Uses of Interface
com.asakusafw.runtime.directio.DataFormat

Packages that use DataFormat

Package	Description
com.asakusafw.directio.hive.orc	ORCFile format bridge between Asakusa and Hive.
com.asakusafw.directio.hive.parquet	Parquet file format bridge between Asakusa and Hive.
com.asakusafw.runtime.directio	Direct I/O access from Asakusa Framework.
com.asakusafw.runtime.directio.api	User APIs of Direct I/O.
com.asakusafw.runtime.directio.api.legacy	The legacy Direct I/O implementations.
com.asakusafw.runtime.directio.hadoop	Direct I/O access for Hadoop Environment.
com.asakusafw.runtime.io.csv.directio	Direct I/O adapters for CSV format.
com.asakusafw.runtime.io.json.directio	Direct I/O adapters of JSON files.
com.asakusafw.runtime.io.line.directio	Direct I/O adapters of line read/write.
com.asakusafw.runtime.io.text.directio	Direct I/O adapters of formatted text files.
com.asakusafw.runtime.stage.directio	Map Reduce stage elements for Direct I/O.
com.asakusafw.testdriver	Test driver classes.
com.asakusafw.testdriver.directio.api	Tester APIs of Direct I/O.
com.asakusafw.vocabulary.directio	Asakusa DSL vocabularies extensions for Direct I/O.

Uses of DataFormat in com.asakusafw.directio.hive.orc

Classes in com.asakusafw.directio.hive.orc that implement DataFormat

Modifier and Type	Class and Description
class	AbstractOrcFileFormat<T> An abstract implementation of HadoopFileFormat for ORCFile.
class	OrcFileFormat<T> An implementation of DataFormat for ORCFile.

Uses of DataFormat in com.asakusafw.directio.hive.parquet

Classes in com.asakusafw.directio.hive.parquet that implement DataFormat

Modifier and Type	Class and Description
class	AbstractParquetFileFormat<T> An abstract implementation of HadoopFileFormat for Parquet.
class	ParquetFileFormat<T> An implementation of DataFormat for Parquet file.

Uses of DataFormat in com.asakusafw.runtime.directio

Subinterfaces of DataFormat in com.asakusafw.runtime.directio

Modifier and Type	Interface and Description
interface	FragmentableDataFormat<T> Data model format which can be fragmented.

Classes in com.asakusafw.runtime.directio that implement DataFormat

Modifier and Type	Class and Description
class	BinaryStreamFormat<T> Data model format of InputStream / OutputStream .

Methods in com.asakusafw.runtime.directio that return DataFormat

Modifier and Type	Method and Description
DataFormat<T>	SimpleDataDefinition.getDataFormat()
DataFormat<T>	DataDefinition.getDataFormat() Returns the data format.

Methods in com.asakusafw.runtime.directio with parameters of type DataFormat

Modifier and Type	Method and Description
static <T> DataDefinition<T>	SimpleDataDefinition.newInstance(Class<?> dataClass, DataFormat<T> dataFormat) Creates a new instance.
static <T> DataDefinition<T>	SimpleDataDefinition.newInstance(Class<?> dataClass, DataFormat<T> dataFormat, DataFilter<?> dataFilter) Creates a new instance.

Uses of DataFormat in com.asakusafw.runtime.directio.api

Method parameters in com.asakusafw.runtime.directio.api with type arguments of type DataFormat

Modifier and Type	Method and Description
static <T> ModelInput<T>	DirectIo.open(Class<? extends DataFormat<T>> formatClass, String basePath, String resourcePattern) Returns data model objects from Direct I/O data sources.
<T> ModelInput<T>	DirectIoApi.open(Class<? extends DataFormat<T>> formatClass, String basePath, String resourcePattern) Returns data model objects from Direct I/O data sources.
<T> ModelInput<T>	DirectIoDelegate.open(Class<? extends DataFormat<T>> formatClass, String basePath, String resourcePattern)

Uses of DataFormat in com.asakusafw.runtime.directio.api.legacy

Method parameters in com.asakusafw.runtime.directio.api.legacy with type arguments of type DataFormat

Modifier and Type	Method and Description
static <T> ModelInput<T>	LegacyDirectIo.open(Class<? extends DataFormat<T>> formatClass, String basePath, String resourcePattern) Returns data model objects from Direct I/O data sources.

Uses of DataFormat in com.asakusafw.runtime.directio.hadoop

Subinterfaces of DataFormat in com.asakusafw.runtime.directio.hadoop

Modifier and Type	Interface and Description
interface	StripedDataFormat<T> Data model format which has striping information in the format.

Classes in com.asakusafw.runtime.directio.hadoop that implement DataFormat

Modifier and Type	Class and Description
class	ConfigurableBinaryStreamFormat<T> BinaryStreamFormat with Configurable.
class	HadoopFileFormat<T> Data model format of InputStream / OutputStream .
class	HadoopFileFormatAdapter<T> Adapts BinaryStreamFormat into HadoopFileFormat.
class	SequenceFileFormat<K,V,T> Data model format of InputStream / OutputStream .

Methods in com.asakusafw.runtime.directio.hadoop with parameters of type DataFormat

Modifier and Type	Method and Description
static <T> HadoopFileFormat<T>	HadoopDataSourceUtil.toHadoopFileFormat(org.apache.hadoop.conf.Configuration configuration, DataFormat<T> format) Converts DataFormat into an equivalent HadoopFileFormat.

Uses of DataFormat in com.asakusafw.runtime.io.csv.directio

Classes in com.asakusafw.runtime.io.csv.directio that implement DataFormat

Modifier and Type	Class and Description
class	AbstractCsvStreamFormat<T> An abstract implementation of Direct I/O data format for CSV files.

Uses of DataFormat in com.asakusafw.runtime.io.json.directio

Classes in com.asakusafw.runtime.io.json.directio that implement DataFormat

Modifier and Type	Class and Description
class	AbstractJsonStreamFormat<T> An abstract implementation of Direct I/O data format for JSON files.

Uses of DataFormat in com.asakusafw.runtime.io.line.directio

Classes in com.asakusafw.runtime.io.line.directio that implement DataFormat

Modifier and Type	Class and Description
class	AbstractLineStreamFormat<T> An abstract implementation of Direct I/O data format for line read/write.

Uses of DataFormat in com.asakusafw.runtime.io.text.directio

Classes in com.asakusafw.runtime.io.text.directio that implement DataFormat

Modifier and Type	Class and Description
class	AbstractTextStreamFormat<T> An abstract implementation of Direct I/O data format for formatted text files.

Uses of DataFormat in com.asakusafw.runtime.stage.directio

Constructor parameters in com.asakusafw.runtime.stage.directio with type arguments of type DataFormat

Constructor and Description
AbstractNoReduceDirectOutputMapper(Class<? extends T> dataType, String rawBasePath, String rawResourcePath, Class<? extends DataFormat<? super T>> dataFormatClass) Creates a new instance.
AbstractNoReduceDirectOutputMapper(Class<? extends T> dataType, String outputId, String rawBasePath, String rawResourcePath, Class<? extends DataFormat<? super T>> dataFormatClass) Creates a new instance.
DirectOutputSpec(Class<? extends org.apache.hadoop.io.Writable> valueType, String path, Class<? extends DataFormat<?>> formatClass, Class<? extends StringTemplate> namingClass, Class<? extends DirectOutputOrder> orderClass) Creates a new instance.
DirectOutputSpec(Class<? extends org.apache.hadoop.io.Writable> valueType, String outputId, String path, Class<? extends DataFormat<?>> formatClass, Class<? extends StringTemplate> namingClass, Class<? extends DirectOutputOrder> orderClass) Creates a new instance.

Uses of DataFormat in com.asakusafw.testdriver

Method parameters in com.asakusafw.testdriver with type arguments of type DataFormat

Modifier and Type	Method and Description
<T> void	OperatorTestEnvironment.dump(Class<T> dataType, Iterable<? extends T> objects, Class<? extends DataFormat<? super T>> formatClass, File outputPath) Stores objects into the given sink.
S	FlowDriverOutput.dumpActual(Class<? extends DataFormat<? super T>> formatClass, File outputPath) Enables to store the result data set of this output.
<T> DataLoader<T>	OperatorTestEnvironment.loader(Class<T> dataType, Class<? extends DataFormat<? super T>> formatClass, File file) Returns a new data loader.
<T> DataLoader<T>	TesterBase.loader(Class<T> dataType, Class<? extends DataFormat<? super T>> formatClass, File file) Returns a new data loader.
<T> DataLoader<T>	OperatorTestEnvironment.loader(Class<T> dataType, Class<? extends DataFormat<? super T>> formatClass, String sourcePath) Returns a new data loader.
<T> DataLoader<T>	TesterBase.loader(Class<T> dataType, Class<? extends DataFormat<? super T>> formatClass, String sourcePath) Returns a new data loader.
S	FlowDriverOutput.prepare(Class<? extends DataFormat<? super T>> formatClass, File sourceFile) Sets the test data set for this input.
S	FlowDriverInput.prepare(Class<? extends DataFormat<? super T>> formatClass, File sourceFile) Sets the test data set for this input.
S	FlowDriverInput.prepare(Class<? extends DataFormat<? super T>> formatClass, File sourceFile, Consumer<? super T> transformer) Sets the test data set for this input.
S	FlowDriverOutput.prepare(Class<? extends DataFormat<? super T>> formatClass, String sourcePath) Sets the test data set for this input.
S	FlowDriverInput.prepare(Class<? extends DataFormat<? super T>> formatClass, String sourcePath) Sets the test data set for this input.
S	FlowDriverInput.prepare(Class<? extends DataFormat<? super T>> formatClass, String sourcePath, Consumer<? super T> transformer) Sets the test data set for this input.
protected <T> DataModelSinkFactory	DriverElementBase.toDataModelSinkFactory(DataModelDefinition<T> definition, Class<? extends DataFormat<? super T>> formatClass, File destinationFile) Converts an output path to DataModelSinkFactory to write to the path.
protected <T> DataModelSourceFactory	DriverElementBase.toDataModelSourceFactory(DataModelDefinition<T> definition, Class<? extends DataFormat<? super T>> formatClass, File sourceFile) Loads a source file DataModelSourceFactory which provides data models.
protected <T> DataModelSourceFactory	DriverElementBase.toDataModelSourceFactory(DataModelDefinition<T> definition, Class<? extends DataFormat<? super T>> formatClass, String sourcePath) Loads a source file DataModelSourceFactory which provides data models.
S	FlowDriverOutput.verify(Class<? extends DataFormat<? super T>> formatClass, File expectedFile, ModelVerifier<? super T> modelVerifier) Enables to verify the results of this output.
S	FlowDriverOutput.verify(Class<? extends DataFormat<? super T>> formatClass, File expectedFile, String verifyRulePath) Enables to verify the results of this output.
S	FlowDriverOutput.verify(Class<? extends DataFormat<? super T>> formatClass, File expectedFile, String verifyRulePath, ModelTester<? super T> tester) Enables to verify the results of this output.
S	FlowDriverOutput.verify(Class<? extends DataFormat<? super T>> formatClass, String expectedPath, ModelVerifier<? super T> modelVerifier) Enables to verify the results of this output.
S	FlowDriverOutput.verify(Class<? extends DataFormat<? super T>> formatClass, String expectedPath, String verifyRulePath) Enables to verify the results of this output.
S	FlowDriverOutput.verify(Class<? extends DataFormat<? super T>> formatClass, String expectedPath, String verifyRulePath, ModelTester<? super T> tester) Enables to verify the results of this output.

Uses of DataFormat in com.asakusafw.testdriver.directio.api

Method parameters in com.asakusafw.testdriver.directio.api with type arguments of type DataFormat

Modifier and Type	Method and Description
DirectIoTester.DirectIoResource	DirectIoTester.resource(Class<? extends DataFormat<?>> formatClass, String basePath, String resourcePattern) Declares about Direct I/O resource.

Uses of DataFormat in com.asakusafw.vocabulary.directio

Methods in com.asakusafw.vocabulary.directio that return types with arguments of type DataFormat

Modifier and Type	Method and Description
abstract Class<? extends DataFormat<?>>	DirectFileOutputDescription.getFormat() Returns an implementation of DataFormat class.
abstract Class<? extends DataFormat<?>>	DirectFileInputDescription.getFormat() Returns an implementation of DataFormat class.