com.asakusafw.directio.hive.parquet

Class Compatibility

java.lang.Object

com.asakusafw.directio.hive.parquet.Compatibility

Direct Known Subclasses:

CompatibilityV1, CompatibilityV2

public abstract class Compatibility
extends Object

compatibility layer for Direct I/O Parquet support.

Since:

0.10.3

Constructor Summary

Constructors

Constructor and Description
Compatibility()

Method Summary

Modifier and Type	Method and Description
abstract List<DirectInputFragment>	computeInputFragments(AbstractParquetFileFormat<?> format, StripedDataFormat.InputContext context) delegate from StripedDataFormat.computeInputFragments(InputContext).
abstract <T> ModelInput<T>	createInput(AbstractParquetFileFormat<T> format, Class<? extends T> dataType, org.apache.hadoop.fs.FileSystem fileSystem, org.apache.hadoop.fs.Path path, long offset, long fragmentSize, Counter counter) delegate from AbstractParquetFileFormat.createInput(Class, FileSystem, Path, long, long, Counter).
abstract <T> ModelOutput<T>	createOutput(AbstractParquetFileFormat<T> format, Class<? extends T> dataType, org.apache.hadoop.fs.FileSystem fileSystem, org.apache.hadoop.fs.Path path, Counter counter) delegate from AbstractParquetFileFormat.createOutput(Class, FileSystem, Path, Counter).
abstract Optional<?>	findValueDriver(org.apache.hadoop.hive.serde2.typeinfo.TypeInfo typeInfo, Class<?> valueClass) delegate to ParquetValueDrivers.find().
abstract Optional<? extends Enum<?>>	findVersionId(String name) returns the version symbol if the name.
abstract Class<? extends Enum<?>>	getCompressionCodecNameClass() returns the compression kind class.
static Compatibility	getInstance() returns the compatibility instance for this environment.
protected abstract int	getPriority() returns the priority of this compatibility layer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Compatibility

public Compatibility()

Method Detail

getInstance

public static Compatibility getInstance()

returns the compatibility instance for this environment.

Returns:

the compatibility instance

getPriority

protected abstract int getPriority()

returns the priority of this compatibility layer.

Returns:

the priority (higher value is higher priority), or negative value if it is not available

computeInputFragments

public abstract List<DirectInputFragment> computeInputFragments(AbstractParquetFileFormat<?> format,
                                                                StripedDataFormat.InputContext context)
                                                         throws IOException,
                                                                InterruptedException

delegate from StripedDataFormat.computeInputFragments(InputContext).

Parameters:

format - the source format

context - the current input context

Returns:

the computed result

Throws:

IOException - if failed to compute fragments by I/O error

InterruptedException - if interrupted while computing fragments

createInput

public abstract <T> ModelInput<T> createInput(AbstractParquetFileFormat<T> format,
                                              Class<? extends T> dataType,
                                              org.apache.hadoop.fs.FileSystem fileSystem,
                                              org.apache.hadoop.fs.Path path,
                                              long offset,
                                              long fragmentSize,
                                              Counter counter)
                                       throws IOException,
                                              InterruptedException

delegate from AbstractParquetFileFormat.createInput(Class, FileSystem, Path, long, long, Counter).

Type Parameters:

T - the data type

Parameters:

format - the source format

dataType - the target data type

fileSystem - the file system to open the target path

path - the path to the target file

offset - starting stream offset

fragmentSize - suggested fragment bytes count, or -1 as infinite

counter - the current counter

Returns:

the created reader

Throws:

IOException - if failed to create reader

InterruptedException - if interrupted

IllegalArgumentException - if this does not support target property sequence, or any parameter is null

createOutput

public abstract <T> ModelOutput<T> createOutput(AbstractParquetFileFormat<T> format,
                                                Class<? extends T> dataType,
                                                org.apache.hadoop.fs.FileSystem fileSystem,
                                                org.apache.hadoop.fs.Path path,
                                                Counter counter)
                                         throws IOException,
                                                InterruptedException

delegate from AbstractParquetFileFormat.createOutput(Class, FileSystem, Path, Counter).

Type Parameters:

T - the data type

Parameters:

format - the source format

dataType - the target data type

fileSystem - the file system to open the target path

path - the path to the target file

counter - the current counter

Returns:

the created writer

Throws:

IOException - if failed to create writer

InterruptedException - if interrupted

IllegalArgumentException - if this does not support property sequence, or any parameter is null

findVersionId

public abstract Optional<? extends Enum<?>> findVersionId(String name)

returns the version symbol if the name.

Parameters:

name - the version name

Returns:

the related symbol, or empty if it is not defined

getCompressionCodecNameClass

public abstract Class<? extends Enum<?>> getCompressionCodecNameClass()

returns the compression kind class.

Returns:

the compression kind class

findValueDriver

public abstract Optional<?> findValueDriver(org.apache.hadoop.hive.serde2.typeinfo.TypeInfo typeInfo,
                                            Class<?> valueClass)

delegate to ParquetValueDrivers.find().

Parameters:

typeInfo - the Hive type info

valueClass - the ValueOption type

Returns:

the corresponded ParquetValueDriver, or empty if it is not found