HadoopDataSourceProfile (Asakusa Framework 0.10.4 API)

java.lang.Object
- com.asakusafw.runtime.directio.hadoop.HadoopDataSourceProfile

```
public class HadoopDataSourceProfile
extends Object
```
A structured profile for HadoopDataSource.

Since:

0.2.5

Version:

0.9.0

Field Summary

Fields
Modifier and Type	Field and Description
`static String`	`KEY_COMBINE_BLOCKS` The property key name for `isCombineBlocks()`.
`static String`	`KEY_KEEPALIVE_INTERVAL` The property key name for `getKeepAliveInterval()`.
`static String`	`KEY_LEGACY_FRAGMENT_MIN` the Hadoop configuration key of whether or not use the minimum value between `getMinimumFragmentSize()` and `FragmentableDataFormat.getMinimumFragmentSize()`.
`static String`	`KEY_MIN_FRAGMENT` The property key name for `getMinimumFragmentSize(FragmentableDataFormat)`.
`static String`	`KEY_OUTPUT_STAGING` The property key name for `isOutputStaging()`.
`static String`	`KEY_OUTPUT_STREAMING` The property key name for `isOutputStreaming()`.
`static String`	`KEY_PATH` The property key name for `getFileSystemPath()`.
`static String`	`KEY_PREF_FRAGMENT` The property key name for `getPreferredFragmentSize(FragmentableDataFormat)`.
`static String`	`KEY_ROLLFORWARD_THREADS` The property key name of number of threads for moving files in roll-forward operation.
`static String`	`KEY_SPLIT_BLOCKS` The property key name for `isSplitBlocks()`.
`static String`	`KEY_TEMP` The property key name for `getTemporaryFileSystemPath()`.

Constructor Summary

Constructors
Constructor and Description
`HadoopDataSourceProfile(org.apache.hadoop.conf.Configuration conf, String id, String contextPath, org.apache.hadoop.fs.Path fileSystemPath, org.apache.hadoop.fs.Path temporaryPath)` Creates a new instance.

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`static HadoopDataSourceProfile`	`convert(DirectDataSourceProfile profile, org.apache.hadoop.conf.Configuration conf)` Converts the `DirectDataSourceProfile` into this profile.
`String`	`getContextPath()` Returns the logical context path.
`org.apache.hadoop.fs.FileSystem`	`getFileSystem()` Returns the file system for the this datastore.
`org.apache.hadoop.fs.Path`	`getFileSystemPath()` Returns the mapping target path.
`String`	`getId()` Return the ID of this datasource.
`long`	`getKeepAliveInterval()` Returns keep-alive interval.
`org.apache.hadoop.fs.LocalFileSystem`	`getLocalFileSystem()` Returns the local file system for the this datastore.
`long`	`getMinimumFragmentSize()` Returns the minimum fragment size.
`long`	`getMinimumFragmentSize(FragmentableDataFormat<?> format)` Returns the minimum fragment size.
`long`	`getPreferredFragmentSize()` Returns the preferred fragment size.
`long`	`getPreferredFragmentSize(FragmentableDataFormat<?> format)` Returns the preferred fragment size.
`int`	`getRollforwardThreads()` Returns the number of threads to move staged files to committed area.
`org.apache.hadoop.fs.Path`	`getTemporaryFileSystemPath()` Returns the temporary root path.
`boolean`	`isCombineBlocks()` Returns whether combines multiple blocks into a fragment for optimization.
`boolean`	`isOutputStaging()` Returns whether output staging is required.
`boolean`	`isOutputStreaming()` Returns whether output streaming is required.
`boolean`	`isSplitBlocks()` Returns whether split DFS block into multiple splits for optimization.
`void`	`setCombineBlocks(boolean combine)` Sets whether combines blocks for optimization.
`void`	`setKeepAliveInterval(long interval)` Sets keep-alive interval.
`void`	`setMinimumFragmentSize(long size)` Configures the minimum fragment size in bytes.
`void`	`setOutputStaging(boolean required)` Sets whether output staging is required.
`void`	`setOutputStreaming(boolean required)` Sets whether output streaming is required.
`void`	`setPreferredFragmentSize(long size)` Configures the preferred fragment size in bytes.
`void`	`setRollforwardThreads(int threads)` Sets the number of threads to move staged files to committed area.
`void`	`setSplitBlocks(boolean split)` Sets whether splits blocks for optimization.
`String`	`toString()`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - KEY_PATH
```
public static final String KEY_PATH
```
    The property key name for getFileSystemPath(). Default is FileSystem.getWorkingDirectory().
    
    See Also:
    
    Constant Field Values
  - KEY_TEMP
```
public static final String KEY_TEMP
```
    The property key name for getTemporaryFileSystemPath().
    
    See Also:
    
    Constant Field Values
  - KEY_OUTPUT_STAGING
```
public static final String KEY_OUTPUT_STAGING
```
    The property key name for isOutputStaging().
    
    See Also:
    
    Constant Field Values
  - KEY_OUTPUT_STREAMING
```
public static final String KEY_OUTPUT_STREAMING
```
    The property key name for isOutputStreaming().
    
    See Also:
    
    Constant Field Values
  - KEY_MIN_FRAGMENT
```
public static final String KEY_MIN_FRAGMENT
```
    The property key name for getMinimumFragmentSize(FragmentableDataFormat).
    
    See Also:
    
    Constant Field Values
  - KEY_PREF_FRAGMENT
```
public static final String KEY_PREF_FRAGMENT
```
    The property key name for getPreferredFragmentSize(FragmentableDataFormat).
    
    See Also:
    
    Constant Field Values
  - KEY_SPLIT_BLOCKS
```
public static final String KEY_SPLIT_BLOCKS
```
    The property key name for isSplitBlocks().
    
    See Also:
    
    Constant Field Values
  - KEY_COMBINE_BLOCKS
```
public static final String KEY_COMBINE_BLOCKS
```
    The property key name for isCombineBlocks().
    
    See Also:
    
    Constant Field Values
  - KEY_KEEPALIVE_INTERVAL
```
public static final String KEY_KEEPALIVE_INTERVAL
```
    The property key name for getKeepAliveInterval().
    
    Since:
    
    0.2.6
    
    See Also:
    
    Constant Field Values
  - KEY_ROLLFORWARD_THREADS
```
public static final String KEY_ROLLFORWARD_THREADS
```
    The property key name of number of threads for moving files in roll-forward operation.
    
    Since:
    
    0.9.0
    
    See Also:
    
    Constant Field Values
  - KEY_LEGACY_FRAGMENT_MIN
```
public static final String KEY_LEGACY_FRAGMENT_MIN
```
    the Hadoop configuration key of whether or not use the minimum value between getMinimumFragmentSize() and FragmentableDataFormat.getMinimumFragmentSize(). After the issue was fixed, we use the maximum value of them.
    
    Since:
    
    0.10.3
    
    See Also:
    
    GitHub issue page , Constant Field Values
- Constructor Detail
  - HadoopDataSourceProfile
```
public HadoopDataSourceProfile(org.apache.hadoop.conf.Configuration conf,
                               String id,
                               String contextPath,
                               org.apache.hadoop.fs.Path fileSystemPath,
                               org.apache.hadoop.fs.Path temporaryPath)
                        throws IOException
```
    Creates a new instance.
    
    Parameters:
    
    conf - the current configuration
    
    id - the ID of this datasource
    
    contextPath - the logical context path
    
    fileSystemPath - the mapping target path
    
    temporaryPath - the temporary root path
    
    Throws:
    
    IOException - if failed to create profile
    
    IllegalArgumentException - if some parameters were null
- Method Detail
  - getId
```
public String getId()
```
    Return the ID of this datasource.
    
    Returns:
    
    the ID
  - getContextPath
```
public String getContextPath()
```
    Returns the logical context path.
    
    Returns:
    
    the logical context path
  - getFileSystemPath
```
public org.apache.hadoop.fs.Path getFileSystemPath()
```
    Returns the mapping target path.
    
    Returns:
    
    the mapping target path
  - getTemporaryFileSystemPath
```
public org.apache.hadoop.fs.Path getTemporaryFileSystemPath()
```
    Returns the temporary root path.
    
    Returns:
    
    the temporary root path
  - getFileSystem
```
public org.apache.hadoop.fs.FileSystem getFileSystem()
```
    Returns the file system for the this datastore.
    
    Returns:
    
    the file system object
  - getLocalFileSystem
```
public org.apache.hadoop.fs.LocalFileSystem getLocalFileSystem()
```
    Returns the local file system for the this datastore.
    
    Returns:
    
    the local file system object
  - getMinimumFragmentSize
```
public long getMinimumFragmentSize(FragmentableDataFormat<?> format)
                            throws IOException,
                                   InterruptedException
```
    Returns the minimum fragment size.
    
    Parameters:
    
    format - target format
    
    Returns:
    
    the minimum fragment size, or < 0 if fragmentation is restricted
    
    Throws:
    
    IOException - if failed to compute size by I/O error
    
    InterruptedException - if interrupted
    
    IllegalArgumentException - if some parameters were null
  - getMinimumFragmentSize
```
public long getMinimumFragmentSize()
```
    Returns the minimum fragment size.
    
    Returns:
    
    the minimum fragment size, or < 0 if fragmentation is restricted
  - setMinimumFragmentSize
```
public void setMinimumFragmentSize(long size)
```
    Configures the minimum fragment size in bytes.
    
    Parameters:
    
    size - the size, or <= 0 to restrict fragmentation
  - getPreferredFragmentSize
```
public long getPreferredFragmentSize(FragmentableDataFormat<?> format)
                              throws IOException,
                                     InterruptedException
```
    Returns the preferred fragment size.
    
    Parameters:
    
    format - target format
    
    Returns:
    
    the preferred fragment size
    
    Throws:
    
    IOException - if failed to compute size by I/O error
    
    InterruptedException - if interrupted
    
    IllegalArgumentException - if some parameters were null
  - getPreferredFragmentSize
```
public long getPreferredFragmentSize()
```
    Returns the preferred fragment size.
    
    Returns:
    
    the preferred fragment size
  - setPreferredFragmentSize
```
public void setPreferredFragmentSize(long size)
```
    Configures the preferred fragment size in bytes.
    
    Parameters:
    
    size - the size
  - isSplitBlocks
```
public boolean isSplitBlocks()
```
    Returns whether split DFS block into multiple splits for optimization.
    
    Returns:
    
    the true to split, otherwise false
  - setSplitBlocks
```
public void setSplitBlocks(boolean split)
```
    Sets whether splits blocks for optimization.
    
    Parameters:
    
    split - true to split, otherwise false
  - isCombineBlocks
```
public boolean isCombineBlocks()
```
    Returns whether combines multiple blocks into a fragment for optimization.
    
    Returns:
    
    the true to combine, otherwise false
  - setCombineBlocks
```
public void setCombineBlocks(boolean combine)
```
    Sets whether combines blocks for optimization.
    
    Parameters:
    
    combine - true to combine, otherwise false
  - isOutputStaging
```
public boolean isOutputStaging()
```
    Returns whether output staging is required.
    
    Returns:
    
    true to required, otherwise false.
  - setOutputStaging
```
public void setOutputStaging(boolean required)
```
    Sets whether output staging is required.
    
    Parameters:
    
    required - true to required, otherwise false
  - isOutputStreaming
```
public boolean isOutputStreaming()
```
    Returns whether output streaming is required.
    
    Returns:
    
    true to required, otherwise false.
  - setOutputStreaming
```
public void setOutputStreaming(boolean required)
```
    Sets whether output streaming is required.
    
    Parameters:
    
    required - true to required, otherwise false
  - getKeepAliveInterval
```
public long getKeepAliveInterval()
```
    Returns keep-alive interval.
    
    Returns:
    
    keep-alive interval in ms, or 0 if keep-alive is disabled
    
    Since:
    
    0.2.6
  - setKeepAliveInterval
```
public void setKeepAliveInterval(long interval)
```
    Sets keep-alive interval.
    
    Parameters:
    
    interval - keep-alive interval in ms, or 0 to disable keep-alive
    
    Since:
    
    0.2.6
  - getRollforwardThreads
```
public int getRollforwardThreads()
```
    Returns the number of threads to move staged files to committed area.
    
    Returns:
    
    the number of threads
    
    Since:
    
    0.9.0
  - setRollforwardThreads
```
public void setRollforwardThreads(int threads)
```
    Sets the number of threads to move staged files to committed area.
    
    Parameters:
    
    threads - the number of threads
    
    Since:
    
    0.9.0
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class Object
  - convert
```
public static HadoopDataSourceProfile convert(DirectDataSourceProfile profile,
                                              org.apache.hadoop.conf.Configuration conf)
                                       throws IOException
```
    Converts the DirectDataSourceProfile into this profile.
    
    Parameters:
    
    profile - target profile
    
    conf - Hadoop configuration
    
    Returns:
    
    the converted profile
    
    Throws:
    
    IOException - if failed to convert
    
    IllegalArgumentException - if some parameters were null

Class HadoopDataSourceProfile

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

KEY_PATH

KEY_TEMP

KEY_OUTPUT_STAGING

KEY_OUTPUT_STREAMING

KEY_MIN_FRAGMENT

KEY_PREF_FRAGMENT

KEY_SPLIT_BLOCKS

KEY_COMBINE_BLOCKS

KEY_KEEPALIVE_INTERVAL

KEY_ROLLFORWARD_THREADS

KEY_LEGACY_FRAGMENT_MIN

Constructor Detail

HadoopDataSourceProfile

Method Detail

getId

getContextPath

getFileSystemPath

getTemporaryFileSystemPath

getFileSystem

getLocalFileSystem

getMinimumFragmentSize

getMinimumFragmentSize

setMinimumFragmentSize

getPreferredFragmentSize

getPreferredFragmentSize

setPreferredFragmentSize

isSplitBlocks

setSplitBlocks

isCombineBlocks

setCombineBlocks

isOutputStaging

setOutputStaging

isOutputStreaming

setOutputStreaming

getKeepAliveInterval

setKeepAliveInterval

getRollforwardThreads

setRollforwardThreads

toString

convert