com.asakusafw.runtime.stage.temporary

Class TemporaryStorage

java.lang.Object

com.asakusafw.runtime.stage.temporary.TemporaryStorage

public final class TemporaryStorage
extends Object

Access to the temporary storage.

Since:

0.2.5

Version:

0.7.1

Method Summary

Modifier and Type	Method and Description
static List<org.apache.hadoop.fs.Path>	list(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path pathPattern) Resolves the raw path pattern into the concrete path list.
static List<org.apache.hadoop.fs.FileStatus>	listStatus(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path pathPattern) Resolves the raw path pattern into the concrete file status list.
static <V> ModelInput<V>	openInput(org.apache.hadoop.conf.Configuration conf, Class<V> dataType, org.apache.hadoop.fs.FileStatus status, InputStream input) Opens a temporary ModelInput for the specified path.
static <V> ModelInput<V>	openInput(org.apache.hadoop.conf.Configuration conf, Class<V> dataType, org.apache.hadoop.fs.Path path) Opens a temporary ModelInput for the specified path.
static <V> ModelOutput<V>	openOutput(org.apache.hadoop.conf.Configuration conf, Class<V> dataType, OutputStream output) Opens a temporary ModelOutput for the specified output.
static <V> ModelOutput<V>	openOutput(org.apache.hadoop.conf.Configuration conf, Class<V> dataType, OutputStream output, org.apache.hadoop.io.compress.CompressionCodec compressionCodec) Opens a temporary ModelOutput for the specified output.
static <V> ModelOutput<V>	openOutput(org.apache.hadoop.conf.Configuration conf, Class<V> dataType, org.apache.hadoop.fs.Path path) Opens a temporary ModelOutput for the specified path.
static <V> ModelOutput<V>	openOutput(org.apache.hadoop.conf.Configuration conf, Class<V> dataType, org.apache.hadoop.fs.Path path, org.apache.hadoop.io.compress.CompressionCodec compressionCodec) Opens a temporary ModelOutput for the specified path.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

list

public static List<org.apache.hadoop.fs.Path> list(org.apache.hadoop.conf.Configuration conf,
                                                   org.apache.hadoop.fs.Path pathPattern)
                                            throws IOException

Resolves the raw path pattern into the concrete path list.

Parameters:

conf - current configuration

pathPattern - path pattern which describes temporary storage

Returns:

the resolved paths

Throws:

IOException - if failed to resolve path pattern

IllegalArgumentException - if some parameters were null

listStatus

public static List<org.apache.hadoop.fs.FileStatus> listStatus(org.apache.hadoop.conf.Configuration conf,
                                                               org.apache.hadoop.fs.Path pathPattern)
                                                        throws IOException

Resolves the raw path pattern into the concrete file status list.

Parameters:

conf - current configuration

pathPattern - path pattern which describes temporary storage

Returns:

the resolved file status

Throws:

IOException - if failed to resolve path pattern

IllegalArgumentException - if some parameters were null

Since:

0.7.1

openInput

public static <V> ModelInput<V> openInput(org.apache.hadoop.conf.Configuration conf,
                                          Class<V> dataType,
                                          org.apache.hadoop.fs.Path path)
                                   throws IOException

Opens a temporary ModelInput for the specified path.

Type Parameters:

V - data type

Parameters:

conf - configuration

dataType - data type

path - source path (must not contain wildcards)

Returns:

the opened ModelInput

Throws:

IOException - if failed to open input

IllegalArgumentException - if some parameters were null

openInput

public static <V> ModelInput<V> openInput(org.apache.hadoop.conf.Configuration conf,
                                          Class<V> dataType,
                                          org.apache.hadoop.fs.FileStatus status,
                                          InputStream input)
                                   throws IOException

Opens a temporary ModelInput for the specified path.

Type Parameters:

V - data type

Parameters:

conf - configuration

dataType - data type

status - source file status

input - source file content

Returns:

the opened ModelInput

Throws:

IOException - if failed to open input

IllegalArgumentException - if some parameters were null

openOutput

public static <V> ModelOutput<V> openOutput(org.apache.hadoop.conf.Configuration conf,
                                            Class<V> dataType,
                                            org.apache.hadoop.fs.Path path)
                                     throws IOException

Opens a temporary ModelOutput for the specified path.

Type Parameters:

V - data type

Parameters:

conf - configuration

dataType - data type

path - target path

Returns:

the opened ModelOutput

Throws:

IOException - if failed to open output

IllegalArgumentException - if some parameters were null

openOutput

public static <V> ModelOutput<V> openOutput(org.apache.hadoop.conf.Configuration conf,
                                            Class<V> dataType,
                                            org.apache.hadoop.fs.Path path,
                                            org.apache.hadoop.io.compress.CompressionCodec compressionCodec)
                                     throws IOException

Opens a temporary ModelOutput for the specified path.

Type Parameters:

V - data type

Parameters:

conf - configuration

dataType - data type

path - target path

compressionCodec - compression codec, or null if not compressed

Returns:

the opened ModelOutput

Throws:

IOException - if failed to open output

IllegalArgumentException - if some parameters were null

openOutput

public static <V> ModelOutput<V> openOutput(org.apache.hadoop.conf.Configuration conf,
                                            Class<V> dataType,
                                            OutputStream output)
                                     throws IOException

Opens a temporary ModelOutput for the specified output.

Type Parameters:

V - data type

Parameters:

conf - configuration

dataType - data type

output - target output stream

Returns:

the opened ModelOutput

Throws:

IOException - if failed to open output

IllegalArgumentException - if some parameters were null

openOutput

public static <V> ModelOutput<V> openOutput(org.apache.hadoop.conf.Configuration conf,
                                            Class<V> dataType,
                                            OutputStream output,
                                            org.apache.hadoop.io.compress.CompressionCodec compressionCodec)
                                     throws IOException

Opens a temporary ModelOutput for the specified output.

Type Parameters:

V - data type

Parameters:

conf - configuration

dataType - data type

output - target output stream

compressionCodec - compression codec, or null if not compressed

Returns:

the opened ModelOutput

Throws:

IOException - if failed to open output

IllegalArgumentException - if some parameters were null