com.asakusafw.runtime.directio

Interface DirectDataSource

All Known Implementing Classes:

AbstractDirectDataSource, HadoopDataSource, HadoopDataSourceCore, KeepAliveDataSource

public interface DirectDataSource

API of direct accessor to data sources. Clients should not implement this interface directly.

Output Staging

Using this interface, output to resources is performed in the following sequence:

1. Prepare the staging output area in setupTransactionOutput(OutputTransactionContext).
2. Prepare temporary output area for each attempt in setupAttemptOutput(OutputAttemptContext).
3. Create output contents onto temporary output area in openOutput(OutputAttemptContext, DataDefinition, String, String, Counter).
4. Move output contents from temporary area to staging area in commitAttemptOutput(OutputAttemptContext).
5. Cleanup output contents on temporary area in cleanupAttemptOutput(OutputAttemptContext).
6. Move output contents from staging area to final area in commitTransactionOutput(OutputTransactionContext).
7. Cleanup output contents on staging area in cleanupTransactionOutput(OutputTransactionContext).

Since:

0.2.5

Version:

0.10.0

See Also:

AbstractDirectDataSource

Method Summary

Modifier and Type	Method and Description
void	cleanupAttemptOutput(OutputAttemptContext context) Cleanup output on the attempt.
void	cleanupTransactionOutput(OutputTransactionContext context) Cleanup output on the transaction.
void	commitAttemptOutput(OutputAttemptContext context) Commits output on the attempt.
void	commitTransactionOutput(OutputTransactionContext context) Commits output on the transaction.
boolean	delete(String basePath, ResourcePattern resourcePattern, boolean recursive, Counter counter) Deletes resources.
<T> List<DirectInputFragment>	findInputFragments(DataDefinition<T> definition, String basePath, ResourcePattern resourcePattern) Collects input fragments about target resources.
default <T> Optional<T>	findProperty(Class<T> propertyType) Returns the property of this data source.
List<ResourceInfo>	list(String basePath, ResourcePattern resourcePattern, Counter counter) List resources.
<T> ModelInput<T>	openInput(DataDefinition<T> definition, DirectInputFragment fragment, Counter counter) Opens a fragment for input.
<T> ModelOutput<T>	openOutput(OutputAttemptContext context, DataDefinition<T> definition, String basePath, String resourcePath, Counter counter) Opens a resource for output.
String	path(String basePath) Returns a textually representation of the target path.
String	path(String basePath, ResourcePattern resourcePattern) Returns a textually representation of the target path pattern.
void	setupAttemptOutput(OutputAttemptContext context) Prepares output area for the attempt.
void	setupTransactionOutput(OutputTransactionContext context) Prepares output area for the transaction.

Method Detail

path

String path(String basePath)

Returns a textually representation of the target path.

Parameters:

basePath - base path of target resources

Returns:

the corresponded path

Throws:

IllegalArgumentException - if some parameters were null

Since:

0.10.0

path

String path(String basePath,
            ResourcePattern resourcePattern)

Returns a textually representation of the target path pattern.

Parameters:

basePath - base path of target resources

resourcePattern - search pattern of target resources from basePath

Returns:

the corresponded path

Throws:

IllegalArgumentException - if some parameters were null

Since:

0.8.1

findInputFragments

<T> List<DirectInputFragment> findInputFragments(DataDefinition<T> definition,
                                                 String basePath,
                                                 ResourcePattern resourcePattern)
                                          throws IOException,
                                                 InterruptedException

Collects input fragments about target resources.

Type Parameters:

T - target data type

Parameters:

definition - the data definition

basePath - base path of target resources

resourcePattern - search pattern of target resources from basePath

Returns:

the found fragments of resources

Throws:

IOException - if failed to find resources by I/O error

InterruptedException - if interrupted while finding fragments

IllegalArgumentException - if some parameters were null

openInput

<T> ModelInput<T> openInput(DataDefinition<T> definition,
                            DirectInputFragment fragment,
                            Counter counter)
                     throws IOException,
                            InterruptedException

Opens a fragment for input.

Type Parameters:

T - target data type

Parameters:

definition - the data definition

fragment - target fragment

counter - counter object

Returns:

ModelInput to obtain contents in the fragment

Throws:

IOException - if failed to open the fragment by I/O error

InterruptedException - if interrupted

IllegalArgumentException - if some parameters were null

openOutput

<T> ModelOutput<T> openOutput(OutputAttemptContext context,
                              DataDefinition<T> definition,
                              String basePath,
                              String resourcePath,
                              Counter counter)
                       throws IOException,
                              InterruptedException

Opens a resource for output.

Type Parameters:

T - target data type

Parameters:

context - current output attempt context

definition - the data definition

basePath - base path of target resource

resourcePath - target resource path from basePath

counter - counter object

Returns:

ModelOutput to put contents to the resource

Throws:

IOException - if failed to open the resource by I/O error

InterruptedException - if interrupted

IllegalArgumentException - if some parameters were null

See Also:

Output Staging

list

List<ResourceInfo> list(String basePath,
                        ResourcePattern resourcePattern,
                        Counter counter)
                 throws IOException,
                        InterruptedException

List resources.

Parameters:

basePath - base path of target resources

resourcePattern - search pattern of target resources from basePath

counter - the resource counter

Returns:

the found resources, or an empty list if there are no resources

Throws:

IOException - if failed to list resources by I/O error

InterruptedException - if interrupted

IllegalArgumentException - if some parameters were null

delete

boolean delete(String basePath,
               ResourcePattern resourcePattern,
               boolean recursive,
               Counter counter)
        throws IOException,
               InterruptedException

Deletes resources.

Parameters:

basePath - base path of target resources

resourcePattern - search pattern of target resources from basePath

recursive - also deletes containers and thier components recursively (only if this datasource supports containers)

counter - the resource counter

Returns:

true if successfully deleted, otherwise false

Throws:

IOException - if failed to delete resources by I/O error

InterruptedException - if interrupted

IllegalArgumentException - if some parameters were null

setupAttemptOutput

void setupAttemptOutput(OutputAttemptContext context)
                 throws IOException,
                        InterruptedException

Prepares output area for the attempt.

Parameters:

context - current attempt context

Throws:

IOException - if failed by I/O error

InterruptedException - if interrupted

IllegalArgumentException - if some parameters were null

See Also:

Output Staging

commitAttemptOutput

void commitAttemptOutput(OutputAttemptContext context)
                  throws IOException,
                         InterruptedException

Commits output on the attempt.

Parameters:

context - current attempt context

Throws:

IOException - if failed by I/O error

InterruptedException - if interrupted

IllegalArgumentException - if some parameters were null

See Also:

Output Staging

cleanupAttemptOutput

void cleanupAttemptOutput(OutputAttemptContext context)
                   throws IOException,
                          InterruptedException

Cleanup output on the attempt.

Parameters:

context - current attempt context

Throws:

IOException - if failed by I/O error

InterruptedException - if interrupted

IllegalArgumentException - if some parameters were null

See Also:

Output Staging

setupTransactionOutput

void setupTransactionOutput(OutputTransactionContext context)
                     throws IOException,
                            InterruptedException

Prepares output area for the transaction.

Parameters:

context - current transaction context

Throws:

IOException - if failed by I/O error

InterruptedException - if interrupted

IllegalArgumentException - if some parameters were null

See Also:

Output Staging

commitTransactionOutput

void commitTransactionOutput(OutputTransactionContext context)
                      throws IOException,
                             InterruptedException

Commits output on the transaction.

Parameters:

context - current transaction context

Throws:

IOException - if failed by I/O error

InterruptedException - if interrupted

IllegalArgumentException - if some parameters were null

See Also:

Output Staging

cleanupTransactionOutput

void cleanupTransactionOutput(OutputTransactionContext context)
                       throws IOException,
                              InterruptedException

Cleanup output on the transaction.

Parameters:

context - current transaction context

Throws:

IOException - if failed by I/O error

InterruptedException - if interrupted

IllegalArgumentException - if some parameters were null

See Also:

Output Staging

findProperty

default <T> Optional<T> findProperty(Class<T> propertyType)

Returns the property of this data source.

Type Parameters:

T - the property type

Parameters:

propertyType - the property type

Returns:

the corresponded property, or empty if it is not defined

Since:

0.10.0