com.asakusafw.yaess.basic

Class ProcessHadoopScriptHandler

java.lang.Object

com.asakusafw.yaess.core.ExecutionScriptHandlerBase

com.asakusafw.yaess.basic.ProcessHadoopScriptHandler

All Implemented Interfaces:

ExecutionScriptHandler<HadoopScript>, HadoopScriptHandler, Service

Direct Known Subclasses:

BasicHadoopScriptHandler, SshHadoopScriptHandler

public abstract class ProcessHadoopScriptHandler
extends ExecutionScriptHandlerBase
implements HadoopScriptHandler

An abstract implementations of process-based HadoopScriptHandler. This handler just launches a command with following arguments in its tail.

1. class name
2. batch-id
3. flow-id
4. execution-id
5. batch-arguments
6. hadoop properties (with "-D")

Additionally, the handler lanuches a command if hadoop.cleanup is true.

1. CLEANUP_STAGE_CLASS
2. batch-id
3. flow-id
4. execution-id
5. batch-arguments
6. hadoop properties (with "-D")

Profile format

# <position> = 0, 1, 2, ...
# <prefix command token> can contain "@[position],"
# this will be replaced as original command tokens (0-origin position)
hadoop = <this class name>
hadoop.env.ASAKUSA_HOME = ${ASAKUSA_HOME}
hadoop.command.<position> = $<prefix command token>
hadoop.cleanup = whether enables cleanup
hadoop.env.<key> = $<extra environment variables>
hadoop.prop.<key> = $<extra Hadoop properties>

Since:

0.2.3

Version:

0.5.0

Field Summary

Fields

Modifier and Type	Field and Description
static String	CLEANUP_STAGE_CLASS The class name of cleanup stage client.
static String	KEY_CLEANUP (sub) key name of cleanup enabled.
static String	KEY_WORKING_DIRECTORY Deprecated. cleanup is obsoleted
static String	PATH_EXECUTE The path to the Hadoop execution executable file (relative path from Asakusa home).
static String	VAR_BATCH_ID Variable name of batch ID.
static String	VAR_EXECUTION_ID Variable name of execution ID.
static String	VAR_FLOW_ID Variable name of flow ID.

Fields inherited from interface com.asakusafw.yaess.core.ExecutionScriptHandler

DEFAULT_RESOURCE_ID, ENV_EXTENSION_PREFIX, KEY_ENV_PREFIX, KEY_PROP_PREFIX, KEY_RESOURCE

Constructor Summary

Constructors

Constructor and Description
ProcessHadoopScriptHandler()

Method Summary

Modifier and Type	Method and Description
void	cleanUp(ExecutionMonitor monitor, ExecutionContext context) Cleanup the target environment.
protected abstract void	configureExtension(ServiceProfile<?> profile) Configures this handler internally (extension point).
protected void	doConfigure(ServiceProfile<?> profile, Map<String,String> desiredProperties, Map<String,String> desiredEnvironmentVariables) Configures this handler internally (extention point).
void	execute(ExecutionMonitor monitor, ExecutionContext context, HadoopScript script) Executes the specified script.
protected abstract ProcessExecutor	getCommandExecutor() Returns command executor for this handler (extension point).
protected String	getPathSegmentSeparator() Returns the path segment separator.

Methods inherited from class com.asakusafw.yaess.core.ExecutionScriptHandlerBase

configure, getEnvironmentVariables, getHandlerId, getProperties, getResourceId, setUp, voidCleanUp, voidSetUp

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.asakusafw.yaess.core.ExecutionScriptHandler

getEnvironmentVariables, getHandlerId, getProperties, getResourceId, setUp

Methods inherited from interface com.asakusafw.yaess.core.Service

configure

Field Detail

CLEANUP_STAGE_CLASS

public static final String CLEANUP_STAGE_CLASS

The class name of cleanup stage client.

Since:

0.4.0

See Also:

Constant Field Values

KEY_WORKING_DIRECTORY

@Deprecated
public static final String KEY_WORKING_DIRECTORY

Deprecated. cleanup is obsoleted

(sub) key name of working directory.

See Also:

Constant Field Values

KEY_CLEANUP

public static final String KEY_CLEANUP

(sub) key name of cleanup enabled.

See Also:

Constant Field Values

PATH_EXECUTE

public static final String PATH_EXECUTE

The path to the Hadoop execution executable file (relative path from Asakusa home).

See Also:

Constant Field Values

VAR_BATCH_ID

public static final String VAR_BATCH_ID

Variable name of batch ID.

See Also:

Constant Field Values

VAR_FLOW_ID

public static final String VAR_FLOW_ID

Variable name of flow ID.

See Also:

Constant Field Values

VAR_EXECUTION_ID

public static final String VAR_EXECUTION_ID

Variable name of execution ID.

See Also:

Constant Field Values

Constructor Detail

ProcessHadoopScriptHandler

public ProcessHadoopScriptHandler()

Method Detail

doConfigure

protected final void doConfigure(ServiceProfile<?> profile,
                                 Map<String,String> desiredProperties,
                                 Map<String,String> desiredEnvironmentVariables)
                          throws InterruptedException,
                                 IOException

Description copied from class: ExecutionScriptHandlerBase

Configures this handler internally (extention point).

Specified by:

doConfigure in class ExecutionScriptHandlerBase

Parameters:

profile - the profile of this service

desiredProperties - the current desired system/hadoop properties (configurable)

desiredEnvironmentVariables - the current desired environment variables (configurable)

Throws:

InterruptedException - if interrupted in configuration

IOException - if failed to configure this service

configureExtension

protected abstract void configureExtension(ServiceProfile<?> profile)
                                    throws InterruptedException,
                                           IOException

Configures this handler internally (extension point).

Parameters:

profile - the profile of this service

Throws:

InterruptedException - if interrupted in configuration

IOException - if failed to configure this service

getCommandExecutor

protected abstract ProcessExecutor getCommandExecutor()

Returns command executor for this handler (extension point).

Returns:

command executor

execute

public final void execute(ExecutionMonitor monitor,
                          ExecutionContext context,
                          HadoopScript script)
                   throws InterruptedException,
                          IOException

Description copied from interface: ExecutionScriptHandler

Executes the specified script.

Specified by:

execute in interface ExecutionScriptHandler<HadoopScript>

Parameters:

monitor - the progress monitor of the execution

context - current execution context

script - target script

Throws:

InterruptedException - if the execution was interrupted

IOException - if execution was failed

cleanUp

public void cleanUp(ExecutionMonitor monitor,
                    ExecutionContext context)
             throws InterruptedException,
                    IOException

Description copied from class: ExecutionScriptHandlerBase

Cleanup the target environment.

Specified by:

cleanUp in interface ExecutionScriptHandler<HadoopScript>

Overrides:

cleanUp in class ExecutionScriptHandlerBase

Parameters:

monitor - the progress monitor of the operation

context - the current execution context

Throws:

InterruptedException - if this operation is interrupted

IOException - if failed to setup the target environment

getPathSegmentSeparator

protected String getPathSegmentSeparator()

Returns the path segment separator.

Returns:

the path segment separator string