Package weka.knowledgeflow

Interface StepManager

  • All Known Implementing Classes:
    StepManagerImpl
    public interface StepManager
    Client public interface for the StepManager. Step implementations should only use this interface
    Author:
    Mark Hall (mhall{[at]}pentaho{[dot]}com)

    • Method Detail

      • getName

        java.lang.String getName()
        Get the name of the step managed by this StepManager
        Returns:
        the name of the managed step

      • getManagedStep

        Step getManagedStep()
        Get the actual step managed by this step manager
        Returns:
        the Step managed by this step manager

      • getExecutionEnvironment

        ExecutionEnvironment getExecutionEnvironment()
        Get the executing environment. This contains information such as whether the flow is running in headless environment, what environment variables are available and methods to execute units of work in parallel.
        Returns:
        the execution environment

      • getSettings

        Settings getSettings()
        Get the knowledge flow settings
        Returns:
        the knowledge flow settings

      • numIncomingConnections

        int numIncomingConnections()
        Get the number of steps that are connected with incoming connections
        Returns:
        the number of incoming connections

      • numOutgoingConnections

        int numOutgoingConnections()
        Get the number of steps that are connected with outgoing connections
        Returns:
        the number of outgoing connections

      • numIncomingConnectionsOfType

        int numIncomingConnectionsOfType​(java.lang.String connectionName)
        Get the number of steps that are connected with the given incoming connection type
        Parameters:
        connectionName - the type of the incoming connection
        Returns:
        the number of steps connected with the specified incoming connection type

      • numOutgoingConnectionsOfType

        int numOutgoingConnectionsOfType​(java.lang.String connectionName)
        Get the number of steps that are connected with the given outgoing connection type
        Parameters:
        connectionName - the type of the outgoing connection
        Returns:
        the number of steps connected with the specified outgoing connection type

      • getIncomingConnectedStepsOfConnectionType

        java.util.List<StepManager> getIncomingConnectedStepsOfConnectionType​(java.lang.String connectionName)
        Get a list of steps that are the source of incoming connections of the given type
        Parameters:
        connectionName - the name of the incoming connection to get a list of steps for
        Returns:
        a list of steps that are the source of incoming connections of the given type

      • getIncomingConnectedStepWithName

        StepManager getIncomingConnectedStepWithName​(java.lang.String stepName)
        Get the named step that is connected with an incoming connection.
        Parameters:
        stepName - the name of the step to get
        Returns:
        the step connected with an incoming connection or null if the named step is not connected

      • getOutgoingConnectedStepWithName

        StepManager getOutgoingConnectedStepWithName​(java.lang.String stepName)
        Get a named step connected to this step with an outgoing connection
        Parameters:
        stepName - the name of the step to look for
        Returns:
        the connected step

      • getOutgoingConnectedStepsOfConnectionType

        java.util.List<StepManager> getOutgoingConnectedStepsOfConnectionType​(java.lang.String connectionName)
        Get a list of downstream steps connected to this step with the given connection type.
        Parameters:
        connectionName - the name of the outgoing connection
        Returns:
        a list of downstream steps connected to this one with the named connection type

      • getIncomingConnections

        java.util.Map<java.lang.String,​java.util.List<StepManager>> getIncomingConnections()
        Get a Map of all incoming connections. Map is keyed by connection type; values are lists of steps
        Returns:
        a Map of incoming connections

      • getOutgoingConnections

        java.util.Map<java.lang.String,​java.util.List<StepManager>> getOutgoingConnections()
        Get a Map of all outgoing connections. Map is keyed by connection type; values are lists of steps
        Returns:
        a Map of outgoing connections

      • outputData

        void outputData​(java.lang.String outgoingConnectionName,
                Data data)
         throws WekaException
        Output data to all steps connected with the supplied outgoing connection type. Populates the source and connection name in the supplied Data object for the client
        Parameters:
        outgoingConnectionName - the type of the outgoing connection to send data to
        data - a single Data object to send
        Throws:
        WekaException - if a problem occurs

      • outputData

        void outputData​(Data... data)
         throws WekaException
        Output one or more Data objects to all relevant steps. Populates the source in each Data object for the client, HOWEVER, the client must have populated the connection type in each Data object to be output so that the StepManager knows which connected steps to send the data to. Also notifies any registered StepOutputListeners. Note that the downstream step(s)' processIncoming() method is called in a separate thread for batch connections. Furthermore, if multiple Data objects are supplied via the varargs argument, and a target step will receive more than one of the Data objects, then they will be passed on to the step in question sequentially within the same thread of execution.
        Parameters:
        data - one or more Data objects to be sent
        Throws:
        WekaException - if a problem occurs

      • outputData

        void outputData​(java.lang.String outgoingConnectionName,
                java.lang.String stepName,
                Data data)
         throws WekaException
        Output a single Data object to the named step with the supplied outgoing connection type
        Parameters:
        outgoingConnectionName - the name of the outgoing connection
        stepName - the name of the step to send the data to
        data - the data to send
        Throws:
        WekaException - if a problem occurs

      • getIncomingStructureForConnectionType

        Instances getIncomingStructureForConnectionType​(java.lang.String connectionName)
                                         throws WekaException
        Attempt to retrieve the structure (as a header-only set of instances) for the named incoming connection type. Assumes that there is only one step connected with the supplied incoming connection type.
        Parameters:
        connectionName - the type of the incoming connection to get the structure for
        Returns:
        the structure of the data for the specified incoming connection, or null if the structure can't be determined (or represented as an Instances object)
        Throws:
        WekaException - if a problem occurs

      • getIncomingStructureForConnectionType

        Instances getIncomingStructureForConnectionType​(java.lang.String connectionName,
                                                Environment env)
                                         throws WekaException
        Attempt to retrieve the structure (as a header-only set of instances) for the named incoming connection type. Assumes that there is only one step connected with the supplied incoming connection type.
        Parameters:
        connectionName - the type of the incoming connection to get the structure for
        env - the Environment to use
        Returns:
        the structure of the data for the specified incoming connection, or null if the structure can't be determined (or represented as an Instances object)
        Throws:
        WekaException - if a problem occurs

      • getIncomingStructureFromStep

        Instances getIncomingStructureFromStep​(StepManager sourceStep,
                                       java.lang.String connectionName)
                                throws WekaException
        Attempt to get the incoming structure (as a header-only set of instances) from the given managed step for the given connection type.
        Parameters:
        sourceStep - the step manager managing the source step
        connectionName - the name of the connection to attempt to get the structure for
        Returns:
        the structure as a header-only set of instances, or null if the source step can't determine this at present or if it can't be represented as a set of instances.
        Throws:
        WekaException - if a problem occurs

      • isStepBusy

        boolean isStepBusy()
        Returns true if, at this time, the step managed by this step manager is currently busy with processing
        Returns:
        true if the step managed by this step manager is busy

      • isStopRequested

        boolean isStopRequested()
        Return true if a stop has been requested by the runtime environment
        Returns:
        true if a stop has been requested

      • isStepFinished

        boolean isStepFinished()
        Return true if the current step is finished.
        Returns:
        true if the current step is finished

      • processing

        void processing()
        Step implementations processing batch data should call this to indicate that they have started some processing. Calling this should set the busy flag to true.

      • finished

        void finished()
        Step implementations processing batch data should call this to indicate that they have finished all processing. Calling this should set the busy flag to false.

      • interrupted

        void interrupted()
        Step implementations processing batch data should call this as soon as they have finished processing after a stop has been requested. Calling this should set the busy flag to false.

      • isStreamFinished

        boolean isStreamFinished​(Data data)
        Returns true if this data object marks the end of an incremental stream. Note - does not check that the data object is actually an incremental one of some sort! Just checks to see if the CON_AUX_DATA_INCREMENTAL_STREAM_END flag is set to true or not;
        Parameters:
        data - the data element to check
        Returns:
        true if the data element is flagged as end of stream

      • throughputUpdateStart

        void throughputUpdateStart()
        Start a throughput measurement. Should only be used by steps that are processing instance streams. Call just before performing a unit of work for an incoming instance.

      • throughputUpdateEnd

        void throughputUpdateEnd()
        End a throughput measurement. Should only be used by steps that are processing instance streams. Call just after finishing a unit of work for an incoming instance

      • throughputFinished

        void throughputFinished​(Data... data)
                 throws WekaException
        Signal that throughput measurement has finished. Should only be used by steps that are emitting incremental data. Call as the completion of an data stream.
        Parameters:
        data - one or more Data events (with appropriate connection type set) to pass on to downstream connected steps. These are used to carry any final data and to inform the downstream step(s) that the stream has ended
        Throws:
        WekaException - if a problem occurs

      • logLow

        void logLow​(java.lang.String message)
        Log a message at the "low" level
        Parameters:
        message - the message to log

      • logBasic

        void logBasic​(java.lang.String message)
        Log a message at the "basic" level
        Parameters:
        message - the message to log

      • logDetailed

        void logDetailed​(java.lang.String message)
        Log a message at the "detailed" level
        Parameters:
        message - the message to log

      • logDebug

        void logDebug​(java.lang.String message)
        Log a message at the "debug" level
        Parameters:
        message - the message to log

      • logWarning

        void logWarning​(java.lang.String message)
        Log a warning message. Always makes it into the log regardless of what logging level the user has specified.
        Parameters:
        message - the message to log

      • logError

        void logError​(java.lang.String message,
              java.lang.Throwable cause)
        Log an error message. Always makes it into the log regardless of what logging level the user has specified. Causes all flow execution to halt. Prints an exception to the log if supplied.
        Parameters:
        message - the message to log
        cause - the optional Throwable to log

      • log

        void log​(java.lang.String message,
         LoggingLevel level)
        Write a message to the log at the given logging level
        Parameters:
        message - the message to write
        level - the level for the message

      • statusMessage

        void statusMessage​(java.lang.String message)
        Write a status message
        Parameters:
        message - the message

      • getLog

        Logger getLog()
        Get the log
        Returns:
        the log object

      • getLoggingLevel

        LoggingLevel getLoggingLevel()
        Get the currently set logging level
        Returns:
        the currently set logging level

      • environmentSubstitute

        java.lang.String environmentSubstitute​(java.lang.String source)
        Substitute all known environment variables in the given string
        Parameters:
        source - the source string
        Returns:
        the source string with all known variables resolved

      • getInfoStep

        Step getInfoStep​(java.lang.Class stepClass)
          throws WekaException
        Returns a reference to the step being managed if it has one or more outgoing CON_INFO connections and the managed step is of the supplied class
        Parameters:
        stepClass - the expected class of the step
        Returns:
        the step being managed if outgoing CON_INFO connections are present and the step is of the supplied class
        Throws:
        WekaException - if there are no outgoing CON_INFO connections or the managed step is the wrong type

      • getInfoStep

        Step getInfoStep()
          throws WekaException
        Returns a reference to the step being managed if it has one or more outgoing CON_INFO connections.
        Returns:
        the step being managed if outgoing CON_INFO connections are present
        Throws:
        WekaException - if there are no outgoing CON_INFO connections

      • findStepInFlow

        StepManager findStepInFlow​(java.lang.String stepNameToFind)
        Finds a named step in the current flow. Returns null if the named step is not present in the flow
        Parameters:
        stepNameToFind - the name of the step to find
        Returns:
        the StepManager of the named step, or null if the step does not exist in the current flow.

      • stepIsResourceIntensive

        boolean stepIsResourceIntensive()
        Returns true if the step managed by this step manager has been marked as being resource (cpu/memory) intensive.
        Returns:
        true if the managed step is resource intensive

      • setStepIsResourceIntensive

        void setStepIsResourceIntensive​(boolean isResourceIntensive)
        Mark the step managed by this step manager as resource intensive
        Parameters:
        isResourceIntensive - true if the step managed by this step manager is resource intensive

      • setStepMustRunSingleThreaded

        void setStepMustRunSingleThreaded​(boolean mustRunSingleThreaded)
        Marked the step managed by this step manager as one that must run single-threaded. I.e. in an executor service with one worker thread, thus effectively preventing more than one copy of the step from executing at any one point in time
        Parameters:
        mustRunSingleThreaded - true if the managed step must run single-threaded

      • getStepMustRunSingleThreaded

        boolean getStepMustRunSingleThreaded()
        Returns true if the step managed by this step manager has been marked as one that must run single-threaded. I.e. in an executor service with one worker thread, thus effectively preventing more than one copy of the step from executing at any one point in time
        Parameters:
        mustRunSingleThreaded - true if the managed step must run single-threaded