Package weka.knowledgeflow.steps

Class MakeResourceIntensive

  • All Implemented Interfaces:
    java.io.Serializable, BaseStepExtender, Step
    @KFStep(name="MakeResourceIntensive",
        category="Flow",
        toolTipText="Makes downstream connected steps resource intensive (or not). This shifts processing of such steps between the main step executor<br>service and the high resource executor service or vice versa.",
        iconPath="weka/gui/knowledgeflow/icons/DiamondPlain.gif")
public class MakeResourceIntensive
extends BaseStep
    A Step that makes downstream steps that are directly connected to this step resource intensive (or not). This overrides whatever the downstream step may (or may not) have declared in it's KFStep class annotation with regards to whether it is resource intensive (cpu or memory). The Knowledge Flow execution environment uses two executor services - a primary one to execute batch processing for steps; and a secondary one for executing StepTasks (which are assumed to be resource intensive by default) or for executing batch processing for a Step when it declares itself to be resource intensive. This secondary executor service uses a limited (typically <= num cpu cores) number of threads. Steps that involve potentially intensive (cpu/memory) processing should declare themselves resource intensive so that less taxing steps (and the UI) get cpu cycles. E.g. the Classifier Step is resource intensive so that processing cross-validation folds in parallel for a large data set or computationally intensive classifier does not blow out memory or bog the system down.
    Version:
    $Revision: $
    Author:
    Mark Hall (mhall{[at]}pentaho{[dot]}com)
    See Also:
    Serialized Form

    • Constructor Detail

      • MakeResourceIntensive

        public MakeResourceIntensive()

    • Method Detail

      • setMakeResourceIntensive

        @OptionMetadata(displayName="Make downstream step(s) high resource",
                description="<html>Makes downstream connected steps resource intensive (or not)<br>This shifts processing of such steps between the main step executor service and the high resource executor service or vice versa.</html>")
public void setMakeResourceIntensive​(boolean resourceIntensive)
        Set whether downstream steps are to be made resource intensive or not
        Parameters:
        resourceIntensive - true if the downstream connected steps are to be made resource intensive

      • getMakeResourceIntensive

        public boolean getMakeResourceIntensive()
        Get whether downstream steps are to be made resource intensive
        Returns:
        true if downstream connected steps are to be made resource intensive

      • getIncomingConnectionTypes

        public java.util.List<java.lang.String> getIncomingConnectionTypes()
        Get a list of incoming connection types that this step can accept. Ideally (and if appropriate), this should take into account the state of the step and any existing incoming connections. E.g. a step might be able to accept one (and only one) incoming batch data connection.
        Returns:
        a list of incoming connections that this step can accept given its current state

      • getOutgoingConnectionTypes

        public java.util.List<java.lang.String> getOutgoingConnectionTypes()
        Get a list of outgoing connection types that this step can produce. Ideally (and if appropriate), this should take into account the state of the step and the incoming connections. E.g. depending on what incoming connection is present, a step might be able to produce a trainingSet output, a testSet output or neither, but not both.
        Returns:
        a list of outgoing connections that this step can produce