Package weka.classifiers.functions

Class SMO

  • All Implemented Interfaces:
    java.io.Serializable, java.lang.Cloneable, Classifier, BatchPredictor, CapabilitiesHandler, CapabilitiesIgnorer, CommandlineRunnable, OptionHandler, RevisionHandler, TechnicalInformationHandler, WeightedInstancesHandler
    public class SMO
extends AbstractClassifier
implements WeightedInstancesHandler, TechnicalInformationHandler
    Implements John Platt's sequential minimal optimization algorithm for training a support vector classifier.

    This implementation globally replaces all missing values and transforms nominal attributes into binary ones. It also normalizes all attributes by default. (In that case the coefficients in the output are based on the normalized data, not the original data --- this is important for interpreting the classifier.)

    Multi-class problems are solved using pairwise classification (aka 1-vs-1).

    To obtain proper probability estimates, use the option that fits calibration models to the outputs of the support vector machine. In the multi-class case, the predicted probabilities are coupled using Hastie and Tibshirani's pairwise coupling method.

    Note: for improved speed normalization should be turned off when operating on SparseInstances.

    For more information on the SMO algorithm, see

    J. Platt: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In B. Schoelkopf and C. Burges and A. Smola, editors, Advances in Kernel Methods - Support Vector Learning, 1998.

    S.S. Keerthi, S.K. Shevade, C. Bhattacharyya, K.R.K. Murthy (2001). Improvements to Platt's SMO Algorithm for SVM Classifier Design. Neural Computation. 13(3):637-649.

    Trevor Hastie, Robert Tibshirani: Classification by Pairwise Coupling. In: Advances in Neural Information Processing Systems, 1998.

    BibTeX: 
     @incollection{Platt1998,
    author = {J. Platt},
    booktitle = {Advances in Kernel Methods - Support Vector Learning},
    editor = {B. Schoelkopf and C. Burges and A. Smola},
    publisher = {MIT Press},
    title = {Fast Training of Support Vector Machines using Sequential Minimal Optimization},
    year = {1998},
    URL = {http://research.microsoft.com/\~jplatt/smo.html},
    PS = {http://research.microsoft.com/\~jplatt/smo-book.ps.gz},
    PDF = {http://research.microsoft.com/\~jplatt/smo-book.pdf}
 }
 
 @article{Keerthi2001,
    author = {S.S. Keerthi and S.K. Shevade and C. Bhattacharyya and K.R.K. Murthy},
    journal = {Neural Computation},
    number = {3},
    pages = {637-649},
    title = {Improvements to Platt's SMO Algorithm for SVM Classifier Design},
    volume = {13},
    year = {2001},
    PS = {http://guppy.mpe.nus.edu.sg/\~mpessk/svm/smo_mod_nc.ps.gz}
 }
 
 @inproceedings{Hastie1998,
    author = {Trevor Hastie and Robert Tibshirani},
    booktitle = {Advances in Neural Information Processing Systems},
    editor = {Michael I. Jordan and Michael J. Kearns and Sara A. Solla},
    publisher = {MIT Press},
    title = {Classification by Pairwise Coupling},
    volume = {10},
    year = {1998},
    PS = {http://www-stat.stanford.edu/\~hastie/Papers/2class.ps}
 }


    Valid options are: 

     -no-checks
  Turns off all checks - use with caution!
  Turning them off assumes that data is purely numeric, doesn't
  contain any missing values, and has a nominal class. Turning them
  off also means that no header information will be stored if the
  machine is linear. Finally, it also assumes that no instance has
  a weight equal to 0.
  (default: checks on)
     -C <double>
  The complexity constant C. (default 1)
     -N
  Whether to 0=normalize/1=standardize/2=neither. (default 0=normalize)
     -L <double>
  The tolerance parameter. (default 1.0e-3)
     -P <double>
  The epsilon for round-off error. (default 1.0e-12)
     -M
  Fit calibration models to SVM outputs. 
     -V <double>
  The number of folds for the internal
  cross-validation. (default -1, use training data)
     -W <double>
  The random number seed. (default 1)
     -K <classname and parameters>
  The Kernel to use.
  (default: weka.classifiers.functions.supportVector.PolyKernel)
     -calibrator <scheme specification>
  Full name of calibration model, followed by options.
  (default: "weka.classifiers.functions.Logistic")
     -output-debug-info
  If set, classifier is run in debug mode and
  may output additional info to the console
     -do-not-check-capabilities
  If set, classifier capabilities are not checked before classifier is built
  (use with caution).
     -num-decimal-places
  The number of decimal places for the output of numbers in the model (default 2).
     
 Options specific to kernel weka.classifiers.functions.supportVector.PolyKernel:
 
     -E <num>
  The Exponent to use.
  (default: 1.0)
     -L
  Use lower-order terms.
  (default: no)
     -C <num>
  The size of the cache (a prime number), 0 for full cache and 
  -1 to turn it off.
  (default: 250007)
     -output-debug-info
  Enables debugging output (if available) to be printed.
  (default: off)
     -no-checks
  Turns off all checks - use with caution!
  (default: checks on)
     
 Options specific to calibrator weka.classifiers.functions.Logistic:
 
     -C
  Use conjugate gradient descent rather than BFGS updates.
     -R <ridge>
  Set the ridge in the log-likelihood.
     -M <number>
  Set the maximum number of iterations (default -1, until convergence).
     -output-debug-info
  If set, classifier is run in debug mode and
  may output additional info to the console
     -do-not-check-capabilities
  If set, classifier capabilities are not checked before classifier is built
  (use with caution).
     -num-decimal-places
  The number of decimal places for the output of numbers in the model (default 2).
    Version:
    $Revision: 14825 $
    Author:
    Eibe Frank (eibe@cs.waikato.ac.nz), Shane Legg (shane@intelligenesis.net) (sparse vector code), Stuart Inglis (stuart@reeltwo.com) (sparse vector code)
    See Also:
    Serialized Form

    • Field Detail

      • FILTER_NORMALIZE

        public static final int FILTER_NORMALIZE
        filter: Normalize training data
        See Also:
        Constant Field Values

      • FILTER_STANDARDIZE

        public static final int FILTER_STANDARDIZE
        filter: Standardize training data
        See Also:
        Constant Field Values

      • FILTER_NONE

        public static final int FILTER_NONE
        filter: No normalization/standardization
        See Also:
        Constant Field Values

      • TAGS_FILTER

        public static final Tag[] TAGS_FILTER
        The filter to apply to the training data

    • Constructor Detail

      • SMO

        public SMO()

    • Method Detail

      • globalInfo

        public java.lang.String globalInfo()
        Returns a string describing classifier
        Returns:
        a description suitable for displaying in the explorer/experimenter gui

      • getTechnicalInformation

        public TechnicalInformation getTechnicalInformation()
        Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.
        Specified by:
        getTechnicalInformation in interface TechnicalInformationHandler
        Returns:
        the technical information about this class

      • turnChecksOff

        public void turnChecksOff()
        Turns off checks for missing values, etc. Use with caution.

      • turnChecksOn

        public void turnChecksOn()
        Turns on checks for missing values, etc.

      • buildClassifier

        public void buildClassifier​(Instances insts)
                     throws java.lang.Exception
        Method for building the classifier. Implements a one-against-one wrapper for multi-class problems.
        Specified by:
        buildClassifier in interface Classifier
        Parameters:
        insts - the set of training instances
        Throws:
        java.lang.Exception - if the classifier can't be built successfully

      • distributionForInstance

        public double[] distributionForInstance​(Instance inst)
                                 throws java.lang.Exception
        Estimates class probabilities for given instance.
        Specified by:
        distributionForInstance in interface Classifier
        Overrides:
        distributionForInstance in class AbstractClassifier
        Parameters:
        inst - the instance to compute the probabilities for
        Returns:
        an array containing the estimated membership probabilities of the test instance in each class or the numeric prediction
        Throws:
        java.lang.Exception - in case of an error

      • obtainVotes

        public int[] obtainVotes​(Instance inst)
                  throws java.lang.Exception
        Returns an array of votes for the given instance.
        Parameters:
        inst - the instance
        Returns:
        array of votex
        Throws:
        java.lang.Exception - if something goes wrong

      • sparseWeights

        public double[][][] sparseWeights()
        Returns the weights in sparse format.

      • sparseIndices

        public int[][][] sparseIndices()
        Returns the indices in sparse format.

      • bias

        public double[][] bias()
        Returns the bias of each binary SMO.

      • numClassAttributeValues

        public int numClassAttributeValues()

      • classAttributeNames

        public java.lang.String[] classAttributeNames()

      • attributeNames

        public java.lang.String[][][] attributeNames()
        Returns the attribute names.

      • setOptions

        public void setOptions​(java.lang.String[] options)
                throws java.lang.Exception
        Parses a given list of options.

        Valid options are: 

         -no-checks
    Turns off all checks - use with caution!
    Turning them off assumes that data is purely numeric, doesn't
    contain any missing values, and has a nominal class. Turning them
    off also means that no header information will be stored if the
    machine is linear. Finally, it also assumes that no instance has
    a weight equal to 0.
    (default: checks on)
         -C <double>
    The complexity constant C. (default 1)
         -N
    Whether to 0=normalize/1=standardize/2=neither. (default 0=normalize)
         -L <double>
    The tolerance parameter. (default 1.0e-3)
         -P <double>
    The epsilon for round-off error. (default 1.0e-12)
         -M
    Fit calibration models to SVM outputs. 
         -V <double>
    The number of folds for the internal
    cross-validation. (default -1, use training data)
         -W <double>
    The random number seed. (default 1)
         -K <classname and parameters>
    The Kernel to use.
    (default: weka.classifiers.functions.supportVector.PolyKernel)
         -calibrator <scheme specification>
    Full name of calibration model, followed by options.
    (default: "weka.classifiers.functions.Logistic")
         -output-debug-info
    If set, classifier is run in debug mode and
    may output additional info to the console
         -do-not-check-capabilities
    If set, classifier capabilities are not checked before classifier is built
    (use with caution).
         -num-decimal-places
    The number of decimal places for the output of numbers in the model (default 2).
         
   Options specific to kernel weka.classifiers.functions.supportVector.PolyKernel:
   
         -E <num>
    The Exponent to use.
    (default: 1.0)
         -L
    Use lower-order terms.
    (default: no)
         -C <num>
    The size of the cache (a prime number), 0 for full cache and 
    -1 to turn it off.
    (default: 250007)
         -output-debug-info
    Enables debugging output (if available) to be printed.
    (default: off)
         -no-checks
    Turns off all checks - use with caution!
    (default: checks on)
         
   Options specific to calibrator weka.classifiers.functions.Logistic:
   
         -C
    Use conjugate gradient descent rather than BFGS updates.
         -R <ridge>
    Set the ridge in the log-likelihood.
         -M <number>
    Set the maximum number of iterations (default -1, until convergence).
         -output-debug-info
    If set, classifier is run in debug mode and
    may output additional info to the console
         -do-not-check-capabilities
    If set, classifier capabilities are not checked before classifier is built
    (use with caution).
         -num-decimal-places
    The number of decimal places for the output of numbers in the model (default 2).
        Specified by:
        setOptions in interface OptionHandler
        Overrides:
        setOptions in class AbstractClassifier
        Parameters:
        options - the list of options as an array of strings
        Throws:
        java.lang.Exception - if an option is not supported

      • getOptions

        public java.lang.String[] getOptions()
        Gets the current settings of the classifier.
        Specified by:
        getOptions in interface OptionHandler
        Overrides:
        getOptions in class AbstractClassifier
        Returns:
        an array of strings suitable for passing to setOptions

      • setChecksTurnedOff

        public void setChecksTurnedOff​(boolean value)
        Disables or enables the checks (which could be time-consuming). Use with caution!
        Parameters:
        value - if true turns off all checks

      • getChecksTurnedOff

        public boolean getChecksTurnedOff()
        Returns whether the checks are turned off or not.
        Returns:
        true if the checks are turned off

      • checksTurnedOffTipText

        public java.lang.String checksTurnedOffTipText()
        Returns the tip text for this property
        Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

      • kernelTipText

        public java.lang.String kernelTipText()
        Returns the tip text for this property
        Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

      • setKernel

        public void setKernel​(Kernel value)
        sets the kernel to use
        Parameters:
        value - the kernel to use

      • getKernel

        public Kernel getKernel()
        Returns the kernel to use
        Returns:
        the current kernel

      • calibratorTipText

        public java.lang.String calibratorTipText()
        Returns the tip text for this property
        Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

      • setCalibrator

        public void setCalibrator​(Classifier value)
        sets the calibrator to use
        Parameters:
        value - the calibrator to use

      • getCalibrator

        public Classifier getCalibrator()
        Returns the calibrator to use
        Returns:
        the current calibrator

      • cTipText

        public java.lang.String cTipText()
        Returns the tip text for this property
        Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

      • getC

        public double getC()
        Get the value of C.
        Returns:
        Value of C.

      • setC

        public void setC​(double v)
        Set the value of C.
        Parameters:
        v - Value to assign to C.

      • toleranceParameterTipText

        public java.lang.String toleranceParameterTipText()
        Returns the tip text for this property
        Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

      • getToleranceParameter

        public double getToleranceParameter()
        Get the value of tolerance parameter.
        Returns:
        Value of tolerance parameter.

      • setToleranceParameter

        public void setToleranceParameter​(double v)
        Set the value of tolerance parameter.
        Parameters:
        v - Value to assign to tolerance parameter.

      • epsilonTipText

        public java.lang.String epsilonTipText()
        Returns the tip text for this property
        Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

      • getEpsilon

        public double getEpsilon()
        Get the value of epsilon.
        Returns:
        Value of epsilon.

      • setEpsilon

        public void setEpsilon​(double v)
        Set the value of epsilon.
        Parameters:
        v - Value to assign to epsilon.

      • filterTypeTipText

        public java.lang.String filterTypeTipText()
        Returns the tip text for this property
        Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

      • getFilterType

        public SelectedTag getFilterType()
        Gets how the training data will be transformed. Will be one of FILTER_NORMALIZE, FILTER_STANDARDIZE, FILTER_NONE.
        Returns:
        the filtering mode

      • setFilterType

        public void setFilterType​(SelectedTag newType)
        Sets how the training data will be transformed. Should be one of FILTER_NORMALIZE, FILTER_STANDARDIZE, FILTER_NONE.
        Parameters:
        newType - the new filtering mode

      • buildCalibrationModelsTipText

        public java.lang.String buildCalibrationModelsTipText()
        Returns the tip text for this property
        Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

      • getBuildCalibrationModels

        public boolean getBuildCalibrationModels()
        Get the value of buildCalibrationModels.
        Returns:
        Value of buildCalibrationModels.

      • setBuildCalibrationModels

        public void setBuildCalibrationModels​(boolean newbuildCalibrationModels)
        Set the value of buildCalibrationModels.
        Parameters:
        newbuildCalibrationModels - Value to assign to buildCalibrationModels.

      • numFoldsTipText

        public java.lang.String numFoldsTipText()
        Returns the tip text for this property
        Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

      • getNumFolds

        public int getNumFolds()
        Get the value of numFolds.
        Returns:
        Value of numFolds.

      • setNumFolds

        public void setNumFolds​(int newnumFolds)
        Set the value of numFolds.
        Parameters:
        newnumFolds - Value to assign to numFolds.

      • randomSeedTipText

        public java.lang.String randomSeedTipText()
        Returns the tip text for this property
        Returns:
        tip text for this property suitable for displaying in the explorer/experimenter gui

      • getRandomSeed

        public int getRandomSeed()
        Get the value of randomSeed.
        Returns:
        Value of randomSeed.

      • setRandomSeed

        public void setRandomSeed​(int newrandomSeed)
        Set the value of randomSeed.
        Parameters:
        newrandomSeed - Value to assign to randomSeed.

      • toString

        public java.lang.String toString()
        Prints out the classifier.
        Overrides:
        toString in class java.lang.Object
        Returns:
        a description of the classifier as a string

      • main

        public static void main​(java.lang.String[] argv)
        Main method for testing this class.