Class LogitBoost

All Implemented Interfaces:
Serializable, Cloneable, Sourcable, CapabilitiesHandler, OptionHandler, Randomizable, RevisionHandler, TechnicalInformationHandler, WeightedInstancesHandler

Class for performing additive logistic regression.
This class performs classification using a regression scheme as the base learner, and can handle multi-class problems. For more information, see

J. Friedman, T. Hastie, R. Tibshirani (1998). Additive Logistic Regression: a Statistical View of Boosting. Stanford University.

Can do efficient internal cross-validation to determine appropriate number of iterations.

BibTeX:

 @techreport{Friedman1998,
    address = {Stanford University},
    author = {J. Friedman and T. Hastie and R. Tibshirani},
    title = {Additive Logistic Regression: a Statistical View of Boosting},
    year = {1998},
    PS = {http://www-stat.stanford.edu/\~jhf/ftp/boost.ps}
 }
 

Valid options are:

 -Q
  Use resampling instead of reweighting for boosting.
 -P <percent>
  Percentage of weight mass to base training on.
  (default 100, reduce to around 90 speed up)
 -F <num>
  Number of folds for internal cross-validation.
  (default 0 -- no cross-validation)
 -R <num>
  Number of runs for internal cross-validation.
  (default 1)
 -L <num>
  Threshold on the improvement of the likelihood.
  (default -Double.MAX_VALUE)
 -H <num>
  Shrinkage parameter.
  (default 1)
 -S <num>
  Random number seed.
  (default 1)
 -I <num>
  Number of iterations.
  (default 10)
 -D
  If set, classifier is run in debug mode and
  may output additional info to the console
 -W
  Full name of base classifier.
  (default: weka.classifiers.trees.DecisionStump)
 
 Options specific to classifier weka.classifiers.trees.DecisionStump:
 
 -D
  If set, classifier is run in debug mode and
  may output additional info to the console
Options after -- are passed to the designated learner.

Version:
$Revision: 9371 $
Author:
Len Trigg (trigg@cs.waikato.ac.nz), Eibe Frank (eibe@cs.waikato.ac.nz)
See Also:
  • Constructor Details

    • LogitBoost

      public LogitBoost()
      Constructor.
  • Method Details

    • globalInfo

      public String globalInfo()
      Returns a string describing classifier
      Returns:
      a description suitable for displaying in the explorer/experimenter gui
    • getTechnicalInformation

      public TechnicalInformation getTechnicalInformation()
      Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.
      Specified by:
      getTechnicalInformation in interface TechnicalInformationHandler
      Returns:
      the technical information about this class
    • listOptions

      public Enumeration listOptions()
      Returns an enumeration describing the available options.
      Specified by:
      listOptions in interface OptionHandler
      Overrides:
      listOptions in class RandomizableIteratedSingleClassifierEnhancer
      Returns:
      an enumeration of all the available options.
    • setOptions

      public void setOptions(String[] options) throws Exception
      Parses a given list of options.

      Valid options are:

       -Q
        Use resampling instead of reweighting for boosting.
       -P <percent>
        Percentage of weight mass to base training on.
        (default 100, reduce to around 90 speed up)
       -F <num>
        Number of folds for internal cross-validation.
        (default 0 -- no cross-validation)
       -R <num>
        Number of runs for internal cross-validation.
        (default 1)
       -L <num>
        Threshold on the improvement of the likelihood.
        (default -Double.MAX_VALUE)
       -H <num>
        Shrinkage parameter.
        (default 1)
       -S <num>
        Random number seed.
        (default 1)
       -I <num>
        Number of iterations.
        (default 10)
       -D
        If set, classifier is run in debug mode and
        may output additional info to the console
       -W
        Full name of base classifier.
        (default: weka.classifiers.trees.DecisionStump)
       
       Options specific to classifier weka.classifiers.trees.DecisionStump:
       
       -D
        If set, classifier is run in debug mode and
        may output additional info to the console
      Options after -- are passed to the designated learner.

      Specified by:
      setOptions in interface OptionHandler
      Overrides:
      setOptions in class RandomizableIteratedSingleClassifierEnhancer
      Parameters:
      options - the list of options as an array of strings
      Throws:
      Exception - if an option is not supported
    • getOptions

      public String[] getOptions()
      Gets the current settings of the Classifier.
      Specified by:
      getOptions in interface OptionHandler
      Overrides:
      getOptions in class RandomizableIteratedSingleClassifierEnhancer
      Returns:
      an array of strings suitable for passing to setOptions
    • shrinkageTipText

      public String shrinkageTipText()
      Returns the tip text for this property
      Returns:
      tip text for this property suitable for displaying in the explorer/experimenter gui
    • getShrinkage

      public double getShrinkage()
      Get the value of Shrinkage.
      Returns:
      Value of Shrinkage.
    • setShrinkage

      public void setShrinkage(double newShrinkage)
      Set the value of Shrinkage.
      Parameters:
      newShrinkage - Value to assign to Shrinkage.
    • likelihoodThresholdTipText

      public String likelihoodThresholdTipText()
      Returns the tip text for this property
      Returns:
      tip text for this property suitable for displaying in the explorer/experimenter gui
    • getLikelihoodThreshold

      public double getLikelihoodThreshold()
      Get the value of Precision.
      Returns:
      Value of Precision.
    • setLikelihoodThreshold

      public void setLikelihoodThreshold(double newPrecision)
      Set the value of Precision.
      Parameters:
      newPrecision - Value to assign to Precision.
    • numRunsTipText

      public String numRunsTipText()
      Returns the tip text for this property
      Returns:
      tip text for this property suitable for displaying in the explorer/experimenter gui
    • getNumRuns

      public int getNumRuns()
      Get the value of NumRuns.
      Returns:
      Value of NumRuns.
    • setNumRuns

      public void setNumRuns(int newNumRuns)
      Set the value of NumRuns.
      Parameters:
      newNumRuns - Value to assign to NumRuns.
    • numFoldsTipText

      public String numFoldsTipText()
      Returns the tip text for this property
      Returns:
      tip text for this property suitable for displaying in the explorer/experimenter gui
    • getNumFolds

      public int getNumFolds()
      Get the value of NumFolds.
      Returns:
      Value of NumFolds.
    • setNumFolds

      public void setNumFolds(int newNumFolds)
      Set the value of NumFolds.
      Parameters:
      newNumFolds - Value to assign to NumFolds.
    • useResamplingTipText

      public String useResamplingTipText()
      Returns the tip text for this property
      Returns:
      tip text for this property suitable for displaying in the explorer/experimenter gui
    • setUseResampling

      public void setUseResampling(boolean r)
      Set resampling mode
      Parameters:
      r - true if resampling should be done
    • getUseResampling

      public boolean getUseResampling()
      Get whether resampling is turned on
      Returns:
      true if resampling output is on
    • weightThresholdTipText

      public String weightThresholdTipText()
      Returns the tip text for this property
      Returns:
      tip text for this property suitable for displaying in the explorer/experimenter gui
    • setWeightThreshold

      public void setWeightThreshold(int threshold)
      Set weight thresholding
      Parameters:
      threshold - the percentage of weight mass used for training
    • getWeightThreshold

      public int getWeightThreshold()
      Get the degree of weight thresholding
      Returns:
      the percentage of weight mass used for training
    • getCapabilities

      public Capabilities getCapabilities()
      Returns default capabilities of the classifier.
      Specified by:
      getCapabilities in interface CapabilitiesHandler
      Overrides:
      getCapabilities in class SingleClassifierEnhancer
      Returns:
      the capabilities of this classifier
      See Also:
    • buildClassifier

      public void buildClassifier(Instances data) throws Exception
      Builds the boosted classifier
      Overrides:
      buildClassifier in class IteratedSingleClassifierEnhancer
      Parameters:
      data - the data to train the classifier with
      Throws:
      Exception - if building fails, e.g., can't handle data
    • classifiers

      public Classifier[][] classifiers()
      Returns the array of classifiers that have been built.
      Returns:
      the built classifiers
    • distributionForInstance

      public double[] distributionForInstance(Instance instance) throws Exception
      Calculates the class membership probabilities for the given test instance.
      Overrides:
      distributionForInstance in class Classifier
      Parameters:
      instance - the instance to be classified
      Returns:
      predicted class probability distribution
      Throws:
      Exception - if instance could not be classified successfully
    • toSource

      public String toSource(String className) throws Exception
      Returns the boosted model as Java source code.
      Specified by:
      toSource in interface Sourcable
      Parameters:
      className - the classname in the generated code
      Returns:
      the tree as Java source code
      Throws:
      Exception - if something goes wrong
    • toString

      public String toString()
      Returns description of the boosted classifier.
      Overrides:
      toString in class Object
      Returns:
      description of the boosted classifier as a string
    • getRevision

      public String getRevision()
      Returns the revision string.
      Specified by:
      getRevision in interface RevisionHandler
      Overrides:
      getRevision in class Classifier
      Returns:
      the revision
    • main

      public static void main(String[] argv)
      Main method for testing this class.
      Parameters:
      argv - the options