SG++-Doxygen-Documentation
python.controller.CheckpointController.CheckpointController Class Reference

Class for handling events for storing and restoring of checkpoints. More...

Inheritance diagram for python.controller.CheckpointController.CheckpointController:

Public Member Functions

def __init__ (self, title, path=None, interval=None, fold=None)
 Constructor save checkpoint files will have a name like title.iteration. More...
 
def composeName (self, iteration=None, fold=None)
 Composes checkpoint file name from path title and iteration number. More...
 
def generateFoldValidationJob (self, email="")
 Generates a SGE array job script for concurrent performing of cross-fold validation computations. More...
 
def handleLearningEvent (self, subject, status)
 
def loadAll (self, iteration)
 Loads the Learner object with corresponding Grid and LearnedKnowledge. More...
 
def loadGrid (self, iteration)
 Loads sg.Grid from the checkpoint of given iteration. More...
 
def loadLearnedKnowledge (self, iteration)
 Loads knowledge from the checkpoint of given iteration. More...
 
def saveAll (self, iteration)
 Saves current Grid, LearnedKnowledge and Learner objects. More...
 
def saveGrid (self, iteration)
 Saves current Grid to the checkpoint with given iteration. More...
 
def saveLearnedKnowledge (self, iteration)
 Saves current LearnedKnowldge object to the checkpoint with given iteration. More...
 
def saveLearner (self, iteration)
 Saves the current Learner object to the checkpoint with given iteration. More...
 
def setGrid (self, grid)
 Setter for the current Grid object. More...
 
def setLearnedKnowledge (self, knowledge)
 Setter for the current LearnedKnowledge object. More...
 
def setLearner (self, learner)
 Setter for current Learner object. More...
 

Static Public Attributes

 fold = None
 Fold number if cross-validation is used. More...
 
 grid = None
 Reference to the Grid object. More...
 
 interval = None
 Interval, which determines how often checkpoints should be set. More...
 
 knowledge = None
 Reference to the LearnedKnowledge object. More...
 
 lastIteration = None
 Number of the last iteration. More...
 
 learner = None
 Reference to the Learner object. More...
 
 path = None
 Path to the checkpoint files. More...
 
 title = None
 Checkpoint title. More...
 

Detailed Description

Class for handling events for storing and restoring of checkpoints.

Responsible for storing of checkpoints during learning process and restoring of learning from some checkpoint.

Memento Design Pattern

CheckpointController has an ability to save the states of Learner, its Grid and LearnedKnowledge for different iterations of training process. To insure flexibility and extensibility of the system the design pattern Memento with several modifications was used.

Roles

  • Originator: Grid, LearnedKnowledge, Learner
  • Memento: Grid object itself for Grid, DataVector with alpha values for LearnedKnowledge object, json structure for Learner obejects
  • Caretaker: CeckpointController

Why should we use Memento pattern here?

  • Simplification of inner structure of classes, since they don't need to organized and store different states by themselves;
  • Decoupling of objects of business logic and their representations allows to assure compatibility of between different versions and support of different input/output formats without modification of business classes;
  • Client or Caretaker don't have to know about the organization of originator to store/restore checkpoints.

Enhancements
To be able able to use the states after the program has once terminated, we have to make memento objects persistent. For this purpose the Classes GridFormatter, LearnedKnowledgeFormatter and LearnerFormatter are used, which provides the functionality of serialization and desiralization of corresponding objects into file or any other stream, e.g. socket or terminal.

For example, to save the Grid object in file, controller requests its memento object with states and then calls method GridFormatter().serializeToFile(<Grid memento object>, <file name>).

How to restore a checkpoint

Supposed, you have already learned and stored checkpoints and now you have files checkpoint.6.arff.gz, checkpoint.6.grid.gz and checkpoint.6.learner.gz from the 6th iteration stored in your working directory. Now you can

Constructor & Destructor Documentation

◆ __init__()

def python.controller.CheckpointController.CheckpointController.__init__ (   self,
  title,
  path = None,
  interval = None,
  fold = None 
)

Constructor save checkpoint files will have a name like title.iteration.

[grid | learner | arff].gz

Parameters
titlestring title for checkpoints
pathstring absolute path to the checkpoint files
intervalinteger defines the number of iteration between saved checkpoints, e.g. if interval = 1, only states from 0,5th, 10th, 15th ... iteration will be saved
foldthe folding level, if n-fold cross-validation is used

References python.controller.CheckpointController.CheckpointController.fold, python.controller.CheckpointController.CheckpointController.interval, python.controller.CheckpointController.CheckpointController.path, and python.controller.CheckpointController.CheckpointController.title.

Member Function Documentation

◆ composeName()

◆ generateFoldValidationJob()

def python.controller.CheckpointController.CheckpointController.generateFoldValidationJob (   self,
  email = "" 
)

Generates a SGE array job script for concurrent performing of cross-fold validation computations.

The script can be then lunched using <code>qsub -t 1-XXX <scriptname>.sge.job

Parameters
emailString with email-address, the status information from SGE should be sent to

References python.controller.CheckpointController.CheckpointController.composeName(), python.controller.CheckpointController.CheckpointController.fold, sgpp::base::OperationMultipleEval.grid, sgpp::base::OperationFirstMomentModPolyClenshawCurtis.grid, sgpp::base::OperationFirstMomentPoly.grid, sgpp::base::OperationSecondMomentPolyBoundary.grid, sgpp::base::OperationSecondMomentPolyClenshawCurtis.grid, sgpp::base::OperationFirstMomentPolyBoundary.grid, sgpp::base::OperationFirstMomentPolyClenshawCurtis.grid, sgpp::base::OperationFirstMomentBspline.grid, sgpp::base::OperationFirstMomentBsplineBoundary.grid, sgpp::base::OperationFirstMomentModBspline.grid, sgpp::base::OperationSecondMomentPoly.grid, sgpp::base::OperationSecondMomentBspline.grid, sgpp::base::OperationSecondMomentBsplineBoundary.grid, sgpp::base::OperationSecondMomentModBspline.grid, sgpp::base::OperationFirstMomentModPoly.grid, sgpp::base::OperationSecondMomentModPoly.grid, sgpp::base::OperationSecondMomentModPolyClenshawCurtis.grid, sgpp::base::OperationSecondMomentPolyClenshawCurtisBoundary.grid, sgpp::base::OperationFirstMomentPolyClenshawCurtisBoundary.grid, sgpp::base::OperationSecondMomentModBsplineClenshawCurtis.grid, sgpp::base::OperationFirstMomentBsplineClenshawCurtis.grid, sgpp::base::OperationFirstMomentModBsplineClenshawCurtis.grid, sgpp::base::OperationSecondMomentBsplineClenshawCurtis.grid, sgpp::base::OperationHierarchisationFundamentalSpline.grid, sgpp::base::OperationHierarchisationModFundamentalSpline.grid, sgpp::base::OperationArbitraryBoundaryHierarchisation.grid, sgpp::base::DehierarchisationFundamentalSpline.grid, sgpp::base::HierarchisationModFundamentalSpline.grid, sgpp::base::HierarchisationFundamentalSpline.grid, sgpp::base::DehierarchisationModFundamentalSpline.grid, sgpp::base::OperationQuadratureMC.grid, python.controller.CheckpointController.CheckpointController.grid, sgpp::combigrid::LTwoScalarProductHashMapNakBsplineBoundaryCombigrid.grid, sgpp::base::ImpurityRefinementIndicator.grid, sgpp::base::ForwardSelectorRefinementIndicator.grid, python.controller.CheckpointController.CheckpointController.knowledge, python.controller.CheckpointController.CheckpointController.learner, python.controller.CheckpointController.CheckpointController.saveGrid(), python.controller.CheckpointController.CheckpointController.saveLearnedKnowledge(), and python.controller.CheckpointController.CheckpointController.title.

◆ handleLearningEvent()

def python.controller.CheckpointController.CheckpointController.handleLearningEvent (   self,
  subject,
  status 
)

◆ loadAll()

def python.controller.CheckpointController.CheckpointController.loadAll (   self,
  iteration 
)

Loads the Learner object with corresponding Grid and LearnedKnowledge.

Parameters
iterationInteger iteration number.
Returns
the Learner object.

References python.controller.CheckpointController.CheckpointController.__getCOperatorType(), python.controller.CheckpointController.CheckpointController.__getLearnerEventControllers(), python.controller.CheckpointController.CheckpointController.__getSolverEventControllers(), python.controller.CheckpointController.CheckpointController.__loadLearner(), sgpp::op_factory.createOperationIdentity(), sgpp::op_factory.createOperationLaplace(), sgpp::base::OperationMultipleEval.grid, sgpp::base::OperationFirstMomentPolyBoundary.grid, sgpp::base::OperationFirstMomentPolyClenshawCurtis.grid, sgpp::base::OperationFirstMomentBspline.grid, sgpp::base::OperationFirstMomentModBspline.grid, sgpp::base::OperationFirstMomentBsplineBoundary.grid, sgpp::base::OperationSecondMomentBspline.grid, sgpp::base::OperationSecondMomentBsplineBoundary.grid, sgpp::base::OperationFirstMomentModPoly.grid, sgpp::base::OperationSecondMomentModBspline.grid, sgpp::base::OperationSecondMomentModPoly.grid, sgpp::base::OperationSecondMomentPolyClenshawCurtis.grid, sgpp::base::OperationSecondMomentPolyBoundary.grid, sgpp::base::OperationFirstMomentModPolyClenshawCurtis.grid, sgpp::base::OperationSecondMomentModPolyClenshawCurtis.grid, sgpp::base::OperationSecondMomentPoly.grid, sgpp::base::OperationFirstMomentPoly.grid, sgpp::base::OperationSecondMomentPolyClenshawCurtisBoundary.grid, sgpp::base::OperationFirstMomentPolyClenshawCurtisBoundary.grid, sgpp::base::OperationFirstMomentBsplineClenshawCurtis.grid, sgpp::base::OperationFirstMomentModBsplineClenshawCurtis.grid, sgpp::base::OperationSecondMomentBsplineClenshawCurtis.grid, sgpp::base::OperationSecondMomentModBsplineClenshawCurtis.grid, sgpp::base::OperationHierarchisationFundamentalSpline.grid, sgpp::base::OperationHierarchisationModFundamentalSpline.grid, sgpp::base::OperationArbitraryBoundaryHierarchisation.grid, sgpp::base::HierarchisationFundamentalSpline.grid, sgpp::base::DehierarchisationFundamentalSpline.grid, sgpp::base::HierarchisationModFundamentalSpline.grid, sgpp::base::DehierarchisationModFundamentalSpline.grid, sgpp::base::OperationQuadratureMC.grid, python.controller.CheckpointController.CheckpointController.grid, sgpp::combigrid::LTwoScalarProductHashMapNakBsplineBoundaryCombigrid.grid, sgpp::base::ForwardSelectorRefinementIndicator.grid, sgpp::base::ImpurityRefinementIndicator.grid, python.controller.CheckpointController.CheckpointController.knowledge, python.controller.CheckpointController.CheckpointController.learner, python.controller.CheckpointController.CheckpointController.loadGrid(), python.controller.CheckpointController.CheckpointController.loadLearnedKnowledge(), python.controller.CheckpointController.CheckpointController.setGrid(), and python.controller.CheckpointController.CheckpointController.setLearnedKnowledge().

◆ loadGrid()

def python.controller.CheckpointController.CheckpointController.loadGrid (   self,
  iteration 
)

Loads sg.Grid from the checkpoint of given iteration.

Parameters
iterationinteger iteration number
Returns
Grid object

References python.controller.CheckpointController.CheckpointController.composeName().

Referenced by python.controller.CheckpointController.CheckpointController.loadAll().

◆ loadLearnedKnowledge()

def python.controller.CheckpointController.CheckpointController.loadLearnedKnowledge (   self,
  iteration 
)

Loads knowledge from the checkpoint of given iteration.

Parameters
iterationinteger iteration number
Returns
LearnedKnowledge object

References python.controller.CheckpointController.CheckpointController.composeName().

Referenced by python.controller.CheckpointController.CheckpointController.loadAll().

◆ saveAll()

def python.controller.CheckpointController.CheckpointController.saveAll (   self,
  iteration 
)

◆ saveGrid()

def python.controller.CheckpointController.CheckpointController.saveGrid (   self,
  iteration 
)

Saves current Grid to the checkpoint with given iteration.

Parameters
iterationinteger iteration number

References python.controller.CheckpointController.CheckpointController.composeName(), sgpp::base::OperationMultipleEval.grid, sgpp::base::OperationFirstMomentPoly.grid, sgpp::base::OperationFirstMomentPolyBoundary.grid, sgpp::base::OperationSecondMomentPolyClenshawCurtis.grid, sgpp::base::OperationFirstMomentPolyClenshawCurtis.grid, sgpp::base::OperationSecondMomentPolyBoundary.grid, sgpp::base::OperationSecondMomentPoly.grid, sgpp::base::OperationSecondMomentModPolyClenshawCurtis.grid, sgpp::base::OperationFirstMomentBspline.grid, sgpp::base::OperationFirstMomentBsplineBoundary.grid, sgpp::base::OperationSecondMomentBspline.grid, sgpp::base::OperationFirstMomentModBspline.grid, sgpp::base::OperationSecondMomentBsplineBoundary.grid, sgpp::base::OperationSecondMomentModBspline.grid, sgpp::base::OperationFirstMomentModPoly.grid, sgpp::base::OperationSecondMomentModPoly.grid, sgpp::base::OperationFirstMomentModPolyClenshawCurtis.grid, sgpp::base::OperationSecondMomentPolyClenshawCurtisBoundary.grid, sgpp::base::OperationFirstMomentPolyClenshawCurtisBoundary.grid, sgpp::base::OperationSecondMomentModBsplineClenshawCurtis.grid, sgpp::base::OperationFirstMomentBsplineClenshawCurtis.grid, sgpp::base::OperationFirstMomentModBsplineClenshawCurtis.grid, sgpp::base::OperationSecondMomentBsplineClenshawCurtis.grid, sgpp::base::OperationHierarchisationFundamentalSpline.grid, sgpp::base::OperationHierarchisationModFundamentalSpline.grid, sgpp::base::OperationArbitraryBoundaryHierarchisation.grid, sgpp::base::HierarchisationFundamentalSpline.grid, sgpp::base::DehierarchisationFundamentalSpline.grid, sgpp::base::HierarchisationModFundamentalSpline.grid, sgpp::base::DehierarchisationModFundamentalSpline.grid, sgpp::base::OperationQuadratureMC.grid, python.controller.CheckpointController.CheckpointController.grid, sgpp::combigrid::LTwoScalarProductHashMapNakBsplineBoundaryCombigrid.grid, sgpp::base::ImpurityRefinementIndicator.grid, and sgpp::base::ForwardSelectorRefinementIndicator.grid.

Referenced by python.controller.CheckpointController.CheckpointController.generateFoldValidationJob(), and python.controller.CheckpointController.CheckpointController.saveAll().

◆ saveLearnedKnowledge()

def python.controller.CheckpointController.CheckpointController.saveLearnedKnowledge (   self,
  iteration 
)

◆ saveLearner()

def python.controller.CheckpointController.CheckpointController.saveLearner (   self,
  iteration 
)

◆ setGrid()

def python.controller.CheckpointController.CheckpointController.setGrid (   self,
  grid 
)

Setter for the current Grid object.

Parameters
gridThe Grid object.

References sgpp::base::OperationMultipleEval.grid, sgpp::base::OperationFirstMomentBspline.grid, sgpp::base::OperationFirstMomentPolyBoundary.grid, sgpp::base::OperationFirstMomentPolyClenshawCurtis.grid, sgpp::base::OperationFirstMomentBsplineBoundary.grid, sgpp::base::OperationSecondMomentPolyClenshawCurtis.grid, sgpp::base::OperationSecondMomentPolyBoundary.grid, sgpp::base::OperationSecondMomentPoly.grid, sgpp::base::OperationFirstMomentModBspline.grid, sgpp::base::OperationSecondMomentBspline.grid, sgpp::base::OperationSecondMomentBsplineBoundary.grid, sgpp::base::OperationFirstMomentModPoly.grid, sgpp::base::OperationSecondMomentModBspline.grid, sgpp::base::OperationSecondMomentModPoly.grid, sgpp::base::OperationFirstMomentModPolyClenshawCurtis.grid, sgpp::base::OperationSecondMomentModPolyClenshawCurtis.grid, sgpp::base::OperationFirstMomentPoly.grid, sgpp::base::OperationSecondMomentPolyClenshawCurtisBoundary.grid, sgpp::base::OperationFirstMomentPolyClenshawCurtisBoundary.grid, sgpp::base::OperationSecondMomentModBsplineClenshawCurtis.grid, sgpp::base::OperationFirstMomentBsplineClenshawCurtis.grid, sgpp::base::OperationFirstMomentModBsplineClenshawCurtis.grid, sgpp::base::OperationSecondMomentBsplineClenshawCurtis.grid, sgpp::base::OperationHierarchisationFundamentalSpline.grid, sgpp::base::OperationHierarchisationModFundamentalSpline.grid, sgpp::base::OperationArbitraryBoundaryHierarchisation.grid, sgpp::base::DehierarchisationFundamentalSpline.grid, sgpp::base::HierarchisationFundamentalSpline.grid, sgpp::base::HierarchisationModFundamentalSpline.grid, sgpp::base::DehierarchisationModFundamentalSpline.grid, sgpp::base::OperationQuadratureMC.grid, python.controller.CheckpointController.CheckpointController.grid, sgpp::combigrid::LTwoScalarProductHashMapNakBsplineBoundaryCombigrid.grid, sgpp::base::ImpurityRefinementIndicator.grid, and sgpp::base::ForwardSelectorRefinementIndicator.grid.

Referenced by python.controller.CheckpointController.CheckpointController.loadAll(), and python.controller.CheckpointController.CheckpointController.setLearner().

◆ setLearnedKnowledge()

def python.controller.CheckpointController.CheckpointController.setLearnedKnowledge (   self,
  knowledge 
)

◆ setLearner()

def python.controller.CheckpointController.CheckpointController.setLearner (   self,
  learner 
)

Member Data Documentation

◆ fold

◆ grid

python.controller.CheckpointController.CheckpointController.grid = None
static

Reference to the Grid object.

Referenced by python.uq.dists.SGDEdist.SGDEdist.__str__(), python.learner.Learner.Learner.applyData(), python.tools.Matrix.ApplyMatrix(), python.uq.dists.SGDEdist.SGDEdist.cdf(), python.uq.learner.Interpolant.Interpolant.doLearningIteration(), python.learner.Learner.Learner.doLearningIteration(), python.learner.Classifier.Classifier.evalError(), python.uq.learner.Interpolant.Interpolant.evalError(), python.tools.Matrix.generateb(), python.controller.CheckpointController.CheckpointController.generateFoldValidationJob(), python.uq.learner.SimulationLearner.SimulationLearner.getCollocationNodes(), python.uq.learner.SimulationLearner.SimulationLearner.getGrid(), python.uq.learner.SimulationLearner.SimulationLearner.getLearner(), python.uq.learner.Regressor.Regressor.learnData(), python.learner.Learner.Learner.learnData(), python.uq.learner.Regressor.Regressor.learnDataWithFolding(), python.learner.Learner.Learner.learnDataWithFolding(), python.uq.learner.Regressor.Regressor.learnDataWithTest(), python.learner.Learner.Learner.learnDataWithTest(), python.controller.CheckpointController.CheckpointController.loadAll(), python.uq.operations.forcePositivity.operationMakePositive.OperationMakePositive.makePositive(), python.uq.operations.forcePositivity.operationMakePositiveFast.OperationMakePositiveFast.makePositive(), python.uq.dists.SGDEdist.SGDEdist.mean(), python.uq.dists.SGDEdist.SGDEdist.pdf(), python.uq.dists.SGDEdist.SGDEdist.ppf(), python.learner.Classifier.Classifier.refineGrid(), python.learner.Regressor.Regressor.refineGrid(), python.uq.learner.Regressor.Regressor.refineGrid(), python.uq.learner.SimulationLearner.SimulationLearner.refineGrid(), python.controller.CheckpointController.CheckpointController.saveGrid(), python.controller.CheckpointController.CheckpointController.setGrid(), python.uq.learner.Learner.Learner.setGrid(), python.learner.Learner.Learner.setGrid(), python.uq.operations.forcePositivity.localFullGridSearch.LocalFullGrid.split(), python.uq.dists.SGDEdist.SGDEdist.toJson(), python.learner.Classifier.Classifier.updateResults(), python.learner.Regressor.Regressor.updateResults(), python.uq.learner.Regressor.Regressor.updateResults(), and python.uq.dists.SGDEdist.SGDEdist.var().

◆ interval

python.controller.CheckpointController.CheckpointController.interval = None
static

◆ knowledge

◆ lastIteration

python.controller.CheckpointController.CheckpointController.lastIteration = None
static

Number of the last iteration.

◆ learner

◆ path

python.controller.CheckpointController.CheckpointController.path = None
static

◆ title


The documentation for this class was generated from the following file: