SG++
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
sgpp::optimization::optimizer Namespace Reference

Namespace containing methods for gradient-based and gradient-free optimization. More...

Classes

class  AdaptiveGradientDescent
 Gradient descent with adaptive step size. More...
 
class  AdaptiveNewton
 Newton method with adaptive step size. More...
 
class  AugmentedLagrangian
 Augmented Lagrangian method for constrained optimization. More...
 
class  BFGS
 BFGS method for unconstrained optimization. More...
 
class  CMAES
 Gradient-free CMA-ES method. More...
 
class  ConstrainedOptimizer
 Abstract class for solving constrained optimization problems. More...
 
class  DifferentialEvolution
 Gradient-free Differential Evolution method. More...
 
class  GradientDescent
 Gradient-based method of steepest descent. More...
 
class  LeastSquaresOptimizer
 Abstract class for solving non-linear least squares problems. More...
 
class  LevenbergMarquardt
 Levenberg-Marquardt algorithm for least squares optimization. More...
 
class  LogBarrier
 Log Barrier method for constrained optimization. More...
 
class  MultiStart
 Meta optimization algorithm calling local algorithm multiple times. More...
 
class  NelderMead
 Gradient-free Nelder-Mead method. More...
 
class  Newton
 Gradient-based nonlinear conjugate gradient method. More...
 
class  NLCG
 Gradient-based nonlinear conjugate gradient method. More...
 
class  Rprop
 Rprop method for unconstrained optimization. More...
 
class  SquaredPenalty
 Squared Penalty method for constrained optimization. More...
 
class  UnconstrainedOptimizer
 Abstract class for optimizing objective functions. More...
 

Functions

bool lineSearchArmijo (ScalarFunction &f, double beta, double gamma, double tol, double eps, const base::DataVector &x, double fx, base::DataVector &gradFx, const base::DataVector &s, base::DataVector &y, size_t &evalCounter)
 Line search (1D optimization on a line) with Armijo's rule used in gradient-based optimization. More...
 

Detailed Description

Namespace containing methods for gradient-based and gradient-free optimization.

See Module sgpp::optimization for more details.

Function Documentation

bool sgpp::optimization::optimizer::lineSearchArmijo ( ScalarFunction &  f,
double  beta,
double  gamma,
double  tol,
double  eps,
const base::DataVector &  x,
double  fx,
base::DataVector &  gradFx,
const base::DataVector &  s,
base::DataVector &  y,
size_t &  evalCounter 
)
inline

Line search (1D optimization on a line) with Armijo's rule used in gradient-based optimization.

Armijo's rule calculates \(\sigma = \beta^k\) for \(k = 0, 1, \dotsc\) for a fixed \(\beta \in (0, 1)\) and checks if \(\vec{y} = \vec{x} + \sigma\vec{s}\) lies in \([0, 1]^d\) and whether the objective function value improvement meets the condition \(f(\vec{x}) - f(\vec{y}) \ge \gamma\sigma (-\nabla f(\vec{x}) \cdot \vec{s})\) for \(\gamma \in (0, 1)\) fixed.

The return value states whether the relative improvement (depending on two tolerances) is big enough to continue the optimization algorithm.

Parameters
fobjective function
beta\(\beta \in (0, 1)\)
gamma\(\gamma \in (0, 1)\)
toltolerance 1 (positive)
epstolerance 2 (positive)
xpoint to start the line search in
fxobjective function value in x
gradFxobjective function gradient in x
ssearch direction (should be normalized)
[out]ynew point, must have the same size as x before calling this function
[in,out]evalCounterthis variable will be increased by the number of evaluations of f while searching
Returns
whether the new point reaches an acceptable improvement

References sgpp::base::DataVector::dotProduct(), sgpp::optimization::ScalarFunction::eval(), and sgpp::base::DataVector::getSize().

Referenced by sgpp::optimization::optimizer::GradientDescent::optimize(), sgpp::optimization::optimizer::NLCG::optimize(), and sgpp::optimization::optimizer::Newton::optimize().