Class to host all methods to perform Bayesian Optimization. More...

#include <BayesianOptimization.hpp>

Public Member Functions
double	acquisitionOuter (const base::DataVector &inp)
	Wrapper function for use in optimizer. More...

	BayesianOptimization (const std::vector< BOConfig > &initialConfigs)
	Constructor setting up Gaussian Process. More...

void	decomposeCholesky (base::DataMatrix &km, base::DataMatrix &gnew)
	Perform a Cholesky Decomposition. More...

base::DataVector	fitScales ()
	fitting the Gaussian Process to the data in the form of scaling the hyperparameters in relation to each other by likelihood maximization More...

double	kernel (double distance)
	kernel function More...

double	likelihood (const base::DataVector &inp)
	Gaussian Process likelihood to perform likelihood maximization over. More...

BOConfig	main (BOConfig &prototype)
	main routine to find new sample point More...

double	mean (base::DataVector &knew)

void	setScales (base::DataVector nscales, double factor)

void	solveCholeskySystem (base::DataMatrix &gmatrix, base::DataVector &x)
	Solve a system of linear equations using previously decomposed matrix. More...

void	updateGP (BOConfig &newConfig, bool normalize)
	Gaussian Process update step. More...

double	var (base::DataVector &knew, double kself)

Static Public Member Functions
static double	acquisitionEI (double dMean, double dVar, double bestsofar)
	Implementation of mathematical formulation of the expected improvement acquisition function. More...

Protected Attributes
std::vector< BOConfig >	allConfigs
	existing sample points in the Gaussian Process More...

double	bestsofar
	best score encountered so far (used for expected improvement calculation) More...

bool	decomFailed = false
	debugging variable for numerical instabilities More...

base::DataMatrix	gleft
	Cholesky Decomposition of the Gram matrix. More...

base::DataMatrix	kernelmatrix
	Gram matrix containing all kernel values between all existing samples. More...

double	maxofmax
	debugging variable for numerical instabilities More...

base::DataVector	rawScores
	score values belonging to all existing sample points More...

base::DataVector	scales
	scales of the hyperparameter space used to compute the kernel (set by fitScales()) More...

bool	screwedvar
	debugging variable for numerical instabilities More...

base::DataVector	transformedOutput
	solution of Kx = s where K is the Gram matrix and s the scores of the samples More...

Detailed Description

Class to host all methods to perform Bayesian Optimization.

Constructor & Destructor Documentation

◆ BayesianOptimization()

sgpp::datadriven::BayesianOptimization::BayesianOptimization ( const std::vector< BOConfig > & initialConfigs )

explicit

Constructor setting up Gaussian Process.

Parameters

initialConfigs non-empty vector of initial points to build the Gaussian Process

References allConfigs, bestsofar, python.statsfileInfo::i, sgpp::base::DataVector::max(), sgpp::base::DataVector::min(), sgpp::base::DataVector::normalize(), rawScores, scales, setScales(), sgpp::base::DataVector::sub(), and sgpp::base::DataVector::sum().

Member Function Documentation

◆ acquisitionEI()

double sgpp::datadriven::BayesianOptimization::acquisitionEI	(	double	dMean,
		double	dVar,
		double	bestsofar
	)

static

Implementation of mathematical formulation of the expected improvement acquisition function.

Referenced by acquisitionOuter().

◆ acquisitionOuter()

double sgpp::datadriven::BayesianOptimization::acquisitionOuter ( const base::DataVector & inp )

Wrapper function for use in optimizer.

Parameters

inp	point in continuous optimization space

Returns: score to optimize on

References acquisitionEI(), allConfigs, bestsofar, python.statsfileInfo::i, kernel(), m, mean(), scales, and var().

Referenced by main().

◆ decomposeCholesky()

void sgpp::datadriven::BayesianOptimization::decomposeCholesky	(	base::DataMatrix &	km,
		base::DataMatrix &	gnew
	)

Perform a Cholesky Decomposition.

Parameters

km	input matrix
gnew	output (triangular) matrix

References decomFailed, sgpp::base::DataMatrix::get(), sgpp::base::DataMatrix::getNrows(), python.statsfileInfo::i, python.utils.statsfile2gnuplot::j, and sgpp::base::DataMatrix::set().

Referenced by likelihood(), setScales(), and updateGP().

◆ fitScales()

base::DataVector sgpp::datadriven::BayesianOptimization::fitScales ( )

fitting the Gaussian Process to the data in the form of scaling the hyperparameters in relation to each other by likelihood maximization

References likelihood(), and scales.

Referenced by sgpp::datadriven::BoHyperparameterOptimizer::run().

◆ kernel()

double sgpp::datadriven::BayesianOptimization::kernel ( double distance )

kernel function

Parameters

distance as computed according to some spacial representation

Returns: kernel value representing variance

Referenced by acquisitionOuter(), likelihood(), setScales(), and updateGP().

◆ likelihood()

double sgpp::datadriven::BayesianOptimization::likelihood ( const base::DataVector & inp )

Gaussian Process likelihood to perform likelihood maximization over.

Parameters

inp	vector containing scales of the hyperparameter space

Returns: value representative of the likelihood

References allConfigs, decomposeCholesky(), sgpp::base::DataVector::dotProduct(), sgpp::base::DataMatrix::get(), python.statsfileInfo::i, kernel(), sgpp::combigrid::pow(), rawScores, solveCholeskySystem(), and analyse_erg::tmp.

Referenced by fitScales().

◆ main()

BOConfig sgpp::datadriven::BayesianOptimization::main ( BOConfig & prototype )

main routine to find new sample point

Parameters

prototype baseline BOConfig

Returns: new sample point

References acquisitionOuter(), allConfigs, sgpp::datadriven::BOConfig::calcDiscDistance(), sgpp::datadriven::BOConfig::getContSize(), sgpp::optimization::optimizer::UnconstrainedOptimizer::getOptimalPoint(), sgpp::optimization::optimizer::UnconstrainedOptimizer::getOptimalValue(), sgpp::datadriven::BOConfig::nextDisc(), sgpp::optimization::optimizer::MultiStart::optimize(), scales, and sgpp::datadriven::BOConfig::setCont().

Referenced by sgpp::datadriven::BoHyperparameterOptimizer::run().

◆ mean()

double sgpp::datadriven::BayesianOptimization::mean ( base::DataVector & knew )

References sgpp::base::DataVector::dotProduct(), and transformedOutput.

Referenced by acquisitionOuter(), python.uq.analysis.mc.MCAnalysis.MCAnalysis::computeMoments(), and python.uq.analysis.asgc.ASGCAnalysis.ASGCAnalysis::computeMoments().

◆ setScales()

void sgpp::datadriven::BayesianOptimization::setScales	(	base::DataVector	nscales,
		double	factor
	)

References sgpp::base::DataVector::add(), allConfigs, decomposeCholesky(), gleft, python.statsfileInfo::i, kernel(), kernelmatrix, sgpp::base::DataVector::mult(), sgpp::combigrid::pow(), rawScores, scales, sgpp::base::DataMatrix::set(), solveCholeskySystem(), analyse_erg::tmp, and transformedOutput.

Referenced by BayesianOptimization(), and sgpp::datadriven::BoHyperparameterOptimizer::run().

◆ solveCholeskySystem()

void sgpp::datadriven::BayesianOptimization::solveCholeskySystem	(	base::DataMatrix &	gmatrix,
		base::DataVector &	x
	)

Solve a system of linear equations using previously decomposed matrix.

Parameters

gmatrix	decomposed matrix
x	target vector

References sgpp::base::DataMatrix::get(), and python.statsfileInfo::i.

Referenced by likelihood(), setScales(), updateGP(), and var().

◆ updateGP()

void sgpp::datadriven::BayesianOptimization::updateGP	(	BOConfig &	newConfig,
		bool	normalize
	)

Gaussian Process update step.

Incorporates most recent sample into Gaussian Process.

References allConfigs, sgpp::base::DataMatrix::appendCol(), sgpp::base::DataMatrix::appendRow(), bestsofar, decomFailed, decomposeCholesky(), sgpp::base::DataMatrix::getNcols(), sgpp::datadriven::BOConfig::getScore(), gleft, python.statsfileInfo::i, kernel(), kernelmatrix, sgpp::base::DataVector::max(), maxofmax, sgpp::base::DataVector::min(), sgpp::base::DataMatrix::mult(), sgpp::base::DataVector::normalize(), sgpp::combigrid::pow(), rawScores, scales, screwedvar, sgpp::base::DataMatrix::set(), solveCholeskySystem(), sgpp::base::DataVector::sub(), sgpp::base::DataVector::sum(), analyse_erg::tmp, and transformedOutput.

Referenced by sgpp::datadriven::BoHyperparameterOptimizer::run().

◆ var()

double sgpp::datadriven::BayesianOptimization::var	(	base::DataVector &	knew,
		double	kself
	)

References python.uq.tools::check(), sgpp::base::DataVector::dotProduct(), gleft, kernelmatrix, maxofmax, sgpp::base::DataMatrix::mult(), screwedvar, solveCholeskySystem(), and analyse_erg::tmp.

Referenced by acquisitionOuter(), python.uq.analysis.mc.MCAnalysis.MCAnalysis::computeMoments(), and python.uq.analysis.asgc.ASGCAnalysis.ASGCAnalysis::computeMoments().

Member Data Documentation

◆ allConfigs

std::vector<BOConfig> sgpp::datadriven::BayesianOptimization::allConfigs

protected

existing sample points in the Gaussian Process

Referenced by acquisitionOuter(), BayesianOptimization(), likelihood(), main(), setScales(), and updateGP().

◆ bestsofar

double sgpp::datadriven::BayesianOptimization::bestsofar

protected

best score encountered so far (used for expected improvement calculation)

Referenced by acquisitionOuter(), BayesianOptimization(), and updateGP().

◆ decomFailed

bool sgpp::datadriven::BayesianOptimization::decomFailed = false

protected

debugging variable for numerical instabilities

Referenced by decomposeCholesky(), and updateGP().

◆ gleft

base::DataMatrix sgpp::datadriven::BayesianOptimization::gleft

protected

Cholesky Decomposition of the Gram matrix.

Referenced by setScales(), updateGP(), and var().

◆ kernelmatrix

base::DataMatrix sgpp::datadriven::BayesianOptimization::kernelmatrix

protected

Gram matrix containing all kernel values between all existing samples.

Referenced by setScales(), updateGP(), and var().

◆ maxofmax

double sgpp::datadriven::BayesianOptimization::maxofmax

protected

debugging variable for numerical instabilities

Referenced by updateGP(), and var().

◆ rawScores

base::DataVector sgpp::datadriven::BayesianOptimization::rawScores

protected

score values belonging to all existing sample points

Referenced by BayesianOptimization(), likelihood(), setScales(), and updateGP().

◆ scales

base::DataVector sgpp::datadriven::BayesianOptimization::scales

protected

scales of the hyperparameter space used to compute the kernel (set by fitScales())

Referenced by acquisitionOuter(), BayesianOptimization(), fitScales(), main(), setScales(), and updateGP().

◆ screwedvar

bool sgpp::datadriven::BayesianOptimization::screwedvar

protected

debugging variable for numerical instabilities

Referenced by updateGP(), and var().

◆ transformedOutput

base::DataVector sgpp::datadriven::BayesianOptimization::transformedOutput

protected

solution of Kx = s where K is the Gram matrix and s the scores of the samples

Referenced by mean(), setScales(), and updateGP().

The documentation for this class was generated from the following files:

datadriven/src/sgpp/datadriven/datamining/modules/hpo/bo/BayesianOptimization.hpp
datadriven/src/sgpp/datadriven/datamining/modules/hpo/bo/BayesianOptimization.cpp

Public Member Functions

Static Public Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ BayesianOptimization()

Member Function Documentation

◆ acquisitionEI()

◆ acquisitionOuter()

◆ decomposeCholesky()

◆ fitScales()

◆ kernel()

◆ likelihood()

◆ main()

◆ mean()

◆ setScales()

◆ solveCholeskySystem()

◆ updateGP()

◆ var()

Member Data Documentation

◆ allConfigs

◆ bestsofar

◆ decomFailed

◆ gleft

◆ kernelmatrix

◆ maxofmax

◆ rawScores

◆ scales

◆ screwedvar

◆ transformedOutput