SG++
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
Classification Example

This example shows how classification specific refinement strategies are used.

To do classification, for each class a PDF is approximated with LearnerSGDE and the class with the highest probability gets assigned for new data points to be classified. The ripley data sets is used, although the small number of training data poitns in combination with only a basic setup does not yield good results for any refinement strategy. This example is merely a tech-example.

Helper to create learner

sgpp::datadriven::LearnerSGDE createSGDELearner(size_t dim, size_t level,
double lambda);

Helper to evaluate the classifiers

std::vector<std::string> doClassification(std::vector<sgpp::base::Grid*> grids,
std::vector<sgpp::base::DataVector*> alphas,
int main() {

Get the training/test data

std::string basePath = "../../datasets/ripley/ripleyGarcke";
sgpp::datadriven::ARFFTools::readARFF(basePath + ".train.arff");
sgpp::datadriven::ARFFTools::readARFF(basePath + ".test.arff");
sgpp::base::DataMatrix dataTrain = datasetTr.getData();
sgpp::base::DataVector targetTrain = datasetTr.getTargets();
sgpp::base::DataMatrix dataTest = datasetTs.getData();
sgpp::base::DataVector targetTest = datasetTs.getTargets();
std::cout << "Read training data: " << dataTrain.getNrows() << std::endl;
std::cout << "Read test data : " << dataTest.getNrows() << std::endl;

Only uint class labels starting 0 and incrementing by 1 per class are possible right no (to match grid-indices in vectors). For use in DataVector class labels are cast to double. Preprocess to have class label 0, 1, ... -1 -> 0 and 1 -> 1

for (size_t i = 0; i < targetTrain.getSize(); i++) {
if (targetTrain.get(i) < 0.0) {
targetTrain.set(i, 0.0);
} else {
targetTrain.set(i, 1.0);
}
}
for (size_t i = 0; i < targetTest.getSize(); i++) {
if (targetTest.get(i) < 0.0) {
targetTest.set(i, 0.0);
} else {
targetTest.set(i, 1.0);
}
}
std::cout << "Preprocessing the data" << std::endl;

Split Training data according to class

sgpp::base::DataMatrix dataCl1(0.0, dataTrain.getNcols());
sgpp::base::DataMatrix dataCl2(0.0, dataTrain.getNcols());
sgpp::base::DataVector row(dataTrain.getNcols());
for (size_t i = 0; i < dataTrain.getNrows(); i++) {
dataTrain.getRow(i, row);
if (targetTrain.get(i) < 1) {
dataCl1.appendRow(row);
} else {
dataCl2.appendRow(row);
}
}
std::cout << "Data points of class -1.0 (= 0): " << dataCl1.getNrows() << std::endl;
std::cout << "Data points of class +1.0 (= 1): " << dataCl2.getNrows() << std::endl;

Approximate a probability density function for the class data using LearnerSGDE, one for each class. Initialize the learners with the data

double lambda = 1e-5;
sgpp::datadriven::LearnerSGDE learner1 = createSGDELearner(2, 3, lambda);
sgpp::datadriven::LearnerSGDE learner2 = createSGDELearner(2, 3, lambda);
learner1.initialize(dataCl1);
learner2.initialize(dataCl2);

Bundle grids and surplus vector pointer needed for refinement and evaluation

std::vector<sgpp::base::Grid*> grids;
std::vector<sgpp::base::DataVector*> alphas;
grids.push_back(learner1.getGrid().get());
grids.push_back(learner2.getGrid().get());
alphas.push_back(learner1.getSurpluses().get());
alphas.push_back(learner2.getSurpluses().get());

Create refinement functors

size_t numRefinements = 3;
bool levelPenalize = false; // Multiplies penalzing term for fine levels
bool preCompute = true; // Precomputes and caches evals for zrcr & grid
// Surplus refinement
numRefinements,
levelPenalize);
// Grid point-based refinement
numRefinements,
levelPenalize,
preCompute);

Data-based refinement. Needs a problem dependent coeffA. The values were determined by testing (aim at ~10 % of the training data is to be marked relevant. Cross-validation or similar can/should be employed to determine this value.

std::vector<double> coeffA;
coeffA.push_back(1.2);
coeffA.push_back(1.2);
&dataTrain,
&targetTrain,
numRefinements,
levelPenalize,
coeffA);

Choose the refinement functor to be used

// fun = &funSrpl;
// fun = &funGrid;
fun = &funZrcr;
// fun = &funData;

Repeat alternating refinement and training for n steps and do evaluation after each step Uses the refinement strategy defined in fun An initial evaluation with the regular grid is done at "step 0"

size_t numSteps = 5;
std::vector<std::string> eval = doClassification(grids, alphas, dataTest, targetTest);
std::cout << "Evaluation:" << std::endl << std::endl;
std::cout << " Step | c=1 c=2 | total" << std::endl;
std::cout << "------------------------------" << std::endl;
std::cout << " 0 | " << eval.at(0) << " | " << eval.at(1) << std::endl;
for (size_t i = 1; i < numSteps + 1; i++) {
if (preCompute) {
// precompute the evals. Needs to be done once per step, before
// any refinement is done
}

Helper function It configures and creates a SGDE learner with meaningful parameters

sgpp::datadriven::LearnerSGDE createSGDELearner(size_t dim, size_t level,
double lambda) {
gridConfig.dim_ = dim;
gridConfig.level_ = static_cast<int>(level);
// configure adaptive refinement
adaptConfig.numRefinements_ = 0;
adaptConfig.noPoints_ = 10;
// configure solver
solverConfig.maxIterations_ = 1000;
solverConfig.eps_ = 1e-10;
solverConfig.threshold_ = 1e-10;
// configure regularization
regularizationConfig.regType_ =
// configure learner
crossvalidationConfig;
crossvalidationConfig.enable_ = false;
crossvalidationConfig.kfold_ = 3;
crossvalidationConfig.lambda_ = 3.16228e-06;
crossvalidationConfig.lambdaStart_ = lambda;
crossvalidationConfig.lambdaEnd_ = lambda;
crossvalidationConfig.lambdaSteps_ = 3;
crossvalidationConfig.logScale_ = true;
crossvalidationConfig.shuffle_ = true;
crossvalidationConfig.seed_ = 1234567;
crossvalidationConfig.silent_ = true;
sgpp::datadriven::LearnerSGDE learner(gridConfig,
adaptConfig,
solverConfig,
regularizationConfig,
crossvalidationConfig);
return learner;
}

Helper function it does the classification, gets the predictions and generates some error-output

std::vector<std::string> doClassification(std::vector<sgpp::base::Grid*> grids,
std::vector<sgpp::base::DataVector*> alphas,
sgpp::base::DataVector& testLabel) {
double best_eval = 0.0;
double eval = 0.0;
sgpp::base::DataVector indices(testData.getNrows());
sgpp::base::DataVector evals(testData.getNrows());
std::vector<std::unique_ptr<sgpp::base::OperationEval>> evalOps;
for (size_t i = 0; i < grids.size(); i++) {
std::unique_ptr<sgpp::base::OperationEval>
evalOps.push_back(std::move(e));
}