LogitBoost Classifier Demo

Several ways to use LogitBoosting Algorithm classifier

Contents

Load dataset

% Name of matlab dataset
% Available names : [cancer, wine, iris, crab, glass, simpleclass, thyroid]
datasetName = 'iris';

% This is just an example to load matlab datasets
% you can load datasets from different sources with different ways
% as long as the you provide x the training instance which is a matrix
% of size(Number of Instance,Number of Features) and y which is the
% label matrix having size(Number of Instance, 1)
load(strcat(datasetName, '_dataset'));
eval(sprintf('x = %sInputs;', datasetName));
eval(sprintf('y = %sTargets;', datasetName));
x = x';
y = y';
numClasses = size(y, 2);
[~,y] = max(y,[],2);

numFeatures = size(x, 2);
numInstances = size(x, 1);

% display dataset info
disp(['Dataset Name ' datasetName]);
disp(['Number of Classes ' num2str(numClasses)]);
disp(['Number of Instances ' num2str(numInstances)]);
disp(['Number of Features ' num2str(numFeatures)]);
Dataset Name iris
Number of Classes 3
Number of Instances 150
Number of Features 4

Basic Usage

fprintf('===========\n');
fprintf('Basic Usage\n');
fprintf('===========\n');

% create LogitBoost classifier Using svm regressor
lcl = LogitBooster(SVMRegressor(), numClasses);

% train logitboost classifier
fprintf('\tTraining Classifier for 5 iterations');
[lcl, learnErr] = learn(lcl, x, y, 5);
fprintf('\tLearning Error %f\n', learnErr);

fprintf('\t-----\n');
fprintf('\tTesting Classifier\n');
outs = computeOutputs(lcl, x);
% other way to calculate error either on training dataset or other dataset
err = sum(outs ~= y) / numInstances;
fprintf('\tLearning Error %f\n', err);

% comparing outputs of first 5 instances in predicted and target outputs
fprintf('\t[predicted outputs : correct outputs]\n');
fprintf('\t\t%d\t\t%d\t\n',outs(1:5, :), y(1:5 , :));
===========
Basic Usage
===========
	Training Classifier for 5 iterations============
Logit Boost
============
Iteration 0
Iteration 1
Iteration 2
Iteration 3
Iteration 4
			Logit Boost Training is Done with err = 0.053333
	Learning Error 0.053333
	-----
	Testing Classifier
	Learning Error 0.053333
	[predicted outputs : correct outputs]
		1		1	
		1		1	
		1		1	
		1		1	
		1		1	

Display Classifier

fprintf('==================\n');
fprintf('Display Classifier\n');
fprintf('==================\n');

fprintf('\tDisplay Classifier Before Learning\n\t===>\n');
lcl = LogitBooster(SVMRegressor(), numClasses);
display(lcl);
fprintf('\t<===\n');

fprintf('\tDisplay Classifier After Learning\n===>\n');
lcl = learn(lcl, x, y, 2);
display(lcl);
fprintf('\t<===\n');
==================
Display Classifier
==================
	Display Classifier Before Learning
	===>
Logit AdaBooster Classifier
  classifier is not trained
	<===
	Display Classifier After Learning
===>
============
Logit Boost
============
Iteration 0
Iteration 1
			Logit Boost Training is Done with err = 0.266667
Logit AdaBooster Classifier
  classifier is trained, the learnt paramters are:\n
	No target detection rate was specified
	Threshold = 0.000000
	Number of stages = 1
	<===

Add More Boosting Stages

fprintf('========================\n');
fprintf('Add More Boosting Stages\n');
fprintf('========================\n');

lcl = LogitBooster(SVMRegressor(), numClasses);

fprintf('\tTraining Classifier for 3 iterations\n');
lcl = learn(lcl, x, y, 3);
fprintf('\t------------\n');
fprintf('\tTraining Classifier for 3 more iterations\n');
[~, learnErr] = learn(lcl, x, y, 6);
fprintf('\tLearning Error %f\n', learnErr);
========================
Add More Boosting Stages
========================
	Training Classifier for 3 iterations
============
Logit Boost
============
Iteration 0
Iteration 1
Iteration 2
			Logit Boost Training is Done with err = 0.133333
	------------
	Training Classifier for 3 more iterations
============
Logit Boost
============
classifier was trained before
Iteration 3
Iteration 4
Iteration 5
			Logit Boost Training is Done with err = 0.046667
	Learning Error 0.046667

Add Boost Stages till reaching a required Err Bound

fprintf('===================================================\n');
fprintf('Add Boost Stages till reaching a required Err Bound\n');
fprintf('===================================================\n');

lcl = LogitBooster(SVMRegressor(), numClasses);
[lcl, learnErr] = learn(lcl, x, y, Inf, NaN, 0.02);
fprintf('\tLearning Error %f\n', learnErr);
===================================================
Add Boost Stages till reaching a required Err Bound
===================================================
============
Logit Boost
============
Iteration 0
			Currently, boosted classifier's error = 0.333333
Iteration 1
			Currently, boosted classifier's error = 0.266667
Iteration 2
			Currently, boosted classifier's error = 0.133333
Iteration 3
			Currently, boosted classifier's error = 0.060000
Iteration 4
			Currently, boosted classifier's error = 0.053333
Iteration 5
			Currently, boosted classifier's error = 0.046667
Iteration 6
			Currently, boosted classifier's error = 0.026667
Iteration 7
			Currently, boosted classifier's error = 0.033333
Iteration 8
			Currently, boosted classifier's error = 0.026667
Iteration 9
			Currently, boosted classifier's error = 0.020000
			Logit Boost Training is Done with err = 0.020000
	Learning Error 0.020000