LogitBoost Classifier Demo
Several ways to use LogitBoosting Algorithm classifier
Contents
Load dataset
% Name of matlab dataset % Available names : [cancer, wine, iris, crab, glass, simpleclass, thyroid] datasetName = 'iris'; % This is just an example to load matlab datasets % you can load datasets from different sources with different ways % as long as the you provide x the training instance which is a matrix % of size(Number of Instance,Number of Features) and y which is the % label matrix having size(Number of Instance, 1) load(strcat(datasetName, '_dataset')); eval(sprintf('x = %sInputs;', datasetName)); eval(sprintf('y = %sTargets;', datasetName)); x = x'; y = y'; numClasses = size(y, 2); [~,y] = max(y,[],2); numFeatures = size(x, 2); numInstances = size(x, 1); % display dataset info disp(['Dataset Name ' datasetName]); disp(['Number of Classes ' num2str(numClasses)]); disp(['Number of Instances ' num2str(numInstances)]); disp(['Number of Features ' num2str(numFeatures)]);
Dataset Name iris Number of Classes 3 Number of Instances 150 Number of Features 4
Basic Usage
fprintf('===========\n'); fprintf('Basic Usage\n'); fprintf('===========\n'); % create LogitBoost classifier Using svm regressor lcl = LogitBooster(SVMRegressor(), numClasses); % train logitboost classifier fprintf('\tTraining Classifier for 5 iterations'); [lcl, learnErr] = learn(lcl, x, y, 5); fprintf('\tLearning Error %f\n', learnErr); fprintf('\t-----\n'); fprintf('\tTesting Classifier\n'); outs = computeOutputs(lcl, x); % other way to calculate error either on training dataset or other dataset err = sum(outs ~= y) / numInstances; fprintf('\tLearning Error %f\n', err); % comparing outputs of first 5 instances in predicted and target outputs fprintf('\t[predicted outputs : correct outputs]\n'); fprintf('\t\t%d\t\t%d\t\n',outs(1:5, :), y(1:5 , :));
=========== Basic Usage =========== Training Classifier for 5 iterations============ Logit Boost ============ Iteration 0 Iteration 1 Iteration 2 Iteration 3 Iteration 4 Logit Boost Training is Done with err = 0.053333 Learning Error 0.053333 ----- Testing Classifier Learning Error 0.053333 [predicted outputs : correct outputs] 1 1 1 1 1 1 1 1 1 1
Display Classifier
fprintf('==================\n'); fprintf('Display Classifier\n'); fprintf('==================\n'); fprintf('\tDisplay Classifier Before Learning\n\t===>\n'); lcl = LogitBooster(SVMRegressor(), numClasses); display(lcl); fprintf('\t<===\n'); fprintf('\tDisplay Classifier After Learning\n===>\n'); lcl = learn(lcl, x, y, 2); display(lcl); fprintf('\t<===\n');
================== Display Classifier ================== Display Classifier Before Learning ===> Logit AdaBooster Classifier classifier is not trained <=== Display Classifier After Learning ===> ============ Logit Boost ============ Iteration 0 Iteration 1 Logit Boost Training is Done with err = 0.266667 Logit AdaBooster Classifier classifier is trained, the learnt paramters are:\n No target detection rate was specified Threshold = 0.000000 Number of stages = 1 <===
Add More Boosting Stages
fprintf('========================\n'); fprintf('Add More Boosting Stages\n'); fprintf('========================\n'); lcl = LogitBooster(SVMRegressor(), numClasses); fprintf('\tTraining Classifier for 3 iterations\n'); lcl = learn(lcl, x, y, 3); fprintf('\t------------\n'); fprintf('\tTraining Classifier for 3 more iterations\n'); [~, learnErr] = learn(lcl, x, y, 6); fprintf('\tLearning Error %f\n', learnErr);
======================== Add More Boosting Stages ======================== Training Classifier for 3 iterations ============ Logit Boost ============ Iteration 0 Iteration 1 Iteration 2 Logit Boost Training is Done with err = 0.133333 ------------ Training Classifier for 3 more iterations ============ Logit Boost ============ classifier was trained before Iteration 3 Iteration 4 Iteration 5 Logit Boost Training is Done with err = 0.046667 Learning Error 0.046667
Add Boost Stages till reaching a required Err Bound
fprintf('===================================================\n'); fprintf('Add Boost Stages till reaching a required Err Bound\n'); fprintf('===================================================\n'); lcl = LogitBooster(SVMRegressor(), numClasses); [lcl, learnErr] = learn(lcl, x, y, Inf, NaN, 0.02); fprintf('\tLearning Error %f\n', learnErr);
=================================================== Add Boost Stages till reaching a required Err Bound =================================================== ============ Logit Boost ============ Iteration 0 Currently, boosted classifier's error = 0.333333 Iteration 1 Currently, boosted classifier's error = 0.266667 Iteration 2 Currently, boosted classifier's error = 0.133333 Iteration 3 Currently, boosted classifier's error = 0.060000 Iteration 4 Currently, boosted classifier's error = 0.053333 Iteration 5 Currently, boosted classifier's error = 0.046667 Iteration 6 Currently, boosted classifier's error = 0.026667 Iteration 7 Currently, boosted classifier's error = 0.033333 Iteration 8 Currently, boosted classifier's error = 0.026667 Iteration 9 Currently, boosted classifier's error = 0.020000 Logit Boost Training is Done with err = 0.020000 Learning Error 0.020000