www.gusucode.com > stats 源码程序 matlab案例代码 > stats/CompareAccuraciesOfTwoDifferentClassificationModelsExample.m
%% Compare Accuracies of Two Different Classification Models % Train two classification models using different algorithms. Conduct a % statistical test comparing the misclassification rates of the two models % on a held-out set. %% % Load the |ionosphere| data set. % Copyright 2015 The MathWorks, Inc. load ionosphere; %% % Create a partition that evenly splits the data into training and testing % sets. rng(1); % For reproducibility CVP = cvpartition(Y,'holdout',0.5); idxTrain = training(CVP); % Training-set indices idxTest = test(CVP); % Test-set indices %% % |CVP| is a cross-validation partition object that specifies the training % and test sets. %% % Train an SVM model and an ensemble of 100 bagged classification trees. % For the SVM model, specify to use the radial basis function kernel and a % heuristic procedure to determine the kernel scale. MdlSVM = fitcsvm(X(idxTrain,:),Y(idxTrain),'Standardize',true,... 'KernelFunction','RBF','KernelScale','auto'); MdlBag = fitensemble(X(idxTrain,:),Y(idxTrain),'Bag',100,'Tree',... 'Type','classification'); %% % |MdlSVM| is a trained |ClassificationSVM| model. |MdlBag| is a trained % |ClassificationBaggedEnsemble| model. %% % Label the test-set observations using the trained models. YhatSVM = predict(MdlSVM,X(idxTest,:)); YhatBag = predict(MdlBag,X(idxTest,:)); %% % |YhatSVM| and |YhatBag| are vectors continuing the predicted % class labels of the respective models. %% % Test whether the two models have equal predictive accuracies. h = testcholdout(YhatSVM,YhatBag,Y(idxTest)) %% % |h = 0| indicates to not reject the null hypothesis that the two models % have equal predictive accuracies.