www.gusucode.com > stats 源码程序 matlab案例代码 > stats/RefitNCAModelForRegressionWithModifiedSettingsExample.m

    %% Refit NCA Model for Regression with Modified Settings
%%
% Load the sample data.
load(fullfile(matlabroot,'examples','stats','robotarm.mat'))
%%
% The robotarm (pumadyn32nm) dataset is created using a robot arm simulator
% with 7168 training and 1024 test observations with 32 features [1], [2].
% This is a preprocessed version of the original data set. Data are
% preprocessed by subtracting off a linear regression fit followed by
% normalization of all features to unit variance.
%%
% Compute the generalization error without feature selection.
nca = fsrnca(Xtrain,ytrain,'FitMethod','none','Standardize',1);
L = loss(nca,Xtest,ytest)

%%
% Now, refit the model and compute the prediction loss with feature
% selection, with $\lambda$ = 0 (no regularization term) and compare to the
% previous loss value, to determine feature selection seems necessary for
% this problem. For the settings that you do not change, |refit| uses the
% settings of the initial model |nca|. For example, it uses the feature
% weights found in |nca| as the initial feature weights.
nca2 = refit(nca,'FitMethod','exact','Lambda',0);
L2 = loss(nca2,Xtest,ytest)
%%
% The decrease in the loss suggests that feature selection is necessary.
%%
% Plot the feature weights.
figure()
plot(nca2.FeatureWeights,'ro')
%%
% Tuning the regularization parameter usually improves the results. Suppose
% that, after tuning $\lambda$ using cross-validation as in <docid:stats_ug.bve4g0q-1>, the best $\lambda$
% value found is 0.0035. Refit the nca model using this $\lambda$
% value and stochastic gradient descent as the solver. Compute the
% prediction loss.
nca3 = refit(nca2,'FitMethod','exact','Lambda',0.0035,...
          'Solver','sgd');
L3 = loss(nca3,Xtest,ytest)

%%
% Plot the feature weights.
figure()
plot(nca3.FeatureWeights,'ro')
%%
% After tuning the regularization parameter, the loss decreased even more
% and the software identified four of the features as relevant.
%%
% *References* 
%
% [1] Rasmussen, C. E., R. M. Neal, G. E. Hinton, D. van Campand, M. Revow,
% Z. Ghahramani, R. Kustra, R. Tibshirani. The DELVE Manual, 1996,
% http://mlg.eng.cam.ac.uk/pub/pdf/RasNeaHinetal96.pdf
%
% [2] http://www.cs.toronto.edu/~delve/data/datasets.html