本文参考机器学习周志华
基础内容如下
由上述原理,给出利用不使用核函数和软间隔的SVM处理用LAD降至2维的iris数据集的MATLAB源程序
main.m
tic
clear;clc
%导入数据
load matlab.mat
global D
D.X1 = D_new(1:48,:);
D.Y1 = ones(size(D.X1,1),1);
D.X2 = D_new(51:end-2,:);
D.Y2 = -ones(size(D.X2,1),1);
% 使用Matlab自带SVM对LDA处理后的iris数据二分类
% svmModel = fitcsvm([D.X1;D.X2],[D.Y1;D.Y2],'kernelFunction','rbf');
% Y_pred = predict(svmModel,[D_new(49:50,:);D_new(end-1:end,:)]);
%% 使用自己求解SVM对LDA处理后的iris数据二分类
D.Label = [D.Y1;D.Y2];
D.Data = [D.X1;D.X2];
figure(1)
hold on
scatter(D.X1(:,1),D.X1(:,2),'r');
scatter(D.X2(:,1),D.X2(:,2),'g');
ITER = 100;
objf = +inf;
for Times = 1:ITER
x0 = rand(size(D.X1,1)+size(D.X2,1),1);
obj = 0;
for i = 1:size(D.Data,1)
for j = 1:size(D.Data,1)
obj = obj+1/2*(x0(i)*x0(j)*D.Label(i)*D.Label(j)*D.Data(i)*D.Data(j)');
end
obj = obj-x0(i);
end
if obj<objf
objf = obj;
bestx0 = x0;
end
end
[alpha,val] = fmincon(@obj_fun,bestx0,[],[],[D.Label'],0,zeros(size(D.X1,1)+size(D.X2,1),1),[],[],[]);
% h = [];
% for i = 1:size(D.Data,1)
% for j = 1:size(D.Data,1)
% h(i,j) = D.Data(i,:)*D.Data(j,:)'*D.Label(i)*D.Label(j);
% end
% end
% f = -1*ones(size(D.Data,1),1);
% [alpha,val] = quadprog(h,f,[],[],D.Label',0,zeros(size(D.Data,1),1),[]);
alpha(alpha<1e-3) = 0;
w = zeros(1,size(D.Data,2));
for i = 1:size(D.Label,1)
w = w+alpha(i).*D.Label(i).*D.Data(i,:);
end
[row,~] = find(alpha~=0);
b = D.Label(row(1))-w*D.Data(row(1),:)';
plot([0,-b/w(1)],[-b/w(2),0])
toc
Obj_fun
function f = obj_fun(x)
global D
Data = [D.X1;D.X2];
Label = [D.Y1;D.Y2];
sum = 0;
for i = 1:size(Data,1)
for j = 1:size(Data,1)
sum = sum+1/2*(x(i)*x(j)*Label(i)*Label(j)*Data(i)*Data(j)');
end
sum = sum-x(i);
end
f = sum;
其中注释段代码为使用二次规划求解,直接运行是先由100次蒙特卡洛模拟生成初始解,再将初始解代入非线性规划求得的,二者在运行时间上差了5倍,但为非二次规划问题提供思路。
值得注意的是,如果将两类的标签置为0和1则不能得到正确的结果,只有将标签置为1和-1时才能得到正确的SVM二分类结果
积累:是否可以不定义全局变量global D,而使用其他手段定义目标函数Obj_fun