04-04 AdaBoost算法代码(鸢尾花分类)

2022-11-17 18:11:22

AdaBoost算法代码(鸢尾花分类)
一、导入模块
二、导入数据
三、构造决策边界
四、训练模型

更新、更全的《机器学习》的更新网站，更有python、go、数据结构与算法、爬虫、人工智能教学等着你：https://www.cnblogs.com/nickchen121/p/11686958.html

AdaBoost算法代码(鸢尾花分类)

一、导入模块

import numpy as np
import matplotlib.pyplot as plt
from matplotlib.colors import ListedColormap
from matplotlib.font_manager import FontProperties
from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import AdaBoostClassifier
%matplotlib inline
font = FontProperties(fname='/Library/Fonts/Heiti.ttc')

二、导入数据

X = iris_data.data[:, [2, 3]]
y = iris_data.target
label_list = ['山鸢尾', '杂色鸢尾', '维吉尼亚鸢尾']

三、构造决策边界

def plot_decision_regions(X, y, classifier=None):
    marker_list = ['o', 'x', 's']
    color_list = ['r', 'b', 'g']
    cmap = ListedColormap(color_list[:len(np.unique(y))])
x1_min, x1_max = X[:, <span class="hljs-number">0</span>].<span class="hljs-built_in">min</span>()<span class="hljs-number">-1</span>, X[:, <span class="hljs-number">0</span>].<span class="hljs-built_in">max</span>()+<span class="hljs-number">1</span>
x2_min, x2_max = X[:, <span class="hljs-number">1</span>].<span class="hljs-built_in">min</span>()<span class="hljs-number">-1</span>, X[:, <span class="hljs-number">1</span>].<span class="hljs-built_in">max</span>()+<span class="hljs-number">1</span>
t1 = np.linspace(x1_min, x1_max, <span class="hljs-number">666</span>)
t2 = np.linspace(x2_min, x2_max, <span class="hljs-number">666</span>)

x1, x2 = np.meshgrid(t1, t2)
y_hat = classifier.predict(np.array([x1.ravel(), x2.ravel()]).T)
y_hat = y_hat.reshape(x1.shape)
plt.contourf(x1, x2, y_hat, alpha=<span class="hljs-number">0.2</span>, cmap=cmap)
plt.xlim(x1_min, x1_max)
plt.ylim(x2_min, x2_max)

<span class="hljs-keyword">for</span> ind, clas <span class="hljs-keyword">in</span> <span class="hljs-built_in">enumerate</span>(np.unique(y)):
    plt.scatter(X[y == clas, <span class="hljs-number">0</span>], X[y == clas, <span class="hljs-number">1</span>], alpha=<span class="hljs-number">0.8</span>, s=<span class="hljs-number">50</span>,
                c=color_list[ind], marker=marker_list[ind], label=label_list[clas])

四、训练模型

4.1 训练模型(n_e=10, l_r=0.8)

adbt = AdaBoostClassifier(DecisionTreeClassifier(max_depth=2, min_samples_split=20, min_samples_leaf=5),
                          algorithm="SAMME", n_estimators=10, learning_rate=0.8)
adbt.fit(X, y)

AdaBoostClassifier(algorithm='SAMME',
          base_estimator=DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=2,
            max_features=None, max_leaf_nodes=None,
            min_impurity_decrease=0.0, min_impurity_split=None,
            min_samples_leaf=5, min_samples_split=20,
            min_weight_fraction_leaf=0.0, presort=False, random_state=None,
            splitter='best'),
          learning_rate=0.8, n_estimators=10, random_state=None)

4.2 可视化

plot_decision_regions(X, y, classifier=adbt)
plt.xlabel('花瓣长度（cm）', fontproperties=font)
plt.ylabel('花瓣宽度（cm）', fontproperties=font)
plt.title('AdaBoost算法代码(鸢尾花分类, n_e=10, l_r=0.8)',
          fontproperties=font, fontsize=20)
plt.legend(prop=font)
plt.show()

print("Score:{}".format(adbt.score(X, y)))

Score:0.9866666666666667

4.3 训练模型(n_estimators=300, learning_rate=0.8)

adbt = AdaBoostClassifier(DecisionTreeClassifier(max_depth=2, min_samples_split=20, min_samples_leaf=5),
                          algorithm="SAMME", n_estimators=300, learning_rate=0.8)
adbt.fit(X, y)
print("Score:{}".format(adbt.score(X, y)))

Score:0.9933333333333333

由于样本太少，可能效果不明显，但是对比上一个模型可以发现，相同步长的情况下，如果弱学习个数越多，拟合效果越好，但如果过多则可能过拟合。

4.4 训练模型(n_estimators=300, learning_rate=0.5)

adbt = AdaBoostClassifier(DecisionTreeClassifier(max_depth=2, min_samples_split=20, min_samples_leaf=5),
                          algorithm="SAMME", n_estimators=300, learning_rate=0.001)
adbt.fit(X, y)
print("Score:{}".format(adbt.score(X, y)))

Score:0.9533333333333334

相同迭代次数的情况下，对比上一个模型可以发现，如果步长越大，则模型效果越好。

4.5 训练模型(n_estimators=600, learning_rate=0.7)

adbt = AdaBoostClassifier(DecisionTreeClassifier(max_depth=2, min_samples_split=20, min_samples_leaf=5),
                          algorithm="SAMME", n_estimators=600, learning_rate=0.8)
adbt.fit(X, y)
print("Score:{}".format(adbt.score(X, y)))

Score:0.9933333333333333

对比第二个模型，可以发现即使增加迭代次数，算法准确率也没有提高，所以n_estimators=300的时候其实算法就已经收敛了。

码农公寓