|
$k$-近邻算法（kNN）的工作原理：存在一个训练样本集，样本集中的每个数据都存在标签，即我们知道样本集中每一数据与所属分类的对于关系。输入没有标签的新数据后，将新数据的每一个特征与样本集中数据对应的特征进行比较，然后算法提取样本集中特征最相似数据（最近邻）的分类标签。一般来说，我们只选择样本数据集中前 $k$ 个最相似的数据，这就是$k$-近邻算法中$k$的出处，通常$k$是不大于20的整数。最后，选择$k$个最相似的数据中出现次数最多的分类，作为新数据的分类。 |
1. Putting the kNN classification algorithm into action
For every point in our dataset:
calculate the distance between inX and the current point
sort the distances in increasing order
take k items with lowest distances to inX
find the majority class among these items
return the majority class as our prediction for the class of inX
一个简单的例子：kNN.py
# coding=utf-8
from numpy import *
import operator def createDataSet():
group = array([[1.0,1.1], [1.0,1.0], [0,0], [0,0.1]])
labels = ['A', 'A', 'B', 'B']
return group, labels def classify0(inX, dataSet, labels, k):
dataSetSize = dataSet.shape[0]
diffMat = tile(inX, (dataSetSize,1)) - dataSet
sqDiffMat = diffMat**2
sqDistances = sqDiffMat.sum(axis=1) # 按行求和
distances = sqDistances**0.5
sortedDistIndicies = distances.argsort() # 将索引按照距离从小到大顺序排列
classCount={} # 以dict形式存储
for i in range(k):
voteIlabel = labels[sortedDistIndicies[i]] # 第i最靠近的样本的标签
# dict是按照key-value的形式构成的，classCount.get(voteIlabel,0)是取出classCount中key是voteIlabel的value，如果key不存在，则定义返回0
classCount[voteIlabel] = classCount.get(voteIlabel,0) +1
# classCount.items()以(key, value) tuple 形式返回list
sortedClassCount = sorted(classCount.items(),
key=operator.itemgetter(1), reverse=True)
return sortedClassCount[0][0] group, labels = createDataSet()
label = classify0([0.2,0.2], group, labels, 3)
print label
createDataSet() 函数为我们准备了四个简单的训练数据。
classify0() 函数是一个简单的$k$-近邻算法实现，函数有四个参数：待预测样本的输入特征inX，训练样本集特征集合dataSet，训练样本集标签向量labels，最近邻数目$k$。
2. Example: improving matches from a dating site with kNN
Example: using kNN on results from a dating site
1. Collect: Text file provided.
2. Prepare: Parse a text file in Python.
3. Analyze: Use Matplotlib to make 2D plots of our data.
4. Train: Doesn’t apply to the kNN algorithm.
5. Test: Write a function to use some portion of the data Hellen gave us as test examples. The test examples are classified against the non-test examples. If the predicted class doesn’t match the real class, we’ll count that as an error.
6. Use: Build a simple command-line program Hellen can use to predict whether she’ll like someone based on a few inputs.
2.1 Prepare: parsing data from a text file
所有训练数据存放在文本文件datingTestSet.txt中，样本容量大小为1000。样本主要包含以下三种特征：
■ Number of frequent flyer miles earned per year
■ Percentage of time spent playing video games
■ Liters of ice cream consumed per week
设计分类器之前，我们首先要把原始数据读入到python中。在kNN.py中创建名为file2matrix的函数，以此来处理原始数据。该函数的输入为文件名字符串，输出为训练样本矩阵和类标签向量。
40920 8.326976 0.953952 largeDoses
14488 7.153469 1.673904 smallDoses
26052 1.441871 0.805124 didntLike
75136 13.147394 0.428964 didntLike
38344 1.669788 0.134296 didntLike
72993 10.141740 1.032955 didntLike
35948 6.830792 1.213192 largeDoses
42666 13.276369 0.543880 largeDoses
67497 8.631577 0.749278 didntLike
35483 12.273169 1.508053 largeDoses
50242 3.723498 0.831917 didntLike
63275 8.385879 1.669485 didntLike
5569 4.875435 0.728658 smallDoses
51052 4.680098 0.625224 didntLike
77372 15.299570 0.331351 didntLike
43673 1.889461 0.191283 didntLike
61364 7.516754 1.269164 didntLike
69673 14.239195 0.261333 didntLike
15669 0.000000 1.250185 smallDoses
28488 10.528555 1.304844 largeDoses
6487 3.540265 0.822483 smallDoses
37708 2.991551 0.833920 didntLike
22620 5.297865 0.638306 smallDoses
28782 6.593803 0.187108 largeDoses
19739 2.816760 1.686209 smallDoses
36788 12.458258 0.649617 largeDoses
5741 0.000000 1.656418 smallDoses
28567 9.968648 0.731232 largeDoses
6808 1.364838 0.640103 smallDoses
41611 0.230453 1.151996 didntLike
36661 11.865402 0.882810 largeDoses
43605 0.120460 1.352013 didntLike
15360 8.545204 1.340429 largeDoses
63796 5.856649 0.160006 didntLike
10743 9.665618 0.778626 smallDoses
70808 9.778763 1.084103 didntLike
72011 4.932976 0.632026 didntLike
5914 2.216246 0.587095 smallDoses
14851 14.305636 0.632317 largeDoses
33553 12.591889 0.686581 largeDoses
44952 3.424649 1.004504 didntLike
17934 0.000000 0.147573 smallDoses
27738 8.533823 0.205324 largeDoses
29290 9.829528 0.238620 largeDoses
42330 11.492186 0.263499 largeDoses
36429 3.570968 0.832254 didntLike
39623 1.771228 0.207612 didntLike
32404 3.513921 0.991854 didntLike
27268 4.398172 0.975024 didntLike
5477 4.276823 1.174874 smallDoses
14254 5.946014 1.614244 smallDoses
68613 13.798970 0.724375 didntLike
41539 10.393591 1.663724 largeDoses
7917 3.007577 0.297302 smallDoses
21331 1.031938 0.486174 smallDoses
8338 4.751212 0.064693 smallDoses
5176 3.692269 1.655113 smallDoses
18983 10.448091 0.267652 largeDoses
68837 10.585786 0.329557 didntLike
13438 1.604501 0.069064 smallDoses
48849 3.679497 0.961466 didntLike
12285 3.795146 0.696694 smallDoses
7826 2.531885 1.659173 smallDoses
5565 9.733340 0.977746 smallDoses
10346 6.093067 1.413798 smallDoses
1823 7.712960 1.054927 smallDoses
9744 11.470364 0.760461 largeDoses
16857 2.886529 0.934416 smallDoses
39336 10.054373 1.138351 largeDoses
65230 9.972470 0.881876 didntLike
2463 2.335785 1.366145 smallDoses
27353 11.375155 1.528626 largeDoses
16191 0.000000 0.605619 smallDoses
12258 4.126787 0.357501 smallDoses
42377 6.319522 1.058602 didntLike
25607 8.680527 0.086955 largeDoses
77450 14.856391 1.129823 didntLike
58732 2.454285 0.222380 didntLike
46426 7.292202 0.548607 largeDoses
32688 8.745137 0.857348 largeDoses
64890 8.579001 0.683048 didntLike
8554 2.507302 0.869177 smallDoses
28861 11.415476 1.505466 largeDoses
42050 4.838540 1.680892 didntLike
32193 10.339507 0.583646 largeDoses
64895 6.573742 1.151433 didntLike
2355 6.539397 0.462065 smallDoses
0 2.209159 0.723567 smallDoses
70406 11.196378 0.836326 didntLike
57399 4.229595 0.128253 didntLike
41732 9.505944 0.005273 largeDoses
11429 8.652725 1.348934 largeDoses
75270 17.101108 0.490712 didntLike
5459 7.871839 0.717662 smallDoses
73520 8.262131 1.361646 didntLike
40279 9.015635 1.658555 largeDoses
21540 9.215351 0.806762 largeDoses
17694 6.375007 0.033678 smallDoses
22329 2.262014 1.022169 didntLike
46570 5.677110 0.709469 didntLike
42403 11.293017 0.207976 largeDoses
33654 6.590043 1.353117 didntLike
9171 4.711960 0.194167 smallDoses
28122 8.768099 1.108041 largeDoses
34095 11.502519 0.545097 largeDoses
1774 4.682812 0.578112 smallDoses
40131 12.446578 0.300754 largeDoses
13994 12.908384 1.657722 largeDoses
77064 12.601108 0.974527 didntLike
11210 3.929456 0.025466 smallDoses
6122 9.751503 1.182050 largeDoses
15341 3.043767 0.888168 smallDoses
44373 4.391522 0.807100 didntLike
28454 11.695276 0.679015 largeDoses
63771 7.879742 0.154263 didntLike
9217 5.613163 0.933632 smallDoses
69076 9.140172 0.851300 didntLike
24489 4.258644 0.206892 didntLike
16871 6.799831 1.221171 smallDoses
39776 8.752758 0.484418 largeDoses
5901 1.123033 1.180352 smallDoses
40987 10.833248 1.585426 largeDoses
7479 3.051618 0.026781 smallDoses
38768 5.308409 0.030683 largeDoses
4933 1.841792 0.028099 smallDoses
32311 2.261978 1.605603 didntLike
26501 11.573696 1.061347 largeDoses
37433 8.038764 1.083910 largeDoses
23503 10.734007 0.103715 largeDoses
68607 9.661909 0.350772 didntLike
27742 9.005850 0.548737 largeDoses
11303 0.000000 0.539131 smallDoses
0 5.757140 1.062373 smallDoses
32729 9.164656 1.624565 largeDoses
24619 1.318340 1.436243 didntLike
42414 14.075597 0.695934 largeDoses
20210 10.107550 1.308398 largeDoses
33225 7.960293 1.219760 largeDoses
54483 6.317292 0.018209 didntLike
18475 12.664194 0.595653 largeDoses
33926 2.906644 0.581657 didntLike
43865 2.388241 0.913938 didntLike
26547 6.024471 0.486215 largeDoses
44404 7.226764 1.255329 largeDoses
16674 4.183997 1.275290 smallDoses
8123 11.850211 1.096981 largeDoses
42747 11.661797 1.167935 largeDoses
56054 3.574967 0.494666 didntLike
10933 0.000000 0.107475 smallDoses
18121 7.937657 0.904799 largeDoses
11272 3.365027 1.014085 smallDoses
16297 0.000000 0.367491 smallDoses
28168 13.860672 1.293270 largeDoses
40963 10.306714 1.211594 largeDoses
31685 7.228002 0.670670 largeDoses
55164 4.508740 1.036192 didntLike
17595 0.366328 0.163652 smallDoses
1862 3.299444 0.575152 smallDoses
57087 0.573287 0.607915 didntLike
63082 9.183738 0.012280 didntLike
51213 7.842646 1.060636 largeDoses
6487 4.750964 0.558240 smallDoses
4805 11.438702 1.556334 largeDoses
30302 8.243063 1.122768 largeDoses
68680 7.949017 0.271865 didntLike
17591 7.875477 0.227085 smallDoses
74391 9.569087 0.364856 didntLike
37217 7.750103 0.869094 largeDoses
42814 0.000000 1.515293 didntLike
14738 3.396030 0.633977 smallDoses
19896 11.916091 0.025294 largeDoses
14673 0.460758 0.689586 smallDoses
32011 13.087566 0.476002 largeDoses
58736 4.589016 1.672600 didntLike
54744 8.397217 1.534103 didntLike
29482 5.562772 1.689388 didntLike
27698 10.905159 0.619091 largeDoses
11443 1.311441 1.169887 smallDoses
56117 10.647170 0.980141 largeDoses
39514 0.000000 0.481918 didntLike
26627 8.503025 0.830861 largeDoses
16525 0.436880 1.395314 smallDoses
24368 6.127867 1.102179 didntLike
22160 12.112492 0.359680 largeDoses
6030 1.264968 1.141582 smallDoses
6468 6.067568 1.327047 smallDoses
22945 8.010964 1.681648 largeDoses
18520 3.791084 0.304072 smallDoses
34914 11.773195 1.262621 largeDoses
6121 8.339588 1.443357 smallDoses
38063 2.563092 1.464013 didntLike
23410 5.954216 0.953782 didntLike
35073 9.288374 0.767318 largeDoses
52914 3.976796 1.043109 didntLike
16801 8.585227 1.455708 largeDoses
9533 1.271946 0.796506 smallDoses
16721 0.000000 0.242778 smallDoses
5832 0.000000 0.089749 smallDoses
44591 11.521298 0.300860 largeDoses
10143 1.139447 0.415373 smallDoses
21609 5.699090 1.391892 smallDoses
23817 2.449378 1.322560 didntLike
15640 0.000000 1.228380 smallDoses
8847 3.168365 0.053993 smallDoses
50939 10.428610 1.126257 largeDoses
28521 2.943070 1.446816 didntLike
32901 10.441348 0.975283 largeDoses
42850 12.478764 1.628726 largeDoses
13499 5.856902 0.363883 smallDoses
40345 2.476420 0.096075 didntLike
43547 1.826637 0.811457 didntLike
70758 4.324451 0.328235 didntLike
19780 1.376085 1.178359 smallDoses
44484 5.342462 0.394527 didntLike
54462 11.835521 0.693301 largeDoses
20085 12.423687 1.424264 largeDoses
42291 12.161273 0.071131 largeDoses
47550 8.148360 1.649194 largeDoses
11938 1.531067 1.549756 smallDoses
40699 3.200912 0.309679 didntLike
70908 8.862691 0.530506 didntLike
73989 6.370551 0.369350 didntLike
11872 2.468841 0.145060 smallDoses
48463 11.054212 0.141508 largeDoses
15987 2.037080 0.715243 smallDoses
70036 13.364030 0.549972 didntLike
32967 10.249135 0.192735 largeDoses
63249 10.464252 1.669767 didntLike
42795 9.424574 0.013725 largeDoses
14459 4.458902 0.268444 smallDoses
19973 0.000000 0.575976 smallDoses
5494 9.686082 1.029808 largeDoses
67902 13.649402 1.052618 didntLike
25621 13.181148 0.273014 largeDoses
27545 3.877472 0.401600 didntLike
58656 1.413952 0.451380 didntLike
7327 4.248986 1.430249 smallDoses
64555 8.779183 0.845947 didntLike
8998 4.156252 0.097109 smallDoses
11752 5.580018 0.158401 smallDoses
76319 15.040440 1.366898 didntLike
27665 12.793870 1.307323 largeDoses
67417 3.254877 0.669546 didntLike
21808 10.725607 0.588588 largeDoses
15326 8.256473 0.765891 smallDoses
20057 8.033892 1.618562 largeDoses
79341 10.702532 0.204792 didntLike
15636 5.062996 1.132555 smallDoses
35602 10.772286 0.668721 largeDoses
28544 1.892354 0.837028 didntLike
57663 1.019966 0.372320 didntLike
78727 15.546043 0.729742 didntLike
68255 11.638205 0.409125 didntLike
14964 3.427886 0.975616 smallDoses
21835 11.246174 1.475586 largeDoses
7487 0.000000 0.645045 smallDoses
8700 0.000000 1.424017 smallDoses
26226 8.242553 0.279069 largeDoses
65899 8.700060 0.101807 didntLike
6543 0.812344 0.260334 smallDoses
46556 2.448235 1.176829 didntLike
71038 13.230078 0.616147 didntLike
47657 0.236133 0.340840 didntLike
19600 11.155826 0.335131 largeDoses
37422 11.029636 0.505769 largeDoses
1363 2.901181 1.646633 smallDoses
26535 3.924594 1.143120 didntLike
47707 2.524806 1.292848 didntLike
38055 3.527474 1.449158 didntLike
6286 3.384281 0.889268 smallDoses
10747 0.000000 1.107592 smallDoses
44883 11.898890 0.406441 largeDoses
56823 3.529892 1.375844 didntLike
68086 11.442677 0.696919 didntLike
70242 10.308145 0.422722 didntLike
11409 8.540529 0.727373 smallDoses
67671 7.156949 1.691682 didntLike
61238 0.720675 0.847574 didntLike
17774 0.229405 1.038603 smallDoses
53376 3.399331 0.077501 didntLike
30930 6.157239 0.580133 didntLike
28987 1.239698 0.719989 didntLike
13655 6.036854 0.016548 smallDoses
7227 5.258665 0.933722 smallDoses
40409 12.393001 1.571281 largeDoses
13605 9.627613 0.935842 smallDoses
26400 11.130453 0.597610 largeDoses
13491 8.842595 0.349768 largeDoses
30232 10.690010 1.456595 largeDoses
43253 5.714718 1.674780 largeDoses
55536 3.052505 1.335804 didntLike
8807 0.000000 0.059025 smallDoses
25783 9.945307 1.287952 largeDoses
22812 2.719723 1.142148 didntLike
77826 11.154055 1.608486 didntLike
38172 2.687918 0.660836 didntLike
31676 10.037847 0.962245 largeDoses
74038 12.404762 1.112080 didntLike
44738 10.237305 0.633422 largeDoses
17410 4.745392 0.662520 smallDoses
5688 4.639461 1.569431 smallDoses
36642 3.149310 0.639669 didntLike
29956 13.406875 1.639194 largeDoses
60350 6.068668 0.881241 didntLike
23758 9.477022 0.899002 largeDoses
25780 3.897620 0.560201 smallDoses
11342 5.463615 1.203677 smallDoses
36109 3.369267 1.575043 didntLike
14292 5.234562 0.825954 smallDoses
11160 0.000000 0.722170 smallDoses
23762 12.979069 0.504068 largeDoses
39567 5.376564 0.557476 didntLike
25647 13.527910 1.586732 largeDoses
14814 2.196889 0.784587 smallDoses
73590 10.691748 0.007509 didntLike
35187 1.659242 0.447066 didntLike
49459 8.369667 0.656697 largeDoses
31657 13.157197 0.143248 largeDoses
6259 8.199667 0.908508 smallDoses
33101 4.441669 0.439381 largeDoses
27107 9.846492 0.644523 largeDoses
17824 0.019540 0.977949 smallDoses
43536 8.253774 0.748700 largeDoses
67705 6.038620 1.509646 didntLike
35283 6.091587 1.694641 largeDoses
71308 8.986820 1.225165 didntLike
31054 11.508473 1.624296 largeDoses
52387 8.807734 0.713922 largeDoses
40328 0.000000 0.816676 didntLike
34844 8.889202 1.665414 largeDoses
11607 3.178117 0.542752 smallDoses
64306 7.013795 0.139909 didntLike
32721 9.605014 0.065254 largeDoses
33170 1.230540 1.331674 didntLike
37192 10.412811 0.890803 largeDoses
13089 0.000000 0.567161 smallDoses
66491 9.699991 0.122011 didntLike
15941 0.000000 0.061191 smallDoses
4272 4.455293 0.272135 smallDoses
48812 3.020977 1.502803 didntLike
28818 8.099278 0.216317 largeDoses
35394 1.157764 1.603217 didntLike
71791 10.105396 0.121067 didntLike
40668 11.230148 0.408603 largeDoses
39580 9.070058 0.011379 largeDoses
11786 0.566460 0.478837 smallDoses
19251 0.000000 0.487300 smallDoses
56594 8.956369 1.193484 largeDoses
54495 1.523057 0.620528 didntLike
11844 2.749006 0.169855 smallDoses
45465 9.235393 0.188350 largeDoses
31033 10.555573 0.403927 largeDoses
16633 6.956372 1.519308 smallDoses
13887 0.636281 1.273984 smallDoses
52603 3.574737 0.075163 didntLike
72000 9.032486 1.461809 didntLike
68497 5.958993 0.023012 didntLike
35135 2.435300 1.211744 didntLike
26397 10.539731 1.638248 largeDoses
7313 7.646702 0.056513 smallDoses
91273 20.919349 0.644571 didntLike
24743 1.424726 0.838447 didntLike
31690 6.748663 0.890223 largeDoses
15432 2.289167 0.114881 smallDoses
58394 5.548377 0.402238 didntLike
33962 6.057227 0.432666 didntLike
31442 10.828595 0.559955 largeDoses
31044 11.318160 0.271094 largeDoses
29938 13.265311 0.633903 largeDoses
9875 0.000000 1.496715 smallDoses
51542 6.517133 0.402519 largeDoses
11878 4.934374 1.520028 smallDoses
69241 10.151738 0.896433 didntLike
37776 2.425781 1.559467 didntLike
68997 9.778962 1.195498 didntLike
67416 12.219950 0.657677 didntLike
59225 7.394151 0.954434 didntLike
29138 8.518535 0.742546 largeDoses
5962 2.798700 0.662632 smallDoses
10847 0.637930 0.617373 smallDoses
70527 10.750490 0.097415 didntLike
9610 0.625382 0.140969 smallDoses
64734 10.027968 0.282787 didntLike
25941 9.817347 0.364197 largeDoses
2763 0.646828 1.266069 smallDoses
55601 3.347111 0.914294 didntLike
31128 11.816892 0.193798 largeDoses
5181 0.000000 1.480198 smallDoses
69982 10.945666 0.993219 didntLike
52440 10.244706 0.280539 largeDoses
57350 2.579801 1.149172 didntLike
57869 2.630410 0.098869 didntLike
56557 11.746200 1.695517 largeDoses
42342 8.104232 1.326277 largeDoses
15560 12.409743 0.790295 largeDoses
34826 12.167844 1.328086 largeDoses
8569 3.198408 0.299287 smallDoses
77623 16.055513 0.541052 didntLike
78184 7.138659 0.158481 didntLike
7036 4.831041 0.761419 smallDoses
69616 10.082890 1.373611 didntLike
21546 10.066867 0.788470 largeDoses
36715 8.129538 0.329913 largeDoses
20522 3.012463 1.138108 smallDoses
42349 3.720391 0.845974 didntLike
9037 0.773493 1.148256 smallDoses
26728 10.962941 1.037324 largeDoses
587 0.177621 0.162614 smallDoses
48915 3.085853 0.967899 didntLike
9824 8.426781 0.202558 smallDoses
4135 1.825927 1.128347 smallDoses
9666 2.185155 1.010173 smallDoses
59333 7.184595 1.261338 didntLike
36198 0.000000 0.116525 didntLike
34909 8.901752 1.033527 largeDoses
47516 2.451497 1.358795 didntLike
55807 3.213631 0.432044 didntLike
14036 3.974739 0.723929 smallDoses
42856 9.601306 0.619232 largeDoses
64007 8.363897 0.445341 didntLike
59428 6.381484 1.365019 didntLike
13730 0.000000 1.403914 smallDoses
41740 9.609836 1.438105 largeDoses
63546 9.904741 0.985862 didntLike
30417 7.185807 1.489102 largeDoses
69636 5.466703 1.216571 didntLike
64660 0.000000 0.915898 didntLike
14883 4.575443 0.535671 smallDoses
7965 3.277076 1.010868 smallDoses
68620 10.246623 1.239634 didntLike
8738 2.341735 1.060235 smallDoses
7544 3.201046 0.498843 smallDoses
6377 6.066013 0.120927 smallDoses
36842 8.829379 0.895657 largeDoses
81046 15.833048 1.568245 didntLike
67736 13.516711 1.220153 didntLike
32492 0.664284 1.116755 didntLike
39299 6.325139 0.605109 largeDoses
77289 8.677499 0.344373 didntLike
33835 8.188005 0.964896 largeDoses
71890 9.414263 0.384030 didntLike
32054 9.196547 1.138253 largeDoses
38579 10.202968 0.452363 largeDoses
55984 2.119439 1.481661 didntLike
72694 13.635078 0.858314 didntLike
42299 0.083443 0.701669 didntLike
26635 9.149096 1.051446 largeDoses
8579 1.933803 1.374388 smallDoses
37302 14.115544 0.676198 largeDoses
22878 8.933736 0.943352 largeDoses
4364 2.661254 0.946117 smallDoses
4985 0.988432 1.305027 smallDoses
37068 2.063741 1.125946 didntLike
41137 2.220590 0.690754 didntLike
67759 6.424849 0.806641 didntLike
11831 1.156153 1.613674 smallDoses
34502 3.032720 0.601847 didntLike
4088 3.076828 0.952089 smallDoses
15199 0.000000 0.318105 smallDoses
17309 7.750480 0.554015 largeDoses
42816 10.958135 1.482500 largeDoses
43751 10.222018 0.488678 largeDoses
58335 2.367988 0.435741 didntLike
75039 7.686054 1.381455 didntLike
42878 11.464879 1.481589 largeDoses
42770 11.075735 0.089726 largeDoses
8848 3.543989 0.345853 smallDoses
31340 8.123889 1.282880 largeDoses
41413 4.331769 0.754467 largeDoses
12731 0.120865 1.211961 smallDoses
22447 6.116109 0.701523 largeDoses
33564 7.474534 0.505790 largeDoses
48907 8.819454 0.649292 largeDoses
8762 6.802144 0.615284 smallDoses
46696 12.666325 0.931960 largeDoses
36851 8.636180 0.399333 largeDoses
67639 11.730991 1.289833 didntLike
171 8.132449 0.039062 smallDoses
26674 10.296589 1.496144 largeDoses
8739 7.583906 1.005764 smallDoses
66668 9.777806 0.496377 didntLike
68732 8.833546 0.513876 didntLike
69995 4.907899 1.518036 didntLike
82008 8.362736 1.285939 didntLike
25054 9.084726 1.606312 largeDoses
33085 14.164141 0.560970 largeDoses
41379 9.080683 0.989920 largeDoses
39417 6.522767 0.038548 largeDoses
12556 3.690342 0.462281 smallDoses
39432 3.563706 0.242019 didntLike
38010 1.065870 1.141569 didntLike
69306 6.683796 1.456317 didntLike
38000 1.712874 0.243945 didntLike
46321 13.109929 1.280111 largeDoses
66293 11.327910 0.780977 didntLike
22730 4.545711 1.233254 didntLike
5952 3.367889 0.468104 smallDoses
72308 8.326224 0.567347 didntLike
60338 8.978339 1.442034 didntLike
13301 5.655826 1.582159 smallDoses
27884 8.855312 0.570684 largeDoses
11188 6.649568 0.544233 smallDoses
56796 3.966325 0.850410 didntLike
8571 1.924045 1.664782 smallDoses
4914 6.004812 0.280369 smallDoses
10784 0.000000 0.375849 smallDoses
39296 9.923018 0.092192 largeDoses
13113 2.389084 0.119284 smallDoses
70204 13.663189 0.133251 didntLike
46813 11.434976 0.321216 largeDoses
11697 0.358270 1.292858 smallDoses
44183 9.598873 0.223524 largeDoses
2225 6.375275 0.608040 smallDoses
29066 11.580532 0.458401 largeDoses
4245 5.319324 1.598070 smallDoses
34379 4.324031 1.603481 didntLike
44441 2.358370 1.273204 didntLike
2022 0.000000 1.182708 smallDoses
26866 12.824376 0.890411 largeDoses
57070 1.587247 1.456982 didntLike
32932 8.510324 1.520683 largeDoses
51967 10.428884 1.187734 largeDoses
44432 8.346618 0.042318 largeDoses
67066 7.541444 0.809226 didntLike
17262 2.540946 1.583286 smallDoses
79728 9.473047 0.692513 didntLike
14259 0.352284 0.474080 smallDoses
6122 0.000000 0.589826 smallDoses
76879 12.405171 0.567201 didntLike
11426 4.126775 0.871452 smallDoses
2493 0.034087 0.335848 smallDoses
19910 1.177634 0.075106 smallDoses
10939 0.000000 0.479996 smallDoses
17716 0.994909 0.611135 smallDoses
31390 11.053664 1.180117 largeDoses
20375 0.000000 1.679729 smallDoses
26309 2.495011 1.459589 didntLike
33484 11.516831 0.001156 largeDoses
45944 9.213215 0.797743 largeDoses
4249 5.332865 0.109288 smallDoses
6089 0.000000 1.689771 smallDoses
7513 0.000000 1.126053 smallDoses
27862 12.640062 1.690903 largeDoses
39038 2.693142 1.317518 didntLike
19218 3.328969 0.268271 smallDoses
62911 7.193166 1.117456 didntLike
77758 6.615512 1.521012 didntLike
27940 8.000567 0.835341 largeDoses
2194 4.017541 0.512104 smallDoses
37072 13.245859 0.927465 largeDoses
15585 5.970616 0.813624 smallDoses
25577 11.668719 0.886902 largeDoses
8777 4.283237 1.272728 smallDoses
29016 10.742963 0.971401 largeDoses
21910 12.326672 1.592608 largeDoses
12916 0.000000 0.344622 smallDoses
10976 0.000000 0.922846 smallDoses
79065 10.602095 0.573686 didntLike
36759 10.861859 1.155054 largeDoses
50011 1.229094 1.638690 didntLike
1155 0.410392 1.313401 smallDoses
71600 14.552711 0.616162 didntLike
30817 14.178043 0.616313 largeDoses
54559 14.136260 0.362388 didntLike
29764 0.093534 1.207194 didntLike
69100 10.929021 0.403110 didntLike
47324 11.432919 0.825959 largeDoses
73199 9.134527 0.586846 didntLike
44461 5.071432 1.421420 didntLike
45617 11.460254 1.541749 largeDoses
28221 11.620039 1.103553 largeDoses
7091 4.022079 0.207307 smallDoses
6110 3.057842 1.631262 smallDoses
79016 7.782169 0.404385 didntLike
18289 7.981741 0.929789 largeDoses
43679 4.601363 0.268326 didntLike
22075 2.595564 1.115375 didntLike
23535 10.049077 0.391045 largeDoses
25301 3.265444 1.572970 smallDoses
32256 11.780282 1.511014 largeDoses
36951 3.075975 0.286284 didntLike
31290 1.795307 0.194343 didntLike
38953 11.106979 0.202415 largeDoses
35257 5.994413 0.800021 didntLike
25847 9.706062 1.012182 largeDoses
32680 10.582992 0.836025 largeDoses
62018 7.038266 1.458979 didntLike
9074 0.023771 0.015314 smallDoses
33004 12.823982 0.676371 largeDoses
44588 3.617770 0.493483 didntLike
32565 8.346684 0.253317 largeDoses
38563 6.104317 0.099207 didntLike
75668 16.207776 0.584973 didntLike
9069 6.401969 1.691873 smallDoses
53395 2.298696 0.559757 didntLike
28631 7.661515 0.055981 largeDoses
71036 6.353608 1.645301 didntLike
71142 10.442780 0.335870 didntLike
37653 3.834509 1.346121 didntLike
76839 10.998587 0.584555 didntLike
9916 2.695935 1.512111 smallDoses
38889 3.356646 0.324230 didntLike
39075 14.677836 0.793183 largeDoses
48071 1.551934 0.130902 didntLike
7275 2.464739 0.223502 smallDoses
41804 1.533216 1.007481 didntLike
35665 12.473921 0.162910 largeDoses
67956 6.491596 0.032576 didntLike
41892 10.506276 1.510747 largeDoses
38844 4.380388 0.748506 didntLike
74197 13.670988 1.687944 didntLike
14201 8.317599 0.390409 smallDoses
3908 0.000000 0.556245 smallDoses
2459 0.000000 0.290218 smallDoses
32027 10.095799 1.188148 largeDoses
12870 0.860695 1.482632 smallDoses
9880 1.557564 0.711278 smallDoses
72784 10.072779 0.756030 didntLike
17521 0.000000 0.431468 smallDoses
50283 7.140817 0.883813 largeDoses
33536 11.384548 1.438307 largeDoses
9452 3.214568 1.083536 smallDoses
37457 11.720655 0.301636 largeDoses
17724 6.374475 1.475925 largeDoses
43869 5.749684 0.198875 largeDoses
264 3.871808 0.552602 smallDoses
25736 8.336309 0.636238 largeDoses
39584 9.710442 1.503735 largeDoses
31246 1.532611 1.433898 didntLike
49567 9.785785 0.984614 largeDoses
7052 2.633627 1.097866 smallDoses
35493 9.238935 0.494701 largeDoses
10986 1.205656 1.398803 smallDoses
49508 3.124909 1.670121 didntLike
5734 7.935489 1.585044 smallDoses
65479 12.746636 1.560352 didntLike
77268 10.732563 0.545321 didntLike
28490 3.977403 0.766103 didntLike
13546 4.194426 0.450663 smallDoses
37166 9.610286 0.142912 largeDoses
16381 4.797555 1.260455 smallDoses
10848 1.615279 0.093002 smallDoses
35405 4.614771 1.027105 didntLike
15917 0.000000 1.369726 smallDoses
6131 0.608457 0.512220 smallDoses
67432 6.558239 0.667579 didntLike
30354 12.315116 0.197068 largeDoses
69696 7.014973 1.494616 didntLike
33481 8.822304 1.194177 largeDoses
43075 10.086796 0.570455 largeDoses
38343 7.241614 1.661627 largeDoses
14318 4.602395 1.511768 smallDoses
5367 7.434921 0.079792 smallDoses
37894 10.467570 1.595418 largeDoses
36172 9.948127 0.003663 largeDoses
40123 2.478529 1.568987 didntLike
10976 5.938545 0.878540 smallDoses
12705 0.000000 0.948004 smallDoses
12495 5.559181 1.357926 smallDoses
35681 9.776654 0.535966 largeDoses
46202 3.092056 0.490906 didntLike
11505 0.000000 1.623311 smallDoses
22834 4.459495 0.538867 didntLike
49901 8.334306 1.646600 largeDoses
71932 11.226654 0.384686 didntLike
13279 3.904737 1.597294 smallDoses
49112 7.038205 1.211329 largeDoses
77129 9.836120 1.054340 didntLike
37447 1.990976 0.378081 didntLike
62397 9.005302 0.485385 didntLike
0 1.772510 1.039873 smallDoses
15476 0.458674 0.819560 smallDoses
40625 10.003919 0.231658 largeDoses
36706 0.520807 1.476008 didntLike
28580 10.678214 1.431837 largeDoses
25862 4.425992 1.363842 didntLike
63488 12.035355 0.831222 didntLike
33944 10.606732 1.253858 largeDoses
30099 1.568653 0.684264 didntLike
13725 2.545434 0.024271 smallDoses
36768 10.264062 0.982593 largeDoses
64656 9.866276 0.685218 didntLike
14927 0.142704 0.057455 smallDoses
43231 9.853270 1.521432 largeDoses
66087 6.596604 1.653574 didntLike
19806 2.602287 1.321481 smallDoses
41081 10.411776 0.664168 largeDoses
10277 7.083449 0.622589 smallDoses
7014 2.080068 1.254441 smallDoses
17275 0.522844 1.622458 smallDoses
31600 10.362000 1.544827 largeDoses
59956 3.412967 1.035410 didntLike
42181 6.796548 1.112153 largeDoses
51743 4.092035 0.075804 didntLike
5194 2.763811 1.564325 smallDoses
30832 12.547439 1.402443 largeDoses
7976 5.708052 1.596152 smallDoses
14602 4.558025 0.375806 smallDoses
41571 11.642307 0.438553 largeDoses
55028 3.222443 0.121399 didntLike
5837 4.736156 0.029871 smallDoses
39808 10.839526 0.836323 largeDoses
20944 4.194791 0.235483 smallDoses
22146 14.936259 0.888582 largeDoses
42169 3.310699 1.521855 didntLike
7010 2.971931 0.034321 smallDoses
3807 9.261667 0.537807 smallDoses
29241 7.791833 1.111416 largeDoses
52696 1.480470 1.028750 didntLike
42545 3.677287 0.244167 didntLike
24437 2.202967 1.370399 didntLike
16037 5.796735 0.935893 smallDoses
8493 3.063333 0.144089 smallDoses
68080 11.233094 0.492487 didntLike
59016 1.965570 0.005697 didntLike
11810 8.616719 0.137419 smallDoses
68630 6.609989 1.083505 didntLike
7629 1.712639 1.086297 smallDoses
71992 10.117445 1.299319 didntLike
13398 0.000000 1.104178 smallDoses
26241 9.824777 1.346821 largeDoses
11160 1.653089 0.980949 smallDoses
76701 18.178822 1.473671 didntLike
32174 6.781126 0.885340 largeDoses
45043 8.206750 1.549223 largeDoses
42173 10.081853 1.376745 largeDoses
69801 6.288742 0.112799 didntLike
41737 3.695937 1.543589 didntLike
46979 6.726151 1.069380 largeDoses
79267 12.969999 1.568223 didntLike
4615 2.661390 1.531933 smallDoses
32907 7.072764 1.117386 largeDoses
37444 9.123366 1.318988 largeDoses
569 3.743946 1.039546 smallDoses
8723 2.341300 0.219361 smallDoses
6024 0.541913 0.592348 smallDoses
52252 2.310828 1.436753 didntLike
8358 6.226597 1.427316 smallDoses
26166 7.277876 0.489252 largeDoses
18471 0.000000 0.389459 smallDoses
3386 7.218221 1.098828 smallDoses
41544 8.777129 1.111464 largeDoses
10480 2.813428 0.819419 smallDoses
5894 2.268766 1.412130 smallDoses
7273 6.283627 0.571292 smallDoses
22272 7.520081 1.626868 largeDoses
31369 11.739225 0.027138 largeDoses
10708 3.746883 0.877350 smallDoses
69364 12.089835 0.521631 didntLike
37760 12.310404 0.259339 largeDoses
13004 0.000000 0.671355 smallDoses
37885 2.728800 0.331502 didntLike
52555 10.814342 0.607652 largeDoses
38997 12.170268 0.844205 largeDoses
69698 6.698371 0.240084 didntLike
11783 3.632672 1.643479 smallDoses
47636 10.059991 0.892361 largeDoses
15744 1.887674 0.756162 smallDoses
69058 8.229125 0.195886 didntLike
33057 7.817082 0.476102 largeDoses
28681 12.277230 0.076805 largeDoses
34042 10.055337 1.115778 largeDoses
29928 3.596002 1.485952 didntLike
9734 2.755530 1.420655 smallDoses
7344 7.780991 0.513048 smallDoses
7387 0.093705 0.391834 smallDoses
33957 8.481567 0.520078 largeDoses
9936 3.865584 0.110062 smallDoses
36094 9.683709 0.779984 largeDoses
39835 10.617255 1.359970 largeDoses
64486 7.203216 1.624762 didntLike
0 7.601414 1.215605 smallDoses
39539 1.386107 1.417070 didntLike
66972 9.129253 0.594089 didntLike
15029 1.363447 0.620841 smallDoses
44909 3.181399 0.359329 didntLike
38183 13.365414 0.217011 largeDoses
37372 4.207717 1.289767 didntLike
0 4.088395 0.870075 smallDoses
17786 3.327371 1.142505 smallDoses
39055 1.303323 1.235650 didntLike
37045 7.999279 1.581763 largeDoses
6435 2.217488 0.864536 smallDoses
72265 7.751808 0.192451 didntLike
28152 14.149305 1.591532 largeDoses
25931 8.765721 0.152808 largeDoses
7538 3.408996 0.184896 smallDoses
1315 1.251021 0.112340 smallDoses
12292 6.160619 1.537165 smallDoses
49248 1.034538 1.585162 didntLike
9025 0.000000 1.034635 smallDoses
13438 2.355051 0.542603 smallDoses
69683 6.614543 0.153771 didntLike
25374 10.245062 1.450903 largeDoses
55264 3.467074 1.231019 didntLike
38324 7.487678 1.572293 largeDoses
69643 4.624115 1.185192 didntLike
44058 8.995957 1.436479 largeDoses
41316 11.564476 0.007195 largeDoses
29119 3.440948 0.078331 didntLike
51656 1.673603 0.732746 didntLike
3030 4.719341 0.699755 smallDoses
35695 10.304798 1.576488 largeDoses
1537 2.086915 1.199312 smallDoses
9083 6.338220 1.131305 smallDoses
47744 8.254926 0.710694 largeDoses
71372 16.067108 0.974142 didntLike
37980 1.723201 0.310488 didntLike
42385 3.785045 0.876904 didntLike
22687 2.557561 0.123738 didntLike
39512 9.852220 1.095171 largeDoses
11885 3.679147 1.557205 smallDoses
4944 9.789681 0.852971 smallDoses
73230 14.958998 0.526707 didntLike
17585 11.182148 1.288459 largeDoses
68737 7.528533 1.657487 didntLike
13818 5.253802 1.378603 smallDoses
31662 13.946752 1.426657 largeDoses
86686 15.557263 1.430029 didntLike
43214 12.483550 0.688513 largeDoses
24091 2.317302 1.411137 didntLike
52544 10.069724 0.766119 largeDoses
61861 5.792231 1.615483 didntLike
47903 4.138435 0.475994 didntLike
37190 12.929517 0.304378 largeDoses
6013 9.378238 0.307392 smallDoses
27223 8.361362 1.643204 largeDoses
69027 7.939406 1.325042 didntLike
78642 10.735384 0.705788 didntLike
30254 11.592723 0.286188 largeDoses
21704 10.098356 0.704748 largeDoses
34985 9.299025 0.545337 largeDoses
31316 11.158297 0.218067 largeDoses
76368 16.143900 0.558388 didntLike
27953 10.971700 1.221787 largeDoses
152 0.000000 0.681478 smallDoses
9146 3.178961 1.292692 smallDoses
75346 17.625350 0.339926 didntLike
26376 1.995833 0.267826 didntLike
35255 10.640467 0.416181 largeDoses
19198 9.628339 0.985462 largeDoses
12518 4.662664 0.495403 smallDoses
25453 5.754047 1.382742 smallDoses
12530 0.000000 0.037146 smallDoses
62230 9.334332 0.198118 didntLike
9517 3.846162 0.619968 smallDoses
71161 10.685084 0.678179 didntLike
1593 4.752134 0.359205 smallDoses
33794 0.697630 0.966786 didntLike
39710 10.365836 0.505898 largeDoses
16941 0.461478 0.352865 smallDoses
69209 11.339537 1.068740 didntLike
4446 5.420280 0.127310 smallDoses
9347 3.469955 1.619947 smallDoses
55635 8.517067 0.994858 largeDoses
65889 8.306512 0.413690 didntLike
10753 2.628690 0.444320 smallDoses
7055 0.000000 0.802985 smallDoses
7905 0.000000 1.170397 smallDoses
53447 7.298767 1.582346 largeDoses
9194 7.331319 1.277988 smallDoses
61914 9.392269 0.151617 didntLike
15630 5.541201 1.180596 smallDoses
79194 15.149460 0.537540 didntLike
12268 5.515189 0.250562 smallDoses
33682 7.728898 0.920494 largeDoses
26080 11.318785 1.510979 largeDoses
19119 3.574709 1.531514 smallDoses
30902 7.350965 0.026332 largeDoses
63039 7.122363 1.630177 didntLike
51136 1.828412 1.013702 didntLike
35262 10.117989 1.156862 largeDoses
42776 11.309897 0.086291 largeDoses
64191 8.342034 1.388569 didntLike
15436 0.241714 0.715577 smallDoses
14402 10.482619 1.694972 smallDoses
6341 9.289510 1.428879 smallDoses
14113 4.269419 0.134181 smallDoses
6390 0.000000 0.189456 smallDoses
8794 0.817119 0.143668 smallDoses
43432 1.508394 0.652651 didntLike
38334 9.359918 0.052262 largeDoses
34068 10.052333 0.550423 largeDoses
30819 11.111660 0.989159 largeDoses
22239 11.265971 0.724054 largeDoses
28725 10.383830 0.254836 largeDoses
57071 3.878569 1.377983 didntLike
72420 13.679237 0.025346 didntLike
28294 10.526846 0.781569 largeDoses
9896 0.000000 0.924198 smallDoses
65821 4.106727 1.085669 didntLike
7645 8.118856 1.470686 smallDoses
71289 7.796874 0.052336 didntLike
5128 2.789669 1.093070 smallDoses
13711 6.226962 0.287251 smallDoses
22240 10.169548 1.660104 largeDoses
15092 0.000000 1.370549 smallDoses
5017 7.513353 0.137348 smallDoses
10141 8.240793 0.099735 smallDoses
35570 14.612797 1.247390 largeDoses
46893 3.562976 0.445386 didntLike
8178 3.230482 1.331698 smallDoses
55783 3.612548 1.551911 didntLike
1148 0.000000 0.332365 smallDoses
10062 3.931299 0.487577 smallDoses
74124 14.752342 1.155160 didntLike
66603 10.261887 1.628085 didntLike
11893 2.787266 1.570402 smallDoses
50908 15.112319 1.324132 largeDoses
39891 5.184553 0.223382 largeDoses
65915 3.868359 0.128078 didntLike
65678 3.507965 0.028904 didntLike
62996 11.019254 0.427554 didntLike
36851 3.812387 0.655245 didntLike
36669 11.056784 0.378725 largeDoses
38876 8.826880 1.002328 largeDoses
26878 11.173861 1.478244 largeDoses
46246 11.506465 0.421993 largeDoses
12761 7.798138 0.147917 largeDoses
35282 10.155081 1.370039 largeDoses
68306 10.645275 0.693453 didntLike
31262 9.663200 1.521541 largeDoses
34754 10.790404 1.312679 largeDoses
13408 2.810534 0.219962 smallDoses
30365 9.825999 1.388500 largeDoses
10709 1.421316 0.677603 smallDoses
24332 11.123219 0.809107 largeDoses
45517 13.402206 0.661524 largeDoses
6178 1.212255 0.836807 smallDoses
10639 1.568446 1.297469 smallDoses
29613 3.343473 1.312266 didntLike
22392 5.400155 0.193494 didntLike
51126 3.818754 0.590905 didntLike
53644 7.973845 0.307364 largeDoses
51417 9.078824 0.734876 largeDoses
24859 0.153467 0.766619 didntLike
61732 8.325167 0.028479 didntLike
71128 7.092089 1.216733 didntLike
27276 5.192485 1.094409 largeDoses
30453 10.340791 1.087721 largeDoses
18670 2.077169 1.019775 smallDoses
70600 10.151966 0.993105 didntLike
12683 0.046826 0.809614 smallDoses
81597 11.221874 1.395015 didntLike
69959 14.497963 1.019254 didntLike
8124 3.554508 0.533462 smallDoses
18867 3.522673 0.086725 smallDoses
80886 14.531655 0.380172 didntLike
55895 3.027528 0.885457 didntLike
31587 1.845967 0.488985 didntLike
10591 10.226164 0.804403 largeDoses
70096 10.965926 1.212328 didntLike
53151 2.129921 1.477378 didntLike
11992 0.000000 1.606849 smallDoses
33114 9.489005 0.827814 largeDoses
7413 0.000000 1.020797 smallDoses
10583 0.000000 1.270167 smallDoses
58668 6.556676 0.055183 didntLike
35018 9.959588 0.060020 largeDoses
70843 7.436056 1.479856 didntLike
14011 0.404888 0.459517 smallDoses
35015 9.952942 1.650279 largeDoses
70839 15.600252 0.021935 didntLike
3024 2.723846 0.387455 smallDoses
5526 0.513866 1.323448 smallDoses
5113 0.000000 0.861859 smallDoses
20851 7.280602 1.438470 smallDoses
40999 9.161978 1.110180 largeDoses
15823 0.991725 0.730979 smallDoses
35432 7.398380 0.684218 largeDoses
53711 12.149747 1.389088 largeDoses
64371 9.149678 0.874905 didntLike
9289 9.666576 1.370330 smallDoses
60613 3.620110 0.287767 didntLike
18338 5.238800 1.253646 smallDoses
22845 14.715782 1.503758 largeDoses
74676 14.445740 1.211160 didntLike
34143 13.609528 0.364240 largeDoses
14153 3.141585 0.424280 smallDoses
9327 0.000000 0.120947 smallDoses
18991 0.454750 1.033280 smallDoses
9193 0.510310 0.016395 smallDoses
2285 3.864171 0.616349 smallDoses
9493 6.724021 0.563044 smallDoses
2371 4.289375 0.012563 smallDoses
13963 0.000000 1.437030 smallDoses
2299 3.733617 0.698269 smallDoses
5262 2.002589 1.380184 smallDoses
4659 2.502627 0.184223 smallDoses
17582 6.382129 0.876581 smallDoses
27750 8.546741 0.128706 largeDoses
9868 2.694977 0.432818 smallDoses
18333 3.951256 0.333300 smallDoses
3780 9.856183 0.329181 smallDoses
18190 2.068962 0.429927 smallDoses
11145 3.410627 0.631838 smallDoses
68846 9.974715 0.669787 didntLike
26575 10.650102 0.866627 largeDoses
48111 9.134528 0.728045 largeDoses
43757 7.882601 1.332446 largeDoses
这里我们还需要先把标签替换成数字型的以便识别，largeDoses -> 3, smallDoses -> 2, didntLike -> 1
def file2matrix(filename):
fr = open(filename)
arrayOLines = fr.readlines()
numberOfLines = len(arrayOLines)
returnMat = zeros((numberOfLines,3))
classLabelVector = []
index = 0;
for line in arrayOLines:
line = line.strip() # 去掉所有的回车符
listFromLine = line.split('\t') # 按照tab字符分割成list
returnMat[index,:] = listFromLine[0:3]
# 我们必须明确地将标签值转换成整型，否则python语言会将其当做字符串处理
classLabelVector.append(int(listFromLine[-1]))
index += 1
return returnMat, classLabelVector
接下来我们可以采用图形化的方式直观的展示数据。
2.2 Analyze: creating scatter plots with Matplotlib
我们在cmd中定位到源码所在路径，然后绘制原始数据的散点图：
>>> import kNN
>>> datingDataMat, datingLabels = kNN.file2matrix('datingTestSet2.txt')
>>> import matplotlib
>>> import matplotlib.pyplot as plt
>>> fig = plt.figure()
>>> ax = fig.add_subplot(111)
>>> ax.scatter(datingDataMat[:,1], datingDataMat[:,2])
>>> plt.show()
由于没有使用样本分类的label，我们很难从上图看到任何有用的数据模式信息。为了更好的理解数据信息，我们可以使用色彩或者其他记号来区别标记
datingDataMat, datingLabels = file2matrix('datingTestSet2.txt')
from matplotlib.font_manager import FontProperties
import matplotlib.pyplot as plt
zhfont1 = FontProperties(fname='C:\Windows\Fonts\simkai.ttf',size=16)
fig = plt.figure()
ax = fig.add_subplot(111)
ax.scatter(datingDataMat[:,1], datingDataMat[:,2], 15.0*array(datingLabels), 15.0*array(datingLabels))
plt.xlabel(u'玩游戏所耗时间百分比', fontproperties=zhfont1)
plt.ylabel(u'每周消费的冰淇淋公升数', fontproperties=zhfont1)
plt.show()
2.3 Prepare: normalizing numeric values
newValue = (oldValue-min)/(max-min)
def autoNorm(dataSet):
minVals = dataSet.min(0) # 比较每行获得特征矩阵最小值
maxVals = dataSet.max(0) # 比较每行获得特征矩阵最大值
ranges = maxVals - minVals
normDataSet = zeros(shape(dataSet))
m = dataSet.shape[0] # 特征向量行数
normDataSet = dataSet - tile(minVals, (m,1)) # tile函数将变量内容复制成dataSet同样大小的矩阵
normDataSet = normDataSet/tile(ranges, (m,1))
return normDataSet, ranges, minVals
2.4 Test: testing the classifier as a whole program
机器学习算法一个很重要的工作就是评估算法的正确率，通常我们只提供已有数据的90%作为训练样本来训练分类器，而使用其余的10%数据去测试分类器，检测分类器的正确率。
这里我们可以随机选择这10%的测试样本，也可以顺序选择。
最终我们可以选择错误率来检测分类器的性能。
def datingClassTest():
hoRatio = 0.10 # 测试集所占比例
datingDataMat, datingLabels = file2matrix('datingTestSet2.txt') # 读入样本集
normMat ,ranges, minVals = autoNorm(datingDataMat) # 特征归一化
m = normMat.shape[0]
numTestVecs = int(m*hoRatio)
errorCount = 0.0
# 对于每个测试样本，测试结果，并统计错误次数
for i in range(numTestVecs):
classifierResult = classify0(normMat[i,:], normMat[numTestVecs:m,:], datingLabels[numTestVecs:m], 3)
print "the classifier came back with: %d, the real answer is: %d" % (classifierResult, datingLabels[i])
if classifierResult != datingLabels[i]:
errorCount += 1.0
print "the total error rate is: %f" % (errorCount/float(numTestVecs))
2.5 Use: putting together a useful system
def classifyPerson():
resultList = ['not at all', 'in small doses', 'in large doses']
percentTats = float(raw_input("percentage of time spent playing video games?"))
ffMiles = float(raw_input("ferquent fliter miles earned per year?"))
iceCream = float(raw_input("liters of ice cream consumed per year?"))
datingDataMat, datingLabels = file2matrix('datingTestSet2.txt') # 读入样本集
normMat, ranges, minVals = autoNorm(datingDataMat) # 特征归一化
inArr = array([ffMiles, percentTats, iceCream]) # 预测样本特征
classifierResult = classify0((inArr-minVals)/ranges, normMat, datingLabels, 3)
print "You will probably like the person: ", resultList[classifierResult-1]
到此为止，一个简单的约会对象匹配算法就完成了！
3. Example: a handwriting recognition system
3.1 Prepare: converting images into test vectors
为了简单起见，这里构造的系统只能识别数字0-9。需要识别的数字已经使用图形处理软件，处理成具有相同的色彩和大小：宽高是32像素$\times$32像素的黑白图像。尽管采用文本格式存储图形不能有效地利用内存空间，但是为了方便理解，我们还是将图像转换为文本格式。
def img2vector(filename):
returnVect = zeros((1,1024))
fr = open(filename)
for i in range(32):
lineStr = fr.readline() # 读取一行数据
for j in range(32):
returnVect[0,32*i+j] = int(lineStr[j]) # 数据默认都是以字符形式读入，需要强制转换
return returnVect testVector = img2vector('testDigits/0_13.txt')
print testVector[0,0:31]
这样我们就可以借助于之前的kNN算法代码测试了
Test: kNN on handwritten digits
from os import listdir
def handwritingClasstest():
hwLabels = []
trainingFileList = listdir('trainingDigits') # 获得路径下所有文件名
m = len(trainingFileList) # 训练样本数
trainingMat = zeros((m,1024))
for i in range(m):
fileNameStr = trainingFileList[i]
fileStr = fileNameStr.split('.')[0]
classNumStr = int(fileStr.split('_')[0]) # 训练样本标签
hwLabels.append(classNumStr)
trainingMat[i,:] = img2vector('trainingDigits/%s' % fileNameStr) # 读取样本
testFileList = listdir('testDigits')
errorCount = 0.0
mTest = len(testFileList) # 测试样本数
for i in range(mTest):
fileNameStr = testFileList[i]
fileStr = fileNameStr.split('.')[0]
classNumStr = int(fileStr.split('_')[0]) # 测试样本标签
vectorUnderTest = img2vector('testDigits/%s' % fileNameStr)
classifierResult = classify0(vectorUnderTest, trainingMat, hwLabels, 3) # 预测样本类别
print "the classifier came back with: %d, the real answer is: %d" % (classifierResult, classNumStr)
if classifierResult != classNumStr:
errorCount += 1.0 # 统计错误分类数
print "\nthe total number of errors is: %d" % errorCount
print "\nthe total error rate is: %f" % (errorCount/mTest) handwritingClasstest()
实际使用这个算法是，算法的执行效率并不高。因为预测时，测试样本要和所有样本做距离计算。后面我们会使用一种$k$决策树算法来节省计算开销。