ML之xgboost：利用xgboost算法(sklearn+3Split+调参曲线)训练mushroom蘑菇数据集(22+1,6513+1611)来预测蘑菇是否毒性(二分类预测)
目录
输出结果
设计思路
核心代码
eval_set = [(X_train_part, y_train_part), (X_validate, y_validate)]
bst.fit(X_train_part, y_train_part, eval_metric=["error", "logloss"], eval_set=eval_set, verbose=True)
preds = bst.predict(X_test)
predictions = [round(value) for value in preds]
test_accuracy = accuracy_score(y_test, predictions)
print("【max_depth=2,lr=0.1】Test Accuracy: %.2f%%" % (test_accuracy * 100.0))
results = bst.evals_result()
更多输出
X_train： (6513, 126)
X_test： (1611, 126)
After split(33%)，X_train_part： (4363, 126)
After split(33%)，X_validate： (2150, 126)
[0] validation_0-error:0.045611 validation_0-logloss:0.614637 validation_1-error:0.048372 validation_1-logloss:0.615401
[1] validation_0-error:0.041256 validation_0-logloss:0.549907 validation_1-error:0.042326 validation_1-logloss:0.550696
[2] validation_0-error:0.045611 validation_0-logloss:0.49543 validation_1-error:0.048372 validation_1-logloss:0.496777
[3] validation_0-error:0.041256 validation_0-logloss:0.449089 validation_1-error:0.042326 validation_1-logloss:0.450412
[4] validation_0-error:0.041256 validation_0-logloss:0.409231 validation_1-error:0.042326 validation_1-logloss:0.410717
[5] validation_0-error:0.041256 validation_0-logloss:0.373748 validation_1-error:0.042326 validation_1-logloss:0.375653
[6] validation_0-error:0.023378 validation_0-logloss:0.343051 validation_1-error:0.023256 validation_1-logloss:0.344738
[7] validation_0-error:0.041256 validation_0-logloss:0.315369 validation_1-error:0.042326 validation_1-logloss:0.317409
[8] validation_0-error:0.041256 validation_0-logloss:0.290912 validation_1-error:0.042326 validation_1-logloss:0.292587
[9] validation_0-error:0.023378 validation_0-logloss:0.269356 validation_1-error:0.023256 validation_1-logloss:0.271103
[10] validation_0-error:0.00573 validation_0-logloss:0.249593 validation_1-error:0.006512 validation_1-logloss:0.251354
[11] validation_0-error:0.01719 validation_0-logloss:0.228658 validation_1-error:0.017674 validation_1-logloss:0.230144
[12] validation_0-error:0.01719 validation_0-logloss:0.210442 validation_1-error:0.017674 validation_1-logloss:0.21167
[13] validation_0-error:0.01719 validation_0-logloss:0.194562 validation_1-error:0.017674 validation_1-logloss:0.19555
[14] validation_0-error:0.01719 validation_0-logloss:0.1807 validation_1-error:0.017674 validation_1-logloss:0.181463
[15] validation_0-error:0.01719 validation_0-logloss:0.168585 validation_1-error:0.017674 validation_1-logloss:0.169138
[16] validation_0-error:0.01719 validation_0-logloss:0.157988 validation_1-error:0.017674 validation_1-logloss:0.158345
[17] validation_0-error:0.01719 validation_0-logloss:0.149407 validation_1-error:0.017674 validation_1-logloss:0.149731
[18] validation_0-error:0.0259 validation_0-logloss:0.140835 validation_1-error:0.024651 validation_1-logloss:0.140979
[19] validation_0-error:0.022003 validation_0-logloss:0.133937 validation_1-error:0.020465 validation_1-logloss:0.13405
[20] validation_0-error:0.022003 validation_0-logloss:0.126967 validation_1-error:0.020465 validation_1-logloss:0.126914
[21] validation_0-error:0.022003 validation_0-logloss:0.121386 validation_1-error:0.020465 validation_1-logloss:0.121303
[22] validation_0-error:0.022003 validation_0-logloss:0.115692 validation_1-error:0.020465 validation_1-logloss:0.115456
[23] validation_0-error:0.022003 validation_0-logloss:0.111147 validation_1-error:0.020465 validation_1-logloss:0.110881
[24] validation_0-error:0.022003 validation_0-logloss:0.106477 validation_1-error:0.020465 validation_1-logloss:0.10607
[25] validation_0-error:0.022003 validation_0-logloss:0.102434 validation_1-error:0.020465 validation_1-logloss:0.102319
[26] validation_0-error:0.022003 validation_0-logloss:0.098434 validation_1-error:0.020465 validation_1-logloss:0.09819
[27] validation_0-error:0.022003 validation_0-logloss:0.094875 validation_1-error:0.020465 validation_1-logloss:0.094824
[28] validation_0-error:0.022003 validation_0-logloss:0.091579 validation_1-error:0.020465 validation_1-logloss:0.091784
[29] validation_0-error:0.013294 validation_0-logloss:0.086202 validation_1-error:0.013488 validation_1-logloss:0.086807
[30] validation_0-error:0.022003 validation_0-logloss:0.083247 validation_1-error:0.020465 validation_1-logloss:0.083741
[31] validation_0-error:0.022003 validation_0-logloss:0.080496 validation_1-error:0.020465 validation_1-logloss:0.080924
[32] validation_0-error:0.022003 validation_0-logloss:0.077298 validation_1-error:0.020465 validation_1-logloss:0.077394
[33] validation_0-error:0.015815 validation_0-logloss:0.074507 validation_1-error:0.016279 validation_1-logloss:0.074765
[34] validation_0-error:0.022003 validation_0-logloss:0.071848 validation_1-error:0.020465 validation_1-logloss:0.071811
[35] validation_0-error:0.010543 validation_0-logloss:0.069488 validation_1-error:0.009302 validation_1-logloss:0.069385
[36] validation_0-error:0.001834 validation_0-logloss:0.067147 validation_1-error:0.002326 validation_1-logloss:0.067341
[37] validation_0-error:0.001834 validation_0-logloss:0.06504 validation_1-error:0.002326 validation_1-logloss:0.065406
[38] validation_0-error:0.001834 validation_0-logloss:0.062898 validation_1-error:0.002326 validation_1-logloss:0.063381
[39] validation_0-error:0.001834 validation_0-logloss:0.060837 validation_1-error:0.002326 validation_1-logloss:0.061088
[40] validation_0-error:0.001834 validation_0-logloss:0.058894 validation_1-error:0.002326 validation_1-logloss:0.059039
[41] validation_0-error:0.001834 validation_0-logloss:0.057112 validation_1-error:0.002326 validation_1-logloss:0.057326
[42] validation_0-error:0.001834 validation_0-logloss:0.055391 validation_1-error:0.002326 validation_1-logloss:0.05543
[43] validation_0-error:0.001834 validation_0-logloss:0.053745 validation_1-error:0.002326 validation_1-logloss:0.053871
[44] validation_0-error:0.001834 validation_0-logloss:0.052198 validation_1-error:0.002326 validation_1-logloss:0.052235
[45] validation_0-error:0.001834 validation_0-logloss:0.050776 validation_1-error:0.002326 validation_1-logloss:0.051033
[46] validation_0-error:0.001834 validation_0-logloss:0.049351 validation_1-error:0.002326 validation_1-logloss:0.04973
[47] validation_0-error:0.001834 validation_0-logloss:0.047848 validation_1-error:0.002326 validation_1-logloss:0.048287
[48] validation_0-error:0.001834 validation_0-logloss:0.046406 validation_1-error:0.002326 validation_1-logloss:0.046702
[49] validation_0-error:0.001834 validation_0-logloss:0.045141 validation_1-error:0.002326 validation_1-logloss:0.045492
[50] validation_0-error:0.001834 validation_0-logloss:0.043917 validation_1-error:0.002326 validation_1-logloss:0.044133
[51] validation_0-error:0.001834 validation_0-logloss:0.042729 validation_1-error:0.002326 validation_1-logloss:0.042999
[52] validation_0-error:0.001834 validation_0-logloss:0.041608 validation_1-error:0.002326 validation_1-logloss:0.041807
[53] validation_0-error:0.001834 validation_0-logloss:0.040493 validation_1-error:0.002326 validation_1-logloss:0.040855
[54] validation_0-error:0.001834 validation_0-logloss:0.039457 validation_1-error:0.002326 validation_1-logloss:0.039871
[55] validation_0-error:0.001834 validation_0-logloss:0.038452 validation_1-error:0.002326 validation_1-logloss:0.038755
[56] validation_0-error:0.001834 validation_0-logloss:0.037478 validation_1-error:0.002326 validation_1-logloss:0.037717
[57] validation_0-error:0.001834 validation_0-logloss:0.036439 validation_1-error:0.002326 validation_1-logloss:0.036777
[58] validation_0-error:0.001834 validation_0-logloss:0.035552 validation_1-error:0.002326 validation_1-logloss:0.035936
[59] validation_0-error:0.001834 validation_0-logloss:0.034694 validation_1-error:0.002326 validation_1-logloss:0.034984
[60] validation_0-error:0.001834 validation_0-logloss:0.033826 validation_1-error:0.002326 validation_1-logloss:0.034132
[61] validation_0-error:0.001834 validation_0-logloss:0.032959 validation_1-error:0.002326 validation_1-logloss:0.033348
[62] validation_0-error:0.001834 validation_0-logloss:0.032192 validation_1-error:0.002326 validation_1-logloss:0.032526
[63] validation_0-error:0.001834 validation_0-logloss:0.031476 validation_1-error:0.002326 validation_1-logloss:0.031754
[64] validation_0-error:0.001834 validation_0-logloss:0.030756 validation_1-error:0.002326 validation_1-logloss:0.031081
[65] validation_0-error:0.001834 validation_0-logloss:0.030038 validation_1-error:0.002326 validation_1-logloss:0.030377
[66] validation_0-error:0.001834 validation_0-logloss:0.029332 validation_1-error:0.002326 validation_1-logloss:0.029594
[67] validation_0-error:0.001834 validation_0-logloss:0.028703 validation_1-error:0.002326 validation_1-logloss:0.029079
[68] validation_0-error:0.001834 validation_0-logloss:0.028064 validation_1-error:0.002326 validation_1-logloss:0.028391
[69] validation_0-error:0.001834 validation_0-logloss:0.027404 validation_1-error:0.002326 validation_1-logloss:0.027725
[70] validation_0-error:0.001834 validation_0-logloss:0.026824 validation_1-error:0.002326 validation_1-logloss:0.027187
[71] validation_0-error:0.001834 validation_0-logloss:0.026268 validation_1-error:0.002326 validation_1-logloss:0.026565
[72] validation_0-error:0.001834 validation_0-logloss:0.025679 validation_1-error:0.002326 validation_1-logloss:0.025982
[73] validation_0-error:0.001834 validation_0-logloss:0.025153 validation_1-error:0.002326 validation_1-logloss:0.025413
[74] validation_0-error:0.001834 validation_0-logloss:0.02461 validation_1-error:0.002326 validation_1-logloss:0.024927
[75] validation_0-error:0.001834 validation_0-logloss:0.0241 validation_1-error:0.002326 validation_1-logloss:0.02446
[76] validation_0-error:0.001834 validation_0-logloss:0.023615 validation_1-error:0.002326 validation_1-logloss:0.023921
[77] validation_0-error:0.001834 validation_0-logloss:0.023118 validation_1-error:0.002326 validation_1-logloss:0.023423
[78] validation_0-error:0.001834 validation_0-logloss:0.022671 validation_1-error:0.002326 validation_1-logloss:0.023015
[79] validation_0-error:0.001834 validation_0-logloss:0.022244 validation_1-error:0.002326 validation_1-logloss:0.022538
[80] validation_0-error:0.001834 validation_0-logloss:0.021793 validation_1-error:0.002326 validation_1-logloss:0.022087
[81] validation_0-error:0.001834 validation_0-logloss:0.021396 validation_1-error:0.002326 validation_1-logloss:0.021654
[82] validation_0-error:0.001834 validation_0-logloss:0.020948 validation_1-error:0.002326 validation_1-logloss:0.021198
[83] validation_0-error:0.001834 validation_0-logloss:0.020559 validation_1-error:0.002326 validation_1-logloss:0.020806
[84] validation_0-error:0.001834 validation_0-logloss:0.020144 validation_1-error:0.002326 validation_1-logloss:0.020388
[85] validation_0-error:0.001834 validation_0-logloss:0.019775 validation_1-error:0.002326 validation_1-logloss:0.020057
[86] validation_0-error:0.001834 validation_0-logloss:0.019029 validation_1-error:0.002326 validation_1-logloss:0.019235
[87] validation_0-error:0.001834 validation_0-logloss:0.018672 validation_1-error:0.002326 validation_1-logloss:0.018823
[88] validation_0-error:0.001834 validation_0-logloss:0.018313 validation_1-error:0.002326 validation_1-logloss:0.018507
[89] validation_0-error:0.001834 validation_0-logloss:0.017989 validation_1-error:0.002326 validation_1-logloss:0.01815
[90] validation_0-error:0.001834 validation_0-logloss:0.017376 validation_1-error:0.002326 validation_1-logloss:0.01748
[91] validation_0-error:0.001834 validation_0-logloss:0.017087 validation_1-error:0.002326 validation_1-logloss:0.017189
[92] validation_0-error:0.001834 validation_0-logloss:0.016778 validation_1-error:0.002326 validation_1-logloss:0.016877
[93] validation_0-error:0.001834 validation_0-logloss:0.016458 validation_1-error:0.002326 validation_1-logloss:0.016527
[94] validation_0-error:0.001834 validation_0-logloss:0.015932 validation_1-error:0.002326 validation_1-logloss:0.015956
[95] validation_0-error:0.001834 validation_0-logloss:0.015645 validation_1-error:0.002326 validation_1-logloss:0.015665
[96] validation_0-error:0.001834 validation_0-logloss:0.015379 validation_1-error:0.002326 validation_1-logloss:0.015397
[97] validation_0-error:0.001834 validation_0-logloss:0.015116 validation_1-error:0.002326 validation_1-logloss:0.01513
[98] validation_0-error:0.001834 validation_0-logloss:0.014883 validation_1-error:0.002326 validation_1-logloss:0.014893
[99] validation_0-error:0.001834 validation_0-logloss:0.01464 validation_1-error:0.002326 validation_1-logloss:0.014624
【max_depth=2,lr=0.1】Test Accuracy: 99.81%
{'validation_0': {'error': [0.045611, 0.041256, 0.045611, 0.041256, 0.041256, 0.041256, 0.023378, 0.041256, 0.041256, 0.023378, 0.00573, 0.01719, 0.01719, 0.01719, 0.01719, 0.01719, 0.01719, 0.01719, 0.0259, 0.022003, 0.022003, 0.022003, 0.022003, 0.022003, 0.022003, 0.022003, 0.022003, 0.022003, 0.022003, 0.013294, 0.022003, 0.022003, 0.022003, 0.015815, 0.022003, 0.010543, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834, 0.001834], 'logloss': [0.614637, 0.549907, 0.49543, 0.449089, 0.409231, 0.373748, 0.343051, 0.315369, 0.290912, 0.269356, 0.249593, 0.228658, 0.210442, 0.194562, 0.1807, 0.168585, 0.157988, 0.149407, 0.140835, 0.133937, 0.126967, 0.121386, 0.115692, 0.111147, 0.106477, 0.102434, 0.098434, 0.094875, 0.091579, 0.086202, 0.083247, 0.080496, 0.077298, 0.074507, 0.071848, 0.069488, 0.067147, 0.06504, 0.062898, 0.060837, 0.058894, 0.057112, 0.055391, 0.053745, 0.052198, 0.050776, 0.049351, 0.047848, 0.046406, 0.045141, 0.043917, 0.042729, 0.041608, 0.040493, 0.039457, 0.038452, 0.037478, 0.036439, 0.035552, 0.034694, 0.033826, 0.032959, 0.032192, 0.031476, 0.030756, 0.030038, 0.029332, 0.028703, 0.028064, 0.027404, 0.026824, 0.026268, 0.025679, 0.025153, 0.02461, 0.0241, 0.023615, 0.023118, 0.022671, 0.022244, 0.021793, 0.021396, 0.020948, 0.020559, 0.020144, 0.019775, 0.019029, 0.018672, 0.018313, 0.017989, 0.017376, 0.017087, 0.016778, 0.016458, 0.015932, 0.015645, 0.015379, 0.015116, 0.014883, 0.01464]}, 'validation_1': {'error': [0.048372, 0.042326, 0.048372, 0.042326, 0.042326, 0.042326, 0.023256, 0.042326, 0.042326, 0.023256, 0.006512, 0.017674, 0.017674, 0.017674, 0.017674, 0.017674, 0.017674, 0.017674, 0.024651, 0.020465, 0.020465, 0.020465, 0.020465, 0.020465, 0.020465, 0.020465, 0.020465, 0.020465, 0.020465, 0.013488, 0.020465, 0.020465, 0.020465, 0.016279, 0.020465, 0.009302, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326, 0.002326], 'logloss': [0.615401, 0.550696, 0.496777, 0.450412, 0.410717, 0.375653, 0.344738, 0.317409, 0.292587, 0.271103, 0.251354, 0.230144, 0.21167, 0.19555, 0.181463, 0.169138, 0.158345, 0.149731, 0.140979, 0.13405, 0.126914, 0.121303, 0.115456, 0.110881, 0.10607, 0.102319, 0.09819, 0.094824, 0.091784, 0.086807, 0.083741, 0.080924, 0.077394, 0.074765, 0.071811, 0.069385, 0.067341, 0.065406, 0.063381, 0.061088, 0.059039, 0.057326, 0.05543, 0.053871, 0.052235, 0.051033, 0.04973, 0.048287, 0.046702, 0.045492, 0.044133, 0.042999, 0.041807, 0.040855, 0.039871, 0.038755, 0.037717, 0.036777, 0.035936, 0.034984, 0.034132, 0.033348, 0.032526, 0.031754, 0.031081, 0.030377, 0.029594, 0.029079, 0.028391, 0.027725, 0.027187, 0.026565, 0.025982, 0.025413, 0.024927, 0.02446, 0.023921, 0.023423, 0.023015, 0.022538, 0.022087, 0.021654, 0.021198, 0.020806, 0.020388, 0.020057, 0.019235, 0.018823, 0.018507, 0.01815, 0.01748, 0.017189, 0.016877, 0.016527, 0.015956, 0.015665, 0.015397, 0.01513, 0.014893, 0.014624]}}