xgboost参数详解
官方参数介绍看这里：
Parameters (official guide)
General Parameters（常规参数）
1.booster [default=gbtree]：选择基分类器，gbtree: tree-based models/gblinear: linear models
2.silent [default=0]:设置成1则没有运行信息输出，最好是设置为0.
3.nthread [default to maximum number of threads available if not set]：线程数
Booster Parameters（模型参数）
1.eta [default=0.3]:shrinkage参数，用于更新叶子节点权重时，乘以该系数，避免步长过大。参数值越大，越可能无法收敛。把学习率 eta 设置的小一些，小学习率可以使得后面的学习更加仔细。
2.min_child_weight [default=1]:这个参数默认是 1，是每个叶子里面 h 的和至少是多少，对正负样本不均衡时的 0-1 分类而言，假设 h 在 0.01 附近，min_child_weight 为 1 意味着叶子节点中最少需要包含 100 个样本。这个参数非常影响结果，控制叶子节点中二阶导的和的最小值，该参数值越小，越容易 overfitting。
3.max_depth [default=6]: 每颗树的最大深度，树高越深，越容易过拟合。
4.max_leaf_nodes:最大叶结点数，与max_depth作用有点重合。
5.gamma [default=0]：后剪枝时，用于控制是否后剪枝的参数。
6.max_delta_step [default=0]：这个参数在更新步骤中起作用，如果取0表示没有约束，如果取正值则使得更新步骤更加保守。可以防止做太大的更新步子，使更新更加平缓。
7.subsample [default=1]：样本随机采样，较低的值使得算法更加保守，防止过拟合，但是太小的值也会造成欠拟合。
8.colsample_bytree [default=1]：列采样，对每棵树的生成用的特征进行列采样.一般设置为： 0.5-1
9.lambda [default=1]：控制模型复杂度的权重值的L2正则化项参数，参数越大，模型越不容易过拟合。
10.alpha [default=0]:控制模型复杂程度的权重值的 L1 正则项参数，参数值越大，模型越不容易过拟合。
11.scale_pos_weight [default=1]：如果取值大于0的话，在类别样本不平衡的情况下有助于快速收敛。
Learning Task Parameters（学习任务参数）
1.objective [default=reg:linear]：定义最小化损失函数类型，常用参数：
binary:logistic –logistic regression for binary classification, returns predicted probability (not class)
multi:softmax –multiclass classification using the softmax objective, returns predicted class (not probabilities)
you also need to set an additional num_class (number of classes) parameter defining the number of unique classes
multi:softprob –same as softmax, but returns predicted probability of each data point belonging to each class.
2.eval_metric [ default according to objective ]：
The metric to be used for validation data.
The default values are rmse for regression and error for classification.
Typical values are:
rmse – root mean square error
mae – mean absolute error
logloss – negative log-likelihood
error – Binary classification error rate (0.5 threshold)
merror – Multiclass classification error rate
mlogloss – Multiclass logloss
auc: Area under the curve
3.seed [default=0]：
The random number seed. 随机种子，用于产生可复现的结果
Can be used for generating reproducible results and also for parameter tuning.
注意: python sklearn style参数名会有所变化
eta –> learning_rate
lambda –> reg_lambda
alpha –> reg_alpha
参数解释：
Parameter for Tree Booster
- eta [default=0.3]
- 为了防止过拟合，更新过程中用到的收缩步长。在每次提升计算之后，算法会直接获得新特征的权重。 eta通过缩减特征的权重使提升计算过程更加保守。
缺省值为0.3 - 取值范围为：[0,1]
- 为了防止过拟合，更新过程中用到的收缩步长。在每次提升计算之后，算法会直接获得新特征的权重。 eta通过缩减特征的权重使提升计算过程更加保守。
- gamma [default=0]
- minimum loss reduction required to make a further partition on a leaf node of the tree. the larger, the more conservative the algorithm will be.
- range: [0,∞]
- max_depth [default=6]
- 数的最大深度。
缺省值为6 - 取值范围为：[1,∞]
- 数的最大深度。
- min_child_weight [default=1]
- 孩子节点中最小的样本权重和。如果一个叶子节点的样本权重和小于min_child_weight则拆分过程结束。在现行回归模型中，这个参数是指建立每个模型所需要的最小样本数。该成熟越大算法越conservative
- 取值范围为: [0,∞]
- max_delta_step [default=0]
- Maximum delta step we allow each tree’s weight estimation to be. If the value is set to 0, it means there is no constraint. If it is set to a positive value, it can help making the update step more conservative. Usually this parameter is not needed, but it might help in logistic regression when class is extremely imbalanced. Set it to value of 1-10 might help control the update
- 取值范围为：[0,∞]
- subsample [default=1]
- 用于训练模型的子样本占整个样本集合的比例。如果设置为0.5则意味着XGBoost将随机的冲整个样本集合中随机的抽取出50%的子样本建立树模型，这能够防止过拟合。
- 取值范围为：(0,1]
- colsample_bytree [default=1]
- 在建立树时对特征采样的比例。
缺省值为1 - 取值范围：(0,1]
- 在建立树时对特征采样的比例。
Parameter for Linear Booster
- lambda [default=0]
- L2 正则的惩罚系数
- alpha [default=0]
- L1 正则的惩罚系数
- lambda_bias
- 在偏置上的L2正则。
缺省值为0（在L1上没有偏置项的正则，因为L1时偏置不重要）
- 在偏置上的L2正则。
Task Parameters
- objective [ default=reg:linear ]
- 定义学习任务及相应的学习目标，可选的目标函数如下：
- “reg:linear” –线性回归。
- “reg:logistic” –逻辑回归。
- “binary:logistic” –二分类的逻辑回归问题，输出为概率。
- “binary:logitraw” –二分类的逻辑回归问题，输出的结果为wTx。
- “count:poisson” –计数问题的poisson回归，输出结果为poisson分布。
- 在poisson回归中，max_delta_step的缺省值为0.7。(used to safeguard optimization)
- “multi:softmax” –让XGBoost采用softmax目标函数处理多分类问题，同时需要设置参数num_class（类别个数）
- “multi:softprob” –和softmax一样，但是输出的是ndata * nclass的向量，可以将该向量reshape成ndata行nclass列的矩阵。没行数据表示样本所属于每个类别的概率。
- “rank:pairwise” –set XGBoost to do ranking task by minimizing the pairwise loss
- base_score [ default=0.5 ]
- the initial prediction score of all instances, global bias
- eval_metric [ default according to objective ]
- 校验数据所需要的评价指标，不同的目标函数将会有缺省的评价指标（rmse for regression, and error for classification, mean average precision for ranking）
- 用户可以添加多种评价指标，对于Python用户要以list传递参数对给程序，而不是map参数list参数不会覆盖’eval_metric’
- The choices are listed below:
- “rmse”: root mean square error
- “logloss”: negative log-likelihood
- “error”: Binary classification error rate. It is calculated as #(wrong cases)/#(all cases). For the predictions, the evaluation will regard the instances with prediction value larger than 0.5 as positive instances, and the others as negative instances.
- “merror”: Multiclass classification error rate. It is calculated as #(wrong cases)/#(all cases).
- “mlogloss”: Multiclass logloss
- “auc”: Area under the curve for ranking evaluation.
- “ndcg”:Normalized Discounted Cumulative Gain
- “map”:Mean average precision
- “ndcg@n”,”map@n”: n can be assigned as an integer to cut off the top positions in the lists for evaluation.
- “ndcg-“,”map-“,”ndcg@n-“,”map@n-“: In XGBoost, NDCG and MAP will evaluate the score of a list without any positive samples as 1. By adding “-” in the evaluation metric XGBoost will evaluate these score as 0 to be consistent under some conditions.
training repeatively
- seed [ default=0 ]
- 随机数的种子。
缺省值为0
- 随机数的种子。
Console Parameters
The following parameters are only used in the console version of xgboost
* use_buffer [ default=1 ]
- 是否为输入创建二进制的缓存文件，缓存文件可以加速计算。缺省值为1
* num_round
- boosting迭代计算次数。
* data
- 输入数据的路径
* test:data
- 测试数据的路径
* save_period [default=0]
- 表示保存第i*save_period次迭代的模型。例如save_period=10表示每隔10迭代计算XGBoost将会保存中间结果，设置为0表示每次计算的模型都要保持。
* task [default=train] options: train, pred, eval, dump
- train：训练明显
- pred：对测试数据进行预测
- eval：通过eval[name]=filenam定义评价指标
- dump：将学习模型保存成文本格式
* model_in [default=NULL]
- 指向模型的路径在test, eval, dump都会用到，如果在training中定义XGBoost将会接着输入模型继续训练
* model_out [default=NULL]
- 训练完成后模型的保持路径，如果没有定义则会输出类似0003.model这样的结果，0003是第三次训练的模型结果。
* model_dir [default=models]
- 输出模型所保存的路径。
* fmap
- feature map, used for dump model
* name_dump [default=dump.txt]
- name of model dump file
* name_pred [default=pred.txt]
- 预测结果文件
* pred_margin [default=0]
- 输出预测的边界，而不是转换后的概率
回归
#include <xgboost/c_api.h>
void usexgb()
{
// create the train data
const int cols = 3, rows = 10;
float train[rows][cols];
for (int i = 0; i<rows; i++)
for (int j = 0; j<cols; j++)
train[i][j] = (i + 1) * (j + 1);
float train_labels[rows];
for (int i = 0; i<rows; i++)
train_labels[i] = 1 + i*i;
// convert to DMatrix
DMatrixHandle h_train[1];
XGDMatrixCreateFromMat((float *)train, rows, cols, -1, &h_train[0]);
// load the labels
XGDMatrixSetFloatInfo(h_train[0], "label", train_labels, rows);
// read back the labels, just a sanity check
bst_ulong bst_result;
const float *out_floats;
XGDMatrixGetFloatInfo(h_train[0], "label", &bst_result, &out_floats);
for (unsigned int i = 0; i<bst_result; i++)
std::cout << "label[" << i << "]=" << out_floats[i] << std::endl;
// create the booster and load some parameters
BoosterHandle h_booster;
XGBoosterCreate(h_train, 1, &h_booster);
XGBoosterSetParam(h_booster, "booster", "gbtree");
XGBoosterSetParam(h_booster, "objective", "reg:linear");
XGBoosterSetParam(h_booster, "max_depth", "5");
XGBoosterSetParam(h_booster, "eta", "0.1");
XGBoosterSetParam(h_booster, "min_child_weight", "1");
XGBoosterSetParam(h_booster, "subsample", "0.5");
XGBoosterSetParam(h_booster, "colsample_bytree", "1");
XGBoosterSetParam(h_booster, "num_parallel_tree", "1");
// perform 200 learning iterations
for (int iter = 0; iter<200; iter++)
XGBoosterUpdateOneIter(h_booster, iter, h_train[0]);
// predict
const int sample_rows = 10;
float test[sample_rows][cols];
for (int i = 0; i<sample_rows; i++)
for (int j = 0; j<cols; j++)
test[i][j] = (i + 1) * (j + 1);
DMatrixHandle h_test;
XGDMatrixCreateFromMat((float *)test, sample_rows, cols, -1, &h_test);
bst_ulong out_len;
const float *f;
XGBoosterPredict(h_booster, h_test, 0, 0, &out_len, &f);
for (unsigned int i = 0; i<out_len; i++)
std::cout << "prediction[" << i << "]=" << f[i] << std::endl;
// free xgboost internal structures
XGDMatrixFree(h_train[0]);
XGDMatrixFree(h_test);
XGBoosterFree(h_booster);
}
二分类
void usexgbClassify()
{
// create the train data
const int cols = 3, rows = 100;
float train[rows][cols];
for (int i = 0; i<rows; i++)
for (int j = 0; j<cols; j++)
train[i][j] = (i + 1) * (j + 1);
float train_labels[rows];
for (int i = 0; i < rows; i++)
{
train_labels[i] = 0;
if (i > rows / 2)
{
train_labels[i] = 1;
}
}
// convert to DMatrix
DMatrixHandle h_train[1];
XGDMatrixCreateFromMat((float *)train, rows, cols, -1, &h_train[0]);
// load the labels
//XGDMatrixSetFloatInfo(h_train[0], "label", train_labels, rows);
XGDMatrixSetFloatInfo(h_train[0], "label", train_labels, rows);
// read back the labels, just a sanity check
bst_ulong bst_result;
const float *out_floats;
XGDMatrixGetFloatInfo(h_train[0], "label", &bst_result, &out_floats);
for (unsigned int i = 0; i<bst_result; i++)
std::cout << "label[" << i << "]=" << out_floats[i] << std::endl;
// create the booster and load some parameters
BoosterHandle h_booster;
XGBoosterCreate(h_train, 1, &h_booster);
XGBoosterSetParam(h_booster, "booster", "gbtree");
XGBoosterSetParam(h_booster, "objective", "binary:logistic");
XGBoosterSetParam(h_booster, "max_depth", "5");
XGBoosterSetParam(h_booster, "eta", "0.2");
XGBoosterSetParam(h_booster, "min_child_weight", "1");
XGBoosterSetParam(h_booster, "subsample", "0.5");
XGBoosterSetParam(h_booster, "colsample_bytree", "1");
XGBoosterSetParam(h_booster, "num_parallel_tree", "1");
XGBoosterSetParam(h_booster, "silent", "1");
// perform 200 learning iterations
for (int iter = 0; iter<200; iter++)
XGBoosterUpdateOneIter(h_booster, iter, h_train[0]);
// predict
const int sample_rows = 10;
float test[sample_rows][cols];
for (int i = 45; i<45+sample_rows; i++)
for (int j = 0; j<cols; j++)
test[i-45][j] = (i + 1) * (j + 1);
DMatrixHandle h_test;
XGDMatrixCreateFromMat((float *)test, sample_rows, cols, -1, &h_test);
bst_ulong out_len;
const float *f;
XGBoosterPredict(h_booster, h_test, 0, 0, &out_len, &f);
for (unsigned int i = 0; i<out_len; i++)
std::cout << "prediction[" << i << "]=" << f[i] << std::endl;
// free xgboost internal structures
XGDMatrixFree(h_train[0]);
XGDMatrixFree(h_test);
XGBoosterFree(h_booster);
}
3.多分类
void getTestData(float ** &dataset, float * &label, int flag)
{
if (flag == 0)
{
int feature = 3;
int length = 100;
float* data_mem = new float[length * feature];
dataset = new float*[length];
label = new float[length];
for (int i = 0; i < length; i++)
{
dataset[i] = data_mem + i* feature;
for (int j = 0; j < feature; j++)
{
dataset[i][j] = (i + 1) * (j + 1);
}
if (i < 30)
{
label[i] = 0;
}
else if (i < 60)
{
label[i] = 1;
}
else
{
label[i] = 2;
}
}
for (int i = 0; i < length; i++)
{
for (int j = 0; j < feature; j++)
{
cout << dataset[i][j]<<" ";
}
cout << label[i]<< endl;
}
}
}
void xgboostTrain(float **&dataset, float *&label, int rows, int cols, string modelpath)
{
DMatrixHandle h_train[1];
XGDMatrixCreateFromMat((float *)dataset[0], rows, cols, -1, &h_train[0]); //注意此处使用的是 dataset[0]......如果是二维数组则可直接写成dataset
// load the labels
XGDMatrixSetFloatInfo(h_train[0], "label", label, rows);
// create the booster and load some parameters
BoosterHandle h_booster;
XGBoosterCreate(h_train, 1, &h_booster);
XGBoosterSetParam(h_booster, "booster", "gbtree");
XGBoosterSetParam(h_booster, "objective", "multi:softmax");
XGBoosterSetParam(h_booster, "max_depth", "5");
XGBoosterSetParam(h_booster, "eta", "0.2");
XGBoosterSetParam(h_booster, "min_child_weight", "1");
XGBoosterSetParam(h_booster, "subsample", "0.5");
XGBoosterSetParam(h_booster, "colsample_bytree", "1");
XGBoosterSetParam(h_booster, "num_parallel_tree", "1");
XGBoosterSetParam(h_booster, "silent", "1");
XGBoosterSetParam(h_booster, "num_class", "3");
// perform 200 learning iterations
for (int iter = 0; iter < 200; iter++)
{
XGBoosterUpdateOneIter(h_booster, iter, h_train[0]);
}
XGBoosterSaveModel(h_booster, modelpath.c_str());
// predict
const int sample_rows = 100;
float test[sample_rows][3];
for (int i = 0; i < sample_rows; i++)
for (int j = 0; j < 3; j++)
test[i][j] = (i + 1) * (j + 1);
DMatrixHandle h_test;
XGDMatrixCreateFromMat((float *)test, sample_rows, cols, -1.0, &h_test);
bst_ulong out_len;
const float *f;
XGBoosterPredict(h_booster, h_test, 0, 0, &out_len, &f);
for (unsigned int i = 0; i<out_len; i++)
std::cout << "prediction[" << i << "]=" << f[i] << std::endl;
// free xgboost internal structures
XGDMatrixFree(h_train[0]);
XGDMatrixFree(h_test);
XGBoosterFree(h_booster);
}
人脸属性分类
#include <iostream>
#include <vector>
#include <fstream>
#include <istream>
#include <string>
#include <io.h>
#include <direct.h>
#include "../CascadRegression/MCLC.h"
#include <opencv2/opencv.hpp>
#include <xgboost/c_api.h>
using namespace std;
using namespace caffe;
using namespace cv;
#define DEVICE 0
#define FEATUREDIM 128
#define FLAG 4
const int TRAIN = 50000;
const int TEST = 10973;
string age[] = { "婴儿", "少年", "青年", "中年", "老人" }; // 0-6 , 7-17 ,18-40 ,41-65 ,66+
string emotion[] = { "中立","微笑", "大笑", "生气", "悲伤", "惊讶" };
string race[] = { "黄", "黑", "白" };
string six[] = { "男", "女" };
//string path = "D:\\face\\face_ldmk_trainset\\result_img_128\\";
struct Attributed
{
string filename;
int age_label;
int six_label;
int race_label;
int emotion_label;
};
template <class Type>
Type stringToNum(const string& str)
{
istringstream iss(str);
Type num;
iss >> num;
return num;
}
void splitString(const string& s, vector<string>& v, const string& c)
{
string::size_type pos1, pos2;
pos2 = s.find(c);
pos1 = 0;
while (string::npos != pos2)
{
v.push_back(s.substr(pos1, pos2 - pos1));
pos1 = pos2 + c.size();
pos2 = s.find(c, pos1);
}
if (pos1 != s.length()) {
v.push_back(s.substr(pos1));
}
}
void parseData(string buf, Attributed& mark)
{
vector<string> result;
splitString(buf, result, " ");
mark.filename = result[0];
mark.age_label = stringToNum<int>(result[1]);
mark.emotion_label = stringToNum<int>(result[2]);
mark.race_label = stringToNum<int>(result[3]);
mark.six_label = stringToNum<int>(result[4]);
}
void parseAgeSixData(string buf, Attributed& mark)
{
vector<string> result;
splitString(buf, result, " ");
mark.filename = result[0];
mark.age_label = stringToNum<int>(result[1]);
mark.six_label = stringToNum<int>(result[2]);
}
void readData(string filePath, vector<Attributed> & result, int count = INT_MAX)
{
ifstream fileA(filePath);
if (!fileA)
{
cout << "没有找到需要读取的 " << filePath << " 请将文件放到指定位置再次运行本程序。" << endl << " 按任意键以退出";
return;
}
for (int i = 0; !fileA.eof() && (i < count); i++)
{
Attributed atb;
string buf;
getline(fileA, buf, '\n');
if (buf == "")
{
cout << "buf is empty." << endl;
continue;
}
//parseData(buf, atb);
parseAgeSixData(buf, atb);
result.push_back(atb);
}
fileA.close();
}
void writeDatatoFile(std::string filePath, vector<Attributed> & data)
{
if (filePath == "" || data.size() == 0)
{
return;
}
ofstream agefile_train;
ofstream racefile_train;
ofstream sixfile_train;
ofstream emmotionfile_train;
ofstream data_train;
ofstream agefile_test;
ofstream racefile_test;
ofstream sixfile_test;
ofstream emmotionfile_test;
ofstream data_test;
agefile_train.open(filePath + "age_train.txt", ios::app);
racefile_train.open(filePath + "race_train.txt", ios::app);
sixfile_train.open(filePath + "six_train.txt", ios::app);
emmotionfile_train.open(filePath + "emmotion_train.txt", ios::app);
data_train.open(filePath + "data_train.txt", ios::app);
agefile_test.open(filePath + "age_test.txt", ios::app);
racefile_test.open(filePath + "race_test.txt", ios::app);
sixfile_test.open(filePath + "six_test.txt", ios::app);
emmotionfile_test.open(filePath + "emmotion_test.txt", ios::app);
data_test.open(filePath + "data_test.txt", ios::app);
int length = data.size();
int count = 0;
for (int i = 0; i < length; i++)
{
string dir = "D:\\face\\face_ldmk_trainset\\result_img_48\\" + data[i].filename;
if (_access(dir.c_str(), 0) == -1)
{
continue;
}
string ageline = data[i].filename;
ageline.append(" ");
ageline.append(to_string(data[i].age_label));
string raceline = data[i].filename;
raceline.append(" ");
raceline.append(to_string(data[i].race_label));
string sixline = data[i].filename;
sixline.append(" ");
sixline.append(to_string(data[i].six_label));
string emotionline = data[i].filename;
emotionline.append(" ");
emotionline.append(to_string(data[i].emotion_label));
string dataline = data[i].filename;
dataline.append(" ");
dataline.append(to_string(data[i].age_label));
dataline.append(" ");
dataline.append(to_string(data[i].emotion_label));
dataline.append(" ");
dataline.append(to_string(data[i].race_label));
dataline.append(" ");
dataline.append(to_string(data[i].six_label));
if (count < 50000)
{
agefile_train << ageline << "\n";
racefile_train << raceline << "\n";
sixfile_train << sixline << "\n";
emmotionfile_train << emotionline << "\n";
data_train << dataline << "\n";
}
else
{
agefile_test << ageline << "\n";
racefile_test << raceline << "\n";
sixfile_test << sixline << "\n";
emmotionfile_test << emotionline << "\n";
data_test << dataline << "\n";
}
count++;
}
agefile_test.close();
racefile_test.close();
sixfile_test.close();
emmotionfile_test.close();
agefile_train.close();
racefile_train.close();
sixfile_train.close();
emmotionfile_train.close();
data_train.close();
data_test.close();
}
double extractFeature(MCLC & mclc, int id, Mat &img, int dims, vector<float> & feature)
{
double time = 0;
double t0 = (double)cvGetTickCount();
vector<Mat> imgdata;
imgdata.push_back(img);
unordered_map<std::string, DataBlob> result = mclc.Forward(imgdata, id);
double t1 = (double)cvGetTickCount();
time = (t1 - t0) / ((double)cvGetTickFrequency() * 1000);
//cout << "time is:" << time << " ms" << endl;
for (unordered_map<string, DataBlob>::iterator iter = result.begin(); iter != result.end(); iter++) {
//cout << "key value is" << iter->first << " the mapped value is " << " result" << endl;
string key = iter->first;
DataBlob result = iter->second;
if (result.name == "fc5")
{
for (int i = 0; i < dims; i++)
{
float x = *(result.data);
feature.push_back(x);
(result.data)++;
}
}
}
return time;
}
void getAllFeatureData(MCLC & mclc, int id, vector<Attributed> &data, float ** &dataset, float ** &label, string path)
{
int length = data.size();
float* data_mem = new float[length*FEATUREDIM];
float* label_mem = new float[FLAG*length];
dataset = new float*[length];
label = new float*[FLAG];
for (int i = 0; i < FLAG; i++)
{
label[i] = label_mem + i * length;
}
for (int i = 0; i < length; i++)
{
if (i % 1000 == 0)
{
cout << "extractFeature is:" << i << endl;
}
vector<float> feature;
Mat img = imread(path + data[i].filename);
extractFeature(mclc, id, img, FEATUREDIM, feature);
dataset[i] = data_mem + i* FEATUREDIM;
for (int j = 0; j < FEATUREDIM; j++)
{
dataset[i][j] = feature[j];
}
label[0][i] = data[i].age_label;
label[1][i] = data[i].emotion_label;
label[2][i] = data[i].race_label;
label[3][i] = data[i].six_label;
}
}
void getTestData(float ** &dataset, float * &label, int flag)
{
if (flag == 0)
{
int feature = 3;
int length = 100;
float* data_mem = new float[length * feature];
dataset = new float*[length];
label = new float[length];
for (int i = 0; i < length; i++)
{
dataset[i] = data_mem + i* feature;
for (int j = 0; j < feature; j++)
{
dataset[i][j] = (i + 1) * (j + 1);
}
if (i < 30)
{
label[i] = 0;
}
else if (i < 60)
{
label[i] = 1;
}
else
{
label[i] = 2;
}
}
for (int i = 0; i < length; i++)
{
for (int j = 0; j < feature; j++)
{
cout << dataset[i][j] << " ";
}
cout << label[i] << endl;
}
}
}
void xgboostTrain(float ** &dataset, float ** &label, int rows, int cols, string modelpath, int flag, int num_class)
{
DMatrixHandle h_train[1];
XGDMatrixCreateFromMat((float *)dataset[0], rows, cols, -1, &h_train[0]);
// load the labels
XGDMatrixSetFloatInfo(h_train[0], "label", label[flag], rows);
// create the booster and load some parameters
BoosterHandle h_booster;
XGBoosterCreate(h_train, 1, &h_booster);
XGBoosterSetParam(h_booster, "booster", "gbtree");
XGBoosterSetParam(h_booster, "objective", "multi:softmax");
XGBoosterSetParam(h_booster, "max_depth", "7");
XGBoosterSetParam(h_booster, "eta", "0.1");
XGBoosterSetParam(h_booster, "min_child_weight", "1");
XGBoosterSetParam(h_booster, "subsample", "0.6");
XGBoosterSetParam(h_booster, "colsample_bytree", "1");
XGBoosterSetParam(h_booster, "num_parallel_tree", "1");
XGBoosterSetParam(h_booster, "silent", "1");
XGBoosterSetParam(h_booster, "num_class", to_string(num_class).c_str());
XGBoosterSetParam(h_booster, "lambda", "0.01");
//XGBoosterSetParam(h_booster, "scale_pos_weight", "");
// perform 200 learning iterations
for (int iter = 0; iter < 1000; iter++)
{
if (iter % 100 == 0)
{
cout <<"train iter :"<<iter<< endl;
}
XGBoosterUpdateOneIter(h_booster, iter, h_train[0]);
}
XGBoosterSaveModel(h_booster, modelpath.c_str());
// free xgboost internal structures
XGDMatrixFree(h_train[0]);
XGBoosterFree(h_booster);
}
void xgboostPredict(float **&dataset, float **&label, int rows, int cols, string modelpath, int flag)
{
BoosterHandle h_booster;
const float *f;
bst_ulong out_len;
DMatrixHandle h_test;
double t0 = (double)cvGetTickCount();
XGDMatrixCreateFromMat((float *)dataset[0], rows, cols, -1.0, &h_test);
XGBoosterCreate(&h_test, 1, &h_booster);
int ret = XGBoosterLoadModel(h_booster, modelpath.c_str()); //加载模型
XGBoosterPredict(h_booster, h_test, 0, 0, &out_len, &f); //预测
double t1 = (double)cvGetTickCount();
cout << "["<<flag<<"] predict time is:" << ((t1 - t0) / ((double)cvGetTickFrequency() * 1000) )/TEST <<" ms"<< endl;
float error = 0;
for (unsigned int i = 0; i < out_len; i++)
{
if (f[i] != label[flag][i])
{
//cout << " prediction error :[" << i << "]= " << f[i] << endl; //average rate:86.5215 %
error += 1;
}
}
// free xgboost internal structures
XGDMatrixFree(h_test);
XGBoosterFree(h_booster);
cout << "[" << flag << "] average rate:" << (rows - error) * 100 / rows << " %" << endl;
}
void usexgb()
{
// create the train data
const int cols = 3, rows = 10;
float train[rows][cols];
for (int i = 0; i < rows; i++)
for (int j = 0; j < cols; j++)
train[i][j] = (i + 1) * (j + 1);
float train_labels[rows];
for (int i = 0; i < rows; i++)
train_labels[i] = 1 + i*i;
// convert to DMatrix
DMatrixHandle h_train[1];
XGDMatrixCreateFromMat((float *)train, rows, cols, -1, &h_train[0]);
// load the labels
XGDMatrixSetFloatInfo(h_train[0], "label", train_labels, rows);
// read back the labels, just a sanity check
bst_ulong bst_result;
const float *out_floats;
XGDMatrixGetFloatInfo(h_train[0], "label", &bst_result, &out_floats);
for (unsigned int i = 0; i < bst_result; i++)
std::cout << "label[" << i << "]=" << out_floats[i] << std::endl;
// create the booster and load some parameters
BoosterHandle h_booster;
XGBoosterCreate(h_train, 1, &h_booster);
XGBoosterSetParam(h_booster, "booster", "gbtree");
XGBoosterSetParam(h_booster, "objective", "reg:linear");
XGBoosterSetParam(h_booster, "max_depth", "5");
XGBoosterSetParam(h_booster, "eta", "0.1");
XGBoosterSetParam(h_booster, "min_child_weight", "1");
XGBoosterSetParam(h_booster, "subsample", "0.5");
XGBoosterSetParam(h_booster, "colsample_bytree", "1");
XGBoosterSetParam(h_booster, "num_parallel_tree", "1");
// perform 200 learning iterations
for (int iter = 0; iter < 200; iter++)
XGBoosterUpdateOneIter(h_booster, iter, h_train[0]);
// predict
const int sample_rows = 10;
float test[sample_rows][cols];
for (int i = 0; i < sample_rows; i++)
for (int j = 0; j < cols; j++)
test[i][j] = (i + 1) * (j + 1);
DMatrixHandle h_test;
XGDMatrixCreateFromMat((float *)test, sample_rows, cols, -1, &h_test);
bst_ulong out_len;
const float *f;
XGBoosterPredict(h_booster, h_test, 0, 0, &out_len, &f);
for (unsigned int i = 0; i < out_len; i++)
std::cout << "prediction[" << i << "]=" << f[i] << std::endl;
// free xgboost internal structures
XGDMatrixFree(h_train[0]);
XGDMatrixFree(h_test);
XGBoosterFree(h_booster);
}
void usexgbClassify()
{
// create the train data
const int cols = 3, rows = 100;
float train[rows][cols];
for (int i = 0; i < rows; i++)
for (int j = 0; j < cols; j++)
train[i][j] = (i + 1) * (j + 1);
float train_labels[rows];
for (int i = 0; i < rows; i++)
{
train_labels[i] = 0;
if (i > rows / 2)
{
train_labels[i] = 1;
}
}
// convert to DMatrix
DMatrixHandle h_train[1];
XGDMatrixCreateFromMat((float *)train, rows, cols, -1, &h_train[0]);
// load the labels
//XGDMatrixSetFloatInfo(h_train[0], "label", train_labels, rows);
XGDMatrixSetFloatInfo(h_train[0], "label", train_labels, rows);
// read back the labels, just a sanity check
bst_ulong bst_result;
const float *out_floats;
XGDMatrixGetFloatInfo(h_train[0], "label", &bst_result, &out_floats);
for (unsigned int i = 0; i < bst_result; i++)
std::cout << "label[" << i << "]=" << out_floats[i] << std::endl;
// create the booster and load some parameters
BoosterHandle h_booster;
XGBoosterCreate(h_train, 1, &h_booster);
XGBoosterSetParam(h_booster, "booster", "gbtree");
XGBoosterSetParam(h_booster, "objective", "binary:logistic");
XGBoosterSetParam(h_booster, "max_depth", "5");
XGBoosterSetParam(h_booster, "eta", "0.2");
XGBoosterSetParam(h_booster, "min_child_weight", "1");
XGBoosterSetParam(h_booster, "subsample", "0.5");
XGBoosterSetParam(h_booster, "colsample_bytree", "1");
XGBoosterSetParam(h_booster, "num_parallel_tree", "1");
XGBoosterSetParam(h_booster, "silent", "1");
// perform 200 learning iterations
for (int iter = 0; iter < 200; iter++)
XGBoosterUpdateOneIter(h_booster, iter, h_train[0]);
// predict
const int sample_rows = 10;
float test[sample_rows][cols];
for (int i = 45; i < 45 + sample_rows; i++)
for (int j = 0; j < cols; j++)
test[i - 45][j] = (i + 1) * (j + 1);
DMatrixHandle h_test;
XGDMatrixCreateFromMat((float *)test, sample_rows, cols, -1, &h_test);
bst_ulong out_len;
const float *f;
XGBoosterPredict(h_booster, h_test, 0, 0, &out_len, &f);
for (unsigned int i = 0; i < out_len; i++)
std::cout << "prediction[" << i << "]=" << f[i] << std::endl;
// free xgboost internal structures
XGDMatrixFree(h_train[0]);
XGDMatrixFree(h_test);
XGBoosterFree(h_booster);
}
int main()
{
string base = "E:\\work\\other_attribution\\model\\";
string protext = "E:\\work\\other_attribution\\model\\test_500000.prototxt";
string model = "E:\\work\\other_attribution\\model\\save_path\\_iter_500000.caffemodel";
//string file_train = path + "data_train.txt";
//string file_test = path + "data_test.txt";
string file_train = "D:\\face\\face_ldmk_trainset\\imdb\\imdb_result_img_128\\imdb.txt";
string file_test = "D:\\face\\face_ldmk_trainset\\wiki\\wiki_result_img_128\\wiki.txt";
int num_c[] = {5, 6, 3, 2};
string model_path[] = {base+ "xgb_age.model" , base + "xgb_emotion.model", base + "xgb_race.model" ,base + "xgb_sex.model" };
MCLC mclc;
int id = mclc.AddNet(protext, model, DEVICE);
vector<Attributed> data;
readData(file_train, data);
cout << "read Data size is:" << data.size() << endl;
float **dataset = NULL;
float **label = NULL;
//getTestData(dataset, label, 0);
getAllFeatureData(mclc, id, data, dataset, label, "D:\\face\\face_ldmk_trainset\\imdb\\imdb_result_img_128\\");
for (int i = 3; i < FLAG; i++)
{
xgboostTrain(dataset, label, TRAIN, FEATUREDIM, model_path[i], i, num_c[i]);
}
float **dataset_test = NULL;
float **label_test = NULL;
data.clear();
vector<Attributed>test_data;
readData(file_test, test_data);
getAllFeatureData(mclc, id, test_data, dataset_test, label_test, "D:\\face\\face_ldmk_trainset\\wiki\\wiki_result_img_128\\");
for (int i = 3; i < FLAG; i++)
{
xgboostPredict(dataset_test, label_test, TEST, FEATUREDIM, model_path[i], i);
}
system("PAUSE");
return 0;
}
/*
const int cols=3,rows=100;
float train[rows][cols];
for (int i=0;i<rows;i++)
for (int j=0;j<cols;j++)
train[i][j] = (i+1) * (j+1);
float train_labels[rows];
for (int i=0;i<50;i++)
train_labels[i] = 0;
for (int i=50;i<rows;i++)
train_labels[i] = 1;
// convert to DMatrix
DMatrixHandle h_train[1];
XGDMatrixCreateFromMat((float *) train, rows, cols, -1, &h_train[0]);
// load the labels
XGDMatrixSetFloatInfo(h_train[0], "label", train_labels, rows);
// read back the labels, just a sanity check
bst_ulong bst_result;
const float *out_floats;
XGDMatrixGetFloatInfo(h_train[0], "label" , &bst_result, &out_floats);
for (unsigned int i=0;i<bst_result;i++)
std::cout << "label[" << i << "]=" << out_floats[i] << std::endl;
// create the booster and load some parameters
BoosterHandle h_booster;
XGBoosterCreate(h_train, 1, &h_booster);
XGBoosterSetParam(h_booster, "objective", "binary:logistic");
XGBoosterSetParam(h_booster, "eval_metric", "error");
XGBoosterSetParam(h_booster, "silent", "0");
XGBoosterSetParam(h_booster, "max_depth", "9");
XGBoosterSetParam(h_booster, "eta", "0.1");
XGBoosterSetParam(h_booster, "min_child_weight", "3");
XGBoosterSetParam(h_booster, "gamma", "0.6");
XGBoosterSetParam(h_booster, "colsample_bytree", "1");
XGBoosterSetParam(h_booster, "subsample", "1");
XGBoosterSetParam(h_booster, "reg_alpha", "10");
// perform 200 learning iterations
for (int iter=0; iter<10; iter++)
XGBoosterUpdateOneIter(h_booster, iter, h_train[0]);
// predict
const int sample_rows = 100;
float test[sample_rows][cols];
for (int i=0;i<sample_rows;i++)
for (int j=0;j<cols;j++)
test[i][j] = (i+1) * (j+1);
DMatrixHandle h_test;
XGDMatrixCreateFromMat((float *) test, sample_rows, cols, -1, &h_test);
bst_ulong out_len;
const float *f;
XGBoosterPredict(h_booster, h_test, 0,0,&out_len,&f);
for (unsigned int i=0;i<out_len;i++)
std::cout << "prediction[" << i << "]=" << f[i] << std::endl;
// free xgboost internal structures
XGDMatrixFree(h_train[0]);
XGDMatrixFree(h_test);
XGBoosterFree(h_booster);
*/