ML之XGBoost：XGBoost参数调优的优秀外文翻译—《XGBoost中的参数调优完整指南(带python中的代码)》(二)

2023-07-27 14:01:22

原文题目：《Complete Guide to Parameter Tuning in XGBoost with codes in Python》
原文地址：https://www.analyticsvidhya.com/blog/2016/03/complete-guide-parameter-tuning-xgboost-with-codes-python/
所有权为原文所有，本文只负责翻译。

2. xgboost参数/XGBoost Parameters

The overall parameters have been divided into 3 categories by XGBoost authors:
XGBoost作者将总体参数分为3类：

General Parameters: Guide the overall functioning
一般参数：指导整体功能
Booster Parameters: Guide the individual booster (tree/regression) at each step
Booster参数：在每个步骤中引导单个助推器（树/回归）
Learning Task Parameters: Guide the optimization performed
学习任务参数：指导优化执行

I will give analogies to GBM here and highly recommend to read this article to learn from the very basics.
我将在此对GBM进行类比，并强烈建议阅读本文以从非常基础的内容中学习。

一般参数/General Parameters

These define the overall functionality of XGBoost.
这些定义了XGBoost的整体功能。

booster [default=gbtree] 助推器[默认值=gbtree]
- Select the type of model to run at each iteration. It has 2 options:
  选择要在每次迭代中运行的模型类型。它有两种选择：
  - gbtree: tree-based models gbtree:基于树的模型
  - gblinear: linear models GBLinear:线性模型
silent [default=0]: silent [默认值=0]：
- Silent mode is activated is set to 1, i.e. no running messages will be printed.
  silent模式激活设置为1，即不会打印正在运行的消息。
- It’s generally good to keep it 0 as the messages might help in understanding the model.
  一般来说，最好保持0，因为消息可能有助于理解模型。
nthread [default to maximum number of threads available if not set]
nthread[默认为最大可用线程数（如果未设置）]
- This is used for parallel processing and number of cores in the system should be entered
  这用于并行处理，应输入系统中的内核数。
- If you wish to run on all cores, value should not be entered and algorithm will detect automatically
  如果您希望在所有核心上运行，则不应输入值，算法将自动检测。

There are 2 more parameters which are set automatically by XGBoost and you need not worry about them. Lets move on to Booster parameters.
还有两个参数是由xgboost自动设置的，您不必担心它们。让我们继续讨论助推器参数。

Booster参数/Booster Parameters

Though there are 2 types of boosters, I’ll consider only tree booster here because it always outperforms the linear booster and thus the later is rarely used.
虽然有两种助推器，但这里我只考虑树助推器，因为它总是优于线性助推器，因此很少使用后者。

eta [default=0.3] eta[默认值=0.3]
- Analogous to learning rate in GBM 类似于GBM中的学习率
- Makes the model more robust by shrinking the weights on each step 通过收缩每一步的权重，使模型更加健壮
- Typical final values to be used: 0.01-0.2 使用的典型最终值：0.01-0.2
min_child_weight [default=1] 最小子权重[默认值=1]
- Defines the minimum sum of weights of all observations required in a child.
  定义子级中所需的所有观察值的最小权重之和。
- This is similar to min_child_leaf in GBM but not exactly. This refers to min “sum of weights” of observations while GBM has min “number of observations”.
  这与GBM中的Min_Child_Leaf类似，但不完全相同。这是指观测值的最小“权重和”，而GBM的最小“观测数”。
- Used to control over-fitting. Higher values prevent a model from learning relations which might be highly specific to the particular sample selected for a tree.
  用于控制装配。较高的值会阻止模型学习关系，这可能与为树选择的特定样本高度相关。
- Too high values can lead to under-fitting hence, it should be tuned using CV.
  过高的数值会导致拟合不足，因此应使用cv对其进行调整。
max_depth [default=6] 最大深度[默认值=6]
- The maximum depth of a tree, same as GBM. 树的最大深度，与gbm相同。
- Used to control over-fitting as higher depth will allow model to learn relations very specific to a particular sample.
  用于控制拟合，因为更高的深度将允许模型学习特定于特定样本的关系。
- Should be tuned using CV.
  应该使用cv进行调整。
- Typical values: 3-10
  典型值：3-10
max_leaf_nodes 最大叶节点数
- The maximum number of terminal nodes or leaves in a tree.
  树中终端节点或叶的最大数目。
- Can be defined in place of max_depth. Since binary trees are created, a depth of ‘n’ would produce a maximum of 2^n leaves.
  可在最大深度处定义。由于创建了二叉树，“n”的深度最多可生成2^n个叶。
- If this is defined, GBM will ignore max_depth.
  如果定义了这一点，GBM将忽略最大深度。
gamma [default=0] gamma[默认值=0]
- A node is split only when the resulting split gives a positive reduction in the loss function. Gamma specifies the minimum loss reduction required to make a split.
  只有当所产生的拆分使损失函数正减少时，才会拆分节点。gamma指定进行分割所需的最小损失减少。
- Makes the algorithm conservative. The values can vary depending on the loss function and should be tuned.
  使算法保守。这些值可能因损失函数而变化，应进行调整。
max_delta_step [default=0] 最大增量步进[默认值=0]
- In maximum delta step we allow each tree’s weight estimation to be. If the value is set to 0, it means there is no constraint. If it is set to a positive value, it can help making the update step more conservative.
  在最大增量步骤中，我们允许每棵树的权重估计为。如果该值设置为0，则表示没有约束。如果将其设置为正值，将有助于使更新步骤更加保守。
- Usually this parameter is not needed, but it might help in logistic regression when class is extremely imbalanced.
  通常不需要这个参数，但当类极不平衡时，它可能有助于逻辑回归。
- This is generally not used but you can explore further if you wish.
  这通常不使用，但如果您愿意，您可以进一步探索。
subsample [default=1] 子样本[默认值=1]
- Same as the subsample of GBM. Denotes the fraction of observations to be randomly samples for each tree.
  与GBM的子样本相同。表示每棵树随机采样的观测分数。
- Lower values make the algorithm more conservative and prevents overfitting but too small values might lead to under-fitting.
  值越小，算法越保守，防止过拟合，但过小的值可能导致过拟合。
- Typical values: 0.5-1
  典型值：0.5-1
colsample_bytree [default=1] colsample_bytree [默认值=1]
- Similar to max_features in GBM. Denotes the fraction of columns to be randomly samples for each tree.
  类似于GBM中的max_功能。表示每棵树随机采样的列的分数。
- Typical values: 0.5-1
  典型值：0.5-1
colsample_bylevel [default=1] colsample_bylevel [默认值=1]
- Denotes the subsample ratio of columns for each split, in each level.
  表示每个级别中每个拆分的列的子样本比率。
- I don’t use this often because subsample and colsample_bytree will do the job for you. but you can explore further if you feel so.
  我不经常使用这个，因为Subsample和Colsample Bytree将为您完成这项工作。但是如果您觉得这样，您可以进一步探索。
lambda [default=1] lambda[默认值=1]
- L2 regularization term on weights (analogous to Ridge regression)
  权的L2正则化项（类似于岭回归）
- This used to handle the regularization part of XGBoost. Though many data scientists don’t use it often, it should be explored to reduce overfitting.
  用于处理xgboost的正则化部分。虽然许多数据科学家不经常使用它，但应该探索它来减少过度拟合。
alpha [default=0] alpha[默认值=0]
- L1 regularization term on weight (analogous to Lasso regression)
  L1重量上的正则化项（类似于lasso回归）
- Can be used in case of very high dimensionality so that the algorithm runs faster when implemented
  可以在高维情况下使用，以便算法在实现时运行更快
scale_pos_weight [default=1] scale_pos_weight [默认值=1]
- A value greater than 0 should be used in case of high class imbalance as it helps in faster convergence.
  大于0的值应用于高级不平衡的情况，因为它有助于更快的收敛。

学习任务参数/Learning Task Parameters

These parameters are used to define the optimization objective the metric to be calculated at each step.
这些参数用于定义优化目标，即在每个步骤中要计算的度量。

objective [default=reg:linear] 目标[默认值=reg：线性]
- This defines the loss function to be minimized. Mostly used values are:
  这定义了要最小化的损失函数。最常用的值是：
  - binary:logistic –logistic regression for binary classification, returns predicted probability (not class)
    二进制：logistic–logistic回归用于二进制分类，返回预测概率（非类别）
  - multi:softmax –multiclass classification using the softmax objective, returns predicted class (not probabilities)
    multi:softmax–使用softmax目标的多类分类，返回预测类（不是概率）
    - you also need to set an additional num_class (number of classes) parameter defining the number of unique classes
      您还需要设置一个额外的num_class（类数）参数来定义唯一类的数目。
  - multi:softprob –same as softmax, but returns predicted probability of each data point belonging to each class.
    multi:softprob–与softmax相同，但返回属于每个类的每个数据点的预测概率。
eval_metric [ default according to objective ] 评估指标[根据目标默认]
- The metric to be used for validation data.
  用于验证数据的度量。
- The default values are rmse for regression and error for classification.
  默认值为回归的RMSE和分类的错误。
- Typical values are: 典型值为：
  - rmse – root mean square error rmse–均方根误差
  - mae – mean absolute error 平均绝对误差
  - logloss – negative log-likelihood 对数损失–负对数可能性
  - error – Binary classification error rate (0.5 threshold) 错误–二进制分类错误率（0.5阈值）
  - merror – Multiclass classification error rate 多类分类错误率
  - mlogloss – Multiclass logloss 多类日志损失
  - auc: Area under the curve AUC：曲线下面积
seed [default=0] 种子[默认值=0]
- The random number seed.
  随机数种子。
- Can be used for generating reproducible results and also for parameter tuning.
  可用于生成可重复的结果，也可用于参数调整。

If you’ve been using Scikit-Learn till now, these parameter names might not look familiar. A good news is that xgboost module in python has an sklearn wrapper called XGBClassifier. It uses sklearn style naming convention. The parameters names which will change are:
如果您到目前为止一直在使用scikit-learn，这些参数名称可能看起来不熟悉。一个好消息是，python中的xgboost模块有一个名为xgbclassifier的sklearn包装器。它使用sklearn样式命名约定。将更改的参数名称为：

eta –> learning_rate
lambda –> reg_lambda
alpha –> reg_alpha

You must be wondering that we have defined everything except something similar to the “n_estimators” parameter in GBM. Well this exists as a parameter in XGBClassifier. However, it has to be passed as “num_boosting_rounds” while calling the fit function in the standard xgboost implementation.
您一定想知道，除了类似于GBM中的“n_Estimators”参数之外，我们已经定义了所有内容。这在XGBClassifier中作为一个参数存在。但是，在标准xgboost实现中调用fit函数时，必须将其作为“num-booting-rounds”传递。

I recommend you to go through the following parts of xgboost guide to better understand the parameters and codes:
我建议您仔细阅读xgboost指南的以下部分，以便更好地了解参数和代码：

XGBoost Parameters (official guide) xgboost参数（官方指南）
XGBoost Demo Codes (xgboost GitHub repository) xgboost演示代码（xgboost github存储库）
Python API Reference (official guide) python api参考（官方指南）

码农公寓

2. xgboost参数/XGBoost Parameters

一般参数/General Parameters

Booster参数/Booster Parameters

​​​​​​​学习任务参数/Learning Task Parameters

相关文章

学习任务参数/Learning Task Parameters