前言
- 本文基于孕妇吸烟与胎儿问题中数据集与前期处理
- 针对随机森林与LGBM模型网格搜索效率低，使用贝叶斯调参提高效率
- 有关于贝叶斯优化包相关参数说明详解可以看GitHub地址
- 将处理好的数据用
dill包进行封装，大家在尝试运行时，需要安装dill包 - 数据文件下载地址
数据导入
- 基于
jupyter notebook的魔术命令，如果不在jupyter notebook中运行，请将%号去掉
# 如果已经安装过dill包可以不要下面的魔术命令
%pip install dill
import dill
# 如果执行这步出错，请检查错误信息
# 是否有包未安装，比如bayes_opt包或lightgbm包
dill.load_session('C:/Users/lenovo/Desktop/data.pkl')
# 若未报错，则数据导入成功
- 基于
jupyter notebook的魔术命令，如果不在jupyter notebook中运行，请将%号去掉
%pip install bayesian-optimization
随机森林贝叶斯调参
导入包
# 贝叶斯调参优化随机森林
from sklearn.model_selection import cross_val_score
from bayes_opt import BayesianOptimization
构造黑盒函数
- 构造黑盒函数，即需要优化的目标，这里我选取的是10折交叉检验的分数值。当然你也可以在
RandomForestRegressor()后使用.fit(train_x, train_y).score(test_x, test_y)将模型在测试集上表现作为最优化目标 - 因为
bayes_opt库只支持最大值，所以最后的输出如果是求最小值，那么需要在前面加上负号，以转为最大值。这里使用neg_mean_squared_error作为最大化目标 - 由于
bayes优化只能优化连续超参数，因此要加上int()转为离散超参数
# 函数中包含需要调整的参数
def rf_cv(n_estimators, min_samples_split, max_features, max_depth, min_samples_leaf, max_leaf_nodes):
# 交叉检验，得到的评分为贝叶斯调参优化目标
val = cross_val_score(
# 由于bayes优化只能优化连续超参数，因此要加上int()转为离散超参数
RandomForestRegressor(n_estimators=int(n_estimators),
min_samples_split=int(min_samples_split),
min_samples_leaf = int(min_samples_leaf),
max_leaf_nodes = int(max_leaf_nodes),
max_features=int(max_features)
max_depth=int(max_depth),
random_state=42,
oob_score = 'True'),
X_train, y_train, scoring='neg_mean_squared_error', cv=10
).mean()
return val
确定域空间
# 规定各参数搜索范围
rf_bo = BayesianOptimization(rf_cv,
{'n_estimators': (800, 1500),
'min_samples_split': (2, 20),
'max_features': (1, 6),
'max_depth': (3, 10),
'min_samples_leaf' : (2,20),
'max_leaf_nodes':(10,40)})
rf_bo.maximize()
输出：
| iter | target | max_depth | max_fe... | max_le... | min_sa... | min_sa... | n_esti... |
-------------------------------------------------------------------------------------------------
| 1 | -246.0 | 4.341 | 3.478 | 27.05 | 2.973 | 16.09 | 882.3 |
| 2 | -264.1 | 4.023 | 1.761 | 31.94 | 19.03 | 5.529 | 1.338e+0 |
| 3 | -245.4 | 5.505 | 4.8 | 23.88 | 4.304 | 12.89 | 1.147e+0 |
| 4 | -247.3 | 9.276 | 2.907 | 33.51 | 7.78 | 6.09 | 1.492e+0 |
| 5 | -257.1 | 6.966 | 1.406 | 25.3 | 14.24 | 11.24 | 925.4 |
| 6 | -245.3 | 5.946 | 4.281 | 26.73 | 5.724 | 13.45 | 1.147e+0 |
| 7 | -250.0 | 3.0 | 6.0 | 29.3 | 2.0 | 20.0 | 826.7 |
| 8 | -272.6 | 3.0 | 1.0 | 38.17 | 20.0 | 2.0 | 1.097e+0 |
| 9 | -248.6 | 8.698 | 5.645 | 23.75 | 2.0 | 20.0 | 1.182e+0 |
| 10 | -264.9 | 10.0 | 1.0 | 10.0 | 20.0 | 2.0 | 858.3 |
| 11 | -272.4 | 3.0 | 1.0 | 17.23 | 20.0 | 2.0 | 1.165e+0 |
| 12 | -247.0 | 4.871 | 5.969 | 31.59 | 2.493 | 16.71 | 1.134e+0 |
| 13 | -253.0 | 10.0 | 6.0 | 40.0 | 2.0 | 20.0 | 893.9 |
| 14 | -247.9 | 8.938 | 5.776 | 10.13 | 3.715 | 19.55 | 1.129e+0 |
| 15 | -246.0 | 8.643 | 4.938 | 16.8 | 12.07 | 17.0 | 1.48e+03 |
| 16 | -272.3 | 3.046 | 1.13 | 30.6 | 20.0 | 4.148 | 1.47e+03 |
| 17 | -247.3 | 10.0 | 5.957 | 19.54 | 4.089 | 17.66 | 1.495e+0 |
| 18 | -248.7 | 6.204 | 3.098 | 10.2 | 18.7 | 11.91 | 1.495e+0 |
| 19 | -249.2 | 3.0 | 5.068 | 39.19 | 2.0 | 20.0 | 865.9 |
| 20 | -248.0 | 6.247 | 5.639 | 30.41 | 16.99 | 19.59 | 1.499e+0 |
| 21 | -252.9 | 10.0 | 6.0 | 40.0 | 2.0 | 20.0 | 1.161e+0 |
| 22 | -249.2 | 10.0 | 6.0 | 22.95 | 19.86 | 20.0 | 1.136e+0 |
| 23 | -251.1 | 10.0 | 6.0 | 29.33 | 2.0 | 20.0 | 1.207e+0 |
| 24 | -265.7 | 10.0 | 1.0 | 10.0 | 2.0 | 2.0 | 1.487e+0 |
| 25 | -248.4 | 9.63 | 5.778 | 22.09 | 2.579 | 2.131 | 1.132e+0 |
| 26 | -250.4 | 3.485 | 5.665 | 34.65 | 19.73 | 17.61 | 881.4 |
| 27 | -248.1 | 3.569 | 4.336 | 11.09 | 7.701 | 17.22 | 892.6 |
| 28 | -272.5 | 3.0 | 1.0 | 22.71 | 2.0 | 2.0 | 892.6 |
| 29 | -248.5 | 6.826 | 2.856 | 17.2 | 10.92 | 19.96 | 879.0 |
| 30 | -247.6 | 6.9 | 4.121 | 39.17 | 3.153 | 19.32 | 1.49e+03 |
=================================================================================================
- 可以看到当迭代到30次时，就已经找到了较为理想的参数
最优参
# 使用max得到最优参
rf_bo.max
输出：
{'target': -245.31420890294567,
'params': {'max_depth': 5.946464811467048,
'max_features': 4.280789636350939,
'max_leaf_nodes': 26.726081689294052,
'min_samples_leaf': 5.724298965332711,
'min_samples_split': 13.447191575489041,
'n_estimators': 1147.3188404305388}}
- 使用最优参(记得该取整的要取整)再次训练模型，得到相对理想的模型。
使用最优参训练模型
# 基于sklearn封装的随机森林
from sklearn.ensemble import RandomForestRegressor
# 在RandomForestRegressor()括号中填入最优参
# 这里以上面我得到的最优参为例
rf_reg = RandomForestRegressor(max_depth = 5,
max_features = 4,
max_leaf_nodes = 26,
min_samples_leaf = 5,
min_samples_split = 13,
n_estimators = 1147)
# 在训练集上训练模型
rf_reg.fit(X_train, y_train)
# 查看模型在训练集上均方误差
print('train_mes : {:.3f}'.format(mean_squared_error(y_train, rf_reg.predict(X_train))))
LGBM贝叶斯调参
- 同样的，先构造黑箱函数，然后确定域空间，唯一不同的这次我们规定随机搜索的步数和贝叶斯优化的步数。这样会增加搜索的时间，但也许能比默认步数搜索到更好的参数
LGBM模型中L1、L2正则化等参数允许为浮点类型，所以不需要使用int()进行类型转换。
def lgbm_cv(n_estimators,max_depth,num_leaves,min_child_samples,reg_alpha,reg_lambda,subsample,colsample_bytree):
val = cross_val_score(
LGBMRegressor(learning_rate = 0.001,
n_estimators=int(n_estimators),
max_depth=int(max_depth),
num_leaves = int(num_leaves),
min_child_samples = int(min_child_samples),
reg_alpha = reg_alpha, # float
reg_lambda = reg_lambda,
subsample = subsample,
colsample_bytree = colsample_bytree),
X_train, y_train, scoring='neg_mean_squared_error', cv=10).mean()
return val
lgbm_bo = BayesianOptimization(lgbm_cv,
{'n_estimators': (1500, 3000),
'max_depth': (2, 10),
'num_leaves': (5, 20),
'min_child_samples': (3, 100),
'reg_alpha' : (0.1,1),
'reg_lambda':(0.001,1),
'subsample':(0.8,1),
'colsample_bytree':(0.8,1)})
确定迭代次数
- 迭代次数由两部分组成，随机搜索的步数和贝叶斯优化的步数，贝叶斯优化的步数要多一点，步骤越多，就越有可能找到一个好的最大值。随机探索可以通过扩大探索空间而有所帮助。
# n_iter贝叶斯优化次数,init_points随机优化次数
lgbm_bo.maximize(n_iter = 100,init_points = 50)
输出：
| iter | target | colsam... | max_depth | min_ch... | n_esti... | num_le... | reg_alpha | reg_la... | subsample |
-------------------------------------------------------------------------------------------------------------------------
| 1 | -251.5 | 0.9299 | 6.878 | 82.45 | 2.147e+0 | 11.21 | 0.3167 | 0.2107 | 0.8781 |
| 2 | -252.4 | 0.8555 | 4.824 | 58.81 | 2.055e+0 | 11.71 | 0.7608 | 0.2855 | 0.8864 |
| 3 | -249.1 | 0.96 | 7.997 | 14.63 | 2.232e+0 | 19.55 | 0.1661 | 0.6375 | 0.8603 |
| 4 | -249.4 | 0.9018 | 8.336 | 20.44 | 2.155e+0 | 18.32 | 0.6761 | 0.1784 | 0.9618 |
| 5 | -249.9 | 0.8464 | 8.894 | 76.6 | 2.907e+0 | 15.3 | 0.1962 | 0.9202 | 0.9374 |
| 6 | -256.1 | 0.9289 | 9.386 | 70.46 | 1.546e+0 | 7.014 | 0.4163 | 0.9269 | 0.8498 |
| 7 | -250.1 | 0.8198 | 5.917 | 22.09 | 1.997e+0 | 8.863 | 0.2419 | 0.08677 | 0.8001 |
| 8 | -250.1 | 0.84 | 4.637 | 25.22 | 2.121e+0 | 13.66 | 0.7642 | 0.04577 | 0.9546 |
| 9 | -248.0 | 0.8454 | 8.498 | 15.58 | 2.894e+0 | 18.63 | 0.1802 | 0.4565 | 0.8858 |
| 10 | -255.2 | 0.8839 | 7.783 | 53.08 | 1.979e+0 | 5.731 | 0.4185 | 0.5306 | 0.9554 |
| 11 | -249.2 | 0.944 | 4.216 | 88.81 | 2.929e+0 | 10.36 | 0.5145 | 0.1131 | 0.9041 |
| 12 | -247.2 | 0.8847 | 5.88 | 12.0 | 2.059e+0 | 13.3 | 0.6635 | 0.09008 | 0.9531 |
| 13 | -250.2 | 0.8651 | 3.733 | 50.73 | 2.875e+0 | 16.17 | 0.542 | 0.847 | 0.9391 |
| 14 | -251.2 | 0.8654 | 8.812 | 92.12 | 2.522e+0 | 12.46 | 0.3778 | 0.6421 | 0.9782 |
| 15 | -251.3 | 0.9663 | 6.259 | 45.49 | 2.908e+0 | 17.26 | 0.3127 | 0.04984 | 0.9103 |
| 16 | -248.5 | 0.8887 | 6.157 | 79.92 | 2.83e+03 | 7.47 | 0.7573 | 0.1109 | 0.8294 |
| 17 | -254.0 | 0.9811 | 4.113 | 83.44 | 1.704e+0 | 8.561 | 0.787 | 0.1622 | 0.9373 |
| 18 | -250.7 | 0.9525 | 6.545 | 94.2 | 2.758e+0 | 8.907 | 0.6938 | 0.3916 | 0.9476 |
| 19 | -251.0 | 0.9972 | 6.941 | 46.37 | 2.577e+0 | 12.63 | 0.7774 | 0.9983 | 0.8211 |
| 20 | -246.4 | 0.8041 | 3.084 | 3.125 | 2.505e+0 | 17.59 | 0.4361 | 0.7433 | 0.9699 |
| 21 | -249.1 | 0.8621 | 8.701 | 65.99 | 2.938e+0 | 19.25 | 0.9019 | 0.2082 | 0.8993 |
| 22 | -249.5 | 0.9085 | 3.629 | 23.46 | 2.138e+0 | 12.78 | 0.99 | 0.6435 | 0.9545 |
| 23 | -259.9 | 0.8006 | 2.359 | 33.99 | 1.649e+0 | 16.87 | 0.9538 | 0.7214 | 0.8083 |
| 24 | -253.8 | 0.9543 | 9.966 | 76.27 | 1.817e+0 | 15.48 | 0.953 | 0.4465 | 0.8373 |
| 25 | -250.0 | 0.9024 | 9.932 | 87.61 | 2.563e+0 | 17.52 | 0.6365 | 0.3728 | 0.9358 |
| 26 | -247.7 | 0.9368 | 9.499 | 26.12 | 2.618e+0 | 6.258 | 0.5139 | 0.01136 | 0.8822 |
| 27 | -256.5 | 0.8403 | 2.212 | 72.82 | 1.975e+0 | 9.344 | 0.7336 | 0.9867 | 0.986 |
| 28 | -250.4 | 0.8356 | 6.36 | 25.71 | 2.362e+0 | 12.92 | 0.9792 | 0.1116 | 0.9806 |
| 29 | -250.4 | 0.967 | 6.722 | 72.78 | 2.598e+0 | 11.43 | 0.3715 | 0.7805 | 0.9167 |
| 30 | -253.8 | 0.9365 | 2.325 | 42.29 | 2.333e+0 | 6.774 | 0.7346 | 0.5393 | 0.9785 |
| 31 | -252.4 | 0.984 | 7.318 | 44.95 | 1.856e+0 | 11.53 | 0.364 | 0.06497 | 0.9884 |
| 32 | -255.7 | 0.895 | 4.489 | 41.63 | 1.858e+0 | 5.682 | 0.1547 | 0.7007 | 0.9882 |
| 33 | -248.9 | 0.9449 | 7.396 | 32.28 | 2.355e+0 | 14.05 | 0.8676 | 0.9371 | 0.8904 |
| 34 | -251.9 | 0.8728 | 7.131 | 58.31 | 2.399e+0 | 13.46 | 0.8022 | 0.9378 | 0.9732 |
| 35 | -252.0 | 0.859 | 5.741 | 43.33 | 1.914e+0 | 11.67 | 0.4837 | 0.1606 | 0.9066 |
| 36 | -256.2 | 0.9248 | 3.282 | 44.33 | 1.549e+0 | 17.83 | 0.2039 | 0.1553 | 0.8587 |
| 37 | -250.0 | 0.8103 | 5.894 | 43.89 | 2.872e+0 | 11.2 | 0.9715 | 0.1911 | 0.8768 |
| 38 | -247.4 | 0.9543 | 4.029 | 29.43 | 2.807e+0 | 18.67 | 0.6966 | 0.3294 | 0.8734 |
| 39 | -251.7 | 0.8035 | 9.248 | 49.61 | 2.935e+0 | 19.69 | 0.4769 | 0.7963 | 0.8527 |
| 40 | -253.8 | 0.8426 | 7.232 | 73.47 | 1.953e+0 | 9.829 | 0.4052 | 0.6666 | 0.8147 |
| 41 | -253.7 | 0.8878 | 9.484 | 77.37 | 1.981e+0 | 10.31 | 0.9634 | 0.37 | 0.8724 |
| 42 | -248.7 | 0.837 | 5.001 | 5.432 | 1.895e+0 | 10.73 | 0.4941 | 0.07088 | 0.9292 |
| 43 | -252.1 | 0.8235 | 2.448 | 9.134 | 2.106e+0 | 16.9 | 0.8737 | 0.3368 | 0.8999 |
| 44 | -252.3 | 0.927 | 7.326 | 91.65 | 2.058e+0 | 11.56 | 0.7419 | 0.4703 | 0.8103 |
| 45 | -250.4 | 0.9256 | 7.025 | 64.44 | 2.401e+0 | 16.9 | 0.107 | 0.7307 | 0.8434 |
| 46 | -248.8 | 0.9423 | 3.798 | 29.89 | 2.524e+0 | 12.89 | 0.7705 | 0.741 | 0.9162 |
| 47 | -258.6 | 0.9142 | 5.767 | 76.21 | 1.505e+0 | 6.561 | 0.2224 | 0.2147 | 0.9764 |
| 48 | -252.4 | 0.9271 | 8.811 | 6.559 | 1.796e+0 | 5.845 | 0.542 | 0.8878 | 0.9614 |
| 49 | -258.8 | 0.8216 | 7.332 | 98.04 | 1.61e+03 | 5.039 | 0.4765 | 0.09902 | 0.8323 |
| 50 | -251.2 | 0.8809 | 3.795 | 97.97 | 2.588e+0 | 5.977 | 0.9903 | 0.1333 | 0.8626 |
| 51 | -252.7 | 0.8713 | 3.811 | 55.85 | 2.138e+0 | 17.5 | 0.527 | 0.4825 | 0.8298 |
| 52 | -255.0 | 0.8587 | 9.695 | 56.51 | 1.657e+0 | 14.53 | 0.6904 | 0.6378 | 0.9473 |
| 53 | -254.0 | 0.971 | 2.733 | 91.54 | 2.319e+0 | 5.299 | 0.6559 | 0.9105 | 0.9984 |
| 54 | -252.5 | 0.915 | 9.869 | 92.64 | 2.222e+0 | 10.35 | 0.9431 | 0.4694 | 0.821 |
| 55 | -251.9 | 0.9004 | 7.716 | 73.5 | 2.288e+0 | 11.8 | 0.4452 | 0.4667 | 0.9693 |
| 56 | -251.1 | 0.9678 | 4.192 | 96.36 | 2.536e+0 | 14.14 | 0.1397 | 0.04324 | 0.8681 |
| 57 | -253.8 | 0.8579 | 4.479 | 72.29 | 1.916e+0 | 17.29 | 0.8099 | 0.4709 | 0.9209 |
| 58 | -251.7 | 0.8303 | 5.407 | 92.42 | 2.351e+0 | 9.32 | 0.1488 | 0.5252 | 0.8027 |
| 59 | -250.2 | 0.8626 | 9.889 | 37.49 | 2.543e+0 | 11.68 | 0.3817 | 0.1709 | 0.8627 |
| 60 | -250.7 | 0.9972 | 8.553 | 32.94 | 1.672e+0 | 15.26 | 0.3957 | 0.1086 | 0.8383 |
| 61 | -251.9 | 0.8949 | 5.338 | 26.2 | 1.942e+0 | 7.799 | 0.8794 | 0.1337 | 0.8964 |
| 62 | -246.9 | 0.8566 | 3.851 | 16.47 | 2.076e+0 | 18.73 | 0.5352 | 0.4207 | 0.9956 |
| 63 | -245.1 | 0.9937 | 4.547 | 3.305 | 2.867e+0 | 7.185 | 0.8233 | 0.4205 | 0.9417 |
| 64 | -250.5 | 0.9768 | 8.349 | 85.95 | 2.332e+0 | 14.78 | 0.3782 | 0.8485 | 0.9663 |
| 65 | -250.4 | 0.8401 | 9.422 | 32.77 | 2.428e+0 | 18.25 | 0.7231 | 0.7265 | 0.8794 |
| 66 | -247.4 | 0.9794 | 4.493 | 8.738 | 2.042e+0 | 16.57 | 0.6579 | 0.6725 | 0.844 |
| 67 | -248.6 | 0.946 | 4.873 | 34.24 | 2.044e+0 | 16.74 | 0.3182 | 0.2692 | 0.8474 |
| 68 | -254.3 | 0.8538 | 5.687 | 79.42 | 1.824e+0 | 18.33 | 0.7764 | 0.6676 | 0.8872 |
| 69 | -255.9 | 0.8359 | 8.635 | 76.64 | 1.688e+0 | 9.01 | 0.6837 | 0.9395 | 0.9813 |
| 70 | -252.5 | 0.9696 | 9.777 | 42.32 | 2.846e+0 | 14.44 | 0.4389 | 0.1839 | 0.9697 |
| 71 | -251.0 | 0.8293 | 7.864 | 97.97 | 2.745e+0 | 15.07 | 0.4306 | 0.3661 | 0.8145 |
| 72 | -254.7 | 0.844 | 2.044 | 82.31 | 2.124e+0 | 12.8 | 0.9643 | 0.9277 | 0.8329 |
| 73 | -248.6 | 0.9647 | 4.654 | 79.34 | 2.879e+0 | 14.25 | 0.3899 | 0.4517 | 0.8848 |
| 74 | -253.5 | 0.9759 | 9.409 | 34.56 | 1.831e+0 | 6.315 | 0.2175 | 0.9694 | 0.9046 |
| 75 | -249.8 | 0.9993 | 3.751 | 53.76 | 2.705e+0 | 18.52 | 0.9932 | 0.3771 | 0.9974 |
| 76 | -248.2 | 0.8568 | 7.918 | 26.47 | 2.832e+0 | 9.567 | 0.3081 | 0.2001 | 0.9637 |
| 77 | -250.6 | 0.9161 | 9.572 | 21.78 | 2.08e+03 | 17.8 | 0.659 | 0.2496 | 0.8105 |
| 78 | -256.7 | 0.8722 | 5.862 | 96.38 | 1.661e+0 | 17.3 | 0.8781 | 0.3713 | 0.8406 |
| 79 | -249.6 | 0.9668 | 3.886 | 67.11 | 2.591e+0 | 6.384 | 0.2177 | 0.1065 | 0.9837 |
| 80 | -249.0 | 0.8911 | 5.993 | 29.05 | 2.946e+0 | 13.13 | 0.902 | 0.1512 | 0.9985 |
| 81 | -249.2 | 0.8635 | 3.71 | 43.96 | 2.767e+0 | 18.02 | 0.9057 | 0.6716 | 0.8505 |
| 82 | -253.0 | 0.9688 | 3.429 | 79.2 | 2.066e+0 | 6.202 | 0.9512 | 0.6868 | 0.8608 |
| 83 | -250.9 | 0.952 | 4.57 | 75.44 | 2.293e+0 | 17.13 | 0.6878 | 0.9675 | 0.8407 |
| 84 | -254.5 | 0.9119 | 3.122 | 82.4 | 1.808e+0 | 12.66 | 0.9125 | 0.5803 | 0.9666 |
| 85 | -257.6 | 0.9048 | 3.93 | 95.5 | 1.624e+0 | 10.92 | 0.6581 | 0.2187 | 0.9817 |
| 86 | -257.4 | 0.8564 | 2.41 | 22.95 | 1.807e+0 | 8.449 | 0.3729 | 0.761 | 0.8282 |
| 87 | -248.5 | 0.8595 | 4.704 | 5.026 | 1.914e+0 | 17.77 | 0.5784 | 0.7281 | 0.8922 |
| 88 | -248.6 | 0.9925 | 2.062 | 13.15 | 2.681e+0 | 12.09 | 0.1668 | 0.6771 | 0.8767 |
| 89 | -247.2 | 0.9495 | 8.553 | 6.885 | 2.643e+0 | 14.56 | 0.8508 | 0.2082 | 0.9777 |
| 90 | -249.9 | 0.8889 | 5.049 | 75.7 | 2.95e+03 | 19.67 | 0.5517 | 0.6659 | 0.9488 |
| 91 | -252.1 | 0.8089 | 5.886 | 56.19 | 2.138e+0 | 7.085 | 0.1107 | 0.08426 | 0.8332 |
| 92 | -250.7 | 0.8242 | 8.274 | 80.49 | 2.32e+03 | 6.095 | 0.5902 | 0.6058 | 0.9501 |
| 93 | -257.2 | 0.9934 | 2.024 | 65.59 | 1.808e+0 | 8.934 | 0.1617 | 0.9746 | 0.8435 |
| 94 | -250.5 | 0.8985 | 2.545 | 84.7 | 2.795e+0 | 19.0 | 0.1595 | 0.3708 | 0.8454 |
| 95 | -250.3 | 0.8932 | 8.764 | 47.66 | 2.538e+0 | 19.69 | 0.5193 | 0.4982 | 0.8129 |
| 96 | -251.0 | 0.9709 | 7.095 | 48.09 | 2.861e+0 | 11.9 | 0.3469 | 0.7833 | 0.9071 |
| 97 | -252.4 | 0.9348 | 8.922 | 77.9 | 2.059e+0 | 15.69 | 0.8596 | 0.5381 | 0.9972 |
| 98 | -253.6 | 0.9101 | 2.922 | 11.67 | 1.967e+0 | 15.83 | 0.9068 | 0.7492 | 0.8623 |
| 99 | -250.2 | 0.8583 | 8.64 | 89.44 | 2.598e+0 | 18.2 | 0.1822 | 0.4276 | 0.8993 |
| 100 | -246.6 | 0.8264 | 5.264 | 10.81 | 2.47e+03 | 7.567 | 0.6361 | 0.3497 | 0.8002 |
| 101 | -250.1 | 0.8995 | 3.704 | 52.65 | 2.985e+0 | 19.72 | 0.11 | 0.1371 | 0.8817 |
| 102 | -248.8 | 0.8594 | 5.012 | 4.972 | 2.131e+0 | 8.508 | 0.3815 | 0.945 | 0.9192 |
| 103 | -247.6 | 0.9669 | 3.601 | 29.7 | 2.932e+0 | 18.49 | 0.3248 | 0.7928 | 0.8384 |
| 104 | -256.3 | 0.9432 | 9.293 | 54.1 | 1.761e+0 | 5.085 | 0.3883 | 0.5722 | 0.9298 |
| 105 | -257.2 | 0.9938 | 2.443 | 84.29 | 1.823e+0 | 18.0 | 0.3708 | 0.7673 | 0.9651 |
| 106 | -252.1 | 0.9587 | 8.62 | 42.49 | 2.116e+0 | 19.23 | 0.7527 | 0.4659 | 0.9647 |
| 107 | -253.4 | 0.8556 | 9.788 | 62.2 | 1.972e+0 | 19.38 | 0.9473 | 0.1454 | 0.8905 |
| 108 | -253.2 | 0.8099 | 7.696 | 50.18 | 2.162e+0 | 15.46 | 0.6554 | 0.827 | 0.8181 |
| 109 | -251.1 | 0.8175 | 8.761 | 13.57 | 1.69e+03 | 8.422 | 0.8813 | 0.9929 | 0.9977 |
| 110 | -251.1 | 0.8626 | 6.849 | 92.65 | 2.571e+0 | 8.158 | 0.758 | 0.8469 | 0.9977 |
| 111 | -252.6 | 0.8674 | 7.393 | 27.39 | 1.724e+0 | 18.88 | 0.7886 | 0.4438 | 0.8879 |
| 112 | -251.1 | 0.9263 | 4.146 | 78.4 | 2.257e+0 | 14.98 | 0.2221 | 0.145 | 0.9518 |
| 113 | -247.5 | 0.8534 | 6.048 | 10.58 | 2.222e+0 | 8.573 | 0.4608 | 0.3796 | 0.99 |
| 114 | -246.1 | 0.8489 | 3.597 | 6.778 | 2.394e+0 | 9.335 | 0.5718 | 0.1133 | 0.8493 |
| 115 | -250.9 | 0.9191 | 7.406 | 27.09 | 2.777e+0 | 19.52 | 0.6142 | 0.7267 | 0.8041 |
| 116 | -248.9 | 0.8929 | 7.28 | 38.2 | 2.463e+0 | 8.195 | 0.4663 | 0.3756 | 0.8455 |
| 117 | -252.0 | 0.9764 | 8.82 | 12.13 | 1.569e+0 | 7.788 | 0.7776 | 0.3066 | 0.8528 |
| 118 | -254.7 | 0.9845 | 6.606 | 88.3 | 1.692e+0 | 15.52 | 0.1237 | 0.8686 | 0.9593 |
| 119 | -249.9 | 0.9242 | 4.519 | 52.74 | 2.537e+0 | 19.79 | 0.4742 | 0.5048 | 0.9595 |
| 120 | -254.5 | 0.8294 | 4.186 | 32.16 | 1.826e+0 | 6.485 | 0.4382 | 0.6719 | 0.9082 |
| 121 | -257.4 | 0.8519 | 7.839 | 87.22 | 1.519e+0 | 13.29 | 0.3876 | 0.01006 | 0.9211 |
| 122 | -253.1 | 0.8557 | 4.863 | 99.88 | 2.196e+0 | 6.027 | 0.2864 | 0.08695 | 0.8477 |
| 123 | -255.3 | 0.9123 | 3.487 | 58.82 | 1.743e+0 | 16.01 | 0.981 | 0.2148 | 0.9191 |
| 124 | -252.9 | 0.8671 | 6.105 | 34.59 | 1.822e+0 | 7.959 | 0.5157 | 0.4601 | 0.9288 |
| 125 | -251.6 | 0.823 | 4.232 | 4.69 | 1.63e+03 | 17.58 | 0.3137 | 0.896 | 0.8394 |
| 126 | -251.1 | 0.9605 | 9.27 | 3.885 | 1.594e+0 | 9.155 | 0.3988 | 0.04495 | 0.8383 |
| 127 | -251.1 | 0.8553 | 8.43 | 92.19 | 2.549e+0 | 15.68 | 0.4312 | 0.3145 | 0.8269 |
| 128 | -253.7 | 0.8167 | 6.511 | 63.97 | 1.761e+0 | 18.79 | 0.4067 | 0.4545 | 0.8913 |
| 129 | -250.5 | 0.9509 | 3.603 | 86.18 | 2.351e+0 | 6.903 | 0.858 | 0.1375 | 0.887 |
| 130 | -251.3 | 0.8577 | 7.442 | 58.83 | 2.718e+0 | 18.86 | 0.3148 | 0.3542 | 0.8545 |
| 131 | -252.5 | 0.8522 | 5.962 | 44.5 | 1.682e+0 | 17.12 | 0.1616 | 0.242 | 0.8918 |
| 132 | -256.4 | 0.9798 | 4.611 | 95.35 | 1.608e+0 | 9.765 | 0.6692 | 0.7236 | 0.8428 |
| 133 | -251.3 | 0.9747 | 8.909 | 39.93 | 2.322e+0 | 13.09 | 0.7278 | 0.2057 | 0.9541 |
| 134 | -258.2 | 0.898 | 2.558 | 77.46 | 1.824e+0 | 17.6 | 0.184 | 0.882 | 0.8166 |
| 135 | -250.7 | 0.9765 | 8.294 | 36.04 | 1.986e+0 | 15.15 | 0.1793 | 0.0547 | 0.8536 |
| 136 | -256.1 | 0.8067 | 8.756 | 78.74 | 1.653e+0 | 9.253 | 0.4187 | 0.356 | 0.8482 |
| 137 | -255.0 | 0.9928 | 6.42 | 93.78 | 1.784e+0 | 18.29 | 0.3528 | 0.7969 | 0.9533 |
| 138 | -250.7 | 0.9377 | 9.27 | 93.17 | 2.997e+0 | 13.43 | 0.2546 | 0.3083 | 0.8694 |
| 139 | -259.6 | 0.8726 | 8.204 | 78.49 | 1.566e+0 | 5.512 | 0.6164 | 0.7091 | 0.8871 |
| 140 | -253.0 | 0.9226 | 2.858 | 49.18 | 2.565e+0 | 14.22 | 0.7601 | 0.5902 | 0.8829 |
| 141 | -255.2 | 0.8894 | 2.576 | 54.63 | 2.112e+0 | 16.73 | 0.9314 | 0.9692 | 0.9501 |
| 142 | -251.9 | 0.8662 | 2.44 | 42.66 | 2.677e+0 | 7.367 | 0.5093 | 0.6031 | 0.8568 |
| 143 | -246.4 | 0.8408 | 5.624 | 9.557 | 2.655e+0 | 17.48 | 0.6699 | 0.01308 | 0.9373 |
| 144 | -256.3 | 0.8831 | 4.472 | 72.16 | 1.638e+0 | 8.795 | 0.748 | 0.7101 | 0.8074 |
| 145 | -254.3 | 0.9842 | 5.52 | 54.41 | 1.66e+03 | 10.72 | 0.8502 | 0.954 | 0.8083 |
| 146 | -250.6 | 0.977 | 5.445 | 60.33 | 2.612e+0 | 18.05 | 0.5285 | 0.458 | 0.9856 |
| 147 | -249.9 | 0.9103 | 5.405 | 91.39 | 2.967e+0 | 12.09 | 0.6206 | 0.8223 | 0.812 |
| 148 | -252.6 | 0.9774 | 4.257 | 42.9 | 1.731e+0 | 15.42 | 0.9746 | 0.8842 | 0.8383 |
| 149 | -254.0 | 0.858 | 2.464 | 43.46 | 2.257e+0 | 13.9 | 0.3589 | 0.9985 | 0.8966 |
| 150 | -253.4 | 0.9733 | 9.152 | 55.79 | 2.118e+0 | 5.46 | 0.8561 | 0.062 | 0.8039 |
输出最优参
lgbm_bo.max
输出：
{'target': -243.40837561015314,
'params': {'colsample_bytree': 0.9035439141128341,
'max_depth': 3.167260637404743,
'min_child_samples': 9.324254670380586,
'n_estimators': 2816.120214269747,
'num_leaves': 17.035319354824765,
'reg_alpha': 0.33257959049618924,
'reg_lambda': 0.5556596408409464,
'subsample': 0.8898942099929366}}
- 使用最优参(记得该取整的要取整)再次训练模型，得到相对理想的模型。
写在最后
- 如果大家想更好的使用贝叶斯优化包，可以读一读优化包的GitHub说明，里面有基于经验范围的精密搜索、经验函数等一些参数的调整等，或许能提升优化器性能。
|
