{
  "id": 81603,
  "title": "some question in using hyperopt ",
  "url": "/competitions/elo-merchant-category-recommendation/discussion/81603",
  "author_name": "",
  "post_date": "2019-02-23T09:47:06.710390200Z",
  "votes": 1,
  "comment_count": 5,
  "views": 0,
  "content": "<p>I want to tuning parameter by  hyperopt .Howerver , I got the lower score parameter .  </p>\n\n<h2>Ex . I got the parms by finishing run hyperopt:</h2>\n\n<p>{'colsample_bytree': 0.6761269574650649,\n 'learning_rate': 0.04519033564007213,\n 'max_depth': 0,\n 'min_child_weight': 30.199691742071565,\n 'min_data_in_leaf': 6,\n 'min_split_gain': 10.282274656774836,\n 'num_leaves': 37,\n 'other_rate': 0.027204317754823286,\n 'reg_alpha': 5.670906506486702,\n 'reg_lambda': 7.676023413606416,\n 'subsample': 0.8265019828109162,\n 'top_rate': 0.8678565632906692} <br>\n<strong>CV :3.6612530092950726</strong></p>\n\n<h2>But I find a parms beeter than it in log:</h2>\n\n<p>loss 3.654 params {'boosting': 'goss', 'colsample_bytree': 0.6761269574650649, 'learning_rate': 0.04519033564007213, 'max_depth': 8, 'metric': 'rmse', 'min_child_weight': 30.199691742071565, 'min_data_in_leaf': 26, 'min_split_gain': 10.282274656774836, 'num_leaves': 68, 'objective': 'root_mean_squared_error', 'other_rate': 0.027204317754823286, 'reg_alpha': 5.670906506486702, 'reg_lambda': 7.676023413606416, 'subsample': 0.8265019828109162, 'task': 'train', 'top_rate': 0.8678565632906692} <br>\n<strong>CV：3.654</strong></p>\n\n<h2>Here My code:</h2>\n\n<pre><code>train_X_reg=train_df[features]\ntrain_Y_reg=train_df.target\n#make scorer\nrmse_scorer = make_scorer(rmse,greater_is_better =False)\ndef hyperopt_lgb_reg(params):\n     clf =lgb.LGBMRegressor(**params)\n     return -cross_val_score(clf, train_X_reg, train_Y_reg,   scoring=rmse_scorer,cv=5).mean()\n\ndef f_reg(parms):\n  rmse_loss = hyperopt_lgb_reg(parms)\n  print(\"loss {:.3f} params {}\".format(rmse_loss, parms))\n  return {'loss': rmse_loss, 'status': STATUS_OK}\n\n space_dt_reg = {\n 'task':'train',\n'max_depth': hp.choice('max_depth', range(8,40)),\n'num_leaves': hp.choice('num_leaves', range(31,70)),\n'objective': 'root_mean_squared_error',\n'metric': 'rmse',\n'boosting': 'goss',\n'min_child_weight': hp.uniform('min_child_weight', 20,50),\n'subsample': hp.uniform('subsample', 0.8,1),\n'reg_lambda':hp.uniform('reg_lambda',5,15),\n'min_split_gain':hp.uniform('min_split_gain',8,15),\n'colsample_bytree':hp.uniform('colsample_bytree',0.5,1),\n'min_data_in_leaf':hp.choice('min_data_in_leaf',range(20,30)),\n'top_rate':hp.uniform('top_rate',0.7,0.9),\n'reg_alpha':hp.uniform('reg_alpha',1,10),\n'learning_rate':hp.uniform('learning_rate',0.01,0.05),\n'other_rate':hp.uniform('other_rate',0.01,0.09)}\n\ntrials_reg = Trials()\n\nbest_reg = fmin(f_reg, space_dt_reg, algo=tpe.suggest, max_evals=100, trials=trials_reg)\n\nprint(best_reg)\n</code></pre>",
  "messages": [
    {
      "id": "476838",
      "postDate": "02/23/2019 09:47:06",
      "content": "<p>I want to tuning parameter by  hyperopt .Howerver , I got the lower score parameter .  </p>\n\n<h2>Ex . I got the parms by finishing run hyperopt:</h2>\n\n<p>{'colsample_bytree': 0.6761269574650649,\n 'learning_rate': 0.04519033564007213,\n 'max_depth': 0,\n 'min_child_weight': 30.199691742071565,\n 'min_data_in_leaf': 6,\n 'min_split_gain': 10.282274656774836,\n 'num_leaves': 37,\n 'other_rate': 0.027204317754823286,\n 'reg_alpha': 5.670906506486702,\n 'reg_lambda': 7.676023413606416,\n 'subsample': 0.8265019828109162,\n 'top_rate': 0.8678565632906692} <br>\n<strong>CV :3.6612530092950726</strong></p>\n\n<h2>But I find a parms beeter than it in log:</h2>\n\n<p>loss 3.654 params {'boosting': 'goss', 'colsample_bytree': 0.6761269574650649, 'learning_rate': 0.04519033564007213, 'max_depth': 8, 'metric': 'rmse', 'min_child_weight': 30.199691742071565, 'min_data_in_leaf': 26, 'min_split_gain': 10.282274656774836, 'num_leaves': 68, 'objective': 'root_mean_squared_error', 'other_rate': 0.027204317754823286, 'reg_alpha': 5.670906506486702, 'reg_lambda': 7.676023413606416, 'subsample': 0.8265019828109162, 'task': 'train', 'top_rate': 0.8678565632906692} <br>\n<strong>CV：3.654</strong></p>\n\n<h2>Here My code:</h2>\n\n<pre><code>train_X_reg=train_df[features]\ntrain_Y_reg=train_df.target\n#make scorer\nrmse_scorer = make_scorer(rmse,greater_is_better =False)\ndef hyperopt_lgb_reg(params):\n     clf =lgb.LGBMRegressor(**params)\n     return -cross_val_score(clf, train_X_reg, train_Y_reg,   scoring=rmse_scorer,cv=5).mean()\n\ndef f_reg(parms):\n  rmse_loss = hyperopt_lgb_reg(parms)\n  print(\"loss {:.3f} params {}\".format(rmse_loss, parms))\n  return {'loss': rmse_loss, 'status': STATUS_OK}\n\n space_dt_reg = {\n 'task':'train',\n'max_depth': hp.choice('max_depth', range(8,40)),\n'num_leaves': hp.choice('num_leaves', range(31,70)),\n'objective': 'root_mean_squared_error',\n'metric': 'rmse',\n'boosting': 'goss',\n'min_child_weight': hp.uniform('min_child_weight', 20,50),\n'subsample': hp.uniform('subsample', 0.8,1),\n'reg_lambda':hp.uniform('reg_lambda',5,15),\n'min_split_gain':hp.uniform('min_split_gain',8,15),\n'colsample_bytree':hp.uniform('colsample_bytree',0.5,1),\n'min_data_in_leaf':hp.choice('min_data_in_leaf',range(20,30)),\n'top_rate':hp.uniform('top_rate',0.7,0.9),\n'reg_alpha':hp.uniform('reg_alpha',1,10),\n'learning_rate':hp.uniform('learning_rate',0.01,0.05),\n'other_rate':hp.uniform('other_rate',0.01,0.09)}\n\ntrials_reg = Trials()\n\nbest_reg = fmin(f_reg, space_dt_reg, algo=tpe.suggest, max_evals=100, trials=trials_reg)\n\nprint(best_reg)\n</code></pre>",
      "rawMarkdown": "I want to tuning parameter by  hyperopt .Howerver , I got the lower score parameter .  \n## Ex . I got the parms by finishing run hyperopt: \n{'colsample_bytree': 0.6761269574650649,\n 'learning_rate': 0.04519033564007213,\n 'max_depth': 0,\n 'min_child_weight': 30.199691742071565,\n 'min_data_in_leaf': 6,\n 'min_split_gain': 10.282274656774836,\n 'num_leaves': 37,\n 'other_rate': 0.027204317754823286,\n 'reg_alpha': 5.670906506486702,\n 'reg_lambda': 7.676023413606416,\n 'subsample': 0.8265019828109162,\n 'top_rate': 0.8678565632906692}    \n**CV :3.6612530092950726**\n\n## But I find a parms beeter than it in log:\nloss 3.654 params {'boosting': 'goss', 'colsample_bytree': 0.6761269574650649, 'learning_rate': 0.04519033564007213, 'max_depth': 8, 'metric': 'rmse', 'min_child_weight': 30.199691742071565, 'min_data_in_leaf': 26, 'min_split_gain': 10.282274656774836, 'num_leaves': 68, 'objective': 'root_mean_squared_error', 'other_rate': 0.027204317754823286, 'reg_alpha': 5.670906506486702, 'reg_lambda': 7.676023413606416, 'subsample': 0.8265019828109162, 'task': 'train', 'top_rate': 0.8678565632906692}  \n**CV：3.654**\n\n## Here My code:\n\n    train_X_reg=train_df[features]\n    train_Y_reg=train_df.target\n    #make scorer\n    rmse_scorer = make_scorer(rmse,greater_is_better =False)\n    def hyperopt_lgb_reg(params):\n         clf =lgb.LGBMRegressor(**params)\n         return -cross_val_score(clf, train_X_reg, train_Y_reg,   scoring=rmse_scorer,cv=5).mean()\n     \n    def f_reg(parms):\n      rmse_loss = hyperopt_lgb_reg(parms)\n      print(\"loss {:.3f} params {}\".format(rmse_loss, parms))\n      return {'loss': rmse_loss, 'status': STATUS_OK}\n    \n     space_dt_reg = {\n     'task':'train',\n    'max_depth': hp.choice('max_depth', range(8,40)),\n    'num_leaves': hp.choice('num_leaves', range(31,70)),\n    'objective': 'root_mean_squared_error',\n    'metric': 'rmse',\n    'boosting': 'goss',\n    'min_child_weight': hp.uniform('min_child_weight', 20,50),\n    'subsample': hp.uniform('subsample', 0.8,1),\n    'reg_lambda':hp.uniform('reg_lambda',5,15),\n    'min_split_gain':hp.uniform('min_split_gain',8,15),\n    'colsample_bytree':hp.uniform('colsample_bytree',0.5,1),\n    'min_data_in_leaf':hp.choice('min_data_in_leaf',range(20,30)),\n    'top_rate':hp.uniform('top_rate',0.7,0.9),\n    'reg_alpha':hp.uniform('reg_alpha',1,10),\n    'learning_rate':hp.uniform('learning_rate',0.01,0.05),\n    'other_rate':hp.uniform('other_rate',0.01,0.09)}\n\n    trials_reg = Trials()\n   \n    best_reg = fmin(f_reg, space_dt_reg, algo=tpe.suggest, max_evals=100, trials=trials_reg)\n\n    print(best_reg)",
      "votes": null
    },
    {
      "id": "476840",
      "postDate": "02/23/2019 09:52:20",
      "content": "<p>I would remove the minus in the line below</p>\n\n<pre><code>return -cross_val_score(clf, train_X_reg, train_Y_reg,   scoring=rmse_scorer,cv=5).mean()</code></pre><code>\n\nor replace fmin by fmax in this line : \n</code><pre><code>best_reg = fmin(f_reg, space_dt_reg, algo=tpe.suggest, max_evals=100, trials=trials_reg)</code></pre>",
      "rawMarkdown": "I would remove the minus in the line below\n\n<pre><code>return -cross_val_score(clf, train_X_reg, train_Y_reg,   scoring=rmse_scorer,cv=5).mean()</code></pre><code><code>\n\nor replace fmin by fmax in this line : \n<pre><code>best_reg = fmin(f_reg, space_dt_reg, algo=tpe.suggest, max_evals=100, trials=trials_reg)</code></pre></code></code>",
      "votes": null
    },
    {
      "id": "476842",
      "postDate": "02/23/2019 09:58:16",
      "content": "<p>beacuse I set </p>\n\n<pre><code>greater_is_better =False in \nrmse_scorer = make_scorer(rmse,greater_is_better =False)\n</code></pre>\n\n<p>so fun   <strong>cross_val_score</strong>   return neg value</p>",
      "rawMarkdown": "beacuse I set \n\n    greater_is_better =False in \n    rmse_scorer = make_scorer(rmse,greater_is_better =False)\nso fun   **cross_val_score**   return neg value",
      "votes": null
    },
    {
      "id": "476844",
      "postDate": "02/23/2019 10:01:15",
      "content": "<p>As you return the value with a minus greater should be better I guess</p>",
      "rawMarkdown": "As you return the value with a minus greater should be better I guess",
      "votes": null
    },
    {
      "id": "476849",
      "postDate": "02/23/2019 10:11:00",
      "content": "<p>thx .I will try it now</p>",
      "rawMarkdown": "thx .I will try it now",
      "votes": null
    },
    {
      "id": "476888",
      "postDate": "02/23/2019 11:51:20",
      "content": "<p>everything is not getting better  :(</p>",
      "rawMarkdown": "everything is not getting better  :(",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 476840,
      "author_name": "francoisdubois",
      "author_url": "",
      "post_date": "02/23/2019 09:52:20",
      "content": "<p>I would remove the minus in the line below</p>\n\n<pre><code>return -cross_val_score(clf, train_X_reg, train_Y_reg,   scoring=rmse_scorer,cv=5).mean()</code></pre><code>\n\nor replace fmin by fmax in this line : \n</code><pre><code>best_reg = fmin(f_reg, space_dt_reg, algo=tpe.suggest, max_evals=100, trials=trials_reg)</code></pre>",
      "votes": null,
      "replies": [
        {
          "id": 476842,
          "author_name": "bobbyhuang",
          "author_url": "",
          "post_date": "02/23/2019 09:58:16",
          "content": "<p>beacuse I set </p>\n\n<pre><code>greater_is_better =False in \nrmse_scorer = make_scorer(rmse,greater_is_better =False)\n</code></pre>\n\n<p>so fun   <strong>cross_val_score</strong>   return neg value</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 476844,
          "author_name": "francoisdubois",
          "author_url": "",
          "post_date": "02/23/2019 10:01:15",
          "content": "<p>As you return the value with a minus greater should be better I guess</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 476849,
          "author_name": "bobbyhuang",
          "author_url": "",
          "post_date": "02/23/2019 10:11:00",
          "content": "<p>thx .I will try it now</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 476888,
          "author_name": "bobbyhuang",
          "author_url": "",
          "post_date": "02/23/2019 11:51:20",
          "content": "<p>everything is not getting better  :(</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "476838": "I want to tuning parameter by  hyperopt .Howerver , I got the lower score parameter .  \n## Ex . I got the parms by finishing run hyperopt: \n{'colsample_bytree': 0.6761269574650649,\n 'learning_rate': 0.04519033564007213,\n 'max_depth': 0,\n 'min_child_weight': 30.199691742071565,\n 'min_data_in_leaf': 6,\n 'min_split_gain': 10.282274656774836,\n 'num_leaves': 37,\n 'other_rate': 0.027204317754823286,\n 'reg_alpha': 5.670906506486702,\n 'reg_lambda': 7.676023413606416,\n 'subsample': 0.8265019828109162,\n 'top_rate': 0.8678565632906692}    \n**CV :3.6612530092950726**\n\n## But I find a parms beeter than it in log:\nloss 3.654 params {'boosting': 'goss', 'colsample_bytree': 0.6761269574650649, 'learning_rate': 0.04519033564007213, 'max_depth': 8, 'metric': 'rmse', 'min_child_weight': 30.199691742071565, 'min_data_in_leaf': 26, 'min_split_gain': 10.282274656774836, 'num_leaves': 68, 'objective': 'root_mean_squared_error', 'other_rate': 0.027204317754823286, 'reg_alpha': 5.670906506486702, 'reg_lambda': 7.676023413606416, 'subsample': 0.8265019828109162, 'task': 'train', 'top_rate': 0.8678565632906692}  \n**CV：3.654**\n\n## Here My code:\n\n    train_X_reg=train_df[features]\n    train_Y_reg=train_df.target\n    #make scorer\n    rmse_scorer = make_scorer(rmse,greater_is_better =False)\n    def hyperopt_lgb_reg(params):\n         clf =lgb.LGBMRegressor(**params)\n         return -cross_val_score(clf, train_X_reg, train_Y_reg,   scoring=rmse_scorer,cv=5).mean()\n     \n    def f_reg(parms):\n      rmse_loss = hyperopt_lgb_reg(parms)\n      print(\"loss {:.3f} params {}\".format(rmse_loss, parms))\n      return {'loss': rmse_loss, 'status': STATUS_OK}\n    \n     space_dt_reg = {\n     'task':'train',\n    'max_depth': hp.choice('max_depth', range(8,40)),\n    'num_leaves': hp.choice('num_leaves', range(31,70)),\n    'objective': 'root_mean_squared_error',\n    'metric': 'rmse',\n    'boosting': 'goss',\n    'min_child_weight': hp.uniform('min_child_weight', 20,50),\n    'subsample': hp.uniform('subsample', 0.8,1),\n    'reg_lambda':hp.uniform('reg_lambda',5,15),\n    'min_split_gain':hp.uniform('min_split_gain',8,15),\n    'colsample_bytree':hp.uniform('colsample_bytree',0.5,1),\n    'min_data_in_leaf':hp.choice('min_data_in_leaf',range(20,30)),\n    'top_rate':hp.uniform('top_rate',0.7,0.9),\n    'reg_alpha':hp.uniform('reg_alpha',1,10),\n    'learning_rate':hp.uniform('learning_rate',0.01,0.05),\n    'other_rate':hp.uniform('other_rate',0.01,0.09)}\n\n    trials_reg = Trials()\n   \n    best_reg = fmin(f_reg, space_dt_reg, algo=tpe.suggest, max_evals=100, trials=trials_reg)\n\n    print(best_reg)",
    "476840": "I would remove the minus in the line below\n\n<pre><code>return -cross_val_score(clf, train_X_reg, train_Y_reg,   scoring=rmse_scorer,cv=5).mean()</code></pre><code><code>\n\nor replace fmin by fmax in this line : \n<pre><code>best_reg = fmin(f_reg, space_dt_reg, algo=tpe.suggest, max_evals=100, trials=trials_reg)</code></pre></code></code>",
    "476842": "beacuse I set \n\n    greater_is_better =False in \n    rmse_scorer = make_scorer(rmse,greater_is_better =False)\nso fun   **cross_val_score**   return neg value",
    "476844": "As you return the value with a minus greater should be better I guess",
    "476849": "thx .I will try it now",
    "476888": "everything is not getting better  :("
  },
  "source": "meta"
}