{
  "id": 451896,
  "title": "How to optimize model hyperparameters",
  "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/451896",
  "author_name": "Renato Reggiani",
  "post_date": "2023-10-31T00:31:19.358000",
  "votes": 3,
  "comment_count": 0,
  "views": 0,
  "content": "<p>After the  <a href=\"https://www.kaggle.com/yeoyunsianggeremie\" target=\"_blank\">@yeoyunsianggeremie</a> 's excellent work on how to perform CV in this competition, I made this <a href=\"https://www.kaggle.com/code/renatoreggiani/zzzs-how-to-optimize-models\" target=\"_blank\">notebook</a> to optimize model hyperparameters. </p>\n<p>I ran it with LBGM but you can change the model in this function:</p>\n<pre><code>def objective(trial):\n\n    lgb_params = {\n        : trial.suggest_int(, 500, 700),\n        : trial.suggest_float(, 1e-3, 5e-2, =),\n        : trial.suggest_int(, 3, 6),\n        : trial.suggest_float(, 0.01, 0.05, =0.01),\n        : trial.suggest_float(, 0.8, 0.9, =0.01),\n        : trial.suggest_int(, 200, 500),\n    }\n\n    model = LGBMClassifier(**lgb_params, =51)\n\n    score = cv_score(model, =SPLITS)\n\n\n    return np.mean(score)\n</code></pre>\n<p>Feel free to comment and improve this discussion</p>",
  "messages": [
    {
      "id": 2505905,
      "postDate": "2023-10-31T00:31:19.357Z",
      "content": "<p>After the  <a href=\"https://www.kaggle.com/yeoyunsianggeremie\" target=\"_blank\">@yeoyunsianggeremie</a> 's excellent work on how to perform CV in this competition, I made this <a href=\"https://www.kaggle.com/code/renatoreggiani/zzzs-how-to-optimize-models\" target=\"_blank\">notebook</a> to optimize model hyperparameters. </p>\n<p>I ran it with LBGM but you can change the model in this function:</p>\n<pre><code>def objective(trial):\n\n    lgb_params = {\n        : trial.suggest_int(, 500, 700),\n        : trial.suggest_float(, 1e-3, 5e-2, =),\n        : trial.suggest_int(, 3, 6),\n        : trial.suggest_float(, 0.01, 0.05, =0.01),\n        : trial.suggest_float(, 0.8, 0.9, =0.01),\n        : trial.suggest_int(, 200, 500),\n    }\n\n    model = LGBMClassifier(**lgb_params, =51)\n\n    score = cv_score(model, =SPLITS)\n\n\n    return np.mean(score)\n</code></pre>\n<p>Feel free to comment and improve this discussion</p>",
      "rawMarkdown": "After the  @yeoyunsianggeremie 's excellent work on how to perform CV in this competition, I made this [notebook](https://www.kaggle.com/code/renatoreggiani/zzzs-how-to-optimize-models) to optimize model hyperparameters. \n\nI ran it with LBGM but you can change the model in this function:\n\n```\ndef objective(trial):\n    \n    lgb_params = {\n        'n_estimators': trial.suggest_int('n_estimators', 500, 700),\n        'learning_rate': trial.suggest_float('learning_rate', 1e-3, 5e-2, log=True),\n        'max_depth': trial.suggest_int('max_depth', 3, 6),\n        'reg_alpha': trial.suggest_float('subsample', 0.01, 0.05, step=0.01),\n        'colsample_bytree': trial.suggest_float('colsample_bytree', 0.8, 0.9, step=0.01),\n        'min_child_weight': trial.suggest_int('min_child_weight', 200, 500),\n    }\n    \n    model = LGBMClassifier(**lgb_params, random_state=51)\n    \n    score = cv_score(model, SPLITS=SPLITS)\n#     scores.append(score)\n    \n    return np.mean(score)\n```\n\nFeel free to comment and improve this discussion",
      "votes": 3
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2505905": "After the  @yeoyunsianggeremie 's excellent work on how to perform CV in this competition, I made this [notebook](https://www.kaggle.com/code/renatoreggiani/zzzs-how-to-optimize-models) to optimize model hyperparameters. \n\nI ran it with LBGM but you can change the model in this function:\n\n```\ndef objective(trial):\n    \n    lgb_params = {\n        'n_estimators': trial.suggest_int('n_estimators', 500, 700),\n        'learning_rate': trial.suggest_float('learning_rate', 1e-3, 5e-2, log=True),\n        'max_depth': trial.suggest_int('max_depth', 3, 6),\n        'reg_alpha': trial.suggest_float('subsample', 0.01, 0.05, step=0.01),\n        'colsample_bytree': trial.suggest_float('colsample_bytree', 0.8, 0.9, step=0.01),\n        'min_child_weight': trial.suggest_int('min_child_weight', 200, 500),\n    }\n    \n    model = LGBMClassifier(**lgb_params, random_state=51)\n    \n    score = cv_score(model, SPLITS=SPLITS)\n#     scores.append(score)\n    \n    return np.mean(score)\n```\n\nFeel free to comment and improve this discussion"
  }
}