{
  "id": 80978,
  "title": "Does anyone share a way to optimize model parameters?",
  "url": "/competitions/elo-merchant-category-recommendation/discussion/80978",
  "author_name": "Kang0709",
  "post_date": "2019-02-18T11:44:51.414000",
  "votes": 2,
  "comment_count": 16,
  "views": 0,
  "content": "<p>hello everyone, \nDoes anyone share a way to optimize LGB or XGB model parameters?</p>",
  "messages": [
    {
      "id": 473891,
      "postDate": "2019-02-18T16:37:15.447Z",
      "content": "<p>I used the mlr library in r to optimize xgboost parameters.  And, I know people use optuna in python to optimize parameters.\n<a href=\"https://github.com/mlr-org/mlr/\">https://github.com/mlr-org/mlr/</a>\n<a href=\"https://github.com/pfnet/optuna\">https://github.com/pfnet/optuna</a></p>",
      "rawMarkdown": "I used the mlr library in r to optimize xgboost parameters.  And, I know people use optuna in python to optimize parameters.\nhttps://github.com/mlr-org/mlr/\nhttps://github.com/pfnet/optuna\n",
      "votes": 3,
      "replies": [
        {
          "id": 474135,
          "postDate": "2019-02-19T01:10:10.420Z",
          "content": "<p>thanks!</p>",
          "rawMarkdown": "thanks!"
        }
      ]
    },
    {
      "id": 474216,
      "postDate": "2019-02-19T04:50:30.480Z",
      "content": "<p>For python, You can check \n1) <a href=\"https://www.kaggle.com/sz8416/simple-bayesian-optimization-for-lightgbm\">https://www.kaggle.com/sz8416/simple-bayesian-optimization-for-lightgbm</a>\n2) <a href=\"https://www.kaggle.com/tilii7/bayesian-optimization-of-xgboost-parameters\">https://www.kaggle.com/tilii7/bayesian-optimization-of-xgboost-parameters</a>\n3) <a href=\"https://github.com/WillKoehrsen/hyperparameter-optimization/blob/master/Bayesian%20Hyperparameter%20Optimization%20of%20Gradient%20Boosting%20Machine.ipynb\">https://github.com/WillKoehrsen/hyperparameter-optimization/blob/master/Bayesian%20Hyperparameter%20Optimization%20of%20Gradient%20Boosting%20Machine.ipynb</a></p>",
      "rawMarkdown": "For python, You can check \n1) https://www.kaggle.com/sz8416/simple-bayesian-optimization-for-lightgbm\n2) https://www.kaggle.com/tilii7/bayesian-optimization-of-xgboost-parameters\n3) https://github.com/WillKoehrsen/hyperparameter-optimization/blob/master/Bayesian%20Hyperparameter%20Optimization%20of%20Gradient%20Boosting%20Machine.ipynb",
      "votes": 4,
      "replies": [
        {
          "id": 474233,
          "postDate": "2019-02-19T05:32:37.723Z",
          "content": "<p>thank you!!</p>",
          "rawMarkdown": "thank you!!"
        }
      ]
    },
    {
      "id": 475217,
      "postDate": "2019-02-20T13:04:45.867Z",
      "content": "<p>Personally my favorite tools are :\n- Scikit-Optimize : <a href=\"https://scikit-optimize.github.io/\">https://scikit-optimize.github.io/</a>\n- Spearmint : <a href=\"https://github.com/JasperSnoek/spearmint\">https://github.com/JasperSnoek/spearmint</a> \nAnd rightnow I'm trying BenderOpt : <a href=\"https://github.com/Dreem-Organization/benderopt\">https://github.com/Dreem-Organization/benderopt</a> .\nThere is some examples in their repo.</p>",
      "rawMarkdown": "Personally my favorite tools are :\n- Scikit-Optimize : https://scikit-optimize.github.io/\n- Spearmint : https://github.com/JasperSnoek/spearmint \nAnd rightnow I'm trying BenderOpt : https://github.com/Dreem-Organization/benderopt .\nThere is some examples in their repo.",
      "votes": 1
    },
    {
      "id": 474282,
      "postDate": "2019-02-19T06:55:20.273Z",
      "content": "<p>I find that for me random grid search is the most efficient way in terms of time and result </p>",
      "rawMarkdown": "I find that for me random grid search is the most efficient way in terms of time and result ",
      "votes": 1
    },
    {
      "id": 473693,
      "postDate": "2019-02-18T11:44:51.413Z",
      "content": "<p>hello everyone, \nDoes anyone share a way to optimize LGB or XGB model parameters?</p>",
      "rawMarkdown": "hello everyone, \nDoes anyone share a way to optimize LGB or XGB model parameters?",
      "votes": 2
    },
    {
      "id": 477295,
      "postDate": "2019-02-24T10:01:05.140Z",
      "content": "<p>Beware, hyper parameter optimization is the best way I know to overfit.  Don't do too much, and cross validate using different random seeds for instance.</p>",
      "rawMarkdown": "Beware, hyper parameter optimization is the best way I know to overfit.  Don't do too much, and cross validate using different random seeds for instance.\n"
    },
    {
      "id": 475243,
      "postDate": "2019-02-20T13:48:44.340Z",
      "content": "<p>hyperopt:  <a href=\"https://github.com/hyperopt/hyperopt\">https://github.com/hyperopt/hyperopt</a>\noptuna:  <a href=\"https://github.com/pfnet/optuna\">https://github.com/pfnet/optuna</a></p>",
      "rawMarkdown": "hyperopt:  https://github.com/hyperopt/hyperopt\noptuna:  https://github.com/pfnet/optuna"
    },
    {
      "id": 475154,
      "postDate": "2019-02-20T10:52:43.130Z",
      "content": "<p>Having error to run from bayes_opt import BayesianOptimization.</p>\n\n<p>ModuleNotFoundError: No module named 'bayes_opt'</p>\n\n<p>I installed with \"pip install bayesian-optimization\". <a href=\"https://github.com/fmfn/BayesianOptimization\">https://github.com/fmfn/BayesianOptimization</a>\non Windows10, py3.6, anaconda  dist, conda env</p>\n\n<p>(tf-keras-conda-p36) C:\\Users\\Roman\\K\\elo\\code&gt;pip install bayesian-optimization\nRequirement already satisfied: bayesian-optimization in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (1.0.1)\nRequirement already satisfied: scikit-learn&gt;=0.18.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (0.20.2)\nRequirement already satisfied: numpy&gt;=1.9.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (1.14.0)\nRequirement already satisfied: scipy&gt;=0.14.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (1.0.0)</p>\n\n<p>Are there any hidden tricks for installation or conflicts with existing packages?  Thanks</p>",
      "rawMarkdown": "Having error to run from bayes_opt import BayesianOptimization.\n\nModuleNotFoundError: No module named 'bayes_opt'\n\nI installed with \"pip install bayesian-optimization\". https://github.com/fmfn/BayesianOptimization\non Windows10, py3.6, anaconda  dist, conda env\n\n(tf-keras-conda-p36) C:\\Users\\Roman\\K\\elo\\code&gt;pip install bayesian-optimization\nRequirement already satisfied: bayesian-optimization in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (1.0.1)\nRequirement already satisfied: scikit-learn&gt;=0.18.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (0.20.2)\nRequirement already satisfied: numpy&gt;=1.9.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (1.14.0)\nRequirement already satisfied: scipy&gt;=0.14.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (1.0.0)\n\nAre there any hidden tricks for installation or conflicts with existing packages?  Thanks",
      "replies": [
        {
          "id": 475258,
          "postDate": "2019-02-20T14:17:14.403Z",
          "content": "<p>You should write, from bayes_opt instead of bayesopt.</p>",
          "rawMarkdown": "You should write, from bayes_opt instead of bayesopt."
        },
        {
          "id": 475464,
          "postDate": "2019-02-20T19:29:26.417Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 475467,
          "postDate": "2019-02-20T19:34:23.963Z",
          "content": "<p>it was formatting issue earlier. dropped underscore. still have the above error.</p>",
          "rawMarkdown": "it was formatting issue earlier. dropped underscore. still have the above error."
        },
        {
          "id": 475483,
          "postDate": "2019-02-20T19:45:55.363Z",
          "content": "<p>That's weird. I checked on Windows as well and it is working. Can you do one thing. Just do \n\"!pip install bayesian-optimization\" from your script first and then do \"from bayes_opt import BayesianOptimization\".</p>",
          "rawMarkdown": "That's weird. I checked on Windows as well and it is working. Can you do one thing. Just do \n\"!pip install bayesian-optimization\" from your script first and then do \"from bayes_opt import BayesianOptimization\"."
        },
        {
          "id": 475529,
          "postDate": "2019-02-20T21:26:39.207Z",
          "content": "<p>I was in the above conda env and started the jupyter notebook. The same error</p>\n\n<p>!pip install bayesian-optimization\nRequirement already satisfied: bayesian-optimization in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (1.0.1)\nRequirement already satisfied: numpy&gt;=1.9.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (1.14.0)\nRequirement already satisfied: scipy&gt;=0.14.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (1.0.0)\nRequirement already satisfied: scikit-learn&gt;=0.18.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (0.20.2)</p>\n\n<p>from bayes_opt import BayesianOptimization</p>\n\n<hr>\n\n<p>ModuleNotFoundError                       Traceback (most recent call last)</p>",
          "rawMarkdown": "I was in the above conda env and started the jupyter notebook. The same error\n\n!pip install bayesian-optimization\nRequirement already satisfied: bayesian-optimization in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (1.0.1)\nRequirement already satisfied: numpy&gt;=1.9.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (1.14.0)\nRequirement already satisfied: scipy&gt;=0.14.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (1.0.0)\nRequirement already satisfied: scikit-learn&gt;=0.18.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (0.20.2)\n\nfrom bayes_opt import BayesianOptimization\n\n---------------------------------------------------------------------------\nModuleNotFoundError                       Traceback (most recent call last)\n"
        },
        {
          "id": 475560,
          "postDate": "2019-02-20T22:22:32.583Z",
          "content": "<p>Sorry, but i don't know why is that happening. May be someone else can help.</p>",
          "rawMarkdown": "Sorry, but i don't know why is that happening. May be someone else can help."
        },
        {
          "id": 476108,
          "postDate": "2019-02-21T16:08:47.200Z",
          "content": "<p>Finally, setup on Ubuntu with conda. </p>",
          "rawMarkdown": "Finally, setup on Ubuntu with conda. ",
          "votes": 1
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 473891,
      "author_name": "Kyle Peters",
      "author_url": "",
      "post_date": "2019-02-18T16:37:15.447000",
      "content": "<p>I used the mlr library in r to optimize xgboost parameters.  And, I know people use optuna in python to optimize parameters.\n<a href=\"https://github.com/mlr-org/mlr/\">https://github.com/mlr-org/mlr/</a>\n<a href=\"https://github.com/pfnet/optuna\">https://github.com/pfnet/optuna</a></p>",
      "votes": 3,
      "replies": [
        {
          "id": 474135,
          "author_name": "Kang0709",
          "author_url": "",
          "post_date": "2019-02-19T01:10:10.420000",
          "content": "<p>thanks!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 474216,
      "author_name": "HarshitMehta",
      "author_url": "",
      "post_date": "2019-02-19T04:50:30.480000",
      "content": "<p>For python, You can check \n1) <a href=\"https://www.kaggle.com/sz8416/simple-bayesian-optimization-for-lightgbm\">https://www.kaggle.com/sz8416/simple-bayesian-optimization-for-lightgbm</a>\n2) <a href=\"https://www.kaggle.com/tilii7/bayesian-optimization-of-xgboost-parameters\">https://www.kaggle.com/tilii7/bayesian-optimization-of-xgboost-parameters</a>\n3) <a href=\"https://github.com/WillKoehrsen/hyperparameter-optimization/blob/master/Bayesian%20Hyperparameter%20Optimization%20of%20Gradient%20Boosting%20Machine.ipynb\">https://github.com/WillKoehrsen/hyperparameter-optimization/blob/master/Bayesian%20Hyperparameter%20Optimization%20of%20Gradient%20Boosting%20Machine.ipynb</a></p>",
      "votes": 4,
      "replies": [
        {
          "id": 474233,
          "author_name": "Kang0709",
          "author_url": "",
          "post_date": "2019-02-19T05:32:37.723000",
          "content": "<p>thank you!!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 475217,
      "author_name": "Arthur  Llau",
      "author_url": "",
      "post_date": "2019-02-20T13:04:45.867000",
      "content": "<p>Personally my favorite tools are :\n- Scikit-Optimize : <a href=\"https://scikit-optimize.github.io/\">https://scikit-optimize.github.io/</a>\n- Spearmint : <a href=\"https://github.com/JasperSnoek/spearmint\">https://github.com/JasperSnoek/spearmint</a> \nAnd rightnow I'm trying BenderOpt : <a href=\"https://github.com/Dreem-Organization/benderopt\">https://github.com/Dreem-Organization/benderopt</a> .\nThere is some examples in their repo.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 474282,
      "author_name": "AmirH",
      "author_url": "",
      "post_date": "2019-02-19T06:55:20.273000",
      "content": "<p>I find that for me random grid search is the most efficient way in terms of time and result </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 477295,
      "author_name": "CPMP",
      "author_url": "",
      "post_date": "2019-02-24T10:01:05.140000",
      "content": "<p>Beware, hyper parameter optimization is the best way I know to overfit.  Don't do too much, and cross validate using different random seeds for instance.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 475243,
      "author_name": "fwj7",
      "author_url": "",
      "post_date": "2019-02-20T13:48:44.340000",
      "content": "<p>hyperopt:  <a href=\"https://github.com/hyperopt/hyperopt\">https://github.com/hyperopt/hyperopt</a>\noptuna:  <a href=\"https://github.com/pfnet/optuna\">https://github.com/pfnet/optuna</a></p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 475154,
      "author_name": "Roman",
      "author_url": "",
      "post_date": "2019-02-20T10:52:43.130000",
      "content": "<p>Having error to run from bayes_opt import BayesianOptimization.</p>\n\n<p>ModuleNotFoundError: No module named 'bayes_opt'</p>\n\n<p>I installed with \"pip install bayesian-optimization\". <a href=\"https://github.com/fmfn/BayesianOptimization\">https://github.com/fmfn/BayesianOptimization</a>\non Windows10, py3.6, anaconda  dist, conda env</p>\n\n<p>(tf-keras-conda-p36) C:\\Users\\Roman\\K\\elo\\code&gt;pip install bayesian-optimization\nRequirement already satisfied: bayesian-optimization in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (1.0.1)\nRequirement already satisfied: scikit-learn&gt;=0.18.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (0.20.2)\nRequirement already satisfied: numpy&gt;=1.9.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (1.14.0)\nRequirement already satisfied: scipy&gt;=0.14.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (1.0.0)</p>\n\n<p>Are there any hidden tricks for installation or conflicts with existing packages?  Thanks</p>",
      "votes": 0,
      "replies": [
        {
          "id": 475258,
          "author_name": "HarshitMehta",
          "author_url": "",
          "post_date": "2019-02-20T14:17:14.403000",
          "content": "<p>You should write, from bayes_opt instead of bayesopt.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 475464,
          "author_name": "",
          "author_url": "",
          "post_date": "2019-02-20T19:29:26.417000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 475467,
          "author_name": "Roman",
          "author_url": "",
          "post_date": "2019-02-20T19:34:23.963000",
          "content": "<p>it was formatting issue earlier. dropped underscore. still have the above error.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 475483,
          "author_name": "HarshitMehta",
          "author_url": "",
          "post_date": "2019-02-20T19:45:55.363000",
          "content": "<p>That's weird. I checked on Windows as well and it is working. Can you do one thing. Just do \n\"!pip install bayesian-optimization\" from your script first and then do \"from bayes_opt import BayesianOptimization\".</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 475529,
          "author_name": "Roman",
          "author_url": "",
          "post_date": "2019-02-20T21:26:39.207000",
          "content": "<p>I was in the above conda env and started the jupyter notebook. The same error</p>\n\n<p>!pip install bayesian-optimization\nRequirement already satisfied: bayesian-optimization in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (1.0.1)\nRequirement already satisfied: numpy&gt;=1.9.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (1.14.0)\nRequirement already satisfied: scipy&gt;=0.14.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (1.0.0)\nRequirement already satisfied: scikit-learn&gt;=0.18.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (0.20.2)</p>\n\n<p>from bayes_opt import BayesianOptimization</p>\n\n<hr>\n\n<p>ModuleNotFoundError                       Traceback (most recent call last)</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 475560,
          "author_name": "HarshitMehta",
          "author_url": "",
          "post_date": "2019-02-20T22:22:32.583000",
          "content": "<p>Sorry, but i don't know why is that happening. May be someone else can help.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 476108,
          "author_name": "Roman",
          "author_url": "",
          "post_date": "2019-02-21T16:08:47.200000",
          "content": "<p>Finally, setup on Ubuntu with conda. </p>",
          "votes": 1,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "473891": "I used the mlr library in r to optimize xgboost parameters.  And, I know people use optuna in python to optimize parameters.\nhttps://github.com/mlr-org/mlr/\nhttps://github.com/pfnet/optuna\n",
    "474216": "For python, You can check \n1) https://www.kaggle.com/sz8416/simple-bayesian-optimization-for-lightgbm\n2) https://www.kaggle.com/tilii7/bayesian-optimization-of-xgboost-parameters\n3) https://github.com/WillKoehrsen/hyperparameter-optimization/blob/master/Bayesian%20Hyperparameter%20Optimization%20of%20Gradient%20Boosting%20Machine.ipynb",
    "475217": "Personally my favorite tools are :\n- Scikit-Optimize : https://scikit-optimize.github.io/\n- Spearmint : https://github.com/JasperSnoek/spearmint \nAnd rightnow I'm trying BenderOpt : https://github.com/Dreem-Organization/benderopt .\nThere is some examples in their repo.",
    "474282": "I find that for me random grid search is the most efficient way in terms of time and result ",
    "473693": "hello everyone, \nDoes anyone share a way to optimize LGB or XGB model parameters?",
    "477295": "Beware, hyper parameter optimization is the best way I know to overfit.  Don't do too much, and cross validate using different random seeds for instance.\n",
    "475243": "hyperopt:  https://github.com/hyperopt/hyperopt\noptuna:  https://github.com/pfnet/optuna",
    "475154": "Having error to run from bayes_opt import BayesianOptimization.\n\nModuleNotFoundError: No module named 'bayes_opt'\n\nI installed with \"pip install bayesian-optimization\". https://github.com/fmfn/BayesianOptimization\non Windows10, py3.6, anaconda  dist, conda env\n\n(tf-keras-conda-p36) C:\\Users\\Roman\\K\\elo\\code&gt;pip install bayesian-optimization\nRequirement already satisfied: bayesian-optimization in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (1.0.1)\nRequirement already satisfied: scikit-learn&gt;=0.18.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (0.20.2)\nRequirement already satisfied: numpy&gt;=1.9.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (1.14.0)\nRequirement already satisfied: scipy&gt;=0.14.0 in c:\\users\\roman\\anaconda3\\envs\\tf-keras-conda-p36\\lib\\site-packages (from bayesian-optimization) (1.0.0)\n\nAre there any hidden tricks for installation or conflicts with existing packages?  Thanks"
  }
}