If you are tuning hp, I highly recommend you install a GBT library that has support for GPU. \nFor instance, you can use catboost since it is quite easy to install:
\n\nconda config --add channels conda-forge and conda install catboost.
Then:
\n\n\n from catboost import CatBoostRegressor\n model = CatBoostRegressor(loss_function='RMSE', eval_metric='MAE', task_type=\"GPU\")\n
I hope this helps some people. :)
", "messages": [ { "id": "569235", "postDate": "07/06/2019 09:58:38", "content": "If you are tuning hp, I highly recommend you install a GBT library that has support for GPU. \nFor instance, you can use catboost since it is quite easy to install:
\n\nconda config --add channels conda-forge and conda install catboost.
Then:
\n\n\n from catboost import CatBoostRegressor\n model = CatBoostRegressor(loss_function='RMSE', eval_metric='MAE', task_type=\"GPU\")\n
I hope this helps some people. :)
", "rawMarkdown": "If you are tuning hp, I highly recommend you install a GBT library that has support for GPU. \nFor instance, you can use [**catboost**](https://catboost.ai/docs/features/training-on-gpu.html) since it is quite easy to install: \n\n`conda config --add channels conda-forge` and `conda install catboost`.\n\nThen: \n\n```\n from catboost import CatBoostRegressor\n model = CatBoostRegressor(loss_function='RMSE', eval_metric='MAE', task_type=\"GPU\")\n```\n\nI hope this helps some people. :)", "votes": null }, { "id": "569339", "postDate": "07/06/2019 13:37:06", "content": "xgboost and lightgbm also have gpu acceleration.
", "rawMarkdown": "xgboost and lightgbm also have gpu acceleration.", "votes": null }, { "id": "570472", "postDate": "07/08/2019 10:50:28", "content": "To my practice catBoost performs much worse than lightGBM on mae, may I ask what hyper-params are you using?
", "rawMarkdown": "To my practice catBoost performs much worse than lightGBM on mae, may I ask what hyper-params are you using?", "votes": null }, { "id": "574966", "postDate": "07/14/2019 19:05:17", "content": "Not sure but I could'n wait the end of working catboost on the dataset where lightgbm worked well. Are there any features to start working with catboost?
", "rawMarkdown": "Not sure but I could'n wait the end of working catboost on the dataset where lightgbm worked well. Are there any features to start working with catboost?", "votes": null }, { "id": "575022", "postDate": "07/14/2019 22:25:56", "content": "Thanks @yassinealouini for the recommendation.
\n\nI recently installed the GPU build of LightGBM and did some testing on the 1JHN type with around 300 features using 4 threads. Here are some observations:\n- For max_bin of 255, there is no difference between cpu and gpu\n- For max_bin of 127, there is a difference of 1.66 secs per 1000 rounds between cpu and gpu. For 10,000 rounds, 3 folds, and 200 hyperopt iterations that is a savings of approx 2.8 hrs\n- For max_bin of 63, there is a difference of 8 secs per 1000 rounds between cpu and gpu. For 10,000 rounds, 3 folds, and 200 hyperopt iterations that is a savings of approx 13.5 hrs.
\n\nI am new to trying LightGBM in GPU and it looks like having a lower value of max_bin in your hyperopt experimentations will give a good boost in time savings.
", "rawMarkdown": "Thanks @yassinealouini for the recommendation.\n\nI recently installed the GPU build of LightGBM and did some testing on the 1JHN type with around 300 features using 4 threads. Here are some observations:\n- For max_bin of 255, there is no difference between cpu and gpu\n- For max_bin of 127, there is a difference of 1.66 secs per 1000 rounds between cpu and gpu. For 10,000 rounds, 3 folds, and 200 hyperopt iterations that is a savings of approx 2.8 hrs\n- For max_bin of 63, there is a difference of 8 secs per 1000 rounds between cpu and gpu. For 10,000 rounds, 3 folds, and 200 hyperopt iterations that is a savings of approx 13.5 hrs.\n\nI am new to trying LightGBM in GPU and it looks like having a lower value of max_bin in your hyperopt experimentations will give a good boost in time savings.", "votes": null }, { "id": "577485", "postDate": "07/16/2019 18:11:35", "content": "In Kaggle kernels you can run LGBM, CatBoost, and XGBoost all with GPU. Note that by default they all run in CPU mode. Instructions about converting all to GPU is here
", "rawMarkdown": "In Kaggle kernels you can run LGBM, CatBoost, and XGBoost all with GPU. Note that by default they all run in CPU mode. Instructions about converting all to GPU is [here][1]\n\n[1]: https://www.kaggle.com/c/santander-customer-transaction-prediction/discussion/89004", "votes": null }, { "id": "577567", "postDate": "07/16/2019 19:47:08", "content": "Would love to try GPU with lightgbm on my Windows 10 machine - but have really died a slow death trying to follow the build directions to create the library. Anyone know of a nice set of directions that have worked with little pain?
", "rawMarkdown": "Would love to try GPU with lightgbm on my Windows 10 machine - but have really died a slow death trying to follow the build directions to create the library. Anyone know of a nice set of directions that have worked with little pain?", "votes": null }, { "id": "577572", "postDate": "07/16/2019 19:51:04", "content": "Here is a Kaggle kernel. Not sure if that works for Windows though
", "rawMarkdown": "[Here][1] is a Kaggle kernel. Not sure if that works for Windows though\n\n[1]: https://www.kaggle.com/vinhnguyen/gpu-acceleration-for-lightgbm/", "votes": null }, { "id": "577623", "postDate": "07/16/2019 21:03:36", "content": "I confirm that Catboost doesn't require any installation. Just set GPU on and task_type = \"GPU\".\nHere I show also how to enable GPU with LGBM.
Thanks for the try - not even close to working. Tried to follow the flow in a command window but lost it pretty quick.
\n\nSince I got it removed I will give the directions on the github one more time.
", "rawMarkdown": "Thanks for the try - not even close to working. Tried to follow the flow in a command window but lost it pretty quick.\n\nSince I got it removed I will give the directions on the github one more time.", "votes": null }, { "id": "577769", "postDate": "07/17/2019 03:55:30", "content": "Have you tried pip install lightgbm --install-option=--gpu ? It is the documented way on https://github.com/microsoft/LightGBM/tree/master/python-package
Yes - tried that and a few other version besides. The problem seems to be that both Windows 10 and I are too ignorant to compile the library. I think I have successfully installed all the stuff needed (VS 2019, cmake, boost, etc) but windows cannot find the right paths and errors out with your suggestion trying to compile. I jumped on an open issue on the lightgbm github by someone to improve ability to get Windows build. Did get a response today - they have asked me to send the link to the example for Linux that Chris posted - sounds like they were going to ask the author of that kernel to see if he would do a Windows version, Would still be a ugly path and probably not work for most systems but might provide a good trail to follow. If you give me a clean enough trail I often can find my way home.
", "rawMarkdown": "Yes - tried that and a few other version besides. The problem seems to be that both Windows 10 and I are too ignorant to compile the library. I think I have successfully installed all the stuff needed (VS 2019, cmake, boost, etc) but windows cannot find the right paths and errors out with your suggestion trying to compile. I jumped on an open issue on the lightgbm github by someone to improve ability to get Windows build. Did get a response today - they have asked me to send the link to the example for Linux that Chris posted - sounds like they were going to ask the author of that kernel to see if he would do a Windows version, Would still be a ugly path and probably not work for most systems but might provide a good trail to follow. If you give me a clean enough trail I often can find my way home.", "votes": null } ], "comments": [ { "id": 569339, "author_name": "cpmpml", "author_url": "", "post_date": "07/06/2019 13:37:06", "content": "xgboost and lightgbm also have gpu acceleration.
", "votes": null, "replies": [] }, { "id": 570472, "author_name": "littlebeandog", "author_url": "", "post_date": "07/08/2019 10:50:28", "content": "To my practice catBoost performs much worse than lightGBM on mae, may I ask what hyper-params are you using?
", "votes": null, "replies": [ { "id": 574966, "author_name": "kirill702b", "author_url": "", "post_date": "07/14/2019 19:05:17", "content": "Not sure but I could'n wait the end of working catboost on the dataset where lightgbm worked well. Are there any features to start working with catboost?
", "votes": null, "replies": [] } ] }, { "id": 575022, "author_name": "hidasri", "author_url": "", "post_date": "07/14/2019 22:25:56", "content": "Thanks @yassinealouini for the recommendation.
\n\nI recently installed the GPU build of LightGBM and did some testing on the 1JHN type with around 300 features using 4 threads. Here are some observations:\n- For max_bin of 255, there is no difference between cpu and gpu\n- For max_bin of 127, there is a difference of 1.66 secs per 1000 rounds between cpu and gpu. For 10,000 rounds, 3 folds, and 200 hyperopt iterations that is a savings of approx 2.8 hrs\n- For max_bin of 63, there is a difference of 8 secs per 1000 rounds between cpu and gpu. For 10,000 rounds, 3 folds, and 200 hyperopt iterations that is a savings of approx 13.5 hrs.
\n\nI am new to trying LightGBM in GPU and it looks like having a lower value of max_bin in your hyperopt experimentations will give a good boost in time savings.
", "votes": null, "replies": [] }, { "id": 577485, "author_name": "cdeotte", "author_url": "", "post_date": "07/16/2019 18:11:35", "content": "In Kaggle kernels you can run LGBM, CatBoost, and XGBoost all with GPU. Note that by default they all run in CPU mode. Instructions about converting all to GPU is here
", "votes": null, "replies": [] }, { "id": 577567, "author_name": "pcjimmmy", "author_url": "", "post_date": "07/16/2019 19:47:08", "content": "Would love to try GPU with lightgbm on my Windows 10 machine - but have really died a slow death trying to follow the build directions to create the library. Anyone know of a nice set of directions that have worked with little pain?
", "votes": null, "replies": [ { "id": 577572, "author_name": "cdeotte", "author_url": "", "post_date": "07/16/2019 19:51:04", "content": "Here is a Kaggle kernel. Not sure if that works for Windows though
", "votes": null, "replies": [] }, { "id": 577704, "author_name": "pcjimmmy", "author_url": "", "post_date": "07/17/2019 01:42:49", "content": "Thanks for the try - not even close to working. Tried to follow the flow in a command window but lost it pretty quick.
\n\nSince I got it removed I will give the directions on the github one more time.
", "votes": null, "replies": [] }, { "id": 577769, "author_name": "cpmpml", "author_url": "", "post_date": "07/17/2019 03:55:30", "content": "Have you tried pip install lightgbm --install-option=--gpu ? It is the documented way on https://github.com/microsoft/LightGBM/tree/master/python-package
Yes - tried that and a few other version besides. The problem seems to be that both Windows 10 and I are too ignorant to compile the library. I think I have successfully installed all the stuff needed (VS 2019, cmake, boost, etc) but windows cannot find the right paths and errors out with your suggestion trying to compile. I jumped on an open issue on the lightgbm github by someone to improve ability to get Windows build. Did get a response today - they have asked me to send the link to the example for Linux that Chris posted - sounds like they were going to ask the author of that kernel to see if he would do a Windows version, Would still be a ugly path and probably not work for most systems but might provide a good trail to follow. If you give me a clean enough trail I often can find my way home.
", "votes": null, "replies": [] } ] }, { "id": 577623, "author_name": "raimonds1993", "author_url": "", "post_date": "07/16/2019 21:03:36", "content": "I confirm that Catboost doesn't require any installation. Just set GPU on and task_type = \"GPU\".\nHere I show also how to enable GPU with LGBM.