As far as I know, CatBoost and XGBoost are almost equally good as LightGBM. But no one is using them? Why? Blending LGB,CAT,XGB can give better results right?
", "messages": [ { "id": "680719", "postDate": "11/25/2019 05:50:08", "content": "As far as I know, CatBoost and XGBoost are almost equally good as LightGBM. But no one is using them? Why? Blending LGB,CAT,XGB can give better results right?
", "rawMarkdown": "As far as I know, CatBoost and XGBoost are almost equally good as LightGBM. But no one is using them? Why? Blending LGB,CAT,XGB can give better results right?", "votes": null }, { "id": "680732", "postDate": "11/25/2019 06:33:18", "content": "XGB is too slow
", "rawMarkdown": "XGB is too slow", "votes": null }, { "id": "680761", "postDate": "11/25/2019 07:17:43", "content": "We all know that LightGBM is almost 10 times faster than XGBoost. This means that you can just try one model with XGBoost whereas you can try 10 different models with LightGBM. Feature engineering steps require so many trial and errors. That's why, most of kagglers spend time on LightGBM in the early steps of the competition.
\n\nOn the other hand, it is a fact that XGBoost provides more accurate models. You would do your final attempt with XGBoost when you finish the feature engineering steps.
", "rawMarkdown": "We all know that LightGBM is almost 10 times faster than XGBoost. This means that you can just try one model with XGBoost whereas you can try 10 different models with LightGBM. Feature engineering steps require so many trial and errors. That's why, most of kagglers spend time on LightGBM in the early steps of the competition.\n\nOn the other hand, it is a fact that XGBoost provides more accurate models. You would do your final attempt with XGBoost when you finish the feature engineering steps.", "votes": null }, { "id": "680863", "postDate": "11/25/2019 10:46:31", "content": "XGB is here:
\n\nhttps://www.kaggle.com/sheriytm/simple-xgboost-regressor-model
\n\nand catboost https://www.kaggle.com/kyakovlev/ashrae-catboost
", "rawMarkdown": "XGB is here:\n\nhttps://www.kaggle.com/sheriytm/simple-xgboost-regressor-model\n\nand catboost https://www.kaggle.com/kyakovlev/ashrae-catboost", "votes": null }, { "id": "680903", "postDate": "11/25/2019 11:40:01", "content": "I see the same as ahao. XGBoost is slower than LightGBM in my experience. I don't know about CatBoost speed
", "rawMarkdown": "I see the same as ahao. XGBoost is slower than LightGBM in my experience. I don't know about CatBoost speed", "votes": null }, { "id": "680915", "postDate": "11/25/2019 11:52:50", "content": "But in most recent competitions from last 1 year, most of the people use LightGBM for final predictions in kaggle. Why XGBoost is better as a fact?
", "rawMarkdown": "But in most recent competitions from last 1 year, most of the people use LightGBM for final predictions in kaggle. Why XGBoost is better as a fact?", "votes": null }, { "id": "680916", "postDate": "11/25/2019 11:53:23", "content": "CatBoost is faster or same speed.
", "rawMarkdown": "CatBoost is faster or same speed.", "votes": null }, { "id": "680917", "postDate": "11/25/2019 11:54:17", "content": "but not good scores :/
\n\nIs it because, they are not performing good? I tried CatBoost and got 1.21 score, while I got 1.08 with LightGBM. Maybe I did not tune Catboost properly.
", "rawMarkdown": "but not good scores :/ \n\nIs it because, they are not performing good? I tried CatBoost and got 1.21 score, while I got 1.08 with LightGBM. Maybe I did not tune Catboost properly.", "votes": null }, { "id": "681258", "postDate": "11/25/2019 23:00:45", "content": "Because this is a web scraping competition, data leakage is the king and kills all the algorithms
", "rawMarkdown": "Because this is a web scraping competition, data leakage is the king and kills all the algorithms", "votes": null }, { "id": "681300", "postDate": "11/26/2019 00:33:53", "content": "LightGBM does a great job of directly handling categorical values (as long as you correctly declare them), and thus can achieve more power with fewer estimators than XGB. Since it's faster even with the same number of estimators, it's really no contest for this contest.
", "rawMarkdown": "LightGBM does a great job of directly handling categorical values (as long as you correctly declare them), and thus can achieve more power with fewer estimators than XGB. Since it's faster even with the *same* number of estimators, it's really no contest for this contest.", "votes": null }, { "id": "681499", "postDate": "11/26/2019 07:03:27", "content": "I see the same pattern @keshardev \nI think that is related to features as meter building_id and site_id.\nI think that's because how catboost calculate the numerical representation of categorical. \nIn particular i think the problem arise because test set has 1 observations for every timestamp (it has not gaps) so the calculation are different between train and test.\nAlso in the discussion \"Final thoughts...\" The author says he observed a better score by dropping building_id and site_id.
", "rawMarkdown": "I see the same pattern @keshardev \nI think that is related to features as meter building_id and site_id.\nI think that's because how catboost calculate the numerical representation of categorical. \nIn particular i think the problem arise because test set has 1 observations for every timestamp (it has not gaps) so the calculation are different between train and test.\nAlso in the discussion \"Final thoughts...\" The author says he observed a better score by dropping building_id and site_id.", "votes": null }, { "id": "681502", "postDate": "11/26/2019 07:04:46", "content": "Because how it calculates the bin. They are more accurate then lightgbm one (which uses hist)
", "rawMarkdown": "Because how it calculates the bin. They are more accurate then lightgbm one (which uses hist)", "votes": null }, { "id": "681913", "postDate": "11/26/2019 17:12:48", "content": "LightGBM CPU version is blazing fast compared to XGBoost. Even for GPU versions of both algorithms, in my tests, I have seen slightly faster runtimes on LightGBM. I haven't tested Catboost for speed though.
", "rawMarkdown": "LightGBM CPU version is blazing fast compared to XGBoost. Even for GPU versions of both algorithms, in my tests, I have seen slightly faster runtimes on LightGBM. I haven't tested Catboost for speed though.", "votes": null }, { "id": "682185", "postDate": "11/27/2019 03:55:27", "content": "Using catboost with NO category features - on my dual GPU machine it out runs lightgbm hands down.
\n\nAfter lots of attempts I have given up getting lightgbm built with GPU on both Windows and Ubuntu, so only running it with CPU.
\n\nUsing catboost WITH category features - lightgbm on CPU is the winner by landslide. Catboost looks busy as heck - both GPU's at full speed with 96% of memory in use - but its at least 20 times slower on basic feature set for this competition vs lightgbm.
", "rawMarkdown": "Using catboost with NO category features - on my dual GPU machine it out runs lightgbm hands down.\n\nAfter lots of attempts I have given up getting lightgbm built with GPU on both Windows and Ubuntu, so only running it with CPU.\n\nUsing catboost WITH category features - lightgbm on CPU is the winner by landslide. Catboost looks busy as heck - both GPU's at full speed with 96% of memory in use - but its at least 20 times slower on basic feature set for this competition vs lightgbm.", "votes": null }, { "id": "682393", "postDate": "11/27/2019 11:31:52", "content": "I try to use Catboost, but it doesn't allow me to use too many features( leading to the kernels died), and the best score I got is 1.17. And I fail to run Xgboost until now. Maybe I write the wrong code as it's the first kaggle competition for me.
", "rawMarkdown": "I try to use Catboost, but it doesn't allow me to use too many features( leading to the kernels died), and the best score I got is 1.17. And I fail to run Xgboost until now. Maybe I write the wrong code as it's the first kaggle competition for me.", "votes": null }, { "id": "682476", "postDate": "11/27/2019 13:38:56", "content": "Could you please elaborate on LGBM GPU speed? I tried it on the 5mln rows sample of ASHRAE data and so virtually no difference with the CPU version speed. Moreover, unlike Pytorch or TF2 GPU versions, LGBM only loaded my GPU (nvidia 1660ti) by 6-7% (!!!)
", "rawMarkdown": "Could you please elaborate on LGBM GPU speed? I tried it on the 5mln rows sample of ASHRAE data and so virtually no difference with the CPU version speed. Moreover, unlike Pytorch or TF2 GPU versions, LGBM only loaded my GPU (nvidia 1660ti) by 6-7% (!!!)", "votes": null }, { "id": "683190", "postDate": "11/28/2019 07:23:13", "content": "Is the accuracy of GPU and CPU version same?
", "rawMarkdown": "Is the accuracy of GPU and CPU version same?", "votes": null }, { "id": "683371", "postDate": "11/28/2019 11:02:47", "content": "LeakBoost is the best choice for this competition.
", "rawMarkdown": "LeakBoost is the best choice for this competition.", "votes": null }, { "id": "688772", "postDate": "12/06/2019 04:32:54", "content": "I tried both XGB and CatBoost,with XGB I get 1.13 lb score,but it is hard to optimize it，because xgb is too slow.And catboost seems to require memory consumption too much,my catboost died each time I try it
", "rawMarkdown": "I tried both XGB and CatBoost,with XGB I get 1.13 lb score,but it is hard to optimize it，because xgb is too slow.And catboost seems to require memory consumption too much,my catboost died each time I try it", "votes": null }, { "id": "692122", "postDate": "12/10/2019 23:10:45", "content": "I believe NuGene from Sonaosft can accomplish this no problem.
", "rawMarkdown": "I believe NuGene from Sonaosft can accomplish this no problem.", "votes": null } ], "comments": [ { "id": 680732, "author_name": "xhzs2333", "author_url": "", "post_date": "11/25/2019 06:33:18", "content": "XGB is too slow
", "votes": null, "replies": [] }, { "id": 680761, "author_name": "serengil", "author_url": "", "post_date": "11/25/2019 07:17:43", "content": "We all know that LightGBM is almost 10 times faster than XGBoost. This means that you can just try one model with XGBoost whereas you can try 10 different models with LightGBM. Feature engineering steps require so many trial and errors. That's why, most of kagglers spend time on LightGBM in the early steps of the competition.
\n\nOn the other hand, it is a fact that XGBoost provides more accurate models. You would do your final attempt with XGBoost when you finish the feature engineering steps.
", "votes": null, "replies": [ { "id": 680915, "author_name": "keshardev", "author_url": "", "post_date": "11/25/2019 11:52:50", "content": "But in most recent competitions from last 1 year, most of the people use LightGBM for final predictions in kaggle. Why XGBoost is better as a fact?
", "votes": null, "replies": [] }, { "id": 681502, "author_name": "stenford23", "author_url": "", "post_date": "11/26/2019 07:04:46", "content": "Because how it calculates the bin. They are more accurate then lightgbm one (which uses hist)
", "votes": null, "replies": [] } ] }, { "id": 680863, "author_name": "sheriytm", "author_url": "", "post_date": "11/25/2019 10:46:31", "content": "XGB is here:
\n\nhttps://www.kaggle.com/sheriytm/simple-xgboost-regressor-model
\n\nand catboost https://www.kaggle.com/kyakovlev/ashrae-catboost
", "votes": null, "replies": [ { "id": 680917, "author_name": "keshardev", "author_url": "", "post_date": "11/25/2019 11:54:17", "content": "but not good scores :/
\n\nIs it because, they are not performing good? I tried CatBoost and got 1.21 score, while I got 1.08 with LightGBM. Maybe I did not tune Catboost properly.
", "votes": null, "replies": [] }, { "id": 681499, "author_name": "stenford23", "author_url": "", "post_date": "11/26/2019 07:03:27", "content": "I see the same pattern @keshardev \nI think that is related to features as meter building_id and site_id.\nI think that's because how catboost calculate the numerical representation of categorical. \nIn particular i think the problem arise because test set has 1 observations for every timestamp (it has not gaps) so the calculation are different between train and test.\nAlso in the discussion \"Final thoughts...\" The author says he observed a better score by dropping building_id and site_id.
", "votes": null, "replies": [] } ] }, { "id": 680903, "author_name": "esonnen", "author_url": "", "post_date": "11/25/2019 11:40:01", "content": "I see the same as ahao. XGBoost is slower than LightGBM in my experience. I don't know about CatBoost speed
", "votes": null, "replies": [ { "id": 680916, "author_name": "keshardev", "author_url": "", "post_date": "11/25/2019 11:53:23", "content": "CatBoost is faster or same speed.
", "votes": null, "replies": [] } ] }, { "id": 681258, "author_name": "lftuwujie", "author_url": "", "post_date": "11/25/2019 23:00:45", "content": "Because this is a web scraping competition, data leakage is the king and kills all the algorithms
", "votes": null, "replies": [] }, { "id": 681300, "author_name": "purist1024", "author_url": "", "post_date": "11/26/2019 00:33:53", "content": "LightGBM does a great job of directly handling categorical values (as long as you correctly declare them), and thus can achieve more power with fewer estimators than XGB. Since it's faster even with the same number of estimators, it's really no contest for this contest.
", "votes": null, "replies": [] }, { "id": 681913, "author_name": "sandy1112", "author_url": "", "post_date": "11/26/2019 17:12:48", "content": "LightGBM CPU version is blazing fast compared to XGBoost. Even for GPU versions of both algorithms, in my tests, I have seen slightly faster runtimes on LightGBM. I haven't tested Catboost for speed though.
", "votes": null, "replies": [ { "id": 682476, "author_name": "poltigo", "author_url": "", "post_date": "11/27/2019 13:38:56", "content": "Could you please elaborate on LGBM GPU speed? I tried it on the 5mln rows sample of ASHRAE data and so virtually no difference with the CPU version speed. Moreover, unlike Pytorch or TF2 GPU versions, LGBM only loaded my GPU (nvidia 1660ti) by 6-7% (!!!)
", "votes": null, "replies": [] }, { "id": 683190, "author_name": "keshardev", "author_url": "", "post_date": "11/28/2019 07:23:13", "content": "Is the accuracy of GPU and CPU version same?
", "votes": null, "replies": [] } ] }, { "id": 682185, "author_name": "pcjimmmy", "author_url": "", "post_date": "11/27/2019 03:55:27", "content": "Using catboost with NO category features - on my dual GPU machine it out runs lightgbm hands down.
\n\nAfter lots of attempts I have given up getting lightgbm built with GPU on both Windows and Ubuntu, so only running it with CPU.
\n\nUsing catboost WITH category features - lightgbm on CPU is the winner by landslide. Catboost looks busy as heck - both GPU's at full speed with 96% of memory in use - but its at least 20 times slower on basic feature set for this competition vs lightgbm.
", "votes": null, "replies": [] }, { "id": 682393, "author_name": "good19961121", "author_url": "", "post_date": "11/27/2019 11:31:52", "content": "I try to use Catboost, but it doesn't allow me to use too many features( leading to the kernels died), and the best score I got is 1.17. And I fail to run Xgboost until now. Maybe I write the wrong code as it's the first kaggle competition for me.
", "votes": null, "replies": [] }, { "id": 683371, "author_name": "bandits", "author_url": "", "post_date": "11/28/2019 11:02:47", "content": "LeakBoost is the best choice for this competition.
", "votes": null, "replies": [] }, { "id": 688772, "author_name": "decarmelo", "author_url": "", "post_date": "12/06/2019 04:32:54", "content": "I tried both XGB and CatBoost,with XGB I get 1.13 lb score,but it is hard to optimize it，because xgb is too slow.And catboost seems to require memory consumption too much,my catboost died each time I try it
", "votes": null, "replies": [] }, { "id": 692122, "author_name": "aaronmarinucci", "author_url": "", "post_date": "12/10/2019 23:10:45", "content": "I believe NuGene from Sonaosft can accomplish this no problem.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "680719": "As far as I know, CatBoost and XGBoost are almost equally good as LightGBM. But no one is using them? Why? Blending LGB,CAT,XGB can give better results right?", "680732": "XGB is too slow", "680761": "We all know that LightGBM is almost 10 times faster than XGBoost. This means that you can just try one model with XGBoost whereas you can try 10 different models with LightGBM. Feature engineering steps require so many trial and errors. That's why, most of kagglers spend time on LightGBM in the early steps of the competition.\n\nOn the other hand, it is a fact that XGBoost provides more accurate models. You would do your final attempt with XGBoost when you finish the feature engineering steps.", "680863": "XGB is here:\n\nhttps://www.kaggle.com/sheriytm/simple-xgboost-regressor-model\n\nand catboost https://www.kaggle.com/kyakovlev/ashrae-catboost", "680903": "I see the same as ahao. XGBoost is slower than LightGBM in my experience. I don't know about CatBoost speed", "680915": "But in most recent competitions from last 1 year, most of the people use LightGBM for final predictions in kaggle. Why XGBoost is better as a fact?", "680916": "CatBoost is faster or same speed.", "680917": "but not good scores :/ \n\nIs it because, they are not performing good? I tried CatBoost and got 1.21 score, while I got 1.08 with LightGBM. Maybe I did not tune Catboost properly.", "681258": "Because this is a web scraping competition, data leakage is the king and kills all the algorithms", "681300": "LightGBM does a great job of directly handling categorical values (as long as you correctly declare them), and thus can achieve more power with fewer estimators than XGB. Since it's faster even with the *same* number of estimators, it's really no contest for this contest.", "681499": "I see the same pattern @keshardev \nI think that is related to features as meter building_id and site_id.\nI think that's because how catboost calculate the numerical representation of categorical. \nIn particular i think the problem arise because test set has 1 observations for every timestamp (it has not gaps) so the calculation are different between train and test.\nAlso in the discussion \"Final thoughts...\" The author says he observed a better score by dropping building_id and site_id.", "681502": "Because how it calculates the bin. They are more accurate then lightgbm one (which uses hist)", "681913": "LightGBM CPU version is blazing fast compared to XGBoost. Even for GPU versions of both algorithms, in my tests, I have seen slightly faster runtimes on LightGBM. I haven't tested Catboost for speed though.", "682185": "Using catboost with NO category features - on my dual GPU machine it out runs lightgbm hands down.\n\nAfter lots of attempts I have given up getting lightgbm built with GPU on both Windows and Ubuntu, so only running it with CPU.\n\nUsing catboost WITH category features - lightgbm on CPU is the winner by landslide. Catboost looks busy as heck - both GPU's at full speed with 96% of memory in use - but its at least 20 times slower on basic feature set for this competition vs lightgbm.", "682393": "I try to use Catboost, but it doesn't allow me to use too many features( leading to the kernels died), and the best score I got is 1.17. And I fail to run Xgboost until now. Maybe I write the wrong code as it's the first kaggle competition for me.", "682476": "Could you please elaborate on LGBM GPU speed? I tried it on the 5mln rows sample of ASHRAE data and so virtually no difference with the CPU version speed. Moreover, unlike Pytorch or TF2 GPU versions, LGBM only loaded my GPU (nvidia 1660ti) by 6-7% (!!!)", "683190": "Is the accuracy of GPU and CPU version same?", "683371": "LeakBoost is the best choice for this competition.", "688772": "I tried both XGB and CatBoost,with XGB I get 1.13 lb score,but it is hard to optimize it，because xgb is too slow.And catboost seems to require memory consumption too much,my catboost died each time I try it", "692122": "I believe NuGene from Sonaosft can accomplish this no problem." }, "source": "meta" }