Update 2019.07.10
\n\nTwo days struggling and finally figure out by brute-force searching over all hyper-params.....\nTo make catboost grows similar to lightgbm, set 'boosting_type': 'Plain', then two model are about at same level.\nIn practice catboost ask for less tunning, and also can use GPU to accelarate.\nEspecially on dataset of this competition, CAT is about twice slower than LGB on CPU only, and about 5 to 10 faster on GPU.\nHope these information helpful as I have wasted two days on this shit - -!
As far as I know, catboost should have competitive performance as lightgbm.\nBut In practice I just can not make catboost as good as lightgbm using same set of features.
\n\nSpecifically, I use the features set from Giba's Kernel (https://www.kaggle.com/titericz/giba-r-data-table-simple-features-1-17-lb).
\n\nTake 1JHC for example\nFor catboost I tunned a lot of hyperparams, I use LR=0.03, n_estimators=10000, depth=9, and i could get cv about -0.4.\nFor lightgbm, same LR, n_esimators and depth and could reach about -0.6.\nLightgbm almost always performs better than catboost under same hyperparams.
\n\nDid I miss something important of catboost ?\nAnd Is there way to make catboost train the same way as lightgbm?
", "messages": [ { "id": 571085, "postDate": "2019-07-09T06:27:35.857Z", "content": "Update 2019.07.10
\n\nTwo days struggling and finally figure out by brute-force searching over all hyper-params.....\nTo make catboost grows similar to lightgbm, set 'boosting_type': 'Plain', then two model are about at same level.\nIn practice catboost ask for less tunning, and also can use GPU to accelarate.\nEspecially on dataset of this competition, CAT is about twice slower than LGB on CPU only, and about 5 to 10 faster on GPU.\nHope these information helpful as I have wasted two days on this shit - -!
As far as I know, catboost should have competitive performance as lightgbm.\nBut In practice I just can not make catboost as good as lightgbm using same set of features.
\n\nSpecifically, I use the features set from Giba's Kernel (https://www.kaggle.com/titericz/giba-r-data-table-simple-features-1-17-lb).
\n\nTake 1JHC for example\nFor catboost I tunned a lot of hyperparams, I use LR=0.03, n_estimators=10000, depth=9, and i could get cv about -0.4.\nFor lightgbm, same LR, n_esimators and depth and could reach about -0.6.\nLightgbm almost always performs better than catboost under same hyperparams.
\n\nDid I miss something important of catboost ?\nAnd Is there way to make catboost train the same way as lightgbm?
", "rawMarkdown": "Update 2019.07.10\n\nTwo days struggling and finally figure out by brute-force searching over all hyper-params.....\nTo make catboost grows similar to lightgbm, set `'boosting_type': 'Plain'`, then two model are about at same level.\nIn practice catboost ask for less tunning, and also can use GPU to accelarate.\nEspecially on dataset of this competition, CAT is about twice slower than LGB on CPU only, and about 5 to 10 faster on GPU.\nHope these information helpful as I have wasted two days on this shit - -!\n\n----\nAs far as I know, catboost should have competitive performance as lightgbm.\nBut In practice I just can not make catboost as good as lightgbm using same set of features.\n\nSpecifically, I use the features set from Giba's Kernel (https://www.kaggle.com/titericz/giba-r-data-table-simple-features-1-17-lb). \n\nTake 1JHC for example\nFor catboost I tunned a lot of hyperparams, I use LR=0.03, n_estimators=10000, depth=9, and i could get cv about -0.4.\nFor lightgbm, same LR, n_esimators and depth and could reach about -0.6.\nLightgbm almost always performs better than catboost under same hyperparams.\n\nDid I miss something important of catboost ?\nAnd Is there way to make catboost train the same way as lightgbm?", "votes": 5 }, { "id": 595464, "postDate": "2019-08-09T09:00:30.223Z", "content": "The benefit of catboost is that it requires very less tuning of parameters due to the constrained way in which it builds the tree. Specifically, at depth d, all nodes are of same feature in a tree. And it is faster than lightgbm with GPU. Performance wise, for me, catboost is better than lightgbm. (I've not spent time tuning lightgbm)
", "rawMarkdown": "The benefit of catboost is that it requires very less tuning of parameters due to the constrained way in which it builds the tree. Specifically, at depth d, all nodes are of same feature in a tree. And it is faster than lightgbm with GPU. Performance wise, for me, catboost is better than lightgbm. (I've not spent time tuning lightgbm)" }, { "id": 571403, "postDate": "2019-07-09T15:05:41.320Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 571462, "postDate": "2019-07-09T15:52:24.587Z", "content": "Yes, catboost depends mostly on lr and depth. Of course other params affects, but no matter how i tunned them, there's always big gap between lgb and cat.
", "rawMarkdown": "Yes, catboost depends mostly on lr and depth. Of course other params affects, but no matter how i tunned them, there's always big gap between lgb and cat." }, { "id": 571484, "postDate": "2019-07-09T16:35:17.357Z", "rawMarkdown": "", "isDeleted": true }, { "id": 571718, "postDate": "2019-07-10T02:50:14.010Z", "content": "The lightgbm let you reach the -1.479?
", "rawMarkdown": "The lightgbm let you reach the -1.479?" }, { "id": 571904, "postDate": "2019-07-10T08:16:47.213Z", "content": "yep, it's over 20000 rounds.
", "rawMarkdown": "yep, it's over 20000 rounds." }, { "id": 571941, "postDate": "2019-07-10T08:56:24.517Z", "content": "@timmmmmms \nAndrew Lukyanenko wrote in this thread that they got to seventh place with lightGBM. If I am not mistaken, seventh place that time was already around -2.
", "rawMarkdown": "@timmmmmms \nAndrew Lukyanenko wrote in [this thread](https://www.kaggle.com/c/champs-scalar-coupling/discussion/98038#latest-570752) that they got to seventh place with lightGBM. If I am not mistaken, seventh place that time was already around -2." }, { "id": 571946, "postDate": "2019-07-10T09:02:07.167Z", "content": "\n\n\nAndrew Lukyanenko wrote in this thread that they got to seventh place with lightGBM.
\n
No, he said he is using lgb. He didn't say what his team mates are using.
", "rawMarkdown": "> Andrew Lukyanenko wrote in this thread that they got to seventh place with lightGBM.\n\nNo, he said he is using lgb. He didn't say what his team mates are using.", "votes": 1 }, { "id": 574953, "postDate": "2019-07-14T18:49:42.800Z", "content": "-1.478 - hell yeah!=)
", "rawMarkdown": "-1.478 - hell yeah!=)", "votes": 1 }, { "id": 574959, "postDate": "2019-07-14T18:54:04.180Z", "content": "To be serious I tried fully connected nns and could get only two times worse result result in comparison with lgbm. I can't suppose the reason. But I think graph nns can bring new possibilities.
", "rawMarkdown": "To be serious I tried fully connected nns and could get only two times worse result result in comparison with lgbm. I can't suppose the reason. But I think graph nns can bring new possibilities.", "votes": 1 } ] } ], "comments": [ { "id": 595464, "author_name": "Ashesh.", "author_url": "", "post_date": "2019-08-09T09:00:30.223000", "content": "The benefit of catboost is that it requires very less tuning of parameters due to the constrained way in which it builds the tree. Specifically, at depth d, all nodes are of same feature in a tree. And it is faster than lightgbm with GPU. Performance wise, for me, catboost is better than lightgbm. (I've not spent time tuning lightgbm)
", "votes": 0, "replies": [] }, { "id": 571403, "author_name": "", "author_url": "", "post_date": "2019-07-09T15:05:41.320000", "content": "", "votes": 0, "replies": [ { "id": 571462, "author_name": "littlebeandog", "author_url": "", "post_date": "2019-07-09T15:52:24.587000", "content": "Yes, catboost depends mostly on lr and depth. Of course other params affects, but no matter how i tunned them, there's always big gap between lgb and cat.
", "votes": 0, "replies": [] }, { "id": 571484, "author_name": "", "author_url": "", "post_date": "2019-07-09T16:35:17.357000", "content": "", "votes": 0, "replies": [] }, { "id": 571718, "author_name": "Timmmmmms", "author_url": "", "post_date": "2019-07-10T02:50:14.010000", "content": "The lightgbm let you reach the -1.479?
", "votes": 0, "replies": [] }, { "id": 571904, "author_name": "littlebeandog", "author_url": "", "post_date": "2019-07-10T08:16:47.213000", "content": "yep, it's over 20000 rounds.
", "votes": 0, "replies": [] }, { "id": 571941, "author_name": "somnambWl", "author_url": "", "post_date": "2019-07-10T08:56:24.517000", "content": "@timmmmmms \nAndrew Lukyanenko wrote in this thread that they got to seventh place with lightGBM. If I am not mistaken, seventh place that time was already around -2.
", "votes": 0, "replies": [] }, { "id": 571946, "author_name": "CPMP", "author_url": "", "post_date": "2019-07-10T09:02:07.167000", "content": "\n\n\nAndrew Lukyanenko wrote in this thread that they got to seventh place with lightGBM.
\n
No, he said he is using lgb. He didn't say what his team mates are using.
", "votes": 1, "replies": [] }, { "id": 574953, "author_name": "Kirill Balakhonov", "author_url": "", "post_date": "2019-07-14T18:49:42.800000", "content": "-1.478 - hell yeah!=)
", "votes": 1, "replies": [] }, { "id": 574959, "author_name": "Kirill Balakhonov", "author_url": "", "post_date": "2019-07-14T18:54:04.180000", "content": "To be serious I tried fully connected nns and could get only two times worse result result in comparison with lgbm. I can't suppose the reason. But I think graph nns can bring new possibilities.
", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "571085": "Update 2019.07.10\n\nTwo days struggling and finally figure out by brute-force searching over all hyper-params.....\nTo make catboost grows similar to lightgbm, set `'boosting_type': 'Plain'`, then two model are about at same level.\nIn practice catboost ask for less tunning, and also can use GPU to accelarate.\nEspecially on dataset of this competition, CAT is about twice slower than LGB on CPU only, and about 5 to 10 faster on GPU.\nHope these information helpful as I have wasted two days on this shit - -!\n\n----\nAs far as I know, catboost should have competitive performance as lightgbm.\nBut In practice I just can not make catboost as good as lightgbm using same set of features.\n\nSpecifically, I use the features set from Giba's Kernel (https://www.kaggle.com/titericz/giba-r-data-table-simple-features-1-17-lb). \n\nTake 1JHC for example\nFor catboost I tunned a lot of hyperparams, I use LR=0.03, n_estimators=10000, depth=9, and i could get cv about -0.4.\nFor lightgbm, same LR, n_esimators and depth and could reach about -0.6.\nLightgbm almost always performs better than catboost under same hyperparams.\n\nDid I miss something important of catboost ?\nAnd Is there way to make catboost train the same way as lightgbm?", "595464": "The benefit of catboost is that it requires very less tuning of parameters due to the constrained way in which it builds the tree. Specifically, at depth d, all nodes are of same feature in a tree. And it is faster than lightgbm with GPU. Performance wise, for me, catboost is better than lightgbm. (I've not spent time tuning lightgbm)", "571403": "" } }