Hi everyone! \nI got 3.66x on local and got 3.708 on LB with 5 fold LGB. What is your best model?
", "messages": [ { "id": 432332, "postDate": "2018-12-03T18:20:56.003Z", "content": "Hi everyone! \nI got 3.66x on local and got 3.708 on LB with 5 fold LGB. What is your best model?
", "rawMarkdown": "Hi everyone! \nI got 3.66x on local and got 3.708 on LB with 5 fold LGB. What is your best model? ", "votes": 29 }, { "id": 432389, "postDate": "2018-12-03T19:45:48.200Z", "content": "it's the impact of the negative outliers. 98% of observations are between -1 and 1. With a chunk of outliers with target = -33, the squared error is enormous. Thus the goal is predict ourliers accurately, not the 98% of the rest of the data.
", "rawMarkdown": "it's the impact of the negative outliers. 98% of observations are between -1 and 1. With a chunk of outliers with target = -33, the squared error is enormous. Thus the goal is predict ourliers accurately, not the 98% of the rest of the data. ", "votes": 8 }, { "id": 451276, "postDate": "2019-01-06T18:48:49.327Z", "content": "5 Fold LGB ------> CV: 3.65510 LB: 3.683\n( No feature elimination , no outlier handling , No count Vector , no FM - Just aggregated features and elapsed time)
", "rawMarkdown": "5 Fold LGB ------> CV: 3.65510 LB: 3.683\n( No feature elimination , no outlier handling , No count Vector , no FM - Just aggregated features and elapsed time)", "votes": 5, "replies": [ { "id": 451345, "postDate": "2019-01-06T22:42:14.887Z", "content": "3.683? That was amazing! The improvement is from parameter tuning?
", "rawMarkdown": "3.683? That was amazing! The improvement is from parameter tuning?" }, { "id": 451520, "postDate": "2019-01-07T08:18:36.753Z", "content": "Wow, Gap is very small !!
", "rawMarkdown": "Wow, Gap is very small !!" }, { "id": 452059, "postDate": "2019-01-08T06:00:53.773Z", "content": "That is really amazing, I haven't do (FM/outlier) but I only got 3.687.\nMay we have a talk?
", "rawMarkdown": "That is really amazing, I haven't do (FM/outlier) but I only got 3.687.\nMay we have a talk?" }, { "id": 452451, "postDate": "2019-01-08T18:35:02.730Z", "content": "Your's is amazing as well! It was single model's result without any stacking? I have even smaller CV but LB is around 3.692. Maybe we can be a team?
", "rawMarkdown": "Your's is amazing as well! It was single model's result without any stacking? I have even smaller CV but LB is around 3.692. Maybe we can be a team?" }, { "id": 452635, "postDate": "2019-01-09T01:16:01.927Z", "content": "We can have a talk if u have some insights ?
", "rawMarkdown": "We can have a talk if u have some insights ?" }, { "id": 452729, "postDate": "2019-01-09T04:59:25.333Z", "content": "Can you send me a kaggle message including your contact info so that we can have a talk? I cannot send messages anymore. Thanks!
", "rawMarkdown": "Can you send me a kaggle message including your contact info so that we can have a talk? I cannot send messages anymore. Thanks!" }, { "id": 459250, "postDate": "2019-01-21T12:56:48.450Z", "content": "This is quite amazing, my CV is 3.646, but my LB is only 3.688...........
", "rawMarkdown": "This is quite amazing, my CV is 3.646, but my LB is only 3.688..........." }, { "id": 462876, "postDate": "2019-01-29T03:26:38.767Z", "content": "Hi Long Yin, what is your current best single model ?
", "rawMarkdown": "Hi Long Yin, what is your current best single model ?" }, { "id": 462927, "postDate": "2019-01-29T05:53:29.137Z", "content": "CV: 3.635, LB: 3.674. The rest is coming from the public kernel and I do some change with the binary classification.
", "rawMarkdown": "CV: 3.635, LB: 3.674. The rest is coming from the public kernel and I do some change with the binary classification.", "votes": 1 }, { "id": 462930, "postDate": "2019-01-29T05:56:06.413Z", "content": "is this your single model?
", "rawMarkdown": "is this your single model?" } ] }, { "id": 438141, "postDate": "2018-12-13T07:00:27.437Z", "content": "LightGBM 5fold CV 3.65313-> LB 3.689
", "rawMarkdown": "LightGBM 5fold CV 3.65313-> LB 3.689", "votes": 5, "replies": [ { "id": 438477, "postDate": "2018-12-13T19:04:44.870Z", "content": "Interesting, I also got a 5fold CV 3.653 but it only scored 5fold CV 3.653-> LB 3.703. Looks like there's lots of variation.
", "rawMarkdown": "Interesting, I also got a 5fold CV 3.653 but it only scored 5fold CV 3.653-> LB 3.703. Looks like there's lots of variation.", "votes": 1 }, { "id": 438795, "postDate": "2018-12-14T07:38:45.697Z", "content": "frankly, I just do a small change of the public kernel.
", "rawMarkdown": "frankly, I just do a small change of the public kernel. " }, { "id": 438909, "postDate": "2018-12-14T11:48:04.547Z", "content": "did u include merchant info in your model?
", "rawMarkdown": "did u include merchant info in your model?" }, { "id": 439126, "postDate": "2018-12-14T19:18:40.083Z", "content": "No, I didn't
", "rawMarkdown": "No, I didn't" }, { "id": 441739, "postDate": "2018-12-19T01:52:41.100Z", "content": "Hi SomethingIsWrong, how many feature are you using?
", "rawMarkdown": "Hi SomethingIsWrong, how many feature are you using?" }, { "id": 441752, "postDate": "2018-12-19T02:40:16.543Z", "content": "I use Chau Ngoc Huynh's kernel features. still dealing with gap of cv and LB, haven't add features.
", "rawMarkdown": "I use Chau Ngoc Huynh's kernel features. still dealing with gap of cv and LB, haven't add features.", "votes": 5 }, { "id": 442083, "postDate": "2018-12-19T13:16:02.603Z", "content": "So LGB params gave u the boost?
", "rawMarkdown": "So LGB params gave u the boost?", "votes": 5 }, { "id": 446534, "postDate": "2018-12-28T08:49:19.787Z", "content": "Amazing! Could you share some resources to deal with gap of cv and LB? Many thanks!
", "rawMarkdown": "Amazing! Could you share some resources to deal with gap of cv and LB? Many thanks!", "votes": 5 }, { "id": 459252, "postDate": "2019-01-21T12:58:03.027Z", "content": "Would you mind sharing your CV now?
", "rawMarkdown": "Would you mind sharing your CV now?" }, { "id": 459339, "postDate": "2019-01-21T15:46:20.713Z", "content": "My teammate has better one, mine best one is 3.639.
", "rawMarkdown": "My teammate has better one, mine best one is 3.639." } ] }, { "id": 432508, "postDate": "2018-12-04T01:12:11.490Z", "content": "Hi，my CV is 3.65810 and LB:3.697 with 5 fold LGB
", "rawMarkdown": "Hi，my CV is 3.65810 and LB:3.697 with 5 fold LGB", "votes": 5 }, { "id": 449346, "postDate": "2019-01-03T02:14:10.643Z", "content": "LightGBM 5fold CV 3.6532-->lb3.687\nLightGBM 5fold CV 3.65101-->lb3.687
", "rawMarkdown": "LightGBM 5fold CV 3.6532-->lb3.687\nLightGBM 5fold CV 3.65101-->lb3.687\n", "votes": 3, "replies": [ { "id": 449347, "postDate": "2019-01-03T02:15:42.400Z", "content": "Interesting that Lb is higher/close to cv.
", "rawMarkdown": "Interesting that Lb is higher/close to cv." }, { "id": 450421, "postDate": "2019-01-04T22:52:30.133Z", "content": "I am new to Kaggle, could you share some of your idea on hyper-p tunning?
", "rawMarkdown": "I am new to Kaggle, could you share some of your idea on hyper-p tunning?" }, { "id": 452128, "postDate": "2019-01-08T08:32:26.870Z", "content": "https://medium.com/district-data-labs/parameter-tuning-with-hyperopt-faa86acdfdce
", "rawMarkdown": "https://medium.com/district-data-labs/parameter-tuning-with-hyperopt-faa86acdfdce" }, { "id": 452731, "postDate": "2019-01-09T04:59:44.747Z", "content": "Thank you Manoj!
", "rawMarkdown": "Thank you Manoj!", "votes": 1 } ] }, { "id": 463821, "postDate": "2019-01-30T17:13:53.813Z", "content": "30 features learning rate 0.1 no post-processing\nLB 3.686
", "rawMarkdown": "30 features learning rate 0.1 no post-processing\nLB 3.686\n", "votes": 4, "replies": [ { "id": 463830, "postDate": "2019-01-30T17:24:52.283Z", "content": "awesome !. less is more :)
", "rawMarkdown": "awesome !. less is more :)" } ] }, { "id": 436151, "postDate": "2018-12-09T17:33:39.457Z", "content": "LightGBM 5fold CV 3.64841 -> LB 3.691.
\n\nI struggle to make stacking works in this competition so far, so perhaps working on a good single model might goes a long way in this competition :)
", "rawMarkdown": "LightGBM 5fold CV 3.64841 -> LB 3.691.\n\nI struggle to make stacking works in this competition so far, so perhaps working on a good single model might goes a long way in this competition :)", "votes": 3 }, { "id": 433188, "postDate": "2018-12-04T20:18:34.787Z", "content": "Update: 3.707 based on this kernel https://www.kaggle.com/konradb/lgb-fe-lb-3-707
", "rawMarkdown": "Update: 3.707 based on this kernel https://www.kaggle.com/konradb/lgb-fe-lb-3-707", "votes": 3 }, { "id": 432723, "postDate": "2018-12-04T08:52:02.693Z", "content": "3.658x on 5fold CatBoost => 3.74 on LB.\nMy current best LB model (3.71) has 3.67 on CV.\nIt's my first Kaggle competition, so I was quite concerned before I read this topic.
", "rawMarkdown": "3.658x on 5fold CatBoost => 3.74 on LB.\nMy current best LB model (3.71) has 3.67 on CV.\nIt's my first Kaggle competition, so I was quite concerned before I read this topic.", "votes": 3, "replies": [ { "id": 432736, "postDate": "2018-12-04T09:08:46.370Z", "content": "I'll give catboost a try but not sure about parameters. I've not used it too much not familiar with cb parameters. Did you run any parameter search?
", "rawMarkdown": "I'll give catboost a try but not sure about parameters. I've not used it too much not familiar with cb parameters. Did you run any parameter search? ", "votes": 2 }, { "id": 432759, "postDate": "2018-12-04T09:33:32.810Z", "content": "I'm using CatBoost mainly because it handles categorical features pretty well. As I'm new to all that, I decided not to waste time on trying different methods of encoding, but let CatBost do its job.\nCatBoost is fast on GPU, so I've been able to try a lot of parameters. The most useful for CV score were decreasing learning rate and increasing regularization parameter. But I got much worse scores on LB :)
", "rawMarkdown": "I'm using CatBoost mainly because it handles categorical features pretty well. As I'm new to all that, I decided not to waste time on trying different methods of encoding, but let CatBost do its job.\nCatBoost is fast on GPU, so I've been able to try a lot of parameters. The most useful for CV score were decreasing learning rate and increasing regularization parameter. But I got much worse scores on LB :)", "votes": 1 }, { "id": 434047, "postDate": "2018-12-05T20:24:22.653Z", "content": "I WAS VERY WRONG ABOUT CATBOOST!
\n\nI got 3.695 on LB with same features with LGB instead of 3.71 with CatBoost. So think twice before using it.
", "rawMarkdown": "I WAS VERY WRONG ABOUT CATBOOST!\n\nI got 3.695 on LB with same features with LGB instead of 3.71 with CatBoost. So think twice before using it.", "votes": 2 } ] }, { "id": 432343, "postDate": "2018-12-03T18:35:13.017Z", "content": "Hello, I got CV: 3.666426 and LB: 3.705 but also CV: 3.66377 and LB: 3.708 :/
", "rawMarkdown": "Hello, I got CV: 3.666426 and LB: 3.705 but also CV: 3.66377 and LB: 3.708 :/", "votes": 3, "replies": [ { "id": 432346, "postDate": "2018-12-03T18:39:42.707Z", "content": "Same here. Added some features, lower RMSE on local and got worse score on LB. Can't understand lol
", "rawMarkdown": "Same here. Added some features, lower RMSE on local and got worse score on LB. Can't understand lol ", "votes": 4 }, { "id": 434441, "postDate": "2018-12-06T12:08:38.813Z", "content": "Adding a new feature greatly affects the model. The feature can be strong, but the parameters of the old model may not allow to see its positive effect.
", "rawMarkdown": "Adding a new feature greatly affects the model. The feature can be strong, but the parameters of the old model may not allow to see its positive effect." } ] }, { "id": 437758, "postDate": "2018-12-12T12:56:50.283Z", "content": "10 fold, CV: 3.64342, LB: 3.694
", "rawMarkdown": "10 fold, CV: 3.64342, LB: 3.694", "votes": 4 }, { "id": 433374, "postDate": "2018-12-05T02:26:55.683Z", "content": "5fold lgb with cv: 3.650, lb: 3.697
", "rawMarkdown": "5fold lgb with cv: 3.650, lb: 3.697 ", "votes": 4 }, { "id": 463796, "postDate": "2019-01-30T16:22:07.583Z", "content": "CV 3.655 LB 3.678
", "rawMarkdown": "CV 3.655 LB 3.678", "votes": 1, "replies": [ { "id": 463804, "postDate": "2019-01-30T16:32:36.290Z", "content": "impressive result, thank you for sharing
", "rawMarkdown": "impressive result, thank you for sharing" }, { "id": 463812, "postDate": "2019-01-30T16:46:43.703Z", "content": "Nice result. Mine is Cv 3.651 and Lb 3.685. Could you tell, how many features you used ?
", "rawMarkdown": "Nice result. Mine is Cv 3.651 and Lb 3.685. Could you tell, how many features you used ?" }, { "id": 463815, "postDate": "2019-01-30T16:53:54.580Z", "content": "between 200 and 230 features，all features are aggs
", "rawMarkdown": "between 200 and 230 features，all features are aggs" } ] }, { "id": 438099, "postDate": "2018-12-13T05:11:42.373Z", "content": "LightGBM 5fold CV 3.6598 -> LB 3.692.
\n\nNote: I've had local runs with CV 3.6525 -> LB 3.695 to 3.698 as well. Anyone have tips on how to make local CV move in the same direction as LB?
", "rawMarkdown": "LightGBM 5fold CV 3.6598 -> LB 3.692.\n\nNote: I've had local runs with CV 3.6525 -> LB 3.695 to 3.698 as well. Anyone have tips on how to make local CV move in the same direction as LB?", "votes": 1, "replies": [ { "id": 438655, "postDate": "2018-12-14T01:53:37.837Z", "content": "is your result reproducible ?
", "rawMarkdown": "is your result reproducible ? " }, { "id": 438838, "postDate": "2018-12-14T09:09:19.150Z", "content": "Yes, assuming I run the exact same version of the model. From future iterations I got the same score 3 more times and with different CV for each of them ranging from 3.6515 to 3.6598.
", "rawMarkdown": "Yes, assuming I run the exact same version of the model. From future iterations I got the same score 3 more times and with different CV for each of them ranging from 3.6515 to 3.6598." }, { "id": 438873, "postDate": "2018-12-14T10:11:30.260Z", "content": "I am not able to reproduce..any hint would be appreciated..
", "rawMarkdown": "I am not able to reproduce..any hint would be appreciated.." } ] }, { "id": 435587, "postDate": "2018-12-08T10:22:29.787Z", "content": "LightGBM with 5 folds: CV: 3.65274 → LB: 3.699
", "rawMarkdown": "LightGBM with 5 folds: CV: 3.65274 → LB: 3.699", "votes": 1 }, { "id": 434688, "postDate": "2018-12-06T19:45:55.103Z", "content": "5 Fold LGB ------> CV: 3.67083 LB: 3.70
", "rawMarkdown": "5 Fold LGB ------> CV: 3.67083 LB: 3.70", "votes": 1, "replies": [ { "id": 434707, "postDate": "2018-12-06T20:32:26.410Z", "content": "Wow, that's right on :O
", "rawMarkdown": "Wow, that's right on :O", "votes": 1 } ] }, { "id": 434439, "postDate": "2018-12-06T11:59:36.980Z", "content": "5 fold LGB with CV: 3.6724, LB: 3.700
", "rawMarkdown": "5 fold LGB with CV: 3.6724, LB: 3.700", "votes": 1 }, { "id": 434309, "postDate": "2018-12-06T07:22:04.107Z", "content": "5fold lgb with cv: 3.6611, lb: 3.704
", "rawMarkdown": "5fold lgb with cv: 3.6611, lb: 3.704", "votes": 1 }, { "id": 459606, "postDate": "2019-01-22T04:17:35.020Z", "content": "CV 3.651 LB 3.685
", "rawMarkdown": "CV 3.651 LB 3.685", "votes": 2 }, { "id": 441511, "postDate": "2018-12-18T18:14:35.477Z", "content": "5 fold Light GBM. CV mean score: 3.6549 and LB: 3.698
\n\n", "rawMarkdown": "5 fold Light GBM. CV mean score: 3.6549 and LB: 3.698\n\nhttps://www.kaggle.com/roydatascience/elo-merchant-recommendation-fathers-day-specials?scriptVersionId=8424127", "votes": 2 }, { "id": 435441, "postDate": "2018-12-08T03:24:00.017Z", "content": "5 Fold LGB ------> CV: 3.65650 LB: 3.690\n5 Fold LGB ------> CV:3.65381 LB: 3.691\nI don't have good offline consistency here.
", "rawMarkdown": "5 Fold LGB ------> CV: 3.65650 LB: 3.690\n5 Fold LGB ------> CV:3.65381 LB: 3.691\nI don't have good offline consistency here.", "votes": 2 }, { "id": 433195, "postDate": "2018-12-04T20:27:59.130Z", "content": "My most successful single model is a CatBoost with a CV of 3.6674 which translates to a 3.706 LB. Still ways to go!
", "rawMarkdown": "My most successful single model is a CatBoost with a CV of 3.6674 which translates to a 3.706 LB. Still ways to go!", "votes": 2 }, { "id": 432711, "postDate": "2018-12-04T08:31:38.550Z", "content": "10fold lgb ,cv: 3.6678 lb: 3.709
", "rawMarkdown": "10fold lgb ,cv: 3.6678 lb: 3.709\n", "votes": 2 }, { "id": 465107, "postDate": "2019-02-02T10:32:47.320Z", "content": "CV 3.6360, LB 3.679.
\nI guess this is nearly the best cv score open in disucssion?
Hi all, Is anyone who can hit 3.680± based on single model ,then willing to team up with us ? we need more features but not ensemble&post_processing
", "rawMarkdown": "Hi all, Is anyone who can hit 3.680± based on single model ,then willing to team up with us ? we need more features but not ensemble&post_processing", "replies": [ { "id": 462852, "postDate": "2019-01-29T02:31:48.500Z", "content": "We need new features too, we have two positions.
", "rawMarkdown": "We need new features too, we have two positions.", "votes": 1 }, { "id": 462857, "postDate": "2019-01-29T02:39:47.940Z", "content": "@SomethingIsWrong:\nWhat is your single model LGB score in terms of CV and LB.? My best single model is: 3.680.(LB).
", "rawMarkdown": "@SomethingIsWrong:\nWhat is your single model LGB score in terms of CV and LB.? My best single model is: 3.680.(LB)." }, { "id": 462938, "postDate": "2019-01-29T06:17:31.927Z", "content": "haha our team only have 1 position,please let me know if someone willing to join us
", "rawMarkdown": "haha our team only have 1 position,please let me know if someone willing to join us" }, { "id": 462940, "postDate": "2019-01-29T06:18:04.787Z", "content": "and @SubikashPal ,may u wanna team up with us?
", "rawMarkdown": "and @SubikashPal ,may u wanna team up with us?" }, { "id": 462946, "postDate": "2019-01-29T06:27:50.007Z", "content": "let me send u a message.
", "rawMarkdown": "let me send u a message." }, { "id": 463046, "postDate": "2019-01-29T09:55:51.463Z", "content": "I haven't received ur email now, please let me know if u have send
", "rawMarkdown": "I haven't received ur email now, please let me know if u have send" }, { "id": 463281, "postDate": "2019-01-29T18:29:04.627Z", "content": "tried to send merge request but probably wouldn't be able to do as combined submission exceeds the limit.\nbad luck for me.
", "rawMarkdown": "tried to send merge request but probably wouldn't be able to do as combined submission exceeds the limit.\nbad luck for me." }, { "id": 463440, "postDate": "2019-01-30T01:37:42.957Z", "content": "haha it's really a pity for us....looking forward to future cooperation....
", "rawMarkdown": "haha it's really a pity for us....looking forward to future cooperation...." } ] }, { "id": 459773, "postDate": "2019-01-22T10:48:40.303Z", "content": "I'm at CV 3.657 -> LB 3.696 running on Kaggle Kernels, same code CV 3.654 -> LB 3.692 running on my server.
\n\nUpdate: same model now at CV 3.6454 LB 3.686 after changing one silly little thing in FE code.
", "rawMarkdown": "I'm at CV 3.657 -> LB 3.696 running on Kaggle Kernels, same code CV 3.654 -> LB 3.692 running on my server.\n\nUpdate: same model now at CV 3.6454 LB 3.686 after changing one silly little thing in FE code. ", "replies": [ { "id": 459877, "postDate": "2019-01-22T14:51:03.697Z", "content": "What do you think is causing this difference in running by Kaggle or by your server?
", "rawMarkdown": "What do you think is causing this difference in running by Kaggle or by your server?" }, { "id": 459880, "postDate": "2019-01-22T14:58:29.357Z", "content": "This is really strange !, you may wanna check the version of your major package
", "rawMarkdown": "This is really strange !, you may wanna check the version of your major package " }, { "id": 459889, "postDate": "2019-01-22T15:22:28.680Z", "content": "If you are using seed , can u remove the same and see the behavior?
", "rawMarkdown": "If you are using seed , can u remove the same and see the behavior?" }, { "id": 459969, "postDate": "2019-01-22T18:13:59.203Z", "content": "It's very odd. Maybe different library versions, I'll need to check. It also may be related to the way I'm doing feature engineering ... looking into it
", "rawMarkdown": "It's very odd. Maybe different library versions, I'll need to check. It also may be related to the way I'm doing feature engineering ... looking into it" }, { "id": 460091, "postDate": "2019-01-23T01:35:47.953Z", "content": "maybe the diff datetime.datetime.today() cause that
", "rawMarkdown": "maybe the diff datetime.datetime.today() cause that" }, { "id": 461954, "postDate": "2019-01-27T10:21:54.457Z", "content": "Yep, that was it. Playing with date information provided a small improvement but it won't help me break 3.69 with a single model. My best model is now 3.691 on LB.
\n\nI haven't looked at interactions between categorical features (CountVectoriers, FFM, ...), maybe that's the key to the next jump. But right now that seems too obvious to be that important, there are too many spooky things with this dataset like how sum(category_1) on historical transactions improves CV by -0.006 but makes LB worse by +0.004.
", "rawMarkdown": "Yep, that was it. Playing with date information provided a small improvement but it won't help me break 3.69 with a single model. My best model is now 3.691 on LB.\n\nI haven't looked at interactions between categorical features (CountVectoriers, FFM, ...), maybe that's the key to the next jump. But right now that seems too obvious to be that important, there are too many spooky things with this dataset like how sum(category_1) on historical transactions improves CV by -0.006 but makes LB worse by +0.004." }, { "id": 461970, "postDate": "2019-01-27T10:58:02.857Z", "content": "may i know the best score u hit based on single model?
", "rawMarkdown": "may i know the best score u hit based on single model?", "votes": 1 }, { "id": 461976, "postDate": "2019-01-27T11:08:34.497Z", "content": "I'm currently stuck at LB 3.691.
", "rawMarkdown": "I'm currently stuck at LB 3.691. " }, { "id": 461978, "postDate": "2019-01-27T11:16:12.257Z", "content": "that's fine but can not work for me....thanks
", "rawMarkdown": "that's fine but can not work for me....thanks" }, { "id": 461980, "postDate": "2019-01-27T11:17:37.127Z", "content": "@SubikashPal hi SubikashPal ,may i know the score u hit based on single model?
", "rawMarkdown": "@SubikashPal hi SubikashPal ,may i know the score u hit based on single model?" }, { "id": 462013, "postDate": "2019-01-27T13:23:55.350Z", "content": "3.81 LB...
", "rawMarkdown": "3.81 LB..." }, { "id": 462020, "postDate": "2019-01-27T14:01:32.730Z", "content": "u used ensemble to hit 3.673 based on 3.681.....maybe i can hit 3.665 based on this haha
", "rawMarkdown": "u used ensemble to hit 3.673 based on 3.681.....maybe i can hit 3.665 based on this haha" }, { "id": 462023, "postDate": "2019-01-27T14:06:58.213Z", "content": "Waoo.. You guys are so good .I have used Sample kernel to do post processing. Did not spend too much time there. \nStill trying to improve Single model, My guess is that its possible to achieve 3.67* using single model.\nBTW whats your single model score?
", "rawMarkdown": "Waoo.. You guys are so good .I have used Sample kernel to do post processing. Did not spend too much time there. \nStill trying to improve Single model, My guess is that its possible to achieve 3.67* using single model.\nBTW whats your single model score? " }, { "id": 462030, "postDate": "2019-01-27T14:15:41.380Z", "content": "only 3.685 and we need more features now, i haven't try a lot but no gains haha
", "rawMarkdown": "only 3.685 and we need more features now, i haven't try a lot but no gains haha" }, { "id": 462050, "postDate": "2019-01-27T15:00:15.610Z", "content": "so ensemble or post processing gave u so much gain?
", "rawMarkdown": "so ensemble or post processing gave u so much gain?" }, { "id": 462254, "postDate": "2019-01-28T01:14:05.833Z", "content": "not all in ensemble but any other tricks
", "rawMarkdown": "not all in ensemble but any other tricks" } ] }, { "id": 459538, "postDate": "2019-01-22T00:15:39.947Z", "content": "Got 3.63029 CV and 3.700 LB w/ xgboost. Time to move on to another model..
", "rawMarkdown": "Got 3.63029 CV and 3.700 LB w/ xgboost. Time to move on to another model.." }, { "id": 458394, "postDate": "2019-01-19T14:12:21.077Z", "content": "5 Fold LGB\n CV:3.66700 LB:3.692
", "rawMarkdown": "5 Fold LGB\n CV:3.66700 LB:3.692" }, { "id": 456916, "postDate": "2019-01-16T18:50:12.340Z", "content": "i apologize for asking this stupid question. please excuse me(im a beginner to machine learning) and hopefuly you can explain what i am doing incorrectly.
\n\nhow is everyone calculating the cv score. my cv scores are coming as ~0.06 but it gives me a LB of ~3.9. how is everyones cv close to LB?
\n\nmy code for calculating cv is below
\n\nfrom sklearn.model_selection import cross_val_score
\n\ncv_score = cross_val_score(search.best_estimator_, \\\n x_train,\\\n y_train,\\\n cv = cv,\n scoring = score\n )
\n\nbest_estimator : is a XGBRegressor in my case
\n\ncv = KFold
\n\nscoring: mean_squared_error (i just used a fancy make_scorer object)
", "rawMarkdown": "i apologize for asking this stupid question. please excuse me(im a beginner to machine learning) and hopefuly you can explain what i am doing incorrectly.\n\n\nhow is everyone calculating the cv score. my cv scores are coming as ~0.06 but it gives me a LB of ~3.9. how is everyones cv close to LB?\n\nmy code for calculating cv is below\n\nfrom sklearn.model_selection import cross_val_score\n\ncv_score = cross_val_score(search.best_estimator_, \\\n x_train,\\\n y_train,\\\n cv = cv,\n scoring = score\n )\n\n\nbest_estimator : is a XGBRegressor in my case\n\ncv = KFold\n\nscoring: mean__squared__error (i just used a fancy make_scorer object)\n", "replies": [ { "id": 456931, "postDate": "2019-01-16T19:08:23.673Z", "content": "You are using MAE, instead you should use MSE. Just change the scoring to 'meansquarederror' and you should get the consistent result with LB. \nhttps://medium.com/human-in-a-machine-world/mae-and-rmse-which-metric-is-better-e60ac3bde13d\nDifference in MSE and MAE
hi, thnx for replying. iactually used 'mean_squared_error'.. let me update the comment..i mistyped.
\n\neven after using mse , i still get cv scores close to 0.06. whereas lb is 3.9 :(
", "rawMarkdown": "hi, thnx for replying. iactually used 'mean_squared_error'.. let me update the comment..i mistyped.\n\neven after using mse , i still get cv scores close to 0.06. whereas lb is 3.9 :(" }, { "id": 456941, "postDate": "2019-01-16T19:16:00.353Z", "content": "Can you upload the snapshot of your metric?
", "rawMarkdown": "Can you upload the snapshot of your metric?" }, { "id": 456943, "postDate": "2019-01-16T19:18:34.353Z", "content": "i am sending the results from my gridsearch using this particular dataset.
\n\ni used 10 fold cross validation.
\n\n{'boost_regressor': {'training_time': 75375.482, 'testing_time': 24.132999999991966, 'r2_score': -0.05332302990461146, 'mean_squared_error': 0.06819035509887827, 'root_mean_squared_error': 0.06636031940833483, 'best_estimator': Pipeline(memory=None,\n steps=[('scaler', StandardScaler(copy=True, with_mean=True, with_std=True)), ('boost_regressor', XGBRegressor(base_score=0.5, booster='gblinear', colsample_bylevel=0.8,\n colsample_bytree=1, gamma=0, learning_rate=0.02, max_delta_step=0,\n max_depth=8, min_child_weight=1, missing=None, n_estimators=100,\n n_jobs=1, nthread=None, objective='reg:linear', random_state=0,\n reg_alpha=0, reg_lambda=1, scale_pos_weight=1, seed=None,\n silent=True, subsample=0.8))]), 'cv_score': array([0.06616927, 0.06745297, 0.06768812, 0.06574583, 0.06672648,\n 0.06410755, 0.06530131, 0.06701193, 0.06513149, 0.06826817])}}
", "rawMarkdown": "i am sending the results from my gridsearch using this particular dataset.\n\ni used 10 fold cross validation.\n\n{'boost_regressor': {'training_time': 75375.482, 'testing_time': 24.132999999991966, 'r2_score': -0.05332302990461146, 'mean_squared_error': 0.06819035509887827, 'root_mean_squared_error': 0.06636031940833483, 'best_estimator': Pipeline(memory=None,\n steps=[('scaler', StandardScaler(copy=True, with_mean=True, with_std=True)), ('boost_regressor', XGBRegressor(base_score=0.5, booster='gblinear', colsample_bylevel=0.8,\n colsample_bytree=1, gamma=0, learning_rate=0.02, max_delta_step=0,\n max_depth=8, min_child_weight=1, missing=None, n_estimators=100,\n n_jobs=1, nthread=None, objective='reg:linear', random_state=0,\n reg_alpha=0, reg_lambda=1, scale_pos_weight=1, seed=None,\n silent=True, subsample=0.8))]), 'cv_score': array([0.06616927, 0.06745297, 0.06768812, 0.06574583, 0.06672648,\n 0.06410755, 0.06530131, 0.06701193, 0.06513149, 0.06826817])}}" }, { "id": 456948, "postDate": "2019-01-16T19:23:00.370Z", "content": "Are you scaling your target variable too?
", "rawMarkdown": "Are you scaling your target variable too?", "votes": 1 }, { "id": 456954, "postDate": "2019-01-16T19:32:03.343Z", "content": "you are a rockstar!! i was using Pipeline object of sklearn.. living in delirium that it would apply the pipeline to given y values as well. whereas pipeline only applies it on x values. let me run similar scaling on y values and get back to you. thank you thank you thank you!!!
", "rawMarkdown": "you are a rockstar!! i was using Pipeline object of sklearn.. living in delirium that it would apply the pipeline to given y values as well. whereas pipeline only applies it on x values. let me run similar scaling on y values and get back to you. thank you thank you thank you!!!" } ] }, { "id": 451763, "postDate": "2019-01-07T16:18:40.847Z", "content": "LightGBM 5fold CV 3.64593 --> lb 3.695
\n\nGap is huge. Whats going wrong ?, How can i reduce the gap ?
", "rawMarkdown": "LightGBM 5fold CV 3.64593 --> lb 3.695\n\nGap is huge. Whats going wrong ?, How can i reduce the gap ?", "replies": [ { "id": 455323, "postDate": "2019-01-13T15:53:47.657Z", "content": "LightGBM 5fold CV 3.64438 --> lb 3.691
", "rawMarkdown": "LightGBM 5fold CV 3.64438 --> lb 3.691" }, { "id": 455335, "postDate": "2019-01-13T16:12:16.267Z", "content": "So how did you reduce the gap? lol
", "rawMarkdown": "So how did you reduce the gap? lol " }, { "id": 455497, "postDate": "2019-01-14T03:56:31.690Z", "content": "`from scipy.stats import ks_2samp\n list_p_value =[]\n for i in tqdm(df_train_columns):\n list_p_value.append(ks_2samp(df_test[i] , df_train[i])[1]) \n Se = pd.Series(list_p_value, index = df_train_columns).sort_values() \n list_discarded = list(Se[Se < .1].index)`\ni removed features in \"list_discarded\". Although gap is still quite high :(
", "rawMarkdown": "i removed features in \"list_discarded\". Although gap is still quite high :(", "votes": 1 }, { "id": 455535, "postDate": "2019-01-14T06:06:04.150Z", "content": "Smart way to selecting features, I'll give it a try. Thanks for the code
", "rawMarkdown": "Smart way to selecting features, I'll give it a try. Thanks for the code " } ] }, { "id": 450954, "postDate": "2019-01-06T06:09:30.143Z", "content": "what's the meaning of \"single model\"? without any post-processing for outliers?
", "rawMarkdown": "what's the meaning of \"single model\"? without any post-processing for outliers?", "replies": [ { "id": 450993, "postDate": "2019-01-06T08:30:23.363Z", "content": "Let's say without anything. Blending, stacking, post-processing etc.
", "rawMarkdown": "Let's say without anything. Blending, stacking, post-processing etc." } ] }, { "id": 450535, "postDate": "2019-01-05T07:40:19.513Z", "content": "Best Single Model :\nLightGBM 5 fold CV 3.64771 and LB 0.3695\nrest is about stacking/blending and post processing as discussed on :https://www.kaggle.com/waitingli/combining-your-model-with-a-model-without-outlier/comments#442188
", "rawMarkdown": "Best Single Model :\nLightGBM 5 fold CV 3.64771 and LB 0.3695\nrest is about stacking/blending and post processing as discussed on :https://www.kaggle.com/waitingli/combining-your-model-with-a-model-without-outlier/comments#442188", "replies": [ { "id": 455504, "postDate": "2019-01-14T04:13:05.510Z", "content": "LB 3.695 -->> 3.671 is achieved through stacking and post processing ? !!
", "rawMarkdown": "LB 3.695 -->> 3.671 is achieved through stacking and post processing ? !!" } ] }, { "id": 449867, "postDate": "2019-01-03T22:02:21.490Z", "content": "xgb,cv3.650,lb3.695
", "rawMarkdown": "xgb,cv3.650,lb3.695", "replies": [ { "id": 449957, "postDate": "2019-01-04T02:52:16.037Z", "content": "xgb 5 fold cv 3.652, lb3.693
", "rawMarkdown": "xgb 5 fold cv 3.652, lb3.693" }, { "id": 452111, "postDate": "2019-01-08T07:49:59.227Z", "content": "your cv has increased and lb improved. how are you achieving this ?
", "rawMarkdown": "your cv has increased and lb improved. how are you achieving this ? " }, { "id": 452136, "postDate": "2019-01-08T08:44:52.770Z", "content": "feature selection maybe?
", "rawMarkdown": "feature selection maybe?", "votes": 1 } ] }, { "id": 448835, "postDate": "2019-01-02T07:42:52.533Z", "content": "5Fold LGB CV score 3.651 and LB 3.693
", "rawMarkdown": "5Fold LGB CV score 3.651 and LB 3.693", "replies": [ { "id": 448956, "postDate": "2019-01-02T12:37:01.627Z", "content": "Great job. is there any specific feature that made such a progress?
", "rawMarkdown": "Great job. is there any specific feature that made such a progress?" }, { "id": 448961, "postDate": "2019-01-02T12:45:13.853Z", "content": "Thanks. Seems like purchase amount features have high split and gain for my model & data.
", "rawMarkdown": "Thanks. Seems like purchase amount features have high split and gain for my model & data.", "votes": 1 }, { "id": 448962, "postDate": "2019-01-02T12:51:07.250Z", "content": "Thank you. Are u using count vectorizer for features?
", "rawMarkdown": "Thank you. Are u using count vectorizer for features?" }, { "id": 448968, "postDate": "2019-01-02T13:02:42.360Z", "content": "Nope, just aggregated features. Haven't tried cv yet. Want to try CV and FFM/FM for interactions...
", "rawMarkdown": "Nope, just aggregated features. Haven't tried cv yet. Want to try CV and FFM/FM for interactions..." }, { "id": 448998, "postDate": "2019-01-02T14:04:04.660Z", "content": "Great. Only using aggregated features gives u such a boost. Feature selection?
", "rawMarkdown": "Great. Only using aggregated features gives u such a boost. Feature selection?" }, { "id": 449065, "postDate": "2019-01-02T15:41:50.163Z", "content": "Yes, I've done some feature selection. IMHO exatcly less is more for this competition.
", "rawMarkdown": "Yes, I've done some feature selection. IMHO exatcly [less is more](https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73937) for this competition. " }, { "id": 451186, "postDate": "2019-01-06T14:43:18.377Z", "content": "@SubikashPal Hi Subikash. I was curious about how count vectorizer could be used in this context as based on my understanding count vectorizer is applicable for text features.
\n\nPardon me if the question seems basic as I am a beginner in this field.
", "rawMarkdown": "@SubikashPal Hi Subikash. I was curious about how count vectorizer could be used in this context as based on my understanding count vectorizer is applicable for text features.\n\nPardon me if the question seems basic as I am a beginner in this field." }, { "id": 451194, "postDate": "2019-01-06T14:59:06.270Z", "content": "I had no success using the count vectorizer here. I am not using now.
", "rawMarkdown": "I had no success using the count vectorizer here. I am not using now. " }, { "id": 459654, "postDate": "2019-01-22T06:14:48.960Z", "content": "@Anurag, you can count how many times each factor level of category_2 etc have been seen across each card_id. basically, groupby transactions data using card_id and 'category_2' and count for each level of category_2. Can do this for other features as well.
", "rawMarkdown": "@Anurag, you can count how many times each factor level of category_2 etc have been seen across each card_id. basically, groupby transactions data using card_id and 'category_2' and count for each level of category_2. Can do this for other features as well. " }, { "id": 459927, "postDate": "2019-01-22T16:23:26.743Z", "content": "Thanks Harshit for your response. This helps clarify my doubt.
", "rawMarkdown": "Thanks Harshit for your response. This helps clarify my doubt." } ] }, { "id": 436582, "postDate": "2018-12-10T15:21:20.720Z", "content": "3.648 on a 20 % held out test set. \n3.708 in 3 fold CV for model selection.\n3.742 LB.\nSingle model
", "rawMarkdown": "3.648 on a 20 % held out test set. \n3.708 in 3 fold CV for model selection.\n3.742 LB.\nSingle model" }, { "id": 434823, "postDate": "2018-12-07T02:19:22.550Z", "content": "5 fold LFB. CV mean score: 3.6634, std: 0.0171. LB: 3.71
", "rawMarkdown": "5 fold LFB. CV mean score: 3.6634, std: 0.0171. LB: 3.71" }, { "id": 464025, "postDate": "2019-01-31T04:31:41.763Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 452057, "postDate": "2019-01-08T05:58:57.787Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 452159, "postDate": "2019-01-08T09:46:35.367Z", "content": "My LB score is 3.692. can we have a talk on Wechat(shiganquan) ? I'm a student at ECNU.
", "rawMarkdown": "My LB score is 3.692. can we have a talk on Wechat(shiganquan) ? I'm a student at ECNU." }, { "id": 453728, "postDate": "2019-01-10T17:12:15.970Z", "content": "cv 3.651 lb 3.688, single best model
", "rawMarkdown": "cv 3.651 lb 3.688, single best model" }, { "id": 459211, "postDate": "2019-01-21T11:49:19.857Z", "content": "cv 3.650 lb 3.692, single best model
", "rawMarkdown": "cv 3.650 lb 3.692, single best model" } ] } ], "comments": [ { "id": 432389, "author_name": "whamp", "author_url": "", "post_date": "2018-12-03T19:45:48.200000", "content": "it's the impact of the negative outliers. 98% of observations are between -1 and 1. With a chunk of outliers with target = -33, the squared error is enormous. Thus the goal is predict ourliers accurately, not the 98% of the rest of the data.
", "votes": 8, "replies": [] }, { "id": 451276, "author_name": "SubikashPal", "author_url": "", "post_date": "2019-01-06T18:48:49.327000", "content": "5 Fold LGB ------> CV: 3.65510 LB: 3.683\n( No feature elimination , no outlier handling , No count Vector , no FM - Just aggregated features and elapsed time)
", "votes": 5, "replies": [ { "id": 451345, "author_name": "HenryWu", "author_url": "", "post_date": "2019-01-06T22:42:14.887000", "content": "3.683? That was amazing! The improvement is from parameter tuning?
", "votes": 0, "replies": [] }, { "id": 451520, "author_name": "Manoj", "author_url": "", "post_date": "2019-01-07T08:18:36.753000", "content": "Wow, Gap is very small !!
", "votes": 0, "replies": [] }, { "id": 452059, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-08T06:00:53.773000", "content": "That is really amazing, I haven't do (FM/outlier) but I only got 3.687.\nMay we have a talk?
", "votes": 0, "replies": [] }, { "id": 452451, "author_name": "HenryWu", "author_url": "", "post_date": "2019-01-08T18:35:02.730000", "content": "Your's is amazing as well! It was single model's result without any stacking? I have even smaller CV but LB is around 3.692. Maybe we can be a team?
", "votes": 0, "replies": [] }, { "id": 452635, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-09T01:16:01.927000", "content": "We can have a talk if u have some insights ?
", "votes": 0, "replies": [] }, { "id": 452729, "author_name": "HenryWu", "author_url": "", "post_date": "2019-01-09T04:59:25.333000", "content": "Can you send me a kaggle message including your contact info so that we can have a talk? I cannot send messages anymore. Thanks!
", "votes": 0, "replies": [] }, { "id": 459250, "author_name": "LongYin/杰少", "author_url": "", "post_date": "2019-01-21T12:56:48.450000", "content": "This is quite amazing, my CV is 3.646, but my LB is only 3.688...........
", "votes": 0, "replies": [] }, { "id": 462876, "author_name": "Manoj", "author_url": "", "post_date": "2019-01-29T03:26:38.767000", "content": "Hi Long Yin, what is your current best single model ?
", "votes": 0, "replies": [] }, { "id": 462927, "author_name": "LongYin/杰少", "author_url": "", "post_date": "2019-01-29T05:53:29.137000", "content": "CV: 3.635, LB: 3.674. The rest is coming from the public kernel and I do some change with the binary classification.
", "votes": 1, "replies": [] }, { "id": 462930, "author_name": "SubikashPal", "author_url": "", "post_date": "2019-01-29T05:56:06.413000", "content": "is this your single model?
", "votes": 0, "replies": [] } ] }, { "id": 438141, "author_name": "Marcus Lin", "author_url": "", "post_date": "2018-12-13T07:00:27.437000", "content": "LightGBM 5fold CV 3.65313-> LB 3.689
", "votes": 5, "replies": [ { "id": 438477, "author_name": "Fredrik Jonsson", "author_url": "", "post_date": "2018-12-13T19:04:44.870000", "content": "Interesting, I also got a 5fold CV 3.653 but it only scored 5fold CV 3.653-> LB 3.703. Looks like there's lots of variation.
", "votes": 1, "replies": [] }, { "id": 438795, "author_name": "Marcus Lin", "author_url": "", "post_date": "2018-12-14T07:38:45.697000", "content": "frankly, I just do a small change of the public kernel.
", "votes": 0, "replies": [] }, { "id": 438909, "author_name": "SubikashPal", "author_url": "", "post_date": "2018-12-14T11:48:04.547000", "content": "did u include merchant info in your model?
", "votes": 0, "replies": [] }, { "id": 439126, "author_name": "Marcus Lin", "author_url": "", "post_date": "2018-12-14T19:18:40.083000", "content": "No, I didn't
", "votes": 0, "replies": [] }, { "id": 441739, "author_name": "Truoc Pham", "author_url": "", "post_date": "2018-12-19T01:52:41.100000", "content": "Hi SomethingIsWrong, how many feature are you using?
", "votes": 0, "replies": [] }, { "id": 441752, "author_name": "Marcus Lin", "author_url": "", "post_date": "2018-12-19T02:40:16.543000", "content": "I use Chau Ngoc Huynh's kernel features. still dealing with gap of cv and LB, haven't add features.
", "votes": 5, "replies": [] }, { "id": 442083, "author_name": "SubikashPal", "author_url": "", "post_date": "2018-12-19T13:16:02.603000", "content": "So LGB params gave u the boost?
", "votes": 5, "replies": [] }, { "id": 446534, "author_name": "Qing Liu", "author_url": "", "post_date": "2018-12-28T08:49:19.787000", "content": "Amazing! Could you share some resources to deal with gap of cv and LB? Many thanks!
", "votes": 5, "replies": [] }, { "id": 459252, "author_name": "LongYin/杰少", "author_url": "", "post_date": "2019-01-21T12:58:03.027000", "content": "Would you mind sharing your CV now?
", "votes": 0, "replies": [] }, { "id": 459339, "author_name": "Marcus Lin", "author_url": "", "post_date": "2019-01-21T15:46:20.713000", "content": "My teammate has better one, mine best one is 3.639.
", "votes": 0, "replies": [] } ] }, { "id": 432508, "author_name": "Ethan", "author_url": "", "post_date": "2018-12-04T01:12:11.490000", "content": "Hi，my CV is 3.65810 and LB:3.697 with 5 fold LGB
", "votes": 5, "replies": [] }, { "id": 449346, "author_name": "ggsddu", "author_url": "", "post_date": "2019-01-03T02:14:10.643000", "content": "LightGBM 5fold CV 3.6532-->lb3.687\nLightGBM 5fold CV 3.65101-->lb3.687
", "votes": 3, "replies": [ { "id": 449347, "author_name": "Fredrik Jonsson", "author_url": "", "post_date": "2019-01-03T02:15:42.400000", "content": "Interesting that Lb is higher/close to cv.
", "votes": 0, "replies": [] }, { "id": 450421, "author_name": "HenryWu", "author_url": "", "post_date": "2019-01-04T22:52:30.133000", "content": "I am new to Kaggle, could you share some of your idea on hyper-p tunning?
", "votes": 0, "replies": [] }, { "id": 452128, "author_name": "Manoj", "author_url": "", "post_date": "2019-01-08T08:32:26.870000", "content": "https://medium.com/district-data-labs/parameter-tuning-with-hyperopt-faa86acdfdce
", "votes": 0, "replies": [] }, { "id": 452731, "author_name": "HenryWu", "author_url": "", "post_date": "2019-01-09T04:59:44.747000", "content": "Thank you Manoj!
", "votes": 1, "replies": [] } ] }, { "id": 463821, "author_name": "snovik", "author_url": "", "post_date": "2019-01-30T17:13:53.813000", "content": "30 features learning rate 0.1 no post-processing\nLB 3.686
", "votes": 4, "replies": [ { "id": 463830, "author_name": "Manoj", "author_url": "", "post_date": "2019-01-30T17:24:52.283000", "content": "awesome !. less is more :)
", "votes": 0, "replies": [] } ] }, { "id": 436151, "author_name": "Yifan Xie", "author_url": "", "post_date": "2018-12-09T17:33:39.457000", "content": "LightGBM 5fold CV 3.64841 -> LB 3.691.
\n\nI struggle to make stacking works in this competition so far, so perhaps working on a good single model might goes a long way in this competition :)
", "votes": 3, "replies": [] }, { "id": 433188, "author_name": "Konrad Banachewicz", "author_url": "", "post_date": "2018-12-04T20:18:34.787000", "content": "Update: 3.707 based on this kernel https://www.kaggle.com/konradb/lgb-fe-lb-3-707
", "votes": 3, "replies": [] }, { "id": 432723, "author_name": "Sergey Sergeev", "author_url": "", "post_date": "2018-12-04T08:52:02.693000", "content": "3.658x on 5fold CatBoost => 3.74 on LB.\nMy current best LB model (3.71) has 3.67 on CV.\nIt's my first Kaggle competition, so I was quite concerned before I read this topic.
", "votes": 3, "replies": [ { "id": 432736, "author_name": "silverstone", "author_url": "", "post_date": "2018-12-04T09:08:46.370000", "content": "I'll give catboost a try but not sure about parameters. I've not used it too much not familiar with cb parameters. Did you run any parameter search?
", "votes": 2, "replies": [] }, { "id": 432759, "author_name": "Sergey Sergeev", "author_url": "", "post_date": "2018-12-04T09:33:32.810000", "content": "I'm using CatBoost mainly because it handles categorical features pretty well. As I'm new to all that, I decided not to waste time on trying different methods of encoding, but let CatBost do its job.\nCatBoost is fast on GPU, so I've been able to try a lot of parameters. The most useful for CV score were decreasing learning rate and increasing regularization parameter. But I got much worse scores on LB :)
", "votes": 1, "replies": [] }, { "id": 434047, "author_name": "Sergey Sergeev", "author_url": "", "post_date": "2018-12-05T20:24:22.653000", "content": "I WAS VERY WRONG ABOUT CATBOOST!
\n\nI got 3.695 on LB with same features with LGB instead of 3.71 with CatBoost. So think twice before using it.
", "votes": 2, "replies": [] } ] }, { "id": 432343, "author_name": "Dimitrios Effrosynidis", "author_url": "", "post_date": "2018-12-03T18:35:13.017000", "content": "Hello, I got CV: 3.666426 and LB: 3.705 but also CV: 3.66377 and LB: 3.708 :/
", "votes": 3, "replies": [ { "id": 432346, "author_name": "silverstone", "author_url": "", "post_date": "2018-12-03T18:39:42.707000", "content": "Same here. Added some features, lower RMSE on local and got worse score on LB. Can't understand lol
", "votes": 4, "replies": [] }, { "id": 434441, "author_name": "Aleksandr Kosolapov", "author_url": "", "post_date": "2018-12-06T12:08:38.813000", "content": "Adding a new feature greatly affects the model. The feature can be strong, but the parameters of the old model may not allow to see its positive effect.
", "votes": 0, "replies": [] } ] }, { "id": 437758, "author_name": "Qing Liu", "author_url": "", "post_date": "2018-12-12T12:56:50.283000", "content": "10 fold, CV: 3.64342, LB: 3.694
", "votes": 4, "replies": [] }, { "id": 433374, "author_name": "Chau Huynh", "author_url": "", "post_date": "2018-12-05T02:26:55.683000", "content": "5fold lgb with cv: 3.650, lb: 3.697
", "votes": 4, "replies": [] }, { "id": 463796, "author_name": "Orange", "author_url": "", "post_date": "2019-01-30T16:22:07.583000", "content": "CV 3.655 LB 3.678
", "votes": 1, "replies": [ { "id": 463804, "author_name": "Konstantin Yakovlev", "author_url": "", "post_date": "2019-01-30T16:32:36.290000", "content": "impressive result, thank you for sharing
", "votes": 0, "replies": [] }, { "id": 463812, "author_name": "HarshitMehta", "author_url": "", "post_date": "2019-01-30T16:46:43.703000", "content": "Nice result. Mine is Cv 3.651 and Lb 3.685. Could you tell, how many features you used ?
", "votes": 0, "replies": [] }, { "id": 463815, "author_name": "Orange", "author_url": "", "post_date": "2019-01-30T16:53:54.580000", "content": "between 200 and 230 features，all features are aggs
", "votes": 0, "replies": [] } ] }, { "id": 438099, "author_name": "Lorenzo Ampil", "author_url": "", "post_date": "2018-12-13T05:11:42.373000", "content": "LightGBM 5fold CV 3.6598 -> LB 3.692.
\n\nNote: I've had local runs with CV 3.6525 -> LB 3.695 to 3.698 as well. Anyone have tips on how to make local CV move in the same direction as LB?
", "votes": 1, "replies": [ { "id": 438655, "author_name": "SubikashPal", "author_url": "", "post_date": "2018-12-14T01:53:37.837000", "content": "is your result reproducible ?
", "votes": 0, "replies": [] }, { "id": 438838, "author_name": "Lorenzo Ampil", "author_url": "", "post_date": "2018-12-14T09:09:19.150000", "content": "Yes, assuming I run the exact same version of the model. From future iterations I got the same score 3 more times and with different CV for each of them ranging from 3.6515 to 3.6598.
", "votes": 0, "replies": [] }, { "id": 438873, "author_name": "SubikashPal", "author_url": "", "post_date": "2018-12-14T10:11:30.260000", "content": "I am not able to reproduce..any hint would be appreciated..
", "votes": 0, "replies": [] } ] }, { "id": 435587, "author_name": "Marnix Koops", "author_url": "", "post_date": "2018-12-08T10:22:29.787000", "content": "LightGBM with 5 folds: CV: 3.65274 → LB: 3.699
", "votes": 1, "replies": [] }, { "id": 434688, "author_name": "akshay", "author_url": "", "post_date": "2018-12-06T19:45:55.103000", "content": "5 Fold LGB ------> CV: 3.67083 LB: 3.70
", "votes": 1, "replies": [ { "id": 434707, "author_name": "Peter Hurford", "author_url": "", "post_date": "2018-12-06T20:32:26.410000", "content": "Wow, that's right on :O
", "votes": 1, "replies": [] } ] }, { "id": 434439, "author_name": "Aleksandr Kosolapov", "author_url": "", "post_date": "2018-12-06T11:59:36.980000", "content": "5 fold LGB with CV: 3.6724, LB: 3.700
", "votes": 1, "replies": [] }, { "id": 434309, "author_name": "Namit Ohri", "author_url": "", "post_date": "2018-12-06T07:22:04.107000", "content": "5fold lgb with cv: 3.6611, lb: 3.704
", "votes": 1, "replies": [] }, { "id": 459606, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-22T04:17:35.020000", "content": "CV 3.651 LB 3.685
", "votes": 2, "replies": [] }, { "id": 441511, "author_name": "Ashish Gupta", "author_url": "", "post_date": "2018-12-18T18:14:35.477000", "content": "5 fold Light GBM. CV mean score: 3.6549 and LB: 3.698
\n\n", "votes": 2, "replies": [] }, { "id": 435441, "author_name": "FISH", "author_url": "", "post_date": "2018-12-08T03:24:00.017000", "content": "5 Fold LGB ------> CV: 3.65650 LB: 3.690\n5 Fold LGB ------> CV:3.65381 LB: 3.691\nI don't have good offline consistency here.
", "votes": 2, "replies": [] }, { "id": 433195, "author_name": "Fredrik Jonsson", "author_url": "", "post_date": "2018-12-04T20:27:59.130000", "content": "My most successful single model is a CatBoost with a CV of 3.6674 which translates to a 3.706 LB. Still ways to go!
", "votes": 2, "replies": [] }, { "id": 432711, "author_name": "Konrad Banachewicz", "author_url": "", "post_date": "2018-12-04T08:31:38.550000", "content": "10fold lgb ,cv: 3.6678 lb: 3.709
", "votes": 2, "replies": [] }, { "id": 465107, "author_name": "pocket", "author_url": "", "post_date": "2019-02-02T10:32:47.320000", "content": "CV 3.6360, LB 3.679.
\nI guess this is nearly the best cv score open in disucssion?
Hi all, Is anyone who can hit 3.680± based on single model ,then willing to team up with us ? we need more features but not ensemble&post_processing
", "votes": 0, "replies": [ { "id": 462852, "author_name": "Marcus Lin", "author_url": "", "post_date": "2019-01-29T02:31:48.500000", "content": "We need new features too, we have two positions.
", "votes": 1, "replies": [] }, { "id": 462857, "author_name": "SubikashPal", "author_url": "", "post_date": "2019-01-29T02:39:47.940000", "content": "@SomethingIsWrong:\nWhat is your single model LGB score in terms of CV and LB.? My best single model is: 3.680.(LB).
", "votes": 0, "replies": [] }, { "id": 462938, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-29T06:17:31.927000", "content": "haha our team only have 1 position,please let me know if someone willing to join us
", "votes": 0, "replies": [] }, { "id": 462940, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-29T06:18:04.787000", "content": "and @SubikashPal ,may u wanna team up with us?
", "votes": 0, "replies": [] }, { "id": 462946, "author_name": "SubikashPal", "author_url": "", "post_date": "2019-01-29T06:27:50.007000", "content": "let me send u a message.
", "votes": 0, "replies": [] }, { "id": 463046, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-29T09:55:51.463000", "content": "I haven't received ur email now, please let me know if u have send
", "votes": 0, "replies": [] }, { "id": 463281, "author_name": "SubikashPal", "author_url": "", "post_date": "2019-01-29T18:29:04.627000", "content": "tried to send merge request but probably wouldn't be able to do as combined submission exceeds the limit.\nbad luck for me.
", "votes": 0, "replies": [] }, { "id": 463440, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-30T01:37:42.957000", "content": "haha it's really a pity for us....looking forward to future cooperation....
", "votes": 0, "replies": [] } ] }, { "id": 459773, "author_name": "Mendrika Ramarlina", "author_url": "", "post_date": "2019-01-22T10:48:40.303000", "content": "I'm at CV 3.657 -> LB 3.696 running on Kaggle Kernels, same code CV 3.654 -> LB 3.692 running on my server.
\n\nUpdate: same model now at CV 3.6454 LB 3.686 after changing one silly little thing in FE code.
", "votes": 0, "replies": [ { "id": 459877, "author_name": "Eric Reis", "author_url": "", "post_date": "2019-01-22T14:51:03.697000", "content": "What do you think is causing this difference in running by Kaggle or by your server?
", "votes": 0, "replies": [] }, { "id": 459880, "author_name": "Manoj", "author_url": "", "post_date": "2019-01-22T14:58:29.357000", "content": "This is really strange !, you may wanna check the version of your major package
", "votes": 0, "replies": [] }, { "id": 459889, "author_name": "SubikashPal", "author_url": "", "post_date": "2019-01-22T15:22:28.680000", "content": "If you are using seed , can u remove the same and see the behavior?
", "votes": 0, "replies": [] }, { "id": 459969, "author_name": "Mendrika Ramarlina", "author_url": "", "post_date": "2019-01-22T18:13:59.203000", "content": "It's very odd. Maybe different library versions, I'll need to check. It also may be related to the way I'm doing feature engineering ... looking into it
", "votes": 0, "replies": [] }, { "id": 460091, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-23T01:35:47.953000", "content": "maybe the diff datetime.datetime.today() cause that
", "votes": 0, "replies": [] }, { "id": 461954, "author_name": "Mendrika Ramarlina", "author_url": "", "post_date": "2019-01-27T10:21:54.457000", "content": "Yep, that was it. Playing with date information provided a small improvement but it won't help me break 3.69 with a single model. My best model is now 3.691 on LB.
\n\nI haven't looked at interactions between categorical features (CountVectoriers, FFM, ...), maybe that's the key to the next jump. But right now that seems too obvious to be that important, there are too many spooky things with this dataset like how sum(category_1) on historical transactions improves CV by -0.006 but makes LB worse by +0.004.
", "votes": 0, "replies": [] }, { "id": 461970, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-27T10:58:02.857000", "content": "may i know the best score u hit based on single model?
", "votes": 1, "replies": [] }, { "id": 461976, "author_name": "Mendrika Ramarlina", "author_url": "", "post_date": "2019-01-27T11:08:34.497000", "content": "I'm currently stuck at LB 3.691.
", "votes": 0, "replies": [] }, { "id": 461978, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-27T11:16:12.257000", "content": "that's fine but can not work for me....thanks
", "votes": 0, "replies": [] }, { "id": 461980, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-27T11:17:37.127000", "content": "@SubikashPal hi SubikashPal ,may i know the score u hit based on single model?
", "votes": 0, "replies": [] }, { "id": 462013, "author_name": "SubikashPal", "author_url": "", "post_date": "2019-01-27T13:23:55.350000", "content": "3.81 LB...
", "votes": 0, "replies": [] }, { "id": 462020, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-27T14:01:32.730000", "content": "u used ensemble to hit 3.673 based on 3.681.....maybe i can hit 3.665 based on this haha
", "votes": 0, "replies": [] }, { "id": 462023, "author_name": "SubikashPal", "author_url": "", "post_date": "2019-01-27T14:06:58.213000", "content": "Waoo.. You guys are so good .I have used Sample kernel to do post processing. Did not spend too much time there. \nStill trying to improve Single model, My guess is that its possible to achieve 3.67* using single model.\nBTW whats your single model score?
", "votes": 0, "replies": [] }, { "id": 462030, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-27T14:15:41.380000", "content": "only 3.685 and we need more features now, i haven't try a lot but no gains haha
", "votes": 0, "replies": [] }, { "id": 462050, "author_name": "SubikashPal", "author_url": "", "post_date": "2019-01-27T15:00:15.610000", "content": "so ensemble or post processing gave u so much gain?
", "votes": 0, "replies": [] }, { "id": 462254, "author_name": "Neo Zhao", "author_url": "", "post_date": "2019-01-28T01:14:05.833000", "content": "not all in ensemble but any other tricks
", "votes": 0, "replies": [] } ] }, { "id": 459538, "author_name": "Eric Reis", "author_url": "", "post_date": "2019-01-22T00:15:39.947000", "content": "Got 3.63029 CV and 3.700 LB w/ xgboost. Time to move on to another model..
", "votes": 0, "replies": [] }, { "id": 458394, "author_name": "fwj7", "author_url": "", "post_date": "2019-01-19T14:12:21.077000", "content": "5 Fold LGB\n CV:3.66700 LB:3.692
", "votes": 0, "replies": [] }, { "id": 456916, "author_name": "somaktukai", "author_url": "", "post_date": "2019-01-16T18:50:12.340000", "content": "i apologize for asking this stupid question. please excuse me(im a beginner to machine learning) and hopefuly you can explain what i am doing incorrectly.
\n\nhow is everyone calculating the cv score. my cv scores are coming as ~0.06 but it gives me a LB of ~3.9. how is everyones cv close to LB?
\n\nmy code for calculating cv is below
\n\nfrom sklearn.model_selection import cross_val_score
\n\ncv_score = cross_val_score(search.best_estimator_, \\\n x_train,\\\n y_train,\\\n cv = cv,\n scoring = score\n )
\n\nbest_estimator : is a XGBRegressor in my case
\n\ncv = KFold
\n\nscoring: mean_squared_error (i just used a fancy make_scorer object)
", "votes": 0, "replies": [ { "id": 456931, "author_name": "akshay", "author_url": "", "post_date": "2019-01-16T19:08:23.673000", "content": "You are using MAE, instead you should use MSE. Just change the scoring to 'meansquarederror' and you should get the consistent result with LB. \nhttps://medium.com/human-in-a-machine-world/mae-and-rmse-which-metric-is-better-e60ac3bde13d\nDifference in MSE and MAE
hi, thnx for replying. iactually used 'mean_squared_error'.. let me update the comment..i mistyped.
\n\neven after using mse , i still get cv scores close to 0.06. whereas lb is 3.9 :(
", "votes": 0, "replies": [] }, { "id": 456941, "author_name": "akshay", "author_url": "", "post_date": "2019-01-16T19:16:00.353000", "content": "Can you upload the snapshot of your metric?
", "votes": 0, "replies": [] }, { "id": 456943, "author_name": "somaktukai", "author_url": "", "post_date": "2019-01-16T19:18:34.353000", "content": "i am sending the results from my gridsearch using this particular dataset.
\n\ni used 10 fold cross validation.
\n\n{'boost_regressor': {'training_time': 75375.482, 'testing_time': 24.132999999991966, 'r2_score': -0.05332302990461146, 'mean_squared_error': 0.06819035509887827, 'root_mean_squared_error': 0.06636031940833483, 'best_estimator': Pipeline(memory=None,\n steps=[('scaler', StandardScaler(copy=True, with_mean=True, with_std=True)), ('boost_regressor', XGBRegressor(base_score=0.5, booster='gblinear', colsample_bylevel=0.8,\n colsample_bytree=1, gamma=0, learning_rate=0.02, max_delta_step=0,\n max_depth=8, min_child_weight=1, missing=None, n_estimators=100,\n n_jobs=1, nthread=None, objective='reg:linear', random_state=0,\n reg_alpha=0, reg_lambda=1, scale_pos_weight=1, seed=None,\n silent=True, subsample=0.8))]), 'cv_score': array([0.06616927, 0.06745297, 0.06768812, 0.06574583, 0.06672648,\n 0.06410755, 0.06530131, 0.06701193, 0.06513149, 0.06826817])}}
", "votes": 0, "replies": [] }, { "id": 456948, "author_name": "akshay", "author_url": "", "post_date": "2019-01-16T19:23:00.370000", "content": "Are you scaling your target variable too?
", "votes": 1, "replies": [] }, { "id": 456954, "author_name": "somaktukai", "author_url": "", "post_date": "2019-01-16T19:32:03.343000", "content": "you are a rockstar!! i was using Pipeline object of sklearn.. living in delirium that it would apply the pipeline to given y values as well. whereas pipeline only applies it on x values. let me run similar scaling on y values and get back to you. thank you thank you thank you!!!
", "votes": 0, "replies": [] } ] }, { "id": 451763, "author_name": "", "author_url": "", "post_date": "2019-01-07T16:18:40.847000", "content": "", "votes": 0, "replies": [ { "id": 455323, "author_name": "", "author_url": "", "post_date": "2019-01-13T15:53:47.657000", "content": "", "votes": 0, "replies": [] }, { "id": 455335, "author_name": "", "author_url": "", "post_date": "2019-01-13T16:12:16.267000", "content": "", "votes": 0, "replies": [] }, { "id": 455497, "author_name": "", "author_url": "", "post_date": "2019-01-14T03:56:31.690000", "content": "", "votes": 4, "replies": [] }, { "id": 455503, "author_name": "", "author_url": "", "post_date": "2019-01-14T04:10:37.717000", "content": "", "votes": 1, "replies": [] }, { "id": 455535, "author_name": "", "author_url": "", "post_date": "2019-01-14T06:06:04.150000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 450954, "author_name": "", "author_url": "", "post_date": "2019-01-06T06:09:30.143000", "content": "", "votes": 0, "replies": [ { "id": 450993, "author_name": "", "author_url": "", "post_date": "2019-01-06T08:30:23.363000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 450535, "author_name": "", "author_url": "", "post_date": "2019-01-05T07:40:19.513000", "content": "", "votes": 0, "replies": [ { "id": 455504, "author_name": "", "author_url": "", "post_date": "2019-01-14T04:13:05.510000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 449867, "author_name": "", "author_url": "", "post_date": "2019-01-03T22:02:21.490000", "content": "", "votes": 0, "replies": [ { "id": 449957, "author_name": "", "author_url": "", "post_date": "2019-01-04T02:52:16.037000", "content": "", "votes": 0, "replies": [] }, { "id": 452111, "author_name": "", "author_url": "", "post_date": "2019-01-08T07:49:59.227000", "content": "", "votes": 0, "replies": [] }, { "id": 452136, "author_name": "", "author_url": "", "post_date": "2019-01-08T08:44:52.770000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 448835, "author_name": "", "author_url": "", "post_date": "2019-01-02T07:42:52.533000", "content": "", "votes": 0, "replies": [ { "id": 448956, "author_name": "", "author_url": "", "post_date": "2019-01-02T12:37:01.627000", "content": "", "votes": 0, "replies": [] }, { "id": 448961, "author_name": "", "author_url": "", "post_date": "2019-01-02T12:45:13.853000", "content": "", "votes": 1, "replies": [] }, { "id": 448962, "author_name": "", "author_url": "", "post_date": "2019-01-02T12:51:07.250000", "content": "", "votes": 0, "replies": [] }, { "id": 448968, "author_name": "", "author_url": "", "post_date": "2019-01-02T13:02:42.360000", "content": "", "votes": 0, "replies": [] }, { "id": 448998, "author_name": "", "author_url": "", "post_date": "2019-01-02T14:04:04.660000", "content": "", "votes": 0, "replies": [] }, { "id": 449065, "author_name": "", "author_url": "", "post_date": "2019-01-02T15:41:50.163000", "content": "", "votes": 0, "replies": [] }, { "id": 451186, "author_name": "", "author_url": "", "post_date": "2019-01-06T14:43:18.377000", "content": "", "votes": 0, "replies": [] }, { "id": 451194, "author_name": "", "author_url": "", "post_date": "2019-01-06T14:59:06.270000", "content": "", "votes": 0, "replies": [] }, { "id": 459654, "author_name": "", "author_url": "", "post_date": "2019-01-22T06:14:48.960000", "content": "", "votes": 0, "replies": [] }, { "id": 459927, "author_name": "", "author_url": "", "post_date": "2019-01-22T16:23:26.743000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 436582, "author_name": "", "author_url": "", "post_date": "2018-12-10T15:21:20.720000", "content": "", "votes": 0, "replies": [] }, { "id": 434823, "author_name": "", "author_url": "", "post_date": "2018-12-07T02:19:22.550000", "content": "", "votes": 0, "replies": [] }, { "id": 464025, "author_name": "", "author_url": "", "post_date": "2019-01-31T04:31:41.763000", "content": "", "votes": 1, "replies": [] }, { "id": 452057, "author_name": "", "author_url": "", "post_date": "2019-01-08T05:58:57.787000", "content": "", "votes": 0, "replies": [ { "id": 452159, "author_name": "", "author_url": "", "post_date": "2019-01-08T09:46:35.367000", "content": "", "votes": 0, "replies": [] }, { "id": 453728, "author_name": "", "author_url": "", "post_date": "2019-01-10T17:12:15.970000", "content": "", "votes": 0, "replies": [] }, { "id": 459211, "author_name": "", "author_url": "", "post_date": "2019-01-21T11:49:19.857000", "content": "", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "432332": "Hi everyone! \nI got 3.66x on local and got 3.708 on LB with 5 fold LGB. What is your best model? ", "432389": "it's the impact of the negative outliers. 98% of observations are between -1 and 1. With a chunk of outliers with target = -33, the squared error is enormous. Thus the goal is predict ourliers accurately, not the 98% of the rest of the data. ", "451276": "5 Fold LGB ------> CV: 3.65510 LB: 3.683\n( No feature elimination , no outlier handling , No count Vector , no FM - Just aggregated features and elapsed time)", "438141": "LightGBM 5fold CV 3.65313-> LB 3.689", "432508": "Hi，my CV is 3.65810 and LB:3.697 with 5 fold LGB", "449346": "LightGBM 5fold CV 3.6532-->lb3.687\nLightGBM 5fold CV 3.65101-->lb3.687\n", "463821": "30 features learning rate 0.1 no post-processing\nLB 3.686\n", "436151": "LightGBM 5fold CV 3.64841 -> LB 3.691.\n\nI struggle to make stacking works in this competition so far, so perhaps working on a good single model might goes a long way in this competition :)", "433188": "Update: 3.707 based on this kernel https://www.kaggle.com/konradb/lgb-fe-lb-3-707", "432723": "3.658x on 5fold CatBoost => 3.74 on LB.\nMy current best LB model (3.71) has 3.67 on CV.\nIt's my first Kaggle competition, so I was quite concerned before I read this topic.", "432343": "Hello, I got CV: 3.666426 and LB: 3.705 but also CV: 3.66377 and LB: 3.708 :/", "437758": "10 fold, CV: 3.64342, LB: 3.694", "433374": "5fold lgb with cv: 3.650, lb: 3.697 ", "463796": "CV 3.655 LB 3.678", "438099": "LightGBM 5fold CV 3.6598 -> LB 3.692.\n\nNote: I've had local runs with CV 3.6525 -> LB 3.695 to 3.698 as well. Anyone have tips on how to make local CV move in the same direction as LB?", "435587": "LightGBM with 5 folds: CV: 3.65274 → LB: 3.699", "434688": "5 Fold LGB ------> CV: 3.67083 LB: 3.70", "434439": "5 fold LGB with CV: 3.6724, LB: 3.700", "434309": "5fold lgb with cv: 3.6611, lb: 3.704", "459606": "CV 3.651 LB 3.685", "441511": "5 fold Light GBM. CV mean score: 3.6549 and LB: 3.698\n\nhttps://www.kaggle.com/roydatascience/elo-merchant-recommendation-fathers-day-specials?scriptVersionId=8424127", "435441": "5 Fold LGB ------> CV: 3.65650 LB: 3.690\n5 Fold LGB ------> CV:3.65381 LB: 3.691\nI don't have good offline consistency here.", "433195": "My most successful single model is a CatBoost with a CV of 3.6674 which translates to a 3.706 LB. Still ways to go!", "432711": "10fold lgb ,cv: 3.6678 lb: 3.709\n", "465107": "CV 3.6360, LB 3.679. \nI guess this is nearly the best cv score open in disucssion?", "462839": "Hi all, Is anyone who can hit 3.680± based on single model ,then willing to team up with us ? we need more features but not ensemble&post_processing", "459773": "I'm at CV 3.657 -> LB 3.696 running on Kaggle Kernels, same code CV 3.654 -> LB 3.692 running on my server.\n\nUpdate: same model now at CV 3.6454 LB 3.686 after changing one silly little thing in FE code. ", "459538": "Got 3.63029 CV and 3.700 LB w/ xgboost. Time to move on to another model..", "458394": "5 Fold LGB\n CV:3.66700 LB:3.692", "456916": "i apologize for asking this stupid question. please excuse me(im a beginner to machine learning) and hopefuly you can explain what i am doing incorrectly.\n\n\nhow is everyone calculating the cv score. my cv scores are coming as ~0.06 but it gives me a LB of ~3.9. how is everyones cv close to LB?\n\nmy code for calculating cv is below\n\nfrom sklearn.model_selection import cross_val_score\n\ncv_score = cross_val_score(search.best_estimator_, \\\n x_train,\\\n y_train,\\\n cv = cv,\n scoring = score\n )\n\n\nbest_estimator : is a XGBRegressor in my case\n\ncv = KFold\n\nscoring: mean__squared__error (i just used a fancy make_scorer object)\n", "451763": "LightGBM 5fold CV 3.64593 --> lb 3.695\n\nGap is huge. Whats going wrong ?, How can i reduce the gap ?", "450954": "what's the meaning of \"single model\"? without any post-processing for outliers?", "450535": "Best Single Model :\nLightGBM 5 fold CV 3.64771 and LB 0.3695\nrest is about stacking/blending and post processing as discussed on :https://www.kaggle.com/waitingli/combining-your-model-with-a-model-without-outlier/comments#442188", "449867": "xgb,cv3.650,lb3.695", "448835": "5Fold LGB CV score 3.651 and LB 3.693", "436582": "3.648 on a 20 % held out test set. \n3.708 in 3 fold CV for model selection.\n3.742 LB.\nSingle model", "434823": "5 fold LFB. CV mean score: 3.6634, std: 0.0171. LB: 3.71", "464025": "", "452057": "" } }