How many features does your best single model has?
\n\nMine has 176. CV - 3.648, LB - 3.687.
\n\nI am experimenting with various feature selection techniques to reduce the number, but have little success.
", "messages": [ { "id": 454914, "postDate": "2019-01-12T13:45:41.343Z", "content": "How many features does your best single model has?
\n\nMine has 176. CV - 3.648, LB - 3.687.
\n\nI am experimenting with various feature selection techniques to reduce the number, but have little success.
", "rawMarkdown": "How many features does your best single model has? \n\nMine has **176**. CV - 3.648, LB - 3.687.\n\nI am experimenting with various feature selection techniques to reduce the number, but have little success.", "votes": 19 }, { "id": 457613, "postDate": "2019-01-17T19:42:44.840Z", "content": "30 features LB 3.694
", "rawMarkdown": "30 features LB 3.694", "votes": 1 }, { "id": 455604, "postDate": "2019-01-14T08:38:14.453Z", "content": "92 features; \nLightGBM 5fold CV 3.64491 --> lb 3.690
", "rawMarkdown": "92 features; \nLightGBM 5fold CV 3.64491 --> lb 3.690", "votes": 1 }, { "id": 454953, "postDate": "2019-01-12T15:57:49.623Z", "content": "100 features aprox . I always try to find noisy features.
", "rawMarkdown": "100 features aprox . I always try to find noisy features.", "votes": 1, "replies": [ { "id": 454972, "postDate": "2019-01-12T16:51:00.347Z", "content": "sorry if my question is silly but can you tell me how do you find the noisy features in your model. is it that you compare the gain and split of each feature or any other method??
", "rawMarkdown": "sorry if my question is silly but can you tell me how do you find the noisy features in your model. is it that you compare the gain and split of each feature or any other method??" }, { "id": 455120, "postDate": "2019-01-13T02:51:21.543Z", "content": "I use two methods, the first phase is gain and split for each feature and in the second phase, once I have features selected with this method I select based on this selected set with another method I can´t reveal but it´s more about detect features that seem to be important but it isn´t.
", "rawMarkdown": "I use two methods, the first phase is gain and split for each feature and in the second phase, once I have features selected with this method I select based on this selected set with another method I can´t reveal but it´s more about detect features that seem to be important but it isn´t.\n\n", "votes": 2 }, { "id": 457075, "postDate": "2019-01-16T22:50:07.260Z", "content": "Is your first phase similar to boruta?
", "rawMarkdown": "Is your first phase similar to boruta?" } ] }, { "id": 456560, "postDate": "2019-01-16T03:56:10.703Z", "content": "175 features cv 3.6240 lb 3.672
", "rawMarkdown": "175 features cv 3.6240 lb 3.672", "votes": 2, "replies": [ { "id": 456612, "postDate": "2019-01-16T07:27:10.620Z", "content": "Nice feature engineering.
", "rawMarkdown": "Nice feature engineering." }, { "id": 456664, "postDate": "2019-01-16T09:02:12.527Z", "content": "Thanks. I only use LGB so far. For regression task, I prefer NN, I will try it later.
", "rawMarkdown": "Thanks. I only use LGB so far. For regression task, I prefer NN, I will try it later.\n" }, { "id": 463422, "postDate": "2019-01-30T00:45:37.260Z", "content": "@daishu, may I know whether NN stands for neural network?
", "rawMarkdown": "@daishu, may I know whether NN stands for neural network?" }, { "id": 479078, "postDate": "2019-02-27T00:46:41.607Z", "content": "@Frank Puk\nSorry for my late reply. Your are right. But nn do not work in this competition.
", "rawMarkdown": "@Frank Puk\nSorry for my late reply. Your are right. But nn do not work in this competition." } ] }, { "id": 455229, "postDate": "2019-01-13T10:42:47.063Z", "content": "I have 192 features for my best single model, seems we all kind of converge on a similar range :)
", "rawMarkdown": "I have 192 features for my best single model, seems we all kind of converge on a similar range :)", "votes": 2, "replies": [ { "id": 455552, "postDate": "2019-01-14T06:44:16.563Z", "content": "would you mind tell us what's your best single model 'score on lb and local?
", "rawMarkdown": "would you mind tell us what's your best single model 'score on lb and local?", "votes": 1 }, { "id": 455611, "postDate": "2019-01-14T08:43:36.903Z", "content": "Hi Yifan Xie, what is the CV score and LB score, if you can tell , Thanks !
", "rawMarkdown": "Hi Yifan Xie, what is the CV score and LB score, if you can tell , Thanks !" } ] }, { "id": 455870, "postDate": "2019-01-14T18:22:56.003Z", "content": "awesome score
", "rawMarkdown": "awesome score" }, { "id": 455609, "postDate": "2019-01-14T08:41:11.357Z", "content": "Along with features , if everyone can add CV score and Lb score then that would be even better :)
", "rawMarkdown": "Along with features , if everyone can add CV score and Lb score then that would be even better :)", "replies": [ { "id": 455616, "postDate": "2019-01-14T08:55:01.113Z", "content": "@Manoj, @bestpredict - my best single model on CV & LB is probably overfitting, but here it is:\nCV3.615717, LB3.667
", "rawMarkdown": "@Manoj, @bestpredict - my best single model on CV & LB is probably overfitting, but here it is:\nCV3.615717, LB3.667", "votes": 6 }, { "id": 455622, "postDate": "2019-01-14T09:01:34.927Z", "content": "Wow !!, probably not. all the best :)
", "rawMarkdown": "Wow !!, probably not. all the best :)" }, { "id": 455629, "postDate": "2019-01-14T09:07:38.637Z", "content": "I will explain a bit more after the competition closes, if by that time it is still relevant :)
\n\nThis model doesn't really contribute to my overall stacking due to the way I create it -> I don't actually include it in my current best CV/LB overall solution. I am still experimenting
", "rawMarkdown": "I will explain a bit more after the competition closes, if by that time it is still relevant :) \n\nThis model doesn't really contribute to my overall stacking due to the way I create it -> I don't actually include it in my current best CV/LB overall solution. I am still experimenting \n\n", "votes": 4 }, { "id": 455634, "postDate": "2019-01-14T09:15:27.093Z", "content": "Awesome score!!!
", "rawMarkdown": "Awesome score!!!", "votes": 1 }, { "id": 455681, "postDate": "2019-01-14T11:04:56.600Z", "content": "Awesome single model!
", "rawMarkdown": "Awesome single model!", "votes": 1 }, { "id": 455768, "postDate": "2019-01-14T14:43:36.180Z", "content": "@somethingiswrong and @WangJiaFei, how many features do you guys have in your best single model? and what is the corresponding CV/LB?
", "rawMarkdown": "@somethingiswrong and @WangJiaFei, how many features do you guys have in your best single model? and what is the corresponding CV/LB? " }, { "id": 456021, "postDate": "2019-01-15T01:05:01.097Z", "content": "Our single model is not strong.
", "rawMarkdown": "Our single model is not strong." }, { "id": 456059, "postDate": "2019-01-15T03:34:41.250Z", "content": "bad single model too, so sad!
", "rawMarkdown": "bad single model too, so sad!" }, { "id": 456134, "postDate": "2019-01-15T08:00:37.937Z", "content": "Nice score.
", "rawMarkdown": "Nice score." } ] }, { "id": 455110, "postDate": "2019-01-13T02:01:26.617Z", "content": "With 161 features, CV = 3.65108 and LB = 3.685
", "rawMarkdown": "With 161 features, CV = 3.65108 and LB = 3.685", "replies": [ { "id": 455612, "postDate": "2019-01-14T08:44:29.920Z", "content": "Hi Harshit, what is the CV score and LB score of single model, if you can tell , Thanks !
", "rawMarkdown": "Hi Harshit, what is the CV score and LB score of single model, if you can tell , Thanks !" }, { "id": 463426, "postDate": "2019-01-30T00:54:53.783Z", "content": "CV is 3.652 and LB is 3.685, Stuck over there. Adding features is not helping anymore.
", "rawMarkdown": "CV is 3.652 and LB is 3.685, Stuck over there. Adding features is not helping anymore." } ] }, { "id": 455108, "postDate": "2019-01-13T01:55:55.380Z", "content": "212 features. I am planing to do reduction in the future, but CV rmse is around 1.5 The CV score does not tell me that I should bother reducing it for now. Or should I trim it?
", "rawMarkdown": "212 features. I am planing to do reduction in the future, but CV rmse is around 1.5 The CV score does not tell me that I should bother reducing it for now. Or should I trim it?", "replies": [ { "id": 455131, "postDate": "2019-01-13T03:19:50.817Z", "content": "Your Cv is 1.5, assuming you are training the model after removing the outliers. I would suggest to not remove outliers and then train the model.
", "rawMarkdown": "Your Cv is 1.5, assuming you are training the model after removing the outliers. I would suggest to not remove outliers and then train the model. ", "votes": 3 }, { "id": 455299, "postDate": "2019-01-13T14:33:35.780Z", "content": "hi，can you tell me why training the model with the outliers is better, does this make a effective comparison between LB and CV? thanks !
", "rawMarkdown": "hi，can you tell me why training the model with the outliers is better, does this make a effective comparison between LB and CV? thanks !" }, { "id": 455304, "postDate": "2019-01-13T14:56:37.420Z", "content": "Yes it will. Test set definitely has outliers in it.
", "rawMarkdown": "Yes it will. Test set definitely has outliers in it. " }, { "id": 455491, "postDate": "2019-01-14T03:17:12.590Z", "content": "Thank you for your reply.
", "rawMarkdown": "Thank you for your reply." }, { "id": 455546, "postDate": "2019-01-14T06:25:04.780Z", "content": "But just to add one thing to clarify stuff, building one model on dataset without outliers will help in post processing as mentioned here \nhttps://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/75173 and that can give boost.
", "rawMarkdown": "But just to add one thing to clarify stuff, building one model on dataset without outliers will help in post processing as mentioned here \nhttps://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/75173 and that can give boost.", "votes": 2 } ] }, { "id": 460081, "postDate": "2019-01-23T00:22:18.687Z", "rawMarkdown": "", "isDeleted": true }, { "id": 455381, "postDate": "2019-01-13T17:29:58.060Z", "rawMarkdown": "", "isDeleted": true }, { "id": 454924, "postDate": "2019-01-12T14:04:54.110Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 457613, "author_name": "Vadim Nareyko", "author_url": "", "post_date": "2019-01-17T19:42:44.840000", "content": "30 features LB 3.694
", "votes": 1, "replies": [] }, { "id": 455604, "author_name": "Manoj", "author_url": "", "post_date": "2019-01-14T08:38:14.453000", "content": "92 features; \nLightGBM 5fold CV 3.64491 --> lb 3.690
", "votes": 1, "replies": [] }, { "id": 454953, "author_name": "Icfstat", "author_url": "", "post_date": "2019-01-12T15:57:49.623000", "content": "100 features aprox . I always try to find noisy features.
", "votes": 1, "replies": [ { "id": 454972, "author_name": "aashish malik", "author_url": "", "post_date": "2019-01-12T16:51:00.347000", "content": "sorry if my question is silly but can you tell me how do you find the noisy features in your model. is it that you compare the gain and split of each feature or any other method??
", "votes": 0, "replies": [] }, { "id": 455120, "author_name": "Icfstat", "author_url": "", "post_date": "2019-01-13T02:51:21.543000", "content": "I use two methods, the first phase is gain and split for each feature and in the second phase, once I have features selected with this method I select based on this selected set with another method I can´t reveal but it´s more about detect features that seem to be important but it isn´t.
", "votes": 2, "replies": [] }, { "id": 457075, "author_name": "Moiz", "author_url": "", "post_date": "2019-01-16T22:50:07.260000", "content": "Is your first phase similar to boruta?
", "votes": 0, "replies": [] } ] }, { "id": 456560, "author_name": "Correlation", "author_url": "", "post_date": "2019-01-16T03:56:10.703000", "content": "175 features cv 3.6240 lb 3.672
", "votes": 2, "replies": [ { "id": 456612, "author_name": "LongYin/杰少", "author_url": "", "post_date": "2019-01-16T07:27:10.620000", "content": "Nice feature engineering.
", "votes": 0, "replies": [] }, { "id": 456664, "author_name": "Correlation", "author_url": "", "post_date": "2019-01-16T09:02:12.527000", "content": "Thanks. I only use LGB so far. For regression task, I prefer NN, I will try it later.
", "votes": 0, "replies": [] }, { "id": 463422, "author_name": "FP", "author_url": "", "post_date": "2019-01-30T00:45:37.260000", "content": "@daishu, may I know whether NN stands for neural network?
", "votes": 0, "replies": [] }, { "id": 479078, "author_name": "Correlation", "author_url": "", "post_date": "2019-02-27T00:46:41.607000", "content": "@Frank Puk\nSorry for my late reply. Your are right. But nn do not work in this competition.
", "votes": 0, "replies": [] } ] }, { "id": 455229, "author_name": "Yifan Xie", "author_url": "", "post_date": "2019-01-13T10:42:47.063000", "content": "I have 192 features for my best single model, seems we all kind of converge on a similar range :)
", "votes": 2, "replies": [ { "id": 455552, "author_name": "AIFIRST", "author_url": "", "post_date": "2019-01-14T06:44:16.563000", "content": "would you mind tell us what's your best single model 'score on lb and local?
", "votes": 1, "replies": [] }, { "id": 455611, "author_name": "Manoj", "author_url": "", "post_date": "2019-01-14T08:43:36.903000", "content": "Hi Yifan Xie, what is the CV score and LB score, if you can tell , Thanks !
", "votes": 0, "replies": [] } ] }, { "id": 455870, "author_name": "ALOK PRATAP", "author_url": "", "post_date": "2019-01-14T18:22:56.003000", "content": "awesome score
", "votes": 0, "replies": [] }, { "id": 455609, "author_name": "Manoj", "author_url": "", "post_date": "2019-01-14T08:41:11.357000", "content": "Along with features , if everyone can add CV score and Lb score then that would be even better :)
", "votes": 0, "replies": [ { "id": 455616, "author_name": "Yifan Xie", "author_url": "", "post_date": "2019-01-14T08:55:01.113000", "content": "@Manoj, @bestpredict - my best single model on CV & LB is probably overfitting, but here it is:\nCV3.615717, LB3.667
", "votes": 6, "replies": [] }, { "id": 455622, "author_name": "Manoj", "author_url": "", "post_date": "2019-01-14T09:01:34.927000", "content": "Wow !!, probably not. all the best :)
", "votes": 0, "replies": [] }, { "id": 455629, "author_name": "Yifan Xie", "author_url": "", "post_date": "2019-01-14T09:07:38.637000", "content": "I will explain a bit more after the competition closes, if by that time it is still relevant :)
\n\nThis model doesn't really contribute to my overall stacking due to the way I create it -> I don't actually include it in my current best CV/LB overall solution. I am still experimenting
", "votes": 4, "replies": [] }, { "id": 455634, "author_name": "Marcus Lin", "author_url": "", "post_date": "2019-01-14T09:15:27.093000", "content": "Awesome score!!!
", "votes": 1, "replies": [] }, { "id": 455681, "author_name": "Jianfei Wang", "author_url": "", "post_date": "2019-01-14T11:04:56.600000", "content": "Awesome single model!
", "votes": 1, "replies": [] }, { "id": 455768, "author_name": "Yifan Xie", "author_url": "", "post_date": "2019-01-14T14:43:36.180000", "content": "@somethingiswrong and @WangJiaFei, how many features do you guys have in your best single model? and what is the corresponding CV/LB?
", "votes": 0, "replies": [] }, { "id": 456021, "author_name": "Marcus Lin", "author_url": "", "post_date": "2019-01-15T01:05:01.097000", "content": "Our single model is not strong.
", "votes": 0, "replies": [] }, { "id": 456059, "author_name": "Jianfei Wang", "author_url": "", "post_date": "2019-01-15T03:34:41.250000", "content": "bad single model too, so sad!
", "votes": 0, "replies": [] }, { "id": 456134, "author_name": "LongYin/杰少", "author_url": "", "post_date": "2019-01-15T08:00:37.937000", "content": "Nice score.
", "votes": 0, "replies": [] } ] }, { "id": 455110, "author_name": "HarshitMehta", "author_url": "", "post_date": "2019-01-13T02:01:26.617000", "content": "With 161 features, CV = 3.65108 and LB = 3.685
", "votes": 0, "replies": [ { "id": 455612, "author_name": "Manoj", "author_url": "", "post_date": "2019-01-14T08:44:29.920000", "content": "Hi Harshit, what is the CV score and LB score of single model, if you can tell , Thanks !
", "votes": 0, "replies": [] }, { "id": 463426, "author_name": "HarshitMehta", "author_url": "", "post_date": "2019-01-30T00:54:53.783000", "content": "CV is 3.652 and LB is 3.685, Stuck over there. Adding features is not helping anymore.
", "votes": 0, "replies": [] } ] }, { "id": 455108, "author_name": "sergeman", "author_url": "", "post_date": "2019-01-13T01:55:55.380000", "content": "212 features. I am planing to do reduction in the future, but CV rmse is around 1.5 The CV score does not tell me that I should bother reducing it for now. Or should I trim it?
", "votes": 0, "replies": [ { "id": 455131, "author_name": "HarshitMehta", "author_url": "", "post_date": "2019-01-13T03:19:50.817000", "content": "Your Cv is 1.5, assuming you are training the model after removing the outliers. I would suggest to not remove outliers and then train the model.
", "votes": 3, "replies": [] }, { "id": 455299, "author_name": "fwj7", "author_url": "", "post_date": "2019-01-13T14:33:35.780000", "content": "hi，can you tell me why training the model with the outliers is better, does this make a effective comparison between LB and CV? thanks !
", "votes": 0, "replies": [] }, { "id": 455304, "author_name": "HarshitMehta", "author_url": "", "post_date": "2019-01-13T14:56:37.420000", "content": "Yes it will. Test set definitely has outliers in it.
", "votes": 0, "replies": [] }, { "id": 455491, "author_name": "fwj7", "author_url": "", "post_date": "2019-01-14T03:17:12.590000", "content": "Thank you for your reply.
", "votes": 0, "replies": [] }, { "id": 455546, "author_name": "HarshitMehta", "author_url": "", "post_date": "2019-01-14T06:25:04.780000", "content": "But just to add one thing to clarify stuff, building one model on dataset without outliers will help in post processing as mentioned here \nhttps://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/75173 and that can give boost.
", "votes": 2, "replies": [] } ] }, { "id": 460081, "author_name": "", "author_url": "", "post_date": "2019-01-23T00:22:18.687000", "content": "", "votes": 0, "replies": [] }, { "id": 455381, "author_name": "", "author_url": "", "post_date": "2019-01-13T17:29:58.060000", "content": "", "votes": 0, "replies": [] }, { "id": 454924, "author_name": "", "author_url": "", "post_date": "2019-01-12T14:04:54.110000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "454914": "How many features does your best single model has? \n\nMine has **176**. CV - 3.648, LB - 3.687.\n\nI am experimenting with various feature selection techniques to reduce the number, but have little success.", "457613": "30 features LB 3.694", "455604": "92 features; \nLightGBM 5fold CV 3.64491 --> lb 3.690", "454953": "100 features aprox . I always try to find noisy features.", "456560": "175 features cv 3.6240 lb 3.672", "455229": "I have 192 features for my best single model, seems we all kind of converge on a similar range :)", "455870": "awesome score", "455609": "Along with features , if everyone can add CV score and Lb score then that would be even better :)", "455110": "With 161 features, CV = 3.65108 and LB = 3.685", "455108": "212 features. I am planing to do reduction in the future, but CV rmse is around 1.5 The CV score does not tell me that I should bother reducing it for now. Or should I trim it?", "460081": "", "455381": "", "454924": "" } }