Hello kagglers,
\n\nThis is my first kaggle competition. The community here is amazing and very inspiring. I've learnt much much more from the kernels here than any other online resources. I started out very excited. But I'm basically stuck at the same point for the past few days. I'm running out of ideas and getting frustrated. So I'm here seeking your help to shepherd me in the right direction. π
\n\nI'm using Stratified KFold with lgbm as done here . I had some success extracting some of the features mentioned in the kernel before referring it.
\n\nMy best score so far is 3.695 (with ideas picked from the kernels & discussions) on LB with LGBM single model without any stacking, blending
\n\nNot able to reproduce my results consistently.
\nThe same code is giving multiple CV scores for same LB score. For eg.\n3.65377489489624\n3.65462490707428\n3.654910478618945\nIs it because I used datetime.datetime.today() while calculating aggregates? I tried setting to Nov 1st 2018. It brought down my CV score. So I didn't venture further
Inconsistent gap between LB & local score:
\n\nI'm trying to compute share of wallet - customer spend per category. Purchase amount aggregates on category_2, category_3, category_1 are bringing down score.\nAdding sum & mean aggregates is worsening my scores. I read in the forums SubikashPal were able to achieve 3.68xx score on LB just by aggregating without using merchant info. And making small changes to public kernel was able give SomethingIsWrong a big push. That's amazing. I'm just too blind may be.
\n\nNot sure if I'm doing some glaring mistakes or if I'm sidetracked or if they really didn't work for you either.
\n\nπIf you can give me any cues for improving the single model score 3.695 I'd be indebted to you π
\n\n```\ndef agg_on_cat(df, category, feature):\n temp_df = df.pivot_table(index='card_id', columns=category, aggfunc={feature: ['sum', 'mean']})\n cols = [category + '{0[2]}{0[0]}_{01}'.format(col) for col in temp_df.columns.tolist()]\n temp_df.columns = cols\n return temp_df
\n\ndef get_cat_agg(df):\n agg_df = agg_on_cat(df, 'category_1', 'purchase_amount')\n agg_df = pd.merge(agg_df, agg_on_cat(df, 'category_2', 'purchase_amount'), on='card_id', how='left')\n agg_df = pd.merge(agg_df, agg_on_cat(df, 'category_3', 'purchase_amount'), on='card_id', how='left')\n agg_df = pd.merge(agg_df, agg_on_cat(df, 'authorized_flag', 'purchase_amount'), on='card_id', how='left')\n return agg_df\n```
", "messages": [ { "id": "458542", "postDate": "01/19/2019 22:22:37", "content": "Hello kagglers,
\n\nThis is my first kaggle competition. The community here is amazing and very inspiring. I've learnt much much more from the kernels here than any other online resources. I started out very excited. But I'm basically stuck at the same point for the past few days. I'm running out of ideas and getting frustrated. So I'm here seeking your help to shepherd me in the right direction. π
\n\nI'm using Stratified KFold with lgbm as done here . I had some success extracting some of the features mentioned in the kernel before referring it.
\n\nMy best score so far is 3.695 (with ideas picked from the kernels & discussions) on LB with LGBM single model without any stacking, blending
\n\nNot able to reproduce my results consistently.
\nThe same code is giving multiple CV scores for same LB score. For eg.\n3.65377489489624\n3.65462490707428\n3.654910478618945\nIs it because I used datetime.datetime.today() while calculating aggregates? I tried setting to Nov 1st 2018. It brought down my CV score. So I didn't venture further
Inconsistent gap between LB & local score:
\n\nI'm trying to compute share of wallet - customer spend per category. Purchase amount aggregates on category_2, category_3, category_1 are bringing down score.\nAdding sum & mean aggregates is worsening my scores. I read in the forums SubikashPal were able to achieve 3.68xx score on LB just by aggregating without using merchant info. And making small changes to public kernel was able give SomethingIsWrong a big push. That's amazing. I'm just too blind may be.
\n\nNot sure if I'm doing some glaring mistakes or if I'm sidetracked or if they really didn't work for you either.
\n\nπIf you can give me any cues for improving the single model score 3.695 I'd be indebted to you π
\n\n```\ndef agg_on_cat(df, category, feature):\n temp_df = df.pivot_table(index='card_id', columns=category, aggfunc={feature: ['sum', 'mean']})\n cols = [category + '{0[2]}{0[0]}_{01}'.format(col) for col in temp_df.columns.tolist()]\n temp_df.columns = cols\n return temp_df
\n\ndef get_cat_agg(df):\n agg_df = agg_on_cat(df, 'category_1', 'purchase_amount')\n agg_df = pd.merge(agg_df, agg_on_cat(df, 'category_2', 'purchase_amount'), on='card_id', how='left')\n agg_df = pd.merge(agg_df, agg_on_cat(df, 'category_3', 'purchase_amount'), on='card_id', how='left')\n agg_df = pd.merge(agg_df, agg_on_cat(df, 'authorized_flag', 'purchase_amount'), on='card_id', how='left')\n return agg_df\n```
", "rawMarkdown": "Hello kagglers,\n\nThis is my first kaggle competition. The community here is amazing and very inspiring. I've learnt much much more from the kernels here than any other online resources. I started out very excited. But I'm basically stuck at the same point for the past few days. I'm running out of ideas and getting frustrated. So I'm here seeking your help to shepherd me in the right direction. π\n\nI'm using Stratified KFold with lgbm as done [here](https://www.kaggle.com/chauhuynh/my-first-kernel-3-699) . I had some success extracting some of the features mentioned in the kernel before referring it.\n\nMy best score so far is 3.695 (with ideas picked from the kernels & discussions) on LB with LGBM single model without any stacking, blending\n\nProblem 1:\n----------\n\nNot able to reproduce my results consistently. \nThe same code is giving multiple CV scores for same LB score. For eg.\n3.65377489489624\n3.65462490707428\n3.654910478618945\nIs it because I used `datetime.datetime.today()` while calculating aggregates? I tried setting to Nov 1st 2018. It brought down my CV score. So I didn't venture further\n\nProblem 2:\n----------\n\nInconsistent gap between LB & local score:\n\n1. **3.65975**96743401666, LB - 3.707 \n2. **3.65672**6211308068, LB - 3.696 \n3. **3.65548**01486969906, LB- 3.695\n4. **3.65420**8034404632, LB - 3.697 π€―\n5. **3.65565**48946965446, LB - 3.696\n6. **3.65493**381575084, LB - 3.697 \n\nProblem 3:\n----------\n\nI'm trying to compute share of wallet - customer spend per category. Purchase amount aggregates on category_2, category_3, category_1 are bringing down score.\nAdding sum & mean aggregates is worsening my scores. [I read in the forums SubikashPal were able to achieve 3.68xx score on LB just by aggregating without using merchant info.](https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/77652) [And making small changes to public kernel was able give SomethingIsWrong a big push](https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73517#438795). That's amazing. I'm just too blind may be.\n\nWhat I've tried\n---------------\n\n1. Adding repurchase rate, weekend, avg spend per transaction, recency, participation rate - worked thanks @yifanxie, @chauhuynh\n2. Monthly aggregates & successive aggregates as mentioned [here](https://www.kaggle.com/fabiendaniel/elo-world)\n3. Using Neural net Embeddings brought the CV score to 3.69715275269681. Didn't improve further with current features (It's based on fast.ai. I'll publish a kernel if you're curious)\nEdit: [Kernel is available now. Using latest fastai version but not with Kfold][1]\n4. Separated auth & non-auth transactions and aggregated separately - worsened\n\nNot sure if I'm doing some glaring mistakes or if I'm sidetracked or if they really didn't work for you either.\n\nWhat I'm yet to try:\n--------------------\n\n1. Stacking, blending\n2. Add a meta info column for outlier classification and blend it as suggested in discussion forums\n3. Use merchant info and aggregate\n4. Binning \n5. No of times customer switches between merchant within same category \n6. FM as mentioned in forums\n7. Feature elimination and others mentioned [here](https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73937)\n\nπIf you can give me any cues for improving the single model score 3.695 I'd be indebted to you π\n\n### snippet\n```\ndef agg_on_cat(df, category, feature):\n temp_df = df.pivot_table(index='card_id', columns=category, aggfunc={feature: ['sum', 'mean']})\n cols = [category + '_{0[2]}_{0[0]}_{0[1]}'.format(col) for col in temp_df.columns.tolist()]\n temp_df.columns = cols\n return temp_df\n\ndef get_cat_agg(df):\n agg_df = agg_on_cat(df, 'category_1', 'purchase_amount')\n agg_df = pd.merge(agg_df, agg_on_cat(df, 'category_2', 'purchase_amount'), on='card_id', how='left')\n agg_df = pd.merge(agg_df, agg_on_cat(df, 'category_3', 'purchase_amount'), on='card_id', how='left')\n agg_df = pd.merge(agg_df, agg_on_cat(df, 'authorized_flag', 'purchase_amount'), on='card_id', how='left')\n return agg_df\n```\n\n\n [1]: https://www.kaggle.com/chandureddyvari/sprinkling-in-some-deep-learning-using-fast-ai/", "votes": null }, { "id": "458922", "postDate": "01/20/2019 20:15:47", "content": "Problem 2 is a common problem that is often times just an artifact of random noise... see \"Mind the Gap and Trust your CV\"
", "rawMarkdown": "Problem 2 is a common problem that is often times just an artifact of random noise... see [\"Mind the Gap and Trust your CV\"](https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73936)", "votes": null }, { "id": "458944", "postDate": "01/20/2019 21:16:52", "content": "You should try to do feature selection. It helped me a lot to improve my lb score. Also I'm curios about NN embeddings and waiting for your kernel...
", "rawMarkdown": "You should try to do feature selection. It helped me a lot to improve my lb score. Also I'm curios about NN embeddings and waiting for your kernel...", "votes": null }, { "id": "459047", "postDate": "01/21/2019 04:23:35", "content": "Sure. I'll publish it in 2 days. Any kernels of old competitions you can point me to for feature extraction?
", "rawMarkdown": "Sure. I'll publish it in 2 days. Any kernels of old competitions you can point me to for feature extraction?", "votes": null }, { "id": "459048", "postDate": "01/21/2019 04:25:21", "content": "Thanks for the reply, I'll trust the model with best CV score for blending and stacking.
", "rawMarkdown": "Thanks for the reply, I'll trust the model with best CV score for blending and stacking.", "votes": null }, { "id": "459065", "postDate": "01/21/2019 05:25:13", "content": "Check this out: https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73937
", "rawMarkdown": "Check this out: https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73937", "votes": null }, { "id": "459384", "postDate": "01/21/2019 16:32:26", "content": "Thanks for the pointer. I'll try them for sure. You had any luck aggregating on categories? Because that's what is worsening my score. Just want to clarify if it helped you. Please let me know
", "rawMarkdown": "Thanks for the pointer. I'll try them for sure. You had any luck aggregating on categories? Because that's what is worsening my score. Just want to clarify if it helped you. Please let me know", "votes": null }, { "id": "459410", "postDate": "01/21/2019 17:13:05", "content": "TBH I have not spent too much time on feature extraction. Just using aggregated data and some feature interactions with feature selection. Now planning to start over and try to work on category features. So I'll let you know...
", "rawMarkdown": "TBH I have not spent too much time on feature extraction. Just using aggregated data and some feature interactions with feature selection. Now planning to start over and try to work on category features. So I'll let you know...", "votes": null }, { "id": "459534", "postDate": "01/21/2019 23:56:01", "content": "i recommend you to try more aggregation. I reached at LB 3.688 only using aggregation(single model). and some people reached 3.680 :D
", "rawMarkdown": "i recommend you to try more aggregation. I reached at LB 3.688 only using aggregation(single model). and some people reached 3.680 :D", "votes": null }, { "id": "459551", "postDate": "01/22/2019 01:08:30", "content": "β Inconsistencies should be able to fixed by fixed date and removing the seed.\nβ concentrate on date features/elapsed date. You should be able to mine some really interesting features.\nβ I would suggest u spend some time on merchant features on historical transactions.\nβ aggregation based on categories is really very frustrating but can lead to some top features.
\n\nAll these helped me to score 3.84 single model LGB.
\n\nCurious to see your NN embedding stuff.
", "rawMarkdown": "β Inconsistencies should be able to fixed by fixed date and removing the seed.\nβ concentrate on date features/elapsed date. You should be able to mine some really interesting features.\nβ I would suggest u spend some time on merchant features on historical transactions.\nβ aggregation based on categories is really very frustrating but can lead to some top features.\n\nAll these helped me to score 3.84 single model LGB.\n\nCurious to see your NN embedding stuff.", "votes": null }, { "id": "459553", "postDate": "01/22/2019 01:17:02", "content": "@silverstone.\nWhich feature selection are u using? borota ot Null feature importance. I am having 140+ features and not able to improve a single point removing any features. But i know there is a scope. I spend lots of time there without success. Any clue here βΊοΈ
", "rawMarkdown": "silverstone.\nWhich feature selection are u using? borota ot Null feature importance. I am having 140+ features and not able to improve a single point removing any features. But i know there is a scope. I spend lots of time there without success. Any clue here βΊοΈ", "votes": null }, { "id": "459597", "postDate": "01/22/2019 03:55:01", "content": "Wow that's awesome. I've tried aggregation over categories and authorised trans etc. and all but it just brought down my score. I'll visit again and will try to do more aggregation on all the categorical variables. Thanks for the headsup!
", "rawMarkdown": "Wow that's awesome. I've tried aggregation over categories and authorised trans etc. and all but it just brought down my score. I'll visit again and will try to do more aggregation on all the categorical variables. Thanks for the headsup!", "votes": null }, { "id": "459608", "postDate": "01/22/2019 04:40:00", "content": "Yes. I also try to aggregation more and more when i reached in 3.67x. Good Luck !!
", "rawMarkdown": "Yes. I also try to aggregation more and more when i reached in 3.67x. Good Luck !!", "votes": null }, { "id": "459632", "postDate": "01/22/2019 05:19:38", "content": "I'm using a mix of null importances & boruta's results with different parameters. Like a said now I reviewing whole process.
", "rawMarkdown": "I'm using a mix of null importances & boruta's results with different parameters. Like a said now I reviewing whole process.", "votes": null }, { "id": "459635", "postDate": "01/22/2019 05:24:19", "content": "How many aggregated features do you have? More than 400? I think more features increase the converge time and lead overfitting. at least for me :(
", "rawMarkdown": "How many aggregated features do you have? More than 400? I think more features increase the converge time and lead overfitting. at least for me :(", "votes": null }, { "id": "459692", "postDate": "01/22/2019 07:23:30", "content": "@silverstone1903 i use a 170 features. i also think that more features can be lead to overfitting. but i try to do more aggregation to believe feature selection :D
", "rawMarkdown": "silverstone1903 i use a 170 features. i also think that more features can be lead to overfitting. but i try to do more aggregation to believe feature selection :D", "votes": null }, { "id": "459838", "postDate": "01/22/2019 13:18:40", "content": "That's a very helpful cue. Thank you!! Sure I'll publish the kernel soon. Possibly by tomorrow.
", "rawMarkdown": "That's a very helpful cue. Thank you!! Sure I'll publish the kernel soon. Possibly by tomorrow.", "votes": null }, { "id": "459840", "postDate": "01/22/2019 13:21:32", "content": "Thank you for your suggestions. You guys are really awesome. Good luck!
", "rawMarkdown": "Thank you for your suggestions. You guys are really awesome. Good luck!", "votes": null }, { "id": "459989", "postDate": "01/22/2019 19:38:25", "content": "Here's the link to kernel guys https://www.kaggle.com/chandureddyvari/sprinkling-in-some-deep-learning-using-fast-ai
", "rawMarkdown": "Here's the link to kernel guys https://www.kaggle.com/chandureddyvari/sprinkling-in-some-deep-learning-using-fast-ai", "votes": null }, { "id": "460068", "postDate": "01/22/2019 23:21:01", "content": "This is neural embbeding??? Thanks for sharing. i first saw it :D
", "rawMarkdown": "This is neural embbeding??? Thanks for sharing. i first saw it :D", "votes": null }, { "id": "460120", "postDate": "01/23/2019 03:02:02", "content": "Although it may sound meaningless to you, discussion is extremely helpful and peers here are generally helpful. So don't give up and just continue. Take a break and then continue. Just continue no matter what.
", "rawMarkdown": "Although it may sound meaningless to you, discussion is extremely helpful and peers here are generally helpful. So don't give up and just continue. Take a break and then continue. Just continue no matter what.", "votes": null }, { "id": "460202", "postDate": "01/23/2019 07:27:34", "content": "Yes. People who won 2nd place for Rossmann competition had good success with it supposedly without much feature engineering as first place people. Do try it on your current feature set and let me know if it gave you some boost on LB.
", "rawMarkdown": "Yes. People who won 2nd place for Rossmann competition had good success with it supposedly without much feature engineering as first place people. Do try it on your current feature set and let me know if it gave you some boost on LB.", "votes": null }, { "id": "460204", "postDate": "01/23/2019 07:30:25", "content": "Thanks for the inspiring words Frank :)
", "rawMarkdown": "Thanks for the inspiring words Frank :)", "votes": null } ], "comments": [ { "id": 458922, "author_name": "peterhurford", "author_url": "", "post_date": "01/20/2019 20:15:47", "content": "Problem 2 is a common problem that is often times just an artifact of random noise... see \"Mind the Gap and Trust your CV\"
", "votes": null, "replies": [ { "id": 459048, "author_name": "chandureddyvari", "author_url": "", "post_date": "01/21/2019 04:25:21", "content": "Thanks for the reply, I'll trust the model with best CV score for blending and stacking.
", "votes": null, "replies": [] }, { "id": 460120, "author_name": "pukkinming", "author_url": "", "post_date": "01/23/2019 03:02:02", "content": "Although it may sound meaningless to you, discussion is extremely helpful and peers here are generally helpful. So don't give up and just continue. Take a break and then continue. Just continue no matter what.
", "votes": null, "replies": [] }, { "id": 460204, "author_name": "chandureddyvari", "author_url": "", "post_date": "01/23/2019 07:30:25", "content": "Thanks for the inspiring words Frank :)
", "votes": null, "replies": [] } ] }, { "id": 458944, "author_name": "silverstone1903", "author_url": "", "post_date": "01/20/2019 21:16:52", "content": "You should try to do feature selection. It helped me a lot to improve my lb score. Also I'm curios about NN embeddings and waiting for your kernel...
", "votes": null, "replies": [ { "id": 459047, "author_name": "chandureddyvari", "author_url": "", "post_date": "01/21/2019 04:23:35", "content": "Sure. I'll publish it in 2 days. Any kernels of old competitions you can point me to for feature extraction?
", "votes": null, "replies": [] }, { "id": 459065, "author_name": "silverstone1903", "author_url": "", "post_date": "01/21/2019 05:25:13", "content": "Check this out: https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73937
", "votes": null, "replies": [] }, { "id": 459384, "author_name": "chandureddyvari", "author_url": "", "post_date": "01/21/2019 16:32:26", "content": "Thanks for the pointer. I'll try them for sure. You had any luck aggregating on categories? Because that's what is worsening my score. Just want to clarify if it helped you. Please let me know
", "votes": null, "replies": [] }, { "id": 459410, "author_name": "silverstone1903", "author_url": "", "post_date": "01/21/2019 17:13:05", "content": "TBH I have not spent too much time on feature extraction. Just using aggregated data and some feature interactions with feature selection. Now planning to start over and try to work on category features. So I'll let you know...
", "votes": null, "replies": [] }, { "id": 459553, "author_name": "subikashpal", "author_url": "", "post_date": "01/22/2019 01:17:02", "content": "@silverstone.\nWhich feature selection are u using? borota ot Null feature importance. I am having 140+ features and not able to improve a single point removing any features. But i know there is a scope. I spend lots of time there without success. Any clue here βΊοΈ
", "votes": null, "replies": [] }, { "id": 459632, "author_name": "silverstone1903", "author_url": "", "post_date": "01/22/2019 05:19:38", "content": "I'm using a mix of null importances & boruta's results with different parameters. Like a said now I reviewing whole process.
", "votes": null, "replies": [] } ] }, { "id": 459534, "author_name": "chocozzz", "author_url": "", "post_date": "01/21/2019 23:56:01", "content": "i recommend you to try more aggregation. I reached at LB 3.688 only using aggregation(single model). and some people reached 3.680 :D
", "votes": null, "replies": [ { "id": 459597, "author_name": "chandureddyvari", "author_url": "", "post_date": "01/22/2019 03:55:01", "content": "Wow that's awesome. I've tried aggregation over categories and authorised trans etc. and all but it just brought down my score. I'll visit again and will try to do more aggregation on all the categorical variables. Thanks for the headsup!
", "votes": null, "replies": [] }, { "id": 459608, "author_name": "chocozzz", "author_url": "", "post_date": "01/22/2019 04:40:00", "content": "Yes. I also try to aggregation more and more when i reached in 3.67x. Good Luck !!
", "votes": null, "replies": [] }, { "id": 459635, "author_name": "silverstone1903", "author_url": "", "post_date": "01/22/2019 05:24:19", "content": "How many aggregated features do you have? More than 400? I think more features increase the converge time and lead overfitting. at least for me :(
", "votes": null, "replies": [] }, { "id": 459692, "author_name": "chocozzz", "author_url": "", "post_date": "01/22/2019 07:23:30", "content": "@silverstone1903 i use a 170 features. i also think that more features can be lead to overfitting. but i try to do more aggregation to believe feature selection :D
", "votes": null, "replies": [] }, { "id": 459840, "author_name": "chandureddyvari", "author_url": "", "post_date": "01/22/2019 13:21:32", "content": "Thank you for your suggestions. You guys are really awesome. Good luck!
", "votes": null, "replies": [] } ] }, { "id": 459551, "author_name": "subikashpal", "author_url": "", "post_date": "01/22/2019 01:08:30", "content": "β Inconsistencies should be able to fixed by fixed date and removing the seed.\nβ concentrate on date features/elapsed date. You should be able to mine some really interesting features.\nβ I would suggest u spend some time on merchant features on historical transactions.\nβ aggregation based on categories is really very frustrating but can lead to some top features.
\n\nAll these helped me to score 3.84 single model LGB.
\n\nCurious to see your NN embedding stuff.
", "votes": null, "replies": [ { "id": 459838, "author_name": "chandureddyvari", "author_url": "", "post_date": "01/22/2019 13:18:40", "content": "That's a very helpful cue. Thank you!! Sure I'll publish the kernel soon. Possibly by tomorrow.
", "votes": null, "replies": [] } ] }, { "id": 459989, "author_name": "chandureddyvari", "author_url": "", "post_date": "01/22/2019 19:38:25", "content": "Here's the link to kernel guys https://www.kaggle.com/chandureddyvari/sprinkling-in-some-deep-learning-using-fast-ai
", "votes": null, "replies": [ { "id": 460068, "author_name": "chocozzz", "author_url": "", "post_date": "01/22/2019 23:21:01", "content": "This is neural embbeding??? Thanks for sharing. i first saw it :D
", "votes": null, "replies": [] }, { "id": 460202, "author_name": "chandureddyvari", "author_url": "", "post_date": "01/23/2019 07:27:34", "content": "Yes. People who won 2nd place for Rossmann competition had good success with it supposedly without much feature engineering as first place people. Do try it on your current feature set and let me know if it gave you some boost on LB.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "458542": "Hello kagglers,\n\nThis is my first kaggle competition. The community here is amazing and very inspiring. I've learnt much much more from the kernels here than any other online resources. I started out very excited. But I'm basically stuck at the same point for the past few days. I'm running out of ideas and getting frustrated. So I'm here seeking your help to shepherd me in the right direction. π\n\nI'm using Stratified KFold with lgbm as done [here](https://www.kaggle.com/chauhuynh/my-first-kernel-3-699) . I had some success extracting some of the features mentioned in the kernel before referring it.\n\nMy best score so far is 3.695 (with ideas picked from the kernels & discussions) on LB with LGBM single model without any stacking, blending\n\nProblem 1:\n----------\n\nNot able to reproduce my results consistently. \nThe same code is giving multiple CV scores for same LB score. For eg.\n3.65377489489624\n3.65462490707428\n3.654910478618945\nIs it because I used `datetime.datetime.today()` while calculating aggregates? I tried setting to Nov 1st 2018. It brought down my CV score. So I didn't venture further\n\nProblem 2:\n----------\n\nInconsistent gap between LB & local score:\n\n1. **3.65975**96743401666, LB - 3.707 \n2. **3.65672**6211308068, LB - 3.696 \n3. **3.65548**01486969906, LB- 3.695\n4. **3.65420**8034404632, LB - 3.697 π€―\n5. **3.65565**48946965446, LB - 3.696\n6. **3.65493**381575084, LB - 3.697 \n\nProblem 3:\n----------\n\nI'm trying to compute share of wallet - customer spend per category. Purchase amount aggregates on category_2, category_3, category_1 are bringing down score.\nAdding sum & mean aggregates is worsening my scores. [I read in the forums SubikashPal were able to achieve 3.68xx score on LB just by aggregating without using merchant info.](https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/77652) [And making small changes to public kernel was able give SomethingIsWrong a big push](https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73517#438795). That's amazing. I'm just too blind may be.\n\nWhat I've tried\n---------------\n\n1. Adding repurchase rate, weekend, avg spend per transaction, recency, participation rate - worked thanks @yifanxie, @chauhuynh\n2. Monthly aggregates & successive aggregates as mentioned [here](https://www.kaggle.com/fabiendaniel/elo-world)\n3. Using Neural net Embeddings brought the CV score to 3.69715275269681. Didn't improve further with current features (It's based on fast.ai. I'll publish a kernel if you're curious)\nEdit: [Kernel is available now. Using latest fastai version but not with Kfold][1]\n4. Separated auth & non-auth transactions and aggregated separately - worsened\n\nNot sure if I'm doing some glaring mistakes or if I'm sidetracked or if they really didn't work for you either.\n\nWhat I'm yet to try:\n--------------------\n\n1. Stacking, blending\n2. Add a meta info column for outlier classification and blend it as suggested in discussion forums\n3. Use merchant info and aggregate\n4. Binning \n5. No of times customer switches between merchant within same category \n6. FM as mentioned in forums\n7. Feature elimination and others mentioned [here](https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73937)\n\nπIf you can give me any cues for improving the single model score 3.695 I'd be indebted to you π\n\n### snippet\n```\ndef agg_on_cat(df, category, feature):\n temp_df = df.pivot_table(index='card_id', columns=category, aggfunc={feature: ['sum', 'mean']})\n cols = [category + '_{0[2]}_{0[0]}_{0[1]}'.format(col) for col in temp_df.columns.tolist()]\n temp_df.columns = cols\n return temp_df\n\ndef get_cat_agg(df):\n agg_df = agg_on_cat(df, 'category_1', 'purchase_amount')\n agg_df = pd.merge(agg_df, agg_on_cat(df, 'category_2', 'purchase_amount'), on='card_id', how='left')\n agg_df = pd.merge(agg_df, agg_on_cat(df, 'category_3', 'purchase_amount'), on='card_id', how='left')\n agg_df = pd.merge(agg_df, agg_on_cat(df, 'authorized_flag', 'purchase_amount'), on='card_id', how='left')\n return agg_df\n```\n\n\n [1]: https://www.kaggle.com/chandureddyvari/sprinkling-in-some-deep-learning-using-fast-ai/", "458922": "Problem 2 is a common problem that is often times just an artifact of random noise... see [\"Mind the Gap and Trust your CV\"](https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73936)", "458944": "You should try to do feature selection. It helped me a lot to improve my lb score. Also I'm curios about NN embeddings and waiting for your kernel...", "459047": "Sure. I'll publish it in 2 days. Any kernels of old competitions you can point me to for feature extraction?", "459048": "Thanks for the reply, I'll trust the model with best CV score for blending and stacking.", "459065": "Check this out: https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73937", "459384": "Thanks for the pointer. I'll try them for sure. You had any luck aggregating on categories? Because that's what is worsening my score. Just want to clarify if it helped you. Please let me know", "459410": "TBH I have not spent too much time on feature extraction. Just using aggregated data and some feature interactions with feature selection. Now planning to start over and try to work on category features. So I'll let you know...", "459534": "i recommend you to try more aggregation. I reached at LB 3.688 only using aggregation(single model). and some people reached 3.680 :D", "459551": "β Inconsistencies should be able to fixed by fixed date and removing the seed.\nβ concentrate on date features/elapsed date. You should be able to mine some really interesting features.\nβ I would suggest u spend some time on merchant features on historical transactions.\nβ aggregation based on categories is really very frustrating but can lead to some top features.\n\nAll these helped me to score 3.84 single model LGB.\n\nCurious to see your NN embedding stuff.", "459553": "silverstone.\nWhich feature selection are u using? borota ot Null feature importance. I am having 140+ features and not able to improve a single point removing any features. But i know there is a scope. I spend lots of time there without success. Any clue here βΊοΈ", "459597": "Wow that's awesome. I've tried aggregation over categories and authorised trans etc. and all but it just brought down my score. I'll visit again and will try to do more aggregation on all the categorical variables. Thanks for the headsup!", "459608": "Yes. I also try to aggregation more and more when i reached in 3.67x. Good Luck !!", "459632": "I'm using a mix of null importances & boruta's results with different parameters. Like a said now I reviewing whole process.", "459635": "How many aggregated features do you have? More than 400? I think more features increase the converge time and lead overfitting. at least for me :(", "459692": "silverstone1903 i use a 170 features. i also think that more features can be lead to overfitting. but i try to do more aggregation to believe feature selection :D", "459838": "That's a very helpful cue. Thank you!! Sure I'll publish the kernel soon. Possibly by tomorrow.", "459840": "Thank you for your suggestions. You guys are really awesome. Good luck!", "459989": "Here's the link to kernel guys https://www.kaggle.com/chandureddyvari/sprinkling-in-some-deep-learning-using-fast-ai", "460068": "This is neural embbeding??? Thanks for sharing. i first saw it :D", "460120": "Although it may sound meaningless to you, discussion is extremely helpful and peers here are generally helpful. So don't give up and just continue. Take a break and then continue. Just continue no matter what.", "460202": "Yes. People who won 2nd place for Rossmann competition had good success with it supposedly without much feature engineering as first place people. Do try it on your current feature set and let me know if it gave you some boost on LB.", "460204": "Thanks for the inspiring words Frank :)" }, "source": "meta" }