The trick shared by @30CrMnSiA is amazing, any other tricks you learned in this competition?
", "messages": [ { "id": "479162", "postDate": "02/27/2019 01:58:09", "content": "The trick shared by @30CrMnSiA is amazing, any other tricks you learned in this competition?
", "rawMarkdown": "The trick shared by @30CrMnSiA is amazing, any other tricks you learned in this competition?", "votes": null }, { "id": "479174", "postDate": "02/27/2019 02:06:57", "content": "One small idea is separate the History data into two parts according to the authorized_code, and extract features separately, which help get 0.003 boost at about 3.690 public lb.
", "rawMarkdown": "One small idea is separate the History data into two parts according to the authorized_code, and extract features separately, which help get 0.003 boost at about 3.690 public lb.", "votes": null }, { "id": "479178", "postDate": "02/27/2019 02:13:48", "content": "I have the same idea with @Wang Jianfei. Another small trick is from @raddar's kernel target - true meaning revealed!: inverse transform the purchase_amount, since I generate lots of features based on this, it helps to improve both CV and LB.
Decrpt the purchase amount using Raddar's formula for new_transactions dataset and apply a log1p transform to it, this helped in removing outliers ! 0.04 boost :)
", "rawMarkdown": "Decrpt the purchase amount using Raddar's formula for new_transactions dataset and apply a log1p transform to it, this helped in removing outliers ! 0.04 boost :)", "votes": null }, { "id": "479199", "postDate": "02/27/2019 02:32:43", "content": "I did the same split by auth code and split by cat_1 as well.
", "rawMarkdown": "I did the same split by auth code and split by cat_1 as well.", "votes": null }, { "id": "479201", "postDate": "02/27/2019 02:37:59", "content": "Feature reduction. Here is my post on feature selection techniques https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/82052
", "rawMarkdown": "Feature reduction. Here is my post on feature selection techniques https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/82052", "votes": null }, { "id": "479202", "postDate": "02/27/2019 02:40:58", "content": "about the detail, how much imp with this operation?
", "rawMarkdown": "about the detail, how much imp with this operation?", "votes": null }, { "id": "479204", "postDate": "02/27/2019 02:43:44", "content": "another big imp is from my teammate@ salamatabout time features, give us from 3.685 to 3.675 public lb, which should help in private lb.
", "rawMarkdown": "another big imp is from my teammate@ [salamat][1]about time features, give us from 3.685 to 3.675 public lb, which should help in private lb.\n\n\n [1]: https://www.kaggle.com/salamatburj", "votes": null }, { "id": "479698", "postDate": "02/27/2019 10:52:17", "content": "I turned the target to 0 and positive only (target+33.2) and used Poisson regression, getting predictions back to original values (prediction-33.2)\nthis added good diversity to my stacking
", "rawMarkdown": "I turned the target to 0 and positive only (target+33.2) and used Poisson regression, getting predictions back to original values (prediction-33.2)\nthis added good diversity to my stacking", "votes": null }, { "id": "479774", "postDate": "02/27/2019 12:16:31", "content": "Yes, you are really creative! and details about the boost of lb and cv?
", "rawMarkdown": "Yes, you are really creative! and details about the boost of lb and cv?", "votes": null }, { "id": "479777", "postDate": "02/27/2019 12:21:50", "content": "Im glad it helped!
", "rawMarkdown": "Im glad it helped!", "votes": null }, { "id": "479801", "postDate": "02/27/2019 12:40:58", "content": "Can't say for sure but correlation with my base model was lower than simply changing the hyperparameters with same loss function , so provided good diversity for stacking
", "rawMarkdown": "Can't say for sure but correlation with my base model was lower than simply changing the hyperparameters with same loss function , so provided good diversity for stacking", "votes": null }, { "id": "479844", "postDate": "02/27/2019 13:43:24", "content": "I used the idea with having one model including outliers and one without outlier. The slight innovation from my side was using a third model, a classifier, to choose the breakpoint. Looking at the ROC, I choose the breakpoint as argmax(truePositiveRate - falsePositiveRate), which meant between 32000-35000 false positives usually were using the model including outliers, and around 1800 true positives, whereas the remaining set used the regressor with no outliers. The intuition being that the outliers missed by this threshold could be considered completely unpredictable, and thus should not impact the predictions for the rest of the dataset.
\n\nFor my best submission (which I ended up not choosing for final scoring, instead opting for a bigger blend that ended up performing worse on private lb), this simple blend yielded 3.604 private lb, 3.679 public lb, down from 3.609/3.682 for the raw single model.
", "rawMarkdown": "I used the idea with having one model including outliers and one without outlier. The slight innovation from my side was using a third model, a classifier, to choose the breakpoint. Looking at the ROC, I choose the breakpoint as argmax(truePositiveRate - falsePositiveRate), which meant between 32000-35000 false positives usually were using the model including outliers, and around 1800 true positives, whereas the remaining set used the regressor with no outliers. The intuition being that the outliers missed by this threshold could be considered completely unpredictable, and thus should not impact the predictions for the rest of the dataset.\n\nFor my best submission (which I ended up not choosing for final scoring, instead opting for a bigger blend that ended up performing worse on private lb), this simple blend yielded 3.604 private lb, 3.679 public lb, down from 3.609/3.682 for the raw single model.", "votes": null }, { "id": "479872", "postDate": "02/27/2019 14:19:33", "content": "The following gave me 0.00241 CV improvement:
\n\n\nhistory_df[(history_df['is_historical'] == 1) &(history_df['category_1'] == 0)].copy()\n\n\nhistory_df[(history_df['is_historical'] == 0) & (history_df['category_1'] == 1)].copy()\n
The following gave me 0.00195 CV improvement:
\n\n\nhistory_df[(history_df['is_historical'] == 1) & (history_df['authorized_flag'] == 1)].copy()\n
Very creative, thanks for sharing!
", "rawMarkdown": "Very creative, thanks for sharing!", "votes": null } ], "comments": [ { "id": 479174, "author_name": "codlife", "author_url": "", "post_date": "02/27/2019 02:06:57", "content": "One small idea is separate the History data into two parts according to the authorized_code, and extract features separately, which help get 0.003 boost at about 3.690 public lb.
", "votes": null, "replies": [ { "id": 479199, "author_name": "kingychiu", "author_url": "", "post_date": "02/27/2019 02:32:43", "content": "I did the same split by auth code and split by cat_1 as well.
", "votes": null, "replies": [] }, { "id": 479202, "author_name": "codlife", "author_url": "", "post_date": "02/27/2019 02:40:58", "content": "about the detail, how much imp with this operation?
", "votes": null, "replies": [] }, { "id": 479204, "author_name": "codlife", "author_url": "", "post_date": "02/27/2019 02:43:44", "content": "another big imp is from my teammate@ salamatabout time features, give us from 3.685 to 3.675 public lb, which should help in private lb.
", "votes": null, "replies": [] }, { "id": 479872, "author_name": "kingychiu", "author_url": "", "post_date": "02/27/2019 14:19:33", "content": "The following gave me 0.00241 CV improvement:
\n\n\nhistory_df[(history_df['is_historical'] == 1) &(history_df['category_1'] == 0)].copy()\n\n\nhistory_df[(history_df['is_historical'] == 0) & (history_df['category_1'] == 1)].copy()\n
The following gave me 0.00195 CV improvement:
\n\n\nhistory_df[(history_df['is_historical'] == 1) & (history_df['authorized_flag'] == 1)].copy()\n
I have the same idea with @Wang Jianfei. Another small trick is from @raddar's kernel target - true meaning revealed!: inverse transform the purchase_amount, since I generate lots of features based on this, it helps to improve both CV and LB.
Im glad it helped!
", "votes": null, "replies": [] } ] }, { "id": 479185, "author_name": "yyhhlancelot", "author_url": "", "post_date": "02/27/2019 02:20:33", "content": "", "votes": null, "replies": [] }, { "id": 479192, "author_name": "prashanththangavel", "author_url": "", "post_date": "02/27/2019 02:28:19", "content": "Decrpt the purchase amount using Raddar's formula for new_transactions dataset and apply a log1p transform to it, this helped in removing outliers ! 0.04 boost :)
", "votes": null, "replies": [] }, { "id": 479201, "author_name": "sheriytm", "author_url": "", "post_date": "02/27/2019 02:37:59", "content": "Feature reduction. Here is my post on feature selection techniques https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/82052
", "votes": null, "replies": [] }, { "id": 479698, "author_name": "tpthegreat", "author_url": "", "post_date": "02/27/2019 10:52:17", "content": "I turned the target to 0 and positive only (target+33.2) and used Poisson regression, getting predictions back to original values (prediction-33.2)\nthis added good diversity to my stacking
", "votes": null, "replies": [ { "id": 479774, "author_name": "codlife", "author_url": "", "post_date": "02/27/2019 12:16:31", "content": "Yes, you are really creative! and details about the boost of lb and cv?
", "votes": null, "replies": [] }, { "id": 479801, "author_name": "tpthegreat", "author_url": "", "post_date": "02/27/2019 12:40:58", "content": "Can't say for sure but correlation with my base model was lower than simply changing the hyperparameters with same loss function , so provided good diversity for stacking
", "votes": null, "replies": [] }, { "id": 479949, "author_name": "brunogc", "author_url": "", "post_date": "02/27/2019 15:56:07", "content": "Very creative, thanks for sharing!
", "votes": null, "replies": [] } ] }, { "id": 479844, "author_name": "reimersholme", "author_url": "", "post_date": "02/27/2019 13:43:24", "content": "I used the idea with having one model including outliers and one without outlier. The slight innovation from my side was using a third model, a classifier, to choose the breakpoint. Looking at the ROC, I choose the breakpoint as argmax(truePositiveRate - falsePositiveRate), which meant between 32000-35000 false positives usually were using the model including outliers, and around 1800 true positives, whereas the remaining set used the regressor with no outliers. The intuition being that the outliers missed by this threshold could be considered completely unpredictable, and thus should not impact the predictions for the rest of the dataset.
\n\nFor my best submission (which I ended up not choosing for final scoring, instead opting for a bigger blend that ended up performing worse on private lb), this simple blend yielded 3.604 private lb, 3.679 public lb, down from 3.609/3.682 for the raw single model.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "479162": "The trick shared by @30CrMnSiA is amazing, any other tricks you learned in this competition?", "479174": "One small idea is separate the History data into two parts according to the authorized_code, and extract features separately, which help get 0.003 boost at about 3.690 public lb.", "479178": "I have the same idea with [@Wang Jianfei](https://www.kaggle.com/codlife). Another small trick is from [@raddar](https://www.kaggle.com/raddar)'s kernel [target - true meaning revealed!](https://www.kaggle.com/raddar/target-true-meaning-revealed): inverse transform the `purchase_amount`, since I generate lots of features based on this, it helps to improve both CV and LB.", "479185": "", "479192": "Decrpt the purchase amount using Raddar's formula for new_transactions dataset and apply a log1p transform to it, this helped in removing outliers ! 0.04 boost :)", "479199": "I did the same split by auth code and split by cat_1 as well.", "479201": "Feature reduction. Here is my post on feature selection techniques https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/82052", "479202": "about the detail, how much imp with this operation?", "479204": "another big imp is from my teammate@ [salamat][1]about time features, give us from 3.685 to 3.675 public lb, which should help in private lb.\n\n\n [1]: https://www.kaggle.com/salamatburj", "479698": "I turned the target to 0 and positive only (target+33.2) and used Poisson regression, getting predictions back to original values (prediction-33.2)\nthis added good diversity to my stacking", "479774": "Yes, you are really creative! and details about the boost of lb and cv?", "479777": "Im glad it helped!", "479801": "Can't say for sure but correlation with my base model was lower than simply changing the hyperparameters with same loss function , so provided good diversity for stacking", "479844": "I used the idea with having one model including outliers and one without outlier. The slight innovation from my side was using a third model, a classifier, to choose the breakpoint. Looking at the ROC, I choose the breakpoint as argmax(truePositiveRate - falsePositiveRate), which meant between 32000-35000 false positives usually were using the model including outliers, and around 1800 true positives, whereas the remaining set used the regressor with no outliers. The intuition being that the outliers missed by this threshold could be considered completely unpredictable, and thus should not impact the predictions for the rest of the dataset.\n\nFor my best submission (which I ended up not choosing for final scoring, instead opting for a bigger blend that ended up performing worse on private lb), this simple blend yielded 3.604 private lb, 3.679 public lb, down from 3.609/3.682 for the raw single model.", "479872": "The following gave me 0.00241 CV improvement:\n\n```\nhistory_df[(history_df['is_historical'] == 1) &(history_df['category_1'] == 0)].copy()\n```\n```\nhistory_df[(history_df['is_historical'] == 0) & (history_df['category_1'] == 1)].copy()\n```\n\nThe following gave me 0.00195 CV improvement:\n\n```\nhistory_df[(history_df['is_historical'] == 1) & (history_df['authorized_flag'] == 1)].copy()\n```", "479949": "Very creative, thanks for sharing!" }, "source": "meta" }