{ "id": 82093, "title": "10th place solution", "url": "/competitions/elo-merchant-category-recommendation/discussion/82093", "author_name": "YuryBolkonsky", "post_date": "2019-02-27T06:30:41.874000", "votes": 48, "comment_count": 23, "views": 0, "content": "

Thanks to all competitors for this wonderfull day) I am very happy to get first Gold and become Kaggle Master)

\n\n

PREPROCESSING and FE

\n\n
    \n
  1. Merging historical data and new_data with authorized_flag=1
    2. \n
  2. Count Vectorizer merchant id, subsector id, merchant category id + PCA, SVD
    3. \n
  3. Grouping month lag data (auth month-13, auth month-12 and etc.) for count purchases, and sum adjusted purchase_amount, std, min, max
    4. \n
  4. Ratio every auth_month_purchase_amount-13,...auth month purchase amount+2 to previous with interval 2,4,6
    5. \n
  5. Predicting month lag 3 and 4 and calculating ratio for them
    6. \n
  6. Simple Rolling mean and SimpleExpSmoothing over auth purchase amount-13, ...auth purchase amount+4 over month lags
    7. \n
  7. Calculating avg, min, max purchase time elapsed periods beetween\ntransactions
    8. \n
\n\n

FEATURE SELECTION

\n\n

Finally, i created almost 6500 features and created different features sets using

\n\n
    \n
  1. Boruta (it took almost 8 hours to select 500 best features)
    2. \n
  2. Selecting best features based on Catboost, LGBM, XGB
    3. \n
  3. Deleting features based on Adversarial validation
    4. \n
\n\n

Best Single Model and Ensembling

\n\n

My best single model is Catboost over 5 stratified folds CV 3.645 std 0.012 LB 3.671. Also tried DeepFM but CV 3.656 std 0.016 and LB 3.682. Other models, including LGBM sometimes got even better CV score, but were worse on LB.

\n\n

Finally, i collected 47 models with outliers, and 35 models without them. For combining submissions, also created classification model for outliers ROC 0.907 F1 0.1025

\n\n
    \n
  1. Stacking models with outliers CV 3.637 std 0.013 LB 3.669
    2. \n
  2. Stacking models without outliers CV 1.544 std 0.011
    3. \n
  3. Combining models, replacing 10000 values
    4. \n
\n\n

Thanks for sharing ideas end experience

\n\n

@raddar Post about real meaning of target and @peterhurford for this feature selection post

", "messages": [ { "id": 479444, "postDate": "2019-02-27T06:30:41.873Z", "content": "

Thanks to all competitors for this wonderfull day) I am very happy to get first Gold and become Kaggle Master)

\n\n

PREPROCESSING and FE

\n\n
    \n
  1. Merging historical data and new_data with authorized_flag=1
    2. \n
  2. Count Vectorizer merchant id, subsector id, merchant category id + PCA, SVD
    3. \n
  3. Grouping month lag data (auth month-13, auth month-12 and etc.) for count purchases, and sum adjusted purchase_amount, std, min, max
    4. \n
  4. Ratio every auth_month_purchase_amount-13,...auth month purchase amount+2 to previous with interval 2,4,6
    5. \n
  5. Predicting month lag 3 and 4 and calculating ratio for them
    6. \n
  6. Simple Rolling mean and SimpleExpSmoothing over auth purchase amount-13, ...auth purchase amount+4 over month lags
    7. \n
  7. Calculating avg, min, max purchase time elapsed periods beetween\ntransactions
    8. \n
\n\n

FEATURE SELECTION

\n\n

Finally, i created almost 6500 features and created different features sets using

\n\n
    \n
  1. Boruta (it took almost 8 hours to select 500 best features)
    2. \n
  2. Selecting best features based on Catboost, LGBM, XGB
    3. \n
  3. Deleting features based on Adversarial validation
    4. \n
\n\n

Best Single Model and Ensembling

\n\n

My best single model is Catboost over 5 stratified folds CV 3.645 std 0.012 LB 3.671. Also tried DeepFM but CV 3.656 std 0.016 and LB 3.682. Other models, including LGBM sometimes got even better CV score, but were worse on LB.

\n\n

Finally, i collected 47 models with outliers, and 35 models without them. For combining submissions, also created classification model for outliers ROC 0.907 F1 0.1025

\n\n
    \n
  1. Stacking models with outliers CV 3.637 std 0.013 LB 3.669
    2. \n
  2. Stacking models without outliers CV 1.544 std 0.011
    3. \n
  3. Combining models, replacing 10000 values
    4. \n
\n\n

Thanks for sharing ideas end experience

\n\n

@raddar Post about real meaning of target and @peterhurford for this feature selection post

", "rawMarkdown": "Thanks to all competitors for this wonderfull day) I am very happy to get first Gold and become Kaggle Master)\n\n**PREPROCESSING and FE**\n\n 1. Merging historical data and new_data with authorized_flag=1\n 2. Count Vectorizer merchant id, subsector id, merchant category id + PCA, SVD\n 3. Grouping month lag data (auth month-13, auth month-12 and etc.) for count purchases, and sum adjusted purchase_amount, std, min, max\n 4. Ratio every auth_month_purchase_amount-13,...auth month purchase amount+2 to previous with interval 2,4,6\n 5. Predicting month lag 3 and 4 and calculating ratio for them\n 6. Simple Rolling mean and SimpleExpSmoothing over auth purchase amount-13, ...auth purchase amount+4 over month lags\n 7. Calculating avg, min, max purchase time elapsed periods beetween\n transactions\n\n**FEATURE SELECTION**\n\nFinally, i created almost 6500 features and created different features sets using\n\n 1. Boruta (it took almost 8 hours to select 500 best features)\n 2. Selecting best features based on Catboost, LGBM, XGB\n 3. Deleting features based on Adversarial validation\n\n**Best Single Model and Ensembling**\n\nMy best single model is Catboost over 5 stratified folds CV 3.645 std 0.012 LB 3.671. Also tried DeepFM but CV 3.656 std 0.016 and LB 3.682. Other models, including LGBM sometimes got even better CV score, but were worse on LB. \n\n**Finally, i collected 47 models with outliers, and 35 models without them**. For combining submissions, also created classification model for outliers ROC 0.907 F1 0.1025\n\n 1. Stacking models with outliers CV 3.637 std 0.013 LB 3.669\n 2. Stacking models without outliers CV 1.544 std 0.011\n 3. Combining models, replacing 10000 values\n\n**Thanks for sharing ideas end experience**\n\n[@raddar][1] Post about [real meaning of target ][2] and [@peterhurford][3] for this [feature selection post][4]\n\n\n [1]: https://www.kaggle.com/raddar\n [2]: https://www.kaggle.com/raddar/target-true-meaning-revealed/\n [3]: https://www.kaggle.com/peterhurford\n [4]: https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73937", "votes": 48 }, { "id": 739465, "postDate": "2020-02-07T21:06:14Z", "content": "

@bolkonsky, congratulations! Thanks for sharing your approaches and magic! I added your post to my collection of the best kernels and posts of winners of Kaggle Prize Competitions: [Data Science for tabular data: Advanced Techniques] (https://www.kaggle.com/vbmokin/data-science-for-tabular-data-advanced-techniques)

", "rawMarkdown": "@bolkonsky, congratulations! Thanks for sharing your approaches and magic! I added your post to my collection of the best kernels and posts of winners of Kaggle Prize Competitions: [Data Science for tabular data: Advanced Techniques] (https://www.kaggle.com/vbmokin/data-science-for-tabular-data-advanced-techniques) ", "votes": 2 }, { "id": 479696, "postDate": "2019-02-27T10:49:25.953Z", "content": "

Good job!!!\nYour feature selection based on adversarial validation is new to me, I'll sure try it in the future!!\nThank you for sharing

", "rawMarkdown": "Good job!!!\nYour feature selection based on adversarial validation is new to me, I'll sure try it in the future!!\nThank you for sharing", "votes": 1, "replies": [ { "id": 479757, "postDate": "2019-02-27T11:58:03.823Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 479469, "postDate": "2019-02-27T07:04:51.367Z", "content": "

Congrats! \nClean solution! Thank you for sharing.\nPlease let me ask a question about Feature Selection.

\n\n
\n

Deleting features based on Adversarial validation

\n
\n\n

Does this mean that you excluded features that are useful for classifying Train and Test?

", "rawMarkdown": "Congrats! \nClean solution! Thank you for sharing.\nPlease let me ask a question about Feature Selection.\n> Deleting features based on Adversarial validation \n \nDoes this mean that you excluded features that are useful for classifying Train and Test?", "votes": 1, "replies": [ { "id": 479513, "postDate": "2019-02-27T07:55:38.627Z", "content": "

Thanks. Yes, i deleted features which were most important in classifying Train and Test

", "rawMarkdown": "Thanks. Yes, i deleted features which were most important in classifying Train and Test", "votes": 1 }, { "id": 479786, "postDate": "2019-02-27T12:29:40.330Z", "content": "

Thank you!

", "rawMarkdown": "Thank you!" }, { "id": 529120, "postDate": "2019-05-09T08:22:06.793Z", "content": "

thank you for sharing a awesome solution!\nbut i have a question about adversarial validation in part of features selection .\nIs that because the feature cause overfitting or something what?\nhope your reply

", "rawMarkdown": "thank you for sharing a awesome solution!\nbut i have a question about adversarial validation in part of features selection .\nIs that because the feature cause overfitting or something what?\nhope your reply" } ] }, { "id": 529125, "postDate": "2019-05-09T08:28:06.743Z", "content": "

and one more question, you select features from different models by the fixed number or ratio? and you select the feature that preform well at all of models or one of them?\nthank you! :)

", "rawMarkdown": "and one more question, you select features from different models by the fixed number or ratio? and you select the feature that preform well at all of models or one of them?\nthank you! :)" }, { "id": 482606, "postDate": "2019-03-03T10:59:51.040Z", "content": "

Hello. Congratulations !!! I have questions. Ratio every authmonthpurchase_amount-13,…auth month purchase amount+2 to previous with interval 2,4,6 <- what's mean of interval 2,4,6 ??? You did authmonthpurchase_amount-11/authmonthpurchase_amount-13 , authmonthpurchase_amount-9/authmonthpurchase_amount-13???

", "rawMarkdown": "Hello. Congratulations !!! I have questions. `Ratio every authmonthpurchase_amount-13,…auth month purchase amount+2 to previous with interval 2,4,6` <- what's mean of interval 2,4,6 ??? You did `authmonthpurchase_amount-11/authmonthpurchase_amount-13` , `authmonthpurchase_amount-9/authmonthpurchase_amount-13`???", "replies": [ { "id": 484955, "postDate": "2019-03-06T17:29:35.690Z", "content": "

Yes, you are right

", "rawMarkdown": "Yes, you are right" } ] }, { "id": 482407, "postDate": "2019-03-02T23:37:34.517Z", "content": "

Congratulations!\n\"5 . Predicting month lag 3 and 4 and calculating ratio for them\"
\nWhat's that mean? There is no month lag 3 and 4 in the data.

", "rawMarkdown": "Congratulations!\n\"5 . Predicting month lag 3 and 4 and calculating ratio for them\" \nWhat's that mean? There is no month lag 3 and 4 in the data." }, { "id": 480319, "postDate": "2019-02-28T03:37:04.183Z", "content": "

Congratulations and thank you for sharing.

", "rawMarkdown": "Congratulations and thank you for sharing." }, { "id": 480299, "postDate": "2019-02-28T02:41:20.257Z", "content": "

Congrats on your first gold!

", "rawMarkdown": "Congrats on your first gold!" }, { "id": 479708, "postDate": "2019-02-27T11:02:42.563Z", "content": "

My catboost didn't yield better results than my lgbm. What is the param that you have used?

", "rawMarkdown": "My catboost didn't yield better results than my lgbm. What is the param that you have used?", "replies": [ { "id": 479766, "postDate": "2019-02-27T12:09:33.933Z", "content": "

It depends on what params you are using with lgbm. I used these params in one of my model. As you can see it's very simple. No hyperparams tuned

\n\n

params = {'iterations': 15000, \n 'learning_rate': 0.01, \n 'eval_metric': 'RMSE', \n 'metric_period': 200,\n 'l2_leaf_reg': 10,\n 'od_type':'Iter',\n 'od_wait': 1000,\n 'task_type': 'GPU',\n 'depth':8,\n 'random_seed': 0,\n 'use_best_model': True}

", "rawMarkdown": "It depends on what params you are using with lgbm. I used these params in one of my model. As you can see it's very simple. No hyperparams tuned\n\nparams = {'iterations': 15000, \n 'learning_rate': 0.01, \n 'eval_metric': 'RMSE', \n 'metric_period': 200,\n 'l2_leaf_reg': 10,\n 'od_type':'Iter',\n 'od_wait': 1000,\n 'task_type': 'GPU',\n 'depth':8,\n 'random_seed': 0,\n 'use_best_model': True}", "votes": 3 }, { "id": 479866, "postDate": "2019-02-27T14:11:53.613Z", "content": "

Thanks for your reply. I have tried to use classification to predict outliers but it give the highest prob of less than 0.5 only. If i multiplied it by -33, my CV score and PB was horrible. I mean do you select top10% card_id from classification and then add it up with all card_id from your regression model? Mind to share on this part if it is not too much to ask?

", "rawMarkdown": "Thanks for your reply. I have tried to use classification to predict outliers but it give the highest prob of less than 0.5 only. If i multiplied it by -33, my CV score and PB was horrible. I mean do you select top10% card_id from classification and then add it up with all card_id from your regression model? Mind to share on this part if it is not too much to ask?" } ] }, { "id": 479563, "postDate": "2019-02-27T08:46:33.533Z", "content": "

Congrats!\nI try the Count Vectorizer for merchant_id, subsector_id.etc too, but failed to boost public lb. Besides i try to use NMF/SVD for card-maerchant-category matrix failed boost public lb too. So can you share your result about this? thank you very much!

", "rawMarkdown": "Congrats!\nI try the Count Vectorizer for merchant_id, subsector_id.etc too, but failed to boost public lb. Besides i try to use NMF/SVD for card-maerchant-category matrix failed boost public lb too. So can you share your result about this? thank you very much!", "replies": [ { "id": 479683, "postDate": "2019-02-27T10:39:57.577Z", "content": "

I tried Count Vectorizer. PCA for merchantid (15 components), SVD for subsectorid, merchant_subcategory_id (10 components). So, i tried to reach cum explained_variance_ratio_ at least 0.8.

\n\n

So in some features sets i used Count Vectorizer matrix + top50 features, in other features + pca, svd. It helps my model to add CV a little in single model 0.0007.

", "rawMarkdown": "I tried Count Vectorizer. PCA for merchantid (15 components), SVD for subsectorid, merchant_subcategory_id (10 components). So, i tried to reach cum explained_variance_ratio_ at least 0.8.\n \nSo in some features sets i used Count Vectorizer matrix + top50 features, in other features + pca, svd. It helps my model to add CV a little in single model 0.0007." }, { "id": 479769, "postDate": "2019-02-27T12:13:23.460Z", "content": "

Yes， thank you, for me should help cv too, but don't see the boost in public lb and give up.

", "rawMarkdown": "Yes， thank you, for me should help cv too, but don't see the boost in public lb and give up." } ] }, { "id": 479552, "postDate": "2019-02-27T08:36:06.200Z", "content": "

Congrats! ِPure ML solution and acting bravely!

", "rawMarkdown": "Congrats! ِPure ML solution and acting bravely! " }, { "id": 479473, "postDate": "2019-02-27T07:10:57.030Z", "content": "

Congratulations @bolkonsky Thanks for sharing the solution.

", "rawMarkdown": "Congratulations @bolkonsky Thanks for sharing the solution." }, { "id": 2069654, "postDate": "2022-12-19T08:23:05.740Z", "content": "

Great Job!
\nThank you for sharing.

", "rawMarkdown": "Great Job!\nThank you for sharing." }, { "id": 480499, "postDate": "2019-02-28T08:49:13.003Z", "content": "

great, thank you for sharing

", "rawMarkdown": "great, thank you for sharing" } ], "comments": [ { "id": 739465, "author_name": "Vitalii Mokin", "author_url": "", "post_date": "2020-02-07T21:06:14", "content": "

@bolkonsky, congratulations! Thanks for sharing your approaches and magic! I added your post to my collection of the best kernels and posts of winners of Kaggle Prize Competitions: [Data Science for tabular data: Advanced Techniques] (https://www.kaggle.com/vbmokin/data-science-for-tabular-data-advanced-techniques)

", "votes": 2, "replies": [] }, { "id": 479696, "author_name": "AmirH", "author_url": "", "post_date": "2019-02-27T10:49:25.953000", "content": "

Good job!!!\nYour feature selection based on adversarial validation is new to me, I'll sure try it in the future!!\nThank you for sharing

", "votes": 1, "replies": [ { "id": 479757, "author_name": "", "author_url": "", "post_date": "2019-02-27T11:58:03.823000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 479469, "author_name": "holygo_1208", "author_url": "", "post_date": "2019-02-27T07:04:51.367000", "content": "

Congrats! \nClean solution! Thank you for sharing.\nPlease let me ask a question about Feature Selection.

\n\n
\n

Deleting features based on Adversarial validation

\n
\n\n

Does this mean that you excluded features that are useful for classifying Train and Test?

", "votes": 1, "replies": [ { "id": 479513, "author_name": "YuryBolkonsky", "author_url": "", "post_date": "2019-02-27T07:55:38.627000", "content": "

Thanks. Yes, i deleted features which were most important in classifying Train and Test

", "votes": 1, "replies": [] }, { "id": 479786, "author_name": "holygo_1208", "author_url": "", "post_date": "2019-02-27T12:29:40.330000", "content": "

Thank you!

", "votes": 0, "replies": [] }, { "id": 529120, "author_name": "just go", "author_url": "", "post_date": "2019-05-09T08:22:06.793000", "content": "

thank you for sharing a awesome solution!\nbut i have a question about adversarial validation in part of features selection .\nIs that because the feature cause overfitting or something what?\nhope your reply

", "votes": 0, "replies": [] } ] }, { "id": 529125, "author_name": "just go", "author_url": "", "post_date": "2019-05-09T08:28:06.743000", "content": "

and one more question, you select features from different models by the fixed number or ratio? and you select the feature that preform well at all of models or one of them?\nthank you! :)

", "votes": 0, "replies": [] }, { "id": 482606, "author_name": "choco", "author_url": "", "post_date": "2019-03-03T10:59:51.040000", "content": "

Hello. Congratulations !!! I have questions. Ratio every authmonthpurchase_amount-13,…auth month purchase amount+2 to previous with interval 2,4,6 <- what's mean of interval 2,4,6 ??? You did authmonthpurchase_amount-11/authmonthpurchase_amount-13 , authmonthpurchase_amount-9/authmonthpurchase_amount-13???

", "votes": 0, "replies": [ { "id": 484955, "author_name": "YuryBolkonsky", "author_url": "", "post_date": "2019-03-06T17:29:35.690000", "content": "

Yes, you are right

", "votes": 0, "replies": [] } ] }, { "id": 482407, "author_name": "Ning Jia", "author_url": "", "post_date": "2019-03-02T23:37:34.517000", "content": "

Congratulations!\n\"5 . Predicting month lag 3 and 4 and calculating ratio for them\"
\nWhat's that mean? There is no month lag 3 and 4 in the data.

", "votes": 0, "replies": [] }, { "id": 480319, "author_name": "Nathan Dixon", "author_url": "", "post_date": "2019-02-28T03:37:04.183000", "content": "

Congratulations and thank you for sharing.

", "votes": 0, "replies": [] }, { "id": 480299, "author_name": "Anthony Chiu", "author_url": "", "post_date": "2019-02-28T02:41:20.257000", "content": "

Congrats on your first gold!

", "votes": 0, "replies": [] }, { "id": 479708, "author_name": "Mr Loke", "author_url": "", "post_date": "2019-02-27T11:02:42.563000", "content": "

My catboost didn't yield better results than my lgbm. What is the param that you have used?

", "votes": 0, "replies": [ { "id": 479766, "author_name": "YuryBolkonsky", "author_url": "", "post_date": "2019-02-27T12:09:33.933000", "content": "

It depends on what params you are using with lgbm. I used these params in one of my model. As you can see it's very simple. No hyperparams tuned

\n\n

params = {'iterations': 15000, \n 'learning_rate': 0.01, \n 'eval_metric': 'RMSE', \n 'metric_period': 200,\n 'l2_leaf_reg': 10,\n 'od_type':'Iter',\n 'od_wait': 1000,\n 'task_type': 'GPU',\n 'depth':8,\n 'random_seed': 0,\n 'use_best_model': True}

", "votes": 3, "replies": [] }, { "id": 479866, "author_name": "Mr Loke", "author_url": "", "post_date": "2019-02-27T14:11:53.613000", "content": "

Thanks for your reply. I have tried to use classification to predict outliers but it give the highest prob of less than 0.5 only. If i multiplied it by -33, my CV score and PB was horrible. I mean do you select top10% card_id from classification and then add it up with all card_id from your regression model? Mind to share on this part if it is not too much to ask?

", "votes": 0, "replies": [] } ] }, { "id": 479563, "author_name": "Jianfei Wang", "author_url": "", "post_date": "2019-02-27T08:46:33.533000", "content": "

Congrats!\nI try the Count Vectorizer for merchant_id, subsector_id.etc too, but failed to boost public lb. Besides i try to use NMF/SVD for card-maerchant-category matrix failed boost public lb too. So can you share your result about this? thank you very much!

", "votes": 0, "replies": [ { "id": 479683, "author_name": "YuryBolkonsky", "author_url": "", "post_date": "2019-02-27T10:39:57.577000", "content": "

I tried Count Vectorizer. PCA for merchantid (15 components), SVD for subsectorid, merchant_subcategory_id (10 components). So, i tried to reach cum explained_variance_ratio_ at least 0.8.

\n\n

So in some features sets i used Count Vectorizer matrix + top50 features, in other features + pca, svd. It helps my model to add CV a little in single model 0.0007.

", "votes": 0, "replies": [] }, { "id": 479769, "author_name": "Jianfei Wang", "author_url": "", "post_date": "2019-02-27T12:13:23.460000", "content": "

Yes， thank you, for me should help cv too, but don't see the boost in public lb and give up.

", "votes": 0, "replies": [] } ] }, { "id": 479552, "author_name": "Möbius", "author_url": "", "post_date": "2019-02-27T08:36:06.200000", "content": "

Congrats! ِPure ML solution and acting bravely!

", "votes": 0, "replies": [] }, { "id": 479473, "author_name": "Karthik Chowdary Tsaliki", "author_url": "", "post_date": "2019-02-27T07:10:57.030000", "content": "

Congratulations @bolkonsky Thanks for sharing the solution.

", "votes": 0, "replies": [] }, { "id": 2069654, "author_name": "Alec.Baghramian", "author_url": "", "post_date": "2022-12-19T08:23:05.740000", "content": "

Great Job!
\nThank you for sharing.

", "votes": 0, "replies": [] }, { "id": 480499, "author_name": "andie", "author_url": "", "post_date": "2019-02-28T08:49:13.003000", "content": "

great, thank you for sharing

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "479444": "Thanks to all competitors for this wonderfull day) I am very happy to get first Gold and become Kaggle Master)\n\n**PREPROCESSING and FE**\n\n 1. Merging historical data and new_data with authorized_flag=1\n 2. Count Vectorizer merchant id, subsector id, merchant category id + PCA, SVD\n 3. Grouping month lag data (auth month-13, auth month-12 and etc.) for count purchases, and sum adjusted purchase_amount, std, min, max\n 4. Ratio every auth_month_purchase_amount-13,...auth month purchase amount+2 to previous with interval 2,4,6\n 5. Predicting month lag 3 and 4 and calculating ratio for them\n 6. Simple Rolling mean and SimpleExpSmoothing over auth purchase amount-13, ...auth purchase amount+4 over month lags\n 7. Calculating avg, min, max purchase time elapsed periods beetween\n transactions\n\n**FEATURE SELECTION**\n\nFinally, i created almost 6500 features and created different features sets using\n\n 1. Boruta (it took almost 8 hours to select 500 best features)\n 2. Selecting best features based on Catboost, LGBM, XGB\n 3. Deleting features based on Adversarial validation\n\n**Best Single Model and Ensembling**\n\nMy best single model is Catboost over 5 stratified folds CV 3.645 std 0.012 LB 3.671. Also tried DeepFM but CV 3.656 std 0.016 and LB 3.682. Other models, including LGBM sometimes got even better CV score, but were worse on LB. \n\n**Finally, i collected 47 models with outliers, and 35 models without them**. For combining submissions, also created classification model for outliers ROC 0.907 F1 0.1025\n\n 1. Stacking models with outliers CV 3.637 std 0.013 LB 3.669\n 2. Stacking models without outliers CV 1.544 std 0.011\n 3. Combining models, replacing 10000 values\n\n**Thanks for sharing ideas end experience**\n\n[@raddar][1] Post about [real meaning of target ][2] and [@peterhurford][3] for this [feature selection post][4]\n\n\n [1]: https://www.kaggle.com/raddar\n [2]: https://www.kaggle.com/raddar/target-true-meaning-revealed/\n [3]: https://www.kaggle.com/peterhurford\n [4]: https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73937", "739465": "@bolkonsky, congratulations! Thanks for sharing your approaches and magic! I added your post to my collection of the best kernels and posts of winners of Kaggle Prize Competitions: [Data Science for tabular data: Advanced Techniques] (https://www.kaggle.com/vbmokin/data-science-for-tabular-data-advanced-techniques) ", "479696": "Good job!!!\nYour feature selection based on adversarial validation is new to me, I'll sure try it in the future!!\nThank you for sharing", "479469": "Congrats! \nClean solution! Thank you for sharing.\nPlease let me ask a question about Feature Selection.\n> Deleting features based on Adversarial validation \n \nDoes this mean that you excluded features that are useful for classifying Train and Test?", "529125": "and one more question, you select features from different models by the fixed number or ratio? and you select the feature that preform well at all of models or one of them?\nthank you! :)", "482606": "Hello. Congratulations !!! I have questions. `Ratio every authmonthpurchase_amount-13,…auth month purchase amount+2 to previous with interval 2,4,6` <- what's mean of interval 2,4,6 ??? You did `authmonthpurchase_amount-11/authmonthpurchase_amount-13` , `authmonthpurchase_amount-9/authmonthpurchase_amount-13`???", "482407": "Congratulations!\n\"5 . Predicting month lag 3 and 4 and calculating ratio for them\" \nWhat's that mean? There is no month lag 3 and 4 in the data.", "480319": "Congratulations and thank you for sharing.", "480299": "Congrats on your first gold!", "479708": "My catboost didn't yield better results than my lgbm. What is the param that you have used?", "479563": "Congrats!\nI try the Count Vectorizer for merchant_id, subsector_id.etc too, but failed to boost public lb. Besides i try to use NMF/SVD for card-maerchant-category matrix failed boost public lb too. So can you share your result about this? thank you very much!", "479552": "Congrats! ِPure ML solution and acting bravely! ", "479473": "Congratulations @bolkonsky Thanks for sharing the solution.", "2069654": "Great Job!\nThank you for sharing.", "480499": "great, thank you for sharing" }, "complete": true, "source": "existing_discussion", "competition_id": 10445, "competition_slug": "elo-merchant-category-recommendation", "team_id": 2638058, "solution_url": "/competitions/elo-merchant-category-recommendation/writeups/ods-ai-yurybolkonskiy-10th-place-solution" }