{
  "id": 76950,
  "title": "Manual predictions...",
  "url": "/competitions/elo-merchant-category-recommendation/discussion/76950",
  "author_name": "",
  "post_date": "2019-01-08T07:38:57.423363Z",
  "votes": 39,
  "comment_count": 25,
  "views": 0,
  "content": "<p>As discussed <a href=\"https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73922\">here</a>, manually setting predictions for possible outliers has a great effect on LB score...</p>\n\n<p>My LB score was 3.681, using single LightGBM model with feature selection as well as post processing introduced by <a href=\"https://www.kaggle.com/waitingli/combining-your-model-with-a-model-without-outlier\">李胃疼疼疼</a>. Then, I set predictions for card_ids with 'outlier probability' greater than 0.5 to -33.21928095 just to see what happens. Setting manual predictions like this brought my LB score down to 3.674. Then, I changed the probability threshold to 0.45 and it lowered my LB even more to 3.671...</p>\n\n<p>Now, I don't think manual predictions like this won't do Elo any good. Also, private LB is another story and it all depends on how many outliers there will be in the private dataset...</p>\n\n<p>So what should we do?</p>",
  "messages": [
    {
      "id": "452102",
      "postDate": "01/08/2019 07:38:57",
      "content": "<p>As discussed <a href=\"https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73922\">here</a>, manually setting predictions for possible outliers has a great effect on LB score...</p>\n\n<p>My LB score was 3.681, using single LightGBM model with feature selection as well as post processing introduced by <a href=\"https://www.kaggle.com/waitingli/combining-your-model-with-a-model-without-outlier\">李胃疼疼疼</a>. Then, I set predictions for card_ids with 'outlier probability' greater than 0.5 to -33.21928095 just to see what happens. Setting manual predictions like this brought my LB score down to 3.674. Then, I changed the probability threshold to 0.45 and it lowered my LB even more to 3.671...</p>\n\n<p>Now, I don't think manual predictions like this won't do Elo any good. Also, private LB is another story and it all depends on how many outliers there will be in the private dataset...</p>\n\n<p>So what should we do?</p>",
      "rawMarkdown": "As discussed [here][1], manually setting predictions for possible outliers has a great effect on LB score...\n\nMy LB score was 3.681, using single LightGBM model with feature selection as well as post processing introduced by [李胃疼疼疼][2]. Then, I set predictions for card_ids with 'outlier probability' greater than 0.5 to -33.21928095 just to see what happens. Setting manual predictions like this brought my LB score down to 3.674. Then, I changed the probability threshold to 0.45 and it lowered my LB even more to 3.671...\n\nNow, I don't think manual predictions like this won't do Elo any good. Also, private LB is another story and it all depends on how many outliers there will be in the private dataset...\n\nSo what should we do?\n\n  [1]: https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73922\n  [2]: https://www.kaggle.com/waitingli/combining-your-model-with-a-model-without-outlier",
      "votes": null
    },
    {
      "id": "452142",
      "postDate": "01/08/2019 08:56:13",
      "content": "<p>I think that outliers classification performance should be improved.\nI made a classifier model with few features, and I got an AUC of 0.86945.\nAnd I tried to apply post processing like you, but the leader board showed 14.669.\nAlso, using -15 instead of -33.21928095 got a score of 7.125.</p>",
      "rawMarkdown": "I think that outliers classification performance should be improved.\nI made a classifier model with few features, and I got an AUC of 0.86945.\nAnd I tried to apply post processing like you, but the leader board showed 14.669.\nAlso, using -15 instead of -33.21928095 got a score of 7.125.",
      "votes": null
    },
    {
      "id": "452161",
      "postDate": "01/08/2019 09:50:12",
      "content": "<p>I agree. I used logloss as a metric so I'm not sure about AUC but I think you are assigning way too many outlier values. Too many non-outliers being predicted as outliers, thus a better score with -15 than with -33.21928095.</p>",
      "rawMarkdown": "I agree. I used logloss as a metric so I'm not sure about AUC but I think you are assigning way too many outlier values. Too many non-outliers being predicted as outliers, thus a better score with -15 than with -33.21928095.",
      "votes": null
    },
    {
      "id": "452383",
      "postDate": "01/08/2019 16:45:25",
      "content": "<p>Thank you for sharing !</p>\n\n<p>Please let me question if you do not mind.\nHow much card_ids are greater than 0.5 when you set the threshold to 0.5?\nAnd how much in the case of 0.45?</p>\n\n<p>In my case, the biggest outlier probability was 0.368.\nTherefore, when I set  the threshold to 0.5, there was no data exceeding the threshold.</p>",
      "rawMarkdown": "Thank you for sharing !\n\nPlease let me question if you do not mind.\nHow much card_ids are greater than 0.5 when you set the threshold to 0.5?\nAnd how much in the case of 0.45?\n\nIn my case, the biggest outlier probability was 0.368.\nTherefore, when I set  the threshold to 0.5, there was no data exceeding the threshold.",
      "votes": null
    },
    {
      "id": "452596",
      "postDate": "01/08/2019 23:49:54",
      "content": "<p>In my case, the number of card_ids with 'outlier probability' greater than 0.5 is around 20 and with the threshold 0.45, it's around 50. Hope this helps.</p>",
      "rawMarkdown": "In my case, the number of card_ids with 'outlier probability' greater than 0.5 is around 20 and with the threshold 0.45, it's around 50. Hope this helps.",
      "votes": null
    },
    {
      "id": "452692",
      "postDate": "01/09/2019 03:50:19",
      "content": "<p>How much normal value?\nthanks</p>",
      "rawMarkdown": "How much normal value?\nthanks",
      "votes": null
    },
    {
      "id": "452753",
      "postDate": "01/09/2019 05:53:09",
      "content": "<p>Predictions for the rest of the card_ids are made by my original model.</p>",
      "rawMarkdown": "Predictions for the rest of the card_ids are made by my original model.",
      "votes": null
    },
    {
      "id": "452767",
      "postDate": "01/09/2019 06:10:26",
      "content": "<p>Sorry, I didn't communicate my question clearly. \nI mean, how much outlier and normal value with 'outlier probability' greater than 0.5 or 0.45  in your valid set?</p>",
      "rawMarkdown": "Sorry, I didn't communicate my question clearly. \nI mean, how much outlier and normal value with 'outlier probability' greater than 0.5 or 0.45  in your valid set?",
      "votes": null
    },
    {
      "id": "452778",
      "postDate": "01/09/2019 06:26:01",
      "content": "<p>I'd say around 60-70 % of the card_ids with out-of-the-fold 'outlier probability' greater than 0.5 are actually outliers. Probably a bit lower with the threshold 0.45.</p>",
      "rawMarkdown": "I'd say around 60-70 % of the card_ids with out-of-the-fold 'outlier probability' greater than 0.5 are actually outliers. Probably a bit lower with the threshold 0.45.",
      "votes": null
    },
    {
      "id": "452792",
      "postDate": "01/09/2019 06:50:07",
      "content": "<p>Thanks,  In my valid set, just around 50% sample with 'outlier probability' greater than 0.45. I would not like to set it to -33.21928095 manually which will increas the risk of over-fitting.</p>",
      "rawMarkdown": "Thanks,  In my valid set, just around 50% sample with 'outlier probability' greater than 0.45. I would not like to set it to -33.21928095 manually which will increas the risk of over-fitting.",
      "votes": null
    },
    {
      "id": "452838",
      "postDate": "01/09/2019 08:32:02",
      "content": "<p>Yeah we need to be careful with overfitting to the public LB. </p>",
      "rawMarkdown": "Yeah we need to be careful with overfitting to the public LB.",
      "votes": null
    },
    {
      "id": "452935",
      "postDate": "01/09/2019 11:28:58",
      "content": "<p>it's hard to get a reasonable and not-overfitting threshold!</p>",
      "rawMarkdown": "it's hard to get a reasonable and not-overfitting threshold!",
      "votes": null
    },
    {
      "id": "452965",
      "postDate": "01/09/2019 12:20:37",
      "content": "<p>Interesting. I've never get the outlier probability for the test set more than 0.35. Thus, it's difficult to set the margin for outlier probability.  Manual predictions lead to overfitting.</p>",
      "rawMarkdown": "Interesting. I've never get the outlier probability for the test set more than 0.35. Thus, it's difficult to set the margin for outlier probability.  Manual predictions lead to overfitting.",
      "votes": null
    },
    {
      "id": "453808",
      "postDate": "01/10/2019 20:24:43",
      "content": "<p>So the top positions in the LB are based on this approach?</p>",
      "rawMarkdown": "So the top positions in the LB are based on this approach?",
      "votes": null
    },
    {
      "id": "454012",
      "postDate": "01/11/2019 02:54:13",
      "content": "<p>I'm pretty sure they've used other FE/Post Processing ideas to be on the top of the LB but who knows.</p>",
      "rawMarkdown": "I'm pretty sure they've used other FE/Post Processing ideas to be on the top of the LB but who knows.",
      "votes": null
    },
    {
      "id": "454098",
      "postDate": "01/11/2019 06:06:41",
      "content": "<p>I think that misclassify one outlier may cause great public leaderboard change, and so does private leaderboard due to square error metrice, so it would be a great risk using manual prediction (assign entity to -33.2) for final submission.</p>",
      "rawMarkdown": "I think that misclassify one outlier may cause great public leaderboard change, and so does private leaderboard due to square error metrice, so it would be a great risk using manual prediction (assign entity to -33.2) for final submission.",
      "votes": null
    },
    {
      "id": "454252",
      "postDate": "01/11/2019 10:07:48",
      "content": "<p>I am a novice.I think this method is high risk and high profit.The CV score of the model excluding outliers could easily reach about 1.5.If you can classify the vast majority of outliers,you've won the game,but in view of the sample imbalance, it's really hard.</p>",
      "rawMarkdown": "I am a novice.I think this method is high risk and high profit.The CV score of the model excluding outliers could easily reach about 1.5.If you can classify the vast majority of outliers,you've won the game,but in view of the sample imbalance, it's really hard.",
      "votes": null
    },
    {
      "id": "454961",
      "postDate": "01/12/2019 16:32:25",
      "content": "<p>Auc metric is the proportion of correctly ordered pairs of ones and zeros. But, if there are 1.3% emissions in the test sample, the metric should not be very indicative. It is much better to look at the recall at the top 10%-score on the test sample, which was predicted by the algorithm.</p>",
      "rawMarkdown": "Auc metric is the proportion of correctly ordered pairs of ones and zeros. But, if there are 1.3% emissions in the test sample, the metric should not be very indicative. It is much better to look at the recall at the top 10%-score on the test sample, which was predicted by the algorithm.",
      "votes": null
    },
    {
      "id": "455122",
      "postDate": "01/13/2019 02:55:21",
      "content": "<p>Yes!</p>",
      "rawMarkdown": "Yes!",
      "votes": null
    },
    {
      "id": "457291",
      "postDate": "01/17/2019 07:31:54",
      "content": "<p>I see many kagglers try to solve this problem by creating a binary classifier to judge if it is a outlier. </p>",
      "rawMarkdown": "I see many kagglers try to solve this problem by creating a binary classifier to judge if it is a outlier.",
      "votes": null
    },
    {
      "id": "457432",
      "postDate": "01/17/2019 12:10:02",
      "content": "<p>When i try to create a binary classifier for outlier , it seems all fold log_loss become 0..Why ? any idea ?</p>",
      "rawMarkdown": "When i try to create a binary classifier for outlier , it seems all fold log_loss become 0..Why ? any idea ?",
      "votes": null
    },
    {
      "id": "457478",
      "postDate": "01/17/2019 14:17:59",
      "content": "<p>maybe you didn`t drop the 'target' in train set...data leakage...</p>",
      "rawMarkdown": "maybe you didn`t drop the 'target' in train set...data leakage...",
      "votes": null
    },
    {
      "id": "457808",
      "postDate": "01/18/2019 05:46:41",
      "content": "<p>Thanks ..It worked .. Just a small clarification ... target - is Loyalty score and Outliers - Is the actual response variable ( Rename to target in this case) for the binary classifier then how it creates a data leakage even if we run without deleting it..I might have made mistake but just a thought ..</p>",
      "rawMarkdown": "Thanks ..It worked .. Just a small clarification ... target - is Loyalty score and Outliers - Is the actual response variable ( Rename to target in this case) for the binary classifier then how it creates a data leakage even if we run without deleting it..I might have made mistake but just a thought ..",
      "votes": null
    },
    {
      "id": "459447",
      "postDate": "01/21/2019 18:46:34",
      "content": "<p><a href=\"/kailex\">@kailex</a>, do you think using 3 standard deviations as the cutoff margin will work?</p>",
      "rawMarkdown": "kailex, do you think using 3 standard deviations as the cutoff margin will work?",
      "votes": null
    },
    {
      "id": "459491",
      "postDate": "01/21/2019 21:33:37",
      "content": "<p>It's difficult to say. The only way is to try it yourself.</p>",
      "rawMarkdown": "It's difficult to say. The only way is to try it yourself.",
      "votes": null
    },
    {
      "id": "463428",
      "postDate": "01/30/2019 00:59:25",
      "content": "<p>I guess there is no way to know unless the data from private LB is known. So that's a bet. Single model is safer in this regard.</p>",
      "rawMarkdown": "I guess there is no way to know unless the data from private LB is known. So that's a bet. Single model is safer in this regard.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 452142,
      "author_name": "hatunina",
      "author_url": "",
      "post_date": "01/08/2019 08:56:13",
      "content": "<p>I think that outliers classification performance should be improved.\nI made a classifier model with few features, and I got an AUC of 0.86945.\nAnd I tried to apply post processing like you, but the leader board showed 14.669.\nAlso, using -15 instead of -33.21928095 got a score of 7.125.</p>",
      "votes": null,
      "replies": [
        {
          "id": 452161,
          "author_name": "ykskks",
          "author_url": "",
          "post_date": "01/08/2019 09:50:12",
          "content": "<p>I agree. I used logloss as a metric so I'm not sure about AUC but I think you are assigning way too many outlier values. Too many non-outliers being predicted as outliers, thus a better score with -15 than with -33.21928095.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 454961,
          "author_name": "evgenikasatkin",
          "author_url": "",
          "post_date": "01/12/2019 16:32:25",
          "content": "<p>Auc metric is the proportion of correctly ordered pairs of ones and zeros. But, if there are 1.3% emissions in the test sample, the metric should not be very indicative. It is much better to look at the recall at the top 10%-score on the test sample, which was predicted by the algorithm.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 452383,
      "author_name": "fuumin621",
      "author_url": "",
      "post_date": "01/08/2019 16:45:25",
      "content": "<p>Thank you for sharing !</p>\n\n<p>Please let me question if you do not mind.\nHow much card_ids are greater than 0.5 when you set the threshold to 0.5?\nAnd how much in the case of 0.45?</p>\n\n<p>In my case, the biggest outlier probability was 0.368.\nTherefore, when I set  the threshold to 0.5, there was no data exceeding the threshold.</p>",
      "votes": null,
      "replies": [
        {
          "id": 452596,
          "author_name": "ykskks",
          "author_url": "",
          "post_date": "01/08/2019 23:49:54",
          "content": "<p>In my case, the number of card_ids with 'outlier probability' greater than 0.5 is around 20 and with the threshold 0.45, it's around 50. Hope this helps.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 452692,
          "author_name": "daishu",
          "author_url": "",
          "post_date": "01/09/2019 03:50:19",
          "content": "<p>How much normal value?\nthanks</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 452753,
          "author_name": "ykskks",
          "author_url": "",
          "post_date": "01/09/2019 05:53:09",
          "content": "<p>Predictions for the rest of the card_ids are made by my original model.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 452767,
          "author_name": "daishu",
          "author_url": "",
          "post_date": "01/09/2019 06:10:26",
          "content": "<p>Sorry, I didn't communicate my question clearly. \nI mean, how much outlier and normal value with 'outlier probability' greater than 0.5 or 0.45  in your valid set?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 452778,
          "author_name": "ykskks",
          "author_url": "",
          "post_date": "01/09/2019 06:26:01",
          "content": "<p>I'd say around 60-70 % of the card_ids with out-of-the-fold 'outlier probability' greater than 0.5 are actually outliers. Probably a bit lower with the threshold 0.45.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 452792,
          "author_name": "daishu",
          "author_url": "",
          "post_date": "01/09/2019 06:50:07",
          "content": "<p>Thanks,  In my valid set, just around 50% sample with 'outlier probability' greater than 0.45. I would not like to set it to -33.21928095 manually which will increas the risk of over-fitting.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 452838,
          "author_name": "ykskks",
          "author_url": "",
          "post_date": "01/09/2019 08:32:02",
          "content": "<p>Yeah we need to be careful with overfitting to the public LB. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 452935,
          "author_name": "codlife",
          "author_url": "",
          "post_date": "01/09/2019 11:28:58",
          "content": "<p>it's hard to get a reasonable and not-overfitting threshold!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 452965,
      "author_name": "kailex",
      "author_url": "",
      "post_date": "01/09/2019 12:20:37",
      "content": "<p>Interesting. I've never get the outlier probability for the test set more than 0.35. Thus, it's difficult to set the margin for outlier probability.  Manual predictions lead to overfitting.</p>",
      "votes": null,
      "replies": [
        {
          "id": 459447,
          "author_name": "pukkinming",
          "author_url": "",
          "post_date": "01/21/2019 18:46:34",
          "content": "<p><a href=\"/kailex\">@kailex</a>, do you think using 3 standard deviations as the cutoff margin will work?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 459491,
          "author_name": "kailex",
          "author_url": "",
          "post_date": "01/21/2019 21:33:37",
          "content": "<p>It's difficult to say. The only way is to try it yourself.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 463428,
          "author_name": "pukkinming",
          "author_url": "",
          "post_date": "01/30/2019 00:59:25",
          "content": "<p>I guess there is no way to know unless the data from private LB is known. So that's a bet. Single model is safer in this regard.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 453808,
      "author_name": "deffro",
      "author_url": "",
      "post_date": "01/10/2019 20:24:43",
      "content": "<p>So the top positions in the LB are based on this approach?</p>",
      "votes": null,
      "replies": [
        {
          "id": 454012,
          "author_name": "ykskks",
          "author_url": "",
          "post_date": "01/11/2019 02:54:13",
          "content": "<p>I'm pretty sure they've used other FE/Post Processing ideas to be on the top of the LB but who knows.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 454098,
      "author_name": "marcuslin",
      "author_url": "",
      "post_date": "01/11/2019 06:06:41",
      "content": "<p>I think that misclassify one outlier may cause great public leaderboard change, and so does private leaderboard due to square error metrice, so it would be a great risk using manual prediction (assign entity to -33.2) for final submission.</p>",
      "votes": null,
      "replies": [
        {
          "id": 455122,
          "author_name": "codlife",
          "author_url": "",
          "post_date": "01/13/2019 02:55:21",
          "content": "<p>Yes!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 454252,
      "author_name": "fangwenjun",
      "author_url": "",
      "post_date": "01/11/2019 10:07:48",
      "content": "<p>I am a novice.I think this method is high risk and high profit.The CV score of the model excluding outliers could easily reach about 1.5.If you can classify the vast majority of outliers,you've won the game,but in view of the sample imbalance, it's really hard.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 457291,
      "author_name": "longyin2",
      "author_url": "",
      "post_date": "01/17/2019 07:31:54",
      "content": "<p>I see many kagglers try to solve this problem by creating a binary classifier to judge if it is a outlier. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 457432,
      "author_name": "leninsolver",
      "author_url": "",
      "post_date": "01/17/2019 12:10:02",
      "content": "<p>When i try to create a binary classifier for outlier , it seems all fold log_loss become 0..Why ? any idea ?</p>",
      "votes": null,
      "replies": [
        {
          "id": 457478,
          "author_name": "fangwenjun",
          "author_url": "",
          "post_date": "01/17/2019 14:17:59",
          "content": "<p>maybe you didn`t drop the 'target' in train set...data leakage...</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 457808,
          "author_name": "leninsolver",
          "author_url": "",
          "post_date": "01/18/2019 05:46:41",
          "content": "<p>Thanks ..It worked .. Just a small clarification ... target - is Loyalty score and Outliers - Is the actual response variable ( Rename to target in this case) for the binary classifier then how it creates a data leakage even if we run without deleting it..I might have made mistake but just a thought ..</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "452102": "As discussed [here][1], manually setting predictions for possible outliers has a great effect on LB score...\n\nMy LB score was 3.681, using single LightGBM model with feature selection as well as post processing introduced by [李胃疼疼疼][2]. Then, I set predictions for card_ids with 'outlier probability' greater than 0.5 to -33.21928095 just to see what happens. Setting manual predictions like this brought my LB score down to 3.674. Then, I changed the probability threshold to 0.45 and it lowered my LB even more to 3.671...\n\nNow, I don't think manual predictions like this won't do Elo any good. Also, private LB is another story and it all depends on how many outliers there will be in the private dataset...\n\nSo what should we do?\n\n  [1]: https://www.kaggle.com/c/elo-merchant-category-recommendation/discussion/73922\n  [2]: https://www.kaggle.com/waitingli/combining-your-model-with-a-model-without-outlier",
    "452142": "I think that outliers classification performance should be improved.\nI made a classifier model with few features, and I got an AUC of 0.86945.\nAnd I tried to apply post processing like you, but the leader board showed 14.669.\nAlso, using -15 instead of -33.21928095 got a score of 7.125.",
    "452161": "I agree. I used logloss as a metric so I'm not sure about AUC but I think you are assigning way too many outlier values. Too many non-outliers being predicted as outliers, thus a better score with -15 than with -33.21928095.",
    "452383": "Thank you for sharing !\n\nPlease let me question if you do not mind.\nHow much card_ids are greater than 0.5 when you set the threshold to 0.5?\nAnd how much in the case of 0.45?\n\nIn my case, the biggest outlier probability was 0.368.\nTherefore, when I set  the threshold to 0.5, there was no data exceeding the threshold.",
    "452596": "In my case, the number of card_ids with 'outlier probability' greater than 0.5 is around 20 and with the threshold 0.45, it's around 50. Hope this helps.",
    "452692": "How much normal value?\nthanks",
    "452753": "Predictions for the rest of the card_ids are made by my original model.",
    "452767": "Sorry, I didn't communicate my question clearly. \nI mean, how much outlier and normal value with 'outlier probability' greater than 0.5 or 0.45  in your valid set?",
    "452778": "I'd say around 60-70 % of the card_ids with out-of-the-fold 'outlier probability' greater than 0.5 are actually outliers. Probably a bit lower with the threshold 0.45.",
    "452792": "Thanks,  In my valid set, just around 50% sample with 'outlier probability' greater than 0.45. I would not like to set it to -33.21928095 manually which will increas the risk of over-fitting.",
    "452838": "Yeah we need to be careful with overfitting to the public LB.",
    "452935": "it's hard to get a reasonable and not-overfitting threshold!",
    "452965": "Interesting. I've never get the outlier probability for the test set more than 0.35. Thus, it's difficult to set the margin for outlier probability.  Manual predictions lead to overfitting.",
    "453808": "So the top positions in the LB are based on this approach?",
    "454012": "I'm pretty sure they've used other FE/Post Processing ideas to be on the top of the LB but who knows.",
    "454098": "I think that misclassify one outlier may cause great public leaderboard change, and so does private leaderboard due to square error metrice, so it would be a great risk using manual prediction (assign entity to -33.2) for final submission.",
    "454252": "I am a novice.I think this method is high risk and high profit.The CV score of the model excluding outliers could easily reach about 1.5.If you can classify the vast majority of outliers,you've won the game,but in view of the sample imbalance, it's really hard.",
    "454961": "Auc metric is the proportion of correctly ordered pairs of ones and zeros. But, if there are 1.3% emissions in the test sample, the metric should not be very indicative. It is much better to look at the recall at the top 10%-score on the test sample, which was predicted by the algorithm.",
    "455122": "Yes!",
    "457291": "I see many kagglers try to solve this problem by creating a binary classifier to judge if it is a outlier.",
    "457432": "When i try to create a binary classifier for outlier , it seems all fold log_loss become 0..Why ? any idea ?",
    "457478": "maybe you didn`t drop the 'target' in train set...data leakage...",
    "457808": "Thanks ..It worked .. Just a small clarification ... target - is Loyalty score and Outliers - Is the actual response variable ( Rename to target in this case) for the binary classifier then how it creates a data leakage even if we run without deleting it..I might have made mistake but just a thought ..",
    "459447": "kailex, do you think using 3 standard deviations as the cutoff margin will work?",
    "459491": "It's difficult to say. The only way is to try it yourself.",
    "463428": "I guess there is no way to know unless the data from private LB is known. So that's a bet. Single model is safer in this regard."
  },
  "source": "meta"
}