{
  "id": 73571,
  "title": "How do we deal with the 2207 train values with target < -30?",
  "url": "/competitions/elo-merchant-category-recommendation/discussion/73571",
  "author_name": "",
  "post_date": "2018-12-04T05:32:09.926978800Z",
  "votes": 5,
  "comment_count": 9,
  "views": 0,
  "content": "<p>Has anyone managed to come up with an idea for this yet? The minimum value I'm predicting for my test set is around -15, so these particular rows in the training set could just potentially be outliers which the Elo people encoded with a certain unrealistically low value? </p>",
  "messages": [
    {
      "id": "432622",
      "postDate": "12/04/2018 05:32:09",
      "content": "<p>Has anyone managed to come up with an idea for this yet? The minimum value I'm predicting for my test set is around -15, so these particular rows in the training set could just potentially be outliers which the Elo people encoded with a certain unrealistically low value? </p>",
      "rawMarkdown": "Has anyone managed to come up with an idea for this yet? The minimum value I'm predicting for my test set is around -15, so these particular rows in the training set could just potentially be outliers which the Elo people encoded with a certain unrealistically low value?",
      "votes": null
    },
    {
      "id": "432703",
      "postDate": "12/04/2018 08:16:38",
      "content": "<p>The entire dataset is supposed to be simulated, so your explanation is quite probable.</p>",
      "rawMarkdown": "The entire dataset is supposed to be simulated, so your explanation is quite probable.",
      "votes": null
    },
    {
      "id": "434308",
      "postDate": "12/06/2018 07:19:05",
      "content": "<p>Since most of the rmse score is apparently due to these outliers I tried to classify them first and use that information for the regression, but not working yet.l</p>",
      "rawMarkdown": "Since most of the rmse score is apparently due to these outliers I tried to classify them first and use that information for the regression, but not working yet.l",
      "votes": null
    },
    {
      "id": "434406",
      "postDate": "12/06/2018 10:53:00",
      "content": "<p>Predicting the outliers seems very important in my model.</p>",
      "rawMarkdown": "Predicting the outliers seems very important in my model.",
      "votes": null
    },
    {
      "id": "435978",
      "postDate": "12/09/2018 07:33:44",
      "content": "<p>I've found that -33.21928095 (all scores &lt; 30 are equal to that value) is a -10*log2(10), so it looks like some synthetical value. \nIt may be a result of some value overflow or it was added as a synthetic noise to train data?</p>",
      "rawMarkdown": "I've found that -33.21928095 (all scores &lt; 30 are equal to that value) is a -10*log2(10), so it looks like some synthetical value. \nIt may be a result of some value overflow or it was added as a synthetic noise to train data?",
      "votes": null
    },
    {
      "id": "435980",
      "postDate": "12/09/2018 07:44:56",
      "content": "<p>My guess would be the latter, but in that case, why give it such a very formulaic value? </p>",
      "rawMarkdown": "My guess would be the latter, but in that case, why give it such a very formulaic value?",
      "votes": null
    },
    {
      "id": "445951",
      "postDate": "12/27/2018 09:05:18",
      "content": "<p>has anyone tried removing the outliers from data and then train it</p>",
      "rawMarkdown": "has anyone tried removing the outliers from data and then train it",
      "votes": null
    },
    {
      "id": "446188",
      "postDate": "12/27/2018 17:06:50",
      "content": "<p>At this point I'm currently focusing all my efforts in the Outliers Classification, I tried so many feature combinations with the Regression part and none of them seem to have effect improving RMSE, I even tried trend features over time.</p>",
      "rawMarkdown": "At this point I'm currently focusing all my efforts in the Outliers Classification, I tried so many feature combinations with the Regression part and none of them seem to have effect improving RMSE, I even tried trend features over time.",
      "votes": null
    },
    {
      "id": "446470",
      "postDate": "12/28/2018 06:01:04",
      "content": "<p>Nice observation!</p>",
      "rawMarkdown": "Nice observation!",
      "votes": null
    },
    {
      "id": "466858",
      "postDate": "02/06/2019 04:50:09",
      "content": "<p>spot on! Thx, I was wondering why that number!</p>",
      "rawMarkdown": "spot on! Thx, I was wondering why that number!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 432703,
      "author_name": "konradb",
      "author_url": "",
      "post_date": "12/04/2018 08:16:38",
      "content": "<p>The entire dataset is supposed to be simulated, so your explanation is quite probable.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 434308,
      "author_name": "guntherthepenguin",
      "author_url": "",
      "post_date": "12/06/2018 07:19:05",
      "content": "<p>Since most of the rmse score is apparently due to these outliers I tried to classify them first and use that information for the regression, but not working yet.l</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 434406,
      "author_name": "jfr311",
      "author_url": "",
      "post_date": "12/06/2018 10:53:00",
      "content": "<p>Predicting the outliers seems very important in my model.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 435978,
      "author_name": "andreylat",
      "author_url": "",
      "post_date": "12/09/2018 07:33:44",
      "content": "<p>I've found that -33.21928095 (all scores &lt; 30 are equal to that value) is a -10*log2(10), so it looks like some synthetical value. \nIt may be a result of some value overflow or it was added as a synthetic noise to train data?</p>",
      "votes": null,
      "replies": [
        {
          "id": 435980,
          "author_name": "delayedkarma",
          "author_url": "",
          "post_date": "12/09/2018 07:44:56",
          "content": "<p>My guess would be the latter, but in that case, why give it such a very formulaic value? </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 446470,
          "author_name": "salamatburj",
          "author_url": "",
          "post_date": "12/28/2018 06:01:04",
          "content": "<p>Nice observation!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 466858,
          "author_name": "liwenhuang",
          "author_url": "",
          "post_date": "02/06/2019 04:50:09",
          "content": "<p>spot on! Thx, I was wondering why that number!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 445951,
      "author_name": "abhinav97dutt",
      "author_url": "",
      "post_date": "12/27/2018 09:05:18",
      "content": "<p>has anyone tried removing the outliers from data and then train it</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 446188,
      "author_name": "cv13j0",
      "author_url": "",
      "post_date": "12/27/2018 17:06:50",
      "content": "<p>At this point I'm currently focusing all my efforts in the Outliers Classification, I tried so many feature combinations with the Regression part and none of them seem to have effect improving RMSE, I even tried trend features over time.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "432622": "Has anyone managed to come up with an idea for this yet? The minimum value I'm predicting for my test set is around -15, so these particular rows in the training set could just potentially be outliers which the Elo people encoded with a certain unrealistically low value?",
    "432703": "The entire dataset is supposed to be simulated, so your explanation is quite probable.",
    "434308": "Since most of the rmse score is apparently due to these outliers I tried to classify them first and use that information for the regression, but not working yet.l",
    "434406": "Predicting the outliers seems very important in my model.",
    "435978": "I've found that -33.21928095 (all scores &lt; 30 are equal to that value) is a -10*log2(10), so it looks like some synthetical value. \nIt may be a result of some value overflow or it was added as a synthetic noise to train data?",
    "435980": "My guess would be the latter, but in that case, why give it such a very formulaic value?",
    "445951": "has anyone tried removing the outliers from data and then train it",
    "446188": "At this point I'm currently focusing all my efforts in the Outliers Classification, I tried so many feature combinations with the Regression part and none of them seem to have effect improving RMSE, I even tried trend features over time.",
    "446470": "Nice observation!",
    "466858": "spot on! Thx, I was wondering why that number!"
  },
  "source": "meta"
}