{
  "id": 11342,
  "title": "Is the test class distribution the same as in training?",
  "url": "/competitions/datasciencebowl/discussion/11342",
  "author_name": "",
  "post_date": "2014-12-24T10:44:01.390Z",
  "votes": null,
  "comment_count": 2,
  "views": 1280,
  "content": "<p>Hi all,</p>\n<p>Just wondering if I might be missing out on something trivial. It seems like the test set behaves very differently than my internally kept validation set. It's making the difference between a ~1.5 submission and a ~2.8 actual score.</p>\n<p>For such a large dataset with samples chosen at random, this doesn't seem reasonable. But I'm pretty sure I don't have a bug or some sort of internal leak.</p>\n<p>Did anyone else get an indication that class distribution&nbsp;in the test set might be significantly different than the ones in the public training set?&nbsp;</p>\n<p>Best,</p>\n<p>Uri</p>",
  "messages": [
    {
      "id": "61819",
      "postDate": "12/24/2014 10:44:01",
      "content": "<p>Hi all,</p>\n<p>Just wondering if I might be missing out on something trivial. It seems like the test set behaves very differently than my internally kept validation set. It's making the difference between a ~1.5 submission and a ~2.8 actual score.</p>\n<p>For such a large dataset with samples chosen at random, this doesn't seem reasonable. But I'm pretty sure I don't have a bug or some sort of internal leak.</p>\n<p>Did anyone else get an indication that class distribution&nbsp;in the test set might be significantly different than the ones in the public training set?&nbsp;</p>\n<p>Best,</p>\n<p>Uri</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "61820",
      "postDate": "12/24/2014 10:48:12",
      "content": "<p>Thats a huge difference.</p>\n<p>I got 1.5 valid (20% random) and 1.75 leaderboard&nbsp;logloss.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "65235",
      "postDate": "03/02/2015 13:20:50",
      "content": "<p>Hi Uri, how and when did you separate your own test sample?</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 61820,
      "author_name": "mjahrer",
      "author_url": "",
      "post_date": "12/24/2014 10:48:12",
      "content": "<p>Thats a huge difference.</p>\n<p>I got 1.5 valid (20% random) and 1.75 leaderboard&nbsp;logloss.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 65235,
      "author_name": "atreo1",
      "author_url": "",
      "post_date": "03/02/2015 13:20:50",
      "content": "<p>Hi Uri, how and when did you separate your own test sample?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "61819": "",
    "61820": "",
    "65235": ""
  },
  "source": "meta"
}