{
  "id": 5595,
  "title": "Train and test data from the same set of SE sites?",
  "url": "/competitions/facebook-recruiting-iii-keyword-extraction/discussion/5595",
  "author_name": "",
  "post_date": "2013-08-31T04:29:59.387Z",
  "votes": null,
  "comment_count": 6,
  "views": 2371,
  "content": "<p>Can we assume that the train and test data comes from the same set of StackExchange sites? Or maybe more precise: does a significant number of test examples come from StackExchange sites that didn't contribute any examples to the training set?</p>\n<p>&nbsp;</p>\n<p>Simplified example (just in case I can't explain what I mean clearly):&nbsp;</p>\n<p>Is it possible, that training data consists of questions from {StackOverflow, Chess, AskUbuntu, Bicycles}, and the test data contains questions from&nbsp;{StackOverflow, Chess, Homebrewing}?</p>\n<p>&nbsp;</p>\n<p>Thanks!</p>",
  "messages": [
    {
      "id": "29824",
      "postDate": "08/31/2013 04:29:59",
      "content": "<p>Can we assume that the train and test data comes from the same set of StackExchange sites? Or maybe more precise: does a significant number of test examples come from StackExchange sites that didn't contribute any examples to the training set?</p>\n<p>&nbsp;</p>\n<p>Simplified example (just in case I can't explain what I mean clearly):&nbsp;</p>\n<p>Is it possible, that training data consists of questions from {StackOverflow, Chess, AskUbuntu, Bicycles}, and the test data contains questions from&nbsp;{StackOverflow, Chess, Homebrewing}?</p>\n<p>&nbsp;</p>\n<p>Thanks!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "30061",
      "postDate": "09/03/2013 15:06:45",
      "content": "<p>They come from the same sites.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "30462",
      "postDate": "09/08/2013 01:17:12",
      "content": "<p>And with the same frequency from each site?</p>\n<p>(Ie, if in<em> training</em> 30% are from Chess and 70% from AskUbuntu then in<em> test ~</em>30% will be from Chess and ~70% from AskUbuntu?)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "30540",
      "postDate": "09/09/2013 16:45:28",
      "content": "<p>We will not be revealing the mix of questions.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "30580",
      "postDate": "09/09/2013 18:53:05",
      "content": "<p>Wow now, William... You start sounding like Mr Spock himself!</p>\n<p>No offense meant, of course - I really like Mr Spock and his style, and I think you use it in a most appropriate way here... :-)</p>\n<p>Anyway, I think this (i.e. the mix of questions) is not what Sam above asks for; he is just asking if the (whatever...) mix is the same in both the training and test sets...</p>\n<p>Best</p>\n<p>des</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "30581",
      "postDate": "09/09/2013 19:00:33",
      "content": "<p>Right, so the answer is still a &quot;not telling&quot; for either interpretation of his question. &nbsp;If you're making a tag recommender, it should not be particularly sensitive to the mixture of the test set compared to the training set (in the limit of lots of questions, ignoring semi-supervised techniques).</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "30582",
      "postDate": "09/09/2013 19:05:22",
      "content": "<p>Agree - especially since we are talking about <em>really lots</em> of questions here.</p>\n<p>Thanx</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 30061,
      "author_name": "wcukierski",
      "author_url": "",
      "post_date": "09/03/2013 15:06:45",
      "content": "<p>They come from the same sites.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 30462,
      "author_name": "sambhagwat",
      "author_url": "",
      "post_date": "09/08/2013 01:17:12",
      "content": "<p>And with the same frequency from each site?</p>\n<p>(Ie, if in<em> training</em> 30% are from Chess and 70% from AskUbuntu then in<em> test ~</em>30% will be from Chess and ~70% from AskUbuntu?)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 30540,
      "author_name": "wcukierski",
      "author_url": "",
      "post_date": "09/09/2013 16:45:28",
      "content": "<p>We will not be revealing the mix of questions.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 30580,
      "author_name": "ctsats",
      "author_url": "",
      "post_date": "09/09/2013 18:53:05",
      "content": "<p>Wow now, William... You start sounding like Mr Spock himself!</p>\n<p>No offense meant, of course - I really like Mr Spock and his style, and I think you use it in a most appropriate way here... :-)</p>\n<p>Anyway, I think this (i.e. the mix of questions) is not what Sam above asks for; he is just asking if the (whatever...) mix is the same in both the training and test sets...</p>\n<p>Best</p>\n<p>des</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 30581,
      "author_name": "wcukierski",
      "author_url": "",
      "post_date": "09/09/2013 19:00:33",
      "content": "<p>Right, so the answer is still a &quot;not telling&quot; for either interpretation of his question. &nbsp;If you're making a tag recommender, it should not be particularly sensitive to the mixture of the test set compared to the training set (in the limit of lots of questions, ignoring semi-supervised techniques).</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 30582,
      "author_name": "ctsats",
      "author_url": "",
      "post_date": "09/09/2013 19:05:22",
      "content": "<p>Agree - especially since we are talking about <em>really lots</em> of questions here.</p>\n<p>Thanx</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "29824": "",
    "30061": "",
    "30462": "",
    "30540": "",
    "30580": "",
    "30581": "",
    "30582": ""
  },
  "source": "meta"
}