{
  "id": 6231,
  "title": "Is it okay predict only tags from the train set?",
  "url": "/competitions/facebook-recruiting-iii-keyword-extraction/discussion/6231",
  "author_name": "",
  "post_date": "2013-11-05T04:54:06.037Z",
  "votes": null,
  "comment_count": 5,
  "views": 2398,
  "content": "<p>I guess there a two ways to approach this problem.</p>\n<ol><ol>\n<li><span style=\"line-height: 1.4\">Use the list of tags in the trainset and rank them for each question in the testset.</span></li>\n<li><span style=\"line-height: 1.4\">Extract the tags from the text data itself.</span></li>\n</ol></ol>\n<p><span style=\"line-height: 1.4\">From the posts of the admins here I get the idea that the first approach is&nbsp;</span>preferred (because it would generalize better to new data). Is that the case?</p>",
  "messages": [
    {
      "id": "33232",
      "postDate": "11/05/2013 04:54:06",
      "content": "<p>I guess there a two ways to approach this problem.</p>\n<ol><ol>\n<li><span style=\"line-height: 1.4\">Use the list of tags in the trainset and rank them for each question in the testset.</span></li>\n<li><span style=\"line-height: 1.4\">Extract the tags from the text data itself.</span></li>\n</ol></ol>\n<p><span style=\"line-height: 1.4\">From the posts of the admins here I get the idea that the first approach is&nbsp;</span>preferred (because it would generalize better to new data). Is that the case?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "33271",
      "postDate": "11/05/2013 15:04:56",
      "content": "<p>I guess the best way would be using both approaches, but my current solution only uses the first one. I have not found yet a good way to deal with the second one.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "33342",
      "postDate": "11/07/2013 01:35:45",
      "content": "<p>If you keep in mind that only users with higher rep can create new tags, then the best approach is to stick with only defined tags. &nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "33343",
      "postDate": "11/07/2013 03:41:03",
      "content": "<p>[quote=Alessandro Sena;33342]</p>\n<p>If you keep in mind that only users with higher rep can create new tags, then the best approach is to stick with only defined tags. &nbsp;</p>\n<p>[/quote]</p>\n<p>&nbsp;</p>\n<p>Alessandro: True. But you're still making a big assumption that all existing tags are defined in the training set.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "33694",
      "postDate": "11/14/2013 06:40:23",
      "content": "<p>I think you probably want to do a mix of both. So you might be able to &quot;boost&quot; the probability of a keyword you have extracted from the text if it is a keyword in the training set.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "33752",
      "postDate": "11/15/2013 15:26:36",
      "content": "<p>I'm currently working on the first assumption.</p>\n<p>Let's consider an email with a question about Google Analytics. If you don't know what Analytics is (and your learning machine surely doesn't) how can the model differentiate between the English term and the Google service? I don't think we're supposed to create a model that can understand human language with this level of details.</p>\n<p>In any case, I'm looking to see whether someone can get results with the second approach</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 33271,
      "author_name": "emiliogortizg",
      "author_url": "",
      "post_date": "11/05/2013 15:04:56",
      "content": "<p>I guess the best way would be using both approaches, but my current solution only uses the first one. I have not found yet a good way to deal with the second one.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 33342,
      "author_name": "alessandrosena",
      "author_url": "",
      "post_date": "11/07/2013 01:35:45",
      "content": "<p>If you keep in mind that only users with higher rep can create new tags, then the best approach is to stick with only defined tags. &nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 33343,
      "author_name": "rohits",
      "author_url": "",
      "post_date": "11/07/2013 03:41:03",
      "content": "<p>[quote=Alessandro Sena;33342]</p>\n<p>If you keep in mind that only users with higher rep can create new tags, then the best approach is to stick with only defined tags. &nbsp;</p>\n<p>[/quote]</p>\n<p>&nbsp;</p>\n<p>Alessandro: True. But you're still making a big assumption that all existing tags are defined in the training set.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 33694,
      "author_name": "nikhilagarwal1",
      "author_url": "",
      "post_date": "11/14/2013 06:40:23",
      "content": "<p>I think you probably want to do a mix of both. So you might be able to &quot;boost&quot; the probability of a keyword you have extracted from the text if it is a keyword in the training set.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 33752,
      "author_name": "sergiotorassa",
      "author_url": "",
      "post_date": "11/15/2013 15:26:36",
      "content": "<p>I'm currently working on the first assumption.</p>\n<p>Let's consider an email with a question about Google Analytics. If you don't know what Analytics is (and your learning machine surely doesn't) how can the model differentiate between the English term and the Google service? I don't think we're supposed to create a model that can understand human language with this level of details.</p>\n<p>In any case, I'm looking to see whether someone can get results with the second approach</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "33232": "",
    "33271": "",
    "33342": "",
    "33343": "",
    "33694": "",
    "33752": ""
  },
  "source": "meta"
}