{
  "id": 5849,
  "title": "Clarification on \"no external data\" rule",
  "url": "/competitions/facebook-recruiting-iii-keyword-extraction/discussion/5849",
  "author_name": "",
  "post_date": "2013-09-23T04:08:05.313Z",
  "votes": 1,
  "comment_count": 2,
  "views": 1438,
  "content": "<p>Edit: Nevermind, I understand the restriction now.</p>\n<p>[quote=William Cukierski;29807]The no scraping and no external data rules are there <br>to prevent things like:</p>\n<ul>\n<li>Searching for the text, scraping the actual Stack page, and extracting the tags</li>\n</ul>\n<ul>\n<li>Searching the text and using external data/corpuses/pages to do word associationetc.</li>\n</ul>\n<p>To give a concrete answer, your model should take text and spit out tags. <br>If we pulled new data at the end of the competition, you should be able <br>to rerun your method and achieve similar performance without the need to<br> hand-code new assumptions.</p>\n<p>[/quote]</p>\n<p><a href=\"http://www.kaggle.com/c/facebook-recruiting-iii-keyword-extraction/forums/t/5592/restrictions/29807#post29807\">source</a></p>\n<p>[quote=William Cukierski;29810]</p>\n<p>Another question that comes up often with NLP problems is whether you <br>can use things like a parts of speech tagger. &nbsp;We consider these tools <br>more than data, and generally allow them. &nbsp;In other words, you shouldn't<br> have to reinvent the English language to solve an NLP problem. However,<br> it can be a slippery slope with these things, so it's best to post to <br>the forums if you think what you are using is more data than tool.</p>\n<p>[/quote]</p>\n<p><a href=\"http://www.kaggle.com/c/facebook-recruiting-iii-keyword-extraction/forums/t/5592/restrictions/29810#post29810\">source</a></p>\n<p>&nbsp;</p>\n<p><del>I am thinking about using a part of speech tagger, which itself classifies words using &quot;external data.&quot; Specifically, the data comes from the <a href=\"http://nltk.org/nltk_data/\">NLTK</a>. Would this practice be excepted by the rules of the competition?</del></p>",
  "messages": [
    {
      "id": "31369",
      "postDate": "09/23/2013 04:08:05",
      "content": "<p>Edit: Nevermind, I understand the restriction now.</p>\n<p>[quote=William Cukierski;29807]The no scraping and no external data rules are there <br>to prevent things like:</p>\n<ul>\n<li>Searching for the text, scraping the actual Stack page, and extracting the tags</li>\n</ul>\n<ul>\n<li>Searching the text and using external data/corpuses/pages to do word associationetc.</li>\n</ul>\n<p>To give a concrete answer, your model should take text and spit out tags. <br>If we pulled new data at the end of the competition, you should be able <br>to rerun your method and achieve similar performance without the need to<br> hand-code new assumptions.</p>\n<p>[/quote]</p>\n<p><a href=\"http://www.kaggle.com/c/facebook-recruiting-iii-keyword-extraction/forums/t/5592/restrictions/29807#post29807\">source</a></p>\n<p>[quote=William Cukierski;29810]</p>\n<p>Another question that comes up often with NLP problems is whether you <br>can use things like a parts of speech tagger. &nbsp;We consider these tools <br>more than data, and generally allow them. &nbsp;In other words, you shouldn't<br> have to reinvent the English language to solve an NLP problem. However,<br> it can be a slippery slope with these things, so it's best to post to <br>the forums if you think what you are using is more data than tool.</p>\n<p>[/quote]</p>\n<p><a href=\"http://www.kaggle.com/c/facebook-recruiting-iii-keyword-extraction/forums/t/5592/restrictions/29810#post29810\">source</a></p>\n<p>&nbsp;</p>\n<p><del>I am thinking about using a part of speech tagger, which itself classifies words using &quot;external data.&quot; Specifically, the data comes from the <a href=\"http://nltk.org/nltk_data/\">NLTK</a>. Would this practice be excepted by the rules of the competition?</del></p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "31540",
      "postDate": "09/25/2013 15:08:51",
      "content": "<p>What about using a set of stop words?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "31544",
      "postDate": "09/25/2013 15:57:53",
      "content": "<p>sorry, found the answer. &nbsp;It appears to be yes.</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 31540,
      "author_name": "dopeytaylor",
      "author_url": "",
      "post_date": "09/25/2013 15:08:51",
      "content": "<p>What about using a set of stop words?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 31544,
      "author_name": "dopeytaylor",
      "author_url": "",
      "post_date": "09/25/2013 15:57:53",
      "content": "<p>sorry, found the answer. &nbsp;It appears to be yes.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "31369": "",
    "31540": "",
    "31544": ""
  },
  "source": "meta"
}