{
  "id": 8110,
  "title": "Can we use a word list?",
  "url": "/competitions/billion-word-imputation/discussion/8110",
  "author_name": "",
  "post_date": "2014-05-10T10:59:33.420Z",
  "votes": 1,
  "comment_count": 2,
  "views": 1657,
  "content": "<p>The rules state:</p>\n<p>&quot;You may only use the provided training data to train your model.&quot;</p>\n<p>However it would be very useful to be able to classify the words in the training data, (verb, noun, name, etc).&nbsp;</p>\n<p>So my question is: it allowed to use a word list or dictionary?</p>",
  "messages": [
    {
      "id": "44337",
      "postDate": "05/10/2014 10:59:33",
      "content": "<p>The rules state:</p>\n<p>&quot;You may only use the provided training data to train your model.&quot;</p>\n<p>However it would be very useful to be able to classify the words in the training data, (verb, noun, name, etc).&nbsp;</p>\n<p>So my question is: it allowed to use a word list or dictionary?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "44344",
      "postDate": "05/10/2014 14:19:50",
      "content": "<p>Obviously not an admin, but I remember this question popping up before, and the answer being:</p>\n<p>You do not have to re-invent natural language parsing: stopword-lists, POS tags, dictionary-based spelling correction. is ok.</p>\n<p>Unfortunately I can not find this post, so better to wait on a competition admin for confirmation. While&nbsp;we are on this subject:</p>\n<p>http://www.statmt.org/lm-benchmark/ has a file with output probabilities. Do I have to generate my own or can I use that one?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "44347",
      "postDate": "05/10/2014 17:30:22",
      "content": "<p>Both the file of probs that you reference and the much improved file that I've made available are only on the first ten parts of the heldout data.&nbsp;&nbsp; There are 50 parts in total, so these probs won't help you very much.</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 44344,
      "author_name": "triskelion",
      "author_url": "",
      "post_date": "05/10/2014 14:19:50",
      "content": "<p>Obviously not an admin, but I remember this question popping up before, and the answer being:</p>\n<p>You do not have to re-invent natural language parsing: stopword-lists, POS tags, dictionary-based spelling correction. is ok.</p>\n<p>Unfortunately I can not find this post, so better to wait on a competition admin for confirmation. While&nbsp;we are on this subject:</p>\n<p>http://www.statmt.org/lm-benchmark/ has a file with output probabilities. Do I have to generate my own or can I use that one?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 44347,
      "author_name": "tonyrobinson",
      "author_url": "",
      "post_date": "05/10/2014 17:30:22",
      "content": "<p>Both the file of probs that you reference and the much improved file that I've made available are only on the first ten parts of the heldout data.&nbsp;&nbsp; There are 50 parts in total, so these probs won't help you very much.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "44337": "",
    "44344": "",
    "44347": ""
  },
  "source": "meta"
}