{
  "id": 6201,
  "title": "Encoding of the Training Dataset",
  "url": "/competitions/facebook-recruiting-iii-keyword-extraction/discussion/6201",
  "author_name": "",
  "post_date": "2013-11-02T06:38:49.257Z",
  "votes": null,
  "comment_count": 3,
  "views": 1557,
  "content": "<p>Hi,</p>\n<p>I am using Python 2.x for this project. And while reading the training data I am getting UnicodeError. Can you please let me know what is the encoding of the training file is?</p>\n<p>UnicodeEncodeError: 'ascii' codec can't encode character u'\\u201c' in position 215: ordinal not in range(128)</p>\n<p>Did anybody else is using Python run into this kind of problem. Any directions would be helpful.</p>\n<p>Thanks</p>\n<p>Deb</p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>",
  "messages": [
    {
      "id": "33097",
      "postDate": "11/02/2013 06:38:49",
      "content": "<p>Hi,</p>\n<p>I am using Python 2.x for this project. And while reading the training data I am getting UnicodeError. Can you please let me know what is the encoding of the training file is?</p>\n<p>UnicodeEncodeError: 'ascii' codec can't encode character u'\\u201c' in position 215: ordinal not in range(128)</p>\n<p>Did anybody else is using Python run into this kind of problem. Any directions would be helpful.</p>\n<p>Thanks</p>\n<p>Deb</p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "33107",
      "postDate": "11/02/2013 15:19:46",
      "content": "<p>Best to use <a href=\"StackOverflow.com\">StackOverflow.com</a> for these sorts of questions, you'll find the answer there. It helps to post the actual lines of code causing the issue.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "33308",
      "postDate": "11/06/2013 12:08:52",
      "content": "<p>Use the codecs module in python.</p>\n<p style=\"padding-left: 30px\">import codecs<br>f = codecs.open('unicode.rst', encoding='utf-8')<br>for line in f:<br> print repr(line)</p>\n<p style=\"padding-left: 30px\">&nbsp;</p>\n<p>Reference:&nbsp;http://docs.python.org/2/howto/unicode.html</p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "33489",
      "postDate": "11/10/2013 02:43:18",
      "content": "<p>For anyone else using Python 3.x, unicode handling changed. See this article: &nbsp;</p>\n<p>http://python-notes.curiousefficiency.org/en/latest/python3/text_file_processing.html#files-in-an-ascii-compatible-encoding-best-effort-is-acceptable</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 33107,
      "author_name": "smcinerney",
      "author_url": "",
      "post_date": "11/02/2013 15:19:46",
      "content": "<p>Best to use <a href=\"StackOverflow.com\">StackOverflow.com</a> for these sorts of questions, you'll find the answer there. It helps to post the actual lines of code causing the issue.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 33308,
      "author_name": "praveen97uma",
      "author_url": "",
      "post_date": "11/06/2013 12:08:52",
      "content": "<p>Use the codecs module in python.</p>\n<p style=\"padding-left: 30px\">import codecs<br>f = codecs.open('unicode.rst', encoding='utf-8')<br>for line in f:<br> print repr(line)</p>\n<p style=\"padding-left: 30px\">&nbsp;</p>\n<p>Reference:&nbsp;http://docs.python.org/2/howto/unicode.html</p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 33489,
      "author_name": "mrosett",
      "author_url": "",
      "post_date": "11/10/2013 02:43:18",
      "content": "<p>For anyone else using Python 3.x, unicode handling changed. See this article: &nbsp;</p>\n<p>http://python-notes.curiousefficiency.org/en/latest/python3/text_file_processing.html#files-in-an-ascii-compatible-encoding-best-effort-is-acceptable</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "33097": "",
    "33107": "",
    "33308": "",
    "33489": ""
  },
  "source": "meta"
}