{
  "id": 9550,
  "title": "Mistake about the provided python code?",
  "url": "/competitions/avito-prohibited-content/discussion/9550",
  "author_name": "",
  "post_date": "2014-06-25T02:30:25.050Z",
  "votes": null,
  "comment_count": 30,
  "views": 8358,
  "content": "<p>Dear Friends,</p>\n\n<p>Is there a mistake about the provided Python code?</p>\n\n<p>~</p>\n<p>I download the python code, changed the data folder path, removed the comment symbols &quot;#&quot; for the first 4 lines of main() function, and then run it with Python 2.7.</p>\n\n<p>~</p>\n<p>But then I came across following error (at line 58):</p>\n<p>AttributeError: 'NoneType' object has no attribute 'decode',</p>\n<p>item = {featureName:featureValue.decode('utf-8') for featureName,featureValue in item.iteritems()}</p>\n\n<p>~</p>\n<p>I have also attached a picture to this post about this.&nbsp;</p>\n<p>~</p>\n<p>Any ideas? Thanks in advance!</p>\n\n<p>Best wishes,</p>\n<p>Shize</p>",
  "messages": [
    {
      "id": "49522",
      "postDate": "06/25/2014 02:30:25",
      "content": "<p>Dear Friends,</p>\n\n<p>Is there a mistake about the provided Python code?</p>\n\n<p>~</p>\n<p>I download the python code, changed the data folder path, removed the comment symbols &quot;#&quot; for the first 4 lines of main() function, and then run it with Python 2.7.</p>\n\n<p>~</p>\n<p>But then I came across following error (at line 58):</p>\n<p>AttributeError: 'NoneType' object has no attribute 'decode',</p>\n<p>item = {featureName:featureValue.decode('utf-8') for featureName,featureValue in item.iteritems()}</p>\n\n<p>~</p>\n<p>I have also attached a picture to this post about this.&nbsp;</p>\n<p>~</p>\n<p>Any ideas? Thanks in advance!</p>\n\n<p>Best wishes,</p>\n<p>Shize</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49523",
      "postDate": "06/25/2014 03:07:52",
      "content": "<p>Hi Shize,</p>\n\n<p>One idea: You could try adding an exception to the code. Something like:</p>\n<p>.</p>\n<p>{featureName:featureValue.decode('utf-8') for featureName,featureValue in item.iteritems() if featureValue not None}</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49533",
      "postDate": "06/25/2014 05:53:00",
      "content": "<p>Thanks Triskelion ... Just a very minor correction -</p>\n\n<p>{featureName:featureValue.decode('utf-8') for featureName,featureValue in item.iteritems() if featureValue<em> is</em> not None}</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49735",
      "postDate": "06/27/2014 21:28:04",
      "content": "<p>Is this code supposed to reproduce the .89 benchmark? I've run it, and my submission gives me 0.28...</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49736",
      "postDate": "06/27/2014 21:57:42",
      "content": "<p>Hi Giulio,</p>\n<p>This sample code will reproduce the 0.89 benchmark. As mentioned above, after some very minor changes such as changing&nbsp;data path,&nbsp;adding an exception to the code&nbsp;{featureName:featureValue.decode('utf-8') for featureName,featureValue in item.iteritems() if featureValue is not None}, then the code generate the 0.89 benchmark for me.</p>\n<p>Just double check your code to eliminate some possible typos.</p>\n<p>Have a great day!</p>\n<p>Best wishes,</p>\n<p>Shize</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "49737",
      "postDate": "06/27/2014 22:10:25",
      "content": "<p>Thanks Shize!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "50081",
      "postDate": "07/06/2014 17:21:07",
      "content": "<p>Hi ,</p>\n\n<p>I am getting this error while executing the code.</p>\n\n<p>File &quot;C:\\Python27\\lib\\site-packages\\scipy\\sparse\\coo.py&quot;, line 313, in tocsr<br> data = np.empty(self.nnz, dtype=upcast(self.dtype))<br>MemoryError</p>\n\n\n<p>Thanks</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "51262",
      "postDate": "07/28/2014 11:24:56",
      "content": "<p>I am unable to reproduce 0.89 benchmark. The score was 0.37.</p>\n<p>I have made following changes:</p>\n<p>item = {featureName:featureValue.decode('utf-8') for featureName,featureValue in item.iteritems() if featureValue is not None}</p>\n<p>Changed the alpha value in SGD classifier to 1e-5, n_ter = 100</p>\n<p>Uncommented the 4 lines in main.</p>\n<p>What al are the other changes to be made to reproduce 0.89 benchmark</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "51398",
      "postDate": "07/31/2014 06:17:16",
      "content": "<p>&nbsp;I need to change to change the code to &quot;clf.fit(trainFeatures.<strong>todense(),</strong>trainTargets)&quot; or else there is an error in np.asarray() function (which gets called by clf.fit).</p>\n\n<p>Did others also make this change to the code for it to run ? The problem is that this loses the whole purpose of using a sparse matrix and runs into MemoryError. I am using a machine with 25GB RAM and still can not get this to run with a sample of 300K.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "51403",
      "postDate": "07/31/2014 07:33:16",
      "content": "<p>Hi,Avani</p>\n\n<p>You can not change the sparse matrix to dense. because even you only run with a sample of 300K, if todense() used, you &nbsp;need roughly 111GB to save this matrix(300K rows, 50K features, 64bit per element), you can use some algorithem which support sparse matrix.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "51426",
      "postDate": "07/31/2014 17:09:04",
      "content": "<p>Yes, that totally makes sense. My question is around the existing python code given to us and the benchmark of 0.89. How did people get it to run with the existing code:&nbsp;&quot;clf.fit(trainFeatures,trainTargets)&quot;</p>\n\n<p>I get this error:</p>\n<p>[LINE:159]# INFO [2014-07-31 17:08:40,598] Feature preparation done, fitting model...<br>Traceback (most recent call last):<br> File &quot;avito_ProhibitedContent_SampleCode.py&quot;, line 184, in &lt;module&gt;<br> main()<br> File &quot;avito_ProhibitedContent_SampleCode.py&quot;, line 165, in main<br> clf.fit(trainFeatures,trainTargets)<br> File &quot;/usr/lib/pymodules/python2.7/sklearn/linear_model/base.py&quot;, line 430, in fit<br> self._fit_binary(X, y, sample_weight)<br> File &quot;/usr/lib/pymodules/python2.7/sklearn/linear_model/stochastic_gradient.py&quot;, line 131, in _fit_binary<br> X = np.asarray(X, dtype=np.float64, order='C')<br> File &quot;/usr/lib/python2.7/dist-packages/numpy/core/numeric.py&quot;, line 235, in asarray<br> return array(a, dtype, copy=False, order=order)<br>ValueError: setting an array element with a sequence.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "51431",
      "postDate": "07/31/2014 18:11:53",
      "content": "",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "51455",
      "postDate": "08/01/2014 06:00:02",
      "content": "<p>I suppose SGDClassifier supports it too in scikit. Isn't the 0.89 benchmark using SGDClassifier ?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "51505",
      "postDate": "08/01/2014 22:03:02",
      "content": "<p>Resolved : I had an older version of scikit. Got 0.15 now and it all worked.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "51550",
      "postDate": "08/03/2014 15:00:59",
      "content": "<p>I'm also having trouble running the sample code provided. I encounter a UnicodeEncodeError:&nbsp;'ascii' codec can't encode character u'\\u0438' in position 0: ordinal not in range(128) </p>\n\n<p>From what I can tell, this error seems to occur when you don't explicitly try to decode a unicode character -- it seems that the code is explicitly decoding the character, with decode( )&nbsp;at line 21. Anyone else encounter the same problem, or know how to fix it?&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "51566",
      "postDate": "08/03/2014 20:39:15",
      "content": "<p>@orangerv: I'm getting the same error too. Not yet found a resolution.visited the forum to check if somebody had raised this problem..glad u had :)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "51568",
      "postDate": "08/03/2014 21:26:11",
      "content": "<p>I get this error on one of my machines. Not sure why, but I was able to fix it with these lines:</p>\n<p><code>import sys<br>reload(sys)<br>sys.setdefaultencoding(&quot;utf-8&quot;)<br></code></p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "51595",
      "postDate": "08/04/2014 12:13:20",
      "content": "<p>@Eben: Was it an Windows machine? (Mine is a windows) And did u have those lines as first lines of execution in the sample code? As I try, the program just stopped adter the &nbsp;setDefautlencoding line. Not sure how to debug.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "51596",
      "postDate": "08/04/2014 12:27:56",
      "content": "<p>I guess the following line,</p>\n<p>stopwords= frozenset(word.decode('utf-8') for word in nltk.corpus.stopwords.words(&quot;russian&quot;) if word!=&quot;&#1085;&#1077;&quot;)</p>\n<p>must be coded as,</p>\n<p>stopwords= frozenset(word.decode('utf-8') for word in nltk.corpus.stopwords.words(&quot;russian&quot;) if word.<em><strong>decode</strong></em>('utf-8')!=&quot;&#1085;&#1077;&quot;)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "51797",
      "postDate": "08/09/2014 15:13:25",
      "content": "<p>Yup, explicitly decoding that line worked!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52231",
      "postDate": "08/20/2014 14:31:47",
      "content": "<p>hi,</p>\n<p>I am trying to run the sample code that was provided and got this error that says it is unable to find train_data.pkl file in my data folder. Am i missing anything here? please unblock me</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52278",
      "postDate": "08/21/2014 04:39:00",
      "content": "<p>uncomment first four lines</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52444",
      "postDate": "08/25/2014 23:30:49",
      "content": "<p>I am new to python and I&nbsp;am not able to run the sample code. I have made the following changes&nbsp;</p>\n<p>- &nbsp;foldername, uncommented 4 lines</p>\n<p>- Explicitly decoded (line 21)</p>\n<p>- Added the exception (line 58)</p>\n<p>Still when I run the code, I am getting some syntax error at&nbsp;<span style=\"text-decoration: underline\"><strong>&quot;</strong></span>&nbsp;in the below line .&nbsp;</p>\n<p>if len(re.findall(ur&quot;[&#1072;-&#1103;]&nbsp;<strong><span style=\"text-decoration: underline\">&quot;</span>&nbsp;</strong>,w))&gt;len(re.findall(ur&quot;[a-z]&quot;,w)): (line 35)</p>\n\n<p>Someone please highlight the issue.&nbsp;thanks.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52460",
      "postDate": "08/26/2014 14:31:17",
      "content": "<p>I think the error is in the below line around <strong>'for'</strong></p>\n<p>{featureName:featureValue.decode('utf-8') <strong>for</strong> featureName,featureValue in item.iteritems() if featureValue is not None}</p>\n\n<p>Can anyone please suggest me the workaround.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52461",
      "postDate": "08/26/2014 15:09:17",
      "content": "<p>May be its caused due to improper indentation. Could you post the complete error message.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52462",
      "postDate": "08/26/2014 16:42:33",
      "content": "<p>Manavender, can you help me with the indentation in below two changes. Currently I am using them in a single line.</p>\n<p>Line 58&nbsp;</p>\n<p>item = {featureName:featureValue.decode('utf-8') for featureName,featureValue in item.iteritems() if featureValue is not None}</p>\n\n<p>Line 21</p>\n<p>stopwords= frozenset(word.decode('utf-8') for word in nltk.corpus.stopwords.words(&quot;russian&quot;) if word.decode('utf-8')!=&quot;&#1085;&#1077;&quot;)</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52465",
      "postDate": "08/26/2014 17:25:35",
      "content": "<p>Can you post your file.. Its hard to figure out what the problem is without error message and the contents of the file&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52475",
      "postDate": "08/26/2014 19:20:30",
      "content": "<p>Here is the updated code. Please help me to figure out the error.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52476",
      "postDate": "08/26/2014 19:35:51",
      "content": "<p>Can you please post the error message also.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52482",
      "postDate": "08/26/2014 23:08:08",
      "content": "<p>invalid syntax</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "52487",
      "postDate": "08/27/2014 03:44:23",
      "content": "<p>Its running on my machine with Ubuntu.. not sure what the problem is :(</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 49523,
      "author_name": "triskelion",
      "author_url": "",
      "post_date": "06/25/2014 03:07:52",
      "content": "<p>Hi Shize,</p>\n\n<p>One idea: You could try adding an exception to the code. Something like:</p>\n<p>.</p>\n<p>{featureName:featureValue.decode('utf-8') for featureName,featureValue in item.iteritems() if featureValue not None}</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49533,
      "author_name": "mathusuthan",
      "author_url": "",
      "post_date": "06/25/2014 05:53:00",
      "content": "<p>Thanks Triskelion ... Just a very minor correction -</p>\n\n<p>{featureName:featureValue.decode('utf-8') for featureName,featureValue in item.iteritems() if featureValue<em> is</em> not None}</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49735,
      "author_name": "adjgiulio",
      "author_url": "",
      "post_date": "06/27/2014 21:28:04",
      "content": "<p>Is this code supposed to reproduce the .89 benchmark? I've run it, and my submission gives me 0.28...</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49736,
      "author_name": "sushize",
      "author_url": "",
      "post_date": "06/27/2014 21:57:42",
      "content": "<p>Hi Giulio,</p>\n<p>This sample code will reproduce the 0.89 benchmark. As mentioned above, after some very minor changes such as changing&nbsp;data path,&nbsp;adding an exception to the code&nbsp;{featureName:featureValue.decode('utf-8') for featureName,featureValue in item.iteritems() if featureValue is not None}, then the code generate the 0.89 benchmark for me.</p>\n<p>Just double check your code to eliminate some possible typos.</p>\n<p>Have a great day!</p>\n<p>Best wishes,</p>\n<p>Shize</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 49737,
      "author_name": "adjgiulio",
      "author_url": "",
      "post_date": "06/27/2014 22:10:25",
      "content": "<p>Thanks Shize!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 50081,
      "author_name": "emohit",
      "author_url": "",
      "post_date": "07/06/2014 17:21:07",
      "content": "<p>Hi ,</p>\n\n<p>I am getting this error while executing the code.</p>\n\n<p>File &quot;C:\\Python27\\lib\\site-packages\\scipy\\sparse\\coo.py&quot;, line 313, in tocsr<br> data = np.empty(self.nnz, dtype=upcast(self.dtype))<br>MemoryError</p>\n\n\n<p>Thanks</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 51262,
      "author_name": "novice7",
      "author_url": "",
      "post_date": "07/28/2014 11:24:56",
      "content": "<p>I am unable to reproduce 0.89 benchmark. The score was 0.37.</p>\n<p>I have made following changes:</p>\n<p>item = {featureName:featureValue.decode('utf-8') for featureName,featureValue in item.iteritems() if featureValue is not None}</p>\n<p>Changed the alpha value in SGD classifier to 1e-5, n_ter = 100</p>\n<p>Uncommented the 4 lines in main.</p>\n<p>What al are the other changes to be made to reproduce 0.89 benchmark</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 51398,
      "author_name": "avanigoe",
      "author_url": "",
      "post_date": "07/31/2014 06:17:16",
      "content": "<p>&nbsp;I need to change to change the code to &quot;clf.fit(trainFeatures.<strong>todense(),</strong>trainTargets)&quot; or else there is an error in np.asarray() function (which gets called by clf.fit).</p>\n\n<p>Did others also make this change to the code for it to run ? The problem is that this loses the whole purpose of using a sparse matrix and runs into MemoryError. I am using a machine with 25GB RAM and still can not get this to run with a sample of 300K.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 51403,
      "author_name": "kevinandhu",
      "author_url": "",
      "post_date": "07/31/2014 07:33:16",
      "content": "<p>Hi,Avani</p>\n\n<p>You can not change the sparse matrix to dense. because even you only run with a sample of 300K, if todense() used, you &nbsp;need roughly 111GB to save this matrix(300K rows, 50K features, 64bit per element), you can use some algorithem which support sparse matrix.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 51426,
      "author_name": "avanigoe",
      "author_url": "",
      "post_date": "07/31/2014 17:09:04",
      "content": "<p>Yes, that totally makes sense. My question is around the existing python code given to us and the benchmark of 0.89. How did people get it to run with the existing code:&nbsp;&quot;clf.fit(trainFeatures,trainTargets)&quot;</p>\n\n<p>I get this error:</p>\n<p>[LINE:159]# INFO [2014-07-31 17:08:40,598] Feature preparation done, fitting model...<br>Traceback (most recent call last):<br> File &quot;avito_ProhibitedContent_SampleCode.py&quot;, line 184, in &lt;module&gt;<br> main()<br> File &quot;avito_ProhibitedContent_SampleCode.py&quot;, line 165, in main<br> clf.fit(trainFeatures,trainTargets)<br> File &quot;/usr/lib/pymodules/python2.7/sklearn/linear_model/base.py&quot;, line 430, in fit<br> self._fit_binary(X, y, sample_weight)<br> File &quot;/usr/lib/pymodules/python2.7/sklearn/linear_model/stochastic_gradient.py&quot;, line 131, in _fit_binary<br> X = np.asarray(X, dtype=np.float64, order='C')<br> File &quot;/usr/lib/python2.7/dist-packages/numpy/core/numeric.py&quot;, line 235, in asarray<br> return array(a, dtype, copy=False, order=order)<br>ValueError: setting an array element with a sequence.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 51431,
      "author_name": "avanigoe",
      "author_url": "",
      "post_date": "07/31/2014 18:11:53",
      "content": "",
      "votes": null,
      "replies": []
    },
    {
      "id": 51455,
      "author_name": "avanigoe",
      "author_url": "",
      "post_date": "08/01/2014 06:00:02",
      "content": "<p>I suppose SGDClassifier supports it too in scikit. Isn't the 0.89 benchmark using SGDClassifier ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 51505,
      "author_name": "avanigoe",
      "author_url": "",
      "post_date": "08/01/2014 22:03:02",
      "content": "<p>Resolved : I had an older version of scikit. Got 0.15 now and it all worked.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 51550,
      "author_name": "orangerv",
      "author_url": "",
      "post_date": "08/03/2014 15:00:59",
      "content": "<p>I'm also having trouble running the sample code provided. I encounter a UnicodeEncodeError:&nbsp;'ascii' codec can't encode character u'\\u0438' in position 0: ordinal not in range(128) </p>\n\n<p>From what I can tell, this error seems to occur when you don't explicitly try to decode a unicode character -- it seems that the code is explicitly decoding the character, with decode( )&nbsp;at line 21. Anyone else encounter the same problem, or know how to fix it?&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 51566,
      "author_name": "muthujothi",
      "author_url": "",
      "post_date": "08/03/2014 20:39:15",
      "content": "<p>@orangerv: I'm getting the same error too. Not yet found a resolution.visited the forum to check if somebody had raised this problem..glad u had :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 51568,
      "author_name": "emolson",
      "author_url": "",
      "post_date": "08/03/2014 21:26:11",
      "content": "<p>I get this error on one of my machines. Not sure why, but I was able to fix it with these lines:</p>\n<p><code>import sys<br>reload(sys)<br>sys.setdefaultencoding(&quot;utf-8&quot;)<br></code></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 51595,
      "author_name": "muthujothi",
      "author_url": "",
      "post_date": "08/04/2014 12:13:20",
      "content": "<p>@Eben: Was it an Windows machine? (Mine is a windows) And did u have those lines as first lines of execution in the sample code? As I try, the program just stopped adter the &nbsp;setDefautlencoding line. Not sure how to debug.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 51596,
      "author_name": "muthujothi",
      "author_url": "",
      "post_date": "08/04/2014 12:27:56",
      "content": "<p>I guess the following line,</p>\n<p>stopwords= frozenset(word.decode('utf-8') for word in nltk.corpus.stopwords.words(&quot;russian&quot;) if word!=&quot;&#1085;&#1077;&quot;)</p>\n<p>must be coded as,</p>\n<p>stopwords= frozenset(word.decode('utf-8') for word in nltk.corpus.stopwords.words(&quot;russian&quot;) if word.<em><strong>decode</strong></em>('utf-8')!=&quot;&#1085;&#1077;&quot;)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 51797,
      "author_name": "orangerv",
      "author_url": "",
      "post_date": "08/09/2014 15:13:25",
      "content": "<p>Yup, explicitly decoding that line worked!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52231,
      "author_name": "pastry",
      "author_url": "",
      "post_date": "08/20/2014 14:31:47",
      "content": "<p>hi,</p>\n<p>I am trying to run the sample code that was provided and got this error that says it is unable to find train_data.pkl file in my data folder. Am i missing anything here? please unblock me</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52278,
      "author_name": "datanovice111981",
      "author_url": "",
      "post_date": "08/21/2014 04:39:00",
      "content": "<p>uncomment first four lines</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52444,
      "author_name": "anshul",
      "author_url": "",
      "post_date": "08/25/2014 23:30:49",
      "content": "<p>I am new to python and I&nbsp;am not able to run the sample code. I have made the following changes&nbsp;</p>\n<p>- &nbsp;foldername, uncommented 4 lines</p>\n<p>- Explicitly decoded (line 21)</p>\n<p>- Added the exception (line 58)</p>\n<p>Still when I run the code, I am getting some syntax error at&nbsp;<span style=\"text-decoration: underline\"><strong>&quot;</strong></span>&nbsp;in the below line .&nbsp;</p>\n<p>if len(re.findall(ur&quot;[&#1072;-&#1103;]&nbsp;<strong><span style=\"text-decoration: underline\">&quot;</span>&nbsp;</strong>,w))&gt;len(re.findall(ur&quot;[a-z]&quot;,w)): (line 35)</p>\n\n<p>Someone please highlight the issue.&nbsp;thanks.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52460,
      "author_name": "anshul",
      "author_url": "",
      "post_date": "08/26/2014 14:31:17",
      "content": "<p>I think the error is in the below line around <strong>'for'</strong></p>\n<p>{featureName:featureValue.decode('utf-8') <strong>for</strong> featureName,featureValue in item.iteritems() if featureValue is not None}</p>\n\n<p>Can anyone please suggest me the workaround.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52461,
      "author_name": "mrkaggler",
      "author_url": "",
      "post_date": "08/26/2014 15:09:17",
      "content": "<p>May be its caused due to improper indentation. Could you post the complete error message.&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52462,
      "author_name": "anshul",
      "author_url": "",
      "post_date": "08/26/2014 16:42:33",
      "content": "<p>Manavender, can you help me with the indentation in below two changes. Currently I am using them in a single line.</p>\n<p>Line 58&nbsp;</p>\n<p>item = {featureName:featureValue.decode('utf-8') for featureName,featureValue in item.iteritems() if featureValue is not None}</p>\n\n<p>Line 21</p>\n<p>stopwords= frozenset(word.decode('utf-8') for word in nltk.corpus.stopwords.words(&quot;russian&quot;) if word.decode('utf-8')!=&quot;&#1085;&#1077;&quot;)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52465,
      "author_name": "mrkaggler",
      "author_url": "",
      "post_date": "08/26/2014 17:25:35",
      "content": "<p>Can you post your file.. Its hard to figure out what the problem is without error message and the contents of the file&nbsp;</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52475,
      "author_name": "anshul",
      "author_url": "",
      "post_date": "08/26/2014 19:20:30",
      "content": "<p>Here is the updated code. Please help me to figure out the error.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52476,
      "author_name": "mrkaggler",
      "author_url": "",
      "post_date": "08/26/2014 19:35:51",
      "content": "<p>Can you please post the error message also.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52482,
      "author_name": "anshul",
      "author_url": "",
      "post_date": "08/26/2014 23:08:08",
      "content": "<p>invalid syntax</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 52487,
      "author_name": "mrkaggler",
      "author_url": "",
      "post_date": "08/27/2014 03:44:23",
      "content": "<p>Its running on my machine with Ubuntu.. not sure what the problem is :(</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "49522": "",
    "49523": "",
    "49533": "",
    "49735": "",
    "49736": "",
    "49737": "",
    "50081": "",
    "51262": "",
    "51398": "",
    "51403": "",
    "51426": "",
    "51431": "",
    "51455": "",
    "51505": "",
    "51550": "",
    "51566": "",
    "51568": "",
    "51595": "",
    "51596": "",
    "51797": "",
    "52231": "",
    "52278": "",
    "52444": "",
    "52460": "",
    "52461": "",
    "52462": "",
    "52465": "",
    "52475": "",
    "52476": "",
    "52482": "",
    "52487": ""
  },
  "source": "meta"
}