{
  "id": 5565,
  "title": "Test.csv errors",
  "url": "/competitions/accelerometer-biometric-competition/discussion/5565",
  "author_name": "",
  "post_date": "2013-08-28T01:36:20.057Z",
  "votes": null,
  "comment_count": 1,
  "views": 917,
  "content": "<p>&nbsp;</p>\n<p>After running on the train data for the last few weeks, I switched over to the test.csv data today.</p>\n<p>My code crashed after an 1 hour 48 minutes of processing.&nbsp;</p>\n<p>Apparently the T column in the Test data is being read incorrectly or the data is bad ( I am not sure which). &nbsp; </p>\n<p>Using Pandas, I got 3 different data types in my T column:</p>\n<p><code>23,468,256 integers<br></code><code>3,276,800 floats<br></code><code>262,144 strings<br></code></p>\n<p>&nbsp;</p>\n<p>Here is my code:</p>\n<p>import pandas as pd</p>\n<p>test = pd.read_csv('test.csv')</p>\n<p>test.sort(columns=['SequenceId','T'])</p>\n<p>&nbsp;</p>\n<p>Anyone else experience this?&nbsp;</p>\n<p>Thanks!</p>\n<p>&nbsp;</p>",
  "messages": [
    {
      "id": "29614",
      "postDate": "08/28/2013 01:36:20",
      "content": "<p>&nbsp;</p>\n<p>After running on the train data for the last few weeks, I switched over to the test.csv data today.</p>\n<p>My code crashed after an 1 hour 48 minutes of processing.&nbsp;</p>\n<p>Apparently the T column in the Test data is being read incorrectly or the data is bad ( I am not sure which). &nbsp; </p>\n<p>Using Pandas, I got 3 different data types in my T column:</p>\n<p><code>23,468,256 integers<br></code><code>3,276,800 floats<br></code><code>262,144 strings<br></code></p>\n<p>&nbsp;</p>\n<p>Here is my code:</p>\n<p>import pandas as pd</p>\n<p>test = pd.read_csv('test.csv')</p>\n<p>test.sort(columns=['SequenceId','T'])</p>\n<p>&nbsp;</p>\n<p>Anyone else experience this?&nbsp;</p>\n<p>Thanks!</p>\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "29647",
      "postDate": "08/28/2013 14:51:12",
      "content": "<p>Update...</p>\n<p>I downloaded the test.csv.zip again and the problem was magically resolved!</p>\n<p>It was either bad originally ( I downloaded early on)&nbsp; or was corrupted in the download/extract process.&nbsp;</p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 29647,
      "author_name": "geringer",
      "author_url": "",
      "post_date": "08/28/2013 14:51:12",
      "content": "<p>Update...</p>\n<p>I downloaded the test.csv.zip again and the problem was magically resolved!</p>\n<p>It was either bad originally ( I downloaded early on)&nbsp; or was corrupted in the download/extract process.&nbsp;</p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>\n<p>&nbsp;</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "29614": "",
    "29647": ""
  },
  "source": "meta"
}