{
  "id": 21605,
  "title": "Reading Data",
  "url": "/competitions/grupo-bimbo-inventory-demand/discussion/21605",
  "author_name": "",
  "post_date": "2016-06-11T20:35:12.030Z",
  "votes": null,
  "comment_count": 3,
  "views": 1048,
  "content": "<p>I want to read data directly from the url without saving the files into my disk. I am getting an error:</p>\n\n<p><em>Error in open.connection(file, &quot;rt&quot;) : cannot open the connection\nIn addition: Warning message:\nIn open.connection(file, &quot;rt&quot;) :\n  cannot open zip file '/tmp/RtmpmUp42a/file255a3b8d714'</em></p>\n\n<p>This is the code I am using:</p>\n\n<p>temp &lt;- tempfile()</p>\n\n<p>download.file(&quot;<a href=\"https://www.kaggle.com/c/grupo-bimbo-inventory-demand/download/town_state.csv.zip\">https://www.kaggle.com/c/grupo-bimbo-inventory-demand/download/town_state.csv.zip</a>&quot;,temp)</p>\n\n<p>data &lt;- read.table(unz(temp, &quot;town_state.csv&quot;))</p>\n\n<p>Does anyone know what is wrong?</p>\n\n<p>As an additional question, how are most people reading in the data? Are you downloading on your local machine and reading into an IDE like R Studio?</p>",
  "messages": [
    {
      "id": "123426",
      "postDate": "06/11/2016 20:35:12",
      "content": "<p>I want to read data directly from the url without saving the files into my disk. I am getting an error:</p>\n\n<p><em>Error in open.connection(file, &quot;rt&quot;) : cannot open the connection\nIn addition: Warning message:\nIn open.connection(file, &quot;rt&quot;) :\n  cannot open zip file '/tmp/RtmpmUp42a/file255a3b8d714'</em></p>\n\n<p>This is the code I am using:</p>\n\n<p>temp &lt;- tempfile()</p>\n\n<p>download.file(&quot;<a href=\"https://www.kaggle.com/c/grupo-bimbo-inventory-demand/download/town_state.csv.zip\">https://www.kaggle.com/c/grupo-bimbo-inventory-demand/download/town_state.csv.zip</a>&quot;,temp)</p>\n\n<p>data &lt;- read.table(unz(temp, &quot;town_state.csv&quot;))</p>\n\n<p>Does anyone know what is wrong?</p>\n\n<p>As an additional question, how are most people reading in the data? Are you downloading on your local machine and reading into an IDE like R Studio?</p>",
      "rawMarkdown": "I want to read data directly from the url without saving the files into my disk. I am getting an error:\r\n\r\n*Error in open.connection(file, \"rt\") : cannot open the connection\r\nIn addition: Warning message:\r\nIn open.connection(file, \"rt\") :\r\n  cannot open zip file '/tmp/RtmpmUp42a/file255a3b8d714'*\r\n\r\nThis is the code I am using:\r\n\r\ntemp <- tempfile()\r\n\r\ndownload.file(\"https://www.kaggle.com/c/grupo-bimbo-inventory-demand/download/town_state.csv.zip\",temp)\r\n\r\ndata <- read.table(unz(temp, \"town_state.csv\"))\r\n\r\n\r\nDoes anyone know what is wrong?\r\n\r\nAs an additional question, how are most people reading in the data? Are you downloading on your local machine and reading into an IDE like R Studio?",
      "votes": null
    },
    {
      "id": "123524",
      "postDate": "06/12/2016 14:28:18",
      "content": "<p>Try data.table::fread.</p>\n\n<p>Note that this dataset is pretty large, and you're probably better off iterating through the file in chunks, rather than all at once</p>",
      "rawMarkdown": "Try data.table::fread.\r\n\r\nNote that this dataset is pretty large, and you're probably better off iterating through the file in chunks, rather than all at once",
      "votes": null
    },
    {
      "id": "123586",
      "postDate": "06/12/2016 22:07:22",
      "content": "<p>Try using python rather than R......its pretty much fast in python compared to R data.table.</p>\n\n<p>Hope it helps.\nThank-you.</p>",
      "rawMarkdown": "Try using python rather than R......its pretty much fast in python compared to R data.table.\r\n\r\nHope it helps.\r\nThank-you.",
      "votes": null
    },
    {
      "id": "123595",
      "postDate": "06/13/2016 01:14:58",
      "content": "<p>Thanks. So it looks like most people are downloading on their local machine or dropbox and then reading it using fread and using only parts of the dataset, correct? </p>",
      "rawMarkdown": "Thanks. So it looks like most people are downloading on their local machine or dropbox and then reading it using fread and using only parts of the dataset, correct?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 123524,
      "author_name": "zachmayer",
      "author_url": "",
      "post_date": "06/12/2016 14:28:18",
      "content": "<p>Try data.table::fread.</p>\n\n<p>Note that this dataset is pretty large, and you're probably better off iterating through the file in chunks, rather than all at once</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 123586,
      "author_name": "",
      "author_url": "",
      "post_date": "06/12/2016 22:07:22",
      "content": "<p>Try using python rather than R......its pretty much fast in python compared to R data.table.</p>\n\n<p>Hope it helps.\nThank-you.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 123595,
      "author_name": "sharathlives",
      "author_url": "",
      "post_date": "06/13/2016 01:14:58",
      "content": "<p>Thanks. So it looks like most people are downloading on their local machine or dropbox and then reading it using fread and using only parts of the dataset, correct? </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "123426": "I want to read data directly from the url without saving the files into my disk. I am getting an error:\r\n\r\n*Error in open.connection(file, \"rt\") : cannot open the connection\r\nIn addition: Warning message:\r\nIn open.connection(file, \"rt\") :\r\n  cannot open zip file '/tmp/RtmpmUp42a/file255a3b8d714'*\r\n\r\nThis is the code I am using:\r\n\r\ntemp <- tempfile()\r\n\r\ndownload.file(\"https://www.kaggle.com/c/grupo-bimbo-inventory-demand/download/town_state.csv.zip\",temp)\r\n\r\ndata <- read.table(unz(temp, \"town_state.csv\"))\r\n\r\n\r\nDoes anyone know what is wrong?\r\n\r\nAs an additional question, how are most people reading in the data? Are you downloading on your local machine and reading into an IDE like R Studio?",
    "123524": "Try data.table::fread.\r\n\r\nNote that this dataset is pretty large, and you're probably better off iterating through the file in chunks, rather than all at once",
    "123586": "Try using python rather than R......its pretty much fast in python compared to R data.table.\r\n\r\nHope it helps.\r\nThank-you.",
    "123595": "Thanks. So it looks like most people are downloading on their local machine or dropbox and then reading it using fread and using only parts of the dataset, correct?"
  },
  "source": "meta"
}