{
  "id": 71015,
  "title": "Unable to load the V2 datasets using kaggle kernels due to a lack of memory.",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/71015",
  "author_name": "",
  "post_date": "2018-11-09T09:35:43.819208900Z",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hi, \nI was working with the first version of the datasets using a kaggle kernel and everything was fine, but recently the V2 has replaced it, and now it seems that there isn't enough RAM (14 gb) for the datasets to be loaded (at least not using read.csv()). \nSince it's a reoccuring issue for me when working with competition datasets,  I was wondering if there was another way to load the files without taking all the RAM both in python and R?\nThanks.</p>",
  "messages": [
    {
      "id": "418100",
      "postDate": "11/09/2018 09:35:43",
      "content": "<p>Hi, \nI was working with the first version of the datasets using a kaggle kernel and everything was fine, but recently the V2 has replaced it, and now it seems that there isn't enough RAM (14 gb) for the datasets to be loaded (at least not using read.csv()). \nSince it's a reoccuring issue for me when working with competition datasets,  I was wondering if there was another way to load the files without taking all the RAM both in python and R?\nThanks.</p>",
      "rawMarkdown": "Hi, \nI was working with the first version of the datasets using a kaggle kernel and everything was fine, but recently the V2 has replaced it, and now it seems that there isn't enough RAM (14 gb) for the datasets to be loaded (at least not using read.csv()). \nSince it's a reoccuring issue for me when working with competition datasets,  I was wondering if there was another way to load the files without taking all the RAM both in python and R?\nThanks.",
      "votes": null
    },
    {
      "id": "418123",
      "postDate": "11/09/2018 10:30:00",
      "content": "<p>Just totally recleaned my laptop just to be able to unzip the file, but didn't think about this yet. I'm gonna try this:\n<a href=\"https://stackoverflow.com/questions/26063231/read-specific-columns-with-pandas-or-other-python-module\">https://stackoverflow.com/questions/26063231/read-specific-columns-with-pandas-or-other-python-module</a></p>",
      "rawMarkdown": "Just totally recleaned my laptop just to be able to unzip the file, but didn't think about this yet. I'm gonna try this:\nhttps://stackoverflow.com/questions/26063231/read-specific-columns-with-pandas-or-other-python-module",
      "votes": null
    },
    {
      "id": "418129",
      "postDate": "11/09/2018 10:44:33",
      "content": "<p>Ok I'll look into it, hope it helps.\nThanks.</p>",
      "rawMarkdown": "Ok I'll look into it, hope it helps.\nThanks.",
      "votes": null
    },
    {
      "id": "418154",
      "postDate": "11/09/2018 11:29:29",
      "content": "<p>Started out using this:\ndata = pd.read_csv('train_v2.csv', nrows=200)\nJust to get the first 200 rows. Now trying to convert it to a flat file. But got some issues with expected values while trying to read json.</p>",
      "rawMarkdown": "Started out using this:\ndata = pd.read_csv('train_v2.csv', nrows=200)\nJust to get the first 200 rows. Now trying to convert it to a flat file. But got some issues with expected values while trying to read json.",
      "votes": null
    },
    {
      "id": "418581",
      "postDate": "11/10/2018 07:08:02",
      "content": "<p><a href=\"https://www.kaggle.com/burritodan/gstore-1-flatten\">https://www.kaggle.com/burritodan/gstore-1-flatten</a></p>",
      "rawMarkdown": "https://www.kaggle.com/burritodan/gstore-1-flatten",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 418123,
      "author_name": "ndekker",
      "author_url": "",
      "post_date": "11/09/2018 10:30:00",
      "content": "<p>Just totally recleaned my laptop just to be able to unzip the file, but didn't think about this yet. I'm gonna try this:\n<a href=\"https://stackoverflow.com/questions/26063231/read-specific-columns-with-pandas-or-other-python-module\">https://stackoverflow.com/questions/26063231/read-specific-columns-with-pandas-or-other-python-module</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 418129,
          "author_name": "antonylavoye",
          "author_url": "",
          "post_date": "11/09/2018 10:44:33",
          "content": "<p>Ok I'll look into it, hope it helps.\nThanks.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 418154,
          "author_name": "ndekker",
          "author_url": "",
          "post_date": "11/09/2018 11:29:29",
          "content": "<p>Started out using this:\ndata = pd.read_csv('train_v2.csv', nrows=200)\nJust to get the first 200 rows. Now trying to convert it to a flat file. But got some issues with expected values while trying to read json.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 418581,
      "author_name": "burritodan",
      "author_url": "",
      "post_date": "11/10/2018 07:08:02",
      "content": "<p><a href=\"https://www.kaggle.com/burritodan/gstore-1-flatten\">https://www.kaggle.com/burritodan/gstore-1-flatten</a></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "418100": "Hi, \nI was working with the first version of the datasets using a kaggle kernel and everything was fine, but recently the V2 has replaced it, and now it seems that there isn't enough RAM (14 gb) for the datasets to be loaded (at least not using read.csv()). \nSince it's a reoccuring issue for me when working with competition datasets,  I was wondering if there was another way to load the files without taking all the RAM both in python and R?\nThanks.",
    "418123": "Just totally recleaned my laptop just to be able to unzip the file, but didn't think about this yet. I'm gonna try this:\nhttps://stackoverflow.com/questions/26063231/read-specific-columns-with-pandas-or-other-python-module",
    "418129": "Ok I'll look into it, hope it helps.\nThanks.",
    "418154": "Started out using this:\ndata = pd.read_csv('train_v2.csv', nrows=200)\nJust to get the first 200 rows. Now trying to convert it to a flat file. But got some issues with expected values while trying to read json.",
    "418581": "https://www.kaggle.com/burritodan/gstore-1-flatten"
  },
  "source": "meta"
}