{
  "id": 24346,
  "title": "Pandas.to_sparse() and converting to xgb.Dmatrix()",
  "url": "/competitions/bosch-production-line-performance/discussion/24346",
  "author_name": "",
  "post_date": "2016-10-12T21:03:43.750Z",
  "votes": null,
  "comment_count": 1,
  "views": 389,
  "content": "<p>Hi kagglers, If anyone know how to get xgb.Dmatrix from sparse pandas dataframe without converting it to dense please help.</p>\n\n<p>In details: for example I have pandas.to_sparse() with 1GB in RAM and I want to make Dmatrix from it without any serious peaks in RAM.</p>",
  "messages": [
    {
      "id": "139199",
      "postDate": "10/12/2016 21:03:43",
      "content": "<p>Hi kagglers, If anyone know how to get xgb.Dmatrix from sparse pandas dataframe without converting it to dense please help.</p>\n\n<p>In details: for example I have pandas.to_sparse() with 1GB in RAM and I want to make Dmatrix from it without any serious peaks in RAM.</p>",
      "rawMarkdown": "Hi kagglers, If anyone know how to get xgb.Dmatrix from sparse pandas dataframe without converting it to dense please help.\r\n\r\n\r\nIn details: for example I have pandas.to_sparse() with 1GB in RAM and I want to make Dmatrix from it without any serious peaks in RAM.",
      "votes": null
    },
    {
      "id": "139461",
      "postDate": "10/14/2016 09:06:29",
      "content": "<p>Have you tried to pass it directly to the DMatrix constructor?</p>\n\n<p>My experience with sparse pandas dataframes is mixed.  I could not save them.  This may be better in pandas 19.0, I used 18.1.</p>",
      "rawMarkdown": "Have you tried to pass it directly to the DMatrix constructor?\r\n\r\nMy experience with sparse pandas dataframes is mixed.  I could not save them.  This may be better in pandas 19.0, I used 18.1.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 139461,
      "author_name": "cpmpml",
      "author_url": "",
      "post_date": "10/14/2016 09:06:29",
      "content": "<p>Have you tried to pass it directly to the DMatrix constructor?</p>\n\n<p>My experience with sparse pandas dataframes is mixed.  I could not save them.  This may be better in pandas 19.0, I used 18.1.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "139199": "Hi kagglers, If anyone know how to get xgb.Dmatrix from sparse pandas dataframe without converting it to dense please help.\r\n\r\n\r\nIn details: for example I have pandas.to_sparse() with 1GB in RAM and I want to make Dmatrix from it without any serious peaks in RAM.",
    "139461": "Have you tried to pass it directly to the DMatrix constructor?\r\n\r\nMy experience with sparse pandas dataframes is mixed.  I could not save them.  This may be better in pandas 19.0, I used 18.1."
  },
  "source": "meta"
}