{
  "id": 11123,
  "title": "parallelizing cforest ?",
  "url": "/competitions/avazu-ctr-prediction/discussion/11123",
  "author_name": "",
  "post_date": "2014-12-03T13:16:55.897Z",
  "votes": null,
  "comment_count": 2,
  "views": 1198,
  "content": "<p>Hi&nbsp;</p>\n<p>First of all this is my first competition in Kaggle and I am new to data science. I am trying to use cforest (R 'party' package) for checking the relation between 'click' and some of the other parameters. But it is giving me the error :</p>\n<p>&nbsp; &nbsp;Error: cannot allocate vector of size 18.4 Gb<br>In addition: Warning messages:<br>1: In model.matrix.default(~x - 1) :<br> Reached total allocation of 16080Mb: see help(memory.size)<br>2: In model.matrix.default(~x - 1) :<br> Reached total allocation of 16080Mb: see help(memory.size)<br>3: In model.matrix.default(~x - 1) :<br> Reached total allocation of 16080Mb: see help(memory.size)<br>4: In model.matrix.default(~x - 1) :<br> Reached total allocation of 16080Mb: see help(memory.size)</p>\n\n<p>My system has 16gb ram and 4 cores , I tried using random forest also and parallelized it using foreach but there also I am getting an error that rforest</p>\n<p>&quot;Can not handle categorical predictors with more than 53 categories.&quot;</p>\n\n<p>Please suggest me an approach</p>",
  "messages": [
    {
      "id": "59459",
      "postDate": "12/03/2014 13:16:55",
      "content": "<p>Hi&nbsp;</p>\n<p>First of all this is my first competition in Kaggle and I am new to data science. I am trying to use cforest (R 'party' package) for checking the relation between 'click' and some of the other parameters. But it is giving me the error :</p>\n<p>&nbsp; &nbsp;Error: cannot allocate vector of size 18.4 Gb<br>In addition: Warning messages:<br>1: In model.matrix.default(~x - 1) :<br> Reached total allocation of 16080Mb: see help(memory.size)<br>2: In model.matrix.default(~x - 1) :<br> Reached total allocation of 16080Mb: see help(memory.size)<br>3: In model.matrix.default(~x - 1) :<br> Reached total allocation of 16080Mb: see help(memory.size)<br>4: In model.matrix.default(~x - 1) :<br> Reached total allocation of 16080Mb: see help(memory.size)</p>\n\n<p>My system has 16gb ram and 4 cores , I tried using random forest also and parallelized it using foreach but there also I am getting an error that rforest</p>\n<p>&quot;Can not handle categorical predictors with more than 53 categories.&quot;</p>\n\n<p>Please suggest me an approach</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "59462",
      "postDate": "12/03/2014 14:13:38",
      "content": "<p>Try &nbsp;to use &nbsp; gc()</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "161709",
      "postDate": "02/15/2017 10:11:54",
      "content": "<p>As the Error message clearly states, you have a factor variable in your dataset that contains more than 53 levels (categories) - remove it or transform it (to character, numeric, etc.) and your problem will be solved.</p>",
      "rawMarkdown": "As the Error message clearly states, you have a factor variable in your dataset that contains more than 53 levels (categories) - remove it or transform it (to character, numeric, etc.) and your problem will be solved.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 59462,
      "author_name": "",
      "author_url": "",
      "post_date": "12/03/2014 14:13:38",
      "content": "<p>Try &nbsp;to use &nbsp; gc()</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 161709,
      "author_name": "kkulma",
      "author_url": "",
      "post_date": "02/15/2017 10:11:54",
      "content": "<p>As the Error message clearly states, you have a factor variable in your dataset that contains more than 53 levels (categories) - remove it or transform it (to character, numeric, etc.) and your problem will be solved.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "59459": "",
    "59462": "",
    "161709": "As the Error message clearly states, you have a factor variable in your dataset that contains more than 53 levels (categories) - remove it or transform it (to character, numeric, etc.) and your problem will be solved."
  },
  "source": "meta"
}