{
  "id": 440592,
  "title": "Tackling out of memory issue on test set",
  "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/440592",
  "author_name": "",
  "post_date": "2023-09-15T14:24:57.051224500Z",
  "votes": 5,
  "comment_count": 2,
  "views": 0,
  "content": "<p>If you are facing out of memory issue on test set, here are some things that might work for you</p>\n<ol>\n<li>Try to use optimal data types for features</li>\n<li>Reduce number of features</li>\n<li>Batch wise predictions i.e. load only a set of series_id at once and predict</li>\n<li>Submit only inference notebook, try saving your model to a pkl file maybe, it also reduces the evaluation time</li>\n</ol>\n<p>Here is a code for loading one series at a time and predicting<br>\n<a href=\"https://www.kaggle.com/code/anilreddyvv/tackling-out-of-memory-issue-on-test-set?scriptVersionId=142873650\" target=\"_blank\">Tackling out of memory issue on test set</a></p>",
  "messages": [
    {
      "id": "2440461",
      "postDate": "09/15/2023 14:24:57",
      "content": "<p>If you are facing out of memory issue on test set, here are some things that might work for you</p>\n<ol>\n<li>Try to use optimal data types for features</li>\n<li>Reduce number of features</li>\n<li>Batch wise predictions i.e. load only a set of series_id at once and predict</li>\n<li>Submit only inference notebook, try saving your model to a pkl file maybe, it also reduces the evaluation time</li>\n</ol>\n<p>Here is a code for loading one series at a time and predicting<br>\n<a href=\"https://www.kaggle.com/code/anilreddyvv/tackling-out-of-memory-issue-on-test-set?scriptVersionId=142873650\" target=\"_blank\">Tackling out of memory issue on test set</a></p>",
      "rawMarkdown": "If you are facing out of memory issue on test set, here are some things that might work for you\n1. Try to use optimal data types for features\n2. Reduce number of features\n3. Batch wise predictions i.e. load only a set of series_id at once and predict\n4. Submit only inference notebook, try saving your model to a pkl file maybe, it also reduces the evaluation time\n\nHere is a code for loading one series at a time and predicting\n[Tackling out of memory issue on test set](https://www.kaggle.com/code/anilreddyvv/tackling-out-of-memory-issue-on-test-set?scriptVersionId=142873650)",
      "votes": null
    },
    {
      "id": "2440560",
      "postDate": "09/15/2023 15:23:21",
      "content": "<p>Impressive. For the 4th point I just noticed that we can actually upload our pretrained model \"as a dataset\" .</p>\n<p>Example, <a href=\"https://www.kaggle.com/datasets/carlmcbrideellis/tabpfn-019-whl\" target=\"_blank\">TabPFN model</a></p>",
      "rawMarkdown": "Impressive. For the 4th point I just noticed that we can actually upload our pretrained model \"as a dataset\" .\n\nExample, [TabPFN model](https://www.kaggle.com/datasets/carlmcbrideellis/tabpfn-019-whl)",
      "votes": null
    },
    {
      "id": "2440583",
      "postDate": "09/15/2023 15:32:32",
      "content": "<p>Yes <a href=\"https://www.kaggle.com/rimbax\" target=\"_blank\">@rimbax</a>, I am also using model as dataset for submissions</p>",
      "rawMarkdown": "Yes @rimbax, I am also using model as dataset for submissions",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2440560,
      "author_name": "rimbax",
      "author_url": "",
      "post_date": "09/15/2023 15:23:21",
      "content": "<p>Impressive. For the 4th point I just noticed that we can actually upload our pretrained model \"as a dataset\" .</p>\n<p>Example, <a href=\"https://www.kaggle.com/datasets/carlmcbrideellis/tabpfn-019-whl\" target=\"_blank\">TabPFN model</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 2440583,
          "author_name": "anilreddyvv",
          "author_url": "",
          "post_date": "09/15/2023 15:32:32",
          "content": "<p>Yes <a href=\"https://www.kaggle.com/rimbax\" target=\"_blank\">@rimbax</a>, I am also using model as dataset for submissions</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2440461": "If you are facing out of memory issue on test set, here are some things that might work for you\n1. Try to use optimal data types for features\n2. Reduce number of features\n3. Batch wise predictions i.e. load only a set of series_id at once and predict\n4. Submit only inference notebook, try saving your model to a pkl file maybe, it also reduces the evaluation time\n\nHere is a code for loading one series at a time and predicting\n[Tackling out of memory issue on test set](https://www.kaggle.com/code/anilreddyvv/tackling-out-of-memory-issue-on-test-set?scriptVersionId=142873650)",
    "2440560": "Impressive. For the 4th point I just noticed that we can actually upload our pretrained model \"as a dataset\" .\n\nExample, [TabPFN model](https://www.kaggle.com/datasets/carlmcbrideellis/tabpfn-019-whl)",
    "2440583": "Yes @rimbax, I am also using model as dataset for submissions"
  },
  "source": "meta"
}