{
  "id": 438290,
  "title": "OOM submission & size of the hidden test set?",
  "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/438290",
  "author_name": "",
  "post_date": "2023-09-10T11:48:51.202925200Z",
  "votes": 4,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I keep getting <code>Notebook Out of Memory</code> message when doing a submission.</p>\n<p>It started to get OOM when I crafted 14 features (all with dtype of float), previously it was fine with 10 features.</p>\n<p>I tried to delete variables with high-usage memory during inference of the test set (e.g. <code>del(train)</code>), but nothing changed.</p>\n<p>Questions:</p>\n<ol>\n<li>Any suggestion to solve this OOM issue?</li>\n<li>From the product side, is it possible to make unsuccessful submission not being counted from the daily submissions limit?</li>\n<li>Can participant know more of details about the hidden test set? At least the size in MB or the number of rows, unique series ID, or the statistics of the timesteps.</li>\n</ol>\n<p>I think it will be much helpful for participants to calculate/project their notebook capacity for the submission. Thanks a lot.</p>\n<p>EDIT 1 - 23/09/10 19:07+7:<br>\nCan the participant know the size of the hidden test set <em>for both public and private LB</em>? I mean, if one's submission is successful in public LB, it is not necessarily true in the private LB since the hidden test size is 3x larger than in the public LB (75% = 25% x 3).</p>",
  "messages": [
    {
      "id": "2431767",
      "postDate": "09/10/2023 11:48:51",
      "content": "<p>I keep getting <code>Notebook Out of Memory</code> message when doing a submission.</p>\n<p>It started to get OOM when I crafted 14 features (all with dtype of float), previously it was fine with 10 features.</p>\n<p>I tried to delete variables with high-usage memory during inference of the test set (e.g. <code>del(train)</code>), but nothing changed.</p>\n<p>Questions:</p>\n<ol>\n<li>Any suggestion to solve this OOM issue?</li>\n<li>From the product side, is it possible to make unsuccessful submission not being counted from the daily submissions limit?</li>\n<li>Can participant know more of details about the hidden test set? At least the size in MB or the number of rows, unique series ID, or the statistics of the timesteps.</li>\n</ol>\n<p>I think it will be much helpful for participants to calculate/project their notebook capacity for the submission. Thanks a lot.</p>\n<p>EDIT 1 - 23/09/10 19:07+7:<br>\nCan the participant know the size of the hidden test set <em>for both public and private LB</em>? I mean, if one's submission is successful in public LB, it is not necessarily true in the private LB since the hidden test size is 3x larger than in the public LB (75% = 25% x 3).</p>",
      "rawMarkdown": "I keep getting `Notebook Out of Memory` message when doing a submission.\n\nIt started to get OOM when I crafted 14 features (all with dtype of float), previously it was fine with 10 features.\n\nI tried to delete variables with high-usage memory during inference of the test set (e.g. `del(train)`), but nothing changed.\n\nQuestions:\n1. Any suggestion to solve this OOM issue?\n2. From the product side, is it possible to make unsuccessful submission not being counted from the daily submissions limit?\n3. Can participant know more of details about the hidden test set? At least the size in MB or the number of rows, unique series ID, or the statistics of the timesteps.\n\nI think it will be much helpful for participants to calculate/project their notebook capacity for the submission. Thanks a lot.\n\nEDIT 1 - 23/09/10 19:07+7:\nCan the participant know the size of the hidden test set *for both public and private LB*? I mean, if one's submission is successful in public LB, it is not necessarily true in the private LB since the hidden test size is 3x larger than in the public LB (75% = 25% x 3).",
      "votes": null
    },
    {
      "id": "2432471",
      "postDate": "09/10/2023 21:17:57",
      "content": "<p>Hi, I have the same problem, when I submit I get OOM error. Tried gc.collect(), del train, not using certain features, but nothing worked 🧐</p>",
      "rawMarkdown": "Hi, I have the same problem, when I submit I get OOM error. Tried gc.collect(), del train, not using certain features, but nothing worked 🧐",
      "votes": null
    },
    {
      "id": "2432715",
      "postDate": "09/11/2023 05:01:46",
      "content": "<p>To avoid the problem of a large hidden test set, perhaps streaming will ensure you stay within memory limits.</p>",
      "rawMarkdown": "To avoid the problem of a large hidden test set, perhaps streaming will ensure you stay within memory limits.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2432471,
      "author_name": "josipvrdoljak",
      "author_url": "",
      "post_date": "09/10/2023 21:17:57",
      "content": "<p>Hi, I have the same problem, when I submit I get OOM error. Tried gc.collect(), del train, not using certain features, but nothing worked 🧐</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2432715,
      "author_name": "jerryqu",
      "author_url": "",
      "post_date": "09/11/2023 05:01:46",
      "content": "<p>To avoid the problem of a large hidden test set, perhaps streaming will ensure you stay within memory limits.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2431767": "I keep getting `Notebook Out of Memory` message when doing a submission.\n\nIt started to get OOM when I crafted 14 features (all with dtype of float), previously it was fine with 10 features.\n\nI tried to delete variables with high-usage memory during inference of the test set (e.g. `del(train)`), but nothing changed.\n\nQuestions:\n1. Any suggestion to solve this OOM issue?\n2. From the product side, is it possible to make unsuccessful submission not being counted from the daily submissions limit?\n3. Can participant know more of details about the hidden test set? At least the size in MB or the number of rows, unique series ID, or the statistics of the timesteps.\n\nI think it will be much helpful for participants to calculate/project their notebook capacity for the submission. Thanks a lot.\n\nEDIT 1 - 23/09/10 19:07+7:\nCan the participant know the size of the hidden test set *for both public and private LB*? I mean, if one's submission is successful in public LB, it is not necessarily true in the private LB since the hidden test size is 3x larger than in the public LB (75% = 25% x 3).",
    "2432471": "Hi, I have the same problem, when I submit I get OOM error. Tried gc.collect(), del train, not using certain features, but nothing worked 🧐",
    "2432715": "To avoid the problem of a large hidden test set, perhaps streaming will ensure you stay within memory limits."
  },
  "source": "meta"
}