{
  "id": 491399,
  "title": "Validation set too large? ",
  "url": "/competitions/byu-locating-bacterial-flagellar-motors-v2/discussion/491399",
  "author_name": "",
  "post_date": "2024-04-05T18:17:23.690423600Z",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I'm using the same framework I did for the other Kaggle competition and it keeps giving me this error when I try to make predictions: Your notebook tried to allocate more memory than is available. It has restarted. </p>\n<p>I'm working in a Kaggle notebook through the competition (so the data is already uploaded)</p>",
  "messages": [
    {
      "id": "2737362",
      "postDate": "04/05/2024 18:17:23",
      "content": "<p>I'm using the same framework I did for the other Kaggle competition and it keeps giving me this error when I try to make predictions: Your notebook tried to allocate more memory than is available. It has restarted. </p>\n<p>I'm working in a Kaggle notebook through the competition (so the data is already uploaded)</p>",
      "rawMarkdown": "I'm using the same framework I did for the other Kaggle competition and it keeps giving me this error when I try to make predictions: Your notebook tried to allocate more memory than is available. It has restarted. \n\nI'm working in a Kaggle notebook through the competition (so the data is already uploaded)",
      "votes": null
    },
    {
      "id": "2737364",
      "postDate": "04/05/2024 18:18:23",
      "content": "<p>The validation set for the other competition had 439 images, this one has 1190</p>",
      "rawMarkdown": "The validation set for the other competition had 439 images, this one has 1190",
      "votes": null
    },
    {
      "id": "2737519",
      "postDate": "04/05/2024 19:39:12",
      "content": "<p>Good question! That likely has something to do with the server limits on Kaggle's end. Are you using the accelerator options Kaggle offers or just using CPU?</p>",
      "rawMarkdown": "Good question! That likely has something to do with the server limits on Kaggle's end. Are you using the accelerator options Kaggle offers or just using CPU?",
      "votes": null
    },
    {
      "id": "2737527",
      "postDate": "04/05/2024 19:43:58",
      "content": "<p>When exactly is the error occurring? Are you using the GPU or CPU? Try and find exactly where you are running out of memory. It is interesting to me that this happens when predicting and not when training. What is the batch size you are using?</p>",
      "rawMarkdown": "When exactly is the error occurring? Are you using the GPU or CPU? Try and find exactly where you are running out of memory. It is interesting to me that this happens when predicting and not when training. What is the batch size you are using?",
      "votes": null
    },
    {
      "id": "2737538",
      "postDate": "04/05/2024 19:47:48",
      "content": "<p>I don't think there is any problem with the size of the validation set. It just matters how you utilize the GPU/CPU memory allotted to you.  </p>",
      "rawMarkdown": "I don't think there is any problem with the size of the validation set. It just matters how you utilize the GPU/CPU memory allotted to you.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2737364,
      "author_name": "nathanieldriggs",
      "author_url": "",
      "post_date": "04/05/2024 18:18:23",
      "content": "<p>The validation set for the other competition had 439 images, this one has 1190</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2737519,
      "author_name": "elonsdale",
      "author_url": "",
      "post_date": "04/05/2024 19:39:12",
      "content": "<p>Good question! That likely has something to do with the server limits on Kaggle's end. Are you using the accelerator options Kaggle offers or just using CPU?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2737527,
      "author_name": "braxtonowens",
      "author_url": "",
      "post_date": "04/05/2024 19:43:58",
      "content": "<p>When exactly is the error occurring? Are you using the GPU or CPU? Try and find exactly where you are running out of memory. It is interesting to me that this happens when predicting and not when training. What is the batch size you are using?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2737538,
          "author_name": "braxtonowens",
          "author_url": "",
          "post_date": "04/05/2024 19:47:48",
          "content": "<p>I don't think there is any problem with the size of the validation set. It just matters how you utilize the GPU/CPU memory allotted to you.  </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2737362": "I'm using the same framework I did for the other Kaggle competition and it keeps giving me this error when I try to make predictions: Your notebook tried to allocate more memory than is available. It has restarted. \n\nI'm working in a Kaggle notebook through the competition (so the data is already uploaded)",
    "2737364": "The validation set for the other competition had 439 images, this one has 1190",
    "2737519": "Good question! That likely has something to do with the server limits on Kaggle's end. Are you using the accelerator options Kaggle offers or just using CPU?",
    "2737527": "When exactly is the error occurring? Are you using the GPU or CPU? Try and find exactly where you are running out of memory. It is interesting to me that this happens when predicting and not when training. What is the batch size you are using?",
    "2737538": "I don't think there is any problem with the size of the validation set. It just matters how you utilize the GPU/CPU memory allotted to you."
  },
  "source": "meta"
}