{
  "id": 407500,
  "title": "My RAM is limited after using cv2 to load images, How can I do to avoid this problem?",
  "url": "/competitions/benetech-making-graphs-accessible/discussion/407500",
  "author_name": "#1BuBu",
  "post_date": "2023-05-06T18:11:59.672000",
  "votes": 1,
  "comment_count": 6,
  "views": 0,
  "content": "<p>I use cv2 to load 60000+ images, It needs lots of RAM and My kernel will restart after exceed the RAM limit (13GB).</p>",
  "messages": [
    {
      "id": 2248437,
      "postDate": "2023-05-06T21:01:05.117Z",
      "content": "<p><a href=\"https://www.kaggle.com/yicheung1\" target=\"_blank\">@yicheung1</a> <br>\nI also got the same problem but if you have money you can buy ram on google colab <br>\nIf you have large data set try to reduce means use 50% of the data set to train your model some it gives result </p>",
      "rawMarkdown": "@yicheung1 \nI also got the same problem but if you have money you can buy ram on google colab \nIf you have large data set try to reduce means use 50% of the data set to train your model some it gives result ",
      "votes": 1,
      "replies": [
        {
          "id": 2251014,
          "postDate": "2023-05-09T02:13:12.770Z",
          "content": "<p>Thank you very much and I use less data finally.</p>",
          "rawMarkdown": "Thank you very much and I use less data finally."
        }
      ]
    },
    {
      "id": 2248324,
      "postDate": "2023-05-06T18:11:59.673Z",
      "content": "<p>I use cv2 to load 60000+ images, It needs lots of RAM and My kernel will restart after exceed the RAM limit (13GB).</p>",
      "rawMarkdown": "I use cv2 to load 60000+ images, It needs lots of RAM and My kernel will restart after exceed the RAM limit (13GB).",
      "votes": 1
    },
    {
      "id": 2248336,
      "postDate": "2023-05-06T18:30:08.563Z",
      "content": "<p>Nothing to do about that. You need more RAM. 13GB is not very much, for images you need much more. Now you can try GoogleColab but I doubt you will have more RAM. What you can do is parsing your training on less images. Try to get a balanced distribution on the small training dataset, though.<br>\nGood luck!</p>",
      "rawMarkdown": "Nothing to do about that. You need more RAM. 13GB is not very much, for images you need much more. Now you can try GoogleColab but I doubt you will have more RAM. What you can do is parsing your training on less images. Try to get a balanced distribution on the small training dataset, though.\nGood luck!",
      "replies": [
        {
          "id": 2251017,
          "postDate": "2023-05-09T02:13:53.310Z",
          "content": "<p>Thanks, and I will try ColabPro.</p>",
          "rawMarkdown": "Thanks, and I will try ColabPro."
        },
        {
          "id": 2254703,
          "postDate": "2023-05-11T07:15:53.543Z",
          "content": "<p>I am sorry to bother you again, I am using Colab pro now. But as you know, it's a code competition and I can just submit notebook. Do you know some methods to submit colab notebook?</p>",
          "rawMarkdown": "I am sorry to bother you again, I am using Colab pro now. But as you know, it's a code competition and I can just submit notebook. Do you know some methods to submit colab notebook?"
        }
      ]
    },
    {
      "id": 2265929,
      "postDate": "2023-05-19T15:47:01.620Z",
      "content": "<p>I wish you health</p>",
      "rawMarkdown": "I wish you health",
      "votes": 2,
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 2248437,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-05-06T21:01:05.117000",
      "content": "<p><a href=\"https://www.kaggle.com/yicheung1\" target=\"_blank\">@yicheung1</a> <br>\nI also got the same problem but if you have money you can buy ram on google colab <br>\nIf you have large data set try to reduce means use 50% of the data set to train your model some it gives result </p>",
      "votes": 1,
      "replies": [
        {
          "id": 2251014,
          "author_name": "#1BuBu",
          "author_url": "",
          "post_date": "2023-05-09T02:13:12.770000",
          "content": "<p>Thank you very much and I use less data finally.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2248336,
      "author_name": "Catadanna",
      "author_url": "",
      "post_date": "2023-05-06T18:30:08.563000",
      "content": "<p>Nothing to do about that. You need more RAM. 13GB is not very much, for images you need much more. Now you can try GoogleColab but I doubt you will have more RAM. What you can do is parsing your training on less images. Try to get a balanced distribution on the small training dataset, though.<br>\nGood luck!</p>",
      "votes": 0,
      "replies": [
        {
          "id": 2251017,
          "author_name": "#1BuBu",
          "author_url": "",
          "post_date": "2023-05-09T02:13:53.310000",
          "content": "<p>Thanks, and I will try ColabPro.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 2254703,
          "author_name": "#1BuBu",
          "author_url": "",
          "post_date": "2023-05-11T07:15:53.543000",
          "content": "<p>I am sorry to bother you again, I am using Colab pro now. But as you know, it's a code competition and I can just submit notebook. Do you know some methods to submit colab notebook?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2265929,
      "author_name": "Stiv job fan",
      "author_url": "",
      "post_date": "2023-05-19T15:47:01.620000",
      "content": "<p>I wish you health</p>",
      "votes": 2,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2248437": "@yicheung1 \nI also got the same problem but if you have money you can buy ram on google colab \nIf you have large data set try to reduce means use 50% of the data set to train your model some it gives result ",
    "2248324": "I use cv2 to load 60000+ images, It needs lots of RAM and My kernel will restart after exceed the RAM limit (13GB).",
    "2248336": "Nothing to do about that. You need more RAM. 13GB is not very much, for images you need much more. Now you can try GoogleColab but I doubt you will have more RAM. What you can do is parsing your training on less images. Try to get a balanced distribution on the small training dataset, though.\nGood luck!",
    "2265929": "I wish you health"
  }
}