{
  "id": 331043,
  "title": "Kernal Restart without Error..... !!",
  "url": "/competitions/be-honest-what-do-you-think-of-this-image/discussion/331043",
  "author_name": "",
  "post_date": "2022-06-15T14:15:12.147560800Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I have Loaded 20k image with 512x512 Resolutation in tensor Format….. Succesfully Loaded but during model.fit() GPU and RAM are Free but kernal Restart without error</p>\n<p>My Notebook,<br>\n<a href=\"https://www.kaggle.com/code/muki2003/image-interest-evaluator-tensorflow\" target=\"_blank\">https://www.kaggle.com/code/muki2003/image-interest-evaluator-tensorflow</a></p>",
  "messages": [
    {
      "id": "1821414",
      "postDate": "06/15/2022 14:15:12",
      "content": "<p>I have Loaded 20k image with 512x512 Resolutation in tensor Format….. Succesfully Loaded but during model.fit() GPU and RAM are Free but kernal Restart without error</p>\n<p>My Notebook,<br>\n<a href=\"https://www.kaggle.com/code/muki2003/image-interest-evaluator-tensorflow\" target=\"_blank\">https://www.kaggle.com/code/muki2003/image-interest-evaluator-tensorflow</a></p>",
      "rawMarkdown": "I have Loaded 20k image with 512x512 Resolutation in tensor Format..... Succesfully Loaded but during model.fit() GPU and RAM are Free but kernal Restart without error\n\nMy Notebook,\nhttps://www.kaggle.com/code/muki2003/image-interest-evaluator-tensorflow",
      "votes": null
    },
    {
      "id": "1822069",
      "postDate": "06/16/2022 03:51:07",
      "content": "<p>There could me many reasons for this, <br>\nbut the most common that i face when doing Image classification are , </p>\n<ol>\n<li>The RAM available is not enough.</li>\n</ol>\n<p>If your images are stored as a numpy array, then convert it to a generator as follows </p>\n<pre><code>train_gen = ImageDataGenerator().flow_from_directory(PATH_TO_TRAIN_IMAGES, target_size=(256, 256), batch_size=1)\n</code></pre>\n<p>The batch size of the above code is set to 1 so it loads the image in batches of 1 .</p>\n<p>Just use your training set to see if it fits . You will get to know if the RAM you have is not enough . If this does not work, proceed to the next step .</p>\n<ol>\n<li>If the available RAM is not the issue, then try to check if the weights of the model that you have trained before is a problem.</li>\n</ol>\n<pre><code>for i, layer in enumerate(model.layers):\n    print(i, layer.name)\n</code></pre>\n<p>Try to see the above output. You might notice some hidden layers that have been trained with certain weights but were not removed before you save the new trained model. Try to remove those hidden layers and retrain the model with your new dataset. </p>\n<ol>\n<li>Last but not least, even after completing the above 2 steps , you still face the issue , please comment and i'll try to give you a hand.</li>\n</ol>",
      "rawMarkdown": "There could me many reasons for this, \nbut the most common that i face when doing Image classification are , \n\n1. The RAM available is not enough.\n\nIf your images are stored as a numpy array, then convert it to a generator as follows \n\n```\ntrain_gen = ImageDataGenerator().flow_from_directory(PATH_TO_TRAIN_IMAGES, target_size=(256, 256), batch_size=1)\n```\n\nThe batch size of the above code is set to 1 so it loads the image in batches of 1 .\n\nJust use your training set to see if it fits . You will get to know if the RAM you have is not enough . If this does not work, proceed to the next step .\n\n2. If the available RAM is not the issue, then try to check if the weights of the model that you have trained before is a problem.\n\n```\nfor i, layer in enumerate(model.layers):\n    print(i, layer.name)\n```\n\nTry to see the above output. You might notice some hidden layers that have been trained with certain weights but were not removed before you save the new trained model. Try to remove those hidden layers and retrain the model with your new dataset. \n\n3. Last but not least, even after completing the above 2 steps , you still face the issue , please comment and i'll try to give you a hand.",
      "votes": null
    },
    {
      "id": "1822495",
      "postDate": "06/16/2022 12:48:54",
      "content": "<p>Thx for this Information,</p>\n<ol>\n<li>ImageDataGenerator().flow_from_directory() I can't Use This Becoz I need to Predict the Interest of the Image not a classification problem</li>\n<li>Model weights are good….</li>\n</ol>\n<p>next(iter(train_ds)) also restart the kernal but train_ds.sample(1) works fine……</p>",
      "rawMarkdown": "Thx for this Information,\n\n1. ImageDataGenerator().flow_from_directory() I can't Use This Becoz I need to Predict the Interest of the Image not a classification problem\n2. Model weights are good....\n\nnext(iter(train_ds)) also restart the kernal but train_ds.sample(1) works fine......",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1822069,
      "author_name": "thedevastator",
      "author_url": "",
      "post_date": "06/16/2022 03:51:07",
      "content": "<p>There could me many reasons for this, <br>\nbut the most common that i face when doing Image classification are , </p>\n<ol>\n<li>The RAM available is not enough.</li>\n</ol>\n<p>If your images are stored as a numpy array, then convert it to a generator as follows </p>\n<pre><code>train_gen = ImageDataGenerator().flow_from_directory(PATH_TO_TRAIN_IMAGES, target_size=(256, 256), batch_size=1)\n</code></pre>\n<p>The batch size of the above code is set to 1 so it loads the image in batches of 1 .</p>\n<p>Just use your training set to see if it fits . You will get to know if the RAM you have is not enough . If this does not work, proceed to the next step .</p>\n<ol>\n<li>If the available RAM is not the issue, then try to check if the weights of the model that you have trained before is a problem.</li>\n</ol>\n<pre><code>for i, layer in enumerate(model.layers):\n    print(i, layer.name)\n</code></pre>\n<p>Try to see the above output. You might notice some hidden layers that have been trained with certain weights but were not removed before you save the new trained model. Try to remove those hidden layers and retrain the model with your new dataset. </p>\n<ol>\n<li>Last but not least, even after completing the above 2 steps , you still face the issue , please comment and i'll try to give you a hand.</li>\n</ol>",
      "votes": null,
      "replies": [
        {
          "id": 1822495,
          "author_name": "muki2003",
          "author_url": "",
          "post_date": "06/16/2022 12:48:54",
          "content": "<p>Thx for this Information,</p>\n<ol>\n<li>ImageDataGenerator().flow_from_directory() I can't Use This Becoz I need to Predict the Interest of the Image not a classification problem</li>\n<li>Model weights are good….</li>\n</ol>\n<p>next(iter(train_ds)) also restart the kernal but train_ds.sample(1) works fine……</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1821414": "I have Loaded 20k image with 512x512 Resolutation in tensor Format..... Succesfully Loaded but during model.fit() GPU and RAM are Free but kernal Restart without error\n\nMy Notebook,\nhttps://www.kaggle.com/code/muki2003/image-interest-evaluator-tensorflow",
    "1822069": "There could me many reasons for this, \nbut the most common that i face when doing Image classification are , \n\n1. The RAM available is not enough.\n\nIf your images are stored as a numpy array, then convert it to a generator as follows \n\n```\ntrain_gen = ImageDataGenerator().flow_from_directory(PATH_TO_TRAIN_IMAGES, target_size=(256, 256), batch_size=1)\n```\n\nThe batch size of the above code is set to 1 so it loads the image in batches of 1 .\n\nJust use your training set to see if it fits . You will get to know if the RAM you have is not enough . If this does not work, proceed to the next step .\n\n2. If the available RAM is not the issue, then try to check if the weights of the model that you have trained before is a problem.\n\n```\nfor i, layer in enumerate(model.layers):\n    print(i, layer.name)\n```\n\nTry to see the above output. You might notice some hidden layers that have been trained with certain weights but were not removed before you save the new trained model. Try to remove those hidden layers and retrain the model with your new dataset. \n\n3. Last but not least, even after completing the above 2 steps , you still face the issue , please comment and i'll try to give you a hand.",
    "1822495": "Thx for this Information,\n\n1. ImageDataGenerator().flow_from_directory() I can't Use This Becoz I need to Predict the Interest of the Image not a classification problem\n2. Model weights are good....\n\nnext(iter(train_ds)) also restart the kernal but train_ds.sample(1) works fine......"
  },
  "source": "meta"
}