One of the images in the stage2 test set is grayscale and does not have a channel dimension. So if you do len(image.shape), you'll get 2 as the answer rather than 3 which is the case for all the others. Also this image is read in as np.uint16 rather than np.uint8. I've had to make some changes in my code to deal with this, so I expect others might as well.
", "messages": [ { "id": "312630", "postDate": "04/12/2018 04:17:17", "content": "One of the images in the stage2 test set is grayscale and does not have a channel dimension. So if you do len(image.shape), you'll get 2 as the answer rather than 3 which is the case for all the others. Also this image is read in as np.uint16 rather than np.uint8. I've had to make some changes in my code to deal with this, so I expect others might as well.
", "rawMarkdown": "One of the images in the stage2 test set is grayscale and does not have a channel dimension. So if you do len(image.shape), you'll get 2 as the answer rather than 3 which is the case for all the others. Also this image is read in as np.uint16 rather than np.uint8. I've had to make some changes in my code to deal with this, so I expect others might as well.", "votes": null }, { "id": "312685", "postDate": "04/12/2018 06:32:18", "content": "@Robert thanks for letting us know. I hope this belongs to the \"non-scientific alternation\" criteria and we can adapt our code to those non png and non 3 dimensional data.
", "rawMarkdown": "Robert thanks for letting us know. I hope this belongs to the \"non-scientific alternation\" criteria and we can adapt our code to those non png and non 3 dimensional data.", "votes": null }, { "id": "312713", "postDate": "04/12/2018 07:12:47", "content": "Yes, it's 646f5e00a2db3add97fb80a83ef3c07edd1b17b1b0d47c2bd650cdcab9f322c0
", "rawMarkdown": "Yes, it's 646f5e00a2db3add97fb80a83ef3c07edd1b17b1b0d47c2bd650cdcab9f322c0", "votes": null }, { "id": "312723", "postDate": "04/12/2018 07:33:32", "content": "perfect, thank you, I'll wait a little bit to see all problematic test images being reported. Have real work and life to catch up with after the end of the first stage. :)
", "rawMarkdown": "perfect, thank you, I'll wait a little bit to see all problematic test images being reported. Have real work and life to catch up with after the end of the first stage. :)", "votes": null }, { "id": "312871", "postDate": "04/12/2018 13:42:42", "content": "I think it would be ok to make changes to accomodate this image. The changes wouldn't affect the model, but allow someone to use the image. If you didn't then your code might not be able to handle the image because of the array shape, the stage1 train set didn't have any image shapes like this.
", "rawMarkdown": "I think it would be ok to make changes to accomodate this image. The changes wouldn't affect the model, but allow someone to use the image. If you didn't then your code might not be able to handle the image because of the array shape, the stage1 train set didn't have any image shapes like this.", "votes": null }, { "id": "312874", "postDate": "04/12/2018 13:45:29", "content": "Yep. This was a mistake. It should be RGB.
", "rawMarkdown": "Yep. This was a mistake. It should be RGB.", "votes": null }, { "id": "312918", "postDate": "04/12/2018 14:32:23", "content": "Thanks for the confirmation, @Allen.
", "rawMarkdown": "Thanks for the confirmation, @Allen.", "votes": null }, { "id": "313185", "postDate": "04/12/2018 21:48:36", "content": "I used cv2.imread to read this image. It indeed gives a warning like\" PLTE: ignored in grayscale PNG\". Otherwise, the image.shape has 3 channels and the type is np.unit8. It seems no worries if the CV2 is used.
", "rawMarkdown": "I used cv2.imread to read this image. It indeed gives a warning like\" PLTE: ignored in grayscale PNG\". Otherwise, the image.shape has 3 channels and the type is np.unit8. It seems no worries if the CV2 is used.", "votes": null }, { "id": "313190", "postDate": "04/12/2018 22:01:41", "content": "@terracotta, we didn't hit any problems or warnings reading stage 2 test images, because we used cv2.imread as well, because openCV is way faster than skimage, so no code change is needed for us to read those files in the jpeg/tf format, I guess openCV is smart enough to figure out the header of a file
\n\nimg = cv2.imread(os.path.join(base_path, image_name + '.png'))[:,:,:num_channels]
Hi Robert, thanks for posting this issue. I had the same issue. But as by @Nicole Finnie I didn't encounter this issue while I used OpenCV. However, I find few images are empty like the one with the ID : 'db45ed183873b71ccae114645660e026cdefa0d5dda09718203f9a18530b6bbc' . Is it a mistake or done on purpose to test the algorithm?
", "rawMarkdown": "Hi Robert, thanks for posting this issue. I had the same issue. But as by @Nicole Finnie I didn't encounter this issue while I used OpenCV. However, I find few images are empty like the one with the ID : 'db45ed183873b71ccae114645660e026cdefa0d5dda09718203f9a18530b6bbc' . Is it a mistake or done on purpose to test the algorithm?", "votes": null }, { "id": "313959", "postDate": "04/14/2018 07:02:28", "content": "@VikramanK, using openCV to read the file you'd only hit a warning. not sure why black images are there, I haven't gotten that point yet. Your RLE code should just add this imageID followed by an empty string. And I took a look at the image in question, it's not black, you can equalize the image to see it better.
", "rawMarkdown": "VikramanK, using openCV to read the file you'd only hit a warning. not sure why black images are there, I haven't gotten that point yet. Your RLE code should just add this imageID followed by an empty string. And I took a look at the image in question, it's not black, you can equalize the image to see it better.", "votes": null }, { "id": "313961", "postDate": "04/14/2018 07:10:26", "content": "@Nicole Finnie, I crosschecked with imread function from Skimage for that particular test ID I can confirm to you that it is a black image. As expected my model predicted an empty image. However during RLE encoding part, that particular image ID was not included. If you encounter this situation you can see this useful post by @See https://www.kaggle.com/c/data-science-bowl-2018/discussion/54295#313957
", "rawMarkdown": "Nicole Finnie, I crosschecked with imread function from Skimage for that particular test ID I can confirm to you that it is a black image. As expected my model predicted an empty image. However during RLE encoding part, that particular image ID was not included. If you encounter this situation you can see this useful post by @See https://www.kaggle.com/c/data-science-bowl-2018/discussion/54295#313957", "votes": null }, { "id": "313967", "postDate": "04/14/2018 07:27:43", "content": "@VikramanK, I don't know how you cross checked using imread, it's not black. But your model may not be able to detect any nuclei in it. You can see the histogram here.\nhttps://drive.google.com/open?id=1wxHXR2OeEvdEQdNk9sX6OX0sadiFjbOV
@Nicole Finnie,\nThanks for sharing the histogram. I think I am wrong. I just plot the images and compared them. I randomly chose around 30 images. These images were completely dark to my naked eye! As you say, my model could not identify any nuclei in it! \nThanks again :)
", "rawMarkdown": "Nicole Finnie,\nThanks for sharing the histogram. I think I am wrong. I just plot the images and compared them. I randomly chose around 30 images. These images were completely dark to my naked eye! As you say, my model could not identify any nuclei in it! \nThanks again :)", "votes": null }, { "id": "314036", "postDate": "04/14/2018 13:15:38", "content": "Good catch! I am using skimage and I was pulling my hair out trying to figure out what was going wrong! My input script doesn't flag the exact image that is causing the issue.
", "rawMarkdown": "Good catch! I am using skimage and I was pulling my hair out trying to figure out what was going wrong! My input script doesn't flag the exact image that is causing the issue.", "votes": null }, { "id": "521028", "postDate": "04/22/2019 07:40:23", "content": "I edit this image to RGB format with ID: 646f5e00a2db3add97fb80a83ef3c07edd1b17b1b0d47c2bd650cdcab9f322c0.\nMaybe direct saving it and replacing the original one can help fix it.
", "rawMarkdown": "I edit this image to RGB format with ID: 646f5e00a2db3add97fb80a83ef3c07edd1b17b1b0d47c2bd650cdcab9f322c0.\nMaybe direct saving it and replacing the original one can help fix it.", "votes": null } ], "comments": [ { "id": 312685, "author_name": "nicolefinnie", "author_url": "", "post_date": "04/12/2018 06:32:18", "content": "@Robert thanks for letting us know. I hope this belongs to the \"non-scientific alternation\" criteria and we can adapt our code to those non png and non 3 dimensional data.
", "votes": null, "replies": [ { "id": 312871, "author_name": "robertkag", "author_url": "", "post_date": "04/12/2018 13:42:42", "content": "I think it would be ok to make changes to accomodate this image. The changes wouldn't affect the model, but allow someone to use the image. If you didn't then your code might not be able to handle the image because of the array shape, the stage1 train set didn't have any image shapes like this.
", "votes": null, "replies": [] }, { "id": 312874, "author_name": "agoodman", "author_url": "", "post_date": "04/12/2018 13:45:29", "content": "Yep. This was a mistake. It should be RGB.
", "votes": null, "replies": [] }, { "id": 312918, "author_name": "nicolefinnie", "author_url": "", "post_date": "04/12/2018 14:32:23", "content": "Thanks for the confirmation, @Allen.
", "votes": null, "replies": [] }, { "id": 313185, "author_name": "ybwu01", "author_url": "", "post_date": "04/12/2018 21:48:36", "content": "I used cv2.imread to read this image. It indeed gives a warning like\" PLTE: ignored in grayscale PNG\". Otherwise, the image.shape has 3 channels and the type is np.unit8. It seems no worries if the CV2 is used.
", "votes": null, "replies": [] }, { "id": 313190, "author_name": "nicolefinnie", "author_url": "", "post_date": "04/12/2018 22:01:41", "content": "@terracotta, we didn't hit any problems or warnings reading stage 2 test images, because we used cv2.imread as well, because openCV is way faster than skimage, so no code change is needed for us to read those files in the jpeg/tf format, I guess openCV is smart enough to figure out the header of a file
\n\nimg = cv2.imread(os.path.join(base_path, image_name + '.png'))[:,:,:num_channels]
Hi Robert, thanks for posting this issue. I had the same issue. But as by @Nicole Finnie I didn't encounter this issue while I used OpenCV. However, I find few images are empty like the one with the ID : 'db45ed183873b71ccae114645660e026cdefa0d5dda09718203f9a18530b6bbc' . Is it a mistake or done on purpose to test the algorithm?
", "votes": null, "replies": [] }, { "id": 313959, "author_name": "nicolefinnie", "author_url": "", "post_date": "04/14/2018 07:02:28", "content": "@VikramanK, using openCV to read the file you'd only hit a warning. not sure why black images are there, I haven't gotten that point yet. Your RLE code should just add this imageID followed by an empty string. And I took a look at the image in question, it's not black, you can equalize the image to see it better.
", "votes": null, "replies": [] }, { "id": 313961, "author_name": "vikramank", "author_url": "", "post_date": "04/14/2018 07:10:26", "content": "@Nicole Finnie, I crosschecked with imread function from Skimage for that particular test ID I can confirm to you that it is a black image. As expected my model predicted an empty image. However during RLE encoding part, that particular image ID was not included. If you encounter this situation you can see this useful post by @See https://www.kaggle.com/c/data-science-bowl-2018/discussion/54295#313957
", "votes": null, "replies": [] }, { "id": 313967, "author_name": "nicolefinnie", "author_url": "", "post_date": "04/14/2018 07:27:43", "content": "@VikramanK, I don't know how you cross checked using imread, it's not black. But your model may not be able to detect any nuclei in it. You can see the histogram here.\nhttps://drive.google.com/open?id=1wxHXR2OeEvdEQdNk9sX6OX0sadiFjbOV
@Nicole Finnie,\nThanks for sharing the histogram. I think I am wrong. I just plot the images and compared them. I randomly chose around 30 images. These images were completely dark to my naked eye! As you say, my model could not identify any nuclei in it! \nThanks again :)
", "votes": null, "replies": [] } ] }, { "id": 312713, "author_name": "mpware", "author_url": "", "post_date": "04/12/2018 07:12:47", "content": "Yes, it's 646f5e00a2db3add97fb80a83ef3c07edd1b17b1b0d47c2bd650cdcab9f322c0
", "votes": null, "replies": [ { "id": 312723, "author_name": "nicolefinnie", "author_url": "", "post_date": "04/12/2018 07:33:32", "content": "perfect, thank you, I'll wait a little bit to see all problematic test images being reported. Have real work and life to catch up with after the end of the first stage. :)
", "votes": null, "replies": [] } ] }, { "id": 314036, "author_name": "dstjhb", "author_url": "", "post_date": "04/14/2018 13:15:38", "content": "Good catch! I am using skimage and I was pulling my hair out trying to figure out what was going wrong! My input script doesn't flag the exact image that is causing the issue.
", "votes": null, "replies": [] }, { "id": 521028, "author_name": "renicorn", "author_url": "", "post_date": "04/22/2019 07:40:23", "content": "I edit this image to RGB format with ID: 646f5e00a2db3add97fb80a83ef3c07edd1b17b1b0d47c2bd650cdcab9f322c0.\nMaybe direct saving it and replacing the original one can help fix it.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "312630": "One of the images in the stage2 test set is grayscale and does not have a channel dimension. So if you do len(image.shape), you'll get 2 as the answer rather than 3 which is the case for all the others. Also this image is read in as np.uint16 rather than np.uint8. I've had to make some changes in my code to deal with this, so I expect others might as well.", "312685": "Robert thanks for letting us know. I hope this belongs to the \"non-scientific alternation\" criteria and we can adapt our code to those non png and non 3 dimensional data.", "312713": "Yes, it's 646f5e00a2db3add97fb80a83ef3c07edd1b17b1b0d47c2bd650cdcab9f322c0", "312723": "perfect, thank you, I'll wait a little bit to see all problematic test images being reported. Have real work and life to catch up with after the end of the first stage. :)", "312871": "I think it would be ok to make changes to accomodate this image. The changes wouldn't affect the model, but allow someone to use the image. If you didn't then your code might not be able to handle the image because of the array shape, the stage1 train set didn't have any image shapes like this.", "312874": "Yep. This was a mistake. It should be RGB.", "312918": "Thanks for the confirmation, @Allen.", "313185": "I used cv2.imread to read this image. It indeed gives a warning like\" PLTE: ignored in grayscale PNG\". Otherwise, the image.shape has 3 channels and the type is np.unit8. It seems no worries if the CV2 is used.", "313190": "terracotta, we didn't hit any problems or warnings reading stage 2 test images, because we used cv2.imread as well, because openCV is way faster than skimage, so no code change is needed for us to read those files in the jpeg/tf format, I guess openCV is smart enough to figure out the header of a file\n\n`img = cv2.imread(os.path.join(base_path, image_name + '.png'))[:,:,:num_channels]`", "313924": "Hi Robert, thanks for posting this issue. I had the same issue. But as by @Nicole Finnie I didn't encounter this issue while I used OpenCV. However, I find few images are empty like the one with the ID : 'db45ed183873b71ccae114645660e026cdefa0d5dda09718203f9a18530b6bbc' . Is it a mistake or done on purpose to test the algorithm?", "313959": "VikramanK, using openCV to read the file you'd only hit a warning. not sure why black images are there, I haven't gotten that point yet. Your RLE code should just add this imageID followed by an empty string. And I took a look at the image in question, it's not black, you can equalize the image to see it better.", "313961": "Nicole Finnie, I crosschecked with imread function from Skimage for that particular test ID I can confirm to you that it is a black image. As expected my model predicted an empty image. However during RLE encoding part, that particular image ID was not included. If you encounter this situation you can see this useful post by @See https://www.kaggle.com/c/data-science-bowl-2018/discussion/54295#313957", "313967": "VikramanK, I don't know how you cross checked using `imread`, it's not black. But your model may not be able to detect any nuclei in it. You can see the histogram here.\nhttps://drive.google.com/open?id=1wxHXR2OeEvdEQdNk9sX6OX0sadiFjbOV", "313970": "Nicole Finnie,\nThanks for sharing the histogram. I think I am wrong. I just plot the images and compared them. I randomly chose around 30 images. These images were completely dark to my naked eye! As you say, my model could not identify any nuclei in it! \nThanks again :)", "314036": "Good catch! I am using skimage and I was pulling my hair out trying to figure out what was going wrong! My input script doesn't flag the exact image that is causing the issue.", "521028": "I edit this image to RGB format with ID: 646f5e00a2db3add97fb80a83ef3c07edd1b17b1b0d47c2bd650cdcab9f322c0.\nMaybe direct saving it and replacing the original one can help fix it." }, "source": "meta" }