I'm trying to make a benchmark submission with a single image-sized mask for each image, while for some reason I'm getting an out of bounds exception:
\n\n\n\n\nIndex was outside the bounds of the array.
\n
Here's the python code producing the csv file.
\n\nfrom imageio import imread\nfrom glob import glob\n\nfolders = glob(\"../data/stage1_test/*\")\n\ndef FullImageMaskRLE(w, h): \n return \"1 {}\".format(w * h)\n\nprint(\"ImageId,EncodedPixels\")\nfor folder in folders:\n img_id = folder.split('/')[-1]\n img_file, = glob(folder + \"/images/*.png\")\n img_data = imread(img_file)\n print(\"{},{}\".format(img_id,\n FullImageMaskRLE(img_data.shape[1], img_data.shape[0])))\nAny ideas why this might be happening?
", "messages": [ { "id": "275677", "postDate": "01/29/2018 19:18:09", "content": "I'm trying to make a benchmark submission with a single image-sized mask for each image, while for some reason I'm getting an out of bounds exception:
\n\n\n\n\nIndex was outside the bounds of the array.
\n
Here's the python code producing the csv file.
\n\nfrom imageio import imread\nfrom glob import glob\n\nfolders = glob(\"../data/stage1_test/*\")\n\ndef FullImageMaskRLE(w, h): \n return \"1 {}\".format(w * h)\n\nprint(\"ImageId,EncodedPixels\")\nfor folder in folders:\n img_id = folder.split('/')[-1]\n img_file, = glob(folder + \"/images/*.png\")\n img_data = imread(img_file)\n print(\"{},{}\".format(img_id,\n FullImageMaskRLE(img_data.shape[1], img_data.shape[0])))\nAny ideas why this might be happening?
", "rawMarkdown": "I'm trying to make a benchmark submission with a single image-sized mask for each image, while for some reason I'm getting an out of bounds exception:\n\n> Index was outside the bounds of the array.\n\nHere's the python code producing the csv file.\n\n from imageio import imread\n from glob import glob\n \n folders = glob(\"../data/stage1_test/*\")\n \n def FullImageMaskRLE(w, h): \n return \"1 {}\".format(w * h)\n \n print(\"ImageId,EncodedPixels\")\n for folder in folders:\n img_id = folder.split('/')[-1]\n img_file, = glob(folder + \"/images/*.png\")\n img_data = imread(img_file)\n print(\"{},{}\".format(img_id,\n FullImageMaskRLE(img_data.shape[1], img_data.shape[0])))\n\nAny ideas why this might be happening?", "votes": null }, { "id": "275682", "postDate": "01/29/2018 19:34:47", "content": "Not sure, but maybe the start value should be 0 instead of 1
", "rawMarkdown": "Not sure, but maybe the start value should be 0 instead of 1", "votes": null }, { "id": "275683", "postDate": "01/29/2018 19:36:57", "content": "Well, it is stated on the 'Evaluation' page that it starts from 1. Also, I tried just posting the last scanline mask with this logic and it was accepted just fine...
", "rawMarkdown": "Well, it is stated on the 'Evaluation' page that it starts from 1. Also, I tried just posting the last scanline mask with this logic and it was accepted just fine...", "votes": null }, { "id": "275784", "postDate": "01/30/2018 01:08:05", "content": "I am getting the same error for multiple masks per image. Hope that someone from kaggle explain what it is or which entry gives that error.
", "rawMarkdown": "I am getting the same error for multiple masks per image. Hope that someone from kaggle explain what it is or which entry gives that error.", "votes": null }, { "id": "275841", "postDate": "01/30/2018 04:37:21", "content": "I was getting the same error because there was an issue with resizing (upsampling) mask images to original test image size.
", "rawMarkdown": "I was getting the same error because there was an issue with resizing (upsampling) mask images to original test image size.", "votes": null }, { "id": "275945", "postDate": "01/30/2018 09:43:32", "content": "I checked the mask sizes and they are similar to the original image size. Finally I decided to add all the masks together and use the RLE method used in the unet kernel and that submission worked.
", "rawMarkdown": "I checked the mask sizes and they are similar to the original image size. Finally I decided to add all the masks together and use the RLE method used in the unet kernel and that submission worked.", "votes": null }, { "id": "276245", "postDate": "01/31/2018 02:59:47", "content": "I'm under the impression that each column needs to be encoded separately, so a run length shouldn't be longer than the height of the image
", "rawMarkdown": "I'm under the impression that each column needs to be encoded separately, so a run length shouldn't be longer than the height of the image", "votes": null }, { "id": "276293", "postDate": "01/31/2018 05:34:53", "content": "Hmm, have you tried that (for a full-image mask)? For me it didn't work (for both (w, h) and (h, w)):
\n\n' '.join([str(i) + ' ' + str(w) for i in range(1, w * h, w)])\nunless there's a mistake here...
", "rawMarkdown": "Hmm, have you tried that (for a full-image mask)? For me it didn't work (for both (w, h) and (h, w)):\n\n\n ' '.join([str(i) + ' ' + str(w) for i in range(1, w * h, w)])\n\nunless there's a mistake here...", "votes": null }, { "id": "279125", "postDate": "02/07/2018 12:06:14", "content": "This happened when the submission contained a \"blank\"/missing EncodedPixels row followed by a non-blank row:
\n\n\nca20076870e8fb604e61802605a9ac45419c82dd3e23404c56c4869f9502a5ef,\nca20076870e8fb604e61802605a9ac45419c82dd3e23404c56c4869f9502a5ef,29153 1\n
how do you solve it?
", "rawMarkdown": "how do you solve it?", "votes": null }, { "id": "279602", "postDate": "02/08/2018 10:36:29", "content": "Could someone from the organizers pls react to this? I don't see how the code from the OP should produce a faulty submission. Unless I'm mistaken, this is clearly a bug in the evaluation code, which could easily lead to wrong scores in case, for example, some ground truth image(s) for the test set have wrong dimensions.
", "rawMarkdown": "Could someone from the organizers pls react to this? I don't see how the code from the OP should produce a faulty submission. Unless I'm mistaken, this is clearly a bug in the evaluation code, which could easily lead to wrong scores in case, for example, some ground truth image(s) for the test set have wrong dimensions.", "votes": null }, { "id": "279820", "postDate": "02/08/2018 19:24:34", "content": "I believe the error you encountered:
\n\nPython\nIndex was outside the bounds of the array.\n
is correct. It’s difficult to ascertain If there’s a problem without more context (e.g. FullImageMaskRLE’s body).
\n\nTry verifying that the run-length decoded label matrix doesn’t exceed the image’s shape.
", "rawMarkdown": "I believe the error you encountered:\n\n```Python\nIndex was outside the bounds of the array.\n```\n\nis correct. It’s difficult to ascertain If there’s a problem without more context (e.g. FullImageMaskRLE’s body).\n\nTry verifying that the run-length decoded label matrix doesn’t exceed the image’s shape.", "votes": null }, { "id": "279825", "postDate": "02/08/2018 19:34:17", "content": "I created a sample CSV (for one image). You might find it helpful:
\n\nhttps://gist.github.com/0x00b1/2dff1f78497b1bbe6fc5b19be1664c68
", "rawMarkdown": "I created a sample CSV (for one image). You might find it helpful:\n\nhttps://gist.github.com/0x00b1/2dff1f78497b1bbe6fc5b19be1664c68", "votes": null }, { "id": "279835", "postDate": "02/08/2018 20:01:10", "content": "The code in OP is a MWE, FullImageMaskRLE's body is at lines 6-7.
", "rawMarkdown": "The code in OP is a MWE, FullImageMaskRLE's body is at lines 6-7.", "votes": null }, { "id": "279839", "postDate": "02/08/2018 20:07:46", "content": "Thanks, but I'm not saying I cant submit anything. Submissions with more or less adequate masks work. I'm just saying a submission of type \"id,1 img_size\" doesn't.
", "rawMarkdown": "Thanks, but I'm not saying I cant submit anything. Submissions with more or less adequate masks work. I'm just saying a submission of type \"id,1 img_size\" doesn't.", "votes": null }, { "id": "280386", "postDate": "02/09/2018 21:29:06", "content": "So you’re asking why you can’t submit a bad mask (e.g. a mask that covers every pixel)? Why do you think this is a major problem? I’m not following.
", "rawMarkdown": "So you’re asking why you can’t submit a bad mask (e.g. a mask that covers every pixel)? Why do you think this is a major problem? I’m not following.", "votes": null }, { "id": "280607", "postDate": "02/10/2018 12:40:43", "content": "Well, it's only \"bad\" in terms of highlighting the nuclei, it's still a \"good\" mask from the point of the evaluation rules and it deserves to be evaluated without a problem. The fact that evaluation code chokes on it indicates there's a problem with the code. It may be an innocent problem only affecting a mask like this, or it may be a major problem affecting all the scores (for example if some ground truth for the test set has wrong dimensions).\nI'm surprised no one else seems to be concerned about this...
", "rawMarkdown": "Well, it's only \"bad\" in terms of highlighting the nuclei, it's still a \"good\" mask from the point of the evaluation rules and it deserves to be evaluated without a problem. The fact that evaluation code chokes on it indicates there's a problem with the code. It may be an innocent problem only affecting a mask like this, or it may be a major problem affecting all the scores (for example if some ground truth for the test set has wrong dimensions).\nI'm surprised no one else seems to be concerned about this...", "votes": null }, { "id": "280608", "postDate": "02/10/2018 12:49:10", "content": "I agree with you
", "rawMarkdown": "I agree with you", "votes": null }, { "id": "280764", "postDate": "02/10/2018 23:44:32", "content": "Hello @SiLiKhon, I'm looking into this locally. The index oob seems to come from histogram2d when calculating the 2d histogram. I'll be looking into this in more detail on Monday and have a fix if there needs to be one.
Hi @SiLiKhon, I did some experiments locally and confirm this to be an edge case where if you submit the entire image to be the object, it throws off the histogram methods (histogram2d, histogram) because the metric code assumes there to be a background in the image that is not part of the nuclei.
We decided not to fix the metric code to accommodate for this edge case, since we would have to re-write the logic about the background. Please avoid submitting the entire image size as the prediction, for example, if you submit imageid, 1, 65535 instead of imageid, 1, 65536 you won't hit the error.
Hi Wendy Kan,\nThanks a lot for looking into this. And sorry for the false alarm then...\nYes, I can totally understand such a tiny thing is not worth fixing.
", "rawMarkdown": "Hi Wendy Kan,\nThanks a lot for looking into this. And sorry for the false alarm then...\nYes, I can totally understand such a tiny thing is not worth fixing.", "votes": null } ], "comments": [ { "id": 275682, "author_name": "ironbar", "author_url": "", "post_date": "01/29/2018 19:34:47", "content": "Not sure, but maybe the start value should be 0 instead of 1
", "votes": null, "replies": [ { "id": 275683, "author_name": "silikhon", "author_url": "", "post_date": "01/29/2018 19:36:57", "content": "Well, it is stated on the 'Evaluation' page that it starts from 1. Also, I tried just posting the last scanline mask with this logic and it was accepted just fine...
", "votes": null, "replies": [] } ] }, { "id": 275784, "author_name": "akilaw", "author_url": "", "post_date": "01/30/2018 01:08:05", "content": "I am getting the same error for multiple masks per image. Hope that someone from kaggle explain what it is or which entry gives that error.
", "votes": null, "replies": [ { "id": 275841, "author_name": "vijaybj", "author_url": "", "post_date": "01/30/2018 04:37:21", "content": "I was getting the same error because there was an issue with resizing (upsampling) mask images to original test image size.
", "votes": null, "replies": [] }, { "id": 275945, "author_name": "akilaw", "author_url": "", "post_date": "01/30/2018 09:43:32", "content": "I checked the mask sizes and they are similar to the original image size. Finally I decided to add all the masks together and use the RLE method used in the unet kernel and that submission worked.
", "votes": null, "replies": [] } ] }, { "id": 276245, "author_name": "bencopeland", "author_url": "", "post_date": "01/31/2018 02:59:47", "content": "I'm under the impression that each column needs to be encoded separately, so a run length shouldn't be longer than the height of the image
", "votes": null, "replies": [ { "id": 276293, "author_name": "silikhon", "author_url": "", "post_date": "01/31/2018 05:34:53", "content": "Hmm, have you tried that (for a full-image mask)? For me it didn't work (for both (w, h) and (h, w)):
\n\n' '.join([str(i) + ' ' + str(w) for i in range(1, w * h, w)])\nunless there's a mistake here...
", "votes": null, "replies": [] } ] }, { "id": 279125, "author_name": "spnrpa", "author_url": "", "post_date": "02/07/2018 12:06:14", "content": "This happened when the submission contained a \"blank\"/missing EncodedPixels row followed by a non-blank row:
\n\n\nca20076870e8fb604e61802605a9ac45419c82dd3e23404c56c4869f9502a5ef,\nca20076870e8fb604e61802605a9ac45419c82dd3e23404c56c4869f9502a5ef,29153 1\n
how do you solve it?
", "votes": null, "replies": [] } ] }, { "id": 279602, "author_name": "silikhon", "author_url": "", "post_date": "02/08/2018 10:36:29", "content": "Could someone from the organizers pls react to this? I don't see how the code from the OP should produce a faulty submission. Unless I'm mistaken, this is clearly a bug in the evaluation code, which could easily lead to wrong scores in case, for example, some ground truth image(s) for the test set have wrong dimensions.
", "votes": null, "replies": [] }, { "id": 279820, "author_name": "agoodman", "author_url": "", "post_date": "02/08/2018 19:24:34", "content": "I believe the error you encountered:
\n\nPython\nIndex was outside the bounds of the array.\n
is correct. It’s difficult to ascertain If there’s a problem without more context (e.g. FullImageMaskRLE’s body).
\n\nTry verifying that the run-length decoded label matrix doesn’t exceed the image’s shape.
", "votes": null, "replies": [ { "id": 279825, "author_name": "agoodman", "author_url": "", "post_date": "02/08/2018 19:34:17", "content": "I created a sample CSV (for one image). You might find it helpful:
\n\nhttps://gist.github.com/0x00b1/2dff1f78497b1bbe6fc5b19be1664c68
", "votes": null, "replies": [] }, { "id": 279835, "author_name": "silikhon", "author_url": "", "post_date": "02/08/2018 20:01:10", "content": "The code in OP is a MWE, FullImageMaskRLE's body is at lines 6-7.
", "votes": null, "replies": [] }, { "id": 279839, "author_name": "silikhon", "author_url": "", "post_date": "02/08/2018 20:07:46", "content": "Thanks, but I'm not saying I cant submit anything. Submissions with more or less adequate masks work. I'm just saying a submission of type \"id,1 img_size\" doesn't.
", "votes": null, "replies": [] }, { "id": 280386, "author_name": "agoodman", "author_url": "", "post_date": "02/09/2018 21:29:06", "content": "So you’re asking why you can’t submit a bad mask (e.g. a mask that covers every pixel)? Why do you think this is a major problem? I’m not following.
", "votes": null, "replies": [] }, { "id": 280607, "author_name": "silikhon", "author_url": "", "post_date": "02/10/2018 12:40:43", "content": "Well, it's only \"bad\" in terms of highlighting the nuclei, it's still a \"good\" mask from the point of the evaluation rules and it deserves to be evaluated without a problem. The fact that evaluation code chokes on it indicates there's a problem with the code. It may be an innocent problem only affecting a mask like this, or it may be a major problem affecting all the scores (for example if some ground truth for the test set has wrong dimensions).\nI'm surprised no one else seems to be concerned about this...
", "votes": null, "replies": [] }, { "id": 280608, "author_name": "ironbar", "author_url": "", "post_date": "02/10/2018 12:49:10", "content": "I agree with you
", "votes": null, "replies": [] }, { "id": 280764, "author_name": "wendykan", "author_url": "", "post_date": "02/10/2018 23:44:32", "content": "Hello @SiLiKhon, I'm looking into this locally. The index oob seems to come from histogram2d when calculating the 2d histogram. I'll be looking into this in more detail on Monday and have a fix if there needs to be one.
Hi @SiLiKhon, I did some experiments locally and confirm this to be an edge case where if you submit the entire image to be the object, it throws off the histogram methods (histogram2d, histogram) because the metric code assumes there to be a background in the image that is not part of the nuclei.
We decided not to fix the metric code to accommodate for this edge case, since we would have to re-write the logic about the background. Please avoid submitting the entire image size as the prediction, for example, if you submit imageid, 1, 65535 instead of imageid, 1, 65536 you won't hit the error.
Hi Wendy Kan,\nThanks a lot for looking into this. And sorry for the false alarm then...\nYes, I can totally understand such a tiny thing is not worth fixing.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "275677": "I'm trying to make a benchmark submission with a single image-sized mask for each image, while for some reason I'm getting an out of bounds exception:\n\n> Index was outside the bounds of the array.\n\nHere's the python code producing the csv file.\n\n from imageio import imread\n from glob import glob\n \n folders = glob(\"../data/stage1_test/*\")\n \n def FullImageMaskRLE(w, h): \n return \"1 {}\".format(w * h)\n \n print(\"ImageId,EncodedPixels\")\n for folder in folders:\n img_id = folder.split('/')[-1]\n img_file, = glob(folder + \"/images/*.png\")\n img_data = imread(img_file)\n print(\"{},{}\".format(img_id,\n FullImageMaskRLE(img_data.shape[1], img_data.shape[0])))\n\nAny ideas why this might be happening?", "275682": "Not sure, but maybe the start value should be 0 instead of 1", "275683": "Well, it is stated on the 'Evaluation' page that it starts from 1. Also, I tried just posting the last scanline mask with this logic and it was accepted just fine...", "275784": "I am getting the same error for multiple masks per image. Hope that someone from kaggle explain what it is or which entry gives that error.", "275841": "I was getting the same error because there was an issue with resizing (upsampling) mask images to original test image size.", "275945": "I checked the mask sizes and they are similar to the original image size. Finally I decided to add all the masks together and use the RLE method used in the unet kernel and that submission worked.", "276245": "I'm under the impression that each column needs to be encoded separately, so a run length shouldn't be longer than the height of the image", "276293": "Hmm, have you tried that (for a full-image mask)? For me it didn't work (for both (w, h) and (h, w)):\n\n\n ' '.join([str(i) + ' ' + str(w) for i in range(1, w * h, w)])\n\nunless there's a mistake here...", "279125": "This happened when the submission contained a \"blank\"/missing EncodedPixels row followed by a non-blank row:\n\n```\nca20076870e8fb604e61802605a9ac45419c82dd3e23404c56c4869f9502a5ef,\nca20076870e8fb604e61802605a9ac45419c82dd3e23404c56c4869f9502a5ef,29153 1\n```", "279467": "how do you solve it?", "279602": "Could someone from the organizers pls react to this? I don't see how the code from the OP should produce a faulty submission. Unless I'm mistaken, this is clearly a bug in the evaluation code, which could easily lead to wrong scores in case, for example, some ground truth image(s) for the test set have wrong dimensions.", "279820": "I believe the error you encountered:\n\n```Python\nIndex was outside the bounds of the array.\n```\n\nis correct. It’s difficult to ascertain If there’s a problem without more context (e.g. FullImageMaskRLE’s body).\n\nTry verifying that the run-length decoded label matrix doesn’t exceed the image’s shape.", "279825": "I created a sample CSV (for one image). You might find it helpful:\n\nhttps://gist.github.com/0x00b1/2dff1f78497b1bbe6fc5b19be1664c68", "279835": "The code in OP is a MWE, FullImageMaskRLE's body is at lines 6-7.", "279839": "Thanks, but I'm not saying I cant submit anything. Submissions with more or less adequate masks work. I'm just saying a submission of type \"id,1 img_size\" doesn't.", "280386": "So you’re asking why you can’t submit a bad mask (e.g. a mask that covers every pixel)? Why do you think this is a major problem? I’m not following.", "280607": "Well, it's only \"bad\" in terms of highlighting the nuclei, it's still a \"good\" mask from the point of the evaluation rules and it deserves to be evaluated without a problem. The fact that evaluation code chokes on it indicates there's a problem with the code. It may be an innocent problem only affecting a mask like this, or it may be a major problem affecting all the scores (for example if some ground truth for the test set has wrong dimensions).\nI'm surprised no one else seems to be concerned about this...", "280608": "I agree with you", "280764": "Hello @SiLiKhon, I'm looking into this locally. The index oob seems to come from `histogram2d` when calculating the 2d histogram. I'll be looking into this in more detail on Monday and have a fix if there needs to be one.", "281488": "Hi @SiLiKhon, I did some experiments locally and confirm this to be an edge case where if you submit the entire image to be the object, it throws off the histogram methods (`histogram2d`, `histogram`) because the metric code assumes there to be a background in the image that is not part of the nuclei. \n\nWe decided not to fix the metric code to accommodate for this edge case, since we would have to re-write the logic about the background. Please avoid submitting the entire image size as the prediction, for example, if you submit `imageid, 1, 65535` instead of `imageid, 1, 65536` you won't hit the error.", "281587": "Hi Wendy Kan,\nThanks a lot for looking into this. And sorry for the false alarm then...\nYes, I can totally understand such a tiny thing is not worth fixing." }, "source": "meta" }