Has anyone encountered \"Submission Scoring Error\"?
\n\nI've added a postprocessing function to clear and fuse some bounding boxes and after that my submission is no longer valid. \nAm I right that \"Submission Scoring Error\" is present only in the case when the generated csv file cannot be scored, e.g. missing image_ids or corrupt bounding boxes?\nWould appreciate any help from the community, especially would love to hear from someone who has successfully dealt with the problem.
", "messages": [ { "id": 918068, "postDate": "2020-07-07T00:20:48.423Z", "content": "Has anyone encountered \"Submission Scoring Error\"?
\n\nI've added a postprocessing function to clear and fuse some bounding boxes and after that my submission is no longer valid. \nAm I right that \"Submission Scoring Error\" is present only in the case when the generated csv file cannot be scored, e.g. missing image_ids or corrupt bounding boxes?\nWould appreciate any help from the community, especially would love to hear from someone who has successfully dealt with the problem.
", "rawMarkdown": "Has anyone encountered \"Submission Scoring Error\"? \n\nI've added a postprocessing function to clear and fuse some bounding boxes and after that my submission is no longer valid. \nAm I right that \"Submission Scoring Error\" is present only in the case when the generated csv file cannot be scored, e.g. missing image_ids or corrupt bounding boxes?\nWould appreciate any help from the community, especially would love to hear from someone who has successfully dealt with the problem.\n\n", "votes": 2 }, { "id": 918729, "postDate": "2020-07-07T12:39:32.560Z", "content": "I had a similar issue. I played with pseudo-labelling and reached the 6h GPU limit when submitting
", "rawMarkdown": "I had a similar issue. I played with pseudo-labelling and reached the 6h GPU limit when submitting", "votes": 1 }, { "id": 918686, "postDate": "2020-07-07T11:59:51.543Z", "content": "I've got the same issue bescause of post-processing too.
", "rawMarkdown": "I've got the same issue bescause of post-processing too.", "votes": 1, "replies": [ { "id": 918701, "postDate": "2020-07-07T12:11:15.203Z", "content": "have you solved it by any chance?
", "rawMarkdown": "have you solved it by any chance? " }, { "id": 918709, "postDate": "2020-07-07T12:19:09.493Z", "content": "Solved by not using post-processing x) Joking.\nI made two different approaches of post-processing and weirdly both send me the same error. \nI am working on that, I'll let you know if I get smth.
", "rawMarkdown": "Solved by not using post-processing x) Joking.\nI made two different approaches of post-processing and weirdly both send me the same error. \nI am working on that, I'll let you know if I get smth." }, { "id": 918978, "postDate": "2020-07-07T16:13:32.110Z", "content": "that would help, thanks!
", "rawMarkdown": "that would help, thanks!" }, { "id": 925282, "postDate": "2020-07-12T01:15:13.640Z", "content": "looks like I encounter this error once doing bbox postprocessing and trying to remove some of the bboxes. is your postprocessing similar to mine?
", "rawMarkdown": "looks like I encounter this error once doing bbox postprocessing and trying to remove some of the bboxes. is your postprocessing similar to mine?" }, { "id": 925307, "postDate": "2020-07-12T02:14:12.960Z", "content": "Not exactly, but it could cause the removal of some bboxes => I don't believe that is the problem for me. Why could removing some bboxes causes error. NMS, NMW, WBF, ..., are all removing some boxes. Maybe the problem is in your way of removing the boxes?
", "rawMarkdown": "Not exactly, but it could cause the removal of some bboxes => I don't believe that is the problem for me. Why could removing some bboxes causes error. NMS, NMW, WBF, ..., are all removing some boxes. Maybe the problem is in your way of removing the boxes? " }, { "id": 956556, "postDate": "2020-08-03T15:43:22.337Z", "content": "@ffares i m unable to figure out sub scoring error with 1024 images. did u resolve the error\nWhat could be the reason. \n1) Not all images 1024,can it be resolved by merely putting resize 1024 ?\n2) What to do with blanks ,with no wheat heads ?
", "rawMarkdown": "@ffares i m unable to figure out sub scoring error with 1024 images. did u resolve the error\nWhat could be the reason. \n1) Not all images 1024,can it be resolved by merely putting resize 1024 ?\n2) What to do with blanks ,with no wheat heads ?" }, { "id": 956589, "postDate": "2020-08-03T16:04:11.633Z", "content": "my error was the following: if no wheat heads were detected, I was still doing some permutations with the numpy ndarray of size 0... hope it helps
", "rawMarkdown": "my error was the following: if no wheat heads were detected, I was still doing some permutations with the numpy ndarray of size 0... hope it helps", "votes": 1 }, { "id": 956605, "postDate": "2020-08-03T16:14:10.387Z", "content": "@sevakon how u handled different size images of test no 1024 ,we wont know which img is not 1024 so we can rescale its bboxes
", "rawMarkdown": "@sevakon how u handled different size images of test no 1024 ,we wont know which img is not 1024 so we can rescale its bboxes" }, { "id": 956688, "postDate": "2020-08-03T17:32:40.493Z", "content": "well, you could check for image size once reading the image, e.g. with opencv, and keep the aspect ratio for later
", "rawMarkdown": "well, you could check for image size once reading the image, e.g. with opencv, and keep the aspect ratio for later" } ] }, { "id": 918672, "postDate": "2020-07-07T11:46:59.040Z", "content": "I've tried so many things and none of them fixed the problem, unfortunately.
\n\nI don't even know where the issue might be. \nI do resize the input image to 1024 pixels, scaling the resulting values back to input resolution after the prediction phase. \nMy score is a float value, my bbox is integer values. I did probing with the first test image having zero predictions and the last having one prediction and that submission did not fail, so the issue is not zero or one predicted bbox. It is just one function (which is in a try-except block) that apparently screws something, even if I check for the bbox correctness.
", "rawMarkdown": "I've tried so many things and none of them fixed the problem, unfortunately.\n\nI don't even know where the issue might be. \nI do resize the input image to 1024 pixels, scaling the resulting values back to input resolution after the prediction phase. \nMy score is a float value, my bbox is integer values. I did probing with the first test image having zero predictions and the last having one prediction and that submission did not fail, so the issue is not zero or one predicted bbox. It is just one function (which is in a try-except block) that apparently screws something, even if I check for the bbox correctness. ", "votes": 1 }, { "id": 918098, "postDate": "2020-07-07T01:43:27.937Z", "content": "Hmm… I’ve also got the same submission scoring error for my kernel today.
", "rawMarkdown": "Hmm… I’ve also got the same submission scoring error for my kernel today.", "replies": [ { "id": 919874, "postDate": "2020-07-08T06:45:40.837Z", "content": "Update: I solved it by catching a potential error around weighted box fusion. It sometimes raises numerical exceptions when input boxes are empty.
", "rawMarkdown": "Update: I solved it by catching a potential error around weighted box fusion. It sometimes raises numerical exceptions when input boxes are empty.", "votes": 2 }, { "id": 920049, "postDate": "2020-07-08T09:08:43.833Z", "content": "interesting, thanks! looks like the scoring error may occur when the script is failing with an exception
", "rawMarkdown": "interesting, thanks! looks like the scoring error may occur when the script is failing with an exception" }, { "id": 920510, "postDate": "2020-07-08T16:34:11.580Z", "rawMarkdown": "", "votes": -2, "isDeleted": true } ] } ], "comments": [ { "id": 918729, "author_name": "Alex", "author_url": "", "post_date": "2020-07-07T12:39:32.560000", "content": "I had a similar issue. I played with pseudo-labelling and reached the 6h GPU limit when submitting
", "votes": 1, "replies": [] }, { "id": 918686, "author_name": "Fares FOURATI", "author_url": "", "post_date": "2020-07-07T11:59:51.543000", "content": "I've got the same issue bescause of post-processing too.
", "votes": 1, "replies": [ { "id": 918701, "author_name": "Seva Konjahhin", "author_url": "", "post_date": "2020-07-07T12:11:15.203000", "content": "have you solved it by any chance?
", "votes": 0, "replies": [] }, { "id": 918709, "author_name": "Fares FOURATI", "author_url": "", "post_date": "2020-07-07T12:19:09.493000", "content": "Solved by not using post-processing x) Joking.\nI made two different approaches of post-processing and weirdly both send me the same error. \nI am working on that, I'll let you know if I get smth.
", "votes": 0, "replies": [] }, { "id": 918978, "author_name": "Seva Konjahhin", "author_url": "", "post_date": "2020-07-07T16:13:32.110000", "content": "that would help, thanks!
", "votes": 0, "replies": [] }, { "id": 925282, "author_name": "Seva Konjahhin", "author_url": "", "post_date": "2020-07-12T01:15:13.640000", "content": "looks like I encounter this error once doing bbox postprocessing and trying to remove some of the bboxes. is your postprocessing similar to mine?
", "votes": 0, "replies": [] }, { "id": 925307, "author_name": "Fares FOURATI", "author_url": "", "post_date": "2020-07-12T02:14:12.960000", "content": "Not exactly, but it could cause the removal of some bboxes => I don't believe that is the problem for me. Why could removing some bboxes causes error. NMS, NMW, WBF, ..., are all removing some boxes. Maybe the problem is in your way of removing the boxes?
", "votes": 0, "replies": [] }, { "id": 956556, "author_name": "Jaideep", "author_url": "", "post_date": "2020-08-03T15:43:22.337000", "content": "@ffares i m unable to figure out sub scoring error with 1024 images. did u resolve the error\nWhat could be the reason. \n1) Not all images 1024,can it be resolved by merely putting resize 1024 ?\n2) What to do with blanks ,with no wheat heads ?
", "votes": 0, "replies": [] }, { "id": 956589, "author_name": "Seva Konjahhin", "author_url": "", "post_date": "2020-08-03T16:04:11.633000", "content": "my error was the following: if no wheat heads were detected, I was still doing some permutations with the numpy ndarray of size 0... hope it helps
", "votes": 1, "replies": [] }, { "id": 956605, "author_name": "Jaideep", "author_url": "", "post_date": "2020-08-03T16:14:10.387000", "content": "@sevakon how u handled different size images of test no 1024 ,we wont know which img is not 1024 so we can rescale its bboxes
", "votes": 0, "replies": [] }, { "id": 956688, "author_name": "Seva Konjahhin", "author_url": "", "post_date": "2020-08-03T17:32:40.493000", "content": "well, you could check for image size once reading the image, e.g. with opencv, and keep the aspect ratio for later
", "votes": 0, "replies": [] } ] }, { "id": 918672, "author_name": "Seva Konjahhin", "author_url": "", "post_date": "2020-07-07T11:46:59.040000", "content": "I've tried so many things and none of them fixed the problem, unfortunately.
\n\nI don't even know where the issue might be. \nI do resize the input image to 1024 pixels, scaling the resulting values back to input resolution after the prediction phase. \nMy score is a float value, my bbox is integer values. I did probing with the first test image having zero predictions and the last having one prediction and that submission did not fail, so the issue is not zero or one predicted bbox. It is just one function (which is in a try-except block) that apparently screws something, even if I check for the bbox correctness.
", "votes": 1, "replies": [] }, { "id": 918098, "author_name": "Peiyuan Liao", "author_url": "", "post_date": "2020-07-07T01:43:27.937000", "content": "Hmm… I’ve also got the same submission scoring error for my kernel today.
", "votes": 0, "replies": [ { "id": 919874, "author_name": "Peiyuan Liao", "author_url": "", "post_date": "2020-07-08T06:45:40.837000", "content": "Update: I solved it by catching a potential error around weighted box fusion. It sometimes raises numerical exceptions when input boxes are empty.
", "votes": 2, "replies": [] }, { "id": 920049, "author_name": "Seva Konjahhin", "author_url": "", "post_date": "2020-07-08T09:08:43.833000", "content": "interesting, thanks! looks like the scoring error may occur when the script is failing with an exception
", "votes": 0, "replies": [] }, { "id": 920510, "author_name": "", "author_url": "", "post_date": "2020-07-08T16:34:11.580000", "content": "", "votes": -2, "replies": [] } ] } ], "raw_markdown_by_id": { "918068": "Has anyone encountered \"Submission Scoring Error\"? \n\nI've added a postprocessing function to clear and fuse some bounding boxes and after that my submission is no longer valid. \nAm I right that \"Submission Scoring Error\" is present only in the case when the generated csv file cannot be scored, e.g. missing image_ids or corrupt bounding boxes?\nWould appreciate any help from the community, especially would love to hear from someone who has successfully dealt with the problem.\n\n", "918729": "I had a similar issue. I played with pseudo-labelling and reached the 6h GPU limit when submitting", "918686": "I've got the same issue bescause of post-processing too.", "918672": "I've tried so many things and none of them fixed the problem, unfortunately.\n\nI don't even know where the issue might be. \nI do resize the input image to 1024 pixels, scaling the resulting values back to input resolution after the prediction phase. \nMy score is a float value, my bbox is integer values. I did probing with the first test image having zero predictions and the last having one prediction and that submission did not fail, so the issue is not zero or one predicted bbox. It is just one function (which is in a try-except block) that apparently screws something, even if I check for the bbox correctness. ", "918098": "Hmm… I’ve also got the same submission scoring error for my kernel today." } }