Dear wheat-head detecting colleagues,
\n\nplease find below a short summary-EDA for noise in the data.
\n\nList of images with bboxes without wheat.\nAre huge bboxes normal?\n[https://www.kaggle.com/c/global-wheat-detection/discussion/157009](Label noise)
\n\nThere are huge bboxes without wheat in them.\nRoughly 12 images where found so far, but there are most probably more!\nWe can either REMOVE the whole image or only the relevant noisy Bbox. Indices of he bboxes in train.csv where shared, so we can remove them easily.\nImportant note: Removing the noisy falsy bboes resulted in HIGHER CV but LOWER LB. Probably the LB-data also contains falsy bboxses.\nThere are also super small bboxes without any wheat in them.
\n\n(I will **highlight **important parts)
\n\n2020/05/06\nSource:\n> Thank for noticing this problem !\n1- It's not normal\n2- I'll investigate the origin of the problem…\nIs it happening on all locations ?
\n\n2020/05/07\nSource:\n> For your information, the huge bbox comes from a weird bug of our annotation tool which assigned same id to different boxes, resulting in a super large \"merged box\". There are not too much very problematic edge cases like that but I will send to Kaggle an update version of such boxes, because we still have the individual \"segmentation mask\". Still there will be quite large boxes because in some images the wheat head is quite close to the camera, but at least it is a real life edgee case to handle and not a problem with the data :)\nAnother issue I've noticed working on that there are some too small boxes are present in the file. I thought I filtered them before but apparently not. This small boxes often have a height or width below 10 and was too difficult to notice by a human reviewer on our tool. Most of the problems will be solved in the update and again it represents very few bounding boxes (apparently 170 on almost 150k).\nI hope it help you with the competition and thank you again for your feedback ! This kind of competition is really new for our community, that's the reason there are few mistakes at the launch, but we really want to build the fairest competition possible !\nEtienne
\n\n2020/06/22\nSource:
\n\n> Dear all,\nI really understand your point. The problem comes from an export from our bbox tool that have merge two bboxes into one *(if you look at all the huge bboxes, the two wheat heads are in the corner) . A **corrected version of the training dataset and test set is ready but there are several concerns from the technical team that it could cause problem*.\nI agree that we failed as organizers to provide the best experience for competitors, but we are currently** trying our best to correct the situation**.\nBest,\nEtienne
\n\n2020/06/24\nSource:
\n\n>tanulsingh077 thank you very much for your question. We are currently investigating what would be the best solution to solve the bug.
\n\n---> Currently it's inconclusive if and when there will be an update.
\n\nEDIT:
\n\nThere will be an competition update for the private test-dataset! only, see: https://www.kaggle.com/c/global-wheat-detection/discussion/163291
", "messages": [ { "id": "899505", "postDate": "06/24/2020 09:25:10", "content": "Dear wheat-head detecting colleagues,
\n\nplease find below a short summary-EDA for noise in the data.
\n\nList of images with bboxes without wheat.\nAre huge bboxes normal?\n[https://www.kaggle.com/c/global-wheat-detection/discussion/157009](Label noise)
\n\nThere are huge bboxes without wheat in them.\nRoughly 12 images where found so far, but there are most probably more!\nWe can either REMOVE the whole image or only the relevant noisy Bbox. Indices of he bboxes in train.csv where shared, so we can remove them easily.\nImportant note: Removing the noisy falsy bboes resulted in HIGHER CV but LOWER LB. Probably the LB-data also contains falsy bboxses.\nThere are also super small bboxes without any wheat in them.
\n\n(I will **highlight **important parts)
\n\n2020/05/06\nSource:\n> Thank for noticing this problem !\n1- It's not normal\n2- I'll investigate the origin of the problem…\nIs it happening on all locations ?
\n\n2020/05/07\nSource:\n> For your information, the huge bbox comes from a weird bug of our annotation tool which assigned same id to different boxes, resulting in a super large \"merged box\". There are not too much very problematic edge cases like that but I will send to Kaggle an update version of such boxes, because we still have the individual \"segmentation mask\". Still there will be quite large boxes because in some images the wheat head is quite close to the camera, but at least it is a real life edgee case to handle and not a problem with the data :)\nAnother issue I've noticed working on that there are some too small boxes are present in the file. I thought I filtered them before but apparently not. This small boxes often have a height or width below 10 and was too difficult to notice by a human reviewer on our tool. Most of the problems will be solved in the update and again it represents very few bounding boxes (apparently 170 on almost 150k).\nI hope it help you with the competition and thank you again for your feedback ! This kind of competition is really new for our community, that's the reason there are few mistakes at the launch, but we really want to build the fairest competition possible !\nEtienne
\n\n2020/06/22\nSource:
\n\n> Dear all,\nI really understand your point. The problem comes from an export from our bbox tool that have merge two bboxes into one *(if you look at all the huge bboxes, the two wheat heads are in the corner) . A **corrected version of the training dataset and test set is ready but there are several concerns from the technical team that it could cause problem*.\nI agree that we failed as organizers to provide the best experience for competitors, but we are currently** trying our best to correct the situation**.\nBest,\nEtienne
\n\n2020/06/24\nSource:
\n\n>tanulsingh077 thank you very much for your question. We are currently investigating what would be the best solution to solve the bug.
\n\n---> Currently it's inconclusive if and when there will be an update.
\n\nEDIT:
\n\nThere will be an competition update for the private test-dataset! only, see: https://www.kaggle.com/c/global-wheat-detection/discussion/163291
", "rawMarkdown": "Dear wheat-head detecting colleagues,\n\nplease find below a short summary-EDA for noise in the data.\n\n## **Noise in the data:**\n[List of images with bboxes without wheat.](https://www.kaggle.com/c/global-wheat-detection/discussion/159578)\n[Are huge bboxes normal?](https://www.kaggle.com/c/global-wheat-detection/discussion/149032)\n[https://www.kaggle.com/c/global-wheat-detection/discussion/157009](Label noise)\n\nThere are **huge bboxes** without wheat in them.\nRoughly 12 images where found so far, but there are most probably more!\nWe can either REMOVE the whole image or only the relevant noisy Bbox. Indices of he bboxes in train.csv where shared, so we can remove them easily.\nImportant note: Removing the noisy falsy bboes resulted in HIGHER CV but LOWER LB. Probably the LB-data also contains falsy bboxses.\nThere are also [super small bboxes](https://www.kaggle.com/c/global-wheat-detection/discussion/149032) without any wheat in them.\n\n## **Hosts comments on changing the data & reducing noise**:\n\n(I will **highlight **important parts)\n\n**2020/05/06**\n[Source](https://www.kaggle.com/c/global-wheat-detection/discussion/149032):\n> Thank for noticing this problem !\n1- **It's not normal**\n2- I'll investigate the origin of the problem…\nIs it happening on all locations ?\n\n**2020/05/07**\n[Source](https://www.kaggle.com/c/global-wheat-detection/discussion/149032):\n> For your information, the huge bbox comes from a **weird bug of our annotation tool** which assigned same id to different boxes, resulting in a **super large \"merged box\".** There are not too much very problematic edge cases like that but I will send to Kaggle an update version of such boxes, because we still have the individual \"segmentation mask\". Still there will be quite large boxes because in some images the wheat head is quite close to the camera, but at least it is a real life edgee case to handle and not a problem with the data :)\nAnother issue I've noticed working on that there are **some too small boxes** are present in the file. I thought I filtered them before but apparently not. This small boxes often have a height or width below 10 and was too difficult to notice by a human reviewer on our tool. Most of the problems will be solved in the update and again it represents very few bounding boxes (apparently 170 on almost 150k).\nI hope it help you with the competition and thank you again for your feedback ! This kind of competition is really new for our community, that's the reason there are few mistakes at the launch, but we really want to build the fairest competition possible !\nEtienne\n\n**2020/06/22**\n[Source](https://www.kaggle.com/c/global-wheat-detection/discussion/159585):\n\n> Dear all,\nI really understand your point. The problem comes from an **export from our bbox tool that have merge two bboxes into one **(if you look at all the huge bboxes, the two wheat heads are in the corner) . A ***corrected version of the training dataset and test set is ready but there are several concerns from the technical team that it could cause problem***.\nI agree that we failed as organizers to provide the best experience for competitors, but we are currently** trying our best to correct the situation**.\nBest,\nEtienne\n\n**2020/06/24**\n[Source](https://www.kaggle.com/c/global-wheat-detection/discussion/159585):\n\n>tanulsingh077 thank you very much for your question. **We are currently investigating what would be the best solution to solve the bug**.\n\n\n\n---> **Currently it's inconclusive if and when there will be an update.**\n\n\nEDIT:\n\nThere will be an competition update for the private test-dataset! only, see: https://www.kaggle.com/c/global-wheat-detection/discussion/163291", "votes": null }, { "id": "905326", "postDate": "06/28/2020 12:56:05", "content": "Dear @bendvd, we are trying to follow the multiple threads (see above) with the discussions about annotation-tool induced noise.
\n\nOf course noise is the salt in every ML-competition-soup, but it would clearly help if you could make a statement if \n1) an update for train/test data is coming or not.\n2) test data was created with the same annotation tool.
\n\nThanks so far for the challenging and interesting competition!
", "rawMarkdown": "Dear @bendvd, we are trying to follow the multiple threads (see above) with the discussions about annotation-tool induced noise.\n\nOf course noise is the salt in every ML-competition-soup, but it would clearly help if you could make a statement if \n1) an update for train/test data is coming or not.\n2) test data was created with the same annotation tool.\n\nThanks so far for the challenging and interesting competition!", "votes": null }, { "id": "908174", "postDate": "06/30/2020 12:20:55", "content": "I agree organizers should clarify, this is getting confusing. Thinking about the time, I dont think they will update the train/test set as the mention in the comment: \" A corrected version of the training dataset and test set is ready but there are several concerns from the technical team that it could cause problem.\" Indeed it will cause problem.
", "rawMarkdown": "I agree organizers should clarify, this is getting confusing. Thinking about the time, I dont think they will update the train/test set as the mention in the comment: \" A corrected version of the training dataset and test set is ready but there are several concerns from the technical team that it could cause problem.\" Indeed it will cause problem.", "votes": null }, { "id": "922086", "postDate": "07/09/2020 19:47:25", "content": "Dear @ahmmadrashid , dont miss this update: https://www.kaggle.com/c/global-wheat-detection/discussion/163291 --> quite important!
", "rawMarkdown": "Dear @ahmmadrashid , dont miss this update: https://www.kaggle.com/c/global-wheat-detection/discussion/163291 --> quite important!", "votes": null } ], "comments": [ { "id": 905326, "author_name": "ChristianDenich", "author_url": "", "post_date": "06/28/2020 12:56:05", "content": "Dear @bendvd, we are trying to follow the multiple threads (see above) with the discussions about annotation-tool induced noise.
\n\nOf course noise is the salt in every ML-competition-soup, but it would clearly help if you could make a statement if \n1) an update for train/test data is coming or not.\n2) test data was created with the same annotation tool.
\n\nThanks so far for the challenging and interesting competition!
", "votes": null, "replies": [ { "id": 908174, "author_name": "ahmmadrashid", "author_url": "", "post_date": "06/30/2020 12:20:55", "content": "I agree organizers should clarify, this is getting confusing. Thinking about the time, I dont think they will update the train/test set as the mention in the comment: \" A corrected version of the training dataset and test set is ready but there are several concerns from the technical team that it could cause problem.\" Indeed it will cause problem.
", "votes": null, "replies": [] }, { "id": 922086, "author_name": "ChristianDenich", "author_url": "", "post_date": "07/09/2020 19:47:25", "content": "Dear @ahmmadrashid , dont miss this update: https://www.kaggle.com/c/global-wheat-detection/discussion/163291 --> quite important!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "899505": "Dear wheat-head detecting colleagues,\n\nplease find below a short summary-EDA for noise in the data.\n\n## **Noise in the data:**\n[List of images with bboxes without wheat.](https://www.kaggle.com/c/global-wheat-detection/discussion/159578)\n[Are huge bboxes normal?](https://www.kaggle.com/c/global-wheat-detection/discussion/149032)\n[https://www.kaggle.com/c/global-wheat-detection/discussion/157009](Label noise)\n\nThere are **huge bboxes** without wheat in them.\nRoughly 12 images where found so far, but there are most probably more!\nWe can either REMOVE the whole image or only the relevant noisy Bbox. Indices of he bboxes in train.csv where shared, so we can remove them easily.\nImportant note: Removing the noisy falsy bboes resulted in HIGHER CV but LOWER LB. Probably the LB-data also contains falsy bboxses.\nThere are also [super small bboxes](https://www.kaggle.com/c/global-wheat-detection/discussion/149032) without any wheat in them.\n\n## **Hosts comments on changing the data & reducing noise**:\n\n(I will **highlight **important parts)\n\n**2020/05/06**\n[Source](https://www.kaggle.com/c/global-wheat-detection/discussion/149032):\n> Thank for noticing this problem !\n1- **It's not normal**\n2- I'll investigate the origin of the problem…\nIs it happening on all locations ?\n\n**2020/05/07**\n[Source](https://www.kaggle.com/c/global-wheat-detection/discussion/149032):\n> For your information, the huge bbox comes from a **weird bug of our annotation tool** which assigned same id to different boxes, resulting in a **super large \"merged box\".** There are not too much very problematic edge cases like that but I will send to Kaggle an update version of such boxes, because we still have the individual \"segmentation mask\". Still there will be quite large boxes because in some images the wheat head is quite close to the camera, but at least it is a real life edgee case to handle and not a problem with the data :)\nAnother issue I've noticed working on that there are **some too small boxes** are present in the file. I thought I filtered them before but apparently not. This small boxes often have a height or width below 10 and was too difficult to notice by a human reviewer on our tool. Most of the problems will be solved in the update and again it represents very few bounding boxes (apparently 170 on almost 150k).\nI hope it help you with the competition and thank you again for your feedback ! This kind of competition is really new for our community, that's the reason there are few mistakes at the launch, but we really want to build the fairest competition possible !\nEtienne\n\n**2020/06/22**\n[Source](https://www.kaggle.com/c/global-wheat-detection/discussion/159585):\n\n> Dear all,\nI really understand your point. The problem comes from an **export from our bbox tool that have merge two bboxes into one **(if you look at all the huge bboxes, the two wheat heads are in the corner) . A ***corrected version of the training dataset and test set is ready but there are several concerns from the technical team that it could cause problem***.\nI agree that we failed as organizers to provide the best experience for competitors, but we are currently** trying our best to correct the situation**.\nBest,\nEtienne\n\n**2020/06/24**\n[Source](https://www.kaggle.com/c/global-wheat-detection/discussion/159585):\n\n>tanulsingh077 thank you very much for your question. **We are currently investigating what would be the best solution to solve the bug**.\n\n\n\n---> **Currently it's inconclusive if and when there will be an update.**\n\n\nEDIT:\n\nThere will be an competition update for the private test-dataset! only, see: https://www.kaggle.com/c/global-wheat-detection/discussion/163291", "905326": "Dear @bendvd, we are trying to follow the multiple threads (see above) with the discussions about annotation-tool induced noise.\n\nOf course noise is the salt in every ML-competition-soup, but it would clearly help if you could make a statement if \n1) an update for train/test data is coming or not.\n2) test data was created with the same annotation tool.\n\nThanks so far for the challenging and interesting competition!", "908174": "I agree organizers should clarify, this is getting confusing. Thinking about the time, I dont think they will update the train/test set as the mention in the comment: \" A corrected version of the training dataset and test set is ready but there are several concerns from the technical team that it could cause problem.\" Indeed it will cause problem.", "922086": "Dear @ahmmadrashid , dont miss this update: https://www.kaggle.com/c/global-wheat-detection/discussion/163291 --> quite important!" }, "source": "meta" }