![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F864684%2F1a71a1af2f70063be07e190d9b0b11aa%2Flabel_nosie_2.jpg?generation=1591654182556080&alt=media\")
\nThese poor annotations hurt the CV quite much.Hello all!
\nThank you for your feedback and suggestions since the launch of the competition. We heard you and it has taken us some time to dig into the issues, thank you for your patience!
\nWe have been working with the host to find a solution that addresses the key problems without introducing risk to the work you have done. The issues, as we understand them, are:
\nFixing all of these problems, while they are individually relatively minor, would involve a collectively massive amount of risk to the competition. Updating the training set, test set, solution, and metric could potentially break many past submissions (there are currently nearly 18,000 of these). The benefit of fixing everything, in the absence of definitive competition-breaking issues, is outweighed by the mess it would cause the community.
\nAfter talking with the host, the critical problem is that there are bounding boxes in the test set which are incorrect. In the interests of scoring this competition as fairly as possible, we will correct the private leaderboard’s bounding boxes. Your current techniques and scores will stand, and no submissions will potentially be invalidated. Your final submissions, however, will be scored based on a more correct set of data.
\nThere is no perfect solution for this situation, and despite our and the host’s best efforts we realize that this is not ideal. We appreciate the work you all have already put into addressing this issue. We believe that this change addresses the main risk in the competition data while allowing your current work to stand. We have learned from past competitions that even making a few changes to a competition can make small issues much worse.
\nTogether with the host we worked hard to find the best way forward for this competition and we want to thank you all for your patience and for all of the work you have put into the competition so far.
", "messages": [ { "id": 911122, "postDate": "2020-07-01T15:22:52.413Z", "content": "Hello all!
\nThank you for your feedback and suggestions since the launch of the competition. We heard you and it has taken us some time to dig into the issues, thank you for your patience!
\nWe have been working with the host to find a solution that addresses the key problems without introducing risk to the work you have done. The issues, as we understand them, are:
\nFixing all of these problems, while they are individually relatively minor, would involve a collectively massive amount of risk to the competition. Updating the training set, test set, solution, and metric could potentially break many past submissions (there are currently nearly 18,000 of these). The benefit of fixing everything, in the absence of definitive competition-breaking issues, is outweighed by the mess it would cause the community.
\nAfter talking with the host, the critical problem is that there are bounding boxes in the test set which are incorrect. In the interests of scoring this competition as fairly as possible, we will correct the private leaderboard’s bounding boxes. Your current techniques and scores will stand, and no submissions will potentially be invalidated. Your final submissions, however, will be scored based on a more correct set of data.
\nThere is no perfect solution for this situation, and despite our and the host’s best efforts we realize that this is not ideal. We appreciate the work you all have already put into addressing this issue. We believe that this change addresses the main risk in the competition data while allowing your current work to stand. We have learned from past competitions that even making a few changes to a competition can make small issues much worse.
\nTogether with the host we worked hard to find the best way forward for this competition and we want to thank you all for your patience and for all of the work you have put into the competition so far.
", "rawMarkdown": "Hello all!\n\nThank you for your feedback and suggestions since the launch of the competition. We heard you and it has taken us some time to dig into the issues, thank you for your patience! \n\nWe have been working with the host to find a solution that addresses the key problems without introducing risk to the work you have done. The issues, as we understand them, are:\n- ~170 incorrectly-sized bounding boxes, out of ~190,000\n- <20 images in test that are not the same size as the others\n- A metric with top-end thresholds that are possibly too restrictive given the problem \n\nFixing all of these problems, while they are individually relatively minor, would involve a collectively massive amount of risk to the competition. Updating the training set, test set, solution, and metric could potentially break many past submissions (there are currently nearly 18,000 of these). The benefit of fixing everything, in the absence of definitive competition-breaking issues, is outweighed by the mess it would cause the community.\n\nAfter talking with the host, the critical problem is that there are bounding boxes in the test set which are incorrect. In the interests of scoring this competition as fairly as possible, we will correct the private leaderboard’s bounding boxes. Your current techniques and scores will stand, and no submissions will potentially be invalidated. Your final submissions, however, will be scored based on a more correct set of data. \n\nThere is no perfect solution for this situation, and despite our and the host’s best efforts we realize that this is not ideal. We appreciate the work you all have already put into addressing this issue. We believe that this change addresses the main risk in the competition data while allowing your current work to stand. We have learned from past competitions that even making a few changes to a competition can make small issues much worse.\n\nTogether with the host we worked hard to find the best way forward for this competition and we want to thank you all for your patience and for all of the work you have put into the competition so far.", "votes": 26 }, { "id": 911589, "postDate": "2020-07-01T20:58:55.980Z", "content": "So it will be hard to separate wheat from chaff.
", "rawMarkdown": "So it will be hard to separate wheat from chaff. ", "votes": 17, "replies": [ { "id": 923640, "postDate": "2020-07-11T03:31:54.857Z", "content": "Nice. Now show yourself out.
", "rawMarkdown": "Nice. Now show yourself out. ", "votes": -3 }, { "id": 924030, "postDate": "2020-07-11T08:16:18.657Z", "content": "😂
", "rawMarkdown": "😂 " } ] }, { "id": 911142, "postDate": "2020-07-01T15:32:37.883Z", "content": "Thanks for the information.
\n\nSo, as far as I understand: the only change is in private test dataset? Train and public test are unchanged?
\n\nI'm not sure how will this help to build a robust solution.
", "rawMarkdown": "Thanks for the information.\n\nSo, as far as I understand: the only change is in private test dataset? Train and public test are unchanged?\n\nI'm not sure how will this help to build a robust solution.", "votes": 6, "replies": [ { "id": 911173, "postDate": "2020-07-01T15:45:38.263Z", "content": "Agreed, not sure why anyone would downvote you? It doesn't help us at all develop a better solution. But hopefully it will at least result in the better, more generalized models winning.
", "rawMarkdown": "Agreed, not sure why anyone would downvote you? It doesn't help us at all develop a better solution. But hopefully it will at least result in the better, more generalized models winning.", "votes": 2 }, { "id": 911200, "postDate": "2020-07-01T15:56:22.690Z", "content": "That's definitely the idea: updating train and public test would have created a fairly huge amount of risk for the small amount of benefit provided. Updating private test hopefully allows better, more generalized models to win, without potentially breaking a lot of competitors' submissions.
", "rawMarkdown": "That's definitely the idea: updating train and public test would have created a fairly huge amount of risk for the small amount of benefit provided. Updating private test hopefully allows better, more generalized models to win, without potentially breaking a lot of competitors' submissions." }, { "id": 911382, "postDate": "2020-07-01T17:36:40.910Z", "content": "I understand the risk.\nSo how about providing the incorrectly-sized bounding boxes as \"additional data\" instead of updating train data.\nI think this would not create any risk.\nI don't want to use my time to find bad bounding boxes :(
", "rawMarkdown": "I understand the risk.\nSo how about providing the incorrectly-sized bounding boxes as \"additional data\" instead of updating train data.\nI think this would not create any risk.\nI don't want to use my time to find bad bounding boxes :(", "votes": 20 }, { "id": 920969, "postDate": "2020-07-09T01:46:45.830Z", "rawMarkdown": "", "votes": -1, "isDeleted": true } ] }, { "id": 911222, "postDate": "2020-07-01T16:03:59.863Z", "content": "TL; DR\nFilter the bboxes, train, trust CV, avoid shakeup.
", "rawMarkdown": "TL; DR\nFilter the bboxes, train, trust CV, avoid shakeup.", "votes": 4 }, { "id": 912847, "postDate": "2020-07-02T19:06:20.540Z", "content": "I understand that the ~170 incorrectly-sized bounding boxes are going to be fixed in the private.\nMay I confirm if poorly labeled binding boxes are fixed in the private set as well?\nhttps://www.kaggle.com/c/global-wheat-detection/discussion/157009\n\nThese poor annotations hurt the CV quite much.
The competition organizer has previously noted that:\n> I acknowledge the fact that this dataset is labeled slightly differently than the rest. In the test set, the bbox are tight to the wheat head (you need to trust me on this one ^^).
\n\nI presume this to mean these kinds of noisy labels will not be in the test set. Although he says to trust him, I have trust issues with this competition 😂
", "rawMarkdown": "The competition organizer has previously noted that:\n> I acknowledge the fact that this dataset is labeled slightly differently than the rest. In the test set, the bbox are tight to the wheat head (you need to trust me on this one ^^).\n\nI presume this to mean these kinds of noisy labels will not be in the test set. Although he says to trust him, I have trust issues with this competition 😂 ", "votes": 6 }, { "id": 913100, "postDate": "2020-07-03T01:08:48.573Z", "content": "Thanks, I overlooked that.\nLet's trust the host on this commit!
", "rawMarkdown": "Thanks, I overlooked that.\nLet's trust the host on this commit!", "votes": 2 } ] }, { "id": 912567, "postDate": "2020-07-02T15:17:07.173Z", "content": "Is the private dataset already updated?
", "rawMarkdown": "Is the private dataset already updated?", "replies": [ { "id": 912584, "postDate": "2020-07-02T15:35:24.763Z", "content": "They are only updating the 38% of the test set that is used for final scoring. Therefor it doesn't matter when they update the dataset as it won't be used until the competition deadline.
", "rawMarkdown": "They are only updating the 38% of the test set that is used for final scoring. Therefor it doesn't matter when they update the dataset as it won't be used until the competition deadline." } ] }, { "id": 911275, "postDate": "2020-07-01T16:32:25.460Z", "content": "Thanks for your update.\nHowever, I think this change will make the ranking on public leaderboard less believable, cause the distribution of private and public test set are no longer the same. Remember this computation is also the challenge of CVPPP workshop, top 5 will have the chance to publish papers in the ECCV2020 workshop proceedings. \nSo a question comes to me, my ranking on the public leaderboard is currently around 10. Should I prepare to submit a paper to CVPPP?
", "rawMarkdown": "Thanks for your update.\nHowever, I think this change will make the ranking on public leaderboard less believable, cause the distribution of private and public test set are no longer the same. Remember this computation is also the challenge of CVPPP workshop, top 5 will have the chance to publish papers in the ECCV2020 workshop proceedings. \nSo a question comes to me, my ranking on the public leaderboard is currently around 10. Should I prepare to submit a paper to CVPPP?", "replies": [ { "id": 911289, "postDate": "2020-07-01T16:37:18.410Z", "content": "You can submit a paper independent of your position on the leaderboard. Read where it talks about the workshop again, it says
\n\n\n\n\nThe five best paper submissions (evaluated both on their rank in the competition and their academic contribution) will be invited to present at CVPPP and be published in the ECCV workshop proceedings.
\n
Meaning that the five best paper submissions (judged on BOTH the LB score and academic contribution) will be selected to present and will be published. So you can be 10th and still be selected.
", "rawMarkdown": "You can submit a paper independent of your position on the leaderboard. Read where it talks about the workshop again, it says\n> The five best paper submissions (evaluated both on their rank in the competition and their academic contribution) will be invited to present at CVPPP and be published in the ECCV workshop proceedings.\n\nMeaning that the five best paper submissions (judged on BOTH the LB score and academic contribution) will be selected to present and will be published. So you can be 10th and still be selected." }, { "id": 911296, "postDate": "2020-07-01T16:41:57.530Z", "content": "Yes, you are right. it worth trying.
", "rawMarkdown": "Yes, you are right. it worth trying. " }, { "id": 911361, "postDate": "2020-07-01T17:16:14.237Z", "content": "Definitely! Especially considering where you are on the LB right now. I believe the conference allows for preprints to be posted so if nothing else, you can put your article on arxiv or another preprint site! Best of luck to you
", "rawMarkdown": "Definitely! Especially considering where you are on the LB right now. I believe the conference allows for preprints to be posted so if nothing else, you can put your article on arxiv or another preprint site! Best of luck to you" }, { "id": 912049, "postDate": "2020-07-02T07:15:10.190Z", "content": "Yes, the paper track is definitely here to encourage publications of novel solutions, even if the LB score is not the best.
", "rawMarkdown": "Yes, the paper track is definitely here to encourage publications of novel solutions, even if the LB score is not the best. " } ] }, { "id": 911147, "postDate": "2020-07-01T15:35:19.757Z", "content": "Thank you for providing this update... Just to verify I understand this correctly, the only data that will be verified is the 2nd round (38%) of the test data? If instead it is the entire private test set being updated, when will this update take place?
\n\nAlso, you missed a very important question that I have been waiting to hear back on for over three weeks! Please let me know if correcting training labels manually is permissible and if it considered as external data that must be shared with other competitors?
", "rawMarkdown": "Thank you for providing this update... Just to verify I understand this correctly, the only data that will be verified is the 2nd round (38%) of the test data? If instead it is the entire private test set being updated, when will this update take place?\n\nAlso, you missed a very important question that I have been [waiting to hear back on for over three weeks](https://www.kaggle.com/c/global-wheat-detection/discussion/157009#907262)! Please let me know if correcting training labels manually is permissible and if it considered as external data that must be shared with other competitors?", "replies": [ { "id": 911265, "postDate": "2020-07-01T16:24:34.483Z", "content": "Thanks for your questions.
\nCorrect, the only data that will be updated will be that 38% of data. The actual number of updated samples will be small, based on the issues we've seen.
I apologize for missing your earlier question. Thanks for your patience. Yes, it's permissible to manually correct training labels without providing them as external data.
Thank you very much for taking the time to answer. I look forward to sharing my much revised version of the training dataset after the competition! :D
", "rawMarkdown": "Thank you very much for taking the time to answer. I look forward to sharing my much revised version of the training dataset after the competition! :D" }, { "id": 911272, "postDate": "2020-07-01T16:30:13.923Z", "content": "There is also a few questions listed in the External Data thread which I believe could use some clarification. Thanks!
", "rawMarkdown": "There is also [a few questions listed in the External Data thread](https://www.kaggle.com/c/global-wheat-detection/discussion/148561#902535) which I believe could use some clarification. Thanks!", "votes": 1 }, { "id": 912052, "postDate": "2020-07-02T07:15:44.773Z", "content": "Thank you for your patience!
", "rawMarkdown": "Thank you for your patience! " }, { "id": 912640, "postDate": "2020-07-02T16:11:34.600Z", "content": "\n\n\n<20 images in test that are not the same size as the others
\n
@philculliton @bendvd How are the images with different size handled? Are they excluded from scoring? If not, please answer this question.
", "rawMarkdown": "> <20 images in test that are not the same size as the others\n\n@philculliton @bendvd How are the images with different size handled? Are they excluded from scoring? If not, please answer [this question](https://www.kaggle.com/c/global-wheat-detection/discussion/149949#901848).", "votes": 1 }, { "id": 913513, "postDate": "2020-07-03T08:55:00.840Z", "content": "They will be included. These images are in the private test set and will be corrected at the end.
", "rawMarkdown": "They will be included. These images are in the private test set and will be corrected at the end. " } ] } ], "comments": [ { "id": 911589, "author_name": "Bojan Tunguz", "author_url": "", "post_date": "2020-07-01T20:58:55.980000", "content": "So it will be hard to separate wheat from chaff.
", "votes": 17, "replies": [ { "id": 923640, "author_name": "Joseph Ngari", "author_url": "", "post_date": "2020-07-11T03:31:54.857000", "content": "Nice. Now show yourself out.
", "votes": -3, "replies": [] }, { "id": 924030, "author_name": "Rony K Roy", "author_url": "", "post_date": "2020-07-11T08:16:18.657000", "content": "😂
", "votes": 0, "replies": [] } ] }, { "id": 911142, "author_name": "Andrey Lukyanenko", "author_url": "", "post_date": "2020-07-01T15:32:37.883000", "content": "Thanks for the information.
\n\nSo, as far as I understand: the only change is in private test dataset? Train and public test are unchanged?
\n\nI'm not sure how will this help to build a robust solution.
", "votes": 6, "replies": [ { "id": 911173, "author_name": "Matt", "author_url": "", "post_date": "2020-07-01T15:45:38.263000", "content": "Agreed, not sure why anyone would downvote you? It doesn't help us at all develop a better solution. But hopefully it will at least result in the better, more generalized models winning.
", "votes": 2, "replies": [] }, { "id": 911200, "author_name": "Phil Culliton", "author_url": "", "post_date": "2020-07-01T15:56:22.690000", "content": "That's definitely the idea: updating train and public test would have created a fairly huge amount of risk for the small amount of benefit provided. Updating private test hopefully allows better, more generalized models to win, without potentially breaking a lot of competitors' submissions.
", "votes": 0, "replies": [] }, { "id": 911382, "author_name": "tito", "author_url": "", "post_date": "2020-07-01T17:36:40.910000", "content": "I understand the risk.\nSo how about providing the incorrectly-sized bounding boxes as \"additional data\" instead of updating train data.\nI think this would not create any risk.\nI don't want to use my time to find bad bounding boxes :(
", "votes": 20, "replies": [] }, { "id": 920969, "author_name": "", "author_url": "", "post_date": "2020-07-09T01:46:45.830000", "content": "", "votes": -1, "replies": [] } ] }, { "id": 911222, "author_name": "Mighty Rains", "author_url": "", "post_date": "2020-07-01T16:03:59.863000", "content": "TL; DR\nFilter the bboxes, train, trust CV, avoid shakeup.
", "votes": 4, "replies": [] }, { "id": 912847, "author_name": "arutema47", "author_url": "", "post_date": "2020-07-02T19:06:20.540000", "content": "I understand that the ~170 incorrectly-sized bounding boxes are going to be fixed in the private.\nMay I confirm if poorly labeled binding boxes are fixed in the private set as well?\nhttps://www.kaggle.com/c/global-wheat-detection/discussion/157009\n\nThese poor annotations hurt the CV quite much.
The competition organizer has previously noted that:\n> I acknowledge the fact that this dataset is labeled slightly differently than the rest. In the test set, the bbox are tight to the wheat head (you need to trust me on this one ^^).
\n\nI presume this to mean these kinds of noisy labels will not be in the test set. Although he says to trust him, I have trust issues with this competition 😂
", "votes": 6, "replies": [] }, { "id": 913100, "author_name": "arutema47", "author_url": "", "post_date": "2020-07-03T01:08:48.573000", "content": "Thanks, I overlooked that.\nLet's trust the host on this commit!
", "votes": 2, "replies": [] } ] }, { "id": 912567, "author_name": "Rifat Rashid", "author_url": "", "post_date": "2020-07-02T15:17:07.173000", "content": "Is the private dataset already updated?
", "votes": 0, "replies": [ { "id": 912584, "author_name": "Matt", "author_url": "", "post_date": "2020-07-02T15:35:24.763000", "content": "They are only updating the 38% of the test set that is used for final scoring. Therefor it doesn't matter when they update the dataset as it won't be used until the competition deadline.
", "votes": 0, "replies": [] } ] }, { "id": 911275, "author_name": "Qingyuan Wang", "author_url": "", "post_date": "2020-07-01T16:32:25.460000", "content": "Thanks for your update.\nHowever, I think this change will make the ranking on public leaderboard less believable, cause the distribution of private and public test set are no longer the same. Remember this computation is also the challenge of CVPPP workshop, top 5 will have the chance to publish papers in the ECCV2020 workshop proceedings. \nSo a question comes to me, my ranking on the public leaderboard is currently around 10. Should I prepare to submit a paper to CVPPP?
", "votes": 0, "replies": [ { "id": 911289, "author_name": "Matt", "author_url": "", "post_date": "2020-07-01T16:37:18.410000", "content": "You can submit a paper independent of your position on the leaderboard. Read where it talks about the workshop again, it says
\n\n\n\n\nThe five best paper submissions (evaluated both on their rank in the competition and their academic contribution) will be invited to present at CVPPP and be published in the ECCV workshop proceedings.
\n
Meaning that the five best paper submissions (judged on BOTH the LB score and academic contribution) will be selected to present and will be published. So you can be 10th and still be selected.
", "votes": 0, "replies": [] }, { "id": 911296, "author_name": "Qingyuan Wang", "author_url": "", "post_date": "2020-07-01T16:41:57.530000", "content": "Yes, you are right. it worth trying.
", "votes": 0, "replies": [] }, { "id": 911361, "author_name": "Matt", "author_url": "", "post_date": "2020-07-01T17:16:14.237000", "content": "Definitely! Especially considering where you are on the LB right now. I believe the conference allows for preprints to be posted so if nothing else, you can put your article on arxiv or another preprint site! Best of luck to you
", "votes": 0, "replies": [] }, { "id": 912049, "author_name": "Etienne David", "author_url": "", "post_date": "2020-07-02T07:15:10.190000", "content": "Yes, the paper track is definitely here to encourage publications of novel solutions, even if the LB score is not the best.
", "votes": 0, "replies": [] } ] }, { "id": 911147, "author_name": "Matt", "author_url": "", "post_date": "2020-07-01T15:35:19.757000", "content": "Thank you for providing this update... Just to verify I understand this correctly, the only data that will be verified is the 2nd round (38%) of the test data? If instead it is the entire private test set being updated, when will this update take place?
\n\nAlso, you missed a very important question that I have been waiting to hear back on for over three weeks! Please let me know if correcting training labels manually is permissible and if it considered as external data that must be shared with other competitors?
", "votes": 0, "replies": [ { "id": 911265, "author_name": "Phil Culliton", "author_url": "", "post_date": "2020-07-01T16:24:34.483000", "content": "Thanks for your questions.
\nCorrect, the only data that will be updated will be that 38% of data. The actual number of updated samples will be small, based on the issues we've seen.
I apologize for missing your earlier question. Thanks for your patience. Yes, it's permissible to manually correct training labels without providing them as external data.
Thank you very much for taking the time to answer. I look forward to sharing my much revised version of the training dataset after the competition! :D
", "votes": 0, "replies": [] }, { "id": 911272, "author_name": "Matt", "author_url": "", "post_date": "2020-07-01T16:30:13.923000", "content": "There is also a few questions listed in the External Data thread which I believe could use some clarification. Thanks!
", "votes": 1, "replies": [] }, { "id": 912052, "author_name": "Etienne David", "author_url": "", "post_date": "2020-07-02T07:15:44.773000", "content": "Thank you for your patience!
", "votes": 0, "replies": [] }, { "id": 912640, "author_name": "See--", "author_url": "", "post_date": "2020-07-02T16:11:34.600000", "content": "\n\n\n<20 images in test that are not the same size as the others
\n
@philculliton @bendvd How are the images with different size handled? Are they excluded from scoring? If not, please answer this question.
", "votes": 1, "replies": [] }, { "id": 913513, "author_name": "Etienne David", "author_url": "", "post_date": "2020-07-03T08:55:00.840000", "content": "They will be included. These images are in the private test set and will be corrected at the end.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "911122": "Hello all!\n\nThank you for your feedback and suggestions since the launch of the competition. We heard you and it has taken us some time to dig into the issues, thank you for your patience! \n\nWe have been working with the host to find a solution that addresses the key problems without introducing risk to the work you have done. The issues, as we understand them, are:\n- ~170 incorrectly-sized bounding boxes, out of ~190,000\n- <20 images in test that are not the same size as the others\n- A metric with top-end thresholds that are possibly too restrictive given the problem \n\nFixing all of these problems, while they are individually relatively minor, would involve a collectively massive amount of risk to the competition. Updating the training set, test set, solution, and metric could potentially break many past submissions (there are currently nearly 18,000 of these). The benefit of fixing everything, in the absence of definitive competition-breaking issues, is outweighed by the mess it would cause the community.\n\nAfter talking with the host, the critical problem is that there are bounding boxes in the test set which are incorrect. In the interests of scoring this competition as fairly as possible, we will correct the private leaderboard’s bounding boxes. Your current techniques and scores will stand, and no submissions will potentially be invalidated. Your final submissions, however, will be scored based on a more correct set of data. \n\nThere is no perfect solution for this situation, and despite our and the host’s best efforts we realize that this is not ideal. We appreciate the work you all have already put into addressing this issue. We believe that this change addresses the main risk in the competition data while allowing your current work to stand. We have learned from past competitions that even making a few changes to a competition can make small issues much worse.\n\nTogether with the host we worked hard to find the best way forward for this competition and we want to thank you all for your patience and for all of the work you have put into the competition so far.", "911589": "So it will be hard to separate wheat from chaff. ", "911142": "Thanks for the information.\n\nSo, as far as I understand: the only change is in private test dataset? Train and public test are unchanged?\n\nI'm not sure how will this help to build a robust solution.", "911222": "TL; DR\nFilter the bboxes, train, trust CV, avoid shakeup.", "912847": "I understand that the ~170 incorrectly-sized bounding boxes are going to be fixed in the private.\nMay I confirm if poorly labeled binding boxes are fixed in the private set as well?\nhttps://www.kaggle.com/c/global-wheat-detection/discussion/157009\n\nThese poor annotations hurt the CV quite much.\n", "912567": "Is the private dataset already updated?", "911275": "Thanks for your update.\nHowever, I think this change will make the ranking on public leaderboard less believable, cause the distribution of private and public test set are no longer the same. Remember this computation is also the challenge of CVPPP workshop, top 5 will have the chance to publish papers in the ECCV2020 workshop proceedings. \nSo a question comes to me, my ranking on the public leaderboard is currently around 10. Should I prepare to submit a paper to CVPPP?", "911147": "Thank you for providing this update... Just to verify I understand this correctly, the only data that will be verified is the 2nd round (38%) of the test data? If instead it is the entire private test set being updated, when will this update take place?\n\nAlso, you missed a very important question that I have been [waiting to hear back on for over three weeks](https://www.kaggle.com/c/global-wheat-detection/discussion/157009#907262)! Please let me know if correcting training labels manually is permissible and if it considered as external data that must be shared with other competitors?" } }