\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3613976%2F7494c7032ecaf2c89f0660b597455c95%2FSelection_353.png?generation=1598259043685478&alt=media\")
Dear all,
\nThanks for your immense support, which I did not expect at all!
\nThe kaggle team tried rerunning my notebooks, but failed due to missing datasets. I reran one of the chosen notebooks as a late submission, and the results would get me a silver medal.
\n
However, there is no way I can provide a solid proof that the model that I used for this was the same model that I used on the old dataset because I uploaded the same model again couple of days before the first deadline (though they have the same name \"sota_75.pth\", not enough facts).
\nSo there is nothing the kaggle team can do as it would be unfair for them to accept this solution.
\nI wish the kaggle team and the hosts were more interactive throughout and even after the competition. I received an explanation only after I received such a massive support. The participants of the competition thought that the private set was already updated when the first results were revealed giving false hopes to the winners (I was in top 50), and if it was updated before the deadline or if the hosts and the kaggle team let us know to save all the required datasets, there would be no such problems. I am glad that the kaggle team removed almost all of the yolov5 submissions though.
\n3 months of hard work and receiving such an unsatisfactory and unfair conclusion makes you feel empty, and I am sure there are more participants like me …
", "messages": [ { "id": 983430, "postDate": "2020-08-24T09:56:03.130Z", "content": "Dear all,
\nThanks for your immense support, which I did not expect at all!
\nThe kaggle team tried rerunning my notebooks, but failed due to missing datasets. I reran one of the chosen notebooks as a late submission, and the results would get me a silver medal.
\n
However, there is no way I can provide a solid proof that the model that I used for this was the same model that I used on the old dataset because I uploaded the same model again couple of days before the first deadline (though they have the same name \"sota_75.pth\", not enough facts).
\nSo there is nothing the kaggle team can do as it would be unfair for them to accept this solution.
\nI wish the kaggle team and the hosts were more interactive throughout and even after the competition. I received an explanation only after I received such a massive support. The participants of the competition thought that the private set was already updated when the first results were revealed giving false hopes to the winners (I was in top 50), and if it was updated before the deadline or if the hosts and the kaggle team let us know to save all the required datasets, there would be no such problems. I am glad that the kaggle team removed almost all of the yolov5 submissions though.
\n3 months of hard work and receiving such an unsatisfactory and unfair conclusion makes you feel empty, and I am sure there are more participants like me …
", "rawMarkdown": "Dear all,\n\nThanks for your immense support, which I did not expect at all! \n\nThe kaggle team tried rerunning my notebooks, but failed due to missing datasets. I reran one of the chosen notebooks as a late submission, and the results would get me a silver medal. \n\n\nHowever, there is no way I can provide a solid proof that the model that I used for this was the same model that I used on the old dataset because I uploaded the same model again couple of days before the first deadline (though they have the same name \"sota_75.pth\", not enough facts).\n\nSo there is nothing the kaggle team can do as it would be unfair for them to accept this solution.\n\nI wish the kaggle team and the hosts were more interactive throughout and even after the competition. I received an explanation only after I received such a massive support. The participants of the competition thought that the private set was already updated when the first results were revealed giving false hopes to the winners (I was in top 50), and if it was updated before the deadline or if the hosts and the kaggle team let us know to save all the required datasets, there would be no such problems. I am glad that the kaggle team removed almost all of the yolov5 submissions though.\n\n3 months of hard work and receiving such an unsatisfactory and unfair conclusion makes you feel empty, and I am sure there are more participants like me ... ", "votes": 38 }, { "id": 983560, "postDate": "2020-08-24T12:05:13.657Z", "content": "Although it took the collective support of the entire Kaggle community to help extract it, I'm glad you at least received some sort of communication back.
\n\n\nSo there is nothing the kaggle team can do
\n
🤔 But there are very simple things Kaggle can do:
\nsubmission.csv for the test set. They can still look into those those files. Forget data science; even from a basic engineering standpoint, doing so is a lot less resource costly than rerunning 2245*2 kernels.A lot of work needs to go into the UX for code-competitions. Clarifying that datasets can't be removed (even after the submission deadline of the competition) would have gone a long way. In fact, just disabling their deletion. Allowing a dataset to get deleted is like manually allowing a submission to get altered after the competitions end. Any of us would have gladly made our datasets public rather than deleting them, since the goal was just to free up private-dataset space so we could participate in the two other concurrently running comps that were open at the time.
\nThe way all this was mishandled has left an extremely sour taste for Kaggle in my mouth. Especially the communication aspect.
", "rawMarkdown": "Although it took the collective support of the entire Kaggle community to help extract it, I'm glad you at least received some sort of communication back.\n\n> So there is nothing the kaggle team can do\n\n🤔 But there **are** very simple things Kaggle can do:\n\n1. Examining the inference code of your notebook is sufficient to determine the model architecture you used because it is not possible to load up yolov5 weights into an effdet model. In your case, this verification would have to be done with the version of the model submitted before the original competition deadline; [in my case](https://www.kaggle.com/c/global-wheat-detection/discussion/175935), I never ever even resubmitted / reuploaded the dataset.\n1. Over the course of three months, you've made numerous submissions against the LB. I am sure you didn't delete the datasets of ALL of them. Does it make sense to completely annihilate your work—as if you were a dirty cheater—versus give you a rank based upon the private score corresponding to your highest public LB position?\n1. What is the purpose of re-running the notebooks, especially 18 days after the conclusion of the competition? Unless the actual GT labeling changed, which is highly unlikely (I don't know— @stavness did you guys relabel the private dataset +2week after the competition?) then the recorded submission score of the kernel is more than sufficient 'proof'. In fact, it's the very best proof that exist. What are we trying to prove here anyway? Only the in-the-money winners have the burden of reproducibility. Heck, even if kaggle decided \"hey, let's only use a subset of the predicted values to calculate the final score\", each kernel persists its `submission.csv` for the test set. They can still look into those those files. Forget data science; even from a basic engineering standpoint, doing so is a lot less resource costly than rerunning 2245*2 kernels.\n\nA lot of work needs to go into the UX for code-competitions. Clarifying that datasets can't be removed (even after the submission deadline of the competition) would have gone a long way. In fact, just disabling their deletion. Allowing a dataset to get deleted is like manually allowing a submission to get altered after the competitions end. Any of us would have gladly made our datasets public rather than deleting them, since the goal was just to free up private-dataset space so we could participate in the two other concurrently running comps that were open at the time.\n\nThe way all this was mishandled has left an extremely sour taste for Kaggle in my mouth. Especially the communication aspect.", "votes": 12, "replies": [ { "id": 983581, "postDate": "2020-08-24T12:19:00.603Z", "content": "It's really hard to understand why not just recalculate the results from our final submitted CSV files.
", "rawMarkdown": "It's really hard to understand why not just recalculate the results from our final submitted CSV files.", "votes": 1 }, { "id": 983617, "postDate": "2020-08-24T12:57:53.380Z", "content": "Thanks for your suggestions! The current kaggle staff in this competition are reluctant to make any changes. Just received a message, which basically says they cannot reverse their decision. Maybe other kaggle staff could help, lots of people offered to contact @inversion
", "rawMarkdown": "Thanks for your suggestions! The current kaggle staff in this competition are reluctant to make any changes. Just received a message, which basically says they cannot reverse their decision. Maybe other kaggle staff could help, lots of people offered to contact @inversion ", "votes": 1 }, { "id": 983694, "postDate": "2020-08-24T14:16:10Z", "content": "Perhaps I can give some further details.
\nFirst, requests like this take some time to process, requiring a number of people to coordinate and investigate. And the initial request came in the middle an unusually heavy compliance workload, and then continued discussion into the weekend. Apologies for the slow response. We always try to do better in this regard.
\n\n\nClarifying that datasets can't be removed . . . or lust disabling their deletion.
\n
It would be good to have something that clarifies when there is risk of removing a dataset linked to a re-run competition. But we wouldn't prevent it, as that is considered user data, and the user is free to delete user data at any time.
\n\n\nespecially 18 days after the conclusion of the competition
\n
It wouldn't have mattered. The datasets were deleted right after submissions were selected. 4 datasets were created for the first submission. Then they were deleted. Then 4 more datasets were created for the second submission, and then deleted. This caused the models to fail on re-run.
\nUnlike the original thread discussing this, no one was \"removed for being honest\". The situation is that we are absolutely not going attempt to manually replicate a person's code that failed, particularly when we don't have the original datasets. Is it unfortunate? Yes. But it should be clear to everyone that there are multiple reasons why this isn't viable.
", "rawMarkdown": "Perhaps I can give some further details. \n\nFirst, requests like this take some time to process, requiring a number of people to coordinate and investigate. And the initial request came in the middle an unusually heavy compliance workload, and then continued discussion into the weekend. Apologies for the slow response. We always try to do better in this regard.\n\n> Clarifying that datasets can't be removed . . . or lust disabling their deletion. \n\nIt would be good to have something that clarifies when there is risk of removing a dataset linked to a re-run competition. But we wouldn't prevent it, as that is considered user data, and the user is free to delete user data at any time.\n\n> especially 18 days after the conclusion of the competition\n\nIt wouldn't have mattered. The datasets were deleted right after submissions were selected. 4 datasets were created for the first submission. Then they were deleted. Then 4 more datasets were created for the second submission, and then deleted. This caused the models to fail on re-run.\n\nUnlike the original thread discussing this, no one was \"removed for being honest\". The situation is that we are absolutely not going attempt to manually replicate a person's code that failed, particularly when we don't have the original datasets. Is it unfortunate? Yes. But it should be clear to everyone that there are multiple reasons why this isn't viable.", "votes": 2 }, { "id": 983768, "postDate": "2020-08-24T15:25:01.597Z", "content": "The reference about 18 days after was specific to my case, where I deleted my wheat models dataset closer to the end of Melanoma competition, in order to make room for it. If a dset is deleted right after a submission (before the competition closes), well then of course it cannot be re-ran upon the close of said competition. But in my case I deleted them weeks after the reported submission end timeline (Aug 4th) of this comp; in fact, days after they'd already been reran against private lb multiple times for the various cleanups.
", "rawMarkdown": "The reference about 18 days after was specific to my case, where I deleted my wheat models dataset closer to the end of *Melanoma* competition, in order to make room for it. If a dset is deleted right after a submission (before the competition closes), well then of course it cannot be re-ran upon the close of said competition. But in my case I deleted them weeks after the reported submission end timeline (Aug 4th) of this comp; in fact, days after they'd already been reran against private lb multiple times for the various cleanups.", "votes": 4 }, { "id": 983825, "postDate": "2020-08-24T16:06:37.237Z", "content": "Hi, our team never deleted our datasets, we just turned off the versioning during the re-run period. We now have the similar situation like them. For our case, the unversioned dataset was actually what we used in the selected two submissions. We do not have the problem of proving the dataset is the original one because it is never changed. Now we turn the setting back, is there still no way to re-run our submissions? See here for more details.
", "rawMarkdown": "Hi, our team never deleted our datasets, we just turned off the versioning during the re-run period. We now have the similar situation like them. For our case, the unversioned dataset was actually what we used in the selected two submissions. We do not have the problem of proving the dataset is the original one because it is never changed. Now we turn the setting back, is there still no way to re-run our submissions? See [here](https://www.kaggle.com/c/global-wheat-detection/discussion/176066) for more details.", "votes": 3 }, { "id": 984305, "postDate": "2020-08-25T03:03:38.557Z", "content": "@inversion Could you have a look at our team's situation here?
\n\n\nour team never deleted our datasets, we just turned off the versioning during the re-run period. We now have the similar situation like them. For our case, the unversioned dataset was actually what we used in the selected two submissions. We do not have the problem of proving the dataset is the original one because it is never changed. Now we turn the setting back, is there still no way to re-run our submissions? See here for more details.
\n
We send the email to compliance team ,but it seems that they didn't read the details of our feedback and just give the same reply email they sent to other teams. Except that, we didn't receive any offical reply in the kaggle forum.
\nWe spent lost of time in this competition and it's so sad to waiting a such long time without any positive reply.
", "rawMarkdown": "@inversion Could you have a look at our team's situation here?\n> our team never deleted our datasets, we just turned off the versioning during the re-run period. We now have the similar situation like them. For our case, the unversioned dataset was actually what we used in the selected two submissions. We do not have the problem of proving the dataset is the original one because it is never changed. Now we turn the setting back, is there still no way to re-run our submissions? See here for more details.\n\nWe send the email to [compliance team](https://www.kaggle.com/compliance) ,but it seems that they didn't read the details of our feedback and just give the same reply email they sent to other teams. Except that, we didn't receive any offical reply in the kaggle forum.\n\nWe spent lost of time in this competition and it's so sad to waiting a such long time without any positive reply.\n", "votes": 2 }, { "id": 984308, "postDate": "2020-08-25T03:13:06.007Z", "content": "What's more, there are several teams met the same situation in the forum. I think we all deserve a more detailed explanation.
", "rawMarkdown": "What's more, there are several teams met the same situation in the forum. I think we all deserve a more detailed explanation.", "votes": 3 }, { "id": 984327, "postDate": "2020-08-25T03:32:11.217Z", "content": "\n\nFirst, requests like this take some time to process, requiring a number of people to coordinate and investigate. And the initial request came in the middle an unusually heavy compliance workload, and then continued discussion into the weekend.
\n
Thanks so much for the effort for removing uncompliance submission.
\nI can't agree more that it's import to remove those uncompliance submission. But I think the deleted decision for those who didn't submit yolov5 should be made more carefully.Because each submission means servarl monsths of efforts.And the dataset shouldn't change is never mention before.
", "rawMarkdown": "> First, requests like this take some time to process, requiring a number of people to coordinate and investigate. And the initial request came in the middle an unusually heavy compliance workload, and then continued discussion into the weekend. \n\nThanks so much for the effort for removing uncompliance submission.\n\nI can't agree more that it's import to remove those uncompliance submission. But I think the deleted decision for those who didn't submit yolov5 should be made more carefully.Because each submission means servarl monsths of efforts.And the dataset shouldn't change is never mention before.", "votes": 2 } ] }, { "id": 983535, "postDate": "2020-08-24T11:43:49.493Z", "content": "Congratulations with first silver medal!
", "rawMarkdown": "Congratulations with first silver medal!", "votes": 3, "replies": [ { "id": 983580, "postDate": "2020-08-24T12:18:53.867Z", "content": "What silver medal?
", "rawMarkdown": "What silver medal?", "votes": 3 }, { "id": 983604, "postDate": "2020-08-24T12:44:26.253Z", "content": "Thanks for your support, Anuar! I wish the result was reflected in the leaderboard …
", "rawMarkdown": "Thanks for your support, Anuar! I wish the result was reflected in the leaderboard ...", "votes": 4 }, { "id": 983607, "postDate": "2020-08-24T12:45:37.480Z", "content": "@authman which has been stolen
", "rawMarkdown": "@authman which has been stolen", "votes": 3 } ] }, { "id": 983774, "postDate": "2020-08-24T15:31:30.307Z", "content": "Congrats @aidynub and I want to see you in other competitions.
", "rawMarkdown": "Congrats @aidynub and I want to see you in other competitions.\n", "votes": 1 }, { "id": 986625, "postDate": "2020-08-26T16:30:57.857Z", "content": "This competition host is pretty disappointing…but good for you to get a silver medal.
", "rawMarkdown": "This competition host is pretty disappointing...but good for you to get a silver medal.", "votes": -2 }, { "id": 992046, "postDate": "2020-08-30T21:06:21.330Z", "content": "I'm sorry for you. I also got kicked from Melanoma competition for sharing a notebook to someone who I wanted to look over my notebook to see if he can find my PyTorch XLA error (if you worked with PyTorch XLA, you know how buggy that thing is) I never had a bad intention nor I wanted to cheat in anyway but Kaggle support never solved the problem with me, they just think that I \"cheated\" and I cant prove that I did not….Eventhough I was \"only\" in Top 30% I wanted to reflect my score on my profile. Seems like they always handle the problems like that. I wish you all the best for your next competition.
", "rawMarkdown": "I'm sorry for you. I also got kicked from Melanoma competition for sharing a notebook to someone who I wanted to look over my notebook to see if he can find my PyTorch XLA error (if you worked with PyTorch XLA, you know how buggy that thing is) I never had a bad intention nor I wanted to cheat in anyway but Kaggle support never solved the problem with me, they just think that I \"cheated\" and I cant prove that I did not....Eventhough I was \"only\" in Top 30% I wanted to reflect my score on my profile. Seems like they always handle the problems like that. I wish you all the best for your next competition." }, { "id": 990086, "postDate": "2020-08-29T10:33:19.503Z", "content": "I'm sorry to read it went this way, losing all that work must be really hard :-/
", "rawMarkdown": "I'm sorry to read it went this way, losing all that work must be really hard :-/" }, { "id": 989212, "postDate": "2020-08-28T16:14:40.367Z", "content": "Hopefully a learning experience for the Kaggle team in communication with users, but at the very least they did get to it
", "rawMarkdown": "Hopefully a learning experience for the Kaggle team in communication with users, but at the very least they did get to it" }, { "id": 991431, "postDate": "2020-08-30T12:10:01.413Z", "rawMarkdown": "", "isDeleted": true }, { "id": 987474, "postDate": "2020-08-27T09:45:39.610Z", "rawMarkdown": "", "votes": -3, "isDeleted": true }, { "id": 987342, "postDate": "2020-08-27T07:09:24.180Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 983560, "author_name": "عثمان", "author_url": "", "post_date": "2020-08-24T12:05:13.657000", "content": "Although it took the collective support of the entire Kaggle community to help extract it, I'm glad you at least received some sort of communication back.
\n\n\nSo there is nothing the kaggle team can do
\n
🤔 But there are very simple things Kaggle can do:
\nsubmission.csv for the test set. They can still look into those those files. Forget data science; even from a basic engineering standpoint, doing so is a lot less resource costly than rerunning 2245*2 kernels.A lot of work needs to go into the UX for code-competitions. Clarifying that datasets can't be removed (even after the submission deadline of the competition) would have gone a long way. In fact, just disabling their deletion. Allowing a dataset to get deleted is like manually allowing a submission to get altered after the competitions end. Any of us would have gladly made our datasets public rather than deleting them, since the goal was just to free up private-dataset space so we could participate in the two other concurrently running comps that were open at the time.
\nThe way all this was mishandled has left an extremely sour taste for Kaggle in my mouth. Especially the communication aspect.
", "votes": 12, "replies": [ { "id": 983581, "author_name": "Lost Stars5", "author_url": "", "post_date": "2020-08-24T12:19:00.603000", "content": "It's really hard to understand why not just recalculate the results from our final submitted CSV files.
", "votes": 1, "replies": [] }, { "id": 983617, "author_name": "Aidyn Ubingazhibov", "author_url": "", "post_date": "2020-08-24T12:57:53.380000", "content": "Thanks for your suggestions! The current kaggle staff in this competition are reluctant to make any changes. Just received a message, which basically says they cannot reverse their decision. Maybe other kaggle staff could help, lots of people offered to contact @inversion
", "votes": 1, "replies": [] }, { "id": 983694, "author_name": "inversion", "author_url": "", "post_date": "2020-08-24T14:16:10", "content": "Perhaps I can give some further details.
\nFirst, requests like this take some time to process, requiring a number of people to coordinate and investigate. And the initial request came in the middle an unusually heavy compliance workload, and then continued discussion into the weekend. Apologies for the slow response. We always try to do better in this regard.
\n\n\nClarifying that datasets can't be removed . . . or lust disabling their deletion.
\n
It would be good to have something that clarifies when there is risk of removing a dataset linked to a re-run competition. But we wouldn't prevent it, as that is considered user data, and the user is free to delete user data at any time.
\n\n\nespecially 18 days after the conclusion of the competition
\n
It wouldn't have mattered. The datasets were deleted right after submissions were selected. 4 datasets were created for the first submission. Then they were deleted. Then 4 more datasets were created for the second submission, and then deleted. This caused the models to fail on re-run.
\nUnlike the original thread discussing this, no one was \"removed for being honest\". The situation is that we are absolutely not going attempt to manually replicate a person's code that failed, particularly when we don't have the original datasets. Is it unfortunate? Yes. But it should be clear to everyone that there are multiple reasons why this isn't viable.
", "votes": 2, "replies": [] }, { "id": 983768, "author_name": "عثمان", "author_url": "", "post_date": "2020-08-24T15:25:01.597000", "content": "The reference about 18 days after was specific to my case, where I deleted my wheat models dataset closer to the end of Melanoma competition, in order to make room for it. If a dset is deleted right after a submission (before the competition closes), well then of course it cannot be re-ran upon the close of said competition. But in my case I deleted them weeks after the reported submission end timeline (Aug 4th) of this comp; in fact, days after they'd already been reran against private lb multiple times for the various cleanups.
", "votes": 4, "replies": [] }, { "id": 983825, "author_name": "Qingyuan Wang", "author_url": "", "post_date": "2020-08-24T16:06:37.237000", "content": "Hi, our team never deleted our datasets, we just turned off the versioning during the re-run period. We now have the similar situation like them. For our case, the unversioned dataset was actually what we used in the selected two submissions. We do not have the problem of proving the dataset is the original one because it is never changed. Now we turn the setting back, is there still no way to re-run our submissions? See here for more details.
", "votes": 3, "replies": [] }, { "id": 984305, "author_name": "Dapeng", "author_url": "", "post_date": "2020-08-25T03:03:38.557000", "content": "@inversion Could you have a look at our team's situation here?
\n\n\nour team never deleted our datasets, we just turned off the versioning during the re-run period. We now have the similar situation like them. For our case, the unversioned dataset was actually what we used in the selected two submissions. We do not have the problem of proving the dataset is the original one because it is never changed. Now we turn the setting back, is there still no way to re-run our submissions? See here for more details.
\n
We send the email to compliance team ,but it seems that they didn't read the details of our feedback and just give the same reply email they sent to other teams. Except that, we didn't receive any offical reply in the kaggle forum.
\nWe spent lost of time in this competition and it's so sad to waiting a such long time without any positive reply.
", "votes": 2, "replies": [] }, { "id": 984308, "author_name": "Dapeng", "author_url": "", "post_date": "2020-08-25T03:13:06.007000", "content": "What's more, there are several teams met the same situation in the forum. I think we all deserve a more detailed explanation.
", "votes": 3, "replies": [] }, { "id": 984327, "author_name": "Dapeng", "author_url": "", "post_date": "2020-08-25T03:32:11.217000", "content": "\n\nFirst, requests like this take some time to process, requiring a number of people to coordinate and investigate. And the initial request came in the middle an unusually heavy compliance workload, and then continued discussion into the weekend.
\n
Thanks so much for the effort for removing uncompliance submission.
\nI can't agree more that it's import to remove those uncompliance submission. But I think the deleted decision for those who didn't submit yolov5 should be made more carefully.Because each submission means servarl monsths of efforts.And the dataset shouldn't change is never mention before.
", "votes": 2, "replies": [] } ] }, { "id": 983535, "author_name": "Aimoldin Anuar [dsmlkz]", "author_url": "", "post_date": "2020-08-24T11:43:49.493000", "content": "Congratulations with first silver medal!
", "votes": 3, "replies": [ { "id": 983580, "author_name": "عثمان", "author_url": "", "post_date": "2020-08-24T12:18:53.867000", "content": "What silver medal?
", "votes": 3, "replies": [] }, { "id": 983604, "author_name": "Aidyn Ubingazhibov", "author_url": "", "post_date": "2020-08-24T12:44:26.253000", "content": "Thanks for your support, Anuar! I wish the result was reflected in the leaderboard …
", "votes": 4, "replies": [] }, { "id": 983607, "author_name": "Zhanseri Ikram", "author_url": "", "post_date": "2020-08-24T12:45:37.480000", "content": "@authman which has been stolen
", "votes": 3, "replies": [] } ] }, { "id": 983774, "author_name": "Heroseo", "author_url": "", "post_date": "2020-08-24T15:31:30.307000", "content": "Congrats @aidynub and I want to see you in other competitions.
", "votes": 1, "replies": [] }, { "id": 986625, "author_name": "豆柴金鯱", "author_url": "", "post_date": "2020-08-26T16:30:57.857000", "content": "This competition host is pretty disappointing…but good for you to get a silver medal.
", "votes": -2, "replies": [] }, { "id": 992046, "author_name": "Ali Abdin", "author_url": "", "post_date": "2020-08-30T21:06:21.330000", "content": "I'm sorry for you. I also got kicked from Melanoma competition for sharing a notebook to someone who I wanted to look over my notebook to see if he can find my PyTorch XLA error (if you worked with PyTorch XLA, you know how buggy that thing is) I never had a bad intention nor I wanted to cheat in anyway but Kaggle support never solved the problem with me, they just think that I \"cheated\" and I cant prove that I did not….Eventhough I was \"only\" in Top 30% I wanted to reflect my score on my profile. Seems like they always handle the problems like that. I wish you all the best for your next competition.
", "votes": 0, "replies": [] }, { "id": 990086, "author_name": "Elena Itzcovich", "author_url": "", "post_date": "2020-08-29T10:33:19.503000", "content": "I'm sorry to read it went this way, losing all that work must be really hard :-/
", "votes": 0, "replies": [] }, { "id": 989212, "author_name": "///", "author_url": "", "post_date": "2020-08-28T16:14:40.367000", "content": "Hopefully a learning experience for the Kaggle team in communication with users, but at the very least they did get to it
", "votes": 0, "replies": [] }, { "id": 991431, "author_name": "", "author_url": "", "post_date": "2020-08-30T12:10:01.413000", "content": "", "votes": 0, "replies": [] }, { "id": 987474, "author_name": "", "author_url": "", "post_date": "2020-08-27T09:45:39.610000", "content": "", "votes": -3, "replies": [] }, { "id": 987342, "author_name": "", "author_url": "", "post_date": "2020-08-27T07:09:24.180000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "983430": "Dear all,\n\nThanks for your immense support, which I did not expect at all! \n\nThe kaggle team tried rerunning my notebooks, but failed due to missing datasets. I reran one of the chosen notebooks as a late submission, and the results would get me a silver medal. \n\n\nHowever, there is no way I can provide a solid proof that the model that I used for this was the same model that I used on the old dataset because I uploaded the same model again couple of days before the first deadline (though they have the same name \"sota_75.pth\", not enough facts).\n\nSo there is nothing the kaggle team can do as it would be unfair for them to accept this solution.\n\nI wish the kaggle team and the hosts were more interactive throughout and even after the competition. I received an explanation only after I received such a massive support. The participants of the competition thought that the private set was already updated when the first results were revealed giving false hopes to the winners (I was in top 50), and if it was updated before the deadline or if the hosts and the kaggle team let us know to save all the required datasets, there would be no such problems. I am glad that the kaggle team removed almost all of the yolov5 submissions though.\n\n3 months of hard work and receiving such an unsatisfactory and unfair conclusion makes you feel empty, and I am sure there are more participants like me ... ", "983560": "Although it took the collective support of the entire Kaggle community to help extract it, I'm glad you at least received some sort of communication back.\n\n> So there is nothing the kaggle team can do\n\n🤔 But there **are** very simple things Kaggle can do:\n\n1. Examining the inference code of your notebook is sufficient to determine the model architecture you used because it is not possible to load up yolov5 weights into an effdet model. In your case, this verification would have to be done with the version of the model submitted before the original competition deadline; [in my case](https://www.kaggle.com/c/global-wheat-detection/discussion/175935), I never ever even resubmitted / reuploaded the dataset.\n1. Over the course of three months, you've made numerous submissions against the LB. I am sure you didn't delete the datasets of ALL of them. Does it make sense to completely annihilate your work—as if you were a dirty cheater—versus give you a rank based upon the private score corresponding to your highest public LB position?\n1. What is the purpose of re-running the notebooks, especially 18 days after the conclusion of the competition? Unless the actual GT labeling changed, which is highly unlikely (I don't know— @stavness did you guys relabel the private dataset +2week after the competition?) then the recorded submission score of the kernel is more than sufficient 'proof'. In fact, it's the very best proof that exist. What are we trying to prove here anyway? Only the in-the-money winners have the burden of reproducibility. Heck, even if kaggle decided \"hey, let's only use a subset of the predicted values to calculate the final score\", each kernel persists its `submission.csv` for the test set. They can still look into those those files. Forget data science; even from a basic engineering standpoint, doing so is a lot less resource costly than rerunning 2245*2 kernels.\n\nA lot of work needs to go into the UX for code-competitions. Clarifying that datasets can't be removed (even after the submission deadline of the competition) would have gone a long way. In fact, just disabling their deletion. Allowing a dataset to get deleted is like manually allowing a submission to get altered after the competitions end. Any of us would have gladly made our datasets public rather than deleting them, since the goal was just to free up private-dataset space so we could participate in the two other concurrently running comps that were open at the time.\n\nThe way all this was mishandled has left an extremely sour taste for Kaggle in my mouth. Especially the communication aspect.", "983535": "Congratulations with first silver medal!", "983774": "Congrats @aidynub and I want to see you in other competitions.\n", "986625": "This competition host is pretty disappointing...but good for you to get a silver medal.", "992046": "I'm sorry for you. I also got kicked from Melanoma competition for sharing a notebook to someone who I wanted to look over my notebook to see if he can find my PyTorch XLA error (if you worked with PyTorch XLA, you know how buggy that thing is) I never had a bad intention nor I wanted to cheat in anyway but Kaggle support never solved the problem with me, they just think that I \"cheated\" and I cant prove that I did not....Eventhough I was \"only\" in Top 30% I wanted to reflect my score on my profile. Seems like they always handle the problems like that. I wish you all the best for your next competition.", "990086": "I'm sorry to read it went this way, losing all that work must be really hard :-/", "989212": "Hopefully a learning experience for the Kaggle team in communication with users, but at the very least they did get to it", "991431": "", "987474": "", "987342": "" } }