@Admins and Kaggle Management
\n\nIt is clear that the code upload rule consists of two parts:
\n\nFirst part is the code upload timeline, that stands for all participants.
\n\nSecond part is that in accordance with the FAQ if you don't upload code then you can still be on the leaderboard, except for the case that you finish in a prize winning position and your submission will be removed.
\n\nIt seems the rule has been enforced only partially, without previous notice and retrospectively. If the change was published in time then all participants would be discouraged to not upload their model and would almost certainly upload their code, because getting results in such a competition requires effort, time and money, and wasting all that for such a simple thing is irrational.
\n\nWhere can we appeal this decision and where can we formally express our disagreement about the consistency of your decisions?
\n\n@All kagglers who disagree with the removal decision please upvote and appeal.
\n\nEdit: The fact that by removing all participants, the top left pixel benchmark earns a bronze medal and sufficient ranking points gives an idea about the admins' decision and the suitability of this competition's results.
\n\nEdit2:\nIt was not included in the rules https://www.kaggle.com/c/data-science-bowl-2018/rules
\n\nIt was not included in the FAQ https://www.kaggle.com/two-stage-faq
\n\n\n\n\nWill I still be on the leaderboard if I don't upload a model but do\n submit in the second stage?
\n \nYes. However, if you do not upload a model and finish in a prize\n position, your team will be removed from the competition standings\n entirely.
\n
Ex post facto laws are expressly forbidden by the United States Constitution in Article 1, Section 9, Clause 3 (with respect to federal laws) and Article 1, Section 10 (with respect to state laws).
", "messages": [ { "id": 317318, "postDate": "2018-04-21T08:01:29.997Z", "content": "@Admins and Kaggle Management
\n\nIt is clear that the code upload rule consists of two parts:
\n\nFirst part is the code upload timeline, that stands for all participants.
\n\nSecond part is that in accordance with the FAQ if you don't upload code then you can still be on the leaderboard, except for the case that you finish in a prize winning position and your submission will be removed.
\n\nIt seems the rule has been enforced only partially, without previous notice and retrospectively. If the change was published in time then all participants would be discouraged to not upload their model and would almost certainly upload their code, because getting results in such a competition requires effort, time and money, and wasting all that for such a simple thing is irrational.
\n\nWhere can we appeal this decision and where can we formally express our disagreement about the consistency of your decisions?
\n\n@All kagglers who disagree with the removal decision please upvote and appeal.
\n\nEdit: The fact that by removing all participants, the top left pixel benchmark earns a bronze medal and sufficient ranking points gives an idea about the admins' decision and the suitability of this competition's results.
\n\nEdit2:\nIt was not included in the rules https://www.kaggle.com/c/data-science-bowl-2018/rules
\n\nIt was not included in the FAQ https://www.kaggle.com/two-stage-faq
\n\n\n\n\nWill I still be on the leaderboard if I don't upload a model but do\n submit in the second stage?
\n \nYes. However, if you do not upload a model and finish in a prize\n position, your team will be removed from the competition standings\n entirely.
\n
Ex post facto laws are expressly forbidden by the United States Constitution in Article 1, Section 9, Clause 3 (with respect to federal laws) and Article 1, Section 10 (with respect to state laws).
", "rawMarkdown": "@Admins and Kaggle Management\n\nIt is clear that the code upload rule consists of two parts: \n\nFirst part is the code upload timeline, that stands for all participants. \n\nSecond part is that in accordance with the FAQ if you don't upload code then you can still be on the leaderboard, except for the case that you finish in a prize winning position and your submission will be removed.\n\nIt seems the rule has been enforced only partially, without previous notice and retrospectively. If the change was published in time then all participants would be discouraged to not upload their model and would almost certainly upload their code, because getting results in such a competition requires effort, time and money, and wasting all that for such a simple thing is irrational. \n\nWhere can we appeal this decision and where can we formally express our disagreement about the consistency of your decisions?\n\n@All kagglers who disagree with the removal decision please upvote and appeal.\n\nEdit: The fact that by removing all participants, the top left pixel benchmark earns a bronze medal and sufficient ranking points gives an idea about the admins' decision and the suitability of this competition's results.\n\nEdit2:\nIt was not included in the rules https://www.kaggle.com/c/data-science-bowl-2018/rules\n\nIt was not included in the FAQ https://www.kaggle.com/two-stage-faq\n\n> Will I still be on the leaderboard if I don't upload a model but do\n> submit in the second stage?\n> \n> Yes. However, if you do not upload a model and finish in a prize\n> position, your team will be removed from the competition standings\n> entirely.\n\nEx post facto laws are expressly forbidden by the United States Constitution in Article 1, Section 9, Clause 3 (with respect to federal laws) and Article 1, Section 10 (with respect to state laws).", "votes": 74 }, { "id": 317362, "postDate": "2018-04-21T09:37:12.460Z", "content": "The rules haven't changed. Last year DSB had the same rules, same timeline wordings, non-winners were not removed for not uploading models. On top of that, current FAQ states:
\n\n\n\n\nWill I still be on the leaderboard if I don't upload a model but do submit in the second stage?\n Yes. However, if you do not upload a model and finish in a prize position, your team will be removed from the competition standings entirely.
\n
Given these two facts, one could easily think above statement should hold under current rules. And it had been always like that for recent two stage competitions.
\n\nWhy Kaggle now chose to inforce differently for the same RULES, without ANY NOTICE? One Kaggle admin mentioned they couldn't implement this in previous competitions due to administrative reasons but why not NOTIFYING us from the beginning that something is changed? OR is there any hidden reason for the change AFTER the competition ended?
\n\nIn addition, the requirement for model uploading is not even put under RULES tab, but under timeline. Since it's not under RULES, it's only for those who want the win the prize right? If it's for all, why didn't they bother putting that important piece of information under RULES tab in the first place, given that the RULES also have a timeline?
\n\nMore than half number of teams are removed just shows something very wrong from the organizing point of view. We need some proper explanations.
\n\n@William, @Wendy
", "rawMarkdown": "The rules haven't changed. Last year DSB had the same rules, [same timeline wordings][1], non-winners were not removed for not uploading models. On top of that, current FAQ states:\n\n> Will I still be on the leaderboard if I don't upload a model but do submit in the second stage?\nYes. However, if you do not upload a model and finish in a prize position, your team will be removed from the competition standings entirely.\n\nGiven these two facts, one could easily think above statement should hold under current rules. And it had been always like that for recent two stage competitions. \n\nWhy Kaggle now chose to inforce differently for the same RULES, without ANY NOTICE? One Kaggle admin mentioned they couldn't implement this in previous competitions due to administrative reasons but why not NOTIFYING us from the beginning that something is changed? OR is there any hidden reason for the change AFTER the competition ended?\n\n\nIn addition, the requirement for model uploading is not even put under RULES tab, but under timeline. Since it's not under RULES, it's only for those who want the win the prize right? If it's for all, why didn't they bother putting that important piece of information under RULES tab in the first place, given that the RULES also have a timeline?\n\n\nMore than half number of teams are removed just shows something very wrong from the organizing point of view. We need some proper explanations. \n\n@William, @Wendy\n\n\n [1]: https://www.kaggle.com/c/data-science-bowl-2017#timeline", "votes": 16 }, { "id": 317325, "postDate": "2018-04-21T08:06:47.667Z", "content": "I had the same idea, just removed my post to not dilute impact. I have not been removed, but I support this, because the rules + FAQ were ambiguous. It is because of this ambiguity that I wrote my post on 'Cheating is encouraged'. The new rule is great IMHO, but it should be enforced in future competitions, not this one, because it has been decided after stage 1 ended.
", "rawMarkdown": "I had the same idea, just removed my post to not dilute impact. I have not been removed, but I support this, because the rules + FAQ were ambiguous. It is because of this ambiguity that I wrote my post on 'Cheating is encouraged'. The new rule is great IMHO, but it should be enforced in future competitions, not this one, because it has been decided after stage 1 ended.", "votes": 16, "replies": [ { "id": 317329, "postDate": "2018-04-21T08:10:57.883Z", "content": "@CPMP your agreement is respected, since you were the topic starter for the need to change the FAQ. The main issue is that they didn't change it in time and they now do it retrospectively.
", "rawMarkdown": "@CPMP your agreement is respected, since you were the topic starter for the need to change the FAQ. The main issue is that they didn't change it in time and they now do it retrospectively.", "votes": 3 }, { "id": 317331, "postDate": "2018-04-21T08:12:58.183Z", "content": "\n\n\nThe main issue is that they didn't change it in time and they now do it retrospectively.
\n
I agree, I wrote that too in the comment of my other post, see https://www.kaggle.com/c/data-science-bowl-2018/discussion/54575#316064
\n\nI fully support the change, but for next competition, not this one. I wish they has changed their FAQ in time to avoid all this mess.
", "rawMarkdown": "> The main issue is that they didn't change it in time and they now do it retrospectively.\n\nI agree, I wrote that too in the comment of my other post, see https://www.kaggle.com/c/data-science-bowl-2018/discussion/54575#316064\n\nI fully support the change, but for next competition, not this one. I wish they has changed their FAQ in time to avoid all this mess.", "votes": 5 }, { "id": 317364, "postDate": "2018-04-21T09:39:11.170Z", "content": "CPMP, when you first made your suggestion, I strongly disagreed - it rankled me that your default assumption is that everyone intends to cheat (it still does). Punishing people for honest bugfixes is as much against the spirit of friendly competition as, say, secretively adjusting your training methods.
\n\nThat change would make Kaggle a somewhat less friendly place. But now I can see the value in it.
", "rawMarkdown": "CPMP, when you first made your suggestion, I strongly disagreed - it rankled me that your default assumption is that everyone intends to cheat (it still does). Punishing people for honest bugfixes is as much against the spirit of friendly competition as, say, secretively adjusting your training methods.\n\nThat change would make Kaggle a somewhat less friendly place. But now I can see the value in it." }, { "id": 317370, "postDate": "2018-04-21T09:53:44.200Z", "content": "@Snorkle, Who said people should be punished for bug fixes?
", "rawMarkdown": "@Snorkle, Who said people should be punished for bug fixes?" }, { "id": 317372, "postDate": "2018-04-21T10:00:54.343Z", "content": "Yeah, sorry, I left out a lot of context.
\n\nYour proposed change was that people be prevented from uploading predictions for stage 2 unless they got code in for stage 1. If we're going with that suggestion, I'd prefer a slightly different soln, where the upload button says \"feel free to get a score, but you missed the model deadline - sorry mate\". For submitting models, it'd be ideal if Kaggle retains two model uploads for verification: your last upload, and the last upload before the deadline. For prizes (or for any medals that actually matter - like the gold you recently got stripped of), they could diff them if they needed to make a judgement call.
\n\nedit: running competitions is hard. I'm glad it's not me doing it.
", "rawMarkdown": "Yeah, sorry, I left out a lot of context.\n\nYour proposed change was that people be prevented from uploading predictions for stage 2 unless they got code in for stage 1. If we're going with that suggestion, I'd prefer a slightly different soln, where the upload button says \"feel free to get a score, but you missed the model deadline - sorry mate\". For submitting models, it'd be ideal if Kaggle retains two model uploads for verification: your last upload, and the last upload before the deadline. For prizes (or for any medals that actually matter - like the gold you recently got stripped of), they could diff them if they needed to make a judgement call.\n\nedit: running competitions is hard. I'm glad it's not me doing it.", "votes": 1 } ] }, { "id": 317527, "postDate": "2018-04-21T19:39:10.573Z", "content": "I disagree with the removal decision taken after stage 1.
\n\nSince I didn't expect to be prize eligible I didn't bother to upload my code. Now, I'm a bit surprised that my former top 14% result has been removed from the LB. By the way, my code is based on my publicly available kernel https://www.kaggle.com/raoulma/nuclei-dsb-2018-tensorflow-u-net-score-0-352 which has also supported other participants in entering the whole contest.
\n\nFurthermore, I'm curious - does kaggle really check all the code of non-prize eligible candidates for correctly reproducing the submitted csv file ? If not, then it shouldn't matter if you upload code or not, as long as you are not prize eligible.
", "rawMarkdown": "I disagree with the removal decision taken after stage 1. \n\nSince I didn't expect to be prize eligible I didn't bother to upload my code. Now, I'm a bit surprised that my former top 14% result has been removed from the LB. By the way, my code is based on my publicly available kernel https://www.kaggle.com/raoulma/nuclei-dsb-2018-tensorflow-u-net-score-0-352 which has also supported other participants in entering the whole contest. \n\nFurthermore, I'm curious - does kaggle really check all the code of non-prize eligible candidates for correctly reproducing the submitted csv file ? If not, then it shouldn't matter if you upload code or not, as long as you are not prize eligible. \n\n", "votes": 12, "replies": [ { "id": 318188, "postDate": "2018-04-23T11:08:42.540Z", "content": "Hi Raoul
\n\nI've finished reading your notebook... It's absolutely great for those like me who wanna dive later into images segmentation. I really love the general purpose network class.
\n\nIt strikes me even more to know that your submissions were removed while the code is based on a baseline kernel that you were very kind to publish.
\n\nKaggle should really take a lot of things into consideration before making such cruel decisions.
", "rawMarkdown": "Hi Raoul\n\nI've finished reading your notebook... It's absolutely great for those like me who wanna dive later into images segmentation. I really love the general purpose network class. \n\nIt strikes me even more to know that your submissions were removed while the code is based on a baseline kernel that you were very kind to publish. \n\n\nKaggle should really take a lot of things into consideration before making such cruel decisions.", "votes": 6 }, { "id": 318204, "postDate": "2018-04-23T11:48:05.957Z", "content": "@Raoul, \nI'm sad to know that you're removed from the LB even though your kernel is still public. Being novice in such competitions, your kernel was really helpful for me to understand how things work. It's disappointing that admins don't even have time to see publicly submitted kernels and it's importance.
\n\nI don't know how much interaction these quality control inspectors (who are enforcing martial laws as soon as they wake up) have with competitions. I am sorry to say this but this issue really needs to be escalated in order to prevent further destruction in competition tier from few people wearing Kaggle badge but no knowledge or interaction with competitions or community.
", "rawMarkdown": "@Raoul, \nI'm sad to know that you're removed from the LB even though your kernel is still public. Being novice in such competitions, your kernel was really helpful for me to understand how things work. It's disappointing that admins don't even have time to see publicly submitted kernels and it's importance. \n\nI don't know how much interaction these **quality control inspectors** (who are enforcing martial laws as soon as they wake up) have with competitions. I am sorry to say this but this issue really needs to be escalated in order to prevent further destruction in competition tier from few people wearing Kaggle badge but no knowledge or interaction with competitions or community. ", "votes": 4 }, { "id": 318331, "postDate": "2018-04-23T16:16:22.820Z", "content": "@Serigne and @Pranav, thanks for your comments. I just hope that the kaggle team realizes their mistake and is professional enough to correct for it.
", "rawMarkdown": "@Serigne and @Pranav, thanks for your comments. I just hope that the kaggle team realizes their mistake and is professional enough to correct for it. ", "votes": 3 } ] }, { "id": 317454, "postDate": "2018-04-21T16:22:27.663Z", "content": "My LB standing actually benefited due to people getting removed, but I don't feel this is right.\nI agree that Kaggle should not remove people from the LB just because they didn't upload a model.\nThe FAQ makes it clear that uploading a model is not required to stay on the LB.
", "rawMarkdown": "My LB standing actually benefited due to people getting removed, but I don't feel this is right.\nI agree that Kaggle should not remove people from the LB just because they didn't upload a model.\nThe FAQ makes it clear that uploading a model is not required to stay on the LB.\n", "votes": 9 }, { "id": 317724, "postDate": "2018-04-22T11:11:00.493Z", "content": "I did not participate in the competition but I am wholeheartedly with you.
\n\nThis is the kind of competitions that require lot of time, effort and computational resources. So LB removal should not be taken so lightly, except in cases of obvious cheating.
", "rawMarkdown": "I did not participate in the competition but I am wholeheartedly with you.\n\nThis is the kind of competitions that require lot of time, effort and computational resources. So LB removal should not be taken so lightly, except in cases of obvious cheating.", "votes": 10 }, { "id": 317355, "postDate": "2018-04-21T09:04:00.473Z", "content": "I'm not affected but I also disagree with the removal. The timeline in the official rules that everyone agreed upon doesn't even mention that you have to upload. In my opinion the official rules should have been updated before the end of stage1 or atleast a link to the stage2 faq should have been provided on the competition overview, not only in the community discussions.
", "rawMarkdown": "I'm not affected but I also disagree with the removal. The timeline in the official [rules](https://www.kaggle.com/c/data-science-bowl-2018/rules) that everyone agreed upon doesn't even mention that you have to upload. In my opinion the official rules should have been updated before the end of stage1 or atleast a link to the stage2 faq should have been provided on the competition overview, not only in the community discussions. ", "votes": 9 }, { "id": 317366, "postDate": "2018-04-21T09:41:47.100Z", "content": "I have up voted this. I am a solo participant, and this is my first Kaggle competition. My LB for stage 2 was 0.488. However, I have been removed from the leaderboard and the only reason I can think of, is because of late submission of my models. Kaggle sent me an email as a reminder to select my 2 models, and I did that on time, but Kaggle did not remind me or provide instruction to upload models. I only found out how to upload models after reading discussion in Kaggle, that you have to go to Team - but since I am solo, I was expecting this upload feature to be made available in the Submission.
\n\nSo, to my knowledge, I have complied with rules set by Kaggle in FAQ and Rules.
\n\nSpecifically, in Kaggle's FAQ - it's clearly stated: Will I still be on the leaderboard if I don't upload a model but do submit in the second stage? Ans: Yes. However, if you do not upload a model and finish in a prize position, your team will be removed from the competition standings entirely.
\n\nI hope that Kaggle will put me and others with the same situation back into the leaderboard, thank you.
", "rawMarkdown": "I have up voted this. I am a solo participant, and this is my first Kaggle competition. My LB for stage 2 was 0.488. However, I have been removed from the leaderboard and the only reason I can think of, is because of late submission of my models. Kaggle sent me an email as a reminder to select my 2 models, and I did that on time, **but Kaggle did not remind me or provide instruction to upload models**. I only found out how to upload models after reading discussion in Kaggle, that you have to go to Team - but since I am solo, I was expecting this upload feature to be made available in the Submission. \n\nSo, to my knowledge, I have complied with rules set by Kaggle in [FAQ][1] and [Rules][1]. \n\nSpecifically, in Kaggle's [FAQ][2] - it's clearly stated: **Will I still be on the leaderboard if I don't upload a model but do submit in the second stage?** Ans: **Yes.** However, if you do not upload a model **and** finish in a prize position, your team will be removed from the competition standings entirely.\n\nI hope that Kaggle will put me and others with the same situation back into the leaderboard, thank you.\n\n [1]: https://www.kaggle.com/c/data-science-bowl-2018/rules\n [2]: https://www.kaggle.com/two-stage-faq", "votes": 8, "replies": [ { "id": 317449, "postDate": "2018-04-21T16:05:41.133Z", "content": "Yes - thank you it is exactly that line in the FAQ that made me think I didn't need to upload a model. I have also been removed.
\n\nAfter the months of work I put into this competition, I think they could have put more effort into making sure the rules were clear and unambiguous.
", "rawMarkdown": "Yes - thank you it is exactly that line in the FAQ that made me think I didn't need to upload a model. I have also been removed. \n\nAfter the months of work I put into this competition, I think they could have put more effort into making sure the rules were clear and unambiguous. ", "votes": 4 } ] }, { "id": 317988, "postDate": "2018-04-23T02:05:03.387Z", "content": "Hi, this is my first competition and I put a lot of time on it, and now my results are removed!!\nIt's upset enough that the system didn't submit my best result (LB 0.479), but I'm here to learn and I can accept that. But remove all my results? Please! I used Kaggle kernels all the time with absolutely no outside data, why it doesn't count?
", "rawMarkdown": "Hi, this is my first competition and I put a lot of time on it, and now my results are removed!!\nIt's upset enough that the system didn't submit my best result (LB 0.479), but I'm here to learn and I can accept that. But remove all my results? Please! I used Kaggle kernels all the time with absolutely no outside data, why it doesn't count?", "votes": 6 }, { "id": 317421, "postDate": "2018-04-21T14:05:29.543Z", "content": "Competely agree with the points you make. It was a great learning experience but a really frustrating end to the competition.
", "rawMarkdown": "Competely agree with the points you make. It was a great learning experience but a really frustrating end to the competition. ", "votes": 5 }, { "id": 317363, "postDate": "2018-04-21T09:38:41.487Z", "content": "Not only not-uploaded models have been removed, I uploaded my model but have been removed, first time it happens to me in a competition.
\n\nJust filled in compliance form to find out what has happened.
", "rawMarkdown": "Not only not-uploaded models have been removed, I uploaded my model but have been removed, first time it happens to me in a competition. \n\nJust filled in compliance form to find out what has happened.", "votes": 5, "replies": [ { "id": 317424, "postDate": "2018-04-21T14:12:34.400Z", "content": "I am in the same condition. And I have uploaded the model in stage 1, which I think as stated is the only situation for uploading model. I have sent e-mails but I don't get a detailed reason for removing. Could you please tell me the link where I could fill in the compliance form?
", "rawMarkdown": "I am in the same condition. And I have uploaded the model in stage 1, which I think as stated is the only situation for uploading model. I have sent e-mails but I don't get a detailed reason for removing. Could you please tell me the link where I could fill in the compliance form?" }, { "id": 317433, "postDate": "2018-04-21T14:58:44.403Z", "content": "Sure, its here:
\n\nhttps://www.kaggle.com/compliance
\n\nBe careful though, to fill it in the proper way, I posted also the link as thread here, with small advice on how to/not to use it, https://www.kaggle.com/c/data-science-bowl-2018/discussion/55041 , in case there are more in our situation.
\n\nGood luck :-)
", "rawMarkdown": "Sure, its here:\n\nhttps://www.kaggle.com/compliance\n\nBe careful though, to fill it in the proper way, I posted also the link as thread here, with small advice on how to/not to use it, https://www.kaggle.com/c/data-science-bowl-2018/discussion/55041 , in case there are more in our situation. \n\nGood luck :-)" }, { "id": 317587, "postDate": "2018-04-21T23:00:38.437Z", "content": "We have also uploaded our models and have been removed from the leaderboard as well. It'd be nice to know why it happened so that we can at least do it right next time. It's terrible to work on something for months and got nothing out of it at the end without even knowing why.
", "rawMarkdown": "We have also uploaded our models and have been removed from the leaderboard as well. It'd be nice to know why it happened so that we can at least do it right next time. It's terrible to work on something for months and got nothing out of it at the end without even knowing why.", "votes": 1 }, { "id": 317620, "postDate": "2018-04-22T02:32:36.837Z", "content": "Thank you very much! Let's keep in touch to see what will happen.
", "rawMarkdown": "Thank you very much! Let's keep in touch to see what will happen." }, { "id": 317627, "postDate": "2018-04-22T03:13:49.657Z", "content": "I also file a complain and I attached my code. I hope they can put me back on LB.
", "rawMarkdown": "I also file a complain and I attached my code. I hope they can put me back on LB." } ] }, { "id": 317334, "postDate": "2018-04-21T08:24:39.073Z", "content": "I disagree with the removal decision taken after stage 1 end
", "rawMarkdown": "I disagree with the removal decision taken after stage 1 end", "votes": 6 }, { "id": 317384, "postDate": "2018-04-21T10:53:45.533Z", "content": "Upvoted this. Think things are handled pretty badly, removing half of the leaderboard without clear explanation. If the reason really is not uploading of model, then it wasn't explicitly stated. If it is other reasons, at least an explanation could be given.
", "rawMarkdown": "Upvoted this. Think things are handled pretty badly, removing half of the leaderboard without clear explanation. If the reason really is not uploading of model, then it wasn't explicitly stated. If it is other reasons, at least an explanation could be given.", "votes": 6 }, { "id": 317348, "postDate": "2018-04-21T08:50:11.720Z", "content": "The rules is the rule. If change it all the time，It was rude to the contestants.
", "rawMarkdown": "The rules is the rule. If change it all the time，It was rude to the contestants.", "votes": 6 }, { "id": 317990, "postDate": "2018-04-23T02:11:38.127Z", "content": "I agree with you. Kaggle should get us back to the LB. There should not have a change of the rules during the competition!!!
", "rawMarkdown": "I agree with you. Kaggle should get us back to the LB. There should not have a change of the rules during the competition!!!", "votes": 5 }, { "id": 317712, "postDate": "2018-04-22T10:27:25.880Z", "content": "I just checked the LB, to take a screenshot about my status and I am not on the LB... It was a pretty big surprise, but as I saw this thread I knew what happened...\nI did not upload my code on purpose because I knew I won't win and I also totally forget about it (because of University and work stuff)... But I've worked on my model/solution for more than a month and now there is no sign on the LB.
\n\nI don't know what to say. It is really sad that they do such thing against their own rules.
", "rawMarkdown": "I just checked the LB, to take a screenshot about my status and I am not on the LB... It was a pretty big surprise, but as I saw this thread I knew what happened...\nI did not upload my code on purpose because I knew I won't win and I also totally forget about it (because of University and work stuff)... But I've worked on my model/solution for more than a month and now there is no sign on the LB.\n\nI don't know what to say. It is really sad that they do such thing against their own rules.", "votes": 3 }, { "id": 317332, "postDate": "2018-04-21T08:17:55.277Z", "content": "I also was confused by the rules ,everyone in this competition pay many effort ,although I enjoy the process,but the result is also very import .
", "rawMarkdown": "I also was confused by the rules ,everyone in this competition pay many effort ,although I enjoy the process,but the result is also very import .", "votes": 3 }, { "id": 317327, "postDate": "2018-04-21T08:09:00.937Z", "content": "I did not know everyone had to upload their model.....this was my first competition and I thought only prize winners were required to upload their model as per the rules.Maybe I was confused...anyways I learned a lot and it was fun.
", "rawMarkdown": "I did not know everyone had to upload their model.....this was my first competition and I thought only prize winners were required to upload their model as per the rules.Maybe I was confused...anyways I learned a lot and it was fun.", "votes": 3 }, { "id": 317356, "postDate": "2018-04-21T09:13:44.790Z", "content": "Thank you for the thread! I disagree with the removal decision!
", "rawMarkdown": "Thank you for the thread! I disagree with the removal decision!", "votes": 1 }, { "id": 317337, "postDate": "2018-04-21T08:29:32.790Z", "content": "can't agree any more. I need a reasonable explanation on this issue.
", "rawMarkdown": "can't agree any more. I need a reasonable explanation on this issue.", "votes": 1 }, { "id": 319406, "postDate": "2018-04-26T01:32:10.147Z", "content": "Mine was also removed. Anyone from this competition host, please answer this thread.
", "rawMarkdown": "Mine was also removed. Anyone from this competition host, please answer this thread.", "replies": [ { "id": 319502, "postDate": "2018-04-26T07:25:51.340Z", "content": "@Ildoo Kim , Kaggle will reinstate people back if the reason for being removed was not uploading your model before the deadline of the first stage. See the official response here.\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/55228
", "rawMarkdown": "@Ildoo Kim , Kaggle will reinstate people back if the reason for being removed was not uploading your model before the deadline of the first stage. See the official response here.\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/55228" }, { "id": 319528, "postDate": "2018-04-26T08:30:45.530Z", "content": "As I remember, I uploaded.
", "rawMarkdown": "As I remember, I uploaded. " }, { "id": 319583, "postDate": "2018-04-26T11:07:47.590Z", "content": "@Ildoo Kim, wait and see if your team will be reinstated, if not, you can follow the steps in this discussion. They haven't started reinstating people yet, may be busy evaluating the solutions in the prize.\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/55041
", "rawMarkdown": "@Ildoo Kim, wait and see if your team will be reinstated, if not, you can follow the steps in this discussion. They haven't started reinstating people yet, may be busy evaluating the solutions in the prize.\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/55041" }, { "id": 319643, "postDate": "2018-04-26T14:03:45.047Z", "content": "I have filled the compliance form, my result was removed due to not uploading model. \nThis was my first competition, i did not upload any models or code..
", "rawMarkdown": "I have filled the compliance form, my result was removed due to not uploading model. \nThis was my first competition, i did not upload any models or code.. " } ] }, { "id": 318524, "postDate": "2018-04-24T01:45:17.103Z", "content": "FYI, Maggie from Customer Success at Kaggle has provided updates Change of Course - Leaderboard Reinstatement. This should be good news for those impacted.
\n\n@Maggie, thank you very much for listening to our feedbacks and taking necessary actions to get this corrected, very much appreciated. We'll wait for the reinstatement then.
", "rawMarkdown": "FYI, Maggie from Customer Success at Kaggle has provided updates [Change of Course - Leaderboard Reinstatement][1]. This should be good news for those impacted.\n\n@Maggie, thank you very much for listening to our feedbacks and taking necessary actions to get this corrected, very much appreciated. We'll wait for the reinstatement then.\n\n\n [1]: https://www.kaggle.com/c/data-science-bowl-2018/discussion/55228" }, { "id": 318513, "postDate": "2018-04-24T01:01:57.660Z", "content": "I agree with you, why my team has been removed
", "rawMarkdown": "I agree with you, why my team has been removed" }, { "id": 318218, "postDate": "2018-04-23T12:19:32.920Z", "content": "This is my second 2 stage competition and I didn't get removed from the LB for the 1st competition. How interesting.
", "rawMarkdown": "This is my second 2 stage competition and I didn't get removed from the LB for the 1st competition. How interesting." }, { "id": 318136, "postDate": "2018-04-23T08:23:25.330Z", "content": "I agreed with you
", "rawMarkdown": "I agreed with you" }, { "id": 318077, "postDate": "2018-04-23T06:29:25.570Z", "content": "I have already selected two results to submit, but the ranking disappear after 2 days. It difficult to understand what happen
", "rawMarkdown": "I have already selected two results to submit, but the ranking disappear after 2 days. It difficult to understand what happen" }, { "id": 317335, "postDate": "2018-04-21T08:24:42.070Z", "content": "This was my first competition and was a solo participant, so was not aware to upload model, big lesson learned.
\n\nIs there a way to get the result back. I only have a snapshot of my rank 22 which was a silver eligible. I was not aware i had to upload my model etc.. Also in submission page only result upload was there..
\n\nPlease help..
", "rawMarkdown": "This was my first competition and was a solo participant, so was not aware to upload model, big lesson learned.\n\nIs there a way to get the result back. I only have a snapshot of my rank 22 which was a silver eligible. I was not aware i had to upload my model etc.. Also in submission page only result upload was there.. \n\nPlease help.." }, { "id": 317662, "postDate": "2018-04-22T07:12:38.207Z", "rawMarkdown": "", "isDeleted": true }, { "id": 317529, "postDate": "2018-04-21T19:40:27.497Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 317362, "author_name": "Plankton", "author_url": "", "post_date": "2018-04-21T09:37:12.460000", "content": "The rules haven't changed. Last year DSB had the same rules, same timeline wordings, non-winners were not removed for not uploading models. On top of that, current FAQ states:
\n\n\n\n\nWill I still be on the leaderboard if I don't upload a model but do submit in the second stage?\n Yes. However, if you do not upload a model and finish in a prize position, your team will be removed from the competition standings entirely.
\n
Given these two facts, one could easily think above statement should hold under current rules. And it had been always like that for recent two stage competitions.
\n\nWhy Kaggle now chose to inforce differently for the same RULES, without ANY NOTICE? One Kaggle admin mentioned they couldn't implement this in previous competitions due to administrative reasons but why not NOTIFYING us from the beginning that something is changed? OR is there any hidden reason for the change AFTER the competition ended?
\n\nIn addition, the requirement for model uploading is not even put under RULES tab, but under timeline. Since it's not under RULES, it's only for those who want the win the prize right? If it's for all, why didn't they bother putting that important piece of information under RULES tab in the first place, given that the RULES also have a timeline?
\n\nMore than half number of teams are removed just shows something very wrong from the organizing point of view. We need some proper explanations.
\n\n@William, @Wendy
", "votes": 16, "replies": [] }, { "id": 317325, "author_name": "CPMP", "author_url": "", "post_date": "2018-04-21T08:06:47.667000", "content": "I had the same idea, just removed my post to not dilute impact. I have not been removed, but I support this, because the rules + FAQ were ambiguous. It is because of this ambiguity that I wrote my post on 'Cheating is encouraged'. The new rule is great IMHO, but it should be enforced in future competitions, not this one, because it has been decided after stage 1 ended.
", "votes": 16, "replies": [ { "id": 317329, "author_name": "gavrand", "author_url": "", "post_date": "2018-04-21T08:10:57.883000", "content": "@CPMP your agreement is respected, since you were the topic starter for the need to change the FAQ. The main issue is that they didn't change it in time and they now do it retrospectively.
", "votes": 3, "replies": [] }, { "id": 317331, "author_name": "CPMP", "author_url": "", "post_date": "2018-04-21T08:12:58.183000", "content": "\n\n\nThe main issue is that they didn't change it in time and they now do it retrospectively.
\n
I agree, I wrote that too in the comment of my other post, see https://www.kaggle.com/c/data-science-bowl-2018/discussion/54575#316064
\n\nI fully support the change, but for next competition, not this one. I wish they has changed their FAQ in time to avoid all this mess.
", "votes": 5, "replies": [] }, { "id": 317364, "author_name": "Snorkle", "author_url": "", "post_date": "2018-04-21T09:39:11.170000", "content": "CPMP, when you first made your suggestion, I strongly disagreed - it rankled me that your default assumption is that everyone intends to cheat (it still does). Punishing people for honest bugfixes is as much against the spirit of friendly competition as, say, secretively adjusting your training methods.
\n\nThat change would make Kaggle a somewhat less friendly place. But now I can see the value in it.
", "votes": 0, "replies": [] }, { "id": 317370, "author_name": "CPMP", "author_url": "", "post_date": "2018-04-21T09:53:44.200000", "content": "@Snorkle, Who said people should be punished for bug fixes?
", "votes": 0, "replies": [] }, { "id": 317372, "author_name": "Snorkle", "author_url": "", "post_date": "2018-04-21T10:00:54.343000", "content": "Yeah, sorry, I left out a lot of context.
\n\nYour proposed change was that people be prevented from uploading predictions for stage 2 unless they got code in for stage 1. If we're going with that suggestion, I'd prefer a slightly different soln, where the upload button says \"feel free to get a score, but you missed the model deadline - sorry mate\". For submitting models, it'd be ideal if Kaggle retains two model uploads for verification: your last upload, and the last upload before the deadline. For prizes (or for any medals that actually matter - like the gold you recently got stripped of), they could diff them if they needed to make a judgement call.
\n\nedit: running competitions is hard. I'm glad it's not me doing it.
", "votes": 1, "replies": [] } ] }, { "id": 317527, "author_name": "Raoul", "author_url": "", "post_date": "2018-04-21T19:39:10.573000", "content": "I disagree with the removal decision taken after stage 1.
\n\nSince I didn't expect to be prize eligible I didn't bother to upload my code. Now, I'm a bit surprised that my former top 14% result has been removed from the LB. By the way, my code is based on my publicly available kernel https://www.kaggle.com/raoulma/nuclei-dsb-2018-tensorflow-u-net-score-0-352 which has also supported other participants in entering the whole contest.
\n\nFurthermore, I'm curious - does kaggle really check all the code of non-prize eligible candidates for correctly reproducing the submitted csv file ? If not, then it shouldn't matter if you upload code or not, as long as you are not prize eligible.
", "votes": 12, "replies": [ { "id": 318188, "author_name": "Serigne ", "author_url": "", "post_date": "2018-04-23T11:08:42.540000", "content": "Hi Raoul
\n\nI've finished reading your notebook... It's absolutely great for those like me who wanna dive later into images segmentation. I really love the general purpose network class.
\n\nIt strikes me even more to know that your submissions were removed while the code is based on a baseline kernel that you were very kind to publish.
\n\nKaggle should really take a lot of things into consideration before making such cruel decisions.
", "votes": 6, "replies": [] }, { "id": 318204, "author_name": "Pranav Pandya", "author_url": "", "post_date": "2018-04-23T11:48:05.957000", "content": "@Raoul, \nI'm sad to know that you're removed from the LB even though your kernel is still public. Being novice in such competitions, your kernel was really helpful for me to understand how things work. It's disappointing that admins don't even have time to see publicly submitted kernels and it's importance.
\n\nI don't know how much interaction these quality control inspectors (who are enforcing martial laws as soon as they wake up) have with competitions. I am sorry to say this but this issue really needs to be escalated in order to prevent further destruction in competition tier from few people wearing Kaggle badge but no knowledge or interaction with competitions or community.
", "votes": 4, "replies": [] }, { "id": 318331, "author_name": "Raoul", "author_url": "", "post_date": "2018-04-23T16:16:22.820000", "content": "@Serigne and @Pranav, thanks for your comments. I just hope that the kaggle team realizes their mistake and is professional enough to correct for it.
", "votes": 3, "replies": [] } ] }, { "id": 317454, "author_name": "Daniel J", "author_url": "", "post_date": "2018-04-21T16:22:27.663000", "content": "My LB standing actually benefited due to people getting removed, but I don't feel this is right.\nI agree that Kaggle should not remove people from the LB just because they didn't upload a model.\nThe FAQ makes it clear that uploading a model is not required to stay on the LB.
", "votes": 9, "replies": [] }, { "id": 317724, "author_name": "Serigne ", "author_url": "", "post_date": "2018-04-22T11:11:00.493000", "content": "I did not participate in the competition but I am wholeheartedly with you.
\n\nThis is the kind of competitions that require lot of time, effort and computational resources. So LB removal should not be taken so lightly, except in cases of obvious cheating.
", "votes": 10, "replies": [] }, { "id": 317355, "author_name": "Jean-Marc Wanka", "author_url": "", "post_date": "2018-04-21T09:04:00.473000", "content": "I'm not affected but I also disagree with the removal. The timeline in the official rules that everyone agreed upon doesn't even mention that you have to upload. In my opinion the official rules should have been updated before the end of stage1 or atleast a link to the stage2 faq should have been provided on the competition overview, not only in the community discussions.
", "votes": 9, "replies": [] }, { "id": 317366, "author_name": "Sumiko", "author_url": "", "post_date": "2018-04-21T09:41:47.100000", "content": "I have up voted this. I am a solo participant, and this is my first Kaggle competition. My LB for stage 2 was 0.488. However, I have been removed from the leaderboard and the only reason I can think of, is because of late submission of my models. Kaggle sent me an email as a reminder to select my 2 models, and I did that on time, but Kaggle did not remind me or provide instruction to upload models. I only found out how to upload models after reading discussion in Kaggle, that you have to go to Team - but since I am solo, I was expecting this upload feature to be made available in the Submission.
\n\nSo, to my knowledge, I have complied with rules set by Kaggle in FAQ and Rules.
\n\nSpecifically, in Kaggle's FAQ - it's clearly stated: Will I still be on the leaderboard if I don't upload a model but do submit in the second stage? Ans: Yes. However, if you do not upload a model and finish in a prize position, your team will be removed from the competition standings entirely.
\n\nI hope that Kaggle will put me and others with the same situation back into the leaderboard, thank you.
", "votes": 8, "replies": [ { "id": 317449, "author_name": "JamesEverard", "author_url": "", "post_date": "2018-04-21T16:05:41.133000", "content": "Yes - thank you it is exactly that line in the FAQ that made me think I didn't need to upload a model. I have also been removed.
\n\nAfter the months of work I put into this competition, I think they could have put more effort into making sure the rules were clear and unambiguous.
", "votes": 4, "replies": [] } ] }, { "id": 317988, "author_name": "ryan", "author_url": "", "post_date": "2018-04-23T02:05:03.387000", "content": "Hi, this is my first competition and I put a lot of time on it, and now my results are removed!!\nIt's upset enough that the system didn't submit my best result (LB 0.479), but I'm here to learn and I can accept that. But remove all my results? Please! I used Kaggle kernels all the time with absolutely no outside data, why it doesn't count?
", "votes": 6, "replies": [] }, { "id": 317421, "author_name": "DeepNet", "author_url": "", "post_date": "2018-04-21T14:05:29.543000", "content": "Competely agree with the points you make. It was a great learning experience but a really frustrating end to the competition.
", "votes": 5, "replies": [] }, { "id": 317363, "author_name": "miguel perez", "author_url": "", "post_date": "2018-04-21T09:38:41.487000", "content": "Not only not-uploaded models have been removed, I uploaded my model but have been removed, first time it happens to me in a competition.
\n\nJust filled in compliance form to find out what has happened.
", "votes": 5, "replies": [ { "id": 317424, "author_name": "Leo Gu", "author_url": "", "post_date": "2018-04-21T14:12:34.400000", "content": "I am in the same condition. And I have uploaded the model in stage 1, which I think as stated is the only situation for uploading model. I have sent e-mails but I don't get a detailed reason for removing. Could you please tell me the link where I could fill in the compliance form?
", "votes": 0, "replies": [] }, { "id": 317433, "author_name": "miguel perez", "author_url": "", "post_date": "2018-04-21T14:58:44.403000", "content": "Sure, its here:
\n\nhttps://www.kaggle.com/compliance
\n\nBe careful though, to fill it in the proper way, I posted also the link as thread here, with small advice on how to/not to use it, https://www.kaggle.com/c/data-science-bowl-2018/discussion/55041 , in case there are more in our situation.
\n\nGood luck :-)
", "votes": 0, "replies": [] }, { "id": 317587, "author_name": "photogazer", "author_url": "", "post_date": "2018-04-21T23:00:38.437000", "content": "We have also uploaded our models and have been removed from the leaderboard as well. It'd be nice to know why it happened so that we can at least do it right next time. It's terrible to work on something for months and got nothing out of it at the end without even knowing why.
", "votes": 1, "replies": [] }, { "id": 317620, "author_name": "Leo Gu", "author_url": "", "post_date": "2018-04-22T02:32:36.837000", "content": "Thank you very much! Let's keep in touch to see what will happen.
", "votes": 0, "replies": [] }, { "id": 317627, "author_name": "Wei Zheng", "author_url": "", "post_date": "2018-04-22T03:13:49.657000", "content": "I also file a complain and I attached my code. I hope they can put me back on LB.
", "votes": 0, "replies": [] } ] }, { "id": 317334, "author_name": "Giba", "author_url": "", "post_date": "2018-04-21T08:24:39.073000", "content": "I disagree with the removal decision taken after stage 1 end
", "votes": 6, "replies": [] }, { "id": 317384, "author_name": "Hon Keat", "author_url": "", "post_date": "2018-04-21T10:53:45.533000", "content": "Upvoted this. Think things are handled pretty badly, removing half of the leaderboard without clear explanation. If the reason really is not uploading of model, then it wasn't explicitly stated. If it is other reasons, at least an explanation could be given.
", "votes": 6, "replies": [] }, { "id": 317348, "author_name": "yyll008", "author_url": "", "post_date": "2018-04-21T08:50:11.720000", "content": "The rules is the rule. If change it all the time，It was rude to the contestants.
", "votes": 6, "replies": [] }, { "id": 317990, "author_name": "zjj421", "author_url": "", "post_date": "2018-04-23T02:11:38.127000", "content": "I agree with you. Kaggle should get us back to the LB. There should not have a change of the rules during the competition!!!
", "votes": 5, "replies": [] }, { "id": 317712, "author_name": "Gábor Vecsei", "author_url": "", "post_date": "2018-04-22T10:27:25.880000", "content": "I just checked the LB, to take a screenshot about my status and I am not on the LB... It was a pretty big surprise, but as I saw this thread I knew what happened...\nI did not upload my code on purpose because I knew I won't win and I also totally forget about it (because of University and work stuff)... But I've worked on my model/solution for more than a month and now there is no sign on the LB.
\n\nI don't know what to say. It is really sad that they do such thing against their own rules.
", "votes": 3, "replies": [] }, { "id": 317332, "author_name": "yaya", "author_url": "", "post_date": "2018-04-21T08:17:55.277000", "content": "I also was confused by the rules ,everyone in this competition pay many effort ,although I enjoy the process,but the result is also very import .
", "votes": 3, "replies": [] }, { "id": 317327, "author_name": "Kunal.S.Kasodekar", "author_url": "", "post_date": "2018-04-21T08:09:00.937000", "content": "I did not know everyone had to upload their model.....this was my first competition and I thought only prize winners were required to upload their model as per the rules.Maybe I was confused...anyways I learned a lot and it was fun.
", "votes": 3, "replies": [] }, { "id": 317356, "author_name": "Shitian Ni", "author_url": "", "post_date": "2018-04-21T09:13:44.790000", "content": "Thank you for the thread! I disagree with the removal decision!
", "votes": 1, "replies": [] }, { "id": 317337, "author_name": "Zheng Li", "author_url": "", "post_date": "2018-04-21T08:29:32.790000", "content": "can't agree any more. I need a reasonable explanation on this issue.
", "votes": 1, "replies": [] }, { "id": 319406, "author_name": "Ildoo Kim", "author_url": "", "post_date": "2018-04-26T01:32:10.147000", "content": "Mine was also removed. Anyone from this competition host, please answer this thread.
", "votes": 0, "replies": [ { "id": 319502, "author_name": "Nicole Finnie", "author_url": "", "post_date": "2018-04-26T07:25:51.340000", "content": "@Ildoo Kim , Kaggle will reinstate people back if the reason for being removed was not uploading your model before the deadline of the first stage. See the official response here.\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/55228
", "votes": 0, "replies": [] }, { "id": 319528, "author_name": "Ildoo Kim", "author_url": "", "post_date": "2018-04-26T08:30:45.530000", "content": "As I remember, I uploaded.
", "votes": 0, "replies": [] }, { "id": 319583, "author_name": "Nicole Finnie", "author_url": "", "post_date": "2018-04-26T11:07:47.590000", "content": "@Ildoo Kim, wait and see if your team will be reinstated, if not, you can follow the steps in this discussion. They haven't started reinstating people yet, may be busy evaluating the solutions in the prize.\nhttps://www.kaggle.com/c/data-science-bowl-2018/discussion/55041
", "votes": 0, "replies": [] }, { "id": 319643, "author_name": "Samrat Saha", "author_url": "", "post_date": "2018-04-26T14:03:45.047000", "content": "I have filled the compliance form, my result was removed due to not uploading model. \nThis was my first competition, i did not upload any models or code..
", "votes": 0, "replies": [] } ] }, { "id": 318524, "author_name": "Sumiko", "author_url": "", "post_date": "2018-04-24T01:45:17.103000", "content": "FYI, Maggie from Customer Success at Kaggle has provided updates Change of Course - Leaderboard Reinstatement. This should be good news for those impacted.
\n\n@Maggie, thank you very much for listening to our feedbacks and taking necessary actions to get this corrected, very much appreciated. We'll wait for the reinstatement then.
", "votes": 0, "replies": [] }, { "id": 318513, "author_name": "notbad", "author_url": "", "post_date": "2018-04-24T01:01:57.660000", "content": "I agree with you, why my team has been removed
", "votes": 0, "replies": [] }, { "id": 318218, "author_name": "Wei Zheng", "author_url": "", "post_date": "2018-04-23T12:19:32.920000", "content": "This is my second 2 stage competition and I didn't get removed from the LB for the 1st competition. How interesting.
", "votes": 0, "replies": [] }, { "id": 318136, "author_name": "Nafisur Rahman", "author_url": "", "post_date": "2018-04-23T08:23:25.330000", "content": "I agreed with you
", "votes": 0, "replies": [] }, { "id": 318077, "author_name": "Ho-Kin", "author_url": "", "post_date": "2018-04-23T06:29:25.570000", "content": "I have already selected two results to submit, but the ranking disappear after 2 days. It difficult to understand what happen
", "votes": 0, "replies": [] }, { "id": 317335, "author_name": "Samrat Saha", "author_url": "", "post_date": "2018-04-21T08:24:42.070000", "content": "This was my first competition and was a solo participant, so was not aware to upload model, big lesson learned.
\n\nIs there a way to get the result back. I only have a snapshot of my rank 22 which was a silver eligible. I was not aware i had to upload my model etc.. Also in submission page only result upload was there..
\n\nPlease help..
", "votes": 0, "replies": [] }, { "id": 317662, "author_name": "", "author_url": "", "post_date": "2018-04-22T07:12:38.207000", "content": "", "votes": 0, "replies": [] }, { "id": 317529, "author_name": "", "author_url": "", "post_date": "2018-04-21T19:40:27.497000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "317318": "@Admins and Kaggle Management\n\nIt is clear that the code upload rule consists of two parts: \n\nFirst part is the code upload timeline, that stands for all participants. \n\nSecond part is that in accordance with the FAQ if you don't upload code then you can still be on the leaderboard, except for the case that you finish in a prize winning position and your submission will be removed.\n\nIt seems the rule has been enforced only partially, without previous notice and retrospectively. If the change was published in time then all participants would be discouraged to not upload their model and would almost certainly upload their code, because getting results in such a competition requires effort, time and money, and wasting all that for such a simple thing is irrational. \n\nWhere can we appeal this decision and where can we formally express our disagreement about the consistency of your decisions?\n\n@All kagglers who disagree with the removal decision please upvote and appeal.\n\nEdit: The fact that by removing all participants, the top left pixel benchmark earns a bronze medal and sufficient ranking points gives an idea about the admins' decision and the suitability of this competition's results.\n\nEdit2:\nIt was not included in the rules https://www.kaggle.com/c/data-science-bowl-2018/rules\n\nIt was not included in the FAQ https://www.kaggle.com/two-stage-faq\n\n> Will I still be on the leaderboard if I don't upload a model but do\n> submit in the second stage?\n> \n> Yes. However, if you do not upload a model and finish in a prize\n> position, your team will be removed from the competition standings\n> entirely.\n\nEx post facto laws are expressly forbidden by the United States Constitution in Article 1, Section 9, Clause 3 (with respect to federal laws) and Article 1, Section 10 (with respect to state laws).", "317362": "The rules haven't changed. Last year DSB had the same rules, [same timeline wordings][1], non-winners were not removed for not uploading models. On top of that, current FAQ states:\n\n> Will I still be on the leaderboard if I don't upload a model but do submit in the second stage?\nYes. However, if you do not upload a model and finish in a prize position, your team will be removed from the competition standings entirely.\n\nGiven these two facts, one could easily think above statement should hold under current rules. And it had been always like that for recent two stage competitions. \n\nWhy Kaggle now chose to inforce differently for the same RULES, without ANY NOTICE? One Kaggle admin mentioned they couldn't implement this in previous competitions due to administrative reasons but why not NOTIFYING us from the beginning that something is changed? OR is there any hidden reason for the change AFTER the competition ended?\n\n\nIn addition, the requirement for model uploading is not even put under RULES tab, but under timeline. Since it's not under RULES, it's only for those who want the win the prize right? If it's for all, why didn't they bother putting that important piece of information under RULES tab in the first place, given that the RULES also have a timeline?\n\n\nMore than half number of teams are removed just shows something very wrong from the organizing point of view. We need some proper explanations. \n\n@William, @Wendy\n\n\n [1]: https://www.kaggle.com/c/data-science-bowl-2017#timeline", "317325": "I had the same idea, just removed my post to not dilute impact. I have not been removed, but I support this, because the rules + FAQ were ambiguous. It is because of this ambiguity that I wrote my post on 'Cheating is encouraged'. The new rule is great IMHO, but it should be enforced in future competitions, not this one, because it has been decided after stage 1 ended.", "317527": "I disagree with the removal decision taken after stage 1. \n\nSince I didn't expect to be prize eligible I didn't bother to upload my code. Now, I'm a bit surprised that my former top 14% result has been removed from the LB. By the way, my code is based on my publicly available kernel https://www.kaggle.com/raoulma/nuclei-dsb-2018-tensorflow-u-net-score-0-352 which has also supported other participants in entering the whole contest. \n\nFurthermore, I'm curious - does kaggle really check all the code of non-prize eligible candidates for correctly reproducing the submitted csv file ? If not, then it shouldn't matter if you upload code or not, as long as you are not prize eligible. \n\n", "317454": "My LB standing actually benefited due to people getting removed, but I don't feel this is right.\nI agree that Kaggle should not remove people from the LB just because they didn't upload a model.\nThe FAQ makes it clear that uploading a model is not required to stay on the LB.\n", "317724": "I did not participate in the competition but I am wholeheartedly with you.\n\nThis is the kind of competitions that require lot of time, effort and computational resources. So LB removal should not be taken so lightly, except in cases of obvious cheating.", "317355": "I'm not affected but I also disagree with the removal. The timeline in the official [rules](https://www.kaggle.com/c/data-science-bowl-2018/rules) that everyone agreed upon doesn't even mention that you have to upload. In my opinion the official rules should have been updated before the end of stage1 or atleast a link to the stage2 faq should have been provided on the competition overview, not only in the community discussions. ", "317366": "I have up voted this. I am a solo participant, and this is my first Kaggle competition. My LB for stage 2 was 0.488. However, I have been removed from the leaderboard and the only reason I can think of, is because of late submission of my models. Kaggle sent me an email as a reminder to select my 2 models, and I did that on time, **but Kaggle did not remind me or provide instruction to upload models**. I only found out how to upload models after reading discussion in Kaggle, that you have to go to Team - but since I am solo, I was expecting this upload feature to be made available in the Submission. \n\nSo, to my knowledge, I have complied with rules set by Kaggle in [FAQ][1] and [Rules][1]. \n\nSpecifically, in Kaggle's [FAQ][2] - it's clearly stated: **Will I still be on the leaderboard if I don't upload a model but do submit in the second stage?** Ans: **Yes.** However, if you do not upload a model **and** finish in a prize position, your team will be removed from the competition standings entirely.\n\nI hope that Kaggle will put me and others with the same situation back into the leaderboard, thank you.\n\n [1]: https://www.kaggle.com/c/data-science-bowl-2018/rules\n [2]: https://www.kaggle.com/two-stage-faq", "317988": "Hi, this is my first competition and I put a lot of time on it, and now my results are removed!!\nIt's upset enough that the system didn't submit my best result (LB 0.479), but I'm here to learn and I can accept that. But remove all my results? Please! I used Kaggle kernels all the time with absolutely no outside data, why it doesn't count?", "317421": "Competely agree with the points you make. It was a great learning experience but a really frustrating end to the competition. ", "317363": "Not only not-uploaded models have been removed, I uploaded my model but have been removed, first time it happens to me in a competition. \n\nJust filled in compliance form to find out what has happened.", "317334": "I disagree with the removal decision taken after stage 1 end", "317384": "Upvoted this. Think things are handled pretty badly, removing half of the leaderboard without clear explanation. If the reason really is not uploading of model, then it wasn't explicitly stated. If it is other reasons, at least an explanation could be given.", "317348": "The rules is the rule. If change it all the time，It was rude to the contestants.", "317990": "I agree with you. Kaggle should get us back to the LB. There should not have a change of the rules during the competition!!!", "317712": "I just checked the LB, to take a screenshot about my status and I am not on the LB... It was a pretty big surprise, but as I saw this thread I knew what happened...\nI did not upload my code on purpose because I knew I won't win and I also totally forget about it (because of University and work stuff)... But I've worked on my model/solution for more than a month and now there is no sign on the LB.\n\nI don't know what to say. It is really sad that they do such thing against their own rules.", "317332": "I also was confused by the rules ,everyone in this competition pay many effort ,although I enjoy the process,but the result is also very import .", "317327": "I did not know everyone had to upload their model.....this was my first competition and I thought only prize winners were required to upload their model as per the rules.Maybe I was confused...anyways I learned a lot and it was fun.", "317356": "Thank you for the thread! I disagree with the removal decision!", "317337": "can't agree any more. I need a reasonable explanation on this issue.", "319406": "Mine was also removed. Anyone from this competition host, please answer this thread.", "318524": "FYI, Maggie from Customer Success at Kaggle has provided updates [Change of Course - Leaderboard Reinstatement][1]. This should be good news for those impacted.\n\n@Maggie, thank you very much for listening to our feedbacks and taking necessary actions to get this corrected, very much appreciated. We'll wait for the reinstatement then.\n\n\n [1]: https://www.kaggle.com/c/data-science-bowl-2018/discussion/55228", "318513": "I agree with you, why my team has been removed", "318218": "This is my second 2 stage competition and I didn't get removed from the LB for the 1st competition. How interesting.", "318136": "I agreed with you", "318077": "I have already selected two results to submit, but the ranking disappear after 2 days. It difficult to understand what happen", "317335": "This was my first competition and was a solo participant, so was not aware to upload model, big lesson learned.\n\nIs there a way to get the result back. I only have a snapshot of my rank 22 which was a silver eligible. I was not aware i had to upload my model etc.. Also in submission page only result upload was there.. \n\nPlease help..", "317662": "", "317529": "" } }