Since all the kernels have been rerun, should we get another chance to choose the submissions again? the pseudo training is not stable and such a rerun is totally not fair!
", "messages": [ { "id": "965988", "postDate": "08/11/2020 03:16:49", "content": "Since all the kernels have been rerun, should we get another chance to choose the submissions again? the pseudo training is not stable and such a rerun is totally not fair!
", "rawMarkdown": "Since all the kernels have been rerun, should we get another chance to choose the submissions again? the pseudo training is not stable and such a rerun is totally not fair!", "votes": null }, { "id": "966500", "postDate": "08/11/2020 13:19:07", "content": "To be fair, I don't understand why they reran the kernels. This heavily affects pseudo labeling and thus to me this is a valid question.
", "rawMarkdown": "To be fair, I don't understand why they reran the kernels. This heavily affects pseudo labeling and thus to me this is a valid question.", "votes": null }, { "id": "967380", "postDate": "08/12/2020 08:08:07", "content": "Honestly, after seeing scores on the private set, you can not allow people to choose again. One of my submissions got luckier than the others on the private set and now I know which one. I should not be allowed to select that one now. ( I did not use PL ).
\nBut yes, rerunning seems to be a mistake. I don’t understand why they couldn’t just update the GT and rescore the submissions.
\nOr if they planned to rerun and for whatever reason they want solutions which are stable under a rerun, then why they would not just let the competitors know about that they will rerun? And why they don’t take the time to explain why are they doing a rerun?
\nIt seems like they not only ignored questions about this competition, but they also did not even bother to read the discussions, people warned about reruns when they first mentioned it.
\nI find this attitude weird for a competition organized by a University also linked to some scientific workshop. Scientist usually care more about stuff, e.g.: look at the Cornell Bird Identification challenge. The organizers are very active and engage in the discussions.
", "rawMarkdown": "Honestly, after seeing scores on the private set, you can not allow people to choose again. One of my submissions got luckier than the others on the private set and now I know which one. I should not be allowed to select that one now. ( I did not use PL ).\n\nBut yes, rerunning seems to be a mistake. I don’t understand why they couldn’t just update the GT and rescore the submissions. \n\nOr if they planned to rerun and for whatever reason they want solutions which are stable under a rerun, then why they would not just let the competitors know about that they will rerun? And why they don’t take the time to explain why are they doing a rerun?\n\nIt seems like they not only ignored questions about this competition, but they also did not even bother to read the discussions, people warned about reruns when they first mentioned it.\n\nI find this attitude weird for a competition organized by a University also linked to some scientific workshop. Scientist usually care more about stuff, e.g.: look at the Cornell Bird Identification challenge. The organizers are very active and engage in the discussions.", "votes": null } ], "comments": [ { "id": 966500, "author_name": "philippsinger", "author_url": "", "post_date": "08/11/2020 13:19:07", "content": "To be fair, I don't understand why they reran the kernels. This heavily affects pseudo labeling and thus to me this is a valid question.
", "votes": null, "replies": [] }, { "id": 967380, "author_name": "riblidezso", "author_url": "", "post_date": "08/12/2020 08:08:07", "content": "Honestly, after seeing scores on the private set, you can not allow people to choose again. One of my submissions got luckier than the others on the private set and now I know which one. I should not be allowed to select that one now. ( I did not use PL ).
\nBut yes, rerunning seems to be a mistake. I don’t understand why they couldn’t just update the GT and rescore the submissions.
\nOr if they planned to rerun and for whatever reason they want solutions which are stable under a rerun, then why they would not just let the competitors know about that they will rerun? And why they don’t take the time to explain why are they doing a rerun?
\nIt seems like they not only ignored questions about this competition, but they also did not even bother to read the discussions, people warned about reruns when they first mentioned it.
\nI find this attitude weird for a competition organized by a University also linked to some scientific workshop. Scientist usually care more about stuff, e.g.: look at the Cornell Bird Identification challenge. The organizers are very active and engage in the discussions.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "965988": "Since all the kernels have been rerun, should we get another chance to choose the submissions again? the pseudo training is not stable and such a rerun is totally not fair!", "966500": "To be fair, I don't understand why they reran the kernels. This heavily affects pseudo labeling and thus to me this is a valid question.", "967380": "Honestly, after seeing scores on the private set, you can not allow people to choose again. One of my submissions got luckier than the others on the private set and now I know which one. I should not be allowed to select that one now. ( I did not use PL ).\n\nBut yes, rerunning seems to be a mistake. I don’t understand why they couldn’t just update the GT and rescore the submissions. \n\nOr if they planned to rerun and for whatever reason they want solutions which are stable under a rerun, then why they would not just let the competitors know about that they will rerun? And why they don’t take the time to explain why are they doing a rerun?\n\nIt seems like they not only ignored questions about this competition, but they also did not even bother to read the discussions, people warned about reruns when they first mentioned it.\n\nI find this attitude weird for a competition organized by a University also linked to some scientific workshop. Scientist usually care more about stuff, e.g.: look at the Cornell Bird Identification challenge. The organizers are very active and engage in the discussions." }, "source": "meta" }