After looking at scores of my selected submissions and reading most of the related comments, here is a short summary of what, I think, has happened
\nAs a conclusion, in my opinion, the current state of the public LB is questionable as it takes maximum over the old and new scores for each team. Not sure why the interim updates were not implemented as a part of the private LB, where only selected submissions are compared.
\nUPD1: LB is going to be rescored in order to fix the issue. Thanks to @inversion for the prompt reaction.
\nUPD2: LB update that fixes the issue seems to be completed, at least all the 1,639 teams from the last day competition stats are there now.
", "messages": [ { "id": "2510299", "postDate": "11/02/2023 21:35:37", "content": "After looking at scores of my selected submissions and reading most of the related comments, here is a short summary of what, I think, has happened
\nAs a conclusion, in my opinion, the current state of the public LB is questionable as it takes maximum over the old and new scores for each team. Not sure why the interim updates were not implemented as a part of the private LB, where only selected submissions are compared.
\nUPD1: LB is going to be rescored in order to fix the issue. Thanks to @inversion for the prompt reaction.
\nUPD2: LB update that fixes the issue seems to be completed, at least all the 1,639 teams from the last day competition stats are there now.
", "rawMarkdown": "After looking at scores of my selected submissions and reading most of the related comments, here is a short summary of what, I think, has happened\n\n- two selected submissions were rescored on the new data and their new scores replaced the old public scores;\n- all the unselected submissions were not rescored and maintain their old public scores;\n- public LB continues to function as a public LB, meaning that for each team it shows the overall best public score of all the submissions (whether selected or not).\n\nAs a conclusion, in my opinion, the current state of the public LB is questionable as it takes maximum over the old and new scores for each team. Not sure why the interim updates were not implemented as a part of the private LB, where only selected submissions are compared. \n\n**UPD1:** LB is going to be [rescored](https://www.kaggle.com/competitions/cafa-5-protein-function-prediction/discussion/452639) in order to fix the issue. Thanks to @inversion for the prompt reaction.\n\n**UPD2:** LB update that fixes the issue seems to be completed, at least all the 1,639 teams from [the last day competition stats](https://www.kaggle.com/competitions/cafa-5-protein-function-prediction/discussion/433184) are there now.", "votes": null }, { "id": "2510310", "postDate": "11/02/2023 21:53:01", "content": "I think you're right. So if that is the case, what is really the leading score in the LB. I personally don't like the current arrangement.
\nSo did your 2 selected submissions improve?
\nOne of my selections remained the same and the other deteriorated.
No, both my scores went down unfortunately.
", "rawMarkdown": "No, both my scores went down unfortunately.", "votes": null }, { "id": "2515511", "postDate": "11/07/2023 01:58:05", "content": "Dear @inversion, I just found that many teams (including mine) lost one submission on LB. Is it a part of some new update or simply a bug?
", "rawMarkdown": "Dear @inversion, I just found that many teams (including mine) lost one submission on LB. Is it a part of some new update or simply a bug?", "votes": null }, { "id": "2515534", "postDate": "11/07/2023 02:19:26", "content": "I will look into this first thing tomorrow morning.
", "rawMarkdown": "I will look into this first thing tomorrow morning.", "votes": null }, { "id": "2515561", "postDate": "11/07/2023 02:54:09", "content": "I see one of my submission running, is one more LB update happening?
", "rawMarkdown": "I see one of my submission running, is one more LB update happening?", "votes": null }, { "id": "2515805", "postDate": "11/07/2023 08:56:10", "content": "Both of our submissions got an \"error\" and we disappeared from the leaderboardâ€¦đŸ˜…
", "rawMarkdown": "Both of our submissions got an \"error\" and we disappeared from the leaderboard...đŸ˜…", "votes": null }, { "id": "2515854", "postDate": "11/07/2023 09:31:00", "content": "same here.
", "rawMarkdown": "same here.", "votes": null }, { "id": "2515917", "postDate": "11/07/2023 10:44:24", "content": "It's been more than 8 hours since my notice, submission is still running, will update once it is done.
", "rawMarkdown": "It's been more than 8 hours since my notice, submission is still running, will update once it is done.", "votes": null }, { "id": "2516335", "postDate": "11/07/2023 16:36:46", "content": "Everything should be fixed. Sorry for the hiccup!
\n(You still may see an error message associated with the submission. That is a bug on our end that we'll address.)
", "rawMarkdown": "Everything should be fixed. Sorry for the hiccup!\n\n(You still may see an error message associated with the submission. That is a bug on our end that we'll address.)", "votes": null }, { "id": "2516597", "postDate": "11/07/2023 19:57:07", "content": "Thank you for the quick fix @inversion!
", "rawMarkdown": "Thank you for the quick fix @inversion!", "votes": null } ], "comments": [ { "id": 2510310, "author_name": "victorogobi", "author_url": "", "post_date": "11/02/2023 21:53:01", "content": "I think you're right. So if that is the case, what is really the leading score in the LB. I personally don't like the current arrangement.
\nSo did your 2 selected submissions improve?
\nOne of my selections remained the same and the other deteriorated.
No, both my scores went down unfortunately.
", "votes": null, "replies": [] } ] }, { "id": 2515511, "author_name": "kononenko", "author_url": "", "post_date": "11/07/2023 01:58:05", "content": "Dear @inversion, I just found that many teams (including mine) lost one submission on LB. Is it a part of some new update or simply a bug?
", "votes": null, "replies": [ { "id": 2515534, "author_name": "inversion", "author_url": "", "post_date": "11/07/2023 02:19:26", "content": "I will look into this first thing tomorrow morning.
", "votes": null, "replies": [ { "id": 2515561, "author_name": "anilreddyvv", "author_url": "", "post_date": "11/07/2023 02:54:09", "content": "I see one of my submission running, is one more LB update happening?
", "votes": null, "replies": [] }, { "id": 2515805, "author_name": "jcerpentier", "author_url": "", "post_date": "11/07/2023 08:56:10", "content": "Both of our submissions got an \"error\" and we disappeared from the leaderboardâ€¦đŸ˜…
", "votes": null, "replies": [ { "id": 2515854, "author_name": "daishu", "author_url": "", "post_date": "11/07/2023 09:31:00", "content": "same here.
", "votes": null, "replies": [ { "id": 2515917, "author_name": "anilreddyvv", "author_url": "", "post_date": "11/07/2023 10:44:24", "content": "It's been more than 8 hours since my notice, submission is still running, will update once it is done.
", "votes": null, "replies": [ { "id": 2516335, "author_name": "inversion", "author_url": "", "post_date": "11/07/2023 16:36:46", "content": "Everything should be fixed. Sorry for the hiccup!
\n(You still may see an error message associated with the submission. That is a bug on our end that we'll address.)
", "votes": null, "replies": [ { "id": 2516597, "author_name": "kononenko", "author_url": "", "post_date": "11/07/2023 19:57:07", "content": "Thank you for the quick fix @inversion!
", "votes": null, "replies": [] } ] } ] } ] } ] } ] } ] } ], "raw_markdown_by_id": { "2510299": "After looking at scores of my selected submissions and reading most of the related comments, here is a short summary of what, I think, has happened\n\n- two selected submissions were rescored on the new data and their new scores replaced the old public scores;\n- all the unselected submissions were not rescored and maintain their old public scores;\n- public LB continues to function as a public LB, meaning that for each team it shows the overall best public score of all the submissions (whether selected or not).\n\nAs a conclusion, in my opinion, the current state of the public LB is questionable as it takes maximum over the old and new scores for each team. Not sure why the interim updates were not implemented as a part of the private LB, where only selected submissions are compared. \n\n**UPD1:** LB is going to be [rescored](https://www.kaggle.com/competitions/cafa-5-protein-function-prediction/discussion/452639) in order to fix the issue. Thanks to @inversion for the prompt reaction.\n\n**UPD2:** LB update that fixes the issue seems to be completed, at least all the 1,639 teams from [the last day competition stats](https://www.kaggle.com/competitions/cafa-5-protein-function-prediction/discussion/433184) are there now.", "2510310": "I think you're right. So if that is the case, what is really the leading score in the LB. I personally don't like the current arrangement.\nSo did your 2 selected submissions improve?\nOne of my selections remained the same and the other deteriorated.", "2510314": "No, both my scores went down unfortunately.", "2515511": "Dear @inversion, I just found that many teams (including mine) lost one submission on LB. Is it a part of some new update or simply a bug?", "2515534": "I will look into this first thing tomorrow morning.", "2515561": "I see one of my submission running, is one more LB update happening?", "2515805": "Both of our submissions got an \"error\" and we disappeared from the leaderboard...đŸ˜…", "2515854": "same here.", "2515917": "It's been more than 8 hours since my notice, submission is still running, will update once it is done.", "2516335": "Everything should be fixed. Sorry for the hiccup!\n\n(You still may see an error message associated with the submission. That is a bug on our end that we'll address.)", "2516597": "Thank you for the quick fix @inversion!" }, "source": "meta" }