I just joined a team and realized that the 5 daily submission attempts are shared among all members. This is irreversible once you join — you can’t go back to competing individually if you want to stay in the team.
\nThis creates an interesting situation: teams formed at the beginning of the competition have only 5 total daily attempts. However, if several users decide to form a team on the last day, each of them will already have made their 5 individual submissions, meaning that, in practice, that group will have made 5 × number of participants submissions before joining.
\nDoesn’t that seem like a disadvantage for those who collaborate from the start? Or do you think it’s a fair way to balance the risk of coordinating at the end?
\nLet’s not forget that it’s possible to coordinate outside Kaggle and take advantage of this situation. It shouldn’t be considered cheating, right?
", "messages": [ { "id": "3308057", "postDate": "10/28/2025 13:45:10", "content": "I just joined a team and realized that the 5 daily submission attempts are shared among all members. This is irreversible once you join — you can’t go back to competing individually if you want to stay in the team.
\nThis creates an interesting situation: teams formed at the beginning of the competition have only 5 total daily attempts. However, if several users decide to form a team on the last day, each of them will already have made their 5 individual submissions, meaning that, in practice, that group will have made 5 × number of participants submissions before joining.
\nDoesn’t that seem like a disadvantage for those who collaborate from the start? Or do you think it’s a fair way to balance the risk of coordinating at the end?
\nLet’s not forget that it’s possible to coordinate outside Kaggle and take advantage of this situation. It shouldn’t be considered cheating, right?
", "rawMarkdown": "I just joined a team and realized that the 5 daily submission attempts are shared among all members. This is irreversible once you join — you can’t go back to competing individually if you want to stay in the team.\n\nThis creates an interesting situation: teams formed at the beginning of the competition have only 5 total daily attempts. However, if several users decide to form a team on the last day, each of them will already have made their 5 individual submissions, meaning that, in practice, that group will have made 5 × number of participants submissions before joining.\n\nDoesn’t that seem like a disadvantage for those who collaborate from the start? Or do you think it’s a fair way to balance the risk of coordinating at the end?\n\nLet’s not forget that it’s possible to coordinate outside Kaggle and take advantage of this situation. It shouldn’t be considered cheating, right?", "votes": null }, { "id": "3308122", "postDate": "10/28/2025 16:10:15", "content": "This is an interesting perspective, but keep in mind that:
\n\n\nThe leaderboard was designed to display method performance on a relatively small selection of proteins from the test superset. Some distribution shift between the sample of proteins used for the leaderboard and the final evaluation sample is to be expected.
\n
The evaluation from the 5 daily submissions may not be as valuable a signal as a well-constructed validation set and thoughtful validation pipeline run off-platform.
", "rawMarkdown": "This is an interesting perspective, but keep in mind that: \n>The leaderboard was designed to display method performance on a relatively small selection of proteins from the test superset. Some distribution shift between the sample of proteins used for the leaderboard and the final evaluation sample is to be expected. \n\nThe evaluation from the 5 daily submissions may not be as valuable a signal as a well-constructed validation set and thoughtful validation pipeline run off-platform.", "votes": null }, { "id": "3308131", "postDate": "10/28/2025 16:54:47", "content": "Yes, but at this stage that signal is extremely valuable for exploratory analysis and reverse-modeling of the expected dataset.
\nFor instance, I'm still aligning my reference dataset to my predictions — there are many open questions related to annotation datasources, evidence codes, taxonomy filters, redundancy strategies and so on. In that context, there’s no better way to gain insights than through iterative submissions.
\nEven if the leaderboard subset is not fully representative, those submission results provide indirect validation that helps refine both the model and the preprocessing assumptions.
\nMaybe, what I should do is use CAFA4/5 as reference first.
", "rawMarkdown": "Yes, but at this stage that signal is extremely valuable for exploratory analysis and reverse-modeling of the expected dataset.\n\nFor instance, I'm still aligning my reference dataset to my predictions — there are many open questions related to annotation datasources, evidence codes, taxonomy filters, redundancy strategies and so on. In that context, there’s no better way to gain insights than through iterative submissions.\n\nEven if the leaderboard subset is not fully representative, those submission results provide indirect validation that helps refine both the model and the preprocessing assumptions.\n\nMaybe, what I should do is use CAFA4/5 as reference first.", "votes": null }, { "id": "3308149", "postDate": "10/28/2025 17:58:36", "content": "There are some advantage for teaming up later, but they may not be what you thought.
\n\n\nHowever, if several users decide to form a team on the last day, each of them will already have made their 5 individual submissions, meaning that, in practice, that group will have made 5 × number of participants submissions before joining.
\n
Not possible to form a team on the last day - it has to be done at least a week before the deadline. A sum of all submission for those who wish to form a team can't be larger than the maximum number of submissions a single member could have made up to that same day. So no, what you describe would not be possible and the merger wouldn't go. In case it is not clear: let's say that after 10 days (meaning a maximum of 50 submissions allowed) you decide to team up with someone, and both of you have made 30 individual submissions. You would have to wait 2 days without either of you submitting before you'd be allowed to team up. At day 12 the maximum number of submissions would rise to 60, in which case your combined submission total would be fine.
\n\n\nLet’s not forget that it’s possible to coordinate outside Kaggle and take advantage of this situation. It shouldn’t be considered cheating, right?
\n
This is absolutely considered cheating and it is not allowed. You can share the code publicly, that's allowed. Those who share the code privately with non-teammates, if caught, will be disqualified.
", "rawMarkdown": "There are some advantage for teaming up later, but they may not be what you thought.\n\n> However, if several users decide to form a team on the last day, each of them will already have made their 5 individual submissions, meaning that, in practice, that group will have made 5 × number of participants submissions before joining.\n\nNot possible to form a team on the last day - it has to be done at least a week before the deadline. A sum of all submission for those who wish to form a team can't be larger than the maximum number of submissions a single member could have made up to that same day. So no, what you describe would not be possible and the merger wouldn't go. In case it is not clear: let's say that after 10 days (meaning a maximum of 50 submissions allowed) you decide to team up with someone, and both of you have made 30 individual submissions. You would have to wait 2 days without either of you submitting before you'd be allowed to team up. At day 12 the maximum number of submissions would rise to 60, in which case your combined submission total would be fine.\n\n> Let’s not forget that it’s possible to coordinate outside Kaggle and take advantage of this situation. It shouldn’t be considered cheating, right?\n\nThis is absolutely considered cheating and it is not allowed. You can share the code publicly, that's allowed. Those who share the code privately with non-teammates, if caught, will be disqualified.", "votes": null }, { "id": "3308160", "postDate": "10/28/2025 18:43:12", "content": "If I want to make 5 daily submissions on my own, I need to compete solo. Got it.
\nIt does feel like a bit of a penalty for teams, but that’s okay.
\nNever mind, it’s not really worth it. It’s fine as it is.\nThanks
", "rawMarkdown": "If I want to make 5 daily submissions on my own, I need to compete solo. Got it.\n\nIt does feel like a bit of a penalty for teams, but that’s okay.\n\nNever mind, it’s not really worth it. It’s fine as it is.\nThanks", "votes": null }, { "id": "3308172", "postDate": "10/28/2025 19:12:20", "content": "\n\nIt does feel like a bit of a penalty for teams, but that’s okay.
\n
In a competition all of us are considered teams - it's just that most teams have a single member. That's why you can go to a Team tag and change a team name that will display on the LB without changing your username. Whether a team has a single member or 5, the number of allowed submissions is the same.
\nTeams with multiple members have an advantage of combining multiple points of view and extra expertise. If on top of that multi-member teams would be allowed [max_submissions * #_of_members], their advantage would further grow.
", "rawMarkdown": "> It does feel like a bit of a penalty for teams, but that’s okay.\n\nIn a competition all of us are considered teams - it's just that most teams have a single member. That's why you can go to a [**Team tag**](https://www.kaggle.com/competitions/cafa-6-protein-function-prediction/team) and change a team name that will display on the LB without changing your username. Whether a team has a single member or 5, the number of allowed submissions is the same.\n\nTeams with multiple members have an advantage of combining multiple points of view and extra expertise. If on top of that multi-member teams would be allowed [max_submissions * #_of_members], their advantage would further grow.", "votes": null }, { "id": "3308487", "postDate": "10/29/2025 14:42:50", "content": "Sorry, this is a conversation that doesn’t really lead anywhere.\nBetter to focus on other things. Thanks a lot for your clarifications — let’s just enjoy the competition!
", "rawMarkdown": "Sorry, this is a conversation that doesn’t really lead anywhere.\nBetter to focus on other things. Thanks a lot for your clarifications — let’s just enjoy the competition!", "votes": null } ], "comments": [ { "id": 3308122, "author_name": "claradepaolis", "author_url": "", "post_date": "10/28/2025 16:10:15", "content": "This is an interesting perspective, but keep in mind that:
\n\n\nThe leaderboard was designed to display method performance on a relatively small selection of proteins from the test superset. Some distribution shift between the sample of proteins used for the leaderboard and the final evaluation sample is to be expected.
\n
The evaluation from the 5 daily submissions may not be as valuable a signal as a well-constructed validation set and thoughtful validation pipeline run off-platform.
", "votes": null, "replies": [ { "id": 3308131, "author_name": "xaxipirulazo", "author_url": "", "post_date": "10/28/2025 16:54:47", "content": "Yes, but at this stage that signal is extremely valuable for exploratory analysis and reverse-modeling of the expected dataset.
\nFor instance, I'm still aligning my reference dataset to my predictions — there are many open questions related to annotation datasources, evidence codes, taxonomy filters, redundancy strategies and so on. In that context, there’s no better way to gain insights than through iterative submissions.
\nEven if the leaderboard subset is not fully representative, those submission results provide indirect validation that helps refine both the model and the preprocessing assumptions.
\nMaybe, what I should do is use CAFA4/5 as reference first.
", "votes": null, "replies": [] } ] }, { "id": 3308149, "author_name": "tilii7", "author_url": "", "post_date": "10/28/2025 17:58:36", "content": "There are some advantage for teaming up later, but they may not be what you thought.
\n\n\nHowever, if several users decide to form a team on the last day, each of them will already have made their 5 individual submissions, meaning that, in practice, that group will have made 5 × number of participants submissions before joining.
\n
Not possible to form a team on the last day - it has to be done at least a week before the deadline. A sum of all submission for those who wish to form a team can't be larger than the maximum number of submissions a single member could have made up to that same day. So no, what you describe would not be possible and the merger wouldn't go. In case it is not clear: let's say that after 10 days (meaning a maximum of 50 submissions allowed) you decide to team up with someone, and both of you have made 30 individual submissions. You would have to wait 2 days without either of you submitting before you'd be allowed to team up. At day 12 the maximum number of submissions would rise to 60, in which case your combined submission total would be fine.
\n\n\nLet’s not forget that it’s possible to coordinate outside Kaggle and take advantage of this situation. It shouldn’t be considered cheating, right?
\n
This is absolutely considered cheating and it is not allowed. You can share the code publicly, that's allowed. Those who share the code privately with non-teammates, if caught, will be disqualified.
", "votes": null, "replies": [ { "id": 3308160, "author_name": "xaxipirulazo", "author_url": "", "post_date": "10/28/2025 18:43:12", "content": "If I want to make 5 daily submissions on my own, I need to compete solo. Got it.
\nIt does feel like a bit of a penalty for teams, but that’s okay.
\nNever mind, it’s not really worth it. It’s fine as it is.\nThanks
", "votes": null, "replies": [ { "id": 3308172, "author_name": "tilii7", "author_url": "", "post_date": "10/28/2025 19:12:20", "content": "\n\nIt does feel like a bit of a penalty for teams, but that’s okay.
\n
In a competition all of us are considered teams - it's just that most teams have a single member. That's why you can go to a Team tag and change a team name that will display on the LB without changing your username. Whether a team has a single member or 5, the number of allowed submissions is the same.
\nTeams with multiple members have an advantage of combining multiple points of view and extra expertise. If on top of that multi-member teams would be allowed [max_submissions * #_of_members], their advantage would further grow.
", "votes": null, "replies": [ { "id": 3308487, "author_name": "xaxipirulazo", "author_url": "", "post_date": "10/29/2025 14:42:50", "content": "Sorry, this is a conversation that doesn’t really lead anywhere.\nBetter to focus on other things. Thanks a lot for your clarifications — let’s just enjoy the competition!
", "votes": null, "replies": [] } ] } ] } ] } ], "raw_markdown_by_id": { "3308057": "I just joined a team and realized that the 5 daily submission attempts are shared among all members. This is irreversible once you join — you can’t go back to competing individually if you want to stay in the team.\n\nThis creates an interesting situation: teams formed at the beginning of the competition have only 5 total daily attempts. However, if several users decide to form a team on the last day, each of them will already have made their 5 individual submissions, meaning that, in practice, that group will have made 5 × number of participants submissions before joining.\n\nDoesn’t that seem like a disadvantage for those who collaborate from the start? Or do you think it’s a fair way to balance the risk of coordinating at the end?\n\nLet’s not forget that it’s possible to coordinate outside Kaggle and take advantage of this situation. It shouldn’t be considered cheating, right?", "3308122": "This is an interesting perspective, but keep in mind that: \n>The leaderboard was designed to display method performance on a relatively small selection of proteins from the test superset. Some distribution shift between the sample of proteins used for the leaderboard and the final evaluation sample is to be expected. \n\nThe evaluation from the 5 daily submissions may not be as valuable a signal as a well-constructed validation set and thoughtful validation pipeline run off-platform.", "3308131": "Yes, but at this stage that signal is extremely valuable for exploratory analysis and reverse-modeling of the expected dataset.\n\nFor instance, I'm still aligning my reference dataset to my predictions — there are many open questions related to annotation datasources, evidence codes, taxonomy filters, redundancy strategies and so on. In that context, there’s no better way to gain insights than through iterative submissions.\n\nEven if the leaderboard subset is not fully representative, those submission results provide indirect validation that helps refine both the model and the preprocessing assumptions.\n\nMaybe, what I should do is use CAFA4/5 as reference first.", "3308149": "There are some advantage for teaming up later, but they may not be what you thought.\n\n> However, if several users decide to form a team on the last day, each of them will already have made their 5 individual submissions, meaning that, in practice, that group will have made 5 × number of participants submissions before joining.\n\nNot possible to form a team on the last day - it has to be done at least a week before the deadline. A sum of all submission for those who wish to form a team can't be larger than the maximum number of submissions a single member could have made up to that same day. So no, what you describe would not be possible and the merger wouldn't go. In case it is not clear: let's say that after 10 days (meaning a maximum of 50 submissions allowed) you decide to team up with someone, and both of you have made 30 individual submissions. You would have to wait 2 days without either of you submitting before you'd be allowed to team up. At day 12 the maximum number of submissions would rise to 60, in which case your combined submission total would be fine.\n\n> Let’s not forget that it’s possible to coordinate outside Kaggle and take advantage of this situation. It shouldn’t be considered cheating, right?\n\nThis is absolutely considered cheating and it is not allowed. You can share the code publicly, that's allowed. Those who share the code privately with non-teammates, if caught, will be disqualified.", "3308160": "If I want to make 5 daily submissions on my own, I need to compete solo. Got it.\n\nIt does feel like a bit of a penalty for teams, but that’s okay.\n\nNever mind, it’s not really worth it. It’s fine as it is.\nThanks", "3308172": "> It does feel like a bit of a penalty for teams, but that’s okay.\n\nIn a competition all of us are considered teams - it's just that most teams have a single member. That's why you can go to a [**Team tag**](https://www.kaggle.com/competitions/cafa-6-protein-function-prediction/team) and change a team name that will display on the LB without changing your username. Whether a team has a single member or 5, the number of allowed submissions is the same.\n\nTeams with multiple members have an advantage of combining multiple points of view and extra expertise. If on top of that multi-member teams would be allowed [max_submissions * #_of_members], their advantage would further grow.", "3308487": "Sorry, this is a conversation that doesn’t really lead anywhere.\nBetter to focus on other things. Thanks a lot for your clarifications — let’s just enjoy the competition!" }, "source": "meta" }