{ "id": 542076, "title": "Private Leaderboard - how does it work for this competition?", "url": "/competitions/brist1d/discussion/542076", "author_name": "", "post_date": "2024-10-22T22:36:12.295512700Z", "votes": 2, "comment_count": 5, "views": 0, "content": "

I can see from the Data description that:

\n
\n

This leaderboard is calculated with approximately 29% of the test data. The final results will be based on the other 71%, so the final standings may be different.

\n
\n

Does this mean that all test features (Private & Public) are contained in test.csv (minus the target of bg+1:00), but that only 29% of these rows are matched against ground truth and scored for the Public Leaderboard?

\n

Could you confirm, if possible, please @samajames?

", "messages": [ { "id": "3025555", "postDate": "10/22/2024 22:36:12", "content": "

I can see from the Data description that:

\n
\n

This leaderboard is calculated with approximately 29% of the test data. The final results will be based on the other 71%, so the final standings may be different.

\n
\n

Does this mean that all test features (Private & Public) are contained in test.csv (minus the target of bg+1:00), but that only 29% of these rows are matched against ground truth and scored for the Public Leaderboard?

\n

Could you confirm, if possible, please @samajames?

", "rawMarkdown": "I can see from the Data description that:\n\n>This leaderboard is calculated with approximately 29% of the test data. The final results will be based on the other 71%, so the final standings may be different.\n\nDoes this mean that all test features (Private & Public) are contained in `test.csv` (minus the target of bg+1:00), but that only 29% of these rows are matched against ground truth and scored for the Public Leaderboard?\n\nCould you confirm, if possible, please @samajames?", "votes": null }, { "id": "3025571", "postDate": "10/23/2024 00:00:09", "content": "

Yes,you're right.

", "rawMarkdown": "Yes,you're right.", "votes": null }, { "id": "3025952", "postDate": "10/23/2024 09:48:25", "content": "

Hi @jagofc,

\n

Yes, as @yunsuxiaozi says, you're correct in your interpretation of that. The remaining 71% is matched against the ground truth for the private Leaderboard.

", "rawMarkdown": "Hi @jagofc,\n\nYes, as @yunsuxiaozi says, you're correct in your interpretation of that. The remaining 71% is matched against the ground truth for the private Leaderboard.", "votes": null }, { "id": "3026057", "postDate": "10/23/2024 12:09:04", "content": "

Great, thank you @samajames & @yunsuxiaozi. So we should not be expecting any unseen participant ids? I noted that we span p_01 to p_24 in the test.csv, but we don't have data for 24 participants when summing up the participants across both train and test datasets. I was just double checking that these aren't going to suddenly appear?

\n

If your intention is to test these models on a broader population after the competition, @samajames, would you like for us to account for unseen participants too?

", "rawMarkdown": "Great, thank you @samajames & @yunsuxiaozi. So we should not be expecting any unseen participant ids? I noted that we span p_01 to p_24 in the test.csv, but we don't have data for 24 participants when summing up the participants across both train and test datasets. I was just double checking that these aren't going to suddenly appear?\n\nIf your intention is to test these models on a broader population after the competition, @samajames, would you like for us to account for unseen participants too?", "votes": null }, { "id": "3026220", "postDate": "10/23/2024 14:46:28", "content": "

No, there won't be any unseen participant ids beyond those that appear in the test.csv used in the competition. The missing participant ids are from study participants who didn't have enough data and so were excluded from the competition. When the full dataset is released after the competition there will be some additional participants that don't appear in the competition dataset.

\n

My aim is for the competition to form a benchmark that researchers can then compare models they develop against when working with the full dataset. I wouldn't worry about trying to account for unseen participants in your submission beyond the new ones that appear in the test.csv that aren't in the train.csv.

", "rawMarkdown": "No, there won't be any unseen participant ids beyond those that appear in the **test.csv** used in the competition. The missing participant ids are from study participants who didn't have enough data and so were excluded from the competition. When the full dataset is released after the competition there will be some additional participants that don't appear in the competition dataset.\n\nMy aim is for the competition to form a benchmark that researchers can then compare models they develop against when working with the full dataset. I wouldn't worry about trying to account for unseen participants in your submission beyond the new ones that appear in the **test.csv** that aren't in the **train.csv**.", "votes": null }, { "id": "3026352", "postDate": "10/23/2024 17:21:52", "content": "

Awesome, thanks for the context @samajames, very helpful.

", "rawMarkdown": "Awesome, thanks for the context @samajames, very helpful.", "votes": null } ], "comments": [ { "id": 3025571, "author_name": "yunsuxiaozi", "author_url": "", "post_date": "10/23/2024 00:00:09", "content": "

Yes,you're right.

", "votes": null, "replies": [] }, { "id": 3025952, "author_name": "samajames", "author_url": "", "post_date": "10/23/2024 09:48:25", "content": "

Hi @jagofc,

\n

Yes, as @yunsuxiaozi says, you're correct in your interpretation of that. The remaining 71% is matched against the ground truth for the private Leaderboard.

", "votes": null, "replies": [ { "id": 3026057, "author_name": "jagofc", "author_url": "", "post_date": "10/23/2024 12:09:04", "content": "

Great, thank you @samajames & @yunsuxiaozi. So we should not be expecting any unseen participant ids? I noted that we span p_01 to p_24 in the test.csv, but we don't have data for 24 participants when summing up the participants across both train and test datasets. I was just double checking that these aren't going to suddenly appear?

\n

If your intention is to test these models on a broader population after the competition, @samajames, would you like for us to account for unseen participants too?

", "votes": null, "replies": [ { "id": 3026220, "author_name": "samajames", "author_url": "", "post_date": "10/23/2024 14:46:28", "content": "

No, there won't be any unseen participant ids beyond those that appear in the test.csv used in the competition. The missing participant ids are from study participants who didn't have enough data and so were excluded from the competition. When the full dataset is released after the competition there will be some additional participants that don't appear in the competition dataset.

\n

My aim is for the competition to form a benchmark that researchers can then compare models they develop against when working with the full dataset. I wouldn't worry about trying to account for unseen participants in your submission beyond the new ones that appear in the test.csv that aren't in the train.csv.

", "votes": null, "replies": [ { "id": 3026352, "author_name": "jagofc", "author_url": "", "post_date": "10/23/2024 17:21:52", "content": "

Awesome, thanks for the context @samajames, very helpful.

", "votes": null, "replies": [] } ] } ] } ] } ], "raw_markdown_by_id": { "3025555": "I can see from the Data description that:\n\n>This leaderboard is calculated with approximately 29% of the test data. The final results will be based on the other 71%, so the final standings may be different.\n\nDoes this mean that all test features (Private & Public) are contained in `test.csv` (minus the target of bg+1:00), but that only 29% of these rows are matched against ground truth and scored for the Public Leaderboard?\n\nCould you confirm, if possible, please @samajames?", "3025571": "Yes,you're right.", "3025952": "Hi @jagofc,\n\nYes, as @yunsuxiaozi says, you're correct in your interpretation of that. The remaining 71% is matched against the ground truth for the private Leaderboard.", "3026057": "Great, thank you @samajames & @yunsuxiaozi. So we should not be expecting any unseen participant ids? I noted that we span p_01 to p_24 in the test.csv, but we don't have data for 24 participants when summing up the participants across both train and test datasets. I was just double checking that these aren't going to suddenly appear?\n\nIf your intention is to test these models on a broader population after the competition, @samajames, would you like for us to account for unseen participants too?", "3026220": "No, there won't be any unseen participant ids beyond those that appear in the **test.csv** used in the competition. The missing participant ids are from study participants who didn't have enough data and so were excluded from the competition. When the full dataset is released after the competition there will be some additional participants that don't appear in the competition dataset.\n\nMy aim is for the competition to form a benchmark that researchers can then compare models they develop against when working with the full dataset. I wouldn't worry about trying to account for unseen participants in your submission beyond the new ones that appear in the **test.csv** that aren't in the **train.csv**.", "3026352": "Awesome, thanks for the context @samajames, very helpful." }, "source": "meta" }