Just joined and I am seeing only four files in the test folder. Whatsup with that?
", "messages": [ { "id": "1254742", "postDate": "03/28/2021 04:15:06", "content": "Just joined and I am seeing only four files in the test folder. Whatsup with that?
", "rawMarkdown": "Just joined and I am seeing only four files in the test folder. Whatsup with that?", "votes": null }, { "id": "1254756", "postDate": "03/28/2021 04:51:52", "content": "These are for your local \"validation\", just to ensure that your code works as expected before you submit.
\nIn reality, there are about 8000 hidden test case json files,
\nout of which 12% i.e. about 1000 files are used to display Public Leaderboard.
@garrettpartenza I was wondering the same for the last three days.. but as @sirishks said, when you hit the submit buttom 8000 hidden test cases are considered and predictions on all of them are made..
\nBut for god's sake, that should be stated somewhere! :-)
", "rawMarkdown": "garrettpartenza I was wondering the same for the last three days.. but as @sirishks said, when you hit the submit buttom 8000 hidden test cases are considered and predictions on all of them are made.. \n\nBut for god's sake, that should be stated somewhere! :-)", "votes": null }, { "id": "1261248", "postDate": "04/02/2021 21:13:06", "content": "@sirishks I still didn't get completely. We are not submitting model, only predictions for 4 publications we are submitting. How this 1000 or 8000 publications id concept is coming. Please help me to understand. And how they are testing models ( which we didn't submit)
", "rawMarkdown": "sirishks I still didn't get completely. We are not submitting model, only predictions for 4 publications we are submitting. How this 1000 or 8000 publications id concept is coming. Please help me to understand. And how they are testing models ( which we didn't submit)", "votes": null }, { "id": "1262728", "postDate": "04/04/2021 16:54:43", "content": "@garrettpartenza My comment should have been clearer: your code should be in a way that it doesn't matter whether there are four or 8000 publications in the test set. In this way, the hidden publications in test set are also considered and you get your score based on the 12% of the test set.
\nAs stated when you submit: \"In this competition, we will privately re-run your selected Notebook Version with a hidden test set substituted into the competition dataset. We then extract your chosen Output File from the re-run and use that to determine your score.\"
", "rawMarkdown": "garrettpartenza My comment should have been clearer: your code should be in a way that it doesn't matter whether there are four or 8000 publications in the test set. In this way, the hidden publications in test set are also considered and you get your score based on the 12% of the test set.\n\nAs stated when you submit: \"In this competition, we will privately re-run your selected Notebook Version with a hidden test set substituted into the competition dataset. We then extract your chosen Output File from the re-run and use that to determine your score.\"", "votes": null }, { "id": "1289817", "postDate": "05/01/2021 11:59:28", "content": "@rajpur There is a hidden test set of 8000 publications which is not made available to the public. This is mentioned in the \"Data\" tab of the competition page. During the final submission, we have to select the notebook of our choice over which the predictions for the hidden test set will be computed.
", "rawMarkdown": "rajpur There is a hidden test set of 8000 publications which is not made available to the public. This is mentioned in the \"Data\" tab of the competition page. During the final submission, we have to select the notebook of our choice over which the predictions for the hidden test set will be computed.", "votes": null } ], "comments": [ { "id": 1254756, "author_name": "sirishks", "author_url": "", "post_date": "03/28/2021 04:51:52", "content": "These are for your local \"validation\", just to ensure that your code works as expected before you submit.
\nIn reality, there are about 8000 hidden test case json files,
\nout of which 12% i.e. about 1000 files are used to display Public Leaderboard.
@garrettpartenza I was wondering the same for the last three days.. but as @sirishks said, when you hit the submit buttom 8000 hidden test cases are considered and predictions on all of them are made..
\nBut for god's sake, that should be stated somewhere! :-)
", "votes": null, "replies": [] }, { "id": 1261248, "author_name": "rajpur", "author_url": "", "post_date": "04/02/2021 21:13:06", "content": "@sirishks I still didn't get completely. We are not submitting model, only predictions for 4 publications we are submitting. How this 1000 or 8000 publications id concept is coming. Please help me to understand. And how they are testing models ( which we didn't submit)
", "votes": null, "replies": [ { "id": 1289817, "author_name": "kartikbhargav", "author_url": "", "post_date": "05/01/2021 11:59:28", "content": "@rajpur There is a hidden test set of 8000 publications which is not made available to the public. This is mentioned in the \"Data\" tab of the competition page. During the final submission, we have to select the notebook of our choice over which the predictions for the hidden test set will be computed.
", "votes": null, "replies": [] } ] }, { "id": 1262728, "author_name": "hannes82", "author_url": "", "post_date": "04/04/2021 16:54:43", "content": "@garrettpartenza My comment should have been clearer: your code should be in a way that it doesn't matter whether there are four or 8000 publications in the test set. In this way, the hidden publications in test set are also considered and you get your score based on the 12% of the test set.
\nAs stated when you submit: \"In this competition, we will privately re-run your selected Notebook Version with a hidden test set substituted into the competition dataset. We then extract your chosen Output File from the re-run and use that to determine your score.\"
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1254742": "Just joined and I am seeing only four files in the test folder. Whatsup with that?", "1254756": "These are for your local \"validation\", just to ensure that your code works as expected before you submit.\nIn reality, there are about 8000 hidden test case json files,\nout of which 12% i.e. about 1000 files are used to display Public Leaderboard.", "1256272": "garrettpartenza I was wondering the same for the last three days.. but as @sirishks said, when you hit the submit buttom 8000 hidden test cases are considered and predictions on all of them are made.. \n\nBut for god's sake, that should be stated somewhere! :-)", "1261248": "sirishks I still didn't get completely. We are not submitting model, only predictions for 4 publications we are submitting. How this 1000 or 8000 publications id concept is coming. Please help me to understand. And how they are testing models ( which we didn't submit)", "1262728": "garrettpartenza My comment should have been clearer: your code should be in a way that it doesn't matter whether there are four or 8000 publications in the test set. In this way, the hidden publications in test set are also considered and you get your score based on the 12% of the test set.\n\nAs stated when you submit: \"In this competition, we will privately re-run your selected Notebook Version with a hidden test set substituted into the competition dataset. We then extract your chosen Output File from the re-run and use that to determine your score.\"", "1289817": "rajpur There is a hidden test set of 8000 publications which is not made available to the public. This is mentioned in the \"Data\" tab of the competition page. During the final submission, we have to select the notebook of our choice over which the predictions for the hidden test set will be computed." }, "source": "meta" }