On the leaderboard section, it's written that the result will be based on 38% of the test dataset and the current public leaderboard is calculated on 62% of the dataset. So my question is: Will the final submission be evaluated ONLY on 38% of the test data or all the test data (62% + 38%)?
", "messages": [ { "id": "898682", "postDate": "06/23/2020 17:09:19", "content": "On the leaderboard section, it's written that the result will be based on 38% of the test dataset and the current public leaderboard is calculated on 62% of the dataset. So my question is: Will the final submission be evaluated ONLY on 38% of the test data or all the test data (62% + 38%)?
", "rawMarkdown": "On the leaderboard section, it's written that the result will be based on 38% of the test dataset and the current public leaderboard is calculated on 62% of the dataset. So my question is: Will the final submission be evaluated **ONLY** on 38% of the test data or all the test data (62% + 38%)?", "votes": null }, { "id": "898755", "postDate": "06/23/2020 17:59:17", "content": "Public LB is 62% of the test set, the final/private is 38%
", "rawMarkdown": "Public LB is 62% of the test set, the final/private is 38%", "votes": null }, { "id": "899236", "postDate": "06/24/2020 05:12:52", "content": "So that means only 38% of test data right? I want to estimate the runtime of my inferences.
", "rawMarkdown": "So that means only 38% of test data right? I want to estimate the runtime of my inferences.", "votes": null }, { "id": "899261", "postDate": "06/24/2020 05:54:37", "content": "As a follow up, do you guys know if you can make a submission longer than the time limit in the requirements? As in, does my system have to perform inference on the 62% in less than 6 hours or can it be 6 hours on just the private test data?
", "rawMarkdown": "As a follow up, do you guys know if you can make a submission longer than the time limit in the requirements? As in, does my system have to perform inference on the 62% in less than 6 hours or can it be 6 hours on just the private test data?", "votes": null }, { "id": "899285", "postDate": "06/24/2020 06:14:46", "content": "@matthewmasters Hmm...Good question! In my question, I was assuming the point that our submission should run at max 6 hours on just private test data (6 hours on 38% of the data). But now I'm curious. Let's ask the host: @bendvd
", "rawMarkdown": "matthewmasters Hmm...Good question! In my question, I was assuming the point that our submission should run at max 6 hours on just private test data (6 hours on 38% of the data). But now I'm curious. Let's ask the host: @bendvd", "votes": null }, { "id": "899297", "postDate": "06/24/2020 06:24:48", "content": "When you commit your notebook, it has access to the publicly available test set (10 samples). When you submit the results, Kaggle re-runs your notebook. This \"re-run\" has to finish in 6 hours. During this, your script has access to ALL samples (~1200). Kaggle will calculate both the public and the private LB. (The private LB is ready as well, they just don't show it to us)
", "rawMarkdown": "When you **commit** your notebook, it has access to the publicly available test set (10 samples). When you **submit** the results, Kaggle re-runs your notebook. This \"re-run\" has to finish in 6 hours. During this, your script has access to **ALL** samples (~1200). Kaggle will calculate both the public and the private LB. (The private LB is ready as well, they just don't show it to us)", "votes": null }, { "id": "899323", "postDate": "06/24/2020 06:56:39", "content": "@pestipeti Thanks for explaining better! I understand it now
", "rawMarkdown": "pestipeti Thanks for explaining better! I understand it now", "votes": null }, { "id": "900989", "postDate": "06/25/2020 07:34:39", "content": "I can only confirm @pestipeti answer :)
", "rawMarkdown": "I can only confirm @pestipeti answer :)", "votes": null }, { "id": "901027", "postDate": "06/25/2020 07:58:44", "content": "@bendvd Thanks!
", "rawMarkdown": "bendvd Thanks!", "votes": null } ], "comments": [ { "id": 898755, "author_name": "pestipeti", "author_url": "", "post_date": "06/23/2020 17:59:17", "content": "Public LB is 62% of the test set, the final/private is 38%
", "votes": null, "replies": [ { "id": 899236, "author_name": "kaushal2896", "author_url": "", "post_date": "06/24/2020 05:12:52", "content": "So that means only 38% of test data right? I want to estimate the runtime of my inferences.
", "votes": null, "replies": [] }, { "id": 899261, "author_name": "matthewmasters", "author_url": "", "post_date": "06/24/2020 05:54:37", "content": "As a follow up, do you guys know if you can make a submission longer than the time limit in the requirements? As in, does my system have to perform inference on the 62% in less than 6 hours or can it be 6 hours on just the private test data?
", "votes": null, "replies": [] }, { "id": 899285, "author_name": "kaushal2896", "author_url": "", "post_date": "06/24/2020 06:14:46", "content": "@matthewmasters Hmm...Good question! In my question, I was assuming the point that our submission should run at max 6 hours on just private test data (6 hours on 38% of the data). But now I'm curious. Let's ask the host: @bendvd
", "votes": null, "replies": [] }, { "id": 899297, "author_name": "pestipeti", "author_url": "", "post_date": "06/24/2020 06:24:48", "content": "When you commit your notebook, it has access to the publicly available test set (10 samples). When you submit the results, Kaggle re-runs your notebook. This \"re-run\" has to finish in 6 hours. During this, your script has access to ALL samples (~1200). Kaggle will calculate both the public and the private LB. (The private LB is ready as well, they just don't show it to us)
", "votes": null, "replies": [] }, { "id": 899323, "author_name": "matthewmasters", "author_url": "", "post_date": "06/24/2020 06:56:39", "content": "@pestipeti Thanks for explaining better! I understand it now
", "votes": null, "replies": [] }, { "id": 900989, "author_name": "bendvd", "author_url": "", "post_date": "06/25/2020 07:34:39", "content": "I can only confirm @pestipeti answer :)
", "votes": null, "replies": [] }, { "id": 901027, "author_name": "kaushal2896", "author_url": "", "post_date": "06/25/2020 07:58:44", "content": "@bendvd Thanks!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "898682": "On the leaderboard section, it's written that the result will be based on 38% of the test dataset and the current public leaderboard is calculated on 62% of the dataset. So my question is: Will the final submission be evaluated **ONLY** on 38% of the test data or all the test data (62% + 38%)?", "898755": "Public LB is 62% of the test set, the final/private is 38%", "899236": "So that means only 38% of test data right? I want to estimate the runtime of my inferences.", "899261": "As a follow up, do you guys know if you can make a submission longer than the time limit in the requirements? As in, does my system have to perform inference on the 62% in less than 6 hours or can it be 6 hours on just the private test data?", "899285": "matthewmasters Hmm...Good question! In my question, I was assuming the point that our submission should run at max 6 hours on just private test data (6 hours on 38% of the data). But now I'm curious. Let's ask the host: @bendvd", "899297": "When you **commit** your notebook, it has access to the publicly available test set (10 samples). When you **submit** the results, Kaggle re-runs your notebook. This \"re-run\" has to finish in 6 hours. During this, your script has access to **ALL** samples (~1200). Kaggle will calculate both the public and the private LB. (The private LB is ready as well, they just don't show it to us)", "899323": "pestipeti Thanks for explaining better! I understand it now", "900989": "I can only confirm @pestipeti answer :)", "901027": "bendvd Thanks!" }, "source": "meta" }