Hi,
\n\nWhy does everyone who submitted a score for stage two have a score of zero? Is this because the scoring function hasn’t been updated yet or could this be that our models are over fitting? Also, to fix the problem of not predicting a mask for each image, my submission code returns a rle of “1 1” for each image without a predicted mask, could this also be causing the low scores on the leaderboard?
\n\nThanks
", "messages": [ { "id": "312647", "postDate": "04/12/2018 05:03:24", "content": "Hi,
\n\nWhy does everyone who submitted a score for stage two have a score of zero? Is this because the scoring function hasn’t been updated yet or could this be that our models are over fitting? Also, to fix the problem of not predicting a mask for each image, my submission code returns a rle of “1 1” for each image without a predicted mask, could this also be causing the low scores on the leaderboard?
\n\nThanks
", "rawMarkdown": "Hi,\n\nWhy does everyone who submitted a score for stage two have a score of zero? Is this because the scoring function hasn’t been updated yet or could this be that our models are over fitting? Also, to fix the problem of not predicting a mask for each image, my submission code returns a rle of “1 1” for each image without a predicted mask, could this also be causing the low scores on the leaderboard?\n\nThanks", "votes": null }, { "id": "312671", "postDate": "04/12/2018 05:53:24", "content": "I think may be they are just giving zero for all the submissions until the end of stage 2 so that we cannot probe the leader board.
", "rawMarkdown": "I think may be they are just giving zero for all the submissions until the end of stage 2 so that we cannot probe the leader board.", "votes": null }, { "id": "312672", "postDate": "04/12/2018 06:03:04", "content": "\"Each row in your submission represents a single predicted nucleus segmentation for the given ImageId.\"
\n\nI GUESS it should not have any row if the image has no nucleus.
\n\nBy the way, I got 0 score, too.
", "rawMarkdown": "\"Each row in your submission represents a single predicted nucleus segmentation for the given ImageId.\"\n\nI GUESS it should not have any row if the image has no nucleus.\n\nBy the way, I got 0 score, too.", "votes": null }, { "id": "312728", "postDate": "04/12/2018 07:56:00", "content": "I think this is right.
", "rawMarkdown": "I think this is right.", "votes": null }, { "id": "312729", "postDate": "04/12/2018 08:03:09", "content": "If I submit multiple results, which one should be chosen as the final score.
", "rawMarkdown": "If I submit multiple results, which one should be chosen as the final score.", "votes": null }, { "id": "312755", "postDate": "04/12/2018 09:14:04", "content": "I think if you are not sure, better skip selecting, best score will be automatically selected for you.
", "rawMarkdown": "I think if you are not sure, better skip selecting, best score will be automatically selected for you.", "votes": null }, { "id": "312757", "postDate": "04/12/2018 09:18:17", "content": "No, it selects best public score, which is not the same, especially if all public scores are 0...
\n\nIt is much safer to select yourself.
", "rawMarkdown": "No, it selects best public score, which is not the same, especially if all public scores are 0...\n\nIt is much safer to select yourself.", "votes": null }, { "id": "312762", "postDate": "04/12/2018 09:36:14", "content": "I didn't know that, thanks! Could you please elaborate - \"Your final score may not be based on the same exact subset of data as the public leaderboard, but rather a different private data subset of your full submission — your public score is only a rough indication of what your final score is.\" It seems for private LB there is some another private data set.
", "rawMarkdown": "I didn't know that, thanks! Could you please elaborate - \"Your final score may not be based on the same exact subset of data as the public leaderboard, but rather a different private data subset of your full submission — your public score is only a rough indication of what your final score is.\" It seems for private LB there is some another private data set.", "votes": null }, { "id": "312782", "postDate": "04/12/2018 10:20:15", "content": "I think it's better if someone from kaggle explained about this.
", "rawMarkdown": "I think it's better if someone from kaggle explained about this.", "votes": null }, { "id": "312785", "postDate": "04/12/2018 10:24:35", "content": "Yes, right.
", "rawMarkdown": "Yes, right.", "votes": null }, { "id": "312788", "postDate": "04/12/2018 10:40:24", "content": "Does the current leaderboard reflects actual positions?
", "rawMarkdown": "Does the current leaderboard reflects actual positions?", "votes": null }, { "id": "312794", "postDate": "04/12/2018 10:46:45", "content": "I am really confused by the instructions on \"My Submissions\" page.
", "rawMarkdown": "I am really confused by the instructions on \"My Submissions\" page.", "votes": null }, { "id": "313089", "postDate": "04/12/2018 19:07:18", "content": "I don't think so. It seems to be ordered by time of 1st submission.
", "rawMarkdown": "I don't think so. It seems to be ordered by time of 1st submission.", "votes": null }, { "id": "313095", "postDate": "04/12/2018 19:10:23", "content": "lb is behaving as fifo\n\"inversionbronze inversion 3 months ago Reply\nHi Sukjae - Yes, if scores are identical, the first one submitted gets the higher placing.\"\nbug design understanding :)
", "rawMarkdown": "lb is behaving as fifo\n\"inversionbronze inversion 3 months ago Reply\nHi Sukjae - Yes, if scores are identical, the first one submitted gets the higher placing.\"\nbug design understanding :)", "votes": null }, { "id": "313422", "postDate": "04/13/2018 08:31:52", "content": "Which actually mean LB is sorted by time of 1st submission.
", "rawMarkdown": "Which actually mean LB is sorted by time of 1st submission.", "votes": null }, { "id": "313729", "postDate": "04/13/2018 19:04:27", "content": "In order to prevent probing, all the images in stage 2 test set are either in private leaderboard or ignored. Therefore your public leaderboard scores will always be 0.
", "rawMarkdown": "In order to prevent probing, all the images in stage 2 test set are either in private leaderboard or ignored. Therefore your public leaderboard scores will always be 0.", "votes": null }, { "id": "313927", "postDate": "04/14/2018 05:03:49", "content": "This is such a brillant idea that I cannot say how smart it is. LB probing is the only thing negative I can think of in Kaggle competitions.
", "rawMarkdown": "This is such a brillant idea that I cannot say how smart it is. LB probing is the only thing negative I can think of in Kaggle competitions.", "votes": null } ], "comments": [ { "id": 312671, "author_name": "akilaw", "author_url": "", "post_date": "04/12/2018 05:53:24", "content": "I think may be they are just giving zero for all the submissions until the end of stage 2 so that we cannot probe the leader board.
", "votes": null, "replies": [ { "id": 312728, "author_name": "ildoonet", "author_url": "", "post_date": "04/12/2018 07:56:00", "content": "I think this is right.
", "votes": null, "replies": [] }, { "id": 312729, "author_name": "zmingen", "author_url": "", "post_date": "04/12/2018 08:03:09", "content": "If I submit multiple results, which one should be chosen as the final score.
", "votes": null, "replies": [] }, { "id": 312755, "author_name": "vijaybj", "author_url": "", "post_date": "04/12/2018 09:14:04", "content": "I think if you are not sure, better skip selecting, best score will be automatically selected for you.
", "votes": null, "replies": [] }, { "id": 312757, "author_name": "cpmpml", "author_url": "", "post_date": "04/12/2018 09:18:17", "content": "No, it selects best public score, which is not the same, especially if all public scores are 0...
\n\nIt is much safer to select yourself.
", "votes": null, "replies": [] }, { "id": 312762, "author_name": "vijaybj", "author_url": "", "post_date": "04/12/2018 09:36:14", "content": "I didn't know that, thanks! Could you please elaborate - \"Your final score may not be based on the same exact subset of data as the public leaderboard, but rather a different private data subset of your full submission — your public score is only a rough indication of what your final score is.\" It seems for private LB there is some another private data set.
", "votes": null, "replies": [] }, { "id": 312782, "author_name": "akilaw", "author_url": "", "post_date": "04/12/2018 10:20:15", "content": "I think it's better if someone from kaggle explained about this.
", "votes": null, "replies": [] }, { "id": 312785, "author_name": "vijaybj", "author_url": "", "post_date": "04/12/2018 10:24:35", "content": "Yes, right.
", "votes": null, "replies": [] } ] }, { "id": 312672, "author_name": "jason71995", "author_url": "", "post_date": "04/12/2018 06:03:04", "content": "\"Each row in your submission represents a single predicted nucleus segmentation for the given ImageId.\"
\n\nI GUESS it should not have any row if the image has no nucleus.
\n\nBy the way, I got 0 score, too.
", "votes": null, "replies": [] }, { "id": 312788, "author_name": "dicksonchin93", "author_url": "", "post_date": "04/12/2018 10:40:24", "content": "Does the current leaderboard reflects actual positions?
", "votes": null, "replies": [ { "id": 312794, "author_name": "vijaybj", "author_url": "", "post_date": "04/12/2018 10:46:45", "content": "I am really confused by the instructions on \"My Submissions\" page.
", "votes": null, "replies": [] }, { "id": 313089, "author_name": "antorsae", "author_url": "", "post_date": "04/12/2018 19:07:18", "content": "I don't think so. It seems to be ordered by time of 1st submission.
", "votes": null, "replies": [] } ] }, { "id": 313095, "author_name": "andre1233", "author_url": "", "post_date": "04/12/2018 19:10:23", "content": "lb is behaving as fifo\n\"inversionbronze inversion 3 months ago Reply\nHi Sukjae - Yes, if scores are identical, the first one submitted gets the higher placing.\"\nbug design understanding :)
", "votes": null, "replies": [ { "id": 313422, "author_name": "vijaybj", "author_url": "", "post_date": "04/13/2018 08:31:52", "content": "Which actually mean LB is sorted by time of 1st submission.
", "votes": null, "replies": [] } ] }, { "id": 313729, "author_name": "wendykan", "author_url": "", "post_date": "04/13/2018 19:04:27", "content": "In order to prevent probing, all the images in stage 2 test set are either in private leaderboard or ignored. Therefore your public leaderboard scores will always be 0.
", "votes": null, "replies": [ { "id": 313927, "author_name": "cpmpml", "author_url": "", "post_date": "04/14/2018 05:03:49", "content": "This is such a brillant idea that I cannot say how smart it is. LB probing is the only thing negative I can think of in Kaggle competitions.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "312647": "Hi,\n\nWhy does everyone who submitted a score for stage two have a score of zero? Is this because the scoring function hasn’t been updated yet or could this be that our models are over fitting? Also, to fix the problem of not predicting a mask for each image, my submission code returns a rle of “1 1” for each image without a predicted mask, could this also be causing the low scores on the leaderboard?\n\nThanks", "312671": "I think may be they are just giving zero for all the submissions until the end of stage 2 so that we cannot probe the leader board.", "312672": "\"Each row in your submission represents a single predicted nucleus segmentation for the given ImageId.\"\n\nI GUESS it should not have any row if the image has no nucleus.\n\nBy the way, I got 0 score, too.", "312728": "I think this is right.", "312729": "If I submit multiple results, which one should be chosen as the final score.", "312755": "I think if you are not sure, better skip selecting, best score will be automatically selected for you.", "312757": "No, it selects best public score, which is not the same, especially if all public scores are 0...\n\nIt is much safer to select yourself.", "312762": "I didn't know that, thanks! Could you please elaborate - \"Your final score may not be based on the same exact subset of data as the public leaderboard, but rather a different private data subset of your full submission — your public score is only a rough indication of what your final score is.\" It seems for private LB there is some another private data set.", "312782": "I think it's better if someone from kaggle explained about this.", "312785": "Yes, right.", "312788": "Does the current leaderboard reflects actual positions?", "312794": "I am really confused by the instructions on \"My Submissions\" page.", "313089": "I don't think so. It seems to be ordered by time of 1st submission.", "313095": "lb is behaving as fifo\n\"inversionbronze inversion 3 months ago Reply\nHi Sukjae - Yes, if scores are identical, the first one submitted gets the higher placing.\"\nbug design understanding :)", "313422": "Which actually mean LB is sorted by time of 1st submission.", "313729": "In order to prevent probing, all the images in stage 2 test set are either in private leaderboard or ignored. Therefore your public leaderboard scores will always be 0.", "313927": "This is such a brillant idea that I cannot say how smart it is. LB probing is the only thing negative I can think of in Kaggle competitions." }, "source": "meta" }