Hello there!
\n\nHow do you think should we expect a shake-up in this competition? Here is my opinion.
\n\nI think we should expect a huge shake-up just cause three following reasons:\n1) A lot of broken boxes in train, public and private datasets. I'm sure that a lot of people, in particular even those guys who are in the medal zone now, don't know this fact. And all it cost to read the necessary discussion.\n2) A lot of people using Yolov5, and again in particular in the medal zone. I'm talking exactly about their the best scores, they got it by Yolov5. A part of them will just choose another submit and apparently will drop down, another part of them will state in the final submission Yolov5 solution and will definitely drop down in the private LB.\n3) A lot of people (mostly Yolov5 users) using submit-notebooks with the overfitted parameters (like thresholds) and will use them for the final submissions. They thank the creators of these notebooks handsomely, and then will fiercely blame the organizers for the shake-up.
\n\nWhy am I saying this? As I said above, in my opinion, there will be a shake-up after all. Any shake-up causes a lot of dissatisfaction among the participants, but this is usually due to poorly compiled public and private datasets relative to each other. However, this time the matter is different, everything is quite natural. What do you think?
", "messages": [ { "id": 956287, "postDate": "2020-08-03T11:44:53.250Z", "content": "I agree, this competition have shake-up factor.
\n1. Many participants have YoloV5 public score. \n2. Private LB has new source test dataset, we didn't know whether difficult or not.\n3. hyperparameter is very sensitive(threshold, IoU etc...). we difficultly choose best hyperparameter\n4. Private LB(only) dataset is modified bbox. but Public LB is not modified
Good points and good luck for your compliant solution.
", "rawMarkdown": "Good points and good luck for your compliant solution.", "votes": 1 }, { "id": 957035, "postDate": "2020-08-04T02:35:56.783Z", "content": "Thanks! Good luck too!
", "rawMarkdown": "Thanks! Good luck too!", "votes": 1 }, { "id": 957037, "postDate": "2020-08-04T02:36:36.953Z", "content": "add One thing(No4)
", "rawMarkdown": "add One thing(No4)", "votes": 1 }, { "id": 957305, "postDate": "2020-08-04T07:58:18.660Z", "content": "It's easy to choose the best hyperparameters only for public leaderboard but they might ruin a lot of good scores at the private leaderboard.
", "rawMarkdown": "It's easy to choose the best hyperparameters only for public leaderboard but they might ruin a lot of good scores at the private leaderboard.", "votes": 1 }, { "id": 957393, "postDate": "2020-08-04T09:11:10.287Z", "content": "I will submit a low score and a high score,hhhhhhhh.
", "rawMarkdown": "I will submit a low score and a high score,hhhhhhhh." }, { "id": 957579, "postDate": "2020-08-04T12:22:20.467Z", "rawMarkdown": "", "isDeleted": true }, { "id": 957681, "postDate": "2020-08-04T13:42:25.930Z", "content": "If some one selects one yolo and one non yolo.. will that person still be disqualified ?\nhow do we know kaggle has excluded Yolo submission and disqualified participants
", "rawMarkdown": "If some one selects one yolo and one non yolo.. will that person still be disqualified ?\nhow do we know kaggle has excluded Yolo submission and disqualified participants", "votes": 2 }, { "id": 957696, "postDate": "2020-08-04T13:51:15.763Z", "content": "well, you will know after you're ejected out of the leaderboard.. ;)
", "rawMarkdown": "well, you will know after you're ejected out of the leaderboard.. ;)", "votes": 1 }, { "id": 957706, "postDate": "2020-08-04T13:58:16.837Z", "content": "Even if their Yolov5 submissions are detected, those people won't be disqualified. In the worst case, they will just lose these submissions and their second submissions will be used for the final score (if these are not Yolov5 solutions). In the best case for them, these \"cheating\" solutions won't be even detected and will be used for the final score on the private leaderboard. Likely, hosts will check manually only gold medal owners' submissions.
", "rawMarkdown": "Even if their Yolov5 submissions are detected, those people won't be disqualified. In the worst case, they will just lose these submissions and their second submissions will be used for the final score (if these are not Yolov5 solutions). In the best case for them, these \"cheating\" solutions won't be even detected and will be used for the final score on the private leaderboard. Likely, hosts will check manually only gold medal owners' submissions.", "votes": 1 }, { "id": 957710, "postDate": "2020-08-04T14:02:24.913Z", "content": "is it necessary to clean output folder so to just have .csv file or we are good as long as long as score comes up even if not cleaned ?
", "rawMarkdown": "is it necessary to clean output folder so to just have .csv file or we are good as long as long as score comes up even if not cleaned ?" }, { "id": 957857, "postDate": "2020-08-04T15:43:39.330Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 956059, "postDate": "2020-08-03T07:49:47.073Z", "content": "Lots of people are using the same public kernels, so I expect >100 shakeups and downs.\nJust a few parameter tweaks and even random seeds will cause big shakes in such situations.
", "rawMarkdown": "Lots of people are using the same public kernels, so I expect >100 shakeups and downs.\nJust a few parameter tweaks and even random seeds will cause big shakes in such situations.", "votes": 6, "replies": [ { "id": 956655, "postDate": "2020-08-03T17:08:01.110Z", "content": "pub lb set size is 62% which is quite high.. what can remaining 38% can do much to score if u are using compliant model
", "rawMarkdown": " pub lb set size is 62% which is quite high.. what can remaining 38% can do much to score if u are using compliant model ", "votes": -5 }, { "id": 957046, "postDate": "2020-08-04T02:45:29.913Z", "content": "I think largest data science challenge is how we prepare for the unknown hidden private set.\nLet's see how people dealed with it. ;)
", "rawMarkdown": "I think largest data science challenge is how we prepare for the unknown hidden private set.\nLet's see how people dealed with it. ;)", "votes": 2 }, { "id": 957337, "postDate": "2020-08-04T08:22:57.777Z", "content": "@kyoshioka47 is 38% test set is contained in the test set or that is separate entirely ?
", "rawMarkdown": "@kyoshioka47 is 38% test set is contained in the test set or that is separate entirely ?" } ] }, { "id": 956027, "postDate": "2020-08-03T07:04:33.403Z", "content": "Hello there!
\n\nHow do you think should we expect a shake-up in this competition? Here is my opinion.
\n\nI think we should expect a huge shake-up just cause three following reasons:\n1) A lot of broken boxes in train, public and private datasets. I'm sure that a lot of people, in particular even those guys who are in the medal zone now, don't know this fact. And all it cost to read the necessary discussion.\n2) A lot of people using Yolov5, and again in particular in the medal zone. I'm talking exactly about their the best scores, they got it by Yolov5. A part of them will just choose another submit and apparently will drop down, another part of them will state in the final submission Yolov5 solution and will definitely drop down in the private LB.\n3) A lot of people (mostly Yolov5 users) using submit-notebooks with the overfitted parameters (like thresholds) and will use them for the final submissions. They thank the creators of these notebooks handsomely, and then will fiercely blame the organizers for the shake-up.
\n\nWhy am I saying this? As I said above, in my opinion, there will be a shake-up after all. Any shake-up causes a lot of dissatisfaction among the participants, but this is usually due to poorly compiled public and private datasets relative to each other. However, this time the matter is different, everything is quite natural. What do you think?
", "rawMarkdown": "Hello there!\n\nHow do you think should we expect a shake-up in this competition? Here is my opinion.\n\nI think we should expect a huge shake-up just cause three following reasons:\n1) A lot of broken boxes in train, public and private datasets. I'm sure that a lot of people, in particular even those guys who are in the medal zone now, don't know this fact. And all it cost to read the necessary discussion.\n2) A lot of people using Yolov5, and again in particular in the medal zone. I'm talking exactly about their the best scores, they got it by Yolov5. A part of them will just choose another submit and apparently will drop down, another part of them will state in the final submission Yolov5 solution and will definitely drop down in the private LB.\n3) A lot of people (mostly Yolov5 users) using submit-notebooks with the overfitted parameters (like thresholds) and will use them for the final submissions. They thank the creators of these notebooks handsomely, and then will fiercely blame the organizers for the shake-up.\n\nWhy am I saying this? As I said above, in my opinion, there will be a shake-up after all. Any shake-up causes a lot of dissatisfaction among the participants, but this is usually due to poorly compiled public and private datasets relative to each other. However, this time the matter is different, everything is quite natural. What do you think?\n\n ", "votes": 5 }, { "id": 957489, "postDate": "2020-08-04T11:01:18.597Z", "content": "From results of my local experiments, I am expecting a 0.005-0.01 shake without the YoloV5 factor.
", "rawMarkdown": "From results of my local experiments, I am expecting a 0.005-0.01 shake without the YoloV5 factor.", "votes": 3, "replies": [ { "id": 957582, "postDate": "2020-08-04T12:23:57.067Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 956889, "postDate": "2020-08-03T21:52:48.723Z", "content": "Training labels are not clean and private set labels are adjusted, so yes.
", "rawMarkdown": "Training labels are not clean and private set labels are adjusted, so yes.", "replies": [ { "id": 956942, "postDate": "2020-08-04T00:42:27.123Z", "content": "Hello!The boxes in test is correct , is it right?
", "rawMarkdown": "Hello!The boxes in test is correct , is it right?" }, { "id": 957041, "postDate": "2020-08-04T02:40:23.470Z", "content": "In the private set, yes
", "rawMarkdown": "In the private set, yes" }, { "id": 957044, "postDate": "2020-08-04T02:43:29.877Z", "content": "Here is information.
\nhttps://www.kaggle.com/c/global-wheat-detection/discussion/163291#911147
I think correct might be a bit too optimistic given how error prone all the other datasets are. My initial assumption was the labels would be great given that there was such an extensive labeling process but when I actually looked at them in detail I discovered otherwise.
", "rawMarkdown": "I think correct might be a bit too optimistic given how error prone all the other datasets are. My initial assumption was the labels would be great given that there was such an extensive labeling process but when I actually looked at them in detail I discovered otherwise. " }, { "id": 958309, "postDate": "2020-08-04T23:50:18.933Z", "content": "Very less scope of shake up ,other than y5 ,every thing will stay same. So selecting top non y5 will be good
", "rawMarkdown": "Very less scope of shake up ,other than y5 ,every thing will stay same. So selecting top non y5 will be good" } ] }, { "id": 958722, "postDate": "2020-08-05T05:54:55.810Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 956287, "author_name": "tereka", "author_url": "", "post_date": "2020-08-03T11:44:53.250000", "content": "I agree, this competition have shake-up factor.
\n1. Many participants have YoloV5 public score. \n2. Private LB has new source test dataset, we didn't know whether difficult or not.\n3. hyperparameter is very sensitive(threshold, IoU etc...). we difficultly choose best hyperparameter\n4. Private LB(only) dataset is modified bbox. but Public LB is not modified
Good points and good luck for your compliant solution.
", "votes": 1, "replies": [] }, { "id": 957035, "author_name": "tereka", "author_url": "", "post_date": "2020-08-04T02:35:56.783000", "content": "Thanks! Good luck too!
", "votes": 1, "replies": [] }, { "id": 957037, "author_name": "tereka", "author_url": "", "post_date": "2020-08-04T02:36:36.953000", "content": "add One thing(No4)
", "votes": 1, "replies": [] }, { "id": 957305, "author_name": "Vadim Timakin", "author_url": "", "post_date": "2020-08-04T07:58:18.660000", "content": "It's easy to choose the best hyperparameters only for public leaderboard but they might ruin a lot of good scores at the private leaderboard.
", "votes": 1, "replies": [] }, { "id": 957393, "author_name": "Zekun", "author_url": "", "post_date": "2020-08-04T09:11:10.287000", "content": "I will submit a low score and a high score,hhhhhhhh.
", "votes": 0, "replies": [] }, { "id": 957579, "author_name": "", "author_url": "", "post_date": "2020-08-04T12:22:20.467000", "content": "", "votes": 0, "replies": [] }, { "id": 957681, "author_name": "Jaideep", "author_url": "", "post_date": "2020-08-04T13:42:25.930000", "content": "If some one selects one yolo and one non yolo.. will that person still be disqualified ?\nhow do we know kaggle has excluded Yolo submission and disqualified participants
", "votes": 2, "replies": [] }, { "id": 957696, "author_name": "arutema47", "author_url": "", "post_date": "2020-08-04T13:51:15.763000", "content": "well, you will know after you're ejected out of the leaderboard.. ;)
", "votes": 1, "replies": [] }, { "id": 957706, "author_name": "Vadim Timakin", "author_url": "", "post_date": "2020-08-04T13:58:16.837000", "content": "Even if their Yolov5 submissions are detected, those people won't be disqualified. In the worst case, they will just lose these submissions and their second submissions will be used for the final score (if these are not Yolov5 solutions). In the best case for them, these \"cheating\" solutions won't be even detected and will be used for the final score on the private leaderboard. Likely, hosts will check manually only gold medal owners' submissions.
", "votes": 1, "replies": [] }, { "id": 957710, "author_name": "Jaideep", "author_url": "", "post_date": "2020-08-04T14:02:24.913000", "content": "is it necessary to clean output folder so to just have .csv file or we are good as long as long as score comes up even if not cleaned ?
", "votes": 0, "replies": [] }, { "id": 957857, "author_name": "", "author_url": "", "post_date": "2020-08-04T15:43:39.330000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 956059, "author_name": "arutema47", "author_url": "", "post_date": "2020-08-03T07:49:47.073000", "content": "Lots of people are using the same public kernels, so I expect >100 shakeups and downs.\nJust a few parameter tweaks and even random seeds will cause big shakes in such situations.
", "votes": 6, "replies": [ { "id": 956655, "author_name": "Jaideep", "author_url": "", "post_date": "2020-08-03T17:08:01.110000", "content": "pub lb set size is 62% which is quite high.. what can remaining 38% can do much to score if u are using compliant model
", "votes": -5, "replies": [] }, { "id": 957046, "author_name": "arutema47", "author_url": "", "post_date": "2020-08-04T02:45:29.913000", "content": "I think largest data science challenge is how we prepare for the unknown hidden private set.\nLet's see how people dealed with it. ;)
", "votes": 2, "replies": [] }, { "id": 957337, "author_name": "Jaideep", "author_url": "", "post_date": "2020-08-04T08:22:57.777000", "content": "@kyoshioka47 is 38% test set is contained in the test set or that is separate entirely ?
", "votes": 0, "replies": [] } ] }, { "id": 957489, "author_name": "Peiyuan Liao", "author_url": "", "post_date": "2020-08-04T11:01:18.597000", "content": "From results of my local experiments, I am expecting a 0.005-0.01 shake without the YoloV5 factor.
", "votes": 3, "replies": [ { "id": 957582, "author_name": "", "author_url": "", "post_date": "2020-08-04T12:23:57.067000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 956889, "author_name": "sin", "author_url": "", "post_date": "2020-08-03T21:52:48.723000", "content": "Training labels are not clean and private set labels are adjusted, so yes.
", "votes": 0, "replies": [ { "id": 956942, "author_name": "Zekun", "author_url": "", "post_date": "2020-08-04T00:42:27.123000", "content": "Hello!The boxes in test is correct , is it right?
", "votes": 0, "replies": [] }, { "id": 957041, "author_name": "sin", "author_url": "", "post_date": "2020-08-04T02:40:23.470000", "content": "In the private set, yes
", "votes": 0, "replies": [] }, { "id": 957044, "author_name": "tereka", "author_url": "", "post_date": "2020-08-04T02:43:29.877000", "content": "Here is information.
\nhttps://www.kaggle.com/c/global-wheat-detection/discussion/163291#911147
I think correct might be a bit too optimistic given how error prone all the other datasets are. My initial assumption was the labels would be great given that there was such an extensive labeling process but when I actually looked at them in detail I discovered otherwise.
", "votes": 0, "replies": [] }, { "id": 958309, "author_name": "Jaideep", "author_url": "", "post_date": "2020-08-04T23:50:18.933000", "content": "Very less scope of shake up ,other than y5 ,every thing will stay same. So selecting top non y5 will be good
", "votes": 0, "replies": [] } ] }, { "id": 958722, "author_name": "", "author_url": "", "post_date": "2020-08-05T05:54:55.810000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "956287": "I agree, this competition have shake-up factor. \n1. Many participants have YoloV5 public score. \n2. Private LB has new source test dataset, we didn't know whether difficult or not.\n3. hyperparameter is very sensitive(threshold, IoU etc...). we difficultly choose best hyperparameter\n4. Private LB(only) dataset is modified bbox. but Public LB is not modified", "956059": "Lots of people are using the same public kernels, so I expect >100 shakeups and downs.\nJust a few parameter tweaks and even random seeds will cause big shakes in such situations.", "956027": "Hello there!\n\nHow do you think should we expect a shake-up in this competition? Here is my opinion.\n\nI think we should expect a huge shake-up just cause three following reasons:\n1) A lot of broken boxes in train, public and private datasets. I'm sure that a lot of people, in particular even those guys who are in the medal zone now, don't know this fact. And all it cost to read the necessary discussion.\n2) A lot of people using Yolov5, and again in particular in the medal zone. I'm talking exactly about their the best scores, they got it by Yolov5. A part of them will just choose another submit and apparently will drop down, another part of them will state in the final submission Yolov5 solution and will definitely drop down in the private LB.\n3) A lot of people (mostly Yolov5 users) using submit-notebooks with the overfitted parameters (like thresholds) and will use them for the final submissions. They thank the creators of these notebooks handsomely, and then will fiercely blame the organizers for the shake-up.\n\nWhy am I saying this? As I said above, in my opinion, there will be a shake-up after all. Any shake-up causes a lot of dissatisfaction among the participants, but this is usually due to poorly compiled public and private datasets relative to each other. However, this time the matter is different, everything is quite natural. What do you think?\n\n ", "957489": "From results of my local experiments, I am expecting a 0.005-0.01 shake without the YoloV5 factor.", "956889": "Training labels are not clean and private set labels are adjusted, so yes.", "958722": "" } }