This is what we learnt from shaking down from 10th public to 80th private!
\nIn our validation, and on the public leaderboard we found that with our faster rcnn model we got a boost in the score by expanding box sizes one pixel in all directions. Without this trick our frcnn score would have placed 17th public and 27th private. However, using late submissions we found that if we shrink the box size by 1 pixel in all directions we would see a lower public score at 89th but the private score would have just got into the gold zone.
\nSome analysis one of my teammates did during the comp suggested that the box sizes may be smaller in the private dataset. The post-comp insight we found from this is that rather than doing a universal box size adjustment it may have been better to make box size adjustments based on the initial box size. We believe it would have been effective to make the small boxes smaller and the large boxes larger. If there is enough interest in this topic I can make a kernel to explore this further.
", "messages": [ { "id": "982260", "postDate": "08/23/2020 07:48:48", "content": "This is what we learnt from shaking down from 10th public to 80th private!
\nIn our validation, and on the public leaderboard we found that with our faster rcnn model we got a boost in the score by expanding box sizes one pixel in all directions. Without this trick our frcnn score would have placed 17th public and 27th private. However, using late submissions we found that if we shrink the box size by 1 pixel in all directions we would see a lower public score at 89th but the private score would have just got into the gold zone.
\nSome analysis one of my teammates did during the comp suggested that the box sizes may be smaller in the private dataset. The post-comp insight we found from this is that rather than doing a universal box size adjustment it may have been better to make box size adjustments based on the initial box size. We believe it would have been effective to make the small boxes smaller and the large boxes larger. If there is enough interest in this topic I can make a kernel to explore this further.
", "rawMarkdown": "This is what we learnt from shaking down from 10th public to 80th private!\n\nIn our validation, and on the public leaderboard we found that with our faster rcnn model we got a boost in the score by expanding box sizes one pixel in all directions. Without this trick our frcnn score would have placed 17th public and 27th private. However, using late submissions we found that if we shrink the box size by 1 pixel in all directions we would see a lower public score at 89th but the private score would have just got into the gold zone.\n\nSome analysis one of my teammates did during the comp suggested that the box sizes may be smaller in the private dataset. The post-comp insight we found from this is that rather than doing a universal box size adjustment it may have been better to make box size adjustments based on the initial box size. We believe it would have been effective to make the small boxes smaller and the large boxes larger. If there is enough interest in this topic I can make a kernel to explore this further.", "votes": null }, { "id": "982900", "postDate": "08/23/2020 19:30:00", "content": "Nice findings. I also thought about expanding/contracting the predictions but thought it may lead to overfitting on the LB.
", "rawMarkdown": "Nice findings. I also thought about expanding/contracting the predictions but thought it may lead to overfitting on the LB.", "votes": null }, { "id": "983298", "postDate": "08/24/2020 07:02:19", "content": "We knew there was a risk to this approach but since we saw an improvement in our validation and on the leaderboard with this box size adjustment then it was reasonable to try it. This thread especially this comment discusses how others worked on box size adjustment with more success.
\nThe shakedown was disappointing but I don't regret trying this, at least we learnt something through the process.
", "rawMarkdown": "We knew there was a risk to this approach but since we saw an improvement in our validation and on the leaderboard with this box size adjustment then it was reasonable to try it. This [thread](https://www.kaggle.com/c/global-wheat-detection/discussion/176109) especially this [comment](https://www.kaggle.com/c/global-wheat-detection/discussion/176109#979137) discusses how others worked on box size adjustment with more success.\n\nThe shakedown was disappointing but I don't regret trying this, at least we learnt something through the process.", "votes": null } ], "comments": [ { "id": 982900, "author_name": "matthewmasters", "author_url": "", "post_date": "08/23/2020 19:30:00", "content": "Nice findings. I also thought about expanding/contracting the predictions but thought it may lead to overfitting on the LB.
", "votes": null, "replies": [ { "id": 983298, "author_name": "datahobbit", "author_url": "", "post_date": "08/24/2020 07:02:19", "content": "We knew there was a risk to this approach but since we saw an improvement in our validation and on the leaderboard with this box size adjustment then it was reasonable to try it. This thread especially this comment discusses how others worked on box size adjustment with more success.
\nThe shakedown was disappointing but I don't regret trying this, at least we learnt something through the process.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "982260": "This is what we learnt from shaking down from 10th public to 80th private!\n\nIn our validation, and on the public leaderboard we found that with our faster rcnn model we got a boost in the score by expanding box sizes one pixel in all directions. Without this trick our frcnn score would have placed 17th public and 27th private. However, using late submissions we found that if we shrink the box size by 1 pixel in all directions we would see a lower public score at 89th but the private score would have just got into the gold zone.\n\nSome analysis one of my teammates did during the comp suggested that the box sizes may be smaller in the private dataset. The post-comp insight we found from this is that rather than doing a universal box size adjustment it may have been better to make box size adjustments based on the initial box size. We believe it would have been effective to make the small boxes smaller and the large boxes larger. If there is enough interest in this topic I can make a kernel to explore this further.", "982900": "Nice findings. I also thought about expanding/contracting the predictions but thought it may lead to overfitting on the LB.", "983298": "We knew there was a risk to this approach but since we saw an improvement in our validation and on the leaderboard with this box size adjustment then it was reasonable to try it. This [thread](https://www.kaggle.com/c/global-wheat-detection/discussion/176109) especially this [comment](https://www.kaggle.com/c/global-wheat-detection/discussion/176109#979137) discusses how others worked on box size adjustment with more success.\n\nThe shakedown was disappointing but I don't regret trying this, at least we learnt something through the process." }, "source": "meta" }