{
  "id": 172458,
  "title": "5th Place Solution (EfficientDet)",
  "url": "/competitions/global-wheat-detection/discussion/172458",
  "author_name": "tereka",
  "post_date": "2020-08-05T06:04:51.187000",
  "votes": 50,
  "comment_count": 12,
  "views": 0,
  "content": "<p>Thank you for host and competitors.<br>\nIt is interesting competition for me.</p>\n<h1>TL;DR</h1>\n<ol>\n<li>EfficientDetB3 5Fold -&gt; 10 image Test with Pseudo Labeling</li>\n<li>EfficientDetB5 All Training(EMA) -&gt; Prediction MultiScaleTraning(1280,768,1024+4Flip) 0.766</li>\n<li>EfficientDetB5 All + EfficientDetB4 All + EfficientDetB5 All(With Pseudo) Public 0.7752</li>\n</ol>\n<h1>Solution</h1>\n<h2>1st Phase</h2>\n<p>EfficientDetB3 for test 10 images pseudo labeling<br>\nTest image is \"test source\", I want to get test source knowledge</p>\n<h2>2nd Phase</h2>\n<p>I used EMA for the ensemble technique in 2nd phase.</p>\n<ol>\n<li>AdamW 100epoch 640 batchsize 4(for bn parameter turning)</li>\n<li>AdamW 40epoch 1024 batchsize 1(bn layer freeze)</li>\n</ol>\n<p>use Cosine Annealing</p>\n<p>Augmentation</p>\n<ul>\n<li>Mixup</li>\n<li>Mosaic</li>\n<li>Scale</li>\n<li>Hue</li>\n<li>Random Brightness</li>\n<li>Cutout</li>\n<li>GridMask</li>\n</ul>\n<h2>3rd Phase(Kernel)</h2>\n<h3>Prediction</h3>\n<p>EfficientDetB5 All + Predict MultiScale(768, 1024, 1280 * 4Flip) -&gt; Pseudo Labeling of All test. </p>\n<h3>Pseudo Labeling</h3>\n<p>Traning Parameter as follows</p>\n<ul>\n<li>EfficientDet B5 image size 1024  </li>\n<li>Epoch 5</li>\n<li>Use EMA</li>\n<li>Mixed Precision(AMP)</li>\n</ul>\n<h3>Ensemble</h3>\n<p>EfficientDetB5 All + EfficientDetB4 All + EfficientDetB5 All(With Pseudo) using WBF</p>\n<h1>Why Shakedown?</h1>\n<ol>\n<li>failed Threshold turning.(lower threshold is better in private)  <br>\nWhen setting threshold 0.1, the score go to 0.695. but I can not get selection.</li>\n<li>Use more heavy augmentation for robustness  </li>\n</ol>",
  "messages": [
    {
      "id": 958739,
      "postDate": "2020-08-05T06:04:51.187Z",
      "content": "<p>Thank you for host and competitors.<br>\nIt is interesting competition for me.</p>\n<h1>TL;DR</h1>\n<ol>\n<li>EfficientDetB3 5Fold -&gt; 10 image Test with Pseudo Labeling</li>\n<li>EfficientDetB5 All Training(EMA) -&gt; Prediction MultiScaleTraning(1280,768,1024+4Flip) 0.766</li>\n<li>EfficientDetB5 All + EfficientDetB4 All + EfficientDetB5 All(With Pseudo) Public 0.7752</li>\n</ol>\n<h1>Solution</h1>\n<h2>1st Phase</h2>\n<p>EfficientDetB3 for test 10 images pseudo labeling<br>\nTest image is \"test source\", I want to get test source knowledge</p>\n<h2>2nd Phase</h2>\n<p>I used EMA for the ensemble technique in 2nd phase.</p>\n<ol>\n<li>AdamW 100epoch 640 batchsize 4(for bn parameter turning)</li>\n<li>AdamW 40epoch 1024 batchsize 1(bn layer freeze)</li>\n</ol>\n<p>use Cosine Annealing</p>\n<p>Augmentation</p>\n<ul>\n<li>Mixup</li>\n<li>Mosaic</li>\n<li>Scale</li>\n<li>Hue</li>\n<li>Random Brightness</li>\n<li>Cutout</li>\n<li>GridMask</li>\n</ul>\n<h2>3rd Phase(Kernel)</h2>\n<h3>Prediction</h3>\n<p>EfficientDetB5 All + Predict MultiScale(768, 1024, 1280 * 4Flip) -&gt; Pseudo Labeling of All test. </p>\n<h3>Pseudo Labeling</h3>\n<p>Traning Parameter as follows</p>\n<ul>\n<li>EfficientDet B5 image size 1024  </li>\n<li>Epoch 5</li>\n<li>Use EMA</li>\n<li>Mixed Precision(AMP)</li>\n</ul>\n<h3>Ensemble</h3>\n<p>EfficientDetB5 All + EfficientDetB4 All + EfficientDetB5 All(With Pseudo) using WBF</p>\n<h1>Why Shakedown?</h1>\n<ol>\n<li>failed Threshold turning.(lower threshold is better in private)  <br>\nWhen setting threshold 0.1, the score go to 0.695. but I can not get selection.</li>\n<li>Use more heavy augmentation for robustness  </li>\n</ol>",
      "rawMarkdown": "Thank you for host and competitors.\nIt is interesting competition for me.\n\n# TL;DR\n1. EfficientDetB3 5Fold -&gt; 10 image Test with Pseudo Labeling\n2. EfficientDetB5 All Training(EMA) -&gt; Prediction MultiScaleTraning(1280,768,1024+4Flip) 0.766\n3. EfficientDetB5 All + EfficientDetB4 All + EfficientDetB5 All(With Pseudo) Public 0.7752\n\n# Solution\n## 1st Phase\nEfficientDetB3 for test 10 images pseudo labeling\nTest image is \"test source\", I want to get test source knowledge\n\n## 2nd Phase\nI used EMA for the ensemble technique in 2nd phase.\n\n1. AdamW 100epoch 640 batchsize 4(for bn parameter turning)\n2. AdamW 40epoch 1024 batchsize 1(bn layer freeze)\n\nuse Cosine Annealing\n\nAugmentation\n- Mixup\n- Mosaic\n- Scale\n- Hue\n- Random Brightness\n- Cutout\n- GridMask\n\n## 3rd Phase(Kernel)\n### Prediction\nEfficientDetB5 All + Predict MultiScale(768, 1024, 1280 * 4Flip) -&gt; Pseudo Labeling of All test. \n\n### Pseudo Labeling\nTraning Parameter as follows\n- EfficientDet B5 image size 1024  \n- Epoch 5\n- Use EMA\n- Mixed Precision(AMP)\n\n### Ensemble\nEfficientDetB5 All + EfficientDetB4 All + EfficientDetB5 All(With Pseudo) using WBF\n\n# Why Shakedown?\n1. failed Threshold turning.(lower threshold is better in private)  \n   When setting threshold 0.1, the score go to 0.695. but I can not get selection.\n2. Use more heavy augmentation for robustness  \n",
      "votes": 50
    },
    {
      "id": 958743,
      "postDate": "2020-08-05T06:07:20.717Z",
      "content": "<p>I've been waiting for your solution! After such a dominant public LB, I was quite surprised at your private LB. It all makes a ton of sense now, and it's a bit shocking how similar your architecture is to NVNN. Thanks so much for sharing!</p>",
      "rawMarkdown": "I've been waiting for your solution! After such a dominant public LB, I was quite surprised at your private LB. It all makes a ton of sense now, and it's a bit shocking how similar your architecture is to NVNN. Thanks so much for sharing!",
      "votes": 1,
      "replies": [
        {
          "id": 958759,
          "postDate": "2020-08-05T06:13:48.643Z",
          "content": "<p>Thanks, my early plan have pseudo labeling experiment. <br>\nbecause very long kernel time and get domain gap. <br>\nbut 0.7700 is my best public score w/o pseudo labeling </p>",
          "rawMarkdown": "Thanks, my early plan have pseudo labeling experiment.    \nbecause very long kernel time and get domain gap.  \nbut 0.7700 is my best public score w/o pseudo labeling ",
          "votes": 1
        }
      ]
    },
    {
      "id": 959379,
      "postDate": "2020-08-05T14:42:04.400Z",
      "content": "<p>Thanks for sharing!</p>\n\n<p>\"failed Threshold turning.(lower threshold is better in private)\"\nDo you mean the iou threshold?</p>",
      "rawMarkdown": "Thanks for sharing!\n\n\"failed Threshold turning.(lower threshold is better in private)\"\nDo you mean the iou threshold?",
      "replies": [
        {
          "id": 959385,
          "postDate": "2020-08-05T14:45:15.790Z",
          "content": "<p>No. It is detection threshold.</p>",
          "rawMarkdown": "No. It is detection threshold.",
          "votes": 1
        }
      ]
    },
    {
      "id": 958846,
      "postDate": "2020-08-05T07:10:31.687Z",
      "content": "<p>I'm looking forward to your solution!\nMay I ask that do you plan to share your kernel later?</p>\n\n<p><a href=\"/tereka\">@tereka</a> </p>",
      "rawMarkdown": "I'm looking forward to your solution!\nMay I ask that do you plan to share your kernel later?\n\n@tereka ",
      "replies": [
        {
          "id": 959389,
          "postDate": "2020-08-05T14:46:18.900Z",
          "content": "<p>I have plan that will publish</p>",
          "rawMarkdown": "I have plan that will publish",
          "votes": 1
        }
      ]
    },
    {
      "id": 958827,
      "postDate": "2020-08-05T06:49:16.677Z",
      "content": "<p>Thanks for sharing the writeup!</p>\n\n<blockquote>\n  <p>EfficientDetB5 All + EfficientDetB4 All + EfficientDetB5 All(With Pseudo) Public 0.7752</p>\n</blockquote>\n\n<p>Did ensembling models only with Psuedo get worse results?</p>",
      "rawMarkdown": "Thanks for sharing the writeup!\n&gt;EfficientDetB5 All + EfficientDetB4 All + EfficientDetB5 All(With Pseudo) Public 0.7752\n\nDid ensembling models only with Psuedo get worse results?",
      "replies": [
        {
          "id": 959108,
          "postDate": "2020-08-05T10:54:59.710Z",
          "content": "<p>a little. \nEfficientDetB5 All(With Pseudo) only Public 0.7747(+- 0.0001)</p>",
          "rawMarkdown": "a little. \nEfficientDetB5 All(With Pseudo) only Public 0.7747(+- 0.0001)",
          "votes": 1
        }
      ]
    },
    {
      "id": 958751,
      "postDate": "2020-08-05T06:10:45Z",
      "rawMarkdown": "",
      "isDeleted": true,
      "replies": [
        {
          "id": 958764,
          "postDate": "2020-08-05T06:14:54.210Z",
          "content": "<p>Thanks! maybe the test will be public. I prepare training notebook(many junk codes...)</p>",
          "rawMarkdown": "Thanks! maybe the test will be public. I prepare training notebook(many junk codes...)",
          "votes": 2
        }
      ]
    },
    {
      "id": 958792,
      "postDate": "2020-08-05T06:27:16.083Z",
      "content": "<p>Excellent work! Thanks for sharing.</p>",
      "rawMarkdown": "Excellent work! Thanks for sharing.",
      "votes": 1
    },
    {
      "id": 959648,
      "postDate": "2020-08-05T19:21:00.683Z",
      "content": "<p>Thank you for sharing</p>",
      "rawMarkdown": "Thank you for sharing"
    }
  ],
  "comments": [
    {
      "id": 958743,
      "author_name": "Stanley Zheng",
      "author_url": "",
      "post_date": "2020-08-05T06:07:20.717000",
      "content": "<p>I've been waiting for your solution! After such a dominant public LB, I was quite surprised at your private LB. It all makes a ton of sense now, and it's a bit shocking how similar your architecture is to NVNN. Thanks so much for sharing!</p>",
      "votes": 1,
      "replies": [
        {
          "id": 958759,
          "author_name": "tereka",
          "author_url": "",
          "post_date": "2020-08-05T06:13:48.643000",
          "content": "<p>Thanks, my early plan have pseudo labeling experiment. <br>\nbecause very long kernel time and get domain gap. <br>\nbut 0.7700 is my best public score w/o pseudo labeling </p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 959379,
      "author_name": "mknzfr",
      "author_url": "",
      "post_date": "2020-08-05T14:42:04.400000",
      "content": "<p>Thanks for sharing!</p>\n\n<p>\"failed Threshold turning.(lower threshold is better in private)\"\nDo you mean the iou threshold?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 959385,
          "author_name": "tereka",
          "author_url": "",
          "post_date": "2020-08-05T14:45:15.790000",
          "content": "<p>No. It is detection threshold.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 958846,
      "author_name": "Heroseo",
      "author_url": "",
      "post_date": "2020-08-05T07:10:31.687000",
      "content": "<p>I'm looking forward to your solution!\nMay I ask that do you plan to share your kernel later?</p>\n\n<p><a href=\"/tereka\">@tereka</a> </p>",
      "votes": 0,
      "replies": [
        {
          "id": 959389,
          "author_name": "tereka",
          "author_url": "",
          "post_date": "2020-08-05T14:46:18.900000",
          "content": "<p>I have plan that will publish</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 958827,
      "author_name": "arutema47",
      "author_url": "",
      "post_date": "2020-08-05T06:49:16.677000",
      "content": "<p>Thanks for sharing the writeup!</p>\n\n<blockquote>\n  <p>EfficientDetB5 All + EfficientDetB4 All + EfficientDetB5 All(With Pseudo) Public 0.7752</p>\n</blockquote>\n\n<p>Did ensembling models only with Psuedo get worse results?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 959108,
          "author_name": "tereka",
          "author_url": "",
          "post_date": "2020-08-05T10:54:59.710000",
          "content": "<p>a little. \nEfficientDetB5 All(With Pseudo) only Public 0.7747(+- 0.0001)</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 958751,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-08-05T06:10:45",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 958764,
          "author_name": "tereka",
          "author_url": "",
          "post_date": "2020-08-05T06:14:54.210000",
          "content": "<p>Thanks! maybe the test will be public. I prepare training notebook(many junk codes...)</p>",
          "votes": 2,
          "replies": []
        }
      ]
    },
    {
      "id": 958792,
      "author_name": "alan chen",
      "author_url": "",
      "post_date": "2020-08-05T06:27:16.083000",
      "content": "<p>Excellent work! Thanks for sharing.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 959648,
      "author_name": "zijinwangzhe",
      "author_url": "",
      "post_date": "2020-08-05T19:21:00.683000",
      "content": "<p>Thank you for sharing</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "958739": "Thank you for host and competitors.\nIt is interesting competition for me.\n\n# TL;DR\n1. EfficientDetB3 5Fold -&gt; 10 image Test with Pseudo Labeling\n2. EfficientDetB5 All Training(EMA) -&gt; Prediction MultiScaleTraning(1280,768,1024+4Flip) 0.766\n3. EfficientDetB5 All + EfficientDetB4 All + EfficientDetB5 All(With Pseudo) Public 0.7752\n\n# Solution\n## 1st Phase\nEfficientDetB3 for test 10 images pseudo labeling\nTest image is \"test source\", I want to get test source knowledge\n\n## 2nd Phase\nI used EMA for the ensemble technique in 2nd phase.\n\n1. AdamW 100epoch 640 batchsize 4(for bn parameter turning)\n2. AdamW 40epoch 1024 batchsize 1(bn layer freeze)\n\nuse Cosine Annealing\n\nAugmentation\n- Mixup\n- Mosaic\n- Scale\n- Hue\n- Random Brightness\n- Cutout\n- GridMask\n\n## 3rd Phase(Kernel)\n### Prediction\nEfficientDetB5 All + Predict MultiScale(768, 1024, 1280 * 4Flip) -&gt; Pseudo Labeling of All test. \n\n### Pseudo Labeling\nTraning Parameter as follows\n- EfficientDet B5 image size 1024  \n- Epoch 5\n- Use EMA\n- Mixed Precision(AMP)\n\n### Ensemble\nEfficientDetB5 All + EfficientDetB4 All + EfficientDetB5 All(With Pseudo) using WBF\n\n# Why Shakedown?\n1. failed Threshold turning.(lower threshold is better in private)  \n   When setting threshold 0.1, the score go to 0.695. but I can not get selection.\n2. Use more heavy augmentation for robustness  \n",
    "958743": "I've been waiting for your solution! After such a dominant public LB, I was quite surprised at your private LB. It all makes a ton of sense now, and it's a bit shocking how similar your architecture is to NVNN. Thanks so much for sharing!",
    "959379": "Thanks for sharing!\n\n\"failed Threshold turning.(lower threshold is better in private)\"\nDo you mean the iou threshold?",
    "958846": "I'm looking forward to your solution!\nMay I ask that do you plan to share your kernel later?\n\n@tereka ",
    "958827": "Thanks for sharing the writeup!\n&gt;EfficientDetB5 All + EfficientDetB4 All + EfficientDetB5 All(With Pseudo) Public 0.7752\n\nDid ensembling models only with Psuedo get worse results?",
    "958751": "",
    "958792": "Excellent work! Thanks for sharing.",
    "959648": "Thank you for sharing"
  }
}