{
  "id": 153084,
  "title": "Best single model",
  "url": "/competitions/global-wheat-detection/discussion/153084",
  "author_name": "Matt",
  "post_date": "2020-05-23T04:34:48.250000",
  "votes": 37,
  "comment_count": 81,
  "views": 0,
  "content": "<p>I'm curious, what is your best single model? No pseudolabeling, no ensembling.</p>\n\n<p>Mine:\n0.7088 LB\nEpochs: 50\nDataset: 1024^2 images\nDetector: Faster RCNN\nBackbone: resnet50\nAugmentation: flipping, random hsv, random brightness/contrast, custom \"stitching\" augmentation</p>",
  "messages": [
    {
      "id": 857938,
      "postDate": "2020-05-23T04:34:48.250Z",
      "content": "<p>I'm curious, what is your best single model? No pseudolabeling, no ensembling.</p>\n\n<p>Mine:\n0.7088 LB\nEpochs: 50\nDataset: 1024^2 images\nDetector: Faster RCNN\nBackbone: resnet50\nAugmentation: flipping, random hsv, random brightness/contrast, custom \"stitching\" augmentation</p>",
      "rawMarkdown": "I'm curious, what is your best single model? No pseudolabeling, no ensembling.\n\nMine:\n0.7088 LB\nEpochs: 50\nDataset: 1024^2 images\nDetector: Faster RCNN\nBackbone: resnet50\nAugmentation: flipping, random hsv, random brightness/contrast, custom \"stitching\" augmentation\n",
      "votes": 37
    },
    {
      "id": 921815,
      "postDate": "2020-07-09T15:21:51.560Z",
      "content": "<p>YOLOV5 out.....</p>\n\n<ol>\n<li>EffDet5</li>\n<li>1024 as input</li>\n<li>single fold - 80/20 split</li>\n<li>with some training tricks\n(Gradient Accumulation for bigger batch size, nvidia apex, personal implementation of data augmentation)</li>\n<li>epochs: 60\nLB: 0.7516 (with TTA)</li>\n</ol>",
      "rawMarkdown": "YOLOV5 out.....\n\n1. EffDet5\n2. 1024 as input\n3. single fold - 80/20 split\n4. with some training tricks\n (Gradient Accumulation for bigger batch size, nvidia apex, personal implementation of data augmentation)\n5. epochs: 60\nLB: 0.7516 (with TTA)",
      "votes": 10,
      "replies": [
        {
          "id": 927397,
          "postDate": "2020-07-13T11:01:15.180Z",
          "content": "<p>Hello! I'm tried Gradient Accumulation too, but it seems that the train loss is lower but valid loss is higher than ever before, can you tell me how did you use this trick? Thanks!</p>",
          "rawMarkdown": "Hello! I'm tried Gradient Accumulation too, but it seems that the train loss is lower but valid loss is higher than ever before, can you tell me how did you use this trick? Thanks!"
        },
        {
          "id": 927406,
          "postDate": "2020-07-13T11:08:40.140Z",
          "content": "<p>Hi Wuxin, \nIf I am not wrong... I guess you even need to change learning rate and scheduler...</p>",
          "rawMarkdown": "Hi Wuxin, \nIf I am not wrong... I guess you even need to change learning rate and scheduler...\n"
        },
        {
          "id": 927410,
          "postDate": "2020-07-13T11:13:01.003Z",
          "content": "<p>Thank! I use learning rate 0.0001 and cos scheduler. I'll try other hyperparamters.</p>",
          "rawMarkdown": "Thank! I use learning rate 0.0001 and cos scheduler. I'll try other hyperparamters."
        },
        {
          "id": 937456,
          "postDate": "2020-07-21T03:19:36.083Z",
          "content": "<p>How much time did it take?</p>",
          "rawMarkdown": "How much time did it take?"
        },
        {
          "id": 938067,
          "postDate": "2020-07-21T10:22:46.443Z",
          "content": "<p>about one day.</p>",
          "rawMarkdown": "about one day."
        }
      ]
    },
    {
      "id": 919612,
      "postDate": "2020-07-08T02:33:28.993Z",
      "content": "<p>EffDet5: 0.7508(with pseudo labels)\nYolov5: out😂 .</p>",
      "rawMarkdown": "EffDet5: 0.7508(with pseudo labels)\nYolov5: out😂 .",
      "votes": 5
    },
    {
      "id": 881426,
      "postDate": "2020-06-11T03:09:43.433Z",
      "content": "<p>[Updated]\n1.  EffDet5\n2. 1024 as input\n3. single fold - 80/20 split\n4. with some training tricks(Gradient Accumulation for bigger batch size, nvidia apex) \n5. epochs: 60 </p>\n\n<p>LB: 0.7487(with TTA)</p>",
      "rawMarkdown": "[Updated]\n1.  EffDet5\n2. 1024 as input\n3. single fold - 80/20 split\n4. with some training tricks(Gradient Accumulation for bigger batch size, nvidia apex) \n5. epochs: 60 \n\nLB: 0.7487(with TTA)\n",
      "votes": 6,
      "replies": [
        {
          "id": 891744,
          "postDate": "2020-06-18T12:34:08.080Z",
          "content": "<p>Did gradient accumulation boost your score ? \nI was thinking on using it for 1024 images, it did not come to my mind for 512px</p>",
          "rawMarkdown": "Did gradient accumulation boost your score ? \nI was thinking on using it for 1024 images, it did not come to my mind for 512px",
          "votes": 1
        },
        {
          "id": 891899,
          "postDate": "2020-06-18T14:17:01.993Z",
          "content": "<p>Yes, it worked for me. You should change learning rate and lr scheduler.</p>",
          "rawMarkdown": "Yes, it worked for me. You should change learning rate and lr scheduler.",
          "votes": 4
        },
        {
          "id": 917330,
          "postDate": "2020-07-06T12:20:03.857Z",
          "content": "<p>I implemented all training tricks on effdet5, but model is not generalizing well. Any clue about how to change the learning rate and the scheduler?</p>",
          "rawMarkdown": "I implemented all training tricks on effdet5, but model is not generalizing well. Any clue about how to change the learning rate and the scheduler?"
        }
      ]
    },
    {
      "id": 858185,
      "postDate": "2020-05-23T09:28:33.293Z",
      "content": "<p>I think my score is overfitted. I got 0.7265 for a single FasterRCNN model 2 weeks ago. Then I tried many better ideas, but none of them can reach 0.72.</p>",
      "rawMarkdown": "I think my score is overfitted. I got 0.7265 for a single FasterRCNN model 2 weeks ago. Then I tried many better ideas, but none of them can reach 0.72.",
      "votes": 5,
      "replies": [
        {
          "id": 858194,
          "postDate": "2020-05-23T09:34:27.397Z",
          "content": "<p>Interesting, thank you for sharing!</p>",
          "rawMarkdown": "Interesting, thank you for sharing!"
        }
      ]
    },
    {
      "id": 902238,
      "postDate": "2020-06-26T02:46:49.770Z",
      "content": "<p>[update]\nEffDet5\n512 input\nsingle fold\n40 epochs  --&gt;  50 epochs\n0.7393 --&gt; 0.7441 with TTA (no pseudo labels)</p>",
      "rawMarkdown": "[update]\nEffDet5\n512 input\nsingle fold\n40 epochs  --&gt;  50 epochs\n0.7393 --&gt; 0.7441 with TTA (no pseudo labels)",
      "votes": 3,
      "replies": [
        {
          "id": 903129,
          "postDate": "2020-06-26T15:35:42.107Z",
          "content": "<p>Wow, impressive! Good work</p>",
          "rawMarkdown": "Wow, impressive! Good work"
        }
      ]
    },
    {
      "id": 920470,
      "postDate": "2020-07-08T16:08:41.567Z",
      "content": "<p>Effdet5 512 0.7393 no pseudo no tta\nYolov5 😨😨😨</p>",
      "rawMarkdown": "Effdet5 512 0.7393 no pseudo no tta\nYolov5 😨😨😨",
      "votes": 4,
      "replies": [
        {
          "id": 931655,
          "postDate": "2020-07-16T11:00:53.800Z",
          "content": "<p>Awesome! Couldyou tell me your training strategy?</p>",
          "rawMarkdown": "Awesome! Couldyou tell me your training strategy?"
        }
      ]
    },
    {
      "id": 912549,
      "postDate": "2020-07-02T15:01:57.950Z",
      "content": "<p>EffDet4\n768 input\nsingle fold\n30 epochs\n0.7267\nAbout 8 hours on a local 2080ti FP32</p>",
      "rawMarkdown": "EffDet4\n768 input\nsingle fold\n30 epochs\n0.7267\nAbout 8 hours on a local 2080ti FP32",
      "votes": 1,
      "replies": [
        {
          "id": 917464,
          "postDate": "2020-07-06T14:32:35.663Z",
          "rawMarkdown": "",
          "votes": -2,
          "isDeleted": true
        }
      ]
    },
    {
      "id": 891853,
      "postDate": "2020-06-18T13:49:14.310Z",
      "content": "<p>Hi <a href=\"/matthewmasters\">@matthewmasters</a>, I am trying Effdet-5 on 1024 but I am running out of memory on Colab and 2070...\nHow are you dealing with this?? Any suggestions??</p>",
      "rawMarkdown": "Hi @matthewmasters, I am trying Effdet-5 on 1024 but I am running out of memory on Colab and 2070...\nHow are you dealing with this?? Any suggestions??",
      "votes": 1,
      "replies": [
        {
          "id": 891908,
          "postDate": "2020-06-18T14:24:46.493Z",
          "content": "<p>Gradient Accumulation</p>",
          "rawMarkdown": "Gradient Accumulation",
          "votes": 1
        },
        {
          "id": 892324,
          "postDate": "2020-06-18T20:00:35.197Z",
          "content": "<p>I am not using effdet so I wouldn't know exactly.</p>",
          "rawMarkdown": "I am not using effdet so I wouldn't know exactly."
        },
        {
          "id": 911672,
          "postDate": "2020-07-01T23:32:38.383Z",
          "content": "<p><a href=\"/nitindatta\">@nitindatta</a> \nTry reducing the batch size.</p>",
          "rawMarkdown": "@nitindatta \nTry reducing the batch size."
        }
      ]
    },
    {
      "id": 879220,
      "postDate": "2020-06-09T10:01:46.893Z",
      "content": "<p>Single fcos dcnv2 X 101 FPN model with augmentation got 0.7322. </p>",
      "rawMarkdown": "Single fcos dcnv2 X 101 FPN model with augmentation got 0.7322. ",
      "votes": 1,
      "replies": [
        {
          "id": 879343,
          "postDate": "2020-06-09T12:08:41.120Z",
          "content": "<p>Very well done!</p>",
          "rawMarkdown": "Very well done!"
        }
      ]
    },
    {
      "id": 858201,
      "postDate": "2020-05-23T09:40:16.383Z",
      "content": "<p><a href=\"/matthewmasters\">@matthewmasters</a> your current LB (0.7222) score is from a single model too (one fold, no tta, no ensemble, etc)?</p>",
      "rawMarkdown": "@matthewmasters your current LB (0.7222) score is from a single model too (one fold, no tta, no ensemble, etc)?",
      "votes": 1,
      "replies": [
        {
          "id": 858225,
          "postDate": "2020-05-23T10:03:45.123Z",
          "content": "<p>No, that score was my first time using ensembling and tta</p>",
          "rawMarkdown": "No, that score was my first time using ensembling and tta",
          "votes": 1
        }
      ]
    },
    {
      "id": 857947,
      "postDate": "2020-05-23T05:04:59.927Z",
      "content": "<p>Do  you  have  a  try  about   CascadeRcnn ?</p>",
      "rawMarkdown": "Do  you  have  a  try  about   CascadeRcnn ?",
      "votes": 1,
      "replies": [
        {
          "id": 857950,
          "postDate": "2020-05-23T05:07:15.977Z",
          "content": "<p>I haven't. I have only taken a brief look at the paper to see what it was. It looks promising so I'll have to investigate it more at some point</p>",
          "rawMarkdown": "I haven't. I have only taken a brief look at the paper to see what it was. It looks promising so I'll have to investigate it more at some point"
        }
      ]
    },
    {
      "id": 891732,
      "postDate": "2020-06-18T12:24:45.387Z",
      "content": "<p>[Update]\nyolov5x\nLB: 0.7526\n<a href=\"https://www.kaggle.com/orkatz2/yolov5-fake-or-real-single-model-l-b-0-73\">https://www.kaggle.com/orkatz2/yolov5-fake-or-real-single-model-l-b-0-73</a></p>",
      "rawMarkdown": "[Update]\nyolov5x\nLB: 0.7526\nhttps://www.kaggle.com/orkatz2/yolov5-fake-or-real-single-model-l-b-0-73",
      "votes": 2,
      "replies": [
        {
          "id": 892329,
          "postDate": "2020-06-18T20:01:43.767Z",
          "content": "<p>Wow, I think that's highest yet! Great work. Does this include any tricks (TTA, ensemble, pseudolabel)?</p>",
          "rawMarkdown": "Wow, I think that's highest yet! Great work. Does this include any tricks (TTA, ensemble, pseudolabel)?",
          "votes": 1
        },
        {
          "id": 900001,
          "postDate": "2020-06-24T15:10:38.923Z",
          "content": "<p>Yes, the yolov5 implementation have some hyperparameters about augmentation set by default. Actually, flipping is almost hardcoded :D</p>",
          "rawMarkdown": "Yes, the yolov5 implementation have some hyperparameters about augmentation set by default. Actually, flipping is almost hardcoded :D",
          "votes": 1
        },
        {
          "id": 900937,
          "postDate": "2020-06-25T06:45:20.803Z",
          "content": "<p>Actually YoloV5's data loader class has almost all the augmentations which are being used in this competition. That's great! Take a look at this: <a href=\"https://github.com/ultralytics/yolov5/blob/master/utils/datasets.py\">https://github.com/ultralytics/yolov5/blob/master/utils/datasets.py</a></p>\n\n<p>The code is also super readable with great documentation, unlike DetectoRS and MMDetection, which is super complicated for beginners like me. </p>",
          "rawMarkdown": "Actually YoloV5's data loader class has almost all the augmentations which are being used in this competition. That's great! Take a look at this: https://github.com/ultralytics/yolov5/blob/master/utils/datasets.py\n\nThe code is also super readable with great documentation, unlike DetectoRS and MMDetection, which is super complicated for beginners like me. ",
          "votes": 5
        },
        {
          "id": 902227,
          "postDate": "2020-06-26T02:32:31.313Z",
          "content": "<p>&gt; unlike DetectoRS and MMDetection, which is super complicated</p>\n\n<p>I am playing with mmdetection these 10 days and totally agree with you now.</p>",
          "rawMarkdown": "&gt; unlike DetectoRS and MMDetection, which is super complicated\n\nI am playing with mmdetection these 10 days and totally agree with you now.",
          "votes": 2
        }
      ]
    },
    {
      "id": 880365,
      "postDate": "2020-06-10T08:21:33.767Z",
      "content": "<p>EffDet5, \n512 input, \nsingle fold -  80/20 split <br>\n0.7354</p>",
      "rawMarkdown": "EffDet5, \n512 input, \nsingle fold -  80/20 split  \n0.7354",
      "votes": 2,
      "replies": [
        {
          "id": 880614,
          "postDate": "2020-06-10T12:40:46.420Z",
          "content": "<p>Wow, I'm surprised you were able to get so far with 512 images, well done!</p>",
          "rawMarkdown": "Wow, I'm surprised you were able to get so far with 512 images, well done!"
        },
        {
          "id": 880935,
          "postDate": "2020-06-10T16:45:14.227Z",
          "content": "<p><a href=\"/matthewmasters\">@matthewmasters</a> <br>\nDo you find a big performance difference between training with 512 v.s. 1024?</p>",
          "rawMarkdown": "@matthewmasters  \nDo you find a big performance difference between training with 512 v.s. 1024?"
        },
        {
          "id": 880938,
          "postDate": "2020-06-10T16:47:32.407Z",
          "content": "<p>i haven't tried bigger than 512</p>",
          "rawMarkdown": "i haven't tried bigger than 512\n"
        },
        {
          "id": 881331,
          "postDate": "2020-06-10T23:44:15.343Z",
          "content": "<p>thank you for sharing, \nwhat is your batch size?</p>",
          "rawMarkdown": "thank you for sharing, \nwhat is your batch size?\n"
        },
        {
          "id": 881332,
          "postDate": "2020-06-10T23:44:26.400Z",
          "content": "<p><a href=\"/alexlwh\">@alexlwh</a> from my experience, 1024 has always been better than 512. I have also tried ensembling both together but it was worse than 1024 alone</p>",
          "rawMarkdown": "@alexlwh from my experience, 1024 has always been better than 512. I have also tried ensembling both together but it was worse than 1024 alone"
        },
        {
          "id": 881549,
          "postDate": "2020-06-11T06:46:30.740Z",
          "content": "<p>Larger should be better. However I am relaying on kernels and need to use smaller sizes. </p>\n\n<p>Batch size is 4.</p>\n\n<p>PS: I am not using pseudo labels - yet</p>",
          "rawMarkdown": "Larger should be better. However I am relaying on kernels and need to use smaller sizes. \n\nBatch size is 4.\n\nPS: I am not using pseudo labels - yet",
          "votes": 1
        },
        {
          "id": 882068,
          "postDate": "2020-06-11T15:03:00.790Z",
          "content": "<p>Relying on kernels as in you do preprocessing with some kernel or you just mean within the neural network?</p>",
          "rawMarkdown": "Relying on kernels as in you do preprocessing with some kernel or you just mean within the neural network?"
        }
      ]
    },
    {
      "id": 877762,
      "postDate": "2020-06-08T00:11:13.913Z",
      "content": "<p>So far I'm at 0.7118 with a single model: Resnet101 + FasterRCNN (Pytorch) + ~60 epochs + a bunch of stock albumentations augmentations + minor data cleaning + minor model tweaks + one fold 80/20 split. No ensembing, no tta, no pseudolabeling, no WBF. </p>",
      "rawMarkdown": "So far I'm at 0.7118 with a single model: Resnet101 + FasterRCNN (Pytorch) + ~60 epochs + a bunch of stock albumentations augmentations + minor data cleaning + minor model tweaks + one fold 80/20 split. No ensembing, no tta, no pseudolabeling, no WBF. ",
      "votes": 2,
      "replies": [
        {
          "id": 877764,
          "postDate": "2020-06-08T00:17:02.053Z",
          "content": "<p>Well done, I'm sure ensembling and TTA will bring that score up to silver/gold level</p>",
          "rawMarkdown": "Well done, I'm sure ensembling and TTA will bring that score up to silver/gold level",
          "votes": 1
        },
        {
          "id": 877930,
          "postDate": "2020-06-08T05:55:14.913Z",
          "content": "<p>Thanks Matthew! 😃  I spent too much time trying to generate extra training images with CycleGAN and it didn't really work out. So I feel like I'm playing a bit of catch up at the moment. </p>",
          "rawMarkdown": "Thanks Matthew! 😃  I spent too much time trying to generate extra training images with CycleGAN and it didn't really work out. So I feel like I'm playing a bit of catch up at the moment. ",
          "votes": 1
        },
        {
          "id": 877946,
          "postDate": "2020-06-08T06:23:57.283Z",
          "content": "<p>Yeah, I hear you. I've tried many things up to this point that didn't work out. I also tried using CycleGAN and some other adversarial networks to solve the \"domain generalization\" problem but no success.</p>",
          "rawMarkdown": "Yeah, I hear you. I've tried many things up to this point that didn't work out. I also tried using CycleGAN and some other adversarial networks to solve the \"domain generalization\" problem but no success.",
          "votes": 3
        }
      ]
    },
    {
      "id": 857944,
      "postDate": "2020-05-23T04:58:35.847Z",
      "content": "<p>My score is quite low:\n0.6624 public LB\nepochs: 5\nimage size: 1024 x 1024\nDetector: Faster RCNN\nbackbone: ResNet101\nAugs: flipping, random hsv, random brightness/contrast</p>\n\n<p>BTW how much time it took to train for 150 epochs?! <a href=\"/matthewmasters\">@matthewmasters</a> . In my case when I trained more than 5 epochs (15 epochs), score decreased because of overfitting (I think)</p>",
      "rawMarkdown": "My score is quite low:\n0.6624 public LB\nepochs: 5\nimage size: 1024 x 1024\nDetector: Faster RCNN\nbackbone: ResNet101\nAugs: flipping, random hsv, random brightness/contrast\n\nBTW how much time it took to train for 150 epochs?! @matthewmasters . In my case when I trained more than 5 epochs (15 epochs), score decreased because of overfitting (I think)",
      "votes": 2,
      "replies": [
        {
          "id": 857949,
          "postDate": "2020-05-23T05:05:21.267Z",
          "content": "<p>Sorry, that was actually a mistake, it was just 50 epochs. I corrected the original post.\nIt still took a long time though. Around 16 hours on my 2070.</p>",
          "rawMarkdown": "Sorry, that was actually a mistake, it was just 50 epochs. I corrected the original post.\nIt still took a long time though. Around 16 hours on my 2070."
        },
        {
          "id": 857952,
          "postDate": "2020-05-23T05:09:06.253Z",
          "content": "<p>Cool! So on kaggle kernels with K80 it will take even longer...</p>",
          "rawMarkdown": "Cool! So on kaggle kernels with K80 it will take even longer...",
          "votes": -1
        },
        {
          "id": 866947,
          "postDate": "2020-05-29T20:13:11.437Z",
          "content": "<p>Do you thing this resolution of images is necessary?</p>",
          "rawMarkdown": "Do you thing this resolution of images is necessary?\n"
        },
        {
          "id": 867058,
          "postDate": "2020-05-30T00:58:29.533Z",
          "content": "<p>From my results, 1024 was always better. Makes sense as some of the more crowded regions are difficult to discern between plants at lower resolution.</p>",
          "rawMarkdown": "From my results, 1024 was always better. Makes sense as some of the more crowded regions are difficult to discern between plants at lower resolution."
        }
      ]
    },
    {
      "id": 957748,
      "postDate": "2020-08-04T14:33:01.290Z",
      "content": "<p><strong>AlexeyAB/darknet YOLOv4 with pretrained mish-416 weights</strong>\n- Single fold selected from stratified k-folds (K=5)\n- 608x608 resolution for width and height, but using 1024x1024 as input\n- About 12,000 iterations to get best weights\n- lr=0.0005\n- Data augmentation: <code>mosaic=1</code>, <code>blur=1</code>, <code>gaussian_noise=1</code>\n- TTA + WBF\n- Pseudo Labeling for 1,000 iterations\n- Public LB: 0.7352</p>",
      "rawMarkdown": "**AlexeyAB/darknet YOLOv4 with pretrained mish-416 weights**\n- Single fold selected from stratified k-folds (K=5)\n- 608x608 resolution for width and height, but using 1024x1024 as input\n- About 12,000 iterations to get best weights\n- lr=0.0005\n- Data augmentation: `mosaic=1`, `blur=1`, `gaussian_noise=1`\n- TTA + WBF\n- Pseudo Labeling for 1,000 iterations\n- Public LB: 0.7352\n"
    },
    {
      "id": 955912,
      "postDate": "2020-08-03T04:36:23.203Z",
      "content": "<p>What was the detection threshold you used while inferencing?</p>",
      "rawMarkdown": "What was the detection threshold you used while inferencing?",
      "replies": [
        {
          "id": 955929,
          "postDate": "2020-08-03T05:12:28.583Z",
          "content": "<p>Me? 0.5 for FasterRCNN</p>",
          "rawMarkdown": "Me? 0.5 for FasterRCNN"
        }
      ]
    },
    {
      "id": 936777,
      "postDate": "2020-07-20T14:19:08.177Z",
      "content": "<p>efficientdet-5 512X512<br>\nsingle model <br>\nlb = 0.7504</p>",
      "rawMarkdown": "efficientdet-5 512X512\nsingle model \nlb = 0.7504",
      "replies": [
        {
          "id": 937449,
          "postDate": "2020-07-21T03:09:33.057Z",
          "content": "<p>For how many epochs?\n&amp; what was the learning rate?</p>",
          "rawMarkdown": "For how many epochs?\n&amp; what was the learning rate?"
        },
        {
          "id": 938208,
          "postDate": "2020-07-21T12:00:08.300Z",
          "content": "<p>78-80 epoch<br>\nlr = 0.001 with ex decay</p>",
          "rawMarkdown": "78-80 epoch\nlr = 0.001 with ex decay"
        },
        {
          "id": 938335,
          "postDate": "2020-07-21T13:15:25.503Z",
          "content": "<p><a href=\"/ajax0564\">@ajax0564</a>  Hello! What is ex decay mean? Is it ExponentialLR?</p>",
          "rawMarkdown": "@ajax0564  Hello! What is ex decay mean? Is it ExponentialLR?"
        },
        {
          "id": 938456,
          "postDate": "2020-07-21T14:43:39.860Z",
          "content": "<p>ExponentialLR?</p>",
          "rawMarkdown": "ExponentialLR?"
        },
        {
          "id": 938966,
          "postDate": "2020-07-21T22:42:28.157Z",
          "content": "<p>😂 <a href=\"https://pytorch.org/docs/stable/_modules/torch/optim/lr_scheduler.html#ExponentialLR\">This one</a></p>",
          "rawMarkdown": "😂 [This one](https://pytorch.org/docs/stable/_modules/torch/optim/lr_scheduler.html#ExponentialLR)"
        }
      ]
    },
    {
      "id": 911956,
      "postDate": "2020-07-02T06:03:56.417Z",
      "content": "<p>Can I ask are you guys using kaggle GPU/TPU kernels, local GPUs, colab pro, vast.ai? </p>\n\n<p>Thanks</p>",
      "rawMarkdown": "Can I ask are you guys using kaggle GPU/TPU kernels, local GPUs, colab pro, vast.ai? \n\nThanks",
      "replies": [
        {
          "id": 911967,
          "postDate": "2020-07-02T06:15:03.423Z",
          "content": "<p>Hi <a href=\"/bopengiowa\">@bopengiowa</a> \nMost of them use colab or colab pro, very few who train on local and there might be some who train on kaggle but kaggle GPU is mostly used for inference and submission purposes..</p>",
          "rawMarkdown": "Hi @bopengiowa \nMost of them use colab or colab pro, very few who train on local and there might be some who train on kaggle but kaggle GPU is mostly used for inference and submission purposes..",
          "votes": 2
        },
        {
          "id": 911968,
          "postDate": "2020-07-02T06:15:59.867Z",
          "content": "<p>cool thanks</p>",
          "rawMarkdown": "cool thanks"
        }
      ]
    },
    {
      "id": 881217,
      "postDate": "2020-06-10T20:30:36.457Z",
      "content": "<p><a href=\"/matthewmasters\">@matthewmasters</a> \nI'm trying with mask-rcnn, but I know, maybe it's not the proper choice for this task (due to its instance seg parts). After playing with many configurations param set up, I've reached <a href=\"https://www.kaggle.com/ipythonx/gwd-with-mask-rcnn-and-coco-transfer-learning\">0.6449</a> scores on it. I don't have (still) pure intention to participate seriously in this competition.  But just wondering if anyone has tried it already for this task.</p>",
      "rawMarkdown": "@matthewmasters \nI'm trying with mask-rcnn, but I know, maybe it's not the proper choice for this task (due to its instance seg parts). After playing with many configurations param set up, I've reached [0.6449](https://www.kaggle.com/ipythonx/gwd-with-mask-rcnn-and-coco-transfer-learning) scores on it. I don't have (still) pure intention to participate seriously in this competition.  But just wondering if anyone has tried it already for this task."
    },
    {
      "id": 868000,
      "postDate": "2020-05-30T20:08:13.750Z",
      "content": "<p>Thanks for the sharing. \nA few questions about your train setting:\n1. Did you entirely do fine-tuning on the head in the 50 epochs, or you unfreeze all layers in the later stage of the 50 epochs? \n2. Did you use the default pretrained FPN? (i.e. COCO resnet50)</p>",
      "rawMarkdown": "Thanks for the sharing. \nA few questions about your train setting:\n1. Did you entirely do fine-tuning on the head in the 50 epochs, or you unfreeze all layers in the later stage of the 50 epochs? \n2. Did you use the default pretrained FPN? (i.e. COCO resnet50)",
      "replies": [
        {
          "id": 868116,
          "postDate": "2020-05-31T00:08:41.300Z",
          "content": "<ol>\n<li>Not sure exactly what you mean. The head is trained the entire 50 epochs.</li>\n<li>Yes</li>\n</ol>\n\n<p>My model is very close to the default one provided by torchvision</p>",
          "rawMarkdown": "1. Not sure exactly what you mean. The head is trained the entire 50 epochs.\n2. Yes\n\nMy model is very close to the default one provided by torchvision"
        },
        {
          "id": 868941,
          "postDate": "2020-05-31T15:28:35.737Z",
          "content": "<p>sorry for the confusion in Q1. \nI mean did you fine-tune your backbone in the 50 epochs?</p>",
          "rawMarkdown": "sorry for the confusion in Q1. \nI mean did you fine-tune your backbone in the 50 epochs?"
        }
      ]
    },
    {
      "id": 859818,
      "postDate": "2020-05-24T19:08:24.870Z",
      "content": "<p>For the moment, FasterRCNN, 0.67, with few epochs and augmentations. </p>",
      "rawMarkdown": "For the moment, FasterRCNN, 0.67, with few epochs and augmentations. "
    },
    {
      "id": 858914,
      "postDate": "2020-05-24T00:00:55.133Z",
      "content": "<p>Hello! I'm using fasterrcnn too. And my augmentation is also flipping, random hsv, randombrightnesscontrast, cutout. But it just reach 69.23 LB. So what about custom \"stitching\" augmentation mean? Is it cutmix?</p>",
      "rawMarkdown": "Hello! I'm using fasterrcnn too. And my augmentation is also flipping, random hsv, randombrightnesscontrast, cutout. But it just reach 69.23 LB. So what about custom \"stitching\" augmentation mean? Is it cutmix?",
      "replies": [
        {
          "id": 858922,
          "postDate": "2020-05-24T00:28:39.837Z",
          "content": "<p>I don't wish to share my exact augmentation but yes it is similar to cutmix but patches from different images can be mixed together too</p>",
          "rawMarkdown": "I don't wish to share my exact augmentation but yes it is similar to cutmix but patches from different images can be mixed together too"
        },
        {
          "id": 858956,
          "postDate": "2020-05-24T01:41:59.570Z",
          "content": "<p>Thanks! It seems that we can do a lot augmentation to improve the performance. Just for beginners like me it's hard. But I'll do my best to write some augmentation function.</p>",
          "rawMarkdown": "Thanks! It seems that we can do a lot augmentation to improve the performance. Just for beginners like me it's hard. But I'll do my best to write some augmentation function."
        },
        {
          "id": 859221,
          "postDate": "2020-05-24T08:50:44.493Z",
          "content": "<p><a href=\"/whurobin\">@whurobin</a> Which implementation of cutout are you using? And are you augmenting the images on the fly or creating a larger dataset before training? Thanks in advance!</p>",
          "rawMarkdown": "@whurobin Which implementation of cutout are you using? And are you augmenting the images on the fly or creating a larger dataset before training? Thanks in advance!"
        },
        {
          "id": 859290,
          "postDate": "2020-05-24T09:49:55.347Z",
          "content": "<p>Question wasn't at me but just my two cents: if your going to use cutout, do it on the fly.</p>",
          "rawMarkdown": "Question wasn't at me but just my two cents: if your going to use cutout, do it on the fly.",
          "votes": 1
        },
        {
          "id": 859465,
          "postDate": "2020-05-24T12:54:55.120Z",
          "content": "<p>I just use Albu's cutout, and using it during training.</p>",
          "rawMarkdown": "I just use Albu's cutout, and using it during training.",
          "votes": 1
        },
        {
          "id": 859589,
          "postDate": "2020-05-24T15:07:10.177Z",
          "content": "<p>Okay Thanks <a href=\"/matthewmasters\">@matthewmasters</a> and <a href=\"/whurobin\">@whurobin</a> </p>",
          "rawMarkdown": "Okay Thanks @matthewmasters and @whurobin "
        },
        {
          "id": 862658,
          "postDate": "2020-05-26T18:09:53.063Z",
          "content": "<p>how many holes are you using?</p>",
          "rawMarkdown": "how many holes are you using?"
        },
        {
          "id": 862854,
          "postDate": "2020-05-26T21:57:44.353Z",
          "content": "<p>I don't know whether my hyperparameter was good or not, I use 8 holes.</p>",
          "rawMarkdown": "I don't know whether my hyperparameter was good or not, I use 8 holes."
        }
      ]
    },
    {
      "id": 858017,
      "postDate": "2020-05-23T06:37:47.850Z",
      "content": "<p>Can u tell how long it took to train 50 epochs?</p>",
      "rawMarkdown": "Can u tell how long it took to train 50 epochs?",
      "replies": [
        {
          "id": 858018,
          "postDate": "2020-05-23T06:38:48.993Z",
          "content": "<p>Yeah I mentioned it in another comment. Around 16 hours on my 2070.</p>",
          "rawMarkdown": "Yeah I mentioned it in another comment. Around 16 hours on my 2070.",
          "votes": 1
        },
        {
          "id": 858022,
          "postDate": "2020-05-23T06:41:57.417Z",
          "content": "<p>Okk..Thanks!!</p>",
          "rawMarkdown": "Okk..Thanks!!"
        }
      ]
    },
    {
      "id": 920447,
      "postDate": "2020-07-08T15:50:38.590Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 921815,
      "author_name": "Chanhu",
      "author_url": "",
      "post_date": "2020-07-09T15:21:51.560000",
      "content": "<p>YOLOV5 out.....</p>\n\n<ol>\n<li>EffDet5</li>\n<li>1024 as input</li>\n<li>single fold - 80/20 split</li>\n<li>with some training tricks\n(Gradient Accumulation for bigger batch size, nvidia apex, personal implementation of data augmentation)</li>\n<li>epochs: 60\nLB: 0.7516 (with TTA)</li>\n</ol>",
      "votes": 10,
      "replies": [
        {
          "id": 927397,
          "author_name": "27Wilson",
          "author_url": "",
          "post_date": "2020-07-13T11:01:15.180000",
          "content": "<p>Hello! I'm tried Gradient Accumulation too, but it seems that the train loss is lower but valid loss is higher than ever before, can you tell me how did you use this trick? Thanks!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 927406,
          "author_name": "Nitin Datta",
          "author_url": "",
          "post_date": "2020-07-13T11:08:40.140000",
          "content": "<p>Hi Wuxin, \nIf I am not wrong... I guess you even need to change learning rate and scheduler...</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 927410,
          "author_name": "27Wilson",
          "author_url": "",
          "post_date": "2020-07-13T11:13:01.003000",
          "content": "<p>Thank! I use learning rate 0.0001 and cos scheduler. I'll try other hyperparamters.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 937456,
          "author_name": "AnkurSingh",
          "author_url": "",
          "post_date": "2020-07-21T03:19:36.083000",
          "content": "<p>How much time did it take?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 938067,
          "author_name": "Chanhu",
          "author_url": "",
          "post_date": "2020-07-21T10:22:46.443000",
          "content": "<p>about one day.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 919612,
      "author_name": "ld",
      "author_url": "",
      "post_date": "2020-07-08T02:33:28.993000",
      "content": "<p>EffDet5: 0.7508(with pseudo labels)\nYolov5: out😂 .</p>",
      "votes": 5,
      "replies": []
    },
    {
      "id": 881426,
      "author_name": "Chanhu",
      "author_url": "",
      "post_date": "2020-06-11T03:09:43.433000",
      "content": "<p>[Updated]\n1.  EffDet5\n2. 1024 as input\n3. single fold - 80/20 split\n4. with some training tricks(Gradient Accumulation for bigger batch size, nvidia apex) \n5. epochs: 60 </p>\n\n<p>LB: 0.7487(with TTA)</p>",
      "votes": 6,
      "replies": [
        {
          "id": 891744,
          "author_name": "Alex",
          "author_url": "",
          "post_date": "2020-06-18T12:34:08.080000",
          "content": "<p>Did gradient accumulation boost your score ? \nI was thinking on using it for 1024 images, it did not come to my mind for 512px</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 891899,
          "author_name": "Chanhu",
          "author_url": "",
          "post_date": "2020-06-18T14:17:01.993000",
          "content": "<p>Yes, it worked for me. You should change learning rate and lr scheduler.</p>",
          "votes": 4,
          "replies": []
        },
        {
          "id": 917330,
          "author_name": "ademdum",
          "author_url": "",
          "post_date": "2020-07-06T12:20:03.857000",
          "content": "<p>I implemented all training tricks on effdet5, but model is not generalizing well. Any clue about how to change the learning rate and the scheduler?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 858185,
      "author_name": "nvnn",
      "author_url": "",
      "post_date": "2020-05-23T09:28:33.293000",
      "content": "<p>I think my score is overfitted. I got 0.7265 for a single FasterRCNN model 2 weeks ago. Then I tried many better ideas, but none of them can reach 0.72.</p>",
      "votes": 5,
      "replies": [
        {
          "id": 858194,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-05-23T09:34:27.397000",
          "content": "<p>Interesting, thank you for sharing!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 902238,
      "author_name": "Qingyao Shuai",
      "author_url": "",
      "post_date": "2020-06-26T02:46:49.770000",
      "content": "<p>[update]\nEffDet5\n512 input\nsingle fold\n40 epochs  --&gt;  50 epochs\n0.7393 --&gt; 0.7441 with TTA (no pseudo labels)</p>",
      "votes": 3,
      "replies": [
        {
          "id": 903129,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-06-26T15:35:42.107000",
          "content": "<p>Wow, impressive! Good work</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 920470,
      "author_name": "DungNB",
      "author_url": "",
      "post_date": "2020-07-08T16:08:41.567000",
      "content": "<p>Effdet5 512 0.7393 no pseudo no tta\nYolov5 😨😨😨</p>",
      "votes": 4,
      "replies": [
        {
          "id": 931655,
          "author_name": "27Wilson",
          "author_url": "",
          "post_date": "2020-07-16T11:00:53.800000",
          "content": "<p>Awesome! Couldyou tell me your training strategy?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 912549,
      "author_name": "peachthief",
      "author_url": "",
      "post_date": "2020-07-02T15:01:57.950000",
      "content": "<p>EffDet4\n768 input\nsingle fold\n30 epochs\n0.7267\nAbout 8 hours on a local 2080ti FP32</p>",
      "votes": 1,
      "replies": [
        {
          "id": 917464,
          "author_name": "",
          "author_url": "",
          "post_date": "2020-07-06T14:32:35.663000",
          "content": "",
          "votes": -2,
          "replies": []
        }
      ]
    },
    {
      "id": 891853,
      "author_name": "Nitin Datta",
      "author_url": "",
      "post_date": "2020-06-18T13:49:14.310000",
      "content": "<p>Hi <a href=\"/matthewmasters\">@matthewmasters</a>, I am trying Effdet-5 on 1024 but I am running out of memory on Colab and 2070...\nHow are you dealing with this?? Any suggestions??</p>",
      "votes": 1,
      "replies": [
        {
          "id": 891908,
          "author_name": "Chanhu",
          "author_url": "",
          "post_date": "2020-06-18T14:24:46.493000",
          "content": "<p>Gradient Accumulation</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 892324,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-06-18T20:00:35.197000",
          "content": "<p>I am not using effdet so I wouldn't know exactly.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 911672,
          "author_name": "Zaber Ibn Abdul Hakim",
          "author_url": "",
          "post_date": "2020-07-01T23:32:38.383000",
          "content": "<p><a href=\"/nitindatta\">@nitindatta</a> \nTry reducing the batch size.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 879220,
      "author_name": "sm047",
      "author_url": "",
      "post_date": "2020-06-09T10:01:46.893000",
      "content": "<p>Single fcos dcnv2 X 101 FPN model with augmentation got 0.7322. </p>",
      "votes": 1,
      "replies": [
        {
          "id": 879343,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-06-09T12:08:41.120000",
          "content": "<p>Very well done!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 858201,
      "author_name": "Peter",
      "author_url": "",
      "post_date": "2020-05-23T09:40:16.383000",
      "content": "<p><a href=\"/matthewmasters\">@matthewmasters</a> your current LB (0.7222) score is from a single model too (one fold, no tta, no ensemble, etc)?</p>",
      "votes": 1,
      "replies": [
        {
          "id": 858225,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-05-23T10:03:45.123000",
          "content": "<p>No, that score was my first time using ensembling and tta</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 857947,
      "author_name": "哈尔的移动城堡",
      "author_url": "",
      "post_date": "2020-05-23T05:04:59.927000",
      "content": "<p>Do  you  have  a  try  about   CascadeRcnn ?</p>",
      "votes": 1,
      "replies": [
        {
          "id": 857950,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-05-23T05:07:15.977000",
          "content": "<p>I haven't. I have only taken a brief look at the paper to see what it was. It looks promising so I'll have to investigate it more at some point</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 891732,
      "author_name": "OrKatz",
      "author_url": "",
      "post_date": "2020-06-18T12:24:45.387000",
      "content": "<p>[Update]\nyolov5x\nLB: 0.7526\n<a href=\"https://www.kaggle.com/orkatz2/yolov5-fake-or-real-single-model-l-b-0-73\">https://www.kaggle.com/orkatz2/yolov5-fake-or-real-single-model-l-b-0-73</a></p>",
      "votes": 2,
      "replies": [
        {
          "id": 892329,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-06-18T20:01:43.767000",
          "content": "<p>Wow, I think that's highest yet! Great work. Does this include any tricks (TTA, ensemble, pseudolabel)?</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 900001,
          "author_name": "ademdum",
          "author_url": "",
          "post_date": "2020-06-24T15:10:38.923000",
          "content": "<p>Yes, the yolov5 implementation have some hyperparameters about augmentation set by default. Actually, flipping is almost hardcoded :D</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 900937,
          "author_name": "Kaushal Shah",
          "author_url": "",
          "post_date": "2020-06-25T06:45:20.803000",
          "content": "<p>Actually YoloV5's data loader class has almost all the augmentations which are being used in this competition. That's great! Take a look at this: <a href=\"https://github.com/ultralytics/yolov5/blob/master/utils/datasets.py\">https://github.com/ultralytics/yolov5/blob/master/utils/datasets.py</a></p>\n\n<p>The code is also super readable with great documentation, unlike DetectoRS and MMDetection, which is super complicated for beginners like me. </p>",
          "votes": 5,
          "replies": []
        },
        {
          "id": 902227,
          "author_name": "yama",
          "author_url": "",
          "post_date": "2020-06-26T02:32:31.313000",
          "content": "<p>&gt; unlike DetectoRS and MMDetection, which is super complicated</p>\n\n<p>I am playing with mmdetection these 10 days and totally agree with you now.</p>",
          "votes": 2,
          "replies": []
        }
      ]
    },
    {
      "id": 880365,
      "author_name": "Miroslav Valan",
      "author_url": "",
      "post_date": "2020-06-10T08:21:33.767000",
      "content": "<p>EffDet5, \n512 input, \nsingle fold -  80/20 split <br>\n0.7354</p>",
      "votes": 2,
      "replies": [
        {
          "id": 880614,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-06-10T12:40:46.420000",
          "content": "<p>Wow, I'm surprised you were able to get so far with 512 images, well done!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 880935,
          "author_name": "Alex Lau",
          "author_url": "",
          "post_date": "2020-06-10T16:45:14.227000",
          "content": "<p><a href=\"/matthewmasters\">@matthewmasters</a> <br>\nDo you find a big performance difference between training with 512 v.s. 1024?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 880938,
          "author_name": "Miroslav Valan",
          "author_url": "",
          "post_date": "2020-06-10T16:47:32.407000",
          "content": "<p>i haven't tried bigger than 512</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 881331,
          "author_name": "Blonde",
          "author_url": "",
          "post_date": "2020-06-10T23:44:15.343000",
          "content": "<p>thank you for sharing, \nwhat is your batch size?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 881332,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-06-10T23:44:26.400000",
          "content": "<p><a href=\"/alexlwh\">@alexlwh</a> from my experience, 1024 has always been better than 512. I have also tried ensembling both together but it was worse than 1024 alone</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 881549,
          "author_name": "Miroslav Valan",
          "author_url": "",
          "post_date": "2020-06-11T06:46:30.740000",
          "content": "<p>Larger should be better. However I am relaying on kernels and need to use smaller sizes. </p>\n\n<p>Batch size is 4.</p>\n\n<p>PS: I am not using pseudo labels - yet</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 882068,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-06-11T15:03:00.790000",
          "content": "<p>Relying on kernels as in you do preprocessing with some kernel or you just mean within the neural network?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 877762,
      "author_name": "Ryan Epp",
      "author_url": "",
      "post_date": "2020-06-08T00:11:13.913000",
      "content": "<p>So far I'm at 0.7118 with a single model: Resnet101 + FasterRCNN (Pytorch) + ~60 epochs + a bunch of stock albumentations augmentations + minor data cleaning + minor model tweaks + one fold 80/20 split. No ensembing, no tta, no pseudolabeling, no WBF. </p>",
      "votes": 2,
      "replies": [
        {
          "id": 877764,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-06-08T00:17:02.053000",
          "content": "<p>Well done, I'm sure ensembling and TTA will bring that score up to silver/gold level</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 877930,
          "author_name": "Ryan Epp",
          "author_url": "",
          "post_date": "2020-06-08T05:55:14.913000",
          "content": "<p>Thanks Matthew! 😃  I spent too much time trying to generate extra training images with CycleGAN and it didn't really work out. So I feel like I'm playing a bit of catch up at the moment. </p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 877946,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-06-08T06:23:57.283000",
          "content": "<p>Yeah, I hear you. I've tried many things up to this point that didn't work out. I also tried using CycleGAN and some other adversarial networks to solve the \"domain generalization\" problem but no success.</p>",
          "votes": 3,
          "replies": []
        }
      ]
    },
    {
      "id": 857944,
      "author_name": "Kaushal Shah",
      "author_url": "",
      "post_date": "2020-05-23T04:58:35.847000",
      "content": "<p>My score is quite low:\n0.6624 public LB\nepochs: 5\nimage size: 1024 x 1024\nDetector: Faster RCNN\nbackbone: ResNet101\nAugs: flipping, random hsv, random brightness/contrast</p>\n\n<p>BTW how much time it took to train for 150 epochs?! <a href=\"/matthewmasters\">@matthewmasters</a> . In my case when I trained more than 5 epochs (15 epochs), score decreased because of overfitting (I think)</p>",
      "votes": 2,
      "replies": [
        {
          "id": 857949,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-05-23T05:05:21.267000",
          "content": "<p>Sorry, that was actually a mistake, it was just 50 epochs. I corrected the original post.\nIt still took a long time though. Around 16 hours on my 2070.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 857952,
          "author_name": "Kaushal Shah",
          "author_url": "",
          "post_date": "2020-05-23T05:09:06.253000",
          "content": "<p>Cool! So on kaggle kernels with K80 it will take even longer...</p>",
          "votes": -1,
          "replies": []
        },
        {
          "id": 866947,
          "author_name": "Marcelo Sánchez Ortega",
          "author_url": "",
          "post_date": "2020-05-29T20:13:11.437000",
          "content": "<p>Do you thing this resolution of images is necessary?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 867058,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-05-30T00:58:29.533000",
          "content": "<p>From my results, 1024 was always better. Makes sense as some of the more crowded regions are difficult to discern between plants at lower resolution.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 957748,
      "author_name": "Mark Peng",
      "author_url": "",
      "post_date": "2020-08-04T14:33:01.290000",
      "content": "<p><strong>AlexeyAB/darknet YOLOv4 with pretrained mish-416 weights</strong>\n- Single fold selected from stratified k-folds (K=5)\n- 608x608 resolution for width and height, but using 1024x1024 as input\n- About 12,000 iterations to get best weights\n- lr=0.0005\n- Data augmentation: <code>mosaic=1</code>, <code>blur=1</code>, <code>gaussian_noise=1</code>\n- TTA + WBF\n- Pseudo Labeling for 1,000 iterations\n- Public LB: 0.7352</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 955912,
      "author_name": "DeepUnderstanding",
      "author_url": "",
      "post_date": "2020-08-03T04:36:23.203000",
      "content": "<p>What was the detection threshold you used while inferencing?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 955929,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-08-03T05:12:28.583000",
          "content": "<p>Me? 0.5 for FasterRCNN</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 936777,
      "author_name": "Ankit Maurya",
      "author_url": "",
      "post_date": "2020-07-20T14:19:08.177000",
      "content": "<p>efficientdet-5 512X512<br>\nsingle model <br>\nlb = 0.7504</p>",
      "votes": 0,
      "replies": [
        {
          "id": 937449,
          "author_name": "AnkurSingh",
          "author_url": "",
          "post_date": "2020-07-21T03:09:33.057000",
          "content": "<p>For how many epochs?\n&amp; what was the learning rate?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 938208,
          "author_name": "Ankit Maurya",
          "author_url": "",
          "post_date": "2020-07-21T12:00:08.300000",
          "content": "<p>78-80 epoch<br>\nlr = 0.001 with ex decay</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 938335,
          "author_name": "27Wilson",
          "author_url": "",
          "post_date": "2020-07-21T13:15:25.503000",
          "content": "<p><a href=\"/ajax0564\">@ajax0564</a>  Hello! What is ex decay mean? Is it ExponentialLR?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 938456,
          "author_name": "Ankit Maurya",
          "author_url": "",
          "post_date": "2020-07-21T14:43:39.860000",
          "content": "<p>ExponentialLR?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 938966,
          "author_name": "27Wilson",
          "author_url": "",
          "post_date": "2020-07-21T22:42:28.157000",
          "content": "<p>😂 <a href=\"https://pytorch.org/docs/stable/_modules/torch/optim/lr_scheduler.html#ExponentialLR\">This one</a></p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 911956,
      "author_name": "Bo Peng",
      "author_url": "",
      "post_date": "2020-07-02T06:03:56.417000",
      "content": "<p>Can I ask are you guys using kaggle GPU/TPU kernels, local GPUs, colab pro, vast.ai? </p>\n\n<p>Thanks</p>",
      "votes": 0,
      "replies": [
        {
          "id": 911967,
          "author_name": "Nitin Datta",
          "author_url": "",
          "post_date": "2020-07-02T06:15:03.423000",
          "content": "<p>Hi <a href=\"/bopengiowa\">@bopengiowa</a> \nMost of them use colab or colab pro, very few who train on local and there might be some who train on kaggle but kaggle GPU is mostly used for inference and submission purposes..</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 911968,
          "author_name": "Bo Peng",
          "author_url": "",
          "post_date": "2020-07-02T06:15:59.867000",
          "content": "<p>cool thanks</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 881217,
      "author_name": "Innat",
      "author_url": "",
      "post_date": "2020-06-10T20:30:36.457000",
      "content": "<p><a href=\"/matthewmasters\">@matthewmasters</a> \nI'm trying with mask-rcnn, but I know, maybe it's not the proper choice for this task (due to its instance seg parts). After playing with many configurations param set up, I've reached <a href=\"https://www.kaggle.com/ipythonx/gwd-with-mask-rcnn-and-coco-transfer-learning\">0.6449</a> scores on it. I don't have (still) pure intention to participate seriously in this competition.  But just wondering if anyone has tried it already for this task.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 868000,
      "author_name": "Alex Lau",
      "author_url": "",
      "post_date": "2020-05-30T20:08:13.750000",
      "content": "<p>Thanks for the sharing. \nA few questions about your train setting:\n1. Did you entirely do fine-tuning on the head in the 50 epochs, or you unfreeze all layers in the later stage of the 50 epochs? \n2. Did you use the default pretrained FPN? (i.e. COCO resnet50)</p>",
      "votes": 0,
      "replies": [
        {
          "id": 868116,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-05-31T00:08:41.300000",
          "content": "<ol>\n<li>Not sure exactly what you mean. The head is trained the entire 50 epochs.</li>\n<li>Yes</li>\n</ol>\n\n<p>My model is very close to the default one provided by torchvision</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 868941,
          "author_name": "Alex Lau",
          "author_url": "",
          "post_date": "2020-05-31T15:28:35.737000",
          "content": "<p>sorry for the confusion in Q1. \nI mean did you fine-tune your backbone in the 50 epochs?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 859818,
      "author_name": "Fares FOURATI",
      "author_url": "",
      "post_date": "2020-05-24T19:08:24.870000",
      "content": "<p>For the moment, FasterRCNN, 0.67, with few epochs and augmentations. </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 858914,
      "author_name": "27Wilson",
      "author_url": "",
      "post_date": "2020-05-24T00:00:55.133000",
      "content": "<p>Hello! I'm using fasterrcnn too. And my augmentation is also flipping, random hsv, randombrightnesscontrast, cutout. But it just reach 69.23 LB. So what about custom \"stitching\" augmentation mean? Is it cutmix?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 858922,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-05-24T00:28:39.837000",
          "content": "<p>I don't wish to share my exact augmentation but yes it is similar to cutmix but patches from different images can be mixed together too</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 858956,
          "author_name": "27Wilson",
          "author_url": "",
          "post_date": "2020-05-24T01:41:59.570000",
          "content": "<p>Thanks! It seems that we can do a lot augmentation to improve the performance. Just for beginners like me it's hard. But I'll do my best to write some augmentation function.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 859221,
          "author_name": "Kaushal Shah",
          "author_url": "",
          "post_date": "2020-05-24T08:50:44.493000",
          "content": "<p><a href=\"/whurobin\">@whurobin</a> Which implementation of cutout are you using? And are you augmenting the images on the fly or creating a larger dataset before training? Thanks in advance!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 859290,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-05-24T09:49:55.347000",
          "content": "<p>Question wasn't at me but just my two cents: if your going to use cutout, do it on the fly.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 859465,
          "author_name": "27Wilson",
          "author_url": "",
          "post_date": "2020-05-24T12:54:55.120000",
          "content": "<p>I just use Albu's cutout, and using it during training.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 859589,
          "author_name": "Kaushal Shah",
          "author_url": "",
          "post_date": "2020-05-24T15:07:10.177000",
          "content": "<p>Okay Thanks <a href=\"/matthewmasters\">@matthewmasters</a> and <a href=\"/whurobin\">@whurobin</a> </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 862658,
          "author_name": "Marcelo Sánchez Ortega",
          "author_url": "",
          "post_date": "2020-05-26T18:09:53.063000",
          "content": "<p>how many holes are you using?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 862854,
          "author_name": "27Wilson",
          "author_url": "",
          "post_date": "2020-05-26T21:57:44.353000",
          "content": "<p>I don't know whether my hyperparameter was good or not, I use 8 holes.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 858017,
      "author_name": "Prakhar Rai",
      "author_url": "",
      "post_date": "2020-05-23T06:37:47.850000",
      "content": "<p>Can u tell how long it took to train 50 epochs?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 858018,
          "author_name": "Matt",
          "author_url": "",
          "post_date": "2020-05-23T06:38:48.993000",
          "content": "<p>Yeah I mentioned it in another comment. Around 16 hours on my 2070.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 858022,
          "author_name": "Prakhar Rai",
          "author_url": "",
          "post_date": "2020-05-23T06:41:57.417000",
          "content": "<p>Okk..Thanks!!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 920447,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-07-08T15:50:38.590000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "857938": "I'm curious, what is your best single model? No pseudolabeling, no ensembling.\n\nMine:\n0.7088 LB\nEpochs: 50\nDataset: 1024^2 images\nDetector: Faster RCNN\nBackbone: resnet50\nAugmentation: flipping, random hsv, random brightness/contrast, custom \"stitching\" augmentation\n",
    "921815": "YOLOV5 out.....\n\n1. EffDet5\n2. 1024 as input\n3. single fold - 80/20 split\n4. with some training tricks\n (Gradient Accumulation for bigger batch size, nvidia apex, personal implementation of data augmentation)\n5. epochs: 60\nLB: 0.7516 (with TTA)",
    "919612": "EffDet5: 0.7508(with pseudo labels)\nYolov5: out😂 .",
    "881426": "[Updated]\n1.  EffDet5\n2. 1024 as input\n3. single fold - 80/20 split\n4. with some training tricks(Gradient Accumulation for bigger batch size, nvidia apex) \n5. epochs: 60 \n\nLB: 0.7487(with TTA)\n",
    "858185": "I think my score is overfitted. I got 0.7265 for a single FasterRCNN model 2 weeks ago. Then I tried many better ideas, but none of them can reach 0.72.",
    "902238": "[update]\nEffDet5\n512 input\nsingle fold\n40 epochs  --&gt;  50 epochs\n0.7393 --&gt; 0.7441 with TTA (no pseudo labels)",
    "920470": "Effdet5 512 0.7393 no pseudo no tta\nYolov5 😨😨😨",
    "912549": "EffDet4\n768 input\nsingle fold\n30 epochs\n0.7267\nAbout 8 hours on a local 2080ti FP32",
    "891853": "Hi @matthewmasters, I am trying Effdet-5 on 1024 but I am running out of memory on Colab and 2070...\nHow are you dealing with this?? Any suggestions??",
    "879220": "Single fcos dcnv2 X 101 FPN model with augmentation got 0.7322. ",
    "858201": "@matthewmasters your current LB (0.7222) score is from a single model too (one fold, no tta, no ensemble, etc)?",
    "857947": "Do  you  have  a  try  about   CascadeRcnn ?",
    "891732": "[Update]\nyolov5x\nLB: 0.7526\nhttps://www.kaggle.com/orkatz2/yolov5-fake-or-real-single-model-l-b-0-73",
    "880365": "EffDet5, \n512 input, \nsingle fold -  80/20 split  \n0.7354",
    "877762": "So far I'm at 0.7118 with a single model: Resnet101 + FasterRCNN (Pytorch) + ~60 epochs + a bunch of stock albumentations augmentations + minor data cleaning + minor model tweaks + one fold 80/20 split. No ensembing, no tta, no pseudolabeling, no WBF. ",
    "857944": "My score is quite low:\n0.6624 public LB\nepochs: 5\nimage size: 1024 x 1024\nDetector: Faster RCNN\nbackbone: ResNet101\nAugs: flipping, random hsv, random brightness/contrast\n\nBTW how much time it took to train for 150 epochs?! @matthewmasters . In my case when I trained more than 5 epochs (15 epochs), score decreased because of overfitting (I think)",
    "957748": "**AlexeyAB/darknet YOLOv4 with pretrained mish-416 weights**\n- Single fold selected from stratified k-folds (K=5)\n- 608x608 resolution for width and height, but using 1024x1024 as input\n- About 12,000 iterations to get best weights\n- lr=0.0005\n- Data augmentation: `mosaic=1`, `blur=1`, `gaussian_noise=1`\n- TTA + WBF\n- Pseudo Labeling for 1,000 iterations\n- Public LB: 0.7352\n",
    "955912": "What was the detection threshold you used while inferencing?",
    "936777": "efficientdet-5 512X512\nsingle model \nlb = 0.7504",
    "911956": "Can I ask are you guys using kaggle GPU/TPU kernels, local GPUs, colab pro, vast.ai? \n\nThanks",
    "881217": "@matthewmasters \nI'm trying with mask-rcnn, but I know, maybe it's not the proper choice for this task (due to its instance seg parts). After playing with many configurations param set up, I've reached [0.6449](https://www.kaggle.com/ipythonx/gwd-with-mask-rcnn-and-coco-transfer-learning) scores on it. I don't have (still) pure intention to participate seriously in this competition.  But just wondering if anyone has tried it already for this task.",
    "868000": "Thanks for the sharing. \nA few questions about your train setting:\n1. Did you entirely do fine-tuning on the head in the 50 epochs, or you unfreeze all layers in the later stage of the 50 epochs? \n2. Did you use the default pretrained FPN? (i.e. COCO resnet50)",
    "859818": "For the moment, FasterRCNN, 0.67, with few epochs and augmentations. ",
    "858914": "Hello! I'm using fasterrcnn too. And my augmentation is also flipping, random hsv, randombrightnesscontrast, cutout. But it just reach 69.23 LB. So what about custom \"stitching\" augmentation mean? Is it cutmix?",
    "858017": "Can u tell how long it took to train 50 epochs?",
    "920447": ""
  }
}