{
  "id": 94102,
  "title": "3rd place solution",
  "url": "/competitions/cassava-disease/writeups/appian-3rd-place-solution",
  "author_name": "",
  "post_date": "2019-06-02T10:15:51.300Z",
  "votes": 24,
  "comment_count": 15,
  "views": 0,
  "content": "<h3>Train on train images</h3>\n\n<p>First I trained 5-folds se_resnext50_32x4d using train images. All the models were trained on kaggle kernels for a few hours.</p>\n\n<ul>\n<li>lr: 1e-4 </li>\n<li>batch size: 20</li>\n<li>input size: 500x500x3</li>\n<li>augmentations: random crop, random erasing, hflip, vflip, random affine, </li>\n<li>tta: 10 times</li>\n</ul>\n\n<p>cv: 0.9204 | 0.9257 | 0.9213 | 0.9062 | 0.9179</p>\n\n<p>By averaging 5-folds, it scored <strong>0.92052</strong> on public lb and <strong>0.93507</strong> on private lb.</p>\n\n<h3>Train on pseudo extra/test images and train images</h3>\n\n<p>I used previous 5-folds models to provide pseudo labels on extra/test images.</p>\n\n<p>This model alone achieved <strong>0.9158</strong> on all of train images as evaluation data. I used this model to train further with 5-folds train images.</p>\n\n<p>cv: 0.9399 | 0.9346 | 0.9302 | 0.9159 | 0.9285</p>\n\n<p>This cv is a little bit more promising than previous 5-folds but lb score remained almost the same. </p>\n\n<p>If you have any questions, I will try to answer. </p>",
  "messages": [
    {
      "id": "541224",
      "postDate": "06/02/2019 03:49:12",
      "content": "<h3>Train on train images</h3>\n\n<p>First I trained 5-folds se_resnext50_32x4d using train images. All the models were trained on kaggle kernels for a few hours.</p>\n\n<ul>\n<li>lr: 1e-4 </li>\n<li>batch size: 20</li>\n<li>input size: 500x500x3</li>\n<li>augmentations: random crop, random erasing, hflip, vflip, random affine, </li>\n<li>tta: 10 times</li>\n</ul>\n\n<p>cv: 0.9204 | 0.9257 | 0.9213 | 0.9062 | 0.9179</p>\n\n<p>By averaging 5-folds, it scored <strong>0.92052</strong> on public lb and <strong>0.93507</strong> on private lb.</p>\n\n<h3>Train on pseudo extra/test images and train images</h3>\n\n<p>I used previous 5-folds models to provide pseudo labels on extra/test images.</p>\n\n<p>This model alone achieved <strong>0.9158</strong> on all of train images as evaluation data. I used this model to train further with 5-folds train images.</p>\n\n<p>cv: 0.9399 | 0.9346 | 0.9302 | 0.9159 | 0.9285</p>\n\n<p>This cv is a little bit more promising than previous 5-folds but lb score remained almost the same. </p>\n\n<p>If you have any questions, I will try to answer. </p>",
      "rawMarkdown": "### Train on train images\n\nFirst I trained 5-folds se\\_resnext50\\_32x4d using train images. All the models were trained on kaggle kernels for a few hours.\n\n- lr: 1e-4 \n- batch size: 20\n- input size: 500x500x3\n- augmentations: random crop, random erasing, hflip, vflip, random affine, \n- tta: 10 times\n\ncv: 0.9204 | 0.9257 | 0.9213 | 0.9062 | 0.9179\n\nBy averaging 5-folds, it scored **0.92052** on public lb and **0.93507** on private lb.\n\n\n### Train on pseudo extra/test images and train images\n\nI used previous 5-folds models to provide pseudo labels on extra/test images.\n\nThis model alone achieved **0.9158** on all of train images as evaluation data. I used this model to train further with 5-folds train images.\n\ncv: 0.9399 | 0.9346 | 0.9302 | 0.9159 | 0.9285\n\nThis cv is a little bit more promising than previous 5-folds but lb score remained almost the same. \n\nIf you have any questions, I will try to answer.",
      "votes": null
    },
    {
      "id": "541256",
      "postDate": "06/02/2019 05:27:27",
      "content": "<p>Thanks for your solution. Why did you choose se_resnext50_32x4d ? Is there any specific reason ? Is it better than the Densenet / Resnet models ? Is there anything which did not work for you ?</p>\n\n<p>Another request : Would you like to share the code in whatever form it might be so that newbies like me can learn from it ?</p>",
      "rawMarkdown": "Thanks for your solution. Why did you choose se_resnext50_32x4d ? Is there any specific reason ? Is it better than the Densenet / Resnet models ? Is there anything which did not work for you ?\n\nAnother request : Would you like to share the code in whatever form it might be so that newbies like me can learn from it ?",
      "votes": null
    },
    {
      "id": "541366",
      "postDate": "06/02/2019 10:10:12",
      "content": "<p>I normally choose resnet50 and se_resnext50 as the next model to see what difference it makes. If se_resnext50 improves cv/lb then I try a larger model such as se_resnext101. In case of this competition I could not see noticeble difference between se_resnext50 and se_resnext101 and chose the former as a main architecture.\nThe reason for this choices are based on the report described in <a href=\"https://arxiv.org/pdf/1810.00736.pdf\">https://arxiv.org/pdf/1810.00736.pdf</a> . se_resnext family reports relatively good accuracy considering relatively low computational complexity.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/541366/13356/figure1.png\" alt=\"\"></p>",
      "rawMarkdown": "I normally choose resnet50 and se\\_resnext50 as the next model to see what difference it makes. If se\\_resnext50 improves cv/lb then I try a larger model such as se\\_resnext101. In case of this competition I could not see noticeble difference between se\\_resnext50 and se\\_resnext101 and chose the former as a main architecture.\nThe reason for this choices are based on the report described in https://arxiv.org/pdf/1810.00736.pdf . se\\_resnext family reports relatively good accuracy considering relatively low computational complexity.\n\n![](https://storage.googleapis.com/kaggle-forum-message-attachments/541366/13356/figure1.png)",
      "votes": null
    },
    {
      "id": "541397",
      "postDate": "06/02/2019 11:33:57",
      "content": "<p>Thanks for the detailed answer for choosing the model architecture. Very informative.</p>",
      "rawMarkdown": "Thanks for the detailed answer for choosing the model architecture. Very informative.",
      "votes": null
    },
    {
      "id": "544056",
      "postDate": "06/05/2019 04:39:29",
      "content": "<p><strong>One more question : How many epochs did you use ?</strong></p>\n\n<p>I tried replicating the results in \n<a href=\"https://www.kaggle.com/ambarish/icassava-2019-cadene-models\">https://www.kaggle.com/ambarish/icassava-2019-cadene-models</a></p>\n\n<p>fastai\nwith 80 epochs\nsame model se_resnext50_32x4d\nlearning rate varying from 3e-3,3e-2\ninput size 500\ntta 8\nbatch 20</p>\n\n<p>The results were better than my previous results with private 0.91077\nThank you for the tips .</p>\n\n<p>This is nowhere close to your 3rd place solution.</p>\n\n<p>Your learning rate is much lower 1e-4 but that would mean increasing epochs.</p>\n\n<p>I am using fastai which is adding some custom layers at the head of the model.</p>\n\n<p>Is your architecture using any custom layers ?  I am surely missing something ? Any suggestions/enlightment would help? Any code snippet would also help ?</p>\n\n<p>Cheers,\nBukun</p>",
      "rawMarkdown": "**One more question : How many epochs did you use ?**\n\nI tried replicating the results in \nhttps://www.kaggle.com/ambarish/icassava-2019-cadene-models\n\nfastai\nwith 80 epochs\nsame model se_resnext50_32x4d\nlearning rate varying from 3e-3,3e-2\ninput size 500\ntta 8\nbatch 20\n\n\nThe results were better than my previous results with private 0.91077\nThank you for the tips .\n\nThis is nowhere close to your 3rd place solution.\n\nYour learning rate is much lower 1e-4 but that would mean increasing epochs.\n\nI am using fastai which is adding some custom layers at the head of the model.\n\nIs your architecture using any custom layers ?  I am surely missing something ? Any suggestions/enlightment would help? Any code snippet would also help ?\n\nCheers,\nBukun",
      "votes": null
    },
    {
      "id": "547183",
      "postDate": "06/07/2019 11:28:25",
      "content": "<ul>\n<li>It only took 8-15 epochs to reach the best cv. </li>\n<li>I used Adam and 1e-4 was not that low for Adam. What optimizer are you using?</li>\n<li>I didn't use any custom layers but trained only dense layer for first 2 epochs with smaller lr.</li>\n</ul>",
      "rawMarkdown": "It only took 8-15 epochs to reach the best cv. \n- I used Adam and 1e-4 was not that low for Adam. What optimizer are you using?\n- I didn't use any custom layers but trained only dense layer for first 2 epochs with smaller lr.",
      "votes": null
    },
    {
      "id": "547709",
      "postDate": "06/08/2019 06:38:14",
      "content": "<p>Thanks for your answer and for being very patient with my incessant questions.</p>\n\n<p>I tried to build a PyTorch model <a href=\"https://www.kaggle.com/ambarish/icassava-2019-cadene-models-2\">https://www.kaggle.com/ambarish/icassava-2019-cadene-models-2</a>\nfrom scratch using\n- Adam\n- same model seresnext5032x4d\n-  First 2 epoch 8e-5 and from the 3rd epoch to 15 epochs 1e-4\n- Changed only the last 2 layers\n- 15 epochs\n<strong>- Accuracy score = .8</strong>\n- Training only the last 2 layers since it was changed\n- image size 500\n- batch size </p>\n\n<p>I am surely doing something wrong.</p>\n\n<p>I tried training the model as a whole but it ran out of memory. Hence training only the changed layers.</p>\n\n<p>Looking forward for your suggestions/comments</p>",
      "rawMarkdown": "Thanks for your answer and for being very patient with my incessant questions.\n\nI tried to build a PyTorch model https://www.kaggle.com/ambarish/icassava-2019-cadene-models-2\nfrom scratch using\n- Adam\n- same model seresnext5032x4d\n-  First 2 epoch 8e-5 and from the 3rd epoch to 15 epochs 1e-4\n- Changed only the last 2 layers\n- 15 epochs\n**- Accuracy score = .8**\n- Training only the last 2 layers since it was changed\n- image size 500\n- batch size \n\nI am surely doing something wrong.\n\nI tried training the model as a whole but it ran out of memory. Hence training only the changed layers.\n\nLooking forward for your suggestions/comments",
      "votes": null
    },
    {
      "id": "549151",
      "postDate": "06/10/2019 11:04:21",
      "content": "<p>Thanks for details. Do you use pytorch? I don't see random erasing transform anywhere.</p>",
      "rawMarkdown": "Thanks for details. Do you use pytorch? I don't see random erasing transform anywhere.",
      "votes": null
    },
    {
      "id": "550658",
      "postDate": "06/11/2019 22:15:31",
      "content": "<p>Hello, can you please post your email address here? For some reason Kaggle isn't allowing me to contact you. I wanted to send you instructions for presenting your work, prizes etc. Thanks and great job.</p>",
      "rawMarkdown": "Hello, can you please post your email address here? For some reason Kaggle isn't allowing me to contact you. I wanted to send you instructions for presenting your work, prizes etc. Thanks and great job.",
      "votes": null
    },
    {
      "id": "551421",
      "postDate": "06/12/2019 17:28:32",
      "content": "<p>I think entire layers should be trained after training the dense layer. If it runs out of memory, you can try reducing batch size or input image size. </p>\n\n<p>This is the augmentations I used just in case it might help improving. You can skip randomerasing when you do inference.</p>\n\n<p>from torchvision import transforms as T</p>\n\n<p><code>\ndef train_transform(size):\n    return T.Compose([\n        T.RandomApply([T.RandomAffine(45, shear=15)], 0.8),\n        RandomResizedCropV2(size, scale=(0.6, 1.0), ratio=(3/5, 5/3)),\n        T.RandomHorizontalFlip(),\n        T.RandomVerticalFlip(),\n        T.ToTensor(),\n        T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),\n        RandomErasing(probability=0.3, sh=0.3),\n    ])\n</code></p>",
      "rawMarkdown": "I think entire layers should be trained after training the dense layer. If it runs out of memory, you can try reducing batch size or input image size. \n\nThis is the augmentations I used just in case it might help improving. You can skip randomerasing when you do inference.\n\nfrom torchvision import transforms as T\n\n```\ndef train_transform(size):\n    return T.Compose([\n        T.RandomApply([T.RandomAffine(45, shear=15)], 0.8),\n        RandomResizedCropV2(size, scale=(0.6, 1.0), ratio=(3/5, 5/3)),\n        T.RandomHorizontalFlip(),\n        T.RandomVerticalFlip(),\n        T.ToTensor(),\n        T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),\n        RandomErasing(probability=0.3, sh=0.3),\n    ])\n```",
      "votes": null
    },
    {
      "id": "551427",
      "postDate": "06/12/2019 17:37:26",
      "content": "<p><a href=\"/tgebru123\">@tgebru123</a> \nI sent you an message. My email address should be there. \nThanks.</p>",
      "rawMarkdown": "tgebru123 \nI sent you an message. My email address should be there. \nThanks.",
      "votes": null
    },
    {
      "id": "551434",
      "postDate": "06/12/2019 17:50:11",
      "content": "<p>I have responded to your email yesterday and sent another one today--I'm assuming thats you. Please check and let me know if you haven't received it. </p>",
      "rawMarkdown": "I have responded to your email yesterday and sent another one today--I'm assuming thats you. Please check and let me know if you haven't received it.",
      "votes": null
    },
    {
      "id": "551441",
      "postDate": "06/12/2019 18:03:23",
      "content": "<p>That's strange. </p>\n\n<p>Let me confirm. \n- What I sent was a message sent via kaggle.\n- I sent just a half hour ago (might be delayed because it's relayed by kaggle. I have no idea).\n- This was the first time I sent to you.\n- I haven't received anything from you yet.</p>",
      "rawMarkdown": "That's strange. \n\nLet me confirm. \n- What I sent was a message sent via kaggle.\n- I sent just a half hour ago (might be delayed because it's relayed by kaggle. I have no idea).\n- This was the first time I sent to you.\n- I haven't received anything from you yet.",
      "votes": null
    },
    {
      "id": "552445",
      "postDate": "06/14/2019 01:55:56",
      "content": "<p>I just sent it through Kaggle. Did you receive it?</p>",
      "rawMarkdown": "I just sent it through Kaggle. Did you receive it?",
      "votes": null
    },
    {
      "id": "552693",
      "postDate": "06/14/2019 10:57:12",
      "content": "<p>Yes, </p>",
      "rawMarkdown": "Yes,",
      "votes": null
    },
    {
      "id": "553162",
      "postDate": "06/15/2019 08:14:49",
      "content": "<p>I replied. Let me know if you have not received just in case.</p>",
      "rawMarkdown": "I replied. Let me know if you have not received just in case.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 541256,
      "author_name": "ambarish",
      "author_url": "",
      "post_date": "06/02/2019 05:27:27",
      "content": "<p>Thanks for your solution. Why did you choose se_resnext50_32x4d ? Is there any specific reason ? Is it better than the Densenet / Resnet models ? Is there anything which did not work for you ?</p>\n\n<p>Another request : Would you like to share the code in whatever form it might be so that newbies like me can learn from it ?</p>",
      "votes": null,
      "replies": [
        {
          "id": 541366,
          "author_name": "appian",
          "author_url": "",
          "post_date": "06/02/2019 10:10:12",
          "content": "<p>I normally choose resnet50 and se_resnext50 as the next model to see what difference it makes. If se_resnext50 improves cv/lb then I try a larger model such as se_resnext101. In case of this competition I could not see noticeble difference between se_resnext50 and se_resnext101 and chose the former as a main architecture.\nThe reason for this choices are based on the report described in <a href=\"https://arxiv.org/pdf/1810.00736.pdf\">https://arxiv.org/pdf/1810.00736.pdf</a> . se_resnext family reports relatively good accuracy considering relatively low computational complexity.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/541366/13356/figure1.png\" alt=\"\"></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 541397,
          "author_name": "ambarish",
          "author_url": "",
          "post_date": "06/02/2019 11:33:57",
          "content": "<p>Thanks for the detailed answer for choosing the model architecture. Very informative.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 544056,
      "author_name": "ambarish",
      "author_url": "",
      "post_date": "06/05/2019 04:39:29",
      "content": "<p><strong>One more question : How many epochs did you use ?</strong></p>\n\n<p>I tried replicating the results in \n<a href=\"https://www.kaggle.com/ambarish/icassava-2019-cadene-models\">https://www.kaggle.com/ambarish/icassava-2019-cadene-models</a></p>\n\n<p>fastai\nwith 80 epochs\nsame model se_resnext50_32x4d\nlearning rate varying from 3e-3,3e-2\ninput size 500\ntta 8\nbatch 20</p>\n\n<p>The results were better than my previous results with private 0.91077\nThank you for the tips .</p>\n\n<p>This is nowhere close to your 3rd place solution.</p>\n\n<p>Your learning rate is much lower 1e-4 but that would mean increasing epochs.</p>\n\n<p>I am using fastai which is adding some custom layers at the head of the model.</p>\n\n<p>Is your architecture using any custom layers ?  I am surely missing something ? Any suggestions/enlightment would help? Any code snippet would also help ?</p>\n\n<p>Cheers,\nBukun</p>",
      "votes": null,
      "replies": [
        {
          "id": 547183,
          "author_name": "appian",
          "author_url": "",
          "post_date": "06/07/2019 11:28:25",
          "content": "<ul>\n<li>It only took 8-15 epochs to reach the best cv. </li>\n<li>I used Adam and 1e-4 was not that low for Adam. What optimizer are you using?</li>\n<li>I didn't use any custom layers but trained only dense layer for first 2 epochs with smaller lr.</li>\n</ul>",
          "votes": null,
          "replies": []
        },
        {
          "id": 547709,
          "author_name": "ambarish",
          "author_url": "",
          "post_date": "06/08/2019 06:38:14",
          "content": "<p>Thanks for your answer and for being very patient with my incessant questions.</p>\n\n<p>I tried to build a PyTorch model <a href=\"https://www.kaggle.com/ambarish/icassava-2019-cadene-models-2\">https://www.kaggle.com/ambarish/icassava-2019-cadene-models-2</a>\nfrom scratch using\n- Adam\n- same model seresnext5032x4d\n-  First 2 epoch 8e-5 and from the 3rd epoch to 15 epochs 1e-4\n- Changed only the last 2 layers\n- 15 epochs\n<strong>- Accuracy score = .8</strong>\n- Training only the last 2 layers since it was changed\n- image size 500\n- batch size </p>\n\n<p>I am surely doing something wrong.</p>\n\n<p>I tried training the model as a whole but it ran out of memory. Hence training only the changed layers.</p>\n\n<p>Looking forward for your suggestions/comments</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 550658,
          "author_name": "tgebru123",
          "author_url": "",
          "post_date": "06/11/2019 22:15:31",
          "content": "<p>Hello, can you please post your email address here? For some reason Kaggle isn't allowing me to contact you. I wanted to send you instructions for presenting your work, prizes etc. Thanks and great job.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 551421,
          "author_name": "appian",
          "author_url": "",
          "post_date": "06/12/2019 17:28:32",
          "content": "<p>I think entire layers should be trained after training the dense layer. If it runs out of memory, you can try reducing batch size or input image size. </p>\n\n<p>This is the augmentations I used just in case it might help improving. You can skip randomerasing when you do inference.</p>\n\n<p>from torchvision import transforms as T</p>\n\n<p><code>\ndef train_transform(size):\n    return T.Compose([\n        T.RandomApply([T.RandomAffine(45, shear=15)], 0.8),\n        RandomResizedCropV2(size, scale=(0.6, 1.0), ratio=(3/5, 5/3)),\n        T.RandomHorizontalFlip(),\n        T.RandomVerticalFlip(),\n        T.ToTensor(),\n        T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),\n        RandomErasing(probability=0.3, sh=0.3),\n    ])\n</code></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 551427,
          "author_name": "appian",
          "author_url": "",
          "post_date": "06/12/2019 17:37:26",
          "content": "<p><a href=\"/tgebru123\">@tgebru123</a> \nI sent you an message. My email address should be there. \nThanks.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 551434,
          "author_name": "tgebru123",
          "author_url": "",
          "post_date": "06/12/2019 17:50:11",
          "content": "<p>I have responded to your email yesterday and sent another one today--I'm assuming thats you. Please check and let me know if you haven't received it. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 551441,
          "author_name": "appian",
          "author_url": "",
          "post_date": "06/12/2019 18:03:23",
          "content": "<p>That's strange. </p>\n\n<p>Let me confirm. \n- What I sent was a message sent via kaggle.\n- I sent just a half hour ago (might be delayed because it's relayed by kaggle. I have no idea).\n- This was the first time I sent to you.\n- I haven't received anything from you yet.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 552445,
          "author_name": "tgebru123",
          "author_url": "",
          "post_date": "06/14/2019 01:55:56",
          "content": "<p>I just sent it through Kaggle. Did you receive it?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 552693,
          "author_name": "appian",
          "author_url": "",
          "post_date": "06/14/2019 10:57:12",
          "content": "<p>Yes, </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 553162,
          "author_name": "appian",
          "author_url": "",
          "post_date": "06/15/2019 08:14:49",
          "content": "<p>I replied. Let me know if you have not received just in case.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 549151,
      "author_name": "liberifatali",
      "author_url": "",
      "post_date": "06/10/2019 11:04:21",
      "content": "<p>Thanks for details. Do you use pytorch? I don't see random erasing transform anywhere.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "541224": "### Train on train images\n\nFirst I trained 5-folds se\\_resnext50\\_32x4d using train images. All the models were trained on kaggle kernels for a few hours.\n\n- lr: 1e-4 \n- batch size: 20\n- input size: 500x500x3\n- augmentations: random crop, random erasing, hflip, vflip, random affine, \n- tta: 10 times\n\ncv: 0.9204 | 0.9257 | 0.9213 | 0.9062 | 0.9179\n\nBy averaging 5-folds, it scored **0.92052** on public lb and **0.93507** on private lb.\n\n\n### Train on pseudo extra/test images and train images\n\nI used previous 5-folds models to provide pseudo labels on extra/test images.\n\nThis model alone achieved **0.9158** on all of train images as evaluation data. I used this model to train further with 5-folds train images.\n\ncv: 0.9399 | 0.9346 | 0.9302 | 0.9159 | 0.9285\n\nThis cv is a little bit more promising than previous 5-folds but lb score remained almost the same. \n\nIf you have any questions, I will try to answer.",
    "541256": "Thanks for your solution. Why did you choose se_resnext50_32x4d ? Is there any specific reason ? Is it better than the Densenet / Resnet models ? Is there anything which did not work for you ?\n\nAnother request : Would you like to share the code in whatever form it might be so that newbies like me can learn from it ?",
    "541366": "I normally choose resnet50 and se\\_resnext50 as the next model to see what difference it makes. If se\\_resnext50 improves cv/lb then I try a larger model such as se\\_resnext101. In case of this competition I could not see noticeble difference between se\\_resnext50 and se\\_resnext101 and chose the former as a main architecture.\nThe reason for this choices are based on the report described in https://arxiv.org/pdf/1810.00736.pdf . se\\_resnext family reports relatively good accuracy considering relatively low computational complexity.\n\n![](https://storage.googleapis.com/kaggle-forum-message-attachments/541366/13356/figure1.png)",
    "541397": "Thanks for the detailed answer for choosing the model architecture. Very informative.",
    "544056": "**One more question : How many epochs did you use ?**\n\nI tried replicating the results in \nhttps://www.kaggle.com/ambarish/icassava-2019-cadene-models\n\nfastai\nwith 80 epochs\nsame model se_resnext50_32x4d\nlearning rate varying from 3e-3,3e-2\ninput size 500\ntta 8\nbatch 20\n\n\nThe results were better than my previous results with private 0.91077\nThank you for the tips .\n\nThis is nowhere close to your 3rd place solution.\n\nYour learning rate is much lower 1e-4 but that would mean increasing epochs.\n\nI am using fastai which is adding some custom layers at the head of the model.\n\nIs your architecture using any custom layers ?  I am surely missing something ? Any suggestions/enlightment would help? Any code snippet would also help ?\n\nCheers,\nBukun",
    "547183": "It only took 8-15 epochs to reach the best cv. \n- I used Adam and 1e-4 was not that low for Adam. What optimizer are you using?\n- I didn't use any custom layers but trained only dense layer for first 2 epochs with smaller lr.",
    "547709": "Thanks for your answer and for being very patient with my incessant questions.\n\nI tried to build a PyTorch model https://www.kaggle.com/ambarish/icassava-2019-cadene-models-2\nfrom scratch using\n- Adam\n- same model seresnext5032x4d\n-  First 2 epoch 8e-5 and from the 3rd epoch to 15 epochs 1e-4\n- Changed only the last 2 layers\n- 15 epochs\n**- Accuracy score = .8**\n- Training only the last 2 layers since it was changed\n- image size 500\n- batch size \n\nI am surely doing something wrong.\n\nI tried training the model as a whole but it ran out of memory. Hence training only the changed layers.\n\nLooking forward for your suggestions/comments",
    "549151": "Thanks for details. Do you use pytorch? I don't see random erasing transform anywhere.",
    "550658": "Hello, can you please post your email address here? For some reason Kaggle isn't allowing me to contact you. I wanted to send you instructions for presenting your work, prizes etc. Thanks and great job.",
    "551421": "I think entire layers should be trained after training the dense layer. If it runs out of memory, you can try reducing batch size or input image size. \n\nThis is the augmentations I used just in case it might help improving. You can skip randomerasing when you do inference.\n\nfrom torchvision import transforms as T\n\n```\ndef train_transform(size):\n    return T.Compose([\n        T.RandomApply([T.RandomAffine(45, shear=15)], 0.8),\n        RandomResizedCropV2(size, scale=(0.6, 1.0), ratio=(3/5, 5/3)),\n        T.RandomHorizontalFlip(),\n        T.RandomVerticalFlip(),\n        T.ToTensor(),\n        T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),\n        RandomErasing(probability=0.3, sh=0.3),\n    ])\n```",
    "551427": "tgebru123 \nI sent you an message. My email address should be there. \nThanks.",
    "551434": "I have responded to your email yesterday and sent another one today--I'm assuming thats you. Please check and let me know if you haven't received it.",
    "551441": "That's strange. \n\nLet me confirm. \n- What I sent was a message sent via kaggle.\n- I sent just a half hour ago (might be delayed because it's relayed by kaggle. I have no idea).\n- This was the first time I sent to you.\n- I haven't received anything from you yet.",
    "552445": "I just sent it through Kaggle. Did you receive it?",
    "552693": "Yes,",
    "553162": "I replied. Let me know if you have not received just in case."
  },
  "source": "meta"
}