{
  "id": 141805,
  "title": "2nd place solution",
  "url": "/competitions/evohackaton/writeups/ods-ai-powerrangers-2nd-place-solution",
  "author_name": "",
  "post_date": "2020-04-07T15:02:35.395004500Z",
  "votes": 5,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Big thanks <a href=\"http://best-kyiv.org/\">BEST</a> for organizing such an outstanding event. It was a great experience for everyone.\nThanks for all my team members (@yaroslav4klymchuk , @vladyelisieiev, @zekamrozek )</p>\n\n<h1>Solution</h1>\n\n<h2>Initial thougths about data</h2>\n\n<p>Data was mostly very clean with only small noise in labeling, which resulted in ZERO shake-up in LB\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1690820%2F728aae9f9361a761139f2756ff8a87b5%2Fnoshakeup.png?generation=1586270843647464&amp;alt=media\" alt=\"\">\nSo we did not do any pre-processing </p>\n\n<h2>CV policy</h2>\n\n<p>We used simple StratifiedKFold for Cross-Validation and got a very good correlation with public score</p>\n\n<h2>Modeling</h2>\n\n<p>We tried several models:\n1. Started from resnet34 to make some basic experiments\n2. Switched to SE-ResNeXt50_32x4d and trained it only for 3 folds (because of Disk out of space :) )\n3. Also tried mobilenet_v2 but it was to shallow for our task </p>\n\n<h2>Augmentation</h2>\n\n<p>We used a pack of augmentations from <a href=\"https://github.com/qubvel/segmentation_models.pytorch/blob/master/examples/cars%20segmentation%20%28camvid%29.ipynb\">segmentation models pytorch</a> and , for really, did not tune this stuff.</p>\n\n<h2>Optimization</h2>\n\n<p>We used simple Adam with bigger LR for classifier and ReduceLROnPlateau as a scheduler. As loss function we used simple CE loss.</p>\n\n<h2>Post-processing</h2>\n\n<p>We used flips TTA for creating final submissions and blend models from different folds. Also we used  <a href=\"https://pytorch.org/blog/stochastic-weight-averaging-in-pytorch/\">SWA</a> for blending checkpoints from different epochs.</p>\n\n<p>Thanks again for all participants and congratulations to winners!!!</p>",
  "messages": [
    {
      "id": "800597",
      "postDate": "04/07/2020 15:02:35",
      "content": "<p>Big thanks <a href=\"http://best-kyiv.org/\">BEST</a> for organizing such an outstanding event. It was a great experience for everyone.\nThanks for all my team members (@yaroslav4klymchuk , @vladyelisieiev, @zekamrozek )</p>\n\n<h1>Solution</h1>\n\n<h2>Initial thougths about data</h2>\n\n<p>Data was mostly very clean with only small noise in labeling, which resulted in ZERO shake-up in LB\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1690820%2F728aae9f9361a761139f2756ff8a87b5%2Fnoshakeup.png?generation=1586270843647464&amp;alt=media\" alt=\"\">\nSo we did not do any pre-processing </p>\n\n<h2>CV policy</h2>\n\n<p>We used simple StratifiedKFold for Cross-Validation and got a very good correlation with public score</p>\n\n<h2>Modeling</h2>\n\n<p>We tried several models:\n1. Started from resnet34 to make some basic experiments\n2. Switched to SE-ResNeXt50_32x4d and trained it only for 3 folds (because of Disk out of space :) )\n3. Also tried mobilenet_v2 but it was to shallow for our task </p>\n\n<h2>Augmentation</h2>\n\n<p>We used a pack of augmentations from <a href=\"https://github.com/qubvel/segmentation_models.pytorch/blob/master/examples/cars%20segmentation%20%28camvid%29.ipynb\">segmentation models pytorch</a> and , for really, did not tune this stuff.</p>\n\n<h2>Optimization</h2>\n\n<p>We used simple Adam with bigger LR for classifier and ReduceLROnPlateau as a scheduler. As loss function we used simple CE loss.</p>\n\n<h2>Post-processing</h2>\n\n<p>We used flips TTA for creating final submissions and blend models from different folds. Also we used  <a href=\"https://pytorch.org/blog/stochastic-weight-averaging-in-pytorch/\">SWA</a> for blending checkpoints from different epochs.</p>\n\n<p>Thanks again for all participants and congratulations to winners!!!</p>",
      "rawMarkdown": "Big thanks [BEST](http://best-kyiv.org/) for organizing such an outstanding event. It was a great experience for everyone.\nThanks for all my team members (@yaroslav4klymchuk , @vladyelisieiev, @zekamrozek )\n\n# Solution\n## Initial thougths about data \nData was mostly very clean with only small noise in labeling, which resulted in ZERO shake-up in LB\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1690820%2F728aae9f9361a761139f2756ff8a87b5%2Fnoshakeup.png?generation=1586270843647464&amp;alt=media)\nSo we did not do any pre-processing \n\n## CV policy\nWe used simple StratifiedKFold for Cross-Validation and got a very good correlation with public score\n\n## Modeling\nWe tried several models:\n1. Started from resnet34 to make some basic experiments\n2. Switched to SE-ResNeXt50_32x4d and trained it only for 3 folds (because of Disk out of space :) )\n3. Also tried mobilenet_v2 but it was to shallow for our task \n\n## Augmentation\nWe used a pack of augmentations from [segmentation models pytorch](https://github.com/qubvel/segmentation_models.pytorch/blob/master/examples/cars%20segmentation%20(camvid).ipynb) and , for really, did not tune this stuff.\n\n## Optimization\nWe used simple Adam with bigger LR for classifier and ReduceLROnPlateau as a scheduler. As loss function we used simple CE loss.\n\n## Post-processing\nWe used flips TTA for creating final submissions and blend models from different folds. Also we used  [SWA](https://pytorch.org/blog/stochastic-weight-averaging-in-pytorch/) for blending checkpoints from different epochs.\n\nThanks again for all participants and congratulations to winners!!!",
      "votes": null
    },
    {
      "id": "802756",
      "postDate": "04/09/2020 19:41:33",
      "content": "<p>Thanks and congratulations. Excellent approach!</p>",
      "rawMarkdown": "Thanks and congratulations. Excellent approach!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 802756,
      "author_name": "",
      "author_url": "",
      "post_date": "04/09/2020 19:41:33",
      "content": "<p>Thanks and congratulations. Excellent approach!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "800597": "Big thanks [BEST](http://best-kyiv.org/) for organizing such an outstanding event. It was a great experience for everyone.\nThanks for all my team members (@yaroslav4klymchuk , @vladyelisieiev, @zekamrozek )\n\n# Solution\n## Initial thougths about data \nData was mostly very clean with only small noise in labeling, which resulted in ZERO shake-up in LB\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1690820%2F728aae9f9361a761139f2756ff8a87b5%2Fnoshakeup.png?generation=1586270843647464&amp;alt=media)\nSo we did not do any pre-processing \n\n## CV policy\nWe used simple StratifiedKFold for Cross-Validation and got a very good correlation with public score\n\n## Modeling\nWe tried several models:\n1. Started from resnet34 to make some basic experiments\n2. Switched to SE-ResNeXt50_32x4d and trained it only for 3 folds (because of Disk out of space :) )\n3. Also tried mobilenet_v2 but it was to shallow for our task \n\n## Augmentation\nWe used a pack of augmentations from [segmentation models pytorch](https://github.com/qubvel/segmentation_models.pytorch/blob/master/examples/cars%20segmentation%20(camvid).ipynb) and , for really, did not tune this stuff.\n\n## Optimization\nWe used simple Adam with bigger LR for classifier and ReduceLROnPlateau as a scheduler. As loss function we used simple CE loss.\n\n## Post-processing\nWe used flips TTA for creating final submissions and blend models from different folds. Also we used  [SWA](https://pytorch.org/blog/stochastic-weight-averaging-in-pytorch/) for blending checkpoints from different epochs.\n\nThanks again for all participants and congratulations to winners!!!",
    "802756": "Thanks and congratulations. Excellent approach!"
  },
  "source": "meta"
}