{
  "id": 122167,
  "title": "3rd place Solution",
  "url": "/competitions/Kannada-MNIST/writeups/haiyang-liu-3rd-place-solution",
  "author_name": "",
  "post_date": "2019-12-18T07:40:06.972094600Z",
  "votes": 30,
  "comment_count": 11,
  "views": 0,
  "content": "<p>Thanks a lot to Kaggle and <a href=\"/higgstachyon\">@higgstachyon</a> for hosting this competition, it's suitable for me to start kaggle competitions, congratulations to all the winners!</p>\n\n<p>My method is very sample, just try to avoid overfitting. \nAnd the two versions code are available in my github: there are 10 tricks in the code, 3 final used.\n<a href=\"https://github.com/HaiyangLiu1997/Kaggle_Kannada_MNIST_Pytorch\">https://github.com/HaiyangLiu1997/Kaggle_Kannada_MNIST_Pytorch</a></p>\n\n<h3>What worked for me:</h3>\n\n<ol>\n<li>pseudo-labeling, increase my score from 99.280 to 99.420(private score)</li>\n<li>5 model embedding, average voting. increase my score from 99.180 to 99.280(private score)</li>\n<li>using all data to train, after choosing best model base on my local 5 KFold CV </li>\n<li>fix the random seed at first. </li>\n</ol>\n\n<h3>What didn't worked for me:</h3>\n\n<p><strong>the didn't work, just mean no significant improve, of course the result will increase 00.020 sometimes</strong>\n1. warm-up\n2. multi-lr in different layers\n3. more complex model, such MobileNetV3, DenseNet\n4. small batchsize\n5. TTA\n6. Focus loss\n7. label smoothing</p>\n\n<h3>My consideration</h3>\n\n<p><strong>I'm curious about the First and Second place methods, I think it will have more useful things than my simple solution</strong>\nMore details are in github. Thanks for reading!</p>",
  "messages": [
    {
      "id": "697629",
      "postDate": "12/18/2019 07:40:06",
      "content": "<p>Thanks a lot to Kaggle and <a href=\"/higgstachyon\">@higgstachyon</a> for hosting this competition, it's suitable for me to start kaggle competitions, congratulations to all the winners!</p>\n\n<p>My method is very sample, just try to avoid overfitting. \nAnd the two versions code are available in my github: there are 10 tricks in the code, 3 final used.\n<a href=\"https://github.com/HaiyangLiu1997/Kaggle_Kannada_MNIST_Pytorch\">https://github.com/HaiyangLiu1997/Kaggle_Kannada_MNIST_Pytorch</a></p>\n\n<h3>What worked for me:</h3>\n\n<ol>\n<li>pseudo-labeling, increase my score from 99.280 to 99.420(private score)</li>\n<li>5 model embedding, average voting. increase my score from 99.180 to 99.280(private score)</li>\n<li>using all data to train, after choosing best model base on my local 5 KFold CV </li>\n<li>fix the random seed at first. </li>\n</ol>\n\n<h3>What didn't worked for me:</h3>\n\n<p><strong>the didn't work, just mean no significant improve, of course the result will increase 00.020 sometimes</strong>\n1. warm-up\n2. multi-lr in different layers\n3. more complex model, such MobileNetV3, DenseNet\n4. small batchsize\n5. TTA\n6. Focus loss\n7. label smoothing</p>\n\n<h3>My consideration</h3>\n\n<p><strong>I'm curious about the First and Second place methods, I think it will have more useful things than my simple solution</strong>\nMore details are in github. Thanks for reading!</p>",
      "rawMarkdown": "Thanks a lot to Kaggle and @higgstachyon for hosting this competition, it's suitable for me to start kaggle competitions, congratulations to all the winners!\n\nMy method is very sample, just try to avoid overfitting. \nAnd the two versions code are available in my github: there are 10 tricks in the code, 3 final used.\nhttps://github.com/HaiyangLiu1997/Kaggle_Kannada_MNIST_Pytorch\n\n### What worked for me:\n1. pseudo-labeling, increase my score from 99.280 to 99.420(private score)\n2. 5 model embedding, average voting. increase my score from 99.180 to 99.280(private score)\n3. using all data to train, after choosing best model base on my local 5 KFold CV \n4. fix the random seed at first. \n\n### What didn't worked for me:\n**the didn't work, just mean no significant improve, of course the result will increase 00.020 sometimes**\n1. warm-up\n2. multi-lr in different layers\n3. more complex model, such MobileNetV3, DenseNet\n4. small batchsize\n5. TTA\n6. Focus loss\n7. label smoothing\n\n### My consideration\n**I'm curious about the First and Second place methods, I think it will have more useful things than my simple solution**\nMore details are in github. Thanks for reading!",
      "votes": null
    },
    {
      "id": "697780",
      "postDate": "12/18/2019 11:53:12",
      "content": "<p>nice sharing, so many useful trials in the code!</p>",
      "rawMarkdown": "nice sharing, so many useful trials in the code!",
      "votes": null
    },
    {
      "id": "697787",
      "postDate": "12/18/2019 12:06:39",
      "content": "<p>Very interesting, thank you for sharing! </p>",
      "rawMarkdown": "Very interesting, thank you for sharing!",
      "votes": null
    },
    {
      "id": "697847",
      "postDate": "12/18/2019 13:39:53",
      "content": "<p>Thanks for sharing</p>",
      "rawMarkdown": "Thanks for sharing",
      "votes": null
    },
    {
      "id": "697881",
      "postDate": "12/18/2019 14:08:41",
      "content": "<p>good work!!</p>",
      "rawMarkdown": "good work!!",
      "votes": null
    },
    {
      "id": "698004",
      "postDate": "12/18/2019 16:54:06",
      "content": "<p>Thanks for sharing your solution details and your code. Congratulations!</p>",
      "rawMarkdown": "Thanks for sharing your solution details and your code. Congratulations!",
      "votes": null
    },
    {
      "id": "698598",
      "postDate": "12/19/2019 12:54:02",
      "content": "<p>This is nice...</p>",
      "rawMarkdown": "This is nice...",
      "votes": null
    },
    {
      "id": "698601",
      "postDate": "12/19/2019 13:03:53",
      "content": "<p>Congrats!🎉  and Thanks for sharing!!😄 👍 </p>",
      "rawMarkdown": "Congrats!🎉  and Thanks for sharing!!😄 👍",
      "votes": null
    },
    {
      "id": "699314",
      "postDate": "12/20/2019 10:00:08",
      "content": "<p>Congratulations !\nLooks like pseudo-labeling was the key here</p>",
      "rawMarkdown": "Congratulations !\nLooks like pseudo-labeling was the key here",
      "votes": null
    },
    {
      "id": "699761",
      "postDate": "12/20/2019 21:30:44",
      "content": "<p>nice effort...</p>",
      "rawMarkdown": "nice effort...",
      "votes": null
    },
    {
      "id": "700540",
      "postDate": "12/22/2019 07:07:20",
      "content": "<p>really insightful , thanks and would love to connect !</p>",
      "rawMarkdown": "really insightful , thanks and would love to connect !",
      "votes": null
    },
    {
      "id": "700616",
      "postDate": "12/22/2019 10:06:26",
      "content": "<p>Congrats! and thaks for sharing.</p>",
      "rawMarkdown": "Congrats! and thaks for sharing.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 697780,
      "author_name": "pleaseenjoy",
      "author_url": "",
      "post_date": "12/18/2019 11:53:12",
      "content": "<p>nice sharing, so many useful trials in the code!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 697787,
      "author_name": "ricardoamferreira",
      "author_url": "",
      "post_date": "12/18/2019 12:06:39",
      "content": "<p>Very interesting, thank you for sharing! </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 697847,
      "author_name": "oldmon",
      "author_url": "",
      "post_date": "12/18/2019 13:39:53",
      "content": "<p>Thanks for sharing</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 697881,
      "author_name": "dsv61999",
      "author_url": "",
      "post_date": "12/18/2019 14:08:41",
      "content": "<p>good work!!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 698004,
      "author_name": "datahobbit",
      "author_url": "",
      "post_date": "12/18/2019 16:54:06",
      "content": "<p>Thanks for sharing your solution details and your code. Congratulations!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 698598,
      "author_name": "christiansly",
      "author_url": "",
      "post_date": "12/19/2019 12:54:02",
      "content": "<p>This is nice...</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 698601,
      "author_name": "mashlyn",
      "author_url": "",
      "post_date": "12/19/2019 13:03:53",
      "content": "<p>Congrats!🎉  and Thanks for sharing!!😄 👍 </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 699314,
      "author_name": "hemanth007",
      "author_url": "",
      "post_date": "12/20/2019 10:00:08",
      "content": "<p>Congratulations !\nLooks like pseudo-labeling was the key here</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 699761,
      "author_name": "ahmedelgammal",
      "author_url": "",
      "post_date": "12/20/2019 21:30:44",
      "content": "<p>nice effort...</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 700540,
      "author_name": "ravijoe",
      "author_url": "",
      "post_date": "12/22/2019 07:07:20",
      "content": "<p>really insightful , thanks and would love to connect !</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 700616,
      "author_name": "corochann",
      "author_url": "",
      "post_date": "12/22/2019 10:06:26",
      "content": "<p>Congrats! and thaks for sharing.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "697629": "Thanks a lot to Kaggle and @higgstachyon for hosting this competition, it's suitable for me to start kaggle competitions, congratulations to all the winners!\n\nMy method is very sample, just try to avoid overfitting. \nAnd the two versions code are available in my github: there are 10 tricks in the code, 3 final used.\nhttps://github.com/HaiyangLiu1997/Kaggle_Kannada_MNIST_Pytorch\n\n### What worked for me:\n1. pseudo-labeling, increase my score from 99.280 to 99.420(private score)\n2. 5 model embedding, average voting. increase my score from 99.180 to 99.280(private score)\n3. using all data to train, after choosing best model base on my local 5 KFold CV \n4. fix the random seed at first. \n\n### What didn't worked for me:\n**the didn't work, just mean no significant improve, of course the result will increase 00.020 sometimes**\n1. warm-up\n2. multi-lr in different layers\n3. more complex model, such MobileNetV3, DenseNet\n4. small batchsize\n5. TTA\n6. Focus loss\n7. label smoothing\n\n### My consideration\n**I'm curious about the First and Second place methods, I think it will have more useful things than my simple solution**\nMore details are in github. Thanks for reading!",
    "697780": "nice sharing, so many useful trials in the code!",
    "697787": "Very interesting, thank you for sharing!",
    "697847": "Thanks for sharing",
    "697881": "good work!!",
    "698004": "Thanks for sharing your solution details and your code. Congratulations!",
    "698598": "This is nice...",
    "698601": "Congrats!🎉  and Thanks for sharing!!😄 👍",
    "699314": "Congratulations !\nLooks like pseudo-labeling was the key here",
    "699761": "nice effort...",
    "700540": "really insightful , thanks and would love to connect !",
    "700616": "Congrats! and thaks for sharing."
  },
  "source": "meta"
}