{
  "id": 247067,
  "title": "How to choice submission for private LB",
  "url": "/competitions/classify-leaves/discussion/247067",
  "author_name": "",
  "post_date": "2021-06-18T03:09:13.356109500Z",
  "votes": 3,
  "comment_count": 5,
  "views": 0,
  "content": "<p>I choose two sub with public LB 0.97772 and 0.98613. (my best LB score is 0.98795)<br>\nI don't choose the best submission because I believe too high score in this noisy labeled dataset may suffer from over-fitting.</p>\n<p><img src=\"https://z3.ax1x.com/2021/06/18/RSlu9g.png\" alt=\"sub\"></p>",
  "messages": [
    {
      "id": "1354953",
      "postDate": "06/18/2021 03:09:13",
      "content": "<p>I choose two sub with public LB 0.97772 and 0.98613. (my best LB score is 0.98795)<br>\nI don't choose the best submission because I believe too high score in this noisy labeled dataset may suffer from over-fitting.</p>\n<p><img src=\"https://z3.ax1x.com/2021/06/18/RSlu9g.png\" alt=\"sub\"></p>",
      "rawMarkdown": "I choose two sub with public LB 0.97772 and 0.98613. (my best LB score is 0.98795)\nI don't choose the best submission because I believe too high score in this noisy labeled dataset may suffer from over-fitting.\n\n![sub](https://z3.ax1x.com/2021/06/18/RSlu9g.png)",
      "votes": null
    },
    {
      "id": "1355583",
      "postDate": "06/18/2021 11:45:19",
      "content": "<p>请问你觉得大概什么样的结果算是比较好的呢，顺便分享一下你的思路呗</p>",
      "rawMarkdown": "请问你觉得大概什么样的结果算是比较好的呢，顺便分享一下你的思路呗",
      "votes": null
    },
    {
      "id": "1356416",
      "postDate": "06/19/2021 02:36:36",
      "content": "<p>我的个人经验是:</p>\n<ol>\n<li>选取<strong>合适的验证方法</strong>。比如AUC，mAP(mean average precision)，micro/macro F-beta score 等，根据任务和数据分布选择最适合的验证方法。一般来说，ACC(Precision)是较为不可靠的验证指标，尤其是正负样本不均衡的情况下(比如正负样本9:1,全预测1也有90%分)。</li>\n<li><strong>相信本地交叉验证</strong>结果而不是线上LB。因为线上测试数据量更小，而交叉验证数据量却是整个训练数据集。</li>\n<li><strong>相信自己</strong>。一些降低模型预测方差的方法，如TTA（test-time augmentation），Blending等集成方法，能降低shake风险（防止public LB分很高而private LB分很低）。一些自己的经验相信有效的方法，不要因为不稳定的测试集而轻易放弃。</li>\n<li>对于CV任务，<strong>优秀的特征提取</strong>更为重要。使用metric learning等方法可以获取更好的特征提取，这样的模型通常会更鲁棒更值得相信。（目前我只提交了baseline，还尚未尝试这点）。</li>\n</ol>\n<hr>\n<p>update:</p>\n<ol>\n<li><strong>相信沐神</strong>。<a href=\"https://arxiv.org/pdf/1812.01187.pdf\" target=\"_blank\">Bag of Tricks</a> | <a href=\"https://arxiv.org/abs/2004.08955\" target=\"_blank\">ResNeSt</a></li>\n</ol>",
      "rawMarkdown": "我的个人经验是:\n1. 选取**合适的验证方法**。比如AUC，mAP(mean average precision)，micro/macro F-beta score 等，根据任务和数据分布选择最适合的验证方法。一般来说，ACC(Precision)是较为不可靠的验证指标，尤其是正负样本不均衡的情况下(比如正负样本9:1,全预测1也有90%分)。\n2. **相信本地交叉验证**结果而不是线上LB。因为线上测试数据量更小，而交叉验证数据量却是整个训练数据集。\n3. **相信自己**。一些降低模型预测方差的方法，如TTA（test-time augmentation），Blending等集成方法，能降低shake风险（防止public LB分很高而private LB分很低）。一些自己的经验相信有效的方法，不要因为不稳定的测试集而轻易放弃。\n4. 对于CV任务，**优秀的特征提取**更为重要。使用metric learning等方法可以获取更好的特征提取，这样的模型通常会更鲁棒更值得相信。（目前我只提交了baseline，还尚未尝试这点）。\n\n---------\n\nupdate:\n5. **相信沐神**。[Bag of Tricks](https://arxiv.org/pdf/1812.01187.pdf) | [ResNeSt](https://arxiv.org/abs/2004.08955)",
      "votes": null
    },
    {
      "id": "1362883",
      "postDate": "06/23/2021 18:01:02",
      "content": "<p><em>In last two competitions, I submitted the wrong submission twice , both of the time I followed somebody else's method and I thought that I wasn't good enough to research and study and I shouldn't submit my method and mine method was the best submission both of the times, I got so poor private score than expected ,after these two competitions I released that we should respect our own work too, like I am doing kaggle continuously form last two months and this is the the lesson I have got that everyone should believe in their own working methodology more than others.</em></p>\n<p>And <a href=\"https://www.kaggle.com/seefun\" target=\"_blank\">@seefun</a>  well done man, that you have this skill,  excellent mate.</p>",
      "rawMarkdown": "*In last two competitions, I submitted the wrong submission twice , both of the time I followed somebody else's method and I thought that I wasn't good enough to research and study and I shouldn't submit my method and mine method was the best submission both of the times, I got so poor private score than expected ,after these two competitions I released that we should respect our own work too, like I am doing kaggle continuously form last two months and this is the the lesson I have got that everyone should believe in their own working methodology more than others.*\n\nAnd @seefun  well done man, that you have this skill,  excellent mate.",
      "votes": null
    },
    {
      "id": "1365877",
      "postDate": "06/26/2021 08:00:36",
      "content": "<p>翻车了 应该是测试集里太多重复样本和噪声标注了</p>",
      "rawMarkdown": "翻车了 应该是测试集里太多重复样本和噪声标注了",
      "votes": null
    },
    {
      "id": "1367800",
      "postDate": "06/28/2021 05:19:12",
      "content": "<p>哈哈哈相信沐神没错了，ResNeSt真的效果极佳😄</p>",
      "rawMarkdown": "哈哈哈相信沐神没错了，ResNeSt真的效果极佳😄",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1355583,
      "author_name": "crazywinds",
      "author_url": "",
      "post_date": "06/18/2021 11:45:19",
      "content": "<p>请问你觉得大概什么样的结果算是比较好的呢，顺便分享一下你的思路呗</p>",
      "votes": null,
      "replies": [
        {
          "id": 1356416,
          "author_name": "seefun",
          "author_url": "",
          "post_date": "06/19/2021 02:36:36",
          "content": "<p>我的个人经验是:</p>\n<ol>\n<li>选取<strong>合适的验证方法</strong>。比如AUC，mAP(mean average precision)，micro/macro F-beta score 等，根据任务和数据分布选择最适合的验证方法。一般来说，ACC(Precision)是较为不可靠的验证指标，尤其是正负样本不均衡的情况下(比如正负样本9:1,全预测1也有90%分)。</li>\n<li><strong>相信本地交叉验证</strong>结果而不是线上LB。因为线上测试数据量更小，而交叉验证数据量却是整个训练数据集。</li>\n<li><strong>相信自己</strong>。一些降低模型预测方差的方法，如TTA（test-time augmentation），Blending等集成方法，能降低shake风险（防止public LB分很高而private LB分很低）。一些自己的经验相信有效的方法，不要因为不稳定的测试集而轻易放弃。</li>\n<li>对于CV任务，<strong>优秀的特征提取</strong>更为重要。使用metric learning等方法可以获取更好的特征提取，这样的模型通常会更鲁棒更值得相信。（目前我只提交了baseline，还尚未尝试这点）。</li>\n</ol>\n<hr>\n<p>update:</p>\n<ol>\n<li><strong>相信沐神</strong>。<a href=\"https://arxiv.org/pdf/1812.01187.pdf\" target=\"_blank\">Bag of Tricks</a> | <a href=\"https://arxiv.org/abs/2004.08955\" target=\"_blank\">ResNeSt</a></li>\n</ol>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1367800,
          "author_name": "charlesyyun",
          "author_url": "",
          "post_date": "06/28/2021 05:19:12",
          "content": "<p>哈哈哈相信沐神没错了，ResNeSt真的效果极佳😄</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1362883,
      "author_name": "sayantankirtaniya",
      "author_url": "",
      "post_date": "06/23/2021 18:01:02",
      "content": "<p><em>In last two competitions, I submitted the wrong submission twice , both of the time I followed somebody else's method and I thought that I wasn't good enough to research and study and I shouldn't submit my method and mine method was the best submission both of the times, I got so poor private score than expected ,after these two competitions I released that we should respect our own work too, like I am doing kaggle continuously form last two months and this is the the lesson I have got that everyone should believe in their own working methodology more than others.</em></p>\n<p>And <a href=\"https://www.kaggle.com/seefun\" target=\"_blank\">@seefun</a>  well done man, that you have this skill,  excellent mate.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1365877,
      "author_name": "seefun",
      "author_url": "",
      "post_date": "06/26/2021 08:00:36",
      "content": "<p>翻车了 应该是测试集里太多重复样本和噪声标注了</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1354953": "I choose two sub with public LB 0.97772 and 0.98613. (my best LB score is 0.98795)\nI don't choose the best submission because I believe too high score in this noisy labeled dataset may suffer from over-fitting.\n\n![sub](https://z3.ax1x.com/2021/06/18/RSlu9g.png)",
    "1355583": "请问你觉得大概什么样的结果算是比较好的呢，顺便分享一下你的思路呗",
    "1356416": "我的个人经验是:\n1. 选取**合适的验证方法**。比如AUC，mAP(mean average precision)，micro/macro F-beta score 等，根据任务和数据分布选择最适合的验证方法。一般来说，ACC(Precision)是较为不可靠的验证指标，尤其是正负样本不均衡的情况下(比如正负样本9:1,全预测1也有90%分)。\n2. **相信本地交叉验证**结果而不是线上LB。因为线上测试数据量更小，而交叉验证数据量却是整个训练数据集。\n3. **相信自己**。一些降低模型预测方差的方法，如TTA（test-time augmentation），Blending等集成方法，能降低shake风险（防止public LB分很高而private LB分很低）。一些自己的经验相信有效的方法，不要因为不稳定的测试集而轻易放弃。\n4. 对于CV任务，**优秀的特征提取**更为重要。使用metric learning等方法可以获取更好的特征提取，这样的模型通常会更鲁棒更值得相信。（目前我只提交了baseline，还尚未尝试这点）。\n\n---------\n\nupdate:\n5. **相信沐神**。[Bag of Tricks](https://arxiv.org/pdf/1812.01187.pdf) | [ResNeSt](https://arxiv.org/abs/2004.08955)",
    "1362883": "*In last two competitions, I submitted the wrong submission twice , both of the time I followed somebody else's method and I thought that I wasn't good enough to research and study and I shouldn't submit my method and mine method was the best submission both of the times, I got so poor private score than expected ,after these two competitions I released that we should respect our own work too, like I am doing kaggle continuously form last two months and this is the the lesson I have got that everyone should believe in their own working methodology more than others.*\n\nAnd @seefun  well done man, that you have this skill,  excellent mate.",
    "1365877": "翻车了 应该是测试集里太多重复样本和噪声标注了",
    "1367800": "哈哈哈相信沐神没错了，ResNeSt真的效果极佳😄"
  },
  "source": "meta"
}