{
  "id": 164431,
  "title": "What's your local validation stats for YoloV5?",
  "url": "/competitions/global-wheat-detection/discussion/164431",
  "author_name": "",
  "post_date": "2020-07-06T08:36:00.010035600Z",
  "votes": 16,
  "comment_count": 13,
  "views": 0,
  "content": "<p>I have been pretty frustrated for not being able to create a stable CV-Public Leaderboard correspondence with my YoloV5 pipeline, so I'm gonna share my local results here. Any additional reports are welcomed!</p>\n\n<p>(Also it would be unfair to not report the split I'm using, below is the random split used by most public kernels, fold 0.)</p>\n\n<p><code>\nmAP@.5: 0.976\nmAP@.5:.95: 0.612\nPrecision: 0.841\nRecall: 0.973\n</code></p>",
  "messages": [
    {
      "id": "917094",
      "postDate": "07/06/2020 08:36:00",
      "content": "<p>I have been pretty frustrated for not being able to create a stable CV-Public Leaderboard correspondence with my YoloV5 pipeline, so I'm gonna share my local results here. Any additional reports are welcomed!</p>\n\n<p>(Also it would be unfair to not report the split I'm using, below is the random split used by most public kernels, fold 0.)</p>\n\n<p><code>\nmAP@.5: 0.976\nmAP@.5:.95: 0.612\nPrecision: 0.841\nRecall: 0.973\n</code></p>",
      "rawMarkdown": "I have been pretty frustrated for not being able to create a stable CV-Public Leaderboard correspondence with my YoloV5 pipeline, so I'm gonna share my local results here. Any additional reports are welcomed!\n\n(Also it would be unfair to not report the split I'm using, below is the random split used by most public kernels, fold 0.)\n\n```\nmAP@.5: 0.976\nmAP@.5:.95: 0.612\nPrecision: 0.841\nRecall: 0.973\n```",
      "votes": null
    },
    {
      "id": "917837",
      "postDate": "07/06/2020 19:07:20",
      "content": "<p>Thanks for sharing!</p>",
      "rawMarkdown": "Thanks for sharing!",
      "votes": null
    },
    {
      "id": "917895",
      "postDate": "07/06/2020 19:34:42",
      "content": "<p>5 fold stratified Kfold on num_target:\nonly using fold 0\nMAP@.5:.95: 0.5489\nLB with pseudo labeling 0.7624</p>\n\n<p>But this split is not stable either</p>",
      "rawMarkdown": "5 fold stratified Kfold on num_target:\nonly using fold 0\nMAP@.5:.95: 0.5489\nLB with pseudo labeling 0.7624\n\nBut this split is not stable either",
      "votes": null
    },
    {
      "id": "918134",
      "postDate": "07/07/2020 02:46:42",
      "content": "<p>Awesome work！I got <strong>mAP@.5:.95: 0.5541</strong> with other cv split. But the correspondence of CV and LB is also unstable.\nMaybe we should try new stable split or choose more convincing metric.</p>",
      "rawMarkdown": "Awesome work！I got **mAP@.5:.95: 0.5541** with other cv split. But the correspondence of CV and LB is also unstable.\nMaybe we should try new stable split or choose more convincing metric.",
      "votes": null
    },
    {
      "id": "918331",
      "postDate": "07/07/2020 07:23:56",
      "content": "<p>I also confuse it,  fitness = 61.62   LB=74.94\n                             fitness = 64.47   LB = 73.20  this is weird, What's you next plan?</p>",
      "rawMarkdown": "I also confuse it,  fitness = 61.62   LB=74.94\n                             fitness = 64.47   LB = 73.20  this is weird, What's you next plan?",
      "votes": null
    },
    {
      "id": "918554",
      "postDate": "07/07/2020 10:09:36",
      "content": "<p>Honestly I'm not sure at this point. Feels like until someone figures out a way to effectively validate model performance, or find out a way to generalize model to unknown distribution, the entire competition is just a huge lottery.</p>",
      "rawMarkdown": "Honestly I'm not sure at this point. Feels like until someone figures out a way to effectively validate model performance, or find out a way to generalize model to unknown distribution, the entire competition is just a huge lottery.",
      "votes": null
    },
    {
      "id": "918587",
      "postDate": "07/07/2020 10:39:49",
      "content": "<p>Maybe we need split the data by source!</p>",
      "rawMarkdown": "Maybe we need split the data by source!",
      "votes": null
    },
    {
      "id": "918617",
      "postDate": "07/07/2020 11:11:35",
      "content": "<p>What LB results do you have with this model?</p>",
      "rawMarkdown": "What LB results do you have with this model?",
      "votes": null
    },
    {
      "id": "918741",
      "postDate": "07/07/2020 13:05:06",
      "content": "<p>Tried that already. Didn't quite work.</p>",
      "rawMarkdown": "Tried that already. Didn't quite work.",
      "votes": null
    },
    {
      "id": "918744",
      "postDate": "07/07/2020 13:05:46",
      "content": "<p>Abysmal. Also sensitive w. the choice of WBF/NMS parameters, so I choose not to share.</p>",
      "rawMarkdown": "Abysmal. Also sensitive w. the choice of WBF/NMS parameters, so I choose not to share.",
      "votes": null
    },
    {
      "id": "920341",
      "postDate": "07/08/2020 14:17:01",
      "content": "<p><a href=\"/bixiaopeng\">@bixiaopeng</a> already tried.. i get lower lb score.</p>\n\n<p>are you using this metric:\n<a href=\"https://www.kaggle.com/pestipeti/competition-metric-details-script\">https://www.kaggle.com/pestipeti/competition-metric-details-script</a></p>\n\n<p>or the one in the yolov5 repo?</p>",
      "rawMarkdown": "bixiaopeng already tried.. i get lower lb score.\n\nare you using this metric:\nhttps://www.kaggle.com/pestipeti/competition-metric-details-script\n\nor the one in the yolov5 repo?",
      "votes": null
    },
    {
      "id": "920354",
      "postDate": "07/08/2020 14:31:32",
      "content": "<p>yolov5 repo, I try only use map0.5~0.75,but not get better.</p>",
      "rawMarkdown": "yolov5 repo, I try only use map0.5~0.75,but not get better.",
      "votes": null
    },
    {
      "id": "922409",
      "postDate": "07/10/2020 05:04:28",
      "content": "<p>The one with highest LB I got is look like\nmAP@.5: 0.971\nmAP@.5:.95: 0.598\nPrecision: 0.835\nRecall: 0.971\nSplits are unstable. I am trying to figure out the best possible split. With the current status of the competition, I am confused that even the leader-board even matters. </p>",
      "rawMarkdown": "The one with highest LB I got is look like\nmAP@.5: 0.971\nmAP@.5:.95: 0.598\nPrecision: 0.835\nRecall: 0.971\nSplits are unstable. I am trying to figure out the best possible split. With the current status of the competition, I am confused that even the leader-board even matters.",
      "votes": null
    },
    {
      "id": "981064",
      "postDate": "08/22/2020 06:30:34",
      "content": "<p>Hi,did you change your files in yolov5 such as dataset.py?</p>",
      "rawMarkdown": "Hi,did you change your files in yolov5 such as dataset.py?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 981064,
      "author_name": "zekunn",
      "author_url": "",
      "post_date": "08/22/2020 06:30:34",
      "content": "<p>Hi,did you change your files in yolov5 such as dataset.py?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 917837,
      "author_name": "salmaneunus",
      "author_url": "",
      "post_date": "07/06/2020 19:07:20",
      "content": "<p>Thanks for sharing!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 917895,
      "author_name": "jielu0728",
      "author_url": "",
      "post_date": "07/06/2020 19:34:42",
      "content": "<p>5 fold stratified Kfold on num_target:\nonly using fold 0\nMAP@.5:.95: 0.5489\nLB with pseudo labeling 0.7624</p>\n\n<p>But this split is not stable either</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 918134,
      "author_name": "dandingclam",
      "author_url": "",
      "post_date": "07/07/2020 02:46:42",
      "content": "<p>Awesome work！I got <strong>mAP@.5:.95: 0.5541</strong> with other cv split. But the correspondence of CV and LB is also unstable.\nMaybe we should try new stable split or choose more convincing metric.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 918331,
      "author_name": "bixiaopeng",
      "author_url": "",
      "post_date": "07/07/2020 07:23:56",
      "content": "<p>I also confuse it,  fitness = 61.62   LB=74.94\n                             fitness = 64.47   LB = 73.20  this is weird, What's you next plan?</p>",
      "votes": null,
      "replies": [
        {
          "id": 918554,
          "author_name": "alexanderliao",
          "author_url": "",
          "post_date": "07/07/2020 10:09:36",
          "content": "<p>Honestly I'm not sure at this point. Feels like until someone figures out a way to effectively validate model performance, or find out a way to generalize model to unknown distribution, the entire competition is just a huge lottery.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 918587,
          "author_name": "bixiaopeng",
          "author_url": "",
          "post_date": "07/07/2020 10:39:49",
          "content": "<p>Maybe we need split the data by source!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 918741,
          "author_name": "alexanderliao",
          "author_url": "",
          "post_date": "07/07/2020 13:05:06",
          "content": "<p>Tried that already. Didn't quite work.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 920341,
          "author_name": "geno87",
          "author_url": "",
          "post_date": "07/08/2020 14:17:01",
          "content": "<p><a href=\"/bixiaopeng\">@bixiaopeng</a> already tried.. i get lower lb score.</p>\n\n<p>are you using this metric:\n<a href=\"https://www.kaggle.com/pestipeti/competition-metric-details-script\">https://www.kaggle.com/pestipeti/competition-metric-details-script</a></p>\n\n<p>or the one in the yolov5 repo?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 920354,
          "author_name": "bixiaopeng",
          "author_url": "",
          "post_date": "07/08/2020 14:31:32",
          "content": "<p>yolov5 repo, I try only use map0.5~0.75,but not get better.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 918617,
      "author_name": "smolsnastya",
      "author_url": "",
      "post_date": "07/07/2020 11:11:35",
      "content": "<p>What LB results do you have with this model?</p>",
      "votes": null,
      "replies": [
        {
          "id": 918744,
          "author_name": "alexanderliao",
          "author_url": "",
          "post_date": "07/07/2020 13:05:46",
          "content": "<p>Abysmal. Also sensitive w. the choice of WBF/NMS parameters, so I choose not to share.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 922409,
      "author_name": "ahmmadrashid",
      "author_url": "",
      "post_date": "07/10/2020 05:04:28",
      "content": "<p>The one with highest LB I got is look like\nmAP@.5: 0.971\nmAP@.5:.95: 0.598\nPrecision: 0.835\nRecall: 0.971\nSplits are unstable. I am trying to figure out the best possible split. With the current status of the competition, I am confused that even the leader-board even matters. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "917094": "I have been pretty frustrated for not being able to create a stable CV-Public Leaderboard correspondence with my YoloV5 pipeline, so I'm gonna share my local results here. Any additional reports are welcomed!\n\n(Also it would be unfair to not report the split I'm using, below is the random split used by most public kernels, fold 0.)\n\n```\nmAP@.5: 0.976\nmAP@.5:.95: 0.612\nPrecision: 0.841\nRecall: 0.973\n```",
    "917837": "Thanks for sharing!",
    "917895": "5 fold stratified Kfold on num_target:\nonly using fold 0\nMAP@.5:.95: 0.5489\nLB with pseudo labeling 0.7624\n\nBut this split is not stable either",
    "918134": "Awesome work！I got **mAP@.5:.95: 0.5541** with other cv split. But the correspondence of CV and LB is also unstable.\nMaybe we should try new stable split or choose more convincing metric.",
    "918331": "I also confuse it,  fitness = 61.62   LB=74.94\n                             fitness = 64.47   LB = 73.20  this is weird, What's you next plan?",
    "918554": "Honestly I'm not sure at this point. Feels like until someone figures out a way to effectively validate model performance, or find out a way to generalize model to unknown distribution, the entire competition is just a huge lottery.",
    "918587": "Maybe we need split the data by source!",
    "918617": "What LB results do you have with this model?",
    "918741": "Tried that already. Didn't quite work.",
    "918744": "Abysmal. Also sensitive w. the choice of WBF/NMS parameters, so I choose not to share.",
    "920341": "bixiaopeng already tried.. i get lower lb score.\n\nare you using this metric:\nhttps://www.kaggle.com/pestipeti/competition-metric-details-script\n\nor the one in the yolov5 repo?",
    "920354": "yolov5 repo, I try only use map0.5~0.75,but not get better.",
    "922409": "The one with highest LB I got is look like\nmAP@.5: 0.971\nmAP@.5:.95: 0.598\nPrecision: 0.835\nRecall: 0.971\nSplits are unstable. I am trying to figure out the best possible split. With the current status of the competition, I am confused that even the leader-board even matters.",
    "981064": "Hi,did you change your files in yolov5 such as dataset.py?"
  },
  "source": "meta"
}