{
  "id": 46628,
  "title": "How do you choose metrics for this kind of task?",
  "url": "/competitions/dog-breed-identification/discussion/46628",
  "author_name": "scott huang(黄)",
  "post_date": "2017-12-31T01:50:38.219000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>say that I use 3600 images for validation. 30 per class on average. if I modify some hyper-parameter, cause 1 image in every class change their labels. that's 120 images.  This will cause accuracy changes in total by 3.3%.  3.3% increase should be a good result, but because only 1 image changes in each class, the result maybe Fraudulent.</p>\n\n<p>I know that the math is not thorough. my point is that If I use accuracy as metric, How big the dataset I need to make sure my experiments result is valid?\nor there are better metric for this task?</p>",
  "messages": [
    {
      "id": 263611,
      "postDate": "2017-12-31T01:50:38.220Z",
      "content": "<p>say that I use 3600 images for validation. 30 per class on average. if I modify some hyper-parameter, cause 1 image in every class change their labels. that's 120 images.  This will cause accuracy changes in total by 3.3%.  3.3% increase should be a good result, but because only 1 image changes in each class, the result maybe Fraudulent.</p>\n\n<p>I know that the math is not thorough. my point is that If I use accuracy as metric, How big the dataset I need to make sure my experiments result is valid?\nor there are better metric for this task?</p>",
      "rawMarkdown": "say that I use 3600 images for validation. 30 per class on average. if I modify some hyper-parameter, cause 1 image in every class change their labels. that's 120 images.  This will cause accuracy changes in total by 3.3%.  3.3% increase should be a good result, but because only 1 image changes in each class, the result maybe Fraudulent.\n\nI know that the math is not thorough. my point is that If I use accuracy as metric, How big the dataset I need to make sure my experiments result is valid?\nor there are better metric for this task?"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "263611": "say that I use 3600 images for validation. 30 per class on average. if I modify some hyper-parameter, cause 1 image in every class change their labels. that's 120 images.  This will cause accuracy changes in total by 3.3%.  3.3% increase should be a good result, but because only 1 image changes in each class, the result maybe Fraudulent.\n\nI know that the math is not thorough. my point is that If I use accuracy as metric, How big the dataset I need to make sure my experiments result is valid?\nor there are better metric for this task?"
  }
}