{
  "id": 11540,
  "title": "Training  data quality and image size?",
  "url": "/competitions/datasciencebowl/discussion/11540",
  "author_name": "Charles Roberson",
  "post_date": "2015-01-17T20:06:07.277000",
  "votes": 3,
  "comment_count": 0,
  "views": 997,
  "content": "<p>Given that I and others have seen results suggesting that size doesn't matter, but one of the admins seemed to suggest it does, I looked into the training data distribution.</p>\n<p>I set the largest possible image size to 160,000 pixels and then counted the number of training images in each 10% range. &nbsp;Here are the results:</p>\n<p>10 % 28285 <br> 20 % 1163 <br> 30 % 469 <br> 40 % 228 <br> 50 % 98 <br> 60 % 42 <br> 70 % 25 <br> 80 % 14 <br> 90 % 8 <br> 100 % 1</p>\n<p>&nbsp; As you can see, 28,285 images are less than or equal to 16K pixels and only one image is greater than or equal to 160K pixels. 1,163 images are between 16K and 32K pixels.&nbsp;</p>\n<p>&nbsp;To me, &quot;size matters&quot; means that an object in an image of 40x20 that looks the same as an object in an image of 400x200 is not the same.</p>\n<p>So, is the training data a good sampling of the data or should we get a better sampling?</p>\n<p>&nbsp; Charles</p>",
  "messages": [
    {
      "id": 62843,
      "postDate": "2015-01-17T20:06:07.277Z",
      "content": "<p>Given that I and others have seen results suggesting that size doesn't matter, but one of the admins seemed to suggest it does, I looked into the training data distribution.</p>\n<p>I set the largest possible image size to 160,000 pixels and then counted the number of training images in each 10% range. &nbsp;Here are the results:</p>\n<p>10 % 28285 <br> 20 % 1163 <br> 30 % 469 <br> 40 % 228 <br> 50 % 98 <br> 60 % 42 <br> 70 % 25 <br> 80 % 14 <br> 90 % 8 <br> 100 % 1</p>\n<p>&nbsp; As you can see, 28,285 images are less than or equal to 16K pixels and only one image is greater than or equal to 160K pixels. 1,163 images are between 16K and 32K pixels.&nbsp;</p>\n<p>&nbsp;To me, &quot;size matters&quot; means that an object in an image of 40x20 that looks the same as an object in an image of 400x200 is not the same.</p>\n<p>So, is the training data a good sampling of the data or should we get a better sampling?</p>\n<p>&nbsp; Charles</p>",
      "votes": 3
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "62843": ""
  }
}