{
  "id": 158889,
  "title": "Correct way to do image augmentations",
  "url": "/competitions/global-wheat-detection/discussion/158889",
  "author_name": "",
  "post_date": "2020-06-15T17:39:58.861204100Z",
  "votes": null,
  "comment_count": 4,
  "views": 0,
  "content": "<p>When we apply image augmentations, is it conventional to \n1) keep the existing data and append to the existing data transformed images, which will increase the number of images in the dataset\nor\n2) transform the existing dataset. This will keep the dataset size fixed.</p>\n\n<p>It seems 2 is done in the examples, but I thought the purpose of image augmentation was to increase the dataset size.</p>",
  "messages": [
    {
      "id": "887492",
      "postDate": "06/15/2020 17:39:58",
      "content": "<p>When we apply image augmentations, is it conventional to \n1) keep the existing data and append to the existing data transformed images, which will increase the number of images in the dataset\nor\n2) transform the existing dataset. This will keep the dataset size fixed.</p>\n\n<p>It seems 2 is done in the examples, but I thought the purpose of image augmentation was to increase the dataset size.</p>",
      "rawMarkdown": "When we apply image augmentations, is it conventional to \n1) keep the existing data and append to the existing data transformed images, which will increase the number of images in the dataset\nor\n2) transform the existing dataset. This will keep the dataset size fixed.\n\nIt seems 2 is done in the examples, but I thought the purpose of image augmentation was to increase the dataset size.",
      "votes": null
    },
    {
      "id": "887503",
      "postDate": "06/15/2020 17:45:07",
      "content": "<p>See <a href=\"https://www.kaggle.com/c/global-wheat-detection/discussion/153536\">here</a></p>",
      "rawMarkdown": "See [here](https://www.kaggle.com/c/global-wheat-detection/discussion/153536)",
      "votes": null
    },
    {
      "id": "887513",
      "postDate": "06/15/2020 17:52:40",
      "content": "<p>You can add a third method which is:</p>\n\n<p>3) Transform the existing dataset during the training  process (or onfly) with a certain probability for each kind of augmentation (random augmentations). In that way you make sure you're augmenting the size of your data and make sure that you're model is seeing multiple variations of your images including the original ones. </p>\n\n<p>Of course I recommend the third one. You can find some kernels using that.</p>\n\n<p>You're welcome!</p>",
      "rawMarkdown": "You can add a third method which is:\n\n3) Transform the existing dataset during the training  process (or onfly) with a certain probability for each kind of augmentation (random augmentations). In that way you make sure you're augmenting the size of your data and make sure that you're model is seeing multiple variations of your images including the original ones. \n\nOf course I recommend the third one. You can find some kernels using that.\n\nYou're welcome!",
      "votes": null
    },
    {
      "id": "887585",
      "postDate": "06/15/2020 18:48:35",
      "content": "<p><a href=\"/ffares\">@ffares</a> I thought about it and now it makes sense to me. Originally I thought if you have a data transformation with probability p, then either the dataset will or will not contain the transformed image. Now I remembered that the model trains for multiple epochs, and so there is a chance that <em>both</em> the transformed image and the original image will be seen, and so the dataset size will in effect be larger. Sorry, I am a deep learning newbie.</p>",
      "rawMarkdown": "ffares I thought about it and now it makes sense to me. Originally I thought if you have a data transformation with probability p, then either the dataset will or will not contain the transformed image. Now I remembered that the model trains for multiple epochs, and so there is a chance that *both* the transformed image and the original image will be seen, and so the dataset size will in effect be larger. Sorry, I am a deep learning newbie.",
      "votes": null
    },
    {
      "id": "887653",
      "postDate": "06/15/2020 19:37:03",
      "content": "<p>Exactly! </p>\n\n<p>A journey of a thousand miles begins with a single step :)</p>",
      "rawMarkdown": "Exactly! \n\nA journey of a thousand miles begins with a single step :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 887503,
      "author_name": "matthewmasters",
      "author_url": "",
      "post_date": "06/15/2020 17:45:07",
      "content": "<p>See <a href=\"https://www.kaggle.com/c/global-wheat-detection/discussion/153536\">here</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 887513,
      "author_name": "ffares",
      "author_url": "",
      "post_date": "06/15/2020 17:52:40",
      "content": "<p>You can add a third method which is:</p>\n\n<p>3) Transform the existing dataset during the training  process (or onfly) with a certain probability for each kind of augmentation (random augmentations). In that way you make sure you're augmenting the size of your data and make sure that you're model is seeing multiple variations of your images including the original ones. </p>\n\n<p>Of course I recommend the third one. You can find some kernels using that.</p>\n\n<p>You're welcome!</p>",
      "votes": null,
      "replies": [
        {
          "id": 887585,
          "author_name": "aamster",
          "author_url": "",
          "post_date": "06/15/2020 18:48:35",
          "content": "<p><a href=\"/ffares\">@ffares</a> I thought about it and now it makes sense to me. Originally I thought if you have a data transformation with probability p, then either the dataset will or will not contain the transformed image. Now I remembered that the model trains for multiple epochs, and so there is a chance that <em>both</em> the transformed image and the original image will be seen, and so the dataset size will in effect be larger. Sorry, I am a deep learning newbie.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 887653,
          "author_name": "ffares",
          "author_url": "",
          "post_date": "06/15/2020 19:37:03",
          "content": "<p>Exactly! </p>\n\n<p>A journey of a thousand miles begins with a single step :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "887492": "When we apply image augmentations, is it conventional to \n1) keep the existing data and append to the existing data transformed images, which will increase the number of images in the dataset\nor\n2) transform the existing dataset. This will keep the dataset size fixed.\n\nIt seems 2 is done in the examples, but I thought the purpose of image augmentation was to increase the dataset size.",
    "887503": "See [here](https://www.kaggle.com/c/global-wheat-detection/discussion/153536)",
    "887513": "You can add a third method which is:\n\n3) Transform the existing dataset during the training  process (or onfly) with a certain probability for each kind of augmentation (random augmentations). In that way you make sure you're augmenting the size of your data and make sure that you're model is seeing multiple variations of your images including the original ones. \n\nOf course I recommend the third one. You can find some kernels using that.\n\nYou're welcome!",
    "887585": "ffares I thought about it and now it makes sense to me. Originally I thought if you have a data transformation with probability p, then either the dataset will or will not contain the transformed image. Now I remembered that the model trains for multiple epochs, and so there is a chance that *both* the transformed image and the original image will be seen, and so the dataset size will in effect be larger. Sorry, I am a deep learning newbie.",
    "887653": "Exactly! \n\nA journey of a thousand miles begins with a single step :)"
  },
  "source": "meta"
}