{
  "id": 153536,
  "title": "Why augmenting data on the fly is better approach than creating a new larger dataset? [solved]",
  "url": "/competitions/global-wheat-detection/discussion/153536",
  "author_name": "",
  "post_date": "2020-05-25T08:26:29.370971900Z",
  "votes": 1,
  "comment_count": 5,
  "views": 0,
  "content": "<p>This might be a very basic question for others but I had this doubt until today when I realized the intuition behind on the fly data augmentation while training and why it's better than creating a larger augmented dataset. Here's the reason (I'm not an expert please correct me if I'm wrong): </p>\n\n<p>When you create a larger dataset using any augmentations, you randomly apply those augmentations on the images and save them. Now when you train on those images, your model gets to see only those augmentations at each epoch (no new augmentations are randomly generated at every epoch in this process and thus this limits the number of possibilities for each image). </p>\n\n<p>But when you augment the data on the fly, each image can have different augmentation applied at each epoch (because of randomness) but this is not the case with the previous approach. And that's why I think the second approach works well.</p>\n\n<p>I'm sharing this because of <a href=\"https://qr.ae/pNycwG\">this</a></p>",
  "messages": [
    {
      "id": "860356",
      "postDate": "05/25/2020 08:26:29",
      "content": "<p>This might be a very basic question for others but I had this doubt until today when I realized the intuition behind on the fly data augmentation while training and why it's better than creating a larger augmented dataset. Here's the reason (I'm not an expert please correct me if I'm wrong): </p>\n\n<p>When you create a larger dataset using any augmentations, you randomly apply those augmentations on the images and save them. Now when you train on those images, your model gets to see only those augmentations at each epoch (no new augmentations are randomly generated at every epoch in this process and thus this limits the number of possibilities for each image). </p>\n\n<p>But when you augment the data on the fly, each image can have different augmentation applied at each epoch (because of randomness) but this is not the case with the previous approach. And that's why I think the second approach works well.</p>\n\n<p>I'm sharing this because of <a href=\"https://qr.ae/pNycwG\">this</a></p>",
      "rawMarkdown": "This might be a very basic question for others but I had this doubt until today when I realized the intuition behind on the fly data augmentation while training and why it's better than creating a larger augmented dataset. Here's the reason (I'm not an expert please correct me if I'm wrong): \n\nWhen you create a larger dataset using any augmentations, you randomly apply those augmentations on the images and save them. Now when you train on those images, your model gets to see only those augmentations at each epoch (no new augmentations are randomly generated at every epoch in this process and thus this limits the number of possibilities for each image). \n\nBut when you augment the data on the fly, each image can have different augmentation applied at each epoch (because of randomness) but this is not the case with the previous approach. And that's why I think the second approach works well.\n\nI'm sharing this because of [this](https://qr.ae/pNycwG)",
      "votes": null
    },
    {
      "id": "860362",
      "postDate": "05/25/2020 08:34:06",
      "content": "<p>Yes, that is correct. Augmentations should almost always be applied on the fly for this reason. One exception is sometimes people will create a larger dataset containing all possible combinations of rotations/flipping to shave off time when loading data. </p>",
      "rawMarkdown": "Yes, that is correct. Augmentations should almost always be applied on the fly for this reason. One exception is sometimes people will create a larger dataset containing all possible combinations of rotations/flipping to shave off time when loading data.",
      "votes": null
    },
    {
      "id": "860399",
      "postDate": "05/25/2020 09:27:08",
      "content": "<p>Yes, correct! Thanks for validating.</p>",
      "rawMarkdown": "Yes, correct! Thanks for validating.",
      "votes": null
    },
    {
      "id": "860457",
      "postDate": "05/25/2020 10:33:51",
      "content": "<p>You're right. The model trained with on the fly augmentations generalizes better as he's seeing more samples comparing to another model having the same epochs and using offline augmentations.</p>",
      "rawMarkdown": "You're right. The model trained with on the fly augmentations generalizes better as he's seeing more samples comparing to another model having the same epochs and using offline augmentations.",
      "votes": null
    },
    {
      "id": "860525",
      "postDate": "05/25/2020 11:34:57",
      "content": "<p>Do you mean \"create a larger dataset\" without contain the origin dataset ?</p>\n\n<p>I tried to create a larger dataset using augmentations + the origin dataset: LB: 0.6380.\nBut only fly with same augmentations: LB: 0.6359.</p>\n\n<p>So I still confuse or depend on the random of augmentations with probably p</p>",
      "rawMarkdown": "Do you mean \"create a larger dataset\" without contain the origin dataset ?\n\nI tried to create a larger dataset using augmentations + the origin dataset: LB: 0.6380.\nBut only fly with same augmentations: LB: 0.6359.\n\nSo I still confuse or depend on the random of augmentations with probably p",
      "votes": null
    },
    {
      "id": "860542",
      "postDate": "05/25/2020 11:57:00",
      "content": "<p>In any case, the model will get to see limited and fixed augmentations. Practically I've tried only with static augmented dataset (original dataset + augmented images). Will try on the fly augmentations and update this.</p>",
      "rawMarkdown": "In any case, the model will get to see limited and fixed augmentations. Practically I've tried only with static augmented dataset (original dataset + augmented images). Will try on the fly augmentations and update this.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 860362,
      "author_name": "matthewmasters",
      "author_url": "",
      "post_date": "05/25/2020 08:34:06",
      "content": "<p>Yes, that is correct. Augmentations should almost always be applied on the fly for this reason. One exception is sometimes people will create a larger dataset containing all possible combinations of rotations/flipping to shave off time when loading data. </p>",
      "votes": null,
      "replies": [
        {
          "id": 860399,
          "author_name": "kaushal2896",
          "author_url": "",
          "post_date": "05/25/2020 09:27:08",
          "content": "<p>Yes, correct! Thanks for validating.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 860457,
      "author_name": "ffares",
      "author_url": "",
      "post_date": "05/25/2020 10:33:51",
      "content": "<p>You're right. The model trained with on the fly augmentations generalizes better as he's seeing more samples comparing to another model having the same epochs and using offline augmentations.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 860525,
      "author_name": "truonghoang",
      "author_url": "",
      "post_date": "05/25/2020 11:34:57",
      "content": "<p>Do you mean \"create a larger dataset\" without contain the origin dataset ?</p>\n\n<p>I tried to create a larger dataset using augmentations + the origin dataset: LB: 0.6380.\nBut only fly with same augmentations: LB: 0.6359.</p>\n\n<p>So I still confuse or depend on the random of augmentations with probably p</p>",
      "votes": null,
      "replies": [
        {
          "id": 860542,
          "author_name": "kaushal2896",
          "author_url": "",
          "post_date": "05/25/2020 11:57:00",
          "content": "<p>In any case, the model will get to see limited and fixed augmentations. Practically I've tried only with static augmented dataset (original dataset + augmented images). Will try on the fly augmentations and update this.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "860356": "This might be a very basic question for others but I had this doubt until today when I realized the intuition behind on the fly data augmentation while training and why it's better than creating a larger augmented dataset. Here's the reason (I'm not an expert please correct me if I'm wrong): \n\nWhen you create a larger dataset using any augmentations, you randomly apply those augmentations on the images and save them. Now when you train on those images, your model gets to see only those augmentations at each epoch (no new augmentations are randomly generated at every epoch in this process and thus this limits the number of possibilities for each image). \n\nBut when you augment the data on the fly, each image can have different augmentation applied at each epoch (because of randomness) but this is not the case with the previous approach. And that's why I think the second approach works well.\n\nI'm sharing this because of [this](https://qr.ae/pNycwG)",
    "860362": "Yes, that is correct. Augmentations should almost always be applied on the fly for this reason. One exception is sometimes people will create a larger dataset containing all possible combinations of rotations/flipping to shave off time when loading data.",
    "860399": "Yes, correct! Thanks for validating.",
    "860457": "You're right. The model trained with on the fly augmentations generalizes better as he's seeing more samples comparing to another model having the same epochs and using offline augmentations.",
    "860525": "Do you mean \"create a larger dataset\" without contain the origin dataset ?\n\nI tried to create a larger dataset using augmentations + the origin dataset: LB: 0.6380.\nBut only fly with same augmentations: LB: 0.6359.\n\nSo I still confuse or depend on the random of augmentations with probably p",
    "860542": "In any case, the model will get to see limited and fixed augmentations. Practically I've tried only with static augmented dataset (original dataset + augmented images). Will try on the fly augmentations and update this."
  },
  "source": "meta"
}