{
  "id": 396149,
  "title": "ImageDataGenerator",
  "url": "/competitions/digit-recognizer/discussion/396149",
  "author_name": "Sebastien Gac",
  "post_date": "2023-03-20T14:42:32.498000",
  "votes": 0,
  "comment_count": 0,
  "views": null,
  "content": "<p>Hi!</p>\n<p>How does ImageDataGenerator work?<br>\nDoes it create new batches of entirely new images (modified from image from the existing dataset) so that total number of images = (initial number of images) x (batch size)?<br>\nOr does it make batches from modified existing images so that total number of images = initial number of images?</p>\n<p>When I use model.fit it looks like no new batches are generated. Instead batches are made from the existing dataset.</p>\n<p>Here is part of my code:</p>\n<p>\"<br>\nfrom sklearn.model_selection import train_test_split<br>\nfrom keras.preprocessing import image</p>\n<p>gen = image.ImageDataGenerator()</p>\n<p>X = X_train<br>\ny = y_train<br>\nX_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.10, random_state=42)<br>\nbatches = gen.flow(X_train, y_train, batch_size=64)<br>\nval_batches = gen.flow(X_val, y_val, batch_size=64)</p>\n<p>history=model.fit(batches, steps_per_epoch=batches.n, validation_data=val_batches, <br>\n                  validation_steps=val_batches.n, epochs=3)<br>\n\"</p>\n<p>Thanks<br>\nSebastien</p>",
  "messages": [
    {
      "id": 2189492,
      "postDate": "2023-03-20T14:42:32.497Z",
      "content": "<p>Hi!</p>\n<p>How does ImageDataGenerator work?<br>\nDoes it create new batches of entirely new images (modified from image from the existing dataset) so that total number of images = (initial number of images) x (batch size)?<br>\nOr does it make batches from modified existing images so that total number of images = initial number of images?</p>\n<p>When I use model.fit it looks like no new batches are generated. Instead batches are made from the existing dataset.</p>\n<p>Here is part of my code:</p>\n<p>\"<br>\nfrom sklearn.model_selection import train_test_split<br>\nfrom keras.preprocessing import image</p>\n<p>gen = image.ImageDataGenerator()</p>\n<p>X = X_train<br>\ny = y_train<br>\nX_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.10, random_state=42)<br>\nbatches = gen.flow(X_train, y_train, batch_size=64)<br>\nval_batches = gen.flow(X_val, y_val, batch_size=64)</p>\n<p>history=model.fit(batches, steps_per_epoch=batches.n, validation_data=val_batches, <br>\n                  validation_steps=val_batches.n, epochs=3)<br>\n\"</p>\n<p>Thanks<br>\nSebastien</p>",
      "rawMarkdown": "Hi!\n\nHow does ImageDataGenerator work?\nDoes it create new batches of entirely new images (modified from image from the existing dataset) so that total number of images = (initial number of images) x (batch size)?\nOr does it make batches from modified existing images so that total number of images = initial number of images?\n\nWhen I use model.fit it looks like no new batches are generated. Instead batches are made from the existing dataset.\n\nHere is part of my code:\n\n\"\nfrom sklearn.model_selection import train_test_split\nfrom keras.preprocessing import image\n\ngen = image.ImageDataGenerator()\n\nX = X_train\ny = y_train\nX_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.10, random_state=42)\nbatches = gen.flow(X_train, y_train, batch_size=64)\nval_batches = gen.flow(X_val, y_val, batch_size=64)\n\nhistory=model.fit(batches, steps_per_epoch=batches.n, validation_data=val_batches, \n                  validation_steps=val_batches.n, epochs=3)\n\"\n\nThanks\nSebastien"
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2189492": "Hi!\n\nHow does ImageDataGenerator work?\nDoes it create new batches of entirely new images (modified from image from the existing dataset) so that total number of images = (initial number of images) x (batch size)?\nOr does it make batches from modified existing images so that total number of images = initial number of images?\n\nWhen I use model.fit it looks like no new batches are generated. Instead batches are made from the existing dataset.\n\nHere is part of my code:\n\n\"\nfrom sklearn.model_selection import train_test_split\nfrom keras.preprocessing import image\n\ngen = image.ImageDataGenerator()\n\nX = X_train\ny = y_train\nX_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.10, random_state=42)\nbatches = gen.flow(X_train, y_train, batch_size=64)\nval_batches = gen.flow(X_val, y_val, batch_size=64)\n\nhistory=model.fit(batches, steps_per_epoch=batches.n, validation_data=val_batches, \n                  validation_steps=val_batches.n, epochs=3)\n\"\n\nThanks\nSebastien"
  }
}