{
  "id": 100945,
  "title": "Batch size 16->64 make result worse, but why?",
  "url": "/competitions/generative-dog-images/discussion/100945",
  "author_name": "",
  "post_date": "2019-07-22T09:10:58.112911800Z",
  "votes": 3,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I tested different batch size in the same kernel and the result shows 16 is better than 64.\nThe LB score from 76-&gt;94.\nI thought bigger batch size could make the gradient aim to a better direction.\nSo is anyone got the same result and is there any research about the correlation in gan and batch size?</p>",
  "messages": [
    {
      "id": "581693",
      "postDate": "07/22/2019 09:10:58",
      "content": "<p>I tested different batch size in the same kernel and the result shows 16 is better than 64.\nThe LB score from 76-&gt;94.\nI thought bigger batch size could make the gradient aim to a better direction.\nSo is anyone got the same result and is there any research about the correlation in gan and batch size?</p>",
      "rawMarkdown": "I tested different batch size in the same kernel and the result shows 16 is better than 64.\nThe LB score from 76-&gt;94.\nI thought bigger batch size could make the gradient aim to a better direction.\nSo is anyone got the same result and is there any research about the correlation in gan and batch size?",
      "votes": null
    },
    {
      "id": "581816",
      "postDate": "07/22/2019 12:41:53",
      "content": "<p>A smaller batch size makes the training way slower . So I have made my peace with 32 . Per epoch it takes about 30/35 seconds . </p>",
      "rawMarkdown": "A smaller batch size makes the training way slower . So I have made my peace with 32 . Per epoch it takes about 30/35 seconds .",
      "votes": null
    },
    {
      "id": "581910",
      "postDate": "07/22/2019 14:22:57",
      "content": "<p>I think maybe discriminator could converge quickly with larger batch size, and if you are using dcgan, generator will get no gradient if discriminator is converged, the loss will be a constant (it's not a good idea to let discriminator could easily classify real and generated samples when generator is bad, wgan is designed for this. And larger batch size may cause mode collapse ( if I'm wrong please correct me :)</p>",
      "rawMarkdown": "I think maybe discriminator could converge quickly with larger batch size, and if you are using dcgan, generator will get no gradient if discriminator is converged, the loss will be a constant (it's not a good idea to let discriminator could easily classify real and generated samples when generator is bad, wgan is designed for this. And larger batch size may cause mode collapse ( if I'm wrong please correct me :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 581816,
      "author_name": "phoenix9032",
      "author_url": "",
      "post_date": "07/22/2019 12:41:53",
      "content": "<p>A smaller batch size makes the training way slower . So I have made my peace with 32 . Per epoch it takes about 30/35 seconds . </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 581910,
      "author_name": "jionie",
      "author_url": "",
      "post_date": "07/22/2019 14:22:57",
      "content": "<p>I think maybe discriminator could converge quickly with larger batch size, and if you are using dcgan, generator will get no gradient if discriminator is converged, the loss will be a constant (it's not a good idea to let discriminator could easily classify real and generated samples when generator is bad, wgan is designed for this. And larger batch size may cause mode collapse ( if I'm wrong please correct me :)</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "581693": "I tested different batch size in the same kernel and the result shows 16 is better than 64.\nThe LB score from 76-&gt;94.\nI thought bigger batch size could make the gradient aim to a better direction.\nSo is anyone got the same result and is there any research about the correlation in gan and batch size?",
    "581816": "A smaller batch size makes the training way slower . So I have made my peace with 32 . Per epoch it takes about 30/35 seconds .",
    "581910": "I think maybe discriminator could converge quickly with larger batch size, and if you are using dcgan, generator will get no gradient if discriminator is converged, the loss will be a constant (it's not a good idea to let discriminator could easily classify real and generated samples when generator is bad, wgan is designed for this. And larger batch size may cause mode collapse ( if I'm wrong please correct me :)"
  },
  "source": "meta"
}