{
  "id": 99254,
  "title": "Optimal number of epochs for GAN training?",
  "url": "/competitions/generative-dog-images/discussion/99254",
  "author_name": "",
  "post_date": "2019-07-09T23:58:19.443665400Z",
  "votes": null,
  "comment_count": 4,
  "views": 0,
  "content": "<p>I had initially planned to use the FID metric as an early stopping criterion during offline training but that's clearly not feasible given the time taken to compute it. Has anyone come across some interesting techniques/ideas related to this?</p>",
  "messages": [
    {
      "id": "571651",
      "postDate": "07/09/2019 23:58:19",
      "content": "<p>I had initially planned to use the FID metric as an early stopping criterion during offline training but that's clearly not feasible given the time taken to compute it. Has anyone come across some interesting techniques/ideas related to this?</p>",
      "rawMarkdown": "I had initially planned to use the FID metric as an early stopping criterion during offline training but that's clearly not feasible given the time taken to compute it. Has anyone come across some interesting techniques/ideas related to this?",
      "votes": null
    },
    {
      "id": "571677",
      "postDate": "07/10/2019 01:06:08",
      "content": "<p>it really depends on the GAN architecture, the quality of the original images, loss functions etc \nbut normally a high number, greater than 100 (I personally have not yet tested with epoch greater than 500)</p>",
      "rawMarkdown": "it really depends on the GAN architecture, the quality of the original images, loss functions etc \nbut normally a high number, greater than 100 (I personally have not yet tested with epoch greater than 500)",
      "votes": null
    },
    {
      "id": "571846",
      "postDate": "07/10/2019 06:35:39",
      "content": "<p>For simple GAN architectures I found 100 epoch performs pretty well but going 120+ degrades score considerably. Though it's possible there were spikes and drops in perfomance and I just found one of these.\nWith more recent sophisticated architectures it might be possible to train your net as long as you can without performance drop.</p>",
      "rawMarkdown": "For simple GAN architectures I found 100 epoch performs pretty well but going 120+ degrades score considerably. Though it's possible there were spikes and drops in perfomance and I just found one of these.\nWith more recent sophisticated architectures it might be possible to train your net as long as you can without performance drop.",
      "votes": null
    },
    {
      "id": "571870",
      "postDate": "07/10/2019 07:08:31",
      "content": "<p><a href=\"/dvorobiev\">@dvorobiev</a> can you specify those <em>sophisticated architectures</em>?could be a nice tip :) </p>",
      "rawMarkdown": "dvorobiev can you specify those *sophisticated architectures*?could be a nice tip :)",
      "votes": null
    },
    {
      "id": "571881",
      "postDate": "07/10/2019 07:29:32",
      "content": "<p>In Self-Attention GAN paper the authors claim spectral norm with two-timescale learning rates (TTUR) helps to achieve a monotonic decrease of FID values when the number of iterations goes up.</p>\n\n<p>Though even the spectral norm could be enough given the kernel restrictions...</p>",
      "rawMarkdown": "In Self-Attention GAN paper the authors claim spectral norm with two-timescale learning rates (TTUR) helps to achieve a monotonic decrease of FID values when the number of iterations goes up.\n\nThough even the spectral norm could be enough given the kernel restrictions...",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 571677,
      "author_name": "jesucristo",
      "author_url": "",
      "post_date": "07/10/2019 01:06:08",
      "content": "<p>it really depends on the GAN architecture, the quality of the original images, loss functions etc \nbut normally a high number, greater than 100 (I personally have not yet tested with epoch greater than 500)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 571846,
      "author_name": "dvorobiev",
      "author_url": "",
      "post_date": "07/10/2019 06:35:39",
      "content": "<p>For simple GAN architectures I found 100 epoch performs pretty well but going 120+ degrades score considerably. Though it's possible there were spikes and drops in perfomance and I just found one of these.\nWith more recent sophisticated architectures it might be possible to train your net as long as you can without performance drop.</p>",
      "votes": null,
      "replies": [
        {
          "id": 571870,
          "author_name": "jesucristo",
          "author_url": "",
          "post_date": "07/10/2019 07:08:31",
          "content": "<p><a href=\"/dvorobiev\">@dvorobiev</a> can you specify those <em>sophisticated architectures</em>?could be a nice tip :) </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 571881,
          "author_name": "dvorobiev",
          "author_url": "",
          "post_date": "07/10/2019 07:29:32",
          "content": "<p>In Self-Attention GAN paper the authors claim spectral norm with two-timescale learning rates (TTUR) helps to achieve a monotonic decrease of FID values when the number of iterations goes up.</p>\n\n<p>Though even the spectral norm could be enough given the kernel restrictions...</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "571651": "I had initially planned to use the FID metric as an early stopping criterion during offline training but that's clearly not feasible given the time taken to compute it. Has anyone come across some interesting techniques/ideas related to this?",
    "571677": "it really depends on the GAN architecture, the quality of the original images, loss functions etc \nbut normally a high number, greater than 100 (I personally have not yet tested with epoch greater than 500)",
    "571846": "For simple GAN architectures I found 100 epoch performs pretty well but going 120+ degrades score considerably. Though it's possible there were spikes and drops in perfomance and I just found one of these.\nWith more recent sophisticated architectures it might be possible to train your net as long as you can without performance drop.",
    "571870": "dvorobiev can you specify those *sophisticated architectures*?could be a nice tip :)",
    "571881": "In Self-Attention GAN paper the authors claim spectral norm with two-timescale learning rates (TTUR) helps to achieve a monotonic decrease of FID values when the number of iterations goes up.\n\nThough even the spectral norm could be enough given the kernel restrictions..."
  },
  "source": "meta"
}