{
  "id": 99613,
  "title": "Error in Starter Code",
  "url": "/competitions/generative-dog-images/discussion/99613",
  "author_name": "",
  "post_date": "2019-07-12T16:15:14.969338300Z",
  "votes": 25,
  "comment_count": 7,
  "views": 0,
  "content": "<h2>Increase your LB 20 points!</h2>\n\n<p>If you are using the PyTorch Kaggle kernel starter code, I would like to point out that it contains an error. We need to un-normalize the images before saving (and displaying). The images will appear lighter and your LB will increase by up to 20 points. (The top images score LB 130, the bottom score LB 110). This also makes viewing the images more enjoyable.</p>\n\n<h2>Dark Images</h2>\n\n<pre><code>save_image(gen_images[i_image, :, :, :], os.path.join('../output_images', f'image_{i_batch+i_image:05d}.png'))\n</code></pre>\n\n<p><img src=\"http://playagricola.com/Kaggle/dark71219.png\" alt=\"image\"></p>\n\n<h2>Light Images</h2>\n\n<pre><code>save_image( (gen_images[i_image, :, :, :]+1.)/2. , os.path.join('../output_images', f'image_{i_batch+i_image:05d}.png'))\n</code></pre>\n\n<p><img src=\"http://playagricola.com/Kaggle/light71219.png\" alt=\"image\"></p>",
  "messages": [
    {
      "id": "573697",
      "postDate": "07/12/2019 16:15:14",
      "content": "<h2>Increase your LB 20 points!</h2>\n\n<p>If you are using the PyTorch Kaggle kernel starter code, I would like to point out that it contains an error. We need to un-normalize the images before saving (and displaying). The images will appear lighter and your LB will increase by up to 20 points. (The top images score LB 130, the bottom score LB 110). This also makes viewing the images more enjoyable.</p>\n\n<h2>Dark Images</h2>\n\n<pre><code>save_image(gen_images[i_image, :, :, :], os.path.join('../output_images', f'image_{i_batch+i_image:05d}.png'))\n</code></pre>\n\n<p><img src=\"http://playagricola.com/Kaggle/dark71219.png\" alt=\"image\"></p>\n\n<h2>Light Images</h2>\n\n<pre><code>save_image( (gen_images[i_image, :, :, :]+1.)/2. , os.path.join('../output_images', f'image_{i_batch+i_image:05d}.png'))\n</code></pre>\n\n<p><img src=\"http://playagricola.com/Kaggle/light71219.png\" alt=\"image\"></p>",
      "rawMarkdown": "## Increase your LB 20 points!\nIf you are using the PyTorch Kaggle kernel starter code, I would like to point out that it contains an error. We need to un-normalize the images before saving (and displaying). The images will appear lighter and your LB will increase by up to 20 points. (The top images score LB 130, the bottom score LB 110). This also makes viewing the images more enjoyable.\n\n## Dark Images\n    save_image(gen_images[i_image, :, :, :], os.path.join('../output_images', f'image_{i_batch+i_image:05d}.png'))\n![image](http://playagricola.com/Kaggle/dark71219.png)\n## Light Images\n    save_image( (gen_images[i_image, :, :, :]+1.)/2. , os.path.join('../output_images', f'image_{i_batch+i_image:05d}.png'))\n![image](http://playagricola.com/Kaggle/light71219.png)",
      "votes": null
    },
    {
      "id": "573702",
      "postDate": "07/12/2019 16:20:16",
      "content": "<p>I have tried this trick and I confirm . It increases LB score drastically . .</p>",
      "rawMarkdown": "I have tried this trick and I confirm . It increases LB score drastically . .",
      "votes": null
    },
    {
      "id": "573712",
      "postDate": "07/12/2019 16:42:35",
      "content": "<p>deja vu? \nI somehow remember you pointed this out (or something very similar) somewhere in a discussion. I fixed it in my kernel which is currently running.\nThanks again!</p>",
      "rawMarkdown": "deja vu? \nI somehow remember you pointed this out (or something very similar) somewhere in a discussion. I fixed it in my kernel which is currently running.\nThanks again!",
      "votes": null
    },
    {
      "id": "573719",
      "postDate": "07/12/2019 16:53:38",
      "content": "<p>Great. I added it as a footnote to my RaLSGAN discussion post <a href=\"https://www.kaggle.com/c/generative-dog-images/discussion/99485\">here</a>. I just made this separate post in case people missed it. There are a few public kernels that would increase their LB with this change. Also Nirjhar pointed this out previously <a href=\"https://www.kaggle.com/c/generative-dog-images/discussion/99162\">here</a></p>",
      "rawMarkdown": "Great. I added it as a footnote to my RaLSGAN discussion post [here][1]. I just made this separate post in case people missed it. There are a few public kernels that would increase their LB with this change. Also Nirjhar pointed this out previously [here][2]\n\n[1]: https://www.kaggle.com/c/generative-dog-images/discussion/99485\n[2]: https://www.kaggle.com/c/generative-dog-images/discussion/99162",
      "votes": null
    },
    {
      "id": "573738",
      "postDate": "07/12/2019 17:16:33",
      "content": "<p>Yes! Now I remember. I learn a lot from you... more than my university. </p>",
      "rawMarkdown": "Yes! Now I remember. I learn a lot from you... more than my university.",
      "votes": null
    },
    {
      "id": "574362",
      "postDate": "07/13/2019 17:19:58",
      "content": "<p><code>Chris</code> How you interpret <code>G</code> and <code>D</code> loss?</p>",
      "rawMarkdown": "`Chris` How you interpret `G` and `D` loss?",
      "votes": null
    },
    {
      "id": "574379",
      "postDate": "07/13/2019 18:19:26",
      "content": "<p>The Generator loss shows how often the Generator fools the Discriminator with 0 being 100% and 1 being 37% (when using basic loss function). The Discriminator loss shows both how well the Discriminator is identifying real images and how well its identifying fake images. And the value is harder to interpret, see below.</p>\n\n<h2>Long Answer</h2>\n\n<p>G and D loss are the average loss per image fed into a network. The interpretation of the value depends on the type of loss. If you're using basic GAN (or DCGAN) loss then, <code>gen_loss = -log ( D(x_f) )</code> from <a href=\"https://www.kaggle.com/c/generative-dog-images/discussion/99485\">here</a>. Here <code>D(x_f)</code> is the probability that <code>x_f</code> is real  (where x_f is a generated image). So <code>gen_loss</code> is the average negative log of the probability of a generated image being declared real.</p>\n\n<h3>Generator</h3>\n\n<p>If <code>gen_loss = 0.3 = -log(0.75)</code> then the Generator is fooling the Discriminator 75% of the time. \nIf <code>gen_loss = 0.7 = -log(0.5)</code> then the Generator is fooling the Discriminator 50% of the time. \nIf <code>gen_loss = 1.4 = -log(0.25)</code> then the Generator is fooling the Discriminator 25% of the time.</p>\n\n<h3>Discriminator</h3>\n\n<p>The Discriminator loss is trickier to interpret because it is the sum of two losses, <code>disc_loss = -log (D(x_r)) - log (1-D(x_f))</code>. So you must use the <code>gen_loss</code> to help interpret the <code>disc_loss</code>:</p>\n\n<p>If Generator is fooling the Discriminator 25% of the time...\nthen <code>disc_loss = 0.3+0.3</code> means the Discriminator is correctly identifying real images 75% of the time. \nthen <code>disc_loss = 0.7+0.3</code> means the Discriminator is correctly identifying real images 50% of the time. \nthen <code>disc_loss = 1.4+0.3</code> means the Discriminator is correctly identifying real images 25% of the time.</p>\n\n<p>If Generator is fooling the Discriminator 50% of the time...\nthen <code>disc_loss = 0.3+0.7</code> means the Discriminator is correctly identifying real images 75% of the time. \nthen <code>disc_loss = 0.7+0.7</code> means the Discriminator is correctly identifying real images 50% of the time. \nthen <code>disc_loss = 1.4+0.7</code> means the Discriminator is correctly identifying real images 25% of the time.</p>\n\n<p>If Generator is fooling the Discriminator 75% of the time...\nthen <code>disc_loss = 0.3+1.4</code> means the Discriminator is correctly identifying real images 75% of the time. \nthen <code>disc_loss = 0.7+1.4</code> means the Discriminator is correctly identifying real images 50% of the time. \nthen <code>disc_loss = 1.4+1.4</code> means the Discriminator is correctly identifying real images 25% of the time.</p>\n\n<h3>RaGAN and LSGAN</h3>\n\n<p>The above are not true for RaGAN, LSGAN, RaLSGAN etc because they use different loss functions.</p>",
      "rawMarkdown": "The Generator loss shows how often the Generator fools the Discriminator with 0 being 100% and 1 being 37% (when using basic loss function). The Discriminator loss shows both how well the Discriminator is identifying real images and how well its identifying fake images. And the value is harder to interpret, see below.\n\n## Long Answer\nG and D loss are the average loss per image fed into a network. The interpretation of the value depends on the type of loss. If you're using basic GAN (or DCGAN) loss then, `gen_loss = -log ( D(x_f) )` from [here][1]. Here `D(x_f)` is the probability that `x_f` is real  (where x_f is a generated image). So `gen_loss` is the average negative log of the probability of a generated image being declared real.\n\n### Generator\nIf `gen_loss = 0.3 = -log(0.75)` then the Generator is fooling the Discriminator 75% of the time. \nIf `gen_loss = 0.7 = -log(0.5)` then the Generator is fooling the Discriminator 50% of the time. \nIf `gen_loss = 1.4 = -log(0.25)` then the Generator is fooling the Discriminator 25% of the time.\n\n### Discriminator\nThe Discriminator loss is trickier to interpret because it is the sum of two losses, `disc_loss = -log (D(x_r)) - log (1-D(x_f))`. So you must use the `gen_loss` to help interpret the `disc_loss`:\n\nIf Generator is fooling the Discriminator 25% of the time...\nthen `disc_loss = 0.3+0.3` means the Discriminator is correctly identifying real images 75% of the time. \nthen `disc_loss = 0.7+0.3` means the Discriminator is correctly identifying real images 50% of the time. \nthen `disc_loss = 1.4+0.3` means the Discriminator is correctly identifying real images 25% of the time.\n\nIf Generator is fooling the Discriminator 50% of the time...\nthen `disc_loss = 0.3+0.7` means the Discriminator is correctly identifying real images 75% of the time. \nthen `disc_loss = 0.7+0.7` means the Discriminator is correctly identifying real images 50% of the time. \nthen `disc_loss = 1.4+0.7` means the Discriminator is correctly identifying real images 25% of the time.\n\nIf Generator is fooling the Discriminator 75% of the time...\nthen `disc_loss = 0.3+1.4` means the Discriminator is correctly identifying real images 75% of the time. \nthen `disc_loss = 0.7+1.4` means the Discriminator is correctly identifying real images 50% of the time. \nthen `disc_loss = 1.4+1.4` means the Discriminator is correctly identifying real images 25% of the time.\n### RaGAN and LSGAN\nThe above are not true for RaGAN, LSGAN, RaLSGAN etc because they use different loss functions.\n\n[1]: https://www.kaggle.com/c/generative-dog-images/discussion/99485",
      "votes": null
    },
    {
      "id": "574385",
      "postDate": "07/13/2019 18:33:42",
      "content": "<p>Thanks <code>Chris</code> 👍 </p>",
      "rawMarkdown": "Thanks `Chris` 👍",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 573702,
      "author_name": "phoenix9032",
      "author_url": "",
      "post_date": "07/12/2019 16:20:16",
      "content": "<p>I have tried this trick and I confirm . It increases LB score drastically . .</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 573712,
      "author_name": "timetraveller98",
      "author_url": "",
      "post_date": "07/12/2019 16:42:35",
      "content": "<p>deja vu? \nI somehow remember you pointed this out (or something very similar) somewhere in a discussion. I fixed it in my kernel which is currently running.\nThanks again!</p>",
      "votes": null,
      "replies": [
        {
          "id": 573719,
          "author_name": "cdeotte",
          "author_url": "",
          "post_date": "07/12/2019 16:53:38",
          "content": "<p>Great. I added it as a footnote to my RaLSGAN discussion post <a href=\"https://www.kaggle.com/c/generative-dog-images/discussion/99485\">here</a>. I just made this separate post in case people missed it. There are a few public kernels that would increase their LB with this change. Also Nirjhar pointed this out previously <a href=\"https://www.kaggle.com/c/generative-dog-images/discussion/99162\">here</a></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 573738,
          "author_name": "timetraveller98",
          "author_url": "",
          "post_date": "07/12/2019 17:16:33",
          "content": "<p>Yes! Now I remember. I learn a lot from you... more than my university. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 574362,
      "author_name": "coolcoder22",
      "author_url": "",
      "post_date": "07/13/2019 17:19:58",
      "content": "<p><code>Chris</code> How you interpret <code>G</code> and <code>D</code> loss?</p>",
      "votes": null,
      "replies": [
        {
          "id": 574379,
          "author_name": "cdeotte",
          "author_url": "",
          "post_date": "07/13/2019 18:19:26",
          "content": "<p>The Generator loss shows how often the Generator fools the Discriminator with 0 being 100% and 1 being 37% (when using basic loss function). The Discriminator loss shows both how well the Discriminator is identifying real images and how well its identifying fake images. And the value is harder to interpret, see below.</p>\n\n<h2>Long Answer</h2>\n\n<p>G and D loss are the average loss per image fed into a network. The interpretation of the value depends on the type of loss. If you're using basic GAN (or DCGAN) loss then, <code>gen_loss = -log ( D(x_f) )</code> from <a href=\"https://www.kaggle.com/c/generative-dog-images/discussion/99485\">here</a>. Here <code>D(x_f)</code> is the probability that <code>x_f</code> is real  (where x_f is a generated image). So <code>gen_loss</code> is the average negative log of the probability of a generated image being declared real.</p>\n\n<h3>Generator</h3>\n\n<p>If <code>gen_loss = 0.3 = -log(0.75)</code> then the Generator is fooling the Discriminator 75% of the time. \nIf <code>gen_loss = 0.7 = -log(0.5)</code> then the Generator is fooling the Discriminator 50% of the time. \nIf <code>gen_loss = 1.4 = -log(0.25)</code> then the Generator is fooling the Discriminator 25% of the time.</p>\n\n<h3>Discriminator</h3>\n\n<p>The Discriminator loss is trickier to interpret because it is the sum of two losses, <code>disc_loss = -log (D(x_r)) - log (1-D(x_f))</code>. So you must use the <code>gen_loss</code> to help interpret the <code>disc_loss</code>:</p>\n\n<p>If Generator is fooling the Discriminator 25% of the time...\nthen <code>disc_loss = 0.3+0.3</code> means the Discriminator is correctly identifying real images 75% of the time. \nthen <code>disc_loss = 0.7+0.3</code> means the Discriminator is correctly identifying real images 50% of the time. \nthen <code>disc_loss = 1.4+0.3</code> means the Discriminator is correctly identifying real images 25% of the time.</p>\n\n<p>If Generator is fooling the Discriminator 50% of the time...\nthen <code>disc_loss = 0.3+0.7</code> means the Discriminator is correctly identifying real images 75% of the time. \nthen <code>disc_loss = 0.7+0.7</code> means the Discriminator is correctly identifying real images 50% of the time. \nthen <code>disc_loss = 1.4+0.7</code> means the Discriminator is correctly identifying real images 25% of the time.</p>\n\n<p>If Generator is fooling the Discriminator 75% of the time...\nthen <code>disc_loss = 0.3+1.4</code> means the Discriminator is correctly identifying real images 75% of the time. \nthen <code>disc_loss = 0.7+1.4</code> means the Discriminator is correctly identifying real images 50% of the time. \nthen <code>disc_loss = 1.4+1.4</code> means the Discriminator is correctly identifying real images 25% of the time.</p>\n\n<h3>RaGAN and LSGAN</h3>\n\n<p>The above are not true for RaGAN, LSGAN, RaLSGAN etc because they use different loss functions.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 574385,
          "author_name": "coolcoder22",
          "author_url": "",
          "post_date": "07/13/2019 18:33:42",
          "content": "<p>Thanks <code>Chris</code> 👍 </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "573697": "## Increase your LB 20 points!\nIf you are using the PyTorch Kaggle kernel starter code, I would like to point out that it contains an error. We need to un-normalize the images before saving (and displaying). The images will appear lighter and your LB will increase by up to 20 points. (The top images score LB 130, the bottom score LB 110). This also makes viewing the images more enjoyable.\n\n## Dark Images\n    save_image(gen_images[i_image, :, :, :], os.path.join('../output_images', f'image_{i_batch+i_image:05d}.png'))\n![image](http://playagricola.com/Kaggle/dark71219.png)\n## Light Images\n    save_image( (gen_images[i_image, :, :, :]+1.)/2. , os.path.join('../output_images', f'image_{i_batch+i_image:05d}.png'))\n![image](http://playagricola.com/Kaggle/light71219.png)",
    "573702": "I have tried this trick and I confirm . It increases LB score drastically . .",
    "573712": "deja vu? \nI somehow remember you pointed this out (or something very similar) somewhere in a discussion. I fixed it in my kernel which is currently running.\nThanks again!",
    "573719": "Great. I added it as a footnote to my RaLSGAN discussion post [here][1]. I just made this separate post in case people missed it. There are a few public kernels that would increase their LB with this change. Also Nirjhar pointed this out previously [here][2]\n\n[1]: https://www.kaggle.com/c/generative-dog-images/discussion/99485\n[2]: https://www.kaggle.com/c/generative-dog-images/discussion/99162",
    "573738": "Yes! Now I remember. I learn a lot from you... more than my university.",
    "574362": "`Chris` How you interpret `G` and `D` loss?",
    "574379": "The Generator loss shows how often the Generator fools the Discriminator with 0 being 100% and 1 being 37% (when using basic loss function). The Discriminator loss shows both how well the Discriminator is identifying real images and how well its identifying fake images. And the value is harder to interpret, see below.\n\n## Long Answer\nG and D loss are the average loss per image fed into a network. The interpretation of the value depends on the type of loss. If you're using basic GAN (or DCGAN) loss then, `gen_loss = -log ( D(x_f) )` from [here][1]. Here `D(x_f)` is the probability that `x_f` is real  (where x_f is a generated image). So `gen_loss` is the average negative log of the probability of a generated image being declared real.\n\n### Generator\nIf `gen_loss = 0.3 = -log(0.75)` then the Generator is fooling the Discriminator 75% of the time. \nIf `gen_loss = 0.7 = -log(0.5)` then the Generator is fooling the Discriminator 50% of the time. \nIf `gen_loss = 1.4 = -log(0.25)` then the Generator is fooling the Discriminator 25% of the time.\n\n### Discriminator\nThe Discriminator loss is trickier to interpret because it is the sum of two losses, `disc_loss = -log (D(x_r)) - log (1-D(x_f))`. So you must use the `gen_loss` to help interpret the `disc_loss`:\n\nIf Generator is fooling the Discriminator 25% of the time...\nthen `disc_loss = 0.3+0.3` means the Discriminator is correctly identifying real images 75% of the time. \nthen `disc_loss = 0.7+0.3` means the Discriminator is correctly identifying real images 50% of the time. \nthen `disc_loss = 1.4+0.3` means the Discriminator is correctly identifying real images 25% of the time.\n\nIf Generator is fooling the Discriminator 50% of the time...\nthen `disc_loss = 0.3+0.7` means the Discriminator is correctly identifying real images 75% of the time. \nthen `disc_loss = 0.7+0.7` means the Discriminator is correctly identifying real images 50% of the time. \nthen `disc_loss = 1.4+0.7` means the Discriminator is correctly identifying real images 25% of the time.\n\nIf Generator is fooling the Discriminator 75% of the time...\nthen `disc_loss = 0.3+1.4` means the Discriminator is correctly identifying real images 75% of the time. \nthen `disc_loss = 0.7+1.4` means the Discriminator is correctly identifying real images 50% of the time. \nthen `disc_loss = 1.4+1.4` means the Discriminator is correctly identifying real images 25% of the time.\n### RaGAN and LSGAN\nThe above are not true for RaGAN, LSGAN, RaLSGAN etc because they use different loss functions.\n\n[1]: https://www.kaggle.com/c/generative-dog-images/discussion/99485",
    "574385": "Thanks `Chris` 👍"
  },
  "source": "meta"
}