{
  "id": 99354,
  "title": "semi-supervised GAN",
  "url": "/competitions/generative-dog-images/discussion/99354",
  "author_name": "",
  "post_date": "2019-07-10T15:43:54.738991100Z",
  "votes": 2,
  "comment_count": 3,
  "views": 0,
  "content": "<p>In supervised learning, we have a training set of inputs and class labels. We train a model that takes as input and gives as output. </p>\n\n<p>In semi-supervised learning, our goal is still to train a model that takes as input and generates as output. However, not all of our training examples have a label. We need to develop an algorithm that is able to get better at classification by studying both labeled pairs and unlabeled examples.</p>\n\n<p>For example, to do this for the SVHN dataset, we'll turn the GAN discriminator into an 11 class discriminator. It will recognize the 10 different classes of real SVHN digits, as well as an 11th class of fake images that come from the generator. </p>\n\n<p>The discriminator will get to train on real labeled images, real unlabeled images, and fake images. By drawing on three sources of data instead of just one, it will generalize to the test set much better than a traditional classifier trained on only one source of data.</p>\n\n<p>In the original work by Tim Salimans at OpenAI, a GAN using more tricks and more runtime reaches over 94% accuracy using only 1,000 labeled examples. [1]</p>\n\n<p>[1] <a href=\"https://arxiv.org/pdf/1606.03498.pdf\">https://arxiv.org/pdf/1606.03498.pdf</a></p>",
  "messages": [
    {
      "id": "572190",
      "postDate": "07/10/2019 15:43:54",
      "content": "<p>In supervised learning, we have a training set of inputs and class labels. We train a model that takes as input and gives as output. </p>\n\n<p>In semi-supervised learning, our goal is still to train a model that takes as input and generates as output. However, not all of our training examples have a label. We need to develop an algorithm that is able to get better at classification by studying both labeled pairs and unlabeled examples.</p>\n\n<p>For example, to do this for the SVHN dataset, we'll turn the GAN discriminator into an 11 class discriminator. It will recognize the 10 different classes of real SVHN digits, as well as an 11th class of fake images that come from the generator. </p>\n\n<p>The discriminator will get to train on real labeled images, real unlabeled images, and fake images. By drawing on three sources of data instead of just one, it will generalize to the test set much better than a traditional classifier trained on only one source of data.</p>\n\n<p>In the original work by Tim Salimans at OpenAI, a GAN using more tricks and more runtime reaches over 94% accuracy using only 1,000 labeled examples. [1]</p>\n\n<p>[1] <a href=\"https://arxiv.org/pdf/1606.03498.pdf\">https://arxiv.org/pdf/1606.03498.pdf</a></p>",
      "rawMarkdown": "In supervised learning, we have a training set of inputs and class labels. We train a model that takes as input and gives as output. \n\nIn semi-supervised learning, our goal is still to train a model that takes as input and generates as output. However, not all of our training examples have a label. We need to develop an algorithm that is able to get better at classification by studying both labeled pairs and unlabeled examples.\n\nFor example, to do this for the SVHN dataset, we'll turn the GAN discriminator into an 11 class discriminator. It will recognize the 10 different classes of real SVHN digits, as well as an 11th class of fake images that come from the generator. \n\nThe discriminator will get to train on real labeled images, real unlabeled images, and fake images. By drawing on three sources of data instead of just one, it will generalize to the test set much better than a traditional classifier trained on only one source of data.\n\nIn the original work by Tim Salimans at OpenAI, a GAN using more tricks and more runtime reaches over 94% accuracy using only 1,000 labeled examples. [1]\n\n[1] https://arxiv.org/pdf/1606.03498.pdf",
      "votes": null
    },
    {
      "id": "572207",
      "postDate": "07/10/2019 16:04:21",
      "content": "<p>Yes semi-supervised GANs can learn quickly. Here are 3 ways to incorporate labels as elaborated by Goodfellow <a href=\"https://www.quora.com/Can-Generative-Adversarial-networks-use-multi-class-labels\">here</a>\n* CGAN (Conditional GAN) <a href=\"https://arxiv.org/abs/1411.1784\">here</a>: You add class labels as inputs but Discriminator still has 1 output.\n* CatGAN (Category GAN) <a href=\"https://arxiv.org/abs/1511.06390\">here</a>: Your Discriminator has N outputs (for N classes of images). Discriminator trains for real images to be classified correctly. Discriminator trains for fake images to score uniform probability across all classes. Generator trains for fake images to score one category strongly. Afterward CatGAN Discriminator is better than typical classifier.\n* SGAN (Semi-supervised GAN) <a href=\"https://arxiv.org/abs/1606.01583\">here</a>: Your Discriminator has N+1 outputs (for N classes of images and 1 for fake).</p>",
      "rawMarkdown": "Yes semi-supervised GANs can learn quickly. Here are 3 ways to incorporate labels as elaborated by Goodfellow [here][4]\n* CGAN (Conditional GAN) [here][1]: You add class labels as inputs but Discriminator still has 1 output.\n* CatGAN (Category GAN) [here][2]: Your Discriminator has N outputs (for N classes of images). Discriminator trains for real images to be classified correctly. Discriminator trains for fake images to score uniform probability across all classes. Generator trains for fake images to score one category strongly. Afterward CatGAN Discriminator is better than typical classifier.\n* SGAN (Semi-supervised GAN) [here][3]: Your Discriminator has N+1 outputs (for N classes of images and 1 for fake).\n\n[1]: https://arxiv.org/abs/1411.1784\n[2]: https://arxiv.org/abs/1511.06390\n[3]: https://arxiv.org/abs/1606.01583\n[4]: https://www.quora.com/Can-Generative-Adversarial-networks-use-multi-class-labels",
      "votes": null
    },
    {
      "id": "572463",
      "postDate": "07/11/2019 01:04:03",
      "content": "<p>Thanks for sharing, Chris.</p>",
      "rawMarkdown": "Thanks for sharing, Chris.",
      "votes": null
    },
    {
      "id": "572464",
      "postDate": "07/11/2019 01:06:38",
      "content": "<p>Refer to the kernel: <a href=\"https://www.kaggle.com/grapestone5321/semi-supervised-gan\">https://www.kaggle.com/grapestone5321/semi-supervised-gan</a></p>",
      "rawMarkdown": "Refer to the kernel: https://www.kaggle.com/grapestone5321/semi-supervised-gan",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 572207,
      "author_name": "cdeotte",
      "author_url": "",
      "post_date": "07/10/2019 16:04:21",
      "content": "<p>Yes semi-supervised GANs can learn quickly. Here are 3 ways to incorporate labels as elaborated by Goodfellow <a href=\"https://www.quora.com/Can-Generative-Adversarial-networks-use-multi-class-labels\">here</a>\n* CGAN (Conditional GAN) <a href=\"https://arxiv.org/abs/1411.1784\">here</a>: You add class labels as inputs but Discriminator still has 1 output.\n* CatGAN (Category GAN) <a href=\"https://arxiv.org/abs/1511.06390\">here</a>: Your Discriminator has N outputs (for N classes of images). Discriminator trains for real images to be classified correctly. Discriminator trains for fake images to score uniform probability across all classes. Generator trains for fake images to score one category strongly. Afterward CatGAN Discriminator is better than typical classifier.\n* SGAN (Semi-supervised GAN) <a href=\"https://arxiv.org/abs/1606.01583\">here</a>: Your Discriminator has N+1 outputs (for N classes of images and 1 for fake).</p>",
      "votes": null,
      "replies": [
        {
          "id": 572463,
          "author_name": "grapestone5321",
          "author_url": "",
          "post_date": "07/11/2019 01:04:03",
          "content": "<p>Thanks for sharing, Chris.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 572464,
      "author_name": "grapestone5321",
      "author_url": "",
      "post_date": "07/11/2019 01:06:38",
      "content": "<p>Refer to the kernel: <a href=\"https://www.kaggle.com/grapestone5321/semi-supervised-gan\">https://www.kaggle.com/grapestone5321/semi-supervised-gan</a></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "572190": "In supervised learning, we have a training set of inputs and class labels. We train a model that takes as input and gives as output. \n\nIn semi-supervised learning, our goal is still to train a model that takes as input and generates as output. However, not all of our training examples have a label. We need to develop an algorithm that is able to get better at classification by studying both labeled pairs and unlabeled examples.\n\nFor example, to do this for the SVHN dataset, we'll turn the GAN discriminator into an 11 class discriminator. It will recognize the 10 different classes of real SVHN digits, as well as an 11th class of fake images that come from the generator. \n\nThe discriminator will get to train on real labeled images, real unlabeled images, and fake images. By drawing on three sources of data instead of just one, it will generalize to the test set much better than a traditional classifier trained on only one source of data.\n\nIn the original work by Tim Salimans at OpenAI, a GAN using more tricks and more runtime reaches over 94% accuracy using only 1,000 labeled examples. [1]\n\n[1] https://arxiv.org/pdf/1606.03498.pdf",
    "572207": "Yes semi-supervised GANs can learn quickly. Here are 3 ways to incorporate labels as elaborated by Goodfellow [here][4]\n* CGAN (Conditional GAN) [here][1]: You add class labels as inputs but Discriminator still has 1 output.\n* CatGAN (Category GAN) [here][2]: Your Discriminator has N outputs (for N classes of images). Discriminator trains for real images to be classified correctly. Discriminator trains for fake images to score uniform probability across all classes. Generator trains for fake images to score one category strongly. Afterward CatGAN Discriminator is better than typical classifier.\n* SGAN (Semi-supervised GAN) [here][3]: Your Discriminator has N+1 outputs (for N classes of images and 1 for fake).\n\n[1]: https://arxiv.org/abs/1411.1784\n[2]: https://arxiv.org/abs/1511.06390\n[3]: https://arxiv.org/abs/1606.01583\n[4]: https://www.quora.com/Can-Generative-Adversarial-networks-use-multi-class-labels",
    "572463": "Thanks for sharing, Chris.",
    "572464": "Refer to the kernel: https://www.kaggle.com/grapestone5321/semi-supervised-gan"
  },
  "source": "meta"
}