{ "id": 104170, "title": "My approach (gold?)", "url": "/competitions/generative-dog-images/discussion/104170", "author_name": "Nanashi", "post_date": "2019-08-14T22:47:38.073000", "votes": 26, "comment_count": 19, "views": 0, "content": "
\n

I want to share this and go to sleep... Good luck!

\n
\n\n

As you can read at The best GAN performance and goog luck! SNGAN really outperforms, so I decided to focus on that.

\n\n
    \n
  1. I read the original paper: Spectral Normalization for Generative Adversarial Networks
    2. \n
  2. Following the original authors, I found out this amazing paper: cGANs with Projection Discriminator
    3. \n
\n\n

\"\"

\n\n

\"\"

\n\n

The performance using projection was amazing!

\n\n

This is the original repo where you can find the code: pfnet-research

\n\n

\"\"\n\"\"

\n\n

In this other repo, you can find another implementation: sngan_projection_TensorFlow

\n\n\n\n

And finally I found this one: Spectral Normalization and Projection Discriminator(Pytorch)

\n\n

\"\"

\n\n

I tried a lot of architectures: GANs, DCGANs, WGAN, RaLSGAN, CycleGAN... in this case we have labels so CGANs are a great option + projection and other tricks...\nMy 2nd option are VAE-GANs ;)

\n\n

I took holidays and I couldn't work too much in this approach ... hope this helps and please let me know if you realized about this :) the papers are from 2018 so this technique is like a \"new\" approach I guess, probably that's why is not so famous. I teamed up because I didn't have time... but my teammates couldn't work at all on these approaches so their solution is compete different from this.

", "messages": [ { "id": 599387, "postDate": "2019-08-14T22:47:38.073Z", "content": "
\n

I want to share this and go to sleep... Good luck!

\n
\n\n

As you can read at The best GAN performance and goog luck! SNGAN really outperforms, so I decided to focus on that.

\n\n
    \n
  1. I read the original paper: Spectral Normalization for Generative Adversarial Networks
    2. \n
  2. Following the original authors, I found out this amazing paper: cGANs with Projection Discriminator
    3. \n
\n\n

\"\"

\n\n

\"\"

\n\n

The performance using projection was amazing!

\n\n

This is the original repo where you can find the code: pfnet-research

\n\n

\"\"\n\"\"

\n\n

In this other repo, you can find another implementation: sngan_projection_TensorFlow

\n\n\n\n

And finally I found this one: Spectral Normalization and Projection Discriminator(Pytorch)

\n\n

\"\"

\n\n

I tried a lot of architectures: GANs, DCGANs, WGAN, RaLSGAN, CycleGAN... in this case we have labels so CGANs are a great option + projection and other tricks...\nMy 2nd option are VAE-GANs ;)

\n\n

I took holidays and I couldn't work too much in this approach ... hope this helps and please let me know if you realized about this :) the papers are from 2018 so this technique is like a \"new\" approach I guess, probably that's why is not so famous. I teamed up because I didn't have time... but my teammates couldn't work at all on these approaches so their solution is compete different from this.

", "rawMarkdown": "> I want to share this and go to sleep... Good luck!\n\nAs you can read at [The best GAN performance and goog luck!](https://www.kaggle.com/c/generative-dog-images/discussion/104167#latest-599374) SNGAN really outperforms, so I decided to focus on that.\n\n1. I read the original paper: [Spectral Normalization for Generative Adversarial Networks](https://arxiv.org/pdf/1802.05957.pdf)\n2. Following the original authors, I found out this amazing paper: [cGANs with Projection Discriminator](https://arxiv.org/pdf/1802.05637.pdf)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2779868%2Fc503bb6e1dc565237b8f08dab530d18d%2FScreenshot%20from%202019-08-15%2000-12-39.png?generation=1565820871058717&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2779868%2F88fcdaf6f6a3f2fbaa3eaafd6a4d57ab%2FScreenshot%20from%202019-08-15%2000-13-12.png?generation=1565820872307693&alt=media)\n\nThe performance using **projection** was amazing!\n\nThis is the original repo where you can find the code: [pfnet-research](https://github.com/pfnet-research/sngan_projection)\n\n![](https://raw.githubusercontent.com/pfnet-research/sngan_projection/master/demo/dog_and_cat_1x1_long.gif)\n![](https://raw.githubusercontent.com/pfnet-research/sngan_projection/master/demo/images_dog_and_cat_64.jpg)\n\nIn this other repo, you can find another implementation: [sngan_projection_TensorFlow](https://github.com/MingtaoGuo/sngan_projection_TensorFlow)\n\n- conditional_batchnorm\n- projection\n\nAnd finally I found this one: [Spectral Normalization and Projection Discriminator(Pytorch)](https://github.com/balansky/pytorch_gan)\n\n![](https://raw.githubusercontent.com/balansky/pytorch_gan/master/samples/250000_img.png)\n\nI tried a lot of architectures: GANs, DCGANs, WGAN, RaLSGAN, CycleGAN... in this case we have labels so CGANs are a great option + projection and other tricks...\nMy 2nd option are VAE-GANs ;)\n\nI took holidays and I couldn't work too much in this approach ... hope this helps and please let me know if you realized about this :) the papers are from 2018 so this technique is like a \"new\" approach I guess, probably that's why is not so famous. I teamed up because I didn't have time... but my teammates couldn't work at all on these approaches so their solution is compete different from this.\n\n", "votes": 26 }, { "id": 599402, "postDate": "2019-08-14T23:13:50.517Z", "content": "

Great work Nanashi, I also used that GAN :-) I found the GitHub the day before yesterday so I didn't have time to tune it well. I only got it to LB 47 with two overnight experiments. What settings did you use and what preprocessing did you do to the dog images? (I used the original GitHub from the authors).

\n\n

", "rawMarkdown": "Great work Nanashi, I also used that GAN :-) I found the GitHub the day before yesterday so I didn't have time to tune it well. I only got it to LB 47 with two overnight experiments. What settings did you use and what preprocessing did you do to the dog images? (I used the original GitHub from the authors).\n \n \n", "votes": 1, "replies": [ { "id": 599594, "postDate": "2019-08-15T06:56:53.280Z", "content": "

thank you so much @cdeotte :) I think I'll score around 30-40. As I said I couldn't work so much in this approach... but I think it's really interesting, both papers are great.

\n\n

btw, I never thought of using biggan, really cool 👍

\n\n

PS: I can't see the picture...

", "rawMarkdown": "thank you so much @cdeotte :) I think I'll score around 30-40. As I said I couldn't work so much in this approach... but I think it's really interesting, both papers are great.\n\nbtw, I never thought of using biggan, really cool 👍\n\nPS: I can't see the picture..." }, { "id": 599602, "postDate": "2019-08-15T07:05:03.387Z", "content": "

also, I'm a noob so I have to ask...

\n\n
\n

how did you face this comp?

\n
\n\n

I mean, did you try several architectures until BigGan? did you find BigGan last weeks? or you knew about it before?

\n\n

Thanks!

", "rawMarkdown": "also, I'm a noob so I have to ask...\n> how did you face this comp?\n\nI mean, did you try several architectures until BigGan? did you find BigGan last weeks? or you knew about it before?\n\nThanks!" }, { "id": 599629, "postDate": "2019-08-15T07:45:20.740Z", "content": "

I started with WGAN / DCGAN. Then I watched a few talks from CVPR 2018, namely Ian Goodfellow speech on general state of GAN models. So I found Self-Attention GAN first and was able to get ~40 MiFID out of it. I don't remember how I found BigGAN, but it took me a while to set it up in a way, which gave me the better perfomance than SAGAN.

", "rawMarkdown": "I started with WGAN / DCGAN. Then I watched a few talks from CVPR 2018, namely Ian Goodfellow speech on general state of GAN models. So I found Self-Attention GAN first and was able to get ~40 MiFID out of it. I don't remember how I found BigGAN, but it took me a while to set it up in a way, which gave me the better perfomance than SAGAN.", "votes": 1 }, { "id": 599683, "postDate": "2019-08-15T08:47:41.887Z", "content": "

Congrats ans thanks for your sharing! @dvorobiev \nI saw BigGAN also, but I don't have time to implement it...

", "rawMarkdown": "Congrats ans thanks for your sharing! @dvorobiev \nI saw BigGAN also, but I don't have time to implement it..." }, { "id": 599711, "postDate": "2019-08-15T09:32:33.113Z", "content": "

Thanks. I didn't implemented it. I just took the existing code and swap data processing pipeline:\nhttps://github.com/ajbrock/BigGAN-PyTorch

", "rawMarkdown": "Thanks. I didn't implemented it. I just took the existing code and swap data processing pipeline:\nhttps://github.com/ajbrock/BigGAN-PyTorch" }, { "id": 599729, "postDate": "2019-08-15T09:50:32.530Z", "content": "

thank you so much for this information 😄

", "rawMarkdown": "thank you so much for this information 😄 " } ] }, { "id": 599393, "postDate": "2019-08-14T22:57:11.057Z", "content": "

Nice post

", "rawMarkdown": "Nice post", "votes": 1 }, { "id": 599599, "postDate": "2019-08-15T07:00:58.743Z", "content": "

Also I tought about SuperResolution GANs, are really impressive :) But but vacations come first ;) And I teamed up but my mates were also too busy and me alone couldn't work on all of this .... \nAnyway, BigGan is the winning horse

", "rawMarkdown": "Also I tought about SuperResolution GANs, are really impressive :) But but vacations come first ;) And I teamed up but my mates were also too busy and me alone couldn't work on all of this .... \nAnyway, BigGan is the winning horse", "replies": [ { "id": 599630, "postDate": "2019-08-15T07:49:13.537Z", "content": "

Great job nanashi! \nYou were working in silence. Thank you for sharing your work :)\nPs- who keeps downvoting you? It's weird.

", "rawMarkdown": "Great job nanashi! \nYou were working in silence. Thank you for sharing your work :)\nPs- who keeps downvoting you? It's weird. ", "votes": 1 }, { "id": 599666, "postDate": "2019-08-15T08:33:35.080Z", "content": "

hahaha thanks @timetraveller98 I guess my serial downvoter is back!\nI would have liked to share my SNGAN kernel, but I didn't have time... probably these days I'll publish it ;)

", "rawMarkdown": "hahaha thanks @timetraveller98 I guess my serial downvoter is back!\nI would have liked to share my SNGAN kernel, but I didn't have time... probably these days I'll publish it ;)", "votes": 1 } ] }, { "id": 600150, "postDate": "2019-08-15T17:51:23.263Z", "content": "

Really nice!\nThanks for sharing the info.

", "rawMarkdown": "Really nice!\nThanks for sharing the info." }, { "id": 599778, "postDate": "2019-08-15T10:50:52.473Z", "content": "

Good job man...Glad to see a great knowledge treasure at one place. Try to implement all...

", "rawMarkdown": "Good job man...Glad to see a great knowledge treasure at one place. Try to implement all..." }, { "id": 599542, "postDate": "2019-08-15T05:13:38.023Z", "content": "

Good job. Your LB score?

", "rawMarkdown": "Good job. Your LB score?" }, { "id": 599517, "postDate": "2019-08-15T04:18:12.020Z", "content": "

Awesome work!

", "rawMarkdown": "Awesome work!" }, { "id": 599494, "postDate": "2019-08-15T03:34:43.527Z", "content": "

Thanks for your sharing!! \nI also use the conditional_batchnorm\nMy best model is Resnet block DCGAN add conditional batchnorm + RalSGAN's loss function add label classification loss, my great score is 44 MiFiD

", "rawMarkdown": "Thanks for your sharing!! \nI also use the conditional_batchnorm\nMy best model is Resnet block DCGAN add conditional batchnorm + RalSGAN's loss function add label classification loss, my great score is 44 MiFiD", "replies": [ { "id": 599597, "postDate": "2019-08-15T06:58:19.853Z", "content": "

yeah! conditional_batchnorm helped a lot! Probbaly add projection to a DCGAN would be an amazing idea

", "rawMarkdown": "yeah! conditional_batchnorm helped a lot! Probbaly add projection to a DCGAN would be an amazing idea" } ] }, { "id": 599405, "postDate": "2019-08-14T23:16:32.640Z", "content": "

Nice post, Nanashi !!\nI also saw these great repositories, but I couldn't reimplemented in kaggle kernels, and my public score resulted in not good score...\nhow is your socore ?? Thank you !

", "rawMarkdown": "Nice post, Nanashi !!\nI also saw these great repositories, but I couldn't reimplemented in kaggle kernels, and my public score resulted in not good score...\nhow is your socore ?? Thank you !" }, { "id": 610330, "postDate": "2019-08-28T17:03:13.760Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 599402, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-08-14T23:13:50.517000", "content": "

Great work Nanashi, I also used that GAN :-) I found the GitHub the day before yesterday so I didn't have time to tune it well. I only got it to LB 47 with two overnight experiments. What settings did you use and what preprocessing did you do to the dog images? (I used the original GitHub from the authors).

\n\n

", "votes": 1, "replies": [ { "id": 599594, "author_name": "Nanashi", "author_url": "", "post_date": "2019-08-15T06:56:53.280000", "content": "

thank you so much @cdeotte :) I think I'll score around 30-40. As I said I couldn't work so much in this approach... but I think it's really interesting, both papers are great.

\n\n

btw, I never thought of using biggan, really cool 👍

\n\n

PS: I can't see the picture...

", "votes": 0, "replies": [] }, { "id": 599602, "author_name": "Nanashi", "author_url": "", "post_date": "2019-08-15T07:05:03.387000", "content": "

also, I'm a noob so I have to ask...

\n\n
\n

how did you face this comp?

\n
\n\n

I mean, did you try several architectures until BigGan? did you find BigGan last weeks? or you knew about it before?

\n\n

Thanks!

", "votes": 0, "replies": [] }, { "id": 599629, "author_name": "Dmitry Vorobiev", "author_url": "", "post_date": "2019-08-15T07:45:20.740000", "content": "

I started with WGAN / DCGAN. Then I watched a few talks from CVPR 2018, namely Ian Goodfellow speech on general state of GAN models. So I found Self-Attention GAN first and was able to get ~40 MiFID out of it. I don't remember how I found BigGAN, but it took me a while to set it up in a way, which gave me the better perfomance than SAGAN.

", "votes": 1, "replies": [] }, { "id": 599683, "author_name": "0xFunky", "author_url": "", "post_date": "2019-08-15T08:47:41.887000", "content": "

Congrats ans thanks for your sharing! @dvorobiev \nI saw BigGAN also, but I don't have time to implement it...

", "votes": 0, "replies": [] }, { "id": 599711, "author_name": "Dmitry Vorobiev", "author_url": "", "post_date": "2019-08-15T09:32:33.113000", "content": "

Thanks. I didn't implemented it. I just took the existing code and swap data processing pipeline:\nhttps://github.com/ajbrock/BigGAN-PyTorch

", "votes": 0, "replies": [] }, { "id": 599729, "author_name": "Nanashi", "author_url": "", "post_date": "2019-08-15T09:50:32.530000", "content": "

thank you so much for this information 😄

", "votes": 0, "replies": [] } ] }, { "id": 599393, "author_name": "Yirun Zhang", "author_url": "", "post_date": "2019-08-14T22:57:11.057000", "content": "

Nice post

", "votes": 1, "replies": [] }, { "id": 599599, "author_name": "Nanashi", "author_url": "", "post_date": "2019-08-15T07:00:58.743000", "content": "

Also I tought about SuperResolution GANs, are really impressive :) But but vacations come first ;) And I teamed up but my mates were also too busy and me alone couldn't work on all of this .... \nAnyway, BigGan is the winning horse

", "votes": 0, "replies": [ { "id": 599630, "author_name": "timetraveller", "author_url": "", "post_date": "2019-08-15T07:49:13.537000", "content": "

Great job nanashi! \nYou were working in silence. Thank you for sharing your work :)\nPs- who keeps downvoting you? It's weird.

", "votes": 1, "replies": [] }, { "id": 599666, "author_name": "Nanashi", "author_url": "", "post_date": "2019-08-15T08:33:35.080000", "content": "

hahaha thanks @timetraveller98 I guess my serial downvoter is back!\nI would have liked to share my SNGAN kernel, but I didn't have time... probably these days I'll publish it ;)

", "votes": 1, "replies": [] } ] }, { "id": 600150, "author_name": "Ziad Magdy", "author_url": "", "post_date": "2019-08-15T17:51:23.263000", "content": "

Really nice!\nThanks for sharing the info.

", "votes": 0, "replies": [] }, { "id": 599778, "author_name": "VinayVikram", "author_url": "", "post_date": "2019-08-15T10:50:52.473000", "content": "

Good job man...Glad to see a great knowledge treasure at one place. Try to implement all...

", "votes": 0, "replies": [] }, { "id": 599542, "author_name": "baigou", "author_url": "", "post_date": "2019-08-15T05:13:38.023000", "content": "

Good job. Your LB score?

", "votes": 0, "replies": [] }, { "id": 599517, "author_name": "Mighty Rains", "author_url": "", "post_date": "2019-08-15T04:18:12.020000", "content": "

Awesome work!

", "votes": 0, "replies": [] }, { "id": 599494, "author_name": "0xFunky", "author_url": "", "post_date": "2019-08-15T03:34:43.527000", "content": "

Thanks for your sharing!! \nI also use the conditional_batchnorm\nMy best model is Resnet block DCGAN add conditional batchnorm + RalSGAN's loss function add label classification loss, my great score is 44 MiFiD

", "votes": 0, "replies": [ { "id": 599597, "author_name": "Nanashi", "author_url": "", "post_date": "2019-08-15T06:58:19.853000", "content": "

yeah! conditional_batchnorm helped a lot! Probbaly add projection to a DCGAN would be an amazing idea

", "votes": 0, "replies": [] } ] }, { "id": 599405, "author_name": "kumalab", "author_url": "", "post_date": "2019-08-14T23:16:32.640000", "content": "

Nice post, Nanashi !!\nI also saw these great repositories, but I couldn't reimplemented in kaggle kernels, and my public score resulted in not good score...\nhow is your socore ?? Thank you !

", "votes": 0, "replies": [] }, { "id": 610330, "author_name": "", "author_url": "", "post_date": "2019-08-28T17:03:13.760000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "599387": "> I want to share this and go to sleep... Good luck!\n\nAs you can read at [The best GAN performance and goog luck!](https://www.kaggle.com/c/generative-dog-images/discussion/104167#latest-599374) SNGAN really outperforms, so I decided to focus on that.\n\n1. I read the original paper: [Spectral Normalization for Generative Adversarial Networks](https://arxiv.org/pdf/1802.05957.pdf)\n2. Following the original authors, I found out this amazing paper: [cGANs with Projection Discriminator](https://arxiv.org/pdf/1802.05637.pdf)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2779868%2Fc503bb6e1dc565237b8f08dab530d18d%2FScreenshot%20from%202019-08-15%2000-12-39.png?generation=1565820871058717&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2779868%2F88fcdaf6f6a3f2fbaa3eaafd6a4d57ab%2FScreenshot%20from%202019-08-15%2000-13-12.png?generation=1565820872307693&alt=media)\n\nThe performance using **projection** was amazing!\n\nThis is the original repo where you can find the code: [pfnet-research](https://github.com/pfnet-research/sngan_projection)\n\n![](https://raw.githubusercontent.com/pfnet-research/sngan_projection/master/demo/dog_and_cat_1x1_long.gif)\n![](https://raw.githubusercontent.com/pfnet-research/sngan_projection/master/demo/images_dog_and_cat_64.jpg)\n\nIn this other repo, you can find another implementation: [sngan_projection_TensorFlow](https://github.com/MingtaoGuo/sngan_projection_TensorFlow)\n\n- conditional_batchnorm\n- projection\n\nAnd finally I found this one: [Spectral Normalization and Projection Discriminator(Pytorch)](https://github.com/balansky/pytorch_gan)\n\n![](https://raw.githubusercontent.com/balansky/pytorch_gan/master/samples/250000_img.png)\n\nI tried a lot of architectures: GANs, DCGANs, WGAN, RaLSGAN, CycleGAN... in this case we have labels so CGANs are a great option + projection and other tricks...\nMy 2nd option are VAE-GANs ;)\n\nI took holidays and I couldn't work too much in this approach ... hope this helps and please let me know if you realized about this :) the papers are from 2018 so this technique is like a \"new\" approach I guess, probably that's why is not so famous. I teamed up because I didn't have time... but my teammates couldn't work at all on these approaches so their solution is compete different from this.\n\n", "599402": "Great work Nanashi, I also used that GAN :-) I found the GitHub the day before yesterday so I didn't have time to tune it well. I only got it to LB 47 with two overnight experiments. What settings did you use and what preprocessing did you do to the dog images? (I used the original GitHub from the authors).\n \n \n", "599393": "Nice post", "599599": "Also I tought about SuperResolution GANs, are really impressive :) But but vacations come first ;) And I teamed up but my mates were also too busy and me alone couldn't work on all of this .... \nAnyway, BigGan is the winning horse", "600150": "Really nice!\nThanks for sharing the info.", "599778": "Good job man...Glad to see a great knowledge treasure at one place. Try to implement all...", "599542": "Good job. Your LB score?", "599517": "Awesome work!", "599494": "Thanks for your sharing!! \nI also use the conditional_batchnorm\nMy best model is Resnet block DCGAN add conditional batchnorm + RalSGAN's loss function add label classification loss, my great score is 44 MiFiD", "599405": "Nice post, Nanashi !!\nI also saw these great repositories, but I couldn't reimplemented in kaggle kernels, and my public score resulted in not good score...\nhow is your socore ?? Thank you !", "610330": "" } }