![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F316036%2Fc971fbc7413bc7161bca99fd04a0c3bf%2Fexperience.JPG?generation=1566137754427868&alt=media\")
Hi All\nFirst, I want to thank Kaggle for holding this great competition, this is first GAN competition, I really enjoy and learned a lot from this competition, thanks everyone for great discussion and great kernel, thanks @cdeotte for detail explanation of GAN and good kernel share.
\n\nSecond, Very thanks a great RaLSGAN kernel from @sakami, I fork it and do some experience to get my final solution, LB44.69, Below is my final submission kernel : https://www.kaggle.com/super13579/ralsgan-dogs-resnet-cbn?scriptVersionId=18722384
\n\nTriple the convolution output channel count on Discirminator, LB 69 (original kernel) to LB 54\n
Add Pixel normalize and spectral normalize on Generator, LB 54 to LB 52
\n\nI add the Aux layer on the last of discriminator, and choose a good weight to merge to loss function (fine-tune manually ), LB 52 to LB 49\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F316036%2F99b6b3274307a43255f302cf23b53c45%2Fexperience1.JPG?generation=1566138320702129&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F316036%2F8280dbba30b3340ab7d0259960c1d2da%2Fexperience2.JPG?generation=1566138413927201&alt=media\")
I saw recent GAN paper all use ResNet block and conditional batch normalize, so I try to implement them, ResNet Block for Generator and Discriminator, CBN for Generatro, LB 49 to LB 44, you can refer my final kernel to see how to implement them.
\n\nThis is dog image from my best score kenrl, some head are missing XD\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F316036%2F4cc0a8e2c2d0cb91924ce42ecc25dc00%2FDog%20image.JPG?generation=1566139332159491&alt=media\")
I think BigGAN is the biggest winner in this competition, I've saw some paper mention it, but I thought it will exceed the limit time (9hours), cause it looks like a \"Big\" model, so I didn't implement it, only implement Resnet Block and CBN, I learned a lot of GAN from this competition, thanks everyone for great discussion and great kernel! GAN is very interesting!
", "messages": [ { "id": "602075", "postDate": "08/18/2019 14:38:52", "content": "Hi All\nFirst, I want to thank Kaggle for holding this great competition, this is first GAN competition, I really enjoy and learned a lot from this competition, thanks everyone for great discussion and great kernel, thanks @cdeotte for detail explanation of GAN and good kernel share.
\n\nSecond, Very thanks a great RaLSGAN kernel from @sakami, I fork it and do some experience to get my final solution, LB44.69, Below is my final submission kernel : https://www.kaggle.com/super13579/ralsgan-dogs-resnet-cbn?scriptVersionId=18722384
\n\nTriple the convolution output channel count on Discirminator, LB 69 (original kernel) to LB 54\n
Add Pixel normalize and spectral normalize on Generator, LB 54 to LB 52
\n\nI add the Aux layer on the last of discriminator, and choose a good weight to merge to loss function (fine-tune manually ), LB 52 to LB 49\n
I saw recent GAN paper all use ResNet block and conditional batch normalize, so I try to implement them, ResNet Block for Generator and Discriminator, CBN for Generatro, LB 49 to LB 44, you can refer my final kernel to see how to implement them.
\n\nThis is dog image from my best score kenrl, some head are missing XD\n
I think BigGAN is the biggest winner in this competition, I've saw some paper mention it, but I thought it will exceed the limit time (9hours), cause it looks like a \"Big\" model, so I didn't implement it, only implement Resnet Block and CBN, I learned a lot of GAN from this competition, thanks everyone for great discussion and great kernel! GAN is very interesting!
", "rawMarkdown": "Hi All\nFirst, I want to thank Kaggle for holding this great competition, this is first GAN competition, I really enjoy and learned a lot from this competition, thanks everyone for great discussion and great kernel, thanks @cdeotte for detail explanation of GAN and good kernel share.\n\nSecond, Very thanks a great RaLSGAN kernel from @sakami, I fork it and do some experience to get my final solution, LB44.69, Below is my final submission kernel : https://www.kaggle.com/super13579/ralsgan-dogs-resnet-cbn?scriptVersionId=18722384\n \n## Journal to LB 44.69\n### 1. Add convolution output channel count on Discirminator, LB 69 to 54\nTriple the convolution output channel count on Discirminator, LB 69 (original kernel) to LB 54\n\n\n### 2. Add Pixel Normalize and spectral normalize on Generator, LB 54 to 52\nAdd Pixel normalize and spectral normalize on Generator, LB 54 to LB 52\n\n### 3. Using Breed Label, LB 52 to 49\nI add the Aux layer on the last of discriminator, and choose a good weight to merge to loss function (fine-tune manually ), LB 52 to LB 49\n\n\n\n\n### 4. Add ResNet Block and Conditional Batch normalize, LB 49 to 44\nI saw recent GAN paper all use ResNet block and conditional batch normalize, so I try to implement them, ResNet Block for Generator and Discriminator, CBN for Generatro, LB 49 to LB 44, you can refer my final kernel to see how to implement them.\n\nThis is dog image from my best score kenrl, some head are missing XD\n\n\n\n## Summary\nI think BigGAN is the biggest winner in this competition, I've saw some paper mention it, but I thought it will exceed the limit time (9hours), cause it looks like a \"Big\" model, so I didn't implement it, only implement Resnet Block and CBN, I learned a lot of GAN from this competition, thanks everyone for great discussion and great kernel! GAN is very interesting!", "votes": null }, { "id": "602176", "postDate": "08/18/2019 17:51:17", "content": "Congrats... Thank you for Sharing your Amazing Work...!! @super13579
", "rawMarkdown": "Congrats... Thank you for Sharing your Amazing Work...!! @super13579", "votes": null }, { "id": "602295", "postDate": "08/18/2019 21:35:34", "content": "\n\n\nI've saw some paper mention it, but I thought it will exceed the limit time (9hours), cause it looks like a \"Big\" model, so I didn't implement it.
\n
Same. I read countless times on the forums how bigGAN, proGAN and styleGAN are too big to train within 9 hours so I never bothered training them. I am not repeating such a mistake ever again.
", "rawMarkdown": ">I've saw some paper mention it, but I thought it will exceed the limit time (9hours), cause it looks like a \"Big\" model, so I didn't implement it.\n\nSame. I read countless times on the forums how bigGAN, proGAN and styleGAN are too big to train within 9 hours so I never bothered training them. I am not repeating such a mistake ever again.", "votes": null }, { "id": "602412", "postDate": "08/19/2019 03:55:43", "content": "Congrats and great explanation by the way. Thanks for sharing @super13579 .
", "rawMarkdown": "Congrats and great explanation by the way. Thanks for sharing @super13579 .", "votes": null }, { "id": "603146", "postDate": "08/19/2019 23:44:45", "content": "Great solution, funkyboy. Thank you for sharing!
", "rawMarkdown": "Great solution, funkyboy. Thank you for sharing!", "votes": null }, { "id": "612457", "postDate": "08/29/2019 19:55:31", "content": "Congrats Funkyboy on achieving 13th place! You built a great GAN and this is a great write up. It shows how each technique improves GANs. Well done!
", "rawMarkdown": "Congrats Funkyboy on achieving 13th place! You built a great GAN and this is a great write up. It shows how each technique improves GANs. Well done!", "votes": null }, { "id": "613100", "postDate": "08/30/2019 08:07:59", "content": "Funkyboy - Thanks so much for a Non-Funky great solution and even greater writeup . Thanks much and congratulations :D
", "rawMarkdown": "Funkyboy - Thanks so much for a Non-Funky great solution and even greater writeup . Thanks much and congratulations :D", "votes": null }, { "id": "613622", "postDate": "08/30/2019 17:07:28", "content": "Great work @super13579 .. Good write up..
", "rawMarkdown": "Great work @super13579 .. Good write up..", "votes": null } ], "comments": [ { "id": 602176, "author_name": "veeralakrishna", "author_url": "", "post_date": "08/18/2019 17:51:17", "content": "Congrats... Thank you for Sharing your Amazing Work...!! @super13579
", "votes": null, "replies": [] }, { "id": 602295, "author_name": "timetraveller98", "author_url": "", "post_date": "08/18/2019 21:35:34", "content": "\n\n\nI've saw some paper mention it, but I thought it will exceed the limit time (9hours), cause it looks like a \"Big\" model, so I didn't implement it.
\n
Same. I read countless times on the forums how bigGAN, proGAN and styleGAN are too big to train within 9 hours so I never bothered training them. I am not repeating such a mistake ever again.
", "votes": null, "replies": [] }, { "id": 602412, "author_name": "joelhanson", "author_url": "", "post_date": "08/19/2019 03:55:43", "content": "Congrats and great explanation by the way. Thanks for sharing @super13579 .
", "votes": null, "replies": [] }, { "id": 603146, "author_name": "sakami", "author_url": "", "post_date": "08/19/2019 23:44:45", "content": "Great solution, funkyboy. Thank you for sharing!
", "votes": null, "replies": [] }, { "id": 612457, "author_name": "cdeotte", "author_url": "", "post_date": "08/29/2019 19:55:31", "content": "Congrats Funkyboy on achieving 13th place! You built a great GAN and this is a great write up. It shows how each technique improves GANs. Well done!
", "votes": null, "replies": [] }, { "id": 613100, "author_name": "phoenix9032", "author_url": "", "post_date": "08/30/2019 08:07:59", "content": "Funkyboy - Thanks so much for a Non-Funky great solution and even greater writeup . Thanks much and congratulations :D
", "votes": null, "replies": [] }, { "id": 613622, "author_name": "manojprabhaakr", "author_url": "", "post_date": "08/30/2019 17:07:28", "content": "Great work @super13579 .. Good write up..
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "602075": "Hi All\nFirst, I want to thank Kaggle for holding this great competition, this is first GAN competition, I really enjoy and learned a lot from this competition, thanks everyone for great discussion and great kernel, thanks @cdeotte for detail explanation of GAN and good kernel share.\n\nSecond, Very thanks a great RaLSGAN kernel from @sakami, I fork it and do some experience to get my final solution, LB44.69, Below is my final submission kernel : https://www.kaggle.com/super13579/ralsgan-dogs-resnet-cbn?scriptVersionId=18722384\n \n## Journal to LB 44.69\n### 1. Add convolution output channel count on Discirminator, LB 69 to 54\nTriple the convolution output channel count on Discirminator, LB 69 (original kernel) to LB 54\n\n\n### 2. Add Pixel Normalize and spectral normalize on Generator, LB 54 to 52\nAdd Pixel normalize and spectral normalize on Generator, LB 54 to LB 52\n\n### 3. Using Breed Label, LB 52 to 49\nI add the Aux layer on the last of discriminator, and choose a good weight to merge to loss function (fine-tune manually ), LB 52 to LB 49\n\n\n\n\n### 4. Add ResNet Block and Conditional Batch normalize, LB 49 to 44\nI saw recent GAN paper all use ResNet block and conditional batch normalize, so I try to implement them, ResNet Block for Generator and Discriminator, CBN for Generatro, LB 49 to LB 44, you can refer my final kernel to see how to implement them.\n\nThis is dog image from my best score kenrl, some head are missing XD\n\n\n\n## Summary\nI think BigGAN is the biggest winner in this competition, I've saw some paper mention it, but I thought it will exceed the limit time (9hours), cause it looks like a \"Big\" model, so I didn't implement it, only implement Resnet Block and CBN, I learned a lot of GAN from this competition, thanks everyone for great discussion and great kernel! GAN is very interesting!", "602176": "Congrats... Thank you for Sharing your Amazing Work...!! @super13579", "602295": ">I've saw some paper mention it, but I thought it will exceed the limit time (9hours), cause it looks like a \"Big\" model, so I didn't implement it.\n\nSame. I read countless times on the forums how bigGAN, proGAN and styleGAN are too big to train within 9 hours so I never bothered training them. I am not repeating such a mistake ever again.", "602412": "Congrats and great explanation by the way. Thanks for sharing @super13579 .", "603146": "Great solution, funkyboy. Thank you for sharing!", "612457": "Congrats Funkyboy on achieving 13th place! You built a great GAN and this is a great write up. It shows how each technique improves GANs. Well done!", "613100": "Funkyboy - Thanks so much for a Non-Funky great solution and even greater writeup . Thanks much and congratulations :D", "613622": "Great work @super13579 .. Good write up.." }, "source": "meta" }