![\"image\"](\"http://playagricola.com/Kaggle/dogs381419.png\")
Hi everybody ! \nThe code for my solution is available here : https://www.kaggle.com/theoviel/conditional-progan-30-public\nI spent some time yersteday cleaning it and commenting it a bit.
\n\nOverall, I took a PyTorch ProGAN implementation from GitHub about 20 days after the beginning and sticked with it for the rest of the comp. I quickly reached 32 but couldn't get much further. Guess I should've moved to BigGan which seemed to perform better :)
\n\nI'm definitely going to enjoy reading other's solutions though.\nSpecial thanks to the Kaggle team for the extra effort put into the competition, hopefully reviewing kernels won't be too painful.
", "messages": [ { "id": "599699", "postDate": "08/15/2019 09:13:21", "content": "Hi everybody ! \nThe code for my solution is available here : https://www.kaggle.com/theoviel/conditional-progan-30-public\nI spent some time yersteday cleaning it and commenting it a bit.
\n\nOverall, I took a PyTorch ProGAN implementation from GitHub about 20 days after the beginning and sticked with it for the rest of the comp. I quickly reached 32 but couldn't get much further. Guess I should've moved to BigGan which seemed to perform better :)
\n\nI'm definitely going to enjoy reading other's solutions though.\nSpecial thanks to the Kaggle team for the extra effort put into the competition, hopefully reviewing kernels won't be too painful.
", "rawMarkdown": "Hi everybody ! \nThe code for my solution is available here : https://www.kaggle.com/theoviel/conditional-progan-30-public\nI spent some time yersteday cleaning it and commenting it a bit.\n\nOverall, I took a PyTorch ProGAN implementation from GitHub about 20 days after the beginning and sticked with it for the rest of the comp. I quickly reached 32 but couldn't get much further. Guess I should've moved to BigGan which seemed to perform better :)\n\nI'm definitely going to enjoy reading other's solutions though.\nSpecial thanks to the Kaggle team for the extra effort put into the competition, hopefully reviewing kernels won't be too painful.", "votes": null }, { "id": "599732", "postDate": "08/15/2019 09:52:17", "content": "ProGan? Really interesting, it's the first time I read about it :) Thanks! @theoviel
", "rawMarkdown": "ProGan? Really interesting, it's the first time I read about it :) Thanks! @theoviel", "votes": null }, { "id": "599781", "postDate": "08/15/2019 10:55:09", "content": "Thank you for sharing!
", "rawMarkdown": "Thank you for sharing!", "votes": null }, { "id": "599842", "postDate": "08/15/2019 11:47:39", "content": "I use VGAN and I quickly reached 32 too. Unfortunately, I can't improve it for many days. But I think VGAN can get a better score than 32.
", "rawMarkdown": "I use VGAN and I quickly reached 32 too. Unfortunately, I can't improve it for many days. But I think VGAN can get a better score than 32.", "votes": null }, { "id": "599859", "postDate": "08/15/2019 11:57:22", "content": "Congrats Theo. LB 30 is amazing. I read that paper on ProGAN. I like the idea. In mathematics, we often solve PDEs by first solving a coarse solution and progressively solving with more resolution. It works well. I wanted to play with those ideas but it seemed too much work to implement. Thanks for coding it up and sharing a Kaggle kernel.
", "rawMarkdown": "Congrats Theo. LB 30 is amazing. I read that [paper][1] on ProGAN. I like the idea. In mathematics, we often solve PDEs by first solving a coarse solution and progressively solving with more resolution. It works well. I wanted to play with those ideas but it seemed too much work to implement. Thanks for coding it up and sharing a Kaggle kernel.\n\n[1]: https://arxiv.org/abs/1710.10196", "votes": null }, { "id": "599867", "postDate": "08/15/2019 12:07:32", "content": "I've tried a custom implementation of ProGAN (basically, the same tricks and architecture as my DCGAN but trained layer after layers). Unfortunately, wasn't able to get anything beyond 75, and there were issues with convergence in some setups. But probably I did something wrong there.
\n\nThe implementation is very rude with explicitly repeated code patterns instead of looping :) Though it was simple to do it in this way when running out of time.
", "rawMarkdown": "I've tried a custom implementation of [ProGAN](https://www.kaggle.com/purplejester/gan-xx-pgan-attempt) (basically, the same tricks and architecture as my DCGAN but trained layer after layers). Unfortunately, wasn't able to get anything beyond 75, and there were issues with convergence in some setups. But probably I did something wrong there. \n\nThe implementation is very rude with explicitly repeated code patterns instead of looping :) Though it was simple to do it in this way when running out of time.", "votes": null }, { "id": "599898", "postDate": "08/15/2019 12:38:14", "content": "Thanks Chris! The paper is indeed great, lots of tricks and everything.\nI wish I did the implementation, but I pretty much took the architecture from https://github.com/akanimax/pro_gan_pytorch\nMy first tries at manually implementing the arch were not that successful.
", "rawMarkdown": "Thanks Chris! The paper is indeed great, lots of tricks and everything.\nI wish I did the implementation, but I pretty much took the architecture from https://github.com/akanimax/pro_gan_pytorch\nMy first tries at manually implementing the arch were not that successful.", "votes": null }, { "id": "599899", "postDate": "08/15/2019 12:38:25", "content": "ProGAN has more than the progressive growing trick, it has pixelnorm, minibatch std, ema weight decay and other interesting stuff, that definitely help the nn converging.
", "rawMarkdown": "ProGAN has more than the progressive growing trick, it has pixelnorm, minibatch std, ema weight decay and other interesting stuff, that definitely help the nn converging.", "votes": null }, { "id": "599900", "postDate": "08/15/2019 12:38:58", "content": "Did not read about VGan, i'll take a look at it though!
", "rawMarkdown": "Did not read about VGan, i'll take a look at it though!", "votes": null }, { "id": "599957", "postDate": "08/15/2019 13:38:10", "content": "Here is the paper:https://xbpeng.github.io/projects/VDB/2019_VDB.pdf
", "rawMarkdown": "Here is the paper:https://xbpeng.github.io/projects/VDB/2019_VDB.pdf", "votes": null }, { "id": "600127", "postDate": "08/15/2019 17:22:29", "content": "Great Work!
", "rawMarkdown": "Great Work!", "votes": null }, { "id": "600166", "postDate": "08/15/2019 18:33:24", "content": "Thanks for Sharing. Congrants for this Amazing score and take your time for commenting what you do.
", "rawMarkdown": "Thanks for Sharing. Congrants for this Amazing score and take your time for commenting what you do.", "votes": null }, { "id": "600181", "postDate": "08/15/2019 19:00:26", "content": "Yeah, I did some of these as well, at least PixelNorm, Minibatch std, and SpectralNorm. However, I didn't do any EMA (though it really helped with DCGAN), or weight decay. In my case, cosine annealing with restarts for learning rate undermined the performance instead. But sure enough, you're right and I should be more careful with it.
", "rawMarkdown": "Yeah, I did some of these as well, at least PixelNorm, Minibatch std, and SpectralNorm. However, I didn't do any EMA (though it really helped with DCGAN), or weight decay. In my case, cosine annealing with restarts for learning rate undermined the performance instead. But sure enough, you're right and I should be more careful with it.", "votes": null }, { "id": "600225", "postDate": "08/15/2019 20:10:41", "content": "Below are samples of Theo's dogs. They look great
\n\n
wooow. ProGAN, sounds very cool.
", "rawMarkdown": "wooow. ProGAN, sounds very cool.", "votes": null }, { "id": "601889", "postDate": "08/18/2019 09:50:40", "content": "I didn’t know VGAN, I’m reading now. In my setting, adding uniform noise to discriminator inputs really worked well. But I think that incorporating VGAN architectures in other GAN settings can improve score more than my choice. Thanks for sharing, @adiachen !
", "rawMarkdown": "I didn’t know VGAN, I’m reading now. In my setting, adding uniform noise to discriminator inputs really worked well. But I think that incorporating VGAN architectures in other GAN settings can improve score more than my choice. Thanks for sharing, @adiachen !", "votes": null } ], "comments": [ { "id": 599732, "author_name": "jesucristo", "author_url": "", "post_date": "08/15/2019 09:52:17", "content": "ProGan? Really interesting, it's the first time I read about it :) Thanks! @theoviel
", "votes": null, "replies": [] }, { "id": 599781, "author_name": "matthiaspilz", "author_url": "", "post_date": "08/15/2019 10:55:09", "content": "Thank you for sharing!
", "votes": null, "replies": [] }, { "id": 599842, "author_name": "adiachen", "author_url": "", "post_date": "08/15/2019 11:47:39", "content": "I use VGAN and I quickly reached 32 too. Unfortunately, I can't improve it for many days. But I think VGAN can get a better score than 32.
", "votes": null, "replies": [ { "id": 599900, "author_name": "theoviel", "author_url": "", "post_date": "08/15/2019 12:38:58", "content": "Did not read about VGan, i'll take a look at it though!
", "votes": null, "replies": [] }, { "id": 599957, "author_name": "adiachen", "author_url": "", "post_date": "08/15/2019 13:38:10", "content": "Here is the paper:https://xbpeng.github.io/projects/VDB/2019_VDB.pdf
", "votes": null, "replies": [] }, { "id": 601889, "author_name": "yukia18", "author_url": "", "post_date": "08/18/2019 09:50:40", "content": "I didn’t know VGAN, I’m reading now. In my setting, adding uniform noise to discriminator inputs really worked well. But I think that incorporating VGAN architectures in other GAN settings can improve score more than my choice. Thanks for sharing, @adiachen !
", "votes": null, "replies": [] } ] }, { "id": 599859, "author_name": "cdeotte", "author_url": "", "post_date": "08/15/2019 11:57:22", "content": "Congrats Theo. LB 30 is amazing. I read that paper on ProGAN. I like the idea. In mathematics, we often solve PDEs by first solving a coarse solution and progressively solving with more resolution. It works well. I wanted to play with those ideas but it seemed too much work to implement. Thanks for coding it up and sharing a Kaggle kernel.
", "votes": null, "replies": [ { "id": 599898, "author_name": "theoviel", "author_url": "", "post_date": "08/15/2019 12:38:14", "content": "Thanks Chris! The paper is indeed great, lots of tricks and everything.\nI wish I did the implementation, but I pretty much took the architecture from https://github.com/akanimax/pro_gan_pytorch\nMy first tries at manually implementing the arch were not that successful.
", "votes": null, "replies": [] } ] }, { "id": 599867, "author_name": "purplejester", "author_url": "", "post_date": "08/15/2019 12:07:32", "content": "I've tried a custom implementation of ProGAN (basically, the same tricks and architecture as my DCGAN but trained layer after layers). Unfortunately, wasn't able to get anything beyond 75, and there were issues with convergence in some setups. But probably I did something wrong there.
\n\nThe implementation is very rude with explicitly repeated code patterns instead of looping :) Though it was simple to do it in this way when running out of time.
", "votes": null, "replies": [ { "id": 599899, "author_name": "theoviel", "author_url": "", "post_date": "08/15/2019 12:38:25", "content": "ProGAN has more than the progressive growing trick, it has pixelnorm, minibatch std, ema weight decay and other interesting stuff, that definitely help the nn converging.
", "votes": null, "replies": [] }, { "id": 600181, "author_name": "purplejester", "author_url": "", "post_date": "08/15/2019 19:00:26", "content": "Yeah, I did some of these as well, at least PixelNorm, Minibatch std, and SpectralNorm. However, I didn't do any EMA (though it really helped with DCGAN), or weight decay. In my case, cosine annealing with restarts for learning rate undermined the performance instead. But sure enough, you're right and I should be more careful with it.
", "votes": null, "replies": [] } ] }, { "id": 600127, "author_name": "akshayakula", "author_url": "", "post_date": "08/15/2019 17:22:29", "content": "Great Work!
", "votes": null, "replies": [] }, { "id": 600166, "author_name": "marcovasquez", "author_url": "", "post_date": "08/15/2019 18:33:24", "content": "Thanks for Sharing. Congrants for this Amazing score and take your time for commenting what you do.
", "votes": null, "replies": [] }, { "id": 600225, "author_name": "cdeotte", "author_url": "", "post_date": "08/15/2019 20:10:41", "content": "Below are samples of Theo's dogs. They look great
\n\n
wooow. ProGAN, sounds very cool.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "599699": "Hi everybody ! \nThe code for my solution is available here : https://www.kaggle.com/theoviel/conditional-progan-30-public\nI spent some time yersteday cleaning it and commenting it a bit.\n\nOverall, I took a PyTorch ProGAN implementation from GitHub about 20 days after the beginning and sticked with it for the rest of the comp. I quickly reached 32 but couldn't get much further. Guess I should've moved to BigGan which seemed to perform better :)\n\nI'm definitely going to enjoy reading other's solutions though.\nSpecial thanks to the Kaggle team for the extra effort put into the competition, hopefully reviewing kernels won't be too painful.", "599732": "ProGan? Really interesting, it's the first time I read about it :) Thanks! @theoviel", "599781": "Thank you for sharing!", "599842": "I use VGAN and I quickly reached 32 too. Unfortunately, I can't improve it for many days. But I think VGAN can get a better score than 32.", "599859": "Congrats Theo. LB 30 is amazing. I read that [paper][1] on ProGAN. I like the idea. In mathematics, we often solve PDEs by first solving a coarse solution and progressively solving with more resolution. It works well. I wanted to play with those ideas but it seemed too much work to implement. Thanks for coding it up and sharing a Kaggle kernel.\n\n[1]: https://arxiv.org/abs/1710.10196", "599867": "I've tried a custom implementation of [ProGAN](https://www.kaggle.com/purplejester/gan-xx-pgan-attempt) (basically, the same tricks and architecture as my DCGAN but trained layer after layers). Unfortunately, wasn't able to get anything beyond 75, and there were issues with convergence in some setups. But probably I did something wrong there. \n\nThe implementation is very rude with explicitly repeated code patterns instead of looping :) Though it was simple to do it in this way when running out of time.", "599898": "Thanks Chris! The paper is indeed great, lots of tricks and everything.\nI wish I did the implementation, but I pretty much took the architecture from https://github.com/akanimax/pro_gan_pytorch\nMy first tries at manually implementing the arch were not that successful.", "599899": "ProGAN has more than the progressive growing trick, it has pixelnorm, minibatch std, ema weight decay and other interesting stuff, that definitely help the nn converging.", "599900": "Did not read about VGan, i'll take a look at it though!", "599957": "Here is the paper:https://xbpeng.github.io/projects/VDB/2019_VDB.pdf", "600127": "Great Work!", "600166": "Thanks for Sharing. Congrants for this Amazing score and take your time for commenting what you do.", "600181": "Yeah, I did some of these as well, at least PixelNorm, Minibatch std, and SpectralNorm. However, I didn't do any EMA (though it really helped with DCGAN), or weight decay. In my case, cosine annealing with restarts for learning rate undermined the performance instead. But sure enough, you're right and I should be more careful with it.", "600225": "Below are samples of Theo's dogs. They look great\n\n", "600657": "wooow. ProGAN, sounds very cool.", "601889": "I didn’t know VGAN, I’m reading now. In my setting, adding uniform noise to discriminator inputs really worked well. But I think that incorporating VGAN architectures in other GAN settings can improve score more than my choice. Thanks for sharing, @adiachen !" }, "source": "meta" }