Hi
\n\nHere is our kernel: https://www.kaggle.com/shaygu/t3h-d0g-mak3rz-public-43-fid we specifically attached the version that calculates FID every few epochs, we think it brings more insight into the network and also that's the method we used for optimization .
\n\nOur knowledge was very limited walking into this competition, and we joined little more then a month ago, so we decided to start with the best public model and see where it gets us. Our steps along the way are further discussed in the kernel, however, I will briefly touch on the points also here.
\n\nUsing a lot more parameters for the network, improved our score from 60 to 55. Calculating the FID every few epochs allowed us to observe which network performs better (more consistently), which lead to ability to introduce cyclic learning rate (efficiently now) and small hyper parameter optimizations got us to ~50. The last big step was changing the weight initialization to a broader distribution (0,0.15-1 instead of the 0,0.02 commonly used) got us to ~46 and from there to 43 its minor optimizations.
\n\nIt was very interesting competition and we've learned a lot. Hopefully, there will be another GAN competition in the near future.
\n\n", "messages": [ { "id": "599717", "postDate": "08/15/2019 09:44:30", "content": "Hi
\n\nHere is our kernel: https://www.kaggle.com/shaygu/t3h-d0g-mak3rz-public-43-fid we specifically attached the version that calculates FID every few epochs, we think it brings more insight into the network and also that's the method we used for optimization .
\n\nOur knowledge was very limited walking into this competition, and we joined little more then a month ago, so we decided to start with the best public model and see where it gets us. Our steps along the way are further discussed in the kernel, however, I will briefly touch on the points also here.
\n\nUsing a lot more parameters for the network, improved our score from 60 to 55. Calculating the FID every few epochs allowed us to observe which network performs better (more consistently), which lead to ability to introduce cyclic learning rate (efficiently now) and small hyper parameter optimizations got us to ~50. The last big step was changing the weight initialization to a broader distribution (0,0.15-1 instead of the 0,0.02 commonly used) got us to ~46 and from there to 43 its minor optimizations.
\n\nIt was very interesting competition and we've learned a lot. Hopefully, there will be another GAN competition in the near future.
\n\n", "rawMarkdown": "Hi \n\nHere is our kernel: https://www.kaggle.com/shaygu/t3h-d0g-mak3rz-public-43-fid we specifically attached the version that calculates FID every few epochs, we think it brings more insight into the network and also that's the method we used for optimization .\n\nOur knowledge was very limited walking into this competition, and we joined little more then a month ago, so we decided to start with the best public model and see where it gets us. Our steps along the way are further discussed in the kernel, however, I will briefly touch on the points also here. \n\nUsing a lot more parameters for the network, improved our score from 60 to 55. Calculating the FID every few epochs allowed us to observe which network performs better (more consistently), which lead to ability to introduce cyclic learning rate (efficiently now) and small hyper parameter optimizations got us to ~50. The last big step was changing the weight initialization to a broader distribution (0,0.15-1 instead of the 0,0.02 commonly used) got us to ~46 and from there to 43 its minor optimizations. \n\nIt was very interesting competition and we've learned a lot. Hopefully, there will be another GAN competition in the near future.\n\n@eviatarai @yogevh", "votes": null }, { "id": "600400", "postDate": "08/16/2019 04:19:17", "content": "Amazing Work... Thank you for sharing...!! @shaygu
", "rawMarkdown": "Amazing Work... Thank you for sharing...!! @shaygu", "votes": null }, { "id": "600479", "postDate": "08/16/2019 06:32:54", "content": "@veeralakrishna Thanks for the reply, glad you liked it!
", "rawMarkdown": "veeralakrishna Thanks for the reply, glad you liked it!", "votes": null }, { "id": "600756", "postDate": "08/16/2019 14:19:01", "content": "nice work @shaygu . Thanks for sharing
", "rawMarkdown": "nice work @shaygu . Thanks for sharing", "votes": null }, { "id": "600770", "postDate": "08/16/2019 14:37:05", "content": "@utkukubilay Thanks, I appriciate the reply!
", "rawMarkdown": "utkukubilay Thanks, I appriciate the reply!", "votes": null } ], "comments": [ { "id": 600400, "author_name": "veeralakrishna", "author_url": "", "post_date": "08/16/2019 04:19:17", "content": "Amazing Work... Thank you for sharing...!! @shaygu
", "votes": null, "replies": [ { "id": 600479, "author_name": "shaygu", "author_url": "", "post_date": "08/16/2019 06:32:54", "content": "@veeralakrishna Thanks for the reply, glad you liked it!
", "votes": null, "replies": [] } ] }, { "id": 600756, "author_name": "utkukubilay", "author_url": "", "post_date": "08/16/2019 14:19:01", "content": "nice work @shaygu . Thanks for sharing
", "votes": null, "replies": [ { "id": 600770, "author_name": "shaygu", "author_url": "", "post_date": "08/16/2019 14:37:05", "content": "@utkukubilay Thanks, I appriciate the reply!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "599717": "Hi \n\nHere is our kernel: https://www.kaggle.com/shaygu/t3h-d0g-mak3rz-public-43-fid we specifically attached the version that calculates FID every few epochs, we think it brings more insight into the network and also that's the method we used for optimization .\n\nOur knowledge was very limited walking into this competition, and we joined little more then a month ago, so we decided to start with the best public model and see where it gets us. Our steps along the way are further discussed in the kernel, however, I will briefly touch on the points also here. \n\nUsing a lot more parameters for the network, improved our score from 60 to 55. Calculating the FID every few epochs allowed us to observe which network performs better (more consistently), which lead to ability to introduce cyclic learning rate (efficiently now) and small hyper parameter optimizations got us to ~50. The last big step was changing the weight initialization to a broader distribution (0,0.15-1 instead of the 0,0.02 commonly used) got us to ~46 and from there to 43 its minor optimizations. \n\nIt was very interesting competition and we've learned a lot. Hopefully, there will be another GAN competition in the near future.\n\n@eviatarai @yogevh", "600400": "Amazing Work... Thank you for sharing...!! @shaygu", "600479": "veeralakrishna Thanks for the reply, glad you liked it!", "600756": "nice work @shaygu . Thanks for sharing", "600770": "utkukubilay Thanks, I appriciate the reply!" }, "source": "meta" }