Thank you for all the participants in this competition.\nI provide my solution that gets 45.6 in public LB.
\n\nMy refactored code is available at https://github.com/fujibo/kaggle-dog.
\n\nMy code is based on SN-GAN with projection discriminator https://github.com/pfnet-research/sngan_projection.\nI add the following modifications to the base code.\n- Apply spectral normalization to the generator\n- Increase the learning rate and reduce the number of critics of the discriminator\nThese techniques are used in SA-GAN, which outperforms SN-GAN.\nI did not use the original SA-GAN code because I am not good at tensorflow.
\n\nMoreover, I tuned hyper-parameters as follows.\n- change batch size from 64 to 128\n- change iteration_decay_start from 0 to 18000\n- change iteration to 24000
Thank you for all the participants in this competition.\nI provide my solution that gets 45.6 in public LB.
\n\nMy refactored code is available at https://github.com/fujibo/kaggle-dog.
\n\nMy code is based on SN-GAN with projection discriminator https://github.com/pfnet-research/sngan_projection.\nI add the following modifications to the base code.\n- Apply spectral normalization to the generator\n- Increase the learning rate and reduce the number of critics of the discriminator\nThese techniques are used in SA-GAN, which outperforms SN-GAN.\nI did not use the original SA-GAN code because I am not good at tensorflow.
\n\nMoreover, I tuned hyper-parameters as follows.\n- change batch size from 64 to 128\n- change iteration_decay_start from 0 to 18000\n- change iteration to 24000
Great job. Congratulations on a great score. What learning rate did you use and how many channels did you change the ResNet discriminator to?
\n\nMy final solution also used the code from https://github.com/pfnet-research/sngan_projection. To increase training time, I cropped squares from the original image bounding boxes, resized them to 80x80, and saved them to the hard drive. This allowed me to perform 33000 iterations with batch size = 32 and score LB 47. (Exponential decay starting at 27000). I left learning rates and discriminator channels unchanged.
", "rawMarkdown": "Great job. Congratulations on a great score. What learning rate did you use and how many channels did you change the ResNet discriminator to?\n\nMy final solution also used the code from https://github.com/pfnet-research/sngan_projection. To increase training time, I cropped squares from the original image bounding boxes, resized them to 80x80, and saved them to the hard drive. This allowed me to perform 33000 iterations with batch size = 32 and score LB 47. (Exponential decay starting at 27000). I left learning rates and discriminator channels unchanged.", "replies": [ { "id": 601080, "postDate": "2019-08-17T01:44:54.277Z", "content": "Thank you for your question.
\n\nI changed alpha from 0.0002 to 0.0005 for a discriminator and n_dis from 5 to 2. I did not change the channel size. I cropped squares following bounding boxes and resized so that the length of shorter edge is 64, then I randomly cropped to 64x64. I set the random seed to 42.
\n\nAfter getting my best score, I changed my code to save the resized images to the hard drive and increased iterations, but I got scores around 46.8 (almost the same score with you). Moreover, when I changed the random seed to 0 without increasing iterations, I got 52.2. From here, I realized that I happened to find good hyper-parameters.
", "rawMarkdown": "Thank you for your question.\n\nI changed alpha from 0.0002 to 0.0005 for a discriminator and n_dis from 5 to 2. I did not change the channel size. I cropped squares following bounding boxes and resized so that the length of shorter edge is 64, then I randomly cropped to 64x64. I set the random seed to 42.\n\nAfter getting my best score, I changed my code to save the resized images to the hard drive and increased iterations, but I got scores around 46.8 (almost the same score with you). Moreover, when I changed the random seed to 0 without increasing iterations, I got 52.2. From here, I realized that I happened to find good hyper-parameters.", "votes": 1 } ] }, { "id": 600593, "postDate": "2019-08-16T10:30:36.307Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 600616, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-08-16T11:13:26.990000", "content": "Great job. Congratulations on a great score. What learning rate did you use and how many channels did you change the ResNet discriminator to?
\n\nMy final solution also used the code from https://github.com/pfnet-research/sngan_projection. To increase training time, I cropped squares from the original image bounding boxes, resized them to 80x80, and saved them to the hard drive. This allowed me to perform 33000 iterations with batch size = 32 and score LB 47. (Exponential decay starting at 27000). I left learning rates and discriminator channels unchanged.
", "votes": 0, "replies": [ { "id": 601080, "author_name": "fujibo", "author_url": "", "post_date": "2019-08-17T01:44:54.277000", "content": "Thank you for your question.
\n\nI changed alpha from 0.0002 to 0.0005 for a discriminator and n_dis from 5 to 2. I did not change the channel size. I cropped squares following bounding boxes and resized so that the length of shorter edge is 64, then I randomly cropped to 64x64. I set the random seed to 42.
\n\nAfter getting my best score, I changed my code to save the resized images to the hard drive and increased iterations, but I got scores around 46.8 (almost the same score with you). Moreover, when I changed the random seed to 0 without increasing iterations, I got 52.2. From here, I realized that I happened to find good hyper-parameters.
", "votes": 1, "replies": [] } ] }, { "id": 600593, "author_name": "", "author_url": "", "post_date": "2019-08-16T10:30:36.307000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "600406": "Thank you for all the participants in this competition.\nI provide my solution that gets 45.6 in public LB.\n\n## Solution\nMy refactored code is available at https://github.com/fujibo/kaggle-dog.\n\nMy code is based on SN-GAN with projection discriminator https://github.com/pfnet-research/sngan_projection.\nI add the following modifications to the base code.\n- Apply spectral normalization to the generator\n- Increase the learning rate and reduce the number of critics of the discriminator\nThese techniques are used in SA-GAN, which outperforms SN-GAN.\nI did not use the original SA-GAN code because I am not good at tensorflow.\n\nMoreover, I tuned hyper-parameters as follows.\n- change `batch size` from 64 to 128\n- change `iteration_decay_start` from 0 to 18000\n- change `iteration` to 24000\n\n## Insights\n- For reducing training time, I produced dog images with 32x32 pixels followed by bilinear resizing, but obtained a worse score.\n- Different trials have different training time somehow, that is, trial A cannot finish 27000 iterations, while trial B can finish 28000 iterations.", "600616": "Great job. Congratulations on a great score. What learning rate did you use and how many channels did you change the ResNet discriminator to?\n\nMy final solution also used the code from https://github.com/pfnet-research/sngan_projection. To increase training time, I cropped squares from the original image bounding boxes, resized them to 80x80, and saved them to the hard drive. This allowed me to perform 33000 iterations with batch size = 32 and score LB 47. (Exponential decay starting at 27000). I left learning rates and discriminator channels unchanged.", "600593": "" } }