![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F366730%2Fda51ab6eaed5fa9ae5edf95ce808847c%2Fgan_metrics_correlation.png?generation=1565879608939273&alt=media\")
Thank you all for the competition, especially the authors of hot topics and kernels!
\n\nI want to ask: how did you design experiments in order to select hyperparameters, the number of epochs and other for your model? What metrics did you look at and what code did you use?
\n\nI used a modified code from the repository to calculate the metrics for each epoch.\nhttps://github.com/xuqiantong/GAN-Metrics
\n\nMy script has been preserved for every epoch:\n- Model checkpoints\n- Generated dogs (fixed noise) for visual check\n- 31 metrics
\n\nThe script counts 31 metrics:\n'wasserstein', 'mmd', 'acc', 'acc_real', 'acc_fake', 'precision', 'recall' for 4 pretrained model layers (28)\ninception_score + modescore + fid (3)
\n\nA little later, I noticed that they all correlate very strongly and I left only Inception Score + FID. But in the end, I still look only at the FID.\n
However, when I started submitting, I realized that the best number of the epoch and parameters are not the best on the public leaderboard. (from the choice of the epoch number score could significantly change) So I did not understand what is going on.
", "messages": [ { "id": 600012, "postDate": "2019-08-15T14:35:11.463Z", "content": "Thank you all for the competition, especially the authors of hot topics and kernels!
\n\nI want to ask: how did you design experiments in order to select hyperparameters, the number of epochs and other for your model? What metrics did you look at and what code did you use?
\n\nI used a modified code from the repository to calculate the metrics for each epoch.\nhttps://github.com/xuqiantong/GAN-Metrics
\n\nMy script has been preserved for every epoch:\n- Model checkpoints\n- Generated dogs (fixed noise) for visual check\n- 31 metrics
\n\nThe script counts 31 metrics:\n'wasserstein', 'mmd', 'acc', 'acc_real', 'acc_fake', 'precision', 'recall' for 4 pretrained model layers (28)\ninception_score + modescore + fid (3)
\n\nA little later, I noticed that they all correlate very strongly and I left only Inception Score + FID. But in the end, I still look only at the FID.\n
However, when I started submitting, I realized that the best number of the epoch and parameters are not the best on the public leaderboard. (from the choice of the epoch number score could significantly change) So I did not understand what is going on.
", "rawMarkdown": "Thank you all for the competition, especially the authors of hot topics and kernels!\n\nI want to ask: how did you design experiments in order to select hyperparameters, the number of epochs and other for your model? What metrics did you look at and what code did you use?\n\nI used a modified code from the repository to calculate the metrics for each epoch.\nhttps://github.com/xuqiantong/GAN-Metrics\n\nMy script has been preserved for every epoch:\n- Model checkpoints\n- Generated dogs (fixed noise) for visual check\n- 31 metrics\n\nThe script counts 31 metrics:\n'wasserstein', 'mmd', 'acc', 'acc_real', 'acc_fake', 'precision', 'recall' for 4 pretrained model layers (28)\ninception_score + modescore + fid (3)\n\nA little later, I noticed that they all correlate very strongly and I left only Inception Score + FID. But in the end, I still look only at the FID.\n\n\nHowever, when I started submitting, I realized that the best number of the epoch and parameters are not the best on the public leaderboard. (from the choice of the epoch number score could significantly change) So I did not understand what is going on.", "votes": 2 } ], "comments": [], "raw_markdown_by_id": { "600012": "Thank you all for the competition, especially the authors of hot topics and kernels!\n\nI want to ask: how did you design experiments in order to select hyperparameters, the number of epochs and other for your model? What metrics did you look at and what code did you use?\n\nI used a modified code from the repository to calculate the metrics for each epoch.\nhttps://github.com/xuqiantong/GAN-Metrics\n\nMy script has been preserved for every epoch:\n- Model checkpoints\n- Generated dogs (fixed noise) for visual check\n- 31 metrics\n\nThe script counts 31 metrics:\n'wasserstein', 'mmd', 'acc', 'acc_real', 'acc_fake', 'precision', 'recall' for 4 pretrained model layers (28)\ninception_score + modescore + fid (3)\n\nA little later, I noticed that they all correlate very strongly and I left only Inception Score + FID. But in the end, I still look only at the FID.\n\n\nHowever, when I started submitting, I realized that the best number of the epoch and parameters are not the best on the public leaderboard. (from the choice of the epoch number score could significantly change) So I did not understand what is going on." } }