![\"image\"](\"http://playagricola.com/Kaggle/warp7719.png\")
I think most people know already that memorized GAN is allowed (at least for now). \nThis is really disappointing! Coz it's absolutely meaningless.
\n\nI understand it's really hard to check but as long as the rules are clear and you are willing to check the winners and the gold medals (guess just 11 teams), most people will not use memorized GAN, since it will easily goes to the top of the LB and be disqualified. Even for silver medals, it's gonna be hard to tune your model for not being checked!
\n\nSo please do not allow memorized GAN in this competition!
", "messages": [ { "id": 572880, "postDate": "2019-07-11T14:23:09.330Z", "content": "I think most people know already that memorized GAN is allowed (at least for now). \nThis is really disappointing! Coz it's absolutely meaningless.
\n\nI understand it's really hard to check but as long as the rules are clear and you are willing to check the winners and the gold medals (guess just 11 teams), most people will not use memorized GAN, since it will easily goes to the top of the LB and be disqualified. Even for silver medals, it's gonna be hard to tune your model for not being checked!
\n\nSo please do not allow memorized GAN in this competition!
", "rawMarkdown": "I think most people know already that memorized GAN is allowed (at least for now). \nThis is really disappointing! Coz it's absolutely meaningless.\n\nI understand it's really hard to check but as long as the rules are clear and you are willing to check the winners and the gold medals (guess just 11 teams), most people will not use memorized GAN, since it will easily goes to the top of the LB and be disqualified. Even for silver medals, it's gonna be hard to tune your model for not being checked!\n\nSo please do not allow memorized GAN in this competition!", "votes": 17 }, { "id": 573002, "postDate": "2019-07-11T16:38:42.817Z", "content": "We should ask @cdeotte to check all our models!! ahahaha
", "rawMarkdown": "We should ask @cdeotte to check all our models!! ahahaha", "votes": 4, "replies": [ { "id": 573011, "postDate": "2019-07-11T16:50:17.353Z", "content": "haha, that's a lot of work! I see that your kernel is scoring LB 84 now. Great work!
\n\nI've been playing around with it. Specifically, I converted the loss functions from DCGAN to RaLSGAN explained here. Then I changed real_label = 1.0 back to one for the new loss functions. I also changed picture display to plt.imshow((img+1.)/2.) to fix the dark images.
Seems amazing. I will try these changes too. I'm trying now to work on a new architecture. I will share more if it works.
", "rawMarkdown": "Seems amazing. I will try these changes too. I'm trying now to work on a new architecture. I will share more if it works. ", "votes": 2 } ] }, { "id": 578680, "postDate": "2019-07-18T02:46:45.663Z", "content": "I think the reason why memorized GANs has not been banned is that the score in private LB will be calculated by another dataset. While memorized GANs will not get a good score because of overfitting public dataset. Kaggle have confidence on it.
", "rawMarkdown": "I think the reason why memorized GANs has not been banned is that the score in private LB will be calculated by another dataset. While memorized GANs will not get a good score because of overfitting public dataset. Kaggle have confidence on it. ", "votes": 1, "replies": [ { "id": 579171, "postDate": "2019-07-18T15:12:15.510Z", "content": "Memorizer Gan will score well on private, as demonstrated here https://www.kaggle.com/jesucristo/private-lb-simulation-i\nThey have not been banned yet (imho) because it is painful to manually check all submissions to see if the gan memorizes or actually learns something
", "rawMarkdown": "Memorizer Gan will score well on private, as demonstrated here https://www.kaggle.com/jesucristo/private-lb-simulation-i\nThey have not been banned yet (imho) because it is painful to manually check all submissions to see if the gan memorizes or actually learns something", "votes": 3 }, { "id": 579177, "postDate": "2019-07-18T15:20:47.037Z", "content": "I'm curious how Kaggle will pick the 11 gold winners. Kaggle will need to review hundreds of top private leaderboard submissions before they will find a non-memorizer GAN. Will they just assume that if your LB score is below 40 you are using a memorizer GAN. In that case, if you make a generalizer GAN that is too good, you won't win.
", "rawMarkdown": "I'm curious how Kaggle will pick the 11 gold winners. Kaggle will need to review hundreds of top private leaderboard submissions before they will find a non-memorizer GAN. Will they just assume that if your LB score is below 40 you are using a memorizer GAN. In that case, if you make a generalizer GAN that is too good, you won't win.", "votes": 3 }, { "id": 579192, "postDate": "2019-07-18T15:37:57.463Z", "content": "If the guy that has the best generelizer GAN is known, then you just have to check the people that are ranked after him until there is enough legit models to fill the gold (silver?) area. \nIf you're in the top 5 you'll have to share your code to claim your prize money so hopefully people with strong legit GANs will let themselves know.
", "rawMarkdown": "If the guy that has the best generelizer GAN is known, then you just have to check the people that are ranked after him until there is enough legit models to fill the gold (silver?) area. \nIf you're in the top 5 you'll have to share your code to claim your prize money so hopefully people with strong legit GANs will let themselves know.", "votes": 1 }, { "id": 579543, "postDate": "2019-07-19T00:22:24.247Z", "content": "Thanks for pointing out that~ It's getting complicated ！
", "rawMarkdown": "Thanks for pointing out that~ It's getting complicated ！" }, { "id": 579750, "postDate": "2019-07-19T07:00:22.990Z", "content": "@cdeotte Chris, since you know everything about this competition :) could you please explain how magic number 40 was calculated for threshold? As far as I understand you can take memorizer GAN and make it a little bit worse so just some threshold number won't help or am I wrong?
", "rawMarkdown": "@cdeotte Chris, since you know everything about this competition :) could you please explain how magic number 40 was calculated for threshold? As far as I understand you can take memorizer GAN and make it a little bit worse so just some threshold number won't help or am I wrong?" }, { "id": 579796, "postDate": "2019-07-19T08:14:20.523Z", "content": "maybe kaggle should organize another competition to classify kernels of this competition into generalizing GANs or memorizer GANs and use the winner's solution to rank participants of this competition xD
", "rawMarkdown": "maybe kaggle should organize another competition to classify kernels of this competition into generalizing GANs or memorizer GANs and use the winner's solution to rank participants of this competition xD" }, { "id": 579802, "postDate": "2019-07-19T08:28:40.443Z", "content": "In Kaggle Freesound 2019, it took 2 weeks to release a LB, and 27 days to finalise the LB. It really depends on how much effort the sponsors want to put in. I will be extremely disappointed if any overt memorizer gans pick up medals.
", "rawMarkdown": "In Kaggle Freesound 2019, it took 2 weeks to release a LB, and 27 days to finalise the LB. It really depends on how much effort the sponsors want to put in. I will be extremely disappointed if any overt memorizer gans pick up medals. ", "votes": 3 }, { "id": 579903, "postDate": "2019-07-19T11:35:15.343Z", "content": "@demonplus (Anna). I just picked 40 arbitrarily since that was the best traditional GAN for a while. Now it's down to 25. My point is, if we could look at all the submitted images right now we would notice that the highest LB scores of 5 thru 40 would just look like original training images to the human eye (with basic preprocessing like crops, rotations etc). So according to the rules, Kaggle could visually inspect them and declare them too similar to original training images and disqualify them all. Kaggle may only accept submissions with warped funny looking dogs.
\n\nSo then the question is, should we apply a warping preprocess so even our traditional GANs don't get accidentally disqualified if they are too good?\n
\n\n\nSo then the question is, should we apply a warping preprocess so even our traditional GANs don't get accidentally disqualified if they are too good?
\n
I noticed that even in my best generated images there are almost always missing small 'scene' details like collars, toys or other minor stuff. Real images looks a lot more feature-rich\nAnd there is much less variety of ground textures.
", "rawMarkdown": "> So then the question is, should we apply a warping preprocess so even our traditional GANs don't get accidentally disqualified if they are too good?\n\nI noticed that even in my best generated images there are almost always missing small 'scene' details like collars, toys or other minor stuff. Real images looks a lot more feature-rich\nAnd there is much less variety of ground textures.", "votes": 2 }, { "id": 579935, "postDate": "2019-07-19T12:17:45.383Z", "content": "I noticed that about your images. Some of your images look like a high resolution dog was cut out of a training image and placed on a mostly solid color background.
\n\nThat makes me wonder. What if I built a CNN that memorizes dog bodies and faces and places them on solid color backgrounds. Is that a valid solution? There is so much gray zone in this comp.
", "rawMarkdown": "I noticed that about your images. Some of your images look like a high resolution dog was cut out of a training image and placed on a mostly solid color background.\n\nThat makes me wonder. What if I built a CNN that memorizes dog bodies and faces and places them on solid color backgrounds. Is that a valid solution? There is so much gray zone in this comp." }, { "id": 579941, "postDate": "2019-07-19T12:23:03.453Z", "content": "@cdeotte I don't think that's valid because you're still memorizing and applying minor transformations to original images. \nHowever a CGAN that has information on the background could be a legit alternative option.
", "rawMarkdown": "@cdeotte I don't think that's valid because you're still memorizing and applying minor transformations to original images. \nHowever a CGAN that has information on the background could be a legit alternative option." }, { "id": 579953, "postDate": "2019-07-19T12:48:32.210Z", "content": "> I noticed that even in my best generated images there are almost always missing small 'scene' details like collars, toys or other minor stuff. Real images looks a lot more feature-rich\nAnd there is much less variety of ground textures.
\n\nIt's interesting that your GAN realizes that the dogs are the most important objects, and that the collars, toys, background are not important. I think this is the advantage of using a CGAN or ACGAN. It forces the GAN to recognize the importance of the dogs and encourages your GAN to draw high resolution dogs (and low res other stuff).
", "rawMarkdown": "> I noticed that even in my best generated images there are almost always missing small 'scene' details like collars, toys or other minor stuff. Real images looks a lot more feature-rich\nAnd there is much less variety of ground textures.\n\nIt's interesting that your GAN realizes that the dogs are the most important objects, and that the collars, toys, background are not important. I think this is the advantage of using a CGAN or ACGAN. It forces the GAN to recognize the importance of the dogs and encourages your GAN to draw high resolution dogs (and low res other stuff)." }, { "id": 579957, "postDate": "2019-07-19T12:59:03.243Z", "content": "Yeah, I have the same thoughts. My previous simpler nets were drawing a lot more grass-looking textures and other background things
", "rawMarkdown": "Yeah, I have the same thoughts. My previous simpler nets were drawing a lot more grass-looking textures and other background things", "votes": 1 } ] }, { "id": 573147, "postDate": "2019-07-11T21:06:39.440Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 573002, "author_name": "François", "author_url": "", "post_date": "2019-07-11T16:38:42.817000", "content": "We should ask @cdeotte to check all our models!! ahahaha
", "votes": 4, "replies": [ { "id": 573011, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-11T16:50:17.353000", "content": "haha, that's a lot of work! I see that your kernel is scoring LB 84 now. Great work!
\n\nI've been playing around with it. Specifically, I converted the loss functions from DCGAN to RaLSGAN explained here. Then I changed real_label = 1.0 back to one for the new loss functions. I also changed picture display to plt.imshow((img+1.)/2.) to fix the dark images.
Seems amazing. I will try these changes too. I'm trying now to work on a new architecture. I will share more if it works.
", "votes": 2, "replies": [] } ] }, { "id": 578680, "author_name": "Jar", "author_url": "", "post_date": "2019-07-18T02:46:45.663000", "content": "I think the reason why memorized GANs has not been banned is that the score in private LB will be calculated by another dataset. While memorized GANs will not get a good score because of overfitting public dataset. Kaggle have confidence on it.
", "votes": 1, "replies": [ { "id": 579171, "author_name": "Theo Viel", "author_url": "", "post_date": "2019-07-18T15:12:15.510000", "content": "Memorizer Gan will score well on private, as demonstrated here https://www.kaggle.com/jesucristo/private-lb-simulation-i\nThey have not been banned yet (imho) because it is painful to manually check all submissions to see if the gan memorizes or actually learns something
", "votes": 3, "replies": [] }, { "id": 579177, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-18T15:20:47.037000", "content": "I'm curious how Kaggle will pick the 11 gold winners. Kaggle will need to review hundreds of top private leaderboard submissions before they will find a non-memorizer GAN. Will they just assume that if your LB score is below 40 you are using a memorizer GAN. In that case, if you make a generalizer GAN that is too good, you won't win.
", "votes": 3, "replies": [] }, { "id": 579192, "author_name": "Theo Viel", "author_url": "", "post_date": "2019-07-18T15:37:57.463000", "content": "If the guy that has the best generelizer GAN is known, then you just have to check the people that are ranked after him until there is enough legit models to fill the gold (silver?) area. \nIf you're in the top 5 you'll have to share your code to claim your prize money so hopefully people with strong legit GANs will let themselves know.
", "votes": 1, "replies": [] }, { "id": 579543, "author_name": "Jar", "author_url": "", "post_date": "2019-07-19T00:22:24.247000", "content": "Thanks for pointing out that~ It's getting complicated ！
", "votes": 0, "replies": [] }, { "id": 579750, "author_name": "Anna Novikova", "author_url": "", "post_date": "2019-07-19T07:00:22.990000", "content": "@cdeotte Chris, since you know everything about this competition :) could you please explain how magic number 40 was calculated for threshold? As far as I understand you can take memorizer GAN and make it a little bit worse so just some threshold number won't help or am I wrong?
", "votes": 0, "replies": [] }, { "id": 579796, "author_name": "timetraveller", "author_url": "", "post_date": "2019-07-19T08:14:20.523000", "content": "maybe kaggle should organize another competition to classify kernels of this competition into generalizing GANs or memorizer GANs and use the winner's solution to rank participants of this competition xD
", "votes": 0, "replies": [] }, { "id": 579802, "author_name": "robga", "author_url": "", "post_date": "2019-07-19T08:28:40.443000", "content": "In Kaggle Freesound 2019, it took 2 weeks to release a LB, and 27 days to finalise the LB. It really depends on how much effort the sponsors want to put in. I will be extremely disappointed if any overt memorizer gans pick up medals.
", "votes": 3, "replies": [] }, { "id": 579903, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-19T11:35:15.343000", "content": "@demonplus (Anna). I just picked 40 arbitrarily since that was the best traditional GAN for a while. Now it's down to 25. My point is, if we could look at all the submitted images right now we would notice that the highest LB scores of 5 thru 40 would just look like original training images to the human eye (with basic preprocessing like crops, rotations etc). So according to the rules, Kaggle could visually inspect them and declare them too similar to original training images and disqualify them all. Kaggle may only accept submissions with warped funny looking dogs.
\n\nSo then the question is, should we apply a warping preprocess so even our traditional GANs don't get accidentally disqualified if they are too good?\n
\n\n\nSo then the question is, should we apply a warping preprocess so even our traditional GANs don't get accidentally disqualified if they are too good?
\n
I noticed that even in my best generated images there are almost always missing small 'scene' details like collars, toys or other minor stuff. Real images looks a lot more feature-rich\nAnd there is much less variety of ground textures.
", "votes": 2, "replies": [] }, { "id": 579935, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-19T12:17:45.383000", "content": "I noticed that about your images. Some of your images look like a high resolution dog was cut out of a training image and placed on a mostly solid color background.
\n\nThat makes me wonder. What if I built a CNN that memorizes dog bodies and faces and places them on solid color backgrounds. Is that a valid solution? There is so much gray zone in this comp.
", "votes": 0, "replies": [] }, { "id": 579941, "author_name": "Theo Viel", "author_url": "", "post_date": "2019-07-19T12:23:03.453000", "content": "@cdeotte I don't think that's valid because you're still memorizing and applying minor transformations to original images. \nHowever a CGAN that has information on the background could be a legit alternative option.
", "votes": 0, "replies": [] }, { "id": 579953, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-19T12:48:32.210000", "content": "> I noticed that even in my best generated images there are almost always missing small 'scene' details like collars, toys or other minor stuff. Real images looks a lot more feature-rich\nAnd there is much less variety of ground textures.
\n\nIt's interesting that your GAN realizes that the dogs are the most important objects, and that the collars, toys, background are not important. I think this is the advantage of using a CGAN or ACGAN. It forces the GAN to recognize the importance of the dogs and encourages your GAN to draw high resolution dogs (and low res other stuff).
", "votes": 0, "replies": [] }, { "id": 579957, "author_name": "Dmitry Vorobiev", "author_url": "", "post_date": "2019-07-19T12:59:03.243000", "content": "Yeah, I have the same thoughts. My previous simpler nets were drawing a lot more grass-looking textures and other background things
", "votes": 1, "replies": [] } ] }, { "id": 573147, "author_name": "", "author_url": "", "post_date": "2019-07-11T21:06:39.440000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "572880": "I think most people know already that memorized GAN is allowed (at least for now). \nThis is really disappointing! Coz it's absolutely meaningless.\n\nI understand it's really hard to check but as long as the rules are clear and you are willing to check the winners and the gold medals (guess just 11 teams), most people will not use memorized GAN, since it will easily goes to the top of the LB and be disqualified. Even for silver medals, it's gonna be hard to tune your model for not being checked!\n\nSo please do not allow memorized GAN in this competition!", "573002": "We should ask @cdeotte to check all our models!! ahahaha", "578680": "I think the reason why memorized GANs has not been banned is that the score in private LB will be calculated by another dataset. While memorized GANs will not get a good score because of overfitting public dataset. Kaggle have confidence on it. ", "573147": "" } }