![\"image\"](\"http://playagricola.com/Kaggle/doga71019.png\")
As the leaderboard is ruined by Memorization Gan, which may not be allowed. However, I believe many people who use traditional Gan want to know there position and how far can traditional Gan goes, so I create this topic, anyone who use traditional Gan could comments about your Gan's score, as a reference to other people.\nAs for me, my best score of traditional Gan is 96.18. \nWish and wait for more comments. Thanks.
\n\nUpdate: Using Ralsgan helps me break to 74.36. And it's seems like Memorization Gan will be acceptable. Though it may cause over fit, I still think it would perform better than traditional Gan since no matter how big the difference public data and mysterious data have, dogs are always dogs, it will not change just because of the changing of data.
\n\nUpdate: 42 by now.
", "messages": [ { "id": 571707, "postDate": "2019-07-10T02:16:02.103Z", "content": "As the leaderboard is ruined by Memorization Gan, which may not be allowed. However, I believe many people who use traditional Gan want to know there position and how far can traditional Gan goes, so I create this topic, anyone who use traditional Gan could comments about your Gan's score, as a reference to other people.\nAs for me, my best score of traditional Gan is 96.18. \nWish and wait for more comments. Thanks.
\n\nUpdate: Using Ralsgan helps me break to 74.36. And it's seems like Memorization Gan will be acceptable. Though it may cause over fit, I still think it would perform better than traditional Gan since no matter how big the difference public data and mysterious data have, dogs are always dogs, it will not change just because of the changing of data.
\n\nUpdate: 42 by now.
", "rawMarkdown": "As the leaderboard is ruined by Memorization Gan, which may not be allowed. However, I believe many people who use traditional Gan want to know there position and how far can traditional Gan goes, so I create this topic, anyone who use traditional Gan could comments about your Gan's score, as a reference to other people.\nAs for me, my best score of traditional Gan is 96.18. \nWish and wait for more comments. Thanks.\n\nUpdate: Using Ralsgan helps me break to 74.36. And it's seems like Memorization Gan will be acceptable. Though it may cause over fit, I still think it would perform better than traditional Gan since no matter how big the difference public data and mysterious data have, dogs are always dogs, it will not change just because of the changing of data.\n\nUpdate: 42 by now.", "votes": 15 }, { "id": 572897, "postDate": "2019-07-11T14:45:47.087Z", "content": "I am struggle at 40+. My best score for now is 39.13987, but I think it was mostly luck rather than some clever changes.\nDunno how to get below 40.
", "rawMarkdown": "I am struggle at 40+. My best score for now is 39.13987, but I think it was mostly luck rather than some clever changes.\nDunno how to get below 40.", "votes": 7, "replies": [ { "id": 572926, "postDate": "2019-07-11T15:18:29.807Z", "content": "Wow 40 is amazing! Great job Dmitry!
", "rawMarkdown": "Wow 40 is amazing! Great job Dmitry!", "votes": 1 }, { "id": 572934, "postDate": "2019-07-11T15:23:52.337Z", "content": "Thanks
", "rawMarkdown": "Thanks" }, { "id": 574135, "postDate": "2019-07-13T09:33:31.763Z", "content": "The better your score, the harder it is to improve. And as you're probably at the top right now it is gonna be complicated :)
", "rawMarkdown": "The better your score, the harder it is to improve. And as you're probably at the top right now it is gonna be complicated :) ", "votes": 2 } ] }, { "id": 579776, "postDate": "2019-07-19T07:30:24.283Z", "content": "LB 40+ with StyleGAN.\nI feel this competition is fun, though there is a problem on rules.
", "rawMarkdown": "LB 40+ with StyleGAN.\nI feel this competition is fun, though there is a problem on rules.", "votes": 5, "replies": [ { "id": 579794, "postDate": "2019-07-19T08:12:26.500Z", "content": "Wow, that's cool :) I thought it was impossible to train it in the 9 hours limit
", "rawMarkdown": "Wow, that's cool :) I thought it was impossible to train it in the 9 hours limit" }, { "id": 579801, "postDate": "2019-07-19T08:27:21.067Z", "content": "Yeah, I modified network shallower. And still struggling with kernel time limit 🏃 ⏲
", "rawMarkdown": "Yeah, I modified network shallower. And still struggling with kernel time limit 🏃 ⏲ ", "votes": 1 }, { "id": 579960, "postDate": "2019-07-19T13:06:42.070Z", "content": "StyleGAN is conditional right ? The dog labels are used ?
", "rawMarkdown": "StyleGAN is conditional right ? The dog labels are used ? " }, { "id": 579974, "postDate": "2019-07-19T13:39:54.997Z", "content": "I think, no. At least I didn't find such option out of the box. If I understand correctly, the point of styleGAN is that you can manipulate the various features of generated images scaling up and down parts of the noise vector. Maybe the breed itself can be encoded as a combination of these features eventually. But there is no explicit option to pass additional classes
", "rawMarkdown": "I think, no. At least I didn't find such option out of the box. If I understand correctly, the point of styleGAN is that you can manipulate the various features of generated images scaling up and down parts of the noise vector. Maybe the breed itself can be encoded as a combination of these features eventually. But there is no explicit option to pass additional classes" } ] }, { "id": 574134, "postDate": "2019-07-13T09:32:13.993Z", "content": "57-ish with 300 epochs, which took me about 7 hours.\nBut GANs are instable and the metric varies a lot between two sets of images from the same generator so it is quite hard to tweak models.
", "rawMarkdown": "57-ish with 300 epochs, which took me about 7 hours.\nBut GANs are instable and the metric varies a lot between two sets of images from the same generator so it is quite hard to tweak models.", "votes": 5 }, { "id": 571994, "postDate": "2019-07-10T10:11:30.903Z", "content": "My 2 best scores are Memorizer GAN = 5.5 :-) and Traditional GAN = 65 after 150 epochs (6.5 hours):
\n\n
Hey, I just have a question your Memorizer GAN. For this competition, they won't take a Memorizer GAN into consideration as a final submission right?
", "rawMarkdown": "Hey, I just have a question your Memorizer GAN. For this competition, they won't take a Memorizer GAN into consideration as a final submission right?", "votes": 1 }, { "id": 580331, "postDate": "2019-07-20T02:29:51.417Z", "content": "It is unclear. A Kaggle admin said that the Memorizer GAN is valid here. But the original rules imply that memorizing and regurgitating images (or slightly altered) is invalid. So I don't know.
", "rawMarkdown": "It is unclear. A Kaggle admin said that the Memorizer GAN is valid [here][1]. But the original rules imply that memorizing and regurgitating images (or slightly altered) is invalid. So I don't know.\n\n[1]: https://www.kaggle.com/c/generative-dog-images/discussion/98183#572299" }, { "id": 580338, "postDate": "2019-07-20T02:35:48.497Z", "content": "Thank you for the information.
", "rawMarkdown": "Thank you for the information." } ] }, { "id": 571816, "postDate": "2019-07-10T05:43:31.680Z", "content": "I'm at 71. I think most people 40+ are using genuine gans.
\n\nI bet a whole bunch of people with 7-40 scores also have a good scoring traditional GAN. It is disappointing to see kaggle regulars, who I respect greatly, submitting LB killing scores. What good does it do to be the 50th person with a score of 7.2?
", "rawMarkdown": "I'm at 71. I think most people 40+ are using genuine gans.\n\nI bet a whole bunch of people with 7-40 scores also have a good scoring traditional GAN. It is disappointing to see kaggle regulars, who I respect greatly, submitting LB killing scores. What good does it do to be the 50th person with a score of 7.2?", "votes": 3 }, { "id": 572481, "postDate": "2019-07-11T01:42:37.060Z", "content": "I'm at 47.7 with 1000epochs
", "rawMarkdown": "I'm at 47.7 with 1000epochs", "votes": 4, "replies": [ { "id": 572839, "postDate": "2019-07-11T13:06:06.450Z", "content": "1000 epochs?! \nI think this competition is a lot about optimization. The more epochs I could fit, the better score I get.
", "rawMarkdown": "1000 epochs?! \nI think this competition is a lot about optimization. The more epochs I could fit, the better score I get.", "votes": 1 } ] }, { "id": 578965, "postDate": "2019-07-18T10:56:09.247Z", "content": "Official starter kernel trained on both original images and bbox cropped images with batch size=96 to 800 epochs got LB 62
Update: batch size=16 and 400 epochs got LB 52
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2748588%2F0d7b03d698f0c59edb5c6e6f102c5a99%2F_15634257189249.png?generation=1563425806245778&alt=media\")
\nmy rals gan with reranking labels
I'm at 64 with some change of DCGAN
", "rawMarkdown": "I'm at 64 with some change of DCGAN", "votes": 1 }, { "id": 572745, "postDate": "2019-07-11T11:01:54.927Z", "content": "At 72 now with traditional DCGAN . Did not crop dog images using bounding box during this run .
", "rawMarkdown": "At 72 now with traditional DCGAN . Did not crop dog images using bounding box during this run .", "votes": 1, "replies": [ { "id": 572765, "postDate": "2019-07-11T11:33:41.997Z", "content": "Nice work! I also used dcgan before but can only get the score more than 100. Perhaps missed something.
", "rawMarkdown": "Nice work! I also used dcgan before but can only get the score more than 100. Perhaps missed something." }, { "id": 572845, "postDate": "2019-07-11T13:16:33.187Z", "content": "@leonshangguan I think winning solution will balance between the complexity of the model and the number of epochs they will be able to squeze given kernels limits. More complex models have more weights and greater potential, but they are slower and tend to use more time to reach great performance, whereas lighter models can be trained quickly with many epochs
", "rawMarkdown": "@leonshangguan I think winning solution will balance between the complexity of the model and the number of epochs they will be able to squeze given kernels limits. More complex models have more weights and greater potential, but they are slower and tend to use more time to reach great performance, whereas lighter models can be trained quickly with many epochs", "votes": 4 } ] }, { "id": 571911, "postDate": "2019-07-10T08:24:20.077Z", "content": "~63 with ~1000 epochs
", "rawMarkdown": "~63 with ~1000 epochs", "votes": 1, "replies": [ { "id": 571914, "postDate": "2019-07-10T08:26:25.173Z", "content": "one silly question may be , how did you manage to run epochs which are 30 seconds long ? Or did you train it outside the Kernel ?
", "rawMarkdown": "one silly question may be , how did you manage to run epochs which are 30 seconds long ? Or did you train it outside the Kernel ?" }, { "id": 571923, "postDate": "2019-07-10T08:35:18.303Z", "content": "I think because my GAN is in pure TensorFlow and really simple, also I keep all the preprocessed data in RAM, I was able to even go for ~1500-1600 epochs with the change of some parameters. In kernel now I check the time from the start of the training, when it passes 32000s I stop training. Reading the data and generating images takes ~200 s more, this way I squeze as much as I can from 9h (32400s) kernels limit.
", "rawMarkdown": "I think because my GAN is in pure TensorFlow and really simple, also I keep all the preprocessed data in RAM, I was able to even go for ~1500-1600 epochs with the change of some parameters. In kernel now I check the time from the start of the training, when it passes 32000s I stop training. Reading the data and generating images takes ~200 s more, this way I squeze as much as I can from 9h (32400s) kernels limit.", "votes": 5 }, { "id": 571929, "postDate": "2019-07-10T08:43:47.130Z", "content": "Yes , thats the trick i am coding now . Time decides what is the number of epoch :) . Thats really cool suggestion thanks .
", "rawMarkdown": "Yes , thats the trick i am coding now . Time decides what is the number of epoch :) . Thats really cool suggestion thanks .", "votes": 1 }, { "id": 571992, "postDate": "2019-07-10T10:09:07.757Z", "content": "Wow, cool. I need to optimize my TensorFlow code. Each of my epochs (all 20000 training images) is taking 70 seconds each.
", "rawMarkdown": "Wow, cool. I need to optimize my TensorFlow code. Each of my epochs (all 20000 training images) is taking 70 seconds each.", "votes": 1 }, { "id": 571997, "postDate": "2019-07-10T10:15:02.337Z", "content": "@cdeotte please check the below Kernel this has a per epoch speed of 35 seconds . I just implemented this for my torch code to be able to run more epoch. Not sure if any idea can be taken up for your work \nhttps://www.kaggle.com/speedwagon/ram-dataloader
", "rawMarkdown": "@cdeotte please check the below Kernel this has a per epoch speed of 35 seconds . I just implemented this for my torch code to be able to run more epoch. Not sure if any idea can be taken up for your work \nhttps://www.kaggle.com/speedwagon/ram-dataloader " } ] }, { "id": 571729, "postDate": "2019-07-10T03:27:42.597Z", "content": "95.18 with Ralsgan.
", "rawMarkdown": "95.18 with Ralsgan.", "votes": 1, "replies": [ { "id": 580957, "postDate": "2019-07-21T06:26:44.197Z", "content": "So far 57 after 100 epoch, 46 after 300 epoch, 41 after 550 epoch. Used DCGAN.
", "rawMarkdown": "So far 57 after 100 epoch, 46 after 300 epoch, 41 after 550 epoch. Used DCGAN.", "votes": 2 } ] }, { "id": 574731, "postDate": "2019-07-14T12:01:14.373Z", "content": "Here's a basic TensorFlow DCGAN using RaLS loss. (It also uses bounding box info with inscribed cropped squares described below. And it randomly flips images horizontally with 50% probability). It achieves score LB 72. This is a baseline. To beat this, you need to be creative and add things. (It achieves LB 85/80/75/73/72 after 100/125/150/175/200 epochs).
\n\n![\"image\"](\"http://playagricola.com/Kaggle/score71419.png\")
UPDATE: Here's a basic PyTorch DCGAN using RaLS loss (and inscribed bounding box crop and random horizontal flips). It looks like it performs the same as TensorFlow. (It achieves LB 90/80/77/77/74 after 100/125/150/175/200 epochs).\n![\"image\"](\"http://playagricola.com/Kaggle/scoreb71419.png\")
Great! \nBefore LB 40 (I think someone got that with traditional GAN) I should try breaking the baseline you've provided. \nI made a similar GAN with RaLS loss and trained it for 1k epochs, my LB score was ~91\nI made it on pytorch, something must be wrong with it.\nI haven't used the box annotations, maybe that.
", "rawMarkdown": "Great! \nBefore LB 40 (I think someone got that with traditional GAN) I should try breaking the baseline you've provided. \nI made a similar GAN with RaLS loss and trained it for 1k epochs, my LB score was ~91\nI made it on pytorch, something must be wrong with it.\nI haven't used the box annotations, maybe that. " }, { "id": 574743, "postDate": "2019-07-14T12:16:21.483Z", "content": "This GAN uses the box annotations with inscribed square and random horizontal flipping. (I'll add that comment above). For some reason, I've found that TensorFlow does better than PyTorch with seemingly the same code
\n\nEDIT: I'm not a PyTorch expert but does the starter code adjust the Discriminator weights before we input fake images for the purpose of adjusting the Generator weights? (PyTorch inputs 1 batch of real and 2 batch of fake during training). The TensorFlow code adjusts the weights at the same time. (TensorFlow inputs 1 batch real and 1 batch fake).
\n\nThe PyTorch tutorial is here, the TensorFlow tutorial is here
", "rawMarkdown": "This GAN uses the box annotations with inscribed square and random horizontal flipping. (I'll add that comment above). For some reason, I've found that TensorFlow does better than PyTorch with seemingly the same code\n\nEDIT: I'm not a PyTorch expert but does the starter code adjust the Discriminator weights **before** we input fake images for the purpose of adjusting the Generator weights? (PyTorch inputs 1 batch of real and 2 batch of fake during training). The TensorFlow code adjusts the weights at the same time. (TensorFlow inputs 1 batch real and 1 batch fake).\n\nThe PyTorch tutorial is [here][1], the TensorFlow tutorial is [here][2]\n\n[1]: https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html\n[2]: https://www.tensorflow.org/beta/tutorials/generative/dcgan", "votes": 1 }, { "id": 574746, "postDate": "2019-07-14T12:21:26.150Z", "content": ">I've found that TensorFlow does better than PyTorch with seemingly the same code
\n\nI see, I must get used to using TensorFlow. Now is the time.\nThank you!
", "rawMarkdown": ">I've found that TensorFlow does better than PyTorch with seemingly the same code\n\nI see, I must get used to using TensorFlow. Now is the time.\nThank you! " }, { "id": 574750, "postDate": "2019-07-14T12:24:32.330Z", "content": "See my editted comment above. It may be because PyTorch updates G and D differently than TensorFlow. If this is the case and cause, maybe it can be changed.
", "rawMarkdown": "See my editted comment above. It may be because PyTorch updates G and D differently than TensorFlow. If this is the case and cause, maybe it can be changed.", "votes": 1 }, { "id": 574753, "postDate": "2019-07-14T12:29:10.180Z", "content": "An advantage of PyTorch is that it is easy to make fast (and better functions for data augmentation). Vlad's public RaLSGAN PyTorch kernel here can do 1000 iterations in 9 hours! He uses his RAM dataloader. I haven't been able to get TensorFlow that fast in eager_execution (easy programming) mode. If I can't speed up EE, then I'll have to reprogram my TensorFlow GAN in (difficult) Sessions and Graph mode.
", "rawMarkdown": "An advantage of PyTorch is that it is easy to make fast (and better functions for data augmentation). Vlad's public RaLSGAN PyTorch kernel [here][1] can do 1000 iterations in 9 hours! He uses his RAM dataloader. I haven't been able to get TensorFlow that fast in eager_execution (easy programming) mode. If I can't speed up EE, then I'll have to reprogram my TensorFlow GAN in (difficult) Sessions and Graph mode.\n\n[1]: https://www.kaggle.com/speedwagon/ralsgan-dogs", "votes": 1 }, { "id": 574754, "postDate": "2019-07-14T12:29:50.650Z", "content": "Yes, I found that odd in the started code. I have already changed it to the way weights are updated in your kernels (at the same time). \nI just needed a reason to learn tensorflow and I found it xD
", "rawMarkdown": "Yes, I found that odd in the started code. I have already changed it to the way weights are updated in your kernels (at the same time). \nI just needed a reason to learn tensorflow and I found it xD", "votes": 1 }, { "id": 574990, "postDate": "2019-07-14T20:04:05.450Z", "content": "I re-ran PyTorch starter code with RaLS loss, inscribed bounding box square, random horizontal flips. It performs the same as TensorFlow. I added the plot above.
", "rawMarkdown": "I re-ran PyTorch starter code with RaLS loss, inscribed bounding box square, random horizontal flips. It performs the same as TensorFlow. I added the plot above.", "votes": 2 } ] }, { "id": 571722, "postDate": "2019-07-10T03:00:50.680Z", "content": "Mine so far is 99.78 after 200 epochs . But I saw some people have gone till 50s as well .
", "rawMarkdown": "Mine so far is 99.78 after 200 epochs . But I saw some people have gone till 50s as well .", "votes": 2 }, { "id": 574188, "postDate": "2019-07-13T10:55:07.600Z", "content": "Leon, are you using the bounding boxes to extract the dogs? With the full images, my best score is 75 with just dogs my best is 65.
", "rawMarkdown": "Leon, are you using the bounding boxes to extract the dogs? With the full images, my best score is 75 with just dogs my best is 65.", "replies": [ { "id": 574212, "postDate": "2019-07-13T12:11:10.037Z", "content": "In my case, crop the data made worse score. Perhaps because of my crop function, I crop with the max(xdiff, ydiff) in order to make the image in square size.
", "rawMarkdown": "In my case, crop the data made worse score. Perhaps because of my crop function, I crop with the max(xdiff, ydiff) in order to make the image in square size." }, { "id": 574219, "postDate": "2019-07-13T13:04:49.043Z", "content": "Do you mean if the bounding box is 50 wide by 100 tall, then you used a square that is 100 by 100 circumscribed (then resize 64x64)? I score 65 by using a square that is 50 by 50 inscribed (then resize to 64x64). I use min(xdiff,ydiff) and locate it at (0,0) relative to bounding box.
If you circumscribe then some of your crops contains black pixels when there was no image for the circumscribed to use.
", "rawMarkdown": "Do you mean if the bounding box is 50 wide by 100 tall, then you used a square that is 100 by 100 circumscribed (then resize 64x64)? I score 65 by using a square that is 50 by 50 inscribed (then resize to 64x64). I use `min(xdiff,ydiff)` and locate it at (0,0) relative to bounding box.\n\nIf you circumscribe then some of your crops contains black pixels when there was no image for the circumscribed to use." }, { "id": 574230, "postDate": "2019-07-13T13:31:29.413Z", "content": "Yeah, that's exactly what I did. I will try your method. I crop in that way because I don't want to loss any information though it produces noise, but it seems not as well as I expected.
", "rawMarkdown": "Yeah, that's exactly what I did. I will try your method. I crop in that way because I don't want to loss any information though it produces noise, but it seems not as well as I expected.", "votes": 1 }, { "id": 574258, "postDate": "2019-07-13T14:28:40.670Z", "rawMarkdown": "", "isDeleted": true }, { "id": 574263, "postDate": "2019-07-13T14:34:13.843Z", "content": "Note that when I use min, I don't center the square. For a tall rectangle (h>w), I move square up. For a wide rectangle (w>h), I move square left.
Thanks, by crop in min(xdiff, ydiff), my GAN reaches LB 68 now.
", "rawMarkdown": "Thanks, by crop in min(xdiff, ydiff), my GAN reaches LB 68 now.", "votes": 1 } ] }, { "id": 572921, "postDate": "2019-07-11T15:16:09.877Z", "content": "Since memorizer GANs like chris' kernel are allowed (source). Should we focus less on \"pure\" GANs? Also, my best score till now is 105 with a simple DCGAN. I am currently running my own version of \"RaSLGAN\" here, I am hoping to get a better score.
", "rawMarkdown": "Since memorizer GANs like chris' kernel are allowed ([source](https://www.kaggle.com/c/generative-dog-images/discussion/98183#572299)). Should we focus less on \"pure\" GANs? Also, my best score till now is 105 with a simple DCGAN. I am currently running my own version of \"RaSLGAN\" here, I am hoping to get a better score.", "replies": [ { "id": 572929, "postDate": "2019-07-11T15:19:59.307Z", "content": "Has anyone tried starting with a Memorizer GAN (where Generator has DCGAN architecture). Then training a (DCGAN) Discriminator to match the Generator's performance. And then continuing with traditional iterations of GAN?
", "rawMarkdown": "Has anyone tried starting with a Memorizer GAN (where Generator has DCGAN architecture). Then training a (DCGAN) Discriminator to match the Generator's performance. And then continuing with traditional iterations of GAN?", "votes": 1 }, { "id": 579894, "postDate": "2019-07-19T11:26:38.923Z", "content": "@timetraveller98 imo there is a non zero probability that Memorizer GANs will not be eligible for gold as it is not very hard to check 10~20 kernels. You could try to temper it not to have such a low score, but then you would be close to the best non-memorizer gans, so what would be the point ? 😃
\n\nAnyway, if you find it fun to hyperoptimize a memorizer gan go ahead, though I doubt that you will take much pleasure from it after a few hours. 😄
", "rawMarkdown": "@timetraveller98 imo there is a non zero probability that Memorizer GANs will not be eligible for gold as it is not very hard to check 10~20 kernels. You could try to temper it not to have such a low score, but then you would be close to the best non-memorizer gans, so what would be the point ? 😃\n\nAnyway, if you find it fun to hyperoptimize a memorizer gan go ahead, though I doubt that you will take much pleasure from it after a few hours. 😄" } ] }, { "id": 571787, "postDate": "2019-07-10T05:08:52.917Z", "content": "85.72 with 200 epochs
", "rawMarkdown": "85.72 with 200 epochs", "replies": [ { "id": 571796, "postDate": "2019-07-10T05:19:36.280Z", "content": "Thats awesome !
", "rawMarkdown": "Thats awesome !" } ] } ], "comments": [ { "id": 572897, "author_name": "Dmitry Vorobiev", "author_url": "", "post_date": "2019-07-11T14:45:47.087000", "content": "I am struggle at 40+. My best score for now is 39.13987, but I think it was mostly luck rather than some clever changes.\nDunno how to get below 40.
", "votes": 7, "replies": [ { "id": 572926, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-11T15:18:29.807000", "content": "Wow 40 is amazing! Great job Dmitry!
", "votes": 1, "replies": [] }, { "id": 572934, "author_name": "Dmitry Vorobiev", "author_url": "", "post_date": "2019-07-11T15:23:52.337000", "content": "Thanks
", "votes": 0, "replies": [] }, { "id": 574135, "author_name": "Theo Viel", "author_url": "", "post_date": "2019-07-13T09:33:31.763000", "content": "The better your score, the harder it is to improve. And as you're probably at the top right now it is gonna be complicated :)
", "votes": 2, "replies": [] } ] }, { "id": 579776, "author_name": "tkato", "author_url": "", "post_date": "2019-07-19T07:30:24.283000", "content": "LB 40+ with StyleGAN.\nI feel this competition is fun, though there is a problem on rules.
", "votes": 5, "replies": [ { "id": 579794, "author_name": "Dmitry Vorobiev", "author_url": "", "post_date": "2019-07-19T08:12:26.500000", "content": "Wow, that's cool :) I thought it was impossible to train it in the 9 hours limit
", "votes": 0, "replies": [] }, { "id": 579801, "author_name": "tkato", "author_url": "", "post_date": "2019-07-19T08:27:21.067000", "content": "Yeah, I modified network shallower. And still struggling with kernel time limit 🏃 ⏲
", "votes": 1, "replies": [] }, { "id": 579960, "author_name": "Nirjhar Roy", "author_url": "", "post_date": "2019-07-19T13:06:42.070000", "content": "StyleGAN is conditional right ? The dog labels are used ?
", "votes": 0, "replies": [] }, { "id": 579974, "author_name": "Dmitry Vorobiev", "author_url": "", "post_date": "2019-07-19T13:39:54.997000", "content": "I think, no. At least I didn't find such option out of the box. If I understand correctly, the point of styleGAN is that you can manipulate the various features of generated images scaling up and down parts of the noise vector. Maybe the breed itself can be encoded as a combination of these features eventually. But there is no explicit option to pass additional classes
", "votes": 0, "replies": [] } ] }, { "id": 574134, "author_name": "Theo Viel", "author_url": "", "post_date": "2019-07-13T09:32:13.993000", "content": "57-ish with 300 epochs, which took me about 7 hours.\nBut GANs are instable and the metric varies a lot between two sets of images from the same generator so it is quite hard to tweak models.
", "votes": 5, "replies": [] }, { "id": 571994, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-10T10:11:30.903000", "content": "My 2 best scores are Memorizer GAN = 5.5 :-) and Traditional GAN = 65 after 150 epochs (6.5 hours):
\n\n
Hey, I just have a question your Memorizer GAN. For this competition, they won't take a Memorizer GAN into consideration as a final submission right?
", "votes": 1, "replies": [] }, { "id": 580331, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-20T02:29:51.417000", "content": "It is unclear. A Kaggle admin said that the Memorizer GAN is valid here. But the original rules imply that memorizing and regurgitating images (or slightly altered) is invalid. So I don't know.
", "votes": 0, "replies": [] }, { "id": 580338, "author_name": "Aravind Manoharan ", "author_url": "", "post_date": "2019-07-20T02:35:48.497000", "content": "Thank you for the information.
", "votes": 0, "replies": [] } ] }, { "id": 571816, "author_name": "robga", "author_url": "", "post_date": "2019-07-10T05:43:31.680000", "content": "I'm at 71. I think most people 40+ are using genuine gans.
\n\nI bet a whole bunch of people with 7-40 scores also have a good scoring traditional GAN. It is disappointing to see kaggle regulars, who I respect greatly, submitting LB killing scores. What good does it do to be the 50th person with a score of 7.2?
", "votes": 3, "replies": [] }, { "id": 572481, "author_name": "Tom", "author_url": "", "post_date": "2019-07-11T01:42:37.060000", "content": "I'm at 47.7 with 1000epochs
", "votes": 4, "replies": [ { "id": 572839, "author_name": "timetraveller", "author_url": "", "post_date": "2019-07-11T13:06:06.450000", "content": "1000 epochs?! \nI think this competition is a lot about optimization. The more epochs I could fit, the better score I get.
", "votes": 1, "replies": [] } ] }, { "id": 578965, "author_name": "T Nakamura", "author_url": "", "post_date": "2019-07-18T10:56:09.247000", "content": "Official starter kernel trained on both original images and bbox cropped images with batch size=96 to 800 epochs got LB 62
Update: batch size=16 and 400 epochs got LB 52
\nmy rals gan with reranking labels
I'm at 64 with some change of DCGAN
", "votes": 1, "replies": [] }, { "id": 572745, "author_name": "Nirjhar Roy", "author_url": "", "post_date": "2019-07-11T11:01:54.927000", "content": "At 72 now with traditional DCGAN . Did not crop dog images using bounding box during this run .
", "votes": 1, "replies": [ { "id": 572765, "author_name": "Zhongkai Shangguan", "author_url": "", "post_date": "2019-07-11T11:33:41.997000", "content": "Nice work! I also used dcgan before but can only get the score more than 100. Perhaps missed something.
", "votes": 0, "replies": [] }, { "id": 572845, "author_name": "Kostya Atarik", "author_url": "", "post_date": "2019-07-11T13:16:33.187000", "content": "@leonshangguan I think winning solution will balance between the complexity of the model and the number of epochs they will be able to squeze given kernels limits. More complex models have more weights and greater potential, but they are slower and tend to use more time to reach great performance, whereas lighter models can be trained quickly with many epochs
", "votes": 4, "replies": [] } ] }, { "id": 571911, "author_name": "Kostya Atarik", "author_url": "", "post_date": "2019-07-10T08:24:20.077000", "content": "~63 with ~1000 epochs
", "votes": 1, "replies": [ { "id": 571914, "author_name": "Nirjhar Roy", "author_url": "", "post_date": "2019-07-10T08:26:25.173000", "content": "one silly question may be , how did you manage to run epochs which are 30 seconds long ? Or did you train it outside the Kernel ?
", "votes": 0, "replies": [] }, { "id": 571923, "author_name": "Kostya Atarik", "author_url": "", "post_date": "2019-07-10T08:35:18.303000", "content": "I think because my GAN is in pure TensorFlow and really simple, also I keep all the preprocessed data in RAM, I was able to even go for ~1500-1600 epochs with the change of some parameters. In kernel now I check the time from the start of the training, when it passes 32000s I stop training. Reading the data and generating images takes ~200 s more, this way I squeze as much as I can from 9h (32400s) kernels limit.
", "votes": 5, "replies": [] }, { "id": 571929, "author_name": "Nirjhar Roy", "author_url": "", "post_date": "2019-07-10T08:43:47.130000", "content": "Yes , thats the trick i am coding now . Time decides what is the number of epoch :) . Thats really cool suggestion thanks .
", "votes": 1, "replies": [] }, { "id": 571992, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-10T10:09:07.757000", "content": "Wow, cool. I need to optimize my TensorFlow code. Each of my epochs (all 20000 training images) is taking 70 seconds each.
", "votes": 1, "replies": [] }, { "id": 571997, "author_name": "Nirjhar Roy", "author_url": "", "post_date": "2019-07-10T10:15:02.337000", "content": "@cdeotte please check the below Kernel this has a per epoch speed of 35 seconds . I just implemented this for my torch code to be able to run more epoch. Not sure if any idea can be taken up for your work \nhttps://www.kaggle.com/speedwagon/ram-dataloader
", "votes": 0, "replies": [] } ] }, { "id": 571729, "author_name": "jionie", "author_url": "", "post_date": "2019-07-10T03:27:42.597000", "content": "95.18 with Ralsgan.
", "votes": 1, "replies": [ { "id": 580957, "author_name": "jionie", "author_url": "", "post_date": "2019-07-21T06:26:44.197000", "content": "So far 57 after 100 epoch, 46 after 300 epoch, 41 after 550 epoch. Used DCGAN.
", "votes": 2, "replies": [] } ] }, { "id": 574731, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-14T12:01:14.373000", "content": "Here's a basic TensorFlow DCGAN using RaLS loss. (It also uses bounding box info with inscribed cropped squares described below. And it randomly flips images horizontally with 50% probability). It achieves score LB 72. This is a baseline. To beat this, you need to be creative and add things. (It achieves LB 85/80/75/73/72 after 100/125/150/175/200 epochs).
\n\n
UPDATE: Here's a basic PyTorch DCGAN using RaLS loss (and inscribed bounding box crop and random horizontal flips). It looks like it performs the same as TensorFlow. (It achieves LB 90/80/77/77/74 after 100/125/150/175/200 epochs).\n
Great! \nBefore LB 40 (I think someone got that with traditional GAN) I should try breaking the baseline you've provided. \nI made a similar GAN with RaLS loss and trained it for 1k epochs, my LB score was ~91\nI made it on pytorch, something must be wrong with it.\nI haven't used the box annotations, maybe that.
", "votes": 0, "replies": [] }, { "id": 574743, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-14T12:16:21.483000", "content": "This GAN uses the box annotations with inscribed square and random horizontal flipping. (I'll add that comment above). For some reason, I've found that TensorFlow does better than PyTorch with seemingly the same code
\n\nEDIT: I'm not a PyTorch expert but does the starter code adjust the Discriminator weights before we input fake images for the purpose of adjusting the Generator weights? (PyTorch inputs 1 batch of real and 2 batch of fake during training). The TensorFlow code adjusts the weights at the same time. (TensorFlow inputs 1 batch real and 1 batch fake).
\n\nThe PyTorch tutorial is here, the TensorFlow tutorial is here
", "votes": 1, "replies": [] }, { "id": 574746, "author_name": "timetraveller", "author_url": "", "post_date": "2019-07-14T12:21:26.150000", "content": ">I've found that TensorFlow does better than PyTorch with seemingly the same code
\n\nI see, I must get used to using TensorFlow. Now is the time.\nThank you!
", "votes": 0, "replies": [] }, { "id": 574750, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-14T12:24:32.330000", "content": "See my editted comment above. It may be because PyTorch updates G and D differently than TensorFlow. If this is the case and cause, maybe it can be changed.
", "votes": 1, "replies": [] }, { "id": 574753, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-14T12:29:10.180000", "content": "An advantage of PyTorch is that it is easy to make fast (and better functions for data augmentation). Vlad's public RaLSGAN PyTorch kernel here can do 1000 iterations in 9 hours! He uses his RAM dataloader. I haven't been able to get TensorFlow that fast in eager_execution (easy programming) mode. If I can't speed up EE, then I'll have to reprogram my TensorFlow GAN in (difficult) Sessions and Graph mode.
", "votes": 1, "replies": [] }, { "id": 574754, "author_name": "timetraveller", "author_url": "", "post_date": "2019-07-14T12:29:50.650000", "content": "Yes, I found that odd in the started code. I have already changed it to the way weights are updated in your kernels (at the same time). \nI just needed a reason to learn tensorflow and I found it xD
", "votes": 1, "replies": [] }, { "id": 574990, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-14T20:04:05.450000", "content": "I re-ran PyTorch starter code with RaLS loss, inscribed bounding box square, random horizontal flips. It performs the same as TensorFlow. I added the plot above.
", "votes": 2, "replies": [] } ] }, { "id": 571722, "author_name": "Nirjhar Roy", "author_url": "", "post_date": "2019-07-10T03:00:50.680000", "content": "Mine so far is 99.78 after 200 epochs . But I saw some people have gone till 50s as well .
", "votes": 2, "replies": [] }, { "id": 574188, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-13T10:55:07.600000", "content": "Leon, are you using the bounding boxes to extract the dogs? With the full images, my best score is 75 with just dogs my best is 65.
", "votes": 0, "replies": [ { "id": 574212, "author_name": "Zhongkai Shangguan", "author_url": "", "post_date": "2019-07-13T12:11:10.037000", "content": "In my case, crop the data made worse score. Perhaps because of my crop function, I crop with the max(xdiff, ydiff) in order to make the image in square size.
", "votes": 0, "replies": [] }, { "id": 574219, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-13T13:04:49.043000", "content": "Do you mean if the bounding box is 50 wide by 100 tall, then you used a square that is 100 by 100 circumscribed (then resize 64x64)? I score 65 by using a square that is 50 by 50 inscribed (then resize to 64x64). I use min(xdiff,ydiff) and locate it at (0,0) relative to bounding box.
If you circumscribe then some of your crops contains black pixels when there was no image for the circumscribed to use.
", "votes": 0, "replies": [] }, { "id": 574230, "author_name": "Zhongkai Shangguan", "author_url": "", "post_date": "2019-07-13T13:31:29.413000", "content": "Yeah, that's exactly what I did. I will try your method. I crop in that way because I don't want to loss any information though it produces noise, but it seems not as well as I expected.
", "votes": 1, "replies": [] }, { "id": 574258, "author_name": "", "author_url": "", "post_date": "2019-07-13T14:28:40.670000", "content": "", "votes": 0, "replies": [] }, { "id": 574263, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-13T14:34:13.843000", "content": "Note that when I use min, I don't center the square. For a tall rectangle (h>w), I move square up. For a wide rectangle (w>h), I move square left.
Thanks, by crop in min(xdiff, ydiff), my GAN reaches LB 68 now.
", "votes": 1, "replies": [] } ] }, { "id": 572921, "author_name": "timetraveller", "author_url": "", "post_date": "2019-07-11T15:16:09.877000", "content": "Since memorizer GANs like chris' kernel are allowed (source). Should we focus less on \"pure\" GANs? Also, my best score till now is 105 with a simple DCGAN. I am currently running my own version of \"RaSLGAN\" here, I am hoping to get a better score.
", "votes": 0, "replies": [ { "id": 572929, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-11T15:19:59.307000", "content": "Has anyone tried starting with a Memorizer GAN (where Generator has DCGAN architecture). Then training a (DCGAN) Discriminator to match the Generator's performance. And then continuing with traditional iterations of GAN?
", "votes": 1, "replies": [] }, { "id": 579894, "author_name": "Guillaume Desforges", "author_url": "", "post_date": "2019-07-19T11:26:38.923000", "content": "@timetraveller98 imo there is a non zero probability that Memorizer GANs will not be eligible for gold as it is not very hard to check 10~20 kernels. You could try to temper it not to have such a low score, but then you would be close to the best non-memorizer gans, so what would be the point ? 😃
\n\nAnyway, if you find it fun to hyperoptimize a memorizer gan go ahead, though I doubt that you will take much pleasure from it after a few hours. 😄
", "votes": 0, "replies": [] } ] }, { "id": 571787, "author_name": "r41", "author_url": "", "post_date": "2019-07-10T05:08:52.917000", "content": "85.72 with 200 epochs
", "votes": 0, "replies": [ { "id": 571796, "author_name": "Nirjhar Roy", "author_url": "", "post_date": "2019-07-10T05:19:36.280000", "content": "Thats awesome !
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "571707": "As the leaderboard is ruined by Memorization Gan, which may not be allowed. However, I believe many people who use traditional Gan want to know there position and how far can traditional Gan goes, so I create this topic, anyone who use traditional Gan could comments about your Gan's score, as a reference to other people.\nAs for me, my best score of traditional Gan is 96.18. \nWish and wait for more comments. Thanks.\n\nUpdate: Using Ralsgan helps me break to 74.36. And it's seems like Memorization Gan will be acceptable. Though it may cause over fit, I still think it would perform better than traditional Gan since no matter how big the difference public data and mysterious data have, dogs are always dogs, it will not change just because of the changing of data.\n\nUpdate: 42 by now.", "572897": "I am struggle at 40+. My best score for now is 39.13987, but I think it was mostly luck rather than some clever changes.\nDunno how to get below 40.", "579776": "LB 40+ with StyleGAN.\nI feel this competition is fun, though there is a problem on rules.", "574134": "57-ish with 300 epochs, which took me about 7 hours.\nBut GANs are instable and the metric varies a lot between two sets of images from the same generator so it is quite hard to tweak models.", "571994": "My 2 best scores are Memorizer GAN = 5.5 :-) and Traditional GAN = 65 after 150 epochs (6.5 hours):\n\n", "571816": "I'm at 71. I think most people 40+ are using genuine gans.\n\nI bet a whole bunch of people with 7-40 scores also have a good scoring traditional GAN. It is disappointing to see kaggle regulars, who I respect greatly, submitting LB killing scores. What good does it do to be the 50th person with a score of 7.2?", "572481": "I'm at 47.7 with 1000epochs", "578965": "Official starter kernel trained on both original images and bbox cropped images with batch size=96 to 800 epochs got LB 62