Do GANs memorize or generalize? This question fascinates me. As a math teacher, I often wonder if my students have memorized the formulas or understand the formulas.
\n\nI have many students with high grades and low understanding. And many students with low grades and high understanding. In the same way that it is hard for a teacher to evaluate students, it will be hard for Kaggle to evaluate GANs
\n\nI posted a kernel that explores this concept here (and another here). Note that I purposely made a simple example. I could have made a full GAN with both a generator and discriminator and only showed the training images to the discriminator. Then I could have the discriminator teach the generator to memorize the images.
", "messages": [ { "id": 567449, "postDate": "2019-07-03T14:34:18.807Z", "content": "Do GANs memorize or generalize? This question fascinates me. As a math teacher, I often wonder if my students have memorized the formulas or understand the formulas.
\n\nI have many students with high grades and low understanding. And many students with low grades and high understanding. In the same way that it is hard for a teacher to evaluate students, it will be hard for Kaggle to evaluate GANs
\n\nI posted a kernel that explores this concept here (and another here). Note that I purposely made a simple example. I could have made a full GAN with both a generator and discriminator and only showed the training images to the discriminator. Then I could have the discriminator teach the generator to memorize the images.
", "rawMarkdown": "Do GANs memorize or generalize? This question fascinates me. As a math teacher, I often wonder if my students have memorized the formulas or understand the formulas.\n\nI have many students with high grades and low understanding. And many students with low grades and high understanding. In the same way that it is hard for a teacher to evaluate students, it will be hard for Kaggle to evaluate GANs\n\nI posted a kernel that explores this concept [here][1] (and another [here][2]). Note that I purposely made a simple example. I could have made a full GAN with both a generator and discriminator and only showed the training images to the discriminator. Then I could have the discriminator teach the generator to memorize the images.\n\n[1]: https://www.kaggle.com/cdeotte/supervised-generative-dog-net\n[2]: https://www.kaggle.com/cdeotte/dog-memorizer-gan", "votes": 55 }, { "id": 567458, "postDate": "2019-07-03T14:49:23.143Z", "content": "Wonderful kernel and explanation, as always!\nIt will start a great discussion of what is allowed in this competition and how the metric can be changed to pick that up.
\n\nI hope the hosts will also have a word on this ;)
", "rawMarkdown": "Wonderful kernel and explanation, as always!\nIt will start a great discussion of what is allowed in this competition and how the metric can be changed to pick that up. \n\nI hope the hosts will also have a word on this ;)", "votes": 4, "replies": [ { "id": 567697, "postDate": "2019-07-03T21:12:24.477Z", "content": "Thanks Ilu. I think one solution would be for the Kaggle judges to verify the winners' Generative Network by choosing and submitting 10000 random seeds of their (the judges') choice. And then computing the MiFID metric of those images to make sure it is similar to Private LB score.
\n\n(And judges can verify that the range of the image space is greater than 10000 by observing a few random latent space walks).
", "rawMarkdown": "Thanks Ilu. I think one solution would be for the Kaggle judges to verify the winners' Generative Network by choosing and submitting 10000 random seeds of their (the judges') choice. And then computing the MiFID metric of those images to make sure it is similar to Private LB score.\n\n(And judges can verify that the range of the image space is greater than 10000 by observing a few random latent space walks).", "votes": 1 } ] }, { "id": 570494, "postDate": "2019-07-08T11:27:09.680Z", "content": "It is probably difficult for you as a teacher to develop some formal approach to distinguish low understanding and good understanding speaking about your students. In this competition I think it is difficult for Kaggle to create some formal approach for distinguishing between two types of GAN architectures (generalizing and memorizing).
\n\nWithout this approach memorizing GAN just wins this competition.
", "rawMarkdown": "It is probably difficult for you as a teacher to develop some formal approach to distinguish low understanding and good understanding speaking about your students. In this competition I think it is difficult for Kaggle to create some formal approach for distinguishing between two types of GAN architectures (generalizing and memorizing).\n\nWithout this approach memorizing GAN just wins this competition.", "votes": 1 }, { "id": 570487, "postDate": "2019-07-08T11:13:23.317Z", "content": "Nice read
", "rawMarkdown": "Nice read ", "votes": 1 }, { "id": 570232, "postDate": "2019-07-08T03:21:24.557Z", "content": "nice
", "rawMarkdown": "nice", "votes": 1 }, { "id": 570097, "postDate": "2019-07-07T19:12:53.927Z", "content": "Great kernel and explanation,thank u mr deotte
", "rawMarkdown": "Great kernel and explanation,thank u mr deotte", "votes": 1 }, { "id": 567685, "postDate": "2019-07-03T20:45:09.860Z", "content": "Yes in many cases provided with enough degrees of freedom you can over-fit (sometimes unintentionally) neural network to work like lookup-table without any generalization. GAN architecture is not different since it allows generator to over-fit on train data.\nI'd rather bother about other question. Here we have formal task with formal metric. Why some method are considered as cheating based on our subjective assumptions about meaning of vague term \"generalization\"?
", "rawMarkdown": "Yes in many cases provided with enough degrees of freedom you can over-fit (sometimes unintentionally) neural network to work like lookup-table without any generalization. GAN architecture is not different since it allows generator to over-fit on train data.\nI'd rather bother about other question. Here we have formal task with formal metric. Why some method are considered as cheating based on our subjective assumptions about meaning of vague term \"generalization\"?", "votes": 1, "replies": [ { "id": 567693, "postDate": "2019-07-03T21:01:29.790Z", "content": "The problem is that the MiFID metric is easily fooled. You can submit crops of original images and it won't penalize them as duplicates. Kaggle realizes this, so they added
\n\n> To enforce and prevent cheating, we reserve the right to: (a) Visually inspect all participants' submitted images, (b) review any submitted source code, (c) use these reviews to identify violators or determine winners, and (d) disqualify participants from the competition who are found in violation.
\n\nThis says that humans (Kaggle judges) will review the images to make sure they are \"original\". At what point does a GAN start making \"original\" images? Is 99% one image with 1% new stuff considered \"original\"?
", "rawMarkdown": "The problem is that the MiFID metric is easily fooled. You can submit crops of original images and it won't penalize them as duplicates. Kaggle realizes this, so they added \n\n> To enforce and prevent cheating, we reserve the right to: (a) Visually inspect all participants' submitted images, (b) review any submitted source code, (c) use these reviews to identify violators or determine winners, and (d) disqualify participants from the competition who are found in violation. \n\nThis says that humans (Kaggle judges) will review the images to make sure they are \"original\". At what point does a GAN start making \"original\" images? Is 99% one image with 1% new stuff considered \"original\"?" }, { "id": 567694, "postDate": "2019-07-03T21:03:52.147Z", "content": "If memorization is not cheating, then this will be a way to win this competition. \nAnd to my understanding, memorizing NNs are of no use at all to anyone. So it would be a lot of wasted work for everyone in this competition.
\n\nAlso, I find it hard to believe that the authors will check all medal ranks. That would be a lot of work.
", "rawMarkdown": "If memorization is not cheating, then this will be a way to win this competition. \nAnd to my understanding, memorizing NNs are of no use at all to anyone. So it would be a lot of wasted work for everyone in this competition. \n\nAlso, I find it hard to believe that the authors will check all medal ranks. That would be a lot of work. ", "votes": 5 }, { "id": 567701, "postDate": "2019-07-03T21:17:56.420Z", "content": "You can't just forbid memorization because it is in the heart of ML. I understand that organizers want models learn patterns, not images, but you can't control and easily interpret machine learning models. \nMiFID metric can be fooled, but organizers could:\n1) Request that the output of kernel is serialized model of Generator network not images.\n2) Restrict dimension of input and number of parameters of Generator network\n3) Ensure that private image set doesn't include any training images
", "rawMarkdown": "You can't just forbid memorization because it is in the heart of ML. I understand that organizers want models learn patterns, not images, but you can't control and easily interpret machine learning models. \nMiFID metric can be fooled, but organizers could:\n1) Request that the output of kernel is serialized model of Generator network not images.\n2) Restrict dimension of input and number of parameters of Generator network\n3) Ensure that private image set doesn't include any training images" }, { "id": 568665, "postDate": "2019-07-05T09:21:39.637Z", "content": "I agree with Ilu
\n\n\n\n\nIf memorization is not cheating, then this will be a way to win this competition.
\n
It will be very hard to beat a Memorizer GAN. I posted a competitive one here to demonstrate their unfair advantage.
", "rawMarkdown": "I agree with Ilu\n> If memorization is not cheating, then this will be a way to win this competition. \n\nIt will be very hard to beat a Memorizer GAN. I posted a competitive one [here][1] to demonstrate their unfair advantage.\n\n[1]: https://www.kaggle.com/cdeotte/dog-memorizer-gan", "votes": 3 }, { "id": 568891, "postDate": "2019-07-05T16:09:35.390Z", "content": "More than just \"hard\". FID scores from the very best GANs in current research don't reach values as low as your memorizing Kernel, @cdeotte \nIf i remember correctly they are around 8 for dogs right now with many many days of training.
", "rawMarkdown": "More than just \"hard\". FID scores from the very best GANs in current research don't reach values as low as your memorizing Kernel, @cdeotte \nIf i remember correctly they are around 8 for dogs right now with many many days of training. ", "votes": 2 } ] }, { "id": 567622, "postDate": "2019-07-03T19:01:01.590Z", "content": "Thank you so much sir. I have never thought about GANs this way. I am flattered with your understanding about GANs. Do you have a blog where you write about these topic , I will throughly enjoy reading it.
", "rawMarkdown": "Thank you so much sir. I have never thought about GANs this way. I am flattered with your understanding about GANs. Do you have a blog where you write about these topic , I will throughly enjoy reading it.", "votes": 1, "replies": [ { "id": 567698, "postDate": "2019-07-03T21:14:24.310Z", "content": "Thanks Rahul. Currently my only blog is Kaggle :-)
", "rawMarkdown": "Thanks Rahul. Currently my only blog is Kaggle :-)", "votes": 2 } ] }, { "id": 567547, "postDate": "2019-07-03T17:04:19.380Z", "content": "Good to see you Chris, again !\nI thought that all generative models, if tuned well, do generalize. No? I have never worked with GAN, so looking forward to the reasoning from others.
", "rawMarkdown": "Good to see you Chris, again !\nI thought that all generative models, if tuned well, do generalize. No? I have never worked with GAN, so looking forward to the reasoning from others.", "votes": 1 }, { "id": 569753, "postDate": "2019-07-07T08:59:41.373Z", "content": "@cdeotte great discussion. A couple of months ago I read this article in relation to an evaluation metric which seems to be more robust to memorization. \nTOWARDS GAN BENCHMARKS WHICH REQUIRE GENERALIZATION
", "rawMarkdown": "@cdeotte great discussion. A couple of months ago I read this article in relation to an evaluation metric which seems to be more robust to memorization. \n[TOWARDS GAN BENCHMARKS WHICH REQUIRE GENERALIZATION](https://openreview.net/pdf?id=HkxKH2AcFm)", "votes": 2, "replies": [ { "id": 570114, "postDate": "2019-07-07T19:33:54.093Z", "content": "Thanks for the link. I'll check it out.
", "rawMarkdown": "Thanks for the link. I'll check it out.", "votes": 1 } ] }, { "id": 568549, "postDate": "2019-07-05T05:07:15.863Z", "content": "I think on simple dataset like mnist, GAN is easily able to generalize(Like we able to understand easy formulas quickly.) and when things came to complex problems, GAN memorizes.
\n\nBasically It also depend on Generator and descriminator architecture.
\n\nif we define too many neurons obviously it will memorize each example and if we lower the no of neurons it will generalize them and a single neuron will try to store information about more then 1 example which will generalize the technique but it will fail to give good output.
\n\nTo make good GAN
\n\nArchitecture should be good
\n\nUse batch norm in each layer
\n\nnever forget to apply dropouts( If using too many neurons )
\n\nintroduce different type of loss like log loss, rmse, pre pixel loss, In some cases style loss also do great.
\n\nIf GAN is input based (input(img)->GAN->output(img)) use resnet as it is able to preserve maximum information.
\n\nDont Up-sample with Transposed Convolution( create checker board pattern), use simple image resize function
", "rawMarkdown": "I think on simple dataset like mnist, GAN is easily able to generalize(Like we able to understand easy formulas quickly.) and when things came to complex problems, GAN memorizes. \n\nBasically It also depend on Generator and descriminator architecture.\n\nif we define too many neurons obviously it will memorize each example and if we lower the no of neurons it will generalize them and a single neuron will try to store information about more then 1 example which will generalize the technique but it will fail to give good output.\n\nTo make good GAN \n\nArchitecture should be good\n\nUse batch norm in each layer\n\nnever forget to apply dropouts( If using too many neurons )\n\nintroduce different type of loss like log loss, rmse, pre pixel loss, In some cases style loss also do great.\n\nIf GAN is input based (input(img)->GAN->output(img)) use resnet as it is able to preserve maximum information.\n\nDont Up-sample with Transposed Convolution( create checker board pattern), use simple image resize function", "votes": 2 }, { "id": 567532, "postDate": "2019-07-03T16:42:25.587Z", "content": "Super interesting topic! I think this kind of discussions give a lot of value to the community
", "rawMarkdown": "Super interesting topic! I think this kind of discussions give a lot of value to the community ", "votes": 2 }, { "id": 586224, "postDate": "2019-07-28T20:25:45.830Z", "content": "Nice read, but still, memorization is pretty much unfair and not very interesting in the point of view of this competition. But I guess life is unfair :)
", "rawMarkdown": "Nice read, but still, memorization is pretty much unfair and not very interesting in the point of view of this competition. But I guess life is unfair :)" }, { "id": 567810, "postDate": "2019-07-04T03:20:02.203Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 567458, "author_name": "Pascal Pfeiffer", "author_url": "", "post_date": "2019-07-03T14:49:23.143000", "content": "Wonderful kernel and explanation, as always!\nIt will start a great discussion of what is allowed in this competition and how the metric can be changed to pick that up.
\n\nI hope the hosts will also have a word on this ;)
", "votes": 4, "replies": [ { "id": 567697, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-03T21:12:24.477000", "content": "Thanks Ilu. I think one solution would be for the Kaggle judges to verify the winners' Generative Network by choosing and submitting 10000 random seeds of their (the judges') choice. And then computing the MiFID metric of those images to make sure it is similar to Private LB score.
\n\n(And judges can verify that the range of the image space is greater than 10000 by observing a few random latent space walks).
", "votes": 1, "replies": [] } ] }, { "id": 570494, "author_name": "Anna Novikova", "author_url": "", "post_date": "2019-07-08T11:27:09.680000", "content": "It is probably difficult for you as a teacher to develop some formal approach to distinguish low understanding and good understanding speaking about your students. In this competition I think it is difficult for Kaggle to create some formal approach for distinguishing between two types of GAN architectures (generalizing and memorizing).
\n\nWithout this approach memorizing GAN just wins this competition.
", "votes": 1, "replies": [] }, { "id": 570487, "author_name": "shilpa.rpns", "author_url": "", "post_date": "2019-07-08T11:13:23.317000", "content": "Nice read
", "votes": 1, "replies": [] }, { "id": 570232, "author_name": "muhammad umar", "author_url": "", "post_date": "2019-07-08T03:21:24.557000", "content": "nice
", "votes": 1, "replies": [] }, { "id": 570097, "author_name": "Hamza Topal", "author_url": "", "post_date": "2019-07-07T19:12:53.927000", "content": "Great kernel and explanation,thank u mr deotte
", "votes": 1, "replies": [] }, { "id": 567685, "author_name": "Anton Chikin", "author_url": "", "post_date": "2019-07-03T20:45:09.860000", "content": "Yes in many cases provided with enough degrees of freedom you can over-fit (sometimes unintentionally) neural network to work like lookup-table without any generalization. GAN architecture is not different since it allows generator to over-fit on train data.\nI'd rather bother about other question. Here we have formal task with formal metric. Why some method are considered as cheating based on our subjective assumptions about meaning of vague term \"generalization\"?
", "votes": 1, "replies": [ { "id": 567693, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-03T21:01:29.790000", "content": "The problem is that the MiFID metric is easily fooled. You can submit crops of original images and it won't penalize them as duplicates. Kaggle realizes this, so they added
\n\n> To enforce and prevent cheating, we reserve the right to: (a) Visually inspect all participants' submitted images, (b) review any submitted source code, (c) use these reviews to identify violators or determine winners, and (d) disqualify participants from the competition who are found in violation.
\n\nThis says that humans (Kaggle judges) will review the images to make sure they are \"original\". At what point does a GAN start making \"original\" images? Is 99% one image with 1% new stuff considered \"original\"?
", "votes": 0, "replies": [] }, { "id": 567694, "author_name": "Pascal Pfeiffer", "author_url": "", "post_date": "2019-07-03T21:03:52.147000", "content": "If memorization is not cheating, then this will be a way to win this competition. \nAnd to my understanding, memorizing NNs are of no use at all to anyone. So it would be a lot of wasted work for everyone in this competition.
\n\nAlso, I find it hard to believe that the authors will check all medal ranks. That would be a lot of work.
", "votes": 5, "replies": [] }, { "id": 567701, "author_name": "Anton Chikin", "author_url": "", "post_date": "2019-07-03T21:17:56.420000", "content": "You can't just forbid memorization because it is in the heart of ML. I understand that organizers want models learn patterns, not images, but you can't control and easily interpret machine learning models. \nMiFID metric can be fooled, but organizers could:\n1) Request that the output of kernel is serialized model of Generator network not images.\n2) Restrict dimension of input and number of parameters of Generator network\n3) Ensure that private image set doesn't include any training images
", "votes": 0, "replies": [] }, { "id": 568665, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-05T09:21:39.637000", "content": "I agree with Ilu
\n\n\n\n\nIf memorization is not cheating, then this will be a way to win this competition.
\n
It will be very hard to beat a Memorizer GAN. I posted a competitive one here to demonstrate their unfair advantage.
", "votes": 3, "replies": [] }, { "id": 568891, "author_name": "Pascal Pfeiffer", "author_url": "", "post_date": "2019-07-05T16:09:35.390000", "content": "More than just \"hard\". FID scores from the very best GANs in current research don't reach values as low as your memorizing Kernel, @cdeotte \nIf i remember correctly they are around 8 for dogs right now with many many days of training.
", "votes": 2, "replies": [] } ] }, { "id": 567622, "author_name": "rahul", "author_url": "", "post_date": "2019-07-03T19:01:01.590000", "content": "Thank you so much sir. I have never thought about GANs this way. I am flattered with your understanding about GANs. Do you have a blog where you write about these topic , I will throughly enjoy reading it.
", "votes": 1, "replies": [ { "id": 567698, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-03T21:14:24.310000", "content": "Thanks Rahul. Currently my only blog is Kaggle :-)
", "votes": 2, "replies": [] } ] }, { "id": 567547, "author_name": "Piyush Paliwal", "author_url": "", "post_date": "2019-07-03T17:04:19.380000", "content": "Good to see you Chris, again !\nI thought that all generative models, if tuned well, do generalize. No? I have never worked with GAN, so looking forward to the reasoning from others.
", "votes": 1, "replies": [] }, { "id": 569753, "author_name": "JasonX", "author_url": "", "post_date": "2019-07-07T08:59:41.373000", "content": "@cdeotte great discussion. A couple of months ago I read this article in relation to an evaluation metric which seems to be more robust to memorization. \nTOWARDS GAN BENCHMARKS WHICH REQUIRE GENERALIZATION
", "votes": 2, "replies": [ { "id": 570114, "author_name": "Chris Deotte", "author_url": "", "post_date": "2019-07-07T19:33:54.093000", "content": "Thanks for the link. I'll check it out.
", "votes": 1, "replies": [] } ] }, { "id": 568549, "author_name": "Mohit Duklan", "author_url": "", "post_date": "2019-07-05T05:07:15.863000", "content": "I think on simple dataset like mnist, GAN is easily able to generalize(Like we able to understand easy formulas quickly.) and when things came to complex problems, GAN memorizes.
\n\nBasically It also depend on Generator and descriminator architecture.
\n\nif we define too many neurons obviously it will memorize each example and if we lower the no of neurons it will generalize them and a single neuron will try to store information about more then 1 example which will generalize the technique but it will fail to give good output.
\n\nTo make good GAN
\n\nArchitecture should be good
\n\nUse batch norm in each layer
\n\nnever forget to apply dropouts( If using too many neurons )
\n\nintroduce different type of loss like log loss, rmse, pre pixel loss, In some cases style loss also do great.
\n\nIf GAN is input based (input(img)->GAN->output(img)) use resnet as it is able to preserve maximum information.
\n\nDont Up-sample with Transposed Convolution( create checker board pattern), use simple image resize function
", "votes": 2, "replies": [] }, { "id": 567532, "author_name": "Federico Raimondi Cominesi", "author_url": "", "post_date": "2019-07-03T16:42:25.587000", "content": "Super interesting topic! I think this kind of discussions give a lot of value to the community
", "votes": 2, "replies": [] }, { "id": 586224, "author_name": "Matthieu Cordier", "author_url": "", "post_date": "2019-07-28T20:25:45.830000", "content": "Nice read, but still, memorization is pretty much unfair and not very interesting in the point of view of this competition. But I guess life is unfair :)
", "votes": 0, "replies": [] }, { "id": 567810, "author_name": "", "author_url": "", "post_date": "2019-07-04T03:20:02.203000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "567449": "Do GANs memorize or generalize? This question fascinates me. As a math teacher, I often wonder if my students have memorized the formulas or understand the formulas.\n\nI have many students with high grades and low understanding. And many students with low grades and high understanding. In the same way that it is hard for a teacher to evaluate students, it will be hard for Kaggle to evaluate GANs\n\nI posted a kernel that explores this concept [here][1] (and another [here][2]). Note that I purposely made a simple example. I could have made a full GAN with both a generator and discriminator and only showed the training images to the discriminator. Then I could have the discriminator teach the generator to memorize the images.\n\n[1]: https://www.kaggle.com/cdeotte/supervised-generative-dog-net\n[2]: https://www.kaggle.com/cdeotte/dog-memorizer-gan", "567458": "Wonderful kernel and explanation, as always!\nIt will start a great discussion of what is allowed in this competition and how the metric can be changed to pick that up. \n\nI hope the hosts will also have a word on this ;)", "570494": "It is probably difficult for you as a teacher to develop some formal approach to distinguish low understanding and good understanding speaking about your students. In this competition I think it is difficult for Kaggle to create some formal approach for distinguishing between two types of GAN architectures (generalizing and memorizing).\n\nWithout this approach memorizing GAN just wins this competition.", "570487": "Nice read ", "570232": "nice", "570097": "Great kernel and explanation,thank u mr deotte", "567685": "Yes in many cases provided with enough degrees of freedom you can over-fit (sometimes unintentionally) neural network to work like lookup-table without any generalization. GAN architecture is not different since it allows generator to over-fit on train data.\nI'd rather bother about other question. Here we have formal task with formal metric. Why some method are considered as cheating based on our subjective assumptions about meaning of vague term \"generalization\"?", "567622": "Thank you so much sir. I have never thought about GANs this way. I am flattered with your understanding about GANs. Do you have a blog where you write about these topic , I will throughly enjoy reading it.", "567547": "Good to see you Chris, again !\nI thought that all generative models, if tuned well, do generalize. No? I have never worked with GAN, so looking forward to the reasoning from others.", "569753": "@cdeotte great discussion. A couple of months ago I read this article in relation to an evaluation metric which seems to be more robust to memorization. \n[TOWARDS GAN BENCHMARKS WHICH REQUIRE GENERALIZATION](https://openreview.net/pdf?id=HkxKH2AcFm)", "568549": "I think on simple dataset like mnist, GAN is easily able to generalize(Like we able to understand easy formulas quickly.) and when things came to complex problems, GAN memorizes. \n\nBasically It also depend on Generator and descriminator architecture.\n\nif we define too many neurons obviously it will memorize each example and if we lower the no of neurons it will generalize them and a single neuron will try to store information about more then 1 example which will generalize the technique but it will fail to give good output.\n\nTo make good GAN \n\nArchitecture should be good\n\nUse batch norm in each layer\n\nnever forget to apply dropouts( If using too many neurons )\n\nintroduce different type of loss like log loss, rmse, pre pixel loss, In some cases style loss also do great.\n\nIf GAN is input based (input(img)->GAN->output(img)) use resnet as it is able to preserve maximum information.\n\nDont Up-sample with Transposed Convolution( create checker board pattern), use simple image resize function", "567532": "Super interesting topic! I think this kind of discussions give a lot of value to the community ", "586224": "Nice read, but still, memorization is pretty much unfair and not very interesting in the point of view of this competition. But I guess life is unfair :)", "567810": "" } }