{"cells":[{"metadata":{},"cell_type":"markdown","source":"# How to Paint with MORE Generative Adversarial Networks\n\n*This is part 2, see part 1 [here](https://www.kaggle.com/jesperdramsch/how-to-paint-with-gans-least-squares-gan-starter)*\n\nYou can create Monets with Generative Adversarial Networks (GAN) in a few different ways. We can generate them from scratch using one GAN, where the GAN basically imagines a Monet from scratch. GANs are technically two networks that work against each other, illustrated below. The artist (generator) draws its inspiration from a noise sample and creates a rendering of the data you are trying to generate with said GAN. The private investigator (discriminator) randomly gets assigned real and fake data to investigate. \n\n![](https://media1.tenor.com/images/4c6c1a33f7e10573c7c9a5d574de417a/tenor.gif?itemid=8224930)\n\nBecause us machine learning scientists had a bit too much GPUs running idle, people came up with the idea of cycle-consistent GANs. CycleGANs were introduced for [unpaired Image-to-Image Translation](https://junyanz.github.io/CycleGAN/), for when you don't have Monet available to paint your favorite subject. They're pretty useful generally and have been applied in many domains and style transfer applications. The problem?! This is now two GANs to train that perforn the forward and backward transformation to create the new style from nothing. Finally we can compare apples to oranges.\n\n![](https://junyanz.github.io/CycleGAN/images/objects.jpg)\n\nIn this tutorial we'll look in-depth into:\n\n- Data Augmentation\n- Underlying Neural Network Architectures\n- CycleGAN architectures\n- Better Loss functions\n\nBonus? Their failure modes are hilarious:\n\n![](https://camo.githubusercontent.com/757b691307b52fe8a0806dde3a560dc068dbf5b3/68747470733a2f2f6a756e79616e7a2e6769746875622e696f2f4379636c6547414e2f696d616765732f6661696c7572655f707574696e2e6a7067)\n\nThe main idea behind a CycleGAN is that two Generative Adversarial Networks are trained. Network one learning the forward transformation to the target domain and the second network learning the inverse transformation back to the original image domain. Pairing this with the GAN loss of creating \"believable\" images, at least according to the discriminator of the GAN, yields some surprisingly good transformations.\n\n*This copies in part from my [Intro to Deepfakes](https://www.kaggle.com/jesperdramsch/intro-to-deep-fakes-videos-and-metadata-eda) if you're interested to learn how GANs are used to alter images, videos, and sounds. (Did I mention they're quite versatile?!) This also builds on the Baseline Tutorial. [Please head over and upvote!](https://www.kaggle.com/amyjang/monet-cyclegan-tutorial)*"},{"metadata":{},"cell_type":"markdown","source":"## Introduction and Setup\n\nFor this tutorial, we will be using the TFRecord dataset. Import the following packages and change the accelerator to TPU. Because TPUs are pretty awesome.\n\n![](https://i.imgur.com/hRLjugH.png)"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import tensorflow as tf\nfrom tensorflow import keras\nfrom tensorflow.keras import layers\nimport tensorflow_addons as tfa\nimport tensorflow_datasets as tfds\n\nfrom kaggle_datasets import KaggleDatasets\nimport matplotlib.pyplot as plt\nimport numpy as np\n\nfrom functools import partial\nfrom albumentations import (\n    Compose, RandomBrightness, JpegCompression, HueSaturationValue, RandomContrast, HorizontalFlip,\n    Rotate\n)\n\ntry:\n    tpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n    print('Device:', tpu.master())\n    tf.config.experimental_connect_to_cluster(tpu)\n    tf.tpu.experimental.initialize_tpu_system(tpu)\n    strategy = tf.distribute.experimental.TPUStrategy(tpu)\nexcept:\n    strategy = tf.distribute.get_strategy()\nprint('Number of replicas:', strategy.num_replicas_in_sync)\n\nAUTOTUNE = tf.data.experimental.AUTOTUNE\n\nprint(tf.__version__)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Load in the data\n\nWe want to keep our photo dataset and our Monet dataset separate. First, load in the filenames of the TFRecords. We'll load both for the CycleGAN. For the first GAN we only need the Monets as training data.\n\nAll the images for the competition are already sized to `256 x 256`. As these images are RGB images, set the channel to 3. Additionally, we need to scale the images to a `[-1, 1]` scale. Because we are building a generative model, we don't need the labels or the image id so we'll only return the image from the TFRecord."},{"metadata":{"trusted":true},"cell_type":"code","source":"GCS_PATH = KaggleDatasets().get_gcs_path()\n\nMONET_FILENAMES = tf.io.gfile.glob(str(GCS_PATH + '/monet_tfrec/*.tfrec'))\nprint('Monet TFRecord Files:', len(MONET_FILENAMES))\n\nPHOTO_FILENAMES = tf.io.gfile.glob(str(GCS_PATH + '/photo_tfrec/*.tfrec'))\nprint('Photo TFRecord Files:', len(PHOTO_FILENAMES))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"You can see I put down a bit of augmentation using `random_jitter` and `flip` to increase our data set, because we simply don't have enough data for"},{"metadata":{"trusted":true},"cell_type":"code","source":"IMAGE_SIZE = [256, 256]\n\ndef normalize(image):\n    return (tf.cast(image, tf.float32) / 127.5) - 1\n\ndef decode_image(image):\n    #image = tf.image.decode_jpeg(image, channels=3)\n    #image = tf.reshape(image, [256, 256, 3])\n    image = tf.image.decode_jpeg(image, channels=3)\n    #image = (tf.cast(image, tf.float32) / 127.5) - 1\n    image = tf.reshape(image, [*IMAGE_SIZE, 3])\n    return image\n\ndef random_crop(image):\n    cropped_image = tf.image.random_crop(image, size=[256, 256, 3])\n    return cropped_image\n\ndef random_jitter(image):\n    # resizing to 286 x 286 x 3 \n    image = tf.image.resize(image, [int(256*1.3), int(256*1.3)],\n                          method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)\n    # randomly cropping to 256 x 256 x 3\n    image = random_crop(image)\n    # random mirroring\n    return image\n\ndef flip(image):\n    return tf.image.flip_left_right(image)\n\ndef preprocess_image_train(image, label=None):\n    image = random_jitter(image)\n    return image\n\ndef read_tfrecord(example):\n    tfrecord_format = {\n        \"image_name\": tf.io.FixedLenFeature([], tf.string),\n        \"image\": tf.io.FixedLenFeature([], tf.string),\n        \"target\": tf.io.FixedLenFeature([], tf.string)\n    }\n    example = tf.io.parse_single_example(example, tfrecord_format)\n    image = decode_image(example['image'])\n    return image\n\ndef load_dataset(filenames, labeled=False, ordered=False, repeats=200):\n    dataset = tf.data.TFRecordDataset(filenames)\n    dataset = dataset.map(read_tfrecord, num_parallel_calls=AUTOTUNE)\n    dataset = dataset.concatenate(dataset.map(flip, num_parallel_calls=AUTOTUNE).shuffle(100000))\n    dataset = dataset.concatenate(dataset.map(random_jitter, num_parallel_calls=AUTOTUNE).shuffle(10000, reshuffle_each_iteration=True).repeat(repeats))\n    dataset = dataset.map(normalize, num_parallel_calls=AUTOTUNE).shuffle(10000)\n    return dataset","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Then load the data and display the first images to see if it all worked out. Which of course it does, because it's taken directly from the tutorial."},{"metadata":{"trusted":true},"cell_type":"code","source":"monet_ds = load_dataset(MONET_FILENAMES, labeled=True, repeats=50).batch(100, drop_remainder=True)\nphoto_ds = load_dataset(PHOTO_FILENAMES, labeled=True, repeats=2  ).batch(100, drop_remainder=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def view_image(ds, rows=2):\n    image = next(iter(ds)) # extract 1 batch from the dataset\n    image = image.numpy()\n\n    fig = plt.figure(figsize=(22, rows * 5.05 ))\n    for i in range(5 * rows):\n        ax = fig.add_subplot(rows, 5, i+1, xticks=[], yticks=[])\n        ax.imshow(image[i] / 2 + .5)\n\nview_image(monet_ds)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"view_image(photo_ds)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Build the DCGAN\n## Network Upsample and Downsample\n\nThis one is the same from part1.\n\nThe `downsample`, as the name suggests, reduces the 2D dimensions, the width and height, of the image by the stride. The stride is the length of the step the filter takes. Since the stride is 2, the filter is applied to every other pixel, hence reducing the weight and height by 2.\n\nWe'll be using an instance normalization instead of batch normalization. As the instance normalization is not standard in the TensorFlow API, we'll use the layer from TensorFlow Add-ons."},{"metadata":{"trusted":true},"cell_type":"code","source":"OUTPUT_CHANNELS = 3\nLATENT_DIM = 1024\n\ndef downsample(filters, size, apply_instancenorm=True):\n    initializer = tf.random_normal_initializer(0., 0.02)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    result = keras.Sequential()\n    result.add(layers.Conv2D(filters, size, padding='same',\n                             kernel_initializer=initializer, use_bias=False))\n    result.add(layers.MaxPool2D())\n\n    if apply_instancenorm:\n        result.add(tfa.layers.InstanceNormalization(gamma_initializer=gamma_init))\n\n    result.add(layers.LeakyReLU())\n\n    return result","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`Upsample` does the opposite of downsample and increases the dimensions of the of the image. `Conv2DTranspose` does basically the opposite of a `Conv2D` layer."},{"metadata":{"trusted":true},"cell_type":"code","source":"def upsample(filters, size, apply_dropout=False):\n    initializer = tf.random_normal_initializer(0., 0.02)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    result = keras.Sequential()\n    result.add(layers.Conv2DTranspose(filters, size, strides=2,\n                                      padding='same',\n                                      kernel_initializer=initializer,\n                                      use_bias=False))\n\n    result.add(tfa.layers.InstanceNormalization(gamma_initializer=gamma_init))\n\n    if apply_dropout:\n        result.add(layers.Dropout(0.5))\n\n    result.add(layers.LeakyReLU())\n\n    return result","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Build Network\nThe generator first downsamples the input image and then upsample while establishing long skip connections. Skip connections are a way to help bypass the vanishing gradient problem by concatenating the output of a layer to multiple layers instead of only one. Here we concatenate the output of the downsample layer to the upsample layer in a symmetrical fashion. Unets are pretty versatile and help out in our Generator to distill the input image to a lower dimension and then back to the full size at the target.\n\n![](https://i.imgur.com/7GE9nY1.png)\n[Source](https://github.com/HarisIqbal88/PlotNeuralNet)"},{"metadata":{"trusted":true},"cell_type":"code","source":"EPOCHS = 25\n\nLR_G = 2e-4\nLR_D = 2e-4\nbeta_1 = .5\n\nreal_label = .9\nfake_label = 0","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def CycleGenerator():\n    inputs = layers.Input(shape=[256,256,3])\n\n    # bs = batch size\n    down_stack = [\n        downsample(64, 4, apply_instancenorm=False), # (bs, 128, 128, 64)\n        downsample(128, 4), # (bs, 64, 64, 128)\n        downsample(256, 4), # (bs, 32, 32, 256)\n        downsample(512, 4), # (bs, 16, 16, 512)\n        downsample(512, 4), # (bs, 8, 8, 512)\n        downsample(512, 4), # (bs, 4, 4, 512)\n        downsample(512, 4), # (bs, 2, 2, 512)\n        downsample(512, 4), # (bs, 1, 1, 512)\n    ]\n\n    up_stack = [\n        upsample(512, 4, apply_dropout=True), # (bs, 2, 2, 1024)\n        upsample(512, 4, apply_dropout=True), # (bs, 4, 4, 1024)\n        upsample(512, 4, apply_dropout=True), # (bs, 8, 8, 1024)\n        upsample(512, 4), # (bs, 16, 16, 1024)\n        upsample(256, 4), # (bs, 32, 32, 512)\n        upsample(128, 4), # (bs, 64, 64, 256)\n        upsample(64, 4), # (bs, 128, 128, 128)\n    ]\n\n    initializer = tf.random_normal_initializer(0., 0.02)\n    last = layers.Conv2DTranspose(OUTPUT_CHANNELS, 4,\n                                  strides=2,\n                                  padding='same',\n                                  kernel_initializer=initializer,\n                                  activation='tanh') # (bs, 256, 256, 3)\n\n    x = inputs\n\n    # Downsampling through the model\n    skips = []\n    for down in down_stack:\n        x = down(x)\n        skips.append(x)\n\n    skips = reversed(skips[:-1])\n\n    # Upsampling and establishing the skip connections\n    for up, skip in zip(up_stack, skips):\n        x = up(x)\n        x = layers.Concatenate()([x, skip])\n\n    x = last(x)\n\n    return keras.Model(inputs=inputs, outputs=x)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The discriminator does not need a Unet, just a nice simple downsample to get a simple `fake` or `real` represented in numbers."},{"metadata":{"trusted":true},"cell_type":"code","source":"def CycleDiscriminator():\n    initializer = tf.random_normal_initializer(0., 0.02)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    inp = layers.Input(shape=[256, 256, 3], name='input_image')\n\n    x = inp\n\n    down1 = downsample(64, 4, False)(x) # (bs, 128, 128, 64)\n    down2 = downsample(128, 4)(down1) # (bs, 64, 64, 128)\n    down3 = downsample(256, 4)(down2) # (bs, 32, 32, 256)\n\n    zero_pad1 = layers.ZeroPadding2D()(down3) # (bs, 34, 34, 256)\n    conv = layers.Conv2D(512, 4, strides=1,\n                         kernel_initializer=initializer,\n                         use_bias=False)(zero_pad1) # (bs, 31, 31, 512)\n\n    norm1 = tfa.layers.InstanceNormalization(gamma_initializer=gamma_init)(conv)\n\n    leaky_relu = layers.LeakyReLU()(norm1)\n\n    zero_pad2 = layers.ZeroPadding2D()(leaky_relu) # (bs, 33, 33, 512)\n\n    last_conv = layers.Conv2D(1, 4, strides=1,\n                         kernel_initializer=initializer)(zero_pad2) # (bs, 30, 30, 1)\n\n    last_relu = layers.LeakyReLU(alpha=0.2)(last_conv)\n    last_pool = layers.Flatten()(last_relu)\n    last = layers.Dense(1, activation='sigmoid')(last_pool)\n\n    return tf.keras.Model(inputs=inp, outputs=last)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    monet_cycle_generator = CycleGenerator() # transforms photos to Monet-esque paintings\n    photo_cycle_generator = CycleGenerator() # transforms Monet paintings to be more like photos\n\n    monet_cycle_discriminator = CycleDiscriminator() # differentiates real Monet paintings and generated Monet paintings\n    photo_cycle_discriminator = CycleDiscriminator() # differentiates real photos and generated photos","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Build the CycleGAN model\n\nWe will subclass a `tf.keras.Model` so that we can run `fit()` later to train our model. During the training step, the model transforms a photo to a Monet painting and then back to a photo. The difference between the original photo and the twice-transformed photo is the cycle-consistency loss. We want the original photo and the twice-transformed photo to be similar to one another.\n\nThe way this works is by having one GAN for the forwards transformation and one GAN for the backwards transformation. So from image domain $X \\rightarrow Y$ and backwards $Y \\rightarrow X$. The resulting images are each evaluated by the standard discriminators of the GANs.\n![](https://i.imgur.com/05Cjt6e.png)\n\nThe losses are defined in the next section."},{"metadata":{"trusted":true},"cell_type":"code","source":"class CycleGan(keras.Model):\n    def __init__(\n        self,\n        monet_generator,\n        photo_generator,\n        monet_discriminator,\n        photo_discriminator,\n        lambda_cycle=10,\n        real_label=.5\n    ):\n        super(CycleGan, self).__init__()\n        self.m_gen = monet_generator\n        self.p_gen = photo_generator\n        self.m_disc = monet_discriminator\n        self.p_disc = photo_discriminator\n        self.lambda_cycle = lambda_cycle\n        self.real_label = real_label\n        \n    def compile(\n        self,\n        m_gen_optimizer,\n        p_gen_optimizer,\n        m_disc_optimizer,\n        p_disc_optimizer,\n        gen_loss_fn,\n        disc_loss_fn,\n        cycle_loss_fn,\n        identity_loss_fn\n    ):\n        super(CycleGan, self).compile()\n        self.m_gen_optimizer = m_gen_optimizer\n        self.p_gen_optimizer = p_gen_optimizer\n        self.m_disc_optimizer = m_disc_optimizer\n        self.p_disc_optimizer = p_disc_optimizer\n        self.gen_loss_fn = gen_loss_fn\n        self.disc_loss_fn = disc_loss_fn\n        self.cycle_loss_fn = cycle_loss_fn\n        self.identity_loss_fn = identity_loss_fn\n        \n    def train_step(self, batch_data):\n        real_monet, real_photo = batch_data\n        \n        batch_size = tf.shape(real_photo)[0]\n        labels_real = tf.zeros((batch_size, 1)) + self.real_label\n        labels_real += 0.05 * tf.random.uniform(tf.shape(labels_real))        \n        \n        with tf.GradientTape(persistent=True) as tape:\n            # photo to monet back to photo\n            fake_monet = self.m_gen(real_photo, training=True)\n            cycled_photo = self.p_gen(fake_monet, training=True)\n\n            # monet to photo back to monet\n            fake_photo = self.p_gen(real_monet, training=True)\n            cycled_monet = self.m_gen(fake_photo, training=True)\n\n            # generating itself\n            same_monet = self.m_gen(real_monet, training=True)\n            same_photo = self.p_gen(real_photo, training=True)\n\n            # discriminator used to check, inputing real images\n            disc_real_monet = self.m_disc(real_monet, training=True)\n            disc_real_photo = self.p_disc(real_photo, training=True)\n\n            # discriminator used to check, inputing fake images\n            disc_fake_monet = self.m_disc(fake_monet, training=True)\n            disc_fake_photo = self.p_disc(fake_photo, training=True)\n\n            # evaluates generator loss\n            monet_gen_loss = self.gen_loss_fn(disc_real_monet, disc_fake_monet, labels_real)\n            photo_gen_loss = self.gen_loss_fn(disc_real_photo, disc_fake_photo, labels_real)\n\n            # evaluates total cycle consistency loss\n            total_cycle_loss = self.cycle_loss_fn(real_monet, cycled_monet, self.lambda_cycle) + self.cycle_loss_fn(real_photo, cycled_photo, self.lambda_cycle)\n\n            # evaluates total generator loss\n            total_monet_gen_loss = monet_gen_loss + total_cycle_loss + self.identity_loss_fn(real_monet, same_monet, self.lambda_cycle)\n            total_photo_gen_loss = photo_gen_loss + total_cycle_loss + self.identity_loss_fn(real_photo, same_photo, self.lambda_cycle)\n\n            # evaluates discriminator loss\n            monet_disc_loss = self.disc_loss_fn(disc_real_monet, disc_fake_monet, labels_real)\n            photo_disc_loss = self.disc_loss_fn(disc_real_photo, disc_fake_photo, labels_real)\n\n        # Calculate the gradients for generator and discriminator\n        monet_generator_gradients = tape.gradient(total_monet_gen_loss,\n                                                  self.m_gen.trainable_variables)\n        photo_generator_gradients = tape.gradient(total_photo_gen_loss,\n                                                  self.p_gen.trainable_variables)\n\n        monet_discriminator_gradients = tape.gradient(monet_disc_loss,\n                                                      self.m_disc.trainable_variables)\n        photo_discriminator_gradients = tape.gradient(photo_disc_loss,\n                                                      self.p_disc.trainable_variables)\n\n        # Apply the gradients to the optimizer\n        self.m_gen_optimizer.apply_gradients(zip(monet_generator_gradients,\n                                                 self.m_gen.trainable_variables))\n\n        self.p_gen_optimizer.apply_gradients(zip(photo_generator_gradients,\n                                                 self.p_gen.trainable_variables))\n\n        self.m_disc_optimizer.apply_gradients(zip(monet_discriminator_gradients,\n                                                  self.m_disc.trainable_variables))\n\n        self.p_disc_optimizer.apply_gradients(zip(photo_discriminator_gradients,\n                                                  self.p_disc.trainable_variables))\n        \n        return {\n            \"monet_gen_loss\": total_monet_gen_loss,\n            \"photo_gen_loss\": total_photo_gen_loss,\n            \"monet_disc_loss\": monet_disc_loss,\n            \"photo_disc_loss\": photo_disc_loss\n        }","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Define loss functions\n\nThe discriminator loss function below compares real images to a matrix of 1s and fake images to a matrix of 0s. The perfect discriminator will output all 1s for real images and all 0s for fake images. The discriminator loss outputs the average of the real and generated loss.\n\nThe generator wants to fool the discriminator into thinking the generated image is real. The perfect generator will have the discriminator output only 1s. Thus, it compares the generated image to a matrix of 1s to find the loss.\n\nWe could probably check if the Least Squares would also perform better for the Cycle GAN, so instead of just stupidly copying the starter, let's see if we can get this to improve the solution!"},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    def discriminator_loss(predictions_real, predictions_gen, labels_real):\n        return (tf.reduce_mean((predictions_gen  - tf.reduce_mean(predictions_real) + labels_real) ** 2) +\n                tf.reduce_mean((predictions_real - tf.reduce_mean(predictions_gen)  - labels_real) ** 2))/2\n    \n    def generator_loss(predictions_real, predictions_gen, labels_real):\n        return (tf.reduce_mean((predictions_real - tf.reduce_mean(predictions_gen)  + labels_real) ** 2) +\n                tf.reduce_mean((predictions_gen  - tf.reduce_mean(predictions_real) - labels_real) ** 2)) / 2","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## More Loss Functions\nWe want our original photo and the twice transformed photo to be similar to one another. Thus, we can calculate the cycle consistency loss be finding the average of their difference."},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    def calc_cycle_loss(real_image, cycled_image, LAMBDA):\n        loss1 = tf.reduce_mean(tf.abs(real_image - cycled_image))\n\n        return LAMBDA * loss1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The identity loss compares the image with its generator (i.e. photo with photo generator). If given a photo as input, we want it to generate the same image as the image was originally a photo. The identity loss compares the input with the output of the generator."},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    def identity_loss(real_image, same_image, LAMBDA):\n        loss = tf.reduce_mean(tf.abs(real_image - same_image))\n        return LAMBDA * 0.5 * loss","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Train the CycleGAN\n\nLet's compile our model. Since we used `tf.keras.Model` to build our CycleGAN, we can just ude the `fit` function to train our model. You know the drill already."},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    monet_generator_optimizer = tf.keras.optimizers.Adam(LR_G, beta_1=0.5)\n    photo_generator_optimizer = tf.keras.optimizers.Adam(LR_G, beta_1=0.5)\n\n    monet_discriminator_optimizer = tf.keras.optimizers.Adam(LR_D, beta_1=0.5)\n    photo_discriminator_optimizer = tf.keras.optimizers.Adam(LR_D, beta_1=0.5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Double the optimizers double the fun!"},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    cycle_gan_model = CycleGan(\n        monet_cycle_generator, photo_cycle_generator, monet_cycle_discriminator, photo_cycle_discriminator, real_label=0.66\n    )\n\n    cycle_gan_model.compile(\n        m_gen_optimizer = monet_generator_optimizer,\n        p_gen_optimizer = photo_generator_optimizer,\n        m_disc_optimizer = monet_discriminator_optimizer,\n        p_disc_optimizer = photo_discriminator_optimizer,\n        gen_loss_fn = generator_loss,\n        disc_loss_fn = discriminator_loss,\n        cycle_loss_fn = calc_cycle_loss,\n        identity_loss_fn = identity_loss\n    )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"And finally we get to train!"},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"cycle_gan_model.fit(\n    tf.data.Dataset.zip((monet_ds, photo_ds)),\n    epochs=EPOCHS\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Visualize our Monet-esque photos\nAnd now the CycleGAN with the augmented training data and LS Gan. Probably not that much different from before, but it's worth a try, right?"},{"metadata":{"trusted":true},"cell_type":"code","source":"_, ax = plt.subplots(2, 5, figsize=(25, 5))\nfor i, img in enumerate(photo_ds.take(5)):\n    prediction = monet_cycle_generator(img, training=False)[0].numpy()\n    prediction = (prediction * 127.5 + 127.5).astype(np.uint8)\n    img = (img[0] * 127.5 + 127.5).numpy().astype(np.uint8)\n\n    ax[0, i].imshow(img)\n    ax[1, i].imshow(prediction)\n    ax[0, i].set_title(\"Input Photo\")\n    ax[1, i].set_title(\"Monet-esque\")\n    ax[0, i].axis(\"off\")\n    ax[1, i].axis(\"off\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Create submission files\nWe'll create the second submission file, as this is more of a tutorial so people can have a look at the outputs. Definitely make sure to play around with it!"},{"metadata":{"trusted":true},"cell_type":"code","source":"import PIL\n! mkdir ../images","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for i, img in enumerate(photo_ds.take(9999)):\n    prediction = monet_cycle_generator(img, training=False)[0].numpy()\n    prediction = (prediction * 127.5 + 127.5).astype(np.uint8)\n    im = PIL.Image.fromarray(prediction)\n    im.save(\"../images/\" + str(i) + \".jpg\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import shutil\nshutil.make_archive(\"/kaggle/working/images\", 'zip', \"/kaggle/images\")","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}