{"cells":[{"metadata":{},"cell_type":"markdown","source":"## CycleGAN\n\nThis is based on Amy Jane's Notebook. I made mondifications on the codes. \nIn order to have a better understand how CycleGAN code works, I added inline comments with the code. \nHopefully, this will help others to understand also."},{"metadata":{},"cell_type":"markdown","source":"## Setting up for using TPU \n**Notes**:\n- tf.distribute.cluster_resolver: This library contains all implementations of ClusterResolvers. ClusterResolvers are a way of specifying cluster information for distributed execution. \n- TPUClusterResolver: This an implementation of cluster resolver for Google Cloud TPUs. \n   "},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"import tensorflow as tf\nfrom   tensorflow import keras\nfrom   tensorflow.keras import layers\nimport tensorflow_addons as tfa\nimport tensorflow_datasets as tfds\n\nfrom   kaggle_datasets import KaggleDatasets\nimport matplotlib.pyplot as plt\nimport numpy as np\n\n\ntry:\n    # Initialize a cluster resolver\n    tpu = tf.distribute.cluster_resolver.TPUClusterResolver()   \n    \n    # Show the connection string (device) when creating a session\n    print('Device:', tpu.master())   \n    \n    # Make devices on the cluster available to use\n    tf.config.experimental_connect_to_cluster(tpu)     \n    # Initialize the tpu device\n    tf.tpu.experimental.initialize_tpu_system(tpu)              \n    \n    # Synchronous training on TPUs and TPU Pods.\n    #    While using distribution strategies, the variables created \n    #    within the strategy's scope will be replicated across all \n    #    the replicas and can be kept in sync using all-reduce algorithms\n    strategy = tf.distribute.experimental.TPUStrategy(tpu)  \nexcept:\n    strategy = tf.distribute.get_strategy()\n\n# Show number of replicas\nprint('Number of replicas:', strategy.num_replicas_in_sync)     \n\n#show the version of Tensorflow\nprint(tf.__version__)                                          ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Load in the data\nThe following directories contains files in TFRecords format: \n> 1. **monet_tfrec**: 300 Monet paintings sized 256x256 \n> 2. **photo_tfrec**: 7028 photos sized 256x256 \n\nKeep the photo dataset and the Monet dataset separately:\n> - stores a list of files from the Photo directores.\n> - store a list of file from the Monet directories\n\nNote: There are 5 Monet TFRecord files, and 20 Photo TFRecord files. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# Get the Google Cloud Storage path URI (GCS path) for Kaggle Datasets\nGCS_PATH = KaggleDatasets().get_gcs_path()  \n\n# Obtain two lists of files that match the given patterns specified in str() \nMONET_FILENAMES = tf.io.gfile.glob(str(GCS_PATH + '/monet_tfrec/*.tfrec'))\nPHOTO_FILENAMES = tf.io.gfile.glob(str(GCS_PATH + '/photo_tfrec/*.tfrec'))\nprint('# of Monet TFRecord Files:', len(MONET_FILENAMES))\nprint('# of Photo TFRecord Files:', len(PHOTO_FILENAMES))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Define Function for reading the images\nAll the images for the competition are already sized to `256 x 256`. As these images are RGB images, set the channel to 3. Additionally, we need to scale the images to a `[-1, 1]` scale. \nSince we are building a generative model, we don't need the labels or the image id so we'll only return the image from the TFRecord."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Define the Image height and width (i,e 256 X 256)\nIMG_HEIGHT = 256\nIMG_WIDTH  = 256 \n\ndef decode_image(image):\n    # Decode a JPEG-encoded image to a uint8 tensor.\n    image = tf.image.decode_jpeg(image, channels=3)\n    \n    # Normalize the image to the range of the tanh activation function [-1, 1] for \n    # inputs to the generator and discriminator in GAN model \n    # (i.e. the pixel values are divided by (255/2) to form a value of in a range of [0, 2] and then subtract by 1\n    # to result into a range of [-1, 1])\n    image = (tf.cast(image, tf.float32) / 127.5) - 1        \n    \n    # Reshape the tensor using (256, 256, 3) where 3 is number of channels: Red, Green, and Blue \n    image = tf.reshape(image, [IMG_HEIGHT, IMG_WIDTH, 3])             \n    return image\n\ndef read_tfrecord(example):\n    # Define TFRecord format \n    tfrecord_format = {\n        \"image_name\": tf.io.FixedLenFeature([], tf.string),\n        \"image\":      tf.io.FixedLenFeature([], tf.string),\n        \"target\":     tf.io.FixedLenFeature([], tf.string)\n    }\n    # Parse a single example\n    example = tf.io.parse_single_example(example, tfrecord_format)  \n    # Decode a JPEG image to a uint8 tensor by calling decode_image()\n    image = decode_image(example['image'])                          \n    return image                                                    # Return an image tensor\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Define the function to extract the image from the files."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Set it to tf.data.experimental.AUTOTUNE which will prompt \n# the tf.data runtime to tune the value dynamically at runtime.\nAUTOTUNE = tf.data.experimental.AUTOTUNE  \n\ndef load_dataset(filenames, labeled=True, ordered=False):\n    dataset = tf.data.TFRecordDataset(filenames)\n    # map a dataset with a mapping function read_tfrecord and \n    # Number of parallel calls is set to AUTOTUNE constant previously defined\n    dataset = dataset.map(read_tfrecord, num_parallel_calls=AUTOTUNE)\n    return dataset","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Load the data and display the images"},{"metadata":{"trusted":true},"cell_type":"code","source":"def view_image(ds, nrows=1, ncols=5):\n    ds_iter = iter(ds)\n    # image = next(iter(ds)) # extract 1 from the dataset\n    # image = image.numpy()  # convert the image tensor to NumPy ndarrays.\n\n    fig = plt.figure(figsize=(25, nrows * 5.05 )) # figsize with Width, Height\n    \n    # loop thru all the images (number of rows * number of columns)\n    for i in range(ncols * nrows):\n        image = next(ds_iter)\n        image = image.numpy()\n        ax = fig.add_subplot(nrows, ncols, i+1, xticks=[], yticks=[])\n        ax.imshow(image[0] * 0.5 + .5) # rescale the data in [0, 1] for display","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BATCHSIZE= 1\nmonet_ds = load_dataset(MONET_FILENAMES, labeled=True).batch(BATCHSIZE, drop_remainder=True)\nphoto_ds = load_dataset(PHOTO_FILENAMES, labeled=True).batch(BATCHSIZE, drop_remainder=True)","execution_count":null,"outputs":[]},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"view_image(monet_ds,2, 5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"view_image(photo_ds,2,5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Build the Deep Convolutional Generative Adversarial Networks (DCGAN)\n## Network Upsample and Downsample\nThe `downsample`, as the name suggests, reduces the 2D dimensions, the width and height, of the image by the stride. The stride is the length of the step the filter takes. Since the stride is 2, the filter is applied to every other pixel, hence reducing the weight and height by 2.\n\nWe'll be using an instance normalization instead of batch normalization. As the instance normalization is not standard in the TensorFlow API, we'll use the layer from TensorFlow Add-ons."},{"metadata":{"trusted":true},"cell_type":"code","source":"OUTPUT_CHANNELS = 3\n\n# Define downsample function\n\ndef downsample(filters, kernel_size, apply_norm=True):\n    # Define a random Gaussian weight initializer with a mean of 0 and a standard deviation of 0.02 for the kernel\n    initializer = tf.random_normal_initializer(0., 0.02)\n    # Define gamma initializer for Instance Normalization layer (i.e. gamma_initializer in InstanceNormalization)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    # create a sequential model\n    result = keras.Sequential() \n    # add a Conv2D layer: \n    result.add(layers.Conv2D(filters, kernel_size, strides=2, padding='same',\n                             kernel_initializer=initializer,\n                             use_bias=False)) # no bias vector\n\n    # Apply normization, if True\n    if apply_norm:\n        # CycleGAN uses instance normalization instead of batch normalization.\n        result.add(tfa.layers.InstanceNormalization(gamma_initializer=gamma_init))\n        # result.add(tfa.layers.BatchNormalization(gamma_initializer=gamma_init))\n    \n    # The best practice for GANs (both Generator/Discrimator) is to use Leaky ReLU that allows some values less than zero and \n    # learns where the cut-off should be in each node. \n    # Use the default Negative slope coefficient = 0.3 \n    result.add(layers.LeakyReLU(0.3))\n\n    return result","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`Upsample` does the opposite of downsample and increases the dimensions of the of the image. `Conv2DTranspose` does basically the opposite of a `Conv2D` layer."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Define upsampling function\n\ndef upsample(filters, kernel_size, apply_dropout=False, dropout=0.5):\n    # Define a random Gaussian weight initializer with a mean of 0 and a standard deviation of 0.02 for the kernel\n    initializer = tf.random_normal_initializer(0., 0.02)\n    # Degome gamma initializer for Instance Normalization layer (i.e. gamma_initializer in InstanceNormalization)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    # Create a sequential mode\n    result = keras.Sequential() \n    # add a Conv2DTranspose layer: \n    result.add(layers.Conv2DTranspose(filters, kernel_size, strides=2,\n                                      padding='same',\n                                      kernel_initializer=initializer,\n                                      use_bias=False))\n\n    # CycleGAN uses instance normalization instead of batch normalization.\n    result.add(tfa.layers.InstanceNormalization(gamma_initializer=gamma_init))\n\n    \n    # Add a Dropout layer randomly sets input units to 0 with a frequency of rate 0.5 at each step\n    if apply_dropout:\n        result.add(layers.Dropout(dropout))\n\n    # The best practice for GANs (both Generator/Discrimator) is to use Leaky ReLU that allows some values less than zero and \n    # learns where the cut-off should be in each node. \n    # Use the default Negative slope coefficient = 0.3 \n    result.add(layers.LeakyReLU(0.3))\n\n    return result","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Build the Generator\n\n- The architecture of generator is a modified U-Net.\n- Each block in the **encoder** is (Conv -> Maxpool2D -> InstanceNorm -> Leaky ReLU)\n- Each block in the **decoder** is (Transposed Conv -> InstanceNorm -> Dropout(applied to the first 3 blocks) -> Leaky ReLU)\n- There are skip connections between the encoder and decoder (as in U-Net).\n\n* The generator first downsamples the input image and then upsample while establishing long skip connections. Skip connections are a way to help bypass the vanishing gradient problem by concatenating the output of a layer to multiple layers instead of only one. Here we concatenate the output of the downsample layer to the upsample layer in a symmetrical fashion."},{"metadata":{"trusted":true},"cell_type":"code","source":"def Generator():\n    inputs = layers.Input(shape=[256,256,3])\n\n    # bs = batch size\n    down_stack = [\n        downsample(64, 4, apply_norm=False), # (bs, 128, 128, 64)\n        downsample(128, 4), # (bs, 64, 64, 128)\n        downsample(256, 4), # (bs, 32, 32, 256)\n        downsample(512, 4), # (bs, 16, 16, 512)\n        downsample(512, 4), # (bs, 8, 8, 512)\n        downsample(512, 4), # (bs, 4, 4, 512)\n        downsample(512, 4), # (bs, 2, 2, 512)\n        downsample(512, 4), # (bs, 1, 1, 512)\n    ]\n\n    up_stack = [\n        upsample(512, 4, apply_dropout=True, dropout=0.5), # (bs, 2, 2, 1024)\n        upsample(512, 4, apply_dropout=True, dropout=0.5), # (bs, 4, 4, 1024)\n        upsample(512, 4, apply_dropout=True, dropout=0.5), # (bs, 8, 8, 1024)\n        upsample(512, 4), # (bs, 16, 16, 1024)\n        upsample(256, 4), # (bs, 32, 32, 512)\n        upsample(128, 4), # (bs, 64, 64, 256)\n        upsample(64, 4),  # (bs, 128, 128, 128)\n    ]\n\n    # Define a random Gaussian weight initializer with a mean of 0 and a standard deviation of 0.02 for the kernel\n    initializer = tf.random_normal_initializer(0., 0.02)\n    # The Generator uses the hyperbolic tangent (tanh) activation function in the last (outupt) layer \n    last = layers.Conv2DTranspose(OUTPUT_CHANNELS, 4,\n                                  strides=2,\n                                  padding='same',\n                                  kernel_initializer=initializer,\n                                  activation='tanh') # (bs, 256, 256, 3)\n    \n    # Initialize x with the input layer\n    x = inputs\n\n    # Downsampling through the model\n    skips = []\n    for down in down_stack:\n        x = down(x)\n        skips.append(x)\n\n    skips = reversed(skips[:-1])\n\n    # Upsampling and establishing the skip connections\n    for up, skip in zip(up_stack, skips):\n        x = up(x)\n        x = layers.Concatenate()([x, skip])\n\n    x = last(x)\n\n    return keras.Model(inputs=inputs, outputs=x)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Build the discriminator\n\n- The Discriminator is a PatchGAN.\n- Each block in the discriminator is (Conv -> InstanceNorm -> Leaky ReLU)\n- The shape of the output after the last layer is (batch_size, 30, 30, 1)\n- Each 30x30 patch of the output classifies a 70x70 portion of the input image (such an architecture is called a PatchGAN).\n- Discriminator receives 2 inputs.\n- Input image and the target image, which it should classify as real.\n- Input image and the generated image (output of generator), which it should classify as fake.\n- We concatenate these 2 inputs together in the code (tf.concat([inp, tar], axis=-1))\n- The discriminator takes in the input image and classifies it as real or fake (generated). Instead of outputing a single node, the discriminator outputs a smaller 2D image with higher pixel values indicating a real classification and lower values indicating a fake classification.\n- This patchGAN is nothing but a convolution network. The difference between patchGAN and normal convolution network is that instead of producing output as single scalar vector it generates an NxN array. This NxN array maps to the patch from the input images. And then takes an average to classify the whole image as real or fake."},{"metadata":{"trusted":true},"cell_type":"code","source":"def Discriminator(notarget=True):\n    # Define a random Gaussian weight initializer with a mean of 0 and a standard deviation of 0.02\n    initializer = tf.random_normal_initializer(0., 0.02)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    inp = layers.Input(shape=[256, 256, 3], name='input_image')\n    if notarget:\n        x   = inp  # (bs, 256, 256, 3)\n    else:\n        tar = tf.keras.layers.Input(shape=[256, 256, 3], name='target_image')\n        x   = tf.keras.layers.concatenate([inp, tar])  # (bs, 256, 256, 3*2)\n\n    down1 = downsample(64,  4, False)(x) # (bs, 128, 128, 64)\n    down2 = downsample(128, 4)(down1)   # (bs, 64, 64, 128)\n    down3 = downsample(256, 4)(down2)   # (bs, 32, 32, 256)\n\n    zero_pad1 = layers.ZeroPadding2D()(down3) # (bs, 34, 34, 256) \n    \n    # Strides is set to 1 and padding is default (i.e. valid) in the Discriminator()\n    conv = layers.Conv2D(512, 4, \n                         strides=1,\n                         kernel_initializer=initializer,\n                         use_bias=False)(zero_pad1) # (bs, 31, 31, 512)\n\n    instancenorm = tfa.layers.InstanceNormalization(gamma_initializer=gamma_init)(conv)\n\n    leaky_relu = layers.LeakyReLU(0.3)(instancenorm)\n\n    zero_pad2 = layers.ZeroPadding2D()(leaky_relu) # (bs, 33, 33, 512)\n\n    # Strides is set to 1 and padding is default (i.e. valid) in the Discriminator()\n    # last is 30 X 30 array\n    last = layers.Conv2D(1, 4, \n                         strides=1, \n                         kernel_initializer=initializer)(zero_pad2) # (bs, 30, 30, 1)\n\n    return tf.keras.Model(inputs=inp, outputs=last)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Build the CycleGAN model¶\n* We will subclass a tf.keras.Model so that we can run fit() later to train our model. During the training step, the model transforms a photo to a Monet painting and then back to a photo. The difference between the original photo and the twice-transformed photo is the cycle-consistency loss. We want the original photo and the twice-transformed photo to be similar to one another."},{"metadata":{},"cell_type":"markdown","source":"## Define 2 generators and 2 Discrimators:\nThey are :\n- Monet Generator learns to transform Photo image to Monet painting (i.e. monet_generator)\n- Photo Generator learns to transform Monet painting to Photo image (i.e. photo_generator) \n- Discriminator for Monet learns to differentiate between Monet painting and generated Monet painting (i.e. monet_discriminator)\n- Discriminator for Photo learns to differentiate between Photo image and generated Photo image (i.e. photo_discriminator)\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Open a strategy scope \n# Define the Generators and Discrimators for CycleGAN\nwith strategy.scope():\n    monet_generator     = Generator() # transforms photos to Monet-esque paintings\n    photo_generator     = Generator() # transforms Monet paintings to be more like photos\n    \n    monet_discriminator = Discriminator() # differentiates real Monet paintings and generated Monet paintings\n    photo_discriminator = Discriminator() # differentiates real photos and generated photos\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Define the CycleGan Training Function"},{"metadata":{"trusted":true},"cell_type":"code","source":"class CycleGan(keras.Model):\n    def __init__(\n        # Attributes\n        self,\n        monet_generator,\n        photo_generator,\n        monet_discriminator,\n        photo_discriminator,\n        lambda_cycle=10,\n    ):\n        super(CycleGan, self).__init__()\n        self.m_gen        = monet_generator\n        self.p_gen        = photo_generator\n        self.m_disc       = monet_discriminator\n        self.p_disc       = photo_discriminator\n        self.lambda_cycle = lambda_cycle\n        \n    def compile(\n        self,\n        m_gen_optimizer,\n        p_gen_optimizer,\n        m_disc_optimizer,\n        p_disc_optimizer,\n        gen_loss_fn,\n        disc_loss_fn,\n        cycle_loss_fn,\n        identity_loss_fn\n    ):\n        super(CycleGan, self).compile()\n        self.m_gen_optimizer  = m_gen_optimizer\n        self.p_gen_optimizer  = p_gen_optimizer\n        self.m_disc_optimizer = m_disc_optimizer\n        self.p_disc_optimizer = p_disc_optimizer\n        self.gen_loss_fn      = gen_loss_fn\n        self.disc_loss_fn     = disc_loss_fn\n        self.cycle_loss_fn    = cycle_loss_fn\n        self.identity_loss_fn = identity_loss_fn\n        \n    # Defining the training procedure\n        \n    def train_step(self, batch_data):\n        real_monet, real_photo = batch_data\n        \n        with tf.GradientTape(persistent=True) as tape:\n            # Calculate losses for Monet Processing \n            fake_monet        = self.m_gen(real_photo, training=True)  \n            disc_fake_monet   = self.m_disc(fake_monet, training=True) \n            disc_real_monet   = self.m_disc(real_monet, training=True)\n            same_monet        = self.m_gen(real_monet, training=True)\n            monet_gen_loss    = self.gen_loss_fn(disc_fake_monet)\n            monet_disc_loss   = self.disc_loss_fn(disc_real_monet, disc_fake_monet)\n            monet_identity_loss = self.identity_loss_fn(real_monet, same_monet, self.lambda_cycle)\n\n            # Calculate losses for Photo Processing\n            fake_photo        = self.p_gen(real_monet, training=True)\n            disc_fake_photo   = self.p_disc(fake_photo, training=True)\n            disc_real_photo   = self.p_disc(real_photo, training=True)\n            same_photo        = self.p_gen(real_photo, training=True)\n            photo_gen_loss    = self.gen_loss_fn(disc_fake_photo)\n            photo_disc_loss   = self.disc_loss_fn(disc_real_photo, disc_fake_photo)\n            photo_identity_loss = self.identity_loss_fn(real_photo, same_photo, self.lambda_cycle)\n            \n            # Calculate total cycled losses\n            cycled_photo      = self.p_gen(fake_monet, training=True) \n            cycled_monet      = self.m_gen(fake_photo, training=True)\n            photo_cycled_loss = self.cycle_loss_fn(real_photo, cycled_photo, self.lambda_cycle) \n            monet_cycled_loss = self.cycle_loss_fn(real_monet, cycled_monet, self.lambda_cycle)\n            total_cycled_loss  = photo_cycled_loss + monet_cycled_loss\n\n            # evaluates total generator loss\n            total_monet_gen_loss = monet_gen_loss + monet_identity_loss + total_cycled_loss\n            total_photo_gen_loss = photo_gen_loss + photo_identity_loss + total_cycled_loss\n \n            \n        # Calculate the gradients for generator and discriminator\n        monet_generator_gradients     = tape.gradient(total_monet_gen_loss, self.m_gen.trainable_variables)\n        photo_generator_gradients     = tape.gradient(total_photo_gen_loss, self.p_gen.trainable_variables)\n        monet_discriminator_gradients = tape.gradient(monet_disc_loss, self.m_disc.trainable_variables)\n        photo_discriminator_gradients = tape.gradient(photo_disc_loss, self.p_disc.trainable_variables)\n\n        # Apply the gradients to the optimizer\n        self.m_gen_optimizer.apply_gradients(zip(monet_generator_gradients, self.m_gen.trainable_variables))\n        self.p_gen_optimizer.apply_gradients(zip(photo_generator_gradients, self.p_gen.trainable_variables))\n        self.m_disc_optimizer.apply_gradients(zip(monet_discriminator_gradients, self.m_disc.trainable_variables))\n        self.p_disc_optimizer.apply_gradients(zip(photo_discriminator_gradients, self.p_disc.trainable_variables))\n        \n        return {\n            \"monet_gen_loss\": total_monet_gen_loss,\n            \"photo_gen_loss\": total_photo_gen_loss,\n            \"monet_disc_loss\": monet_disc_loss,\n            \"photo_disc_loss\": photo_disc_loss\n        }","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Initialize the optimizers for all the generators and the discriminators.\n#### Define Adam optimizer for Generators and Discriminators"},{"metadata":{"trusted":true},"cell_type":"code","source":"# The Adam optimizer with tuned hyperparameters is used for training. \n# The learning rate is using 0.0002.\n# The momentum term β1 is specified to 0.5 helped stabilize training.\nwith strategy.scope():\n    monet_generator_optimizer     = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n    photo_generator_optimizer     = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n\n    monet_discriminator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n    photo_discriminator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Define Loss Functions\n- Discriminator Loss Function\n- Generator Loss Function\n- Cycle Consistency Loss Function\n- Identity Loss Function"},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    loss_obj = tf.keras.losses.BinaryCrossentropy(from_logits=True,\n                                                  reduction=tf.keras.losses.Reduction.NONE)\n\n    # The discriminator loss function:\n    # It compares real images to a matrix of 1s and fake images to a matrix of 0s. \n    # The perfect discriminator will output all 1s for real images and all 0s for fake images. \n    # The discriminator loss outputs the average of the real and generated loss.\n    def discriminator_loss(real, generated):\n        real_loss       = loss_obj(tf.ones_like(real), real)\n        generated_loss  = loss_obj(tf.zeros_like(generated), generated)\n        avg_disc_loss   = (real_loss + generated_loss) * 0.5\n        return avg_disc_loss\n\n    # The generator loss function:\n    # The generator wants to fool the discriminator into thinking the generated image is real.\n    # The perfect generator will have the discriminator output only 1s. \n    # Thus, it compares the generated image to a matrix of 1s to find the loss.\n    def generator_loss(generated):\n        return loss_obj(tf.ones_like(generated), generated)\n \n    # The cycle consistency loss is calculatd by finding the average of their difference.\n    def calc_cycle_loss(real_image, cycled_image, LAMBDA):\n        loss1 = tf.reduce_mean(tf.abs(real_image - cycled_image))\n        return LAMBDA * loss1\n\n    # The identity loss compares the image with its generator (i.e. photo with photo generator). \n    # If given a photo as input, we want it to generate the same image as the image was originally a photo. \n    # The identity loss compares the input with the output of the generator.\n    def identity_loss(real_image, same_image, LAMBDA):\n        loss = tf.reduce_mean(tf.abs(real_image - same_image))\n        return LAMBDA * 0.5 * loss","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    \n    # Create a cycle_gan_model\n    cycle_gan_model = CycleGan(monet_generator, \n                               photo_generator, \n                               monet_discriminator, \n                               photo_discriminator\n    )\n\n    # Configure the cycle_gan_model with optimizers and loss function \n    cycle_gan_model.compile(\n        m_gen_optimizer  = monet_generator_optimizer,\n        p_gen_optimizer  = photo_generator_optimizer,\n        m_disc_optimizer = monet_discriminator_optimizer,\n        p_disc_optimizer = photo_discriminator_optimizer,\n        gen_loss_fn      = generator_loss,\n        disc_loss_fn     = discriminator_loss,\n        cycle_loss_fn    = calc_cycle_loss,\n        identity_loss_fn = identity_loss\n    )\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Train the model\nEPOCHS=25\ncycle_gan_model.fit(tf.data.Dataset.zip((monet_ds, photo_ds)), epochs=EPOCHS)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"_, ax = plt.subplots(5, 2, figsize=(25, 25))\nfor i, img in enumerate(photo_ds.take(5)):\n    prediction = monet_generator(img, training=False)[0].numpy()\n    \n    # Convert the data from the range [-1,1] to [0, 255] by multiplying 127.5 and plus 127.5.\n    prediction = (prediction * 127.5 + 127.5).astype(np.uint8)\n    img        = (img[0] * 127.5 + 127.5).numpy().astype(np.uint8)\n\n    ax[i, 0].imshow(img)\n    ax[i, 1].imshow(prediction)\n    ax[i, 0].set_title(\"Input Photo\")\n    ax[i, 1].set_title(\"Monet-esque\")\n    ax[i, 0].axis(\"off\")\n    ax[i, 1].axis(\"off\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import shutil\nimport PIL\n! mkdir ../images\n\ni = 1\nfor img in photo_ds:\n    prediction = monet_generator(img, training=False)[0].numpy()\n    prediction = (prediction * 127.5 + 127.5).astype(np.uint8)\n    im = PIL.Image.fromarray(prediction)\n    im.save(\"../images/\" + str(i) + \".jpg\")\n    i += 1\n\nshutil.make_archive(\"/kaggle/working/images\", 'zip', \"/kaggle/images\")","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}