{"cells":[{"metadata":{},"cell_type":"markdown","source":"# 1. Adversarial ML\n\n**Adversarial ML** is a research field that causes a state-of-the-art neural network to mis-classify any input image to whatever class we choose. This is done simply by adding a small amount of 'specialized' noise to the input image. The changes are imperceptible to humans, but it fools the neural network.\n\nFor instance, in the below image of Willy Wonka, the Inception model originally classified the image as 'sunglasses' (score 31.48%). But once we added adversarial noise, the Inception model classifies the image as a 'bookcase' (score 99.03%).\n\n

\n \"neofetch\"\n

\n\nAn adversarial example is a sample of input data which has been modified very slightly in a way that is intended to cause a machine learning classifier to misclassify it. In many cases, these modifications can be so subtle that a human\nobserver does not even notice the modification at all, yet the classifier still makes a mistake. \n\nThis is a very serious problem! Adversarial examples pose security concerns because they could be used to perform an attack on machine learning systems, even if the adversary has no access to the underlying model. Imagine what would happen if a self-driving car ignores a stop-sign or a person crossing the road because its neural networks have mis-classified the input images."},{"metadata":{},"cell_type":"markdown","source":"# 2. Introduction to GAN\n\n##### Analogy to GAN\n\nLet's take a theoretical example of the process of **money counterfeiting**. In this process, we can imagine two types agents: **a criminal and cop**. Let us look into their competing objectives:\n\n* **Criminal's Objective**: The main objective of the criminal is to come up with complex ways of counterfeiting money such that the Cop cannot distinguish between counterfeited money and real money.\n* **Cop's Objective**: The main objective of the cop is to come up with complex ways so as to distinguish between counterfeited money and real money.\n\nAs this process progresses the cop develops more and more sophisticated technology to detect money counterfeiting and criminal develops more and more sophisticated technology to counterfeit money. This is the basis of what is called an **Adversarial Process**.\n\nGenerative Adversarial Networks take advantage of Adversarial Processes to train two Neural Networks who compete with each other until a desirable equilibrium is reached."},{"metadata":{},"cell_type":"markdown","source":"# 2.1 Discriminator Algorithms\nClassification algorithms that constructs decision boundaries that separates classes are called **Discriminative models**. Discriminative algorithms try to classify input data; that is, given the features of a data instance, they predict a label or category to which that data belongs. \n\n

\n \"neofetch\"\n

\n\nDiscriminative algorithms **map features to labels**. \n"},{"metadata":{},"cell_type":"markdown","source":"# 2.2 Generative Algirithms\nGenerative algorithms performs the functionality of attempting to **predict features given a certain label**. They aim at learning the true data distribution of the training set so as to generate new data points with some variations. \n

\n \"neofetch\"\n

"},{"metadata":{},"cell_type":"markdown","source":"# 2.2.1 Gnerative Adversarial Networks\nGAN’S are generative models that will learn the distribution of data to generate new points. GANs were first invented by Ian Goodfellow in 2014. \n\nImagine a random image(sampled from gaussian distribution) sent through a neural network.The output of the network is also of same size(image).Now if we could tell the network to tweak it’s parameters until the output image is close to our reference image but different,we get a generative model."},{"metadata":{},"cell_type":"markdown","source":"# Introduction and Setup\n\nThis notebook utilizes a CycleGAN architecture to add Monet-style to photos. For this tutorial, we will be using the TFRecord dataset. Import the following packages and change the accelerator to TPU.\n\nFor more information, check out [TensorFlow](https://www.tensorflow.org/tutorials/generative/cyclegan) and [Keras](https://keras.io/examples/generative/cyclegan/) CycleGAN documentation pages."},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import os, random, json, PIL, shutil, re\nimport tensorflow as tf\nfrom tensorflow import keras\nfrom tensorflow.keras import layers\nimport tensorflow_addons as tfa\n\nfrom kaggle_datasets import KaggleDatasets\nimport matplotlib.pyplot as plt\nimport numpy as np\n\ntry:\n tpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n print('Device:', tpu.master())\n tf.config.experimental_connect_to_cluster(tpu)\n tf.tpu.experimental.initialize_tpu_system(tpu)\n strategy = tf.distribute.experimental.TPUStrategy(tpu)\nexcept:\n strategy = tf.distribute.get_strategy()\nprint('Number of replicas:', strategy.num_replicas_in_sync)\n\nAUTOTUNE = tf.data.experimental.AUTOTUNE\n \nprint(tf.__version__)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Load in the data\n\nWe want to keep our photo dataset and our Monet dataset separate. First, load in the filenames of the TFRecords."},{"metadata":{"trusted":true},"cell_type":"code","source":"GCS_PATH = KaggleDatasets().get_gcs_path()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Setting the Contants"},{"metadata":{"trusted":true},"cell_type":"code","source":"BATCH_SIZE = 1\nEPOCHS = 60","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"MONET_FILENAMES = tf.io.gfile.glob(str(GCS_PATH + '/monet_tfrec/*.tfrec'))\nprint('Monet TFRecord Files:', len(MONET_FILENAMES))\n\nPHOTO_FILENAMES = tf.io.gfile.glob(str(GCS_PATH + '/photo_tfrec/*.tfrec'))\nprint('Photo TFRecord Files:', len(PHOTO_FILENAMES))\n\ndef count_data_items(filenames):\n n = [int(re.compile(r\"-([0-9]*)\\.\").search(filename).group(1)) for filename in filenames]\n return np.sum(n)\n\nn_monet_samples = count_data_items(MONET_FILENAMES)\nprint(f'Monet image files:,{n_monet_samples}' )\nn_photo_samples = count_data_items(PHOTO_FILENAMES)\nprint(f'Photo image files:,{n_photo_samples}' )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"All the images for the competition are already sized to 256x256. As these images are RGB images, set the channel to 3. Additionally, we need to scale the images to a [-1, 1] scale. Because we are building a generative model, we don't need the labels or the image id so we'll only return the image from the TFRecord."},{"metadata":{"trusted":true},"cell_type":"code","source":"IMAGE_SIZE = [256, 256]\n\ndef decode_image(image):\n image = tf.image.decode_jpeg(image, channels=3)\n image = (tf.cast(image, tf.float32) / 127.5) - 1\n image = tf.reshape(image, [*IMAGE_SIZE, 3])\n return image\n\ndef read_tfrecord(example):\n tfrecord_format = {\n \"image_name\": tf.io.FixedLenFeature([], tf.string),\n \"image\": tf.io.FixedLenFeature([], tf.string),\n \"target\": tf.io.FixedLenFeature([], tf.string)\n }\n example = tf.io.parse_single_example(example, tfrecord_format)\n image = decode_image(example['image'])\n return image","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Define the function to extract the image from the files."},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_dataset(filenames, labeled=True, ordered=False):\n dataset = tf.data.TFRecordDataset(filenames)\n dataset = dataset.map(read_tfrecord, num_parallel_calls=AUTOTUNE)\n return dataset","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_gan_dataset(monet_files, photo_files, augment=None, repeat=True, shuffle=True, batch_size=1):\n\n monet_ds = load_dataset(monet_files)\n photo_ds = load_dataset(photo_files)\n\n if repeat:\n monet_ds = monet_ds.repeat()\n photo_ds = photo_ds.repeat()\n if shuffle:\n monet_ds = monet_ds.shuffle(2048)\n photo_ds = photo_ds.shuffle(2048)\n \n monet_ds = monet_ds.batch(batch_size, drop_remainder=True)\n photo_ds = photo_ds.batch(batch_size, drop_remainder=True)\n monet_ds = monet_ds.cache()\n photo_ds = photo_ds.cache()\n monet_ds = monet_ds.prefetch(AUTO)\n photo_ds = photo_ds.prefetch(AUTO)\n \n gan_ds = tf.data.Dataset.zip((monet_ds, photo_ds))\n \n return gan_ds\n\ndef display_samples(ds, row, col):\n ds_iter = iter(ds)\n plt.figure(figsize=(15, int(15*row/col)))\n for j in range(row*col):\n example_sample = next(ds_iter)\n plt.subplot(row,col,j+1)\n plt.axis('off')\n plt.imshow(example_sample[0] * 0.5 + 0.5)\n plt.show()\n \ndef display_generated_samples(ds, model, n_samples):\n ds_iter = iter(ds)\n for n_sample in range(n_samples):\n example_sample = next(ds_iter)\n generated_sample = model.predict(example_sample)\n \n plt.subplot(121)\n plt.title(\"input image\")\n plt.imshow(example_sample[0] * 0.5 + 0.5)\n plt.axis('off')\n \n plt.subplot(122)\n plt.title(\"Generated image\")\n plt.imshow(generated_sample[0] * 0.5 + 0.5)\n plt.axis('off')\n plt.show()\n \ndef predict_and_save(input_ds, generator_model, output_path):\n i = 1\n for img in input_ds:\n prediction = generator_model(img, training=False)[0].numpy() # make predition\n prediction = (prediction * 127.5 + 127.5).astype(np.uint8) # re-scale\n im = PIL.Image.fromarray(prediction)\n im.save(f'{output_path}{str(i)}.jpg')\n i += 1\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Display few Monet paitings"},{"metadata":{"trusted":true},"cell_type":"code","source":"display_samples(load_dataset(MONET_FILENAMES).batch(1), 4, 6)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Display few Photo Samples"},{"metadata":{"trusted":true},"cell_type":"code","source":"display_samples(load_dataset(PHOTO_FILENAMES).batch(1), 4, 6)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's load in our datasets."},{"metadata":{"trusted":true},"cell_type":"code","source":"monet_ds = load_dataset(MONET_FILENAMES, labeled=True).batch(1)\nphoto_ds = load_dataset(PHOTO_FILENAMES, labeled=True).batch(1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"example_monet = next(iter(monet_ds))\nexample_photo = next(iter(photo_ds))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's visualize a photo example and a Monet example."},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.subplot(121)\nplt.title('Photo')\nplt.imshow(example_photo[0] * 0.5 + 0.5)\n\nplt.subplot(122)\nplt.title('Monet')\nplt.imshow(example_monet[0] * 0.5 + 0.5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Build the generator\n\nWe'll be using a UNET architecture for our CycleGAN. To build our generator, let's first define our `downsample` and `upsample` methods.\n\nThe `downsample`, as the name suggests, reduces the 2D dimensions, the width and height, of the image by the stride. The stride is the length of the step the filter takes. Since the stride is 2, the filter is applied to every other pixel, hence reducing the weight and height by 2.\n\nWe'll be using an instance normalization instead of batch normalization. As the instance normalization is not standard in the TensorFlow API, we'll use the layer from TensorFlow Add-ons."},{"metadata":{"trusted":true},"cell_type":"code","source":"OUTPUT_CHANNELS = 3\n\ndef downsample(filters, size, apply_instancenorm=True):\n initializer = tf.random_normal_initializer(0., 0.02)\n gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n result = keras.Sequential()\n result.add(layers.Conv2D(filters, size, strides=2, padding='same',\n kernel_initializer=initializer, use_bias=False))\n\n if apply_instancenorm:\n result.add(tfa.layers.InstanceNormalization(gamma_initializer=gamma_init))\n\n result.add(layers.LeakyReLU())\n\n return result","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`Upsample` does the opposite of downsample and increases the dimensions of the of the image. `Conv2DTranspose` does basically the opposite of a `Conv2D` layer."},{"metadata":{"trusted":true},"cell_type":"code","source":"def upsample(filters, size, apply_dropout=False):\n initializer = tf.random_normal_initializer(0., 0.02)\n gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n result = keras.Sequential()\n result.add(layers.Conv2DTranspose(filters, size, strides=2,\n padding='same',\n kernel_initializer=initializer,\n use_bias=False))\n\n result.add(tfa.layers.InstanceNormalization(gamma_initializer=gamma_init))\n\n if apply_dropout:\n result.add(layers.Dropout(0.5))\n\n result.add(layers.ReLU())\n\n return result","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's build our generator!\n\nThe generator first downsamples the input image and then upsample while establishing long skip connections. Skip connections are a way to help bypass the vanishing gradient problem by concatenating the output of a layer to multiple layers instead of only one. Here we concatenate the output of the downsample layer to the upsample layer in a symmetrical fashion."},{"metadata":{"trusted":true},"cell_type":"code","source":"def Generator():\n inputs = layers.Input(shape=[256,256,3])\n\n # bs = batch size\n down_stack = [\n downsample(64, 4, apply_instancenorm=False), # (bs, 128, 128, 64)\n downsample(128, 4), # (bs, 64, 64, 128)\n downsample(256, 4), # (bs, 32, 32, 256)\n downsample(512, 4), # (bs, 16, 16, 512)\n downsample(512, 4), # (bs, 8, 8, 512)\n downsample(512, 4), # (bs, 4, 4, 512)\n downsample(512, 4), # (bs, 2, 2, 512)\n downsample(512, 4), # (bs, 1, 1, 512)\n ]\n\n up_stack = [\n upsample(512, 4, apply_dropout=True), # (bs, 2, 2, 1024)\n upsample(512, 4, apply_dropout=True), # (bs, 4, 4, 1024)\n upsample(512, 4, apply_dropout=True), # (bs, 8, 8, 1024)\n upsample(512, 4), # (bs, 16, 16, 1024)\n upsample(256, 4), # (bs, 32, 32, 512)\n upsample(128, 4), # (bs, 64, 64, 256)\n upsample(64, 4), # (bs, 128, 128, 128)\n ]\n\n initializer = tf.random_normal_initializer(0., 0.02)\n last = layers.Conv2DTranspose(OUTPUT_CHANNELS, 4,\n strides=2,\n padding='same',\n kernel_initializer=initializer,\n activation='tanh') # (bs, 256, 256, 3)\n\n x = inputs\n\n # Downsampling through the model\n skips = []\n for down in down_stack:\n x = down(x)\n skips.append(x)\n\n skips = reversed(skips[:-1])\n\n # Upsampling and establishing the skip connections\n for up, skip in zip(up_stack, skips):\n x = up(x)\n x = layers.Concatenate()([x, skip])\n\n x = last(x)\n\n return keras.Model(inputs=inputs, outputs=x)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Build the discriminator\n\nThe discriminator takes in the input image and classifies it as real or fake (generated). Instead of outputing a single node, the discriminator outputs a smaller 2D image with higher pixel values indicating a real classification and lower values indicating a fake classification."},{"metadata":{"trusted":true},"cell_type":"code","source":"def Discriminator():\n initializer = tf.random_normal_initializer(0., 0.02)\n gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n inp = layers.Input(shape=[256, 256, 3], name='input_image')\n\n x = inp\n\n down1 = downsample(64, 4, False)(x) # (bs, 128, 128, 64)\n down2 = downsample(128, 4)(down1) # (bs, 64, 64, 128)\n down3 = downsample(256, 4)(down2) # (bs, 32, 32, 256)\n\n zero_pad1 = layers.ZeroPadding2D()(down3) # (bs, 34, 34, 256)\n conv = layers.Conv2D(512, 4, strides=1,\n kernel_initializer=initializer,\n use_bias=False)(zero_pad1) # (bs, 31, 31, 512)\n\n norm1 = tfa.layers.InstanceNormalization(gamma_initializer=gamma_init)(conv)\n\n leaky_relu = layers.LeakyReLU()(norm1)\n\n zero_pad2 = layers.ZeroPadding2D()(leaky_relu) # (bs, 33, 33, 512)\n\n last = layers.Conv2D(1, 4, strides=1,\n kernel_initializer=initializer)(zero_pad2) # (bs, 30, 30, 1)\n\n return tf.keras.Model(inputs=inp, outputs=last)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n monet_generator = Generator() # transforms photos to Monet-esque paintings\n photo_generator = Generator() # transforms Monet paintings to be more like photos\n\n monet_discriminator = Discriminator() # differentiates real Monet paintings and generated Monet paintings\n photo_discriminator = Discriminator() # differentiates real photos and generated photos","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since our generators are not trained yet, the generated Monet-esque photo does not show what is expected at this point."},{"metadata":{"trusted":true},"cell_type":"code","source":"to_monet = monet_generator(example_photo)\n\nplt.subplot(1, 2, 1)\nplt.title(\"Original Photo\")\nplt.imshow(example_photo[0] * 0.5 + 0.5)\n\nplt.subplot(1, 2, 2)\nplt.title(\"Monet-esque Photo\")\nplt.imshow(to_monet[0] * 0.5 + 0.5)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Build the CycleGAN model\n\nWe will subclass a `tf.keras.Model` so that we can run `fit()` later to train our model. During the training step, the model transforms a photo to a Monet painting and then back to a photo. The difference between the original photo and the twice-transformed photo is the cycle-consistency loss. We want the original photo and the twice-transformed photo to be similar to one another.\n\nThe losses are defined in the next section."},{"metadata":{"trusted":true},"cell_type":"code","source":"class CycleGan(keras.Model):\n def __init__(\n self,\n monet_generator,\n photo_generator,\n monet_discriminator,\n photo_discriminator,\n lambda_cycle=10,\n ):\n super(CycleGan, self).__init__()\n self.m_gen = monet_generator\n self.p_gen = photo_generator\n self.m_disc = monet_discriminator\n self.p_disc = photo_discriminator\n self.lambda_cycle = lambda_cycle\n \n def compile(\n self,\n m_gen_optimizer,\n p_gen_optimizer,\n m_disc_optimizer,\n p_disc_optimizer,\n gen_loss_fn,\n disc_loss_fn,\n cycle_loss_fn,\n identity_loss_fn\n ):\n super(CycleGan, self).compile()\n self.m_gen_optimizer = m_gen_optimizer\n self.p_gen_optimizer = p_gen_optimizer\n self.m_disc_optimizer = m_disc_optimizer\n self.p_disc_optimizer = p_disc_optimizer\n self.gen_loss_fn = gen_loss_fn\n self.disc_loss_fn = disc_loss_fn\n self.cycle_loss_fn = cycle_loss_fn\n self.identity_loss_fn = identity_loss_fn\n \n def train_step(self, batch_data):\n real_monet, real_photo = batch_data\n \n with tf.GradientTape(persistent=True) as tape:\n # photo to monet back to photo\n fake_monet = self.m_gen(real_photo, training=True)\n cycled_photo = self.p_gen(fake_monet, training=True)\n\n # monet to photo back to monet\n fake_photo = self.p_gen(real_monet, training=True)\n cycled_monet = self.m_gen(fake_photo, training=True)\n\n # generating itself\n same_monet = self.m_gen(real_monet, training=True)\n same_photo = self.p_gen(real_photo, training=True)\n\n # discriminator used to check, inputing real images\n disc_real_monet = self.m_disc(real_monet, training=True)\n disc_real_photo = self.p_disc(real_photo, training=True)\n\n # discriminator used to check, inputing fake images\n disc_fake_monet = self.m_disc(fake_monet, training=True)\n disc_fake_photo = self.p_disc(fake_photo, training=True)\n\n # evaluates generator loss\n monet_gen_loss = self.gen_loss_fn(disc_fake_monet)\n photo_gen_loss = self.gen_loss_fn(disc_fake_photo)\n\n # evaluates total cycle consistency loss\n total_cycle_loss = self.cycle_loss_fn(real_monet, cycled_monet, self.lambda_cycle) + self.cycle_loss_fn(real_photo, cycled_photo, self.lambda_cycle)\n\n # evaluates total generator loss\n total_monet_gen_loss = monet_gen_loss + total_cycle_loss + self.identity_loss_fn(real_monet, same_monet, self.lambda_cycle)\n total_photo_gen_loss = photo_gen_loss + total_cycle_loss + self.identity_loss_fn(real_photo, same_photo, self.lambda_cycle)\n\n # evaluates discriminator loss\n monet_disc_loss = self.disc_loss_fn(disc_real_monet, disc_fake_monet)\n photo_disc_loss = self.disc_loss_fn(disc_real_photo, disc_fake_photo)\n\n # Calculate the gradients for generator and discriminator\n monet_generator_gradients = tape.gradient(total_monet_gen_loss,\n self.m_gen.trainable_variables)\n photo_generator_gradients = tape.gradient(total_photo_gen_loss,\n self.p_gen.trainable_variables)\n\n monet_discriminator_gradients = tape.gradient(monet_disc_loss,\n self.m_disc.trainable_variables)\n photo_discriminator_gradients = tape.gradient(photo_disc_loss,\n self.p_disc.trainable_variables)\n\n # Apply the gradients to the optimizer\n self.m_gen_optimizer.apply_gradients(zip(monet_generator_gradients,\n self.m_gen.trainable_variables))\n\n self.p_gen_optimizer.apply_gradients(zip(photo_generator_gradients,\n self.p_gen.trainable_variables))\n\n self.m_disc_optimizer.apply_gradients(zip(monet_discriminator_gradients,\n self.m_disc.trainable_variables))\n\n self.p_disc_optimizer.apply_gradients(zip(photo_discriminator_gradients,\n self.p_disc.trainable_variables))\n \n return {\n \"monet_gen_loss\": total_monet_gen_loss,\n \"photo_gen_loss\": total_photo_gen_loss,\n \"monet_disc_loss\": monet_disc_loss,\n \"photo_disc_loss\": photo_disc_loss\n }","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Define loss functions\n\nThe discriminator loss function below compares real images to a matrix of 1s and fake images to a matrix of 0s. The perfect discriminator will output all 1s for real images and all 0s for fake images. The discriminator loss outputs the average of the real and generated loss."},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n def discriminator_loss(real, generated):\n real_loss = tf.keras.losses.BinaryCrossentropy(from_logits=True, reduction=tf.keras.losses.Reduction.NONE)(tf.ones_like(real), real)\n\n generated_loss = tf.keras.losses.BinaryCrossentropy(from_logits=True, reduction=tf.keras.losses.Reduction.NONE)(tf.zeros_like(generated), generated)\n\n total_disc_loss = real_loss + generated_loss\n\n return total_disc_loss * 0.5","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The generator wants to fool the discriminator into thinking the generated image is real. The perfect generator will have the discriminator output only 1s. Thus, it compares the generated image to a matrix of 1s to find the loss."},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n def generator_loss(generated):\n return tf.keras.losses.BinaryCrossentropy(from_logits=True, reduction=tf.keras.losses.Reduction.NONE)(tf.ones_like(generated), generated)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We want our original photo and the twice transformed photo to be similar to one another. Thus, we can calculate the cycle consistency loss be finding the average of their difference."},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n def calc_cycle_loss(real_image, cycled_image, LAMBDA):\n loss1 = tf.reduce_mean(tf.abs(real_image - cycled_image))\n\n return LAMBDA * loss1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The identity loss compares the image with its generator (i.e. photo with photo generator). If given a photo as input, we want it to generate the same image as the image was originally a photo. The identity loss compares the input with the output of the generator."},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n def identity_loss(real_image, same_image, LAMBDA):\n loss = tf.reduce_mean(tf.abs(real_image - same_image))\n return LAMBDA * 0.5 * loss","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Train the CycleGAN\n\nLet's compile our model. Since we used `tf.keras.Model` to build our CycleGAN, we can just ude the `fit` function to train our model."},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n monet_generator_optimizer = tf.keras.optimizers.Adam(1e-4, beta_1=0.5)\n photo_generator_optimizer = tf.keras.optimizers.Adam(1e-4, beta_1=0.5)\n\n monet_discriminator_optimizer = tf.keras.optimizers.Adam(3e-4, beta_1=0.5)\n photo_discriminator_optimizer = tf.keras.optimizers.Adam(3e-4, beta_1=0.5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n cycle_gan_model = CycleGan(\n monet_generator, photo_generator, monet_discriminator, photo_discriminator\n )\n\n cycle_gan_model.compile(\n m_gen_optimizer = monet_generator_optimizer,\n p_gen_optimizer = photo_generator_optimizer,\n m_disc_optimizer = monet_discriminator_optimizer,\n p_disc_optimizer = photo_discriminator_optimizer,\n gen_loss_fn = generator_loss,\n disc_loss_fn = discriminator_loss,\n cycle_loss_fn = calc_cycle_loss,\n identity_loss_fn = identity_loss\n )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Setting the epochs = 100; earlier it was 25\ncycle_gan_model.fit(\n tf.data.Dataset.zip((monet_ds, photo_ds)),\n batch_size = BATCH_SIZE,\n# steps_per_epoch = (n_monet_samples//BATCH_SIZE),\n epochs=EPOCHS \n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Visualize our Monet-esque photos"},{"metadata":{"trusted":true},"cell_type":"code","source":"_, ax = plt.subplots(5, 2, figsize=(12, 12))\nfor i, img in enumerate(photo_ds.take(5)):\n prediction = monet_generator(img, training=False)[0].numpy()\n prediction = (prediction * 127.5 + 127.5).astype(np.uint8)\n img = (img[0] * 127.5 + 127.5).numpy().astype(np.uint8)\n\n ax[i, 0].imshow(img)\n ax[i, 1].imshow(prediction)\n ax[i, 0].set_title(\"Input Photo\")\n ax[i, 1].set_title(\"Monet-esque\")\n ax[i, 0].axis(\"off\")\n ax[i, 1].axis(\"off\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Create submission file"},{"metadata":{"trusted":true},"cell_type":"code","source":"import PIL\n! mkdir ../images","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"i = 1\nfor img in photo_ds:\n prediction = monet_generator(img, training=False)[0].numpy()\n prediction = (prediction * 127.5 + 127.5).astype(np.uint8)\n im = PIL.Image.fromarray(prediction)\n im.save(\"../images/\" + str(i) + \".jpg\")\n i += 1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import shutil\nshutil.make_archive(\"/kaggle/working/images\", 'zip', \"/kaggle/images\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}