{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## Using CycleGAN to generate Monet-styled images\n\n<img src= \"https://d32dm0rphc51dk.cloudfront.net/95EYpbkhcWsEKG18njyhCw/large.jpg\" style='width: 400px;'>","metadata":{}},{"cell_type":"markdown","source":"## Problem statement\n\nThis is a project about creating & building a GAN (CycleGAN) model, **to generate Monet-styled images from casual photos.**\n\nI chose CycleGAN after multiple failed attemps of training a **DCGAN** model for this problem, where I was trying to modify the loss function to make the training results more consistent.\n\nUnlike problems that include generating a face of people or animal based on multiple good frontal shots of their faces, training a general DCGAN model to create a filter based on **paintings of different landscapes**, provided me with tons of new problems to solve. I tried creating new loss functions that considered **color loss**, but they did not work well.\n\nHowever, **[CycleGAN](https://junyanz.github.io/CycleGAN/)** is specifically designed for these tasks, by ensuring that the output from the generator can be translated back to the original image, by introducing the **cycle consistency loss.**\n\nThere are multiple paintings and photos provided for training, and there are two file formats **(.jpg, .tfrec)** available for each group.\n\nI will use **Keras** library to create / import some models, and optimize & evaluate the model parameters to finalize the submission output.\n\nCode for importing & printing image files was borrowed from **[Amy Jang's public notebook](https://www.kaggle.com/code/amyjang/monet-cyclegan-tutorial)**, and the training pipeline is designed based on this **[official Keras tutorial](https://keras.io/examples/generative/cyclegan/)**. \n\n## Environment setup & importing data","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport tensorflow as tf\nimport tensorflow_addons as tfa\nimport tensorflow_datasets as tfds\nimport matplotlib.image as mpimg\nimport matplotlib.pyplot as plt\n\nimport keras,os\n\nfrom keras.models import Sequential, Model\nfrom keras.losses import BinaryCrossentropy\nfrom keras.optimizers import Adam\nfrom kaggle_datasets import KaggleDatasets\n\nfrom tensorflow.keras.callbacks import Callback\nfrom tensorflow.keras import layers\n\nos.system('pip install visualkeras')\nimport visualkeras","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:05:27.812778Z","iopub.execute_input":"2023-03-20T19:05:27.813138Z","iopub.status.idle":"2023-03-20T19:05:48.382834Z","shell.execute_reply.started":"2023-03-20T19:05:27.813102Z","shell.execute_reply":"2023-03-20T19:05:48.381722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"GCS_PATH = KaggleDatasets().get_gcs_path()\n\nMONET_FILENAMES = tf.io.gfile.glob(str(GCS_PATH + '/monet_tfrec/*.tfrec'))\nprint('Monet TFRecord Files:', len(MONET_FILENAMES))\n\nPHOTO_FILENAMES = tf.io.gfile.glob(str(GCS_PATH + '/photo_tfrec/*.tfrec'))\nprint('Photo TFRecord Files:', len(PHOTO_FILENAMES))","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:05:48.386917Z","iopub.execute_input":"2023-03-20T19:05:48.389054Z","iopub.status.idle":"2023-03-20T19:05:50.394893Z","shell.execute_reply.started":"2023-03-20T19:05:48.389011Z","shell.execute_reply":"2023-03-20T19:05:50.393770Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"IMAGE_SIZE = (256, 256, 3)\nBATCH_SIZE = 1","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:05:50.396479Z","iopub.execute_input":"2023-03-20T19:05:50.397352Z","iopub.status.idle":"2023-03-20T19:05:50.402669Z","shell.execute_reply.started":"2023-03-20T19:05:50.397313Z","shell.execute_reply":"2023-03-20T19:05:50.401166Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def decode_image(image):\n    image = tf.image.decode_jpeg(image, channels=3)\n    image = (tf.cast(image, tf.float32) / 127.5) - 1\n    image = tf.reshape(image, [*IMAGE_SIZE])\n    return image\n\ndef read_tfrecord(example):\n    tfrecord_format = {\n        \"image_name\": tf.io.FixedLenFeature([], tf.string),\n        \"image\": tf.io.FixedLenFeature([], tf.string),\n        \"target\": tf.io.FixedLenFeature([], tf.string)\n    }\n    example = tf.io.parse_single_example(example, tfrecord_format)\n    image = decode_image(example['image'])\n    return image\n\ndef load_dataset(filenames, labeled=True, ordered=False):\n    dataset = tf.data.TFRecordDataset(filenames)\n    dataset = dataset.map(read_tfrecord, num_parallel_calls=tf.data.AUTOTUNE)\n    return dataset\n\nmonet_ds = load_dataset(MONET_FILENAMES, labeled=True).batch(BATCH_SIZE)\nphoto_ds = load_dataset(PHOTO_FILENAMES, labeled=True).batch(BATCH_SIZE)","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:05:50.405740Z","iopub.execute_input":"2023-03-20T19:05:50.406893Z","iopub.status.idle":"2023-03-20T19:05:52.998747Z","shell.execute_reply.started":"2023-03-20T19:05:50.406848Z","shell.execute_reply":"2023-03-20T19:05:52.997736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"example_monet = next(iter(monet_ds))\nexample_photo = next(iter(photo_ds))\n\nplt.subplot(121)\nplt.title('Photo')\nplt.imshow(example_photo[0] * 0.5 + 0.5)\n\nplt.subplot(122)\nplt.title('Monet')\nplt.imshow(example_monet[0] * 0.5 + 0.5)","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:05:53.000443Z","iopub.execute_input":"2023-03-20T19:05:53.000830Z","iopub.status.idle":"2023-03-20T19:05:57.032194Z","shell.execute_reply.started":"2023-03-20T19:05:53.000789Z","shell.execute_reply":"2023-03-20T19:05:57.031315Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Basic EDA\n\nThere are **300** paintings of monet provided, along with over **7000** photos to convert.\n\nIn the painting, I see more **abstract figures** instead of smooth and sharp edges, plus there's many **detailed contrasts** too. Model layers may need a larger kernel for these paintings.\n\nAlso, the **texture and base color palletes** are not the same, so the generator model would have to learn those features also.\n\nMost of them are paintings of nature landscape, and I think that it may be the reason that the photos provided are mostly about landscapes.\n\n## Defining model architecture\n\nThe CycleGAN architecture is explained in depth in [this blog post](https://hardikbansal.github.io/CycleGANBlog/).\n\nFirst, there will be **two generators** that converts photo to monet, and vice versa. There will be **two discriminators** also, that discriminates generated photos and generated monets.\n\nThe **adversarial loss** is gained by discriminating on generated images, just like in DCGAN. And in CycleGAN, there are two more losses introduced.\n\nThe first one is the **cycle-consistency loss**, which ensures that the generators are consistent and invertible, and this loss is caculated by reconstructing images through two separate generators.\n\nThe second one is the **identity loss**, which is used to ensure that the generator does not change the style or contents of the image unnecessarily.\n\nBy adding all these three losses, the final loss is gained, then passed onto the gradient calculation section.\n\n## Building blocks for the models","metadata":{}},{"cell_type":"code","source":"kernel_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\ngamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\ndef downsample(\n    x,\n    filters,\n    activation,\n    kernel_size=(4, 4),\n    strides=(2, 2),\n    padding=\"same\",\n    kernel_initializer=kernel_init,\n    gamma_initializer=gamma_init,\n    use_bias=False,\n):\n    x = layers.Conv2D(\n        filters,\n        kernel_size,\n        strides=strides,\n        kernel_initializer=kernel_initializer,\n        padding=padding,\n        use_bias=use_bias,\n    )(x)\n    x = tfa.layers.InstanceNormalization(gamma_initializer=gamma_initializer)(x)\n    if activation:\n        x = activation(x)\n    return x\n\ndef upsample(\n    x,\n    filters,\n    activation,\n    kernel_size=(4, 4),\n    strides=(2, 2),\n    padding=\"same\",\n    kernel_initializer=kernel_init,\n    gamma_initializer=gamma_init,\n    use_bias=False,\n):\n    x = layers.Conv2DTranspose(\n        filters,\n        kernel_size,\n        strides=strides,\n        padding=padding,\n        kernel_initializer=kernel_initializer,\n        use_bias=use_bias,\n    )(x)\n    x = tfa.layers.InstanceNormalization(gamma_initializer=gamma_initializer)(x)\n    if activation:\n        x = activation(x)\n    return x","metadata":{"_kg_hide-output":false,"execution":{"iopub.status.busy":"2023-03-20T19:05:57.033333Z","iopub.execute_input":"2023-03-20T19:05:57.033788Z","iopub.status.idle":"2023-03-20T19:05:57.047881Z","shell.execute_reply.started":"2023-03-20T19:05:57.033751Z","shell.execute_reply":"2023-03-20T19:05:57.045690Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Building the generators","metadata":{}},{"cell_type":"code","source":"def get_generator(\n    filters=64,\n    num_downsampling_blocks=3,\n    num_upsample_blocks=3,\n    kernel_initializer=kernel_init,\n    gamma_initializer=gamma_init\n):\n    img_input = layers.Input(shape=IMAGE_SIZE)\n    \n    x = layers.Conv2D(filters, (6, 6), padding='same', kernel_initializer=kernel_initializer, use_bias=False)(img_input)\n    x = tfa.layers.InstanceNormalization(gamma_initializer=gamma_initializer)(x)\n    x = layers.Activation(\"relu\")(x)\n    \n    # Downsampling blocks x 3\n    for _ in range(num_downsampling_blocks):\n        filters *= 2\n        x = downsample(x, filters=filters, activation=layers.Activation(\"relu\"))\n        \n    # Upsampling blocks x 3\n    for _ in range(num_upsample_blocks):\n        filters //= 2\n        x = upsample(x, filters, activation=layers.Activation(\"relu\"))\n    \n    # Final block\n    x = layers.Conv2D(3, (6, 6), padding='same', use_bias=False)(x)\n    x = layers.Activation(\"tanh\")(x)\n    \n    model = keras.models.Model(img_input, x)\n    \n    return model","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:05:57.050616Z","iopub.execute_input":"2023-03-20T19:05:57.051893Z","iopub.status.idle":"2023-03-20T19:05:57.065255Z","shell.execute_reply.started":"2023-03-20T19:05:57.051854Z","shell.execute_reply":"2023-03-20T19:05:57.064611Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Building the discriminators","metadata":{}},{"cell_type":"code","source":"def get_discriminator(\n    filters=64,\n    num_downsampling=3,\n    kernel_initializer=keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n):\n    img_input = layers.Input(shape=IMAGE_SIZE)\n    \n    x = layers.Conv2D(filters, (4, 4), strides=(2, 2), padding=\"same\", kernel_initializer=kernel_initializer)(img_input)\n    x = layers.LeakyReLU(0.2)(x)\n\n    # Downsampling blocks x 3\n    num_filters = filters\n    for num_downsample_block in range(3):\n        num_filters *= 2\n        if num_downsample_block < 2:\n            x = downsample(\n                x, \n                filters=num_filters,\n                activation=layers.LeakyReLU(0.2),\n                kernel_size=(4, 4),\n                strides=(2, 2),\n            )\n        else:\n            x = downsample(\n                x,\n                filters=num_filters,\n                activation=layers.LeakyReLU(0.2),\n                kernel_size=(4, 4),\n                strides=(1, 1),\n            )\n\n    # Final block\n    x = layers.Conv2D(1, (4, 4), strides=(1, 1), padding=\"same\")(x)\n\n    model = keras.models.Model(inputs=img_input, outputs=x)\n    \n    return model","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:05:57.066594Z","iopub.execute_input":"2023-03-20T19:05:57.067591Z","iopub.status.idle":"2023-03-20T19:05:57.079772Z","shell.execute_reply.started":"2023-03-20T19:05:57.067553Z","shell.execute_reply":"2023-03-20T19:05:57.078868Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"gen_monet = get_generator()\ngen_photo = get_generator()\n\ndisc_monet = get_discriminator()\ndisc_photo = get_discriminator()","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:05:57.082088Z","iopub.execute_input":"2023-03-20T19:05:57.082402Z","iopub.status.idle":"2023-03-20T19:05:58.328843Z","shell.execute_reply.started":"2023-03-20T19:05:57.082351Z","shell.execute_reply":"2023-03-20T19:05:58.327770Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualizing the models","metadata":{}},{"cell_type":"code","source":"visualkeras.layered_view(gen_monet, legend=True, scale_xy=0.5)","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:05:58.332721Z","iopub.execute_input":"2023-03-20T19:05:58.333039Z","iopub.status.idle":"2023-03-20T19:05:58.364927Z","shell.execute_reply.started":"2023-03-20T19:05:58.333010Z","shell.execute_reply":"2023-03-20T19:05:58.363816Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"visualkeras.layered_view(disc_monet, legend=True, scale_xy=0.5)","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:05:58.366636Z","iopub.execute_input":"2023-03-20T19:05:58.367082Z","iopub.status.idle":"2023-03-20T19:05:58.388298Z","shell.execute_reply.started":"2023-03-20T19:05:58.367033Z","shell.execute_reply":"2023-03-20T19:05:58.386498Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Examining the visualizations of the models\n\nThe generator downsamples the data using **Conv2D**, and upsamples the data again with **Conv2DTranspose** like an autoencoder, hopefully capturing major image features in the process.\n\nThere are multiple **InstanceNormalization** layers used, because images in this problem should be considered on a per image basis.\n\nThe final output dimension of the generator is same as the input, **(BATCH_SIZE * 256 * 256 * 3)**. This will allow generator to accept raw images.\n\nThe discriminator downsamples the data, and reduces the dimension to a single unit with the last Conv2D layer.","metadata":{}},{"cell_type":"markdown","source":"## Building the CycleGAN model","metadata":{}},{"cell_type":"code","source":"class CycleGAN(keras.Model):\n    def __init__(\n        self,\n        gen_monet,\n        gen_photo,\n        disc_monet,\n        disc_photo,\n        lambda_cycle=10.0,\n        lambda_identity=0.5,\n    ):\n        super().__init__()\n        self.gen_monet = gen_monet\n        self.gen_photo = gen_photo\n        self.disc_monet = disc_monet\n        self.disc_photo = disc_photo\n        self.lambda_cycle = lambda_cycle\n        self.lambda_identity = lambda_identity\n\n    def compile(\n        self,\n        gen_monet_opt,\n        gen_photo_opt,\n        disc_monet_opt,\n        disc_photo_opt,\n        gen_loss_fn,\n        disc_loss_fn,\n    ):\n        super().compile()\n        self.gen_monet_opt = gen_monet_opt\n        self.gen_photo_opt = gen_photo_opt\n        self.disc_monet_opt = disc_monet_opt\n        self.disc_photo_opt = disc_photo_opt\n        self.generator_loss_fn = gen_loss_fn\n        self.discriminator_loss_fn = disc_loss_fn\n        self.cycle_loss_fn = keras.losses.MeanAbsoluteError()\n        self.identity_loss_fn = keras.losses.MeanAbsoluteError()\n\n    def train_step(self, batch_data):\n        \n        real_monet, real_photo = batch_data\n\n        with tf.GradientTape(persistent=True) as tape:\n            # Real photo to fake monet \n            fake_monet = self.gen_monet(real_photo, training=True)\n            # Real monet to fake photo\n            fake_photo = self.gen_photo(real_monet, training=True)\n\n            # Cycle (Zebra to fake horse to fake zebra) y -> x -> y\n            cycled_monet = self.gen_monet(fake_photo, training=True)\n            # Cycle (Horse to fake zebra to fake horse) x -> y -> x\n            cycled_photo = self.gen_photo(fake_monet, training=True)\n\n            # Identity mapping\n            same_monet = self.gen_monet(real_monet, training=True)\n            same_photo = self.gen_photo(real_photo, training=True)\n\n            # Discriminator output\n            disc_real_monet = self.disc_monet(real_monet, training=True)\n            disc_fake_monet = self.disc_monet(fake_monet, training=True)\n\n            disc_real_photo = self.disc_photo(real_photo, training=True)\n            disc_fake_photo = self.disc_photo(fake_photo, training=True)\n\n            # Generator adverserial loss\n            gen_monet_loss = self.generator_loss_fn(disc_fake_monet)\n            gen_photo_loss = self.generator_loss_fn(disc_fake_photo)\n\n            # Generator cycle loss\n            cycle_loss_monet = self.cycle_loss_fn(real_monet, cycled_monet) * self.lambda_cycle\n            cycle_loss_photo = self.cycle_loss_fn(real_photo, cycled_photo) * self.lambda_cycle\n\n            # Generator identity loss\n            id_loss_monet = (\n                self.identity_loss_fn(real_monet, same_monet)\n                * self.lambda_cycle\n                * self.lambda_identity\n            )\n            id_loss_photo = (\n                self.identity_loss_fn(real_photo, same_photo)\n                * self.lambda_cycle\n                * self.lambda_identity\n            )\n\n            # Total generator loss\n            total_loss_monet = gen_monet_loss + cycle_loss_monet + id_loss_monet\n            total_loss_photo = gen_photo_loss + cycle_loss_photo + id_loss_photo\n\n            # Discriminator loss\n            disc_monet_loss = self.discriminator_loss_fn(disc_real_monet, disc_fake_monet)\n            disc_photo_loss = self.discriminator_loss_fn(disc_real_photo, disc_fake_photo)\n\n        # Get the gradients for the generators\n        grads_monet = tape.gradient(total_loss_monet, self.gen_monet.trainable_variables)\n        grads_photo = tape.gradient(total_loss_photo, self.gen_photo.trainable_variables)\n\n        # Get the gradients for the discriminators\n        disc_monet_grads = tape.gradient(disc_monet_loss, self.disc_monet.trainable_variables)\n        disc_photo_grads = tape.gradient(disc_photo_loss, self.disc_photo.trainable_variables)\n\n        # Update the weights of the generators\n        self.gen_monet_opt.apply_gradients(\n            zip(grads_monet, self.gen_monet.trainable_variables)\n        )\n        self.gen_photo_opt.apply_gradients(\n            zip(grads_photo, self.gen_photo.trainable_variables)\n        )\n\n        # Update the weights of the discriminators\n        self.disc_monet_opt.apply_gradients(\n            zip(disc_monet_grads, self.disc_monet.trainable_variables)\n        )\n        self.disc_photo_opt.apply_gradients(\n            zip(disc_photo_grads, self.disc_photo.trainable_variables)\n        )\n\n        return {\n            \"monet_loss\": total_loss_monet,\n            \"photo_loss\": total_loss_photo,\n            \"disc_monet_loss\": disc_monet_loss,\n            \"disc_photo_loss\": disc_photo_loss,\n        }","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:05:58.389836Z","iopub.execute_input":"2023-03-20T19:05:58.390369Z","iopub.status.idle":"2023-03-20T19:05:58.408039Z","shell.execute_reply.started":"2023-03-20T19:05:58.390323Z","shell.execute_reply":"2023-03-20T19:05:58.406997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Defining base loss functions","metadata":{}},{"cell_type":"code","source":"# Loss function for evaluating adversarial loss\nadv_loss_fn = keras.losses.MeanSquaredError()\n\n# Define the loss function for the generators\ndef generator_loss_fn(fake):\n    fake_loss = adv_loss_fn(tf.ones_like(fake), fake)\n    return fake_loss\n\n# Define the loss function for the discriminators\ndef discriminator_loss_fn(real, fake):\n    real_loss = adv_loss_fn(tf.ones_like(real), real)\n    fake_loss = adv_loss_fn(tf.zeros_like(fake), fake)\n    return (real_loss + fake_loss) * 0.5","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:05:58.409457Z","iopub.execute_input":"2023-03-20T19:05:58.409989Z","iopub.status.idle":"2023-03-20T19:05:58.420653Z","shell.execute_reply.started":"2023-03-20T19:05:58.409954Z","shell.execute_reply":"2023-03-20T19:05:58.419612Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Compiling & training the CycleGAN model","metadata":{}},{"cell_type":"code","source":"EPOCH_COUNT = 100","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:05:58.422018Z","iopub.execute_input":"2023-03-20T19:05:58.422617Z","iopub.status.idle":"2023-03-20T19:05:58.433836Z","shell.execute_reply.started":"2023-03-20T19:05:58.422581Z","shell.execute_reply":"2023-03-20T19:05:58.432066Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_gan = CycleGAN(\n    gen_monet=gen_monet, gen_photo=gen_photo, disc_monet=disc_monet, disc_photo=disc_photo\n)\n\nmodel_gan.compile(\n    gen_monet_opt=keras.optimizers.Adam(learning_rate=2e-4, beta_1=0.5),\n    gen_photo_opt=keras.optimizers.Adam(learning_rate=2e-4, beta_1=0.5),\n    disc_monet_opt=keras.optimizers.Adam(learning_rate=1e-4, beta_1=0.5),\n    disc_photo_opt=keras.optimizers.Adam(learning_rate=1e-4, beta_1=0.5),\n    gen_loss_fn=generator_loss_fn,\n    disc_loss_fn=discriminator_loss_fn,\n)\n\nhist = model_gan.fit(\n    tf.data.Dataset.zip((monet_ds, photo_ds)),\n    epochs=EPOCH_COUNT,\n    verbose=0\n)","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:05:58.436243Z","iopub.execute_input":"2023-03-20T19:05:58.437296Z","iopub.status.idle":"2023-03-20T19:10:11.851496Z","shell.execute_reply.started":"2023-03-20T19:05:58.437255Z","shell.execute_reply":"2023-03-20T19:10:11.850510Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Evaluating the CycleGAN model","metadata":{}},{"cell_type":"code","source":"plt.plot(hist.history[\"monet_loss\"])\nplt.plot(hist.history['photo_loss'])\nplt.plot(hist.history['disc_monet_loss'])\nplt.plot(hist.history['disc_photo_loss'])\n\nplt.title(\"Model Evaluation\")\nplt.ylabel(\"Loss\")\nplt.xlabel(\"Epoch\")\nplt.legend([\"Generator (Photo -> Monet)\", \"Generator (Monet -> Photo)\", \"Monet Discriminator\", \"Photo Discriminator\"])\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:10:11.853597Z","iopub.execute_input":"2023-03-20T19:10:11.854321Z","iopub.status.idle":"2023-03-20T19:10:12.089496Z","shell.execute_reply.started":"2023-03-20T19:10:11.854280Z","shell.execute_reply":"2023-03-20T19:10:12.088426Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Examining loss plot of the model\n\nThe slope oscillates a bit, but the overall trend is consistent. The orange loss of the generator model that converts **paintings into photos** had a larger loss compared to the blue loss of the generator that converts **photos into paintings**, and this may either imply that the latter conversion is easier, or the generator model's architecture is more suited for the latter translation.","metadata":{}},{"cell_type":"code","source":"rows, cols = 2, 4\n\nfig, axes = plt.subplots(rows, cols, figsize=(12, 6))\n\nfor i, photo in enumerate(photo_ds.take(4)):\n    monet = gen_monet(photo, training=False)[0].numpy()\n    monet = (monet * 127.5 + 127.5).astype(np.uint8)\n    photo = (photo[0] * 127.5 + 127.5).numpy().astype(np.uint8)\n    \n    row, col = i // 2, i % 2\n\n    axes[row, 2 * col].imshow(photo)\n    axes[row, 2 * col].axis('off')\n    axes[row, 2 * col].set_title(\"Photo\")\n    axes[row, 2 * col + 1].imshow(monet)\n    axes[row, 2 * col + 1].axis('off')\n    axes[row, 2 * col + 1].set_title(\"Monet-styled\")\n    \nplt.subplots_adjust(wspace = 0.05, hspace = 0.3)\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:10:12.091236Z","iopub.execute_input":"2023-03-20T19:10:12.091991Z","iopub.status.idle":"2023-03-20T19:10:14.990124Z","shell.execute_reply.started":"2023-03-20T19:10:12.091952Z","shell.execute_reply":"2023-03-20T19:10:14.987375Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Examining translated images\n\nThe **colors and the contrasts** of generated images resemble the ones of Monet, and the **texture** of the image also feels like a canvas oil paint.\n\nIt looks like the generator models succeeded in capturing major features! Yay!\n\n## Conclusion\n\n**CycleGAN** showed sufficient power in translating images between domains.\n\nAlso, the model showed slow, but **more consistent learning speed** compared to traditional GANs, having three different losses being calculated. The more I trained, the better the output became. There was no large-scale oscillations of the generator and discriminator fighting over each other, but the losses still decreased for both models quite consistently.\n\nI am very happy with the results, and plan to use the model architecture for future projects also.\n\n## Submission","metadata":{}},{"cell_type":"code","source":"import PIL\n! mkdir ../images\n\ni = 1\nfor img in photo_ds:\n    prediction = gen_monet(img, training=False)[0].numpy()\n    prediction = (prediction * 127.5 + 127.5).astype(np.uint8)\n    im = PIL.Image.fromarray(prediction)\n    im.save(\"../images/\" + str(i) + \".jpg\")\n    i += 1\n    \nimport shutil\nshutil.make_archive(\"/kaggle/working/images\", 'zip', \"/kaggle/images\")","metadata":{"execution":{"iopub.status.busy":"2023-03-20T19:10:14.991853Z","iopub.execute_input":"2023-03-20T19:10:14.992702Z"},"trusted":true},"execution_count":null,"outputs":[]}]}