{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.12.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceType":"competition","sourceId":21755,"databundleVersionId":1475600}],"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## CSCA5642:Generative Adversarial Network to create Monet Artworks ##\n### This project implements an Generative Adversial Network to transform real-world photos into Monet-style painting. ###\n\n* Author: Alexander Meau  \n* Email: alme9155@colorado.edu \n* GitHub: [https://github.com/alme9155/csca-5642-week5/tree/main](https://github.com/alme9155/csca-5642-week5/tree/main)  ","metadata":{}},{"cell_type":"markdown","source":"## I. Brief Description of the Project and Data ##\n\nThis project aims to create a generative adversarial network (GAN) that generates images resembling Monet's paintings.\n- **The requirement of this competition is to apply a Generative Adversarial Network (GAN) to generate artwork resembling Monet's paintings.**\n- **A Generative Adversarial Network (GAN) is a class of machine learning models consisting of a generator and a discriminator trained to optimize the generation of realistic outputs.**\n- Submission will be evaluated by inception score MiFID(Memorization-informed Fréchet Inception Distance)\n- MiFID score is a modification of Fréchet Inception Distance (FID).\n- The smaller the MiFID value, the better the generated images will be. \n\n### Dataset: ####\n* The dataset contains 4 data directories: \"monet_tfrec\", \"photo_tfrec\", \"monet_jpg\", \"photo_jpg\"\n* The monet_tfrec and monet_jpg directories contain the same painting images\n* The photo_tfrec and photo_jpg directories contain the same photos.\n\n|Directory        | Dimension      | Size      | Structure   |\n|:----------------|----------------|:----------|:-----------:|\n| monet_tfrec | 300   | 256x256 | JPEG |\n| photo_tfrec | 300   | 256x256 | JPEG |\n| monet_jpg   | 7028  | 256x256 | JPEG |\n| photo_jpg   | 7028  | 256x256 | JPEG |\n\n\n### Competition Rules ###\n* Create an Images.zip that contains 7,000-10,000 images sized 256x256.","metadata":{}},{"cell_type":"markdown","source":"## II. Exploratory Data Analysis (EDA) ##\n- Review the total number of monet paintings in tref format (File Count)\n- Review the total number of photos in tfref format (File Count)\n- Inspect image dimension (width x height) of monets paintings\n- Inspect image dimension (width x height) of photos\n- Display sample monet paints and photos","metadata":{}},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under t 4ehe input directory\n\nimport os\n\njpg_file_count =0\ntfrec_file_count = 0\n\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    print(f\"[{dirname}]\")\n\n    for filename in filenames:\n        if filename.endswith('.jpg'):\n            jpg_file_count +=1\n            if jpg_file_count < 3:\n                print(\"\\t\"+os.path.join(dirname, filename))\n            elif jpg_file_count == 3:\n                print(\"\\t...\")\n        elif filename.endswith('.tfrec'):\n            tfrec_file_count +=1\n            if tfrec_file_count < 3:\n                print(\"\\t\"+os.path.join(dirname, filename))\n            elif tfrec_file_count == 3:\n                print(\"\\t...\")\n        else:        \n            print(os.path.join(dirname, filename))\n    if jpg_file_count > 0:\n        print(f\"\\n\\tTotal number of .jpg files in {dirname}: {jpg_file_count}\")\n        jpg_file_count =0\n    if tfrec_file_count > 0:\n        print(f\"\\n\\tTotal number of .tfrec files in {dirname}: {tfrec_file_count}\")\n        tfrec_file_count =0\n        \n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-19T07:11:36.887399Z","iopub.execute_input":"2026-02-19T07:11:36.887652Z","iopub.status.idle":"2026-02-19T07:11:52.127645Z","shell.execute_reply.started":"2026-02-19T07:11:36.887617Z","shell.execute_reply":"2026-02-19T07:11:52.126950Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Display sample files\nimport os\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport random\nimport cv2\nfrom kaggle_datasets import KaggleDatasets\n\nINPUT_DIR = \"/kaggle/input/gan-getting-started\"\nmonet_jpg_dir = os.path.join(INPUT_DIR, 'monet_jpg')\nmonet_tfrec_dir = os.path.join(INPUT_DIR, 'monet_tfrec')\nphoto_jpg_dir = os.path.join(INPUT_DIR, 'photo_jpg')\nphoto_tfrec_dir = os.path.join(INPUT_DIR, 'photo_tfrec')\n\n# Show five random photo images\ndef show_random_photos(folder, label, n=5):\n    sample_files = random.sample(os.listdir(folder), n)\n    plt.figure(figsize=(15, 3))\n    for i, file in enumerate(sample_files):\n        img = cv2.imread(os.path.join(folder, file))\n        img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n        plt.subplot(1, n, i+1)\n        plt.imshow(img)\n        plt.axis('off')\n        plt.title(f\"{label} {i+1}\")\n    plt.tight_layout()\n    plt.show()\n\n\n# Run both\nprint(\"Showing 5 random Photo images:\")\nshow_random_photos(photo_jpg_dir, 'Photo', n=5)\nprint(\"Showing 5 random Monet paintings:\")\nshow_random_photos(monet_jpg_dir, 'Monet Painting', n=5)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-19T07:12:16.040503Z","iopub.execute_input":"2026-02-19T07:12:16.041132Z","iopub.status.idle":"2026-02-19T07:12:17.414105Z","shell.execute_reply.started":"2026-02-19T07:12:16.041102Z","shell.execute_reply":"2026-02-19T07:12:17.413335Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# examine images in data directories\nimport os\nimport cv2\nfrom PIL import Image\n\n# data directory\nINPUT_DIR = \"/kaggle/input/gan-getting-started\"\nmonet_jpg_dir = os.path.join(INPUT_DIR, 'monet_jpg')\nmonet_tfrec_dir = os.path.join(INPUT_DIR, 'monet_tfrec')\nphoto_jpg_dir = os.path.join(INPUT_DIR, 'photo_jpg')\nphoto_tfrec_dir = os.path.join(INPUT_DIR, 'photo_tfrec')\n\nEXPECTED_SIZE = (256, 256)\nEXPECTED_MODE = \"RGB\"  \nPIXEL_RANGE = (-1.0, 1.0)\n\ndef examine_files(image_dir):\n    for filename in os.listdir(image_dir):\n        if filename.lower().endswith(('.jpg', '.jpeg', '.png')):\n            img_path = os.path.join(image_dir, filename)\n            with Image.open(img_path) as img:\n                # Check size and mode\n                if img.size != EXPECTED_SIZE or img.mode != EXPECTED_MODE:\n                    print(\"Needs resizing and preprocessing:\")\n                    print(f\"File: {filename}\")\n                    print(f\"Size: {img.size}, Mode: {img.mode}\")\n                    break\n                    \n                image_np = np.asarray(img).astype(np.float32)\n                image_np = (image_np / 127.5) - 1.0 \n                \n                # Check pixel range\n                if not (np.min(image_np) >= -1.0 and np.max(image_np) <= 1.0):\n                    print(\"Pixel values in {filename} out of range [-1, 1]:\")\n                    print(f\"File: {filename}\")\n                    print(f\"Pixel min: {np.min(image_np)}, max: {np.max(image_np)}\")\n                    break\n    else:\n        print(f\"All images in {image_dir} are (Dimension) 256×256, (Channel) RGB, and pixel range to [-1, 1]\")\n        print(f\"No further data pre-processing is required for {image_dir}\")\n\nexamine_files(monet_jpg_dir)\nexamine_files(photo_jpg_dir)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-19T07:12:21.567309Z","iopub.execute_input":"2026-02-19T07:12:21.568129Z","iopub.status.idle":"2026-02-19T07:13:20.009856Z","shell.execute_reply.started":"2026-02-19T07:12:21.568099Z","shell.execute_reply":"2026-02-19T07:13:20.009099Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import os\nimport random\nimport numpy as np\nfrom PIL import Image\nimport matplotlib.pyplot as plt\n\ndef RGB_histogram(img_path, title):\n    \"\"\"Display RGB histogram for a given image.\"\"\"\n    img = Image.open(img_path).convert('RGB')  # Ensure 3-channel RGB\n    img_np = np.array(img)\n    \n    plt.figure(figsize=(6, 3))\n    for i, color in enumerate(['r', 'g', 'b']):\n        plt.hist(img_np[:, :, i].ravel(), bins=256, color=color, alpha=0.5, label=color)\n    \n    plt.legend()\n    plt.title(title)\n    plt.tight_layout()\n    plt.show()\n\n# randomly pick a file for inspection.\ndef get_random_image_path(directory):\n    image_files = [f for f in os.listdir(directory) if f.lower().endswith(('.jpg', '.jpeg', '.png'))]\n    if not image_files:\n        raise FileNotFoundError(f\"No image files found in {directory}\")\n    return os.path.join(directory, random.choice(image_files))\n\n# Example directories\nINPUT_DIR = \"/kaggle/input/gan-getting-started\"\nmonet_jpg_dir = os.path.join(INPUT_DIR, 'monet_jpg')\nphoto_jpg_dir = os.path.join(INPUT_DIR, 'photo_jpg')\nrandom_monet = get_random_image_path(monet_jpg_dir)\nrandom_photo = get_random_image_path(photo_jpg_dir)\n\nRGB_histogram(random_monet, \"Random Monet Image RGB Histogram\")\nRGB_histogram(random_photo, \"Random Photo Image RGB Histogram\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-19T07:13:41.208431Z","iopub.execute_input":"2026-02-19T07:13:41.209134Z","iopub.status.idle":"2026-02-19T07:13:43.156849Z","shell.execute_reply.started":"2026-02-19T07:13:41.209106Z","shell.execute_reply":"2026-02-19T07:13:43.156135Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Brief Conclusion of EDA ###\n\n* All image files in the corresponding image directory are already normalized to the required output dimensions 256x256x3 channels.\n* No further pre-processing is required before applying an adversarial generative neural network.\n  - [ monet_tfrec ] : 300 Monet paintings sized 256x256 with RGB channels in JPEG format\n  - [ photo_tfrec ]: 300 Monet paintings sized 256x256 with RGB channels in TFRecord format\n  - [ monet_jpg ] : 7028 photos sized 256x256 with RGB channels in JPEG format\n  - [ photo_jpg ] : 7028 photos sized 256x256 with RGB channels in TFRecord format","metadata":{}},{"cell_type":"markdown","source":"## III. Generative Neural Network Architecture: CycleGAN ##\n\n### Model Description ###\n- Following the tutorial on the Kaggle completion, I have decided to use the CycleGAN architecture for this project.\n- CycleGAN is a dual DCGAN design (Deep Convolution GAN) to deploy two generators and two discriminators.\n- This bi-directional setup will enable translation between domains: A-> B and B -> A.\n- - Key Components of CycleGAN are to deploy two generators and two discriminators in the setup.\n  - Two Generators:\n      - G: X -> Y (e.g., Photo → Monet)\n      - F: Y -> X (e.g., Monet → Photo)\n  - Two Discriminators:\n      - D_Y distinguish real Y from fake Y (G(X)) (e.g. real Monet)\n      - D_X distinguish real X from fake X (F(Y)) (e.g. real Photo)\n  - By applying a cycle-consistency constraint, it ensures the generated image not only carries Monet's impressionistic style but also looks realistic.\n  - Otherwise, the generated image of a zebra might have an extra limb\n\nRef: [https://www.kaggle.com/code/amyjang/monet-cyclegan-tutorial/notebook](https://www.kaggle.com/code/amyjang/monet-cyclegan-tutorial/notebook)","metadata":{}},{"cell_type":"code","source":"# CycleGAN class definition\nimport tensorflow as tf\nfrom tensorflow.keras import layers, optimizers\nimport time\n\nclass CycleGAN(tf.keras.Model):\n    def __init__(self, img_size=256, lambda_cycle=10):\n        super(CycleGAN, self).__init__()\n        self.img_size = img_size\n        self.lambda_cycle = lambda_cycle\n        self.loss_obj = tf.keras.losses.BinaryCrossentropy(from_logits=True)     \n        self.build_models()\n        self.gen_learning_rate=2e-4\n        self.disc_learning_rate=2e-4\n        print('CycleGAN::Init() complete.')\n\n    def residual_block(self, filters):\n        return [\n            layers.Conv2D(filters, kernel_size=3, padding='same', use_bias=False),\n            layers.BatchNormalization(),\n            layers.ReLU(),\n            layers.Conv2D(filters, kernel_size=3, padding='same', use_bias=False),\n            layers.BatchNormalization()\n        ]\n    \n    def build_generator(self):\n        # kernel size: 256 x 256 x RGB channel\n        inputs = tf.keras.layers.Input(shape=[self.img_size, self.img_size, 3])\n        x = inputs\n\n        # down stack\n        for layer in [\n            layers.Conv2D(64, 4, strides=2, padding='same', use_bias=False),\n            layers.LeakyReLU(),\n            layers.Conv2D(128, 4, strides=2, padding='same', use_bias=False),\n            layers.BatchNormalization(),\n            layers.LeakyReLU(),\n            layers.Conv2D(256, 4, strides=2, padding='same', use_bias=False),\n            layers.BatchNormalization(),\n            layers.LeakyReLU(),\n        ]:\n            x = layer(x)\n\n        # residual block\n        for _ in range(6):\n            res = x\n            for layer in self.residual_block(256):\n                res = layer(res)\n            x = layers.Add()([x, res])\n\n        # up stack\n        for layer in [\n            layers.Conv2DTranspose(128, 4, strides=2, padding='same', use_bias=False),\n            layers.BatchNormalization(),\n            layers.ReLU(),\n            layers.Conv2DTranspose(64, 4, strides=2, padding='same', use_bias=False),\n            layers.BatchNormalization(),\n            layers.ReLU(),\n        ]:\n            x = layer(x)\n\n        #last output\n        x = tf.keras.layers.Conv2DTranspose(3, 4, strides=2, padding='same', activation='tanh')(x)\n        return tf.keras.Model(inputs=inputs, outputs=x)\n\n    def build_discriminator(self):\n        # discriminator only has down stack with final output\n        inputs = layers.Input(shape=[self.img_size, self.img_size, 3])\n        x = inputs\n\n        # down stack\n        for layer in [\n            layers.Conv2D(64, 4, strides=2, padding='same', use_bias=False),\n            layers.LeakyReLU(),\n            layers.Conv2D(128, 4, strides=2, padding='same', use_bias=False),\n            layers.BatchNormalization(),\n            layers.LeakyReLU(),\n            layers.Conv2D(256, 4, strides=2, padding='same', use_bias=False),\n            layers.BatchNormalization(),\n            layers.LeakyReLU(),\n            layers.Conv2D(512, 4, strides=2, padding='same', use_bias=False),\n            layers.BatchNormalization(),\n            layers.LeakyReLU(),\n            layers.Conv2D(1, 4, strides=1, padding='same') # Outputs BinaryCrossentropy(from_logits=True).\n        ]:\n            x = layer(x)            \n        return tf.keras.Model(inputs=inputs, outputs=x)\n\n    def build_models(self):\n        self.gen_photo_to_monet = self.build_generator()\n        self.gen_monet_to_photo = self.build_generator()\n        self.disc_photo = self.build_discriminator()\n        self.disc_monet = self.build_discriminator()\n\n    def generator_loss(self, discriminator_output):\n        return self.loss_obj(tf.ones_like(discriminator_output), discriminator_output)\n\n    def discriminator_loss(self, real_output, fake_output):\n        real_loss = self.loss_obj(tf.ones_like(real_output), real_output)\n        fake_loss = self.loss_obj(tf.zeros_like(fake_output), fake_output)\n        return (real_loss + fake_loss) * 0.5\n\n    def cycle_loss_fn(self, real_image, cycled_image, lambda_weight):\n        return lambda_weight * tf.reduce_mean(tf.abs(real_image - cycled_image))\n\n    def identity_loss_fn(self, real_image, same_image, lambda_weight):\n        return 0.5 * lambda_weight * tf.reduce_mean(tf.abs(real_image - same_image))    \n    \n    def compile(self, gen_learning_rate=None, disc_learning_rate=None):\n        super().compile()\n        gen_lr = gen_learning_rate if gen_learning_rate is not None else self.gen_learning_rate\n        disc_lr = disc_learning_rate if disc_learning_rate is not None else self.disc_learning_rate\n        self.gen_photo_to_monet_optimizer = optimizers.Adam(gen_lr, beta_1=0.5)\n        self.gen_monet_to_photo_optimizer = optimizers.Adam(gen_lr, beta_1=0.5)\n        self.disc_photo_optimizer = optimizers.Adam(disc_lr, beta_1=0.5)\n        self.disc_monet_optimizer = optimizers.Adam(disc_lr, beta_1=0.5)          \n        print('CycleGAN::compile() complete.')\n\n    @tf.function\n    def train_step(self, batch_data):\n        real_photo, real_monet = batch_data\n\n        with tf.GradientTape(persistent=True) as tape:\n            # Generate images\n            fake_monet = self.gen_photo_to_monet(real_photo, training=True)\n            fake_photo = self.gen_monet_to_photo(real_monet, training=True)\n            cycled_photo = self.gen_monet_to_photo(fake_monet, training=True)\n            cycled_monet = self.gen_photo_to_monet(fake_photo, training=True)\n            same_photo = self.gen_monet_to_photo(real_photo, training=True)\n            same_monet = self.gen_photo_to_monet(real_monet, training=True)\n\n            # Discriminator outputs\n            disc_real_photo = self.disc_photo(real_photo, training=True)\n            disc_fake_photo = self.disc_photo(fake_photo, training=True)\n            disc_real_monet = self.disc_monet(real_monet, training=True)\n            disc_fake_monet = self.disc_monet(fake_monet, training=True)\n\n            # Generator losses\n            gen_photo_to_monet_loss = self.generator_loss(disc_fake_monet)\n            gen_monet_to_photo_loss = self.generator_loss(disc_fake_photo)\n            cycle_loss = tf.reduce_mean(tf.abs(real_photo - cycled_photo)) + tf.reduce_mean(tf.abs(real_monet - cycled_monet))\n            identity_loss = tf.reduce_mean(tf.abs(real_photo - same_photo)) + tf.reduce_mean(tf.abs(real_monet - same_monet))\n\n            total_gen_photo_to_monet_loss = gen_photo_to_monet_loss + self.lambda_cycle * cycle_loss + 0.5 * self.lambda_cycle * identity_loss\n            total_gen_monet_to_photo_loss = gen_monet_to_photo_loss + self.lambda_cycle * cycle_loss + 0.5 * self.lambda_cycle * identity_loss\n\n            # Discriminator losses\n            disc_photo_loss = self.discriminator_loss(disc_real_photo, disc_fake_photo)\n            disc_monet_loss = self.discriminator_loss(disc_real_monet, disc_fake_monet)\n\n        # Gradients\n        grads_gen_photo_to_monet = tape.gradient(total_gen_photo_to_monet_loss, self.gen_photo_to_monet.trainable_variables)\n        grads_gen_monet_to_photo = tape.gradient(total_gen_monet_to_photo_loss, self.gen_monet_to_photo.trainable_variables)\n        grads_disc_photo = tape.gradient(disc_photo_loss, self.disc_photo.trainable_variables)\n        grads_disc_monet = tape.gradient(disc_monet_loss, self.disc_monet.trainable_variables)\n\n        # Clip gradients\n        grads_gen_photo_to_monet = [tf.clip_by_norm(g, 1.0) for g in grads_gen_photo_to_monet]\n        grads_gen_monet_to_photo = [tf.clip_by_norm(g, 1.0) for g in grads_gen_monet_to_photo]\n        grads_disc_photo = [tf.clip_by_norm(g, 1.0) for g in grads_disc_photo]\n        grads_disc_monet = [tf.clip_by_norm(g, 1.0) for g in grads_disc_monet]\n\n        # Apply gradients\n        self.gen_photo_to_monet_optimizer.apply_gradients(zip(grads_gen_photo_to_monet, self.gen_photo_to_monet.trainable_variables))\n        self.gen_monet_to_photo_optimizer.apply_gradients(zip(grads_gen_monet_to_photo, self.gen_monet_to_photo.trainable_variables))\n        self.disc_photo_optimizer.apply_gradients(zip(grads_disc_photo, self.disc_photo.trainable_variables))\n        self.disc_monet_optimizer.apply_gradients(zip(grads_disc_monet, self.disc_monet.trainable_variables))\n\n        del tape\n        return {\n            'photo_gen_loss': total_gen_photo_to_monet_loss,\n            'monet_gen_loss': total_gen_monet_to_photo_loss,\n            'photo_disc_loss': disc_photo_loss,\n            'monet_disc_loss': disc_monet_loss\n        }\n\n    def fit(self, photo_dataset, monet_dataset, epochs=20):\n        steps_per_epoch = sum(1 for _ in photo_dataset)\n        if steps_per_epoch == 0:\n            raise ValueError(\"photo_dataset is empty or has unknown size.\")\n\n        self.loss_history = {\n            'photo_gen_loss': [],\n            'monet_gen_loss': [],\n            'photo_disc_loss': [],\n            'monet_disc_loss': []\n        }\n        \n        for epoch in range(epochs):\n            print(f\"\\nEpoch {epoch+1}/{epochs}\")\n            progbar = tf.keras.utils.Progbar(steps_per_epoch, stateful_metrics=[\n                'photo_gen_loss', 'monet_gen_loss', 'photo_disc_loss', 'monet_disc_loss'\n            ])\n\n            epoch_losses = {\n                'photo_gen_loss': 0.0,\n                'monet_gen_loss': 0.0,\n                'photo_disc_loss': 0.0,\n                'monet_disc_loss': 0.0\n            }\n    \n            # Zip datasets and run training steps\n            for step, (image_photo, image_monet) in enumerate(tf.data.Dataset.zip((photo_dataset, monet_dataset))):\n                losses = self.train_step((image_photo, image_monet))\n\n                # Accumulate total\n                for k in epoch_losses:\n                    epoch_losses[k] += float(losses[k])\n                \n                progbar.update(step + 1, [\n                    ('photo_gen_loss', float(losses['photo_gen_loss'])),\n                    ('monet_gen_loss', float(losses['monet_gen_loss'])),\n                    ('photo_disc_loss', float(losses['photo_disc_loss'])),\n                    ('monet_disc_loss', float(losses['monet_disc_loss']))\n                ])\n                \n            for k in self.loss_history:\n                self.loss_history[k].append(epoch_losses[k] / steps_per_epoch)\n        print(\"CycleGAN::Training() complete.\")\nprint('CycleGAN Model Defined..')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-19T07:13:52.943445Z","iopub.execute_input":"2026-02-19T07:13:52.944108Z","iopub.status.idle":"2026-02-19T07:14:16.959754Z","shell.execute_reply.started":"2026-02-19T07:13:52.944079Z","shell.execute_reply":"2026-02-19T07:14:16.959097Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## IV. Model Training ##\n\n* Load Monet and Photo Dataset from jpg files.\n* Dataset has 7000 photo and 300 images in jpg format, but only 20 photos and 5 images in tensorflow native tfref format.\n* Train CycleGAN model with loaded jpg dataset\n \nRef: [https://www.kaggle.com/code/amyjang/monet-cyclegan-tutorial/notebook](https://www.kaggle.com/code/amyjang/monet-cyclegan-tutorial/notebook)","metadata":{}},{"cell_type":"code","source":"# load dataset\nimport tensorflow as tf\nfrom kaggle_datasets import KaggleDatasets\nimport os\nimport matplotlib.pyplot as plt\n\nIMAGE_SIZE = [256, 256]\nBATCH_SIZE = 1 #no batch normalization for CycleGAN\n\n# Get GCS path for TFRecord data\nGCS_PATH = KaggleDatasets().get_gcs_path('gan-getting-started')\n\nINPUT_DIR = \"/kaggle/input/gan-getting-started\"\nmonet_jpg_dir = os.path.join(INPUT_DIR, 'monet_jpg')\nphoto_jpg_dir = os.path.join(INPUT_DIR, 'photo_jpg')\n\ndef decode_jpg(file_path):\n    image = tf.io.read_file(file_path)\n    image = tf.image.decode_jpeg(image, channels=3)\n    image = tf.image.resize(image, IMAGE_SIZE)\n    image = (tf.cast(image, tf.float32) / 127.5) - 1 \n    return image\n\ndef load_jpg_dataset(image_dir):\n    jpg_files = tf.data.Dataset.list_files(os.path.join(image_dir, '*.jpg'), shuffle=True)\n    dataset = (jpg_files\n               .map(decode_jpg, num_parallel_calls=tf.data.AUTOTUNE)\n               .cache()\n               .shuffle(1024)\n               .batch(BATCH_SIZE)\n               .prefetch(tf.data.AUTOTUNE))\n    return dataset\n\ndef load_dataset(tfrecord_paths):\n    return (tf.data.TFRecordDataset(tfrecord_paths)\n            .map(read_tfrecord, num_parallel_calls=tf.data.AUTOTUNE)\n            .cache()\n            .shuffle(1024)\n            .batch(BATCH_SIZE)\n            .prefetch(tf.data.AUTOTUNE))\n\n\nmonet_dataset = load_jpg_dataset(monet_jpg_dir)\nphoto_dataset = load_jpg_dataset(photo_jpg_dir)\nmonet_file_count = len(os.listdir(monet_jpg_dir))\nphoto_file_count = len(os.listdir(photo_jpg_dir))\n\nprint(f\"Monet Dataset has {monet_file_count} files.\")\n# photo_dataset = load_dataset(photo_tfrec_path)\nprint(f\"Photo Dataset has {photo_file_count} files.\"  )\nprint('Monet and Photo Dataset loaded....')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-19T07:18:30.244280Z","iopub.execute_input":"2026-02-19T07:18:30.244925Z","iopub.status.idle":"2026-02-19T07:18:32.393239Z","shell.execute_reply.started":"2026-02-19T07:18:30.244890Z","shell.execute_reply":"2026-02-19T07:18:32.392526Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import tensorflow as tf\nimport os\nimport zipfile\nimport shutil\nimport numpy as np\nimport PIL.Image\nimport random\nimport matplotlib.pyplot as plt\n\nIMG_SIZE = 256\nEPOCHS = 25\nOUTPUT_DIR = \"../generated_images\"\n\nmodel = CycleGAN(img_size=IMG_SIZE)\nmodel.compile()\nhistory = model.fit(photo_dataset, monet_dataset, epochs=EPOCHS)\nprint(\"Model Training complete.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-19T07:18:40.106471Z","iopub.execute_input":"2026-02-19T07:18:40.106981Z","iopub.status.idle":"2026-02-19T07:36:39.494299Z","shell.execute_reply.started":"2026-02-19T07:18:40.106953Z","shell.execute_reply":"2026-02-19T07:36:39.493527Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\n# plot loss over training epoch\nplt.figure(figsize=(12, 6))\n\nfor k, v in model.loss_history.items():\n    plt.plot(v, label=k)\n\nplt.title(\"Loss Over Epochs\")\nplt.xlabel(\"Epoch\")\nplt.ylabel(\"Loss\")\nplt.legend()\nplt.grid(True)\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-19T07:36:50.917239Z","iopub.execute_input":"2026-02-19T07:36:50.917841Z","iopub.status.idle":"2026-02-19T07:36:51.077103Z","shell.execute_reply.started":"2026-02-19T07:36:50.917812Z","shell.execute_reply":"2026-02-19T07:36:51.076551Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## VI. Results and Analysis before Hyper-parameter Tuning ##\n\nOver the course of 25 training epochs, the training shows moderate fluctuation while using CycleGAN to convert photos into Monet-style paintings:\n### Generator Loss ###\n* Epoch 1-5: A sharp early drop in generator loss indicates successful initial learning. The generator is getting better at fooling the discriminator.\n* Epoch 6-25: Noticeable fluctuations are observed, with spikes occurring around Epochs 10, 15, and 20—suggesting instability in adversarial dynamics.\n\n### Discriminator Loss ###\n* The discriminator loss generally declines over time, with relatively minor spikes.\n* The lowest loss values appear around Epochs 8 and 9, indicating the discriminator's strongest performance during training.","metadata":{}},{"cell_type":"markdown","source":"## VII. Hyper-parameter Tuning with adjusting Lambda value. ###\n\n* CycleGAN use two GAN model photo -> Monet, and Monet -> Photo.\n* Lambda value in CycleGAN is a weighting factor that controls the range of cycle-consistency loss.\n\nThree lambdha values will be evaluted:\n- 5\n- 10\n- 15","metadata":{}},{"cell_type":"code","source":"### Sub-class with custom fit() method to examine lambda hyper-parameter.\nimport tensorflow as tf\nfrom tensorflow.keras import optimizers, utils\nimport time\n\n\nclass CustomCycleGANLambdaTuning(CycleGAN):\n    def fit(self, photo_dataset, monet_dataset, epochs=25):\n        # Determine steps per epoch safely\n        cardinality = tf.data.experimental.cardinality(photo_dataset)\n        if cardinality == tf.data.INFINITE_CARDINALITY or cardinality == tf.data.UNKNOWN_CARDINALITY:\n            steps_per_epoch = 300  # Fallback default value\n        else:\n            steps_per_epoch = cardinality.numpy()\n\n        loss_history = {\n            'photo_gen_loss': [],\n            'monet_gen_loss': [],\n            'photo_disc_loss': [],\n            'monet_disc_loss': []\n        }\n\n        for epoch in range(epochs):\n            print(f\"\\nEpoch {epoch + 1}/{epochs}\")\n            progbar = utils.Progbar(steps_per_epoch, stateful_metrics=[\n                'photo_gen_loss', 'monet_gen_loss', 'photo_disc_loss', 'monet_disc_loss'])\n\n            epoch_losses = {\n                'photo_gen_loss': 0.0,\n                'monet_gen_loss': 0.0,\n                'photo_disc_loss': 0.0,\n                'monet_disc_loss': 0.0\n            }\n\n            for step, (image_photo, image_monet) in enumerate(tf.data.Dataset.zip((photo_dataset, monet_dataset))):\n                losses = self.train_step((image_photo, image_monet))\n\n                # Accumulate epoch loss\n                for key in epoch_losses:\n                    epoch_losses[key] += float(losses[key])\n\n                progbar.update(step + 1, [\n                    ('photo_gen_loss', float(losses['photo_gen_loss'])),\n                    ('monet_gen_loss', float(losses['monet_gen_loss'])),\n                    ('photo_disc_loss', float(losses['photo_disc_loss'])),\n                    ('monet_disc_loss', float(losses['monet_disc_loss']))\n                ])\n\n                if step + 1 >= steps_per_epoch:\n                    break\n\n            # Store average loss per epoch\n            for key in loss_history:\n                loss_history[key].append(epoch_losses[key] / steps_per_epoch)\n\n        print(\"Training complete.\")\n        return loss_history\n\n\nlambda_list = [5, 10, 15]\nEPOCHS = 10\nresults = {}\n\nfor lambda_value in lambda_list:\n    print(f\"\\nTraining CycleGAN with lambda cycle = {lambda_value}\")\n    model = CustomCycleGANLambdaTuning(img_size=256, lambda_cycle=lambda_value)\n    model.compile()\n    loss_history = model.fit(photo_dataset, monet_dataset, epochs=EPOCHS)\n    results[f'lambda_{lambda_value}'] = loss_history['photo_gen_loss']","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-19T07:36:58.654916Z","iopub.execute_input":"2026-02-19T07:36:58.655417Z","iopub.status.idle":"2026-02-19T07:59:17.461831Z","shell.execute_reply.started":"2026-02-19T07:36:58.655387Z","shell.execute_reply":"2026-02-19T07:59:17.461230Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\nplt.figure(figsize=(10, 6))\nfor lambda_value in lambda_list:\n    key = f'lambda_{lambda_value}'\n    if key in results:\n        plt.plot(range(1, len(results[key]) + 1), results[key], label=f'lambda = {lambda_value}')\n    else:\n        print(f\"Warning: {key} not found in results.\")\n\nplt.title('Photo Generator Loss vs Epochs for Different Lambda')\nplt.xlabel('Epoch')\nplt.ylabel('Photo Generator Loss')\nplt.legend()\nplt.grid(True)\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-19T07:59:30.042817Z","iopub.execute_input":"2026-02-19T07:59:30.043541Z","iopub.status.idle":"2026-02-19T07:59:30.202531Z","shell.execute_reply.started":"2026-02-19T07:59:30.043512Z","shell.execute_reply":"2026-02-19T07:59:30.201943Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## VIII. Results and Interprtation on Lambda hyper-parameter Tuning ##\n- Conclusion: **Lambda value =10 remains the best option balance image stylization and realism.**\n### Result ###\n\n\n### Interpretation ###\n- A small lambda value places less emphasis on reconstruction consistency. This allows the model to produce images that look more like Monet paintings stylistically, but the object structure may become less realistic.\n- A large lambda value, on the other hand, places greater emphasis on reconstruction consistency. This encourages the model to preserve object structure more faithfully, but the resulting images may appear less stylized and closer to the original input.","metadata":{}},{"cell_type":"markdown","source":"## IX. Hyper-parameter Tuning with Early Stopping Interpretation ###\n**Based upon previous learning loss statistics, early-stopping with patience value =3 will return best result.**\n\n* Lowest combined generator loss (photo + monet) occurs at Epoch 11: 5.9172 + 5.8281 = 11.7453\n* After Epoch 11, generator loss increase again:\n  - Epoch 12: 17.07 (big spike)\n  - Epoch 13: 14.94 (still higher than 11.7453)\n  - Epoch 14: 12.56 (still higher than 11.7453)\n  - Epoch 15: 18.2 (another spike)\n* Thus, early stopping with patience value = 3 would allow\n  - Allow small fluctuation but still prevent overfitting of memorization of training photo.","metadata":{}},{"cell_type":"markdown","source":"## X. Prepare Best model for competition submission ##\nBest model configuration\n- lambda =10\n- Early stopping with patience value =3","metadata":{}},{"cell_type":"code","source":"import tensorflow as tf\nimport time\nfrom tensorflow.keras import optimizers, layers\nimport os\nimport shutil\n\nclass CustomCycleGANFinalSubmission(CycleGAN):\n    def fit(self, photo_dataset, monet_dataset, epochs=25, patience=3, weight_photo=0.9, weight_monet=0.1):\n        cardinality = tf.data.experimental.cardinality(photo_dataset)\n        if cardinality == tf.data.INFINITE_CARDINALITY or cardinality == tf.data.UNKNOWN_CARDINALITY:\n            steps_per_epoch = 300\n        else:\n            steps_per_epoch = cardinality.numpy()\n\n        best_loss = float('inf')\n        wait = 0\n\n        epoch_loss_history = {\n            'photo_gen_loss': [],\n            'monet_gen_loss': [],\n            'photo_disc_loss': [],\n            'monet_disc_loss': [],\n            'avg_total_gen_loss': []\n        }\n\n        for epoch in range(epochs):\n            print(f\"\\nEpoch {epoch+1}/{epochs}\")\n            progbar = tf.keras.utils.Progbar(steps_per_epoch, stateful_metrics=[\n                'photo_gen_loss', 'monet_gen_loss', 'photo_disc_loss', 'monet_disc_loss'\n            ])\n\n            epoch_losses = {\n                'photo_gen_loss': 0.0,\n                'monet_gen_loss': 0.0,\n                'photo_disc_loss': 0.0,\n                'monet_disc_loss': 0.0\n            }\n\n            for step, (image_photo, image_monet) in enumerate(tf.data.Dataset.zip((photo_dataset, monet_dataset))):\n                losses = self.train_step((image_photo, image_monet))\n                for key in epoch_losses:\n                    epoch_losses[key] += float(losses[key])\n                progbar.update(step + 1, [\n                    ('photo_gen_loss', float(losses['photo_gen_loss'])),\n                    ('monet_gen_loss', float(losses['monet_gen_loss'])),\n                    ('photo_disc_loss', float(losses['photo_disc_loss'])),\n                    ('monet_disc_loss', float(losses['monet_disc_loss']))\n                ])\n\n            for key in epoch_losses:\n                epoch_losses[key] /= steps_per_epoch\n                epoch_loss_history[key].append(epoch_losses[key])\n\n            avg_total_gen_loss = (\n                weight_photo * epoch_losses['photo_gen_loss'] +\n                weight_monet * epoch_losses['monet_gen_loss']\n            )\n            epoch_loss_history['avg_total_gen_loss'].append(avg_total_gen_loss)\n\n            if avg_total_gen_loss < best_loss:\n                best_loss = avg_total_gen_loss\n                wait = 0\n            else:\n                wait += 1\n                if wait >= patience:\n                    print(f\"Early stopping at epoch {epoch+1} with best weighted gen loss {best_loss:.4f}\")\n                    break\n\n        print(\"CustomCycleGANEarlyStopping::Training() complete.\")\n        return epoch_loss_history\n\n\n\nIMG_SIZE = 256\nEPOCHS = 25\nFINAL_OUTPUT_DIR = \"../final_generated_images\"\nif os.path.exists(FINAL_OUTPUT_DIR):\n    shutil.rmtree(FINAL_OUTPUT_DIR)\nos.makedirs(FINAL_OUTPUT_DIR, exist_ok=True)\n\nfinal_model = CustomCycleGANFinalSubmission(img_size=IMG_SIZE)\nfinal_model.compile()\nhistory = final_model.fit(\n    photo_dataset, monet_dataset,\n    epochs=EPOCHS,\n    patience=3,\n    weight_photo=1.0,  # emphasize photo → monet direction\n    weight_monet=0.0   # deprioritize monet → photo\n)\nprint(\"Final Model Training complete.\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-19T07:59:39.823673Z","iopub.execute_input":"2026-02-19T07:59:39.824028Z","iopub.status.idle":"2026-02-19T08:10:38.785200Z","shell.execute_reply.started":"2026-02-19T07:59:39.824003Z","shell.execute_reply":"2026-02-19T08:10:38.784445Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## XI. Prepare submission zip file ##\n* Generate 7000 images in jpg file\n* Zip jpg files to be submitted to the competition.","metadata":{}},{"cell_type":"code","source":"import tensorflow as tf\nimport os\nimport io\nimport zipfile\nimport shutil\n\nOUTPUT_ZIPFILE = \"images.zip\"\nTMP_DIR = \"../tmp\"\nNUM_IMAGES = 7000 \n\n#Clean up temporary directory if used\nif os.path.exists(TMP_DIR):\n    shutil.rmtree(TMP_DIR)\nif os.path.exists(OUTPUT_ZIPFILE):\n    os.remove(OUTPUT_ZIPFILE)\n\n\n#Generate zip file\nwith zipfile.ZipFile(OUTPUT_ZIPFILE, \"w\", compression=zipfile.ZIP_DEFLATED) as z:\n    i = 1\n    print(f\"Generating [{NUM_IMAGES}] images...\")\n    for photo_batch in photo_dataset.repeat().take(NUM_IMAGES):  # Repeat dataset to generate enough images\n        generated = model.gen_photo_to_monet(photo_batch, training=False)\n        for img in generated:\n            if i > NUM_IMAGES:\n                break\n            img = (img.numpy() * 127.5 + 127.5).astype(np.uint8)\n            im = PIL.Image.fromarray(img)\n            # Save image to bytes buffer\n            buffer = io.BytesIO()\n            im.save(buffer, format=\"JPEG\")\n            # Write to zip file\n            z.writestr(f\"image_{i}.jpg\", buffer.getvalue())\n            i += 1\n\nprint(f\"Submission zip file [{OUTPUT_ZIPFILE}] ready.\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-19T08:11:01.671151Z","iopub.execute_input":"2026-02-19T08:11:01.671444Z","iopub.status.idle":"2026-02-19T08:16:22.264203Z","shell.execute_reply.started":"2026-02-19T08:11:01.671420Z","shell.execute_reply":"2026-02-19T08:16:22.263566Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\n# plot loss over training epoch\nplt.figure(figsize=(12, 6))\n\nfor k, v in history.items():\n    plt.plot(v, label=k)\n\nplt.title(\"Loss Over Epochs\")\nplt.xlabel(\"Epoch\")\nplt.ylabel(\"Loss\")\nplt.legend()\nplt.grid(True)\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-19T08:16:42.635996Z","iopub.execute_input":"2026-02-19T08:16:42.636286Z","iopub.status.idle":"2026-02-19T08:16:42.826704Z","shell.execute_reply.started":"2026-02-19T08:16:42.636262Z","shell.execute_reply":"2026-02-19T08:16:42.826158Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## XII. Conclusion ##\n\nThe requirement of this competition is to apply a Generative Adversarial Network (GAN) to generate artwork resembling Monet's paintings.\nIn this project, I have chosen CycleGAN (Generative Adversial Network) for this competition.\n- Benefit of CycleGAN is that it ensures the generated image not only carries Monet's impressionistic style but also looks realistic.\n- CycleGAN is a dual DCGAN design (Deep Convolution GAN) to deploy two generators and two discriminators.\n- This bi-directional setup will enable translation between domains: A-> B and B -> A.\n- Key Components of CycleGAN are to deploy two generators and two discriminators in the setup.\n  - Two Generators:\n      - G: X -> Y (e.g., Photo → Monet)\n      - F: Y -> X (e.g., Monet → Photo)\n  - Two Discriminators:\n      - D_Y distinguish real Y from fake Y (G(X)) (e.g. real Monet)\n      - D_X distinguish real X from fake X (F(Y)) (e.g. real Photo)\n  - By applying a cycle-consistency constraint, it ensures the generated image not only carries Monet's impressionistic style but also looks realistic.\n\n### What improve the performance of the model ###\n- Best hyper-perimeter for the competition:\n  - Image size=256 x256\n  - Buffer size=1 (No batch normalization)\n  - Early stopping with 3 patienc value. \n  - lambda cycle=10\n- A lambda_cycle value of 10.0 provides a good balance between style transfer and content preservation in the CycleGAN model.\n- Cycle-consistency loss enhances content preservation in generated Monet-style images by ensuring that the mappings between photos and paintings are reversible.\n- This helps reduce the MiFID (Memorization-informed Fréchet Inception Distance) score by maintaining content fidelity, aligning the generated images closely with the target Monet style.\n\n### What didn't improve the performance of the model ###\n- Insufficient Training Duration:\n  - Training for only 10 epochs, as in the provided code, does not sufficiently reduce the generator and discriminator losses.\n  - Under-training leads to poor style transfer and higher MiFID scores.\n- Over-training:\n  - On the other extermem, long training time might lead not reduce learning loss because model might have memorize the training dataset.","metadata":{}},{"cell_type":"code","source":"# Randomly pick 5 generated images, and display generated images side by side with original photos\nFINAL_OUTPUT_DIR = \"../final_generated_images\"\n\ni = 1\nfor photo in photo_dataset.unbatch().take(10):\n    input_img = tf.expand_dims(photo, axis=0)  \n    generated = model.gen_photo_to_monet(input_img, training=False)[0]\n    \n    original_img = (photo.numpy() * 127.5 + 127.5).astype(np.uint8)\n    generated_img = (generated.numpy() * 127.5 + 127.5).astype(np.uint8)\n    PIL.Image.fromarray(generated_img).save(os.path.join(FINAL_OUTPUT_DIR, f\"{i}.jpg\"))\n    \n    # Display images side by side\n    plt.figure(figsize=(6, 3))\n    plt.subplot(1, 2, 1)\n    plt.imshow(original_img)\n    plt.title(\"Original Photo\")\n    plt.axis(\"off\")\n\n    plt.subplot(1, 2, 2)\n    plt.imshow(generated_img)\n    plt.title(\"Generated Monet\")\n    plt.axis(\"off\")\n    \n    plt.tight_layout()\n    plt.show()\n    i += 1","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-02-19T08:16:50.505634Z","iopub.execute_input":"2026-02-19T08:16:50.505958Z","iopub.status.idle":"2026-02-19T08:16:52.601424Z","shell.execute_reply.started":"2026-02-19T08:16:50.505932Z","shell.execute_reply":"2026-02-19T08:16:52.600858Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## XIII. Future Works and References ##\n- Experiment with Diffsion GAN to leverage strength of Transfer and GAN: [https://ieeexplore.ieee.org/document/10678380](https://ieeexplore.ieee.org/document/10678380)\n- Experiment with Vision Transformer: [https://paperswithcode.com/method/vision-transformer](https://paperswithcode.com/method/vision-transformer)\n\n### References ###\n- Ref: TensorFlow CycleGAN [https://www.tensorflow.org/tutorials/generative/cyclegan](https://www.tensorflow.org/tutorials/generative/cyclegan) \n","metadata":{}}]}