{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":21755,"databundleVersionId":1475600,"sourceType":"competition"}],"dockerImageVersionId":30918,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"\"\"\"\n# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session\n\"\"\"","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:18.755715Z","iopub.execute_input":"2025-02-28T14:04:18.756020Z","iopub.status.idle":"2025-02-28T14:04:18.764030Z","shell.execute_reply.started":"2025-02-28T14:04:18.755986Z","shell.execute_reply":"2025-02-28T14:04:18.763215Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Introduction\n\n**Brief description of the problem and data**\n\nOn this excerces we have in our hands 300 of beautiful painting made by Monet.\nThis guy was a French painter and founder of Impressionism painting who is seen as a key precursor to modernism, that lived from 1840 to 1926.*\n\nOn this exercice the objective is to make a digital Monet, this fake monet is an AI that looks a a beutiful landscape from outsuide and paints as Monet. \n\nThis task is going to need a GAN.\n\nGAN are defined by amazon web services as a deep learning architecture where you train two neural networks to compete with each other to generate new, more authentic data from a given training data set.\n\nIn this case we have some images from Monet paintings. They are in .jpg and .tfrec formats. This .tfrec is basically \"a custom TensorFlow format for storing a sequence of binary records. TFRecords are highly optimized for TensorFlow, which lead to ... efficient form of data storage and faster read speed compared to other types of formats\".*\n\nAs in this case i do't intent to use TPU we will use images in .jpg format.\n\n**Analysis**\n\nThis is not going to be an easy task, those are very preatty, and also preatty heterogeneus.\n\nAfter researching a little bit, and wasting a hole previous notbook building another GAN that failed. I camed into a tutorial in keras that trains a CycleGAN for horses to look like zebras. Voila, lest go.\n\nMy hole notbook is just an adaptation of their tutorial, and hase more comments in it. The turorial is available in https://keras.io/examples/generative/cyclegan/\n\nWe are now going to:\n1. Transform our data to look similar to data on the tutorial\n2. Construct Cycle GAN\n\nMost of the info is commented on code\n\n**Cycle GAN\\***\n\nCycleGAN allows image-to-image translation without paired images. It learns the features of the target style and transforms images from the original style to match it. \n\nIn fact we need a group of networks that generates images as follows:\n\nOriginal Image (Monet painting) **Pases trough →** Generator G(trained digital Monet) **Creates→** Transformed Image (Fake Painting)\n\nBut also for training we need a generator that reverses the proces, so: Transformed Image (Fake Painting) → Generator F **(Creates →)** Reconstructed Image (Original Photo)\n\n\\* <br> https://keras.io/examples/keras_recipes/creating_tfrecords/ <br> https://en.wikipedia.org/wiki/Claude_Monet <br>https://www.geeksforgeeks.org/cycle-generative-adversarial-network-cyclegan-2/","metadata":{}},{"cell_type":"markdown","source":"# EDA\n\nLest go with data reading some basic exploration on paintings, RBG of the mean and dominant colors.","metadata":{}},{"cell_type":"code","source":"import os\nimport keras\nimport glob\nimport cv2\n\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport tensorflow as tf\nimport tensorflow_datasets as tfds\n\nfrom PIL import Image\nfrom sklearn.cluster import KMeans\nfrom keras import layers, ops","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T23:04:51.920411Z","iopub.execute_input":"2025-02-28T23:04:51.920613Z","iopub.status.idle":"2025-02-28T23:05:05.422859Z","shell.execute_reply.started":"2025-02-28T23:04:51.920594Z","shell.execute_reply":"2025-02-28T23:05:05.421993Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#RAW data\nimgs = glob.glob(\"/kaggle/input/gan-getting-started/monet_jpg/*.jpg\")\nphotos = glob.glob(\"/kaggle/input/gan-getting-started/photo_jpg/*.jpg\")\nN = min(10, len(imgs)) #Images to display on the output (EDA section)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:23.451572Z","iopub.execute_input":"2025-02-28T14:04:23.452145Z","iopub.status.idle":"2025-02-28T14:04:23.472040Z","shell.execute_reply.started":"2025-02-28T14:04:23.452119Z","shell.execute_reply":"2025-02-28T14:04:23.471363Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#NN\n#Looking at some preatty paintings\nprint(f'Number of Monet images: {len(imgs)}')\nimg = Image.open(imgs[0])\nprint(f'Size is {img.size}')\n\nfig, axes = plt.subplots(2, 5, figsize=(12, 6))\n\nfor ax, imgs in zip(axes.flatten(), imgs[:10]): #[x:y] desde x:y imágenes\n    img = Image.open(imgs)\n    ax.imshow(img)\n    ax.axis(\"off\")\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:23.472710Z","iopub.execute_input":"2025-02-28T14:04:23.472928Z","iopub.status.idle":"2025-02-28T14:04:24.141322Z","shell.execute_reply.started":"2025-02-28T14:04:23.472909Z","shell.execute_reply":"2025-02-28T14:04:24.140254Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#NN\n#Looking at some preatty Landscapes\nprint(f'Number of Landscapes: {len(photos)}')\nprint(f'Size is {img.size} px')\n\n#IMAGEN\nimg = Image.open(photos[0])\nfig, axes = plt.subplots(2, 5, figsize=(12, 6))\n\nfor ax, lands in zip(axes.flatten(), photos[:10]): #[x:y] desde x:y imágenes\n    img = Image.open(lands)\n    ax.imshow(img)\n    ax.axis(\"off\")\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:24.142351Z","iopub.execute_input":"2025-02-28T14:04:24.142672Z","iopub.status.idle":"2025-02-28T14:04:24.961189Z","shell.execute_reply.started":"2025-02-28T14:04:24.142645Z","shell.execute_reply":"2025-02-28T14:04:24.960118Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#NN\n#RGB hystograms of individual monet paintings\nimgs = glob.glob(\"/kaggle/input/gan-getting-started/monet_jpg/*.jpg\")\n\n# Create subplots for displaying images and histograms\nfig, axes = plt.subplots(2, 5, figsize=(12, 6))\n\n# Iterate through images and axes\nfor idx, img_path in enumerate(imgs[:5]): # [:5] shoul match with subplot size (2, 5)\n    # Open the image with PIL\n    img = Image.open(img_path)\n    \n    # Show the image in the first column of the current row (left side)\n    ax_img = axes[0, idx]  # First row for images\n    ax_img.imshow(img)\n    ax_img.axis(\"off\")\n\n    # Convert PIL image to NumPy array (RGB)\n    img_rgb = np.array(img.convert('RGB'))\n    # Convert to BGR for OpenCV processing (as OpenCV uses BGR)\n    img_bgr = cv2.cvtColor(img_rgb, cv2.COLOR_RGB2BGR)\n    # Plot the RGB histogram in the second column of the current row (right side)\n    ax_hist = axes[1, idx]  # Second row for histograms\n    colors = ('r', 'g', 'b')\n    for channel, col in enumerate(colors):\n        hist = cv2.calcHist([img_bgr], [channel], None, [256], [0, 256])\n        ax_hist.plot(hist, color=col, label=f'{col.upper()} channel')\n    \n    ax_hist.set_title(\"RGB Histogram\")\n    ax_hist.set_xlabel(\"Pixel Intensity\")\n    ax_hist.set_ylabel(\"Frequency\")\n    ax_hist.legend()\n\nplt.tight_layout()\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:24.962242Z","iopub.execute_input":"2025-02-28T14:04:24.962696Z","iopub.status.idle":"2025-02-28T14:04:26.107678Z","shell.execute_reply.started":"2025-02-28T14:04:24.962648Z","shell.execute_reply":"2025-02-28T14:04:26.106710Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"How pictures are a lot diferent between each other. This is going to be hard for our robotic Monet","metadata":{}},{"cell_type":"code","source":"#NN\n#Averaged RBG histogram on paintings\n# Initialize accumulators for histograms per channel\nhist_r = np.zeros((256, 1), dtype=np.float64)\nhist_g = np.zeros((256, 1), dtype=np.float64)\nhist_b = np.zeros((256, 1), dtype=np.float64)\ntotal_pixels = 0\n\n# Loop through all images in the dataset\nfor image_path in imgs:\n    img = cv2.imread(image_path)\n    if img is None:\n        continue\n    # Convert from BGR to RGB\n    img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    \n    # Update histograms for each channel\n    hist_r += cv2.calcHist([img_rgb], [0], None, [256], [0, 256])\n    hist_g += cv2.calcHist([img_rgb], [1], None, [256], [0, 256])\n    hist_b += cv2.calcHist([img_rgb], [2], None, [256], [0, 256])\n    \n    # Count total pixels processed (images might have different sizes)\n    total_pixels += img.shape[0] * img.shape[1]\n\n# Plot aggregated histograms\nplt.figure(figsize=(10, 5))\nplt.plot(hist_r, color='r', label='Red')\nplt.plot(hist_g, color='g', label='Green')\nplt.plot(hist_b, color='b', label='Blue')\nplt.title(\"Aggregated RGB Histogram (raw counts)\")\nplt.xlabel(\"Pixel Intensity\")\nplt.ylabel(\"Frequency\")\nplt.legend()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:26.108632Z","iopub.execute_input":"2025-02-28T14:04:26.108911Z","iopub.status.idle":"2025-02-28T14:04:27.435000Z","shell.execute_reply.started":"2025-02-28T14:04:26.108889Z","shell.execute_reply":"2025-02-28T14:04:27.434075Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#NN\n# Bright or light on individual pintings\n#Create subplots with N rows and 2 columns (one for image, one for its brightness histogram)\nfig, axes = plt.subplots(nrows=N, ncols=2, figsize=(12, N * 3), squeeze=False)\n\nfor i, image_path in enumerate(imgs[:N]):\n    # Read image using OpenCV\n    img = cv2.imread(image_path)\n    if img is None:\n        continue\n    # Convert image to RGB (for display) and to grayscale (for brightness analysis)\n    img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n    \n    # Display the original image in the left subplot\n    axes[i, 0].imshow(img_rgb)\n    axes[i, 0].set_title(\"Image\")\n    axes[i, 0].axis(\"off\")\n    \n    # Compute the brightness histogram for the grayscale image\n    hist = cv2.calcHist([gray], [0], None, [256], [0, 256])\n    axes[i, 1].plot(hist, color='black')\n    axes[i, 1].set_title(\"Brightness Histogram\")\n    axes[i, 1].set_xlabel(\"Pixel Intensity\")\n    axes[i, 1].set_ylabel(\"Frequency\")\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:27.437611Z","iopub.execute_input":"2025-02-28T14:04:27.437930Z","iopub.status.idle":"2025-02-28T14:04:29.883387Z","shell.execute_reply.started":"2025-02-28T14:04:27.437905Z","shell.execute_reply":"2025-02-28T14:04:29.882481Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#NN\n#Bright of ligh on all paintings\n#Initialize an accumulator for the brightness histogram and a counter for total pixels\nhist_sum = np.zeros((256, 1), dtype=np.float64)\ntotal_pixels = 0\n\n# Loop over all images and accumulate the brightness histogram\nfor image_path in imgs:\n    img = cv2.imread(image_path)\n    if img is None:\n        continue\n    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n    hist = cv2.calcHist([gray], [0], None, [256], [0, 256])\n    hist_sum += hist\n    total_pixels += gray.shape[0] * gray.shape[1]\n\n# Option 1: Plot the raw aggregated brightness histogram\nplt.figure(figsize=(10, 5))\nplt.plot(hist_sum, color='black')\nplt.title(\"Aggregated Brightness Histogram (Raw Counts)\")\nplt.xlabel(\"Pixel Intensity\")\nplt.ylabel(\"Frequency\")\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:29.885012Z","iopub.execute_input":"2025-02-28T14:04:29.885248Z","iopub.status.idle":"2025-02-28T14:04:30.843454Z","shell.execute_reply.started":"2025-02-28T14:04:29.885229Z","shell.execute_reply":"2025-02-28T14:04:30.842542Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# CycleGAN 1. Data transformation\n\nIt was fun to know color and light as a person who likes painting it was fun.\nBut lest go with our AI Monet.\n\nIn tutorial data is organized in a very specific way, as code is not that easy to follow, the less i change the less i break so transform data to be similar is key part.","metadata":{}},{"cell_type":"code","source":"#Read data\n# Get lists of file paths for each domain\nmonet_paths = glob.glob(\"/kaggle/input/gan-getting-started/monet_jpg/*.jpg\")\nphoto_paths = glob.glob(\"/kaggle/input/gan-getting-started/photo_jpg/*.jpg\")\n\n# Create datasets from file paths\nmonet_ds = tf.data.Dataset.from_tensor_slices(monet_paths)\nphoto_ds = tf.data.Dataset.from_tensor_slices(photo_paths)\n\n# Function to load an image, convert pixel values to [-1, 1], and assign a label.\ndef load_monet_image(path):\n    image = tf.io.read_file(path)\n    image = tf.image.decode_jpeg(image, channels=3)\n    image = tf.image.convert_image_dtype(image, tf.float32)  # Scales to [0, 1]\n    image = (image - 0.5) * 2  # Now in [-1, 1]\n    return image\n\ndef load_photo_image(path):\n    image = tf.io.read_file(path)\n    image = tf.image.decode_jpeg(image, channels=3)\n    image = tf.image.convert_image_dtype(image, tf.float32)  # Scales to [0, 1]\n    image = (image - 0.5) * 2  # Now in [-1, 1]\n    return image\n\n# Map the functions to load and decode images with labels\nmonet_ds = (\n    monet_ds.map(load_monet_image, num_parallel_calls=tf.data.AUTOTUNE)\n    .shuffle(buffer_size=1000)\n)\n\nphoto_ds = (\n     photo_ds.map(load_photo_image, num_parallel_calls=tf.data.AUTOTUNE)\n     .shuffle(buffer_size=1000)\n )\n\n# Organize datasets in a dictionary similar to CycleGAN's \"trainA\" and \"trainB\"\ndataset = {\n    \"trainA\": monet_ds,\n    \"trainB\": photo_ds\n}\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:30.844443Z","iopub.execute_input":"2025-02-28T14:04:30.844699Z","iopub.status.idle":"2025-02-28T14:04:31.312798Z","shell.execute_reply.started":"2025-02-28T14:04:30.844673Z","shell.execute_reply":"2025-02-28T14:04:31.312117Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#buffer and batches\nbuffer_size = 256\nbatch_size = 1\n\n#loading datasets\ntrain_monet, train_photos = dataset[\"trainA\"], dataset[\"trainB\"]\ntrain_monet = (\n    train_monet.map(lambda img: img, num_parallel_calls=tf.data.AUTOTUNE)\n    .cache()\n    .shuffle(buffer_size)\n    .batch(batch_size)\n)\ntrain_photos = (\n    train_photos.map(lambda img: img, num_parallel_calls=tf.data.AUTOTUNE)\n    .cache()\n    .shuffle(buffer_size)\n    .batch(batch_size)\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:31.313615Z","iopub.execute_input":"2025-02-28T14:04:31.313940Z","iopub.status.idle":"2025-02-28T14:04:31.975145Z","shell.execute_reply.started":"2025-02-28T14:04:31.313909Z","shell.execute_reply":"2025-02-28T14:04:31.974161Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#NN\n# Extract one sample from each dataset (assuming each sample is a tuple: (image, label))\n#Random because of .shuffle on dataset\nimgA = next(iter(dataset[\"trainA\"].take(1)))\nimgB = next(iter(dataset[\"trainB\"].take(1)))\n\n# Create a figure with 2 rows and 1 column (2x1 subplot)\nfig, axes = plt.subplots(nrows=1, ncols=2, figsize=(6, 12))\n\n# Plot the first sample (Monet's Painting)\naxes[0].imshow((imgA.numpy() + 1) / 2)  # Convert from [-1, 1] to [0, 1] for display\naxes[0].set_title(\"Monet's Painting\")\naxes[0].axis(\"off\")\n\n# Plot the second sample (Real Landscape)\naxes[1].imshow((imgB.numpy() + 1) / 2)  # Convert from [-1, 1] to [0, 1] for display\naxes[1].set_title(\"Real Landscape\")\naxes[1].axis(\"off\")\n\nplt.tight_layout()\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:31.976154Z","iopub.execute_input":"2025-02-28T14:04:31.976483Z","iopub.status.idle":"2025-02-28T14:04:33.205431Z","shell.execute_reply.started":"2025-02-28T14:04:31.976451Z","shell.execute_reply":"2025-02-28T14:04:33.204481Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Cycle GAN 2. Constructing NN\n\n## Important variables:\n\n**kernel_initializer (kernel_init):**\n\nThis initializer is used by layers like Conv2D or Conv2DTranspose to set the initial values for the convolutional filters. Random normal distribution with a mean of 0.0 and a standard deviation of 0.02. This choice is common in GAN architectures because initializing weights with small random values can help stabilize training.\n\n**gamma_initializer (gamma_init):**\nThis is used by normalization layers (in this case, GroupNormalization) for the scaling parameter (often referred to as \"gamma\"). It controls how much the normalized output is scaled before being passed to the next layer. Using a similar random normal initializer (mean=0.0, stddev=0.02) is also a common practice in GANs to ensure that the initial scaling doesn't cause large deviations, helping to maintain stable gradients during training. Is a learned parameter.\n\n","metadata":{}},{"cell_type":"code","source":"#Important Variables\nautotune = tf.data.AUTOTUNE\n#Kernel_init\nkernel_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n# Gamma initializer for instance normalization.\ngamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n#img size\ninput_img_size = (256, 256, 3)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:33.206422Z","iopub.execute_input":"2025-02-28T14:04:33.206736Z","iopub.status.idle":"2025-02-28T14:04:33.212137Z","shell.execute_reply.started":"2025-02-28T14:04:33.206709Z","shell.execute_reply":"2025-02-28T14:04:33.211201Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Basic Functions","metadata":{}},{"cell_type":"code","source":"#Building blocks used in the CycleGAN generators and discriminators\n\n\"\"\"\n-------------------\nBlock of RNN\nRecieves x, type of activation function, kernel init, size, stride, uses padding same, gamma init an bias as false\n    \n    1. Convolution + normalization + activation funtion\n    2. Convolution + normalization\n    3. Residual or skip conection from initial x to final x\n\ndim = x.shape[-1]: Number of channels of RGB input - In a residual block, the goal is to learn a transformation that can be added back to the original input (the skip connection). For this addition to work, the input and the transformed output must have the same shape. By using x.shape[-1] (the number of channels in the input) as the number of filters for the convolutional layers, you ensure that the output of the block has the same number of channels as the input, making the element-wise addition possible. This design is a standard practice in architectures like ResNet and CycleGAN, where preserving the feature dimensions across a block helps in learning the residual mapping effectively.\n\n\"\"\"\n\ndef residual_block(\n    x,\n    activation,\n    kernel_initializer=kernel_init,\n    kernel_size=(3, 3),\n    strides=(1, 1),\n    padding=\"same\",  # Use 'same' padding instead of reflection padding\n    gamma_initializer=gamma_init,\n    use_bias=False,\n    ):\n    \n    dim = x.shape[-1] \n    input_tensor = x\n\n    # First convolutional layer with same padding\n    x = layers.Conv2D(\n        dim, # dim is the number of filters\n        kernel_size,\n        strides=strides,\n        kernel_initializer=kernel_initializer,\n        padding=padding,\n        use_bias=use_bias,\n    )(input_tensor)\n    x = keras.layers.GroupNormalization(groups=1, gamma_initializer=gamma_initializer)(x)\n    x = activation(x)\n\n    # Second convolutional layer with same padding\n    x = layers.Conv2D(\n        dim,\n        kernel_size,\n        strides=strides,\n        kernel_initializer=kernel_initializer,\n        padding=padding,\n        use_bias=use_bias,\n    )(x)\n    x = keras.layers.GroupNormalization(groups=1, gamma_initializer=gamma_initializer)(x)\n\n    # Add the input tensor (residual connection)\n    x = layers.add([input_tensor, x])\n    return x\n    \n\"\"\"\n-------------------\nDownsample function \nRecieves x, type of activation function, kernel init, size, stride, uses padding same, gamma init an bias as false\n    \n    1. Convolution + normalization (with a bigger stride to downsample)\n\"\"\"\n\ndef downsample(\n    x,\n    filters,\n    activation,\n    kernel_initializer=kernel_init,\n    kernel_size=(3, 3),\n    strides=(2, 2),\n    padding=\"same\",\n    gamma_initializer=gamma_init,\n    use_bias=False,\n):\n    x = layers.Conv2D(\n        filters,\n        kernel_size,\n        strides=strides,\n        kernel_initializer=kernel_initializer,\n        padding=padding,\n        use_bias=use_bias,\n    )(x)\n    x = keras.layers.GroupNormalization(groups=1, gamma_initializer=gamma_initializer)(\n        x\n    )\n    if activation:\n        x = activation(x)\n    return x\n    \n\"\"\"\n-------------------\nUpsample function \nRecieves x, type of activation function, kernel init, size, stride, uses padding same, gamma init an bias as false\n    \n    1. Inverse - Convolution + normalization (with a bigger stride to downsample)\n\"\"\"\n\ndef upsample(\n    x,\n    filters,\n    activation,\n    kernel_size=(3, 3),\n    strides=(2, 2),\n    padding=\"same\",\n    kernel_initializer=kernel_init,\n    gamma_initializer=gamma_init,\n    use_bias=False,\n):\n    x = layers.Conv2DTranspose(\n        filters,\n        kernel_size,\n        strides=strides,\n        padding=padding,\n        kernel_initializer=kernel_initializer,\n        use_bias=use_bias,\n    )(x)\n    x = keras.layers.GroupNormalization(groups=1, gamma_initializer=gamma_initializer)(\n        x\n    )\n    if activation:\n        x = activation(x)\n    return x","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:33.213272Z","iopub.execute_input":"2025-02-28T14:04:33.213649Z","iopub.status.idle":"2025-02-28T14:04:33.226741Z","shell.execute_reply.started":"2025-02-28T14:04:33.213616Z","shell.execute_reply":"2025-02-28T14:04:33.226058Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## RESNet Generator (Function)","metadata":{}},{"cell_type":"code","source":"\"\"\"\n--------------\n1. Convolution + Normalization + activation\n2. Downsampling - uses funtion defined previously\n3. Residual blocks - uses funtion defined previously\n3. Upsampling - uses funtion defined previously\n4. Final convolution and activtion\n\n\"\"\"\ndef get_resnet_generator(\n    filters=64,\n    num_downsampling_blocks=2,\n    num_residual_blocks=9,\n    num_upsample_blocks=2,\n    gamma_initializer=gamma_init,\n    name=None,\n):\n    img_input = layers.Input(shape=input_img_size, name=name + \"_img_input\")\n    x = img_input\n    x = layers.Conv2D(filters, (7, 7), padding=\"same\", kernel_initializer=kernel_init, use_bias=False)(x)\n    x = keras.layers.GroupNormalization(groups=1, gamma_initializer=gamma_initializer)(x)\n    x = layers.Activation(\"relu\")(x)\n\n    # Downsampling\n    for _ in range(num_downsampling_blocks):\n        filters *= 2\n        x = downsample(x, filters=filters, activation=layers.Activation(\"relu\"))\n\n    # Residual blocks\n    for _ in range(num_residual_blocks):\n        x = residual_block(x, activation=layers.Activation(\"relu\"))\n\n    # Upsampling\n    for _ in range(num_upsample_blocks):\n        filters //= 2\n        x = upsample(x, filters, activation=layers.Activation(\"relu\"))\n\n    # Final block\n    x = layers.Conv2D(3, (7, 7), padding=\"same\")(x)\n    x = layers.Activation(\"tanh\")(x)\n\n    model = keras.models.Model(img_input, x, name=name)\n    return model","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:33.227641Z","iopub.execute_input":"2025-02-28T14:04:33.227960Z","iopub.status.idle":"2025-02-28T14:04:33.243523Z","shell.execute_reply.started":"2025-02-28T14:04:33.227928Z","shell.execute_reply":"2025-02-28T14:04:33.242815Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Discriminator (Function)","metadata":{}},{"cell_type":"code","source":"def get_discriminator(filters=64, kernel_initializer=kernel_init, num_downsampling=3, name=None):\n    img_input = layers.Input(shape=input_img_size, name=name + \"_img_input\")\n    x = layers.Conv2D(filters,(4, 4),strides=(2, 2), padding=\"same\", kernel_initializer=kernel_initializer,)(img_input)\n    x = layers.LeakyReLU(0.2)(x)\n\n    num_filters = filters\n    for num_downsample_block in range(3):\n        num_filters *= 2\n        if num_downsample_block < 2:\n            x = downsample(\n                x,\n                filters=num_filters,\n                activation=layers.LeakyReLU(0.2),\n                kernel_size=(4, 4),\n                strides=(2, 2),\n            )\n        else:\n            x = downsample(\n                x,\n                filters=num_filters,\n                activation=layers.LeakyReLU(0.2),\n                kernel_size=(4, 4),\n                strides=(1, 1),\n            )\n\n    x = layers.Conv2D(1, (4, 4), strides=(1, 1), padding=\"same\", kernel_initializer=kernel_initializer)(x)\n\n    model = keras.models.Model(inputs=img_input, outputs=x, name=name)\n    return model","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:33.244415Z","iopub.execute_input":"2025-02-28T14:04:33.244682Z","iopub.status.idle":"2025-02-28T14:04:33.262653Z","shell.execute_reply.started":"2025-02-28T14:04:33.244660Z","shell.execute_reply":"2025-02-28T14:04:33.261945Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Creating Networks ","metadata":{}},{"cell_type":"code","source":"#\n#Get the generators\ngen_G = get_resnet_generator(name=\"generator_G\") # Translate features from monet to photos\ngen_F = get_resnet_generator(name=\"generator_F\") # reverse proces photos to monet\n\n# Get the discriminators\ndisc_X = get_discriminator(name=\"discriminator_X\") #Discrimination on monet real or fake\ndisc_Y = get_discriminator(name=\"discriminator_Y\") #Discriminator on photo real or fake","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:33.263435Z","iopub.execute_input":"2025-02-28T14:04:33.263669Z","iopub.status.idle":"2025-02-28T14:04:34.812548Z","shell.execute_reply.started":"2025-02-28T14:04:33.263649Z","shell.execute_reply":"2025-02-28T14:04:34.811861Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Cycle GAN 3. Teaching to paint class CycleGan\n\nThis is a little hard to follow but we basically need to update weights and unify all NN for them to work toghether\n\nSome definition here:\n\n**Lambda_cycle** = \"cycle consistency loss\" - generador G outputs : G(X) ---> gen F(G(x)) shoud go back to X, diference is an hyperparameter to tune\n\n**Lambda_identity** = \"identity loss\" - Image from domain No. 2 on gen G shoud not make changes Y = G(Y).","metadata":{}},{"cell_type":"code","source":"class CycleGan(keras.Model):\n    def __init__(\n        self,\n        generator_G,\n        generator_F,\n        discriminator_X,\n        discriminator_Y,\n        lambda_cycle=10.0,\n        lambda_identity=0.5,\n    ):\n        super().__init__() # calls init from parent class (keras.Model)\n        self.gen_G = generator_G\n        self.gen_F = generator_F\n        self.disc_X = discriminator_X\n        self.disc_Y = discriminator_Y\n        self.lambda_cycle = lambda_cycle\n        self.lambda_identity = lambda_identity\n        \n    def call(self, inputs):\n        return (\n            self.disc_X(inputs),\n            self.disc_Y(inputs),\n            self.gen_G(inputs),\n            self.gen_F(inputs),\n        )\n\n    def compile(\n        self,\n        gen_G_optimizer,\n        gen_F_optimizer,\n        disc_X_optimizer,\n        disc_Y_optimizer,\n        gen_loss_fn,\n        disc_loss_fn,\n    ):\n        super().compile()\n        self.gen_G_optimizer = gen_G_optimizer\n        self.gen_F_optimizer = gen_F_optimizer\n        self.disc_X_optimizer = disc_X_optimizer\n        self.disc_Y_optimizer = disc_Y_optimizer\n        self.generator_loss_fn = gen_loss_fn\n        self.discriminator_loss_fn = disc_loss_fn\n        self.cycle_loss_fn = keras.losses.MeanAbsoluteError()\n        self.identity_loss_fn = keras.losses.MeanAbsoluteError()\n\n    def train_step(self, batch_data):\n        # x is monet and y is photo\n        real_x, real_y = batch_data\n\n        # For CycleGAN, we need to calculate different\n        # kinds of losses for the generators and discriminators.\n        # We will perform the following steps here:\n        #\n        # 1. Pass real images through the generators and get the generated images\n        # 2. Pass the generated images back to the generators to check if we\n        #    can predict the original image from the generated image.\n        # 3. Do an identity mapping of the real images using the generators.\n        # 4. Pass the generated images in 1) to the corresponding discriminators.\n        # 5. Calculate the generators total loss (adversarial + cycle + identity)\n        # 6. Calculate the discriminators loss\n        # 7. Update the weights of the generators\n        # 8. Update the weights of the discriminators\n        # 9. Return the losses in a dictionary\n\n        with tf.GradientTape(persistent=True) as tape:\n            # monet to fake photo\n            fake_y = self.gen_G(real_x, training=True)\n            # photo to fake monet -> y2x\n            fake_x = self.gen_F(real_y, training=True)\n\n            # Cycle (monet to fake photo to fake monet): x -> y -> x\n            cycled_x = self.gen_F(fake_y, training=True)\n            # Cycle (photo to fake moent to fake photo) y -> x -> y\n            cycled_y = self.gen_G(fake_x, training=True)\n\n            # Identity mapping\n            same_x = self.gen_F(real_x, training=True)\n            same_y = self.gen_G(real_y, training=True)\n\n            # Discriminator output\n            disc_real_x = self.disc_X(real_x, training=True)\n            disc_fake_x = self.disc_X(fake_x, training=True)\n\n            disc_real_y = self.disc_Y(real_y, training=True)\n            disc_fake_y = self.disc_Y(fake_y, training=True)\n\n            # Generator adversarial loss\n            gen_G_loss = self.generator_loss_fn(disc_fake_y)\n            gen_F_loss = self.generator_loss_fn(disc_fake_x)\n\n            # Generator cycle loss\n            cycle_loss_G = self.cycle_loss_fn(real_y, cycled_y) * self.lambda_cycle\n            cycle_loss_F = self.cycle_loss_fn(real_x, cycled_x) * self.lambda_cycle\n\n            # Generator identity loss\n            id_loss_G = (\n                self.identity_loss_fn(real_y, same_y)\n                * self.lambda_cycle\n                * self.lambda_identity\n            )\n            id_loss_F = (\n                self.identity_loss_fn(real_x, same_x)\n                * self.lambda_cycle\n                * self.lambda_identity\n            )\n\n            # Total generator loss\n            total_loss_G = gen_G_loss + cycle_loss_G + id_loss_G\n            total_loss_F = gen_F_loss + cycle_loss_F + id_loss_F\n\n            # Discriminator loss\n            disc_X_loss = self.discriminator_loss_fn(disc_real_x, disc_fake_x)\n            disc_Y_loss = self.discriminator_loss_fn(disc_real_y, disc_fake_y)\n\n        # Get the gradients for the generators\n        grads_G = tape.gradient(total_loss_G, self.gen_G.trainable_variables)\n        grads_F = tape.gradient(total_loss_F, self.gen_F.trainable_variables)\n\n        # Get the gradients for the discriminators\n        disc_X_grads = tape.gradient(disc_X_loss, self.disc_X.trainable_variables)\n        disc_Y_grads = tape.gradient(disc_Y_loss, self.disc_Y.trainable_variables)\n\n        # Update the weights of the generators\n        self.gen_G_optimizer.apply_gradients(\n            zip(grads_G, self.gen_G.trainable_variables)\n        )\n        self.gen_F_optimizer.apply_gradients(\n            zip(grads_F, self.gen_F.trainable_variables)\n        )\n\n        # Update the weights of the discriminators\n        self.disc_X_optimizer.apply_gradients(\n            zip(disc_X_grads, self.disc_X.trainable_variables)\n        )\n        self.disc_Y_optimizer.apply_gradients(\n            zip(disc_Y_grads, self.disc_Y.trainable_variables)\n        )\n\n        return {\n            \"G_loss\": total_loss_G,\n            \"F_loss\": total_loss_F,\n            \"D_X_loss\": disc_X_loss,\n            \"D_Y_loss\": disc_Y_loss,\n        }\n        ","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T17:49:57.338446Z","iopub.execute_input":"2025-02-28T17:49:57.338861Z","iopub.status.idle":"2025-02-28T17:49:57.353813Z","shell.execute_reply.started":"2025-02-28T17:49:57.338823Z","shell.execute_reply":"2025-02-28T17:49:57.352674Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Loss Functions","metadata":{}},{"cell_type":"code","source":"# Loss function for evaluating adversarial loss\nadv_loss_fn = keras.losses.MeanSquaredError()\n\n# Define the loss function for the generators\ndef generator_loss_fn(fake):\n    fake_loss = adv_loss_fn(ops.ones_like(fake), fake)\n    return fake_loss\n\n\n# Define the loss function for the discriminators\ndef discriminator_loss_fn(real, fake):\n    real_loss = adv_loss_fn(ops.ones_like(real), real)\n    fake_loss = adv_loss_fn(ops.zeros_like(fake), fake)\n    return (real_loss + fake_loss) * 0.5","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:04:35.110189Z","iopub.execute_input":"2025-02-28T14:04:35.110586Z","iopub.status.idle":"2025-02-28T14:04:35.126083Z","shell.execute_reply.started":"2025-02-28T14:04:35.110546Z","shell.execute_reply":"2025-02-28T14:04:35.125236Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Checkpoint functions","metadata":{}},{"cell_type":"code","source":"#Save images and checkpoints\n\nclass GANMonitor(keras.callbacks.Callback):\n    \"\"\"A callback to generate and show images after each epoch\n    Modify \n     if (epoch + 1) % 1 == 0 for changing every X epoch\n    Modify\n     (self, num_img=4) for taking more pictures\n    \n    \"\"\"\n\n    def __init__(self, num_img=4):\n        self.num_img = num_img\n\n    def on_epoch_end(self, epoch, logs=None):\n        \n        if (epoch + 1) % 1 == 0: #Optionally recreate every x epoch\n            \n            _, ax = plt.subplots(4, 2, figsize=(12, 12))\n            for i, img in enumerate(train_monet.take(self.num_img)):\n                prediction = self.model.gen_G(img)[0].numpy() #AI paints\n                prediction = (prediction * 127.5 + 127.5).astype(np.uint8) #Converts -1 to 1 to a readable image\n                img = (img[0] * 127.5 + 127.5).numpy().astype(np.uint8)\n    \n                ax[i, 0].imshow(img)\n                ax[i, 1].imshow(prediction)\n                ax[i, 0].set_title(\"Input image\")\n                ax[i, 1].set_title(\"Translated image\")\n                ax[i, 0].axis(\"off\")\n                ax[i, 1].axis(\"off\")\n\n            plt.show()\n            plt.close()\n#End of class\n\npainter = GANMonitor ()\n\n\"\"\"\n-------\nCheckpoints\n-------\n\"\"\"\ncheckpoint_filepath = \"/kaggle/working/model_checkpoints/cyclegan{epoch}_checkpoint.weights.h5\"\ncheckpoint = keras.callbacks.ModelCheckpoint(\n    filepath=checkpoint_filepath, save_weights_only=True, save_freq=600\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:06:03.968039Z","iopub.execute_input":"2025-02-28T14:06:03.968378Z","iopub.status.idle":"2025-02-28T14:06:03.975377Z","shell.execute_reply.started":"2025-02-28T14:06:03.968355Z","shell.execute_reply":"2025-02-28T14:06:03.974539Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Compile and train","metadata":{}},{"cell_type":"code","source":"# Create cycle gan model\n\"\"\"\n---------------\nModel from nothing\n---------------\n\"\"\"\ncycle_gan_model = CycleGan(\n    generator_G=gen_G, generator_F=gen_F, discriminator_X=disc_X, discriminator_Y=disc_Y\n)\n\n# Compile (final configuration) of the model\n\ncycle_gan_model.compile(\n    gen_G_optimizer=keras.optimizers.Adam(learning_rate=2e-4, beta_1=0.5),\n    gen_F_optimizer=keras.optimizers.Adam(learning_rate=2e-4, beta_1=0.5),\n    disc_X_optimizer=keras.optimizers.Adam(learning_rate=2e-4, beta_1=0.5),\n    disc_Y_optimizer=keras.optimizers.Adam(learning_rate=2e-4, beta_1=0.5),\n    gen_loss_fn=generator_loss_fn,\n    disc_loss_fn=discriminator_loss_fn,\n)\n\n\"\"\"\n----------------\nModel from checkpoint - if want to retrain after x epoch\n---------------\n\"\"\"\n# cycle_gan_model.load_weights(\"/kaggle/working/model_checkpoints/cyclegan_weights_epoch05.h5\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:06:13.372434Z","iopub.execute_input":"2025-02-28T14:06:13.372753Z","iopub.status.idle":"2025-02-28T14:06:13.397204Z","shell.execute_reply.started":"2025-02-28T14:06:13.372728Z","shell.execute_reply":"2025-02-28T14:06:13.396514Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#debugging\nmini_monet = train_monet.take(5)\nmini_photos = train_photos.take(5)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:11:45.250743Z","iopub.status.idle":"2025-02-28T14:11:45.251059Z","shell.execute_reply":"2025-02-28T14:11:45.250945Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Here we will train the model \ncycle_gan_model.fit(tf.data.Dataset.zip((train_monet, train_photos)),\n    epochs=100,\n    callbacks=[painter, checkpoint]\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T14:12:07.275056Z","iopub.execute_input":"2025-02-28T14:12:07.275414Z","iopub.status.idle":"2025-02-28T16:24:07.268134Z","shell.execute_reply.started":"2025-02-28T14:12:07.275382Z","shell.execute_reply":"2025-02-28T16:24:07.267180Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Results - After the training\n\nSo with that preview i have to say it was very good for making Fake Monet from real ones. Is time to see if it can also paint a landscape.\nI have to say that learning is no allways consistent, and some epochs are better than others. GAN training is unstable and has some oscillations. Also some pictures seems \"easier\" than others wich can mean filters are better for capturing their essence on ResNet. \n\nDelicate balance between the generator and discriminator— is hard to mantein and if one starts to overpower the other, the quality can drop.\n\nAnother explanations on inestability are because os:\n\n**Mode Collapse:** The generator might start to produce a limited variety of outputs (mode collapse) after a certain point, leading to less diverse or lower-quality images.\n\n**Noisy Training Signal:** The adversarial loss can be quite noisy, causing fluctuations in image quality as the model continually adjusts to the discriminator's feedback.\n\n## Final appreciations\n\nThe main and big difference from original to fake can be the little details, so i went asking to a famous transformer (CHAT-GPT) what can be the reasons and this is \"his\" answer:\n\"\nDownsampling/Upsampling Loss:\n    Many generators include downsampling (to capture global context) followed by upsampling. This process can cause a loss of fine details because high-frequency information may be smoothed out during downsampling.\n\nKernel Size and Receptive Field:\n    Convolutional layers have a limited receptive field determined by the kernel size. If the kernels are too large or the network isn’t deep enough, they might not focus on fine details, leading to a blurred reconstruction of high-frequency signals.\n\nLoss Function Emphasis:\n    Standard adversarial and cycle consistency losses often focus on global structure rather than fine textures. Without an additional loss (like perceptual or texture loss), the generator might prioritize overall coherence at the expense of sharp, detailed features.\n\nArchitecture Choices:\n    Certain architectural choices (e.g., the use of transposed convolutions) can introduce artifacts or fail to perfectly reconstruct high-frequency components.\n\nIn practice, if preserving high-frequency details is important, researchers often experiment with different kernel sizes, skip connections, or add extra loss terms that specifically encourage the retention of fine details.\n\n\"\n\nI guess we need to play with a little bit more with Learning Rate and Hyperparameters, maybe even padding or more advancer preproccessing if you want better and more consistant images. But this has already taken a lot of time. And i am preatty happy with what we did and leaned here.\n    ","metadata":{}},{"cell_type":"code","source":"\"\"\"\n----------------\nModel from checkpoint - Saving for using it later on\n\nUse favorite epoc weights and don't .fit an you will have Fake Monet from previous epoch\n\n28, 31, 35, 45, 53, 55, and 71 are promising\nAfter loading wights you could safe hole model\ntf.saved_model.save(cycle_gan_model, \"/kaggle/working/saved_cyclegan_model\") \n---------------\n\ncycle_gan_model.load_weights(\"/kaggle/working/model_checkpoints/cyclegan71_checkpoint.weights.h5\")\n\"\"\"\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T17:50:04.012885Z","iopub.execute_input":"2025-02-28T17:50:04.013301Z","iopub.status.idle":"2025-02-28T17:50:04.734976Z","shell.execute_reply.started":"2025-02-28T17:50:04.013264Z","shell.execute_reply":"2025-02-28T17:50:04.733725Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Show painted the landscapes\n_, ax = plt.subplots(4, 2, figsize=(10, 15))\nfor i, img in enumerate(train_photos.take(4)):\n    prediction = cycle_gan_model.gen_F(img, training=False)[0].numpy()\n    prediction = (prediction * 127.5 + 127.5).astype(np.uint8)\n    img = (img[0] * 127.5 + 127.5).numpy().astype(np.uint8)\n\n    ax[i, 0].imshow(img)\n    ax[i, 1].imshow(prediction)\n    ax[i, 0].set_title(\"Input image\")\n    ax[i, 1].set_title(\"Translated image\")\n    ax[i, 0].axis(\"off\")\n    ax[i, 1].axis(\"off\")\n\nplt.tight_layout()\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T16:29:50.865748Z","iopub.execute_input":"2025-02-28T16:29:50.866103Z","iopub.status.idle":"2025-02-28T16:29:53.101489Z","shell.execute_reply.started":"2025-02-28T16:29:50.866076Z","shell.execute_reply":"2025-02-28T16:29:53.100087Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# rm -rf /kaggle/working/saved_cyclegan","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-02-28T17:52:27.182457Z","iopub.execute_input":"2025-02-28T17:52:27.182934Z","iopub.status.idle":"2025-02-28T17:52:27.775489Z","shell.execute_reply.started":"2025-02-28T17:52:27.182890Z","shell.execute_reply":"2025-02-28T17:52:27.774509Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}