{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":21755,"databundleVersionId":1475600,"sourceType":"competition"}],"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# INTRODUCTION & SETUP\nThis notebook trains a CycleGAN to transfer a Monet styling to real photos. CycleGANs are a type of adversarial network commonly used in style transfer. In this implementation, generator networks learn to create fake Monet images from real photos and fake photos from Monet images, while discriminator networks learn to distinguish real and fake Monets and photos. The generators and disciminators compete during training, each pushing the other to improve.\n\nThe data for this project comes from the [Monet Paintings dataset](https://www.kaggle.com/c/gan-getting-started/data) on Kaggle. The dataset contains 300 Monet paintings and 7028 real photos saved in tfrec or jpg fiels. All images are 256 x 256. \n\nExplanations of the CycleGAN design and implementation are mixed in with the code below. Along the way, I've also written comments to myself and my future self to better understand CycleGANs, the TensorFlow and Keras APIs, and a few object-oriented programming concepts.","metadata":{}},{"cell_type":"markdown","source":"### INSTALL LIBRARIES","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport matplotlib.pyplot as plt\nimport datetime\nfrom kaggle_datasets import KaggleDatasets\n# !pip install tensorflow==2.13\nimport tensorflow as tf\nfrom tensorflow import keras\nfrom keras.losses import BinaryCrossentropy\nfrom keras.callbacks import ModelCheckpoint, TensorBoard\nfrom keras import Input, Model, Sequential\nfrom keras.layers import Conv2D, Conv2DTranspose, ZeroPadding2D, ReLU, LeakyReLU, Dropout, GroupNormalization, Concatenate\nfrom keras.layers import Lambda, RandomFlip, RandomRotation, RandomZoom, RandomBrightness, RandomContrast, RandomTranslation, RandomCrop","metadata":{"id":"jM5YzQRK74ff","execution":{"iopub.status.busy":"2024-02-18T04:18:22.728340Z","iopub.execute_input":"2024-02-18T04:18:22.729198Z","iopub.status.idle":"2024-02-18T04:18:35.240445Z","shell.execute_reply.started":"2024-02-18T04:18:22.729165Z","shell.execute_reply":"2024-02-18T04:18:35.239670Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### TENSORFLOW CONFIGURATION (TPU, GPU, OR CPU)\n\nThe code below sets the model training strategy according to the processor type, and outputs the processor details. Kaggle's GPU T4 X2 and TPU VM v3-8 were not compatible with TensorFlow due to incompatibilities with CUDA version 12.2 and possibly with cuDNN. Using Kaggle's GPU P100 returned warning messages that hinted at compatibility problems, but the model compiled and trained without issue. ","metadata":{"id":"XNxXF77j74fg"}},{"cell_type":"code","source":"try:\n    tpu = tf.distribute.cluster_resolver.TPUClusterResolver() # TPU detection\n    tf.config.experimental_connect_to_cluster(tpu) # Connects to the given cluster\n    tf.tpu.experimental.initialize_tpu_system(tpu) # Initializes the TPU system\n    strategy = tf.distribute.TPUStrategy(tpu) # TPU distribution strategy\n    print('Running on TPUs')\n\nexcept ValueError:\n    print(\"TPU not available.\")\n    if tf.config.list_physical_devices('GPU'):\n        strategy = tf.distribute.MirroredStrategy()\n        print(\"Running on GPU.\")\n    else:\n        strategy = tf.distribute.get_strategy()\n        print(\"Running on CPU.\")\n\nprint('Number of accelerators (CPU, GPU, or TPU):', strategy.num_replicas_in_sync)\n\nprint(\"Tensor Flow Version:\", tf.__version__)\n\nfrom psutil import virtual_memory\nram_gb = virtual_memory().total / 1e9\nprint('Your runtime has {:.1f} gigabytes of available RAM\\n'.format(ram_gb))\n\nif ram_gb < 20:\n  print('Not using a high-RAM runtime')\nelse:\n  print('You are using a high-RAM runtime!')","metadata":{"id":"0lN7nYta74fh","outputId":"a45ed48a-5292-40f7-9c53-62740d877ede","execution":{"iopub.status.busy":"2024-02-18T04:18:35.241887Z","iopub.execute_input":"2024-02-18T04:18:35.242394Z","iopub.status.idle":"2024-02-18T04:18:35.862212Z","shell.execute_reply.started":"2024-02-18T04:18:35.242368Z","shell.execute_reply":"2024-02-18T04:18:35.861210Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# LOAD, PARSE, AND PREPROCESS DATA\nWe will use the TensorFlow's TFRecord files (`.tfrec`) provided by Kaggle. `.tfrec` files are binary, record-oriented files that integrate with TensorFlow's ecosystem and are optimimzed for fast read and write times. Parsing the data involves conveting the serialized tfrecords into TensorFlow dictionaries, decoding the TensorFlow string into a 3-D tensor, setting the data type to float32, normalizing pixel intensities to [-1, 1], and sizing the images to 256 x 256.","metadata":{"id":"UmGh6b6G74fh"}},{"cell_type":"code","source":"# parse_tfrecord converts the serialized TFRecord into a 3-D tensor, 256x256x3\ndef parse_tfrecord_fn(example):\n  # `feature_description` specifies the structure of data stored in the TFRecord file\n  # tf.string is a Tensor data type that can hold byte strings, like binary, jpeg-encoded images\n    feature_description = {\n        'image': tf.io.FixedLenFeature([], tf.string)\n    }\n\n    # convert serialized tfrecords into Tensor Flow 'dict', {'image': tf.string}.\n    example = tf.io.parse_single_example(example, feature_description)\n\n    # decode the tf.string into a 3-D tensor with dtype `tf.uint8`\n    image = tf.io.decode_jpeg(example['image'], channels=3)\n\n    # Set datatype to float32 before normalizing\n#     image = tf.cast(image, tf.float32)\n    image = tf.cast(image, tf.float16)\n\n    # Normalizing to [-1, 1] is standard practice and speeds up training.\n    image = (image / 127.5) - 1\n\n    # Resize for GAN input 256x256\n    image = tf.image.resize(image, [256, 256])\n    return image\n\n# Define a function to extract images from the TFRecord files\ndef load_dataset(filenames):\n    raw_dataset = tf.data.TFRecordDataset(filenames)\n    parsed_dataset = raw_dataset.map(parse_tfrecord_fn)\n    return parsed_dataset","metadata":{"id":"kFEwGcpS74fh","execution":{"iopub.status.busy":"2024-02-18T04:18:35.863622Z","iopub.execute_input":"2024-02-18T04:18:35.863912Z","iopub.status.idle":"2024-02-18T04:18:35.960002Z","shell.execute_reply.started":"2024-02-18T04:18:35.863888Z","shell.execute_reply":"2024-02-18T04:18:35.959011Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### LOAD MONET AND PHOTOS DATASETS","metadata":{}},{"cell_type":"code","source":"\"\"\"\nCode Notes:\nio  -->  \"Input/Output\", provides the interface between the program and the operating system\ngfile  --> \"Generic File\", provides an API that is close to Python's file object, but based on TensorFlow's C++ FileSystem API\nglob --> finds all the pathnames matching a specified pattern according to the rules used by the Unix shell\n\"\"\"\nGCS_PATH = KaggleDatasets().get_gcs_path()\n\nmonet_filenames = tf.io.gfile.glob(str(GCS_PATH + '/monet_tfrec/*.tfrec'))\nprint('Monet TFRecord Files:', len(monet_filenames))\nphotos_filenames = tf.io.gfile.glob(str(GCS_PATH + '/photo_tfrec/*.tfrec'))\nprint(\"Photos TFRecord Files:\", len(photos_filenames))\n\nmonet_dataset = load_dataset(monet_filenames)\nphotos_dataset = load_dataset(photos_filenames)\n\nmonet_originals = load_dataset(monet_filenames)\nphoto_originals = load_dataset(photos_filenames)","metadata":{"execution":{"iopub.status.busy":"2024-02-18T04:18:35.962032Z","iopub.execute_input":"2024-02-18T04:18:35.962321Z","iopub.status.idle":"2024-02-18T04:18:36.631359Z","shell.execute_reply.started":"2024-02-18T04:18:35.962296Z","shell.execute_reply":"2024-02-18T04:18:36.630452Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### DATA AUGMENTATION\n\nMy first training attempts transferred convincing monet-like styling to photos, but the generated images had clear artifacts. Faint gridlines and other subtle abnormalities were added to most photos. Large monochromatic areas like skies and bodies of water had a tendancy to pick up chattery patterns that humans would cleary recognize as \"not-Monet\". In an attempt to reduce these artifacts and improve the model fit, I augmented both datasets by jittering images (enlarge and randomly crop), randomly flipping and rotating images, and randomly tweaking the contrast and brightness. These approaches should help with general model training becuase they effectively increase the size of both datasets, and the monet dataset is otherwise small at 300 images. These methods seemed to reduce the artifacts, but did not eliminate them. ","metadata":{}},{"cell_type":"code","source":"# Random Jitter\ndef resize(image, height, width):\n    image = tf.image.resize(image, [height, width], method=tf.image.ResizeMethod.NEAREST_NEIGHBOR)\n    return image\n\ndef random_crop(image):\n    cropped_image = tf.image.random_crop(image, size=[256, 256, 3])\n    return cropped_image\n\n@tf.function()\ndef RandomJitter(image):\n    image = resize(image, 286, 286)\n    image = random_crop(image) # back to 256 x 256 x 3\n    image = tf.image.random_flip_left_right(image)\n\n    return image\n\n# Data augmentation\ndata_augmentation = Sequential([\n    RandomFlip(\"horizontal_and_vertical\"),\n    RandomRotation(factor=0.2), # '.2' means .2 * pie = 36 degrees\n    RandomZoom(height_factor=0.2),\n    RandomBrightness(factor=0.2, value_range=[0.0, 1.0]),\n    RandomContrast(factor=0.2),\n    Lambda(RandomJitter)\n])\n\ndef apply_augmentation(image):\n  image = data_augmentation(image, training=True)\n  return image\n\nAUTOTUNE = tf.data.AUTOTUNE # For dataset configuration\n\nmonet_dataset = monet_dataset.map(apply_augmentation, num_parallel_calls = AUTOTUNE)\nphotos_dataset = photos_dataset.map(apply_augmentation, num_parallel_calls = AUTOTUNE)","metadata":{"id":"-cV9GH5-GAUT","outputId":"6155bacd-d08a-444f-9ece-be7513c6e5d2","execution":{"iopub.status.busy":"2024-02-18T04:18:36.632546Z","iopub.execute_input":"2024-02-18T04:18:36.632832Z","iopub.status.idle":"2024-02-18T04:18:37.810007Z","shell.execute_reply.started":"2024-02-18T04:18:36.632807Z","shell.execute_reply":"2024-02-18T04:18:37.809011Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### BATCH AND BUFFER THE DATASETS\nNormally, I would use a batch size based on a power of 2 for computational efficiency. Batch sizes of 32, 64, 128, and 256 are fairly standard. However, for tasks involving fine-grained updates, small batches allow for more frequent updates and can help convergence. Batch sizes as small as 1 can help the CycleGAN learn mappings between individual images from the two domains. \n\nAlso, our CycleGAN model architecture requires the monet and photo batches to be the same size. With 300 and 7028 elements in the monet and photos datasets, batch size of 1, 2, and 4 would ensure equal batch sizes in both datasets and avoid dropped remainders. Other CycleGANs have succesfully used a batch size of 1, so we'll use the same.\n\nA buffer size of 1024 should be large enough to ensure good randomization of image order from one epoch to the next, and minimize slowdowns.","metadata":{}},{"cell_type":"markdown","source":"### TESTING AND TROUBLESHOOTING CODE (Uncomment as needed)\nWhile troubleshooting, I found it helpful to reduce the dataset sizes to 10 each. This temporary measures allowed me to itterate through changes without waiting a long time to see the output or discover an error.","metadata":{}},{"cell_type":"code","source":"photos_dataset = photos_dataset.take(5)\nmonet_dataset = monet_dataset.take(5)","metadata":{"execution":{"iopub.status.busy":"2024-02-18T04:18:37.814237Z","iopub.execute_input":"2024-02-18T04:18:37.814789Z","iopub.status.idle":"2024-02-18T04:18:37.824401Z","shell.execute_reply.started":"2024-02-18T04:18:37.814763Z","shell.execute_reply":"2024-02-18T04:18:37.823493Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Batch and shuffle the dataset\nBUFFER_SIZE = 1024\nBATCH_SIZE = 1\n\nmonet_dataset = monet_dataset.batch(BATCH_SIZE, drop_remainder=True).shuffle(BUFFER_SIZE)\nphotos_dataset = photos_dataset.batch(BATCH_SIZE, drop_remainder=True).shuffle(BUFFER_SIZE)\nphoto_originals = photo_originals.batch(BATCH_SIZE, drop_remainder=True).shuffle(BUFFER_SIZE) # For illustrating style-transfer at the notebook's end.\n\n# Apply prefetching with AUTOTUNE at the end of the pipeline\nmonet_dataset = monet_dataset.prefetch(AUTOTUNE)\nphotos_dataset = photos_dataset.prefetch(AUTOTUNE)","metadata":{"execution":{"iopub.status.busy":"2024-02-18T04:18:37.825477Z","iopub.execute_input":"2024-02-18T04:18:37.825737Z","iopub.status.idle":"2024-02-18T04:18:37.847975Z","shell.execute_reply.started":"2024-02-18T04:18:37.825716Z","shell.execute_reply":"2024-02-18T04:18:37.847120Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### SAMPLE MONET AND PHOTO\n\nTensorFlow operates in eager execution mode, which means operations return concerete values instead of computational graphs. This means TensorFlow tensors can be direclty accessed and manipulated by other APIs, like NumPy and Matplotlib, as demonstrated by the plots below.","metadata":{"id":"N3J8MPvX74fi"}},{"cell_type":"code","source":"example_monet = next(iter(monet_dataset))\nexample_photo = next(iter(photos_dataset))\n\nprint(f'Monet and Photo data type: {type(example_monet)}')\n\nplt.subplot(121)\nplt.title('Monet')\nplt.imshow(example_monet[0] * 0.5 + 0.5);\n\nplt.subplot(122)\nplt.title('Photo')\nplt.imshow(example_photo[0] * 0.5 + 0.5);","metadata":{"id":"ceynFVAR74fi","outputId":"21a0521b-41d6-41f9-b1c9-f9fae433f1a6","execution":{"iopub.status.busy":"2024-02-18T04:18:37.849075Z","iopub.execute_input":"2024-02-18T04:18:37.849407Z","iopub.status.idle":"2024-02-18T04:18:40.187264Z","shell.execute_reply.started":"2024-02-18T04:18:37.849375Z","shell.execute_reply":"2024-02-18T04:18:40.186347Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# DESIGN CYCLEGAN\n\nMy strategy is to use a UNET architecture to build a generator and discriminator. UNET architecture is utilized in the TensorFlow guides for [CycleGAN](https://www.tensorflow.org/tutorials/generative/cyclegan) and [pix2pix](https://www.tensorflow.org/tutorials/generative/pix2pix) and is described in [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://lmb.informatik.uni-freiburg.de/Publications/2015/RFB15a/)\n\nA small difference between my model and those in the aforementioned TensorFlow guides is that we use instance normalization instead of batch normalization. Instance normalization performs normalization for each specific instance (image or photo) independently. It computes the mean and variance for each channel in each training example. Instance normalization is often used in style transfer models, where the style of individual images can matter more than the average style of a batch of images.\n\nUnfotunately, Keras does not have a built-in instance normalization layer. While `tfa.layers.InstanceNormalization` is available from the TensorFlow Addons library, the module is no longer maintained. To future proof the code, I will use GroupNormalization and set the group size equal to the number of input channels. With this setting, GroupNormalization is effectively instance normalization, as each channel of each image will be normalized to a mean of 0 and standard deviation of 1.\n\nBuilding the CycleGAN generator will involve multiple downsample blocks to encode basic features and multiple upsample blocks to generate the output. To simplify the code, I create downsample and upsample funtions that combine standard layers (Conv2D/Conv2DTranspose, GroupNormalization, Dropout, and LeakyReLu/Relu) into blocks. The blocks are shorter and easier to work with, and still allow specification of filter number and kernel size. This technique is demonstrated in the [Pix2Pixtutorial](https://colab.research.google.com/corgiredirector?site=https%3A%2F%2Fwww.tensorflow.org%2Ftutorials%2Fgenerative%2Fpix2pix) and by Amy Jang in her [tutorial](https://www.kaggle.com/code/amyjang/monet-cyclegan-tutorial).","metadata":{"id":"YF1ERL8o74fi"}},{"cell_type":"markdown","source":"### DOWNSAMPLE FUNCTION","metadata":{"id":"ojZaqLg-74fi"}},{"cell_type":"code","source":"OUTPUT_CHANNELS = 3\n\ndef downsample(filters, kernel_size=4, strides=2, apply_instancenorm=True):\n    initializer = tf.random_normal_initializer(0., 0.02)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    result = Sequential()\n    result.add(Conv2D(filters,\n                      kernel_size,\n                      strides=strides,\n                      padding='same',\n                      kernel_initializer=initializer,\n                      use_bias=False))\n\n    if apply_instancenorm:\n        result.add(GroupNormalization(gamma_initializer=gamma_init,\n                                      groups=filters))\n\n    result.add(LeakyReLU())\n\n    return result","metadata":{"id":"SDhaapZD74fi","execution":{"iopub.status.busy":"2024-02-18T04:18:40.188657Z","iopub.execute_input":"2024-02-18T04:18:40.189083Z","iopub.status.idle":"2024-02-18T04:18:40.196999Z","shell.execute_reply.started":"2024-02-18T04:18:40.189050Z","shell.execute_reply":"2024-02-18T04:18:40.195996Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### UPSAMPLE FUNCTION","metadata":{"id":"VyMxjyKs74fi"}},{"cell_type":"code","source":"def upsample(filters, kernel_size=4, strides=2, apply_dropout=False):\n    initializer = tf.random_normal_initializer(0., 0.02)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    result = keras.Sequential()\n    result.add(Conv2DTranspose(filters,\n                               kernel_size,\n                               strides=strides,\n                               padding='same',\n                               kernel_initializer=initializer,\n                               use_bias=False))\n\n    result.add(GroupNormalization(gamma_initializer=gamma_init,\n                                  groups=filters))\n\n    if apply_dropout:\n        result.add(Dropout(0.5))\n\n    result.add(ReLU())\n\n    return result","metadata":{"id":"1pHJQggQ74fi","execution":{"iopub.status.busy":"2024-02-18T04:18:40.202196Z","iopub.execute_input":"2024-02-18T04:18:40.202741Z","iopub.status.idle":"2024-02-18T04:18:40.226096Z","shell.execute_reply.started":"2024-02-18T04:18:40.202710Z","shell.execute_reply":"2024-02-18T04:18:40.225226Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### GENERATOR FUNCTION\n\nIn the generator function, each downsample and upsample block is composed of a 2D convolutional (down) or 2D transpose convolutional layer (up), a group normalization layer, a dropout layer, and a ReLU or LeakyReLU activation function. The final generator layer is a convolutional layer with a tanh activation function. The generator function also uses **skip connections** to concatenate identically shaped outputs from the downsample and upsample blocks. Skip connections help to preserve the original image information and reduce the vanishing gradient problem. These connections \"skip\" layers to concatenate input and output layers. The skip connections are visible as right arcing lines in the architectural diagram that follows the function definitions.","metadata":{"id":"LNm_HmAA74fi"}},{"cell_type":"code","source":"def Generator():\n    inputs = Input(shape=[256,256,3])\n\n    # bs = batch size\n    down_stack = [\n        downsample(filters=64,\n                   apply_instancenorm=False),   # output dim = (bs, 128, 128, 64)\n        downsample(128), # output dim = (bs, 64, 64, 128)\n        downsample(256), # output dim = (bs, 32, 32, 256)\n        downsample(512), # output dim = (bs, 16, 16, 512)\n        downsample(512), # output dim = (bs, 8, 8, 512)\n        downsample(512), # output dim = (bs, 4, 4, 512)\n        downsample(512), # output dim = (bs, 2, 2, 512)\n        downsample(512), # output dim = (bs, 1, 1, 512)\n    ]\n    \n    # The output of each upsample layer is double the number of filters, because\n    # the skip connections concatenate the output of the downsample layer to  \n    # a corresponding upsample layer\n    \n    up_stack = [\n        upsample(filters=512,\n                 apply_dropout=True), # concat'd output dim = (bs, 2, 2, 1024)\n        upsample(512, apply_dropout=True), # concat'd output dim = (bs, 4, 4, 1024)\n        upsample(512, apply_dropout=True), # concat'd output dim = (bs, 8, 8, 1024)\n        upsample(512), # concat'd output dim = (bs, 16, 16, 1024)\n        upsample(256), # concat'd output dim = (bs, 32, 32, 512)\n        upsample(128), # concat'd output dim = (bs, 64, 64, 256)\n        upsample(64), # concat'd output dim = (bs, 128, 128, 128)\n    ]\n\n    initializer = tf.random_normal_initializer(0., 0.02)\n    last = Conv2DTranspose(OUTPUT_CHANNELS,\n                           kernel_size = 4,\n                           strides=2,\n                           padding='same',\n                           kernel_initializer=initializer,\n                           activation='tanh') # (bs, 256, 256, 3)\n\n    x = inputs\n\n    # Downsampling through the model\n    skips = []\n    for down in down_stack:\n        x = down(x)\n        skips.append(x)\n\n    skips = reversed(skips[:-1])\n\n    # Upsampling and creating skip connections\n    for up, skip in zip(up_stack, skips):\n        x = up(x)\n        x = Concatenate()([x, skip])\n\n    x = last(x)\n\n    return keras.Model(inputs=inputs, outputs=x)","metadata":{"id":"cSEVUHaT74fi","execution":{"iopub.status.busy":"2024-02-18T04:18:40.227363Z","iopub.execute_input":"2024-02-18T04:18:40.227644Z","iopub.status.idle":"2024-02-18T04:18:40.244032Z","shell.execute_reply.started":"2024-02-18T04:18:40.227621Z","shell.execute_reply":"2024-02-18T04:18:40.243300Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### DISCRIMINATOR\nIn the discriminator function, each downsample block is composed of a convolutional layer, a group normalization layer, a dropout layer, and a LeakyReLU activation function.","metadata":{"id":"XmJy9nni74fj"}},{"cell_type":"code","source":"def Discriminator():\n    initializer = tf.random_normal_initializer(0., 0.02)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    inp = Input(shape=[256, 256, 3], name='input_image')\n\n    x = inp\n\n    down1 = downsample(filters=64,\n                       apply_instancenorm=False)(x)\n    down2 = downsample(128)(down1)\n    down3 = downsample(256)(down2)\n\n    zero_pad1 = ZeroPadding2D()(down3)\n\n    conv = Conv2D(filters=512,\n                         kernel_size=4,\n                         strides=1,\n                         kernel_initializer=initializer,\n                         use_bias=False)(zero_pad1)\n\n    norm1 = GroupNormalization(gamma_initializer=gamma_init,\n                               groups=512)(conv)\n\n    leaky_relu = LeakyReLU()(norm1)\n\n    zero_pad2 = ZeroPadding2D()(leaky_relu)\n\n    last = Conv2D(filters=1, kernel_size=4, strides=1,\n                         kernel_initializer=initializer)(zero_pad2)\n\n    return Model(inputs=inp, outputs=last)","metadata":{"id":"T5Bp1Syx74fj","execution":{"iopub.status.busy":"2024-02-18T04:18:40.245040Z","iopub.execute_input":"2024-02-18T04:18:40.245294Z","iopub.status.idle":"2024-02-18T04:18:40.274206Z","shell.execute_reply.started":"2024-02-18T04:18:40.245272Z","shell.execute_reply":"2024-02-18T04:18:40.273473Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### APPLY DISTRIBUTION STRATEGY TO MODELS\n","metadata":{"id":"75pH5HHK74fj"}},{"cell_type":"code","source":"with strategy.scope():\n    monet_generator = Generator()\n    photo_generator = Generator()\n\n    monet_discriminator = Discriminator()\n    photo_discriminator = Discriminator()","metadata":{"id":"CPkS62fE74fj","execution":{"iopub.status.busy":"2024-02-18T04:18:40.275227Z","iopub.execute_input":"2024-02-18T04:18:40.275479Z","iopub.status.idle":"2024-02-18T04:18:45.025247Z","shell.execute_reply.started":"2024-02-18T04:18:40.275449Z","shell.execute_reply":"2024-02-18T04:18:45.024260Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### PHOTO AND MONET-LIKE FROM UNTRAINED GENERATOR","metadata":{}},{"cell_type":"code","source":"example_photo = next(iter(photos_dataset))\nmonet_like = monet_generator(example_photo)\n\nplt.subplot(121)\nplt.title(\"Example Photo\")\nplt.imshow(example_photo[0] * 0.5 + 0.5)\n\n\nplt.subplot(122)\nplt.title(\"Monet-Like \\n(from untrained generator)\")\nplt.imshow(monet_like[0] * 0.5 + 0.5)\n\n","metadata":{"execution":{"iopub.status.busy":"2024-02-18T04:18:45.026497Z","iopub.execute_input":"2024-02-18T04:18:45.026786Z","iopub.status.idle":"2024-02-18T04:18:49.043348Z","shell.execute_reply.started":"2024-02-18T04:18:45.026762Z","shell.execute_reply":"2024-02-18T04:18:49.042391Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ARCHITECTURE DIAGRAM & SUMMARY","metadata":{"id":"8oHBDo8BdV4C"}},{"cell_type":"markdown","source":"### GENERATOR ARCHITECTURE","metadata":{"id":"WBZWqU1VDfBf"}},{"cell_type":"code","source":"generator = Generator()\ntf.keras.utils.plot_model(generator, show_shapes=True, dpi=64)","metadata":{"id":"9d5Ro8QMczwr","outputId":"d82d6350-c39a-4262-c760-a882c1b806be","execution":{"iopub.status.busy":"2024-02-18T04:18:49.044546Z","iopub.execute_input":"2024-02-18T04:18:49.044834Z","iopub.status.idle":"2024-02-18T04:18:51.221707Z","shell.execute_reply.started":"2024-02-18T04:18:49.044809Z","shell.execute_reply":"2024-02-18T04:18:51.220754Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### GENERATOR SUMMARY","metadata":{"id":"OR1YEvzZ74fj"}},{"cell_type":"code","source":"print(\"Monet Generator Model Summary:\", monet_generator.summary())","metadata":{"id":"qexvMZxe74fj","outputId":"f750ff97-d819-4c15-9dce-477027d037ea","execution":{"iopub.status.busy":"2024-02-18T04:18:51.222850Z","iopub.execute_input":"2024-02-18T04:18:51.223153Z","iopub.status.idle":"2024-02-18T04:18:51.287298Z","shell.execute_reply.started":"2024-02-18T04:18:51.223128Z","shell.execute_reply":"2024-02-18T04:18:51.284619Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### DISCRIMINATOR ARCHITECTURE","metadata":{"id":"dBlWH6LNDhfA"}},{"cell_type":"code","source":"discriminator = Discriminator()\ntf.keras.utils.plot_model(discriminator, show_shapes=True, dpi=64)","metadata":{"id":"o5FAttupdbLy","execution":{"iopub.status.busy":"2024-02-18T04:18:51.288323Z","iopub.execute_input":"2024-02-18T04:18:51.288601Z","iopub.status.idle":"2024-02-18T04:18:51.711022Z","shell.execute_reply.started":"2024-02-18T04:18:51.288578Z","shell.execute_reply":"2024-02-18T04:18:51.710007Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### DISCRIMINATOR SUMMARY","metadata":{"id":"bgGEKEesEJBi"}},{"cell_type":"code","source":"print(\"Monet Discriminator Model Summary:\", monet_discriminator.summary())","metadata":{"id":"alZa-MZvcbqV","execution":{"iopub.status.busy":"2024-02-18T04:18:51.712158Z","iopub.execute_input":"2024-02-18T04:18:51.712433Z","iopub.status.idle":"2024-02-18T04:18:51.743824Z","shell.execute_reply.started":"2024-02-18T04:18:51.712409Z","shell.execute_reply":"2024-02-18T04:18:51.742944Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# CLASS CYCLGAN (MODEL ENCAPSULATION)\n\nHere I define a class \"CycleGan\", which will construct a TensorFlow model from the generators and discriminator functions defined earlier. Below are important Python and object-oriented-progreamming concepts represented in this code.\n\n1) The CycleGan class **encapsulates** everything related to the CycleGAN model, including the generator and discriminator components and how compilation and training are performed.\n\n2) `class CycleGan(keras.Model)` sets up **inheritance** of methods from keras.Model. Instances of the CycleGan class will have fit(), evaluate(), and predict() methods from Keras, among others.\n\n3) The **super()** command is used to call specific methods from the keras.Models superclass. The inheritance described in 2) makes the keras.Model methods available, but the super() command applies those methods in specific functions within the CycleGan class. In this case, the two super() commands ensure that the initialization (__init__) and compilation (compile) logic from the keras.Model superclass are integrated into CycleGan.  \n\n4) The 'train_step' method is an example of **polymorphism**, where `train_step` from the keras.Model is overridden by the custom `train_step` code written into this class definition.","metadata":{"id":"Fv3rAo7_74fj"}},{"cell_type":"code","source":"class CycleGan(keras.Model):\n    def __init__(\n        self,\n        monet_generator,\n        photo_generator,\n        monet_discriminator,\n        photo_discriminator,\n        lambda_cycle=10,\n    ):\n        super().__init__()\n        self.m_gen = monet_generator\n        self.p_gen = photo_generator\n        self.m_disc = monet_discriminator\n        self.p_disc = photo_discriminator\n        self.lambda_cycle = lambda_cycle\n\n    def compile(\n        self,\n        m_gen_optimizer,\n        p_gen_optimizer,\n        m_disc_optimizer,\n        p_disc_optimizer,\n        gen_loss_fn,\n        disc_loss_fn,\n        cycle_loss_fn,\n        identity_loss_fn\n    ):\n        super().compile()\n        self.m_gen_optimizer = m_gen_optimizer\n        self.p_gen_optimizer = p_gen_optimizer\n        self.m_disc_optimizer = m_disc_optimizer\n        self.p_disc_optimizer = p_disc_optimizer\n        self.gen_loss_fn = gen_loss_fn\n        self.disc_loss_fn = disc_loss_fn\n        self.cycle_loss_fn = cycle_loss_fn\n        self.identity_loss_fn = identity_loss_fn\n\n    def train_step(self, batch_data):\n        real_monet, real_photo = batch_data\n\n        with tf.GradientTape(persistent=True) as tape:\n            # generating a cycled photo: photo --> fake monet --> photo\n            fake_monet = self.m_gen(real_photo, training=True)\n            cycled_photo = self.p_gen(fake_monet, training=True)\n\n            # generating a cycled monet: monet --> fake photo --> monet\n            fake_photo = self.p_gen(real_monet, training=True)\n            cycled_monet = self.m_gen(fake_photo, training=True)\n\n            # generating same monet and same photo\n            same_monet = self.m_gen(real_monet, training=True)\n            same_photo = self.p_gen(real_photo, training=True)\n\n            # discriminating real images\n            disc_real_monet = self.m_disc(real_monet, training=True)\n            disc_real_photo = self.p_disc(real_photo, training=True)\n\n            # discriminating fake images\n            disc_fake_monet = self.m_disc(fake_monet, training=True)\n            disc_fake_photo = self.p_disc(fake_photo, training=True)\n\n            # generator losses on fake images\n            monet_gen_loss = self.gen_loss_fn(disc_fake_monet)\n            photo_gen_loss = self.gen_loss_fn(disc_fake_photo)\n\n            # total cycle consistency loss\n            total_cycle_loss = self.cycle_loss_fn(real_monet, cycled_monet, self.lambda_cycle) + self.cycle_loss_fn(real_photo, cycled_photo, self.lambda_cycle)\n\n            # total generator losses\n            total_monet_gen_loss = monet_gen_loss + total_cycle_loss + self.identity_loss_fn(real_monet, same_monet, self.lambda_cycle)\n            total_photo_gen_loss = photo_gen_loss + total_cycle_loss + self.identity_loss_fn(real_photo, same_photo, self.lambda_cycle)\n\n            # discriminator losses\n            monet_disc_loss = self.disc_loss_fn(disc_real_monet, disc_fake_monet)\n            photo_disc_loss = self.disc_loss_fn(disc_real_photo, disc_fake_photo)\n\n        # Calculate the gradients for generator and discriminator\n        monet_generator_gradients = tape.gradient(total_monet_gen_loss,\n                                                  self.m_gen.trainable_variables)\n        photo_generator_gradients = tape.gradient(total_photo_gen_loss,\n                                                  self.p_gen.trainable_variables)\n\n        monet_discriminator_gradients = tape.gradient(monet_disc_loss,\n                                                      self.m_disc.trainable_variables)\n        photo_discriminator_gradients = tape.gradient(photo_disc_loss,\n                                                      self.p_disc.trainable_variables)\n\n        # Apply the gradients to the optimizer\n        self.m_gen_optimizer.apply_gradients(zip(monet_generator_gradients,\n                                                 self.m_gen.trainable_variables))\n\n        self.p_gen_optimizer.apply_gradients(zip(photo_generator_gradients,\n                                                 self.p_gen.trainable_variables))\n\n        self.m_disc_optimizer.apply_gradients(zip(monet_discriminator_gradients,\n                                                  self.m_disc.trainable_variables))\n\n        self.p_disc_optimizer.apply_gradients(zip(photo_discriminator_gradients,\n                                                  self.p_disc.trainable_variables))\n\n\n        return {\n            \"monet_gen_loss\": total_monet_gen_loss,\n            \"photo_gen_loss\": total_photo_gen_loss,\n            \"monet_disc_loss\": monet_disc_loss,\n            \"photo_disc_loss\": photo_disc_loss\n        }","metadata":{"id":"Yo5fk4F474fj","execution":{"iopub.status.busy":"2024-02-18T04:18:51.745250Z","iopub.execute_input":"2024-02-18T04:18:51.745633Z","iopub.status.idle":"2024-02-18T04:18:51.766555Z","shell.execute_reply.started":"2024-02-18T04:18:51.745599Z","shell.execute_reply":"2024-02-18T04:18:51.765718Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# LOSS FUNCTIONS AND OPTIMIZERS\nLoss metrics for CycleGANs. include generator and discriminator losses, cycle consistency loss and identity loss. Cycle consistency loss measures how well an input image is reconstructed after a round trip transformation, e.g. monet --> photo --> monet. Identity loss measures how much an image changes when passesd through the generator of its own domain, e.g. monet --> monet.\n\n`strategy.scope()` tells TensorFlow to distribute the model's operations according to the strategy defined in the \"CONFIGURE TENSORFLOW\" section near the beginning of this document.","metadata":{"id":"zIpV_Mru74fj"}},{"cell_type":"code","source":"with strategy.scope():\n    def discriminator_loss(real, generated):\n        real_loss = BinaryCrossentropy(from_logits=True, reduction=tf.keras.losses.Reduction.NONE)(tf.ones_like(real), real)\n        generated_loss = BinaryCrossentropy(from_logits=True, reduction=tf.keras.losses.Reduction.NONE)(tf.zeros_like(generated), generated)\n        total_disc_loss = real_loss + generated_loss\n        \n        return tf.reduce_mean(total_disc_loss) * 0.5 # Reduce to scalar for TensorBoard or plotting loss vs epoch\n#         return total_disc_loss * 0.5 # Keep as an array if not using TensorBoard or plotting loss vs epoch","metadata":{"id":"JdFUJHCy74fj","execution":{"iopub.status.busy":"2024-02-18T04:18:51.768045Z","iopub.execute_input":"2024-02-18T04:18:51.768362Z","iopub.status.idle":"2024-02-18T04:18:51.782109Z","shell.execute_reply.started":"2024-02-18T04:18:51.768329Z","shell.execute_reply":"2024-02-18T04:18:51.781143Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with strategy.scope():\n    def generator_loss(generated):\n        \n        return tf.reduce_mean(BinaryCrossentropy(from_logits=True,reduction=tf.keras.losses.Reduction.NONE)\n                         (tf.ones_like(generated), generated)) # Reduce to scalar for TensorBoard\n#         return (BinaryCrossentropy(from_logits=True, reduction=tf.keras.losses.Reduction.NONE)\n#                 (tf.ones_like(generated), generated))# Keep as an array if not using TensorBoard\n\n","metadata":{"id":"CYzwOboE74fj","execution":{"iopub.status.busy":"2024-02-18T04:18:51.783206Z","iopub.execute_input":"2024-02-18T04:18:51.783494Z","iopub.status.idle":"2024-02-18T04:18:51.794236Z","shell.execute_reply.started":"2024-02-18T04:18:51.783464Z","shell.execute_reply":"2024-02-18T04:18:51.793235Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with strategy.scope():\n    def calc_cycle_loss(real_image, cycled_image, LAMBDA):\n        loss1 = tf.reduce_mean(tf.abs(real_image - cycled_image))\n\n        return LAMBDA * loss1","metadata":{"id":"iKVnGGOk74fj","execution":{"iopub.status.busy":"2024-02-18T04:18:51.795329Z","iopub.execute_input":"2024-02-18T04:18:51.795601Z","iopub.status.idle":"2024-02-18T04:18:51.805406Z","shell.execute_reply.started":"2024-02-18T04:18:51.795578Z","shell.execute_reply":"2024-02-18T04:18:51.804475Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with strategy.scope():\n    def identity_loss(real_image, same_image, LAMBDA):\n        loss = tf.reduce_mean(tf.abs(real_image - same_image))\n        return LAMBDA * 0.5 * loss","metadata":{"id":"VeJWQZYJ74fk","execution":{"iopub.status.busy":"2024-02-18T04:18:51.806433Z","iopub.execute_input":"2024-02-18T04:18:51.806983Z","iopub.status.idle":"2024-02-18T04:18:51.816477Z","shell.execute_reply.started":"2024-02-18T04:18:51.806951Z","shell.execute_reply":"2024-02-18T04:18:51.815772Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Use `legacy.Adam` when training on M1 laptop. Use `Adam` when training on Google Colab or Kaggle Notebook\nwith strategy.scope():\n    monet_generator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n    photo_generator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n\n    monet_discriminator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n    photo_discriminator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)","metadata":{"id":"X43qaIo874fk","execution":{"iopub.status.busy":"2024-02-18T04:18:51.817785Z","iopub.execute_input":"2024-02-18T04:18:51.818077Z","iopub.status.idle":"2024-02-18T04:18:51.836425Z","shell.execute_reply.started":"2024-02-18T04:18:51.818054Z","shell.execute_reply":"2024-02-18T04:18:51.835503Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# INSTANTIATE MODEL","metadata":{"id":"1m0KoxBCdElG"}},{"cell_type":"code","source":"with strategy.scope():\n    cycle_gan_model = CycleGan(\n        monet_generator, photo_generator, monet_discriminator, photo_discriminator\n    )\n\n    cycle_gan_model.compile(\n        m_gen_optimizer = monet_generator_optimizer,\n        p_gen_optimizer = photo_generator_optimizer,\n        m_disc_optimizer = monet_discriminator_optimizer,\n        p_disc_optimizer = photo_discriminator_optimizer,\n        gen_loss_fn = generator_loss,\n        disc_loss_fn = discriminator_loss,\n        cycle_loss_fn = calc_cycle_loss,\n        identity_loss_fn = identity_loss\n    )","metadata":{"id":"Nsm46R1S74fk","execution":{"iopub.status.busy":"2024-02-18T04:18:51.837623Z","iopub.execute_input":"2024-02-18T04:18:51.838129Z","iopub.status.idle":"2024-02-18T04:18:51.863791Z","shell.execute_reply.started":"2024-02-18T04:18:51.838098Z","shell.execute_reply":"2024-02-18T04:18:51.863055Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# FIT CYCLGAN\n\nBelow, I setup callbacks that save the model weights every 10 epochs. This isn't necessary, but I'd like to see examples of Monet-like images produced by the generator trained to different epochs. I'm hoping the Monet-like images will be noticeably improved across the different epochs, as this would help determine when the model is sufficiently trained. Knowing when the model is appropriately trained is difficult due to the four CycleGAN losses that represent competition between generating and disciminating images. I don't believe that any single loss is sufficiently informative to guide training, and even considering the losses in aggregate may not suffice. My reasoning is that while the Monet generator improves, so should the Monet Discriminator. If the generator and disciminator improve in roughly equal proportions, the losses may change very little or unpredictably. The same should be true for the photo generator and disciminator. However, the generator losses include cycle loss and identity loss, which (theoretically) can be reduced with training, even if the generator and disciminator remain balanced. In short, scrutiny of the losses might, or might not allow us to pinpoint when training should stop.","metadata":{"id":"Q8wvQFTSeNsW"}},{"cell_type":"code","source":"# Set up checkpoint callback\ncheckpoint_path = 'checkpoints/'\ncheckpoint_name = 'cp-{epoch:04d}.ckpt'\nfilepath = checkpoint_path + checkpoint_name\ncheckpoint_callback = ModelCheckpoint(\n    filepath=filepath,\n    verbose=0,\n    save_weights_only=True,\n    save_freq=12000) # Save CKPT every 40 epochs (1 epoch = 300 batches)\n#     save_freq=400)  # TROUBLESHOOTING -> Save CKPT every 40 epochs (1 epoch = 10 batches)\n    \n# If checkpoints exist, load the latest one\nlatest_checkpoint = tf.train.latest_checkpoint(checkpoint_path)\nif latest_checkpoint:\n    cycle_gan_model.load_weights(latest_checkpoint)\n    print(f\"Loaded weights from {latest_checkpoint}\")\nelse:\n    print(\"No checkpoint found. Starting training from scratch.\")\n    ","metadata":{"id":"602y1moV9Ohr","outputId":"4491af72-efad-4098-eca7-f65167b340ec","execution":{"iopub.status.busy":"2024-02-18T04:18:51.864923Z","iopub.execute_input":"2024-02-18T04:18:51.865545Z","iopub.status.idle":"2024-02-18T04:18:51.872307Z","shell.execute_reply.started":"2024-02-18T04:18:51.865514Z","shell.execute_reply":"2024-02-18T04:18:51.871509Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history = cycle_gan_model.fit(\n    tf.data.Dataset.zip((monet_dataset, photos_dataset)),\n    verbose=0,\n    epochs=160,\n    callbacks = [checkpoint_callback]\n)","metadata":{"id":"gYKB399c74fk","outputId":"0f56423f-d5b2-45ba-fc2f-414e099c7a7e","execution":{"iopub.status.busy":"2024-02-18T04:18:51.873585Z","iopub.execute_input":"2024-02-18T04:18:51.874088Z","iopub.status.idle":"2024-02-18T04:30:27.981322Z","shell.execute_reply.started":"2024-02-18T04:18:51.874064Z","shell.execute_reply":"2024-02-18T04:30:27.980497Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# MODEL EVALUATION\n\nThe line graph below shows dramatic loss oscillations for the Monet and photo generators. From about epochs 10 to 40, the generator losses and disciminator losses seem to move in opposite direction. Presumabely, improvements in disciminator function result in higher generator loss, and vice versa. The ideal number of training epochs is not obvious. While it might be tempting to choose an epoch with the lowest generator loss, a change in generator loss could represent diminished generator performance, or an improvement in discriminator performance. I've submitted notebooks with 25 and 50 training epochs thus far and obtained Kaggle scores (Memorization-Informated Frechet Inception Distance) of 71.9 and 73.0 respectively. Is this difference purely random, or is 50 epochs too many?  Hard to say!\n\nFor this last submission, I used `tf.reduce_mean` in the generator and discriminator loss calculations to produce plottable scalar values, as shown below. I don't expect the change to have a large impact on training, but comparing this submission to earlier submissions won't be exactly \"apples-to-apples\". I will train to 40 epochs before generating the Monet-like images for submission.","metadata":{}},{"cell_type":"code","source":"plt.plot(cycle_gan_model.history.history['monet_gen_loss'], label = 'Monet generator')\nplt.plot(cycle_gan_model.history.history['monet_disc_loss'], label = 'Monet discriminator')\nplt.plot(cycle_gan_model.history.history['photo_gen_loss'], label = 'photo generator')\nplt.plot(cycle_gan_model.history.history['photo_disc_loss'], label = 'photo discriminator')\nplt.title(\"Model Loss over epochs\")\nplt.xlabel(\"Epoch\")\nplt.ylabel('Loss')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-02-18T04:30:27.986263Z","iopub.execute_input":"2024-02-18T04:30:27.986555Z","iopub.status.idle":"2024-02-18T04:30:28.286731Z","shell.execute_reply.started":"2024-02-18T04:30:27.986530Z","shell.execute_reply":"2024-02-18T04:30:28.285671Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### MONET-LIKE IMAGES AFTER 40, 80, 120 and 160 TRAINING EPOCH\nMemorization-Informed Frechet Inception Distance (MiFID) is the competition metric. However, I'd like to see if noticeable differences exist in the quality of style transfer at different timepoints in training. The plot below shows monet-like generated images from the CycleGAN model trained to 40, 80, 120 and 160 epochs.\n","metadata":{}},{"cell_type":"code","source":"fix, ax = plt.subplots(5, 5, figsize=(20,20))\n\nphotos = []\nfor i, photo in enumerate(photo_originals.take(5)):\n    photos.append(photo)\n    photo = (photo[0]*127.5 + 127.5).numpy().astype(np.uint8)\n    ax[i, 0].imshow(photo)\n    ax[i, 0].set_title('Original Photo')\n    ax[i, 0].axis('off')\n\nfor col, ckpt_num in enumerate(['0040','0080', '0120', '0160']):\n    checkpoint_name = 'checkpoints/cp-' + ckpt_num + '.ckpt'\n    cycle_gan_model.load_weights(checkpoint_name)\n    for i, photo in enumerate(photos):\n        monet_like = cycle_gan_model.m_gen(photo, training= False)[0].numpy()   \n        monet_like = (monet_like*127.5 + 127.5).astype(np.uint8)\n        ax[i, col+1].imshow(monet_like)\n        ax[i, col+1].set_title('Monet-like (epoch ' + ckpt_num + ')')\n        ax[i, col+1].axis('off')\n","metadata":{"execution":{"iopub.status.busy":"2024-02-18T04:41:25.689967Z","iopub.execute_input":"2024-02-18T04:41:25.690313Z","iopub.status.idle":"2024-02-18T04:41:40.076659Z","shell.execute_reply.started":"2024-02-18T04:41:25.690289Z","shell.execute_reply":"2024-02-18T04:41:40.075086Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### FAKE PHOTOS GENERATED FROM MONETS","metadata":{}},{"cell_type":"code","source":"fix, ax = plt.subplots(5, 5, figsize=(20,20))\n\nmonets = []\nfor i, monet in enumerate(monet_originals.take(5)):\n    monets.append(monet)\n    monet = (monet[0]*127.5 + 127.5).numpy().astype(np.uint8)\n    ax[i, 0].imshow(monet)\n    ax[i, 0].set_title('Original Monet')\n    ax[i, 0].axis('off')\n\nfor col, ckpt_num in enumerate(['0040','0080', '0120', '0160']):\n    checkpoint_name = 'checkpoints/cp-' + ckpt_num + '.ckpt'\n    cycle_gan_model.load_weights(checkpoint_name)\n    for i, monet in enumerate(monets):\n        fake_photo = cycle_gan_model.m_gen(monet, training= False)[0].numpy()   \n        fake_photo = (fake_photo*127.5 + 127.5).astype(np.uint8)\n        ax[i, col+1].imshow(fake_photo)\n        ax[i, col+1].set_title('fake_photo (epoch ' + ckpt_num + ')')\n        ax[i, col+1].axis('off')\n","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# GENERATE & ZIP MONETS FOR KAGGLE\nLastly, I use the trained CycleGAN Monet generator to transfer styling to the 7028 original photos. Following the competition recommendation, I avoid saving individual image files, which could exceed the Kaggle limits on output file number. Instead, the Monet-like images are saved into a zip file as they are produced. \n\nThanks for reading my notebook and offerring any comments or suggestions!","metadata":{}},{"cell_type":"code","source":"# import os\n# import zipfile\n# import PIL\n# from PIL import Image\n# import io\n\n# zip_filename = '/kaggle/working/images.zip'\n\n# with zipfile.ZipFile(zip_filename, 'a') as myzip:\n#     i = 1\n#     for batch in photo_originals:\n#         for image in (batch):\n            \n#             fake_monet = cycle_gan_model.m_gen(tf.expand_dims(image, 0), training=False)[0].numpy()\n#             fake_monet = (fake_monet * 127.5 + 127.5).astype(np.uint8)\n            \n#             fake_monet = PIL.Image.fromarray(fake_monet)\n            \n#             img_byte_arr = io.BytesIO()\n#             fake_monet.save(img_byte_arr, format='JPEG', quality=95)\n#             img_byte_arr = img_byte_arr.getvalue()\n            \n#             fake_monet_filename = str(i) +'.jpg'\n#             myzip.writestr(fake_monet_filename, img_byte_arr)\n            \n#             i += 1","metadata":{"id":"00Tg52gY5Dfo","execution":{"iopub.status.busy":"2024-02-18T04:30:37.839517Z","iopub.status.idle":"2024-02-18T04:30:37.839891Z","shell.execute_reply.started":"2024-02-18T04:30:37.839712Z","shell.execute_reply":"2024-02-18T04:30:37.839727Z"},"trusted":true},"execution_count":null,"outputs":[]}]}