{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import tensorflow as tf\nfrom tensorflow import keras\nfrom tensorflow.keras import layers\nimport tensorflow_addons as tfa\n\nfrom kaggle_datasets import KaggleDatasets\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport os, re\nimport pandas as pd","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Notes\n\nIn this notebook I implemented a CycleGAN Network with real faces and faces from japanese paintings in the 16th and 17th century. \n\nI web-scraped the Kaokore images from a list of URL's, but I just had to convert the real faces from an existing dataset. Links to the notebooks where I created each of these TFrecords are linked below.\n\n-- \n\nFeel free to train the CycleGAN from scratch, but I have also provided an option to use existing model weights below! Also dont forget to turn on TPU's! \n\nThanks for coming across my notebook, hope you like it!\n\n\nTFrecords Datasets:\n\n- [Kaokore Faces TFrecords](https://www.kaggle.com/brendanartley/kaokore-faces)\n\n- [Flickr-Faces-TFrecords- 512x512](https://www.kaggle.com/brendanartley/flickrfacestfrecords)\n\nOriginal Datasets:\n\n- [Arnaud58's Flickr Faces Data](https://www.kaggle.com/arnaud58/flickrfaceshq-dataset-ffhq)\n- [Kaokore Dataset](https://github.com/rois-codh/kaokore)\n"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","collapsed":true,"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":false},"cell_type":"markdown","source":"### Detecting TPU"},{"metadata":{"trusted":true},"cell_type":"code","source":"try:\n    tpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n    print('Device:', tpu.master())\n    tf.config.experimental_connect_to_cluster(tpu)\n    tf.tpu.experimental.initialize_tpu_system(tpu)\n    strategy = tf.distribute.experimental.TPUStrategy(tpu)\nexcept:\n    strategy = tf.distribute.get_strategy()\nprint('Number of replicas:', strategy.num_replicas_in_sync)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Variables"},{"metadata":{"trusted":true},"cell_type":"code","source":"KAOKORE_GCS_PATH = KaggleDatasets().get_gcs_path('kaokore-faces')\nFLICKR_GCS_PATH = KaggleDatasets().get_gcs_path('flickrfacestfrecords')\nAUTOTUNE = tf.data.experimental.AUTOTUNE\nBATCH_SIZE = 16 * strategy.num_replicas_in_sync\n\n\nIMAGE_SIZE = [256,256]\nOUTPUT_CHANNELS = 3","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#this function counts number of images in all TFRecords\ndef count_data_items(filenames):\n    n = [int(re.compile(r'-([0-9]*)\\.').search(filename).group(1)) for filename in filenames]\n    return np.sum(n)\n\nKAOKORE_FILENAMES = tf.io.gfile.glob(str(KAOKORE_GCS_PATH + '/*.tfrec'))\nFLICKR_FILENAMES = tf.io.gfile.glob(str(FLICKR_GCS_PATH + '/*.tfrec'))\n\nNUM_KAOKORE_IMAGES = count_data_items(KAOKORE_FILENAMES)\nNUM_FLICKR_IMAGES = count_data_items(FLICKR_FILENAMES)\n\nprint('KAOKORE Face Images: {}\\nFlickr Face Images: {}'.format(NUM_KAOKORE_IMAGES, NUM_FLICKR_IMAGES))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Simple Augmentation\n\nAdding some simple rotatation and flips. "},{"metadata":{},"cell_type":"markdown","source":"### Loading Functions"},{"metadata":{"trusted":true},"cell_type":"code","source":"def decode_image(image):\n    image = tf.image.decode_jpeg(image, channels=3)\n    image = (tf.cast(image, tf.float32) / 127.5) - 1\n    image = tf.image.resize(image, [*IMAGE_SIZE])\n    image = tf.reshape(image, [*IMAGE_SIZE, 3]) #asterix before removes list brackets\n    return image\n\ndef read_tfrecord(example):\n    tfrecord_format = {\n        \"image\": tf.io.FixedLenFeature([], tf.string),\n        \"image_id\": tf.io.FixedLenFeature([], tf.string),\n    }\n    example = tf.io.parse_single_example(example, tfrecord_format)\n    image = decode_image(example['image'])\n    return image","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_dataset(filenames, labeled=True, ordered=False):\n    dataset = tf.data.TFRecordDataset(filenames)\n    dataset = dataset.map(read_tfrecord, num_parallel_calls=AUTOTUNE)\n    dataset = dataset.repeat()\n    dataset = dataset.shuffle(2048)\n    return dataset","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Visualizing Image Samples\n\nPlotting examples of the TFrecords. Also good checkpoint to know that the input images are formatted correctly before training the model."},{"metadata":{"trusted":true},"cell_type":"code","source":"#re-run cell to get different variations of photo and kaokore paintings\nkaokore_ds = load_dataset(KAOKORE_FILENAMES, labeled=True).batch(1)\nflickr_ds = load_dataset(FLICKR_FILENAMES, labeled=True).batch(1)\n\nexample_kaokore = next(iter(kaokore_ds))\nexample_flickr = next(iter(flickr_ds))\n\nplt.subplots(2,2,figsize=(10,10))\n\nplt.subplot(121)\nplt.title('Real Face')\nplt.imshow(example_flickr[0] * 0.5 + 0.5)\n\nplt.subplot(122)\nplt.title('Kaokore Face')\nplt.imshow(example_kaokore[0] * 0.5 + 0.5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Building Generator\n\nDownsample - Reduces the image dimensions by the stride length in the Conv2D layer.\n\nUpsample - Increases the image dimensions by using the Conv2DTranspose layer which bascically does the reverse of the conv2d layer.\n\nNOTE: Got the structure for upsample and downsample layers from Amy Jang.\n\n- [Amy Jang's GAN Tutorial](https://www.kaggle.com/amyjang/monet-cyclegan-tutorial)"},{"metadata":{"trusted":true},"cell_type":"code","source":"def downsample(filters, size, apply_instancenorm=True):\n    initializer = tf.random_normal_initializer(0., 0.02)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    result = keras.Sequential()\n    result.add(layers.Conv2D(filters, size, strides=2, padding='same',\n                             kernel_initializer=initializer, use_bias=False))\n\n    if apply_instancenorm:\n        result.add(tfa.layers.InstanceNormalization(gamma_initializer=gamma_init))\n\n    result.add(layers.LeakyReLU()) #anything below zero is zero\n\n    return result\n\ndef upsample(filters, size, apply_dropout=False):\n    initializer = tf.random_normal_initializer(0., 0.02)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    result = keras.Sequential()\n    result.add(layers.Conv2DTranspose(filters, size, strides=2,\n                                      padding='same',\n                                      kernel_initializer=initializer,\n                                      use_bias=False))\n\n    result.add(tfa.layers.InstanceNormalization(gamma_initializer=gamma_init))\n\n    if apply_dropout:\n        result.add(layers.Dropout(0.5))\n\n    result.add(layers.ReLU())\n\n    return result","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In the next cell we define out generator model.\n\nThe generator is downsampling our images and then upsampling whilst adding some long skip connections.\n\nSkip connections are a strategy used to combat the vanishing gradient that often occurs in GAN's. They\nbasically skip some layers in the neural network and feed the output to layers further along the model. \n\nSee here for skip connection info -> [link](https://theaisummer.com/skip-connections/#:~:text=At%20present%2C%20skip%20connection%20is,module%20in%20many%20convolutional%20architectures.&text=Skip%20connections%20in%20deep%20architectures,of%20only%20the%20next%20one)"},{"metadata":{"trusted":true},"cell_type":"code","source":"def Generator():\n    inputs = layers.Input(shape=[256,256,3])\n\n    # bs = batch size\n    down_stack = [\n        downsample(64, 4, apply_instancenorm=False), # (bs, 128, 128, 64)\n        downsample(128, 4), # (bs, 64, 64, 128)\n        downsample(256, 4), # (bs, 32, 32, 256)\n        downsample(512, 4), # (bs, 16, 16, 512)\n        downsample(512, 4), # (bs, 8, 8, 512)\n        downsample(512, 4), # (bs, 4, 4, 512)\n        downsample(512, 4), # (bs, 2, 2, 512)\n        downsample(512, 4), # (bs, 1, 1, 512)\n    ]\n\n    up_stack = [\n        upsample(512, 4, apply_dropout=True), # (bs, 2, 2, 1024)\n        upsample(512, 4, apply_dropout=True), # (bs, 4, 4, 1024)\n        upsample(512, 4, apply_dropout=True), # (bs, 8, 8, 1024)\n        upsample(512, 4), # (bs, 16, 16, 1024)\n        upsample(256, 4), # (bs, 32, 32, 512)\n        upsample(128, 4), # (bs, 64, 64, 256)\n        upsample(64, 4), # (bs, 128, 128, 128)\n    ]\n\n    initializer = tf.random_normal_initializer(0., 0.02)\n    last = layers.Conv2DTranspose(OUTPUT_CHANNELS, 4,\n                                  strides=2,\n                                  padding='same',\n                                  kernel_initializer=initializer,\n                                  activation='tanh') # (bs, 256, 256, 3)\n\n    x = inputs\n\n    # Downsampling through the model\n    skips = []\n    for down in down_stack:\n        x = down(x)\n        skips.append(x)\n\n    skips = reversed(skips[:-1])\n\n    # Upsampling and establishing the skip connections\n    for up, skip in zip(up_stack, skips):\n        x = up(x)\n        x = layers.Concatenate()([x, skip])\n\n    x = last(x)\n\n    return keras.Model(inputs=inputs, outputs=x)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Building the discriminator\n\nThe discriminator outputs a small 2D image with high pixel values if it thinks the image is real, and outputs a small 2D image with low pixel values if it thinks that the image is made by the generator.\n\nThese outputs are what help us determine how effective the disriminators and generators truly are."},{"metadata":{"trusted":true},"cell_type":"code","source":"def Discriminator():\n    initializer = tf.random_normal_initializer(0., 0.02)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    inp = layers.Input(shape=[256, 256, 3], name='input_image')\n\n    x = inp\n\n    down1 = downsample(64, 4, False)(x) # (bs, 128, 128, 64)\n    down2 = downsample(128, 4)(down1) # (bs, 64, 64, 128)\n    down3 = downsample(256, 4)(down2) # (bs, 32, 32, 256)\n\n    zero_pad1 = layers.ZeroPadding2D()(down3) # (bs, 34, 34, 256)\n    conv = layers.Conv2D(512, 4, strides=1,\n                         kernel_initializer=initializer,\n                         use_bias=False)(zero_pad1) # (bs, 31, 31, 512)\n\n    norm1 = tfa.layers.InstanceNormalization(gamma_initializer=gamma_init)(conv)\n\n    leaky_relu = layers.LeakyReLU()(norm1)\n\n    zero_pad2 = layers.ZeroPadding2D()(leaky_relu) # (bs, 33, 33, 512)\n\n    last = layers.Conv2D(1, 4, strides=1,\n                         kernel_initializer=initializer)(zero_pad2) # (bs, 30, 30, 1)\n\n    return tf.keras.Model(inputs=inp, outputs=last)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Defining Genenerators and Discriminators\n\nDefining model within strategy.scope to train cycleGAN on TPU's."},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    kaokore_generator = Generator() # transforms photos to kaokore-esque paintings\n    photo_generator = Generator() # transforms kaokore paintings to be more like photos\n\n    kaokore_discriminator = Discriminator() # differentiates real kaokore painting and generated kaokore painting\n    photo_discriminator = Discriminator() # differentiates real photos and generated photos","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The generator is not trained yet and therefore isnt outputting anything remotely close to what we are looking for at the moment. \n\nThe hope is that the generator learns to beat the disriminator by generating and image from the original photo that represents the face of a 16th century japanese face."},{"metadata":{"trusted":true},"cell_type":"code","source":"to_kaokore = kaokore_generator(example_flickr)\n\nplt.subplots(2,2,figsize=(10,10))\n\nplt.subplot(1, 2, 1)\nplt.title(\"Original Photo\")\nplt.imshow(example_flickr[0] * 0.5 + 0.5)\n\nplt.subplot(1, 2, 2)\nplt.title(\"Kaokore-esque Photo\")\nplt.imshow(to_kaokore[0] * 0.5 + 0.5)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Defining Model SubClass\n\nIn the next cell we are building a subclass of tf.keras.model.\n\nDuring each training step the model is transforming a photo to a kaokore painting, and then transforming back into a photo. We therefore are training both the kaokore_generator and the photo_generator at the same time.\n\nThe difference between the input image and the resulting image is measured by the cycle-consistency loss which is a custom loss function defined later in this notebook."},{"metadata":{"trusted":true},"cell_type":"code","source":"class CycleGan(keras.Model):\n    def __init__(\n        self,\n        kaokore_generator,\n        photo_generator,\n        kaokore_discriminator,\n        photo_discriminator,\n        lambda_cycle=10,\n    ):\n        super(CycleGan, self).__init__()\n        self.k_gen = kaokore_generator\n        self.p_gen = photo_generator\n        self.k_disc = kaokore_discriminator\n        self.p_disc = photo_discriminator\n        self.lambda_cycle = lambda_cycle\n        \n    def compile(\n        self,\n        k_gen_optimizer,\n        p_gen_optimizer,\n        k_disc_optimizer,\n        p_disc_optimizer,\n        gen_loss_fn,\n        disc_loss_fn,\n        cycle_loss_fn,\n        identity_loss_fn\n    ):\n        super(CycleGan, self).compile()\n        self.k_gen_optimizer = k_gen_optimizer\n        self.p_gen_optimizer = p_gen_optimizer\n        self.k_disc_optimizer = k_disc_optimizer\n        self.p_disc_optimizer = p_disc_optimizer\n        self.gen_loss_fn = gen_loss_fn\n        self.disc_loss_fn = disc_loss_fn\n        self.cycle_loss_fn = cycle_loss_fn\n        self.identity_loss_fn = identity_loss_fn\n        \n    def train_step(self, batch_data):\n        real_kaokore, real_photo = batch_data\n        \n        with tf.GradientTape(persistent=True) as tape:\n            # photo to kaokore back to photo\n            fake_kaokore = self.k_gen(real_photo, training=True)\n            cycled_photo = self.p_gen(fake_kaokore, training=True)\n\n            # kaokore to photo back to kaokore\n            fake_photo = self.p_gen(real_kaokore, training=True)\n            cycled_kaokore = self.k_gen(fake_photo, training=True)\n\n            # generating itself\n            same_kaokore = self.k_gen(real_kaokore, training=True)\n            same_photo = self.p_gen(real_photo, training=True)\n\n            # discriminator used to check, inputing real images\n            disc_real_kaokore = self.k_disc(real_kaokore, training=True)\n            disc_real_photo = self.p_disc(real_photo, training=True)\n\n            # discriminator used to check, inputing fake images\n            disc_fake_kaokore = self.k_disc(fake_kaokore, training=True)\n            disc_fake_photo = self.p_disc(fake_photo, training=True)\n\n            # evaluates generator loss\n            kaokore_gen_loss = self.gen_loss_fn(disc_fake_kaokore)\n            photo_gen_loss = self.gen_loss_fn(disc_fake_photo)\n\n            # evaluates total cycle consistency loss\n            total_cycle_loss = self.cycle_loss_fn(real_kaokore, cycled_kaokore, self.lambda_cycle) + self.cycle_loss_fn(real_photo, cycled_photo, self.lambda_cycle)\n\n            # evaluates total generator loss\n            total_kaokore_gen_loss = kaokore_gen_loss + total_cycle_loss + self.identity_loss_fn(real_kaokore, same_kaokore, self.lambda_cycle)\n            total_photo_gen_loss = photo_gen_loss + total_cycle_loss + self.identity_loss_fn(real_photo, same_photo, self.lambda_cycle)\n\n            # evaluates discriminator loss\n            kaokore_disc_loss = self.disc_loss_fn(disc_real_kaokore, disc_fake_kaokore)\n            photo_disc_loss = self.disc_loss_fn(disc_real_photo, disc_fake_photo)\n\n        # Calculate the gradients for generator and discriminator\n        kaokore_generator_gradients = tape.gradient(total_kaokore_gen_loss,\n                                                  self.k_gen.trainable_variables)\n        photo_generator_gradients = tape.gradient(total_photo_gen_loss,\n                                                  self.p_gen.trainable_variables)\n\n        kaokore_discriminator_gradients = tape.gradient(kaokore_disc_loss,\n                                                      self.k_disc.trainable_variables)\n        photo_discriminator_gradients = tape.gradient(photo_disc_loss,\n                                                      self.p_disc.trainable_variables)\n\n        # Apply the gradients to the optimizer\n        self.k_gen_optimizer.apply_gradients(zip(kaokore_generator_gradients,\n                                                 self.k_gen.trainable_variables))\n\n        self.p_gen_optimizer.apply_gradients(zip(photo_generator_gradients,\n                                                 self.p_gen.trainable_variables))\n\n        self.k_disc_optimizer.apply_gradients(zip(kaokore_discriminator_gradients,\n                                                  self.k_disc.trainable_variables))\n\n        self.p_disc_optimizer.apply_gradients(zip(photo_discriminator_gradients,\n                                                  self.p_disc.trainable_variables))\n        \n        return {\n            \"kaokore_gen_loss\": total_kaokore_gen_loss,\n            \"photo_gen_loss\": total_photo_gen_loss,\n            \"kaokore_disc_loss\": kaokore_disc_loss,\n            \"photo_disc_loss\": photo_disc_loss\n        }","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Defining Custom Loss Functions\nThe discriminator loss function below compares real images to a matrix of 1s and fake images to a matrix of 0s. \n\nThe perfect discriminator will output all 1s for real images and all 0s for fake images.\n\nThe discriminator loss output is the average of the real and generated loss."},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    def discriminator_loss(real, generated):\n        real_loss = tf.keras.losses.BinaryCrossentropy(from_logits=True, reduction=tf.keras.losses.Reduction.NONE)(tf.ones_like(real), real)\n\n        generated_loss = tf.keras.losses.BinaryCrossentropy(from_logits=True, reduction=tf.keras.losses.Reduction.NONE)(tf.zeros_like(generated), generated)\n\n        total_disc_loss = real_loss + generated_loss\n\n        return total_disc_loss * 0.5","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The generator wants to fool the discriminator into thinking the generated image is real. The perfect generator will have the discriminator output only 1s. \n\nThus, it compares the generated image to a matrix of 1s to find the loss.\n\nThis constant battle between the discriminator and generator is why training a GAN is much harder than a single NN."},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    def generator_loss(generated):\n        return tf.keras.losses.BinaryCrossentropy(from_logits=True, reduction=tf.keras.losses.Reduction.NONE)(tf.ones_like(generated), generated)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In the cycleGAN two transformations occur ie.\n\nPhoto (input) -> Kaokore (generated) -> Photo (generated)\n\nThe following calc_cycle_loss is the difference between the real image, and the image that has gone through our model."},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    def calc_cycle_loss(real_image, cycled_image, LAMBDA):\n        loss1 = tf.reduce_mean(tf.abs(real_image - cycled_image))\n\n        return LAMBDA * loss1","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The final loss function we have is the identity_loss which compares the input of the generator with the output of the generator. (ie difference between input photo + generated photo)"},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    def identity_loss(real_image, same_image, LAMBDA):\n        loss = tf.reduce_mean(tf.abs(real_image - same_image))\n        return LAMBDA * 0.5 * loss","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Compile + Train CycleGAN\n\n\nWe are using the Adam optimizer for both the discriminator and the generators here. \n\nMore information in regards to each parameter for Adam can be found in TensorFlow docs here --> [link](https://www.tensorflow.org/api_docs/python/tf/keras/optimizers/Adam)"},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    kaokore_generator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n    photo_generator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n\n    kaokore_discriminator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n    photo_discriminator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In the cell below we can define our generators and discriminators, and compile our model. All the tedious code has been completed above and the following is relatively simple.\n\nNote if we wanted to alter the loss functions we would have to change the subclass code, and the respective loss functions above. \n\nOther TF-GAN Loss functions to try: --> [link](https://developers.google.com/machine-learning/gan/loss)\n\n- Minimax Loss, Modified Minimax Los, Wasserstein Loss"},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    cycle_gan_model = CycleGan(\n        kaokore_generator, photo_generator, kaokore_discriminator, photo_discriminator\n    )\n\n    cycle_gan_model.compile(\n        k_gen_optimizer = kaokore_generator_optimizer,\n        p_gen_optimizer = photo_generator_optimizer,\n        k_disc_optimizer = kaokore_discriminator_optimizer,\n        p_disc_optimizer = photo_discriminator_optimizer,\n        gen_loss_fn = generator_loss,\n        disc_loss_fn = discriminator_loss,\n        cycle_loss_fn = calc_cycle_loss,\n        identity_loss_fn = identity_loss\n    )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"I included the steps_per_epoch as the NUM_KAOKORE_IMAGES/8 here or 1106.\n\nSince we have such a large number of images in each dataset there will be considerable advantage in training the GAN for a longer time period. \n\nI started off training the model with the following fit parameters. (4hrs on TPU's)\n\n- 100 epochs \n- 1106 steps_per_epoch."},{"metadata":{"trusted":true},"cell_type":"code","source":"history = cycle_gan_model.fit(\n            tf.data.Dataset.zip((kaokore_ds, flickr_ds)),\n            epochs=100,\n            steps_per_epoch=NUM_KAOKORE_IMAGES/8,\n        )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cycle_gan_model.save_weights('./CycleGAN_trained_TPU_model_weights.h5')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Pre-Trained Weights\n\nIf you dont want to train the model from scratch, I have saved pretrained model weights to a datasets so that you can just visualize the work of the kaokore_generator."},{"metadata":{"trusted":true},"cell_type":"code","source":"#.built allows for the cutstom model weights to be loaded into the model\n# cycle_gan_model.built = True\n# cycle_gan_model.load_weights('../input/cyclegan-kaokore-model-weights/CycleGAN_trained_TPU_model_weights.h5')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Visualize Generated KaoKore Faces\n\nThere are roughly 52,000 images that could randomly be selected here. Some of the generated images come out great and other may not look as good. \n\nSince this is a CycleGAN and we have two generator we can also do this the other way around. Simply replace the name of the generator and image_ds to do! \n\nNote: the photo_generator is much harder to train!\n\nAlso, feel free to re-run the following kernel as many times as you would like to get a different set of images!"},{"metadata":{"trusted":true},"cell_type":"code","source":"_, ax = plt.subplots(6, 2, figsize=(15, 30))\nfor i, img in enumerate(flickr_ds.take(6)):\n    prediction = kaokore_generator(img, training=False)[0].numpy()\n    prediction = (prediction * 127.5 + 127.5).astype(np.uint8)\n    img = (img[0] * 127.5 + 127.5).numpy().astype(np.uint8)\n\n    ax[i, 0].imshow(img)\n    ax[i, 1].imshow(prediction)\n    ax[i, 0].set_title(\"Flickr Photo\")\n    ax[i, 1].set_title(\"Kaokore-esque\")\n    ax[i, 0].axis(\"off\")\n    ax[i, 1].axis(\"off\")","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}