{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"#### What are you trying to do in this notebook?\nWe recognize the works of artists through their unique style, such as color choices or brush strokes. Artists like Claude Monet can now be imitated with algorithms thanks to generative adversarial networks (GANs). In this competition, I will bring that style to my photos or recreate the style from scratch!\n\nComputer vision has advanced tremendously in recent years and GANs are now capable of mimicking objects in a very convincing way. But creating museum-worthy masterpieces is thought of to be, well, more art than science. So can (data) science, in the form of GANs, trick classifiers into believing I’ve created a true Monet? That’s the challenge I’ll take on!\n\n#### Why are you trying it?\nA GAN consists of at least two neural networks: a generator model and a discriminator model. The generator is a neural network that creates the images. For this competition, I generate images in the style of Monet. This generator is trained using a discriminator. The two models will work against each other, with the generator trying to trick the discriminator, and the discriminator trying to accurately classify the real vs. generated images.\n\nMy task is to build a GAN that generates 7,000 to 10,000 Monet-style images.","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:10:56.430453Z","iopub.execute_input":"2021-12-27T02:10:56.431610Z","iopub.status.idle":"2021-12-27T02:11:03.710273Z","shell.execute_reply.started":"2021-12-27T02:10:56.431478Z","shell.execute_reply":"2021-12-27T02:11:03.709286Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"BATCH_SIZE = 4\nimport re\nimport os\nimport numpy as np\nimport pandas as pd\nimport tensorflow as tf\nfrom tensorflow import keras\nfrom tensorflow.keras import layers\nimport tensorflow_addons as tfa\nimport matplotlib.pyplot as plt\nimport matplotlib.image as mpimg\nfrom tensorflow.keras import Model, losses, optimizers\nfrom tensorflow.keras.callbacks import Callback\nfrom kaggle_datasets import KaggleDatasets","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:03.712965Z","iopub.execute_input":"2021-12-27T02:11:03.713205Z","iopub.status.idle":"2021-12-27T02:11:10.149430Z","shell.execute_reply.started":"2021-12-27T02:11:03.713176Z","shell.execute_reply":"2021-12-27T02:11:10.148277Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"try:\n    tpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n    print('Device:', tpu.master())\n    tf.config.experimental_connect_to_cluster(tpu)\n    tf.tpu.experimental.initialize_tpu_system(tpu)\n    tpu_strategy = tf.distribute.experimental.TPUStrategy(tpu)\nexcept:\n    tpu_strategy = tf.distribute.get_strategy()\nprint('Number of replicas:', tpu_strategy.num_replicas_in_sync)\n\nAUTO_TUNE = tf.data.experimental.AUTOTUNE\n    \nprint(\"version:\",tf.__version__)","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:10.150798Z","iopub.execute_input":"2021-12-27T02:11:10.151127Z","iopub.status.idle":"2021-12-27T02:11:16.699869Z","shell.execute_reply.started":"2021-12-27T02:11:10.151091Z","shell.execute_reply":"2021-12-27T02:11:16.698567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"GCS_PATH = KaggleDatasets().get_gcs_path('gan-getting-started')\n\nmonets_tfr = tf.io.gfile.glob(str(GCS_PATH + '/monet_tfrec/*.tfrec'))\nphotos_tfr = tf.io.gfile.glob(str(GCS_PATH + '/photo_tfrec/*.tfrec'))","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:16.701519Z","iopub.execute_input":"2021-12-27T02:11:16.701761Z","iopub.status.idle":"2021-12-27T02:11:17.441533Z","shell.execute_reply.started":"2021-12-27T02:11:16.701733Z","shell.execute_reply":"2021-12-27T02:11:17.440660Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def count_data_items(filenames):\n    n = [int(re.compile(r\"-([0-9]*)\\.\").search(filename).group(1)) for filename in filenames]\n    return np.sum(n)\n\nmonet_jpg = count_data_items(monets_tfr)\nphoto_jpg = count_data_items(photos_tfr)\n\nEPOCHS = 30\n\nprint(\"Monet TFRecord files:\", len(monets_tfr))\nprint(\"Monet image files:\", monet_jpg)\nprint(\"Photo TFRecord files:\", len(photos_tfr))\nprint(\"Photo image files:\", photo_jpg)\nprint(\"EPOCHS:\",EPOCHS)","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:17.443587Z","iopub.execute_input":"2021-12-27T02:11:17.444081Z","iopub.status.idle":"2021-12-27T02:11:17.455625Z","shell.execute_reply.started":"2021-12-27T02:11:17.444043Z","shell.execute_reply":"2021-12-27T02:11:17.454671Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"IMAGE_SIZE = [256, 256]","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:17.456918Z","iopub.execute_input":"2021-12-27T02:11:17.457350Z","iopub.status.idle":"2021-12-27T02:11:17.469717Z","shell.execute_reply.started":"2021-12-27T02:11:17.457318Z","shell.execute_reply":"2021-12-27T02:11:17.468849Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def decode_image(image):\n    image = tf.image.decode_jpeg(image, channels=3)\n    image = (tf.cast(image, tf.float32) / 127.5) - 1\n    image = tf.reshape(image, [*IMAGE_SIZE, 3])\n    return image","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:17.470815Z","iopub.execute_input":"2021-12-27T02:11:17.471125Z","iopub.status.idle":"2021-12-27T02:11:17.482483Z","shell.execute_reply.started":"2021-12-27T02:11:17.471064Z","shell.execute_reply":"2021-12-27T02:11:17.481515Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def read_tfrecord(example):\n    tfrecord_format = {\n        \"image_name\": tf.io.FixedLenFeature([], tf.string),\n        \"image\": tf.io.FixedLenFeature([], tf.string),\n        \"target\": tf.io.FixedLenFeature([], tf.string)\n    }\n    example = tf.io.parse_single_example(example, tfrecord_format)\n    image = decode_image(example['image'])\n    return image","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:17.483791Z","iopub.execute_input":"2021-12-27T02:11:17.484048Z","iopub.status.idle":"2021-12-27T02:11:17.495667Z","shell.execute_reply.started":"2021-12-27T02:11:17.484008Z","shell.execute_reply":"2021-12-27T02:11:17.494923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def data_augment(image):\n    p_rotate = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    p_spatial = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    p_crop = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    if p_crop > .5:\n        image = tf.image.resize(image, [286, 286]) #resizing to 286 x 286 x 3\n        image = tf.image.random_crop(image, size=[256, 256, 3]) # randomly cropping to 256 x 256 x 3\n        if p_crop > .9:\n            image = tf.image.resize(image, [300, 300])\n            image = tf.image.random_crop(image, size=[256, 256, 3])\n    \n    if p_rotate > .9:\n        image = tf.image.rot90(image, k=3) # rotate 270º\n    elif p_rotate > .7:\n        image = tf.image.rot90(image, k=2) # rotate 180º\n    elif p_rotate > .5:\n        image = tf.image.rot90(image, k=1) # rotate 90º\n        \n        ## random mirroring\n    if p_spatial > .6:\n        image = tf.image.random_flip_left_right(image)\n        image = tf.image.random_flip_up_down(image)\n        if p_spatial > .9:\n            image = tf.image.transpose(image)\n    \n    return image","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:17.497327Z","iopub.execute_input":"2021-12-27T02:11:17.498172Z","iopub.status.idle":"2021-12-27T02:11:17.510822Z","shell.execute_reply.started":"2021-12-27T02:11:17.498122Z","shell.execute_reply":"2021-12-27T02:11:17.510057Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def load_dataset(filenames):\n    data = tf.data.TFRecordDataset(filenames)\n    data = data.map(read_tfrecord, num_parallel_calls=AUTO_TUNE)\n    return data","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:17.512349Z","iopub.execute_input":"2021-12-27T02:11:17.513239Z","iopub.status.idle":"2021-12-27T02:11:17.528066Z","shell.execute_reply.started":"2021-12-27T02:11:17.513195Z","shell.execute_reply":"2021-12-27T02:11:17.527348Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_gan_dataset(monet_files, photo_files, augment=None, repeat=True, shuffle=True, batch_size=1):\n\n    monet_ds = load_dataset(monet_files)\n    photo_ds = load_dataset(photo_files)\n    \n    if augment:\n        monet_ds = monet_ds.map(augment, num_parallel_calls=AUTO_TUNE)\n        photo_ds = photo_ds.map(augment, num_parallel_calls=AUTO_TUNE)\n        \n    if repeat:\n        monet_ds = monet_ds.repeat()\n        photo_ds = photo_ds.repeat()\n    if shuffle:\n        monet_ds = monet_ds.shuffle(2048)\n        photo_ds = photo_ds.shuffle(2048)\n        \n    monet_ds = monet_ds.batch(batch_size, drop_remainder=True)\n    photo_ds = photo_ds.batch(batch_size, drop_remainder=True)\n    monet_ds = monet_ds.cache()\n    photo_ds = photo_ds.cache()\n    monet_ds = monet_ds.prefetch(AUTO_TUNE)\n    photo_ds = photo_ds.prefetch(AUTO_TUNE)\n    \n    gan_ds = tf.data.Dataset.zip((monet_ds, photo_ds))\n    \n    return gan_ds\n\ndata = get_gan_dataset(monets_tfr, photos_tfr, augment=data_augment, repeat=True, shuffle=True, batch_size=BATCH_SIZE)","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:17.529749Z","iopub.execute_input":"2021-12-27T02:11:17.530038Z","iopub.status.idle":"2021-12-27T02:11:18.572170Z","shell.execute_reply.started":"2021-12-27T02:11:17.530006Z","shell.execute_reply":"2021-12-27T02:11:18.571209Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"example_monet , example_photo = next(iter(data))","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:18.573453Z","iopub.execute_input":"2021-12-27T02:11:18.573707Z","iopub.status.idle":"2021-12-27T02:11:24.916788Z","shell.execute_reply.started":"2021-12-27T02:11:18.573676Z","shell.execute_reply":"2021-12-27T02:11:24.915731Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Visualizing the real photo\nplt.subplot(121)\nplt.title('Real photo')\nplt.imshow(example_photo[2] * 0.5 + 0.5)\n\n# Visualizing the Monet painting\nplt.subplot(122)\nplt.title('Monet painting')\nplt.imshow(example_monet[2]* 0.5 + 0.5)","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:24.918544Z","iopub.execute_input":"2021-12-27T02:11:24.918795Z","iopub.status.idle":"2021-12-27T02:11:25.950098Z","shell.execute_reply.started":"2021-12-27T02:11:24.918764Z","shell.execute_reply":"2021-12-27T02:11:25.948917Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def display_samples(dataset, nrows, ncols):\n    ds_iter = iter(dataset)\n    plt.figure(figsize=(15, int(15*nrows/ncols)))\n    for j in range(nrows*ncols):\n        monet_sample = next(ds_iter)\n        plt.subplot(nrows,ncols,j+1)\n        plt.axis('off')\n        plt.imshow(monet_sample[0] * 0.5 + 0.5)\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:25.955228Z","iopub.execute_input":"2021-12-27T02:11:25.955646Z","iopub.status.idle":"2021-12-27T02:11:25.964285Z","shell.execute_reply.started":"2021-12-27T02:11:25.955592Z","shell.execute_reply":"2021-12-27T02:11:25.963230Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_samples(load_dataset(monets_tfr).batch(1), 5, 4)","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:25.965988Z","iopub.execute_input":"2021-12-27T02:11:25.966464Z","iopub.status.idle":"2021-12-27T02:11:28.095732Z","shell.execute_reply.started":"2021-12-27T02:11:25.966416Z","shell.execute_reply":"2021-12-27T02:11:28.094077Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def downsample(filters, size, apply_instancenorm=True):\n    initializer = tf.random_normal_initializer(0., 0.04)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n     \n    result = keras.Sequential()\n    # Convolutional layer\n    result.add(layers.Conv2D(filters, size, strides=2, padding='same',\n                             kernel_initializer=initializer, use_bias=False))\n # Normalization layer\n    if apply_instancenorm:\n        result.add(tfa.layers.InstanceNormalization(gamma_initializer=gamma_init))\n # Activation layer\n    result.add(layers.LeakyReLU())\n\n    return result","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:28.097049Z","iopub.execute_input":"2021-12-27T02:11:28.097299Z","iopub.status.idle":"2021-12-27T02:11:28.104708Z","shell.execute_reply.started":"2021-12-27T02:11:28.097268Z","shell.execute_reply":"2021-12-27T02:11:28.103860Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def upsample(filters, size, apply_dropout=False):\n     # Normalization layer\n    initializer = tf.random_normal_initializer(0., 0.04)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n     # Transpose convolutional layer\n    result = keras.Sequential()\n    result.add(layers.Conv2DTranspose(filters, size, strides=2,\n                                      padding='same',\n                                      kernel_initializer=initializer,\n                                      use_bias=False))\n#Instance Normalization\n    result.add(tfa.layers.InstanceNormalization(gamma_initializer=gamma_init))\n# Dropout layer\n    if apply_dropout:\n        result.add(layers.Dropout(0.5))\n# Activation layer\n    result.add(layers.ReLU())\n\n    return result","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:28.105997Z","iopub.execute_input":"2021-12-27T02:11:28.106321Z","iopub.status.idle":"2021-12-27T02:11:28.123148Z","shell.execute_reply.started":"2021-12-27T02:11:28.106284Z","shell.execute_reply":"2021-12-27T02:11:28.122239Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"OUTPUT_CHANNELS = 3\ndef Generator_PM():\n    data = layers.Input(shape=[256,256,3])\n\n    # bs = batch size\n    down_sample = [\n        downsample(64, 4, apply_instancenorm=False), # (bs, 128, 128, 64)\n        downsample(128, 4), # (bs, 64, 64, 128)\n        downsample(256, 4), # (bs, 32, 32, 256)\n        downsample(512, 4), # (bs, 16, 16, 512)\n        downsample(512, 4), # (bs, 8, 8, 512)\n        downsample(512, 4), # (bs, 4, 4, 512)\n        downsample(512, 4), # (bs, 2, 2, 512)\n        downsample(512, 4), # (bs, 1, 1, 512)\n    ]\n\n    up_sample = [\n        upsample(512, 4, apply_dropout=True), \n        upsample(512, 4, apply_dropout=True), # (bs, 4, 4, 1024)\n        upsample(512, 4, apply_dropout=True), # (bs, 8, 8, 1024)\n        upsample(512, 4), # (bs, 16, 16, 1024)\n        upsample(256, 4), # (bs, 32, 32, 512)\n        upsample(128, 4), # (bs, 64, 64, 256)\n        upsample(64, 4), # (bs, 128, 128, 128)\n    ]\n    \n    initialize = tf.random_normal_initializer(0., 0.02)\n    final = layers.Conv2DTranspose(3, 7,\n                                  strides=2,\n                                  padding='same',\n                                  kernel_initializer=initialize,\n                                  activation='tanh') # (bs, 256, 256, 3)\n\n    inputs = data\n\n    # Downsampling through the model\n    skip_connection = []\n    for down in down_sample:\n        inputs = down(inputs)\n        skip_connection.append(inputs)\n        \n    skip_connection = reversed(skip_connection[:-1])\n\n    # Upsampling and establishing the skip connections\n    for up, skip in zip(up_sample, skip_connection):\n        inputs = up(inputs)\n        inputs = layers.Concatenate()([inputs, skip])\n\n    inputs = final(inputs)\n\n    return keras.Model(inputs=data, outputs=inputs)    ","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:28.124454Z","iopub.execute_input":"2021-12-27T02:11:28.125011Z","iopub.status.idle":"2021-12-27T02:11:28.139970Z","shell.execute_reply.started":"2021-12-27T02:11:28.124974Z","shell.execute_reply":"2021-12-27T02:11:28.138687Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"generator = Generator_PM()\ntf.keras.utils.plot_model(generator, show_shapes=True, dpi=64)","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:28.141278Z","iopub.execute_input":"2021-12-27T02:11:28.141728Z","iopub.status.idle":"2021-12-27T02:11:31.208014Z","shell.execute_reply.started":"2021-12-27T02:11:28.141690Z","shell.execute_reply":"2021-12-27T02:11:31.206915Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def Discriminator_PM():\n    initialize = tf.random_normal_initializer(0., 0.02)\n    init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    data = layers.Input(shape=[256, 256, 3], name='input_image')\n\n    inputs = data\n\n    down1 = downsample(64, 4, False)(inputs) # (bs, 128, 128, 64)\n    down2 = downsample(128, 4)(down1) # (bs, 64, 64, 128)\n    down3 = downsample(256, 4)(down2) # (bs, 32, 32, 256)\n\n    zero_pad1 = layers.ZeroPadding2D()(down3) # (bs, 34, 34, 256)\n    conv = layers.Conv2D(512, 7, strides=2,\n                         kernel_initializer=initialize,\n                         use_bias=False)(zero_pad1) # (bs, 31, 31, 512)\n\n    norm1 = tfa.layers.InstanceNormalization(gamma_initializer=init)(conv)\n\n    leaky_relu = layers.LeakyReLU()(norm1)\n    \n    zero_pad2 = layers.ZeroPadding2D()(leaky_relu) # (bs, 33, 33, 512)\n\n    final = layers.Conv2D(1, 7, strides=2,\n                         kernel_initializer=initialize)(zero_pad2) # (bs, 30, 30, 1)\n\n    return tf.keras.Model(inputs=inputs, outputs=final)","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:31.210408Z","iopub.execute_input":"2021-12-27T02:11:31.210824Z","iopub.status.idle":"2021-12-27T02:11:31.225014Z","shell.execute_reply.started":"2021-12-27T02:11:31.210771Z","shell.execute_reply":"2021-12-27T02:11:31.223633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"discriminator_y = Discriminator_PM()\ntf.keras.utils.plot_model(discriminator_y, show_shapes=True, dpi=64)","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:31.226131Z","iopub.execute_input":"2021-12-27T02:11:31.226735Z","iopub.status.idle":"2021-12-27T02:11:31.675916Z","shell.execute_reply.started":"2021-12-27T02:11:31.226696Z","shell.execute_reply":"2021-12-27T02:11:31.674950Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with tpu_strategy.scope():\n    monet_generator = Generator_PM() # transforms photos to Monet-esque paintings\n    photo_generator = Generator_PM() # transforms Monet paintings to be more like photos\n\n    monet_discriminator = Discriminator_PM() # differentiates real Monet paintings and generated Monet paintings\n    photo_discriminator = Discriminator_PM() ","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:31.678144Z","iopub.execute_input":"2021-12-27T02:11:31.678605Z","iopub.status.idle":"2021-12-27T02:11:41.863320Z","shell.execute_reply.started":"2021-12-27T02:11:31.678563Z","shell.execute_reply":"2021-12-27T02:11:41.862131Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class CycleGan(keras.Model):\n    def __init__(\n        self,\n        monet_generator,\n        photo_generator,\n        monet_discriminator,\n        photo_discriminator,\n        lambda_cycle=20,\n    ):\n        super(CycleGan, self).__init__()\n        self.m_gen = monet_generator\n        self.p_gen = photo_generator\n        self.m_disc = monet_discriminator\n        self.p_disc = photo_discriminator\n        self.lambda_cycle = lambda_cycle\n        \n    def compile(\n        self,\n        m_gen_optimizer,\n        p_gen_optimizer,\n        m_disc_optimizer,\n        p_disc_optimizer,\n        gen_loss_fn,\n        disc_loss_fn,\n        cycle_loss_fn,\n        identity_loss_fn\n    ):\n        super(CycleGan, self).compile()\n        self.m_gen_optimizer = m_gen_optimizer\n        self.p_gen_optimizer = p_gen_optimizer\n        self.m_disc_optimizer = m_disc_optimizer\n        self.p_disc_optimizer = p_disc_optimizer\n        self.gen_loss_fn = gen_loss_fn\n        self.disc_loss_fn = disc_loss_fn\n        self.cycle_loss_fn = cycle_loss_fn\n        self.identity_loss_fn = identity_loss_fn\n        \n    def train_step(self, batch_data):\n        real_monet, real_photo = batch_data\n        \n        with tf.GradientTape(persistent=True) as tape:\n            # photo to monet back to photo\n            fake_monet = self.m_gen(real_photo, training=True)\n            cycled_photo = self.p_gen(fake_monet, training=True)\n\n            # monet to photo back to monet\n            fake_photo = self.p_gen(real_monet, training=True)\n            cycled_monet = self.m_gen(fake_photo, training=True)\n\n            # generating itself\n            same_monet = self.m_gen(real_monet, training=True)\n            same_photo = self.p_gen(real_photo, training=True)\n\n            # discriminator used to check, inputing real images\n            disc_real_monet = self.m_disc(real_monet, training=True)\n            disc_real_photo = self.p_disc(real_photo, training=True)\n\n            # discriminator used to check, inputing fake images\n            disc_fake_monet = self.m_disc(fake_monet, training=True)\n            disc_fake_photo = self.p_disc(fake_photo, training=True)\n            \n             # evaluates generator loss\n            monet_gen_loss = self.gen_loss_fn(disc_fake_monet)\n            photo_gen_loss = self.gen_loss_fn(disc_fake_photo)\n\n            # evaluates total cycle consistency loss\n            total_cycle_loss = self.cycle_loss_fn(real_monet, cycled_monet, self.lambda_cycle) + self.cycle_loss_fn(real_photo, cycled_photo, self.lambda_cycle)\n\n            # evaluates total generator loss\n            total_monet_gen_loss = monet_gen_loss + total_cycle_loss + self.identity_loss_fn(real_monet, same_monet, self.lambda_cycle)\n            total_photo_gen_loss = photo_gen_loss + total_cycle_loss + self.identity_loss_fn(real_photo, same_photo, self.lambda_cycle)\n\n            # evaluates discriminator loss\n            monet_disc_loss = self.disc_loss_fn(disc_real_monet, disc_fake_monet)\n            photo_disc_loss = self.disc_loss_fn(disc_real_photo, disc_fake_photo)\n\n        # Calculate the gradients for generator and discriminator\n        monet_generator_gradients = tape.gradient(total_monet_gen_loss,\n                                                  self.m_gen.trainable_variables)\n        photo_generator_gradients = tape.gradient(total_photo_gen_loss,\n                                                  self.p_gen.trainable_variables)\n        \n        monet_discriminator_gradients = tape.gradient(monet_disc_loss,\n                                                      self.m_disc.trainable_variables)\n        photo_discriminator_gradients = tape.gradient(photo_disc_loss,\n                                                      self.p_disc.trainable_variables)\n\n        # Apply the gradients to the optimizer\n        self.m_gen_optimizer.apply_gradients(zip(monet_generator_gradients,\n                                                 self.m_gen.trainable_variables))\n\n        self.p_gen_optimizer.apply_gradients(zip(photo_generator_gradients,\n                                                 self.p_gen.trainable_variables))\n\n        self.m_disc_optimizer.apply_gradients(zip(monet_discriminator_gradients,\n                                                  self.m_disc.trainable_variables))\n\n        self.p_disc_optimizer.apply_gradients(zip(photo_discriminator_gradients,\n                                                  self.p_disc.trainable_variables))\n        \n        return {\n            \"monet_gen_loss\": total_monet_gen_loss,\n            \"photo_gen_loss\": total_photo_gen_loss,\n            \"monet_disc_loss\": monet_disc_loss,\n            \"photo_disc_loss\": photo_disc_loss\n        }","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:41.865128Z","iopub.execute_input":"2021-12-27T02:11:41.865454Z","iopub.status.idle":"2021-12-27T02:11:41.892214Z","shell.execute_reply.started":"2021-12-27T02:11:41.865417Z","shell.execute_reply":"2021-12-27T02:11:41.891123Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with tpu_strategy.scope():\n    # Discriminator loss {0: fake, 1: real} (The discriminator loss outputs the average of the real and generated loss)\n    def discriminator_loss(real, generated):\n        real_loss = losses.BinaryCrossentropy(from_logits=True, reduction=losses.Reduction.NONE)(tf.ones_like(real), real)\n\n        generated_loss = losses.BinaryCrossentropy(from_logits=True, reduction=losses.Reduction.NONE)(tf.zeros_like(generated), generated)\n\n        total_disc_loss = real_loss + generated_loss\n\n        return total_disc_loss * 0.5\n    # Generator loss\n    def generator_loss(generated):\n        return losses.BinaryCrossentropy(from_logits=True, reduction=losses.Reduction.NONE)(tf.ones_like(generated), generated)\n    \n    \n    # Cycle consistency loss (measures if original photo and the twice transformed photo to be similar to one another)\n    with tpu_strategy.scope():\n        def calc_cycle_loss(real_image, cycled_image, LAMBDA):\n            loss1 = tf.reduce_mean(tf.abs(real_image - cycled_image))\n\n            return LAMBDA * loss1\n\n    # Identity loss (compares the image with its generator (i.e. photo with photo generator))\n    with tpu_strategy.scope():\n        def identity_loss(real_image, same_image, LAMBDA):\n            loss = tf.reduce_mean(tf.abs(real_image - same_image))\n            return LAMBDA * 0.5 * loss","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:41.893742Z","iopub.execute_input":"2021-12-27T02:11:41.894047Z","iopub.status.idle":"2021-12-27T02:11:41.912721Z","shell.execute_reply.started":"2021-12-27T02:11:41.894001Z","shell.execute_reply":"2021-12-27T02:11:41.911207Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with tpu_strategy.scope():\n    monet_generator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n    photo_generator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n\n    monet_discriminator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n    photo_discriminator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:41.914284Z","iopub.execute_input":"2021-12-27T02:11:41.914628Z","iopub.status.idle":"2021-12-27T02:11:41.927444Z","shell.execute_reply.started":"2021-12-27T02:11:41.914593Z","shell.execute_reply":"2021-12-27T02:11:41.926132Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with tpu_strategy.scope():\n    cycle_gan_model = CycleGan(\n        monet_generator, photo_generator, \n        monet_discriminator, photo_discriminator\n    )\n\n    cycle_gan_model.compile(\n        m_gen_optimizer = monet_generator_optimizer,\n        p_gen_optimizer = photo_generator_optimizer,\n        m_disc_optimizer = monet_discriminator_optimizer,\n        p_disc_optimizer = photo_discriminator_optimizer,\n        gen_loss_fn = generator_loss,\n        disc_loss_fn = discriminator_loss,\n        cycle_loss_fn = calc_cycle_loss,\n        identity_loss_fn = identity_loss\n    )","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:41.928988Z","iopub.execute_input":"2021-12-27T02:11:41.929281Z","iopub.status.idle":"2021-12-27T02:11:42.001981Z","shell.execute_reply.started":"2021-12-27T02:11:41.929247Z","shell.execute_reply":"2021-12-27T02:11:42.001015Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cycle_gan_model.fit(\n    data,\n    epochs=30,\n    steps_per_epoch=(max(monet_jpg, photo_jpg)//BATCH_SIZE),\n)","metadata":{"execution":{"iopub.status.busy":"2021-12-27T02:11:42.003789Z","iopub.execute_input":"2021-12-27T02:11:42.004211Z","iopub.status.idle":"2021-12-27T04:18:46.408244Z","shell.execute_reply.started":"2021-12-27T02:11:42.004162Z","shell.execute_reply":"2021-12-27T04:18:46.407175Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import PIL\ndef predict_and_save(input_ds, generator_model, output_path):\n    i = 1\n    for img in input_ds:\n        prediction = generator_model(img, training=False)[0].numpy() # make predition\n        prediction = (prediction * 127.5 + 127.5).astype(np.uint8)   # re-scale\n        im = PIL.Image.fromarray(prediction)\n        im.save(f'{output_path}{str(i)}.jpg')\n        i += 1","metadata":{"execution":{"iopub.status.busy":"2021-12-27T04:18:46.409734Z","iopub.execute_input":"2021-12-27T04:18:46.410087Z","iopub.status.idle":"2021-12-27T04:18:46.419014Z","shell.execute_reply.started":"2021-12-27T04:18:46.410048Z","shell.execute_reply":"2021-12-27T04:18:46.417997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nos.makedirs('../images/') # Create folder to save generated images\n\npredict_and_save(load_dataset(photos_tfr).batch(1), monet_generator, '../images/')","metadata":{"execution":{"iopub.status.busy":"2021-12-27T04:18:46.420803Z","iopub.execute_input":"2021-12-27T04:18:46.421337Z","iopub.status.idle":"2021-12-27T04:55:08.595321Z","shell.execute_reply.started":"2021-12-27T04:18:46.421299Z","shell.execute_reply":"2021-12-27T04:55:08.594026Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import shutil\nshutil.make_archive(\"/kaggle/working/images\", 'zip', \"/kaggle/images\")","metadata":{"execution":{"iopub.status.busy":"2021-12-27T04:55:08.597380Z","iopub.execute_input":"2021-12-27T04:55:08.597664Z","iopub.status.idle":"2021-12-27T04:55:12.681043Z","shell.execute_reply.started":"2021-12-27T04:55:08.597632Z","shell.execute_reply":"2021-12-27T04:55:12.680134Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f\"Number of generated samples: {len([name for name in os.listdir('../images/') if os.path.isfile(os.path.join('../images/', name))])}\")","metadata":{"execution":{"iopub.status.busy":"2021-12-27T04:55:12.682660Z","iopub.execute_input":"2021-12-27T04:55:12.683490Z","iopub.status.idle":"2021-12-27T04:55:12.747504Z","shell.execute_reply.started":"2021-12-27T04:55:12.683451Z","shell.execute_reply":"2021-12-27T04:55:12.746501Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Did it work?\nThe dataset contains four directories: monet_tfrec, photo_tfrec, monet_jpg, and photo_jpg. The monet_tfrec and monet_jpg directories contain the same painting images, and the photo_tfrec and photo_jpg directories contain the same photos. The monet directories contain Monet paintings. Use these images to train your model.\n\nThe photo directories contain photos. Add Monet-style to these images and submit your generated jpeg images as a zip file. Other photos outside of this dataset can be transformed but keep your submission file limited to 10,000 images.\n\n#### What did you not understand about this process?\nWell, everything provides in the competition data page. I've no problem while working on it. If you guys don't understand the thing that I'll do in this notebook then please comment on this notebook.\n\n#### What else do you think you can try as part of this approach?\nMonet-style art can be created from scratch using other GAN architectures like DCGAN. Become more familiar with these concepts:\n\n- Computer vision\n- Generative models\n- Tensor Processing Units (TPUs)\n- TFRecords format for TensorFlow\n","metadata":{}}]}