{"cells":[{"metadata":{},"cell_type":"markdown","source":"# [](http://)CycleGAN with pretraining of generators and discriminators"},{"metadata":{},"cell_type":"markdown","source":"\n## Contribution:\nI suggest that discriminators and  generators for unpaired image-to-image translation should be pre-trained separately on the problems of classification (for the discriminator) and image scaling (for the generator). It will be advantageous to eliminate such hyperparameter as choosing an initial random initialization.\n\n\n## Identifying problems\n\n* Cycle consistency loss is the most important problem of CycleGAN.\n* Small domain dataset. If one of the sets of images is relatively small (only 300 Monet paintings), then Monet discriminator overfits. The discriminator remembers all 300 of them and accurately determines them. In the case of testing with other real Monet paintings, the discriminator does not recognize them and confidently considers them to be generated fakes.\n* Too many hyperparameters. Even initial initialization affects the learning outcome.\n\n\n## 1. Cycle consistency loss\n\nWhy are we using Cycle consistency loss? We assume that translated image can be reconstructed. But this is not the case. Some data has been lost. If we convert zebras to horses, we lose stripes and the models cannot restore stripes in the correct places. If the generator turns the horses back into zebras, it could place stripes in completely wrong places. What does the cycle consistency loss say? A very bad zebra! What will the model do next time? It will encode stripes on the \"horses\" image to reconstruct later!\n\nTo demonstrate the possibilities of encoding, I have inverted the Monet generator's adversarial loss asking \"make the least real Monet!\". The result is gray squares. But from these gray squares the photo generator reconstructed the original photos!\n\nBut this is only one part of the Cycle consistency loss problem. As for image-to-image translation, we want the model to be accurate. A tree must be translated as a tree. All semantics must be preserved. But there is no motivation for this in Cycle consistency loss. The only motivation is to encode the original image into a translated image.\n\nSo Cycle consistency loss is useless. Then duplicating generators and discriminators is also useless. We should start with a regular GAN looking for good objective function."},{"metadata":{},"cell_type":"markdown","source":"## 2. Random initialization problem\n\nThe learning outcome depends on the type of initialization significantly. This is an essential hyperparameter, can we avoid brute-force? My hypothesis: if the generator and the discriminator are pre-trained separately from each other on supervised learning problems, such as classification? Let me give you an example from human activity. The competitive mode significantly improves the development of a person's skills. Human play is an element of learning. But do they start playing without any training? First, children learn the basic skills of walking, running, hitting the ball - and only then they compete in the game of football. Children under one year old do not play football! Then why is the neural network trained exclusively in competition? Maybe, first, the deep layers of the neural network should become suitable for use in similar tasks, in which it is possible to achieve good results, and then the competitive mode will perfect the skills of generating / recognizing fakes, surpassing humans?"},{"metadata":{},"cell_type":"markdown","source":"## Related works\n\nUnpaired image-to-image translation frameworks have been proposed in 2017 [1, 2, 3]. I found two different approaches: \n\n* UNIT [2] (Unsupervised Image-to-Image Translation Networks) combines variational autoencoders (VAEs) with Coupled Generative Adversarial Networks (CoGAN) [4], a GAN framework where two generators share weights to learn the joint distribution of images in cross domains; \n* CycleGAN [1] and DiscoGAN [3] preserve key attributes between the input and the translated image by utilizing a cycle consistency loss.\n\nThere are many papers about Cycle consistency loss problem. They are usually of two types:\n* Improvements of CycleGAN:  [5] (StarGAN transfers images between multiple domains and uses Domain Classification Loss as part of full objective functions), [6] (adding noise for defence against a self-adversarial attack), [7] (training stability using Wasserstein loss), [8] (proposing multi-cycle translation with mutual information constraints - MCMI), [9] (HarmonicGAN adds additional smoothing to CycleGAN), [10] (Lipschitz Regularized CycleGAN for Improving Semantic Robustness in Unpaired Image-to-image Translation). \n* Without cycle consistency loss: DistanceGAN [11], geometry-consistent generative adversarial network GcGAN [12] and  Contrastive learning for unpaired image-to-image translation CUT [13] have been proposed.  [14] propose to learn common latent space with siamese network additional to adversarial network without cycle consistency loss. \n\nCerfGAN [15] only needs two networks (decoder and multi-class discriminator that works also as encoder) to solve image-to-image translation problems, but it uses reconstruction loss just as CycleGAN does. \n\nRandom initialization problem was noted in [16]: “Training Stability: Domain adaptation approaches that rely on some form of adversarial training are sensitive to random initialization. To address this, we incorporate a task–specific loss trained on both source and generated images and a pixel similarity regularization that allows us to avoid mode collapse and stabilize training. By using these tools, we are able to reduce variance of performance for the same hyperparameters across different random initializations of our model “\n\nTransfer learning for image-to-image translation was noted only in [17]. They introduce DeepI2I model, but it obtains inferior results on limited datasets. To mitigate this problem they propose to initialize deepI2I from a pre-trained GAN. So they are the first to study transfer learning for I2I networks. Another important deepI2I feature is that it doesn't use pixelwise cycle consistency loss. Instead it compares outputs of hidden layers of discriminator. \n"},{"metadata":{},"cell_type":"markdown","source":"## Notebook\n\n\nIn this notebook, I did my best to make CycleGAN work. One of the most important things is LSE loss instead of BCE loss. \nI print the FID metric after every epoch. And I also implemented  generators/discriminators pretraining."},{"metadata":{},"cell_type":"markdown","source":"References\n\n[1] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017. \n\n[2] M.-Y. Liu, T. Breuel, and J. Kautz. Unsupervised image-to-image translation networks. arXiv preprint arXiv:1703.00848, 2017. \n\n[3] T. Kim, M. Cha, H. Kim, J. K. Lee, and J. Kim. Learning to discover cross-domain relations with generative adversarial networks. In Proceedings of the 34th International Conference on Machine Learning (ICML), pages 1857–1865, 2017\n\n[4] M.-Y. Liu and O. Tuzel. Coupled generative adversarial networks. In Advances in Neural Information Processing Systems (NIPS), pages 469–477, 2016.\n\n[5] Choi, Y., Choi, M., Kim, M., Ha, J.W., Kim, S., Choo, J.: Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In: Conference on Computer Vision and Pattern Recognition. pp. 8789–8797 (2018)\n\n[[6] D. Bashkirova, B. Usman, and K. Saenko. Adversarial self-defense for cycle-consistent gans. In Advances in Neural Information Processing Systems, pages 635–645, 2019](https://arxiv.org/abs/1908.01517)\n\n[7] Arjovsky, Martín et al. “Wasserstein GAN.” ArXiv abs/1701.07875 (2017)\n\n[[8] Xu, X. et al. “MCMI: Multi-Cycle Image Translation with Mutual Information Constraints.” ArXiv abs/2007.02919 (2020)](https://arxiv.org/pdf/2007.02919.pdf)\n\n[[9] Rui Zhang, Tomas Pfister, and Jia Li. Harmonic unpaired image-to-image translation. arXiv preprint arXiv:1902.09727, 2019](https://arxiv.org/abs/1902.09727)\n\n[[10] Jia, Z., Yuan, B., Wang, K., Wu, H., Clifford, D., Yuan, Z., & Su, H. (2020). Lipschitz Regularized CycleGAN for Improving Semantic Robustness in Unpaired Image-to-image Translation. ArXiv, abs/2012.04932](https://arxiv.org/abs/2012.04932)\n\n[[11] S. Benaim and L. Wolf. One-sided unsupervised domain mapping. In NIPS, 2017 ](https://arxiv.org/abs/1706.00826)\n\n[[12] Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Batmanghelich, Kun Zhang, and Dacheng Tao. Geometry-consistent generative adversarial networks for one-sided unsupervised domain mapping. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2427–2436, 2019](https://arxiv.org/abs/1809.05852)\n\n[[13] Taesung Park, Alexei A Efros, Richard Zhang, and JunYan Zhu. Contrastive learning for unpaired image-to-image translation. In European Conference on Computer Vision, pages 319–345. Springer, 2020](https://arxiv.org/abs/2007.15651)\n\n[[14] Amodio, M., Krishnaswamy, S.: Travelgan: Image-to-image translation by transformation vector learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 8983–8992 (2019)](https://arxiv.org/abs/1902.09631)\n\n[[15] X. Liu, S. Zhang, H. Liu, X. Liu, and R. Ji. Cerfgan: A compact, effective, robust, and fast model for unsupervised multi-domain image-to-image translation. arXiv preprint arXiv:1805.10871v2, 2018](https://arxiv.org/pdf/1805.10871.pdf)\n\n[[16] Konstantinos Bousmalis, Nathan Silberman, David Dohan, Dumitru Erhan, and Dilip Krishnan. Unsupervised pixel-level domain adaptation with generative adversarial networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017b.](https://arxiv.org/pdf/1612.05424)\n\n[[17] Wang, Y., Yu, L., & Weijer, J.V. (2020). DeepI2I: Enabling Deep Hierarchical Image-to-Image Translation by Transferring from GANs. ArXiv, abs/2011.05867.](https://arxiv.org/pdf/2011.05867.pdf)\n\n"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import tensorflow as tf\nimport tensorflow_hub as hub\nimport tensorflow_probability as tfp\nimport os, random, json, PIL, shutil, re, gc\nfrom tensorflow import keras\nfrom tensorflow.keras import layers\nimport tensorflow_addons as tfa\nfrom scipy import linalg\nfrom kaggle_datasets import KaggleDatasets\nimport matplotlib.pyplot as plt\nimport numpy as np\ndef seed_everything(seed=0):\n    random.seed(seed)\n    np.random.seed(seed)\n    tf.random.set_seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    os.environ['TF_DETERMINISTIC_OPS'] = '1'\nSEED = 0\nseed_everything(SEED)\n\n\n%matplotlib inline\ntry:\n    tpu = tf.distribute.cluster_resolver.TPUClusterResolver()\n    print('Device:', tpu.master())\n    tf.config.experimental_connect_to_cluster(tpu)\n    tf.tpu.experimental.initialize_tpu_system(tpu)\n    strategy = tf.distribute.experimental.TPUStrategy(tpu)\nexcept:\n    strategy = tf.distribute.get_strategy()\nprint('Number of replicas:', strategy.num_replicas_in_sync)\n\nAUTOTUNE = tf.data.experimental.AUTOTUNE\ntf.__version__\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"GCS_PATH = KaggleDatasets().get_gcs_path(\"gan-getting-started\")\nGCS_PATH_1367 = KaggleDatasets().get_gcs_path(\"monet-tfrecords-256x256\")\n\nMONET_FILENAMES = tf.io.gfile.glob(str(GCS_PATH + '/monet_tfrec/*.tfrec'))\nPHOTO_FILENAMES = tf.io.gfile.glob(str(GCS_PATH + '/photo_tfrec/*.tfrec'))\n\nMONET_FILENAMES_1367 = tf.io.gfile.glob(str(GCS_PATH_1367 + '/mon*.tfrec'))\n\nm=MONET_FILENAMES_1367\np=PHOTO_FILENAMES\nPRETRAIN_FILENAMES=[m[0],p[0],m[1],p[1],m[2],p[2],m[3],p[3],m[0],p[4],m[1],p[5],m[2],p[6],m[3],p[7],m[0],p[8],m[1],p[9],m[2],p[10],m[3],p[11],m[0],p[12],m[1],p[13],m[2],p[14],m[3],p[15],m[0],p[16],m[1],p[17],m[2],p[18],m[4],p[19]]\nPRETRAIN_LABELS=[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*274+[0]*352+[1]*271+[0]*350\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"IMAGE_SIZE = [256, 256]\n\ndef decode_image(image):\n    image = tf.image.decode_jpeg(image, channels=3)\n    image = (tf.cast(image, tf.float32) / 127.5) - 1\n    image = tf.reshape(image, [*IMAGE_SIZE, 3])\n    return image\n\ndef read_tfrecord(example):\n    tfrecord_format = {\n        \"image\": tf.io.FixedLenFeature([], tf.string)\n    }\n    example = tf.io.parse_single_example(example, tfrecord_format)\n    image = decode_image(example['image'])\n    return image","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def data_augment_color(image):\n    image = tf.image.random_flip_left_right(image)\n    image = (image + 1) / 2\n    image = tf.image.random_saturation(image, 0.7, 1.2)\n    image = tf.clip_by_value(image, 0, 1) \n    image = (image - 0.5) * 2    \n    return image\n\n\ndef pretrain_data_augment(image):\n\n        image = tf.image.resize(image, [286, 286])\n        image = tf.image.random_crop(image, size=[256, 256, 3])\n\n   \n        return image\n\ndef to3030(label):\n    return np.ones((30,30,1),dtype=\"float32\")*label","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"###### from pats notebook https://www.kaggle.com/swepat/cyclegan-to-generate-monet-style-images   #############\ndef data_augment(image):\n    p_rotate = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    p_spatial = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    p_crop = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    \n    # Apply jitter\n    if p_crop > .5:\n        image = tf.image.resize(image, [286, 286])\n        image = tf.image.random_crop(image, size=[256, 256, 3])\n        if p_crop > .9:\n            image = tf.image.resize(image, [300, 300])\n            image = tf.image.random_crop(image, size=[256, 256, 3])\n    \n    # Random rotation\n    if p_rotate > .9:\n        image = tf.image.rot90(image, k=3) # rotate 270º\n    elif p_rotate > .7:\n        image = tf.image.rot90(image, k=2) # rotate 180º\n    elif p_rotate > .5:\n        image = tf.image.rot90(image, k=1) # rotate 90º\n    \n    # Random mirroring\n    if p_spatial > .6:\n        image = tf.image.random_flip_left_right(image)\n        image = tf.image.random_flip_up_down(image)\n        if p_spatial > .9:\n            image = tf.image.transpose(image)\n    \n    return image","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BATCH_SIZE = 1\n# EPOCHS = 5\n\ndef load_dataset(filenames):\n    dataset = tf.data.TFRecordDataset(filenames)\n    dataset = dataset.map(read_tfrecord, num_parallel_calls=AUTOTUNE)\n    return dataset\n\nphoto_ds = load_dataset(PHOTO_FILENAMES)\nmonet_ds = load_dataset(MONET_FILENAMES)\n\n# photo_ds_val=photo_ds.skip(6038)\n# monet_ds_val = load_dataset(MONET_FILENAMES_1367)\n# val_ds=tf.data.Dataset.zip((monet_ds_val, photo_ds_val)).batch(64)\n\n# photo_ds=photo_ds.take(6038)\n\nmonet_ds = monet_ds.repeat()\nphoto_ds = photo_ds.repeat()\n\n# monet_ds = monet_ds.map(data_augment, num_parallel_calls=AUTOTUNE)\n# photo_ds = photo_ds.map(data_augment, num_parallel_calls=AUTOTUNE)\n\n# photo_ds=photo_ds.batch(BATCH_SIZE)\n# monet_ds = monet_ds.batch(BATCH_SIZE)\n  \ngan_ds = tf.data.Dataset.zip((monet_ds, photo_ds)).batch(BATCH_SIZE).prefetch(AUTOTUNE)\n\nfast_photo_ds = load_dataset(PHOTO_FILENAMES).batch(32*strategy.num_replicas_in_sync).prefetch(32)\n\nmonet_ds_fid = load_dataset(MONET_FILENAMES).batch(32*strategy.num_replicas_in_sync).prefetch(32)\n\n\npretrain_ds = load_dataset(PRETRAIN_FILENAMES)\n\npretrain_labels=tf.data.Dataset.from_tensor_slices(np.array(PRETRAIN_LABELS,dtype=\"float32\"))\npretrain_labels=pretrain_labels.map(to3030,num_parallel_calls=AUTOTUNE)\npretrain_trans=pretrain_ds.map(pretrain_data_augment, num_parallel_calls=AUTOTUNE)\npretrain_ds=tf.data.Dataset.zip((pretrain_ds, pretrain_trans,pretrain_labels))\nval_pretrain_ds=pretrain_ds.skip(11894).batch(32, drop_remainder=True).prefetch(2)\npretrain_ds=pretrain_ds.take(11894).shuffle(400).batch(32, drop_remainder=True).prefetch(2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n#         inception_model = tf.keras.applications.InceptionV3(input_shape=(256,256,3),pooling=\"avg\",include_top=False)\n    inception_model = tf.keras.applications.InceptionV3(input_shape=(256,256,3),pooling=\"avg\",include_top=False)\n\n#     mix3  = inception_model.get_layer(\"mixed4\").output\n#     f0 = tf.keras.layers.GlobalMaxPooling2D()(mix3)\n\n#     inception_model = tf.keras.Model(inputs=inception_model.input, outputs=f0)\n    inception_model.trainable = False\n\n    \n    \ndef calculate_activation_statistics_mod(images,fid_model):\n        act=fid_model.predict(images)\n        mu = np.mean(act, axis=0)\n        sigma = np.cov(act, rowvar=False)\n        return mu, sigma\nmyFID_mu2, myFID_sigma2 = calculate_activation_statistics_mod(monet_ds_fid,inception_model)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(myFID_mu2.shape,myFID_sigma2.shape)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def calculate_frechet_distance(mu1,sigma1,mu2,sigma2):\n        fid_epsilon = 1e-14\n        mu1 = np.atleast_1d(mu1)\n        mu2 = np.atleast_1d(mu2)\n        sigma1 = np.atleast_2d(sigma1)\n        sigma2 = np.atleast_2d(sigma2)\n\n        assert mu1.shape == mu2.shape, 'Training and test mean vectors have different lengths'\n        assert sigma1.shape == sigma2.shape, 'Training and test covariances have different dimensions'\n\n        # product might be almost singular\n        covmean, _ = linalg.sqrtm(sigma1.dot(sigma2), disp=False)\n        if not np.isfinite(covmean).all():\n            msg = f'fid calculation produces singular product; adding {fid_epsilon} to diagonal of cov estimates'\n            warnings.warn(msg)\n            offset = np.eye(sigma1.shape[0]) * fid_epsilon\n            covmean = linalg.sqrtm((sigma1 + offset).dot(sigma2 + offset))\n            \n        # numerical error might give slight imaginary component\n        if np.iscomplexobj(covmean):\n            if not np.allclose(np.diagonal(covmean).imag, 0, atol=1e-3):\n                m = np.max(np.abs(covmean.imag))\n                raise ValueError(f'Imaginary component {m}')\n            covmean = covmean.real\n        tr_covmean = np.trace(covmean)\n        return (mu1 - mu2).dot(mu1 - mu2) + np.trace(sigma1) + np.trace(sigma2) - 2 * tr_covmean\n\n\n    \ndef FID(images,gen_model,inception_model=inception_model,myFID_mu2=myFID_mu2, myFID_sigma2=myFID_sigma2):\n            with strategy.scope():\n                inp = layers.Input(shape=[256, 256, 3], name='input_image')\n                x  = gen_model(inp)\n                x=inception_model(x)\n                fid_model = tf.keras.Model(inputs=inp, outputs=x)\n                \n            mu1, sigma1 = calculate_activation_statistics_mod(images,fid_model)\n\n            fid_value = calculate_frechet_distance(mu1, sigma1,myFID_mu2, myFID_sigma2)\n\n            return fid_value","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\nOUTPUT_CHANNELS = 3\n\ndef downsample(filters, size, apply_instancenorm=True):\n    initializer = tf.random_normal_initializer(0., 0.02)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    result = keras.Sequential()\n    result.add(layers.Conv2D(filters, size, strides=2, padding='same',\n                             kernel_initializer=initializer, use_bias=False))\n\n    if apply_instancenorm:\n        result.add(tfa.layers.InstanceNormalization(gamma_initializer=gamma_init))\n\n    result.add(layers.LeakyReLU())\n\n    return result","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def upsample(filters, size, apply_dropout=False):\n    initializer = tf.random_normal_initializer(0., 0.02)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    result = keras.Sequential()\n    result.add(layers.Conv2DTranspose(filters, size, strides=2,\n                                      padding='same',\n                                      kernel_initializer=initializer,\n                                      use_bias=False))\n\n    result.add(tfa.layers.InstanceNormalization(gamma_initializer=gamma_init))\n\n    if apply_dropout:\n        result.add(layers.Dropout(0.5))\n\n    result.add(layers.ReLU())\n\n    return result","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def Generator():\n    inputs = layers.Input(shape=[256,256,3])\n\n    # bs = batch size\n    down_stack = [\n        downsample(64, 4, apply_instancenorm=False), # (bs, 128, 128, 64)\n        downsample(128, 4), # (bs, 64, 64, 128)\n        downsample(256, 4), # (bs, 32, 32, 256)\n        downsample(512, 4), # (bs, 16, 16, 512)\n        downsample(512, 4), # (bs, 8, 8, 512)\n        downsample(512, 4), # (bs, 4, 4, 512)\n        downsample(512, 4), # (bs, 2, 2, 512)\n        downsample(512, 4), # (bs, 1, 1, 512)\n    ]\n\n    up_stack = [\n        upsample(512, 4, apply_dropout=True), # (bs, 2, 2, 1024)\n        upsample(512, 4, apply_dropout=True), # (bs, 4, 4, 1024)\n        upsample(512, 4, apply_dropout=True), # (bs, 8, 8, 1024)\n        upsample(512, 4), # (bs, 16, 16, 1024)\n        upsample(256, 4), # (bs, 32, 32, 512)\n        upsample(128, 4), # (bs, 64, 64, 256)\n        upsample(64, 4), # (bs, 128, 128, 128)\n    ]\n\n    initializer = tf.random_normal_initializer(0., 0.02)\n    last = layers.Conv2DTranspose(OUTPUT_CHANNELS, 4,\n                                  strides=2,\n                                  padding='same',\n                                  kernel_initializer=initializer,\n                                  activation='tanh') # (bs, 256, 256, 3)\n\n    x = inputs\n\n    # Downsampling through the model\n    skips = []\n    for down in down_stack:\n        x = down(x)\n        skips.append(x)\n\n    skips = reversed(skips[:-1])\n\n    # Upsampling and establishing the skip connections\n    for up, skip in zip(up_stack, skips):\n        x = up(x)\n        x = layers.Concatenate()([x, skip])\n\n    x = last(x)\n\n    return keras.Model(inputs=inputs, outputs=x)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def Discriminator():\n    initializer = tf.random_normal_initializer(0., 0.02)\n    gamma_init = keras.initializers.RandomNormal(mean=0.0, stddev=0.02)\n\n    inp = layers.Input(shape=[256, 256, 3], name='input_image')\n\n    x = inp\n\n    down1 = downsample(64, 4, False)(x) # (bs, 128, 128, 64)\n    down2 = downsample(128, 4)(down1) # (bs, 64, 64, 128)\n    down3 = downsample(256, 4)(down2) # (bs, 32, 32, 256)\n\n    zero_pad1 = layers.ZeroPadding2D()(down3) # (bs, 34, 34, 256)\n    conv = layers.Conv2D(512, 4, strides=1,\n                         kernel_initializer=initializer,\n                         use_bias=False)(zero_pad1) # (bs, 31, 31, 512)\n\n    norm1 = tfa.layers.InstanceNormalization(gamma_initializer=gamma_init)(conv)\n\n    leaky_relu = layers.LeakyReLU()(norm1)\n\n    zero_pad2 = layers.ZeroPadding2D()(leaky_relu) # (bs, 33, 33, 512)\n\n    last = layers.Conv2D(1, 4, strides=1,\n                         kernel_initializer=initializer)(zero_pad2) # (bs, 30, 30, 1)\n    \n    \n\n    return tf.keras.Model(inputs=inp, outputs=last)\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    monet_generator = Generator() # transforms photos to Monet-esque paintings\n    photo_generator = Generator() # transforms Monet paintings to be more like photos\n\n    monet_discriminator = Discriminator() # differentiates real Monet paintings and generated Monet paintings\n    photo_discriminator = Discriminator() # differentiates real photos and generated photos","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class CycleGan(keras.Model):\n    def __init__(\n        self,\n        monet_generator,\n        photo_generator,\n        monet_discriminator,\n        photo_discriminator,\n        monet_expert,\n        lambda_cycle=10,\n        lambda_id=3,\n#         lambda_GP=10,        \n    ):\n        super(CycleGan, self).__init__()\n        self.m_gen = monet_generator\n        self.p_gen = photo_generator\n        self.m_disc = monet_discriminator\n        self.p_disc = photo_discriminator\n        self.monet_expert = monet_expert\n        self.lambda_cycle = lambda_cycle\n        self.lambda_id = lambda_id\n#         self.lambda_GP = lambda_GP\n\n\n        \n    def compile(\n        self,\n        m_gen_optimizer,\n        p_gen_optimizer,\n        m_disc_optimizer,\n        p_disc_optimizer,\n        gen_loss_fn,\n        disc_loss_fn,\n        cycle_loss_fn,\n        identity_loss_fn,\n        expert_loss_fn,\n        \n    ):\n        super(CycleGan, self).compile()\n        self.m_gen_optimizer = m_gen_optimizer\n        self.p_gen_optimizer = p_gen_optimizer\n        self.m_disc_optimizer = m_disc_optimizer\n        self.p_disc_optimizer = p_disc_optimizer\n        self.gen_loss_fn = gen_loss_fn\n        self.disc_loss_fn = disc_loss_fn\n        self.cycle_loss_fn = cycle_loss_fn\n        self.identity_loss_fn = identity_loss_fn\n        self.expert_loss_fn = expert_loss\n        \n    \n    def train_step(self, batch_data):\n        real_monet, real_photo = batch_data\n      \n        with tf.GradientTape(persistent=True) as tape:\n            # photo to monet \n            \n            fake_monet = self.m_gen(real_photo, training=True)\n            fake_photo = self.p_gen(real_monet, training=True)\n            \n            same_monet = self.m_gen(real_monet, training=True)\n            same_photo = self.p_gen(real_photo, training=True)\n\n            # discriminator used to check, inputing real images\n            disc_real_monet = self.m_disc(real_monet, training=True)\n            disc_real_photo = self.p_disc(real_photo, training=True)\n            # discriminator used to check, inputing fake images\n            disc_fake_monet = self.m_disc(fake_monet, training=True)\n            disc_fake_photo = self.p_disc(fake_photo, training=True)\n            \n            # evaluates discriminator loss\n            monet_disc_loss = self.disc_loss_fn(disc_real_monet, disc_fake_monet)\n            photo_disc_loss = self.disc_loss_fn(disc_real_photo, disc_fake_photo)\n\n            monet_expert_loss=self.expert_loss_fn(self.monet_expert(fake_monet, training=False),tf.ones_like(fake_monet))\n            photo_expert_loss=self.expert_loss_fn(self.monet_expert(fake_photo, training=False),tf.zeros_like(fake_photo))\n\n\n                # back to monet \n            cycled_monet = self.m_gen(fake_photo, training=True)\n            cycled_photo = self.p_gen(fake_monet, training=True)\n\n\n                # evaluates generator loss\n            monet_gen_loss = self.gen_loss_fn(disc_fake_monet)\n            photo_gen_loss = self.gen_loss_fn(disc_fake_photo)\n\n\n                # evaluates total cycle consistency loss\n            cycle_loss_mpm = self.cycle_loss_fn(real_monet, cycled_monet, self.lambda_cycle)\n            cycle_loss_pmp = self.cycle_loss_fn(real_photo, cycled_photo, self.lambda_cycle)\n            total_cycle_loss = cycle_loss_mpm + cycle_loss_pmp\n\n                # evaluates total generator loss\n            total_monet_gen_loss = monet_gen_loss + total_cycle_loss + self.identity_loss_fn(real_monet, same_monet, self.lambda_id) + monet_expert_loss\n            total_photo_gen_loss = photo_gen_loss + total_cycle_loss + self.identity_loss_fn(real_photo, same_photo, self.lambda_id) + photo_expert_loss\n#             total_monet_gen_loss*=tf.dtypes.cast(tf.random.uniform(shape=[1])[0]<0.50, tf.float32)             \n#             total_photo_gen_loss*=tf.dtypes.cast(tf.random.uniform(shape=[1])[0]<0.50, tf.float32)\n\n            \n        # Calculate the gradients for generator and discriminator\n        monet_discriminator_gradients = tape.gradient(monet_disc_loss,\n                                                      self.m_disc.trainable_variables)\n        photo_discriminator_gradients = tape.gradient(photo_disc_loss,\n                                                      self.p_disc.trainable_variables)\n        monet_generator_gradients = tape.gradient(total_monet_gen_loss,\n                                                      self.m_gen.trainable_variables)\n        photo_generator_gradients = tape.gradient(total_photo_gen_loss,\n                                                      self.p_gen.trainable_variables)\n\n        self.m_gen_optimizer.apply_gradients(zip(monet_generator_gradients,\n                                                     self.m_gen.trainable_variables))\n\n        self.p_gen_optimizer.apply_gradients(zip(photo_generator_gradients,\n                                                     self.p_gen.trainable_variables))\n\n\n        # Apply the gradients to the optimizer\n\n        self.m_disc_optimizer.apply_gradients(zip(monet_discriminator_gradients,\n                                                  self.m_disc.trainable_variables))\n\n        self.p_disc_optimizer.apply_gradients(zip(photo_discriminator_gradients,\n                                                  self.p_disc.trainable_variables))\n        \n        return {\n            \"monet_disc_loss\": monet_disc_loss,\n            \"photo_disc_loss\": photo_disc_loss,\n            \"disc_real_monet\": disc_real_monet,\n            \"disc_fake_monet\": disc_fake_monet,            \n            \"disc_real_photo\": disc_real_photo,            \n            \"disc_fake_photo\": disc_fake_photo,            \n            \"monet_expert_loss\": monet_expert_loss,            \n            \"photo_expert_loss\": photo_expert_loss,            \n\n        }\n    ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class Pretrain_CycleGan(keras.Model):\n    def __init__(\n        self,\n        monet_generator,\n        photo_generator,\n        monet_discriminator,\n        photo_discriminator,\n        monet_expert,\n        lambda_cycle=25,\n    ):\n        super(Pretrain_CycleGan, self).__init__()\n        self.m_gen = monet_generator\n        self.p_gen = photo_generator\n        self.m_disc = monet_discriminator\n        self.p_disc = photo_discriminator\n        self.m_expert = monet_expert\n        self.lambda_cycle = lambda_cycle\n        \n    def compile(\n        self,\n        m_gen_optimizer,\n        p_gen_optimizer,\n        m_disc_optimizer,\n        p_disc_optimizer,\n        m_expert_optimizer,\n        gen_loss_fn,\n        disc_loss_fn,\n        expert_loss_fn,\n    ):\n        super(Pretrain_CycleGan, self).compile()\n        self.m_gen_optimizer = m_gen_optimizer\n        self.p_gen_optimizer = p_gen_optimizer\n        self.m_disc_optimizer = m_disc_optimizer\n        self.p_disc_optimizer = p_disc_optimizer\n        self.m_expert_optimizer = m_expert_optimizer\n        self.expert_loss_fn = expert_loss\n        \n\n        self.gen_loss_fn = gen_loss_fn\n        self.disc_loss_fn = disc_loss_fn\n\n        \n    def train_step(self, batch_data):\n        real_image, transformed_image, label_is_monet = batch_data\n        \n        with tf.GradientTape(persistent=True) as tape:\n\n            gen_monet = self.m_gen(real_image, training=True)\n            gen_photo = self.p_gen(real_image, training=True)\n\n            disc_real_monet = self.m_disc(real_image, training=True)\n            disc_real_photo = self.p_disc(real_image, training=True)\n#             expert_monet = self.m_expert(real_image, training=True)\n#             expert_monet = tf.reduce_mean(self.m_expert(real_image, training=True),axis=[1,2,3])\n            monet_exp_loss=self.expert_loss_fn(self.m_expert(real_image, training=True),label_is_monet)\n         \n\n            # evaluates generator loss\n            monet_gen_loss = self.gen_loss_fn(gen_monet,transformed_image,1)\n            photo_gen_loss = self.gen_loss_fn(gen_photo,transformed_image,1)\n\n            # evaluates discriminator loss\n            monet_disc_loss = self.disc_loss_fn(disc_real_monet, tf.ones_like(disc_real_monet)*label_is_monet)\n            photo_disc_loss = self.disc_loss_fn(disc_real_photo, tf.ones_like(disc_real_photo)*(1-label_is_monet))\n#             monet_exp_loss = self.disc_loss_fn(expert_monet, tf.ones_like(expert_monet)*label_is_monet)\n\n\n        # Calculate the gradients for generator and discriminator\n        monet_generator_gradients = tape.gradient(monet_gen_loss,\n                                                  self.m_gen.trainable_variables)\n        photo_generator_gradients = tape.gradient(photo_gen_loss,\n                                                  self.p_gen.trainable_variables)\n\n        monet_discriminator_gradients = tape.gradient(monet_disc_loss,\n                                                      self.m_disc.trainable_variables)\n        photo_discriminator_gradients = tape.gradient(photo_disc_loss,\n                                                      self.p_disc.trainable_variables)\n\n        monet_expert_gradients = tape.gradient(monet_exp_loss,\n                                                      self.m_expert.trainable_variables)\n\n        # Apply the gradients to the optimizer\n        self.m_gen_optimizer.apply_gradients(zip(monet_generator_gradients,\n                                                 self.m_gen.trainable_variables))\n\n        self.p_gen_optimizer.apply_gradients(zip(photo_generator_gradients,\n                                                 self.p_gen.trainable_variables))\n\n        self.m_disc_optimizer.apply_gradients(zip(monet_discriminator_gradients,\n                                                  self.m_disc.trainable_variables))\n\n        self.p_disc_optimizer.apply_gradients(zip(photo_discriminator_gradients,\n                                                  self.p_disc.trainable_variables))\n        \n        self.m_expert_optimizer.apply_gradients(zip(monet_expert_gradients,\n                                                  self.m_expert.trainable_variables))\n\n        \n        return {\n                \"monet_gen_loss\": monet_gen_loss,\n                \"photo_gen_loss\": photo_gen_loss,\n                \"monet_disc_loss\": monet_disc_loss,\n                \"photo_disc_loss\": photo_disc_loss,\n                \"monet_exp_loss\" : monet_exp_loss\n        }\n\n\n    def test_step(self, batch_data):\n            real_image, transformed_image, label_is_monet = batch_data\n        \n            gen_monet = self.m_gen(real_image, training=True)\n            gen_photo = self.p_gen(real_image, training=True)\n\n            disc_real_monet = self.m_disc(real_image, training=True)\n            disc_real_photo = self.p_disc(real_image, training=True)\n#             expert_monet = self.m_expert(real_image, training=True)\n#             disc_real_monet = tf.reduce_mean(self.m_disc(real_image, training=True),axis=[1,2,3])\n#             disc_real_photo = tf.reduce_mean(self.p_disc(real_image, training=True),axis=[1,2,3])\n            monet_exp_loss=self.expert_loss_fn(self.m_expert(real_image, training=True),label_is_monet)\n\n            # evaluates generator loss\n            monet_gen_loss = self.gen_loss_fn(gen_monet,transformed_image,1)\n            photo_gen_loss = self.gen_loss_fn(gen_photo,transformed_image,1)\n\n            # evaluates discriminator loss\n            monet_disc_loss = self.disc_loss_fn(disc_real_monet, label_is_monet)\n            photo_disc_loss = self.disc_loss_fn(disc_real_photo, 1-label_is_monet)\n  \n\n            return {\n                \"monet_gen_loss\": monet_gen_loss,\n                \"photo_gen_loss\": photo_gen_loss,\n                \"monet_disc_loss\": monet_disc_loss,\n                \"photo_disc_loss\": photo_disc_loss,\n                \"monet_exp_loss\" : monet_exp_loss\n            }","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    def discriminator_loss(real, generated):\n        real_loss = tf.square(tf.ones_like(real) - real)\n\n        generated_loss = tf.square(generated)\n\n        total_disc_loss = real_loss + generated_loss\n\n        return total_disc_loss * 0.5\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    def generator_loss(generated):\n        return tf.square(tf.ones_like(generated) - generated)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    def calc_cycle_loss(real_image, cycled_image, LAMBDA):\n#         loss1 = tf.reduce_mean(tf.abs(real_image - cycled_image))\n#         loss1 = tf.reduce_mean(tf.keras.losses.Huber(0.5,reduction=tf.keras.losses.Reduction.NONE)(real_image, cycled_image))\n        loss1 = tf.reduce_mean(tf.keras.losses.Huber(0.5,reduction=tf.keras.losses.Reduction.NONE)(inception_model(real_image), inception_model(cycled_image)))\n\n        return LAMBDA * loss1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    def identity_loss(real_image, translated_image, LAMBDA):\n#         loss = tf.reduce_mean(tf.keras.losses.Huber(0.5,reduction=tf.keras.losses.Reduction.NONE)(real_image, translated_image))\n#         loss = tf.reduce_mean(tf.abs(real_image - translated_image))\n        loss = tf.reduce_mean(tf.keras.losses.Huber(0.5,reduction=tf.keras.losses.Reduction.NONE)(tf.nn.avg_pool2d(real_image, ksize=32, strides=16, padding=\"VALID\"), tf.nn.avg_pool2d(translated_image, ksize=32, strides=16, padding=\"VALID\")))\n        return LAMBDA *  loss","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    def pretrain_identity_loss(real_image, translated_image, LAMBDA):\n        loss = tf.reduce_mean(tf.abs(real_image - translated_image))\n        return LAMBDA *  loss","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    def pretrain_disc_loss(image,label):\n        return tf.square(label - image)\n\n    def expert_loss(generated,label):\n        return tf.keras.losses.BinaryCrossentropy(from_logits=True,reduction=tf.keras.losses.Reduction.SUM)(tf.reduce_mean(label,axis=[1,2,3]),tf.reduce_mean(generated,axis=[1,2,3]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n\n    monet_generator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n    photo_generator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n\n    monet_discriminator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n    photo_discriminator_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n    monet_expert_optimizer = tf.keras.optimizers.Adam(2e-4, beta_1=0.5)\n    \n    pt_monet_generator = Generator() # transforms photos to Monet-esque paintings\n    pt_photo_generator = Generator() # transforms Monet paintings to be more like photos\n\n    pt_monet_discriminator = Discriminator() # differentiates real Monet paintings and generated Monet paintings\n    pt_photo_discriminator = Discriminator() # differentiates real photos and generated photos\n    pt_monet_expert = Discriminator() # differentiates Monet paintings from photos\n    monet_expert = Discriminator() # differentiates Monet paintings from photos","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    cycle_gan_model = CycleGan(\n        monet_generator, photo_generator, monet_discriminator, photo_discriminator,monet_expert\n    )\n\n    cycle_gan_model.compile(\n        m_gen_optimizer = monet_generator_optimizer,\n        p_gen_optimizer = photo_generator_optimizer,\n        m_disc_optimizer = monet_discriminator_optimizer,\n        p_disc_optimizer = photo_discriminator_optimizer,\n        gen_loss_fn = generator_loss,\n        disc_loss_fn = discriminator_loss,\n        cycle_loss_fn = calc_cycle_loss,\n        identity_loss_fn = identity_loss,\n        expert_loss_fn = expert_loss,\n    )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with strategy.scope():\n    pretrain_cycle_gan_model = Pretrain_CycleGan(pt_monet_generator, pt_photo_generator, pt_monet_discriminator, pt_photo_discriminator,pt_monet_expert)\n\n    pretrain_cycle_gan_model.compile(\n        m_gen_optimizer = monet_generator_optimizer,\n        p_gen_optimizer = photo_generator_optimizer,\n        m_disc_optimizer = monet_discriminator_optimizer,\n        p_disc_optimizer = photo_discriminator_optimizer,\n        m_expert_optimizer = monet_expert_optimizer,\n        gen_loss_fn = pretrain_identity_loss,\n        disc_loss_fn = pretrain_disc_loss,\n        expert_loss_fn = expert_loss,\n    )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"hist=pretrain_cycle_gan_model.fit(pretrain_ds,validation_data=val_pretrain_ds, epochs=33).history","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"pretrain_cycle_gan_model.save_weights('premonet.h5')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cycle_gan_model.built = True\ncycle_gan_model.load_weights('premonet.h5')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"! rm premonet.h5\n# callbacks = [keras.callbacks.ModelCheckpoint(filepath='monet.h5',save_weights_only=True,save_best_only=True, monitor='val_total_cycle_loss', verbose=1)]\ndisc_m_loss1=[]\ndisc_p_loss1=[]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nfids=[]\nbest_fid=999999999\nfor epoch in range(1,48):\n\n    print(\"Epoch = \",epoch)\n    hist=cycle_gan_model.fit(gan_ds,steps_per_epoch=1500, epochs=1).history\n#     disc_m_loss1.append(hist[\"monet_disc_loss1\"][0][0][0])\n#     disc_p_loss1.append(hist[\"photo_disc_loss1\"][0][0][0])\n    if epoch>35:\n        cur_fid=FID(fast_photo_ds,monet_generator)\n        fids.append(cur_fid)\n        print(\"After epoch #{} FID = {}\\n\".format(epoch,cur_fid))\n    if epoch>42:\n            best_fid=cur_fid\n            monet_generator.save('monet_generator_'+str(epoch)+'.h5')\n\n\n# hist=cycle_gan_model.fit(gan_ds,steps_per_epoch=30, epochs=EPOCHS).history\n# hist=cycle_gan_model.fit(gan_ds,steps_per_epoch=30,validation_data=([1]), epochs=3).history\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# plt.plot(disc_m_loss1, label='monet_disc_loss1')\n# plt.plot(disc_p_loss1, label='photo_disc_loss1')\nplt.plot(np.array(fids), label='FID')\n\nplt.legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# !conda install -y gdown \n# import gdown \n# url = 'https://drive.google.com/uc?export=download&id=18UWaVxb_UHDMq4KzJqHqGSPizXXy4H7' \n# output = 'photo.jpg'\n# gdown.download(url, output)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# cycle_gan_model.built = True\n# cycle_gan_model.load_weights('monet.h5')\n# with strategy.scope():\n#     monet_generator = tf.keras.models.load_model('monet_generator.h5')\n#     cycle_gan_model.save_weights('monet.h5')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"_, ax = plt.subplots(5, 3, figsize=(32, 32))\nfor i, img in enumerate(photo_ds.batch(1).take(5)):\n    prediction = monet_generator(img, training=False)\n    cycledphoto = photo_generator(prediction, training=False)\n    prediction = (prediction * 127.5 + 127.5)[0].numpy().astype(np.uint8)\n    cycledphoto = (cycledphoto * 127.5 + 127.5)[0].numpy().astype(np.uint8)\n\n    img = (img[0] * 127.5 + 127.5).numpy().astype(np.uint8)\n\n    ax[i, 0].imshow(img)\n    ax[i, 1].imshow(prediction)\n    ax[i, 2].imshow(cycledphoto)\n    ax[i, 0].set_title(\"Input Photo\")\n    ax[i, 1].set_title(\"Monet-esque\")\n    ax[i, 2].set_title(\"Cycled Photo\")\n    ax[i, 0].axis(\"off\")\n    ax[i, 1].axis(\"off\")\n    ax[i, 2].axis(\"off\")\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"_, ax = plt.subplots(5, 3, figsize=(32, 32))\nfor i, img in enumerate(monet_ds.batch(1).take(5)):\n    prediction = photo_generator(img, training=False)\n    cycledphoto = monet_generator(prediction, training=False)\n    prediction = (prediction * 127.5 + 127.5)[0].numpy().astype(np.uint8)\n    cycledphoto = (cycledphoto * 127.5 + 127.5)[0].numpy().astype(np.uint8)\n\n    img = (img[0] * 127.5 + 127.5).numpy().astype(np.uint8)\n\n    ax[i, 0].imshow(img)\n    ax[i, 1].imshow(prediction)\n    ax[i, 2].imshow(cycledphoto)\n    ax[i, 0].set_title(\"Input Monet\")\n    ax[i, 1].set_title(\"Generated Photo\")\n    ax[i, 2].set_title(\"Cycled Monet\")\n    ax[i, 0].axis(\"off\")\n    ax[i, 1].axis(\"off\")\n    ax[i, 2].axis(\"off\")\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ds_iter = iter(photo_ds.batch(1))\nfor n_sample in range(8):\n        example_sample = next(ds_iter)\n        generated_sample = monet_generator(example_sample)\n        \n        f = plt.figure(figsize=(32, 32))\n        \n        plt.subplot(121)\n        plt.title('Input image')\n        plt.imshow(example_sample[0] * 0.5 + 0.5)\n        plt.axis('off')\n        \n        plt.subplot(122)\n        plt.title('Generated image')\n        plt.imshow(generated_sample[0] * 0.5 + 0.5)\n        plt.axis('off')\n        plt.show()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ds_iter = iter(monet_ds.batch(1))\nfor n_sample in range(8):\n\n        example_sample = next(ds_iter)\n        generated_sample = photo_generator(example_sample)\n        \n        f = plt.figure(figsize=(24, 24))\n        \n        plt.subplot(121)\n        plt.title('Input image')\n        plt.imshow(example_sample[0] * 0.5 + 0.5)\n        plt.axis('off')\n        \n        plt.subplot(122)\n        plt.title('Generated image')\n        plt.imshow(generated_sample[0] * 0.5 + 0.5)\n        plt.axis('off')\n        plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import PIL\n! mkdir ../images","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\ni = 1\nfor img in fast_photo_ds:\n    prediction = monet_generator(img, training=False).numpy()\n    prediction = (prediction * 127.5 + 127.5).astype(np.uint8)\n    for pred in prediction:\n        im = PIL.Image.fromarray(pred)\n        im.save(\"../images/\" + str(i) + \".jpg\")\n        i += 1\n    \n    \n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import shutil\nshutil.make_archive(\"/kaggle/working/images\", 'zip', \"/kaggle/images\")","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}