{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Photo to Monet — CycleGAN⚡PyTorch Lightning","metadata":{"papermill":{"duration":0.012975,"end_time":"2023-03-19T01:57:31.956786","exception":false,"start_time":"2023-03-19T01:57:31.943811","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"*Updates: Upgraded code for compatibility with Lightning 2.0.*","metadata":{}},{"cell_type":"markdown","source":"This notebook aims to implement CycleGAN in PyTorch Lightning. For simplicity, we only look at the results in a single direction (photo-to-Monet translation) here. The list of references is as follows:\n* [Documentation](https://pytorch-lightning.readthedocs.io/en/stable/notebooks/lightning_examples/basic-gan.html) for basic GAN in PyTorch Lightning.\n* [Kaggle tutorial](https://www.kaggle.com/code/amyjang/monet-cyclegan-tutorial/notebook). We adapt the U-Net generator from here.\n* Original [paper](https://arxiv.org/abs/1703.10593) and [code](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix) for CycleGAN. We also examine the ResNet generator used in the original implementation. To stabilize training, the author used an image buffer to store previously generated images to update the discriminator.\n* Original [paper](https://arxiv.org/abs/1611.04076) for LSGAN, which has shown to outperform BCE loss and is used in the original CycleGAN implementation. BCE loss may be susceptible to vanishing gradient problems and cause ineffective learning. For better training stability, we use LSGAN, which adopts the mean squared error for the adversarial criterion.\n\nMore work can be done to include evaluation metrics like the inception score (IS) or Fréchet inception distance (FID).","metadata":{"papermill":{"duration":0.010961,"end_time":"2023-03-19T01:57:31.979038","exception":false,"start_time":"2023-03-19T01:57:31.968077","status":"completed"},"tags":[]}},{"cell_type":"code","source":"!pip install deepspeed","metadata":{"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import glob\nimport os\nimport shutil\n\nimport deepspeed as ds\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport pytorch_lightning as L\nimport torch\nimport torch.nn.functional as F\nimport torchvision.transforms as T\nfrom pytorch_lightning.utilities import CombinedLoader\nfrom torch import nn\nfrom torch.utils.data import DataLoader, Dataset\nfrom torchvision.io import read_image\nfrom torchvision.utils import make_grid, save_image\n\n_ = L.seed_everything(0, workers=True)","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","papermill":{"duration":12.994572,"end_time":"2023-03-19T01:57:44.984673","exception":false,"start_time":"2023-03-19T01:57:31.990101","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---","metadata":{"papermill":{"duration":0.01099,"end_time":"2023-03-19T01:57:45.007205","exception":false,"start_time":"2023-03-19T01:57:44.996215","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# 1. Data Preprocessing","metadata":{"papermill":{"duration":0.011194,"end_time":"2023-03-19T01:57:45.030579","exception":false,"start_time":"2023-03-19T01:57:45.019385","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def show_img(img_tensor, nrow, title=\"\"):\n    img_tensor = img_tensor.detach().cpu() * 0.5 + 0.5\n    img_grid = make_grid(img_tensor, nrow=nrow).permute(1, 2, 0)\n    plt.figure(figsize=(18, 8))\n    plt.imshow(img_grid)\n    plt.axis(\"off\")\n    plt.title(title)\n    plt.show()","metadata":{"_kg_hide-input":false,"papermill":{"duration":0.020996,"end_time":"2023-03-19T01:57:45.062727","exception":false,"start_time":"2023-03-19T01:57:45.041731","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Augmenting the images.","metadata":{"papermill":{"duration":0.010988,"end_time":"2023-03-19T01:57:45.084838","exception":false,"start_time":"2023-03-19T01:57:45.073850","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"Before loading the datasets, we define `CustomTransform` for image augmentation. This improves learning by introducing more variety in the images during training instead of learning from the same set of images, especially when we only have 300 Monet paintings. We look at some basic image transformations:\n* Scaling the images larger using `Resize` and then randomly cropping to the original size of 256 with `RandomCrop` to obtain slightly different images.\n* Randomly flipping the images horizontally using `RandomHorizontalFlip`. The photos and Monet paintings do not greatly depend on the horizontal orientation.\n* Randomly changing the colors of the images using `ColorJitter`. This could mimic different lighting conditions for the photos and introduce variability in the colors of the Monet paintings.\n\nOther possible transformations can be found [here](https://pytorch.org/vision/stable/transforms.html). These transformations are only needed during model training/fitting, and we specify this using the `stage` argument. Finally, the images are scaled down for better convergence.","metadata":{"papermill":{"duration":0.010992,"end_time":"2023-03-19T01:57:45.107100","exception":false,"start_time":"2023-03-19T01:57:45.096108","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class CustomTransform(object):\n    def __init__(self, load_dim=286, target_dim=256):\n        self.transform_train = T.Compose([\n            T.Resize((load_dim, load_dim), antialias=True),\n            T.RandomCrop((target_dim, target_dim)),\n            T.RandomHorizontalFlip(p=0.5),\n            T.ColorJitter(brightness=0.2, contrast=0.2,\n                          saturation=0.2, hue=0.1),\n        ])\n        \n        # ensure images outside of training dataset are also of the same size\n        self.transform = T.Resize((target_dim, target_dim), antialias=True)\n        \n    def __call__(self, img, stage):\n        if stage == \"fit\":\n            img = self.transform_train(img)\n        else:\n            img = self.transform(img)\n        return img * 2 - 1","metadata":{"papermill":{"duration":0.021699,"end_time":"2023-03-19T01:57:45.140105","exception":false,"start_time":"2023-03-19T01:57:45.118406","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Storing the datasets.","metadata":{"papermill":{"duration":0.011074,"end_time":"2023-03-19T01:57:45.162356","exception":false,"start_time":"2023-03-19T01:57:45.151282","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"To load and store the datasets, we define a custom `Dataset` involving two main methods:\n* `__len__` to retrieve the size of the dataset.\n* `__getitem__` to get the i-th sample of images after performing the transformations described above.\n\nSimilarly, we define the `stage` argument to differentiate between the training dataset and prediction dataset when performing the transformations. Different instances of `CustomDataset` will be used to retrieve the photos and Monet paintings separately. We look at combining them while iterating through the datasets later.","metadata":{"papermill":{"duration":0.011056,"end_time":"2023-03-19T01:57:45.184590","exception":false,"start_time":"2023-03-19T01:57:45.173534","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class CustomDataset(Dataset):\n    def __init__(self, filenames, transform, stage):\n        self.filenames = filenames\n        self.transform = transform\n        self.stage = stage\n        \n    def __len__(self):\n        return len(self.filenames)\n    \n    def __getitem__(self, idx):\n        img_name = self.filenames[idx]\n        img = read_image(img_name) / 255.0\n        return self.transform(img, stage=self.stage)","metadata":{"papermill":{"duration":0.025818,"end_time":"2023-03-19T01:57:45.221633","exception":false,"start_time":"2023-03-19T01:57:45.195815","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Iterating through the datasets.","metadata":{"papermill":{"duration":0.011352,"end_time":"2023-03-19T01:57:45.244912","exception":false,"start_time":"2023-03-19T01:57:45.233560","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"To prepare the datasets, we load them into `DataLoader` separately, which can then iterate through the datasets as needed. Because the training dataset contains both the Monet paintings and photos, we pass both dataloaders into `CombinedLoader` for training. We specify the sampling mode using `mode=\"max_size_cycle\"` to stop after one complete pass of the larger dataset of photos while cycling through the smaller dataset of Monet paintings. Other modes can be found [here](https://lightning.ai/docs/pytorch/stable/api/lightning.pytorch.utilities.combined_loader.html). In contrast, we fix the validation, test and prediction datasets to only contain photos for simplicity since we are trying to generate Monet-style images in this notebook.\n\nTo organize all the steps described above for processing data, we define a custom `LightningDataModule`. A datamodule involves many methods, but we are mainly concerned with:\n* `setup` to create the different instances of `CustomDataset` defined above.\n* `train_dataloader` to load both the Monet paintings and photos for training.\n* `val_dataloader` to load only 1 batch of photos and use it repeatedly to visualize the progress of adding Monet style at different points of training.\n* `test_dataloader` to load a small subset of photos to test the results of photo-to-Monet translation on different sample photos after training. \n* `predict_dataloader` to load the entire dataset of photos and add Monet style for submission.\n\nOther possible methods can be found [here](https://pytorch-lightning.readthedocs.io/en/stable/data/datamodule.html). We limit the number of batches to be loaded for the validation and test datasets later on instead of manually extracting here.","metadata":{"papermill":{"duration":0.011068,"end_time":"2023-03-19T01:57:45.267418","exception":false,"start_time":"2023-03-19T01:57:45.256350","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# change to True to set the configuration for debugging mode\nDEBUG = False\n\nDM_CONFIG = {    \n    # the directories where the Monet paintings and photos are loaded from\n    \"monet_dir\": os.path.join(\"/kaggle/input/gan-getting-started/monet_jpg\", \"*.jpg\"),\n    \"photo_dir\": os.path.join(\"/kaggle/input/gan-getting-started/photo_jpg\", \"*.jpg\"),\n    \n    \"loader_config\": {\n        # the number of subprocesses (excluding the main process) used for data loading\n        \"num_workers\": os.cpu_count(),\n        \n        # enable faster data transfer to GPU during training\n        \"pin_memory\": torch.cuda.is_available(),\n    },\n    \n    # the validation/test batch size (mainly for visualization purposes)\n    \"sample_size\": 5,\n    \n    # the training/prediction batch size\n    \"batch_size\": 1 if not DEBUG else 1,\n}","metadata":{"papermill":{"duration":0.019339,"end_time":"2023-03-19T01:57:45.297921","exception":false,"start_time":"2023-03-19T01:57:45.278582","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class CustomDataModule(L.LightningDataModule):\n    def __init__(\n        self,\n        monet_dir,\n        photo_dir, \n        loader_config,\n        sample_size,\n        batch_size,\n    ):\n        super().__init__()\n        self.loader_config = loader_config\n        self.sample_size = sample_size\n        self.batch_size = batch_size       \n            \n        # store file paths\n        self.monet_filenames = sorted(glob.glob(monet_dir))\n        self.photo_filenames = sorted(glob.glob(photo_dir))\n        \n        # define transformations for image augmentation\n        self.transform = CustomTransform()\n        \n    def setup(self, stage):\n        if stage == \"fit\":\n            self.train_monet = CustomDataset(self.monet_filenames, self.transform, stage)\n            self.train_photo = CustomDataset(self.photo_filenames, self.transform, stage)\n            \n        if stage in [\"fit\", \"test\", \"predict\"]:\n            # to be used for test and prediction datasets as well\n            self.valid_photo = CustomDataset(self.photo_filenames, self.transform, None)\n            \n    def train_dataloader(self):\n        loader_config = {\n            \"shuffle\": True,\n            \"drop_last\": True,\n            \"batch_size\": self.batch_size,\n            **self.loader_config,\n        }\n        loader_monet = DataLoader(self.train_monet, **loader_config)\n        loader_photo = DataLoader(self.train_photo, **loader_config)\n        loaders = {\"monet\": loader_monet, \"photo\": loader_photo}\n        return CombinedLoader(loaders, mode=\"max_size_cycle\")\n    \n    def val_dataloader(self):\n        return DataLoader(\n            self.valid_photo,\n            batch_size=self.sample_size,\n            **self.loader_config,\n        )\n    \n    def test_dataloader(self):\n        return self.val_dataloader()\n    \n    def predict_dataloader(self):\n        return DataLoader(\n            self.valid_photo,\n            batch_size=self.batch_size,\n            **self.loader_config,\n        )","metadata":{"papermill":{"duration":0.022601,"end_time":"2023-03-19T01:57:45.331838","exception":false,"start_time":"2023-03-19T01:57:45.309237","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We check that the datamodule defined is working as intended by visualizing samples of the images below.","metadata":{"papermill":{"duration":0.011244,"end_time":"2023-03-19T01:57:45.354335","exception":false,"start_time":"2023-03-19T01:57:45.343091","status":"completed"},"tags":[]}},{"cell_type":"code","source":"dm_sample = CustomDataModule(batch_size=5, **{k: v for k, v in DM_CONFIG.items() if k != \"batch_size\"})\ndm_sample.setup(\"fit\")\n\ntrain_loader = dm_sample.train_dataloader()\nimgs = next(iter(train_loader))\n\nshow_img(imgs[\"monet\"], nrow=5, title=\"Augmented Monet Paintings\")\nshow_img(imgs[\"photo\"], nrow=5, title=\"Augmented Photos\")","metadata":{"papermill":{"duration":1.301484,"end_time":"2023-03-19T01:57:46.667097","exception":false,"start_time":"2023-03-19T01:57:45.365613","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---","metadata":{"papermill":{"duration":0.035229,"end_time":"2023-03-19T01:57:46.737485","exception":false,"start_time":"2023-03-19T01:57:46.702256","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# 2. Building CycleGAN Architecture","metadata":{"papermill":{"duration":0.032736,"end_time":"2023-03-19T01:57:46.803510","exception":false,"start_time":"2023-03-19T01:57:46.770774","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"### U-Net generator.","metadata":{}},{"cell_type":"markdown","source":"<img src=\"https://lh5.googleusercontent.com/9kNO6hxYJmpcfG5bOjnDazieeLC7Q8jZJi3gTtnJelbkOUL7Xz9e-3F_SNuxPpo4fZ4=w2400\" width=\"600\"/>\n\n*Example of the U-Net architecture [[source](https://paperswithcode.com/method/u-net)].*","metadata":{"papermill":{"duration":0.038219,"end_time":"2023-03-19T01:57:46.874277","exception":false,"start_time":"2023-03-19T01:57:46.836058","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"A common architecture for the CycleGAN generator is the U-Net. U-Net is a network which consists of a sequence of downsampling blocks followed by a sequence of upsampling blocks, giving it the U-shaped architecture. In the upsampling path, we concatenate the outputs of the upsampling blocks and the outputs of the downsampling blocks symmetrically. This can be seen as a kind of skip connection, facilitating information flow in deep networks and reducing the impact of vanishing gradients.","metadata":{"papermill":{"duration":0.032886,"end_time":"2023-03-19T01:57:46.941760","exception":false,"start_time":"2023-03-19T01:57:46.908874","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"### ResNet generator.","metadata":{}},{"cell_type":"markdown","source":"<img src=\"https://lh3.googleusercontent.com/drive-viewer/AAOQEOR8E8EPpiR5A_GVPeY6B6-6RJIymhoJy8Jf2ERMK8sbuIEKz1Z1uQZ9HEAOR9S8ZNzR6SLNHH69ooVwYgvfSxDMnrBFfQ=s1600\" width=\"300\"/>\n\n*Example of a residual block [[source](https://paperswithcode.com/method/residual-block)].*","metadata":{}},{"cell_type":"markdown","source":"In the original CycleGAN implementation, the author used a ResNet generator. Similar to the U-Net architecture, the ResNet generator consists of the downsampling path and upsampling path. The difference is that the ResNet generator does not have the long skip connections from the concatenations of outputs. Instead, the ResNet generator uses residual blocks between the two paths. These residual blocks have short skip connections where the original input is added to the output.","metadata":{}},{"cell_type":"markdown","source":"### Downsampling blocks.","metadata":{"papermill":{"duration":0.032774,"end_time":"2023-03-19T01:57:47.007590","exception":false,"start_time":"2023-03-19T01:57:46.974816","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"The downsampling blocks use convolution layers to increase the number of feature maps while reducing the dimensions of the 2D image.","metadata":{"papermill":{"duration":0.032597,"end_time":"2023-03-19T01:57:47.073128","exception":false,"start_time":"2023-03-19T01:57:47.040531","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class Downsampling(nn.Module):\n    def __init__(\n        self,\n        in_channels,\n        out_channels,\n        kernel_size=4,\n        stride=2,\n        padding=1,\n        norm=True,\n        lrelu=True,\n    ):\n        super().__init__()\n        self.block = nn.Sequential(\n            nn.Conv2d(in_channels, out_channels,\n                      kernel_size=kernel_size, stride=stride, padding=padding, bias=not norm),\n        )\n        if norm:\n            self.block.append(nn.InstanceNorm2d(out_channels, affine=True))\n        if lrelu is not None:\n            self.block.append(nn.LeakyReLU(0.2, True) if lrelu else nn.ReLU(True))\n        \n    def forward(self, x):\n        return self.block(x)","metadata":{"papermill":{"duration":0.044778,"end_time":"2023-03-19T01:57:47.150705","exception":false,"start_time":"2023-03-19T01:57:47.105927","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Upsampling blocks.","metadata":{"papermill":{"duration":0.032956,"end_time":"2023-03-19T01:57:47.216301","exception":false,"start_time":"2023-03-19T01:57:47.183345","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"On the other hand, the upsampling blocks contain transposed convolution layers, which combine the learned features to output an image with the original size.","metadata":{"papermill":{"duration":0.032817,"end_time":"2023-03-19T01:57:47.282273","exception":false,"start_time":"2023-03-19T01:57:47.249456","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class Upsampling(nn.Module):\n    def __init__(\n        self,\n        in_channels,\n        out_channels,\n        kernel_size=4,\n        stride=2,\n        padding=1,\n        output_padding=0,\n        dropout=False,\n    ):\n        super().__init__()\n        self.block = nn.Sequential(\n            nn.ConvTranspose2d(in_channels, out_channels,\n                               kernel_size=kernel_size, stride=stride, \n                               padding=padding, output_padding=output_padding, bias=False),\n            nn.InstanceNorm2d(out_channels, affine=True),\n        )\n        if dropout:\n            self.block.append(nn.Dropout(0.5))\n        self.block.append(nn.ReLU(True))\n        \n    def forward(self, x):\n        return self.block(x)","metadata":{"papermill":{"duration":0.044362,"end_time":"2023-03-19T01:57:47.359186","exception":false,"start_time":"2023-03-19T01:57:47.314824","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Residual blocks.","metadata":{}},{"cell_type":"markdown","source":"As described above, the residual blocks have convolution layers where the original input is added to the output.","metadata":{}},{"cell_type":"code","source":"class ResBlock(nn.Module):\n    def __init__(self, in_channels, kernel_size=3, padding=1):\n        super().__init__()\n        self.block = nn.Sequential(\n            nn.ReflectionPad2d(padding),\n            Downsampling(in_channels, in_channels,\n                         kernel_size=kernel_size, stride=1, padding=0, lrelu=False),\n            nn.ReflectionPad2d(padding),\n            Downsampling(in_channels, in_channels,\n                         kernel_size=kernel_size, stride=1, padding=0, lrelu=None),\n        )\n        \n    def forward(self, x):\n        return x + self.block(x)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Building the generator.","metadata":{}},{"cell_type":"markdown","source":"With the building blocks defined, we can now build our CycleGAN generator. For reference, the output size of each block is commented below.","metadata":{}},{"cell_type":"code","source":"class UNetGenerator(nn.Module):\n    def __init__(self, hid_channels, in_channels, out_channels):\n        super().__init__()\n        self.downsampling_path = nn.Sequential(\n            Downsampling(in_channels, hid_channels, norm=False), # 64x128x128\n            Downsampling(hid_channels, hid_channels*2), # 128x64x64\n            Downsampling(hid_channels*2, hid_channels*4), # 256x32x32\n            Downsampling(hid_channels*4, hid_channels*8), # 512x16x16\n            Downsampling(hid_channels*8, hid_channels*8), # 512x8x8\n            Downsampling(hid_channels*8, hid_channels*8), # 512x4x4\n            Downsampling(hid_channels*8, hid_channels*8), # 512x2x2\n            Downsampling(hid_channels*8, hid_channels*8, norm=False), # 512x1x1, instance norm does not work on 1x1\n        )\n        self.upsampling_path = nn.Sequential(\n            Upsampling(hid_channels*8, hid_channels*8, dropout=True), # (512+512)x2x2\n            Upsampling(hid_channels*16, hid_channels*8, dropout=True), # (512+512)x4x4\n            Upsampling(hid_channels*16, hid_channels*8, dropout=True), # (512+512)x8x8\n            Upsampling(hid_channels*16, hid_channels*8), # (512+512)x16x16\n            Upsampling(hid_channels*16, hid_channels*4), # (256+256)x32x32\n            Upsampling(hid_channels*8, hid_channels*2), # (128+128)x64x64\n            Upsampling(hid_channels*4, hid_channels), # (64+64)x128x128\n        )\n        self.feature_block = nn.Sequential(\n            nn.ConvTranspose2d(hid_channels*2, out_channels,\n                               kernel_size=4, stride=2, padding=1), # 3x256x256\n            nn.Tanh(),\n        )\n        \n    def forward(self, x):\n        skips = []\n        for down in self.downsampling_path:\n            x = down(x)\n            skips.append(x)\n        skips = reversed(skips[:-1])\n\n        for up, skip in zip(self.upsampling_path, skips):\n            x = up(x)\n            x = torch.cat([x, skip], dim=1)\n        return self.feature_block(x)\n    \nclass ResNetGenerator(nn.Module):\n    def __init__(self, hid_channels, in_channels, out_channels, num_resblocks):\n        super().__init__()\n        self.model = nn.Sequential(\n            # downsampling path\n            nn.ReflectionPad2d(3),\n            Downsampling(in_channels, hid_channels,\n                         kernel_size=7, stride=1, padding=0, lrelu=False), # 64x256x256\n            Downsampling(hid_channels, hid_channels*2, kernel_size=3, lrelu=False), # 128x128x128\n            Downsampling(hid_channels*2, hid_channels*4, kernel_size=3, lrelu=False), # 256x64x64\n            \n            # residual blocks\n            *[ResBlock(hid_channels*4) for _ in range(num_resblocks)], # 256x64x64\n            \n            # upsampling path\n            Upsampling(hid_channels*4, hid_channels*2, kernel_size=3, output_padding=1), # 128x128x128\n            Upsampling(hid_channels*2, hid_channels, kernel_size=3, output_padding=1), # 64x256x256\n            nn.ReflectionPad2d(3),\n            nn.Conv2d(hid_channels, out_channels, kernel_size=7, stride=1, padding=0), # 3x256x256\n            nn.Tanh(),\n        )\n        \n    def forward(self, x):\n        return self.model(x)\n    \ndef get_gen(gen_name, hid_channels, num_resblocks, in_channels=3, out_channels=3):\n    if gen_name == \"unet\":\n        return UNetGenerator(hid_channels, in_channels, out_channels)\n    elif gen_name == \"resnet\":\n        return ResNetGenerator(hid_channels, in_channels, out_channels, num_resblocks)\n    else:\n        raise NotImplementedError(f\"Generator name '{gen_name}' not recognized.\")","metadata":{"papermill":{"duration":0.047175,"end_time":"2023-03-19T01:57:48.061105","exception":false,"start_time":"2023-03-19T01:57:48.013930","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### PatchGAN discriminator.","metadata":{"papermill":{"duration":0.033167,"end_time":"2023-03-19T01:57:48.132136","exception":false,"start_time":"2023-03-19T01:57:48.098969","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"<img src=\"https://lh6.googleusercontent.com/UhJiaTOQWgfHQlWq50IMGBvdkJ3NDggC449cxud8XVlSxUrule8f5LyoLUV8aaYemGw=w2400\" width=\"300\"/>\n\n*Diagram of how the PatchGAN discriminator works [[source](https://www.researchgate.net/figure/PatchGAN-discriminator-Each-value-of-the-output-matrix-represents-the-probability-of_fig1_323904616)].*","metadata":{"papermill":{"duration":0.033278,"end_time":"2023-03-19T01:57:48.198341","exception":false,"start_time":"2023-03-19T01:57:48.165063","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"Unlike conventional networks that output a single probability of the input image being real or fake, CycleGAN uses the PatchGAN discriminator that outputs a matrix of values. Intuitively, each value of the output matrix checks the corresponding portion of the input image. Values closer to 1 indicate real classification and values closer to 0 indicate fake classification.","metadata":{"papermill":{"duration":0.034539,"end_time":"2023-03-19T01:57:48.266305","exception":false,"start_time":"2023-03-19T01:57:48.231766","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"### Building the discriminator.","metadata":{}},{"cell_type":"markdown","source":"In general, the PatchGAN discriminator consists of a sequence of convolution layers, which can be built using the downsampling blocks defined earlier. For reference, the output size of each block is commented below.","metadata":{}},{"cell_type":"code","source":"class Discriminator(nn.Module):\n    def __init__(self, hid_channels, in_channels=3):\n        super().__init__()\n        self.block = nn.Sequential(\n            Downsampling(in_channels, hid_channels, norm=False), # 64x128x128\n            Downsampling(hid_channels, hid_channels*2), # 128x64x64\n            Downsampling(hid_channels*2, hid_channels*4), # 256x32x32\n            Downsampling(hid_channels*4, hid_channels*8, stride=1), # 512x31x31\n            nn.Conv2d(hid_channels*8, 1, kernel_size=4, padding=1), # 1x30x30\n        )\n        \n    def forward(self, x):\n        return self.block(x)","metadata":{"papermill":{"duration":0.043903,"end_time":"2023-03-19T01:57:48.343071","exception":false,"start_time":"2023-03-19T01:57:48.299168","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Image buffer.","metadata":{}},{"cell_type":"markdown","source":"The original CycleGAN implementation updates the discriminator using a history of generated images instead of the latest images generated. This is done by setting up an image buffer that stores previously generated images. With probability 0.5, each newly generated image is swapped with a previously generated image stored in the buffer. This stabilizes training by giving the discriminator access to past information.","metadata":{}},{"cell_type":"code","source":"# adapted from https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix/blob/master/util/image_pool.py\nclass ImageBuffer(object):\n    def __init__(self, buffer_size):\n        self.buffer_size = buffer_size\n        if self.buffer_size > 0:\n            # the current capacity of the buffer\n            self.curr_cap = 0\n            # initialize buffer as empty list\n            self.buffer = []\n    \n    def __call__(self, imgs):\n        # the buffer is not used\n        if self.buffer_size == 0:\n            return imgs\n        \n        return_imgs = []\n        for img in imgs:\n            img = img.unsqueeze(dim=0)\n            \n            # fill buffer to maximum capacity\n            if self.curr_cap < self.buffer_size:\n                self.curr_cap += 1\n                self.buffer.append(img)\n                return_imgs.append(img)\n            else:\n                p = np.random.uniform(low=0., high=1.)\n                \n                # swap images between input and buffer with probability 0.5\n                if p > 0.5:\n                    idx = np.random.randint(low=0, high=self.buffer_size)\n                    tmp = self.buffer[idx].clone()\n                    self.buffer[idx] = img\n                    return_imgs.append(tmp)\n                else:\n                    return_imgs.append(img)\n        return torch.cat(return_imgs, dim=0)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### CycleGAN.","metadata":{"papermill":{"duration":0.044686,"end_time":"2023-03-19T01:57:48.419994","exception":false,"start_time":"2023-03-19T01:57:48.375308","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"With the generator and discriminator architectures defined, we can now build CycleGAN, which consists of two generators and two discriminators:\n* Generator for photo-to-Monet translation (`gen_PM`).\n* Generator for Monet-to-photo translation (`gen_MP`).\n* Discriminator for Monet paintings (`disc_M`).\n* Discriminator for photos (`disc_P`).\n\nWe use the Adam optimizer for model training. To optimize the parameters, we need to define the loss functions:\n* **Discriminator loss**. For real images fed into the discriminator, the output matrix is compared against a matrix of 1s using the mean squared error. For fake images, the output matrix is compared against a matrix of 0s. This suggests that to minimize loss, the perfect discriminator outputs a matrix of 1s for real images and a matrix of 0s for fake images.\n* **Generator loss**. This is composed of three different loss functions below.\n  * **Adversarial loss**. Fake images are fed into the discriminator and the output matrix is compared against a matrix of 1s using the mean squared error. To minimize loss, the generator needs to 'fool' the discriminator into thinking that the fake images are real and output a matrix of 1s.\n  * **Identity loss**. When a Monet painting is fed into the photo-to-Monet generator, we should get back the same Monet painting because nothing needs to be transformed. The same applies for photos fed into the Monet-to-photo generator. To encourage identity mapping, the difference in pixel values between the input image and generated image is measured using the l1 loss.\n  * **Cycle loss**. When a Monet painting is fed into the Monet-to-photo generator, and the generated image is fed back into the photo-to-Monet generator, it should transform back into the original Monet painting. The same applies for photos passed to the two generators to get back the original photos. To preserve information throughout this cycle, the l1 loss is used to measure the difference between the original image and the reconstructed image.\n\nFrom the above, the mean squared error and the l1 loss are defined as the **adversarial criterion** and the **reconstruction criterion** respectively.","metadata":{"papermill":{"duration":0.033641,"end_time":"2023-03-19T01:57:48.487251","exception":false,"start_time":"2023-03-19T01:57:48.453610","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"### Building the CycleGAN model.","metadata":{}},{"cell_type":"markdown","source":"To organize the code for modeling, we define the loss functions within `LightningModule`, together with the following methods:\n* `forward` to generate Monet-style images given input photos.\n* `setup` to initialize the weights using the normal distribution and the biases to 0s.\n* `configure_optimizers` to set up the Adam optimizers and learning rate schedules. We use a constant learning rate and then linearly decay over the remaining training steps.\n* `training_step` to compute the loss functions for the generators and discriminators and update weights.\n* `validation_step` to visualize the progress of adding Monet style to the same batch of photos during different points of training.\n* `test_step` to test the results of photo-to-Monet translation on other sample photos after training.\n* `predict_step` to run the `forward` method during prediction.\n* `on_train_epoch_start` to record the learning rates used in the current epoch.\n* `on_train_epoch_end` to print the state of the current epoch and update learning rates.\n* `on_train_end` to print a simple message indicating the end of training.\n* `on_predict_epoch_end` to print a message specifying the number of images generated.\n\nOther useful methods can be found [here](https://pytorch-lightning.readthedocs.io/en/stable/common/lightning_module.html).","metadata":{"papermill":{"duration":0.033641,"end_time":"2023-03-19T01:57:48.487251","exception":false,"start_time":"2023-03-19T01:57:48.453610","status":"completed"},"tags":[]}},{"cell_type":"code","source":"MODEL_CONFIG = {\n    # the type of generator, and the number of residual blocks if ResNet generator is used\n    \"gen_name\": \"unet\", # types: 'unet', 'resnet'\n    \"num_resblocks\": 6,\n    \n    # the number of filters in the first layer for the generators and discriminators\n    \"hid_channels\": 64,\n    \n    # using DeepSpeed's FusedAdam (currently GPU only) is slightly faster\n    \"optimizer\": ds.ops.adam.FusedAdam if torch.cuda.is_available() else torch.optim.Adam,\n    \n    # the learning rate and beta parameters for the Adam optimizer\n    \"lr\": 2e-4,\n    \"betas\": (0.5, 0.999),\n    \n    # the weights used in the identity loss and cycle loss\n    \"lambda_idt\": 0.5,\n    \"lambda_cycle\": (10, 10), # (MPM direction, PMP direction)\n    \n    # the size of the buffer that stores previously generated images\n    \"buffer_size\": 100,\n    \n    # the number of epochs for training\n    \"num_epochs\": 18 if not DEBUG else 2,\n    \n    # the number of epochs before starting the learning rate decay\n    \"decay_epochs\": 18 if not DEBUG else 1,\n}","metadata":{"papermill":{"duration":0.042883,"end_time":"2023-03-19T01:57:48.564164","exception":false,"start_time":"2023-03-19T01:57:48.521281","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class CycleGAN(L.LightningModule):\n    def __init__(\n        self,\n        gen_name,\n        num_resblocks,\n        hid_channels,\n        optimizer,\n        lr,\n        betas,\n        lambda_idt,\n        lambda_cycle,\n        buffer_size,\n        num_epochs,\n        decay_epochs,\n    ):\n        super().__init__()\n        self.save_hyperparameters(ignore=[\"optimizer\"])\n        self.optimizer = optimizer\n        self.automatic_optimization = False\n        \n        # define generators and discriminators\n        self.gen_PM = get_gen(gen_name, hid_channels, num_resblocks)\n        self.gen_MP = get_gen(gen_name, hid_channels, num_resblocks)\n        self.disc_M = Discriminator(hid_channels)\n        self.disc_P = Discriminator(hid_channels)\n        \n        # initialize buffers to store fake images\n        self.buffer_fake_M = ImageBuffer(buffer_size)\n        self.buffer_fake_P = ImageBuffer(buffer_size)\n        \n    def forward(self, img):\n        return self.gen_PM(img)   \n            \n    def init_weights(self):\n        def init_fn(m):\n            if isinstance(m, (nn.Conv2d, nn.ConvTranspose2d, nn.InstanceNorm2d)):\n                nn.init.normal_(m.weight, 0.0, 0.02)\n                if m.bias is not None:\n                    nn.init.constant_(m.bias, 0.0)\n        \n        for net in [self.gen_PM, self.gen_MP, self.disc_M, self.disc_P]:\n            net.apply(init_fn)\n        \n    def setup(self, stage):\n        if stage == \"fit\":\n            self.init_weights()\n            print(\"Model initialized.\")\n            \n    def get_lr_scheduler(self, optimizer):\n        def lr_lambda(epoch):\n            len_decay_phase = self.hparams.num_epochs - self.hparams.decay_epochs + 1.0\n            curr_decay_step = max(0, epoch - self.hparams.decay_epochs + 1.0)\n            val = 1.0 - curr_decay_step / len_decay_phase\n            return max(0.0, val)\n        \n        return torch.optim.lr_scheduler.LambdaLR(optimizer, lr_lambda=lr_lambda)\n    \n    def configure_optimizers(self):\n        opt_config = {\n            \"lr\": self.hparams.lr,\n            \"betas\": self.hparams.betas,\n        }\n        opt_gen = self.optimizer(\n            list(self.gen_PM.parameters()) + list(self.gen_MP.parameters()),\n            **opt_config,\n        )\n        opt_disc = self.optimizer(\n            list(self.disc_M.parameters()) + list(self.disc_P.parameters()),\n            **opt_config,\n        )\n        optimizers = [opt_gen, opt_disc]\n        schedulers = [self.get_lr_scheduler(opt) for opt in optimizers]\n        return optimizers, schedulers\n        \n    def adv_criterion(self, y_hat, y):\n        return F.mse_loss(y_hat, y)\n    \n    def recon_criterion(self, y_hat, y):\n        return F.l1_loss(y_hat, y)\n    \n    def get_adv_loss(self, fake, disc):\n        fake_hat = disc(fake)\n        real_labels = torch.ones_like(fake_hat)\n        adv_loss = self.adv_criterion(fake_hat, real_labels)\n        return adv_loss\n    \n    def get_idt_loss(self, real, idt, lambda_cycle):\n        idt_loss = self.recon_criterion(idt, real)\n        return self.hparams.lambda_idt * lambda_cycle * idt_loss\n    \n    def get_cycle_loss(self, real, recon, lambda_cycle):\n        cycle_loss = self.recon_criterion(recon, real)\n        return lambda_cycle * cycle_loss\n    \n    def get_gen_loss(self):\n        # calculate adversarial loss\n        adv_loss_PM = self.get_adv_loss(self.fake_M, self.disc_M)\n        adv_loss_MP = self.get_adv_loss(self.fake_P, self.disc_P)\n        total_adv_loss = adv_loss_PM + adv_loss_MP\n        \n        # calculate identity loss\n        lambda_cycle = self.hparams.lambda_cycle\n        idt_loss_MM = self.get_idt_loss(self.real_M, self.idt_M, lambda_cycle[0])\n        idt_loss_PP = self.get_idt_loss(self.real_P, self.idt_P, lambda_cycle[1])\n        total_idt_loss = idt_loss_MM + idt_loss_PP\n        \n        # calculate cycle loss\n        cycle_loss_MPM = self.get_cycle_loss(self.real_M, self.recon_M, lambda_cycle[0])\n        cycle_loss_PMP = self.get_cycle_loss(self.real_P, self.recon_P, lambda_cycle[1])\n        total_cycle_loss = cycle_loss_MPM + cycle_loss_PMP\n        \n        # combine losses\n        gen_loss = total_adv_loss + total_idt_loss + total_cycle_loss\n        return gen_loss\n    \n    def get_disc_loss(self, real, fake, disc):\n        # calculate loss on real images\n        real_hat = disc(real)\n        real_labels = torch.ones_like(real_hat)\n        real_loss = self.adv_criterion(real_hat, real_labels)\n        \n        # calculate loss on fake images\n        fake_hat = disc(fake.detach())\n        fake_labels = torch.zeros_like(fake_hat)\n        fake_loss = self.adv_criterion(fake_hat, fake_labels)\n        \n        # combine losses\n        disc_loss = (fake_loss + real_loss) * 0.5\n        return disc_loss\n    \n    def get_disc_loss_M(self):\n        fake_M = self.buffer_fake_M(self.fake_M)\n        return self.get_disc_loss(self.real_M, fake_M, self.disc_M)\n    \n    def get_disc_loss_P(self):\n        fake_P = self.buffer_fake_P(self.fake_P)\n        return self.get_disc_loss(self.real_P, fake_P, self.disc_P)\n    \n    def training_step(self, batch, batch_idx):\n        self.real_M = batch[\"monet\"]\n        self.real_P = batch[\"photo\"]\n        opt_gen, opt_disc = self.optimizers()\n\n        # generate fake images\n        self.fake_M = self.gen_PM(self.real_P)\n        self.fake_P = self.gen_MP(self.real_M)\n        \n        # generate identity images\n        self.idt_M = self.gen_PM(self.real_M)\n        self.idt_P = self.gen_MP(self.real_P)\n        \n        # reconstruct images\n        self.recon_M = self.gen_PM(self.fake_P)\n        self.recon_P = self.gen_MP(self.fake_M)\n    \n        # train generators\n        self.toggle_optimizer(opt_gen)\n        gen_loss = self.get_gen_loss()        \n        opt_gen.zero_grad()\n        self.manual_backward(gen_loss)\n        opt_gen.step()\n        self.untoggle_optimizer(opt_gen)\n        \n        # train discriminators\n        self.toggle_optimizer(opt_disc)\n        disc_loss_M = self.get_disc_loss_M()\n        disc_loss_P = self.get_disc_loss_P()\n        opt_disc.zero_grad()\n        self.manual_backward(disc_loss_M)\n        self.manual_backward(disc_loss_P)\n        opt_disc.step()\n        self.untoggle_optimizer(opt_disc)\n        \n        # record training losses\n        metrics = {\n            \"gen_loss\": gen_loss,\n            \"disc_loss_M\": disc_loss_M,\n            \"disc_loss_P\": disc_loss_P,\n        }\n        self.log_dict(metrics, on_step=False, on_epoch=True, prog_bar=True)\n        \n    def validation_step(self, batch, batch_idx):\n        self.display_results(batch, batch_idx, \"validate\")\n    \n    def test_step(self, batch, batch_idx):\n        self.display_results(batch, batch_idx, \"test\")\n        \n    def predict_step(self, batch, batch_idx):\n        return self(batch)\n    \n    def display_results(self, batch, batch_idx, stage):\n        real_P = batch\n        fake_M = self(real_P)\n        \n        if stage == \"validate\":\n            title = f\"Epoch {self.current_epoch+1}: Photo-to-Monet Translation\"\n        else:\n            title = f\"Sample {batch_idx+1}: Photo-to-Monet Translation\"\n\n        show_img(\n            torch.cat([real_P, fake_M], dim=0),\n            nrow=len(real_P),\n            title=title,\n        )\n    \n    def on_train_epoch_start(self):\n        # record learning rates\n        curr_lr = self.lr_schedulers()[0].get_last_lr()[0]\n        self.log(\"lr\", curr_lr, on_step=False, on_epoch=True, prog_bar=True)\n        \n    def on_train_epoch_end(self):\n        # update learning rates\n        for sch in self.lr_schedulers():\n            sch.step()\n        \n        # print current state of epoch\n        logged_values = self.trainer.progress_bar_metrics\n        print(\n            f\"Epoch {self.current_epoch+1}\",\n            *[f\"{k}: {v:.5f}\" for k, v in logged_values.items()],\n            sep=\" - \",\n        )\n        \n    def on_train_end(self):\n        print(\"Training ended.\")\n        \n    def on_predict_epoch_end(self):\n        predictions = self.trainer.predict_loop.predictions\n        num_batches = len(predictions)\n        batch_size = predictions[0].shape[0]\n        last_batch_diff = batch_size - predictions[-1].shape[0]\n        print(f\"Number of images generated: {num_batches*batch_size-last_batch_diff}.\")","metadata":{"papermill":{"duration":0.064437,"end_time":"2023-03-19T01:57:48.661507","exception":false,"start_time":"2023-03-19T01:57:48.597070","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---","metadata":{"papermill":{"duration":0.033909,"end_time":"2023-03-19T01:57:48.728836","exception":false,"start_time":"2023-03-19T01:57:48.694927","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# 3. Model Training","metadata":{"papermill":{"duration":0.032703,"end_time":"2023-03-19T01:57:48.794464","exception":false,"start_time":"2023-03-19T01:57:48.761761","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"To start training the model, we use `Trainer` to automatically handle the training loop by running the `fit` method.","metadata":{"papermill":{"duration":0.032684,"end_time":"2023-03-19T01:57:48.859924","exception":false,"start_time":"2023-03-19T01:57:48.827240","status":"completed"},"tags":[]}},{"cell_type":"code","source":"TRAIN_CONFIG = {\n    \"accelerator\": \"gpu\" if torch.cuda.is_available() else \"cpu\",\n    \n    # train on 16-bit precision\n    \"precision\": \"16-mixed\" if torch.cuda.is_available() else 32,\n    \n    # train on single GPU\n    \"devices\": 1,\n    \n    # save checkpoint only for last epoch by default\n    \"enable_checkpointing\": True,\n    \n    # disable logging for simplicity\n    \"logger\": False,\n    \n    # the number of epochs for training (we limit the number of train/predict batches during debugging)\n    \"max_epochs\": MODEL_CONFIG[\"num_epochs\"],\n    \"limit_train_batches\": 1.0 if not DEBUG else 2,\n    \"limit_predict_batches\": 1.0 if not DEBUG else 5,\n    \n    # the maximum amount of time for training, in case we exceed run-time of 5 hours\n    \"max_time\": {\"hours\": 4, \"minutes\": 55},\n    \n    # use a small subset of photos for validation/testing (we limit here for flexibility)\n    \"limit_val_batches\": 1,\n    \"limit_test_batches\": 5,\n    \n    # disable sanity check before starting the training routine\n    \"num_sanity_val_steps\": 0,\n    \n    # the frequency to visualize the progress of adding Monet style\n    \"check_val_every_n_epoch\": 6 if not DEBUG else 1,\n}","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dm = CustomDataModule(**DM_CONFIG)\nmodel = CycleGAN(**MODEL_CONFIG)\ntrainer = L.Trainer(**TRAIN_CONFIG)\n\ntrainer.fit(model, datamodule=dm)","metadata":{"papermill":{"duration":15422.135167,"end_time":"2023-03-19T06:14:51.028526","exception":false,"start_time":"2023-03-19T01:57:48.893359","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Testing the model on other sample photos.","metadata":{}},{"cell_type":"code","source":"_ = trainer.test(model, datamodule=dm)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"---","metadata":{"papermill":{"duration":0.147734,"end_time":"2023-03-19T06:14:52.876958","exception":false,"start_time":"2023-03-19T06:14:52.729224","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# 4. Submission","metadata":{"papermill":{"duration":0.149119,"end_time":"2023-03-19T06:14:53.175117","exception":false,"start_time":"2023-03-19T06:14:53.025998","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"Computing the predictions can be done by running the `predict` method to generate the Monet-style images given the input photos.","metadata":{"papermill":{"duration":0.148192,"end_time":"2023-03-19T06:14:53.473205","exception":false,"start_time":"2023-03-19T06:14:53.325013","status":"completed"},"tags":[]}},{"cell_type":"code","source":"predictions = trainer.predict(model, datamodule=dm)","metadata":{"papermill":{"duration":126.035671,"end_time":"2023-03-19T06:16:59.658291","exception":false,"start_time":"2023-03-19T06:14:53.622620","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Saving the generated images.","metadata":{"papermill":{"duration":0.146966,"end_time":"2023-03-19T06:16:59.953278","exception":false,"start_time":"2023-03-19T06:16:59.806312","status":"completed"},"tags":[]}},{"cell_type":"code","source":"os.makedirs(\"../images\", exist_ok=True)\n\nidx = 0\nfor tensor in predictions:\n    for monet in tensor:\n        save_image(\n            monet.float().squeeze() * 0.5 + 0.5, \n            fp=f\"../images/{idx}.jpg\",\n        )\n        idx += 1\n\nshutil.make_archive(\"/kaggle/working/images\", \"zip\", \"/kaggle/images\")","metadata":{"papermill":{"duration":31.787796,"end_time":"2023-03-19T06:17:31.887070","exception":false,"start_time":"2023-03-19T06:17:00.099274","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":" ---","metadata":{"papermill":{"duration":0.149773,"end_time":"2023-03-19T06:17:32.198249","exception":false,"start_time":"2023-03-19T06:17:32.048476","status":"completed"},"tags":[]}}]}