{"metadata":{"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":21755,"databundleVersionId":1475600,"sourceType":"competition"}],"dockerImageVersionId":30716,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Requirements","metadata":{}},{"cell_type":"code","source":"import cv2\nimport os \nimport torch.nn as nn\nimport torch\nfrom torch.utils.data import DataLoader, Dataset\nimport random\nfrom PIL import Image\nimport torch.nn.functional as F\nimport matplotlib.pyplot as plt","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.129859Z","iopub.execute_input":"2024-06-06T15:41:40.130248Z","iopub.status.idle":"2024-06-06T15:41:40.136535Z","shell.execute_reply.started":"2024-06-06T15:41:40.130219Z","shell.execute_reply":"2024-06-06T15:41:40.135359Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data ","metadata":{}},{"cell_type":"code","source":"class MonetDataset(Dataset):\n    def __init__(self, root, transforms=None):\n        self.root_path = root\n        self.photo_paths = os.listdir(os.path.join(root, 'photo_jpg'))\n        self.monet_paths = os.listdir(os.path.join(root, 'monet_jpg'))\n        self.transform = transforms\n\n    def __len__(self):\n        return len(self.monet_paths)\n\n    def __getitem__(self, idx):\n        photoPath = os.path.join(self.root_path, 'photo_jpg', self.photo_paths[idx])\n        monetPath = os.path.join(self.root_path, 'monet_jpg', self.monet_paths[idx])\n        \n        if self.transform:\n            photo = self.transform(self.read_img(photoPath))\n            monet = self.transform(self.read_img(monetPath))\n\n        else:\n            photo = self.read_img(photoPath)\n            monet = self.read_img(monetPath)\n\n        return photo, monet\n\n    def to_tensor(self, img):\n        return torch.tensor(img, dtype=torch.float32).permute(2, 0, 1)\n    \n    def read_img(self, path):\n        return cv2.cvtColor(cv2.imread(path), cv2.COLOR_BGR2RGB)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.149384Z","iopub.execute_input":"2024-06-06T15:41:40.149777Z","iopub.status.idle":"2024-06-06T15:41:40.161237Z","shell.execute_reply.started":"2024-06-06T15:41:40.149744Z","shell.execute_reply":"2024-06-06T15:41:40.159966Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Sample","metadata":{}},{"cell_type":"code","source":"sample = MonetDataset('/kaggle/input/gan-getting-started')\nprint(len(sample))\nphoto, monet = sample[0]","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.163755Z","iopub.execute_input":"2024-06-06T15:41:40.164103Z","iopub.status.idle":"2024-06-06T15:41:40.186980Z","shell.execute_reply.started":"2024-06-06T15:41:40.164075Z","shell.execute_reply":"2024-06-06T15:41:40.185948Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(photo.shape)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.188263Z","iopub.execute_input":"2024-06-06T15:41:40.188581Z","iopub.status.idle":"2024-06-06T15:41:40.194125Z","shell.execute_reply.started":"2024-06-06T15:41:40.188554Z","shell.execute_reply":"2024-06-06T15:41:40.193040Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(10, 5))\nfig, ax = plt.subplots(1, 2)\nax[0].imshow(photo)\nax[1].imshow(monet)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.195310Z","iopub.execute_input":"2024-06-06T15:41:40.195638Z","iopub.status.idle":"2024-06-06T15:41:40.593172Z","shell.execute_reply.started":"2024-06-06T15:41:40.195610Z","shell.execute_reply":"2024-06-06T15:41:40.592043Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model","metadata":{}},{"cell_type":"markdown","source":"## Generator","metadata":{}},{"cell_type":"markdown","source":"### ResNet","metadata":{}},{"cell_type":"code","source":"def weight_init(m):\n    if any(isinstance(m, _m) for _m in [nn.Conv2d, nn.ConvTranspose2d]):\n        nn.init.normal_(m.weight, mean=0.0, std=0.02)\n        if m.bias is not None:\n            nn.init.constant_(m.bias, 0)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.596683Z","iopub.execute_input":"2024-06-06T15:41:40.597495Z","iopub.status.idle":"2024-06-06T15:41:40.604745Z","shell.execute_reply.started":"2024-06-06T15:41:40.597453Z","shell.execute_reply":"2024-06-06T15:41:40.603605Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class ResidualBlock(nn.Module):\n    def __init__(self, in_channels, out_channels, kernel_size=3, stride=1, isDropout=False):\n        super().__init__()\n        self._isDropout = isDropout\n        # 第一個卷積層\n        self.padding1 = nn.ReflectionPad2d(kernel_size//2)\n        self.conv1 = nn.Conv2d(in_channels, out_channels, kernel_size, stride=stride)\n        self.bn1 = nn.InstanceNorm2d(out_channels)\n        \n        # 第二個卷積層\n        self.padding2 = nn.ReflectionPad2d(kernel_size//2)\n        self.conv2 = nn.Conv2d(out_channels, out_channels, kernel_size, stride=stride)\n        self.bn2 = nn.InstanceNorm2d(out_channels)\n        \n        # 捷徑連接\n        self.shortcut = nn.Sequential()\n        if stride != 1 or in_channels != out_channels:\n            # 捷徑路徑上的1x1卷積調整維度和步長\n            self.shortcut = nn.Sequential(\n                nn.ReflectionPad2d(kernel_size//2),\n                nn.Conv2d(in_channels, out_channels, kernel_size=1, stride=stride, bias=False),\n                nn.BatchNorm2d(out_channels)\n            )\n\n        self.relu = nn.ReLU(inplace=True)\n    \n    def forward(self, x):\n        identity = self.shortcut(x)\n        \n        out = self.padding1(x)\n        out = self.conv1(out)\n        out = self.bn1(out)\n\n        out = self.relu(out)\n\n        out = self.padding2(out)\n        out = self.conv2(out)\n        out = self.bn2(out)\n\n        out += identity  # 將捷徑的結果加到主路徑的輸出上\n        out = self.relu(out)\n        \n        if self._isDropout:\n            out = F.dropout(out, 0.5)\n\n        return out","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.605982Z","iopub.execute_input":"2024-06-06T15:41:40.606292Z","iopub.status.idle":"2024-06-06T15:41:40.619298Z","shell.execute_reply.started":"2024-06-06T15:41:40.606266Z","shell.execute_reply":"2024-06-06T15:41:40.618313Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class ResizeConv2d(nn.Module):\n    def __init__(self, in_channels, out_channels, kernel_size, stride, scale_factor=2):\n        super().__init__()\n        self.scale_factor = scale_factor\n        self.conv = nn.Conv2d(in_channels, out_channels, kernel_size, stride, padding=kernel_size//2)\n\n    def forward(self, x):\n        out = nn.functional.interpolate(x, scale_factor=self.scale_factor, mode='bilinear')\n        out = self.conv(out)\n        return out","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.620731Z","iopub.execute_input":"2024-06-06T15:41:40.621142Z","iopub.status.idle":"2024-06-06T15:41:40.632962Z","shell.execute_reply.started":"2024-06-06T15:41:40.621116Z","shell.execute_reply":"2024-06-06T15:41:40.631928Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class DownSample(nn.Module):\n    def __init__(self, in_channels, out_channels, kernel_size, stride, isDropout=False, isNorm=True, isActivation=True):\n        super().__init__()\n        self._isNorm = isNorm\n        self._isActiviation = isActivation\n        self._isDropout = isDropout\n        self.padding = nn.ReflectionPad2d(kernel_size//2)\n        self.conv = nn.Conv2d(in_channels, out_channels, kernel_size, stride)\n        self.norm = nn.InstanceNorm2d(out_channels)\n        self.relu = nn.ReLU(inplace=True)\n\n    def forward(self, x):\n        out = self.padding(x)\n        out = self.conv(out)\n        if self._isNorm:\n            out = self.norm(out)\n        if self._isActiviation:\n            out = self.relu(out)\n        if self._isDropout:\n            out = F.dropout(out, 0.5)\n        return out","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.634287Z","iopub.execute_input":"2024-06-06T15:41:40.634607Z","iopub.status.idle":"2024-06-06T15:41:40.647237Z","shell.execute_reply.started":"2024-06-06T15:41:40.634580Z","shell.execute_reply":"2024-06-06T15:41:40.646258Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class UpSample(nn.Module):\n    def __init__(self, in_channels, out_channels, kernel_size, stride, usingConv=\"Interpolate\", isDropout=False, isNorm=True, isActivation=True):\n        super().__init__()\n        self._dropout = isDropout\n        self._isNorm = isNorm\n        self._isActiviation = isActivation\n        if usingConv == \"Interpolate\":\n            self.conv = ResizeConv2d(in_channels, out_channels, kernel_size, stride)\n        else:\n            self.conv = nn.ConvTranspose2d(in_channels, out_channels, kernel_size, stride, padding=kernel_size//2, output_padding=1)\n        self.norm = nn.InstanceNorm2d(out_channels)\n        self.relu = nn.ReLU(inplace=True)\n\n    def forward(self, x):\n        out = self.conv(x)\n        if self._isNorm:\n            out = self.norm(out)\n        if self._dropout:\n            out = F.dropout(out, 0.5)\n        if self._isActiviation:\n            out = self.relu(out)\n        return out","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.648340Z","iopub.execute_input":"2024-06-06T15:41:40.648612Z","iopub.status.idle":"2024-06-06T15:41:40.658974Z","shell.execute_reply.started":"2024-06-06T15:41:40.648587Z","shell.execute_reply":"2024-06-06T15:41:40.657898Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### U-Net","metadata":{}},{"cell_type":"code","source":"class DoubleConv(nn.Module):\n    def __init__(self, in_channel, out_channel, isDropout=False):\n        super().__init__()\n        self._isDropout = isDropout\n        self.conv1 = nn.Conv2d(in_channel, out_channel, kernel_size=3, padding=1)\n        self.conv2 = nn.Conv2d(out_channel, out_channel, kernel_size=3, padding=1)\n        self.relu = nn.ReLU(inplace=True)\n        self.norm = nn.InstanceNorm2d(out_channel)\n    \n    def forward(self, x):\n        x = self.conv1(x)\n        x = self.norm(x)\n        x = self.relu(x)\n        \n        x = self.conv2(x)\n        x = self.norm(x)\n        x = self.relu(x)\n        \n        return x","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.660130Z","iopub.execute_input":"2024-06-06T15:41:40.660438Z","iopub.status.idle":"2024-06-06T15:41:40.672306Z","shell.execute_reply.started":"2024-06-06T15:41:40.660410Z","shell.execute_reply":"2024-06-06T15:41:40.671162Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class DownBlock(nn.Module):\n    def __init__(self, in_channel, out_channel, isDropout=False):\n        super().__init__()\n        self._isDropout = isDropout\n        self.conv = DoubleConv(in_channel, out_channel)\n        self.pooling = nn.MaxPool2d(kernel_size=2, stride=2)\n    \n    def forward(self, x):\n        skip_out = self.conv(x)\n        down_out = self.pooling(skip_out)\n        \n        if self._isDropout:\n            down_out = F.dropout(down_out, 0.5)\n            skip_out = F.dropout(skip_out, 0.5)\n            \n        return (down_out, skip_out)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.673754Z","iopub.execute_input":"2024-06-06T15:41:40.674133Z","iopub.status.idle":"2024-06-06T15:41:40.683812Z","shell.execute_reply.started":"2024-06-06T15:41:40.674105Z","shell.execute_reply":"2024-06-06T15:41:40.682846Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class UpBlock(nn.Module):\n    def __init__(self, in_channel, out_channel, isDropout=False):\n        super().__init__()\n        self._isDropout = isDropout\n        self.conv1 = nn.ConvTranspose2d(in_channel-out_channel, in_channel-out_channel, kernel_size=2, stride=2)\n        self.conv2 = DoubleConv(in_channel, out_channel)\n    \n    def forward(self, down_input, skip_input):\n        x = self.conv1(down_input)\n        x = torch.cat([x, skip_input], dim=1)\n        \n        x = self.conv2(x)\n\n        if self._isDropout:\n            x = F.dropout(x, 0.5)\n            \n        return x","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.685186Z","iopub.execute_input":"2024-06-06T15:41:40.686015Z","iopub.status.idle":"2024-06-06T15:41:40.697847Z","shell.execute_reply.started":"2024-06-06T15:41:40.685978Z","shell.execute_reply":"2024-06-06T15:41:40.696845Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Build up generator","metadata":{}},{"cell_type":"code","source":"class Generator_ResNet(nn.Module):\n    def __init__(self, residual_blocks=9):\n        super().__init__()\n        \n        self.network = []\n        #c7s1-64\n        self.network.append(DownSample(in_channels=3, out_channels=64, kernel_size=7, stride=1))\n        #d128\n        self.network.append(DownSample(in_channels=64, out_channels=128, kernel_size=3, stride=2))\n        #d256\n        self.network.append(DownSample(in_channels=128, out_channels=256, kernel_size=3, stride=2))\n        #residual blocks\n        for _ in range(residual_blocks):\n            self.network.append(ResidualBlock(256, 256))\n        #u128\n        self.network.append(UpSample(in_channels=256, out_channels=128, kernel_size=3, stride=1, usingConv=\"Interpolate\", isDropout=True))\n        #u64\n        self.network.append(UpSample(in_channels=128, out_channels=64, kernel_size=3, stride=1, usingConv=\"Interpolate\", isDropout=True))\n        #c7s1-3\n        self.network.append(DownSample(in_channels=64, out_channels=3, kernel_size=7, stride=1, isActivation=False))\n        self.network.append(nn.Tanh())\n\n        self.model = nn.Sequential(*self.network)\n        self.apply(weight_init)\n        \n    def forward(self, x):\n        return self.model(x)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.699164Z","iopub.execute_input":"2024-06-06T15:41:40.699493Z","iopub.status.idle":"2024-06-06T15:41:40.711823Z","shell.execute_reply.started":"2024-06-06T15:41:40.699444Z","shell.execute_reply":"2024-06-06T15:41:40.710775Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class Generator_UNet(nn.Module):\n    def __init__(self, _in_channel=3, _out_channel=3):\n        super().__init__()\n        \n        self.network = []\n        \n        #Down\n        self.downBlock1 = DownBlock(in_channel=_in_channel, out_channel=64)\n        self.downBlock2 = DownBlock(in_channel=64, out_channel=128)\n        self.downBlock3 = DownBlock(in_channel=128, out_channel=256)\n        self.downBlock4 = DownBlock(in_channel=256, out_channel=512)\n        \n        #BottleNeck\n        self.bottleneck = DoubleConv(512, 1024)\n        \n        #Up\n        self.upBlock1 = UpBlock(in_channel=1024+512, out_channel=512, isDropout=True)\n        self.upBlock2 = UpBlock(in_channel=512+256, out_channel=256, isDropout=True)\n        self.upBlock3 = UpBlock(in_channel=256+128, out_channel=128, isDropout=True)\n        self.upBlock4 = UpBlock(in_channel=128+64, out_channel=64)\n                \n        #Final\n        self.finalConv = nn.Conv2d(64, _out_channel, kernel_size=1)\n                \n    def forward(self, x):\n        x, skip_out_1 = self.downBlock1(x)\n        x, skip_out_2 = self.downBlock2(x)\n        x, skip_out_3 = self.downBlock3(x)\n        x, skip_out_4 = self.downBlock4(x)\n        \n        x = self.bottleneck(x)\n        \n        x = self.upBlock1(x, skip_out_4)\n        x = self.upBlock2(x, skip_out_3)\n        x = self.upBlock3(x, skip_out_2)\n        x = self.upBlock4(x, skip_out_1)\n        \n        out = self.finalConv(x)\n        return out","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.717667Z","iopub.execute_input":"2024-06-06T15:41:40.718019Z","iopub.status.idle":"2024-06-06T15:41:40.730439Z","shell.execute_reply.started":"2024-06-06T15:41:40.717990Z","shell.execute_reply":"2024-06-06T15:41:40.729436Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Test","metadata":{}},{"cell_type":"code","source":"gen = Generator_UNet()","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:40.731641Z","iopub.execute_input":"2024-06-06T15:41:40.731950Z","iopub.status.idle":"2024-06-06T15:41:41.128889Z","shell.execute_reply.started":"2024-06-06T15:41:40.731922Z","shell.execute_reply":"2024-06-06T15:41:41.127997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"gen = Generator_ResNet()","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:41.130084Z","iopub.execute_input":"2024-06-06T15:41:41.130377Z","iopub.status.idle":"2024-06-06T15:41:41.386204Z","shell.execute_reply.started":"2024-06-06T15:41:41.130351Z","shell.execute_reply":"2024-06-06T15:41:41.384829Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(photo.shape)\ntest = gen(sample.to_tensor(photo).unsqueeze(0))\nprint(test.shape)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:41.387580Z","iopub.execute_input":"2024-06-06T15:41:41.387919Z","iopub.status.idle":"2024-06-06T15:41:42.364667Z","shell.execute_reply.started":"2024-06-06T15:41:41.387860Z","shell.execute_reply":"2024-06-06T15:41:42.363538Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(10, 5))\nfig, ax = plt.subplots(1, 2)\nax[0].imshow(photo)\nax[1].imshow(test.detach().numpy().squeeze().transpose(1, 2, 0))\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:42.366043Z","iopub.execute_input":"2024-06-06T15:41:42.366372Z","iopub.status.idle":"2024-06-06T15:41:42.961980Z","shell.execute_reply.started":"2024-06-06T15:41:42.366344Z","shell.execute_reply":"2024-06-06T15:41:42.960939Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def unnormalize(img, mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]):\n    for t, m, s in zip(img, mean, std):\n        t.mul_(s).add_(s)\n        \n    return img","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:42.963418Z","iopub.execute_input":"2024-06-06T15:41:42.963781Z","iopub.status.idle":"2024-06-06T15:41:42.970650Z","shell.execute_reply.started":"2024-06-06T15:41:42.963751Z","shell.execute_reply":"2024-06-06T15:41:42.969598Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Discriminator","metadata":{}},{"cell_type":"markdown","source":"### Component","metadata":{}},{"cell_type":"code","source":"class DownSample_Dis(nn.Module):\n    def __init__(self, in_channels, out_channels, kernel_size, stride, isNorm=True):\n        super().__init__()\n        self._isNorm = isNorm\n        \n        self.conv = nn.Conv2d(in_channels, out_channels, kernel_size, stride, padding=kernel_size//2)\n        self.norm = nn.InstanceNorm2d(out_channels)\n        self.relu = nn.LeakyReLU(negative_slope=0.2)\n        self.apply(weight_init)\n        \n    def forward(self, x):\n        out = self.conv(x)\n        if self._isNorm:\n            out = self.norm(out)\n        out = self.relu(out)\n        return out","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:42.971905Z","iopub.execute_input":"2024-06-06T15:41:42.972238Z","iopub.status.idle":"2024-06-06T15:41:42.981381Z","shell.execute_reply.started":"2024-06-06T15:41:42.972211Z","shell.execute_reply":"2024-06-06T15:41:42.980435Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class Discriminator(nn.Module):\n    def __init__(self):\n        super().__init__()\n\n        self.network = []\n        self.network.append(DownSample_Dis(in_channels=3, out_channels=64, kernel_size=4, stride=2, isNorm=False))\n        self.network.append(DownSample_Dis(in_channels=64, out_channels=128, kernel_size=4, stride=2))\n        self.network.append(DownSample_Dis(in_channels=128, out_channels=256, kernel_size=4, stride=2))\n        self.network.append(DownSample_Dis(in_channels=256, out_channels=512, kernel_size=4, stride=2))\n        self.network.append(nn.Conv2d(in_channels=512, out_channels=1, kernel_size=4, stride=1))\n        \n        self.model = nn.Sequential(*self.network)\n        self.apply(weight_init)\n\n    def forward(self, x):\n        x = self.model(x)\n        x = nn.Dropout(0.5)(x)\n        return F.avg_pool2d(x, x.size()[2:]).view(x.size()[0], -1)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:42.982690Z","iopub.execute_input":"2024-06-06T15:41:42.983077Z","iopub.status.idle":"2024-06-06T15:41:42.995858Z","shell.execute_reply.started":"2024-06-06T15:41:42.983048Z","shell.execute_reply":"2024-06-06T15:41:42.994954Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class MultiScaleDiscriminator(nn.Module):\n    def __init__(self, num_scales=3):\n        super(MultiScaleDiscriminator, self).__init__()\n        self.discriminators = nn.ModuleList([Discriminator() for _ in range(num_scales)])\n        self.downsample = nn.AvgPool2d(3, stride=2, padding=[1, 1], count_include_pad=False)\n\n    def forward(self, x):\n        results = []\n        for D in self.discriminators:\n            results.append(D(x))\n            x = self.downsample(x)\n        return results","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Test","metadata":{}},{"cell_type":"code","source":"dis = Discriminator()","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:42.997186Z","iopub.execute_input":"2024-06-06T15:41:42.997571Z","iopub.status.idle":"2024-06-06T15:41:43.090235Z","shell.execute_reply.started":"2024-06-06T15:41:42.997533Z","shell.execute_reply":"2024-06-06T15:41:43.089107Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"t = dis(test)\nprint(t)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:43.091584Z","iopub.execute_input":"2024-06-06T15:41:43.091940Z","iopub.status.idle":"2024-06-06T15:41:43.155450Z","shell.execute_reply.started":"2024-06-06T15:41:43.091901Z","shell.execute_reply":"2024-06-06T15:41:43.154377Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training","metadata":{}},{"cell_type":"markdown","source":"### Hyperparameter","metadata":{}},{"cell_type":"code","source":"device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\nlambda_cycle = 10\nlambda_identity = 0.5 * lambda_cycle\nlambda_perceptual = 0.5 * lambda_cycle\nnum_epochs = 20\ndecay_epochs = 20\nbuffer_size = 50\nlearning_rate = 2e-4\n# Model\ngen_type = \"UNet\"\nusing_data_aug = True\nusing_identity_loss = False\nusing_perceptual_loss = True\nusing_multiscale_discriminator = True","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:43.157154Z","iopub.execute_input":"2024-06-06T15:41:43.157451Z","iopub.status.idle":"2024-06-06T15:41:43.183395Z","shell.execute_reply.started":"2024-06-06T15:41:43.157426Z","shell.execute_reply":"2024-06-06T15:41:43.182177Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if gen_type == \"ResNet\":\n    gen_AB = Generator_ResNet()\n    gen_BA = Generator_ResNet()\nelse:\n    gen_AB = Generator_UNet()\n    gen_AB.apply(weight_init)\n    gen_BA = Generator_UNet()\n    gen_BA.apply(weight_init)\n\nif using_multiscale_discriminator:\n    dis_A = MultiScaleDiscriminator()\n    dis_B = MultiScaleDiscriminator()\nelse:\n    dis_A = Discriminator()\n    dis_B = Discriminator()\n\ngen_AB.to(device)\ngen_BA.to(device)\ndis_A.to(device)\ndis_B.to(device)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:43.184757Z","iopub.execute_input":"2024-06-06T15:41:43.185178Z","iopub.status.idle":"2024-06-06T15:41:44.576874Z","shell.execute_reply.started":"2024-06-06T15:41:43.185148Z","shell.execute_reply":"2024-06-06T15:41:44.575443Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport torchvision.models as models\n\n# 加載預訓練的VGG19網絡，只需要用到features部分\nvgg = models.vgg19(pretrained=True).features.cuda().eval()\n\n# 凍結VGG19網絡的參數，不需要進行梯度更新\nfor param in vgg.parameters():\n    param.requires_grad = False\n\n# 定義感知損失函數\ncriterion_perceptual = nn.MSELoss()\n\ndef perceptual_loss(fake_img, real_img):\n    fake_features = vgg(fake_img)\n    real_features = vgg(real_img)\n    return criterion_perceptual(fake_features, real_features)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.577849Z","iopub.status.idle":"2024-06-06T15:41:44.578255Z","shell.execute_reply.started":"2024-06-06T15:41:44.578067Z","shell.execute_reply":"2024-06-06T15:41:44.578085Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# loss function\ncriterion_GAN = nn.MSELoss()\ncriterion_cycle = nn.L1Loss()\ncriterion_identity = nn.L1Loss()","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.579516Z","iopub.status.idle":"2024-06-06T15:41:44.579884Z","shell.execute_reply.started":"2024-06-06T15:41:44.579697Z","shell.execute_reply":"2024-06-06T15:41:44.579713Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from torchvision import transforms\n\ntransform = transforms.Compose([\n    transforms.ToPILImage(),\n    transforms.ToTensor(),\n    transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])\n])\n\ntransform_Augmentation = transforms.Compose([\n    transforms.ToPILImage(),\n    transforms.Resize((286, 286), Image.BICUBIC),\n    transforms.RandomCrop((256, 256)),\n    transforms.RandomHorizontalFlip(),\n    transforms.ToTensor(),\n    transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])\n])","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.581362Z","iopub.status.idle":"2024-06-06T15:41:44.581712Z","shell.execute_reply.started":"2024-06-06T15:41:44.581542Z","shell.execute_reply":"2024-06-06T15:41:44.581558Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Dataloader\nfrom torch.utils.data import DataLoader\n\nif using_data_aug:\n    monetDataset = MonetDataset('/kaggle/input/gan-getting-started', transform_Augmentation)\nelse:\n    monetDataset = MonetDataset('/kaggle/input/gan-getting-started', transform)\n\n#train_set, vaild_set = torch.utils.data.random_split(monetDataset, [len(monetDataset) - 50, 50])\n#train_dataloader = DataLoader(dataset=train_set, batch_size=8, shuffle=True)\n#vaild_dataloader = DataLoader(dataset=vaild_set, batch_size=1, shuffle=True)\ndataloader = DataLoader(dataset=monetDataset, batch_size=1, shuffle=True, pin_memory=True)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.583122Z","iopub.status.idle":"2024-06-06T15:41:44.583490Z","shell.execute_reply.started":"2024-06-06T15:41:44.583310Z","shell.execute_reply":"2024-06-06T15:41:44.583326Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from torch.optim import lr_scheduler\nimport itertools\n\n# Optimizer & LR scheduler\noptimizer_G = torch.optim.Adam(itertools.chain(gen_AB.parameters(), gen_BA.parameters()), lr=learning_rate, betas=(0.5, 0.999))\noptimizer_D = torch.optim.Adam(itertools.chain(dis_A.parameters(), dis_B.parameters()), lr=learning_rate, betas=(0.5, 0.999))\n\nLambdaLR = lambda epoch: 1 - max(0, epoch - decay_epochs + 1) / (num_epochs - decay_epochs + 1)\nscheduler_G = lr_scheduler.LambdaLR(optimizer_G, lr_lambda=LambdaLR)\nscheduler_D = lr_scheduler.LambdaLR(optimizer_D, lr_lambda=LambdaLR)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.584407Z","iopub.status.idle":"2024-06-06T15:41:44.584777Z","shell.execute_reply.started":"2024-06-06T15:41:44.584594Z","shell.execute_reply":"2024-06-06T15:41:44.584610Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class ImageBuffer():\n    def __init__(self):\n        self.buffer = []\n        self.max_size = 50\n    \n    def push_and_pop(self, datas):\n        return_imgs = []\n        for element in datas.data:\n            element = torch.unsqueeze(element, 0)\n            if len(self.buffer) < self.max_size:\n                self.buffer.append(element)\n                return_imgs.append(element)\n            else:\n                if random.uniform(0, 1) > 0.5:\n                    i = random.randint(0, self.max_size - 1)\n                    return_imgs.append(self.buffer[i].clone())\n                    self.buffer[i] = element\n                else:\n                    return_imgs.append(element)\n                    \n        return torch.cat(return_imgs, 0)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.586616Z","iopub.status.idle":"2024-06-06T15:41:44.586999Z","shell.execute_reply.started":"2024-06-06T15:41:44.586782Z","shell.execute_reply":"2024-06-06T15:41:44.586797Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_image(amount):\n    _, ax = plt.subplots(amount, 2, figsize=(12, 12))\n\n    if amount == 1:\n        ax = [ax]\n    \n    for i in range(amount):\n        photo, _ = monetDataset[i]\n        photo = photo.unsqueeze(0)\n\n        with torch.no_grad():\n            monet = gen_AB(photo.to(device)).cpu().detach()\n\n        photo = unnormalize(photo)\n        monet = unnormalize(monet)\n\n        ax[i][0].imshow(photo.squeeze().permute(1, 2, 0))\n        ax[i][1].imshow(monet.squeeze().permute(1, 2, 0))\n        ax[i][0].axis(\"off\")\n        ax[i][1].axis(\"off\")\n      \n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.588252Z","iopub.status.idle":"2024-06-06T15:41:44.588633Z","shell.execute_reply.started":"2024-06-06T15:41:44.588442Z","shell.execute_reply":"2024-06-06T15:41:44.588459Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from collections import defaultdict\nfrom tqdm.auto import tqdm\nlogger = list()\n\nfor epoch in range(1, num_epochs+1):\n    loss_batch = defaultdict(float)\n    x_fake_A_buffer = ImageBuffer()\n    x_fake_B_buffer = ImageBuffer()\n    \n    total_g_loss = 0.0\n    total_d_loss = 0.0\n    \n    for i, data in enumerate(tqdm(dataloader)):\n        # Data\n        x_A = data[0].to(device) #photo\n        x_B = data[1].to(device) #monet\n\n        # Generators\n        gen_AB.zero_grad()\n        gen_BA.zero_grad()\n\n        # Generate\n        x_fake_A = gen_BA(x_B)\n        x_fake_B = gen_AB(x_A)\n\n        # Adversarial\n        x_adv_A = dis_A(x_fake_A)\n        x_adv_B = dis_B(x_fake_B)\n\n        # Cycle\n        x_cycle_A = gen_BA(x_fake_B)\n        x_cycle_B = gen_AB(x_fake_A)\n\n\n        g_loss_adv_A = 0\n        g_loss_adv_B = 0\n        if using_multiscale_discriminator:\n            for adv_A, adv_B in zip(x_adv_A, x_adv_B):\n                g_loss_adv_A += criterion_GAN(adv_A, torch.ones_like(adv_A))\n                g_loss_adv_B += criterion_GAN(adv_B, torch.ones_like(adv_B))\n\n            g_loss_adv_A /= len(x_adv_A)\n            g_loss_adv_B /= len(x_adv_B)\n        else:\n            g_loss_adv_A = criterion_GAN(x_adv_A, torch.ones_like(x_adv_A))\n            g_loss_adv_B = criterion_GAN(x_adv_B, torch.ones_like(x_adv_B))\n\n\n        # Cycle loss\n        g_loss_cycle_A = criterion_cycle(x_cycle_A, x_A) * lambda_cycle \n        g_loss_cycle_B = criterion_cycle(x_cycle_B, x_B) * lambda_cycle\n\n        # Identity loss\n        g_loss_identity_A = criterion_identity(x_cycle_A, x_A) * lambda_identity if using_identity_loss else 0\n        g_loss_identity_B = criterion_identity(x_cycle_B, x_B) * lambda_identity if using_identity_loss else 0\n\n        # Perceptual loss\n        g_loss_perceptual_A = criterion_perceptual(x_cycle_A, x_A) * lambda_perceptual if using_perceptual_loss else 0\n        g_loss_perceptual_B = criterion_perceptual(x_cycle_B, x_B) * lambda_perceptual if using_perceptual_loss else 0\n\n        g_loss_A = g_loss_adv_A + g_loss_cycle_A + g_loss_identity_A + g_loss_perceptual_A\n        g_loss_B = g_loss_adv_B + g_loss_cycle_B + g_loss_identity_B + g_loss_perceptual_B\n        g_loss = (g_loss_A + g_loss_B) / 2 \n              \n        g_loss.backward()\n        total_g_loss += g_loss.item()\n\n        optimizer_G.step()\n\n        # Discriminators\n        dis_A.zero_grad()\n        dis_B.zero_grad()\n\n        # photo\n        x_fake_A = x_fake_A_buffer.push_and_pop(x_fake_A)\n        d_real_A = dis_A(x_A)\n        d_fake_A = dis_A(x_fake_A.detach())\n        \n        # monet\n        x_fake_B = x_fake_B_buffer.push_and_pop(x_fake_B)\n        d_real_B = dis_B(x_B)\n        d_fake_B = dis_B(x_fake_B.detach())\n        \n        d_loss_A = 0 \n        d_loss_B = 0\n        # Discriminator Loss\n        if using_multiscale_discriminator:\n            for real_A, fake_A, real_B, fake_B in zip(d_real_A, d_fake_A, d_real_B, d_fake_B):\n                d_loss_A += criterion_GAN(real_A, torch.ones_like(real_A)) + criterion_GAN(fake_A, torch.zeros_like(fake_A))\n                d_loss_B += criterion_GAN(real_B, torch.ones_like(real_B)) + criterion_GAN(fake_B, torch.zeros_like(fake_B))\n                \n            d_loss_A /= len(d_real_A)\n            d_loss_A /= len(d_real_B)\n        else:\n            d_loss_A = criterion_GAN(d_real_A, torch.ones_like(d_real_A)) + criterion_GAN(d_fake_A, torch.zeros_like(d_fake_A))\n            d_loss_B = criterion_GAN(d_real_B, torch.ones_like(d_real_B)) + criterion_GAN(d_fake_B, torch.zeros_like(d_fake_B))\n\n        d_loss = (d_loss_A + d_loss_B) / 2\n\n        d_loss.backward()\n        total_d_loss += d_loss.item()\n        optimizer_D.step()\n    \n    avg_g_loss = total_g_loss / len(dataloader)\n    avg_d_loss = total_d_loss / len(dataloader)\n    \n    logger.append({\n        'g_loss': avg_g_loss,\n        'd_loss': avg_d_loss\n    })\n\n    scheduler_G.step()\n    scheduler_D.step()\n    \n    print(f\"Epoch: {epoch} | Generator Loss: {avg_g_loss} | Discriminator Loss: {avg_d_loss}\")\n    plot_image(3)\n    #torch.save(gen_AB.state_dict(), f'gen_AB_{epoch}.pth')\n        ","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.590895Z","iopub.status.idle":"2024-06-06T15:41:44.591325Z","shell.execute_reply.started":"2024-06-06T15:41:44.591139Z","shell.execute_reply":"2024-06-06T15:41:44.591156Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Training Detail","metadata":{}},{"cell_type":"code","source":"g_loss = []\nd_loss = []\n\nfor log in logger:\n    g_loss.append(log['g_loss'])\n    d_loss.append(log['d_loss'])","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.593456Z","iopub.status.idle":"2024-06-06T15:41:44.593841Z","shell.execute_reply.started":"2024-06-06T15:41:44.593635Z","shell.execute_reply":"2024-06-06T15:41:44.593651Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.xlabel(\"Epochs\")\nplt.ylabel(\"Losses\")\nplt.plot(g_loss, 'r', label='Generator Loss')\nplt.plot(d_loss, 'b', label='Descriminator Loss')\nplt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.596597Z","iopub.status.idle":"2024-06-06T15:41:44.597246Z","shell.execute_reply.started":"2024-06-06T15:41:44.596960Z","shell.execute_reply":"2024-06-06T15:41:44.596984Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"gen_AB.eval()\nplot_image(3)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.599143Z","iopub.status.idle":"2024-06-06T15:41:44.599538Z","shell.execute_reply.started":"2024-06-06T15:41:44.599351Z","shell.execute_reply":"2024-06-06T15:41:44.599369Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Output","metadata":{}},{"cell_type":"code","source":"class PhotoDataset(Dataset):\n    def __init__(self, root, transforms=None):\n        self.root_path = root\n        self.photo_paths = os.listdir(os.path.join(root, 'photo_jpg'))\n        self.transform = transforms\n\n    def __len__(self):\n        return len(self.photo_paths)\n\n    def __getitem__(self, idx):\n        photoPath = os.path.join(self.root_path, 'photo_jpg', self.photo_paths[idx])\n\n        if self.transform:\n            photo = self.transform(self.read_img(photoPath))\n\n        else:\n            photo = self.read_img(photoPath)\n\n        return photo\n\n    def to_tensor(self, img):\n        return torch.tensor(img, dtype=torch.float32).permute(2, 0, 1)\n    \n    def read_img(self, path):\n        return cv2.cvtColor(cv2.imread(path), cv2.COLOR_BGR2RGB)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.600802Z","iopub.status.idle":"2024-06-06T15:41:44.601210Z","shell.execute_reply.started":"2024-06-06T15:41:44.601028Z","shell.execute_reply":"2024-06-06T15:41:44.601045Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fin_ds = PhotoDataset('../input/gan-getting-started', transforms=transform)\nfin_dl = DataLoader(fin_ds, batch_size=1)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.603065Z","iopub.status.idle":"2024-06-06T15:41:44.603437Z","shell.execute_reply.started":"2024-06-06T15:41:44.603253Z","shell.execute_reply":"2024-06-06T15:41:44.603270Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.makedirs('../images', exist_ok=True)","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.604641Z","iopub.status.idle":"2024-06-06T15:41:44.605051Z","shell.execute_reply.started":"2024-06-06T15:41:44.604826Z","shell.execute_reply":"2024-06-06T15:41:44.604843Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"t = transforms.ToPILImage()","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.606610Z","iopub.status.idle":"2024-06-06T15:41:44.606984Z","shell.execute_reply.started":"2024-06-06T15:41:44.606781Z","shell.execute_reply":"2024-06-06T15:41:44.606796Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"gen_AB.eval()\n\nfor i, photo in enumerate(tqdm(fin_dl)):\n    photo = photo.to(device) #photo\n    with torch.no_grad():\n        pred_monet = gen_AB(photo).cpu().detach()\n    pred_monet = unnormalize(pred_monet)\n    img = t(pred_monet.squeeze())\n    img.save(f'../images/{i+1}.jpg')","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.608528Z","iopub.status.idle":"2024-06-06T15:41:44.609029Z","shell.execute_reply.started":"2024-06-06T15:41:44.608696Z","shell.execute_reply":"2024-06-06T15:41:44.608711Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import shutil\nshutil.make_archive(\"/kaggle/working/images\", 'zip', \"/kaggle/images\")","metadata":{"execution":{"iopub.status.busy":"2024-06-06T15:41:44.610815Z","iopub.status.idle":"2024-06-06T15:41:44.611340Z","shell.execute_reply.started":"2024-06-06T15:41:44.611076Z","shell.execute_reply":"2024-06-06T15:41:44.611099Z"},"trusted":true},"execution_count":null,"outputs":[]}]}