{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"
In this notebook, I have tried my best to explain the GANs with the help of illustrations and in-depth descriptions along with its implementation from scratch on the Stanford Dogs dataset. Feedback would be greatly appreciated :)
\n\n
","metadata":{}},{"cell_type":"markdown","source":"# Generative Adversarial Networks (GANs)\n\nGANs are a type of neural network that we use to generate new images. They consist of two parts:\n\n- **Generator**: We use this to create new images from random noise.\n- **Discriminator**: We use this to evaluate the generated images against real-world examples.\n\nBoth parts start from scratch and learn together during training.
\n
\n

\n\n## How GANs Work\n\nWe input a random noise signal into the **Generator** (G) and it outputs an image. The **Discriminator** (A) learns to distinguish between different objects or features. Once trained, it can identify whether an input image matches the features it has learned.\n\n## GANs Process\n\nIn the first step, we input a random noise signal into the generator to create images. These images are used to train the discriminator, which outputs probabilities. We calculate errors and backpropagate them through the discriminator to update the weights.\n\nNext, we train the generator using the images it created. The generator learns by tricking the discriminator into outputting false positives. We calculate errors and backpropagate them through the generator to update the weights.\n\nIn the second step, we further train the generator and discriminator. The generator improves by learning from its mistakes through backpropagation.\n\n\n
\n

","metadata":{}},{"cell_type":"markdown","source":"

\n# Importing the libraries","metadata":{}},{"cell_type":"code","source":"from __future__ import print_function\nimport time\nimport os\nimport torch\nimport torch.nn as nn\nimport torch.nn.parallel\nimport torch.optim as optim\nimport torch.utils.data\nimport torchvision.datasets as dset\nimport torchvision.transforms as transforms\nimport torchvision.utils as vutils\nfrom torch.autograd import Variable\nimport matplotlib.pyplot as plt\nfrom skimage.transform import resize\n\nimport numpy as np\nfrom torch import nn, optim\nimport torch.nn.functional as F\nfrom torchvision import datasets, transforms\nfrom torchvision.utils import save_image\nimport matplotlib.pyplot as plt\nimport matplotlib.image as mpimg\nfrom tqdm import tqdm_notebook as tqdm","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-04T19:48:17.980935Z","iopub.execute_input":"2023-11-04T19:48:17.981324Z","iopub.status.idle":"2023-11-04T19:48:17.992588Z","shell.execute_reply.started":"2023-11-04T19:48:17.981280Z","shell.execute_reply":"2023-11-04T19:48:17.991437Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### About the Dataset\nWe will be using Stanford Dogs dataset which contains images of 120 breeds of dogs from all over the world","metadata":{}},{"cell_type":"code","source":"import random\n# Define the path to the directory containing dog images\nPATH = '/kaggle/input/stanford-dogs-dataset/images/Images/'\n\n# Get the list of image filenames in the directory\nimages = os.listdir(PATH)\n\n# Display the count of dog pictures in the directory\nprint(f'Total dog pictures available: {len(os.listdir(PATH))}')\n\n# Create subplots to display random images\nfig, axes = plt.subplots(nrows=5, ncols=5, figsize=(8,8))\n\n# Iterate through the subplots and display random dog images\nfor indx, axis in enumerate(axes.flatten()):\n PATH_TEMP = random.choice(os.listdir(PATH))\n # Get a random index within the range of available images\n rnd_indx = np.random.randint(0, len(os.listdir(PATH+'/'+PATH_TEMP)))\n \n # Load and display the image using Matplotlib\n images = os.listdir(PATH+'/'+PATH_TEMP+'/')\n img = plt.imread(PATH+'/'+PATH_TEMP + '/'+images[rnd_indx])\n # Resize the image to a constant size (e.g., 64x64)\n \n img = resize(img, (400,400))\n imgplot = axis.imshow(img)\n axis.set_axis_off() # Turn off the axis for better visualization\n\n","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-04T19:52:32.638351Z","iopub.execute_input":"2023-11-04T19:52:32.638661Z","iopub.status.idle":"2023-11-04T19:52:35.579725Z","shell.execute_reply.started":"2023-11-04T19:52:32.638637Z","shell.execute_reply":"2023-11-04T19:52:35.578756Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Image Preprocessing\nWe apply transformations to our images, resize them to 64x64 pixels, and apply random color jittering, rotation, and horizontal flipping. We convert these transformed images into tensors and normalize them. We load these images in batches of 32 from a folder and shuffle them for randomness. This approach enhances the diversity and size of our dataset, improving our model’s robustness.","metadata":{}},{"cell_type":"code","source":"batch_size = 32\nimage_size = 64\n\nrandom_transforms = [transforms.ColorJitter(), transforms.RandomRotation(degrees=20)]\ntransform = transforms.Compose([transforms.Resize(64),\n transforms.CenterCrop(64),\n transforms.RandomHorizontalFlip(p=0.5),\n transforms.RandomApply(random_transforms, p=0.2),\n transforms.ToTensor(),\n transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n\ntrain_data = datasets.ImageFolder(PATH, transform=transform)\ntrain_loader = torch.utils.data.DataLoader(train_data, shuffle=True,\n batch_size=batch_size)\n \nimgs, label = next(iter(train_loader))\nimgs = imgs.numpy().transpose(0, 2, 3, 1)","metadata":{"execution":{"iopub.status.busy":"2023-11-04T19:52:24.720679Z","iopub.execute_input":"2023-11-04T19:52:24.721038Z","iopub.status.idle":"2023-11-04T19:52:32.636750Z","shell.execute_reply.started":"2023-11-04T19:52:24.721013Z","shell.execute_reply":"2023-11-04T19:52:32.635864Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, axs = plt.subplots(5,5, figsize = (8,8))\n\nfor i in range(5):\n for j in range(5):\n axs[i,j].imshow(imgs[5*i+j])\n axs[i,j].axis('off')\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-11-04T19:52:41.630488Z","iopub.execute_input":"2023-11-04T19:52:41.630895Z","iopub.status.idle":"2023-11-04T19:52:42.509217Z","shell.execute_reply.started":"2023-11-04T19:52:41.630856Z","shell.execute_reply":"2023-11-04T19:52:42.508264Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Weights\n### Defining the weights_init function","metadata":{}},{"cell_type":"code","source":"def weights_init(m):\n \"\"\"\n Takes as input a neural network m that will initialize all its weights.\n \"\"\"\n classname = m.__class__.__name__\n if classname.find('Conv') != -1:\n m.weight.data.normal_(0.0, 0.02)\n elif classname.find('BatchNorm') != -1:\n m.weight.data.normal_(1.0, 0.02)\n m.bias.data.fill_(0)","metadata":{"execution":{"iopub.status.busy":"2023-11-04T19:58:05.021652Z","iopub.execute_input":"2023-11-04T19:58:05.022048Z","iopub.status.idle":"2023-11-04T19:58:05.027924Z","shell.execute_reply.started":"2023-11-04T19:58:05.022021Z","shell.execute_reply":"2023-11-04T19:58:05.027055Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Generator","metadata":{}},{"cell_type":"code","source":"class G(nn.Module):\n def __init__(self):\n # Used to inherit the torch.nn Module\n super(G, self).__init__()\n # Meta Module - consists of different layers of Modules\n self.main = nn.Sequential(\n nn.ConvTranspose2d(100, 512, 4, stride=1, padding=0, bias=False),\n nn.BatchNorm2d(512),\n nn.ReLU(True),\n nn.ConvTranspose2d(512, 256, 4, stride=2, padding=1, bias=False),\n nn.BatchNorm2d(256),\n nn.ReLU(True),\n nn.ConvTranspose2d(256, 128, 4, stride=2, padding=1, bias=False),\n nn.BatchNorm2d(128),\n nn.ReLU(True),\n nn.ConvTranspose2d(128, 64, 4, stride=2, padding=1, bias=False),\n nn.BatchNorm2d(64),\n nn.ReLU(True),\n nn.ConvTranspose2d(64, 3, 4, stride=2, padding=1, bias=False),\n nn.Tanh()\n )\n \n def forward(self, input):\n output = self.main(input)\n return output\n\n# Creating the generator\nnetG = G()\nnetG.apply(weights_init)","metadata":{"execution":{"iopub.status.busy":"2023-11-04T19:58:06.231265Z","iopub.execute_input":"2023-11-04T19:58:06.231982Z","iopub.status.idle":"2023-11-04T19:58:06.306123Z","shell.execute_reply.started":"2023-11-04T19:58:06.231948Z","shell.execute_reply":"2023-11-04T19:58:06.305184Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Discriminator","metadata":{}},{"cell_type":"code","source":"# Defining the discriminator\nclass D(nn.Module):\n def __init__(self):\n super(D, self).__init__()\n self.main = nn.Sequential(\n nn.Conv2d(3, 64, 4, stride=2, padding=1, bias=False),\n nn.LeakyReLU(negative_slope=0.2, inplace=True),\n nn.Conv2d(64, 128, 4, stride=2, padding=1, bias=False),\n nn.BatchNorm2d(128),\n nn.LeakyReLU(negative_slope=0.2, inplace=True),\n nn.Conv2d(128, 256, 4, stride=2, padding=1, bias=False),\n nn.BatchNorm2d(256),\n nn.LeakyReLU(negative_slope=0.2, inplace=True),\n nn.Conv2d(256, 512, 4, stride=2, padding=1, bias=False),\n nn.BatchNorm2d(512),\n nn.LeakyReLU(negative_slope=0.2, inplace=True),\n nn.Conv2d(512, 1, 4, stride=1, padding=0, bias=False),\n nn.Sigmoid()\n )\n \n def forward(self, input):\n output = self.main(input)\n # .view(-1) = Flattens the output into 1D instead of 2D\n return output.view(-1)\n \n \n# Creating the discriminator\nnetD = D()\nnetD.apply(weights_init)\n","metadata":{"execution":{"iopub.status.busy":"2023-11-04T19:58:06.841453Z","iopub.execute_input":"2023-11-04T19:58:06.841808Z","iopub.status.idle":"2023-11-04T19:58:06.899551Z","shell.execute_reply.started":"2023-11-04T19:58:06.841781Z","shell.execute_reply":"2023-11-04T19:58:06.898596Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Another setup","metadata":{}},{"cell_type":"code","source":"class Generator(nn.Module):\n def __init__(self, nz=128, channels=3):\n super(Generator, self).__init__()\n \n self.nz = nz\n self.channels = channels\n \n def convlayer(n_input, n_output, k_size=4, stride=2, padding=0):\n block = [\n nn.ConvTranspose2d(n_input, n_output, kernel_size=k_size, stride=stride, padding=padding, bias=False),\n nn.BatchNorm2d(n_output),\n nn.ReLU(inplace=True),\n ]\n return block\n\n self.model = nn.Sequential(\n *convlayer(self.nz, 1024, 4, 1, 0), # Fully connected layer via convolution.\n *convlayer(1024, 512, 4, 2, 1),\n *convlayer(512, 256, 4, 2, 1),\n *convlayer(256, 128, 4, 2, 1),\n *convlayer(128, 64, 4, 2, 1),\n nn.ConvTranspose2d(64, self.channels, 3, 1, 1),\n nn.Tanh()\n )\n\n def forward(self, z):\n z = z.view(-1, self.nz, 1, 1)\n img = self.model(z)\n return img\n\n \nclass Discriminator(nn.Module):\n def __init__(self, channels=3):\n super(Discriminator, self).__init__()\n \n self.channels = channels\n\n def convlayer(n_input, n_output, k_size=4, stride=2, padding=0, bn=False):\n block = [nn.Conv2d(n_input, n_output, kernel_size=k_size, stride=stride, padding=padding, bias=False)]\n if bn:\n block.append(nn.BatchNorm2d(n_output))\n block.append(nn.LeakyReLU(0.2, inplace=True))\n return block\n\n self.model = nn.Sequential(\n *convlayer(self.channels, 32, 4, 2, 1),\n *convlayer(32, 64, 4, 2, 1),\n *convlayer(64, 128, 4, 2, 1, bn=True),\n *convlayer(128, 256, 4, 2, 1, bn=True),\n nn.Conv2d(256, 1, 4, 1, 0, bias=False), # FC with Conv.\n )\n\n def forward(self, imgs):\n logits = self.model(imgs)\n out = torch.sigmoid(logits)\n \n return out.view(-1, 1)","metadata":{"execution":{"iopub.status.busy":"2023-11-04T19:58:07.733882Z","iopub.execute_input":"2023-11-04T19:58:07.734246Z","iopub.status.idle":"2023-11-04T19:58:07.750190Z","shell.execute_reply.started":"2023-11-04T19:58:07.734218Z","shell.execute_reply":"2023-11-04T19:58:07.749198Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training","metadata":{}},{"cell_type":"code","source":"!mkdir results\n!ls","metadata":{"execution":{"iopub.status.busy":"2023-11-04T19:58:09.040980Z","iopub.execute_input":"2023-11-04T19:58:09.041976Z","iopub.status.idle":"2023-11-04T19:58:10.980785Z","shell.execute_reply.started":"2023-11-04T19:58:09.041932Z","shell.execute_reply":"2023-11-04T19:58:10.979745Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Parameters","metadata":{}},{"cell_type":"code","source":"batch_size = 32\nLR_G = 0.001\nLR_D = 0.0005\n\nbeta1 = 0.5\nepochs = 100\n\nreal_label = 0.9\nfake_label = 0\nnz = 128\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Initialize models and optimizers","metadata":{}},{"cell_type":"code","source":"netG = Generator(nz).to(device)\nnetD = Discriminator().to(device)\n\ncriterion = nn.BCELoss()\n\noptimizerD = optim.Adam(netD.parameters(), lr=LR_D, betas=(beta1, 0.999))\noptimizerG = optim.Adam(netG.parameters(), lr=LR_G, betas=(beta1, 0.999))\n\nfixed_noise = torch.randn(25, nz, 1, 1, device=device)\n\nG_losses = []\nD_losses = []\nepoch_time = []","metadata":{"execution":{"iopub.status.busy":"2023-11-04T19:59:48.371108Z","iopub.execute_input":"2023-11-04T19:59:48.371844Z","iopub.status.idle":"2023-11-04T19:59:52.757511Z","shell.execute_reply.started":"2023-11-04T19:59:48.371813Z","shell.execute_reply":"2023-11-04T19:59:52.756464Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Plot Loss per EPOCH**\n> plot_loss()","metadata":{}},{"cell_type":"code","source":"def plot_loss (G_losses, D_losses, epoch):\n plt.figure(figsize=(10,5))\n plt.title(\"Generator and Discriminator Loss - EPOCH \"+ str(epoch))\n plt.plot(G_losses,label=\"G\")\n plt.plot(D_losses,label=\"D\")\n plt.xlabel(\"iterations\")\n plt.ylabel(\"Loss\")\n plt.legend()\n plt.show()","metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"execution":{"iopub.status.busy":"2023-11-04T19:59:52.759382Z","iopub.execute_input":"2023-11-04T19:59:52.759677Z","iopub.status.idle":"2023-11-04T19:59:52.765233Z","shell.execute_reply.started":"2023-11-04T19:59:52.759653Z","shell.execute_reply":"2023-11-04T19:59:52.764267Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Show generated images**\n> show_generated_img()","metadata":{}},{"cell_type":"code","source":"def show_generated_img(n_images=5):\n sample = []\n for _ in range(n_images):\n noise = torch.randn(1, nz, 1, 1, device=device)\n gen_image = netG(noise).to(\"cpu\").clone().detach().squeeze(0)\n gen_image = gen_image.numpy().transpose(1, 2, 0)\n sample.append(gen_image)\n \n figure, axes = plt.subplots(1, len(sample), figsize = (64,64))\n for index, axis in enumerate(axes):\n axis.axis('off')\n image_array = sample[index]\n axis.imshow(image_array)\n \n plt.show()\n plt.close()","metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"execution":{"iopub.status.busy":"2023-11-04T19:59:52.766420Z","iopub.execute_input":"2023-11-04T19:59:52.766800Z","iopub.status.idle":"2023-11-04T19:59:52.779162Z","shell.execute_reply.started":"2023-11-04T19:59:52.766762Z","shell.execute_reply":"2023-11-04T19:59:52.778337Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Training Loop","metadata":{}},{"cell_type":"code","source":"for epoch in range(epochs):\n \n start = time.time()\n for ii, (real_images, train_labels) in tqdm(enumerate(train_loader), total=len(train_loader)):\n ############################\n # (1) Update D network: maximize log(D(x)) + log(1 - D(G(z)))\n ###########################\n # train with real\n netD.zero_grad()\n real_images = real_images.to(device)\n batch_size = real_images.size(0)\n labels = torch.full((batch_size, 1), real_label, device=device)\n\n output = netD(real_images)\n errD_real = criterion(output, labels)\n errD_real.backward()\n D_x = output.mean().item()\n\n # train with fake\n noise = torch.randn(batch_size, nz, 1, 1, device=device)\n fake = netG(noise)\n labels.fill_(fake_label)\n output = netD(fake.detach())\n errD_fake = criterion(output, labels)\n errD_fake.backward()\n D_G_z1 = output.mean().item()\n errD = errD_real + errD_fake\n optimizerD.step()\n\n ############################\n # (2) Update G network: maximize log(D(G(z)))\n ###########################\n netG.zero_grad()\n labels.fill_(real_label) # fake labels are real for generator cost\n output = netD(fake)\n errG = criterion(output, labels)\n errG.backward()\n D_G_z2 = output.mean().item()\n optimizerG.step()\n \n # Save Losses for plotting later\n G_losses.append(errG.item())\n D_losses.append(errD.item())\n \n if (ii+1) % (len(train_loader)//2) == 0:\n print('[%d/%d][%d/%d] Loss_D: %.4f Loss_G: %.4f D(x): %.4f D(G(z)): %.4f / %.4f'\n % (epoch + 1, epochs, ii+1, len(train_loader),\n errD.item(), errG.item(), D_x, D_G_z1, D_G_z2))\n \n plot_loss (G_losses, D_losses, epoch)\n G_losses = []\n D_losses = []\n if epoch % 10 == 0:\n show_generated_img()\n\n epoch_time.append(time.time()- start)\n \n# valid_image = netG(fixed_noise)","metadata":{"_kg_hide-output":false,"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-11-04T19:59:52.992835Z","iopub.execute_input":"2023-11-04T19:59:52.993567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print (\">> average EPOCH duration = \", np.mean(epoch_time))","metadata":{"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Generation example\n","metadata":{}},{"cell_type":"code","source":"show_generated_img(7)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if not os.path.exists('../output_images'):\n os.mkdir('../output_images')\n \nim_batch_size = 50\nn_images=10000\n\nfor i_batch in tqdm(range(0, n_images, im_batch_size)):\n gen_z = torch.randn(im_batch_size, nz, 1, 1, device=device)\n gen_images = netG(gen_z)\n images = gen_images.to(\"cpu\").clone().detach()\n images = images.numpy().transpose(0, 2, 3, 1)\n for i_image in range(gen_images.size(0)):\n save_image(gen_images[i_image, :, :, :], os.path.join('../output_images', f'image_{i_batch+i_image:05d}.png'))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize=(25, 16))\n# display 10 images from each class\nfor i, j in enumerate(images[:32]):\n ax = fig.add_subplot(4, 8, i + 1, xticks=[], yticks=[])\n plt.imshow(j)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]}]}