![](\"https://miro.medium.com/max/1070/1*Zx-ZMLKab7VOCQTxdZ1OAw.gif\")
\n\n","metadata":{"trusted":true}},{"cell_type":"markdown","source":"Let us implement a convolution operation on a 1 channel image with a 3x3 kernel.","metadata":{"trusted":true}},{"cell_type":"code","source":"def apply_kernel(image, kernel):\n ri, ci = image.shape # image dimensions\n rk, ck = kernel.shape # kernel dimensions\n ro, co = ri-rk+1, ci-ck+1 # output dimensions, No padding and no striding\n output = torch.zeros([ro, co])\n for i in range(ro): \n for j in range(co):\n output[i,j] = torch.sum(image[i:i+rk,j:j+ck] * kernel) \n return output","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_image = torch.tensor([\n [4, 4, 3, 1, 0], \n [0, 0, 1, 4, 1], \n [4, 1, 3, 3, 4], \n [2, 0, 0, 3, 3], \n [3, 0, 0, 0, 1]\n], dtype=torch.float32)\n\nsample_kernel = torch.tensor([\n [1, 1, 1], \n [1, 1, 1], \n [1, 1, 1]\n], dtype=torch.float32) #blur filter\n\napply_kernel(sample_image, sample_kernel)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The `Conv2d` layer transforms a 3-channel image to a 16-channel *feature map*, and the `MaxPool2d` layer halves the height and width. The feature map gets smaller as we add more layers, until we are finally left with a small feature map, which can be flattened into a vector. We can then add some fully connected layers at the end to get vector of size 10 for each image.\n\n![](\"https://i.imgur.com/KKtPOKE.png\")
\n\nLet's define the model by extending an `ImageClassificationBase` class which contains helper methods for training & validation.","metadata":{"trusted":true}},{"cell_type":"code","source":"def accuracy(outputs,labels):\n _,preds= torch.max(outputs, dim=1)\n return torch.tensor(torch.sum(preds==labels).item()/len(preds))\n\nclass ImageClassificationBase(nn.Module):\n def training_step(self,batch):\n images,labels= batch\n out= self(images) #Generate predictions\n loss= F.cross_entropy(out,labels) #calculate loss\n return loss\n \n def validation_step(self,batch):\n images, labels = batch\n out = self(images) # Generate predictions\n loss= F.cross_entropy(out,labels) # calculate loss\n acc= accuracy(out,labels)\n return {'val_loss':loss.detach(), 'val_acc':acc}\n \n def validation_epoch_end(self, outputs):\n batch_losses = [x['val_loss'] for x in outputs]\n epoch_loss= torch.stack(batch_losses).mean() # Stacking losses to combine losses and calculate average\n batch_accs = [x['val_acc'] for x in outputs]\n epoch_acc = torch.stack(batch_accs).mean() # Combine accuracies\n return {'val_loss': epoch_loss.item(), 'val_acc': epoch_acc.item()}\n \n def epoch_end(self,epoch,result):\n print(\"Epoch [{}], train_loss: {:.4f}, val_loss: {:.4f}, val_acc: {:.4f}\".format(\n epoch, result['train_loss'], result['val_loss'], result['val_acc']))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"`detach()` detaches the output from the computationnal graph. So no gradient will be backproped along this variable.\n\n`torch.no_grad` says that no operation should build the graph.\n\n\nThe difference is that one refers to only a given variable on which it’s called. The other affects all operations taking place within the with statement.tensor.detach() creates a tensor that shares storage with tensor that does not require grad. It detaches the output from the computational graph. So no gradient will be backpropagated along this variable.\n\nThe wrapper with torch.no_grad() temporarily set all the requires_grad flag to false. torch.no_grad says that no operation should build the graph.\n\nThe difference is that one refers to only a given variable on which it is called. The other affects all operations taking place within the with statement. Also, torch.no_grad will use less memory because it knows from the beginning that no gradients are needed so it doesn’t need to keep intermediary results.","metadata":{"trusted":true}},{"cell_type":"markdown","source":"We'll use `nn.Sequential` to chain the layers and activations functions into a single network architecture.","metadata":{"trusted":true}},{"cell_type":"code","source":"class Cifar10CnnModel(ImageClassificationBase):\n def __init__(self):\n super().__init__()\n self.network = nn.Sequential(\n nn.Conv2d(3, 32, kernel_size=3, padding=1),\n nn.ReLU(),\n nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1),\n nn.ReLU(),\n nn.MaxPool2d(2, 2), # output: 64 x 16 x 16\n\n nn.Conv2d(64, 128, kernel_size=3, stride=1, padding=1),\n nn.ReLU(),\n nn.Conv2d(128, 128, kernel_size=3, stride=1, padding=1),\n nn.ReLU(),\n nn.MaxPool2d(2, 2), # output: 128 x 8 x 8\n\n nn.Conv2d(128, 256, kernel_size=3, stride=1, padding=1),\n nn.ReLU(),\n nn.Conv2d(256, 256, kernel_size=3, stride=1, padding=1),\n nn.ReLU(),\n nn.MaxPool2d(2, 2), # output: 256 x 4 x 4\n\n nn.Flatten(), \n nn.Linear(256*4*4, 1024),\n nn.ReLU(),\n nn.Linear(1024, 512),\n nn.ReLU(),\n nn.Linear(512, 10))\n \n def forward(self, xb):\n return self.network(xb)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model= Cifar10CnnModel()\nmodel","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for images, labels in train_dl:\n print('images.shape:', images.shape)\n out = model(images)\n print('out.shape:',out.shape)\n print('out[0]',out[0])\n break","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"out.shape","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_default_device():\n \"\"\"Pick GPU if available, else CPU\"\"\"\n if torch.cuda.is_available():\n return torch.device('cuda')\n else:\n return torch.device('cpu')\n \ndef to_device(data, device):\n \"\"\"Move tensor(s) to chosen device\"\"\"\n if isinstance(data, (list,tuple)):\n return [to_device(x, device) for x in data]\n return data.to(device, non_blocking=True)\n\nclass DeviceDataLoader():\n \"\"\"Wrap a dataloader to move data to a device\"\"\"\n def __init__(self, dl, device):\n self.dl = dl\n self.device = device\n \n def __iter__(self):\n \"\"\"Yield a batch of data after moving it to device\"\"\"\n for b in self.dl: \n yield to_device(b, self.device)\n\n def __len__(self):\n \"\"\"Number of batches\"\"\"\n return len(self.dl)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Based on where you run this notebook, your default device could be a CPU (`torch.device('cpu')`) or a GPU (`torch.device('cuda')`)","metadata":{"trusted":true}},{"cell_type":"code","source":"device = get_default_device()\ndevice","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can now wrap our training and validation data loaders using `DeviceDataLoader` for automatically transferring batches of data to the GPU (if available), and use `to_device` to move our model to the GPU (if available).","metadata":{}},{"cell_type":"code","source":"train_dl = DeviceDataLoader(train_dl, device)\nval_dl = DeviceDataLoader(val_dl, device) # load data to device (GPU if available)\nto_device(model, device) # move model to GPU if available","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# Training the Model","metadata":{}},{"cell_type":"markdown","source":"m","metadata":{}},{"cell_type":"code","source":"#In this mode, the result of every computation will have requires_grad=False, even when the inputs have requires_grad=True.\n@torch.no_grad() \ndef evaluate(model, val_loader):\n model.eval() # Setting model to evaluation mode, the model can adjust its behavior regarding some operations, like Dropout.\n outputs = [model.validation_step(batch) for batch in val_loader]\n return model.validation_epoch_end(outputs)\n \ndef fit(epochs, lr, model, train_loader, val_loader, opt_func= torch.optim.SGD):\n history=[]\n optimizer= opt_func(model.parameters(),lr) # model paramters w.r.t calculate derivative of loss\n for epoch in range(epochs):\n # Training phase\n model.train() # Setting model to training mode\n train_losses=[]\n for batch in train_loader:\n loss= model.training_step(batch)\n train_losses.append(loss)\n loss.backward() #compute gradients\n optimizer.step()\n optimizer.zero_grad() # zero the gradients\n #Validation phase\n result= evaluate(model,val_loader)\n result['train_loss'] = torch.stack(train_losses).mean().item()\n model.epoch_end(epoch, result)\n history.append(result)\n return history\n ","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The initial accuracy is around 10% in one epoch, which is expected model is not trained enough.","metadata":{}},{"cell_type":"code","source":"evaluate(model,val_dl)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"num_epochs = 15\nopt_func = torch.optim.Adam\nlr = 0.001","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"history = fit(num_epochs, lr, model, train_dl, val_dl, opt_func)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Plot valdation set accuracies to study how the model improves over time.","metadata":{}},{"cell_type":"code","source":"def plot_accuracies(history):\n accuracies=[x['val_acc'] for x in history]\n plt.plot(accuracies,'-x')\n plt.xlabel('Epoch')\n plt.ylabel('Accuracy')\n plt.title('Accuracy vs No of epochs')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(10,6))\nplot_accuracies(history)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Our model reaches an accuracy of around 75%, and by looking at the graph, it seems unlikely that the model will achieve an accuracy higher than 80% even after training for a long time. This suggests that we might need to use a more powerful model to capture the relationship between the images and the labels more accurately.","metadata":{}},{"cell_type":"code","source":"def plot_losses(history):\n train_losses = [x.get('train_loss') for x in history]\n val_losses = [x['val_loss'] for x in history]\n plt.plot(train_losses, '-bx')\n plt.plot(val_losses, '-rx')\n plt.xlabel('epoch')\n plt.ylabel('loss')\n plt.legend(['Training', 'Validation'])\n plt.title('Loss vs. No. of epochs');","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(10,6))\nplot_losses(history)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Initialy, both the training and validation losses seem to decrease over time. However, if we train the model for long enough,training loss continues to decrease, while the validation loss stops decreasing, and even starts to increase after a certain point(overfitting).","metadata":{}},{"cell_type":"markdown","source":"\n# Testing with individual images","metadata":{}},{"cell_type":"code","source":"test_dataset = ImageFolder(data_dir+'/test', transform=ToTensor())\ntest_dataset","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def predict_image(img, model):\n xb= to_device(img.unsqueeze(0),device) # unsqueeze turns an n-dimensionsal tensor into an n+1-dimensional one. But since it is ambiguous which axis the new dimension lies across, this needs to be specified.\n # Get predictions from model\n yb = model(xb)\n # Pick index/label with highest probability\n _, preds= torch.max(yb, dim=1)\n return dataset.classes[preds[0].item()]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img,label= test_dataset[8]\nplt.imshow(img.permute(1,2,0))\nprint('Label:', dataset.classes[label], ', Predicted:', predict_image(img, model))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img,label= test_dataset[1220]\nplt.imshow(img.permute(1,2,0))\nprint('Label:', dataset.classes[label], ', Predicted:', predict_image(img, model))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img,label= test_dataset[345]\nplt.imshow(img.permute(1,2,0))\nprint('Label:', dataset.classes[label], ', Predicted:', predict_image(img, model))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img,label= test_dataset[6153]\nplt.imshow(img.permute(1,2,0))\nprint('Label:', dataset.classes[label], ', Predicted:', predict_image(img, model))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img,label= test_dataset[456]\nplt.imshow(img.permute(1,2,0))\nprint('Label:', dataset.classes[label], ', Predicted:', predict_image(img, model))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"img,label= test_dataset[10]\nplt.imshow(img.permute(1,2,0))\nprint('Label:', dataset.classes[label], ', Predicted:', predict_image(img, model))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"`Identifying where our model performs poorly can help us improve the model, by collecting more training data, \nincreasing/decreasing the complexity of the model, and changing the hypeparameters.`","metadata":{}},{"cell_type":"code","source":"img,label= test_dataset[1432]\nplt.imshow(img.permute(1,2,0))\nprint('Label:', dataset.classes[label], ', Predicted:', predict_image(img, model))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_loader=DeviceDataLoader(DataLoader(test_dataset,batch_size),device)\ntest_result = evaluate(model, test_loader)\ntest_result","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"