{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## A Complete Walk Through to LeNet-5 Classification Tutorial Using Pytorch\nReference link: https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-10-10T10:28:50.332041Z","iopub.execute_input":"2021-10-10T10:28:50.332417Z","iopub.status.idle":"2021-10-10T10:28:50.359113Z","shell.execute_reply.started":"2021-10-10T10:28:50.332313Z","shell.execute_reply":"2021-10-10T10:28:50.358236Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Steps to Follow for Training an Image Classifier\n\n1- Loading and Normalize CIFAR-10 dataset from torchvision <br>\n2- Define a Convolutional Neural Network <br>\n3- Define loss criterion and optimizer<br>\n4- Train the network<br>\n5- Test the network<br>","metadata":{}},{"cell_type":"markdown","source":"### Step 01: Loading and Normalize CIFAR-10 Dataset","metadata":{}},{"cell_type":"code","source":"# importing torch, torchvision and transform\nimport torch\nimport torchvision\nimport torchvision.transforms as transforms","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:28:50.360507Z","iopub.execute_input":"2021-10-10T10:28:50.360964Z","iopub.status.idle":"2021-10-10T10:28:51.468165Z","shell.execute_reply.started":"2021-10-10T10:28:50.360931Z","shell.execute_reply":"2021-10-10T10:28:51.467269Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Torchvision loads the images into PILLOW Image or PILImage format which is the python default. We need to convert this into Tensor. We're using ToTensor for converting PILImage [0, 255] into float (C X H X W) with shape [0, 1] and then to normalize [-1, 1] by using the mean and standard deviation of 0.5.","metadata":{}},{"cell_type":"code","source":"transform = transforms.Compose([transforms.ToTensor(),\\\n                              transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:28:51.469481Z","iopub.execute_input":"2021-10-10T10:28:51.469710Z","iopub.status.idle":"2021-10-10T10:28:51.474156Z","shell.execute_reply.started":"2021-10-10T10:28:51.469685Z","shell.execute_reply":"2021-10-10T10:28:51.473154Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Transforms.Normalize will normalize the image into [-1, 1] using the formula *\"image = (image - mean) / std\"*. And the way it is written *\"transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))\"* due to CIFAR 10 dataset contains color images with Red, Green and Blue channels for each mean and std value for each channel.","metadata":{}},{"cell_type":"code","source":"batch_size = 4\n\ntrainset = torchvision.datasets.CIFAR10(root = './data',\\\n                                       train = True, download = True, transform = transform)\ntrainloader = torch.utils.data.DataLoader(trainset, batch_size = batch_size, shuffle = True, num_workers=2)\ntestset = torchvision.datasets.CIFAR10(root = './data',\\\n                                      train = False, download = True, transform = transform)\ntestloader = torch.utils.data.DataLoader(testset, batch_size = batch_size, shuffle = False, num_workers=2)\n\nclasses = ('plane', 'car', 'bird', 'cat','deer', 'dog', 'frog', 'horse', 'ship', 'truck')","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:28:51.475906Z","iopub.execute_input":"2021-10-10T10:28:51.476113Z","iopub.status.idle":"2021-10-10T10:28:57.544262Z","shell.execute_reply.started":"2021-10-10T10:28:51.476091Z","shell.execute_reply":"2021-10-10T10:28:57.542626Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Dataset and Dataloader\nDataset is responsible for reading and transform the datapoint in a dataset. We're using dataloader to shuffle the data and batching for the input to the network.<br>\nTransform -> Dataset <br>\nShuffle -> Dataloader<br>\nBatching -> Dataloader","metadata":{}},{"cell_type":"markdown","source":"### Displaying the image\nThe below code will diaply some random images in the dataset. First it need to be unnormalize the image then convert it to the numpy array and then finally called a plt.imshow() function to display the image","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport numpy as np\n\ndef imshow(img):\n    img = img / 2 + 0.5     # unnormalize\n    npimg = img.numpy()\n    plt.imshow(np.transpose(npimg, (1, 2, 0)))\n    plt.show()\n\n\n# get some random training images\ndataiter = iter(trainloader)\nimages, labels = dataiter.next()\n\n# show images\nimshow(torchvision.utils.make_grid(images))\n# print labels\nprint(' '.join('%5s' % classes[labels[j]] for j in range(batch_size)))","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:28:57.548147Z","iopub.execute_input":"2021-10-10T10:28:57.548422Z","iopub.status.idle":"2021-10-10T10:28:57.962384Z","shell.execute_reply.started":"2021-10-10T10:28:57.548391Z","shell.execute_reply":"2021-10-10T10:28:57.961364Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Step 02: Define a Convolutional Neural Netwok","metadata":{}},{"cell_type":"code","source":"import torch.nn as nn\nimport torch.nn.functional as F","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:28:57.964487Z","iopub.execute_input":"2021-10-10T10:28:57.965256Z","iopub.status.idle":"2021-10-10T10:28:57.970309Z","shell.execute_reply.started":"2021-10-10T10:28:57.965208Z","shell.execute_reply":"2021-10-10T10:28:57.969032Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class Net(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.conv1 = nn.Conv2d(3, 6, 5)\n        self.pool = nn.MaxPool2d(2, 2)\n        self.conv2 = nn.Conv2d(6, 16, 5)\n        self.fc1 = nn.Linear(16 * 5 * 5, 120)\n        self.fc2 = nn.Linear(120, 84)\n        self.fc3 = nn.Linear(84, 10)\n\n    def forward(self, x):\n        x = self.pool(F.relu(self.conv1(x)))\n        x = self.pool(F.relu(self.conv2(x)))\n        x = torch.flatten(x, 1) # flatten all dimensions except batch\n        x = F.relu(self.fc1(x))\n        x = F.relu(self.fc2(x))\n        x = self.fc3(x)\n        return x\n\n\nnet = Net()","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:28:57.971477Z","iopub.execute_input":"2021-10-10T10:28:57.971794Z","iopub.status.idle":"2021-10-10T10:28:57.989449Z","shell.execute_reply.started":"2021-10-10T10:28:57.971757Z","shell.execute_reply":"2021-10-10T10:28:57.988317Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Explaination:\nnn.Conv2d(in_channels, out_channels, kernel_size)<br>\nnn.MaxPool2d(H, W)<br>\nself.conv1 -> input channels=3, output channels=6, kernel size=5<br>\nself.conv2 -> input channels=6, output channels=16, kernel size=5<br>\nself.fc1 -> input=16x16x5, output=120<br>\nself.fc2 -> input=120, output=84<br>\nself.fc3 -> input=84, output=10<br>\n\nThe MaxPool layer doesn't have weights, therefore it can be utilized after each convolutional layer.\n\n","metadata":{}},{"cell_type":"markdown","source":"## Step 03: Define loss criterion and optimizer","metadata":{}},{"cell_type":"code","source":"import torch.optim as optim\n\ncriterion = nn.CrossEntropyLoss() #using CrossEntropyLoss() for classification problem\noptimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:28:57.991058Z","iopub.execute_input":"2021-10-10T10:28:57.991394Z","iopub.status.idle":"2021-10-10T10:28:57.998104Z","shell.execute_reply.started":"2021-10-10T10:28:57.991360Z","shell.execute_reply":"2021-10-10T10:28:57.997130Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Step 04: Train the network","metadata":{}},{"cell_type":"code","source":"for epoch in range(2):  # loop over the dataset multiple times\n\n    running_loss = 0.0\n    for i, data in enumerate(trainloader, 0):\n        # get the inputs; data is a list of [inputs, labels]\n        inputs, labels = data\n\n        # zero the parameter gradients\n        optimizer.zero_grad()\n\n        # forward + backward + optimize\n        outputs = net(inputs)\n        loss = criterion(outputs, labels)\n        loss.backward()\n        optimizer.step()\n\n        # print statistics\n        running_loss += loss.item()\n        if i % 2000 == 1999:    # print every 2000 mini-batches\n            print('[%d, %5d] loss: %.3f' %\n                  (epoch + 1, i + 1, running_loss / 2000))\n            running_loss = 0.0\n\nprint('Finished Training')","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:28:57.999633Z","iopub.execute_input":"2021-10-10T10:28:58.000036Z","iopub.status.idle":"2021-10-10T10:31:06.017299Z","shell.execute_reply.started":"2021-10-10T10:28:57.999982Z","shell.execute_reply":"2021-10-10T10:31:06.016303Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Saving the model\nPATH = './cifar_net.pth'\ntorch.save(net.state_dict(), PATH)","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:32:18.002785Z","iopub.execute_input":"2021-10-10T10:32:18.003096Z","iopub.status.idle":"2021-10-10T10:32:18.010928Z","shell.execute_reply.started":"2021-10-10T10:32:18.003060Z","shell.execute_reply":"2021-10-10T10:32:18.010289Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dataiter = iter(testloader)\nimages, labels = dataiter.next()\n\n# print images\nimshow(torchvision.utils.make_grid(images))\nprint('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:33:02.364565Z","iopub.execute_input":"2021-10-10T10:33:02.365313Z","iopub.status.idle":"2021-10-10T10:33:02.626142Z","shell.execute_reply.started":"2021-10-10T10:33:02.365275Z","shell.execute_reply":"2021-10-10T10:33:02.624779Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"net = Net()\nnet.load_state_dict(torch.load(PATH))","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:33:29.688415Z","iopub.execute_input":"2021-10-10T10:33:29.689183Z","iopub.status.idle":"2021-10-10T10:33:29.700627Z","shell.execute_reply.started":"2021-10-10T10:33:29.689126Z","shell.execute_reply":"2021-10-10T10:33:29.699993Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"outputs = net(images)","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:33:44.817760Z","iopub.execute_input":"2021-10-10T10:33:44.818570Z","iopub.status.idle":"2021-10-10T10:33:44.825103Z","shell.execute_reply.started":"2021-10-10T10:33:44.818527Z","shell.execute_reply":"2021-10-10T10:33:44.824494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"_, predicted = torch.max(outputs, 1)\n\nprint('Predicted: ', ' '.join('%5s' % classes[predicted[j]]\n                              for j in range(4)))","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:34:00.924838Z","iopub.execute_input":"2021-10-10T10:34:00.925651Z","iopub.status.idle":"2021-10-10T10:34:00.930874Z","shell.execute_reply.started":"2021-10-10T10:34:00.925615Z","shell.execute_reply":"2021-10-10T10:34:00.930319Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"correct = 0\ntotal = 0\n# since we're not training, we don't need to calculate the gradients for our outputs\nwith torch.no_grad():\n    for data in testloader:\n        images, labels = data\n        # calculate outputs by running images through the network\n        outputs = net(images)\n        # the class with the highest energy is what we choose as prediction\n        _, predicted = torch.max(outputs.data, 1)\n        total += labels.size(0)\n        correct += (predicted == labels).sum().item()\n\nprint('Accuracy of the network on the 10000 test images: %d %%' % (\n    100 * correct / total))","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:35:05.704370Z","iopub.execute_input":"2021-10-10T10:35:05.705063Z","iopub.status.idle":"2021-10-10T10:35:12.871809Z","shell.execute_reply.started":"2021-10-10T10:35:05.705027Z","shell.execute_reply":"2021-10-10T10:35:12.870792Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# prepare to count predictions for each class\ncorrect_pred = {classname: 0 for classname in classes}\ntotal_pred = {classname: 0 for classname in classes}\n\n# again no gradients needed\nwith torch.no_grad():\n    for data in testloader:\n        images, labels = data\n        outputs = net(images)\n        _, predictions = torch.max(outputs, 1)\n        # collect the correct predictions for each class\n        for label, prediction in zip(labels, predictions):\n            if label == prediction:\n                correct_pred[classes[label]] += 1\n            total_pred[classes[label]] += 1\n\n\n# print accuracy for each class\nfor classname, correct_count in correct_pred.items():\n    accuracy = 100 * float(correct_count) / total_pred[classname]\n    print(\"Accuracy for class {:5s} is: {:.1f} %\".format(classname,\n                                                   accuracy))","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:35:44.285328Z","iopub.execute_input":"2021-10-10T10:35:44.285660Z","iopub.status.idle":"2021-10-10T10:35:52.101279Z","shell.execute_reply.started":"2021-10-10T10:35:44.285624Z","shell.execute_reply":"2021-10-10T10:35:52.100604Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n\n# Assuming that we are on a CUDA machine, this should print a CUDA device:\n\nprint(device)","metadata":{"execution":{"iopub.status.busy":"2021-10-10T10:36:24.405593Z","iopub.execute_input":"2021-10-10T10:36:24.405892Z","iopub.status.idle":"2021-10-10T10:36:24.413105Z","shell.execute_reply.started":"2021-10-10T10:36:24.405863Z","shell.execute_reply":"2021-10-10T10:36:24.412021Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### The notebook is still in the process of improvement, if you learn something new, don't forget to upvote. Thanks ","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}}]}