{"nbformat_minor": 1, "cells": [{"source": ["# Pytorch Convolution Neural Network "], "cell_type": "markdown", "metadata": {"_uuid": "2430cad589e13a115f77d5963d35b4a07d1548e6", "_cell_guid": "75651683-fb32-43cd-8b34-8eb87e297ca8"}}, {"source": ["In this notebook I am going to build a convolution neural network using torch.nn module. We are going to train the CNN on MNIST data set . "], "cell_type": "markdown", "metadata": {"_uuid": "8d8dca65653c45ed75e1eb89aa6de7045709735d", "_cell_guid": "d9a6e318-a004-4363-9d1b-8fd7978bd143"}}, {"metadata": {"_uuid": "991df7918a2b37a1e9a0fd183f5e393daac82772", "_cell_guid": "b98bf7d9-7676-49df-b59f-05466d666c16"}, "source": ["# This Python 3 environment comes with many helpful analytics libraries installed\n", "# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n", "# For example, here's several helpful packages to load in \n", "\n", "import numpy as np # linear algebra\n", "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n", "\n", "# Input data files are available in the \"../input/\" directory.\n", "# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n", "\n", "from subprocess import check_output\n", "print(check_output([\"ls\", \"../input\"]).decode(\"utf8\"))\n", "\n", "# Any results you write to the current directory are saved as output."], "cell_type": "code", "outputs": [], "execution_count": 6}, {"source": ["Here first we have to import several torch libraries like torch.nn , torch.data.utils, torchvision etc."], "cell_type": "markdown", "metadata": {"_uuid": "2e34fb48f1d3bbe097d8a31f7f92de16bf9a7565", "_cell_guid": "d59dacd4-0500-4e74-9ac7-47c6f16b5fc3"}}, {"metadata": {"collapsed": true, "_uuid": "97cfca70477ffd7b4bb084384e10184e300c2a62", "_cell_guid": "64e682e6-826b-442d-aeb0-6f9a5b756e96"}, "source": ["df = pd.read_csv(\"../input/train.csv\")"], "cell_type": "code", "outputs": [], "execution_count": 7}, {"metadata": {"_uuid": "096b7ce8f555205e2f8336d980dab1389234afb9", "_cell_guid": "bf88f044-e2d7-411d-907e-232e4d889ab5"}, "source": ["import torch\n", "import torch.nn as nn\n", "import torch.utils.data as data\n", "import torchvision.datasets as dsets\n", "import torchvision.transforms as transforms\n", "from torch.autograd import Variable"], "cell_type": "code", "outputs": [], "execution_count": 10}, {"source": ["A very important part in building any deep NN in pytorch is first building a class conating the data . This data.datatset class wil contain  init() , getitem() , len() . After defining the class we create an object of that class . Then call the dataloader on the data with    batch_size = 50 .  "], "cell_type": "markdown", "metadata": {"_uuid": "d9d57561bf4633b780cc3ca5e63143b2c25eaa4a", "_cell_guid": "41d9f20a-6178-4009-a4ce-76b30255c211"}}, {"metadata": {"collapsed": true, "_uuid": "397c8f7f9065d95163c89ef9ea256737a2a6fb4a", "_cell_guid": "384fdfca-4bc2-474f-8c76-37a183352065"}, "source": ["class Mnist(data.Dataset):\n", "    def __init__(self):\n", "        train_X = df[0:20000]\n", "        train_Y = train_X.label.as_matrix().tolist()\n", "        train_X = train_X.drop(\"label\",axis=1).as_matrix().reshape(20000,1,28,28)\n", "        self.datalist = train_X\n", "        self.labellist = train_Y\n", "\n", "\n", "    def __getitem__(self, index):\n", "        return torch.Tensor(self.datalist[index].astype(float)), self.labellist[index]\n", "\n", "    def __len__(self):\n", "        return self.datalist.shape[0]\n", "\n", "train_data = Mnist()\n", "train_loader = torch.utils.data.DataLoader(dataset=train_data,\n", "                                           batch_size=50, \n", "                                           shuffle=True,\n", "                                           num_workers=2)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"source": ["Now we create the nn.module class CNN . It must contain two methods init ( ) and forward ( ) . In init ( ) we define the sequential steps of the CNN model using Conv2d , ReLU , MaxPool2d etc. In forward method we do the forward propagation part . "], "cell_type": "markdown", "metadata": {"_uuid": "4773ad2c658e56de2d57d406d63eb01830e494c8", "_cell_guid": "437fe13c-45c7-4d21-b7bf-10400f499b46"}}, {"metadata": {"collapsed": true, "_uuid": "e33cb98156db0af64faa493efb80ed86935c092e", "_cell_guid": "96371adb-fef6-4a6a-9b69-9e124b109b90"}, "source": ["class CNN(nn.Module):\n", "    def __init__(self):\n", "        super(CNN,self).__init__()\n", "        self.layer1 = nn.Sequential(\n", "            nn.Conv2d(1, 20, kernel_size=5),\n", "            nn.BatchNorm2d(20),\n", "            nn.ReLU(),\n", "            nn.MaxPool2d(2))\n", "        self.fc = nn.Linear(12*12*20, 10)\n", "\n", "\n", "    def forward(self, x):\n", "        out = self.layer1(x)\n", "        out = out.view(out.size(0),-1)\n", "        out = self.fc(out)\n", "        return out\n", "    \n", "cnn = CNN()"], "cell_type": "code", "outputs": [], "execution_count": null}, {"source": ["After craeting an instance cnn of class CNN now we define the loss function and the optimizer technique , which is SGD here with learning rate = .004"], "cell_type": "markdown", "metadata": {"_uuid": "512c275630a5680527ce15e65538b28f483664a1", "_cell_guid": "c808abba-a060-4c85-8df5-e0ffe9f46913"}}, {"metadata": {"collapsed": true, "_uuid": "f004a344b54575c0072f7e92b2cfd5646edd8268", "_cell_guid": "6471fe1b-26b0-43d5-bf81-b47f0936355f"}, "source": ["loss_fn = nn.CrossEntropyLoss()\n", "optimizer = torch.optim.SGD(cnn.parameters(), lr=0.001)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"source": ["Here we start the actual CNN implemetation through iterating over allthe examples . Here we are only running 5 iterations and in each epoch we first call cnn on each image then we do the backward propagation and update the parameters . "], "cell_type": "markdown", "metadata": {"_uuid": "8d5f06dc73538e1c055538583b47e993e9fa6bff", "_cell_guid": "ee08fb9c-5c5b-45e9-b2a5-ad25ce64382c"}}, {"metadata": {"collapsed": true, "_uuid": "80ebd54a4c6f68c5017a4c2cedfbe7e72af9f0cf", "_cell_guid": "3f7a70d7-e137-4dd0-bdef-e19296e772e0"}, "source": ["for epoch in range(5):\n", "    for i, (images, labels) in enumerate(train_loader):\n", "        images = Variable(images)\n", "        labels = Variable(labels)\n", "\n", "        optimizer.zero_grad()\n", "        outputs = cnn(images)\n", "\n", "        loss = loss_fn(outputs, labels)\n", "        loss.backward()\n", "        optimizer.step()\n", "\n", "        if (i+1) % 100 == 0:\n", "            print ('Epoch [%d/%d], Iter [%d/%d] Loss: %.4f' %(epoch+1, 5, i+1, len(train_data)//50,  loss.data[0]))\n"], "cell_type": "code", "outputs": [], "execution_count": null}, {"source": ["So ,finally we see that in just 3 epochs the loss value has come down to 0.45 from 2.04 . "], "cell_type": "markdown", "metadata": {"_uuid": "d3610ef4a859283f28d77fc6bf4a1ed1401cbd37", "_cell_guid": "7f6321da-605f-4b2a-8b00-a0a830ae92c1"}}, {"source": ["Now we will be create a test data and apply our cnn on that."], "cell_type": "markdown", "metadata": {"_uuid": "8c573e44c5b64c38f553a709d9bd3fdd3ce852c3", "_cell_guid": "edc0c907-1cc9-47ee-ae4c-81d51f5cb2db"}}, {"metadata": {"collapsed": true, "_uuid": "15408be9ab8517fb3494adbe9b0b312d84472ef0", "_cell_guid": "8d9bb530-b847-4fe4-8e52-de82c4845e6d"}, "source": ["testX = df[20000:22000]\n", "testY = testX['label'].values\n", "testX = testX.drop('label',axis=1).as_matrix().reshape(2000,1,28,28).astype(float)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"collapsed": true, "_uuid": "b1eeee158ec598f8b690f474f60718e7c0123713", "_cell_guid": "6bbd3740-6e6d-45b9-a416-a17e81da2699"}, "source": ["testX = Variable(torch.Tensor(testX))\n", "pred = cnn(testX)\n", "_, predlabel = torch.max(pred.data, 1)"], "cell_type": "code", "outputs": [], "execution_count": null}, {"source": ["We run the cnn on the test data and find accuarcy of  90.7%"], "cell_type": "markdown", "metadata": {"_uuid": "adc3b1729c4d89a6d7c3455f1f5ecf0ef9f4807e", "_cell_guid": "199bd963-f702-43fe-93d6-7878b3a23da3"}}, {"metadata": {"collapsed": true, "_uuid": "ee96e566e75a0f9c3f8d06e0d47560d8a440a7ab", "_cell_guid": "98c221ab-2e76-47cd-a986-11ea4e4c91cf"}, "source": ["np.sum(predlabel.numpy()==testY)/2000"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"_uuid": "ef09837c7fb19aa6ed67c202b8ab6c88f3bafa2a", "_cell_guid": "dbbbda14-9361-4eba-8a4f-1e5a829991f7"}, "source": ["from sklearn.metrics import classification_report\n", "print(classification_report(testY,predlabel))"], "cell_type": "code", "outputs": [], "execution_count": null}, {"metadata": {"collapsed": true}, "source": [], "cell_type": "code", "outputs": [], "execution_count": null}], "nbformat": 4, "metadata": {"language_info": {"codemirror_mode": {"version": 3, "name": "ipython"}, "mimetype": "text/x-python", "version": "3.6.3", "pygments_lexer": "ipython3", "file_extension": ".py", "name": "python", "nbconvert_exporter": "python"}, "kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"}}}