{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.10","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":16017,"databundleVersionId":708189,"sourceType":"competition"}],"dockerImageVersionId":30580,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<center><b><h4>If this notebook was helpful to you in any way, then please do consider providing feedback-it would mean a lot! :)</b></center>","metadata":{}},{"cell_type":"markdown","source":"# Libraries and Modules\n\nThe following libraries and modules are imported at the beginning of the notebook:\n\n- `numpy`: a library for numerical computing in Python\n- `pandas`: a library for data manipulation and analysis\n- `os`: a module for interacting with the operating system\n\n- `matplotlib.pyplot`: a module for creating visualizations in Python\n- `PIL.Image`: a module for working with images in Python\n- `torch`: a library for machine learning in Python\n- `torch.nn`: a module for building neural networks in PyTorch\n- `torch.optim`: a module for optimizing neural networks in PyTorch\n- `torch.nn.functional`: a module for applying various functions to tensors in PyTorch\n- `torchvision`: a library for computer vision tasks in PyTorch\n- `torchvision.models`: a module containing pre-trained models for computer vision tasks in PyTorch\n- `torch.utils.data`: a module for working with datasets in PyTorch\n","metadata":{}},{"cell_type":"code","source":"%matplotlib inline\n\n# Importing necessary libraries and modules\nimport numpy as np # for numerical computing\nimport pandas as pd # for manipulation and analysis\nimport os # for interacting with the operating system \nimport matplotlib.pyplot as plt # for creating visualizations\nfrom PIL import Image # for working with images\n\nimport torch # for machine learning\nimport torch.nn as nn # for building neural networks in PyTorch\nimport torch.optim as optim # for optimizing neural networks in PyTorch\nimport torchvision # for computer vision tasks in PyTorch\nfrom torchvision import models, transforms # for loading and preprocessing image data\nimport torchvision.models as models # for using pre-trained models in PyTorch\nfrom torch.utils.data import DataLoader, Dataset # for working with datasets in PyTorch\n","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-06-14T17:51:47.175045Z","iopub.execute_input":"2023-06-14T17:51:47.175767Z","iopub.status.idle":"2023-06-14T17:51:47.183613Z","shell.execute_reply.started":"2023-06-14T17:51:47.175732Z","shell.execute_reply":"2023-06-14T17:51:47.182705Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<br><br>\n# Setting Things Up\n","metadata":{}},{"cell_type":"markdown","source":"We define the file paths we would be using as part of this notebook","metadata":{}},{"cell_type":"code","source":"# Defining the paths for the training and testing data\nbase_dir = '../input/'\ntrain_csv = '../input/Kannada-MNIST/train.csv'\ntest_csv = '../input/Kannada-MNIST/test.csv'\n\n\n# Set the device which we will be using for the model\ndevice = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")","metadata":{"execution":{"iopub.status.busy":"2023-06-14T18:15:03.474434Z","iopub.execute_input":"2023-06-14T18:15:03.475161Z","iopub.status.idle":"2023-06-14T18:15:03.480055Z","shell.execute_reply.started":"2023-06-14T18:15:03.475129Z","shell.execute_reply":"2023-06-14T18:15:03.479111Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\nFinally, we define custom dataset class for loading and preprocessing image data. It includes several functions for working with image files and applying transformations to the images.\n\n\nThe `manualDataset` class is defined with several parameters, including the file paths for the images, the split between training and testing data, and whether or not to use a validation split. The class includes functions for loading and preprocessing the image data, including applying transformations to the images and extracting the labels from the file names.\n\n\nThe `__init__()` function initializes the class with the specified parameters and defines the transformations to be applied to the images. The `__len__()` function returns the length of the dataset, and the `__getitem__()` function loads and preprocesses the image data and returns it along with the corresponding label.","metadata":{}},{"cell_type":"code","source":"\n\n\n# Defining a custom dataset class for loading and preprocessing image data\nclass manualDataset(Dataset):\n    def __init__(self, file_path, split='train', val_split=False):\n        df = pd.read_csv(file_path)\n       \n        self.split = split\n        self.val_split = val_split\n       \n        if self.split=='test' and val_split==False :\n            # test data\n            df = df.drop(columns=[\"id\"])\n            self.raw_files = df.values.reshape((-1,28,28)).astype(np.uint8)[:,:,:,None]\n            \n            self.labels = None\n        else:\n            # training data\n            self.raw_files = df.iloc[:,1:].values.reshape((-1,28,28)).astype(np.uint8)[:,:,:,None]\n            self.labels = torch.from_numpy(df.iloc[:,0].values)\n            print(self.raw_files.shape)\n        if(val_split):\n            self.raw_files = self.raw_files[:10]\n          \n       \n        # Defining the transformations to be applied to the images\n        self.train_transform = transforms.Compose([ transforms.ToPILImage(),transforms.RandomHorizontalFlip(), transforms.ToTensor()])\n        self.test_transform = transforms.Compose([ transforms.ToTensor()])\n          \n\n        \n    def __len__(self):\n        return len(self.raw_files)\n    \n    def __getitem__(self, idx):\n        # Reading in the image file\n        raw = self.raw_files[idx]\n      \n        \n        # Applying the appropriate transformation based on the split\n        if self.split == 'train' or self.val_split:\n            raw = self.train_transform(raw)\n            label = self.labels[idx]\n            return raw, label\n        elif self.split == 'test':\n            raw = self.test_transform(raw)\n            return raw\n        \n        \n        \n        \n","metadata":{"execution":{"iopub.status.busy":"2023-06-14T18:23:52.663239Z","iopub.execute_input":"2023-06-14T18:23:52.663858Z","iopub.status.idle":"2023-06-14T18:23:52.679610Z","shell.execute_reply.started":"2023-06-14T18:23:52.663821Z","shell.execute_reply":"2023-06-14T18:23:52.678786Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<br><br>\n# Datasets and Dataloaders Setup\n\n\nNow we create instances of the `manualDataset` class for the training, testing, and validation data. The `split` parameter is used to specify whether the data is for training or testing, and the `val_split` parameter is used to specify whether to use a validation split.\n\nWe then defines the batch sizes for the training and testing data, and creates instances of the `DataLoader` class for the training, testing, and validation data. The `batch_size` parameter is used to specify the number of samples per batch, and the `shuffle` parameter is used to specify whether to shuffle the training data.\n\n","metadata":{}},{"cell_type":"code","source":"# Creating instances of the manualDataset class for the training, testing, and validation datasets\ntrain_dataset = manualDataset(train_csv, split='train')\ntest_dataset = manualDataset(test_csv, split='test')\nval_dataset = manualDataset(train_csv, split='test', val_split=True)\n\n# Defining the batch sizes for the training and testing data\ntrain_batch_size = 32\ntest_batch_size = 1\n\n# Creating instances of the DataLoader class for the training, testing, and validation data\ntrain_loader = DataLoader(train_dataset, batch_size=train_batch_size, shuffle=True)\ntest_loader = DataLoader(test_dataset, batch_size=test_batch_size)\nval_loader = DataLoader(val_dataset, batch_size=test_batch_size)\n","metadata":{"execution":{"iopub.status.busy":"2023-06-14T18:23:53.105407Z","iopub.execute_input":"2023-06-14T18:23:53.105826Z","iopub.status.idle":"2023-06-14T18:24:00.526677Z","shell.execute_reply.started":"2023-06-14T18:23:53.105793Z","shell.execute_reply":"2023-06-14T18:24:00.525491Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<br><br>\n# Creating a Convolutional Network\n\n","metadata":{}},{"cell_type":"code","source":"class CNN(nn.Module):    \n    def __init__(self):\n        super().__init__()\n        \n        # CNN layers\n        self.cnn = nn.Sequential(\n            nn.Conv2d(1, 32, kernel_size=3, stride=1, padding=1),  # Convolutional layer with 32 output channels\n            nn.BatchNorm2d(32),  # Batch normalization to normalize the outputs\n            nn.ReLU(inplace=True),  # ReLU activation function for introducing non-linearity\n            nn.Conv2d(32, 32, kernel_size=3, stride=1, padding=1),  # Another convolutional layer with 32 output channels\n            nn.BatchNorm2d(32),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(kernel_size=2, stride=2),  # Max pooling layer to reduce the spatial dimensions\n            nn.Conv2d(32, 64, kernel_size=3, padding=1),  # Convolutional layer with 64 output channels\n            nn.BatchNorm2d(64),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(64, 64, kernel_size=3, padding=1),  # Another convolutional layer with 64 output channels\n            nn.BatchNorm2d(64),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(kernel_size=2, stride=2)  # Max pooling layer to reduce the spatial dimensions\n        )\n        \n        # Fully connected layers\n        self.classifier = nn.Sequential(\n            nn.Dropout(p=0.5),  # Dropout layer to prevent overfitting\n            nn.Linear(64 * 7 * 7, 512),  # Fully connected layer with 512 output units\n            nn.BatchNorm1d(512),\n            nn.ReLU(inplace=True),\n            nn.Dropout(p=0.5),\n            nn.Linear(512, 512),  # Another fully connected layer with 512 output units\n            nn.BatchNorm1d(512),\n            nn.ReLU(inplace=True),\n            nn.Dropout(p=0.5),\n            nn.Linear(512, 10)  # Final fully connected layer with 10 output units (corresponding to classes)\n        )\n        \n    def forward(self, x):\n        x = self.cnn(x)  # Pass the input through the CNN layers\n        x = x.view(x.size(0), -1)  # Flatten the output for the fully connected layers\n        x = self.classifier(x)  # Pass the flattened output through the fully connected layers\n        \n        return x\n","metadata":{"execution":{"iopub.status.busy":"2023-06-14T18:24:00.528747Z","iopub.execute_input":"2023-06-14T18:24:00.529294Z","iopub.status.idle":"2023-06-14T18:24:00.541204Z","shell.execute_reply.started":"2023-06-14T18:24:00.529259Z","shell.execute_reply":"2023-06-14T18:24:00.540226Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Visualizing Our Training Dataset\n","metadata":{}},{"cell_type":"markdown","source":"We also visualize the training data distribution(ie. number of values for each label)","metadata":{}},{"cell_type":"code","source":"train_df = pd.read_csv(train_csv)\n#plt.rcParams['figure.figsize'] = (8, 5)\nplt.bar(train_df['label'].value_counts().index, train_df['label'].value_counts())\nplt.xticks(np.arange(10))\nplt.xlabel('Class', fontsize=16)\nplt.ylabel('Count', fontsize=16)\nplt.grid('on', axis='y')","metadata":{"execution":{"iopub.status.busy":"2023-06-17T17:00:30.601737Z","iopub.execute_input":"2023-06-17T17:00:30.602312Z","iopub.status.idle":"2023-06-17T17:00:30.997823Z","shell.execute_reply.started":"2023-06-17T17:00:30.602276Z","shell.execute_reply":"2023-06-17T17:00:30.996141Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now, we visualize a few of the training images along with their labels","metadata":{}},{"cell_type":"code","source":"\n_,ax = plt.subplots(3,3,figsize=(6,6))\ntorch.manual_seed(81)\nvisualize_loader = DataLoader(train_dataset, batch_size=1,shuffle=True)\n\nfor i, (data,label) in enumerate(visualize_loader):\n    \n    \n    ax[i//3][i%3].imshow(data.cpu().squeeze())\n    ax[i//3][i%3].axis('off')\n    ax[i//3][i%3].set_title(f\"Label: {label.item()}\",y=-0.3)\n    if(i==8): break\nplt.subplots_adjust(hspace=0.5)\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2023-06-17T17:00:30.998751Z","iopub.status.idle":"2023-06-17T17:00:30.999485Z","shell.execute_reply.started":"2023-06-17T17:00:30.999247Z","shell.execute_reply":"2023-06-17T17:00:30.999270Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<br><br>\n## Model Setup & Training\n\nThe code begins by defining the loss function and optimizer for training the model using Cross Entropy Loss` and Stochastic gradient descent, respectively. The `params` parameter is used to specify the parameters to optimize\n\nThe number of epochs to train the model is defined using the `epochs` variable. The model is then set to training mode using `model.train()`.\n\nThe model is trained for the specified number of epochs using a nested loop. The outer loop iterates through the epochs, and the inner loop iterates through the training data using the `train_loader` DataLoader.\n\nFor each batch of data, the data and labels are moved to the appropriate device (GPU or CPU) using `to()`. The output of the model is computed using `model()`, the loss is calculated using `nn.CELoss()`, and backpropagation is performed using `backward()`. The `optimizer.step()` function is used to update the model parameters.\n\n\n\nAfter training the model for each epoch, the model is set to evaluation mode using `model.eval()`. The validation data is then iterated through using the `val_loader` DataLoader object.\n\n","metadata":{}},{"cell_type":"code","source":"model = CNN()\nmodel = model.to(device)\n# Defining the loss function and optimizer for training the model\ncriterion = nn.CrossEntropyLoss()\noptimizer = optim.SGD(params=model.parameters(), lr=0.001, momentum=0.9)\n\n# Defining the number of epochs to train the model\nepochs = 10\n\n# Setting the model to training mode\nmodel.train()\n\n# Training the model for the specified number of epochs\nfor epoch in range(epochs):\n    epoch_loss = 0\n    epoch_accuracy = 0\n    \n    # Iterating through the training data\n    for data, label in train_loader:\n        data = data.to(device)\n        label = label.to(device)\n        optimizer.zero_grad()\n        output = model(data)\n  \n     \n        loss = criterion(output, label)\n     \n      \n        \n        loss.backward()\n        optimizer.step()\n        \n        epoch_loss += (loss.item())/len(train_loader)\n        \n    print('Epoch : {},  train loss : {}'.format(epoch+1,epoch_loss))\n    \n    # Setting the model to evaluation mode\n    model.eval()\n    with torch.no_grad():\n        epoch_val_accuracy=0\n        epoch_val_loss =0\n        \n        # Iterating through the validation data\n        for data, label in val_loader:\n            data = data.to(device)\n            label = label.to(device)\n     \n            output = model(data)\n           \n\n            \n            val_output = model(data)\n            val_loss = criterion(val_output,label)\n            \n            epoch_val_loss += val_loss/ len(val_loader)\n            \n        print('Epoch : {} , val_loss : {}'.format(epoch+1,epoch_val_loss))\n","metadata":{"execution":{"iopub.status.busy":"2023-06-17T17:00:31.001578Z","iopub.status.idle":"2023-06-17T17:00:31.002018Z","shell.execute_reply.started":"2023-06-17T17:00:31.001795Z","shell.execute_reply":"2023-06-17T17:00:31.001816Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<br><br>\n# Visualizing Predictions\nWe will visualize the first 9 predictions made by the model","metadata":{}},{"cell_type":"code","source":"model.eval()\n_,ax = plt.subplots(3,3,figsize=(6,6))\nfor i, data in enumerate(test_loader):\n    \n \n    data = data.cuda()\n\n    output = model(data)\n    \n\n    pred = output.cpu().data.max(1, keepdim=True)[1]\n    \n    ax[i//3][i%3].imshow(data.cpu().squeeze())\n    ax[i//3][i%3].axis('off')\n    ax[i//3][i%3].set_title(f\"Prediction: {pred.item()}\",y=-0.3)\n    if(i==8): break\nplt.subplots_adjust(hspace=0.5)\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2023-06-14T18:21:50.837861Z","iopub.status.idle":"2023-06-14T18:21:50.838598Z","shell.execute_reply.started":"2023-06-14T18:21:50.838354Z","shell.execute_reply":"2023-06-14T18:21:50.838377Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<br><br>\n## Submission\n\nFinally, we inference the model on the test dataset for submission\n","metadata":{}},{"cell_type":"code","source":"model.eval()\ntest_pred = torch.LongTensor()\n\nfor i, data in enumerate(test_loader):\n    \n    if torch.cuda.is_available():\n        data = data.cuda()\n\n    output = model(data)\n\n    pred = output.cpu().data.max(1, keepdim=True)[1]\n    test_pred = torch.cat((test_pred, pred), dim=0)\n    \nprint(test_pred.shape)\nsubmission_df = pd.read_csv(\"../input/Kannada-MNIST/sample_submission.csv\")\nsubmission_df['label'] = test_pred.numpy().squeeze()\nsubmission_df.to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2023-06-14T18:31:29.805316Z","iopub.execute_input":"2023-06-14T18:31:29.805714Z","iopub.status.idle":"2023-06-14T18:31:36.560566Z","shell.execute_reply.started":"2023-06-14T18:31:29.805686Z","shell.execute_reply":"2023-06-14T18:31:36.559592Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}