{"metadata":{"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":16017,"databundleVersionId":708189,"sourceType":"competition"}],"dockerImageVersionId":30733,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.13"},"papermill":{"default_parameters":{},"duration":634.137694,"end_time":"2024-07-05T17:26:50.694072","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-07-05T17:16:16.556378","version":"2.5.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"import torch\nfrom torch.utils.data import DataLoader,Dataset\nimport torch.nn as nn\nimport torchvision\nimport torchvision.transforms as transforms\nfrom torch.utils.data.sampler import SubsetRandomSampler\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport pandas as pd","metadata":{"papermill":{"duration":5.620805,"end_time":"2024-07-05T17:16:24.763139","exception":false,"start_time":"2024-07-05T17:16:19.142334","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:28:22.838003Z","iopub.execute_input":"2024-07-08T10:28:22.838410Z","iopub.status.idle":"2024-07-08T10:28:27.992357Z","shell.execute_reply.started":"2024-07-08T10:28:22.838375Z","shell.execute_reply":"2024-07-08T10:28:27.991544Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## STEP 1: Init dataset and dataloader","metadata":{"papermill":{"duration":0.004905,"end_time":"2024-07-05T17:16:24.773474","exception":false,"start_time":"2024-07-05T17:16:24.768569","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class TrainMNIST(Dataset):\n    def __init__(self,file_path,transform=None):\n        self.data = pd.read_csv(file_path)\n        self.transform = transform\n\n    def __len__(self):\n        return len(self.data)\n\n    def __getitem__(self, idx):\n        img = self.data.iloc[idx,1:].values.astype(np.uint8).reshape((28, 28, 1))\n        label = self.data.iloc[idx,0]\n        if self.transform is not None:\n            img = self.transform(img)\n        return img,label","metadata":{"papermill":{"duration":0.016325,"end_time":"2024-07-05T17:16:24.794795","exception":false,"start_time":"2024-07-05T17:16:24.778470","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:28:27.993807Z","iopub.execute_input":"2024-07-08T10:28:27.994202Z","iopub.status.idle":"2024-07-08T10:28:28.000675Z","shell.execute_reply.started":"2024-07-08T10:28:27.994176Z","shell.execute_reply":"2024-07-08T10:28:27.999706Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class TestMNIST(Dataset):\n    def __init__(self,file_path,transform=None):\n        self.data = pd.read_csv(file_path)\n        self.transform = transform\n\n    def __len__(self):\n        return len(self.data)\n\n    def __getitem__(self, idx):\n        img = self.data.iloc[idx,1:].values.astype(np.uint8).reshape((28, 28, 1))\n        \n        if self.transform is not None:\n            img = self.transform(img)\n        return img","metadata":{"papermill":{"duration":0.014215,"end_time":"2024-07-05T17:16:24.814259","exception":false,"start_time":"2024-07-05T17:16:24.800044","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:28:28.001711Z","iopub.execute_input":"2024-07-08T10:28:28.001953Z","iopub.status.idle":"2024-07-08T10:28:28.026676Z","shell.execute_reply.started":"2024-07-08T10:28:28.001931Z","shell.execute_reply":"2024-07-08T10:28:28.025921Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_data = TrainMNIST('/kaggle/input/Kannada-MNIST/train.csv', transform=transforms.ToTensor())\ntest_data = TestMNIST('/kaggle/input/Kannada-MNIST/test.csv', transform=transforms.ToTensor())","metadata":{"papermill":{"duration":5.75838,"end_time":"2024-07-05T17:16:30.577708","exception":false,"start_time":"2024-07-05T17:16:24.819328","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:28:28.028268Z","iopub.execute_input":"2024-07-08T10:28:28.028555Z","iopub.status.idle":"2024-07-08T10:28:32.847914Z","shell.execute_reply.started":"2024-07-08T10:28:28.028530Z","shell.execute_reply":"2024-07-08T10:28:32.847113Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"num_workers = 0\nbatch_size = 20\ntrain_loader = torch.utils.data.DataLoader(train_data, batch_size=batch_size,\n     num_workers=num_workers,)\ntest_loader = torch.utils.data.DataLoader(test_data, batch_size=batch_size, \n    num_workers=num_workers)","metadata":{"papermill":{"duration":0.014691,"end_time":"2024-07-05T17:16:30.597797","exception":false,"start_time":"2024-07-05T17:16:30.583106","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:30:04.102602Z","iopub.execute_input":"2024-07-08T10:30:04.103002Z","iopub.status.idle":"2024-07-08T10:30:04.108203Z","shell.execute_reply.started":"2024-07-08T10:30:04.102973Z","shell.execute_reply":"2024-07-08T10:30:04.107299Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n%matplotlib inline\n    \n# obtain one batch of training images\ndataiter = iter(train_loader)\nimages, labels = next(dataiter)\nimages = images.numpy()\n\n# plot the images in the batch, along with the corresponding labels\nfig = plt.figure(figsize=(25, 4))\nfor idx in np.arange(20):\n    ax = fig.add_subplot(2, int(20/2), idx+1, xticks=[], yticks=[])\n    ax.imshow(np.squeeze(images[idx]), cmap='gray')\n    # print out the correct label for each image\n    # .item() gets the value contained in a Tensor\n    ax.set_title(str(labels[idx].item()))","metadata":{"papermill":{"duration":1.108106,"end_time":"2024-07-05T17:16:31.711290","exception":false,"start_time":"2024-07-05T17:16:30.603184","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:30:06.752453Z","iopub.execute_input":"2024-07-08T10:30:06.753325Z","iopub.status.idle":"2024-07-08T10:30:07.858510Z","shell.execute_reply.started":"2024-07-08T10:30:06.753291Z","shell.execute_reply":"2024-07-08T10:30:07.857035Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# STEP 2: Define the model","metadata":{"papermill":{"duration":0.005343,"end_time":"2024-07-05T17:16:31.722251","exception":false,"start_time":"2024-07-05T17:16:31.716908","status":"completed"},"tags":[]}},{"cell_type":"code","source":"fc1 = nn.Linear(28*28, 12)","metadata":{"papermill":{"duration":0.016097,"end_time":"2024-07-05T17:16:31.743836","exception":false,"start_time":"2024-07-05T17:16:31.727739","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:30:10.780469Z","iopub.execute_input":"2024-07-08T10:30:10.781366Z","iopub.status.idle":"2024-07-08T10:30:10.789787Z","shell.execute_reply.started":"2024-07-08T10:30:10.781333Z","shell.execute_reply":"2024-07-08T10:30:10.788983Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch.nn.functional as F\n\n\nclass Net(nn.Module):\n    def __init__(self):\n        super(Net, self).__init__()\n        hidden_1 = 512\n        hidden_2 = 512\n        hidden_3=512\n        self.fc1 = nn.Linear(28*28, hidden_1)\n        self.fc2=nn.Linear(hidden_1, hidden_2)\n        self.fc3=nn.Linear(hidden_2, hidden_3)\n        self.fc4 = nn.Linear(hidden_3, 10)\n    def forward(self,x):\n        x = x.flatten(start_dim=1)\n        x = F.tanh(self.fc1(x))\n        x = F.tanh(self.fc2(x))\n        x = F.tanh(self.fc3(x))\n        x = self.fc4(x)\n        return x\ndevice = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\nmodel=Net()\nmodel.to(device)\nprint(model)","metadata":{"papermill":{"duration":0.029477,"end_time":"2024-07-05T17:16:31.779122","exception":false,"start_time":"2024-07-05T17:16:31.749645","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:30:15.612056Z","iopub.execute_input":"2024-07-08T10:30:15.612408Z","iopub.status.idle":"2024-07-08T10:30:15.801620Z","shell.execute_reply.started":"2024-07-08T10:30:15.612379Z","shell.execute_reply":"2024-07-08T10:30:15.800650Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# STEP 3: Define the loss function and optimizer","metadata":{"papermill":{"duration":0.00544,"end_time":"2024-07-05T17:16:31.790247","exception":false,"start_time":"2024-07-05T17:16:31.784807","status":"completed"},"tags":[]}},{"cell_type":"code","source":"loss=nn.CrossEntropyLoss() #cross entropy loss for classification\noptimizer=torch.optim.Adam(model.parameters(),lr=0.001) #Adamtive optimizer","metadata":{"papermill":{"duration":0.013993,"end_time":"2024-07-05T17:16:31.809774","exception":false,"start_time":"2024-07-05T17:16:31.795781","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:30:18.107394Z","iopub.execute_input":"2024-07-08T10:30:18.108366Z","iopub.status.idle":"2024-07-08T10:30:18.113037Z","shell.execute_reply.started":"2024-07-08T10:30:18.108331Z","shell.execute_reply":"2024-07-08T10:30:18.112143Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# STEP 4: Train the model","metadata":{"papermill":{"duration":0.005255,"end_time":"2024-07-05T17:16:31.820694","exception":false,"start_time":"2024-07-05T17:16:31.815439","status":"completed"},"tags":[]}},{"cell_type":"code","source":"n_epochs = 20\ntrain_loss_min = np.Inf\n\n\n\nfor epoch in range(n_epochs):\n    train_loss = 0.0\n    model.train()\n    for data, target in train_loader:\n        data, target = data.to(device), target.to(device)\n        optimizer.zero_grad()  # clear the gradients of all optimized variable\n        output = model(data)\n        loss_output = loss(output, target)\n        loss_output.backward() #Derivative\n        optimizer.step() #Update weights\n        train_loss += loss_output.item()*data.size(0)\n        mean_loss = train_loss/len(train_loader.dataset)\n\n    print(f'Epoch: {epoch} \\tTraining Loss: {mean_loss:.6f}')\n    if mean_loss <= train_loss_min:\n        print('Loss decreased ({:.6f} --> {:.6f}).  Saving model ...'.format(train_loss_min,mean_loss))\n        torch.save(model.state_dict(), 'model.pt') #Save model\n        train_loss_min = mean_loss","metadata":{"papermill":{"duration":613.293463,"end_time":"2024-07-05T17:26:45.120331","exception":false,"start_time":"2024-07-05T17:16:31.826868","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:30:28.368413Z","iopub.execute_input":"2024-07-08T10:30:28.368824Z","iopub.status.idle":"2024-07-08T10:35:51.354163Z","shell.execute_reply.started":"2024-07-08T10:30:28.368793Z","shell.execute_reply":"2024-07-08T10:35:51.353244Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.load_state_dict(torch.load('model.pt'))","metadata":{"papermill":{"duration":0.0212,"end_time":"2024-07-05T17:26:45.148399","exception":false,"start_time":"2024-07-05T17:26:45.127199","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:35:59.199190Z","iopub.execute_input":"2024-07-08T10:35:59.199562Z","iopub.status.idle":"2024-07-08T10:35:59.210478Z","shell.execute_reply.started":"2024-07-08T10:35:59.199531Z","shell.execute_reply":"2024-07-08T10:35:59.209573Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Test the model","metadata":{"papermill":{"duration":0.006741,"end_time":"2024-07-05T17:26:45.162271","exception":false,"start_time":"2024-07-05T17:26:45.155530","status":"completed"},"tags":[]}},{"cell_type":"code","source":"dataiter=iter(test_loader)\nimages=next(dataiter)\noutput=model(images.to(device))\n_, preds = torch.max(output, 1)\nimages = images.numpy()\nfig = plt.figure(figsize=(25, 4))\nfor idx in range(test_loader.batch_size):\n    ax = fig.add_subplot(2, int(test_loader.batch_size/2), idx+1, xticks=[], yticks=[])\n    ax.imshow(np.squeeze(images[idx]), cmap='gray')\n    ax.set_title(str(preds[idx].item()), color=(\"green\" ))","metadata":{"papermill":{"duration":1.244729,"end_time":"2024-07-05T17:26:46.414114","exception":false,"start_time":"2024-07-05T17:26:45.169385","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:36:01.777253Z","iopub.execute_input":"2024-07-08T10:36:01.777624Z","iopub.status.idle":"2024-07-08T10:36:02.787149Z","shell.execute_reply.started":"2024-07-08T10:36:01.777593Z","shell.execute_reply":"2024-07-08T10:36:02.786230Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Predict Test Data and Submit","metadata":{"papermill":{"duration":0.007269,"end_time":"2024-07-05T17:26:46.429064","exception":false,"start_time":"2024-07-05T17:26:46.421795","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def predict(data,dataloader):\n    rs=[]\n    for idx,batch in enumerate(dataloader):\n        output=model(batch.to(device))\n        _, preds = torch.max(output, 1)\n        rs.append(preds.cpu().numpy())\n    return rs","metadata":{"papermill":{"duration":0.017882,"end_time":"2024-07-05T17:26:46.454902","exception":false,"start_time":"2024-07-05T17:26:46.437020","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:36:06.979383Z","iopub.execute_input":"2024-07-08T10:36:06.979770Z","iopub.status.idle":"2024-07-08T10:36:06.985440Z","shell.execute_reply.started":"2024-07-08T10:36:06.979739Z","shell.execute_reply":"2024-07-08T10:36:06.984382Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"new_batch=1\ntest_loader = torch.utils.data.DataLoader(test_data, batch_size=new_batch, \n    num_workers=num_workers)\nrs=predict(test_data,test_loader)\n","metadata":{"papermill":{"duration":2.937746,"end_time":"2024-07-05T17:26:49.400206","exception":false,"start_time":"2024-07-05T17:26:46.462460","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:36:10.375395Z","iopub.execute_input":"2024-07-08T10:36:10.375777Z","iopub.status.idle":"2024-07-08T10:36:13.451222Z","shell.execute_reply.started":"2024-07-08T10:36:10.375745Z","shell.execute_reply":"2024-07-08T10:36:13.450242Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission=pd.DataFrame(data={\n    'id':np.arange(len(rs)),\n    'label':np.concatenate(rs)\n})","metadata":{"papermill":{"duration":0.021192,"end_time":"2024-07-05T17:26:49.429259","exception":false,"start_time":"2024-07-05T17:26:49.408067","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:36:16.566715Z","iopub.execute_input":"2024-07-08T10:36:16.567084Z","iopub.status.idle":"2024-07-08T10:36:16.575625Z","shell.execute_reply.started":"2024-07-08T10:36:16.567054Z","shell.execute_reply":"2024-07-08T10:36:16.574707Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.to_csv('submission.csv',index=False)","metadata":{"papermill":{"duration":0.024692,"end_time":"2024-07-05T17:26:49.461818","exception":false,"start_time":"2024-07-05T17:26:49.437126","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-07-08T10:36:20.920643Z","iopub.execute_input":"2024-07-08T10:36:20.920994Z","iopub.status.idle":"2024-07-08T10:36:20.933332Z","shell.execute_reply.started":"2024-07-08T10:36:20.920966Z","shell.execute_reply":"2024-07-08T10:36:20.932491Z"},"trusted":true},"execution_count":null,"outputs":[]}]}