{"metadata":{"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":16017,"databundleVersionId":708189,"sourceType":"competition"}],"dockerImageVersionId":30636,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.12"},"papermill":{"default_parameters":{},"duration":722.817104,"end_time":"2024-01-21T14:01:27.194341","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-01-21T13:49:24.377237","version":"2.4.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","papermill":{"duration":0.726774,"end_time":"2024-01-21T13:49:28.484567","exception":false,"start_time":"2024-01-21T13:49:27.757793","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-01-21T14:23:56.412761Z","iopub.execute_input":"2024-01-21T14:23:56.413196Z","iopub.status.idle":"2024-01-21T14:23:56.801113Z","shell.execute_reply.started":"2024-01-21T14:23:56.413162Z","shell.execute_reply":"2024-01-21T14:23:56.800182Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 1. Import libraries","metadata":{"papermill":{"duration":0.005867,"end_time":"2024-01-21T13:49:28.496848","exception":false,"start_time":"2024-01-21T13:49:28.490981","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import torch\nfrom torch.utils.data import DataLoader, Dataset\nimport torchvision\nimport torchvision.transforms as transforms\nfrom torch.utils.data.sampler import SubsetRandomSampler\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd","metadata":{"papermill":{"duration":3.70959,"end_time":"2024-01-21T13:49:32.213305","exception":false,"start_time":"2024-01-21T13:49:28.503715","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-01-21T14:23:56.802886Z","iopub.execute_input":"2024-01-21T14:23:56.803266Z","iopub.status.idle":"2024-01-21T14:23:58.632294Z","shell.execute_reply.started":"2024-01-21T14:23:56.803239Z","shell.execute_reply":"2024-01-21T14:23:58.631499Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 2. Define custom dataset class","metadata":{"papermill":{"duration":0.005962,"end_time":"2024-01-21T13:49:32.226353","exception":false,"start_time":"2024-01-21T13:49:32.220391","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class TrainMNIST(Dataset):\n    def __init__(self, file_path, transform = None):\n        self.data = pd.read_csv(file_path)\n        self.transform = transform\n    \n    def __len__(self):\n        return len(self.data)\n    \n    def __getitem__(self, index):\n        #load image as ndarray type (Height * Width * Channels)\n        image = self.data.iloc[index, 1:].values.astype(np.uint8).reshape((28,28,1))\n        label = self.data.iloc[index, 0]\n        \n        if self.transform is not None:\n            image = self.transform(image)\n    \n        return image, label\n\n\nclass TestMNIST(Dataset):\n    def __init__(self, file_path, transform = None):\n        self.data = pd.read_csv(file_path)\n        self.transform = transform\n        \n    def __len__(self):\n        return len(self.data)\n    \n    def __getitem__(self, index):\n        #load image as ndarray type (Height * Width * Channels)\n        image = self.data.iloc[index, 1:].values.astype(np.uint8).reshape((28,28,1))\n        \n        if self.transform is not None:\n            image = self.transform(image)\n    \n        return image","metadata":{"papermill":{"duration":0.017989,"end_time":"2024-01-21T13:49:32.250293","exception":false,"start_time":"2024-01-21T13:49:32.232304","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-01-21T14:23:58.633345Z","iopub.execute_input":"2024-01-21T14:23:58.633708Z","iopub.status.idle":"2024-01-21T14:23:58.643140Z","shell.execute_reply.started":"2024-01-21T14:23:58.633685Z","shell.execute_reply":"2024-01-21T14:23:58.642204Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 3. Load and Visualize the data","metadata":{"papermill":{"duration":0.005666,"end_time":"2024-01-21T13:49:32.261808","exception":false,"start_time":"2024-01-21T13:49:32.256142","status":"completed"},"tags":[]}},{"cell_type":"code","source":"train_data = TrainMNIST('/kaggle/input/Kannada-MNIST/train.csv', transform=transforms.ToTensor())\ntest_data = TestMNIST('/kaggle/input/Kannada-MNIST/test.csv', transform=transforms.ToTensor())","metadata":{"papermill":{"duration":4.835317,"end_time":"2024-01-21T13:49:37.103108","exception":false,"start_time":"2024-01-21T13:49:32.267791","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-01-21T14:23:58.644412Z","iopub.execute_input":"2024-01-21T14:23:58.644658Z","iopub.status.idle":"2024-01-21T14:24:04.220034Z","shell.execute_reply.started":"2024-01-21T14:23:58.644636Z","shell.execute_reply":"2024-01-21T14:24:04.219126Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Count samples\nnum_train_samples = len(train_data)\nnum_test_samples = len(test_data)\nprint('Number of samples in trainset:  ',num_train_samples)\nprint('Number of samples in trainset:  ',num_test_samples)","metadata":{"papermill":{"duration":0.014413,"end_time":"2024-01-21T13:49:37.123960","exception":false,"start_time":"2024-01-21T13:49:37.109547","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-01-21T14:24:04.223358Z","iopub.execute_input":"2024-01-21T14:24:04.223753Z","iopub.status.idle":"2024-01-21T14:24:04.228520Z","shell.execute_reply.started":"2024-01-21T14:24:04.223727Z","shell.execute_reply":"2024-01-21T14:24:04.227632Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"num_workers = 0\nbatch_size = 20\n\ntrain_loader = torch.utils.data.DataLoader(train_data, batch_size = batch_size, \n                                           num_workers = num_workers)","metadata":{"papermill":{"duration":0.01293,"end_time":"2024-01-21T13:49:37.142887","exception":false,"start_time":"2024-01-21T13:49:37.129957","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-01-21T14:24:04.229671Z","iopub.execute_input":"2024-01-21T14:24:04.229995Z","iopub.status.idle":"2024-01-21T14:24:04.242057Z","shell.execute_reply.started":"2024-01-21T14:24:04.229970Z","shell.execute_reply":"2024-01-21T14:24:04.241314Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"## Visualize train.csv\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\n#obtain one batch of training images\ndataiter = iter(train_loader)\nimages, labels = next(dataiter)\nimages = images.numpy()\n\n# plot the images in the batch, along with the corresponding labels\nfig = plt.figure(figsize = (25,4))\nfor idx in np.arange(20):\n    ax = fig.add_subplot(2, int(20/2), idx+1, xticks=[], yticks=[])\n    ax.imshow(np.squeeze(images[idx]), cmap = 'gray')\n    ax.set_title(str(labels[idx].item()))","metadata":{"papermill":{"duration":1.254378,"end_time":"2024-01-21T13:49:38.403245","exception":false,"start_time":"2024-01-21T13:49:37.148867","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-01-21T14:24:04.243173Z","iopub.execute_input":"2024-01-21T14:24:04.243748Z","iopub.status.idle":"2024-01-21T14:24:05.589338Z","shell.execute_reply.started":"2024-01-21T14:24:04.243723Z","shell.execute_reply":"2024-01-21T14:24:05.588438Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 4. Build a simple Neural Network","metadata":{"papermill":{"duration":0.007363,"end_time":"2024-01-21T13:49:38.417274","exception":false,"start_time":"2024-01-21T13:49:38.409911","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import torch.nn as nn\nimport torch.nn.functional as F\n\nclass Net(nn.Module):\n    def __init__(self):\n        super(Net, self).__init__()\n        hidden_1 = 512\n        hidden_2 = 512\n        self.fc1 = nn.Linear(28*28, hidden_1)\n        self.fc2 = nn.Linear(hidden_1, hidden_2)\n        self.fc3 = nn.Linear(hidden_2, 10)\n        \n    def forward(self, x):\n        #flatten image input\n        x = x.view(-1, 28*28)\n        # add hidden layer, with relu activation function\n        x = F.relu(self.fc1(x))\n        x = F.relu(self.fc2(x))\n        x = self.fc3(x)\n        return x\n    \nmodel = Net()\nprint(model)","metadata":{"papermill":{"duration":0.02473,"end_time":"2024-01-21T13:49:38.448439","exception":false,"start_time":"2024-01-21T13:49:38.423709","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-01-21T14:24:05.590521Z","iopub.execute_input":"2024-01-21T14:24:05.590816Z","iopub.status.idle":"2024-01-21T14:24:05.606320Z","shell.execute_reply.started":"2024-01-21T14:24:05.590789Z","shell.execute_reply":"2024-01-21T14:24:05.605456Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 5. Loss Function and Optimizer","metadata":{"papermill":{"duration":0.006307,"end_time":"2024-01-21T13:49:38.461335","exception":false,"start_time":"2024-01-21T13:49:38.455028","status":"completed"},"tags":[]}},{"cell_type":"code","source":"criterion = nn.CrossEntropyLoss()\noptimizer = torch.optim.SGD(model.parameters(), lr=0.01)","metadata":{"papermill":{"duration":0.013483,"end_time":"2024-01-21T13:49:38.481346","exception":false,"start_time":"2024-01-21T13:49:38.467863","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-01-21T14:24:05.607460Z","iopub.execute_input":"2024-01-21T14:24:05.607738Z","iopub.status.idle":"2024-01-21T14:24:05.612577Z","shell.execute_reply.started":"2024-01-21T14:24:05.607714Z","shell.execute_reply":"2024-01-21T14:24:05.611505Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 6. Train the network","metadata":{"papermill":{"duration":0.006234,"end_time":"2024-01-21T13:49:38.494050","exception":false,"start_time":"2024-01-21T13:49:38.487816","status":"completed"},"tags":[]}},{"cell_type":"code","source":"n_epochs = 40\ntrain_loss_min = np.Inf\n\nfor epoch in range(n_epochs):\n    train_loss = 0.0\n    model.train()\n    for data, target in train_loader:\n        optimizer.zero_grad()   # clear the gradients of all optimized variables\n        output = model(data)\n        loss = criterion(output, target)\n        loss.backward()\n        optimizer.step()   # perform a single optimization step (parameter update)\n        train_loss += loss.item()*data.size(0)\n        \n    train_loss = train_loss/len(train_loader.dataset)\n    print('Epoch: {} \\t Training loss {:.6f}'.format(epoch+1, train_loss))\n    \n    # save model if validation loss has decreased\n    if train_loss <= train_loss_min:\n        print('Training loss decreased ({:.6f} --> {:.6f}). Saving model...'.format(train_loss_min,\n                                                                                    train_loss))\n        torch.save(model.state_dict(),'model.pt')\n        train_loss_min = train_loss","metadata":{"papermill":{"duration":703.872593,"end_time":"2024-01-21T14:01:22.373070","exception":false,"start_time":"2024-01-21T13:49:38.500477","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-01-21T14:24:05.613807Z","iopub.execute_input":"2024-01-21T14:24:05.614227Z","iopub.status.idle":"2024-01-21T14:36:26.153624Z","shell.execute_reply.started":"2024-01-21T14:24:05.614202Z","shell.execute_reply":"2024-01-21T14:36:26.152551Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Load the model with the lowest validation loss","metadata":{"papermill":{"duration":0.009938,"end_time":"2024-01-21T14:01:22.393574","exception":false,"start_time":"2024-01-21T14:01:22.383636","status":"completed"},"tags":[]}},{"cell_type":"code","source":"model.load_state_dict(torch.load('model.pt'))","metadata":{"papermill":{"duration":0.021599,"end_time":"2024-01-21T14:01:22.425446","exception":false,"start_time":"2024-01-21T14:01:22.403847","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-01-21T14:36:26.155011Z","iopub.execute_input":"2024-01-21T14:36:26.155914Z","iopub.status.idle":"2024-01-21T14:36:26.166369Z","shell.execute_reply.started":"2024-01-21T14:36:26.155863Z","shell.execute_reply":"2024-01-21T14:36:26.165444Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 7. Predict Kannada digit based on input","metadata":{"papermill":{"duration":0.010063,"end_time":"2024-01-21T14:01:22.445908","exception":false,"start_time":"2024-01-21T14:01:22.435845","status":"completed"},"tags":[]}},{"cell_type":"code","source":"test_loader = torch.utils.data.DataLoader(test_data, batch_size=1,num_workers=num_workers)\n\n\ndef predict(model, dataloader):\n    prediction_list = []\n    for i, batch in enumerate(dataloader):\n        outputs = model(batch)\n        _, predicted = torch.max(outputs.data, 1) \n        prediction_list.append(predicted.cpu())\n    return prediction_list\n\n\npredictions = predict(model,test_loader)\n# predictions = np.array(predictions)  # error: The requested array has an inhomogeneous shape after 1 dimensions.\noutput = []\nfor i in predictions:\n    output.append(i.item())\noutput = np.array(output)","metadata":{"papermill":{"duration":3.108585,"end_time":"2024-01-21T14:01:25.564642","exception":false,"start_time":"2024-01-21T14:01:22.456057","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-01-21T14:36:26.167956Z","iopub.execute_input":"2024-01-21T14:36:26.168440Z","iopub.status.idle":"2024-01-21T14:36:29.296887Z","shell.execute_reply.started":"2024-01-21T14:36:26.168414Z","shell.execute_reply":"2024-01-21T14:36:29.295904Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 8. Create file submission on kaggle","metadata":{"papermill":{"duration":0.009972,"end_time":"2024-01-21T14:01:25.585258","exception":false,"start_time":"2024-01-21T14:01:25.575286","status":"completed"},"tags":[]}},{"cell_type":"code","source":"test_df = pd.read_csv(\"/kaggle/input/Kannada-MNIST/test.csv\")\n\nsubmission = pd.DataFrame(data={\n    \"id\": test_df['id'],\n    \"label\": output\n})\nsubmission.to_csv(\"submission.csv\", index=None)","metadata":{"papermill":{"duration":0.323971,"end_time":"2024-01-21T14:01:25.919392","exception":false,"start_time":"2024-01-21T14:01:25.595421","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-01-21T14:36:29.298260Z","iopub.execute_input":"2024-01-21T14:36:29.298567Z","iopub.status.idle":"2024-01-21T14:36:29.633897Z","shell.execute_reply.started":"2024-01-21T14:36:29.298541Z","shell.execute_reply":"2024-01-21T14:36:29.632856Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Kết quả thực tế trên Kaggle: Accuracy ~ 93%","metadata":{"papermill":{"duration":0.010628,"end_time":"2024-01-21T14:01:25.940785","exception":false,"start_time":"2024-01-21T14:01:25.930157","status":"completed"},"tags":[]}},{"cell_type":"code","source":"","metadata":{"papermill":{"duration":0.010128,"end_time":"2024-01-21T14:01:25.961298","exception":false,"start_time":"2024-01-21T14:01:25.951170","status":"completed"},"tags":[]},"execution_count":null,"outputs":[]}]}