{"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":3004,"databundleVersionId":861823,"sourceType":"competition"}],"dockerImageVersionId":30627,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Handwritten Digit Classification with Modified AlexNet (98.97% Accuracy)\n\n## Overview:\nIn this notebook, we have tackled the problem of handwritten digit classification using a modified AlexNet architecture. Our goal was to achieve high accuracy in recognizing handwritten digits from the MNIST dataset.\n\n## Key Steps:\n### 1. Data Loading:\n* We loaded the MNIST dataset, which consists of grayscale images of handwritten digits (0-9) and their corresponding labels.\n### 2. Data Preprocessing:\n* Data augmentation techniques, such as rotation, translation, and zoom, were applied to the training dataset to increase its diversity.\n* We converted the data into PyTorch tensors and performed normalization.\n### 3. Model Architecture:\n* We defined a modified AlexNet model with additional Batch Normalization layers.\n* The model consists of convolutional layers, ReLU activation functions, Batch Normalization, max-pooling, and fully connected layers.\n### 4. Training:\n* The model was trained on the training dataset with custom data augmentation.\n* We used cross-entropy loss and stochastic gradient descent (SGD) for optimization.\n* Training progress was monitored, and we aimed to avoid overfitting.\n### 5. Validation:\n* Model performance was evaluated on a validation dataset to ensure generalization.\n* We tracked validation accuracy to monitor model performance.\n### 6. Testing and Submission:\n* The trained model was used to make predictions on a test dataset.\n* Predictions were prepared for submission to the Kaggle competition.\n### 7. Kaggle Submission:\n* The model achieved an impressive accuracy of 98.97% on the Kaggle competition.\n* The notebook was prepared for submission to the competition.\n### 8. Conclusion:\n* Our modified AlexNet model demonstrated excellent performance in classifying handwritten digits, achieving a high accuracy rate. This notebook showcases the steps taken to preprocess data, design a neural network, train the model, and achieve success in the Kaggle competition.","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n\nfrom sklearn.model_selection import train_test_split\n\nimport torch\nfrom torchvision import transforms\nfrom torchvision.transforms.functional import to_pil_image\nfrom torch.utils.data import Dataset, DataLoader, TensorDataset, ConcatDataset\nimport torch.nn as nn\n\n!pip install torchsummary\nfrom torchsummary import summary","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:50:35.339987Z","iopub.execute_input":"2024-01-09T05:50:35.340381Z","iopub.status.idle":"2024-01-09T05:50:47.537840Z","shell.execute_reply.started":"2024-01-09T05:50:35.340339Z","shell.execute_reply":"2024-01-09T05:50:47.536769Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"look for any GPU accelaration if you can get.\n* my personal laptop is a macbook, so I use 'MPS' as device while running locally\n* on kaggle we have free GPU, so feel free use the 'CUDA' as device on kaggle","metadata":{}},{"cell_type":"code","source":"# Check PyTorch has access to MPS (Metal Performance Shader, Apple's GPU architecture)\nprint(f\"Is MPS (Metal Performance Shader) built? {torch.backends.mps.is_built()}\")\nprint(f\"Is MPS available? {torch.backends.mps.is_available()}\")\nprint(f\"Is cuda available? {torch.cuda.is_available()}\")\n# Set the device      \ndevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"\nprint(f\"Using device: {device}\")","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:50:47.540050Z","iopub.execute_input":"2024-01-09T05:50:47.540395Z","iopub.status.idle":"2024-01-09T05:50:47.546935Z","shell.execute_reply.started":"2024-01-09T05:50:47.540351Z","shell.execute_reply":"2024-01-09T05:50:47.545954Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"seeting the local paths to read the data from.\nif you're not sure you can use the following code, to understand the folders, file structure on kaggle:","metadata":{}},{"cell_type":"code","source":"import os\nfor root, folders, filenames in os.walk('/kaggle/input'):\n   print(root, folders)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"as we know the paths, set them to the respective variables.","metadata":{}},{"cell_type":"code","source":"# Load the CSV files\ntrain_csv_path = '/kaggle/input/digit-recognizer/train.csv'\ntest_csv_path = '/kaggle/input/digit-recognizer/test.csv'","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:50:47.548403Z","iopub.execute_input":"2024-01-09T05:50:47.548795Z","iopub.status.idle":"2024-01-09T05:50:47.561584Z","shell.execute_reply.started":"2024-01-09T05:50:47.548757Z","shell.execute_reply":"2024-01-09T05:50:47.560736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"load the data to respective data frames.","metadata":{}},{"cell_type":"code","source":"# Load the data into DataFrames\ntrain_df = pd.read_csv(train_csv_path)\ntest_df = pd.read_csv(test_csv_path)","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:50:47.563041Z","iopub.execute_input":"2024-01-09T05:50:47.563340Z","iopub.status.idle":"2024-01-09T05:50:52.333930Z","shell.execute_reply.started":"2024-01-09T05:50:47.563314Z","shell.execute_reply":"2024-01-09T05:50:52.332770Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Explore the data","metadata":{}},{"cell_type":"code","source":"# Show the shape of datasets\nprint(\"Training Set Shape:\", train_df.shape)\nprint(\"Test Set Shape:\", test_df.shape)","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:50:52.337419Z","iopub.execute_input":"2024-01-09T05:50:52.337728Z","iopub.status.idle":"2024-01-09T05:50:52.343442Z","shell.execute_reply.started":"2024-01-09T05:50:52.337694Z","shell.execute_reply":"2024-01-09T05:50:52.342321Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"look for any missing values","metadata":{}},{"cell_type":"code","source":"# Check for missing values in the datasets\nprint(\"Missing values in training data:\", train_df.isnull().sum().sum())\nprint(\"Missing values in test data:\", test_df.isnull().sum().sum())","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:50:52.344534Z","iopub.execute_input":"2024-01-09T05:50:52.344779Z","iopub.status.idle":"2024-01-09T05:50:52.400661Z","shell.execute_reply.started":"2024-01-09T05:50:52.344756Z","shell.execute_reply":"2024-01-09T05:50:52.399737Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"see the data understand the structure, here all 784 pixels of a 28 x 28 are prestend as columns","metadata":{}},{"cell_type":"code","source":"print(\"\\nFirst 5 Rows of Training Data:\\n\", train_df.head())","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:50:52.401978Z","iopub.execute_input":"2024-01-09T05:50:52.402346Z","iopub.status.idle":"2024-01-09T05:50:52.413933Z","shell.execute_reply.started":"2024-01-09T05:50:52.402306Z","shell.execute_reply":"2024-01-09T05:50:52.412857Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"lets try to see some sample data","metadata":{}},{"cell_type":"code","source":"# Function to display images in the dataset\ndef show_images(data, num_images=5):\n    data = data.iloc[:,1:].values\n    fig, axes = plt.subplots(1, num_images, figsize=(10,2))\n    \n    for i in range(num_images):\n        img = data[i].reshape(28,28)\n        axes[i].imshow(img, cmap='gray')\n        axes[i].set_title(f\"Label: {train_df.iloc[i, 0]}\")\n        axes[i].axis('off')\n    plt.tight_layout()\n    plt.show()\n    return\n\nshow_images(train_df, 10)","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:50:52.415269Z","iopub.execute_input":"2024-01-09T05:50:52.415608Z","iopub.status.idle":"2024-01-09T05:50:53.192043Z","shell.execute_reply.started":"2024-01-09T05:50:52.415563Z","shell.execute_reply":"2024-01-09T05:50:53.191083Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"let's prepare our dataset\n* firstly normalize the data by dividing all the coloumns expect the label by 255\n* now, we can split our data into training and validation data\n* once we have the data, we can make our dataset out of this data","metadata":{}},{"cell_type":"code","source":"# Separate features and labels\nX = train_df.drop(columns=['label']).values / 255.0\ny = train_df['label'].values\n\n# Split the dataset into training (80%) and validation (20%) sets\nX_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=55)\n\n# Convert data to PyTorch tensors\nX_train_tensor = torch.tensor(X_train, dtype=torch.float32).view(-1, 1, 28, 28).to(device)\ny_train_tensor = torch.tensor(y_train, dtype=torch.long).to(device)\nX_val_tensor = torch.tensor(X_val, dtype=torch.float32).view(-1, 1, 28, 28).to(device)\ny_val_tensor = torch.tensor(y_val, dtype=torch.long).to(device)\n\n# Create datasets and data loaders\noriginal_dataset = TensorDataset(X_train_tensor, y_train_tensor)\nval_dataset = TensorDataset(X_val_tensor, y_val_tensor)","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:50:53.193112Z","iopub.execute_input":"2024-01-09T05:50:53.193415Z","iopub.status.idle":"2024-01-09T05:50:53.850042Z","shell.execute_reply.started":"2024-01-09T05:50:53.193388Z","shell.execute_reply":"2024-01-09T05:50:53.849014Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"since, we have less data we can agument training data by applying some transformation.\n* we can't use horizontal flip, vertical flip transformation that can entirly change the label.\n* I tried shear transformation too, but accuracy is better when I just used rotation but not sheer.","metadata":{}},{"cell_type":"code","source":"# Define custom data augmentation transformations\ncustom_transform = transforms.Compose([\n    transforms.RandomRotation(degrees=(-15, 15)),  # Random rotation between -15 and 15 degrees\n    transforms.RandomAffine(degrees=0, translate=(0.1, 0.1)),  # Random translation by up to 10% of image size\n    transforms.RandomAffine(degrees=0, scale=(0.9, 1.1)),  # Random zoom between 0.9x and 1.1x\n    #transforms.RandomAffine(degrees=0, shear=(-10, 10)),  # Random shear between -10 and 10 degrees\n    transforms.ToTensor(),                       # Convert to tensor\n])","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:50:53.851341Z","iopub.execute_input":"2024-01-09T05:50:53.851745Z","iopub.status.idle":"2024-01-09T05:50:53.859251Z","shell.execute_reply.started":"2024-01-09T05:50:53.851708Z","shell.execute_reply":"2024-01-09T05:50:53.858170Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"lets genarate some agumented images and see how they look","metadata":{}},{"cell_type":"code","source":"# # Select three sample images and their labels\nsample_indices = [44, 37, 97]\noriginal_images = [to_pil_image(X_train_tensor[i]) for i in sample_indices]\nlabels = [y_train[i].item() for i in sample_indices]\n\n# Create subplots for each row of images\nfig, axes = plt.subplots(3, 10, figsize=(10,4))\n\n# Loop over each row\nfor row_index, (original_image, label) in enumerate(zip(original_images, labels)):\n    augmented_samples = [custom_transform(original_image).view((28,28,1)) for _ in range(9)]\n\n    # Original Image on the left with label\n    #plt.subplot(3, 10, row_index * 10 + 1)\n    axes[row_index][0].imshow(original_image, cmap='gray')\n    axes[row_index][0].set_title(f\"Label: {label}\")\n    axes[row_index][0].axis('off')\n\n    # Augmented Images on the right\n    for i, augmented_img in enumerate(augmented_samples):\n        #axes[row_index][i+2].subplot(3, 10, row_index * 10 + i + 2)\n        axes[row_index][i+1].imshow(augmented_img, cmap='gray')\n        axes[row_index][i+1].set_title(f\"{i+1}\")\n        axes[row_index][i+1].axis('off')\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:50:53.860966Z","iopub.execute_input":"2024-01-09T05:50:53.861359Z","iopub.status.idle":"2024-01-09T05:50:55.939768Z","shell.execute_reply.started":"2024-01-09T05:50:53.861329Z","shell.execute_reply":"2024-01-09T05:50:55.938844Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"now, we can combine the original training set and the newly genarted images.\nhere I just used the transformation once on whole dataset. I think we can improve the accuracy if we can use the transformation one more time. since we are biuilding a big architure for the task have more data will favour the task in the genral cases.","metadata":{}},{"cell_type":"code","source":"# Create augmented training dataset\naugmented_data = [(custom_transform(to_pil_image(x)), y) for x, y in zip(X_train_tensor, y_train_tensor)]\n\n# Extract the augmented data into separate tensors\nX_train_tensor_augmented = torch.stack([x for x, _ in augmented_data]).to(device)  #Convert PIL Images to tensors\ny_train_tensor_augmented = torch.tensor([y for _, y in augmented_data]).to(device)\n\n# Now you have X_train_tensor_augmented and y_train_tensor_augmented\n# You can use these tensors to create a TensorDataset if needed\naugmented_dataset = TensorDataset(X_train_tensor_augmented, y_train_tensor_augmented)\n\n# Combine both datasets\ncombined_dataset = ConcatDataset([original_dataset, augmented_dataset])\n\n# Create data loaders\nbatch_size = 64\ntrain_loader = DataLoader(combined_dataset, batch_size=batch_size, shuffle=True)\nval_loader = DataLoader(val_dataset, batch_size=batch_size)\n\n# Print the number of samples in the training and validation sets\nprint(f\"Number of training samples: {len(train_loader.dataset)}\")\nprint(f\"Number of validation samples: {len(val_loader.dataset)}\")","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:50:55.940972Z","iopub.execute_input":"2024-01-09T05:50:55.941238Z","iopub.status.idle":"2024-01-09T05:51:12.512346Z","shell.execute_reply.started":"2024-01-09T05:50:55.941215Z","shell.execute_reply":"2024-01-09T05:51:12.511402Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"since the model is big for the data, to avoid overfitting I've considered BatchNorm after after every convolution layer. also dropout layers in the fully connected network.","metadata":{}},{"cell_type":"code","source":"import torch.nn as nn\n\nclass ModifiedAlexNet(nn.Module):\n    def __init__(self, num_classes = 10):\n        super(ModifiedAlexNet, self).__init__()\n        self.features = nn.Sequential(\n            nn.Conv2d(1, 32, kernel_size=5, stride = 1, padding=0),\n            nn.ReLU(inplace=True),\n            nn.BatchNorm2d(32),\n            nn.Conv2d(32, 64, kernel_size=5, stride = 1, padding=0),\n            nn.ReLU(inplace=True),\n            nn.BatchNorm2d(64),\n            nn.MaxPool2d(kernel_size=3, stride = 1),\n            nn.Conv2d(64, 128, kernel_size=3, stride = 1, padding=0),\n            nn.ReLU(inplace=True),\n            nn.BatchNorm2d(128),\n            nn.Conv2d(128, 256, kernel_size=3, stride = 1, padding=0),\n            nn.ReLU(inplace=True),\n            nn.BatchNorm2d(256),\n            nn.MaxPool2d(kernel_size=3, stride = 1), # Max-pooling after the last convolutional layer\n        )\n        self.avgpool = nn.AdaptiveAvgPool2d((6, 6))\n        self.classifier = nn.Sequential(\n            nn.Dropout(0.6),\n            nn.Linear(256 * 6 * 6, 4096),\n            nn.ReLU(inplace=True),\n            nn.Dropout(0.6),\n            nn.Linear(4096, 4096),\n            nn.ReLU(inplace=True),\n            nn.Linear(4096, num_classes),\n        )\n\n    def forward(self, x):\n        x=x.to(device)\n        x = self.features(x)\n        x = self.avgpool(x)\n        x = x.view(x.size(0), 256 * 6 * 6)\n        x = self.classifier(x)\n        return x\n    \n# Create the Modified AlexNet model\nmodel = ModifiedAlexNet().to(device)\n\n# Print the model architecture\nprint(model)","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:51:12.513718Z","iopub.execute_input":"2024-01-09T05:51:12.514655Z","iopub.status.idle":"2024-01-09T05:51:13.088573Z","shell.execute_reply.started":"2024-01-09T05:51:12.514617Z","shell.execute_reply":"2024-01-09T05:51:13.087573Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"here is the summary of the model.","metadata":{}},{"cell_type":"code","source":"summary(model, input_size = (1, 28, 28), batch_size = -1)","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:51:13.092680Z","iopub.execute_input":"2024-01-09T05:51:13.092960Z","iopub.status.idle":"2024-01-09T05:51:14.489265Z","shell.execute_reply.started":"2024-01-09T05:51:13.092935Z","shell.execute_reply":"2024-01-09T05:51:14.487888Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training\nI observed that validation loss is increasing after 2 epochs. so the model is overfitting to the data, \nit's better to use L2 reguralization, to avoid overfitting. so weight decay along with sochastic gradient descent.if you can train for more epochs, you should most probably see better results.","metadata":{}},{"cell_type":"code","source":"import torch.optim as optim\n\n# Define hyperparameters\nlearning_rate = 0.001\nweight_decay = 0.001  # L2 regularization strength\n\n# Define the loss function and optimizer with L2 regularization\ncriterion = nn.CrossEntropyLoss()\noptimizer = optim.SGD(model.parameters(), lr=learning_rate, weight_decay=weight_decay)\n\n# Number of training epochs\nnum_epochs = 10\n\n# Create empty lists to store losses\ntrain_losses = []\nval_losses = []\n\n# Training loop\nfor epoch in range(num_epochs):\n    model.train()  # Set model to training mode\n    running_loss = 0.0\n\n    for inputs, labels in train_loader:\n        optimizer.zero_grad()\n        outputs = model(inputs)\n        loss = criterion(outputs, labels)\n        loss.backward()\n        optimizer.step()\n        running_loss += loss.item()\n\n    # Calculate average training loss for this epoch\n    avg_train_loss = running_loss / len(train_loader)\n    train_losses.append(avg_train_loss)\n\n    # Validation loop\n    model.eval()  # Set model to evaluation mode\n    running_loss = 0.0\n\n    with torch.no_grad():\n        for inputs, labels in val_loader:\n            outputs = model(inputs)\n            loss = criterion(outputs, labels)\n            running_loss += loss.item()\n\n    # Calculate average validation loss for this epoch\n    avg_val_loss = running_loss / len(val_loader)\n    val_losses.append(avg_val_loss)\n\n    # Print training and validation losses for this epoch\n    print(f'Epoch [{epoch + 1}/{num_epochs}] - Train Loss: {avg_train_loss:.4f}, Validation Loss: {avg_val_loss:.4f}')\n    \nprint(\"Finished Training\")","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:51:14.490670Z","iopub.execute_input":"2024-01-09T05:51:14.490986Z","iopub.status.idle":"2024-01-09T05:55:22.037986Z","shell.execute_reply.started":"2024-01-09T05:51:14.490959Z","shell.execute_reply":"2024-01-09T05:55:22.036927Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"now evaluate model on validation dataset.\nwah !! we have a 99% accuracy","metadata":{}},{"cell_type":"code","source":"# Assuming you have a DataLoader for the validation set named 'val_loader'\nmodel.eval()  # Set model to evaluation mode\ncorrect = 0\ntotal = 0\n\nwith torch.no_grad():\n    for inputs, labels in val_loader:\n        outputs = model(inputs)\n        _, predicted = torch.max(outputs, 1)\n        total += labels.size(0)\n        correct += (predicted == labels).sum().item()\n\naccuracy = 100 * correct / total\nprint(f'Validation Accuracy: {accuracy:.2f}%')","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:55:22.039293Z","iopub.execute_input":"2024-01-09T05:55:22.039605Z","iopub.status.idle":"2024-01-09T05:55:22.999270Z","shell.execute_reply.started":"2024-01-09T05:55:22.039579Z","shell.execute_reply":"2024-01-09T05:55:22.998112Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"let's see some misclassified images and their labels. I felt the model is doing OK.\nyou can try running for 5 more epochs and see if there are any improvements.","metadata":{}},{"cell_type":"code","source":"model.eval()  # Set model to evaluation mode\nmisclassified_samples = []\n\nwith torch.no_grad():\n    for inputs, labels in val_loader:\n        outputs = model(inputs)\n        _, predicted = torch.max(outputs, 1)\n        misclassified_indices = (predicted != labels).nonzero()  # Find misclassified samples\n        for index in misclassified_indices:\n            misclassified_samples.append((inputs[index], labels[index], predicted[index]))\n\nfig, axes = plt.subplots(1, 10, figsize=(10,2))\n\n# Print or plot misclassified samples\nfor row_index, (input_img, true_label, predicted_label) in enumerate(misclassified_samples[11:21]):\n    # Original Image on the left with label\n    #axes[row_index].subplot(1, 10, row_index + 1)\n    input_img = input_img[0].squeeze().cpu().numpy()  # Reshape and convert to a NumPy array\n    axes[row_index].imshow(input_img, cmap='gray')\n    axes[row_index].set_title(f'T: {true_label.item()}, P: {predicted_label.item()}')\n    axes[row_index].axis('off')\n    \nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:55:23.000628Z","iopub.execute_input":"2024-01-09T05:55:23.001180Z","iopub.status.idle":"2024-01-09T05:55:24.825504Z","shell.execute_reply.started":"2024-01-09T05:55:23.001150Z","shell.execute_reply":"2024-01-09T05:55:24.824577Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"finally, we made it to the predictions.\nand make the submission dataframe ready.","metadata":{}},{"cell_type":"code","source":"test_tensor = torch.tensor(test_df.values, dtype=torch.float32).view(-1, 1, 28, 28).to(device)\n\n# Set your model to evaluation mode\nmodel.eval()\n\n# Create an empty list to store the predictions\npredictions = []\n\n# Iterate through the test data in batches (you can adjust the batch size)\nbatch_size = 64\nwith torch.no_grad():\n    for i in range(0, len(test_tensor), batch_size):\n        # Get a batch of test data\n        batch = test_tensor[i:i+batch_size]\n\n        # Forward pass to make predictions\n        outputs = model(batch)\n\n        # Get the predicted class (index with the highest probability)\n        _, predicted = torch.max(outputs, 1)\n\n        # Append the predictions to the list\n        predictions.extend(predicted.cpu().numpy())\n\n# Convert the list of predictions to a Pandas Series\npredictions_series = pd.Series(predictions, name='Label')\n\n# Create a DataFrame with the 'ImageId' column\nimage_ids = pd.Series(range(1, len(test_df) + 1), name='ImageId')\n\n# Concatenate the 'ImageId' and 'Label' columns\nsubmission = pd.concat([image_ids, predictions_series], axis=1)\nsubmission.head()","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:55:24.826728Z","iopub.execute_input":"2024-01-09T05:55:24.827012Z","iopub.status.idle":"2024-01-09T05:55:27.817990Z","shell.execute_reply.started":"2024-01-09T05:55:24.826986Z","shell.execute_reply":"2024-01-09T05:55:27.816977Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"let's see some of the test samples and their predicitons.","metadata":{}},{"cell_type":"code","source":"# Select 10 random indices from the test dataset\nrandom_indices = np.random.choice(len(test_df), 10, replace=False)\n\nfig, axes = plt.subplots(1, 10, figsize=(15, 2))\n\n# Print or plot randomly selected samples\nfor i in range(0,10):\n    input_img = test_df.iloc[random_indices[i]].values.reshape(28,28)\n    axes[i].imshow(input_img, cmap='gray')\n    axes[i].set_title(f'Predicted: {submission.iloc[random_indices[i]][\"Label\"]}')\n    axes[i].axis('off')\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:55:27.819436Z","iopub.execute_input":"2024-01-09T05:55:27.820067Z","iopub.status.idle":"2024-01-09T05:55:28.520721Z","shell.execute_reply.started":"2024-01-09T05:55:27.820029Z","shell.execute_reply":"2024-01-09T05:55:28.519709Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"we can make our first CNN submission.","metadata":{}},{"cell_type":"code","source":"# Export the submission DataFrame to a CSV file\nsubmission.to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2024-01-09T05:55:28.521864Z","iopub.execute_input":"2024-01-09T05:55:28.522163Z","iopub.status.idle":"2024-01-09T05:55:28.585904Z","shell.execute_reply.started":"2024-01-09T05:55:28.522135Z","shell.execute_reply":"2024-01-09T05:55:28.584914Z"},"trusted":true},"execution_count":null,"outputs":[]}]}