{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt # for plotting beautiful graphs\n\n# train test split from sklearn\nfrom sklearn.model_selection import train_test_split\n\n#Pytorch Stuff \nimport torch\nimport torch.nn as nn\nfrom torch import optim\nfrom torch.utils.data import DataLoader,Dataset\n\n# What's in the current directory?\nimport os\nprint(os.listdir(\"../input\"))","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2021-06-14T16:59:06.117746Z","iopub.execute_input":"2021-06-14T16:59:06.118209Z","iopub.status.idle":"2021-06-14T16:59:08.382756Z","shell.execute_reply.started":"2021-06-14T16:59:06.118083Z","shell.execute_reply":"2021-06-14T16:59:08.380674Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Given the increasing popularity of Pytorch among Kaggle users I wanted to learn the framework. I decided to try and recreate this [MNIST Tensorflow Notebook](https://www.kaggle.com/brendanartley/mnist-keras-cnn-99-6) using Pytorch. \n\nPytorch Users: Feel free to give me suggestions and let me know how I can improve my code!\n\n-- --\n\nHere are some Kaggle notebooks that helped me understand Pytorch so far.\n\n- [Pytorch Tutorial for Deep Learning Lovers](https://www.kaggle.com/kanncaa1/pytorch-tutorial-for-deep-learning-lovers)\n\n- [CNN with PyTorch (0.995 Accuracy)CNN with PyTorch (0.995 Accuracy)](https://www.kaggle.com/juiyangchang/cnn-with-pytorch-0-995-accuracy/datahttps://www.kaggle.com/juiyangchang/cnn-with-pytorch-0-995-accuracy/data)\n\n- [Shervine's Blog - Pytorch How To Generate Data](https://stanford.edu/~shervine/blog/pytorch-how-to-generate-data-parallel)\n\n- [Pytorch Dataset and Dataloader Notebook](https://www.kaggle.com/pinocookie/pytorch-dataset-and-dataloader)\n","metadata":{}},{"cell_type":"markdown","source":"### Set Seed\n\nSetting a seed so that the notebook results are reproducable.","metadata":{}},{"cell_type":"code","source":"class my_params:\n    seed = 1\n    batch_size = 256\n    test_size = 0.1","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:08.384506Z","iopub.execute_input":"2021-06-14T16:59:08.384848Z","iopub.status.idle":"2021-06-14T16:59:08.391626Z","shell.execute_reply.started":"2021-06-14T16:59:08.384802Z","shell.execute_reply":"2021-06-14T16:59:08.38855Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Loading Data\n\nReading the csv files into pandas. Usually image data is not stored using a CSV file, but since MNIST images are so small we are able to do so.","metadata":{}},{"cell_type":"code","source":"train_pd = pd.read_csv(\"../input/digit-recognizer/train.csv\", dtype=np.float32)\nfinal_test = pd.read_csv(\"../input/digit-recognizer/test.csv\", dtype=np.float32)\nsample_sub = pd.read_csv(\"../input/digit-recognizer/sample_submission.csv\")\ntrain_pd.info()","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:08.394748Z","iopub.execute_input":"2021-06-14T16:59:08.395355Z","iopub.status.idle":"2021-06-14T16:59:16.410381Z","shell.execute_reply.started":"2021-06-14T16:59:08.395299Z","shell.execute_reply":"2021-06-14T16:59:16.409151Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### GPU or CPU\n\nThe following cell detects if we are able to use GPUs in the current notebook environement. If it is available we will train the model on GPUs, otherwise we will train the Model on CPUs.","metadata":{}},{"cell_type":"code","source":"device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n\n# Assuming that we are on a CUDA machine, this should print a CUDA device:\n\nprint(device)","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:16.412575Z","iopub.execute_input":"2021-06-14T16:59:16.41301Z","iopub.status.idle":"2021-06-14T16:59:16.503478Z","shell.execute_reply.started":"2021-06-14T16:59:16.412965Z","shell.execute_reply":"2021-06-14T16:59:16.502049Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Creating Numpy Arrays\n\nIn the following, we are seperating the pixel columns from the label columns, and converting the pandas dataframe to a numpy array.\n\nWe also create training and validation datasets. ","metadata":{}},{"cell_type":"code","source":"# Seperate the features and labels\nlabels_np = train_pd.label.values \nfeatures_np = train_pd.loc[:, train_pd.columns != 'label'].values/255 \n\n# Split into training and validation set\nfeatures_train, features_val, labels_train, labels_val = train_test_split(features_np, labels_np, \n                                                                            test_size = my_params.test_size, \n                                                                            random_state = my_params.seed\n                                                                           )","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:16.50803Z","iopub.execute_input":"2021-06-14T16:59:16.508398Z","iopub.status.idle":"2021-06-14T16:59:16.983699Z","shell.execute_reply.started":"2021-06-14T16:59:16.508337Z","shell.execute_reply":"2021-06-14T16:59:16.982597Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Creating Tensors and Pytorch Datasets\n\n\nIn the following cell we are converting our numpy array's into Torch Tensors. \n\n\"Tensors are similar to NumPy’s ndarrays, except that tensors can run on GPUs or other hardware accelerators\"\n\nRead more here --> [Tensors Tutorial](https://pytorch.org/tutorials/beginner/basics/tensorqs_tutorial.html)\n","metadata":{"execution":{"iopub.status.busy":"2021-06-07T20:49:19.208593Z","iopub.execute_input":"2021-06-07T20:49:19.208975Z","iopub.status.idle":"2021-06-07T20:49:19.214249Z","shell.execute_reply.started":"2021-06-07T20:49:19.208941Z","shell.execute_reply":"2021-06-07T20:49:19.213019Z"}}},{"cell_type":"code","source":"# create feature and labels tensor for train set. As you remember we need variable to accumulate gradients. Therefore first we create tensor, then we will create variable\nfeaturesTrain = torch.from_numpy(features_train)\nlabelsTrain = torch.from_numpy(labels_train).type(torch.LongTensor) # data type is long - aka integer\n\n# create feature and targets tensor for validation set.\nfeaturesVal = torch.from_numpy(features_val)\nlabelsVal = torch.from_numpy(labels_val).type(torch.LongTensor) # data type is long - aka integer","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:16.985407Z","iopub.execute_input":"2021-06-14T16:59:16.985806Z","iopub.status.idle":"2021-06-14T16:59:17.013889Z","shell.execute_reply.started":"2021-06-14T16:59:16.985758Z","shell.execute_reply":"2021-06-14T16:59:17.012672Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Data Augmentation\n\nThe following cell is optional. With the model structure I use, I found that I get better accuracy without data augmentation. \n\nSee next section to enable/disable Data Augmentation.","metadata":{}},{"cell_type":"code","source":"from torchvision import transforms\n\ntrain_transforms = transforms.Compose([transforms.ToPILImage(),\n                                transforms.RandomRotation(degrees=(-10, 10)),\n                                transforms.RandomAffine(0, translate=(0.1,0.1)),\n                                transforms.ToTensor()])","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:17.015936Z","iopub.execute_input":"2021-06-14T16:59:17.016477Z","iopub.status.idle":"2021-06-14T16:59:17.194364Z","shell.execute_reply.started":"2021-06-14T16:59:17.016427Z","shell.execute_reply":"2021-06-14T16:59:17.193281Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Torch Dataset\n\nA custom Torch Dataset class must implement three functions: __init__, __len__, and __getitem__. \n\n__init__ is run once when creating the object. We initialize the object with data, labels, and transformations if we choose to do so. By default Transforms is set to None.\n","metadata":{}},{"cell_type":"code","source":"class custom_mnist(Dataset):\n    \n    def __init__(self, feat_tens, label_tens, transform=None):\n        self.data = feat_tens.reshape(len(feat_tens), 1, 28, 28)\n        self.label_data = label_tens\n        self.transform = transform\n        \n    def __len__(self):\n        return len(self.data)\n    \n    def __getitem__(self, index):\n        image = self.data[index]\n        label = self.label_data[index]\n        \n        if self.transform is not None:\n            image = self.transform(image)\n            \n        return image, label","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:17.197984Z","iopub.execute_input":"2021-06-14T16:59:17.198453Z","iopub.status.idle":"2021-06-14T16:59:17.206788Z","shell.execute_reply.started":"2021-06-14T16:59:17.198407Z","shell.execute_reply":"2021-06-14T16:59:17.205144Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here we are instantiating both the training and validation datasets. \n\nWe pass these datasets into Pytorch Dataloaders, which retrieve features and labels one batch at a time. The data is reshuffled every epoch so each batch is different through every epoch.","metadata":{}},{"cell_type":"code","source":"#creating the custom datasets - pass 3rd variable transform = train_transforms for data augmentation\ntrain_dataset = custom_mnist(featuresTrain, labelsTrain) \nval_dataset = custom_mnist(featuresVal, labelsVal)\n\n#Dataloaders\ntrain_loader = torch.utils.data.DataLoader(train_dataset, batch_size = my_params.batch_size, shuffle = True)\nval_loader = torch.utils.data.DataLoader(val_dataset) #contains all imgs in val_set\n\"\"\"we can only use the entire dataset in the dataloader because the images are very small and\ncan fit into memory. Will have to find workaround with larger image datasets\"\"\"\n\nprint(\"train_loader: {}\".format(len(train_loader)), \"\\nval_loader: {}\".format(len(val_loader)))","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:17.20965Z","iopub.execute_input":"2021-06-14T16:59:17.21017Z","iopub.status.idle":"2021-06-14T16:59:17.224486Z","shell.execute_reply.started":"2021-06-14T16:59:17.210098Z","shell.execute_reply":"2021-06-14T16:59:17.223499Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Visualizing Image\n\nHere we are retrieving one image and label from the training_dataset. If we instantiate the train_dataset again in the cell above we can see that the image created below will be different every time. ","metadata":{}},{"cell_type":"code","source":"# we can access and get data with index by __getitem__(index)\nimg, lab = train_dataset.__getitem__(3)\n\n#The images are getting loaded correctly\nprint(\"image_shape: \", img.shape)\nprint(\"image_label: \", lab)\nplt.imshow(np.squeeze(img))","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:17.225862Z","iopub.execute_input":"2021-06-14T16:59:17.22627Z","iopub.status.idle":"2021-06-14T16:59:17.433876Z","shell.execute_reply.started":"2021-06-14T16:59:17.226218Z","shell.execute_reply":"2021-06-14T16:59:17.432847Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Defining CNN Class\n\nFor image classification CNN Models are very effective.\n\nWe use the following layers in the CNN Model. \n\n- Convolutional Layers - Building blocks of CNNs and what do the \"heavy computation\"\n- Pooling Layers - Steps along image - reduces parameters and decreases likelihood of overfitting\n- Batch Normalization Layer - Scales down outliers, and forces NN to not relying too much on a Particular Weight\n- Dropout Layer - Regularization Technique that randomly drops a percentage of neurons to avoid overfitting (usually 20% - 50%)\n- Flatten Layer - Flattens the input as a 1D vector\n- Output Layer - Units equals number of classes (predicted probability of each class)\n- Linear Layer - Fully connected layer which performs a linear operation on the layer's input\n\n\nNote: I attempted to build the same model as I did in Tensorflow here --> [MNIST Tensorflow Notebook](https://www.kaggle.com/brendanartley/mnist-keras-cnn-99-6)","metadata":{}},{"cell_type":"code","source":"class NeuralNetwork(nn.Module):\n    def __init__(self):\n        \"\"\"super() function gives access to methods and properties of parent class\"\"\" \n        super(NeuralNetwork, self).__init__()\n        \n        self.features = torch.nn.Sequential(\n            #Layer 1: in_channel on first layer = number of color channels\n            nn.Conv2d(in_channels=1, out_channels=32, kernel_size=3, padding=1),\n            nn.ReLU(),\n\n            #Layer 2: Batch Normalization - num_features is input channels from previous layer\n            nn.BatchNorm2d(num_features=32),\n\n            #Layer 3: Conv2d\n            nn.Conv2d(in_channels=32, out_channels=32, kernel_size=3, padding=1),\n            nn.ReLU(),\n\n            #Layer 4: Batch Normalization\n            nn.BatchNorm2d(num_features=32),\n\n            #Layer 5: Max pooling Layer\n            nn.MaxPool2d(kernel_size = (2,2), stride=2),\n            nn.Dropout(0.25),\n\n            #Layer 6: Conv2d\n            nn.Conv2d(in_channels=32, out_channels=64, kernel_size=3, padding=1),\n            nn.ReLU(),\n\n            #Layer 7: Batch Normalization\n            nn.BatchNorm2d(num_features=64),\n\n            #Layer 8: Conv2d\n            nn.Conv2d(in_channels=64, out_channels=64, kernel_size=3, padding=1),\n            nn.ReLU(),\n\n            #Layer 9: Batch Normalization\n            nn.BatchNorm2d(num_features=64),\n\n            #Layer 10: Max pooling Layer\n            nn.MaxPool2d(kernel_size = (2,2), stride=2),\n            nn.Dropout(0.25),\n\n            #Layer 11: Flatten Layer\n            nn.Flatten(),\n\n            #Layer 12: Linear Layer - AKA dense layer in keras\n            nn.Linear(in_features=3136, out_features=512),\n\n            #Layer 13: Batch Normalization - 1d because we used a flatten layer\n            nn.BatchNorm1d(num_features=512),\n            nn.Dropout(0.25),\n\n            #Layer 14: Linear Layer\n            nn.Linear(in_features=512, out_features=1024),\n\n            #Layer 15: Batch Normalization\n            nn.BatchNorm1d(num_features=1024),\n            nn.Dropout(0.50),\n\n            #Layer 16: Linear Layer - Output Layer\n            nn.Linear(in_features=1024, out_features=10),\n            nn.ReLU(),\n            nn.Softmax(dim=1),\n            \n        )\n        \n    def forward(self, x):\n        logits = self.features(x)\n        return logits\n    \nNeuralNetwork()#printing the model to see layers","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:17.43553Z","iopub.execute_input":"2021-06-14T16:59:17.435942Z","iopub.status.idle":"2021-06-14T16:59:17.492916Z","shell.execute_reply.started":"2021-06-14T16:59:17.4359Z","shell.execute_reply":"2021-06-14T16:59:17.491523Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Instantiate Model, Loss, Optimizer, Variables\n\nIn the following cell I instantiate the model, loss function, optimizer, varaiables, and lists for the upcoming model training.\n\n","metadata":{}},{"cell_type":"code","source":"# Instantiate our model\nmodel = NeuralNetwork()\n# Define our loss function\ncriterion = nn.CrossEntropyLoss()\n# Define Optimizer\noptimizer = torch.optim.Adamax(model.parameters(), lr=0.0001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0.8)\n\nepochs = 40\nsteps = 0\ntrain_losses, val_losses, val_accs = [], [], []\nearly_stop_count = 0","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:17.494829Z","iopub.execute_input":"2021-06-14T16:59:17.495296Z","iopub.status.idle":"2021-06-14T16:59:17.527174Z","shell.execute_reply.started":"2021-06-14T16:59:17.495232Z","shell.execute_reply":"2021-06-14T16:59:17.526096Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Double checking wether we are training on GPU's or on CPU's.","metadata":{}},{"cell_type":"code","source":"if torch.cuda.is_available():\n    model = model.cuda()\n    criterion = criterion.cuda()\n    print(\"GPU Training\")\n    \nelse:\n    print(\"CPU Training\")","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:17.528768Z","iopub.execute_input":"2021-06-14T16:59:17.529311Z","iopub.status.idle":"2021-06-14T16:59:22.510822Z","shell.execute_reply.started":"2021-06-14T16:59:17.529218Z","shell.execute_reply":"2021-06-14T16:59:22.509736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Training Loop\n\nThis is where Pytorch really differs from Tensorflow. \n\nWe have to define the training loop in a much more pythonic way. I have provided comments throughout the following cell which explain the components of the training loop.\n\nI found the Pytorch Tutorials really helpful when building the training loop --> [Pytorch Tutorial](https://pytorch.org/tutorials/beginner/basics/optimization_tutorial.html)","metadata":{}},{"cell_type":"code","source":"print(\"---- Starting Model Training ----\\n\")\nfor e in range(epochs):\n    \n    #Early-Stopping - if training accuracy has not improved for a # of epochs, stop training\n    if early_stop_count >= 10:\n        print(\"\\nValidation Accuracy not improved for {} epochs. val_acc: {}.\".format(early_stop_count, sorted(val_accs)[-1]))\n        break\n            \n    print(\"\\n -- Epoch {}/{} -- \".format(e+1, epochs))\n    running_loss = 0\n    steps = 0\n    for images, labels in train_loader:\n        if torch.cuda.is_available(): #have to send to GPU if available\n            images = images.to(device)\n            labels = labels.to(device)\n        steps += 1\n        # Prevent accumulation of gradients\n        optimizer.zero_grad()\n        # Make predictions\n        log_ps = model(images)\n        loss = criterion(log_ps, labels)\n        #backprop\n        loss.backward()\n        optimizer.step()\n        \n        running_loss += loss.item()\n        if steps % len(train_loader) == 0: #on last batch, compute loss/acc\n            val_loss = 0\n            val_accuracy = 0\n            train_accuracy = 0\n\n            # Turn off gradients for validation\n            with torch.no_grad():\n                model.eval() #have to set model to eval to make predictions\n                \n                #validation validation\n                for images, labels in val_loader:\n                    if torch.cuda.is_available():\n                        images = images.to(device)\n                        labels = labels.to(device)\n                    log_ps = model(images)\n                    val_loss += criterion(log_ps, labels)\n\n                    ps = torch.exp(log_ps)\n                    # Get our top predictions\n                    top_p, top_class = ps.topk(1, dim=1)\n                    equals = top_class == labels.view(*top_class.shape)\n                    val_accuracy += torch.mean(equals.type(torch.FloatTensor))\n                \n                #training validation\n                for images, labels in train_loader:\n                    if torch.cuda.is_available():\n                        images = images.to(device)\n                        labels = labels.to(device)\n                    log_ps = model(images)\n\n                    ps = torch.exp(log_ps)\n                    # Get our top predictions\n                    top_p, top_class = ps.topk(1, dim=1)\n                    equals = top_class == labels.view(*top_class.shape)\n                    train_accuracy += torch.mean(equals.type(torch.FloatTensor))\n            \n            #set model to train before start of next epoch\n            model.train()\n\n            train_losses.append(running_loss/len(train_loader))\n            val_losses.append(val_loss/len(val_loader))\n            val_accs.append(val_accuracy/len(val_loader))\n            \n            #saving model if val_accuracy is best so far\n            if val_accs[-1] == sorted(val_accs)[-1]:\n                best_model_params = model.state_dict()\n                early_stop_count = 0\n                print(\"+\")\n                \n            #printing epoch stats\n            print(\"Training Loss: {:.4f} \".format(train_losses[-1]),\n                  \"Training Acc: {:.4f}\".format(train_accuracy/len(train_loader)),\n                  \"Valid Loss: {:.4f} \".format(val_losses[-1]),\n                  \"Valid Accuracy: {:.4f}\".format(val_accs[-1]))\n            \n            #early_stopping counter\n            early_stop_count += 1\n            \n            if e == (epochs-1):\n                print(\"\\nTop Validation Accuracy: {}\".format(sorted(val_accs)[-1]))","metadata":{"execution":{"iopub.status.busy":"2021-06-14T17:00:07.723984Z","iopub.execute_input":"2021-06-14T17:00:07.725027Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Restore Top Model Parameters\n\nIn the following cell we are restoring the model parameters that scored best on the validation dataset. See line 70 in the cell above to see how we save model parameters.","metadata":{}},{"cell_type":"code","source":"model.load_state_dict(best_model_params)","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:40.751942Z","iopub.status.idle":"2021-06-14T16:59:40.752398Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Data Preparation for Prediction\n\nNow that we have fully trained the model we need to instantiate the Test Images and make predictions on these images. ","metadata":{}},{"cell_type":"code","source":"#creating np array from test_data - and creating label array w/ same shape\nfinal_test_np = final_test.values/255\ntest_labels = np.zeros(final_test_np.shape)\n\n#creating torch tensor from the numpy arrays\ntest_tn = torch.from_numpy(final_test_np)\ntest_labels = torch.from_numpy(test_labels)","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:40.753676Z","iopub.status.idle":"2021-06-14T16:59:40.754321Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Again, we are creating a custom dataset and passing it through a dataloader to make predictions on the images.\n\nIf anyone with more Pytorch experience could weigh on the efficiency/effectiveness of this method of prediction please let me know!","metadata":{}},{"cell_type":"code","source":"class test_mnist(Dataset):\n    \n    def __init__(self, feat_tens):\n        self.data = feat_tens.reshape(len(feat_tens), 1, 28, 28)\n        \n    def __len__(self):\n        return len(self.data)\n    \n    def __getitem__(self, index):\n        # load image as ndarray type (Height * Width * Channels)\n        # Note: converting dtype to np.uint8 [Unsigned integer (0 to 255)]\n        # in this example, i don't use ToTensor() method of torchvision.transforms\n        # so you can convert numpy ndarray shape to tensor in PyTorch (H, W, C) --> (C, H, W)\n        image = self.data[index]\n            \n        return image\n    \n#Creating Dataset and Dataloader\nsubmission_dataset = test_mnist(test_tn)\nsubmission_loader = torch.utils.data.DataLoader(submission_dataset, batch_size = 256, shuffle = False)","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:40.755534Z","iopub.status.idle":"2021-06-14T16:59:40.756494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The following cell goes through all the images in the submission loader, and takes the class with the highest predicted probability. These are appended to a list, and converted to a pandas dataframe to be scored!","metadata":{}},{"cell_type":"code","source":"# Making it submission ready\nsubmission = [['ImageId', 'Label']]\n\n# Turn off gradients for validation\nwith torch.no_grad():\n    model.eval()\n    image_id = 1\n    for images in submission_loader:\n        if torch.cuda.is_available():\n            images = images.to(device)\n        log_ps = model(images)\n        ps = torch.exp(log_ps)\n        top_p, top_class = ps.topk(1, dim=1)\n        \n        for prediction in top_class:\n            submission.append([image_id, prediction.item()])\n            image_id += 1\n            \nsubmission_df = pd.DataFrame(submission)\nsubmission_df.columns = submission_df.iloc[0]\nsubmission_df = submission_df.drop(0, axis=0)\n\nsubmission_df.to_csv(\"submission.csv\", index=False)","metadata":{"execution":{"iopub.status.busy":"2021-06-14T16:59:40.758066Z","iopub.status.idle":"2021-06-14T16:59:40.758822Z"},"trusted":true},"execution_count":null,"outputs":[]}]}