{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Dogs Vs Cats with Torch\n\nI made this notebook to learn Pytorch,since I’m moving from Tensorflow to PyTorch.","metadata":{}},{"cell_type":"markdown","source":"# Imports\n\n* `zipfile` - To extract the train and test images from ZIP files\n* `time` - Save the timestamp when some metric was save on disk\n* `os` - Useful for navigate through the files\n* `random` - Select a random number for a seed\n* `numpy` - For shuffling and math in general\n* `pandas` - Create submission CSV\n* `shutil` - Move images\n* `PIL.Image` - Load evaluation images\n* `collections` - Create nested dictionaries\n* `tqdm` - Fancy progressbars\n* `torch.utils.data.DataLoader` - Create batches in an easy way\n* `torch.utils.data.Dataset` - Create a custom Dataset for evaluation\n* `torch.utils.data.sampler.SubsetRandomSampler` - Choose samples from a subset of indices\n* `torchvision.datasets` - Load images and labels from a root directory\n* `torchvision.transforms` - Apply transformations on a given dataset\n* `torchvision.models` - Load pretrained models\n* `torchvision.utils.makegrid` - Plot multiple images\n* `torch` - General methods from Pytorch\n* `torch.nn` - Modules to build neural net layers\n* `torch.nn.functional` - Methods like activation functions\n* `torch.optim` - Optimizers\n* `matplotlib.pyplot` - General ploting\n* `matplotlib.style` - Change plots style","metadata":{}},{"cell_type":"code","source":"import zipfile\nimport time\nimport os\nimport random\nimport numpy as np\nimport pandas as pd\nimport shutil\nfrom PIL import Image\nimport collections\nfrom tqdm import tqdm\nfrom torch.utils.data import DataLoader, Dataset\nfrom torch.utils.data.sampler import SubsetRandomSampler\nfrom torchvision import datasets, transforms, models\nfrom torchvision.utils import make_grid\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nimport matplotlib.pyplot as plt\nfrom matplotlib import style\n%matplotlib inline","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"If we re-run without restart the kernel the `metrics.log` keeps on disk and will accumulate with the metrics from the last run.\n\nTo prevent that let's remove it","metadata":{}},{"cell_type":"code","source":"%rm '/kaggle/working/metrics.log'","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Choose the device to run the model on training/validation/evaluation","metadata":{}},{"cell_type":"code","source":"device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# DataLoader\n\nUnzip the train and test sets to `/kaggle/working/` directory","metadata":{}},{"cell_type":"code","source":"# Extrair os ZIPs\nwith zipfile.ZipFile(\"../input/dogs-vs-cats/train.zip\", \"r\") as unzip:\n    unzip.extractall(\".\")\n\n    \nwith zipfile.ZipFile(\"../input/dogs-vs-cats/test1.zip\", \"r\") as unzip:\n    unzip.extractall(\".\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Create another directory with 2 folders inside (1 folder to each class/label). If they exist it will not create again.","metadata":{}},{"cell_type":"code","source":"os.makedirs(\"my_train/dogs\", exist_ok=True)\nos.makedirs(\"my_train/cats\", exist_ok=True)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here the images from `train` are moved to the correct label folder inside `my_train`.\n\nThe images name start with the correct label, so it's easy to know what is the correct folder to move","metadata":{}},{"cell_type":"code","source":"root = \"train\"\nimgs = os.walk(root).__next__()[2]\nfolders = {\n    \"cat\": \"my_train/cats\",\n    \"dog\": \"my_train/dogs\"\n}\n\nfor img in tqdm(imgs):\n    label = img.split(\".\")[0]\n    \n    old_path = os.path.join(root, img)\n    new_path = os.path.join(folders[label], img)\n    \n    shutil.move(old_path, new_path)\n    \n    ","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Here the images from `my_train` are transformed and split into 2 datasets: `train` and `val` - one for training, another one for validation (during the training).\n\nProbably you are asking: \"Why you put those specific values on the Normalize method?\" - When we use pretrained ImageNet datasets, those values was estimated by them using millions of images, so it seems a good estimation.\n\nAnd why 2 arrays with 3 values each?\n\nThe first array  is for the mean and the second array is for the standard deviation. And we use 3 values, because the images are normalized in-depth (in the channels dimension) and we have RGB images, so we have 3 channels (1 value for each channel).","metadata":{}},{"cell_type":"code","source":"# Image size\nIMG_SIZE = 260\n\n# Transformations\ndata_transforms = transforms.Compose([\n    transforms.Resize((IMG_SIZE, IMG_SIZE)),\n    transforms.ToTensor(),\n    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),\n    transforms.RandomHorizontalFlip(0.5)\n])\n\n# Load the images and labels\ndataset = datasets.ImageFolder(root=\"my_train\", transform=data_transforms)\n\n# Get the name for each numeric label\nclasses = dataset.classes\n\n# 20% of the train dataset will be used for\n# validation\nval_split = 0.2\ndataset_size = len(dataset)\nidxs = list(range(dataset_size))\n\n# Number of validation images\nsplit = int(val_split * dataset_size)\n\n# Set a random seed and shuffle the ids\nnp.random.seed(random.randint(0, 99999))\nnp.random.shuffle(idxs)\n\ndataset_size = {\n    \"train\": dataset_size - split,\n    \"val\": split\n}\n\n# Set data samplers to select N unique images for\n# each dataset\ntrain_sampler = SubsetRandomSampler(idxs[:-split])\nval_sampler = SubsetRandomSampler(idxs[-split:])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Split the dataset in 2 dataloaders and each dataloader will get a subset of unique images, divided in batches.","metadata":{}},{"cell_type":"code","source":"dataloaders = {\n    \"train\": DataLoader(dataset, batch_size=64, sampler=train_sampler),\n    \"val\": DataLoader(dataset, batch_size=64, sampler=val_sampler),\n}","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Just a simple function to show 4 images from the train dataloader.\n\nSome people don't understand why we need this lines:\\\n`mean = np.array([0.485, 0.456, 0.406])\nstd = np.array([0.229, 0.224, 0.225])\ninp = std * inp + mean\ninp = np.clip(inp, 0, 1)`\n\nIf you're reading the markdown cells, you know that we use the mean and the standar deviaton to normalize the images. So the first 2 lines are explained. Now, why `inp` is `std * inp + mean` ?\n\nIn statistics we have a thing called ***Standard Scores*** or ***Z-score*** and we can use it to normalize values (only if we know the mean and std of a population). The expression is:\\\n$\\frac{X - \\mu}{\\sigma}$, where $X$ is our image, $\\mu$ the mean and $\\sigma$ the std.\n\nMore info about that [here](https://en.wikipedia.org/wiki/Standard_score)\n\nSo our images were normalized with that expression and matplotlib will not plot the image as we expect, because the pixel values are normalized. To \"unnormalize\" we need to reverse the ***z-score*** expression:\\\n$X * \\sigma + \\mu$\n\nAs you can see we reverse all the operations from the previous expression, and that converted in code is:\\\n`std * inp + mean`\n\nNow, what the `clip()` method does? Clip will grab all the values and translate them into an interval, in this case: [0,1] - If during the \"unnormalize\" some values were < 0 or > 1, they're transformed to a value close to 0 (if value < 0) or close to 1 (if value > 1)","metadata":{}},{"cell_type":"code","source":"def imshow(inp, title=None):\n    inp = inp.numpy().transpose((1,2,0))\n    mean = np.array([0.485, 0.456, 0.406])\n    std = np.array([0.229, 0.224, 0.225])\n    inp = std * inp + mean\n    inp = np.clip(inp, 0, 1)\n    plt.grid(False)\n    plt.imshow(inp)\n    \n    if title != None:\n        plt.title(title)\n    \n    plt.pause(0.001)\n\n    \nfeatures, labels = next(iter(dataloaders[\"train\"]))\nfeatures = features[:4]\nlabels = labels[:4]\nout = make_grid(features)\n\nimshow(out, title=[classes[x] for x in labels])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Test DataLoader\nThis custom dataset just grab all file names on `test1` directory and when we loop through it it will read each image, apply transformations (if we passed them) and return the image and the name of the image (it will be usefull when creating the submission CSV)","metadata":{}},{"cell_type":"code","source":"class TestDataset(Dataset):\n    \n    def __init__(self, root_dir, transform=None):\n        self.root_dir = root_dir\n        self.paths = os.walk(root_dir).__next__()[2]\n        self.transform = transform\n    \n    def __len__(self):\n        return len(self.paths)\n    \n    def __getitem__(self, idx):\n        \n        if torch.is_tensor(idx):\n            idx = idx.tolist()\n        \n        img_path = os.path.join(self.root_dir, self.paths[idx])\n        \n        img = Image.open(img_path)\n        \n        if self.transform != None:\n            img = self.transform(img)\n        \n        return img, self.paths[idx].split(\".\")[0]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Create a dataloader with the custom dataset. The transforms are the same, except the `RandomFlip` that was excluded because the images shouldn't be transformed","metadata":{}},{"cell_type":"code","source":"test_dataset = TestDataset(root_dir=\"./test1\", transform=transforms.Compose([\n    transforms.Resize((IMG_SIZE, IMG_SIZE)),\n    transforms.ToTensor(),\n    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])\n]))\n\ntest_dataloader = DataLoader(test_dataset, batch_size=64)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"imgs, _ = next(iter(test_dataloader))\nimgs = imgs[:4]\nout = make_grid(imgs)\nimshow(out)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Train Function","metadata":{}},{"cell_type":"markdown","source":"The most important function, here the train and validation happens","metadata":{}},{"cell_type":"code","source":"def train(model, loss_fn, optimizer, num_epochs=5, model_name=\"model\", lr_scheduler=None):\n    \n    for epoch in range(num_epochs):\n        print(\"Epoch {}/{}\\n\".format(epoch+1, num_epochs))\n        \n        for step in dataloaders:\n            \n            # Set all layers to trainning=True\n            if step == \"train\":\n                model.train()\n            \n            #Set all layers to trainning=False\n            else:\n                model.eval()\n            \n            # Loss\n            l = 0\n            # Accuracy\n            acc = 0\n            \n            for X_batch, y_batch in tqdm(dataloaders[step]):\n                X_batch, y_batch = X_batch.to(device), y_batch.to(device)\n                \n                # Zero gradient, each batch should have your own gradient\n                optimizer.zero_grad()\n                \n                # Training\n                # Gradients set to True in all layers\n                with torch.set_grad_enabled(step == \"train\"):\n                    preds = model(X_batch)\n                    \n                    # max return a tuple with the max values and their indices\n                    # I only want the indices because CrossEntropy returns an Tensor\n                    # with all the unnormalized probabilities from each label\n                    # The indice from the highest prob is the predicted label\n                    _, max_idxs = torch.max(preds, dim=1)\n                    \n                    \n                    # Calculate errors\n                    loss = loss_fn(preds, y_batch)\n                    \n                    # Real train right here\n                    if step == \"train\":\n                        ## L2 regularization \n                        ## (doesn't worth it with my own model)\n                        #l2_factor = 0.005\n                        #l2_reg = l2_factor * np.sum([(w**2).sum() for w in model.parameters()])\n                        #loss = loss + l2_reg\n                        \n                        # Backprop + Weighs update\n                        loss.backward()\n                        optimizer.step()\n                    \n                # Running metrics\n                l += loss.item() * X_batch.size(0)\n                acc += torch.sum(max_idxs == y_batch.data)\n                \n                # Write on file the batch metrics\n                with open(model_name+\".log\", \"a\") as f:\n                    f.write(\"{},{},{},{}\\n\".format(\n                        round(time.time(), 3),\n                        step,\n                        torch.sum(max_idxs == y_batch.data).item() / y_batch.size(0),\n                        loss.item()\n                    ))\n                \n            # Calculate the average metrics per epoch\n            epoch_loss = l / dataset_size[step]\n            epoch_acc = acc.double() / dataset_size[step]\n            \n            # Learning rate update\n            # We can change the learning rate in each epoch\n            # using learning rate scheduler\n            if step == \"train\" and lr_scheduler != None:\n                lr_scheduler.step()\n            \n            print(\"{} Acc: {:.3f} - Loss: {:.3f}\".format(step, epoch_acc, epoch_loss))\n        print()\n    \n    return model","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Custom Model\n\nHere is my test cell, where I tried to wrote some custom models, but they din't work well. \n\n**You can skip to the next cell, because this model will not be used.**","metadata":{}},{"cell_type":"code","source":"class CNN(nn.Module):\n    def __init__(self):\n        \n        super().__init__()\n        \n        self.conv1 = nn.Conv2d(3, 64, 5, stride=(2,2))\n        self.res_conv1 = None\n        self.conv2 = nn.Conv2d(64, 64, 3, padding=3//2)\n        self.conv3 = nn.Conv2d(64, 64, 3, padding=3//2)\n        self.res_conv3 = None\n        self.conv4 = nn.Conv2d(64, 64, 3, padding=3//2)\n        self.conv5 = nn.Conv2d(64, 64, 3, padding=3//2)\n        \n         \n        \n        x = torch.randn(3, IMG_SIZE, IMG_SIZE).view(-1, 3, IMG_SIZE, IMG_SIZE)\n        self.get_flatten = None\n        \n        # Get number of parameters in the last conv. layer\n        self.forward_convolutions(x)\n        \n        self.dropout = nn.Dropout(0.5)\n        \n        self.fc1 = nn.Linear(self.get_flatten, 2)\n    \n    def forward_convolutions(self, X):\n        X = F.relu( self.conv1(X) )\n        self.res_conv1 = X\n        \n        X = F.relu( self.conv2(X) )\n        X = F.relu( self.conv3(X) )\n        X = self.res_conv1 + X\n        self.res_conv3 = X\n        \n        X = F.relu( self.conv4(X) )\n        X = F.relu( self.conv5(X) )\n        X = self.res_conv3 + X\n        \n        \n        # If we are trying to find the number of\n        # parameters in the last convolution\n        if self.get_flatten == None:\n            self.get_flatten = 1\n\n            for sz in X.size()[1:]:\n                self.get_flatten *= sz\n        \n        \n        return X\n    \n    def forward(self, X):\n        X = self.forward_convolutions(X)\n        \n        X = X.view(-1, self.get_flatten)\n        X = self.dropout(X)\n        \n        return self.fc1(X)\n\n    \n    def calc_outputs(self):\n        # Calculate the output size for\n        # each Conv2d layer\n        \n        conv_idx = 1\n        \n        out_h_prev = out_w_prev = IMG_SIZE\n        \n        for layer in self.children():\n            if isinstance(layer, nn.Conv2d):\n                inp = (out_h_prev, out_w_prev)\n                out = layer.out_channels\n                k = layer.kernel_size\n                s = layer.stride\n                p = layer.padding\n\n                out_h = ( ( inp[0] + (2*p[0]) - k[0] ) // s[0] ) + 1\n                out_w = ( ( inp[1] + (2*p[1]) - k[1] ) // s[1] ) + 1 \n                \n                print(\"Output Convolution Layer {} = ({},{})\".format(\n                    conv_idx,\n                    out_h//2, # convs are divided by 2 because they're all pooled by a 2x2 pool window\n                    out_w//2  # with 2x2 stride, making the size of the feature map being half\n                ))\n                \n                out_h_prev, out_w_prev = out_h, out_w\n                \n                conv_idx += 1","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Just a test for checking the final output of our feature extractor**","metadata":{}},{"cell_type":"code","source":"#test = CNN()\n\n#test.calc_outputs()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n# Train","metadata":{}},{"cell_type":"markdown","source":"Here I tested different parameters for my own model and for the resnet18 pretrained model.\n\nI realize:\n* Dropout with 50% of the neurons temporarily \"dead\" decrease the validation loss\n* Adam optimizer starting with 0.001 learning rate converge pretty fast\n* Step learning rate with gamma = 5 and only change the learning rate on the 5th epoch, helps to increase the accuracy and decrease the loss of the validation set","metadata":{}},{"cell_type":"code","source":"## After some tests I found that 10 epochs are enough for the resnet18 model\nmodel_names = {\n    #\"model_cnn_3conv,3k,1epoch\": 1,\n    #\"model_cnn_3conv,3k,3epoch\": 3,\n    #\"model_resnet18,5epoch\": 5,\n    \"model_resnet18,10epoch\": 10,\n}\n\nfor k,v in model_names.items():\n    ## Transfer Learning\n    model = models.resnet18(pretrained=True)\n    n_features = model.fc.in_features\n    \n    # Freeze layers, except our new Linear layer\n    for layer in model.parameters():\n        layer.requires_grad = False\n    \n    model.fc = nn.Sequential(collections.OrderedDict([\n        (\"fc_dropout1\", nn.Dropout(0.5)),\n        (\"fc_softmax\", nn.Linear(n_features, 2))\n    ]))\n    model = model.to(device)\n    \n    \n    ## Own Model\n    #model = CNN().to(device)\n    \n    # Loss function\n    loss_fn = nn.CrossEntropyLoss()\n\n    # Optimizer\n    optimizer = optim.Adam(model.parameters(), lr=0.001)\n\n    ## Decay LR by a factor of 5 every 5 epochs\n    exp_lr_scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=5, gamma=5)\n    model = train(model, loss_fn, optimizer, num_epochs=v, \n                  model_name=k, lr_scheduler=exp_lr_scheduler)\n    \n    ## When we have mutliple model_names it's good to delete\n    ## the model object, to prevent using the previous trained\n    ## weights\n    #del model","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Visualize metrics","metadata":{}},{"cell_type":"markdown","source":"Here starts a big dictionary is created to save all the train and test accuracies and losses (those that we save in a file during the train/validation.","metadata":{}},{"cell_type":"code","source":"nested_dict = lambda: collections.defaultdict(nested_dict)\n\nmodels_metrics = nested_dict()\n\nfor k,v in model_names.items():\n\n    train_accs = []\n    train_losses =[]\n    val_accs = []\n    val_losses = []\n    losses_removed = 0\n    with open(k+\".log\") as f:\n        for line in f:\n            acc = line.split(\",\")[2]\n            loss = line.split(\",\")[3]\n            \n            # The model goes really bad with some batches\n            # To keep the plot with values between 0 and 1\n            # I delete all the losses greater than 1\n            if float(loss) > 1.0:\n                losses_removed += 1\n                continue\n                \n\n            if \"train\" in line:\n                train_accs.append(float(acc))\n                train_losses.append(float(loss))\n            else:\n                val_accs.append(float(acc))\n                val_losses.append(float(loss))\n    \n    models_metrics[k][\"train_accs\"] = train_accs\n    models_metrics[k][\"train_losses\"] = train_losses\n    models_metrics[k][\"val_accs\"] = val_accs\n    models_metrics[k][\"val_losses\"] = val_losses","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Smooth functions","metadata":{}},{"cell_type":"markdown","source":"Because the batch metrics can be really instable, I tried to implement 2 methods to smooth the plotted lines.\n\nThe **Hanning Window** is a function used in signal processing to smooth values. It is defined as:\\\n$\\frac{1}{2} - \\frac{1}{2}cos(\\frac{2\\pi n}{M-1})$, where $n$ is the current value and $M$ is the total of values.\n\nIf we multiply the result by the current value, it should be smoothed (Numerically it becomes smaller, but visually the difference is almost imperceptible).\n\n\nThe **Moving Average** basically is a mean that use all the previous values to compute it. So when we reach the last value, the first value will not impact much in the average. I tried to implement the version used in Tensorboard, but it doesn't work (probably because I need to set the exponents for each value). So I just gave up and I used the hanning window.","metadata":{}},{"cell_type":"code","source":"def hanning_window(metric):\n    for i in range(len(metric)):\n        n = metric[i]\n        h_metric = 0.5 - 0.5 * np.cos( (2*np.pi*n) / (len(metric)-1) )\n        metric[i] *= n\n\n    return metric\n\n\ndef moving_average(metric, w):\n    \n    last_smooth = metric[0]\n    smoothed_metric = []\n    \n    for i, p in enumerate(metric):\n        if i == 0:\n            smooth = p\n        else:\n            smooth = last_smooth * w + (1 - w) + p \n        \n        smoothed_metric.append( smooth )\n        last_smooth = smooth\n        \n    return smoothed_metric","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Compute the for each metric and transform the metric saved values using the Hanning Window ","metadata":{}},{"cell_type":"code","source":"for k,v in model_names.items():\n    \n    mean_train_acc, mean_train_loss = np.mean(models_metrics[k][\"train_accs\"]), np.mean(models_metrics[k][\"train_losses\"])\n    mean_val_acc, mean_val_loss = np.mean(models_metrics[k][\"val_accs\"]), np.mean(models_metrics[k][\"val_losses\"])\n    \n    models_metrics[k][\"mean_train_acc\"] = mean_train_acc\n    models_metrics[k][\"mean_train_loss\"] = mean_train_loss\n    models_metrics[k][\"mean_val_acc\"] = mean_val_acc\n    models_metrics[k][\"mean_val_loss\"] = mean_val_loss\n\n    \n        \n    models_metrics[k][\"train_accs\"] = hanning_window(models_metrics[k][\"train_accs\"])\n    models_metrics[k][\"train_losses\"] = hanning_window(models_metrics[k][\"train_losses\"])\n    models_metrics[k][\"val_accs\"] = hanning_window(models_metrics[k][\"val_accs\"])\n    models_metrics[k][\"val_losses\"] = hanning_window(models_metrics[k][\"val_losses\"])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now I used the big dictionary to plot train accuracy/loss and validation accuracy/loss ","metadata":{}},{"cell_type":"code","source":"style.use(\"ggplot\")\n\nfor k,v in model_names.items():\n    f, (ax1, ax2) = plt.subplots(2, 1, figsize=(12, 6))\n\n    ax1.set_title(k)\n    \n    ax1.plot(np.arange(len(models_metrics[k][\"train_accs\"])), models_metrics[k][\"train_accs\"], label=\"Acc - Mean: {:.2f}\".format(models_metrics[k][\"mean_train_acc\"]))\n    ax1.plot(np.arange(len(models_metrics[k][\"train_losses\"])), models_metrics[k][\"train_losses\"], label=\"Loss - Mean: {:.2f}\".format(models_metrics[k][\"mean_train_loss\"]))\n\n    ax1.legend()\n    \n    ax2.plot(np.arange(len(models_metrics[k][\"val_accs\"])), models_metrics[k][\"val_accs\"], label=\"Val Acc - Mean: {:.2f}\".format(models_metrics[k][\"mean_val_acc\"]))\n    ax2.plot(np.arange(len(models_metrics[k][\"val_losses\"])), models_metrics[k][\"val_losses\"], label=\"Val Loss - Mean: {:.2f}\".format(models_metrics[k][\"mean_val_loss\"]))\n    ax2.legend()\n    \n    \n    # Save the plot as image to analyze and compare with\n    # other tests I made\n    plt.savefig(k+\".png\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Evaluate","metadata":{}},{"cell_type":"markdown","source":"Here I just used the test dataloader to predict the label from each image and place the image name as key and the predicted label as value of a dictionary","metadata":{}},{"cell_type":"code","source":"\npred_dict = dict()\n\nwith torch.no_grad():\n    for X_batch,names in tqdm(test_dataloader):\n        X_batch = X_batch.to(device)\n        \n        preds = model(X_batch)\n        \n        _, ys = torch.max(preds, dim=1)\n        \n        for y, name in zip(ys, names):\n            pred_dict[int(name)] = y.item()\n    ","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"With the dictionary a DataFrame is created using the submission structure","metadata":{}},{"cell_type":"code","source":"df_test = pd.DataFrame(pred_dict.items(), columns=[\"id\", \"label\"])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Save the datatframe as a CSV file","metadata":{}},{"cell_type":"code","source":"df_test.to_csv(\"submission.csv\", index=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}