{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<a id=\"toc\"></a>\n# Table of content\n\n* [Introduction](#introduction)\n* [Setup, imports, constants](#sic)\n    * [Setting up the environment](#sic.env)\n    * [Imports](#sic.imports)\n    * [Constants](#sic.consts)\n* [EDA (Exploratory Data Analysis)](#EDA)\n* [Model](#model)\n    * [Dataloaders](#model.dls)\n    * [CNN learner](#model.cnn)\n    * [Fine tuning the model](#model.finetune)\n* [Make predictions](#predictions)\n* [Conclusion](#conclusion)","metadata":{}},{"cell_type":"markdown","source":"<a id=\"introduction\"></a>\n# Introduction\n[[back to top]](#toc)","metadata":{}},{"cell_type":"markdown","source":"In this notebook I'm implementing a convolutional neural network which is capable of classifing pictures of dogs by their breeds. The source of the data is the competition named [Dog Breed Identification](https://www.kaggle.com/c/dog-breed-identification/overview/description). The aim of this notebook is to setup a convolutional neural network in the simplest possible way using [fast.ai](https://fast.ai), hence there are no optimizations in the code. The aim is not to create a model which is capable of perdicting the breeds with state-of-the-art accuracy but to give a high level overview of how to easily setup a convolutional neural network and use it to make predictions.\n\nIn the [first chapter](#sic) the environment is set, the required libraries are imported and the constants are defined.\n\n[Next](#EDA), a basic exploration of the provided input is performed.\n\nThe model is created and trained in the [third section](#model). First, the data is loaded in the necessary format, then the model is initialized with weights from ResNet. Then the fine-tuning takes place based on the input images.\n\nThe [fourth chapter](#predictions) describes how to make predictions based on the test data and how to save the outcome in the expected format.","metadata":{}},{"cell_type":"markdown","source":"<a id=\"sic\"></a>\n# Setup, imports, constants\n[[back to top]](#toc)","metadata":{}},{"cell_type":"markdown","source":"This chapter consists of three sections. The [first](#sic.env) describes how to setup the environment, what libraries to install, etc. The [second](#sic.imports) explains what to import and why. The [last subchapter](#sic.consts) introduces the constants used throughout this notebook.","metadata":{}},{"cell_type":"markdown","source":"<a id=\"sic.env\"></a>\n## Setting up the environment\n[[back to top]](#toc)","metadata":{}},{"cell_type":"markdown","source":"This notebook uses fast.ai 2.5.2 and for some reason it fails to work together with PyTorch 1.9.1. The problem is that certain operations throw the following error: `RuntimeError: solve: MAGMA library not found in compilation. Please rebuild with MAGMA.` As of october 2021 one work-around is to downgrade PyTorch from the latest (1.9.1) to 1.9.0. See [this discussion](https://www.kaggle.com/product-feedback/279990) for further details.\n\nThe following cell performs the downgrading of PyTorch and some related packages. Torchvision, torchaudio and torchtext needs to be downgraded to prevent having conflicting versions installed.","metadata":{}},{"cell_type":"code","source":"!pip install --user torch==1.9.0 torchvision==0.10.0 torchaudio==0.9.0 torchtext==0.10.0","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:38:16.128940Z","iopub.execute_input":"2021-11-03T08:38:16.129315Z","iopub.status.idle":"2021-11-03T08:39:38.796596Z","shell.execute_reply.started":"2021-11-03T08:38:16.129225Z","shell.execute_reply":"2021-11-03T08:39:38.795361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"sic.imports\"></a>\n## Imports\n[[back to top]](#toc)","metadata":{}},{"cell_type":"markdown","source":"One of the comfortable aspects of fast.ai is that it is made really easy to import the necessary packages, all we need to do is import everything from their data & vision packages as the first two lines of codes shows.\n\nImporting panda, os & stats etc. is necessary to have a better insights into our data. See the [EDA chapter](#EDA).","metadata":{}},{"cell_type":"code","source":"from fastai.data.all import *\nfrom fastai.vision.all import *\n\nimport numpy as np\nimport pandas as pd\nimport os\n\nfrom functools import cmp_to_key\nfrom PIL import Image\nfrom scipy import stats\n\nprint(\"Import finished.\")","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:39:38.799377Z","iopub.execute_input":"2021-11-03T08:39:38.799774Z","iopub.status.idle":"2021-11-03T08:39:41.633976Z","shell.execute_reply.started":"2021-11-03T08:39:38.799724Z","shell.execute_reply":"2021-11-03T08:39:41.632892Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"sic.consts\"></a>\n## Constants\n[[back to top]](#toc)","metadata":{}},{"cell_type":"markdown","source":"The constants mostly store paths such as path to the working folder (`path`), path to the csv storing the labels (`labels_csv_path`) and the path to the train folder (`train_path`) & test folder (`test_path`).\n\nThe number of epochs is also set here (`number_of_epochs`).","metadata":{}},{"cell_type":"code","source":"# paths\npath = '../input/dog-breed-identification/'\nlabels_csv_path = path + 'labels.csv'\nsample_submission_csv_path = path + 'sample_submission.csv'\nsubmission_csv_path =  './submission.csv'\ntrain_path = path + 'train'\ntest_path = path + 'test'\n\nnumber_of_epochs = 10\n\nprint(f'Constants are set. Fine tuning takes {number_of_epochs} epochs.')","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:39:41.636408Z","iopub.execute_input":"2021-11-03T08:39:41.637179Z","iopub.status.idle":"2021-11-03T08:39:41.650312Z","shell.execute_reply.started":"2021-11-03T08:39:41.637128Z","shell.execute_reply":"2021-11-03T08:39:41.648949Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"EDA\"></a>\n# EDA (Exploratory Data Analysis)\n[[back to top]](#toc)","metadata":{}},{"cell_type":"markdown","source":"The aim of any exploratory data analysis is to have a better understanding of the data to which the model needs to be fit, e.g. understand the shape/size/amount/distribution etc. of the input. Obviously, the analysis depends on the task at hand (what we want to get out of the model i.e.: image classification in this case) and on the model which we plan to deploy.\n\nThe very first step is to check the [data section](https://www.kaggle.com/c/dog-breed-identification/data) of the competition.\n\nAfter that, I'm checking the content of the `labels.csv` using pandas.","metadata":{}},{"cell_type":"code","source":"labels_df = pd.read_csv(labels_csv_path)\nprint(f'The shape of the labels: {labels_df.shape}')","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:39:41.654259Z","iopub.execute_input":"2021-11-03T08:39:41.655455Z","iopub.status.idle":"2021-11-03T08:39:41.707193Z","shell.execute_reply.started":"2021-11-03T08:39:41.655361Z","shell.execute_reply":"2021-11-03T08:39:41.706142Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Pandas' `read_csv` reads everything from the given file (`labels.csv` in this case) and stores the content of the csv file in a so called dataframe. Then, we check the shape (number of rows & number of columns) of the dataframe.","metadata":{}},{"cell_type":"code","source":"labels_df.head","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:39:41.709057Z","iopub.execute_input":"2021-11-03T08:39:41.709645Z","iopub.status.idle":"2021-11-03T08:39:41.741067Z","shell.execute_reply.started":"2021-11-03T08:39:41.709581Z","shell.execute_reply":"2021-11-03T08:39:41.739551Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The `head` prints the first & last five entries of the dataframe. The first column contains the unique ID, the other column stores the breed of the dog on the given picture. Next, I'm checking the number of pictures in the train folder...","metadata":{}},{"cell_type":"code","source":"print(len(os.listdir(train_path)))","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:39:41.743983Z","iopub.execute_input":"2021-11-03T08:39:41.744835Z","iopub.status.idle":"2021-11-03T08:39:42.744943Z","shell.execute_reply.started":"2021-11-03T08:39:41.744770Z","shell.execute_reply":"2021-11-03T08:39:42.743517Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"...which is (unsurprisingly) 10222 just like in the `label.csv` file.","metadata":{}},{"cell_type":"markdown","source":"The number of images in the test folder is...","metadata":{}},{"cell_type":"code","source":"print(len(os.listdir(test_path)))","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:39:42.746961Z","iopub.execute_input":"2021-11-03T08:39:42.747680Z","iopub.status.idle":"2021-11-03T08:39:43.675183Z","shell.execute_reply.started":"2021-11-03T08:39:42.747614Z","shell.execute_reply":"2021-11-03T08:39:43.674046Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"That's how many items needs to be predicted.","metadata":{}},{"cell_type":"markdown","source":"Next, I'm checking the number of breeds (the unique occurrences in the breed column):","metadata":{}},{"cell_type":"code","source":"unique_breeds = pd.unique(labels_df['breed'])\nprint(len(unique_breeds))","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:39:43.677628Z","iopub.execute_input":"2021-11-03T08:39:43.678559Z","iopub.status.idle":"2021-11-03T08:39:43.695795Z","shell.execute_reply.started":"2021-11-03T08:39:43.678492Z","shell.execute_reply":"2021-11-03T08:39:43.694354Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It's 120, just like it's written in the [data section](https://www.kaggle.com/c/dog-breed-identification/data).","metadata":{}},{"cell_type":"code","source":"print(stats.describe(labels_df.value_counts('breed')))","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:39:43.697663Z","iopub.execute_input":"2021-11-03T08:39:43.698269Z","iopub.status.idle":"2021-11-03T08:39:43.716321Z","shell.execute_reply.started":"2021-11-03T08:39:43.698220Z","shell.execute_reply":"2021-11-03T08:39:43.714953Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Dataframe's `value_counts` returns the number of images for each breed. The code snippet above returns some pieces of statistic. For example the breed with the smallest number of images has 66 images, the one with the largest has 126, the mean is 85, etc. This data can be important and might require further investigation depending on the model deployed.","metadata":{}},{"cell_type":"markdown","source":"Next, I would like to know if the images are the same size, so I pick two randomly and check...","metadata":{}},{"cell_type":"code","source":"a_img_path = '../input/dog-breed-identification/train/000bec180eb18c7604dcecc8fe0dba07.jpg'\nb_img_path = '../input/dog-breed-identification/train/002a283a315af96eaea0e28e7163b21b.jpg'\n\na_img = Image.open(a_img_path)\nb_img = Image.open(b_img_path)\n\nprint(a_img.size)\nprint(b_img.size)","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:39:43.722770Z","iopub.execute_input":"2021-11-03T08:39:43.723106Z","iopub.status.idle":"2021-11-03T08:39:43.763260Z","shell.execute_reply.started":"2021-11-03T08:39:43.723073Z","shell.execute_reply":"2021-11-03T08:39:43.760887Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"... it turns out that the dimensions might differ. It is important, because certain models require the input images to be the same size, therefore it will be necessary to resize them.","metadata":{}},{"cell_type":"markdown","source":"Lastly, I'm curious to understand the content of `sample_submission.csv`.","metadata":{}},{"cell_type":"code","source":"sample_df = pd.read_csv(sample_submission_csv_path)\nsample_df.shape","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:39:43.765564Z","iopub.execute_input":"2021-11-03T08:39:43.766049Z","iopub.status.idle":"2021-11-03T08:39:44.555820Z","shell.execute_reply.started":"2021-11-03T08:39:43.765992Z","shell.execute_reply":"2021-11-03T08:39:44.554542Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The number of rows is 10357, so apparently there is an entry for each image in the test folder. The number of columns is 121, the first is the image ID and the rest stores the probabilities for each breed.","metadata":{}},{"cell_type":"markdown","source":"Do ids in `sample_submission.csv` are in alphabetical order?","metadata":{}},{"cell_type":"code","source":"sample_df['id'].is_monotonic_increasing","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:39:44.558006Z","iopub.execute_input":"2021-11-03T08:39:44.558626Z","iopub.status.idle":"2021-11-03T08:39:44.577573Z","shell.execute_reply.started":"2021-11-03T08:39:44.558578Z","shell.execute_reply":"2021-11-03T08:39:44.576399Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Yes, they are. It means that if the sample submission file is to be reused, we have to make sure that when the predictions are made the test input is in alphabetical order too.","metadata":{}},{"cell_type":"markdown","source":"<a id=\"model\"></a>\n# Model\n[[back to top]](#toc)","metadata":{}},{"cell_type":"markdown","source":"This chapter is divided into three subchapters.\n\nThe first step of creating a model is to read the available data from the disk. Fast.ai provides [an API to handle data named datablock](https://docs.fast.ai/tutorial.datablock). Using that API `DataLoaders` can be created which are able to read, store and make the data available to the model (Fast.ai's `DataLoaders` (plural) are not to be confused with PyTorch's `DataLoader` (singular), though the two are related). Loading the data is the subject of the [first subchapter](#model.dls).\n\nThe [second subchapter](#model.cnn) describes how to create a convolutional network and how to get an insight of the structure of the model.\n\nThe [third subchapter](#model.finetune) explains how to train the model easily and how to evaluate the results.","metadata":{}},{"cell_type":"markdown","source":"<a id=\"model.dls\"></a>\n## Dataloaders\n[[back to top]](#toc)","metadata":{}},{"cell_type":"markdown","source":"Fast.ai introduces the concept of `DataLoaders` (plural). They are explained in their excellent tutorial [here](https://colab.research.google.com/github/fastai/fastbook/blob/master/02_production.ipynb#scrollTo=b4NxWFFV6CNq) (see the other chapters of their book [here](https://course.fast.ai/start_colab#Opening-a-chapter-of-the-book)). The idea is that it makes sense to handle the training and validation `DataLoader` (singular) together. Apart from holding the training & validation dataloaders together fast.ai's `DataLoaders` also provide additional functionality and there are classes inherited from them.\n\nOne of the inherited classes is `ImageDataLoaders` which has a factory method named `from_csv` (see its documentation [here](https://docs.fast.ai/vision.data.html#ImageDataLoaders.from_csv)). By default the `from_csv` method expects a csv file which has two columns: the first contains the IDs of the images and the second contains the labels. Luckily we have exactly this, but it is possible to set the behaviour of the method to handle a different setup as well (see documentation).\n\n`ImageDataLoaders#from_csv` expects a csv file whose name is provided in the `csv` parameter and its path is given in the `path` parameter (so the location of the csv file is `path`\\\\`csv`). The first column of the csv file contains the names of the images and `ImageDataLoaders` will search for the images in the directory provided in the `folder` parameter (note, that the `folder` is relative to `path`, just like the `csv`).\n\nThe csv file contains the ids, but not the filenames of the images. Luckily, the filenames are in the format of `ID.jpg`. That's when the `suff` parameter comes handy because it can add a suffix (most likely the file extension) to an ID of an image. So if the ID of an image is `IMAGE_ID`, `from_csv` reads that image from `path`\\\\`folder`\\\\`IMAGE_ID`\\.`suff`.\n\n\nThe `item_tfms` parameter defines the transformation which has to be performed on each image before feeding it into the model. In our case it is resizing the images, because our model will expect that each item has the same size and it turned out during the data analysis in the previous paragraphs that the dimensions of the input images might differ from each other.\n\nCalling `ImageDataLoaders#from_csv` with these parameters will read all the images from `path`\\\\`folder` with the IDs provided in the csv file and resize them to the same dimensions. It will also create and store the training and the validation set. The validation set is 20% of the total data by default and its members are selected randomly.","metadata":{}},{"cell_type":"code","source":"dls = ImageDataLoaders.from_csv(\n    path=path,\n    csv='labels.csv',\n    folder='train',\n    suff='.jpg',\n    item_tfms=Resize(256)\n)\n\nprint(\"ImageDataloaders initialized.\")","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:39:44.579683Z","iopub.execute_input":"2021-11-03T08:39:44.580190Z","iopub.status.idle":"2021-11-03T08:39:49.394101Z","shell.execute_reply.started":"2021-11-03T08:39:44.580136Z","shell.execute_reply":"2021-11-03T08:39:49.392871Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"model.cnn\"></a>\n## CNN learner\n[[back to top]](#toc)","metadata":{}},{"cell_type":"markdown","source":"Once the `DataLoaders` (`dls`) are ready it is fairly simple to initialize a convolutional neural network using fast.ai.","metadata":{}},{"cell_type":"code","source":"learner = cnn_learner(dls, resnet34, metrics=error_rate)","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:39:49.396435Z","iopub.execute_input":"2021-11-03T08:39:49.397164Z","iopub.status.idle":"2021-11-03T08:39:53.142626Z","shell.execute_reply.started":"2021-11-03T08:39:49.397104Z","shell.execute_reply":"2021-11-03T08:39:53.139476Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The method named`cnn_learner` creates a convolutional neural network which reads the training and validation set from `dls`.\n\nThe model is based on ResNet-34. ResNet-34 is a pre-trained model, which is trained on the ImageNet dataset to be capable of distinguishing images (see more details [here](https://www.kaggle.com/pytorch/resnet34/home)). The idea is that it might be more effective to further improve a model which is already capable of recognizing everyday objects, rather than training one from scratch (transfer learning).\n\nThe `metrics` parameter defines how the performance of the model should be evaluated. In this case it's `error_rate` (how many breeds it misses).\n\nThere are plenty of other parameters of `CNNLearner`, but they either have a default value or deducted based on other parameters. For example one could wonder what the loss function is...","metadata":{}},{"cell_type":"code","source":"print(learner.loss_func)","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:39:53.145275Z","iopub.execute_input":"2021-11-03T08:39:53.146211Z","iopub.status.idle":"2021-11-03T08:39:53.154175Z","shell.execute_reply.started":"2021-11-03T08:39:53.146157Z","shell.execute_reply":"2021-11-03T08:39:53.152841Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"...well, it's cross-entropy loss, which makes sense in the case of image classification problems.","metadata":{}},{"cell_type":"markdown","source":"The model itself is also accessable:","metadata":{}},{"cell_type":"code","source":"print(learner.model)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-11-03T08:39:53.156192Z","iopub.execute_input":"2021-11-03T08:39:53.157570Z","iopub.status.idle":"2021-11-03T08:39:53.171377Z","shell.execute_reply.started":"2021-11-03T08:39:53.157484Z","shell.execute_reply":"2021-11-03T08:39:53.169850Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"model.finetune\"></a>\n## Fine tuning the model\n[[back to top]](#toc)","metadata":{}},{"cell_type":"markdown","source":"Once the model is ready, fast.ai makes it fairly straightforward to train it. The most basic option is to call the [`fine_tune`](https://docs.fast.ai/callback.schedule.html#Learner.fine_tune) method on the learner. The provided parameter determines the number of epochs (10 in this case).\n\n(Note, that `fine_tune` performs certain optimization unlike [`fit_one_cycle`](https://docs.fast.ai/callback.schedule.html#Learner.fit_one_cycle). The easiest way to understand the difference between the two is to check the code itself.)","metadata":{}},{"cell_type":"code","source":"learner.fine_tune(number_of_epochs)","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:39:53.173910Z","iopub.execute_input":"2021-11-03T08:39:53.174448Z","iopub.status.idle":"2021-11-03T08:43:59.488563Z","shell.execute_reply.started":"2021-11-03T08:39:53.174375Z","shell.execute_reply":"2021-11-03T08:43:59.487473Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The data above shows how the model trains, i.e. how the loss changes on the training and validation set and how the error_rate improves. Note, that while the loss on the training set constantly declines epoch by epoch, the validation loss and the error_rate might get worse compared to a previous iteration.","metadata":{}},{"cell_type":"markdown","source":"It is possible to have a more detailed overview of the performance of the learner than what we get during training. Fast.ai provides a class named `ClassificationInterpration` which is able to show how accurately the model guesses a certain breed. (See Wikipedia for [F1-score](https://en.wikipedia.org/wiki/F-score#Definition), [precision, recall](https://en.wikipedia.org/wiki/Precision_and_recall) and [accuracy](https://en.wikipedia.org/wiki/Accuracy_and_precision))\n\nIt can happen that the model barely recognizes a certain type of breed. In that case it might worth looking at the input data because it might happen that that specific breed is underrepresented in our data and thus the model doesn't have the chance to learn to recognize it. In such cases the modification of the input data set might be necessary.","metadata":{}},{"cell_type":"code","source":"interp = ClassificationInterpretation.from_learner(learner)\ninterp.print_classification_report()","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:43:59.490406Z","iopub.execute_input":"2021-11-03T08:43:59.491644Z","iopub.status.idle":"2021-11-03T08:44:24.107522Z","shell.execute_reply.started":"2021-11-03T08:43:59.491579Z","shell.execute_reply":"2021-11-03T08:44:24.106180Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We also have the chance to deep dive into the actual operations performed by `fine_tune` using the `show_training_loop` method on the learner. This call prints the callback functions called during training.","metadata":{}},{"cell_type":"code","source":"learner.show_training_loop()","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:44:24.110301Z","iopub.execute_input":"2021-11-03T08:44:24.110734Z","iopub.status.idle":"2021-11-03T08:44:24.135882Z","shell.execute_reply.started":"2021-11-03T08:44:24.110684Z","shell.execute_reply":"2021-11-03T08:44:24.134384Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It is even possible to plot the images with the worst losses. It is useful because that way it's possible to filter out invalid data. For example if the task is to identify the breed of a dog on a picture and there is no dog on the image, then that input is invalid.","metadata":{}},{"cell_type":"code","source":"interp.plot_top_losses(5, nrows=5)","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:44:24.138595Z","iopub.execute_input":"2021-11-03T08:44:24.139209Z","iopub.status.idle":"2021-11-03T08:44:25.092339Z","shell.execute_reply.started":"2021-11-03T08:44:24.139158Z","shell.execute_reply":"2021-11-03T08:44:25.090993Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"predictions\"></a>\n# Make predictions\n[[back top top]](#toc)","metadata":{}},{"cell_type":"markdown","source":"Once the model is trained, the next step is to make actual predictions. Bulk predictions can be made by calling the `get_preds` method of the learner. The function `get_preds` expects a dataloader which stores the test data.\n\nA `Learner` always holds a `DataLoaders` object which by default has two dataloaders: one for training and one for validation. \n\nThe `DataLoaders` class has a method named `test_dl`. As the pydoc describes:\n\n> \"Create a test dataloader from `test_items` using validation transforms of `dls`\"\n\nWhat this means it that `test_dl` reads in the data provided in `test_items` and applies the transformations on the input images with which the original `DataLoaders` object was created. In our case it means that `test_items` are resized to 256\\*256 just like the training items were. Probably the easiset is to check the [related code](https://github.com/fastai/fastai/blob/f8b74ef5b320512a2bb4a6c3cb17a5e917b7d6a3/fastai/data/core.py#L394) in fast.ai.","metadata":{}},{"cell_type":"markdown","source":"It seems to be reasonable to provide the model with the input files sorted alphabetically. This is what the next code snippet does.","metadata":{}},{"cell_type":"code","source":"def get_id_from_image_path(image_path):\n    image_id, _ = os.path.splitext(os.path.basename(image_path))\n    return image_id\n        \n\ndef cmp_path(item1, item2):\n    id1 = get_id_from_image_path(item1)\n    id2 = get_id_from_image_path(item2)\n    \n    if id1 < id2:\n        return -1\n    if id1 > id2:\n        return 1\n    \n    return 0\n\n\nsorted_test_image_files = sorted(get_image_files(test_path), key=cmp_to_key(cmp_path))\nprint(\"Test images are sorted alphabetically.\")","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:44:25.094154Z","iopub.execute_input":"2021-11-03T08:44:25.096671Z","iopub.status.idle":"2021-11-03T08:44:36.924321Z","shell.execute_reply.started":"2021-11-03T08:44:25.096606Z","shell.execute_reply":"2021-11-03T08:44:36.923114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The `test_datalaoder` is created using the previously described `test_dl` method and then the prediction takes place.","metadata":{}},{"cell_type":"code","source":"test_dataloader = learner.dls.test_dl(sorted_test_image_files)\npreds, _ = learner.get_preds(dl=test_dataloader)\n\nprint(\"Predictions are ready.\")","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:44:36.925879Z","iopub.execute_input":"2021-11-03T08:44:36.926566Z","iopub.status.idle":"2021-11-03T08:46:24.308771Z","shell.execute_reply.started":"2021-11-03T08:44:36.926502Z","shell.execute_reply":"2021-11-03T08:46:24.307139Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"preds.shape","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:46:24.311459Z","iopub.execute_input":"2021-11-03T08:46:24.312033Z","iopub.status.idle":"2021-11-03T08:46:24.321167Z","shell.execute_reply.started":"2021-11-03T08:46:24.311972Z","shell.execute_reply":"2021-11-03T08:46:24.319246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The dimensions of the predictions are 10357 \\* 120. 10357 is the number of the input images and 120 is the number of breeds ([see the EDA section](#EDA)). This is exactly the expected format (see `sample_submission.csv`).","metadata":{}},{"cell_type":"markdown","source":"The code snippet in the next cell puts the end results together: stores the IDs of the input pictures in an array, transposes it (changes the dimensions from (1, n), to (n, 1), and then puts the output tensor next to the ID column (`np.hstack`). Finally the results are written into the `submission.csv`.","metadata":{}},{"cell_type":"code","source":"submission = pd.read_csv(sample_submission_csv_path)\n\nsorted_ids = list(map(lambda x: get_id_from_image_path(x) ,sorted_test_image_files))\nid_col = np.transpose([np.asarray(sorted_ids)])\n\nres = np.hstack((id_col, preds.detach().cpu().numpy()))\nres_df = pd.DataFrame(res, columns=submission.columns)\n\nres_df.to_csv(submission_csv_path, index=False)\n\nprint(f'Predictions are saved to {submission_csv_path}')","metadata":{"execution":{"iopub.status.busy":"2021-11-03T08:55:12.173746Z","iopub.execute_input":"2021-11-03T08:55:12.174069Z","iopub.status.idle":"2021-11-03T08:55:14.918608Z","shell.execute_reply.started":"2021-11-03T08:55:12.174022Z","shell.execute_reply":"2021-11-03T08:55:14.917525Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id =\"conclusion\"></a>\n# Conclusion\n[[back to top]](#toc)","metadata":{}},{"cell_type":"markdown","source":"In this notebook a convolutional neural network was trained to predict dog breeds from pictures. The aim was to understand what is the simplest way to initialize and to train a cnn using fast.ai. The aim was not to achieve state-of-the-art results and apply advanced optimization technics.\n\nThe code itself is indeed concise:\n\n1. Create a dataloaders object:\n\n    `dls = ImageDataLoaders.from_csv(\n    path=path, csv='labels.csv', folder='train', suff='.jpg', item_tfms=Resize(256))`\n        \n        \n2. Initialize a convolutional neural network based on ResNet:\n\n    `learner = cnn_learner(dls, resnet34, metrics=error_rate)`\n\n3. Fine tune the model for couple of epochs:\n\n    `learner.fine_tune(number_of_epochs)`\n\n4. Make predictions for the test data:\n\n    `test_dataloader = learner.dls.test_dl(sorted_test_image_files)\n    preds, _ = learner.get_preds(dl=test_dataloader)`\n\n\nThis couple of lines of codes is still capable of achieving \\~0.64 score on the leaderboard (~80% of accuracy). The model is far from being state-of-the art but it is perfect to use as a baseline to which further improvements could be compared.\n\nFurther fine tuning of the model is out of the scope of this notebook. However, there are plenty of room for improvement. One way to go would be applying transformations on the input images (data augmentation). This could be implemented by passing the `batch_tfms` parameter to the `ImageDataLoaders`. It would also make sense to try transfer learning based on other pre-trained models as well. Another idea would be to use `fit_one_cycle` and experience with its parameters instead of just simply calling`fine_tune`.\n\nHowever, this code is a good start to get a baseline model which performs reasonably well.","metadata":{}}]}