{"cells":[{"metadata":{},"cell_type":"markdown","source":"**Thanks for checking out this notebook.\n**\nIf you find this helpful, please show your love and support by upvoting it too."},{"metadata":{},"cell_type":"markdown","source":"# Understanding the dataset\n**Let us print the dataset**"},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nprint(\"There are following directories and files in this dataset\")\nprint(*list(os.listdir(\"../input/dogs-vs-cats-redux-kernels-edition\")),sep = \"\\n\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 1. sample_submission.csv\n**This csv file is an example of csv file(of the predictions by your model) to be uploaded for this competition's evaluation.****"},{"metadata":{},"cell_type":"markdown","source":"# 2. Train.zip\n**This zip file contains the training data which will be used for training our model**"},{"metadata":{},"cell_type":"markdown","source":"# 3. Test.zip\n**This zip file contains the training data which will be used for training our model**"},{"metadata":{},"cell_type":"markdown","source":"# Importing necessary libraries\n**Import Pandas - For data analysis Import Fastai - For training of deep learning model and predictions.\n\n\nNote: We are using Fastai2 (course version 4) which is the latest version. (Not previous version of Fastai- which is version 1)**"},{"metadata":{"trusted":true},"cell_type":"code","source":"import glob\nimport numpy as np\nimport pandas as pd\nimport fastai\nfrom fastai.vision.all import *","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Defining the variables and assigning the paths for this notebook\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"path = Path('../input/dogs-vs-cats-redux-kernels-edition')\nimage_path = Path('./train')\n\nsample_submission_file = Path('../input/dogs-vs-cats-redux-kernels-edition/sample_submission.csv')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Checking the data for model training\n**We will now check the images availabe to us:**"},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"import zipfile\nwith zipfile.ZipFile(\"../input/dogs-vs-cats-redux-kernels-edition/train.zip\",\"r\") as x:\n    x.extractall(\"./\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Training data\n**Let us see the training data present in \"train.zip\" file and available in train folder **"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Number of images present in train folder are: \")\nlen(glob.glob('./train/*.jpg'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"First few samples of training data is as follows: \\n\")\nPath('./train').ls(5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Creating the Pandas Dataframe to be used for creating an ImageDataLoader"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_img_files = get_image_files('./train')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"img_file_names = np.array([f'{x}' for x in sorted(train_img_files)])\nimg_file_names","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"label_names = np.array([(1 if 'dog' in file_name else 0) for file_name in img_file_names])\nlabel_names","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = pd.DataFrame(img_file_names, columns=['id'])\ntrain_df['label'] = label_names\nprint(\"Size of Training data \\n\", train_df.shape)\nprint(\"----------------------------------------------------------\")\nprint(\"\\nFirst few samples of data are \\n\",train_df.head())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Creating the image data loader"},{"metadata":{"trusted":true},"cell_type":"code","source":"dls = ImageDataLoaders.from_df(train_df, path='.', \n                         valid_pct=0.2, seed=42, \n                         fn_col='id', \n                         label_col='label', \n                         label_delim=None, \n                          \n                         item_tfms=Resize(128), batch_tfms=None, \n                         bs=64, \n                         )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Let us check the device type of our \"ImageDataLoader\" to make sure that we are using \"GPU\"**"},{"metadata":{"trusted":true},"cell_type":"code","source":"image_data_loader.device","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Let us check few random images from our ImageDataLoader's batch to make sure that images and labels appears correctly in it.\n**\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"dls.show_batch()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(dls.train_ds), len(dls.valid_ds)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Trainnig the image recognizer model\n\n#We create a CNN (convolutional neural network) with the following specific details:\n\nWhat data we want to train it on? </br> Our data to be used for training is \"dls\"\n\nWhich architecture to use? </br> We are using Resnet34\n\nwhat metric to use for our training evaluation? </br> We have specified it as \"error_rate\"\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = cnn_learner(dls, resnet34, metrics=error_rate)\nlearn.fine_tune(4,freeze_epochs = 4)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.fit_one_cycle(6,lr_max=slice(1e-6,1e-4))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Bring it on - Test data !!"},{"metadata":{},"cell_type":"markdown","source":"# Test data\n**Let us see the test data present in \"test.zip\" file and available in test folder **"},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"!rm -r './train'\nwith zipfile.ZipFile(\"../input/dogs-vs-cats-redux-kernels-edition/test.zip\",\"r\") as y:\n    y.extractall(\"./\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Number of images present in test folder are: \")\nlen(glob.glob('./test/*.jpg'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"First few samples of test data is as follows: \\n\")\nPath('./test').ls(5)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Defining the variables and assigning the paths for test dataset**"},{"metadata":{"trusted":true},"cell_type":"code","source":"test_image_files = get_image_files('./test')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Let us create a ImageDataLoader of our test data set**"},{"metadata":{"trusted":true},"cell_type":"code","source":"fnames = [f.name for f in test_image_files]\ntst_dl = dls.test_dl(test_image_files)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Make the predictions using our trained model called \"learn\".\n\nIgnoring the first two outputs from the model, let us take our final result stored in variable \"preds\"\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"_,_,preds = learn.get_preds(dl=tst_dl, with_decoded=True)\n\nsub = pd.DataFrame(fnames, columns=['id'])\nsub['label'] = preds","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"!rm -r './test'\nsub = pd.DataFrame(fnames, columns=['id'])\nsub['label'] = preds","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_sub = pd.read_csv(sample_submission_file)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}