{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\n# for dirname, _, filenames in os.walk('/kaggle/input'):\n#     for filename in filenames:\n#         print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-02-17T17:12:26.887935Z","iopub.execute_input":"2022-02-17T17:12:26.888690Z","iopub.status.idle":"2022-02-17T17:12:26.917274Z","shell.execute_reply.started":"2022-02-17T17:12:26.888586Z","shell.execute_reply":"2022-02-17T17:12:26.916664Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Hi everyone,\nI am not new to kaggle community, but I just took my start up for working on kaggle regularly. I am going to work on this Cat-vs-dog competition, which is not very difficult problem as our images are very well arranged with good naming convention. So we will further arrange the pictures in respective directories so that we can benefit from ***ImageDataGenerator*** class API to get the image batches and their labels from directory names cats, dogs etc. , which is made very easy by keras Class method flow_from_directory. we will see this action in a moment ","metadata":{}},{"cell_type":"markdown","source":"First thing first , we will unzip our picture data as show below, unzip -q means , this unzip will be done in quiet mode, so as not show the full uncompressing logs","metadata":{}},{"cell_type":"code","source":"!unzip -q ../input/dogs-vs-cats-redux-kernels-edition/train.zip","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:12:57.283463Z","iopub.execute_input":"2022-02-17T17:12:57.283735Z","iopub.status.idle":"2022-02-17T17:13:07.543580Z","shell.execute_reply.started":"2022-02-17T17:12:57.283707Z","shell.execute_reply":"2022-02-17T17:13:07.542685Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ls -l -t /kaggle/working/train | head -3","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:18:56.653218Z","iopub.execute_input":"2022-02-17T17:18:56.653853Z","iopub.status.idle":"2022-02-17T17:18:57.619136Z","shell.execute_reply.started":"2022-02-17T17:18:56.653803Z","shell.execute_reply":"2022-02-17T17:18:57.618392Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!unzip -q ../input/dogs-vs-cats-redux-kernels-edition/test.zip","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:18:59.820315Z","iopub.execute_input":"2022-02-17T17:18:59.820610Z","iopub.status.idle":"2022-02-17T17:19:05.325814Z","shell.execute_reply.started":"2022-02-17T17:18:59.820578Z","shell.execute_reply":"2022-02-17T17:19:05.324908Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"check the naming pattern of our files with the follwing command:","metadata":{}},{"cell_type":"code","source":"ls -l -t /kaggle/working/test | head -3","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:19:09.386744Z","iopub.execute_input":"2022-02-17T17:19:09.387054Z","iopub.status.idle":"2022-02-17T17:19:10.171761Z","shell.execute_reply.started":"2022-02-17T17:19:09.387020Z","shell.execute_reply":"2022-02-17T17:19:10.170992Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"That shows pattern *cat.****.jpg* , likewise *dog.****.jpg* in the file names. So here we can move the files in to two new folders cat and dog so that our keras API can make use of directory names to label our image batches","metadata":{}},{"cell_type":"code","source":"cd /kaggle/working/train\n","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:19:12.247776Z","iopub.execute_input":"2022-02-17T17:19:12.248049Z","iopub.status.idle":"2022-02-17T17:19:12.256163Z","shell.execute_reply.started":"2022-02-17T17:19:12.248017Z","shell.execute_reply":"2022-02-17T17:19:12.255031Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"mkdir cat dog","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:19:17.250726Z","iopub.execute_input":"2022-02-17T17:19:17.251304Z","iopub.status.idle":"2022-02-17T17:19:17.911369Z","shell.execute_reply.started":"2022-02-17T17:19:17.251263Z","shell.execute_reply":"2022-02-17T17:19:17.910448Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"move all dog images to dog directory","metadata":{}},{"cell_type":"code","source":"mv  ./dog.* ./dog\n","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:19:19.465870Z","iopub.execute_input":"2022-02-17T17:19:19.466170Z","iopub.status.idle":"2022-02-17T17:19:20.761493Z","shell.execute_reply.started":"2022-02-17T17:19:19.466134Z","shell.execute_reply":"2022-02-17T17:19:20.760567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"move all cat images to cat directory","metadata":{}},{"cell_type":"code","source":"mv ./cat.* ./cat","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:19:22.442638Z","iopub.execute_input":"2022-02-17T17:19:22.442915Z","iopub.status.idle":"2022-02-17T17:19:23.385394Z","shell.execute_reply.started":"2022-02-17T17:19:22.442886Z","shell.execute_reply":"2022-02-17T17:19:23.384497Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"cd /kaggle/working/test\n","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:19:25.132918Z","iopub.execute_input":"2022-02-17T17:19:25.133208Z","iopub.status.idle":"2022-02-17T17:19:25.141359Z","shell.execute_reply.started":"2022-02-17T17:19:25.133175Z","shell.execute_reply":"2022-02-17T17:19:25.140491Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ls -l -t /kaggle/working/test | head -3","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:19:27.482332Z","iopub.execute_input":"2022-02-17T17:19:27.482879Z","iopub.status.idle":"2022-02-17T17:19:28.227816Z","shell.execute_reply.started":"2022-02-17T17:19:27.482840Z","shell.execute_reply":"2022-02-17T17:19:28.227030Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In test directory the image names do not contain any names like cat or dog, wo do not know the picture is of cat or dog without opeing it. That is why we will make predictions on testing data after our model is fully  trained on trainging data and validated on validation data.\n\n\nLet us start making our image batches for training and validation subset using ImageGenerator class.\n\nIt has three very important benefits according to my knowledge:\n1. It can infer label names from the folder / directory name\n2. Image generator object can be used directly in the fit method for training and fit method will directly iterate through the batches of images during training and validation epochs\n3. It can augment your train and validation image data on the runtime of the model, and exposes different orientations of an image in space, rotation, hrozontal shifting, vertical shifting etc, so that we transform each image in a way that our model will hardly able to identify  it as the same image being sent for training again and again","metadata":{}},{"cell_type":"code","source":"from keras.preprocessing.image import ImageDataGenerator\nval_batch = 10\ntrain_batch = 32\ntrain_datagen = ImageDataGenerator(\n        rescale=1./255,\n        shear_range=0.4,\n        zoom_range=0.3,\n        validation_split=0.30,\n        horizontal_flip=True,\n        )\ntrain_generator = train_datagen.flow_from_directory(\n        '/kaggle/working/train',\n        target_size=(130, 130),\n        batch_size=train_batch,\n        class_mode='binary',\n        subset = 'training',\n        color_mode = 'rgb',\n        shuffle = True,\n        )\nvalidation_generator = train_datagen.flow_from_directory(\n        '/kaggle/working/train',\n        target_size=(130, 130),\n        batch_size=val_batch,\n        class_mode='binary',\n        subset = 'validation',\n        color_mode = 'rgb',\n        shuffle= True)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:19:31.639838Z","iopub.execute_input":"2022-02-17T17:19:31.640183Z","iopub.status.idle":"2022-02-17T17:19:32.419125Z","shell.execute_reply.started":"2022-02-17T17:19:31.640147Z","shell.execute_reply":"2022-02-17T17:19:32.418396Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As you can see that ImageGenerator class has done all the job for us , which has to be done manually by loops otherwise.It has returned us Image generator object which will yield batches of images and the respective labels in tuples. The images has been segmented in two classes for training 17500 ( 70%) of total 25000 images, where as validation set has 7500 (30 %) of total 25000 images.We can see example of some image batches and class labels like below:","metadata":{}},{"cell_type":"code","source":"print(train_generator.class_indices)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:19:36.031872Z","iopub.execute_input":"2022-02-17T17:19:36.032408Z","iopub.status.idle":"2022-02-17T17:19:36.037193Z","shell.execute_reply.started":"2022-02-17T17:19:36.032369Z","shell.execute_reply":"2022-02-17T17:19:36.036312Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see train generator elements how does it look like. Lets see the first element of the train_generator.It will return a tuple of 32 images and 32 labels. Because our each training batch containes 32 images in it as specified by variable *train_batch = 32*","metadata":{}},{"cell_type":"code","source":"print(train_generator[0])","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:19:45.947505Z","iopub.execute_input":"2022-02-17T17:19:45.948065Z","iopub.status.idle":"2022-02-17T17:19:46.135296Z","shell.execute_reply.started":"2022-02-17T17:19:45.948018Z","shell.execute_reply":"2022-02-17T17:19:46.134569Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"our first batch contains 32 images and 32 labels","metadata":{}},{"cell_type":"code","source":"print(train_generator[0][0].shape)\nprint(train_generator[0][1].shape)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:19:51.636012Z","iopub.execute_input":"2022-02-17T17:19:51.638608Z","iopub.status.idle":"2022-02-17T17:19:52.195846Z","shell.execute_reply.started":"2022-02-17T17:19:51.638567Z","shell.execute_reply":"2022-02-17T17:19:52.195079Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"you can observe here the inside out of image generator object. as our batch size for training is 32 and total images were 17500. Hence 17500 / 32 = 546.8 batches, lets see that our 547th batch got how many pictures in it? Interesting isnt it ?","metadata":{}},{"cell_type":"code","source":"print(train_generator[546][0].shape)\nprint(train_generator[546][1].shape)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:19:54.224594Z","iopub.execute_input":"2022-02-17T17:19:54.225249Z","iopub.status.idle":"2022-02-17T17:19:54.516622Z","shell.execute_reply.started":"2022-02-17T17:19:54.225204Z","shell.execute_reply":"2022-02-17T17:19:54.515878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"decimal value in 546.8 i.e. 0.8 shows there were 28 pictures left in the end, and our final batch has 28 pictures in it, whereas all other batches from 0 ~ 545 have 32 pictures and respective labels in it. likwise you can claculate batches in validation dataset.","metadata":{}},{"cell_type":"code","source":"from tensorflow import keras\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Dense, Dropout,Flatten\nmy_base_model = keras.applications.DenseNet201(\n    weights=\"imagenet\",  # Load weights pre-trained on ImageNet.\n    input_shape=(130, 130, 3),\n    include_top=False,\n)  # Do not include the ImageNet classifier at the top.\n\n# Freeze the base_model\nmy_base_model.trainable = False\n\n# Create new model on top\nmy_inputs = keras.Input(shape=(130, 130, 3))\n# x = data_augmentation(inputs)  # Apply random data augmentation\n\n# Pre-trained Xception weights requires that input be scaled\n# from (0, 255) to a range of (-1., +1.), the rescaling layer\n# outputs: `(inputs * scale) + offset`\n# scale_layer = keras.layers.Rescaling(scale=1 / 127.5, offset=-1)\n# x = scale_layer(x)\n\n# The base model contains batchnorm layers. We want to keep them in inference mode\n# when we unfreeze the base model for fine-tuning, so we make sure that the\n# base_model is running in inference mode here.\nx = my_base_model(my_inputs, training=False)\nx = keras.layers.GlobalAveragePooling2D()(x)\n# x = keras.layers.Dropout(0.2)(x)  # Regularize with dropout\n# x = keras.layers.MaxPooling2D()(x)\nx = keras.layers.Dense(128, activation='relu')(x)\nx = keras.layers.Dense(128, activation='relu')(x)\nmy_outputs = keras.layers.Dense(1, activation='sigmoid')(x)\nmy_model = keras.Model(my_inputs,my_outputs)\n\nmy_model.summary()","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:19:56.933440Z","iopub.execute_input":"2022-02-17T17:19:56.933701Z","iopub.status.idle":"2022-02-17T17:20:05.511005Z","shell.execute_reply.started":"2022-02-17T17:19:56.933672Z","shell.execute_reply":"2022-02-17T17:20:05.510304Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"my_model.compile(\n    optimizer=keras.optimizers.Adam(),\n    loss=keras.losses.BinaryCrossentropy(from_logits=False),\n    metrics=['accuracy']\n)\nearly_stopping  = keras.callbacks.EarlyStopping(min_delta=0.001, patience=5)\n\nepochs = 30\nmy_model.fit(train_generator, \n             epochs=epochs, \n             validation_data=validation_generator,\n            callbacks=[early_stopping])","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:20:20.274013Z","iopub.execute_input":"2022-02-17T17:20:20.274299Z","iopub.status.idle":"2022-02-17T17:51:11.907640Z","shell.execute_reply.started":"2022-02-17T17:20:20.274271Z","shell.execute_reply":"2022-02-17T17:51:11.906952Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_datagen = ImageDataGenerator(\n    rescale = 1./255\n)\ntest_generator = test_datagen.flow_from_directory(\n        '/kaggle/input/dogsvscatsmytestdata/training_set/',\n        target_size = (130,130),\n        batch_size = 32,\n        class_mode='binary',\n        color_mode = 'rgb',\n)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:51:20.134762Z","iopub.execute_input":"2022-02-17T17:51:20.135029Z","iopub.status.idle":"2022-02-17T17:51:24.690447Z","shell.execute_reply.started":"2022-02-17T17:51:20.135000Z","shell.execute_reply":"2022-02-17T17:51:24.688663Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"my_model.evaluate(test_generator)","metadata":{"execution":{"iopub.status.busy":"2022-02-17T17:51:29.347037Z","iopub.execute_input":"2022-02-17T17:51:29.347730Z","iopub.status.idle":"2022-02-17T17:52:51.512539Z","shell.execute_reply.started":"2022-02-17T17:51:29.347692Z","shell.execute_reply":"2022-02-17T17:52:51.510970Z"},"trusted":true},"execution_count":null,"outputs":[]}]}