{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**Building a strong image classification model from less data**\n\nThe implementation is a slight variation of the one in https://gist.github.com/fchollet/0830affa1f7f19fd47b06d4cf89ed44d\n\nMainly, in this kernel , the method flow(x,y) is used whereas, in the above gist, method flow_from_directory(directory) is used.\nFor more info, you can refer https://keras.io/preprocessing/image/\n\nThe change is made to have an appropriate kernel to deal with the way data is structured in kaggle. Appropriate changes in other parts of the source code is also done.","metadata":{"_uuid":"cac9915711fdcec7e8eb6431fd8dcae21649b6aa","_cell_guid":"9619b018-f7a0-4d13-90b4-f769a8906003"}},{"cell_type":"code","source":"!unzip ../input/dogs-vs-cats-redux-kernels-edition/train.zip ","metadata":{"execution":{"iopub.status.busy":"2022-10-25T08:57:55.984269Z","iopub.execute_input":"2022-10-25T08:57:55.98469Z","iopub.status.idle":"2022-10-25T08:58:10.243146Z","shell.execute_reply.started":"2022-10-25T08:57:55.984626Z","shell.execute_reply":"2022-10-25T08:58:10.241888Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!unzip ../input/dogs-vs-cats-redux-kernels-edition/test.zip","metadata":{"execution":{"iopub.status.busy":"2022-10-25T08:56:28.356663Z","iopub.execute_input":"2022-10-25T08:56:28.357005Z","iopub.status.idle":"2022-10-25T08:56:35.730392Z","shell.execute_reply.started":"2022-10-25T08:56:28.356961Z","shell.execute_reply":"2022-10-25T08:56:35.728708Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#**Perform the necessary imports.**","metadata":{"_uuid":"16fefbb9857aead05fe80ecbf3ffbabfae4fca22","_cell_guid":"e10726c1-a487-4d89-bf8f-7ff532592440","execution":{"iopub.status.busy":"2022-10-25T09:16:30.190896Z","iopub.execute_input":"2022-10-25T09:16:30.191574Z","iopub.status.idle":"2022-10-25T09:16:30.196179Z","shell.execute_reply.started":"2022-10-25T09:16:30.191514Z","shell.execute_reply":"2022-10-25T09:16:30.19542Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os, cv2, re, random\nimport numpy as np\nimport pandas as pd\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.preprocessing.image import img_to_array, load_img\nfrom keras import layers, models, optimizers\nfrom keras import backend as K\nfrom sklearn.model_selection import train_test_split","metadata":{"_uuid":"49532c85c74dee05663bd7eda324d3df493c60ed","_cell_guid":"4f4ac6a4-708c-46ab-8269-e01978300bde","execution":{"iopub.status.busy":"2022-10-25T09:16:32.83488Z","iopub.execute_input":"2022-10-25T09:16:32.835439Z","iopub.status.idle":"2022-10-25T09:16:32.842744Z","shell.execute_reply.started":"2022-10-25T09:16:32.835347Z","shell.execute_reply":"2022-10-25T09:16:32.841887Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Data dimensions and paths**","metadata":{"_uuid":"d57ed5db51e8c287e62a8b40d264c42ca64a9ecf","_cell_guid":"5fa55078-4136-47a1-9914-1cac54eec440"}},{"cell_type":"code","source":"img_width = 100\nimg_height = 100\nTRAIN_DIR = \"./train\"\nTEST_DIR = \"./test\"\ntrain_images_dogs_cats = [TRAIN_DIR+i for i in os.listdir(TRAIN_DIR)] # use this for full dataset\ntest_images_dogs_cats = [TEST_DIR+i for i in os.listdir(TEST_DIR)]","metadata":{"_uuid":"2eb716c4e9ad7d0bd7293f0f156b5a751263b800","_cell_guid":"25980d83-1f66-420b-a1dc-501699c3d707","execution":{"iopub.status.busy":"2022-10-25T09:22:54.466944Z","iopub.execute_input":"2022-10-25T09:22:54.467722Z","iopub.status.idle":"2022-10-25T09:22:54.504017Z","shell.execute_reply.started":"2022-10-25T09:22:54.467626Z","shell.execute_reply":"2022-10-25T09:22:54.503057Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Helper function to sort the image files based on the numeric value in each file name.**","metadata":{"_uuid":"505895b51eb3ed5a1dee521c9adc0864639d0b96","_cell_guid":"7fd765e1-e4e1-4b59-b7b2-33336afd4848"}},{"cell_type":"code","source":"def atoi(text):\n    return int(text) if text.isdigit() else text\n\ndef natural_keys(text):\n    return [ atoi(c) for c in re.split('(\\d+)', text) ]","metadata":{"_uuid":"775ea273467ff9d32fbf2c1b2158b57f473618b5","_cell_guid":"a11beea5-afc8-42ac-9688-c9bd45ea19aa","execution":{"iopub.status.busy":"2022-10-25T09:22:56.361601Z","iopub.execute_input":"2022-10-25T09:22:56.362275Z","iopub.status.idle":"2022-10-25T09:22:56.367542Z","shell.execute_reply.started":"2022-10-25T09:22:56.362219Z","shell.execute_reply":"2022-10-25T09:22:56.366739Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Sort the traning set. Use 1300 images each of cats and dogs instead of all 25000 to speed up the learning process.**\n\n**Sort the test set**","metadata":{"_uuid":"11a7c4072cdedd8f5b79d4bfcec3d1f31d448404","_cell_guid":"d4c8b492-315c-4c38-9b1e-32c05e368027"}},{"cell_type":"code","source":"train_images_dogs_cats.sort(key=natural_keys)\ntrain_images_dogs_cats = train_images_dogs_cats[0:1300] + train_images_dogs_cats[12500:13800] \n\ntest_images_dogs_cats.sort(key=natural_keys)","metadata":{"_uuid":"22c6c18de15a48f12e9f2120ea56f6e603bc6329","_cell_guid":"001b4ea8-a12f-4290-9a0d-6990b7deb2f7","execution":{"iopub.status.busy":"2022-10-25T09:22:57.954375Z","iopub.execute_input":"2022-10-25T09:22:57.955024Z","iopub.status.idle":"2022-10-25T09:22:58.169576Z","shell.execute_reply.started":"2022-10-25T09:22:57.954954Z","shell.execute_reply":"2022-10-25T09:22:58.168878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Now the images have to be represented in numbers. For this, using the openCV library read and resize the image.  **\n\n**Generate labels for the supervised learning set.**\n\n**Below is the helper function to do so.**","metadata":{"_uuid":"5d3a332070aadd998dcb3ac506f086d1cbe37b20","_cell_guid":"58d15bea-7674-46ff-ba0d-af6a920590be"}},{"cell_type":"code","source":"def prepare_data(list_of_images):\n    \"\"\"\n    Returns two arrays: \n        x is an array of resized images\n        y is an array of labels\n    \"\"\"\n    x = [] # images as arrays\n    y = [] # labels\n    \n    for image in list_of_images:\n        x.append(cv2.resize(cv2.imread(image), (img_width,img_height), interpolation=cv2.INTER_CUBIC))\n    \n    for i in list_of_images:\n        if 'dog' in i:\n            y.append(1)\n        elif 'cat' in i:\n            y.append(0)\n        #else:\n            #print('neither cat nor dog name present in images')\n            \n    return x, y","metadata":{"_uuid":"5dcd4727d558b622fa5533f203834e86c758dace","_cell_guid":"be6eb2de-4302-4731-ba11-75cdb114a0df","execution":{"iopub.status.busy":"2022-10-25T09:23:00.336968Z","iopub.execute_input":"2022-10-25T09:23:00.337641Z","iopub.status.idle":"2022-10-25T09:23:00.346481Z","shell.execute_reply.started":"2022-10-25T09:23:00.337584Z","shell.execute_reply":"2022-10-25T09:23:00.345543Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Generate X and Y using the helper function above**\n\n**Since K.image_data_format() is channel_last,  input_shape to the first keras layer will be (img_width, img_height, 3). '3' since it is a color image**","metadata":{"_uuid":"c42578104846f709ec064b3fa82c861597dd18f1","_cell_guid":"eaddc700-7124-4526-9615-46e96144af58"}},{"cell_type":"code","source":"X, Y = prepare_data(train_images_dogs_cats)\nprint(K.image_data_format())","metadata":{"_uuid":"0111553c02dd7e4d622aeb65c3d307038f32f000","_cell_guid":"9add14aa-3cd6-4160-a3f3-20a2f6b61af9","execution":{"iopub.status.busy":"2022-10-25T09:23:02.016756Z","iopub.execute_input":"2022-10-25T09:23:02.017258Z","iopub.status.idle":"2022-10-25T09:23:02.04661Z","shell.execute_reply.started":"2022-10-25T09:23:02.01721Z","shell.execute_reply":"2022-10-25T09:23:02.045427Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Split the data set containing 2600 images into 2 parts, training set and validation set. Later, you will see that accuracy and loss on the validation set will also be reported while fitting the model using training set.**","metadata":{"_uuid":"13d1080aa11311bd3fe29857f621503797111080","_cell_guid":"110e7b06-d53d-4be4-ae4f-ca62a9beed89"}},{"cell_type":"code","source":"# First split the data in two sets, 80% for training, 20% for Val/Test)\nX_train, X_val, Y_train, Y_val = train_test_split(X,Y, test_size=0.2, random_state=1)","metadata":{"_uuid":"26eea9b27bcbf6a0ca3eba6f91cf5cace580f442","_cell_guid":"6a94930d-9c33-4361-ba3d-ef1f29707bce","collapsed":true,"jupyter":{"outputs_hidden":true}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"nb_train_samples = len(X_train)\nnb_validation_samples = len(X_val)\nbatch_size = 16","metadata":{"_uuid":"c5e3ba21cd44491307721204db024166a5028499","_cell_guid":"34977dad-9c51-4946-a465-d79138306b03","collapsed":true,"jupyter":{"outputs_hidden":true}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**We will be using the Sequential model from Keras to form the Neural Network. Sequential Model is  used to construct simple models with linear stack of layers. **\n\n**More info on Sequential model and Keras in general at https://keras.io/getting-started/sequential-model-guide/ and https://github.com/keras-team/keras**","metadata":{"_uuid":"6b998b641c185a10e809d95762b57a13a0065f56","_cell_guid":"f4cfc349-2ad8-46f8-9816-80b086401eff"}},{"cell_type":"code","source":"model = models.Sequential()\n\nmodel.add(layers.Conv2D(32, (3, 3), input_shape=(img_width, img_height, 3)))\nmodel.add(layers.Activation('relu'))\nmodel.add(layers.MaxPooling2D(pool_size=(2, 2)))\n\nmodel.add(layers.Conv2D(32, (3, 3)))\nmodel.add(layers.Activation('relu'))\nmodel.add(layers.MaxPooling2D(pool_size=(2, 2)))\n\nmodel.add(layers.Conv2D(64, (3, 3)))\nmodel.add(layers.Activation('relu'))\nmodel.add(layers.MaxPooling2D(pool_size=(2, 2)))\n\nmodel.add(layers.Flatten())\nmodel.add(layers.Dense(64))\nmodel.add(layers.Activation('relu'))\nmodel.add(layers.Dropout(0.5))\nmodel.add(layers.Dense(1))\nmodel.add(layers.Activation('sigmoid'))\n\nmodel.compile(loss='binary_crossentropy',\n              optimizer='rmsprop',\n              metrics=['accuracy'])\n\nmodel.summary()","metadata":{"_uuid":"f27c9488fc9767db7ef6beb75b45959852e6a018","_cell_guid":"e72ad7e4-0692-4dd2-a49f-47640caf2e52","collapsed":true,"jupyter":{"outputs_hidden":true}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**This is the augmentation configuration we will use for training and validation**","metadata":{"_uuid":"cc941a4d3d7cef29c74b4c28bb8849c2ebae5a40","_cell_guid":"3342dd4d-9dc1-454a-9f09-0674c8ac41d1"}},{"cell_type":"code","source":"train_datagen = ImageDataGenerator(\n    rescale=1. / 255,\n    shear_range=0.2,\n    zoom_range=0.2,\n    horizontal_flip=True)\n\nval_datagen = ImageDataGenerator(\n    rescale=1. / 255,\n    shear_range=0.2,\n    zoom_range=0.2,\n    horizontal_flip=True)","metadata":{"_uuid":"8a40cd117597246908aa7ef02172d08096548ba2","_cell_guid":"cf7cd57a-4532-4d79-a2dc-431b7bf174eb","collapsed":true,"jupyter":{"outputs_hidden":true}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Prepare generators for training and validation sets**","metadata":{"_uuid":"c78ba8af934502e7a736e1671a4ef73452ace722","_cell_guid":"1fffe8d6-e21b-4354-8313-323bfbb517da"}},{"cell_type":"code","source":"train_generator = train_datagen.flow(np.array(X_train), Y_train, batch_size=batch_size)\nvalidation_generator = val_datagen.flow(np.array(X_val), Y_val, batch_size=batch_size)","metadata":{"_uuid":"3b91d6026f7a71f7a5f8347032fd03c6dc929337","_cell_guid":"36388127-2d62-4aac-999e-04decf9bba30","collapsed":true,"jupyter":{"outputs_hidden":true}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Start training the model!**\n\n**For better accuracy and lower loss, we are using an epoch of 30. Epoch value can be increased for better results.**","metadata":{"_uuid":"bf95bd03922a7449939b135c632079968487324f","_cell_guid":"00cab697-e7c4-41a3-9014-1f7406c2d104"}},{"cell_type":"code","source":"history = model.fit_generator(\n    train_generator, \n    steps_per_epoch=nb_train_samples // batch_size,\n    epochs=30,\n    validation_data=validation_generator,\n    validation_steps=nb_validation_samples // batch_size\n)\n","metadata":{"_uuid":"d46dd6f0d43b62a27e92fe1c0088d787709fc584","_cell_guid":"446727ad-a903-45c2-bf87-97152984baa9","collapsed":true,"jupyter":{"outputs_hidden":true}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Well done for a small training set and a small Epoch! Accuracy on the training set is close to 84% while on the validation set is close to 79%**","metadata":{"_uuid":"8d4932c5555b6274a8a0b16c3fb770c003a1d380","_cell_guid":"cf363841-697a-4b38-9415-a1f57b8f16cb"}},{"cell_type":"markdown","source":"**Saving the model in Keras is simple as this! ** \n\n**It is quite helpful for reuse.**","metadata":{"_uuid":"3d679de7accbe4df7daac93969900f4f2c8c3d75","_cell_guid":"e5ba296f-049f-407a-ae30-22a2226d4810"}},{"cell_type":"code","source":"model.save_weights('model_wieghts.h5')\nmodel.save('model_keras.h5')","metadata":{"_uuid":"c3dc5f39f8e8227b6fbc99dab80af96a48aa9c1b","_cell_guid":"32aa2713-f32d-458e-9f9c-8c0dcb5f50f0","collapsed":true,"jupyter":{"outputs_hidden":true}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Time to predict classification using the model on the test set.**\n\n**Generate X_test and Y_test**","metadata":{"_uuid":"a1889aed06ff0c9d9a4f5095cc38649c2606dbca","_cell_guid":"7ed718a4-826e-4b83-9f2e-2156f6a75f45"}},{"cell_type":"code","source":"X_test, Y_test = prepare_data(test_images_dogs_cats) #Y_test in this case will be []","metadata":{"_uuid":"f807eefa26fcb70a4b9f7eb8b6c06e50109f65fc","_cell_guid":"490f5e45-eefd-4a46-b3bd-3e7e09cf695d","collapsed":true,"jupyter":{"outputs_hidden":true}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**This is the augmentation configuration we will use for testing. Only rescaling.**","metadata":{"_uuid":"c5c594867c18faeb5053bc2c819f78514f07a493","_cell_guid":"8a84efef-c2a3-4cda-a3a0-a6c2d6032238"}},{"cell_type":"code","source":"test_datagen = ImageDataGenerator(rescale=1. / 255)","metadata":{"_uuid":"33af89c39a57edfb49dca2a33c159c96e7d63fe2","_cell_guid":"a2c141c2-ec9e-4eab-8ea8-fa1a7485e506","collapsed":true,"jupyter":{"outputs_hidden":true}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Prepare generator for test set and start predicting on it.**","metadata":{"_uuid":"3c074c8a7974d1c2b01d07d37a2f3bab88fc9951","_cell_guid":"db815324-2ca3-49ce-a005-a1578f179092"}},{"cell_type":"code","source":"test_generator = val_datagen.flow(np.array(X_test), batch_size=batch_size)\nprediction_probabilities = model.predict_generator(test_generator, verbose=1)","metadata":{"_uuid":"49a4cc7684dfde31cd2dfd2921bfa15bfae19672","_cell_guid":"112ff470-4976-496c-bb0c-2c6eddddb528","collapsed":true,"jupyter":{"outputs_hidden":true}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Generate .csv for submission**","metadata":{"_uuid":"e299a1c8f9a927a2896a6dacb016bba9c83d0eea","_cell_guid":"ae9bb1ac-91a2-43c4-a57d-e87651c4d3e4"}},{"cell_type":"code","source":"counter = range(1, len(test_images_dogs_cats) + 1)\nsolution = pd.DataFrame({\"id\": counter, \"label\":list(prediction_probabilities)})\ncols = ['label']\n\nfor col in cols:\n    solution[col] = solution[col].map(lambda x: str(x).lstrip('[').rstrip(']')).astype(float)\n\nsolution.to_csv(\"dogsVScats.csv\", index = False)","metadata":{"_uuid":"75ff4f7fa16b6fc9758bcea0dd2107b65740cfda","_cell_guid":"bc3fada0-c12a-4b36-a3a1-eaa7195e9b19","collapsed":true,"jupyter":{"outputs_hidden":true}},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Kindly upvote if you find this kernel useful**","metadata":{"_uuid":"f333a76d6514546f410d0e2679431515681bb87f","_cell_guid":"edaa9c27-5d0a-4488-b67b-525f7f71377f"}}]}