{"cells":[{"metadata":{"_uuid":"52ba7940f430ddbcfe5c608e3fac6d969f89b836"},"cell_type":"markdown","source":"Hello All ! This is pretty basic tutorial to start off with **Convolutional networks**.\n\nLet's start with a bit of introduction Convolutional neural networks are primarily used to classify images or identify pattern similarities between them. So a convolutional network receives a normal color image as a rectangular box whose width and height are measured by the number of pixels along those dimensions, and whose depth is three layers deep, one for each letter in RGB. Those depth layers are referred to as **channels**. For simplification needs we will only consider gray scale image here.\n\nAs images move through a convolutional network, different patterns are recognised just like a normal neural network. But here rather than focussing on one pixel at a time, a convolutional net takes in square patches of pixels and passes them through a **filter**. That filter is also a square matrix smaller than the image itself, and equal in size to the patch. It is also called a **kernel**.\n![](https://ahmedbesbes.com/images/GIF.gif)\n\n\nWell that's it with the theory let's get started with the practical.\n\n\n"},{"metadata":{"_uuid":"be4e519695677a33858690096ddc849c25130a47"},"cell_type":"markdown","source":"\n**Import Libraries**\n\nThere are basically 4 type of libraries which you have to import\n\n1. Pandas :- For reading / writing data\n2. Matplotlib to display images\n3. Tensorflow Keras models :- Need a model to predict right !! \n4. Tensorflow Keras layers :- Every NN needs layers and CNN needs well a couple of layers.\n\nLayers needed by CNN\n1. Conv2D :- Basic Convolutional layer . Here we will be using a 64 neuron layer\n2. Dense :- Dense layer is needed by every neural network to finally output the result however every once in while using a Dense layer helps in making model learn.\n3. MaxPooling :- CNN has a concept of max pooling. After every convoulution we get some values in a kernel. However in max pooling we select max kernel value.\n4. Flatten:- Conv2D layer returns doesn't return a flatten data hence we need Flatten layer before feeding it into final Dense layer\n"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport cv2\nimport matplotlib.pyplot as plt\nimport tensorflow as tf\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Dense, Flatten, Dropout, Activation, Conv2D, MaxPooling2D\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))\n\nimport zipfile\n\nwith zipfile.ZipFile(\"../input/train.zip\",\"r\") as z:\n    z.extractall(\".\")\n    \nwith zipfile.ZipFile(\"../input/test1.zip\",\"r\") as z:\n    z.extractall(\".\")\n\n# Any results you write to the current directory are saved as output.\n\nprint(os.listdir(\"/kaggle/working/train\"))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2f12a4e690a72c7520e58e75c76f371bf4bca7be"},"cell_type":"markdown","source":"We need to train a model first so we will check training data In the below code we are iterating through all images in train folder and then we will split image name with deliminiter \".\" We have names like dog.0, dog.1, cat.2 etc.. Hence after splitting we are gonna get results like \"dog', \"cat\" as category value of the image. To make this example more easy we will consider dog as \"1\" and cat as \"0\"\n\nNow every image is actually a set of pixels so how to get our computer know that. Its simple convert all those pixels into an array. So we are going to use here a **cv2** library to read our image into an array and also it will read as a gray scale image.\n\n>cv2.imread(os.path.join(path,p),cv2.IMREAD_GRAYSCALE)\n\nNow we have got here images of all sizes . We have landscape, portrait etc etc.. We need to make them all of a single size so it can be analysed pretty easily. How to do that very very simple again. Use cv2\n\n>cv2.resize(img_array, dsize=(80, 80))\n\nOk so we have got image array and its resized but do you believe whatever I just did was correct. Was the resizing of 80 X 80 good or is it bad. Should check it. How can we do that. There is one answer matplotlib. Using the below code we can display the image.\n\n>plt.imshow(new_img_array,cmap=\"gray\")\n\nPlease run the below the code to get better understanding. I have applied break here to just display 1 image. You can try out with 50 X 50 or 100 X100 to see the difference.\n\n\n"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"main_dir = \"/kaggle/working/\"\ntrain_dir = \"train\"\npath = os.path.join(main_dir,train_dir)\n\nfor p in os.listdir(path):\n    category = p.split(\".\")[0]\n    img_array = cv2.imread(os.path.join(path,p),cv2.IMREAD_GRAYSCALE)\n    new_img_array = cv2.resize(img_array, dsize=(80, 80))\n    plt.imshow(new_img_array,cmap=\"gray\")\n    break\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4c70b551142eb6ece6b4a3946e760086e969a64e"},"cell_type":"markdown","source":"Okay so the above code was more for understanding purpose. Nowe we will get to the real part of coding here.\n\nDeclare your training array X and your target array y. Here X will be the array of pixels and y will be value 0 or 1 indicating its a dog or cat\nWrite convert function to map category \"dog\" or \"cat\" into 1 and 0\n\nCreate a function create_test_data which takes all training images into a loop. Converts into image array.Resize image into 80 X80. Append image into X array. Append category value into y array."},{"metadata":{"trusted":true,"_uuid":"cfa2a50c1ea569d97806510e0e313e1ce9d7452b"},"cell_type":"code","source":"X = []\ny = []\nconvert = lambda category : int(category == 'dog')\ndef create_test_data(path):\n    for p in os.listdir(path):\n        category = p.split(\".\")[0]\n        category = convert(category)\n        img_array = cv2.imread(os.path.join(path,p),cv2.IMREAD_GRAYSCALE)\n        new_img_array = cv2.resize(img_array, dsize=(80, 80))\n        X.append(new_img_array)\n        y.append(category)\n    ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"59b1ed87293db99778edda93d5950d5e13a0019b"},"cell_type":"markdown","source":"Now what..\n\nCall the function\n\nBut also later convert X and y into numpy array We also have to reshape X with the below code\n\nnp.array(X).reshape(-1, 80,80,1)\n\nI really don't know why we are using reshaping here. If anyone knows please do write in comments\n"},{"metadata":{"trusted":true,"_uuid":"6e30159db4207451532ac085e617bc2669c93714"},"cell_type":"code","source":"create_test_data(path)\nX = np.array(X).reshape(-1, 80,80,1)\ny = np.array(y)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"dfbac0e37b746d069807f1428ef517602eeadfa9"},"cell_type":"markdown","source":"If you want to save your processed training (X) and target (y) you can use **pickle**. Please refer the below code for this. I wrote this to experiment but its not really needed. But anyways I still think its better to learn. :)"},{"metadata":{"trusted":true,"_uuid":"6fc971001358f8cdeefedc884b85240e8c82dc35"},"cell_type":"code","source":"import pickle\n\npickle.dump( X, open( \"train_x\", \"wb\" ) )\npickle.dump( y, open( \"train_y\", \"wb\" ) )\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ea5dec80ecb8e958dd2730ee7c911d055da93bf1"},"cell_type":"markdown","source":"If you see the values of X you can see a variety of values between 0- 255 . Its because every pixel has different density of black and white. But with the wide range of values it becomes difficult for a training model to learn ( sometimes memorize ). \n\nHow to resolve this And you guessed it right . You can **normalize** the data. We can use Keras normalize here also . But well we already know all values are having range between 0-255 so we can just divide it by 255 and get all values scaled between 0 -1\n\nThat's what we have done below. You can skip this step to see the difference between accuracy. Don't believe everything I say. Experiment and see for yourself\n"},{"metadata":{"trusted":true,"_uuid":"aac4daf7957d1429a9d47e19c75459c217f04d50"},"cell_type":"code","source":"#Normalize data\nX = X/255.0","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"47bf804edac40561e5c9d13e2a6f29e96ff31ecd"},"cell_type":"markdown","source":"Enough of data processing I wanna train :)\nHere are the steps to do define our CNN model\n\n1. Define a Sequential model\n2. Start adding layers to it.\n3. First we will add a Conv2D layer with 64 nodes and kernel size of (3,3). You can also experiment with different values here like 32, 128 etc. Also we have to specify input shape which is your X shape. Activation we will take 'relu' for now however there are many others to experiment with.\n4. Now after every Conv layer we always do max pooling so we will add max pooling layer with a size of (2,2) \n5. We will repeat this combination again because come on 2 is better than one. Haha. We you can also add 3 or more convolution layers but keep in mind the more layers you add more time it will take to train.\n6. But we don't have much time so we will add a flatten layer now. As we have to feed our data to Dense layer later.\n7. We will now add a Dense layer of 64 nodes. Note for all these layers we are using activation as 'relu' because I found results better with this. You can skip specifying activation but this might make a model a conveniently linear which might not work for us.\n8. In the end for getting our result we will add final Dense layer . Activation can be sigmoid or softmax (if you need probability use sigmoid else use softmax). Here I have used sigmoid.\n9. Finally we will compile the model . There are 3 things to mention here . Loss, Optimizer, Metrics\n\n**Loss** :- To make our model better we either minimize loss or maximize accuracy. NN always minimize loss. To measure it we can use different formulas like 'categorical_crossentropy' or 'binary_crossentropy'. Here I have used binary_crossentropy\n\n**Optimizer** :- If you know a lil bit about mathematics of machine learning you might be familier with local minima or global minima or cost function. To minimize cost function we use different methods For ex :- like gradient descent, stochastic gradient descent. So these are call optimizers. We are using a default one here which is **adam**\n\n**Metrics** :- This is to denote the measure of your model. Can be accuracy or some other metric.\n        "},{"metadata":{"trusted":true,"_uuid":"80c4b5ece66f7a069cfb6194779dc02ef76ce3e8"},"cell_type":"code","source":"model = Sequential()\n# Adds a densely-connected layer with 64 units to the model:\nmodel.add(Conv2D(64,(3,3), activation = 'relu', input_shape = X.shape[1:]))\nmodel.add(MaxPooling2D(pool_size = (2,2)))\n# Add another:\nmodel.add(Conv2D(64,(3,3), activation = 'relu'))\nmodel.add(MaxPooling2D(pool_size = (2,2)))\n\nmodel.add(Flatten())\nmodel.add(Dense(64, activation='relu'))\n# Add a softmax layer with 10 output units:\nmodel.add(Dense(1, activation='sigmoid'))\n\nmodel.compile(optimizer=\"adam\",\n              loss='binary_crossentropy',\n              metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0f7a138933cea22e5d6ebb8098e3d5728d47740e"},"cell_type":"markdown","source":"Now we will fit our model with training data.\n\nEpochs :- How many times our model will go through data\n\nBatch size :- How much amount of data at once you wanna pass through the model\n\nvalidation_split :- How much amount of data (in this case its 20 %) you will need to check cross validation error"},{"metadata":{"trusted":true,"_uuid":"ec02e9538761ef147ba6e0c2129dba117a37efa1"},"cell_type":"code","source":"model.fit(X, y, epochs=10, batch_size=32, validation_split=0.2)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"351552012321cc626c0177be12aee5c797ea72df"},"cell_type":"markdown","source":"Now we have to preporcess our test data also same as that our training data."},{"metadata":{"trusted":true,"_uuid":"6b55df2d3396f6d27d4e5d4ea4584a9e1733bc92"},"cell_type":"code","source":"train_dir = \"test1\"\npath = os.path.join(main_dir,train_dir)\n#os.listdir(path)\n\nX_test = []\nid_line = []\ndef create_test1_data(path):\n    for p in os.listdir(path):\n        id_line.append(p.split(\".\")[0])\n        img_array = cv2.imread(os.path.join(path,p),cv2.IMREAD_GRAYSCALE)\n        new_img_array = cv2.resize(img_array, dsize=(80, 80))\n        X_test.append(new_img_array)\ncreate_test1_data(path)\nX_test = np.array(X_test).reshape(-1,80,80,1)\nX_test = X_test/255","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"35c262be8b002a0cb484de9d22f14ba5d2557baf"},"cell_type":"markdown","source":"Ok. Now the time has come to finally\n**PREDICT**\n\nso feed your model with test data to predict"},{"metadata":{"trusted":true,"_uuid":"7b902cc4d61cfaa770fdf3b674b399e87fd62ac5"},"cell_type":"code","source":"predictions = model.predict(X_test)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"edca7d26b6ce321154679e3d8eb7b6631380e241"},"cell_type":"markdown","source":"We are rounding the result here as we used sigmoid function and we got the probablity values in our predicted dataset"},{"metadata":{"trusted":true,"_uuid":"87ea9ff5a4e9cfa0ac21a1af18b08e86b9183425"},"cell_type":"code","source":"predicted_val = [int(round(p[0])) for p in predictions]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"0cebf5eada571b472b883efe532b921c59629dfd"},"cell_type":"markdown","source":"Now you have to make submission data frame to submit your resultset"},{"metadata":{"trusted":true,"_uuid":"1496f4fb9e8752d359a5f0a6ccee6c544ee896a8"},"cell_type":"code","source":"submission_df = pd.DataFrame({'id':id_line, 'label':predicted_val})","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f599430d2f3062ec7a726bff3caef2a5ffd01166"},"cell_type":"markdown","source":"Write your data frame to a csv file"},{"metadata":{"trusted":true,"_uuid":"2031743817724557d2b65b34541ade436133c657"},"cell_type":"code","source":"submission_df.to_csv(\"submission.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9d6b641de67262d0274ca2d3b92ef83edc927d42"},"cell_type":"markdown","source":"And Voila we are done !!\n\nPlease do read this kernel and leave your comments for feedback and questions.\n\nThanks all :)"},{"metadata":{"trusted":true,"_uuid":"0963b84bf9f6e27235b37d1133c37868791a6eb9"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}