{"cells":[{"metadata":{},"cell_type":"markdown","source":"1. Introduction\n1. Data preparation\n    1. Load data\n    1. Normalization, Reshape and Label encoding\n    1. Visualize test and train sample\n1. Model Building\n    1. Split training and valdiation set\n    1. Define the model architechture\n    1. Set the optimizer and annealer\n    1. Data augmentation\n    1. Train model\n1. Evaluate the model\n    1. Training and validation curves\n    1. Visualize Prediction\n1. Prediction and submition\n    1. Predict and Submit results\n* * * *\n### 1. Introduction\n\nThis kernel is basic start in deep learning. \n\nCIFAR-10 (Canadian Institute For Advanced Research) is the type “hello world” dataset of computer vision. This dataset is a collection of images that are commonly used to train machine learning and computer vision algorithms. It is one of the most widely used datasets for machine learning research. The CIFAR-10 dataset contains 60,000 32x32 color images in 10 different classes. The 10 different classes represent airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks. There are 6,000 images of each class. In this competition, your goal is to correctly identify different object from a dataset of tens of thousands of color images.\n* * * *"},{"metadata":{"trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# import backend\nimport tensorflow as  tf\n\n# Model architecture\nfrom sklearn.model_selection import train_test_split\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Dropout, Flatten, Conv2D\nfrom keras.layers import MaxPool2D, Activation, MaxPooling2D\nfrom keras.layers.normalization import BatchNormalization\n\n# Annealer\nfrom keras.callbacks import LearningRateScheduler\n\n# Data processing\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.utils import to_categorical\nfrom keras.preprocessing import image\n\n# Visualization\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n\n# Progressor\nfrom tqdm import tqdm\nimport h5py\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Dataset Description:**\n\n* **train.7z -** a folder containing the training images in png format\n* **test.7z -** a folder containing the test images in png format\n* **trainLabels.csv -** the training labels\n* **sampleSubmission.csv -** We have to predict labels for all 300,000 images.\n\nTrain and Test files are present a .7z compressed format. You can process file from .7z extension code is given below, but it will take long enough time to process.\n<pre><code>\n# Reading .7z content\n!pip install pyunpack # install decoder package\n!pip install patool # install requirement\n\nfrom pyunpack import Archive\n\nprint(\"start unpacking train data\")\n%time Archive('/kaggle/input/cifar-10/train.7z').extractall('/kaggle/working/dataset/', auto_create_dir=True, patool_path=None)\nprint(\"unpacking finished\")\nprint(\"start unpacking test data\")\n%time Archive('/kaggle/input/cifar-10/test.7z').extractall('/kaggle/working/dataset/', auto_create_dir=True, patool_path=None)\nprint(\"unpacking finished\")\n\n### This will take approx 2-3 hr to read file from test.7z.\n</code></pre>\n\nI downloaded the zip file, and processed it and created files in batches.You can use this data for your work, data is organised as follow:\n* **train_data-** it is generated from train.7z\n* **test_data_1-** first 50k examples from test.7z\n* **test_data_2-** index 50k to 100k examples from test.7z\n* **test_data_3- ** index 100k to 150k examples from test.7z\n* **test_data_4-** index 150k to 200k examples from test.7z\n* **test_data_5- ** index 200k to 250k examples from test.7z\n* **test_data_6-** index 250k to 300k examples from test.7z\n"},{"metadata":{},"cell_type":"markdown","source":"### 2. Data Prepration\n**Load Data**"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"# Read File\nsample_submission = pd.read_csv('../input/cifar-10/sampleSubmission.csv')\ntrain_labels = pd.read_csv('../input/cifar-10/trainLabels.csv')\n\nprint(\"Number of training sample: \",train_labels.shape[0])\nprint(\"Number of test samples: \", sample_submission.shape[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import h5py\nwith h5py.File('../input/cifardata/train_data.h5', 'r') as file:\n    #for key in file.keys():\n     #   print(key)\n    train_ids = pd.DataFrame(np.array(np.squeeze(file['train_ids'])),columns=['id'])\n    train_data = np.array(file['train_images']).reshape(-1, 32, 32, 3)\n    \nwith h5py.File('../input/cifardata/test_data_1.h5', 'r') as file:\n    test_ids_1 = pd.DataFrame(np.array(np.squeeze(file['test_ids'])),columns=['id'])\n    test_data_1 = np.array(file['test_images']).reshape(-1, 32, 32, 3)\n    \nwith h5py.File('../input/cifardata/test_data_2.h5', 'r') as file:\n    test_ids_2 = pd.DataFrame(np.array(np.squeeze(file['test_ids'])),columns=['id'])\n    test_data_2 = np.array(file['test_images']).reshape(-1, 32, 32, 3)\n    \n    \nwith h5py.File('../input/cifardata/test_data_3.h5', 'r') as file:\n    test_ids_3 = pd.DataFrame(np.array(np.squeeze(file['test_ids'])),columns=['id'])\n    test_data_3 = np.array(file['test_images']).reshape(-1, 32, 32, 3)\n    \n    \nwith h5py.File('../input/cifardata/test_data_4.h5', 'r') as file:\n    test_ids_4 = pd.DataFrame(np.array(np.squeeze(file['test_ids'])),columns=['id'])\n    test_data_4 = np.array(file['test_images']).reshape(-1, 32, 32, 3)\n    \n    \nwith h5py.File('../input/cifardata/test_data_5.h5', 'r') as file:\n    test_ids_5 = pd.DataFrame(np.array(np.squeeze(file['test_ids'])),columns=['id'])\n    test_data_5 = np.array(file['test_images']).reshape(-1, 32, 32, 3)\n    \n    \nwith h5py.File('../input/cifardata/test_data_6.h5', 'r') as file:\n    test_ids_6 = pd.DataFrame(np.array(np.squeeze(file['test_ids'])),columns=['id'])\n    test_data_6 = np.array(file['test_images']).reshape(-1, 32, 32, 3)\n    \n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_data = np.concatenate([test_data_1, test_data_2, test_data_3, test_data_4, test_data_5, test_data_6], axis=0)\ntest_ids = pd.concat([test_ids_1, test_ids_2, test_ids_3, test_ids_4, test_ids_5, test_ids_6], axis=0).reset_index()\n#print(test_data.head())\nprint(test_ids.head())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# check load test data is consistent are not\ntest_ids.id.value_counts().sort_index() - sample_submission.id.value_counts().sort_index()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_submission.id.value_counts().sort_index()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# check train data is consistent or not\nsum(train_ids.id == train_labels.id)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# shape of data\nprint(train_data.shape)\nprint(test_data.shape)\nprint(train_ids.shape)\nprint(test_ids.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Distribution of classes in training samples\n\ntrain_labels.label.value_counts().plot(kind='bar', title='Distribution of classes')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Normalization, Reshape and Label Encoding**\n\n***Normalization***\n* We perform a grayscale normalization to reduce the effect of illumination's differences.\n* If we perform normalization, CNN works faster.\n***Reshape***\n* Train and test images (32 x 32)\n* We reshape all data to -1*28x28x1 4D matrices.\n\n***Label Encoding***\n* Encode labels to one hot vectors\n> 2 => [0,0,1,0,0,0,0,0,0,0]\n> 4 => [0,0,0,0,1,0,0,0,0,0]"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Function to reshape and scaling image\ndef Scale_Reshape(x):\n    x_min = x.min(axis=(1, 2), keepdims=True)\n    x_max = x.max(axis=(1, 2), keepdims=True)\n\n    x = (x - x_min)/(x_max-x_min)\n    \n    x = x.reshape(-1, 32, 32, 3)\n    return x\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Training data processing\ntrain = Scale_Reshape(train_data)\n\n# Test data processing\ntest = Scale_Reshape(test_data)\n\n# Label processing\n\nY=train_labels['label']\n# convert to one-hot\nY = pd.get_dummies(Y)\n\n\nprint(\"train shape: \", train.shape)\nprint(\"test shape: \", test.shape)\nprint(\"one-hot label shape: \", Y.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Label encoding\nfrom sklearn.preprocessing import LabelEncoder\n\nlb_make = LabelEncoder()\nlabel_int = train_labels[['label']].copy()\nlabel_int.label = lb_make.fit_transform(label_int.label)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Visualize test and train sample**\n##### Train sample"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# visualizing training samples\nplt.figure(figsize=(15,5))\nfor i in range(40):  \n    plt.subplot(4, 10, i+1)\n    plt.imshow(train_data[i].reshape((32, 32, 3)),cmap=plt.cm.hsv)\n    plt.title(f\"{train_labels.label[i]}\")\n    plt.axis('off')\nplt.subplots_adjust(wspace=0.3, hspace=0.3)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"##### Test sample"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# visualizing test samples\nplt.figure(figsize=(15,5))\nfor i in range(40):  \n    plt.subplot(4, 10, i+1)\n    plt.imshow(test[i].reshape((32,32, 3)),cmap=plt.cm.hsv)\n    plt.axis('off')\nplt.subplots_adjust(wspace=0.0, hspace=0.0)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 3. Model Building: "},{"metadata":{},"cell_type":"markdown","source":"**Split training and valdiation set**"},{"metadata":{"trusted":true},"cell_type":"code","source":"# split training and validation set.\nX_train, X_val, Y_train, Y_val = train_test_split(train, Y, random_state=0, test_size=0.1)\nprint(\"X_train shape: \", X_train.shape)\nprint(\"Y_train shape: \", Y_train.shape)\nprint(\"X_val shape: \", X_val.shape)\nprint(\"Y_val shape: \", Y_val.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Define the model architechture**\n\n**Convolutional neural networks (CNNs)** are the current **state-of-the-art model architecture** for image classification tasks. CNNs apply a series of filters to the raw pixel data of an image to extract and learn higher-level features, which the model can then use for classification. CNNs contains different components:\n\n***Convolutional layers***, which apply a specified number of convolution filters to the image. For each subregion, the layer performs a set of mathematical operations to produce a single value in the output **feature map**. Convolutional layers then typically apply a **ReLU activation** function) to the output to introduce **nonlinearities** into the model.\n\n***BatchNormalization layers***, Batch Normalization is a technical trick to make training faster.\n\n***Dropout layers***, Dropout is a regularization method, where the layer randomly replaces a proportion of its weights to zero for each training sample. This forces the net to learn features in a distributed way, not relying to much on a particular weight, and therefore improves generalization.\n\n***Activation layers***, Activation functions are really important for a any Neural Network to learn and make sense of something really complicated and Non-linear complex functional mappings between the inputs and response variable.They introduce **non-linear properties** to our Network.\n\n***Pooling layers***, which downsample the image data extracted by the convolutional layers to reduce the dimensionality of the feature map in order to decrease processing time. A commonly used pooling algorithm is max pooling, which extracts subregions of the feature map (e.g., 2x2-pixel tiles), keeps their maximum value, and discards all other values. it gives **rotation invariant** feature extraction ability to model. \n\n***Dense (fully connected) layers***, which perform classification on the features extracted by the convolutional layers and downsampled by the pooling layers. In a dense layer, every node in the layer is connected to every node in the preceding layer.\n\nTypically, a CNN is composed of a stack of convolutional modules that perform feature extraction. Each module consists of a convolutional layer followed by a pooling layer. The last convolutional module is followed by one or more dense layers that perform classification. The final dense layer in a CNN contains a single node for each target class in the model (all the possible classes the model may predict), with a softmax activation function to generate a value between 0–1 for each node (the sum of all these softmax values is equal to 1). We can interpret the softmax values for a given image as relative measurements of how likely it is that the image falls into each target class.\n\n"},{"metadata":{},"cell_type":"markdown","source":"Let's build a model to classify the images in the CIFAR-10 dataset using the following CNN architecture:\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# BUILD CONVOLUTIONAL NEURAL NETWORKS\n\nmodel = Sequential()\n\nmodel.add(Conv2D(32,  kernel_size = 3,kernel_initializer='he_normal', activation='relu', input_shape = (32, 32, 3)))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(0.2))\nmodel.add(Conv2D(64, kernel_size = 3, kernel_initializer='he_normal', strides=1, activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(MaxPooling2D((2, 2)))\nmodel.add(Conv2D(128, kernel_size = 3, strides=1, kernel_initializer='he_normal' ,padding='same', activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(MaxPooling2D((2, 2)))\nmodel.add(Conv2D(64, kernel_size = 3,kernel_initializer='he_normal', activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(MaxPooling2D((2, 2)))\nmodel.add(Dropout(0.2))\n\n\nmodel.add(Flatten())\nmodel.add(Dense(512,kernel_initializer='he_normal', activation = \"relu\"))\nmodel.add(Dropout(0.2))\nmodel.add(Dense(10, kernel_initializer='glorot_uniform', activation = \"softmax\"))\n\n\n# Compile the model\nmodel.compile(loss=\"categorical_crossentropy\", optimizer=\"Nadam\", metrics=[\"accuracy\"])\n\n# Summary of model\nmodel.summary()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Set the optimizer and annealer**\n\nWe train once with a smaller learning rate to ensure convergence. We then speed things up, only to reduce the learning rate by 10% every epoch. Keras has a function for this:"},{"metadata":{"_kg_hide-output":true,"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"\"\"\"\nimport math\n# learning rate schedule\ndef step_decay(epoch):\n    initial_lrate = 0.1\n    drop = 0.5\n    epochs_drop = 3.0\n    lrate = initial_lrate * math.pow(drop, math.floor((1+epoch)/epochs_drop))\n    return lrate\n\n# learning schedule callback\nannealer = LearningRateScheduler(step_decay)\ncallbacks_list = [annealer]\n\"\"\"","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Data augmentation**"},{"metadata":{},"cell_type":"markdown","source":"In order to avoid overfitting problem, we need to expand artificially our dataset. We can make existing dataset even larger by altering the training data with small transformations.\n\nFor the data augmentation, i choosed to :\n\n* Randomly rotate some training images by 10 degrees\n* Randomly Zoom by 10% some training images\n* Randomly shift images horizontally by 10% of the width\n* Randomly shift images vertically by 10% of the height\n* Randomly flip images horizontally.\n\nI did not apply a vertical_flip since it could have lead to misclassify symetrical object like aeroplane and bird, truck and shi."},{"metadata":{"trusted":true},"cell_type":"code","source":"# data augumetation\ndatagen = ImageDataGenerator(\n        rotation_range=0,  \n        zoom_range = 0.0,  \n        width_shift_range=0.1, \n        height_shift_range=0.1,\n        horizontal_flip=False)\n\n# data generator model to train and validation set\nbatch_size_1 = 500\ntrain_gen = datagen.flow(X_train, Y_train, batch_size=batch_size_1)\nval_gen = datagen.flow(X_val, Y_val, batch_size=batch_size_1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Generated sample**"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# visualizing augumented image\nX_train_augmented = X_train[9,].reshape((1,32,32,3))\nY_train_augmented = np.array(Y_train.iloc[9,:]).reshape((1,10))\nplt.figure(figsize=(15,4.5))\nfor i in range(30):  \n    plt.subplot(3, 10, i+1)\n    X_train2, Y_train2 = datagen.flow(X_train_augmented,Y_train_augmented).next()\n    plt.imshow(X_train2[0].reshape((32,32,3)),cmap=plt.cm.gray)\n    plt.axis('off')\n    if i==9: X_train_augmented = X_train[2000,].reshape((1,32,32,3))\n    if i==19: X_train_augmented = X_train[1180,].reshape((1,32,32,3))\nplt.subplots_adjust(wspace=-0.1, hspace=-0.1)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Train model**"},{"metadata":{"trusted":true},"cell_type":"code","source":"# traing parameters\nepochs = 2\nbatch_size = 32","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Fit the model\nhistory = model.fit_generator(train_gen, \n                              epochs = epochs, \n                              steps_per_epoch = X_train.shape[0] // batch_size,\n                              validation_data = val_gen,\n                              validation_steps = X_val.shape[0] // batch_size,\n                              \n                              verbose=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"final_loss, final_acc = model.evaluate(X_val, Y_val, verbose=1)\nprint(\"Final loss: {0:.4f}, final accuracy: {1:.4f}\".format(final_loss, final_acc))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 4. Evaluate the model\n\n**Training and validation curves**\n\n"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Plot the loss and accuracy curves for training and validation \nfig, ax = plt.subplots(2,1, figsize=(15, 5))\nax[0].plot(history.history['loss'], color='b', label=\"Training loss\")\nax[0].plot(history.history['val_loss'], color='r', label=\"validation loss\",axes =ax[0])\nlegend = ax[0].legend(loc='best', shadow=True)\n\nax[1].plot(history.history['accuracy'], color='b', label=\"Training accuracy\")\nax[1].plot(history.history['val_accuracy'], color='r',label=\"Validation accuracy\")\nlegend = ax[1].legend(loc='best', shadow=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Visualize Result**"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# making predictions\nprediction = model.predict_classes(X_val)\n\n# PREVIEW PREDICTIONS\nplt.figure(figsize=(20,8))\nfor i in range(40):  \n    plt.subplot(4, 10, i+1)\n    plt.imshow(X_val[i].reshape((32,32,3)),cmap=plt.cm.gray)\n    plt.title(f\"predict={Y.columns.values[prediction[i]]}\")\n    plt.axis('off')\nplt.subplots_adjust(wspace=0.3, hspace=0.3)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 5. Prediction and submition\n**Predict and Submit results**"},{"metadata":{"trusted":true},"cell_type":"code","source":"# prediction on test\nprediction = model.predict_classes(test)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"***Predicted samples distribution***"},{"metadata":{"trusted":true},"cell_type":"code","source":"#test_ids.head()\ntest_ids['label'] =str(0)\nprint(test_ids.head())\nfor i in tqdm(range(sample_submission.shape[0])):\n    test_ids.loc[i, 'label'] = Y.columns.values[prediction[i]]\n\nprint(test_ids.head())\n    ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"final_file = pd.merge((sample_submission, teat_ids), how='inner', left_on='id', right_on='id')\nfinal_file.to_csv('samplefile.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# make submission\n\"\"\"\nfor i in tqdm(range(sample_submission.shape[0])):\n    if i<100000 and i>=0:\n        sample_submission.loc[i, 'label'] = Y.columns.values[prediction_1[i]]\n    elif i <200000 and i>=100000:\n        sample_submission.loc[i, 'label'] = Y.columns.values[prediction_2[i-100000]]\n    elif i<300000 and i >=200000:\n        sample_submission.loc[i, 'label'] = Y.columns.values[prediction_3[i-200000]]\n\"\"\"       \n#sample_submission.to_csv('sampleSubmission.csv')\n#sample_submissions.head(20)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# distribution of predicted class\n#sample_submission.label.value_counts().plot(kind='bar', title='Pridicted class ditribution', figsize=(15, 4.5))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### References: "},{"metadata":{},"cell_type":"markdown","source":"https://keras.io/models/sequential/\nhttps://keras.io/layers/core/\nhttps://keras.io/layers/convolutional/\nhttps://keras.io/layers/pooling/\nhttps://www.kaggle.com/yassineghouzam/introduction-to-cnn-keras-0-997-top-6\nhttps://www.kaggle.com/toregil/welcome-to-deep-learning-cnn-99\nhttps://www.kaggle.com/kanncaa1/convolutional-neural-network-cnn-tutorial\nhttps://github.com/MazenAly/Cifar100"},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"\"\"\"\n# create model\nmodel=Sequential()\n\n#model.add(Lambda(standardize,input_shape=(28,28,1)))    \nmodel.add(Conv2D(filters=32, kernel_size = (5,5), activation=\"relu\", input_shape=(32,32,1)))\nmodel.add(Conv2D(filters=64, kernel_size = (3,3), activation=\"relu\"))\nmodel.add(MaxPooling2D(pool_size=(2,2)))\nmodel.add(BatchNormalization())\nmodel.add(Conv2D(filters=64, kernel_size = (3,3), activation=\"relu\"))\n#model.add(Dropout(0.25))\nmodel.add(Conv2D(filters=64, kernel_size = (3,3), activation=\"relu\"))\nmodel.add(MaxPooling2D(pool_size=(2,2)))\nmodel.add(BatchNormalization())    \nmodel.add(Conv2D(filters=128, kernel_size = (3,3), activation=\"relu\"))\n\nmodel.add(MaxPooling2D(pool_size=(2,2)))\n    \nmodel.add(Flatten())\nmodel.add(BatchNormalization())\nmodel.add(Dense(256,activation=\"relu\"))\nmodel.add(Dropout(0.25)) \nmodel.add(Dense(10,activation=\"softmax\"))\n\"\"\"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}