{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom keras.utils.np_utils import to_categorical\nfrom keras.models import Sequential, load_model\nfrom keras.layers import Convolution2D, MaxPooling2D, Dense, Dropout, Activation, Flatten\nfrom keras.callbacks import EarlyStopping, ReduceLROnPlateau, ModelCheckpoint\nimport matplotlib.pyplot as plt\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"79f7345f9fde025cb4fbfef4513a826573b7b09b"},"cell_type":"markdown","source":"Firstly, let's import train and test file and look at their shape. The sample method will show the sample data of the provided dataframe."},{"metadata":{"trusted":true,"_uuid":"ddcdc268249189f210aca8b1783fe79e74dd3014","scrolled":true},"cell_type":"code","source":"train_df = pd.read_csv(\"../input/train.csv\")\nprint(\"Shape of train.csv file \",train_df.shape)\ntest_df = pd.read_csv(\"../input/test.csv\")\nprint(\"Shape of test.csv file\", test_df.shape)\ntrain_df.sample(5)\ntest_df.sample(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ab4ae0cdc11ffa59fe3bd90a2d4a2508597dcd23","scrolled":false},"cell_type":"code","source":"Y_train = train_df.iloc[:, 0].values\nX_train = train_df.iloc[:, 1:].values\nX_test = test_df.iloc[:, 0:].values\nX_train = X_train.astype('float32')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"f0624dc094a7d89e1f4a83c49b601bed0d97fdd5"},"cell_type":"markdown","source":"**Iloc** method returns the Series based on provided indices.  X_train stores the array of all values of pixels while Y_train stores the array of labels. And, we convert the type of X_train dataframe to float32. "},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"ac3ac24514b22b59db0adc2873d770c86cbd379c"},"cell_type":"code","source":"print(\"training data: {}\".format(X_train.shape))\nprint(\"training labels {}\".format(Y_train.shape))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d6d60b773f685b8d9367dfeecfe48baf132e56de"},"cell_type":"markdown","source":"Did you notice the change in shape? The column of the X_train dataframe reduced by 1. ;) "},{"metadata":{"trusted":true,"_uuid":"90b91a90918f3e7b34b2e22729eb50bff7904b31","scrolled":true},"cell_type":"code","source":"img_width=28\nimg_height=28\nimg_depth=1\n\nX_train = X_train.reshape(len(X_train), img_width, img_height, img_depth)\nX_test = X_test.reshape(len(X_test), img_width, img_height, img_depth)\nX_train = X_train.astype('float32')\nX_test = X_test.astype('float32')\n\nX_train /= 255\nX_test /= 255\n\nprint(\"Training matrix shape\", X_train.shape)\nprint(\"Testing matrix shape\", X_test.shape)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c412ee73a2675fe9282150d0e0be8a98ddd18467"},"cell_type":"markdown","source":"Reshaping of dataframe is done with reshape method. Division changes the pixel values ranging from 0 to 1."},{"metadata":{"_uuid":"61a8fde516dabbcdb291541e783c0bca33136b07"},"cell_type":"markdown","source":"Okay, let's check our train data before doing anything to make sure we have valid input data."},{"metadata":{"trusted":true,"_uuid":"9bd6734bff7a65ffb732cdbef98472aa84619e4d"},"cell_type":"code","source":"plt.figure(figsize=(12,10))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c89fe9bb2d9cfb4d8d18840cf23bfe1700facc75","scrolled":true},"cell_type":"code","source":"for i in range(9):\n    plt.subplot(3,3,i+1)\n    plt.imshow(X_train[i].reshape(28,28), cmap='gray', interpolation='none')\n    plt.title(\"Class {}\".format(Y_train[i]))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d934a8f9460cbc269bab48fed92bee4bf221a8dc"},"cell_type":"markdown","source":"Now, we will train the model and predict the given digits."},{"metadata":{"trusted":true,"_uuid":"7202b2ade446f5b81ee4e190792005bc2b0145f0","scrolled":true},"cell_type":"code","source":"Y_train = to_categorical(Y_train, num_classes=10)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9dc6baf1f7b9897f58600ddcbc91406686cae5e0"},"cell_type":"markdown","source":"to_categorical converts a class vector (integers) to binary class matrix."},{"metadata":{"_uuid":"075e446130bb48a52c80f6faa570eabd6d65b655"},"cell_type":"markdown","source":"So, it's time for stacking the layers into our model. Here, we will be using the Convolution2D. "},{"metadata":{"trusted":true,"_uuid":"9edb49aae5a6da9b674cfafb1a28b1a6a9d58d8f"},"cell_type":"code","source":"model = Sequential()\nmodel.add(Convolution2D(32,(3, 3), activation='relu', input_shape=(28,28, 1)))\nmodel.add(Convolution2D(32, (3, 3), activation='relu', input_shape=(28,28, 1)))\nmodel.add(MaxPooling2D(pool_size=(2,2)))\nmodel.add(Dropout(0.25))\n \nmodel.add(Flatten())\nmodel.add(Dense(128, activation='relu'))\nmodel.add(Dropout(0.5))\nmodel.add(Dense(10, activation='softmax'))\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4c87a6c4d17d1f81ca87fc050138361ff83dec4c"},"cell_type":"markdown","source":"Now, let's have a peak at our model."},{"metadata":{"trusted":true,"_uuid":"27af7fe254b450cecf61a7c45ab0c13e0f7fdc6c"},"cell_type":"code","source":"model.summary()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2eb51eb5f38388c09d0e611f2a23bf5044e74412"},"cell_type":"markdown","source":"So, we use categorical_crossentropy for loss and adam optimizer for optimization."},{"metadata":{"trusted":true,"_uuid":"fb95bb340bce24a8e80726127261d6c75509ebfb"},"cell_type":"code","source":"model.compile(loss='categorical_crossentropy',\n              optimizer='adam',\n              metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"25fe581e28661b3038b4e40cc03655d51f9c9f42"},"cell_type":"markdown","source":"Finally, we have come to the important step here. We will be fitting our model ."},{"metadata":{"trusted":true,"_uuid":"b7bc31f0f8e3f5e1390939cc15b5bfa61618b783"},"cell_type":"code","source":"saved_path='cnn_model.h5'\nmodel_checkpoint = ModelCheckpoint(filepath=saved_path, monitor='val_loss', verbose=1, save_best_only=True, mode='auto')\nearly_stop = EarlyStopping(monitor='val_loss', min_delta=0.5, patience=6, verbose=1, mode='auto')\nreduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=5, min_lr=0.001, mode='auto')\ncallback_list = [model_checkpoint, early_stop, reduce_lr]\n\ntrain_history = model.fit(X_train, Y_train, batch_size=32, \n                          epochs=15, verbose=1, callbacks=callback_list, validation_split=0.2)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4693d1ecf784803b3dc4b7cf968fa8e8a358f443"},"cell_type":"markdown","source":"Now, let's do the fun part. **Prediction**."},{"metadata":{"trusted":true,"_uuid":"6150b10f462a9390489135c73f9e3dca8fc66007"},"cell_type":"code","source":"model = load_model('cnn_model.h5')\nlabels = model.predict_classes(X_test)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d2521fcc4dd0f671464b2ea3b6deea14c27747f4"},"cell_type":"markdown","source":"Let's check if our results are valid."},{"metadata":{"trusted":true,"_uuid":"c7d7f9fb2479f6583c6e62d3f8d78fcd697831c5"},"cell_type":"code","source":"plt.figure(figsize=(12,10))\nfor i in range(9):\n    plt.subplot(3,3,i+1)\n    plt.imshow(X_test[i].reshape(28,28), cmap='gray', interpolation='none')\n    plt.title(\"predicted class {}\".format(labels[i]))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5463497dd30a933f557ac14b51252bb6ed9bbafe"},"cell_type":"markdown","source":"Seems okay, doesn't it?"},{"metadata":{"_uuid":"d543040dabbe95e90a2d65d839a7ee2f08065ddf"},"cell_type":"markdown","source":"So, now we will submit our result through csv file.\nFirst step is to create a dataframe and conversion of the dataframe to csv file is done."},{"metadata":{"trusted":true,"_uuid":"df2c6eafc71f8f2b6a7e52dbfd2fdea3a10ce5f7"},"cell_type":"code","source":"submit_df = pd.DataFrame({'ImageId': list(range(1, 28001)), 'Label': labels})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b285576b4f0d9f7e8fe9395ee167abd79b949685"},"cell_type":"code","source":"submit_df.to_csv('submit.csv', index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}