{"cells":[{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import numpy as np   # linear algebra\nimport pandas as pd  # data processing\nimport matplotlib.pyplot as plt  # figures\nfrom keras.utils import to_categorical\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 1. Upload data and prepare them for Machine Training"},{"metadata":{"_cell_guid":"","_uuid":"","trusted":true},"cell_type":"code","source":"#sample_submission = pd.read_csv(\"../input/digit-recognizer/sample_submission.csv\")\ntest = pd.read_csv(\"../input/digit-recognizer/test.csv\")\ntrain = pd.read_csv(\"../input/digit-recognizer/train.csv\")\n\ntrain_data = train.iloc[:, 1:]                         # pixel data are indexed\ntrain_labels = train['label'].values.astype('float32') # just a one-dimensional ndarray, with answers for 42000 samples\n\n# return numpy representation of DataFrame object\ntest_data = test.to_numpy(dtype='float32') # does exactly the same as the line above but for the test dataset\n\n# the last index is added for further use in augmentations\ntrain_data = train_data.to_numpy(dtype='float32').reshape(train_data.shape[0], 28, 28, 1)\ntest_data = test_data.reshape(test_data.shape[0], 28, 28, 1)\n\ntrain_data = train_data/255  \ntest_data = test_data/255  \n\ntrain_labels = to_categorical(train_labels)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 2. Define the architecture of the network\n\n### What are Convolutional Neural Networks?\n\n> Pixels are most useful in the context of their neighbors. Objects in images are made up of small, localized features. Convolutional layers consist of a set of filters, which you can think of as just 2d matrices of numbers. We can use an input image and a filter to produce an output image by convolving the filter with the input image. In general, convolution helps us look for specific localized image features (like edges) that we can use later in the network. **Padding:** often, we would prefer to have the output image be the same size as the input image. To do this, we add zeros around the image so we can overlay the filter in more places. This is called “same” padding, since the input and output have the same dimensions. Not using any padding, which is what we’ve been doing and will continue to do for this post, is sometimes referred to as “valid” padding. CNNs include convolutional layers that use a set of filters to turn input images into output images. A convolutional layer’s primary parameter is the number of filters it has. **Pooling layers** reduce the size of the input it’s given by pooling values together in the input."},{"metadata":{"trusted":true},"cell_type":"code","source":"from __future__ import print_function\nimport keras\nfrom keras.datasets import mnist\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Dropout, Flatten\nfrom keras.layers import Conv2D, MaxPooling2D\nfrom keras import backend as K\n\ndef define_model():\n\n    model = Sequential()\n\n    # Kernel size is the size of the filter matrix for our convolution.\n    model.add(Conv2D(filters = 64, kernel_size = (5,5), padding ='Same', activation ='relu', input_shape = (28,28,1)))\n    model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2)))\n    model.add(Conv2D(filters = 64, kernel_size = (5,5), padding ='Same', activation ='relu'))\n    model.add(MaxPooling2D(pool_size=(2,2)))\n    model.add(Dropout(0.25))\n\n    model.add(Conv2D(filters = 32, kernel_size = (2,2), padding ='Same', activation ='relu'))\n    model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2)))\n    model.add(Conv2D(filters = 32, kernel_size = (2,2), padding ='Same', activation ='relu'))\n    model.add(MaxPooling2D(pool_size=(2,2), strides=(2,2)))\n    model.add(Dropout(0.25))\n\n    # Flatten layer serves as a connection between the convolution and dense layers.\n    model.add(Flatten())\n\n    #Dense is a standard layer type\n    model.add(Dense(128, activation = \"relu\"))\n    model.add(Dropout(0.15))\n    model.add(Dense(64, activation = \"relu\"))\n    model.add(Dense(10, activation = \"softmax\"))\n    model.compile(loss=keras.losses.categorical_crossentropy, optimizer=keras.optimizers.Adagrad(lr=0.01), metrics=['accuracy'])\n    #model.compile(loss=keras.losses.categorical_crossentropy, optimizer=keras.optimizers.RMSprop(lr=0.001, rho=0.9, epsilon=1e-08, decay=0.0), metrics=['accuracy'])\n    \n    return model","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 3. Train the model"},{"metadata":{"trusted":true},"cell_type":"code","source":"fit = model.fit(train_data, train_labels, batch_size=32, epochs=30, verbose=1, validation_split=0.05, shuffle=True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 4. Make predictions on a test dataset"},{"metadata":{"trusted":true},"cell_type":"code","source":"predictions = model.predict_classes(test_data, batch_size=32, verbose=0)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 5. Write the subsmission file"},{"metadata":{"trusted":true},"cell_type":"code","source":"submissions = pd.DataFrame({'ImageId':list(range(1,len(predictions) + 1)), \"Label\": predictions})\nsubmissions.to_csv(\"DR.csv\", index=False, header=True)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}