{"cells":[{"metadata":{"_uuid":"eca212d6fceb041a191a8a1ce8a429b2dd41553a"},"cell_type":"markdown","source":"<center>\n<h1> Danbury AI June 2018: Workshop Part 4</h1>\n<h2>Street View House Numbers</h2>\n</center>\nSVHN is a real-world image dataset for developing machine learning and object recognition algorithms with minimal requirement on data preprocessing and formatting. It can be seen as similar in flavor to MNIST (e.g., the images are of small cropped digits), but comes from a significantly harder, unsolved, real world problem (recognizing digits and numbers in natural scene images). SVHN is obtained from house numbers in Google Street View images.\n\n* 10 classes, 1 for each digit. Digit '1' has label 1, '9' has label 9 and '0' has label 10.\n*  MNIST-like 32-by-32 images centered around a single character (many of the images do contain some distractors at the sides).\n\nYuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, Andrew Y. Ng Reading Digits in Natural Images with Unsupervised Feature Learning NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011. \n\nWeb source: http://ufldl.stanford.edu/housenumbers/\n\nHere is a selection of what the individual training images look like:\n![](http://ufldl.stanford.edu/housenumbers/32x32eg.png)"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"collapsed":true},"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport scipy.io as sio\nimport os\nimport matplotlib.pyplot as plt\nfrom ipywidgets import interact\nfrom keras.layers import Input, Dense, Dropout,Conv2D,MaxPooling2D,Flatten\nfrom keras.layers import GlobalMaxPooling2D,UpSampling2D,GlobalMaxPooling1D\nfrom keras.models import Model\nfrom keras.utils import to_categorical\nfrom sklearn.model_selection import train_test_split\n\n# Print out the folders where our datasets live. \nprint(\"Datasets: {0}\".format(os.listdir(\"../input/danbury-ai-june-2018\")))","execution_count":29,"outputs":[]},{"metadata":{"_uuid":"1134e71f2b333e7dc58ede0f07b79a3361b9be6b"},"cell_type":"markdown","source":"Here we load in numpy matricies for the training images (train_x), training labels (train_y), and test images (test_x). "},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true,"collapsed":true},"cell_type":"code","source":"# The training images. \nX = np.load(\"../input/danbury-ai-june-2018/train_x.npy\")\ny = np.load(\"../input/danbury-ai-june-2018/train_y.npy\")\n\n# We subtract 1 from the labels in order to scale the the labels between 0,9. \ny = y - 1\n\n# These are the images we will need to predict lables for. \ntest  = np.load(\"../input/danbury-ai-june-2018/test.npy\")","execution_count":34,"outputs":[]},{"metadata":{"_uuid":"3bfb5bd58a9a94e29090c5e43a9dce52791f0057"},"cell_type":"markdown","source":"We will now do our standard training/validation split so we can evaluate our model's generalization characteristics. "},{"metadata":{"trusted":true,"_uuid":"99b12de614d841cd97104f2052de0b99c208748b","collapsed":true},"cell_type":"code","source":"X_train, X_validation, y_train , y_validation = train_test_split(X,to_categorical(y,10), test_size=0.2)","execution_count":35,"outputs":[]},{"metadata":{"_uuid":"5848e8d32764eb481974a560f3c8c287260e8afb"},"cell_type":"markdown","source":"Here we define a convolutional neural network with a max pooling layer and gobal pooling layer. "},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"fa16a2434fb3946e88f5e42972de7fe63262c95f"},"cell_type":"code","source":"def makeModel(inputSize):\n    inputs = Input(shape=inputSize,name=\"input\")\n    x = Conv2D(32, (3, 3), padding='same', activation='relu')(inputs)\n    x = MaxPooling2D(pool_size=(2, 2), strides=None, padding='valid')(x)\n    x = Conv2D(32, (3, 3), padding='same', activation='relu')(x)\n    x = GlobalMaxPooling2D()(x)\n    out = Dense(10,activation='softmax', name=\"output\")(x)\n\n    model = Model(inputs=inputs, outputs=out)\n    model.compile(optimizer=\"adam\",loss=\"categorical_crossentropy\",metrics=['acc'])\n    \n    return model","execution_count":36,"outputs":[]},{"metadata":{"_uuid":"55aa0fcb42c0309b4654125c7d6d54beb9f5c57f"},"cell_type":"markdown","source":"We will now train our model. "},{"metadata":{"trusted":true,"_uuid":"1e1719476303c1d504c04d808c4efbece9573fc6","collapsed":true},"cell_type":"code","source":"model2 = makeModel((32,32,3,))\nmodel2.summary()\nhist2 = model2.fit(X_train, y_train, batch_size=100,epochs=10, validation_data=(X_validation,y_validation))","execution_count":37,"outputs":[]},{"metadata":{"_uuid":"5bea008d3a556e9cb6ecdecf1be86f3944a9f80b"},"cell_type":"markdown","source":"Now that we have trained our network, let's look at the training loss over the training epochs. "},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"ac065a07803c4a80a6ed0f3c99a56cd3be1a2971"},"cell_type":"code","source":"def learningCurves(hist):\n    histAcc_train = hist.history['acc']\n    histLoss_train = hist.history['loss']\n    histAcc_validation = hist.history['val_acc']\n    histLoss_validation = hist.history['val_loss']\n    maxValAcc = np.max(histAcc_validation)\n    minValLoss = np.min(histLoss_validation)\n\n    plt.figure(figsize=(12,12))\n    epochs = len(histAcc_train)\n\n    plt.plot(range(epochs),histLoss_train, label=\"Training Loss\", color=\"#acc6ef\")\n    plt.plot(range(epochs),histLoss_validation, label=\"Validation Loss\", color=\"#a7e295\")\n\n    plt.scatter(np.argmin(histLoss_validation),minValLoss,zorder=10,color=\"green\")\n\n    plt.xlabel('Epochs',fontsize=14)\n    plt.title(\"Learning Curves\",fontsize=20)\n\n    plt.legend()\n    plt.show()\n\n    print(\"Max validation accuracy: {0}\".format(maxValAcc))\n    print(\"Minimum validation loss: {0}\".format(minValLoss))\n    \nlearningCurves(hist2)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3f57e5894047d31dba8a18916b7dc42009fb117e"},"cell_type":"markdown","source":"Predict the class lables with our model "},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"8ff83aa2fdfa4bf76643ebeb03ed596e9ee4b85d"},"cell_type":"code","source":"pred = model2.predict(test)\npred = np.argmax(pred,1)\npred = pred + 1","execution_count":38,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"edd758d15b34a45a7b85ab37aa195a166f6e1fcb"},"cell_type":"code","source":"submission = pd.DataFrame.from_items([\n    ('id',list(range(pred.shape[0]))),\n    ('label', pred)])\n\nsubmission.to_csv('submission.csv', index = False)","execution_count":39,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}