{"cells":[{"metadata":{"_cell_guid":"94fe1125-1985-4d4e-85a6-88e2a021c068","_uuid":"9334801b37c78e693f84cd0657a220c458b6cf47"},"cell_type":"markdown","source":"# MNIST Autoencoder Experiments 1\n** March 2018 **\n\n** Andrew Riberio @ [github](https://github.com/Andrewnetwork) **\n\nThis is an interactive notebook for exploring a simple non-convolutional mnist autoencoder. In this lab we look at how an autoencoder is able to build up a representation of the zero digit class. We explore through interactive widgets various properties, including the denosing behavior of our autoencoder ( it makes 0 look more like 0's and other digits look zeroish ). \n\n**Note**: This is an interactive notebook and requires you to fork this kernel to run the interactive elements or download a local version of this notebook and run it with jupyter notebooks or jupyter lab. You will need to have the dependencies listed. \n\n\nResources\n* https://www.tensorflow.org/get_started/mnist/beginners\n* https://blog.keras.io/building-autoencoders-in-keras.html"},{"metadata":{"_cell_guid":"bd3e851f-e083-48c8-88ae-f94131f3b3e1","_uuid":"bcde0ce0f11142543cbf6d047d2bf081b95c2cac"},"cell_type":"markdown","source":"## 1. Libraries and Config"},{"metadata":{"_cell_guid":"73733fd8-6784-4430-bd6e-91fbe9b7d7d3","_uuid":"12dec2d6ae6de4ada89be23ff06db782c336cae9","collapsed":true,"trusted":false},"cell_type":"code","source":"import matplotlib.pyplot as plt\nimport numpy as np\nfrom keras.layers import Input, Dense, Conv2D, MaxPooling2D, UpSampling2D\nfrom keras.models import Model\nfrom keras import losses\nfrom keras.callbacks import EarlyStopping\nfrom tensorflow.examples.tutorials.mnist import input_data\nfrom keras import backend as K\nimport tensorflow as tf\nfrom ipywidgets import interact_manual,interact\nimport pandas as pd\nfrom keras.models import load_model","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"09e88ebd-3224-4b63-81af-76af3bd042d7","_uuid":"f46ddb418cb4d20a14759c672c051275d0da9a06"},"cell_type":"markdown","source":"#### 1.2. Start tensorflow session and have keras use it. "},{"metadata":{"_cell_guid":"2b8c5cbe-9aa4-42be-8c26-d22dc033cf4f","_uuid":"343cb2cde25feb6e08bcb0803d96353aee6aa16c","collapsed":true,"trusted":false},"cell_type":"code","source":"sess = tf.Session(config=tf.ConfigProto(allow_soft_placement=True,log_device_placement=True,device_count = {'CPU' : 1, 'GPU' : 1}))\nK.set_session(sess)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"49c1ca4f-6896-44a2-8929-82ef78bd73e0","_uuid":"9549ef13c22b5aa5c2578d03fe151ca3de380304"},"cell_type":"markdown","source":"## 2. Gathering and Working with MNIST Data"},{"metadata":{"_cell_guid":"c1edb045-4ad6-450c-9797-d41c4f3dbf78","_uuid":"a811e24324d7f6a2bba27fdd3f4fbfc49cfe3b2b"},"cell_type":"markdown","source":"#### 2.1. Loading the dataset object into the mnist variable and printing the dimensions of the data matrix and label vector. "},{"metadata":{"_cell_guid":"fa96cee9-f516-46ee-9b8e-b33aba670cdf","_uuid":"2534ac6f8a90c78cc26e3857960ef8edd48f771d","collapsed":true,"trusted":false},"cell_type":"code","source":"mnist = input_data.read_data_sets(\"MNIST_data/\", one_hot=False)\nprint(\"Shape of the image data matrix: {0}\".format(mnist.train.images.shape))\nprint(\"Shape of the label data: {0}\".format(mnist.train.labels.shape))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"90643fa5-61a4-4d7e-a0d0-cf673dca6192","_uuid":"c8a38025f9df5c3a8a9c2eee256102f7e3815890"},"cell_type":"markdown","source":"####  2.2. Visualizing an image in our training set. "},{"metadata":{"_cell_guid":"cc384f00-1fda-4bb9-8775-521bb3d87521","_uuid":"d48cb9bf945606c8d34b49373ef06a915e81efa2","trusted":false,"collapsed":true},"cell_type":"code","source":"exampleIndex = 3\nexImage      = mnist.train.images[exampleIndex ,:]\nexImageLabel = mnist.train.labels[exampleIndex]\nplt.imshow(exImage.reshape(28,28),cmap='gray')\nplt.show()\nprint(\"Image label: {0}\".format(exImageLabel))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"f6c30fbc-22ac-4e7a-b590-437a46d6884f","_uuid":"85a6b347621679f9c03ec298870fe8dd5f6d611e"},"cell_type":"markdown","source":"#### 2.3. Split images into digit class tensors."},{"metadata":{"_cell_guid":"bcef8db6-053b-4d4d-9264-bbbee369fe68","_uuid":"dbcb6016d0f5c518995db5c2363cc3d1c5c04091","collapsed":true,"trusted":false},"cell_type":"code","source":"# First we will zip the training labels with the training images\ndataWithLabels = zip(mnist.train.labels, mnist.train.images)\n\n# Now let's turn this into a dictionary where subsets of the images in respect \n# to digit class are stored via the corresponding key.\n\n# Init dataDict with keys [0,9] and empty lists.\ndigitDict = {}\nfor i in range(0,10):\n    digitDict[i] = []\n\n# Assign a list of image vectors to each corresponding digit class index. \nfor i in dataWithLabels:\n    digitDict[i[0]].append(i[1])\n\n# Convert the lists into numpy matricies. (could be done above, but I claim ignorace)\nfor i in range(0,10):\n    digitDict[i] = np.matrix(digitDict[i])\n    print(\"Digit {0} matrix shape: {1}\".format(i,digitDict[i].shape))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"15158afc-c8b4-499f-99e9-1f2985311d29","_uuid":"d1ee22acfdc8b9d9d10e415f06514775bfeafa24"},"cell_type":"markdown","source":"## 3. Anomoly Detection for Non-Zero Class Digits"},{"metadata":{"_cell_guid":"776aa4f6-b7b3-45eb-94ab-9290426fa62f","_uuid":"27f1105e72437f42abe39bd8c8d48af80e15d29c"},"cell_type":"markdown","source":"#### 3.1. Compile a simple single layer non-convolutional autoencoder for the digit class 0. "},{"metadata":{"_cell_guid":"16a9679f-e437-4587-ac4c-5a54f9e5a74b","_uuid":"04f324b253dd55c51c47571b1bb52da3aff14807","collapsed":true,"trusted":false},"cell_type":"code","source":"def simpleAE(encoding_dim = 32, input_dim = 784):\n\n    # this is our input placeholder\n    input_img = Input(shape=(input_dim,))\n    # \"encoded\" is the encoded representation of the input\n    encoded = Dense(encoding_dim, activation='relu')(input_img)\n    # \"decoded\" is the lossy reconstruction of the input\n    decoded = Dense(input_dim, activation='sigmoid')(encoded)\n\n    # this model maps an input to its reconstruction\n    return Model(input_img, decoded)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"a99bde5e-593b-401d-8a5b-1586dbba9d5e","_uuid":"1652b1bfddc4bc733cee43022a363e63f16bd76a","collapsed":true,"trusted":false},"cell_type":"code","source":"ae1 = simpleAE(64,784)\nae1.compile(optimizer='adadelta', loss='binary_crossentropy')","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"c6aa5d3f-6ebe-4c1b-b5ec-d29dfd697718","_uuid":"6d90d42db76c1f7b90f9a618f3407f90237c6ebd"},"cell_type":"markdown","source":"#### 3.2. Train the autoencoder and log the loss over epoch history. "},{"metadata":{"_cell_guid":"80c2f47b-0d4b-42be-a88f-023b684cd483","_uuid":"ba15a5650f448d3086a1f436fe739d74d30d9145","collapsed":true,"trusted":false},"cell_type":"code","source":"modelURI = \"models/mnist_zero_autoencoder_1_64.h5\"\n\nprint(\"Loading model...\")\n\ntry:\n    # Load model if found. \n    ae1 = load_model(modelURI)\n    print(\"Model found and loaded.\")\nexcept:\n    # Train model if model cannot be loaded. \n    print(\"Model not found. Training model..\")\n    \n    history = ae1.fit(digitDict[0], digitDict[0],\n                    epochs=100,\n                    batch_size=20,\n                    shuffle=True,verbose=1)\n    \n    # Save our Model\n    print(\"Model Saved.\")\n    ae1.save(modelURI)\n    \n    # summarize history for loss\n    plt.plot(history.history['loss'])\n    plt.title('model loss')\n    plt.ylabel('loss')\n    plt.xlabel('epoch')\n    plt.legend(['train', 'test'], loc='upper left')\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"04a202ac-6a3c-4f78-9782-35bb0b67cd96","_uuid":"56b5e7b4062a6bb6b6d85fa54375cfa03afbd52e"},"cell_type":"markdown","source":"#### 3.3. Explore trained autoencoder reconstruction results ( Interactive )"},{"metadata":{"_cell_guid":"cd456365-ca06-48d7-9dc6-e3eaf3754bf8","_uuid":"73be4fafc2043175c8431c69d20bec3ae044a853","trusted":false,"collapsed":true},"cell_type":"code","source":"def dispRes1(digitClass=0,nthDigit=0):\n    actual = digitDict[digitClass][nthDigit,:]\n    pred = ae1.predict(actual)\n    res = losses.mean_absolute_error(actual,pred)\n    \n    fig = plt.figure(figsize=(10, 10))\n    fig.suptitle(\"Mean Absolute Reconstruction Error: {0:.5}%\".format(sess.run(res)[0]*100), fontsize=16,y=0.73)\n    \n    ax1 = fig.add_subplot(1,2,1)\n    ax1.axis('off')\n    ax1.set_title(\"Original Image\")\n    ax1.imshow(actual.reshape(28,28),cmap=\"gray\")\n    \n    ax2 = fig.add_subplot(1,2,2)\n    ax2.axis('off')\n    ax2.set_title(\"Reconstructed Image\")\n    ax2.imshow(pred.reshape(28,28),cmap=\"gray\")\n    \n    plt.show()\n    \ninteract_manual(dispRes1,digitClass=(0,9),nthDigit=(0,1000))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"58fb3fad-8401-4aa6-8fb8-dc8941c35cf5","_uuid":"bf2e1eec4f3220378db3f2ade447bdc07382b3cc"},"cell_type":"markdown","source":"#### 3.4. Reconstruction Error Distributions ( Interactive )\nWe will now explore the reconstruction error distributions over the digit classes. "},{"metadata":{"_cell_guid":"1b454c6e-f86e-414c-b7a6-b536b0566940","_uuid":"7da3818f7463b3ff1e10dba379144cf6b8e74490","collapsed":true,"trusted":false},"cell_type":"code","source":"classLosses = []\nfor digitClass in range(0,10):\n    lossTensor = losses.mean_absolute_error(digitDict[digitClass],ae1.predict(digitDict[digitClass]))\n    res = sess.run(lossTensor)\n    res.sort()\n    classLosses.append(res)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"4a320b7f-7f8b-4b08-95f2-41419cadba6e","_uuid":"50f43d71f414dadbc604228084fa257ef2ab7d98","trusted":false,"collapsed":true},"cell_type":"code","source":"def dispDists(digitClass=0):\n    l = len(classLosses[digitClass])\n    plt.figure(figsize=(6, 6))\n    plt.hist(classLosses[digitClass],bins=20,range=(0, 0.125))\n    plt.show()\n    \ninteract(dispDists,digitClass=(0,9))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"1ac28be7-a0db-45c1-b629-3b6768930dbb","_uuid":"d7a16ff440bd8e3f51bc1fdea7ea68bffd04cf6a"},"cell_type":"markdown","source":"We will now plot the histograms of the reconstruction error distributions over the digit classes."},{"metadata":{"_cell_guid":"eac1474d-f37c-4027-ad15-4b20f3950cea","_uuid":"464aec4daf28d39d05b94696b0f4b9793577bc8a","trusted":false,"collapsed":true},"cell_type":"code","source":"plt.figure(figsize=(10, 10))\nfor digitClass in range(0,10):\n    l = len(classLosses[digitClass])\n    plt.legend(['0','1','2','3','4','5','6','7','8','9'], loc='upper right')\n    plt.hist(classLosses[digitClass],bins=20,range=(0, 0.125))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"9505d878-83e7-4db2-a350-716c93514d35","_uuid":"30809e32f33cfb3afa0715cc01609924c39d5d63","trusted":false,"collapsed":true},"cell_type":"code","source":"# Summary Statistics for Digit Reconstruction Loss\npd.DataFrame(classLosses).transpose().describe()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"bbef559f-2b9f-478b-b619-4b008dc5f8a6","_uuid":"024c28edebfee6e0819a137bb425308f2ef21ec5"},"cell_type":"markdown","source":"#### 3.5. Classify reconstruction loss. \n**3.5.1. Create loss classifier network architecture **"},{"metadata":{"_cell_guid":"3389679a-8583-42ec-b178-5789fedd5ab6","_uuid":"366f327a34f68f1d97518d7bad7c0b02e4863a90","collapsed":true,"trusted":false},"cell_type":"code","source":"def simpleNN(encoding_dim = 10, input_dim = 1):\n\n    # this is our input placeholder\n    input_img = Input(shape=(input_dim,))\n    # \"encoded\" is the encoded representation of the input\n    encoded = Dense(encoding_dim, activation='relu')(input_img)\n    # \"decoded\" is the lossy reconstruction of the input\n    decoded = Dense(1, activation='sigmoid')(encoded)\n\n    # this model maps an input to its reconstruction\n    return Model(input_img, decoded)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"ba0ece69-af15-4d10-87f6-e4a46f5b719d","_uuid":"d40169c2524648182d2a575bb3a9a60dc1a7d797"},"cell_type":"markdown","source":"**3.5.2. Preprocess data for classifier**"},{"metadata":{"_cell_guid":"d36eb0da-ba80-45c4-95a0-4ca3ce87343f","_uuid":"c981727529f33501ff08cb8c168ea3bfbf7733fc","collapsed":true,"trusted":false},"cell_type":"code","source":"# Preprocess Data for Classifier\nzeroClass = classLosses[0]\notherClass = np.concatenate(classLosses[1:10])\nlabels = np.concatenate([np.zeros(len(zeroClass)),np.ones(len(otherClass))])\ndata = np.concatenate([zeroClass,otherClass])","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"9bbe59d1-1437-449f-9ea6-ad856f58df58","_uuid":"74b7df0cf663b3ec1e7796fab8b166dae33bd5d4"},"cell_type":"markdown","source":"**3.5.3. Train loss classifier**"},{"metadata":{"_cell_guid":"1b8e4673-cc1d-43fd-bab4-619214d81ae3","_uuid":"f42918e7908a79f4a8dac213e88c3e79e3893a24","collapsed":true,"trusted":false},"cell_type":"code","source":"modelURI = \"models/reconstruction_loss_1_64.h5\"\n\nprint(\"Loading model...\")\n\ntry:\n    # Load model if found. \n    snn = load_model(modelURI)\n    print(\"Model found and loaded.\")\nexcept:\n    # Train model if model cannot be loaded. \n    print(\"Model not found. Training model..\")\n    \n    snn = simpleNN()\n    snn.compile(optimizer='adadelta', loss='binary_crossentropy')\n    history = snn.fit(data, labels,\n                        epochs=25,\n                        batch_size=20,\n                        shuffle=True,verbose=1)\n    \n    # Save our Model\n    print(\"Model Saved.\")\n    snn.save(modelURI)\n    \n    # summarize history for loss\n    plt.plot(history.history['loss'])\n    plt.title('model loss')\n    plt.ylabel('loss')\n    plt.xlabel('epoch')\n    plt.legend(['train', 'test'], loc='upper left')\n    plt.show()\n","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"f5828713-c569-4864-b7ad-485b51bfa7da","_uuid":"31a23d85a9928ebcc960bce21aa8b65dcaaacda2"},"cell_type":"markdown","source":"**3.5.4. Visualize results.**"},{"metadata":{"_cell_guid":"184006bf-9945-4285-95b5-d90d54028cc7","_uuid":"8dd142ce6a682d6e67f660aadc4ef09bb7110773","trusted":false,"collapsed":true},"cell_type":"code","source":"def dispRes1(digitClass=0,nthDigit=0):\n    classificationThresh = 0.5\n    actual = digitDict[digitClass][nthDigit,:]\n    pred = ae1.predict(actual)\n    res = losses.mean_absolute_error(actual,pred)\n    res = sess.run(res)\n    \n    fig = plt.figure(figsize=(10, 10))\n    fig.suptitle(\"Mean Absolute Reconstruction Error: {0:.5}%\".format(res[0]*100), fontsize=16,y=0.73)\n    \n    ax1 = fig.add_subplot(1,2,1)\n    ax1.axis('off')\n    ax1.set_title(\"Original Image\")\n    ax1.imshow(actual.reshape(28,28),cmap=\"gray\")\n    \n    ax2 = fig.add_subplot(1,2,2)\n    ax2.axis('off')\n    ax2.set_title(\"Reconstructed Image\")\n    ax2.imshow(pred.reshape(28,28),cmap=\"gray\")\n    \n    plt.show()\n    \n    snnRes = snn.predict(res)[0][0]\n    \n    if snnRes < classificationThresh:\n        print(\"Classification 0 : [{0}]\".format( 1-snnRes ))\n    else:\n        print(\"Classification ANOMALY : [{0}]\".format( snnRes ))\n        \n    \n    \n    \ninteract_manual(dispRes1,digitClass=(0,9),nthDigit=(0,1000))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"908b0892-8ae9-4e49-aa8b-6114ea41a655","_uuid":"f12efca40f731458707cf058985d04d677ed0aba"},"cell_type":"markdown","source":"**3.5.5. Evaluate the classification accuracy**"},{"metadata":{"_cell_guid":"08600e9a-a341-49da-94e6-08de998fe019","_uuid":"4c64fd3d3afccb462c45e4c88408990b4c460909","collapsed":true,"trusted":false},"cell_type":"code","source":"classificationThresh = 0.5\n\n# Get all the training images. \nactual = mnist.train.images\n\n# Pass all the images through the trained zero digit autoencoder. \npred = ae1.predict(actual)\n\n# Calculuate the reconstruction loss for each image passed through the autoencoder. \nres = losses.mean_absolute_error(actual,pred)\n\n# Use tensorflow to compute the value. \nres = sess.run(res)\n\n# Pass the losses through the loss classifier. \nsnnRes = snn.predict(res)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"5ae669c0-fadd-4d99-94a7-f48848393e49","_uuid":"54bd4db12c8a2ff32c53bd95faa992db6aa28055","collapsed":true,"trusted":false},"cell_type":"code","source":"def tresholdFn( loss ):\n    if loss < classificationThresh:\n        return 0\n    else:\n        return 1\n\npredictions = np.vectorize(tresholdFn)(snnRes)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"534f8446-fee9-4f9f-9c86-b56eff137421","_uuid":"25f8a0a244c33fe898fa8b5a95dc53976dd0daa7","collapsed":true,"trusted":false},"cell_type":"code","source":"nData = mnist.train.labels.shape[0]\nnFalsePos = 0\nfalsePosIdxs = []\nnFalseNeg = 0\nfalseNegIdxs = []\n\nfor index,prediction,label,in zip(range(nData),predictions,mnist.train.labels):\n    # Compute the number of false negatives. \n    if label == 0 and prediction == 1:\n        nFalseNeg += 1\n        falseNegIdxs.append(index)\n        \n    # Compute the number of false positives. \n    if label != 0 and prediction == 0:\n        nFalsePos += 1\n        falsePosIdxs.append(index)\n    \nprint(\"Number of false negatives: {0}\".format(nFalseNeg))\nprint(\"Percentage of false negatives: {0:.3}%\".format(nFalseNeg/digitDict[0].shape[0]*100))\nprint(\"---\")\nprint(\"Number of false positives: {0}\".format(nFalsePos))\nprint(\"Percentage of false positives: {0:.6}%\".format(nFalsePos/otherClass.shape[0]*100))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"4061a0ec-5250-4479-b48f-10fd61c59944","_uuid":"9144dbfcdd786f2013c9bfa92e71cc2aae372fbc","collapsed":true,"trusted":false},"cell_type":"code","source":"autoResults = ae1.predict(mnist.train.images)\nlossRes = losses.mean_absolute_error(mnist.train.images,autoResults)\nlossRes = sess.run(lossRes)","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"a952b4ab-98bb-4583-a46d-152c6cbf8901","_uuid":"02ba1ecf05e5d66e0fb0baeda8beee20d82cb508"},"cell_type":"markdown","source":"Visualize False Positives. False positives are non-zero characters that are classified as zeros. "},{"metadata":{"_cell_guid":"63392faa-dc2c-48ac-b00a-2896d6f631b9","_uuid":"e7176c5e5642763d36ea0101d74a0f5b966d5129","trusted":false,"collapsed":true},"cell_type":"code","source":"def displayFalsePos(idx = 0):\n    classificationThresh = 0.5\n    \n    indexVis = falsePosIdxs[idx]\n    \n    fig = plt.figure(figsize=(10, 10))\n    fig.suptitle(\"False Positives - Mean Absolute Reconstruction Error: {0:.5}%\".format(lossRes[indexVis]*100), fontsize=16,y=0.73)\n    \n    ax1 = fig.add_subplot(1,2,1)\n    ax1.axis('off')\n    ax1.set_title(\"Original Image\")\n    ax1.imshow(mnist.train.images[indexVis,:].reshape(28,28),cmap=\"gray\")\n    \n    ax2 = fig.add_subplot(1,2,2)\n    ax2.axis('off')\n    ax2.set_title(\"Reconstructed Image\")\n    ax2.imshow(autoResults[indexVis,:].reshape(28,28),cmap=\"gray\")\n    \n    plt.show()\n    \n    print(\"Absolute index value: {0}.\".format(indexVis))\n    print(\"Image label: {0}.\".format(mnist.train.labels[indexVis]))\n        \ninteract(displayFalsePos,idx = (0,len(falsePosIdxs)-1))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"051b3b67-5098-4c43-8f66-b95413bfd627","_uuid":"5079ddb689aca4c4c4f71ac362adb2b1db1a0a01","trusted":false,"collapsed":true},"cell_type":"code","source":"def displayFalseNeg(idx = 0):\n    classificationThresh = 0.5\n    \n    indexVis = falseNegIdxs[idx]\n    \n    fig = plt.figure(figsize=(10, 10))\n    fig.suptitle(\"False Negatives - Mean Absolute Reconstruction Error: {0:.5}%\".format(lossRes[indexVis]*100), fontsize=16,y=0.73)\n    \n    ax1 = fig.add_subplot(1,2,1)\n    ax1.axis('off')\n    ax1.set_title(\"Original Image\")\n    ax1.imshow(mnist.train.images[indexVis,:].reshape(28,28),cmap=\"gray\")\n    \n    ax2 = fig.add_subplot(1,2,2)\n    ax2.axis('off')\n    ax2.set_title(\"Reconstructed Image\")\n    ax2.imshow(autoResults[indexVis,:].reshape(28,28),cmap=\"gray\")\n    \n    plt.show()\n    \n    print(\"Absolute index value: {0}.\".format(indexVis))\n    print(\"Image label: {0}.\".format(mnist.train.labels[indexVis]))\n        \ninteract(displayFalseNeg,idx = (0,len(falseNegIdxs)-1))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"2444b404-d1a7-4440-9ff5-4acff3b390b4","_uuid":"d19ff4a0d82aea30b16d9f2c3b647a47b74a2b92"},"cell_type":"markdown","source":"Number of false negatives: 757\nPercentage of false negatives: 13.9%\n---\nNumber of false positives: 394\nPercentage of false positives: 0.79506%"},{"metadata":{"_cell_guid":"e2d7d416-b0fa-423e-9ca6-97625394245b","_uuid":"9e4307659e7389214bf7123ac263b2ce236c918b","collapsed":true,"trusted":false},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"language_info":{"version":"3.6.4","file_extension":".py","codemirror_mode":{"version":3,"name":"ipython"},"mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3"},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"}},"nbformat":4,"nbformat_minor":1}