{"metadata": {"kernelspec": {"display_name": "Python 3", "name": "python3", "language": "python"}, "language_info": {"mimetype": "text/x-python", "version": "3.6.3", "name": "python", "pygments_lexer": "ipython3", "file_extension": ".py", "codemirror_mode": {"version": 3, "name": "ipython"}, "nbconvert_exporter": "python"}}, "nbformat_minor": 1, "nbformat": 4, "cells": [{"source": ["<h2>Neural Network with Logistic Regression</h2>\n", "\n", "As the title says, this is a logistic regression neural network, the final accuracy is of ~85%.\n", "\n", "<b>First we start by importing needed libraries for this project.</b>"], "metadata": {"_cell_guid": "d613aeb7-f119-4d2d-8f46-c904176a0709", "_uuid": "1edbde3fab2080e1570029a671bb251839759951"}, "cell_type": "markdown"}, {"source": ["import numpy as np\n", "import csv\n", "import matplotlib.pyplot as plt\n", "import matplotlib.cm as cm\n", "from random import randint\n", "%matplotlib inline"], "metadata": {"_cell_guid": "8d09a4b2-fd95-4b5c-89ac-ffb60a049d20", "_uuid": "ad17bf2e793b7b43c2a8646dc1886b47d6fc9b2b", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["<b>Let's define the functions that we will need in this neural network.</b>\n", "<br/><br/><br/>\n", "<b>We'll first start with the forward prop functions, and then do the same for the backward prop.</b>"], "metadata": {"_cell_guid": "7eafe0ca-945a-4e03-9624-1c6efb433e96", "_uuid": "6d786fab918326d5902f7865e53c731ae4612a4a"}, "cell_type": "markdown"}, {"source": ["function linear_forward, calculates Z which is by definition WX + b, in our case X will be replaced by A which is the activation matrix for this layer. linear forward then saves A,W,b in a cache variable, so that we don't have to calculate them each time, and then returns Z and cache."], "metadata": {"_cell_guid": "ddfcca50-d94e-4305-906b-62647225ac06", "_uuid": "9d8cc67f324c7cfb67e03a9cd884fdbdb8e0d9c3"}, "cell_type": "markdown"}, {"source": ["def linear_forward(A, W, b):\n", "    Z = np.dot(W, A) + b\n", "    \n", "    assert(Z.shape == (W.shape[0], A.shape[1]))\n", "    cache = (A, W, b)\n", "    \n", "    return Z, cache"], "metadata": {"_cell_guid": "8988c631-89fb-4ce8-98e2-c73a382f9aed", "_uuid": "5f0cac9d34f3fe8054c6115b64bda70fab0c69da", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["function linear_activation_forward will work based on the activation we send to it, in this case there are 3 activation functions, sigmoid, relu and tanh (tanh won't really be used now), linear_activation_forward will calculate Z, and linear cache using linear_forward function, then based on the activation we send to it, it will calculate A... We'll also save Z in activation_cache, so that we don't have to recalculate each time."], "metadata": {"_cell_guid": "a74e5bac-9eb4-424e-ab88-f748ea33f389", "_uuid": "5c319643226cccfe77a7c521cd74496252981438"}, "cell_type": "markdown"}, {"source": ["def linear_activation_forward(A_prev, W, b, activation):\n", "    Z, linear_cache = linear_forward(A_prev, W, b)\n", "    if activation == \"sigmoid\":\n", "        A, activation_cache = sigmoid(Z)\n", "    elif activation == \"relu\":\n", "        A, activation_cache = relu(Z)\n", "    elif activation == \"tanh\":\n", "        A, activation_cache = tanh(Z)\n", "    \n", "    assert (A.shape == (W.shape[0], A_prev.shape[1]))\n", "    cache = (linear_cache, activation_cache)\n", "\n", "    return A, cache"], "metadata": {"_cell_guid": "c839d06c-4dcb-4623-aed8-a3d932aef8ff", "_uuid": "4935075bf311e60d5bdee5e7fb1e9962695dc3c2", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["function L_model_forward is where all the logic for forward propagation lies. We'll use the functions we defined above to do all the forward propagation, and generate an output of the neural net AL."], "metadata": {"_cell_guid": "2d8ec8ef-8859-4686-bd13-50b6ec7d41c8", "_uuid": "1b004934fd37269ecc827701dcbafcd408164e8f"}, "cell_type": "markdown"}, {"source": ["def L_model_forward(X, parameters):\n", "    caches = []\n", "    A = X\n", "    L = len(parameters) // 2\n", "    \n", "    # Implement [LINEAR -> RELU]*(L-1). Add \"cache\" to the \"caches\" list.\n", "    for l in range(1, L):\n", "        A_prev = A \n", "        A, cache = linear_activation_forward(A_prev, parameters[\"W\"+str(l)], parameters[\"b\"+str(l)], 'relu')\n", "        caches.append(cache)\n", "    \n", "    AL, cache = linear_activation_forward(A, parameters[\"W\"+str(L)], parameters[\"b\"+str(L)], 'sigmoid')\n", "    caches.append(cache)\n", "    \n", "    assert(AL.shape == (10,X.shape[1]))\n", "            \n", "    return AL, caches"], "metadata": {"_cell_guid": "71c4fc34-a856-4468-b482-324b2eb5c627", "_uuid": "817ed486f969efeca0acdfc4f4962788c525a585", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["function compute cost function, computes the cost of our neural network, with L2 reguliraztion."], "metadata": {"_cell_guid": "1619d728-068e-47dd-9327-0896f80db9ab", "_uuid": "09046d5bf128544cbf9745ed40b4de541901e465"}, "cell_type": "markdown"}, {"source": ["def compute_cost(AL, Y, parameters, layers, lambd):\n", "    m = Y.shape[1]\n", "    p = 0\n", "    for i in range(1, layers):\n", "        p += np.sum(np.square(parameters[\"W\" + str(i)]))\n", "    logprobs = np.multiply(-np.log(AL),Y) + (lambd * p)/ (2 * m)\n", "    cost = 1./m * np.nansum(logprobs)\n", "    #cost = (- np.sum(np.dot(Y, np.log(AL.T))) / m) \n", "    cost = np.squeeze(cost)\n", "    assert(cost.shape == ())\n", "    return cost"], "metadata": {"_cell_guid": "a1f42cbc-4cf8-4bca-98b6-098683420e7a", "_uuid": "2cb93976604e401f6eea629d8d8ca639c0e8a67f", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["<b>Same as we did for forward propagation, we will do here, we will define 3 functions to do our backward propagation.</b> <br/><br/>\n", "linear_backward function gets A, W, b from our cache parameter, then calculates dW, db, and dA and returns them."], "metadata": {"_cell_guid": "f82fb352-5d7c-475a-8a40-648d332fcf81", "_uuid": "b2bd9e61292c6ba5cf8fb056cd9c81c50aebed76"}, "cell_type": "markdown"}, {"source": ["def linear_backward(dZ, cache, lambd):\n", "    A_prev, W, b = cache\n", "    m = A_prev.shape[1]\n", "\n", "    dW = (np.dot(dZ, A_prev.T) / m) + ((lambd * W) / m) \n", "    db = np.sum(dZ, axis = 1, keepdims = True) / m\n", "    dA_prev = np.dot(W.T, dZ)\n", "    \n", "    assert (dA_prev.shape == A_prev.shape)\n", "    assert (dW.shape == W.shape)\n", "    assert (db.shape == b.shape)\n", "    \n", "    return dA_prev, dW, db"], "metadata": {"_cell_guid": "33c93664-e4a9-4b30-97e7-9ae48258ea6f", "_uuid": "b52a1a9caa0073bfdffc54c95673bd8d9752ab40", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["linear_activation_backward takes in the dA, and cache, and activation method, then calculates dZ based on the activation we pass to it, after calculating dZ it calculates dA_prev, dW and db using linear_backward function defined above."], "metadata": {"_cell_guid": "c87db254-efe7-4094-93ee-6496aca05319", "_uuid": "8f9d0ea5d1601d377df4b7f6319321b7f14f4288"}, "cell_type": "markdown"}, {"source": ["def linear_activation_backward(dA, cache, activation, lambd):\n", "    linear_cache, activation_cache = cache\n", "    \n", "    if activation == \"relu\":\n", "        dZ = relu_backward(dA, activation_cache)\n", "    elif activation == \"sigmoid\":\n", "        dZ = sigmoid_backward(dA, activation_cache)\n", "    elif activation == \"tanh\":\n", "        dZ = tanh_backward(dA, activation_cache)\n", "    dA_prev, dW, db = linear_backward(dZ, linear_cache, lambd)\n", "    return dA_prev, dW, db"], "metadata": {"_cell_guid": "5b760b6e-d815-431c-81b4-ecd861a3e4ed", "_uuid": "d29735f58277056b1c343c6c5557937543ef05b6", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["function L_model_backward is where all the logic for backward propagation lies. We'll use the functions we defined above to do all the backward propagation, and generate the gradients for each of our layers."], "metadata": {"_cell_guid": "39bd444b-10d4-400e-8072-45f0ce859c20", "_uuid": "4386bc076c97dfb6acc44a5537142a386930b74e"}, "cell_type": "markdown"}, {"source": ["def L_model_backward(AL, Y, caches, lambd):\n", "    grads = {}\n", "    L = len(caches) # the number of layers\n", "    m = AL.shape[1]\n", "    Y = Y.reshape(AL.shape) # after this line, Y is the same shape as AL\n", "    \n", "    dAL = - (np.divide(Y, AL) - np.divide(1 - Y, 1 - AL)) \n", "    \n", "    current_cache = caches\n", "    grads[\"dA\" + str(L)], grads[\"dW\" + str(L)], grads[\"db\" + str(L)] = linear_activation_backward(dAL, current_cache[L-1], activation = \"sigmoid\", lambd=lambd)\n", "    \n", "    for l in reversed(range(L-1)):\n", "        current_cache = caches\n", "        dA_prev_temp, dW_temp, db_temp = linear_activation_backward(grads[\"dA\" + str(l+2)], current_cache[l], activation = \"relu\", lambd=lambd)\n", "        grads[\"dA\" + str(l + 1)] = dA_prev_temp\n", "        grads[\"dW\" + str(l + 1)] = dW_temp\n", "        grads[\"db\" + str(l + 1)] = db_temp\n", "    return grads"], "metadata": {"_cell_guid": "9922d9c3-6383-432e-b76e-816399c45e51", "_uuid": "f8d4c191427cb9fc9c9a5c8698ed84573086e18b", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["<b>Join them all together</b>\n", "<br /><br />\n", "Finally we join the whole logic together into L_layer_model function which will initialize our parameters and run forward and backward propagation using the functions defined above for a certain number of iterations provided as a parameter. Learning rate is also defined as a parameter for this function to be passed to update_parameters function."], "metadata": {"_cell_guid": "5eeee8f2-debb-44b1-8429-cbf7052e96a8", "_uuid": "510765dbfab3c99cf8ec3a7cfd29c6dd6142e45e"}, "cell_type": "markdown"}, {"source": ["def L_layer_model(X, Y, layers_dims, data_test,labels_test_dense, learning_rate = 0.0075, num_iterations = 3000, print_cost=False, lambd=0.1):#lr was 0.009\n", "    costs = []                         # keep track of cost\n", "    accuracies = []\n", "    v_accuracies = []\n", "    parameters = initialize_parameters_deep(layers_dims)\n", "    m = Y.shape[1]\n", "    # Loop (gradient descent)\n", "    for i in range(0, num_iterations):\n", "        AL, caches = L_model_forward(X, parameters)\n", "        cost = compute_cost(AL, Y, parameters, len(layers_dims), lambd)\n", "        grads = L_model_backward(AL, Y, caches, lambd)\n", "        parameters = update_parameters(parameters, grads, learning_rate)\n", "        if print_cost and i % 100 == 0:\n", "            print (\"Cost after iteration %i: %f\" %(i, cost))\n", "            costs.append(cost)\n", "            accuracy = float(np.sum(np.argmax(Y, axis = 0) == np.argmax(AL, axis = 0)))*100.0/float(42000)\n", "            accuracies.append(accuracy)\n", "            print(\"Train Accuracy: \"+str(accuracy))\n", "            validation = predict(parameters, data_test, layers_dims)\n", "            accuracy = (np.sum(validation == labels_test_dense) / testPart) * 100\n", "            v_accuracies.append(accuracy);\n", "            print(\"Validation Accuracy: \" + str(accuracy) + \"%\")\n", "            \n", "    # plot the cost and accuries\n", "    plt.plot(np.squeeze(costs), label=\"cost\")\n", "    plt.ylabel('cost')\n", "    plt.xlabel('iterations (per tens)')\n", "    plt.title(\"Learning rate =\" + str(learning_rate))\n", "    plt.legend()\n", "    plt.show()\n", "    plt.close()\n", "    plt.plot(np.squeeze(accuracies), label=\"training accuracy\")\n", "    plt.plot(np.squeeze(v_accuracies), label=\"validation accuracy\")\n", "    plt.ylabel('accuracies')\n", "    plt.xlabel('iterations (per tens)')\n", "    plt.title(\"Learning rate =\" + str(learning_rate))\n", "    plt.legend()\n", "    plt.show()\n", "    \n", "    return parameters"], "metadata": {"_cell_guid": "3257ffa7-cd6f-4833-a245-a8724c902bc7", "_uuid": "dac20ed98cc4a89bea11e2fc80572210d493d45e", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["<b>Now we will define our helpers functions</b>\n", "<br /><br />\n", "function initialize_parameters_deep will initialize W and b for the whole neural network, that's why we're passing layer_dims list to this function, layer_dims contains our network layers, and the nodes they have. W and b are randomly initialized for breaking the symmetry in our neural network."], "metadata": {"_cell_guid": "65f0fa09-7ef9-4ebb-8fa9-c0570f6a7d8a", "_uuid": "1d10a6328bbe2958175601bcc0397931e3ae6622"}, "cell_type": "markdown"}, {"source": ["def initialize_parameters_deep(layer_dims):\n", "    parameters = {}\n", "    L = len(layer_dims) # number of layers in the network\n", "    \n", "    for l in range(1, L):\n", "        parameters['W' + str(l)] = np.random.randn(layer_dims[l], layer_dims[l-1]) * 0.1\n", "        parameters['b' + str(l)] = np.zeros((layer_dims[l], 1))\n", "        \n", "        assert(parameters['W' + str(l)].shape == (layer_dims[l], layer_dims[l-1]))\n", "        assert(parameters['b' + str(l)].shape == (layer_dims[l], 1))\n", "\n", "        \n", "    return parameters"], "metadata": {"_cell_guid": "af9694f9-4b88-465b-a02e-c843acf3bc62", "_uuid": "f49228ec70c563f76e1a577a9a02293d4726e81d", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["function update_parameters will update the parameters of each layer in our neural net based on the gradients for that layer and the learning rate we provide."], "metadata": {"_cell_guid": "22832e1d-28ca-4898-9285-1d08ddf0ab27", "_uuid": "8d73197b4f70869ae53d59c3ec8edba1618d81c6"}, "cell_type": "markdown"}, {"source": ["def update_parameters(parameters, grads, learning_rate):\n", "    L = len(parameters) // 2 # number of layers in the neural network\n", "\n", "    for l in range(L):\n", "        parameters[\"W\" + str(l+1)] = parameters[\"W\" + str(l+1)] - (learning_rate * grads[\"dW\" + str(l+1)])\n", "        parameters[\"b\" + str(l+1)] = parameters[\"b\" + str(l+1)] - (learning_rate * grads[\"db\" + str(l+1)])\n", "        \n", "    return parameters"], "metadata": {"_cell_guid": "963d4ed1-38ab-4d33-b7f8-b04fc79d4e89", "_uuid": "cf1e3e3a9a32e98d37eef9f6c8780ab54602caaf", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["functions sigmoid, relu, tanh, sigmoid_backward, relu_backward, and tanh_backward are pretty understandable by their names..."], "metadata": {"_cell_guid": "7232c9b7-e82a-48aa-bf58-92388a3e7511", "_uuid": "02c2ef38094c8bb9fb477c4f8f603d0dd0ed1cb5"}, "cell_type": "markdown"}, {"source": ["def sigmoid(Z):\n", "    A = 1/(1+np.exp(-Z))\n", "    cache = Z\n", "    \n", "    return A, cache"], "metadata": {"_cell_guid": "579b817e-520a-4bd7-921b-1592b3959ee7", "_uuid": "3347c48133699dc410d25a936a400a4e1ae84ae0", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["def relu(Z):\n", "    A = np.maximum(0,Z)\n", "    assert(A.shape == Z.shape)\n", "    cache = Z \n", "    return A, cache"], "metadata": {"_cell_guid": "54960c0a-7619-4d06-86bf-3682e45b1e73", "_uuid": "c99d11405e67f906d2806c4e0fc1d3c61c592821", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["def sigmoid_backward(dA, cache):\n", "    Z = cache\n", "    \n", "    s = 1/(1+np.exp(-Z))\n", "    dZ = dA * s * (1-s)\n", "    \n", "    assert (dZ.shape == Z.shape)\n", "    \n", "    return dZ"], "metadata": {"_cell_guid": "ab2ed29d-c51c-459c-9d3d-27818d21b570", "_uuid": "c68e3dc4749de37e0404e34f1330fceafe1b9af5", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["def relu_backward(dA, cache):\n", "    Z = cache\n", "    dZ = np.array(dA, copy=True) # just converting dz to a correct object.\n", "    \n", "    # When z <= 0, you should set dz to 0 as well. \n", "    dZ[Z <= 0] = 0\n", "    \n", "    assert (dZ.shape == Z.shape)\n", "    \n", "    return dZ"], "metadata": {"_cell_guid": "e8169f7a-78b5-4920-a866-b94bf605e975", "_uuid": "a85ec1e122ae9d889a7223aeadbef1c8f2622572", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["function dense_to_1hot changes our labels to one hot lists for exaple a vector \n", "<br /><br />\n", "[1,2,4,9,0] \n", "<br /><br />will become:\n", "<br /><br />\n", "[\n", "[0,1,0,0,0,0,0,0,0,0],[0,0,1,0,0,0,0,0,0,0],[0,0,0,0,1,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,1],[1,0,0,0,0,0,0,0,0,0]\n", "]\n", "<br /><br />\n", "shape1 is passed to the function in order for it to know the length of the one hot vector.\n"], "metadata": {"_cell_guid": "54af4857-e2c1-4c5d-8a3e-5cce62cbb538", "_uuid": "6cdbe486596c6979c577ac31e86b1a0ae505e567"}, "cell_type": "markdown"}, {"source": ["def dense_to_1hot(labels, shape1):\n", "    shape0 = labels.shape[0]\n", "    one_hot = np.zeros((shape0, shape1))\n", "    one_hot[np.arange(shape0), labels] = 1\n", "    return one_hot"], "metadata": {"_cell_guid": "b4a7a1a3-40a9-41d4-ac5e-8f3be080195d", "_uuid": "98a7f0d8177e9fa7dcbef28e42cf27d01e27a973", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["predict function and write predictions are pretty well self explanatory."], "metadata": {"_cell_guid": "884fd95c-2cf6-4055-bad6-023ded3ae3b8", "_uuid": "e9eb0d34faaacb94f32afa1c4ee604744887bc8b"}, "cell_type": "markdown"}, {"source": ["def predict(parameters, data, layers_dims):\n", "    layers = len(layers_dims)\n", "    A = []\n", "    A.append(data)\n", "    for i in range(1, layers):\n", "        A.append(np.dot(parameters[\"W\" + str(i)], A[i-1]) + parameters[\"b\" + str(i)])\n", "    predictions = np.argmax(A[layers-1], axis=0)\n", "    return predictions"], "metadata": {"_cell_guid": "7529a297-bbec-46d9-a095-1947d4c4d2fb", "_uuid": "0588e3a3f7e51c9cb2196ce29c11402b793f7e49", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["def write_predictions(predictions):\n", "    with open('../input/submission_nn.csv', 'w') as subs:\n", "        subs.write(\"ImageId,Label\\n\")\n", "        for i, pred in enumerate(predictions):\n", "            subs.write(str(i+1)+','+str(pred)+'\\n')"], "metadata": {"_cell_guid": "5c42e35a-925d-4da3-b434-c798f523287d", "_uuid": "bd86b4bdf679d4c419bdd7b55569325ef4ea5162", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["<b>Run it part</b>\n", "<br /><br />\n", "In this part of the notebook, we will read set up some variables, read our data from the training set, split it into training and validation, and reshape it and transform it as needed, and at last we'll make sure everything is as it should be..."], "metadata": {"_cell_guid": "8fa69045-e50d-47d4-ab81-9f05d5ba5339", "_uuid": "22da01cbf47dae7a26228cf9034046bbb31fe4a1"}, "cell_type": "markdown"}, {"source": ["totalTrain = 42000\n", "tainPart = 40000\n", "testPart = 2000\n", "data = []\n", "with open('../input/train.csv', 'r') as train:\n", "    pixelReader = csv.reader(train, delimiter=',')\n", "    next(pixelReader, None)\n", "    for row in pixelReader:\n", "        data.append(row[0:])\n", "        \n", "data = np.array(data).astype(int)\n", "np.random.shuffle(data)\n", "data_train, data_test = data[:tainPart,:], data[tainPart:,:]\n", "labels_train_dense = data_train[:, 0]\n", "labels_test_dense = data_test[:, 0]\n", "\n", "data_train = np.multiply(data_train[:, 1:].T, 1/255)\n", "data_test = np.multiply(data_test[:, 1:].T, 1/255)\n", "assert(data_train.shape == (784, tainPart))\n", "assert(data_test.shape == (784, testPart))\n", "\n", "labels_train = dense_to_1hot(labels_train_dense, 10).T\n", "labels_test = dense_to_1hot(labels_test_dense, 10).T\n", "assert(labels_train.shape == (10, tainPart))\n", "assert(labels_test.shape == (10, testPart))\n", "\n", "image_to_show = 10\n", "plt.axis('off')\n", "plt.imshow(data_train[:, image_to_show].reshape(28, 28),  cmap=cm.binary)"], "metadata": {"scrolled": true, "_cell_guid": "168cdb79-7b34-4f75-9d9c-64df38047545", "_uuid": "07dd27e12ae3281c008328dc5ad92cf27acc925b", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["Initialize our layer dimensions, and train our neural network using the L_layer_model function we defined above. As we can notice here accuracy on each iteration is going up, and cost is decreasing gradually, which means we're in a good state. At the end 2 graphS will be displayed showing our accuracies and cost curves."], "metadata": {"_cell_guid": "a3a0eb89-fa9e-4ca4-b42b-99c0fb7be4bc", "_uuid": "a4042a259f47241f1d661838d59221c92ba99a22"}, "cell_type": "markdown"}, {"source": ["layers_dims = [784, 40, 30, 10] #  4-layer model\n", "parameters = L_layer_model(data_train, labels_train, layers_dims, data_test, labels_test_dense, num_iterations = 3200, print_cost = True, learning_rate = 0.0075, lambd = 3.3)"], "metadata": {"scrolled": true, "_cell_guid": "f358a0fb-43a6-4af9-9cab-4e9fdceae3b0", "_uuid": "c8ef97e5790a844c599fa843cf1a7afcf47fee39", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": ["<b>Predict on the test data</b>\n", "<br /><br />\n", "If everything is as we want it, we read the test data from the csv, reshape and transform it as needed, then we predict using our model, and at the end we write our predictions to the submission file"], "metadata": {"_cell_guid": "26520fef-277c-443a-98f8-e58a7b947752", "_uuid": "8f835c47bb469e0c0cb60fce9fcf419e2d8dfe50"}, "cell_type": "markdown"}, {"source": ["test = []\n", "with open('../input/test.csv', 'r') as train:\n", "    pixelReader = csv.reader(train, delimiter=',')\n", "    next(pixelReader, None)\n", "    for row in pixelReader:\n", "        test.append(row[0:])\n", "\n", "test = np.array(test).astype(int)\n", "test = np.multiply(test, 1.0 / 255.0)\n", "test = test.T\n", "#uncomment the last line to write your predictions to file\n", "predictions = predict(parameters, test, layers_dims)\n", "#write_predictions(predictions)"], "metadata": {"_cell_guid": "8181ea2e-0478-4393-8ad0-c8e9d4f8271e", "_uuid": "ed60ee1d188d69ed1a6568bd4b49b1699cd89e57", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}, {"source": [], "metadata": {"_cell_guid": "16a5479e-f73b-405a-9666-5e3853cea9dc", "_uuid": "5f59cf604e67c74eccc4855c6dbffbc361a8748c", "collapsed": true}, "outputs": [], "execution_count": null, "cell_type": "code"}]}