{"cells": [{"cell_type": "markdown", "source": ["### Table of Contents\n", "\n", "1. Overview of the Problem set\n", "\n", "2. Load and prep the data\n", "\n", "3. Building the parts of our algorithm.\n", "\n", "4. Merge all functions into a model\n", "\n", "5. Visualizing and Analysis\n", "\n", "6. Lab"], "metadata": {"_cell_guid": "d0301e18-f10e-4852-9204-44d836fc8d5d", "_uuid": "80400d9a1749dd9acd813fbc077f2fb1668a66dd"}}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "8dd5a78e-7ade-4fa3-93c0-b18be1214740", "_uuid": "a0fbf8ebb5f35a7d7fff6dbf61e697ed1df995bb"}, "source": ["import os\n", "import random\n", "import sys\n", "import datetime\n", "## pip3 install opencv-python\n", "import cv2\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "\n", "%matplotlib inline"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "4cd88eaf-cff7-417a-932c-62b68bcdaa63", "_uuid": "9bffff1e8541552d6980fa51b5bcf6b17a923b9b"}, "source": ["## constants\n", "TRAIN_DIR = \"../input/train/\"\n", "TEST_DIR = \"../input/test/\"\n", "TRAIN_SIZE = 25000\n", "TEST_SIZE = 12500\n", "DEV_RATIO = 0.1\n", "IMAGE_HEIGHT = IMAGE_WIDTH = 128\n", "\n", "LEARNING_RATE = 0.0001\n", "MINIBATCH_SIZE = 32\n", "INPUT_SIZE = IMAGE_HEIGHT * IMAGE_WIDTH * 3\n", "OUTPUT_SIZE = 2"], "outputs": []}, {"cell_type": "markdown", "source": ["### Overview of the Problem set\n", "A fun project to differentiate dogs from cats. Dataset is from Kaggle: https://www.kaggle.com/c/dogs-vs-cats-redux-kernels-edition.\n", "\n", "The ./input/train/ dir contains 12500 cat images and 12500 dog images. Each filename contains \"cat\" or \"dog\" as label.\n", "\n", "The ./input/test/ dir contains 12500 images to classify\n"], "metadata": {"_cell_guid": "b951bacf-6d87-43a5-b937-deab5df1a1ef", "_uuid": "32558ac5d02059e297c4d96163ee1f06e7eb7582"}}, {"cell_type": "markdown", "source": ["### Load and prep the data"], "metadata": {"_cell_guid": "f563a837-c719-4937-b918-dd5e59b9e27d", "_uuid": "31090eea5f459eedbcbe84216060e1f75266d54d"}}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "a2d9ceea-d88d-4bb3-b2d5-716c4d1bce5d", "_uuid": "261adc3acc4a549714ee9bcc80044d7a28bc4776"}, "source": ["## data utility functions\n", "def split_data(two_dims_datas, split_ratio=DEV_RATIO):\n", "    left_count = int(two_dims_datas.shape[1] * split_ratio)\n", "    left_datas = two_dims_datas[:, :left_count]\n", "    right_datas = two_dims_datas[:, left_count:]\n", "    print(\"input datas shape: {}, left datas shape:{}, \\\n", "    right datas shape: {}\".format(two_dims_datas.shape, left_datas.shape, right_datas.shape))\n", "    return left_datas, right_datas"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "d0e3a9b2-c177-46d0-8784-df9929f33428", "_uuid": "2a493016660d32d907c36048cf8ffb54a1fff7ad"}, "source": ["def load_data(dirname=TRAIN_DIR, file_count=1000, shuffle=True):\n", "    all_filenames = os.listdir(dirname)\n", "    random.shuffle(all_filenames)\n", "    filenames = all_filenames[:file_count]\n", "    \n", "    ## images\n", "    images = np.zeros((file_count, IMAGE_HEIGHT*IMAGE_WIDTH*3))\n", "    for i in range(file_count):\n", "        imgnd_origin = cv2.imread(dirname+filenames[i])\n", "        imgnd_resized = cv2.resize(imgnd_origin, (IMAGE_HEIGHT, IMAGE_WIDTH), interpolation=cv2.INTER_CUBIC)\n", "        imgnd_flatten = imgnd_resized.reshape(1,-1)\n", "        images[i] = imgnd_flatten\n", "    \n", "    ## labels from filenames\n", "    labels_list = [\"dog\" in filename for filename in filenames]\n", "    labels = np.array(labels_list, dtype='int8').reshape(file_count, 1)\n", "    \n", "    ## shuffle\n", "    if shuffle:\n", "        permutation = list(np.random.permutation(labels.shape[0]))\n", "        labels = labels[permutation, :]\n", "        images = images[permutation, :]\n", "\n", "    ## normalization\n", "    images = images/255.0\n", "    \n", "    return images.T, labels.T"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"_cell_guid": "671fe2ae-1b3b-48d5-bf1f-9e8aead6444d", "_uuid": "71216ef4a6d9156bf5fd6f8bfac2e0d3f9963293"}, "source": ["images, labels = load_data(file_count=200)\n", "dev_images, train_images = split_data(images)\n", "dev_labels, train_labels = split_data(labels)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"scrolled": true, "collapsed": true, "_cell_guid": "7fcbddcd-c0dc-4b20-acd0-6f05279903be", "_uuid": "932e9a5a92afcdf2d265264a0c793850aa5d3d94"}, "source": [], "outputs": []}, {"cell_type": "markdown", "source": ["### Building the parts of our algorithm."], "metadata": {"_cell_guid": "05130aee-f02b-42f4-aa58-837ca2ceb40b", "_uuid": "8c0243bc373cf3c6aa71c7b3e517b5e3baacd174"}}, {"cell_type": "markdown", "source": ["<img src=\"http://p1plx6n23.bkt.clouddn.com/LogReg_kiank.png\">\n", "\n", "Forward Propagation:\n", "- You get X\n", "- You compute $A = \\sigma(w^T X + b) = (a^{(0)}, a^{(1)}, ..., a^{(m-1)}, a^{(m)})$\n", "- You calculate the cost function: $J = -\\frac{1}{m}\\sum_{i=1}^{m}y^{(i)}\\log(a^{(i)})+(1-y^{(i)})\\log(1-a^{(i)})$\n", "\n", "Here are the two formulas you will be using: \n", "\n", "$$ \\frac{\\partial J}{\\partial w} = \\frac{1}{m}X(A-Y)^T\\tag{7}$$\n", "$$ \\frac{\\partial J}{\\partial b} = \\frac{1}{m} \\sum_{i=1}^m (a^{(i)}-y^{(i)})\\tag{8}$$"], "metadata": {"_cell_guid": "9ead5282-0208-437d-8c72-fca6a0256638", "_uuid": "7e5c4aef1dc6bbaf8e09d4dbebd86adde1ed5069"}}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "39b13483-ca1c-4d9f-894c-560dd297d525", "_uuid": "b542bd4238872b6b15150f44c88648d2e840484c"}, "source": ["def sigmoid(z):\n", "    \"\"\"\n", "    Compute the sigmoid of z\n", "\n", "    Arguments:\n", "    z -- A scalar or numpy array of any size.\n", "\n", "    Return:\n", "    s -- sigmoid(z)\n", "    \"\"\"\n", "    \n", "    s = 1.0/(1.0 + np.exp(-1.0 * z))\n", "    \n", "    return s\n", "\n", "def initialize_with_zeros(dim):\n", "    \"\"\"\n", "    This function creates a vector of zeros of shape (dim, 1) for w and initializes b to 0.\n", "    \n", "    Argument:\n", "    dim -- size of the w vector we want (or number of parameters in this case)\n", "    \n", "    Returns:\n", "    w -- initialized vector of shape (dim, 1)\n", "    b -- initialized scalar (corresponds to the bias)\n", "    \"\"\"\n", "    \n", "    w = np.zeros(shape=(dim, 1), dtype=np.float32)\n", "    b = 0\n", "    \n", "    assert(w.shape == (dim, 1))\n", "    assert(isinstance(b, float) or isinstance(b, int))\n", "    \n", "    return w, b\n", "\n", "def propagate(w, b, X, Y):\n", "    \"\"\"\n", "    Implement the cost function and its gradient for the propagation explained above\n", "\n", "    Arguments:\n", "    w -- weights, a numpy array of size (num_px * num_px * 3, 1)\n", "    b -- bias, a scalar\n", "    X -- data of size (num_px * num_px * 3, number of examples)\n", "    Y -- true \"label\" vector (containing 0 if non-cat, 1 if cat) of size (1, number of examples)\n", "\n", "    Return:\n", "    cost -- negative log-likelihood cost for logistic regression\n", "    dw -- gradient of the loss with respect to w, thus same shape as w\n", "    db -- gradient of the loss with respect to b, thus same shape as b\n", "    \n", "    Tips:\n", "    - Write your code step by step for the propagation. np.log(), np.dot()\n", "    \"\"\"\n", "    \n", "    m = X.shape[1]\n", "    \n", "    # FORWARD PROPAGATION (FROM X TO COST)\n", "    #### W: shape of (num_px * num_px * 3, 1)\n", "    #### X: shape of (num_px * num_px * 3, number of examples)\n", "    #### Y: shape of (1, number of examples)\n", "    #print(\"shape of w\", w.shape)\n", "    #print(\"shape of X\", X.shape)\n", "    Z= np.matmul(w.T, X) + b\n", "    #print(\"shape of Z\", Z.shape)\n", "    A = sigmoid(Z)        ##### A shape of (1, number of examples)                            # compute activation\n", "    #print(\"shape of A\", A.shape)\n", "    #print(\"shape of Y\", Y.shape)\n", "    cost = (-1.0) * np.mean(np.multiply(Y, np.log(A)) + np.multiply(1.0-Y, np.log(1.0 - A)), axis=1)                                # compute cost\n", "    #print(\"shape of cost\", cost.shape)\n", "\n", "    dw = np.matmul(X, np.transpose(A - Y)) * (1.0/m)  ### (n_p*n_p*3, m)(1, m).T=(n_p*n_p*3, 1)\n", "    db = np.mean((A - Y))\n", "    #print(\"shape of dw\", dw.shape)\n", "    \n", "    assert(dw.shape == w.shape)\n", "    assert(db.dtype == float)\n", "    cost = np.squeeze(cost)\n", "    assert(cost.shape == ())\n", "    \n", "    grads = {\"dw\": dw,\n", "             \"db\": db}\n", "    \n", "    return grads, cost\n", "\n", "def optimize(w, b, X, Y, num_iterations, learning_rate, print_cost = False):\n", "    \"\"\"\n", "    This function optimizes w and b by running a gradient descent algorithm\n", "    \n", "    Arguments:\n", "    w -- weights, a numpy array of size (num_px * num_px * 3, 1)\n", "    b -- bias, a scalar\n", "    X -- data of shape (num_px * num_px * 3, number of examples)\n", "    Y -- true \"label\" vector (containing 0 if non-cat, 1 if cat), of shape (1, number of examples)\n", "    num_iterations -- number of iterations of the optimization loop\n", "    learning_rate -- learning rate of the gradient descent update rule\n", "    print_cost -- True to print the loss every 100 steps\n", "    \n", "    Returns:\n", "    params -- dictionary containing the weights w and bias b\n", "    grads -- dictionary containing the gradients of the weights and bias with respect to the cost function\n", "    costs -- list of all the costs computed during the optimization, this will be used to plot the learning curve.\n", "    \n", "    Tips:\n", "    You basically need to write down two steps and iterate through them:\n", "        1) Calculate the cost and the gradient for the current parameters. Use propagate().\n", "        2) Update the parameters using gradient descent rule for w and b.\n", "    \"\"\"\n", "    \n", "    costs = []\n", "    \n", "    for i in range(num_iterations):\n", "        \n", "        \n", "        # Cost and gradient calculation \n", "        grads, cost = propagate(w, b, X, Y)\n", "        \n", "        # Retrieve derivatives from grads\n", "        dw = grads[\"dw\"]\n", "        db = grads[\"db\"]\n", "        \n", "        # update rule\n", "        w = w - learning_rate * dw\n", "        b = b - learning_rate * db\n", "        \n", "        # Record the costs\n", "        if i % 100 == 0:\n", "            costs.append(cost)\n", "        \n", "        # Print the cost every 100 training examples\n", "        if print_cost and i % 100 == 0:\n", "            print (\"Cost after iteration %i: %f\" %(i, cost))\n", "    \n", "    params = {\"w\": w,\n", "              \"b\": b}\n", "    \n", "    grads = {\"dw\": dw,\n", "             \"db\": db}\n", "    \n", "    return params, grads, costs\n", "\n", "def predict(w, b, X):\n", "    '''\n", "    Predict whether the label is 0 or 1 using learned logistic regression parameters (w, b)\n", "    \n", "    Arguments:\n", "    w -- weights, a numpy array of size (num_px * num_px * 3, 1)\n", "    b -- bias, a scalar\n", "    X -- data of size (num_px * num_px * 3, number of examples)\n", "    \n", "    Returns:\n", "    Y_prediction -- a numpy array (vector) containing all predictions (0/1) for the examples in X\n", "    '''\n", "    \n", "    m = X.shape[1]\n", "    Y_prediction = np.zeros((1,m))\n", "    w = w.reshape(X.shape[0], 1)\n", "    \n", "    # Compute vector \"A\" predicting the probabilities of a cat being present in the picture\n", "    A = sigmoid(np.matmul(w.T, X) + b)\n", "    \n", "    for i in range(A.shape[1]):\n", "        \n", "        # Convert probabilities A[0,i] to actual predictions p[0,i]\n", "        if A[0, i] > 0.5:\n", "            Y_prediction[0, i] = 1\n", "        else:\n", "            Y_prediction[0, i] = 0\n", "    \n", "    assert(Y_prediction.shape == (1, m))\n", "    \n", "    return Y_prediction\n", "\n", "def predict_test_images(dirname=TEST_DIR, file_count=125):\n", "    test_images, _ = load_data(dirname=dirname, file_count=file_count, shuffle=False)\n", "\n", "    res = predict(d['w'], d['b'], test_images)\n", "    res = res.ravel().astype(int)\n", "\n", "    res = np.dstack((np.arange(1, res.shape[0]+1), res))[0]\n", "    res_fn = 'result{}.csv'.format(datetime.datetime.now())\n", "    np.savetxt(res_fn, res, fmt='%d,%d',header=\"id,label\", comments='')\n", "    return res_fn"], "outputs": []}, {"cell_type": "markdown", "source": ["### Merge all functions into a model"], "metadata": {"_cell_guid": "88827fa2-f717-4c22-8d26-fab0c975619a", "_uuid": "edc9077ef421708c34e008664519b7819dd1d114"}}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "e3da9365-0c89-4710-800b-fb52744566b4", "_uuid": "cc59deb0eab93439a8c8af4d12f861d4021dab4a"}, "source": ["def model(X_train, Y_train, X_test, Y_test, num_iterations = 2000, learning_rate = 0.5, print_cost = False):\n", "    \"\"\"\n", "    Builds the logistic regression model by calling the function you've implemented previously\n", "    \n", "    Arguments:\n", "    X_train -- training set represented by a numpy array of shape (num_px * num_px * 3, m_train)\n", "    Y_train -- training labels represented by a numpy array (vector) of shape (1, m_train)\n", "    X_test -- test set represented by a numpy array of shape (num_px * num_px * 3, m_test)\n", "    Y_test -- test labels represented by a numpy array (vector) of shape (1, m_test)\n", "    num_iterations -- hyperparameter representing the number of iterations to optimize the parameters\n", "    learning_rate -- hyperparameter representing the learning rate used in the update rule of optimize()\n", "    print_cost -- Set to true to print the cost every 100 iterations\n", "    \n", "    Returns:\n", "    d -- dictionary containing information about the model.\n", "    \"\"\"\n", "    \n", "    # initialize parameters with zeros (\u2248 1 line of code)\n", "    w, b = initialize_with_zeros(X_train.shape[0])\n", "\n", "    # Gradient descent (\u2248 1 line of code)\n", "    parameters, grads, costs = optimize(w, b, X_train, Y_train, num_iterations, learning_rate, print_cost = print_cost)\n", "    \n", "    # Retrieve parameters w and b from dictionary \"parameters\"\n", "    w = parameters[\"w\"]\n", "    b = parameters[\"b\"]\n", "    \n", "    # Predict test/train set examples (\u2248 2 lines of code)\n", "    Y_prediction_test = predict(w, b, X_test)\n", "    Y_prediction_train = predict(w, b, X_train)\n", "\n", "    # Print train/test Errors\n", "    print(\"train accuracy: {} %\".format(100 - np.mean(np.abs(Y_prediction_train - Y_train)) * 100))\n", "    print(\"test accuracy: {} %\".format(100 - np.mean(np.abs(Y_prediction_test - Y_test)) * 100))\n", "\n", "    \n", "    d = {\"costs\": costs,\n", "         \"Y_prediction_test\": Y_prediction_test, \n", "         \"Y_prediction_train\" : Y_prediction_train, \n", "         \"w\" : w, \n", "         \"b\" : b,\n", "         \"learning_rate\" : learning_rate,\n", "         \"num_iterations\": num_iterations}\n", "    \n", "    return d"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"_cell_guid": "852c64c2-4579-49f6-82d3-141e86088fe3", "_uuid": "8a90b5901e9bc9eaffb1fc5419d521505c2211ee"}, "source": ["d = model(train_images, train_labels, dev_images, dev_labels, num_iterations = 2000, learning_rate = 0.005, print_cost = True)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"_cell_guid": "c4a62300-a8da-4810-bb96-ab2b5378bc3f", "_uuid": "2de59ed953770eac03d24e6036a76fc9be3e5da9"}, "source": ["predict_test_images(file_count=500)"], "outputs": []}, {"cell_type": "markdown", "source": ["> ### Visualizing and Analysis"], "metadata": {"_cell_guid": "45259e3b-ee50-44fe-81e9-c7300a5bccd0", "_uuid": "b236776246c61724fae451d7781fba6fc99a98a1"}}, {"cell_type": "code", "execution_count": null, "metadata": {"_cell_guid": "14464fee-fda7-405e-99ec-e417de4b8e1d", "_uuid": "419750ce6dbe29e5ca469474ac0e29da10d7ae03"}, "source": ["def plt_costs(d):\n", "    # Plot learning curve (with costs)\n", "    costs = np.squeeze(d['costs'])\n", "    plt.plot(costs)\n", "    plt.ylabel('cost')\n", "    plt.xlabel('iterations (per hundreds)')\n", "    plt.title(\"Learning rate =\" + str(d[\"learning_rate\"]))\n", "    #plt.show()\n", "\n", "plt_costs(d)"], "outputs": []}, {"cell_type": "markdown", "source": ["### Lab"], "metadata": {"_cell_guid": "f1b68758-ff60-4264-9800-8939f69f91c1", "_uuid": "3ad359df0b8ca59670317f108af0d74719e86cc6"}}, {"cell_type": "markdown", "source": ["##### Use more training data "], "metadata": {"_cell_guid": "fabfc24e-e3c6-424b-8278-19a20cbd2c15", "_uuid": "4bd725c8ddc004b5ee3ee3641c275f3c56d9e52f"}}, {"cell_type": "code", "execution_count": null, "metadata": {"_cell_guid": "488f9303-82f4-4352-8754-a1248520dd3a", "_uuid": "8e37729437f2f4465a9ad702c5793ed109a7f66b"}, "source": ["## More Data\n", "m = 1000\n", "images, labels = load_data(file_count=m)\n", "dev_images, train_images = split_data(images)\n", "dev_labels, train_labels = split_data(labels)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"_cell_guid": "a89dfc57-a843-4f40-af5e-4ed717599454", "_uuid": "4291bd0d99f5acb2e61fa414a99e2f654e1687cc"}, "source": ["d = model(train_images, train_labels, dev_images, dev_labels, num_iterations = 2000, learning_rate = 0.0005, print_cost = True)\n", "plt_costs(d)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "2fb73a18-ef01-4de1-b08e-f1d4f2eb98e1", "_uuid": "1a45e0c103fcd20036ac965c125bfe8a20f8d45f"}, "source": [], "outputs": []}, {"cell_type": "markdown", "source": ["##### Use more training data ,less learning_rate"], "metadata": {"_cell_guid": "bfe1b8d9-0fb8-49c8-9b81-9d77762bfdd6", "_uuid": "dbf78b233d3f0c299a2a4a023b9cecd01e9dd9f3"}}, {"cell_type": "code", "execution_count": null, "metadata": {"_cell_guid": "7834b73e-aae4-4097-8257-2b6ddcd38af9", "_uuid": "6af402b9e67da7440292d1b713d40e9e2194ae5c"}, "source": ["d = model(train_images, train_labels, dev_images, dev_labels, num_iterations = 2000, learning_rate = 0.0005, print_cost = True)\n", "plt_costs(d)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "2427ffd7-e5b2-472a-8e08-3f391c0e0ba4", "_uuid": "43aeed81f33972d4aceb090a811f3d420a9f082d"}, "source": [], "outputs": []}, {"cell_type": "markdown", "source": ["##### Use all training data , learning_rate = LEARNING_RATE =  0.001"], "metadata": {"_cell_guid": "66853c61-994a-458e-a779-0587a3a84636", "_uuid": "8d6c3fe9ecbab3f9a952e8380e3657a1d520cb13"}}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "badf692f-f09e-4ae7-813a-838179ec8c1f", "_uuid": "f3591fb51c5bd2372571bb2a2e25a85dbe606ba8"}, "source": ["## More Data\n", "m = TRAIN_SIZE\n", "images, labels = load_data(file_count=m)\n", "dev_images, train_images = split_data(images, split_ratio=DEV_RATIO)\n", "dev_labels, train_labels = split_data(labels, split_ratio=DEV_RATIO)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "844f62fa-87db-410e-9d2d-1ab114189dd2", "_uuid": "ba736690a558e888e5f4e68f3e8255877e957da5"}, "source": ["d = model(train_images, train_labels, dev_images, dev_labels, num_iterations = 200, learning_rate = LEARNING_RATE, print_cost = True)\n", "plt_costs(d)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "624509da-ffa2-4729-9edc-0757cee2c48e", "_uuid": "e2414ecf4c94092eeca32e145913c24b84f4667f"}, "source": ["predict_test_images(file_count=12500)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "0feae3bc-f549-4c33-b1ba-d0e8efdf3dc8", "_uuid": "72781247a29aec6b5df66c1c5c9af7657d71a5e2"}, "source": [], "outputs": []}, {"cell_type": "markdown", "source": ["##### Use all training data , learning_rate =  0.005"], "metadata": {"_cell_guid": "5c25a08d-415d-4cc6-bfc2-09b0faf8184f", "_uuid": "0f685acaab64c75bcecd5356785870bc195eff96"}}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "55cc23ee-6191-4957-afb9-69f940bfe07e", "_uuid": "92370fa13fbbdee4c9c4acdba90672135bf6a0d2"}, "source": ["## More Data\n", "m = TRAIN_SIZE\n", "images, labels = load_data(file_count=m)\n", "dev_images, train_images = split_data(images, split_ratio=DEV_RATIO)\n", "dev_labels, train_labels = split_data(labels, split_ratio=DEV_RATIO)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"scrolled": true, "collapsed": true, "_cell_guid": "79abf706-f069-4fde-99d9-2071bd96780d", "_uuid": "aa6f455168fd1de8cbc772a6f8f955cfd9a6ca72"}, "source": ["d = model(train_images, train_labels, dev_images, dev_labels, num_iterations = 2000, learning_rate = 0.005, print_cost = True)\n", "plt_costs(d)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "2351f80b-3e5d-47e7-a324-b749959b367a", "_uuid": "01c52dda94b0e48d833fcffc9f5e06b20e4f690b"}, "source": ["predict_test_images(file_count=12500)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "6e9c5eed-c8c9-4075-9585-d010ce038d71", "_uuid": "b133b94a916c48b7a7d06fa027662c574c4bc818"}, "source": [], "outputs": []}, {"cell_type": "markdown", "source": ["##### Use all training data , learning_rate =  0.003"], "metadata": {"_cell_guid": "0eb928d9-2f0d-415b-8e21-7502b0a3aad2", "_uuid": "31432e35002d3f28aae57eb688947697acc5f3f9"}}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "31b299bd-dd7a-4622-b76b-0f0b4fd5f539", "_uuid": "8cc6c3bad775fc4b722bdfc38a3981f9ded757c8"}, "source": ["## More Data\n", "m = TRAIN_SIZE\n", "images, labels = load_data(file_count=m)\n", "dev_images, train_images = split_data(images, split_ratio=DEV_RATIO)\n", "dev_labels, train_labels = split_data(labels, split_ratio=DEV_RATIO)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"scrolled": true, "collapsed": true, "_cell_guid": "d7e2333f-2b18-48b3-bdd3-795f08cd1cff", "_uuid": "f662b6a8161081be66310dfc6ebc486e3762b5cf"}, "source": ["d = model(train_images, train_labels, dev_images, dev_labels, num_iterations = 200, learning_rate = 0.003, print_cost = True)\n", "plt_costs(d)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"scrolled": true, "collapsed": true, "_cell_guid": "ec42fee5-b1c2-44dd-9450-6b97908cd67b", "_uuid": "bffe889cd003a7136a4a09fe438d30b3430475e4"}, "source": ["predict_test_images(file_count=12500)"], "outputs": []}, {"cell_type": "markdown", "source": ["##### Use all training data , learning_rate =  0.001"], "metadata": {"_cell_guid": "7f6a3134-0530-42eb-9c7d-0ec4e42947ef", "_uuid": "3c9a2cc846256d0753583f6c808f5ef99a5de02c"}}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "c75ebaa6-ce0b-4505-9dfd-bad62e8b1d91", "_uuid": "6d02061f4bb4d92f0f1d95c18101027d5c00cfd2"}, "source": ["## More Data\n", "m = TRAIN_SIZE\n", "print(\"start load {} data:{}\".format(m,datetime.datetime.now()))\n", "images, labels = load_data(file_count=m)\n", "print(\"end load {} data:{}\".format(m,datetime.datetime.now()))\n", "print(\"start split {} data:{}\".format(m,datetime.datetime.now()))\n", "dev_images, train_images = split_data(images, split_ratio=DEV_RATIO)\n", "dev_labels, train_labels = split_data(labels, split_ratio=DEV_RATIO)\n", "print(\"end split {} data:{}\".format(m,datetime.datetime.now()))"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"scrolled": true, "collapsed": true, "_cell_guid": "c4a2595a-54c0-415c-bb6b-e7040eb00189", "_uuid": "8230553dacd8544131829662d6f74f40250b9759"}, "source": ["print(\"start train model:{}\".format(datetime.datetime.now()))\n", "d = model(train_images, train_labels, dev_images, dev_labels, num_iterations = 500, learning_rate = 0.001, print_cost = True)\n", "print(\"end train model:{}\".format(datetime.datetime.now()))\n", "plt_costs(d)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "f97255a1-94d2-4f8c-98a6-418735822115", "_uuid": "51092006764bbea238e0d0972b708b29e5d1ce44"}, "source": ["print(\"start train model:{}\".format(datetime.datetime.now()))\n", "d = model(train_images, train_labels, dev_images, dev_labels, num_iterations = 500, learning_rate = 0.0005, print_cost = True)\n", "print(\"end train model:{}\".format(datetime.datetime.now()))\n", "plt_costs(d)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "f5546f4f-178f-4e8f-93be-b54395dac659", "_uuid": "7b63262fb82e25e7f8ec22b5098e5d89840e2acc"}, "source": ["def learning_rate_lab(learning_rates=[0.0005]):\n", "    m_l = len(learning_rates)\n", "    d_list = []\n", "    for i in range(m_l):\n", "        print(\"learning_rate:{}, start train model:{}\".format(learning_rates[i], datetime.datetime.now()))\n", "        d = model(train_images, train_labels, dev_images, dev_labels, num_iterations = 500, learning_rate = learning_rates[i], print_cost = True)\n", "        print(\"learning_rate:{}, end train model:{}\".format(learning_rates[i], datetime.datetime.now()))\n", "        d_list.append(d)\n", "    for i in range(len(d_list)):\n", "        plt.subplot(321+i, facecolor='y')\n", "        plt_costs(d_list[i])\n", "    return d_list"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "04c4d16e-7aef-40fa-bc3f-452c16a914fb", "_uuid": "d9c1a662d28217f59e30f360ba43d2fbcbadc138"}, "source": ["learning_rate_lab([0.00007, 0.0001, 0.0005, 0.0007, 0.001, 0.005])"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "0600ed9d-dcf5-4bcf-8fcc-9424d94154d2", "_uuid": "5851c04df1aa5695235268f342368df09e99f9b0"}, "source": ["d_list = learning_rate_lab([0.0007, 0.0008, 0.0009, 0.001])"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true, "_cell_guid": "a6250b8e-0320-417c-bb26-2484f3511560", "_uuid": "90ba1414f490e1615d676bbe388eeaad7b5b0de9"}, "source": [], "outputs": []}], "metadata": {"language_info": {"nbconvert_exporter": "python", "pygments_lexer": "ipython3", "mimetype": "text/x-python", "file_extension": ".py", "version": "3.6.3", "name": "python", "codemirror_mode": {"version": 3, "name": "ipython"}}, "kernelspec": {"name": "python3", "language": "python", "display_name": "Python 3"}}, "nbformat_minor": 1, "nbformat": 4}