{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"provenance":[]},"kernelspec":{"name":"python3","display_name":"Python 3"}},"cells":[{"cell_type":"markdown","metadata":{"id":"r_dQxqilTO6u"},"source":"# Training Deep Convolutional Neural Network\n- Name1, Student's ID1\n- Name2, Student's ID2\n\nName your file to (first 2 student ID digit)_(last 4 student ID digit)*4.ipynb\n\n## Lab Instruction\n\nIn this lab, you will learn to train a deep convolutional neural network using Keras library with Tensorflow backend. We will use  Cat vs Dog dataset.\n\nSee https://www.kaggle.com/c/dogs-vs-cats/data"},{"cell_type":"markdown","metadata":{"id":"5e2iAoo_Sbdb"},"source":"\n### Your Turn! Dog or Cat Application\n\nNow, it time to put everything together and develop and Cat vs Dog classifier model. Assume that you and your company want to get more attraction on your product by launching a new product that can classify whether it is a dog or a cat. You have decide that you want a precision more than 90 - 95% in order to launch to product.\n\nYou have a cat and dog dataset contain total of 25000 images, 12500 for cat and other half for a dog.\n\nAfter successfully develop this model, you can try to play with it to see how it perform. (You can upload your selfies image to see you are a dog or a cat)\n\nSome note before start a project:\n1. You have to load a data into a project using any method\n**suggest**\n   \n\n```\n !wget -c https://s3.amazonaws.com/content.udacity-data.com/nd089/Cat_Dog_data.zip\n !unzip -qq Cat_Dog_data.zip\n```\n\n\n2. You have to plan on how you will split a data. (or no need for suggest method)\n3. You have to preprocess your data before feed into a network. For example, cropping, padding, etc.\n4. You can come up with any model or use pre-train model. It depend on you!\n\nYou have two week for this project so that you can compete with your competitor app! Now, create a new Jupyter notebook and start building a model!"},{"cell_type":"code","metadata":{"id":"1sObcVOjUCOW","colab":{"base_uri":"https://localhost:8080/","height":34},"outputId":"8247634b-c812-4764-e509-73e21a0c4d46"},"source":"from tensorflow import keras\nfrom keras import models\nfrom keras import layers\nfrom keras import optimizers\nfrom keras import callbacks\nfrom keras import backend as K\nfrom keras.preprocessing import image\nfrom keras.datasets import fashion_mnist\nfrom keras.utils import to_categorical\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import confusion_matrix, classification_report\nfrom keras.utils import np_utils\nfrom keras.layers.convolutional import  MaxPooling2D\nimport numpy as np\nimport matplotlib.pyplot as plt\n%matplotlib inline","execution_count":null,"outputs":[{"output_type":"stream","text":["Using TensorFlow backend.\n"],"name":"stderr"}]},{"cell_type":"code","metadata":{"id":"VOihOjVbVElj"},"source":"# %load _utils\nimport matplotlib.pyplot as plt\nfrom sklearn.metrics import confusion_matrix, classification_report\nfrom keras import backend as K\n\n# Test\ndef print_hello():\n    print('Hello')\n\n# define a function to plot the result from training step\ndef show_result(history):\n\n    # Print the result from the last epoch\n    print('Last train accuracy: %s'%history.history['acc'][-1])\n    print('Last validation accuracy: %s'%history.history['val_acc'][-1])\n\n    loss = history.history['loss']\n    val_loss = history.history['val_loss']\n\n    acc = history.history['acc']\n    val_acc = history.history['val_acc']\n\n    epochs = range(1, len(loss) + 1)\n\n    # Define a subplot\n    fig, axs = plt.subplots(1,2,figsize=(15,4))\n\n    # Plot loss\n    loss_plot = axs[0]\n\n    loss_plot.plot(epochs, loss, 'c--', label='Training loss')\n    loss_plot.plot(epochs, val_loss, 'b', label='Validation loss')\n    loss_plot.set_title('Training and validation loss')\n    loss_plot.set_xlabel('Epochs')\n    loss_plot.set_ylabel('Loss')\n    loss_plot.legend()\n\n    # Plot accuracy\n    acc_plot = axs[1]\n\n    acc_plot.plot(epochs, acc, 'c--', label='Training acc')\n    acc_plot.plot(epochs, val_acc, 'b', label='Validation acc')\n    acc_plot.set_title('Training and validation accuracy')\n    acc_plot.set_xlabel('Epochs')\n    acc_plot.set_ylabel('Accuracy')\n    acc_plot.legend()\n\n# Define an evaluation function to print the evaluation result\ndef evaluation_report(model,features,labels):\n\n    # Calculate result\n    result = model.evaluate(features,labels,verbose=False)\n\n    # Predict and convert into a class\n    pred_class = model.predict(features).argmax(axis=1)\n    labels = labels.argmax(axis=1)\n    # Show report\n    print(confusion_matrix(labels,pred_class))\n    print(classification_report(labels,pred_class))\n    print(\"Loss: %s Accuracy: %s\" %(result[0],result[1]))\n\n    return pred_class\n\n\n# Show a subplot of the incorrect predict data\ndef show_false_prediction(predict, feature, label, img_size=28, channel=1):\n\n    false_pred = feature[(predict != label).tolist()]\n    actual_label = label[(predict != label).tolist()]\n    false_label = predict[(predict != label).tolist()]\n    if channel == 3:\n        false_pred = false_pred.reshape(false_pred.shape[0],img_size,img_size,channel)\n    elif channel == 1:\n        false_pred = false_pred.reshape(false_pred.shape[0],img_size,img_size)\n    else:\n        raise ValueError('Must be RGB or gray scale image')\n\n    print(false_pred.shape)\n\n    fig, ax = plt.subplots(3,10,figsize=(15,6))\n    fig.suptitle('The incorrect prediction')\n\n    for i in range(3):\n        for j in range(10):\n            ax[i,j].imshow(false_pred[j + i*10],cmap='gray')\n            ax[i,j].set_title('Pred %s Act %s'%(false_label[j + i*10],actual_label[j + i*10]))\n\n# Show activation value of each layer\ndef show_layer_activation(activation, model,num_layer,num_row=16):\n    layer_names = []\n    for layer in model.layers[:num_layer]:\n        layer_names.append(layer.name)\n\n    images_per_row = num_row\n    for layer_name, layer_activation in zip(layer_names,activation):\n        n_features = layer_activation.shape[-1]\n\n        size = layer_activation.shape[1]\n\n        n_cols = n_features//images_per_row\n        display_grid = np.zeros((size * n_cols, images_per_row * size))\n\n        for col in range(n_cols):\n            for row in range(images_per_row):\n                channel_image = layer_activation[0,:,:,col*images_per_row + row]\n                channel_image -= channel_image.mean()\n                channel_image /= channel_image.std()\n                channel_image *= 64\n                channel_image += 128\n                channel_image = np.clip(channel_image, 0,255).astype('uint8')\n                display_grid[col*size:(col +1)*size,\n                             row*size:(row+1)*size] = channel_image\n        scale = 1./size\n        plt.figure(figsize=(scale*display_grid.shape[1],\n                           scale*display_grid.shape[0]))\n        plt.title(layer_name)\n        plt.grid(False)\n        plt.imshow(display_grid, aspect='auto', cmap='viridis')\n\n\ndef deprocess_image(img):\n\n    # Zero-centering and make sure that std is 0.1\n    img -= img.mean()\n    img /= (img.std() + 1e-5)\n    img *= 0.1\n\n    # Clips to [0,1]\n    img += 0.5\n    img = np.clip(img,0,1)\n\n    # Convert to RGB array\n    img *= 255\n    img = np.clip(img,0,255).astype('uint8')\n\n    return img\n\ndef generate_pattern(model, layer_name , filter_index, size=150):\n    # Build the loss function that maximize the activation of the nth filter of the layer under consideration\n    layer_output = model.get_layer(layer_name).output\n    loss = K.mean(layer_output[:,:,:,filter_index])\n\n    # Compute the gradient of the input picture with regard to this loss\n    grads = K.gradients(loss, model.input)[0]\n\n    # Normalize the gradient\n    grads /= (K.sqrt(K.mean(K.square(grads))) +1e-5)\n\n    # Return the loss and gradient given the input picture\n    iterate = K.function([model.input],[loss, grads])\n\n    # Stars from a gray image with some noise\n    input_img_data = np.random.random((1, size, size, 3)) * 20 + 128.\n\n    # Run gradient ascent for 40 step\n    step = 1.\n    for i in range(40):\n        loss_value, grads_value = iterate([input_img_data])\n        input_img_data += grads_value * step\n\n    img = input_img_data[0]\n    return deprocess_image(img)\n\n### feed layer name. ie, 'conv_1'\ndef visualize_filter(model,layer_name, size= 64, margin = 5):\n\n    # Empty black image to store results\n    results = np.zeros((8 * size + 7 * margin, 8 * size + 7 * margin, 3))\n\n    # iterate over the row of result grid\n    for i in range(8):\n        # Iterate over the column of the result grid\n        for j in range(8):\n            # Generates the pattern for filter i + (j*8) in layer_name\n            filter_img = generate_pattern(model, layer_name, i + (j*8), size=size)\n\n            # Puts the result in the square (i,j) of the results grid\n            horizontal_start = i * size + i * margin\n            horizontal_end = horizontal_start + size\n            vertical_start = j * size + j * margin\n            vertical_end = vertical_start + size\n            results[horizontal_start:horizontal_end,\n                    vertical_start:vertical_end, :] = filter_img\n\n    plt.figure(figsize=(20,20))\n    plt.imshow(results)","execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"mAX39-5qVJwo"},"source":"from keras.preprocessing.image import ImageDataGenerator","execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"bioSUNEcVPOf"},"source":"","execution_count":null,"outputs":[]}]}