{ "cells": [ { "cell_type": "markdown", "metadata": { "_cell_guid": "7baa365a-7329-3540-ebad-70769d4c23e5" }, "source": [ "# Introduction\n", "\n", "This Kernel is forked from Phillipe Loher's TensorFlow Starter Kit and uses the image preprocessing algorithm from gauss256.\n", "\n", "Trying to improve the model by adding dropout and tinkering the ConvNet." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "b50e1f7b-e2d1-d51a-0926-0f0cb37014d2" }, "outputs": [], "source": [ "# These are all the modules we'll be using later\n", "import matplotlib.image as mpimg\n", "import matplotlib.pyplot as plt\n", "from PIL import Image\n", "import PIL\n", "import pandas as pd\n", "import numpy as np\n", "import cv2\n", "import os\n", "import glob\n", "import re\n", "import tensorflow as tf\n", "\n", "# Constants\n", "TRAIN_DIR = '../input/train/'\n", "TEST_DIR = '../input/test/'\n", "IMAGE_SIZE = 150\n", "CHANNELS = 3\n", "\n", "# Sample sizes\n", "TRAINING_SIZE = 1600\n", "VALIDATION_SIZE = 400\n", "TESTING_SIZE = 100\n", "\n", "# CNN parameters\n", "BATCH_SIZE = 16\n", "NUM_HIDDEN = 32\n", "DROPOUT = 0.75\n", "LEARNING_RATE = 0.0001\n", "NUM_STEPS = 1001" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "90a7f0ca-de09-e332-1007-39f615b7de33" }, "source": [ "- To run within a Kaggle Kernel, only use 2000 samples from TRAIN_DIR and 500 samples from TEST_DIR\n", "- Set image size to 96x96 since Kaggle Kernel was running out of memory with 224" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "6de52376-51c3-b21f-aedf-e4f1730d1bd6" }, "source": [ "# Preprocessing\n", "\n", "Normalize the luminance values and resize the images to a standard shape. This is done because the training and test images come in a variety of shapes, sizes, and lighting." ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "816ff167-ef52-2507-d305-bcaaadad753b" }, "source": [ "**Normalize the image luminance**\n", "\n", "It is common in image analysis to normalize the luminance (brightness) values to have mean 0 and standard deviation 1. We do that here and apply a slight contrast stretch, which also ensures the brightness values stay within the bounds of the image encoding.\n", "\n", "The normalization is applied to the luminance, not to the RGB channels individually. We first convert to YCbCr space, operate on the Y channel, and then convert back to RGB." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "b2d2b803-8089-c605-a1a8-db4b987eee62" }, "outputs": [], "source": [ "def norm_image(img):\n", " \"\"\"\n", " Normalize PIL image\n", " \n", " Normalizes luminance to (mean,std)=(0,1), and applies a [1%, 99%] contrast stretch\n", " \"\"\"\n", " img_y, img_b, img_r = img.convert('YCbCr').split()\n", " \n", " img_y_np = np.asarray(img_y).astype(float)\n", "\n", " img_y_np /= 255\n", " img_y_np -= img_y_np.mean()\n", " img_y_np /= img_y_np.std()\n", " scale = np.max([np.abs(np.percentile(img_y_np, 1.0)),\n", " np.abs(np.percentile(img_y_np, 99.0))])\n", " img_y_np = img_y_np / scale\n", " img_y_np = np.clip(img_y_np, -1.0, 1.0)\n", " img_y_np = (img_y_np + 1.0) / 2.0\n", " \n", " img_y_np = (img_y_np * 255 + 0.5).astype(np.uint8)\n", "\n", " img_y = Image.fromarray(img_y_np)\n", "\n", " img_ybr = Image.merge('YCbCr', (img_y, img_b, img_r))\n", " \n", " img_nrm = img_ybr.convert('RGB')\n", " \n", " return img_nrm" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "24af0e4d-fc25-ca1d-5ba4-8386e831883a" }, "source": [ "\n", "**Resize the image**\n", "\n", "We resize the images to be square with a default side length of 224 to be compatible with models trained on ImageNet. The aspect ratio is preserved and gray bars are added as necessary to make the image square." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "88be41f4-9045-9e72-7cf3-0edf6621153a" }, "outputs": [], "source": [ "def resize_image(img, size):\n", " \"\"\"\n", " Resize PIL image\n", " \n", " Resizes image to be square with sidelength size. Pads with black if needed.\n", " \"\"\"\n", " # Resize\n", " n_x, n_y = img.size\n", " if n_y > n_x:\n", " n_y_new = size\n", " n_x_new = int(size * n_x / n_y + 0.5)\n", " else:\n", " n_x_new = size\n", " n_y_new = int(size * n_y / n_x + 0.5)\n", "\n", " img_res = img.resize((n_x_new, n_y_new), resample=PIL.Image.BICUBIC)\n", "\n", " # Pad the borders to create a square image\n", " img_pad = Image.new('RGB', (size, size), (128, 128, 128))\n", " ulc = ((size - n_x_new) // 2, (size - n_y_new) // 2)\n", " img_pad.paste(img_res, ulc)\n", "\n", " return img_pad" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "67239619-7632-6e39-1d40-257300d1dd94" }, "source": [ "**Accumulate the image names**" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "9b3f084a-e71a-10a0-5cb3-3079a9ad9db7" }, "outputs": [], "source": [ "def natural_key(string_):\n", " \"\"\"\n", " Define sort key that is integer-aware\n", " \"\"\"\n", " return [int(s) if s.isdigit() else s for s in re.split(r'(\\d+)', string_)]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "bca16e3c-2b12-a7af-7d7e-bf1df0f5b39a" }, "outputs": [], "source": [ "train_cats = np.array(sorted(glob.glob(os.path.join(TRAIN_DIR, 'cat*.jpg')), key=natural_key))\n", "train_dogs = np.array(sorted(glob.glob(os.path.join(TRAIN_DIR, 'dog*.jpg')), key=natural_key))\n", "\n", "test_all = np.array(sorted(glob.glob(os.path.join(TEST_DIR, '*.jpg')), key=natural_key))" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "c39f6539-b640-0f33-e2f4-e6936a9c23a7" }, "source": [ "**Define the training, validation and testing datasets**" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "7aa44095-297e-84ea-82ec-1980b4f1fe76" }, "outputs": [], "source": [ "def read_images(images):\n", " \"\"\" Load image data into a useful structure. \"\"\"\n", " count = len(images)\n", " data = np.ndarray((count, IMAGE_SIZE, IMAGE_SIZE, CHANNELS), dtype=np.float32)\n", "\n", " for i, image_file in enumerate(images):\n", " # Normalize and resize the image\n", " img = Image.open(image_file)\n", " img_nrm = norm_image(img)\n", " img_res = resize_image(img_nrm, IMAGE_SIZE)\n", " # Store it with a useful format\n", " img_data = np.array(img_res, dtype=np.float32) \n", " data[i] = img_data\n", " return data" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "74ce0a1c-4698-6834-3902-43a4c0b4ac21" }, "outputs": [], "source": [ "np.random.seed(133)\n", "def randomize(dataset, labels=None):\n", " permutation = np.random.permutation(len(dataset))\n", " shuffled_dataset = dataset[permutation]\n", " if labels is not None:\n", " shuffled_labels = labels[permutation]\n", " return shuffled_dataset, shuffled_labels\n", " return shuffled_dataset" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "66b6f1f7-11ad-fa31-e866-f4a7cb8baa61" }, "outputs": [], "source": [ "# Training dataset\n", "train_all = np.append(randomize(train_cats), randomize(train_dogs))\n", "\n", "half_train_size = int(TRAINING_SIZE / 2)\n", "mid_train = int(len(train_all) / 2)\n", "\n", "train_images = np.append(train_all[:half_train_size], train_all[mid_train:mid_train+half_train_size])\n", "train_dataset = read_images(train_images)\n", "train_labels = np.append(np.ones(half_train_size), np.zeros(half_train_size))\n", "train_labels = (np.arange(2) == train_labels[:,None]).astype(np.float32)\n", "\n", "# Validation dataset\n", "valid_all = np.append(train_all[half_train_size:mid_train], train_all[mid_train+half_train_size:])\n", "valid_labels_all = np.append(np.ones(mid_train - half_train_size), np.zeros(mid_train - half_train_size))\n", "valid_images, valid_images_labels = randomize(valid_all, valid_labels_all)\n", "valid_dataset = read_images(valid_images[:VALIDATION_SIZE])\n", "valid_labels = valid_images_labels[:VALIDATION_SIZE]\n", "valid_labels = (np.arange(2) == valid_labels[:,None]).astype(np.float32)\n", "\n", "# Testing dataset\n", "test_images = test_all[:TESTING_SIZE]\n", "test_dataset = read_images(test_images)\n", "\n", "print(\"Train shape:\", train_dataset.shape, train_labels.shape)\n", "print(\"Valid shape:\", valid_dataset.shape, valid_labels.shape)\n", "print(\"Test shape:\", test_dataset.shape)" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "afe1edec-a287-371b-6b0e-a93cc45f1346" }, "source": [ "Just for visualization fun, print images (2 train & 2 valid) after resizing and normalizing. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "a4a83186-c361-4fb0-69a6-4aff689ea523" }, "outputs": [], "source": [ "plt.imshow(train_dataset[0,:,:,:], interpolation='nearest')\n", "plt.figure()\n", "plt.imshow(train_dataset[1000,:,:,:], interpolation='nearest')\n", "plt.figure()\n", "plt.imshow(valid_dataset[0,:,:,:], interpolation='nearest')\n", "plt.figure()\n", "plt.imshow(valid_dataset[1,:,:,:], interpolation='nearest')" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "9e9e7059-0e31-f2d0-8176-32e04cebac09" }, "source": [ "# Run TensorFlow Convolutional Neural Network\n", "\n", "**Define the Graph Model**" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "6796b600-5add-2744-c043-87ada7bfa679" }, "outputs": [], "source": [ "graph = tf.Graph()\n", "\n", "with graph.as_default():\n", " # Input data\n", " tf_train_dataset = tf.placeholder(tf.float32, shape=(BATCH_SIZE, IMAGE_SIZE, IMAGE_SIZE, CHANNELS))\n", " tf_train_labels = tf.placeholder(tf.float32, shape=(BATCH_SIZE, 2))\n", " tf_valid_dataset = tf.constant(valid_dataset)\n", " tf_test_dataset = tf.constant(test_dataset)\n", "\n", " # Variables \n", " HALF_HIDDEN = int(NUM_HIDDEN / 2)\n", " kernel_conv1 = tf.Variable(tf.truncated_normal([3, 3, 3, HALF_HIDDEN], dtype=tf.float32, stddev=1e-1), name='weights_conv1')\n", " biases_conv1 = tf.Variable(tf.constant(0.0, shape=[HALF_HIDDEN], dtype=tf.float32), trainable=True, name='biases_conv1')\n", " kernel_conv2 = tf.Variable(tf.truncated_normal([3, 3, HALF_HIDDEN, HALF_HIDDEN], dtype=tf.float32, stddev=1e-1), name='weights_conv2')\n", " biases_conv2 = tf.Variable(tf.constant(0.0, shape=[HALF_HIDDEN], dtype=tf.float32), trainable=True, name='biases_conv2')\n", " kernel_conv3 = tf.Variable(tf.truncated_normal([3, 3, HALF_HIDDEN, NUM_HIDDEN], dtype=tf.float32, stddev=1e-1), name='weights_conv3')\n", " biases_conv3 = tf.Variable(tf.constant(0.0, shape=[NUM_HIDDEN], dtype=tf.float32), trainable=True, name='biases_conv3')\n", " \n", " fc1w = tf.Variable(tf.truncated_normal([11552, NUM_HIDDEN], dtype=tf.float32, stddev=1e-1), name='weights') \n", " fc1b = tf.Variable(tf.constant(1.0, shape=[NUM_HIDDEN], dtype=tf.float32), trainable=True, name='biases')\n", " fc2w = tf.Variable(tf.truncated_normal([NUM_HIDDEN, 2], dtype=tf.float32, stddev=1e-1), name='weights')\n", " fc2b = tf.Variable(tf.constant(1.0, shape=[2], dtype=tf.float32), trainable=True, name='biases')\n", " \n", " def model(data):\n", " parameters = []\n", " with tf.name_scope('conv1_1') as scope:\n", " conv = tf.nn.conv2d(data, kernel_conv1, [1, 1, 1, 1], padding='SAME')\n", " out = tf.nn.bias_add(conv, biases_conv1)\n", " conv1_1 = tf.nn.relu(out, name=scope)\n", " conv1_1 = tf.nn.dropout(conv1_1, DROPOUT)\n", " parameters += [kernel_conv1, biases_conv1]\n", " \n", " # pool1\n", " pool1 = tf.nn.max_pool(conv1_1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME', name='pool1')\n", " with tf.name_scope('conv2_1') as scope:\n", " conv = tf.nn.conv2d(pool1, kernel_conv2, [1, 1, 1, 1], padding='SAME')\n", " out = tf.nn.bias_add(conv, biases_conv2)\n", " conv2_1 = tf.nn.relu(out, name=scope)\n", " conv2_1 = tf.nn.dropout(conv2_1, DROPOUT)\n", " parameters += [kernel_conv2, biases_conv2]\n", " \n", " # pool2\n", " pool2 = tf.nn.max_pool(conv2_1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME', name='pool2')\n", " with tf.name_scope('conv3_1') as scope:\n", " conv = tf.nn.conv2d(pool2, kernel_conv3, [1, 1, 1, 1], padding='SAME')\n", " out = tf.nn.bias_add(conv, biases_conv3)\n", " conv3_1 = tf.nn.relu(out, name=scope)\n", " conv3_1 = tf.nn.dropout(conv3_1, DROPOUT)\n", " parameters += [kernel_conv3, biases_conv3]\n", " \n", " # pool3\n", " pool3 = tf.nn.max_pool(conv3_1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME', name='pool3')\n", " \n", " # fc1\n", " with tf.name_scope('fc1') as scope:\n", " shape = int(np.prod(pool3.get_shape()[1:])) # except for batch size (the first one), multiple the dimensions\n", " pool3_flat = tf.reshape(pool3, [-1, shape])\n", " fc1l = tf.nn.bias_add(tf.matmul(pool3_flat, fc1w), fc1b)\n", " fc1 = tf.nn.relu(fc1l)\n", " parameters += [fc1w, fc1b]\n", "\n", " # fc3\n", " with tf.name_scope('fc3') as scope:\n", " fc2l = tf.nn.bias_add(tf.matmul(fc1, fc2w), fc2b)\n", " parameters += [fc2w, fc2b]\n", " \n", " return fc2l\n", " \n", " # Loss function\n", " logits = model(tf_train_dataset)\n", " loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=tf_train_labels))\n", " \n", " # Optimizer\n", " optimizer = tf.train.RMSPropOptimizer(LEARNING_RATE).minimize(loss)\n", " \n", " # Predictions for the training, validation, and test data.\n", " train_prediction = tf.nn.softmax(logits)\n", " valid_prediction = tf.nn.softmax(model(tf_valid_dataset))\n", " test_prediction = tf.nn.softmax(model(tf_test_dataset))" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "e462b95c-dac4-656f-5047-1a1bb4be4eff" }, "source": [ "**Training Phase**" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "dcf2124e-bb6c-9aed-6e74-895db982c605" }, "outputs": [], "source": [ "def accuracy(predictions, labels):\n", " return (100.0 * np.sum(np.argmax(predictions, 1) == np.argmax(labels, 1)) / predictions.shape[0])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "5b2e8f17-a494-54ed-bc5c-0d45aab9e58c" }, "outputs": [], "source": [ "# Create a TensorFlow session\n", "with tf.Session(graph=graph) as sess:\n", "\n", " # Initialize the variables\n", " tf.initialize_all_variables().run()\n", "\n", " # Training loop\n", " for step in range(NUM_STEPS):\n", " offset = (step * BATCH_SIZE) % (train_labels.shape[0] - BATCH_SIZE)\n", " batch_data = train_dataset[offset:(offset + BATCH_SIZE), :, :, :]\n", " batch_labels = train_labels[offset:(offset + BATCH_SIZE), :]\n", " feed_dict = {tf_train_dataset : batch_data, tf_train_labels : batch_labels}\n", " _, l, predictions = sess.run([optimizer, loss, train_prediction], feed_dict=feed_dict)\n", " if step % 50 == 0:\n", " print(\"Minibatch loss at step\", step, \":\", l)\n", " print(\"Minibatch accuracy: %.1f%%\" % accuracy(predictions, batch_labels))\n", " print(\"Validation accuracy: %.1f%%\" % accuracy(valid_prediction.eval(), valid_labels))\n", " \n", " result = test_prediction.eval()" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "6aaca0e8-7c7b-bb60-2dae-259fb4562ce0" }, "source": [ "# Get test data predictions" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "04841e0b-cafe-b722-c029-2f713c41a0fd" }, "outputs": [], "source": [ "# TODO" ] } ], "metadata": { "_change_revision": 0, "_is_fork": false, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.0" } }, "nbformat": 4, "nbformat_minor": 0 }