{"nbformat_minor": 1, "metadata": {"language_info": {"file_extension": ".py", "codemirror_mode": {"version": 3, "name": "ipython"}, "name": "python", "mimetype": "text/x-python", "version": "3.6.2", "pygments_lexer": "ipython3", "nbconvert_exporter": "python"}, "kernelspec": {"language": "python", "display_name": "Python 3", "name": "python3"}}, "nbformat": 4, "cells": [{"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true}, "source": ["%matplotlib inline\n", "import pandas as pd\n", "import numpy as np\n", "import matplotlib.pyplot as plt"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["# Data preparations\n", "\n", "Let's read data and review it:"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "source": ["df = pd.read_csv(\"../input/labels.csv\")\n", "df.head()"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {}, "source": ["df.describe(include=\"all\")"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["So - we have 2 fields (id also is a part of image filename - e.g. \"../input/train/${id}.jpg\").\n", "Let's make binary features from breeds:"]}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true}, "source": ["breed_codes = list(set(df[\"breed\"]))\n", "breed_codes.sort()\n", "\n", "for code in breed_codes:\n", "    df[code] = 1.0 * (df[\"breed\"] == code)"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["And review breeds distribution:"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "source": ["from sklearn.preprocessing import LabelEncoder\n", "\n", "plt.hist(LabelEncoder().fit_transform(df[\"breed\"]), bins=50);"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["As you can see - we have small (up to 2 times) class imbalance, so I used sample weighting (given ~=0.005 improvement).\n", "\n", "# Image processing.\n", "\n", "I'll process data next way:\n", "- use pre-trained Resnet50 with imagenet weights and without output dense layer (for feature extraction)\n", "- group train/test image id in batches. For each batch:\n", "    - load images\n", "    - resize images (we'll need 224x224 for Resnet50)\n", "    - apply feature extraction to each image and store result.\n", "\n", "After all I'll have resnet50-based features for each image - so I'll can train my own dense network on it.\n", "\n", "Let's define image loading functions:"]}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true}, "source": ["from scipy.misc import imread, imresize\n", "\n", "\n", "def load_train_image(id):\n", "    return imresize(imread(\"../input/train/{0}.jpg\".format(id)), \n", "                    (224, 224))\n", "\n", "\n", "def load_test_image(id):\n", "    return imresize(imread(\"../input/test/{0}.jpg\".format(id)),\n", "                    (224,224))"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["And some utils functions:"]}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true}, "source": ["from ipywidgets import IntProgress\n", "from IPython.display import display\n", "\n", "\n", "def log_progress(sequence, every=10):\n", "    progress = IntProgress(min=0, max=len(sequence), value=0)\n", "    display(progress)\n", "    for index, record in enumerate(sequence):\n", "        if index % every == 0:\n", "            progress.value = index\n", "        yield record\n", "    progress.value = len(sequence)\n", "    \n", "    \n", "def chunks(lst, size):\n", "    \"\"\"Yield successive n-sized chunks from l.\"\"\"\n", "    result = []\n", "    for i in range(0, len(lst), size):\n", "        result.append(lst[i:i + size])\n", "    return result"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["Let's check image loading:"]}, {"cell_type": "code", "execution_count": null, "metadata": {"scrolled": true}, "source": ["plt.imshow(load_train_image(\"000bec180eb18c7604dcecc8fe0dba07\"));"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {}, "source": ["plt.imshow(load_test_image(\"00a3edd22dc7859c487a64777fc8d093\"));"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["Now let's define resnet50 feature extractor function.\n", "It'll consume:\n", "- image ids\n", "- image loader (function, that return image by id)"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "source": ["from keras.applications import ResNet50\n", "\n", "\n", "resnet = ResNet50(include_top=False, weights='imagenet')\n", "\n", "\n", "def get_resnet_features(ids, loader):\n", "    id_chunks = chunks(ids, 10)\n", "    resnet_output = {}\n", "    for chunk in log_progress(id_chunks, every=1):\n", "        images = []\n", "        for image_id in chunk:\n", "            image = loader(image_id)\n", "            images.append(image)\n", "        predictions = resnet.predict(np.array(images))\n", "        for i, image_id in enumerate(chunk):\n", "            resnet_output[image_id] = predictions[i]\n", "    return resnet_output"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {}, "source": ["train_resnet_features = get_resnet_features(df[\"id\"], load_train_image)"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["# Dense network building/fitting\n", "\n", "Let's build our dense network."]}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true}, "source": ["from keras.models import Sequential\n", "from keras.layers import Flatten, Dense, Dropout\n", "from keras import regularizers\n", "\n", "model = Sequential()\n", "model.add(Flatten(input_shape=train_resnet_features[df[\"id\"][0]].shape))\n", "model.add(Dropout(0.5))\n", "model.add(Dense(2 * len(breed_codes),\n", "                activation='relu'))\n", "model.add(Dropout(0.5))\n", "model.add(Dense(len(breed_codes),\n", "                activation='softmax'))\n", "model.compile(\"sgd\", \"categorical_crossentropy\")"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["We have imbalanced classes, so let's calculate sample weights:"]}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true}, "source": ["class_counts = np.array([(df[\"breed\"] == breed).sum() \n", "                         for breed in breed_codes])\n", "class_weights = class_counts.mean() / class_counts\n", "class_weights_dict = {cls: class_weights[i] for i, cls in enumerate(breed_codes)}\n", "sample_weights = np.array([class_weights_dict[breed]\n", "                           for breed in df[\"breed\"]])"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["Now let's fit network:"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "source": ["from keras.callbacks import EarlyStopping\n", "\n", "\n", "X_train = np.array([train_resnet_features[image_id]\n", "                    for image_id in df[\"id\"]])\n", "y_train = np.array(df[breed_codes])\n", "\n", "model.fit(X_train, y_train, \n", "          epochs=1000,\n", "          sample_weight=sample_weights,\n", "          verbose=True,\n", "          validation_split=0.3,\n", "          callbacks=[\n", "              EarlyStopping(min_delta=1e-4, patience=10)\n", "          ])"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["# Test prediction\n", "\n", "Let's get ids for test images and extract features:"]}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true}, "source": ["from os import listdir\n", "from os.path import isfile, join\n", "\n", "\n", "def get_test_image_ids():\n", "    test_dir = \"../input/test\"\n", "    test_files = filter(isfile, map(lambda fname: join(test_dir, fname), listdir(test_dir)))\n", "    ids = map(lambda fname: fname.split('/')[-1].split('\\\\')[-1].split('.')[0], test_files)\n", "    return list(ids)"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {}, "source": ["test_image_ids = get_test_image_ids()\n", "test_resnet_features = get_resnet_features(test_image_ids, load_test_image)"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["Now let's make prediction for this features"]}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true}, "source": ["X_test = np.array([test_resnet_features[image_id]\n", "                   for image_id in test_image_ids])\n", "test_prediction = model.predict(X_test)"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["And save result:"]}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true}, "source": ["from collections import OrderedDict\n", "\n", "test_df_dict = OrderedDict([(\"id\", test_image_ids)])\n", "for breed_index, breed in enumerate(breed_codes):\n", "    test_df_dict[breed] = test_prediction[:, breed_index]\n", "pd.DataFrame(test_df_dict).to_csv(\"../output/resnet50-dense-dense.csv\", index=False)"], "outputs": []}, {"cell_type": "markdown", "metadata": {}, "source": ["# Model saving\n", "\n", "Let's save full model"]}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true}, "source": ["from keras.layers import InputLayer\n", "\n", "\n", "result_model = Sequential()\n", "result_model.add(InputLayer(input_shape=(224, 224, 3)))\n", "result_model.add(resnet)\n", "result_model.add(model)\n", "result_model.compile(\"sgd\", \"categorical_crossentropy\")\n", "result_model.save(\"../output/result_model.h5\")"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true}, "source": ["result_model.predict(np.array([load_train_image(\"000bec180eb18c7604dcecc8fe0dba07\")]))"], "outputs": []}, {"cell_type": "code", "execution_count": null, "metadata": {"collapsed": true}, "source": [], "outputs": []}]}