{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Prepare the Data"},{"metadata":{},"cell_type":"markdown","source":"### Load library\nload basic libraries"},{"metadata":{"trusted":true},"cell_type":"code","source":"import gc\nimport os\nimport keras\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nfrom tqdm import tqdm\nfrom keras import backend as K\n\n# check keras image data format\nprint(K.image_data_format())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Fix random seed"},{"metadata":{"trusted":true},"cell_type":"code","source":"np.random.seed(42)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### load Files\n\ncheck files and load"},{"metadata":{"trusted":true},"cell_type":"code","source":"DATA_PATH = '../input/2019-3rd-ml-month-with-kakr'\nprint(os.listdir(DATA_PATH))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* **train.csv** - train set's image file name, bbox, class\n* **test.csv** - test set's image file name, bbox, class\n* **sample_submission.csv** - submission file corresponding to test.csv\n* **class.csv** - car label corresponding to dataset's class column\n* **train** - train image files\n* **test** - test image files"},{"metadata":{"trusted":true},"cell_type":"code","source":"TRAIN_IMG_PATH = os.path.join(DATA_PATH, 'train')\nTEST_IMG_PATH = os.path.join(DATA_PATH, 'test')\n\n# load .csv files\ndf_train = pd.read_csv(os.path.join(DATA_PATH, 'train.csv'))\ndf_test = pd.read_csv(os.path.join(DATA_PATH, 'test.csv'))\ndf_class = pd.read_csv(os.path.join(DATA_PATH, 'class.csv'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data Exploration\n\nWe try to solve the general question about the data.  \nsuch as whether the actual data matches the description, how the data is structured, and what kind of distribution it has by class.\n"},{"metadata":{},"cell_type":"markdown","source":"Detailed description of each column in the Data Description.\n\n* **img_file** - image file name associated with each row in the dataset\n* **bbox_x1** - Bounding box x1 coordinates (upper left x)\n* **bbox_y1** - Bounding box y1 coordinates (upper left y)\n* **bbox_x2** - Bounding box x2 coordinates (lower right x)\n* **bbox_y2** - Bounding box y2 coordinates (lower right y)\n* **class** - car model to predict(target)\n* **id** - car class id\n* **name** - actual car model label corresponding to class id"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Change the start value of the class to 0\ndf_train['class'] -= 1\ndf_class['id'] -= 1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_test.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# check missing data\nif set(list(df_train['img_file'])) == set(os.listdir(TRAIN_IMG_PATH)):\n    print('No Train file missing')\nelse:\n    print('missing Train file')\n\nif set(list(df_test['img_file'])) == set(os.listdir(TEST_IMG_PATH)):\n    print('No Test file missing')\nelse:\n    print('missing Test file')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# check number of Data\nprint(\"Number of Train Data : {}\".format(df_train.shape[0]))\nprint(\"Number of Test Data : {}\".format(df_test.shape[0]))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_class.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Number of Target class : {}\".format(df_class.shape[0]))\nprint(\"Number of Target class kinds of Training Data : {}\".format(df_train['class'\n].nunique()))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Class Distribution\n\nThe first thing to doubt about the classification problem is the distribution of the Target Class.  \nYou need to check the Target distribution of the Train set and check the balance.  \n"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(15, 6))\nsns.countplot(df_train['class'], order=df_train[\"class\"].value_counts(ascending=True).index)\nplt.title('Number of data per each class')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cntEachClass = df_train['class'].value_counts(ascending=False)\nprint('Class with most count : {}'.format(cntEachClass.index[0]))\nprint(\"Most Count : {}\".format(cntEachClass.max()))\n\nprint('Class with fewest count : {}'.format(cntEachClass.index[-1]))\nprint(\"Fewest Count : {}\".format(cntEachClass.min()))\n\nprint('Mean : {}'.format(cntEachClass.mean()))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"cntEachClass.describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Image Visualization"},{"metadata":{"trusted":true},"cell_type":"code","source":"import cv2\n\ndef load_image(image):\n    img = cv2.imread(image)\n    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n    return img\n\ntmp_imgs = df_train['img_file'][100:110]\nplt.figure(figsize= (12, 20))\n\nfor num, f_name in enumerate(tmp_imgs, start=1):\n    img = load_image(os.path.join(TRAIN_IMG_PATH, f_name))\n    plt.subplot(5, 2, num)\n    plt.title(f_name)\n    plt.imshow(img)\n    plt.axis('off')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"### Bounding Box\n\nWhat is bounding box?\nbounding box refer to the coordinates of a box that labled with a specific object inside the image.  \nNormally, the upper left corner and the lower right coordinates are given."},{"metadata":{"trusted":true},"cell_type":"code","source":"def draw_rect(img, pos, outline, width):\n    p1 = tuple(pos[0:2])\n    p2 = tuple(pos[2:4])\n    img = cv2.rectangle(img, p1, p2, outline, width)\n    return img\n\ndef make_boxing_img(img_name):\n    if img_name.split('_')[0] == 'train':\n        PATH = TRAIN_IMG_PATH\n        data = df_train\n    elif img_name.split('_')[0] == 'test':\n        PATH = TEST_IMG_PATH\n        data = df_test\n    \n    img = load_image(os.path.join(PATH, img_name))\n    pos = data.loc[data['img_file'] == img_name, ['bbox_x1', 'bbox_y1', 'bbox_x2', 'bbox_y2']].values.reshape(-1)\n    img = draw_rect(img, pos, outline=(255, 0, 0), width=10)\n    return img","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"f_name = \"train_00102.jpg\"\n\nplt.figure(figsize=(20, 10))\nplt.subplot(1, 2, 1)\n\n# Original Image\norigin_img = load_image(os.path.join(TRAIN_IMG_PATH, f_name))\nplt.title('Original Image - {}'.format(f_name))\nplt.imshow(origin_img)\nplt.axis('off')\n\n# Image included bounding box\nplt.subplot(1, 2, 2)\nboxing = make_boxing_img(f_name)\nplt.title('Boxing Image - {}'.format(f_name))\nplt.imshow(boxing)\nplt.axis('off')\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Model\n\nNow let's create an image classification model in earnest."},{"metadata":{"trusted":true},"cell_type":"markdown","source":"### Train Valid Test dataset split"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\nits = np.arange(df_train.shape[0])\ntrain_idx, val_idx = train_test_split(its, train_size = 0.8, random_state=42)\n\nX_train = df_train.iloc[train_idx]\nX_val = df_train.iloc[val_idx]\n\nprint(X_train.shape)\nprint(X_val.shape)\nprint(df_test.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(X_train.head())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"### Generator\n\nGenerator can be really useful in a cloud environment   \nsuch as colab or kaggle kernel and in a typical local environment.\nbecause usually these environment don't have enough memory."},{"metadata":{"trusted":true},"cell_type":"markdown","source":"### Keras DataGenerator\n\nKeras has a really nice generator function.\nKeras ImageDataGenerator allow you to define the generator and give the desired noise to the data at the same time."},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras.applications.xception import Xception, preprocess_input\n\nclass CustomDataGenerator(keras.utils.Sequence):\n    def __init__(self, X, y, batch_size, dim, n_channels, n_classes, shuffle=True):\n        self.dim = dim\n        self.batch_size = batch_size\n        self.X = X.values\n        self.y = y.values if y is not None else y\n        self.n_channels = n_channels\n        self.n_classes = n_classes\n        self.shuffle = shuffle\n        self.indexes = np.arange(len(self.X))\n        self.shuffle_index()\n\n    def shuffle_index(self):\n        if self.shuffle:\n            np.random.shuffle(self.indexes)\n\n    def __len__(self):\n        return int(np.floor(len(self.X) / self.batch_size))\n\n    def __getitem__(self, index):\n        indexes = self.indexes[index * self.batch_size: (index + 1) * self.batch_size]\n        X_temp = [self.X[k] for k in indexes]\n\n        if self.y is not None:\n            y_temp = [self.y[k] for k in indexes]\n            X, y = self.__data_generation(X_temp, y_temp)\n            return X, y\n        else:\n            y_temp = None\n            X = self.__data_generation(X_temp, y_temp)\n            return X\n\n    def __data_generation(self, X_temp, list_y_temp):\n        X = np.empty((self.batch_size, *self.dim, self.n_channels))\n        y = np.empty(self.batch_size, dtype=int)\n        if list_y_temp is not None:\n            for i, (img_path, label) in enumerate(zip(X_temp, list_y_temp)):\n                img = cv2.imread(img_path)\n                img = cv2.resize(img, dsize=self.dim)\n                img = preprocess_input(img)\n\n                X[i] = img\n                y[i] = label\n\n            return X, keras.utils.to_categorical(y, num_classes=self.n_classes)\n        else:\n            for i, img_path in enumerate(X_temp):\n                img = cv2.imread(img_path)\n                img = cv2.resize(img, dsize=self.dim)\n                img = preprocess_input(img)\n\n                X[i] = img\n                return X","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Parameter\nimg_size = (299, 299)\nnb_train_samples = len(X_train)\nnb_validation_samples = len(X_val)\nnb_test_samples = len(df_test)\nepochs = 20\nbatch_size = 32\n\ntrain_generator = CustomDataGenerator((TRAIN_IMG_PATH+'/')+X_train['img_file'], \n                                      X_train['class'], \n                                      dim = img_size, \n                                      batch_size=batch_size, \n                                      n_classes=df_class.shape[0], \n                                      n_channels=3, \n                                      shuffle=False)\n\nvalidation_generator = CustomDataGenerator((TRAIN_IMG_PATH+'/')+X_val['img_file'], \n                                           X_val['class'], \n                                           dim = img_size, \n                                           batch_size=batch_size, \n                                           n_classes=df_class.shape[0], \n                                           n_channels=3, \n                                           shuffle=False)\n\ntest_generator = CustomDataGenerator((TEST_IMG_PATH+'/')+df_test['img_file'], \n                                     None, \n                                     dim = img_size, \n                                     batch_size=1, \n                                     n_classes=df_class.shape[0], \n                                     n_channels=3, \n                                     shuffle=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Loading Pre-trained model - Xception\n\n![Xception](https://cdn-images-1.medium.com/max/1600/1*J8dborzVBRBupJfvR7YhuA.png)"},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras.models import Sequential, load_model\nfrom keras.layers import Dense, GlobalAveragePooling2D\n\nxception_model = Xception(include_top=False, weights='imagenet', input_shape = (299, 299, 3))\n\nmodel = Sequential()\nmodel.add(xception_model)\nmodel.add(GlobalAveragePooling2D())\nmodel.add(Dense(df_class.shape[0], activation='softmax', kernel_initializer='he_normal'))\nmodel.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc'])\n\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### define evaluation metric"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.metrics import f1_score\n\ndef micro_f1(y_true, y_pred):\n    return f1_score(y_true, y_pred, average='micro')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Model Training"},{"metadata":{"trusted":true},"cell_type":"code","source":"from math import ceil\n\ndef get_steps(num_samples, batch_size):\n    return ceil(num_samples / batch_size)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"'''\n%%time\n# running time profiling\n\nfrom keras.callbacks import ModelCheckpoint, EarlyStopping\n\nfilepath = \"my_xception_model_{val_acc:.2f}_{val_loss:.4f}.h5\"\n\nckpt = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=False)\nes = EarlyStopping(monitor='var_acc', patience=3, verbose=1)\n\ncallbackList = [ckpt, es]\n\nhist = model.fit_generator(\n    train_generator,\n    steps_per_epoch = get_steps(nb_train_samples, batch_size),\n    epochs = epochs,\n    validation_data = validation_generator,\n    validation_steps = get_steps(nb_validation_samples, batch_size),\n    callbacks = callbackList\n)\n\ngc.collect()\n'''","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Training result\n\n* **default training** - acc : 0.33, loss : 3.3309\n* **imagenet weight + fine-tuning** - acc : 0.70, loss : 1.4957\n* **imagenet weight + fine-tuning + transfer-learning** - acc : 0.71, loss : 1.3773"},{"metadata":{},"cell_type":"markdown","source":"### Load trained weights"},{"metadata":{"trusted":true},"cell_type":"code","source":"WEIGHT_PATH = '../input/car-model-classification-weight'\nprint(os.listdir(WEIGHT_PATH))\n\nMODEL_PATH = os.path.join(WEIGHT_PATH, 'fine-tuning-transfer-learning.h5')\nprint(MODEL_PATH)\n\nmodel = load_model(MODEL_PATH)\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Evaluate Predict & Make submission"},{"metadata":{},"cell_type":"markdown","source":"### Model Evaluation"},{"metadata":{"trusted":true},"cell_type":"code","source":"val_predict = model.predict_generator(\n    generator = validation_generator,\n    steps = get_steps(nb_validation_samples, batch_size),\n    verbose = 1\n)\nval_predict = np.argmax(val_predict, axis=1)[:X_val.shape[0]]\nf1_score = micro_f1(X_val['class'].values, val_predict)\nprint(\"f1_score : {:.3}\".format(f1_score))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Model predict"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\nprediction = model.predict_generator(\n    generator = test_generator,\n    steps = nb_test_samples,\n    verbose = 1\n)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"### Make submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"predicted_class_indices = np.argmax(prediction, axis=1)[:df_test.shape[0]]\nsubmission = pd.read_csv(os.path.join(DATA_PATH, 'sample_submission.csv'))\n\nsubmission['class'] = predicted_class_indices + 1\nprint(submission['class'])\nsubmission.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}