{"cells":[{"metadata":{},"cell_type":"markdown","source":"

2019 3rd ML month with KaKR

\n
자동차 이미지 데이터셋을 이용한 자동차 차종 분류
\n\n
\n\n
\n \n
\n\n
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n

Kaggle Kerenl Reference

Author
Kernel
TaeJin Kim
[3rd ML Month] Car Model Classification Baseline
Taemyung Heo
[3rd ML Month] Car Image Cropping
Jang
[3rd ML Month] Xception, StratifiedKFold, Ensemble
\n\n
\n\n\n \n \n \n \n \n \n \n \n \n \n \n

Discussion Reference

Author
Discussion
Steve Jang
커널 commit 진행상황 확인하는 tip
\n\n
\n\n### Used Pre-tained Model\n\n| MODEL_N | Pretained Model |\n|:-------:|:-----------------:|\n| 0 | Xception |\n| 1 | InceptionV3 |\n| 2 | InceptionResNetV2 |\n| 3 | EfficientNetB4 |\n| 4 | EfficientNetB5 |\n| 5 | Xception |\n| 6 | InceptionV3 |\n| 7 | InceptionResNetV2 |\n| 8 | EfficientNetB4 |\n| 9 | EfficientNetB5 |\n| 10 | Xception |\n| 11 | InceptionV3 |\n| 12 | InceptionResNetV2 |\n| 13 | EfficientNetB4 |\n| 14 | EfficientNetB5 |\n| 15 | Xception |\n| 16 | InceptionV3 |\n| 17 | InceptionResNetV2 |\n| 18 | EfficientNetB4 |\n| 19 | EfficientNetB5 |\n\n> [EfficientNet](https://github.com/qubvel/efficientnet)\n\nValidation data set is splited with `StratifiedKFold`. \nAs a result, Validation data set (5%) is different per model. \n\n
\n\n### Contents\n\n- EDA\n- Image Preprocessing\n - Cropping & Resize\n - CLAHE (Contrast Limited Adaptive Histogram Equalization)\n- Image Augmentation: CutMixImageDataGenerator + imgaug\n - [CutMixImageDataGenerator_For_Keras](https://github.com/DevBruce/CutMixImageDataGenerator_For_Keras) (Self-Production)\n - [imgaug](https://github.com/aleju/imgaug)\n- Ensemble 10 Saved Model (0 ~ 9)\n\n
\n\n**If `ENSEMBLE_MODE = True`, this kernel just ensembles uploaded saved models. \nSo if you want to train and save a model file, \nyou have to set `ENSEMBLE_MODE = False` and set `MODEL_N` you want to train & save.**\n\n
\n\n### Note\n\n- When all models were ensembled (0 ~ 19), Submit Score dropped. \n(You can see the result in Version 1 of this Kernel) \n\n- When ensembled pretained model EfficientNetB7 (0 ~ 9, total 10 models), Submit Score dropped. \n(You can see the result in Version 2 of this kernel) \n\nSo, This kernel just ensemble 10 models (0 ~ 9)"},{"metadata":{},"cell_type":"markdown","source":"## Install Library\n\n- [EfficientNet](https://github.com/qubvel/efficientnet)\n- [imgaug](https://github.com/aleju/imgaug)"},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install efficientnet==0.0.4\n!pip install git+https://github.com/aleju/imgaug.git","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Model Settings"},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nfrom keras.applications.xception import Xception # (299 x 299)\nfrom keras.applications.inception_v3 import InceptionV3 # (299 x 299)\nfrom keras.applications.inception_resnet_v2 import InceptionResNetV2 # (299 x 299)\nfrom efficientnet import EfficientNetB4, EfficientNetB5 # (299 x 299)\n\n\nMODEL_N = 0 # 0 ~ 19 (!important)\nMODEL_N_STR = '0' + str(MODEL_N) if len(str(MODEL_N)) == 1 else str(MODEL_N)\nMODEL_LIST = (\n Xception, InceptionV3, InceptionResNetV2, EfficientNetB4, EfficientNetB5,\n Xception, InceptionV3, InceptionResNetV2, EfficientNetB4, EfficientNetB5,\n Xception, InceptionV3, InceptionResNetV2, EfficientNetB4, EfficientNetB5,\n Xception, InceptionV3, InceptionResNetV2, EfficientNetB4, EfficientNetB5,\n)\nPRETAINED_MODEL = MODEL_LIST[MODEL_N]\nif MODEL_N in (0, 5, 10, 15): PRETAINED_MODEL_STR = 'Xception'\nelif MODEL_N in (1, 6, 11, 16): PRETAINED_MODEL_STR = 'InceptionV3'\nelif MODEL_N in (2, 7, 12, 17): PRETAINED_MODEL_STR = 'InceptionResNetV2'\nelif MODEL_N in (3, 8, 13, 18): PRETAINED_MODEL_STR = 'EfficientNetB4'\nelif MODEL_N in (4, 9, 14, 19): PRETAINED_MODEL_STR = 'EfficientNetB5'\n\n# Ensemble Mode\nINPUT_PATH_ORIGIN = os.path.join('..', 'input')\nUPLOADED_MODEL_PATH = os.path.join(\n os.path.join(INPUT_PATH_ORIGIN, 'kakr-2019-3rd-cutmix-saved-models'),\n 'kakr_2019_3rd_cutmix_saved_models'\n)\nUPLOADED_MODEL_PATH = os.path.join(UPLOADED_MODEL_PATH, 'kakr_2019_3rd_cutmix_saved_models')\nENSEMBLE_MODE = True # (!importrant)\nif ENSEMBLE_MODE and not os.path.isdir(UPLOADED_MODEL_PATH):\n raise # ENSEMBLE_MODE = True, but no uploaded model files\n\n# Full Data Mode\n# If False, Lightweight Data Mode For Test (Get 20% Of Data)\nFULL_DATA = True\nif FULL_DATA: LIGHT_DATA_TRAIN, LIGHT_DATA_TEST = False, False\nelse: LIGHT_DATA_TRAIN, LIGHT_DATA_TEST = True, True\n \n# Size Of Pretained Model (IMG_SIZE x IMG_SIZE)\nIMG_SIZE = 299","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Initial Settings"},{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","trusted":true},"cell_type":"code","source":"import gc\nimport random\nimport warnings\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport cv2\nimport PIL\nfrom PIL import ImageDraw\nfrom keras import regularizers\nfrom keras import models, layers, optimizers\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.callbacks import EarlyStopping, ReduceLROnPlateau, ModelCheckpoint, Callback\nfrom sklearn.model_selection import train_test_split, StratifiedKFold\nfrom sklearn.utils.class_weight import compute_class_weight\nimport imgaug as ia\nfrom imgaug import augmenters as iaa\n\n\n# Ignore Warnings\nwarnings.filterwarnings('ignore')\n\n# Class Weights\nGET_CLASS_WEIGHTS = False\n\n# Undersampling class 119 for class balance\nUNDERSAMPLING_119 = False\n\n# If EfficientNet\nis_EfficientNet = PRETAINED_MODEL_STR in ('EfficientNetB4', 'EfficientNetB5')\n\n# Constant Variables\nLEARNING_RATE = 1e-4\nOPTIMIZER = optimizers.Adam(lr=LEARNING_RATE)\nEPOCHS = 80 if FULL_DATA else 4\nEPOCHS = 64 if (FULL_DATA and is_EfficientNet) else EPOCHS\nBATCH_SIZE = 16 # if set high value, it would occur ResourceExhaustedError\nINITIALIZER = 'he_normal'\nREGULARIZER = regularizers.l2(1e-3)\nDROPOUT_RATE = 0.5\nPATIENCE = 12 if is_EfficientNet else 16 # Patience Value For Early Stop\nVERBOSE = 2 if FULL_DATA else 1 # Verbosity mode (2: No progress bar)\nRANDOM_SEED = 2019\nia.seed(RANDOM_SEED)\n\n# Set Basic Path\nINPUT_PATH = os.path.join('..', 'input')\nif os.path.isdir(os.path.join(INPUT_PATH, '2019-3rd-ml-month-with-kakr')):\n INPUT_PATH = os.path.join(INPUT_PATH, '2019-3rd-ml-month-with-kakr')\nTRAIN_IMG_PATH = os.path.join(INPUT_PATH, 'train')\nTEST_IMG_PATH = os.path.join(INPUT_PATH, 'test')\n\n# Set Path of Cropped Train Images\nTRAIN_IMG_CROP_PATH = os.path.join('..', 'train_crop')\nif not os.path.exists(TRAIN_IMG_CROP_PATH):\n os.makedirs(TRAIN_IMG_CROP_PATH, exist_ok=True)\n\n# Set Path of Cropped Test Images\nTEST_IMG_CROP_PATH = os.path.join('..', 'test_crop')\nif not os.path.exists(TEST_IMG_CROP_PATH):\n os.makedirs(TEST_IMG_CROP_PATH, exist_ok=True)\n \n# Set Path of Histogram Equalized Train Images\nTRAIN_IMG_PREP_PATH = os.path.join('..', 'train_prep')\nif not os.path.exists(TRAIN_IMG_PREP_PATH):\n os.makedirs(TRAIN_IMG_PREP_PATH, exist_ok=True)\n\n# Set Path of Histogram Equalized Test Images\nTEST_IMG_PREP_PATH = os.path.join('..', 'test_prep')\nif not os.path.exists(TEST_IMG_PREP_PATH):\n os.makedirs(TEST_IMG_PREP_PATH, exist_ok=True)\n\n# Set Path of Model\nMODEL_FILE_SAVE_DIR_PATH = '.'\nif not os.path.exists(MODEL_FILE_SAVE_DIR_PATH):\n os.makedirs(MODEL_FILE_SAVE_PATH, exist_ok=True)\nMODEL_FILE_PATH = os.path.join(\n MODEL_FILE_SAVE_DIR_PATH,\n MODEL_N_STR + '_' + PRETAINED_MODEL_STR + '.hdf5'\n)\n\n# Load DataFrame (Read CSV)\ndf_class = pd.read_csv(os.path.join(INPUT_PATH, 'class.csv')) # Label of class column (df_train)\ndf_train = pd.read_csv(os.path.join(INPUT_PATH, 'train.csv'))\ndf_test = pd.read_csv(os.path.join(INPUT_PATH, 'test.csv'))\n# df_class: | id | name |\n# df_train: | img_file | bbox_x1 | bbox_y1 | bbox_x2 | bbox_y2 | class |\n# df_test: | img_file | bbox_x1 | bbox_y1 | bbox_x2 | bbox_y2 | |\n\n# Number Of Class\nCLASS_NB = len(df_class) # == 196","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# System Log Settings"},{"metadata":{"trusted":true},"cell_type":"code","source":"from datetime import datetime\nfrom pytz import timezone, utc\n\n\nKST = timezone('Asia/Seoul')\n\ndef system_print(string): \n os.system(f'echo \\\"{string}\\\"')\n print(string)\n\n# To Add CallBack List\nclass PrintSystemLogPerEpoch(Callback):\n def on_epoch_begin(self, epoch, logs={}):\n t = utc.localize(datetime.utcnow()).astimezone(KST).time()\n system_print(f'* [Epoch {epoch+1}] begins at {t}')\n def on_epoch_end(self, epoch, logs={}):\n t = utc.localize(datetime.utcnow()).astimezone(KST).time()\n system_print(f'* [Epoch {epoch+1}] ends at {t} | acc={logs[\"acc\"]:0.4f}, val_acc={logs[\"val_acc\"]:0.4f}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Lightweight Data Set For Test"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Get 20% Of Original Train Data Set\nif LIGHT_DATA_TRAIN:\n _, df_train = train_test_split(\n df_train,\n train_size=0.8,\n random_state=RANDOM_SEED,\n stratify=df_train['class'],\n )\n\n# Get 10% Of Original Test Data Set\nif LIGHT_DATA_TEST:\n _, df_test = train_test_split(\n df_test,\n train_size=0.8,\n random_state=RANDOM_SEED,\n )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# EDA"},{"metadata":{},"cell_type":"markdown","source":"## DataFrame Information"},{"metadata":{"trusted":true},"cell_type":"code","source":"print('------ df_class ---' + '-' * 15)\ndf_class.info()\nprint('\\n------ df_train ---' + '-' * 15)\ndf_train.info()\nprint('\\n------ df_test ---' + '-' * 15)\ndf_test.info()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- **There are not null data**\n\n- **There are just two types of data**\n - object\n - int64"},{"metadata":{},"cell_type":"markdown","source":"## Check Missing Data"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Check train image files missing\nif set(df_train.img_file) == set(os.listdir(TRAIN_IMG_PATH)): # Ignore ordering with set\n print(\"Train image files are no Problem\")\nelse: print(\"There is some missing train image files\")\n\n# Check test image files missing\nif set(df_test.img_file) == set(os.listdir(TEST_IMG_PATH)):\n print(\"Test image files are no Problem\")\nelse: print(\"There is some missing test image files\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"car_class_nb = df_class.shape[0]\ntrain_class_nb = df_train['class'].nunique()\nprint(f'Number of car class: {car_class_nb}')\nprint(f'Number of train data class: {train_class_nb}')\n\nif car_class_nb == train_class_nb:\n print(\"No problem\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Class Distribution\n\n- Check balance of number of classes"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_nb_per_class = df_train['class'].value_counts(ascending=True)\ntrain_nb_per_class.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(f'Min Count Class: {train_nb_per_class.index[0]}')\nprint(f'Max Count Class: {train_nb_per_class.index[-1]}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"- Min Count Class: 136 (Value: 30)\n- Max Count Class: 119 (Value: 84)\n- Mean: 51.10\n- Standard Deviation: 5.35"},{"metadata":{},"cell_type":"markdown","source":"### Visualization"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(16,8))\nplt.title('Number of train data per class')\nsns.countplot(df_train['class'])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Class 119 is too high**"},{"metadata":{},"cell_type":"markdown","source":"### Undersampling (Delete Some Class 119 For Class Balance)\n\nDelete 25% rows of class 119. \nAs a result, the number of class 119 will be balanced"},{"metadata":{"trusted":true},"cell_type":"code","source":"random.seed(RANDOM_SEED)\n\nif UNDERSAMPLING_119:\n class_119_idx_bool = (df_train['class'] == 119)\n class_119_idx_nb = sum(class_119_idx_bool)\n class_119_idx_list = list(df_train[class_119_idx_bool].index)\n class_119_undersampling_idx = random.sample(class_119_idx_list, int(class_119_idx_nb*0.25))\n\n # Delete 22 rows(119 Class) For Undersampling\n for idx in class_119_undersampling_idx:\n df_train = df_train.drop(idx)\n\n df_train = df_train.reset_index(drop=True)\n \n # Show Undersampling Result\n plt.figure(figsize=(16,8))\n plt.title('Number of train data per class After Undersampling')\n sns.countplot(df_train['class'])\n plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Get Class Weights\n\nTo Resolve Class Imbalance"},{"metadata":{"trusted":true},"cell_type":"code","source":"if GET_CLASS_WEIGHTS:\n class_weights = compute_class_weight('balanced', np.unique(df_train['class']), df_train['class'])\n class_weights_dict = dict(zip(np.unique(df_train['class']), class_weights))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Load Images"},{"metadata":{"trusted":true},"cell_type":"code","source":"tmp_imgs = df_train['img_file'][:4] # Load 4 images (0, 1, 2, 3)\nplt.figure(figsize=(14,10))\n\nfor idx, img_file_name in enumerate(tmp_imgs, 1):\n img = PIL.Image.open(os.path.join(TRAIN_IMG_PATH, img_file_name))\n plt.subplot(2, 2, idx)\n plt.title(img_file_name)\n plt.imshow(img)\n plt.axis('off')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Image Preprocessing - Crop & Resize"},{"metadata":{"trusted":true},"cell_type":"code","source":"def img_crop_resize(img_file_name, resize_val, margin=6):\n if img_file_name.split('_')[0] == 'train':\n IMG_PATH = TRAIN_IMG_PATH\n data = df_train\n elif img_file_name.split('_')[0] == 'test':\n IMG_PATH = TEST_IMG_PATH\n data = df_test\n \n img = PIL.Image.open(os.path.join(IMG_PATH, img_file_name))\n pos = data.loc[data['img_file'] == img_file_name, ['bbox_x1','bbox_y1', 'bbox_x2', 'bbox_y2']].values.reshape(-1)\n width, height = img.size\n x1 = max(0, pos[0] - margin)\n y1 = max(0, pos[1] - margin)\n x2 = min(pos[2] + margin, width)\n y2 = min(pos[3] + margin, height)\n return img.crop((x1, y1, x2, y2)).resize(resize_val)\n\n\n# Define Function For Test\ndef test_crop(img_file_name):\n # Show Original Image\n img_origin = PIL.Image.open(os.path.join(TRAIN_IMG_PATH, img_file_name))\n plt.figure(figsize=(16,12))\n plt.subplot(1, 2, 1)\n plt.title(f'Original Image - {img_file_name}')\n plt.imshow(img_origin)\n plt.axis('off')\n \n # Show Crop Image\n img_crop = img_crop_resize(img_file_name, resize_val=(IMG_SIZE, IMG_SIZE))\n plt.subplot(1, 2, 2)\n plt.title(f'Cropped & Resized Image - {img_file_name}')\n plt.imshow(img_crop)\n plt.axis('off')\n \n # Show Result\n plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Test bbox and crop\ntest_crop(img_file_name=df_train['img_file'].iloc[100])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Save Crop Images\n\n> Cropped Train Images Path: `'/kaggle/train_crop'` (`../train_crop`) \n> Cropped Test Images Path: `'/kaggle/test_crop'` (`../test_crop`) "},{"metadata":{"trusted":true},"cell_type":"code","source":"# Save Cropped Train Images (Path: /kaggle/working/train_crop)\nif not os.listdir(TRAIN_IMG_CROP_PATH): # If PATH_IMG_TRAIN_CROP is empty\n for idx, row in df_train.iterrows():\n img_file_name = row['img_file']\n img_crop = img_crop_resize(img_file_name, resize_val=(IMG_SIZE, IMG_SIZE))\n img_crop.save(os.path.join(TRAIN_IMG_CROP_PATH, img_file_name))\n\n# Save Cropped Test Images (Path: /kaggle/working/test_crop)\nif not os.listdir(TEST_IMG_CROP_PATH):\n for idx, row in df_test.iterrows():\n img_file_name = row['img_file']\n img_crop = img_crop_resize(img_file_name, resize_val=(IMG_SIZE, IMG_SIZE))\n img_crop.save(os.path.join(TEST_IMG_CROP_PATH, img_file_name))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Image Preprocessing - CLAHE\n\nContrast Limited Adaptive Histogram Equalization"},{"metadata":{"trusted":true},"cell_type":"code","source":"def img_clahe(img_file_name):\n if img_file_name.split('_')[0] == 'train':\n IMG_CROP_PATH = TRAIN_IMG_CROP_PATH\n data = df_train\n elif img_file_name.split('_')[0] == 'test':\n IMG_CROP_PATH = TEST_IMG_CROP_PATH\n data = df_test\n \n # --- Histogram Equalization --- #\n img = cv2.imread(os.path.join(IMG_CROP_PATH, img_file_name))\n img_y_cr_cb = cv2.cvtColor(img, cv2.COLOR_BGR2YCrCb)\n y, cr, cb = cv2.split(img_y_cr_cb)\n\n # Equalize y (CLAHE (Contrast Limited Adaptive Histogram Equalization))\n clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(6,6))\n y_eq = clahe.apply(y)\n img_y_cr_cb_eq = cv2.merge((y_eq, cr, cb))\n img_bgr_eq = cv2.cvtColor(img_y_cr_cb_eq, cv2.COLOR_YCR_CB2BGR)\n img_prep = img_bgr_eq\n\n # --- Convert BGR To RGB (Just On cv2) ---\n # b, g, r = cv2.split(img_bgr_eq)\n # img_prep = cv2.merge((r,g,b))\n return img_prep","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Save CLAHE Images\n\n> CLAHE Train Images Path: `'/kaggle/train_prep'` (`../train_prep`) \n> CLAHE Test Images Path: `'/kaggle/test_prep'` (`../test_prep`) "},{"metadata":{"trusted":true},"cell_type":"code","source":"# Save CLAHE Train Images (Path: ../train_prep)\nif not os.listdir(TRAIN_IMG_PREP_PATH): # If PATH_IMG_TRAIN_PREP is empty\n for idx, row in df_train.iterrows():\n img_file_name = row['img_file']\n img_prep = img_clahe(img_file_name)\n cv2.imwrite(os.path.join(TRAIN_IMG_PREP_PATH, img_file_name), img_prep)\n\n# Save CLAHE Test Images (Path: ../test_prep)\nif not os.listdir(TEST_IMG_PREP_PATH):\n for idx, row in df_test.iterrows():\n img_file_name = row['img_file']\n img_prep = img_clahe(img_file_name)\n cv2.imwrite(os.path.join(TEST_IMG_PREP_PATH, img_file_name), img_prep)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Test CLAHE"},{"metadata":{"trusted":true},"cell_type":"code","source":"def test_clahe(img_file_name):\n # Show Cropped Image\n img_crop = PIL.Image.open(os.path.join(TRAIN_IMG_CROP_PATH, img_file_name))\n plt.figure(figsize=(12, 9))\n plt.subplot(1, 2, 1)\n plt.title(f'Cropped & Resized Image - {img_file_name}')\n plt.imshow(img_crop)\n plt.axis('off')\n\n # Show CLAHE Image\n img_clahe = PIL.Image.open(os.path.join(TRAIN_IMG_PREP_PATH, img_file_name))\n plt.subplot(1, 2, 2)\n plt.title(f'CLAHE - {img_file_name}')\n plt.imshow(img_clahe)\n plt.axis('off')\n \n # Show Result\n plt.show()\n \n# Test CLAHE Image\ntest_clahe(img_file_name=df_train['img_file'].iloc[114])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"---"},{"metadata":{},"cell_type":"markdown","source":"# Split Data With StratifiedKFold\n\n- `idx_split[n][0]` : Splited Train index\n- `idx_split[n][1]` : Splited Validation Index\n- `n_splits=20`: Validation Set Ratio 5%"},{"metadata":{"trusted":true},"cell_type":"code","source":"# class_mode=\"categorical\", y_col=\"class\" column values must be type string (flow method options)\ndf_train['class'] = df_train['class'].astype('str')\n\n# Take Necessary Columns\ndf_train = df_train[['img_file', 'class']]\n\n# K-Fold Cross Validation\nidx_split = list()\nskf = StratifiedKFold(n_splits=20, random_state=RANDOM_SEED)\nfor train_idx, val_idx in skf.split(X=df_train['img_file'], y=df_train['class']):\n idx_split.append((train_idx, val_idx))\n \nX_train = df_train.iloc[idx_split[MODEL_N][0]].reset_index()\nX_val = df_train.iloc[idx_split[MODEL_N][1]].reset_index()\nX_test = df_test[['img_file']]\n\n# Delete Unncessary Data For Memory\ndel df_train\ndel df_test\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Model Definition"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Define Get Model Function\ndef get_model(pretained_model, img_size, initializer, regularizer, optimizer, dropout_rate, class_nb):\n model = models.Sequential()\n model.add(\n pretained_model(\n include_top=False,\n weights='imagenet',\n input_shape=(img_size, img_size, 3)\n )\n )\n model.add(layers.GlobalAveragePooling2D())\n model.add(layers.Dense(\n 1024,\n activation='relu',\n kernel_initializer=initializer,\n kernel_regularizer=regularizer,\n ))\n model.add(layers.Dropout(dropout_rate))\n model.add(layers.Dense(class_nb, activation='softmax'))\n model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['acc'])\n return model\n\n\n# Define Get Step Function\ndef get_steps(num_data, batch_size):\n quotient, remainder = divmod(num_data, batch_size)\n return (quotient + 1) if remainder else quotient","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Callback\n\n- Early Stop\n- Reduce Learning Rate\n- Model Check Point (Save Model File)"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Define Get Callback Function\ndef get_callback_list(model_file_path, patience, verbose, system_print=False):\n early_stop = EarlyStopping(\n monitor='val_loss',\n patience=patience,\n verbose=verbose,\n mode='min',\n )\n reduce_lr = ReduceLROnPlateau(\n monitor='val_loss',\n factor=0.5,\n patience=patience//2,\n verbose=verbose,\n mode='min',\n min_lr=1e-6,\n )\n model_chk = ModelCheckpoint(\n filepath=model_file_path,\n monitor='val_loss',\n verbose=verbose,\n save_best_only=True,\n mode='min',\n )\n callback_list = [early_stop, reduce_lr, model_chk]\n if system_print:\n callback_list.append(PrintSystemLogPerEpoch())\n return callback_list","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Generator"},{"metadata":{},"cell_type":"markdown","source":"## CutMixImageDataGenerator + imgaug\n\n- [CutMixImageDataGenerator_For_Keras](https://github.com/DevBruce/CutMixImageDataGenerator_For_Keras) (Self-Production)\n- [imgaug](https://github.com/aleju/imgaug)\n\nPut CutMixImageDataGenerator (Self-Production) and sequential of imgaug together"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Set imgaug Sequential\nseq = iaa.Sequential(\n [\n iaa.Affine(rotate=(-15, 15)),\n iaa.Fliplr(0.5),\n iaa.GaussianBlur((0, 2.0)),\n iaa.ElasticTransformation(alpha=(0, 70), sigma=9),\n iaa.AdditiveGaussianNoise(scale=(0, 0.05), per_channel=True),\n iaa.ChannelShuffle(p=0.5),\n ],\n random_order=False,\n)\n\nclass CutMixImageDataGenerator():\n def __init__(self, generator1, generator2, img_size, batch_size):\n self.batch_index = 0\n self.samples = generator1.samples\n self.class_indices = generator1.class_indices\n self.generator1 = generator1\n self.generator2 = generator2\n self.img_size = img_size\n self.batch_size = batch_size\n\n def reset_index(self): # Ordering Reset (If Shuffle is True, Shuffle Again)\n self.generator1._set_index_array()\n self.generator2._set_index_array()\n\n def reset(self):\n self.batch_index = 0\n self.generator1.reset()\n self.generator2.reset()\n self.reset_index()\n\n def get_steps_per_epoch(self):\n quotient, remainder = divmod(self.samples, self.batch_size)\n return (quotient + 1) if remainder else quotient\n \n def __len__(self):\n self.get_steps_per_epoch()\n\n def __next__(self):\n if self.batch_index == 0: self.reset()\n\n crt_idx = self.batch_index * self.batch_size\n if self.samples > crt_idx + self.batch_size:\n self.batch_index += 1\n else: # If current index over number of samples\n self.batch_index = 0\n\n reshape_size = self.batch_size\n last_step_start_idx = (self.get_steps_per_epoch()-1) * self.batch_size\n if crt_idx == last_step_start_idx:\n reshape_size = self.samples - last_step_start_idx\n \n X_1, y_1 = self.generator1.next()\n X_2, y_2 = self.generator2.next()\n \n cut_ratio = np.random.beta(a=1, b=1, size=reshape_size)\n cut_ratio = np.clip(cut_ratio, 0.2, 0.8)\n label_ratio = cut_ratio.reshape(reshape_size, 1)\n cut_img = X_2\n\n X = X_1\n for i in range(reshape_size):\n cut_size = int((self.img_size-1) * cut_ratio[i])\n y1 = random.randint(0, (self.img_size-1) - cut_size)\n x1 = random.randint(0, (self.img_size-1) - cut_size)\n y2 = y1 + cut_size\n x2 = x1 + cut_size\n cut_arr = cut_img[i][y1:y2, x1:x2]\n cutmix_img = X_1[i]\n cutmix_img[y1:y2, x1:x2] = cut_arr\n X[i] = cutmix_img\n \n X = seq.augment_images(X) # Sequential of imgaug\n y = y_1 * (1 - (label_ratio ** 2)) + y_2 * (label_ratio ** 2)\n return X, y\n\n def __iter__(self):\n while True:\n yield next(self)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Generator Definition"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Settings For CutMix\ntrain_datagen = ImageDataGenerator(\n brightness_range=(0.6, 1.4),\n horizontal_flip=True,\n vertical_flip=False,\n rescale=1./255,\n)\ntrain_generator1 = train_datagen.flow_from_dataframe(\n dataframe=X_train,\n directory=TRAIN_IMG_PREP_PATH,\n x_col='img_file',\n y_col='class',\n target_size=(IMG_SIZE, IMG_SIZE),\n color_mode='rgb',\n class_mode='categorical',\n batch_size=BATCH_SIZE,\n shuffle=True,\n)\ntrain_generator2 = train_datagen.flow_from_dataframe(\n dataframe=X_train,\n directory=TRAIN_IMG_PREP_PATH,\n x_col='img_file',\n y_col='class',\n target_size=(IMG_SIZE, IMG_SIZE),\n color_mode='rgb',\n class_mode='categorical',\n batch_size=BATCH_SIZE,\n shuffle=True,\n)\n\n# Train Generator (CutMix)\ntrain_generator = CutMixImageDataGenerator(\n generator1=train_generator1,\n generator2=train_generator2,\n img_size=IMG_SIZE,\n batch_size=BATCH_SIZE,\n)\n\n# Validation Generator\nvalid_datagen = ImageDataGenerator(rescale=1./255,)\nvalid_generator = valid_datagen.flow_from_dataframe(\n dataframe=X_val,\n directory=TRAIN_IMG_PREP_PATH,\n x_col='img_file',\n y_col='class',\n target_size=(IMG_SIZE, IMG_SIZE),\n color_mode='rgb',\n class_mode='categorical',\n batch_size=BATCH_SIZE,\n shuffle=False,\n)\n\n# Test Generator\ntest_datagen = ImageDataGenerator(rescale=1./255,)\ntest_generator = test_datagen.flow_from_dataframe(\n dataframe=X_test,\n directory=TEST_IMG_PREP_PATH,\n x_col='img_file',\n y_col=None,\n target_size=(IMG_SIZE, IMG_SIZE),\n color_mode='rgb',\n class_mode=None,\n batch_size=BATCH_SIZE,\n shuffle=False,\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Train & Save"},{"metadata":{"trusted":true},"cell_type":"code","source":"if not ENSEMBLE_MODE:\n # Get Model\n model = get_model(\n pretained_model=PRETAINED_MODEL,\n img_size=IMG_SIZE,\n initializer=INITIALIZER,\n regularizer=REGULARIZER,\n optimizer=OPTIMIZER,\n dropout_rate=DROPOUT_RATE,\n class_nb=CLASS_NB,\n )\n\n # Train With Generators\n history = model.fit_generator(\n generator=train_generator,\n steps_per_epoch=train_generator.get_steps_per_epoch(),\n epochs=EPOCHS,\n verbose=VERBOSE,\n callbacks=get_callback_list(\n model_file_path=MODEL_FILE_PATH,\n patience=PATIENCE,\n verbose=VERBOSE,\n system_print=True,\n ),\n validation_data=valid_generator,\n validation_steps=get_steps(valid_generator.samples, BATCH_SIZE),\n class_weight=class_weights_dict if GET_CLASS_WEIGHTS else None,\n shuffle=False,\n )\n \n # Visualization Loss & Accuracy\n history_info = history.history\n loss = history_info['loss']\n val_loss = history_info['val_loss']\n acc = history_info['acc']\n val_acc = history_info['val_acc']\n epochs = range(1, len(loss)+1)\n\n # Plot\n fig = plt.figure(figsize=(20, 4))\n\n fig.add_subplot(1, 2, 1)\n plt.plot(epochs, loss, 'b', label=f'Training Loss')\n plt.plot(epochs, val_loss,'r', label=f'Validation Loss')\n plt.title(f'Loss Per Epoch ({MODEL_N}.{PRETAINED_MODEL_STR})')\n plt.xlabel('Epochs')\n plt.ylabel('Loss')\n plt.legend()\n\n fig.add_subplot(1, 2, 2)\n plt.plot(epochs, acc, 'b', label='Training Accuracy')\n plt.plot(epochs, val_acc, 'r', label='Validation Accuracy')\n plt.title(f'Accuracy Per Epoch ({MODEL_N}.{PRETAINED_MODEL_STR})')\n plt.xlabel('Epochs')\n plt.ylabel('Accuracy')\n plt.legend()\n\n # Show Visualization Result\n plt.show()\n \n # Prediction\n model.load_weights(MODEL_FILE_PATH)\n pred = model.predict_generator(\n generator=test_generator,\n steps=get_steps(len(X_test), BATCH_SIZE),\n verbose=VERBOSE,\n )\n pred_class_indices = np.argmax(pred, axis=1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Ensemble Models\n\nKernel would die if 20 models are loaded and ensembled at once. \nTo solve this problem, you can use `np.save` and `np.load`. \nFor example, models (0 ~ 9) were ensembled and saved separately with `np.save`, and loaded with `np.load`. \nYou can see this example in Version 1 of this kernel."},{"metadata":{"trusted":true},"cell_type":"code","source":"if ENSEMBLE_MODE:\n UPLOADED_MODEL_LIST = sorted(os.listdir(UPLOADED_MODEL_PATH))[:10]\n\n model_pred_list = list()\n for model_n, model_name in enumerate(UPLOADED_MODEL_LIST):\n \n # Load Model\n model_file_path = os.path.join(UPLOADED_MODEL_PATH, model_name)\n model = get_model(\n pretained_model=MODEL_LIST[model_n],\n img_size=IMG_SIZE,\n initializer=INITIALIZER,\n regularizer=REGULARIZER,\n optimizer=OPTIMIZER,\n dropout_rate=DROPOUT_RATE,\n class_nb=CLASS_NB,\n )\n model.load_weights(model_file_path)\n \n # Prediction\n test_generator.reset()\n pred = model.predict_generator(\n generator=test_generator,\n steps=get_steps(len(X_test), BATCH_SIZE),\n verbose=VERBOSE,\n )\n model_pred_list.append(pred)\n\n # Model (0 ~ 9) Ensemble\n pred_mean = np.mean(model_pred_list, axis=0)\n pred_class_indices = np.argmax(pred_mean, axis=1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"get_class_indices = train_generator.class_indices\nlabels = {val:key for key, val in get_class_indices.items()}\npred_result = [labels[idx] for idx in pred_class_indices]\n\nsubmission = pd.read_csv(os.path.join(INPUT_PATH, 'sample_submission.csv'))\nsubmission['class'] = pred_result\nsubmission.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission.head()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.6"}},"nbformat":4,"nbformat_minor":1}