{"cells":[{"metadata":{},"cell_type":"markdown","source":"# 迁移学习案例研究\n# 四、数据增强：迁移学习，使用vgg16预训练模型作为特征提取器\n## 数据集样本数量受限的迁移学习（使用kaggle Dogs vs. Cats数据集）\n"},{"metadata":{},"cell_type":"markdown","source":"### 一、了解数据集\n\n　　该数据集为猫狗图像数据集，训练集由50000张照片（其中猫25000张，狗25000张），测试集25000张照片（其中其中猫12500张，狗12500张）。"},{"metadata":{"trusted":true},"cell_type":"code","source":"import glob\n\n# 了解数据集的组成\n\ntrain_files = glob.glob('/kaggle/input/dogs-vs-cats/train/train/*')\ntest_files = glob.glob('/kaggle/input/dogs-vs-cats/test1/test1/*')\n\ntrain_cat_files = [file_name for file_name in train_files if 'cat' in file_name]\ntrain_dog_files = [file_name for file_name in train_files if 'dog' in file_name]\n\nprint('train samples of cat:', len(train_cat_files))\nprint('train samples of dog:', len(train_dog_files))\nprint( train_dog_files[0])\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 二、构建小数据集\n\n　　现在我们构造一个小型数据集，即训练图像包含3000张图片，校验图像包含1000张图片，测试图像包含1000张图片（每类中猫狗图片数量相同）。"},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np\nfrom random import shuffle\nfrom keras.preprocessing.image import load_img, img_to_array\nfrom sklearn.preprocessing import LabelEncoder\n\n# 从猫训练数据中随机抽取1500张训练样本\ncat_train = list(np.random.choice(train_cat_files, size=1500, replace=False))\n\n# 从狗训练数据中随机抽取1500张训练样本\ndog_train = list(np.random.choice(train_dog_files, size=1500, replace=False))\n\n# 从猫训练数据中剔除已经抽取的训练样本\ntrain_cat_files = list(set(train_cat_files) - set(cat_train))\n\n# 从狗训练数据中剔除已经抽取的训练样本\ntrain_dog_files = list(set(train_dog_files) - set(dog_train))\n\n# 从猫训练数据中随机抽取500张校验样本\ncat_val = list(np.random.choice(train_cat_files, size=500, replace=False))\n\n# 从狗训练数据中随机抽取500张校验样本\ndog_val = list(np.random.choice(train_dog_files, size=500, replace=False))\n\n# 从猫训练数据中剔除已经抽取的校验样本\ntrain_cat_files = list(set(train_cat_files) - set(cat_val))\n\n# 从狗训练数据中剔除已经抽取的校验样本\ntrain_dog_files = list(set(train_dog_files) - set(dog_val))\n\n# 从猫训练数据中随机抽取500张测试样本\ncat_test = list(np.random.choice(train_cat_files, size=500, replace=False))\n\n# 从狗训练数据中随机抽取500张测试样本\ndog_test = list(np.random.choice(train_dog_files, size=500, replace=False))\n\n# 合并猫狗训练集\ntrain_files = cat_train + dog_train\n# 合并猫狗校验集\nval_files = cat_val + dog_val\n# 合并猫狗测试集\ntest_files = cat_test + dog_test\n\n# 随机化猫狗训练集\nshuffle(train_files)\n\n# 样本尺寸\nIMG_DIM = (150, 150)\n# 从磁盘加载训练集\nx_train = np.array([img_to_array(load_img(image_file, target_size=IMG_DIM)) for image_file in train_files])\n# 从磁盘加载校验集\nx_val = np.array([img_to_array(load_img(image_file, target_size=IMG_DIM)) for image_file in val_files])\n# 从磁盘加载测试集\nx_test = np.array([img_to_array(load_img(image_file, target_size=IMG_DIM)) for image_file in test_files])\n\n# 将训练集列表转换为numpy矩阵\nx_train = np.array(x_train)\n# 将校验集列表转换为numpy矩阵\nx_val = np.array(x_val)\n# 将测试集列表转换为numpy矩阵\nx_test = np.array(x_test)\n\n# 标签编码\ntrain_labels = [fn.split('/')[-1].split('.')[0].strip() for fn in train_files]\nval_labels = [fn.split('/')[-1].split('.')[0].strip() for fn in val_files]\ntest_labels = [fn.split('/')[-1].split('.')[0].strip() for fn in test_files]\nle = LabelEncoder()\nle.fit(train_labels)\ny_train = le.transform(train_labels)\ny_val = le.transform(val_labels)\ny_test = le.transform(test_labels)\n\n\nprint('x_train shape:', x_train.shape)\nprint('y_train shape:', y_train.shape)\nprint('x_validate shape:', x_val.shape)\nprint('y_validate shape:', y_val.shape)\nprint('x_test shape:', x_test.shape)\nprint('y_test shape:', y_test.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 三、数据增强\n\n　　定义数据增强器，直观测试增强的样本效果。"},{"metadata":{"trusted":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\nfrom keras.preprocessing.image import ImageDataGenerator\ntrain_datagen = ImageDataGenerator(rescale=1./255, zoom_range=0.3, rotation_range=50,\n                                   width_shift_range=0.2, height_shift_range=0.2, shear_range=0.2, \n                                   horizontal_flip=True, fill_mode='nearest')\n\nval_datagen = ImageDataGenerator(rescale=1./255)\n\nimg_id = 3\ncat_generator = train_datagen.flow(x_train[img_id:img_id+1], train_labels[img_id:img_id+1],\n                                   batch_size=1)\ncat = [next(cat_generator) for i in range(0,5)]\nfig, ax = plt.subplots(1,5, figsize=(16, 6))\nprint('Labels:', [item[1][0] for item in cat])\nl = [ax[i].imshow(cat[i][0][0]) for i in range(0,5)]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"img_id = 5\ncat_generator = train_datagen.flow(x_train[img_id:img_id+1], train_labels[img_id:img_id+1],\n                                   batch_size=1)\ncat = [next(cat_generator) for i in range(0,5)]\nfig, ax = plt.subplots(1,5, figsize=(16, 6))\nprint('Labels:', [item[1][0] for item in cat])\nl = [ax[i].imshow(cat[i][0][0]) for i in range(0,5)]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 四、使用vgg16网络提取特征\n\n　　vgg16网络各层参数固定。"},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras.applications import vgg16\nfrom keras.models import Model\nimport keras\ninput_shape = (150, 150, 3)\nvgg = vgg16.VGG16(include_top=False, weights='imagenet', \n                                     input_shape=input_shape)\n\noutput = vgg.layers[-1].output\noutput = keras.layers.Flatten()(output)\nvgg_model = Model(vgg.input, output)\n\nvgg_model.trainable = False\nfor layer in vgg_model.layers:\n    layer.trainable = False\n    \nimport pandas as pd\npd.set_option('max_colwidth', -1)\nlayers = [(layer, layer.name, layer.trainable) for layer in vgg_model.layers]\npd.DataFrame(layers, columns=['Layer Type', 'Layer Name', 'Layer Trainable']) \n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 五、输出vgg16网络扁平层信息\n\n　　用vgg16网络扁平层输出作为我们自己网络的输入。"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_imgs_scaled = x_train.astype('float32')\ntrain_imgs_scaled /= 255.0\nbottleneck_feature_example = vgg.predict(train_imgs_scaled[0:1])\nprint(bottleneck_feature_example.shape)\nplt.imshow(bottleneck_feature_example[0][:,:,0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_bottleneck_features(model, input_imgs):\n    features = model.predict(input_imgs, verbose=0)\n    return features\n\ntrain_imgs_scaled = x_train.astype('float32')\ntrain_imgs_scaled /= 255.0\nvalidation_imgs_scaled = x_val.astype('float32')\nvalidation_imgs_scaled /= 255.0\n\ntrain_features_vgg = get_bottleneck_features(vgg_model, train_imgs_scaled)\nvalidation_features_vgg = get_bottleneck_features(vgg_model, validation_imgs_scaled)\n\nprint('Train Bottleneck Features:', train_features_vgg.shape, \n      '\\tValidation Bottleneck Features:', validation_features_vgg.shape)\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 六、定义迁移学习网络模型"},{"metadata":{"trusted":true},"cell_type":"code","source":"from keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout, InputLayer\nfrom keras.models import Sequential\nfrom keras import optimizers\n\ninput_shape = vgg_model.output_shape[1]\n\nmodel = Sequential()\nmodel.add(InputLayer(input_shape=(input_shape,)))\nmodel.add(Dense(512, activation='relu', input_dim=input_shape))\nmodel.add(Dropout(0.3))\nmodel.add(Dense(512, activation='relu'))\nmodel.add(Dropout(0.3))\nmodel.add(Dense(1, activation='sigmoid'))\n\nmodel.compile(loss='binary_crossentropy',\n              optimizer=optimizers.RMSprop(lr=1e-4),\n              metrics=['accuracy'])\n\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 七、训练卷积神经网络\n　　指定训练数据、校验数据，每批样本数量、训练趟数、回显级别。 "},{"metadata":{"trusted":true},"cell_type":"code","source":"batch_size = 30\nepochs = 30\nhistory = model.fit(x=train_features_vgg, y=y_train,\n                    validation_data=(validation_features_vgg, y_val),\n                    batch_size=batch_size,\n                    epochs=epochs,\n                    verbose=1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 八、绘制模型训练的准确度和损失图\n　　通过观察训练和校验的准确度，网络性能有所提升，但仍然存在过拟合问题，我们可以绘制模型训练的准确度和损失图来直观发现拟合问题。"},{"metadata":{"trusted":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\n\nf, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 4))\nt = f.suptitle('Pre-trained CNN Transfer Learn Performance', fontsize=12)\nf.subplots_adjust(top=0.85, wspace=0.3)\n\nepoch_list = list(range(1,31))\nax1.plot(epoch_list, history.history['accuracy'], label='Train Accuracy')\nax1.plot(epoch_list, history.history['val_accuracy'], label='Validation Accuracy')\nax1.set_xticks(np.arange(0, 31, 5))\nax1.set_ylabel('Accuracy Value')\nax1.set_xlabel('Epoch #')\nax1.set_title('Accuracy')\nl1 = ax1.legend(loc=\"best\")\n\nax2.plot(epoch_list, history.history['loss'], label='Train Loss')\nax2.plot(epoch_list, history.history['val_loss'], label='Validation Loss')\nax2.set_xticks(np.arange(0, 31, 5))\nax2.set_ylabel('Loss Value')\nax2.set_xlabel('Epoch #')\nax2.set_title('Loss')\nl2 = ax2.legend(loc=\"best\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 　你可以清楚地观察到2–3趟训练后模型开始在训练数据上出现拟合。该模型在校验集的分类准确率大约为90%，虽然不完美，但有所进步！我们还能够改进该模型吗？"},{"metadata":{},"cell_type":"markdown","source":"### 九、保存模型\n　　保存模型以便后面我们用测试集对该模型的性能进行评估。"},{"metadata":{"trusted":true},"cell_type":"code","source":"model.save('cats_dogs_basic_cnn_re_tl.h5')","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}