{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 细胞图像分割Report - 邓忠寒\n**目标**:设计细胞图像分割模型，并在测试数据上生成细胞分割结果
\n\n**存在问题**:给定的training data过少。在网上搜索大量资料，企图寻找相关的细胞图像数据，最终在Kaggle上找到[2018 Data Science Bowl](https://www.kaggle.com/c/data-science-bowl-2018/data?select=stage1_train_labels.csv.zip)，是我找到唯一一个十分接近目标图像的数据库，但是也仅有600多个training 数据。","metadata":{}},{"cell_type":"markdown","source":"## 1. 什么是图像分割(Image Segmentation)？\n一个图片是由pixels组成，image segmentation是将一个图片分成多个segments的过程。分割的目标是去简单化或者说是改变一个图片的representaion，让它的representation更有意义，也更容易被分析。Image segmentation被用于定位objects。更准确点说，image segementation是把每一个pixel都assign一个label的过程，具有相同label的pixel共享这某一些特性。
\n来源：https://en.wikipedia.org/wiki/Image_segmentation\n","metadata":{}},{"cell_type":"markdown","source":"## 2. 图像分割的种类\n\n
\n图片来源：https://cdn.analyticsvidhya.com/wp-content/uploads/2019/03/Screenshot-from-2019-03-28-12-08-09.png","metadata":{}},{"cell_type":"markdown","source":"实际上我们可以大致的将图片分割化为两大类。
\n\n- semantic segmentation: 如图所示，每一个pixel属于一个特定的class(要么是背景要么是人)。并且，所有来自同一个class的被标示为同一个颜色(背景为黑色，人为粉色)。\n- inseance segmentation: 如图所示，在同一个class里面的不同的object(人)也被标示成了不同的颜色。\n\n综上所示，semantic segmentation目标是classify所有的人作为一个instance，然而instance segmentation目标是把每一个人单独分开。在当前的report中，我们的目标是semantic segmentation。\n\n来源：https://www.analyticsvidhya.com/blog/2019/04/introduction-image-segmentation-techniques-python/","metadata":{}},{"cell_type":"markdown","source":"## 3. 图像分割与物体检测(Object Detection)的区别\n
\n图片来源：cs231n.stanford.edu","metadata":{}},{"cell_type":"markdown","source":"在一个图片中，object detection是在对每一个class建立一个边界框。它告诉了我们物体所在的大概位置(也就是边框的坐标)，但是我们无法得知目标的具体形状。
\n在一个图片中，image segmentation创造一个在pixel-wise的mask给每一个物体，这个技术给予我们更加细化的理解。在医学界，如果我们能很早的检测出病人的癌症，将大大提高其存活率，然而癌症细胞的一个重要的特点，就是它的形状会给予癌症严重性的重要信息。","metadata":{}},{"cell_type":"markdown","source":"## 4. 数据的预处理","metadata":{}},{"cell_type":"code","source":"import tensorflow as tf\nimport os\nimport random\nimport numpy as np\nfrom tqdm import tqdm\nfrom skimage.io import imread,imshow\nfrom skimage.transform import resize\nimport matplotlib.pyplot as plt\nfrom zipfile import ZipFile \nfrom keras import backend as K\nfrom keras.callbacks import EarlyStopping, ModelCheckpoint\nimport keras","metadata":{"execution":{"iopub.status.busy":"2021-08-15T00:51:47.020997Z","iopub.execute_input":"2021-08-15T00:51:47.021354Z","iopub.status.idle":"2021-08-15T00:51:52.662709Z","shell.execute_reply.started":"2021-08-15T00:51:47.021309Z","shell.execute_reply":"2021-08-15T00:51:52.661939Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# set up hyperparameter\nimg_height = 256\nimg_width = 256\nimg_channel = 3","metadata":{"execution":{"iopub.status.busy":"2021-08-15T00:51:52.664620Z","iopub.execute_input":"2021-08-15T00:51:52.665002Z","iopub.status.idle":"2021-08-15T00:51:52.671215Z","shell.execute_reply.started":"2021-08-15T00:51:52.664959Z","shell.execute_reply":"2021-08-15T00:51:52.670486Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def unzip(keyword = None):\n \"\"\"\n input: train / test\n objective: unzip file\n \"\"\"\n file_name = f\"../input/data-science-bowl-2018/stage1_{keyword}.zip\"\n with ZipFile(file_name, 'r') as zip: \n print(f'{keyword} file unzipping')\n zip.extractall(f\"stage1_{keyword}\") \n print('Done!')","metadata":{"execution":{"iopub.status.busy":"2021-08-15T00:51:52.672676Z","iopub.execute_input":"2021-08-15T00:51:52.673276Z","iopub.status.idle":"2021-08-15T00:51:52.684336Z","shell.execute_reply.started":"2021-08-15T00:51:52.673236Z","shell.execute_reply":"2021-08-15T00:51:52.683578Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"unzip('train') # unzip train\nunzip('test') # unzip test","metadata":{"execution":{"iopub.status.busy":"2021-08-15T00:51:52.685934Z","iopub.execute_input":"2021-08-15T00:51:52.686322Z","iopub.status.idle":"2021-08-15T00:51:58.800357Z","shell.execute_reply.started":"2021-08-15T00:51:52.686276Z","shell.execute_reply":"2021-08-15T00:51:58.799424Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# set up train and test path\ntrain_path = \"stage1_train/\"\ntest_path = \"stage1_test/\"","metadata":{"execution":{"iopub.status.busy":"2021-08-15T00:51:58.803405Z","iopub.execute_input":"2021-08-15T00:51:58.803703Z","iopub.status.idle":"2021-08-15T00:51:58.808014Z","shell.execute_reply.started":"2021-08-15T00:51:58.803670Z","shell.execute_reply":"2021-08-15T00:51:58.806669Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# save all the training ids\ntrain_ids = os.listdir(train_path)\nprint(f'There are {len(train_ids)} training samples.')","metadata":{"execution":{"iopub.status.busy":"2021-08-15T00:51:58.810242Z","iopub.execute_input":"2021-08-15T00:51:58.810654Z","iopub.status.idle":"2021-08-15T00:51:58.823053Z","shell.execute_reply.started":"2021-08-15T00:51:58.810616Z","shell.execute_reply":"2021-08-15T00:51:58.822186Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def preprocess(id_, path):\n \"\"\"\n input: list of ids, path of directory\n output: return X_train and y_train\n \"\"\"\n \n # initialize two empty array to store\n # size is (# of training instance, img_size, img_size, img_channel)\n X_train = np.zeros((len(id_), img_height, img_width, img_channel), dtype = np.uint8)\n Y_train = np.zeros((len(id_), img_height, img_width, 1), dtype = np.bool)\n \n # iterate through all the training img, save each training instance into X_train\n # using tqdm is good for us to visualize the process\n for n, id_ in tqdm(enumerate(id_), total = len(id_)): \n cur_path = path + id_\n # read in img as array\n img = imread(cur_path + '/images/' + id_ + '.png')[:,:,:img_channel] \n # resize data to increase the speed of training\n img = resize(img, (img_height, img_width), mode='constant', preserve_range=True)\n # save current img into X_train\n X_train[n] = img \n # for each img, we have several masks\n # we need to iterate through each one \n mask = np.zeros((img_height, img_width, 1), dtype = np.bool)\n for mask_file in os.listdir(cur_path + '/masks/'):\n # read in current mask\n cur_mask = imread(cur_path + '/masks/' + mask_file)\n # resize it and adjust the dimension to 128x128x1\n cur_mask = np.expand_dims(resize(cur_mask, (img_height, img_width), mode = 'constant', preserve_range = True), axis = -1)\n mask = np.maximum(mask, cur_mask)\n Y_train[n] = mask\n return X_train, Y_train","metadata":{"execution":{"iopub.status.busy":"2021-08-15T00:51:58.825049Z","iopub.execute_input":"2021-08-15T00:51:58.825444Z","iopub.status.idle":"2021-08-15T00:51:58.839858Z","shell.execute_reply.started":"2021-08-15T00:51:58.825408Z","shell.execute_reply":"2021-08-15T00:51:58.838818Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train, y_train = preprocess(train_ids, train_path)","metadata":{"execution":{"iopub.status.busy":"2021-08-15T00:51:58.841551Z","iopub.execute_input":"2021-08-15T00:51:58.841803Z","iopub.status.idle":"2021-08-15T00:55:22.394795Z","shell.execute_reply.started":"2021-08-15T00:51:58.841780Z","shell.execute_reply":"2021-08-15T00:55:22.392976Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"模型启发: 因为我们在做的事细胞分割，所以在经过一定的research之后，发现U-Net模型框架比较适合。
\n模型介绍: U-Net是一款为生物医学图像分割创造出的模型，最开始被Freiburg大学的电脑科学院所开发。
\n模型框架:
\n\n\n\n我们可以看到这个框架一共有9个部分，我们可以大致给他分成encoder和decoder两部分。
\n\n- 前面的5个部分通常被视为encoder部分均由两个3x3的conv layer接上一个2x2的max pooling。
\n- 后面的4个部分通常被视为decoder阶段，由一个conv Transposelayer + concat + 两个3x3conv layer。
\n- 在decoder阶段，U-Net框架会拼接之前decoder阶段的部分，对其进行卷积。
\n\nU-Net优点是在较少的training samples仍然能有比较理想的结果，这打破了传统深度学习框架通常需要数以千计或者万计training samples的弊端。这也正符合我们的需要因为我们仅有670training samples。
\n\nLoss Function and Metric
\n- 在做binary classification的时候，binary cross-entropy是一个非常常用的loss function，它度量错误classification的概率。
\n- 另外一个在image segmentation中比较常用的叫做dice coefficient，它的就算是2乘以两个图片overlap的面积再除以所有的面积。这个metric的区间在0到1之间。1代表完美的overlap。
\n\n我们会采用binary cross-entropy作为loss function，采用accuracy和dice coefficient作为metric。","metadata":{}},{"cell_type":"markdown","source":"## 5. 模型建造","metadata":{}},{"cell_type":"code","source":"smooth = 1.\n# define dice coefficient metric\ndef dice_coef(y_true, y_pred):\n # flatten\n y_true_f = K.flatten(y_true)\n y_pred_f = K.flatten(y_pred)\n # find intersection\n intersection = K.sum(y_true_f * y_pred_f)\n # 2 * intersection / overall\n return (2. * intersection + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth)","metadata":{"execution":{"iopub.status.busy":"2021-08-15T01:08:27.169799Z","iopub.execute_input":"2021-08-15T01:08:27.170169Z","iopub.status.idle":"2021-08-15T01:08:27.178257Z","shell.execute_reply.started":"2021-08-15T01:08:27.170138Z","shell.execute_reply":"2021-08-15T01:08:27.177385Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"inputs = tf.keras.layers.Input((img_height, img_width, img_channel))\n# Normalize the data\ns = tf.keras.layers.Lambda(lambda x: x / 255)(inputs)\n\n# Encode\nc1 = tf.keras.layers.Conv2D(16, (3, 3), activation = 'relu', padding = 'same')(s)\nc1 = tf.keras.layers.Conv2D(16, (3, 3), activation = 'relu', padding = 'same')(c1)\np1 = tf.keras.layers.MaxPooling2D((2, 2))(c1)\n\nc2 = tf.keras.layers.Conv2D(32, (3, 3), activation = 'relu', padding = 'same')(p1)\nc2 = tf.keras.layers.Conv2D(32, (3, 3), activation = 'relu', padding = 'same')(c2)\np2 = tf.keras.layers.MaxPooling2D((2, 2))(c2)\n \nc3 = tf.keras.layers.Conv2D(64, (3, 3), activation = 'relu', padding = 'same')(p2)\nc3 = tf.keras.layers.Conv2D(64, (3, 3), activation = 'relu', padding = 'same')(c3)\np3 = tf.keras.layers.MaxPooling2D((2, 2))(c3)\n \nc4 = tf.keras.layers.Conv2D(128, (3, 3), activation = 'relu', padding = 'same')(p3)\nc4 = tf.keras.layers.Conv2D(128, (3, 3), activation = 'relu', padding = 'same')(c4)\np4 = tf.keras.layers.MaxPooling2D((2, 2))(c4)\n \nc5 = tf.keras.layers.Conv2D(256, (3, 3), activation = 'relu', padding = 'same')(p4)\nc5 = tf.keras.layers.Conv2D(256, (3, 3), activation = 'relu', padding = 'same')(c5)\n\n# Decode \nu6 = tf.keras.layers.Conv2DTranspose(128, (2, 2), strides = (2, 2), padding = 'same')(c5)\nu6 = tf.keras.layers.concatenate([u6, c4])\nc6 = tf.keras.layers.Conv2D(128, (3, 3), activation = 'relu', padding = 'same')(u6)\nc6 = tf.keras.layers.Conv2D(128, (3, 3), activation = 'relu', padding = 'same')(c6)\n \nu7 = tf.keras.layers.Conv2DTranspose(64, (2, 2), strides = (2, 2), padding = 'same')(c6)\nu7 = tf.keras.layers.concatenate([u7, c3])\nc7 = tf.keras.layers.Conv2D(64, (3, 3), activation = 'relu', padding = 'same')(u7)\nc7 = tf.keras.layers.Conv2D(64, (3, 3), activation = 'relu', padding = 'same')(c7)\n \nu8 = tf.keras.layers.Conv2DTranspose(32, (2, 2), strides=(2, 2), padding = 'same')(c7)\nu8 = tf.keras.layers.concatenate([u8, c2])\nc8 = tf.keras.layers.Conv2D(32, (3, 3), activation = 'relu', padding = 'same')(u8)\nc8 = tf.keras.layers.Conv2D(32, (3, 3), activation = 'relu', padding = 'same')(c8)\n \nu9 = tf.keras.layers.Conv2DTranspose(16, (2, 2), strides = (2, 2), padding = 'same')(c8)\nu9 = tf.keras.layers.concatenate([u9, c1], axis=3)\nc9 = tf.keras.layers.Conv2D(16, (3, 3), activation = 'relu', padding = 'same')(u9)\nc9 = tf.keras.layers.Conv2D(16, (3, 3), activation = 'relu', padding = 'same')(c9)\n\n# outpput\noutputs = tf.keras.layers.Conv2D(1, (1, 1), activation='sigmoid')(c9)\nmodel = tf.keras.Model(inputs=[inputs], outputs=[outputs])\n\n# compile\nmodel.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy', dice_coef]) ","metadata":{"execution":{"iopub.status.busy":"2021-08-15T01:08:27.447541Z","iopub.execute_input":"2021-08-15T01:08:27.447867Z","iopub.status.idle":"2021-08-15T01:08:27.695884Z","shell.execute_reply.started":"2021-08-15T01:08:27.447820Z","shell.execute_reply":"2021-08-15T01:08:27.695138Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"callbacks = [EarlyStopping(patience = 5, monitor = 'val_loss'), ModelCheckpoint('U-Net-Best.h5', verbose=1, save_best_only = True)]","metadata":{"execution":{"iopub.status.busy":"2021-08-15T01:08:27.880146Z","iopub.execute_input":"2021-08-15T01:08:27.880435Z","iopub.status.idle":"2021-08-15T01:08:27.885895Z","shell.execute_reply.started":"2021-08-15T01:08:27.880407Z","shell.execute_reply":"2021-08-15T01:08:27.884994Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 6. 模型训练","metadata":{}},{"cell_type":"code","source":"results = model.fit(X_train, y_train, validation_split = 0.1, batch_size = 16, epochs = 30, callbacks = callbacks) ","metadata":{"execution":{"iopub.status.busy":"2021-08-15T01:08:31.984443Z","iopub.execute_input":"2021-08-15T01:08:31.984762Z","iopub.status.idle":"2021-08-15T01:09:57.311207Z","shell.execute_reply.started":"2021-08-15T01:08:31.984731Z","shell.execute_reply":"2021-08-15T01:09:57.310317Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def generate_plot(history, keyword = ''):\n plt.plot(history.history[keyword])\n plt.plot(history.history[f'val_{keyword}'])\n plt.title(f'model {keyword}')\n plt.ylabel(f'{keyword}')\n plt.xlabel('epoch')\n plt.legend(['train', 'val'], loc = 'upper left')\n plt.show()","metadata":{"execution":{"iopub.status.busy":"2021-08-15T01:09:59.321782Z","iopub.execute_input":"2021-08-15T01:09:59.322138Z","iopub.status.idle":"2021-08-15T01:09:59.329243Z","shell.execute_reply.started":"2021-08-15T01:09:59.322105Z","shell.execute_reply":"2021-08-15T01:09:59.328137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"generate_plot(results, 'accuracy')","metadata":{"execution":{"iopub.status.busy":"2021-08-15T01:09:59.627419Z","iopub.execute_input":"2021-08-15T01:09:59.627713Z","iopub.status.idle":"2021-08-15T01:09:59.775132Z","shell.execute_reply.started":"2021-08-15T01:09:59.627685Z","shell.execute_reply":"2021-08-15T01:09:59.773647Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"generate_plot(results, 'loss')","metadata":{"execution":{"iopub.status.busy":"2021-08-15T01:09:59.997126Z","iopub.execute_input":"2021-08-15T01:09:59.997393Z","iopub.status.idle":"2021-08-15T01:10:00.141000Z","shell.execute_reply.started":"2021-08-15T01:09:59.997367Z","shell.execute_reply":"2021-08-15T01:10:00.139897Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"generate_plot(results, 'dice_coef')","metadata":{"execution":{"iopub.status.busy":"2021-08-15T01:10:01.922376Z","iopub.execute_input":"2021-08-15T01:10:01.922686Z","iopub.status.idle":"2021-08-15T01:10:02.073212Z","shell.execute_reply.started":"2021-08-15T01:10:01.922656Z","shell.execute_reply":"2021-08-15T01:10:02.072217Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 7. 模型结果分析\n我们可以看到经过几个epoch我们就达到了一个十分可观的回报，我们最好的model的val_loss在0.5左右，val accuracy在0.97左右，val dice coefficient在0.88左右。这个结果还是比较理想的，因为我们的test sample的取样不同于train sample，所以我们也不想把模型过度训练，这样不利于模型generalize。所以我们在callback中设定了earlystopping以防止overfitting。我们将patience设定为5，已经给了model足够的时间。从accuracy和loss还有codice coefficient的图可以看我们的model在validation set的表现是略好于training set。这是我们通常想要的，这可以在一定程度上体现模型的generalize的能力。当然valiadation set还不能完全当作test set，毕竟我们是通过给model看到validation set进行调试的，而test set通常是model完全看不到的。综上所述我们得到了我认为还算不错的结果。接下来我们运用训练好的模型对training data进行呈现。","metadata":{}},{"cell_type":"code","source":"# load best model\nmodel = keras.models.load_model('U-Net-Best.h5', custom_objects = {'dice_coef': dice_coef})\n# make prediction on X_train\npreds_train = model.predict(X_train, verbose = 1)\n# set threshold as 0.5\npreds_train_t = (preds_train > 0.5).astype(np.uint8)","metadata":{"execution":{"iopub.status.busy":"2021-08-15T01:10:05.909025Z","iopub.execute_input":"2021-08-15T01:10:05.909392Z","iopub.status.idle":"2021-08-15T01:10:07.794882Z","shell.execute_reply.started":"2021-08-15T01:10:05.909359Z","shell.execute_reply":"2021-08-15T01:10:07.793894Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 8. 模型在training data预测","metadata":{}},{"cell_type":"code","source":"def show_pred():\n \"\"\"\n Random generate id, show X_train, y_train and prediction\n \"\"\"\n index = random.randint(0, len(preds_train_t))\n imshow(X_train[index])\n plt.title('X_train')\n plt.show()\n imshow(np.squeeze(y_train[index]))\n plt.title('y_train')\n plt.show()\n imshow(np.squeeze(preds_train_t[index]))\n plt.title('Prediction')\n plt.show()","metadata":{"execution":{"iopub.status.busy":"2021-08-15T01:10:08.085184Z","iopub.execute_input":"2021-08-15T01:10:08.085509Z","iopub.status.idle":"2021-08-15T01:10:08.093421Z","shell.execute_reply.started":"2021-08-15T01:10:08.085479Z","shell.execute_reply":"2021-08-15T01:10:08.092244Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"show_pred()","metadata":{"execution":{"iopub.status.busy":"2021-08-15T01:10:08.446205Z","iopub.execute_input":"2021-08-15T01:10:08.446489Z","iopub.status.idle":"2021-08-15T01:10:08.940101Z","shell.execute_reply.started":"2021-08-15T01:10:08.446462Z","shell.execute_reply":"2021-08-15T01:10:08.939070Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 9. 模型在test img上预测","metadata":{}},{"cell_type":"code","source":"# import our test img and reshape the array size\ntest = imread('../input/test111/newtest.png')\ntest_reshape = test.reshape(1, 256, 256, 3)\nimshow(np.squeeze(test_reshape))\nplt.title('test img')\nplt.axis('off')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-08-15T01:10:09.133410Z","iopub.execute_input":"2021-08-15T01:10:09.133707Z","iopub.status.idle":"2021-08-15T01:10:09.209381Z","shell.execute_reply.started":"2021-08-15T01:10:09.133679Z","shell.execute_reply":"2021-08-15T01:10:09.208542Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# make prediction\npred = model.predict(test_reshape)\npred_t = (pred > 0.5).astype(np.uint8)","metadata":{"execution":{"iopub.status.busy":"2021-08-15T01:01:26.335977Z","iopub.execute_input":"2021-08-15T01:01:26.336380Z","iopub.status.idle":"2021-08-15T01:01:26.378524Z","shell.execute_reply.started":"2021-08-15T01:01:26.336322Z","shell.execute_reply":"2021-08-15T01:01:26.377743Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"imshow(np.squeeze(pred_t[0]))\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2021-08-15T01:01:26.380168Z","iopub.execute_input":"2021-08-15T01:01:26.380548Z","iopub.status.idle":"2021-08-15T01:01:26.599763Z","shell.execute_reply.started":"2021-08-15T01:01:26.380512Z","shell.execute_reply":"2021-08-15T01:01:26.598722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## 9. 模型的未来以及改进\n即使U-Net已经表现的很好，但是它仍然有一些缺陷。它依赖于多层级联卷积神经网络，这些级联的网络提取感兴趣的部分进行密集预测，这导致了过度和冗余使用计算资源，因为它重复提取低级特征。
\nfuture work的一个方向是集中在对于attention-Unet的研究。attention在图像分割中是一个在training中highlight only relevent的方式。它大大的减少了计算资源的浪费，给予network更强大的power。[来源](https://towardsdatascience.com/a-detailed-explanation-of-the-attention-u-net-b371a5590831#:~:text=Attention%2C%20in%20the%20context%20of,network%20with%20better%20generalisation%20power.)
\n另一个方向是对于Unet++的研究。Unet++志在通过在encoder和decoder之间加入dense block和conv layers提高分割准确率。[论文链接](https://arxiv.org/abs/1807.10165)","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}