{"cells":[{"metadata":{"_uuid":"125bef9f26b4610096464318d5fa5976fdf9ede8"},"cell_type":"markdown","source":"# Trainable Image Resize\n\nIt was mentioned in the forums https://www.kaggle.com/c/data-science-bowl-2018/discussion/52766 that the fact that images are at different scales can be a significant problem. \nIt was also pointed that people dealt with it before in the following manner:\n\n* train network on raw images\n\n* run image through the net\n\n* estimate object size\n\n* resize based on the object size estimate\n\n* train new network on resize images\n\nWe decided to implement it in the following way:\n\n* train unet for mask and contour prediction\n\n* do some morphological postprocessing mask+contour\n\n* estimage size by:\n\n```python\nclass CellSizer(BaseTransformer):\n    def __init__(self, **kwargs):\n        pass\n\n    def transform(self, labeled_images):\n        mean_sizes = []\n        for image in tqdm(labeled_images):\n            mean_size = mean_cell_size(image)\n            mean_sizes.append(mean_size)\n        return {'sizes': mean_sizes}\n            \ndef mean_cell_size(labeled_image):\n    blob_sizes = itemfreq(labeled_image)\n    if blob_sizes.shape[0]==1:\n        return 0\n    else:\n        blob_sizes = blob_sizes[blob_sizes[:, 0].argsort()][1:, 1]\n        return np.mean(blob_sizes)\n            \n```\n\n* rescaling the image (assuming certain boundaries) with\n\n```python \n\nclass ImageReaderRescaler(BaseTransformer):\n    def __init__(self, min_size, max_size, target_ratio):\n        self.min_size = min_size\n        self.max_size = max_size\n        self.target_ratio = target_ratio\n\n    def _transform(self, sizes, X, y=None, meta=None):\n        raw_images = X[0]\n        raw_images_adj = []\n        for size, raw_image in tqdm(zip(sizes, raw_images)):\n            h_adj, w_adj = self._get_adjusted_image_size(size, from_pil(raw_image))\n            raw_image_adj = resize(from_pil(raw_image), (h_adj, w_adj), \n                                   preserve_range=True).astype(np.uint8)\n            raw_images_adj.append(to_pil(raw_image_adj))\n        X_adj = [raw_images_adj]\n        ...\n        return X_adj, y_adj\n\n    def _get_adjusted_image_size(self, mean_cell_size, img):\n        h, w = img.shape[:2]\n        img_area = h * w\n        \n        if mean_cell_size ==0:\n            adj_ratio = 1.0\n        else:\n            size_ratio = img_area / mean_cell_size\n            adj_ratio = size_ratio / self.target_ratio\n\n        h_adj = int(clip(self.min_size, h * adj_ratio, self.max_size))\n        w_adj = int(clip(self.min_size, w * adj_ratio, self.max_size))\n\n        return h_adj, w_adj\n```\n\n* Finally on such rescaled images we train and predict by using patches of fixed size (say 512x512). For example inference can be done with something like this:\n\n```python\nclass PatchCombiner(BaseTransformer):\n```\n    ...\n```python\n    def _join_output(self, patch_meta, image_patches):\n        image_h = patch_meta['image_h'].unique()[0]\n        image_w = patch_meta['image_w'].unique()[0]\n        prediction_image = np.zeros((image_h, image_w))\n        prediction_image_padded = get_mosaic_padded_image(prediction_image, \n                                                          self.patching_size, \n                                                          self.patching_stride)\n\n        patches_per_image = 0\n        for (y_coordinate, \n             x_coordinate, \n             tta_angle), image_patch in zip(patch_meta[['y_coordinates', \n                                                        'x_coordinates', \n                                                        'tta_angles']].values.tolist(), \n                                            image_patches):\n            patches_per_image += 1\n            image_patch = np.rot90(image_patch, -1 * tta_angle / 90.)\n            (window_y, \n             window_x) = y_coordinate * self.patching_stride, x_coordinate * self.patching_stride\n            prediction_image_padded[window_y:self.patching_size + window_y,\n            window_x:self.patching_size + window_x] += image_patch\n\n        _, h_top, h_bottom, _ = get_padded_size(max(image_h, self.patching_size),\n                                                self.patching_size,\n                                                self.patching_stride)\n        _, w_left, w_right, _ = get_padded_size(max(image_w, self.patching_size),\n                                                self.patching_size,\n                                                self.patching_stride)\n\n        prediction_image = prediction_image_padded[h_top:-h_bottom, w_left:-w_right]\n        prediction_image /= self.normalization_factor\n        return prediction_image\n```\n"},{"metadata":{"_uuid":"16e4e1951b47d45c7d8dd160c5fbe3f4199eaf63"},"cell_type":"markdown","source":"# Full pipeline\nIf you would like to see how we plugged trainable rescale into our pipeline go to [open solution](https://github.com/neptune-ml/open-solution-data-science-bowl-2018)\n\n![full open solution pipeline](https://gist.githubusercontent.com/jakubczakon/10e5eb3d5024cc30cdb056d5acd3d92f/raw/e85c1da3acfe96123d0ff16f8145913ee65e938c/full_pipeline.png)\n\nThe `ImageReaderRescaler` step is defined in the `preprocessing.py` file:\n\nIf you want to use our implementation just go for it!"},{"metadata":{"trusted":false,"collapsed":true,"_uuid":"09f8b3ccdcb73402480d57d1d6aad45eaa8ae2bc"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.5.2"}},"nbformat":4,"nbformat_minor":1}