{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Mantisshrimp begginer's tutorial","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"In this tutorial we'll go through to how to finetune a model in the [wheat](https://www.kaggle.com/c/global-wheat-detection/overview/eccv-2020) dataset, where we are tasked to find the bounding boxes locations of wheat heads in the image.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"## Install packages\nBe sure to turn on internet and gpu accelerator in the `Settings` tab.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install -U 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'\n!pip install mantisshrimp","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Step 0: Imports","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Mantisshrimp is designed with developer productivity as it first goal, and that already starts at the imports level. The recommended way of importing the library is `from mantisshrimp.all import *`, this will import a curated list of packages (like numpy and torch) and all the necessary modules","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"#hide\nfrom mantisshrimp.all import *\nimport pandas as pd","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Step 1: Data parsers","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Often the step of getting our data into a standard format is the most difficult one. Almost all datasets comes in their unique own format, and writing custom code to handle each one of them can be very time consuming\n\nMantisshrimp provides an easy `Parser` interface for handling that, the main idea is that you only have to define how to parse a single sample and the library takes care of the rest","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"But before all, let's get the path to our dataset and read the CSV file using `pandas`","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"source = Path('../input/global-wheat-detection/')\ndf = pd.read_csv(source/'train.csv')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"There are two types of parsers we have to define:\n* `InfoParser`: Parses metadata information about our images, like the image `id`, `file_path`, `height` and `width`\n* `AnnotationParser`: Parses our annotations, like bounding boxes, masks and keypoints","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class WheatInfoParser(InfoParser):\n    def iid(self, o): return o.image_id\n    def height(self, o): return o.height\n    def width(self, o): return o.width\n    def file_path(self, o): return self.source/f'train/{o.image_id}.jpg'\n    def __iter__(self): yield from self.data.itertuples()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class WheatAnnotationParser(AnnotationParser):\n    def iid(self, o): return o.image_id\n    def oid(self, o): return 0\n    def bbox(self, o): return BBox.from_xywh(*np.fromstring(o.bbox[1:-1], sep=','))\n    def __iter__(self): yield from df.itertuples()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Define a `CategoryMap`, each `Category` receives an object id and it's name","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"catmap = CategoryMap([Category(0, 'wheat')])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"parser = DataParser(df, source, catmap=catmap,\n                    img_parser=WheatInfoParser,\n                    annot_parser=WheatAnnotationParser)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The `.parse` method will run the parser over all data points, grouping all images that share the same `id`  \nBy default it returns two lists of `Records` with a 0.8/0.2 `train/validation` split","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_rs,valid_rs = parser.parse()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Step 2: Transforms, Datasets, and DataLoaders","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Mantisshrimp does not implement transforms on it's own, but it instead relies on an easy interface for integrating any transforms library. It currently supports [albumentations](https://github.com/albumentations-team/albumentations) out of the box","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"tfm = AlbuTfm([A.Flip(p=.8), A.ShiftScaleRotate(p=.8, scale_limit=(0,.5))])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since we only defined augmentation transforms, we only want to apply them to the training dataset","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_ds = Dataset(train_rs, tfm)\nvalid_ds = Dataset(valid_rs)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We use `RCNNDataLoader` for creating our data loaders, it receives the same arguments as a standard pytorch `DataLoader`","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_dl = RCNNDataLoader(train_ds, batch_size=12, num_workers=8)\nvalid_dl = RCNNDataLoader(valid_ds, batch_size=12, num_workers=8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Bonus:** Let's take a look at how our transforms are affecting the same training sample:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"items = [train_ds[0] for _ in range(3)]\ngrid2([partial(show_item, o, label=False) for o in items])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Step 3 (optional): Metrics","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Let's use the same evaluation metric that COCO uses","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"metrics = [COCOMetric(valid_rs, catmap)]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Step 3: Model and Training","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Mantisshrimp provides a high and a mid level interface for training. One is not better than the other, they instead serve different purposes:\n* High level interface: For quick prototyping in a jupyter notebook like environment\n* Mid level interface: For more reproducible experiments. Great for writing experiments that can be launched in terminal with different hyperparameters","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"#### High level interface (Learner)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"The `Learner` interface is inspired (and very similar) to the [fastai](https://github.com/fastai/fastai2) `Learner`. It aims to provide a very productive experience for prototyping in a jupyter notebook like environment  \nIt comes bundled with additional functionallity, like inbuilt learning rate schedulers and differential learning rates for training unfreezed models","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Since our problem only contain bounding boxes, we're going to use the `FasterRCNN` model","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"model = MantisFasterRCNN(len(catmap), metrics=metrics)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The `Learner` receives an argument called `opt_fn`, it will call this function passing the model parameters and it expects to receive back a torch `Optimizer`. We are going to use `partial` to pass any additional paramters to our `SGD` optimizer","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"opt_fn = partial(SGD, momentum=.9, weight_decay=5e-4)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = Learner(model, train_dl, valid_dl, opt_fn)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`fit_one_cycle` will train adjusting the learning rate acording with the [1cycle learning rate policy](https://pytorch.org/docs/stable/optim.html#torch.optim.lr_scheduler.OneCycleLR)","execution_count":null},{"metadata":{"scrolled":false,"trusted":true},"cell_type":"code","source":"learn.fit_one_cycle(5, 1e-3)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Mid level interface (Lightning Trainer)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"This is almost pure Lightning, go crazy!  \nFor simplicity, let's just define a model that uses `SGD` and the 1cycle policy like before","execution_count":null},{"metadata":{"trusted":false},"cell_type":"code","source":"class WheatModel(MantisFasterRCNN):\n    def configure_optimizers(self):\n        opt = SGD(params(self), lr=1e-3, momentum=.9, weight_decay=5e-4)\n        sched = OneCycleLR(opt, max_lr=1e-3, total_steps=len(train_dl), pct_start=.3)\n        return [opt], [{'scheduler':sched, 'interval':'step'}]","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"model = WheatModel(len(catmap), metrics=metrics)","execution_count":null,"outputs":[]},{"metadata":{"trusted":false},"cell_type":"code","source":"trainer = Trainer(max_epochs=1, gpus=1)\ntrainer.fit(model, train_dl, valid_dl)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Extra: Visualize results","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Because we are using Lightning logs are automatically saved for us in a tensorboard format. Other cool loggers like [wandb](https://www.wandb.com/) are also provided by lightning!","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Let's take a look at some model predictions (`learn.show_results` if using `Learner`)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"rs = random.choices(valid_rs, k=2)\nims,preds = model.predict(rs=rs)\nshow_preds(ims, preds)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}