{"cells":[{"metadata":{},"cell_type":"markdown","source":"# `fastai2` Starter Kernel\n\nThis kernel will walk you through how to set up the `DataBlock` for this competition!","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"First let's grab `fastai2` (make sure your internet is turned on!)","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"!pip install fastai2 --quiet","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"And now let's grab the `vision` library and set up our `Path`","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from fastai2.vision.all import *\n\npath = Path('../input/global-wheat-detection')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"path.ls()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Our labels are inside of the `train.csv`, let's take alook","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"df = pd.read_csv(path/'train.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df['bbox'].isna().sum()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So we can see that we get an `image_id` and one label per row. We'll need to remember that in a little!","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"imgs = get_image_files(path/'train')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(imgs) == df['image_id'].nunique()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(imgs) - df['image_id'].nunique()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So we have 49 images that *aren't* labelled. Let's build on this now by simply making a custom set of `Paths` that contain our working images, and build a `get_items` for it","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"im_df = df['image_id'].unique()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"im_df = [fn + '.jpg' for fn in im_df]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"im_df[:5]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fns = [Path(str(path/'train') + f'/{fn}') for fn in im_df]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fns[0]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fns[0].name[:-4]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We'll make a `get_items` that simply returns our *good* images","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_items(noop): return fns","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## DataFrame Format:\n\n`image_id` is the same, `bbox` contains bounding boxes, all labels are `wheat`\n\nKeeping everything in the `DataFrame` is super inefficient, so we'll move everything to a NumPy array to load it faster.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"df['label'] = 'wheat'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_np = df.to_numpy()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`get_y` needs to return the coordinates then the label. Let's look at an example quickly","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"coco_source = untar_data(URLs.COCO_TINY)\nimages, lbl_bbox = get_annotations(coco_source/'train.json')\nimg2bbox = dict(zip(images, lbl_bbox))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fn = images[0]; fn","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"img2bbox[fn][0][0]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now that we know the format, let's work with our `DataFrame` to return something `fastai2` wants. First let's convert our bounding boxes into something we can use. NumPy has a nice `np.fromstring`, but it wants just numbers whereas our `DataFrame` will give us: `\"[0,0,0,0]\"`, which we don't want! So we'll replace both of the brackets first. This is just temporary for now though, as we want to make everything run on NumPy for efficiency","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_tmp_bbox(fn):\n    \"Grab bounding boxes from `DataFrame`\"\n    rows = np.where((df_np[:, 0] == fn.name[:-4]))\n    bboxs = df_np[rows][:,3]\n    bboxs = [b.replace('[', '').replace(']', '') for b in bboxs]\n    return np.array([np.fromstring(b, sep=',') for b in bboxs])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We don't require as much for the labels, as all of them are simply a string of \"wheat\"","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_tmp_lbl(fn):\n    \"Grab label from `DataFrame`\"\n    rows = np.where((df_np[:, 0] == fn.name[:-4]))\n    return df_np[rows][:,5]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fnames = df['image_id'].unique(); fnames[:3]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now let's start building our ground truth data. We'll want an initial array to add to","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"bboxs = get_tmp_bbox(fns[0])\nlbls = get_tmp_lbl(fns[0])\narr = np.array([fns[0].name[:-4], bboxs, lbls])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"arr","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"And now we can add the rest of the data","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"for fname in fns[1:]:\n    bbox = get_tmp_bbox(fname)\n    lbl = get_tmp_lbl(fname)\n    arr2 = np.array([fname.name[:-4], bbox, lbl])\n    arr = np.vstack((arr, arr2))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we have our actual data array, we need to make some adjustments. Currently our coordinates are x,y,w,h and we want x1,y1,x2,y2. So let's look at converting those!","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"arr[:,1][0][0][0] + arr[:,1][0][0][2]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To convert it, we need to add our width and height to the respective x and y. We can do this like so:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"arr[:,1][0][1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for i, im in enumerate(arr[:,1]):\n    for j, box in enumerate(im):\n        arr[:,1][i][j][2] = box[0]+box[2]\n        arr[:,1][i][j][3] = box[1]+box[3]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"arr[0][1][0]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"That looks much better!","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"np.save('data.npy', arr)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's make our true `get_bbox` and `get_lbl`. We'll want to first search our NumPy array for a matching filename, then grab the second or third index for the bounding box or the label respectively ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_bbox(fn):\n    \"Gets bounding box from `fn`\"\n    idx = np.where((arr[:,0] == fn.name[:-4]))\n    return arr[idx][0][1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_lbl(fn):\n    \"Get's label from `fn`\"\n    idx = np.where((arr[:,0] == fn.name[:-4]))\n    return arr[idx][0][2]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"For a true test of speed, to get the first value with pandas it takes ~6.4-6.6 milliseconds for each function. Let's see how ours does:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"%%timeit\n_ = get_bbox(fns[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"%%timeit\n_ = get_lbl(fns[0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"*Much* more efficent to use `NumPy` here.\n\n# DataLoaders","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"For our `DataLoaders`, we're going to want to use a `ImageBlock` for our input, and the `BBoxBlock` and `BBoxLblBlock` for our outputs, our custom `get_items`, along with some `get_y`'s. I chose some very simple transforms for us to use here. Finally we need to specify the number of inputs to simply be 1, telling `fastai` we have two outputs","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"wheat = DataBlock(blocks=(ImageBlock, BBoxBlock, BBoxLblBlock),\n                 get_items=get_items,\n                 splitter=RandomSplitter(),\n                 get_y=[get_bbox, get_lbl],\n                 item_tfms=Resize(256, method=ResizeMethod.Pad),\n                 n_inp=1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"And now we can build our `DataLoaders` and you're done!","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"dls = wheat.dataloaders(path,bs=32)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dls.show_batch(max_n=1, figsize=(12,12))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"batch = dls.one_batch()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"batch[1].shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"For a time comparison, the Pandas method took ~17 seconds to build the dataloaders and 1.3 seconds per batch. Using NumPy we reduce this to 90ms and 838ms per batch (with most of that time taken up by shuffling the data)","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}