{"cells":[{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"from fastai.vision import *\nimport fastai\nimport pandas as pd","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This notebook is based on `fast.ai` course `v3` (2019) and version of `fastai` `1.0.57`. It looks like it changes pretty significantly from version to version so many 1-year old notebooks are pretty outdated. \n\nIt covers just the lesson 1: \n- train the model with the default `lr` for a few epochs with image `size=224`;\n- find new `lr` and unfreeze all the layers; "},{"metadata":{"trusted":true},"cell_type":"code","source":"fastai.__version__","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## data preparation"},{"metadata":{},"cell_type":"markdown","source":"There are a few issues with the file system:\n\n- your files are stored in `../input/dog-breed-identification`, not in `../input/`;\n- during different runs of this notebook you may encounter a bug and `len(path_test.ls())` will give you `10358` instead of `10357`; in this case you have to specify `path_test = path / 'test/test'` and `test='test/test'`;\n- finally, you should store the model in `tmp` directory specified below and `'submission.csv'` in the current directory;"},{"metadata":{"trusted":true},"cell_type":"code","source":"Path('../input').ls(), ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"path = Path('../input/dog-breed-identification')\npath_train = path / 'train'\npath_test = path / 'test/test'\npath_model = Path('/tmp/model/')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"path_train, path_test","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"path.ls()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(path_train.ls()), len(path_test.ls())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"list(Path('../input/dog-breed-identification').glob('**'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"bs, size = 64, 224","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data = ImageDataBunch.from_csv(path=path,\n                               folder='train',\n                               csv_labels='labels.csv',\n                               ds_tfms=get_transforms(),\n                               suffix='.jpg',\n                               test='test/test',\n                               size=size,\n                               bs=bs,\n                               num_workers=0).normalize(imagenet_stats)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(data.train_ds), len(data.valid_ds), len(data.test_ds.items)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"8178 + 2044","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"> ## vanilla model"},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = cnn_learner(data, \n                    base_arch=models.resnet50, \n                    metrics=accuracy, \n                    model_dir=path_model)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.lr_find()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.recorder.plot()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So we have to choose a point with the steepest descent where the plot goes down for a while. That's `1e-3` or basically the same as the default `lr`. So we may just train with the default rate."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.fit_one_cycle(8)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.save('stage-1-50-224')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.unfreeze()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.lr_find()\nlearn.recorder.plot()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Here we don't have any steep curves so we may just choose `1e-6` or `1e-5` as the left limit and we choose `1e-4` as the right limit (we may use the rule that it's equal to previous `lr // 10` or we may just use the standard value for it which is `1e-4` as well)."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.fit_one_cycle(4, max_lr=slice(1e-6,1e-4))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So no improvement, let's load the previous model."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.load('stage-1-50-224');","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"## submission"},{"metadata":{"trusted":true},"cell_type":"code","source":"preds, _ = learn.get_preds(DatasetType.Test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data.test_ds.items[0]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"path_test","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_numeric_part(file_path, path_test):\n    low = len(str(path_test)) + 1\n    return str(file_path)[low:-4]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"get_numeric_part(data.test_ds.items[0], path_test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df = pd.DataFrame(preds.numpy(), columns=data.classes)\nfilenames = [get_numeric_part(fp, path_test) for fp in data.test_ds.items]\ndf.insert(0, \"id\", filenames)\ndf_sorted = df.sort_values(by='id')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_sorted.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_sorted.iloc[0, 1:].sum()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_sorted.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"ls","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}