{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Simple Fastai/Pytorch Example"},{"metadata":{},"cell_type":"markdown","source":"Here is my first bare-bones attempt at the MNIST Kannada challenge. This 1st commit notebook contains a fully convolutional network implemented with Fastai/Pytorch. "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"from fastai import *\nfrom fastai.vision import *\nDATAPATH = Path('/kaggle/input/Kannada-MNIST/')\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Data processing"},{"metadata":{"trusted":true},"cell_type":"markdown","source":"The data given to us is in a .csv format. The first column contains the image label/id, and the rest of the columns contain the image pixel values in grayscale. To process the csv into something we can train a network with, I \n- read the csv into a pandas dataframe\n- extract the label/id\n- remove the label/id column and reshape the image data into a 28x28 square\n- give the resulting array an extra dimension to indicate the images are in grayscale"},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_data_labels(csv,label):\n    fileraw = pd.read_csv(csv)\n    labels = fileraw[label].to_numpy()\n    data = fileraw.drop([label],axis=1).to_numpy(dtype=np.float32).reshape((fileraw.shape[0],28,28))\n    data = np.expand_dims(data, axis=1)\n    return data, labels","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Each of the datasets is processed to obtain the data and label/id. "},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data, train_labels = get_data_labels(DATAPATH/'train.csv','label')\ntest_data, test_labels = get_data_labels(DATAPATH/'test.csv','id')\nother_data, other_labels = get_data_labels(DATAPATH/'Dig-MNIST.csv','label')\ntrain_data.shape, train_labels.shape, test_data.shape, test_labels.shape, other_data.shape, other_labels.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To ensure the the data read in properly, I display one. Seems ok."},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.title(f'Training Label: {train_labels[2]}')\nplt.imshow(train_data[2,0],cmap='gray');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Creating a Fastai Databunch"},{"metadata":{},"cell_type":"markdown","source":"I want to use Fastai's code for training the model. First, however, I created a validation dataset from 10% of the training data."},{"metadata":{"trusted":true},"cell_type":"code","source":"np.random.seed(42)\nran_10_pct_idx = (np.random.random_sample(train_labels.shape)) < .1\n\ntrain_90_labels = train_labels[np.invert(ran_10_pct_idx)]\ntrain_90_data = train_data[np.invert(ran_10_pct_idx)]\n\nvalid_10_labels = train_labels[ran_10_pct_idx]\nvalid_10_data = train_data[ran_10_pct_idx]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The image data is currently in numpy arrays. Fastai does not have a built in dataset to handle images in this format, so I created a bare-bones Torch Dataset class. [Code from here](https://docs.fast.ai/basic_data.html)"},{"metadata":{"trusted":true},"cell_type":"code","source":"class ArrayDataset(Dataset):\n    \"Dataset for numpy arrays based on fastai example: \"\n    def __init__(self, x, y):\n        self.x, self.y = x, y\n        self.c = len(np.unique(y))\n    \n    def __len__(self):\n        return len(self.x)\n    \n    def __getitem__(self, i):\n        return self.x[i], self.y[i]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_ds = ArrayDataset(train_90_data,train_90_labels)\nvalid_ds = ArrayDataset(valid_10_data,valid_10_labels)\nother_ds = ArrayDataset(other_data, other_labels)\ntest_ds = ArrayDataset(test_data, test_labels)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Finally, I can create a Databunch, which contains, my training, validation and test sets, along with the batch size."},{"metadata":{"trusted":true},"cell_type":"code","source":"bs = 128\ndatabunch = DataBunch.create(train_ds, valid_ds, test_ds=test_ds, bs=bs)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Simple model: fully convolutional network"},{"metadata":{},"cell_type":"markdown","source":"The 28x28 images are tiny, so I did not want to use a large pretrained model, like ResNet. This is a fully convolutional network created in a blend of in pytorchand Fastai. The 'conv_layer' is a Fastai function that returns a convolutional layer, batchnorm, and RELU. "},{"metadata":{"trusted":true},"cell_type":"code","source":"def conv2(ni,nf,stride=2,ks=3): return conv_layer(ni,nf,stride=stride,ks=ks)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"smallConvolutional = nn.Sequential(\n    conv2(1,8,ks=5),\n    conv2(8,16),\n    conv2(16,32),\n    conv2(32, 16),\n    conv2(16, 10),\n    Flatten()\n)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This 'learner' in Fastai holds the data, model, loss function, and metric of interest."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = Learner(databunch, smallConvolutional, loss_func = nn.CrossEntropyLoss(), metrics=[accuracy] )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This uses a [one cycle policy](https://docs.fast.ai/callbacks.one_cycle.html) for training. Eight cycles a an arbitrary number; the accuracy on the validation set typically reaches ~99.2%."},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.fit_one_cycle(8)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"# Test data set"},{"metadata":{},"cell_type":"markdown","source":"Now, we can get the predictions for the test set. "},{"metadata":{"trusted":true},"cell_type":"code","source":"preds, ids = learn.get_preds(DatasetType.Test)\ny = torch.argmax(preds, dim=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission = pd.DataFrame({ 'id': ids,'label': y })","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission.to_csv(path_or_buf =\"submission.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}