{"cells":[{"metadata":{"trusted":true},"cell_type":"markdown","source":"The objective of this notebook is to show how one of the fastest dimensionality reduction techniques UMAP (which is also a great tSNE alternative) groups similar digits of Kannada MNIST dataset. \n\nAs we know before, the dataset is an MNIST-like dataset with first column as label and 784 (28 x 28) pixel values"},{"metadata":{},"cell_type":"markdown","source":"### UMAP\n\nUniform Manifold Approximation and Projection (UMAP) is a dimension reduction technique that can be used for visualisation similarly to t-SNE, but also for general non-linear dimension reduction."},{"metadata":{},"cell_type":"markdown","source":"### Loading required libraries starting from `umap` "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import umap\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\nplt.figure(figsize=(20,10))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Read Training dataset"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"kannada_train = pd.read_csv(\"../input/Kannada-MNIST/train.csv\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### To reduce the execution time for learning purpose, We'll take only 10K rows randomly sampled out of 60K rows from the training dataset"},{"metadata":{"trusted":true},"cell_type":"code","source":"kannada_train_sampled = kannada_train.sample(n = 10000, random_state = 123)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Looking at the label distribution after random sampling"},{"metadata":{"trusted":true},"cell_type":"code","source":"kannada_train_sampled.groupby('label')['label'].count()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Visualizing a sample digit"},{"metadata":{"trusted":true},"cell_type":"code","source":"\nimage = np.array(kannada_train_sampled.iloc[2,1:785], dtype='float')\npixels = image.reshape((28, 28))\nplt.imshow(pixels, cmap='gray')\nplt.title(kannada_train_sampled.iloc[2,0])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`random_state` parameter is set for reproducibility\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"data = kannada_train_sampled.iloc[:, 1:].values.astype(np.float32)\ntarget = kannada_train_sampled['label'].values","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### UMAP() function for dimensionality reduction. \n\n`UMAP()` takes other hyperparameters like \n\n    n_neighbors\n    min_dist\n    n_components\n    metric\n\nby default, `n_components` is set to 2 which makes the reduced dimension into 2"},{"metadata":{"trusted":true},"cell_type":"code","source":"%time reduce = umap.UMAP(random_state = 123) #just for reproducibility\n%time  embedding = reduce.fit_transform(data)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Converting the output embedding into a dataframe \n### And, Renaming the columns in it for easier access"},{"metadata":{},"cell_type":"markdown","source":"Since we have just two dimensions, let's name it x and y"},{"metadata":{"trusted":true},"cell_type":"code","source":"df = pd.DataFrame(embedding, columns=('x', 'y'))\ndf[\"class\"] = target\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Adding the target column for visualization labelling"},{"metadata":{},"cell_type":"markdown","source":"### Kannada MNIST clustered (dimensions reduced - Visulization) with UMAP"},{"metadata":{},"cell_type":"markdown","source":"A simple scatter plot with x and y, color from target label"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.scatter(df.x, df.y, s= 5, c=target, cmap='Spectral')\nplt.colorbar(boundaries=np.arange(11)-0.5).set_ticks(np.arange(10))\nplt.title('Kannada MNIST clustered (dimensions reduced - Visulization) with UMAP', fontsize=20);\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As, we can see `UMAP` has done a pretty good job in identifying distinct groups in the dataset purely based on the dimensions / columns (pixel values).\n\nUMAP is definitely a good-fast alternative wherever you think tSNE could be useful. "},{"metadata":{},"cell_type":"markdown","source":"### Reference\n\n+ [`umap` Python Package Github](https://github.com/lmcinnes/umap)\n+ [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://arxiv.org/abs/1802.03426)"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}