{"cells":[{"metadata":{},"cell_type":"markdown","source":"## Introduction\n\nThis notebook double checks validtity of un-supervised machine learning model using t-SNE dimentionality reduction technique to show meaningful clusters. \nIn this repsect, thisnotbook uses two popular datasets:\n1. MNIST-digit: 0-9\n2. Fashion MNIST\n\nBoth datasets use pixel of image in csv.\n\n### About Data\nThe MNIST database of handwritten digits, has a training set of 60000 examples, and a test set of 10,000 examples. The training and test dataset rows contain pixels of 28 x 28 handwritten digits flattened into single row i.e. each row has 784 data pixel points."},{"metadata":{},"cell_type":"markdown","source":"## What is in this notebook?\n\nThough, the trainig dataset contains `labels` for the digit representing each data point, this kernel aims to build a un-supervised learning algorithm. \n\nWe use two decomposition techniques to build clusters and plot it using matplotlib and seaborn libraries. "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"#import required libraries\nimport numpy as np\nimport pandas as pd\n\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n\n%matplotlib inline\n\npd.set_option('display.max_rows', None)\npd.set_option('display.max_columns', None)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"*Loading trainig and test sets*"},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train = pd.read_csv(\"../input/digit-recognizer/train.csv\")\ndf_test = pd.read_csv(\"../input/digit-recognizer/test.csv\")\ndf_train.head()\n#df_test.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_test.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Note: Though, training dataset contains column `label` with ground thruth values, and it can be used to build `supervised` classification model. However, we aim to build a `unsupervised` learning model with help of dimensionality reduction techniques i.e., PCA and t-SNE."},{"metadata":{},"cell_type":"markdown","source":"Let's visulaize some images"},{"metadata":{},"cell_type":"raw","source":""},{"metadata":{},"cell_type":"markdown","source":""},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train_pixels = df_train.iloc[:,1:] #slice target column out i.e. 1st column and keep only the pixels\ndf_target = df_train.iloc[:,0]\ndf_train_pixels.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#df_target","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Take single row pixel values (784) and reshape it to 28 x 28 2D matrix where each cell has respective pixel values."},{"metadata":{"trusted":true},"cell_type":"code","source":"#pixels of 1st image\nimage_1 = df_train_pixels.values[0].reshape(28,28)\nplt.imshow(image_1, cmap='gray')\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#pixels of 2nd image\nimage_1 = df_train_pixels.values[1].reshape(28,28)\nplt.imshow(image_1, cmap='gray')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Lets build a facet using matplotlib comprising of top 10 images.** "},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"plt.figure(figsize=(20,15))\nfor index, (image, label) in enumerate(zip(df_train_pixels[0:10], \n                                           df_target[0:10])):\n    plt.subplot(3, 5, index + 1)\n    plt.imshow(np.reshape(df_train_pixels.values[index], (28,28)), cmap=plt.cm.gray)\n    plt.title('Label: %i\\n' % label, fontsize = 20);","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#import sklearn manifold library\nfrom sklearn import manifold","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train_pixels[0:10].to_numpy()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tsne = manifold.TSNE(n_components=2, random_state=1)\ntransf_data = tsne.fit_transform(df_train_pixels[:10000].to_numpy())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# The transformed data is of 10000 data points and 2 t-SNE components\ntransf_data.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Lets convert this numpy array to dataframe\ndf_tsne = pd.DataFrame(np.column_stack((transf_data, df_target[:10000])),\n                      columns = [\"X\", \"y\", \"label\"])\ndf_tsne.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Do we need to change is to int\ndf_tsne['label'] = df_tsne.label.astype(int) ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig= plt.figure(figsize=(12,10))\nsns.scatterplot(data=df_tsne, x=\"X\", y=\"y\", hue=\"label\", palette=\"deep\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Fashion MNIST:\n\nAbove results show, one gets good unsupervised result with t-SNE features for MNIST data set. It would be interesting to double check if t-SNE fails to give meanigful cluster with another data set e.g., Fashin-MNIST. Here, we will be replicating above steps to only double check t-SNE validaity. "},{"metadata":{},"cell_type":"markdown","source":""},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train = pd.read_csv(\"../input/fashionmnist/fashion-mnist_train.csv\")\ndf_test = pd.read_csv(\"../input/fashionmnist/fashion-mnist_test.csv\")\ndf_train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train_pixels = df_train.iloc[:,1:] #slice target column out i.e. 1st column and keep only the pixels\ndf_target = df_train.iloc[:,0]\ndf_train_pixels.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#pixels of 1st image\nimage_1 = df_train_pixels.values[0].reshape(28,28)\nplt.imshow(image_1, cmap='gray')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(20,15))\nfor index, (image, label) in enumerate(zip(df_train_pixels[0:10], \n                                           df_target[0:10])):\n    plt.subplot(3, 5, index + 1)\n    plt.imshow(np.reshape(df_train_pixels.values[index], (28,28)), cmap=plt.cm.gray)\n    plt.title('Label: %i\\n' % label, fontsize = 20);","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#import sklearn manifold library\nfrom sklearn import manifold","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train_pixels[0:10].to_numpy()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"tsne = manifold.TSNE(n_components=2, random_state=1)\ntransf_data = tsne.fit_transform(df_train_pixels[:10000].to_numpy())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# The transformed data is of 10000 data points and 2 t-SNE components\ntransf_data.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Lets convert this numpy array to dataframe\ndf_tsne = pd.DataFrame(np.column_stack((transf_data, df_target[:10000])),\n                      columns = [\"X\", \"y\", \"label\"])\ndf_tsne.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Do we need to change is to int\ndf_tsne['label'] = df_tsne.label.astype(int) ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig= plt.figure(figsize=(12,10))\nsns.scatterplot(data=df_tsne, x=\"X\", y=\"y\", hue=\"label\", palette=\"deep\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":""},{"metadata":{},"cell_type":"markdown","source":"The results from both data sets are interesting. MNIST data shows clear clusters for all the 10 hand written digits. Nevertheless, tSNE reduction in dimension only captured 7 clusters out of 10 fashion items. Therefore, it can be concluded that using t-SNE does give competitive results but it accuracy depends on the underlining data. "},{"metadata":{},"cell_type":"markdown","source":"Reference:\n1. Thakur, Abhishek. Approaching (almost) any machine learning problem. Abhishek Thakur, 2020."},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}