{"cells":[{"metadata":{},"cell_type":"markdown","source":"**Objective:**To visualize the mnist dataset  using various Dimensionality Reduction Techniques in 2-d space\n\n\nDataSet Description:\n  dataset contains the (28*28) image and crossonding pixel values are stored in 784 dimension of vector.\n  dataset contains 42000 images pixel repersentation\n  one column in dataset contains information about digit label"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"df_train=pd.read_csv('/kaggle/input/digit-recognizer/train.csv')\ndf_test=pd.read_csv('/kaggle/input/digit-recognizer/test.csv')\nsubmission=pd.read_csv('/kaggle/input/digit-recognizer/sample_submission.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Separeting image pixel value and labels into seperate variables"},{"metadata":{"trusted":true},"cell_type":"code","source":"data=df_train.drop('label',axis=1)\nlabel=df_train['label']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"ploting the  image  in plot using their vector"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(7,7))\nidx = 25\n\ngrid_data = data.iloc[idx].as_matrix().reshape(28,28)  # reshape from 1d to 2d pixel array\nplt.imshow(grid_data, interpolation = \"none\", cmap = \"gray\")\nplt.show()\n\n\nprint(\"label \"+str(label[idx]))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we are convertiong the pixel value into standared for such that their mean is zero and lies between -1 and 1"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.preprocessing import StandardScaler\nstandardized_data = StandardScaler().fit_transform(data)\nprint(standardized_data.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**PCA(PRINIPLE COMPONENT ANALYSIER):**PCA is a technique which helps us in extracting a new set of variables from an existing large set of variables. These newly extracted variables are called Principal Components\nIN PCA the algorithm first compute covariance matrix using  formula np.matmul(sample_data.T , sample_data)\nthen we compute eigan vector for the covriance matrix\nSelect top k eigan vector \nConcert our datapoint into k-dimensional repersentaion by applying matrix multiplication between vector and data.T\n"},{"metadata":{},"cell_type":"markdown","source":"We can directely apply PCA using sklearn hear we have to pass the dimension of the resultant vector as n_components I am passing it as 2 because we can only visualize 2D and 3D data only."},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn import decomposition\npca = decomposition.PCA()\n# configuring the parameteres\n# the number of components = 2\npca.n_components = 2\npca_data = pca.fit_transform(data)\n\n# pca_reduced will contain the 2-d projects of simple data\nprint(\"shape of pca_reduced.shape = \", pca_data.shape)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"pca_data = np.vstack((pca_data.T, label)).T","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Plotting the result into the 2D plot"},{"metadata":{"trusted":true},"cell_type":"code","source":"import seaborn as sn\npca_df = pd.DataFrame(data=pca_data, columns=(\"1st_component\", \"2nd_component\", \"label\"))\nsn.FacetGrid(pca_df, hue=\"label\", size=6).map(plt.scatter, '1st_component', '2nd_component').add_legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As we can see above that  some points are well seperable but some points are not well seperable. But PCA works quite well and it seperates some of our classes very well but some are not seperable so we need another algorithm for visualizing our data point"},{"metadata":{},"cell_type":"markdown","source":"## T-SNE(T-disstribution Stochastic Neighborhood Embedding)"},{"metadata":{},"cell_type":"markdown","source":"This is also a Dimensionality Reduction algorithm.This algorithm is best Dimensionality Reduction technigue. It works very well for visualization.It tries to preserve neighbourhood structure in d-dimension space to lower dimension space.\n\nFor more details visit\nhttps://distill.pub/2016/misread-tsne/"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.manifold import TSNE\nmodel = TSNE(n_components=2, random_state=0)\ntsne_data = model.fit_transform(data)\ntsne_data = np.vstack((tsne_data.T, label)).T\ntsne_df = pd.DataFrame(data=tsne_data, columns=(\"Dimension_1\", \"Dimension_2\", \"label\"))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sn.FacetGrid(tsne_df, hue=\"label\", size=6).map(plt.scatter, 'Dimension_1', 'Dimension_2').add_legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As we can see that we get better result then PCA  we can also see all the digits clusters are seperable we can get best result buy tuning the value of Perplexity and Step parameter for more details visit \nhttps://distill.pub/2016/misread-tsne/\nhttps://scikit-learn.org/stable/modules/generated/sklearn.manifold.TSNE.html"},{"metadata":{},"cell_type":"markdown","source":"Thanks for viewing the kernel, please upvote if you liked it."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}