{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-07-01T13:46:56.481029Z","iopub.execute_input":"2022-07-01T13:46:56.481672Z","iopub.status.idle":"2022-07-01T13:46:56.490088Z","shell.execute_reply.started":"2022-07-01T13:46:56.481629Z","shell.execute_reply":"2022-07-01T13:46:56.488972Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Going to work with Only train data \ndf = pd.read_csv('/kaggle/input/digit-recognizer/train.csv') ","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:46:58.210327Z","iopub.execute_input":"2022-07-01T13:46:58.21125Z","iopub.status.idle":"2022-07-01T13:47:02.167823Z","shell.execute_reply.started":"2022-07-01T13:46:58.211201Z","shell.execute_reply":"2022-07-01T13:47:02.166577Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:02.170182Z","iopub.execute_input":"2022-07-01T13:47:02.171169Z","iopub.status.idle":"2022-07-01T13:47:02.202116Z","shell.execute_reply.started":"2022-07-01T13:47:02.17112Z","shell.execute_reply":"2022-07-01T13:47:02.200975Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.shape","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:02.203497Z","iopub.execute_input":"2022-07-01T13:47:02.203833Z","iopub.status.idle":"2022-07-01T13:47:02.21118Z","shell.execute_reply.started":"2022-07-01T13:47:02.203803Z","shell.execute_reply":"2022-07-01T13:47:02.209976Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- ***So there are 785 features thats alot of features, now this is the curse of dimensionality and we need to reduce this using dimensionality reduction techniques like Tsne,PCA; here I'm going to use PCA {Principal Component Analysis}.***\n\n- ***Let's ask the question or try to understand why there are so many features , so this is because of there are pictures and each picture is of 28*28 pixels which equals to 784 pixels; they made each pixel a column and first column is the label/target feature which tells us about what digit is in the image.There are 42000 such images , I hope you got the gist of it.***","metadata":{}},{"cell_type":"code","source":"df.sample()","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:02.21441Z","iopub.execute_input":"2022-07-01T13:47:02.215388Z","iopub.status.idle":"2022-07-01T13:47:02.238873Z","shell.execute_reply.started":"2022-07-01T13:47:02.215333Z","shell.execute_reply":"2022-07-01T13:47:02.237764Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***Let me show you how this label is connected using matplotlib***","metadata":{}},{"cell_type":"code","source":"import matplotlib.pyplot as plt\nplt.imshow(df.iloc[30421,1:].values.reshape(28,28))","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:02.240846Z","iopub.execute_input":"2022-07-01T13:47:02.241186Z","iopub.status.idle":"2022-07-01T13:47:02.455696Z","shell.execute_reply.started":"2022-07-01T13:47:02.241148Z","shell.execute_reply":"2022-07-01T13:47:02.454482Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## ***Now I'll run the machine learning algorithm and see how things go without any dimensionality reduction***","metadata":{}},{"cell_type":"code","source":"X= df.iloc[:,1:]\ny = df.iloc[:,0]","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:03.837408Z","iopub.execute_input":"2022-07-01T13:47:03.838184Z","iopub.status.idle":"2022-07-01T13:47:03.843419Z","shell.execute_reply.started":"2022-07-01T13:47:03.838141Z","shell.execute_reply":"2022-07-01T13:47:03.842495Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\nX_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2 , random_state=42)","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:04.388259Z","iopub.execute_input":"2022-07-01T13:47:04.389293Z","iopub.status.idle":"2022-07-01T13:47:06.124666Z","shell.execute_reply.started":"2022-07-01T13:47:04.389251Z","shell.execute_reply":"2022-07-01T13:47:06.123456Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train.shape","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:06.126352Z","iopub.execute_input":"2022-07-01T13:47:06.126682Z","iopub.status.idle":"2022-07-01T13:47:06.133087Z","shell.execute_reply.started":"2022-07-01T13:47:06.126653Z","shell.execute_reply":"2022-07-01T13:47:06.131971Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:06.700205Z","iopub.execute_input":"2022-07-01T13:47:06.700642Z","iopub.status.idle":"2022-07-01T13:47:06.747546Z","shell.execute_reply.started":"2022-07-01T13:47:06.700606Z","shell.execute_reply":"2022-07-01T13:47:06.746292Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# NO missing data , YAY this is a perfect data\n\nfrom sklearn.neighbors import KNeighborsClassifier","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:07.056402Z","iopub.execute_input":"2022-07-01T13:47:07.056823Z","iopub.status.idle":"2022-07-01T13:47:07.201649Z","shell.execute_reply.started":"2022-07-01T13:47:07.056789Z","shell.execute_reply":"2022-07-01T13:47:07.200497Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"knn = KNeighborsClassifier() # using default settings","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:07.372118Z","iopub.execute_input":"2022-07-01T13:47:07.372566Z","iopub.status.idle":"2022-07-01T13:47:07.378323Z","shell.execute_reply.started":"2022-07-01T13:47:07.372527Z","shell.execute_reply":"2022-07-01T13:47:07.377055Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"knn.fit(X_train,y_train)","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:08.334835Z","iopub.execute_input":"2022-07-01T13:47:08.335802Z","iopub.status.idle":"2022-07-01T13:47:08.359285Z","shell.execute_reply.started":"2022-07-01T13:47:08.335759Z","shell.execute_reply":"2022-07-01T13:47:08.358441Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- ***The KNN algorithm assumes that similar things exist in close proximity. In other words, similar things are near to each other.***\n\n- ***So when I'll write knn.predit, it's going to take more time because what I am saying to KNN is to compute the distances from a given point to other points (almost 33.5k ) then sort them to know who are the nearest neighbors and use their label that too in 784 dimensional space.***\n","metadata":{}},{"cell_type":"code","source":"import time \nstart = time.time()\ny_pred = knn.predict(X_test)\nprint(time.time()-start)","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:12.97453Z","iopub.execute_input":"2022-07-01T13:47:12.975564Z","iopub.status.idle":"2022-07-01T13:47:26.094365Z","shell.execute_reply.started":"2022-07-01T13:47:12.975516Z","shell.execute_reply":"2022-07-01T13:47:26.092978Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from sklearn.metrics import accuracy_score\naccuracy_score(y_test,y_pred)","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:28.366613Z","iopub.execute_input":"2022-07-01T13:47:28.367402Z","iopub.status.idle":"2022-07-01T13:47:28.374936Z","shell.execute_reply.started":"2022-07-01T13:47:28.367366Z","shell.execute_reply":"2022-07-01T13:47:28.373821Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# It's not bad but lets use PCA and gain this much accuracy with less no of columns\nfrom sklearn.preprocessing import StandardScaler\nscaler = StandardScaler()","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:31.056079Z","iopub.execute_input":"2022-07-01T13:47:31.05685Z","iopub.status.idle":"2022-07-01T13:47:31.061636Z","shell.execute_reply.started":"2022-07-01T13:47:31.056807Z","shell.execute_reply":"2022-07-01T13:47:31.060439Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ***Step 1 Standardize the data (Not mandatory but gives better results)***","metadata":{}},{"cell_type":"code","source":"X_train = scaler.fit_transform(X_train)\nX_test = scaler.transform(X_test)","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:32.431084Z","iopub.execute_input":"2022-07-01T13:47:32.431581Z","iopub.status.idle":"2022-07-01T13:47:32.938547Z","shell.execute_reply.started":"2022-07-01T13:47:32.431543Z","shell.execute_reply":"2022-07-01T13:47:32.937239Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ***Step 2 Getting the Eigen vectors and values***","metadata":{}},{"cell_type":"code","source":"from sklearn.decomposition import PCA\npca = PCA(n_components = 100 ) # gettng 100 principal components","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:34.617798Z","iopub.execute_input":"2022-07-01T13:47:34.618213Z","iopub.status.idle":"2022-07-01T13:47:34.623732Z","shell.execute_reply.started":"2022-07-01T13:47:34.618177Z","shell.execute_reply":"2022-07-01T13:47:34.622658Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train_trf = pca.fit_transform(X_train)\nX_test_trf = pca.transform(X_test)","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:35.224847Z","iopub.execute_input":"2022-07-01T13:47:35.225264Z","iopub.status.idle":"2022-07-01T13:47:38.795324Z","shell.execute_reply.started":"2022-07-01T13:47:35.225227Z","shell.execute_reply":"2022-07-01T13:47:38.79382Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train_trf.shape","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:38.798089Z","iopub.execute_input":"2022-07-01T13:47:38.798929Z","iopub.status.idle":"2022-07-01T13:47:38.807115Z","shell.execute_reply.started":"2022-07-01T13:47:38.798875Z","shell.execute_reply":"2022-07-01T13:47:38.805853Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# ***Step 3: Training the data***","metadata":{}},{"cell_type":"code","source":"knn = KNeighborsClassifier()","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:38.809142Z","iopub.execute_input":"2022-07-01T13:47:38.809972Z","iopub.status.idle":"2022-07-01T13:47:38.8178Z","shell.execute_reply.started":"2022-07-01T13:47:38.809915Z","shell.execute_reply":"2022-07-01T13:47:38.816476Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"knn.fit(X_train_trf,y_train)","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:40.466022Z","iopub.execute_input":"2022-07-01T13:47:40.466467Z","iopub.status.idle":"2022-07-01T13:47:40.481413Z","shell.execute_reply.started":"2022-07-01T13:47:40.466401Z","shell.execute_reply":"2022-07-01T13:47:40.480624Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"start = time.time()\ny_pred = knn.predict(X_test_trf)\nprint(time.time() - start)","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:41.201149Z","iopub.execute_input":"2022-07-01T13:47:41.202156Z","iopub.status.idle":"2022-07-01T13:47:48.478394Z","shell.execute_reply.started":"2022-07-01T13:47:41.202111Z","shell.execute_reply":"2022-07-01T13:47:48.477205Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"accuracy_score(y_test ,y_pred)","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:51.158646Z","iopub.execute_input":"2022-07-01T13:47:51.160786Z","iopub.status.idle":"2022-07-01T13:47:51.168851Z","shell.execute_reply.started":"2022-07-01T13:47:51.16073Z","shell.execute_reply":"2022-07-01T13:47:51.167906Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"- ***Lets understand now , when we started there were 784 dimensions and when we applied same settings with everything on 100 dimensions which we got using PCA , we got a little less accuracy but we saved computation(time) and dimensionality space is reduced as well. win-win for us.***\n\n- ***Now I'll loop in through multiple ranges / no of dimensions to know how many components are best for this.***","metadata":{}},{"cell_type":"code","source":"for i in range(1,36):\n    pca =PCA(n_components = i)\n    X_train_trf = pca.fit_transform(X_train)\n    X_test_trf =pca.transform(X_test)\n    \n    knn = KNeighborsClassifier()\n    knn.fit(X_train_trf,y_train)\n    y_pred = knn.predict(X_test_trf)\n    acc_score = accuracy_score(y_test,y_pred)\n    print(f'Accuracy with {i} Components is {acc_score}')","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:47:54.105723Z","iopub.execute_input":"2022-07-01T13:47:54.106189Z","iopub.status.idle":"2022-07-01T13:52:02.664164Z","shell.execute_reply.started":"2022-07-01T13:47:54.106152Z","shell.execute_reply":"2022-07-01T13:52:02.663134Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***Now we can see that gradually the accuracy increases and after some principal components it remains same , so we need to find that optimum number but before that let's visualize.***\n\n## ***Second thing for which PCA used is Visualization***","metadata":{}},{"cell_type":"code","source":"# Transforming it into a 2D coordinate system\npca = PCA(n_components = 2)\nX_train_trf = pca.fit_transform(X_train)\nX_test_trf = pca.fit(X_test)","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:52:56.396779Z","iopub.execute_input":"2022-07-01T13:52:56.397219Z","iopub.status.idle":"2022-07-01T13:52:59.811963Z","shell.execute_reply.started":"2022-07-01T13:52:56.397188Z","shell.execute_reply":"2022-07-01T13:52:59.810469Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train_trf","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:53:02.543286Z","iopub.execute_input":"2022-07-01T13:53:02.543682Z","iopub.status.idle":"2022-07-01T13:53:02.551978Z","shell.execute_reply.started":"2022-07-01T13:53:02.543651Z","shell.execute_reply":"2022-07-01T13:53:02.550812Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import plotly.express as px\ny_train_trf = y_train.astype(str)\nfig = px.scatter(x= X_train_trf[:,0],\n                 y = X_train_trf[:,1],\n                 color = y_train_trf,\n                 color_discrete_sequence=px.colors.qualitative.G10)\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:53:03.547773Z","iopub.execute_input":"2022-07-01T13:53:03.548158Z","iopub.status.idle":"2022-07-01T13:53:06.320908Z","shell.execute_reply.started":"2022-07-01T13:53:03.548127Z","shell.execute_reply":"2022-07-01T13:53:06.318972Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***Isn't it beautiful , Now we can easily visualize and because of the power of plotly we can even interact with it , I love doing that but still alot og points are overalapped , so let's get this in 3D.***","metadata":{}},{"cell_type":"code","source":"# Transforming it into a 3D coordinate system\npca = PCA(n_components = 3)\nX_train_trf = pca.fit_transform(X_train)\nX_test_trf = pca.fit(X_test)","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:54:00.752747Z","iopub.execute_input":"2022-07-01T13:54:00.753215Z","iopub.status.idle":"2022-07-01T13:54:04.236799Z","shell.execute_reply.started":"2022-07-01T13:54:00.753179Z","shell.execute_reply":"2022-07-01T13:54:04.235209Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train_trf","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:54:04.24406Z","iopub.execute_input":"2022-07-01T13:54:04.246625Z","iopub.status.idle":"2022-07-01T13:54:04.260359Z","shell.execute_reply.started":"2022-07-01T13:54:04.246567Z","shell.execute_reply":"2022-07-01T13:54:04.25898Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"y_train_trf = y_train.astype(str)\nfig = px.scatter_3d(df , x= X_train_trf[:,0],\n                 y = X_train_trf[:,1],z = X_train_trf[:,2],\n                 color = y_train_trf)\nfig.update_layout(margin = dict(l=20,r=20,t=20,b=20))\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:54:04.327677Z","iopub.execute_input":"2022-07-01T13:54:04.328801Z","iopub.status.idle":"2022-07-01T13:54:04.640111Z","shell.execute_reply.started":"2022-07-01T13:54:04.328746Z","shell.execute_reply":"2022-07-01T13:54:04.638883Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***Just WOW and you can understand alot of different patterns in data using the interactive plot***","metadata":{}},{"cell_type":"code","source":"pca.explained_variance_   ## Top 3 Eigen values ","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:54:11.666982Z","iopub.execute_input":"2022-07-01T13:54:11.667363Z","iopub.status.idle":"2022-07-01T13:54:11.675029Z","shell.execute_reply.started":"2022-07-01T13:54:11.66733Z","shell.execute_reply":"2022-07-01T13:54:11.673752Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pca.components_  ##Eigen vectors","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:54:13.34527Z","iopub.execute_input":"2022-07-01T13:54:13.346264Z","iopub.status.idle":"2022-07-01T13:54:13.355635Z","shell.execute_reply.started":"2022-07-01T13:54:13.346221Z","shell.execute_reply":"2022-07-01T13:54:13.354494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pca.components_.shape ##3 vectors with 784 shape","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:54:15.591485Z","iopub.execute_input":"2022-07-01T13:54:15.592212Z","iopub.status.idle":"2022-07-01T13:54:15.603963Z","shell.execute_reply.started":"2022-07-01T13:54:15.592176Z","shell.execute_reply":"2022-07-01T13:54:15.602675Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pca.explained_variance_ratio_ * 100 ## this gives us the percentage of variance each eigen value have ","metadata":{"execution":{"iopub.status.busy":"2022-07-01T13:55:45.832545Z","iopub.execute_input":"2022-07-01T13:55:45.833757Z","iopub.status.idle":"2022-07-01T13:55:45.841552Z","shell.execute_reply.started":"2022-07-01T13:55:45.833696Z","shell.execute_reply":"2022-07-01T13:55:45.84025Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## ***Let's find out the optimum number of principal components***","metadata":{}},{"cell_type":"code","source":"pca = PCA(n_components = None)\nX_train_trf = pca.fit_transform(X_train)\nX_test_trf = pca.fit(X_test)","metadata":{"execution":{"iopub.status.busy":"2022-07-01T14:01:42.906467Z","iopub.execute_input":"2022-07-01T14:01:42.906951Z","iopub.status.idle":"2022-07-01T14:01:48.56278Z","shell.execute_reply.started":"2022-07-01T14:01:42.906912Z","shell.execute_reply":"2022-07-01T14:01:48.561384Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pca.explained_variance_.shape","metadata":{"execution":{"iopub.status.busy":"2022-07-01T14:02:07.97941Z","iopub.execute_input":"2022-07-01T14:02:07.979859Z","iopub.status.idle":"2022-07-01T14:02:07.986676Z","shell.execute_reply.started":"2022-07-01T14:02:07.979815Z","shell.execute_reply":"2022-07-01T14:02:07.985859Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pca.components_.shape","metadata":{"execution":{"iopub.status.busy":"2022-07-01T14:02:27.273021Z","iopub.execute_input":"2022-07-01T14:02:27.273833Z","iopub.status.idle":"2022-07-01T14:02:27.279822Z","shell.execute_reply.started":"2022-07-01T14:02:27.273746Z","shell.execute_reply":"2022-07-01T14:02:27.278927Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pca.explained_variance_ratio_","metadata":{"execution":{"iopub.status.busy":"2022-07-01T14:02:54.302785Z","iopub.execute_input":"2022-07-01T14:02:54.303388Z","iopub.status.idle":"2022-07-01T14:02:54.321965Z","shell.execute_reply.started":"2022-07-01T14:02:54.303353Z","shell.execute_reply":"2022-07-01T14:02:54.320685Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***Now I'll do cumulative sum on this till I get the 90 percent***","metadata":{}},{"cell_type":"code","source":"np.cumsum(pca.explained_variance_ratio_)","metadata":{"execution":{"iopub.status.busy":"2022-07-01T14:04:57.158574Z","iopub.execute_input":"2022-07-01T14:04:57.159067Z","iopub.status.idle":"2022-07-01T14:04:57.17618Z","shell.execute_reply.started":"2022-07-01T14:04:57.159025Z","shell.execute_reply":"2022-07-01T14:04:57.174961Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt. plot(np.cumsum(pca.explained_variance_ratio_))\nplt.xlabel('No of Principle Components' ,color ='red')\nplt.ylabel('Variance Explained' ,color ='red')","metadata":{"execution":{"iopub.status.busy":"2022-07-01T14:07:16.361546Z","iopub.execute_input":"2022-07-01T14:07:16.361959Z","iopub.status.idle":"2022-07-01T14:07:16.563308Z","shell.execute_reply.started":"2022-07-01T14:07:16.361925Z","shell.execute_reply":"2022-07-01T14:07:16.56204Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"***Now around (200~250) we get 90 %***","metadata":{}},{"cell_type":"markdown","source":"# ***Thanks for giving it a read , Do upvote it if you learned something out of it.***\n\n### Advantages of PCA:\n\n- ***Easy to compute. PCA is based on linear algebra, which is computationally easy to solve by computers.***\n\n- ***Speeds up other machine learning algorithms. Machine learning algorithms converge faster when trained on principal components instead of the original dataset.***\n\n- ***Counteracts the issues of high-dimensional data. High-dimensional data causes regression-based algorithms to overfit easily. By using PCA beforehand to lower the dimensions of the training dataset, we prevent the predictive algorithms from overfitting.*** \n\n### Disadvantages of PCA:\n\n- ***Low interpretability of principal components. Principal components are linear combinations of the features from the original data, but they are not as easy to interpret. For example, it is difficult to tell which are the most important features in the dataset after computing principal components.** *\n\n- ***The trade-off between information loss and dimensionality reduction. Although dimensionality reduction is useful, it comes at a cost. Information loss is a necessary part of PCA. Balancing the trade-off between dimensionality reduction and information loss is unfortunately a necessary compromise that we have to make when using PCA.***","metadata":{}}]}