{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"I am still at begginer level to learn data science, still i will try to explain the logic i have used to solve this problem. I have tried to keep it as simple as possible so that it is easy for other begginers as well. First we will import all the libraries we are going to need:"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nfrom sklearn import datasets\nfrom sklearn.model_selection import train_test_split\nfrom sklearn import svm\nfrom sklearn import metrics\nimport matplotlib.pyplot as plt, matplotlib.image as mpimg\nimport time\nprint('Loading Libraries')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9d2979397e06f03252c3a513c84bdc65701d306a"},"cell_type":"code","source":"ds=pd.read_csv('../input/train.csv')\nprint(\"Data shape of training data set is :\",ds.shape)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1c171f095777dcb6dde7a89a1ad61ef8f5920a98"},"cell_type":"markdown","source":"Lets have a look at columns of training dataset:"},{"metadata":{"trusted":true,"_uuid":"9918c19ab857c981f183ee80ce4d8920e42af0fd"},"cell_type":"code","source":"ds.columns","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"25faeeb8d4a6e6693143fbfbe73ccec755979ef9"},"cell_type":"markdown","source":"As we can see here that there are 785 columns, one is label which is the number written in image, the remaining 784 columns are pixel values. The \"pixel\" columns shows 28X28 image which is reshaped into a 1D array. So i am just going to plot one row from this data set. I have selected digit \"5\" for printing:"},{"metadata":{"trusted":true,"_uuid":"0c85b01463c7a68656034c7a2747d52ae6d5b149"},"cell_type":"code","source":"plot5=ds[ds.label==5]#it will pick all rows in data set which represent digit 5","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"38e738d13a031f9937be0c9f7ef917a9ff91bbda"},"cell_type":"code","source":"plot5=plot5.drop('label',axis=1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b5284c30e40474b008f5ad46606839ad9953c489"},"cell_type":"markdown","source":"Label column is dropped because we are going to pick any row from plot5 data set and then convert it into 28X28 2D array so that we can plot it."},{"metadata":{"trusted":true,"_uuid":"3890dde5254347c66d73aab386d1dffb4caf8200"},"cell_type":"code","source":"indx=plot5.index.max()#used this for row selection, i am just going to plot the data from last row and now i know its index value","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"bc540aea7a3fb684038fad015bc1bb3df9f5f4af"},"cell_type":"code","source":"ax5=plt.imshow(plot5.loc[indx].values.reshape(28,28),cmap='gray')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c59ad3a504a236447239f4077cebed4830f10f47"},"cell_type":"markdown","source":"Now i will split training data set based on columns, label column will be our new target data set and pixel columns will be our features data set:"},{"metadata":{"trusted":true,"_uuid":"d21e67aacba672d38822341e6689f34d3a630af7"},"cell_type":"code","source":"ds_noLabel=ds.drop('label',axis=1) #Features dataset\nds_label=ds['label']#Target dataset","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1004cd81413e4b3fd7f5f6a02633c0f8be7bd3f8"},"cell_type":"markdown","source":"Now i will use train_test_split funtion to divide my dataset into training and testing set. I am doing this because i am going to apply three classifiers on this and then i will compare results of each. Once best classifier is selected, i will then import test data set seperately and make my prediction:"},{"metadata":{"trusted":true,"_uuid":"eac238feaf53c6d9f950848b5966b71b9ddc61fe"},"cell_type":"code","source":"x_train,x_test,y_train,y_test=train_test_split(ds_noLabel,ds_label,train_size=0.7,random_state=23)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"780db5820b029b8e5ae1d579575ef7fefec665ab"},"cell_type":"markdown","source":"Now i will use SVM classifier to make my prediction. SVM stands for Support Vector Machine, it classifies by dividing the feature plane so that each sample belongs to a specific part in that plane. More detail can be read at this link : [https://medium.com/machine-learning-101/chapter-2-svm-support-vector-machine-theory-f0812effc72](http://).\n\nA classifier takes data as input and applies mathematical algorithims to make out a pattern. It applies linear algebra to construct the plane.  There can be different types of algorithms, in case of SVM the algorithms used can be Linear, Polynomial and RBF. \n\nLinear function is best when classes are seperable by a line in plane. But at times seperation is not linear so in that case we use polynomial function i.e. to identify non linear classification. Here is another detailed tutorial for a better understanding [https://blog.statsbot.co/support-vector-machines-tutorial-c1618e635e93](http://)\n\nSVM can be used for classification and regression, but this is a classification problem so we will use SVC classifier.\n\nData set is quite big, so i will just use 8000 samples from training dataset to fit model and make prediction:"},{"metadata":{"trusted":true,"_uuid":"3ec7ea74be1fdc56114a5d96c466d0b4e7302b8e"},"cell_type":"code","source":"#Using linear kernel\nsvcLin=svm.SVC(kernel='linear',random_state=42) #initiate the linear kernel\nsvcLin.fit(x_train[:8000],y_train[:8000])# fit the model on 8000 records\nscoreLin=svcLin.score(x_test,y_test) #check the score\n\n#Using RBF kernel \nsvcRBF=svm.SVC(random_state=42)#initiate the RBF kernel\nsvcRBF.fit(x_train[:8000],y_train[:8000])# fit the model on 8000 records\nscoreRBF=svcRBF.score(x_test,y_test)#check the score\n\nsvcPol=svm.SVC(kernel='poly',random_state=42,degree=2)#initiate the polynomial kernel with degree 2\nsvcPol.fit(x_train[:8000],y_train[:8000])# fit the model on 8000 records\nscorePol=svcPol.score(x_test,y_test)#check the score\n\nscore_df=pd.DataFrame({\"RBF Kernel score\" : scoreRBF,\"Linear kernel score\":scoreLin,\"Polynomial kernel degree 2\":scorePol},index=[\"Score\"])\n#score_df is data set created to compare the score of each kernel\nscore_df","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4454cfad735dad8d67df9ab2fa6120366a4390d8"},"cell_type":"markdown","source":"As we can see that polynomial kernel with degree 2 had best output, so now i am going to import the test data for this problem:"},{"metadata":{"trusted":true,"_uuid":"7835f90af3a0e066345eab82f180848a8941430e"},"cell_type":"code","source":"test_df=pd.read_csv('../input/test.csv')\ntest_df.columns","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"695cc35abdd48bb4dbbe3b45e826c37662e76a1a"},"cell_type":"code","source":"y_pred_final=svcPol.predict(test_df)#Making prediction","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0733c5b4fb3e59586b331182e3669be804280093"},"cell_type":"code","source":"sub_pred=pd.DataFrame({\"ImageId\":test_df.index+1,\"Label\":y_pred_final})\nsub_pred.to_csv('Sub.csv',index=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7ce0c096e7bfaf32704af9912dfa91a028c0763b"},"cell_type":"markdown","source":"So this concludes my tutorial, as I am a begginer so i would really appreciate  any comments or feedback from the community."},{"metadata":{"trusted":true,"_uuid":"736a2fed0ebffec7f1750a5d74c7e88acd1f3465"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}