{"cells":[{"metadata":{"_uuid":"b04513612c5b20286b615161e5e82b03e9ae04f5"},"cell_type":"markdown","source":"DATA MINING PROJECT\n\n"},{"metadata":{"_uuid":"f18c2e61a5301d5da4fa1653a2da2b430ccbe76d"},"cell_type":"markdown","source":"\nIn this competition, the goal is to correctly identify digits from a dataset of tens of thousands of handwritten images.\n\nFor this, we used different data mining operations."},{"metadata":{"_uuid":"e0325292077a54bdac20bf1cd08c60de884414de"},"cell_type":"markdown","source":"First we import the necessary packages:"},{"metadata":{"trusted":true,"_uuid":"b0b96624fd169e91151e1b71dedab215048baeae"},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nfrom sklearn.metrics import accuracy_score\nfrom sklearn.decomposition import PCA\nfrom sklearn.svm import SVC\nfrom sklearn.model_selection import cross_val_score#accuracy ölçmek için\nfrom sklearn.naive_bayes import GaussianNB#one of our classifiers\nfrom sklearn.neural_network import MLPClassifier#neural network\nfrom sklearn.model_selection import train_test_split#datayı bölmek için\nfrom sklearn.ensemble import RandomForestClassifier#another classifier\nfrom sklearn.preprocessing import StandardScaler#datayı normalize etmek için(neural network daha iyi çalışır)\nimport matplotlib.pyplot as plt#grafik çizdirmek için\nfrom sklearn import tree#ANOTHER classifier\nimport seaborn as sns#grafik çizdirmek için\nfrom sklearn.decomposition import PCA#dimension reduction için(kullanmadık)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"dbc8ef9cb730c8d471d4df90ffed478953d0beed"},"cell_type":"markdown","source":"Then we assign the dataset to a variable:"},{"metadata":{"trusted":true,"_uuid":"8939a197b41246719ba2dfaf56f29214982d2c4c"},"cell_type":"code","source":"data=pd.read_csv('../input/train.csv')#train datası\ntest=pd.read_csv('../input/test.csv')#test datası\ny=data.label#etiketler\nx=data.drop('label',axis=1)#label kısmı drop edildi\nx_test=test#test datası","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"433ee28897dc840cc99a4ec16ef4804377b6908a"},"cell_type":"markdown","source":"At this part we are checking dataset for null and missing values:(KDD process; selection)"},{"metadata":{"trusted":true,"_uuid":"bcbc411098355ef316febb3e296ead47cf26666b"},"cell_type":"code","source":"print(data.isnull().any().describe())\nprint(test.isnull().any().describe())","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7b8b0926a829b13a1a35fa1f9080dfc9a787c068"},"cell_type":"markdown","source":"graph to see the distribution of data:(Data Visualisation)"},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"a935cf105cca0e33e5f05c02db28f04ccf135e07"},"cell_type":"code","source":"g=sns.countplot(y)\nplt.show(g)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"66910561058c90821ed6b15e345ad89519852936"},"cell_type":"markdown","source":"PCA is used to eliminate unnecessary components."},{"metadata":{"trusted":true,"_uuid":"9d80baa0048319ebc9bd85f63abc51c5d1706e9f"},"cell_type":"code","source":"pca=PCA(n_components=2, whiten=True)\npca.fit(x)\nX_pca=pca.transform(x)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1a56640c30af14a6a143a5db0a053d42ca28f48a"},"cell_type":"markdown","source":"This picture represents the reflection of numbers onto 2 dimensional area"},{"metadata":{"trusted":true,"scrolled":false,"_uuid":"734331cc017efe941645d261ce066484c1124f1e"},"cell_type":"code","source":"plt.figure(1, figsize=(12,8))\nplt.scatter(X_pca[:,0], X_pca[:,1], c=y, s=10,cmap=plt.get_cmap('jet',10))\nplt.colorbar()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b36250f3c5cd16a7332cd454260410380aca11d2"},"cell_type":"code","source":"pca=PCA()\npca.fit(x)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6cbc81ef36fe4f85ea3e94c48976c09650a75927"},"cell_type":"markdown","source":"The plot below shows that more than 35 components is unnecessary, we can eliminate them"},{"metadata":{"trusted":true,"_uuid":"6720506180f5992c09b3c239f8e701aa862a9fe5"},"cell_type":"code","source":"plt.figure(1,figsize=(12,8))\nplt.xticks(np.arange(0, 800, 30.0))\nplt.plot(pca.explained_variance_,linewidth=2)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ad25ec1fdcf41d7d26c90163160dae73a9a7b1c4"},"cell_type":"markdown","source":" splitting dataset for validation, then apply datasete normalization:(KDD ;prepocessing)"},{"metadata":{"trusted":true,"_uuid":"cfb17a3a29d62b28131cf96b543abc3e487a67a7"},"cell_type":"code","source":"x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.3, random_state=42)\nscaler = StandardScaler()\nscaler.fit(x_train)\nscaler.fit(x_test)\n\nx_train = scaler.transform(x_train)\nx_test = scaler.transform(x_test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1ddbfc11d89b1f17ba8c598c6d3c1698d1fca527"},"cell_type":"code","source":"\nn_components=35\npca=PCA(n_components=n_components, whiten=True)\npca.fit(x_train)\nX_train_pca=pd.DataFrame(pca.transform(x_train))\nX_test_pca=pd.DataFrame(pca.transform(x_test))\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"72518f0da25d05745ea866ccf2e84d8223739d43"},"cell_type":"markdown","source":"PCA transformation is done"},{"metadata":{"_uuid":"1e1dc9141abd5480f823fa83fc83dd0af50c07f2"},"cell_type":"markdown","source":"Preparing the files for the competition"},{"metadata":{"trusted":true,"_uuid":"c2696a7d6d9dc86b38b4edb2a51925c0ec34c5ea"},"cell_type":"code","source":"clf=SVC(C=10, gamma=0.01, kernel=\"rbf\")\nclf.fit(X_train_pca, y_train)\ncompetion_dataset=pd.read_csv(\"../input/test.csv\",dtype=\"uint8\")\ncompetion_dataset=competion_dataset.values\ncompetion_dataset=competion_dataset[:,0:]\ncompetion_dataset_pca=pca.transform(competion_dataset)\ny_pred2=clf.predict(competion_dataset_pca)\nfile_name=\"x_pca_{}_svc_mnist.csv\".format(n_components)\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9b264e60a5a4481be5d246b381e59b7ffff5badc"},"cell_type":"markdown","source":"At this part we defined other classifiers to use:"},{"metadata":{"trusted":true,"_uuid":"405e89a279bcc4fd0d7e0eea248da996ad504671"},"cell_type":"code","source":"model = MLPClassifier(solver='lbfgs', activation='logistic', learning_rate='adaptive')\nclf = tree.DecisionTreeClassifier()\nmodel1 = RandomForestClassifier(random_state=43)\ngnb = GaussianNB()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6fb3e452d3d5295457bc46e9349ed07f568a799e"},"cell_type":"markdown","source":"We trained our model by usuing train data:"},{"metadata":{"trusted":true,"_uuid":"6820b2a3207bdf435a588d81873875e44e1e72ca"},"cell_type":"code","source":"model.fit(X_train_pca, y_train)\nmodel1.fit(X_train_pca, y_train)\nclf.fit(X_train_pca, y_train)\ngnb.fit(X_train_pca, y_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c3944a3d5f766b138548e9bd5727d01e977d377e"},"cell_type":"code","source":"y_pred1=clf.predict(X_test_pca)\ny_pred2=model.predict(X_test_pca)\ny_pred3=model1.predict(X_test_pca)\ny_pred4=clf.predict(X_test_pca)\ny_pred5=gnb.predict(X_test_pca)\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8025b23ab72fc7699f2df0f3368064cf6d694de6"},"cell_type":"markdown","source":"We writed to score of cross validatiion:(KDD process; evaluation)"},{"metadata":{"trusted":true,"_uuid":"3ad42048b7fd5942114edbdc067a36c4884f1969"},"cell_type":"code","source":"print('Accuracy for support vector machines is: ',(accuracy_score(y_pred1, y_test)))\nprint('neural network accuracy:',accuracy_score(y_pred2, y_test))\nprint('random forest:' ,accuracy_score(y_pred3, y_test))\nprint('decision tree:' ,accuracy_score(y_pred4, y_test))\nprint('gaussian naive bayes:' ,accuracy_score(y_pred5, y_test))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"cd598ea317ea349135a2d92d16005ca57e01f917"},"cell_type":"code","source":"results = pd.Series(y_pred1,name=\"Label\")\nsubmission = pd.concat([pd.Series(range(1,28001),name = \"ImageId\"),results],axis = 1)\n#y_pred2=y_pred2.astype(int)\nsubmission.to_csv(file_name,index=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"863ae4b07b0c97394223a689d8c5410d1e7d16a7"},"cell_type":"markdown","source":"We took the best accuracy in Neural Network algorithm ."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"\n\nResult:\n    In this competition, our goal is to correctly identify digits from a dataset of tens of thousands of handwritten images. We used different classifiers and compare their results.\n    We searched convenient algorithms for CSV data and observational data, before we decided to Classification Algorithms of our digits and handwritten images recognizer. Then, we test four different algorisms by using our test data and then we take result from our programming codes.\n    As you can see in above, neural network accuracy with percentage of 0,944 and it gives us to best result at train data. By the way, with the best result of classification we applied neural network algorithm. We finished our procet and we committed to Kaggle.\n\n\n\n\n\n\n\n\n\n\n\n"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}