{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nfrom sklearn.svm import SVC\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.metrics import accuracy_score\nfrom sklearn.model_selection import train_test_split\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Read sample_submission files\nsample_submission = pd.read_csv('/kaggle/input/digit-recognizer/sample_submission.csv')\nsample_submission.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Read train files\ntrain = pd.read_csv('/kaggle/input/digit-recognizer/train.csv')\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Show some images\nfor x in range(0, 20):\n    image = train.loc[x,train.columns != \"label\"]\n    plt.imshow(np.array(image).reshape((28, 28)), cmap=\"gray\")\n    plt.show()\n    \n    plt.hist(image)\n    plt.xlabel(\"Pixel Intensity\")\n    plt.ylabel(\"Counts\")\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Number of train images\nprint(\"Number of images: %d\" % len(train))\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**sklearn.model_selection.train_test_split(*arrays, **options)[source]**\n\n\nSplit arrays or matrices into random train and test subsets\n\nQuick utility that wraps input validation and next(ShuffleSplit().split(X, y)) and application to input data into a single call for splitting (and optionally subsampling) data in a oneliner.\n\n\n>>> X\narray([[0, 1],\n       [2, 3],\n       [4, 5],\n       [6, 7],\n       [8, 9]])\n\n\nX_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.33, random_state=42)\n\n>>> X_train\narray([[4, 5],\n       [0, 1],\n       [6, 7]])\n       \n       \n>>> y_train\n[2, 0, 3]\n\n\n>>> X_test\narray([[2, 3],\n       [8, 9]])\n       \n       \n>>> y_test\n[1, 4]"},{"metadata":{"trusted":true},"cell_type":"code","source":"# split the data\ntrain_images = train.loc[:, train.columns != \"label\"] / 255\ntrain_labels = train.label\n\n\n#Split arrays or matrices into random train and test subsets\n#Quick utility that wraps input validation and next(ShuffleSplit().split(X, y)) and application to input data into a single call for splitting (and optionally subsampling) data in a oneliner.\nx_train, x_test, y_train, y_test = train_test_split(train_images, train_labels, test_size=0.2, random_state=1)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print ('y_test ****************')\nprint(y_test)\nprint ('x_test ****************')\nprint(x_test)\nprint ('y_train ****************')\nprint(y_train)\nprint ('y_train ****************')\nprint(y_train)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**SVC / SVM **\n\nSupport vector machines (SVMs) are a set of supervised learning methods used for classification, regression and outliers detection.\n\nThe advantages of support vector machines are:\n\nEffective in high dimensional spaces.\nStill effective in cases where number of dimensions is greater than the number of samples.\nUses a subset of training points in the decision function (called support vectors), so it is also memory efficient.\nVersatile: different Kernel functions can be specified for the decision function. Common kernels are provided, but it is also possible to specify custom kernels.\nThe disadvantages of support vector machines include:\n\nIf the number of features is much greater than the number of samples, avoid over-fitting in choosing Kernel functions and regularization term is crucial.\nSVMs do not directly provide probability estimates, these are calculated using an expensive five-fold cross-validation (see Scores and probabilities, below).\nThe support vector machines in scikit-learn support both dense (numpy.ndarray and convertible to that by numpy.asarray) and sparse (any scipy.sparse) sample vectors as input. However, to use an SVM to make predictions for sparse data, it must have been fit on such data. For optimal performance, use C-ordered numpy.ndarray (dense) or scipy.sparse.csr_matrix (sparse) with dtype=float64.\n\n\nSVC, NuSVC and LinearSVC are classes capable of performing multi-class classification on a dataset."},{"metadata":{"trusted":true},"cell_type":"code","source":"# this takes about 20 minutes. accuray = 94.0\n\n#SVC classifier\nmodel = SVC()\nmodel.fit(x_train, y_train)\n\n# this takes about 5 min also\ntest_predicts = model.predict(x_test)\nprint(test_predicts)\n\nfrom sklearn.metrics import accuracy_score\ntest_acc = round(accuracy_score(y_test, test_predicts) * 100)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_acc","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**Random forests**\n\n\nRandom forests are an ensemble learning method that can be used for classification. It works by using a multitude of decision trees and it selects the class that is the most often predicted by the trees.\n\nA decision tree contains at each vertex a \"question\" and each descending edge is an \"answer\" to that question. The leaves of the tree are the possible outcomes. A decision tree can be built automatically from a training set.\n\nEach tree of the forest is created using a random sample of the original training set, and by considering only a subset of the features (typically the square root of the number of features). The number of trees is controlled by cross-validation."},{"metadata":{"trusted":true},"cell_type":"code","source":"# KNN - RandomForestClassifier\n# this takes about 10 minutes. accuray = 96.0\n\nmodelRandomForestClassifier = RandomForestClassifier(n_estimators=100)\nmodelRandomForestClassifier.fit(x_train, y_train)\ntestRandomForestClassifier_predicts = modelRandomForestClassifier.predict(x_test)\ntest_acc_RandomForestClassifier = round(accuracy_score(y_test, testRandomForestClassifier_predicts) * 100)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_acc_RandomForestClassifier","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"* 96% acc for RandomForestClassifier\n* 94% acc for SVC"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Read test files & apply model & write the output in a submission file\ntest_for_submission = pd.read_csv('/kaggle/input/digit-recognizer/test.csv')\ntest_for_submission.head()\n\ntest = test_for_submission.loc[:, :] / 255\nsubmit = modelRandomForestClassifier.predict(test)\npd.DataFrame(submit).to_csv('submit.csv', index=False) ","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}