{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"###############################################################\n# NB: shift + tab HOLD FOR 2 SECONDS!\n###############################################################\n\n\n\n# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.\nprint('Getting traing dataset...')\ntrain_data = pd.read_csv('../input/digit-recognizer/train.csv')\nprint('Traing data set obtained \\n')\n\nprint('Getting test dataset...')\ntest_data = pd.read_csv('../input/digit-recognizer/test.csv')\nprint('Test data set obtained \\n')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Each image is 28 pixels in height and 28 pixels in width, for a total of 784 pixels in total. Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of that pixel, with higher numbers meaning darker. This pixel-value is an integer between 0 and 255, inclusive.\n\nThe training data set, (train.csv), has 785 columns. **The first column, called \"label\", is the digit that was drawn by the user**. The rest of the columns contain the pixel-values of the associated image.\n\nEach pixel column in the training set has a name like pixelx, where x is an integer between 0 and 783, inclusive. To locate this pixel on the image, suppose that we have decomposed x as x = i * 28 + j, where i and j are integers between 0 and 27, inclusive. Then pixelx is located on row i and column j of a 28 x 28 matrix, (indexing by zero).\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_img = train_data.drop('label', axis=1)\ntrain_label = train_data['label'].values\ntrain_img","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**RECOVERING IMAGES**\n\nNow we try to print a single image. In order to do so, we need to convert the pandas dataframe into a matrix, i.e. an ordered numpy array. "},{"metadata":{"trusted":true},"cell_type":"code","source":"img_try = train_img.iloc[30].values\nimg_try = np.reshape(img_try, (28, 28))\n\nfig = plt.figure(figsize=(6, 6))\nplt.imshow(img_try)\nplt.title(train_label[30], fontsize=30)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can now define a function that does the same! \n\nIt will eat the dataframe, the label and an integer number that will be the selected row"},{"metadata":{"trusted":true},"cell_type":"code","source":"# The first function return the matrix out from the dataframe\ndef return_matrix(train, n=0):\n    import numpy as np # linear algebra\n    import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n    import matplotlib.pyplot as plt\n    \n    if ((n>=0) & (n<=783)) :\n        img_try = train.iloc[n].values\n        img_try = np.reshape(img_try, (28, 28))\n    else :\n        print('Insert a n between 0 and 783')\n        pass \n    return img_try\n\n# The second function eats both train and test dataframe plus an integer number between 0 and 783 (28x28 =784)\ndef print_matrix(train, test, n=0):\n    import numpy as np # linear algebra\n    import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n    import matplotlib.pyplot as plt\n    \n    if ((n>=0) & (n<=783)) :\n        mat = return_matrix(train, n)\n    \n        fig = plt.figure(figsize=(5, 5))\n        plt.imshow(mat)\n        plt.title(train_label[n], fontsize=30)\n    else : \n        print('Insert a n between 0 and 783')\n        pass\n    \n    return print('Done')\n        ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print_matrix(train_img, train_label, 80)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**DECISION TREE CLASSIFIER**\n\nWe may now start with a CLASSICAL Machine Learning algorithm such a **Decision Tree** classifier. \n\nRecall that now we have a set of features as a pd.dataframe saved as \n> train_img\n\nand a set of associated labels saved as\n> train_label\n\nWe start using the **return_matrix** function we have defined to transform the pd.dataframe into a np.array. NB: it has to be a flat array![](http://)\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"features = []\nlabels = []\n\nprint('Transforming the data... \\n')\n\nfor i in np.arange(0, train_label.size):\n    features.append(train_img.iloc[i].values)\n    labels.append(train_label[i])\n\nprint('Done')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(np.array(features).shape)\nprint(np.array(labels).shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We now need to split the Datasat into Train and Test to perform a supervised machine learning algorithm such as DecisionTreeClassifiare or RandomForestClassifier from scikit-learn"},{"metadata":{"trusted":true},"cell_type":"code","source":"# split into training and testing sets\nfrom sklearn.model_selection import train_test_split\n\nX_train, X_test, Y_train, Y_test = train_test_split(np.array(features), np.array(labels), test_size=0.30)\n\nprint('Training records:',Y_train.size)\nprint('Test records:',Y_test.size)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we import sklearn to create a *pipeline* that \n1. Scales the image using the **MinMaxScaler**\n2. Create a **DecisionTreeClassifier**"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.pipeline import Pipeline\nfrom sklearn.preprocessing import MinMaxScaler\nfrom sklearn.tree import DecisionTreeClassifier\n\n# Convert the training features to floats so they can be scaled\nX_train_float = X_train.astype('float64')\n\n# Our pipeline performs two tasks:\n#   1. Normalize the image arrays\n#   2. Train a classification model - Decision Tree\nimg_pipeline = Pipeline([('norm', MinMaxScaler()),\n                         ('classify', DecisionTreeClassifier()),\n                        ])\n\n# Use the pipeline to fit a model to the training data\nprint(\"Training model...\")\nclf = img_pipeline.fit(X_train_float, Y_train)\n\nprint('classifier trained!')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We now need to evauete the model; we will plot a confusion matrix"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Evaluate classifier\nfrom sklearn import metrics\nfrom sklearn.metrics import accuracy_score, confusion_matrix\nimport numpy as np\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\nclassnames = '0 1 2 3 4 5 6 7 8 9'.split()\n\n# Convert the test features for scaling\nX_test_float = X_test.astype('float64')\n\nprint('Compute predictions: \\n')\npredictions = clf.predict(X_test)\n\nprint('Classifier Metrics:')\nprint(metrics.classification_report(Y_test, predictions, target_names=classnames))\nprint('Accuracy: {:.2%}'.format(metrics.accuracy_score(Y_test, predictions)))\n\nprint(\"\\n Confusion Matrix:\")\ncm = confusion_matrix(Y_test, np.round(predictions, 0))\n\n# Plot confusion matrix as heatmap\nplt.imshow(cm, interpolation=\"nearest\", cmap=plt.cm.Blues)\nplt.colorbar()\ntick_marks = np.arange(len(classnames))\nplt.xticks(tick_marks, classnames, rotation=85)\nplt.yticks(tick_marks, classnames)\nplt.xlabel(\"Predicted Label\")\nplt.ylabel(\"True Label\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we need to do the same for the **TEST** values\n\nThose are objects that have to be classified by the ML algorithm. "},{"metadata":{"trusted":true},"cell_type":"code","source":"test_img = test_data\nX_value = []\n\nprint('Transforming the data... \\n')\n\nfor i in np.arange(0, test_img.shape[0]):\n    X_value.append(test_img.iloc[i].values)\n\nprint('Done')\n\nprint(np.array(X_value).shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**PREDICTIONS**\n\nWe now **predict** what the images are; we define a simple function that eats the classifier and the image and returns the preiction"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Function to predict the class of an image\ndef predict_image(classifier, image_array):\n    import numpy as np\n    \n    # These are the classes our model can predict\n    classnames = '0 1 2 3 4 5 6 7 8 9'.split()\n    \n    # Predict the class of each input image\n    predictions = classifier.predict(image_array)\n    \n    predicted_classes = []\n    for prediction in predictions:\n        # And append the corresponding class name to the results\n        predicted_classes.append(classnames[int(prediction)])\n    # Return the predictions\n    return predicted_classes","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"classnames = '0 1 2 3 4 5 6 7 8 9'.split()\n\nprint('Compute predictions: \\n')\ntrue_predictions = predict_image(clf, np.array(X_value))\nprint('End.')\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We now define a function that, gives the images, the prediction and an integer number between 0 and 27999 (dimension of the unkwnow images dataset) returns the plot of the image with the predicted label as a title"},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_pred(img, pred, n=0):\n    import matplotlib.pyplot as plt\n    import numpy as np\n    \n    if ((n>=0) & (n<np.array(X_value).shape[0])):\n        imag = np.reshape(np.array(img[n]), (28, 28))\n        fig = plt.figure(figsize=(5, 5))\n        plt.imshow(imag)\n        plt.title(pred[n], fontsize=30)\n    else: \n        print('Wrong n \\n')\n        pass\n    \n    return print('Done')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_pred(X_value, true_predictions, 30)\nplot_pred(X_value, true_predictions, 99)\nplot_pred(X_value, true_predictions, 1200)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now, we would like to define a SINGLE FUNCTION that eats the training dataset and spits out the trained classifier, doing all the manipulation internally. \n\nThis means that one can run only the function after having received the dataset. "},{"metadata":{"trusted":true},"cell_type":"code","source":"def decision_tree_classifier_MNIST(train_data, testsize=0.30, criterio='gini'):\n    import numpy as np # linear algebra\n    import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n    import matplotlib.pyplot as plt\n    from sklearn import metrics\n    from sklearn.metrics import accuracy_score, confusion_matrix\n    from sklearn.model_selection import train_test_split\n    \n    # First, we need to transform the data\n    train_img = train_data.drop('label', axis=1)\n    train_label = train_data['label'].values\n    \n    features = []\n    labels = []\n\n    print('Transforming the data...')\n    for i in np.arange(0, train_label.size):\n        features.append(train_img.iloc[i].values)\n        labels.append(train_label[i])\n    print('Transformation done \\n')\n    \n    \n    # Second, we need to split the dataset into training and test subsets\n    print('Splitting the dataset... ')\n    X_train, X_test, Y_train, Y_test = train_test_split(np.array(features), np.array(labels), test_size=testsize)\n    print('Splitting done \\n')\n    \n    # Third, we need to define the classifier\n    from sklearn.pipeline import Pipeline\n    from sklearn.preprocessing import MinMaxScaler\n    from sklearn.tree import DecisionTreeClassifier\n    \n    if (criterio != 'gini') :\n        crit='entropy'\n    else :\n        crit ='gini'\n        \n        # Convert the training features to floats so they can be scaled\n    X_train_float = X_train.astype('float64')\n\n        # Our pipeline performs two tasks:\n        #   1. Normalize the image arrays\n        #   2. Train a classification model - Decision Tree\n    img_pipeline = Pipeline([('norm', MinMaxScaler()),\n                         ('classify', DecisionTreeClassifier(criterion = crit)),\n                        ])\n\n    # Fourth, we train the model\n    print(\"Training model...\")\n    clf = img_pipeline.fit(X_train_float, Y_train)\n    print('Training done. \\n')\n    \n    \n    # Finally, we evaluate the trained model\n    print('Reporting the model...')\n    classnames = '0 1 2 3 4 5 6 7 8 9'.split()\n    print('Compute predictions: \\n')\n    predictions = clf.predict(X_test)\n\n    print('Classifier Metrics:')\n    print(metrics.classification_report(Y_test, predictions, target_names=classnames))\n    print('Accuracy: {:.2%}'.format(metrics.accuracy_score(Y_test, predictions)))\n\n    print(\"\\n Confusion Matrix:\")\n    cm = confusion_matrix(Y_test, np.round(predictions, 0))\n\n        # Plot confusion matrix as heatmap\n    plt.imshow(cm, interpolation=\"nearest\", cmap=plt.cm.Blues)\n    plt.colorbar()\n    tick_marks = np.arange(len(classnames))\n    plt.xticks(tick_marks, classnames, rotation=85)\n    plt.yticks(tick_marks, classnames)\n    plt.xlabel(\"Predicted Label\")\n    plt.ylabel(\"True Label\")\n    plt.show()\n    \n    \n    print('Process ended succesfully!')\n    return clf","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"clf = decision_tree_classifier_MNIST(train_data)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We thus can use our classifier to make prediction:"},{"metadata":{"trusted":true},"cell_type":"code","source":"def predict_image(classifier, image_array):\n    import numpy as np\n    \n    # These are the classes our model can predict\n    classnames = '0 1 2 3 4 5 6 7 8 9'.split()\n    \n    # Predict the class of each input image\n    predictions = classifier.predict(image_array)\n    \n    predicted_classes = []\n    for prediction in predictions:\n        # And append the corresponding class name to the results\n        predicted_classes.append(classnames[int(prediction)])\n    # Return the predictions\n    return predicted_classes\n\ndef evaluate_plot_pred(clf, test_data, n=0):\n    import numpy as np # linear algebra\n    import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n    import matplotlib.pyplot as plt\n    \n    # First need to transform the data\n    test_img = test_data\n    X_value = []\n\n    print('Transforming the data... \\n')\n    for i in np.arange(0, test_img.shape[0]):\n        X_value.append(test_img.iloc[i].values)\n\n    print('Done')\n    \n    \n    # Second, we compute the predictions\n    classnames = '0 1 2 3 4 5 6 7 8 9'.split()\n\n    print('Compute predictions: \\n')\n    true_predictions = predict_image(clf, np.array(X_value)) #notice that we call the function we defined before\n    print('Predictions done.')\n    \n    # Finally, we plot\n    if ((n>=0) & (n<np.array(X_value).shape[0])):\n        imag = np.reshape(np.array(X_value[n]), (28, 28))\n        fig = plt.figure(figsize=(5, 5))\n        plt.imshow(imag)\n        plt.title(true_predictions[n], fontsize=30)\n    else: \n        print('Wrong n \\n')\n        pass\n    \n    print('Done. ')\n    \n    return true_predictions\n    ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"true_predictions = evaluate_plot_pred(clf, test_data, 1200)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}