{"metadata":{"kaggle":{"accelerator":"none","dataSources":[{"sourceId":3004,"databundleVersionId":861823,"sourceType":"competition"}],"dockerImageVersionId":30761,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.12.3"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"import numpy as np\nimport pandas as pd \nimport tensorflow as tf\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Dense\nfrom tensorflow.keras.activations import linear, relu\n\nfrom keras.layers import Dropout\nimport matplotlib.pyplot as plt\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import classification_report","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","execution":{"iopub.execute_input":"2024-08-31T03:57:58.132056Z","iopub.status.busy":"2024-08-31T03:57:58.131494Z","iopub.status.idle":"2024-08-31T03:58:11.894865Z","shell.execute_reply":"2024-08-31T03:58:11.893186Z","shell.execute_reply.started":"2024-08-31T03:57:58.132007Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train = pd.read_csv('/kaggle/input/digit-recognizer/train.csv')","metadata":{"execution":{"iopub.execute_input":"2024-08-31T03:58:11.898838Z","iopub.status.busy":"2024-08-31T03:58:11.897807Z","iopub.status.idle":"2024-08-31T03:58:16.261884Z","shell.execute_reply":"2024-08-31T03:58:16.260522Z","shell.execute_reply.started":"2024-08-31T03:58:11.898769Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.shape","metadata":{"execution":{"iopub.execute_input":"2024-08-31T03:58:16.265251Z","iopub.status.busy":"2024-08-31T03:58:16.263999Z","iopub.status.idle":"2024-08-31T03:58:16.274672Z","shell.execute_reply":"2024-08-31T03:58:16.273183Z","shell.execute_reply.started":"2024-08-31T03:58:16.265183Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.head()","metadata":{"execution":{"iopub.execute_input":"2024-08-31T03:58:16.278179Z","iopub.status.busy":"2024-08-31T03:58:16.277625Z","iopub.status.idle":"2024-08-31T03:58:16.317752Z","shell.execute_reply":"2024-08-31T03:58:16.316258Z","shell.execute_reply.started":"2024-08-31T03:58:16.278110Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X = train.drop('label' , axis=1).to_numpy()\ny = train.label.to_numpy().reshape(-1,1)","metadata":{"execution":{"iopub.execute_input":"2024-08-31T03:58:16.320271Z","iopub.status.busy":"2024-08-31T03:58:16.319670Z","iopub.status.idle":"2024-08-31T03:58:16.421628Z","shell.execute_reply":"2024-08-31T03:58:16.419974Z","shell.execute_reply.started":"2024-08-31T03:58:16.320209Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Visualizing the Data","metadata":{}},{"cell_type":"code","source":"def plot_random_images(x, y, nrows=8, ncols=8, figsize=(5, 5), title=\"\", cmap='gray'):\n    \"\"\"\n    Plots a grid of randomly selected images from the test set with their corresponding labels.\n\n    Parameters:\n    - x_test: The test images (numpy array).\n    - y_test: The test labels (numpy array).\n    - nrows: Number of rows in the grid (default is 8).\n    - ncols: Number of columns in the grid (default is 8).\n    - figsize: Size of the figure (default is (5, 5)).\n    - title: The title of the figure (default is \"Prediction, image\").\n    - cmap: The color map to use for displaying images (default is 'gray').\n    \"\"\"\n    m, n = x.shape\n    fig, axes = plt.subplots(nrows, ncols, figsize=figsize)\n    fig.tight_layout(pad=0.13, rect=[0, 0.03, 1, 0.91])\n\n    for i, ax in enumerate(axes.flat):\n        # Select a random index\n        random_index = np.random.randint(m)\n\n        # Reshape the image\n        X_random_reshaped = x[random_index].reshape((28, 28))\n\n        # Display the image\n        ax.imshow(X_random_reshaped, cmap=cmap)\n\n        # Display the label above the image\n        ax.set_title(y[random_index])\n        ax.set_axis_off()\n\n    # Add a title to the entire figure\n    fig.suptitle(title, fontsize=14)\n    plt.show()\n","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_random_images(X, y,title=\"Label, image\")","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Split Data Set ","metadata":{}},{"cell_type":"code","source":"X_train , X_cv, Y_train,  Y_cv   = train_test_split(X, y, test_size= .2 , random_state=40)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Preprocess the data","metadata":{}},{"cell_type":"code","source":"X_train = X_train.astype('float32') / 255\nX_cv = X_cv.astype('float32') / 255","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train.shape","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Y_train.shape","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Tensorflow Model Implementation","metadata":{}},{"cell_type":"code","source":"tf.random.set_seed(1234) \nmodel = Sequential(\n    [               \n        tf.keras.Input(shape=(784,)),\n        Dense(512,activation='relu',name=\"L1\"), Dropout(0.7),\n        Dense(512,activation='relu',name='L2'),Dropout(0.5),\n        Dense(10,activation=\"linear\",name=\"L3\"),\n        \n    ], name = \"my_model\" \n)","metadata":{"execution":{"iopub.execute_input":"2024-08-31T03:58:23.752455Z","iopub.status.busy":"2024-08-31T03:58:23.752039Z","iopub.status.idle":"2024-08-31T03:58:23.864060Z","shell.execute_reply":"2024-08-31T03:58:23.862385Z","shell.execute_reply.started":"2024-08-31T03:58:23.752411Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.summary()","metadata":{"execution":{"iopub.execute_input":"2024-08-31T03:58:23.866363Z","iopub.status.busy":"2024-08-31T03:58:23.865916Z","iopub.status.idle":"2024-08-31T03:58:23.895226Z","shell.execute_reply":"2024-08-31T03:58:23.893660Z","shell.execute_reply.started":"2024-08-31T03:58:23.866317Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.compile(\n    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n    optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),\n    metrics=['accuracy'],\n    \n)","metadata":{"execution":{"iopub.execute_input":"2024-08-31T03:58:23.897344Z","iopub.status.busy":"2024-08-31T03:58:23.896944Z","iopub.status.idle":"2024-08-31T03:58:23.919275Z","shell.execute_reply":"2024-08-31T03:58:23.917921Z","shell.execute_reply.started":"2024-08-31T03:58:23.897303Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model.fit(\n    X_train,Y_train,\n    batch_size=128,\n    epochs=20,\n    validation_data=(X_cv, Y_cv),\n    verbose=1\n)","metadata":{"execution":{"iopub.execute_input":"2024-08-31T03:58:23.923827Z","iopub.status.busy":"2024-08-31T03:58:23.923362Z","iopub.status.idle":"2024-08-31T04:00:17.135438Z","shell.execute_reply":"2024-08-31T04:00:17.134208Z","shell.execute_reply.started":"2024-08-31T03:58:23.923780Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"prediction = model.predict(X_cv)","metadata":{"execution":{"iopub.execute_input":"2024-08-31T04:00:17.137948Z","iopub.status.busy":"2024-08-31T04:00:17.137392Z","iopub.status.idle":"2024-08-31T04:00:20.868166Z","shell.execute_reply":"2024-08-31T04:00:20.866719Z","shell.execute_reply.started":"2024-08-31T04:00:17.137886Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"prediction_p = tf.nn.softmax(prediction)","metadata":{"execution":{"iopub.execute_input":"2024-08-31T04:00:20.870566Z","iopub.status.busy":"2024-08-31T04:00:20.870076Z","iopub.status.idle":"2024-08-31T04:00:20.882338Z","shell.execute_reply":"2024-08-31T04:00:20.880845Z","shell.execute_reply.started":"2024-08-31T04:00:20.870520Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"prediction_p","metadata":{"execution":{"iopub.execute_input":"2024-08-31T04:00:20.884518Z","iopub.status.busy":"2024-08-31T04:00:20.884036Z","iopub.status.idle":"2024-08-31T04:00:20.895838Z","shell.execute_reply":"2024-08-31T04:00:20.894442Z","shell.execute_reply.started":"2024-08-31T04:00:20.884473Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"yhat = np.argmax(prediction_p, axis=1)","metadata":{"execution":{"iopub.execute_input":"2024-08-31T04:00:20.898314Z","iopub.status.busy":"2024-08-31T04:00:20.897720Z","iopub.status.idle":"2024-08-31T04:00:20.911604Z","shell.execute_reply":"2024-08-31T04:00:20.910013Z","shell.execute_reply.started":"2024-08-31T04:00:20.898249Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"yhat","metadata":{"execution":{"iopub.execute_input":"2024-08-31T04:00:20.914684Z","iopub.status.busy":"2024-08-31T04:00:20.914221Z","iopub.status.idle":"2024-08-31T04:00:20.925018Z","shell.execute_reply":"2024-08-31T04:00:20.923605Z","shell.execute_reply.started":"2024-08-31T04:00:20.914640Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's compare the predictions vs the labels for a random sample of 64 digits.","metadata":{}},{"cell_type":"code","source":"import warnings\nwarnings.simplefilter(action='ignore', category=FutureWarning)\ndef display_random_images_with_predictions(X, y, model, grid_size=(8, 8), img_shape=(28, 28), figsize=(5, 5)):\n    \"\"\"\n    Displays a grid of random images with their true labels and predicted labels.\n\n    Parameters:\n    - X: np.array, 2D array containing the images as flattened vectors.\n    - y: np.array, 2D array containing the true labels.\n    - model: a trained neural network model for making predictions.\n    - grid_size: tuple, size of the grid to display (rows, columns).\n    - img_shape: tuple, the shape of the images to display.\n    - figsize: tuple, the size of the figure.\n    \"\"\"\n    m, n = X.shape\n    fig, axes = plt.subplots(*grid_size, figsize=figsize)\n    fig.tight_layout(pad=0.13, rect=[0, 0.03, 1, 0.91])\n\n    for i, ax in enumerate(axes.flat):\n        # Select random indices\n        random_index = np.random.randint(m)\n\n        # Select rows corresponding to the random indices and reshape the image\n        X_random_reshaped = X[random_index].reshape(img_shape)\n\n        # Display the image\n        ax.imshow(X_random_reshaped, cmap='gray')\n\n        # Predict using the Neural Network\n        prediction = model.predict(X[random_index].reshape(1, n))\n        prediction_p = tf.nn.softmax(prediction)\n        yhat = np.argmax(prediction_p)\n\n        # Display the label above the image\n        ax.set_title(f\"{y[random_index, 0]},{yhat}\", fontsize=10)\n        ax.set_axis_off()\n\n    fig.suptitle(\"Label, yhat\", fontsize=14)\n    plt.show()\n","metadata":{"execution":{"iopub.execute_input":"2024-08-31T04:00:20.927520Z","iopub.status.busy":"2024-08-31T04:00:20.927029Z","iopub.status.idle":"2024-08-31T04:00:20.940870Z","shell.execute_reply":"2024-08-31T04:00:20.939514Z","shell.execute_reply.started":"2024-08-31T04:00:20.927471Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display_random_images_with_predictions(X_cv,Y_cv,model)","metadata":{"execution":{"iopub.execute_input":"2024-08-31T04:00:20.943306Z","iopub.status.busy":"2024-08-31T04:00:20.942829Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(classification_report(Y_cv,yhat))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Great 😋","metadata":{}},{"cell_type":"markdown","source":"Let's look at the misclassified images looks.","metadata":{}},{"cell_type":"code","source":"fig = plt.figure(figsize=(1, 1))\nerrors = np.where(Y_cv != yhat.reshape(-1,1))\nrandom_index = errors[0][0]\nX_random_reshaped = X_cv[random_index].reshape((28, 28))\nplt.imshow(X_random_reshaped, cmap='gray')\nplt.title(f\"{y[random_index]}, {yhat[random_index]}\")\nplt.axis('off')\nplt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Test Data","metadata":{}},{"cell_type":"code","source":"test = pd.read_csv('/kaggle/input/digit-recognizer/test.csv')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.shape","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test.head()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission = pd.DataFrame()  ","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission['ImageId'] = list(range(1, 28001))\nsubmission","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Preprocess the data","metadata":{}},{"cell_type":"code","source":"x_test = test.to_numpy().astype('float32') / 255\nx_test.shape","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"prediction_test = model.predict(x_test)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"prediction_p_test = tf.nn.softmax(prediction_test)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ytest = np.argmax(prediction_p_test, axis=1)","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Displaying a Random Selection of Test Images with Predictions 🤩","metadata":{}},{"cell_type":"code","source":"plot_random_images(x_test , ytest , title='Prediction, image')","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission['Label'] = ytest","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.tail()","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.to_csv('submission.csv' , index=False)","metadata":{},"execution_count":null,"outputs":[]}]}