{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"import os\nfrom IPython.display import Image\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n for filename in filenames:\n print(os.path.join(dirname, filename))","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-08-14T13:17:47.232589Z","iopub.execute_input":"2022-08-14T13:17:47.233125Z","iopub.status.idle":"2022-08-14T13:17:47.277826Z","shell.execute_reply.started":"2022-08-14T13:17:47.233026Z","shell.execute_reply":"2022-08-14T13:17:47.276428Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
\n\"Vectorized Implementation of Simple NN\" \n
\n
\n\n\n
\n
\n
\n*Animation is just for illustration. We are not using this architecture in the notebook\n
\n\n# Introduction\nHellooo everyone, we are together with the new notebook. In this notebook we will be doing vectorized implementation of neural network (NN).\n\n🚀Sooo let's get do it!\n\n## Libraries ⬇\n","metadata":{}},{"cell_type":"code","source":"import numpy as np #we gonna use it for arrays\nimport pandas as pd #implementing data and checking data\nimport matplotlib.pyplot as plt #data visualization\nimport warnings #just to show off some warning. Don't mind this package :)\nwarnings.filterwarnings('ignore')","metadata":{"execution":{"iopub.status.busy":"2023-10-05T17:58:48.985343Z","iopub.execute_input":"2023-10-05T17:58:48.985716Z","iopub.status.idle":"2023-10-05T17:58:49.021528Z","shell.execute_reply.started":"2023-10-05T17:58:48.985645Z","shell.execute_reply":"2023-10-05T17:58:49.020520Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Importing Data ⬇\n","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv('/kaggle/input/digit-recognizer/train.csv')","metadata":{"execution":{"iopub.status.busy":"2023-10-05T17:58:50.576620Z","iopub.execute_input":"2023-10-05T17:58:50.577145Z","iopub.status.idle":"2023-10-05T17:58:52.768635Z","shell.execute_reply.started":"2023-10-05T17:58:50.577108Z","shell.execute_reply":"2023-10-05T17:58:52.767355Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Let's check how our data looks like 🕵️♀️\n","metadata":{}},{"cell_type":"markdown","source":"As we can see from data set we have label colon at first which indicates what are the digits that written in the picture and pixels of pictures. We have **42000** examples (m) and for each example we have **784** pixels that represented as columns.","metadata":{}},{"cell_type":"code","source":"df.head()","metadata":{"execution":{"iopub.status.busy":"2023-10-05T17:58:53.894150Z","iopub.execute_input":"2023-10-05T17:58:53.894481Z","iopub.status.idle":"2023-10-05T17:58:53.919337Z","shell.execute_reply.started":"2023-10-05T17:58:53.894454Z","shell.execute_reply":"2023-10-05T17:58:53.918662Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.info()","metadata":{"execution":{"iopub.status.busy":"2023-10-05T17:58:55.834715Z","iopub.execute_input":"2023-10-05T17:58:55.835074Z","iopub.status.idle":"2023-10-05T17:58:55.878982Z","shell.execute_reply.started":"2023-10-05T17:58:55.835049Z","shell.execute_reply":"2023-10-05T17:58:55.877834Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Checking some examples and preparing our data for model ▶\n","metadata":{}},{"cell_type":"markdown","source":"First things first, we have to start to getting our data ready for model. In this section, we will split our data for **train** and **test** for X and Y. Before we split our data, we will shuffle it first. We just want to sure that our data is mixed well for unbiased results.","metadata":{}},{"cell_type":"code","source":"np.random.seed(123) #fixing randomization to get same results everytime when we use this code\ndata = np.array(df) #get our data into array from DataFrame\nnp.random.shuffle(data) #shuffling our data\n\nX = data[:,1:].T #taking the columns that we have pixels in it. In other words, all columns except labels\nY = data[:,0].reshape((len(df),1)) #assigning our labels to Y which is first column","metadata":{"execution":{"iopub.status.busy":"2023-10-05T17:58:58.022991Z","iopub.execute_input":"2023-10-05T17:58:58.023284Z","iopub.status.idle":"2023-10-05T17:58:58.388638Z","shell.execute_reply.started":"2023-10-05T17:58:58.023264Z","shell.execute_reply":"2023-10-05T17:58:58.387518Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In the code below, We gonna stack our pixels for each pictures horizontally. For example, first picture's pixel will be represented by first column in the **X_train** array. In this case, shape of X_train will be **(number of pixels, number of pictures)**. X_train will look like that ⬇\n
\n
\n
\n *$Px^{i}_{j}$ denotes i. picture j. pixel\n
","metadata":{}},{"cell_type":"code","source":"X_train = X[:,:round(.9*len(df))] # 90% of our data goes to X_train array\nX_train = X_train / 255. #normalizing our data\nX_test = X[:,round(.9*len(df)):] # 10% of our data goes to X_test array\nX_test = X_test / 255.\n\ntrain_labels = Y[:round(.9*len(df)),:] # 90% of our labels to train labels\ntest_labels = Y[round(.9*len(df)):,:] # 10% of our labels to test labels\n\nprint(f\"X_train's shape:{X_train.shape}\")","metadata":{"execution":{"iopub.status.busy":"2023-10-05T17:59:00.241223Z","iopub.execute_input":"2023-10-05T17:59:00.241563Z","iopub.status.idle":"2023-10-05T17:59:00.319364Z","shell.execute_reply.started":"2023-10-05T17:59:00.241536Z","shell.execute_reply":"2023-10-05T17:59:00.317934Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Alright, we have splitted our data for train and test. There is one last step that we have to do which is formatting our **Y_train** and **Y_test**. We gonna use the technique which is calling **\"one hot encoding\"**. We will stack possible outcomes to columns (0 to 9) and we will give **\"1\"** where our label represented, for everything else it will be **\"0\"**. In this case, one column will represent one pictures label. That means we will have **(10,number of examples)** shaped matrix. Our results after this step will look like that ⬇\n\n
\n
","metadata":{}},{"cell_type":"code","source":"Y_train = np.zeros((10,train_labels.shape[0]))\nY_train[np.squeeze(train_labels),np.arange(train_labels.shape[0])] = 1\n\nY_test = np.zeros((10,test_labels.shape[0]))\nY_test[np.squeeze(test_labels),np.arange(test_labels.shape[0])] = 1","metadata":{"execution":{"iopub.status.busy":"2023-10-05T18:01:56.318637Z","iopub.execute_input":"2023-10-05T18:01:56.318972Z","iopub.status.idle":"2023-10-05T18:01:56.325901Z","shell.execute_reply.started":"2023-10-05T18:01:56.318947Z","shell.execute_reply":"2023-10-05T18:01:56.324701Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def check_pictures(image):\n image = image.reshape((28, 28)) * 255\n plt.gray()\n plt.imshow(image, interpolation='nearest')\n plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-10-05T18:01:57.864447Z","iopub.execute_input":"2023-10-05T18:01:57.864781Z","iopub.status.idle":"2023-10-05T18:01:57.869753Z","shell.execute_reply.started":"2023-10-05T18:01:57.864752Z","shell.execute_reply":"2023-10-05T18:01:57.868925Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Here is the some examples that we have\ncheck_pictures(X_train[:,0])\ncheck_pictures(X_train[:,30000])","metadata":{"execution":{"iopub.status.busy":"2023-10-05T18:01:59.250652Z","iopub.execute_input":"2023-10-05T18:01:59.251013Z","iopub.status.idle":"2023-10-05T18:01:59.492340Z","shell.execute_reply.started":"2023-10-05T18:01:59.250982Z","shell.execute_reply":"2023-10-05T18:01:59.491633Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Let's start building our model step by step 🧱\n","metadata":{}},{"cell_type":"markdown","source":"YESS!! This is the time that you've been waiting for. In this section, we are starting to build our model. We are starting with initializing our parameters but before jumping into it, let's check how our model will look like and how neural network works overally. This will be the architacture we will implement ⬇\n
\n
\n
\n*$L_{i}$ denotes i. layer\n
\n\nIn the first layer (L1), we have our examples which donated as X. The X for our model will be X_train for train step and X_test for test step. You might see that they can denote examples (X) as **\"A0\"** (Activation zero) as well. We will have 5 nodes for L2 (Hidden Layer) and 10 nodes for L3 (output layer). We can change our hidden layer's number of nodes (node = the circles you see) but for output layer it must be 10 because the possible out comes just can be (0,1,2,3,4,5,6,7,8,9). Every node at the end will predict the possibility of being corresponding number. For example, first node in the output will predict the given pictures' possibility of being **'0'**. For second node, it will be possibility of being **'1'** and so on. \n\n**So how this circles will predict the posibilities?**\n\nOK, let's understand how NN (neural networks) works generally. We will have some parameters which will be **W** (weights) and **b** (bias). Just keep them in your mind for now. We gonna send our examples (for this notebook all our examples but it might be in batches as well) to our nodes. Every node will calculate some equations itself (we will show these equations later in the notebook). The calcuation results will go in to our activation function (it might be sigmoid, tanh, ReLU which we used in this notebook and Leaky ReLU etc.). So it gives the activations (A) which we will use as the input of next layers' nodes. Every nodes in the next layer will make same calculations as well like we did before but instead of taking X, it will take the outputs of previous layers which is calling activation. Then we gonna push it in to activation function and after last layer we will have our predictions of possibilities. This process that we talked about calling as **'Forward Propagation'**. After having our predictions we gonna compare them with our Y (one hot encoded labels) and compute the **loss** that we have (it will show like how far we are from the actual results).\n\nAfter this step we will go **backwards**. This process as you can understand from the movement calling as **\"Backward Propagation\"**. We will calculate some derivatives corresponding to our loss function which will help us to minimize our loss function. After we have all derivatives we want, we will update our parameters that I said at first **W** and **b**.\nSo for going one time forward and one time backwards, it will count as one iteration. Then we gonna repeat it many times to have the best minimum loss which means more accurate predictions (**gradient descent**). I tried to explain how NN works overally without giving more details. So let's start coding it step by step!","metadata":{}},{"cell_type":"markdown","source":"We are starting with initializing our parameters (W and b). In this step we want to be sure that shapes of $W^{[1]}$, $W^{[2]}$, $b^{[1]}$, $b^{[2]}$ are correct and initialized randomly. For example, $W^{[1]}$ will look like that:\n
\n
\n
\n*$W^{[i]}_{j}$ denotes i.layers j. node's weights\n
","metadata":{}},{"cell_type":"code","source":"def initialize_parameters(n_x,n_h,n_y):\n W1 = np.random.randn(n_h,n_x) * .01 #Random initialization for W1\n b1 = np.random.randn(n_h,1) * .01 #Random initialization for b1\n W2 = np.random.randn(n_y,n_h) * .01 #Random initialization for W2\n b2 = np.random.randn(n_y,1) * .01 #Random initialization for b2\n parameters = {'W1' : W1,\n 'b1' : b1,\n 'W2' : W2,\n 'b2' : b2}\n return parameters","metadata":{"execution":{"iopub.status.busy":"2022-08-14T10:48:08.623194Z","iopub.execute_input":"2022-08-14T10:48:08.623614Z","iopub.status.idle":"2022-08-14T10:48:08.630617Z","shell.execute_reply.started":"2022-08-14T10:48:08.623581Z","shell.execute_reply":"2022-08-14T10:48:08.629507Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We will be using ReLU activation function for our nodes in $L_{2}$. Let's firt check how ReLU looks like and then let's define it.\n
\n
\n\nAs you see from the picture above, ReLU returns **0** for every possible x which is **lower than 0** and **number itself** for every x which is greater than **0**. ","metadata":{}},{"cell_type":"code","source":"def ReLU(Z): ## defining ReLU\n return np.maximum(0,Z)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T10:48:10.441417Z","iopub.execute_input":"2022-08-14T10:48:10.442045Z","iopub.status.idle":"2022-08-14T10:48:10.446198Z","shell.execute_reply.started":"2022-08-14T10:48:10.442008Z","shell.execute_reply":"2022-08-14T10:48:10.445303Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We will use softmax function for our output layer ($L_{3}$) because we are trying to make a multiclass classification. I'm not gonna give more information about it. If you are curious about it there are lots of resources that you can check it out :)","metadata":{}},{"cell_type":"code","source":"def softmax(Z): # defining sofmax function\n return np.exp(Z) / sum(np.exp(Z))","metadata":{"execution":{"iopub.status.busy":"2022-08-14T10:48:12.265529Z","iopub.execute_input":"2022-08-14T10:48:12.266189Z","iopub.status.idle":"2022-08-14T10:48:12.270416Z","shell.execute_reply.started":"2022-08-14T10:48:12.266152Z","shell.execute_reply":"2022-08-14T10:48:12.269306Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Forward propagation**\n\nWe are ready to define our forward propagation step! You can check equations that I've implemented below ⬇\n\n$$Z^{[1]} = W^{[1]} X + b^{[1]}$$\n$$A^{[1]} = g_{\\text{ReLU}}(Z^{[1]}))$$\n$$Z^{[2]} = W^{[2]} A^{[1]} + b^{[2]}$$\n$$A^{[2]} = g_{\\text{softmax}}(Z^{[2]})$$","metadata":{}},{"cell_type":"code","source":"def forward_propagation(A0,parameters):\n # pulling out our parameters from output of initialize_parameters()\n W1 = parameters['W1'] \n b1 = parameters['b1']\n W2 = parameters['W2']\n b2 = parameters['b2']\n \n # implementing our equations for forward propagation\n Z1 = np.dot(W1,A0) + b1\n A1 = ReLU(Z1)\n Z2 = np.dot(W2,A1) + b2\n A2 = softmax(Z2)\n \n outputs = {'Z1' : Z1,\n 'A1' : A1,\n 'Z2' : Z2,\n 'A2' : A2}\n \n return outputs ","metadata":{"execution":{"iopub.status.busy":"2022-08-14T10:49:11.503087Z","iopub.execute_input":"2022-08-14T10:49:11.503583Z","iopub.status.idle":"2022-08-14T10:49:11.510781Z","shell.execute_reply.started":"2022-08-14T10:49:11.503545Z","shell.execute_reply":"2022-08-14T10:49:11.509778Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Compute Loss**\n\nAfter going forward we have to compute our cost (loss) for predictions that we made. Let's see how we implement it. You can see the equation that we will implement below.\n$$J = - \\frac{1}{m} \\sum\\limits_{i = 0}^{m} \\large\\left(\\small y^{(i)}\\log\\left(a^{[2] (i)}\\right) + (1-y^{(i)})\\log\\left(1- a^{[2] (i)}\\right) \\large \\right) $$","metadata":{}},{"cell_type":"code","source":"def loss(Y_hat,Y):\n \n m = Y.shape[1]\n \n logprobs = np.multiply(Y , np.log(Y_hat)) + np.multiply(1-Y,np.log(1-Y_hat))\n loss = (-1/m) * np.sum(logprobs)\n \n return loss","metadata":{"execution":{"iopub.status.busy":"2022-08-14T10:52:48.871556Z","iopub.execute_input":"2022-08-14T10:52:48.872066Z","iopub.status.idle":"2022-08-14T10:52:48.879647Z","shell.execute_reply.started":"2022-08-14T10:52:48.872024Z","shell.execute_reply":"2022-08-14T10:52:48.878284Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Backward Propagation**\n\nWe went through from all layers we have. We compute the cost so what's next? Yes, you are right! We have to go backwards after that. So let's see how we do that. There are the equations that I will use for backward propagation.\n\n$$dZ^{[2]} = A^{[2]} - Y$$\n$$dW^{[2]} = \\frac{1}{m} dZ^{[2]} A^{[1]T}$$\n$$dB^{[2]} = \\frac{1}{m} \\Sigma {dZ^{[2]}}$$\n$$dZ^{[1]} = W^{[2]T} dZ^{[2]} .* g^{[1]\\prime} (z^{[1]})$$\n$$dW^{[1]} = \\frac{1}{m} dZ^{[1]} A^{[0]T}$$\n$$dB^{[1]} = \\frac{1}{m} \\Sigma {dZ^{[1]}}$$\n\nFirst I will implement derivative of ReLU which will be helpful for us while implementing backward propagation. The derivative of ReLU is really easy to implement because the line has **zero slope** for every value which is **lower than zero** and **1** for the values **greater than zero**.","metadata":{}},{"cell_type":"code","source":"def derivative_ReLU(X): \n return X > 0","metadata":{"execution":{"iopub.status.busy":"2022-08-14T10:59:14.172071Z","iopub.execute_input":"2022-08-14T10:59:14.172514Z","iopub.status.idle":"2022-08-14T10:59:14.177064Z","shell.execute_reply.started":"2022-08-14T10:59:14.172480Z","shell.execute_reply":"2022-08-14T10:59:14.176122Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def backward_propagation(parameters,outputs,X,Y):\n # implementing backward propagation equations which is above\n \n m = X.shape[1]\n \n W1 = parameters['W1']\n W2 = parameters['W2']\n \n A1 = outputs['A1']\n Z1 = outputs['Z1']\n A2 = outputs['A2']\n \n dZ2 = A2 - Y\n dW2 = (1/m)*np.dot(dZ2,A1.T)\n db2 = (1/m)*np.sum(dZ2,axis = 1, keepdims=True)\n \n dZ1 = np.dot(W2.T,dZ2) * derivative_ReLU(Z1)\n dW1 = (1/m)*np.dot(dZ1,X.T)\n db1 = (1/m)*np.sum(dZ1,axis = 1, keepdims=True)\n \n derivatives = {\"dW1\": dW1,\n \"db1\": db1,\n \"dW2\": dW2,\n \"db2\": db2}\n \n return derivatives","metadata":{"execution":{"iopub.status.busy":"2022-08-14T10:59:33.994418Z","iopub.execute_input":"2022-08-14T10:59:33.994925Z","iopub.status.idle":"2022-08-14T10:59:34.006445Z","shell.execute_reply.started":"2022-08-14T10:59:33.994884Z","shell.execute_reply":"2022-08-14T10:59:34.005310Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Updating Parameters**\n\nAlright time to update our parameters with help of derivatives we have.\n\n$$W^{[2]} := W^{[2]} - \\alpha dW^{[2]}$$\n$$b^{[2]} := b^{[2]} - \\alpha db^{[2]}$$\n$$W^{[1]} := W^{[1]} - \\alpha dW^{[1]}$$\n$$b^{[1]} := b^{[1]} - \\alpha db^{[1]}$$\n\n**What is $\\alpha$?**\n\nThere is something new which is $\\alpha$ that we haven'talked before. You will hear it as learning rate in the deep learning literature. So what does this $\\alpha$ do? It decides to how big steps that our algorithm (gradient descent) will take. Let's look at it with animation.\n
\n
\n\nAs you can see from that countor plot (for loss) in every iteration the loss converges to zero. So the length of blue arrows actually depends on our $\\alpha$ . So big learning rate means longer arrows. It might look like we have to use big learning rates to converge faster but it doesn't work like that. If you have big learning rate, your algorithm will have trouble to hit minimum and if you use so small number for learning rate it will take lots of time to find minimum. So picking correct learning rate is really important for model. \n\n**Extra information**: As you can see from the animation above the arrows are getting shorter when it converges to minimum. The reason of this, the loss function is a convex function as shown below. When you converge to the minimum point of the function the slope of function shrinks. So the steps that you take becomes smaller.\n
\n
\n\n","metadata":{}},{"cell_type":"code","source":"def update_parameters(parameters,derivatives,alpha):\n \n W1 = parameters['W1']\n b1 = parameters['b1']\n W2 = parameters['W2']\n b2 = parameters['b2']\n \n dW1 = derivatives['dW1']\n db1 = derivatives['db1']\n dW2 = derivatives['dW2']\n db2 = derivatives['db2']\n \n W1 = W1 - alpha*dW1\n b1 = b1 - alpha*db1\n W2 = W2 - alpha*dW2\n b2 = b2 - alpha*db2\n \n parameters = {\"W1\": W1,\n \"b1\": b1,\n \"W2\": W2,\n \"b2\": b2}\n \n return parameters","metadata":{"execution":{"iopub.status.busy":"2022-08-14T11:22:58.740049Z","iopub.execute_input":"2022-08-14T11:22:58.741258Z","iopub.status.idle":"2022-08-14T11:22:58.748346Z","shell.execute_reply.started":"2022-08-14T11:22:58.741216Z","shell.execute_reply":"2022-08-14T11:22:58.747322Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## WE ARE READY TO DEPLOY OUR MODEL 🔜 \n\n\nSo let's bring everything together and make some predictions with it!","metadata":{}},{"cell_type":"code","source":"def model(X,Y,n_h, num_iterations = 500,alpha=1.3):\n \n n_X = X.shape[0]\n n_Y = 10\n \n parameters = initialize_parameters(n_X,n_h,n_Y)\n \n for i in range(num_iterations):\n outputs = forward_propagation(X,parameters)\n\n loss_values = loss(outputs['A2'],Y)\n\n derivatives = backward_propagation(parameters,outputs,X,Y)\n\n parameters = update_parameters(parameters,derivatives,alpha)\n \n if i % 100 == 0:\n print('Loss at {}. iteration is {}'.format(i,loss_values))\n \n \n return parameters","metadata":{"execution":{"iopub.status.busy":"2022-08-14T11:23:01.712829Z","iopub.execute_input":"2022-08-14T11:23:01.713282Z","iopub.status.idle":"2022-08-14T11:23:01.721835Z","shell.execute_reply.started":"2022-08-14T11:23:01.713247Z","shell.execute_reply":"2022-08-14T11:23:01.720485Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Alright after fitting our model we will make some predictions and our predictions will show the possibilities of being any number (0 to 9). We will pick the biggest opportunity as our prediction. Our possibilities ($A^{[2]}$) will look like that:\n
\n
\nand our predictions will look like that in the array:\n
\n
","metadata":{}},{"cell_type":"code","source":"def predict(parameters,X):\n \n outputs = forward_propagation(X,parameters)\n A2 = outputs['A2']\n \n predictions = np.argmax(A2,0)\n predictions =predictions.reshape(X.shape[1],1)\n \n return predictions","metadata":{"execution":{"iopub.status.busy":"2022-08-14T11:23:11.132759Z","iopub.execute_input":"2022-08-14T11:23:11.133221Z","iopub.status.idle":"2022-08-14T11:23:11.139800Z","shell.execute_reply.started":"2022-08-14T11:23:11.133184Z","shell.execute_reply":"2022-08-14T11:23:11.138682Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Fitting our model, making predictions and checking accuracy 🎯\n","metadata":{"execution":{"iopub.status.busy":"2022-08-14T12:13:16.952649Z","iopub.execute_input":"2022-08-14T12:13:16.953628Z","iopub.status.idle":"2022-08-14T12:13:16.960877Z","shell.execute_reply.started":"2022-08-14T12:13:16.953582Z","shell.execute_reply":"2022-08-14T12:13:16.959175Z"}}},{"cell_type":"code","source":"parameters = model(X_train,Y_train,5, num_iterations = 1000,alpha=.3)\npredictions = predict(parameters,X_train)","metadata":{"execution":{"iopub.status.busy":"2022-08-14T11:50:43.837559Z","iopub.execute_input":"2022-08-14T11:50:43.838024Z","iopub.status.idle":"2022-08-14T11:52:36.388836Z","shell.execute_reply.started":"2022-08-14T11:50:43.837986Z","shell.execute_reply":"2022-08-14T11:52:36.387393Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"For every 100 iteration we can see that our loss is shrinking which is good. That means that our model works well!","metadata":{}},{"cell_type":"code","source":"accuracy = (np.sum(predictions == train_labels) / train_labels.shape[0])*100\nprint(f'Training accuracy = %{accuracy}')","metadata":{"execution":{"iopub.status.busy":"2022-08-14T11:53:06.569331Z","iopub.execute_input":"2022-08-14T11:53:06.569744Z","iopub.status.idle":"2022-08-14T11:53:06.576597Z","shell.execute_reply.started":"2022-08-14T11:53:06.569712Z","shell.execute_reply":"2022-08-14T11:53:06.575181Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"predictions = predict(parameters,X_test)\naccuracy = (np.sum(predictions == test_labels) / test_labels.shape[0])*100\nprint(f'Test accuracy = %{accuracy}')","metadata":{"execution":{"iopub.status.busy":"2022-08-14T11:53:08.855527Z","iopub.execute_input":"2022-08-14T11:53:08.855930Z","iopub.status.idle":"2022-08-14T11:53:08.876745Z","shell.execute_reply.started":"2022-08-14T11:53:08.855897Z","shell.execute_reply":"2022-08-14T11:53:08.875176Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"display(predictions[35:40])\ndisplay(test_labels[35:40])","metadata":{"execution":{"iopub.status.busy":"2022-08-14T12:11:16.982073Z","iopub.execute_input":"2022-08-14T12:11:16.982513Z","iopub.status.idle":"2022-08-14T12:11:16.992902Z","shell.execute_reply.started":"2022-08-14T12:11:16.982474Z","shell.execute_reply":"2022-08-14T12:11:16.991786Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Let's check some of the mispredicted examples.","metadata":{}},{"cell_type":"code","source":"print('Our prediction:{}'.format(predictions[18,:]))\ncheck_pictures(X_test[:,18])\n\nprint('Our prediction:{}'.format(predictions[34,:]))\ncheck_pictures(X_test[:,34])\n\nprint('Our prediction:{}'.format(predictions[36,:]))\ncheck_pictures(X_test[:,36])","metadata":{"execution":{"iopub.status.busy":"2022-08-14T12:11:36.955859Z","iopub.execute_input":"2022-08-14T12:11:36.956298Z","iopub.status.idle":"2022-08-14T12:11:37.451214Z","shell.execute_reply.started":"2022-08-14T12:11:36.956264Z","shell.execute_reply":"2022-08-14T12:11:37.449989Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Conclusion 🔚\n","metadata":{}},{"cell_type":"markdown","source":"\nYes, we did it!! Finally we implemented our model from scratch which classifies digits. The accuracy scores that we get for train data is **89.4%** and for test data it's **88.9%** which is good we don't have overfitting problem at all. I haven't talked about overfitting in this notebook but I can talk about it in the coming notebooks if you wanna :) Thanks for reading the notebook until the end and I hope you learned something you don't know through reading it. If you have any question or any part that you haven't understand or you found any part hard to interpret just let me know in the comments! See you in the next notebook 👋","metadata":{}}]}