{"metadata":{"kernelspec":{"name":"python3","display_name":"Python 3","language":"python"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":3004,"databundleVersionId":861823,"sourceType":"competition"}],"dockerImageVersionId":30698,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# IMPORTING THE LİBRARİES","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport seaborn as sns\nimport tensorflow as tf                       \nimport numpy as np                           \nimport matplotlib.pyplot as plt               \n%matplotlib inline","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# LOAD THE DATA","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.datasets.mnist import load_data #fetch the data from keras","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"(X_train, y_train) , (X_test, y_test) = load_data()  #you can think this as train_test_split","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(\"Train size:\", len(X_train))     \nprint(\"Test size:\", len(X_test))  ","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"X_train[24]  #Let's test some images","metadata":{"scrolled":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import matplotlib.pyplot as plt","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plt.imshow(X_train[24], cmap= plt.cm.binary); #visualize the matrix above","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"num_rows, num_cols = 3, 6                                       # Datadan 3 tane row, 6 tane column sectik.\nf, ax = plt.subplots(3, 6, figsize=(12,5),\n                     gridspec_kw={'wspace':0.04, 'hspace':0.03}, \n                     squeeze=True)\n\nfor r in range(num_rows):\n    for c in range(num_cols):\n      \n        image_index = r * 5 + c\n        ax[r,c].axis(\"off\")\n        ax[r,c].imshow( X_train[image_index], cmap='gray')\nplt.show()\nplt.close()","metadata":{"scrolled":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Lets look some matrix values in order to see if data is scaled or not","metadata":{}},{"cell_type":"code","source":"X_train[19].min() #would be 0-1 if it's scaled","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"X_train[19].max() #would be 0-1 if it's scaled","metadata":{"scrolled":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"##  ⚖️ WHY SCALING\n<a id=\"data-scaling\">\nNormalizing an image dataset for CNN means adjusting the pixel values of images so they fall in a similar range, typically between 0 and 1. This helps the CNN learn faster and perform better.\n</a>","metadata":{}},{"cell_type":"code","source":"#Let's scale\nX_train = X_train.astype('float32') #Float conversion is used to prevent rounding numbers to integers\nX_test = X_test.astype('float32')\nX_train /= 255\nX_test /= 255","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"X_train[23].min() #now it's scaled","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"X_train[23].max()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"X_train = X_train.reshape(X_train.shape[0],28, 28, 1) \nX_test = X_test.reshape(X_test.shape[0], 28, 28, 1)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Converting data to tensors for CNN input is crucial because it:\n\n1. Standardizes the input format for efficient computation.\n2. Preserves spatial information crucial for image analysis.\n3. Represents color channels, aiding in feature extraction.\n4. Enables compatibility with CNN architectures, optimizing model performance.","metadata":{}},{"cell_type":"code","source":"X_train.shape, X_test.shape #now it's converted from matrix to tensor","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"X_train[245].shape ","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"The X_train dataset originally had a shape of (60000, 28, 28), which is a 3-dimensional matrix. However, CNN models require input data to be in a 4-dimensional format. Therefore, the dataset was reshaped into the format (60000, 28, 28, 1) by adding an extra dimension (1 for grayscale) to make it compatible for feeding into a CNN model. This reshaping process enables the CNN model to process the image data correctly.","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.utils import to_categorical","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"Y_train = to_categorical(y_train, 10) \nY_test = to_categorical(y_test, 10)\nY_test.shape","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"\nThe code snippet converts class labels in the MNIST dataset, representing digits from 0 to 9, into a format suitable for machine learning algorithms. Instead of using the labels directly, which are single digits, the to_categorical() function transforms them into 10-dimensional vectors. For example, the label \"5\" becomes [0, 0, 0, 0, 0, 1, 0, 0, 0, 0]. This vectorized representation is essential for multiclass classification tasks.","metadata":{}},{"cell_type":"markdown","source":"# 🧠LET'S BUILD OUR MODEL🧠","metadata":{}},{"cell_type":"code","source":"from tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, BatchNormalization, Dropout,Input","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#Defien the input shape\ninput_shape=X_train[123].shape\ninput_shape","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"model = Sequential()\n#model.add(Input(shape=input_shape)) #It helps model to compile better when u add the input as a \"LAYER\"\n#CNN\nmodel.add(Conv2D(16, kernel_size=(3, 3), activation='relu',padding='same',input_shape=(28, 28, 1))) # if we use padding='valid' we would have lost shape of the images\nmodel.add(MaxPooling2D(pool_size=(3, 3)))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(0.1))\n\nmodel.add(Conv2D(32, kernel_size=(3, 3), activation='relu',padding='same'))\nmodel.add(MaxPooling2D(pool_size=(3, 3)))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(0.1))\n\nmodel.add(Flatten())\n#ANN\n\nmodel.add(Dense(128, activation='relu'))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(0.2))\n\n# OUTPUT LAYER\nmodel.add(Dense(10, activation='softmax'))\n\nmodel.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# 🎯MODEL SUMMARY🎯","metadata":{}},{"cell_type":"code","source":"model.summary()","metadata":{"scrolled":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from tensorflow.keras.callbacks import EarlyStopping","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"early_stop = EarlyStopping(monitor=\"val_loss\", \n                           min_delta=0, #val_loss value between two epochs\n                           mode=\"auto\",\n                           restore_best_weights = True, #in case if model misses the best_weights\n                           verbose=1,\n                           patience=50)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"model.fit(X_train,\n          Y_train, #do not forget to use categorical form of Y, if you use y_train you will get dimension error!!!\n          validation_split=0.1,\n          batch_size=512,\n          epochs=100,\n          verbose=1,\n          callbacks=[early_stop])","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"model.history.history","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"summary = pd.DataFrame(model.history.history)\nsummary.head(50)[10:20:1]","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"model.evaluate(X_test,Y_test)[1] #to see test accuracy","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plt.figure(figsize=(10,6))\nplt.plot(summary.accuracy)        \nplt.plot(summary.val_accuracy)\nplt.ylabel(\"Accuracy\")\nplt.xlabel(\"Epoch\")\nplt.legend('labels');","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plt.figure(figsize=(10,6))\nplt.plot(summary.accuracy)        \nplt.plot(summary.val_accuracy)\nplt.ylabel(\"Accuracy\")\nplt.xlabel(\"Epoch\")\nplt.legend('labels');\n\nplt.ylim(0.95,1) #in order to zoom in","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# 🚀 🚀 PREDICTION🚀 🚀 ","metadata":{}},{"cell_type":"code","source":"preds = model.predict(X_test)\npreds","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"preds[0] #We have probabilities of each class we have to select the highest prob value","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"predictions= np.argmax(preds, axis=1)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"predictions #now we can see which label model predicted","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(predictions[:100])","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#  🎯 EVALUATION 🎯","metadata":{}},{"cell_type":"code","source":"from sklearn.metrics import classification_report,confusion_matrix","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(confusion_matrix(y_test, predictions))","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# PREDICTION OF TEST VALUES","metadata":{}},{"cell_type":"code","source":"number_test = X_test[24]\nplt.imshow(number_test); ","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"np.argmax(model.predict(number_test.reshape(1,28,28,1)), axis=1)","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}