{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"'''Data Manipulattion Modules'''\nimport numpy as np\nimport pandas as pd\n\n'''Visualization Modules '''\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n%matplotlib inline\n\n'''Preprocessing libraries'''\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import confusion_matrix\nfrom sklearn import preprocessing\n\n'''keras libraries for CNN'''\nfrom keras.utils.np_utils import to_categorical\nfrom keras.models import Sequential\nfrom keras.layers import Dense,Conv2D,Flatten,MaxPool2D,Dropout,BatchNormalization\nfrom keras.optimizers import RMSprop,Adam\nfrom keras.callbacks import ReduceLROnPlateau","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# **LOADING THE DATA**"},{"metadata":{"trusted":true},"cell_type":"code","source":"train=pd.read_csv('../input/Kannada-MNIST/train.csv')\ntest1=pd.read_csv('../input/Kannada-MNIST/test.csv')\nsample_sub=pd.read_csv('../input/Kannada-MNIST/sample_submission.csv')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# **EXPLORATORY DATA ANALYSIS (EDA)**"},{"metadata":{},"cell_type":"markdown","source":"As per data description, each image is 28 pixels in height and 28 pixels in width, for a total of 784 pixels in total (so each row represent one image in data set). Each pixel has a single pixel-value associated with it, indicating the lightness or darkness of that pixel, with higher numbers meaning darker. This pixel-value is an integer between 0 and 255, inclusive.\n\nEach pixel column in the training set has a name like pixel{x}, where x is an integer between 0 and 783, inclusive. To locate this pixel on the image, suppose that we have decomposed x as x = i * 28 + j, where i and j are integers between 0 and 27, inclusive. Then pixel{x} is located on row i and column j of a 28 x 28 matrix, (indexing by zero).\n\nFor example, pixel31 indicates the pixel that is in the fourth column from the left, and the second row from the top, as in the ascii-diagram below.\n\nVisually, if we omit the \"pixel\" prefix, the pixels make up the image like this:\n\n000 001 002 003 ----------------------------------------------------------------------------------------------------------------------------------------028 029 030 031 032-----------------------------------------------------------------------------------------------------------------------------------------056"},{"metadata":{"trusted":true},"cell_type":"code","source":"train.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test1.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test1.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.label.value_counts()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"1. Training data set has 60000 rows and 785 columns, where as test dataset has 5000 rows and 785 columns.\n2. First column is label and rest of the 784 colummns are pixel0 to pixel 783.\n3. All the numbers in the dataset have equal numbers."},{"metadata":{},"cell_type":"markdown","source":"# **DATA PREPROCESSING**"},{"metadata":{},"cell_type":"markdown","source":"Lets seperate features and labels of our training set.And divide our data in taining and vaildation set"},{"metadata":{"trusted":true},"cell_type":"code","source":"X_train, X_valid, y_train, y_valid =\\\n    train_test_split(train.drop('label', axis=1).values, train.iloc[:,0].values, \n                     test_size=0.3, random_state=1, stratify=train.iloc[:,0])\n\ntest=test1.drop('id',axis=1)\nX_test = test.values\n\nprint('y_train Shape:', y_train.shape)\nprint('X_train Shape:', X_train.shape)\nprint('y_valid Shape:', y_valid.shape)\nprint('X_valid Shape:', X_valid.shape)\nprint('X_test Shape: ', X_test.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Next, we need to reshape our dataset inputs (Xs_train and Xs_test) to the shape that our model expects when we train the model. The first number is the number of images (-1). Then comes the shape of each image (28x28). The last number is 1, which signifies that the images are greyscale. if 3 implies RGB mode"},{"metadata":{"trusted":true},"cell_type":"code","source":"X_train = X_train.reshape(-1, 28, 28, 1)\nX_valid = X_valid.reshape(-1, 28, 28, 1)\nX_test = X_test.reshape(-1, 28, 28, 1)\n\nprint('X_train Shape:', X_train.shape)\nprint('X_valid Shape:', X_valid.shape)\nprint('X_test Shape: ', X_test.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Plot the Images"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(X_train[0][:,:,0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.imshow(X_train[10][:,:,0])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"____________"},{"metadata":{},"cell_type":"markdown","source":"Neural networks process inputs using small weight values, and inputs with large integer values can disrupt or slow down the learning process. Therefore, we need to scale pixel values between 0 and 1. "},{"metadata":{"trusted":true},"cell_type":"code","source":"Xs_train = X_train / 255\nXs_valid = X_valid / 255","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"ENCODING TARGET VALUES.The categorical value assignment can be done using sklearn’s LabelEncoder. Problem with label encoding is that it assumes higher the categorical value, better the category, WHICH WE DONT WANT! lets use Keras inbuild library to_categorical() for one-hot encoding. "},{"metadata":{"trusted":true},"cell_type":"code","source":"y_train=to_categorical(y_train)\ny_valid=to_categorical(y_valid)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# **BUILDING MODEL**"},{"metadata":{"trusted":true},"cell_type":"code","source":"np.random.seed(42)\n\n#The Sequential model is a linear stack of layers.\n\nfrom keras.models import Sequential\n\n# Activation the Model\nKannad_cnn = Sequential()\n\n# adding layers to model\n#BLOCK 1\nKannad_cnn.add(Conv2D(32, (3,3), padding='same', input_shape=(28,28,1), activation = 'relu') )\nKannad_cnn.add(MaxPool2D(2,2))\nKannad_cnn.add(BatchNormalization())\n\n#BLOCK 2\nKannad_cnn.add(Conv2D(64, (3,3), padding='same', activation = 'relu') )\nKannad_cnn.add(MaxPool2D(2,2))\nKannad_cnn.add(BatchNormalization())\n\n#Final Block\nKannad_cnn.add(Flatten())\nKannad_cnn.add(Dense(64, activation='relu'))\nKannad_cnn.add(BatchNormalization())\n\n#In between the convolutional layer and the fully connected layer, there is a ‘Flatten’ layer. Flattening transforms \n#a two-dimensional matrix of features into a vector that can be fed into a fully connected neural network classifier.\n\nKannad_cnn.add(Dense(10, activation='softmax'))\n\nKannad_cnn.summary()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Optimizers update the weight parameters to minimize the loss function. Loss function acts as guides to the terrain telling optimizer if it is moving in the right direction to reach the bottom of the valley, the global minimum."},{"metadata":{"trusted":true},"cell_type":"code","source":"optimizer=Adam(learning_rate=0.001,beta_1=0.9,beta_2=0.999,epsilon=1e-08)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"Kannad_cnn.compile(optimizer=optimizer,loss=['categorical_crossentropy'],metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"h1 = Kannad_cnn.fit(Xs_train, y_train, batch_size=1024, epochs=40, verbose=1, \n                   validation_data = [Xs_valid, y_valid]) ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=[10,5])\nplt.subplot(1,2,1)\nplt.plot(range(1,41), h1.history['accuracy'], label='Training Accuracy')\nplt.plot(range(1,41), h1.history['val_accuracy'], label='Validation Accuracy')\nplt.xlabel('Epoch')\nplt.legend()\n\nplt.subplot(1,2,2)\nplt.plot(range(1,41), h1.history['loss'], label='Training Loss')\nplt.plot(range(1,41), h1.history['val_loss'], label='Validation Loss')\nplt.xlabel('Epoch')\nplt.legend()\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"Y_test = Kannad_cnn.predict_classes(X_test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"my_submission = pd.DataFrame({'id' :test1. iloc[:,0], 'label':Y_test })\n\nprint(my_submission.iloc[:5])\n\nmy_submission.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}