{ "id": 332426, "title": "[Digit Recognizer] Summary of all the top discussions posts ", "url": "/competitions/digit-recognizer/discussion/332426", "author_name": "The Devastator", "post_date": "2022-06-21T15:52:22.688000", "votes": 11, "comment_count": 3, "views": null, "content": "

[Digit Recognizer] Summary of all the top discussions posts

\n

Are 100% accuracies fake?
\nby alifrahman

\n\n

kNN vs. Neural Network
\nby cdeotte

\n\n

simple keras with lb 0.99071
\nby china1000

\n\n
import numpy as np\nimport pandas as pd\nfrom keras import backend as K\nfrom keras.models import Sequential\nfrom keras.layers.convolutional import *\nfrom keras.callbacks import EarlyStopping\nfrom keras.utils.np_utils import to_categorical\nfrom keras.layers.core import Dropout, Dense, Flatten, Activation\n\nK.set_image_dim_ordering('th')\n\ntrain_df, valid_df = pd.read_csv(\"../input/train.csv\"), pd.read_csv(\"../input/test.csv\")\n\nx_train = train_df.drop(['label'], axis=1).values.astype('float32')\nY_train = train_df['label'].values.astype('float32')\nx_valid = valid_df.values.astype('float32')\n\nn_train, n_valid = x_train.shape[0], x_valid.shape[0]\n\nx_train, x_valid = x_train.reshape(n_train,1, 28, 28), x_valid.reshape(n_valid, 1, 28, 28)\nx_train, x_valid = (x_train / 255), (x_valid/255)\n\nmodel = Sequential()\nmodel.add(Convolution2D(64, 3, 3, batch_input_shape=(None, 1, 28, 28), activation='relu', border_mode='valid'))\nmodel.add(MaxPooling2D(pool_size=(3, 3)))\nmodel.add(Convolution2D(64, 2, 2, activation='relu', border_mode='valid'))\nmodel.add(MaxPooling2D(pool_size=(1, 1)))\nmodel.add(Dropout(0.25))\nmodel.add(Flatten())\nmodel.add(Dense(128))\nmodel.add(Activation('relu'))\nmodel.add(Dropout(0.5))\nmodel.add(Dense(10))\nmodel.add(Activation('softmax'))\nmodel.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\n\nearly_stopping = EarlyStopping(monitor='val_acc', patience=1)\nmodel.fit(x_train, to_categorical(Y_train), batch_size = 128, nb_epoch=100, verbose = 2, validation_split = .2)\nyPred = model.predict_classes(x_valid,batch_size=32,verbose=1)\nnp.savetxt('mnist_output.csv', np.c_[range(1,len(yPred)+1),yPred], delimiter=',', header = 'ImageId,Label', comments = '', fmt='%d')\n
\n\n

98.83% with Weka, new NN package
\nby johannesamten

\n\n

The way to get 99.571% accuracy with simple model using keras
\nby yugkhanna

\n\n

Applications of Digit recognizer: Resume tricks
\nby kritidoneria

\n\n
**Writing 'Digit recognizer' and 'MNIST' in your resume is cliché. Instead, try these:**\n- Bank cheque/invoices/documents processing using OCR (quantify the impact in terms of man hours having)\n- Document verification system ( OCR and ID verification)\n- Examination grading by digitization MCQ only)\n- Scientific notes digitizer\n- License-plate reading\n
\n

Sharing Python Code for KNN, SVC and Random Forest
\nby broham

\n\n

Downloading train.csv and test.csv
\nby swethasharma

\n\n

Simple tensorflow CNN - +99%
\nby julienknafou

\n\n

Getting Started - Python Sample Code (Random Forest)
\nby cclark

\n\n
from numpy import genfromtxt, savetxt\nfrom sklearn.ensemble import RandomForestClassifier\n\ndef main():\n    #create the training & test sets, skipping the header row with [1:]\n    dataset = genfromtxt(open('Data/train.csv','r'), delimiter=',', dtype='f8')[1:]\n    target = [x[0] for x in dataset]\n    train = [x[1:] for x in dataset]\n    test = genfromtxt(open('Data/test.csv','r'), delimiter=',', dtype='f8')[1:]\n    rf = RandomForestClassifier(n_estimators=100)\n    rf.fit(train, target)\n    savetxt('Data/submission2.csv', rf.predict(test), delimiter=',', fmt='%f')\n
\n

What is a good score?
\nby sanjaykrishnamurthi

\n\n

You might be cheating without knowing it
\nby samuelcortinhas

\n\n

(X_train_mnist, y_train_mnist), (X_val_mnist, y_val_mnist) = mnist.load_data()

\n

kaggle uses a training/test split of 42k/28k whereas this dataset downloaded from keras has a 60k/10k split. This means you will be using some of kaggles test data to train your algorithm, which artificially improves your score because your algorithm is just memorising a lot of the right answers.
\n'''

\n

How to score 97%, 98%, 99%, and 100%
\nby cdeotte

\n\n

Important Research Papers worth Mentioning
\nby ishandutta

\n\n

A beginner guide to MNIST - 0.9925%
\nby shaygu

\n\n

Reading in Data Directly to fast.ai
\nby steventesta

\n\n
class CustomImageItemList(ImageItemList):\n    def open(self, fn):\n        img = fn.reshape(28,28)\n        img = np.stack((img,)*3, axis=-1) # convert to 3 channels\n        return Image(pil2tensor(img, dtype=np.float32))\n\n    @classmethod\n    def from_csv_custom(cls, path:PathOrStr, csv_name:str, imgIdx:int=1, header:str='infer', **kwargs)->'ItemList':\n        df = pd.read_csv(Path(path)/csv_name, header=header)\n        res = super().from_df(df, path=path, cols=0, **kwargs)\n        # convert pixels to an ndarray\n        res.items = df.iloc[:,imgIdx:].apply(lambda x: x.values / 255.0, axis=1).values\n        return res\n
\n

And usage would be as follows:

\n
test = CustomImageItemList.from_csv_custom(path='./data', csv_name='test.csv', imgIdx=0)\ndata = (CustomImageItemList.from_csv_custom(path='./data', csv_name='train.csv')\n                       .random_split_by_pct(.2)\n                       .label_from_df(cols='label')\n                       .add_test(test, label=0)\n                       .databunch(bs=64, num_workers=0))\n
\n\n

Interactive MNIST Digits Recognition visual Tableau
\nby zinovadr

\n\n

\"\"

", "messages": [ { "id": 1828301, "postDate": "2022-06-21T15:52:22.690Z", "content": "

[Digit Recognizer] Summary of all the top discussions posts

\n

Are 100% accuracies fake?
\nby alifrahman

\n\n

kNN vs. Neural Network
\nby cdeotte

\n\n

simple keras with lb 0.99071
\nby china1000

\n\n
import numpy as np\nimport pandas as pd\nfrom keras import backend as K\nfrom keras.models import Sequential\nfrom keras.layers.convolutional import *\nfrom keras.callbacks import EarlyStopping\nfrom keras.utils.np_utils import to_categorical\nfrom keras.layers.core import Dropout, Dense, Flatten, Activation\n\nK.set_image_dim_ordering('th')\n\ntrain_df, valid_df = pd.read_csv(\"../input/train.csv\"), pd.read_csv(\"../input/test.csv\")\n\nx_train = train_df.drop(['label'], axis=1).values.astype('float32')\nY_train = train_df['label'].values.astype('float32')\nx_valid = valid_df.values.astype('float32')\n\nn_train, n_valid = x_train.shape[0], x_valid.shape[0]\n\nx_train, x_valid = x_train.reshape(n_train,1, 28, 28), x_valid.reshape(n_valid, 1, 28, 28)\nx_train, x_valid = (x_train / 255), (x_valid/255)\n\nmodel = Sequential()\nmodel.add(Convolution2D(64, 3, 3, batch_input_shape=(None, 1, 28, 28), activation='relu', border_mode='valid'))\nmodel.add(MaxPooling2D(pool_size=(3, 3)))\nmodel.add(Convolution2D(64, 2, 2, activation='relu', border_mode='valid'))\nmodel.add(MaxPooling2D(pool_size=(1, 1)))\nmodel.add(Dropout(0.25))\nmodel.add(Flatten())\nmodel.add(Dense(128))\nmodel.add(Activation('relu'))\nmodel.add(Dropout(0.5))\nmodel.add(Dense(10))\nmodel.add(Activation('softmax'))\nmodel.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\n\nearly_stopping = EarlyStopping(monitor='val_acc', patience=1)\nmodel.fit(x_train, to_categorical(Y_train), batch_size = 128, nb_epoch=100, verbose = 2, validation_split = .2)\nyPred = model.predict_classes(x_valid,batch_size=32,verbose=1)\nnp.savetxt('mnist_output.csv', np.c_[range(1,len(yPred)+1),yPred], delimiter=',', header = 'ImageId,Label', comments = '', fmt='%d')\n
\n\n

98.83% with Weka, new NN package
\nby johannesamten

\n\n

The way to get 99.571% accuracy with simple model using keras
\nby yugkhanna

\n\n

Applications of Digit recognizer: Resume tricks
\nby kritidoneria

\n\n
**Writing 'Digit recognizer' and 'MNIST' in your resume is cliché. Instead, try these:**\n- Bank cheque/invoices/documents processing using OCR (quantify the impact in terms of man hours having)\n- Document verification system ( OCR and ID verification)\n- Examination grading by digitization MCQ only)\n- Scientific notes digitizer\n- License-plate reading\n
\n

Sharing Python Code for KNN, SVC and Random Forest
\nby broham

\n\n

Downloading train.csv and test.csv
\nby swethasharma

\n\n

Simple tensorflow CNN - +99%
\nby julienknafou

\n\n

Getting Started - Python Sample Code (Random Forest)
\nby cclark

\n\n
from numpy import genfromtxt, savetxt\nfrom sklearn.ensemble import RandomForestClassifier\n\ndef main():\n    #create the training & test sets, skipping the header row with [1:]\n    dataset = genfromtxt(open('Data/train.csv','r'), delimiter=',', dtype='f8')[1:]\n    target = [x[0] for x in dataset]\n    train = [x[1:] for x in dataset]\n    test = genfromtxt(open('Data/test.csv','r'), delimiter=',', dtype='f8')[1:]\n    rf = RandomForestClassifier(n_estimators=100)\n    rf.fit(train, target)\n    savetxt('Data/submission2.csv', rf.predict(test), delimiter=',', fmt='%f')\n
\n

What is a good score?
\nby sanjaykrishnamurthi

\n\n

You might be cheating without knowing it
\nby samuelcortinhas

\n\n

(X_train_mnist, y_train_mnist), (X_val_mnist, y_val_mnist) = mnist.load_data()

\n

kaggle uses a training/test split of 42k/28k whereas this dataset downloaded from keras has a 60k/10k split. This means you will be using some of kaggles test data to train your algorithm, which artificially improves your score because your algorithm is just memorising a lot of the right answers.
\n'''

\n

How to score 97%, 98%, 99%, and 100%
\nby cdeotte

\n\n

Important Research Papers worth Mentioning
\nby ishandutta

\n\n

A beginner guide to MNIST - 0.9925%
\nby shaygu

\n\n

Reading in Data Directly to fast.ai
\nby steventesta

\n\n
class CustomImageItemList(ImageItemList):\n    def open(self, fn):\n        img = fn.reshape(28,28)\n        img = np.stack((img,)*3, axis=-1) # convert to 3 channels\n        return Image(pil2tensor(img, dtype=np.float32))\n\n    @classmethod\n    def from_csv_custom(cls, path:PathOrStr, csv_name:str, imgIdx:int=1, header:str='infer', **kwargs)->'ItemList':\n        df = pd.read_csv(Path(path)/csv_name, header=header)\n        res = super().from_df(df, path=path, cols=0, **kwargs)\n        # convert pixels to an ndarray\n        res.items = df.iloc[:,imgIdx:].apply(lambda x: x.values / 255.0, axis=1).values\n        return res\n
\n

And usage would be as follows:

\n
test = CustomImageItemList.from_csv_custom(path='./data', csv_name='test.csv', imgIdx=0)\ndata = (CustomImageItemList.from_csv_custom(path='./data', csv_name='train.csv')\n                       .random_split_by_pct(.2)\n                       .label_from_df(cols='label')\n                       .add_test(test, label=0)\n                       .databunch(bs=64, num_workers=0))\n
\n\n

Interactive MNIST Digits Recognition visual Tableau
\nby zinovadr

\n\n

\"\"

", "rawMarkdown": "### [Digit Recognizer] Summary of all the top discussions posts \n\n\n[Are 100% accuracies fake?](https://www.kaggle.com/competitions/digit-recognizer/discussion/183700) \nby [alifrahman](alifrahman)\n\n- A discussion about solutions with 100% acc: They are using the external public MNIST\n- [Here](https://www.kaggle.com/bgmello/how-top-lb-got-their-score-use-mnist-to-learn) is a notebook that shows how to do this\n\n[kNN vs. Neural Network](https://www.kaggle.com/competitions/digit-recognizer/discussion/60305) \nby [cdeotte](cdeotte)\n\n- A discussion on why a neural network outperforms kNN.\n- From the post: \"The problem is that kNN can not learn that pixel #5 is very important and pixel #13 is misleading. A neural network would learn this quickly and classify 100% of the digits correctly.\"\n\n[simple keras with lb 0.99071](https://www.kaggle.com/competitions/digit-recognizer/discussion/26378)\nby [china1000](china1000)\n\n- Sharing an extremely simple keras model (on the discussion message, not a notebook), here it is minimized a bit:\n```python\n\nimport numpy as np\nimport pandas as pd\nfrom keras import backend as K\nfrom keras.models import Sequential\nfrom keras.layers.convolutional import *\nfrom keras.callbacks import EarlyStopping\nfrom keras.utils.np_utils import to_categorical\nfrom keras.layers.core import Dropout, Dense, Flatten, Activation\n\nK.set_image_dim_ordering('th')\n\ntrain_df, valid_df = pd.read_csv(\"../input/train.csv\"), pd.read_csv(\"../input/test.csv\")\n\nx_train = train_df.drop(['label'], axis=1).values.astype('float32')\nY_train = train_df['label'].values.astype('float32')\nx_valid = valid_df.values.astype('float32')\n\nn_train, n_valid = x_train.shape[0], x_valid.shape[0]\n\nx_train, x_valid = x_train.reshape(n_train,1, 28, 28), x_valid.reshape(n_valid, 1, 28, 28)\nx_train, x_valid = (x_train / 255), (x_valid/255)\n\nmodel = Sequential()\nmodel.add(Convolution2D(64, 3, 3, batch_input_shape=(None, 1, 28, 28), activation='relu', border_mode='valid'))\nmodel.add(MaxPooling2D(pool_size=(3, 3)))\nmodel.add(Convolution2D(64, 2, 2, activation='relu', border_mode='valid'))\nmodel.add(MaxPooling2D(pool_size=(1, 1)))\nmodel.add(Dropout(0.25))\nmodel.add(Flatten())\nmodel.add(Dense(128))\nmodel.add(Activation('relu'))\nmodel.add(Dropout(0.5))\nmodel.add(Dense(10))\nmodel.add(Activation('softmax'))\nmodel.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\n\nearly_stopping = EarlyStopping(monitor='val_acc', patience=1)\nmodel.fit(x_train, to_categorical(Y_train), batch_size = 128, nb_epoch=100, verbose = 2, validation_split = .2)\nyPred = model.predict_classes(x_valid,batch_size=32,verbose=1)\nnp.savetxt('mnist_output.csv', np.c_[range(1,len(yPred)+1),yPred], delimiter=',', header = 'ImageId,Label', comments = '', fmt='%d')\n```\n\n- Sharing a [public notebook](https://www.kaggle.com/danielelton/digit-recognizer/keras-cnn-with-explanation/notebook) with a simple keras model that got a public LB score of 0.99071.\n\n[98.83% with Weka, new NN package](https://www.kaggle.com/competitions/digit-recognizer/discussion/7221) \nby [johannesamten](johannesamten)\n\n- The package was used to get 98.83% accuracy.\n- Sharing a [Weka package](https://github.com/amten/NeuralNetwork) for a neural network with dropout and rectified linear units in java. [wow this is an old post..]\n\n[The way to get 99.571% accuracy with simple model using keras](https://www.kaggle.com/competitions/digit-recognizer/discussion/62976)\nby [yugkhanna](yugkhanna)\n\n- Sharing a [medium article](https://medium.com/@yugkhanna1/building-a-5-layer-cnn-model-with-keras-to-identify-handwritten-digits-complex-yet-simple-f06b8b375346) about how to get 99.571% accuracy with a simple model using keras.\n\n[Applications of Digit recognizer: Resume tricks](https://www.kaggle.com/competitions/digit-recognizer/discussion/252787) \nby [kritidoneria](kritidoneria)\n\n- The post suggest that the same infoprmation can expressed in different words and sound much more impressive as a CV line. From the post:\n\n```\n**Writing 'Digit recognizer' and 'MNIST' in your resume is cliché. Instead, try these:**\n- Bank cheque/invoices/documents processing using OCR (quantify the impact in terms of man hours having)\n- Document verification system ( OCR and ID verification)\n- Examination grading by digitization MCQ only)\n- Scientific notes digitizer\n- License-plate reading\n```\n\n[Sharing Python Code for KNN, SVC and Random Forest](https://www.kaggle.com/competitions/digit-recognizer/discussion/4281)\nby [broham](broham)\n\n- Sharing a [GitHub repo](https://github.com/Broham/DigitRecognizer) with Python code for KNN, SVC and Random Forest.\n\n[Downloading train.csv and test.csv](https://www.kaggle.com/competitions/digit-recognizer/discussion/2983) \nby [swethasharma](swethasharma)\n\n- The post discusses a downloading problem that happened before, on the comments it is suggested that the files had been reuploaded [here](https://storage.googleapis.com/kaggle-forum-message-attachments/263398/8134/kaggle-digit-recognizer-data.tar.gz) as well if there is any problem in the future.\n\n[Simple tensorflow CNN - +99%](https://www.kaggle.com/competitions/digit-recognizer/discussion/43349)\nby [julienknafou](julienknafou)\n\n- Sharing a top 1% solution both in the post itself and on [GitHub](https://github.com/KubaMichalczyk/kaggle-predict-future-sales)\n\n[Getting Started - Python Sample Code (Random Forest)](https://www.kaggle.com/competitions/digit-recognizer/discussion/2299) \nby [cclark](cclark)\n\n- Sharing a simple solution to generate the Random Forest benchmark\n```python\nfrom numpy import genfromtxt, savetxt\nfrom sklearn.ensemble import RandomForestClassifier\n\ndef main():\n #create the training & test sets, skipping the header row with [1:]\n dataset = genfromtxt(open('Data/train.csv','r'), delimiter=',', dtype='f8')[1:]\n target = [x[0] for x in dataset]\n train = [x[1:] for x in dataset]\n test = genfromtxt(open('Data/test.csv','r'), delimiter=',', dtype='f8')[1:]\n rf = RandomForestClassifier(n_estimators=100)\n rf.fit(train, target)\n savetxt('Data/submission2.csv', rf.predict(test), delimiter=',', fmt='%f')\n\n```\n\n[What is a good score?](https://www.kaggle.com/competitions/digit-recognizer/discussion/163676)\nby [sanjaykrishnamurthi](sanjaykrishnamurthi)\n\n- Asking what a good score is in the Digit Recognizer competition.\n- From the comments:\n- 'I think if you can cross 99.8% accuracy, it should be good enough. But, with further tweaks like using callback function, augmenting data and using deeper models, you can easily get the accuracy over 99.9%.'\n- see [this](https://www.kaggle.com/c/digit-recognizer/discussion/61480) post\n\n[You might be cheating without knowing it](https://www.kaggle.com/competitions/digit-recognizer/discussion/296110) \nby [samuelcortinhas](samuelcortinhas)\n\n- From the post: '''\nin the top scoring notebooks:\n\n(X_train_mnist, y_train_mnist), (X_val_mnist, y_val_mnist) = mnist.load_data()\n\nkaggle uses a training/test split of 42k/28k whereas this dataset downloaded from keras has a 60k/10k split. This means you will be using some of kaggles test data to train your algorithm, which artificially improves your score because your algorithm is just memorising a lot of the right answers.\n'''\n\n[How to score 97%, 98%, 99%, and 100%](https://www.kaggle.com/competitions/digit-recognizer/discussion/61480) \nby [cdeotte](cdeotte)\n\n- Sharing a histogram of Kaggle leaderboard scores where each bar has range 0.1%.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1723677%2Fb858f6500a16f9a0e3e23ee9126b0069%2Fimg1.png?generation=1602506040379499&alt=media)\n- Sharing a schematic of a CNN architecture.\n- A discussion on how to score 97%, 98%, [99%](https://www.kaggle.com/cdeotte/25-million-images-0-99757-mnist), and 100% on the MNIST dataset. (by simply trying many many many options)\n- Sharing a [link](https://www.kaggle.com/cdeotte/mnist-perfect-100-using-knn) to a notebook that scores 100%.\n\n[Important Research Papers worth Mentioning](https://www.kaggle.com/competitions/digit-recognizer/discussion/161796) \nby [ishandutta](ishandutta)\n\n- Sharing a list of research papers about handwritten digit recognition.\n- [Improved method of handwritten digit recognition tested on MNIST database](https://www.sciencedirect.com/science/article/abs/pii/S0262885604000721)\n- [The MNIST Database of Handwritten Digit Images for Machine Learning Research](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/MNIST-SPM2012.pdf)\n- ]Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition](https://arxiv.org/pdf/1003.0358.pdf)\n- [A novel hybrid CNN–SVM classifier for recognizing handwritten digits](https://www.sciencedirect.com/science/article/abs/pii/S0031320311004006)\n- [Deep Big Multilayer Perceptrons For Digit Recognition](http://people.idsia.ch/~juergen/NNtricks.pdf)\n- [Better digit recognition with a committee of simple Neural Nets](http://www.iapr-tc11.org/archive/icdar2011/fileup/PDF/4520b250.pdf)\n- [Echo State Networks-based Reservoir Computing for MNIST Handwritten Digits Recognition](https://hal.archives-ouvertes.fr/hal-02131170/document)\n- [Extraction Method of Handwritten Digit Recognition Tested on the MNIST Database](https://www.researchgate.net/profile/Badre_Eddine_El_Kessab2/publication/324845327_Extraction_Method_of_Handwritten_Digit_Recognition_Tested_on_the_MNIST_Database/links/5ae7914ea6fdcc03cd8db36d/Extraction-Method-of-Handwritten-Digit-Recognition-Tested-on-the-MNIST-Database.pdf)\n- [Handwritten Digit Recognition Using K-Nearest Neighbour Classifier](https://ieeexplore.ieee.org/abstract/document/6755106)\n\n[A beginner guide to MNIST - 0.9925%](https://www.kaggle.com/competitions/digit-recognizer/discussion/93002) \nby [shaygu](shaygu)\n\n- Sharing a [public notebook](https://www.kaggle.com/shaygu/fast-cnn-for-beginners-0-9923) about a beginner guide to MNIST.\n\n[Reading in Data Directly to fast.ai](https://www.kaggle.com/competitions/digit-recognizer/discussion/80243) \nby [steventesta](steventesta)\n\n- Wrote a simple script for using fast.ai learner directly from the provided csv data.\n```python\nclass CustomImageItemList(ImageItemList):\n def open(self, fn):\n img = fn.reshape(28,28)\n img = np.stack((img,)*3, axis=-1) # convert to 3 channels\n return Image(pil2tensor(img, dtype=np.float32))\n\n @classmethod\n def from_csv_custom(cls, path:PathOrStr, csv_name:str, imgIdx:int=1, header:str='infer', **kwargs)->'ItemList':\n df = pd.read_csv(Path(path)/csv_name, header=header)\n res = super().from_df(df, path=path, cols=0, **kwargs)\n # convert pixels to an ndarray\n res.items = df.iloc[:,imgIdx:].apply(lambda x: x.values / 255.0, axis=1).values\n return res\n```\n\nAnd usage would be as follows:\n\n```python\ntest = CustomImageItemList.from_csv_custom(path='./data', csv_name='test.csv', imgIdx=0)\ndata = (CustomImageItemList.from_csv_custom(path='./data', csv_name='train.csv')\n .random_split_by_pct(.2)\n .label_from_df(cols='label')\n .add_test(test, label=0)\n .databunch(bs=64, num_workers=0))\n```\n\n- Sharing a [public notebook](https://www.kaggle.com/steventesta/digit-recognizer-fast-ai-custom-databunch) about reading in data directly to fast.ai.\n\n[Interactive MNIST Digits Recognition visual Tableau](https://www.kaggle.com/competitions/digit-recognizer/discussion/116424)\nby [zinovadr](zinovadr)\n\n- Sharing an [interactive MNIST Digits Recognition visual Tableau](https://public.tableau.com/profile/adrian.zinovei#!/vizhome/MNIST-DigitsAnalyzerKAGGLE/MNISTANALIZER)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2986379%2F3ccf4c6875ae59ff5a3073c67e075476%2FMNIST%20Digits.gif?generation=1573259567531991&alt=media)\n\n", "votes": 11 }, { "id": 1857699, "postDate": "2022-07-16T09:56:17.067Z", "content": "

GReat summary @thedevastator

", "rawMarkdown": "GReat summary @thedevastator " }, { "id": 1922505, "postDate": "2022-09-01T14:29:53.013Z", "content": "

great. thanks

", "rawMarkdown": "great. thanks" }, { "id": 1877032, "postDate": "2022-07-30T09:29:31.423Z", "content": "

great. thank you

", "rawMarkdown": "great. thank you" } ], "comments": [ { "id": 1857699, "author_name": "sanjay lalwani", "author_url": "", "post_date": "2022-07-16T09:56:17.067000", "content": "

GReat summary @thedevastator

", "votes": 0, "replies": [] }, { "id": 1922505, "author_name": "sheqian", "author_url": "", "post_date": "2022-09-01T14:29:53.013000", "content": "

great. thanks

", "votes": 0, "replies": [] }, { "id": 1877032, "author_name": "Mhd Modar Halimeh", "author_url": "", "post_date": "2022-07-30T09:29:31.423000", "content": "

great. thank you

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1828301": "### [Digit Recognizer] Summary of all the top discussions posts \n\n\n[Are 100% accuracies fake?](https://www.kaggle.com/competitions/digit-recognizer/discussion/183700) \nby [alifrahman](alifrahman)\n\n- A discussion about solutions with 100% acc: They are using the external public MNIST\n- [Here](https://www.kaggle.com/bgmello/how-top-lb-got-their-score-use-mnist-to-learn) is a notebook that shows how to do this\n\n[kNN vs. Neural Network](https://www.kaggle.com/competitions/digit-recognizer/discussion/60305) \nby [cdeotte](cdeotte)\n\n- A discussion on why a neural network outperforms kNN.\n- From the post: \"The problem is that kNN can not learn that pixel #5 is very important and pixel #13 is misleading. A neural network would learn this quickly and classify 100% of the digits correctly.\"\n\n[simple keras with lb 0.99071](https://www.kaggle.com/competitions/digit-recognizer/discussion/26378)\nby [china1000](china1000)\n\n- Sharing an extremely simple keras model (on the discussion message, not a notebook), here it is minimized a bit:\n```python\n\nimport numpy as np\nimport pandas as pd\nfrom keras import backend as K\nfrom keras.models import Sequential\nfrom keras.layers.convolutional import *\nfrom keras.callbacks import EarlyStopping\nfrom keras.utils.np_utils import to_categorical\nfrom keras.layers.core import Dropout, Dense, Flatten, Activation\n\nK.set_image_dim_ordering('th')\n\ntrain_df, valid_df = pd.read_csv(\"../input/train.csv\"), pd.read_csv(\"../input/test.csv\")\n\nx_train = train_df.drop(['label'], axis=1).values.astype('float32')\nY_train = train_df['label'].values.astype('float32')\nx_valid = valid_df.values.astype('float32')\n\nn_train, n_valid = x_train.shape[0], x_valid.shape[0]\n\nx_train, x_valid = x_train.reshape(n_train,1, 28, 28), x_valid.reshape(n_valid, 1, 28, 28)\nx_train, x_valid = (x_train / 255), (x_valid/255)\n\nmodel = Sequential()\nmodel.add(Convolution2D(64, 3, 3, batch_input_shape=(None, 1, 28, 28), activation='relu', border_mode='valid'))\nmodel.add(MaxPooling2D(pool_size=(3, 3)))\nmodel.add(Convolution2D(64, 2, 2, activation='relu', border_mode='valid'))\nmodel.add(MaxPooling2D(pool_size=(1, 1)))\nmodel.add(Dropout(0.25))\nmodel.add(Flatten())\nmodel.add(Dense(128))\nmodel.add(Activation('relu'))\nmodel.add(Dropout(0.5))\nmodel.add(Dense(10))\nmodel.add(Activation('softmax'))\nmodel.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\n\nearly_stopping = EarlyStopping(monitor='val_acc', patience=1)\nmodel.fit(x_train, to_categorical(Y_train), batch_size = 128, nb_epoch=100, verbose = 2, validation_split = .2)\nyPred = model.predict_classes(x_valid,batch_size=32,verbose=1)\nnp.savetxt('mnist_output.csv', np.c_[range(1,len(yPred)+1),yPred], delimiter=',', header = 'ImageId,Label', comments = '', fmt='%d')\n```\n\n- Sharing a [public notebook](https://www.kaggle.com/danielelton/digit-recognizer/keras-cnn-with-explanation/notebook) with a simple keras model that got a public LB score of 0.99071.\n\n[98.83% with Weka, new NN package](https://www.kaggle.com/competitions/digit-recognizer/discussion/7221) \nby [johannesamten](johannesamten)\n\n- The package was used to get 98.83% accuracy.\n- Sharing a [Weka package](https://github.com/amten/NeuralNetwork) for a neural network with dropout and rectified linear units in java. [wow this is an old post..]\n\n[The way to get 99.571% accuracy with simple model using keras](https://www.kaggle.com/competitions/digit-recognizer/discussion/62976)\nby [yugkhanna](yugkhanna)\n\n- Sharing a [medium article](https://medium.com/@yugkhanna1/building-a-5-layer-cnn-model-with-keras-to-identify-handwritten-digits-complex-yet-simple-f06b8b375346) about how to get 99.571% accuracy with a simple model using keras.\n\n[Applications of Digit recognizer: Resume tricks](https://www.kaggle.com/competitions/digit-recognizer/discussion/252787) \nby [kritidoneria](kritidoneria)\n\n- The post suggest that the same infoprmation can expressed in different words and sound much more impressive as a CV line. From the post:\n\n```\n**Writing 'Digit recognizer' and 'MNIST' in your resume is cliché. Instead, try these:**\n- Bank cheque/invoices/documents processing using OCR (quantify the impact in terms of man hours having)\n- Document verification system ( OCR and ID verification)\n- Examination grading by digitization MCQ only)\n- Scientific notes digitizer\n- License-plate reading\n```\n\n[Sharing Python Code for KNN, SVC and Random Forest](https://www.kaggle.com/competitions/digit-recognizer/discussion/4281)\nby [broham](broham)\n\n- Sharing a [GitHub repo](https://github.com/Broham/DigitRecognizer) with Python code for KNN, SVC and Random Forest.\n\n[Downloading train.csv and test.csv](https://www.kaggle.com/competitions/digit-recognizer/discussion/2983) \nby [swethasharma](swethasharma)\n\n- The post discusses a downloading problem that happened before, on the comments it is suggested that the files had been reuploaded [here](https://storage.googleapis.com/kaggle-forum-message-attachments/263398/8134/kaggle-digit-recognizer-data.tar.gz) as well if there is any problem in the future.\n\n[Simple tensorflow CNN - +99%](https://www.kaggle.com/competitions/digit-recognizer/discussion/43349)\nby [julienknafou](julienknafou)\n\n- Sharing a top 1% solution both in the post itself and on [GitHub](https://github.com/KubaMichalczyk/kaggle-predict-future-sales)\n\n[Getting Started - Python Sample Code (Random Forest)](https://www.kaggle.com/competitions/digit-recognizer/discussion/2299) \nby [cclark](cclark)\n\n- Sharing a simple solution to generate the Random Forest benchmark\n```python\nfrom numpy import genfromtxt, savetxt\nfrom sklearn.ensemble import RandomForestClassifier\n\ndef main():\n #create the training & test sets, skipping the header row with [1:]\n dataset = genfromtxt(open('Data/train.csv','r'), delimiter=',', dtype='f8')[1:]\n target = [x[0] for x in dataset]\n train = [x[1:] for x in dataset]\n test = genfromtxt(open('Data/test.csv','r'), delimiter=',', dtype='f8')[1:]\n rf = RandomForestClassifier(n_estimators=100)\n rf.fit(train, target)\n savetxt('Data/submission2.csv', rf.predict(test), delimiter=',', fmt='%f')\n\n```\n\n[What is a good score?](https://www.kaggle.com/competitions/digit-recognizer/discussion/163676)\nby [sanjaykrishnamurthi](sanjaykrishnamurthi)\n\n- Asking what a good score is in the Digit Recognizer competition.\n- From the comments:\n- 'I think if you can cross 99.8% accuracy, it should be good enough. But, with further tweaks like using callback function, augmenting data and using deeper models, you can easily get the accuracy over 99.9%.'\n- see [this](https://www.kaggle.com/c/digit-recognizer/discussion/61480) post\n\n[You might be cheating without knowing it](https://www.kaggle.com/competitions/digit-recognizer/discussion/296110) \nby [samuelcortinhas](samuelcortinhas)\n\n- From the post: '''\nin the top scoring notebooks:\n\n(X_train_mnist, y_train_mnist), (X_val_mnist, y_val_mnist) = mnist.load_data()\n\nkaggle uses a training/test split of 42k/28k whereas this dataset downloaded from keras has a 60k/10k split. This means you will be using some of kaggles test data to train your algorithm, which artificially improves your score because your algorithm is just memorising a lot of the right answers.\n'''\n\n[How to score 97%, 98%, 99%, and 100%](https://www.kaggle.com/competitions/digit-recognizer/discussion/61480) \nby [cdeotte](cdeotte)\n\n- Sharing a histogram of Kaggle leaderboard scores where each bar has range 0.1%.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1723677%2Fb858f6500a16f9a0e3e23ee9126b0069%2Fimg1.png?generation=1602506040379499&alt=media)\n- Sharing a schematic of a CNN architecture.\n- A discussion on how to score 97%, 98%, [99%](https://www.kaggle.com/cdeotte/25-million-images-0-99757-mnist), and 100% on the MNIST dataset. (by simply trying many many many options)\n- Sharing a [link](https://www.kaggle.com/cdeotte/mnist-perfect-100-using-knn) to a notebook that scores 100%.\n\n[Important Research Papers worth Mentioning](https://www.kaggle.com/competitions/digit-recognizer/discussion/161796) \nby [ishandutta](ishandutta)\n\n- Sharing a list of research papers about handwritten digit recognition.\n- [Improved method of handwritten digit recognition tested on MNIST database](https://www.sciencedirect.com/science/article/abs/pii/S0262885604000721)\n- [The MNIST Database of Handwritten Digit Images for Machine Learning Research](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/MNIST-SPM2012.pdf)\n- ]Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition](https://arxiv.org/pdf/1003.0358.pdf)\n- [A novel hybrid CNN–SVM classifier for recognizing handwritten digits](https://www.sciencedirect.com/science/article/abs/pii/S0031320311004006)\n- [Deep Big Multilayer Perceptrons For Digit Recognition](http://people.idsia.ch/~juergen/NNtricks.pdf)\n- [Better digit recognition with a committee of simple Neural Nets](http://www.iapr-tc11.org/archive/icdar2011/fileup/PDF/4520b250.pdf)\n- [Echo State Networks-based Reservoir Computing for MNIST Handwritten Digits Recognition](https://hal.archives-ouvertes.fr/hal-02131170/document)\n- [Extraction Method of Handwritten Digit Recognition Tested on the MNIST Database](https://www.researchgate.net/profile/Badre_Eddine_El_Kessab2/publication/324845327_Extraction_Method_of_Handwritten_Digit_Recognition_Tested_on_the_MNIST_Database/links/5ae7914ea6fdcc03cd8db36d/Extraction-Method-of-Handwritten-Digit-Recognition-Tested-on-the-MNIST-Database.pdf)\n- [Handwritten Digit Recognition Using K-Nearest Neighbour Classifier](https://ieeexplore.ieee.org/abstract/document/6755106)\n\n[A beginner guide to MNIST - 0.9925%](https://www.kaggle.com/competitions/digit-recognizer/discussion/93002) \nby [shaygu](shaygu)\n\n- Sharing a [public notebook](https://www.kaggle.com/shaygu/fast-cnn-for-beginners-0-9923) about a beginner guide to MNIST.\n\n[Reading in Data Directly to fast.ai](https://www.kaggle.com/competitions/digit-recognizer/discussion/80243) \nby [steventesta](steventesta)\n\n- Wrote a simple script for using fast.ai learner directly from the provided csv data.\n```python\nclass CustomImageItemList(ImageItemList):\n def open(self, fn):\n img = fn.reshape(28,28)\n img = np.stack((img,)*3, axis=-1) # convert to 3 channels\n return Image(pil2tensor(img, dtype=np.float32))\n\n @classmethod\n def from_csv_custom(cls, path:PathOrStr, csv_name:str, imgIdx:int=1, header:str='infer', **kwargs)->'ItemList':\n df = pd.read_csv(Path(path)/csv_name, header=header)\n res = super().from_df(df, path=path, cols=0, **kwargs)\n # convert pixels to an ndarray\n res.items = df.iloc[:,imgIdx:].apply(lambda x: x.values / 255.0, axis=1).values\n return res\n```\n\nAnd usage would be as follows:\n\n```python\ntest = CustomImageItemList.from_csv_custom(path='./data', csv_name='test.csv', imgIdx=0)\ndata = (CustomImageItemList.from_csv_custom(path='./data', csv_name='train.csv')\n .random_split_by_pct(.2)\n .label_from_df(cols='label')\n .add_test(test, label=0)\n .databunch(bs=64, num_workers=0))\n```\n\n- Sharing a [public notebook](https://www.kaggle.com/steventesta/digit-recognizer-fast-ai-custom-databunch) about reading in data directly to fast.ai.\n\n[Interactive MNIST Digits Recognition visual Tableau](https://www.kaggle.com/competitions/digit-recognizer/discussion/116424)\nby [zinovadr](zinovadr)\n\n- Sharing an [interactive MNIST Digits Recognition visual Tableau](https://public.tableau.com/profile/adrian.zinovei#!/vizhome/MNIST-DigitsAnalyzerKAGGLE/MNISTANALIZER)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2986379%2F3ccf4c6875ae59ff5a3073c67e075476%2FMNIST%20Digits.gif?generation=1573259567531991&alt=media)\n\n", "1857699": "GReat summary @thedevastator ", "1922505": "great. thanks", "1877032": "great. thank you" } }