{ "id": 12336, "title": "Really low accuracy using R, caret, and nnet", "url": "/competitions/digit-recognizer/discussion/12336", "author_name": "blackfist", "post_date": "2015-01-26T16:30:12.347000", "votes": 0, "comment_count": 2, "views": null, "content": "

I have gone through the data using KNN and came out with an accuracy of around 96%. I decided to try experimenting with neural networks because I saw in one of the threads someone said that those are typically better for handwriting analysis. However, my resulting code is only about 36% accurate. I was wondering if someone could identify what I might be doing wrong.

\n

library("caret")
library(randomForest)

\n

library("doMC")

\n

# I want to leave some cores for other things.
doMC::registerDoMC(cores=detectCores() -3 )

\n

inputData <- read.csv("train.csv")
testData <- read.csv("test.csv")

\n

# Cleaning and tweaking
inputData$label <- as.factor(inputData$label)

\n

# Partition the training data
randomSelect <- createDataPartition(inputData$label, p=0.7, list=F)
modelTrain <- inputData[randomSelect, ]
modelTest <- inputData[-randomSelect, ]

\n

# Consider removing columns with near zero variance
nzv <- nearZeroVar(modelTrain)
modelTrain <- modelTrain[,-nzv]

\n

if(!file.exists("nnModel.Rda")) {
nnStart <- Sys.time()
nnModel <- train(label ~ ., data=modelTrain, method="nnet")
nnEnd <- Sys.time()
save(nnModel, file="nnModel.Rda")
}

\n

nnPredictions <- predict(nnModel, modelTest)

\n

confusionMatrix(modelTest$label, nnPredictions)$overall

\n


Accuracy Kappa AccuracyLower AccuracyUpper AccuracyNull AccuracyPValue McnemarPValue
0.3655129 0.2913060 0.3570951 0.3739928 0.4526040 1.0000000 NaN

", "messages": [ { "id": 110620, "postDate": "2016-03-07T02:08:01.477Z", "content": "

Try to normalize the data before you train the nerual network.

", "rawMarkdown": "Try to normalize the data before you train the nerual network." }, { "id": 63224, "postDate": "2015-01-27T03:48:27.423Z", "content": "

I don't know the package you used. From what I know, there are two common neural network packages in R: neuralnet and nnet. 

\n

I used nnet and got around 98%. I recommend you understand how neural net works before applying them. 

\n

I wrote a blog about recourses I find. You can take it look here.

" }, { "id": 63192, "postDate": "2015-01-26T16:30:12.347Z", "content": "

I have gone through the data using KNN and came out with an accuracy of around 96%. I decided to try experimenting with neural networks because I saw in one of the threads someone said that those are typically better for handwriting analysis. However, my resulting code is only about 36% accurate. I was wondering if someone could identify what I might be doing wrong.

\n

library("caret")
library(randomForest)

\n

library("doMC")

\n

# I want to leave some cores for other things.
doMC::registerDoMC(cores=detectCores() -3 )

\n

inputData <- read.csv("train.csv")
testData <- read.csv("test.csv")

\n

# Cleaning and tweaking
inputData$label <- as.factor(inputData$label)

\n

# Partition the training data
randomSelect <- createDataPartition(inputData$label, p=0.7, list=F)
modelTrain <- inputData[randomSelect, ]
modelTest <- inputData[-randomSelect, ]

\n

# Consider removing columns with near zero variance
nzv <- nearZeroVar(modelTrain)
modelTrain <- modelTrain[,-nzv]

\n

if(!file.exists("nnModel.Rda")) {
nnStart <- Sys.time()
nnModel <- train(label ~ ., data=modelTrain, method="nnet")
nnEnd <- Sys.time()
save(nnModel, file="nnModel.Rda")
}

\n

nnPredictions <- predict(nnModel, modelTest)

\n

confusionMatrix(modelTest$label, nnPredictions)$overall

\n


Accuracy Kappa AccuracyLower AccuracyUpper AccuracyNull AccuracyPValue McnemarPValue
0.3655129 0.2913060 0.3570951 0.3739928 0.4526040 1.0000000 NaN

" } ], "comments": [ { "id": 110620, "author_name": "Fangyu Ding", "author_url": "", "post_date": "2016-03-07T02:08:01.477000", "content": "

Try to normalize the data before you train the nerual network.

", "votes": 0, "replies": [] }, { "id": 63224, "author_name": "", "author_url": "", "post_date": "2015-01-27T03:48:27.423000", "content": "

I don't know the package you used. From what I know, there are two common neural network packages in R: neuralnet and nnet. 

\n

I used nnet and got around 98%. I recommend you understand how neural net works before applying them. 

\n

I wrote a blog about recourses I find. You can take it look here.

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "110620": "Try to normalize the data before you train the nerual network.", "63224": "", "63192": "" } }