{
  "id": 372749,
  "title": "Low evaluation accuracy but high training and validation accuracy",
  "url": "/competitions/digit-recognizer/discussion/372749",
  "author_name": "Harry Wang",
  "post_date": "2022-12-17T18:49:32.143000",
  "votes": 1,
  "comment_count": 2,
  "views": null,
  "content": "<p>Hello everyone,</p>\n<p>During training my CNN reaches training accuracy of 99.9+% and validation accuracy of 98+%. But the test accuracy during submission is only 10%. I've printed out a few test digits and my model all correctly identifies them. Is there anything that I need to do before submission or does anyone know what is going on here? Thanks!</p>",
  "messages": [
    {
      "id": 2068333,
      "postDate": "2022-12-17T18:49:32.143Z",
      "content": "<p>Hello everyone,</p>\n<p>During training my CNN reaches training accuracy of 99.9+% and validation accuracy of 98+%. But the test accuracy during submission is only 10%. I've printed out a few test digits and my model all correctly identifies them. Is there anything that I need to do before submission or does anyone know what is going on here? Thanks!</p>",
      "rawMarkdown": "Hello everyone,\n\nDuring training my CNN reaches training accuracy of 99.9+% and validation accuracy of 98+%. But the test accuracy during submission is only 10%. I've printed out a few test digits and my model all correctly identifies them. Is there anything that I need to do before submission or does anyone know what is going on here? Thanks!",
      "votes": 1
    },
    {
      "id": 2074449,
      "postDate": "2022-12-24T07:31:23.103Z",
      "content": "<p>My suggestion would be to rescale your train and test data. By rescaling I mean dividing your train and test data by 255. It would allow all the pixel values to fall in the range between 0 and 1. Just by rescaling and tuning some hyperparameters, I have achieved 99% accuracy on train data and 98.80% on test data. But you can definitely try data augmentation or any other techniques you can think of.</p>",
      "rawMarkdown": "My suggestion would be to rescale your train and test data. By rescaling I mean dividing your train and test data by 255. It would allow all the pixel values to fall in the range between 0 and 1. Just by rescaling and tuning some hyperparameters, I have achieved 99% accuracy on train data and 98.80% on test data. But you can definitely try data augmentation or any other techniques you can think of."
    },
    {
      "id": 2071582,
      "postDate": "2022-12-21T07:39:56.793Z",
      "content": "<p>As per my experience, your model seems to be getting biased for the training dataset. So , when the model is trying to predict the test data (which is not seen by the model), it is unable to predict it.<br>\nSee if this helps:<br>\n<a href=\"https://arxiv.org/ftp/arxiv/papers/2106/2106.14829.pdf\" target=\"_blank\">https://arxiv.org/ftp/arxiv/papers/2106/2106.14829.pdf</a></p>\n<p>Also, in the test dataset, the images have rotation and different angels. That makes the test data very different from the training data. Because of the change in pixel position, the model is unable to predict the labels.<br>\nTo solve this issue, use data augmentation techniques to produce similar data from training dataset.<br>\nCheck this out:<br>\n<a href=\"https://www.kaggle.com/code/foucardm/data-augmentation-on-the-mnist-dataset/notebook\" target=\"_blank\">https://www.kaggle.com/code/foucardm/data-augmentation-on-the-mnist-dataset/notebook</a></p>",
      "rawMarkdown": "As per my experience, your model seems to be getting biased for the training dataset. So , when the model is trying to predict the test data (which is not seen by the model), it is unable to predict it.\nSee if this helps:\nhttps://arxiv.org/ftp/arxiv/papers/2106/2106.14829.pdf\n\nAlso, in the test dataset, the images have rotation and different angels. That makes the test data very different from the training data. Because of the change in pixel position, the model is unable to predict the labels.\nTo solve this issue, use data augmentation techniques to produce similar data from training dataset.\nCheck this out:\nhttps://www.kaggle.com/code/foucardm/data-augmentation-on-the-mnist-dataset/notebook"
    }
  ],
  "comments": [
    {
      "id": 2074449,
      "author_name": "Pranaw Mishra",
      "author_url": "",
      "post_date": "2022-12-24T07:31:23.103000",
      "content": "<p>My suggestion would be to rescale your train and test data. By rescaling I mean dividing your train and test data by 255. It would allow all the pixel values to fall in the range between 0 and 1. Just by rescaling and tuning some hyperparameters, I have achieved 99% accuracy on train data and 98.80% on test data. But you can definitely try data augmentation or any other techniques you can think of.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2071582,
      "author_name": "Ankit Modak",
      "author_url": "",
      "post_date": "2022-12-21T07:39:56.793000",
      "content": "<p>As per my experience, your model seems to be getting biased for the training dataset. So , when the model is trying to predict the test data (which is not seen by the model), it is unable to predict it.<br>\nSee if this helps:<br>\n<a href=\"https://arxiv.org/ftp/arxiv/papers/2106/2106.14829.pdf\" target=\"_blank\">https://arxiv.org/ftp/arxiv/papers/2106/2106.14829.pdf</a></p>\n<p>Also, in the test dataset, the images have rotation and different angels. That makes the test data very different from the training data. Because of the change in pixel position, the model is unable to predict the labels.<br>\nTo solve this issue, use data augmentation techniques to produce similar data from training dataset.<br>\nCheck this out:<br>\n<a href=\"https://www.kaggle.com/code/foucardm/data-augmentation-on-the-mnist-dataset/notebook\" target=\"_blank\">https://www.kaggle.com/code/foucardm/data-augmentation-on-the-mnist-dataset/notebook</a></p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2068333": "Hello everyone,\n\nDuring training my CNN reaches training accuracy of 99.9+% and validation accuracy of 98+%. But the test accuracy during submission is only 10%. I've printed out a few test digits and my model all correctly identifies them. Is there anything that I need to do before submission or does anyone know what is going on here? Thanks!",
    "2074449": "My suggestion would be to rescale your train and test data. By rescaling I mean dividing your train and test data by 255. It would allow all the pixel values to fall in the range between 0 and 1. Just by rescaling and tuning some hyperparameters, I have achieved 99% accuracy on train data and 98.80% on test data. But you can definitely try data augmentation or any other techniques you can think of.",
    "2071582": "As per my experience, your model seems to be getting biased for the training dataset. So , when the model is trying to predict the test data (which is not seen by the model), it is unable to predict it.\nSee if this helps:\nhttps://arxiv.org/ftp/arxiv/papers/2106/2106.14829.pdf\n\nAlso, in the test dataset, the images have rotation and different angels. That makes the test data very different from the training data. Because of the change in pixel position, the model is unable to predict the labels.\nTo solve this issue, use data augmentation techniques to produce similar data from training dataset.\nCheck this out:\nhttps://www.kaggle.com/code/foucardm/data-augmentation-on-the-mnist-dataset/notebook"
  }
}