After the correct rate reaches 98%, the model becomes weak. No matter how I modify the model, whether it is data enhancement, adding regularization or Dropout is not significant. I want to know if there is any more advanced method to improve the accuracy rate besides these? Please enlighten me.
", "messages": [ { "id": "674766", "postDate": "11/17/2019 03:26:52", "content": "After the correct rate reaches 98%, the model becomes weak. No matter how I modify the model, whether it is data enhancement, adding regularization or Dropout is not significant. I want to know if there is any more advanced method to improve the accuracy rate besides these? Please enlighten me.
", "rawMarkdown": "After the correct rate reaches 98%, the model becomes weak. No matter how I modify the model, whether it is data enhancement, adding regularization or Dropout is not significant. I want to know if there is any more advanced method to improve the accuracy rate besides these? Please enlighten me.", "votes": null }, { "id": "676618", "postDate": "11/19/2019 11:05:51", "content": "hi, you may find this paper useful to read.\nI also highly recommend to take a look at this fine kernel by Chris Deotte.\nFor the Kannada-MNIST competition I am currently using a quite simple model you may find in here .\nWith regards the basic hyper parameters I found only one which helped and this is the batch size. Playing with others did not make significant improvement.\nFurther tricks are the followings: number of epochs, ensemble predicting, usage of the provided test set.\nhope this helps.\nN
", "rawMarkdown": "hi, you may find [this paper](https://arxiv.org/abs/1812.01187?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%253A+arxiv%252FQSXk+%2528ExcitingAds%2521+cs+updates+on+arXiv.org%2529) useful to read.\nI also highly recommend to take a look at [this fine kernel](https://www.kaggle.com/cdeotte/25-million-images-0-99757-mnist) by Chris Deotte.\nFor the Kannada-MNIST competition I am currently using a quite simple model you may find in [here](https://www.kaggle.com/nandor65/c3c3c5c5c7c7c4) .\nWith regards the basic hyper parameters I found only one which helped and this is the batch size. Playing with others did not make significant improvement.\nFurther tricks are the followings: number of epochs, ensemble predicting, usage of the provided test set.\nhope this helps.\nN", "votes": null }, { "id": "678952", "postDate": "11/22/2019 05:13:09", "content": "It helps me a lot. Thank you very much. 😄
", "rawMarkdown": "It helps me a lot. Thank you very much. 😄", "votes": null }, { "id": "680974", "postDate": "11/25/2019 13:23:52", "content": "What do you mean by \"the provided test set\"? Are you referring to the additional Dig-MNIST file?
", "rawMarkdown": "What do you mean by \"the provided test set\"? Are you referring to the additional Dig-MNIST file?", "votes": null }, { "id": "681018", "postDate": "11/25/2019 14:38:01", "content": "I meant the test.csv with 5000 test samples. I think the method is called pseudo labelling and you may find an example kernel here. Meanwhile I added a simple kernel to the site which contains exact same CNN structure which I use in this competition. Pseudo labelling technique should add at least 0.0010 points to your accuracy which does not look like a big number. But having this small figure added to your original accuracy (without pseudo labelling) that may well move your position up on the board.
", "rawMarkdown": "I meant the test.csv with 5000 test samples. I think the method is called pseudo labelling and you may find an example kernel [here](https://www.kaggle.com/jinbao/kannada-mnist-baseline). Meanwhile I added a simple [kernel](https://www.kaggle.com/nandor65/kmnist-trio) to the site which contains exact same CNN structure which I use in this competition. Pseudo labelling technique should add at least 0.0010 points to your accuracy which does not look like a big number. But having this small figure added to your original accuracy (without pseudo labelling) that may well move your position up on the board.", "votes": null }, { "id": "684186", "postDate": "11/29/2019 11:23:18", "content": "Indeed, it's called pseudo labeling, I have found couple of interesting articles about this. Thanks a lot! But how about samples from Dig-MNIST? Do you find them useful as well or are those someway differnet from train/test samples and e.g. introduce more noise or whatnot?
", "rawMarkdown": "Indeed, it's called pseudo labeling, I have found couple of interesting articles about this. Thanks a lot! But how about samples from Dig-MNIST? Do you find them useful as well or are those someway differnet from train/test samples and e.g. introduce more noise or whatnot?", "votes": null }, { "id": "684215", "postDate": "11/29/2019 12:29:37", "content": "hi, I have no any experience with Dig-MNIST set. For training I use train set, then do a pseudo labelling on the test set and then concatenate train and pseudo labelled test set and then I continue training a few epochs more.
", "rawMarkdown": "hi, I have no any experience with Dig-MNIST set. For training I use train set, then do a pseudo labelling on the test set and then concatenate train and pseudo labelled test set and then I continue training a few epochs more.", "votes": null } ], "comments": [ { "id": 676618, "author_name": "nandor65", "author_url": "", "post_date": "11/19/2019 11:05:51", "content": "hi, you may find this paper useful to read.\nI also highly recommend to take a look at this fine kernel by Chris Deotte.\nFor the Kannada-MNIST competition I am currently using a quite simple model you may find in here .\nWith regards the basic hyper parameters I found only one which helped and this is the batch size. Playing with others did not make significant improvement.\nFurther tricks are the followings: number of epochs, ensemble predicting, usage of the provided test set.\nhope this helps.\nN
", "votes": null, "replies": [ { "id": 678952, "author_name": "yonminma", "author_url": "", "post_date": "11/22/2019 05:13:09", "content": "It helps me a lot. Thank you very much. 😄
", "votes": null, "replies": [] }, { "id": 680974, "author_name": "mtszkw", "author_url": "", "post_date": "11/25/2019 13:23:52", "content": "What do you mean by \"the provided test set\"? Are you referring to the additional Dig-MNIST file?
", "votes": null, "replies": [] }, { "id": 681018, "author_name": "nandor65", "author_url": "", "post_date": "11/25/2019 14:38:01", "content": "I meant the test.csv with 5000 test samples. I think the method is called pseudo labelling and you may find an example kernel here. Meanwhile I added a simple kernel to the site which contains exact same CNN structure which I use in this competition. Pseudo labelling technique should add at least 0.0010 points to your accuracy which does not look like a big number. But having this small figure added to your original accuracy (without pseudo labelling) that may well move your position up on the board.
", "votes": null, "replies": [] }, { "id": 684186, "author_name": "mtszkw", "author_url": "", "post_date": "11/29/2019 11:23:18", "content": "Indeed, it's called pseudo labeling, I have found couple of interesting articles about this. Thanks a lot! But how about samples from Dig-MNIST? Do you find them useful as well or are those someway differnet from train/test samples and e.g. introduce more noise or whatnot?
", "votes": null, "replies": [] }, { "id": 684215, "author_name": "nandor65", "author_url": "", "post_date": "11/29/2019 12:29:37", "content": "hi, I have no any experience with Dig-MNIST set. For training I use train set, then do a pseudo labelling on the test set and then concatenate train and pseudo labelled test set and then I continue training a few epochs more.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "674766": "After the correct rate reaches 98%, the model becomes weak. No matter how I modify the model, whether it is data enhancement, adding regularization or Dropout is not significant. I want to know if there is any more advanced method to improve the accuracy rate besides these? Please enlighten me.", "676618": "hi, you may find [this paper](https://arxiv.org/abs/1812.01187?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%253A+arxiv%252FQSXk+%2528ExcitingAds%2521+cs+updates+on+arXiv.org%2529) useful to read.\nI also highly recommend to take a look at [this fine kernel](https://www.kaggle.com/cdeotte/25-million-images-0-99757-mnist) by Chris Deotte.\nFor the Kannada-MNIST competition I am currently using a quite simple model you may find in [here](https://www.kaggle.com/nandor65/c3c3c5c5c7c7c4) .\nWith regards the basic hyper parameters I found only one which helped and this is the batch size. Playing with others did not make significant improvement.\nFurther tricks are the followings: number of epochs, ensemble predicting, usage of the provided test set.\nhope this helps.\nN", "678952": "It helps me a lot. Thank you very much. 😄", "680974": "What do you mean by \"the provided test set\"? Are you referring to the additional Dig-MNIST file?", "681018": "I meant the test.csv with 5000 test samples. I think the method is called pseudo labelling and you may find an example kernel [here](https://www.kaggle.com/jinbao/kannada-mnist-baseline). Meanwhile I added a simple [kernel](https://www.kaggle.com/nandor65/kmnist-trio) to the site which contains exact same CNN structure which I use in this competition. Pseudo labelling technique should add at least 0.0010 points to your accuracy which does not look like a big number. But having this small figure added to your original accuracy (without pseudo labelling) that may well move your position up on the board.", "684186": "Indeed, it's called pseudo labeling, I have found couple of interesting articles about this. Thanks a lot! But how about samples from Dig-MNIST? Do you find them useful as well or are those someway differnet from train/test samples and e.g. introduce more noise or whatnot?", "684215": "hi, I have no any experience with Dig-MNIST set. For training I use train set, then do a pseudo labelling on the test set and then concatenate train and pseudo labelled test set and then I continue training a few epochs more." }, "source": "meta" }