Dear All,
\n\nTo begin with, thank you so very much for all your efforts!\nI am deeply humbled by all of your contributions and the conversations in the forums.\nAt this juncture, I thought I'd share a few points, some of which might have been missed in the paper.
\n\n1: Raw grid-scan sheets\nICYMI, I have open-sourced the raw grid scan sheets from which the digits were segmented out here: https://github.com/vinayprabhu/Kannada_MNIST/tree/master/data/input_scans/\nThis was done to empower you all to write your own digit extraction routine and perform end-to-end classification. Also, because some of these details/resources were amiss in the previous x-MNIST variants I had used before.
\n\n2: Regarding some digit images being 'cut-off' at the border:\nOne valuable feedback I received during the curation of the dataset was to retain these 'cut-off-image' idiosyncrasies in the dataset and not to 'over-clean' the dataset. \nThe reasons for this being:
\n\na) Real-world data is, in fact, almost always messy\nb) Classification accuracy is NOT always the holy grail to be chased after. \nc) Training NNs with messy/noisy input data (especially when the nature of corruption is well understood) brings in a new flavor of challenges that are worth pursuing, from both empirical fronts as well as the theoretical front (think self-supervised learning).\nd) Getting NNs to naturally make non-confident prediction when there is genuine fuzziness in the query image and distributing the softmax masses between the candidate classes in a way a human eye would react to in an important ML challenge
\n\n3: Dig-MNIST augmentation\nICYMI, Dig-MNIST was curated here in the US, with the help of volunteers who were non-native Kannada language speakers and who were seeing these digit-glyphs for the first time ever. This did result in a 'newbie effect' in terms of the smoothness of the curves constituting the digits etc.\nMere drag-and-drop augmentation of the extra-Dig-MNIST digit images will/might in fact, hurt the accuracy of the models you are training. I found this phenomenon fascinating and I can totally see a lot of interesting work coming out of this observation.
\n\n4: Binary neural networks\nI chanced upon this awesome library 'LARQ': https://larq.dev/\nand thought I'd share it with y'all in case you wanted to train and deploy parsimonious models (oft in the order of a few kB!).
\n\nLastly, thank you so very much for your participation and energy!\nKind regards!\n- Vinay
", "messages": [ { "id": "667143", "postDate": "11/06/2019 21:31:53", "content": "Dear All,
\n\nTo begin with, thank you so very much for all your efforts!\nI am deeply humbled by all of your contributions and the conversations in the forums.\nAt this juncture, I thought I'd share a few points, some of which might have been missed in the paper.
\n\n1: Raw grid-scan sheets\nICYMI, I have open-sourced the raw grid scan sheets from which the digits were segmented out here: https://github.com/vinayprabhu/Kannada_MNIST/tree/master/data/input_scans/\nThis was done to empower you all to write your own digit extraction routine and perform end-to-end classification. Also, because some of these details/resources were amiss in the previous x-MNIST variants I had used before.
\n\n2: Regarding some digit images being 'cut-off' at the border:\nOne valuable feedback I received during the curation of the dataset was to retain these 'cut-off-image' idiosyncrasies in the dataset and not to 'over-clean' the dataset. \nThe reasons for this being:
\n\na) Real-world data is, in fact, almost always messy\nb) Classification accuracy is NOT always the holy grail to be chased after. \nc) Training NNs with messy/noisy input data (especially when the nature of corruption is well understood) brings in a new flavor of challenges that are worth pursuing, from both empirical fronts as well as the theoretical front (think self-supervised learning).\nd) Getting NNs to naturally make non-confident prediction when there is genuine fuzziness in the query image and distributing the softmax masses between the candidate classes in a way a human eye would react to in an important ML challenge
\n\n3: Dig-MNIST augmentation\nICYMI, Dig-MNIST was curated here in the US, with the help of volunteers who were non-native Kannada language speakers and who were seeing these digit-glyphs for the first time ever. This did result in a 'newbie effect' in terms of the smoothness of the curves constituting the digits etc.\nMere drag-and-drop augmentation of the extra-Dig-MNIST digit images will/might in fact, hurt the accuracy of the models you are training. I found this phenomenon fascinating and I can totally see a lot of interesting work coming out of this observation.
\n\n4: Binary neural networks\nI chanced upon this awesome library 'LARQ': https://larq.dev/\nand thought I'd share it with y'all in case you wanted to train and deploy parsimonious models (oft in the order of a few kB!).
\n\nLastly, thank you so very much for your participation and energy!\nKind regards!\n- Vinay
", "rawMarkdown": "Dear All,\n\nTo begin with, thank you so very much for all your efforts!\nI am deeply humbled by all of your contributions and the conversations in the forums.\nAt this juncture, I thought I'd share a few points, some of which might have been missed in the paper.\n\n**1: Raw grid-scan sheets**\nICYMI, I have open-sourced the raw grid scan sheets from which the digits were segmented out here: https://github.com/vinayprabhu/Kannada_MNIST/tree/master/data/input_scans/\nThis was done to empower you all to write your own digit extraction routine and perform end-to-end classification. Also, because some of these details/resources were amiss in the previous x-MNIST variants I had used before.\n\n**2: Regarding some digit images being 'cut-off' at the border:**\nOne valuable feedback I received during the curation of the dataset was to retain these 'cut-off-image' idiosyncrasies in the dataset and not to 'over-clean' the dataset. \nThe reasons for this being:\n\na) Real-world data is, in fact, almost always messy\nb) Classification accuracy is *NOT* always the holy grail to be chased after. \nc) Training NNs with messy/noisy input data (especially when the nature of corruption is well understood) brings in a new flavor of challenges that are worth pursuing, from both empirical fronts as well as the theoretical front (think self-supervised learning).\nd) Getting NNs to naturally make non-confident prediction when there is genuine fuzziness in the query image and distributing the softmax masses between the candidate classes in a way a human eye would react to in an important ML challenge\n\n**3: Dig-MNIST augmentation**\nICYMI, Dig-MNIST was curated here in the US, with the help of volunteers who were non-native Kannada language speakers and who were seeing these digit-glyphs for the first time ever. This did result in a 'newbie effect' in terms of the smoothness of the curves constituting the digits etc.\nMere drag-and-drop augmentation of the extra-Dig-MNIST digit images will/might in fact, hurt the accuracy of the models you are training. I found this phenomenon fascinating and I can totally see a lot of interesting work coming out of this observation. \n\n**4: Binary neural networks**\nI chanced upon this awesome library 'LARQ': https://larq.dev/\nand thought I'd share it with y'all in case you wanted to train and deploy parsimonious models (oft in the order of a few kB!).\n\nLastly, thank you so very much for your participation and energy!\nKind regards!\n- Vinay", "votes": null }, { "id": "667159", "postDate": "11/06/2019 22:02:01", "content": "Thanks for creating and providing the dataset! It is great to get started with image classification or to explore what is going on inside the own architecture. \nPutting it here in a kaggle competition makes me certainly spend more time with the data and studying the basics than the original MNIST would have done.
", "rawMarkdown": "Thanks for creating and providing the dataset! It is great to get started with image classification or to explore what is going on inside the own architecture. \nPutting it here in a kaggle competition makes me certainly spend more time with the data and studying the basics than the original MNIST would have done.", "votes": null }, { "id": "668896", "postDate": "11/09/2019 03:22:53", "content": "Volunteers with 'newbie effect' = human augmenter 😜
", "rawMarkdown": "Volunteers with 'newbie effect' = human augmenter 😜", "votes": null }, { "id": "679276", "postDate": "11/22/2019 14:28:10", "content": "Hi Vinay,
\n\nThanks for this competition. It is really fun and I have learned a lot from it. Will you write any paper based on the results? How does the accuracy of the models compare to what you estimated? It seems like the top contenders have a really hard time pushing past 0.99-0.991.
\n\nI would wonder how a human expert would classify the images that the algos get wrong. Are they also impossible for a human reader to classify correctly?
", "rawMarkdown": "Hi Vinay,\n\nThanks for this competition. It is really fun and I have learned a lot from it. Will you write any paper based on the results? How does the accuracy of the models compare to what you estimated? It seems like the top contenders have a really hard time pushing past 0.99-0.991. \n\nI would wonder how a human expert would classify the images that the algos get wrong. Are they also impossible for a human reader to classify correctly?", "votes": null } ], "comments": [ { "id": 667159, "author_name": "joatom", "author_url": "", "post_date": "11/06/2019 22:02:01", "content": "Thanks for creating and providing the dataset! It is great to get started with image classification or to explore what is going on inside the own architecture. \nPutting it here in a kaggle competition makes me certainly spend more time with the data and studying the basics than the original MNIST would have done.
", "votes": null, "replies": [] }, { "id": 668896, "author_name": "awater1223", "author_url": "", "post_date": "11/09/2019 03:22:53", "content": "Volunteers with 'newbie effect' = human augmenter 😜
", "votes": null, "replies": [] }, { "id": 679276, "author_name": "rincewind007", "author_url": "", "post_date": "11/22/2019 14:28:10", "content": "Hi Vinay,
\n\nThanks for this competition. It is really fun and I have learned a lot from it. Will you write any paper based on the results? How does the accuracy of the models compare to what you estimated? It seems like the top contenders have a really hard time pushing past 0.99-0.991.
\n\nI would wonder how a human expert would classify the images that the algos get wrong. Are they also impossible for a human reader to classify correctly?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "667143": "Dear All,\n\nTo begin with, thank you so very much for all your efforts!\nI am deeply humbled by all of your contributions and the conversations in the forums.\nAt this juncture, I thought I'd share a few points, some of which might have been missed in the paper.\n\n**1: Raw grid-scan sheets**\nICYMI, I have open-sourced the raw grid scan sheets from which the digits were segmented out here: https://github.com/vinayprabhu/Kannada_MNIST/tree/master/data/input_scans/\nThis was done to empower you all to write your own digit extraction routine and perform end-to-end classification. Also, because some of these details/resources were amiss in the previous x-MNIST variants I had used before.\n\n**2: Regarding some digit images being 'cut-off' at the border:**\nOne valuable feedback I received during the curation of the dataset was to retain these 'cut-off-image' idiosyncrasies in the dataset and not to 'over-clean' the dataset. \nThe reasons for this being:\n\na) Real-world data is, in fact, almost always messy\nb) Classification accuracy is *NOT* always the holy grail to be chased after. \nc) Training NNs with messy/noisy input data (especially when the nature of corruption is well understood) brings in a new flavor of challenges that are worth pursuing, from both empirical fronts as well as the theoretical front (think self-supervised learning).\nd) Getting NNs to naturally make non-confident prediction when there is genuine fuzziness in the query image and distributing the softmax masses between the candidate classes in a way a human eye would react to in an important ML challenge\n\n**3: Dig-MNIST augmentation**\nICYMI, Dig-MNIST was curated here in the US, with the help of volunteers who were non-native Kannada language speakers and who were seeing these digit-glyphs for the first time ever. This did result in a 'newbie effect' in terms of the smoothness of the curves constituting the digits etc.\nMere drag-and-drop augmentation of the extra-Dig-MNIST digit images will/might in fact, hurt the accuracy of the models you are training. I found this phenomenon fascinating and I can totally see a lot of interesting work coming out of this observation. \n\n**4: Binary neural networks**\nI chanced upon this awesome library 'LARQ': https://larq.dev/\nand thought I'd share it with y'all in case you wanted to train and deploy parsimonious models (oft in the order of a few kB!).\n\nLastly, thank you so very much for your participation and energy!\nKind regards!\n- Vinay", "667159": "Thanks for creating and providing the dataset! It is great to get started with image classification or to explore what is going on inside the own architecture. \nPutting it here in a kaggle competition makes me certainly spend more time with the data and studying the basics than the original MNIST would have done.", "668896": "Volunteers with 'newbie effect' = human augmenter 😜", "679276": "Hi Vinay,\n\nThanks for this competition. It is really fun and I have learned a lot from it. Will you write any paper based on the results? How does the accuracy of the models compare to what you estimated? It seems like the top contenders have a really hard time pushing past 0.99-0.991. \n\nI would wonder how a human expert would classify the images that the algos get wrong. Are they also impossible for a human reader to classify correctly?" }, "source": "meta" }