Hi guys,\nthis is my first try of working with Kaggle MNIST data. \nTried FastAI and loved it.\nYou can find my kernel here: https://www.kaggle.com/zinovadr/mnist-fastai-pytorch-dynamic-tuned
\n\nMy findings: When you're over fitting you model in order to get high score, always keep in mind that there could be some \"funky\" initial training data intentionally putted there to messed up your final sore. \nI am still new to ML/DL - if anyone fine any findings on how to improve it, please share.
\n\nNext step is to try CatBoost.\nregards,\nAdrian
", "messages": [ { "id": "642125", "postDate": "10/05/2019 15:24:58", "content": "Hi guys,\nthis is my first try of working with Kaggle MNIST data. \nTried FastAI and loved it.\nYou can find my kernel here: https://www.kaggle.com/zinovadr/mnist-fastai-pytorch-dynamic-tuned
\n\nMy findings: When you're over fitting you model in order to get high score, always keep in mind that there could be some \"funky\" initial training data intentionally putted there to messed up your final sore. \nI am still new to ML/DL - if anyone fine any findings on how to improve it, please share.
\n\nNext step is to try CatBoost.\nregards,\nAdrian
", "rawMarkdown": "Hi guys,\nthis is my first try of working with Kaggle MNIST data. \nTried FastAI and loved it.\nYou can find my kernel here: [https://www.kaggle.com/zinovadr/mnist-fastai-pytorch-dynamic-tuned](https://www.kaggle.com/zinovadr/mnist-fastai-pytorch-dynamic-tuned)\n\nMy findings: When you're over fitting you model in order to get high score, always keep in mind that there could be some \"funky\" initial training data intentionally putted there to messed up your final sore. \nI am still new to ML/DL - if anyone fine any findings on how to improve it, please share.\n \n\nNext step is to try CatBoost.\nregards,\nAdrian", "votes": null }, { "id": "642266", "postDate": "10/05/2019 19:25:43", "content": "Check out this post by Chris Deotte. It explains how to get the best possible score :)
", "rawMarkdown": "Check out [this post](https://www.kaggle.com/c/digit-recognizer/discussion/61480) by Chris Deotte. It explains how to get the best possible score :)", "votes": null }, { "id": "644246", "postDate": "10/08/2019 14:27:01", "content": "Thanks for the feedback!
", "rawMarkdown": "Thanks for the feedback!", "votes": null }, { "id": "654037", "postDate": "10/21/2019 10:51:12", "content": "Hi, you validation set is to small it is impossible for you to tell how accurate your models is and if you are overfitting (most likely). Add more regularisation and more transforms and you will move higher up.
", "rawMarkdown": "Hi, you validation set is to small it is impossible for you to tell how accurate your models is and if you are overfitting (most likely). Add more regularisation and more transforms and you will move higher up.", "votes": null }, { "id": "654192", "postDate": "10/21/2019 15:23:09", "content": "Total agree with you @rincewind007 \nI think the power is in if you can manage to generate more train data and see how it'll fit in order for the model to be ready to face the last data. \nAnyway taking into consideration the symbols, I'd say that the 99% it is already very close and if this model will be used to recognize entire phrases - it can auto-correct from the context. \nthanks
", "rawMarkdown": "Total agree with you @rincewind007 \nI think the power is in if you can manage to generate more train data and see how it'll fit in order for the model to be ready to face the last data. \nAnyway taking into consideration the symbols, I'd say that the 99% it is already very close and if this model will be used to recognize entire phrases - it can auto-correct from the context. \nthanks", "votes": null }, { "id": "658583", "postDate": "10/26/2019 07:53:41", "content": "Hi @zinovadr , Im also new to ML. Im trying to grasp some things from fast ai. I went through your kernel. Could you explain me how did you calculate the input shape for Linear laye as 3136? `
\n conv2(64,64,stride=2,ks=5),\n nn.Dropout(0.4),
Flatten(),\nnn.Linear(**3136**, 256),`\nIt should be 64*64 = 4096 right? Im confused. Im trying out on a lot of models.
", "rawMarkdown": "Hi @zinovadr , Im also new to ML. Im trying to grasp some things from fast ai. I went through your kernel. Could you explain me how did you calculate the input shape for Linear laye as 3136? ` \n conv2(64,64,stride=2,ks=5),\n nn.Dropout(0.4),\n \n Flatten(),\n nn.Linear(**3136**, 256),`\nIt should be 64*64 = 4096 right? Im confused. Im trying out on a lot of models.", "votes": null }, { "id": "658742", "postDate": "10/26/2019 13:24:32", "content": "great question. \n64 is how many layer filers and it is not dividing the entire size to exact pieces.\nBut honestly you might be right. I tried many changes in order to get more accurate result.\nIn fact it should be a combination of: height x width x channel.\n thanks
", "rawMarkdown": "great question. \n64 is how many layer filers and it is not dividing the entire size to exact pieces.\nBut honestly you might be right. I tried many changes in order to get more accurate result.\nIn fact it should be a combination of: height x width x channel.\n thanks", "votes": null }, { "id": "659110", "postDate": "10/27/2019 05:08:09", "content": "Thanks for the reply. I went on further researching how that number came, and i came to know that it is actually 64*7*7. This number comes from (number of input channels * width * height) after the last conv2d channel of yours. This number 7 comes after two stride2 kernels. 28/2 = 14 / 2 = 7.
", "rawMarkdown": "Thanks for the reply. I went on further researching how that number came, and i came to know that it is actually 64*7*7. This number comes from **(number of input channels * width * height)** after the last conv2d channel of yours. This number 7 comes after two stride2 kernels. 28/2 = 14 / 2 = 7.", "votes": null } ], "comments": [ { "id": 642266, "author_name": "christianwallenwein", "author_url": "", "post_date": "10/05/2019 19:25:43", "content": "Check out this post by Chris Deotte. It explains how to get the best possible score :)
", "votes": null, "replies": [] }, { "id": 644246, "author_name": "zinovadr", "author_url": "", "post_date": "10/08/2019 14:27:01", "content": "Thanks for the feedback!
", "votes": null, "replies": [] }, { "id": 654037, "author_name": "rincewind007", "author_url": "", "post_date": "10/21/2019 10:51:12", "content": "Hi, you validation set is to small it is impossible for you to tell how accurate your models is and if you are overfitting (most likely). Add more regularisation and more transforms and you will move higher up.
", "votes": null, "replies": [ { "id": 654192, "author_name": "zinovadr", "author_url": "", "post_date": "10/21/2019 15:23:09", "content": "Total agree with you @rincewind007 \nI think the power is in if you can manage to generate more train data and see how it'll fit in order for the model to be ready to face the last data. \nAnyway taking into consideration the symbols, I'd say that the 99% it is already very close and if this model will be used to recognize entire phrases - it can auto-correct from the context. \nthanks
", "votes": null, "replies": [] } ] }, { "id": 658583, "author_name": "shreeshakedilaya", "author_url": "", "post_date": "10/26/2019 07:53:41", "content": "Hi @zinovadr , Im also new to ML. Im trying to grasp some things from fast ai. I went through your kernel. Could you explain me how did you calculate the input shape for Linear laye as 3136? `
\n conv2(64,64,stride=2,ks=5),\n nn.Dropout(0.4),
Flatten(),\nnn.Linear(**3136**, 256),`\nIt should be 64*64 = 4096 right? Im confused. Im trying out on a lot of models.
", "votes": null, "replies": [ { "id": 658742, "author_name": "zinovadr", "author_url": "", "post_date": "10/26/2019 13:24:32", "content": "great question. \n64 is how many layer filers and it is not dividing the entire size to exact pieces.\nBut honestly you might be right. I tried many changes in order to get more accurate result.\nIn fact it should be a combination of: height x width x channel.\n thanks
", "votes": null, "replies": [] }, { "id": 659110, "author_name": "shreeshakedilaya", "author_url": "", "post_date": "10/27/2019 05:08:09", "content": "Thanks for the reply. I went on further researching how that number came, and i came to know that it is actually 64*7*7. This number comes from (number of input channels * width * height) after the last conv2d channel of yours. This number 7 comes after two stride2 kernels. 28/2 = 14 / 2 = 7.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "642125": "Hi guys,\nthis is my first try of working with Kaggle MNIST data. \nTried FastAI and loved it.\nYou can find my kernel here: [https://www.kaggle.com/zinovadr/mnist-fastai-pytorch-dynamic-tuned](https://www.kaggle.com/zinovadr/mnist-fastai-pytorch-dynamic-tuned)\n\nMy findings: When you're over fitting you model in order to get high score, always keep in mind that there could be some \"funky\" initial training data intentionally putted there to messed up your final sore. \nI am still new to ML/DL - if anyone fine any findings on how to improve it, please share.\n \n\nNext step is to try CatBoost.\nregards,\nAdrian", "642266": "Check out [this post](https://www.kaggle.com/c/digit-recognizer/discussion/61480) by Chris Deotte. It explains how to get the best possible score :)", "644246": "Thanks for the feedback!", "654037": "Hi, you validation set is to small it is impossible for you to tell how accurate your models is and if you are overfitting (most likely). Add more regularisation and more transforms and you will move higher up.", "654192": "Total agree with you @rincewind007 \nI think the power is in if you can manage to generate more train data and see how it'll fit in order for the model to be ready to face the last data. \nAnyway taking into consideration the symbols, I'd say that the 99% it is already very close and if this model will be used to recognize entire phrases - it can auto-correct from the context. \nthanks", "658583": "Hi @zinovadr , Im also new to ML. Im trying to grasp some things from fast ai. I went through your kernel. Could you explain me how did you calculate the input shape for Linear laye as 3136? ` \n conv2(64,64,stride=2,ks=5),\n nn.Dropout(0.4),\n \n Flatten(),\n nn.Linear(**3136**, 256),`\nIt should be 64*64 = 4096 right? Im confused. Im trying out on a lot of models.", "658742": "great question. \n64 is how many layer filers and it is not dividing the entire size to exact pieces.\nBut honestly you might be right. I tried many changes in order to get more accurate result.\nIn fact it should be a combination of: height x width x channel.\n thanks", "659110": "Thanks for the reply. I went on further researching how that number came, and i came to know that it is actually 64*7*7. This number comes from **(number of input channels * width * height)** after the last conv2d channel of yours. This number 7 comes after two stride2 kernels. 28/2 = 14 / 2 = 7." }, "source": "meta" }