Notebook available on GitHub: https://github.com/kevindewalt/ai-notebooks/blob/master/Corporaci%C3%B3n+Favorita+Grocery+Sales+Forecasting.ipynb
\n\nA few months ago I started trying to apply the techniques Jeremy Howard shares here for making predictions using simple feed-forward neural networks and embedding layers to this contest.
\n\nIt is based on the this paper by Guo and Berkhahn who finished 3rd place in the Rossman Store Sales contest.
\n\nWell ... it didn't work very well. Here is a quick summary of what I tried along with a simple approach which got me the best results.
\n\nFeel free to ask questions or share any advice in the comments.\n\nI'm presuming a lot of knowledge about neural networks. If you take the time to carefully read through my results I'll be happy to share what I know.
\n\nI started by systematically building features with all data provided in the contest following the techniques Jeremy Howard outlines in the link above.
\n\nI generated over 27 category features (e.g. year, month, dayofweek, family, holidays) and 15 continuous features (e.g. oil price, 30-day oil change, days before/after promotion).
\n\nSome of these were quite complex such as days before/after a promotion given that each depends on a store and item.
\n\nAnd I did this for all of the training data, resulting in dataframes > 20 GB.
\n\nThen I built a sophisticated embedding layer and a few dense layers for my model. It took several hours to train on a GTX1080Ti ... and my results were pretty lousy.
\n\nWhile it's always possible I made a mistake, I couldn't get anything below 1.0 on the public leaderboard. So I decided to scrap it and try another approach.
\n\nI decided to start afresh trying replicate the results using a handful of features and less data.
\n\nI started with the 'moving average' technique presnted by Paulo Pinto and others here and tuned the neural network to get a LB score of .547. The 4 features are:
\n\nI used some unorthodox technniques (no dropout, batch sizes of 500,000) which I shared below.
\n\nI got better results and after trying to add lots of additional features never improved on these results.
\n\nThis contest was a good reminder to start with simple models, small amounts of training data and gradually add features.
\n\nMy goal was to learn the techniques presented in the Guo paper for using embedding layers on time-series data. I definitely achieved it and now have a good understanding of the approach.
\n\nmore features\n\nI'd be curious to see if adding oil, weather, holiday, geographical etc. information improves results but I'm not super-optimistic about getting a big boost.
\n\ncustom sample weight and loss fucntions\n\nOn the model side I'd like to figure out why using Keras sample_weight field isn't working and would like to build a custom loss functions with the NWRMSLE instead of MAE. Unfortunately building custom loss functions in Keras gets quite complex using the Keras backend. Keras is better for out-of-the-box functions.
\n\nA good reminder to start simple\n\nUnfortunately most of the data in this contest doesn't appear very predictive. In this case a simpler algorithm like xgboost is probably a better choice.
\n\nObviously ... we would never tackle a client project using this approach. Start simple, learn the data, and get a good baseline is always the right approach.
\n\nHope it helps!
", "messages": [ { "id": "266839", "postDate": "01/09/2018 23:50:14", "content": "Notebook available on GitHub: https://github.com/kevindewalt/ai-notebooks/blob/master/Corporaci%C3%B3n+Favorita+Grocery+Sales+Forecasting.ipynb
\n\nA few months ago I started trying to apply the techniques Jeremy Howard shares here for making predictions using simple feed-forward neural networks and embedding layers to this contest.
\n\nIt is based on the this paper by Guo and Berkhahn who finished 3rd place in the Rossman Store Sales contest.
\n\nWell ... it didn't work very well. Here is a quick summary of what I tried along with a simple approach which got me the best results.
\n\nFeel free to ask questions or share any advice in the comments.\n\nI'm presuming a lot of knowledge about neural networks. If you take the time to carefully read through my results I'll be happy to share what I know.
\n\nI started by systematically building features with all data provided in the contest following the techniques Jeremy Howard outlines in the link above.
\n\nI generated over 27 category features (e.g. year, month, dayofweek, family, holidays) and 15 continuous features (e.g. oil price, 30-day oil change, days before/after promotion).
\n\nSome of these were quite complex such as days before/after a promotion given that each depends on a store and item.
\n\nAnd I did this for all of the training data, resulting in dataframes > 20 GB.
\n\nThen I built a sophisticated embedding layer and a few dense layers for my model. It took several hours to train on a GTX1080Ti ... and my results were pretty lousy.
\n\nWhile it's always possible I made a mistake, I couldn't get anything below 1.0 on the public leaderboard. So I decided to scrap it and try another approach.
\n\nI decided to start afresh trying replicate the results using a handful of features and less data.
\n\nI started with the 'moving average' technique presnted by Paulo Pinto and others here and tuned the neural network to get a LB score of .547. The 4 features are:
\n\nI used some unorthodox technniques (no dropout, batch sizes of 500,000) which I shared below.
\n\nI got better results and after trying to add lots of additional features never improved on these results.
\n\nThis contest was a good reminder to start with simple models, small amounts of training data and gradually add features.
\n\nMy goal was to learn the techniques presented in the Guo paper for using embedding layers on time-series data. I definitely achieved it and now have a good understanding of the approach.
\n\nmore features\n\nI'd be curious to see if adding oil, weather, holiday, geographical etc. information improves results but I'm not super-optimistic about getting a big boost.
\n\ncustom sample weight and loss fucntions\n\nOn the model side I'd like to figure out why using Keras sample_weight field isn't working and would like to build a custom loss functions with the NWRMSLE instead of MAE. Unfortunately building custom loss functions in Keras gets quite complex using the Keras backend. Keras is better for out-of-the-box functions.
\n\nA good reminder to start simple\n\nUnfortunately most of the data in this contest doesn't appear very predictive. In this case a simpler algorithm like xgboost is probably a better choice.
\n\nObviously ... we would never tackle a client project using this approach. Start simple, learn the data, and get a good baseline is always the right approach.
\n\nHope it helps!
", "rawMarkdown": "## Get the notebook\nNotebook available on GitHub: https://github.com/kevindewalt/ai-notebooks/blob/master/Corporaci%C3%B3n+Favorita+Grocery+Sales+Forecasting.ipynb\n\n# Simple 4-feature Neural Network for LB .547\nA few months ago I started trying to apply the techniques Jeremy Howard shares [here](https://github.com/fastai/courses/blob/master/deeplearning2/rossman.ipynb) for making predictions using simple feed-forward neural networks and embedding layers to this contest.\n\nIt is based on the [this paper](https://arxiv.org/abs/1604.06737) by Guo and Berkhahn who finished 3rd place in the [Rossman Store Sales](https://www.kaggle.com/c/rossmann-store-sales) contest.\n\nWell ... it didn't work very well. Here is a quick summary of what I tried along with a simple approach which got me the best results.\n\n#### Feel free to ask questions or share any advice in the comments.\nI'm presuming a lot of knowledge about neural networks. If you take the time to carefully read through my results I'll be happy to share what I know.\n\n## What didn't work - a massive, feature-rich dataset\nI started by systematically building features with all data provided in the contest following the techniques Jeremy Howard outlines in the link above. \n\nI generated over 27 category features (e.g. year, month, dayofweek, family, holidays) and 15 continuous features (e.g. oil price, 30-day oil change, days before/after promotion). \n\nSome of these were quite complex such as days before/after a promotion given that each depends on a store and item.\n\nAnd I did this for **all of the training data**, resulting in dataframes > 20 GB.\n\nThen I built a sophisticated embedding layer and a few dense layers for my model. It took several hours to train on a GTX1080Ti ... and my results were pretty lousy.\n\nWhile it's always possible I made a mistake, I couldn't get anything below 1.0 on the public leaderboard. So I decided to scrap it and try another approach.\n\n## What worked better - 4 features and a few months of training data\nI decided to start afresh trying replicate the results using a handful of features and less data.\n\nI started with the 'moving average' technique presnted by Paulo Pinto and others [here](https://www.kaggle.com/paulorzp/log-ma-and-days-of-week-means-lb-0-529/code) and tuned the neural network to get a LB score of .547. The 4 features are:\n\n+ store number\n+ item number\n+ whether or the item was on promotion\n+ an average of recent sales at the same store/item combinations.\n\nI used some unorthodox technniques (no dropout, batch sizes of 500,000) which I shared below.\n\nI got better results and after trying to add lots of additional features never improved on these results.\n\n## What I would like to try (but probably won't)\nThis contest was a good reminder to start with simple models, small amounts of training data and gradually add features. \n\nMy goal was to learn the techniques presented in the Guo paper for using embedding layers on time-series data. I definitely achieved it and now have a good understanding of the approach.\n\n#### more features\nI'd be curious to see if adding oil, weather, holiday, geographical etc. information improves results but I'm not super-optimistic about getting a big boost.\n\n#### custom sample weight and loss fucntions\nOn the model side I'd like to figure out why using Keras sample_weight field isn't working and would like to build a custom loss functions with the NWRMSLE instead of MAE. Unfortunately building custom loss functions in Keras gets quite complex using the Keras backend. Keras is better for out-of-the-box functions.\n\n#### A good reminder to start simple\nUnfortunately most of the data in this contest doesn't appear very predictive. In this case a simpler algorithm like xgboost is probably a better choice.\n\nObviously ... we would never tackle a client project using this approach. Start simple, learn the data, and get a good baseline is always the right approach.\n\nHope it helps!", "votes": null }, { "id": "266860", "postDate": "01/10/2018 01:24:15", "content": "Really helpful! I like a reminder of \"start with simple models, small amounts of training data and gradually add features\". Thanks for sharing.
", "rawMarkdown": "Really helpful! I like a reminder of \"start with simple models, small amounts of training data and gradually add features\". Thanks for sharing.", "votes": null }, { "id": "266883", "postDate": "01/10/2018 03:58:22", "content": "Thanks for your sharing.
\n\n\n\n\nan average of recent sales at the same store/item combinations.
\n
What do you mean by the above average?
\n\nBesides, in many other kernels, it seems day of the week is a useful feature.
", "rawMarkdown": "Thanks for your sharing. \n\n> an average of recent sales at the same store/item combinations.\n\nWhat do you mean by the above average?\n\nBesides, in many other kernels, it seems day of the week is a useful feature.", "votes": null }, { "id": "266940", "postDate": "01/10/2018 07:07:08", "content": "Thanks Kevin. I tried working using Jeremy's notebook as a starting point as well, but couldn't get it to work.\nI am very happy to see that you made so much progress and curious to take a look at your code.\nThanks for sharing it!
", "rawMarkdown": "Thanks Kevin. I tried working using Jeremy's notebook as a starting point as well, but couldn't get it to work.\nI am very happy to see that you made so much progress and curious to take a look at your code.\nThanks for sharing it!", "votes": null }, { "id": "266944", "postDate": "01/10/2018 07:11:59", "content": "One more comment. If you haven't already, you could post this in the fast.ai forum to bring it to Jeremy's attention. \nI am sure he would be curious to see what you have tried!
", "rawMarkdown": "One more comment. If you haven't already, you could post this in the fast.ai forum to bring it to Jeremy's attention. \nI am sure he would be curious to see what you have tried!", "votes": null }, { "id": "267051", "postDate": "01/10/2018 13:41:56", "content": "You're asking a good question. One challenge in Data Science is our inconsistent use of terminology. When in doubt ... I look at the code.
\n\nUnfortunately that doesn't always help either. Most of my experience is in building products, so I'm used to working with professional programmers who organize their code for readability and reuse. That isn't the case in Data Science - most of these Kernels are organized like experimental lab books where the only relevant issue is the result. So. ..
\n\nIn this case I couldn't understand many of the Kernels until started running the data myself ... looking at the dataframes and seeing what is derived. And then I had to simplify it by running the same code on 1 store or 1 item.
\n\nHere's an example. Suppose you have 2 grocery stores and 2 products:
\n\nStores: A and B.\nProducts: Beer and Diapers.
\n\nStore A sells an average of 5 cases of Beer and 4 boxes of Diapers per week.\nStore B sells an average of 10 cases of Beer and 2 boxes of Diapers per week.
\n\nThat's what I mean by an average of store/item combinations.. Average is grouped across a date.
\n\nOf course you could run the same combinations for a daily interval, monthly interval, or just Tuesdays.
", "rawMarkdown": "You're asking a good question. One challenge in Data Science is our inconsistent use of terminology. When in doubt ... I look at the code.\n\nUnfortunately that doesn't always help either. Most of my experience is in building products, so I'm used to working with professional programmers who organize their code for readability and reuse. That isn't the case in Data Science - most of these Kernels are organized like experimental lab books where the only relevant issue is the result. So. ..\n\nIn this case I couldn't understand many of the Kernels until started running the data myself ... looking at the dataframes and seeing what is derived. And then I had to simplify it by running the same code on 1 store or 1 item.\n\nHere's an example. Suppose you have 2 grocery stores and 2 products:\n\nStores: A and B.\nProducts: Beer and Diapers.\n\nStore A sells an average of 5 cases of Beer and 4 boxes of Diapers per week.\nStore B sells an average of 10 cases of Beer and 2 boxes of Diapers per week. \n\n**That's what I mean by an average of store/item combinations.**. Average is grouped across a date.\n\nOf course you could run the same combinations for a daily interval, monthly interval, or just Tuesdays.", "votes": null }, { "id": "267052", "postDate": "01/10/2018 13:43:26", "content": "Awesome. I can also share my other notebooks if it helps. In particular I have one where I just attempted to derive all of the features using Jeremy's approach. Let me know if you're interested and I'll share on GitHub.
", "rawMarkdown": "Awesome. I can also share my other notebooks if it helps. In particular I have one where I just attempted to derive all of the features using Jeremy's approach. Let me know if you're interested and I'll share on GitHub.", "votes": null }, { "id": "267053", "postDate": "01/10/2018 13:43:37", "content": "Good idea, done. See http://forums.fast.ai/t/corporacion-favorita-grocery-sales-forecasting/8359/5
", "rawMarkdown": "Good idea, done. See http://forums.fast.ai/t/corporacion-favorita-grocery-sales-forecasting/8359/5", "votes": null }, { "id": "267143", "postDate": "01/10/2018 16:55:25", "content": "Here you go, a few others have asked me for it:
\n\nhttps://github.com/kevindewalt/ai-notebooks/blob/master/DataPrep.ipynb
", "rawMarkdown": "Here you go, a few others have asked me for it:\n\nhttps://github.com/kevindewalt/ai-notebooks/blob/master/DataPrep.ipynb", "votes": null }, { "id": "267520", "postDate": "01/11/2018 15:58:53", "content": "Many thanks for your clarification, Kevin. I feel the same: codes of many kernels are hard to read unless you spend a lot of time on it. Thus, it is recommended to publish a kernel using the notebooks with enough explanations (Markdown), instead of just dumping the codes.
", "rawMarkdown": "Many thanks for your clarification, Kevin. I feel the same: codes of many kernels are hard to read unless you spend a lot of time on it. Thus, it is recommended to publish a kernel using the notebooks with enough explanations (Markdown), instead of just dumping the codes.", "votes": null }, { "id": "267560", "postDate": "01/11/2018 18:19:41", "content": "have you benchmarked your 4 feature model with a linear model ? that may take less amount of time. Where you doing this analysis on a PC or on a cloud and if on a cloud which provider ?! I got disappointed by gcloud plus my ignorance !
", "rawMarkdown": "have you benchmarked your 4 feature model with a linear model ? that may take less amount of time. Where you doing this analysis on a PC or on a cloud and if on a cloud which provider ?! I got disappointed by gcloud plus my ignorance !", "votes": null }, { "id": "267562", "postDate": "01/11/2018 18:33:19", "content": "Sorry, I didn't do it on a linear model. A simpler model would definitely be a better choice for a practical implementation with such few features.
\n\nI ran it on our company's Deep Learning rig (affectionately called Drogan): https://blog.prolego.io/build-your-1st-deep-learning-rig-6f749f63798d
", "rawMarkdown": "Sorry, I didn't do it on a linear model. A simpler model would definitely be a better choice for a practical implementation with such few features. \n\nI ran it on our company's Deep Learning rig (affectionately called Drogan): https://blog.prolego.io/build-your-1st-deep-learning-rig-6f749f63798d", "votes": null }, { "id": "400384", "postDate": "10/08/2018 08:05:44", "content": "Thanks for detailed and reproducible notebook.\nWhat is the logical reason behind calculating the day of week historical averages before expanding training dataframe(zero filling)? \nWhy can't we first expand dataframe and then start all calculations?
\n\nEdit:\nI think, reason is probably, zero filled dataset would skew those averages and consequently model will under-forecast.
", "rawMarkdown": "Thanks for detailed and reproducible notebook.\nWhat is the logical reason behind calculating the day of week historical averages before expanding training dataframe(zero filling)? \nWhy can't we first expand dataframe and then start all calculations?\n\nEdit:\nI think, reason is probably, zero filled dataset would skew those averages and consequently model will under-forecast.", "votes": null } ], "comments": [ { "id": 266860, "author_name": "nguyentp", "author_url": "", "post_date": "01/10/2018 01:24:15", "content": "Really helpful! I like a reminder of \"start with simple models, small amounts of training data and gradually add features\". Thanks for sharing.
", "votes": null, "replies": [] }, { "id": 266883, "author_name": "shuhuagao", "author_url": "", "post_date": "01/10/2018 03:58:22", "content": "Thanks for your sharing.
\n\n\n\n\nan average of recent sales at the same store/item combinations.
\n
What do you mean by the above average?
\n\nBesides, in many other kernels, it seems day of the week is a useful feature.
", "votes": null, "replies": [ { "id": 267051, "author_name": "kevindewalt", "author_url": "", "post_date": "01/10/2018 13:41:56", "content": "You're asking a good question. One challenge in Data Science is our inconsistent use of terminology. When in doubt ... I look at the code.
\n\nUnfortunately that doesn't always help either. Most of my experience is in building products, so I'm used to working with professional programmers who organize their code for readability and reuse. That isn't the case in Data Science - most of these Kernels are organized like experimental lab books where the only relevant issue is the result. So. ..
\n\nIn this case I couldn't understand many of the Kernels until started running the data myself ... looking at the dataframes and seeing what is derived. And then I had to simplify it by running the same code on 1 store or 1 item.
\n\nHere's an example. Suppose you have 2 grocery stores and 2 products:
\n\nStores: A and B.\nProducts: Beer and Diapers.
\n\nStore A sells an average of 5 cases of Beer and 4 boxes of Diapers per week.\nStore B sells an average of 10 cases of Beer and 2 boxes of Diapers per week.
\n\nThat's what I mean by an average of store/item combinations.. Average is grouped across a date.
\n\nOf course you could run the same combinations for a daily interval, monthly interval, or just Tuesdays.
", "votes": null, "replies": [] }, { "id": 267520, "author_name": "shuhuagao", "author_url": "", "post_date": "01/11/2018 15:58:53", "content": "Many thanks for your clarification, Kevin. I feel the same: codes of many kernels are hard to read unless you spend a lot of time on it. Thus, it is recommended to publish a kernel using the notebooks with enough explanations (Markdown), instead of just dumping the codes.
", "votes": null, "replies": [] } ] }, { "id": 266940, "author_name": "fabsta", "author_url": "", "post_date": "01/10/2018 07:07:08", "content": "Thanks Kevin. I tried working using Jeremy's notebook as a starting point as well, but couldn't get it to work.\nI am very happy to see that you made so much progress and curious to take a look at your code.\nThanks for sharing it!
", "votes": null, "replies": [ { "id": 267052, "author_name": "kevindewalt", "author_url": "", "post_date": "01/10/2018 13:43:26", "content": "Awesome. I can also share my other notebooks if it helps. In particular I have one where I just attempted to derive all of the features using Jeremy's approach. Let me know if you're interested and I'll share on GitHub.
", "votes": null, "replies": [] }, { "id": 267143, "author_name": "kevindewalt", "author_url": "", "post_date": "01/10/2018 16:55:25", "content": "Here you go, a few others have asked me for it:
\n\nhttps://github.com/kevindewalt/ai-notebooks/blob/master/DataPrep.ipynb
", "votes": null, "replies": [] } ] }, { "id": 266944, "author_name": "fabsta", "author_url": "", "post_date": "01/10/2018 07:11:59", "content": "One more comment. If you haven't already, you could post this in the fast.ai forum to bring it to Jeremy's attention. \nI am sure he would be curious to see what you have tried!
", "votes": null, "replies": [ { "id": 267053, "author_name": "kevindewalt", "author_url": "", "post_date": "01/10/2018 13:43:37", "content": "Good idea, done. See http://forums.fast.ai/t/corporacion-favorita-grocery-sales-forecasting/8359/5
", "votes": null, "replies": [] } ] }, { "id": 267560, "author_name": "huhulimaguli", "author_url": "", "post_date": "01/11/2018 18:19:41", "content": "have you benchmarked your 4 feature model with a linear model ? that may take less amount of time. Where you doing this analysis on a PC or on a cloud and if on a cloud which provider ?! I got disappointed by gcloud plus my ignorance !
", "votes": null, "replies": [ { "id": 267562, "author_name": "kevindewalt", "author_url": "", "post_date": "01/11/2018 18:33:19", "content": "Sorry, I didn't do it on a linear model. A simpler model would definitely be a better choice for a practical implementation with such few features.
\n\nI ran it on our company's Deep Learning rig (affectionately called Drogan): https://blog.prolego.io/build-your-1st-deep-learning-rig-6f749f63798d
", "votes": null, "replies": [] } ] }, { "id": 400384, "author_name": "parthbkgadoya", "author_url": "", "post_date": "10/08/2018 08:05:44", "content": "Thanks for detailed and reproducible notebook.\nWhat is the logical reason behind calculating the day of week historical averages before expanding training dataframe(zero filling)? \nWhy can't we first expand dataframe and then start all calculations?
\n\nEdit:\nI think, reason is probably, zero filled dataset would skew those averages and consequently model will under-forecast.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "266839": "## Get the notebook\nNotebook available on GitHub: https://github.com/kevindewalt/ai-notebooks/blob/master/Corporaci%C3%B3n+Favorita+Grocery+Sales+Forecasting.ipynb\n\n# Simple 4-feature Neural Network for LB .547\nA few months ago I started trying to apply the techniques Jeremy Howard shares [here](https://github.com/fastai/courses/blob/master/deeplearning2/rossman.ipynb) for making predictions using simple feed-forward neural networks and embedding layers to this contest.\n\nIt is based on the [this paper](https://arxiv.org/abs/1604.06737) by Guo and Berkhahn who finished 3rd place in the [Rossman Store Sales](https://www.kaggle.com/c/rossmann-store-sales) contest.\n\nWell ... it didn't work very well. Here is a quick summary of what I tried along with a simple approach which got me the best results.\n\n#### Feel free to ask questions or share any advice in the comments.\nI'm presuming a lot of knowledge about neural networks. If you take the time to carefully read through my results I'll be happy to share what I know.\n\n## What didn't work - a massive, feature-rich dataset\nI started by systematically building features with all data provided in the contest following the techniques Jeremy Howard outlines in the link above. \n\nI generated over 27 category features (e.g. year, month, dayofweek, family, holidays) and 15 continuous features (e.g. oil price, 30-day oil change, days before/after promotion). \n\nSome of these were quite complex such as days before/after a promotion given that each depends on a store and item.\n\nAnd I did this for **all of the training data**, resulting in dataframes > 20 GB.\n\nThen I built a sophisticated embedding layer and a few dense layers for my model. It took several hours to train on a GTX1080Ti ... and my results were pretty lousy.\n\nWhile it's always possible I made a mistake, I couldn't get anything below 1.0 on the public leaderboard. So I decided to scrap it and try another approach.\n\n## What worked better - 4 features and a few months of training data\nI decided to start afresh trying replicate the results using a handful of features and less data.\n\nI started with the 'moving average' technique presnted by Paulo Pinto and others [here](https://www.kaggle.com/paulorzp/log-ma-and-days-of-week-means-lb-0-529/code) and tuned the neural network to get a LB score of .547. The 4 features are:\n\n+ store number\n+ item number\n+ whether or the item was on promotion\n+ an average of recent sales at the same store/item combinations.\n\nI used some unorthodox technniques (no dropout, batch sizes of 500,000) which I shared below.\n\nI got better results and after trying to add lots of additional features never improved on these results.\n\n## What I would like to try (but probably won't)\nThis contest was a good reminder to start with simple models, small amounts of training data and gradually add features. \n\nMy goal was to learn the techniques presented in the Guo paper for using embedding layers on time-series data. I definitely achieved it and now have a good understanding of the approach.\n\n#### more features\nI'd be curious to see if adding oil, weather, holiday, geographical etc. information improves results but I'm not super-optimistic about getting a big boost.\n\n#### custom sample weight and loss fucntions\nOn the model side I'd like to figure out why using Keras sample_weight field isn't working and would like to build a custom loss functions with the NWRMSLE instead of MAE. Unfortunately building custom loss functions in Keras gets quite complex using the Keras backend. Keras is better for out-of-the-box functions.\n\n#### A good reminder to start simple\nUnfortunately most of the data in this contest doesn't appear very predictive. In this case a simpler algorithm like xgboost is probably a better choice.\n\nObviously ... we would never tackle a client project using this approach. Start simple, learn the data, and get a good baseline is always the right approach.\n\nHope it helps!", "266860": "Really helpful! I like a reminder of \"start with simple models, small amounts of training data and gradually add features\". Thanks for sharing.", "266883": "Thanks for your sharing. \n\n> an average of recent sales at the same store/item combinations.\n\nWhat do you mean by the above average?\n\nBesides, in many other kernels, it seems day of the week is a useful feature.", "266940": "Thanks Kevin. I tried working using Jeremy's notebook as a starting point as well, but couldn't get it to work.\nI am very happy to see that you made so much progress and curious to take a look at your code.\nThanks for sharing it!", "266944": "One more comment. If you haven't already, you could post this in the fast.ai forum to bring it to Jeremy's attention. \nI am sure he would be curious to see what you have tried!", "267051": "You're asking a good question. One challenge in Data Science is our inconsistent use of terminology. When in doubt ... I look at the code.\n\nUnfortunately that doesn't always help either. Most of my experience is in building products, so I'm used to working with professional programmers who organize their code for readability and reuse. That isn't the case in Data Science - most of these Kernels are organized like experimental lab books where the only relevant issue is the result. So. ..\n\nIn this case I couldn't understand many of the Kernels until started running the data myself ... looking at the dataframes and seeing what is derived. And then I had to simplify it by running the same code on 1 store or 1 item.\n\nHere's an example. Suppose you have 2 grocery stores and 2 products:\n\nStores: A and B.\nProducts: Beer and Diapers.\n\nStore A sells an average of 5 cases of Beer and 4 boxes of Diapers per week.\nStore B sells an average of 10 cases of Beer and 2 boxes of Diapers per week. \n\n**That's what I mean by an average of store/item combinations.**. Average is grouped across a date.\n\nOf course you could run the same combinations for a daily interval, monthly interval, or just Tuesdays.", "267052": "Awesome. I can also share my other notebooks if it helps. In particular I have one where I just attempted to derive all of the features using Jeremy's approach. Let me know if you're interested and I'll share on GitHub.", "267053": "Good idea, done. See http://forums.fast.ai/t/corporacion-favorita-grocery-sales-forecasting/8359/5", "267143": "Here you go, a few others have asked me for it:\n\nhttps://github.com/kevindewalt/ai-notebooks/blob/master/DataPrep.ipynb", "267520": "Many thanks for your clarification, Kevin. I feel the same: codes of many kernels are hard to read unless you spend a lot of time on it. Thus, it is recommended to publish a kernel using the notebooks with enough explanations (Markdown), instead of just dumping the codes.", "267560": "have you benchmarked your 4 feature model with a linear model ? that may take less amount of time. Where you doing this analysis on a PC or on a cloud and if on a cloud which provider ?! I got disappointed by gcloud plus my ignorance !", "267562": "Sorry, I didn't do it on a linear model. A simpler model would definitely be a better choice for a practical implementation with such few features. \n\nI ran it on our company's Deep Learning rig (affectionately called Drogan): https://blog.prolego.io/build-your-1st-deep-learning-rig-6f749f63798d", "400384": "Thanks for detailed and reproducible notebook.\nWhat is the logical reason behind calculating the day of week historical averages before expanding training dataframe(zero filling)? \nWhy can't we first expand dataframe and then start all calculations?\n\nEdit:\nI think, reason is probably, zero filled dataset would skew those averages and consequently model will under-forecast." }, "source": "meta" }