Hi, I am new to Kaggle and it is the first time I am applying statistical techniques to a dataset of this size, hence the question may sound stupid to more experience kagglers (sorry in advance) . My question is if it is a good idea to calculate sample statistics (like day of the week means, whether series are trending, number of days with promotions, standard deviations, maximum values etc.) and then build a model based on these features. I got inspired by one of the kernels which used moving average forecasts which beaten many of more sophisticated techniques (and ran 100x faster) and was based just on sample statistics without much of a model behind it.
\n\nI was thinking if combining all moving averages features from Pinto`s kernel with some other sample statistics to improve the fit. I tried implementing it (using last 5 months of data), and found that the more complex models performed worse. I ran OLS, Cubist, gradient boosting and neural network (first attempts in everything but OLS and few sad attempts in NN), and the best was OLS with the score of 0.649 and the worst was NN with the score 0.719. Is it due to bias variance trade-off (more complex model do not really predict anything new and just add nosiness to predictions) or overfitting? Should I continue with OLS and make more explicit non-linear features (like onpromotion*(promotion_mean-normal_mean) or is_trending*time*standard_deviation), or is it just going to be a waste of time?
\n\nMany thanks to all the Kagglers who shared their kernels. I have learnt so much just by looking at the code and trying to adapt it for my own tasks. Particularly, data manipulation will never be the same again after I discovered R data.table, dplyr and foreach packages. \nIf you have any advice for a novice like me, I would be very happy to listen to it.
", "messages": [ { "id": 258434, "postDate": "2017-12-16T04:32:38.147Z", "content": "Hi, I am new to Kaggle and it is the first time I am applying statistical techniques to a dataset of this size, hence the question may sound stupid to more experience kagglers (sorry in advance) . My question is if it is a good idea to calculate sample statistics (like day of the week means, whether series are trending, number of days with promotions, standard deviations, maximum values etc.) and then build a model based on these features. I got inspired by one of the kernels which used moving average forecasts which beaten many of more sophisticated techniques (and ran 100x faster) and was based just on sample statistics without much of a model behind it.
\n\nI was thinking if combining all moving averages features from Pinto`s kernel with some other sample statistics to improve the fit. I tried implementing it (using last 5 months of data), and found that the more complex models performed worse. I ran OLS, Cubist, gradient boosting and neural network (first attempts in everything but OLS and few sad attempts in NN), and the best was OLS with the score of 0.649 and the worst was NN with the score 0.719. Is it due to bias variance trade-off (more complex model do not really predict anything new and just add nosiness to predictions) or overfitting? Should I continue with OLS and make more explicit non-linear features (like onpromotion*(promotion_mean-normal_mean) or is_trending*time*standard_deviation), or is it just going to be a waste of time?
\n\nMany thanks to all the Kagglers who shared their kernels. I have learnt so much just by looking at the code and trying to adapt it for my own tasks. Particularly, data manipulation will never be the same again after I discovered R data.table, dplyr and foreach packages. \nIf you have any advice for a novice like me, I would be very happy to listen to it.
", "rawMarkdown": "Hi, I am new to Kaggle and it is the first time I am applying statistical techniques to a dataset of this size, hence the question may sound stupid to more experience kagglers (sorry in advance) . My question is if it is a good idea to calculate sample statistics (like day of the week means, whether series are trending, number of days with promotions, standard deviations, maximum values etc.) and then build a model based on these features. I got inspired by one of the kernels which used moving average forecasts which beaten many of more sophisticated techniques (and ran 100x faster) and was based just on sample statistics without much of a model behind it.\n\nI was thinking if combining all moving averages features from Pinto`s kernel with some other sample statistics to improve the fit. I tried implementing it (using last 5 months of data), and found that the more complex models performed worse. I ran OLS, Cubist, gradient boosting and neural network (first attempts in everything but OLS and few sad attempts in NN), and the best was OLS with the score of 0.649 and the worst was NN with the score 0.719. Is it due to bias variance trade-off (more complex model do not really predict anything new and just add nosiness to predictions) or overfitting? Should I continue with OLS and make more explicit non-linear features (like onpromotion*(promotion_mean-normal_mean) or is_trending*time*standard_deviation), or is it just going to be a waste of time?\n\nMany thanks to all the Kagglers who shared their kernels. I have learnt so much just by looking at the code and trying to adapt it for my own tasks. Particularly, data manipulation will never be the same again after I discovered R data.table, dplyr and foreach packages. \nIf you have any advice for a novice like me, I would be very happy to listen to it." } ], "comments": [], "raw_markdown_by_id": { "258434": "Hi, I am new to Kaggle and it is the first time I am applying statistical techniques to a dataset of this size, hence the question may sound stupid to more experience kagglers (sorry in advance) . My question is if it is a good idea to calculate sample statistics (like day of the week means, whether series are trending, number of days with promotions, standard deviations, maximum values etc.) and then build a model based on these features. I got inspired by one of the kernels which used moving average forecasts which beaten many of more sophisticated techniques (and ran 100x faster) and was based just on sample statistics without much of a model behind it.\n\nI was thinking if combining all moving averages features from Pinto`s kernel with some other sample statistics to improve the fit. I tried implementing it (using last 5 months of data), and found that the more complex models performed worse. I ran OLS, Cubist, gradient boosting and neural network (first attempts in everything but OLS and few sad attempts in NN), and the best was OLS with the score of 0.649 and the worst was NN with the score 0.719. Is it due to bias variance trade-off (more complex model do not really predict anything new and just add nosiness to predictions) or overfitting? Should I continue with OLS and make more explicit non-linear features (like onpromotion*(promotion_mean-normal_mean) or is_trending*time*standard_deviation), or is it just going to be a waste of time?\n\nMany thanks to all the Kagglers who shared their kernels. I have learnt so much just by looking at the code and trying to adapt it for my own tasks. Particularly, data manipulation will never be the same again after I discovered R data.table, dplyr and foreach packages. \nIf you have any advice for a novice like me, I would be very happy to listen to it." } }