Hey guys, I spoke with a Kaggle admin, Wendy, recently, and since we had presented our results as of 2-3 weeks ago at a data science meetup, we are obligated to share them here as well. We began working on this project as a part of our coursework at New York City Data Science Academy (NYCDSA). Working with the two others in my group, we were able to get to ~0.46 with just a few xgboosts and quite a few engineered features.
\n\nMy teammates are:\nhttps://www.kaggle.com/abkosar\nhttps://www.kaggle.com/hcozart
\n\nHere is our blog post regarding the project, which includes all the material we went over during our presentation at our meetup.
\n\n\n\nThanks!
", "messages": [ { "id": "128400", "postDate": "07/19/2016 21:33:13", "content": "Hey guys, I spoke with a Kaggle admin, Wendy, recently, and since we had presented our results as of 2-3 weeks ago at a data science meetup, we are obligated to share them here as well. We began working on this project as a part of our coursework at New York City Data Science Academy (NYCDSA). Working with the two others in my group, we were able to get to ~0.46 with just a few xgboosts and quite a few engineered features.
\n\nMy teammates are:\nhttps://www.kaggle.com/abkosar\nhttps://www.kaggle.com/hcozart
\n\nHere is our blog post regarding the project, which includes all the material we went over during our presentation at our meetup.
\n\n\n\nThanks!
", "rawMarkdown": "Hey guys, I spoke with a Kaggle admin, Wendy, recently, and since we had presented our results as of 2-3 weeks ago at a data science meetup, we are obligated to share them here as well. We began working on this project as a part of our coursework at New York City Data Science Academy (NYCDSA). Working with the two others in my group, we were able to get to ~0.46 with just a few xgboosts and quite a few engineered features.\r\n\r\nMy teammates are:\r\nhttps://www.kaggle.com/abkosar\r\nhttps://www.kaggle.com/hcozart\r\n\r\nHere is our blog post regarding the project, which includes all the material we went over during our presentation at our meetup.\r\n\r\nhttp://blog.nycdatascience.com/student-works/predicting-demand-historical-sales-data-grupo-bimbo-kaggle-competition/\r\n\r\nThanks!", "votes": null }, { "id": "128401", "postDate": "07/19/2016 22:21:05", "content": "Interesting blog.\nThanks
", "rawMarkdown": "Interesting blog.\r\nThanks", "votes": null }, { "id": "128402", "postDate": "07/19/2016 22:32:41", "content": "Lots of good ideas here. Thanks for sharing.
", "rawMarkdown": "Lots of good ideas here. Thanks for sharing.", "votes": null }, { "id": "128433", "postDate": "07/20/2016 06:22:24", "content": "Thanks for sharing.
", "rawMarkdown": "Thanks for sharing.", "votes": null }, { "id": "128439", "postDate": "07/20/2016 07:10:40", "content": "Thank you very much, this helps a bit people with low amount of memory.
", "rawMarkdown": "Thank you very much, this helps a bit people with low amount of memory.", "votes": null }, { "id": "128510", "postDate": "07/20/2016 20:33:12", "content": "Yes, thanks for sharing!
", "rawMarkdown": "Yes, thanks for sharing!", "votes": null }, { "id": "128592", "postDate": "07/21/2016 15:18:41", "content": "Yeah it definitely helped for low memory. We were able to get cross validation running on computers with 8 GB of memory (although that was just testing the lag features + one new feature each time against the benchmark). For the full model running we needed 16 GB, although with this trick: https://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/22330/there-is-hope-with-xgboost-and-small-hardware others may be able to do better.
\n\nAdditionally, (using R) freeing up unused variables with rm() really helped as well (Example: once the xgb.DMatrix for the training set was created, we didn't need the sparse model matrix or the training set data table, so they were removed before the model begins training).
", "rawMarkdown": "Yeah it definitely helped for low memory. We were able to get cross validation running on computers with 8 GB of memory (although that was just testing the lag features + one new feature each time against the benchmark). For the full model running we needed 16 GB, although with this trick: https://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/22330/there-is-hope-with-xgboost-and-small-hardware others may be able to do better. \r\n\r\nAdditionally, (using R) freeing up unused variables with rm() really helped as well (Example: once the xgb.DMatrix for the training set was created, we didn't need the sparse model matrix or the training set data table, so they were removed before the model begins training).", "votes": null }, { "id": "128664", "postDate": "07/22/2016 12:44:28", "content": "Really interesting article, helped me better understand given features, thanks a lot for sharing.
", "rawMarkdown": "Really interesting article, helped me better understand given features, thanks a lot for sharing.", "votes": null }, { "id": "129632", "postDate": "08/01/2016 06:08:30", "content": "Thank you for sharing your solution. How do you get the "Client Count by Town Id" feature? Since there is no client information in the town id data.
", "rawMarkdown": "Thank you for sharing your solution. How do you get the \"Client Count by Town Id\" feature? Since there is no client information in the town id data.", "votes": null }, { "id": "129637", "postDate": "08/01/2016 06:43:17", "content": "This is awesome :) Thanks for sharing :)
", "rawMarkdown": "This is awesome :) Thanks for sharing :)", "votes": null }, { "id": "129664", "postDate": "08/01/2016 12:35:17", "content": "@Li Li I suspect it is done by first merging cliente_tabla into train by Cliente_ID, then merge town_state into train by Agencia_ID -- then you can simply sum the client number of by Town_ID (which I would again assume to be the first 4 digits of the column Town).
", "rawMarkdown": "Li Li I suspect it is done by first merging cliente_tabla into train by Cliente_ID, then merge town_state into train by Agencia_ID -- then you can simply sum the client number of by Town_ID (which I would again assume to be the first 4 digits of the column Town).", "votes": null }, { "id": "129671", "postDate": "08/01/2016 15:19:43", "content": "Yep! That's correct yilisg.
", "rawMarkdown": "Yep! That's correct yilisg.", "votes": null }, { "id": "129682", "postDate": "08/01/2016 17:50:13", "content": "Thank you for sharing!
", "rawMarkdown": "Thank you for sharing!", "votes": null }, { "id": "129967", "postDate": "08/03/2016 04:04:40", "content": "Really nice article. It helped me understand how to convert demand history to features.
", "rawMarkdown": "Really nice article. It helped me understand how to convert demand history to features.", "votes": null }, { "id": "129978", "postDate": "08/03/2016 05:44:58", "content": "Thank you for sharing this.
", "rawMarkdown": "Thank you for sharing this.", "votes": null }, { "id": "130108", "postDate": "08/03/2016 21:49:02", "content": "@kes367, please be careful as public/private is split by week, public is week 10, private is week 11. https://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/21918/public-versus-private-leaderboard-data-split
\n\nSo, if you use week 10's prediction to calculate week 11's Lag1, it will perform worse at the end. Besides, week 11's Lag1 will have too many NA values (around 70% NA). For this lag method, I did several cross validation, the score of week 11 (private) should have worse score (by around 0.003-0.005) compared to week 10 (public).
\n\nIn this case, maybe by assuming all test set as week 10 only, we have more representative Lag1 feature?
", "rawMarkdown": "kes367, please be careful as public/private is split by week, public is week 10, private is week 11. https://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/21918/public-versus-private-leaderboard-data-split\r\n\r\nSo, if you use week 10's prediction to calculate week 11's Lag1, it will perform worse at the end. Besides, week 11's Lag1 will have too many NA values (around 70% NA). For this lag method, I did several cross validation, the score of week 11 (private) should have worse score (by around 0.003-0.005) compared to week 10 (public). \r\n\r\nIn this case, maybe by assuming all test set as week 10 only, we have more representative Lag1 feature?", "votes": null }, { "id": "130116", "postDate": "08/04/2016 00:55:24", "content": "I think the main challenge is to predict demand for those products which we do not have prior demand information, and it took me long enough to realize that.
\n\nIf prior information is given, like the case of training set, i could get ~ 0.32-0.35 rmsle just predicting lg_mean alone, but i realized my LB score was 0.48, which probably means that for many test cases without prior demand, i am predicting wrongly.
\n\nI am trying features like weight and price of products , and they seem to be slightly anticorrelated with log demand (about .22-.30). Hope that i can implement them in time.
", "rawMarkdown": "I think the main challenge is to predict demand for those products which we do not have prior demand information, and it took me long enough to realize that.\r\n\r\n If prior information is given, like the case of training set, i could get ~ 0.32-0.35 rmsle just predicting lg_mean alone, but i realized my LB score was 0.48, which probably means that for many test cases without prior demand, i am predicting wrongly. \r\n\r\nI am trying features like weight and price of products , and they seem to be slightly anticorrelated with log demand (about .22-.30). Hope that i can implement them in time.", "votes": null }, { "id": "130136", "postDate": "08/04/2016 08:14:59", "content": "" ...i could get ~ 0.32-0.35 rmsle just predicting lg_mean alone, but i realized my LB score was 0.48 ..."
\n\n@leexiongan, is 0.32 cross validation score? My best CV score is around 0.44.
\n\nFor this model I use the same features as explained in the blog, only I didn't try k-means clustering, I used Incremental PCA instead, simply because I'm still not sure how to do the k-means ;)
\n\nI added more lag features based on producto-ruta (to existing producto-cliente). This PR lag mean perform very well also. You can play as well with producto-cliente-agencia lag features.
\n\nMost important is feature engineering, e.g. quadratic + cubic feature interactions, etc. Run many times with different subset of train set, then average the results (a kind of bagging regressor). After around a week of 24 hours/day with limited RAM & 2 cores, so far I managed to get public LB score of 0.45+ with this lag method., thanks again.
", "rawMarkdown": "\" *...i could get ~ 0.32-0.35 rmsle just predicting lg_mean alone, but i realized my LB score was 0.48 ...*\"\r\n\r\n@leexiongan, is 0.32 cross validation score? My best CV score is around 0.44. \r\n\r\nFor this model I use the same features as explained in the blog, only I didn't try k-means clustering, I used [Incremental PCA][1] instead, simply because I'm still not sure how to do the k-means ;) \r\n\r\nI added more lag features based on producto-ruta (to existing producto-cliente). This PR lag mean perform very well also. You can play as well with producto-cliente-agencia lag features. \r\n\r\nMost important is feature engineering, e.g. quadratic + cubic feature interactions, etc. Run many times with different subset of train set, then average the results (a kind of bagging regressor). After around a week of 24 hours/day with limited RAM & 2 cores, so far I managed to get public LB score of 0.45+ with this lag method., thanks again. \r\n\r\n [1]: http://scikit-learn.org/stable/auto_examples/decomposition/plot_incremental_pca.html", "votes": null }, { "id": "130137", "postDate": "08/04/2016 09:26:39", "content": "@ aldente \nI think my 0.32-0.35 was is a mistake, but it was CV error, using log_means method, shown here " https://www.kaggle.com/happycube/grupo-bimbo-inventory-demand/log-means-in-python-48578-v2" .
\n\nUsing the lg_means without machine learning model, i compared lg_means with week6, and i got 0.32. I must however point out that i used CV data to calculate the lg_means, which might explain my low cv error. I think my 0.32 is an error.
\n\nOh, i was thinking about simulating noise in the dataset, and since much of the test set lacks lag values, would you thinking removing some lags randomly make the model more robust?
", "rawMarkdown": "aldente \r\nI think my 0.32-0.35 was is a mistake, but it was CV error, using log_means method, shown here \" https://www.kaggle.com/happycube/grupo-bimbo-inventory-demand/log-means-in-python-48578-v2\" . \r\n\r\nUsing the lg_means without machine learning model, i compared lg_means with week6, and i got 0.32. I must however point out that i used CV data to calculate the lg_means, which might explain my low cv error. I think my 0.32 is an error.\r\n\r\nOh, i was thinking about simulating noise in the dataset, and since much of the test set lacks lag values, would you thinking removing some lags randomly make the model more robust?", "votes": null }, { "id": "130260", "postDate": "08/05/2016 03:38:29", "content": "Edited: Nevermind :D Reread your post @aldente and it makes sense now. But yeah either way the lag1 for week 11 is going to be a bit weird.
", "rawMarkdown": "Edited: Nevermind :D Reread your post @aldente and it makes sense now. But yeah either way the lag1 for week 11 is going to be a bit weird.", "votes": null }, { "id": "130347", "postDate": "08/05/2016 21:17:32", "content": "@kes367, thanks.
\n\nI'll select two final submissions: (1) default = week 11's lag1 uses train's week 9, (2) with week 11's lag1 uses week 10's prediction.
\n\nI wonder how low can we go with this lag method? Does full-set fitting perform better than average of subsets? Anyone tried full-set fitting with Xgb's external memory, or Spark? What's your best LB score? Thank you ;)
", "rawMarkdown": "kes367, thanks. \r\n\r\nI'll select two final submissions: (1) default = week 11's lag1 uses train's week 9, (2) with week 11's lag1 uses week 10's prediction. \r\n\r\nI wonder how low can we go with this lag method? Does full-set fitting perform better than average of subsets? Anyone tried full-set fitting with Xgb's external memory, or Spark? What's your best LB score? Thank you ;)", "votes": null } ], "comments": [ { "id": 128401, "author_name": "yangyang", "author_url": "", "post_date": "07/19/2016 22:21:05", "content": "Interesting blog.\nThanks
", "votes": null, "replies": [] }, { "id": 128402, "author_name": "centsoft", "author_url": "", "post_date": "07/19/2016 22:32:41", "content": "Lots of good ideas here. Thanks for sharing.
", "votes": null, "replies": [] }, { "id": 128433, "author_name": "", "author_url": "", "post_date": "07/20/2016 06:22:24", "content": "Thanks for sharing.
", "votes": null, "replies": [] }, { "id": 128439, "author_name": "aldente", "author_url": "", "post_date": "07/20/2016 07:10:40", "content": "Thank you very much, this helps a bit people with low amount of memory.
", "votes": null, "replies": [] }, { "id": 128510, "author_name": "mrooijer", "author_url": "", "post_date": "07/20/2016 20:33:12", "content": "Yes, thanks for sharing!
", "votes": null, "replies": [] }, { "id": 128592, "author_name": "kes367", "author_url": "", "post_date": "07/21/2016 15:18:41", "content": "Yeah it definitely helped for low memory. We were able to get cross validation running on computers with 8 GB of memory (although that was just testing the lag features + one new feature each time against the benchmark). For the full model running we needed 16 GB, although with this trick: https://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/22330/there-is-hope-with-xgboost-and-small-hardware others may be able to do better.
\n\nAdditionally, (using R) freeing up unused variables with rm() really helped as well (Example: once the xgb.DMatrix for the training set was created, we didn't need the sparse model matrix or the training set data table, so they were removed before the model begins training).
", "votes": null, "replies": [] }, { "id": 128664, "author_name": "liyanau", "author_url": "", "post_date": "07/22/2016 12:44:28", "content": "Really interesting article, helped me better understand given features, thanks a lot for sharing.
", "votes": null, "replies": [] }, { "id": 129632, "author_name": "aikinogard", "author_url": "", "post_date": "08/01/2016 06:08:30", "content": "Thank you for sharing your solution. How do you get the "Client Count by Town Id" feature? Since there is no client information in the town id data.
", "votes": null, "replies": [] }, { "id": 129637, "author_name": "vonatzki", "author_url": "", "post_date": "08/01/2016 06:43:17", "content": "This is awesome :) Thanks for sharing :)
", "votes": null, "replies": [] }, { "id": 129664, "author_name": "yilihome", "author_url": "", "post_date": "08/01/2016 12:35:17", "content": "@Li Li I suspect it is done by first merging cliente_tabla into train by Cliente_ID, then merge town_state into train by Agencia_ID -- then you can simply sum the client number of by Town_ID (which I would again assume to be the first 4 digits of the column Town).
", "votes": null, "replies": [] }, { "id": 129671, "author_name": "kes367", "author_url": "", "post_date": "08/01/2016 15:19:43", "content": "Yep! That's correct yilisg.
", "votes": null, "replies": [] }, { "id": 129682, "author_name": "zachmayer", "author_url": "", "post_date": "08/01/2016 17:50:13", "content": "Thank you for sharing!
", "votes": null, "replies": [] }, { "id": 129967, "author_name": "vinayshuddar", "author_url": "", "post_date": "08/03/2016 04:04:40", "content": "Really nice article. It helped me understand how to convert demand history to features.
", "votes": null, "replies": [] }, { "id": 129978, "author_name": "angrycat", "author_url": "", "post_date": "08/03/2016 05:44:58", "content": "Thank you for sharing this.
", "votes": null, "replies": [] }, { "id": 130108, "author_name": "aldente", "author_url": "", "post_date": "08/03/2016 21:49:02", "content": "@kes367, please be careful as public/private is split by week, public is week 10, private is week 11. https://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/21918/public-versus-private-leaderboard-data-split
\n\nSo, if you use week 10's prediction to calculate week 11's Lag1, it will perform worse at the end. Besides, week 11's Lag1 will have too many NA values (around 70% NA). For this lag method, I did several cross validation, the score of week 11 (private) should have worse score (by around 0.003-0.005) compared to week 10 (public).
\n\nIn this case, maybe by assuming all test set as week 10 only, we have more representative Lag1 feature?
", "votes": null, "replies": [] }, { "id": 130116, "author_name": "leexiongan", "author_url": "", "post_date": "08/04/2016 00:55:24", "content": "I think the main challenge is to predict demand for those products which we do not have prior demand information, and it took me long enough to realize that.
\n\nIf prior information is given, like the case of training set, i could get ~ 0.32-0.35 rmsle just predicting lg_mean alone, but i realized my LB score was 0.48, which probably means that for many test cases without prior demand, i am predicting wrongly.
\n\nI am trying features like weight and price of products , and they seem to be slightly anticorrelated with log demand (about .22-.30). Hope that i can implement them in time.
", "votes": null, "replies": [] }, { "id": 130136, "author_name": "aldente", "author_url": "", "post_date": "08/04/2016 08:14:59", "content": "" ...i could get ~ 0.32-0.35 rmsle just predicting lg_mean alone, but i realized my LB score was 0.48 ..."
\n\n@leexiongan, is 0.32 cross validation score? My best CV score is around 0.44.
\n\nFor this model I use the same features as explained in the blog, only I didn't try k-means clustering, I used Incremental PCA instead, simply because I'm still not sure how to do the k-means ;)
\n\nI added more lag features based on producto-ruta (to existing producto-cliente). This PR lag mean perform very well also. You can play as well with producto-cliente-agencia lag features.
\n\nMost important is feature engineering, e.g. quadratic + cubic feature interactions, etc. Run many times with different subset of train set, then average the results (a kind of bagging regressor). After around a week of 24 hours/day with limited RAM & 2 cores, so far I managed to get public LB score of 0.45+ with this lag method., thanks again.
", "votes": null, "replies": [] }, { "id": 130137, "author_name": "leexiongan", "author_url": "", "post_date": "08/04/2016 09:26:39", "content": "@ aldente \nI think my 0.32-0.35 was is a mistake, but it was CV error, using log_means method, shown here " https://www.kaggle.com/happycube/grupo-bimbo-inventory-demand/log-means-in-python-48578-v2" .
\n\nUsing the lg_means without machine learning model, i compared lg_means with week6, and i got 0.32. I must however point out that i used CV data to calculate the lg_means, which might explain my low cv error. I think my 0.32 is an error.
\n\nOh, i was thinking about simulating noise in the dataset, and since much of the test set lacks lag values, would you thinking removing some lags randomly make the model more robust?
", "votes": null, "replies": [] }, { "id": 130260, "author_name": "kes367", "author_url": "", "post_date": "08/05/2016 03:38:29", "content": "Edited: Nevermind :D Reread your post @aldente and it makes sense now. But yeah either way the lag1 for week 11 is going to be a bit weird.
", "votes": null, "replies": [] }, { "id": 130347, "author_name": "aldente", "author_url": "", "post_date": "08/05/2016 21:17:32", "content": "@kes367, thanks.
\n\nI'll select two final submissions: (1) default = week 11's lag1 uses train's week 9, (2) with week 11's lag1 uses week 10's prediction.
\n\nI wonder how low can we go with this lag method? Does full-set fitting perform better than average of subsets? Anyone tried full-set fitting with Xgb's external memory, or Spark? What's your best LB score? Thank you ;)
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "128400": "Hey guys, I spoke with a Kaggle admin, Wendy, recently, and since we had presented our results as of 2-3 weeks ago at a data science meetup, we are obligated to share them here as well. We began working on this project as a part of our coursework at New York City Data Science Academy (NYCDSA). Working with the two others in my group, we were able to get to ~0.46 with just a few xgboosts and quite a few engineered features.\r\n\r\nMy teammates are:\r\nhttps://www.kaggle.com/abkosar\r\nhttps://www.kaggle.com/hcozart\r\n\r\nHere is our blog post regarding the project, which includes all the material we went over during our presentation at our meetup.\r\n\r\nhttp://blog.nycdatascience.com/student-works/predicting-demand-historical-sales-data-grupo-bimbo-kaggle-competition/\r\n\r\nThanks!", "128401": "Interesting blog.\r\nThanks", "128402": "Lots of good ideas here. Thanks for sharing.", "128433": "Thanks for sharing.", "128439": "Thank you very much, this helps a bit people with low amount of memory.", "128510": "Yes, thanks for sharing!", "128592": "Yeah it definitely helped for low memory. We were able to get cross validation running on computers with 8 GB of memory (although that was just testing the lag features + one new feature each time against the benchmark). For the full model running we needed 16 GB, although with this trick: https://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/22330/there-is-hope-with-xgboost-and-small-hardware others may be able to do better. \r\n\r\nAdditionally, (using R) freeing up unused variables with rm() really helped as well (Example: once the xgb.DMatrix for the training set was created, we didn't need the sparse model matrix or the training set data table, so they were removed before the model begins training).", "128664": "Really interesting article, helped me better understand given features, thanks a lot for sharing.", "129632": "Thank you for sharing your solution. How do you get the \"Client Count by Town Id\" feature? Since there is no client information in the town id data.", "129637": "This is awesome :) Thanks for sharing :)", "129664": "Li Li I suspect it is done by first merging cliente_tabla into train by Cliente_ID, then merge town_state into train by Agencia_ID -- then you can simply sum the client number of by Town_ID (which I would again assume to be the first 4 digits of the column Town).", "129671": "Yep! That's correct yilisg.", "129682": "Thank you for sharing!", "129967": "Really nice article. It helped me understand how to convert demand history to features.", "129978": "Thank you for sharing this.", "130108": "kes367, please be careful as public/private is split by week, public is week 10, private is week 11. https://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/21918/public-versus-private-leaderboard-data-split\r\n\r\nSo, if you use week 10's prediction to calculate week 11's Lag1, it will perform worse at the end. Besides, week 11's Lag1 will have too many NA values (around 70% NA). For this lag method, I did several cross validation, the score of week 11 (private) should have worse score (by around 0.003-0.005) compared to week 10 (public). \r\n\r\nIn this case, maybe by assuming all test set as week 10 only, we have more representative Lag1 feature?", "130116": "I think the main challenge is to predict demand for those products which we do not have prior demand information, and it took me long enough to realize that.\r\n\r\n If prior information is given, like the case of training set, i could get ~ 0.32-0.35 rmsle just predicting lg_mean alone, but i realized my LB score was 0.48, which probably means that for many test cases without prior demand, i am predicting wrongly. \r\n\r\nI am trying features like weight and price of products , and they seem to be slightly anticorrelated with log demand (about .22-.30). Hope that i can implement them in time.", "130136": "\" *...i could get ~ 0.32-0.35 rmsle just predicting lg_mean alone, but i realized my LB score was 0.48 ...*\"\r\n\r\n@leexiongan, is 0.32 cross validation score? My best CV score is around 0.44. \r\n\r\nFor this model I use the same features as explained in the blog, only I didn't try k-means clustering, I used [Incremental PCA][1] instead, simply because I'm still not sure how to do the k-means ;) \r\n\r\nI added more lag features based on producto-ruta (to existing producto-cliente). This PR lag mean perform very well also. You can play as well with producto-cliente-agencia lag features. \r\n\r\nMost important is feature engineering, e.g. quadratic + cubic feature interactions, etc. Run many times with different subset of train set, then average the results (a kind of bagging regressor). After around a week of 24 hours/day with limited RAM & 2 cores, so far I managed to get public LB score of 0.45+ with this lag method., thanks again. \r\n\r\n [1]: http://scikit-learn.org/stable/auto_examples/decomposition/plot_incremental_pca.html", "130137": "aldente \r\nI think my 0.32-0.35 was is a mistake, but it was CV error, using log_means method, shown here \" https://www.kaggle.com/happycube/grupo-bimbo-inventory-demand/log-means-in-python-48578-v2\" . \r\n\r\nUsing the lg_means without machine learning model, i compared lg_means with week6, and i got 0.32. I must however point out that i used CV data to calculate the lg_means, which might explain my low cv error. I think my 0.32 is an error.\r\n\r\nOh, i was thinking about simulating noise in the dataset, and since much of the test set lacks lag values, would you thinking removing some lags randomly make the model more robust?", "130260": "Edited: Nevermind :D Reread your post @aldente and it makes sense now. But yeah either way the lag1 for week 11 is going to be a bit weird.", "130347": "kes367, thanks. \r\n\r\nI'll select two final submissions: (1) default = week 11's lag1 uses train's week 9, (2) with week 11's lag1 uses week 10's prediction. \r\n\r\nI wonder how low can we go with this lag method? Does full-set fitting perform better than average of subsets? Anyone tried full-set fitting with Xgb's external memory, or Spark? What's your best LB score? Thank you ;)" }, "source": "meta" }