First, congratulations to top 3 teams! It was a very hard problem and that placement is very well deserved.\nThat was a cool competition, thanks Kaggle and Bimbo for that experience!
\n\nOur solution uses week 9 for validation.\nWe trained more than 10 models, validated in week 9. But the final submission is basically a weighted average of 7 models.
\n\nSince Private LB is only in week 11 we don't have lag1 feature values directly available to use in the models (Probably that is the cause of many positions shakes in the end). So we built some models without lag1 features and some models using the predictions of week10 (Public LB) as the lag1 features of week11(Private LB). \nModels we trained:
\n\nOur best single XGB scored around 0.433x on Public LB ;-D
\n\nGiba
", "messages": [ { "id": "133171", "postDate": "08/31/2016 19:28:28", "content": "First, congratulations to top 3 teams! It was a very hard problem and that placement is very well deserved.\nThat was a cool competition, thanks Kaggle and Bimbo for that experience!
\n\nOur solution uses week 9 for validation.\nWe trained more than 10 models, validated in week 9. But the final submission is basically a weighted average of 7 models.
\n\nSince Private LB is only in week 11 we don't have lag1 feature values directly available to use in the models (Probably that is the cause of many positions shakes in the end). So we built some models without lag1 features and some models using the predictions of week10 (Public LB) as the lag1 features of week11(Private LB). \nModels we trained:
\n\nOur best single XGB scored around 0.433x on Public LB ;-D
\n\nGiba
", "rawMarkdown": "First, congratulations to top 3 teams! It was a very hard problem and that placement is very well deserved.\r\nThat was a cool competition, thanks Kaggle and Bimbo for that experience!\r\n\r\nOur solution uses week 9 for validation.\r\nWe trained more than 10 models, validated in week 9. But the final submission is basically a weighted average of 7 models.\r\n\r\nSince Private LB is only in week 11 we don't have lag1 feature values directly available to use in the models (Probably that is the cause of many positions shakes in the end). So we built some models without lag1 features and some models using the predictions of week10 (Public LB) as the lag1 features of week11(Private LB). \r\nModels we trained:\r\n\r\n- XGB( around +100 features: many combinations of lags)\r\n- LR\r\n- libFM(Sparse Features only)\r\n\r\nOur best single XGB scored around 0.433x on Public LB ;-D\r\n\r\nGiba", "votes": null }, { "id": "133189", "postDate": "08/31/2016 21:02:01", "content": "Congratulations & thank you for sharing @Gilberto & team.
\n\nRegarding this "...and some models using the predictions of week10 (Public LB) as the lag1 features of week11(Private LB)", does it have better or worse score? (In my case it yields worse score.)
\n\nWhat's your best single LibFM score?
", "rawMarkdown": "Congratulations & thank you for sharing @Gilberto & team. \r\n\r\nRegarding this \"...*and some models using the predictions of week10 (Public LB) as the lag1 features of week11(Private LB)*\", does it have better or worse score? (In my case it yields worse score.) \r\n\r\nWhat's your best single LibFM score?", "votes": null }, { "id": "133203", "postDate": "08/31/2016 22:28:11", "content": "Congratulations G & R. You are at #4 now.
", "rawMarkdown": "Congratulations G & R. You are at #4 now.", "votes": null }, { "id": "133213", "postDate": "08/31/2016 23:01:26", "content": "Wow... Jumping positions is always good.
\n\n@aldente: I didn't submitted my libFM but it scored 0.452 at week9.
", "rawMarkdown": "Wow... Jumping positions is always good.\r\n\r\n@aldente: I didn't submitted my libFM but it scored 0.452 at week9.", "votes": null }, { "id": "133232", "postDate": "09/01/2016 01:50:55", "content": "@aldente: The model that used the predictions of week 10 as lag 1 and moved the others lags accordingly did 0.0007 worse in the private leaderboard than a model that replicated the predictions of week 10 in week 11, although in our validation for week 11 it was doing considerably better. So I can't be sure if this is a good strategy or not, although intuitively I think it's a good strategy because that is how time series predictions work. I guess the lag 1 problem did not make so much difference at the end.
", "rawMarkdown": "aldente: The model that used the predictions of week 10 as lag 1 and moved the others lags accordingly did 0.0007 worse in the private leaderboard than a model that replicated the predictions of week 10 in week 11, although in our validation for week 11 it was doing considerably better. So I can't be sure if this is a good strategy or not, although intuitively I think it's a good strategy because that is how time series predictions work. I guess the lag 1 problem did not make so much difference at the end.", "votes": null }, { "id": "133238", "postDate": "09/01/2016 03:01:15", "content": "@Regis A. Ely
\n\nyou may pay attention to the fact that week 10 data are downsampled. Therefore, the regular time series approach (you mentioned) is not a fit!
", "rawMarkdown": "Regis A. Ely \r\n\r\nyou may pay attention to the fact that week 10 data are downsampled. Therefore, the regular time series approach (you mentioned) is not a fit!", "votes": null }, { "id": "133242", "postDate": "09/01/2016 03:11:46", "content": "Thank you @Gilberto, @Regis for the responses. I agree with "that is how time series predictions work", I was also not sure so I decided to select the 2 versions as final submissions. In this competition with small amount of test set @bobye, it performs slightly worse, but in real world I think we should choose it.
", "rawMarkdown": "Thank you @Gilberto, @Regis for the responses. I agree with \"*that is how time series predictions work*\", I was also not sure so I decided to select the 2 versions as final submissions. In this competition with small amount of test set @bobye, it performs slightly worse, but in real world I think we should choose it.", "votes": null }, { "id": "133256", "postDate": "09/01/2016 04:17:52", "content": "I forgot to mention that we were not actually using the predictions of week 10 as lag 1, but rather the mean of the predictions by product and client.
", "rawMarkdown": "I forgot to mention that we were not actually using the predictions of week 10 as lag 1, but rather the mean of the predictions by product and client.", "votes": null }, { "id": "133268", "postDate": "09/01/2016 06:50:51", "content": "[quote=Regis A. Ely;133256]
\n\nI forgot to mention that we were not actually using the predictions of week 10 as lag 1, but rather the mean of the predictions by product and client.
\n\n[/quote]
\n\nThat makes sense. I didn't think in that way!
", "rawMarkdown": "[quote=Regis A. Ely;133256]\r\n\r\nI forgot to mention that we were not actually using the predictions of week 10 as lag 1, but rather the mean of the predictions by product and client.\r\n\r\n[/quote]\r\n\r\nThat makes sense. I didn't think in that way!", "votes": null }, { "id": "133269", "postDate": "09/01/2016 06:51:03", "content": "Thanks for sharing, and congratulations!
", "rawMarkdown": "Thanks for sharing, and congratulations!", "votes": null } ], "comments": [ { "id": 133189, "author_name": "aldente", "author_url": "", "post_date": "08/31/2016 21:02:01", "content": "Congratulations & thank you for sharing @Gilberto & team.
\n\nRegarding this "...and some models using the predictions of week10 (Public LB) as the lag1 features of week11(Private LB)", does it have better or worse score? (In my case it yields worse score.)
\n\nWhat's your best single LibFM score?
", "votes": null, "replies": [] }, { "id": 133203, "author_name": "prakulm", "author_url": "", "post_date": "08/31/2016 22:28:11", "content": "Congratulations G & R. You are at #4 now.
", "votes": null, "replies": [] }, { "id": 133213, "author_name": "titericz", "author_url": "", "post_date": "08/31/2016 23:01:26", "content": "Wow... Jumping positions is always good.
\n\n@aldente: I didn't submitted my libFM but it scored 0.452 at week9.
", "votes": null, "replies": [] }, { "id": 133232, "author_name": "regisely", "author_url": "", "post_date": "09/01/2016 01:50:55", "content": "@aldente: The model that used the predictions of week 10 as lag 1 and moved the others lags accordingly did 0.0007 worse in the private leaderboard than a model that replicated the predictions of week 10 in week 11, although in our validation for week 11 it was doing considerably better. So I can't be sure if this is a good strategy or not, although intuitively I think it's a good strategy because that is how time series predictions work. I guess the lag 1 problem did not make so much difference at the end.
", "votes": null, "replies": [] }, { "id": 133238, "author_name": "amazingbob", "author_url": "", "post_date": "09/01/2016 03:01:15", "content": "@Regis A. Ely
\n\nyou may pay attention to the fact that week 10 data are downsampled. Therefore, the regular time series approach (you mentioned) is not a fit!
", "votes": null, "replies": [] }, { "id": 133242, "author_name": "aldente", "author_url": "", "post_date": "09/01/2016 03:11:46", "content": "Thank you @Gilberto, @Regis for the responses. I agree with "that is how time series predictions work", I was also not sure so I decided to select the 2 versions as final submissions. In this competition with small amount of test set @bobye, it performs slightly worse, but in real world I think we should choose it.
", "votes": null, "replies": [] }, { "id": 133256, "author_name": "regisely", "author_url": "", "post_date": "09/01/2016 04:17:52", "content": "I forgot to mention that we were not actually using the predictions of week 10 as lag 1, but rather the mean of the predictions by product and client.
", "votes": null, "replies": [] }, { "id": 133268, "author_name": "amazingbob", "author_url": "", "post_date": "09/01/2016 06:50:51", "content": "[quote=Regis A. Ely;133256]
\n\nI forgot to mention that we were not actually using the predictions of week 10 as lag 1, but rather the mean of the predictions by product and client.
\n\n[/quote]
\n\nThat makes sense. I didn't think in that way!
", "votes": null, "replies": [] }, { "id": 133269, "author_name": "cpmpml", "author_url": "", "post_date": "09/01/2016 06:51:03", "content": "Thanks for sharing, and congratulations!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "133171": "First, congratulations to top 3 teams! It was a very hard problem and that placement is very well deserved.\r\nThat was a cool competition, thanks Kaggle and Bimbo for that experience!\r\n\r\nOur solution uses week 9 for validation.\r\nWe trained more than 10 models, validated in week 9. But the final submission is basically a weighted average of 7 models.\r\n\r\nSince Private LB is only in week 11 we don't have lag1 feature values directly available to use in the models (Probably that is the cause of many positions shakes in the end). So we built some models without lag1 features and some models using the predictions of week10 (Public LB) as the lag1 features of week11(Private LB). \r\nModels we trained:\r\n\r\n- XGB( around +100 features: many combinations of lags)\r\n- LR\r\n- libFM(Sparse Features only)\r\n\r\nOur best single XGB scored around 0.433x on Public LB ;-D\r\n\r\nGiba", "133189": "Congratulations & thank you for sharing @Gilberto & team. \r\n\r\nRegarding this \"...*and some models using the predictions of week10 (Public LB) as the lag1 features of week11(Private LB)*\", does it have better or worse score? (In my case it yields worse score.) \r\n\r\nWhat's your best single LibFM score?", "133203": "Congratulations G & R. You are at #4 now.", "133213": "Wow... Jumping positions is always good.\r\n\r\n@aldente: I didn't submitted my libFM but it scored 0.452 at week9.", "133232": "aldente: The model that used the predictions of week 10 as lag 1 and moved the others lags accordingly did 0.0007 worse in the private leaderboard than a model that replicated the predictions of week 10 in week 11, although in our validation for week 11 it was doing considerably better. So I can't be sure if this is a good strategy or not, although intuitively I think it's a good strategy because that is how time series predictions work. I guess the lag 1 problem did not make so much difference at the end.", "133238": "Regis A. Ely \r\n\r\nyou may pay attention to the fact that week 10 data are downsampled. Therefore, the regular time series approach (you mentioned) is not a fit!", "133242": "Thank you @Gilberto, @Regis for the responses. I agree with \"*that is how time series predictions work*\", I was also not sure so I decided to select the 2 versions as final submissions. In this competition with small amount of test set @bobye, it performs slightly worse, but in real world I think we should choose it.", "133256": "I forgot to mention that we were not actually using the predictions of week 10 as lag 1, but rather the mean of the predictions by product and client.", "133268": "[quote=Regis A. Ely;133256]\r\n\r\nI forgot to mention that we were not actually using the predictions of week 10 as lag 1, but rather the mean of the predictions by product and client.\r\n\r\n[/quote]\r\n\r\nThat makes sense. I didn't think in that way!", "133269": "Thanks for sharing, and congratulations!" }, "source": "meta" }