First, we want to thanks @sjv. We learned a lot from his prevous sharing.
\n\nSecond, thanks to our team members to stay with me, discuss and work hard together until last minute.
\n\nAlso thanks to @CPMP, his success on WTF with CNN encouraged me to peruse breakthrough with deep learning model.
\n\nWe won't introduce the basic model architecture for wavenet.
\n\nWe just post our understanding and our experience with this model. We think the model and the way to train model is naturally robust.
\n\nFeeding data with mini-batch with randomly sampled 128 sequence. Then randomly to choose the start of decode/target date. So, we could say somehow the model will see different data for each training iteration. because the total dataset is around 170000 (seq) x 365 days. My feeling is the wavenet trained in this way will be good to handle overfitting.
\n\nDuring the training, we set the learning rate as 0.0005 and with a learning decay.
\n\nIn fact, the training loss is hard to get decrease after some iteration.
\n\nWe take the approach for validation is step by step. We keep the last 16 days validation data.
\n\nTrigger by WTF top solution, We think the shifted unit_sales and onpromotion data could be used for capture better quarterly and yearly pattern.
\n\nThe way we feeding data gives us robust. but we think there is a little drawback. which is the prediction from model will be slightly 'overfitted' the last several hundreds or thousands of mini-batch data. So the result has a little fluctuation, not very stable. This experience also shown in Top solution in WTF. Also CPMP mentioned by different cv setting he got different performcen.
\n\nThe best way we thought is averaging. In WTF top solution 30 models prediction averaging. We trained the model and after 5000 mini-batch, model will generate a prediction for every 2000 mini-batch.
\n\nAfter certain iteration, choose 5 model average will give better result than single model. But different 5 or 10 model average output still have a little difference on performance from Local CV.
\n\nSo finally we choose EMA to average on prediction. The parameter used for EMA, is from LocalCV.
\n\nNew Item processing, it might only has minor impact, but because we are hanging on the edge of gold zone, so finally I suggest to do some work on new items. which is agreed by all team member to bet on it for gold medal.
\n\nAfter analysis of the data, we found the on the date 2015-07-01, 2015-09-01, 2015-10-01 and 2016-11-27, there are lots of items start to have unit_sales records. but the hard problem is that we do not know in test period most of new items on which day start to have unit_sale. and on which stores? By checking onpromotion information, all onpromotion with new items only on 08-30 and 08-31. by reviewing the data on 2015-10-01 or 2016-11-27, those 'old new items' is shown onpromotion on that day, not before that. So, we bet the most new items will start have unit_sales on 08-30 and 08-31. Before that there might be only a few new item was sold. We do not use model to predict to prevent overfitting, We only use some value to validate with loss calculation on those 'old new items'
\n\nWe also used Max Observation value cap. etc.
\n\nFinally, thanks to our team member work together to get this very good result.
\n\nUpdate, the new item prediction is not good. checked by late submission. the score got slight dropped less than 0.001 when I add new item prediction
", "messages": [ { "id": "269198", "postDate": "01/16/2018 12:51:54", "content": "First, we want to thanks @sjv. We learned a lot from his prevous sharing.
\n\nSecond, thanks to our team members to stay with me, discuss and work hard together until last minute.
\n\nAlso thanks to @CPMP, his success on WTF with CNN encouraged me to peruse breakthrough with deep learning model.
\n\nWe won't introduce the basic model architecture for wavenet.
\n\nWe just post our understanding and our experience with this model. We think the model and the way to train model is naturally robust.
\n\nFeeding data with mini-batch with randomly sampled 128 sequence. Then randomly to choose the start of decode/target date. So, we could say somehow the model will see different data for each training iteration. because the total dataset is around 170000 (seq) x 365 days. My feeling is the wavenet trained in this way will be good to handle overfitting.
\n\nDuring the training, we set the learning rate as 0.0005 and with a learning decay.
\n\nIn fact, the training loss is hard to get decrease after some iteration.
\n\nWe take the approach for validation is step by step. We keep the last 16 days validation data.
\n\nTrigger by WTF top solution, We think the shifted unit_sales and onpromotion data could be used for capture better quarterly and yearly pattern.
\n\nThe way we feeding data gives us robust. but we think there is a little drawback. which is the prediction from model will be slightly 'overfitted' the last several hundreds or thousands of mini-batch data. So the result has a little fluctuation, not very stable. This experience also shown in Top solution in WTF. Also CPMP mentioned by different cv setting he got different performcen.
\n\nThe best way we thought is averaging. In WTF top solution 30 models prediction averaging. We trained the model and after 5000 mini-batch, model will generate a prediction for every 2000 mini-batch.
\n\nAfter certain iteration, choose 5 model average will give better result than single model. But different 5 or 10 model average output still have a little difference on performance from Local CV.
\n\nSo finally we choose EMA to average on prediction. The parameter used for EMA, is from LocalCV.
\n\nNew Item processing, it might only has minor impact, but because we are hanging on the edge of gold zone, so finally I suggest to do some work on new items. which is agreed by all team member to bet on it for gold medal.
\n\nAfter analysis of the data, we found the on the date 2015-07-01, 2015-09-01, 2015-10-01 and 2016-11-27, there are lots of items start to have unit_sales records. but the hard problem is that we do not know in test period most of new items on which day start to have unit_sale. and on which stores? By checking onpromotion information, all onpromotion with new items only on 08-30 and 08-31. by reviewing the data on 2015-10-01 or 2016-11-27, those 'old new items' is shown onpromotion on that day, not before that. So, we bet the most new items will start have unit_sales on 08-30 and 08-31. Before that there might be only a few new item was sold. We do not use model to predict to prevent overfitting, We only use some value to validate with loss calculation on those 'old new items'
\n\nWe also used Max Observation value cap. etc.
\n\nFinally, thanks to our team member work together to get this very good result.
\n\nUpdate, the new item prediction is not good. checked by late submission. the score got slight dropped less than 0.001 when I add new item prediction
", "rawMarkdown": "First, we want to thanks @sjv. We learned a lot from his prevous sharing.\n\nSecond, thanks to our team members to stay with me, discuss and work hard together until last minute. \n\nAlso thanks to @CPMP, his success on WTF with CNN encouraged me to peruse breakthrough with deep learning model.\n\nWe won't introduce the basic model architecture for wavenet. \n\nWe just post our understanding and our experience with this model. We think the model and the way to train model is naturally robust.\n\nFeeding data with mini-batch with randomly sampled 128 sequence. Then randomly to choose the start of decode/target date. So, we could say somehow the model will see different data for each training iteration. because the total dataset is around 170000 (seq) x 365 days. My feeling is the wavenet trained in this way will be good to handle overfitting.\n\nDuring the training, we set the learning rate as 0.0005 and with a learning decay.\n\nIn fact, the training loss is hard to get decrease after some iteration.\n\nWe take the approach for validation is step by step. We keep the last 16 days validation data. \n\nTrigger by WTF top solution, We think the shifted unit_sales and onpromotion data could be used for capture better quarterly and yearly pattern. \n\nThe way we feeding data gives us robust. but we think there is a little drawback. which is the prediction from model will be slightly 'overfitted' the last several hundreds or thousands of mini-batch data. So the result has a little fluctuation, not very stable. This experience also shown in Top solution in WTF. Also CPMP mentioned by different cv setting he got different performcen.\n\nThe best way we thought is averaging. In WTF top solution 30 models prediction averaging. We trained the model and after 5000 mini-batch, model will generate a prediction for every 2000 mini-batch. \n\nAfter certain iteration, choose 5 model average will give better result than single model. But different 5 or 10 model average output still have a little difference on performance from Local CV. \n\nSo finally we choose EMA to average on prediction. The parameter used for EMA, is from LocalCV. \n\nNew Item processing, it might only has minor impact, but because we are hanging on the edge of gold zone, so finally I suggest to do some work on new items. which is agreed by all team member to bet on it for gold medal.\n\nAfter analysis of the data, we found the on the date 2015-07-01, 2015-09-01, 2015-10-01 and 2016-11-27, there are lots of items start to have unit_sales records. but the hard problem is that we do not know in test period most of new items on which day start to have unit_sale. and on which stores? By checking onpromotion information, all onpromotion with new items only on 08-30 and 08-31. by reviewing the data on 2015-10-01 or 2016-11-27, those 'old new items' is shown onpromotion on that day, not before that. So, we bet the most new items will start have unit_sales on 08-30 and 08-31. Before that there might be only a few new item was sold. We do not use model to predict to prevent overfitting, We only use some value to validate with loss calculation on those 'old new items'\n\nWe also used Max Observation value cap. etc.\n\nFinally, thanks to our team member work together to get this very good result. \n\nUpdate, the new item prediction is not good. checked by late submission. the score got slight dropped less than 0.001 when I add new item prediction", "votes": null }, { "id": "269200", "postDate": "01/16/2018 12:55:12", "content": "Thanks for sharing, glad you found my advice to be of interest. You made the difference with new items, good work on that, and congrats on the result! Prize well deserved.
", "rawMarkdown": "Thanks for sharing, glad you found my advice to be of interest. You made the difference with new items, good work on that, and congrats on the result! Prize well deserved.", "votes": null }, { "id": "269202", "postDate": "01/16/2018 12:59:48", "content": "Thanks a lot!.
", "rawMarkdown": "Thanks a lot!.", "votes": null }, { "id": "269207", "postDate": "01/16/2018 13:05:47", "content": "You too leveraged NN predictions at various training time, will definitely try that next time!
", "rawMarkdown": "You too leveraged NN predictions at various training time, will definitely try that next time!", "votes": null }, { "id": "269211", "postDate": "01/16/2018 13:28:33", "content": "Well deserved score!!, the solution is awesome!! thanks for asking me join your team, but I missed the email😓, I am not good at deep learning, and I think this is the most important thing I need to learn in the coming year!
", "rawMarkdown": "Well deserved score!!, the solution is awesome!! thanks for asking me join your team, but I missed the email😓, I am not good at deep learning, and I think this is the most important thing I need to learn in the coming year!", "votes": null }, { "id": "269224", "postDate": "01/16/2018 13:46:49", "content": "Thanks. My teammates are awesome, but i still think it is kind of pity that we missed cooperation. I notice you since summer too. You always perform good. !
", "rawMarkdown": "Thanks. My teammates are awesome, but i still think it is kind of pity that we missed cooperation. I notice you since summer too. You always perform good. !", "votes": null }, { "id": "269231", "postDate": "01/16/2018 13:56:23", "content": "Very well done and thanks a lot to share :)
", "rawMarkdown": "Very well done and thanks a lot to share :)", "votes": null }, { "id": "269232", "postDate": "01/16/2018 13:58:39", "content": "Yeah, good cooperation between teammates is one of the keys to winning the competition, you guys are awesome! hope to team up in the next time😆!
", "rawMarkdown": "Yeah, good cooperation between teammates is one of the keys to winning the competition, you guys are awesome! hope to team up in the next time😆!", "votes": null }, { "id": "269499", "postDate": "01/16/2018 23:37:19", "content": "Thanks a lot!
", "rawMarkdown": "Thanks a lot!", "votes": null }, { "id": "272069", "postDate": "01/22/2018 07:34:11", "content": "Thanks a lot for your detailed overview! I was wondering is it convenient to share your code?
", "rawMarkdown": "Thanks a lot for your detailed overview! I was wondering is it convenient to share your code?", "votes": null }, { "id": "278474", "postDate": "02/06/2018 08:57:42", "content": "Nice analysis for new items, good job!
", "rawMarkdown": "Nice analysis for new items, good job!", "votes": null } ], "comments": [ { "id": 269200, "author_name": "cpmpml", "author_url": "", "post_date": "01/16/2018 12:55:12", "content": "Thanks for sharing, glad you found my advice to be of interest. You made the difference with new items, good work on that, and congrats on the result! Prize well deserved.
", "votes": null, "replies": [ { "id": 269202, "author_name": "luciusyu", "author_url": "", "post_date": "01/16/2018 12:59:48", "content": "Thanks a lot!.
", "votes": null, "replies": [] }, { "id": 269207, "author_name": "cpmpml", "author_url": "", "post_date": "01/16/2018 13:05:47", "content": "You too leveraged NN predictions at various training time, will definitely try that next time!
", "votes": null, "replies": [] } ] }, { "id": 269211, "author_name": "nickycan", "author_url": "", "post_date": "01/16/2018 13:28:33", "content": "Well deserved score!!, the solution is awesome!! thanks for asking me join your team, but I missed the email😓, I am not good at deep learning, and I think this is the most important thing I need to learn in the coming year!
", "votes": null, "replies": [ { "id": 269224, "author_name": "luciusyu", "author_url": "", "post_date": "01/16/2018 13:46:49", "content": "Thanks. My teammates are awesome, but i still think it is kind of pity that we missed cooperation. I notice you since summer too. You always perform good. !
", "votes": null, "replies": [] }, { "id": 269232, "author_name": "nickycan", "author_url": "", "post_date": "01/16/2018 13:58:39", "content": "Yeah, good cooperation between teammates is one of the keys to winning the competition, you guys are awesome! hope to team up in the next time😆!
", "votes": null, "replies": [] } ] }, { "id": 269231, "author_name": "lesaffrea", "author_url": "", "post_date": "01/16/2018 13:56:23", "content": "Very well done and thanks a lot to share :)
", "votes": null, "replies": [ { "id": 269499, "author_name": "luciusyu", "author_url": "", "post_date": "01/16/2018 23:37:19", "content": "Thanks a lot!
", "votes": null, "replies": [] } ] }, { "id": 272069, "author_name": "floralqy", "author_url": "", "post_date": "01/22/2018 07:34:11", "content": "Thanks a lot for your detailed overview! I was wondering is it convenient to share your code?
", "votes": null, "replies": [] }, { "id": 278474, "author_name": "longyin2", "author_url": "", "post_date": "02/06/2018 08:57:42", "content": "Nice analysis for new items, good job!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "269198": "First, we want to thanks @sjv. We learned a lot from his prevous sharing.\n\nSecond, thanks to our team members to stay with me, discuss and work hard together until last minute. \n\nAlso thanks to @CPMP, his success on WTF with CNN encouraged me to peruse breakthrough with deep learning model.\n\nWe won't introduce the basic model architecture for wavenet. \n\nWe just post our understanding and our experience with this model. We think the model and the way to train model is naturally robust.\n\nFeeding data with mini-batch with randomly sampled 128 sequence. Then randomly to choose the start of decode/target date. So, we could say somehow the model will see different data for each training iteration. because the total dataset is around 170000 (seq) x 365 days. My feeling is the wavenet trained in this way will be good to handle overfitting.\n\nDuring the training, we set the learning rate as 0.0005 and with a learning decay.\n\nIn fact, the training loss is hard to get decrease after some iteration.\n\nWe take the approach for validation is step by step. We keep the last 16 days validation data. \n\nTrigger by WTF top solution, We think the shifted unit_sales and onpromotion data could be used for capture better quarterly and yearly pattern. \n\nThe way we feeding data gives us robust. but we think there is a little drawback. which is the prediction from model will be slightly 'overfitted' the last several hundreds or thousands of mini-batch data. So the result has a little fluctuation, not very stable. This experience also shown in Top solution in WTF. Also CPMP mentioned by different cv setting he got different performcen.\n\nThe best way we thought is averaging. In WTF top solution 30 models prediction averaging. We trained the model and after 5000 mini-batch, model will generate a prediction for every 2000 mini-batch. \n\nAfter certain iteration, choose 5 model average will give better result than single model. But different 5 or 10 model average output still have a little difference on performance from Local CV. \n\nSo finally we choose EMA to average on prediction. The parameter used for EMA, is from LocalCV. \n\nNew Item processing, it might only has minor impact, but because we are hanging on the edge of gold zone, so finally I suggest to do some work on new items. which is agreed by all team member to bet on it for gold medal.\n\nAfter analysis of the data, we found the on the date 2015-07-01, 2015-09-01, 2015-10-01 and 2016-11-27, there are lots of items start to have unit_sales records. but the hard problem is that we do not know in test period most of new items on which day start to have unit_sale. and on which stores? By checking onpromotion information, all onpromotion with new items only on 08-30 and 08-31. by reviewing the data on 2015-10-01 or 2016-11-27, those 'old new items' is shown onpromotion on that day, not before that. So, we bet the most new items will start have unit_sales on 08-30 and 08-31. Before that there might be only a few new item was sold. We do not use model to predict to prevent overfitting, We only use some value to validate with loss calculation on those 'old new items'\n\nWe also used Max Observation value cap. etc.\n\nFinally, thanks to our team member work together to get this very good result. \n\nUpdate, the new item prediction is not good. checked by late submission. the score got slight dropped less than 0.001 when I add new item prediction", "269200": "Thanks for sharing, glad you found my advice to be of interest. You made the difference with new items, good work on that, and congrats on the result! Prize well deserved.", "269202": "Thanks a lot!.", "269207": "You too leveraged NN predictions at various training time, will definitely try that next time!", "269211": "Well deserved score!!, the solution is awesome!! thanks for asking me join your team, but I missed the email😓, I am not good at deep learning, and I think this is the most important thing I need to learn in the coming year!", "269224": "Thanks. My teammates are awesome, but i still think it is kind of pity that we missed cooperation. I notice you since summer too. You always perform good. !", "269231": "Very well done and thanks a lot to share :)", "269232": "Yeah, good cooperation between teammates is one of the keys to winning the competition, you guys are awesome! hope to team up in the next time😆!", "269499": "Thanks a lot!", "272069": "Thanks a lot for your detailed overview! I was wondering is it convenient to share your code?", "278474": "Nice analysis for new items, good job!" }, "source": "meta" }