I was wondering if the organizers could provide some information regarding the items that appear in the test set but not the train set. In particular, are these items lacking prior data because they are newly introduced in the test timeframe? Are they continuing products that just haven't had any data logged? Or are they continuing products that have zero sales for the entirety of the train timeframe? There's probably not a straight answer to this, but any information will help us make informed assumptions for our models.
", "messages": [ { "id": "241016", "postDate": "11/07/2017 21:06:11", "content": "I was wondering if the organizers could provide some information regarding the items that appear in the test set but not the train set. In particular, are these items lacking prior data because they are newly introduced in the test timeframe? Are they continuing products that just haven't had any data logged? Or are they continuing products that have zero sales for the entirety of the train timeframe? There's probably not a straight answer to this, but any information will help us make informed assumptions for our models.
", "rawMarkdown": "I was wondering if the organizers could provide some information regarding the items that appear in the test set but not the train set. In particular, are these items lacking prior data because they are newly introduced in the test timeframe? Are they continuing products that just haven't had any data logged? Or are they continuing products that have zero sales for the entirety of the train timeframe? There's probably not a straight answer to this, but any information will help us make informed assumptions for our models.", "votes": null }, { "id": "241059", "postDate": "11/07/2017 23:43:32", "content": "We have a little bit more than nothing. They are included in the items.csv dataset.
", "rawMarkdown": "We have a little bit more than nothing. They are included in the items.csv dataset.", "votes": null }, { "id": "241060", "postDate": "11/07/2017 23:48:36", "content": "I don't think we'll be given any extra info, for two reasons:
\n\nThe less information we have about the relationship between the train and test sets, the more the competition becomes a lottery. As it is, teams will bet on certain assumptions and those who place the correct bet will have the advantage in the final evaluation. I do agree that if the organizers were concerned about this they would have done something, but I really don't see how it's in their best interest to handle it as they have. Anyways, the best strategy for now is to just do trial and error with different strategies and use the public lb as validation, which kinda sucks.
", "rawMarkdown": "The less information we have about the relationship between the train and test sets, the more the competition becomes a lottery. As it is, teams will bet on certain assumptions and those who place the correct bet will have the advantage in the final evaluation. I do agree that if the organizers were concerned about this they would have done something, but I really don't see how it's in their best interest to handle it as they have. Anyways, the best strategy for now is to just do trial and error with different strategies and use the public lb as validation, which kinda sucks.", "votes": null }, { "id": "241125", "postDate": "11/08/2017 06:42:30", "content": "Thank you for bringing up this issue. I have so far avoided dealing with this issue by predicting zeros for all new item x store combinations. But I guess inevitably I'll be optimizing them without a proper validation set and overfitting the leaderboard at the later stage.
", "rawMarkdown": "Thank you for bringing up this issue. I have so far avoided dealing with this issue by predicting zeros for all new item x store combinations. But I guess inevitably I'll be optimizing them without a proper validation set and overfitting the leaderboard at the later stage.", "votes": null }, { "id": "241352", "postDate": "11/08/2017 16:38:36", "content": "Look at this kernel: https://www.kaggle.com/ericdfr/new-items-in-stores/
\nThere is an unusual number of new items in stores the first day of the test dataset!
\nI guess they remove history of sales for some items in some stores. I don't know why they've done that. It sucks if you want to use a time series forecast approach.
Items that appear in the test set but not the train set are new product introductions. This is very common in our business and we must find ways to predict sales of those products to accurately replenish the stores. A possible way to forecast new product sales is using surrogates. That is finding similar products using family and class attributes, finding sales patterns in those similar products, possibly clustering those similar products and choosing the cluster with the best forecasting results.
", "rawMarkdown": "Items that appear in the test set but not the train set are new product introductions. This is very common in our business and we must find ways to predict sales of those products to accurately replenish the stores. A possible way to forecast new product sales is using surrogates. That is finding similar products using family and class attributes, finding sales patterns in those similar products, possibly clustering those similar products and choosing the cluster with the best forecasting results.", "votes": null }, { "id": "241565", "postDate": "11/09/2017 08:03:04", "content": "Thanks for your reply.
\nIntroduction of new products is a very common case and I only count 60 items that are new in all stores. For me, that's not the problem.
\nThe fact that more than 40k item/store combo are new is the real problem and I think is an uncommon case. Don't you remove records for these products?
\nSee this kernel for more details: https://www.kaggle.com/ericdfr/new-items-in-stores/
We did not manipulate the data in any way, shape or form. It was extracted AS IS from our sales transaction log. At most we curated which stores and items to include in the competition to have a coherent data set that reflects the reality and subtleties of the supermarket business. We will look into the 40k item/store combo to try to find the possible reasons of such a high number and will share with you our findings.
", "rawMarkdown": "We did not manipulate the data in any way, shape or form. It was extracted AS IS from our sales transaction log. At most we curated which stores and items to include in the competition to have a coherent data set that reflects the reality and subtleties of the supermarket business. We will look into the 40k item/store combo to try to find the possible reasons of such a high number and will share with you our findings.", "votes": null }, { "id": "241716", "postDate": "11/09/2017 16:57:25", "content": "The item/store combo is not equal to the new item. They just create the test data for convenience by including all item/store combo although like store A never sells item 1.
", "rawMarkdown": "The item/store combo is not equal to the new item. They just create the test data for convenience by including all item/store combo although like store A never sells item 1.", "votes": null }, { "id": "242124", "postDate": "11/10/2017 17:13:56", "content": "So for each of new items in test set that are not in train can we expect to have >0 sales?
", "rawMarkdown": "So for each of new items in test set that are not in train can we expect to have >0 sales?", "votes": null }, { "id": "243095", "postDate": "11/13/2017 10:58:38", "content": "\n\n\nCorporación Favorita wrote
\n \n\n\nWe will look into the 40k item/store combo to try to find the possible reasons of such a high number and will share with you our findings.
\n
Have you found something about these 40k item/store combos?
", "rawMarkdown": "> **Corporación Favorita wrote**\n> \n> > We will look into the 40k item/store combo to try to find the possible reasons of such a high number and will share with you our findings. \n\nHave you found something about these 40k item/store combos?", "votes": null }, { "id": "243109", "postDate": "11/13/2017 11:43:25", "content": "Yes #MarcellFlygare is right!
", "rawMarkdown": "Yes #MarcellFlygare is right!", "votes": null }, { "id": "246122", "postDate": "11/20/2017 15:46:01", "content": "You could probably divide the training set into two parts, training_ (which contains items not in testing_) and testing_ (which contains items not seen in training_), by taking into account the percentages of unseen in the original train and test datasets. Then, you could probably see how your model is performing on the unseen data.
", "rawMarkdown": "You could probably divide the training set into two parts, training_ (which contains items not in testing_) and testing_ (which contains items not seen in training_), by taking into account the percentages of unseen in the original train and test datasets. Then, you could probably see how your model is performing on the unseen data.", "votes": null }, { "id": "247165", "postDate": "11/22/2017 13:24:03", "content": "\n\n\nEric D wrote
\n \n\n\n \n> Corporación Favorita wrote\n > \n > > We will look into the 40k item/store combo to try to find the possible reasons of such a high number and will share with you our findings.
\nHave you found something about these 40k item/store combos?
\n
No news about that?
", "rawMarkdown": "> **Eric D wrote**\n> \n> > \n> > **Corporación Favorita wrote**\n> > \n> > > We will look into the 40k item/store combo to try to find the possible reasons of such a high number and will share with you our findings. \n> \n> Have you found something about these 40k item/store combos?\n\nNo news about that?", "votes": null }, { "id": "247312", "postDate": "11/22/2017 19:25:04", "content": "I believe FengLi is right.
\n~~
\nThe problem with trying to predict new items is that we do not know which stores the new items are in. That is, if an item starts being sold at one store, 810 (54 stores * 15 days) rows are added to the test set. We do not know which stores actually have a chance of selling that item.
\n~~
\nI did some analysis on this, and I have not come up with an easy way to tell which store an item will be added to - maybe there is a way, e.g: looking at which similar items seem to be 'phased out'. This of course takes us out of business-applicable predicting.
\n~~
\nUnfortunately, I think it is to late for a release of which stores the new items are actually being sold at.
Thank you for inspiring me for solving this problem. I am wondering why the result is worse if I use the unit_sales of the same store and class as the new item for prediction. I cannot think of other methods to predict. I would appreciate it very much if there are any inspirations.
", "rawMarkdown": "Thank you for inspiring me for solving this problem. I am wondering why the result is worse if I use the unit_sales of the same store and class as the new item for prediction. I cannot think of other methods to predict. I would appreciate it very much if there are any inspirations.", "votes": null }, { "id": "257292", "postDate": "12/13/2017 22:33:55", "content": "I'm jumping in here late, regarding the discussion of new products, in order to clarify a bit. As has been mentioned, there are a very small number of new items that were introduced in the test set. That does not explain the apparent high number of new items seen on the first day of the test set. The reason for this is that the training set does not include records for zero sales. The test set, though, includes all store / item combinations, whether or not that item was seen previously in a store.
\n\nIf there are any other questions about this, please let me know. Thanks!
", "rawMarkdown": "I'm jumping in here late, regarding the discussion of new products, in order to clarify a bit. As has been mentioned, there are a very small number of new items that were introduced in the test set. That does not explain the apparent high number of new items seen on the first day of the test set. The reason for this is that the training set does not include records for zero sales. The test set, though, includes all store / item combinations, whether or not that item was seen previously in a store.\n\nIf there are any other questions about this, please let me know. Thanks!", "votes": null }, { "id": "260023", "postDate": "12/19/2017 12:52:09", "content": "So, any info on each item sale onset? Especially the missing ones.
", "rawMarkdown": "So, any info on each item sale onset? Especially the missing ones.", "votes": null }, { "id": "267925", "postDate": "01/12/2018 19:06:44", "content": "Can we assume that all new items (or almost all new items) had at least one sale in at least one store during the test period? Or are there a significant number of new product introductions that do not sell a single unit anywhere in the first 16 days (and such products would be included in the test set)? (And, for that matter, were some of the items in the test set already introduced before the test period and could have been in stores, not selling, for days or months before the test period?)
", "rawMarkdown": "Can we assume that all new items (or almost all new items) had at least one sale in at least one store during the test period? Or are there a significant number of new product introductions that do not sell a single unit anywhere in the first 16 days (and such products would be included in the test set)? (And, for that matter, were some of the items in the test set already introduced before the test period and could have been in stores, not selling, for days or months before the test period?)", "votes": null } ], "comments": [ { "id": 241059, "author_name": "flygare", "author_url": "", "post_date": "11/07/2017 23:43:32", "content": "We have a little bit more than nothing. They are included in the items.csv dataset.
", "votes": null, "replies": [] }, { "id": 241060, "author_name": "konradb", "author_url": "", "post_date": "11/07/2017 23:48:36", "content": "I don't think we'll be given any extra info, for two reasons:
\n\nThe less information we have about the relationship between the train and test sets, the more the competition becomes a lottery. As it is, teams will bet on certain assumptions and those who place the correct bet will have the advantage in the final evaluation. I do agree that if the organizers were concerned about this they would have done something, but I really don't see how it's in their best interest to handle it as they have. Anyways, the best strategy for now is to just do trial and error with different strategies and use the public lb as validation, which kinda sucks.
", "votes": null, "replies": [] }, { "id": 241125, "author_name": "ceshine", "author_url": "", "post_date": "11/08/2017 06:42:30", "content": "Thank you for bringing up this issue. I have so far avoided dealing with this issue by predicting zeros for all new item x store combinations. But I guess inevitably I'll be optimizing them without a proper validation set and overfitting the leaderboard at the later stage.
", "votes": null, "replies": [] }, { "id": 241352, "author_name": "ericdfr", "author_url": "", "post_date": "11/08/2017 16:38:36", "content": "Look at this kernel: https://www.kaggle.com/ericdfr/new-items-in-stores/
\nThere is an unusual number of new items in stores the first day of the test dataset!
\nI guess they remove history of sales for some items in some stores. I don't know why they've done that. It sucks if you want to use a time series forecast approach.
Thank you for inspiring me for solving this problem. I am wondering why the result is worse if I use the unit_sales of the same store and class as the new item for prediction. I cannot think of other methods to predict. I would appreciate it very much if there are any inspirations.
", "votes": null, "replies": [] } ] }, { "id": 241526, "author_name": "gyorgy", "author_url": "", "post_date": "11/09/2017 04:32:26", "content": "Items that appear in the test set but not the train set are new product introductions. This is very common in our business and we must find ways to predict sales of those products to accurately replenish the stores. A possible way to forecast new product sales is using surrogates. That is finding similar products using family and class attributes, finding sales patterns in those similar products, possibly clustering those similar products and choosing the cluster with the best forecasting results.
", "votes": null, "replies": [ { "id": 241565, "author_name": "ericdfr", "author_url": "", "post_date": "11/09/2017 08:03:04", "content": "Thanks for your reply.
\nIntroduction of new products is a very common case and I only count 60 items that are new in all stores. For me, that's not the problem.
\nThe fact that more than 40k item/store combo are new is the real problem and I think is an uncommon case. Don't you remove records for these products?
\nSee this kernel for more details: https://www.kaggle.com/ericdfr/new-items-in-stores/
We did not manipulate the data in any way, shape or form. It was extracted AS IS from our sales transaction log. At most we curated which stores and items to include in the competition to have a coherent data set that reflects the reality and subtleties of the supermarket business. We will look into the 40k item/store combo to try to find the possible reasons of such a high number and will share with you our findings.
", "votes": null, "replies": [] }, { "id": 241716, "author_name": "beedata", "author_url": "", "post_date": "11/09/2017 16:57:25", "content": "The item/store combo is not equal to the new item. They just create the test data for convenience by including all item/store combo although like store A never sells item 1.
", "votes": null, "replies": [] }, { "id": 242124, "author_name": "iamivo", "author_url": "", "post_date": "11/10/2017 17:13:56", "content": "So for each of new items in test set that are not in train can we expect to have >0 sales?
", "votes": null, "replies": [] }, { "id": 243095, "author_name": "ericdfr", "author_url": "", "post_date": "11/13/2017 10:58:38", "content": "\n\n\nCorporación Favorita wrote
\n \n\n\nWe will look into the 40k item/store combo to try to find the possible reasons of such a high number and will share with you our findings.
\n
Have you found something about these 40k item/store combos?
", "votes": null, "replies": [] }, { "id": 247165, "author_name": "ericdfr", "author_url": "", "post_date": "11/22/2017 13:24:03", "content": "\n\n\nEric D wrote
\n \n\n\n \n> Corporación Favorita wrote\n > \n > > We will look into the 40k item/store combo to try to find the possible reasons of such a high number and will share with you our findings.
\nHave you found something about these 40k item/store combos?
\n
No news about that?
", "votes": null, "replies": [] }, { "id": 247312, "author_name": "npa02012", "author_url": "", "post_date": "11/22/2017 19:25:04", "content": "I believe FengLi is right.
\n~~
\nThe problem with trying to predict new items is that we do not know which stores the new items are in. That is, if an item starts being sold at one store, 810 (54 stores * 15 days) rows are added to the test set. We do not know which stores actually have a chance of selling that item.
\n~~
\nI did some analysis on this, and I have not come up with an easy way to tell which store an item will be added to - maybe there is a way, e.g: looking at which similar items seem to be 'phased out'. This of course takes us out of business-applicable predicting.
\n~~
\nUnfortunately, I think it is to late for a release of which stores the new items are actually being sold at.
Yes #MarcellFlygare is right!
", "votes": null, "replies": [] }, { "id": 246122, "author_name": "rawi707", "author_url": "", "post_date": "11/20/2017 15:46:01", "content": "You could probably divide the training set into two parts, training_ (which contains items not in testing_) and testing_ (which contains items not seen in training_), by taking into account the percentages of unseen in the original train and test datasets. Then, you could probably see how your model is performing on the unseen data.
", "votes": null, "replies": [] }, { "id": 257292, "author_name": "inversion", "author_url": "", "post_date": "12/13/2017 22:33:55", "content": "I'm jumping in here late, regarding the discussion of new products, in order to clarify a bit. As has been mentioned, there are a very small number of new items that were introduced in the test set. That does not explain the apparent high number of new items seen on the first day of the test set. The reason for this is that the training set does not include records for zero sales. The test set, though, includes all store / item combinations, whether or not that item was seen previously in a store.
\n\nIf there are any other questions about this, please let me know. Thanks!
", "votes": null, "replies": [ { "id": 260023, "author_name": "rawi707", "author_url": "", "post_date": "12/19/2017 12:52:09", "content": "So, any info on each item sale onset? Especially the missing ones.
", "votes": null, "replies": [] }, { "id": 267925, "author_name": "aharless", "author_url": "", "post_date": "01/12/2018 19:06:44", "content": "Can we assume that all new items (or almost all new items) had at least one sale in at least one store during the test period? Or are there a significant number of new product introductions that do not sell a single unit anywhere in the first 16 days (and such products would be included in the test set)? (And, for that matter, were some of the items in the test set already introduced before the test period and could have been in stores, not selling, for days or months before the test period?)
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "241016": "I was wondering if the organizers could provide some information regarding the items that appear in the test set but not the train set. In particular, are these items lacking prior data because they are newly introduced in the test timeframe? Are they continuing products that just haven't had any data logged? Or are they continuing products that have zero sales for the entirety of the train timeframe? There's probably not a straight answer to this, but any information will help us make informed assumptions for our models.", "241059": "We have a little bit more than nothing. They are included in the items.csv dataset.", "241060": "I don't think we'll be given any extra info, for two reasons:\n\n 1. It would've probably happened by now :-)\n 2. The 'unseen' items are the real challenge here - roughly 20pct of observations in the test set are combinations of store and item that are not present in the training set, so I think methods of handling those will make the ultimate difference. So far most public kernels I've seen just impute 0s, but I'm fairly certain there must be a better way (I just haven't found it yet)...", "241073": "The less information we have about the relationship between the train and test sets, the more the competition becomes a lottery. As it is, teams will bet on certain assumptions and those who place the correct bet will have the advantage in the final evaluation. I do agree that if the organizers were concerned about this they would have done something, but I really don't see how it's in their best interest to handle it as they have. Anyways, the best strategy for now is to just do trial and error with different strategies and use the public lb as validation, which kinda sucks.", "241125": "Thank you for bringing up this issue. I have so far avoided dealing with this issue by predicting zeros for all new item x store combinations. But I guess inevitably I'll be optimizing them without a proper validation set and overfitting the leaderboard at the later stage.", "241352": "Look at this kernel: [https://www.kaggle.com/ericdfr/new-items-in-stores/][1] \nThere is an unusual number of new items in stores the first day of the test dataset! \nI guess they remove history of sales for some items in some stores. I don't know why they've done that. It sucks if you want to use a time series forecast approach.\n\n [1]: https://www.kaggle.com/ericdfr/new-items-in-stores/", "241526": "Items that appear in the test set but not the train set are new product introductions. This is very common in our business and we must find ways to predict sales of those products to accurately replenish the stores. A possible way to forecast new product sales is using surrogates. That is finding similar products using family and class attributes, finding sales patterns in those similar products, possibly clustering those similar products and choosing the cluster with the best forecasting results.", "241565": "Thanks for your reply. \nIntroduction of new products is a very common case and I only count 60 items that are new in all stores. For me, that's not the problem. \nThe fact that more than 40k item/store combo are new is the real problem and I think is an uncommon case. Don't you remove records for these products? \nSee this kernel for more details: https://www.kaggle.com/ericdfr/new-items-in-stores/", "241704": "We did not manipulate the data in any way, shape or form. It was extracted AS IS from our sales transaction log. At most we curated which stores and items to include in the competition to have a coherent data set that reflects the reality and subtleties of the supermarket business. We will look into the 40k item/store combo to try to find the possible reasons of such a high number and will share with you our findings.", "241716": "The item/store combo is not equal to the new item. They just create the test data for convenience by including all item/store combo although like store A never sells item 1.", "242124": "So for each of new items in test set that are not in train can we expect to have >0 sales?", "243095": "> **Corporación Favorita wrote**\n> \n> > We will look into the 40k item/store combo to try to find the possible reasons of such a high number and will share with you our findings. \n\nHave you found something about these 40k item/store combos?", "243109": "Yes #MarcellFlygare is right!", "246122": "You could probably divide the training set into two parts, training_ (which contains items not in testing_) and testing_ (which contains items not seen in training_), by taking into account the percentages of unseen in the original train and test datasets. Then, you could probably see how your model is performing on the unseen data.", "247165": "> **Eric D wrote**\n> \n> > \n> > **Corporación Favorita wrote**\n> > \n> > > We will look into the 40k item/store combo to try to find the possible reasons of such a high number and will share with you our findings. \n> \n> Have you found something about these 40k item/store combos?\n\nNo news about that?", "247312": "I believe FengLi is right. \n~~ \nThe problem with trying to predict new items is that we do not know which stores the new items are in. That is, if an item starts being sold at **one** store, 810 (54 stores * 15 days) rows are added to the test set. We do not know which stores actually have a chance of selling that item. \n~~ \nI did some analysis on this, and I have not come up with an easy way to tell which store an item will be added to - maybe there is a way, e.g: looking at which similar items seem to be 'phased out'. This of course takes us out of business-applicable predicting. \n~~ \nUnfortunately, I think it is to late for a release of which stores the new items are actually being sold at.", "248482": "Thank you for inspiring me for solving this problem. I am wondering why the result is worse if I use the unit_sales of the same store and class as the new item for prediction. I cannot think of other methods to predict. I would appreciate it very much if there are any inspirations.", "257292": "I'm jumping in here late, regarding the discussion of new products, in order to clarify a bit. As has been mentioned, there are a very small number of new items that were introduced in the test set. That does not explain the apparent high number of new items seen on the first day of the test set. The reason for this is that the training set does not include records for zero sales. The test set, though, includes all store / item combinations, whether or not that item was seen previously in a store.\n\nIf there are any other questions about this, please let me know. Thanks!", "260023": "So, any info on each item sale onset? Especially the missing ones.", "267925": "Can we assume that all new items (or almost all new items) had at least one sale in at least one store during the test period? Or are there a significant number of new product introductions that do not sell a single unit anywhere in the first 16 days (and such products would be included in the test set)? (And, for that matter, were some of the items in the test set already introduced before the test period and could have been in stores, not selling, for days or months before the test period?)" }, "source": "meta" }