![\"enter](\"https://s31.postimg.org/pjj4vv1zf/Kaggle_Grupo_Bimbo2.png\")
Grupo Bimbo and Kaggle would like to welcome you to this forecasting challenge. We're quite excited about this dataset because it's very "real-world", has a large enough volume to cause you trouble, but at the same time the structure of the data is simple.
\n\nPlease feel free to ask questions and we'll try to answer them.
\n\nWe hope you'll enjoy the challenge.
\n\n¡Que te diviertas!
", "messages": [ { "id": 1582950, "postDate": "2021-11-15T12:41:02.857Z", "content": "@wendykan
\nHi Wendy and the Grupo Bimbo Dataset author,
I am a PhD candidate from Guangzhou, China. Would you please let me know, if this dataset can be used for academic research purpose? Our team found this Grupo Bimbo Inventory dataset very suitable for our academic research. With permission, we will explicitly point out the data source when using this data set.
\nI'd appreciated if you could tell me whethe we can use this dataset in our academic research. And we promise to use it for academic research purpose only.
\nMany thanks and best wishes,
\nMichelle
For route ID, it says "Several routes = Sales Depot". What does this mean?
", "rawMarkdown": "For route ID, it says \"Several routes = Sales Depot\". What does this mean?", "votes": 14 }, { "id": 122879, "postDate": "2016-06-08T01:14:40.877Z", "content": "Grupo Bimbo and Kaggle would like to welcome you to this forecasting challenge. We're quite excited about this dataset because it's very "real-world", has a large enough volume to cause you trouble, but at the same time the structure of the data is simple.
\n\nPlease feel free to ask questions and we'll try to answer them.
\n\nWe hope you'll enjoy the challenge.
\n\n¡Que te diviertas!
", "rawMarkdown": "Grupo Bimbo and Kaggle would like to welcome you to this forecasting challenge. We're quite excited about this dataset because it's very \"real-world\", has a large enough volume to cause you trouble, but at the same time the structure of the data is simple. \r\n\r\nPlease feel free to ask questions and we'll try to answer them. \r\n\r\nWe hope you'll enjoy the challenge. \r\n\r\n¡Que te diviertas!", "votes": 13 }, { "id": 123506, "postDate": "2016-06-12T10:28:11.807Z", "content": "For every row in the dataset, submission files should contain two columns: id and Demanda_uni_equi. The id corresponds to the column of that id in the test.csv. The Demanda_uni_equi is the predicted demand units and should be integers. The file should contain a header and have the following format:
\n\nHowever, as i see public scripts are making float type predictions. Is that intended?
", "rawMarkdown": "For every row in the dataset, submission files should contain two columns: id and Demanda_uni_equi. The id corresponds to the column of that id in the test.csv. **The Demanda_uni_equi is the predicted demand units and should be integers**. The file should contain a header and have the following format:\r\n\r\nHowever, as i see public scripts are making float type predictions. Is that intended?", "votes": 8 }, { "id": 123710, "postDate": "2016-06-13T16:51:27.673Z", "content": "@Farhani,
\n\nFor example:
\n\nweek 1, I'm delivering 100 bread X to store A. Record shows: 100 sold. \nweek 2, I'm delivering another 150 bread X to store A, and I'm picking up 25 bread X from the ones that are unsold and expired. Record shows: 150 sold, 25 returned.
\n\nSo overall for week 1, net demand is 100-25=75.
\n\nNote: "sold" means "sold to the vendor", not "sold to customer", maybe that's where you're confused?
", "rawMarkdown": "@Farhani, \r\n\r\nFor example:\r\n\r\nweek 1, I'm delivering 100 bread X to store A. Record shows: 100 sold. \r\nweek 2, I'm delivering another 150 bread X to store A, and I'm picking up 25 bread X from the ones that are unsold and expired. Record shows: 150 sold, 25 returned. \r\n\r\nSo overall for week 1, net demand is 100-25=75. \r\n\r\nNote: \"sold\" means \"sold to the vendor\", not \"sold to customer\", maybe that's where you're confused?", "votes": 5 }, { "id": 122976, "postDate": "2016-06-08T23:24:34.103Z", "content": "Just curious,
\n\nFrom a business standpoint: why would you want to forecast the sales per customer like this competition asks for (vs just per store) to manage your inventory? That makes the forecasting process more difficult without gaining much extra insight to plan your production I would think.
", "rawMarkdown": "Just curious,\r\n\r\nFrom a business standpoint: why would you want to forecast the sales per customer like this competition asks for (vs just per store) to manage your inventory? That makes the forecasting process more difficult without gaining much extra insight to plan your production I would think.\r\n", "votes": 6 }, { "id": 122922, "postDate": "2016-06-08T11:28:44.427Z", "content": "https://www.kaggle.com/c/grupo-bimbo-inventory-demand/details/evaluation
\n\nThere's a small mistake on this page:\nIt says the submission column should be 'Demanda_uni_equi', but instead it is actually 'Demanda_uni_equil'
\n\nThis might trip someone else up like it did to me!
", "rawMarkdown": "https://www.kaggle.com/c/grupo-bimbo-inventory-demand/details/evaluation\r\n\r\nThere's a small mistake on this page:\r\nIt says the submission column should be 'Demanda_uni_equi', but instead it is actually 'Demanda_uni_equil'\r\n\r\nThis might trip someone else up like it did to me!", "votes": 6 }, { "id": 123022, "postDate": "2016-06-09T07:26:05.883Z", "content": "@Ajay_Jadhav it is my understanding that one is the number of products sold (integer) and the other one is the total value of those products.
", "rawMarkdown": "@Ajay_Jadhav it is my understanding that one is the number of products sold (integer) and the other one is the total value of those products.", "votes": 3 }, { "id": 123692, "postDate": "2016-06-13T14:34:05.807Z", "content": "Hi,
\n\nI'm having a difficulty in understanding data. For each tuple (Agencia_ID, Canal_ID, Ruta_SAK, Cliente_ID, Producto_ID), there are some missing weeks in training data (as well as test data). I'm assuming Venta_uni_hoy and Dev_uni_proxima are both 0s i.e. there were no demands in those weeks, for some reason. My question is: Does test data include those 0s records? If it doesn't, that can be a source of leak. Winning models will always predict >= 1 whichever a tuple is and the resulting predictions can be a bit off the mark in practice.
\n\nAlso why test data includes only ~3M records per week while train data does ~10M?
", "rawMarkdown": "Hi,\r\n\r\nI'm having a difficulty in understanding data. For each tuple (Agencia_ID, Canal_ID, Ruta_SAK, Cliente_ID, Producto_ID), there are some missing weeks in training data (as well as test data). I'm assuming Venta_uni_hoy and Dev_uni_proxima are both 0s i.e. there were no demands in those weeks, for some reason. My question is: Does test data include those 0s records? If it doesn't, that can be a source of leak. Winning models will always predict >= 1 whichever a tuple is and the resulting predictions can be a bit off the mark in practice.\r\n\r\nAlso why test data includes only ~3M records per week while train data does ~10M?\r\n", "votes": 4 }, { "id": 129541, "postDate": "2016-07-30T22:02:32.797Z", "content": "I am not sure whether I misunderstand the task or there are some strange data errors: \nDoes the data depicted below mean an error because Venta_uni and Dev_uni are interchanged for Semana 3-7?
\n\n
... or is it plausbile that Producto_ID = 32558 (1) can be used for several weeks before it expires, (2) the client has built a huge stockpile before Semana 3 (3) could not sell it to his customers and (4) is now returning the expired products to us week by week before he orders again 1800 units in Semana 8?
\n\nI am asking because there are 302559 rows in train where \nVenta_uni_hoy==0 & Dev_uni_proxima>0 \nGiven the final row in the data above for Semana =8, I would think that my reasoning of reducing stock was implausible. However, when the data looks like this:
![\"enter](\"https://s31.postimg.org/82l1oausb/Kaggle_Grupo_Bimbo3_Kopie.png\")
how am I supposed to know whether there is a mistake or Cliente_ID =2119 is reducing his stock?
", "rawMarkdown": "I am not sure whether I misunderstand the task or there are some strange data errors: \r\nDoes the data depicted below mean an error because Venta_uni and Dev_uni are interchanged for Semana 3-7? \r\n\r\n![enter image description here][1]\r\n\r\n... or is it plausbile that Producto_ID = 32558 (1) can be used for several weeks before it expires, (2) the client has built a huge stockpile before Semana 3 (3) could not sell it to his customers and (4) is now returning the expired products to us week by week before he orders again 1800 units in Semana 8? \r\n\r\nI am asking because there are 302559 rows in train where \r\n`Venta_uni_hoy==0 & Dev_uni_proxima>0` \r\nGiven the final row in the data above for Semana =8, I would think that my reasoning of reducing stock was implausible. However, when the data looks like this: \r\n\r\n![enter image description here][2]\r\n\r\nhow am I supposed to know whether there is a mistake or Cliente_ID =2119 is reducing his stock? \r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n [1]: https://s31.postimg.org/pjj4vv1zf/Kaggle_Grupo_Bimbo2.png\r\n [2]: https://s31.postimg.org/82l1oausb/Kaggle_Grupo_Bimbo3_Kopie.png", "votes": 1 }, { "id": 128354, "postDate": "2016-07-19T12:30:25.577Z", "content": "@ Kyle7Day\nAbout multiple deliveries of the same product to the same client in the same week. I'll use an example as one possible explanation: \nit could mean that the Cliente ordered e.g. 100 pieces of the item, but BIMBO only had 40 available in the clients preferred depot (usually the closest depot I'd assume). So in order to fully supply the client, they need to do another delivery (from another depot) with the remaining 60 pcs. I don't think it is possible to predict these type of events with the available training data. We'd have to know each depot's stock level for that.\nThe way I approach this is by just converting the 2 deliveries into 1 (so in the example you'd end up with is 1 line of 100 pcs delivered, instead of 2 lines with 40 and 60 each). I use a groupby(client, product, week).sum(). Hope this helps.
", "rawMarkdown": "@ Kyle7Day\r\nAbout multiple deliveries of the same product to the same client in the same week. I'll use an example as one possible explanation: \r\nit could mean that the Cliente ordered e.g. 100 pieces of the item, but BIMBO only had 40 available in the clients preferred depot (usually the closest depot I'd assume). So in order to fully supply the client, they need to do another delivery (from another depot) with the remaining 60 pcs. I don't think it is possible to predict these type of events with the available training data. We'd have to know each depot's stock level for that.\r\nThe way I approach this is by just converting the 2 deliveries into 1 (so in the example you'd end up with is 1 line of 100 pcs delivered, instead of 2 lines with 40 and 60 each). I use a groupby(client, product, week).sum(). Hope this helps.", "votes": 1 }, { "id": 127184, "postDate": "2016-07-13T15:57:11.757Z", "content": "@ihopethiswillfi
\n\nOkay, I can see that, assuming that only one Agencia_ID is used for a given week. If this is correct then there would only be one entry in the test data for each set of Semana, Cliente_ID, and Producto_ID. I just checked it and this holds true.
\n\nThe other thing I can check is to make sure there aren't spots in the training data where a client has ordered from multiple Agencia's for the same week. For the training set I get 74,013,023 unique records based on Semana, Cliente_ID, and Producto_ID. That's 167,441 less than the 74,180,464 records. Meaning that for the same week there are multiple orders by some clients for the same product.
\n\nAny ideas on how to handle these cases? Do we just sum and hope it doesn't end up happening in the rest of the test data?
", "rawMarkdown": "@ihopethiswillfi\r\n\r\nOkay, I can see that, assuming that only one Agencia_ID is used for a given week. If this is correct then there would only be one entry in the test data for each set of Semana, Cliente_ID, and Producto_ID. I just checked it and this holds true. \r\n\r\nThe other thing I can check is to make sure there aren't spots in the training data where a client has ordered from multiple Agencia's for the same week. For the training set I get 74,013,023 unique records based on Semana, Cliente_ID, and Producto_ID. That's 167,441 less than the 74,180,464 records. Meaning that for the same week there are multiple orders by some clients for the same product. \r\n\r\nAny ideas on how to handle these cases? Do we just sum and hope it doesn't end up happening in the rest of the test data?\r\n\r\n", "votes": 1 }, { "id": 125808, "postDate": "2016-07-02T22:26:36.483Z", "content": "@Didier Orel
\n\nLet me explain my understandings. I think all data are pulled from the company's relational database, which means each row should indicate an event (delivery or return) triggered. If neither delivery nor return happens, there simply is no record out there --- neither in train nor in test. As such, you can see the competition is not really about demand forecasting (because there is no easy way to know that in the first place). Due to the constraints of available data, the compromise is to forecast the actual quantities if a delivery or return is expected to happen. I know it is a bit weird.
\n\nI expressed my concern in this regard earlier ( https://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/21670/data-censored-not-at-random ). But I don't see any way we can do better. I think this is an issue that should be properly handled by the sponsors, not us competitors.
", "rawMarkdown": "@Didier Orel\r\n\r\nLet me explain my understandings. I think all data are pulled from the company's relational database, which means each row should indicate an event (delivery or return) triggered. If neither delivery nor return happens, there simply is no record out there --- neither in train nor in test. As such, you can see the competition is not really about demand forecasting (because there is no easy way to know that in the first place). Due to the constraints of available data, the compromise is to forecast the actual quantities if a delivery or return is expected to happen. I know it is a bit weird. \r\n\r\nI expressed my concern in this regard earlier ( https://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/21670/data-censored-not-at-random ). But I don't see any way we can do better. I think this is an issue that should be properly handled by the sponsors, not us competitors. \r\n\r\n\r\n\r\n", "votes": 1 }, { "id": 124718, "postDate": "2016-06-21T16:09:12.627Z", "content": "@Israel,
\n\nIt's max(0,demand).
", "rawMarkdown": "@Israel,\r\n\r\nIt's max(0,demand). ", "votes": 1 }, { "id": 124309, "postDate": "2016-06-17T03:07:13.747Z", "content": "@Silogram, it was randomly downsampled.
\n\n@Shanemeister, I don't think it was a non-existent client since there are client ID's. It's just that the dataset that we're given doesn't have a name for that client.
", "rawMarkdown": "@Silogram, it was randomly downsampled. \r\n\r\n@Shanemeister, I don't think it was a non-existent client since there are client ID's. It's just that the dataset that we're given doesn't have a name for that client. ", "votes": 1 }, { "id": 123995, "postDate": "2016-06-14T22:23:25.710Z", "content": "@Wendy,
\n\nI somewhat misunderstood the problem, thank you.
", "rawMarkdown": "@Wendy,\r\n\r\nI somewhat misunderstood the problem, thank you.", "votes": 1 }, { "id": 123711, "postDate": "2016-06-13T16:54:37.083Z", "content": "@Randomness,
\n\nI think route_SAK is the delivery route, the same route would be the same truck delivers those items in a trip.
\n\nOf course, I'm not the data owner, so Grupo Bimbo's answer will be the most correct.
", "rawMarkdown": "@Randomness, \r\n\r\nI think route_SAK is the delivery route, the same route would be the same truck delivers those items in a trip.\r\n \r\nOf course, I'm not the data owner, so Grupo Bimbo's answer will be the most correct. ", "votes": 1 }, { "id": 123504, "postDate": "2016-06-12T10:14:06.407Z", "content": "Hi,
\n\nI have few questions on nature of data & variables
\n\nData:\n Is the sales data provided is Point-of-sales data/ order placed by a store?
\n\nVariables:\n1) How is "Cliente_ID" defined?\n2) How is Route_Id defined?
\n\nThanks,\nSiva Kumar
", "rawMarkdown": "Hi,\r\n\r\nI have few questions on nature of data & variables\r\n\r\nData:\r\n Is the sales data provided is Point-of-sales data/ order placed by a store?\r\n\r\nVariables:\r\n1) How is \"Cliente_ID\" defined?\r\n2) How is Route_Id defined?\r\n\r\nThanks,\r\nSiva Kumar\r\n", "votes": 1 }, { "id": 123281, "postDate": "2016-06-10T20:09:02.133Z", "content": "@Manuel,
\n\nVery few records that had typos or were obvious outliers were removed. What you are seeing (products missing) might be due to the nature of the data, not from the pre-processing.
", "rawMarkdown": "@Manuel, \r\n\r\nVery few records that had typos or were obvious outliers were removed. What you are seeing (products missing) might be due to the nature of the data, not from the pre-processing. ", "votes": 1 }, { "id": 122947, "postDate": "2016-06-08T18:43:16.813Z", "content": "@anokas, thank you for reporting this! The typo is now fixed.
", "rawMarkdown": "@anokas, thank you for reporting this! The typo is now fixed. ", "votes": 1 }, { "id": 122905, "postDate": "2016-06-08T07:19:17.987Z", "content": "[quote=Wendy Kan;122879]\n¡Que te diviertas!\n[/quote]
\n\nAsi lo hare ;)
", "rawMarkdown": "[quote=Wendy Kan;122879]\r\n¡Que te diviertas!\r\n[/quote]\r\n\r\nAsi lo hare ;)\r\n", "votes": 1 }, { "id": 125344, "postDate": "2016-06-28T19:27:38.030Z", "content": "@bobye, it's split by week.
", "rawMarkdown": "@bobye, it's split by week. ", "votes": 2 }, { "id": 124093, "postDate": "2016-06-15T13:39:29.990Z", "content": "@Wendy, was the downsampling of the test set random? If not, can you let us know how the sampling was performed?
", "rawMarkdown": "@Wendy, was the downsampling of the test set random? If not, can you let us know how the sampling was performed?", "votes": 2 }, { "id": 123955, "postDate": "2016-06-14T16:25:00.587Z", "content": "@Naokazu,
\n\nYour assumption is correct that Venta_uni_hoy and Dev_uni_proxima are both 0. The reason behind it is the demand isn't really tested because the supply=0 (Venta_uni_hoy=0), so there can be no returns. Therefore, the experiment there is missing. In that case, the demand is not 0, it's missing. In other cases where Venta_uni_hoy = Dev_uni_proxima != 0, the demand is 0.
", "rawMarkdown": "@Naokazu, \r\n\r\nYour assumption is correct that Venta_uni_hoy and Dev_uni_proxima are both 0. The reason behind it is the demand isn't really tested because the supply=0 (Venta_uni_hoy=0), so there can be no returns. Therefore, the experiment there is missing. In that case, the demand is not 0, it's missing. In other cases where Venta_uni_hoy = Dev_uni_proxima != 0, the demand is 0. ", "votes": 2 }, { "id": 123880, "postDate": "2016-06-14T09:06:47.493Z", "content": "@Wendy,
\n\nYes, there are 0s in training data, but relatively few records. Also Venta_uni_hoy can be 0 only when Dev_uni_proxima is not. I suppose these are somewhat exceptional cases.
\n\ntable(train$Venta_uni_hoy == 0, train$Dev_uni_proxima == 0)\n\n FALSE TRUE\n FALSE 2241902 71636003\n TRUE 302559 0\nWhat I'm talking about is not this type of 0s. Let's see an example.
\n\nfilter(train, Agencia_ID == 1110, Canal_ID == 7, Ruta_SAK == 3301, Cliente_ID == 15766, Producto_ID == 1216) %>%\nselect(Semana:Venta_uni_hoy)\nSource: local data frame [6 x 7]\n\n Semana Agencia_ID Canal_ID Ruta_SAK Cliente_ID Producto_ID Venta_uni_hoy\n (int) (int) (int) (int) (int) (int) (int)\n1 3 1110 7 3301 15766 1216 4\n2 4 1110 7 3301 15766 1216 2\n3 5 1110 7 3301 15766 1216 3\n4 6 1110 7 3301 15766 1216 1\n5 7 1110 7 3301 15766 1216 2\n6 8 1110 7 3301 15766 1216 5\nIn this example, there no record for week 9 in training data. Actually most of tuples don't have complete 7 weeks of logs.
\n\ntrain %>%\ngroup_by(Agencia_ID, Canal_ID, Ruta_SAK, Cliente_ID, Producto_ID) %>%\nsummarise(n_weeks = n()) %>%\nungroup %>%\ngroup_by(n_weeks) %>%\nsummarise(n = n())\n\nSource: local data frame [7 x 2]\n\n n_weeks n\n (int) (int)\n1 1 9600146\n2 2 5235516\n3 3 3653825\n4 4 2236668\n5 5 2337054\n6 6 818204\n7 7 2515235\nMy assumption is Venta_uni_hoy and Dev_uni_proxima are both 0 for this tuple in week 9 in the above example. If my assumption is correct, I think Grupo Bimbo needs to predict such type of 0s i.e. there will be no demands for the week and therefore test set should include such type of records to make models practically usable. Otherwise the fact that a tuple is included in test set simply indicates there were demands, an information leak. (There can be very few exceptional 0s in test data as in training data, though.)
\n\nIf not, I'd like to know why there are some missing weeks in training data.
", "rawMarkdown": "@Wendy,\r\n\r\nYes, there are 0s in training data, but relatively few records. Also Venta_uni_hoy can be 0 only when Dev_uni_proxima is not. I suppose these are somewhat exceptional cases.\r\n\r\n table(train$Venta_uni_hoy == 0, train$Dev_uni_proxima == 0)\r\n \r\n FALSE TRUE\r\n FALSE 2241902 71636003\r\n TRUE 302559 0\r\n\r\nWhat I'm talking about is not this type of 0s. Let's see an example.\r\n\r\n filter(train, Agencia_ID == 1110, Canal_ID == 7, Ruta_SAK == 3301, Cliente_ID == 15766, Producto_ID == 1216) %>%\r\n select(Semana:Venta_uni_hoy)\r\n Source: local data frame [6 x 7]\r\n \r\n Semana Agencia_ID Canal_ID Ruta_SAK Cliente_ID Producto_ID Venta_uni_hoy\r\n (int) (int) (int) (int) (int) (int) (int)\r\n 1 3 1110 7 3301 15766 1216 4\r\n 2 4 1110 7 3301 15766 1216 2\r\n 3 5 1110 7 3301 15766 1216 3\r\n 4 6 1110 7 3301 15766 1216 1\r\n 5 7 1110 7 3301 15766 1216 2\r\n 6 8 1110 7 3301 15766 1216 5\r\n\r\nIn this example, there no record for week 9 in training data. Actually most of tuples don't have complete 7 weeks of logs.\r\n\r\n train %>%\r\n group_by(Agencia_ID, Canal_ID, Ruta_SAK, Cliente_ID, Producto_ID) %>%\r\n summarise(n_weeks = n()) %>%\r\n ungroup %>%\r\n group_by(n_weeks) %>%\r\n summarise(n = n())\r\n\r\n Source: local data frame [7 x 2]\r\n \r\n n_weeks n\r\n (int) (int)\r\n 1 1 9600146\r\n 2 2 5235516\r\n 3 3 3653825\r\n 4 4 2236668\r\n 5 5 2337054\r\n 6 6 818204\r\n 7 7 2515235\r\n\r\n\r\nMy assumption is Venta_uni_hoy and Dev_uni_proxima are both 0 for this tuple in week 9 in the above example. If my assumption is correct, I think Grupo Bimbo needs to predict such type of 0s i.e. there will be no demands for the week and therefore test set should include such type of records to make models practically usable. Otherwise the fact that a tuple is included in test set simply indicates there were demands, an information leak. (There can be very few exceptional 0s in test data as in training data, though.)\r\n\r\nIf not, I'd like to know why there are some missing weeks in training data.\r\n\r\n", "votes": 2 }, { "id": 123263, "postDate": "2016-06-10T18:23:58.803Z", "content": "@rmtogether,
\n\nIt is time series data and the time is marked in the column of "Semana" which means "week". The rows aren't ordered, so all the data of the same Semana happened in the same week, so you can view them as happening at the same time. Yes, you can think of them as transactions. For example, client A purchased product B for X units in this week.
", "rawMarkdown": "@rmtogether, \r\n\r\nIt is time series data and the time is marked in the column of \"Semana\" which means \"week\". The rows aren't ordered, so all the data of the same Semana happened in the same week, so you can view them as happening at the same time. Yes, you can think of them as transactions. For example, client A purchased product B for X units in this week. ", "votes": 2 }, { "id": 123205, "postDate": "2016-06-10T10:41:27.613Z", "content": "Hi Wendy,
\n\nCould you kindly share more about the business purpose of Sales Channel ID? I would like to understand what does Sales Channel mean.
\n\nThanks\nIan
", "rawMarkdown": "Hi Wendy,\r\n\r\nCould you kindly share more about the business purpose of Sales Channel ID? I would like to understand what does Sales Channel mean.\r\n\r\nThanks\r\nIan", "votes": 2 }, { "id": 123112, "postDate": "2016-06-09T19:31:45.507Z", "content": "Does the store we are gonna predict demand sell products of many brands or only Bimbo's? I didnt get it in the problem description.
", "rawMarkdown": "Does the store we are gonna predict demand sell products of many brands or only Bimbo's? I didnt get it in the problem description.", "votes": 2 }, { "id": 132290, "postDate": "2016-08-24T09:39:28.300Z", "content": "Now I'm getting.
", "rawMarkdown": "Now I'm getting." }, { "id": 127089, "postDate": "2016-07-13T09:22:27.283Z", "content": "Hi Kyle7Day
\n\nThe way I see it, Cliente_ID is the unique identifier for a store. When Cliente_ID is related to more than one Agencia_ID, it just means that this store can be supplied by 2 different BIMBO depot's and BIMBO will choose whichever is most convenient for them at the moment they are planning the delivery. Maybe the store is located 20km from one depot and 22km from the other one, so it gives BIMBO 2 options to choose from.
", "rawMarkdown": "Hi Kyle7Day\r\n\r\nThe way I see it, Cliente_ID is the unique identifier for a store. When Cliente_ID is related to more than one Agencia_ID, it just means that this store can be supplied by 2 different BIMBO depot's and BIMBO will choose whichever is most convenient for them at the moment they are planning the delivery. Maybe the store is located 20km from one depot and 22km from the other one, so it gives BIMBO 2 options to choose from. \r\n" }, { "id": 126919, "postDate": "2016-07-12T21:36:39.057Z", "content": "[quote=Wendy Kan;126509]
\n\n@Bindukumar,
\n\nI'm not the owner of the data, but from my understanding:
\n\nTrucks deliver via a Route that has a few stores (aka vendors, aka client). For example, Route 1: client A, B, C, D, E. Channel is a different way to group the clients as they are related to sales.
\n\nOverall, the route is a spatial group of clients, and channel is a financial group of clients.
\n\n[/quote]
\n\nSo what exactly is a unique identifier for a store from the given fields?? I've been working under the view that all five columns were needed to identify a single product of a single store. With "Agencia_ID", "Canal_ID", "Ruta_SAK", "Cliente_ID" indicating a unique store and then "Producto_ID" giving the products of that store.
\n\nI get that "Agencia_ID" is the sales depot. I now understand that Ruta_SAK or route is a spatial group of clients and channel is a financial group. From past postings and review of the data it seems that there are "Cliente_ID" with multiple "Agencia_ID" meaning Clients can have more than one store/location.
\n\nIs it correct to assume "Agencia_ID" and "Cliente_ID" together identify the location/ store? and therefore we are predicting demand by store and product labeled in the data as "Agencia_ID", "Cliente_ID", and "Producto_ID"?
", "rawMarkdown": "[quote=Wendy Kan;126509]\r\n\r\n@Bindukumar,\r\n\r\nI'm not the owner of the data, but from my understanding:\r\n\r\nTrucks deliver via a Route that has a few stores (aka vendors, aka client). For example, Route 1: client A, B, C, D, E. Channel is a different way to group the clients as they are related to sales. \r\n\r\nOverall, the route is a **spatial** group of clients, and channel is a **financial** group of clients. \r\n\r\n[/quote]\r\n\r\nSo what exactly is a unique identifier for a store from the given fields?? I've been working under the view that all five columns were needed to identify a single product of a single store. With \"Agencia_ID\", \"Canal_ID\", \"Ruta_SAK\", \"Cliente_ID\" indicating a unique store and then \"Producto_ID\" giving the products of that store. \r\n\r\nI get that \"Agencia_ID\" is the sales depot. I now understand that Ruta_SAK or route is a spatial group of clients and channel is a financial group. From past postings and review of the data it seems that there are \"Cliente_ID\" with multiple \"Agencia_ID\" meaning Clients can have more than one store/location.\r\n\r\nIs it correct to assume \"Agencia_ID\" and \"Cliente_ID\" together identify the location/ store? and therefore we are predicting demand by store and product labeled in the data as \"Agencia_ID\", \"Cliente_ID\", and \"Producto_ID\"?" }, { "id": 126592, "postDate": "2016-07-10T11:33:12.567Z", "content": "[quote=Wendy Kan;123708]
\n\n@Naokazu,
\n\nThe 0s are included in both test and train data.
\n\nWe had to down-sample the test data to a smaller set because the evaluation for each submission was taking too long due to the volume.
\n\n@raddar,
\n\nGood catch. It means the answer is always an integer.
\n\n[/quote]
\n\nHow are predictions rounded in evaluation function? nearest integer, floor or ceiling?
", "rawMarkdown": "[quote=Wendy Kan;123708]\r\n\r\n@Naokazu, \r\n\r\nThe 0s are included in both test and train data. \r\n\r\nWe had to down-sample the test data to a smaller set because the evaluation for each submission was taking too long due to the volume. \r\n\r\n@raddar, \r\n\r\nGood catch. It means the answer is always an integer. \r\n\r\n[/quote]\r\n\r\nHow are predictions rounded in evaluation function? nearest integer, floor or ceiling?\r\n" }, { "id": 126587, "postDate": "2016-07-10T07:09:04.320Z", "content": "\n\n\nMy guess would be that MLA is the brand and is subsidiary firm from MTA and MTA is subsidiary from CC however since I could not claim this with certainty could you please provide us with information on this matter.
\n
@PhillDonn,\nThe above assumption is highly unlikely. The last word just before the product id seems to be the brand name. (LR=LARA, BIM=BIMBO etc.). The string before the brand seems to be some kind of discount code specific to set of Agencia_ID/Cliente_ID. For e.g.
\n\n49994,Tostado Int 0pct Grasa Azuc 200g WON 49994
\n\n49996,Tostado Int 0pct Grasa Azuc 200g MTA WON 49996
\n\n49997,Tostado Int 0pct Grasa Azuc 200g TAB WON 49997
\n\nThere is no overlap in Agencia_ID/Cliente_ID who consume these different product id. Possibly the first one is with no discount, while the rest two have two different discount codes.
", "rawMarkdown": "> My guess would be that MLA is the brand and is subsidiary firm from MTA and MTA is subsidiary from CC however since I could not claim this with certainty could you please provide us with information on this matter.\r\n\r\n@PhillDonn,\r\nThe above assumption is highly unlikely. The last word just before the product id seems to be the brand name. (LR=LARA, BIM=BIMBO etc.). The string before the brand seems to be some kind of discount code specific to set of Agencia_ID/Cliente_ID. For e.g.\r\n\r\n49994,Tostado Int 0pct Grasa Azuc 200g WON 49994\r\n\r\n49996,Tostado Int 0pct Grasa Azuc 200g MTA WON 49996\r\n\r\n49997,Tostado Int 0pct Grasa Azuc 200g TAB WON 49997\r\n\r\nThere is no overlap in Agencia_ID/Cliente_ID who consume these different product id. Possibly the first one is with no discount, while the rest two have two different discount codes." }, { "id": 126510, "postDate": "2016-07-09T00:50:49.327Z", "content": "@PhillDonn,
\n\nI have inquired folks at Grupo Bimbo and they'll answer your questions on the forum. Before then, please think of this as part of the challenge.
", "rawMarkdown": "@PhillDonn, \r\n\r\nI have inquired folks at Grupo Bimbo and they'll answer your questions on the forum. Before then, please think of this as part of the challenge. " }, { "id": 126509, "postDate": "2016-07-09T00:46:48.810Z", "content": "@Bindukumar,
\n\nI'm not the owner of the data, but from my understanding:
\n\nTrucks deliver via a Route that has a few stores (aka vendors, aka client). For example, Route 1: client A, B, C, D, E. Channel is a different way to group the clients as they are related to sales.
\n\nOverall, the route is a spatial group of clients, and channel is a financial group of clients.
", "rawMarkdown": "@Bindukumar,\r\n\r\nI'm not the owner of the data, but from my understanding:\r\n\r\nTrucks deliver via a Route that has a few stores (aka vendors, aka client). For example, Route 1: client A, B, C, D, E. Channel is a different way to group the clients as they are related to sales. \r\n\r\nOverall, the route is a **spatial** group of clients, and channel is a **financial** group of clients. " }, { "id": 126451, "postDate": "2016-07-08T14:45:45.933Z", "content": "@NeerajSinghSarwan, just buy some RAM to plug into your machine. RAM is relatively cheap nowadays.
", "rawMarkdown": "@NeerajSinghSarwan, just buy some RAM to plug into your machine. RAM is relatively cheap nowadays." }, { "id": 126438, "postDate": "2016-07-08T12:58:25.643Z", "content": "In this competition, you will forecast the demand of a product for a given week, at a particular store. How is the store defined ??? Every week, there are delivery trucks that deliver products to the vendors. What does vendors mean in the scope of the problem. Cliente_ID — Client ID ------->>>> is this vendor or store ??? i am assuming that a client(Cliente_ID — Client ID) may have stores at different locations (Ruta_SAK — Route ID (Several routes = Sales Depot)). in this context what does---->>>> Agencia_ID — Sales Depot ID---->>>>Canal_ID — Sales Channel ID mean. I might look confused and really.............can someone explain thee relations between given variables.
", "rawMarkdown": "In this competition, you will forecast the demand of a product for a given week, at a particular store. How is the store defined ??? Every week, there are delivery trucks that deliver products to the vendors. What does vendors mean in the scope of the problem. Cliente_ID — Client ID ------->>>> is this vendor or store ??? i am assuming that a client(Cliente_ID — Client ID) may have stores at different locations (Ruta_SAK — Route ID (Several routes = Sales Depot)). in this context what does---->>>> Agencia_ID — Sales Depot ID---->>>>Canal_ID — Sales Channel ID mean. I might look confused and really.............can someone explain thee relations between given variables." }, { "id": 126081, "postDate": "2016-07-06T01:41:46.397Z", "content": "Hello Wendy
\n\nAfter some research through the forum I noticed several topics asking the same question that I would like to ask. Unfortunately none of them could answer entirely, therefore I bring this question here.\nProviding the links to these topics:
\n\n\n\n\n\nI’d like to present this sample from the NombreProducto column in producto_tabla dataset:
\n\nGansito Chocolate 52g CC MTA MLA 49498
\n\n‘MLA’ -> Marinela
\n\n‘CC’ -> Coca Cola
\n\nAlthough with certainty I could say that the last uppercase letters describe the products’ brand, I could not find out what do the last two brand names mean. My guess would be that MLA is the brand and is subsidiary firm from MTA and MTA is subsidiary from CC however since I could not claim this with certainty could you please provide us with information on this matter.
\n\nIn addition providing these samples:
\n\nTostada Tubo 3pq 360g MR 890
\n\nTostada Ond Tubo 30p 360g Prom MR 897
\n\nI’d like to ask if you could explain what do ‘pq’ and ‘Prom’ stand for. My guess would be that Prom stands for Promocion which according to google translate it stands for discount and for ‘pq’ I have no idea. I have tried searching for the product names through the web and while for some parts this act was fruitful for others it was not . So if you could provide us with any information on this matter it would be much appreciated.
\n\nAnd finally the last question that I'd like to ask is about using external data. Let's say that I know that certain types of brands only make certain types of products(sweets, baked goods, etc...). Would using this knowledge for creating features violate the competition's rules?
\n\nThank you
", "rawMarkdown": "Hello Wendy \r\n\r\nAfter some research through the forum I noticed several topics asking the same question that I would like to ask. Unfortunately none of them could answer entirely, therefore I bring this question here.\r\nProviding the links to these topics:\r\n\r\nhttps://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/21649/product-file-information-and-assumptions\r\n\r\nhttps://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/21649/product-file-information-and-assumptions\r\n\r\nI’d like to present this sample from the NombreProducto column in producto_tabla dataset:\r\n\r\nGansito Chocolate 52g CC MTA MLA 49498\r\n\r\n‘MLA’ -> Marinela\r\n\r\n‘CC’ -> Coca Cola\r\n\r\nAlthough with certainty I could say that the last uppercase letters describe the products’ brand, I could not find out what do the last two brand names mean. My guess would be that MLA is the brand and is subsidiary firm from MTA and MTA is subsidiary from CC however since I could not claim this with certainty could you please provide us with information on this matter.\r\n\r\nIn addition providing these samples:\r\n\r\nTostada Tubo 3pq 360g MR 890\r\n\r\nTostada Ond Tubo 30p 360g Prom MR 897\r\n\r\nI’d like to ask if you could explain what do ‘pq’ and ‘Prom’ stand for. My guess would be that Prom stands for Promocion which according to google translate it stands for discount and for ‘pq’ I have no idea. I have tried searching for the product names through the web and while for some parts this act was fruitful for others it was not . So if you could provide us with any information on this matter it would be much appreciated.\r\n\r\nAnd finally the last question that I'd like to ask is about using external data. Let's say that I know that certain types of brands only make certain types of products(sweets, baked goods, etc...). Would using this knowledge for creating features violate the competition's rules?\r\n\r\nThank you\r\n" }, { "id": 125968, "postDate": "2016-07-05T04:51:01.187Z", "content": "The note about the client table says:
\n\n\n\n\nThere are duplicate Cliente_ID's in cliente_tabla, which means one Cliente_ID may have multiple NombreCliente that are very similar. This is due to the NombreCliente being noisy and not standardized in the raw data, so it is up to you to decide how to clean up and use this information.
\n
It is not clear whether the noise arises from the DUPLICATION of Cliente_IDs or from the given SPLITTING of NombreCliente or both. It doesn't make sense to me that the latter would CAUSE the former.
\n\nSo my question is: does Cliente_IDs FAITHFULLY (i.e. 1-1) represent individual clients in the training set?
", "rawMarkdown": "The note about the client table says:\r\n\r\n> There are **duplicate Cliente_ID's in cliente_tabla**, which means **one Cliente_ID may have multiple NombreCliente that are very similar**. This is due to **the NombreCliente being noisy and not standardized in the raw data**, so it is up to you to decide how to clean up and use this information. \r\n\r\nIt is not clear whether the noise arises from the DUPLICATION of Cliente_IDs or from the given SPLITTING of NombreCliente or both. It doesn't make sense to me that the latter would CAUSE the former.\r\n\r\nSo my question is: does Cliente_IDs FAITHFULLY (i.e. 1-1) represent individual clients in the training set?" }, { "id": 125933, "postDate": "2016-07-04T16:44:37.370Z", "content": "@Wendy...
\n\nThe hunger- stealth component of the demand is normally calculated as a percent of something (quantities sold, etc)\nIf possible i would like to know the method followed by BINBO. Does it increase the returns in order to have a correct item stock?
", "rawMarkdown": "@Wendy...\r\n\r\nThe hunger- stealth component of the demand is normally calculated as a percent of something (quantities sold, etc)\r\nIf possible i would like to know the method followed by BINBO. Does it increase the returns in order to have a correct item stock?" }, { "id": 125845, "postDate": "2016-07-03T11:35:41.507Z", "content": "@Jianbo : \nYour answer makes a lot of sense, and I tend to agree with you. Thanks a lot
\n\n@Wendy : would be great if you could confirm
", "rawMarkdown": "@Jianbo : \r\nYour answer makes a lot of sense, and I tend to agree with you. Thanks a lot\r\n\r\n@Wendy : would be great if you could confirm" }, { "id": 125791, "postDate": "2016-07-02T18:10:01.340Z", "content": "One point is still not clear for me.\nIn a previous post, Wendy was explaining why some tuples (client, product) don't have any row for certain week. If I got her right, it's because these clients are not delivered this week for this product. \nSo my question is : for each line of the test.csv, should we
\n\nthanks for your clarifications.
", "rawMarkdown": "One point is still not clear for me.\r\nIn a previous post, Wendy was explaining why some tuples (client, product) don't have any row for certain week. If I got her right, it's because these clients are not delivered this week for this product. \r\nSo my question is : for each line of the test.csv, should we\r\n\r\n 1. predict the exact demand, even if the client is not delivered this week for this product ? or\r\n 2. predict if this client will be delivered or not ? meaning that\r\n - if we predict that the client won't be delivered, we input a 0 for this row\r\n - if we predict that the client will be delivered, we predict the Venta_uni_hoy - Dev_uni_proxima which also can be 0, or more \r\n\r\nthanks for your clarifications.\r\n" }, { "id": 125577, "postDate": "2016-06-30T13:59:28.917Z", "content": "You don't need to load all the data in RAM. There are solutions to work with the data from disk in an intelligent way. See https://cran.r-project.org/web/packages/data.table/vignettes/datatable-intro.pdf
\n\nAlso some algorithms allow you to train a model incrementally like Stochastic Gradient Descent.
", "rawMarkdown": "You don't need to load all the data in RAM. There are solutions to work with the data from disk in an intelligent way. See https://cran.r-project.org/web/packages/data.table/vignettes/datatable-intro.pdf\r\n\r\nAlso some algorithms allow you to train a model incrementally like Stochastic Gradient Descent." }, { "id": 125332, "postDate": "2016-06-28T17:52:56.923Z", "content": "Hi @Wendy,
\n\nCould you tell me how the test data subject to the Public LB and Private LB are split? are they split randomly or are they split based on the two different weeks (week 10 and week 11)?
\n\nJianbo
", "rawMarkdown": "Hi @Wendy,\r\n\r\nCould you tell me how the test data subject to the Public LB and Private LB are split? are they split randomly or are they split based on the two different weeks (week 10 and week 11)?\r\n\r\n Jianbo" }, { "id": 125239, "postDate": "2016-06-28T01:38:13.670Z", "content": "Hi Wendy,
\n\nThanks for that.
\n\nJust to recap, so the goal of this predict the right amount of demands for just bakery products or the entire products (49997 items) for over a million stores?
\n\nSugi
", "rawMarkdown": "Hi Wendy,\r\n\r\nThanks for that.\r\n\r\nJust to recap, so the goal of this predict the right amount of demands for just bakery products or the entire products (49997 items) for over a million stores?\r\n\r\nSugi" }, { "id": 125231, "postDate": "2016-06-27T23:01:31.777Z", "content": "@sugianto, the test data has the other 2 weeks of data.
", "rawMarkdown": "@sugianto, the test data has the other 2 weeks of data. " }, { "id": 125227, "postDate": "2016-06-27T20:39:16.207Z", "content": "Hi Admin,
\n\nI am new here and this is my first time doing the competition as this will be my project for Data Science course I am doing.
\n\nForgive me if this has been noted by someone else or i might have missed information on this, but the training data only contain 7 weeks of data (Semana: 3-9)?
\n\nWasnt it supposed to be 9 weeks of data?
\n\nThanks heaps.
", "rawMarkdown": "Hi Admin,\r\n\r\nI am new here and this is my first time doing the competition as this will be my project for Data Science course I am doing.\r\n\r\nForgive me if this has been noted by someone else or i might have missed information on this, but the training data only contain 7 weeks of data (Semana: 3-9)? \r\n\r\nWasnt it supposed to be 9 weeks of data?\r\n\r\nThanks heaps.\r\n" }, { "id": 125046, "postDate": "2016-06-25T01:23:39.253Z", "content": "@Shahnawaz, I'm sorry that this is the best form we can provide.
", "rawMarkdown": "@Shahnawaz, I'm sorry that this is the best form we can provide. " }, { "id": 125041, "postDate": "2016-06-24T21:50:40.430Z", "content": "Hi Wendy,\nWould you ask Bimbo to provide details (Like brand, name, taste, occasion, associated category etc) about products in more cleaner form because that would really help to give good prediction..
", "rawMarkdown": "Hi Wendy,\r\nWould you ask Bimbo to provide details (Like brand, name, taste, occasion, associated category etc) about products in more cleaner form because that would really help to give good prediction.." }, { "id": 124909, "postDate": "2016-06-23T10:38:53.727Z", "content": "I think it could be the final invoice amount which includes tax.
", "rawMarkdown": "I think it could be the final invoice amount which includes tax." }, { "id": 124898, "postDate": "2016-06-23T07:43:35.393Z", "content": "[quote=MCAG;124897]
\n\nHi all,
\n\ni divided sales_this_week by sales_unit_this_week to get the price of every product. But i noticed that was not unique. Even when i looked just for the entries with sales_unit_this_week = 1 by product_id there are different prices (sales_this_week).\nIs this due to discounts or something like that?
\n\n[/quote]
\n\nI think there is also the factor that different clients may sell at different prices, in addition to discount. And there are cases where sell and return prices differ.
", "rawMarkdown": "[quote=MCAG;124897]\r\n\r\nHi all,\r\n\r\ni divided sales_this_week by sales_unit_this_week to get the price of every product. But i noticed that was not unique. Even when i looked just for the entries with sales_unit_this_week = 1 by product_id there are different prices (sales_this_week).\r\nIs this due to discounts or something like that?\r\n\r\n[/quote]\r\n\r\nI think there is also the factor that different clients may sell at different prices, in addition to discount. And there are cases where sell and return prices differ." }, { "id": 124897, "postDate": "2016-06-23T07:29:00.470Z", "content": "Hi all,
\n\ni divided sales_this_week by sales_unit_this_week to get the price of every product. But i noticed that was not unique. Even when i looked just for the entries with sales_unit_this_week = 1 by product_id there are different prices (sales_this_week).\nIs this due to discounts or something like that?
", "rawMarkdown": "Hi all,\r\n\r\ni divided sales_this_week by sales_unit_this_week to get the price of every product. But i noticed that was not unique. Even when i looked just for the entries with sales_unit_this_week = 1 by product_id there are different prices (sales_this_week).\r\nIs this due to discounts or something like that?" }, { "id": 124691, "postDate": "2016-06-21T10:08:25.443Z", "content": "Question
\n\nThe data description says:
\n\n\n\n\nThe demand for a product in a certain week is defined as the sales\n this week subtracted by the return next week.
\n
However later:
\n\n\n\n\nThe adjusted demand (Demanda_uni_equil) is always >= 0 since demand\n should be either 0 or a positive value. The reason that Venta_uni_hoy\n - Dev_uni_proxima sometimes has negative values is that the returns records sometimes carry over a few weeks.
\n
So, what is the difference between demand as defined in the first paragraph and the field "Demanda_uni_equil"? Is it just max(0, demand)? Or you account somewhat for the carried over returns?
@Umar Najeeb
\n\nThat is probably because excel will take a long time to load the training data (which totals 3.2GB uncompressed).
", "rawMarkdown": "@Umar Najeeb\r\n\r\nThat is probably because excel will take a long time to load the training data (which totals 3.2GB uncompressed)." }, { "id": 124445, "postDate": "2016-06-18T16:53:52.727Z", "content": "can someone explain why the data extraction is not working for me ? when I extract the zip file I dont see anything in excel. thanks
", "rawMarkdown": "can someone explain why the data extraction is not working for me ? when I extract the zip file I dont see anything in excel. thanks\r\n" }, { "id": 124308, "postDate": "2016-06-17T02:19:49.837Z", "content": "Am I missing something here? I understand this is real data, which means there is inevitably bad data here.
\n\nThere are approximately 12% of the test records that reference a cliente_id that does not exist in the cliente_tabla. In the test table, there are 6,999,251 records. Of these, 849,128 records are for clients that do not exist in the client table (foreign key constraint is not enforced).
\n\nIf this is real data, I would say that someone was shipping products to a non-existent client.
\n\nI haven't finished looking at the train data, but I suspect we will see the same. In the train data I might expect this as a test of whether we cleanse the data before analyzing. However, I would like clarification on how we should handle the test data that references a client that does not exist?\nThanks in advance!
", "rawMarkdown": "Am I missing something here? I understand this is real data, which means there is inevitably bad data here. \r\n\r\nThere are approximately 12% of the test records that reference a cliente_id that does not exist in the cliente_tabla. In the test table, there are 6,999,251 records. Of these, 849,128 records are for clients that do not exist in the client table (foreign key constraint is not enforced). \r\n\r\nIf this is real data, I would say that someone was shipping products to a non-existent client. \r\n\r\nI haven't finished looking at the train data, but I suspect we will see the same. In the train data I might expect this as a test of whether we cleanse the data before analyzing. However, I would like clarification on how we should handle the test data that references a client that does not exist?\r\nThanks in advance!\r\n\r\n" }, { "id": 124014, "postDate": "2016-06-15T00:28:45.880Z", "content": "[quote=Anton Kargapolov;124013]
\n\n[quote=Wendy Kan;123710]\nFor example:
\n\nweek 1, I'm delivering 100 bread X to store A. Record shows: 100 sold. \nweek 2, I'm delivering another 150 bread X to store A, and I'm picking up 25 bread X from the ones that are unsold and expired. Record shows: 150 sold, 25 returned.
\n\nSo overall for week 1, net demand is 100-25=75. \n[/quote]\nWhen does Bimbo know about these 25 returned breads, when truck arrived with returns or in advance (before next week starts)?\nIf they know what the return is going to be before sending truck, then it is ok.\nIf they know returns (and hence adjustments for previous week) only when truck comes in with returns it means Bimbo can not use successfully model that uses adjusted demand from previous week - because when they do prediction for next week they won't know adjusted amount.
\n\n[/quote]
\n\nI agree with Anton. If the returns are only known once the truck comes in, then perhaps two separate models would need to be developed. One for Sales and one for Returns.
\n\nThere is so much less Returns data (at Product x Customer level) to do forecasting.
\n\nIan
", "rawMarkdown": "[quote=Anton Kargapolov;124013]\r\n\r\n[quote=Wendy Kan;123710]\r\nFor example:\r\n\r\nweek 1, I'm delivering 100 bread X to store A. Record shows: 100 sold. \r\nweek 2, I'm delivering another 150 bread X to store A, and I'm picking up 25 bread X from the ones that are unsold and expired. Record shows: 150 sold, 25 returned. \r\n\r\nSo overall for week 1, net demand is 100-25=75. \r\n[/quote]\r\nWhen does Bimbo know about these 25 returned breads, when truck arrived with returns or in advance (before next week starts)?\r\nIf they know what the return is going to be before sending truck, then it is ok.\r\nIf they know returns (and hence adjustments for previous week) only when truck comes in with returns it means Bimbo can not use successfully model that uses adjusted demand from previous week - because when they do prediction for next week they won't know adjusted amount. \r\n\r\n[/quote]\r\n\r\nI agree with Anton. If the returns are only known once the truck comes in, then perhaps two separate models would need to be developed. One for Sales and one for Returns.\r\n\r\nThere is so much less Returns data (at Product x Customer level) to do forecasting.\r\n\r\nIan" }, { "id": 124013, "postDate": "2016-06-15T00:24:09.640Z", "content": "[quote=Wendy Kan;123710]\nFor example:
\n\nweek 1, I'm delivering 100 bread X to store A. Record shows: 100 sold. \nweek 2, I'm delivering another 150 bread X to store A, and I'm picking up 25 bread X from the ones that are unsold and expired. Record shows: 150 sold, 25 returned.
\n\nSo overall for week 1, net demand is 100-25=75. \n[/quote]\nWhen does Bimbo know about these 25 returned breads, when truck arrives with returns or in advance (before next week starts)?\nIf they know what the return is going to be before sending truck, then it is ok.\nIf they know returns (and hence adjustments for previous week) only when truck comes in with returns it means Bimbo can not use successfully model that uses adjusted demand from previous week - because when they do prediction for next week they won't know adjusted amount.
", "rawMarkdown": "[quote=Wendy Kan;123710]\r\nFor example:\r\n\r\nweek 1, I'm delivering 100 bread X to store A. Record shows: 100 sold. \r\nweek 2, I'm delivering another 150 bread X to store A, and I'm picking up 25 bread X from the ones that are unsold and expired. Record shows: 150 sold, 25 returned. \r\n\r\nSo overall for week 1, net demand is 100-25=75. \r\n[/quote]\r\nWhen does Bimbo know about these 25 returned breads, when truck arrives with returns or in advance (before next week starts)?\r\nIf they know what the return is going to be before sending truck, then it is ok.\r\nIf they know returns (and hence adjustments for previous week) only when truck comes in with returns it means Bimbo can not use successfully model that uses adjusted demand from previous week - because when they do prediction for next week they won't know adjusted amount. " }, { "id": 123966, "postDate": "2016-06-14T17:52:22.720Z", "content": "@Sangxia, You can submit non-integers.
", "rawMarkdown": "@Sangxia, You can submit non-integers. " }, { "id": 123854, "postDate": "2016-06-14T07:07:11.427Z", "content": "[quote=Wendy Kan;123708]
\n\n@Naokazu,
\n\nThe 0s are included in both test and train data.
\n\nWe had to down-sample the test data to a smaller set because the evaluation for each submission was taking too long due to the volume.
\n\n@raddar,
\n\nGood catch. It means the answer is always an integer.
\n\n[/quote]
\n\nCould you clarify on the second question? I understand that the correct answer is an integer, but is it okay to submit non-integral answers?
", "rawMarkdown": "[quote=Wendy Kan;123708]\r\n\r\n@Naokazu, \r\n\r\nThe 0s are included in both test and train data. \r\n\r\nWe had to down-sample the test data to a smaller set because the evaluation for each submission was taking too long due to the volume. \r\n\r\n@raddar, \r\n\r\nGood catch. It means the answer is always an integer. \r\n\r\n[/quote]\r\n\r\nCould you clarify on the second question? I understand that the correct answer is an integer, but is it okay to submit non-integral answers?\r\n" }, { "id": 123708, "postDate": "2016-06-13T16:47:24.093Z", "content": "@Naokazu,
\n\nThe 0s are included in both test and train data.
\n\nWe had to down-sample the test data to a smaller set because the evaluation for each submission was taking too long due to the volume.
\n\n@raddar,
\n\nGood catch. It means the answer is always an integer.
", "rawMarkdown": "@Naokazu, \r\n\r\nThe 0s are included in both test and train data. \r\n\r\nWe had to down-sample the test data to a smaller set because the evaluation for each submission was taking too long due to the volume. \r\n\r\n@raddar, \r\n\r\nGood catch. It means the answer is always an integer. " }, { "id": 123602, "postDate": "2016-06-13T03:46:38.473Z", "content": "Hello All,
\n\nI want to understand the importance of Canal_ID and Route_SUK in predicting the demand for a product / client. I am not seeing any correlation to demand. May be I am missing some. Is there a business sense to these attributes in relation with determining the demand.
\n\nThanks,\nGilli
", "rawMarkdown": "Hello All,\r\n\r\nI want to understand the importance of Canal_ID and Route_SUK in predicting the demand for a product / client. I am not seeing any correlation to demand. May be I am missing some. Is there a business sense to these attributes in relation with determining the demand.\r\n\r\nThanks,\r\nGilli" }, { "id": 123482, "postDate": "2016-06-12T07:07:34.203Z", "content": "tanx
", "rawMarkdown": "tanx" }, { "id": 123438, "postDate": "2016-06-11T22:39:44Z", "content": "Can some one explain to me what this sentence mean?
\n\n\n\n\n" The demand for a product in a certain week is defined as the sales this week subtracted by the return next week."
\n
Especially that "Returns are the products that are unsold and expired" !
\n\nThanks
", "rawMarkdown": "Can some one explain to me what this sentence mean?\r\n\r\n>*\" The demand for a product in a certain week is defined as the sales this week subtracted by the return next week.\"*\r\n\r\nEspecially that *\"Returns are the products that are unsold and expired\"* !\r\n\r\nThanks\r\n" }, { "id": 123270, "postDate": "2016-06-10T18:46:58.827Z", "content": "Hi,
\n\nAré all the transactions in the training set or some aré deleted ?
\n\nI asume that the relative of some products are complete deleted in training set.
", "rawMarkdown": "Hi,\r\n\r\nAré all the transactions in the training set or some aré deleted ?\r\n\r\nI asume that the relative of some products are complete deleted in training set." }, { "id": 123217, "postDate": "2016-06-10T12:01:02.797Z", "content": "[quote=Payam Bahreyni;123151]
\n\nHi there,
\n\nI believe we have to submit product-level forecasts and given the fact that some products exist only in the test set, how we are going to do this? I mean what product-level information we have about them other that store, route, client, etc. ?
\n\nCan you elaborate a little bit?
\n\n[/quote]
\n\nThere are still ways to provide an answer. For example, you could create a global model - for all the products - and use it to forecast unseen ones from the test set.
", "rawMarkdown": "[quote=Payam Bahreyni;123151]\r\n\r\nHi there,\r\n\r\nI believe we have to submit product-level forecasts and given the fact that some products exist **only in the test set**, how we are going to do this? I mean what product-level information we have about them other that store, route, client, etc. ?\r\n\r\nCan you elaborate a little bit?\r\n\r\n[/quote]\r\n\r\nThere are still ways to provide an answer. For example, you could create a global model - for all the products - and use it to forecast unseen ones from the test set.\r\n" }, { "id": 123157, "postDate": "2016-06-10T01:02:29.077Z", "rawMarkdown": "" }, { "id": 123155, "postDate": "2016-06-10T00:42:54.220Z", "content": "Hi,
\n\nIs this a time serial data? Does each row happen after its previous row? (For example, 2nd row is happened after 1st row?)
\n\nAnother question, For example, Row 2 to row 24 is from [Cliente_ID] =15766. \nCan I think this is a "transaction" of products purchased by [Cliente_ID] =15766?\nand the products within this transaction were purchased (barcode scan) in order.
", "rawMarkdown": "Hi, \r\n\r\nIs this a time serial data? Does each row happen after its previous row? (For example, 2nd row is happened after 1st row?)\r\n\r\nAnother question, For example, Row 2 to row 24 is from [Cliente_ID] =15766. \r\nCan I think this is a \"transaction\" of products purchased by [Cliente_ID] =15766?\r\nand the products within this transaction were purchased (barcode scan) in order." }, { "id": 123151, "postDate": "2016-06-10T00:04:28.200Z", "content": "Hi there,
\n\nI believe we have to submit product-level forecasts and given the fact that some products exist only in the test set, how we are going to do this? I mean what product-level information we have about them other that store, route, client, etc. ?
\n\nCan you elaborate a little bit?
", "rawMarkdown": "Hi there,\r\n\r\nI believe we have to submit product-level forecasts and given the fact that some products exist **only in the test set**, how we are going to do this? I mean what product-level information we have about them other that store, route, client, etc. ?\r\n\r\nCan you elaborate a little bit?" }, { "id": 123145, "postDate": "2016-06-09T23:28:39.043Z", "content": "@Snow Dog, that is correct.
", "rawMarkdown": "@Snow Dog, that is correct. " }, { "id": 123143, "postDate": "2016-06-09T23:27:38.303Z", "content": "Wow you're fast, thanks. One more question, why it says from "Thursday to Wednesday" on the data description? Is it the start/end points of a week? Thanks!
", "rawMarkdown": "Wow you're fast, thanks. One more question, why it says from \"Thursday to Wednesday\" on the data description? Is it the start/end points of a week? Thanks!" }, { "id": 123142, "postDate": "2016-06-09T23:21:43.667Z", "content": "@Snow Dog,
\n\nIt's a week counter, like week 3, week 4, etc. It starts at 3 because that's what is available.
", "rawMarkdown": "@Snow Dog, \r\n\r\nIt's a week counter, like week 3, week 4, etc. It starts at 3 because that's what is available. " }, { "id": 123141, "postDate": "2016-06-09T23:17:46.883Z", "content": "I dont' quite get the 'Semana' variable. Is it a 'week counter' or just the encoded days of week?\nIf the former, why it starts at 3? Thanks.
", "rawMarkdown": "I dont' quite get the 'Semana' variable. Is it a 'week counter' or just the encoded days of week?\r\nIf the former, why it starts at 3? Thanks.\r\n" }, { "id": 123137, "postDate": "2016-06-09T23:05:12.890Z", "content": "Hi Cctruc\nIs only for Bimbo´s brands\nRegards
", "rawMarkdown": "Hi Cctruc\r\nIs only for Bimbo´s brands\r\nRegards" }, { "id": 123023, "postDate": "2016-06-09T07:27:57.867Z", "content": "@anokas: Yes you are right. I was expecting some straight forward relation. Anyways. Thanks
", "rawMarkdown": "@anokas: Yes you are right. I was expecting some straight forward relation. Anyways. Thanks \r\n" }, { "id": 123015, "postDate": "2016-06-09T06:50:06.400Z", "content": "Can you please explain relation between Venta_uni_hoy and Venta_hoy. Its specified that 1 is in integer and another is in peso. But what exactly it is ?
", "rawMarkdown": "Can you please explain relation between Venta_uni_hoy and Venta_hoy. Its specified that 1 is in integer and another is in peso. But what exactly it is ?" }, { "id": 122981, "postDate": "2016-06-09T00:04:41.727Z", "content": "I feel Grupo Bimbo is concerned about Delivery boy taking decision about how much of what to place in a store.
", "rawMarkdown": "I feel Grupo Bimbo is concerned about Delivery boy taking decision about how much of what to place in a store." }, { "id": 122980, "postDate": "2016-06-09T00:03:32.260Z", "content": "I feel the store is concerned about Delivery boy taking decision about how much of what to place in a store.
", "rawMarkdown": "I feel the store is concerned about Delivery boy taking decision about how much of what to place in a store." }, { "id": 122946, "postDate": "2016-06-08T18:28:11.980Z", "content": "Love it!, let´s work on that!
", "rawMarkdown": "Love it!, let´s work on that!" }, { "id": 125574, "postDate": "2016-06-30T13:56:14.513Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1582950, "author_name": "chanaha", "author_url": "", "post_date": "2021-11-15T12:41:02.857000", "content": "@wendykan
\nHi Wendy and the Grupo Bimbo Dataset author,
I am a PhD candidate from Guangzhou, China. Would you please let me know, if this dataset can be used for academic research purpose? Our team found this Grupo Bimbo Inventory dataset very suitable for our academic research. With permission, we will explicitly point out the data source when using this data set.
\nI'd appreciated if you could tell me whethe we can use this dataset in our academic research. And we promise to use it for academic research purpose only.
\nMany thanks and best wishes,
\nMichelle
For route ID, it says "Several routes = Sales Depot". What does this mean?
", "votes": 14, "replies": [] }, { "id": 123506, "author_name": "raddar", "author_url": "", "post_date": "2016-06-12T10:28:11.807000", "content": "For every row in the dataset, submission files should contain two columns: id and Demanda_uni_equi. The id corresponds to the column of that id in the test.csv. The Demanda_uni_equi is the predicted demand units and should be integers. The file should contain a header and have the following format:
\n\nHowever, as i see public scripts are making float type predictions. Is that intended?
", "votes": 8, "replies": [] }, { "id": 123710, "author_name": "Wendy Kan", "author_url": "", "post_date": "2016-06-13T16:51:27.673000", "content": "@Farhani,
\n\nFor example:
\n\nweek 1, I'm delivering 100 bread X to store A. Record shows: 100 sold. \nweek 2, I'm delivering another 150 bread X to store A, and I'm picking up 25 bread X from the ones that are unsold and expired. Record shows: 150 sold, 25 returned.
\n\nSo overall for week 1, net demand is 100-25=75.
\n\nNote: "sold" means "sold to the vendor", not "sold to customer", maybe that's where you're confused?
", "votes": 5, "replies": [] }, { "id": 122976, "author_name": "Peter", "author_url": "", "post_date": "2016-06-08T23:24:34.103000", "content": "Just curious,
\n\nFrom a business standpoint: why would you want to forecast the sales per customer like this competition asks for (vs just per store) to manage your inventory? That makes the forecasting process more difficult without gaining much extra insight to plan your production I would think.
", "votes": 6, "replies": [] }, { "id": 122922, "author_name": "anokas", "author_url": "", "post_date": "2016-06-08T11:28:44.427000", "content": "https://www.kaggle.com/c/grupo-bimbo-inventory-demand/details/evaluation
\n\nThere's a small mistake on this page:\nIt says the submission column should be 'Demanda_uni_equi', but instead it is actually 'Demanda_uni_equil'
\n\nThis might trip someone else up like it did to me!
", "votes": 6, "replies": [] }, { "id": 123022, "author_name": "anokas", "author_url": "", "post_date": "2016-06-09T07:26:05.883000", "content": "@Ajay_Jadhav it is my understanding that one is the number of products sold (integer) and the other one is the total value of those products.
", "votes": 3, "replies": [] }, { "id": 123692, "author_name": "Naokazu Mizuta", "author_url": "", "post_date": "2016-06-13T14:34:05.807000", "content": "Hi,
\n\nI'm having a difficulty in understanding data. For each tuple (Agencia_ID, Canal_ID, Ruta_SAK, Cliente_ID, Producto_ID), there are some missing weeks in training data (as well as test data). I'm assuming Venta_uni_hoy and Dev_uni_proxima are both 0s i.e. there were no demands in those weeks, for some reason. My question is: Does test data include those 0s records? If it doesn't, that can be a source of leak. Winning models will always predict >= 1 whichever a tuple is and the resulting predictions can be a bit off the mark in practice.
\n\nAlso why test data includes only ~3M records per week while train data does ~10M?
", "votes": 4, "replies": [] }, { "id": 129541, "author_name": "EnvyMyFieldsMedal", "author_url": "", "post_date": "2016-07-30T22:02:32.797000", "content": "I am not sure whether I misunderstand the task or there are some strange data errors: \nDoes the data depicted below mean an error because Venta_uni and Dev_uni are interchanged for Semana 3-7?
\n\n
... or is it plausbile that Producto_ID = 32558 (1) can be used for several weeks before it expires, (2) the client has built a huge stockpile before Semana 3 (3) could not sell it to his customers and (4) is now returning the expired products to us week by week before he orders again 1800 units in Semana 8?
\n\nI am asking because there are 302559 rows in train where \nVenta_uni_hoy==0 & Dev_uni_proxima>0 \nGiven the final row in the data above for Semana =8, I would think that my reasoning of reducing stock was implausible. However, when the data looks like this:
how am I supposed to know whether there is a mistake or Cliente_ID =2119 is reducing his stock?
", "votes": 1, "replies": [] }, { "id": 128354, "author_name": "ihopethiswillfi", "author_url": "", "post_date": "2016-07-19T12:30:25.577000", "content": "@ Kyle7Day\nAbout multiple deliveries of the same product to the same client in the same week. I'll use an example as one possible explanation: \nit could mean that the Cliente ordered e.g. 100 pieces of the item, but BIMBO only had 40 available in the clients preferred depot (usually the closest depot I'd assume). So in order to fully supply the client, they need to do another delivery (from another depot) with the remaining 60 pcs. I don't think it is possible to predict these type of events with the available training data. We'd have to know each depot's stock level for that.\nThe way I approach this is by just converting the 2 deliveries into 1 (so in the example you'd end up with is 1 line of 100 pcs delivered, instead of 2 lines with 40 and 60 each). I use a groupby(client, product, week).sum(). Hope this helps.
", "votes": 1, "replies": [] }, { "id": 127184, "author_name": "Kyle7Day", "author_url": "", "post_date": "2016-07-13T15:57:11.757000", "content": "@ihopethiswillfi
\n\nOkay, I can see that, assuming that only one Agencia_ID is used for a given week. If this is correct then there would only be one entry in the test data for each set of Semana, Cliente_ID, and Producto_ID. I just checked it and this holds true.
\n\nThe other thing I can check is to make sure there aren't spots in the training data where a client has ordered from multiple Agencia's for the same week. For the training set I get 74,013,023 unique records based on Semana, Cliente_ID, and Producto_ID. That's 167,441 less than the 74,180,464 records. Meaning that for the same week there are multiple orders by some clients for the same product.
\n\nAny ideas on how to handle these cases? Do we just sum and hope it doesn't end up happening in the rest of the test data?
", "votes": 1, "replies": [] }, { "id": 125808, "author_name": "Jianbo Ye", "author_url": "", "post_date": "2016-07-02T22:26:36.483000", "content": "@Didier Orel
\n\nLet me explain my understandings. I think all data are pulled from the company's relational database, which means each row should indicate an event (delivery or return) triggered. If neither delivery nor return happens, there simply is no record out there --- neither in train nor in test. As such, you can see the competition is not really about demand forecasting (because there is no easy way to know that in the first place). Due to the constraints of available data, the compromise is to forecast the actual quantities if a delivery or return is expected to happen. I know it is a bit weird.
\n\nI expressed my concern in this regard earlier ( https://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/21670/data-censored-not-at-random ). But I don't see any way we can do better. I think this is an issue that should be properly handled by the sponsors, not us competitors.
", "votes": 1, "replies": [] }, { "id": 124718, "author_name": "Wendy Kan", "author_url": "", "post_date": "2016-06-21T16:09:12.627000", "content": "@Israel,
\n\nIt's max(0,demand).
", "votes": 1, "replies": [] }, { "id": 124309, "author_name": "Wendy Kan", "author_url": "", "post_date": "2016-06-17T03:07:13.747000", "content": "@Silogram, it was randomly downsampled.
\n\n@Shanemeister, I don't think it was a non-existent client since there are client ID's. It's just that the dataset that we're given doesn't have a name for that client.
", "votes": 1, "replies": [] }, { "id": 123995, "author_name": "Naokazu Mizuta", "author_url": "", "post_date": "2016-06-14T22:23:25.710000", "content": "@Wendy,
\n\nI somewhat misunderstood the problem, thank you.
", "votes": 1, "replies": [] }, { "id": 123711, "author_name": "Wendy Kan", "author_url": "", "post_date": "2016-06-13T16:54:37.083000", "content": "@Randomness,
\n\nI think route_SAK is the delivery route, the same route would be the same truck delivers those items in a trip.
\n\nOf course, I'm not the data owner, so Grupo Bimbo's answer will be the most correct.
", "votes": 1, "replies": [] }, { "id": 123504, "author_name": "Randomness", "author_url": "", "post_date": "2016-06-12T10:14:06.407000", "content": "Hi,
\n\nI have few questions on nature of data & variables
\n\nData:\n Is the sales data provided is Point-of-sales data/ order placed by a store?
\n\nVariables:\n1) How is "Cliente_ID" defined?\n2) How is Route_Id defined?
\n\nThanks,\nSiva Kumar
", "votes": 1, "replies": [] }, { "id": 123281, "author_name": "Wendy Kan", "author_url": "", "post_date": "2016-06-10T20:09:02.133000", "content": "@Manuel,
\n\nVery few records that had typos or were obvious outliers were removed. What you are seeing (products missing) might be due to the nature of the data, not from the pre-processing.
", "votes": 1, "replies": [] }, { "id": 122947, "author_name": "Wendy Kan", "author_url": "", "post_date": "2016-06-08T18:43:16.813000", "content": "@anokas, thank you for reporting this! The typo is now fixed.
", "votes": 1, "replies": [] }, { "id": 122905, "author_name": "NxGTR", "author_url": "", "post_date": "2016-06-08T07:19:17.987000", "content": "[quote=Wendy Kan;122879]\n¡Que te diviertas!\n[/quote]
\n\nAsi lo hare ;)
", "votes": 1, "replies": [] }, { "id": 125344, "author_name": "Wendy Kan", "author_url": "", "post_date": "2016-06-28T19:27:38.030000", "content": "@bobye, it's split by week.
", "votes": 2, "replies": [] }, { "id": 124093, "author_name": "Silogram", "author_url": "", "post_date": "2016-06-15T13:39:29.990000", "content": "@Wendy, was the downsampling of the test set random? If not, can you let us know how the sampling was performed?
", "votes": 2, "replies": [] }, { "id": 123955, "author_name": "Wendy Kan", "author_url": "", "post_date": "2016-06-14T16:25:00.587000", "content": "@Naokazu,
\n\nYour assumption is correct that Venta_uni_hoy and Dev_uni_proxima are both 0. The reason behind it is the demand isn't really tested because the supply=0 (Venta_uni_hoy=0), so there can be no returns. Therefore, the experiment there is missing. In that case, the demand is not 0, it's missing. In other cases where Venta_uni_hoy = Dev_uni_proxima != 0, the demand is 0.
", "votes": 2, "replies": [] }, { "id": 123880, "author_name": "Naokazu Mizuta", "author_url": "", "post_date": "2016-06-14T09:06:47.493000", "content": "@Wendy,
\n\nYes, there are 0s in training data, but relatively few records. Also Venta_uni_hoy can be 0 only when Dev_uni_proxima is not. I suppose these are somewhat exceptional cases.
\n\ntable(train$Venta_uni_hoy == 0, train$Dev_uni_proxima == 0)\n\n FALSE TRUE\n FALSE 2241902 71636003\n TRUE 302559 0\nWhat I'm talking about is not this type of 0s. Let's see an example.
\n\nfilter(train, Agencia_ID == 1110, Canal_ID == 7, Ruta_SAK == 3301, Cliente_ID == 15766, Producto_ID == 1216) %>%\nselect(Semana:Venta_uni_hoy)\nSource: local data frame [6 x 7]\n\n Semana Agencia_ID Canal_ID Ruta_SAK Cliente_ID Producto_ID Venta_uni_hoy\n (int) (int) (int) (int) (int) (int) (int)\n1 3 1110 7 3301 15766 1216 4\n2 4 1110 7 3301 15766 1216 2\n3 5 1110 7 3301 15766 1216 3\n4 6 1110 7 3301 15766 1216 1\n5 7 1110 7 3301 15766 1216 2\n6 8 1110 7 3301 15766 1216 5\nIn this example, there no record for week 9 in training data. Actually most of tuples don't have complete 7 weeks of logs.
\n\ntrain %>%\ngroup_by(Agencia_ID, Canal_ID, Ruta_SAK, Cliente_ID, Producto_ID) %>%\nsummarise(n_weeks = n()) %>%\nungroup %>%\ngroup_by(n_weeks) %>%\nsummarise(n = n())\n\nSource: local data frame [7 x 2]\n\n n_weeks n\n (int) (int)\n1 1 9600146\n2 2 5235516\n3 3 3653825\n4 4 2236668\n5 5 2337054\n6 6 818204\n7 7 2515235\nMy assumption is Venta_uni_hoy and Dev_uni_proxima are both 0 for this tuple in week 9 in the above example. If my assumption is correct, I think Grupo Bimbo needs to predict such type of 0s i.e. there will be no demands for the week and therefore test set should include such type of records to make models practically usable. Otherwise the fact that a tuple is included in test set simply indicates there were demands, an information leak. (There can be very few exceptional 0s in test data as in training data, though.)
\n\nIf not, I'd like to know why there are some missing weeks in training data.
", "votes": 2, "replies": [] }, { "id": 123263, "author_name": "Wendy Kan", "author_url": "", "post_date": "2016-06-10T18:23:58.803000", "content": "@rmtogether,
\n\nIt is time series data and the time is marked in the column of "Semana" which means "week". The rows aren't ordered, so all the data of the same Semana happened in the same week, so you can view them as happening at the same time. Yes, you can think of them as transactions. For example, client A purchased product B for X units in this week.
", "votes": 2, "replies": [] }, { "id": 123205, "author_name": "Ian Lo", "author_url": "", "post_date": "2016-06-10T10:41:27.613000", "content": "Hi Wendy,
\n\nCould you kindly share more about the business purpose of Sales Channel ID? I would like to understand what does Sales Channel mean.
\n\nThanks\nIan
", "votes": 2, "replies": [] }, { "id": 123112, "author_name": "cctruc", "author_url": "", "post_date": "2016-06-09T19:31:45.507000", "content": "Does the store we are gonna predict demand sell products of many brands or only Bimbo's? I didnt get it in the problem description.
", "votes": 2, "replies": [] }, { "id": 132290, "author_name": "yim", "author_url": "", "post_date": "2016-08-24T09:39:28.300000", "content": "Now I'm getting.
", "votes": 0, "replies": [] }, { "id": 127089, "author_name": "ihopethiswillfi", "author_url": "", "post_date": "2016-07-13T09:22:27.283000", "content": "Hi Kyle7Day
\n\nThe way I see it, Cliente_ID is the unique identifier for a store. When Cliente_ID is related to more than one Agencia_ID, it just means that this store can be supplied by 2 different BIMBO depot's and BIMBO will choose whichever is most convenient for them at the moment they are planning the delivery. Maybe the store is located 20km from one depot and 22km from the other one, so it gives BIMBO 2 options to choose from.
", "votes": 0, "replies": [] }, { "id": 126919, "author_name": "", "author_url": "", "post_date": "2016-07-12T21:36:39.057000", "content": "", "votes": 0, "replies": [] }, { "id": 126592, "author_name": "", "author_url": "", "post_date": "2016-07-10T11:33:12.567000", "content": "", "votes": 0, "replies": [] }, { "id": 126587, "author_name": "", "author_url": "", "post_date": "2016-07-10T07:09:04.320000", "content": "", "votes": 0, "replies": [] }, { "id": 126510, "author_name": "", "author_url": "", "post_date": "2016-07-09T00:50:49.327000", "content": "", "votes": 0, "replies": [] }, { "id": 126509, "author_name": "", "author_url": "", "post_date": "2016-07-09T00:46:48.810000", "content": "", "votes": 0, "replies": [] }, { "id": 126451, "author_name": "", "author_url": "", "post_date": "2016-07-08T14:45:45.933000", "content": "", "votes": 0, "replies": [] }, { "id": 126438, "author_name": "", "author_url": "", "post_date": "2016-07-08T12:58:25.643000", "content": "", "votes": 0, "replies": [] }, { "id": 126081, "author_name": "", "author_url": "", "post_date": "2016-07-06T01:41:46.397000", "content": "", "votes": 0, "replies": [] }, { "id": 125968, "author_name": "", "author_url": "", "post_date": "2016-07-05T04:51:01.187000", "content": "", "votes": 0, "replies": [] }, { "id": 125933, "author_name": "", "author_url": "", "post_date": "2016-07-04T16:44:37.370000", "content": "", "votes": 0, "replies": [] }, { "id": 125845, "author_name": "", "author_url": "", "post_date": "2016-07-03T11:35:41.507000", "content": "", "votes": 0, "replies": [] }, { "id": 125791, "author_name": "", "author_url": "", "post_date": "2016-07-02T18:10:01.340000", "content": "", "votes": 0, "replies": [] }, { "id": 125577, "author_name": "", "author_url": "", "post_date": "2016-06-30T13:59:28.917000", "content": "", "votes": 0, "replies": [] }, { "id": 125332, "author_name": "", "author_url": "", "post_date": "2016-06-28T17:52:56.923000", "content": "", "votes": 0, "replies": [] }, { "id": 125239, "author_name": "", "author_url": "", "post_date": "2016-06-28T01:38:13.670000", "content": "", "votes": 0, "replies": [] }, { "id": 125231, "author_name": "", "author_url": "", "post_date": "2016-06-27T23:01:31.777000", "content": "", "votes": 0, "replies": [] }, { "id": 125227, "author_name": "", "author_url": "", "post_date": "2016-06-27T20:39:16.207000", "content": "", "votes": 0, "replies": [] }, { "id": 125046, "author_name": "", "author_url": "", "post_date": "2016-06-25T01:23:39.253000", "content": "", "votes": 0, "replies": [] }, { "id": 125041, "author_name": "", "author_url": "", "post_date": "2016-06-24T21:50:40.430000", "content": "", "votes": 0, "replies": [] }, { "id": 124909, "author_name": "", "author_url": "", "post_date": "2016-06-23T10:38:53.727000", "content": "", "votes": 0, "replies": [] }, { "id": 124898, "author_name": "", "author_url": "", "post_date": "2016-06-23T07:43:35.393000", "content": "", "votes": 0, "replies": [] }, { "id": 124897, "author_name": "", "author_url": "", "post_date": "2016-06-23T07:29:00.470000", "content": "", "votes": 0, "replies": [] }, { "id": 124691, "author_name": "", "author_url": "", "post_date": "2016-06-21T10:08:25.443000", "content": "", "votes": 0, "replies": [] }, { "id": 124482, "author_name": "", "author_url": "", "post_date": "2016-06-19T05:29:27.743000", "content": "", "votes": 0, "replies": [] }, { "id": 124445, "author_name": "", "author_url": "", "post_date": "2016-06-18T16:53:52.727000", "content": "", "votes": 0, "replies": [] }, { "id": 124308, "author_name": "", "author_url": "", "post_date": "2016-06-17T02:19:49.837000", "content": "", "votes": 0, "replies": [] }, { "id": 124014, "author_name": "", "author_url": "", "post_date": "2016-06-15T00:28:45.880000", "content": "", "votes": 0, "replies": [] }, { "id": 124013, "author_name": "", "author_url": "", "post_date": "2016-06-15T00:24:09.640000", "content": "", "votes": 0, "replies": [] }, { "id": 123966, "author_name": "", "author_url": "", "post_date": "2016-06-14T17:52:22.720000", "content": "", "votes": 0, "replies": [] }, { "id": 123854, "author_name": "", "author_url": "", "post_date": "2016-06-14T07:07:11.427000", "content": "", "votes": 0, "replies": [] }, { "id": 123708, "author_name": "", "author_url": "", "post_date": "2016-06-13T16:47:24.093000", "content": "", "votes": 0, "replies": [] }, { "id": 123602, "author_name": "", "author_url": "", "post_date": "2016-06-13T03:46:38.473000", "content": "", "votes": 0, "replies": [] }, { "id": 123482, "author_name": "", "author_url": "", "post_date": "2016-06-12T07:07:34.203000", "content": "", "votes": 0, "replies": [] }, { "id": 123438, "author_name": "", "author_url": "", "post_date": "2016-06-11T22:39:44", "content": "", "votes": 0, "replies": [] }, { "id": 123270, "author_name": "", "author_url": "", "post_date": "2016-06-10T18:46:58.827000", "content": "", "votes": 0, "replies": [] }, { "id": 123217, "author_name": "", "author_url": "", "post_date": "2016-06-10T12:01:02.797000", "content": "", "votes": 0, "replies": [] }, { "id": 123157, "author_name": "", "author_url": "", "post_date": "2016-06-10T01:02:29.077000", "content": "", "votes": 0, "replies": [] }, { "id": 123155, "author_name": "", "author_url": "", "post_date": "2016-06-10T00:42:54.220000", "content": "", "votes": 0, "replies": [] }, { "id": 123151, "author_name": "", "author_url": "", "post_date": "2016-06-10T00:04:28.200000", "content": "", "votes": 0, "replies": [] }, { "id": 123145, "author_name": "", "author_url": "", "post_date": "2016-06-09T23:28:39.043000", "content": "", "votes": 0, "replies": [] }, { "id": 123143, "author_name": "", "author_url": "", "post_date": "2016-06-09T23:27:38.303000", "content": "", "votes": 0, "replies": [] }, { "id": 123142, "author_name": "", "author_url": "", "post_date": "2016-06-09T23:21:43.667000", "content": "", "votes": 0, "replies": [] }, { "id": 123141, "author_name": "", "author_url": "", "post_date": "2016-06-09T23:17:46.883000", "content": "", "votes": 0, "replies": [] }, { "id": 123137, "author_name": "", "author_url": "", "post_date": "2016-06-09T23:05:12.890000", "content": "", "votes": 0, "replies": [] }, { "id": 123023, "author_name": "", "author_url": "", "post_date": "2016-06-09T07:27:57.867000", "content": "", "votes": 0, "replies": [] }, { "id": 123015, "author_name": "", "author_url": "", "post_date": "2016-06-09T06:50:06.400000", "content": "", "votes": 0, "replies": [] }, { "id": 122981, "author_name": "", "author_url": "", "post_date": "2016-06-09T00:04:41.727000", "content": "", "votes": 0, "replies": [] }, { "id": 122980, "author_name": "", "author_url": "", "post_date": "2016-06-09T00:03:32.260000", "content": "", "votes": 0, "replies": [] }, { "id": 122946, "author_name": "", "author_url": "", "post_date": "2016-06-08T18:28:11.980000", "content": "", "votes": 0, "replies": [] }, { "id": 125574, "author_name": "", "author_url": "", "post_date": "2016-06-30T13:56:14.513000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1582950": "@wendykan \nHi Wendy and the Grupo Bimbo Dataset author,\n\nI am a PhD candidate from Guangzhou, China. Would you please let me know, if this dataset can be used for academic research purpose? Our team found this Grupo Bimbo Inventory dataset very suitable for our academic research. With permission, we will explicitly point out the data source when using this data set. \n\nI'd appreciated if you could tell me whethe we can use this dataset in our academic research. And we promise to use it for academic research purpose only.\n\nMany thanks and best wishes,\nMichelle", "123075": "For route ID, it says \"Several routes = Sales Depot\". What does this mean?", "122879": "Grupo Bimbo and Kaggle would like to welcome you to this forecasting challenge. We're quite excited about this dataset because it's very \"real-world\", has a large enough volume to cause you trouble, but at the same time the structure of the data is simple. \r\n\r\nPlease feel free to ask questions and we'll try to answer them. \r\n\r\nWe hope you'll enjoy the challenge. \r\n\r\n¡Que te diviertas!", "123506": "For every row in the dataset, submission files should contain two columns: id and Demanda_uni_equi. The id corresponds to the column of that id in the test.csv. **The Demanda_uni_equi is the predicted demand units and should be integers**. The file should contain a header and have the following format:\r\n\r\nHowever, as i see public scripts are making float type predictions. Is that intended?", "123710": "@Farhani, \r\n\r\nFor example:\r\n\r\nweek 1, I'm delivering 100 bread X to store A. Record shows: 100 sold. \r\nweek 2, I'm delivering another 150 bread X to store A, and I'm picking up 25 bread X from the ones that are unsold and expired. Record shows: 150 sold, 25 returned. \r\n\r\nSo overall for week 1, net demand is 100-25=75. \r\n\r\nNote: \"sold\" means \"sold to the vendor\", not \"sold to customer\", maybe that's where you're confused?", "122976": "Just curious,\r\n\r\nFrom a business standpoint: why would you want to forecast the sales per customer like this competition asks for (vs just per store) to manage your inventory? That makes the forecasting process more difficult without gaining much extra insight to plan your production I would think.\r\n", "122922": "https://www.kaggle.com/c/grupo-bimbo-inventory-demand/details/evaluation\r\n\r\nThere's a small mistake on this page:\r\nIt says the submission column should be 'Demanda_uni_equi', but instead it is actually 'Demanda_uni_equil'\r\n\r\nThis might trip someone else up like it did to me!", "123022": "@Ajay_Jadhav it is my understanding that one is the number of products sold (integer) and the other one is the total value of those products.", "123692": "Hi,\r\n\r\nI'm having a difficulty in understanding data. For each tuple (Agencia_ID, Canal_ID, Ruta_SAK, Cliente_ID, Producto_ID), there are some missing weeks in training data (as well as test data). I'm assuming Venta_uni_hoy and Dev_uni_proxima are both 0s i.e. there were no demands in those weeks, for some reason. My question is: Does test data include those 0s records? If it doesn't, that can be a source of leak. Winning models will always predict >= 1 whichever a tuple is and the resulting predictions can be a bit off the mark in practice.\r\n\r\nAlso why test data includes only ~3M records per week while train data does ~10M?\r\n", "129541": "I am not sure whether I misunderstand the task or there are some strange data errors: \r\nDoes the data depicted below mean an error because Venta_uni and Dev_uni are interchanged for Semana 3-7? \r\n\r\n![enter image description here][1]\r\n\r\n... or is it plausbile that Producto_ID = 32558 (1) can be used for several weeks before it expires, (2) the client has built a huge stockpile before Semana 3 (3) could not sell it to his customers and (4) is now returning the expired products to us week by week before he orders again 1800 units in Semana 8? \r\n\r\nI am asking because there are 302559 rows in train where \r\n`Venta_uni_hoy==0 & Dev_uni_proxima>0` \r\nGiven the final row in the data above for Semana =8, I would think that my reasoning of reducing stock was implausible. However, when the data looks like this: \r\n\r\n![enter image description here][2]\r\n\r\nhow am I supposed to know whether there is a mistake or Cliente_ID =2119 is reducing his stock? \r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n [1]: https://s31.postimg.org/pjj4vv1zf/Kaggle_Grupo_Bimbo2.png\r\n [2]: https://s31.postimg.org/82l1oausb/Kaggle_Grupo_Bimbo3_Kopie.png", "128354": "@ Kyle7Day\r\nAbout multiple deliveries of the same product to the same client in the same week. I'll use an example as one possible explanation: \r\nit could mean that the Cliente ordered e.g. 100 pieces of the item, but BIMBO only had 40 available in the clients preferred depot (usually the closest depot I'd assume). So in order to fully supply the client, they need to do another delivery (from another depot) with the remaining 60 pcs. I don't think it is possible to predict these type of events with the available training data. We'd have to know each depot's stock level for that.\r\nThe way I approach this is by just converting the 2 deliveries into 1 (so in the example you'd end up with is 1 line of 100 pcs delivered, instead of 2 lines with 40 and 60 each). I use a groupby(client, product, week).sum(). Hope this helps.", "127184": "@ihopethiswillfi\r\n\r\nOkay, I can see that, assuming that only one Agencia_ID is used for a given week. If this is correct then there would only be one entry in the test data for each set of Semana, Cliente_ID, and Producto_ID. I just checked it and this holds true. \r\n\r\nThe other thing I can check is to make sure there aren't spots in the training data where a client has ordered from multiple Agencia's for the same week. For the training set I get 74,013,023 unique records based on Semana, Cliente_ID, and Producto_ID. That's 167,441 less than the 74,180,464 records. Meaning that for the same week there are multiple orders by some clients for the same product. \r\n\r\nAny ideas on how to handle these cases? Do we just sum and hope it doesn't end up happening in the rest of the test data?\r\n\r\n", "125808": "@Didier Orel\r\n\r\nLet me explain my understandings. I think all data are pulled from the company's relational database, which means each row should indicate an event (delivery or return) triggered. If neither delivery nor return happens, there simply is no record out there --- neither in train nor in test. As such, you can see the competition is not really about demand forecasting (because there is no easy way to know that in the first place). Due to the constraints of available data, the compromise is to forecast the actual quantities if a delivery or return is expected to happen. I know it is a bit weird. \r\n\r\nI expressed my concern in this regard earlier ( https://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/21670/data-censored-not-at-random ). But I don't see any way we can do better. I think this is an issue that should be properly handled by the sponsors, not us competitors. \r\n\r\n\r\n\r\n", "124718": "@Israel,\r\n\r\nIt's max(0,demand). ", "124309": "@Silogram, it was randomly downsampled. \r\n\r\n@Shanemeister, I don't think it was a non-existent client since there are client ID's. It's just that the dataset that we're given doesn't have a name for that client. ", "123995": "@Wendy,\r\n\r\nI somewhat misunderstood the problem, thank you.", "123711": "@Randomness, \r\n\r\nI think route_SAK is the delivery route, the same route would be the same truck delivers those items in a trip.\r\n \r\nOf course, I'm not the data owner, so Grupo Bimbo's answer will be the most correct. ", "123504": "Hi,\r\n\r\nI have few questions on nature of data & variables\r\n\r\nData:\r\n Is the sales data provided is Point-of-sales data/ order placed by a store?\r\n\r\nVariables:\r\n1) How is \"Cliente_ID\" defined?\r\n2) How is Route_Id defined?\r\n\r\nThanks,\r\nSiva Kumar\r\n", "123281": "@Manuel, \r\n\r\nVery few records that had typos or were obvious outliers were removed. What you are seeing (products missing) might be due to the nature of the data, not from the pre-processing. ", "122947": "@anokas, thank you for reporting this! The typo is now fixed. ", "122905": "[quote=Wendy Kan;122879]\r\n¡Que te diviertas!\r\n[/quote]\r\n\r\nAsi lo hare ;)\r\n", "125344": "@bobye, it's split by week. ", "124093": "@Wendy, was the downsampling of the test set random? If not, can you let us know how the sampling was performed?", "123955": "@Naokazu, \r\n\r\nYour assumption is correct that Venta_uni_hoy and Dev_uni_proxima are both 0. The reason behind it is the demand isn't really tested because the supply=0 (Venta_uni_hoy=0), so there can be no returns. Therefore, the experiment there is missing. In that case, the demand is not 0, it's missing. In other cases where Venta_uni_hoy = Dev_uni_proxima != 0, the demand is 0. ", "123880": "@Wendy,\r\n\r\nYes, there are 0s in training data, but relatively few records. Also Venta_uni_hoy can be 0 only when Dev_uni_proxima is not. I suppose these are somewhat exceptional cases.\r\n\r\n table(train$Venta_uni_hoy == 0, train$Dev_uni_proxima == 0)\r\n \r\n FALSE TRUE\r\n FALSE 2241902 71636003\r\n TRUE 302559 0\r\n\r\nWhat I'm talking about is not this type of 0s. Let's see an example.\r\n\r\n filter(train, Agencia_ID == 1110, Canal_ID == 7, Ruta_SAK == 3301, Cliente_ID == 15766, Producto_ID == 1216) %>%\r\n select(Semana:Venta_uni_hoy)\r\n Source: local data frame [6 x 7]\r\n \r\n Semana Agencia_ID Canal_ID Ruta_SAK Cliente_ID Producto_ID Venta_uni_hoy\r\n (int) (int) (int) (int) (int) (int) (int)\r\n 1 3 1110 7 3301 15766 1216 4\r\n 2 4 1110 7 3301 15766 1216 2\r\n 3 5 1110 7 3301 15766 1216 3\r\n 4 6 1110 7 3301 15766 1216 1\r\n 5 7 1110 7 3301 15766 1216 2\r\n 6 8 1110 7 3301 15766 1216 5\r\n\r\nIn this example, there no record for week 9 in training data. Actually most of tuples don't have complete 7 weeks of logs.\r\n\r\n train %>%\r\n group_by(Agencia_ID, Canal_ID, Ruta_SAK, Cliente_ID, Producto_ID) %>%\r\n summarise(n_weeks = n()) %>%\r\n ungroup %>%\r\n group_by(n_weeks) %>%\r\n summarise(n = n())\r\n\r\n Source: local data frame [7 x 2]\r\n \r\n n_weeks n\r\n (int) (int)\r\n 1 1 9600146\r\n 2 2 5235516\r\n 3 3 3653825\r\n 4 4 2236668\r\n 5 5 2337054\r\n 6 6 818204\r\n 7 7 2515235\r\n\r\n\r\nMy assumption is Venta_uni_hoy and Dev_uni_proxima are both 0 for this tuple in week 9 in the above example. If my assumption is correct, I think Grupo Bimbo needs to predict such type of 0s i.e. there will be no demands for the week and therefore test set should include such type of records to make models practically usable. Otherwise the fact that a tuple is included in test set simply indicates there were demands, an information leak. (There can be very few exceptional 0s in test data as in training data, though.)\r\n\r\nIf not, I'd like to know why there are some missing weeks in training data.\r\n\r\n", "123263": "@rmtogether, \r\n\r\nIt is time series data and the time is marked in the column of \"Semana\" which means \"week\". The rows aren't ordered, so all the data of the same Semana happened in the same week, so you can view them as happening at the same time. Yes, you can think of them as transactions. For example, client A purchased product B for X units in this week. ", "123205": "Hi Wendy,\r\n\r\nCould you kindly share more about the business purpose of Sales Channel ID? I would like to understand what does Sales Channel mean.\r\n\r\nThanks\r\nIan", "123112": "Does the store we are gonna predict demand sell products of many brands or only Bimbo's? I didnt get it in the problem description.", "132290": "Now I'm getting.", "127089": "Hi Kyle7Day\r\n\r\nThe way I see it, Cliente_ID is the unique identifier for a store. When Cliente_ID is related to more than one Agencia_ID, it just means that this store can be supplied by 2 different BIMBO depot's and BIMBO will choose whichever is most convenient for them at the moment they are planning the delivery. Maybe the store is located 20km from one depot and 22km from the other one, so it gives BIMBO 2 options to choose from. \r\n", "126919": "[quote=Wendy Kan;126509]\r\n\r\n@Bindukumar,\r\n\r\nI'm not the owner of the data, but from my understanding:\r\n\r\nTrucks deliver via a Route that has a few stores (aka vendors, aka client). For example, Route 1: client A, B, C, D, E. Channel is a different way to group the clients as they are related to sales. \r\n\r\nOverall, the route is a **spatial** group of clients, and channel is a **financial** group of clients. \r\n\r\n[/quote]\r\n\r\nSo what exactly is a unique identifier for a store from the given fields?? I've been working under the view that all five columns were needed to identify a single product of a single store. With \"Agencia_ID\", \"Canal_ID\", \"Ruta_SAK\", \"Cliente_ID\" indicating a unique store and then \"Producto_ID\" giving the products of that store. \r\n\r\nI get that \"Agencia_ID\" is the sales depot. I now understand that Ruta_SAK or route is a spatial group of clients and channel is a financial group. From past postings and review of the data it seems that there are \"Cliente_ID\" with multiple \"Agencia_ID\" meaning Clients can have more than one store/location.\r\n\r\nIs it correct to assume \"Agencia_ID\" and \"Cliente_ID\" together identify the location/ store? and therefore we are predicting demand by store and product labeled in the data as \"Agencia_ID\", \"Cliente_ID\", and \"Producto_ID\"?", "126592": "[quote=Wendy Kan;123708]\r\n\r\n@Naokazu, \r\n\r\nThe 0s are included in both test and train data. \r\n\r\nWe had to down-sample the test data to a smaller set because the evaluation for each submission was taking too long due to the volume. \r\n\r\n@raddar, \r\n\r\nGood catch. It means the answer is always an integer. \r\n\r\n[/quote]\r\n\r\nHow are predictions rounded in evaluation function? nearest integer, floor or ceiling?\r\n", "126587": "> My guess would be that MLA is the brand and is subsidiary firm from MTA and MTA is subsidiary from CC however since I could not claim this with certainty could you please provide us with information on this matter.\r\n\r\n@PhillDonn,\r\nThe above assumption is highly unlikely. The last word just before the product id seems to be the brand name. (LR=LARA, BIM=BIMBO etc.). The string before the brand seems to be some kind of discount code specific to set of Agencia_ID/Cliente_ID. For e.g.\r\n\r\n49994,Tostado Int 0pct Grasa Azuc 200g WON 49994\r\n\r\n49996,Tostado Int 0pct Grasa Azuc 200g MTA WON 49996\r\n\r\n49997,Tostado Int 0pct Grasa Azuc 200g TAB WON 49997\r\n\r\nThere is no overlap in Agencia_ID/Cliente_ID who consume these different product id. Possibly the first one is with no discount, while the rest two have two different discount codes.", "126510": "@PhillDonn, \r\n\r\nI have inquired folks at Grupo Bimbo and they'll answer your questions on the forum. Before then, please think of this as part of the challenge. ", "126509": "@Bindukumar,\r\n\r\nI'm not the owner of the data, but from my understanding:\r\n\r\nTrucks deliver via a Route that has a few stores (aka vendors, aka client). For example, Route 1: client A, B, C, D, E. Channel is a different way to group the clients as they are related to sales. \r\n\r\nOverall, the route is a **spatial** group of clients, and channel is a **financial** group of clients. ", "126451": "@NeerajSinghSarwan, just buy some RAM to plug into your machine. RAM is relatively cheap nowadays.", "126438": "In this competition, you will forecast the demand of a product for a given week, at a particular store. How is the store defined ??? Every week, there are delivery trucks that deliver products to the vendors. What does vendors mean in the scope of the problem. Cliente_ID — Client ID ------->>>> is this vendor or store ??? i am assuming that a client(Cliente_ID — Client ID) may have stores at different locations (Ruta_SAK — Route ID (Several routes = Sales Depot)). in this context what does---->>>> Agencia_ID — Sales Depot ID---->>>>Canal_ID — Sales Channel ID mean. I might look confused and really.............can someone explain thee relations between given variables.", "126081": "Hello Wendy \r\n\r\nAfter some research through the forum I noticed several topics asking the same question that I would like to ask. Unfortunately none of them could answer entirely, therefore I bring this question here.\r\nProviding the links to these topics:\r\n\r\nhttps://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/21649/product-file-information-and-assumptions\r\n\r\nhttps://www.kaggle.com/c/grupo-bimbo-inventory-demand/forums/t/21649/product-file-information-and-assumptions\r\n\r\nI’d like to present this sample from the NombreProducto column in producto_tabla dataset:\r\n\r\nGansito Chocolate 52g CC MTA MLA 49498\r\n\r\n‘MLA’ -> Marinela\r\n\r\n‘CC’ -> Coca Cola\r\n\r\nAlthough with certainty I could say that the last uppercase letters describe the products’ brand, I could not find out what do the last two brand names mean. My guess would be that MLA is the brand and is subsidiary firm from MTA and MTA is subsidiary from CC however since I could not claim this with certainty could you please provide us with information on this matter.\r\n\r\nIn addition providing these samples:\r\n\r\nTostada Tubo 3pq 360g MR 890\r\n\r\nTostada Ond Tubo 30p 360g Prom MR 897\r\n\r\nI’d like to ask if you could explain what do ‘pq’ and ‘Prom’ stand for. My guess would be that Prom stands for Promocion which according to google translate it stands for discount and for ‘pq’ I have no idea. I have tried searching for the product names through the web and while for some parts this act was fruitful for others it was not . So if you could provide us with any information on this matter it would be much appreciated.\r\n\r\nAnd finally the last question that I'd like to ask is about using external data. Let's say that I know that certain types of brands only make certain types of products(sweets, baked goods, etc...). Would using this knowledge for creating features violate the competition's rules?\r\n\r\nThank you\r\n", "125968": "The note about the client table says:\r\n\r\n> There are **duplicate Cliente_ID's in cliente_tabla**, which means **one Cliente_ID may have multiple NombreCliente that are very similar**. This is due to **the NombreCliente being noisy and not standardized in the raw data**, so it is up to you to decide how to clean up and use this information. \r\n\r\nIt is not clear whether the noise arises from the DUPLICATION of Cliente_IDs or from the given SPLITTING of NombreCliente or both. It doesn't make sense to me that the latter would CAUSE the former.\r\n\r\nSo my question is: does Cliente_IDs FAITHFULLY (i.e. 1-1) represent individual clients in the training set?", "125933": "@Wendy...\r\n\r\nThe hunger- stealth component of the demand is normally calculated as a percent of something (quantities sold, etc)\r\nIf possible i would like to know the method followed by BINBO. Does it increase the returns in order to have a correct item stock?", "125845": "@Jianbo : \r\nYour answer makes a lot of sense, and I tend to agree with you. Thanks a lot\r\n\r\n@Wendy : would be great if you could confirm", "125791": "One point is still not clear for me.\r\nIn a previous post, Wendy was explaining why some tuples (client, product) don't have any row for certain week. If I got her right, it's because these clients are not delivered this week for this product. \r\nSo my question is : for each line of the test.csv, should we\r\n\r\n 1. predict the exact demand, even if the client is not delivered this week for this product ? or\r\n 2. predict if this client will be delivered or not ? meaning that\r\n - if we predict that the client won't be delivered, we input a 0 for this row\r\n - if we predict that the client will be delivered, we predict the Venta_uni_hoy - Dev_uni_proxima which also can be 0, or more \r\n\r\nthanks for your clarifications.\r\n", "125577": "You don't need to load all the data in RAM. There are solutions to work with the data from disk in an intelligent way. See https://cran.r-project.org/web/packages/data.table/vignettes/datatable-intro.pdf\r\n\r\nAlso some algorithms allow you to train a model incrementally like Stochastic Gradient Descent.", "125332": "Hi @Wendy,\r\n\r\nCould you tell me how the test data subject to the Public LB and Private LB are split? are they split randomly or are they split based on the two different weeks (week 10 and week 11)?\r\n\r\n Jianbo", "125239": "Hi Wendy,\r\n\r\nThanks for that.\r\n\r\nJust to recap, so the goal of this predict the right amount of demands for just bakery products or the entire products (49997 items) for over a million stores?\r\n\r\nSugi", "125231": "@sugianto, the test data has the other 2 weeks of data. ", "125227": "Hi Admin,\r\n\r\nI am new here and this is my first time doing the competition as this will be my project for Data Science course I am doing.\r\n\r\nForgive me if this has been noted by someone else or i might have missed information on this, but the training data only contain 7 weeks of data (Semana: 3-9)? \r\n\r\nWasnt it supposed to be 9 weeks of data?\r\n\r\nThanks heaps.\r\n", "125046": "@Shahnawaz, I'm sorry that this is the best form we can provide. ", "125041": "Hi Wendy,\r\nWould you ask Bimbo to provide details (Like brand, name, taste, occasion, associated category etc) about products in more cleaner form because that would really help to give good prediction..", "124909": "I think it could be the final invoice amount which includes tax.", "124898": "[quote=MCAG;124897]\r\n\r\nHi all,\r\n\r\ni divided sales_this_week by sales_unit_this_week to get the price of every product. But i noticed that was not unique. Even when i looked just for the entries with sales_unit_this_week = 1 by product_id there are different prices (sales_this_week).\r\nIs this due to discounts or something like that?\r\n\r\n[/quote]\r\n\r\nI think there is also the factor that different clients may sell at different prices, in addition to discount. And there are cases where sell and return prices differ.", "124897": "Hi all,\r\n\r\ni divided sales_this_week by sales_unit_this_week to get the price of every product. But i noticed that was not unique. Even when i looked just for the entries with sales_unit_this_week = 1 by product_id there are different prices (sales_this_week).\r\nIs this due to discounts or something like that?", "124691": "**Question**\r\n\r\nThe data description says:\r\n\r\n> The demand for a product in a certain week is defined as the sales\r\n> this week subtracted by the return next week.\r\n\r\nHowever later:\r\n\r\n> The adjusted demand (Demanda_uni_equil) is always >= 0 since demand\r\n> should be either 0 or a positive value. The reason that Venta_uni_hoy\r\n> - Dev_uni_proxima sometimes has negative values is that the returns records sometimes carry over a few weeks.\r\n\r\nSo, what is the difference between demand as defined in the first paragraph and the field \"Demanda_uni_equil\"? Is it just `max(0, demand)`? Or you account somewhat for the carried over returns?", "124482": "@Umar Najeeb\r\n\r\nThat is probably because excel will take a long time to load the training data (which totals 3.2GB uncompressed).", "124445": "can someone explain why the data extraction is not working for me ? when I extract the zip file I dont see anything in excel. thanks\r\n", "124308": "Am I missing something here? I understand this is real data, which means there is inevitably bad data here. \r\n\r\nThere are approximately 12% of the test records that reference a cliente_id that does not exist in the cliente_tabla. In the test table, there are 6,999,251 records. Of these, 849,128 records are for clients that do not exist in the client table (foreign key constraint is not enforced). \r\n\r\nIf this is real data, I would say that someone was shipping products to a non-existent client. \r\n\r\nI haven't finished looking at the train data, but I suspect we will see the same. In the train data I might expect this as a test of whether we cleanse the data before analyzing. However, I would like clarification on how we should handle the test data that references a client that does not exist?\r\nThanks in advance!\r\n\r\n", "124014": "[quote=Anton Kargapolov;124013]\r\n\r\n[quote=Wendy Kan;123710]\r\nFor example:\r\n\r\nweek 1, I'm delivering 100 bread X to store A. Record shows: 100 sold. \r\nweek 2, I'm delivering another 150 bread X to store A, and I'm picking up 25 bread X from the ones that are unsold and expired. Record shows: 150 sold, 25 returned. \r\n\r\nSo overall for week 1, net demand is 100-25=75. \r\n[/quote]\r\nWhen does Bimbo know about these 25 returned breads, when truck arrived with returns or in advance (before next week starts)?\r\nIf they know what the return is going to be before sending truck, then it is ok.\r\nIf they know returns (and hence adjustments for previous week) only when truck comes in with returns it means Bimbo can not use successfully model that uses adjusted demand from previous week - because when they do prediction for next week they won't know adjusted amount. \r\n\r\n[/quote]\r\n\r\nI agree with Anton. If the returns are only known once the truck comes in, then perhaps two separate models would need to be developed. One for Sales and one for Returns.\r\n\r\nThere is so much less Returns data (at Product x Customer level) to do forecasting.\r\n\r\nIan", "124013": "[quote=Wendy Kan;123710]\r\nFor example:\r\n\r\nweek 1, I'm delivering 100 bread X to store A. Record shows: 100 sold. \r\nweek 2, I'm delivering another 150 bread X to store A, and I'm picking up 25 bread X from the ones that are unsold and expired. Record shows: 150 sold, 25 returned. \r\n\r\nSo overall for week 1, net demand is 100-25=75. \r\n[/quote]\r\nWhen does Bimbo know about these 25 returned breads, when truck arrives with returns or in advance (before next week starts)?\r\nIf they know what the return is going to be before sending truck, then it is ok.\r\nIf they know returns (and hence adjustments for previous week) only when truck comes in with returns it means Bimbo can not use successfully model that uses adjusted demand from previous week - because when they do prediction for next week they won't know adjusted amount. ", "123966": "@Sangxia, You can submit non-integers. ", "123854": "[quote=Wendy Kan;123708]\r\n\r\n@Naokazu, \r\n\r\nThe 0s are included in both test and train data. \r\n\r\nWe had to down-sample the test data to a smaller set because the evaluation for each submission was taking too long due to the volume. \r\n\r\n@raddar, \r\n\r\nGood catch. It means the answer is always an integer. \r\n\r\n[/quote]\r\n\r\nCould you clarify on the second question? I understand that the correct answer is an integer, but is it okay to submit non-integral answers?\r\n", "123708": "@Naokazu, \r\n\r\nThe 0s are included in both test and train data. \r\n\r\nWe had to down-sample the test data to a smaller set because the evaluation for each submission was taking too long due to the volume. \r\n\r\n@raddar, \r\n\r\nGood catch. It means the answer is always an integer. ", "123602": "Hello All,\r\n\r\nI want to understand the importance of Canal_ID and Route_SUK in predicting the demand for a product / client. I am not seeing any correlation to demand. May be I am missing some. Is there a business sense to these attributes in relation with determining the demand.\r\n\r\nThanks,\r\nGilli", "123482": "tanx", "123438": "Can some one explain to me what this sentence mean?\r\n\r\n>*\" The demand for a product in a certain week is defined as the sales this week subtracted by the return next week.\"*\r\n\r\nEspecially that *\"Returns are the products that are unsold and expired\"* !\r\n\r\nThanks\r\n", "123270": "Hi,\r\n\r\nAré all the transactions in the training set or some aré deleted ?\r\n\r\nI asume that the relative of some products are complete deleted in training set.", "123217": "[quote=Payam Bahreyni;123151]\r\n\r\nHi there,\r\n\r\nI believe we have to submit product-level forecasts and given the fact that some products exist **only in the test set**, how we are going to do this? I mean what product-level information we have about them other that store, route, client, etc. ?\r\n\r\nCan you elaborate a little bit?\r\n\r\n[/quote]\r\n\r\nThere are still ways to provide an answer. For example, you could create a global model - for all the products - and use it to forecast unseen ones from the test set.\r\n", "123157": "", "123155": "Hi, \r\n\r\nIs this a time serial data? Does each row happen after its previous row? (For example, 2nd row is happened after 1st row?)\r\n\r\nAnother question, For example, Row 2 to row 24 is from [Cliente_ID] =15766. \r\nCan I think this is a \"transaction\" of products purchased by [Cliente_ID] =15766?\r\nand the products within this transaction were purchased (barcode scan) in order.", "123151": "Hi there,\r\n\r\nI believe we have to submit product-level forecasts and given the fact that some products exist **only in the test set**, how we are going to do this? I mean what product-level information we have about them other that store, route, client, etc. ?\r\n\r\nCan you elaborate a little bit?", "123145": "@Snow Dog, that is correct. ", "123143": "Wow you're fast, thanks. One more question, why it says from \"Thursday to Wednesday\" on the data description? Is it the start/end points of a week? Thanks!", "123142": "@Snow Dog, \r\n\r\nIt's a week counter, like week 3, week 4, etc. It starts at 3 because that's what is available. ", "123141": "I dont' quite get the 'Semana' variable. Is it a 'week counter' or just the encoded days of week?\r\nIf the former, why it starts at 3? Thanks.\r\n", "123137": "Hi Cctruc\r\nIs only for Bimbo´s brands\r\nRegards", "123023": "@anokas: Yes you are right. I was expecting some straight forward relation. Anyways. Thanks \r\n", "123015": "Can you please explain relation between Venta_uni_hoy and Venta_hoy. Its specified that 1 is in integer and another is in peso. But what exactly it is ?", "122981": "I feel Grupo Bimbo is concerned about Delivery boy taking decision about how much of what to place in a store.", "122980": "I feel the store is concerned about Delivery boy taking decision about how much of what to place in a store.", "122946": "Love it!, let´s work on that!", "125574": "" } }