This whole thing seems a bit strange to me. I might have misunderstood something so please enlighten me and tell me where I go wrong.
\n\nFirst of all, the fact that Grupo Bimbo has to refund store owners for unsold products is a bit surprising. Is it really so that Grupo Bimbo gets to choose the quantity of products to deliver but then has to refund if they over-deliver?
\n\nAssuming that I got the first part correct, I really have to ask what are we actually modeling here? I mean the task is to create a model "to accurately forecast inventory demand". The demand for a product is "defined as the sales this week subtracted by the return next week".
\n\nBut, in our training data, "sales this week" is actually an educated guess done by a random Grupo Bimbo employee. Right? And if there are no returns next week, this guess is also accepted as the correct "demand". But what if the real demand was actually higher than that and the consumers indeed were staring at empty shelves? The training data does not tell us anything about this.
\n\nAssume that you still manage to build a model that perfectly predicts the actual demand and therefore maximises Grupo Bimpo profits. That model now gets penalized in the cases where the guess done by the Grupo Bimbo employee was lower than our perfect prediction. So to build a winning model for this competition you instead have to predict what Grupo Bimbo employees predicted in the past.
", "messages": [ { "id": "123409", "postDate": "06/11/2016 18:21:00", "content": "This whole thing seems a bit strange to me. I might have misunderstood something so please enlighten me and tell me where I go wrong.
\n\nFirst of all, the fact that Grupo Bimbo has to refund store owners for unsold products is a bit surprising. Is it really so that Grupo Bimbo gets to choose the quantity of products to deliver but then has to refund if they over-deliver?
\n\nAssuming that I got the first part correct, I really have to ask what are we actually modeling here? I mean the task is to create a model "to accurately forecast inventory demand". The demand for a product is "defined as the sales this week subtracted by the return next week".
\n\nBut, in our training data, "sales this week" is actually an educated guess done by a random Grupo Bimbo employee. Right? And if there are no returns next week, this guess is also accepted as the correct "demand". But what if the real demand was actually higher than that and the consumers indeed were staring at empty shelves? The training data does not tell us anything about this.
\n\nAssume that you still manage to build a model that perfectly predicts the actual demand and therefore maximises Grupo Bimpo profits. That model now gets penalized in the cases where the guess done by the Grupo Bimbo employee was lower than our perfect prediction. So to build a winning model for this competition you instead have to predict what Grupo Bimbo employees predicted in the past.
", "rawMarkdown": "This whole thing seems a bit strange to me. I might have misunderstood something so please enlighten me and tell me where I go wrong.\r\n\r\nFirst of all, the fact that Grupo Bimbo has to refund store owners for unsold products is a bit surprising. Is it really so that Grupo Bimbo gets to choose the quantity of products to deliver but then has to refund if they over-deliver?\r\n\r\nAssuming that I got the first part correct, I really have to ask what are we actually modeling here? I mean the task is to create a model \"to accurately forecast inventory demand\". The demand for a product is \"defined as the sales this week subtracted by the return next week\". \r\n\r\nBut, in our training data, \"sales this week\" is actually an educated guess done by a random Grupo Bimbo employee. Right? And if there are no returns next week, this guess is also accepted as the correct \"demand\". But what if the real demand was actually higher than that and the consumers indeed were staring at empty shelves? The training data does not tell us anything about this.\r\n\r\nAssume that you still manage to build a model that perfectly predicts the actual demand and therefore maximises Grupo Bimpo profits. That model now gets penalized in the cases where the guess done by the Grupo Bimbo employee was lower than our perfect prediction. So to build a winning model for this competition you instead have to predict what Grupo Bimbo employees predicted in the past.", "votes": null }, { "id": "123413", "postDate": "06/11/2016 18:58:51", "content": "Why do you think that "sales this week" is a guess? Can't it be observed data?
", "rawMarkdown": "Why do you think that \"sales this week\" is a guess? Can't it be observed data?", "votes": null }, { "id": "123416", "postDate": "06/11/2016 19:25:32", "content": "It probably is observed but let's consider an example:
\n\n\n\n\nVenta_uni_hoy: 8 Venta_hoy: 46.08 Dev_uni_proxima: 0 Dev_proxima: 0\n Demanda_uni_equil: 8
\n
I'm currently interpreting this so that they delivered 8 units and sold everything as there were no returns next week. But what if the actual demand was 10 units and 2 consumers were left without their burritos? So why is it correct to assume that the demand was 8?
", "rawMarkdown": "It probably *is* observed but let's consider an example:\r\n\r\n> Venta_uni_hoy: 8 Venta_hoy: 46.08 Dev_uni_proxima: 0 Dev_proxima: 0\r\n> Demanda_uni_equil: 8\r\n\r\nI'm currently interpreting this so that they delivered 8 units and sold everything as there were no returns next week. But what if the actual demand was 10 units and 2 consumers were left without their burritos? So why is it correct to assume that the demand was 8?", "votes": null }, { "id": "123422", "postDate": "06/11/2016 19:52:40", "content": "I see your point now. I think that when dev_proxima equals 0, the real demand will be greater than venta_uni_hoy most of the time. Maybe "demanda" is not the best word to describe the target variable. It's better to move this questions to the welcome page anyway.
", "rawMarkdown": "I see your point now. I think that when dev_proxima equals 0, the real demand will be greater than venta_uni_hoy most of the time. Maybe \"demanda\" is not the best word to describe the target variable. It's better to move this questions to the welcome page anyway.", "votes": null }, { "id": "123470", "postDate": "06/12/2016 04:06:06", "content": "It doesn't work like that at all. Bimbo has two things, big bread factories around the country, and a distribution channel. Their distribution channel is based on trucks that visit retailers every week. Retailers can be big supermarkets (think of walmart), or very small corner stores managed by a single person.
\n\nWhen the sales guy makes their visit, the retailer specifies how much of each product they want to buy based on their own experience, budget, etc. On the next visit , the sales guy retrieves the unsold merchandise that has gone beyond the display date. They would do this to avoid having retailers selling old bread and giving the brand a bad reputation. Thus, it is not unthinkable that they might refund a part of it just to keep the retailers happy even if they screw up a bit (also, Bimbo has some stores that sell the bread that has not gone bad just beyond its display date).
\n\nSo wrapping up ,
\n\nAt least thats the way I see it.
", "rawMarkdown": "It doesn't work like that at all. Bimbo has two things, big bread factories around the country, and a distribution channel. Their distribution channel is based on trucks that visit retailers every week. Retailers can be big supermarkets (think of walmart), or very small corner stores managed by a single person. \r\n\r\nWhen the sales guy makes their visit, the retailer specifies how much of each product they want to buy based on their own experience, budget, etc. On the next visit , the sales guy retrieves the unsold merchandise that has gone beyond the display date. They would do this to avoid having retailers selling old bread and giving the brand a bad reputation. Thus, it is not unthinkable that they might refund a part of it just to keep the retailers happy even if they screw up a bit (also, Bimbo has some stores that sell the bread that has not gone bad just beyond its display date).\r\n\r\nSo wrapping up , \r\n\r\n- \"sales this week\" are the products the retailer decided to buy from the Bimbo sales guy\r\n- You're a bit correct on the demand part. It might be that customers were looking at empty shelves, but you should expect that the retailer notices that and makes a bigger purchase next week.\r\n- You're correct about penalizing a perfect demand forecast model because the retailers often miss to perfectly estimate the demand. However there is no such thing as perfect demand forecasting in the wild at this scale. \r\n\r\nAt least thats the way I see it.", "votes": null }, { "id": "123496", "postDate": "06/12/2016 09:07:54", "content": "Also note that the performance metric used in this problem (RMSLE) penalises under-predicted estimates more (i.e. is better to go above the optimum that below). Which means that a good solution will generally avoid 'having the clients staring at empty shelves' (at the cost of having more returns).
\n\nhttps://www.kaggle.com/wiki/RootMeanSquaredLogarithmicError
", "rawMarkdown": "Also note that the performance metric used in this problem (RMSLE) penalises under-predicted estimates more (i.e. is better to go above the optimum that below). Which means that a good solution will generally avoid 'having the clients staring at empty shelves' (at the cost of having more returns).\r\n\r\nhttps://www.kaggle.com/wiki/RootMeanSquaredLogarithmicError", "votes": null }, { "id": "123515", "postDate": "06/12/2016 12:24:47", "content": "In fact in Demand planning, the model looks like the Newsboy Vendor Problem. But then, we should have a cost of shortage which is the margin that Bimbo looses when it misses a sale and a cost of excess which is what Bimbo refunds to the dealer in case of excess. The newsboy model then proposes quantities that maximize the overall profit. Bimbo pays back dealers to avoid then selling expired products but mainly to avoid having some dealers buying insufficient quantities because they don't want to have access. \nGiven that we don't have these inputs (cost of excess, cost of shortage), we can't go this way and we have to stick on the given data.
", "rawMarkdown": "In fact in Demand planning, the model looks like the Newsboy Vendor Problem. But then, we should have a cost of shortage which is the margin that Bimbo looses when it misses a sale and a cost of excess which is what Bimbo refunds to the dealer in case of excess. The newsboy model then proposes quantities that maximize the overall profit. Bimbo pays back dealers to avoid then selling expired products but mainly to avoid having some dealers buying insufficient quantities because they don't want to have access. \r\nGiven that we don't have these inputs (cost of excess, cost of shortage), we can't go this way and we have to stick on the given data.", "votes": null }, { "id": "123528", "postDate": "06/12/2016 14:41:10", "content": "@Ibrahima, thanks, I learned something new today! (i have zero operations background haha). The second reason to pay back to the retailer is spot on. Would it be possible/accurate to set the cost of excess/shortage as a multiplier of the unitary price?
", "rawMarkdown": "Ibrahima, thanks, I learned something new today! (i have zero operations background haha). The second reason to pay back to the retailer is spot on. Would it be possible/accurate to set the cost of excess/shortage as a multiplier of the unitary price?", "votes": null }, { "id": "123530", "postDate": "06/12/2016 14:51:34", "content": "Thanks everyone for your comments! This was very helpful.
", "rawMarkdown": "Thanks everyone for your comments! This was very helpful.", "votes": null }, { "id": "124303", "postDate": "06/17/2016 01:06:02", "content": "I agree with the poster. This prediction problem is not well defined for the stated business goal. Instead we are creating an algorithm that will perform no better than the human heuristics, with the same errors. I'll illustrate my point with examples and data.
\n\nSorry for the lengthy post, but I need it to prove my point.
\n\nWhy the Objective Function For Prediction Is a Poor Choice
\n\nBelow a small part of the training data is shown (some columns removed, with column names translated to English). It is all the data for client 1367838 with product 30552, and client 1367838 with product 30572.
\n\nWeek Delivery ID,Client ID,Product ID,Units Sold,Units Returned,Adjusted Demand Units\n3,1367838,30552,1276,0,1276\n4,1367838,30552,616,0,616\n5,1367838,30552,616,0,616\n6,1367838,30552,616,0,616\n7,1367838,30552,616,0,616\n3,1367838,30572,1248,0,1248\n4,1367838,30572,611,0,611\n5,1367838,30572,564,0,564\n6,1367838,30572,564,0,564\n7,1367838,30572,94,0,94\nThe key observations:
\n\nEvery entry has zero returned items. In other words, according to the data, at no point has the demand been fully met by the supply.
As a consequence, the adjusted demand that we are meant to predict is by definition equal to the sales guess made by the human.
If we look at product 30572 in week 3, we see that even with 1248 units, not all demand is met. Still, the units sold in the subsequent weeks are 611, 564 or even as low as 94. From the data alone, the predicted correct business decision would be to at least sell 1248, maybe more, in future weeks. However, given that the adjusted demand is here not a function of the market demand, rather a reflection of the human guess, the best value to predict would probably be 564 or some similar measure of central tendency. In short, the objective function of the problem given to us would be minimized if I reproduced the human heuristics, no matter how wrong.
In cases where there are units returned, the adjusted demand is indeed a reflection of what the true demand of the product at that client was. The business optimization then aligns with the objective function given to us. So is data with zero returns the rule of the exception?
\n\nOf all client-product pairs I find that 23783091 or about 92% have no week with non-zero returns. In other words, these are data points where the value we are asked to predict is equal to the human guess, with its shortcomings. (It should be noted that many client-product pairs are sampled only one or two weeks). With this skew, the objective function is likely dominated by points that do not align with business goals.
\n\nTo conclude: a prediction method performs well if it reproduces the human biases and mistakes. It does not propose actions that improve upon the problem of meeting demand without excess supply.
\n\nSome Ideas On How To Recast the Problem To Align With Business Goals
\n\nThe easiest way to make the prediction meaningful to business would be to limit predictions to client-product pairs that are properly sampled in the training data (a few non-zero returns in the data). It would obviously simplify the problem, but arguably too much.
\n\nSince we do not have the luxury to run experiments based on forecasts by a prediction algorithm, we must cast the prediction such that human shortcomings do not skew the objective function from the business goal. One way would be to make this a classification problem, like this:
\n\nGiven a client ID, a product ID and the actual Units Sold, we try to predict:
\n\nThis will have some business relevance. A method that performs well on this classification will educate the person in the warehouse if his/her suggestion for number of unit of product X to sell to client Y is likely too low, too high or about right.
\n\nA problem cast this way, with the appropriate objective function, is not skewed by the human failures. What we instead are doing is to create a prediction of said failures. Can a machine figure out if the proposed sales number is off the mark or not?
\n\nKaggle, how about changing the objective function?
", "rawMarkdown": "I agree with the poster. This prediction problem is not well defined for the stated business goal. Instead we are creating an algorithm that will perform no better than the human heuristics, with the same errors. I'll illustrate my point with examples and data. \r\n\r\n*Sorry for the lengthy post, but I need it to prove my point.*\r\n\r\n**Why the Objective Function For Prediction Is a Poor Choice**\r\n\r\nBelow a small part of the training data is shown (some columns removed, with column names translated to English). It is all the data for client 1367838 with product 30552, and client 1367838 with product 30572. \r\n\r\n Week Delivery ID,Client ID,Product ID,Units Sold,Units Returned,Adjusted Demand Units\r\n 3,1367838,30552,1276,0,1276\r\n 4,1367838,30552,616,0,616\r\n 5,1367838,30552,616,0,616\r\n 6,1367838,30552,616,0,616\r\n 7,1367838,30552,616,0,616\r\n 3,1367838,30572,1248,0,1248\r\n 4,1367838,30572,611,0,611\r\n 5,1367838,30572,564,0,564\r\n 6,1367838,30572,564,0,564\r\n 7,1367838,30572,94,0,94\r\n\r\nThe key observations:\r\n\r\n* Every entry has zero returned items. In other words, according to the data, at no point has the demand been fully met by the supply.\r\n\r\n* As a consequence, the adjusted demand *that we are meant to predict* is by definition equal to the sales guess made by the human.\r\n\r\n* If we look at product 30572 in week 3, we see that even with 1248 units, not all demand is met. Still, the units sold in the subsequent weeks are 611, 564 or even as low as 94. From the data alone, the predicted correct *business decision* would be to at least sell 1248, maybe more, in future weeks. However, given that the adjusted demand is here *not* a function of the market demand, rather a reflection of the human guess, the best value to predict would probably be 564 or some similar measure of central tendency. In short, the objective function of the problem given to us would be minimized if I reproduced the human heuristics, no matter how wrong.\r\n\r\nIn cases where there are units returned, the adjusted demand is indeed a reflection of what the true demand of the product at that client was. The business optimization then aligns with the objective function given to us. So is data with zero returns the rule of the exception?\r\n\r\nOf all client-product pairs I find that 23783091 or about 92% have *no week with non-zero returns.* In other words, these are data points where the value we are asked to predict is equal to the human guess, with its shortcomings. (It should be noted that many client-product pairs are sampled only one or two weeks). With this skew, the objective function is likely dominated by points that do not align with business goals.\r\n\r\nTo conclude: a prediction method performs well if it reproduces the human biases and mistakes. It does not propose actions that improve upon the problem of meeting demand without excess supply.\r\n\r\n**Some Ideas On How To Recast the Problem To Align With Business Goals**\r\n\r\nThe easiest way to make the prediction meaningful to business would be to limit predictions to client-product pairs that are properly sampled in the training data (a few non-zero returns in the data). It would obviously simplify the problem, but arguably too much.\r\n\r\nSince we do not have the luxury to run experiments based on forecasts by a prediction algorithm, we must cast the prediction such that human shortcomings do not skew the objective function from the business goal. One way would be to make this a classification problem, like this:\r\n\r\nGiven a client ID, a product ID *and* the actual Units Sold, we try to predict:\r\n\r\n* Will this fail to meet demand (Units Returned = 0), or\r\n* Will it overshoot demand by some threshold percentage (Units Returned / Units Sold > P), or\r\n* Will it be right between 0 returned and the threshold percentage (0 < Units Returned / Units Sold < P).\r\n\r\nThis will have some business relevance. A method that performs well on this classification will educate the person in the warehouse if his/her suggestion for number of unit of product X to sell to client Y is likely too low, too high or about right. \r\n\r\nA problem cast this way, with the appropriate objective function, is not skewed by the human failures. What we instead are doing is to create a prediction of said failures. Can a machine figure out if the proposed sales number is off the mark or not?\r\n\r\nKaggle, how about changing the objective function?", "votes": null }, { "id": "124311", "postDate": "06/17/2016 03:14:39", "content": "IMO this competition would make a lot more sense if the test dataset included Venta_uni_hoy (Sales units this week) - no need to change the objective function or anything. The logic here is that if you consider Sales units = 100 and you have a model that predicts Demand will be 100 that implies you should increase your Sales units. If your model predicts demand = 10, you should decrease your Sales units. As the competition stands, Sales units is essentially the thing we're trying to predict.
\n\nOf course, I'm not exactly suggesting to change the competition at this point. I'd be okay with it, but some people might cry...
", "rawMarkdown": "IMO this competition would make a lot more sense if the test dataset included Venta_uni_hoy (Sales units this week) - no need to change the objective function or anything. The logic here is that if you consider Sales units = 100 and you have a model that predicts Demand will be 100 that implies you should increase your Sales units. If your model predicts demand = 10, you should decrease your Sales units. As the competition stands, Sales units is essentially the thing we're trying to predict.\r\n\r\nOf course, I'm not exactly suggesting to change the competition at this point. I'd be okay with it, but some people might cry...", "votes": null }, { "id": "124327", "postDate": "06/17/2016 07:23:41", "content": "There's a little point at which I disagree:
\n\n[quote=AndersOhrn;124303]
\n\n[/quote]
\n\nWhy are we assuming that zero returns == the demand was not fully met?
\n\nHave a look at the list of the products we are working with. True, some of them will perish in a week or so, but there will be many others that will (and should) stay on the shelves much more time. And that would invalidate your premise.
\n\nIn my opinion, the definition of the problem is correct; remember that we are having a look at the problem from the seller's point of view -> we don't really have information about the final customers behaviour (at least, can't obtain it easily).
\n\nOn the other hand, the issue with the returns to me means that the competition host just wants to predict how many units should be producing, instead of how many should ship. And what is the simplest way of representing that? ---> Units shipped in week N - returns in week N+1. I think that this is the most simple interpretation.
", "rawMarkdown": "There's a little point at which I disagree:\r\n\r\n[quote=AndersOhrn;124303]\r\n\r\n* Every entry has zero returned items. In other words, according to the data, at no point has the demand been fully met by the supply.\r\n\r\n[/quote]\r\n\r\nWhy are we assuming that zero returns == the demand was not fully met?\r\n\r\nHave a look at the list of the products we are working with. True, some of them will perish in a week or so, but there will be many others that will (and should) stay on the shelves much more time. And that would invalidate your premise.\r\n\r\nIn my opinion, the definition of the problem is correct; remember that we are having a look at the problem from the seller's point of view -> we don't really have information about the final customers behaviour (at least, can't obtain it easily).\r\n\r\nOn the other hand, the issue with the returns to me means that the competition host just wants to predict how many units should be producing, *instead of how many should ship*. And what is the simplest way of representing that? ---> Units shipped in week N - returns in week N+1. I think that this is the most simple interpretation.", "votes": null }, { "id": "124433", "postDate": "06/18/2016 13:58:30", "content": "Consider a hypothetical transition probability matrix. For the following variables,
\n\ndefine the states as:
\n\nstate1<-c("R_t==0 & S_t1 < S_t",\n "R_t==0 & S_t1 == S_t",\n "R_t==0 & S_t1 > S_t",\n "R_t>1 & S_t1 < S_t",\n "R_t>1 & S_t1 == S_t",\n "R_t>1 & S_t1 > S_t"\n ) \nstate2<-c("lead(R_t)==0 & lead(S_t1) < lead(S_t)",\n "lead(R_t)==0 & lead(S_t1) == lead(S_t)",\n "lead(R_t)==0 & lead(S_t1) > lead(S_t)",\n "lead(R_t)>1 & lead(S_t1) < lead(S_t)",\n "lead(R_t)>1 & lead(S_t1) == lead(S_t)",\n "lead(R_t)>1 & lead(S_t1) > lead(S_t)" )\nCrunch numbers...
\n\n> P\n R=0,S1<S R=0,S1=S R=0,S1>S R>0,S1<S R>0,S1=S R>0,S1>S\nR=0,S1<S 0.2126075 0.12479651 0.6563643 0.001592332 0.0007631981 0.003876204\nR=0,S1=S 0.2436116 0.57314601 0.1785044 0.001875952 0.0014560190 0.001405975\nR=0,S1>S 0.6298764 0.15642210 0.2054879 0.005133265 0.0011423148 0.001938022\nR>0,S1<S 0.2139869 0.10401679 0.4260662 0.078604740 0.0228129797 0.154512331\nR>0,S1=S 0.2598516 0.19685838 0.1346320 0.103945578 0.2652071738 0.039505257\nR>0,S1>S 0.4163396 0.08951427 0.1286476 0.258307532 0.0368325215 0.070358465\nThe goal is to fit systems like this at whichever level you determine is the most tractable/least noisy and then "forecast" the states of weeks 10 and 11.
\n\nAs one can see in the matrix, the fact that returns are zero in many cases, does not imply that we cannot learn meaningful information about the underlying demand dynamics.
", "rawMarkdown": "Consider a hypothetical transition probability matrix. For the following variables,\r\n\r\n - S_t = Sales from this week \r\n - R_t = Returns from this week's sales \r\n - S_t1= Sales from next week\r\n\r\ndefine the states as:\r\n\r\n state1<-c(\"R_t==0 & S_t1 < S_t\",\r\n \"R_t==0 & S_t1 == S_t\",\r\n \"R_t==0 & S_t1 > S_t\",\r\n \"R_t>1 & S_t1 < S_t\",\r\n \"R_t>1 & S_t1 == S_t\",\r\n \"R_t>1 & S_t1 > S_t\"\r\n ) \r\n state2<-c(\"lead(R_t)==0 & lead(S_t1) < lead(S_t)\",\r\n \"lead(R_t)==0 & lead(S_t1) == lead(S_t)\",\r\n \"lead(R_t)==0 & lead(S_t1) > lead(S_t)\",\r\n \"lead(R_t)>1 & lead(S_t1) < lead(S_t)\",\r\n \"lead(R_t)>1 & lead(S_t1) == lead(S_t)\",\r\n \"lead(R_t)>1 & lead(S_t1) > lead(S_t)\" )\r\n\r\nCrunch numbers...\r\n\r\n > P\r\n R=0,S1My main point is that the historical data contains the prediction shortcomings of the delivery sales employee. In the problem description it says:
\n\n"Currently, daily inventory calculations are performed by direct delivery sales employees who must single-handedly predict the forces of supply, demand, and hunger based on their personal experiences with each store."
\n\nSo is the assumption that these employees have accurately predicted the amount of product to sell to the retailer? In other words, that in the 92% of cases where there were zero returns the following week that the prediction was perfect and all demand was met? Only then would the demand column be semantically identical to the 8% cases where there were returns.
\n\nI find no evidence in the description that this is a correct assumption. I viewed the business problem to be one of improving upon current approach, such that the employee would sell more product to a client if demand wasn't fully met (empty shelves), and less if demand was exceeded requiring returns to be bought back by Bimbo. But if the goal is to create an algorithm no better than current employees, then of course my critique is invalid.
", "rawMarkdown": "My main point is that the historical data contains the prediction shortcomings of the delivery sales employee. In the problem description it says:\r\n\r\n\"Currently, daily inventory calculations are performed by direct delivery sales employees who must single-handedly predict the forces of supply, demand, and hunger based on their personal experiences with each store.\"\r\n\r\nSo is the assumption that these employees have accurately predicted the amount of product to sell to the retailer? In other words, that in the 92% of cases where there were zero returns the following week that the prediction was perfect and all demand was met? Only then would the demand column be semantically identical to the 8% cases where there were returns.\r\n\r\nI find no evidence in the description that this is a correct assumption. I viewed the business problem to be one of improving upon current approach, such that the employee would sell more product to a client if demand wasn't fully met (empty shelves), and less if demand was exceeded requiring returns to be bought back by Bimbo. But if the goal is to create an algorithm no better than current employees, then of course my critique is invalid.", "votes": null }, { "id": "124496", "postDate": "06/19/2016 11:29:53", "content": "@Metabaron is right, it strange competition.
\n\nThe actual problem is a Theory of Constrain distribution solution out of the book. And it's also bakery to make it complete match with a book: “When I first did the analysis, I noticed that the company assumed that the orders the shops were giving were the best prediction for the next day's demand. That is the assumption that I elected to challenge. I speculated that the demand is higher than the orders because the orders are influenced by the conservatism of the shop owner. As you read, I'd had no idea of the magnitude of the conservatism, but now we are standing on much more solid ground. We know that the conservatism is playing a major role. We know that it is bigger than what I allowed myself to hope for. The resulting increase is not 10 percent or even 30 percent; it is 60 percent.”
\n\nThe issue: it’s difficult to quantify missing sales and implementing TOC solution increased 60% sales according to “The Choice”.\nAlso bakery accepts unsold goods and it is not a complete loss: some of the goods are going back into the same shops as croutons and biscuits.
\n\nAnd if you apply TOC concept to design this competition it only takes to introduce additional column into training/testing dataset like number of customers walked out empty handed from the shop. It can be randomly generated integer value, later updated with observations from the shops, but then ML models out of competition would present a real business value, measurable in cashflow and other accounting practices.
", "rawMarkdown": "Metabaron is right, it strange competition. \r\n\r\nThe actual problem is a Theory of Constrain distribution solution out of the book. And it's also bakery to make it complete match with a book: “When I first did the analysis, I noticed that the company assumed that the orders the shops were giving were the best prediction for the next day's demand. That is the assumption that I elected to challenge. I speculated that the demand is higher than the orders because the orders are influenced by the conservatism of the shop owner. As you read, I'd had no idea of the magnitude of the conservatism, but now we are standing on much more solid ground. We know that the conservatism is playing a major role. We know that it is bigger than what I allowed myself to hope for. The resulting increase is not 10 percent or even 30 percent; it is 60 percent.”\r\n\r\nThe issue: it’s difficult to quantify missing sales and implementing TOC solution increased 60% sales according to “The Choice”.\r\nAlso bakery accepts unsold goods and it is not a complete loss: some of the goods are going back into the same shops as croutons and biscuits. \r\n\r\nAnd if you apply TOC concept to design this competition it only takes to introduce additional column into training/testing dataset like number of customers walked out empty handed from the shop. It can be randomly generated integer value, later updated with observations from the shops, but then ML models out of competition would present a real business value, measurable in cashflow and other accounting practices.", "votes": null }, { "id": "124497", "postDate": "06/19/2016 12:06:50", "content": "All those zeroes in the returns field indicate a avariable that is not normally distributed. That sort of problem is not unusual, and is well researched. Unfortunately, I don't have a link or keyword to help with. It may be feasible to estimate using only the cases where returns are not zero.
\n\nAs for the "refunds," they might actually be credits as opposed to payments.
\n\nWe don't know at what price products were sold, assuming Mexico has laws that forbid price fixing by manufacturers. In the USA, we have a "manufacturers suggest retail price," not a stipulated price, and almost everyone sells beneath that price level. However, convenience stores where you park 50 feet from the shelf sell at a higher price than stores where you park half a mile from the shelf.
\n\nIt looks like demand equals items put on the shelf at the beginning of the week minus returns. The introduction says that "some" breads have a shelf life of only one week. This implies other products have longer shelf, but the examples I saw all showed demand to equal placed product minus returns for the week.
\n\nFinally, there is an odd word in the description -- "supply, demand, and hunger." Since demand means you're in the market with ability to purchase, "hunger" could stand for theft (shrinkage). This could be subject to greater fluctuations in a developing country like Mexico than in other, developed countries where food security is less of an issue. I'm unaware of any other instance where those words are used together. Theft of basic foods like breads is worth measuring.
", "rawMarkdown": "All those zeroes in the returns field indicate a avariable that is not normally distributed. That sort of problem is not unusual, and is well researched. Unfortunately, I don't have a link or keyword to help with. It may be feasible to estimate using only the cases where returns are not zero. \r\n\r\nAs for the \"refunds,\" they might actually be credits as opposed to payments. \r\n\r\nWe don't know at what price products were sold, assuming Mexico has laws that forbid price fixing by manufacturers. In the USA, we have a \"manufacturers suggest retail price,\" not a stipulated price, and almost everyone sells beneath that price level. However, convenience stores where you park 50 feet from the shelf sell at a higher price than stores where you park half a mile from the shelf.\r\n\r\nIt looks like demand equals items put on the shelf at the beginning of the week minus returns. The introduction says that \"some\" breads have a shelf life of only one week. This implies other products have longer shelf, but the examples I saw all showed demand to equal placed product minus returns for the week.\r\n\r\nFinally, there is an odd word in the description -- \"supply, demand, and hunger.\" Since demand means you're in the market with ability to purchase, \"hunger\" could stand for theft (shrinkage). This could be subject to greater fluctuations in a developing country like Mexico than in other, developed countries where food security is less of an issue. I'm unaware of any other instance where those words are used together. Theft of basic foods like breads is worth measuring.", "votes": null } ], "comments": [ { "id": 123413, "author_name": "snowdog", "author_url": "", "post_date": "06/11/2016 18:58:51", "content": "Why do you think that "sales this week" is a guess? Can't it be observed data?
", "votes": null, "replies": [] }, { "id": 123416, "author_name": "lyytinen", "author_url": "", "post_date": "06/11/2016 19:25:32", "content": "It probably is observed but let's consider an example:
\n\n\n\n\nVenta_uni_hoy: 8 Venta_hoy: 46.08 Dev_uni_proxima: 0 Dev_proxima: 0\n Demanda_uni_equil: 8
\n
I'm currently interpreting this so that they delivered 8 units and sold everything as there were no returns next week. But what if the actual demand was 10 units and 2 consumers were left without their burritos? So why is it correct to assume that the demand was 8?
", "votes": null, "replies": [] }, { "id": 123422, "author_name": "snowdog", "author_url": "", "post_date": "06/11/2016 19:52:40", "content": "I see your point now. I think that when dev_proxima equals 0, the real demand will be greater than venta_uni_hoy most of the time. Maybe "demanda" is not the best word to describe the target variable. It's better to move this questions to the welcome page anyway.
", "votes": null, "replies": [] }, { "id": 123470, "author_name": "juanrdz", "author_url": "", "post_date": "06/12/2016 04:06:06", "content": "It doesn't work like that at all. Bimbo has two things, big bread factories around the country, and a distribution channel. Their distribution channel is based on trucks that visit retailers every week. Retailers can be big supermarkets (think of walmart), or very small corner stores managed by a single person.
\n\nWhen the sales guy makes their visit, the retailer specifies how much of each product they want to buy based on their own experience, budget, etc. On the next visit , the sales guy retrieves the unsold merchandise that has gone beyond the display date. They would do this to avoid having retailers selling old bread and giving the brand a bad reputation. Thus, it is not unthinkable that they might refund a part of it just to keep the retailers happy even if they screw up a bit (also, Bimbo has some stores that sell the bread that has not gone bad just beyond its display date).
\n\nSo wrapping up ,
\n\nAt least thats the way I see it.
", "votes": null, "replies": [] }, { "id": 123496, "author_name": "carrdelling", "author_url": "", "post_date": "06/12/2016 09:07:54", "content": "Also note that the performance metric used in this problem (RMSLE) penalises under-predicted estimates more (i.e. is better to go above the optimum that below). Which means that a good solution will generally avoid 'having the clients staring at empty shelves' (at the cost of having more returns).
\n\nhttps://www.kaggle.com/wiki/RootMeanSquaredLogarithmicError
", "votes": null, "replies": [] }, { "id": 123515, "author_name": "ignang", "author_url": "", "post_date": "06/12/2016 12:24:47", "content": "In fact in Demand planning, the model looks like the Newsboy Vendor Problem. But then, we should have a cost of shortage which is the margin that Bimbo looses when it misses a sale and a cost of excess which is what Bimbo refunds to the dealer in case of excess. The newsboy model then proposes quantities that maximize the overall profit. Bimbo pays back dealers to avoid then selling expired products but mainly to avoid having some dealers buying insufficient quantities because they don't want to have access. \nGiven that we don't have these inputs (cost of excess, cost of shortage), we can't go this way and we have to stick on the given data.
", "votes": null, "replies": [] }, { "id": 123528, "author_name": "juanrdz", "author_url": "", "post_date": "06/12/2016 14:41:10", "content": "@Ibrahima, thanks, I learned something new today! (i have zero operations background haha). The second reason to pay back to the retailer is spot on. Would it be possible/accurate to set the cost of excess/shortage as a multiplier of the unitary price?
", "votes": null, "replies": [] }, { "id": 123530, "author_name": "lyytinen", "author_url": "", "post_date": "06/12/2016 14:51:34", "content": "Thanks everyone for your comments! This was very helpful.
", "votes": null, "replies": [] }, { "id": 124303, "author_name": "ajohrn", "author_url": "", "post_date": "06/17/2016 01:06:02", "content": "I agree with the poster. This prediction problem is not well defined for the stated business goal. Instead we are creating an algorithm that will perform no better than the human heuristics, with the same errors. I'll illustrate my point with examples and data.
\n\nSorry for the lengthy post, but I need it to prove my point.
\n\nWhy the Objective Function For Prediction Is a Poor Choice
\n\nBelow a small part of the training data is shown (some columns removed, with column names translated to English). It is all the data for client 1367838 with product 30552, and client 1367838 with product 30572.
\n\nWeek Delivery ID,Client ID,Product ID,Units Sold,Units Returned,Adjusted Demand Units\n3,1367838,30552,1276,0,1276\n4,1367838,30552,616,0,616\n5,1367838,30552,616,0,616\n6,1367838,30552,616,0,616\n7,1367838,30552,616,0,616\n3,1367838,30572,1248,0,1248\n4,1367838,30572,611,0,611\n5,1367838,30572,564,0,564\n6,1367838,30572,564,0,564\n7,1367838,30572,94,0,94\nThe key observations:
\n\nEvery entry has zero returned items. In other words, according to the data, at no point has the demand been fully met by the supply.
As a consequence, the adjusted demand that we are meant to predict is by definition equal to the sales guess made by the human.
If we look at product 30572 in week 3, we see that even with 1248 units, not all demand is met. Still, the units sold in the subsequent weeks are 611, 564 or even as low as 94. From the data alone, the predicted correct business decision would be to at least sell 1248, maybe more, in future weeks. However, given that the adjusted demand is here not a function of the market demand, rather a reflection of the human guess, the best value to predict would probably be 564 or some similar measure of central tendency. In short, the objective function of the problem given to us would be minimized if I reproduced the human heuristics, no matter how wrong.
In cases where there are units returned, the adjusted demand is indeed a reflection of what the true demand of the product at that client was. The business optimization then aligns with the objective function given to us. So is data with zero returns the rule of the exception?
\n\nOf all client-product pairs I find that 23783091 or about 92% have no week with non-zero returns. In other words, these are data points where the value we are asked to predict is equal to the human guess, with its shortcomings. (It should be noted that many client-product pairs are sampled only one or two weeks). With this skew, the objective function is likely dominated by points that do not align with business goals.
\n\nTo conclude: a prediction method performs well if it reproduces the human biases and mistakes. It does not propose actions that improve upon the problem of meeting demand without excess supply.
\n\nSome Ideas On How To Recast the Problem To Align With Business Goals
\n\nThe easiest way to make the prediction meaningful to business would be to limit predictions to client-product pairs that are properly sampled in the training data (a few non-zero returns in the data). It would obviously simplify the problem, but arguably too much.
\n\nSince we do not have the luxury to run experiments based on forecasts by a prediction algorithm, we must cast the prediction such that human shortcomings do not skew the objective function from the business goal. One way would be to make this a classification problem, like this:
\n\nGiven a client ID, a product ID and the actual Units Sold, we try to predict:
\n\nThis will have some business relevance. A method that performs well on this classification will educate the person in the warehouse if his/her suggestion for number of unit of product X to sell to client Y is likely too low, too high or about right.
\n\nA problem cast this way, with the appropriate objective function, is not skewed by the human failures. What we instead are doing is to create a prediction of said failures. Can a machine figure out if the proposed sales number is off the mark or not?
\n\nKaggle, how about changing the objective function?
", "votes": null, "replies": [] }, { "id": 124311, "author_name": "ben519", "author_url": "", "post_date": "06/17/2016 03:14:39", "content": "IMO this competition would make a lot more sense if the test dataset included Venta_uni_hoy (Sales units this week) - no need to change the objective function or anything. The logic here is that if you consider Sales units = 100 and you have a model that predicts Demand will be 100 that implies you should increase your Sales units. If your model predicts demand = 10, you should decrease your Sales units. As the competition stands, Sales units is essentially the thing we're trying to predict.
\n\nOf course, I'm not exactly suggesting to change the competition at this point. I'd be okay with it, but some people might cry...
", "votes": null, "replies": [] }, { "id": 124327, "author_name": "carrdelling", "author_url": "", "post_date": "06/17/2016 07:23:41", "content": "There's a little point at which I disagree:
\n\n[quote=AndersOhrn;124303]
\n\n[/quote]
\n\nWhy are we assuming that zero returns == the demand was not fully met?
\n\nHave a look at the list of the products we are working with. True, some of them will perish in a week or so, but there will be many others that will (and should) stay on the shelves much more time. And that would invalidate your premise.
\n\nIn my opinion, the definition of the problem is correct; remember that we are having a look at the problem from the seller's point of view -> we don't really have information about the final customers behaviour (at least, can't obtain it easily).
\n\nOn the other hand, the issue with the returns to me means that the competition host just wants to predict how many units should be producing, instead of how many should ship. And what is the simplest way of representing that? ---> Units shipped in week N - returns in week N+1. I think that this is the most simple interpretation.
", "votes": null, "replies": [] }, { "id": 124433, "author_name": "theoldfather", "author_url": "", "post_date": "06/18/2016 13:58:30", "content": "Consider a hypothetical transition probability matrix. For the following variables,
\n\ndefine the states as:
\n\nstate1<-c("R_t==0 & S_t1 < S_t",\n "R_t==0 & S_t1 == S_t",\n "R_t==0 & S_t1 > S_t",\n "R_t>1 & S_t1 < S_t",\n "R_t>1 & S_t1 == S_t",\n "R_t>1 & S_t1 > S_t"\n ) \nstate2<-c("lead(R_t)==0 & lead(S_t1) < lead(S_t)",\n "lead(R_t)==0 & lead(S_t1) == lead(S_t)",\n "lead(R_t)==0 & lead(S_t1) > lead(S_t)",\n "lead(R_t)>1 & lead(S_t1) < lead(S_t)",\n "lead(R_t)>1 & lead(S_t1) == lead(S_t)",\n "lead(R_t)>1 & lead(S_t1) > lead(S_t)" )\nCrunch numbers...
\n\n> P\n R=0,S1<S R=0,S1=S R=0,S1>S R>0,S1<S R>0,S1=S R>0,S1>S\nR=0,S1<S 0.2126075 0.12479651 0.6563643 0.001592332 0.0007631981 0.003876204\nR=0,S1=S 0.2436116 0.57314601 0.1785044 0.001875952 0.0014560190 0.001405975\nR=0,S1>S 0.6298764 0.15642210 0.2054879 0.005133265 0.0011423148 0.001938022\nR>0,S1<S 0.2139869 0.10401679 0.4260662 0.078604740 0.0228129797 0.154512331\nR>0,S1=S 0.2598516 0.19685838 0.1346320 0.103945578 0.2652071738 0.039505257\nR>0,S1>S 0.4163396 0.08951427 0.1286476 0.258307532 0.0368325215 0.070358465\nThe goal is to fit systems like this at whichever level you determine is the most tractable/least noisy and then "forecast" the states of weeks 10 and 11.
\n\nAs one can see in the matrix, the fact that returns are zero in many cases, does not imply that we cannot learn meaningful information about the underlying demand dynamics.
", "votes": null, "replies": [] }, { "id": 124480, "author_name": "ajohrn", "author_url": "", "post_date": "06/19/2016 05:18:03", "content": "My main point is that the historical data contains the prediction shortcomings of the delivery sales employee. In the problem description it says:
\n\n"Currently, daily inventory calculations are performed by direct delivery sales employees who must single-handedly predict the forces of supply, demand, and hunger based on their personal experiences with each store."
\n\nSo is the assumption that these employees have accurately predicted the amount of product to sell to the retailer? In other words, that in the 92% of cases where there were zero returns the following week that the prediction was perfect and all demand was met? Only then would the demand column be semantically identical to the 8% cases where there were returns.
\n\nI find no evidence in the description that this is a correct assumption. I viewed the business problem to be one of improving upon current approach, such that the employee would sell more product to a client if demand wasn't fully met (empty shelves), and less if demand was exceeded requiring returns to be bought back by Bimbo. But if the goal is to create an algorithm no better than current employees, then of course my critique is invalid.
", "votes": null, "replies": [] }, { "id": 124496, "author_name": "alexmikhalev", "author_url": "", "post_date": "06/19/2016 11:29:53", "content": "@Metabaron is right, it strange competition.
\n\nThe actual problem is a Theory of Constrain distribution solution out of the book. And it's also bakery to make it complete match with a book: “When I first did the analysis, I noticed that the company assumed that the orders the shops were giving were the best prediction for the next day's demand. That is the assumption that I elected to challenge. I speculated that the demand is higher than the orders because the orders are influenced by the conservatism of the shop owner. As you read, I'd had no idea of the magnitude of the conservatism, but now we are standing on much more solid ground. We know that the conservatism is playing a major role. We know that it is bigger than what I allowed myself to hope for. The resulting increase is not 10 percent or even 30 percent; it is 60 percent.”
\n\nThe issue: it’s difficult to quantify missing sales and implementing TOC solution increased 60% sales according to “The Choice”.\nAlso bakery accepts unsold goods and it is not a complete loss: some of the goods are going back into the same shops as croutons and biscuits.
\n\nAnd if you apply TOC concept to design this competition it only takes to introduce additional column into training/testing dataset like number of customers walked out empty handed from the shop. It can be randomly generated integer value, later updated with observations from the shops, but then ML models out of competition would present a real business value, measurable in cashflow and other accounting practices.
", "votes": null, "replies": [] }, { "id": 124497, "author_name": "matt605", "author_url": "", "post_date": "06/19/2016 12:06:50", "content": "All those zeroes in the returns field indicate a avariable that is not normally distributed. That sort of problem is not unusual, and is well researched. Unfortunately, I don't have a link or keyword to help with. It may be feasible to estimate using only the cases where returns are not zero.
\n\nAs for the "refunds," they might actually be credits as opposed to payments.
\n\nWe don't know at what price products were sold, assuming Mexico has laws that forbid price fixing by manufacturers. In the USA, we have a "manufacturers suggest retail price," not a stipulated price, and almost everyone sells beneath that price level. However, convenience stores where you park 50 feet from the shelf sell at a higher price than stores where you park half a mile from the shelf.
\n\nIt looks like demand equals items put on the shelf at the beginning of the week minus returns. The introduction says that "some" breads have a shelf life of only one week. This implies other products have longer shelf, but the examples I saw all showed demand to equal placed product minus returns for the week.
\n\nFinally, there is an odd word in the description -- "supply, demand, and hunger." Since demand means you're in the market with ability to purchase, "hunger" could stand for theft (shrinkage). This could be subject to greater fluctuations in a developing country like Mexico than in other, developed countries where food security is less of an issue. I'm unaware of any other instance where those words are used together. Theft of basic foods like breads is worth measuring.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "123409": "This whole thing seems a bit strange to me. I might have misunderstood something so please enlighten me and tell me where I go wrong.\r\n\r\nFirst of all, the fact that Grupo Bimbo has to refund store owners for unsold products is a bit surprising. Is it really so that Grupo Bimbo gets to choose the quantity of products to deliver but then has to refund if they over-deliver?\r\n\r\nAssuming that I got the first part correct, I really have to ask what are we actually modeling here? I mean the task is to create a model \"to accurately forecast inventory demand\". The demand for a product is \"defined as the sales this week subtracted by the return next week\". \r\n\r\nBut, in our training data, \"sales this week\" is actually an educated guess done by a random Grupo Bimbo employee. Right? And if there are no returns next week, this guess is also accepted as the correct \"demand\". But what if the real demand was actually higher than that and the consumers indeed were staring at empty shelves? The training data does not tell us anything about this.\r\n\r\nAssume that you still manage to build a model that perfectly predicts the actual demand and therefore maximises Grupo Bimpo profits. That model now gets penalized in the cases where the guess done by the Grupo Bimbo employee was lower than our perfect prediction. So to build a winning model for this competition you instead have to predict what Grupo Bimbo employees predicted in the past.", "123413": "Why do you think that \"sales this week\" is a guess? Can't it be observed data?", "123416": "It probably *is* observed but let's consider an example:\r\n\r\n> Venta_uni_hoy: 8 Venta_hoy: 46.08 Dev_uni_proxima: 0 Dev_proxima: 0\r\n> Demanda_uni_equil: 8\r\n\r\nI'm currently interpreting this so that they delivered 8 units and sold everything as there were no returns next week. But what if the actual demand was 10 units and 2 consumers were left without their burritos? So why is it correct to assume that the demand was 8?", "123422": "I see your point now. I think that when dev_proxima equals 0, the real demand will be greater than venta_uni_hoy most of the time. Maybe \"demanda\" is not the best word to describe the target variable. It's better to move this questions to the welcome page anyway.", "123470": "It doesn't work like that at all. Bimbo has two things, big bread factories around the country, and a distribution channel. Their distribution channel is based on trucks that visit retailers every week. Retailers can be big supermarkets (think of walmart), or very small corner stores managed by a single person. \r\n\r\nWhen the sales guy makes their visit, the retailer specifies how much of each product they want to buy based on their own experience, budget, etc. On the next visit , the sales guy retrieves the unsold merchandise that has gone beyond the display date. They would do this to avoid having retailers selling old bread and giving the brand a bad reputation. Thus, it is not unthinkable that they might refund a part of it just to keep the retailers happy even if they screw up a bit (also, Bimbo has some stores that sell the bread that has not gone bad just beyond its display date).\r\n\r\nSo wrapping up , \r\n\r\n- \"sales this week\" are the products the retailer decided to buy from the Bimbo sales guy\r\n- You're a bit correct on the demand part. It might be that customers were looking at empty shelves, but you should expect that the retailer notices that and makes a bigger purchase next week.\r\n- You're correct about penalizing a perfect demand forecast model because the retailers often miss to perfectly estimate the demand. However there is no such thing as perfect demand forecasting in the wild at this scale. \r\n\r\nAt least thats the way I see it.", "123496": "Also note that the performance metric used in this problem (RMSLE) penalises under-predicted estimates more (i.e. is better to go above the optimum that below). Which means that a good solution will generally avoid 'having the clients staring at empty shelves' (at the cost of having more returns).\r\n\r\nhttps://www.kaggle.com/wiki/RootMeanSquaredLogarithmicError", "123515": "In fact in Demand planning, the model looks like the Newsboy Vendor Problem. But then, we should have a cost of shortage which is the margin that Bimbo looses when it misses a sale and a cost of excess which is what Bimbo refunds to the dealer in case of excess. The newsboy model then proposes quantities that maximize the overall profit. Bimbo pays back dealers to avoid then selling expired products but mainly to avoid having some dealers buying insufficient quantities because they don't want to have access. \r\nGiven that we don't have these inputs (cost of excess, cost of shortage), we can't go this way and we have to stick on the given data.", "123528": "Ibrahima, thanks, I learned something new today! (i have zero operations background haha). The second reason to pay back to the retailer is spot on. Would it be possible/accurate to set the cost of excess/shortage as a multiplier of the unitary price?", "123530": "Thanks everyone for your comments! This was very helpful.", "124303": "I agree with the poster. This prediction problem is not well defined for the stated business goal. Instead we are creating an algorithm that will perform no better than the human heuristics, with the same errors. I'll illustrate my point with examples and data. \r\n\r\n*Sorry for the lengthy post, but I need it to prove my point.*\r\n\r\n**Why the Objective Function For Prediction Is a Poor Choice**\r\n\r\nBelow a small part of the training data is shown (some columns removed, with column names translated to English). It is all the data for client 1367838 with product 30552, and client 1367838 with product 30572. \r\n\r\n Week Delivery ID,Client ID,Product ID,Units Sold,Units Returned,Adjusted Demand Units\r\n 3,1367838,30552,1276,0,1276\r\n 4,1367838,30552,616,0,616\r\n 5,1367838,30552,616,0,616\r\n 6,1367838,30552,616,0,616\r\n 7,1367838,30552,616,0,616\r\n 3,1367838,30572,1248,0,1248\r\n 4,1367838,30572,611,0,611\r\n 5,1367838,30572,564,0,564\r\n 6,1367838,30572,564,0,564\r\n 7,1367838,30572,94,0,94\r\n\r\nThe key observations:\r\n\r\n* Every entry has zero returned items. In other words, according to the data, at no point has the demand been fully met by the supply.\r\n\r\n* As a consequence, the adjusted demand *that we are meant to predict* is by definition equal to the sales guess made by the human.\r\n\r\n* If we look at product 30572 in week 3, we see that even with 1248 units, not all demand is met. Still, the units sold in the subsequent weeks are 611, 564 or even as low as 94. From the data alone, the predicted correct *business decision* would be to at least sell 1248, maybe more, in future weeks. However, given that the adjusted demand is here *not* a function of the market demand, rather a reflection of the human guess, the best value to predict would probably be 564 or some similar measure of central tendency. In short, the objective function of the problem given to us would be minimized if I reproduced the human heuristics, no matter how wrong.\r\n\r\nIn cases where there are units returned, the adjusted demand is indeed a reflection of what the true demand of the product at that client was. The business optimization then aligns with the objective function given to us. So is data with zero returns the rule of the exception?\r\n\r\nOf all client-product pairs I find that 23783091 or about 92% have *no week with non-zero returns.* In other words, these are data points where the value we are asked to predict is equal to the human guess, with its shortcomings. (It should be noted that many client-product pairs are sampled only one or two weeks). With this skew, the objective function is likely dominated by points that do not align with business goals.\r\n\r\nTo conclude: a prediction method performs well if it reproduces the human biases and mistakes. It does not propose actions that improve upon the problem of meeting demand without excess supply.\r\n\r\n**Some Ideas On How To Recast the Problem To Align With Business Goals**\r\n\r\nThe easiest way to make the prediction meaningful to business would be to limit predictions to client-product pairs that are properly sampled in the training data (a few non-zero returns in the data). It would obviously simplify the problem, but arguably too much.\r\n\r\nSince we do not have the luxury to run experiments based on forecasts by a prediction algorithm, we must cast the prediction such that human shortcomings do not skew the objective function from the business goal. One way would be to make this a classification problem, like this:\r\n\r\nGiven a client ID, a product ID *and* the actual Units Sold, we try to predict:\r\n\r\n* Will this fail to meet demand (Units Returned = 0), or\r\n* Will it overshoot demand by some threshold percentage (Units Returned / Units Sold > P), or\r\n* Will it be right between 0 returned and the threshold percentage (0 < Units Returned / Units Sold < P).\r\n\r\nThis will have some business relevance. A method that performs well on this classification will educate the person in the warehouse if his/her suggestion for number of unit of product X to sell to client Y is likely too low, too high or about right. \r\n\r\nA problem cast this way, with the appropriate objective function, is not skewed by the human failures. What we instead are doing is to create a prediction of said failures. Can a machine figure out if the proposed sales number is off the mark or not?\r\n\r\nKaggle, how about changing the objective function?", "124311": "IMO this competition would make a lot more sense if the test dataset included Venta_uni_hoy (Sales units this week) - no need to change the objective function or anything. The logic here is that if you consider Sales units = 100 and you have a model that predicts Demand will be 100 that implies you should increase your Sales units. If your model predicts demand = 10, you should decrease your Sales units. As the competition stands, Sales units is essentially the thing we're trying to predict.\r\n\r\nOf course, I'm not exactly suggesting to change the competition at this point. I'd be okay with it, but some people might cry...", "124327": "There's a little point at which I disagree:\r\n\r\n[quote=AndersOhrn;124303]\r\n\r\n* Every entry has zero returned items. In other words, according to the data, at no point has the demand been fully met by the supply.\r\n\r\n[/quote]\r\n\r\nWhy are we assuming that zero returns == the demand was not fully met?\r\n\r\nHave a look at the list of the products we are working with. True, some of them will perish in a week or so, but there will be many others that will (and should) stay on the shelves much more time. And that would invalidate your premise.\r\n\r\nIn my opinion, the definition of the problem is correct; remember that we are having a look at the problem from the seller's point of view -> we don't really have information about the final customers behaviour (at least, can't obtain it easily).\r\n\r\nOn the other hand, the issue with the returns to me means that the competition host just wants to predict how many units should be producing, *instead of how many should ship*. And what is the simplest way of representing that? ---> Units shipped in week N - returns in week N+1. I think that this is the most simple interpretation.", "124433": "Consider a hypothetical transition probability matrix. For the following variables,\r\n\r\n - S_t = Sales from this week \r\n - R_t = Returns from this week's sales \r\n - S_t1= Sales from next week\r\n\r\ndefine the states as:\r\n\r\n state1<-c(\"R_t==0 & S_t1 < S_t\",\r\n \"R_t==0 & S_t1 == S_t\",\r\n \"R_t==0 & S_t1 > S_t\",\r\n \"R_t>1 & S_t1 < S_t\",\r\n \"R_t>1 & S_t1 == S_t\",\r\n \"R_t>1 & S_t1 > S_t\"\r\n ) \r\n state2<-c(\"lead(R_t)==0 & lead(S_t1) < lead(S_t)\",\r\n \"lead(R_t)==0 & lead(S_t1) == lead(S_t)\",\r\n \"lead(R_t)==0 & lead(S_t1) > lead(S_t)\",\r\n \"lead(R_t)>1 & lead(S_t1) < lead(S_t)\",\r\n \"lead(R_t)>1 & lead(S_t1) == lead(S_t)\",\r\n \"lead(R_t)>1 & lead(S_t1) > lead(S_t)\" )\r\n\r\nCrunch numbers...\r\n\r\n > P\r\n R=0,S1