{
  "id": 70441,
  "title": "Why wouldn't a realistic predictive model not have access to historical values of revenue of the user?",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/70441",
  "author_name": "",
  "post_date": "2018-11-03T17:49:38.726787400Z",
  "votes": 1,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I'm very new to everything but one thing doesn't make sense to me. Why wouldn't the model have access to the historical values of an unique user in a realistic scenario? My prediction rates go from 30% to 90% using the same model if I just consider that I have access to the previous historic purchases of the user (on the validation set).</p>\n\n<p>I can't see how that would be \"data leakage\" that I've learned about but I'd love for someone more experienced go into more detail over this.</p>\n\n<p>Since I might not have been able to explain myself properly, here's an example:</p>\n\n<p><strong>User1 visits for the first time:</strong> \nThey purchase something for 100 and have a revenue sum of 0.</p>\n\n<p><strong>User1 visits for the second time:</strong> \nThey purchase something for 50 and have a revenue sum of 100.</p>\n\n<p><strong>User1 visits for the third time:</strong> \nThey purchase something for 50 and have a revenue sum of 150.</p>\n\n<p>How would this be data leakage? The model would be much more precise in predicting how much the user would spend on the store.</p>",
  "messages": [
    {
      "id": "414839",
      "postDate": "11/03/2018 17:49:38",
      "content": "<p>I'm very new to everything but one thing doesn't make sense to me. Why wouldn't the model have access to the historical values of an unique user in a realistic scenario? My prediction rates go from 30% to 90% using the same model if I just consider that I have access to the previous historic purchases of the user (on the validation set).</p>\n\n<p>I can't see how that would be \"data leakage\" that I've learned about but I'd love for someone more experienced go into more detail over this.</p>\n\n<p>Since I might not have been able to explain myself properly, here's an example:</p>\n\n<p><strong>User1 visits for the first time:</strong> \nThey purchase something for 100 and have a revenue sum of 0.</p>\n\n<p><strong>User1 visits for the second time:</strong> \nThey purchase something for 50 and have a revenue sum of 100.</p>\n\n<p><strong>User1 visits for the third time:</strong> \nThey purchase something for 50 and have a revenue sum of 150.</p>\n\n<p>How would this be data leakage? The model would be much more precise in predicting how much the user would spend on the store.</p>",
      "rawMarkdown": "I'm very new to everything but one thing doesn't make sense to me. Why wouldn't the model have access to the historical values of an unique user in a realistic scenario? My prediction rates go from 30% to 90% using the same model if I just consider that I have access to the previous historic purchases of the user (on the validation set).\n\nI can't see how that would be \"data leakage\" that I've learned about but I'd love for someone more experienced go into more detail over this.\n\nSince I might not have been able to explain myself properly, here's an example:\n\n**User1 visits for the first time:** \nThey purchase something for 100 and have a revenue sum of 0.\n\n**User1 visits for the second time:** \nThey purchase something for 50 and have a revenue sum of 100.\n\n**User1 visits for the third time:** \nThey purchase something for 50 and have a revenue sum of 150.\n\nHow would this be data leakage? The model would be much more precise in predicting how much the user would spend on the store.",
      "votes": null
    },
    {
      "id": "415885",
      "postDate": "11/05/2018 21:19:21",
      "content": "<p>In my opinion and that of other kaggler too (from what I read in other discussions), this competition does not really have real utility in business.</p>\n\n<p>That being said, I do not think, however, that the example you just showed contains data leakage since the information comes from the past and would be included in the trainset.</p>\n\n<p>From machinelearningmastery.com:</p>\n\n<p>Data leakage is when information from outside the training dataset is used to create the model. This additional information can allow the model to learn or know something that it otherwise would not know and in turn invalidate the estimated performance of the mode being constructed.</p>\n\n<p><a href=\"https://machinelearningmastery.com/data-leakage-machine-learning/\">https://machinelearningmastery.com/data-leakage-machine-learning/</a></p>",
      "rawMarkdown": "In my opinion and that of other kaggler too (from what I read in other discussions), this competition does not really have real utility in business.\n\nThat being said, I do not think, however, that the example you just showed contains data leakage since the information comes from the past and would be included in the trainset.\n\nFrom machinelearningmastery.com:\n\nData leakage is when information from outside the training dataset is used to create the model. This additional information can allow the model to learn or know something that it otherwise would not know and in turn invalidate the estimated performance of the mode being constructed.\n\nhttps://machinelearningmastery.com/data-leakage-machine-learning/",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 415885,
      "author_name": "charlesdemontigny",
      "author_url": "",
      "post_date": "11/05/2018 21:19:21",
      "content": "<p>In my opinion and that of other kaggler too (from what I read in other discussions), this competition does not really have real utility in business.</p>\n\n<p>That being said, I do not think, however, that the example you just showed contains data leakage since the information comes from the past and would be included in the trainset.</p>\n\n<p>From machinelearningmastery.com:</p>\n\n<p>Data leakage is when information from outside the training dataset is used to create the model. This additional information can allow the model to learn or know something that it otherwise would not know and in turn invalidate the estimated performance of the mode being constructed.</p>\n\n<p><a href=\"https://machinelearningmastery.com/data-leakage-machine-learning/\">https://machinelearningmastery.com/data-leakage-machine-learning/</a></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "414839": "I'm very new to everything but one thing doesn't make sense to me. Why wouldn't the model have access to the historical values of an unique user in a realistic scenario? My prediction rates go from 30% to 90% using the same model if I just consider that I have access to the previous historic purchases of the user (on the validation set).\n\nI can't see how that would be \"data leakage\" that I've learned about but I'd love for someone more experienced go into more detail over this.\n\nSince I might not have been able to explain myself properly, here's an example:\n\n**User1 visits for the first time:** \nThey purchase something for 100 and have a revenue sum of 0.\n\n**User1 visits for the second time:** \nThey purchase something for 50 and have a revenue sum of 100.\n\n**User1 visits for the third time:** \nThey purchase something for 50 and have a revenue sum of 150.\n\nHow would this be data leakage? The model would be much more precise in predicting how much the user would spend on the store.",
    "415885": "In my opinion and that of other kaggler too (from what I read in other discussions), this competition does not really have real utility in business.\n\nThat being said, I do not think, however, that the example you just showed contains data leakage since the information comes from the past and would be included in the trainset.\n\nFrom machinelearningmastery.com:\n\nData leakage is when information from outside the training dataset is used to create the model. This additional information can allow the model to learn or know something that it otherwise would not know and in turn invalidate the estimated performance of the mode being constructed.\n\nhttps://machinelearningmastery.com/data-leakage-machine-learning/"
  },
  "source": "meta"
}