{
  "id": 66006,
  "title": "Proper way to apply target ?",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/66006",
  "author_name": "",
  "post_date": "2018-09-17T09:31:03.104187800Z",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Haven't done any ML/DL in a long time, and trying to come back by following the fast.ai course, starting with its <a href=\"http://forums.fast.ai/t/another-treat-early-access-to-intro-to-machine-learning-videos/6826\">ML track</a>. \nI would like to try and apply Random forest on this dataset as an exercise, and after studying the data and cleaning it a bit, I'm facing a big issue: How do I add my logTotalRevenue column to the dataset ?</p>\n\n<p>Since the target value is calculated by taking the log of the  sum of \n totals.transactionRevenue for several rows grouped by fullVisitorId, I can only think of 2 options:</p>\n\n<ul>\n<li>recreate a new dataset with only one row per user, data being\nmerged/averaged/etc (feels like we’ll lose information this way), then add the logTotalRevenue column.</li>\n<li>add a new column logTotalRevenue to the existing dataset containing several rows per user, containing the correct target value (ie if a user has 3 transactions, all 3 rows will have the same value for logTotalRevenue).</li>\n</ul>\n\n<p>Which one would you recommend me to dive into in order to continue with a random forests regressor ? Or is there a better way to deal with this data ? Or might it be that random forests is unusable in this case ?</p>\n\n<p>Thank you for your help</p>",
  "messages": [
    {
      "id": "388613",
      "postDate": "09/17/2018 09:31:03",
      "content": "<p>Haven't done any ML/DL in a long time, and trying to come back by following the fast.ai course, starting with its <a href=\"http://forums.fast.ai/t/another-treat-early-access-to-intro-to-machine-learning-videos/6826\">ML track</a>. \nI would like to try and apply Random forest on this dataset as an exercise, and after studying the data and cleaning it a bit, I'm facing a big issue: How do I add my logTotalRevenue column to the dataset ?</p>\n\n<p>Since the target value is calculated by taking the log of the  sum of \n totals.transactionRevenue for several rows grouped by fullVisitorId, I can only think of 2 options:</p>\n\n<ul>\n<li>recreate a new dataset with only one row per user, data being\nmerged/averaged/etc (feels like we’ll lose information this way), then add the logTotalRevenue column.</li>\n<li>add a new column logTotalRevenue to the existing dataset containing several rows per user, containing the correct target value (ie if a user has 3 transactions, all 3 rows will have the same value for logTotalRevenue).</li>\n</ul>\n\n<p>Which one would you recommend me to dive into in order to continue with a random forests regressor ? Or is there a better way to deal with this data ? Or might it be that random forests is unusable in this case ?</p>\n\n<p>Thank you for your help</p>",
      "rawMarkdown": "Haven't done any ML/DL in a long time, and trying to come back by following the fast.ai course, starting with its [ML track](http://forums.fast.ai/t/another-treat-early-access-to-intro-to-machine-learning-videos/6826). \nI would like to try and apply Random forest on this dataset as an exercise, and after studying the data and cleaning it a bit, I'm facing a big issue: How do I add my logTotalRevenue column to the dataset ?\n\nSince the target value is calculated by taking the log of the  sum of \n totals.transactionRevenue for several rows grouped by fullVisitorId, I can only think of 2 options:\n\n - recreate a new dataset with only one row per user, data being\n   merged/averaged/etc (feels like we’ll lose information this way), then add the logTotalRevenue column.\n - add a new column logTotalRevenue to the existing dataset containing several rows per user, containing the correct target value (ie if a user has 3 transactions, all 3 rows will have the same value for logTotalRevenue).\n\nWhich one would you recommend me to dive into in order to continue with a random forests regressor ? Or is there a better way to deal with this data ? Or might it be that random forests is unusable in this case ?\n\nThank you for your help",
      "votes": null
    },
    {
      "id": "392596",
      "postDate": "09/24/2018 03:54:05",
      "content": "<p>In my opinion, the simplest way is to treat each row as 'one training record', train the model, make prediction.</p>\n\n<p>After that,  sum the predicted revenue of the same customer, and then submit the result.</p>\n\n<p>The shortage is: the evaluation is based on the \"summed revenue\", while your model is trained based on purchase order.</p>",
      "rawMarkdown": "In my opinion, the simplest way is to treat each row as 'one training record', train the model, make prediction.\n\nAfter that,  sum the predicted revenue of the same customer, and then submit the result.\n\nThe shortage is: the evaluation is based on the \"summed revenue\", while your model is trained based on purchase order.",
      "votes": null
    },
    {
      "id": "392598",
      "postDate": "09/24/2018 03:58:34",
      "content": "<p>@Ray, this sounds so obvious when you say it ^^ Thank you very much, I'll give it another try soon :)</p>",
      "rawMarkdown": "Ray, this sounds so obvious when you say it ^^ Thank you very much, I'll give it another try soon :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 392596,
      "author_name": "zhangshuhui",
      "author_url": "",
      "post_date": "09/24/2018 03:54:05",
      "content": "<p>In my opinion, the simplest way is to treat each row as 'one training record', train the model, make prediction.</p>\n\n<p>After that,  sum the predicted revenue of the same customer, and then submit the result.</p>\n\n<p>The shortage is: the evaluation is based on the \"summed revenue\", while your model is trained based on purchase order.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 392598,
      "author_name": "fandekasp",
      "author_url": "",
      "post_date": "09/24/2018 03:58:34",
      "content": "<p>@Ray, this sounds so obvious when you say it ^^ Thank you very much, I'll give it another try soon :)</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "388613": "Haven't done any ML/DL in a long time, and trying to come back by following the fast.ai course, starting with its [ML track](http://forums.fast.ai/t/another-treat-early-access-to-intro-to-machine-learning-videos/6826). \nI would like to try and apply Random forest on this dataset as an exercise, and after studying the data and cleaning it a bit, I'm facing a big issue: How do I add my logTotalRevenue column to the dataset ?\n\nSince the target value is calculated by taking the log of the  sum of \n totals.transactionRevenue for several rows grouped by fullVisitorId, I can only think of 2 options:\n\n - recreate a new dataset with only one row per user, data being\n   merged/averaged/etc (feels like we’ll lose information this way), then add the logTotalRevenue column.\n - add a new column logTotalRevenue to the existing dataset containing several rows per user, containing the correct target value (ie if a user has 3 transactions, all 3 rows will have the same value for logTotalRevenue).\n\nWhich one would you recommend me to dive into in order to continue with a random forests regressor ? Or is there a better way to deal with this data ? Or might it be that random forests is unusable in this case ?\n\nThank you for your help",
    "392596": "In my opinion, the simplest way is to treat each row as 'one training record', train the model, make prediction.\n\nAfter that,  sum the predicted revenue of the same customer, and then submit the result.\n\nThe shortage is: the evaluation is based on the \"summed revenue\", while your model is trained based on purchase order.",
    "392598": "Ray, this sounds so obvious when you say it ^^ Thank you very much, I'll give it another try soon :)"
  },
  "source": "meta"
}