{
  "id": 77522,
  "title": "GA Cust. Rev. Prediction test set seems to belong to different breed of data",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/77522",
  "author_name": "",
  "post_date": "2019-01-13T19:12:59.786278300Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I have been playing with Google Analytics Customer Revenue Prediction data sets using xgboost in R for a while and even though the challenge is over I'd like to understand why my results are far away from what stated in the LB. I got my best result RSME = 0.25451 on the test set of 401589 rows (labelled test set) whilst I can get RMSE = 0.05456 on the test CV split of the 1708337 rows in 80% train 20% test. The difference in the results is very high so I wonder if the test set of 401589 rows actually belongs to a different \"breed\"/different context of data. Any comment is welcome, thanks.</p>",
  "messages": [
    {
      "id": "455412",
      "postDate": "01/13/2019 19:12:59",
      "content": "<p>I have been playing with Google Analytics Customer Revenue Prediction data sets using xgboost in R for a while and even though the challenge is over I'd like to understand why my results are far away from what stated in the LB. I got my best result RSME = 0.25451 on the test set of 401589 rows (labelled test set) whilst I can get RMSE = 0.05456 on the test CV split of the 1708337 rows in 80% train 20% test. The difference in the results is very high so I wonder if the test set of 401589 rows actually belongs to a different \"breed\"/different context of data. Any comment is welcome, thanks.</p>",
      "rawMarkdown": "I have been playing with Google Analytics Customer Revenue Prediction data sets using xgboost in R for a while and even though the challenge is over I'd like to understand why my results are far away from what stated in the LB. I got my best result RSME = 0.25451 on the test set of 401589 rows (labelled test set) whilst I can get RMSE = 0.05456 on the test CV split of the 1708337 rows in 80% train 20% test. The difference in the results is very high so I wonder if the test set of 401589 rows actually belongs to a different \"breed\"/different context of data. Any comment is welcome, thanks.",
      "votes": null
    },
    {
      "id": "456333",
      "postDate": "01/15/2019 15:37:23",
      "content": "<p>Remember the test dataset is on future data. So if your 80% train 20% test is not a time-series split, results could be worse than expected.</p>",
      "rawMarkdown": "Remember the test dataset is on future data. So if your 80% train 20% test is not a time-series split, results could be worse than expected.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 456333,
      "author_name": "returnofsputnik",
      "author_url": "",
      "post_date": "01/15/2019 15:37:23",
      "content": "<p>Remember the test dataset is on future data. So if your 80% train 20% test is not a time-series split, results could be worse than expected.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "455412": "I have been playing with Google Analytics Customer Revenue Prediction data sets using xgboost in R for a while and even though the challenge is over I'd like to understand why my results are far away from what stated in the LB. I got my best result RSME = 0.25451 on the test set of 401589 rows (labelled test set) whilst I can get RMSE = 0.05456 on the test CV split of the 1708337 rows in 80% train 20% test. The difference in the results is very high so I wonder if the test set of 401589 rows actually belongs to a different \"breed\"/different context of data. Any comment is welcome, thanks.",
    "456333": "Remember the test dataset is on future data. So if your 80% train 20% test is not a time-series split, results could be worse than expected."
  },
  "source": "meta"
}