{
  "id": 67883,
  "title": "What is up with totals.bounces",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/67883",
  "author_name": "Xavier Bourret Sicotte",
  "post_date": "2018-10-06T18:08:50.458000",
  "votes": 12,
  "comment_count": 0,
  "views": 0,
  "content": "<p>As many of you know already from EDA and pulling your hair out on this data set... </p>\n\n<p>On the training set at the <strong>session level</strong></p>\n\n<ul>\n<li>When totals.bounces = 1, then revenues = 0</li>\n<li>There are  450,630 such sessions in train and 420,948 in test</li>\n</ul>\n\n<p>The obvious explanation is that in 2016, in order to make a purchase, more than one hit was required on the website. It is possible that this was no longer the case in 2017, for example if they introduced a new functionality on the website such as \"pay without clicking\"  but I am working under the assumption that this structural characteristic of the data set still holds. </p>\n\n<p>As a result, I would expect any reasonably good model to find such a pattern and predict 0 for such cases. Looking at the outputs of my best model (LB 1.4334) this is not the case and I struggle to understand the reason behind this as well as the implications this may have for more advanced modeling. </p>\n\n<p><img src=\"https://i.imgur.com/Q4bXgoL.png\" alt=\"enter image description here\"></p>\n\n<p>To summarize my results: </p>\n\n<ul>\n<li>True train sessions with bounce =1 :  Total revenue: 0.0 Total sessions: 450,630</li>\n<li>Predicted train sessions with bounce =1 :  Total revenue: 4474.43 Total sessions: 450,630 </li>\n<li>Predicted test sessions with bounce =1 :  Total revenue: 4823.01 Total sessions: 420,948</li>\n</ul>\n\n<p>Of course, this is nothing compared to the revenues for sessions with zero bounce: </p>\n\n<ul>\n<li>True train sessions with bounce =0 :  205,236 and 453,023 sessions</li>\n<li>Predicted train sessions with bounce =0 :  218,294 and 453,023 sessions</li>\n<li>Predicted test sessions with bounce =0 :  156,518 and 420,948 sessions</li>\n</ul>\n\n<p>... so post processing the predictions by fixing the bounces = 1 to zero revenue makes absolutely no difference (I just tried)</p>\n\n<p>But my concern is more about the potential bias, or skew these sessions introduce in the model. Here is the distribution of predictions made on a <a href=\"https://www.kaggle.com/xavierbourretsicotte/lgbm-error-analysis-negative-downsampling\">negative downsampled</a> model </p>\n\n<p><img src=\"https://i.imgur.com/jTjM9rW.png\" alt=\"enter image description here\"></p>\n\n<p>The actual performance of such a model is awful when measured on the CV and LB, but visually there is something going on...</p>",
  "messages": [
    {
      "id": 399768,
      "postDate": "2018-10-06T18:08:50.460Z",
      "content": "<p>As many of you know already from EDA and pulling your hair out on this data set... </p>\n\n<p>On the training set at the <strong>session level</strong></p>\n\n<ul>\n<li>When totals.bounces = 1, then revenues = 0</li>\n<li>There are  450,630 such sessions in train and 420,948 in test</li>\n</ul>\n\n<p>The obvious explanation is that in 2016, in order to make a purchase, more than one hit was required on the website. It is possible that this was no longer the case in 2017, for example if they introduced a new functionality on the website such as \"pay without clicking\"  but I am working under the assumption that this structural characteristic of the data set still holds. </p>\n\n<p>As a result, I would expect any reasonably good model to find such a pattern and predict 0 for such cases. Looking at the outputs of my best model (LB 1.4334) this is not the case and I struggle to understand the reason behind this as well as the implications this may have for more advanced modeling. </p>\n\n<p><img src=\"https://i.imgur.com/Q4bXgoL.png\" alt=\"enter image description here\"></p>\n\n<p>To summarize my results: </p>\n\n<ul>\n<li>True train sessions with bounce =1 :  Total revenue: 0.0 Total sessions: 450,630</li>\n<li>Predicted train sessions with bounce =1 :  Total revenue: 4474.43 Total sessions: 450,630 </li>\n<li>Predicted test sessions with bounce =1 :  Total revenue: 4823.01 Total sessions: 420,948</li>\n</ul>\n\n<p>Of course, this is nothing compared to the revenues for sessions with zero bounce: </p>\n\n<ul>\n<li>True train sessions with bounce =0 :  205,236 and 453,023 sessions</li>\n<li>Predicted train sessions with bounce =0 :  218,294 and 453,023 sessions</li>\n<li>Predicted test sessions with bounce =0 :  156,518 and 420,948 sessions</li>\n</ul>\n\n<p>... so post processing the predictions by fixing the bounces = 1 to zero revenue makes absolutely no difference (I just tried)</p>\n\n<p>But my concern is more about the potential bias, or skew these sessions introduce in the model. Here is the distribution of predictions made on a <a href=\"https://www.kaggle.com/xavierbourretsicotte/lgbm-error-analysis-negative-downsampling\">negative downsampled</a> model </p>\n\n<p><img src=\"https://i.imgur.com/jTjM9rW.png\" alt=\"enter image description here\"></p>\n\n<p>The actual performance of such a model is awful when measured on the CV and LB, but visually there is something going on...</p>",
      "rawMarkdown": "As many of you know already from EDA and pulling your hair out on this data set... \n\nOn the training set at the **session level**\n\n - When totals.bounces = 1, then revenues = 0\n - There are  450,630 such sessions in train and 420,948 in test\n\nThe obvious explanation is that in 2016, in order to make a purchase, more than one hit was required on the website. It is possible that this was no longer the case in 2017, for example if they introduced a new functionality on the website such as \"pay without clicking\"  but I am working under the assumption that this structural characteristic of the data set still holds. \n\nAs a result, I would expect any reasonably good model to find such a pattern and predict 0 for such cases. Looking at the outputs of my best model (LB 1.4334) this is not the case and I struggle to understand the reason behind this as well as the implications this may have for more advanced modeling. \n\n![enter image description here][1]\n\nTo summarize my results: \n\n - True train sessions with bounce =1 :  Total revenue: 0.0 Total sessions: 450,630\n - Predicted train sessions with bounce =1 :  Total revenue: 4474.43 Total sessions: 450,630 \n - Predicted test sessions with bounce =1 :  Total revenue: 4823.01 Total sessions: 420,948\n\nOf course, this is nothing compared to the revenues for sessions with zero bounce: \n\n- True train sessions with bounce =0 :  205,236 and 453,023 sessions\n- Predicted train sessions with bounce =0 :  218,294 and 453,023 sessions\n- Predicted test sessions with bounce =0 :  156,518 and 420,948 sessions\n\n... so post processing the predictions by fixing the bounces = 1 to zero revenue makes absolutely no difference (I just tried)\n\nBut my concern is more about the potential bias, or skew these sessions introduce in the model. Here is the distribution of predictions made on a [negative downsampled][2] model \n\n![enter image description here][3]\n\nThe actual performance of such a model is awful when measured on the CV and LB, but visually there is something going on...\n\n\n  [1]: https://i.imgur.com/Q4bXgoL.png\n  [2]:https://www.kaggle.com/xavierbourretsicotte/lgbm-error-analysis-negative-downsampling\n  [3]: https://i.imgur.com/jTjM9rW.png",
      "votes": 11
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "399768": "As many of you know already from EDA and pulling your hair out on this data set... \n\nOn the training set at the **session level**\n\n - When totals.bounces = 1, then revenues = 0\n - There are  450,630 such sessions in train and 420,948 in test\n\nThe obvious explanation is that in 2016, in order to make a purchase, more than one hit was required on the website. It is possible that this was no longer the case in 2017, for example if they introduced a new functionality on the website such as \"pay without clicking\"  but I am working under the assumption that this structural characteristic of the data set still holds. \n\nAs a result, I would expect any reasonably good model to find such a pattern and predict 0 for such cases. Looking at the outputs of my best model (LB 1.4334) this is not the case and I struggle to understand the reason behind this as well as the implications this may have for more advanced modeling. \n\n![enter image description here][1]\n\nTo summarize my results: \n\n - True train sessions with bounce =1 :  Total revenue: 0.0 Total sessions: 450,630\n - Predicted train sessions with bounce =1 :  Total revenue: 4474.43 Total sessions: 450,630 \n - Predicted test sessions with bounce =1 :  Total revenue: 4823.01 Total sessions: 420,948\n\nOf course, this is nothing compared to the revenues for sessions with zero bounce: \n\n- True train sessions with bounce =0 :  205,236 and 453,023 sessions\n- Predicted train sessions with bounce =0 :  218,294 and 453,023 sessions\n- Predicted test sessions with bounce =0 :  156,518 and 420,948 sessions\n\n... so post processing the predictions by fixing the bounces = 1 to zero revenue makes absolutely no difference (I just tried)\n\nBut my concern is more about the potential bias, or skew these sessions introduce in the model. Here is the distribution of predictions made on a [negative downsampled][2] model \n\n![enter image description here][3]\n\nThe actual performance of such a model is awful when measured on the CV and LB, but visually there is something going on...\n\n\n  [1]: https://i.imgur.com/Q4bXgoL.png\n  [2]:https://www.kaggle.com/xavierbourretsicotte/lgbm-error-analysis-negative-downsampling\n  [3]: https://i.imgur.com/jTjM9rW.png"
  }
}