{
  "id": 12588,
  "title": "LB paradoxes",
  "url": "/competitions/avazu-ctr-prediction/discussion/12588",
  "author_name": "",
  "post_date": "2015-02-22T21:16:47.010Z",
  "votes": 3,
  "comment_count": 2,
  "views": 1547,
  "content": "<p>I found from the topic &quot;Q&amp;A&quot; that the split of the test data into two parts: public (20%) and private (80%) was conducted at random (uniformly).</p>\n<p>I would be very interested to know how you will explain significant&nbsp;difference between public and private scores despite huge sample sizes?</p>\n<p>Definitely, results of the Avazu/Click Contest were not consistent.<br>And, it is conclusion N1 - this problem is serious, but maybe overlooked.</p>\n<p>Particularly surprising is the fact that private scores are better compared to the public scores. From the point of view of the most basic statistical theory such phenomenon is totally impossible (assuming that the experiment was designed correctly).</p>\n<p><strong>Conclusion N2:</strong><br>I know that such an opinion maybe very unpleasant for some Kagglers, but the whole Avazu/Click LB appears to be invalid (in scientific sense, of course). It is the main problem here - cannot be ignored.</p>\n<p>You are welcomed to post here valid scientific grounds or reasons&nbsp;to reverse two above conclusions.</p>",
  "messages": [
    {
      "id": "64744",
      "postDate": "02/22/2015 21:16:47",
      "content": "<p>I found from the topic &quot;Q&amp;A&quot; that the split of the test data into two parts: public (20%) and private (80%) was conducted at random (uniformly).</p>\n<p>I would be very interested to know how you will explain significant&nbsp;difference between public and private scores despite huge sample sizes?</p>\n<p>Definitely, results of the Avazu/Click Contest were not consistent.<br>And, it is conclusion N1 - this problem is serious, but maybe overlooked.</p>\n<p>Particularly surprising is the fact that private scores are better compared to the public scores. From the point of view of the most basic statistical theory such phenomenon is totally impossible (assuming that the experiment was designed correctly).</p>\n<p><strong>Conclusion N2:</strong><br>I know that such an opinion maybe very unpleasant for some Kagglers, but the whole Avazu/Click LB appears to be invalid (in scientific sense, of course). It is the main problem here - cannot be ignored.</p>\n<p>You are welcomed to post here valid scientific grounds or reasons&nbsp;to reverse two above conclusions.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "64745",
      "postDate": "02/22/2015 22:25:07",
      "content": "<p>According to the leaderboard, you didn't enter this competition or you got removed (again) so not sure what your point is. Maybe you should explain why you were removed from the leaderboard rather than distracting attention away from that fact</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "77716",
      "postDate": "05/09/2015 16:48:44",
      "content": "<p>you may find answer on your question here:</p>\n<p>https://www.kaggle.com/c/restaurant-revenue-prediction/forums/t/14131/multiple-accounts</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 64745,
      "author_name": "domcastro",
      "author_url": "",
      "post_date": "02/22/2015 22:25:07",
      "content": "<p>According to the leaderboard, you didn't enter this competition or you got removed (again) so not sure what your point is. Maybe you should explain why you were removed from the leaderboard rather than distracting attention away from that fact</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 77716,
      "author_name": "",
      "author_url": "",
      "post_date": "05/09/2015 16:48:44",
      "content": "<p>you may find answer on your question here:</p>\n<p>https://www.kaggle.com/c/restaurant-revenue-prediction/forums/t/14131/multiple-accounts</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "64744": "",
    "64745": "",
    "77716": ""
  },
  "source": "meta"
}