{
  "id": 66697,
  "title": "More than 10M single user revenue in test set",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/66697",
  "author_name": "",
  "post_date": "2018-09-24T12:49:27.405649800Z",
  "votes": 30,
  "comment_count": 11,
  "views": 0,
  "content": "<p>Largest revenue in train set is around np.exp(25)//1000000=72K $</p>\n\n<p>Largest revenue in test set is larger than np.exp(30)//1000000=10M $</p>\n\n<p>I tested this using np.clip(pred, 0, 30) on my predictions and I realized that my score is worsened.</p>",
  "messages": [
    {
      "id": "392829",
      "postDate": "09/24/2018 12:49:27",
      "content": "<p>Largest revenue in train set is around np.exp(25)//1000000=72K $</p>\n\n<p>Largest revenue in test set is larger than np.exp(30)//1000000=10M $</p>\n\n<p>I tested this using np.clip(pred, 0, 30) on my predictions and I realized that my score is worsened.</p>",
      "rawMarkdown": "Largest revenue in train set is around np.exp(25)//1000000=72K $\n\nLargest revenue in test set is larger than np.exp(30)//1000000=10M $\n\nI tested this using np.clip(pred, 0, 30) on my predictions and I realized that my score is worsened.",
      "votes": null
    },
    {
      "id": "392967",
      "postDate": "09/24/2018 16:31:19",
      "content": "<p>Setting a ceiling of 35 also makes the LB score substantially worse, meaning that there were dozens of customers who spent more than $2 billion each on Google swag last year. LOL. Clearly there are issues with the test set labels. I'm out till Kaggle addresses the problem.</p>",
      "rawMarkdown": "Setting a ceiling of 35 also makes the LB score substantially worse, meaning that there were dozens of customers who spent more than $2 billion each on Google swag last year. LOL. Clearly there are issues with the test set labels. I'm out till Kaggle addresses the problem.",
      "votes": null
    },
    {
      "id": "393026",
      "postDate": "09/24/2018 18:07:54",
      "content": "<p><a href=\"/philculliton\">@philculliton</a>, can you please look at this issue, and especially check there is no overflow problem when calculating public leaderboard score?</p>",
      "rawMarkdown": "philculliton, can you please look at this issue, and especially check there is no overflow problem when calculating public leaderboard score?",
      "votes": null
    },
    {
      "id": "393111",
      "postDate": "09/24/2018 20:43:08",
      "content": "<p>Great finding, thanks a lot.</p>",
      "rawMarkdown": "Great finding, thanks a lot.",
      "votes": null
    },
    {
      "id": "393123",
      "postDate": "09/24/2018 21:11:24",
      "content": "<p>I see \"Our workers are spending a little extra time on your submission. Grab a coffee and refresh later to see your score.\", But I did not submit anything for few days. It looks like we will have public leaderboard shake-up :)</p>",
      "rawMarkdown": "I see \"Our workers are spending a little extra time on your submission. Grab a coffee and refresh later to see your score.\", But I did not submit anything for few days. It looks like we will have public leaderboard shake-up :)",
      "votes": null
    },
    {
      "id": "393124",
      "postDate": "09/24/2018 21:15:11",
      "content": "<p>log(sum) might beat sum(log).  :)</p>",
      "rawMarkdown": "log(sum) might beat sum(log).  :)",
      "votes": null
    },
    {
      "id": "393129",
      "postDate": "09/24/2018 21:24:10",
      "content": "<p>Wow looks like you single handedly broke the whole leaderboard! Thanks for sharing.</p>",
      "rawMarkdown": "Wow looks like you single handedly broke the whole leaderboard! Thanks for sharing.",
      "votes": null
    },
    {
      "id": "393134",
      "postDate": "09/24/2018 21:36:11",
      "content": "<p>The fix didn't stem solely from this thread (I've been working on it for a bit), but I appreciate the help in tracking down the issue.</p>",
      "rawMarkdown": "The fix didn't stem solely from this thread (I've been working on it for a bit), but I appreciate the help in tracking down the issue.",
      "votes": null
    },
    {
      "id": "393192",
      "postDate": "09/25/2018 00:33:40",
      "content": "<p>Ha! *fixed</p>",
      "rawMarkdown": "Ha! *fixed",
      "votes": null
    },
    {
      "id": "393304",
      "postDate": "09/25/2018 06:17:15",
      "content": "<p>I won't forget your constant belief about 'the mistake' and <a href=\"https://www.kaggle.com/c/ga-customer-revenue-prediction/discussion/65806#388104\">this comment</a>. Thank you for this clear test Ahmet!</p>",
      "rawMarkdown": "I won't forget your constant belief about 'the mistake' and [this comment][1]. Thank you for this clear test Ahmet!\n\n\n  [1]: https://www.kaggle.com/c/ga-customer-revenue-prediction/discussion/65806#388104",
      "votes": null
    },
    {
      "id": "393312",
      "postDate": "09/25/2018 06:44:42",
      "content": "<p>@Phil, How long ago did you realize there was a problem?</p>",
      "rawMarkdown": "Phil, How long ago did you realize there was a problem?",
      "votes": null
    },
    {
      "id": "393580",
      "postDate": "09/25/2018 16:12:14",
      "content": "<p>That's enough to buy over 15,000 pixel 2s.</p>",
      "rawMarkdown": "That's enough to buy over 15,000 pixel 2s.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 392967,
      "author_name": "psilogram",
      "author_url": "",
      "post_date": "09/24/2018 16:31:19",
      "content": "<p>Setting a ceiling of 35 also makes the LB score substantially worse, meaning that there were dozens of customers who spent more than $2 billion each on Google swag last year. LOL. Clearly there are issues with the test set labels. I'm out till Kaggle addresses the problem.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 393026,
      "author_name": "alexfir",
      "author_url": "",
      "post_date": "09/24/2018 18:07:54",
      "content": "<p><a href=\"/philculliton\">@philculliton</a>, can you please look at this issue, and especially check there is no overflow problem when calculating public leaderboard score?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 393111,
      "author_name": "mayer79",
      "author_url": "",
      "post_date": "09/24/2018 20:43:08",
      "content": "<p>Great finding, thanks a lot.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 393123,
      "author_name": "alexfir",
      "author_url": "",
      "post_date": "09/24/2018 21:11:24",
      "content": "<p>I see \"Our workers are spending a little extra time on your submission. Grab a coffee and refresh later to see your score.\", But I did not submit anything for few days. It looks like we will have public leaderboard shake-up :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 393124,
      "author_name": "lishubin",
      "author_url": "",
      "post_date": "09/24/2018 21:15:11",
      "content": "<p>log(sum) might beat sum(log).  :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 393129,
      "author_name": "sdoria",
      "author_url": "",
      "post_date": "09/24/2018 21:24:10",
      "content": "<p>Wow looks like you single handedly broke the whole leaderboard! Thanks for sharing.</p>",
      "votes": null,
      "replies": [
        {
          "id": 393192,
          "author_name": "nnnnick",
          "author_url": "",
          "post_date": "09/25/2018 00:33:40",
          "content": "<p>Ha! *fixed</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 393134,
      "author_name": "philculliton",
      "author_url": "",
      "post_date": "09/24/2018 21:36:11",
      "content": "<p>The fix didn't stem solely from this thread (I've been working on it for a bit), but I appreciate the help in tracking down the issue.</p>",
      "votes": null,
      "replies": [
        {
          "id": 393312,
          "author_name": "psilogram",
          "author_url": "",
          "post_date": "09/25/2018 06:44:42",
          "content": "<p>@Phil, How long ago did you realize there was a problem?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 393304,
      "author_name": "fatihozturk",
      "author_url": "",
      "post_date": "09/25/2018 06:17:15",
      "content": "<p>I won't forget your constant belief about 'the mistake' and <a href=\"https://www.kaggle.com/c/ga-customer-revenue-prediction/discussion/65806#388104\">this comment</a>. Thank you for this clear test Ahmet!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 393580,
      "author_name": "afgonczol",
      "author_url": "",
      "post_date": "09/25/2018 16:12:14",
      "content": "<p>That's enough to buy over 15,000 pixel 2s.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "392829": "Largest revenue in train set is around np.exp(25)//1000000=72K $\n\nLargest revenue in test set is larger than np.exp(30)//1000000=10M $\n\nI tested this using np.clip(pred, 0, 30) on my predictions and I realized that my score is worsened.",
    "392967": "Setting a ceiling of 35 also makes the LB score substantially worse, meaning that there were dozens of customers who spent more than $2 billion each on Google swag last year. LOL. Clearly there are issues with the test set labels. I'm out till Kaggle addresses the problem.",
    "393026": "philculliton, can you please look at this issue, and especially check there is no overflow problem when calculating public leaderboard score?",
    "393111": "Great finding, thanks a lot.",
    "393123": "I see \"Our workers are spending a little extra time on your submission. Grab a coffee and refresh later to see your score.\", But I did not submit anything for few days. It looks like we will have public leaderboard shake-up :)",
    "393124": "log(sum) might beat sum(log).  :)",
    "393129": "Wow looks like you single handedly broke the whole leaderboard! Thanks for sharing.",
    "393134": "The fix didn't stem solely from this thread (I've been working on it for a bit), but I appreciate the help in tracking down the issue.",
    "393192": "Ha! *fixed",
    "393304": "I won't forget your constant belief about 'the mistake' and [this comment][1]. Thank you for this clear test Ahmet!\n\n\n  [1]: https://www.kaggle.com/c/ga-customer-revenue-prediction/discussion/65806#388104",
    "393312": "Phil, How long ago did you realize there was a problem?",
    "393580": "That's enough to buy over 15,000 pixel 2s."
  },
  "source": "meta"
}