{
  "id": 70236,
  "title": "A question on new evaluation rules",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/70236",
  "author_name": "",
  "post_date": "2018-11-01T06:53:25.331481100Z",
  "votes": 1,
  "comment_count": 6,
  "views": 0,
  "content": "<p>1 question needs clarifying: Let's take an hypothetical case: user A and B exist in training set, and I predict they will both generate $10 revenue.</p>\n\n<p>In test period, it turns out that A does not even return, B returns to website but does not buy anything. How will the score be calculated?</p>",
  "messages": [
    {
      "id": "413583",
      "postDate": "11/01/2018 06:53:25",
      "content": "<p>1 question needs clarifying: Let's take an hypothetical case: user A and B exist in training set, and I predict they will both generate $10 revenue.</p>\n\n<p>In test period, it turns out that A does not even return, B returns to website but does not buy anything. How will the score be calculated?</p>",
      "rawMarkdown": "1 question needs clarifying: Let's take an hypothetical case: user A and B exist in training set, and I predict they will both generate $10 revenue.\n\nIn test period, it turns out that A does not even return, B returns to website but does not buy anything. How will the score be calculated?",
      "votes": null
    },
    {
      "id": "413593",
      "postDate": "11/01/2018 07:07:03",
      "content": "<p>From my understanding:\nThe test data that you are trying to predict is:</p>\n\n<p>User A: 0\nUser B: 0</p>\n\n<p>As we are trying to predict purchases, and there is no wording on visits during the test period.</p>\n\n<p>so to your question, from my understanding, i believe the score will be the same for the 2 users, because the actual labels for both of them is 0.</p>",
      "rawMarkdown": "From my understanding:\nThe test data that you are trying to predict is:\n\nUser A: 0\nUser B: 0\n\nAs we are trying to predict purchases, and there is no wording on visits during the test period.\n\nso to your question, from my understanding, i believe the score will be the same for the 2 users, because the actual labels for both of them is 0.",
      "votes": null
    },
    {
      "id": "413605",
      "postDate": "11/01/2018 07:44:29",
      "content": "<p>This makes sense. In this case, we have to be very precise on identifying who will purchase - since the conversion rate is so low, a few false positives (i.e. purchases that do not happen) will ruin the score.</p>",
      "rawMarkdown": "This makes sense. In this case, we have to be very precise on identifying who will purchase - since the conversion rate is so low, a few false positives (i.e. purchases that do not happen) will ruin the score.",
      "votes": null
    },
    {
      "id": "414248",
      "postDate": "11/02/2018 12:06:56",
      "content": "<p>Emm .. That is mean that the we should have a transaction history of every user in the test set ?? </p>",
      "rawMarkdown": "Emm .. That is mean that the we should have a transaction history of every user in the test set ??",
      "votes": null
    },
    {
      "id": "414274",
      "postDate": "11/02/2018 13:17:59",
      "content": "<p>Nope, because the test set is in future, there is no way to know at submission time.</p>",
      "rawMarkdown": "Nope, because the test set is in future, there is no way to know at submission time.",
      "votes": null
    },
    {
      "id": "414949",
      "postDate": "11/04/2018 00:28:02",
      "content": "<p>Emm .. but how we predict the revenue gained from a person without having any information about him ! </p>",
      "rawMarkdown": "Emm .. but how we predict the revenue gained from a person without having any information about him !",
      "votes": null
    },
    {
      "id": "415678",
      "postDate": "11/05/2018 13:30:38",
      "content": "<p>I guess, for brandy new customer during real test period, kaggle won't count them, as none of submission mentioned them. So it's fair for everybody.</p>\n\n<p>I am curious about how to calc those customers whom existed in test_V2 but don't return during real test period. Will Kaggle ignore the predict value for all submission? It should not be fair enough.</p>",
      "rawMarkdown": "I guess, for brandy new customer during real test period, kaggle won't count them, as none of submission mentioned them. So it's fair for everybody.\n\nI am curious about how to calc those customers whom existed in test_V2 but don't return during real test period. Will Kaggle ignore the predict value for all submission? It should not be fair enough.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 413593,
      "author_name": "tpthegreat",
      "author_url": "",
      "post_date": "11/01/2018 07:07:03",
      "content": "<p>From my understanding:\nThe test data that you are trying to predict is:</p>\n\n<p>User A: 0\nUser B: 0</p>\n\n<p>As we are trying to predict purchases, and there is no wording on visits during the test period.</p>\n\n<p>so to your question, from my understanding, i believe the score will be the same for the 2 users, because the actual labels for both of them is 0.</p>",
      "votes": null,
      "replies": [
        {
          "id": 413605,
          "author_name": "comsaint",
          "author_url": "",
          "post_date": "11/01/2018 07:44:29",
          "content": "<p>This makes sense. In this case, we have to be very precise on identifying who will purchase - since the conversion rate is so low, a few false positives (i.e. purchases that do not happen) will ruin the score.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 414248,
          "author_name": "ayoub89",
          "author_url": "",
          "post_date": "11/02/2018 12:06:56",
          "content": "<p>Emm .. That is mean that the we should have a transaction history of every user in the test set ?? </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 414274,
          "author_name": "comsaint",
          "author_url": "",
          "post_date": "11/02/2018 13:17:59",
          "content": "<p>Nope, because the test set is in future, there is no way to know at submission time.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 414949,
          "author_name": "ayoub89",
          "author_url": "",
          "post_date": "11/04/2018 00:28:02",
          "content": "<p>Emm .. but how we predict the revenue gained from a person without having any information about him ! </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 415678,
          "author_name": "elulue",
          "author_url": "",
          "post_date": "11/05/2018 13:30:38",
          "content": "<p>I guess, for brandy new customer during real test period, kaggle won't count them, as none of submission mentioned them. So it's fair for everybody.</p>\n\n<p>I am curious about how to calc those customers whom existed in test_V2 but don't return during real test period. Will Kaggle ignore the predict value for all submission? It should not be fair enough.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "413583": "1 question needs clarifying: Let's take an hypothetical case: user A and B exist in training set, and I predict they will both generate $10 revenue.\n\nIn test period, it turns out that A does not even return, B returns to website but does not buy anything. How will the score be calculated?",
    "413593": "From my understanding:\nThe test data that you are trying to predict is:\n\nUser A: 0\nUser B: 0\n\nAs we are trying to predict purchases, and there is no wording on visits during the test period.\n\nso to your question, from my understanding, i believe the score will be the same for the 2 users, because the actual labels for both of them is 0.",
    "413605": "This makes sense. In this case, we have to be very precise on identifying who will purchase - since the conversion rate is so low, a few false positives (i.e. purchases that do not happen) will ruin the score.",
    "414248": "Emm .. That is mean that the we should have a transaction history of every user in the test set ??",
    "414274": "Nope, because the test set is in future, there is no way to know at submission time.",
    "414949": "Emm .. but how we predict the revenue gained from a person without having any information about him !",
    "415678": "I guess, for brandy new customer during real test period, kaggle won't count them, as none of submission mentioned them. So it's fair for everybody.\n\nI am curious about how to calc those customers whom existed in test_V2 but don't return during real test period. Will Kaggle ignore the predict value for all submission? It should not be fair enough."
  },
  "source": "meta"
}