{
  "id": 71025,
  "title": "Test set contains the target",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/71025",
  "author_name": "",
  "post_date": "2018-11-09T11:08:59.284487600Z",
  "votes": 12,
  "comment_count": 15,
  "views": 0,
  "content": "<p>The new test set (test_v2) contains the target. I confirmed this by submitting the solution and getting a score of 0. Could anyone explain this ?</p>",
  "messages": [
    {
      "id": "418140",
      "postDate": "11/09/2018 11:08:59",
      "content": "<p>The new test set (test_v2) contains the target. I confirmed this by submitting the solution and getting a score of 0. Could anyone explain this ?</p>",
      "rawMarkdown": "The new test set (test_v2) contains the target. I confirmed this by submitting the solution and getting a score of 0. Could anyone explain this ?",
      "votes": null
    },
    {
      "id": "418148",
      "postDate": "11/09/2018 11:20:57",
      "content": "<p>Hello Marwen</p>\n\n<p>PublicLB = test_v2 target\nProbably, in this competition Public LB is completely unreliable</p>",
      "rawMarkdown": "Hello Marwen\n\nPublicLB = test_v2 target\nProbably, in this competition Public LB is completely unreliable",
      "votes": null
    },
    {
      "id": "418163",
      "postDate": "11/09/2018 11:47:29",
      "content": "<p>Bro，you are so brave! </p>",
      "rawMarkdown": "Bro，you are so brave!",
      "votes": null
    },
    {
      "id": "418174",
      "postDate": "11/09/2018 12:01:14",
      "content": "<p>:( How to check actual positions then ?</p>",
      "rawMarkdown": ":( How to check actual positions then ?",
      "votes": null
    },
    {
      "id": "418193",
      "postDate": "11/09/2018 13:09:13",
      "content": "<p>The test_set is what is used to determine the leaderboard... </p>\n\n<p>Hope you all have strong CV strategies !!</p>",
      "rawMarkdown": "The test_set is what is used to determine the leaderboard... \n\nHope you all have strong CV strategies !!",
      "votes": null
    },
    {
      "id": "418231",
      "postDate": "11/09/2018 14:09:33",
      "content": "<p>I want to know what it means。。。。</p>",
      "rawMarkdown": "I want to know what it means。。。。",
      "votes": null
    },
    {
      "id": "418307",
      "postDate": "11/09/2018 16:16:07",
      "content": "<p>test_v2 used to validate time predict.\nReal predict is 2018.12.1-2019.1.31.\nit seems Private Leaderboard score need a long time to see. \ncan have 0 score in Public LB, and pk Private Leaderboard.</p>",
      "rawMarkdown": "test_v2 used to validate time predict.\nReal predict is 2018.12.1-2019.1.31.\nit seems Private Leaderboard score need a long time to see. \ncan have 0 score in Public LB, and pk Private Leaderboard.",
      "votes": null
    },
    {
      "id": "418364",
      "postDate": "11/09/2018 18:23:58",
      "content": "<p>after January 31st 2019...</p>",
      "rawMarkdown": "after January 31st 2019...",
      "votes": null
    },
    {
      "id": "418510",
      "postDate": "11/10/2018 02:03:41",
      "content": "<p>my god</p>",
      "rawMarkdown": "my god",
      "votes": null
    },
    {
      "id": "418571",
      "postDate": "11/10/2018 06:50:42",
      "content": "<p>Will not changes in game rules affect the purchase of google goods? haha</p>",
      "rawMarkdown": "Will not changes in game rules affect the purchase of google goods? haha",
      "votes": null
    },
    {
      "id": "418685",
      "postDate": "11/10/2018 12:21:01",
      "content": "<p>great</p>",
      "rawMarkdown": "great",
      "votes": null
    },
    {
      "id": "418726",
      "postDate": "11/10/2018 13:47:33",
      "content": "<p>After the leak detection by Jerome Vallet, this contest will have two phases. In the first phase the test set contains the target intentionally (it would be like a validation set). The challenge of the second phase is to predict the target in a future time frame.</p>",
      "rawMarkdown": "After the leak detection by Jerome Vallet, this contest will have two phases. In the first phase the test set contains the target intentionally (it would be like a validation set). The challenge of the second phase is to predict the target in a future time frame.",
      "votes": null
    },
    {
      "id": "418776",
      "postDate": "11/10/2018 15:38:52",
      "content": "<p>what happened？ I can't submit！</p>",
      "rawMarkdown": "what happened？ I can't submit！",
      "votes": null
    },
    {
      "id": "419010",
      "postDate": "11/11/2018 04:39:41",
      "content": "<p>how can you get score 0 from test_v2.\nI use follow,but always get 1.7768  .</p><p>\nDoes something wrong in the code, can help?\n</p><p></p>\n\n<pre>import numpy as np\nimport pandas as pd\nimport json\nfrom pandas.io.json import json_normalize\nPATH=\"../input/ga-customer-revenue-prediction/\"\ntest_df=pd.read_csv(PATH+\"test_v2.csv\", converters={'totals': json.loads}, dtype={'fullVisitorId': 'str'},usecols=['fullVisitorId','totals'])\ntest_df.head()\n</pre>\n\n<pre>json_col_df = json_normalize(test_df['totals'])\njson_col_df.columns = [f\"totals.{sub_col}\" for sub_col in json_col_df.columns]\ntest_df = test_df.drop('totals', axis=1).merge(json_col_df, right_index=True, left_index=True)\ntest_df.head()\n</pre>\n\n<pre>test_df.shape\n</pre>\n\n<pre>testsub=test_df[['fullVisitorId','totals.transactionRevenue']].fillna(0).groupby('fullVisitorId').sum()\ntestsub.head()\n</pre>\n\n<pre>testsub['fullVisitorId']=testsub.index\ntestsub['PredictedLogRevenue']=testsub['totals.transactionRevenue'].apply(np.log1p)\ntestsub[['fullVisitorId','PredictedLogRevenue']].head()\n</pre>\n\n<pre>testsub[['fullVisitorId','PredictedLogRevenue']].fillna(0).to_csv(\"testsub.csv\",index=False)\n</pre>",
      "rawMarkdown": "how can you get score 0 from test_v2.\nI use follow,but always get 1.7768  .<p>\nDoes something wrong in the code, can help?\n</p>\n<pre>import numpy as np\nimport pandas as pd\nimport json\nfrom pandas.io.json import json_normalize\nPATH=\"../input/ga-customer-revenue-prediction/\"\ntest_df=pd.read_csv(PATH+\"test_v2.csv\", converters={'totals': json.loads}, dtype={'fullVisitorId': 'str'},usecols=['fullVisitorId','totals'])\ntest_df.head()\n</pre>\n<pre>json_col_df = json_normalize(test_df['totals'])\njson_col_df.columns = [f\"totals.{sub_col}\" for sub_col in json_col_df.columns]\ntest_df = test_df.drop('totals', axis=1).merge(json_col_df, right_index=True, left_index=True)\ntest_df.head()\n</pre>\n<pre>test_df.shape\n</pre>\n<pre>testsub=test_df[['fullVisitorId','totals.transactionRevenue']].fillna(0).groupby('fullVisitorId').sum()\ntestsub.head()\n</pre>\n<pre>testsub['fullVisitorId']=testsub.index\ntestsub['PredictedLogRevenue']=testsub['totals.transactionRevenue'].apply(np.log1p)\ntestsub[['fullVisitorId','PredictedLogRevenue']].head()\n</pre>\n<pre>testsub[['fullVisitorId','PredictedLogRevenue']].fillna(0).to_csv(\"testsub.csv\",index=False)\n</pre>",
      "votes": null
    },
    {
      "id": "440358",
      "postDate": "12/17/2018 12:54:08",
      "content": "<p>hi, does the test set contain the target ？I can't find it. Can you help me?</p>",
      "rawMarkdown": "hi, does the test set contain the target ？I can't find it. Can you help me?",
      "votes": null
    },
    {
      "id": "440362",
      "postDate": "12/17/2018 13:00:29",
      "content": "<p>the test data  contains:\n'totals.hits',\n 'totals.pageviews',\n 'totals.sessionQualityDim',\n 'totals.timeOnSite',\n 'totals.visits'.</p>\n\n<p>But there isn‘t  'totals.transactionRevenue'. \nIs my data different with you? \nif so，can you provide your data? \nI need it so much. <br>\nThanks a lot  !!!!!!</p>",
      "rawMarkdown": "the test data  contains:\n'totals.hits',\n 'totals.pageviews',\n 'totals.sessionQualityDim',\n 'totals.timeOnSite',\n 'totals.visits'.\n\nBut there isn‘t  'totals.transactionRevenue'. \nIs my data different with you? \nif so，can you provide your data? \nI need it so much.          \nThanks a lot  !!!!!!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 418148,
      "author_name": "atsunorifujita",
      "author_url": "",
      "post_date": "11/09/2018 11:20:57",
      "content": "<p>Hello Marwen</p>\n\n<p>PublicLB = test_v2 target\nProbably, in this competition Public LB is completely unreliable</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 418163,
      "author_name": "leaveslei",
      "author_url": "",
      "post_date": "11/09/2018 11:47:29",
      "content": "<p>Bro，you are so brave! </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 418174,
      "author_name": "hiddenbrains",
      "author_url": "",
      "post_date": "11/09/2018 12:01:14",
      "content": "<p>:( How to check actual positions then ?</p>",
      "votes": null,
      "replies": [
        {
          "id": 418364,
          "author_name": "jingobox",
          "author_url": "",
          "post_date": "11/09/2018 18:23:58",
          "content": "<p>after January 31st 2019...</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 418193,
      "author_name": "xtelima",
      "author_url": "",
      "post_date": "11/09/2018 13:09:13",
      "content": "<p>The test_set is what is used to determine the leaderboard... </p>\n\n<p>Hope you all have strong CV strategies !!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 418231,
      "author_name": "z16130110062",
      "author_url": "",
      "post_date": "11/09/2018 14:09:33",
      "content": "<p>I want to know what it means。。。。</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 418307,
      "author_name": "zjp2origin",
      "author_url": "",
      "post_date": "11/09/2018 16:16:07",
      "content": "<p>test_v2 used to validate time predict.\nReal predict is 2018.12.1-2019.1.31.\nit seems Private Leaderboard score need a long time to see. \ncan have 0 score in Public LB, and pk Private Leaderboard.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 418510,
      "author_name": "yangddd",
      "author_url": "",
      "post_date": "11/10/2018 02:03:41",
      "content": "<p>my god</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 418571,
      "author_name": "dcahn12",
      "author_url": "",
      "post_date": "11/10/2018 06:50:42",
      "content": "<p>Will not changes in game rules affect the purchase of google goods? haha</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 418685,
      "author_name": "silenozlt",
      "author_url": "",
      "post_date": "11/10/2018 12:21:01",
      "content": "<p>great</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 418726,
      "author_name": "paulorzp",
      "author_url": "",
      "post_date": "11/10/2018 13:47:33",
      "content": "<p>After the leak detection by Jerome Vallet, this contest will have two phases. In the first phase the test set contains the target intentionally (it would be like a validation set). The challenge of the second phase is to predict the target in a future time frame.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 418776,
      "author_name": "dinglubjut25079",
      "author_url": "",
      "post_date": "11/10/2018 15:38:52",
      "content": "<p>what happened？ I can't submit！</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 419010,
      "author_name": "zjp2origin",
      "author_url": "",
      "post_date": "11/11/2018 04:39:41",
      "content": "<p>how can you get score 0 from test_v2.\nI use follow,but always get 1.7768  .</p><p>\nDoes something wrong in the code, can help?\n</p><p></p>\n\n<pre>import numpy as np\nimport pandas as pd\nimport json\nfrom pandas.io.json import json_normalize\nPATH=\"../input/ga-customer-revenue-prediction/\"\ntest_df=pd.read_csv(PATH+\"test_v2.csv\", converters={'totals': json.loads}, dtype={'fullVisitorId': 'str'},usecols=['fullVisitorId','totals'])\ntest_df.head()\n</pre>\n\n<pre>json_col_df = json_normalize(test_df['totals'])\njson_col_df.columns = [f\"totals.{sub_col}\" for sub_col in json_col_df.columns]\ntest_df = test_df.drop('totals', axis=1).merge(json_col_df, right_index=True, left_index=True)\ntest_df.head()\n</pre>\n\n<pre>test_df.shape\n</pre>\n\n<pre>testsub=test_df[['fullVisitorId','totals.transactionRevenue']].fillna(0).groupby('fullVisitorId').sum()\ntestsub.head()\n</pre>\n\n<pre>testsub['fullVisitorId']=testsub.index\ntestsub['PredictedLogRevenue']=testsub['totals.transactionRevenue'].apply(np.log1p)\ntestsub[['fullVisitorId','PredictedLogRevenue']].head()\n</pre>\n\n<pre>testsub[['fullVisitorId','PredictedLogRevenue']].fillna(0).to_csv(\"testsub.csv\",index=False)\n</pre>",
      "votes": null,
      "replies": [
        {
          "id": 440358,
          "author_name": "a362566282",
          "author_url": "",
          "post_date": "12/17/2018 12:54:08",
          "content": "<p>hi, does the test set contain the target ？I can't find it. Can you help me?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 440362,
          "author_name": "a362566282",
          "author_url": "",
          "post_date": "12/17/2018 13:00:29",
          "content": "<p>the test data  contains:\n'totals.hits',\n 'totals.pageviews',\n 'totals.sessionQualityDim',\n 'totals.timeOnSite',\n 'totals.visits'.</p>\n\n<p>But there isn‘t  'totals.transactionRevenue'. \nIs my data different with you? \nif so，can you provide your data? \nI need it so much. <br>\nThanks a lot  !!!!!!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "418140": "The new test set (test_v2) contains the target. I confirmed this by submitting the solution and getting a score of 0. Could anyone explain this ?",
    "418148": "Hello Marwen\n\nPublicLB = test_v2 target\nProbably, in this competition Public LB is completely unreliable",
    "418163": "Bro，you are so brave!",
    "418174": ":( How to check actual positions then ?",
    "418193": "The test_set is what is used to determine the leaderboard... \n\nHope you all have strong CV strategies !!",
    "418231": "I want to know what it means。。。。",
    "418307": "test_v2 used to validate time predict.\nReal predict is 2018.12.1-2019.1.31.\nit seems Private Leaderboard score need a long time to see. \ncan have 0 score in Public LB, and pk Private Leaderboard.",
    "418364": "after January 31st 2019...",
    "418510": "my god",
    "418571": "Will not changes in game rules affect the purchase of google goods? haha",
    "418685": "great",
    "418726": "After the leak detection by Jerome Vallet, this contest will have two phases. In the first phase the test set contains the target intentionally (it would be like a validation set). The challenge of the second phase is to predict the target in a future time frame.",
    "418776": "what happened？ I can't submit！",
    "419010": "how can you get score 0 from test_v2.\nI use follow,but always get 1.7768  .<p>\nDoes something wrong in the code, can help?\n</p>\n<pre>import numpy as np\nimport pandas as pd\nimport json\nfrom pandas.io.json import json_normalize\nPATH=\"../input/ga-customer-revenue-prediction/\"\ntest_df=pd.read_csv(PATH+\"test_v2.csv\", converters={'totals': json.loads}, dtype={'fullVisitorId': 'str'},usecols=['fullVisitorId','totals'])\ntest_df.head()\n</pre>\n<pre>json_col_df = json_normalize(test_df['totals'])\njson_col_df.columns = [f\"totals.{sub_col}\" for sub_col in json_col_df.columns]\ntest_df = test_df.drop('totals', axis=1).merge(json_col_df, right_index=True, left_index=True)\ntest_df.head()\n</pre>\n<pre>test_df.shape\n</pre>\n<pre>testsub=test_df[['fullVisitorId','totals.transactionRevenue']].fillna(0).groupby('fullVisitorId').sum()\ntestsub.head()\n</pre>\n<pre>testsub['fullVisitorId']=testsub.index\ntestsub['PredictedLogRevenue']=testsub['totals.transactionRevenue'].apply(np.log1p)\ntestsub[['fullVisitorId','PredictedLogRevenue']].head()\n</pre>\n<pre>testsub[['fullVisitorId','PredictedLogRevenue']].fillna(0).to_csv(\"testsub.csv\",index=False)\n</pre>",
    "440358": "hi, does the test set contain the target ？I can't find it. Can you help me?",
    "440362": "the test data  contains:\n'totals.hits',\n 'totals.pageviews',\n 'totals.sessionQualityDim',\n 'totals.timeOnSite',\n 'totals.visits'.\n\nBut there isn‘t  'totals.transactionRevenue'. \nIs my data different with you? \nif so，can you provide your data? \nI need it so much.          \nThanks a lot  !!!!!!"
  },
  "source": "meta"
}