{
  "id": 69928,
  "title": "Why I got 650489 unique \"fullVisitorId\" in test data?",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/69928",
  "author_name": "Rachel",
  "post_date": "2018-10-29T02:18:05.066000",
  "votes": 0,
  "comment_count": 6,
  "views": 0,
  "content": "<p>I got 804684 rows in test data. After the code df.fullVisitorId.nunique(), I got 650489 unique fullVisitorId. But the submission should contain 617242 unique fullVisitorId. I don't know why.</p>",
  "messages": [
    {
      "id": 411980,
      "postDate": "2018-10-29T10:42:46.490Z",
      "content": "<p>I had the same issue. I recreated fullvisitorid from sessionid before aggregating and it was resolved.</p>\n\n<p>test_df['fullVisitorId'] = test_df['sessionId'].str.split('_',expand=True)\ntrain_df['fullVisitorId'] = train_df['sessionId'].str.split('_',expand=True)</p>",
      "rawMarkdown": "I had the same issue. I recreated fullvisitorid from sessionid before aggregating and it was resolved.\n\ntest_df['fullVisitorId'] = test_df['sessionId'].str.split('_',expand=True)\ntrain_df['fullVisitorId'] = train_df['sessionId'].str.split('_',expand=True)\n",
      "replies": [
        {
          "id": 412690,
          "postDate": "2018-10-30T16:00:06.237Z",
          "content": "<p>Thank you! \nColin Pearse's method can also solve this issue. I think it is a simple way. If you would like, you could try it.</p>",
          "rawMarkdown": "Thank you! \nColin Pearse's method can also solve this issue. I think it is a simple way. If you would like, you could try it.",
          "votes": 1
        }
      ]
    },
    {
      "id": 411860,
      "postDate": "2018-10-29T05:54:17.387Z",
      "content": "<p>There are leading zeros in fullVisitorId which are stripped by default. Forcing read_csv to assign string will keep the leading zeros and the number of unique visitors in test.csv will be 617242. (I force visitId too for good measure). IE. <code>test = pd.read_csv(\"../input/test.csv\", dtype={'fullVisitorId': 'str', 'visitId': 'str'})</code></p>",
      "rawMarkdown": "There are leading zeros in fullVisitorId which are stripped by default. Forcing read_csv to assign string will keep the leading zeros and the number of unique visitors in test.csv will be 617242. (I force visitId too for good measure). IE. `test = pd.read_csv(\"../input/test.csv\", dtype={'fullVisitorId': 'str', 'visitId': 'str'})`",
      "replies": [
        {
          "id": 412687,
          "postDate": "2018-10-30T15:58:02.820Z",
          "content": "<p>Thank you!</p>",
          "rawMarkdown": "Thank you!"
        }
      ]
    },
    {
      "id": 411780,
      "postDate": "2018-10-29T02:18:05.067Z",
      "content": "<p>I got 804684 rows in test data. After the code df.fullVisitorId.nunique(), I got 650489 unique fullVisitorId. But the submission should contain 617242 unique fullVisitorId. I don't know why.</p>",
      "rawMarkdown": "I got 804684 rows in test data. After the code df.fullVisitorId.nunique(), I got 650489 unique fullVisitorId. But the submission should contain 617242 unique fullVisitorId. I don't know why."
    },
    {
      "id": 411799,
      "postDate": "2018-10-29T03:30:48.870Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 411795,
      "postDate": "2018-10-29T03:20:37.300Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 411980,
      "author_name": "Jeff",
      "author_url": "",
      "post_date": "2018-10-29T10:42:46.490000",
      "content": "<p>I had the same issue. I recreated fullvisitorid from sessionid before aggregating and it was resolved.</p>\n\n<p>test_df['fullVisitorId'] = test_df['sessionId'].str.split('_',expand=True)\ntrain_df['fullVisitorId'] = train_df['sessionId'].str.split('_',expand=True)</p>",
      "votes": 0,
      "replies": [
        {
          "id": 412690,
          "author_name": "Rachel",
          "author_url": "",
          "post_date": "2018-10-30T16:00:06.237000",
          "content": "<p>Thank you! \nColin Pearse's method can also solve this issue. I think it is a simple way. If you would like, you could try it.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 411860,
      "author_name": "Colin Pearse",
      "author_url": "",
      "post_date": "2018-10-29T05:54:17.387000",
      "content": "<p>There are leading zeros in fullVisitorId which are stripped by default. Forcing read_csv to assign string will keep the leading zeros and the number of unique visitors in test.csv will be 617242. (I force visitId too for good measure). IE. <code>test = pd.read_csv(\"../input/test.csv\", dtype={'fullVisitorId': 'str', 'visitId': 'str'})</code></p>",
      "votes": 0,
      "replies": [
        {
          "id": 412687,
          "author_name": "Rachel",
          "author_url": "",
          "post_date": "2018-10-30T15:58:02.820000",
          "content": "<p>Thank you!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 411799,
      "author_name": "",
      "author_url": "",
      "post_date": "2018-10-29T03:30:48.870000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 411795,
      "author_name": "",
      "author_url": "",
      "post_date": "2018-10-29T03:20:37.300000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "411980": "I had the same issue. I recreated fullvisitorid from sessionid before aggregating and it was resolved.\n\ntest_df['fullVisitorId'] = test_df['sessionId'].str.split('_',expand=True)\ntrain_df['fullVisitorId'] = train_df['sessionId'].str.split('_',expand=True)\n",
    "411860": "There are leading zeros in fullVisitorId which are stripped by default. Forcing read_csv to assign string will keep the leading zeros and the number of unique visitors in test.csv will be 617242. (I force visitId too for good measure). IE. `test = pd.read_csv(\"../input/test.csv\", dtype={'fullVisitorId': 'str', 'visitId': 'str'})`",
    "411780": "I got 804684 rows in test data. After the code df.fullVisitorId.nunique(), I got 650489 unique fullVisitorId. But the submission should contain 617242 unique fullVisitorId. I don't know why.",
    "411799": "",
    "411795": ""
  }
}