{
  "id": 66488,
  "title": "test['fullVisitorId'].nunique() returning 650489 instead of 617242",
  "url": "/competitions/ga-customer-revenue-prediction/discussion/66488",
  "author_name": "",
  "post_date": "2018-09-22T02:46:07.367398600Z",
  "votes": 2,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hi, importing train.csv and test.csv by the <a href=\"https://www.kaggle.com/julian3833/1-quick-start-read-csv-and-flatten-json-fields/notebook\">Júlian Kernel</a> and importing it to work on my Spyder IDE. When i set the command <code>test['fullVisitorId'].nunique()</code> to see the number of unique values of 'fullVisitorId' on my test dataframe, it returns 650489, but it should return 617242, any thoughts?\n<em>PS: If I try this procedure on a kaggle's kernel it returns 617242 correctly</em></p>",
  "messages": [
    {
      "id": "391551",
      "postDate": "09/22/2018 02:46:07",
      "content": "<p>Hi, importing train.csv and test.csv by the <a href=\"https://www.kaggle.com/julian3833/1-quick-start-read-csv-and-flatten-json-fields/notebook\">Júlian Kernel</a> and importing it to work on my Spyder IDE. When i set the command <code>test['fullVisitorId'].nunique()</code> to see the number of unique values of 'fullVisitorId' on my test dataframe, it returns 650489, but it should return 617242, any thoughts?\n<em>PS: If I try this procedure on a kaggle's kernel it returns 617242 correctly</em></p>",
      "rawMarkdown": "Hi, importing train.csv and test.csv by the [Júlian Kernel][1] and importing it to work on my Spyder IDE. When i set the command `test['fullVisitorId'].nunique()` to see the number of unique values of 'fullVisitorId' on my test dataframe, it returns 650489, but it should return 617242, any thoughts?\n*PS: If I try this procedure on a kaggle's kernel it returns 617242 correctly*\n\n\n  [1]: https://www.kaggle.com/julian3833/1-quick-start-read-csv-and-flatten-json-fields/notebook",
      "votes": null
    },
    {
      "id": "391561",
      "postDate": "09/22/2018 03:06:44",
      "content": "<p>Have you checked that you have specified the fullVisitorId as String? I encountered the similar problem before, it was due to the datatype changes during file load/save.</p>",
      "rawMarkdown": "Have you checked that you have specified the fullVisitorId as String? I encountered the similar problem before, it was due to the datatype changes during file load/save.",
      "votes": null
    },
    {
      "id": "391566",
      "postDate": "09/22/2018 03:17:35",
      "content": "<pre><code>test['fullVisitorId'] = test['fullVisitorId'].astype(str)\ntest['fullVisitorId'].nunique()\n</code></pre>\n\n<p>Now returned 620627, lol. It definatelly made a difference but it's still something odd about it, it should return 617242. Thanks anyway man xD</p>",
      "rawMarkdown": "test['fullVisitorId'] = test['fullVisitorId'].astype(str)\n    test['fullVisitorId'].nunique()\n\nNow returned 620627, lol. It definatelly made a difference but it's still something odd about it, it should return 617242. Thanks anyway man xD",
      "votes": null
    },
    {
      "id": "391604",
      "postDate": "09/22/2018 05:11:10",
      "content": "<p>Try to do it during loading the csv file. i.e:</p>\n\n<p>read_csv with this params: dtype={'fullVisitorId': 'str'},</p>",
      "rawMarkdown": "Try to do it during loading the csv file. i.e:\n\nread_csv with this params: dtype={'fullVisitorId': 'str'},",
      "votes": null
    },
    {
      "id": "391835",
      "postDate": "09/22/2018 14:14:12",
      "content": "<p>HOLY SH*T MAN IT WORKED! &lt;3</p>",
      "rawMarkdown": "HOLY SH*T MAN IT WORKED! &lt;3",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 391561,
      "author_name": "kingychiu",
      "author_url": "",
      "post_date": "09/22/2018 03:06:44",
      "content": "<p>Have you checked that you have specified the fullVisitorId as String? I encountered the similar problem before, it was due to the datatype changes during file load/save.</p>",
      "votes": null,
      "replies": [
        {
          "id": 391566,
          "author_name": "wesleyjr01",
          "author_url": "",
          "post_date": "09/22/2018 03:17:35",
          "content": "<pre><code>test['fullVisitorId'] = test['fullVisitorId'].astype(str)\ntest['fullVisitorId'].nunique()\n</code></pre>\n\n<p>Now returned 620627, lol. It definatelly made a difference but it's still something odd about it, it should return 617242. Thanks anyway man xD</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 391604,
          "author_name": "kingychiu",
          "author_url": "",
          "post_date": "09/22/2018 05:11:10",
          "content": "<p>Try to do it during loading the csv file. i.e:</p>\n\n<p>read_csv with this params: dtype={'fullVisitorId': 'str'},</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 391835,
          "author_name": "wesleyjr01",
          "author_url": "",
          "post_date": "09/22/2018 14:14:12",
          "content": "<p>HOLY SH*T MAN IT WORKED! &lt;3</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "391551": "Hi, importing train.csv and test.csv by the [Júlian Kernel][1] and importing it to work on my Spyder IDE. When i set the command `test['fullVisitorId'].nunique()` to see the number of unique values of 'fullVisitorId' on my test dataframe, it returns 650489, but it should return 617242, any thoughts?\n*PS: If I try this procedure on a kaggle's kernel it returns 617242 correctly*\n\n\n  [1]: https://www.kaggle.com/julian3833/1-quick-start-read-csv-and-flatten-json-fields/notebook",
    "391561": "Have you checked that you have specified the fullVisitorId as String? I encountered the similar problem before, it was due to the datatype changes during file load/save.",
    "391566": "test['fullVisitorId'] = test['fullVisitorId'].astype(str)\n    test['fullVisitorId'].nunique()\n\nNow returned 620627, lol. It definatelly made a difference but it's still something odd about it, it should return 617242. Thanks anyway man xD",
    "391604": "Try to do it during loading the csv file. i.e:\n\nread_csv with this params: dtype={'fullVisitorId': 'str'},",
    "391835": "HOLY SH*T MAN IT WORKED! &lt;3"
  },
  "source": "meta"
}