{
  "id": 11309,
  "title": "Searching for keys that do not exist?",
  "url": "/competitions/avazu-ctr-prediction/discussion/11309",
  "author_name": "",
  "post_date": "2014-12-21T23:37:36.473Z",
  "votes": 1,
  "comment_count": 4,
  "views": 1327,
  "content": "<p>When I try and upload, I get this error message</p>\n<p>ERROR: Unable to find 4563025 required key values in the 'id' column ERROR: Unable to find the required key value '10000174058809263569' in the 'id'</p>\n<p>I checked the raw test and sampleSubmission datasets. and they do not seem to contain: id 10000174058809263569.</p>\n<p>I am using following script to output my submission:</p>\n<p>write.csv(baseline, file=&quot;baseline.csv&quot;, quote = FALSE, row.names = FALSE)</p>\n<p>does the precision of the click column matter, I am going to about 10 decimal places, is this too long?&nbsp;</p>\n<p>Wondering what could possibly be the problem.&nbsp; Upon inspection, all of my data appears to be correct?&nbsp; Any insight would be greatly apperciated.&nbsp;</p>",
  "messages": [
    {
      "id": "61641",
      "postDate": "12/21/2014 23:37:36",
      "content": "<p>When I try and upload, I get this error message</p>\n<p>ERROR: Unable to find 4563025 required key values in the 'id' column ERROR: Unable to find the required key value '10000174058809263569' in the 'id'</p>\n<p>I checked the raw test and sampleSubmission datasets. and they do not seem to contain: id 10000174058809263569.</p>\n<p>I am using following script to output my submission:</p>\n<p>write.csv(baseline, file=&quot;baseline.csv&quot;, quote = FALSE, row.names = FALSE)</p>\n<p>does the precision of the click column matter, I am going to about 10 decimal places, is this too long?&nbsp;</p>\n<p>Wondering what could possibly be the problem.&nbsp; Upon inspection, all of my data appears to be correct?&nbsp; Any insight would be greatly apperciated.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "61656",
      "postDate": "12/22/2014 10:59:25",
      "content": "<p>David,</p>\n<p>You are suffering &nbsp;loss of 'floating point precision' when u import the data &nbsp;</p>\n<p>i.e.. &nbsp; the first id in test.csv is&nbsp;</p>\n<p><code>10000174058809263569 &nbsp;# string of id</code></p>\n<p>just cut and paste that number into your R console and hit enter</p>\n<p style=\"text-align: left; padding-left: 30px;\"><code>&gt; 10000174058809263569 &nbsp;<br></code></p>\n<p style=\"text-align: left; padding-left: 30px;\"><code>[1] 10000174058809264128 &nbsp;<strong>#</strong></code><strong><code>this isn't the number i typed in?!</code></strong></p>\n<p style=\"text-align: left; padding-left: 30px;\">&nbsp;</p>\n<p>R is converting your string(id) to its closest binary approximation&nbsp;</p>\n<p>The work-around is to import the id's as a string(character) - this of course uses way more memory and is generally a pain. Remember- &nbsp;the moment that some of those id's becomes a numeric in R - data is lost forever. &nbsp;&nbsp;</p>\n<p><a href=\"http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html\">What every computer scientist should know about floating point arithmetic</a></p>\n<p>The short answer is that the ability for a computer to accurately store data declines and you move away from 1. (R uses IEEE 754 double-precision floating-point numbers.)</p>\n<p>hope this helps</p>\n<p>Tim</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "61694",
      "postDate": "12/22/2014 20:04:13",
      "content": "<p>Hi David,</p>\n<p>I experienced the same problem as you.</p>\n<p>The way I solved it was to set&nbsp;colClasses during reading csv files. &nbsp; &quot;Character&quot; was used for id.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "61699",
      "postDate": "12/22/2014 20:48:41",
      "content": "<p>Hi David,</p>\n<p>I just found there is another link which should be helpful.</p>\n<p>http://www.kaggle.com/c/avazu-ctr-prediction/forums/t/11001/submission-issues-errors</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "61712",
      "postDate": "12/22/2014 23:59:01",
      "content": "<p>Everyone,</p>\n<p>Thanks so much for helping me out with this, it is much apperciated.&nbsp; I am now up and running and having all kinds of fun.&nbsp;</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 61656,
      "author_name": "scharf",
      "author_url": "",
      "post_date": "12/22/2014 10:59:25",
      "content": "<p>David,</p>\n<p>You are suffering &nbsp;loss of 'floating point precision' when u import the data &nbsp;</p>\n<p>i.e.. &nbsp; the first id in test.csv is&nbsp;</p>\n<p><code>10000174058809263569 &nbsp;# string of id</code></p>\n<p>just cut and paste that number into your R console and hit enter</p>\n<p style=\"text-align: left; padding-left: 30px;\"><code>&gt; 10000174058809263569 &nbsp;<br></code></p>\n<p style=\"text-align: left; padding-left: 30px;\"><code>[1] 10000174058809264128 &nbsp;<strong>#</strong></code><strong><code>this isn't the number i typed in?!</code></strong></p>\n<p style=\"text-align: left; padding-left: 30px;\">&nbsp;</p>\n<p>R is converting your string(id) to its closest binary approximation&nbsp;</p>\n<p>The work-around is to import the id's as a string(character) - this of course uses way more memory and is generally a pain. Remember- &nbsp;the moment that some of those id's becomes a numeric in R - data is lost forever. &nbsp;&nbsp;</p>\n<p><a href=\"http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html\">What every computer scientist should know about floating point arithmetic</a></p>\n<p>The short answer is that the ability for a computer to accurately store data declines and you move away from 1. (R uses IEEE 754 double-precision floating-point numbers.)</p>\n<p>hope this helps</p>\n<p>Tim</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 61694,
      "author_name": "newtohere",
      "author_url": "",
      "post_date": "12/22/2014 20:04:13",
      "content": "<p>Hi David,</p>\n<p>I experienced the same problem as you.</p>\n<p>The way I solved it was to set&nbsp;colClasses during reading csv files. &nbsp; &quot;Character&quot; was used for id.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 61699,
      "author_name": "newtohere",
      "author_url": "",
      "post_date": "12/22/2014 20:48:41",
      "content": "<p>Hi David,</p>\n<p>I just found there is another link which should be helpful.</p>\n<p>http://www.kaggle.com/c/avazu-ctr-prediction/forums/t/11001/submission-issues-errors</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 61712,
      "author_name": "davidbess0",
      "author_url": "",
      "post_date": "12/22/2014 23:59:01",
      "content": "<p>Everyone,</p>\n<p>Thanks so much for helping me out with this, it is much apperciated.&nbsp; I am now up and running and having all kinds of fun.&nbsp;</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "61641": "",
    "61656": "",
    "61694": "",
    "61699": "",
    "61712": ""
  },
  "source": "meta"
}