{
  "id": 21554,
  "title": "~10k duplicated client ids",
  "url": "/competitions/grupo-bimbo-inventory-demand/discussion/21554",
  "author_name": "",
  "post_date": "2016-06-09T19:13:26.050Z",
  "votes": 4,
  "comment_count": 4,
  "views": 1132,
  "content": "<p>This really messed up my first submission:</p>\n\n<pre><code>library(data.table)\nclient &lt;- fread('input/cliente_tabla.csv')\nclient[which(duplicated(Cliente_ID) | duplicated(Cliente_ID, fromLast=T)),]\n</code></pre>\n\n<p>They all look pretty similar, so I'm just dropping the duplicates, but this might be a case of a bad SQL join the the data prep or something</p>",
  "messages": [
    {
      "id": "123110",
      "postDate": "06/09/2016 19:13:26",
      "content": "<p>This really messed up my first submission:</p>\n\n<pre><code>library(data.table)\nclient &lt;- fread('input/cliente_tabla.csv')\nclient[which(duplicated(Cliente_ID) | duplicated(Cliente_ID, fromLast=T)),]\n</code></pre>\n\n<p>They all look pretty similar, so I'm just dropping the duplicates, but this might be a case of a bad SQL join the the data prep or something</p>",
      "rawMarkdown": "This really messed up my first submission:\r\n\r\n    library(data.table)\r\n    client <- fread('input/cliente_tabla.csv')\r\n    client[which(duplicated(Cliente_ID) | duplicated(Cliente_ID, fromLast=T)),]\r\n\r\nThey all look pretty similar, so I'm just dropping the duplicates, but this might be a case of a bad SQL join the the data prep or something",
      "votes": null
    },
    {
      "id": "123124",
      "postDate": "06/09/2016 21:35:53",
      "content": "<p>@Zach, </p>\n\n<p>Thanks for sharing. We are aware of this. This is a real world problem where the data that appear in the transactions are noisy and not standardized, so when we normalized the data, we decided to keep all the original client names from the transactions, and leave it to the participants to decide which client names to go with. </p>",
      "rawMarkdown": "Zach, \r\n\r\nThanks for sharing. We are aware of this. This is a real world problem where the data that appear in the transactions are noisy and not standardized, so when we normalized the data, we decided to keep all the original client names from the transactions, and leave it to the participants to decide which client names to go with.",
      "votes": null
    },
    {
      "id": "123125",
      "postDate": "06/09/2016 21:37:38",
      "content": "<p>Makes sense, thanks.  I missed this in the data description, so it seemed worth opening a thread =D</p>",
      "rawMarkdown": "Makes sense, thanks.  I missed this in the data description, so it seemed worth opening a thread =D",
      "votes": null
    },
    {
      "id": "123144",
      "postDate": "06/09/2016 23:27:44",
      "content": "<p>@Zach, good point. I've added it in the data description page. Thanks for the reminder!!</p>",
      "rawMarkdown": "Zach, good point. I've added it in the data description page. Thanks for the reminder!!",
      "votes": null
    },
    {
      "id": "123222",
      "postDate": "06/10/2016 12:38:04",
      "content": "<p>@Wendy thank you!</p>\n\n<p>it actually turns out to not have affected my results very much.  Once I corrected my code to use RMSLE instead of RMSE (whoops!) I got much better results, even with the duplicated rows from the cliente_tabla.</p>",
      "rawMarkdown": "Wendy thank you!\r\n\r\nit actually turns out to not have affected my results very much.  Once I corrected my code to use RMSLE instead of RMSE (whoops!) I got much better results, even with the duplicated rows from the cliente_tabla.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 123124,
      "author_name": "wendykan",
      "author_url": "",
      "post_date": "06/09/2016 21:35:53",
      "content": "<p>@Zach, </p>\n\n<p>Thanks for sharing. We are aware of this. This is a real world problem where the data that appear in the transactions are noisy and not standardized, so when we normalized the data, we decided to keep all the original client names from the transactions, and leave it to the participants to decide which client names to go with. </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 123125,
      "author_name": "zachmayer",
      "author_url": "",
      "post_date": "06/09/2016 21:37:38",
      "content": "<p>Makes sense, thanks.  I missed this in the data description, so it seemed worth opening a thread =D</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 123144,
      "author_name": "wendykan",
      "author_url": "",
      "post_date": "06/09/2016 23:27:44",
      "content": "<p>@Zach, good point. I've added it in the data description page. Thanks for the reminder!!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 123222,
      "author_name": "zachmayer",
      "author_url": "",
      "post_date": "06/10/2016 12:38:04",
      "content": "<p>@Wendy thank you!</p>\n\n<p>it actually turns out to not have affected my results very much.  Once I corrected my code to use RMSLE instead of RMSE (whoops!) I got much better results, even with the duplicated rows from the cliente_tabla.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "123110": "This really messed up my first submission:\r\n\r\n    library(data.table)\r\n    client <- fread('input/cliente_tabla.csv')\r\n    client[which(duplicated(Cliente_ID) | duplicated(Cliente_ID, fromLast=T)),]\r\n\r\nThey all look pretty similar, so I'm just dropping the duplicates, but this might be a case of a bad SQL join the the data prep or something",
    "123124": "Zach, \r\n\r\nThanks for sharing. We are aware of this. This is a real world problem where the data that appear in the transactions are noisy and not standardized, so when we normalized the data, we decided to keep all the original client names from the transactions, and leave it to the participants to decide which client names to go with.",
    "123125": "Makes sense, thanks.  I missed this in the data description, so it seemed worth opening a thread =D",
    "123144": "Zach, good point. I've added it in the data description page. Thanks for the reminder!!",
    "123222": "Wendy thank you!\r\n\r\nit actually turns out to not have affected my results very much.  Once I corrected my code to use RMSLE instead of RMSE (whoops!) I got much better results, even with the duplicated rows from the cliente_tabla."
  },
  "source": "meta"
}