{
  "id": 22155,
  "title": "Leave One Out Encoding",
  "url": "/competitions/grupo-bimbo-inventory-demand/discussion/22155",
  "author_name": "",
  "post_date": "2016-07-11T14:25:16.580Z",
  "votes": 12,
  "comment_count": 6,
  "views": 1385,
  "content": "<p>This uses about 1Gb of memory and projects the IDs to the demand by using Owen Zhang's Leave One Out methodology - you can add noise or incorporate minima and maxima for additional features.  The creation of the actual model is up to you - may I suggest ridge regression or KNN regression to start you off.</p>\n\n<ol>\n<li>Copy the files to the directory of your choice</li>\n<li>Change directory and file names in main.cpp to mirror your own</li>\n<li>From the command line go to your directory</li>\n<li>Type 'cmake .' (Please don't use the quotes ;)</li>\n<li>Type 'make'</li>\n<li>Type './LOO'</li>\n</ol>",
  "messages": [
    {
      "id": "126649",
      "postDate": "07/11/2016 14:25:16",
      "content": "<p>This uses about 1Gb of memory and projects the IDs to the demand by using Owen Zhang's Leave One Out methodology - you can add noise or incorporate minima and maxima for additional features.  The creation of the actual model is up to you - may I suggest ridge regression or KNN regression to start you off.</p>\n\n<ol>\n<li>Copy the files to the directory of your choice</li>\n<li>Change directory and file names in main.cpp to mirror your own</li>\n<li>From the command line go to your directory</li>\n<li>Type 'cmake .' (Please don't use the quotes ;)</li>\n<li>Type 'make'</li>\n<li>Type './LOO'</li>\n</ol>",
      "rawMarkdown": "This uses about 1Gb of memory and projects the IDs to the demand by using Owen Zhang's Leave One Out methodology - you can add noise or incorporate minima and maxima for additional features.  The creation of the actual model is up to you - may I suggest ridge regression or KNN regression to start you off.\r\n\r\n\r\n 1. Copy the files to the directory of your choice\r\n 2. Change directory and file names in main.cpp to mirror your own\r\n 3. From the command line go to your directory\r\n 4. Type 'cmake .' (Please don't use the quotes ;)\r\n 5. Type 'make'\r\n 6. Type './LOO'",
      "votes": null
    },
    {
      "id": "127231",
      "postDate": "07/13/2016 17:52:15",
      "content": "<p>Example of R implementation:</p>\n\n<p>library(data.table) </p>\n\n<p>train &lt;- fread('train.csv')</p>\n\n<p>test &lt;- fread('test.csv')</p>\n\n<p>train[,log_demand := log1p(Demanda_uni_equil)]</p>\n\n<p>mean_PR &lt;- train[, .(MPR = mean(log_demand)), by = .(Producto_ID, Ruta_SAK)] </p>\n\n<p>test &lt;- merge(test, mean_PR, all.x = TRUE, by = c(&quot;Producto_ID&quot;, &quot;Ruta_SAK&quot;))</p>\n\n<p>train[,sm:=sum(log_demand),by = .(Producto_ID, Ruta_SAK)]</p>\n\n<p>train[,ct:=.N, by = .(Producto_ID, Ruta_SAK)]</p>\n\n<p>train[,mean_PR:=(sm-log_demand)/(ct-1)]</p>",
      "rawMarkdown": "Example of R implementation:\r\n\r\n\r\n\r\nlibrary(data.table) \r\n\r\ntrain <- fread('train.csv')\r\n\r\ntest <- fread('test.csv')\r\n\r\ntrain[,log_demand := log1p(Demanda_uni_equil)]\r\n\r\nmean_PR <- train[, .(MPR = mean(log_demand)), by = .(Producto_ID, Ruta_SAK)] \r\n\r\ntest <- merge(test, mean_PR, all.x = TRUE, by = c(\"Producto_ID\", \"Ruta_SAK\"))\r\n\r\ntrain[,sm:=sum(log_demand),by = .(Producto_ID, Ruta_SAK)]\r\n\r\ntrain[,ct:=.N, by = .(Producto_ID, Ruta_SAK)]\r\n\r\ntrain[,mean_PR:=(sm-log_demand)/(ct-1)]",
      "votes": null
    },
    {
      "id": "127342",
      "postDate": "07/13/2016 23:41:03",
      "content": "<p>@Scirpus thanks for sharing this! </p>\n\n<p>any chance you can also share some links or technical information regarding Owen's methodology?\nI suppose this is to handle high cardinality of the categorical data, but still it would be nice to see what are the technical thinking behind :)</p>",
      "rawMarkdown": "Scirpus thanks for sharing this! \r\n\r\nany chance you can also share some links or technical information regarding Owen's methodology?\r\nI suppose this is to handle high cardinality of the categorical data, but still it would be nice to see what are the technical thinking behind :)",
      "votes": null
    },
    {
      "id": "127353",
      "postDate": "07/14/2016 00:50:39",
      "content": "<p><a href=\"http://www.slideshare.net/OwenZhang2/tips-for-data-science-competitions\">http://www.slideshare.net/OwenZhang2/tips-for-data-science-competitions</a></p>",
      "rawMarkdown": "http://www.slideshare.net/OwenZhang2/tips-for-data-science-competitions",
      "votes": null
    },
    {
      "id": "127531",
      "postDate": "07/14/2016 13:37:47",
      "content": "<p>@Bluefool, thanks and now perhaps I can bang my head against some C++ code after all ;)</p>",
      "rawMarkdown": "Bluefool, thanks and now perhaps I can bang my head against some C++ code after all ;)",
      "votes": null
    },
    {
      "id": "127534",
      "postDate": "07/14/2016 13:43:58",
      "content": "<p><a href=\"https://github.com/owenzhang/Kaggle-AmazonChallenge2013/\">You could always look at owens R code</a></p>",
      "rawMarkdown": "[You could always look at owens R code][1]\r\n\r\n\r\n  [1]: https://github.com/owenzhang/Kaggle-AmazonChallenge2013/",
      "votes": null
    },
    {
      "id": "127535",
      "postDate": "07/14/2016 13:47:04",
      "content": "<p>@Scripus thanks :)  </p>",
      "rawMarkdown": "Scripus thanks :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 127231,
      "author_name": "ccccat",
      "author_url": "",
      "post_date": "07/13/2016 17:52:15",
      "content": "<p>Example of R implementation:</p>\n\n<p>library(data.table) </p>\n\n<p>train &lt;- fread('train.csv')</p>\n\n<p>test &lt;- fread('test.csv')</p>\n\n<p>train[,log_demand := log1p(Demanda_uni_equil)]</p>\n\n<p>mean_PR &lt;- train[, .(MPR = mean(log_demand)), by = .(Producto_ID, Ruta_SAK)] </p>\n\n<p>test &lt;- merge(test, mean_PR, all.x = TRUE, by = c(&quot;Producto_ID&quot;, &quot;Ruta_SAK&quot;))</p>\n\n<p>train[,sm:=sum(log_demand),by = .(Producto_ID, Ruta_SAK)]</p>\n\n<p>train[,ct:=.N, by = .(Producto_ID, Ruta_SAK)]</p>\n\n<p>train[,mean_PR:=(sm-log_demand)/(ct-1)]</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 127342,
      "author_name": "yifanxie",
      "author_url": "",
      "post_date": "07/13/2016 23:41:03",
      "content": "<p>@Scirpus thanks for sharing this! </p>\n\n<p>any chance you can also share some links or technical information regarding Owen's methodology?\nI suppose this is to handle high cardinality of the categorical data, but still it would be nice to see what are the technical thinking behind :)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 127353,
      "author_name": "domcastro",
      "author_url": "",
      "post_date": "07/14/2016 00:50:39",
      "content": "<p><a href=\"http://www.slideshare.net/OwenZhang2/tips-for-data-science-competitions\">http://www.slideshare.net/OwenZhang2/tips-for-data-science-competitions</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 127531,
      "author_name": "yifanxie",
      "author_url": "",
      "post_date": "07/14/2016 13:37:47",
      "content": "<p>@Bluefool, thanks and now perhaps I can bang my head against some C++ code after all ;)</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 127534,
      "author_name": "scirpus",
      "author_url": "",
      "post_date": "07/14/2016 13:43:58",
      "content": "<p><a href=\"https://github.com/owenzhang/Kaggle-AmazonChallenge2013/\">You could always look at owens R code</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 127535,
      "author_name": "yifanxie",
      "author_url": "",
      "post_date": "07/14/2016 13:47:04",
      "content": "<p>@Scripus thanks :)  </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "126649": "This uses about 1Gb of memory and projects the IDs to the demand by using Owen Zhang's Leave One Out methodology - you can add noise or incorporate minima and maxima for additional features.  The creation of the actual model is up to you - may I suggest ridge regression or KNN regression to start you off.\r\n\r\n\r\n 1. Copy the files to the directory of your choice\r\n 2. Change directory and file names in main.cpp to mirror your own\r\n 3. From the command line go to your directory\r\n 4. Type 'cmake .' (Please don't use the quotes ;)\r\n 5. Type 'make'\r\n 6. Type './LOO'",
    "127231": "Example of R implementation:\r\n\r\n\r\n\r\nlibrary(data.table) \r\n\r\ntrain <- fread('train.csv')\r\n\r\ntest <- fread('test.csv')\r\n\r\ntrain[,log_demand := log1p(Demanda_uni_equil)]\r\n\r\nmean_PR <- train[, .(MPR = mean(log_demand)), by = .(Producto_ID, Ruta_SAK)] \r\n\r\ntest <- merge(test, mean_PR, all.x = TRUE, by = c(\"Producto_ID\", \"Ruta_SAK\"))\r\n\r\ntrain[,sm:=sum(log_demand),by = .(Producto_ID, Ruta_SAK)]\r\n\r\ntrain[,ct:=.N, by = .(Producto_ID, Ruta_SAK)]\r\n\r\ntrain[,mean_PR:=(sm-log_demand)/(ct-1)]",
    "127342": "Scirpus thanks for sharing this! \r\n\r\nany chance you can also share some links or technical information regarding Owen's methodology?\r\nI suppose this is to handle high cardinality of the categorical data, but still it would be nice to see what are the technical thinking behind :)",
    "127353": "http://www.slideshare.net/OwenZhang2/tips-for-data-science-competitions",
    "127531": "Bluefool, thanks and now perhaps I can bang my head against some C++ code after all ;)",
    "127534": "[You could always look at owens R code][1]\r\n\r\n\r\n  [1]: https://github.com/owenzhang/Kaggle-AmazonChallenge2013/",
    "127535": "Scripus thanks :)"
  },
  "source": "meta"
}