{
  "id": 123288,
  "title": "[46th Pvt LB] 18th Public LB - Very simple solution",
  "url": "/competitions/ashrae-energy-prediction/writeups/wittmannf-46th-pvt-lb-18th-public-lb-very-simple-s",
  "author_name": "",
  "post_date": "2020-01-14T03:47:53.877Z",
  "votes": 31,
  "comment_count": 22,
  "views": 0,
  "content": "<ul>\n<li>Kernel: <a href=\"https://www.kaggle.com/wittmannf/0-939-lb-public-blend-leak-valid-ridgecv/\">https://www.kaggle.com/wittmannf/0-939-lb-public-blend-leak-valid-ridgecv/</a></li>\n</ul>\n\n<p>Most of my time was actually spent on models that turned out to not score as good as the public kernels (NNs and bldg based models). My 18th is thanks to some side experimentation when trying to maximize the tips that have been provided in public Kernels and discussion. </p>\n\n<p>The solution consists of a blend of non-leaked public submissions (I enhanced some of them) using Ridge CV's coefficients when fitting against the leak validation data. For the enhanced models, I discovered that it was more effective to combine them using RidgeCV's coeffs from leak validation data than just averaging them. However, I only had time to that in the <code>simple-data-cleanup-3-models</code>. My public score was 0.939 with leak replacement and 1.04ish without leak replacement. </p>\n\n<p>And here's something interesting: RidgeCV figured out negative coefficient for some submissions, although the sum of all of them was close to 1 (as expected):</p>\n\n<p>```</p>\n\n<h2>Ridge Coefficients</h2>\n\n<p>Sum of coefficients: 0.9994466580815099\nhalf-half-drop-rows-stratify-weekday has weight 0.14\nsimple-data-cleanup-3-models has weight 0.26\nashrae-kfold-lightgbm-without-leak-1-08 has weight -0.23\nanother-1-08-lb-no-leak has weight -0.46\nashrae-kfold-lightgbm-without-building-id has weight 0.19\nashrae-energy-prediction-using-stratified-kfold has weight 0.52\nashrae-lightgbm-without-leak has weight -0.15\nashrae-stratified-kfold-lightgbm has weight 0.23\nashrae-2-lightgbm-without-leak-data has weight 0.50\n```</p>\n\n<p>All credits go to the authors of the public kernels and <a href=\"/yamsam\">@yamsam</a> 's idea of leak validation. My contribution was the idea of combining them using RidgeCV. </p>\n\n<h3>UPDATE</h3>\n\n<p>As mentioned in <a href=\"https://www.kaggle.com/c/ashrae-energy-prediction/discussion/125843\">this discussion</a>, the following kernel would've ranked between 13th and 16th position of private leaderboard:\n- <a href=\"https://www.kaggle.com/wittmannf/blender-leak-validation-inception?scriptVersionId=25306647\">https://www.kaggle.com/wittmannf/blender-leak-validation-inception?scriptVersionId=25306647</a></p>\n\n<p>The only difference is that I performed a second level in the blending process (<code>submission.csv</code> is the output from the first blend):\n<code>\nsubmission_paths = [\n'/kaggle/input/ashrae-half-and-half-w-drop-rows-stratify/submission.csv', #1.106\n'/kaggle/input/ashrae-simple-data-cleanup-lb-1-08-no-leaks/submission.csv',\n'/kaggle/input/ashrae-kfold-lightgbm-without-leak-1-08/submission.csv',\n'/kaggle/input/another-1-08-lb-no-leak/fe2_lgbm.csv',\n'/kaggle/input/ashrae-kfold-lightgbm-without-building-id/submission.csv', #1.098\n'/kaggle/input/ashrae-energy-prediction-using-stratified-kfold/fe2_lgbm.csv', #1.074\n'/kaggle/input/ashrae-lightgbm-without-leak/submission.csv', #1.082\n'/kaggle/input/ashrae-stratified-kfold-lightgbm/submission.csv',#1.075\n'./submission.csv'#1.04\n]\n</code></p>",
  "messages": [
    {
      "id": "703682",
      "postDate": "12/26/2019 13:54:38",
      "content": "<ul>\n<li>Kernel: <a href=\"https://www.kaggle.com/wittmannf/0-939-lb-public-blend-leak-valid-ridgecv/\">https://www.kaggle.com/wittmannf/0-939-lb-public-blend-leak-valid-ridgecv/</a></li>\n</ul>\n\n<p>Most of my time was actually spent on models that turned out to not score as good as the public kernels (NNs and bldg based models). My 18th is thanks to some side experimentation when trying to maximize the tips that have been provided in public Kernels and discussion. </p>\n\n<p>The solution consists of a blend of non-leaked public submissions (I enhanced some of them) using Ridge CV's coefficients when fitting against the leak validation data. For the enhanced models, I discovered that it was more effective to combine them using RidgeCV's coeffs from leak validation data than just averaging them. However, I only had time to that in the <code>simple-data-cleanup-3-models</code>. My public score was 0.939 with leak replacement and 1.04ish without leak replacement. </p>\n\n<p>And here's something interesting: RidgeCV figured out negative coefficient for some submissions, although the sum of all of them was close to 1 (as expected):</p>\n\n<p>```</p>\n\n<h2>Ridge Coefficients</h2>\n\n<p>Sum of coefficients: 0.9994466580815099\nhalf-half-drop-rows-stratify-weekday has weight 0.14\nsimple-data-cleanup-3-models has weight 0.26\nashrae-kfold-lightgbm-without-leak-1-08 has weight -0.23\nanother-1-08-lb-no-leak has weight -0.46\nashrae-kfold-lightgbm-without-building-id has weight 0.19\nashrae-energy-prediction-using-stratified-kfold has weight 0.52\nashrae-lightgbm-without-leak has weight -0.15\nashrae-stratified-kfold-lightgbm has weight 0.23\nashrae-2-lightgbm-without-leak-data has weight 0.50\n```</p>\n\n<p>All credits go to the authors of the public kernels and <a href=\"/yamsam\">@yamsam</a> 's idea of leak validation. My contribution was the idea of combining them using RidgeCV. </p>\n\n<h3>UPDATE</h3>\n\n<p>As mentioned in <a href=\"https://www.kaggle.com/c/ashrae-energy-prediction/discussion/125843\">this discussion</a>, the following kernel would've ranked between 13th and 16th position of private leaderboard:\n- <a href=\"https://www.kaggle.com/wittmannf/blender-leak-validation-inception?scriptVersionId=25306647\">https://www.kaggle.com/wittmannf/blender-leak-validation-inception?scriptVersionId=25306647</a></p>\n\n<p>The only difference is that I performed a second level in the blending process (<code>submission.csv</code> is the output from the first blend):\n<code>\nsubmission_paths = [\n'/kaggle/input/ashrae-half-and-half-w-drop-rows-stratify/submission.csv', #1.106\n'/kaggle/input/ashrae-simple-data-cleanup-lb-1-08-no-leaks/submission.csv',\n'/kaggle/input/ashrae-kfold-lightgbm-without-leak-1-08/submission.csv',\n'/kaggle/input/another-1-08-lb-no-leak/fe2_lgbm.csv',\n'/kaggle/input/ashrae-kfold-lightgbm-without-building-id/submission.csv', #1.098\n'/kaggle/input/ashrae-energy-prediction-using-stratified-kfold/fe2_lgbm.csv', #1.074\n'/kaggle/input/ashrae-lightgbm-without-leak/submission.csv', #1.082\n'/kaggle/input/ashrae-stratified-kfold-lightgbm/submission.csv',#1.075\n'./submission.csv'#1.04\n]\n</code></p>",
      "rawMarkdown": "Kernel: https://www.kaggle.com/wittmannf/0-939-lb-public-blend-leak-valid-ridgecv/\n\nMost of my time was actually spent on models that turned out to not score as good as the public kernels (NNs and bldg based models). My 18th is thanks to some side experimentation when trying to maximize the tips that have been provided in public Kernels and discussion. \n\nThe solution consists of a blend of non-leaked public submissions (I enhanced some of them) using Ridge CV's coefficients when fitting against the leak validation data. For the enhanced models, I discovered that it was more effective to combine them using RidgeCV's coeffs from leak validation data than just averaging them. However, I only had time to that in the `simple-data-cleanup-3-models`. My public score was 0.939 with leak replacement and 1.04ish without leak replacement. \n\nAnd here's something interesting: RidgeCV figured out negative coefficient for some submissions, although the sum of all of them was close to 1 (as expected):\n\n```\n## Ridge Coefficients\nSum of coefficients: 0.9994466580815099\nhalf-half-drop-rows-stratify-weekday has weight 0.14\nsimple-data-cleanup-3-models has weight 0.26\nashrae-kfold-lightgbm-without-leak-1-08 has weight -0.23\nanother-1-08-lb-no-leak has weight -0.46\nashrae-kfold-lightgbm-without-building-id has weight 0.19\nashrae-energy-prediction-using-stratified-kfold has weight 0.52\nashrae-lightgbm-without-leak has weight -0.15\nashrae-stratified-kfold-lightgbm has weight 0.23\nashrae-2-lightgbm-without-leak-data has weight 0.50\n```\n\nAll credits go to the authors of the public kernels and @yamsam 's idea of leak validation. My contribution was the idea of combining them using RidgeCV. \n\n### UPDATE\nAs mentioned in [this discussion](https://www.kaggle.com/c/ashrae-energy-prediction/discussion/125843), the following kernel would've ranked between 13th and 16th position of private leaderboard:\n- https://www.kaggle.com/wittmannf/blender-leak-validation-inception?scriptVersionId=25306647\n\nThe only difference is that I performed a second level in the blending process (`submission.csv` is the output from the first blend):\n```\nsubmission_paths = [\n'/kaggle/input/ashrae-half-and-half-w-drop-rows-stratify/submission.csv', #1.106\n'/kaggle/input/ashrae-simple-data-cleanup-lb-1-08-no-leaks/submission.csv',\n'/kaggle/input/ashrae-kfold-lightgbm-without-leak-1-08/submission.csv',\n'/kaggle/input/another-1-08-lb-no-leak/fe2_lgbm.csv',\n'/kaggle/input/ashrae-kfold-lightgbm-without-building-id/submission.csv', #1.098\n'/kaggle/input/ashrae-energy-prediction-using-stratified-kfold/fe2_lgbm.csv', #1.074\n'/kaggle/input/ashrae-lightgbm-without-leak/submission.csv', #1.082\n'/kaggle/input/ashrae-stratified-kfold-lightgbm/submission.csv',#1.075\n'./submission.csv'#1.04\n]\n```",
      "votes": null
    },
    {
      "id": "703697",
      "postDate": "12/26/2019 14:14:04",
      "content": "<p>You used the leak data for validation which isn't in the private lb, don't you think you might have over-fitted?</p>",
      "rawMarkdown": "You used the leak data for validation which isn't in the private lb, don't you think you might have over-fitted?",
      "votes": null
    },
    {
      "id": "703699",
      "postDate": "12/26/2019 14:21:35",
      "content": "<p>It is a tiny transfer learning. I don't think it causes any overfitting with such a simple model and big data. If we all don't shake-up due to seasonality, I don't think he will shake-up. </p>\n\n<p>Good idea <a href=\"/wittmannf\">@wittmannf</a> ! I was going to do the same but didn't expect public kernels to be that much valuable, therefore didn't invest time on it.</p>",
      "rawMarkdown": "It is a tiny transfer learning. I don't think it causes any overfitting with such a simple model and big data. If we all don't shake-up due to seasonality, I don't think he will shake-up. \n\nGood idea @wittmannf ! I was going to do the same but didn't expect public kernels to be that much valuable, therefore didn't invest time on it.",
      "votes": null
    },
    {
      "id": "703704",
      "postDate": "12/26/2019 14:28:54",
      "content": "<p>We will know who is right in a few days :-)</p>",
      "rawMarkdown": "We will know who is right in a few days :-)",
      "votes": null
    },
    {
      "id": "703706",
      "postDate": "12/26/2019 14:29:13",
      "content": "<p><a href=\"/wittmannf\">@wittmannf</a> , thanks for sharing.</p>",
      "rawMarkdown": "wittmannf , thanks for sharing.",
      "votes": null
    },
    {
      "id": "703727",
      "postDate": "12/26/2019 15:15:57",
      "content": "<p>My solution is very similar wtih yours,hope that we don't have over-fitted.</p>",
      "rawMarkdown": "My solution is very similar wtih yours,hope that we don't have over-fitted.",
      "votes": null
    },
    {
      "id": "703780",
      "postDate": "12/26/2019 16:02:50",
      "content": "<p>Thanks for sharing <a href=\"/wittmannf\">@wittmannf</a>! Congrats !</p>",
      "rawMarkdown": "Thanks for sharing @wittmannf! Congrats !",
      "votes": null
    },
    {
      "id": "703799",
      "postDate": "12/26/2019 16:36:00",
      "content": "<p>I don't think it was overfitting since the score was improving in non-leaked sites. </p>\n\n<p>However, another type of overfitting sometimes happened. The leak validation score was a good reference for the leaderboard as long as I kept the model simple. When trying non-linear models like NNs for combining the submissions, I was able to get great validation scores, but it did not reflect in better leaderboard scores. </p>",
      "rawMarkdown": "I don't think it was overfitting since the score was improving in non-leaked sites. \n\nHowever, another type of overfitting sometimes happened. The leak validation score was a good reference for the leaderboard as long as I kept the model simple. When trying non-linear models like NNs for combining the submissions, I was able to get great validation scores, but it did not reflect in better leaderboard scores.",
      "votes": null
    },
    {
      "id": "703847",
      "postDate": "12/26/2019 17:52:44",
      "content": "<p>We also found out that NNs and building based models didn't work as expected :-) Your blending solution is quite interesting, we didn't try using ridgeCV because we were afraid of negative coefficients, but they could have their sense, if they work out smoothing certain predictions as your strategy demonstrates. What was your C parameters in the end?</p>",
      "rawMarkdown": "We also found out that NNs and building based models didn't work as expected :-) Your blending solution is quite interesting, we didn't try using ridgeCV because we were afraid of negative coefficients, but they could have their sense, if they work out smoothing certain predictions as your strategy demonstrates. What was your C parameters in the end?",
      "votes": null
    },
    {
      "id": "703855",
      "postDate": "12/26/2019 18:07:08",
      "content": "<p>Thanks for sharing! (2 Kernels from the ones you used are mine 😋 )</p>",
      "rawMarkdown": "Thanks for sharing! (2 Kernels from the ones you used are mine 😋 )",
      "votes": null
    },
    {
      "id": "703884",
      "postDate": "12/26/2019 18:58:22",
      "content": "<p>Thanks for sharing.\nThe simplest of ideas are often the hardest to find.</p>",
      "rawMarkdown": "Thanks for sharing.\nThe simplest of ideas are often the hardest to find.",
      "votes": null
    },
    {
      "id": "703961",
      "postDate": "12/26/2019 21:41:33",
      "content": "<p>I think this competition is about blending of blendIng of blending of ... in my impression. XD.</p>",
      "rawMarkdown": "I think this competition is about blending of blendIng of blending of ... in my impression. XD.",
      "votes": null
    },
    {
      "id": "703964",
      "postDate": "12/26/2019 21:51:18",
      "content": "<p>nice solution</p>",
      "rawMarkdown": "nice solution",
      "votes": null
    },
    {
      "id": "704089",
      "postDate": "12/27/2019 03:28:14",
      "content": "<p>Thanks for sharing!! I didn't realize RidgeCV could work.</p>",
      "rawMarkdown": "Thanks for sharing!! I didn't realize RidgeCV could work.",
      "votes": null
    },
    {
      "id": "704197",
      "postDate": "12/27/2019 06:43:00",
      "content": "<p>All the best! Thanks for sharing though <a href=\"/wittmannf\">@wittmannf</a> </p>",
      "rawMarkdown": "All the best! Thanks for sharing though @wittmannf",
      "votes": null
    },
    {
      "id": "712190",
      "postDate": "01/06/2020 23:27:28",
      "content": "<p>Thanks for sharing</p>",
      "rawMarkdown": "Thanks for sharing",
      "votes": null
    },
    {
      "id": "717394",
      "postDate": "01/13/2020 04:33:18",
      "content": "<p>Thank you very much <a href=\"/wittmannf\">@wittmannf</a> for sharing! We would like to add your solution to our analysis of the top processes and models. Please let us know how you would like to be acknowledged. The default will be what you have on your profile. Thanks again!</p>\n\n<p><strong>Also, for everyone else, we are in the process of curating a directory of Notebooks for building performance engineers and analysts from the competition that will serve as a recipe book for beginners as well as experts who are curious to learn about certain topics. We highly encourage you all to share your solutions (or any part of your process) as a notebook for this purpose. We will acknowledge the authors of all notebooks in this process, so please annotate your notebook or profile with how you would like to be acknowledged. We will also include general solutions and process information that is not in notebooks as well. It would also be helpful if you could indicate whether you are an ASHRAE member.</strong></p>",
      "rawMarkdown": "Thank you very much @wittmannf for sharing! We would like to add your solution to our analysis of the top processes and models. Please let us know how you would like to be acknowledged. The default will be what you have on your profile. Thanks again!\n\n**Also, for everyone else, we are in the process of curating a directory of Notebooks for building performance engineers and analysts from the competition that will serve as a recipe book for beginners as well as experts who are curious to learn about certain topics. We highly encourage you all to share your solutions (or any part of your process) as a notebook for this purpose. We will acknowledge the authors of all notebooks in this process, so please annotate your notebook or profile with how you would like to be acknowledged. We will also include general solutions and process information that is not in notebooks as well. It would also be helpful if you could indicate whether you are an ASHRAE member.**",
      "votes": null
    },
    {
      "id": "718109",
      "postDate": "01/14/2020 02:58:37",
      "content": "<p>Thanks <a href=\"/mimoudata\">@mimoudata</a> ! They were very helpful! :) </p>",
      "rawMarkdown": "Thanks @mimoudata ! They were very helpful! :)",
      "votes": null
    },
    {
      "id": "718110",
      "postDate": "01/14/2020 02:59:00",
      "content": "<p>Thanks! and congrats on the 2nd place! </p>",
      "rawMarkdown": "Thanks! and congrats on the 2nd place!",
      "votes": null
    },
    {
      "id": "718111",
      "postDate": "01/14/2020 02:59:44",
      "content": "<p>You are welcome! And congrats on the 1st place!!</p>",
      "rawMarkdown": "You are welcome! And congrats on the 1st place!!",
      "votes": null
    },
    {
      "id": "718116",
      "postDate": "01/14/2020 03:05:39",
      "content": "<p>Hey <a href=\"/lucamassaron\">@lucamassaron</a> , sorry for the delay to reply you! Yeah, turns out negative coeffs actually help. This is something I wasn't expecting. I am quite curious on checking in different datasets. Regarding the C value, next time I run the kernel I will check! But my guess is that gridsearchCV chose a small C value since the amplitude of the weights is not that high. </p>",
      "rawMarkdown": "Hey @lucamassaron , sorry for the delay to reply you! Yeah, turns out negative coeffs actually help. This is something I wasn't expecting. I am quite curious on checking in different datasets. Regarding the C value, next time I run the kernel I will check! But my guess is that gridsearchCV chose a small C value since the amplitude of the weights is not that high.",
      "votes": null
    },
    {
      "id": "718118",
      "postDate": "01/14/2020 03:09:06",
      "content": "<p>Congrats on the 21st place <a href=\"/decarmelo\">@decarmelo</a> ! And we can confirm that we didn't overfit (maybe mine did a little). But I noticed that your LB position actually increased!! </p>",
      "rawMarkdown": "Congrats on the 21st place @decarmelo ! And we can confirm that we didn't overfit (maybe mine did a little). But I noticed that your LB position actually increased!!",
      "votes": null
    },
    {
      "id": "721056",
      "postDate": "01/17/2020 01:21:22",
      "content": "<p>Thanks <a href=\"/claytonmiller\">@claytonmiller</a>! It's okay to use what's on my profile </p>",
      "rawMarkdown": "Thanks @claytonmiller! It's okay to use what's on my profile",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 703697,
      "author_name": "themonologue",
      "author_url": "",
      "post_date": "12/26/2019 14:14:04",
      "content": "<p>You used the leak data for validation which isn't in the private lb, don't you think you might have over-fitted?</p>",
      "votes": null,
      "replies": [
        {
          "id": 703699,
          "author_name": "aerdem4",
          "author_url": "",
          "post_date": "12/26/2019 14:21:35",
          "content": "<p>It is a tiny transfer learning. I don't think it causes any overfitting with such a simple model and big data. If we all don't shake-up due to seasonality, I don't think he will shake-up. </p>\n\n<p>Good idea <a href=\"/wittmannf\">@wittmannf</a> ! I was going to do the same but didn't expect public kernels to be that much valuable, therefore didn't invest time on it.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 703704,
          "author_name": "sheriytm",
          "author_url": "",
          "post_date": "12/26/2019 14:28:54",
          "content": "<p>We will know who is right in a few days :-)</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 703799,
          "author_name": "wittmannf",
          "author_url": "",
          "post_date": "12/26/2019 16:36:00",
          "content": "<p>I don't think it was overfitting since the score was improving in non-leaked sites. </p>\n\n<p>However, another type of overfitting sometimes happened. The leak validation score was a good reference for the leaderboard as long as I kept the model simple. When trying non-linear models like NNs for combining the submissions, I was able to get great validation scores, but it did not reflect in better leaderboard scores. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 704197,
          "author_name": "themonologue",
          "author_url": "",
          "post_date": "12/27/2019 06:43:00",
          "content": "<p>All the best! Thanks for sharing though <a href=\"/wittmannf\">@wittmannf</a> </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 703706,
      "author_name": "sheriytm",
      "author_url": "",
      "post_date": "12/26/2019 14:29:13",
      "content": "<p><a href=\"/wittmannf\">@wittmannf</a> , thanks for sharing.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 703727,
      "author_name": "decarmelo",
      "author_url": "",
      "post_date": "12/26/2019 15:15:57",
      "content": "<p>My solution is very similar wtih yours,hope that we don't have over-fitted.</p>",
      "votes": null,
      "replies": [
        {
          "id": 718118,
          "author_name": "wittmannf",
          "author_url": "",
          "post_date": "01/14/2020 03:09:06",
          "content": "<p>Congrats on the 21st place <a href=\"/decarmelo\">@decarmelo</a> ! And we can confirm that we didn't overfit (maybe mine did a little). But I noticed that your LB position actually increased!! </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 703780,
      "author_name": "caesarlupum",
      "author_url": "",
      "post_date": "12/26/2019 16:02:50",
      "content": "<p>Thanks for sharing <a href=\"/wittmannf\">@wittmannf</a>! Congrats !</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 703847,
      "author_name": "lucamassaron",
      "author_url": "",
      "post_date": "12/26/2019 17:52:44",
      "content": "<p>We also found out that NNs and building based models didn't work as expected :-) Your blending solution is quite interesting, we didn't try using ridgeCV because we were afraid of negative coefficients, but they could have their sense, if they work out smoothing certain predictions as your strategy demonstrates. What was your C parameters in the end?</p>",
      "votes": null,
      "replies": [
        {
          "id": 718116,
          "author_name": "wittmannf",
          "author_url": "",
          "post_date": "01/14/2020 03:05:39",
          "content": "<p>Hey <a href=\"/lucamassaron\">@lucamassaron</a> , sorry for the delay to reply you! Yeah, turns out negative coeffs actually help. This is something I wasn't expecting. I am quite curious on checking in different datasets. Regarding the C value, next time I run the kernel I will check! But my guess is that gridsearchCV chose a small C value since the amplitude of the weights is not that high. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 703855,
      "author_name": "mimoudata",
      "author_url": "",
      "post_date": "12/26/2019 18:07:08",
      "content": "<p>Thanks for sharing! (2 Kernels from the ones you used are mine 😋 )</p>",
      "votes": null,
      "replies": [
        {
          "id": 718109,
          "author_name": "wittmannf",
          "author_url": "",
          "post_date": "01/14/2020 02:58:37",
          "content": "<p>Thanks <a href=\"/mimoudata\">@mimoudata</a> ! They were very helpful! :) </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 703884,
      "author_name": "rohanrao",
      "author_url": "",
      "post_date": "12/26/2019 18:58:22",
      "content": "<p>Thanks for sharing.\nThe simplest of ideas are often the hardest to find.</p>",
      "votes": null,
      "replies": [
        {
          "id": 718110,
          "author_name": "wittmannf",
          "author_url": "",
          "post_date": "01/14/2020 02:59:00",
          "content": "<p>Thanks! and congrats on the 2nd place! </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 703961,
      "author_name": "hanjoonchoe",
      "author_url": "",
      "post_date": "12/26/2019 21:41:33",
      "content": "<p>I think this competition is about blending of blendIng of blending of ... in my impression. XD.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 703964,
      "author_name": "teeyee314",
      "author_url": "",
      "post_date": "12/26/2019 21:51:18",
      "content": "<p>nice solution</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 704089,
      "author_name": "yamsam",
      "author_url": "",
      "post_date": "12/27/2019 03:28:14",
      "content": "<p>Thanks for sharing!! I didn't realize RidgeCV could work.</p>",
      "votes": null,
      "replies": [
        {
          "id": 718111,
          "author_name": "wittmannf",
          "author_url": "",
          "post_date": "01/14/2020 02:59:44",
          "content": "<p>You are welcome! And congrats on the 1st place!!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 712190,
      "author_name": "corochann",
      "author_url": "",
      "post_date": "01/06/2020 23:27:28",
      "content": "<p>Thanks for sharing</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 717394,
      "author_name": "claytonmiller",
      "author_url": "",
      "post_date": "01/13/2020 04:33:18",
      "content": "<p>Thank you very much <a href=\"/wittmannf\">@wittmannf</a> for sharing! We would like to add your solution to our analysis of the top processes and models. Please let us know how you would like to be acknowledged. The default will be what you have on your profile. Thanks again!</p>\n\n<p><strong>Also, for everyone else, we are in the process of curating a directory of Notebooks for building performance engineers and analysts from the competition that will serve as a recipe book for beginners as well as experts who are curious to learn about certain topics. We highly encourage you all to share your solutions (or any part of your process) as a notebook for this purpose. We will acknowledge the authors of all notebooks in this process, so please annotate your notebook or profile with how you would like to be acknowledged. We will also include general solutions and process information that is not in notebooks as well. It would also be helpful if you could indicate whether you are an ASHRAE member.</strong></p>",
      "votes": null,
      "replies": [
        {
          "id": 721056,
          "author_name": "wittmannf",
          "author_url": "",
          "post_date": "01/17/2020 01:21:22",
          "content": "<p>Thanks <a href=\"/claytonmiller\">@claytonmiller</a>! It's okay to use what's on my profile </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "703682": "Kernel: https://www.kaggle.com/wittmannf/0-939-lb-public-blend-leak-valid-ridgecv/\n\nMost of my time was actually spent on models that turned out to not score as good as the public kernels (NNs and bldg based models). My 18th is thanks to some side experimentation when trying to maximize the tips that have been provided in public Kernels and discussion. \n\nThe solution consists of a blend of non-leaked public submissions (I enhanced some of them) using Ridge CV's coefficients when fitting against the leak validation data. For the enhanced models, I discovered that it was more effective to combine them using RidgeCV's coeffs from leak validation data than just averaging them. However, I only had time to that in the `simple-data-cleanup-3-models`. My public score was 0.939 with leak replacement and 1.04ish without leak replacement. \n\nAnd here's something interesting: RidgeCV figured out negative coefficient for some submissions, although the sum of all of them was close to 1 (as expected):\n\n```\n## Ridge Coefficients\nSum of coefficients: 0.9994466580815099\nhalf-half-drop-rows-stratify-weekday has weight 0.14\nsimple-data-cleanup-3-models has weight 0.26\nashrae-kfold-lightgbm-without-leak-1-08 has weight -0.23\nanother-1-08-lb-no-leak has weight -0.46\nashrae-kfold-lightgbm-without-building-id has weight 0.19\nashrae-energy-prediction-using-stratified-kfold has weight 0.52\nashrae-lightgbm-without-leak has weight -0.15\nashrae-stratified-kfold-lightgbm has weight 0.23\nashrae-2-lightgbm-without-leak-data has weight 0.50\n```\n\nAll credits go to the authors of the public kernels and @yamsam 's idea of leak validation. My contribution was the idea of combining them using RidgeCV. \n\n### UPDATE\nAs mentioned in [this discussion](https://www.kaggle.com/c/ashrae-energy-prediction/discussion/125843), the following kernel would've ranked between 13th and 16th position of private leaderboard:\n- https://www.kaggle.com/wittmannf/blender-leak-validation-inception?scriptVersionId=25306647\n\nThe only difference is that I performed a second level in the blending process (`submission.csv` is the output from the first blend):\n```\nsubmission_paths = [\n'/kaggle/input/ashrae-half-and-half-w-drop-rows-stratify/submission.csv', #1.106\n'/kaggle/input/ashrae-simple-data-cleanup-lb-1-08-no-leaks/submission.csv',\n'/kaggle/input/ashrae-kfold-lightgbm-without-leak-1-08/submission.csv',\n'/kaggle/input/another-1-08-lb-no-leak/fe2_lgbm.csv',\n'/kaggle/input/ashrae-kfold-lightgbm-without-building-id/submission.csv', #1.098\n'/kaggle/input/ashrae-energy-prediction-using-stratified-kfold/fe2_lgbm.csv', #1.074\n'/kaggle/input/ashrae-lightgbm-without-leak/submission.csv', #1.082\n'/kaggle/input/ashrae-stratified-kfold-lightgbm/submission.csv',#1.075\n'./submission.csv'#1.04\n]\n```",
    "703697": "You used the leak data for validation which isn't in the private lb, don't you think you might have over-fitted?",
    "703699": "It is a tiny transfer learning. I don't think it causes any overfitting with such a simple model and big data. If we all don't shake-up due to seasonality, I don't think he will shake-up. \n\nGood idea @wittmannf ! I was going to do the same but didn't expect public kernels to be that much valuable, therefore didn't invest time on it.",
    "703704": "We will know who is right in a few days :-)",
    "703706": "wittmannf , thanks for sharing.",
    "703727": "My solution is very similar wtih yours,hope that we don't have over-fitted.",
    "703780": "Thanks for sharing @wittmannf! Congrats !",
    "703799": "I don't think it was overfitting since the score was improving in non-leaked sites. \n\nHowever, another type of overfitting sometimes happened. The leak validation score was a good reference for the leaderboard as long as I kept the model simple. When trying non-linear models like NNs for combining the submissions, I was able to get great validation scores, but it did not reflect in better leaderboard scores.",
    "703847": "We also found out that NNs and building based models didn't work as expected :-) Your blending solution is quite interesting, we didn't try using ridgeCV because we were afraid of negative coefficients, but they could have their sense, if they work out smoothing certain predictions as your strategy demonstrates. What was your C parameters in the end?",
    "703855": "Thanks for sharing! (2 Kernels from the ones you used are mine 😋 )",
    "703884": "Thanks for sharing.\nThe simplest of ideas are often the hardest to find.",
    "703961": "I think this competition is about blending of blendIng of blending of ... in my impression. XD.",
    "703964": "nice solution",
    "704089": "Thanks for sharing!! I didn't realize RidgeCV could work.",
    "704197": "All the best! Thanks for sharing though @wittmannf",
    "712190": "Thanks for sharing",
    "717394": "Thank you very much @wittmannf for sharing! We would like to add your solution to our analysis of the top processes and models. Please let us know how you would like to be acknowledged. The default will be what you have on your profile. Thanks again!\n\n**Also, for everyone else, we are in the process of curating a directory of Notebooks for building performance engineers and analysts from the competition that will serve as a recipe book for beginners as well as experts who are curious to learn about certain topics. We highly encourage you all to share your solutions (or any part of your process) as a notebook for this purpose. We will acknowledge the authors of all notebooks in this process, so please annotate your notebook or profile with how you would like to be acknowledged. We will also include general solutions and process information that is not in notebooks as well. It would also be helpful if you could indicate whether you are an ASHRAE member.**",
    "718109": "Thanks @mimoudata ! They were very helpful! :)",
    "718110": "Thanks! and congrats on the 2nd place!",
    "718111": "You are welcome! And congrats on the 1st place!!",
    "718116": "Hey @lucamassaron , sorry for the delay to reply you! Yeah, turns out negative coeffs actually help. This is something I wasn't expecting. I am quite curious on checking in different datasets. Regarding the C value, next time I run the kernel I will check! But my guess is that gridsearchCV chose a small C value since the amplitude of the weights is not that high.",
    "718118": "Congrats on the 21st place @decarmelo ! And we can confirm that we didn't overfit (maybe mine did a little). But I noticed that your LB position actually increased!!",
    "721056": "Thanks @claytonmiller! It's okay to use what's on my profile"
  },
  "source": "meta"
}