{
  "id": 121131,
  "title": "My First Kaggle Competition",
  "url": "/competitions/ashrae-energy-prediction/discussion/121131",
  "author_name": "Qua[D]",
  "post_date": "2019-12-11T10:28:15.289000",
  "votes": 7,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Hi All,</p>\n\n<p>This is my first Kaggle competition. Although I'm following Kaggle for quite some time now but I guess I was intimated to participate. Also, I don't know much of Deep learning so I don't take part in computer vision challenges (which dominates Kaggle competitions). Let me share my experience with everyone.</p>\n\n<p>I'm working on this data from my work place private network, where I do it whenever I have some free time. So, I don't have access to LightGBM algorithm (as it is not available in CRAN). I'm an RStudio fan boy (is that a good thing?) so I have used 'h2o' package for building models - GLM/RF/GBM (sadly h2o does not support XGB on windows).</p>\n\n<p>I decided to take multiple approaches and not use LightGBM until 1 week prior to the competition deadline (unless I'm able to improve my score beyond the score achieved by people using LightGBM).</p>\n\n<p>Approach 1 - Create models based on Building-Meter combination\nIf we club the columns building id &amp; meter - then there are 2380 such unique id's. For these 2380 id's using the meter_reading column I created 10 clusters. Then I built 10 different models for these 10 clusters.\nUsing GLM I could achieve highest RMSLE on Public LB: 2.11\nUsing DRF I could achieve highest RMSLE on Public LB: 1.53\nUsing GBM I could achieve highest RMSLE on Public LB: 1.31</p>\n\n<p>N.B - I tried creating 12/15/20 clusters but that did not seem to improve the Public LB RMSLE much</p>\n\n<p>Approach 2 - Then I came accross the AC/DC kernel (Thank you <a href=\"/kailex\">@kailex</a> ) while searching for R notebooks to check what other people are doing (I was surprised there are hardly R kernels in the competition!). I got to know the half-half approach. Then used that approach in my GBM model. </p>\n\n<p>Remark - After reading lot of discussion threads I realised that somehow no one is noting down their mean &amp; standard deviation (which is a very good indicator of bias &amp; variance of prediction) of the submission. I had noted my submission mean &amp; sd while I was tuning my GBM model. Below is an account of that -\nmean of prediction 953 (sd 44458.02)  - Public LB: 1.18\nmean of prediction 913 (sd 42121.20)  - Public LB: 1.17\nmean of prediction 885 (sd 39729.71)  - Public LB: 1.19\nmean of prediction 848 (sd 37641.77)  - Public LB: 1.19\nmean of prediction 832 (sd 37447.77)  - Public LB: 1.16\nmean of prediction 811 (sd 35553.71)  - Public LB: 1.18\nmean of prediction 752 (sd 32774.71)  - Public LB: 1.16</p>\n\n<p>Then I split the dataset based on meter type so I had 4 different models. Below is an account of that -\nmean of prediction 1071 (sd 54839.47) - Public LB: 1.17\nmean of prediction 1010 (sd 51735.35) - Public LB: 1.16\nmean of prediction 960  (sd 49148.58) - Public LB: 1.16\nmean of prediction 909  (sd 46561.81) - Public LB: 1.15\nmean of prediction 859  (sd 43975.04) - Public LB: 1.15\nmean of prediction 808  (sd 41388.28) - Public LB: 1.15\nmean of prediction 787  (sd 39012.36) - Public LB: 1.16</p>\n\n<p>After lot of tuning I could achieve highest RMSLE on Public LB: 1.15</p>\n\n<p>Approach 3 - I decided to use the Half Half kernel (Thanks to <a href=\"/rohanrao\">@rohanrao</a> ) and generate the LGBM output and downloaded the output to compare the mean &amp; sd of that output against my best output. </p>\n\n<p>my best submission   : mean = 808-909 ; sd = 41388-56561\nhalf half submission : mean = 555     ; sd = 13123</p>\n\n<p>mean of prediction 555  (sd 13123.09) - Public LB: 1.10\nmean of prediction 499  (sd 11810.78) - Public LB: 1.10 [rank improved by 100] [In this submission I had just reduced the Half-Half prediction by 10%]</p>\n\n<p>My aim of noting mean &amp; sd of submission is to understand the effect of reduction of prediction bias &amp; variance on RMSLE metric. Although, I have not been able to understand it completely but I feel reducing the prediction mean &amp; variance reduces RMSLE to a certain extent then it increases (I guess under prediction is penalised more by RMSLE that's why!). While, same mean (808 vs 811) with different sd (41388 vs 35553) lead to pretty different RMSLE. I hope the experts over here will be able to provide more insights about this.</p>\n\n<p>Quad~</p>\n\n<p>Edit 1 :  I have tested 3 scenarios since yesterday! Different mean &amp; similar standard deviation gave more public LB RMSLE and Similar mean &amp; different standard deviation gave even more public LB RMSLE.\nmean of prediction 499  (sd 11810.78) - Public LB: 1.10\nmean of prediction 427  (sd 10104.78) - Public LB: 1.11\nmean of prediction 432  (sd 21456.80) - Public LB: 1.35\nmean of prediction 340  (sd 1505.76) -   Public LB: 1.08  [Just for the sake of comparison, I ran a 1.08 score kernel &amp; checked these values, thanks <a href=\"/aitude\">@aitude</a> ]\nThere is a huge difference between 1.1 (mean &amp; sd) vs 1.08 (mean &amp; sd). Clearly, 1.08 submission is not under predicting which means 1.1 is over predicting. I'm not able to understand how much it's over predicting.</p>",
  "messages": [
    {
      "id": 692495,
      "postDate": "2019-12-11T10:28:15.290Z",
      "content": "<p>Hi All,</p>\n\n<p>This is my first Kaggle competition. Although I'm following Kaggle for quite some time now but I guess I was intimated to participate. Also, I don't know much of Deep learning so I don't take part in computer vision challenges (which dominates Kaggle competitions). Let me share my experience with everyone.</p>\n\n<p>I'm working on this data from my work place private network, where I do it whenever I have some free time. So, I don't have access to LightGBM algorithm (as it is not available in CRAN). I'm an RStudio fan boy (is that a good thing?) so I have used 'h2o' package for building models - GLM/RF/GBM (sadly h2o does not support XGB on windows).</p>\n\n<p>I decided to take multiple approaches and not use LightGBM until 1 week prior to the competition deadline (unless I'm able to improve my score beyond the score achieved by people using LightGBM).</p>\n\n<p>Approach 1 - Create models based on Building-Meter combination\nIf we club the columns building id &amp; meter - then there are 2380 such unique id's. For these 2380 id's using the meter_reading column I created 10 clusters. Then I built 10 different models for these 10 clusters.\nUsing GLM I could achieve highest RMSLE on Public LB: 2.11\nUsing DRF I could achieve highest RMSLE on Public LB: 1.53\nUsing GBM I could achieve highest RMSLE on Public LB: 1.31</p>\n\n<p>N.B - I tried creating 12/15/20 clusters but that did not seem to improve the Public LB RMSLE much</p>\n\n<p>Approach 2 - Then I came accross the AC/DC kernel (Thank you <a href=\"/kailex\">@kailex</a> ) while searching for R notebooks to check what other people are doing (I was surprised there are hardly R kernels in the competition!). I got to know the half-half approach. Then used that approach in my GBM model. </p>\n\n<p>Remark - After reading lot of discussion threads I realised that somehow no one is noting down their mean &amp; standard deviation (which is a very good indicator of bias &amp; variance of prediction) of the submission. I had noted my submission mean &amp; sd while I was tuning my GBM model. Below is an account of that -\nmean of prediction 953 (sd 44458.02)  - Public LB: 1.18\nmean of prediction 913 (sd 42121.20)  - Public LB: 1.17\nmean of prediction 885 (sd 39729.71)  - Public LB: 1.19\nmean of prediction 848 (sd 37641.77)  - Public LB: 1.19\nmean of prediction 832 (sd 37447.77)  - Public LB: 1.16\nmean of prediction 811 (sd 35553.71)  - Public LB: 1.18\nmean of prediction 752 (sd 32774.71)  - Public LB: 1.16</p>\n\n<p>Then I split the dataset based on meter type so I had 4 different models. Below is an account of that -\nmean of prediction 1071 (sd 54839.47) - Public LB: 1.17\nmean of prediction 1010 (sd 51735.35) - Public LB: 1.16\nmean of prediction 960  (sd 49148.58) - Public LB: 1.16\nmean of prediction 909  (sd 46561.81) - Public LB: 1.15\nmean of prediction 859  (sd 43975.04) - Public LB: 1.15\nmean of prediction 808  (sd 41388.28) - Public LB: 1.15\nmean of prediction 787  (sd 39012.36) - Public LB: 1.16</p>\n\n<p>After lot of tuning I could achieve highest RMSLE on Public LB: 1.15</p>\n\n<p>Approach 3 - I decided to use the Half Half kernel (Thanks to <a href=\"/rohanrao\">@rohanrao</a> ) and generate the LGBM output and downloaded the output to compare the mean &amp; sd of that output against my best output. </p>\n\n<p>my best submission   : mean = 808-909 ; sd = 41388-56561\nhalf half submission : mean = 555     ; sd = 13123</p>\n\n<p>mean of prediction 555  (sd 13123.09) - Public LB: 1.10\nmean of prediction 499  (sd 11810.78) - Public LB: 1.10 [rank improved by 100] [In this submission I had just reduced the Half-Half prediction by 10%]</p>\n\n<p>My aim of noting mean &amp; sd of submission is to understand the effect of reduction of prediction bias &amp; variance on RMSLE metric. Although, I have not been able to understand it completely but I feel reducing the prediction mean &amp; variance reduces RMSLE to a certain extent then it increases (I guess under prediction is penalised more by RMSLE that's why!). While, same mean (808 vs 811) with different sd (41388 vs 35553) lead to pretty different RMSLE. I hope the experts over here will be able to provide more insights about this.</p>\n\n<p>Quad~</p>\n\n<p>Edit 1 :  I have tested 3 scenarios since yesterday! Different mean &amp; similar standard deviation gave more public LB RMSLE and Similar mean &amp; different standard deviation gave even more public LB RMSLE.\nmean of prediction 499  (sd 11810.78) - Public LB: 1.10\nmean of prediction 427  (sd 10104.78) - Public LB: 1.11\nmean of prediction 432  (sd 21456.80) - Public LB: 1.35\nmean of prediction 340  (sd 1505.76) -   Public LB: 1.08  [Just for the sake of comparison, I ran a 1.08 score kernel &amp; checked these values, thanks <a href=\"/aitude\">@aitude</a> ]\nThere is a huge difference between 1.1 (mean &amp; sd) vs 1.08 (mean &amp; sd). Clearly, 1.08 submission is not under predicting which means 1.1 is over predicting. I'm not able to understand how much it's over predicting.</p>",
      "rawMarkdown": "Hi All,\n\nThis is my first Kaggle competition. Although I'm following Kaggle for quite some time now but I guess I was intimated to participate. Also, I don't know much of Deep learning so I don't take part in computer vision challenges (which dominates Kaggle competitions). Let me share my experience with everyone.\n\nI'm working on this data from my work place private network, where I do it whenever I have some free time. So, I don't have access to LightGBM algorithm (as it is not available in CRAN). I'm an RStudio fan boy (is that a good thing?) so I have used 'h2o' package for building models - GLM/RF/GBM (sadly h2o does not support XGB on windows).\n\nI decided to take multiple approaches and not use LightGBM until 1 week prior to the competition deadline (unless I'm able to improve my score beyond the score achieved by people using LightGBM).\n\nApproach 1 - Create models based on Building-Meter combination\nIf we club the columns building id &amp; meter - then there are 2380 such unique id's. For these 2380 id's using the meter_reading column I created 10 clusters. Then I built 10 different models for these 10 clusters.\nUsing GLM I could achieve highest RMSLE on Public LB: 2.11\nUsing DRF I could achieve highest RMSLE on Public LB: 1.53\nUsing GBM I could achieve highest RMSLE on Public LB: 1.31\n\nN.B - I tried creating 12/15/20 clusters but that did not seem to improve the Public LB RMSLE much\n\nApproach 2 - Then I came accross the AC/DC kernel (Thank you @kailex ) while searching for R notebooks to check what other people are doing (I was surprised there are hardly R kernels in the competition!). I got to know the half-half approach. Then used that approach in my GBM model. \n\nRemark - After reading lot of discussion threads I realised that somehow no one is noting down their mean &amp; standard deviation (which is a very good indicator of bias &amp; variance of prediction) of the submission. I had noted my submission mean &amp; sd while I was tuning my GBM model. Below is an account of that -\nmean of prediction 953 (sd 44458.02)  - Public LB: 1.18\nmean of prediction 913 (sd 42121.20)  - Public LB: 1.17\nmean of prediction 885 (sd 39729.71)  - Public LB: 1.19\nmean of prediction 848 (sd 37641.77)  - Public LB: 1.19\nmean of prediction 832 (sd 37447.77)  - Public LB: 1.16\nmean of prediction 811 (sd 35553.71)  - Public LB: 1.18\nmean of prediction 752 (sd 32774.71)  - Public LB: 1.16\n\nThen I split the dataset based on meter type so I had 4 different models. Below is an account of that -\nmean of prediction 1071 (sd 54839.47) - Public LB: 1.17\nmean of prediction 1010 (sd 51735.35) - Public LB: 1.16\nmean of prediction 960  (sd 49148.58) - Public LB: 1.16\nmean of prediction 909  (sd 46561.81) - Public LB: 1.15\nmean of prediction 859  (sd 43975.04) - Public LB: 1.15\nmean of prediction 808  (sd 41388.28) - Public LB: 1.15\nmean of prediction 787  (sd 39012.36) - Public LB: 1.16\n\nAfter lot of tuning I could achieve highest RMSLE on Public LB: 1.15\n\nApproach 3 - I decided to use the Half Half kernel (Thanks to @rohanrao ) and generate the LGBM output and downloaded the output to compare the mean &amp; sd of that output against my best output. \n \nmy best submission   : mean = 808-909 ; sd = 41388-56561\nhalf half submission : mean = 555     ; sd = 13123\n\nmean of prediction 555  (sd 13123.09) - Public LB: 1.10\nmean of prediction 499  (sd 11810.78) - Public LB: 1.10 [rank improved by 100] [In this submission I had just reduced the Half-Half prediction by 10%]\n\nMy aim of noting mean &amp; sd of submission is to understand the effect of reduction of prediction bias &amp; variance on RMSLE metric. Although, I have not been able to understand it completely but I feel reducing the prediction mean &amp; variance reduces RMSLE to a certain extent then it increases (I guess under prediction is penalised more by RMSLE that's why!). While, same mean (808 vs 811) with different sd (41388 vs 35553) lead to pretty different RMSLE. I hope the experts over here will be able to provide more insights about this.\n\nQuad~\n\nEdit 1 :  I have tested 3 scenarios since yesterday! Different mean &amp; similar standard deviation gave more public LB RMSLE and Similar mean &amp; different standard deviation gave even more public LB RMSLE.\nmean of prediction 499  (sd 11810.78) - Public LB: 1.10\nmean of prediction 427  (sd 10104.78) - Public LB: 1.11\nmean of prediction 432  (sd 21456.80) - Public LB: 1.35\nmean of prediction 340  (sd 1505.76) -   Public LB: 1.08  [Just for the sake of comparison, I ran a 1.08 score kernel &amp; checked these values, thanks @aitude ]\nThere is a huge difference between 1.1 (mean &amp; sd) vs 1.08 (mean &amp; sd). Clearly, 1.08 submission is not under predicting which means 1.1 is over predicting. I'm not able to understand how much it's over predicting.",
      "votes": 6
    },
    {
      "id": 692519,
      "postDate": "2019-12-11T11:29:35.460Z",
      "content": "<p>Hi <a href=\"/smydpchowdhury\">@smydpchowdhury</a> </p>\n\n<p>Good luck in your first competition. You made great points, but it is not that simple. You have to understand meanings behind the patterns.</p>",
      "rawMarkdown": "Hi @smydpchowdhury \n\nGood luck in your first competition. You made great points, but it is not that simple. You have to understand meanings behind the patterns.",
      "votes": 1,
      "replies": [
        {
          "id": 693083,
          "postDate": "2019-12-12T02:42:53.980Z",
          "content": "<p><a href=\"/gunesevitan\">@gunesevitan</a> One thing I feel is - submission mean is much closer to 500 than 800 and std dev is closer to 12000 which implies readings for the buildings like 1099, 994 etc are indeed useless for prediction. What's your opinion on that?</p>",
          "rawMarkdown": "@gunesevitan One thing I feel is - submission mean is much closer to 500 than 800 and std dev is closer to 12000 which implies readings for the buildings like 1099, 994 etc are indeed useless for prediction. What's your opinion on that?"
        }
      ]
    },
    {
      "id": 692520,
      "postDate": "2019-12-11T11:33:55.307Z",
      "rawMarkdown": "",
      "votes": 3,
      "isDeleted": true
    },
    {
      "id": 692517,
      "postDate": "2019-12-11T11:14:42.383Z",
      "rawMarkdown": "",
      "votes": -1,
      "isDeleted": true,
      "replies": [
        {
          "id": 693716,
          "postDate": "2019-12-12T18:01:37.550Z",
          "content": "<p>What do you mean with &nbsp;GO FOR PYTHON , R IS DEAD…! ???</p>\n\n<p>Just explain please.</p>",
          "rawMarkdown": "What do you mean with &nbsp;GO FOR PYTHON , R IS DEAD…! ???\n\nJust explain please.",
          "votes": 1
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 692519,
      "author_name": "Gunes Evitan",
      "author_url": "",
      "post_date": "2019-12-11T11:29:35.460000",
      "content": "<p>Hi <a href=\"/smydpchowdhury\">@smydpchowdhury</a> </p>\n\n<p>Good luck in your first competition. You made great points, but it is not that simple. You have to understand meanings behind the patterns.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 693083,
          "author_name": "Qua[D]",
          "author_url": "",
          "post_date": "2019-12-12T02:42:53.980000",
          "content": "<p><a href=\"/gunesevitan\">@gunesevitan</a> One thing I feel is - submission mean is much closer to 500 than 800 and std dev is closer to 12000 which implies readings for the buildings like 1099, 994 etc are indeed useless for prediction. What's your opinion on that?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 692520,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-11T11:33:55.307000",
      "content": "",
      "votes": 3,
      "replies": []
    },
    {
      "id": 692517,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-11T11:14:42.383000",
      "content": "",
      "votes": -1,
      "replies": [
        {
          "id": 693716,
          "author_name": "mezoganet",
          "author_url": "",
          "post_date": "2019-12-12T18:01:37.550000",
          "content": "<p>What do you mean with &nbsp;GO FOR PYTHON , R IS DEAD…! ???</p>\n\n<p>Just explain please.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "692495": "Hi All,\n\nThis is my first Kaggle competition. Although I'm following Kaggle for quite some time now but I guess I was intimated to participate. Also, I don't know much of Deep learning so I don't take part in computer vision challenges (which dominates Kaggle competitions). Let me share my experience with everyone.\n\nI'm working on this data from my work place private network, where I do it whenever I have some free time. So, I don't have access to LightGBM algorithm (as it is not available in CRAN). I'm an RStudio fan boy (is that a good thing?) so I have used 'h2o' package for building models - GLM/RF/GBM (sadly h2o does not support XGB on windows).\n\nI decided to take multiple approaches and not use LightGBM until 1 week prior to the competition deadline (unless I'm able to improve my score beyond the score achieved by people using LightGBM).\n\nApproach 1 - Create models based on Building-Meter combination\nIf we club the columns building id &amp; meter - then there are 2380 such unique id's. For these 2380 id's using the meter_reading column I created 10 clusters. Then I built 10 different models for these 10 clusters.\nUsing GLM I could achieve highest RMSLE on Public LB: 2.11\nUsing DRF I could achieve highest RMSLE on Public LB: 1.53\nUsing GBM I could achieve highest RMSLE on Public LB: 1.31\n\nN.B - I tried creating 12/15/20 clusters but that did not seem to improve the Public LB RMSLE much\n\nApproach 2 - Then I came accross the AC/DC kernel (Thank you @kailex ) while searching for R notebooks to check what other people are doing (I was surprised there are hardly R kernels in the competition!). I got to know the half-half approach. Then used that approach in my GBM model. \n\nRemark - After reading lot of discussion threads I realised that somehow no one is noting down their mean &amp; standard deviation (which is a very good indicator of bias &amp; variance of prediction) of the submission. I had noted my submission mean &amp; sd while I was tuning my GBM model. Below is an account of that -\nmean of prediction 953 (sd 44458.02)  - Public LB: 1.18\nmean of prediction 913 (sd 42121.20)  - Public LB: 1.17\nmean of prediction 885 (sd 39729.71)  - Public LB: 1.19\nmean of prediction 848 (sd 37641.77)  - Public LB: 1.19\nmean of prediction 832 (sd 37447.77)  - Public LB: 1.16\nmean of prediction 811 (sd 35553.71)  - Public LB: 1.18\nmean of prediction 752 (sd 32774.71)  - Public LB: 1.16\n\nThen I split the dataset based on meter type so I had 4 different models. Below is an account of that -\nmean of prediction 1071 (sd 54839.47) - Public LB: 1.17\nmean of prediction 1010 (sd 51735.35) - Public LB: 1.16\nmean of prediction 960  (sd 49148.58) - Public LB: 1.16\nmean of prediction 909  (sd 46561.81) - Public LB: 1.15\nmean of prediction 859  (sd 43975.04) - Public LB: 1.15\nmean of prediction 808  (sd 41388.28) - Public LB: 1.15\nmean of prediction 787  (sd 39012.36) - Public LB: 1.16\n\nAfter lot of tuning I could achieve highest RMSLE on Public LB: 1.15\n\nApproach 3 - I decided to use the Half Half kernel (Thanks to @rohanrao ) and generate the LGBM output and downloaded the output to compare the mean &amp; sd of that output against my best output. \n \nmy best submission   : mean = 808-909 ; sd = 41388-56561\nhalf half submission : mean = 555     ; sd = 13123\n\nmean of prediction 555  (sd 13123.09) - Public LB: 1.10\nmean of prediction 499  (sd 11810.78) - Public LB: 1.10 [rank improved by 100] [In this submission I had just reduced the Half-Half prediction by 10%]\n\nMy aim of noting mean &amp; sd of submission is to understand the effect of reduction of prediction bias &amp; variance on RMSLE metric. Although, I have not been able to understand it completely but I feel reducing the prediction mean &amp; variance reduces RMSLE to a certain extent then it increases (I guess under prediction is penalised more by RMSLE that's why!). While, same mean (808 vs 811) with different sd (41388 vs 35553) lead to pretty different RMSLE. I hope the experts over here will be able to provide more insights about this.\n\nQuad~\n\nEdit 1 :  I have tested 3 scenarios since yesterday! Different mean &amp; similar standard deviation gave more public LB RMSLE and Similar mean &amp; different standard deviation gave even more public LB RMSLE.\nmean of prediction 499  (sd 11810.78) - Public LB: 1.10\nmean of prediction 427  (sd 10104.78) - Public LB: 1.11\nmean of prediction 432  (sd 21456.80) - Public LB: 1.35\nmean of prediction 340  (sd 1505.76) -   Public LB: 1.08  [Just for the sake of comparison, I ran a 1.08 score kernel &amp; checked these values, thanks @aitude ]\nThere is a huge difference between 1.1 (mean &amp; sd) vs 1.08 (mean &amp; sd). Clearly, 1.08 submission is not under predicting which means 1.1 is over predicting. I'm not able to understand how much it's over predicting.",
    "692519": "Hi @smydpchowdhury \n\nGood luck in your first competition. You made great points, but it is not that simple. You have to understand meanings behind the patterns.",
    "692520": "",
    "692517": ""
  }
}