Thanks, Kaggle and ASHRAE for hosting this competition. Maybe, my place isn’t high enough to post my view, and in later competitions I won’t post until top 20-30, but that was my second participation and first silver! So, I have an overwhelming excitement and just have to share my experience with such brilliant minds.
\n\n• CV and final model\n• Clearing dataset\n• Feature engineering\n• Sadly, didn’t try to postprocess
\n\nFor feature selection and engineering in CV I only used leaked data with sites 0,1,2,15. 2 folds (split by months) and compare predictions of 2017-2018 with real readings from leak data. \nWith right parameters it fast and you can actually see if feature useful or not. Yes, you can overfit to this sites, but I created only general features (at least tried), checked performance a few times on public LB and decided to use it.
\n\nFor final model I excluded sites 0,1,2,15 and trained without them with same 2 fold strategy. It improved private score from 1.266 to 1.262.\nI wish I had time to add CatBoost but this time I used only 3 lightgbm models with different seeds and same parameters + blend them with 3 top public kernels without leaks with little different strategy from one another to add diversity.
\n\no Ashrae Energy Prediction using Stratified KFold\no ASHRAE 2*lightGBM without Leak Data\no Ashrae: simple data cleanup (LB 1.08 no leaks)
\n\nBlend weights: 0.66 for my 3 models, rest for public kernels.
\n\nMy best LGBM parameters:
\n\n\n{'num_leaves': 500,
\n 'objective': 'regression',\n 'learning_rate': 0.04,\n 'boosting': 'gbdt',\n 'subsample': 0.4,\n 'feature_fraction': 0.7,\n 'n_jobs': -1,\n 'seed': 19,\n 'metric': 'rmse'}\n
Used Robert Stockton kernel, learned a lot, thanks!\nManually cleaned some worst readings by meter type, by RMSLE error, buildings in sites 9,6,10. \nI don’t think, that I should have done some of the manual cleaning based only on my intuition instead of actual understanding what those outliers mean. Maybe, I cleaned what seemed like an outlier to me but it wasn’t and presented in test data. I think so, because I have 2 models which scores higher than my final submission and best one is single model without late manual cleaning that scores 1.25 on private.
\n\nI manually checked all existed features, features from kernels, features from my ideas, aggregations, frequency encodings and found only few of them useful. I deleted most of the initial features.\nI didn’t do any imputations (score decreased in CV), converted all categorical values to category type, square feet and meter readings to log1p.\nI even downloaded external weather history to each site with weather conditions etc. but it didn’t improve score of my CV so I decided not to use them.
\n\nI had lot of fun struggle through and learned a lot. Hope, you are having a wonderful day and looking forward to see all of you in the next competition!
", "messages": [ { "id": "714331", "postDate": "01/09/2020 09:59:23", "content": "Thanks, Kaggle and ASHRAE for hosting this competition. Maybe, my place isn’t high enough to post my view, and in later competitions I won’t post until top 20-30, but that was my second participation and first silver! So, I have an overwhelming excitement and just have to share my experience with such brilliant minds.
\n\n• CV and final model\n• Clearing dataset\n• Feature engineering\n• Sadly, didn’t try to postprocess
\n\nFor feature selection and engineering in CV I only used leaked data with sites 0,1,2,15. 2 folds (split by months) and compare predictions of 2017-2018 with real readings from leak data. \nWith right parameters it fast and you can actually see if feature useful or not. Yes, you can overfit to this sites, but I created only general features (at least tried), checked performance a few times on public LB and decided to use it.
\n\nFor final model I excluded sites 0,1,2,15 and trained without them with same 2 fold strategy. It improved private score from 1.266 to 1.262.\nI wish I had time to add CatBoost but this time I used only 3 lightgbm models with different seeds and same parameters + blend them with 3 top public kernels without leaks with little different strategy from one another to add diversity.
\n\no Ashrae Energy Prediction using Stratified KFold\no ASHRAE 2*lightGBM without Leak Data\no Ashrae: simple data cleanup (LB 1.08 no leaks)
\n\nBlend weights: 0.66 for my 3 models, rest for public kernels.
\n\nMy best LGBM parameters:
\n\n\n{'num_leaves': 500,
\n 'objective': 'regression',\n 'learning_rate': 0.04,\n 'boosting': 'gbdt',\n 'subsample': 0.4,\n 'feature_fraction': 0.7,\n 'n_jobs': -1,\n 'seed': 19,\n 'metric': 'rmse'}\n
Used Robert Stockton kernel, learned a lot, thanks!\nManually cleaned some worst readings by meter type, by RMSLE error, buildings in sites 9,6,10. \nI don’t think, that I should have done some of the manual cleaning based only on my intuition instead of actual understanding what those outliers mean. Maybe, I cleaned what seemed like an outlier to me but it wasn’t and presented in test data. I think so, because I have 2 models which scores higher than my final submission and best one is single model without late manual cleaning that scores 1.25 on private.
\n\nI manually checked all existed features, features from kernels, features from my ideas, aggregations, frequency encodings and found only few of them useful. I deleted most of the initial features.\nI didn’t do any imputations (score decreased in CV), converted all categorical values to category type, square feet and meter readings to log1p.\nI even downloaded external weather history to each site with weather conditions etc. but it didn’t improve score of my CV so I decided not to use them.
\n\nI had lot of fun struggle through and learned a lot. Hope, you are having a wonderful day and looking forward to see all of you in the next competition!
", "rawMarkdown": "Thanks, Kaggle and ASHRAE for hosting this competition. Maybe, my place isn’t high enough to post my view, and in later competitions I won’t post until top 20-30, but that was my second participation and first silver! So, I have an overwhelming excitement and just have to share my experience with such brilliant minds.\n\n### Key points:\n\n•\tCV and final model\n•\tClearing dataset\n•\tFeature engineering\n•\tSadly, didn’t try to postprocess\n\n### CV and final model\n\nFor feature selection and engineering in CV I only used leaked data with sites 0,1,2,15. 2 folds (split by months) and compare predictions of 2017-2018 with real readings from leak data. \nWith right parameters it fast and you can actually see if feature useful or not. Yes, you can overfit to this sites, but I created only general features (at least tried), checked performance a few times on public LB and decided to use it.\n\nFor final model I excluded sites 0,1,2,15 and trained without them with same 2 fold strategy. It improved private score from 1.266 to 1.262.\nI wish I had time to add CatBoost but this time I used only 3 lightgbm models with different seeds and same parameters + blend them with 3 top public kernels without leaks with little different strategy from one another to add diversity. \n\no\t[Ashrae Energy Prediction using Stratified KFold](https://www.kaggle.com/roydatascience/ashrae-energy-prediction-using-stratified-kfold)\no\t[ASHRAE 2*lightGBM without Leak Data](https://www.kaggle.com/mimoudata/ashrae-2-lightgbm-without-leak-data)\no\t[Ashrae: simple data cleanup (LB 1.08 no leaks)](https://www.kaggle.com/purist1024/ashrae-simple-data-cleanup-lb-1-08-no-leaks)\n\nBlend weights: 0.66 for my 3 models, rest for public kernels. \n\nMy best LGBM parameters:\n \n```\n{'num_leaves': 500, \n 'objective': 'regression',\n 'learning_rate': 0.04,\n 'boosting': 'gbdt',\n 'subsample': 0.4,\n 'feature_fraction': 0.7,\n 'n_jobs': -1,\n 'seed': 19,\n 'metric': 'rmse'}\n```\n\n### Clearing dataset\n\nUsed [Robert Stockton](https://www.kaggle.com/purist1024/ashrae-simple-data-cleanup-lb-1-08-no-leaks) kernel, learned a lot, thanks!\nManually cleaned some worst readings by meter type, by RMSLE error, buildings in sites 9,6,10. \nI don’t think, that I should have done some of the manual cleaning based only on my intuition instead of actual understanding what those outliers mean. Maybe, I cleaned what seemed like an outlier to me but it wasn’t and presented in test data. I think so, because I have 2 models which scores higher than my final submission and best one is single model without late manual cleaning that scores 1.25 on private.\n\n### Feature engineering\n\nI manually checked all existed features, features from kernels, features from my ideas, aggregations, frequency encodings and found only few of them useful. I deleted most of the initial features.\nI didn’t do any imputations (score decreased in CV), converted all categorical values to category type, square feet and meter readings to log1p.\nI even downloaded external weather history to each site with weather conditions etc. but it didn’t improve score of my CV so I decided not to use them. \n\n---\nI had lot of fun struggle through and learned a lot. Hope, you are having a wonderful day and looking forward to see all of you in the next competition!", "votes": null }, { "id": "717362", "postDate": "01/13/2020 03:55:46", "content": "Thank you very much for sharing! We would like to add your solution to our analysis of the top processes and models. Please let us know how you would like to be acknowledged. The default will be what you have on your profile. Thanks again!
\n\nAlso, for everyone else, we are in the process of curating a directory of Notebooks for building performance engineers and analysts from the competition that will serve as a recipe book for beginners as well as experts who are curious to learn about certain topics. We highly encourage you all to share your solutions (or any part of your process) as a notebook for this purpose. We will acknowledge the authors of all notebooks in this process, so please annotate your notebook or profile with how you would like to be acknowledged. We will also include general solutions and process information that is not in notebooks as well. It would also be helpful if you could indicate whether you are an ASHRAE member.
", "rawMarkdown": "Thank you very much for sharing! We would like to add your solution to our analysis of the top processes and models. Please let us know how you would like to be acknowledged. The default will be what you have on your profile. Thanks again!\n\n**Also, for everyone else, we are in the process of curating a directory of Notebooks for building performance engineers and analysts from the competition that will serve as a recipe book for beginners as well as experts who are curious to learn about certain topics. We highly encourage you all to share your solutions (or any part of your process) as a notebook for this purpose. We will acknowledge the authors of all notebooks in this process, so please annotate your notebook or profile with how you would like to be acknowledged. We will also include general solutions and process information that is not in notebooks as well. It would also be helpful if you could indicate whether you are an ASHRAE member.**", "votes": null }, { "id": "717630", "postDate": "01/13/2020 11:26:38", "content": "Profile info would be sufficient, thanks!
", "rawMarkdown": "Profile info would be sufficient, thanks!", "votes": null } ], "comments": [ { "id": 717362, "author_name": "claytonmiller", "author_url": "", "post_date": "01/13/2020 03:55:46", "content": "Thank you very much for sharing! We would like to add your solution to our analysis of the top processes and models. Please let us know how you would like to be acknowledged. The default will be what you have on your profile. Thanks again!
\n\nAlso, for everyone else, we are in the process of curating a directory of Notebooks for building performance engineers and analysts from the competition that will serve as a recipe book for beginners as well as experts who are curious to learn about certain topics. We highly encourage you all to share your solutions (or any part of your process) as a notebook for this purpose. We will acknowledge the authors of all notebooks in this process, so please annotate your notebook or profile with how you would like to be acknowledged. We will also include general solutions and process information that is not in notebooks as well. It would also be helpful if you could indicate whether you are an ASHRAE member.
", "votes": null, "replies": [ { "id": 717630, "author_name": "uspangaliev", "author_url": "", "post_date": "01/13/2020 11:26:38", "content": "Profile info would be sufficient, thanks!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "714331": "Thanks, Kaggle and ASHRAE for hosting this competition. Maybe, my place isn’t high enough to post my view, and in later competitions I won’t post until top 20-30, but that was my second participation and first silver! So, I have an overwhelming excitement and just have to share my experience with such brilliant minds.\n\n### Key points:\n\n•\tCV and final model\n•\tClearing dataset\n•\tFeature engineering\n•\tSadly, didn’t try to postprocess\n\n### CV and final model\n\nFor feature selection and engineering in CV I only used leaked data with sites 0,1,2,15. 2 folds (split by months) and compare predictions of 2017-2018 with real readings from leak data. \nWith right parameters it fast and you can actually see if feature useful or not. Yes, you can overfit to this sites, but I created only general features (at least tried), checked performance a few times on public LB and decided to use it.\n\nFor final model I excluded sites 0,1,2,15 and trained without them with same 2 fold strategy. It improved private score from 1.266 to 1.262.\nI wish I had time to add CatBoost but this time I used only 3 lightgbm models with different seeds and same parameters + blend them with 3 top public kernels without leaks with little different strategy from one another to add diversity. \n\no\t[Ashrae Energy Prediction using Stratified KFold](https://www.kaggle.com/roydatascience/ashrae-energy-prediction-using-stratified-kfold)\no\t[ASHRAE 2*lightGBM without Leak Data](https://www.kaggle.com/mimoudata/ashrae-2-lightgbm-without-leak-data)\no\t[Ashrae: simple data cleanup (LB 1.08 no leaks)](https://www.kaggle.com/purist1024/ashrae-simple-data-cleanup-lb-1-08-no-leaks)\n\nBlend weights: 0.66 for my 3 models, rest for public kernels. \n\nMy best LGBM parameters:\n \n```\n{'num_leaves': 500, \n 'objective': 'regression',\n 'learning_rate': 0.04,\n 'boosting': 'gbdt',\n 'subsample': 0.4,\n 'feature_fraction': 0.7,\n 'n_jobs': -1,\n 'seed': 19,\n 'metric': 'rmse'}\n```\n\n### Clearing dataset\n\nUsed [Robert Stockton](https://www.kaggle.com/purist1024/ashrae-simple-data-cleanup-lb-1-08-no-leaks) kernel, learned a lot, thanks!\nManually cleaned some worst readings by meter type, by RMSLE error, buildings in sites 9,6,10. \nI don’t think, that I should have done some of the manual cleaning based only on my intuition instead of actual understanding what those outliers mean. Maybe, I cleaned what seemed like an outlier to me but it wasn’t and presented in test data. I think so, because I have 2 models which scores higher than my final submission and best one is single model without late manual cleaning that scores 1.25 on private.\n\n### Feature engineering\n\nI manually checked all existed features, features from kernels, features from my ideas, aggregations, frequency encodings and found only few of them useful. I deleted most of the initial features.\nI didn’t do any imputations (score decreased in CV), converted all categorical values to category type, square feet and meter readings to log1p.\nI even downloaded external weather history to each site with weather conditions etc. but it didn’t improve score of my CV so I decided not to use them. \n\n---\nI had lot of fun struggle through and learned a lot. Hope, you are having a wonderful day and looking forward to see all of you in the next competition!", "717362": "Thank you very much for sharing! We would like to add your solution to our analysis of the top processes and models. Please let us know how you would like to be acknowledged. The default will be what you have on your profile. Thanks again!\n\n**Also, for everyone else, we are in the process of curating a directory of Notebooks for building performance engineers and analysts from the competition that will serve as a recipe book for beginners as well as experts who are curious to learn about certain topics. We highly encourage you all to share your solutions (or any part of your process) as a notebook for this purpose. We will acknowledge the authors of all notebooks in this process, so please annotate your notebook or profile with how you would like to be acknowledged. We will also include general solutions and process information that is not in notebooks as well. It would also be helpful if you could indicate whether you are an ASHRAE member.**", "717630": "Profile info would be sufficient, thanks!" }, "source": "meta" }