New to machine learning and data science? No question is too basic or too simple. Feel free to start your own thread, or use this thread as a place to post any first-timer clarifying questions for the Kaggle community to help you with!
\n\nIf you would consider yourself a beginner but don't know where to get started, let other Kagglers help you take your first steps here!
\n\nNew to Kaggle? Take a look at a few videos our very own Dr. Rachael Tatman has put together to learn a bit more about site etiquette, Kaggle lingo, and how to enter a competition using Kaggle Notebooks.
", "messages": [ { "id": 649773, "postDate": "2019-10-15T18:30:54.100Z", "content": "New to machine learning and data science? No question is too basic or too simple. Feel free to start your own thread, or use this thread as a place to post any first-timer clarifying questions for the Kaggle community to help you with!
\n\nIf you would consider yourself a beginner but don't know where to get started, let other Kagglers help you take your first steps here!
\n\nNew to Kaggle? Take a look at a few videos our very own Dr. Rachael Tatman has put together to learn a bit more about site etiquette, Kaggle lingo, and how to enter a competition using Kaggle Notebooks.
", "rawMarkdown": "New to machine learning and data science? No question is too basic or too simple. Feel free to start your own thread, or use this thread as a place to post any first-timer clarifying questions for the Kaggle community to help you with! \n\nIf you would consider yourself a beginner but don't know where to get started, let other Kagglers help you take your first steps here!\n\nNew to Kaggle? Take a look at a few videos our very own Dr. Rachael Tatman has put together to learn a bit more about [site etiquette](https://www.youtube.com/watch?v=aIus8si_Et0), [Kaggle lingo](https://www.youtube.com/watch?v=sEJHyuWKd-s), and [how to enter a competition using Kaggle Notebooks](https://www.youtube.com/watch?&v=GJBOMWpLpTQ).", "votes": 22 }, { "id": 665171, "postDate": "2019-11-04T17:34:50.137Z", "content": "Thanks for creating this thread! I have some basic questions that I need to have answered to better understand the data and the goal of this competition.
\n\nPlease correct me if I am wrong... It seems we are training our model on building and weather data from 2016, and it is our job to predict what the meter will read in 2017 and 2018.
\n\nSo.. I can make predictions for 2017 and 18, and Kaggle and ASHRAE have the true meter readings hidden away? So after I make my predictions and submit them, Kaggle will compare my predictions to the true value and give me an accuracy score?
\n\nIf that is correct, then how are these numbers to be compared to a building that has been overhauled and energy efficiencies have been made? \nIs the idea that all of these buildings will be overhauled sometime in the future to be made more energy efficient, and we will use the most accurate model to make predictions of what energy use would have been if changes were not made, and then make the comparison to find the amount of money the building saved?
\n\nThe above questions have to do with the goal of the competition, now I have a question about the data itself...
\n\nFor the meter types: (0: electricity, 1: chilled water, 2: steam, 3: hot water) are we talking about how much energy each type consumes for a specific building?
\n\nI noticed there are way more entries for electricity than the other types.\nelectricity_count = 12,060,910\nchilled_water_count =4,182,440\nsteam_count = 2,708,713\nhot_water_count = 2,708,713
\n\nIs it safe to assume that some buildings have meters for each type of common electricity uses, and for those buildings the energy consumption would be divided between each use... \nAnd some buildings will have only one meter for electricity, so the meter reading would include all the electricity the building used, including electricity used for chilled water, steam and hot water?
\n\nI really appreciate anyone who takes the time to read this post and confirm or correct my assumptions!\nTHANKS
", "rawMarkdown": "Thanks for creating this thread! I have some basic questions that I need to have answered to better understand the data and the goal of this competition.\n\nPlease correct me if I am wrong... It seems we are training our model on building and weather data from 2016, and it is our job to predict what the meter will read in 2017 and 2018. \n\nSo.. I can make predictions for 2017 and 18, and Kaggle and ASHRAE have the true meter readings hidden away? So after I make my predictions and submit them, Kaggle will compare my predictions to the true value and give me an accuracy score?\n\nIf that is correct, then how are these numbers to be compared to a building that has been overhauled and energy efficiencies have been made? \nIs the idea that all of these buildings will be overhauled sometime in the future to be made more energy efficient, and we will use the most accurate model to make predictions of what energy use would have been if changes were not made, and then make the comparison to find the amount of money the building saved?\n\nThe above questions have to do with the goal of the competition, now I have a question about the data itself...\n\nFor the meter types: (0: electricity, 1: chilled water, 2: steam, 3: hot water) are we talking about how much energy each type consumes for a specific building? \n\nI noticed there are way more entries for electricity than the other types.\nelectricity_count = 12,060,910\nchilled_water_count =4,182,440\nsteam_count = 2,708,713\nhot_water_count = 2,708,713\n\nIs it safe to assume that some buildings have meters for each type of common electricity uses, and for those buildings the energy consumption would be divided between each use... \nAnd some buildings will have only one meter for electricity, so the meter reading would include all the electricity the building used, including electricity used for chilled water, steam and hot water?\n\nI really appreciate anyone who takes the time to read this post and confirm or correct my assumptions!\nTHANKS", "votes": 2, "replies": [ { "id": 668776, "postDate": "2019-11-08T20:37:51.793Z", "content": "Just consider the hot water meter as luxury one. Electricity meter is the basic one available in most home. But hot water meter is a add on. It is available in all home. We should find the meter reading forecast for these meters separately. \nDon't assume that if a building has one meter it does not include all meters. If a building has only electricity meter then no hot water meter is available in the particular building. \nTime series will have trend which will include the over hauled and depreciation of equipment energy consumption over a period of time
", "rawMarkdown": "Just consider the hot water meter as luxury one. Electricity meter is the basic one available in most home. But hot water meter is a add on. It is available in all home. We should find the meter reading forecast for these meters separately. \nDon't assume that if a building has one meter it does not include all meters. If a building has only electricity meter then no hot water meter is available in the particular building. \nTime series will have trend which will include the over hauled and depreciation of equipment energy consumption over a period of time" } ] }, { "id": 706057, "postDate": "2019-12-29T21:44:02.263Z", "content": "New to this... I wish there was systematic progression plan for submissions. Any help to get started would be appreciated.
", "rawMarkdown": "New to this... I wish there was systematic progression plan for submissions. Any help to get started would be appreciated." }, { "id": 705010, "postDate": "2019-12-28T09:49:38.227Z", "content": "Hi, I have not taken part in this competition but using this use case for the learning purpose.
\n\nI hesitated to post the query here but thanks there is thread for newbie. I hope someone will answer my query on data.
\n\nThanks
", "rawMarkdown": "Hi, I have not taken part in this competition but using this use case for the learning purpose.\n\nI hesitated to post the query here but thanks there is thread for newbie. I hope someone will answer my query on data.\n\n1. All given meter reading is taken after performance improvement done, \n If this is true I will need some additional data for training model to predict energy consumption without improvement.\n\tIs that leaked data people talking ?\n2. square_feet data is gross area so it already contains area of all floors\n3. Why weather and meter reading data are given split for train/test ? In test data there is no expected value given.\n\nThanks" }, { "id": 693908, "postDate": "2019-12-12T23:23:13.367Z", "content": "I managed to reduce the memory size and merge the datasets, but when i'm trying to clean them, it keeps giving me memory errors, where do I go from here??
", "rawMarkdown": "I managed to reduce the memory size and merge the datasets, but when i'm trying to clean them, it keeps giving me memory errors, where do I go from here??" }, { "id": 691341, "postDate": "2019-12-10T02:50:43.830Z", "content": "New to this... where should I start?
", "rawMarkdown": "New to this... where should I start?" }, { "id": 675610, "postDate": "2019-11-18T10:15:12.953Z", "content": "Hi,\nHow would you develop a safety rating system for building based on alarm information, response to alarms, maintenance measures in place, inspections carried out etc.\nThanks,\nRP
", "rawMarkdown": "Hi,\nHow would you develop a safety rating system for building based on alarm information, response to alarms, maintenance measures in place, inspections carried out etc.\nThanks,\nRP" }, { "id": 673940, "postDate": "2019-11-15T17:19:31.620Z", "content": "Great, Very useful resources.
", "rawMarkdown": "Great, Very useful resources." }, { "id": 673198, "postDate": "2019-11-14T16:48:36.307Z", "content": "Hi,
\n\nRespected Sir Howard,
\n\nThank you very much for starting this thread.
\n\nIndeed this is a great help for new comers like me.
\n\nI am a chemical engineer who is interested in solving this problem. This is because it involves thermodynamics.
\n\nI am presently studying all the comments and comprehend the Python Notebook of Sir Remis Haroon (ID: remisharoon) in which the solution of this problem has been uploaded.
\n\nI will ask REAL questions, as soon as I don't get suitable answer even after searching on Google.
\n\nCheers!
\n\nWaqas.
", "rawMarkdown": "Hi,\n\nRespected Sir Howard,\n\nThank you very much for starting this thread. \n\nIndeed this is a great help for new comers like me. \n\nI am a chemical engineer who is interested in solving this problem. This is because it involves thermodynamics. \n\nI am presently studying all the comments and comprehend the Python Notebook of Sir Remis Haroon (ID: remisharoon) in which the solution of this problem has been uploaded. \n\nI will ask REAL questions, as soon as I don't get suitable answer even after searching on Google. \n\nCheers!\n\nWaqas. " }, { "id": 672686, "postDate": "2019-11-14T04:18:17.540Z", "content": "i am wondering how to merge weather table and building table, as there is no primary key or unique key in both tables
", "rawMarkdown": "i am wondering how to merge weather table and building table, as there is no primary key or unique key in both tables", "replies": [ { "id": 673059, "postDate": "2019-11-14T13:24:52.997Z", "content": "@knightbyj1221 Every building is corresponding to a site_id in building_metadata.csv. Weather is also associated with site_id.
\n\nbuilding_id -> site_id -> weahter
", "rawMarkdown": "@knightbyj1221 Every building is corresponding to a site\\_id in building\\_metadata.csv. Weather is also associated with site_id.\n\nbuilding\\_id -> site\\_id -> weahter" } ] }, { "id": 649813, "postDate": "2019-10-15T19:29:01.833Z", "content": "thanks for sharing, Already subscribed on Rachael's youtube channel. very cool and funny.
", "rawMarkdown": "thanks for sharing, Already subscribed on Rachael's youtube channel. very cool and funny. " }, { "id": 674047, "postDate": "2019-11-15T20:52:39.377Z", "rawMarkdown": "", "isDeleted": true }, { "id": 663203, "postDate": "2019-11-01T15:47:42.253Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 665171, "author_name": "Chris Murphy", "author_url": "", "post_date": "2019-11-04T17:34:50.137000", "content": "Thanks for creating this thread! I have some basic questions that I need to have answered to better understand the data and the goal of this competition.
\n\nPlease correct me if I am wrong... It seems we are training our model on building and weather data from 2016, and it is our job to predict what the meter will read in 2017 and 2018.
\n\nSo.. I can make predictions for 2017 and 18, and Kaggle and ASHRAE have the true meter readings hidden away? So after I make my predictions and submit them, Kaggle will compare my predictions to the true value and give me an accuracy score?
\n\nIf that is correct, then how are these numbers to be compared to a building that has been overhauled and energy efficiencies have been made? \nIs the idea that all of these buildings will be overhauled sometime in the future to be made more energy efficient, and we will use the most accurate model to make predictions of what energy use would have been if changes were not made, and then make the comparison to find the amount of money the building saved?
\n\nThe above questions have to do with the goal of the competition, now I have a question about the data itself...
\n\nFor the meter types: (0: electricity, 1: chilled water, 2: steam, 3: hot water) are we talking about how much energy each type consumes for a specific building?
\n\nI noticed there are way more entries for electricity than the other types.\nelectricity_count = 12,060,910\nchilled_water_count =4,182,440\nsteam_count = 2,708,713\nhot_water_count = 2,708,713
\n\nIs it safe to assume that some buildings have meters for each type of common electricity uses, and for those buildings the energy consumption would be divided between each use... \nAnd some buildings will have only one meter for electricity, so the meter reading would include all the electricity the building used, including electricity used for chilled water, steam and hot water?
\n\nI really appreciate anyone who takes the time to read this post and confirm or correct my assumptions!\nTHANKS
", "votes": 2, "replies": [ { "id": 668776, "author_name": "saravanansaminathan", "author_url": "", "post_date": "2019-11-08T20:37:51.793000", "content": "Just consider the hot water meter as luxury one. Electricity meter is the basic one available in most home. But hot water meter is a add on. It is available in all home. We should find the meter reading forecast for these meters separately. \nDon't assume that if a building has one meter it does not include all meters. If a building has only electricity meter then no hot water meter is available in the particular building. \nTime series will have trend which will include the over hauled and depreciation of equipment energy consumption over a period of time
", "votes": 0, "replies": [] } ] }, { "id": 706057, "author_name": "Jignesh Patel", "author_url": "", "post_date": "2019-12-29T21:44:02.263000", "content": "New to this... I wish there was systematic progression plan for submissions. Any help to get started would be appreciated.
", "votes": 0, "replies": [] }, { "id": 705010, "author_name": "Piyush Merja", "author_url": "", "post_date": "2019-12-28T09:49:38.227000", "content": "Hi, I have not taken part in this competition but using this use case for the learning purpose.
\n\nI hesitated to post the query here but thanks there is thread for newbie. I hope someone will answer my query on data.
\n\nThanks
", "votes": 0, "replies": [] }, { "id": 693908, "author_name": "Juan Vazquez", "author_url": "", "post_date": "2019-12-12T23:23:13.367000", "content": "I managed to reduce the memory size and merge the datasets, but when i'm trying to clean them, it keeps giving me memory errors, where do I go from here??
", "votes": 0, "replies": [] }, { "id": 691341, "author_name": "Alexis Ankrah", "author_url": "", "post_date": "2019-12-10T02:50:43.830000", "content": "New to this... where should I start?
", "votes": 0, "replies": [] }, { "id": 675610, "author_name": "Roopa Prabhakar", "author_url": "", "post_date": "2019-11-18T10:15:12.953000", "content": "Hi,\nHow would you develop a safety rating system for building based on alarm information, response to alarms, maintenance measures in place, inspections carried out etc.\nThanks,\nRP
", "votes": 0, "replies": [] }, { "id": 673940, "author_name": "Ali Hassan", "author_url": "", "post_date": "2019-11-15T17:19:31.620000", "content": "Great, Very useful resources.
", "votes": 0, "replies": [] }, { "id": 673198, "author_name": "Muhammad Waqas ", "author_url": "", "post_date": "2019-11-14T16:48:36.307000", "content": "Hi,
\n\nRespected Sir Howard,
\n\nThank you very much for starting this thread.
\n\nIndeed this is a great help for new comers like me.
\n\nI am a chemical engineer who is interested in solving this problem. This is because it involves thermodynamics.
\n\nI am presently studying all the comments and comprehend the Python Notebook of Sir Remis Haroon (ID: remisharoon) in which the solution of this problem has been uploaded.
\n\nI will ask REAL questions, as soon as I don't get suitable answer even after searching on Google.
\n\nCheers!
\n\nWaqas.
", "votes": 0, "replies": [] }, { "id": 672686, "author_name": "卞逸杰", "author_url": "", "post_date": "2019-11-14T04:18:17.540000", "content": "i am wondering how to merge weather table and building table, as there is no primary key or unique key in both tables
", "votes": 0, "replies": [ { "id": 673059, "author_name": "Buffalo Spdwy", "author_url": "", "post_date": "2019-11-14T13:24:52.997000", "content": "@knightbyj1221 Every building is corresponding to a site_id in building_metadata.csv. Weather is also associated with site_id.
\n\nbuilding_id -> site_id -> weahter
", "votes": 0, "replies": [] } ] }, { "id": 649813, "author_name": "Adrian Zinovei", "author_url": "", "post_date": "2019-10-15T19:29:01.833000", "content": "thanks for sharing, Already subscribed on Rachael's youtube channel. very cool and funny.
", "votes": 0, "replies": [] }, { "id": 674047, "author_name": "", "author_url": "", "post_date": "2019-11-15T20:52:39.377000", "content": "", "votes": 0, "replies": [] }, { "id": 663203, "author_name": "", "author_url": "", "post_date": "2019-11-01T15:47:42.253000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "649773": "New to machine learning and data science? No question is too basic or too simple. Feel free to start your own thread, or use this thread as a place to post any first-timer clarifying questions for the Kaggle community to help you with! \n\nIf you would consider yourself a beginner but don't know where to get started, let other Kagglers help you take your first steps here!\n\nNew to Kaggle? Take a look at a few videos our very own Dr. Rachael Tatman has put together to learn a bit more about [site etiquette](https://www.youtube.com/watch?v=aIus8si_Et0), [Kaggle lingo](https://www.youtube.com/watch?v=sEJHyuWKd-s), and [how to enter a competition using Kaggle Notebooks](https://www.youtube.com/watch?&v=GJBOMWpLpTQ).", "665171": "Thanks for creating this thread! I have some basic questions that I need to have answered to better understand the data and the goal of this competition.\n\nPlease correct me if I am wrong... It seems we are training our model on building and weather data from 2016, and it is our job to predict what the meter will read in 2017 and 2018. \n\nSo.. I can make predictions for 2017 and 18, and Kaggle and ASHRAE have the true meter readings hidden away? So after I make my predictions and submit them, Kaggle will compare my predictions to the true value and give me an accuracy score?\n\nIf that is correct, then how are these numbers to be compared to a building that has been overhauled and energy efficiencies have been made? \nIs the idea that all of these buildings will be overhauled sometime in the future to be made more energy efficient, and we will use the most accurate model to make predictions of what energy use would have been if changes were not made, and then make the comparison to find the amount of money the building saved?\n\nThe above questions have to do with the goal of the competition, now I have a question about the data itself...\n\nFor the meter types: (0: electricity, 1: chilled water, 2: steam, 3: hot water) are we talking about how much energy each type consumes for a specific building? \n\nI noticed there are way more entries for electricity than the other types.\nelectricity_count = 12,060,910\nchilled_water_count =4,182,440\nsteam_count = 2,708,713\nhot_water_count = 2,708,713\n\nIs it safe to assume that some buildings have meters for each type of common electricity uses, and for those buildings the energy consumption would be divided between each use... \nAnd some buildings will have only one meter for electricity, so the meter reading would include all the electricity the building used, including electricity used for chilled water, steam and hot water?\n\nI really appreciate anyone who takes the time to read this post and confirm or correct my assumptions!\nTHANKS", "706057": "New to this... I wish there was systematic progression plan for submissions. Any help to get started would be appreciated.", "705010": "Hi, I have not taken part in this competition but using this use case for the learning purpose.\n\nI hesitated to post the query here but thanks there is thread for newbie. I hope someone will answer my query on data.\n\n1. All given meter reading is taken after performance improvement done, \n If this is true I will need some additional data for training model to predict energy consumption without improvement.\n\tIs that leaked data people talking ?\n2. square_feet data is gross area so it already contains area of all floors\n3. Why weather and meter reading data are given split for train/test ? In test data there is no expected value given.\n\nThanks", "693908": "I managed to reduce the memory size and merge the datasets, but when i'm trying to clean them, it keeps giving me memory errors, where do I go from here??", "691341": "New to this... where should I start?", "675610": "Hi,\nHow would you develop a safety rating system for building based on alarm information, response to alarms, maintenance measures in place, inspections carried out etc.\nThanks,\nRP", "673940": "Great, Very useful resources.", "673198": "Hi,\n\nRespected Sir Howard,\n\nThank you very much for starting this thread. \n\nIndeed this is a great help for new comers like me. \n\nI am a chemical engineer who is interested in solving this problem. This is because it involves thermodynamics. \n\nI am presently studying all the comments and comprehend the Python Notebook of Sir Remis Haroon (ID: remisharoon) in which the solution of this problem has been uploaded. \n\nI will ask REAL questions, as soon as I don't get suitable answer even after searching on Google. \n\nCheers!\n\nWaqas. ", "672686": "i am wondering how to merge weather table and building table, as there is no primary key or unique key in both tables", "649813": "thanks for sharing, Already subscribed on Rachael's youtube channel. very cool and funny. ", "674047": "", "663203": "" } }