{
  "id": 112841,
  "title": "External Data Disclosure Thread",
  "url": "/competitions/ashrae-energy-prediction/discussion/112841",
  "author_name": "Sohier Dane",
  "post_date": "2019-10-15T18:30:40.038000",
  "votes": 39,
  "comment_count": 88,
  "views": 0,
  "content": "<p>Post links to your external data sources here before the deadline specified in the rules (December 12th).</p>",
  "messages": [
    {
      "id": 649772,
      "postDate": "2019-10-15T18:30:40.040Z",
      "content": "<p>Post links to your external data sources here before the deadline specified in the rules (December 12th).</p>",
      "rawMarkdown": "Post links to your external data sources here before the deadline specified in the rules (December 12th).",
      "votes": 37
    },
    {
      "id": 670787,
      "postDate": "2019-11-11T21:05:45.473Z",
      "content": "<p><a href=\"/sohier\">@sohier</a> We are asking to clarify using web scrapped data as predicted values. That is clearly a data leak.</p>",
      "rawMarkdown": "@sohier We are asking to clarify using web scrapped data as predicted values. That is clearly a data leak.",
      "votes": 25
    },
    {
      "id": 665274,
      "postDate": "2019-11-04T20:35:31.940Z",
      "content": "<p>for a number of buildings / sites it is possible to find information in the Internet using area and construction year as a key.  I made a more detailed post [here] (<a href=\"https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115698\">https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115698</a>).\nBelow is a list of sources that mention buildings from a few areas: </p>\n\n<p><strong>0: Orlando, FL</strong>\n    <a href=\"https://www.oeis.ucf.edu/buildings\">https://www.oeis.ucf.edu/buildings</a></p>\n\n<p><strong>2: Tempe, Arizona</strong> \n    <a href=\"https://www.asu.edu/purchasing/bids/pdfs/231906_ex_6A.pdf\">https://www.asu.edu/purchasing/bids/pdfs/231906_ex_6A.pdf</a></p>\n\n<p><strong>3: DC</strong>\n<a href=\"https://dgs.dc.gov/sites/default/files/dc/sites/dgs/service_content/attachments/Attachment%20A-%20Municipal%20Facilities%5b1%5d.pdf\">https://dgs.dc.gov/sites/default/files/dc/sites/dgs/service_content/attachments/Attachment%20A-%20Municipal%20Facilities%5b1%5d.pdf</a></p>\n\n<p><strong>8: Orlando, FL</strong>\n<a href=\"https://www.orlando.gov/files/sharedassets/public/initiatives/bewes/090418_cityoforlando_bldginfo.pdf\">https://www.orlando.gov/files/sharedassets/public/initiatives/bewes/090418_cityoforlando_bldginfo.pdf</a></p>\n\n<p><strong>9: Austin, TX</strong>\n    <a href=\"http://www.thecb.state.tx.us/reports/xls/7559.xlsx\">http://www.thecb.state.tx.us/reports/xls/7559.xlsx</a></p>\n\n<p><strong>13: TwinCities, MN</strong>\n<a href=\"https://mn.b3benchmarking.com/Report?OrganizationID=1337&amp;ReturnTo=Higher+Education\">https://mn.b3benchmarking.com/Report?OrganizationID=1337&amp;ReturnTo=Higher+Education</a></p>\n\n<p><strong>14: Charlottesville, VA</strong>\n<a href=\"https://www.fm.virginia.edu/docs/conditionReport/2018-19FacilitiesConditionReport.pdf\">https://www.fm.virginia.edu/docs/conditionReport/2018-19FacilitiesConditionReport.pdf</a> (has year built data)</p>",
      "rawMarkdown": "for a number of buildings / sites it is possible to find information in the Internet using area and construction year as a key.  I made a more detailed post [here] (https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115698).\nBelow is a list of sources that mention buildings from a few areas: \n\n**0: Orlando, FL**\n    https://www.oeis.ucf.edu/buildings\n\n**2: Tempe, Arizona** \n    https://www.asu.edu/purchasing/bids/pdfs/231906_ex_6A.pdf\n\n**3: DC**\nhttps://dgs.dc.gov/sites/default/files/dc/sites/dgs/service_content/attachments/Attachment%20A-%20Municipal%20Facilities%5b1%5d.pdf\n\n**8: Orlando, FL**\nhttps://www.orlando.gov/files/sharedassets/public/initiatives/bewes/090418_cityoforlando_bldginfo.pdf\n\n**9: Austin, TX**\n    http://www.thecb.state.tx.us/reports/xls/7559.xlsx\n\n**13: TwinCities, MN**\nhttps://mn.b3benchmarking.com/Report?OrganizationID=1337&amp;ReturnTo=Higher+Education\n\n**14: Charlottesville, VA**\nhttps://www.fm.virginia.edu/docs/conditionReport/2018-19FacilitiesConditionReport.pdf (has year built data)\n",
      "votes": 19,
      "replies": [
        {
          "id": 668345,
          "postDate": "2019-11-08T09:59:33.317Z",
          "content": "<p>hey so just like... based on that I'm gonna go scrape all ucf's energy data... That's within the rules right? I'm allowed to just go get that?</p>\n\n<p>doesn't feel right...</p>",
          "rawMarkdown": "hey so just like... based on that I'm gonna go scrape all ucf's energy data... That's within the rules right? I'm allowed to just go get that?\n\ndoesn't feel right...",
          "votes": 2
        }
      ]
    },
    {
      "id": 673832,
      "postDate": "2019-11-15T14:38:51.157Z",
      "content": "<p>Site 1 UCL building metadata and meter readings\n<a href=\"https://platform.carbonculture.net/communities/ucl/30/apps/assets/list/place/\">https://platform.carbonculture.net/communities/ucl/30/apps/assets/list/place/</a></p>",
      "rawMarkdown": "Site 1 UCL building metadata and meter readings\nhttps://platform.carbonculture.net/communities/ucl/30/apps/assets/list/place/",
      "votes": 18,
      "replies": [
        {
          "id": 673903,
          "postDate": "2019-11-15T16:21:03.310Z",
          "content": "<p>OK, another one revealed ? As I see you climbed up in the LB 😄 </p>",
          "rawMarkdown": "OK, another one revealed ? As I see you climbed up in the LB 😄 "
        },
        {
          "id": 673926,
          "postDate": "2019-11-15T17:01:35.357Z",
          "content": "<p>Thanks for this, but it is terrible news! One more site data publicly available!\nIt´s a pity, this could have been a very interesting competition...</p>",
          "rawMarkdown": "Thanks for this, but it is terrible news! One more site data publicly available!\nIt´s a pity, this could have been a very interesting competition...",
          "votes": 7
        },
        {
          "id": 674815,
          "postDate": "2019-11-17T05:42:33.490Z",
          "content": "<p>Another one? OMG...😭</p>",
          "rawMarkdown": "Another one? OMG...😭"
        },
        {
          "id": 674960,
          "postDate": "2019-11-17T10:42:41.897Z",
          "content": "<p>Has anyone planned to make UCL dataset and to make it public ?</p>\n\n<p>If no one, i will try it next week.</p>",
          "rawMarkdown": "Has anyone planned to make UCL dataset and to make it public ?\n\nIf no one, i will try it next week.",
          "votes": 4
        },
        {
          "id": 675059,
          "postDate": "2019-11-17T14:38:34.457Z",
          "content": "<p><a href=\"/gunesevitan\">@gunesevitan</a> , may I ask if cm.asu.edu is scrapable ? if yes, how do you proceed ? I can't find the actual table or a way to query the website.</p>",
          "rawMarkdown": "@gunesevitan , may I ask if cm.asu.edu is scrapable ? if yes, how do you proceed ? I can't find the actual table or a way to query the website."
        },
        {
          "id": 675067,
          "postDate": "2019-11-17T14:52:56.353Z",
          "content": "<p>It is actually very easy to scrape <a href=\"http://cm.asu.edu/\">http://cm.asu.edu/</a>. It took me 15 minutes to write the spider and you don't even need Scrapy.</p>\n\n<ul>\n<li>Open the website on chrome and copy your session cookie because server won't return a valid response without it</li>\n<li>Find the request that returns building ids from XHR tab in network, and collect the building ids (use session cookie in headers)</li>\n<li>Find the request that returns meter readings and pass building id to it while looping through scraped building ids and accumulate the meter readings in a dataframe (use session cookie in every request)</li>\n</ul>\n\n<p>One big problem is the server returns the response in approximately  3 minutes, so it'll take too much time to scrape it completely</p>",
          "rawMarkdown": "It is actually very easy to scrape http://cm.asu.edu/. It took me 15 minutes to write the spider and you don't even need Scrapy.\n\n* Open the website on chrome and copy your session cookie because server won't return a valid response without it\n* Find the request that returns building ids from XHR tab in network, and collect the building ids (use session cookie in headers)\n* Find the request that returns meter readings and pass building id to it while looping through scraped building ids and accumulate the meter readings in a dataframe (use session cookie in every request)\n\nOne big problem is the server returns the response in approximately  3 minutes, so it'll take too much time to scrape it completely",
          "votes": 5
        },
        {
          "id": 675094,
          "postDate": "2019-11-17T15:40:40.910Z",
          "content": "<p><a href=\"/yamsam\">@yamsam</a>  Yes, <a href=\"https://www.kaggle.com/mpware/ucl-data-leakage-episode-2\">https://www.kaggle.com/mpware/ucl-data-leakage-episode-2</a></p>",
          "rawMarkdown": "@yamsam  Yes, https://www.kaggle.com/mpware/ucl-data-leakage-episode-2",
          "votes": 8
        },
        {
          "id": 675289,
          "postDate": "2019-11-17T21:14:45.563Z",
          "content": "<p>Thank you <a href=\"/gunesevitan\">@gunesevitan</a> <a href=\"/mpware\">@mpware</a> . You are so great!! I will update my kernel.</p>",
          "rawMarkdown": "Thank you @gunesevitan @mpware . You are so great!! I will update my kernel."
        },
        {
          "id": 675819,
          "postDate": "2019-11-18T15:51:26.113Z",
          "content": "<p>Organizers should have provided the link to scrape the data rather than sharing train/test set :D</p>",
          "rawMarkdown": "Organizers should have provided the link to scrape the data rather than sharing train/test set :D"
        },
        {
          "id": 676354,
          "postDate": "2019-11-19T05:28:23.123Z",
          "content": "<p><a href=\"/gunesevitan\">@gunesevitan</a> can't you multi-thread a webscrape? can't each thread request information independently?</p>",
          "rawMarkdown": "@gunesevitan can't you multi-thread a webscrape? can't each thread request information independently?"
        },
        {
          "id": 676362,
          "postDate": "2019-11-19T05:46:23.137Z",
          "content": "<p>You can send multi-threaded requests, but there is no point of doing it. You still have to wait for server to send you response back. The time consuming operation is waiting for response.</p>\n\n<p>That's why web scraping frameworks like Scrapy are single-threaded, but they utilize concurrency with coroutines. Non-blocking IO is way better than multi-threading in web scraping.</p>",
          "rawMarkdown": "You can send multi-threaded requests, but there is no point of doing it. You still have to wait for server to send you response back. The time consuming operation is waiting for response.\n\nThat's why web scraping frameworks like Scrapy are single-threaded, but they utilize concurrency with coroutines. Non-blocking IO is way better than multi-threading in web scraping.",
          "votes": 2
        },
        {
          "id": 676451,
          "postDate": "2019-11-19T07:38:01.073Z",
          "content": "<p>Guys can you please move this valuable discussion on Best Practices on Web Scraping to a separate thread, out of this dataset thread? Other things that you would have to worry about with commercial sites are rate-limiting, rotating IP addresses, etc.</p>",
          "rawMarkdown": "Guys can you please move this valuable discussion on Best Practices on Web Scraping to a separate thread, out of this dataset thread? Other things that you would have to worry about with commercial sites are rate-limiting, rotating IP addresses, etc."
        }
      ]
    },
    {
      "id": 677934,
      "postDate": "2019-11-20T19:44:07.200Z",
      "content": "<p>It is <a href=\"https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115698#latest-674237\">revealed</a> that site 4 is UC Berkeley. The university provides it energy consumption data <a href=\"https://engagementdashboard.com/ucb/ucb/\">publicly</a>. The following kernel extracts the historical energy consumption data for several buildings in the college.</p>\n\n<p><a href=\"https://www.kaggle.com/serengil/ucb-data-leakage-site-4\">https://www.kaggle.com/serengil/ucb-data-leakage-site-4</a></p>",
      "rawMarkdown": "It is [revealed](https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115698#latest-674237) that site 4 is UC Berkeley. The university provides it energy consumption data [publicly](https://engagementdashboard.com/ucb/ucb/). The following kernel extracts the historical energy consumption data for several buildings in the college.\n\n[https://www.kaggle.com/serengil/ucb-data-leakage-site-4](https://www.kaggle.com/serengil/ucb-data-leakage-site-4)",
      "votes": 11
    },
    {
      "id": 676560,
      "postDate": "2019-11-19T09:54:49.810Z",
      "content": "<p>Site 2 is not posted here yet, so let's make it official\n<a href=\"http://cm.asu.edu/\">http://cm.asu.edu/</a></p>",
      "rawMarkdown": "Site 2 is not posted here yet, so let's make it official\nhttp://cm.asu.edu/",
      "votes": 7,
      "replies": [
        {
          "id": 676576,
          "postDate": "2019-11-19T10:19:47.147Z",
          "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F456596%2F364f85d7ae8a1392913a413c49803400%2FCFu8ouq.jpg?generation=1574158751587103&amp;alt=media\" alt=\"\"></p>",
          "rawMarkdown": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F456596%2F364f85d7ae8a1392913a413c49803400%2FCFu8ouq.jpg?generation=1574158751587103&amp;alt=media)\n",
          "votes": 4
        },
        {
          "id": 676746,
          "postDate": "2019-11-19T13:30:13.167Z",
          "content": "<p>sorry somebody was faster than me...</p>",
          "rawMarkdown": "sorry somebody was faster than me...",
          "votes": 2
        }
      ]
    },
    {
      "id": 693900,
      "postDate": "2019-12-12T22:51:22.190Z",
      "content": "<p><strong>Leaks:</strong>\n0: <a href=\"https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels\">https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels</a>\n1: <a href=\"https://www.kaggle.com/mpware/ucl-data-leakage-episode-2\">https://www.kaggle.com/mpware/ucl-data-leakage-episode-2</a>\n2: <strong><a href=\"https://www.kaggle.com/pdnartreb/scrap-asu-data\">https://www.kaggle.com/pdnartreb/scrap-asu-data</a></strong>\n3: <a href=\"https://media.githubusercontent.com/media/buds-lab/island-of-misfit-buildings/master/data/raw/DGS_322_Buildings-15m-By_Building-DST-gap-filled-3-2-18-508pm.csv\">https://media.githubusercontent.com/media/buds-lab/island-of-misfit-buildings/master/data/raw/DGS_322_Buildings-15m-By_Building-DST-gap-filled-3-2-18-508pm.csv</a>\n4:<a href=\"https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings\">https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings</a></p>\n\n<p>15: <a href=\"https://www.kaggle.com/pp2file/ashrae-site15-cornell\">https://www.kaggle.com/pp2file/ashrae-site15-cornell</a></p>\n\n<p><strong>Trends:</strong>\n14: <a href=\"http://energytracker.fm.virginia.edu/index.html\">http://energytracker.fm.virginia.edu/index.html</a>   (can see overall trends using graphs)</p>\n\n<p>University Terms\nSite 0: University of Central Florida \n<a href=\"http://cf.smartcatalogiq.com/en/2015-2016/Catalog/About-College-of-Central-Florida/College-Calendar\">http://cf.smartcatalogiq.com/en/2015-2016/Catalog/About-College-of-Central-Florida/College-Calendar</a>\n<a href=\"http://cf.smartcatalogiq.com/en/2016-2017/Catalog/About-College-of-Central-Florida/College-Calendar\">http://cf.smartcatalogiq.com/en/2016-2017/Catalog/About-College-of-Central-Florida/College-Calendar</a>\n<a href=\"http://cf.smartcatalogiq.com/en/2017-2018/Catalog/About-College-of-Central-Florida/College-Calendar\">http://cf.smartcatalogiq.com/en/2017-2018/Catalog/About-College-of-Central-Florida/College-Calendar</a>\n<a href=\"http://cf.smartcatalogiq.com/en/2018-2019/Catalog/About-College-of-Central-Florida/College-Calendar\">http://cf.smartcatalogiq.com/en/2018-2019/Catalog/About-College-of-Central-Florida/College-Calendar</a></p>\n\n<p>site1: University College London\n<a href=\"https://www.ucl.ac.uk/estates/roombooking/calendar/1516.pdf\">https://www.ucl.ac.uk/estates/roombooking/calendar/1516.pdf</a>\n<a href=\"https://www.ucl.ac.uk/estates/roombooking/calendar/1617.pdf\">https://www.ucl.ac.uk/estates/roombooking/calendar/1617.pdf</a>\n<a href=\"https://www.ucl.ac.uk/estates/roombooking/calendar/1718.pdf\">https://www.ucl.ac.uk/estates/roombooking/calendar/1718.pdf</a>\n<a href=\"https://www.ucl.ac.uk/estates/roombooking/calendar/1819.pdf\">https://www.ucl.ac.uk/estates/roombooking/calendar/1819.pdf</a></p>\n\n<p>site2: Arizona State University \n<a href=\"https://catalog.arizona.edu/calendar/2015-2016-academic-calendar\">https://catalog.arizona.edu/calendar/2015-2016-academic-calendar</a>\n<a href=\"https://catalog.arizona.edu/calendar/2016-2017-academic-calendar\">https://catalog.arizona.edu/calendar/2016-2017-academic-calendar</a>\n<a href=\"https://catalog.arizona.edu/2017-2018-academic-calendar\">https://catalog.arizona.edu/2017-2018-academic-calendar</a>\n<a href=\"https://catalog.arizona.edu/calendar/2018-2019-academic-calendar\">https://catalog.arizona.edu/calendar/2018-2019-academic-calendar</a></p>\n\n<p>Site4: University of California Berkeley\n<a href=\"https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2015-16.pdf\">https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2015-16.pdf</a>\n<a href=\"https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2016-17_V7.pdf\">https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2016-17_V7.pdf</a>\n<a href=\"https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2017-18_V3.pdf\">https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2017-18_V3.pdf</a>\n<a href=\"https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2018-19_V4.pdf\">https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2018-19_V4.pdf</a></p>\n\n<p>site5: University of Southampton \n<a href=\"https://www.southampton.ac.uk/assets/imported/transforms/content-block/UsefulDownloads_Download/A4EE46BFA84746158B25E5CD179BE5DE/term-dates-201020.pdf\">https://www.southampton.ac.uk/assets/imported/transforms/content-block/UsefulDownloads_Download/A4EE46BFA84746158B25E5CD179BE5DE/term-dates-201020.pdf</a></p>\n\n<p>Site 7 and 11: McGill\n<a href=\"https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2015-16\">https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2015-16</a>\n<a href=\"https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2016-17\">https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2016-17</a>\n<a href=\"https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2017-18\">https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2017-18</a>\n<a href=\"https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2018-19\">https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2018-19</a></p>\n\n<p>Site 9: University of Texas\n<a href=\"https://registrar.utexas.edu/calendars/15-16\">https://registrar.utexas.edu/calendars/15-16</a>\n<a href=\"https://registrar.utexas.edu/calendars/16-17\">https://registrar.utexas.edu/calendars/16-17</a>\n<a href=\"https://registrar.utexas.edu/calendars/17-18\">https://registrar.utexas.edu/calendars/17-18</a>\n<a href=\"https://registrar.utexas.edu/calendars/18-19\">https://registrar.utexas.edu/calendars/18-19</a></p>\n\n<p>site10 : Weber State University\n<a href=\"https://www.weber.edu/Registration/Class_Schedule_Archive.html\">https://www.weber.edu/Registration/Class_Schedule_Archive.html</a></p>\n\n<p>Site 13: University of Minnesotta\n<a href=\"http://www.d.umn.edu/calendar/academic_cal_15-16.html\">http://www.d.umn.edu/calendar/academic_cal_15-16.html</a>\n<a href=\"http://www.d.umn.edu/calendar/academic_cal_16-17.html\">http://www.d.umn.edu/calendar/academic_cal_16-17.html</a>\n<a href=\"http://www.d.umn.edu/calendar/academic_cal_17-18.html\">http://www.d.umn.edu/calendar/academic_cal_17-18.html</a>\n<a href=\"http://www.d.umn.edu/calendar/academic_cal_18-19.html\">http://www.d.umn.edu/calendar/academic_cal_18-19.html</a></p>\n\n<p>site14: University of Virginia\n<a href=\"https://www2.virginia.edu/registrar/calenold.html\">https://www2.virginia.edu/registrar/calenold.html</a> (old)\n<a href=\"https://www2.virginia.edu/registrar/calendar.html\">https://www2.virginia.edu/registrar/calendar.html</a> (current)</p>\n\n<p>site15: Cornell\n<a href=\"http://courses.cornell.edu/mime/media/26/11593/2015-2016+Academic+Calendar.pdf\">http://courses.cornell.edu/mime/media/26/11593/2015-2016+Academic+Calendar.pdf</a>\n<a href=\"https://registrar.cornell.edu/academic-calendar/2016-2017\">https://registrar.cornell.edu/academic-calendar/2016-2017</a>\n<a href=\"https://registrar.cornell.edu/academic-calendar/2017-2018\">https://registrar.cornell.edu/academic-calendar/2017-2018</a>\n<a href=\"https://registrar.cornell.edu/academic-calendar/2018-2019\">https://registrar.cornell.edu/academic-calendar/2018-2019</a></p>",
      "rawMarkdown": "**Leaks:**\n0: https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels\n1: https://www.kaggle.com/mpware/ucl-data-leakage-episode-2\n2: **https://www.kaggle.com/pdnartreb/scrap-asu-data**\n3: https://media.githubusercontent.com/media/buds-lab/island-of-misfit-buildings/master/data/raw/DGS_322_Buildings-15m-By_Building-DST-gap-filled-3-2-18-508pm.csv\n4:https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings\n\n15: https://www.kaggle.com/pp2file/ashrae-site15-cornell\n\n**Trends:**\n14: http://energytracker.fm.virginia.edu/index.html   (can see overall trends using graphs)\n\nUniversity Terms\nSite 0: University of Central Florida \nhttp://cf.smartcatalogiq.com/en/2015-2016/Catalog/About-College-of-Central-Florida/College-Calendar\nhttp://cf.smartcatalogiq.com/en/2016-2017/Catalog/About-College-of-Central-Florida/College-Calendar\nhttp://cf.smartcatalogiq.com/en/2017-2018/Catalog/About-College-of-Central-Florida/College-Calendar\nhttp://cf.smartcatalogiq.com/en/2018-2019/Catalog/About-College-of-Central-Florida/College-Calendar\n\nsite1: University College London\nhttps://www.ucl.ac.uk/estates/roombooking/calendar/1516.pdf\nhttps://www.ucl.ac.uk/estates/roombooking/calendar/1617.pdf\nhttps://www.ucl.ac.uk/estates/roombooking/calendar/1718.pdf\nhttps://www.ucl.ac.uk/estates/roombooking/calendar/1819.pdf\n\nsite2: Arizona State University \nhttps://catalog.arizona.edu/calendar/2015-2016-academic-calendar\nhttps://catalog.arizona.edu/calendar/2016-2017-academic-calendar\nhttps://catalog.arizona.edu/2017-2018-academic-calendar\nhttps://catalog.arizona.edu/calendar/2018-2019-academic-calendar\n  \nSite4: University of California Berkeley\nhttps://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2015-16.pdf\nhttps://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2016-17_V7.pdf\nhttps://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2017-18_V3.pdf\nhttps://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2018-19_V4.pdf\n \nsite5: University of Southampton \nhttps://www.southampton.ac.uk/assets/imported/transforms/content-block/UsefulDownloads_Download/A4EE46BFA84746158B25E5CD179BE5DE/term-dates-201020.pdf\n\nSite 7 and 11: McGill\nhttps://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2015-16\nhttps://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2016-17\nhttps://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2017-18\nhttps://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2018-19\n \nSite 9: University of Texas\nhttps://registrar.utexas.edu/calendars/15-16\nhttps://registrar.utexas.edu/calendars/16-17\nhttps://registrar.utexas.edu/calendars/17-18\nhttps://registrar.utexas.edu/calendars/18-19\n\nsite10 : Weber State University\nhttps://www.weber.edu/Registration/Class_Schedule_Archive.html\n\nSite 13: University of Minnesotta\nhttp://www.d.umn.edu/calendar/academic_cal_15-16.html\nhttp://www.d.umn.edu/calendar/academic_cal_16-17.html\nhttp://www.d.umn.edu/calendar/academic_cal_17-18.html\nhttp://www.d.umn.edu/calendar/academic_cal_18-19.html\n\nsite14: University of Virginia\nhttps://www2.virginia.edu/registrar/calenold.html (old)\nhttps://www2.virginia.edu/registrar/calendar.html (current)\n\nsite15: Cornell\nhttp://courses.cornell.edu/mime/media/26/11593/2015-2016+Academic+Calendar.pdf\nhttps://registrar.cornell.edu/academic-calendar/2016-2017\nhttps://registrar.cornell.edu/academic-calendar/2017-2018\nhttps://registrar.cornell.edu/academic-calendar/2018-2019\n\n\n\n\n \n",
      "votes": 7
    },
    {
      "id": 652300,
      "postDate": "2019-10-18T16:40:07.163Z",
      "content": "<p>Pain in the neck to get data - pretty much only one city at a time for me.  But should be some useful info for the heat load part of energy usage.</p>\n\n<p><a href=\"https://www.nrel.gov/gis/data-tools.html\">https://www.nrel.gov/gis/data-tools.html</a></p>",
      "rawMarkdown": "Pain in the neck to get data - pretty much only one city at a time for me.  But should be some useful info for the heat load part of energy usage.\n\nhttps://www.nrel.gov/gis/data-tools.html\n\n\n\n",
      "votes": 8,
      "replies": [
        {
          "id": 668274,
          "postDate": "2019-11-08T07:58:41.830Z",
          "content": "<p><a href=\"/pcjimmmy\">@pcjimmmy</a> cheers for sharing, I assume you're mainly talking about the \"solar data\" stuff here?  <a href=\"https://www.nrel.gov/gis/tools.html\">https://www.nrel.gov/gis/tools.html</a>, specifically, \"Federal Energy Management Program Screening Map\"?</p>",
          "rawMarkdown": "@pcjimmmy cheers for sharing, I assume you're mainly talking about the \"solar data\" stuff here?  https://www.nrel.gov/gis/tools.html, specifically, \"Federal Energy Management Program Screening Map\"?"
        },
        {
          "id": 668577,
          "postDate": "2019-11-08T15:23:30.070Z",
          "content": "<p>yes - solar stuff - tried just adding a few cities at different latitudes - no help - so probably need to know the specific cities for each site.</p>",
          "rawMarkdown": "yes - solar stuff - tried just adding a few cities at different latitudes - no help - so probably need to know the specific cities for each site.",
          "votes": 1
        },
        {
          "id": 668939,
          "postDate": "2019-11-09T06:20:00.840Z",
          "content": "<p>dang, thanks for dropping that nugget of info jimmy! I've done a decent amount of web scraping before so maybe I'll see if there's a way to grab it en-masse and have a go with more data. </p>",
          "rawMarkdown": "dang, thanks for dropping that nugget of info jimmy! I've done a decent amount of web scraping before so maybe I'll see if there's a way to grab it en-masse and have a go with more data. "
        },
        {
          "id": 668954,
          "postDate": "2019-11-09T06:52:34.647Z",
          "content": "<p>Pretty confident on a number of the sites for what city is what  - but have the solar data low on my to do list.  My current features do a very good job of making predictions to match the training year but have made little progress in getting leader board improvements to match.  Pretty sure the solar data will help predict the training year, but not sure it will help for the test years.  </p>\n\n<p>When I do get to add solar data than orlando will be the first city I grab - it's two sites and  I have lots of confidence that the two sites are really orlando.</p>",
          "rawMarkdown": "Pretty confident on a number of the sites for what city is what  - but have the solar data low on my to do list.  My current features do a very good job of making predictions to match the training year but have made little progress in getting leader board improvements to match.  Pretty sure the solar data will help predict the training year, but not sure it will help for the test years.  \n\nWhen I do get to add solar data than orlando will be the first city I grab - it's two sites and  I have lots of confidence that the two sites are really orlando."
        },
        {
          "id": 692864,
          "postDate": "2019-12-11T19:54:55.880Z",
          "content": "<h2>Hourly solar data time series for US &amp; Canada:</h2>\n\n<ul>\n<li><a href=\"https://maps.nrel.gov/nsrdb-viewer/\">https://maps.nrel.gov/nsrdb-viewer/</a>?</li>\n<li>Choose \"PSMv3\" (Physical Solar Model) for satellite-based solar irradiance data</li>\n<li>SELECT YEARS: 2016-2018</li>\n<li>SELECT ATTRIBUTES: select all attributes</li>\n<li>SELECT DOWNLOAD OPTIONS:\n<ul><li>Include leap day</li>\n<li>UTC (don't convert to local time), si its easier to synchronize with the</li>\n<li>yes to half hour intervals. Use only the half-hour values for solar data, and the whole-hour values for the terrestrial parameters (e.g. temperature) to fit missing data.</li>\n<li>Note that you have to check synchronization. I noticed that site0 data should be moved back approx 1.5 hours to synchronize temperature best.</li></ul></li>\n</ul>\n\n<h2>UK data:</h2>\n\n<ul>\n<li>I used terrestrial data. Register for a user account at UK CEDA (Centre for Environmental Data Analysis), then download specific sites via:</li>\n<li><a href=\"https://catalogue.ceda.ac.uk/?q=solar&amp;sort_by=relevance\">https://catalogue.ceda.ac.uk/?q=solar&amp;sort_by=relevance</a></li>\n<li>The sites measure generally only GHI hourly.</li>\n</ul>\n\n<p>Note: Both sources require appropriate attribution when used</p>",
          "rawMarkdown": "##Hourly solar data time series for US &amp; Canada: \n- https://maps.nrel.gov/nsrdb-viewer/?\n- Choose \"PSMv3\" (Physical Solar Model) for satellite-based solar irradiance data\n- SELECT YEARS: 2016-2018\n- SELECT ATTRIBUTES: select all attributes\n- SELECT DOWNLOAD OPTIONS:\n  - Include leap day\n  - UTC (don't convert to local time), si its easier to synchronize with the\n  - yes to half hour intervals. Use only the half-hour values for solar data, and the whole-hour values for the terrestrial parameters (e.g. temperature) to fit missing data.\n  - Note that you have to check synchronization. I noticed that site0 data should be moved back approx 1.5 hours to synchronize temperature best.\n\n##UK data:\n- I used terrestrial data. Register for a user account at UK CEDA (Centre for Environmental Data Analysis), then download specific sites via:\n- https://catalogue.ceda.ac.uk/?q=solar&amp;sort_by=relevance\n- The sites measure generally only GHI hourly.\n\nNote: Both sources require appropriate attribution when used\n"
        }
      ]
    },
    {
      "id": 678161,
      "postDate": "2019-11-21T03:29:45.657Z",
      "content": "<p>I think we would be better finding all the leaks as soon as possible, so we can focus on modeling.\nAttached are the exact locations / stations that provided the weather information for the competition.\nIf one look for universities near by those airports there is a high likelihood of finding the source of buildings energy consumption. I do not think all of them are possible to be scraped.</p>\n\n<p>High probability that site 15 is Cornell and data can be scraped at  <a href=\"http://portal.emcs.cornell.edu\">http://portal.emcs.cornell.edu</a>.\nOthers are just initial guesses.</p>\n\n<p>site,station,city,state,country,lat,lon,altitude,timezone,tz\n0,ORLANDO INTERNATIONAL AIRPORT,ORLANDO,FL,US,+28.434,-81.325,+27.4,-5,US/Eastern\nUniversity of Central Florida</p>\n\n<p>1,HEATHROW,LONDON,,UK,+51.478,-0.461,+25.3,0,Europe/London</p>\n\n<p>2,PHOENIX SKY HARBOR INTL AIRPORT,PHOENIX,AZ,US,+33.428,-112.004,+337.4,-7,US/Mountain\nArizona State University</p>\n\n<p>3,RONALD REAGAN WASHINGTON NATL AP,WASHINGTON,VA,US,+38.847,-77.035,+3.1,-5,US/Eastern</p>\n\n<p>4,METRO OAKLAND INTL AIRPORT,OAKLAND,CA,US,+37.721,-122.221,+1.8,-8,US/Pacific\nUC Berkeley</p>\n\n<p>5,CARDIFF,CARDIFF,,UK,+51.397,-3.343,+67.1,0,Europe/London\n?? Cardiff University</p>\n\n<p>6,CHARLOTTESVILLE-ALBEMARLE ARPT,CHARLOTTESVILLE,VA,US,+38.137,-78.455,+195.1,-5,US/Eastern\n?? University of Virginia</p>\n\n<p>7,OTTAWA CDA RCS ONT,OTTAWA,ON,CA,+45.383,-75.717,+79.0,-5,Canada/Eastern\n? University of Ottawa</p>\n\n<p>8,ORLANDO INTERNATIONAL AIRPORT,ORLANDO,FL,US,+28.434,-81.325,+27.4,-5,US/Eastern\n??</p>\n\n<p>9,AUSTIN-CAMP MABRY,AUSTIN,TX,US,+30.321,-97.760,+204.2,-6,US/Central\n? University of Texas at Austin</p>\n\n<p>10,OGDEN-HINCKLEY AIRPORT,OGDEN,UT,US,+41.196,-112.011,+1362.5,-7,US/Mountain\n?? Weber state university</p>\n\n<p>11,OTTAWA CDA RCS ONT,OTTAWA,ON,CA,+45.383,-75.717,+79.0,-5,Canada/Eastern\n??</p>\n\n<p>12,DUBLIN,DUBLIN,,EI,+53.421,-6.270,+73.8,0,Europe/Dublin\n?? Dublin City University </p>\n\n<p>13,MINNEAPOLIS-ST PAUL INTERNATIONAL AP,MINNEAPOLIS,MN,US,+44.883,-93.229,+265.8,-6,US/Central\n?? University of Minnesota</p>\n\n<p>14,TRENTON MERCER AIRPORT,TRENTON,NJ,US,+40.277,-74.816,+57.9,-5,US/Eastern\n?? Princeton University</p>\n\n<p>15,REGIONAL AIRPORT,ITHACA,NY,US,+42.483,-76.467,+335.0,-5,US/Eastern\nCornell University</p>",
      "rawMarkdown": "I think we would be better finding all the leaks as soon as possible, so we can focus on modeling.\nAttached are the exact locations / stations that provided the weather information for the competition.\nIf one look for universities near by those airports there is a high likelihood of finding the source of buildings energy consumption. I do not think all of them are possible to be scraped.\n\nHigh probability that site 15 is Cornell and data can be scraped at  http://portal.emcs.cornell.edu.\nOthers are just initial guesses.\n\nsite,station,city,state,country,lat,lon,altitude,timezone,tz\n0,ORLANDO INTERNATIONAL AIRPORT,ORLANDO,FL,US,+28.434,-81.325,+27.4,-5,US/Eastern\nUniversity of Central Florida\n\n1,HEATHROW,LONDON,,UK,+51.478,-0.461,+25.3,0,Europe/London\n\n2,PHOENIX SKY HARBOR INTL AIRPORT,PHOENIX,AZ,US,+33.428,-112.004,+337.4,-7,US/Mountain\nArizona State University\n\n3,RONALD REAGAN WASHINGTON NATL AP,WASHINGTON,VA,US,+38.847,-77.035,+3.1,-5,US/Eastern\n\n4,METRO OAKLAND INTL AIRPORT,OAKLAND,CA,US,+37.721,-122.221,+1.8,-8,US/Pacific\nUC Berkeley\n\n5,CARDIFF,CARDIFF,,UK,+51.397,-3.343,+67.1,0,Europe/London\n?? Cardiff University\n\n6,CHARLOTTESVILLE-ALBEMARLE ARPT,CHARLOTTESVILLE,VA,US,+38.137,-78.455,+195.1,-5,US/Eastern\n?? University of Virginia\n\n7,OTTAWA CDA RCS ONT,OTTAWA,ON,CA,+45.383,-75.717,+79.0,-5,Canada/Eastern\n? University of Ottawa\n\n8,ORLANDO INTERNATIONAL AIRPORT,ORLANDO,FL,US,+28.434,-81.325,+27.4,-5,US/Eastern\n??\n\n9,AUSTIN-CAMP MABRY,AUSTIN,TX,US,+30.321,-97.760,+204.2,-6,US/Central\n? University of Texas at Austin\n\n10,OGDEN-HINCKLEY AIRPORT,OGDEN,UT,US,+41.196,-112.011,+1362.5,-7,US/Mountain\n?? Weber state university\n\n11,OTTAWA CDA RCS ONT,OTTAWA,ON,CA,+45.383,-75.717,+79.0,-5,Canada/Eastern\n??\n\n12,DUBLIN,DUBLIN,,EI,+53.421,-6.270,+73.8,0,Europe/Dublin\n?? Dublin City University \n\n13,MINNEAPOLIS-ST PAUL INTERNATIONAL AP,MINNEAPOLIS,MN,US,+44.883,-93.229,+265.8,-6,US/Central\n?? University of Minnesota\n\n14,TRENTON MERCER AIRPORT,TRENTON,NJ,US,+40.277,-74.816,+57.9,-5,US/Eastern\n?? Princeton University\n\n15,REGIONAL AIRPORT,ITHACA,NY,US,+42.483,-76.467,+335.0,-5,US/Eastern\nCornell University",
      "votes": 5,
      "replies": [
        {
          "id": 678207,
          "postDate": "2019-11-21T05:05:27.717Z",
          "content": "<p>I agree. I can confirm Site 15 is Cornell University. It requires some form fuzzy matching (sqft) for 70 of the buildings. The other 54/124 are exact match. <a href=\"https://www.kaggle.com/teeyee314/cornell\">Link </a>to building metadata. That should save you guys some time. Don't forget to upvote my dataset ;)</p>\n\n<p>edit: to anyone looking to scrape/compile/match all the cornell data, you'll need to average the meter readings within each hour (sum and divide by 4) because they are in 15 minute increments, so </p>\n\n<blockquote>\n  <p>00:00:00 = (00:00:00 + 00:15:00 + 00:30:00 + 00:45:00) / 4. </p>\n</blockquote>\n\n<p>And you'll have to deal with the missing readings so good luck.</p>",
          "rawMarkdown": "I agree. I can confirm Site 15 is Cornell University. It requires some form fuzzy matching (sqft) for 70 of the buildings. The other 54/124 are exact match. [Link ](https://www.kaggle.com/teeyee314/cornell)to building metadata. That should save you guys some time. Don't forget to upvote my dataset ;)\n\nedit: to anyone looking to scrape/compile/match all the cornell data, you'll need to average the meter readings within each hour (sum and divide by 4) because they are in 15 minute increments, so \n\n&gt; 00:00:00 = (00:00:00 + 00:15:00 + 00:30:00 + 00:45:00) / 4. \n\nAnd you'll have to deal with the missing readings so good luck.",
          "votes": 3
        },
        {
          "id": 682813,
          "postDate": "2019-11-27T22:37:48.900Z",
          "content": "<p>Someone tried Princeton energy data (site14 potentially)? It is available online at:\n<a href=\"https://tiger-energy.appspot.com/data-download\">https://tiger-energy.appspot.com/data-download</a>\nBut it does not seem to match.</p>",
          "rawMarkdown": "Someone tried Princeton energy data (site14 potentially)? It is available online at:\nhttps://tiger-energy.appspot.com/data-download\nBut it does not seem to match.",
          "votes": 1
        }
      ]
    },
    {
      "id": 682806,
      "postDate": "2019-11-27T22:00:07.787Z",
      "content": "<p>There is a potential data leak for site 9 (Austin, TX):\nMany of the buildings are located within main campus of UT Austin. This may be confirmed by cross-checking competition data vs their buildings list here: <a href=\"https://utdirect.utexas.edu/apps/campus/buildings/nlogon/facilities/UTM/\">https://utdirect.utexas.edu/apps/campus/buildings/nlogon/facilities/UTM/</a>\nThe University seems to have a comprehensive site with energy management information <a href=\"https://utilities.utexas.edu/efficiency/energy-management\">https://utilities.utexas.edu/efficiency/energy-management</a>. Since this site is only available to the campus residents on intranet/VPN, it is not possible to use/access for most of us. However, given the size of UT, it may not be ruled out that some competition participants from U. of Texas may get an advantage. Could someone verify this?</p>",
      "rawMarkdown": "There is a potential data leak for site 9 (Austin, TX):\nMany of the buildings are located within main campus of UT Austin. This may be confirmed by cross-checking competition data vs their buildings list here: https://utdirect.utexas.edu/apps/campus/buildings/nlogon/facilities/UTM/\nThe University seems to have a comprehensive site with energy management information https://utilities.utexas.edu/efficiency/energy-management. Since this site is only available to the campus residents on intranet/VPN, it is not possible to use/access for most of us. However, given the size of UT, it may not be ruled out that some competition participants from U. of Texas may get an advantage. Could someone verify this?",
      "votes": 6
    },
    {
      "id": 692514,
      "postDate": "2019-12-11T11:07:44.967Z",
      "content": "<p>I use the leaked data in the following kernels\n0:<a href=\"https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels\">https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels</a>\n1:<a href=\"https://www.kaggle.com/mpware/ucl-data-leakage-episode-2\">https://www.kaggle.com/mpware/ucl-data-leakage-episode-2</a>\n2:<a href=\"https://www.kaggle.com/pdnartreb/scrap-asu-data\">https://www.kaggle.com/pdnartreb/scrap-asu-data</a>\n4:<a href=\"https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings\">https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings</a>\n15:<a href=\"https://www.kaggle.com/pp2file/ashrae-site15-cornell\">https://www.kaggle.com/pp2file/ashrae-site15-cornell</a>\n(sorry for repeating)</p>\n\n<p>【JUST MY THOUGHT】\n- We can use the above data.\n (I think it is not necessary for each of us to declare the exploitation of these data in this thread. Because these data already meet the external data condition written in rules 7C)\n- There may be other leak, but at least for me there seems to be no other leaked data that meet the condition of external data written in rules. </p>",
      "rawMarkdown": "I use the leaked data in the following kernels\n0:https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels\n1:https://www.kaggle.com/mpware/ucl-data-leakage-episode-2\n2:https://www.kaggle.com/pdnartreb/scrap-asu-data\n4:https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings\n15:https://www.kaggle.com/pp2file/ashrae-site15-cornell\n(sorry for repeating)\n\n【JUST MY THOUGHT】\n- We can use the above data.\n (I think it is not necessary for each of us to declare the exploitation of these data in this thread. Because these data already meet the external data condition written in rules 7C)\n- There may be other leak, but at least for me there seems to be no other leaked data that meet the condition of external data written in rules. ",
      "votes": 4
    },
    {
      "id": 693334,
      "postDate": "2019-12-12T09:12:42.230Z",
      "content": "<p><a href=\"https://github.com/dr-prodigy/python-holidays\">https://github.com/dr-prodigy/python-holidays</a>\n<a href=\"https://www.officeholidays.com/countries/\">https://www.officeholidays.com/countries/</a>\n<a href=\"https://www.timeanddate.com/holidays/\">https://www.timeanddate.com/holidays/</a>\n<a href=\"https://www.gov.uk/bank-holidays\">https://www.gov.uk/bank-holidays</a>\n<a href=\"https://blog.holidaylettings.co.uk/\">https://blog.holidaylettings.co.uk/</a>\n<a href=\"https://www.edwardbryantschool.co.uk/docs/term_dates/term-dates-2018-2019.pdf\">https://www.edwardbryantschool.co.uk/docs/term_dates/term-dates-2018-2019.pdf</a>\n<a href=\"https://www.kaggle.com/c/ashrae-energy-prediction/notebooks\">https://www.kaggle.com/c/ashrae-energy-prediction/notebooks</a></p>",
      "rawMarkdown": "https://github.com/dr-prodigy/python-holidays\nhttps://www.officeholidays.com/countries/\nhttps://www.timeanddate.com/holidays/\nhttps://www.gov.uk/bank-holidays\nhttps://blog.holidaylettings.co.uk/\nhttps://www.edwardbryantschool.co.uk/docs/term_dates/term-dates-2018-2019.pdf\nhttps://www.kaggle.com/c/ashrae-energy-prediction/notebooks",
      "votes": 1
    },
    {
      "id": 691358,
      "postDate": "2019-12-10T03:47:18.583Z",
      "content": "<p><a href=\"https://www.worldweatheronline.com/\">https://www.worldweatheronline.com/</a></p>",
      "rawMarkdown": "https://www.worldweatheronline.com/",
      "votes": 1
    },
    {
      "id": 665141,
      "postDate": "2019-11-04T17:11:20.113Z",
      "content": "<p>There is this <a href=\"https://github.com/buds-lab/the-building-data-genome-project\">Building Data Genome Project Database </a> and mention of Southampton University as data source in the [related article of Miller, Meggers] (<a href=\"http://www.sciencedirect.com/science/article/pii/S1876610217330047\">http://www.sciencedirect.com/science/article/pii/S1876610217330047</a>)\nJudging by sq ft and year built numbers it has about 250-300 overlaps with this contest's buildings table. The possible matches are for site_ids 1,2,4,5,6,13,15. </p>",
      "rawMarkdown": "There is this [Building Data Genome Project Database ](https://github.com/buds-lab/the-building-data-genome-project) and mention of Southampton University as data source in the [related article of Miller, Meggers] (http://www.sciencedirect.com/science/article/pii/S1876610217330047)\nJudging by sq ft and year built numbers it has about 250-300 overlaps with this contest's buildings table. The possible matches are for site_ids 1,2,4,5,6,13,15. ",
      "votes": 1
    },
    {
      "id": 655700,
      "postDate": "2019-10-23T11:40:31.253Z",
      "content": "<p><a href=\"https://www.kaggle.com/selfishgene/historical-hourly-weather-data\">https://www.kaggle.com/selfishgene/historical-hourly-weather-data</a></p>",
      "rawMarkdown": "https://www.kaggle.com/selfishgene/historical-hourly-weather-data",
      "votes": 1
    },
    {
      "id": 688543,
      "postDate": "2019-12-05T18:04:57.400Z",
      "content": "<p>Quite many of the buildings in the dataset are located a university campuses, even many of those with types 'office', 'lodging', 'parking'. It may help to use the academic calendars to predict consumption patterns related to school breaks (these shift a bit from year to year).\nFor starters: \nsite 9: 2015-16 academic calendar for UT Austin <a href=\"https://registrar.utexas.edu/calendars/15-16\">https://registrar.utexas.edu/calendars/15-16</a>\nsite 15: 2015-16 academic calendar for Cornell <a href=\"http://courses.cornell.edu/mime/media/26/11593/2015-2016+Academic+Calendar.pdf\">http://courses.cornell.edu/mime/media/26/11593/2015-2016+Academic+Calendar.pdf</a>\nNotice for example, that spring breaks do not coincide between these two. \nI presume that accounting for academic events (breaks, exams) may improve model results ...</p>",
      "rawMarkdown": "Quite many of the buildings in the dataset are located a university campuses, even many of those with types 'office', 'lodging', 'parking'. It may help to use the academic calendars to predict consumption patterns related to school breaks (these shift a bit from year to year).\nFor starters: \nsite 9: 2015-16 academic calendar for UT Austin https://registrar.utexas.edu/calendars/15-16\nsite 15: 2015-16 academic calendar for Cornell http://courses.cornell.edu/mime/media/26/11593/2015-2016+Academic+Calendar.pdf\nNotice for example, that spring breaks do not coincide between these two. \nI presume that accounting for academic events (breaks, exams) may improve model results ...",
      "votes": 2
    },
    {
      "id": 680509,
      "postDate": "2019-11-24T19:38:00.247Z",
      "content": "<p>Site 15 revealed\n<a href=\"http://portal.emcs.cornell.edu/GannettHealthCenter\">http://portal.emcs.cornell.edu/GannettHealthCenter</a>\nrefering to <a href=\"https://www.kaggle.com/pp2file/ashrae-site15-cornell\">https://www.kaggle.com/pp2file/ashrae-site15-cornell</a></p>",
      "rawMarkdown": "Site 15 revealed\nhttp://portal.emcs.cornell.edu/GannettHealthCenter\nrefering to https://www.kaggle.com/pp2file/ashrae-site15-cornell",
      "votes": 2
    },
    {
      "id": 679916,
      "postDate": "2019-11-23T16:45:46.217Z",
      "content": "<p>Site 3 only building metadata, no meter readings\n<a href=\"https://opendata.dc.gov/datasets/building-energy-benchmarks/data\">https://opendata.dc.gov/datasets/building-energy-benchmarks/data</a></p>",
      "rawMarkdown": "Site 3 only building metadata, no meter readings\nhttps://opendata.dc.gov/datasets/building-energy-benchmarks/data",
      "votes": 2,
      "replies": [
        {
          "id": 680601,
          "postDate": "2019-11-25T00:08:49.977Z",
          "content": "<p>have，data，just you don't find it.</p>",
          "rawMarkdown": "have，data，just you don't find it."
        }
      ]
    },
    {
      "id": 690951,
      "postDate": "2019-12-09T11:44:48.870Z",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a>\nAll these leak data stations used in my submission</p>",
      "rawMarkdown": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nAll these leak data stations used in my submission"
    },
    {
      "id": 664019,
      "postDate": "2019-11-03T03:39:35.503Z",
      "content": "<p>historical weather data in Ireland and London.\n<a href=\"https://www.kaggle.com/conorrot/irish-weather-hourly-data\">https://www.kaggle.com/conorrot/irish-weather-hourly-data</a>\n<a href=\"https://www.kaggle.com/jeanmidev/smart-meters-in-london\">https://www.kaggle.com/jeanmidev/smart-meters-in-london</a></p>",
      "rawMarkdown": "historical weather data in Ireland and London.\nhttps://www.kaggle.com/conorrot/irish-weather-hourly-data\nhttps://www.kaggle.com/jeanmidev/smart-meters-in-london",
      "votes": 2
    },
    {
      "id": 654925,
      "postDate": "2019-10-22T13:52:01.620Z",
      "content": "<p><a href=\"https://github.com/buds-lab/the-building-data-genome-project\">https://github.com/buds-lab/the-building-data-genome-project</a>\n<a href=\"https://github.com/buds-lab/building-prediction-benchmarking\">https://github.com/buds-lab/building-prediction-benchmarking</a></p>",
      "rawMarkdown": "https://github.com/buds-lab/the-building-data-genome-project\nhttps://github.com/buds-lab/building-prediction-benchmarking",
      "votes": 2
    },
    {
      "id": 653619,
      "postDate": "2019-10-20T18:04:35.677Z",
      "content": "<p><a href=\"https://www.eia.gov/\">https://www.eia.gov/</a></p>\n\n<p>Not sure why so many of the datasets only available in monthly chunks but good data here for energy consumption</p>",
      "rawMarkdown": "https://www.eia.gov/\n\nNot sure why so many of the datasets only available in monthly chunks but good data here for energy consumption\n\n",
      "votes": 2
    },
    {
      "id": 670140,
      "postDate": "2019-11-11T04:27:18.330Z",
      "content": "<p>calendar of each year\n<a href=\"https://www.timeanddate.com/calendar/\">https://www.timeanddate.com/calendar/</a></p>",
      "rawMarkdown": "calendar of each year\nhttps://www.timeanddate.com/calendar/"
    },
    {
      "id": 694129,
      "postDate": "2019-12-13T08:16:51.807Z",
      "content": "<p><a href=\"/sohier\">@sohier</a> Do you already know which sites are not scored in private?</p>",
      "rawMarkdown": "@sohier Do you already know which sites are not scored in private?",
      "votes": 1
    },
    {
      "id": 691666,
      "postDate": "2019-12-10T11:38:51.980Z",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a></p>",
      "rawMarkdown": "\nhttps://www.kaggle.com/yamsam/ashrae-leak-data-station",
      "votes": -1
    },
    {
      "id": 691425,
      "postDate": "2019-12-10T05:58:42.970Z",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a></p>",
      "rawMarkdown": "https://www.kaggle.com/yamsam/ashrae-leak-data-station",
      "votes": -1
    },
    {
      "id": 698694,
      "postDate": "2019-12-19T15:31:19.987Z",
      "content": "<p><a href=\"https://www.kaggle.com/mimoudata/ashrae-2-lightgbm-without-leak-data\">https://www.kaggle.com/mimoudata/ashrae-2-lightgbm-without-leak-data</a></p>",
      "rawMarkdown": "https://www.kaggle.com/mimoudata/ashrae-2-lightgbm-without-leak-data"
    },
    {
      "id": 694671,
      "postDate": "2019-12-14T00:40:07.673Z",
      "content": "<p>Was anyone able to fill in year_built and floor_count using these leaks? </p>",
      "rawMarkdown": "Was anyone able to fill in year_built and floor_count using these leaks? "
    },
    {
      "id": 694015,
      "postDate": "2019-12-13T04:02:02.200Z",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a>\nAll these leak data stations used in my submission</p>",
      "rawMarkdown": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nAll these leak data stations used in my submission"
    },
    {
      "id": 693902,
      "postDate": "2019-12-12T22:55:23.413Z",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a>\n<a href=\"https://www.kaggle.com/selfishgene/historical-hourly-weather-data\">https://www.kaggle.com/selfishgene/historical-hourly-weather-data</a></p>",
      "rawMarkdown": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nhttps://www.kaggle.com/selfishgene/historical-hourly-weather-data"
    },
    {
      "id": 693827,
      "postDate": "2019-12-12T20:32:37.580Z",
      "content": "<p>Airport data from OpenFlights.org available at: <a href=\"https://openflights.org/data.html\">https://openflights.org/data.html</a> (and directly from <a href=\"https://raw.githubusercontent.com/jpatokal/openflights/master/data/airports.dat\">https://raw.githubusercontent.com/jpatokal/openflights/master/data/airports.dat</a> )</p>",
      "rawMarkdown": "Airport data from OpenFlights.org available at: https://openflights.org/data.html (and directly from https://raw.githubusercontent.com/jpatokal/openflights/master/data/airports.dat )"
    },
    {
      "id": 693665,
      "postDate": "2019-12-12T16:50:19.327Z",
      "content": "<p>weather information - <a href=\"https://www.ncei.noaa.gov/data/global-hourly/archive/csv/\">https://www.ncei.noaa.gov/data/global-hourly/archive/csv/</a></p>",
      "rawMarkdown": "weather information - https://www.ncei.noaa.gov/data/global-hourly/archive/csv/"
    },
    {
      "id": 693490,
      "postDate": "2019-12-12T12:41:40.163Z",
      "content": "<p>We have used below kernels and data from external website : \n1. <a href=\"https://www.kaggle.com/wuliaokaola/ashrae-maybe-this-can-make-public-lb-some-useful\"> https://www.kaggle.com/wuliaokaola/ashrae-maybe-this-can-make-public-lb-some-useful</a>\n2. <a href=\"https://www.kaggle.com/yamsam/new-ucf-starter-kernel\">https://www.kaggle.com/yamsam/new-ucf-starter-kernel</a>\n3. <a href=\"https://www.kaggle.com/mpware/ucl-data-leakage-episode-2\">https://www.kaggle.com/mpware/ucl-data-leakage-episode-2</a>\n4. <a href=\"https://www.kaggle.com/pdnartreb/asu-buildings-energy-consumption\">https://www.kaggle.com/pdnartreb/asu-buildings-energy-consumption</a>\n5. <a href=\"https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings\">https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings</a>\n6. <a href=\"https://www.kaggle.com/pp2file/ashrae-site15-cornell\">https://www.kaggle.com/pp2file/ashrae-site15-cornell</a>\n7. <a href=\"https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115040#latest-667889\">https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115040#latest-667889</a>\n8. <a href=\"https://www.kaggle.com/patrick0302/locate-cities-according-weather-temperature\">https://www.kaggle.com/patrick0302/locate-cities-according-weather-temperature</a>\n9. <a href=\"https://power.larc.nasa.gov/data-access-viewer/\">https://power.larc.nasa.gov/data-access-viewer/</a></p>",
      "rawMarkdown": "We have used below kernels and data from external website : \n1. [ https://www.kaggle.com/wuliaokaola/ashrae-maybe-this-can-make-public-lb-some-useful](https://www.kaggle.com/wuliaokaola/ashrae-maybe-this-can-make-public-lb-some-useful)\n2. [https://www.kaggle.com/yamsam/new-ucf-starter-kernel](https://www.kaggle.com/yamsam/new-ucf-starter-kernel)\n3. [https://www.kaggle.com/mpware/ucl-data-leakage-episode-2](https://www.kaggle.com/mpware/ucl-data-leakage-episode-2)\n4. [https://www.kaggle.com/pdnartreb/asu-buildings-energy-consumption](https://www.kaggle.com/pdnartreb/asu-buildings-energy-consumption)\n5. [https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings](https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings)\n6. [https://www.kaggle.com/pp2file/ashrae-site15-cornell](https://www.kaggle.com/pp2file/ashrae-site15-cornell)\n7. [https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115040#latest-667889](https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115040#latest-667889)\n8. [https://www.kaggle.com/patrick0302/locate-cities-according-weather-temperature](https://www.kaggle.com/patrick0302/locate-cities-according-weather-temperature)\n9. [https://power.larc.nasa.gov/data-access-viewer/](https://power.larc.nasa.gov/data-access-viewer/)"
    },
    {
      "id": 693447,
      "postDate": "2019-12-12T11:32:51.597Z",
      "content": "<p>I use the leaked data in the following kernels\n0:<a href=\"https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels\">https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels</a>\n1:<a href=\"https://www.kaggle.com/mpware/ucl-data-leakage-episode-2\">https://www.kaggle.com/mpware/ucl-data-leakage-episode-2</a>\n2:<a href=\"https://www.kaggle.com/pdnartreb/scrap-asu-data\">https://www.kaggle.com/pdnartreb/scrap-asu-data</a>\n4:<a href=\"https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings\">https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings</a>\n15:<a href=\"https://www.kaggle.com/pp2file/ashrae-site15-cornell\">https://www.kaggle.com/pp2file/ashrae-site15-cornell</a>\n<a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a>\n(sorry for repeating)</p>",
      "rawMarkdown": "I use the leaked data in the following kernels\n0:https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels\n1:https://www.kaggle.com/mpware/ucl-data-leakage-episode-2\n2:https://www.kaggle.com/pdnartreb/scrap-asu-data\n4:https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings\n15:https://www.kaggle.com/pp2file/ashrae-site15-cornell\nhttps://www.kaggle.com/yamsam/ashrae-leak-data-station\n(sorry for repeating)"
    },
    {
      "id": 693374,
      "postDate": "2019-12-12T10:10:21.210Z",
      "content": "<p><a href=\"https://pypi.org/project/meteocalc/\">https://pypi.org/project/meteocalc/</a></p>\n\n<p>For weather features</p>",
      "rawMarkdown": "https://pypi.org/project/meteocalc/\n\nFor weather features"
    },
    {
      "id": 692810,
      "postDate": "2019-12-11T18:50:34.437Z",
      "content": "<p>As far as i know, those platform can be used to get hourly energy data (and are publicly available data)\n<a href=\"http://cm.asu.edu/\">http://cm.asu.edu/</a>\n<a href=\"https://www.oeis.ucf.edu/\">https://www.oeis.ucf.edu/</a>\n<a href=\"http://portal.emcs.cornell.edu/\">http://portal.emcs.cornell.edu/</a>\n<a href=\"https://platform.carbonculture.net/\">https://platform.carbonculture.net/</a>\n<a href=\"https://engagementdashboard.com/ucb/ucb/\">https://engagementdashboard.com/ucb/ucb/</a>\n<a href=\"https://engagementdashboard.com/ucsb/ucsb\">https://engagementdashboard.com/ucsb/ucsb</a>\n<a href=\"https://engagementdashboard.com/sd36/sd36\">https://engagementdashboard.com/sd36/sd36</a>\n<a href=\"https://tiger-energy.appspot.com/data-download\">https://tiger-energy.appspot.com/data-download</a></p>",
      "rawMarkdown": "As far as i know, those platform can be used to get hourly energy data (and are publicly available data)\nhttp://cm.asu.edu/\nhttps://www.oeis.ucf.edu/\nhttp://portal.emcs.cornell.edu/\nhttps://platform.carbonculture.net/\nhttps://engagementdashboard.com/ucb/ucb/\nhttps://engagementdashboard.com/ucsb/ucsb\nhttps://engagementdashboard.com/sd36/sd36\nhttps://tiger-energy.appspot.com/data-download"
    },
    {
      "id": 692709,
      "postDate": "2019-12-11T16:48:02.630Z",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a>\nAll these leak data stations used in my submission</p>",
      "rawMarkdown": "[https://www.kaggle.com/yamsam/ashrae-leak-data-station](https://www.kaggle.com/yamsam/ashrae-leak-data-station)\nAll these leak data stations used in my submission"
    },
    {
      "id": 692277,
      "postDate": "2019-12-11T05:11:02.493Z",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a>\nAll these leak data stations used in my submission</p>",
      "rawMarkdown": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nAll these leak data stations used in my submission"
    },
    {
      "id": 691951,
      "postDate": "2019-12-10T17:35:00.870Z",
      "content": "<p><a href=\"https://pypi.org/project/holidays/\">holidays library</a></p>",
      "rawMarkdown": "[holidays library](https://pypi.org/project/holidays/)"
    },
    {
      "id": 690817,
      "postDate": "2019-12-09T07:52:20.763Z",
      "content": "<p><a href=\"https://opendata.dc.gov/datasets/building-energy-benchmarks/data\">https://opendata.dc.gov/datasets/building-energy-benchmarks/data</a>\n<a href=\"http://portal.emcs.cornell.edu/GannettHealthCenter\">http://portal.emcs.cornell.edu/GannettHealthCenter</a>\n<a href=\"https://platform.carbonculture.net/communities/ucl/30/apps/assets/list/place/\">https://platform.carbonculture.net/communities/ucl/30/apps/assets/list/place/</a>\n<a href=\"https://www.kaggle.com/selfishgene/historical-hourly-weather-data\">https://www.kaggle.com/selfishgene/historical-hourly-weather-data</a>\n<a href=\"https://www.data.jma.go.jp/gmd/cpd/monitor/climatview/frame.php\">https://www.data.jma.go.jp/gmd/cpd/monitor/climatview/frame.php</a>\n<a href=\"https://www.kaggle.com/selfishgene/historical-hourly-weather-data\">https://www.kaggle.com/selfishgene/historical-hourly-weather-data</a>\n<a href=\"https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/numerical-weather-prediction\">https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/numerical-weather-prediction</a>
<a href=\"https://www.orlando.gov/files/sharedassets/public/initiatives/bewes/090418_cityoforlando_bldginfo.pdf\">https://www.orlando.gov/files/sharedassets/public/initiatives/bewes/090418_cityoforlando_bldginfo.pdf</a>
<a href=\"http://www.thecb.state.tx.us/reports/xls/7559.xlsx\">http://www.thecb.state.tx.us/reports/xls/7559.xlsx</a>
<a href=\"https://mn.b3benchmarking.com/Report?OrganizationID=1337&amp;ReturnTo=Higher+Education\">https://mn.b3benchmarking.com/Report?OrganizationID=1337&amp;ReturnTo=Higher+Education</a>
<a href=\"https://www.fm.virginia.edu/docs/conditionReport/2018\">https://www.fm.virginia.edu/docs/conditionReport/2018</a></p>",
      "rawMarkdown": "https://opendata.dc.gov/datasets/building-energy-benchmarks/data\nhttp://portal.emcs.cornell.edu/GannettHealthCenter\nhttps://platform.carbonculture.net/communities/ucl/30/apps/assets/list/place/\nhttps://www.kaggle.com/selfishgene/historical-hourly-weather-data\nhttps://www.data.jma.go.jp/gmd/cpd/monitor/climatview/frame.php\nhttps://www.kaggle.com/selfishgene/historical-hourly-weather-data\nhttps://www.ncdc.noaa.gov/data-access/model-data/model-datasets/numerical-weather-prediction
https://www.orlando.gov/files/sharedassets/public/initiatives/bewes/090418_cityoforlando_bldginfo.pdf
http://www.thecb.state.tx.us/reports/xls/7559.xlsx
https://mn.b3benchmarking.com/Report?OrganizationID=1337&amp;ReturnTo=Higher+Education
https://www.fm.virginia.edu/docs/conditionReport/2018\n"
    },
    {
      "id": 690811,
      "postDate": "2019-12-09T07:40:34.957Z",
      "content": "<p>Hi Dane,\nYour post on the modified rules says: \"Using publicly available meter readings is within the rules, assuming the general restrictions are also followed such as posting a link to the data in the external data thread by the entry deadline (December 12th).\" So, every participant has to publish the list of external data sources?</p>\n\n<p>Instead, since you will be doing a review after the deadline of December 12, may I request that you publish a list of accepted External data sites? In that way, we do not have to bother about which external link cannot be used.</p>\n\n<p>At this point (thanks to the enterprise of the relevant active Kagglers), I am using the external data sites mentioned in the following Kernels:\nASHRAE - UCF Spider and EDA (Full Test Labels) v3\nUCL: Data Leakage (Episode 2) v1\nASU train and scraped test data v7\nUCB: Data Leakage (Site 4) v14\nASHRAE-site15-cornell v3</p>\n\n<p>Thanks to the initiative of  <a href=\"/gunesevitan\">@gunesevitan</a>, <a href=\"/mpware\">@mpware</a>, <a href=\"/poedator\">@poedator</a>, <a href=\"/pdnartreb\">@pdnartreb</a>, <a href=\"/serengil\">@serengil</a> @IsamuYamashita and <a href=\"/pp2file\">@pp2file</a></p>",
      "rawMarkdown": "Hi Dane,\nYour post on the modified rules says: \"Using publicly available meter readings is within the rules, assuming the general restrictions are also followed such as posting a link to the data in the external data thread by the entry deadline (December 12th).\" So, every participant has to publish the list of external data sources?\n\nInstead, since you will be doing a review after the deadline of December 12, may I request that you publish a list of accepted External data sites? In that way, we do not have to bother about which external link cannot be used.\n\nAt this point (thanks to the enterprise of the relevant active Kagglers), I am using the external data sites mentioned in the following Kernels:\nASHRAE - UCF Spider and EDA (Full Test Labels) v3\nUCL: Data Leakage (Episode 2) v1\nASU train and scraped test data v7\nUCB: Data Leakage (Site 4) v14\nASHRAE-site15-cornell v3\n\nThanks to the initiative of  @gunesevitan, @mpware, @poedator, @pdnartreb, @serengil @IsamuYamashita and @pp2file"
    },
    {
      "id": 690800,
      "postDate": "2019-12-09T07:08:09.030Z",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a>\nAll these leak data stations used in my submission\nAnd data that other kagglers announced</p>",
      "rawMarkdown": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nAll these leak data stations used in my submission\nAnd data that other kagglers announced"
    },
    {
      "id": 690140,
      "postDate": "2019-12-08T03:10:40.597Z",
      "content": "<p><a href=\"https://www.officeholidays.com/\">https://www.officeholidays.com/</a> for holiday info.</p>",
      "rawMarkdown": "https://www.officeholidays.com/ for holiday info."
    },
    {
      "id": 689809,
      "postDate": "2019-12-07T13:17:22.957Z",
      "content": "<p>I used doc2vec with wikipedia dumps <a href=\"https://dumps.wikimedia.org/enwiki/\">https://dumps.wikimedia.org/enwiki/</a>\nIt is probably not relevant to what the thread is meant to be but just in case...</p>",
      "rawMarkdown": "I used doc2vec with wikipedia dumps https://dumps.wikimedia.org/enwiki/\nIt is probably not relevant to what the thread is meant to be but just in case..."
    },
    {
      "id": 689632,
      "postDate": "2019-12-07T07:46:53.417Z",
      "content": "<p>used leaked data from sites 0,1,2,4,15</p>",
      "rawMarkdown": "used leaked data from sites 0,1,2,4,15"
    },
    {
      "id": 689588,
      "postDate": "2019-12-07T05:43:25.253Z",
      "content": "<p>Hi <a href=\"/sohier\">@sohier</a>  As per my understanding site 0,1,2,4 &amp; 15 data has been leaked. Does it mean that these sites will be excluded from private leaderboard test data?</p>",
      "rawMarkdown": "Hi @sohier  As per my understanding site 0,1,2,4 &amp; 15 data has been leaked. Does it mean that these sites will be excluded from private leaderboard test data?"
    },
    {
      "id": 689193,
      "postDate": "2019-12-06T16:00:39.847Z",
      "content": "<p>Would formulas (involving constants only) be considered External Data and must be disclosed?</p>",
      "rawMarkdown": "Would formulas (involving constants only) be considered External Data and must be disclosed?",
      "replies": [
        {
          "id": 689403,
          "postDate": "2019-12-06T22:18:13.617Z",
          "content": "<p>There's no need to worry about something trivial that you might find in a textbook, like unit conversion factors. However, I can't give a blanket pass since \"a formula with constants\" could contain almost anything. If that doesn't answer your question feel free to send me a direct message.</p>",
          "rawMarkdown": "There's no need to worry about something trivial that you might find in a textbook, like unit conversion factors. However, I can't give a blanket pass since \"a formula with constants\" could contain almost anything. If that doesn't answer your question feel free to send me a direct message."
        }
      ]
    },
    {
      "id": 688871,
      "postDate": "2019-12-06T07:31:20.253Z",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a>\nAll these leak data stations used in my submission</p>",
      "rawMarkdown": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nAll these leak data stations used in my submission\n\n"
    },
    {
      "id": 688032,
      "postDate": "2019-12-05T06:45:01.667Z",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a>\nAll these leak data stations used in my submission</p>",
      "rawMarkdown": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nAll these leak data stations used in my submission"
    },
    {
      "id": 686318,
      "postDate": "2019-12-03T02:47:04.580Z",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a>\nAll these leak data stations used in my submission</p>",
      "rawMarkdown": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nAll these leak data stations used in my submission"
    },
    {
      "id": 680603,
      "postDate": "2019-11-25T00:13:50.583Z",
      "content": "<p>so far. maybe have 6 site energy  data  has be leajed.</p>",
      "rawMarkdown": "so far. maybe have 6 site energy  data  has be leajed."
    },
    {
      "id": 678257,
      "postDate": "2019-11-21T06:43:37.260Z",
      "content": "<p>Will publicly available values be excluded from the final validation or are there plans to jitter them such that competition is made more fair in the interest of intellectual honesty?  Doing so could encourage honest Kagglers to continue in earnest...</p>",
      "rawMarkdown": "Will publicly available values be excluded from the final validation or are there plans to jitter them such that competition is made more fair in the interest of intellectual honesty?  Doing so could encourage honest Kagglers to continue in earnest...",
      "replies": [
        {
          "id": 678577,
          "postDate": "2019-11-21T15:13:05.633Z",
          "content": "<p>See <a href=\"https://www.kaggle.com/c/ashrae-energy-prediction/discussion/117357\">https://www.kaggle.com/c/ashrae-energy-prediction/discussion/117357</a></p>",
          "rawMarkdown": "See https://www.kaggle.com/c/ashrae-energy-prediction/discussion/117357"
        }
      ]
    },
    {
      "id": 674110,
      "postDate": "2019-11-15T22:42:42Z",
      "content": "<p>Not sure how useful this is going to be, but I'm planning on experimenting with <a href=\"https://www.esrl.noaa.gov/gmd/grad/solcalc/\">https://www.esrl.noaa.gov/gmd/grad/solcalc/</a> and <a href=\"https://pysolar.readthedocs.io/en/latest/\">pysolar</a>.</p>",
      "rawMarkdown": "Not sure how useful this is going to be, but I'm planning on experimenting with https://www.esrl.noaa.gov/gmd/grad/solcalc/ and [pysolar](https://pysolar.readthedocs.io/en/latest/)."
    },
    {
      "id": 673332,
      "postDate": "2019-11-14T21:05:17.833Z",
      "content": "<p><a href=\"https://www.timeanddate.com/worldclock/\">https://www.timeanddate.com/worldclock/</a></p>",
      "rawMarkdown": "https://www.timeanddate.com/worldclock/\n"
    },
    {
      "id": 671680,
      "postDate": "2019-11-13T03:23:55.450Z",
      "content": "<p><a href=\"https://en.wikipedia.org/wiki/List_of_cities_by_average_temperature\">https://en.wikipedia.org/wiki/List_of_cities_by_average_temperature</a>\n<a href=\"https://www.wunderground.com/\">https://www.wunderground.com/</a>\n<a href=\"https://github.com/buds-lab/island-of-misfit-buildings\">https://github.com/buds-lab/island-of-misfit-buildings</a></p>",
      "rawMarkdown": "https://en.wikipedia.org/wiki/List_of_cities_by_average_temperature\nhttps://www.wunderground.com/\nhttps://github.com/buds-lab/island-of-misfit-buildings"
    },
    {
      "id": 670757,
      "postDate": "2019-11-11T19:50:51.123Z",
      "content": "<p>Lots of weather data on NOAA website:\n<a href=\"https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/numerical-weather-prediction\">https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/numerical-weather-prediction</a>\n<a href=\"https://www.ncdc.noaa.gov/data-access/radar-data/noaa-big-data-project\">https://www.ncdc.noaa.gov/data-access/radar-data/noaa-big-data-project</a>\n<a href=\"https://www.ncdc.noaa.gov/data-access/land-based-station-data\">https://www.ncdc.noaa.gov/data-access/land-based-station-data</a></p>",
      "rawMarkdown": "Lots of weather data on NOAA website:\nhttps://www.ncdc.noaa.gov/data-access/model-data/model-datasets/numerical-weather-prediction\nhttps://www.ncdc.noaa.gov/data-access/radar-data/noaa-big-data-project\nhttps://www.ncdc.noaa.gov/data-access/land-based-station-data"
    },
    {
      "id": 668847,
      "postDate": "2019-11-09T00:24:57.433Z",
      "content": "<p><a href=\"https://www.sports-reference.com/cfb/years\">https://www.sports-reference.com/cfb/years</a></p>\n\n<p>college football schedule - 2016 to present\nhigh consumption of beer - assume the same for energy but TBD</p>",
      "rawMarkdown": "https://www.sports-reference.com/cfb/years\n\ncollege football schedule - 2016 to present\nhigh consumption of beer - assume the same for energy but TBD"
    },
    {
      "id": 664159,
      "postDate": "2019-11-03T08:54:30.943Z",
      "content": "<p><a href=\"https://darksky.net/dev\">https://darksky.net/dev</a></p>",
      "rawMarkdown": "https://darksky.net/dev"
    },
    {
      "id": 662218,
      "postDate": "2019-10-31T08:41:33.913Z",
      "content": "<p><a href=\"https://www.data.jma.go.jp/gmd/cpd/monitor/climatview/frame.php\">https://www.data.jma.go.jp/gmd/cpd/monitor/climatview/frame.php</a>\nIt will help to find site id location</p>",
      "rawMarkdown": "https://www.data.jma.go.jp/gmd/cpd/monitor/climatview/frame.php\nIt will help to find site id location"
    },
    {
      "id": 657441,
      "postDate": "2019-10-25T06:32:10.173Z",
      "content": "<p><a href=\"https://www.ncei.noaa.gov/support/access-data-service-api-user-documentation\">https://www.ncei.noaa.gov/support/access-data-service-api-user-documentation</a></p>\n\n<p><a href=\"https://www.ashrae.org/communities/regions\">https://www.ashrae.org/communities/regions</a></p>",
      "rawMarkdown": "https://www.ncei.noaa.gov/support/access-data-service-api-user-documentation\n\nhttps://www.ashrae.org/communities/regions"
    },
    {
      "id": 692979,
      "postDate": "2019-12-11T23:23:55.783Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 690200,
      "postDate": "2019-12-08T05:41:56.740Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 690195,
      "postDate": "2019-12-08T05:39:29.573Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 674684,
      "postDate": "2019-11-16T22:20:40.497Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 670787,
      "author_name": "WispZero",
      "author_url": "",
      "post_date": "2019-11-11T21:05:45.473000",
      "content": "<p><a href=\"/sohier\">@sohier</a> We are asking to clarify using web scrapped data as predicted values. That is clearly a data leak.</p>",
      "votes": 25,
      "replies": []
    },
    {
      "id": 665274,
      "author_name": "Poe Dator",
      "author_url": "",
      "post_date": "2019-11-04T20:35:31.940000",
      "content": "<p>for a number of buildings / sites it is possible to find information in the Internet using area and construction year as a key.  I made a more detailed post [here] (<a href=\"https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115698\">https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115698</a>).\nBelow is a list of sources that mention buildings from a few areas: </p>\n\n<p><strong>0: Orlando, FL</strong>\n    <a href=\"https://www.oeis.ucf.edu/buildings\">https://www.oeis.ucf.edu/buildings</a></p>\n\n<p><strong>2: Tempe, Arizona</strong> \n    <a href=\"https://www.asu.edu/purchasing/bids/pdfs/231906_ex_6A.pdf\">https://www.asu.edu/purchasing/bids/pdfs/231906_ex_6A.pdf</a></p>\n\n<p><strong>3: DC</strong>\n<a href=\"https://dgs.dc.gov/sites/default/files/dc/sites/dgs/service_content/attachments/Attachment%20A-%20Municipal%20Facilities%5b1%5d.pdf\">https://dgs.dc.gov/sites/default/files/dc/sites/dgs/service_content/attachments/Attachment%20A-%20Municipal%20Facilities%5b1%5d.pdf</a></p>\n\n<p><strong>8: Orlando, FL</strong>\n<a href=\"https://www.orlando.gov/files/sharedassets/public/initiatives/bewes/090418_cityoforlando_bldginfo.pdf\">https://www.orlando.gov/files/sharedassets/public/initiatives/bewes/090418_cityoforlando_bldginfo.pdf</a></p>\n\n<p><strong>9: Austin, TX</strong>\n    <a href=\"http://www.thecb.state.tx.us/reports/xls/7559.xlsx\">http://www.thecb.state.tx.us/reports/xls/7559.xlsx</a></p>\n\n<p><strong>13: TwinCities, MN</strong>\n<a href=\"https://mn.b3benchmarking.com/Report?OrganizationID=1337&amp;ReturnTo=Higher+Education\">https://mn.b3benchmarking.com/Report?OrganizationID=1337&amp;ReturnTo=Higher+Education</a></p>\n\n<p><strong>14: Charlottesville, VA</strong>\n<a href=\"https://www.fm.virginia.edu/docs/conditionReport/2018-19FacilitiesConditionReport.pdf\">https://www.fm.virginia.edu/docs/conditionReport/2018-19FacilitiesConditionReport.pdf</a> (has year built data)</p>",
      "votes": 19,
      "replies": [
        {
          "id": 668345,
          "author_name": "Louka Ewington-Pitsos",
          "author_url": "",
          "post_date": "2019-11-08T09:59:33.317000",
          "content": "<p>hey so just like... based on that I'm gonna go scrape all ucf's energy data... That's within the rules right? I'm allowed to just go get that?</p>\n\n<p>doesn't feel right...</p>",
          "votes": 2,
          "replies": []
        }
      ]
    },
    {
      "id": 673832,
      "author_name": "Gunes Evitan",
      "author_url": "",
      "post_date": "2019-11-15T14:38:51.157000",
      "content": "<p>Site 1 UCL building metadata and meter readings\n<a href=\"https://platform.carbonculture.net/communities/ucl/30/apps/assets/list/place/\">https://platform.carbonculture.net/communities/ucl/30/apps/assets/list/place/</a></p>",
      "votes": 18,
      "replies": [
        {
          "id": 673903,
          "author_name": "Jie Lu",
          "author_url": "",
          "post_date": "2019-11-15T16:21:03.310000",
          "content": "<p>OK, another one revealed ? As I see you climbed up in the LB 😄 </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 673926,
          "author_name": "Panos",
          "author_url": "",
          "post_date": "2019-11-15T17:01:35.357000",
          "content": "<p>Thanks for this, but it is terrible news! One more site data publicly available!\nIt´s a pity, this could have been a very interesting competition...</p>",
          "votes": 7,
          "replies": []
        },
        {
          "id": 674815,
          "author_name": "PGiN",
          "author_url": "",
          "post_date": "2019-11-17T05:42:33.490000",
          "content": "<p>Another one? OMG...😭</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 674960,
          "author_name": "Isamu",
          "author_url": "",
          "post_date": "2019-11-17T10:42:41.897000",
          "content": "<p>Has anyone planned to make UCL dataset and to make it public ?</p>\n\n<p>If no one, i will try it next week.</p>",
          "votes": 4,
          "replies": []
        },
        {
          "id": 675059,
          "author_name": "eagle4",
          "author_url": "",
          "post_date": "2019-11-17T14:38:34.457000",
          "content": "<p><a href=\"/gunesevitan\">@gunesevitan</a> , may I ask if cm.asu.edu is scrapable ? if yes, how do you proceed ? I can't find the actual table or a way to query the website.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 675067,
          "author_name": "Gunes Evitan",
          "author_url": "",
          "post_date": "2019-11-17T14:52:56.353000",
          "content": "<p>It is actually very easy to scrape <a href=\"http://cm.asu.edu/\">http://cm.asu.edu/</a>. It took me 15 minutes to write the spider and you don't even need Scrapy.</p>\n\n<ul>\n<li>Open the website on chrome and copy your session cookie because server won't return a valid response without it</li>\n<li>Find the request that returns building ids from XHR tab in network, and collect the building ids (use session cookie in headers)</li>\n<li>Find the request that returns meter readings and pass building id to it while looping through scraped building ids and accumulate the meter readings in a dataframe (use session cookie in every request)</li>\n</ul>\n\n<p>One big problem is the server returns the response in approximately  3 minutes, so it'll take too much time to scrape it completely</p>",
          "votes": 5,
          "replies": []
        },
        {
          "id": 675094,
          "author_name": "MPWARE",
          "author_url": "",
          "post_date": "2019-11-17T15:40:40.910000",
          "content": "<p><a href=\"/yamsam\">@yamsam</a>  Yes, <a href=\"https://www.kaggle.com/mpware/ucl-data-leakage-episode-2\">https://www.kaggle.com/mpware/ucl-data-leakage-episode-2</a></p>",
          "votes": 8,
          "replies": []
        },
        {
          "id": 675289,
          "author_name": "Isamu",
          "author_url": "",
          "post_date": "2019-11-17T21:14:45.563000",
          "content": "<p>Thank you <a href=\"/gunesevitan\">@gunesevitan</a> <a href=\"/mpware\">@mpware</a> . You are so great!! I will update my kernel.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 675819,
          "author_name": "HarshitMehta",
          "author_url": "",
          "post_date": "2019-11-18T15:51:26.113000",
          "content": "<p>Organizers should have provided the link to scrape the data rather than sharing train/test set :D</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 676354,
          "author_name": "Tim Yee",
          "author_url": "",
          "post_date": "2019-11-19T05:28:23.123000",
          "content": "<p><a href=\"/gunesevitan\">@gunesevitan</a> can't you multi-thread a webscrape? can't each thread request information independently?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 676362,
          "author_name": "Gunes Evitan",
          "author_url": "",
          "post_date": "2019-11-19T05:46:23.137000",
          "content": "<p>You can send multi-threaded requests, but there is no point of doing it. You still have to wait for server to send you response back. The time consuming operation is waiting for response.</p>\n\n<p>That's why web scraping frameworks like Scrapy are single-threaded, but they utilize concurrency with coroutines. Non-blocking IO is way better than multi-threading in web scraping.</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 676451,
          "author_name": "Stephen McInerney",
          "author_url": "",
          "post_date": "2019-11-19T07:38:01.073000",
          "content": "<p>Guys can you please move this valuable discussion on Best Practices on Web Scraping to a separate thread, out of this dataset thread? Other things that you would have to worry about with commercial sites are rate-limiting, rotating IP addresses, etc.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 677934,
      "author_name": "Sefik",
      "author_url": "",
      "post_date": "2019-11-20T19:44:07.200000",
      "content": "<p>It is <a href=\"https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115698#latest-674237\">revealed</a> that site 4 is UC Berkeley. The university provides it energy consumption data <a href=\"https://engagementdashboard.com/ucb/ucb/\">publicly</a>. The following kernel extracts the historical energy consumption data for several buildings in the college.</p>\n\n<p><a href=\"https://www.kaggle.com/serengil/ucb-data-leakage-site-4\">https://www.kaggle.com/serengil/ucb-data-leakage-site-4</a></p>",
      "votes": 11,
      "replies": []
    },
    {
      "id": 676560,
      "author_name": "Gunes Evitan",
      "author_url": "",
      "post_date": "2019-11-19T09:54:49.810000",
      "content": "<p>Site 2 is not posted here yet, so let's make it official\n<a href=\"http://cm.asu.edu/\">http://cm.asu.edu/</a></p>",
      "votes": 7,
      "replies": [
        {
          "id": 676576,
          "author_name": "Jie Lu",
          "author_url": "",
          "post_date": "2019-11-19T10:19:47.147000",
          "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F456596%2F364f85d7ae8a1392913a413c49803400%2FCFu8ouq.jpg?generation=1574158751587103&amp;alt=media\" alt=\"\"></p>",
          "votes": 4,
          "replies": []
        },
        {
          "id": 676746,
          "author_name": "eagle4",
          "author_url": "",
          "post_date": "2019-11-19T13:30:13.167000",
          "content": "<p>sorry somebody was faster than me...</p>",
          "votes": 2,
          "replies": []
        }
      ]
    },
    {
      "id": 693900,
      "author_name": "KublaKahn",
      "author_url": "",
      "post_date": "2019-12-12T22:51:22.190000",
      "content": "<p><strong>Leaks:</strong>\n0: <a href=\"https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels\">https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels</a>\n1: <a href=\"https://www.kaggle.com/mpware/ucl-data-leakage-episode-2\">https://www.kaggle.com/mpware/ucl-data-leakage-episode-2</a>\n2: <strong><a href=\"https://www.kaggle.com/pdnartreb/scrap-asu-data\">https://www.kaggle.com/pdnartreb/scrap-asu-data</a></strong>\n3: <a href=\"https://media.githubusercontent.com/media/buds-lab/island-of-misfit-buildings/master/data/raw/DGS_322_Buildings-15m-By_Building-DST-gap-filled-3-2-18-508pm.csv\">https://media.githubusercontent.com/media/buds-lab/island-of-misfit-buildings/master/data/raw/DGS_322_Buildings-15m-By_Building-DST-gap-filled-3-2-18-508pm.csv</a>\n4:<a href=\"https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings\">https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings</a></p>\n\n<p>15: <a href=\"https://www.kaggle.com/pp2file/ashrae-site15-cornell\">https://www.kaggle.com/pp2file/ashrae-site15-cornell</a></p>\n\n<p><strong>Trends:</strong>\n14: <a href=\"http://energytracker.fm.virginia.edu/index.html\">http://energytracker.fm.virginia.edu/index.html</a>   (can see overall trends using graphs)</p>\n\n<p>University Terms\nSite 0: University of Central Florida \n<a href=\"http://cf.smartcatalogiq.com/en/2015-2016/Catalog/About-College-of-Central-Florida/College-Calendar\">http://cf.smartcatalogiq.com/en/2015-2016/Catalog/About-College-of-Central-Florida/College-Calendar</a>\n<a href=\"http://cf.smartcatalogiq.com/en/2016-2017/Catalog/About-College-of-Central-Florida/College-Calendar\">http://cf.smartcatalogiq.com/en/2016-2017/Catalog/About-College-of-Central-Florida/College-Calendar</a>\n<a href=\"http://cf.smartcatalogiq.com/en/2017-2018/Catalog/About-College-of-Central-Florida/College-Calendar\">http://cf.smartcatalogiq.com/en/2017-2018/Catalog/About-College-of-Central-Florida/College-Calendar</a>\n<a href=\"http://cf.smartcatalogiq.com/en/2018-2019/Catalog/About-College-of-Central-Florida/College-Calendar\">http://cf.smartcatalogiq.com/en/2018-2019/Catalog/About-College-of-Central-Florida/College-Calendar</a></p>\n\n<p>site1: University College London\n<a href=\"https://www.ucl.ac.uk/estates/roombooking/calendar/1516.pdf\">https://www.ucl.ac.uk/estates/roombooking/calendar/1516.pdf</a>\n<a href=\"https://www.ucl.ac.uk/estates/roombooking/calendar/1617.pdf\">https://www.ucl.ac.uk/estates/roombooking/calendar/1617.pdf</a>\n<a href=\"https://www.ucl.ac.uk/estates/roombooking/calendar/1718.pdf\">https://www.ucl.ac.uk/estates/roombooking/calendar/1718.pdf</a>\n<a href=\"https://www.ucl.ac.uk/estates/roombooking/calendar/1819.pdf\">https://www.ucl.ac.uk/estates/roombooking/calendar/1819.pdf</a></p>\n\n<p>site2: Arizona State University \n<a href=\"https://catalog.arizona.edu/calendar/2015-2016-academic-calendar\">https://catalog.arizona.edu/calendar/2015-2016-academic-calendar</a>\n<a href=\"https://catalog.arizona.edu/calendar/2016-2017-academic-calendar\">https://catalog.arizona.edu/calendar/2016-2017-academic-calendar</a>\n<a href=\"https://catalog.arizona.edu/2017-2018-academic-calendar\">https://catalog.arizona.edu/2017-2018-academic-calendar</a>\n<a href=\"https://catalog.arizona.edu/calendar/2018-2019-academic-calendar\">https://catalog.arizona.edu/calendar/2018-2019-academic-calendar</a></p>\n\n<p>Site4: University of California Berkeley\n<a href=\"https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2015-16.pdf\">https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2015-16.pdf</a>\n<a href=\"https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2016-17_V7.pdf\">https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2016-17_V7.pdf</a>\n<a href=\"https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2017-18_V3.pdf\">https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2017-18_V3.pdf</a>\n<a href=\"https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2018-19_V4.pdf\">https://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2018-19_V4.pdf</a></p>\n\n<p>site5: University of Southampton \n<a href=\"https://www.southampton.ac.uk/assets/imported/transforms/content-block/UsefulDownloads_Download/A4EE46BFA84746158B25E5CD179BE5DE/term-dates-201020.pdf\">https://www.southampton.ac.uk/assets/imported/transforms/content-block/UsefulDownloads_Download/A4EE46BFA84746158B25E5CD179BE5DE/term-dates-201020.pdf</a></p>\n\n<p>Site 7 and 11: McGill\n<a href=\"https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2015-16\">https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2015-16</a>\n<a href=\"https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2016-17\">https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2016-17</a>\n<a href=\"https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2017-18\">https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2017-18</a>\n<a href=\"https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2018-19\">https://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2018-19</a></p>\n\n<p>Site 9: University of Texas\n<a href=\"https://registrar.utexas.edu/calendars/15-16\">https://registrar.utexas.edu/calendars/15-16</a>\n<a href=\"https://registrar.utexas.edu/calendars/16-17\">https://registrar.utexas.edu/calendars/16-17</a>\n<a href=\"https://registrar.utexas.edu/calendars/17-18\">https://registrar.utexas.edu/calendars/17-18</a>\n<a href=\"https://registrar.utexas.edu/calendars/18-19\">https://registrar.utexas.edu/calendars/18-19</a></p>\n\n<p>site10 : Weber State University\n<a href=\"https://www.weber.edu/Registration/Class_Schedule_Archive.html\">https://www.weber.edu/Registration/Class_Schedule_Archive.html</a></p>\n\n<p>Site 13: University of Minnesotta\n<a href=\"http://www.d.umn.edu/calendar/academic_cal_15-16.html\">http://www.d.umn.edu/calendar/academic_cal_15-16.html</a>\n<a href=\"http://www.d.umn.edu/calendar/academic_cal_16-17.html\">http://www.d.umn.edu/calendar/academic_cal_16-17.html</a>\n<a href=\"http://www.d.umn.edu/calendar/academic_cal_17-18.html\">http://www.d.umn.edu/calendar/academic_cal_17-18.html</a>\n<a href=\"http://www.d.umn.edu/calendar/academic_cal_18-19.html\">http://www.d.umn.edu/calendar/academic_cal_18-19.html</a></p>\n\n<p>site14: University of Virginia\n<a href=\"https://www2.virginia.edu/registrar/calenold.html\">https://www2.virginia.edu/registrar/calenold.html</a> (old)\n<a href=\"https://www2.virginia.edu/registrar/calendar.html\">https://www2.virginia.edu/registrar/calendar.html</a> (current)</p>\n\n<p>site15: Cornell\n<a href=\"http://courses.cornell.edu/mime/media/26/11593/2015-2016+Academic+Calendar.pdf\">http://courses.cornell.edu/mime/media/26/11593/2015-2016+Academic+Calendar.pdf</a>\n<a href=\"https://registrar.cornell.edu/academic-calendar/2016-2017\">https://registrar.cornell.edu/academic-calendar/2016-2017</a>\n<a href=\"https://registrar.cornell.edu/academic-calendar/2017-2018\">https://registrar.cornell.edu/academic-calendar/2017-2018</a>\n<a href=\"https://registrar.cornell.edu/academic-calendar/2018-2019\">https://registrar.cornell.edu/academic-calendar/2018-2019</a></p>",
      "votes": 7,
      "replies": []
    },
    {
      "id": 652300,
      "author_name": "PC Jimmmy",
      "author_url": "",
      "post_date": "2019-10-18T16:40:07.163000",
      "content": "<p>Pain in the neck to get data - pretty much only one city at a time for me.  But should be some useful info for the heat load part of energy usage.</p>\n\n<p><a href=\"https://www.nrel.gov/gis/data-tools.html\">https://www.nrel.gov/gis/data-tools.html</a></p>",
      "votes": 8,
      "replies": [
        {
          "id": 668274,
          "author_name": "Louka Ewington-Pitsos",
          "author_url": "",
          "post_date": "2019-11-08T07:58:41.830000",
          "content": "<p><a href=\"/pcjimmmy\">@pcjimmmy</a> cheers for sharing, I assume you're mainly talking about the \"solar data\" stuff here?  <a href=\"https://www.nrel.gov/gis/tools.html\">https://www.nrel.gov/gis/tools.html</a>, specifically, \"Federal Energy Management Program Screening Map\"?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 668577,
          "author_name": "PC Jimmmy",
          "author_url": "",
          "post_date": "2019-11-08T15:23:30.070000",
          "content": "<p>yes - solar stuff - tried just adding a few cities at different latitudes - no help - so probably need to know the specific cities for each site.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 668939,
          "author_name": "Louka Ewington-Pitsos",
          "author_url": "",
          "post_date": "2019-11-09T06:20:00.840000",
          "content": "<p>dang, thanks for dropping that nugget of info jimmy! I've done a decent amount of web scraping before so maybe I'll see if there's a way to grab it en-masse and have a go with more data. </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 668954,
          "author_name": "PC Jimmmy",
          "author_url": "",
          "post_date": "2019-11-09T06:52:34.647000",
          "content": "<p>Pretty confident on a number of the sites for what city is what  - but have the solar data low on my to do list.  My current features do a very good job of making predictions to match the training year but have made little progress in getting leader board improvements to match.  Pretty sure the solar data will help predict the training year, but not sure it will help for the test years.  </p>\n\n<p>When I do get to add solar data than orlando will be the first city I grab - it's two sites and  I have lots of confidence that the two sites are really orlando.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 692864,
          "author_name": "Peter G. Schild",
          "author_url": "",
          "post_date": "2019-12-11T19:54:55.880000",
          "content": "<h2>Hourly solar data time series for US &amp; Canada:</h2>\n\n<ul>\n<li><a href=\"https://maps.nrel.gov/nsrdb-viewer/\">https://maps.nrel.gov/nsrdb-viewer/</a>?</li>\n<li>Choose \"PSMv3\" (Physical Solar Model) for satellite-based solar irradiance data</li>\n<li>SELECT YEARS: 2016-2018</li>\n<li>SELECT ATTRIBUTES: select all attributes</li>\n<li>SELECT DOWNLOAD OPTIONS:\n<ul><li>Include leap day</li>\n<li>UTC (don't convert to local time), si its easier to synchronize with the</li>\n<li>yes to half hour intervals. Use only the half-hour values for solar data, and the whole-hour values for the terrestrial parameters (e.g. temperature) to fit missing data.</li>\n<li>Note that you have to check synchronization. I noticed that site0 data should be moved back approx 1.5 hours to synchronize temperature best.</li></ul></li>\n</ul>\n\n<h2>UK data:</h2>\n\n<ul>\n<li>I used terrestrial data. Register for a user account at UK CEDA (Centre for Environmental Data Analysis), then download specific sites via:</li>\n<li><a href=\"https://catalogue.ceda.ac.uk/?q=solar&amp;sort_by=relevance\">https://catalogue.ceda.ac.uk/?q=solar&amp;sort_by=relevance</a></li>\n<li>The sites measure generally only GHI hourly.</li>\n</ul>\n\n<p>Note: Both sources require appropriate attribution when used</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 678161,
      "author_name": "uortolan",
      "author_url": "",
      "post_date": "2019-11-21T03:29:45.657000",
      "content": "<p>I think we would be better finding all the leaks as soon as possible, so we can focus on modeling.\nAttached are the exact locations / stations that provided the weather information for the competition.\nIf one look for universities near by those airports there is a high likelihood of finding the source of buildings energy consumption. I do not think all of them are possible to be scraped.</p>\n\n<p>High probability that site 15 is Cornell and data can be scraped at  <a href=\"http://portal.emcs.cornell.edu\">http://portal.emcs.cornell.edu</a>.\nOthers are just initial guesses.</p>\n\n<p>site,station,city,state,country,lat,lon,altitude,timezone,tz\n0,ORLANDO INTERNATIONAL AIRPORT,ORLANDO,FL,US,+28.434,-81.325,+27.4,-5,US/Eastern\nUniversity of Central Florida</p>\n\n<p>1,HEATHROW,LONDON,,UK,+51.478,-0.461,+25.3,0,Europe/London</p>\n\n<p>2,PHOENIX SKY HARBOR INTL AIRPORT,PHOENIX,AZ,US,+33.428,-112.004,+337.4,-7,US/Mountain\nArizona State University</p>\n\n<p>3,RONALD REAGAN WASHINGTON NATL AP,WASHINGTON,VA,US,+38.847,-77.035,+3.1,-5,US/Eastern</p>\n\n<p>4,METRO OAKLAND INTL AIRPORT,OAKLAND,CA,US,+37.721,-122.221,+1.8,-8,US/Pacific\nUC Berkeley</p>\n\n<p>5,CARDIFF,CARDIFF,,UK,+51.397,-3.343,+67.1,0,Europe/London\n?? Cardiff University</p>\n\n<p>6,CHARLOTTESVILLE-ALBEMARLE ARPT,CHARLOTTESVILLE,VA,US,+38.137,-78.455,+195.1,-5,US/Eastern\n?? University of Virginia</p>\n\n<p>7,OTTAWA CDA RCS ONT,OTTAWA,ON,CA,+45.383,-75.717,+79.0,-5,Canada/Eastern\n? University of Ottawa</p>\n\n<p>8,ORLANDO INTERNATIONAL AIRPORT,ORLANDO,FL,US,+28.434,-81.325,+27.4,-5,US/Eastern\n??</p>\n\n<p>9,AUSTIN-CAMP MABRY,AUSTIN,TX,US,+30.321,-97.760,+204.2,-6,US/Central\n? University of Texas at Austin</p>\n\n<p>10,OGDEN-HINCKLEY AIRPORT,OGDEN,UT,US,+41.196,-112.011,+1362.5,-7,US/Mountain\n?? Weber state university</p>\n\n<p>11,OTTAWA CDA RCS ONT,OTTAWA,ON,CA,+45.383,-75.717,+79.0,-5,Canada/Eastern\n??</p>\n\n<p>12,DUBLIN,DUBLIN,,EI,+53.421,-6.270,+73.8,0,Europe/Dublin\n?? Dublin City University </p>\n\n<p>13,MINNEAPOLIS-ST PAUL INTERNATIONAL AP,MINNEAPOLIS,MN,US,+44.883,-93.229,+265.8,-6,US/Central\n?? University of Minnesota</p>\n\n<p>14,TRENTON MERCER AIRPORT,TRENTON,NJ,US,+40.277,-74.816,+57.9,-5,US/Eastern\n?? Princeton University</p>\n\n<p>15,REGIONAL AIRPORT,ITHACA,NY,US,+42.483,-76.467,+335.0,-5,US/Eastern\nCornell University</p>",
      "votes": 5,
      "replies": [
        {
          "id": 678207,
          "author_name": "Tim Yee",
          "author_url": "",
          "post_date": "2019-11-21T05:05:27.717000",
          "content": "<p>I agree. I can confirm Site 15 is Cornell University. It requires some form fuzzy matching (sqft) for 70 of the buildings. The other 54/124 are exact match. <a href=\"https://www.kaggle.com/teeyee314/cornell\">Link </a>to building metadata. That should save you guys some time. Don't forget to upvote my dataset ;)</p>\n\n<p>edit: to anyone looking to scrape/compile/match all the cornell data, you'll need to average the meter readings within each hour (sum and divide by 4) because they are in 15 minute increments, so </p>\n\n<blockquote>\n  <p>00:00:00 = (00:00:00 + 00:15:00 + 00:30:00 + 00:45:00) / 4. </p>\n</blockquote>\n\n<p>And you'll have to deal with the missing readings so good luck.</p>",
          "votes": 3,
          "replies": []
        },
        {
          "id": 682813,
          "author_name": "MPWARE",
          "author_url": "",
          "post_date": "2019-11-27T22:37:48.900000",
          "content": "<p>Someone tried Princeton energy data (site14 potentially)? It is available online at:\n<a href=\"https://tiger-energy.appspot.com/data-download\">https://tiger-energy.appspot.com/data-download</a>\nBut it does not seem to match.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 682806,
      "author_name": "Poe Dator",
      "author_url": "",
      "post_date": "2019-11-27T22:00:07.787000",
      "content": "<p>There is a potential data leak for site 9 (Austin, TX):\nMany of the buildings are located within main campus of UT Austin. This may be confirmed by cross-checking competition data vs their buildings list here: <a href=\"https://utdirect.utexas.edu/apps/campus/buildings/nlogon/facilities/UTM/\">https://utdirect.utexas.edu/apps/campus/buildings/nlogon/facilities/UTM/</a>\nThe University seems to have a comprehensive site with energy management information <a href=\"https://utilities.utexas.edu/efficiency/energy-management\">https://utilities.utexas.edu/efficiency/energy-management</a>. Since this site is only available to the campus residents on intranet/VPN, it is not possible to use/access for most of us. However, given the size of UT, it may not be ruled out that some competition participants from U. of Texas may get an advantage. Could someone verify this?</p>",
      "votes": 6,
      "replies": []
    },
    {
      "id": 692514,
      "author_name": "Hiroyuki Namba",
      "author_url": "",
      "post_date": "2019-12-11T11:07:44.967000",
      "content": "<p>I use the leaked data in the following kernels\n0:<a href=\"https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels\">https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels</a>\n1:<a href=\"https://www.kaggle.com/mpware/ucl-data-leakage-episode-2\">https://www.kaggle.com/mpware/ucl-data-leakage-episode-2</a>\n2:<a href=\"https://www.kaggle.com/pdnartreb/scrap-asu-data\">https://www.kaggle.com/pdnartreb/scrap-asu-data</a>\n4:<a href=\"https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings\">https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings</a>\n15:<a href=\"https://www.kaggle.com/pp2file/ashrae-site15-cornell\">https://www.kaggle.com/pp2file/ashrae-site15-cornell</a>\n(sorry for repeating)</p>\n\n<p>【JUST MY THOUGHT】\n- We can use the above data.\n (I think it is not necessary for each of us to declare the exploitation of these data in this thread. Because these data already meet the external data condition written in rules 7C)\n- There may be other leak, but at least for me there seems to be no other leaked data that meet the condition of external data written in rules. </p>",
      "votes": 4,
      "replies": []
    },
    {
      "id": 693334,
      "author_name": "Johnny Lee",
      "author_url": "",
      "post_date": "2019-12-12T09:12:42.230000",
      "content": "<p><a href=\"https://github.com/dr-prodigy/python-holidays\">https://github.com/dr-prodigy/python-holidays</a>\n<a href=\"https://www.officeholidays.com/countries/\">https://www.officeholidays.com/countries/</a>\n<a href=\"https://www.timeanddate.com/holidays/\">https://www.timeanddate.com/holidays/</a>\n<a href=\"https://www.gov.uk/bank-holidays\">https://www.gov.uk/bank-holidays</a>\n<a href=\"https://blog.holidaylettings.co.uk/\">https://blog.holidaylettings.co.uk/</a>\n<a href=\"https://www.edwardbryantschool.co.uk/docs/term_dates/term-dates-2018-2019.pdf\">https://www.edwardbryantschool.co.uk/docs/term_dates/term-dates-2018-2019.pdf</a>\n<a href=\"https://www.kaggle.com/c/ashrae-energy-prediction/notebooks\">https://www.kaggle.com/c/ashrae-energy-prediction/notebooks</a></p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 691358,
      "author_name": "suda",
      "author_url": "",
      "post_date": "2019-12-10T03:47:18.583000",
      "content": "<p><a href=\"https://www.worldweatheronline.com/\">https://www.worldweatheronline.com/</a></p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 665141,
      "author_name": "Poe Dator",
      "author_url": "",
      "post_date": "2019-11-04T17:11:20.113000",
      "content": "<p>There is this <a href=\"https://github.com/buds-lab/the-building-data-genome-project\">Building Data Genome Project Database </a> and mention of Southampton University as data source in the [related article of Miller, Meggers] (<a href=\"http://www.sciencedirect.com/science/article/pii/S1876610217330047\">http://www.sciencedirect.com/science/article/pii/S1876610217330047</a>)\nJudging by sq ft and year built numbers it has about 250-300 overlaps with this contest's buildings table. The possible matches are for site_ids 1,2,4,5,6,13,15. </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 655700,
      "author_name": "Isamu",
      "author_url": "",
      "post_date": "2019-10-23T11:40:31.253000",
      "content": "<p><a href=\"https://www.kaggle.com/selfishgene/historical-hourly-weather-data\">https://www.kaggle.com/selfishgene/historical-hourly-weather-data</a></p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 688543,
      "author_name": "Poe Dator",
      "author_url": "",
      "post_date": "2019-12-05T18:04:57.400000",
      "content": "<p>Quite many of the buildings in the dataset are located a university campuses, even many of those with types 'office', 'lodging', 'parking'. It may help to use the academic calendars to predict consumption patterns related to school breaks (these shift a bit from year to year).\nFor starters: \nsite 9: 2015-16 academic calendar for UT Austin <a href=\"https://registrar.utexas.edu/calendars/15-16\">https://registrar.utexas.edu/calendars/15-16</a>\nsite 15: 2015-16 academic calendar for Cornell <a href=\"http://courses.cornell.edu/mime/media/26/11593/2015-2016+Academic+Calendar.pdf\">http://courses.cornell.edu/mime/media/26/11593/2015-2016+Academic+Calendar.pdf</a>\nNotice for example, that spring breaks do not coincide between these two. \nI presume that accounting for academic events (breaks, exams) may improve model results ...</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 680509,
      "author_name": "Jie Lu",
      "author_url": "",
      "post_date": "2019-11-24T19:38:00.247000",
      "content": "<p>Site 15 revealed\n<a href=\"http://portal.emcs.cornell.edu/GannettHealthCenter\">http://portal.emcs.cornell.edu/GannettHealthCenter</a>\nrefering to <a href=\"https://www.kaggle.com/pp2file/ashrae-site15-cornell\">https://www.kaggle.com/pp2file/ashrae-site15-cornell</a></p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 679916,
      "author_name": "Gunes Evitan",
      "author_url": "",
      "post_date": "2019-11-23T16:45:46.217000",
      "content": "<p>Site 3 only building metadata, no meter readings\n<a href=\"https://opendata.dc.gov/datasets/building-energy-benchmarks/data\">https://opendata.dc.gov/datasets/building-energy-benchmarks/data</a></p>",
      "votes": 2,
      "replies": [
        {
          "id": 680601,
          "author_name": "Zhang Yunfei",
          "author_url": "",
          "post_date": "2019-11-25T00:08:49.977000",
          "content": "<p>have，data，just you don't find it.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 690951,
      "author_name": "Dmitrii Maslov",
      "author_url": "",
      "post_date": "2019-12-09T11:44:48.870000",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a>\nAll these leak data stations used in my submission</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 664019,
      "author_name": "marshi",
      "author_url": "",
      "post_date": "2019-11-03T03:39:35.503000",
      "content": "<p>historical weather data in Ireland and London.\n<a href=\"https://www.kaggle.com/conorrot/irish-weather-hourly-data\">https://www.kaggle.com/conorrot/irish-weather-hourly-data</a>\n<a href=\"https://www.kaggle.com/jeanmidev/smart-meters-in-london\">https://www.kaggle.com/jeanmidev/smart-meters-in-london</a></p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 654925,
      "author_name": "KALE",
      "author_url": "",
      "post_date": "2019-10-22T13:52:01.620000",
      "content": "<p><a href=\"https://github.com/buds-lab/the-building-data-genome-project\">https://github.com/buds-lab/the-building-data-genome-project</a>\n<a href=\"https://github.com/buds-lab/building-prediction-benchmarking\">https://github.com/buds-lab/building-prediction-benchmarking</a></p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 653619,
      "author_name": "PC Jimmmy",
      "author_url": "",
      "post_date": "2019-10-20T18:04:35.677000",
      "content": "<p><a href=\"https://www.eia.gov/\">https://www.eia.gov/</a></p>\n\n<p>Not sure why so many of the datasets only available in monthly chunks but good data here for energy consumption</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 670140,
      "author_name": "Ryunosuke Ishizaki",
      "author_url": "",
      "post_date": "2019-11-11T04:27:18.330000",
      "content": "<p>calendar of each year\n<a href=\"https://www.timeanddate.com/calendar/\">https://www.timeanddate.com/calendar/</a></p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 694129,
      "author_name": "Oleg Knaub",
      "author_url": "",
      "post_date": "2019-12-13T08:16:51.807000",
      "content": "<p><a href=\"/sohier\">@sohier</a> Do you already know which sites are not scored in private?</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 691666,
      "author_name": "lmssdd",
      "author_url": "",
      "post_date": "2019-12-10T11:38:51.980000",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a></p>",
      "votes": -1,
      "replies": []
    },
    {
      "id": 691425,
      "author_name": "LJH",
      "author_url": "",
      "post_date": "2019-12-10T05:58:42.970000",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a></p>",
      "votes": -1,
      "replies": []
    },
    {
      "id": 698694,
      "author_name": "Huizi Ren",
      "author_url": "",
      "post_date": "2019-12-19T15:31:19.987000",
      "content": "<p><a href=\"https://www.kaggle.com/mimoudata/ashrae-2-lightgbm-without-leak-data\">https://www.kaggle.com/mimoudata/ashrae-2-lightgbm-without-leak-data</a></p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 694671,
      "author_name": "SDanielZafar",
      "author_url": "",
      "post_date": "2019-12-14T00:40:07.673000",
      "content": "<p>Was anyone able to fill in year_built and floor_count using these leaks? </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 694015,
      "author_name": "Mr Nik",
      "author_url": "",
      "post_date": "2019-12-13T04:02:02.200000",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a>\nAll these leak data stations used in my submission</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 693902,
      "author_name": "Alan Clappis",
      "author_url": "",
      "post_date": "2019-12-12T22:55:23.413000",
      "content": "<p><a href=\"https://www.kaggle.com/yamsam/ashrae-leak-data-station\">https://www.kaggle.com/yamsam/ashrae-leak-data-station</a>\n<a href=\"https://www.kaggle.com/selfishgene/historical-hourly-weather-data\">https://www.kaggle.com/selfishgene/historical-hourly-weather-data</a></p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 693827,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-12T20:32:37.580000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 693665,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-12T16:50:19.327000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 693490,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-12T12:41:40.163000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 693447,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-12T11:32:51.597000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 693374,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-12T10:10:21.210000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 692810,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-11T18:50:34.437000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 692709,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-11T16:48:02.630000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 692277,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-11T05:11:02.493000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 691951,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-10T17:35:00.870000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 690817,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-09T07:52:20.763000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 690811,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-09T07:40:34.957000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 690800,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-09T07:08:09.030000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 690140,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-08T03:10:40.597000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 689809,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-07T13:17:22.957000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 689632,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-07T07:46:53.417000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 689588,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-07T05:43:25.253000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 689193,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-06T16:00:39.847000",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 689403,
          "author_name": "",
          "author_url": "",
          "post_date": "2019-12-06T22:18:13.617000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 688871,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-06T07:31:20.253000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 688032,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-05T06:45:01.667000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 686318,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-03T02:47:04.580000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 680603,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-11-25T00:13:50.583000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 678257,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-11-21T06:43:37.260000",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 678577,
          "author_name": "",
          "author_url": "",
          "post_date": "2019-11-21T15:13:05.633000",
          "content": "",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 674110,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-11-15T22:42:42",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 673332,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-11-14T21:05:17.833000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 671680,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-11-13T03:23:55.450000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 670757,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-11-11T19:50:51.123000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 668847,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-11-09T00:24:57.433000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 664159,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-11-03T08:54:30.943000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 662218,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-10-31T08:41:33.913000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 657441,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-10-25T06:32:10.173000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 692979,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-11T23:23:55.783000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 690200,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-08T05:41:56.740000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 690195,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-12-08T05:39:29.573000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 674684,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-11-16T22:20:40.497000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "649772": "Post links to your external data sources here before the deadline specified in the rules (December 12th).",
    "670787": "@sohier We are asking to clarify using web scrapped data as predicted values. That is clearly a data leak.",
    "665274": "for a number of buildings / sites it is possible to find information in the Internet using area and construction year as a key.  I made a more detailed post [here] (https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115698).\nBelow is a list of sources that mention buildings from a few areas: \n\n**0: Orlando, FL**\n    https://www.oeis.ucf.edu/buildings\n\n**2: Tempe, Arizona** \n    https://www.asu.edu/purchasing/bids/pdfs/231906_ex_6A.pdf\n\n**3: DC**\nhttps://dgs.dc.gov/sites/default/files/dc/sites/dgs/service_content/attachments/Attachment%20A-%20Municipal%20Facilities%5b1%5d.pdf\n\n**8: Orlando, FL**\nhttps://www.orlando.gov/files/sharedassets/public/initiatives/bewes/090418_cityoforlando_bldginfo.pdf\n\n**9: Austin, TX**\n    http://www.thecb.state.tx.us/reports/xls/7559.xlsx\n\n**13: TwinCities, MN**\nhttps://mn.b3benchmarking.com/Report?OrganizationID=1337&amp;ReturnTo=Higher+Education\n\n**14: Charlottesville, VA**\nhttps://www.fm.virginia.edu/docs/conditionReport/2018-19FacilitiesConditionReport.pdf (has year built data)\n",
    "673832": "Site 1 UCL building metadata and meter readings\nhttps://platform.carbonculture.net/communities/ucl/30/apps/assets/list/place/",
    "677934": "It is [revealed](https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115698#latest-674237) that site 4 is UC Berkeley. The university provides it energy consumption data [publicly](https://engagementdashboard.com/ucb/ucb/). The following kernel extracts the historical energy consumption data for several buildings in the college.\n\n[https://www.kaggle.com/serengil/ucb-data-leakage-site-4](https://www.kaggle.com/serengil/ucb-data-leakage-site-4)",
    "676560": "Site 2 is not posted here yet, so let's make it official\nhttp://cm.asu.edu/",
    "693900": "**Leaks:**\n0: https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels\n1: https://www.kaggle.com/mpware/ucl-data-leakage-episode-2\n2: **https://www.kaggle.com/pdnartreb/scrap-asu-data**\n3: https://media.githubusercontent.com/media/buds-lab/island-of-misfit-buildings/master/data/raw/DGS_322_Buildings-15m-By_Building-DST-gap-filled-3-2-18-508pm.csv\n4:https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings\n\n15: https://www.kaggle.com/pp2file/ashrae-site15-cornell\n\n**Trends:**\n14: http://energytracker.fm.virginia.edu/index.html   (can see overall trends using graphs)\n\nUniversity Terms\nSite 0: University of Central Florida \nhttp://cf.smartcatalogiq.com/en/2015-2016/Catalog/About-College-of-Central-Florida/College-Calendar\nhttp://cf.smartcatalogiq.com/en/2016-2017/Catalog/About-College-of-Central-Florida/College-Calendar\nhttp://cf.smartcatalogiq.com/en/2017-2018/Catalog/About-College-of-Central-Florida/College-Calendar\nhttp://cf.smartcatalogiq.com/en/2018-2019/Catalog/About-College-of-Central-Florida/College-Calendar\n\nsite1: University College London\nhttps://www.ucl.ac.uk/estates/roombooking/calendar/1516.pdf\nhttps://www.ucl.ac.uk/estates/roombooking/calendar/1617.pdf\nhttps://www.ucl.ac.uk/estates/roombooking/calendar/1718.pdf\nhttps://www.ucl.ac.uk/estates/roombooking/calendar/1819.pdf\n\nsite2: Arizona State University \nhttps://catalog.arizona.edu/calendar/2015-2016-academic-calendar\nhttps://catalog.arizona.edu/calendar/2016-2017-academic-calendar\nhttps://catalog.arizona.edu/2017-2018-academic-calendar\nhttps://catalog.arizona.edu/calendar/2018-2019-academic-calendar\n  \nSite4: University of California Berkeley\nhttps://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2015-16.pdf\nhttps://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2016-17_V7.pdf\nhttps://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2017-18_V3.pdf\nhttps://registrar.berkeley.edu/sites/default/files/pdf/UCB_AcademicCalendar_2018-19_V4.pdf\n \nsite5: University of Southampton \nhttps://www.southampton.ac.uk/assets/imported/transforms/content-block/UsefulDownloads_Download/A4EE46BFA84746158B25E5CD179BE5DE/term-dates-201020.pdf\n\nSite 7 and 11: McGill\nhttps://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2015-16\nhttps://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2016-17\nhttps://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2017-18\nhttps://www.mcgill.ca/importantdates/key-dates/past-dates/key-dates-2018-19\n \nSite 9: University of Texas\nhttps://registrar.utexas.edu/calendars/15-16\nhttps://registrar.utexas.edu/calendars/16-17\nhttps://registrar.utexas.edu/calendars/17-18\nhttps://registrar.utexas.edu/calendars/18-19\n\nsite10 : Weber State University\nhttps://www.weber.edu/Registration/Class_Schedule_Archive.html\n\nSite 13: University of Minnesotta\nhttp://www.d.umn.edu/calendar/academic_cal_15-16.html\nhttp://www.d.umn.edu/calendar/academic_cal_16-17.html\nhttp://www.d.umn.edu/calendar/academic_cal_17-18.html\nhttp://www.d.umn.edu/calendar/academic_cal_18-19.html\n\nsite14: University of Virginia\nhttps://www2.virginia.edu/registrar/calenold.html (old)\nhttps://www2.virginia.edu/registrar/calendar.html (current)\n\nsite15: Cornell\nhttp://courses.cornell.edu/mime/media/26/11593/2015-2016+Academic+Calendar.pdf\nhttps://registrar.cornell.edu/academic-calendar/2016-2017\nhttps://registrar.cornell.edu/academic-calendar/2017-2018\nhttps://registrar.cornell.edu/academic-calendar/2018-2019\n\n\n\n\n \n",
    "652300": "Pain in the neck to get data - pretty much only one city at a time for me.  But should be some useful info for the heat load part of energy usage.\n\nhttps://www.nrel.gov/gis/data-tools.html\n\n\n\n",
    "678161": "I think we would be better finding all the leaks as soon as possible, so we can focus on modeling.\nAttached are the exact locations / stations that provided the weather information for the competition.\nIf one look for universities near by those airports there is a high likelihood of finding the source of buildings energy consumption. I do not think all of them are possible to be scraped.\n\nHigh probability that site 15 is Cornell and data can be scraped at  http://portal.emcs.cornell.edu.\nOthers are just initial guesses.\n\nsite,station,city,state,country,lat,lon,altitude,timezone,tz\n0,ORLANDO INTERNATIONAL AIRPORT,ORLANDO,FL,US,+28.434,-81.325,+27.4,-5,US/Eastern\nUniversity of Central Florida\n\n1,HEATHROW,LONDON,,UK,+51.478,-0.461,+25.3,0,Europe/London\n\n2,PHOENIX SKY HARBOR INTL AIRPORT,PHOENIX,AZ,US,+33.428,-112.004,+337.4,-7,US/Mountain\nArizona State University\n\n3,RONALD REAGAN WASHINGTON NATL AP,WASHINGTON,VA,US,+38.847,-77.035,+3.1,-5,US/Eastern\n\n4,METRO OAKLAND INTL AIRPORT,OAKLAND,CA,US,+37.721,-122.221,+1.8,-8,US/Pacific\nUC Berkeley\n\n5,CARDIFF,CARDIFF,,UK,+51.397,-3.343,+67.1,0,Europe/London\n?? Cardiff University\n\n6,CHARLOTTESVILLE-ALBEMARLE ARPT,CHARLOTTESVILLE,VA,US,+38.137,-78.455,+195.1,-5,US/Eastern\n?? University of Virginia\n\n7,OTTAWA CDA RCS ONT,OTTAWA,ON,CA,+45.383,-75.717,+79.0,-5,Canada/Eastern\n? University of Ottawa\n\n8,ORLANDO INTERNATIONAL AIRPORT,ORLANDO,FL,US,+28.434,-81.325,+27.4,-5,US/Eastern\n??\n\n9,AUSTIN-CAMP MABRY,AUSTIN,TX,US,+30.321,-97.760,+204.2,-6,US/Central\n? University of Texas at Austin\n\n10,OGDEN-HINCKLEY AIRPORT,OGDEN,UT,US,+41.196,-112.011,+1362.5,-7,US/Mountain\n?? Weber state university\n\n11,OTTAWA CDA RCS ONT,OTTAWA,ON,CA,+45.383,-75.717,+79.0,-5,Canada/Eastern\n??\n\n12,DUBLIN,DUBLIN,,EI,+53.421,-6.270,+73.8,0,Europe/Dublin\n?? Dublin City University \n\n13,MINNEAPOLIS-ST PAUL INTERNATIONAL AP,MINNEAPOLIS,MN,US,+44.883,-93.229,+265.8,-6,US/Central\n?? University of Minnesota\n\n14,TRENTON MERCER AIRPORT,TRENTON,NJ,US,+40.277,-74.816,+57.9,-5,US/Eastern\n?? Princeton University\n\n15,REGIONAL AIRPORT,ITHACA,NY,US,+42.483,-76.467,+335.0,-5,US/Eastern\nCornell University",
    "682806": "There is a potential data leak for site 9 (Austin, TX):\nMany of the buildings are located within main campus of UT Austin. This may be confirmed by cross-checking competition data vs their buildings list here: https://utdirect.utexas.edu/apps/campus/buildings/nlogon/facilities/UTM/\nThe University seems to have a comprehensive site with energy management information https://utilities.utexas.edu/efficiency/energy-management. Since this site is only available to the campus residents on intranet/VPN, it is not possible to use/access for most of us. However, given the size of UT, it may not be ruled out that some competition participants from U. of Texas may get an advantage. Could someone verify this?",
    "692514": "I use the leaked data in the following kernels\n0:https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels\n1:https://www.kaggle.com/mpware/ucl-data-leakage-episode-2\n2:https://www.kaggle.com/pdnartreb/scrap-asu-data\n4:https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings\n15:https://www.kaggle.com/pp2file/ashrae-site15-cornell\n(sorry for repeating)\n\n【JUST MY THOUGHT】\n- We can use the above data.\n (I think it is not necessary for each of us to declare the exploitation of these data in this thread. Because these data already meet the external data condition written in rules 7C)\n- There may be other leak, but at least for me there seems to be no other leaked data that meet the condition of external data written in rules. ",
    "693334": "https://github.com/dr-prodigy/python-holidays\nhttps://www.officeholidays.com/countries/\nhttps://www.timeanddate.com/holidays/\nhttps://www.gov.uk/bank-holidays\nhttps://blog.holidaylettings.co.uk/\nhttps://www.edwardbryantschool.co.uk/docs/term_dates/term-dates-2018-2019.pdf\nhttps://www.kaggle.com/c/ashrae-energy-prediction/notebooks",
    "691358": "https://www.worldweatheronline.com/",
    "665141": "There is this [Building Data Genome Project Database ](https://github.com/buds-lab/the-building-data-genome-project) and mention of Southampton University as data source in the [related article of Miller, Meggers] (http://www.sciencedirect.com/science/article/pii/S1876610217330047)\nJudging by sq ft and year built numbers it has about 250-300 overlaps with this contest's buildings table. The possible matches are for site_ids 1,2,4,5,6,13,15. ",
    "655700": "https://www.kaggle.com/selfishgene/historical-hourly-weather-data",
    "688543": "Quite many of the buildings in the dataset are located a university campuses, even many of those with types 'office', 'lodging', 'parking'. It may help to use the academic calendars to predict consumption patterns related to school breaks (these shift a bit from year to year).\nFor starters: \nsite 9: 2015-16 academic calendar for UT Austin https://registrar.utexas.edu/calendars/15-16\nsite 15: 2015-16 academic calendar for Cornell http://courses.cornell.edu/mime/media/26/11593/2015-2016+Academic+Calendar.pdf\nNotice for example, that spring breaks do not coincide between these two. \nI presume that accounting for academic events (breaks, exams) may improve model results ...",
    "680509": "Site 15 revealed\nhttp://portal.emcs.cornell.edu/GannettHealthCenter\nrefering to https://www.kaggle.com/pp2file/ashrae-site15-cornell",
    "679916": "Site 3 only building metadata, no meter readings\nhttps://opendata.dc.gov/datasets/building-energy-benchmarks/data",
    "690951": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nAll these leak data stations used in my submission",
    "664019": "historical weather data in Ireland and London.\nhttps://www.kaggle.com/conorrot/irish-weather-hourly-data\nhttps://www.kaggle.com/jeanmidev/smart-meters-in-london",
    "654925": "https://github.com/buds-lab/the-building-data-genome-project\nhttps://github.com/buds-lab/building-prediction-benchmarking",
    "653619": "https://www.eia.gov/\n\nNot sure why so many of the datasets only available in monthly chunks but good data here for energy consumption\n\n",
    "670140": "calendar of each year\nhttps://www.timeanddate.com/calendar/",
    "694129": "@sohier Do you already know which sites are not scored in private?",
    "691666": "\nhttps://www.kaggle.com/yamsam/ashrae-leak-data-station",
    "691425": "https://www.kaggle.com/yamsam/ashrae-leak-data-station",
    "698694": "https://www.kaggle.com/mimoudata/ashrae-2-lightgbm-without-leak-data",
    "694671": "Was anyone able to fill in year_built and floor_count using these leaks? ",
    "694015": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nAll these leak data stations used in my submission",
    "693902": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nhttps://www.kaggle.com/selfishgene/historical-hourly-weather-data",
    "693827": "Airport data from OpenFlights.org available at: https://openflights.org/data.html (and directly from https://raw.githubusercontent.com/jpatokal/openflights/master/data/airports.dat )",
    "693665": "weather information - https://www.ncei.noaa.gov/data/global-hourly/archive/csv/",
    "693490": "We have used below kernels and data from external website : \n1. [ https://www.kaggle.com/wuliaokaola/ashrae-maybe-this-can-make-public-lb-some-useful](https://www.kaggle.com/wuliaokaola/ashrae-maybe-this-can-make-public-lb-some-useful)\n2. [https://www.kaggle.com/yamsam/new-ucf-starter-kernel](https://www.kaggle.com/yamsam/new-ucf-starter-kernel)\n3. [https://www.kaggle.com/mpware/ucl-data-leakage-episode-2](https://www.kaggle.com/mpware/ucl-data-leakage-episode-2)\n4. [https://www.kaggle.com/pdnartreb/asu-buildings-energy-consumption](https://www.kaggle.com/pdnartreb/asu-buildings-energy-consumption)\n5. [https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings](https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings)\n6. [https://www.kaggle.com/pp2file/ashrae-site15-cornell](https://www.kaggle.com/pp2file/ashrae-site15-cornell)\n7. [https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115040#latest-667889](https://www.kaggle.com/c/ashrae-energy-prediction/discussion/115040#latest-667889)\n8. [https://www.kaggle.com/patrick0302/locate-cities-according-weather-temperature](https://www.kaggle.com/patrick0302/locate-cities-according-weather-temperature)\n9. [https://power.larc.nasa.gov/data-access-viewer/](https://power.larc.nasa.gov/data-access-viewer/)",
    "693447": "I use the leaked data in the following kernels\n0:https://www.kaggle.com/gunesevitan/ashrae-ucf-spider-and-eda-full-test-labels\n1:https://www.kaggle.com/mpware/ucl-data-leakage-episode-2\n2:https://www.kaggle.com/pdnartreb/scrap-asu-data\n4:https://www.kaggle.com/serengil/ucb-data-leakage-site-4-81-buildings\n15:https://www.kaggle.com/pp2file/ashrae-site15-cornell\nhttps://www.kaggle.com/yamsam/ashrae-leak-data-station\n(sorry for repeating)",
    "693374": "https://pypi.org/project/meteocalc/\n\nFor weather features",
    "692810": "As far as i know, those platform can be used to get hourly energy data (and are publicly available data)\nhttp://cm.asu.edu/\nhttps://www.oeis.ucf.edu/\nhttp://portal.emcs.cornell.edu/\nhttps://platform.carbonculture.net/\nhttps://engagementdashboard.com/ucb/ucb/\nhttps://engagementdashboard.com/ucsb/ucsb\nhttps://engagementdashboard.com/sd36/sd36\nhttps://tiger-energy.appspot.com/data-download",
    "692709": "[https://www.kaggle.com/yamsam/ashrae-leak-data-station](https://www.kaggle.com/yamsam/ashrae-leak-data-station)\nAll these leak data stations used in my submission",
    "692277": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nAll these leak data stations used in my submission",
    "691951": "[holidays library](https://pypi.org/project/holidays/)",
    "690817": "https://opendata.dc.gov/datasets/building-energy-benchmarks/data\nhttp://portal.emcs.cornell.edu/GannettHealthCenter\nhttps://platform.carbonculture.net/communities/ucl/30/apps/assets/list/place/\nhttps://www.kaggle.com/selfishgene/historical-hourly-weather-data\nhttps://www.data.jma.go.jp/gmd/cpd/monitor/climatview/frame.php\nhttps://www.kaggle.com/selfishgene/historical-hourly-weather-data\nhttps://www.ncdc.noaa.gov/data-access/model-data/model-datasets/numerical-weather-prediction
https://www.orlando.gov/files/sharedassets/public/initiatives/bewes/090418_cityoforlando_bldginfo.pdf
http://www.thecb.state.tx.us/reports/xls/7559.xlsx
https://mn.b3benchmarking.com/Report?OrganizationID=1337&amp;ReturnTo=Higher+Education
https://www.fm.virginia.edu/docs/conditionReport/2018\n",
    "690811": "Hi Dane,\nYour post on the modified rules says: \"Using publicly available meter readings is within the rules, assuming the general restrictions are also followed such as posting a link to the data in the external data thread by the entry deadline (December 12th).\" So, every participant has to publish the list of external data sources?\n\nInstead, since you will be doing a review after the deadline of December 12, may I request that you publish a list of accepted External data sites? In that way, we do not have to bother about which external link cannot be used.\n\nAt this point (thanks to the enterprise of the relevant active Kagglers), I am using the external data sites mentioned in the following Kernels:\nASHRAE - UCF Spider and EDA (Full Test Labels) v3\nUCL: Data Leakage (Episode 2) v1\nASU train and scraped test data v7\nUCB: Data Leakage (Site 4) v14\nASHRAE-site15-cornell v3\n\nThanks to the initiative of  @gunesevitan, @mpware, @poedator, @pdnartreb, @serengil @IsamuYamashita and @pp2file",
    "690800": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nAll these leak data stations used in my submission\nAnd data that other kagglers announced",
    "690140": "https://www.officeholidays.com/ for holiday info.",
    "689809": "I used doc2vec with wikipedia dumps https://dumps.wikimedia.org/enwiki/\nIt is probably not relevant to what the thread is meant to be but just in case...",
    "689632": "used leaked data from sites 0,1,2,4,15",
    "689588": "Hi @sohier  As per my understanding site 0,1,2,4 &amp; 15 data has been leaked. Does it mean that these sites will be excluded from private leaderboard test data?",
    "689193": "Would formulas (involving constants only) be considered External Data and must be disclosed?",
    "688871": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nAll these leak data stations used in my submission\n\n",
    "688032": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nAll these leak data stations used in my submission",
    "686318": "https://www.kaggle.com/yamsam/ashrae-leak-data-station\nAll these leak data stations used in my submission",
    "680603": "so far. maybe have 6 site energy  data  has be leajed.",
    "678257": "Will publicly available values be excluded from the final validation or are there plans to jitter them such that competition is made more fair in the interest of intellectual honesty?  Doing so could encourage honest Kagglers to continue in earnest...",
    "674110": "Not sure how useful this is going to be, but I'm planning on experimenting with https://www.esrl.noaa.gov/gmd/grad/solcalc/ and [pysolar](https://pysolar.readthedocs.io/en/latest/).",
    "673332": "https://www.timeanddate.com/worldclock/\n",
    "671680": "https://en.wikipedia.org/wiki/List_of_cities_by_average_temperature\nhttps://www.wunderground.com/\nhttps://github.com/buds-lab/island-of-misfit-buildings",
    "670757": "Lots of weather data on NOAA website:\nhttps://www.ncdc.noaa.gov/data-access/model-data/model-datasets/numerical-weather-prediction\nhttps://www.ncdc.noaa.gov/data-access/radar-data/noaa-big-data-project\nhttps://www.ncdc.noaa.gov/data-access/land-based-station-data",
    "668847": "https://www.sports-reference.com/cfb/years\n\ncollege football schedule - 2016 to present\nhigh consumption of beer - assume the same for energy but TBD",
    "664159": "https://darksky.net/dev",
    "662218": "https://www.data.jma.go.jp/gmd/cpd/monitor/climatview/frame.php\nIt will help to find site id location",
    "657441": "https://www.ncei.noaa.gov/support/access-data-service-api-user-documentation\n\nhttps://www.ashrae.org/communities/regions",
    "692979": "",
    "690200": "",
    "690195": "",
    "674684": ""
  }
}