![\"GEFS](\"http://cs.ou.edu/~djgagne/gefs_mesonet_stations.png\")
You cannot sign up to Kaggle from multiple accounts and therefore you cannot submit from multiple accounts.
\r\n\r\n Privately sharing code or data outside of teams is not permitted.\r\n It's okay to share code if made available to all participants on the forums.\r\n
\r\nKaggle and the competition host have the right to publicly disseminate any entries or models.
\r\n\r\n Winning solutions need to be made available under\r\n a popular OSI-approved license\r\n in order to be eligible for recognition and prize money.\r\n
\r\n\r\n Prizes will be awarded after the winners have posted their solutions to the competition forum.\r\n Winners must post or link to their solutions within seven (7) days of being notified of their winning status.\r\n
\r\n\r\n Team mergers are allowed and can be performed by the team leader. In order to merge, the combined team must have a total submission count less than or equal to the maximum allowed as of the merge date. The maximum allowed is the number of submissions per day multiplied by the number of days the competition has been running.
\r\n
There is no maximum team size.
\r\nYou may submit a maximum of 2 entries per day.
\r\nYou may select up to 2 final submissions for judging.
\r\nStart Date: 7/8/2013 6:45 PM UTC
\r\nMerger Deadline: 11/8/2013 11:59 PM UTC
\r\nEntry Deadline: None
\r\nEnd Date: 11/15/2013 11:59 PM UTC
\r\n\t\r\nWelcome to the American Meteorological Society 2013-2014 Solar Energy Prediction Contest! This contest is organized by the American Meteorological Society Committees on Artificial Intelligence Applications to Environmental Science, Probability and Statistics, and Earth and Energy. Prizes are sponsored by EarthRisk Technologies, Inc.
\r\nRenewable energy sources, such as solar and wind, offer many environmental advantages over fossil fuels for electricity generation, but the energy produced by them fluctuates with changing weather conditions. Electric utility companies need accurate forecasts of energy production in order to have the right balance of renewable and fossil fuels available. Errors in the forecast could lead to large expenses for the utility from excess fuel consumption or emergency purchases of electricity from neighboring utilities. Power forecasts typically are derived from numerical weather prediction models, but statistical and machine learning techniques are increasingly being used in conjunction with the numerical models to produce more accurate forecasts.
\r\n
The goal of this contest is to discover which statistical and machine learning techniques provide the best short term predictions of solar energy production. Contestants will predict the total daily incoming solar energy at 98 Oklahoma Mesonet sites, which will serve as \"solar farms\" for the contest. Input numerical weather prediction data for the contest comes from the NOAA/ESRL Global Ensemble Forecast System (GEFS) Reforecast Version 2. Data include all 11 ensemble members and the forecast timesteps 12, 15, 18, 21, and 24. Locations of the Mesonet sites relative to the GEFS data are shown in the above figure. Training data will come from 1994-2007. Public testing data will be from 2008-2009. Private testing data for a more recent period will be used for the final evaluation.
\r\nDaily solar energy data were provided by the Oklahoma Mesonet with the assistance of Dr. Jeffrey Basara. The GEFS Reforecast Version 2 data were developed and provided by Dr. Thomas Hamill. The contest is being administered by David John Gagne and Dr. Amy McGovern of the University of Oklahoma.
\r\nEarthRisk Technologies creates a market advantage for its clients by uniquely quantifying weather data. Our company is a research pioneer that analyzes extreme weather risk at lead times longer than one week. Our techniques enhance competitive business decisions. TempRisk, the company’s first product suite, is a web-based platform that utilizes historical data, machine learning and predictive analytics to project risk for extreme winter cold and summer heat up to 40 days before it occurs. These patent-pending algorithms were developed in conjunction with Scripps Institution of Oceanography at the University of California San Diego. Energy producers and commodity investment firms currently employ TempRisk in their daily operations. Our customers require a uniquely objective quantitative methods for extreme event prediction. Our products are continuously developing thanks to ongoing support from customer-partners including large energy companies, investment firms, and reinsurance advisors.
EarthRisk's leadership team is excited to be deeply engaged with the American Meteorological Society. In addition to our engagement with the Committee on Artificial Intelligence Applications to Environmental Science, we're also active on the AMS Energy Committee, the Board on Private Sector Meteorology, the Financial Weather/Climate Risk Management Committee, and the Weather Enterprise Economic Evaluation Team. We have a true passion for advancing meteorological methods through the intelligent application of technology and are proud to be part of the Solar Energy Prediction Contest!
Contestants will submit predictions of the total solar daily incoming solar radiation at 98 Oklahoma Mesonet sites for each day specified in the test.csv file. The Mean Absolute Error (MAE) is the metric being used for this competition. It is commonly used in regression problems and by the renewable energy industry to compare forecast performance. The formula is given by:
\r\n\\[ \\text{MAE}=\\frac{1}{SE}\\sum_s^S\\sum_e^E{\\mid F_{se}-O_{se} \\mid} \\]
\r\nUnlike Root Mean Squared Error, MAE does not excessively punish extreme forecasts.
\r\nContestants should submit one csv file with the Date in the first column and each site's prediction in subsequent columns ordered alphabetically. The date should be in YYYYMMDD format. Solar energy predictions should be in units of Joules per square meter.
\r\nDate,ACME,ADAX,ALTU,APAC,...,WIST,WOOD,WYNO\r\n20080101,0,0,0,0,...,0,0,0\r\n20080102,0,0,0,0,...,0,0,0\r\n...\r\n20121130,0,0,0,0,...,0,0,0\r\n", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "data-description", "content": "
The contest training data are separated into 3 files.
\r\ngefs_train.tar.gz and gefs_train.zip contain all of the GEFS training data. The data are in netCDF4 files with each file holding the grids for each ensemble member at every time step for a particular variable. Each netCDF file contains the latitude-longitude grid and timestep values as well as metadata listing the full names of each variable and the associated units. More infromation about the netCDF format and links to open libraries for reading the files can be found here. NetCDF libraries are known to be available for C, Java, Python, R, and MATLAB.
\r\nEach netCDF4 file contains the total data for one of the model variables and are stored in a multidimensional array. The first dimension is the date of the model run and will correspond directly with a row in either the train.csv or sampleSubmission.csv files. The second dimension is the ensemble member that the forecast comes from. The GEFS has 11 ensemble members with perturbed initial conditions. The third dimension is the forecast hour, which runs from 12 to 24 hours in 3 hour increments. All model runs start at 00 UTC, so they will always correspond to the same universal time although local solar time will vary over each year. The fourth and fifth dimensions are the latitude and longitude uniform spatial grid. The longitudes in the file are in positive degrees from the Prime Meridian, so subtracting 360 from them will translate them to a similar range of values as in station_info.csv. A visualization of the grid can be seen on the main page.
\r\n| Variable | \r\nDescription | \r\nUnits | \r\n
| apcp_sfc | \r\n3-Hour accumulated precipitation at the surface | \r\nkg m-2 | \r\n
| dlwrf_sfc | \r\nDownward long-wave radiative flux average at the surface | \r\nW m-2 | \r\n
| dswrf_sfc | \r\nDownward short-wave radiative flux average at the surface | \r\nW m-2 | \r\n
| pres_msl | \r\nAir pressure at mean sea level | \r\nPa | \r\n
| pwat_eatm | \r\nPrecipitable Water over the entire depth of the atmosphere | \r\nkg m-2 | \r\n
| spfh_2m | \r\nSpecific Humidity at 2 m above ground | \r\nkg kg-1 | \r\n
| tcdc_eatm | \r\nTotal cloud cover over the entire depth of the atmosphere | \r\n% | \r\n
| tcolc_eatm | \r\nTotal column-integrated condensate over the entire atmos. | \r\nkg m-2 | \r\n
| tmax_2m | \r\nMaximum Temperature over the past 3 hours at 2 m above the ground | \r\nK | \r\n
| tmin_2m | \r\nMininmum Temperature over the past 3 hours at 2 m above the ground | \r\nK | \r\n
| tmp_2m | \r\nCurrent temperature at 2 m above the ground | \r\nK | \r\n
| tmp_sfc | \r\nTemperature of the surface | \r\nK | \r\n
| ulwrf_sfc | \r\nUpward long-wave radiation at the surface | \r\nW m-2 | \r\n
| ulwrf_tatm | \r\nUpward long-wave radiation at the top of the atmosphere | \r\nW m-2 | \r\n
| uswrf_sfc | \r\nUpward short-wave radiation at the surface | \r\nW m-2 | \r\n
train.csv contains the total daily incoming solar energy in (J m-2) at 98 Oklahoma Mesonet sites that have been in continuous operation since January 1, 1994. The solar energy was directly measured by a pyranometer at each Mesonet site every 5 minutes and summed from sunrise to 23:55 UTC of the date listed in each column.
\r\nstation_info.csv contains the latitude, longitudes, and elevation (meters) of each Mesonet station.
\r\ngefs_elevations.nc is a netCDF4 file that contains the model elevations of the GEFS grid points. Since the model terrain is smoothed compared to the real-world, the true elevation at a particular lat-lon point will likely not match the elevation in the model. The file contains two elevation variables. The elevation_control variable contains the elevations for the GEFS control run, which is the first ensemble member. The elevation_perturbation variable contains the elevations for the GEFS perturbations, which are the other ensemble members. There are up to 300 m differences in the elevations, so using one instead of the other could have an impact on your model. NOTE: If you do include elevation-based interpolation into any submission since September 10 and in your final submission, it must use the elevations from gefs_elevations.nc and not from any outside source.
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Timeline", "content": "All deadlines are at 11:59 PM UTC on the corresponding day unless otherwise noted. The organizers reserve the right to update the contest timeline if they deem it necessary as the contest progresses.
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Prizes", "content": "First Place: $500
\r\nSecond Place: $300
\r\nThird Place: $200
\r\nThe top 3 winners will have their abstract fees waived for the AMS Annual Meeting.
\r\nThe top student winner will have their conference registration and abstract fee waived for the AMS Annual Meeting.
", "isPublished": false, "hasIsPublished": false, "postTitle": "", "hasPostTitle": false, "mimeType": "", "hasMimeType": false }, { "name": "Winners", "content": "First Place team:
\r\nSecond Place:
\r\nThird Place:
\r\nTop student winner (invited to give a talk at AMS Annual Meeting):
\r\n