{
  "id": 129182,
  "title": "15th Place Solution",
  "url": "/competitions/ashrae-energy-prediction/discussion/129182",
  "author_name": "",
  "post_date": "2020-02-06T03:06:49.495000",
  "votes": 10,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Main difficulties in this competition：\n1） Less submissions\n2） Large amount of data - not memory friendly\n3） Data leak dropped the results of the competition.</p>\n\n<p>Step 1：Self Examination\n1） dividing the training set\nAccording to the time series relationships between the test set and the training set in the competition --- train set: 2016, test set: 2017, 2018 --- I divided the training set twice: \ntrain：first 6 months， val: last 6 months\ntrain：first 8 months， val: last 4 months</p>\n\n<p>Step 2：Fast data import and data size reduction\n1) In this competition, participants often use ‘feather’ format to read data, getting noticeable fast importations.\n2) The effort to reduce memory exists throughout the code. The two main ideas are:\n        a) Allocate memory according to data size requirements. Do not use 16 bits if you can use 8 bits.\n        b) Try to ensure that only the data required for the current calculation is retained in the memory. Do not load unnecessary data into the memory. After the calculation is completed, immediately release the memory.</p>\n\n<p>Step 3：Using leak data\nThe leaked data is essentially equivalent to a part of the test set, so it can be used as a test set.\n1) Direct training: increasing the amount of training data\n2) Statistics, discovering uncontrollable situations such as power outages, abnormal weather (because it is time series data)\n3) Used as a test set to evaluate the current model\n4) Used to determine some hyperparameters for model fusion\n5) Submit as a result</p>\n\n<p>Technique Summary：\nWe used 19 features in this competition：</p>\n\n<p>Categorical features： 'weekend', 'hour', 'meter', \"site_id\"</p>\n\n<p>Numerical features： \"building_id\", \"primary_use\"，\"square_feet\", \"year_built\", \"air_temperature\", \"cloud_coverage\",\n              \"dew_temperature\", 'building_mean', 'site_mean', \"wind_direction\",\n              \"wind_speed\", \"precip_depth_1_hr\", 'building_square_mean', 'floor_count', 'site_hour'</p>\n\n<p>building_mean: Count the total number of each group after grouping according to building_id (not sum).</p>\n\n<p>Site_mean: Concating site_id, meter, primary_use and then counting the total count of each group after grouping.</p>\n\n<p>wind_direction: Convert wind direction to ‘cos’, otherwise it will have an adverse effect on the result, because 0 and 360 coexist. </p>\n\n<p>Building_square_mean: After grouping by building_id, we get the log of the energy consumption of each building (the number is too large). Then we divided that by square_feet to get a ranking of the index of similar buildings' energy consumption capabilities.</p>\n\n<p>Site_hour：concat site_id and hour. With such grouping, we get the ‘log’ for the sum of energy consumption (number too large). We get ranking of the index of regional temperature’s impact on energy consumption.</p>\n\n<p>From all given features, remove sea_level_pressure.Based on experimental results, make np.log1p transformations on some features, mainly square_feet.</p>\n\n<p>Model：lightGBM</p>\n\n<p>Prameter：</p>\n\n<p>params = {\n        'boosting_type': 'gbdt',\n        'objective': 'regression',\n        'metric': 'rmse',\n        'learning_rate': 0.1,\n        'num_leaves': 2**8,\n        'feature_fraction': 0.9,\n        'bagging_fraction': 0.9,\n        'subsample_freq': 5,\n        \"reg_lambda\": 2,\n        'reg_alpha': 1,\n        'seed': 0,\n        'early_stopping_rounds': 20\n    }</p>\n\n<p>Kfold：group - first / second half of year；explanation：The temperature distribution is similar and relatively balanced. Later, someone tested the stratified k-fold and got good results:\nThe key of using k-fold is to ensure the similarity of the data. The two are too similar and boring. If they are too different, it does not matter at all, so try to grasp</p>\n\n<p>Data：\nTraining set used all the leaked data\n1）Discard site15 for its low data accuracy\n2)  selectively discards some data from site4 based on calculated distance from 2016</p>\n\n<p>Discard anomalies：\n1） We delete all data of site0 under meter = 0 before 2016-05-20 (from a data analysis perspective, these are all noise)\n2） We delete all cases where the meter reads 0 because we don't think a building will have the meter moving.\n3） We delete the case where meter is huge in building_id = 1099. At the end, we still feel that there are a lot of noise points in the data, because removing many zero-value points will get very good results &lt;0.8, but overfitting is more serious because there are more zero-value points</p>\n\n<p>note：\nDue to the unit inconsistency problem at site0, we converted units. The fact is that its effect is extremely limited (difficult to observe), but it has a limited effect when using kfold, so we keep it.</p>",
  "messages": [
    {
      "id": 738001,
      "postDate": "2020-02-06T03:06:49.497Z",
      "content": "<p>Main difficulties in this competition：\n1） Less submissions\n2） Large amount of data - not memory friendly\n3） Data leak dropped the results of the competition.</p>\n\n<p>Step 1：Self Examination\n1） dividing the training set\nAccording to the time series relationships between the test set and the training set in the competition --- train set: 2016, test set: 2017, 2018 --- I divided the training set twice: \ntrain：first 6 months， val: last 6 months\ntrain：first 8 months， val: last 4 months</p>\n\n<p>Step 2：Fast data import and data size reduction\n1) In this competition, participants often use ‘feather’ format to read data, getting noticeable fast importations.\n2) The effort to reduce memory exists throughout the code. The two main ideas are:\n        a) Allocate memory according to data size requirements. Do not use 16 bits if you can use 8 bits.\n        b) Try to ensure that only the data required for the current calculation is retained in the memory. Do not load unnecessary data into the memory. After the calculation is completed, immediately release the memory.</p>\n\n<p>Step 3：Using leak data\nThe leaked data is essentially equivalent to a part of the test set, so it can be used as a test set.\n1) Direct training: increasing the amount of training data\n2) Statistics, discovering uncontrollable situations such as power outages, abnormal weather (because it is time series data)\n3) Used as a test set to evaluate the current model\n4) Used to determine some hyperparameters for model fusion\n5) Submit as a result</p>\n\n<p>Technique Summary：\nWe used 19 features in this competition：</p>\n\n<p>Categorical features： 'weekend', 'hour', 'meter', \"site_id\"</p>\n\n<p>Numerical features： \"building_id\", \"primary_use\"，\"square_feet\", \"year_built\", \"air_temperature\", \"cloud_coverage\",\n              \"dew_temperature\", 'building_mean', 'site_mean', \"wind_direction\",\n              \"wind_speed\", \"precip_depth_1_hr\", 'building_square_mean', 'floor_count', 'site_hour'</p>\n\n<p>building_mean: Count the total number of each group after grouping according to building_id (not sum).</p>\n\n<p>Site_mean: Concating site_id, meter, primary_use and then counting the total count of each group after grouping.</p>\n\n<p>wind_direction: Convert wind direction to ‘cos’, otherwise it will have an adverse effect on the result, because 0 and 360 coexist. </p>\n\n<p>Building_square_mean: After grouping by building_id, we get the log of the energy consumption of each building (the number is too large). Then we divided that by square_feet to get a ranking of the index of similar buildings' energy consumption capabilities.</p>\n\n<p>Site_hour：concat site_id and hour. With such grouping, we get the ‘log’ for the sum of energy consumption (number too large). We get ranking of the index of regional temperature’s impact on energy consumption.</p>\n\n<p>From all given features, remove sea_level_pressure.Based on experimental results, make np.log1p transformations on some features, mainly square_feet.</p>\n\n<p>Model：lightGBM</p>\n\n<p>Prameter：</p>\n\n<p>params = {\n        'boosting_type': 'gbdt',\n        'objective': 'regression',\n        'metric': 'rmse',\n        'learning_rate': 0.1,\n        'num_leaves': 2**8,\n        'feature_fraction': 0.9,\n        'bagging_fraction': 0.9,\n        'subsample_freq': 5,\n        \"reg_lambda\": 2,\n        'reg_alpha': 1,\n        'seed': 0,\n        'early_stopping_rounds': 20\n    }</p>\n\n<p>Kfold：group - first / second half of year；explanation：The temperature distribution is similar and relatively balanced. Later, someone tested the stratified k-fold and got good results:\nThe key of using k-fold is to ensure the similarity of the data. The two are too similar and boring. If they are too different, it does not matter at all, so try to grasp</p>\n\n<p>Data：\nTraining set used all the leaked data\n1）Discard site15 for its low data accuracy\n2)  selectively discards some data from site4 based on calculated distance from 2016</p>\n\n<p>Discard anomalies：\n1） We delete all data of site0 under meter = 0 before 2016-05-20 (from a data analysis perspective, these are all noise)\n2） We delete all cases where the meter reads 0 because we don't think a building will have the meter moving.\n3） We delete the case where meter is huge in building_id = 1099. At the end, we still feel that there are a lot of noise points in the data, because removing many zero-value points will get very good results &lt;0.8, but overfitting is more serious because there are more zero-value points</p>\n\n<p>note：\nDue to the unit inconsistency problem at site0, we converted units. The fact is that its effect is extremely limited (difficult to observe), but it has a limited effect when using kfold, so we keep it.</p>",
      "rawMarkdown": "Main difficulties in this competition：\n1） Less submissions\n2） Large amount of data - not memory friendly\n3） Data leak dropped the results of the competition.\n\nStep 1：Self Examination\n1） dividing the training set\nAccording to the time series relationships between the test set and the training set in the competition --- train set: 2016, test set: 2017, 2018 --- I divided the training set twice: \ntrain：first 6 months， val: last 6 months\ntrain：first 8 months， val: last 4 months\n\nStep 2：Fast data import and data size reduction\n1) In this competition, participants often use ‘feather’ format to read data, getting noticeable fast importations.\n2) The effort to reduce memory exists throughout the code. The two main ideas are:\n        a) Allocate memory according to data size requirements. Do not use 16 bits if you can use 8 bits.\n        b) Try to ensure that only the data required for the current calculation is retained in the memory. Do not load unnecessary data into the memory. After the calculation is completed, immediately release the memory.\n\nStep 3：Using leak data\nThe leaked data is essentially equivalent to a part of the test set, so it can be used as a test set.\n1) Direct training: increasing the amount of training data\n2) Statistics, discovering uncontrollable situations such as power outages, abnormal weather (because it is time series data)\n3) Used as a test set to evaluate the current model\n4) Used to determine some hyperparameters for model fusion\n5) Submit as a result\n\nTechnique Summary：\nWe used 19 features in this competition：\n\nCategorical features： 'weekend', 'hour', 'meter', \"site_id\"\n\nNumerical features： \"building_id\", \"primary_use\"，\"square_feet\", \"year_built\", \"air_temperature\", \"cloud_coverage\",\n              \"dew_temperature\", 'building_mean', 'site_mean', \"wind_direction\",\n              \"wind_speed\", \"precip_depth_1_hr\", 'building_square_mean', 'floor_count', 'site_hour'\n\nbuilding_mean: Count the total number of each group after grouping according to building_id (not sum).\n\nSite_mean: Concating site_id, meter, primary_use and then counting the total count of each group after grouping.\n\nwind_direction: Convert wind direction to ‘cos’, otherwise it will have an adverse effect on the result, because 0 and 360 coexist. \n\nBuilding_square_mean: After grouping by building_id, we get the log of the energy consumption of each building (the number is too large). Then we divided that by square_feet to get a ranking of the index of similar buildings' energy consumption capabilities.\n\nSite_hour：concat site_id and hour. With such grouping, we get the ‘log’ for the sum of energy consumption (number too large). We get ranking of the index of regional temperature’s impact on energy consumption.\n\nFrom all given features, remove sea_level_pressure.Based on experimental results, make np.log1p transformations on some features, mainly square_feet.\n\nModel：lightGBM\n\nPrameter：\n\nparams = {\n\t\t'boosting_type': 'gbdt',\n\t\t'objective': 'regression',\n\t\t'metric': 'rmse',\n\t\t'learning_rate': 0.1,\n\t\t'num_leaves': 2**8,\n\t\t'feature_fraction': 0.9,\n\t\t'bagging_fraction': 0.9,\n\t\t'subsample_freq': 5,\n\t\t\"reg_lambda\": 2,\n\t\t'reg_alpha': 1,\n\t\t'seed': 0,\n\t\t'early_stopping_rounds': 20\n\t}\n\n Kfold：group - first / second half of year；explanation：The temperature distribution is similar and relatively balanced. Later, someone tested the stratified k-fold and got good results:\nThe key of using k-fold is to ensure the similarity of the data. The two are too similar and boring. If they are too different, it does not matter at all, so try to grasp\n\nData：\nTraining set used all the leaked data\n1）Discard site15 for its low data accuracy\n2)  selectively discards some data from site4 based on calculated distance from 2016\n\nDiscard anomalies：\n1） We delete all data of site0 under meter = 0 before 2016-05-20 (from a data analysis perspective, these are all noise)\n2） We delete all cases where the meter reads 0 because we don't think a building will have the meter moving.\n3） We delete the case where meter is huge in building_id = 1099. At the end, we still feel that there are a lot of noise points in the data, because removing many zero-value points will get very good results &lt;0.8, but overfitting is more serious because there are more zero-value points\n\n\nnote：\nDue to the unit inconsistency problem at site0, we converted units. The fact is that its effect is extremely limited (difficult to observe), but it has a limited effect when using kfold, so we keep it.\n\n",
      "votes": 10
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "738001": "Main difficulties in this competition：\n1） Less submissions\n2） Large amount of data - not memory friendly\n3） Data leak dropped the results of the competition.\n\nStep 1：Self Examination\n1） dividing the training set\nAccording to the time series relationships between the test set and the training set in the competition --- train set: 2016, test set: 2017, 2018 --- I divided the training set twice: \ntrain：first 6 months， val: last 6 months\ntrain：first 8 months， val: last 4 months\n\nStep 2：Fast data import and data size reduction\n1) In this competition, participants often use ‘feather’ format to read data, getting noticeable fast importations.\n2) The effort to reduce memory exists throughout the code. The two main ideas are:\n        a) Allocate memory according to data size requirements. Do not use 16 bits if you can use 8 bits.\n        b) Try to ensure that only the data required for the current calculation is retained in the memory. Do not load unnecessary data into the memory. After the calculation is completed, immediately release the memory.\n\nStep 3：Using leak data\nThe leaked data is essentially equivalent to a part of the test set, so it can be used as a test set.\n1) Direct training: increasing the amount of training data\n2) Statistics, discovering uncontrollable situations such as power outages, abnormal weather (because it is time series data)\n3) Used as a test set to evaluate the current model\n4) Used to determine some hyperparameters for model fusion\n5) Submit as a result\n\nTechnique Summary：\nWe used 19 features in this competition：\n\nCategorical features： 'weekend', 'hour', 'meter', \"site_id\"\n\nNumerical features： \"building_id\", \"primary_use\"，\"square_feet\", \"year_built\", \"air_temperature\", \"cloud_coverage\",\n              \"dew_temperature\", 'building_mean', 'site_mean', \"wind_direction\",\n              \"wind_speed\", \"precip_depth_1_hr\", 'building_square_mean', 'floor_count', 'site_hour'\n\nbuilding_mean: Count the total number of each group after grouping according to building_id (not sum).\n\nSite_mean: Concating site_id, meter, primary_use and then counting the total count of each group after grouping.\n\nwind_direction: Convert wind direction to ‘cos’, otherwise it will have an adverse effect on the result, because 0 and 360 coexist. \n\nBuilding_square_mean: After grouping by building_id, we get the log of the energy consumption of each building (the number is too large). Then we divided that by square_feet to get a ranking of the index of similar buildings' energy consumption capabilities.\n\nSite_hour：concat site_id and hour. With such grouping, we get the ‘log’ for the sum of energy consumption (number too large). We get ranking of the index of regional temperature’s impact on energy consumption.\n\nFrom all given features, remove sea_level_pressure.Based on experimental results, make np.log1p transformations on some features, mainly square_feet.\n\nModel：lightGBM\n\nPrameter：\n\nparams = {\n\t\t'boosting_type': 'gbdt',\n\t\t'objective': 'regression',\n\t\t'metric': 'rmse',\n\t\t'learning_rate': 0.1,\n\t\t'num_leaves': 2**8,\n\t\t'feature_fraction': 0.9,\n\t\t'bagging_fraction': 0.9,\n\t\t'subsample_freq': 5,\n\t\t\"reg_lambda\": 2,\n\t\t'reg_alpha': 1,\n\t\t'seed': 0,\n\t\t'early_stopping_rounds': 20\n\t}\n\n Kfold：group - first / second half of year；explanation：The temperature distribution is similar and relatively balanced. Later, someone tested the stratified k-fold and got good results:\nThe key of using k-fold is to ensure the similarity of the data. The two are too similar and boring. If they are too different, it does not matter at all, so try to grasp\n\nData：\nTraining set used all the leaked data\n1）Discard site15 for its low data accuracy\n2)  selectively discards some data from site4 based on calculated distance from 2016\n\nDiscard anomalies：\n1） We delete all data of site0 under meter = 0 before 2016-05-20 (from a data analysis perspective, these are all noise)\n2） We delete all cases where the meter reads 0 because we don't think a building will have the meter moving.\n3） We delete the case where meter is huge in building_id = 1099. At the end, we still feel that there are a lot of noise points in the data, because removing many zero-value points will get very good results &lt;0.8, but overfitting is more serious because there are more zero-value points\n\n\nnote：\nDue to the unit inconsistency problem at site0, we converted units. The fact is that its effect is extremely limited (difficult to observe), but it has a limited effect when using kfold, so we keep it.\n\n"
  }
}