# [497th place shake down solution] PUBLIC LB 1st place solution

Since we can “feel free to post solutions and write-ups” now according to
[https://www.kaggle.com/c/ashrae-energy-prediction/discussion/122296](https://www.kaggle.com/c/ashrae-energy-prediction/discussion/122296)
I summarize my solution.
I appreciate ASHRAE and Kaggle for this really interesting competition. In spite of some terrible leakage, this competition is not destroyed thanks to sincere kagglers and rational rules clarification by organizers.
[Feature Engineering and Feature Selection]
-Final model used only 18 features.
-I tried a lot of features but many were not useful.
-Though dew temperature has high feature importance, I removed dew temperature because CV was better without it. I think air temperature is more important and dew temperature harms the contribution of air temperature.
-Categorical Multiplication had a slight positive impact. For example site_id_and_meter.
[Validation Strategy]
-Mainly based on [1-6] train [7-12] valid. [1-6] means January to June.
-Also watched [7-12] train [1-6] valid, and [1-9] train [10-12] valid.
-After leakage, I also validated by site0,1,2,15.
[Outlier Removal]
-The most important part.
-I plotted and watched all time-series carefully, and extracted so many rules that may detect outlier. I evaluated over 100 rules by the above Validation Strategy one by one.
-Examples of effective rules: “Meter 0 and long 0 segment”/”Meter 2 and start before April and long 0 segment”/”0 segments in many building at the same time”/”day average is too small in January comparing with December”.
[Single Model]
-Only LGBM.
-I tried CatBoost/time-series based methods but not worked.
-Tuned Parameters: num_leaves=127, feature_fraction=0.6.
[Ensemble]
-Totally 32 LGBM models.
-4 random seed averages for each model.
-[1-6] train model and [7-12] train model, similar to half-and-half kernel.
-Optimized weights for each model (Important).
--Basically, weight for [7-12] model was set higher than weight for [1-6], since [7-12] model was created from recent data.
--Furthermore, weight for [7-12] model was set higher for test data from July to December.
--Weight for [1-6] model was set higher for test data whose time-series has few train data (due to outliers) in [7-12].
[PostProcess]
-Plotted all time-series before submission and extracted some effective postprocess rules that may improve LB score.
--Modified to smaller value by applying piecewise linear function f. For example, 9.8 before log1p is modified to f(9.8)≒9.7. I searched optimal f and LB improved around 0.002. I think it is effective because there is some “unpredictable 0” but there is few “unpredictable large value“ (Important).
--Clipped to [0.1, 10.5] before log1p.
[Leak]
-Replaced leaked label just before submission (site0,1,2,15 and part of site4)
-Used for validation (site0,1,2,15)
-No other external data, no leaked data in training data, no other methods to exploit leakage.
Thank you for all comments! I will try to reply all questions within 24-48 business hours.