![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3474468%2F2dd66875429845e31bf59d96dd346e76%2FTS_0before_meter0.PNG?generation=1573636713340125&alt=media\")
It's been pointed out that the training dataset contains a large number of time series with meter_reading=0 for most of the times before 21th May 2016.
\n\nThe issue seems to occur primarily on buildings with meter type = 0 as in the next figure.
\n\n
I investigated the issue in more detail in this notebook
\n\nThe issue seems to be more general, as it turns out that there are also such time series with different meter type.
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3474468%2Ff5719a41d52a74c581fbb02b18a4d57b%2FTS_0before.PNG?generation=1573637247550021&alt=media\")
There are also series where the issue occurs in the second half of the year or throughout the year.\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3474468%2F34b208d978aec08d77e7fee2e30a813a%2FTS_0after.PNG?generation=1573637349913675&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3474468%2F3dc107a59fb245a586d13da1885c7204%2FTS_0before_after.PNG?generation=1573637376793426&alt=media\")
I removed from the training dataset the sections where the issue occurs and tried a basic LightGBM algorithm.
\n\nAlthough I only removed about 2% of the datapoints, my score went from 1.34 to 1.21!
\n\nAs always, data cleaning is the most effective approach.
\n\nFeel free to try my training data and let me know if it helps your score :)
", "messages": [ { "id": "671894", "postDate": "11/13/2019 09:37:37", "content": "It's been pointed out that the training dataset contains a large number of time series with meter_reading=0 for most of the times before 21th May 2016.
\n\nThe issue seems to occur primarily on buildings with meter type = 0 as in the next figure.
\n\n
I investigated the issue in more detail in this notebook
\n\nThe issue seems to be more general, as it turns out that there are also such time series with different meter type.
\n\n
There are also series where the issue occurs in the second half of the year or throughout the year.\n\n
I removed from the training dataset the sections where the issue occurs and tried a basic LightGBM algorithm.
\n\nAlthough I only removed about 2% of the datapoints, my score went from 1.34 to 1.21!
\n\nAs always, data cleaning is the most effective approach.
\n\nFeel free to try my training data and let me know if it helps your score :)
", "rawMarkdown": "It's been pointed out that the training dataset contains a large number of time series with meter_reading=0 for most of the times before 21th May 2016. \n\nThe issue seems to occur primarily on buildings with meter type = 0 as in the next figure.\n\n\n\nI investigated the issue in more detail in this [notebook](https://www.kaggle.com/robertobianco/time-series-study)\n\nThe issue seems to be more general, as it turns out that there are also such time series with different meter type.\n\n\n\nThere are also series where the issue occurs in the second half of the year or throughout the year.\n\n\n\nI removed from the training dataset the sections where the issue occurs and tried a basic LightGBM algorithm.\n\nAlthough I only removed about 2% of the datapoints, my score went from **1.34 to 1.21**!\n\nAs always, data cleaning is the most effective approach.\n\nFeel free to try my training data and let me know if it helps your score :)", "votes": null }, { "id": "672431", "postDate": "11/13/2019 21:40:32", "content": "Great works!!
", "rawMarkdown": "Great works!!", "votes": null }, { "id": "672668", "postDate": "11/14/2019 03:56:38", "content": "Good job !
", "rawMarkdown": "Good job !", "votes": null } ], "comments": [ { "id": 672431, "author_name": "hanjoonchoe", "author_url": "", "post_date": "11/13/2019 21:40:32", "content": "Great works!!
", "votes": null, "replies": [] }, { "id": 672668, "author_name": "uttam94", "author_url": "", "post_date": "11/14/2019 03:56:38", "content": "Good job !
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "671894": "It's been pointed out that the training dataset contains a large number of time series with meter_reading=0 for most of the times before 21th May 2016. \n\nThe issue seems to occur primarily on buildings with meter type = 0 as in the next figure.\n\n\n\nI investigated the issue in more detail in this [notebook](https://www.kaggle.com/robertobianco/time-series-study)\n\nThe issue seems to be more general, as it turns out that there are also such time series with different meter type.\n\n\n\nThere are also series where the issue occurs in the second half of the year or throughout the year.\n\n\n\nI removed from the training dataset the sections where the issue occurs and tried a basic LightGBM algorithm.\n\nAlthough I only removed about 2% of the datapoints, my score went from **1.34 to 1.21**!\n\nAs always, data cleaning is the most effective approach.\n\nFeel free to try my training data and let me know if it helps your score :)", "672431": "Great works!!", "672668": "Good job !" }, "source": "meta" }