![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2207679%2Fa20687c316be3e920ee62ec105655417%2Findivisual_ele_missing_time.PNG?generation=1571735475020622&alt=media\")
I'm wondering if we treat 'Missing time values'.\n'Missing time values' means the meter couldn't measure the values on time, the missing values would be counted when the meter repaired. The figure shows some individual meter read values. After continue missing values, the peak would appear. The peak must be consumed electricity values at missing value time. Test data has long term and many building data, so the ‘missing time values’ may be included. Or it may be occurred Blackout. In these time series data forecast, how we treat these uncertain accidents?\nIf you know some way, could you show some way?
I'm wondering if we treat 'Missing time values'.\n'Missing time values' means the meter couldn't measure the values on time, the missing values would be counted when the meter repaired. The figure shows some individual meter read values. After continue missing values, the peak would appear. The peak must be consumed electricity values at missing value time. Test data has long term and many building data, so the ‘missing time values’ may be included. Or it may be occurred Blackout. In these time series data forecast, how we treat these uncertain accidents?\nIf you know some way, could you show some way?
Hi there could be a problem with the DAQ system where all values are dropping to zero. \"he peak must be consumed electricity values at missing value time\" if this statement is correct what about other peaks that i can see in the signal. we can cut the blank section and use ramp/half sine/ linear overlap/ sine overlap function to join time series data.
", "rawMarkdown": "Hi there could be a problem with the DAQ system where all values are dropping to zero. \"he peak must be consumed electricity values at missing value time\" if this statement is correct what about other peaks that i can see in the signal. we can cut the blank section and use ramp/half sine/ linear overlap/ sine overlap function to join time series data.", "votes": null }, { "id": "655254", "postDate": "10/22/2019 21:14:39", "content": "To streamline this though process it is useful to know the 3 categories in which missing data can be classified into:\nMissing Completely at Random (MCAR)\nMissing at Random (MAR)\nMissing Not at Random (MNAR)
\n\ndeeper into each one of them and look for potential treatments that can help us circumvent the problems that missing data creates.
\n\nSome ideas:\nTime series imputation\n - Non-time-series specific method\n- mean imputation\nmedian - imputation\nmode - imputation\n- calcucate the appropriate measure and replace NAs with the values. Appropriate for stationary time series, for example, white noise data\n- Random sample imputation\nreplace missing values with observations randomly selected from the remaining (either of it or just some section of it)
\n\nTime-Series specific method\n- Last observation carried forward (LOCF)\n- Next observation carried backward (NOCB)
\n\nThis article can be helpfull: https://www.linkedin.com/pulse/data-cleaning-guide-dealing-na-values-karan-rajwanshi/
", "rawMarkdown": "To streamline this though process it is useful to know the 3 categories in which missing data can be classified into:\nMissing Completely at Random (MCAR)\nMissing at Random (MAR)\nMissing Not at Random (MNAR)\n\ndeeper into each one of them and look for potential treatments that can help us circumvent the problems that missing data creates. \n\nSome ideas:\nTime series imputation\n - Non-time-series specific method\n- mean imputation\nmedian - imputation\nmode - imputation\n- calcucate the appropriate measure and replace NAs with the values. Appropriate for stationary time series, for example, white noise data\n- Random sample imputation\nreplace missing values with observations randomly selected from the remaining (either of it or just some section of it)\n\nTime-Series specific method\n- Last observation carried forward (LOCF)\n- Next observation carried backward (NOCB)\n\nThis article can be helpfull: https://www.linkedin.com/pulse/data-cleaning-guide-dealing-na-values-karan-rajwanshi/", "votes": null }, { "id": "656545", "postDate": "10/24/2019 11:21:15", "content": "Thank you for plite comment. \nI reconsidered and\nMeter_values are lost their time series property \nThis data has time series property, but predict targets are not time series prediction but just regression using weather data. If we interporate with some values, the values must have include weather information. For predicting next year meter values, it is better way to use last year data, but it is not true that next year meter values is the same with last year. So I think that drop missing values is better way.
", "rawMarkdown": "Thank you for plite comment. \nI reconsidered and\nMeter_values are lost their time series property \nThis data has time series property, but predict targets are not time series prediction but just regression using weather data. If we interporate with some values, the values must have include weather information. For predicting next year meter values, it is better way to use last year data, but it is not true that next year meter values is the same with last year. So I think that drop missing values is better way.", "votes": null } ], "comments": [ { "id": 654823, "author_name": "ashishjahagirdar", "author_url": "", "post_date": "10/22/2019 11:03:20", "content": "Hi there could be a problem with the DAQ system where all values are dropping to zero. \"he peak must be consumed electricity values at missing value time\" if this statement is correct what about other peaks that i can see in the signal. we can cut the blank section and use ramp/half sine/ linear overlap/ sine overlap function to join time series data.
", "votes": null, "replies": [] }, { "id": 655254, "author_name": "caesarlupum", "author_url": "", "post_date": "10/22/2019 21:14:39", "content": "To streamline this though process it is useful to know the 3 categories in which missing data can be classified into:\nMissing Completely at Random (MCAR)\nMissing at Random (MAR)\nMissing Not at Random (MNAR)
\n\ndeeper into each one of them and look for potential treatments that can help us circumvent the problems that missing data creates.
\n\nSome ideas:\nTime series imputation\n - Non-time-series specific method\n- mean imputation\nmedian - imputation\nmode - imputation\n- calcucate the appropriate measure and replace NAs with the values. Appropriate for stationary time series, for example, white noise data\n- Random sample imputation\nreplace missing values with observations randomly selected from the remaining (either of it or just some section of it)
\n\nTime-Series specific method\n- Last observation carried forward (LOCF)\n- Next observation carried backward (NOCB)
\n\nThis article can be helpfull: https://www.linkedin.com/pulse/data-cleaning-guide-dealing-na-values-karan-rajwanshi/
", "votes": null, "replies": [] }, { "id": 656545, "author_name": "takumiban", "author_url": "", "post_date": "10/24/2019 11:21:15", "content": "Thank you for plite comment. \nI reconsidered and\nMeter_values are lost their time series property \nThis data has time series property, but predict targets are not time series prediction but just regression using weather data. If we interporate with some values, the values must have include weather information. For predicting next year meter values, it is better way to use last year data, but it is not true that next year meter values is the same with last year. So I think that drop missing values is better way.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "654757": "I'm wondering if we treat 'Missing time values'.\n'Missing time values' means the meter couldn't measure the values on time, the missing values would be counted when the meter repaired. The figure shows some individual meter read values. After continue missing values, the peak would appear. The peak must be consumed electricity values at missing value time. Test data has long term and many building data, so the ‘missing time values’ may be included. Or it may be occurred Blackout. In these time series data forecast, how we treat these uncertain accidents?\nIf you know some way, could you show some way?", "654823": "Hi there could be a problem with the DAQ system where all values are dropping to zero. \"he peak must be consumed electricity values at missing value time\" if this statement is correct what about other peaks that i can see in the signal. we can cut the blank section and use ramp/half sine/ linear overlap/ sine overlap function to join time series data.", "655254": "To streamline this though process it is useful to know the 3 categories in which missing data can be classified into:\nMissing Completely at Random (MCAR)\nMissing at Random (MAR)\nMissing Not at Random (MNAR)\n\ndeeper into each one of them and look for potential treatments that can help us circumvent the problems that missing data creates. \n\nSome ideas:\nTime series imputation\n - Non-time-series specific method\n- mean imputation\nmedian - imputation\nmode - imputation\n- calcucate the appropriate measure and replace NAs with the values. Appropriate for stationary time series, for example, white noise data\n- Random sample imputation\nreplace missing values with observations randomly selected from the remaining (either of it or just some section of it)\n\nTime-Series specific method\n- Last observation carried forward (LOCF)\n- Next observation carried backward (NOCB)\n\nThis article can be helpfull: https://www.linkedin.com/pulse/data-cleaning-guide-dealing-na-values-karan-rajwanshi/", "656545": "Thank you for plite comment. \nI reconsidered and\nMeter_values are lost their time series property \nThis data has time series property, but predict targets are not time series prediction but just regression using weather data. If we interporate with some values, the values must have include weather information. For predicting next year meter values, it is better way to use last year data, but it is not true that next year meter values is the same with last year. So I think that drop missing values is better way." }, "source": "meta" }