{
  "id": 325672,
  "title": "Is there any meaning behind not having a meter reading measurment?",
  "url": "/competitions/energy-anomaly-detection/discussion/325672",
  "author_name": "",
  "post_date": "2022-05-17T16:06:56.059138600Z",
  "votes": null,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Are there any insights behind having a missing <code>meter_reading</code> value in <code>train.csv</code>? I am asking mainly to think about how to handle these missing values.</p>\n<p>Perhaps:</p>\n<ol>\n<li>The measurement is missing for unknown reasons.</li>\n<li>The detector was turned off because the building is not in use.</li>\n</ol>\n<p>Although my initial analysis is based only in <code>train.csv</code>, I don't think this is answered in <code>train_features.csv</code>. Also, there doesn't seem to be a case when there is a missing <code>meter_reading</code> value but a registered anomaly, i.e.</p>\n<pre><code>train_df[train_df['meter_reading'].isna() &amp; (train_df['anomaly'] == 1)] \n</code></pre>\n<p>is empty.</p>\n<p>Thank you for any discussion!</p>",
  "messages": [
    {
      "id": "1793126",
      "postDate": "05/17/2022 16:06:56",
      "content": "<p>Are there any insights behind having a missing <code>meter_reading</code> value in <code>train.csv</code>? I am asking mainly to think about how to handle these missing values.</p>\n<p>Perhaps:</p>\n<ol>\n<li>The measurement is missing for unknown reasons.</li>\n<li>The detector was turned off because the building is not in use.</li>\n</ol>\n<p>Although my initial analysis is based only in <code>train.csv</code>, I don't think this is answered in <code>train_features.csv</code>. Also, there doesn't seem to be a case when there is a missing <code>meter_reading</code> value but a registered anomaly, i.e.</p>\n<pre><code>train_df[train_df['meter_reading'].isna() &amp; (train_df['anomaly'] == 1)] \n</code></pre>\n<p>is empty.</p>\n<p>Thank you for any discussion!</p>",
      "rawMarkdown": "Are there any insights behind having a missing `meter_reading` value in `train.csv`? I am asking mainly to think about how to handle these missing values.\n\nPerhaps:\n\n1. The measurement is missing for unknown reasons.\n2. The detector was turned off because the building is not in use.\n\nAlthough my initial analysis is based only in `train.csv`, I don't think this is answered in `train_features.csv`. Also, there doesn't seem to be a case when there is a missing `meter_reading` value but a registered anomaly, i.e.\n\n```python\ntrain_df[train_df['meter_reading'].isna() & (train_df['anomaly'] == 1)] \n```\n\nis empty.\n\nThank you for any discussion!",
      "votes": null
    },
    {
      "id": "1793172",
      "postDate": "05/17/2022 16:42:38",
      "content": "<p><a href=\"https://www.kaggle.com/santiagoontanon\" target=\"_blank\">@santiagoontanon</a>, Missing <code>meter_reading</code> values denote the instances when the meter was down or yielded noisy signals (that we had removed). Our objective here is to provide cleaner data so that participants can focus more on the modeling rather than doing the cleaning!</p>",
      "rawMarkdown": "santiagoontanon, Missing `meter_reading` values denote the instances when the meter was down or yielded noisy signals (that we had removed). Our objective here is to provide cleaner data so that participants can focus more on the modeling rather than doing the cleaning!",
      "votes": null
    },
    {
      "id": "1793189",
      "postDate": "05/17/2022 16:50:21",
      "content": "<p>Ok, thank you!</p>",
      "rawMarkdown": "Ok, thank you!",
      "votes": null
    },
    {
      "id": "1794123",
      "postDate": "05/18/2022 14:10:12",
      "content": "<p><a href=\"https://www.kaggle.com/samy101\" target=\"_blank\">@samy101</a> - I have a couple of questions as well</p>\n<ol>\n<li><p>For train_features.csv/train.csv - Do you have a what each column means (definition) , I know some are very obvious, like building_id, meter_reading etc.,</p></li>\n<li><p>With regards to your reply to Santiago's question, I agree you don't want us to waste time, but rather focus on the cleaned data. So I am assuming you are referring to train_features.csv that you have provided, am I right? But I notice there are ***NaN ***s for the meter readings in this file as well. Is it ok for us to sort of impute - meaning depending on a particular building_id - create a column called usage may be and set the NaN s to a mean value of their readings? And then proceed to do the modeling?</p></li>\n<li><p>What are the fields for e.g. like <strong><em>gte_meter_building_id_hour, gte_meter_building_id_weekday, gte_meter_building_id_month</em></strong>… etc. are they usage in a weekday or meter reading in that hour/weekday/month for the particular building? I am not clear on these fields, hence your response to my first question will be definitely helpful.</p></li>\n</ol>\n<p>I would appreciate your response to all my questions. TIA.</p>",
      "rawMarkdown": "samy101 - I have a couple of questions as well\n1. For train_features.csv/train.csv - Do you have a what each column means (definition) , I know some are very obvious, like building_id, meter_reading etc.,\n\n2. With regards to your reply to Santiago's question, I agree you don't want us to waste time, but rather focus on the cleaned data. So I am assuming you are referring to train_features.csv that you have provided, am I right? But I notice there are ***NaN ***s for the meter readings in this file as well. Is it ok for us to sort of impute - meaning depending on a particular building_id - create a column called usage may be and set the NaN s to a mean value of their readings? And then proceed to do the modeling?\n\n3. What are the fields for e.g. like ***gte_meter_building_id_hour, gte_meter_building_id_weekday, gte_meter_building_id_month***... etc. are they usage in a weekday or meter reading in that hour/weekday/month for the particular building? I am not clear on these fields, hence your response to my first question will be definitely helpful.\n\nI would appreciate your response to all my questions. TIA.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1793172,
      "author_name": "samy101",
      "author_url": "",
      "post_date": "05/17/2022 16:42:38",
      "content": "<p><a href=\"https://www.kaggle.com/santiagoontanon\" target=\"_blank\">@santiagoontanon</a>, Missing <code>meter_reading</code> values denote the instances when the meter was down or yielded noisy signals (that we had removed). Our objective here is to provide cleaner data so that participants can focus more on the modeling rather than doing the cleaning!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1793189,
          "author_name": "santiagoontanon",
          "author_url": "",
          "post_date": "05/17/2022 16:50:21",
          "content": "<p>Ok, thank you!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1794123,
      "author_name": "pankajashankar",
      "author_url": "",
      "post_date": "05/18/2022 14:10:12",
      "content": "<p><a href=\"https://www.kaggle.com/samy101\" target=\"_blank\">@samy101</a> - I have a couple of questions as well</p>\n<ol>\n<li><p>For train_features.csv/train.csv - Do you have a what each column means (definition) , I know some are very obvious, like building_id, meter_reading etc.,</p></li>\n<li><p>With regards to your reply to Santiago's question, I agree you don't want us to waste time, but rather focus on the cleaned data. So I am assuming you are referring to train_features.csv that you have provided, am I right? But I notice there are ***NaN ***s for the meter readings in this file as well. Is it ok for us to sort of impute - meaning depending on a particular building_id - create a column called usage may be and set the NaN s to a mean value of their readings? And then proceed to do the modeling?</p></li>\n<li><p>What are the fields for e.g. like <strong><em>gte_meter_building_id_hour, gte_meter_building_id_weekday, gte_meter_building_id_month</em></strong>… etc. are they usage in a weekday or meter reading in that hour/weekday/month for the particular building? I am not clear on these fields, hence your response to my first question will be definitely helpful.</p></li>\n</ol>\n<p>I would appreciate your response to all my questions. TIA.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1793126": "Are there any insights behind having a missing `meter_reading` value in `train.csv`? I am asking mainly to think about how to handle these missing values.\n\nPerhaps:\n\n1. The measurement is missing for unknown reasons.\n2. The detector was turned off because the building is not in use.\n\nAlthough my initial analysis is based only in `train.csv`, I don't think this is answered in `train_features.csv`. Also, there doesn't seem to be a case when there is a missing `meter_reading` value but a registered anomaly, i.e.\n\n```python\ntrain_df[train_df['meter_reading'].isna() & (train_df['anomaly'] == 1)] \n```\n\nis empty.\n\nThank you for any discussion!",
    "1793172": "santiagoontanon, Missing `meter_reading` values denote the instances when the meter was down or yielded noisy signals (that we had removed). Our objective here is to provide cleaner data so that participants can focus more on the modeling rather than doing the cleaning!",
    "1793189": "Ok, thank you!",
    "1794123": "samy101 - I have a couple of questions as well\n1. For train_features.csv/train.csv - Do you have a what each column means (definition) , I know some are very obvious, like building_id, meter_reading etc.,\n\n2. With regards to your reply to Santiago's question, I agree you don't want us to waste time, but rather focus on the cleaned data. So I am assuming you are referring to train_features.csv that you have provided, am I right? But I notice there are ***NaN ***s for the meter readings in this file as well. Is it ok for us to sort of impute - meaning depending on a particular building_id - create a column called usage may be and set the NaN s to a mean value of their readings? And then proceed to do the modeling?\n\n3. What are the fields for e.g. like ***gte_meter_building_id_hour, gte_meter_building_id_weekday, gte_meter_building_id_month***... etc. are they usage in a weekday or meter reading in that hour/weekday/month for the particular building? I am not clear on these fields, hence your response to my first question will be definitely helpful.\n\nI would appreciate your response to all my questions. TIA."
  },
  "source": "meta"
}