{
  "id": 325436,
  "title": "Clarity on year_build feature in train_features ",
  "url": "/competitions/energy-anomaly-detection/discussion/325436",
  "author_name": "",
  "post_date": "2022-05-16T13:02:27.616083600Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>year_build field is defined as \"year_build - 1900\" here - <a href=\"https://github.com/buds-lab/ashrae-great-energy-predictor-3-solution-analysis/blob/master/solutions/rank-1/scripts/02_preprocess_data.py\" target=\"_blank\">https://github.com/buds-lab/ashrae-great-energy-predictor-3-solution-analysis/blob/master/solutions/rank-1/scripts/02_preprocess_data.py</a></p>\n<p>What I fail to understand is why most signals have year_built = 255, does this imply so data anomaly? The latest year prior to 255 is 114, which implies a year of build as 2014; a year_built = 255 would imply year of build 2155.</p>\n<h3>code for reproducibility</h3>\n<p><code>r\ntrain_features %&gt;% group_by(year_built) %&gt;% summarise(rc = n()) %&gt;% arrange(desc(rc))</code><br>\nA tibble: 32 × 2<br>\nyear_built    rc<br>\n    <br>\n255    1161101<br>\n76    105408<br>\n66    70271<br>\n19    43920<br>\n60    43919<br>\n110    35058<br>\n114    26345<br>\n104    17568<br>\n105    17568<br>\n68    17567<br>\n67    17566<br>\n64    17564<br>\n0    8784<br>\n6    8784<br>\n9    8784 <br>\n…</p>\n<p>PS. New here, apologies for lack of understanding or improper interpretation in advance</p>",
  "messages": [
    {
      "id": "1791892",
      "postDate": "05/16/2022 13:02:27",
      "content": "<p>year_build field is defined as \"year_build - 1900\" here - <a href=\"https://github.com/buds-lab/ashrae-great-energy-predictor-3-solution-analysis/blob/master/solutions/rank-1/scripts/02_preprocess_data.py\" target=\"_blank\">https://github.com/buds-lab/ashrae-great-energy-predictor-3-solution-analysis/blob/master/solutions/rank-1/scripts/02_preprocess_data.py</a></p>\n<p>What I fail to understand is why most signals have year_built = 255, does this imply so data anomaly? The latest year prior to 255 is 114, which implies a year of build as 2014; a year_built = 255 would imply year of build 2155.</p>\n<h3>code for reproducibility</h3>\n<p><code>r\ntrain_features %&gt;% group_by(year_built) %&gt;% summarise(rc = n()) %&gt;% arrange(desc(rc))</code><br>\nA tibble: 32 × 2<br>\nyear_built    rc<br>\n    <br>\n255    1161101<br>\n76    105408<br>\n66    70271<br>\n19    43920<br>\n60    43919<br>\n110    35058<br>\n114    26345<br>\n104    17568<br>\n105    17568<br>\n68    17567<br>\n67    17566<br>\n64    17564<br>\n0    8784<br>\n6    8784<br>\n9    8784 <br>\n…</p>\n<p>PS. New here, apologies for lack of understanding or improper interpretation in advance</p>",
      "rawMarkdown": "year_build field is defined as \"year_build - 1900\" here - https://github.com/buds-lab/ashrae-great-energy-predictor-3-solution-analysis/blob/master/solutions/rank-1/scripts/02_preprocess_data.py\n\nWhat I fail to understand is why most signals have year_built = 255, does this imply so data anomaly? The latest year prior to 255 is 114, which implies a year of build as 2014; a year_built = 255 would imply year of build 2155.\n\n###code for reproducibility \n`r\ntrain_features %>% group_by(year_built) %>% summarise(rc = n()) %>% arrange(desc(rc))`\nA tibble: 32 × 2\nyear_built\trc\n<int>\t<int>\n255\t1161101\n76\t105408\n66\t70271\n19\t43920\n60\t43919\n110\t35058\n114\t26345\n104\t17568\n105\t17568\n68\t17567\n67\t17566\n64\t17564\n0\t8784\n6\t8784\n9\t8784 \n...\n\nPS. New here, apologies for lack of understanding or improper interpretation in advance",
      "votes": null
    },
    {
      "id": "1791995",
      "postDate": "05/16/2022 14:37:53",
      "content": "<p>Had similar observation for cloud_coverage as well, is 255 a proxy for NA in these fields?</p>",
      "rawMarkdown": "Had similar observation for cloud_coverage as well, is 255 a proxy for NA in these fields?",
      "votes": null
    },
    {
      "id": "1793165",
      "postDate": "05/17/2022 16:34:27",
      "content": "<p><a href=\"https://www.kaggle.com/rohannanaware\" target=\"_blank\">@rohannanaware</a>, Good catch! As per our quick check, 255 is a proxy for missing values! Anyway, we will double check this and fix it soon.</p>",
      "rawMarkdown": "rohannanaware, Good catch! As per our quick check, 255 is a proxy for missing values! Anyway, we will double check this and fix it soon.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1791995,
      "author_name": "rohannanaware",
      "author_url": "",
      "post_date": "05/16/2022 14:37:53",
      "content": "<p>Had similar observation for cloud_coverage as well, is 255 a proxy for NA in these fields?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1793165,
      "author_name": "samy101",
      "author_url": "",
      "post_date": "05/17/2022 16:34:27",
      "content": "<p><a href=\"https://www.kaggle.com/rohannanaware\" target=\"_blank\">@rohannanaware</a>, Good catch! As per our quick check, 255 is a proxy for missing values! Anyway, we will double check this and fix it soon.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1791892": "year_build field is defined as \"year_build - 1900\" here - https://github.com/buds-lab/ashrae-great-energy-predictor-3-solution-analysis/blob/master/solutions/rank-1/scripts/02_preprocess_data.py\n\nWhat I fail to understand is why most signals have year_built = 255, does this imply so data anomaly? The latest year prior to 255 is 114, which implies a year of build as 2014; a year_built = 255 would imply year of build 2155.\n\n###code for reproducibility \n`r\ntrain_features %>% group_by(year_built) %>% summarise(rc = n()) %>% arrange(desc(rc))`\nA tibble: 32 × 2\nyear_built\trc\n<int>\t<int>\n255\t1161101\n76\t105408\n66\t70271\n19\t43920\n60\t43919\n110\t35058\n114\t26345\n104\t17568\n105\t17568\n68\t17567\n67\t17566\n64\t17564\n0\t8784\n6\t8784\n9\t8784 \n...\n\nPS. New here, apologies for lack of understanding or improper interpretation in advance",
    "1791995": "Had similar observation for cloud_coverage as well, is 255 a proxy for NA in these fields?",
    "1793165": "rohannanaware, Good catch! As per our quick check, 255 is a proxy for missing values! Anyway, we will double check this and fix it soon."
  },
  "source": "meta"
}