{
  "id": 335735,
  "title": "Building IDs Unique between Train & Test - Implications",
  "url": "/competitions/energy-anomaly-detection/discussion/335735",
  "author_name": "Rahul",
  "post_date": "2022-07-07T15:57:48.053000",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hello,</p>\n<p>Since the building IDs are completely unique between the train and test sets, we necessarily:</p>\n<ol>\n<li>Cannot learn building specific features</li>\n<li>Must use the expanded feature set in the <code>train_features</code> dataset</li>\n</ol>\n<p>Furthermore, my hypothesis is keeping <code>building_id</code> as a feature in the training model would hamper performance on the training set &amp; the LB. I've seen a few notebooks where <code>building_id</code> is one of the highest ranked features in feature importance plots. Yet, completely useless as a predictor if the test set has unique building IDs.</p>\n<p>Thoughts?</p>",
  "messages": [
    {
      "id": 1847076,
      "postDate": "2022-07-07T15:57:48.053Z",
      "content": "<p>Hello,</p>\n<p>Since the building IDs are completely unique between the train and test sets, we necessarily:</p>\n<ol>\n<li>Cannot learn building specific features</li>\n<li>Must use the expanded feature set in the <code>train_features</code> dataset</li>\n</ol>\n<p>Furthermore, my hypothesis is keeping <code>building_id</code> as a feature in the training model would hamper performance on the training set &amp; the LB. I've seen a few notebooks where <code>building_id</code> is one of the highest ranked features in feature importance plots. Yet, completely useless as a predictor if the test set has unique building IDs.</p>\n<p>Thoughts?</p>",
      "rawMarkdown": "Hello,\n\nSince the building IDs are completely unique between the train and test sets, we necessarily:\n\n1. Cannot learn building specific features\n2. Must use the expanded feature set in the `train_features` dataset\n\nFurthermore, my hypothesis is keeping `building_id` as a feature in the training model would hamper performance on the training set & the LB. I've seen a few notebooks where `building_id` is one of the highest ranked features in feature importance plots. Yet, completely useless as a predictor if the test set has unique building IDs.\n\nThoughts?",
      "votes": 1
    },
    {
      "id": 1847690,
      "postDate": "2022-07-08T04:49:30.800Z",
      "content": "<p>Hi Rahul,</p>\n<p>I had the same thought at first, but actually the <code>building_id</code> implies the order of data collection, which means power meters  with closer <code>building_id</code> should be more similar.<br>\nPerhaps that explains why <code>building_id</code> remains a positive influence on the model (at least it works to my model).</p>\n<p>However, I cannot deny that building_id would cause the overfitting</p>",
      "rawMarkdown": "Hi Rahul,\n\nI had the same thought at first, but actually the `building_id` implies the order of data collection, which means power meters  with closer `building_id` should be more similar.\nPerhaps that explains why `building_id` remains a positive influence on the model (at least it works to my model).\n\nHowever, I cannot deny that building_id would cause the overfitting",
      "replies": [
        {
          "id": 1851886,
          "postDate": "2022-07-11T15:49:34.337Z",
          "content": "<p>Have you tried substituting <code>building_id</code> with a feature deriving the 'closeness' to other buildings?  <code>Site_id</code> should indicate closeness regardless, isn't it.</p>",
          "rawMarkdown": "Have you tried substituting `building_id` with a feature deriving the 'closeness' to other buildings?  `Site_id` should indicate closeness regardless, isn't it.",
          "votes": 1
        },
        {
          "id": 1852764,
          "postDate": "2022-07-12T10:41:52.243Z",
          "content": "<p>That's a good idea!<br>\nActually I've tried <code>site_id</code> and also clustered by K-means, but none of them works…</p>",
          "rawMarkdown": "That's a good idea!\nActually I've tried `site_id` and also clustered by K-means, but none of them works..."
        }
      ]
    },
    {
      "id": 1847691,
      "postDate": "2022-07-08T04:50:37.520Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 1847690,
      "author_name": "Chun Fu",
      "author_url": "",
      "post_date": "2022-07-08T04:49:30.800000",
      "content": "<p>Hi Rahul,</p>\n<p>I had the same thought at first, but actually the <code>building_id</code> implies the order of data collection, which means power meters  with closer <code>building_id</code> should be more similar.<br>\nPerhaps that explains why <code>building_id</code> remains a positive influence on the model (at least it works to my model).</p>\n<p>However, I cannot deny that building_id would cause the overfitting</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1851886,
          "author_name": "Rahul",
          "author_url": "",
          "post_date": "2022-07-11T15:49:34.337000",
          "content": "<p>Have you tried substituting <code>building_id</code> with a feature deriving the 'closeness' to other buildings?  <code>Site_id</code> should indicate closeness regardless, isn't it.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1852764,
          "author_name": "Chun Fu",
          "author_url": "",
          "post_date": "2022-07-12T10:41:52.243000",
          "content": "<p>That's a good idea!<br>\nActually I've tried <code>site_id</code> and also clustered by K-means, but none of them works…</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1847691,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-07-08T04:50:37.520000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1847076": "Hello,\n\nSince the building IDs are completely unique between the train and test sets, we necessarily:\n\n1. Cannot learn building specific features\n2. Must use the expanded feature set in the `train_features` dataset\n\nFurthermore, my hypothesis is keeping `building_id` as a feature in the training model would hamper performance on the training set & the LB. I've seen a few notebooks where `building_id` is one of the highest ranked features in feature importance plots. Yet, completely useless as a predictor if the test set has unique building IDs.\n\nThoughts?",
    "1847690": "Hi Rahul,\n\nI had the same thought at first, but actually the `building_id` implies the order of data collection, which means power meters  with closer `building_id` should be more similar.\nPerhaps that explains why `building_id` remains a positive influence on the model (at least it works to my model).\n\nHowever, I cannot deny that building_id would cause the overfitting",
    "1847691": ""
  }
}