{
  "id": 442928,
  "title": "How to deal with class imbalance of events?",
  "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/442928",
  "author_name": "",
  "post_date": "2023-09-24T20:11:52.353372600Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I am trying to determine the best way of dealing with the class imbalance of the events column: 'onset', 'wakeup', or NaN. The vast majority of the events are 'NaN'. I am considering SMOTE. Curious to hear peoples thoughts.<br>\nThanks. </p>",
  "messages": [
    {
      "id": "2454469",
      "postDate": "09/24/2023 20:11:52",
      "content": "<p>I am trying to determine the best way of dealing with the class imbalance of the events column: 'onset', 'wakeup', or NaN. The vast majority of the events are 'NaN'. I am considering SMOTE. Curious to hear peoples thoughts.<br>\nThanks. </p>",
      "rawMarkdown": "I am trying to determine the best way of dealing with the class imbalance of the events column: 'onset', 'wakeup', or NaN. The vast majority of the events are 'NaN'. I am considering SMOTE. Curious to hear peoples thoughts.\nThanks.",
      "votes": null
    },
    {
      "id": "2455173",
      "postDate": "09/25/2023 11:05:21",
      "content": "<p>Since the dataset is already very large using SMOTE is only going to further cause memory issues. Based on the public notebooks, many have preprocessed to detect whether the person is asleep or not and used that predictions and change in values to detect the onset and wakeup events.Hope this could be helpful</p>",
      "rawMarkdown": "Since the dataset is already very large using SMOTE is only going to further cause memory issues. Based on the public notebooks, many have preprocessed to detect whether the person is asleep or not and used that predictions and change in values to detect the onset and wakeup events.Hope this could be helpful",
      "votes": null
    },
    {
      "id": "2455727",
      "postDate": "09/25/2023 17:02:25",
      "content": "<p>The dataset is imbalanced, and real data will also be imbalanced. I've seen several ideas in notebooks: setting 2 classes, 0 and 1, where 0 represents, for example, the absence of an event, and 1 represents wake up or going to sleep, using models that handle imbalanced datasets, or anomaly detection.</p>",
      "rawMarkdown": "The dataset is imbalanced, and real data will also be imbalanced. I've seen several ideas in notebooks: setting 2 classes, 0 and 1, where 0 represents, for example, the absence of an event, and 1 represents wake up or going to sleep, using models that handle imbalanced datasets, or anomaly detection.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2455173,
      "author_name": "uadithyan",
      "author_url": "",
      "post_date": "09/25/2023 11:05:21",
      "content": "<p>Since the dataset is already very large using SMOTE is only going to further cause memory issues. Based on the public notebooks, many have preprocessed to detect whether the person is asleep or not and used that predictions and change in values to detect the onset and wakeup events.Hope this could be helpful</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2455727,
      "author_name": "michalinahulak",
      "author_url": "",
      "post_date": "09/25/2023 17:02:25",
      "content": "<p>The dataset is imbalanced, and real data will also be imbalanced. I've seen several ideas in notebooks: setting 2 classes, 0 and 1, where 0 represents, for example, the absence of an event, and 1 represents wake up or going to sleep, using models that handle imbalanced datasets, or anomaly detection.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2454469": "I am trying to determine the best way of dealing with the class imbalance of the events column: 'onset', 'wakeup', or NaN. The vast majority of the events are 'NaN'. I am considering SMOTE. Curious to hear peoples thoughts.\nThanks.",
    "2455173": "Since the dataset is already very large using SMOTE is only going to further cause memory issues. Based on the public notebooks, many have preprocessed to detect whether the person is asleep or not and used that predictions and change in values to detect the onset and wakeup events.Hope this could be helpful",
    "2455727": "The dataset is imbalanced, and real data will also be imbalanced. I've seen several ideas in notebooks: setting 2 classes, 0 and 1, where 0 represents, for example, the absence of an event, and 1 represents wake up or going to sleep, using models that handle imbalanced datasets, or anomaly detection."
  },
  "source": "meta"
}