{
  "id": 452535,
  "title": "[Duplicated] train events which only has one event",
  "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/452535",
  "author_name": "",
  "post_date": "2023-11-02T13:50:10.637057900Z",
  "votes": 3,
  "comment_count": 5,
  "views": 0,
  "content": "<p>I participated recently in this competition. And I found the train event which only has one event.</p>\n<pre><code>series_ids = [,\n ,\n ,\n ,\n ]\n</code></pre>\n<p>Is this data as expected?</p>\n<p>It say the dataset was updated[1], is this effect of this update?<br>\nOr, did this records already exist in the original version?</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2F6377d0ecd72717716238791da02c189d%2FScreenshot%202023-11-02%20at%2022.37.36.png?generation=1698932424078778&amp;alt=media\" alt=\"\"></p>\n<p><a href=\"https://www.kaggle.com/ryanholbrook\" target=\"_blank\">@ryanholbrook</a> Could you confirm this?</p>\n<h2>Reference</h2>\n<ul>\n<li>[1] <a href=\"https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697\" target=\"_blank\">https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697</a></li>\n</ul>",
  "messages": [
    {
      "id": "2509654",
      "postDate": "11/02/2023 13:50:10",
      "content": "<p>I participated recently in this competition. And I found the train event which only has one event.</p>\n<pre><code>series_ids = [,\n ,\n ,\n ,\n ]\n</code></pre>\n<p>Is this data as expected?</p>\n<p>It say the dataset was updated[1], is this effect of this update?<br>\nOr, did this records already exist in the original version?</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2F6377d0ecd72717716238791da02c189d%2FScreenshot%202023-11-02%20at%2022.37.36.png?generation=1698932424078778&amp;alt=media\" alt=\"\"></p>\n<p><a href=\"https://www.kaggle.com/ryanholbrook\" target=\"_blank\">@ryanholbrook</a> Could you confirm this?</p>\n<h2>Reference</h2>\n<ul>\n<li>[1] <a href=\"https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697\" target=\"_blank\">https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697</a></li>\n</ul>",
      "rawMarkdown": "I participated recently in this competition. And I found the train event which only has one event.\n\n```python\nseries_ids = ['0ce74d6d2106',\n '154fe824ed87',\n '44a41bba1ee7',\n 'efbfc4526d58',\n 'f8a8da8bdd00']\n```\n\nIs this data as expected?\n\nIt say the dataset was updated[1], is this effect of this update?\nOr, did this records already exist in the original version?\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2F6377d0ecd72717716238791da02c189d%2FScreenshot%202023-11-02%20at%2022.37.36.png?generation=1698932424078778&alt=media)\n\n@ryanholbrook Could you confirm this?\n\n## Reference\n\n- [1] https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697",
      "votes": null
    },
    {
      "id": "2509918",
      "postDate": "11/02/2023 16:26:10",
      "content": "<p>There is always one onset, wakeup pair every night.<br>\nThere are nulls in step and timestamp column in train_events.csv.So if you count the number of events using step column it would give wrong answer. You probably did something like </p>\n<p>`train_events = pd.read_csv(\"/kaggle/input/child-mind-institute-detect-sleep-states/train_events.csv\")<br>\ncounts= train_events.groupby(['series_id', 'night'])['step'].count().reset_index()</p>\n<p>counts.columns = ['series_id', 'night', 'event_counts']</p>\n<p>display(counts[counts['event_counts'] == 1]) <br>\n`<br>\nYou should do this instead</p>\n<p>`train_events = pd.read_csv(\"/kaggle/input/child-mind-institute-detect-sleep-states/train_events.csv\")</p>\n<p>counts= train_events.groupby(['series_id', 'night'])['event'].count().reset_index()<br>\ncounts.columns = ['series_id', 'night', 'event_counts']</p>\n<p>display(counts[counts['event_counts'] == 1])<br>\n`<br>\nor use size method during aggregation</p>",
      "rawMarkdown": "There is always one onset, wakeup pair every night.\nThere are nulls in step and timestamp column in train_events.csv.So if you count the number of events using step column it would give wrong answer. You probably did something like \n\n`train_events = pd.read_csv(\"/kaggle/input/child-mind-institute-detect-sleep-states/train_events.csv\")\ncounts= train_events.groupby(['series_id', 'night'])['step'].count().reset_index()\n\ncounts.columns = ['series_id', 'night', 'event_counts']\n\ndisplay(counts[counts['event_counts'] == 1]) \n`\nYou should do this instead\n\n`train_events = pd.read_csv(\"/kaggle/input/child-mind-institute-detect-sleep-states/train_events.csv\")\n\ncounts= train_events.groupby(['series_id', 'night'])['event'].count().reset_index()\ncounts.columns = ['series_id', 'night', 'event_counts']\n\ndisplay(counts[counts['event_counts'] == 1])\n`\nor use size method during aggregation",
      "votes": null
    },
    {
      "id": "2510348",
      "postDate": "11/02/2023 22:32:32",
      "content": "<p>I dropped null columns.</p>\n<p>They say “at most one valid window should be detected”. I believe both onset and wake-up should not be null or neither of them.</p>\n<blockquote>\n  <p>If no valid sleep window is identifiable, neither an onset nor a wakeup event is recorded for that night.</p>\n</blockquote>",
      "rawMarkdown": "I dropped null columns.\n\nThey say “at most one valid window should be detected”. I believe both onset and wake-up should not be null or neither of them.\n\n> If no valid sleep window is identifiable, neither an onset nor a wakeup event is recorded for that night.",
      "votes": null
    },
    {
      "id": "2510350",
      "postDate": "11/02/2023 22:37:37",
      "content": "<p>Duplicated to this topic.<br>\n<a href=\"https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/442862\" target=\"_blank\">https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/442862</a></p>",
      "rawMarkdown": "Duplicated to this topic.\nhttps://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/442862",
      "votes": null
    },
    {
      "id": "2510499",
      "postDate": "11/03/2023 03:53:27",
      "content": "<p>My bad I understood the question wrongly<br>\nYou are right. They do look like points where events were not annotated for some reason.</p>",
      "rawMarkdown": "My bad I understood the question wrongly\nYou are right. They do look like points where events were not annotated for some reason.",
      "votes": null
    },
    {
      "id": "2510552",
      "postDate": "11/03/2023 04:55:17",
      "content": "<p>No problem. Thanks for commenting.</p>",
      "rawMarkdown": "No problem. Thanks for commenting.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2509918,
      "author_name": "shoabahamed",
      "author_url": "",
      "post_date": "11/02/2023 16:26:10",
      "content": "<p>There is always one onset, wakeup pair every night.<br>\nThere are nulls in step and timestamp column in train_events.csv.So if you count the number of events using step column it would give wrong answer. You probably did something like </p>\n<p>`train_events = pd.read_csv(\"/kaggle/input/child-mind-institute-detect-sleep-states/train_events.csv\")<br>\ncounts= train_events.groupby(['series_id', 'night'])['step'].count().reset_index()</p>\n<p>counts.columns = ['series_id', 'night', 'event_counts']</p>\n<p>display(counts[counts['event_counts'] == 1]) <br>\n`<br>\nYou should do this instead</p>\n<p>`train_events = pd.read_csv(\"/kaggle/input/child-mind-institute-detect-sleep-states/train_events.csv\")</p>\n<p>counts= train_events.groupby(['series_id', 'night'])['event'].count().reset_index()<br>\ncounts.columns = ['series_id', 'night', 'event_counts']</p>\n<p>display(counts[counts['event_counts'] == 1])<br>\n`<br>\nor use size method during aggregation</p>",
      "votes": null,
      "replies": [
        {
          "id": 2510348,
          "author_name": "tatamikenn",
          "author_url": "",
          "post_date": "11/02/2023 22:32:32",
          "content": "<p>I dropped null columns.</p>\n<p>They say “at most one valid window should be detected”. I believe both onset and wake-up should not be null or neither of them.</p>\n<blockquote>\n  <p>If no valid sleep window is identifiable, neither an onset nor a wakeup event is recorded for that night.</p>\n</blockquote>",
          "votes": null,
          "replies": [
            {
              "id": 2510499,
              "author_name": "shoabahamed",
              "author_url": "",
              "post_date": "11/03/2023 03:53:27",
              "content": "<p>My bad I understood the question wrongly<br>\nYou are right. They do look like points where events were not annotated for some reason.</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2510552,
                  "author_name": "tatamikenn",
                  "author_url": "",
                  "post_date": "11/03/2023 04:55:17",
                  "content": "<p>No problem. Thanks for commenting.</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "id": 2510350,
      "author_name": "tatamikenn",
      "author_url": "",
      "post_date": "11/02/2023 22:37:37",
      "content": "<p>Duplicated to this topic.<br>\n<a href=\"https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/442862\" target=\"_blank\">https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/442862</a></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2509654": "I participated recently in this competition. And I found the train event which only has one event.\n\n```python\nseries_ids = ['0ce74d6d2106',\n '154fe824ed87',\n '44a41bba1ee7',\n 'efbfc4526d58',\n 'f8a8da8bdd00']\n```\n\nIs this data as expected?\n\nIt say the dataset was updated[1], is this effect of this update?\nOr, did this records already exist in the original version?\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2F6377d0ecd72717716238791da02c189d%2FScreenshot%202023-11-02%20at%2022.37.36.png?generation=1698932424078778&alt=media)\n\n@ryanholbrook Could you confirm this?\n\n## Reference\n\n- [1] https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/440697",
    "2509918": "There is always one onset, wakeup pair every night.\nThere are nulls in step and timestamp column in train_events.csv.So if you count the number of events using step column it would give wrong answer. You probably did something like \n\n`train_events = pd.read_csv(\"/kaggle/input/child-mind-institute-detect-sleep-states/train_events.csv\")\ncounts= train_events.groupby(['series_id', 'night'])['step'].count().reset_index()\n\ncounts.columns = ['series_id', 'night', 'event_counts']\n\ndisplay(counts[counts['event_counts'] == 1]) \n`\nYou should do this instead\n\n`train_events = pd.read_csv(\"/kaggle/input/child-mind-institute-detect-sleep-states/train_events.csv\")\n\ncounts= train_events.groupby(['series_id', 'night'])['event'].count().reset_index()\ncounts.columns = ['series_id', 'night', 'event_counts']\n\ndisplay(counts[counts['event_counts'] == 1])\n`\nor use size method during aggregation",
    "2510348": "I dropped null columns.\n\nThey say “at most one valid window should be detected”. I believe both onset and wake-up should not be null or neither of them.\n\n> If no valid sleep window is identifiable, neither an onset nor a wakeup event is recorded for that night.",
    "2510350": "Duplicated to this topic.\nhttps://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/442862",
    "2510499": "My bad I understood the question wrongly\nYou are right. They do look like points where events were not annotated for some reason.",
    "2510552": "No problem. Thanks for commenting."
  },
  "source": "meta"
}