{
  "id": 438110,
  "title": "Quick insight: 3 series_ids with potential time zone errors in train_events.csv",
  "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/438110",
  "author_name": "",
  "post_date": "2023-09-09T14:59:50.798570300Z",
  "votes": 4,
  "comment_count": 1,
  "views": 0,
  "content": "<p><strong>Series in question: [\"8a306e0890c0\", \"655f19eabf1e\", \"c6788e579967\"]</strong></p>\n<p>After doing a groupby on ['series_id', 'night'], I found 3 series_ids in train_events.csv which accounted for the \"onset\" step in both time zones:</p>\n<pre><code>series_id     night    event count\n8a306e0890c0         \n655f19eabf1e          \nc6788e579967         \nd2fef7e4defd          \nee4e0e3afd3d          \n                      ..\neef041dd50aa         \n8a22387617c3         \n062dbd4c95e6         \n280e08693c6d         \nc3072a759efb         \n</code></pre>\n<pre><code>index    series_id    night    event    step    timestamp\n  8a306e0890c0        onset    --12T01::-0400\n  8a306e0890c0        onset    --12T01::-0500\n  8a306e0890c0       wakeup    --12T09:03:-0500\n         series_id  night   event     step                 timestamp\n  655f19eabf1e         onset    --12T01:07:-0400\n  655f19eabf1e         onset    --12T01:07:-0500\n  655f19eabf1e        wakeup    --12T07::-0500\n          series_id  night   event      step                 timestamp\n  c6788e579967        onset    --11T01::-0400\n  c6788e579967        onset    --11T01::-0500\n  c6788e579967       wakeup    --11T09::-0500\n          series_id  night   event      step                 timestamp\n  c3072a759efb       wakeup    -03-18T09::-0400\n</code></pre>\n<p>In addition, series [\"c3072a759efb\"] on night 20 doesn't have an onset, only a wakeup step:</p>\n<pre><code>index    series_id    night    event    step    timestamp\n    c3072a759efb      wakeup      -03-16T07:08:-0500\n    c3072a759efb      onset       -03-16T20::-0500\n    c3072a759efb      wakeup      -03-17T08::-0500\n    c3072a759efb      wakeup      -03-18T09::-0400\n    c3072a759efb      onset       -03-18T22::-0400\n    c3072a759efb      wakeup      -03-19T07::-0400\n</code></pre>",
  "messages": [
    {
      "id": "2430753",
      "postDate": "09/09/2023 14:59:50",
      "content": "<p><strong>Series in question: [\"8a306e0890c0\", \"655f19eabf1e\", \"c6788e579967\"]</strong></p>\n<p>After doing a groupby on ['series_id', 'night'], I found 3 series_ids in train_events.csv which accounted for the \"onset\" step in both time zones:</p>\n<pre><code>series_id     night    event count\n8a306e0890c0         \n655f19eabf1e          \nc6788e579967         \nd2fef7e4defd          \nee4e0e3afd3d          \n                      ..\neef041dd50aa         \n8a22387617c3         \n062dbd4c95e6         \n280e08693c6d         \nc3072a759efb         \n</code></pre>\n<pre><code>index    series_id    night    event    step    timestamp\n  8a306e0890c0        onset    --12T01::-0400\n  8a306e0890c0        onset    --12T01::-0500\n  8a306e0890c0       wakeup    --12T09:03:-0500\n         series_id  night   event     step                 timestamp\n  655f19eabf1e         onset    --12T01:07:-0400\n  655f19eabf1e         onset    --12T01:07:-0500\n  655f19eabf1e        wakeup    --12T07::-0500\n          series_id  night   event      step                 timestamp\n  c6788e579967        onset    --11T01::-0400\n  c6788e579967        onset    --11T01::-0500\n  c6788e579967       wakeup    --11T09::-0500\n          series_id  night   event      step                 timestamp\n  c3072a759efb       wakeup    -03-18T09::-0400\n</code></pre>\n<p>In addition, series [\"c3072a759efb\"] on night 20 doesn't have an onset, only a wakeup step:</p>\n<pre><code>index    series_id    night    event    step    timestamp\n    c3072a759efb      wakeup      -03-16T07:08:-0500\n    c3072a759efb      onset       -03-16T20::-0500\n    c3072a759efb      wakeup      -03-17T08::-0500\n    c3072a759efb      wakeup      -03-18T09::-0400\n    c3072a759efb      onset       -03-18T22::-0400\n    c3072a759efb      wakeup      -03-19T07::-0400\n</code></pre>",
      "rawMarkdown": "**Series in question: [\"8a306e0890c0\", \"655f19eabf1e\", \"c6788e579967\"]**\n\nAfter doing a groupby on ['series_id', 'night'], I found 3 series_ids in train_events.csv which accounted for the \"onset\" step in both time zones:\n\n```python\nseries_id     night    event count\n8a306e0890c0  11       3\n655f19eabf1e  3        3\nc6788e579967  20       3\nd2fef7e4defd  3        2\nee4e0e3afd3d  1        2\n                      ..\neef041dd50aa  13       2\n8a22387617c3  34       2\n062dbd4c95e6  20       2\n280e08693c6d  15       2\nc3072a759efb  20       1\n```\n```python\nindex    series_id    night    event    step    timestamp\n7357  8a306e0890c0     11   onset  178464.0  2017-11-12T01:37:00-0400\n7358  8a306e0890c0     11   onset  179184.0  2017-11-12T01:37:00-0500\n7359  8a306e0890c0     11  wakeup  184536.0  2017-11-12T09:03:00-0500\n         series_id  night   event     step                 timestamp\n5362  655f19eabf1e      3   onset  43104.0  2017-11-12T01:07:00-0400\n5363  655f19eabf1e      3   onset  43824.0  2017-11-12T01:07:00-0500\n5364  655f19eabf1e      3  wakeup  48420.0  2017-11-12T07:30:00-0500\n          series_id  night   event      step                 timestamp\n10349  c6788e579967     20   onset  333552.0  2018-11-11T01:16:00-0400\n10350  c6788e579967     20   onset  334272.0  2018-11-11T01:16:00-0500\n10351  c6788e579967     20  wakeup  340404.0  2018-11-11T09:47:00-0500\n          series_id  night   event      step                 timestamp\n10132  c3072a759efb     20  wakeup  343068.0  2018-03-18T09:44:00-0400\n```\n\nIn addition, series [\"c3072a759efb\"] on night 20 doesn't have an onset, only a wakeup step:\n\n```python\nindex    series_id    night    event    step    timestamp\n10129\tc3072a759efb\t18\twakeup\t307356.0\t2018-03-16T07:08:00-0500\n10130\tc3072a759efb\t19\tonset\t317292.0\t2018-03-16T20:56:00-0500\n10131\tc3072a759efb\t19\twakeup\t325884.0\t2018-03-17T08:52:00-0500\n10132\tc3072a759efb\t20\twakeup\t343068.0\t2018-03-18T09:44:00-0400\n10133\tc3072a759efb\t21\tonset\t352356.0\t2018-03-18T22:38:00-0400\n10134\tc3072a759efb\t21\twakeup\t358596.0\t2018-03-19T07:18:00-0400\n```",
      "votes": null
    },
    {
      "id": "2430843",
      "postDate": "09/09/2023 16:16:17",
      "content": "<p>Found the same thing.. i guess some of the onset/wakeups dont \"tie out\" perfectly -- such is life</p>",
      "rawMarkdown": "Found the same thing.. i guess some of the onset/wakeups dont \"tie out\" perfectly -- such is life",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2430843,
      "author_name": "scharf",
      "author_url": "",
      "post_date": "09/09/2023 16:16:17",
      "content": "<p>Found the same thing.. i guess some of the onset/wakeups dont \"tie out\" perfectly -- such is life</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2430753": "**Series in question: [\"8a306e0890c0\", \"655f19eabf1e\", \"c6788e579967\"]**\n\nAfter doing a groupby on ['series_id', 'night'], I found 3 series_ids in train_events.csv which accounted for the \"onset\" step in both time zones:\n\n```python\nseries_id     night    event count\n8a306e0890c0  11       3\n655f19eabf1e  3        3\nc6788e579967  20       3\nd2fef7e4defd  3        2\nee4e0e3afd3d  1        2\n                      ..\neef041dd50aa  13       2\n8a22387617c3  34       2\n062dbd4c95e6  20       2\n280e08693c6d  15       2\nc3072a759efb  20       1\n```\n```python\nindex    series_id    night    event    step    timestamp\n7357  8a306e0890c0     11   onset  178464.0  2017-11-12T01:37:00-0400\n7358  8a306e0890c0     11   onset  179184.0  2017-11-12T01:37:00-0500\n7359  8a306e0890c0     11  wakeup  184536.0  2017-11-12T09:03:00-0500\n         series_id  night   event     step                 timestamp\n5362  655f19eabf1e      3   onset  43104.0  2017-11-12T01:07:00-0400\n5363  655f19eabf1e      3   onset  43824.0  2017-11-12T01:07:00-0500\n5364  655f19eabf1e      3  wakeup  48420.0  2017-11-12T07:30:00-0500\n          series_id  night   event      step                 timestamp\n10349  c6788e579967     20   onset  333552.0  2018-11-11T01:16:00-0400\n10350  c6788e579967     20   onset  334272.0  2018-11-11T01:16:00-0500\n10351  c6788e579967     20  wakeup  340404.0  2018-11-11T09:47:00-0500\n          series_id  night   event      step                 timestamp\n10132  c3072a759efb     20  wakeup  343068.0  2018-03-18T09:44:00-0400\n```\n\nIn addition, series [\"c3072a759efb\"] on night 20 doesn't have an onset, only a wakeup step:\n\n```python\nindex    series_id    night    event    step    timestamp\n10129\tc3072a759efb\t18\twakeup\t307356.0\t2018-03-16T07:08:00-0500\n10130\tc3072a759efb\t19\tonset\t317292.0\t2018-03-16T20:56:00-0500\n10131\tc3072a759efb\t19\twakeup\t325884.0\t2018-03-17T08:52:00-0500\n10132\tc3072a759efb\t20\twakeup\t343068.0\t2018-03-18T09:44:00-0400\n10133\tc3072a759efb\t21\tonset\t352356.0\t2018-03-18T22:38:00-0400\n10134\tc3072a759efb\t21\twakeup\t358596.0\t2018-03-19T07:18:00-0400\n```",
    "2430843": "Found the same thing.. i guess some of the onset/wakeups dont \"tie out\" perfectly -- such is life"
  },
  "source": "meta"
}