{
  "id": 437232,
  "title": "About error tolerance thresholds: 3 seconds/step or 5 seconds/step?",
  "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/437232",
  "author_name": "",
  "post_date": "2023-09-06T00:53:22.746218400Z",
  "votes": 12,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Dear Host,</p>\n<p>Thank you for hosting the competition! Upon reading the Evaluation page, I noticed the mention of error tolerance thresholds being set at:</p>\n<p><code>For both event classes, we use error tolerance thresholds of 1, 3, 5, 7.5, 10, 12.5, 15, 20, 25, 30 in minutes, or 20, 60, 100, 150, 200, 250, 300, 400, 500, 600 in steps.</code></p>\n<p>From this, we can infer that the conversion between steps and seconds is:<br>\n60 seconds/20 steps = 3 seconds/1 step.</p>\n<p>However, after inspecting the train_events data, I observed that the conversion appears to be 5 seconds/1 step.<br>\nFor instance, consider the series with series_id = 038441c925bb on night = 1:<br>\nThe difference between the timestamps is:<br>\n2018-08-15T06:41:00-0400 - 2018-08-14T22:26:00-0400 = 29700 seconds<br>\nThe difference in steps is:<br>\n10932 - 4992 = 5940 steps</p>\n<p>When computing 29700/5940, it precisely equals 5 seconds per step.</p>\n<p>Is my understanding of the error tolerance thresholds incorrect?</p>",
  "messages": [
    {
      "id": "2425473",
      "postDate": "09/06/2023 00:53:22",
      "content": "<p>Dear Host,</p>\n<p>Thank you for hosting the competition! Upon reading the Evaluation page, I noticed the mention of error tolerance thresholds being set at:</p>\n<p><code>For both event classes, we use error tolerance thresholds of 1, 3, 5, 7.5, 10, 12.5, 15, 20, 25, 30 in minutes, or 20, 60, 100, 150, 200, 250, 300, 400, 500, 600 in steps.</code></p>\n<p>From this, we can infer that the conversion between steps and seconds is:<br>\n60 seconds/20 steps = 3 seconds/1 step.</p>\n<p>However, after inspecting the train_events data, I observed that the conversion appears to be 5 seconds/1 step.<br>\nFor instance, consider the series with series_id = 038441c925bb on night = 1:<br>\nThe difference between the timestamps is:<br>\n2018-08-15T06:41:00-0400 - 2018-08-14T22:26:00-0400 = 29700 seconds<br>\nThe difference in steps is:<br>\n10932 - 4992 = 5940 steps</p>\n<p>When computing 29700/5940, it precisely equals 5 seconds per step.</p>\n<p>Is my understanding of the error tolerance thresholds incorrect?</p>",
      "rawMarkdown": "Dear Host,\n\nThank you for hosting the competition! Upon reading the Evaluation page, I noticed the mention of error tolerance thresholds being set at:\n\n`For both event classes, we use error tolerance thresholds of 1, 3, 5, 7.5, 10, 12.5, 15, 20, 25, 30 in minutes, or 20, 60, 100, 150, 200, 250, 300, 400, 500, 600 in steps.`\n\nFrom this, we can infer that the conversion between steps and seconds is:\n60 seconds/20 steps = 3 seconds/1 step.\n\nHowever, after inspecting the train_events data, I observed that the conversion appears to be 5 seconds/1 step.\nFor instance, consider the series with series_id = 038441c925bb on night = 1:\nThe difference between the timestamps is:\n2018-08-15T06:41:00-0400 - 2018-08-14T22:26:00-0400 = 29700 seconds\nThe difference in steps is:\n10932 - 4992 = 5940 steps\n\nWhen computing 29700/5940, it precisely equals 5 seconds per step.\n\nIs my understanding of the error tolerance thresholds incorrect?",
      "votes": null
    },
    {
      "id": "2426591",
      "postDate": "09/06/2023 18:01:47",
      "content": "<p>I'm confused about this too. Just inspecting the data, the timestamps are clearly 5 seconds apart, so the minutes and steps definitions of error thresholds don't match.</p>",
      "rawMarkdown": "I'm confused about this too. Just inspecting the data, the timestamps are clearly 5 seconds apart, so the minutes and steps definitions of error thresholds don't match.",
      "votes": null
    },
    {
      "id": "2426602",
      "postDate": "09/06/2023 18:09:15",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/maruichi01\" target=\"_blank\">@maruichi01</a>,</p>\n<p>Thanks for the heads up. This was an error on our part. The correct steps are <code>[12,  36,  60,  90, 120, 150, 180, 240, 300, 360]</code>. (So 60sec / 12step = 5sec per step.)</p>\n<p>I've updated the evaluation documentation and will rescoring the current submissions with the correct thresholds.</p>",
      "rawMarkdown": "Hi @maruichi01,\n\nThanks for the heads up. This was an error on our part. The correct steps are `[12,  36,  60,  90, 120, 150, 180, 240, 300, 360]`. (So 60sec / 12step = 5sec per step.)\n\nI've updated the evaluation documentation and will rescoring the current submissions with the correct thresholds.",
      "votes": null
    },
    {
      "id": "2426907",
      "postDate": "09/06/2023 23:35:08",
      "content": "<p>Thank you for the quick correction!</p>",
      "rawMarkdown": "Thank you for the quick correction!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2426591,
      "author_name": "apapayawarison",
      "author_url": "",
      "post_date": "09/06/2023 18:01:47",
      "content": "<p>I'm confused about this too. Just inspecting the data, the timestamps are clearly 5 seconds apart, so the minutes and steps definitions of error thresholds don't match.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2426602,
      "author_name": "ryanholbrook",
      "author_url": "",
      "post_date": "09/06/2023 18:09:15",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/maruichi01\" target=\"_blank\">@maruichi01</a>,</p>\n<p>Thanks for the heads up. This was an error on our part. The correct steps are <code>[12,  36,  60,  90, 120, 150, 180, 240, 300, 360]</code>. (So 60sec / 12step = 5sec per step.)</p>\n<p>I've updated the evaluation documentation and will rescoring the current submissions with the correct thresholds.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2426907,
          "author_name": "maruichi01",
          "author_url": "",
          "post_date": "09/06/2023 23:35:08",
          "content": "<p>Thank you for the quick correction!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2425473": "Dear Host,\n\nThank you for hosting the competition! Upon reading the Evaluation page, I noticed the mention of error tolerance thresholds being set at:\n\n`For both event classes, we use error tolerance thresholds of 1, 3, 5, 7.5, 10, 12.5, 15, 20, 25, 30 in minutes, or 20, 60, 100, 150, 200, 250, 300, 400, 500, 600 in steps.`\n\nFrom this, we can infer that the conversion between steps and seconds is:\n60 seconds/20 steps = 3 seconds/1 step.\n\nHowever, after inspecting the train_events data, I observed that the conversion appears to be 5 seconds/1 step.\nFor instance, consider the series with series_id = 038441c925bb on night = 1:\nThe difference between the timestamps is:\n2018-08-15T06:41:00-0400 - 2018-08-14T22:26:00-0400 = 29700 seconds\nThe difference in steps is:\n10932 - 4992 = 5940 steps\n\nWhen computing 29700/5940, it precisely equals 5 seconds per step.\n\nIs my understanding of the error tolerance thresholds incorrect?",
    "2426591": "I'm confused about this too. Just inspecting the data, the timestamps are clearly 5 seconds apart, so the minutes and steps definitions of error thresholds don't match.",
    "2426602": "Hi @maruichi01,\n\nThanks for the heads up. This was an error on our part. The correct steps are `[12,  36,  60,  90, 120, 150, 180, 240, 300, 360]`. (So 60sec / 12step = 5sec per step.)\n\nI've updated the evaluation documentation and will rescoring the current submissions with the correct thresholds.",
    "2426907": "Thank you for the quick correction!"
  },
  "source": "meta"
}