{
  "id": 442092,
  "title": "\"match each ground-truth to the highest-confidence unmatched prediction occurring within the allowed tolerance\"",
  "url": "/competitions/child-mind-institute-detect-sleep-states/discussion/442092",
  "author_name": "MeerkatStatistics",
  "post_date": "2023-09-21T09:26:47.937000",
  "votes": 1,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I'm trying to understand what is needed to submit to this competition. </p>\n<p>Reading the evaluation part, it seems that predictions will be matched against ground truth with thresholds of errors (tolerances), and that for each tolerance the prediction with the highest confidence score in that window will be taken. </p>\n<p>Does this mean it is best to submit multiple events with different degree of confidence scores?</p>\n<p>I.e., <br>\n0,038441c925bb,100,onset,0.1<br>\n1,038441c925bb,150,onset,0.5<br>\n2,038441c925bb,200,onset,0.8<br>\n3,038441c925bb,250,onset,0.3<br>\n4,038441c925bb,300,onset,0.1<br>\netc.</p>\n<p>Is it possible to submit something like that? <br>\nAnd does it even make sense? I mean, what is the benefit of choosing the highest confidence? In my example, if the ground truth was before step 100, what difference does it makes if it is matched against prediction 0, 1, 2…? If its between step 150 and 200, and the current tolerance includes both, again - what difference does it make if it is matched against prediction 1 and 2? </p>\n<p>Would appreciate some clarification about the submission.<br>\n🙏</p>",
  "messages": [
    {
      "id": 2449597,
      "postDate": "2023-09-21T09:26:47.937Z",
      "content": "<p>I'm trying to understand what is needed to submit to this competition. </p>\n<p>Reading the evaluation part, it seems that predictions will be matched against ground truth with thresholds of errors (tolerances), and that for each tolerance the prediction with the highest confidence score in that window will be taken. </p>\n<p>Does this mean it is best to submit multiple events with different degree of confidence scores?</p>\n<p>I.e., <br>\n0,038441c925bb,100,onset,0.1<br>\n1,038441c925bb,150,onset,0.5<br>\n2,038441c925bb,200,onset,0.8<br>\n3,038441c925bb,250,onset,0.3<br>\n4,038441c925bb,300,onset,0.1<br>\netc.</p>\n<p>Is it possible to submit something like that? <br>\nAnd does it even make sense? I mean, what is the benefit of choosing the highest confidence? In my example, if the ground truth was before step 100, what difference does it makes if it is matched against prediction 0, 1, 2…? If its between step 150 and 200, and the current tolerance includes both, again - what difference does it make if it is matched against prediction 1 and 2? </p>\n<p>Would appreciate some clarification about the submission.<br>\n🙏</p>",
      "rawMarkdown": "I'm trying to understand what is needed to submit to this competition. \n\nReading the evaluation part, it seems that predictions will be matched against ground truth with thresholds of errors (tolerances), and that for each tolerance the prediction with the highest confidence score in that window will be taken. \n\nDoes this mean it is best to submit multiple events with different degree of confidence scores?\n\nI.e., \n0,038441c925bb,100,onset,0.1\n1,038441c925bb,150,onset,0.5\n2,038441c925bb,200,onset,0.8\n3,038441c925bb,250,onset,0.3\n4,038441c925bb,300,onset,0.1\netc.\n\nIs it possible to submit something like that? \nAnd does it even make sense? I mean, what is the benefit of choosing the highest confidence? In my example, if the ground truth was before step 100, what difference does it makes if it is matched against prediction 0, 1, 2...? If its between step 150 and 200, and the current tolerance includes both, again - what difference does it make if it is matched against prediction 1 and 2? \n\nWould appreciate some clarification about the submission.\n🙏\n",
      "votes": 1
    },
    {
      "id": 2449824,
      "postDate": "2023-09-21T13:02:07.397Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 2449824,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-09-21T13:02:07.397000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2449597": "I'm trying to understand what is needed to submit to this competition. \n\nReading the evaluation part, it seems that predictions will be matched against ground truth with thresholds of errors (tolerances), and that for each tolerance the prediction with the highest confidence score in that window will be taken. \n\nDoes this mean it is best to submit multiple events with different degree of confidence scores?\n\nI.e., \n0,038441c925bb,100,onset,0.1\n1,038441c925bb,150,onset,0.5\n2,038441c925bb,200,onset,0.8\n3,038441c925bb,250,onset,0.3\n4,038441c925bb,300,onset,0.1\netc.\n\nIs it possible to submit something like that? \nAnd does it even make sense? I mean, what is the benefit of choosing the highest confidence? In my example, if the ground truth was before step 100, what difference does it makes if it is matched against prediction 0, 1, 2...? If its between step 150 and 200, and the current tolerance includes both, again - what difference does it make if it is matched against prediction 1 and 2? \n\nWould appreciate some clarification about the submission.\n🙏\n",
    "2449824": ""
  }
}