The event dates in train_events.csv is aggregated at 1-minute intervals, which corresponds to 12 steps. At first glance, it might seem logical to round steps to the multiple of 12. However, doing so can negatively impact the Event Detection AP score.
\nFor instance, when the error tolerance threshold is set to 12, and if the predicted step is 240, the only true value that matches is 240. However, if the predicted step is something other than a multiple of 12, like 245, then potential true values that fall within the Error Tolerance of 12 could be both 240 and 252.
\nSimilarly, with an error tolerance threshold of 36, if the predicted step is 240, the potential true values are 216, 228, 240, 252, and 264. But, if the predicted step is 245, the true value candidates increase to 216, 228, 240, 252, 264, and 276.
\nIf you're rounding your predicted steps to multiples of 12, consider avoiding rounding. This might improve your score.
", "messages": [ { "id": 2480855, "postDate": "2023-10-13T15:40:25.913Z", "content": "The event dates in train_events.csv is aggregated at 1-minute intervals, which corresponds to 12 steps. At first glance, it might seem logical to round steps to the multiple of 12. However, doing so can negatively impact the Event Detection AP score.
\nFor instance, when the error tolerance threshold is set to 12, and if the predicted step is 240, the only true value that matches is 240. However, if the predicted step is something other than a multiple of 12, like 245, then potential true values that fall within the Error Tolerance of 12 could be both 240 and 252.
\nSimilarly, with an error tolerance threshold of 36, if the predicted step is 240, the potential true values are 216, 228, 240, 252, and 264. But, if the predicted step is 245, the true value candidates increase to 216, 228, 240, 252, 264, and 276.
\nIf you're rounding your predicted steps to multiples of 12, consider avoiding rounding. This might improve your score.
", "rawMarkdown": "The event dates in *train_events.csv* is aggregated at 1-minute intervals, which corresponds to 12 steps. At first glance, it might seem logical to round steps to the multiple of 12. However, doing so can negatively impact the Event Detection AP score.\n\nFor instance, when the error tolerance threshold is set to 12, and if the predicted step is 240, the only true value that matches is 240. However, if the predicted step is something other than a multiple of 12, like 245, then potential true values that fall within the Error Tolerance of 12 could be both 240 and **252**.\n\nSimilarly, with an error tolerance threshold of 36, if the predicted step is 240, the potential true values are 216, 228, 240, 252, and 264. But, if the predicted step is 245, the true value candidates increase to 216, 228, 240, 252, 264, and **276**.\n\nIf you're rounding your predicted steps to multiples of 12, consider avoiding rounding. This might improve your score.", "votes": 38 }, { "id": 2484725, "postDate": "2023-10-16T16:27:26.970Z", "content": "It's possible to get sub-bin level precision with some tricks. ;)
", "rawMarkdown": "It's possible to get sub-bin level precision with some tricks. ;)", "votes": 3, "replies": [ { "id": 2487642, "postDate": "2023-10-18T17:34:16.310Z", "content": "Hey @tolgadincer what’s the sub-bin level precision do you mean, could you give some hints, much appreciate it!
", "rawMarkdown": "Hey @tolgadincer what’s the sub-bin level precision do you mean, could you give some hints, much appreciate it!" } ] }, { "id": 2486563, "postDate": "2023-10-18T02:56:10.620Z", "content": "If you normalize the step to a multiple of 12, you will not predict a step that is not a multiple of 12, such as 245.
\nOf course, I think normalizing to a multiple of 12 is problematic. When the fault tolerance is 12, if the correct step size is 240, then you can only predict that 240， 228 and 252 are both incorrect.
\nBut if the downsampling is still 12 times, but the step size is taken as the middle step between the beginning and end, such as step size 245 between 240 and 251, then if the correct step size is 240, you can predict that both 245 and 233 (between 228 and 239) are correct.
", "rawMarkdown": "If you normalize the step to a multiple of 12, you will not predict a step that is not a multiple of 12, such as 245. \n\nOf course, I think normalizing to a multiple of 12 is problematic. When the fault tolerance is 12, if the correct step size is 240, then you can only predict that 240， 228 and 252 are both incorrect. \n\nBut if the downsampling is still 12 times, but the step size is taken as the middle step between the beginning and end, such as step size 245 between 240 and 251, then if the correct step size is 240, you can predict that both 245 and 233 (between 228 and 239) are correct. ", "votes": 1 }, { "id": 2481191, "postDate": "2023-10-13T22:50:22.220Z", "content": "I would've thought that 240 includes e.g 204?
\nAt least that's how I would interpret tolerance, since you basically should have predicted 204 but you predicted 240 which has a difference of 36 and thus is within the tolerance? But maybe that's not how it is counted.
You can verify using the Event Detection AP
\ntrain_events = pd.read_csv()\n\ntolerances = {\n : [, , , , , , , , , ],\n : [, , , , , , , , , ] \n}\n\ncolumn_names = {\n : ,\n : ,\n : ,\n : ,\n}\n\nsolution = train_events.copy()\nsolution = solution[~solution[].isnull()]\nsubmission = solution.copy()\nsubmission[] = \n\nscore(solution, submission, tolerances, **column_names)\n\n\nsubmission[] += \nscore(solution, submission, tolerances, **column_names)\n\n\nsubmission[] -= \nsubmission[] += \nscore(solution, submission, tolerances, **column_names)\n\nYes I see, that's a good catch!
", "rawMarkdown": "Yes I see, that's a good catch!" } ] } ] }, { "id": 2482472, "postDate": "2023-10-15T04:30:26.997Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2482973, "postDate": "2023-10-15T12:00:23.903Z", "content": "Interesting, thanks for sharing this
", "rawMarkdown": "Interesting, thanks for sharing this", "votes": 1 } ], "comments": [ { "id": 2484725, "author_name": "Tolga", "author_url": "", "post_date": "2023-10-16T16:27:26.970000", "content": "It's possible to get sub-bin level precision with some tricks. ;)
", "votes": 3, "replies": [ { "id": 2487642, "author_name": "Lamal", "author_url": "", "post_date": "2023-10-18T17:34:16.310000", "content": "Hey @tolgadincer what’s the sub-bin level precision do you mean, could you give some hints, much appreciate it!
", "votes": 0, "replies": [] } ] }, { "id": 2486563, "author_name": "batigol", "author_url": "", "post_date": "2023-10-18T02:56:10.620000", "content": "If you normalize the step to a multiple of 12, you will not predict a step that is not a multiple of 12, such as 245.
\nOf course, I think normalizing to a multiple of 12 is problematic. When the fault tolerance is 12, if the correct step size is 240, then you can only predict that 240， 228 and 252 are both incorrect.
\nBut if the downsampling is still 12 times, but the step size is taken as the middle step between the beginning and end, such as step size 245 between 240 and 251, then if the correct step size is 240, you can predict that both 245 and 233 (between 228 and 239) are correct.
", "votes": 1, "replies": [] }, { "id": 2481191, "author_name": "Fritz Cremer", "author_url": "", "post_date": "2023-10-13T22:50:22.220000", "content": "I would've thought that 240 includes e.g 204?
\nAt least that's how I would interpret tolerance, since you basically should have predicted 204 but you predicted 240 which has a difference of 36 and thus is within the tolerance? But maybe that's not how it is counted.
You can verify using the Event Detection AP
\ntrain_events = pd.read_csv()\n\ntolerances = {\n : [, , , , , , , , , ],\n : [, , , , , , , , , ] \n}\n\ncolumn_names = {\n : ,\n : ,\n : ,\n : ,\n}\n\nsolution = train_events.copy()\nsolution = solution[~solution[].isnull()]\nsubmission = solution.copy()\nsubmission[] = \n\nscore(solution, submission, tolerances, **column_names)\n\n\nsubmission[] += \nscore(solution, submission, tolerances, **column_names)\n\n\nsubmission[] -= \nsubmission[] += \nscore(solution, submission, tolerances, **column_names)\n\nYes I see, that's a good catch!
", "votes": 0, "replies": [] } ] } ] }, { "id": 2482472, "author_name": "", "author_url": "", "post_date": "2023-10-15T04:30:26.997000", "content": "", "votes": 0, "replies": [] }, { "id": 2482973, "author_name": "Yutao", "author_url": "", "post_date": "2023-10-15T12:00:23.903000", "content": "Interesting, thanks for sharing this
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "2480855": "The event dates in *train_events.csv* is aggregated at 1-minute intervals, which corresponds to 12 steps. At first glance, it might seem logical to round steps to the multiple of 12. However, doing so can negatively impact the Event Detection AP score.\n\nFor instance, when the error tolerance threshold is set to 12, and if the predicted step is 240, the only true value that matches is 240. However, if the predicted step is something other than a multiple of 12, like 245, then potential true values that fall within the Error Tolerance of 12 could be both 240 and **252**.\n\nSimilarly, with an error tolerance threshold of 36, if the predicted step is 240, the potential true values are 216, 228, 240, 252, and 264. But, if the predicted step is 245, the true value candidates increase to 216, 228, 240, 252, 264, and **276**.\n\nIf you're rounding your predicted steps to multiples of 12, consider avoiding rounding. This might improve your score.", "2484725": "It's possible to get sub-bin level precision with some tricks. ;)", "2486563": "If you normalize the step to a multiple of 12, you will not predict a step that is not a multiple of 12, such as 245. \n\nOf course, I think normalizing to a multiple of 12 is problematic. When the fault tolerance is 12, if the correct step size is 240, then you can only predict that 240， 228 and 252 are both incorrect. \n\nBut if the downsampling is still 12 times, but the step size is taken as the middle step between the beginning and end, such as step size 245 between 240 and 251, then if the correct step size is 240, you can predict that both 245 and 233 (between 228 and 239) are correct. ", "2481191": "I would've thought that 240 includes e.g 204?\nAt least that's how I would interpret tolerance, since you basically should have predicted 204 but you predicted 240 which has a difference of 36 and thus is within the tolerance? But maybe that's not how it is counted.", "2482472": "", "2482973": "Interesting, thanks for sharing this" } }