aggregation_keys = pd.DataFrame(\n [(ev, tol, vid)\n ev tolerances.keys()\n tol tolerances[ev]\n vid ground_truths[series_id_column_name].unique()],\n columns=[event_column_name, , series_id_column_name],\n )\nThis only considers series ids with at least one event. There are actually 8 series with no event and with this implementation, predicting any event on these series doesn't effect the metric. By fixing this, my local score goes down by 0.002.
", "messages": [ { "id": "2506306", "postDate": "10/31/2023 07:56:17", "content": " aggregation_keys = pd.DataFrame(\n [(ev, tol, vid)\n ev tolerances.keys()\n tol tolerances[ev]\n vid ground_truths[series_id_column_name].unique()],\n columns=[event_column_name, , series_id_column_name],\n )\nThis only considers series ids with at least one event. There are actually 8 series with no event and with this implementation, predicting any event on these series doesn't effect the metric. By fixing this, my local score goes down by 0.002.
", "rawMarkdown": "```python\n aggregation_keys = pd.DataFrame(\n [(ev, tol, vid)\n for ev in tolerances.keys()\n for tol in tolerances[ev]\n for vid in ground_truths[series_id_column_name].unique()],\n columns=[event_column_name, 'tolerance', series_id_column_name],\n )\n```\n\nThis only considers series ids with at least one event. There are actually 8 series with no event and with this implementation, predicting any event on these series doesn't effect the metric. By fixing this, my local score goes down by 0.002.", "votes": null }, { "id": "2506746", "postDate": "10/31/2023 14:13:54", "content": "Thanks @aerdem4 , I'll look into it.
", "rawMarkdown": "Thanks @aerdem4 , I'll look into it.", "votes": null }, { "id": "2506840", "postDate": "10/31/2023 15:19:19", "content": "@aerdem4 This seems relevant to my finding: https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/451179#2501458
", "rawMarkdown": "aerdem4 This seems relevant to my finding: https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/451179#2501458", "votes": null }, { "id": "2506848", "postDate": "10/31/2023 15:28:10", "content": "Yes, it is the same but I would say FPs are not punished if they are in a series with no positives.
", "rawMarkdown": "Yes, it is the same but I would say FPs are not punished if they are in a series with no positives.", "votes": null }, { "id": "2513939", "postDate": "11/05/2023 21:48:16", "content": "Are there any updates on this?
", "rawMarkdown": "Are there any updates on this?", "votes": null }, { "id": "2515097", "postDate": "11/06/2023 18:14:17", "content": "Hi @manwithaflower, I'm working on it. I hope to have a fix up in the next couple days. I don't expect there will be much change in the scores, though.
", "rawMarkdown": "Hi @manwithaflower, I'm working on it. I hope to have a fix up in the next couple days. I don't expect there will be much change in the scores, though.", "votes": null }, { "id": "2522475", "postDate": "11/12/2023 17:37:37", "content": "Actually, it would be great to know if there are any series with no labels in the test data. If no, nothing needs to be fixed. We can handle it in our training and post processing strategies.
", "rawMarkdown": "Actually, it would be great to know if there are any series with no labels in the test data. If no, nothing needs to be fixed. We can handle it in our training and post processing strategies.", "votes": null }, { "id": "2536313", "postDate": "11/24/2023 05:49:23", "content": "Very useful thanks bro
", "rawMarkdown": "Very useful thanks bro", "votes": null } ], "comments": [ { "id": 2506746, "author_name": "ryanholbrook", "author_url": "", "post_date": "10/31/2023 14:13:54", "content": "Thanks @aerdem4 , I'll look into it.
", "votes": null, "replies": [ { "id": 2513939, "author_name": "manwithaflower", "author_url": "", "post_date": "11/05/2023 21:48:16", "content": "Are there any updates on this?
", "votes": null, "replies": [ { "id": 2515097, "author_name": "ryanholbrook", "author_url": "", "post_date": "11/06/2023 18:14:17", "content": "Hi @manwithaflower, I'm working on it. I hope to have a fix up in the next couple days. I don't expect there will be much change in the scores, though.
", "votes": null, "replies": [] } ] } ] }, { "id": 2506840, "author_name": "tolgadincer", "author_url": "", "post_date": "10/31/2023 15:19:19", "content": "@aerdem4 This seems relevant to my finding: https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/451179#2501458
", "votes": null, "replies": [ { "id": 2506848, "author_name": "aerdem4", "author_url": "", "post_date": "10/31/2023 15:28:10", "content": "Yes, it is the same but I would say FPs are not punished if they are in a series with no positives.
", "votes": null, "replies": [] } ] }, { "id": 2522475, "author_name": "tolgadincer", "author_url": "", "post_date": "11/12/2023 17:37:37", "content": "Actually, it would be great to know if there are any series with no labels in the test data. If no, nothing needs to be fixed. We can handle it in our training and post processing strategies.
", "votes": null, "replies": [] }, { "id": 2536313, "author_name": "ziyuechen3", "author_url": "", "post_date": "11/24/2023 05:49:23", "content": "Very useful thanks bro
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2506306": "```python\n aggregation_keys = pd.DataFrame(\n [(ev, tol, vid)\n for ev in tolerances.keys()\n for tol in tolerances[ev]\n for vid in ground_truths[series_id_column_name].unique()],\n columns=[event_column_name, 'tolerance', series_id_column_name],\n )\n```\n\nThis only considers series ids with at least one event. There are actually 8 series with no event and with this implementation, predicting any event on these series doesn't effect the metric. By fixing this, my local score goes down by 0.002.", "2506746": "Thanks @aerdem4 , I'll look into it.", "2506840": "aerdem4 This seems relevant to my finding: https://www.kaggle.com/competitions/child-mind-institute-detect-sleep-states/discussion/451179#2501458", "2506848": "Yes, it is the same but I would say FPs are not punished if they are in a series with no positives.", "2513939": "Are there any updates on this?", "2515097": "Hi @manwithaflower, I'm working on it. I hope to have a fix up in the next couple days. I don't expect there will be much change in the scores, though.", "2522475": "Actually, it would be great to know if there are any series with no labels in the test data. If no, nothing needs to be fixed. We can handle it in our training and post processing strategies.", "2536313": "Very useful thanks bro" }, "source": "meta" }