I have read the evaluation process multiple times but am still not able to process how to get the results score column. Also, what is the target column?
", "messages": [ { "id": 2426275, "postDate": "2023-09-06T14:21:04.367Z", "content": "Hi! The score column in your submissions represent the \"confidence\" of your model in assigning a given event at that particular step. The target for evaluation is the step x event type that you assign in each timeseries (i.e., are you placing onset and wake events at the correct locations). Does that help?
Thanks for joining us in the competition!
", "rawMarkdown": "Hi! The `score` column in your submissions represent the \"confidence\" of your model in assigning a given event at that particular step. The `target` for evaluation is the `step` x `event` type that you assign in each timeseries (i.e., are you placing `onset` and `wake` events at the correct locations). Does that help?\n\nThanks for joining us in the competition!", "votes": 5, "replies": [ { "id": 2426597, "postDate": "2023-09-06T18:05:03.810Z", "content": "Confidence makes way more sense as wording than score
Still, a small confusion @gkiar07, why are there only 2 outputs per series_id in submission.csv?
\nOtherwise whatever you mentioned is clear to me now.
", "rawMarkdown": "Still, a small confusion @gkiar07, why are there only 2 outputs per series_id in submission.csv?\n\nOtherwise whatever you mentioned is clear to me now.", "replies": [ { "id": 2427742, "postDate": "2023-09-07T13:01:01.507Z", "content": "Hi @warcoder,
\nThe sample_submission.csv you see on the Data page is just an example. The number of events for each series isn't predefined, so part of the problem is making the \"correct\" number of event detections for a series.
Is it ok to put the predictions of my model for every time step? For example if my model predicts an event occuring at timestep0 with a very low confidence and in that timestep that event does not happen, so a True Negative, will there be a penalty for it? The example submission includes predictions with 0.0 confidence and I could not see anything about True Negatives in the explanation of the score
", "rawMarkdown": "Is it ok to put the predictions of my model for every time step? For example if my model predicts an event occuring at timestep0 with a very low confidence and in that timestep that event does not happen, so a True Negative, will there be a penalty for it? The example submission includes predictions with 0.0 confidence and I could not see anything about True Negatives in the explanation of the score", "votes": 4 }, { "id": 2428086, "postDate": "2023-09-07T16:39:05.353Z", "content": "You should not make predictions for the negative case. It would be best to avoid making an excessive number of predictions, as well, certainly not at every time step.
", "rawMarkdown": "You should not make predictions for the negative case. It would be best to avoid making an excessive number of predictions, as well, certainly not at every time step.", "votes": 3 } ] }, { "id": 2463437, "postDate": "2023-10-01T10:27:04.087Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2478637, "postDate": "2023-10-12T05:58:18.080Z", "content": "So if I am correct score column is just a predict.probability right?
\nThanks,Happy Kaggling
I have read the evaluation process multiple times but am still not able to process how to get the results score column. Also, what is the target column?
", "rawMarkdown": "I have read the evaluation process multiple times but am still not able to process how to get the results score column. Also, what is the target column?", "votes": 6 } ], "comments": [ { "id": 2426275, "author_name": "Greg Kiar", "author_url": "", "post_date": "2023-09-06T14:21:04.367000", "content": "Hi! The score column in your submissions represent the \"confidence\" of your model in assigning a given event at that particular step. The target for evaluation is the step x event type that you assign in each timeseries (i.e., are you placing onset and wake events at the correct locations). Does that help?
Thanks for joining us in the competition!
", "votes": 5, "replies": [ { "id": 2426597, "author_name": "t0xic-wast3", "author_url": "", "post_date": "2023-09-06T18:05:03.810000", "content": "Confidence makes way more sense as wording than score
Still, a small confusion @gkiar07, why are there only 2 outputs per series_id in submission.csv?
\nOtherwise whatever you mentioned is clear to me now.
", "votes": 0, "replies": [ { "id": 2427742, "author_name": "Ryan Holbrook", "author_url": "", "post_date": "2023-09-07T13:01:01.507000", "content": "Hi @warcoder,
\nThe sample_submission.csv you see on the Data page is just an example. The number of events for each series isn't predefined, so part of the problem is making the \"correct\" number of event detections for a series.
Is it ok to put the predictions of my model for every time step? For example if my model predicts an event occuring at timestep0 with a very low confidence and in that timestep that event does not happen, so a True Negative, will there be a penalty for it? The example submission includes predictions with 0.0 confidence and I could not see anything about True Negatives in the explanation of the score
", "votes": 4, "replies": [] }, { "id": 2428086, "author_name": "Ryan Holbrook", "author_url": "", "post_date": "2023-09-07T16:39:05.353000", "content": "You should not make predictions for the negative case. It would be best to avoid making an excessive number of predictions, as well, certainly not at every time step.
", "votes": 3, "replies": [] } ] }, { "id": 2463437, "author_name": "", "author_url": "", "post_date": "2023-10-01T10:27:04.087000", "content": "", "votes": 0, "replies": [] }, { "id": 2478637, "author_name": "Kumar Gaurav", "author_url": "", "post_date": "2023-10-12T05:58:18.080000", "content": "So if I am correct score column is just a predict.probability right?
\nThanks,Happy Kaggling