![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3293834%2Fec44f8b616b42133c65431dc2e7bb12e%2FCMI_1_1.png?generation=1700145287743365&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3293834%2F2d73c04d68e42caf2c8b240f2f16fd30%2Fcmi2_.png?generation=1700146081129651&alt=media\")
While reading @carlmcbrideellis notebook I found these to event_detection_score different.
\n((1.0x28)+(0.1x46))/(28+46) = 0.44054054054054054
\nSo I think the event_detection_score should be 0.44054054054054054 regardless of the submission score.
\nCan anyone explain why the event_detection_score is different when all of the scores are the same across series_ids?
\nI'm pretty stuck 🤔
\nDoes score doesn't matter at all unless we are unifying these values?
While reading @carlmcbrideellis notebook I found these to event_detection_score different.
\n((1.0x28)+(0.1x46))/(28+46) = 0.44054054054054054
\nSo I think the event_detection_score should be 0.44054054054054054 regardless of the submission score.
\nCan anyone explain why the event_detection_score is different when all of the scores are the same across series_ids?
\nI'm pretty stuck 🤔
\nDoes score doesn't matter at all unless we are unifying these values?
Result is calculated for all series_id's combined. 'score' you supply is compared to true match probability, and something similar to ROC curve is constructed. This is not the same as AUC metric, but it is similar.
\nMy point is, 'score' is supposed to represent probability of match to true result. If 'score' closely tracks that probability, result is high. If it does not, result is low.
\nSo in your example i guess second series had a lot lower true match probability, so assigning it lower score improved the result.
", "rawMarkdown": "Result is calculated for all series_id's combined. 'score' you supply is compared to true match probability, and something similar to ROC curve is constructed. This is not the same as AUC metric, but it is similar.\n\nMy point is, 'score' is supposed to represent probability of match to true result. If 'score' closely tracks that probability, result is high. If it does not, result is low.\n\nSo in your example i guess second series had a lot lower true match probability, so assigning it lower score improved the result.", "votes": null }, { "id": "2530069", "postDate": "11/18/2023 18:49:27", "content": "Thanks for your reply first.
\nI tried 0.2, 0.9 etc. instead of 0.1 for the second series but it gave me the same result.
\nAlso tried changing the scores to a random number for my submission.
\nI think it doesn't matter that much if our score is not that bad.
\nSo I'm thinking just assigning the probability as the 'score'.
\nStill stuck with the exact mechanism but probably got enough info about 'score' thing in our submission.
\nThanks for your comment!
Result is calculated for all series_id's combined. 'score' you supply is compared to true match probability, and something similar to ROC curve is constructed. This is not the same as AUC metric, but it is similar.
\nMy point is, 'score' is supposed to represent probability of match to true result. If 'score' closely tracks that probability, result is high. If it does not, result is low.
\nSo in your example i guess second series had a lot lower true match probability, so assigning it lower score improved the result.
", "votes": null, "replies": [ { "id": 2530069, "author_name": "danielchae", "author_url": "", "post_date": "11/18/2023 18:49:27", "content": "Thanks for your reply first.
\nI tried 0.2, 0.9 etc. instead of 0.1 for the second series but it gave me the same result.
\nAlso tried changing the scores to a random number for my submission.
\nI think it doesn't matter that much if our score is not that bad.
\nSo I'm thinking just assigning the probability as the 'score'.
\nStill stuck with the exact mechanism but probably got enough info about 'score' thing in our submission.
\nThanks for your comment!