I just started looking at the competition, and I see that a number of recent submissions on the leaderboard have already reached up to 0.98 AUC. The maximum value for the weighted AUC metric is 1. Am I missing something here?
", "messages": [ { "id": "86424", "postDate": "07/22/2015 06:39:48", "content": "I just started looking at the competition, and I see that a number of recent submissions on the leaderboard have already reached up to 0.98 AUC. The maximum value for the weighted AUC metric is 1. Am I missing something here?
", "rawMarkdown": "I just started looking at the competition, and I see that a number of recent submissions on the leaderboard have already reached up to 0.98 AUC. The maximum value for the weighted AUC metric is 1. Am I missing something here?", "votes": null }, { "id": "86492", "postDate": "07/22/2015 16:28:47", "content": "It is a weighted AUC. For example, random will give you 0.7 and not the usual 0.5
", "rawMarkdown": "It is a weighted AUC. For example, random will give you 0.7 and not the usual 0.5", "votes": null }, { "id": "86499", "postDate": "07/22/2015 17:36:11", "content": "Yes, the maximum value for the weighted ROC AUC is 1. It was right note that for a random classifier it will be 0.7 (not 0.5 as usual). Thus for the weighted AUC you have shifted quality scale.
", "rawMarkdown": "Yes, the maximum value for the weighted ROC AUC is 1. It was right note that for a random classifier it will be 0.7 (not 0.5 as usual). Thus for the weighted AUC you have shifted quality scale.", "votes": null }, { "id": "86501", "postDate": "07/22/2015 17:52:32", "content": "Thanks, got it. I still find it unintuitive that the scores for the competition will be resolved over 1-2 percentile points. Not that it changes anything. :)
", "rawMarkdown": "Thanks, got it. I still find it unintuitive that the scores for the competition will be resolved over 1-2 percentile points. Not that it changes anything. :)", "votes": null }, { "id": "86506", "postDate": "07/22/2015 18:13:39", "content": "These 1-2 percentile points play a crucial role for rare decays. We want to see 5-8 signal events among a lot of background events. Little improvement of the classifier quality can significantly decrease the amount of background events while saving nearly all expected signal events. If there are, say, around 100 background events that passed all preselections, we'll be unable to prove the presence (or absence) of decay.
", "rawMarkdown": "These 1-2 percentile points play a crucial role for rare decays. We want to see 5-8 signal events among a lot of background events. Little improvement of the classifier quality can significantly decrease the amount of background events while saving nearly all expected signal events. If there are, say, around 100 background events that passed all preselections, we'll be unable to prove the presence (or absence) of decay.", "votes": null }, { "id": "86511", "postDate": "07/22/2015 18:53:16", "content": "Thanks for the context, Tatiana.
", "rawMarkdown": "Thanks for the context, Tatiana.", "votes": null } ], "comments": [ { "id": 86492, "author_name": "udibr1", "author_url": "", "post_date": "07/22/2015 16:28:47", "content": "It is a weighted AUC. For example, random will give you 0.7 and not the usual 0.5
", "votes": null, "replies": [] }, { "id": 86499, "author_name": "tataantares", "author_url": "", "post_date": "07/22/2015 17:36:11", "content": "Yes, the maximum value for the weighted ROC AUC is 1. It was right note that for a random classifier it will be 0.7 (not 0.5 as usual). Thus for the weighted AUC you have shifted quality scale.
", "votes": null, "replies": [] }, { "id": 86501, "author_name": "neerajprad", "author_url": "", "post_date": "07/22/2015 17:52:32", "content": "Thanks, got it. I still find it unintuitive that the scores for the competition will be resolved over 1-2 percentile points. Not that it changes anything. :)
", "votes": null, "replies": [] }, { "id": 86506, "author_name": "tataantares", "author_url": "", "post_date": "07/22/2015 18:13:39", "content": "These 1-2 percentile points play a crucial role for rare decays. We want to see 5-8 signal events among a lot of background events. Little improvement of the classifier quality can significantly decrease the amount of background events while saving nearly all expected signal events. If there are, say, around 100 background events that passed all preselections, we'll be unable to prove the presence (or absence) of decay.
", "votes": null, "replies": [] }, { "id": 86511, "author_name": "neerajprad", "author_url": "", "post_date": "07/22/2015 18:53:16", "content": "Thanks for the context, Tatiana.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "86424": "I just started looking at the competition, and I see that a number of recent submissions on the leaderboard have already reached up to 0.98 AUC. The maximum value for the weighted AUC metric is 1. Am I missing something here?", "86492": "It is a weighted AUC. For example, random will give you 0.7 and not the usual 0.5", "86499": "Yes, the maximum value for the weighted ROC AUC is 1. It was right note that for a random classifier it will be 0.7 (not 0.5 as usual). Thus for the weighted AUC you have shifted quality scale.", "86501": "Thanks, got it. I still find it unintuitive that the scores for the competition will be resolved over 1-2 percentile points. Not that it changes anything. :)", "86506": "These 1-2 percentile points play a crucial role for rare decays. We want to see 5-8 signal events among a lot of background events. Little improvement of the classifier quality can significantly decrease the amount of background events while saving nearly all expected signal events. If there are, say, around 100 background events that passed all preselections, we'll be unable to prove the presence (or absence) of decay.", "86511": "Thanks for the context, Tatiana." }, "source": "meta" }