Please be patient with my ignorance.\nI submit "IPSig" transformed so that "0"+"1" in Training.Csv should be flat.\nThis gives my score 0.29 -- I reckon it should be 0.95 or so.\nI then sort Test.Csv on "IPSig" and flatten it in [0;1].\nThen I get a KS of 0.6.\nAm I a complete ignoramus in assuming AUC takes no consideration to absolute values, but only to the order of events?\nI am not using any of the scripts; this is pure enjoyment for me and I write from scratch in C# -- just a simple bug in my code?\n(enclosed image: train signal/background, agreement signal/background and background test/train after transform "0"+"1" flat in Train).\nPlease help!?
", "messages": [ { "id": "89502", "postDate": "08/16/2015 12:12:43", "content": "Please be patient with my ignorance.\nI submit "IPSig" transformed so that "0"+"1" in Training.Csv should be flat.\nThis gives my score 0.29 -- I reckon it should be 0.95 or so.\nI then sort Test.Csv on "IPSig" and flatten it in [0;1].\nThen I get a KS of 0.6.\nAm I a complete ignoramus in assuming AUC takes no consideration to absolute values, but only to the order of events?\nI am not using any of the scripts; this is pure enjoyment for me and I write from scratch in C# -- just a simple bug in my code?\n(enclosed image: train signal/background, agreement signal/background and background test/train after transform "0"+"1" flat in Train).\nPlease help!?
", "rawMarkdown": "Please be patient with my ignorance.\r\nI submit \"IPSig\" transformed so that \"0\"+\"1\" in Training.Csv should be flat.\r\nThis gives my score 0.29 -- I reckon it should be 0.95 or so.\r\nI then sort Test.Csv on \"IPSig\" and flatten it in [0;1].\r\nThen I get a KS of 0.6.\r\nAm I a complete ignoramus in assuming AUC takes no consideration to absolute values, but only to the order of events?\r\nI am not using any of the scripts; this is pure enjoyment for me and I write from scratch in C# -- just a simple bug in my code?\r\n(enclosed image: train signal/background, agreement signal/background and background test/train after transform \"0\"+\"1\" flat in Train).\r\nPlease help!?", "votes": null }, { "id": "89546", "postDate": "08/17/2015 08:11:04", "content": "So, if I sort my entries over "Id", the KS is .59.\nIf I sort on "IPSig", the KS is .6.\nIf I submit in the original order of "Id", the score is .29 for "IPSig"\nDoes this mean the Ids are really not used and the submission must come in the original order of Ids?
", "rawMarkdown": "So, if I sort my entries over \"Id\", the KS is .59.\r\nIf I sort on \"IPSig\", the KS is .6.\r\nIf I submit in the original order of \"Id\", the score is .29 for \"IPSig\"\r\nDoes this mean the Ids are really not used and the submission must come in the original order of Ids?", "votes": null }, { "id": "89622", "postDate": "08/18/2015 00:54:07", "content": "@Glimmung,
\n\nOrder doesn't matter in the evaluation metric, both in KS test and in the weighted AUC calculation. You don't have to submit with the original order of IDs.
\n\nI'm testing it locally but can't reproduce the discrepancy you are describing. If this is still a problem for you, please email me your submission files and I'll test them out.
", "rawMarkdown": "Glimmung,\r\n\r\nOrder doesn't matter in the evaluation metric, both in KS test and in the weighted AUC calculation. You don't have to submit with the original order of IDs. \r\n\r\nI'm testing it locally but can't reproduce the discrepancy you are describing. If this is still a problem for you, please email me your submission files and I'll test them out.", "votes": null }, { "id": "89713", "postDate": "08/18/2015 19:57:25", "content": "Thanks, Wendy, I have tried to send you a file.\nI have added two sorting routines so I can bypass this problem, so it's not a major issue any longer.
", "rawMarkdown": "Thanks, Wendy, I have tried to send you a file.\r\nI have added two sorting routines so I can bypass this problem, so it's not a major issue any longer.", "votes": null }, { "id": "89860", "postDate": "08/19/2015 22:53:59", "content": "@Glimmung,
\n\nThank you so much for reporting this and sending the files. We have fixed this bug now.
", "rawMarkdown": "Glimmung,\r\n\r\nThank you so much for reporting this and sending the files. We have fixed this bug now.", "votes": null } ], "comments": [ { "id": 89546, "author_name": "glimmung", "author_url": "", "post_date": "08/17/2015 08:11:04", "content": "So, if I sort my entries over "Id", the KS is .59.\nIf I sort on "IPSig", the KS is .6.\nIf I submit in the original order of "Id", the score is .29 for "IPSig"\nDoes this mean the Ids are really not used and the submission must come in the original order of Ids?
", "votes": null, "replies": [] }, { "id": 89622, "author_name": "wendykan", "author_url": "", "post_date": "08/18/2015 00:54:07", "content": "@Glimmung,
\n\nOrder doesn't matter in the evaluation metric, both in KS test and in the weighted AUC calculation. You don't have to submit with the original order of IDs.
\n\nI'm testing it locally but can't reproduce the discrepancy you are describing. If this is still a problem for you, please email me your submission files and I'll test them out.
", "votes": null, "replies": [] }, { "id": 89713, "author_name": "glimmung", "author_url": "", "post_date": "08/18/2015 19:57:25", "content": "Thanks, Wendy, I have tried to send you a file.\nI have added two sorting routines so I can bypass this problem, so it's not a major issue any longer.
", "votes": null, "replies": [] }, { "id": 89860, "author_name": "wendykan", "author_url": "", "post_date": "08/19/2015 22:53:59", "content": "@Glimmung,
\n\nThank you so much for reporting this and sending the files. We have fixed this bug now.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "89502": "Please be patient with my ignorance.\r\nI submit \"IPSig\" transformed so that \"0\"+\"1\" in Training.Csv should be flat.\r\nThis gives my score 0.29 -- I reckon it should be 0.95 or so.\r\nI then sort Test.Csv on \"IPSig\" and flatten it in [0;1].\r\nThen I get a KS of 0.6.\r\nAm I a complete ignoramus in assuming AUC takes no consideration to absolute values, but only to the order of events?\r\nI am not using any of the scripts; this is pure enjoyment for me and I write from scratch in C# -- just a simple bug in my code?\r\n(enclosed image: train signal/background, agreement signal/background and background test/train after transform \"0\"+\"1\" flat in Train).\r\nPlease help!?", "89546": "So, if I sort my entries over \"Id\", the KS is .59.\r\nIf I sort on \"IPSig\", the KS is .6.\r\nIf I submit in the original order of \"Id\", the score is .29 for \"IPSig\"\r\nDoes this mean the Ids are really not used and the submission must come in the original order of Ids?", "89622": "Glimmung,\r\n\r\nOrder doesn't matter in the evaluation metric, both in KS test and in the weighted AUC calculation. You don't have to submit with the original order of IDs. \r\n\r\nI'm testing it locally but can't reproduce the discrepancy you are describing. If this is still a problem for you, please email me your submission files and I'll test them out.", "89713": "Thanks, Wendy, I have tried to send you a file.\r\nI have added two sorting routines so I can bypass this problem, so it's not a major issue any longer.", "89860": "Glimmung,\r\n\r\nThank you so much for reporting this and sending the files. We have fixed this bug now." }, "source": "meta" }