Is it just me, or are weird things happening on the leaderboard since the recent Kaggle updates? The MAEs of the same submissions appear to have changed twice over the last 24 hours.
", "messages": [ { "id": "2085", "postDate": "03/28/2011 08:50:17", "content": "Is it just me, or are weird things happening on the leaderboard since the recent Kaggle updates? The MAEs of the same submissions appear to have changed twice over the last 24 hours.
", "rawMarkdown": "", "votes": null }, { "id": "2086", "postDate": "03/28/2011 11:19:52", "content": "I think they are working on it.
", "rawMarkdown": "", "votes": null }, { "id": "2087", "postDate": "03/28/2011 13:07:21", "content": "I'm working on the new site and I made an update last night to fix a few issues which is why you noticed the change.\r\n\r\nThe new code assumed that public scores of 0 were invalid, which isn't the case for this competition :) I fixed that error and now you correctly see that there are three teams with a public score of zero. \r\n\r\nIn addition, I updated the leaderboard to reflect *when* your team first attained the score listed. I think this is more fair in the case of a tie to see who was first with the results. You can see when a team last made an update by hovering over the team name link (i.e. it's in the alt text).\r\n\r\nWhat do you think of the changes?", "rawMarkdown": "", "votes": null }, { "id": "2088", "postDate": "03/28/2011 15:04:59", "content": "I'm fine with anything as long as the scores on the leaderboard are correct :)
\r\nThe only thing I know is that last Saturday, there was one team with 0 error; Sunday, there were no teams with 0 error; and now, there are five teams with 0 error. This made me suspicious, since there was a bug before that caused all-zero rows to get 0 error.
", "rawMarkdown": "", "votes": null }, { "id": "2089", "postDate": "03/28/2011 17:24:13", "content": "It's quite possible there is a bug. I'll look into it and let you know what I find.", "rawMarkdown": "", "votes": null }, { "id": "2090", "postDate": "03/28/2011 18:16:15", "content": "I used to be notified by email whenever a message is posted on this forum, but it is not the case anymore.
", "rawMarkdown": "", "votes": null }, { "id": "2093", "postDate": "03/28/2011 23:04:44", "content": "There was a problem in how the public leaderboard was being scored (only very few cells were being checked). The problem should be fixed now. \r\n\r\nIn order to ensure that all calculations are the same, I reran the scoring with the new code for all submissions (including those made before Kaggle 2.0 went live). \r\n\r\nIt is possible that the public leaderboard is now based on slightly different values that were not previously used before, however it is still a ~30% subset of the data. \r\n\r\nAgain, I want to reiterate that all submissions were re-scored using this new algorithm so that everything should be consistent.\r\n\r\nPlease let me know if you have further issues. Sorry about the hassle this bug caused.", "rawMarkdown": "", "votes": null }, { "id": "2503", "postDate": "04/08/2011 11:41:13", "content": "Sorry for such a basic question, but how does one get to the leaderboard?\r\nThe rules state that it won't be available until May, yet from the discussion it appears to be avialable.\r\nThanks.", "rawMarkdown": "", "votes": null }, { "id": "2509", "postDate": "04/08/2011 13:47:54", "content": "gschmidt: You're referring to the Heritage Health Prize. This is a different competition altogether.", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 2086, "author_name": "ahassaine", "author_url": "", "post_date": "03/28/2011 11:19:52", "content": "I think they are working on it.
", "votes": null, "replies": [] }, { "id": 2087, "author_name": "jeffmoser", "author_url": "", "post_date": "03/28/2011 13:07:21", "content": "I'm working on the new site and I made an update last night to fix a few issues which is why you noticed the change.\r\n\r\nThe new code assumed that public scores of 0 were invalid, which isn't the case for this competition :) I fixed that error and now you correctly see that there are three teams with a public score of zero. \r\n\r\nIn addition, I updated the leaderboard to reflect *when* your team first attained the score listed. I think this is more fair in the case of a tie to see who was first with the results. You can see when a team last made an update by hovering over the team name link (i.e. it's in the alt text).\r\n\r\nWhat do you think of the changes?", "votes": null, "replies": [] }, { "id": 2088, "author_name": "lvdmaaten", "author_url": "", "post_date": "03/28/2011 15:04:59", "content": "I'm fine with anything as long as the scores on the leaderboard are correct :)
\r\nThe only thing I know is that last Saturday, there was one team with 0 error; Sunday, there were no teams with 0 error; and now, there are five teams with 0 error. This made me suspicious, since there was a bug before that caused all-zero rows to get 0 error.
", "votes": null, "replies": [] }, { "id": 2089, "author_name": "jeffmoser", "author_url": "", "post_date": "03/28/2011 17:24:13", "content": "It's quite possible there is a bug. I'll look into it and let you know what I find.", "votes": null, "replies": [] }, { "id": 2090, "author_name": "ahassaine", "author_url": "", "post_date": "03/28/2011 18:16:15", "content": "I used to be notified by email whenever a message is posted on this forum, but it is not the case anymore.
", "votes": null, "replies": [] }, { "id": 2093, "author_name": "jeffmoser", "author_url": "", "post_date": "03/28/2011 23:04:44", "content": "There was a problem in how the public leaderboard was being scored (only very few cells were being checked). The problem should be fixed now. \r\n\r\nIn order to ensure that all calculations are the same, I reran the scoring with the new code for all submissions (including those made before Kaggle 2.0 went live). \r\n\r\nIt is possible that the public leaderboard is now based on slightly different values that were not previously used before, however it is still a ~30% subset of the data. \r\n\r\nAgain, I want to reiterate that all submissions were re-scored using this new algorithm so that everything should be consistent.\r\n\r\nPlease let me know if you have further issues. Sorry about the hassle this bug caused.", "votes": null, "replies": [] }, { "id": 2503, "author_name": "gregs1", "author_url": "", "post_date": "04/08/2011 11:41:13", "content": "Sorry for such a basic question, but how does one get to the leaderboard?\r\nThe rules state that it won't be available until May, yet from the discussion it appears to be avialable.\r\nThanks.", "votes": null, "replies": [] }, { "id": 2509, "author_name": "jeffmoser", "author_url": "", "post_date": "04/08/2011 13:47:54", "content": "gschmidt: You're referring to the Heritage Health Prize. This is a different competition altogether.", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2085": "", "2086": "", "2087": "", "2088": "", "2089": "", "2090": "", "2093": "", "2503": "", "2509": "" }, "source": "meta" }