Just for curiosity, mine is 0.534 for now
", "messages": [ { "id": 767383, "postDate": "2020-03-09T15:00:06.313Z", "content": "Just for curiosity, mine is 0.534 for now
", "rawMarkdown": "Just for curiosity, mine is 0.534 for now", "votes": 10 }, { "id": 767668, "postDate": "2020-03-10T01:24:35.513Z", "content": "Same as Ryan, my posted score on the leaderboard had an inadvertent leak, but my \"real\" best score to date is 0.51010, without any gambling/manual overriding or clipping.
", "rawMarkdown": "Same as Ryan, my posted score on the leaderboard had an inadvertent leak, but my \"real\" best score to date is 0.51010, without any gambling/manual overriding or clipping.", "votes": 5, "replies": [ { "id": 768069, "postDate": "2020-03-10T12:27:50.970Z", "content": "Can I just ask what what is your cross validation scheme?
", "rawMarkdown": "Can I just ask what what is your cross validation scheme?" }, { "id": 768502, "postDate": "2020-03-10T21:53:46.527Z", "content": "Model was trained using 2003-2014 data and then applied to 2015-19 data... I don't have many features in the model so feature selection was somewhat manual, but some techniques were still applied.
", "rawMarkdown": "Model was trained using 2003-2014 data and then applied to 2015-19 data... I don't have many features in the model so feature selection was somewhat manual, but some techniques were still applied.", "votes": 1 }, { "id": 770841, "postDate": "2020-03-13T13:22:30.423Z", "content": "Great score. Can you explain how you train feature-less? any related kernel/model?
", "rawMarkdown": "Great score. Can you explain how you train feature-less? any related kernel/model?" } ] }, { "id": 770347, "postDate": "2020-03-12T20:50:33.150Z", "content": "0.46 roughly, without overrides.
", "rawMarkdown": "0.46 roughly, without overrides. ", "votes": 3, "replies": [ { "id": 770357, "postDate": "2020-03-12T21:00:22.933Z", "content": "I was going to call you out on this - but seeing how well you did last year, I believe you.\nVery nice.\nAlso, if you're on chess.com, my handle is updog and I'm usually 2000 blitz.
", "rawMarkdown": "I was going to call you out on this - but seeing how well you did last year, I believe you.\nVery nice.\nAlso, if you're on chess.com, my handle is updog and I'm usually 2000 blitz. ", "votes": 1 }, { "id": 770360, "postDate": "2020-03-12T21:01:58.367Z", "content": "Great score
", "rawMarkdown": "Great score" }, { "id": 770836, "postDate": "2020-03-13T13:20:16.530Z", "content": "Great score. Do you have any suggestions for us? any magic?
", "rawMarkdown": "Great score. Do you have any suggestions for us? any magic?" } ] }, { "id": 770375, "postDate": "2020-03-12T21:23:39.647Z", "content": "0.507753, no re-weighting, no post-tourney KenPom data.
\n\nWith post-tourney KenPom, I was getting around ~0.401 for Stage1, but I'm pretty sure that was taking advantage of the leak. Using KenPom data from the wayback machine gave me a significant drop in accuracy for every year except last year. I was still working to figure out why performance in last year's competition wasn't severely impacted by the leak. Not sure if I will have the motivation to keep digging, now. 😔
", "rawMarkdown": "0.507753, no re-weighting, no post-tourney KenPom data.\n\nWith post-tourney KenPom, I was getting around ~0.401 for Stage1, but I'm pretty sure that was taking advantage of the leak. Using KenPom data from the wayback machine gave me a significant drop in accuracy for every year except last year. I was still working to figure out why performance in last year's competition wasn't severely impacted by the leak. Not sure if I will have the motivation to keep digging, now. 😔 ", "votes": 1 }, { "id": 769441, "postDate": "2020-03-11T22:52:39.303Z", "content": "0.50874 with clipping
", "rawMarkdown": "0.50874 with clipping", "votes": 1, "replies": [ { "id": 770837, "postDate": "2020-03-13T13:20:46.993Z", "content": "Good job. What was the score before clipping?
", "rawMarkdown": "Good job. What was the score before clipping?" } ] }, { "id": 769172, "postDate": "2020-03-11T15:49:31.263Z", "content": "These are my validation scores:\n- 2015 0.491124\n- 2016 0.526561\n- 2017 0.495399\n- 2018 0.584738\n- 2019 0.477078\n- Mean 0.514980\n- Public LB 0.51051
\n\nI train on 2012-201N to predict 201N+1. Still not getting anywhere near the previous years LB scores
", "rawMarkdown": "These are my validation scores:\n- 2015 0.491124\n- 2016 0.526561\n- 2017 0.495399\n- 2018 0.584738\n- 2019 0.477078\n- Mean 0.514980\n- Public LB 0.51051\n\nI train on 2012-201N to predict 201N+1. Still not getting anywhere near the previous years LB scores\n", "votes": 2 }, { "id": 767484, "postDate": "2020-03-09T17:31:45.647Z", "content": "Unfortunately, my best posted score on the leaderboard had a data leak before I had a complete grasp on how I was breaking out my test/valid/train sets. Wish we could delete our own :/
\n\nThe average of my two entry methods (without leaks) is sitting right around 0.520.
", "rawMarkdown": "Unfortunately, my best posted score on the leaderboard had a data leak before I had a complete grasp on how I was breaking out my test/valid/train sets. Wish we could delete our own :/\n\nThe average of my two entry methods (without leaks) is sitting right around 0.520.", "votes": 2 }, { "id": 767415, "postDate": "2020-03-09T15:47:28.170Z", "content": "I've been waiting for someone to create this discussion. I'll be really curious to read everyone's answer and see who's making progress. I've gotten as low as .5297 (which includes 2014 because we have a baseline for it) but there's still time to get it lower. It's probably a little lower for years 15-19. My model does well in years when favorites do well but isn't generalizing as well as I'd hoped...yet.
", "rawMarkdown": "I've been waiting for someone to create this discussion. I'll be really curious to read everyone's answer and see who's making progress. I've gotten as low as .5297 (which includes 2014 because we have a baseline for it) but there's still time to get it lower. It's probably a little lower for years 15-19. My model does well in years when favorites do well but isn't generalizing as well as I'd hoped...yet. ", "votes": 2 }, { "id": 768270, "postDate": "2020-03-10T15:43:25.647Z", "content": "LB scores with three completely different approach: 0.556,0.552,0.530 without any gambling/tweaks.
", "rawMarkdown": "LB scores with three completely different approach: 0.556,0.552,0.530 without any gambling/tweaks. \n" }, { "id": 769328, "postDate": "2020-03-11T19:21:07.353Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 767668, "author_name": "JasonZivkovic", "author_url": "", "post_date": "2020-03-10T01:24:35.513000", "content": "Same as Ryan, my posted score on the leaderboard had an inadvertent leak, but my \"real\" best score to date is 0.51010, without any gambling/manual overriding or clipping.
", "votes": 5, "replies": [ { "id": 768069, "author_name": "Rok Novosel", "author_url": "", "post_date": "2020-03-10T12:27:50.970000", "content": "Can I just ask what what is your cross validation scheme?
", "votes": 0, "replies": [] }, { "id": 768502, "author_name": "JasonZivkovic", "author_url": "", "post_date": "2020-03-10T21:53:46.527000", "content": "Model was trained using 2003-2014 data and then applied to 2015-19 data... I don't have many features in the model so feature selection was somewhat manual, but some techniques were still applied.
", "votes": 1, "replies": [] }, { "id": 770841, "author_name": "", "author_url": "", "post_date": "2020-03-13T13:22:30.423000", "content": "Great score. Can you explain how you train feature-less? any related kernel/model?
", "votes": 0, "replies": [] } ] }, { "id": 770347, "author_name": "Marcus Harvey", "author_url": "", "post_date": "2020-03-12T20:50:33.150000", "content": "0.46 roughly, without overrides.
", "votes": 3, "replies": [ { "id": 770357, "author_name": "Ben Tesch", "author_url": "", "post_date": "2020-03-12T21:00:22.933000", "content": "I was going to call you out on this - but seeing how well you did last year, I believe you.\nVery nice.\nAlso, if you're on chess.com, my handle is updog and I'm usually 2000 blitz.
", "votes": 1, "replies": [] }, { "id": 770360, "author_name": "Jie Lu", "author_url": "", "post_date": "2020-03-12T21:01:58.367000", "content": "Great score
", "votes": 0, "replies": [] }, { "id": 770836, "author_name": "", "author_url": "", "post_date": "2020-03-13T13:20:16.530000", "content": "Great score. Do you have any suggestions for us? any magic?
", "votes": 0, "replies": [] } ] }, { "id": 770375, "author_name": "Josh Stone", "author_url": "", "post_date": "2020-03-12T21:23:39.647000", "content": "0.507753, no re-weighting, no post-tourney KenPom data.
\n\nWith post-tourney KenPom, I was getting around ~0.401 for Stage1, but I'm pretty sure that was taking advantage of the leak. Using KenPom data from the wayback machine gave me a significant drop in accuracy for every year except last year. I was still working to figure out why performance in last year's competition wasn't severely impacted by the leak. Not sure if I will have the motivation to keep digging, now. 😔
", "votes": 1, "replies": [] }, { "id": 769441, "author_name": "Lu Bin Liu", "author_url": "", "post_date": "2020-03-11T22:52:39.303000", "content": "0.50874 with clipping
", "votes": 1, "replies": [ { "id": 770837, "author_name": "", "author_url": "", "post_date": "2020-03-13T13:20:46.993000", "content": "Good job. What was the score before clipping?
", "votes": 0, "replies": [] } ] }, { "id": 769172, "author_name": "datasaurus", "author_url": "", "post_date": "2020-03-11T15:49:31.263000", "content": "These are my validation scores:\n- 2015 0.491124\n- 2016 0.526561\n- 2017 0.495399\n- 2018 0.584738\n- 2019 0.477078\n- Mean 0.514980\n- Public LB 0.51051
\n\nI train on 2012-201N to predict 201N+1. Still not getting anywhere near the previous years LB scores
", "votes": 2, "replies": [] }, { "id": 767484, "author_name": "Ryan Armstrong", "author_url": "", "post_date": "2020-03-09T17:31:45.647000", "content": "Unfortunately, my best posted score on the leaderboard had a data leak before I had a complete grasp on how I was breaking out my test/valid/train sets. Wish we could delete our own :/
\n\nThe average of my two entry methods (without leaks) is sitting right around 0.520.
", "votes": 2, "replies": [] }, { "id": 767415, "author_name": "Ben Tesch", "author_url": "", "post_date": "2020-03-09T15:47:28.170000", "content": "I've been waiting for someone to create this discussion. I'll be really curious to read everyone's answer and see who's making progress. I've gotten as low as .5297 (which includes 2014 because we have a baseline for it) but there's still time to get it lower. It's probably a little lower for years 15-19. My model does well in years when favorites do well but isn't generalizing as well as I'd hoped...yet.
", "votes": 2, "replies": [] }, { "id": 768270, "author_name": "Kumar Shubham", "author_url": "", "post_date": "2020-03-10T15:43:25.647000", "content": "LB scores with three completely different approach: 0.556,0.552,0.530 without any gambling/tweaks.
", "votes": 0, "replies": [] }, { "id": 769328, "author_name": "", "author_url": "", "post_date": "2020-03-11T19:21:07.353000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "767383": "Just for curiosity, mine is 0.534 for now", "767668": "Same as Ryan, my posted score on the leaderboard had an inadvertent leak, but my \"real\" best score to date is 0.51010, without any gambling/manual overriding or clipping.", "770347": "0.46 roughly, without overrides. ", "770375": "0.507753, no re-weighting, no post-tourney KenPom data.\n\nWith post-tourney KenPom, I was getting around ~0.401 for Stage1, but I'm pretty sure that was taking advantage of the leak. Using KenPom data from the wayback machine gave me a significant drop in accuracy for every year except last year. I was still working to figure out why performance in last year's competition wasn't severely impacted by the leak. Not sure if I will have the motivation to keep digging, now. 😔 ", "769441": "0.50874 with clipping", "769172": "These are my validation scores:\n- 2015 0.491124\n- 2016 0.526561\n- 2017 0.495399\n- 2018 0.584738\n- 2019 0.477078\n- Mean 0.514980\n- Public LB 0.51051\n\nI train on 2012-201N to predict 201N+1. Still not getting anywhere near the previous years LB scores\n", "767484": "Unfortunately, my best posted score on the leaderboard had a data leak before I had a complete grasp on how I was breaking out my test/valid/train sets. Wish we could delete our own :/\n\nThe average of my two entry methods (without leaks) is sitting right around 0.520.", "767415": "I've been waiting for someone to create this discussion. I'll be really curious to read everyone's answer and see who's making progress. I've gotten as low as .5297 (which includes 2014 because we have a baseline for it) but there's still time to get it lower. It's probably a little lower for years 15-19. My model does well in years when favorites do well but isn't generalizing as well as I'd hoped...yet. ", "768270": "LB scores with three completely different approach: 0.556,0.552,0.530 without any gambling/tweaks. \n", "769328": "" } }