When I read the overview to see this is the 7th time this competition is going on my first thought was that I have to first get familiar with all the past 6 competitions before I even look at the data.
\n\nHow should we deal with this one? As another milestone in a yearly competition that is the same, or as a new one every year?
\n\nWill the winners of this year's competition even look at the data? Or just move on from where they stopped last year..?
", "messages": [ { "id": "747208", "postDate": "02/16/2020 05:47:08", "content": "When I read the overview to see this is the 7th time this competition is going on my first thought was that I have to first get familiar with all the past 6 competitions before I even look at the data.
\n\nHow should we deal with this one? As another milestone in a yearly competition that is the same, or as a new one every year?
\n\nWill the winners of this year's competition even look at the data? Or just move on from where they stopped last year..?
", "rawMarkdown": "When I read the overview to see this is the 7th time this competition is going on my first thought was that I have to first get familiar with all the past 6 competitions before I even look at the data.\n\nHow should we deal with this one? As another milestone in a yearly competition that is the same, or as a new one every year?\n\nWill the winners of this year's competition even look at the data? Or just move on from where they stopped last year..?", "votes": null }, { "id": "747601", "postDate": "02/16/2020 16:21:59", "content": "Given that this is a tournament with a lot of inherent uncertainty to the outcomes of most games, the winners of any year could have been different had the outcome of any one game changed. I don't have the winner and runner up's results for every year but I will show how much luck matters...\nLast year's winner won with a .41477 and second place took .42012. These numbers are an avg log loss.\nTotal log loss for each would have been:\n1st place .41477x63 games=26.13051\n2nd place .42012x63 games=26.46756\nThe difference is only .33705 total log loss.
\n\nIf the 1st place winner had one game at 55% confidence while the second place winner had it at 45% confidence and the 1st place winner is right, it would have been a .5978 log loss while second place would have had .7985. Imagine that this game was won by one point on a last second shot. What would have happened had that shot missed? The results would have been flipped. Now first place takes a .7985 log loss while second place takes .5978.\nThe end result changes to:\n1st place moves to second with 26.33118 total points (26.13051 + .7985 - .5978) .417575 avg\n2nd place moves to first with 26.26689 total points (26.46756 +.5978 - .7985) .417315 avg
\n\n*The caveat is now a different team advances which changes what other games are predicted unless this is the final game. That being said, it speaks even more to the point that the outcome of even one game can have a huge impact on the leaderboard.
\n\nThe difference of a model returning a 55% confident prediction vs a 45% confident prediction is minimal. \nMy personal opinion is that there's so much luck involved, most any model has some chance.\nThe advice I would give is to make sure your confidence in any prediction is never > 97%-98%. This threshold is debatable.
", "rawMarkdown": "Given that this is a tournament with a lot of inherent uncertainty to the outcomes of most games, the winners of any year could have been different had the outcome of any one game changed. I don't have the winner and runner up's results for every year but I will show how much luck matters...\nLast year's winner won with a .41477 and second place took .42012. These numbers are an avg log loss.\nTotal log loss for each would have been:\n1st place .41477x63 games=26.13051\n2nd place .42012x63 games=26.46756\nThe difference is only .33705 total log loss.\n\nIf the 1st place winner had one game at 55% confidence while the second place winner had it at 45% confidence and the 1st place winner is right, it would have been a .5978 log loss while second place would have had .7985. Imagine that this game was won by one point on a last second shot. What would have happened had that shot missed? The results would have been flipped. Now first place takes a .7985 log loss while second place takes .5978.\nThe end result changes to:\n1st place moves to second with 26.33118 total points (26.13051 + .7985 - .5978) .417575 avg\n2nd place moves to first with 26.26689 total points (26.46756 +.5978 - .7985) .417315 avg \n\n*The caveat is now a different team advances which changes what other games are predicted unless this is the final game. That being said, it speaks even more to the point that the outcome of even one game can have a huge impact on the leaderboard.\n\nThe difference of a model returning a 55% confident prediction vs a 45% confident prediction is minimal. \nMy personal opinion is that there's so much luck involved, most any model has some chance.\nThe advice I would give is to make sure your confidence in any prediction is never > 97%-98%. This threshold is debatable.", "votes": null }, { "id": "748057", "postDate": "02/17/2020 06:44:58", "content": "Thank you Ben,\nDo you think that this 1st place team from last year will need to work hard to get close results this year, or will they have no problem doing so because they own a very good model from last year?
", "rawMarkdown": "Thank you Ben,\nDo you think that this 1st place team from last year will need to work hard to get close results this year, or will they have no problem doing so because they own a very good model from last year?", "votes": null }, { "id": "748576", "postDate": "02/17/2020 17:23:35", "content": "For the 2019 tournament:\n-Five games went to overtime. \n-Another Eight games were won by one possession (1-3 points).
\nThat's 13 games that were essentially a coin flip for the outcome. Yet another 10 games were won by only two possessions (4-6 points). I'm guessing that other games had lead changes in the last few minutes as well. \nHad any of those games gone differently, there's a good chance there would have been a different winner. The difference between a 65% prediction going for or against a submission is huge. It seems to me that a lot of favorable outcomes have to happen for the winner to win. \nIn my opinion, the only two reasons to use an identical model to last year is because you either don't have time to improve it or can't find a way to do so.
It would be really interesting to know how well a submission file would do if it had the most optimal outcome vs the field.
", "rawMarkdown": "For the 2019 tournament:\n-Five games went to overtime. \n-Another Eight games were won by one possession (1-3 points). \nThat's 13 games that were essentially a coin flip for the outcome. Yet another 10 games were won by only two possessions (4-6 points). I'm guessing that other games had lead changes in the last few minutes as well. \nHad any of those games gone differently, there's a good chance there would have been a different winner. The difference between a 65% prediction going for or against a submission is huge. It seems to me that a lot of favorable outcomes have to happen for the winner to win. \nIn my opinion, the only two reasons to use an identical model to last year is because you either don't have time to improve it or can't find a way to do so. \n\nIt would be really interesting to know how well a submission file would do if it had the most optimal outcome vs the field.", "votes": null }, { "id": "749494", "postDate": "02/18/2020 18:26:53", "content": "Maybe you should enter the analytics challenge if you can figure out what makes these games as wild as they are!
", "rawMarkdown": "Maybe you should enter the [analytics challenge](https://www.kaggle.com/c/march-madness-analytics-2020) if you can figure out what makes these games as wild as they are!", "votes": null }, { "id": "749539", "postDate": "02/18/2020 19:03:15", "content": "I've been considering it. I think there's a lot of very interesting ideas worth exploring. Maybe I just need to watch my Utes play a game or two to get some ideas.
", "rawMarkdown": "I've been considering it. I think there's a lot of very interesting ideas worth exploring. Maybe I just need to watch my Utes play a game or two to get some ideas.", "votes": null } ], "comments": [ { "id": 747601, "author_name": "btesch", "author_url": "", "post_date": "02/16/2020 16:21:59", "content": "Given that this is a tournament with a lot of inherent uncertainty to the outcomes of most games, the winners of any year could have been different had the outcome of any one game changed. I don't have the winner and runner up's results for every year but I will show how much luck matters...\nLast year's winner won with a .41477 and second place took .42012. These numbers are an avg log loss.\nTotal log loss for each would have been:\n1st place .41477x63 games=26.13051\n2nd place .42012x63 games=26.46756\nThe difference is only .33705 total log loss.
\n\nIf the 1st place winner had one game at 55% confidence while the second place winner had it at 45% confidence and the 1st place winner is right, it would have been a .5978 log loss while second place would have had .7985. Imagine that this game was won by one point on a last second shot. What would have happened had that shot missed? The results would have been flipped. Now first place takes a .7985 log loss while second place takes .5978.\nThe end result changes to:\n1st place moves to second with 26.33118 total points (26.13051 + .7985 - .5978) .417575 avg\n2nd place moves to first with 26.26689 total points (26.46756 +.5978 - .7985) .417315 avg
\n\n*The caveat is now a different team advances which changes what other games are predicted unless this is the final game. That being said, it speaks even more to the point that the outcome of even one game can have a huge impact on the leaderboard.
\n\nThe difference of a model returning a 55% confident prediction vs a 45% confident prediction is minimal. \nMy personal opinion is that there's so much luck involved, most any model has some chance.\nThe advice I would give is to make sure your confidence in any prediction is never > 97%-98%. This threshold is debatable.
", "votes": null, "replies": [ { "id": 748057, "author_name": "avivlevi815", "author_url": "", "post_date": "02/17/2020 06:44:58", "content": "Thank you Ben,\nDo you think that this 1st place team from last year will need to work hard to get close results this year, or will they have no problem doing so because they own a very good model from last year?
", "votes": null, "replies": [] }, { "id": 748576, "author_name": "btesch", "author_url": "", "post_date": "02/17/2020 17:23:35", "content": "For the 2019 tournament:\n-Five games went to overtime. \n-Another Eight games were won by one possession (1-3 points).
\nThat's 13 games that were essentially a coin flip for the outcome. Yet another 10 games were won by only two possessions (4-6 points). I'm guessing that other games had lead changes in the last few minutes as well. \nHad any of those games gone differently, there's a good chance there would have been a different winner. The difference between a 65% prediction going for or against a submission is huge. It seems to me that a lot of favorable outcomes have to happen for the winner to win. \nIn my opinion, the only two reasons to use an identical model to last year is because you either don't have time to improve it or can't find a way to do so.
It would be really interesting to know how well a submission file would do if it had the most optimal outcome vs the field.
", "votes": null, "replies": [] }, { "id": 749494, "author_name": "addisonhoward", "author_url": "", "post_date": "02/18/2020 18:26:53", "content": "Maybe you should enter the analytics challenge if you can figure out what makes these games as wild as they are!
", "votes": null, "replies": [] }, { "id": 749539, "author_name": "btesch", "author_url": "", "post_date": "02/18/2020 19:03:15", "content": "I've been considering it. I think there's a lot of very interesting ideas worth exploring. Maybe I just need to watch my Utes play a game or two to get some ideas.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "747208": "When I read the overview to see this is the 7th time this competition is going on my first thought was that I have to first get familiar with all the past 6 competitions before I even look at the data.\n\nHow should we deal with this one? As another milestone in a yearly competition that is the same, or as a new one every year?\n\nWill the winners of this year's competition even look at the data? Or just move on from where they stopped last year..?", "747601": "Given that this is a tournament with a lot of inherent uncertainty to the outcomes of most games, the winners of any year could have been different had the outcome of any one game changed. I don't have the winner and runner up's results for every year but I will show how much luck matters...\nLast year's winner won with a .41477 and second place took .42012. These numbers are an avg log loss.\nTotal log loss for each would have been:\n1st place .41477x63 games=26.13051\n2nd place .42012x63 games=26.46756\nThe difference is only .33705 total log loss.\n\nIf the 1st place winner had one game at 55% confidence while the second place winner had it at 45% confidence and the 1st place winner is right, it would have been a .5978 log loss while second place would have had .7985. Imagine that this game was won by one point on a last second shot. What would have happened had that shot missed? The results would have been flipped. Now first place takes a .7985 log loss while second place takes .5978.\nThe end result changes to:\n1st place moves to second with 26.33118 total points (26.13051 + .7985 - .5978) .417575 avg\n2nd place moves to first with 26.26689 total points (26.46756 +.5978 - .7985) .417315 avg \n\n*The caveat is now a different team advances which changes what other games are predicted unless this is the final game. That being said, it speaks even more to the point that the outcome of even one game can have a huge impact on the leaderboard.\n\nThe difference of a model returning a 55% confident prediction vs a 45% confident prediction is minimal. \nMy personal opinion is that there's so much luck involved, most any model has some chance.\nThe advice I would give is to make sure your confidence in any prediction is never > 97%-98%. This threshold is debatable.", "748057": "Thank you Ben,\nDo you think that this 1st place team from last year will need to work hard to get close results this year, or will they have no problem doing so because they own a very good model from last year?", "748576": "For the 2019 tournament:\n-Five games went to overtime. \n-Another Eight games were won by one possession (1-3 points). \nThat's 13 games that were essentially a coin flip for the outcome. Yet another 10 games were won by only two possessions (4-6 points). I'm guessing that other games had lead changes in the last few minutes as well. \nHad any of those games gone differently, there's a good chance there would have been a different winner. The difference between a 65% prediction going for or against a submission is huge. It seems to me that a lot of favorable outcomes have to happen for the winner to win. \nIn my opinion, the only two reasons to use an identical model to last year is because you either don't have time to improve it or can't find a way to do so. \n\nIt would be really interesting to know how well a submission file would do if it had the most optimal outcome vs the field.", "749494": "Maybe you should enter the [analytics challenge](https://www.kaggle.com/c/march-madness-analytics-2020) if you can figure out what makes these games as wild as they are!", "749539": "I've been considering it. I think there's a lot of very interesting ideas worth exploring. Maybe I just need to watch my Utes play a game or two to get some ideas." }, "source": "meta" }