And finally, here is the prelim play-by-play data. We have a different provider for play-by-play this year, compared to last year. For the most part, the data seems nice and compatible, but this did delay the release of the files this year. Differences I noticed are:\n (1) Player names tend to be formatted like FIRST_LAST rather than LAST_FIRST\n (2) There were a couple of new types of two-point-shot this year: hook (which I mapped into jump) and alleyoop (which I mapped into dunk). If we'd had consistent older data (from previous years) with this information, I might have been tempted to keep them as separate types of shot, but at this late time that seemed unhelpful.\n (3) In previous years there were only three types of rebounds: offensive, defensive, and deadball. This year there was a fourth type of rebound, which was just \"rebound\". I mapped that into deadball since the proportions seemed right.
\n\nAlso, please remember that the EventID's and PlayerID's are temporary in this release. The numerical EventID's and PlayerID's are assigned based on an overall sort. If new player spellings are introduced near the end of the year, those players might appear in the middle of the final sorted list for 2019, and would change the overall sort order. So please don't assume that players having a particular PlayerID, or events having a particular EventID, will retain those same ID's into future releases of these files.
", "rawMarkdown": "And finally, here is the prelim play-by-play data. We have a different provider for play-by-play this year, compared to last year. For the most part, the data seems nice and compatible, but this did delay the release of the files this year. Differences I noticed are:\n (1) Player names tend to be formatted like FIRST_LAST rather than LAST_FIRST\n (2) There were a couple of new types of two-point-shot this year: hook (which I mapped into jump) and alleyoop (which I mapped into dunk). If we'd had consistent older data (from previous years) with this information, I might have been tempted to keep them as separate types of shot, but at this late time that seemed unhelpful.\n (3) In previous years there were only three types of rebounds: offensive, defensive, and deadball. This year there was a fourth type of rebound, which was just \"rebound\". I mapped that into deadball since the proportions seemed right.\n\nAlso, please remember that the EventID's and PlayerID's are temporary in this release. The numerical EventID's and PlayerID's are assigned based on an overall sort. If new player spellings are introduced near the end of the year, those players might appear in the middle of the final sorted list for 2019, and would change the overall sort order. So please don't assume that players having a particular PlayerID, or events having a particular EventID, will retain those same ID's into future releases of these files.\n", "votes": 1, "replies": [ { "id": 492234, "postDate": "2019-03-17T00:20:49.343Z", "content": "Thanks. I agree with your decisions on the mapping, seems like a reasonable compromise.
\n\nRegarding player IDs, have you considered hashing them? Something like name + team + number should be unique.
\n\nOr better, have you considered pulling the data from NCAA, which seems to have done a good job tracking players across seasons? That would certainly make the data more valuable, as models would be able to make better inferences using previous seasons.
", "rawMarkdown": "Thanks. I agree with your decisions on the mapping, seems like a reasonable compromise. \n\nRegarding player IDs, have you considered hashing them? Something like name + team + number should be unique. \n\nOr better, have you considered pulling the data from NCAA, which seems to have done a good job tracking players across seasons? That would certainly make the data more valuable, as models would be able to make better inferences using previous seasons." } ] }, { "id": 491666, "postDate": "2019-03-16T00:55:52.643Z", "content": "When is the final stage 2 data released? After March 17 Selection Sunday, or is there an updated official data set after the First Four games ending evening of Weds March 20?
", "rawMarkdown": "When is the final stage 2 data released? After March 17 Selection Sunday, or is there an updated official data set after the First Four games ending evening of Weds March 20? ", "votes": 1, "replies": [ { "id": 491673, "postDate": "2019-03-16T01:23:59.803Z", "content": "After the selections and pairings are announced on Sunday, there is no further game data you will be receiving, and so the bulk of Stage 2 data will be released as soon as possible on Sunday evening. However, the Massey Ordinals take a few days to completely come in, so we will be releasing updated versions of that file (only) over the course of the next few days, from Sunday up through Wednesday or so.
", "rawMarkdown": "After the selections and pairings are announced on Sunday, there is no further game data you will be receiving, and so the bulk of Stage 2 data will be released as soon as possible on Sunday evening. However, the Massey Ordinals take a few days to completely come in, so we will be releasing updated versions of that file (only) over the course of the next few days, from Sunday up through Wednesday or so.", "votes": 1 } ] }, { "id": 491025, "postDate": "2019-03-15T04:40:27.483Z", "content": "Still running into some delays assembling the data because I don't have access to everything I thought I did. Here are the Massey Ordinals and Regular Season Compact Results, cumulative up through last night, in case that's all you are looking for. I will soon be doing a full post with updated versions of all the \"prelim\" files, including a fake bracket so you can see what the submission format for Stage 2 will look like.
\n\nEvidently there is value on my end as well, to go through a preliminary run-through of the data refresh process! Apologies to those who were counting on the prelim data coming in sooner. But I am doing what I can to resolve this ASAP.
", "rawMarkdown": "Still running into some delays assembling the data because I don't have access to everything I thought I did. Here are the Massey Ordinals and Regular Season Compact Results, cumulative up through last night, in case that's all you are looking for. I will soon be doing a full post with updated versions of all the \"prelim\" files, including a fake bracket so you can see what the submission format for Stage 2 will look like.\n\nEvidently there is value on my end as well, to go through a preliminary run-through of the data refresh process! Apologies to those who were counting on the prelim data coming in sooner. But I am doing what I can to resolve this ASAP.", "votes": 1, "replies": [ { "id": 491386, "postDate": "2019-03-15T15:22:26.913Z", "content": "Great, thank you!
", "rawMarkdown": "Great, thank you!", "votes": 1 } ] }, { "id": 486422, "postDate": "2019-03-08T18:41:42.980Z", "content": "@Jeff: any vague guess as to when the Stage 2 Preview will be out? =D
", "rawMarkdown": "@Jeff: any vague guess as to when the Stage 2 Preview will be out? =D", "votes": 1, "replies": [ { "id": 486477, "postDate": "2019-03-08T20:46:11.183Z", "content": "@Zach, probably Wednesday - give or take a day.
", "rawMarkdown": "@Zach, probably Wednesday - give or take a day.", "votes": 2 }, { "id": 489289, "postDate": "2019-03-13T18:25:29.947Z", "content": "@Jeff How's it going? =D
", "rawMarkdown": "@Jeff How's it going? =D", "votes": 1 }, { "id": 489669, "postDate": "2019-03-14T02:22:13.797Z", "content": "Sometime in the next 24 hours, it looks like
", "rawMarkdown": "Sometime in the next 24 hours, it looks like", "votes": 1 }, { "id": 490519, "postDate": "2019-03-14T16:22:32.013Z", "rawMarkdown": "", "votes": 4, "isDeleted": true } ] }, { "id": 472716, "postDate": "2019-02-16T14:30:40.447Z", "rawMarkdown": "", "votes": 1 }, { "id": 491872, "postDate": "2019-03-16T09:58:34.007Z", "content": "Here are the regular season detailed results, and the conference tourney games listing, as of a few days ago. When we release Stage 2, everything will be brought up through the end of the regular season.
", "rawMarkdown": "Here are the regular season detailed results, and the conference tourney games listing, as of a few days ago. When we release Stage 2, everything will be brought up through the end of the regular season.", "votes": 2 }, { "id": 491776, "postDate": "2019-03-16T05:54:32.873Z", "content": "Here are some more preliminary files. Again I apologize for the raggedness of this, but a data feed that I thought was available this year, turns out not to be, and so a bunch of this had to be done in an unexpected way, and more manually than planned. I will still be releasing it all in one post but I am also making some of these files available right away for people, if you can use them. I still need to do some more work tonight in order to get RegularSeasonDetailedResults and ConferenceTourneyGames and play-by-play ready, but it will happen tonight.
", "rawMarkdown": "Here are some more preliminary files. Again I apologize for the raggedness of this, but a data feed that I thought was available this year, turns out not to be, and so a bunch of this had to be done in an unexpected way, and more manually than planned. I will still be releasing it all in one post but I am also making some of these files available right away for people, if you can use them. I still need to do some more work tonight in order to get RegularSeasonDetailedResults and ConferenceTourneyGames and play-by-play ready, but it will happen tonight.", "votes": 2, "replies": [ { "id": 491795, "postDate": "2019-03-16T06:41:02.507Z", "content": "Hi Jeff
\n\nThanks for your great efforts. It would be better to have RegularSeasonDetailedResults.csv of 2019 ASAP for competitiors.
", "rawMarkdown": "Hi Jeff\n\nThanks for your great efforts. It would be better to have RegularSeasonDetailedResults.csv of 2019 ASAP for competitiors." } ] }, { "id": 472920, "postDate": "2019-02-16T22:26:18.050Z", "content": "We typically try to provide a preview of Stage 2 data before Selection Sunday comes along, by using a hypothetical bracket a couple of weeks in advance, so that you can see what Stage 2 data will look like (including the real partial data from the current season). This lets you make sure your code will work on Stage 2 data, which will be a little different (most notably, it will only be one season's worth of tournament predictions that you submit) and also gives you a sneak peak at the current season's data. However, the official Stage 2 release (including complete data from the current regular season) will not be available until something like 12-24 hours after the tournament pairings are announced on Selection Sunday.
", "rawMarkdown": "We typically try to provide a preview of Stage 2 data before Selection Sunday comes along, by using a hypothetical bracket a couple of weeks in advance, so that you can see what Stage 2 data will look like (including the real partial data from the current season). This lets you make sure your code will work on Stage 2 data, which will be a little different (most notably, it will only be one season's worth of tournament predictions that you submit) and also gives you a sneak peak at the current season's data. However, the official Stage 2 release (including complete data from the current regular season) will not be available until something like 12-24 hours after the tournament pairings are announced on Selection Sunday.", "votes": 2 }, { "id": 492633, "postDate": "2019-03-17T14:30:11.920Z", "content": "Will there be an updated team spellings file this year?
", "rawMarkdown": "Will there be an updated team spellings file this year?", "replies": [ { "id": 492731, "postDate": "2019-03-17T17:24:18.800Z", "content": "I don't think I had any new spellings since Stage 1, so at this point, no.
", "rawMarkdown": "I don't think I had any new spellings since Stage 1, so at this point, no.", "votes": 1 } ] }, { "id": 492588, "postDate": "2019-03-17T13:36:22.177Z", "content": "Thank you Jeff! This competition is always a bunch of hard work all at once!
", "rawMarkdown": "Thank you Jeff! This competition is always a bunch of hard work all at once!" }, { "id": 472733, "postDate": "2019-02-16T14:59:50.757Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 492156, "author_name": "Jeff Sonas", "author_url": "", "post_date": "2019-03-16T19:15:56.410000", "content": "And finally, here is the prelim play-by-play data. We have a different provider for play-by-play this year, compared to last year. For the most part, the data seems nice and compatible, but this did delay the release of the files this year. Differences I noticed are:\n (1) Player names tend to be formatted like FIRST_LAST rather than LAST_FIRST\n (2) There were a couple of new types of two-point-shot this year: hook (which I mapped into jump) and alleyoop (which I mapped into dunk). If we'd had consistent older data (from previous years) with this information, I might have been tempted to keep them as separate types of shot, but at this late time that seemed unhelpful.\n (3) In previous years there were only three types of rebounds: offensive, defensive, and deadball. This year there was a fourth type of rebound, which was just \"rebound\". I mapped that into deadball since the proportions seemed right.
\n\nAlso, please remember that the EventID's and PlayerID's are temporary in this release. The numerical EventID's and PlayerID's are assigned based on an overall sort. If new player spellings are introduced near the end of the year, those players might appear in the middle of the final sorted list for 2019, and would change the overall sort order. So please don't assume that players having a particular PlayerID, or events having a particular EventID, will retain those same ID's into future releases of these files.
", "votes": 1, "replies": [ { "id": 492234, "author_name": "Algoterranean", "author_url": "", "post_date": "2019-03-17T00:20:49.343000", "content": "Thanks. I agree with your decisions on the mapping, seems like a reasonable compromise.
\n\nRegarding player IDs, have you considered hashing them? Something like name + team + number should be unique.
\n\nOr better, have you considered pulling the data from NCAA, which seems to have done a good job tracking players across seasons? That would certainly make the data more valuable, as models would be able to make better inferences using previous seasons.
", "votes": 0, "replies": [] } ] }, { "id": 491666, "author_name": "WakkaDojo", "author_url": "", "post_date": "2019-03-16T00:55:52.643000", "content": "When is the final stage 2 data released? After March 17 Selection Sunday, or is there an updated official data set after the First Four games ending evening of Weds March 20?
", "votes": 1, "replies": [ { "id": 491673, "author_name": "Jeff Sonas", "author_url": "", "post_date": "2019-03-16T01:23:59.803000", "content": "After the selections and pairings are announced on Sunday, there is no further game data you will be receiving, and so the bulk of Stage 2 data will be released as soon as possible on Sunday evening. However, the Massey Ordinals take a few days to completely come in, so we will be releasing updated versions of that file (only) over the course of the next few days, from Sunday up through Wednesday or so.
", "votes": 1, "replies": [] } ] }, { "id": 491025, "author_name": "Jeff Sonas", "author_url": "", "post_date": "2019-03-15T04:40:27.483000", "content": "Still running into some delays assembling the data because I don't have access to everything I thought I did. Here are the Massey Ordinals and Regular Season Compact Results, cumulative up through last night, in case that's all you are looking for. I will soon be doing a full post with updated versions of all the \"prelim\" files, including a fake bracket so you can see what the submission format for Stage 2 will look like.
\n\nEvidently there is value on my end as well, to go through a preliminary run-through of the data refresh process! Apologies to those who were counting on the prelim data coming in sooner. But I am doing what I can to resolve this ASAP.
", "votes": 1, "replies": [ { "id": 491386, "author_name": "Zach", "author_url": "", "post_date": "2019-03-15T15:22:26.913000", "content": "Great, thank you!
", "votes": 1, "replies": [] } ] }, { "id": 486422, "author_name": "Zach", "author_url": "", "post_date": "2019-03-08T18:41:42.980000", "content": "@Jeff: any vague guess as to when the Stage 2 Preview will be out? =D
", "votes": 1, "replies": [ { "id": 486477, "author_name": "Jeff Sonas", "author_url": "", "post_date": "2019-03-08T20:46:11.183000", "content": "@Zach, probably Wednesday - give or take a day.
", "votes": 2, "replies": [] }, { "id": 489289, "author_name": "Zach", "author_url": "", "post_date": "2019-03-13T18:25:29.947000", "content": "@Jeff How's it going? =D
", "votes": 1, "replies": [] }, { "id": 489669, "author_name": "Jeff Sonas", "author_url": "", "post_date": "2019-03-14T02:22:13.797000", "content": "Sometime in the next 24 hours, it looks like
", "votes": 1, "replies": [] }, { "id": 490519, "author_name": "", "author_url": "", "post_date": "2019-03-14T16:22:32.013000", "content": "", "votes": 4, "replies": [] } ] }, { "id": 491872, "author_name": "Jeff Sonas", "author_url": "", "post_date": "2019-03-16T09:58:34.007000", "content": "Here are the regular season detailed results, and the conference tourney games listing, as of a few days ago. When we release Stage 2, everything will be brought up through the end of the regular season.
", "votes": 2, "replies": [] }, { "id": 491776, "author_name": "Jeff Sonas", "author_url": "", "post_date": "2019-03-16T05:54:32.873000", "content": "Here are some more preliminary files. Again I apologize for the raggedness of this, but a data feed that I thought was available this year, turns out not to be, and so a bunch of this had to be done in an unexpected way, and more manually than planned. I will still be releasing it all in one post but I am also making some of these files available right away for people, if you can use them. I still need to do some more work tonight in order to get RegularSeasonDetailedResults and ConferenceTourneyGames and play-by-play ready, but it will happen tonight.
", "votes": 2, "replies": [ { "id": 491795, "author_name": "MuhammedBuyukkinaci", "author_url": "", "post_date": "2019-03-16T06:41:02.507000", "content": "Hi Jeff
\n\nThanks for your great efforts. It would be better to have RegularSeasonDetailedResults.csv of 2019 ASAP for competitiors.
", "votes": 0, "replies": [] } ] }, { "id": 472920, "author_name": "Jeff Sonas", "author_url": "", "post_date": "2019-02-16T22:26:18.050000", "content": "We typically try to provide a preview of Stage 2 data before Selection Sunday comes along, by using a hypothetical bracket a couple of weeks in advance, so that you can see what Stage 2 data will look like (including the real partial data from the current season). This lets you make sure your code will work on Stage 2 data, which will be a little different (most notably, it will only be one season's worth of tournament predictions that you submit) and also gives you a sneak peak at the current season's data. However, the official Stage 2 release (including complete data from the current regular season) will not be available until something like 12-24 hours after the tournament pairings are announced on Selection Sunday.
", "votes": 2, "replies": [] }, { "id": 492633, "author_name": "Zach", "author_url": "", "post_date": "2019-03-17T14:30:11.920000", "content": "Will there be an updated team spellings file this year?
", "votes": 0, "replies": [ { "id": 492731, "author_name": "Jeff Sonas", "author_url": "", "post_date": "2019-03-17T17:24:18.800000", "content": "I don't think I had any new spellings since Stage 1, so at this point, no.
", "votes": 1, "replies": [] } ] }, { "id": 492588, "author_name": "Zach", "author_url": "", "post_date": "2019-03-17T13:36:22.177000", "content": "Thank you Jeff! This competition is always a bunch of hard work all at once!
", "votes": 0, "replies": [] }, { "id": 472733, "author_name": "", "author_url": "", "post_date": "2019-02-16T14:59:50.757000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "492156": "And finally, here is the prelim play-by-play data. We have a different provider for play-by-play this year, compared to last year. For the most part, the data seems nice and compatible, but this did delay the release of the files this year. Differences I noticed are:\n (1) Player names tend to be formatted like FIRST_LAST rather than LAST_FIRST\n (2) There were a couple of new types of two-point-shot this year: hook (which I mapped into jump) and alleyoop (which I mapped into dunk). If we'd had consistent older data (from previous years) with this information, I might have been tempted to keep them as separate types of shot, but at this late time that seemed unhelpful.\n (3) In previous years there were only three types of rebounds: offensive, defensive, and deadball. This year there was a fourth type of rebound, which was just \"rebound\". I mapped that into deadball since the proportions seemed right.\n\nAlso, please remember that the EventID's and PlayerID's are temporary in this release. The numerical EventID's and PlayerID's are assigned based on an overall sort. If new player spellings are introduced near the end of the year, those players might appear in the middle of the final sorted list for 2019, and would change the overall sort order. So please don't assume that players having a particular PlayerID, or events having a particular EventID, will retain those same ID's into future releases of these files.\n", "491666": "When is the final stage 2 data released? After March 17 Selection Sunday, or is there an updated official data set after the First Four games ending evening of Weds March 20? ", "491025": "Still running into some delays assembling the data because I don't have access to everything I thought I did. Here are the Massey Ordinals and Regular Season Compact Results, cumulative up through last night, in case that's all you are looking for. I will soon be doing a full post with updated versions of all the \"prelim\" files, including a fake bracket so you can see what the submission format for Stage 2 will look like.\n\nEvidently there is value on my end as well, to go through a preliminary run-through of the data refresh process! Apologies to those who were counting on the prelim data coming in sooner. But I am doing what I can to resolve this ASAP.", "486422": "@Jeff: any vague guess as to when the Stage 2 Preview will be out? =D", "472716": "", "491872": "Here are the regular season detailed results, and the conference tourney games listing, as of a few days ago. When we release Stage 2, everything will be brought up through the end of the regular season.", "491776": "Here are some more preliminary files. Again I apologize for the raggedness of this, but a data feed that I thought was available this year, turns out not to be, and so a bunch of this had to be done in an unexpected way, and more manually than planned. I will still be releasing it all in one post but I am also making some of these files available right away for people, if you can use them. I still need to do some more work tonight in order to get RegularSeasonDetailedResults and ConferenceTourneyGames and play-by-play ready, but it will happen tonight.", "472920": "We typically try to provide a preview of Stage 2 data before Selection Sunday comes along, by using a hypothetical bracket a couple of weeks in advance, so that you can see what Stage 2 data will look like (including the real partial data from the current season). This lets you make sure your code will work on Stage 2 data, which will be a little different (most notably, it will only be one season's worth of tournament predictions that you submit) and also gives you a sneak peak at the current season's data. However, the official Stage 2 release (including complete data from the current regular season) will not be available until something like 12-24 hours after the tournament pairings are announced on Selection Sunday.", "492633": "Will there be an updated team spellings file this year?", "492588": "Thank you Jeff! This competition is always a bunch of hard work all at once!", "472733": "" } }