Hi everyone, here's an update regarding the stage 2 data. I am making this same post for all three contests.
\n\nI have brought the season 2020 data up to the present, with its ragged termination around day 128-129, and you will see new Stage 2 zip files on the Data page. Thiese data updates are reflected in the various files such as RegularSeasonCompactResults, RegularSeasonDetailedResults, and GameCities, along with several others. In addition, I decided to rebuild the entire history of play-by-play, including for seasons prior to 2020. The play-by-play now includes a file for the 2020 season, up through its termination at day 128/129.
\n\nTherefore there is a new men’s data files zip (MDataFiles_Stage2.zip), a new men’s play-by-play zip (MPlayByPlay_Stage2.zip), and the same two files on the women’s side (WDataFiles_Stage2.zip and WPlayByPlay_Stage2.zip). The file names indicate that they are stage 2 data files. So each prediction contest has two new files available on its Data page, and the analytics contest (which covers both men’s and women’s) has four new files available on its Data page.
\n\nI decided to refresh the entire play-by-play data (including previous years) because I know there were some bug fixes upstream of my work, that would hopefully result in the running score (as the game proceeded) being populated rather than always zero, as well as a significant problem that was fixed in the elapsed seconds of the women’s data. My understanding is that these fixes were implemented after I pulled the play-by-play data for stage 1, and so they should be fixed in this latest stage 2 release, now that I've pulled the data again. The play-by-play files from previous years seem a bit larger now, so I expect there are a few more games covered than before - unless it’s just that there are nonzero running scores in there rather than zeroes, taking up a few extra bytes per row.
\n\nThe Massey Ordinals (men’s only) are only populated up through Day 128 - usually there is a Day 133 but the season ended early. I think there may be one more round of ordinals posted on the site but I am not sure how complete that data is.
\n\nEverything should be up to date now, although obviously there are no new NCAA Tourney seeds or results or anything like that. We don’t even get to know which regions were W, X, Y, or Z! Anyway, these updates should provide useful extra information for anyone still working on the analytics contest, and or people wrapping up their work on the predictions contest. Please post any issues you find in these data releases, to this thread for your contest, and hopefully I can address them.
", "messages": [ { "id": 783954, "postDate": "2020-03-23T20:22:01.193Z", "content": "Hi everyone, here's an update regarding the stage 2 data. I am making this same post for all three contests.
\n\nI have brought the season 2020 data up to the present, with its ragged termination around day 128-129, and you will see new Stage 2 zip files on the Data page. Thiese data updates are reflected in the various files such as RegularSeasonCompactResults, RegularSeasonDetailedResults, and GameCities, along with several others. In addition, I decided to rebuild the entire history of play-by-play, including for seasons prior to 2020. The play-by-play now includes a file for the 2020 season, up through its termination at day 128/129.
\n\nTherefore there is a new men’s data files zip (MDataFiles_Stage2.zip), a new men’s play-by-play zip (MPlayByPlay_Stage2.zip), and the same two files on the women’s side (WDataFiles_Stage2.zip and WPlayByPlay_Stage2.zip). The file names indicate that they are stage 2 data files. So each prediction contest has two new files available on its Data page, and the analytics contest (which covers both men’s and women’s) has four new files available on its Data page.
\n\nI decided to refresh the entire play-by-play data (including previous years) because I know there were some bug fixes upstream of my work, that would hopefully result in the running score (as the game proceeded) being populated rather than always zero, as well as a significant problem that was fixed in the elapsed seconds of the women’s data. My understanding is that these fixes were implemented after I pulled the play-by-play data for stage 1, and so they should be fixed in this latest stage 2 release, now that I've pulled the data again. The play-by-play files from previous years seem a bit larger now, so I expect there are a few more games covered than before - unless it’s just that there are nonzero running scores in there rather than zeroes, taking up a few extra bytes per row.
\n\nThe Massey Ordinals (men’s only) are only populated up through Day 128 - usually there is a Day 133 but the season ended early. I think there may be one more round of ordinals posted on the site but I am not sure how complete that data is.
\n\nEverything should be up to date now, although obviously there are no new NCAA Tourney seeds or results or anything like that. We don’t even get to know which regions were W, X, Y, or Z! Anyway, these updates should provide useful extra information for anyone still working on the analytics contest, and or people wrapping up their work on the predictions contest. Please post any issues you find in these data releases, to this thread for your contest, and hopefully I can address them.
", "rawMarkdown": "Hi everyone, here's an update regarding the stage 2 data. I am making this same post for all three contests. \n\nI have brought the season 2020 data up to the present, with its ragged termination around day 128-129, and you will see new Stage 2 zip files on the Data page. Thiese data updates are reflected in the various files such as RegularSeasonCompactResults, RegularSeasonDetailedResults, and GameCities, along with several others. In addition, I decided to rebuild the entire history of play-by-play, including for seasons prior to 2020. The play-by-play now includes a file for the 2020 season, up through its termination at day 128/129.\n\nTherefore there is a new men’s data files zip (MDataFiles_Stage2.zip), a new men’s play-by-play zip (MPlayByPlay_Stage2.zip), and the same two files on the women’s side (WDataFiles_Stage2.zip and WPlayByPlay_Stage2.zip). The file names indicate that they are stage 2 data files. So each prediction contest has two new files available on its Data page, and the analytics contest (which covers both men’s and women’s) has four new files available on its Data page.\n\nI decided to refresh the entire play-by-play data (including previous years) because I know there were some bug fixes upstream of my work, that would hopefully result in the running score (as the game proceeded) being populated rather than always zero, as well as a significant problem that was fixed in the elapsed seconds of the women’s data. My understanding is that these fixes were implemented after I pulled the play-by-play data for stage 1, and so they should be fixed in this latest stage 2 release, now that I've pulled the data again. The play-by-play files from previous years seem a bit larger now, so I expect there are a few more games covered than before - unless it’s just that there are nonzero running scores in there rather than zeroes, taking up a few extra bytes per row.\n\nThe Massey Ordinals (men’s only) are only populated up through Day 128 - usually there is a Day 133 but the season ended early. I think there may be one more round of ordinals posted on the site but I am not sure how complete that data is.\n\nEverything should be up to date now, although obviously there are no new NCAA Tourney seeds or results or anything like that. We don’t even get to know which regions were W, X, Y, or Z! Anyway, these updates should provide useful extra information for anyone still working on the analytics contest, and or people wrapping up their work on the predictions contest. Please post any issues you find in these data releases, to this thread for your contest, and hopefully I can address them." }, { "id": 784027, "postDate": "2020-03-23T22:27:12.893Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 784027, "author_name": "", "author_url": "", "post_date": "2020-03-23T22:27:12.893000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "783954": "Hi everyone, here's an update regarding the stage 2 data. I am making this same post for all three contests. \n\nI have brought the season 2020 data up to the present, with its ragged termination around day 128-129, and you will see new Stage 2 zip files on the Data page. Thiese data updates are reflected in the various files such as RegularSeasonCompactResults, RegularSeasonDetailedResults, and GameCities, along with several others. In addition, I decided to rebuild the entire history of play-by-play, including for seasons prior to 2020. The play-by-play now includes a file for the 2020 season, up through its termination at day 128/129.\n\nTherefore there is a new men’s data files zip (MDataFiles_Stage2.zip), a new men’s play-by-play zip (MPlayByPlay_Stage2.zip), and the same two files on the women’s side (WDataFiles_Stage2.zip and WPlayByPlay_Stage2.zip). The file names indicate that they are stage 2 data files. So each prediction contest has two new files available on its Data page, and the analytics contest (which covers both men’s and women’s) has four new files available on its Data page.\n\nI decided to refresh the entire play-by-play data (including previous years) because I know there were some bug fixes upstream of my work, that would hopefully result in the running score (as the game proceeded) being populated rather than always zero, as well as a significant problem that was fixed in the elapsed seconds of the women’s data. My understanding is that these fixes were implemented after I pulled the play-by-play data for stage 1, and so they should be fixed in this latest stage 2 release, now that I've pulled the data again. The play-by-play files from previous years seem a bit larger now, so I expect there are a few more games covered than before - unless it’s just that there are nonzero running scores in there rather than zeroes, taking up a few extra bytes per row.\n\nThe Massey Ordinals (men’s only) are only populated up through Day 128 - usually there is a Day 133 but the season ended early. I think there may be one more round of ordinals posted on the site but I am not sure how complete that data is.\n\nEverything should be up to date now, although obviously there are no new NCAA Tourney seeds or results or anything like that. We don’t even get to know which regions were W, X, Y, or Z! Anyway, these updates should provide useful extra information for anyone still working on the analytics contest, and or people wrapping up their work on the predictions contest. Please post any issues you find in these data releases, to this thread for your contest, and hopefully I can address them.", "784027": "" } }