{
  "id": 343911,
  "title": "Provided datasets",
  "url": "/competitions/big-data-derby-2022/discussion/343911",
  "author_name": "",
  "post_date": "2022-08-12T23:13:11.634507200Z",
  "votes": 5,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Are the 4 files under the data section all that will be provided?  The description in the overview alludes to longitudinal movement, dorsal/ventral movement, medial/lateral deviation data, etc. that does not exist in these files.  Other than lat/long data points, the remaining data is more or less publicly available through Equibase and doesn't seem to align with the types insights this competition hopes to extract.</p>",
  "messages": [
    {
      "id": "1896559",
      "postDate": "08/12/2022 23:13:11",
      "content": "<p>Are the 4 files under the data section all that will be provided?  The description in the overview alludes to longitudinal movement, dorsal/ventral movement, medial/lateral deviation data, etc. that does not exist in these files.  Other than lat/long data points, the remaining data is more or less publicly available through Equibase and doesn't seem to align with the types insights this competition hopes to extract.</p>",
      "rawMarkdown": "Are the 4 files under the data section all that will be provided?  The description in the overview alludes to longitudinal movement, dorsal/ventral movement, medial/lateral deviation data, etc. that does not exist in these files.  Other than lat/long data points, the remaining data is more or less publicly available through Equibase and doesn't seem to align with the types insights this competition hopes to extract.",
      "votes": null
    },
    {
      "id": "1896563",
      "postDate": "08/12/2022 23:27:33",
      "content": "<p>Hi there,</p>\n<p>Per competition rules, you are welcome to use publicly available data in addition to the data provided. The description of data provided is data that is currently being collected, but will not be provided for this competition.</p>\n<p>Good luck!</p>",
      "rawMarkdown": "Hi there,\n\nPer competition rules, you are welcome to use publicly available data in addition to the data provided. The description of data provided is data that is currently being collected, but will not be provided for this competition.\n\nGood luck!",
      "votes": null
    },
    {
      "id": "1899096",
      "postDate": "08/15/2022 04:13:15",
      "content": "<p>No column for finishing position (1st place, 2nd place, etc…)?!</p>",
      "rawMarkdown": "No column for finishing position (1st place, 2nd place, etc...)?!",
      "votes": null
    },
    {
      "id": "1900240",
      "postDate": "08/15/2022 20:24:02",
      "content": "<p>I am also looking for the finishing position. Or is it possible to get from the latitude/longitude columns? </p>",
      "rawMarkdown": "I am also looking for the finishing position. Or is it possible to get from the latitude/longitude columns?",
      "votes": null
    },
    {
      "id": "1900468",
      "postDate": "08/16/2022 04:09:12",
      "content": "<p>This could be an interesting thing to analyze, except the basic data provided will not lead to any great revelations regarding horse welfare and safety. I’m not sure even rudimentary handicapping angles could be produced by using just this data. Without medication and breakdown information, there is nothing to glean from coordinate information, except perhaps to validate the shortest distance traveled leads to better placings. Even that is questionable absent track variant data.</p>\n<p>Per the contest rules, you can use other publicly available data as long as it is shared with everyone else. I have a dozen years’ worth of detailed NYRA race data, but it cost me $$$ to obtain, and I couldn’t legally share it with anyone anyway. If I could use that data, who knows what fascinating information could be derived? I would love to take a deeper dive. Even then, you’d need more than a single year’s worth of data, that just isn’t enough given the variety of surfaces, tracks and distances involved to have a large enough sample from which to draw conclusions.</p>\n<p>If you matched the provided coordinate data from 2019 to the equine injury/breakdown database available for NY tracks, at best you might be able to conclude, for example, something like inside to outside moves during races had a more adverse effect than other moves. Even then, there are scant few in-race fatalities from 2019, too few to produce any adequate sample sizes. Plus, that equine database has enough holes in it, i.e. fields left blank, etc. that it cannot be counted on as a reliable source.</p>\n<p>If I could use my data, I likely could produce some salient information, but by and large it would be for the benefit of horseplayers more than anyone else. The heart rate, EKG and other data mentioned, but not provided, would be more valuable, as would the database I assume is maintained somewhere from post-race veterinarian exams for lameness of claiming race participants.</p>\n<p>Alas, I am not a data scientist, but I know enough about data and racing to know which questions to ask and which data might be beneficial to have, and perhaps be a valuable team member. Just from an observational perspective on one aspect mentioned in the overview, I could state fairly confidently that drafting plays no part in dirt racing but may in turf racing, not as a conscious benefit, but simply as a result of trying to save ground without the perils of the kickback that occurs in dirt races. As such, horses in dirt races will rarely line up one behind the other, while in grass racing it is common to see. I couldn’t speculate as to what effect drafting has on animal welfare though.</p>",
      "rawMarkdown": "This could be an interesting thing to analyze, except the basic data provided will not lead to any great revelations regarding horse welfare and safety. I’m not sure even rudimentary handicapping angles could be produced by using just this data. Without medication and breakdown information, there is nothing to glean from coordinate information, except perhaps to validate the shortest distance traveled leads to better placings. Even that is questionable absent track variant data.\n\nPer the contest rules, you can use other publicly available data as long as it is shared with everyone else. I have a dozen years’ worth of detailed NYRA race data, but it cost me $$$ to obtain, and I couldn’t legally share it with anyone anyway. If I could use that data, who knows what fascinating information could be derived? I would love to take a deeper dive. Even then, you’d need more than a single year’s worth of data, that just isn’t enough given the variety of surfaces, tracks and distances involved to have a large enough sample from which to draw conclusions.\n\nIf you matched the provided coordinate data from 2019 to the equine injury/breakdown database available for NY tracks, at best you might be able to conclude, for example, something like inside to outside moves during races had a more adverse effect than other moves. Even then, there are scant few in-race fatalities from 2019, too few to produce any adequate sample sizes. Plus, that equine database has enough holes in it, i.e. fields left blank, etc. that it cannot be counted on as a reliable source.\n\nIf I could use my data, I likely could produce some salient information, but by and large it would be for the benefit of horseplayers more than anyone else. The heart rate, EKG and other data mentioned, but not provided, would be more valuable, as would the database I assume is maintained somewhere from post-race veterinarian exams for lameness of claiming race participants.\n\nAlas, I am not a data scientist, but I know enough about data and racing to know which questions to ask and which data might be beneficial to have, and perhaps be a valuable team member. Just from an observational perspective on one aspect mentioned in the overview, I could state fairly confidently that drafting plays no part in dirt racing but may in turf racing, not as a conscious benefit, but simply as a result of trying to save ground without the perils of the kickback that occurs in dirt races. As such, horses in dirt races will rarely line up one behind the other, while in grass racing it is common to see. I couldn’t speculate as to what effect drafting has on animal welfare though.",
      "votes": null
    },
    {
      "id": "1902980",
      "postDate": "08/17/2022 03:20:53",
      "content": "<p>Hi, I agree with your sentiments exactly Is your data from HDW and HTR?</p>",
      "rawMarkdown": "Hi, I agree with your sentiments exactly Is your data from HDW and HTR?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1896563,
      "author_name": "addisonhoward",
      "author_url": "",
      "post_date": "08/12/2022 23:27:33",
      "content": "<p>Hi there,</p>\n<p>Per competition rules, you are welcome to use publicly available data in addition to the data provided. The description of data provided is data that is currently being collected, but will not be provided for this competition.</p>\n<p>Good luck!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1899096,
      "author_name": "kingstats101",
      "author_url": "",
      "post_date": "08/15/2022 04:13:15",
      "content": "<p>No column for finishing position (1st place, 2nd place, etc…)?!</p>",
      "votes": null,
      "replies": [
        {
          "id": 1900240,
          "author_name": "seraquevence",
          "author_url": "",
          "post_date": "08/15/2022 20:24:02",
          "content": "<p>I am also looking for the finishing position. Or is it possible to get from the latitude/longitude columns? </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1900468,
      "author_name": "stevenr59",
      "author_url": "",
      "post_date": "08/16/2022 04:09:12",
      "content": "<p>This could be an interesting thing to analyze, except the basic data provided will not lead to any great revelations regarding horse welfare and safety. I’m not sure even rudimentary handicapping angles could be produced by using just this data. Without medication and breakdown information, there is nothing to glean from coordinate information, except perhaps to validate the shortest distance traveled leads to better placings. Even that is questionable absent track variant data.</p>\n<p>Per the contest rules, you can use other publicly available data as long as it is shared with everyone else. I have a dozen years’ worth of detailed NYRA race data, but it cost me $$$ to obtain, and I couldn’t legally share it with anyone anyway. If I could use that data, who knows what fascinating information could be derived? I would love to take a deeper dive. Even then, you’d need more than a single year’s worth of data, that just isn’t enough given the variety of surfaces, tracks and distances involved to have a large enough sample from which to draw conclusions.</p>\n<p>If you matched the provided coordinate data from 2019 to the equine injury/breakdown database available for NY tracks, at best you might be able to conclude, for example, something like inside to outside moves during races had a more adverse effect than other moves. Even then, there are scant few in-race fatalities from 2019, too few to produce any adequate sample sizes. Plus, that equine database has enough holes in it, i.e. fields left blank, etc. that it cannot be counted on as a reliable source.</p>\n<p>If I could use my data, I likely could produce some salient information, but by and large it would be for the benefit of horseplayers more than anyone else. The heart rate, EKG and other data mentioned, but not provided, would be more valuable, as would the database I assume is maintained somewhere from post-race veterinarian exams for lameness of claiming race participants.</p>\n<p>Alas, I am not a data scientist, but I know enough about data and racing to know which questions to ask and which data might be beneficial to have, and perhaps be a valuable team member. Just from an observational perspective on one aspect mentioned in the overview, I could state fairly confidently that drafting plays no part in dirt racing but may in turf racing, not as a conscious benefit, but simply as a result of trying to save ground without the perils of the kickback that occurs in dirt races. As such, horses in dirt races will rarely line up one behind the other, while in grass racing it is common to see. I couldn’t speculate as to what effect drafting has on animal welfare though.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1902980,
          "author_name": "bobcajones",
          "author_url": "",
          "post_date": "08/17/2022 03:20:53",
          "content": "<p>Hi, I agree with your sentiments exactly Is your data from HDW and HTR?</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1896559": "Are the 4 files under the data section all that will be provided?  The description in the overview alludes to longitudinal movement, dorsal/ventral movement, medial/lateral deviation data, etc. that does not exist in these files.  Other than lat/long data points, the remaining data is more or less publicly available through Equibase and doesn't seem to align with the types insights this competition hopes to extract.",
    "1896563": "Hi there,\n\nPer competition rules, you are welcome to use publicly available data in addition to the data provided. The description of data provided is data that is currently being collected, but will not be provided for this competition.\n\nGood luck!",
    "1899096": "No column for finishing position (1st place, 2nd place, etc...)?!",
    "1900240": "I am also looking for the finishing position. Or is it possible to get from the latitude/longitude columns?",
    "1900468": "This could be an interesting thing to analyze, except the basic data provided will not lead to any great revelations regarding horse welfare and safety. I’m not sure even rudimentary handicapping angles could be produced by using just this data. Without medication and breakdown information, there is nothing to glean from coordinate information, except perhaps to validate the shortest distance traveled leads to better placings. Even that is questionable absent track variant data.\n\nPer the contest rules, you can use other publicly available data as long as it is shared with everyone else. I have a dozen years’ worth of detailed NYRA race data, but it cost me $$$ to obtain, and I couldn’t legally share it with anyone anyway. If I could use that data, who knows what fascinating information could be derived? I would love to take a deeper dive. Even then, you’d need more than a single year’s worth of data, that just isn’t enough given the variety of surfaces, tracks and distances involved to have a large enough sample from which to draw conclusions.\n\nIf you matched the provided coordinate data from 2019 to the equine injury/breakdown database available for NY tracks, at best you might be able to conclude, for example, something like inside to outside moves during races had a more adverse effect than other moves. Even then, there are scant few in-race fatalities from 2019, too few to produce any adequate sample sizes. Plus, that equine database has enough holes in it, i.e. fields left blank, etc. that it cannot be counted on as a reliable source.\n\nIf I could use my data, I likely could produce some salient information, but by and large it would be for the benefit of horseplayers more than anyone else. The heart rate, EKG and other data mentioned, but not provided, would be more valuable, as would the database I assume is maintained somewhere from post-race veterinarian exams for lameness of claiming race participants.\n\nAlas, I am not a data scientist, but I know enough about data and racing to know which questions to ask and which data might be beneficial to have, and perhaps be a valuable team member. Just from an observational perspective on one aspect mentioned in the overview, I could state fairly confidently that drafting plays no part in dirt racing but may in turf racing, not as a conscious benefit, but simply as a result of trying to save ground without the perils of the kickback that occurs in dirt races. As such, horses in dirt races will rarely line up one behind the other, while in grass racing it is common to see. I couldn’t speculate as to what effect drafting has on animal welfare though.",
    "1902980": "Hi, I agree with your sentiments exactly Is your data from HDW and HTR?"
  },
  "source": "meta"
}