{
  "id": 351147,
  "title": "Updated Data Files - Position at Finish",
  "url": "/competitions/big-data-derby-2022/discussion/351147",
  "author_name": "Joe Appelbaum",
  "post_date": "2022-09-08T17:22:10.839000",
  "votes": 21,
  "comment_count": 14,
  "views": 0,
  "content": "<p>Numerous posts have asked to provide information on the position of each horse at the finish. While a few posts actually showed how to figure this out, because of the timing interval (4x per second) of the data, there is a small chance that a close finish might not be accurately reflected in the data. We have replaced the old files with the new ones (so the names won't look any different) and updated the language in the data tab to include the new field.</p>",
  "messages": [
    {
      "id": 1931457,
      "postDate": "2022-09-08T17:22:10.840Z",
      "content": "<p>Numerous posts have asked to provide information on the position of each horse at the finish. While a few posts actually showed how to figure this out, because of the timing interval (4x per second) of the data, there is a small chance that a close finish might not be accurately reflected in the data. We have replaced the old files with the new ones (so the names won't look any different) and updated the language in the data tab to include the new field.</p>",
      "rawMarkdown": "Numerous posts have asked to provide information on the position of each horse at the finish. While a few posts actually showed how to figure this out, because of the timing interval (4x per second) of the data, there is a small chance that a close finish might not be accurately reflected in the data. We have replaced the old files with the new ones (so the names won't look any different) and updated the language in the data tab to include the new field.",
      "votes": 21
    },
    {
      "id": 1935617,
      "postDate": "2022-09-12T08:34:10.297Z",
      "content": "<p>According to the updated data files, there are 34 races where 2 horses finish at the same position.<br>\nFor example, race number 9 on 2019-02-09 at AQU, horses numbers 4 and 8 finished both at first place.<br>\nDoes this make sense?</p>",
      "rawMarkdown": "According to the updated data files, there are 34 races where 2 horses finish at the same position.\nFor example, race number 9 on 2019-02-09 at AQU, horses numbers 4 and 8 finished both at first place.\nDoes this make sense?",
      "votes": 1,
      "replies": [
        {
          "id": 1935644,
          "postDate": "2022-09-12T09:00:33.517Z",
          "content": "<p>Can't comment on those specific races, but yes it does make sense from a global perspective - dead heats (both horses being awarded first place) do happen in horse racing when the photo finish can't separate the two horses involved</p>",
          "rawMarkdown": "Can't comment on those specific races, but yes it does make sense from a global perspective - dead heats (both horses being awarded first place) do happen in horse racing when the photo finish can't separate the two horses involved",
          "votes": 1
        }
      ]
    },
    {
      "id": 1931786,
      "postDate": "2022-09-09T03:37:44.590Z",
      "content": "<p>In the new transformed files, the header seems to be missing for nyra_start_table.csv<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F885209%2F043cb46db315c9dbb96bd8aef20babc2%2FMissing%20Header.jpg?generation=1662694627115426&amp;alt=media\" alt=\"\"></p>\n<p>Can you please add the headers back.</p>",
      "rawMarkdown": "In the new transformed files, the header seems to be missing for nyra_start_table.csv![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F885209%2F043cb46db315c9dbb96bd8aef20babc2%2FMissing%20Header.jpg?generation=1662694627115426&alt=media)\n\nCan you please add the headers back.",
      "votes": 1,
      "replies": [
        {
          "id": 1931922,
          "postDate": "2022-09-09T05:59:27.260Z",
          "content": "<p>An Example of a nyra_2019_complete.csv is shown below. You can apply the same logic to other CSV.</p>\n<p><a href=\"https://postimg.cc/zbwjRqwV\" target=\"_blank\"><img src=\"https://i.postimg.cc/V6pHp6dB/Screen-Shot-2022-09-09-at-12-54-16-AM.png\" alt=\"Screen-Shot-2022-09-09-at-12-54-16-AM.png\"></a></p>\n<p>Hope this helps.</p>",
          "rawMarkdown": "An Example of a nyra_2019_complete.csv is shown below. You can apply the same logic to other CSV.\n\n[![Screen-Shot-2022-09-09-at-12-54-16-AM.png](https://i.postimg.cc/V6pHp6dB/Screen-Shot-2022-09-09-at-12-54-16-AM.png)](https://postimg.cc/zbwjRqwV)\n\nHope this helps."
        },
        {
          "id": 1932038,
          "postDate": "2022-09-09T08:07:41.937Z",
          "content": "<p>thank you!!</p>",
          "rawMarkdown": "thank you!!",
          "votes": 1
        }
      ]
    },
    {
      "id": 1931570,
      "postDate": "2022-09-08T19:34:52.363Z",
      "content": "<p>Hello Joe,</p>\n<p>Firstly thank you for providing a new feature of finishing place. However, I have already figured out that feature and it exactly matches the one you provided.</p>\n<p>A glance of my ETL data summary can be seen in the below images</p>\n<ol>\n<li><p>My ETL Dataset Overview (has <strong>23 features</strong> also <strong>will add two</strong> more <strong>crucial features</strong>)<br>\n<a href=\"https://postimg.cc/Mfjp5X46\" target=\"_blank\"><img src=\"https://i.postimg.cc/ZnwvTdxd/Screen-Shot-2022-09-08-at-1-35-55-PM.png\" alt=\"Screen-Shot-2022-09-08-at-1-35-55-PM.png\"></a></p></li>\n<li><p>Current Competition Dataset (Not Properly Transformed)<br>\n<a href=\"https://postimg.cc/4nVLDWsP\" target=\"_blank\"><img src=\"https://i.postimg.cc/XNDRyPPR/Screen-Shot-2022-09-08-at-1-20-04-PM.png\" alt=\"Screen-Shot-2022-09-08-at-1-20-04-PM.png\"></a></p></li>\n</ol>\n<p>Secondly, I would appreciate it if by any chance I can get the <strong>AGE</strong> of the program_number(horse) that would be a great help for my Machine Learning model to be more robust against aggregation and group basis.</p>\n<p>Not only but also if I can get the <strong>trainer</strong>, <strong>owner</strong>, <strong>Sire</strong>, and <strong>Dam</strong> names that would be a big help to me as that feature plays a very important role in drafting strategies.</p>\n<p>Hope you can provide those data if possible. Thank you.</p>",
      "rawMarkdown": "Hello Joe,\n\nFirstly thank you for providing a new feature of finishing place. However, I have already figured out that feature and it exactly matches the one you provided.\n\nA glance of my ETL data summary can be seen in the below images\n\n1. My ETL Dataset Overview (has **23 features** also **will add two** more **crucial features**)\n[![Screen-Shot-2022-09-08-at-1-35-55-PM.png](https://i.postimg.cc/ZnwvTdxd/Screen-Shot-2022-09-08-at-1-35-55-PM.png)](https://postimg.cc/Mfjp5X46)\n\n2. Current Competition Dataset (Not Properly Transformed)\n[![Screen-Shot-2022-09-08-at-1-20-04-PM.png](https://i.postimg.cc/XNDRyPPR/Screen-Shot-2022-09-08-at-1-20-04-PM.png)](https://postimg.cc/4nVLDWsP)\n\nSecondly, I would appreciate it if by any chance I can get the **AGE** of the program_number(horse) that would be a great help for my Machine Learning model to be more robust against aggregation and group basis.\n\nNot only but also if I can get the **trainer**, **owner**, **Sire**, and **Dam** names that would be a big help to me as that feature plays a very important role in drafting strategies.\n\nHope you can provide those data if possible. Thank you.",
      "votes": 1
    },
    {
      "id": 1931552,
      "postDate": "2022-09-08T18:57:32.543Z",
      "content": "<p>Will take a look at that.</p>",
      "rawMarkdown": "Will take a look at that.",
      "votes": 1,
      "replies": [
        {
          "id": 1933579,
          "postDate": "2022-09-10T15:42:56.407Z",
          "content": "<p>nyra_complete and nyra_start files do not have column names.</p>\n<p>Here is a notebook with a quick fix to the missing column names when loading the CSV. Note in the data tab for the competition, there is no purse in the complete file, but there is in the new complete file.</p>\n<p>No need for replacing column names or loops.</p>\n<p><a href=\"https://www.kaggle.com/mlconsult/fix-missing-data-column-names\" target=\"_blank\">https://www.kaggle.com/mlconsult/fix-missing-data-column-names</a></p>",
          "rawMarkdown": "nyra_complete and nyra_start files do not have column names.\n\nHere is a notebook with a quick fix to the missing column names when loading the CSV. Note in the data tab for the competition, there is no purse in the complete file, but there is in the new complete file.\n\nNo need for replacing column names or loops.\n\nhttps://www.kaggle.com/mlconsult/fix-missing-data-column-names"
        }
      ]
    },
    {
      "id": 1933632,
      "postDate": "2022-09-10T16:27:13.910Z",
      "content": "<p>Thank you for updating!</p>\n<p>Life hack for beginners:</p>\n<pre><code>df = pd.read_csv('../input/big-data-derby-2022/nyra_2019_complete.csv', header=None)\n\ncols = ['track_id', 'race_date', 'race_number', 'program_number', 'trakus_index', 'latitude', 'longitude', 'distance_id', 'course_type', 'track_condition', 'run_up_distance', 'race_type', 'purse', 'post_time', 'weight_carried', 'jockey', 'odds', 'final_place']\n\ndf.rename({i: col for i, col in enumerate(cols)}, axis=1, inplace=True)\n</code></pre>",
      "rawMarkdown": "Thank you for updating!\n\nLife hack for beginners:\n\n```\ndf = pd.read_csv('../input/big-data-derby-2022/nyra_2019_complete.csv', header=None)\n\ncols = ['track_id', 'race_date', 'race_number', 'program_number', 'trakus_index', 'latitude', 'longitude', 'distance_id', 'course_type', 'track_condition', 'run_up_distance', 'race_type', 'purse', 'post_time', 'weight_carried', 'jockey', 'odds', 'final_place']\n\ndf.rename({i: col for i, col in enumerate(cols)}, axis=1, inplace=True)\n```",
      "votes": 2,
      "replies": [
        {
          "id": 1957429,
          "postDate": "2022-09-27T01:17:08.927Z",
          "content": "<pre><code>df = pd.read_csv(\n    ../input/big-data-derby-2022/nyra_2019_complete.csv, \n    low_memory=False, header=None, names=cols)\n</code></pre>",
          "rawMarkdown": "```\ndf = pd.read_csv(\n    ../input/big-data-derby-2022/nyra_2019_complete.csv, \n    low_memory=False, header=None, names=cols)\n```",
          "votes": 2
        }
      ]
    },
    {
      "id": 1931534,
      "postDate": "2022-09-08T18:44:50.587Z",
      "content": "<p>I think in the new files the first row with the name of the columns is missing, in nyra_2019_complete.csv, and nyra_start_table.csv..(noticed on 8th September)</p>",
      "rawMarkdown": "I think in the new files the first row with the name of the columns is missing, in nyra_2019_complete.csv, and nyra_start_table.csv..(noticed on 8th September)\n ",
      "votes": 2,
      "replies": [
        {
          "id": 1931573,
          "postDate": "2022-09-08T19:37:18.890Z",
          "content": "<p>Correct but you can transform that easily.</p>\n<p>An Example of a nyra_2019_complete.csv is shown below. You can apply the same logic to other CSV also you can enumerate the index and its value.</p>\n<p><a href=\"https://postimg.cc/zbwjRqwV\" target=\"_blank\"><img src=\"https://i.postimg.cc/V6pHp6dB/Screen-Shot-2022-09-09-at-12-54-16-AM.png\" alt=\"Screen-Shot-2022-09-09-at-12-54-16-AM.png\"></a></p>\n<p>Hope this helps.</p>",
          "rawMarkdown": "Correct but you can transform that easily.\n\nAn Example of a nyra_2019_complete.csv is shown below. You can apply the same logic to other CSV also you can enumerate the index and its value.\n\n[![Screen-Shot-2022-09-09-at-12-54-16-AM.png](https://i.postimg.cc/V6pHp6dB/Screen-Shot-2022-09-09-at-12-54-16-AM.png)](https://postimg.cc/zbwjRqwV)\n\nHope this helps."
        },
        {
          "id": 1932452,
          "postDate": "2022-09-09T16:11:18.843Z",
          "content": "<p>Thank you! I know that it's not a big problem, it was just to let people know 😃</p>",
          "rawMarkdown": "Thank you! I know that it's not a big problem, it was just to let people know 😃"
        }
      ]
    },
    {
      "id": 1935033,
      "postDate": "2022-09-11T18:48:46.113Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 1935617,
      "author_name": "shapital",
      "author_url": "",
      "post_date": "2022-09-12T08:34:10.297000",
      "content": "<p>According to the updated data files, there are 34 races where 2 horses finish at the same position.<br>\nFor example, race number 9 on 2019-02-09 at AQU, horses numbers 4 and 8 finished both at first place.<br>\nDoes this make sense?</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1935644,
          "author_name": "Tom Sawczuk",
          "author_url": "",
          "post_date": "2022-09-12T09:00:33.517000",
          "content": "<p>Can't comment on those specific races, but yes it does make sense from a global perspective - dead heats (both horses being awarded first place) do happen in horse racing when the photo finish can't separate the two horses involved</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1931786,
      "author_name": "Mahesh Kulkarni",
      "author_url": "",
      "post_date": "2022-09-09T03:37:44.590000",
      "content": "<p>In the new transformed files, the header seems to be missing for nyra_start_table.csv<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F885209%2F043cb46db315c9dbb96bd8aef20babc2%2FMissing%20Header.jpg?generation=1662694627115426&amp;alt=media\" alt=\"\"></p>\n<p>Can you please add the headers back.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1931922,
          "author_name": "Kush Trivedi",
          "author_url": "",
          "post_date": "2022-09-09T05:59:27.260000",
          "content": "<p>An Example of a nyra_2019_complete.csv is shown below. You can apply the same logic to other CSV.</p>\n<p><a href=\"https://postimg.cc/zbwjRqwV\" target=\"_blank\"><img src=\"https://i.postimg.cc/V6pHp6dB/Screen-Shot-2022-09-09-at-12-54-16-AM.png\" alt=\"Screen-Shot-2022-09-09-at-12-54-16-AM.png\"></a></p>\n<p>Hope this helps.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1932038,
          "author_name": "Mahesh Kulkarni",
          "author_url": "",
          "post_date": "2022-09-09T08:07:41.937000",
          "content": "<p>thank you!!</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1931570,
      "author_name": "Kush Trivedi",
      "author_url": "",
      "post_date": "2022-09-08T19:34:52.363000",
      "content": "<p>Hello Joe,</p>\n<p>Firstly thank you for providing a new feature of finishing place. However, I have already figured out that feature and it exactly matches the one you provided.</p>\n<p>A glance of my ETL data summary can be seen in the below images</p>\n<ol>\n<li><p>My ETL Dataset Overview (has <strong>23 features</strong> also <strong>will add two</strong> more <strong>crucial features</strong>)<br>\n<a href=\"https://postimg.cc/Mfjp5X46\" target=\"_blank\"><img src=\"https://i.postimg.cc/ZnwvTdxd/Screen-Shot-2022-09-08-at-1-35-55-PM.png\" alt=\"Screen-Shot-2022-09-08-at-1-35-55-PM.png\"></a></p></li>\n<li><p>Current Competition Dataset (Not Properly Transformed)<br>\n<a href=\"https://postimg.cc/4nVLDWsP\" target=\"_blank\"><img src=\"https://i.postimg.cc/XNDRyPPR/Screen-Shot-2022-09-08-at-1-20-04-PM.png\" alt=\"Screen-Shot-2022-09-08-at-1-20-04-PM.png\"></a></p></li>\n</ol>\n<p>Secondly, I would appreciate it if by any chance I can get the <strong>AGE</strong> of the program_number(horse) that would be a great help for my Machine Learning model to be more robust against aggregation and group basis.</p>\n<p>Not only but also if I can get the <strong>trainer</strong>, <strong>owner</strong>, <strong>Sire</strong>, and <strong>Dam</strong> names that would be a big help to me as that feature plays a very important role in drafting strategies.</p>\n<p>Hope you can provide those data if possible. Thank you.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1931552,
      "author_name": "Joe Appelbaum",
      "author_url": "",
      "post_date": "2022-09-08T18:57:32.543000",
      "content": "<p>Will take a look at that.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1933579,
          "author_name": "Ken Miller",
          "author_url": "",
          "post_date": "2022-09-10T15:42:56.407000",
          "content": "<p>nyra_complete and nyra_start files do not have column names.</p>\n<p>Here is a notebook with a quick fix to the missing column names when loading the CSV. Note in the data tab for the competition, there is no purse in the complete file, but there is in the new complete file.</p>\n<p>No need for replacing column names or loops.</p>\n<p><a href=\"https://www.kaggle.com/mlconsult/fix-missing-data-column-names\" target=\"_blank\">https://www.kaggle.com/mlconsult/fix-missing-data-column-names</a></p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1933632,
      "author_name": "Dmitry Uarov",
      "author_url": "",
      "post_date": "2022-09-10T16:27:13.910000",
      "content": "<p>Thank you for updating!</p>\n<p>Life hack for beginners:</p>\n<pre><code>df = pd.read_csv('../input/big-data-derby-2022/nyra_2019_complete.csv', header=None)\n\ncols = ['track_id', 'race_date', 'race_number', 'program_number', 'trakus_index', 'latitude', 'longitude', 'distance_id', 'course_type', 'track_condition', 'run_up_distance', 'race_type', 'purse', 'post_time', 'weight_carried', 'jockey', 'odds', 'final_place']\n\ndf.rename({i: col for i, col in enumerate(cols)}, axis=1, inplace=True)\n</code></pre>",
      "votes": 2,
      "replies": [
        {
          "id": 1957429,
          "author_name": "JohnM",
          "author_url": "",
          "post_date": "2022-09-27T01:17:08.927000",
          "content": "<pre><code>df = pd.read_csv(\n    ../input/big-data-derby-2022/nyra_2019_complete.csv, \n    low_memory=False, header=None, names=cols)\n</code></pre>",
          "votes": 2,
          "replies": []
        }
      ]
    },
    {
      "id": 1931534,
      "author_name": "Elisabetta Crescio",
      "author_url": "",
      "post_date": "2022-09-08T18:44:50.587000",
      "content": "<p>I think in the new files the first row with the name of the columns is missing, in nyra_2019_complete.csv, and nyra_start_table.csv..(noticed on 8th September)</p>",
      "votes": 2,
      "replies": [
        {
          "id": 1931573,
          "author_name": "Kush Trivedi",
          "author_url": "",
          "post_date": "2022-09-08T19:37:18.890000",
          "content": "<p>Correct but you can transform that easily.</p>\n<p>An Example of a nyra_2019_complete.csv is shown below. You can apply the same logic to other CSV also you can enumerate the index and its value.</p>\n<p><a href=\"https://postimg.cc/zbwjRqwV\" target=\"_blank\"><img src=\"https://i.postimg.cc/V6pHp6dB/Screen-Shot-2022-09-09-at-12-54-16-AM.png\" alt=\"Screen-Shot-2022-09-09-at-12-54-16-AM.png\"></a></p>\n<p>Hope this helps.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1932452,
          "author_name": "Elisabetta Crescio",
          "author_url": "",
          "post_date": "2022-09-09T16:11:18.843000",
          "content": "<p>Thank you! I know that it's not a big problem, it was just to let people know 😃</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1935033,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-09-11T18:48:46.113000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1931457": "Numerous posts have asked to provide information on the position of each horse at the finish. While a few posts actually showed how to figure this out, because of the timing interval (4x per second) of the data, there is a small chance that a close finish might not be accurately reflected in the data. We have replaced the old files with the new ones (so the names won't look any different) and updated the language in the data tab to include the new field.",
    "1935617": "According to the updated data files, there are 34 races where 2 horses finish at the same position.\nFor example, race number 9 on 2019-02-09 at AQU, horses numbers 4 and 8 finished both at first place.\nDoes this make sense?",
    "1931786": "In the new transformed files, the header seems to be missing for nyra_start_table.csv![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F885209%2F043cb46db315c9dbb96bd8aef20babc2%2FMissing%20Header.jpg?generation=1662694627115426&alt=media)\n\nCan you please add the headers back.",
    "1931570": "Hello Joe,\n\nFirstly thank you for providing a new feature of finishing place. However, I have already figured out that feature and it exactly matches the one you provided.\n\nA glance of my ETL data summary can be seen in the below images\n\n1. My ETL Dataset Overview (has **23 features** also **will add two** more **crucial features**)\n[![Screen-Shot-2022-09-08-at-1-35-55-PM.png](https://i.postimg.cc/ZnwvTdxd/Screen-Shot-2022-09-08-at-1-35-55-PM.png)](https://postimg.cc/Mfjp5X46)\n\n2. Current Competition Dataset (Not Properly Transformed)\n[![Screen-Shot-2022-09-08-at-1-20-04-PM.png](https://i.postimg.cc/XNDRyPPR/Screen-Shot-2022-09-08-at-1-20-04-PM.png)](https://postimg.cc/4nVLDWsP)\n\nSecondly, I would appreciate it if by any chance I can get the **AGE** of the program_number(horse) that would be a great help for my Machine Learning model to be more robust against aggregation and group basis.\n\nNot only but also if I can get the **trainer**, **owner**, **Sire**, and **Dam** names that would be a big help to me as that feature plays a very important role in drafting strategies.\n\nHope you can provide those data if possible. Thank you.",
    "1931552": "Will take a look at that.",
    "1933632": "Thank you for updating!\n\nLife hack for beginners:\n\n```\ndf = pd.read_csv('../input/big-data-derby-2022/nyra_2019_complete.csv', header=None)\n\ncols = ['track_id', 'race_date', 'race_number', 'program_number', 'trakus_index', 'latitude', 'longitude', 'distance_id', 'course_type', 'track_condition', 'run_up_distance', 'race_type', 'purse', 'post_time', 'weight_carried', 'jockey', 'odds', 'final_place']\n\ndf.rename({i: col for i, col in enumerate(cols)}, axis=1, inplace=True)\n```",
    "1931534": "I think in the new files the first row with the name of the columns is missing, in nyra_2019_complete.csv, and nyra_start_table.csv..(noticed on 8th September)\n ",
    "1935033": ""
  }
}