{
  "id": 607720,
  "title": "Clarification on Leaderboard Split(public/private)",
  "url": "/competitions/detecting-reversal-points-in-us-equities/discussion/607720",
  "author_name": "",
  "post_date": "2025-09-15T16:56:39.644005400Z",
  "votes": null,
  "comment_count": 4,
  "views": 0,
  "content": "<p><a href=\"https://www.kaggle.com/ozzymet\" target=\"_blank\">@ozzymet</a> :<br>\nHi, the competition description mentions have a public/private leaderboard split:</p>\n<blockquote>\n  <p>Usage column determines public/private test split:<br>\n  \"Public\": Used for public leaderboard (visible during competition)<br>\n  \"Private\": Used for final private leaderboard (determines winners)<br>\n  Approximately 30% public, 70% private split</p>\n</blockquote>\n<p>However, it appears that the leaderboard is entirely based on all test data, with no actual public/private division.</p>\n<p>Could you confirm that there is no public/private split for this competition?</p>\n<p>Thanks!</p>",
  "messages": [
    {
      "id": "3289219",
      "postDate": "09/15/2025 16:56:39",
      "content": "<p><a href=\"https://www.kaggle.com/ozzymet\" target=\"_blank\">@ozzymet</a> :<br>\nHi, the competition description mentions have a public/private leaderboard split:</p>\n<blockquote>\n  <p>Usage column determines public/private test split:<br>\n  \"Public\": Used for public leaderboard (visible during competition)<br>\n  \"Private\": Used for final private leaderboard (determines winners)<br>\n  Approximately 30% public, 70% private split</p>\n</blockquote>\n<p>However, it appears that the leaderboard is entirely based on all test data, with no actual public/private division.</p>\n<p>Could you confirm that there is no public/private split for this competition?</p>\n<p>Thanks!</p>",
      "rawMarkdown": "ozzymet :\nHi, the competition description mentions have a public/private leaderboard split:\n\n>Usage column determines public/private test split:\n\"Public\": Used for public leaderboard (visible during competition)\n\"Private\": Used for final private leaderboard (determines winners)\nApproximately 30% public, 70% private split\n\nHowever, it appears that the leaderboard is entirely based on all test data, with no actual public/private division.\n\nCould you confirm that there is no public/private split for this competition?\n\nThanks!",
      "votes": null
    },
    {
      "id": "3292798",
      "postDate": "09/22/2025 15:09:21",
      "content": "<p>We have another set to run for final private leaderboard too. Mostly for final validation purposes</p>",
      "rawMarkdown": "We have another set to run for final private leaderboard too. Mostly for final validation purposes",
      "votes": null
    },
    {
      "id": "3293635",
      "postDate": "09/24/2025 09:57:36",
      "content": "<p>Is the timeframe of the hidden private validation set known?<br>\nBecause I think it is not entirely unfeasable to match the \"anonymized\" <code>ticker_id</code> to the real US Stock based on the given features / labels. And if the dataset used for private validation is also in the past (e.g. collected before the competition deadline) one could easily use pre-downloaded historical data to achieve a perfect / near perfect (cheated) score (I assume \"Adam Coles\" is already doing this, as his public LB score seems unrealistic otherwise).<br>\nI am aware that this method is strictily forbidden according to the competition rules:</p>\n<blockquote>\n  <p>External data: Not allowed unless it is explicitly provided or whitelisted in the “Data” tab.<br>\n  No leakage: Do not engineer features that peek into future timestamps.<br>\n  However, this is nowhere mentioned on the Overview / Data page or announced - thus, I think many participants will miss this and it will be quite difficult for the hosts to manually check all submissions for such leakage.</p>\n</blockquote>",
      "rawMarkdown": "Is the timeframe of the hidden private validation set known?\nBecause I think it is not entirely unfeasable to match the \"anonymized\" `ticker_id` to the real US Stock based on the given features / labels. And if the dataset used for private validation is also in the past (e.g. collected before the competition deadline) one could easily use pre-downloaded historical data to achieve a perfect / near perfect (cheated) score (I assume \"Adam Coles\" is already doing this, as his public LB score seems unrealistic otherwise).\nI am aware that this method is strictily forbidden according to the competition rules:\n> External data: Not allowed unless it is explicitly provided or whitelisted in the “Data” tab.\n> No leakage: Do not engineer features that peek into future timestamps.\nHowever, this is nowhere mentioned on the Overview / Data page or announced - thus, I think many participants will miss this and it will be quite difficult for the hosts to manually check all submissions for such leakage.",
      "votes": null
    },
    {
      "id": "3303981",
      "postDate": "10/19/2025 13:15:07",
      "content": "<p>Thanks for clarification. date_ids in your private set are between 2023-04-03 to 2025-01-29 or outside of it? it would be very helpful if you can answer this. Since all dates in public test set are between these dates, this is highly relavant question. Thank you.</p>",
      "rawMarkdown": "Thanks for clarification. date_ids in your private set are between 2023-04-03 to 2025-01-29 or outside of it? it would be very helpful if you can answer this. Since all dates in public test set are between these dates, this is highly relavant question. Thank you.",
      "votes": null
    },
    {
      "id": "3303991",
      "postDate": "10/19/2025 13:39:56",
      "content": "<p>if final standings are calculated based on private set and date ids in private test set are far after 2025-01-31 then all timeseries based modelling and feature engineering will become irrelavant. <br>\neither </p>\n<ol>\n<li>final standing should be based on public test set. OR</li>\n<li>private set should have date_ids within 2023-04-03 to 2025-01-31. if not, must be informed asap so participant can focus on non sequential models.<br>\nI request clarification regarding this. thank you.<br>\n<a href=\"https://www.kaggle.com/ozzymet\" target=\"_blank\">@ozzymet</a> </li>\n</ol>",
      "rawMarkdown": "if final standings are calculated based on private set and date ids in private test set are far after 2025-01-31 then all timeseries based modelling and feature engineering will become irrelavant. \neither \n1. final standing should be based on public test set. OR\n2. private set should have date_ids within 2023-04-03 to 2025-01-31. if not, must be informed asap so participant can focus on non sequential models.\nI request clarification regarding this. thank you.\n@ozzymet",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3292798,
      "author_name": "ozzymet",
      "author_url": "",
      "post_date": "09/22/2025 15:09:21",
      "content": "<p>We have another set to run for final private leaderboard too. Mostly for final validation purposes</p>",
      "votes": null,
      "replies": [
        {
          "id": 3293635,
          "author_name": "patrickaiforfun",
          "author_url": "",
          "post_date": "09/24/2025 09:57:36",
          "content": "<p>Is the timeframe of the hidden private validation set known?<br>\nBecause I think it is not entirely unfeasable to match the \"anonymized\" <code>ticker_id</code> to the real US Stock based on the given features / labels. And if the dataset used for private validation is also in the past (e.g. collected before the competition deadline) one could easily use pre-downloaded historical data to achieve a perfect / near perfect (cheated) score (I assume \"Adam Coles\" is already doing this, as his public LB score seems unrealistic otherwise).<br>\nI am aware that this method is strictily forbidden according to the competition rules:</p>\n<blockquote>\n  <p>External data: Not allowed unless it is explicitly provided or whitelisted in the “Data” tab.<br>\n  No leakage: Do not engineer features that peek into future timestamps.<br>\n  However, this is nowhere mentioned on the Overview / Data page or announced - thus, I think many participants will miss this and it will be quite difficult for the hosts to manually check all submissions for such leakage.</p>\n</blockquote>",
          "votes": null,
          "replies": []
        },
        {
          "id": 3303981,
          "author_name": "meetbabariya",
          "author_url": "",
          "post_date": "10/19/2025 13:15:07",
          "content": "<p>Thanks for clarification. date_ids in your private set are between 2023-04-03 to 2025-01-29 or outside of it? it would be very helpful if you can answer this. Since all dates in public test set are between these dates, this is highly relavant question. Thank you.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 3303991,
          "author_name": "meetbabariya",
          "author_url": "",
          "post_date": "10/19/2025 13:39:56",
          "content": "<p>if final standings are calculated based on private set and date ids in private test set are far after 2025-01-31 then all timeseries based modelling and feature engineering will become irrelavant. <br>\neither </p>\n<ol>\n<li>final standing should be based on public test set. OR</li>\n<li>private set should have date_ids within 2023-04-03 to 2025-01-31. if not, must be informed asap so participant can focus on non sequential models.<br>\nI request clarification regarding this. thank you.<br>\n<a href=\"https://www.kaggle.com/ozzymet\" target=\"_blank\">@ozzymet</a> </li>\n</ol>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3289219": "ozzymet :\nHi, the competition description mentions have a public/private leaderboard split:\n\n>Usage column determines public/private test split:\n\"Public\": Used for public leaderboard (visible during competition)\n\"Private\": Used for final private leaderboard (determines winners)\nApproximately 30% public, 70% private split\n\nHowever, it appears that the leaderboard is entirely based on all test data, with no actual public/private division.\n\nCould you confirm that there is no public/private split for this competition?\n\nThanks!",
    "3292798": "We have another set to run for final private leaderboard too. Mostly for final validation purposes",
    "3293635": "Is the timeframe of the hidden private validation set known?\nBecause I think it is not entirely unfeasable to match the \"anonymized\" `ticker_id` to the real US Stock based on the given features / labels. And if the dataset used for private validation is also in the past (e.g. collected before the competition deadline) one could easily use pre-downloaded historical data to achieve a perfect / near perfect (cheated) score (I assume \"Adam Coles\" is already doing this, as his public LB score seems unrealistic otherwise).\nI am aware that this method is strictily forbidden according to the competition rules:\n> External data: Not allowed unless it is explicitly provided or whitelisted in the “Data” tab.\n> No leakage: Do not engineer features that peek into future timestamps.\nHowever, this is nowhere mentioned on the Overview / Data page or announced - thus, I think many participants will miss this and it will be quite difficult for the hosts to manually check all submissions for such leakage.",
    "3303981": "Thanks for clarification. date_ids in your private set are between 2023-04-03 to 2025-01-29 or outside of it? it would be very helpful if you can answer this. Since all dates in public test set are between these dates, this is highly relavant question. Thank you.",
    "3303991": "if final standings are calculated based on private set and date ids in private test set are far after 2025-01-31 then all timeseries based modelling and feature engineering will become irrelavant. \neither \n1. final standing should be based on public test set. OR\n2. private set should have date_ids within 2023-04-03 to 2025-01-31. if not, must be informed asap so participant can focus on non sequential models.\nI request clarification regarding this. thank you.\n@ozzymet"
  },
  "source": "meta"
}