{
  "id": 619210,
  "title": "End of the contest",
  "url": "/competitions/alpha-radar-solana-sprint/discussion/619210",
  "author_name": "",
  "post_date": "2025-11-13T17:47:15.130429600Z",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi everyone.\nWe have been going through all submissions and will continue to do so through next week. As of right now, no one has been able to hit the 75% threshold from the competition using the contest parameters but we are discussing ways to potentially lower the threshold to keep everything moving forward. We will keep you posted on that, and thank you for all of your submissions. If you haven't yet sent your notebooks, please do. \nThanks\nTwain </p>",
  "messages": [
    {
      "id": "3322780",
      "postDate": "11/13/2025 17:47:15",
      "content": "<p>Hi everyone.\nWe have been going through all submissions and will continue to do so through next week. As of right now, no one has been able to hit the 75% threshold from the competition using the contest parameters but we are discussing ways to potentially lower the threshold to keep everything moving forward. We will keep you posted on that, and thank you for all of your submissions. If you haven't yet sent your notebooks, please do. \nThanks\nTwain </p>",
      "rawMarkdown": "Hi everyone.\nWe have been going through all submissions and will continue to do so through next week. As of right now, no one has been able to hit the 75% threshold from the competition using the contest parameters but we are discussing ways to potentially lower the threshold to keep everything moving forward. We will keep you posted on that, and thank you for all of your submissions. If you haven't yet sent your notebooks, please do. \nThanks\nTwain",
      "votes": null
    },
    {
      "id": "3322942",
      "postDate": "11/13/2025 21:32:36",
      "content": "<p>Only top ten are supposed to send notebook or everybody?</p>",
      "rawMarkdown": "Only top ten are supposed to send notebook or everybody?",
      "votes": null
    },
    {
      "id": "3322943",
      "postDate": "11/13/2025 21:38:42",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/twainayar\" target=\"_blank\">@twainayar</a>, \nDespite many attempts, it seems that we have not been able to join you during the competition which explains our lack of guidance regarding what we submitted as final solutions, and the fact that we optimized jaccard index regardless of the recall constraint.</p>\n<p>As for the notebooks, do you expect them to have the constrained models or the ones that can reproduce leaderboard score ?</p>",
      "rawMarkdown": "Hi @twainayar, \nDespite many attempts, it seems that we have not been able to join you during the competition which explains our lack of guidance regarding what we submitted as final solutions, and the fact that we optimized jaccard index regardless of the recall constraint.\n\nAs for the notebooks, do you expect them to have the constrained models or the ones that can reproduce leaderboard score ?",
      "votes": null
    },
    {
      "id": "3324092",
      "postDate": "11/14/2025 17:31:12",
      "content": "<p>Thanks for the update.</p>\n<p>For context, my approach for selecting final submissions was as followed:\nSince the test-set labels distribution was unknown, I relied on local CV as the only available proxy.\nMy two selected final submissions reached respectively 0.76 and 0.79 recall on local CV.</p>\n<p>These recall levels came from selecting the top-x% of CatBoost probability scores on each run, where x was chosen in order to satisfy recall constraint while preserving Jaccard.\nBeing overly safe by pushing the percentage higher would have unnecessarily hurt Jaccard and handed other competitors an advantage.\nThis fixed recall constraint unfortunately means that selecting a cutoff involves an unavoidable amount of randomness/luck - which is imo a bad metric design - since its success depended on how well local CV happened to align with the hidden test distribution.</p>\n<p>I assumed my chosen submissions were safe enough - apparently not.</p>\n<p>Given your message that no submissions have reached the 75% threshold on the test set, could you clarify a few points?</p>\n<ul>\n<li>Are you able to share the recall values of our submissions on the test set, or alternatively release the test set for post-competition analysis ?</li>\n<li>Will submissions that do not reach 75% recall still be considered eligible for prizes if the threshold is revised?</li>\n<li>If the threshold is revised, can participants select alternative submissions for final ranking? The current submissions were optimized specifically for this specific recall constraint, not purely for maximum Jaccard.</li>\n</ul>\n<p>Thank you</p>",
      "rawMarkdown": "Thanks for the update.\n\nFor context, my approach for selecting final submissions was as followed:\nSince the test-set labels distribution was unknown, I relied on local CV as the only available proxy.\nMy two selected final submissions reached respectively 0.76 and 0.79 recall on local CV.\n\nThese recall levels came from selecting the top-x% of CatBoost probability scores on each run, where x was chosen in order to satisfy recall constraint while preserving Jaccard.\nBeing overly safe by pushing the percentage higher would have unnecessarily hurt Jaccard and handed other competitors an advantage.\nThis fixed recall constraint unfortunately means that selecting a cutoff involves an unavoidable amount of randomness/luck - which is imo a bad metric design - since its success depended on how well local CV happened to align with the hidden test distribution.\n\nI assumed my chosen submissions were safe enough - apparently not.\n\nGiven your message that no submissions have reached the 75% threshold on the test set, could you clarify a few points?\n\n- Are you able to share the recall values of our submissions on the test set, or alternatively release the test set for post-competition analysis ?\n- Will submissions that do not reach 75% recall still be considered eligible for prizes if the threshold is revised?\n- If the threshold is revised, can participants select alternative submissions for final ranking? The current submissions were optimized specifically for this specific recall constraint, not purely for maximum Jaccard.\n \n\nThank you",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3322942,
      "author_name": "dagloxkankwanda",
      "author_url": "",
      "post_date": "11/13/2025 21:32:36",
      "content": "<p>Only top ten are supposed to send notebook or everybody?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3322943,
      "author_name": "yannfb",
      "author_url": "",
      "post_date": "11/13/2025 21:38:42",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/twainayar\" target=\"_blank\">@twainayar</a>, \nDespite many attempts, it seems that we have not been able to join you during the competition which explains our lack of guidance regarding what we submitted as final solutions, and the fact that we optimized jaccard index regardless of the recall constraint.</p>\n<p>As for the notebooks, do you expect them to have the constrained models or the ones that can reproduce leaderboard score ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3324092,
      "author_name": "juliencst",
      "author_url": "",
      "post_date": "11/14/2025 17:31:12",
      "content": "<p>Thanks for the update.</p>\n<p>For context, my approach for selecting final submissions was as followed:\nSince the test-set labels distribution was unknown, I relied on local CV as the only available proxy.\nMy two selected final submissions reached respectively 0.76 and 0.79 recall on local CV.</p>\n<p>These recall levels came from selecting the top-x% of CatBoost probability scores on each run, where x was chosen in order to satisfy recall constraint while preserving Jaccard.\nBeing overly safe by pushing the percentage higher would have unnecessarily hurt Jaccard and handed other competitors an advantage.\nThis fixed recall constraint unfortunately means that selecting a cutoff involves an unavoidable amount of randomness/luck - which is imo a bad metric design - since its success depended on how well local CV happened to align with the hidden test distribution.</p>\n<p>I assumed my chosen submissions were safe enough - apparently not.</p>\n<p>Given your message that no submissions have reached the 75% threshold on the test set, could you clarify a few points?</p>\n<ul>\n<li>Are you able to share the recall values of our submissions on the test set, or alternatively release the test set for post-competition analysis ?</li>\n<li>Will submissions that do not reach 75% recall still be considered eligible for prizes if the threshold is revised?</li>\n<li>If the threshold is revised, can participants select alternative submissions for final ranking? The current submissions were optimized specifically for this specific recall constraint, not purely for maximum Jaccard.</li>\n</ul>\n<p>Thank you</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3322780": "Hi everyone.\nWe have been going through all submissions and will continue to do so through next week. As of right now, no one has been able to hit the 75% threshold from the competition using the contest parameters but we are discussing ways to potentially lower the threshold to keep everything moving forward. We will keep you posted on that, and thank you for all of your submissions. If you haven't yet sent your notebooks, please do. \nThanks\nTwain",
    "3322942": "Only top ten are supposed to send notebook or everybody?",
    "3322943": "Hi @twainayar, \nDespite many attempts, it seems that we have not been able to join you during the competition which explains our lack of guidance regarding what we submitted as final solutions, and the fact that we optimized jaccard index regardless of the recall constraint.\n\nAs for the notebooks, do you expect them to have the constrained models or the ones that can reproduce leaderboard score ?",
    "3324092": "Thanks for the update.\n\nFor context, my approach for selecting final submissions was as followed:\nSince the test-set labels distribution was unknown, I relied on local CV as the only available proxy.\nMy two selected final submissions reached respectively 0.76 and 0.79 recall on local CV.\n\nThese recall levels came from selecting the top-x% of CatBoost probability scores on each run, where x was chosen in order to satisfy recall constraint while preserving Jaccard.\nBeing overly safe by pushing the percentage higher would have unnecessarily hurt Jaccard and handed other competitors an advantage.\nThis fixed recall constraint unfortunately means that selecting a cutoff involves an unavoidable amount of randomness/luck - which is imo a bad metric design - since its success depended on how well local CV happened to align with the hidden test distribution.\n\nI assumed my chosen submissions were safe enough - apparently not.\n\nGiven your message that no submissions have reached the 75% threshold on the test set, could you clarify a few points?\n\n- Are you able to share the recall values of our submissions on the test set, or alternatively release the test set for post-competition analysis ?\n- Will submissions that do not reach 75% recall still be considered eligible for prizes if the threshold is revised?\n- If the threshold is revised, can participants select alternative submissions for final ranking? The current submissions were optimized specifically for this specific recall constraint, not purely for maximum Jaccard.\n \n\nThank you"
  },
  "source": "meta"
}