{
  "id": 316033,
  "title": "Is this a Code Competition? ",
  "url": "/competitions/football-match-probability-prediction/discussion/316033",
  "author_name": "",
  "post_date": "2022-03-30T23:07:03.165622600Z",
  "votes": 2,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Having read the Overview and Discussion in some detail, I am still unsure whether this competition is:</p>\n<p>(1) A Code Competition where submitted notebooks are rerun after the closing date by the organisers on an entirely different hidden dataset, not contained within the currently available test data files. In this first case, submitting the existing sample_submission.csv file would fail to produce a valid score, but submitting a notebook generating {Home, Away, Draw} probabilities of {0.3333333,  0.3333333, 0.3333333} for any input features would return a score of 1.09861. Or is it:</p>\n<p>(2) A file submission competition, where the entries not rerun but are evaluated based on a subset of the predictions contained within the already-submitted submission.csv, using a corresponding subset of the available test data that does not contribute to the visible public LB score (the current leaderboard sugggest a 47%:53% public:private split). In this second case, either submitting the existing sample_submission.csv file or submitting a notebook generating {Home, Away, Draw} probabilities of {0.3333333,  0.3333333, 0.3333333} for any input features would both return a score of 1.09861.</p>\n<p>A clarification from <a href=\"https://www.kaggle.com/octosportio\" target=\"_blank\">@octosportio</a> would be greatly appreciated, thanks.</p>",
  "messages": [
    {
      "id": "1740429",
      "postDate": "03/30/2022 23:07:03",
      "content": "<p>Having read the Overview and Discussion in some detail, I am still unsure whether this competition is:</p>\n<p>(1) A Code Competition where submitted notebooks are rerun after the closing date by the organisers on an entirely different hidden dataset, not contained within the currently available test data files. In this first case, submitting the existing sample_submission.csv file would fail to produce a valid score, but submitting a notebook generating {Home, Away, Draw} probabilities of {0.3333333,  0.3333333, 0.3333333} for any input features would return a score of 1.09861. Or is it:</p>\n<p>(2) A file submission competition, where the entries not rerun but are evaluated based on a subset of the predictions contained within the already-submitted submission.csv, using a corresponding subset of the available test data that does not contribute to the visible public LB score (the current leaderboard sugggest a 47%:53% public:private split). In this second case, either submitting the existing sample_submission.csv file or submitting a notebook generating {Home, Away, Draw} probabilities of {0.3333333,  0.3333333, 0.3333333} for any input features would both return a score of 1.09861.</p>\n<p>A clarification from <a href=\"https://www.kaggle.com/octosportio\" target=\"_blank\">@octosportio</a> would be greatly appreciated, thanks.</p>",
      "rawMarkdown": "Having read the Overview and Discussion in some detail, I am still unsure whether this competition is:\n\n(1) A Code Competition where submitted notebooks are rerun after the closing date by the organisers on an entirely different hidden dataset, not contained within the currently available test data files. In this first case, submitting the existing sample_submission.csv file would fail to produce a valid score, but submitting a notebook generating {Home, Away, Draw} probabilities of {0.3333333,  0.3333333, 0.3333333} for any input features would return a score of 1.09861. Or is it:\n\n(2) A file submission competition, where the entries not rerun but are evaluated based on a subset of the predictions contained within the already-submitted submission.csv, using a corresponding subset of the available test data that does not contribute to the visible public LB score (the current leaderboard sugggest a 47%:53% public:private split). In this second case, either submitting the existing sample_submission.csv file or submitting a notebook generating {Home, Away, Draw} probabilities of {0.3333333,  0.3333333, 0.3333333} for any input features would both return a score of 1.09861.\n\nA clarification from @octosportio would be greatly appreciated, thanks.",
      "votes": null
    },
    {
      "id": "1740692",
      "postDate": "03/31/2022 07:29:14",
      "content": "<p>Hi John,</p>\n<p>Yes and no, we won't rerun on a hidden dataset so 2. is what will happen.</p>\n<p>As we said in another <a href=\"https://www.kaggle.com/competitions/football-match-probability-prediction/discussion/314744\" target=\"_blank\">discussion</a> and due to the time-series nature of the competition, the test set can be predicted using an obvious leakage from the test set. The test set should not be a part of your training at any time (in any competition), I am not sure why some competitors thought otherwise. So we will review the solutions to make sure it is the case.</p>\n<p>Right now <strong>everyone above the Bookmaker score is using the leak and is disqualified</strong>. So for now you are leading the competition.</p>\n<p>That said a part of the test set can not be leaked (10%) but we want to keep the 47%:53% public-private split so you can have a better correlation between your CV, public and private score.</p>\n<p>Hope that helps,</p>\n<p>Good luck</p>",
      "rawMarkdown": "Hi John,\n\nYes and no, we won't rerun on a hidden dataset so 2. is what will happen.\n\nAs we said in another [discussion](https://www.kaggle.com/competitions/football-match-probability-prediction/discussion/314744) and due to the time-series nature of the competition, the test set can be predicted using an obvious leakage from the test set. The test set should not be a part of your training at any time (in any competition), I am not sure why some competitors thought otherwise. So we will review the solutions to make sure it is the case.\n\nRight now **everyone above the Bookmaker score is using the leak and is disqualified**. So for now you are leading the competition.\n\nThat said a part of the test set can not be leaked (10%) but we want to keep the 47%:53% public-private split so you can have a better correlation between your CV, public and private score.\n\nHope that helps,\n\nGood luck",
      "votes": null
    },
    {
      "id": "1740778",
      "postDate": "03/31/2022 09:05:47",
      "content": "<p>In order to check your solution has no leakage, and to make sure your are not using external data, we need to be able to run your solution. </p>",
      "rawMarkdown": "In order to check your solution has no leakage, and to make sure your are not using external data, we need to be able to run your solution.",
      "votes": null
    },
    {
      "id": "1740784",
      "postDate": "03/31/2022 09:10:24",
      "content": "<p>Thanks for the clarifications <a href=\"https://www.kaggle.com/octosportio\" target=\"_blank\">@octosportio</a> - that is very helpful.  </p>",
      "rawMarkdown": "Thanks for the clarifications @octosportio - that is very helpful.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1740692,
      "author_name": "octosportio",
      "author_url": "",
      "post_date": "03/31/2022 07:29:14",
      "content": "<p>Hi John,</p>\n<p>Yes and no, we won't rerun on a hidden dataset so 2. is what will happen.</p>\n<p>As we said in another <a href=\"https://www.kaggle.com/competitions/football-match-probability-prediction/discussion/314744\" target=\"_blank\">discussion</a> and due to the time-series nature of the competition, the test set can be predicted using an obvious leakage from the test set. The test set should not be a part of your training at any time (in any competition), I am not sure why some competitors thought otherwise. So we will review the solutions to make sure it is the case.</p>\n<p>Right now <strong>everyone above the Bookmaker score is using the leak and is disqualified</strong>. So for now you are leading the competition.</p>\n<p>That said a part of the test set can not be leaked (10%) but we want to keep the 47%:53% public-private split so you can have a better correlation between your CV, public and private score.</p>\n<p>Hope that helps,</p>\n<p>Good luck</p>",
      "votes": null,
      "replies": [
        {
          "id": 1740778,
          "author_name": "octosportio",
          "author_url": "",
          "post_date": "03/31/2022 09:05:47",
          "content": "<p>In order to check your solution has no leakage, and to make sure your are not using external data, we need to be able to run your solution. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1740784,
          "author_name": "jbomitchell",
          "author_url": "",
          "post_date": "03/31/2022 09:10:24",
          "content": "<p>Thanks for the clarifications <a href=\"https://www.kaggle.com/octosportio\" target=\"_blank\">@octosportio</a> - that is very helpful.  </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1740429": "Having read the Overview and Discussion in some detail, I am still unsure whether this competition is:\n\n(1) A Code Competition where submitted notebooks are rerun after the closing date by the organisers on an entirely different hidden dataset, not contained within the currently available test data files. In this first case, submitting the existing sample_submission.csv file would fail to produce a valid score, but submitting a notebook generating {Home, Away, Draw} probabilities of {0.3333333,  0.3333333, 0.3333333} for any input features would return a score of 1.09861. Or is it:\n\n(2) A file submission competition, where the entries not rerun but are evaluated based on a subset of the predictions contained within the already-submitted submission.csv, using a corresponding subset of the available test data that does not contribute to the visible public LB score (the current leaderboard sugggest a 47%:53% public:private split). In this second case, either submitting the existing sample_submission.csv file or submitting a notebook generating {Home, Away, Draw} probabilities of {0.3333333,  0.3333333, 0.3333333} for any input features would both return a score of 1.09861.\n\nA clarification from @octosportio would be greatly appreciated, thanks.",
    "1740692": "Hi John,\n\nYes and no, we won't rerun on a hidden dataset so 2. is what will happen.\n\nAs we said in another [discussion](https://www.kaggle.com/competitions/football-match-probability-prediction/discussion/314744) and due to the time-series nature of the competition, the test set can be predicted using an obvious leakage from the test set. The test set should not be a part of your training at any time (in any competition), I am not sure why some competitors thought otherwise. So we will review the solutions to make sure it is the case.\n\nRight now **everyone above the Bookmaker score is using the leak and is disqualified**. So for now you are leading the competition.\n\nThat said a part of the test set can not be leaked (10%) but we want to keep the 47%:53% public-private split so you can have a better correlation between your CV, public and private score.\n\nHope that helps,\n\nGood luck",
    "1740778": "In order to check your solution has no leakage, and to make sure your are not using external data, we need to be able to run your solution.",
    "1740784": "Thanks for the clarifications @octosportio - that is very helpful."
  },
  "source": "meta"
}