{
  "id": 315706,
  "title": "Validation strategy ",
  "url": "/competitions/football-match-probability-prediction/discussion/315706",
  "author_name": "Redha C.",
  "post_date": "2022-03-29T12:00:01.607000",
  "votes": 2,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hi everyone, </p>\n<p>Regarding the validation strategy, I noticed that the train/test split is time based with the train being a 1y and 5 months long and the test being a 7 months long. <br>\nIf I'm not mistaken, the closer the train/validation split is to the train/test one, the better it is. <br>\nTherefore, I made the choice to take a 7 months validation set, juste like the test set and I made sure that the validation set is in the future compared to the train. </p>\n<p>Is this strategy correct for the purpose of the competition please ? </p>\n<p>Huge thanks in advance to anyone to would help me :) </p>",
  "messages": [
    {
      "id": 1738610,
      "postDate": "2022-03-29T12:00:01.607Z",
      "content": "<p>Hi everyone, </p>\n<p>Regarding the validation strategy, I noticed that the train/test split is time based with the train being a 1y and 5 months long and the test being a 7 months long. <br>\nIf I'm not mistaken, the closer the train/validation split is to the train/test one, the better it is. <br>\nTherefore, I made the choice to take a 7 months validation set, juste like the test set and I made sure that the validation set is in the future compared to the train. </p>\n<p>Is this strategy correct for the purpose of the competition please ? </p>\n<p>Huge thanks in advance to anyone to would help me :) </p>",
      "rawMarkdown": "Hi everyone, \n\nRegarding the validation strategy, I noticed that the train/test split is time based with the train being a 1y and 5 months long and the test being a 7 months long. \nIf I'm not mistaken, the closer the train/validation split is to the train/test one, the better it is. \nTherefore, I made the choice to take a 7 months validation set, juste like the test set and I made sure that the validation set is in the future compared to the train. \n\nIs this strategy correct for the purpose of the competition please ? \n\nHuge thanks in advance to anyone to would help me :) ",
      "votes": 2
    },
    {
      "id": 1738901,
      "postDate": "2022-03-29T15:56:03.530Z",
      "content": "<p>I am not sure if your validation set and test set can seem similar just by setting up the data over the same kind of timeframe. </p>",
      "rawMarkdown": "I am not sure if your validation set and test set can seem similar just by setting up the data over the same kind of timeframe. ",
      "replies": [
        {
          "id": 1739613,
          "postDate": "2022-03-30T07:34:42.680Z",
          "content": "<p>Hello Ravi, thanks a lot for your response :)<br>\nWhat can I do to make the validation set look similar to the test set though ?</p>",
          "rawMarkdown": "Hello Ravi, thanks a lot for your response :)\nWhat can I do to make the validation set look similar to the test set though ?"
        },
        {
          "id": 1741524,
          "postDate": "2022-03-31T21:36:52.073Z",
          "content": "<p>You do not need it similar you just have to make sure than the correlation between your CV loss and the leaderboard loss is high. So try different <a href=\"https://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection\" target=\"_blank\">methods</a>.</p>",
          "rawMarkdown": "You do not need it similar you just have to make sure than the correlation between your CV loss and the leaderboard loss is high. So try different [methods](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection).",
          "votes": 1
        }
      ]
    },
    {
      "id": 1739612,
      "postDate": "2022-03-30T07:34:13.810Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 1738901,
      "author_name": "Ravi Ramakrishnan",
      "author_url": "",
      "post_date": "2022-03-29T15:56:03.530000",
      "content": "<p>I am not sure if your validation set and test set can seem similar just by setting up the data over the same kind of timeframe. </p>",
      "votes": 0,
      "replies": [
        {
          "id": 1739613,
          "author_name": "Redha C.",
          "author_url": "",
          "post_date": "2022-03-30T07:34:42.680000",
          "content": "<p>Hello Ravi, thanks a lot for your response :)<br>\nWhat can I do to make the validation set look similar to the test set though ?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1741524,
          "author_name": "octosport.io",
          "author_url": "",
          "post_date": "2022-03-31T21:36:52.073000",
          "content": "<p>You do not need it similar you just have to make sure than the correlation between your CV loss and the leaderboard loss is high. So try different <a href=\"https://scikit-learn.org/stable/modules/classes.html#module-sklearn.model_selection\" target=\"_blank\">methods</a>.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1739612,
      "author_name": "",
      "author_url": "",
      "post_date": "2022-03-30T07:34:13.810000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1738610": "Hi everyone, \n\nRegarding the validation strategy, I noticed that the train/test split is time based with the train being a 1y and 5 months long and the test being a 7 months long. \nIf I'm not mistaken, the closer the train/validation split is to the train/test one, the better it is. \nTherefore, I made the choice to take a 7 months validation set, juste like the test set and I made sure that the validation set is in the future compared to the train. \n\nIs this strategy correct for the purpose of the competition please ? \n\nHuge thanks in advance to anyone to would help me :) ",
    "1738901": "I am not sure if your validation set and test set can seem similar just by setting up the data over the same kind of timeframe. ",
    "1739612": ""
  }
}