{
  "id": 295015,
  "title": "Cross Validation Strategy: Best Score vs. AML approach",
  "url": "/competitions/cs5785-fall-2021-final/discussion/295015",
  "author_name": "Abraham Hill",
  "post_date": "2021-12-14T01:11:29.464000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Depending on which models you are using, you may have noticed (like we did) that doing cross validation using the one standard error rule actually lowered the score slightly. Although the goal is to maximize the score, there are still benefits to using simpler models: they are easier to interpret, and in some cases take less time and memory to run.</p>\n<p>Also, keep in mind that we only use 50% of the testing data in our submissions before the deadline, so it's possible that a submission using the AML method for cross validation that appears to decrease the score will actually <em>increase</em> the score once all the testing data is revealed.</p>\n<p>One strategy is to include submissions with both approaches in the final five, but if you only use one approach, I recommend the AML method!</p>",
  "messages": [
    {
      "id": 1617233,
      "postDate": "2021-12-14T01:11:29.463Z",
      "content": "<p>Depending on which models you are using, you may have noticed (like we did) that doing cross validation using the one standard error rule actually lowered the score slightly. Although the goal is to maximize the score, there are still benefits to using simpler models: they are easier to interpret, and in some cases take less time and memory to run.</p>\n<p>Also, keep in mind that we only use 50% of the testing data in our submissions before the deadline, so it's possible that a submission using the AML method for cross validation that appears to decrease the score will actually <em>increase</em> the score once all the testing data is revealed.</p>\n<p>One strategy is to include submissions with both approaches in the final five, but if you only use one approach, I recommend the AML method!</p>",
      "rawMarkdown": "Depending on which models you are using, you may have noticed (like we did) that doing cross validation using the one standard error rule actually lowered the score slightly. Although the goal is to maximize the score, there are still benefits to using simpler models: they are easier to interpret, and in some cases take less time and memory to run.\n\nAlso, keep in mind that we only use 50% of the testing data in our submissions before the deadline, so it's possible that a submission using the AML method for cross validation that appears to decrease the score will actually *increase* the score once all the testing data is revealed.\n\nOne strategy is to include submissions with both approaches in the final five, but if you only use one approach, I recommend the AML method!\n\n\n ",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1617233": "Depending on which models you are using, you may have noticed (like we did) that doing cross validation using the one standard error rule actually lowered the score slightly. Although the goal is to maximize the score, there are still benefits to using simpler models: they are easier to interpret, and in some cases take less time and memory to run.\n\nAlso, keep in mind that we only use 50% of the testing data in our submissions before the deadline, so it's possible that a submission using the AML method for cross validation that appears to decrease the score will actually *increase* the score once all the testing data is revealed.\n\nOne strategy is to include submissions with both approaches in the final five, but if you only use one approach, I recommend the AML method!\n\n\n "
  }
}