{
  "id": 418456,
  "title": "The MFO imbalance problem",
  "url": "/competitions/cafa-5-protein-function-prediction/discussion/418456",
  "author_name": "ZMC",
  "post_date": "2023-06-20T17:42:33.495000",
  "votes": 6,
  "comment_count": 0,
  "views": 0,
  "content": "<p>I have noticed that my local evaluations of BPO and CCO terms correlates very well with scores from the test set but this does not hold true for MFO terms. As touched upon in EDA of others, MFO terms are both significantly less frequent than BPO and CCO terms as well as being weighted towards a few terms dominating protein-GO frequency histograms (as opposed to BPO in particular). </p>\n<p>My question is that given poor correlation, is it better to trust my local evaluation of model against a larger pool of proteins or stick to the test evaluations being given? There is definitely some shift in protein origin and function between train/test set and I have not obtained any distinct insights as to what those might be. Experienced data scientists please help :) </p>",
  "messages": [
    {
      "id": 2310933,
      "postDate": "2023-06-20T17:42:33.497Z",
      "content": "<p>I have noticed that my local evaluations of BPO and CCO terms correlates very well with scores from the test set but this does not hold true for MFO terms. As touched upon in EDA of others, MFO terms are both significantly less frequent than BPO and CCO terms as well as being weighted towards a few terms dominating protein-GO frequency histograms (as opposed to BPO in particular). </p>\n<p>My question is that given poor correlation, is it better to trust my local evaluation of model against a larger pool of proteins or stick to the test evaluations being given? There is definitely some shift in protein origin and function between train/test set and I have not obtained any distinct insights as to what those might be. Experienced data scientists please help :) </p>",
      "rawMarkdown": "I have noticed that my local evaluations of BPO and CCO terms correlates very well with scores from the test set but this does not hold true for MFO terms. As touched upon in EDA of others, MFO terms are both significantly less frequent than BPO and CCO terms as well as being weighted towards a few terms dominating protein-GO frequency histograms (as opposed to BPO in particular). \n\nMy question is that given poor correlation, is it better to trust my local evaluation of model against a larger pool of proteins or stick to the test evaluations being given? There is definitely some shift in protein origin and function between train/test set and I have not obtained any distinct insights as to what those might be. Experienced data scientists please help :) ",
      "votes": 6
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2310933": "I have noticed that my local evaluations of BPO and CCO terms correlates very well with scores from the test set but this does not hold true for MFO terms. As touched upon in EDA of others, MFO terms are both significantly less frequent than BPO and CCO terms as well as being weighted towards a few terms dominating protein-GO frequency histograms (as opposed to BPO in particular). \n\nMy question is that given poor correlation, is it better to trust my local evaluation of model against a larger pool of proteins or stick to the test evaluations being given? There is definitely some shift in protein origin and function between train/test set and I have not obtained any distinct insights as to what those might be. Experienced data scientists please help :) "
  }
}