{
  "id": 232916,
  "title": "Evaluation metric - \"adni\" vs \"alzheimer s disease neuroimaging initiative adni\"",
  "url": "/competitions/coleridgeinitiative-show-us-the-data/discussion/232916",
  "author_name": "alicefukg",
  "post_date": "2021-04-16T04:00:16.627000",
  "votes": 3,
  "comment_count": 0,
  "views": 0,
  "content": "<p>During model validation, I found that the cleaned_label for one of the doc is \"alzheimer s disease neuroimaging initiative adni\", whereas my model predicted \"adni\". If I use the function as written in the Evaluation section of this challenge (see below) to compute the Jaccard-based FBeta score, I got 0.167. But the two strings should mean the same thing, intuitively. I'm wondering if this is properly addressed in the backend program used to compute scores on submissions.</p>\n<p>def jaccard(str1, str2): <br>\n    a = set(str1.lower().split()) <br>\n    b = set(str2.lower().split())<br>\n    c = a.intersection(b)<br>\n    return float(len(c)) / (len(a) + len(b) - len(c))</p>",
  "messages": [
    {
      "id": 1275190,
      "postDate": "2021-04-16T04:00:16.627Z",
      "content": "<p>During model validation, I found that the cleaned_label for one of the doc is \"alzheimer s disease neuroimaging initiative adni\", whereas my model predicted \"adni\". If I use the function as written in the Evaluation section of this challenge (see below) to compute the Jaccard-based FBeta score, I got 0.167. But the two strings should mean the same thing, intuitively. I'm wondering if this is properly addressed in the backend program used to compute scores on submissions.</p>\n<p>def jaccard(str1, str2): <br>\n    a = set(str1.lower().split()) <br>\n    b = set(str2.lower().split())<br>\n    c = a.intersection(b)<br>\n    return float(len(c)) / (len(a) + len(b) - len(c))</p>",
      "rawMarkdown": "During model validation, I found that the cleaned_label for one of the doc is \"alzheimer s disease neuroimaging initiative adni\", whereas my model predicted \"adni\". If I use the function as written in the Evaluation section of this challenge (see below) to compute the Jaccard-based FBeta score, I got 0.167. But the two strings should mean the same thing, intuitively. I'm wondering if this is properly addressed in the backend program used to compute scores on submissions.\n\ndef jaccard(str1, str2): \n    a = set(str1.lower().split()) \n    b = set(str2.lower().split())\n    c = a.intersection(b)\n    return float(len(c)) / (len(a) + len(b) - len(c))",
      "votes": 3
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1275190": "During model validation, I found that the cleaned_label for one of the doc is \"alzheimer s disease neuroimaging initiative adni\", whereas my model predicted \"adni\". If I use the function as written in the Evaluation section of this challenge (see below) to compute the Jaccard-based FBeta score, I got 0.167. But the two strings should mean the same thing, intuitively. I'm wondering if this is properly addressed in the backend program used to compute scores on submissions.\n\ndef jaccard(str1, str2): \n    a = set(str1.lower().split()) \n    b = set(str2.lower().split())\n    c = a.intersection(b)\n    return float(len(c)) / (len(a) + len(b) - len(c))"
  }
}