Hey, doing a quick EDA, I noticed how the train label data is quite noisy (as you already know for sure), and sometimes clean_text(dataset_label) does not match cleaned_label. The evaluation page says that it should …
Just wanted to let you know. And listen to your opinions about that matter. Thanks!
\nHere you can find a simple notebook that shows the issue: clean_text(dataset_label) != cleaned_label 🧐 !?
", "messages": [ { "id": "1254443", "postDate": "03/27/2021 17:19:30", "content": "Hey, doing a quick EDA, I noticed how the train label data is quite noisy (as you already know for sure), and sometimes clean_text(dataset_label) does not match cleaned_label. The evaluation page says that it should …
Just wanted to let you know. And listen to your opinions about that matter. Thanks!
\nHere you can find a simple notebook that shows the issue: clean_text(dataset_label) != cleaned_label 🧐 !?
", "rawMarkdown": "Hey, doing a quick EDA, I noticed how the train label data is quite noisy (as you already know for sure), and sometimes `clean_text(dataset_label)` does not match `cleaned_label `. The [evaluation](https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/overview/evaluation) page says that it should ...\n\nJust wanted to let you know. And listen to your opinions about that matter. Thanks!\n\nHere you can find a simple notebook that shows the issue: [clean_text(dataset_label) != cleaned_label 🧐 !? ](https://www.kaggle.com/jonathanbesomi/clean-text-dataset-label-cleaned-label)", "votes": null }, { "id": "1254665", "postDate": "03/28/2021 00:02:41", "content": "What do you call noisy? It is clearly said that labels are incomplete, but did you find wrong labels?
", "rawMarkdown": "What do you call noisy? It is clearly said that labels are incomplete, but did you find wrong labels?", "votes": null }, { "id": "1254784", "postDate": "03/28/2021 06:03:38", "content": "Thank you @jonathanbesomi for this insight. We may need to look closely and validate the contents of the target.
", "rawMarkdown": "Thank you @jonathanbesomi for this insight. We may need to look closely and validate the contents of the target.", "votes": null }, { "id": "1255000", "postDate": "03/28/2021 10:19:58", "content": "First, I'm very new to this competition, so I haven't figured out most of the things yet. But, just by looking at some labeled examples, you can quickly see that labels have been extracted using an automated method (I would guess unsupervised, but probably not fully.). This reminds me the Tweet Sentiment challenge …
\nFor instance, I would assume that cleaned labels are normalized, instead is not the case:
\nsars cov 2 genome sequences (244 examples in train), genome sequence of sars cov 2 (222 …), genome sequences of sars cov 2 (158), sars cov 2 genome sequence (134 …) are all cleaned labels in the training dataset. According to the \"Jaccard\" function provided, they are all different, yet it's clear to us that they refer to the actual same dataset …
Hi, thanks for your feedback. The clean_text function that we used did not include .strip() and I've removed it from the Evaluation page, as it will result in labels not matching. The whitespace is ignored in the Jaccard function; we'll be updating that in the Evaluation page to align better with the C# Jaccard code.
While using clean_text function for the text and the dataset_label at the same time, you will get the desired result!
", "rawMarkdown": "While using clean_text function for the text and the dataset_label at the same time, you will get the desired result!", "votes": null } ], "comments": [ { "id": 1254665, "author_name": "cpmpml", "author_url": "", "post_date": "03/28/2021 00:02:41", "content": "What do you call noisy? It is clearly said that labels are incomplete, but did you find wrong labels?
", "votes": null, "replies": [ { "id": 1255000, "author_name": "jonathanbesomi", "author_url": "", "post_date": "03/28/2021 10:19:58", "content": "First, I'm very new to this competition, so I haven't figured out most of the things yet. But, just by looking at some labeled examples, you can quickly see that labels have been extracted using an automated method (I would guess unsupervised, but probably not fully.). This reminds me the Tweet Sentiment challenge …
\nFor instance, I would assume that cleaned labels are normalized, instead is not the case:
\nsars cov 2 genome sequences (244 examples in train), genome sequence of sars cov 2 (222 …), genome sequences of sars cov 2 (158), sars cov 2 genome sequence (134 …) are all cleaned labels in the training dataset. According to the \"Jaccard\" function provided, they are all different, yet it's clear to us that they refer to the actual same dataset …
Thank you @jonathanbesomi for this insight. We may need to look closely and validate the contents of the target.
", "votes": null, "replies": [] }, { "id": 1255966, "author_name": "philculliton", "author_url": "", "post_date": "03/29/2021 12:27:42", "content": "Hi, thanks for your feedback. The clean_text function that we used did not include .strip() and I've removed it from the Evaluation page, as it will result in labels not matching. The whitespace is ignored in the Jaccard function; we'll be updating that in the Evaluation page to align better with the C# Jaccard code.
While using clean_text function for the text and the dataset_label at the same time, you will get the desired result!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1254443": "Hey, doing a quick EDA, I noticed how the train label data is quite noisy (as you already know for sure), and sometimes `clean_text(dataset_label)` does not match `cleaned_label `. The [evaluation](https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/overview/evaluation) page says that it should ...\n\nJust wanted to let you know. And listen to your opinions about that matter. Thanks!\n\nHere you can find a simple notebook that shows the issue: [clean_text(dataset_label) != cleaned_label 🧐 !? ](https://www.kaggle.com/jonathanbesomi/clean-text-dataset-label-cleaned-label)", "1254665": "What do you call noisy? It is clearly said that labels are incomplete, but did you find wrong labels?", "1254784": "Thank you @jonathanbesomi for this insight. We may need to look closely and validate the contents of the target.", "1255000": "First, I'm very new to this competition, so I haven't figured out most of the things yet. But, just by looking at some labeled examples, you can quickly see that labels have been extracted using an automated method (I would guess unsupervised, but probably not fully.). This reminds me the Tweet Sentiment challenge ...\n\nFor instance, I would assume that cleaned labels are normalized, instead is not the case:\n\n`sars cov 2 genome sequences` (244 examples in train), `genome sequence of sars cov 2` (222 ...), `genome sequences of sars cov 2` (158), `sars cov 2 genome sequence` (134 ...) are all cleaned labels in the training dataset. According to the \"Jaccard\" function provided, they are all different, yet it's clear to us that they refer to the actual same dataset ...", "1255966": "Hi, thanks for your feedback. The `clean_text` function that we used did *not* include `.strip()` and I've removed it from the Evaluation page, as it will result in labels not matching. The whitespace is ignored in the Jaccard function; we'll be updating that in the Evaluation page to align better with the C# Jaccard code.", "1271182": "While using clean_text function for the text and the dataset_label at the same time, you will get the desired result!" }, "source": "meta" }