I'm reading the rules, discussion and public notebooks for last three days. But I haven't understood what exactly we have to predict? And if the paper refers to several datasets, how to build the answer so the jaccard similarity to the right answer will be high?
", "messages": [ { "id": "1255480", "postDate": "03/28/2021 20:52:04", "content": "I'm reading the rules, discussion and public notebooks for last three days. But I haven't understood what exactly we have to predict? And if the paper refers to several datasets, how to build the answer so the jaccard similarity to the right answer will be high?
", "rawMarkdown": "I'm reading the rules, discussion and public notebooks for last three days. But I haven't understood what exactly we have to predict? And if the paper refers to several datasets, how to build the answer so the jaccard similarity to the right answer will be high?", "votes": null }, { "id": "1256232", "postDate": "03/29/2021 17:17:16", "content": "it seems the answer is here
\nhttps://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/228339
The task is:
\nIn other words, we are given some sentences and have to find the words that refer to a dataset. For example, the sentences could contain the words \"According to the Aging Integrated Database …\". Then the last three words would refer to a dataset (match these three words).
\nIt is not an information retrieval task (find matching documents to a query), because the datasets are not fixed. The test set can have other datasets.
\nWe are dealing here with named entity recognition (NER). However, the ground truth labels can be noisy (refer to this post). While the text can contain the words \"Aging Integrated Database (AGID)\", the abbreviation \"(AGID)\" might not be included in the ground truth.
\nWhen there are multiple datasets in a text, then you separate the strings by a \"|\". For example, \"Aging Integrated Database|AGID\".
", "rawMarkdown": "The task is: \n\n- given a text from a publication (abstract, introduction, conclusion, ...)\n- predict zero or more strings (name of the dataset) e.g. \"Aging Integrated Database\"\n- with the metric being Jaccard-based F0.5.\n\nIn other words, we are given some sentences and have to find the words that refer to a dataset. For example, the sentences could contain the words \"According to the Aging Integrated Database ...\". Then the last three words would refer to a dataset (match these three words).\n\nIt is not an information retrieval task (find matching documents to a query), because the datasets are not fixed. The test set can have other datasets.\n\nWe are dealing here with named entity recognition (NER). However, the ground truth labels can be noisy (refer to this [post](https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/228337)). While the text can contain the words \"Aging Integrated Database (AGID)\", the abbreviation \"(AGID)\" might not be included in the ground truth.\n\nWhen there are multiple datasets in a text, then you separate the strings by a \"|\". For example, \"Aging Integrated Database|AGID\".", "votes": null }, { "id": "1327678", "postDate": "05/29/2021 14:21:21", "content": "Do we have to find any dataset(present outside the cleaned labels) or the datasets already present in the training data(cleaned labels)….
", "rawMarkdown": "Do we have to find any dataset(present outside the cleaned labels) or the datasets already present in the training data(cleaned labels)....", "votes": null }, { "id": "1327701", "postDate": "05/29/2021 14:48:13", "content": "@eryash15 The hidden test set has datasets which are not present in the training set. Our models should be able to generalize for unknown datasets to ace in this competition.
", "rawMarkdown": "eryash15 The hidden test set has datasets which are not present in the training set. Our models should be able to generalize for unknown datasets to ace in this competition.", "votes": null } ], "comments": [ { "id": 1256232, "author_name": "robotmsu92", "author_url": "", "post_date": "03/29/2021 17:17:16", "content": "it seems the answer is here
\nhttps://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/228339
The task is:
\nIn other words, we are given some sentences and have to find the words that refer to a dataset. For example, the sentences could contain the words \"According to the Aging Integrated Database …\". Then the last three words would refer to a dataset (match these three words).
\nIt is not an information retrieval task (find matching documents to a query), because the datasets are not fixed. The test set can have other datasets.
\nWe are dealing here with named entity recognition (NER). However, the ground truth labels can be noisy (refer to this post). While the text can contain the words \"Aging Integrated Database (AGID)\", the abbreviation \"(AGID)\" might not be included in the ground truth.
\nWhen there are multiple datasets in a text, then you separate the strings by a \"|\". For example, \"Aging Integrated Database|AGID\".
", "votes": null, "replies": [ { "id": 1327678, "author_name": "eryash15", "author_url": "", "post_date": "05/29/2021 14:21:21", "content": "Do we have to find any dataset(present outside the cleaned labels) or the datasets already present in the training data(cleaned labels)….
", "votes": null, "replies": [] }, { "id": 1327701, "author_name": "kartikbhargav", "author_url": "", "post_date": "05/29/2021 14:48:13", "content": "@eryash15 The hidden test set has datasets which are not present in the training set. Our models should be able to generalize for unknown datasets to ace in this competition.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1255480": "I'm reading the rules, discussion and public notebooks for last three days. But I haven't understood what exactly we have to predict? And if the paper refers to several datasets, how to build the answer so the jaccard similarity to the right answer will be high?", "1256232": "it seems the answer is here\nhttps://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/228339", "1256483": "The task is: \n\n- given a text from a publication (abstract, introduction, conclusion, ...)\n- predict zero or more strings (name of the dataset) e.g. \"Aging Integrated Database\"\n- with the metric being Jaccard-based F0.5.\n\nIn other words, we are given some sentences and have to find the words that refer to a dataset. For example, the sentences could contain the words \"According to the Aging Integrated Database ...\". Then the last three words would refer to a dataset (match these three words).\n\nIt is not an information retrieval task (find matching documents to a query), because the datasets are not fixed. The test set can have other datasets.\n\nWe are dealing here with named entity recognition (NER). However, the ground truth labels can be noisy (refer to this [post](https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/228337)). While the text can contain the words \"Aging Integrated Database (AGID)\", the abbreviation \"(AGID)\" might not be included in the ground truth.\n\nWhen there are multiple datasets in a text, then you separate the strings by a \"|\". For example, \"Aging Integrated Database|AGID\".", "1327678": "Do we have to find any dataset(present outside the cleaned labels) or the datasets already present in the training data(cleaned labels)....", "1327701": "eryash15 The hidden test set has datasets which are not present in the training set. Our models should be able to generalize for unknown datasets to ace in this competition." }, "source": "meta" }