{ "id": 232867, "title": "Fully Labeled Example", "url": "/competitions/coleridgeinitiative-show-us-the-data/discussion/232867", "author_name": "Emanuele Guidotti", "post_date": "2021-04-15T19:13:35.049000", "votes": 4, "comment_count": 0, "views": 0, "content": "

As stated in the evaluation instructions:

\n
\n

The objective of the competition is to identify the mention of datasets within scientific publications. Your predictions will be short excerpts from the publications that appear to note a dataset.

\n
\n

However, it seems to me that the notion of dataset (and the corresponding excerpt) is a bit ambiguous. For istance in publication d7531c85-560f-4ed3-bec2-7a17bdb1b5da I would have never labeled NOAA tidal station as a dataset (which instead is reported in the training file). Or at the very least, I would have used the whole mention NOAA tidal station, Sabine Pass North (Station ID 8770570, PID AV1014).

\n

On the other side, I would have labeled some mentions that are not reported in the training file. In this case, I cannot figure out if my (human) understanding of the text is wrong or if this is simply due to the training set not being fully labeled. In other words, I cannot understand if the mention I would have labeled is missing because a) it should really miss or b) it should be labeled but was removed from the training data.

\n

@philculliton @maggiemd Is it possible to reveal the identifier of at least one publication that is guaranteed to be fully labeled in the training file? I guess this would be of great help in better understanding the task, see also:

\n", "messages": [ { "id": 1274974, "postDate": "2021-04-15T19:13:35.050Z", "content": "

As stated in the evaluation instructions:

\n
\n

The objective of the competition is to identify the mention of datasets within scientific publications. Your predictions will be short excerpts from the publications that appear to note a dataset.

\n
\n

However, it seems to me that the notion of dataset (and the corresponding excerpt) is a bit ambiguous. For istance in publication d7531c85-560f-4ed3-bec2-7a17bdb1b5da I would have never labeled NOAA tidal station as a dataset (which instead is reported in the training file). Or at the very least, I would have used the whole mention NOAA tidal station, Sabine Pass North (Station ID 8770570, PID AV1014).

\n

On the other side, I would have labeled some mentions that are not reported in the training file. In this case, I cannot figure out if my (human) understanding of the text is wrong or if this is simply due to the training set not being fully labeled. In other words, I cannot understand if the mention I would have labeled is missing because a) it should really miss or b) it should be labeled but was removed from the training data.

\n

@philculliton @maggiemd Is it possible to reveal the identifier of at least one publication that is guaranteed to be fully labeled in the training file? I guess this would be of great help in better understanding the task, see also:

\n", "rawMarkdown": "As stated in the evaluation instructions:\n> The objective of the competition is to identify the mention of datasets within scientific publications. Your predictions will be short excerpts from the publications that appear to note a dataset.\n\nHowever, it seems to me that the notion of **dataset** (and the corresponding excerpt) is a bit ambiguous. For istance in publication `d7531c85-560f-4ed3-bec2-7a17bdb1b5da` I would have never labeled `NOAA tidal station` as a dataset (which instead is reported in the training file). Or at the very least, I would have used the whole mention `NOAA tidal station, Sabine Pass North (Station ID 8770570, PID AV1014)`.\n\nOn the other side, I would have labeled some mentions that are not reported in the training file. In this case, I cannot figure out if my (human) understanding of the text is wrong or if this is simply due to the training set not being fully labeled. In other words, I cannot understand if the mention I would have labeled is missing because a) it should really miss or b) it should be labeled but was removed from the training data.\n\n@philculliton @maggiemd Is it possible to reveal the identifier of at least one publication that is guaranteed to be fully labeled in the training file? I guess this would be of great help in better understanding the task, see also:\n- https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/231893\n- https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/231255\n- https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/230436\n- https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/230604\n- https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/228322\n- https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/228337", "votes": 4 } ], "comments": [], "raw_markdown_by_id": { "1274974": "As stated in the evaluation instructions:\n> The objective of the competition is to identify the mention of datasets within scientific publications. Your predictions will be short excerpts from the publications that appear to note a dataset.\n\nHowever, it seems to me that the notion of **dataset** (and the corresponding excerpt) is a bit ambiguous. For istance in publication `d7531c85-560f-4ed3-bec2-7a17bdb1b5da` I would have never labeled `NOAA tidal station` as a dataset (which instead is reported in the training file). Or at the very least, I would have used the whole mention `NOAA tidal station, Sabine Pass North (Station ID 8770570, PID AV1014)`.\n\nOn the other side, I would have labeled some mentions that are not reported in the training file. In this case, I cannot figure out if my (human) understanding of the text is wrong or if this is simply due to the training set not being fully labeled. In other words, I cannot understand if the mention I would have labeled is missing because a) it should really miss or b) it should be labeled but was removed from the training data.\n\n@philculliton @maggiemd Is it possible to reveal the identifier of at least one publication that is guaranteed to be fully labeled in the training file? I guess this would be of great help in better understanding the task, see also:\n- https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/231893\n- https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/231255\n- https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/230436\n- https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/230604\n- https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/228322\n- https://www.kaggle.com/c/coleridgeinitiative-show-us-the-data/discussion/228337" } }