{
  "id": 231039,
  "title": "Sentence-level search for dataset titles. ",
  "url": "/competitions/coleridgeinitiative-show-us-the-data/discussion/231039",
  "author_name": "",
  "post_date": "2021-04-06T16:12:24.399282200Z",
  "votes": 10,
  "comment_count": 6,
  "views": 0,
  "content": "<p>I started looking into ways to isolate sentences that contain dataset titles. <br>\nThis notebook repacks train data into a dataframe of sentences and shows 93% accuracy in identification of sentences with titles using just Naïve Bayes: <a href=\"url\" target=\"_blank\">https://www.kaggle.com/poedator/sentence-level-analysis</a></p>\n<p>Such result should be materially improved using neural networks -based classifiers.</p>\n<p>Once the sentences with model titles are clearly identified, they can be processed with some NER model for exact identification of the dataset titles.</p>",
  "messages": [
    {
      "id": "1265103",
      "postDate": "04/06/2021 16:12:24",
      "content": "<p>I started looking into ways to isolate sentences that contain dataset titles. <br>\nThis notebook repacks train data into a dataframe of sentences and shows 93% accuracy in identification of sentences with titles using just Naïve Bayes: <a href=\"url\" target=\"_blank\">https://www.kaggle.com/poedator/sentence-level-analysis</a></p>\n<p>Such result should be materially improved using neural networks -based classifiers.</p>\n<p>Once the sentences with model titles are clearly identified, they can be processed with some NER model for exact identification of the dataset titles.</p>",
      "rawMarkdown": "I started looking into ways to isolate sentences that contain dataset titles. \nThis notebook repacks train data into a dataframe of sentences and shows 93% accuracy in identification of sentences with titles using just Naïve Bayes: [https://www.kaggle.com/poedator/sentence-level-analysis](url)\n\nSuch result should be materially improved using neural networks -based classifiers.\n\nOnce the sentences with model titles are clearly identified, they can be processed with some NER model for exact identification of the dataset titles.",
      "votes": null
    },
    {
      "id": "1265382",
      "postDate": "04/06/2021 20:49:06",
      "content": "<p>I noticed that in the first version there may be cases where train and val sets contain sentences from the same document. To avoid this I updated the notebook so that train-test split is now done on document IDs basis. It still gives 92.5% accuracy.</p>\n<p>One possible reason for this rather high accuracy may be the fact that the 14316 publications from train set that we have refer to a rather small number of datasets (45). </p>",
      "rawMarkdown": "I noticed that in the first version there may be cases where train and val sets contain sentences from the same document. To avoid this I updated the notebook so that train-test split is now done on document IDs basis. It still gives 92.5% accuracy.\n\nOne possible reason for this rather high accuracy may be the fact that the 14316 publications from train set that we have refer to a rather small number of datasets (45).",
      "votes": null
    },
    {
      "id": "1267810",
      "postDate": "04/08/2021 20:13:03",
      "content": "<p>Updated notebook with Transformer model, which shows 99% accuracy on val set in identifying sentences with dataset titles. Same URL: <a href=\"url\" target=\"_blank\">https://www.kaggle.com/poedator/sentence-level-analysis</a></p>",
      "rawMarkdown": "Updated notebook with Transformer model, which shows 99% accuracy on val set in identifying sentences with dataset titles. Same URL: [https://www.kaggle.com/poedator/sentence-level-analysis](url)",
      "votes": null
    },
    {
      "id": "1267965",
      "postDate": "04/09/2021 02:37:33",
      "content": "<p>Such a high accuracy rate! I think there are too few data sets, causing the model to only learn to find these words.</p>",
      "rawMarkdown": "Such a high accuracy rate! I think there are too few data sets, causing the model to only learn to find these words.",
      "votes": null
    },
    {
      "id": "1268113",
      "postDate": "04/09/2021 06:18:51",
      "content": "<p>I tried to make an accurate research by placing sentences from different publication into train and val. Perhaps this exercise should be done properly with 2 separate sets of datasets mentioned. I may try that later.</p>",
      "rawMarkdown": "I tried to make an accurate research by placing sentences from different publication into train and val. Perhaps this exercise should be done properly with 2 separate sets of datasets mentioned. I may try that later.",
      "votes": null
    },
    {
      "id": "1269451",
      "postDate": "04/10/2021 14:43:14",
      "content": "<p>Good work!</p>",
      "rawMarkdown": "Good work!",
      "votes": null
    },
    {
      "id": "1278295",
      "postDate": "04/19/2021 18:04:50",
      "content": "<p>Hi Poe, its nice work thank you.</p>\n<p>Have I misunderstood the task though - I thought we had to teach our models to identify the context in which datasets are mentioned as the test set will have datasets we've never seen before?</p>",
      "rawMarkdown": "Hi Poe, its nice work thank you.\n\nHave I misunderstood the task though - I thought we had to teach our models to identify the context in which datasets are mentioned as the test set will have datasets we've never seen before?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1265382,
      "author_name": "poedator",
      "author_url": "",
      "post_date": "04/06/2021 20:49:06",
      "content": "<p>I noticed that in the first version there may be cases where train and val sets contain sentences from the same document. To avoid this I updated the notebook so that train-test split is now done on document IDs basis. It still gives 92.5% accuracy.</p>\n<p>One possible reason for this rather high accuracy may be the fact that the 14316 publications from train set that we have refer to a rather small number of datasets (45). </p>",
      "votes": null,
      "replies": [
        {
          "id": 1278295,
          "author_name": "cascadenite",
          "author_url": "",
          "post_date": "04/19/2021 18:04:50",
          "content": "<p>Hi Poe, its nice work thank you.</p>\n<p>Have I misunderstood the task though - I thought we had to teach our models to identify the context in which datasets are mentioned as the test set will have datasets we've never seen before?</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1267810,
      "author_name": "poedator",
      "author_url": "",
      "post_date": "04/08/2021 20:13:03",
      "content": "<p>Updated notebook with Transformer model, which shows 99% accuracy on val set in identifying sentences with dataset titles. Same URL: <a href=\"url\" target=\"_blank\">https://www.kaggle.com/poedator/sentence-level-analysis</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1267965,
      "author_name": "zzy990106",
      "author_url": "",
      "post_date": "04/09/2021 02:37:33",
      "content": "<p>Such a high accuracy rate! I think there are too few data sets, causing the model to only learn to find these words.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1268113,
          "author_name": "poedator",
          "author_url": "",
          "post_date": "04/09/2021 06:18:51",
          "content": "<p>I tried to make an accurate research by placing sentences from different publication into train and val. Perhaps this exercise should be done properly with 2 separate sets of datasets mentioned. I may try that later.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1269451,
      "author_name": "zavodrobotov",
      "author_url": "",
      "post_date": "04/10/2021 14:43:14",
      "content": "<p>Good work!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1265103": "I started looking into ways to isolate sentences that contain dataset titles. \nThis notebook repacks train data into a dataframe of sentences and shows 93% accuracy in identification of sentences with titles using just Naïve Bayes: [https://www.kaggle.com/poedator/sentence-level-analysis](url)\n\nSuch result should be materially improved using neural networks -based classifiers.\n\nOnce the sentences with model titles are clearly identified, they can be processed with some NER model for exact identification of the dataset titles.",
    "1265382": "I noticed that in the first version there may be cases where train and val sets contain sentences from the same document. To avoid this I updated the notebook so that train-test split is now done on document IDs basis. It still gives 92.5% accuracy.\n\nOne possible reason for this rather high accuracy may be the fact that the 14316 publications from train set that we have refer to a rather small number of datasets (45).",
    "1267810": "Updated notebook with Transformer model, which shows 99% accuracy on val set in identifying sentences with dataset titles. Same URL: [https://www.kaggle.com/poedator/sentence-level-analysis](url)",
    "1267965": "Such a high accuracy rate! I think there are too few data sets, causing the model to only learn to find these words.",
    "1268113": "I tried to make an accurate research by placing sentences from different publication into train and val. Perhaps this exercise should be done properly with 2 separate sets of datasets mentioned. I may try that later.",
    "1269451": "Good work!",
    "1278295": "Hi Poe, its nice work thank you.\n\nHave I misunderstood the task though - I thought we had to teach our models to identify the context in which datasets are mentioned as the test set will have datasets we've never seen before?"
  },
  "source": "meta"
}