{
  "id": 235430,
  "title": "Curiosity: a dataset my model never detects..",
  "url": "/competitions/coleridgeinitiative-show-us-the-data/discussion/235430",
  "author_name": "",
  "post_date": "2021-04-29T14:07:03.006730400Z",
  "votes": 3,
  "comment_count": 3,
  "views": 0,
  "content": "<p>I just wanted to share a curiosity I can't explain: so my model (BERT) detects a diverse range of unseen datasets in my CV set (many as well that are not annotated in the data). However, my model always fails to detect \"Baltimore Longitudinal Study of Aging (BLSA)\". I find that surprising as the words \"study\" and \"longitudinal\" are relatively clear indicators for a dataset. My model has in fact no difficulty to detect for instance \"National Education Longitudinal Study\".</p>\n<p>Any idea what could be the reason? Do you experience similar results?</p>",
  "messages": [
    {
      "id": "1287903",
      "postDate": "04/29/2021 14:07:03",
      "content": "<p>I just wanted to share a curiosity I can't explain: so my model (BERT) detects a diverse range of unseen datasets in my CV set (many as well that are not annotated in the data). However, my model always fails to detect \"Baltimore Longitudinal Study of Aging (BLSA)\". I find that surprising as the words \"study\" and \"longitudinal\" are relatively clear indicators for a dataset. My model has in fact no difficulty to detect for instance \"National Education Longitudinal Study\".</p>\n<p>Any idea what could be the reason? Do you experience similar results?</p>",
      "rawMarkdown": "I just wanted to share a curiosity I can't explain: so my model (BERT) detects a diverse range of unseen datasets in my CV set (many as well that are not annotated in the data). However, my model always fails to detect \"Baltimore Longitudinal Study of Aging (BLSA)\". I find that surprising as the words \"study\" and \"longitudinal\" are relatively clear indicators for a dataset. My model has in fact no difficulty to detect for instance \"National Education Longitudinal Study\".\n\nAny idea what could be the reason? Do you experience similar results?",
      "votes": null
    },
    {
      "id": "1288296",
      "postDate": "04/29/2021 20:45:21",
      "content": "<p>I am experiencing the the BLSA issue too using BERT.  I have just started looking into it and will share what I find.</p>\n<p>Also maybe you can help me - my BERT works fine with internet on, but if I try and turn off internet to submit my notebook, it throws an error when loaidibg the pretrained model - see below.  Can you shed some light o this ?</p>\n<h1>Tokenizer</h1>\n<p>tokenizer = BertTokenizer.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')</p>",
      "rawMarkdown": "I am experiencing the the BLSA issue too using BERT.  I have just started looking into it and will share what I find.\n\nAlso maybe you can help me - my BERT works fine with internet on, but if I try and turn off internet to submit my notebook, it throws an error when loaidibg the pretrained model - see below.  Can you shed some light o this ?\n\n#Tokenizer\ntokenizer = BertTokenizer.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')",
      "votes": null
    },
    {
      "id": "1288306",
      "postDate": "04/29/2021 21:09:25",
      "content": "<p>Yes you need to save the files of the tokenizer somewhere and then load them with tokenizer = BertTokenizer.from_pretrained(folder_with_tokenizer_files)</p>\n<p>Same for the model, saving the model/weights locally and loading them with <br>\nBertForQuestionAnswering.from_pretrained(pretrained_model_name_or_path)</p>",
      "rawMarkdown": "Yes you need to save the files of the tokenizer somewhere and then load them with tokenizer = BertTokenizer.from_pretrained(folder_with_tokenizer_files)\n\nSame for the model, saving the model/weights locally and loading them with \nBertForQuestionAnswering.from_pretrained(pretrained_model_name_or_path)",
      "votes": null
    },
    {
      "id": "1288358",
      "postDate": "04/29/2021 22:59:04",
      "content": "<p>Thanks <a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> I got it working.  I also got BERT to id the BLSA dataset by tweaking the question I asked.  I just started working with BERT today so I have a ways to go.<br>\nThanks again for you help.</p>",
      "rawMarkdown": "Thanks @hannes82 I got it working.  I also got BERT to id the BLSA dataset by tweaking the question I asked.  I just started working with BERT today so I have a ways to go.\nThanks again for you help.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1288296,
      "author_name": "mlconsult",
      "author_url": "",
      "post_date": "04/29/2021 20:45:21",
      "content": "<p>I am experiencing the the BLSA issue too using BERT.  I have just started looking into it and will share what I find.</p>\n<p>Also maybe you can help me - my BERT works fine with internet on, but if I try and turn off internet to submit my notebook, it throws an error when loaidibg the pretrained model - see below.  Can you shed some light o this ?</p>\n<h1>Tokenizer</h1>\n<p>tokenizer = BertTokenizer.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')</p>",
      "votes": null,
      "replies": [
        {
          "id": 1288306,
          "author_name": "hannes82",
          "author_url": "",
          "post_date": "04/29/2021 21:09:25",
          "content": "<p>Yes you need to save the files of the tokenizer somewhere and then load them with tokenizer = BertTokenizer.from_pretrained(folder_with_tokenizer_files)</p>\n<p>Same for the model, saving the model/weights locally and loading them with <br>\nBertForQuestionAnswering.from_pretrained(pretrained_model_name_or_path)</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1288358,
          "author_name": "mlconsult",
          "author_url": "",
          "post_date": "04/29/2021 22:59:04",
          "content": "<p>Thanks <a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> I got it working.  I also got BERT to id the BLSA dataset by tweaking the question I asked.  I just started working with BERT today so I have a ways to go.<br>\nThanks again for you help.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1287903": "I just wanted to share a curiosity I can't explain: so my model (BERT) detects a diverse range of unseen datasets in my CV set (many as well that are not annotated in the data). However, my model always fails to detect \"Baltimore Longitudinal Study of Aging (BLSA)\". I find that surprising as the words \"study\" and \"longitudinal\" are relatively clear indicators for a dataset. My model has in fact no difficulty to detect for instance \"National Education Longitudinal Study\".\n\nAny idea what could be the reason? Do you experience similar results?",
    "1288296": "I am experiencing the the BLSA issue too using BERT.  I have just started looking into it and will share what I find.\n\nAlso maybe you can help me - my BERT works fine with internet on, but if I try and turn off internet to submit my notebook, it throws an error when loaidibg the pretrained model - see below.  Can you shed some light o this ?\n\n#Tokenizer\ntokenizer = BertTokenizer.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')",
    "1288306": "Yes you need to save the files of the tokenizer somewhere and then load them with tokenizer = BertTokenizer.from_pretrained(folder_with_tokenizer_files)\n\nSame for the model, saving the model/weights locally and loading them with \nBertForQuestionAnswering.from_pretrained(pretrained_model_name_or_path)",
    "1288358": "Thanks @hannes82 I got it working.  I also got BERT to id the BLSA dataset by tweaking the question I asked.  I just started working with BERT today so I have a ways to go.\nThanks again for you help."
  },
  "source": "meta"
}