Thanks for your time!!
\nI am facing an issue, when trying to use BERT from tensorflow in the notebook without internet ON for predictions.
\nI have downloaded TF BERT preprocessing and encoder pre-trained models and created a dataset. From dataset, tried to load these pre-trained models using tensorflow hub and getting errors.
\nFew solutions on the net were like, install tensorflow_text(and import the same), and tensorflow_io etc … none didnt work and will not work, if internet is off.
\nMy question is, is it possible to use the tf hub models for competitions?? OR, huggingFace BERT is the alternative solution??
\n import tensorflow as tf\n import tensorflow_hub as hub\n\n model_path = \"../input/bert-en-uncased-l12-h768-a124-jun5-2021/bert_en_uncased_preprocess_3\"\n preprocessor = tf.keras.models.load_model(model_path)#hub.KerasLayer(hub.load(model_path))\n #\"https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3\")\n #\"https://tfhub.dev/tensorflow/albert_en_preprocess/3\")\n\n encoder = hub.KerasLayer(\n \"../input/bert-en-uncased-l12-h768-a124-jun5-2021/bert_en_uncased_L-12_H-768_A-12_4\",\n #\"https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/3\",\n # \"https://tfhub.dev/tensorflow/albert_en_base/3\",\n trainable=True)\nExperts, please throw some light on using BERT models on Kaggle!
\nThanks.
Thanks for your time!!
\nI am facing an issue, when trying to use BERT from tensorflow in the notebook without internet ON for predictions.
\nI have downloaded TF BERT preprocessing and encoder pre-trained models and created a dataset. From dataset, tried to load these pre-trained models using tensorflow hub and getting errors.
\nFew solutions on the net were like, install tensorflow_text(and import the same), and tensorflow_io etc … none didnt work and will not work, if internet is off.
\nMy question is, is it possible to use the tf hub models for competitions?? OR, huggingFace BERT is the alternative solution??
\n import tensorflow as tf\n import tensorflow_hub as hub\n\n model_path = \"../input/bert-en-uncased-l12-h768-a124-jun5-2021/bert_en_uncased_preprocess_3\"\n preprocessor = tf.keras.models.load_model(model_path)#hub.KerasLayer(hub.load(model_path))\n #\"https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3\")\n #\"https://tfhub.dev/tensorflow/albert_en_preprocess/3\")\n\n encoder = hub.KerasLayer(\n \"../input/bert-en-uncased-l12-h768-a124-jun5-2021/bert_en_uncased_L-12_H-768_A-12_4\",\n #\"https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/3\",\n # \"https://tfhub.dev/tensorflow/albert_en_base/3\",\n trainable=True)\nExperts, please throw some light on using BERT models on Kaggle!
\nThanks.
I created Kaggle datasets for TF huggingface tokenizers and pretrained models as follows:
\nYou can add them as datasets to your notebook to use them.
", "rawMarkdown": "I created Kaggle datasets for TF huggingface tokenizers and pretrained models as follows:\n* [TF BERT Base Uncased Pretrained Model/Tokenizer](https://www.kaggle.com/jeongyoonlee/tfbert-base-uncased)\n* [TF BERT Large Uncased Pretrained Model/Tokenizer](https://www.kaggle.com/jeongyoonlee/tfbert-large-uncased)\n* [TF RoBERTa Base Model/Tokenizer](https://www.kaggle.com/jeongyoonlee/tfroberta-base)\n* [TF RoBERTa Large Model/Tokenizer](https://www.kaggle.com/jeongyoonlee/tfroberta-large)\n* [TF BART Base Pretrained Model/Tokenizer](https://www.kaggle.com/jeongyoonlee/tfbart-base)\n* [TF BART Large Pretrained Model/Tokenizer](https://www.kaggle.com/jeongyoonlee/tfbart-large)\n\nYou can add them as datasets to your notebook to use them.", "votes": null }, { "id": "1338087", "postDate": "06/06/2021 06:56:58", "content": "Great and thanks for the datasets. Checking out your notebooks on these datasets. Shall try out the code from there 👍🙏
\nEDIT
\nI was able to successfully load from above datasets using the code from your notebook link
", "rawMarkdown": "Great and thanks for the datasets. Checking out your notebooks on these datasets. Shall try out the code from there 👍🙏\n\n**EDIT**\n\nI was able to successfully load from above datasets using the code from your notebook [link](https://www.kaggle.com/jeongyoonlee/tf-keras-bert-baseline-training-inference)", "votes": null }, { "id": "1338289", "postDate": "06/06/2021 10:26:45", "content": "Hey @Jeong-Yoon Lee, Thanks for the above datasets. Do you have any documentation on how to save the tf hub models? I have tried using the .pb format but was unable to load the model successfully.
", "rawMarkdown": "Hey @Jeong-Yoon Lee, Thanks for the above datasets. Do you have any documentation on how to save the tf hub models? I have tried using the .pb format but was unable to load the model successfully.", "votes": null }, { "id": "1338798", "postDate": "06/06/2021 18:28:41", "content": "You can save them as HDF5 (tf_model.h5) instead of protobuf as follows:
\nbert_model = TFBertModel.from_pretrained(\"bert-large-uncased\", config=config)\nbert_model.save_pretrained(pretrained_dir)\n@mannyjain Are you asking about, how to download(and then save) from tf hub or do transfer learning on tf hub models and then save?? if it is second one, you can follow as in above comment.
", "rawMarkdown": "mannyjain Are you asking about, how to download(and then save) from tf hub or do transfer learning on tf hub models and then save?? if it is second one, you can follow as in above comment.", "votes": null }, { "id": "1342118", "postDate": "06/09/2021 07:39:46", "content": "@jeongyoonlee which BERT do you suggest for NER task??
\nCased vs uncased
\nbase vs large
\nBERT vs RoBERTa vs BART etc?
\nThanks
Generally speaking, large models perform better but require more resources (time, computation). So during initial prototyping, people experiment with base models, then move to large models later.
\nIf cases are important, I think, as in this competition, use cased. Otherwise use uncased.
\nMost of PyTorch baselines use either BERT or RoBERTa. RoBERTa models seem to perform slightly better than BERT, but at the end, you need to try them all and see which one performs best in each competition.
\nGood luck!
", "rawMarkdown": "Generally speaking, large models perform better but require more resources (time, computation). So during initial prototyping, people experiment with base models, then move to large models later.\n\nIf cases are important, I think, as in this competition, use cased. Otherwise use uncased.\n\nMost of PyTorch baselines use either BERT or RoBERTa. RoBERTa models seem to perform slightly better than BERT, but at the end, you need to try them all and see which one performs best in each competition.\n\nGood luck!", "votes": null } ], "comments": [ { "id": 1338055, "author_name": "jeongyoonlee", "author_url": "", "post_date": "06/06/2021 06:14:00", "content": "I created Kaggle datasets for TF huggingface tokenizers and pretrained models as follows:
\nYou can add them as datasets to your notebook to use them.
", "votes": null, "replies": [ { "id": 1338087, "author_name": "jkreddy", "author_url": "", "post_date": "06/06/2021 06:56:58", "content": "Great and thanks for the datasets. Checking out your notebooks on these datasets. Shall try out the code from there 👍🙏
\nEDIT
\nI was able to successfully load from above datasets using the code from your notebook link
", "votes": null, "replies": [] }, { "id": 1338289, "author_name": "mannyjain", "author_url": "", "post_date": "06/06/2021 10:26:45", "content": "Hey @Jeong-Yoon Lee, Thanks for the above datasets. Do you have any documentation on how to save the tf hub models? I have tried using the .pb format but was unable to load the model successfully.
", "votes": null, "replies": [] }, { "id": 1338798, "author_name": "jeongyoonlee", "author_url": "", "post_date": "06/06/2021 18:28:41", "content": "You can save them as HDF5 (tf_model.h5) instead of protobuf as follows:
\nbert_model = TFBertModel.from_pretrained(\"bert-large-uncased\", config=config)\nbert_model.save_pretrained(pretrained_dir)\n@mannyjain Are you asking about, how to download(and then save) from tf hub or do transfer learning on tf hub models and then save?? if it is second one, you can follow as in above comment.
", "votes": null, "replies": [] } ] }, { "id": 1342118, "author_name": "jkreddy", "author_url": "", "post_date": "06/09/2021 07:39:46", "content": "@jeongyoonlee which BERT do you suggest for NER task??
\nCased vs uncased
\nbase vs large
\nBERT vs RoBERTa vs BART etc?
\nThanks
Generally speaking, large models perform better but require more resources (time, computation). So during initial prototyping, people experiment with base models, then move to large models later.
\nIf cases are important, I think, as in this competition, use cased. Otherwise use uncased.
\nMost of PyTorch baselines use either BERT or RoBERTa. RoBERTa models seem to perform slightly better than BERT, but at the end, you need to try them all and see which one performs best in each competition.
\nGood luck!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1337918": "Thanks for your time!!\n\nI am facing an issue, when trying to use BERT from tensorflow in the notebook without internet ON for predictions. \n\nI have downloaded TF BERT preprocessing and encoder pre-trained models and created a dataset. From dataset, tried to load these pre-trained models using tensorflow hub and getting errors.\n\nFew solutions on the net were like, install tensorflow_text(and import the same), and tensorflow_io etc ... none didnt work and will not work, if internet is off.\n\nMy question is, is it possible to use the tf hub models for competitions?? OR, huggingFace BERT is the alternative solution??\n\n```\n import tensorflow as tf\n import tensorflow_hub as hub\n\n model_path = \"../input/bert-en-uncased-l12-h768-a124-jun5-2021/bert_en_uncased_preprocess_3\"\n preprocessor = tf.keras.models.load_model(model_path)#hub.KerasLayer(hub.load(model_path))\n #\"https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3\")\n #\"https://tfhub.dev/tensorflow/albert_en_preprocess/3\")\n\n encoder = hub.KerasLayer(\n \"../input/bert-en-uncased-l12-h768-a124-jun5-2021/bert_en_uncased_L-12_H-768_A-12_4\",\n #\"https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/3\",\n # \"https://tfhub.dev/tensorflow/albert_en_base/3\",\n trainable=True)\n```\n\nExperts, please throw some light on using BERT models on Kaggle!\nThanks.", "1338055": "I created Kaggle datasets for TF huggingface tokenizers and pretrained models as follows:\n* [TF BERT Base Uncased Pretrained Model/Tokenizer](https://www.kaggle.com/jeongyoonlee/tfbert-base-uncased)\n* [TF BERT Large Uncased Pretrained Model/Tokenizer](https://www.kaggle.com/jeongyoonlee/tfbert-large-uncased)\n* [TF RoBERTa Base Model/Tokenizer](https://www.kaggle.com/jeongyoonlee/tfroberta-base)\n* [TF RoBERTa Large Model/Tokenizer](https://www.kaggle.com/jeongyoonlee/tfroberta-large)\n* [TF BART Base Pretrained Model/Tokenizer](https://www.kaggle.com/jeongyoonlee/tfbart-base)\n* [TF BART Large Pretrained Model/Tokenizer](https://www.kaggle.com/jeongyoonlee/tfbart-large)\n\nYou can add them as datasets to your notebook to use them.", "1338087": "Great and thanks for the datasets. Checking out your notebooks on these datasets. Shall try out the code from there 👍🙏\n\n**EDIT**\n\nI was able to successfully load from above datasets using the code from your notebook [link](https://www.kaggle.com/jeongyoonlee/tf-keras-bert-baseline-training-inference)", "1338289": "Hey @Jeong-Yoon Lee, Thanks for the above datasets. Do you have any documentation on how to save the tf hub models? I have tried using the .pb format but was unable to load the model successfully.", "1338798": "You can save them as HDF5 (tf_model.h5) instead of protobuf as follows:\n```python\nbert_model = TFBertModel.from_pretrained(\"bert-large-uncased\", config=config)\nbert_model.save_pretrained(pretrained_dir)\n```", "1339335": "mannyjain Are you asking about, how to download(and then save) from tf hub or do transfer learning on tf hub models and then save?? if it is second one, you can follow as in above comment.", "1342118": "jeongyoonlee which BERT do you suggest for NER task??\nCased vs uncased\nbase vs large\nBERT vs RoBERTa vs BART etc?\nThanks", "1342457": "Generally speaking, large models perform better but require more resources (time, computation). So during initial prototyping, people experiment with base models, then move to large models later.\n\nIf cases are important, I think, as in this competition, use cased. Otherwise use uncased.\n\nMost of PyTorch baselines use either BERT or RoBERTa. RoBERTa models seem to perform slightly better than BERT, but at the end, you need to try them all and see which one performs best in each competition.\n\nGood luck!" }, "source": "meta" }