All Questions
Tagged with bert-language-model tensorflow2.0
33
questions
12
votes
4
answers
11k
views
Training TFBertForSequenceClassification with custom X and Y data
I am working on a TextClassification problem, for which I am trying to traing my model on TFBertForSequenceClassification given in huggingface-transformers library.
I followed the example given on ...
- 872
10
votes
1
answer
14k
views
How to get intermediate layers' output of pre-trained BERT model in HuggingFace Transformers library?
(I'm following this pytorch tutorial about BERT word embeddings, and in the tutorial the author is access the intermediate layers of the BERT model.)
What I want is to access the last, lets say, 4 ...
- 1,262
3
votes
3
answers
6k
views
Tensorflow 2.X Error - Op type not registered 'CaseFoldUTF8' in binary running on Colab
I have been using BERT encoder from the Tensorflow hub for quite sometime now. Here are the syntaxes:
tfhub_handle_encoder = "https://tfhub.dev/tensorflow/bert_multi_cased_L-12_H-768_A-12/4" ...
- 51
3
votes
0
answers
1k
views
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/
n_epochs = 6
model = CNN_Text()
loss_fn = nn.CrossEntropyLoss(reduction='sum')
optimizer = torch.optim.Adam(filter(lambda p: p.requires_grad, model.parameters()), lr=0.001)
model.cuda()
# Load train ...
- 43
3
votes
1
answer
6k
views
How to get the vocab file for Bert tokenizer from TF Hub
I'm trying to use Bert from TensorFlow Hub and build a tokenizer, this is what I'm doing:
>>> import tensorflow_hub as hub
>>> from bert.tokenization import FullTokenizer
>>&...
- 5,898
2
votes
1
answer
975
views
stacking LSTM layer on top of BERT encoder in Keras
I have been trying to stack a single LSTM layer on top of Bert embeddings, but whilst my model starts to train it fails on the last batch and throws the following error message:
Node: 'model/tf....
- 705
2
votes
1
answer
2k
views
Tensorflow2.4 NotFoundError: No algorithm worked! with Keras Conv1D Layer
I've been looking for a solution to this error for days and I can't find solutions for this:
NotFoundError: 3 root error(s) found.
(0) Not found: No algorithm worked!
[[node model/conv1d/conv1d (...
2
votes
1
answer
3k
views
How to freeze some layers of BERT in fine tuning in tf2.keras
I am trying to fine-tune 'bert-based-uncased' on a dataset for a text classification task. Here is the way I am downloading the model:
import tensorflow as tf
from transformers import ...
- 108
2
votes
0
answers
453
views
Error: module 'tensorflow' has no attribute 'gfile' even after importing tensorflow as tensorflow.compat.v1
I have read solutions to this problem where tensorflow version change to 1.14 worked for people. I was wondering if there are any other solutions without doing the version change. I am working off of ...
- 21
1
vote
1
answer
74
views
How to pass multiple embedded layers into TFDF using Keras?
In the Keras tutorial it shows how to embed a text field and pass it to TFDF in this tutorial.
sentence_encoder_layer = hub.KerasLayer(
"https://tfhub.dev/google/universal-sentence-encoder/4&...
- 13
1
vote
1
answer
725
views
When i use tensorflow 2.8.0 to fine-tune bert meet this bug: hub.KerasLayer(tfhub_handle_preprocess) 'CaseFoldUTF8'?
my code follows:
bert_model_name = 'small_bert/bert_en_uncased_L-4_H-512_A-8'
map_name_to_handle = {'small_bert/bert_en_uncased_L-4_H-512_A-8':
'https://tfhub.dev/tensorflow/small_bert/...
- 31
1
vote
1
answer
3k
views
Getting an error(cannot import name 'BertPreTrainedModel') while importing classification model from simpletransformers
Getting the following error while trying to import the classificationmodel from simpletransformers.
ImportError Traceback (most recent call last)
<ipython-input-1-...
1
vote
0
answers
759
views
how to resolve errors in bert embedding installation
import mxnet as mx
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\USER\anaconda3\envs\gpu3\lib\site-packages\mxnet\__init__.py", ...
- 91
1
vote
1
answer
2k
views
How to run Huggingface BERT tokenizer in offline mode?
While running above code in my work laptop I'm getting the following error, but same error is not there when I run in my personal laptop. I wanted to check whether there is a way to fix this SSL error?...
- 11
1
vote
1
answer
1k
views
BERT error - module 'tensorflow_core.keras.activations' has no attribute 'swish'
I am trying to execute the transformer model but ended up with error.
Python version == 3.7
Tensorflow == 2.0
Transformers == 4.15.0
Source : https://huggingface.co/cross-encoder/nli-deberta-...
- 37
1
vote
0
answers
68
views
Tensorflow NLP with BERT Preprocessing data
so this is a specific question involving two Tensorflow text classification tutorials on tensorflow.org. Sorry if this is the wrong place to ask.
Basically, there are two tutorials, one is "...
1
vote
0
answers
120
views
Customizing TensorFlow Serving for BERT or Transformer
I would like to customize serving_default of BERT or Transformer to make the input as string type and get the prediction as string type. So the model's output should be converted to a sentence ...
- 13
1
vote
0
answers
545
views
How to resolve TypeError: Could not build a TypeSpec for <encoded text> with type list
I'm creating a multiclass classifier in Python using BERT and Tensorflow 2.*, but I'm encountering two errors with my current code that I've been stuck on all day. The first error I'm getting is:
...
- 21
0
votes
1
answer
970
views
Transfer Learning using HuggingFace and Tensorflow with AutoModel does not work
I try to do transfer learning using a HuggingFace pretrained BERT model. I want to use tensorflow API with it. I do not understand why the last line produces an error
from transformers import ...
- 2,561
0
votes
1
answer
2k
views
save and load fine-tuned bert classification model using tensorflow 2.0
I am trying to save a fine-tuned binary classification model based on pretrained Bert module 'uncased_L-12_H-768_A-12'. I'm using tf2.
The code set up the model structure:
bert_classifier, ...
- 1,041
0
votes
1
answer
1k
views
How to optimize fine-tuned BERT's model size in TensorFlow 2.0?
After fine tuning BERT for classification the model size is ~1.3GB while the pre-trained model size was ~400MB. This happens due to additional variables related to Adam saved with the model and can be ...
- 639
0
votes
1
answer
697
views
Get output of the last 4 layers of BERT (Tensorflow)
I was going through Andreas Poyiatzis' article posted on towardsdatascience.com. Below is the link for the article.
https://towardsdatascience.com/nlp-extract-contextualized-word-embeddings-from-bert-...
0
votes
1
answer
91
views
Issue with BERT Preprocessor model in TF2 and python
I am trying to use BERT to do a text classification project. However I keep running into this error
`
ValueError Traceback (most recent call last)
Cell In[37], line 4
...
- 21
0
votes
0
answers
28
views
TypeError: Layer input_spec must be an instance of InputSpec. Got: InputSpec(shape=(None, 48, 768), ndim=3)
I am doing a multi -class classification task, the TensorFlow version is 2.7.0, and the Python version is 3.9. When I create a model, I reported the following errors:
TypeError: Exception encountered ...
0
votes
0
answers
124
views
If mobile BERT model requires DataType.INT32, is it possible to run on Android as a tensorflow lite model?
I am trying to run mobile BERT model on Android and have several issues with this.
After converting model into tflite, format I am able to load it into Android Studio over ML binding. However, if I ...
- 6,449
0
votes
0
answers
168
views
No such module named "official" when I try to use tf-models-official
I am going through a lab walkthrough done by Google Cloud about Transformer and BERT Models and I am working in a JupyterLab notebook and using TensorFlow 2.10.
One of the modules I am trying to call ...
0
votes
2
answers
192
views
InternalError during model.fit() while running on Colab TPU
I am attempting to fine-tune an expertBERT model from TensorFlow Hub on the Sentiment140 TFDS in Google Colab. However, I encounter the following error:
InternalError: RET_CHECK failure (third_party/...
0
votes
1
answer
335
views
BERT Model Fine Tuning and migrating to TF2
I executed this excellent tutorial:
https://towardsdatascience.com/building-a-multi-label-text-classifier-using-bert-and-tensorflow-f188e0ecdc5d
I understood most of it except where model is being ...
- 476
0
votes
0
answers
67
views
Input tensor ValueError when loading model BertClassifier Model
--INTRO
I am training a BERT classifier using the google code from tf-models-official==2.3.0.
Everything works like charm: training, evaluating, saving checkpoints.
--THE PROBLEM
My problem arises ...
- 81
0
votes
1
answer
297
views
Quantization of Bert Classifier Model
I am currently trying to quantize a bert-classifier model but am running into an error, I was wondering if this is even supported at the moment or not? For clarity I am asking if quantization is ...
- 11
0
votes
0
answers
721
views
Why TFBertForTokenClassification.from_pretrained("allenai/scibert_scivocab_uncased",from_pt=True,config=config) gives weights warning?
I want to do Named Entity Recognition on scientific articles.
tokenizer = AutoTokenizer.from_pretrained('allenai/scibert_scivocab_cased')
model=TFBertForTokenClassification.from_pretrained("...
-1
votes
0
answers
14
views
tensorflow bert 'tuple' object has no attribute problem
from tensorflow_text.tools.wordpiece_vocab import bert_vocab_from_dataset as bert_vocab
import pathlib
import tensorflow as tf
BATCH_SIZE = 1028
text_dataset = tf.keras.utils....
