All Questions
Tagged with bert-language-model classification
30
questions
8
votes
4
answers
16k
views
Error importing BERT: module 'tensorflow._api.v2.train' has no attribute 'Optimizer'
I tried to use bert-tensorflow in Google Colab, but I got the following error:
--------------------------------------------------------------------------- AttributeError ...
- 615
4
votes
2
answers
2k
views
BERT get sentence level embedding after fine tuning
I came across this page
1) I would like to get sentence level embedding (embedding given by [CLS] token) after the fine tuning is done. How could I do it?
2) I also noticed that the code on that ...
- 6,258
3
votes
0
answers
3k
views
BERT model classification with many classes
I want to train a BERT model to perform a multiclass text classification. I use transformers and followed this tutorial (https://towardsdatascience.com/multi-class-text-classification-with-deep-...
- 41
2
votes
1
answer
1k
views
Using BERT in order to detect language of a given word
I have words in the Hebrew language. Part of them are originally in English, and part of them are 'Hebrew English', meaning that those are words that are originally from English but are written with ...
- 865
2
votes
0
answers
359
views
How to use run_classifer.py,an example of Pytorch implementation of Bert for classification Task?
How to use the fine-tuned bert pytorch model for classification (CoLa) task?
I do not see the argument --do_predict, in /examples/run_classifier.py.
However, --do_predict exists in the original ...
- 6,934
1
vote
1
answer
1k
views
Calculating Probability of a Classification Model Prediction
I have a classification task. The training data has 50 different labels. The customer wants to differentiate the low probability predictions, meaning that, I have to classify some test data as ...
- 2,749
1
vote
1
answer
1k
views
Is splitting a long document of a dataset for BERT considered bad practice?
I am fine-tuning a BERT model on a labeled dataset with many documents longer than the 512 token limit set by the tokenizer.
Since truncating would lose a lot of data I would rather use, I started ...
- 341
1
vote
1
answer
8k
views
Bert Tokenizing error ValueError: Input nan is not valid. Should be a string, a list/tuple of strings or a list/tuple of integers
I am using the Bert for text classification task , when I try to tokenize one data sample using the code:
encoded_sent = tokenizer.encode(
sentences[7],
...
- 53
1
vote
1
answer
1k
views
Can BERT be used to train non-text sequence data for classification?
Can BERT be used for non-text sequence data? I want to try BERT for sequence classification problems. The data is not text. In other words, I want to train BERT from scratch. How do I do that?
- 1,173
1
vote
0
answers
225
views
RuntimeError: result type Float can't be cast to the desired output type Long
I know there are many errors like this solved already but mine still gives the same error no matter what I tried. I tried to fine-tune a multi-label classification model using BERT pre-train model ...
- 107
1
vote
0
answers
86
views
Confusion Matrix interpretation data perfectly balanced
I have trained a transformer based classifier with 2 classes (0,1) reaching a 91 % accuracy on a perfectly balanced dataset. I printed out the confusion matrix on validation data after had tuned the ...
- 380
1
vote
0
answers
458
views
How to get the predictions of a multi text classifier model using a csv file as test data?
I have created a multi class text classification model using TF and Bert by following this blog post: https://towardsdatascience.com/multi-label-text-classification-using-bert-and-tensorflow-...
- 33
1
vote
1
answer
735
views
How can i calculate all recall accuracy precision and f1 measure for multi class classification in BERT?
from sklearn.metrics import f1_score
def f1_score_func(preds, labels):
preds_flat = np.argmax(preds, axis=1).flatten()
labels_flat = labels.flatten()
return f1_score(labels_flat, ...
- 11
1
vote
0
answers
184
views
Concatenating two pre-trained BERT
max_length = 50
tokenizer = RobertaTokenizer.from_pretrained('roberta-large', do_lower_case=True)
encodings = tokenizer.batch_encode_plus(comments,max_length=max_length,pad_to_max_length=True, ...
- 31
0
votes
2
answers
2k
views
How to get multi class confidence score from fine tuned bert model?
I have used pretrained bert model for intent classification(uncased_L-12_H-768_A-12). The model showing the output as a predicted intent. Below is the code of how i am getting prediction:
sentences = [...
- 31
0
votes
2
answers
2k
views
Fine-tuning BERT For Sequence Classification on sentiment140 dataset gives very POOR results
I'm using :
sentiment140 dataset
BertTokenizerFast for text tokenization
TFBertForSequenceClassification for text classification
And I want to fine-tune the model (TFBertForSequenceClassification) ...
- 503
0
votes
1
answer
709
views
What is the range of BERT CLS values?
As you can see in my title, I am interested in the value range of BERT.
I read through the BERT paper but it is all still a little confusing for me. At the end of a Classification, BERT has the ...
- 76
0
votes
1
answer
3k
views
Creating word embedings from bert and feeding them to random forest for classification
I have used bert base pretrained model with 512 dimensions to generate contextual features. Feeding those vectors to random forest classifier is providing 83 percent accuracy but in various researches ...
- 23
0
votes
0
answers
52
views
Error in applying bert for multi class classification
I'm trying to apply https://github.com/xuyige/BERT4doc-Classification this: how to Fine-Tune BERT for Text Classification for multi-class classification.
I'm getting this error that I don't understand ...
0
votes
1
answer
34
views
transformers refine-tune with different classes
I want to fine-tune a BERT-based already fine-tuned model for classification with 7 classes another time on a 16 class dataset:
MODEL_NAME_OR_PATH = 'some pretrained model for 7 class classification ...
- 4,471
0
votes
1
answer
84
views
Text classification with a Language Model (LM) with class labels existing in text tokens
I have a multi-label text classification task. The train data labels are categories that might exist as tokens in the training data texts. For instance, some observations look like the following:
...
- 27
0
votes
0
answers
3k
views
run time error: torch.cat(): expected a non-empty list of Tensors
this is my code, I got an error and I don't know how to solve it:
token_id = []
attention_masks = []
def preprocessing(input_text, tokenizer):
'''
Returns <class transformers....
0
votes
1
answer
755
views
BERT text clasisification using pytorch
I am trying to build a BERT model for text classification with the help of this code [https://towardsdatascience.com/bert-text-classification-using-pytorch-723dfb8b6b5b]. My dataset contains two ...
0
votes
0
answers
105
views
how to fix error about BERT?just confuse me few days
Traceback (most recent call last):
File "D:/desk/bert-master333/bert-master/run_classifier.py", line 1024, in <module>
tf.app.run()
File "D:\anaconda\envs\tensorflow\lib\...
0
votes
0
answers
617
views
Text classification using BERT model
I have built and trained the BERT model, using this code.
Now I have this data:
and using this built function to classify each row in the text column as 0 or 1.
import random
def predict_emotion(...
- 260
0
votes
1
answer
1k
views
Train loss is decreasing, but accuracy remain the same
this is the train and development cell for multi-label classification task using Roberta (BERT). the first part is training and second part is development (validation). train_dataloader is my train ...
0
votes
0
answers
2k
views
Huggingface BERT SequenceClassification - ValueError: too many values to unpack (expected 2)
The class labels for the two class model is 0, 1, 0, 0, etc. There is only one
label per input sequence. The labels are set in a python list and converted to
torch.Tensor. (reading from a csv file - ...
- 11
0
votes
1
answer
385
views
BERT with train, dev, test, predicion mode
I am doing a text classification task with BERT. I am basically using run_classifier.py.
This code uses train.tsv and dev.tsv (with labels) to fine-tune BERT and a test.tsv (without labels) to make ...
0
votes
1
answer
89
views
RuntimeError, working on IA tryna use a pre-trained BERT model
Hi here is a part of my code to use a pre-trained bert model for classification:
model = BertForSequenceClassification.from_pretrained(
"bert-base-uncased", # Use the 12-layer BERT model, ...
-1
votes
1
answer
553
views
bert model showing TypeError: Layer input_spec must be an instance of InputSpec. Got: InputSpec(shape=(None, 55, 768), ndim=3)
I am trying to use bert pretrained model for intent classification. here is my code in jupyter notebok.
class DataPreparation:
text_column = "text"
label_column = "intent&...
- 31