All Questions
Tagged with bert-language-model google-colaboratory
34
questions
17
votes
5
answers
67k
views
Transformer: Error importing packages. "ImportError: cannot import name 'SAVE_STATE_WARNING' from 'torch.optim.lr_scheduler'"
I am working on a machine learning project on Google Colab, it seems recently there is an issue when trying to import packages from transformers. The error message says:
ImportError: cannot import ...
- 231
5
votes
1
answer
8k
views
run python parameters in Google Colab
I am running a python file in Google Colab and getting an error. I am following a bert text classification example from this link;
https://appliedmachinelearning.blog/2019/03/04/state-of-the-art-text-...
- 1,575
5
votes
2
answers
19k
views
ERROR: file:///content does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found
https://colab.research.google.com/drive/11u6leEKvqE0CCbvDHHKmCxmW5GxyjlBm?usp=sharing
setup.py file is in transformers folder(root directory). But this error occurs when I run
!git clone https://...
- 97
5
votes
0
answers
1k
views
Huggingface Bert TPU fine-tuning works on Colab but not in GCP
I'm trying to fine-tune a Huggingface transformers BERT model on TPU. It works in Colab but fails when I switch to a paid TPU on GCP. Jupyter notebook code is as follows:
[1] model = transformers....
- 145
3
votes
3
answers
2k
views
List index out of range when saving finetuned Tensorflow model
I'm trying to fine-tune a pre-trained BERT model from Huggingface using Tensorflow. Everything runs smoothly and the model builds and trains without error. But when I try to save the model it stops ...
- 47
3
votes
0
answers
2k
views
Bert model giving CUDA out of memory error on google colab
I am using the following tutorial here to train and test a Bertsequenceclassifier model on a dataset of documents of varying lengths (small(0-280), medium(280-10000), large(10000 plus)) on the google ...
- 325
2
votes
1
answer
62
views
BERTopic: add legend to term score decline
I plot the term score decline for a topic model I created on Google Colab with BERTopic. Great function. Works neat! But I need to add a legend. This parameter is not specified in the topic_model....
- 583
2
votes
0
answers
992
views
BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable
I am using BERTopic to perform the topic modelling on 174,827 rows by using the following commands:
from bertopic import BERTopic
topic_model = BERTopic(language="english", ...
- 433
2
votes
0
answers
901
views
simpletransformers model trained on Colab doesn't work locally
I've followed the instructions in this link and trained a BERT model on Google Colab. I then downloaded it and tried to use it locally on my computer. But something went wrong.
Here is my code for ...
- 1,039
1
vote
2
answers
7k
views
BERT Multi-class text classification in Google Colab
I'm working on a data set of social media comments(including youtube links) as input features and the Myers-Biggs Personality Profile as the target label:
type posts
0 INFJ 'http://www....
1
vote
1
answer
1k
views
Codes worked fine one week ago, but keep getting error since yesterday: Fine-tuning Bert model training via PyTorch on Colab
I am new to Bert. Two weeks ago I successfully ran a fine-tuning Bert model on a nlp classification task though the outcome was not brilliant. Yesterday, however, when I tried to run the same code and ...
- 41
1
vote
1
answer
1k
views
DistilBERT, less than 512 tokens, Colab crash
I'm following this guide https://jalammar.github.io/a-visual-guide-to-using-bert-for-the-first-time/ and my text observations contain less than 250-300 words on average, so I don't have 512 tokens in ...
- 11
1
vote
1
answer
4k
views
from summarizer import Summarizer it's not running in Colab
I'm trying to use the BERT Text Summarizer inside Colab but I'm getting the following error
from summarizer import Summarizer
I am getting the error as below,
-------------------------------------...
1
vote
1
answer
2k
views
KeyBERT package is not working on Google Colab
I'm using KeyBERT on Google Colab to extract keywords from the text.
from keybert import KeyBERT
model = KeyBERT('distilbert-base-nli-mean-tokens')
text_keywords = model.extract_keywords(my_long_text)...
- 399
1
vote
0
answers
110
views
How to obtain the [CLS] sentence embedding of multiple sentences successively without facing a RAM crash?
I would like to obtain the [CLS] token's sentence embedding (as it represents the whole sentence's meaning) using BERT. I have many sentences (about 40) that belong to a Document, and 246 such ...
1
vote
0
answers
33
views
efficiently convert words to vectors using bert
I am trying to convert reviews Text to vectors using bert. I am using pretrained bert from tensorflowhub. While encoding it on TPU on google colab it crashes the memory.I have seen some articles most ...
- 23
1
vote
2
answers
8k
views
_batch_encode_plus() got an unexpected keyword argument 'return_attention_masks'
I am studying RoBERTA model to detect emotions in tweets.
On Google colab. Following this Noteboook file from Kaggle - https://www.kaggle.com/ishivinal/tweet-emotions-analysis-using-lstm-glove-roberta?...
0
votes
1
answer
2k
views
Error Running "config = RobertaConfig.from_pretrained( "/Absolute-path-to/BERTweet_base_transformers/config.json""
I'm trying to run the code 'transformers' version of this code to use the new pre-trained BERTweet model and I'm getting an error.
The following lines of code ran successfully in my Google Colab ...
- 581
0
votes
2
answers
395
views
Google Colab unable to Hugging Face model
I like to tag parts of speech using the BERT model. I used the Hugging face library for this purpose.
When I run the model on Hugging face API I got the output
However, when I run the code on Google ...
- 1,436
0
votes
1
answer
490
views
How to run 'run_squad.py' on google colab? It gives 'invalid syntax' error
I downloaded the file first using:
!curl -L -O https://github.com/huggingface/transformers/blob/master/examples/legacy/question-answering/run_squad.py
Then used following code:
!python run_squad.py \...
- 21
0
votes
1
answer
459
views
How to use a custom model with Tensorflow Hub?
My goal is to test out Google's BERT algorithm in Google Colab.
I'd like to use a pre-trained custom model for Finnish (https://github.com/TurkuNLP/FinBERT). The model can not be found on TFHub ...
- 555
0
votes
1
answer
70
views
Google Colab Bert instantiation Error using Tensorflow
I'm trying to construct a Bert Model using Tensorflow on Colab. This code was perfectly working weeks ago. Now if I try to instantiate the model I obtain the following error:
Some weights of the ...
0
votes
0
answers
17
views
BERTModel for named entity transfer learning
import tensorflow as tf
from transformers import BertModel
def create_model(bert_model, max_len):
input_ids = tf.keras.Input(shape=(max_len,), dtype='int32')
attention_masks = tf.keras.Input(...
0
votes
0
answers
43
views
I can't train a model with mobilbert or Bert in tflite-model-maker
I'm trying to create a neural network for text classification, and according to the script from tensorflow and google, you can use MobilBert for this.
First example - tensorflow and Second example - ...
- 1
0
votes
1
answer
40
views
CUDA out of memory when running validation test for SciBERT
I am trying to train SciBERT with a not-too-big dataset (roughly 10000 rows). I have partitioned the dataset as train, validation and loss, with proportion of 0.6, 0.2 and 0.2 respectively. Then I ...
0
votes
0
answers
89
views
BERT model does not seem working properly
I am quite new in BERT. I want to carry out a sentiment analysis based on hotel comments. I have an excel file including comments in a column and corresponding label (0 or 1) in another column. My ...
- 1
0
votes
1
answer
375
views
How to initialize and train an BERT model for NER in collab using data that I tagged manually
I'm trying to use BERT for Named Entity Recognition, the code is the following:
!pip install transformers
!pip install accelerate -U
import transformers
from transformers import ...
- 1
0
votes
0
answers
302
views
Got the answer TypeError: __call__() got an unexpected keyword argument 'max_length' when automatically generating meta descriptions using BERT
I am trying to automatically generate meta descriptions by using Python and BERT on a Google colab sheet, according to this tutorial https://lazarinastoy.com/automatically-generate-meta-descriptions-...
- 1
0
votes
1
answer
267
views
BERT Text Classification Tasks for Beginners
Can anyone list in simple terms tasks involved in building a BERT text classifier for someone new to CS working on their first project? Mine involves taking a list of paragraph length humanitarian aid ...
0
votes
1
answer
2k
views
Slow training of BERT model Hugging face
I am training the binary classfier using BERT model implement in hugging face library
training_args = TrainingArguments(
"deleted_tweets_trainer",
num_train_epochs = ...
- 121
0
votes
0
answers
836
views
Receive "TypeError: 'DistilBertTokenizer' object is not callable" when using KeyBERT on Colab
Running KeyBERT to extract keywords on Google Colab gives with the following codes:
from keybert import KeyBERT
model = KeyBERT('distilbert-base-nli-mean-tokens')
keywords = model.extract_keywords(doc,...
- 399
0
votes
0
answers
767
views
Training a BERT and Running out of memory - Google Colab
I keep running out of memory even after i bought google colab pro which has 25gb RAM usage. I have no idea why is this happening. I tried every kernel possible (Google colab, Google colab pro, Kaggle ...
- 117
0
votes
1
answer
380
views
PermissionDeniedError: From /job:worker/replica:0/task:0
I am using google colaboratory to pretrain bert.My train_data is on google cloud storage,my code is on colab
IS_COLAB_BACKEND = 'COLAB_GPU' in os.environ
if IS_COLAB_BACKEND:
from google.colab ...
-2
votes
1
answer
348
views
extract_features sentence embedding BERT
I'm using this code to get the embeddings of sentences that are in my dataset(I'm using my pretrained model).
`python extract_features.py \
--input_file=/tmp/input.txt \
--output_file=/tmp/...
- 39