Questions tagged [elmo]
A tag for ELMo, a way to model words into a deep contextualized representation. It is part of and developed by AllenNLP.
71
questions
12
votes
1
answer
8k
views
What is the difference between Sentence Encodings and Contextualized Word Embeddings?
I have seen both terms used while reading papers about BERT and ELMo so I wonder if there is a difference between them.
- 133
6
votes
1
answer
379
views
Swap gnews with ELMo in the simple colab tutorial
I'm working on this colab notebook:
https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/tf2_text_classification.ipynb
I'd like to replace the gnews swivel embeddings ...
- 19.2k
4
votes
1
answer
602
views
Training SVM classifier (word embeddings vs. sentence embeddings)
I want to experiment with different embeddings such Word2Vec, ELMo, and BERT but I'm a little confused about whether to use the word embeddings or sentence embeddings, and why. I'm using the ...
- 125
4
votes
1
answer
2k
views
How to get similar words in pre trained ELMO embedding?
How to get similar word for a given word in the pre trained ELMO Embedding? For Example: In Glove, we have glove_model.most_similar() to find the most similar word and its embedding for any given word....
- 89
3
votes
1
answer
2k
views
Strongly increasing memory consumption when using ELMo from Tensorflow-Hub
I am currently trying to compare the similarity of millions of documents. For a first test on a CPU I reduced them to around 50 characters each and try to get the ELMo Embedding for 10 of them at a ...
- 357
3
votes
1
answer
887
views
Understanding ELMo's number of presentations
I am trying my hand at ELMo by simply using it as part of a larger PyTorch model. A basic example is given here.
This is a torch.nn.Module subclass that computes any number of ELMo
...
- 27.7k
3
votes
1
answer
1k
views
Paragraph embedding with ELMo
I'm trying to understand how to prepare paragraphs for ELMo vectorization.
The docs only show how to embed multiple sentences/words at the time.
eg.
sentences = [["the", "cat", "is", "on", "the", "...
- 113
3
votes
1
answer
2k
views
ELMo - How to train trainable parameters
I am new to tensorflow-hub and came across the ELMo model (https://www.tensorflow.org/hub/modules/google/elmo/2).
According to the original paper, the ELMo representation is a weighted average of ...
- 681
3
votes
0
answers
969
views
Using Elmo models to predict the masked word in a sentence
I have a homework which requires me to build an algorithm that can guess a missing word from a sentence. For example, when the input sentence is : " I took my **** for a walk this morning" , I want ...
2
votes
1
answer
3k
views
How to use BERT and Elmo embedding with sklearn
I created a text classifier that uses Tf-Idf using sklearn, and I want to use BERT and Elmo embedding instead of Tf-Idf.
How would one do that ?
I'm getting Bert embedding using the code below:
from ...
- 5,195
2
votes
2
answers
1k
views
error when try to import the Elmo module from TensorFlow Hub?
I'm not able to import the elmo module from TensorFlow Hub. I am able to import other modules and use them successfully. I'm running TF2.0 on a GCP Jupyterlab instance with GPUs.
When I try this:
...
- 2,497
2
votes
1
answer
1k
views
LSTM - Elmo from TF-Hub with TF2.0 pure implementation
I have the following problem mainly due to my lack of expertise with TF in general and TF2.0 in particular.
I try to use Elmo embedding with a keras LSTM implementation.
The code that I am inspired ...
2
votes
2
answers
2k
views
keras pad_sequence for string data type
I have a list of sentences. I want to add padding to them; but when I use keras pad_sequence like this:
from keras.preprocessing.sequence import pad_sequences
s = [["this", "is", "a", "book"], ["this"...
2
votes
1
answer
554
views
Calculate cosine similarity for elmo model
I am trying to calculate the cosine similarity of wordsim set using the Elmo model. This may not make sense since it is designed for sentence word embedding, but I want to see how the model performs ...
- 191
2
votes
1
answer
761
views
confuse about parameter 'tokens_length' of elmo model in tensorflow hub
I'm looking the ELMo model in tensorflow hub and, I'm not very clear about what does tokens_length = [6, 5] means in the flow example use:
(https://tfhub.dev/google/elmo/2)
elmo = hub.Module("https://...
- 542
2
votes
1
answer
246
views
I am trying to use Elmo Embedding using TensorFlow 2 and Getting this error for model.fit command: 'NoneType' object has no attribute 'outer_context'
Importing Elmo Embedding Layer from TF-hub Using TF 2
# Imported Elmo Layer
elmo_model_path = "https://tfhub.dev/google/elmo/3"
elmo_layer = hub.KerasLayer(elmo_model_path, input_shape=[], ...
2
votes
2
answers
444
views
Keras TensorFlow Hub: Getting started with simple ELMO network
I'm trying to get a simple ELMO model working from TensorFlow hub but, it's turning out to be a challenge.
When I run the my code, I'm getting the error: "Inputs to eager execution function ...
- 1,866
2
votes
0
answers
349
views
Can I train an ELMo model from scratch using allennlp.modules.elmo.Elmo?
I've been looking to train my own ELMo model for the past week and came across these two implementations allenai/bilm-tf & allenai/allennlp. I've been facing a few roadblocks for a few techniques ...
- 339
2
votes
0
answers
1k
views
Certificate validation failed for TensorFlow hub, Elmo module
I am trying to import the Elmo module using the url:
elmo = hub.Module("https://tfhub.dev/google/elmo/3", trainable=True)
but am unable to load it as it is giving an error:
URLError: urlopen ...
- 33
2
votes
0
answers
734
views
tensorflow_hub to pull BERT embedding on windows machine
I would like to get BERT embedding using tensorflow hub. I found it very easy to get ELMO embedding and my steps are below. Could anyone explain how to get BERT embedding on a windows machine? I found ...
- 6,258
2
votes
0
answers
307
views
How can we fix following error " InternalError: Unable to get element as bytes"?
I am a student, and I am trying to implement text classification using elmo on keras. I imported elmo layer from tensorflow-hub.
def ELMoEmbedding(x):
return embed(inputs={ "tokens": tf.squeeze(...
- 824
2
votes
0
answers
586
views
What is the difference between word2vec, glove, and elmo? [duplicate]
What is the difference between word2vec, glove, and elmo? According to my understanding all of them are used for training word embedding, am I correct?
- 824
1
vote
1
answer
5k
views
Keras Prediction result (getting score,use of argmax)
I am trying to use the elmo model for text classification for my own dataset. The training is completed and the number of classes is 4(used keras model and elmo embedding).In the prediction, I got a ...
- 23
1
vote
1
answer
1k
views
Using Elmo with tf.Keras throws ValueError: could not convert string to float
I am trying to use Elmo with tf.keras. However, the call to model.fitcauses ValueError: could not convert string to float
Tensorflow version: 1.13.1
Numpy version: 1.14.6
Here is the full code:
...
- 6,470
1
vote
1
answer
581
views
Allennlp: How to load a pretrained ELMo as the embedding of allennlp model?
I am new in allennlp. I trained an elmo model to apply it to other allennlp models as the embedding but failed. It seems that my model is not compatible to the interface the config gives. What can I ...
- 13
1
vote
1
answer
438
views
Keras: How to load a model with pretrained ELMO Layer
I have trained a Deep Learning network that has a pretrained ELMO layer. I've saved the model and weights using the code below.
model.save("model.h5")
model.save_weights("weights.h5&...
- 1,866
1
vote
1
answer
502
views
Is it possible and how to reduce the dimensions of ELMO embeddings using tf-hub elmo model?
I am using ELMO model from tensorflow hub.
import tensorflow as tf
import tensorflow_hub as hub
from tensorflow.contrib import predictor
elmo = hub.Module("https://tfhub.dev/google/elmo/3", ...
- 646
1
vote
1
answer
5k
views
FailedPreconditionError: Error while reading resource variable module/bilm/CNN_proj/W_proj from Container: localhost
I am trying to use pre-trained elmo embeddings in jupyter notebook with python 3.7.
Tensorflow version - 1.14.0
This is my code
def ElmoEmbeddingLayer(x):
print(x.shape)
module = hub.Module("...
- 45
1
vote
1
answer
980
views
Get similarity score between 2 words using Pre trained Bert, Elmo
I am trying to compare Glove, Fasttext, Bert ,Elmo on basis on similarity between 2 words using pre-trained models of Wiki. Glove and Fasttext had pretrained models which could easily be used with ...
- 663
1
vote
0
answers
34
views
tf hub.module can't load elmo. I hope it works normally
When I run the following code with Jupyter Notebook, it originally worked normally, but suddenly an error pops up.
import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
tf....
- 11
1
vote
0
answers
65
views
Error with ELMO embeddings - TensorArray has size zero, but element shape [?,256] is not fully defined. Currently only static shapes are supported
I Am trying to implement ELMO embeddings via tensorflow in a neural network. Here is a code snippet of my network :
def get_elmo_embeds_model():
input_text = tf.keras.layers.Input(shape=(1,), ...
- 11
1
vote
1
answer
2k
views
Tensorflow urllib.error.URLError: <urlopen error Errno 60 Operation timed out>
Recently, I tried to use the elmo in tensorflow, but I meet some mistakes, if you can help me, I would be really appreciate.
this is my test_code:
import tensorflow as tf
import tensorflow_hub as hub
...
- 11
1
vote
0
answers
412
views
Flair Elmo Embeddings use in Pytorch model
I am trying to generate the elmo embeddings on a PyTorch model, on every batch iteration, like:
for batch in iterator:
optimizer.zero_grad()
embeddings = get_elmo_embeddings(batch....
- 11
1
vote
1
answer
282
views
Training data preparation for Training ELMO Embedding from scratch
I am trying to build my own custom chemical domain ELMO embedding. I am following the instruction from https://github.com/allenai/bilm-tf
How do I prepare the training data if I have many multi word ...
- 13
1
vote
1
answer
279
views
Allennlp german elmo model
I want to build a Q&A bot with allennlp. I already found the code for it and an English model - I want to use the bot with german. I already searched for it, but I only found this model which is ...
- 628
1
vote
1
answer
2k
views
TensorFlow - tensorflow.python.framework.errors_impl.FailedPreconditionError
I tried to update the following code from TF 1.7 to TF 1.14 :
def build(self, input_shape):
self.max_length = input_shape[1]
self.word_mapping = [x[1] for x in sorted(self.idx2word.items(), ...
- 41
1
vote
1
answer
1k
views
Problems with variable_scope in ELMo Embedding layer with Keras
I'm trying to use a custom ElmoEmbeddingLayer for my model in Keras with tensorflow-gpu==2.0.0-rc0.
This is the class:
class ElmoEmbeddingLayer(Layer):
def __init__(self, trainable=True, ...
1
vote
0
answers
455
views
How to reshape keras mask within custom layer
Note: I posted about this issue already here. I'm creating a new question because:
1. I think the issue specifically relates to reshaping my mask within my custom layer, but I'm not sure enough of ...
- 346
1
vote
1
answer
208
views
Keras PREDICTION throws 'TypeError: ufunc 'add' did not contain a loop with signature matching types dtype('<U4') dtype('<U4') dtype('<U4')'
I built a slot filling (a type of sequence classification) model with structure: custom ELMo embeddings layer - BiLSTM - CRF.
It trains fine. But on prediction I get:
'TypeError: ufunc 'add' did ...
- 346
1
vote
0
answers
465
views
Not able to install Bert Serving server. "Could not find a version that satisfies the requirement bert-serving-server"
I'm trying to set up bert-as-service on my office machine and I keep getting the same error.
I have Python 3.6.8 :: Anaconda, Inc. and Tensorflow 1.10.0 on my Conda environment and Python 3.7.2 on ...
- 11
1
vote
1
answer
205
views
My ELMo trained model gives output dictionary with different fields from those from the tf.hub model
When using a pre-trained ElMo model, I get an output dictionary different from the output dictionary explained in the published tf.hub model
My output dictionary's signature is
> model = ...
- 2,290
1
vote
1
answer
1k
views
Output from elmo pretrained model
I am working on sentiment analysis. I am using elmo method to get word embeddings. But i am confused with the output this method is giving. Consider the code given in tensor flow website:
elmo = ...
- 355
1
vote
1
answer
3k
views
Comparison among ELMo, BERT, and GloVe
What are the differences among ELMo, BERT, and GloVe in word representation? How differently do they perform word embedding tasks? Which one is better and what advantages and disadvantages does each ...
user14251114
0
votes
1
answer
856
views
Why i getting Highway.forward: `input` must be present when running : from elmoformanylangs import Embedder
I am trying to use ELMoForManyLangs programmatically, by using Embedder python object.
from elmoformanylangs import Embedder
e = Embedder('/content/drive/MyDrive/ColabNotebooks/158', batch_size = 64)
...
0
votes
1
answer
2k
views
RuntimeError: variable_scope module_1/ was unused but the corresponding name_scope was already taken. How can i fix it
i got this problem with elmo and tensorflow and i wanna fix it without downgrade. what should i do
`**CODE**
import tensorflow_hub as hub
import tensorflow as tf
#Elmo
elmo = hub.Module("https://...
- 1
0
votes
1
answer
145
views
Why my ELMo-CNN model gives worse performance than Word2vec?
I want to compare the performance between ELMo and word2vec as word embedding using the CNN model by classifying 4000 tweet data on five class labels, but the results show that ELMo gives worse ...
- 1
0
votes
1
answer
721
views
Elmo embeddings errors
i'm using ELMO to create embeddings for my dataset, i always get this error:
embed=elmo(data,signature="default",as_dict=True)["elmo"]
TypeError: 'AutoTrackable' object is not ...
0
votes
1
answer
173
views
How do I produce ELMo embeddings for tokenised strings without getting "Function call stack: pruned"?
I am trying to produce ELMo embeddings for batches of tokenised strings. However I keep receiving the following error:
Traceback (most recent call last):
File "/home/lorcan/.local/lib/python3.6/...
- 555
0
votes
1
answer
174
views
TF-Hub Elmo uses which word embedding to concatenate with characters in Highway layer
I understand that Elmo uses CNN over characters for character embeddings. However I do not understand how the character embeddings are concatenated with word embeddings in the Highway network. In the ...
- 13
0
votes
0
answers
40
views
Unable to train BiLSTM due to FailedPreconditionError
I am building a BiLSTM model using Elmo embeddings with TensorFlow Hub.
My code looks like this:
import tensorflow as tf
import tensorflow_hub as hub
from keras.layers import Layer
import keras....
