RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
```python from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf") model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-chat-hf") inputs = ... inputs = tokenizer.batch_encode_plus(inputs, return_tensors="pt", padding=True) model.generate(**inputs, **generate_kwargs) ``` RuntimeError: probability tensor contains either `inf`, `nan` or element < 0 I got this error while doing inference for text generation, in particular when the batch size is great than 1. I did not get this error and generate correctly when the batch size is set to 1. Does anyone see the same issue?
Posted by Liyan06 on Jul 19, 2023, 12:38 AM
Tags
Related issues
- struct's lifetime requirements can be bypassed via covariant field assignment
- Proposal to Reorganize the Moderation Team and Review Moderation Policies
- Progressive FPS degradation appears to be related to dota2.com — blocking the domain prevents the performance loss
- proposal: container/...: generic collection types
- Better Custom Endpoint BYOK
- A concrete policy for backward compatibility until Godot 5.0
Raise real-world problems, find people facing the same issue, and discuss solutions.2,500 issues0 comments
github (Other)
struct's lifetime requirements can be bypassed via covariant field assignment
Proposal to Reorganize the Moderation Team and Review Moderation Policies
Progressive FPS degradation appears to be related to dota2.com — blocking the domain prevents the performance loss
proposal: container/...: generic collection types
Better Custom Endpoint BYOK
A concrete policy for backward compatibility until Godot 5.0
New MTP models not supported
Tracking: 429 / Capacity Issues
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
```python from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-2-7b-chat-hf") model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-chat-hf") inputs = ... inputs = tokenizer.batch_encode_plus(inputs, return_tensors="pt", padding=True) model.generate(**inputs, **generate_kwargs) ``` RuntimeError: probability tensor contains either `inf`, `nan` or element < 0 I got this error while doing inference for text generation, in particular when the batch size is great than 1. I did not get this error and generate correctly when the batch size is set to 1. Does anyone see the same issue?
Comments (0)
No comments yet. Start the discussion.