3.9 KiB
ERNIE
Overview
ERNIE is a series of powerful models proposed by baidu, especially in Chinese tasks, including ERNIE1.0, ERNIE2.0, ERNIE3.0, ERNIE-Gram, ERNIE-health, etc.
These models are contributed by nghuyong and the official code can be found in PaddleNLP (in PaddlePaddle).
Usage example
Take ernie-1.0-base-zh as an example:
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("nghuyong/ernie-1.0-base-zh")
model = AutoModel.from_pretrained("nghuyong/ernie-1.0-base-zh")
Model checkpoints
| Model Name | Language | Description |
|---|---|---|
| ernie-1.0-base-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
| ernie-2.0-base-en | English | Layer:12, Heads:12, Hidden:768 |
| ernie-2.0-large-en | English | Layer:24, Heads:16, Hidden:1024 |
| ernie-3.0-base-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
| ernie-3.0-medium-zh | Chinese | Layer:6, Heads:12, Hidden:768 |
| ernie-3.0-mini-zh | Chinese | Layer:6, Heads:12, Hidden:384 |
| ernie-3.0-micro-zh | Chinese | Layer:4, Heads:12, Hidden:384 |
| ernie-3.0-nano-zh | Chinese | Layer:4, Heads:12, Hidden:312 |
| ernie-health-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
| ernie-gram-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
You can find all the supported models from huggingface's model hub: huggingface.co/nghuyong, and model details from paddle's official repo: PaddleNLP and ERNIE.
Resources
- Text classification task guide
- Token classification task guide
- Question answering task guide
- Causal language modeling task guide
- Masked language modeling task guide
- Multiple choice task guide
ErnieConfig
autodoc ErnieConfig - all
Ernie specific outputs
autodoc models.ernie.modeling_ernie.ErnieForPreTrainingOutput
ErnieModel
autodoc ErnieModel - forward
ErnieForPreTraining
autodoc ErnieForPreTraining - forward
ErnieForCausalLM
autodoc ErnieForCausalLM - forward
ErnieForMaskedLM
autodoc ErnieForMaskedLM - forward
ErnieForNextSentencePrediction
autodoc ErnieForNextSentencePrediction - forward
ErnieForSequenceClassification
autodoc ErnieForSequenceClassification - forward
ErnieForMultipleChoice
autodoc ErnieForMultipleChoice - forward
ErnieForTokenClassification
autodoc ErnieForTokenClassification - forward
ErnieForQuestionAnswering
autodoc ErnieForQuestionAnswering - forward