96 lines
4.1 KiB
Markdown
96 lines
4.1 KiB
Markdown
<!--Copyright 2024 The HuggingFace Team. All rights reserved.
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
|
|
the License. You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
|
|
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
|
|
specific language governing permissions and limitations under the License.
|
|
|
|
⚠️ Note that this file is in Markdown but contains specific syntax for our doc-builder (similar to MDX) that may not be
|
|
rendered properly in your Markdown viewer.
|
|
|
|
-->
|
|
|
|
# Phi-3
|
|
|
|
## Overview
|
|
|
|
The Phi-3 model was proposed in [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://arxiv.org/abs/2404.14219) by Microsoft.
|
|
|
|
### Summary
|
|
|
|
The abstract from the Phi-3 paper is the following:
|
|
|
|
We introduce phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3.5 (e.g., phi-3-mini achieves 69% on MMLU and 8.38 on MT-bench), despite being small enough to be deployed on a phone. The innovation lies entirely in our dataset for training, a scaled-up version of the one used for phi-2, composed of heavily filtered web data and synthetic data. The model is also further aligned for robustness, safety, and chat format. We also provide some initial parameter-scaling results with a 7B and 14B models trained for 4.8T tokens, called phi-3-small and phi-3-medium, both significantly more capable than phi-3-mini (e.g., respectively 75% and 78% on MMLU, and 8.7 and 8.9 on MT-bench).
|
|
|
|
The original code for Phi-3 can be found [here](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct).
|
|
|
|
## Usage tips
|
|
|
|
- This model is very similar to `Llama` with the main difference of [`Phi3SuScaledRotaryEmbedding`] and [`Phi3YarnScaledRotaryEmbedding`], where they are used to extend the context of the rotary embeddings. The query, key and values are fused, and the MLP's up and gate projection layers are also fused.
|
|
- The tokenizer used for this model is identical to the [`LlamaTokenizer`], with the exception of additional tokens.
|
|
|
|
## How to use Phi-3
|
|
|
|
<Tip warning={true}>
|
|
|
|
Phi-3 has been integrated in the development version (4.40.0.dev) of `transformers`. Until the official version is released through `pip`, ensure that you are doing one of the following:
|
|
|
|
* When loading the model, ensure that `trust_remote_code=True` is passed as an argument of the `from_pretrained()` function.
|
|
|
|
* Update your local `transformers` to the development version: `pip uninstall -y transformers && pip install git+https://github.com/huggingface/transformers`. The previous command is an alternative to cloning and installing from the source.
|
|
|
|
</Tip>
|
|
|
|
```python
|
|
>>> from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
>>> model = AutoModelForCausalLM.from_pretrained("microsoft/Phi-3-mini-4k-instruct")
|
|
>>> tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")
|
|
|
|
>>> messages = [{"role": "user", "content": "Can you provide ways to eat combinations of bananas and dragonfruits?"}]
|
|
>>> inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
|
|
|
|
>>> outputs = model.generate(inputs, max_new_tokens=32)
|
|
>>> text = tokenizer.batch_decode(outputs)[0]
|
|
>>> print(text)
|
|
<s><|user|>
|
|
Can you provide ways to eat combinations of bananas and dragonfruits?<|end|>
|
|
<|assistant|>
|
|
Certainly! Bananas and dragonfruits can be combined in various delicious ways. Here are some ideas for eating combinations of bananas and
|
|
```
|
|
|
|
## Phi3Config
|
|
|
|
[[autodoc]] Phi3Config
|
|
|
|
<frameworkcontent>
|
|
<pt>
|
|
|
|
## Phi3Model
|
|
|
|
[[autodoc]] Phi3Model
|
|
- forward
|
|
|
|
## Phi3ForCausalLM
|
|
|
|
[[autodoc]] Phi3ForCausalLM
|
|
- forward
|
|
- generate
|
|
|
|
## Phi3ForSequenceClassification
|
|
|
|
[[autodoc]] Phi3ForSequenceClassification
|
|
- forward
|
|
|
|
## Phi3ForTokenClassification
|
|
|
|
[[autodoc]] Phi3ForTokenClassification
|
|
- forward
|
|
|
|
</pt>
|
|
</frameworkcontent>
|