dnrops
/
Skywork
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update README_EN.md (#42) 

Quntization -> Quantization
This commit is contained in:
Ikko Eltociear Ashimine 2023-12-12 11:58:37 +09:00 committed by GitHub
parent 520daef18f
commit e79678ddcd
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
README_EN.md
View File

@ -1131,19 +1131,19 @@ Skywork-13B-Chat：
# Quantization
## 8bit Quntization
## 8bit Quantization
Skywork utilizes the widely-used 8-bit quantization method called [BitsAndBytes](https://github.com/TimDettmers/bitsandbytes). This method allows for quantizing performance with minimal loss and has been seamlessly integrated into the transformers library. Building upon BitsAndBytes, we offer two approaches for utilizing online quantization and offline 8-bit models.
To illustrate the usage of the int8 quantization model, we provide an example. Before you begin, please ensure that you have installed the BitsAndBytes library and the necessary dependencies. For detailed installation instructions, please refer to the [BitsAndBytes](https://github.com/TimDettmers/bitsandbytes) repository.
### Online Quntization
### Online Quantization
```python
model = AutoModelForCausalLM.from_pretrained("skywork-13B-Base", torch_dtype=torch.bfloat16,load_in_8bit=True, trust_remote_code=True).eval()
```
### Offline Quntization
### Offline Quantization
```python
model = AutoModelForCausalLM.from_pretrained("skywork-13B-Base-8bits", device_map="auto", torch_dtype=torch.bfloat16,trust_remote_code=True).eval()