dnrops
/
transformers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[`Mixtral`] update conversion script to reflect new changes (#28068) 

* Update convert_mixtral_weights_to_hf.py

* forward contrib credits from original fix

---------

Co-authored-by: thomasw21 <thomasw21@users.noreply.github.com>
This commit is contained in:
Younes Belkada 2023-12-15 14:05:20 +01:00 committed by GitHub
parent 70a127a37a
commit d269c4b2d7
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
src/transformers/models/mixtral/convert_mixtral_weights_to_hf.py
View File

@ -65,7 +65,7 @@ def write_model(model_path, input_base_path, model_size, safe_serialization=True
num_shards = 1
# For some reason this is a string in the params.json
sliding_window = int(params["sliding_window"])
sliding_window = int(params["sliding_window"]) if "sliding_window" in params else None
n_layers = params["num_hidden_layers"]
n_heads = params["num_attention_heads"]
n_heads_per_shard = n_heads // num_shards