dnrops
/
transformers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
16,028 Commits 481 Branches 161 Tags 1.2 GiB
Commit Graph

16028 Commits

Author SHA1 Message Date
ydshieh 5d8f75dd93 build 2024-05-31 11:57:51 +02:00
ydshieh 225d9aa830 build 2024-05-31 11:57:05 +02:00
ydshieh 5704ef3ad3 build 2024-05-31 11:54:38 +02:00
ydshieh d0a8fe6784 build 2024-05-31 11:51:48 +02:00
ydshieh 5b2fa2e6ce build 2024-05-31 11:02:01 +02:00
Yih-Dar 2b9e252b16
Cleanup docker build (#31119) 
* remove

* build

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-29 19:43:51 +02:00
Dhruv Pai 5c88253556
Add on_optimizer_step to callback options (#31095) 
* Modified test

* Added on_optimizer_step to callbacks

* Move callback after step is called

* Added on optimizer step callback
 2024-05-29 16:20:59 +02:00
Joao Gante 4af705c6ce
Add VLM generation default contributor (#31115) 
* add Raushan

* add Raushan
 2024-05-29 15:17:14 +01:00
Younes Belkada cb879c5801
FIX / Docs: Fix GPTQ expected number of bits (#31111) 
Update overview.md
 2024-05-29 15:56:28 +02:00
Yih-Dar 1f84141391
Fix nightly circleci (#31114) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-29 15:42:39 +02:00
Zach Mueller d16053c867
Rm maintainer + migrate (#31089) 2024-05-29 09:35:37 -04:00
Matt 0bef4a2738
Fix faulty rstrip in module loading (#31108) 2024-05-29 13:33:26 +01:00
Matt 97a58a5d2c
Fix env.py in cases where torch is not present (#31113) 
* Fix env.py in cases where torch is not present

* Simplify the fix (and avoid some issues)
 2024-05-29 13:20:36 +01:00
Huazhong Ji c8861376ad
Improve `transformers-cli env` reporting (#31003) 
* Improve `transformers-cli env` reporting

* move the line `"Using GPU in script?": "<fill in>"` to in if conditional
statement

* same option for npu
 2024-05-29 11:57:54 +01:00
Lucain c3044ec2f3
Use `HF_HUB_OFFLINE` + fix has_file in offline mode (#31016) 
* Fix has_file in offline mode

* harmonize env variable for offline mode

* Switch to HF_HUB_OFFLINE

* fix test

* revert test_offline to test TRANSFORMERS_OFFLINE

* Add new offline test

* merge conflicts

* docs
 2024-05-29 11:55:43 +01:00
Younes Belkada bfe6f513b9
FEAT: Add mistral v3 conversion script (#30981) 
* add mistral v3 conversion script

* Update src/transformers/models/mistral/convert_mistral_weights_to_hf.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* fixup

---------

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
 2024-05-29 11:43:54 +02:00
Raushan Turganbay d521ba5797
Quantized KV cache: update quanto (#31052) 
* quanto latest version was refactored

* add error msg

* incorrect compare sign

* Update src/transformers/cache_utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2024-05-29 14:25:44 +05:00
amyeroberts a564d10afe
Deprecate low use models (#30781) 
* Deprecate models
- graphormer
- time_series_transformer
- xlm_prophetnet
- qdqbert
- nat
- ernie_m
- tvlt
- nezha
- mega
- jukebox
- vit_hybrid
- x_clip
- deta
- speech_to_text_2
- efficientformer
- realm
- gptsan_japanese

* Fix up

* Fix speech2text2 imports

* Make sure message isn't indented

* Fix docstrings

* Correctly map for deprecated models from model_type

* Uncomment out

* Add back time series transformer and x-clip

* Import fix and fix-up

* Fix up with updated ruff
 2024-05-28 18:07:07 +01:00
Younes Belkada 7f08817be4
Docs / Quantization: Redirect deleted page (#31063) 
Update _redirects.yml
 2024-05-28 18:29:22 +02:00
Younes Belkada 3264be4114
TST: Fix instruct-blip tests (#31088) 
* fix flan t5 tests

* better format
 2024-05-28 18:29:11 +02:00
Jonny Li 476890e9ae
Fix DeepSpeed compatibility with weight_norm (#30881) (#31018) 2024-05-28 17:25:15 +01:00
Albert Villanova del Moral aada568f73
Fix PretrainedConfig docstring with deprecated resume_download (#31014) 2024-05-28 17:47:35 +02:00
Yih-Dar 3af7bf30ad
skip `test_multi_gpu_data_parallel_forward` for `vit` and `deit` (#31086) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-28 17:44:52 +02:00
Younes Belkada ab19f907fd
FIX / OPT: Fix OPT multi-GPU training for `OPTForQuestionAnswering` (#31092) 
Update modeling_opt.py
 2024-05-28 17:06:00 +02:00
Younes Belkada 94d416f018
FIX: Add `accelerate` as a hard requirement (#31090) 
add accelerate
 2024-05-28 17:05:44 +02:00
Sigbjørn Skjæret 22dab246c5
Render chat template tojson filter as unicode (#31041) 
* Render chat template tojson filter as unicode

* ruff--
 2024-05-28 15:02:51 +01:00
Younes Belkada 4f98b14465
Docs / PEFT: Add PEFT API documentation (#31078) 
* add peft references

* add peft references

* Update docs/source/en/peft.md

* Update docs/source/en/peft.md
 2024-05-28 15:04:43 +02:00
Raushan Turganbay 779bc360ff
Watermark: fix tests (#30961) 
* fix tests

* style

* Update tests/generation/test_utils.py

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2024-05-28 17:07:42 +05:00
Lysandre Debut a3c7b59e31
Fix failing tokenizer tests (#31083) 
* Fix failing tokenizer tests

* Use small tokenizer

* Fix remaining reference
 2024-05-28 13:34:23 +02:00
NielsRogge 90da0b1c9f
[SuperPoint, PaliGemma] Update docs (#31025) 
* Update docs

* Add PaliGemma resources

* Address comment

* Update docs
 2024-05-28 13:22:06 +02:00
Sina Taslimi 66add161dc
Fix typo in trainer.py (#31048) 2024-05-28 12:09:32 +01:00
Pavel Iakubovskii 98e2d48e9a
Fix OWLv2 post_process_object_detection for multiple images (#31082) 
* Add test for multiple images

* [run slow] owlv2

* Fix box rescaling

* [run slow] owlv2
 2024-05-28 12:06:06 +01:00
Pavel Iakubovskii c31473ed44
Remove float64 cast for OwlVit and OwlV2 to support MPS device (#31071) 
Remove float64
 2024-05-28 11:41:40 +01:00
oOraph 936ab7bae5
fix from_pretrained in offline mode when model is preloaded in cache (#31010) 
* Unit test to verify fix

Signed-off-by: Raphael Glon <oOraph@users.noreply.github.com>

* fix from_pretrained in offline mode when model is preloaded in cache

Signed-off-by: Raphael Glon <oOraph@users.noreply.github.com>

* minor: fmt

Signed-off-by: Raphael Glon <oOraph@users.noreply.github.com>

---------

Signed-off-by: Raphael Glon <oOraph@users.noreply.github.com>
Co-authored-by: Raphael Glon <oOraph@users.noreply.github.com>
 2024-05-28 11:56:05 +02:00
Hengwen Tong 537deb7869
Remove redundant backend checks in training_args.py (#30999) 
* Remove backend checks in training_args.py

* Expilicit initialize the device

---------

Co-authored-by: tonghengwen <tonghengwen@cambricon.com>
 2024-05-28 11:52:47 +02:00
AP dd4654eab7
Update quicktour.md to fix broken link to Glossary (#31072) 
Update quicktour.md to fix broken link

Missing '/' in attention mask link in the transformers quicktour
 2024-05-28 11:50:45 +02:00
Clint Adams e18da4e3f2
fix "piano" typo (#31027) 2024-05-28 11:48:23 +02:00
Yih-Dar 8e3b1fef97
Remove `ninja` from docker image build (#31080) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-28 11:36:26 +02:00
Yih-Dar 8f0f7271d0
use `@main` (#31065) 
use main

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-28 10:53:28 +02:00
Yih-Dar 9d35edbb30
skip `test_model_parallelism` for 2 model test classes (#31067) 
skip

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-27 18:36:39 +02:00
Yoach Lacombe d355741eca
Fix pad_to_max_length Whisper (#30787) 
* fix pad_to_max_length Whisper

* add tests

* make style
 2024-05-27 16:09:05 +02:00
Marc Sun b84cd67526
Fix quanto tests (#31062) 
fix quanto tests
 2024-05-27 15:53:45 +02:00
amyeroberts cd797778e4
Update feature request label in template (#30940) 2024-05-27 15:16:47 +02:00
Eitan Turok 0a064dc0fc
Follow up: Fix link in dbrx.md (#30514) 
* Fix link in dbrx.md

* remove "though this may not be up to date"

---------

Co-authored-by: Lysandre Debut <hi@lysand.re>
 2024-05-27 14:57:43 +02:00
Yih-Dar d7942d9d27
unpin uv (#31055) 
[push-ci-image]

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-27 13:47:47 +02:00
Aymeric Roucher 84c4b72ee9
Redirect transformers_agents doc to agents (#31054) 2024-05-27 10:34:14 +02:00
Pablo Montalvo bdb9106f24
Paligemma- fix devices and dtype assignments (#31008) 
* fix devices and dtype assignments

* [run-slow]paligemma
 2024-05-24 19:02:55 +02:00
Ita Zaporozhets deba7655e6
Add split special tokens (#30772) 
* seems like `split_special_tokens` is used here

* split special token

* add new line at end of file

* moving split special token test to common tests

* added assertions

* test

* fixup

* add co-author

* passing rest of args to gptsan_japanese, fixing tests

* removing direct comparison of fast and slow models

* adding test support for UDOP and LayoutXLM

* ruff fix

* readd check if slow tokenizer

* modify test to handle bos tokens

* removing commented function

* trigger build

* applying review feedback - updated docstrings, var names, and simplified tests

* ruff fixes

* Update tests/test_tokenization_common.py

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* applying feedback, comments

* shutil temp directory fix

---------

Co-authored-by: Arthur Zucker <arthur.zucker@gmail.com>
Co-authored-by: Ita Zaporozhets <itazaporozhets@Itas-MBP.localdomain>
Co-authored-by: itazap <itazap@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
Co-authored-by: Ita Zaporozhets <itazaporozhets@Itas-MacBook-Pro.local>
 2024-05-24 08:38:58 -07:00
BHUVAN M e5103a76cc
added interpolation for vitmae model in pytorch as well as tf. (#30732) 
* added interpolation for vitmae model in pytorch as well as tf.

* Update modeling_vit_mae.py

irreugalr import fixed

* small changes and proper formatting

* changes suggested in review.

* modified decoder interpolate_func

* arguments and docstring fix

* Apply suggestions from code review

doc fixes

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>

---------

Co-authored-by: amyeroberts <22614925+amyeroberts@users.noreply.github.com>
 2024-05-24 16:20:09 +01:00
Yih-Dar a3cdff417b
save the list of new model failures (#31013) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-05-24 15:20:25 +02:00