dnrops
/
transformers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,706 Commits 481 Branches 161 Tags 1.2 GiB
Commit Graph

2169 Commits

Author SHA1 Message Date
dependabot[bot] 05b736c16e
Bump gitpython from 3.1.18 to 3.1.30 in /examples/research_projects/decision_transformer (#21010) 
Bump gitpython in /examples/research_projects/decision_transformer

Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.18 to 3.1.30.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.18...3.1.30)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-01-04 15:36:33 -05:00
milyiyo 3b309818e7
Refactor the function get_results (#20999) 2023-01-04 12:05:36 -05:00
Wang, Yi 9c9fe89f84
[run_clm example] add torch_dtype option for model load. (#20971) 
* [run_clm example] add torch_dtype option for model load.
for BLOOM 175B model. peak memory will reduce about 350G for inference. the weight of BLOOM in model hub is bfloat16

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

* add other type in option

* fix style

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
 2023-01-03 09:33:11 -05:00
Alex Hedges 0b686a8a1e
Remove non-breaking spaces (#20929) 
* Remove non-breaking space in comment

It was likely added unintionally.

* Remove remaining non-breaking spaces
 2022-12-29 02:12:40 -05:00
Márton Makrai 3830b3f74a
Fixes typo in the help text for --max_length (#20883) 2022-12-24 02:07:06 -05:00
NielsRogge d87e381f93
[Examples] Update big table (#20845) 
Update big table

Co-authored-by: Niels Rogge <nielsrogge@Nielss-MBP.localdomain>
 2022-12-21 11:34:31 +01:00
fzyzcjy ae3cbbcaf6
Fix tiny typo (#20841) 
* Fix typo

* Update README.md

* Update run_mlm_flax_stream.py

* Update README.md
 2022-12-20 03:17:59 -05:00
dependabot[bot] da95f6ca4c
Bump certifi in /examples/research_projects/visual_bert (#20673) 
Bumps [certifi](https://github.com/certifi/python-certifi) from 2020.6.20 to 2022.12.7.
- [Release notes](https://github.com/certifi/python-certifi/releases)
- [Commits](https://github.com/certifi/python-certifi/compare/2020.06.20...2022.12.07)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-08 11:15:42 -05:00
dependabot[bot] efd7c021ee
Bump certifi in /examples/research_projects/decision_transformer (#20677) 
Bumps [certifi](https://github.com/certifi/python-certifi) from 2021.10.8 to 2022.12.7.
- [Release notes](https://github.com/certifi/python-certifi/releases)
- [Commits](https://github.com/certifi/python-certifi/compare/2021.10.08...2022.12.07)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-08 11:15:11 -05:00
dependabot[bot] 9e33e19bf5
Bump certifi in /examples/research_projects/lxmert (#20672) 
Bumps [certifi](https://github.com/certifi/python-certifi) from 2020.6.20 to 2022.12.7.
- [Release notes](https://github.com/certifi/python-certifi/releases)
- [Commits](https://github.com/certifi/python-certifi/compare/2020.06.20...2022.12.07)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-12-08 11:14:54 -05:00
Emmanuel Schmidbauer 0526a075c5
run_speech_recognition_seq2seq.py: add cache_dir param to dataset (#20540) 2022-12-07 18:23:16 +00:00
Francisco Kurucz f821bea0ad
Fix link to speech encoder decoder model in speech recognition readme (#20633) 2022-12-06 15:46:41 -05:00
Wang, Yi ae06bce888
exclude jit time from the speed metric calculation of evaluation and prediction (#20553) 
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>

Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
 2022-12-06 07:37:01 -05:00
Sylvain Gugger 60d1f31bb0
v4.26.0.dev0 2022-12-01 16:19:33 -05:00
Wang, Yi d752337baa
QnA example: add speed metric (#20522) 2022-12-01 12:04:19 -05:00
Pi Esposito fb2b45e562
add in layer gpt2 tokenizer (#20421) 
* add minimal working gpt2 tokenizer

* graph mode and output equivalence tests working

* not today tensorflow. serialization test passing!

* fix style, documentation, docstrings and all that jazz

* passing consistency checks

* move keras nlp to tf dependencies

* fix tf modeling utils and gpt2 attention to enable compiling

* fix (I hope) keras nlp dependencies

* rever changes on generation

* remove debug prints

* remove redundant tf dummy objects

* add from config, get config and max length settings to address review

* let flake ignore the error on distillation you are welcome

* test from config

* add padding test

* address sgugger review
 2022-11-29 10:02:40 -05:00
Katie Le 667ccea722
Replace assertion with ValueError exceptions in run_image_captioning_flax.py (#20365) 
* replace 4 asserts with ValueError exception for control flow

* Update examples/flax/image-captioning/run_image_captioning_flax.py

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

* Update examples/flax/image-captioning/run_image_captioning_flax.py

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

* reformatted file

* uninstalled trasformers and applied make style

Co-authored-by: Bibi <Bibi@katies-mac.local>
Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
 2022-11-28 15:06:25 +00:00
dependabot[bot] 624ae09f5c
Bump pillow in /examples/research_projects/decision_transformer (#20378) 
Bumps [pillow](https://github.com/python-pillow/Pillow) from 9.0.1 to 9.3.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/9.0.1...9.3.0)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-22 08:15:42 -05:00
Zachary Mueller 9d1ef009b8
Fix flakey test with seed (#20318) 2022-11-18 11:33:25 -05:00
Sylvain Gugger a3f7458066
Pin to the right version... 2022-11-18 07:12:55 -05:00
Sanchit Gandhi c29a2f7c9c
[ASR Examples] Update README for Whisper (#20230) 
* [ASR Examples] Update README for seq2seq

* add language info

* add training results

* re-word
 2022-11-18 11:24:25 +00:00
Zachary Mueller 441811ecd7
Fix summarization script (#20286) 2022-11-16 15:57:07 -05:00
Jiahao Li 9681f052a1
Fix result saving errors of pytorch examples (#20276) 2022-11-16 09:51:04 -05:00
Zachary Mueller 822ae69c1b
Update reqs to include min gather_for_metrics Accelerate version (#20242) 
* Update reqs to include min gather_for_metrics Accelerate version

* Other reqs
 2022-11-15 13:28:00 -05:00
Muhammad Sakib Khan Inan 777b1bfe62
New logging support to "Trainer" Class (ClearML Logger) (#20184) 
* Init Update

* ClearML Callbacks integration

* update corrections

* args reporting updated

* {'tensorboard': False, 'pytorch': False}

* ClearML Tests added

* add clearml

* output_uri=True in Task.init

* reformatted integrations.py

* reformatted and fixed

* IF-ELSE statement issue on "has_clearml" resolved

* Add clearml in main callback docs

* Add additional clearml documentation

* Update src/transformers/integrations.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Accept suggestion

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Accept suggestion

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Small change in comments

* Make style clearml

* Accept suggestion

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Victor Sonck <victor.sonck@gmail.com>
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-11-15 10:08:59 -05:00
Yih-Dar cf7b98b807
Fix `run_clip.py` (#20234) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2022-11-15 15:45:21 +01:00
Ming Liu 36b063ed4f
Update README.md (#20188) 
There is typo in the original hyperlink.

Below is the original version:
Based on the script [`run_translation_no_trainer.py`](https://github.com/huggingface/transformers/blob/main/examples/pytorch/translation/**run_translationn_no_trainer.py**).
 2022-11-14 12:53:02 -05:00
Sanchit Gandhi af1a7c8ca3
[Examples] Generalise Seq2Seq ASR to handle Whisper (#19519) 
* merge conflicts

* bos and eos in datacollator

* (temp) hardcode removal of attention mask

* freeze encoder

* actually freeze encoder

* set max length / num beams according to gen kwargs

* (temp) fix tests

* don't pop attn mask

* override return attention mask config from Hub

* Hub configs updated 🤗

* final fixes

* update type annotations

* backward comp
 2022-11-14 17:45:46 +00:00
Joao Gante f270b960d6
Generate: move generation_*.py src files into generation/*.py (#20096) 
* move generation_*.py src files into generation/*.py

* populate generation.__init__ with lazy loading

* move imports and references from generation.xxx.object to generation.object
 2022-11-09 15:34:08 +00:00
Tom Aarsen b8112eddec
Replace unsupported facebookresearch/bitsandbytes (#20093) 
With https://github.com/TimDettmers/bitsandbytes, which is by the same author and is still being updated
 2022-11-07 08:52:03 -05:00
bhuang 3502c202f9
Update README.md (#20063) 2022-11-04 08:56:54 -04:00
Sylvain Gugger 06886d5a68
Only resize embeddings when necessary (#20043) 
* Only resize embeddings when necessary

* Add comment
 2022-11-03 12:05:04 -04:00
amyeroberts a6b7759880
Add Image Processors (#19796) 
* Add CLIP image processor

* Crop size as dict too

* Update warning

* Actually use logger this time

* Normalize doesn't change dtype of input

* Add perceiver image processor

* Tidy up

* Add DPT image processor

* Add Vilt image processor

* Tidy up

* Add poolformer image processor

* Tidy up

* Add LayoutLM v2 and v3 imsge processors

* Tidy up

* Add Flava image processor

* Tidy up

* Add deit image processor

* Tidy up

* Add ConvNext image processor

* Tidy up

* Add levit image processor

* Add segformer image processor

* Add in post processing

* Fix up

* Add ImageGPT image processor

* Fixup

* Add mobilevit image processor

* Tidy up

* Add postprocessing

* Fixup

* Add VideoMAE image processor

* Tidy up

* Add ImageGPT image processor

* Fixup

* Add ViT image processor

* Tidy up

* Add beit image processor

* Add mobilevit image processor

* Tidy up

* Add postprocessing

* Fixup

* Fix up

* Fix flava and remove tree module

* Fix image classification pipeline failing tests

* Update feature extractor in trainer scripts

* Update pad_if_smaller to accept tuple and int size

* Update for image segmentation pipeline

* Update src/transformers/models/perceiver/image_processing_perceiver.py

Co-authored-by: Alara Dirik <8944735+alaradirik@users.noreply.github.com>

* Update src/transformers/image_processing_utils.py

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* Update src/transformers/models/beit/image_processing_beit.py

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>

* PR comments - docstrings; remove accidentally added resize; var names

* Update docstrings

* Add exception if size is not in the right format

* Fix exception check

* Fix up

* Use shortest_edge in tuple in script

Co-authored-by: Alara Dirik <8944735+alaradirik@users.noreply.github.com>
Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
 2022-11-02 11:57:36 +00:00
Sylvain Gugger c3a93d8d82
v4.25.0.dev0 2022-10-31 21:48:40 -04:00
Sanchit Gandhi f38a145418
[ASR] Update 'tasks' for model card (#19986) 2022-10-31 16:50:17 +00:00
regisss 5d2d51a0fb
Fix LR (#19875) 2022-10-26 08:35:53 -04:00
GMFTBY 71786b10c5
Adding the state-of-the-art contrastive search decoding methods for the codebase of generation_utils.py (#19477) 
* add: the contrastive search for generaton_utils

* add: testing scripts for contrastive search under examples/text-generation

* update the quality of codes

* revise the docstring; make the generation_contrastive_search.py scripts;

* revise the examples/pytorch/text-generation/run_generation_contrastive_search.py to the auto-APIs format

* revise the necessary documents

* fix: revise the docstring of generation_contrastive_search.py

* Fix the code indentation

* fix: revise the nits and examples in contrastive_search docstring.

* fix the copyright

* delete generation_contrastive_search.py

* revise the logic in contrastive_search

* update the intergration test and the docstring

* run the tests over

* add the slow decorate to the contrastive_search intergrate test

* add more test

* do the style, quality, consistency checks
 2022-10-19 10:17:46 +01:00
amyeroberts 31ec424b3d
Add decorator to flaky test (#19674) 2022-10-18 18:51:37 +01:00
David Yang a23819ed6a
Clean up deprecation warnings (#19654) 
* Clean up deprecation warnings

Notes:
Changed some strings in tests to raw strings, which will change the literal content of the strings as they are fed into whatever machine handles them.
Test cases for past in the past/past_key_values switch changed/removed due to warning of impending removal

* Add PILImageResampling abstraction for PIL.Image.Resampling
 2022-10-18 13:34:47 -04:00
Yifan Yang 94d7c3ba44
[Examples] make default preprocessing_num_workers=1 (#19684) 
* [Examples] make default preprocessing_num_workers=1

* [Examples] revert changes in research projects
 2022-10-17 14:17:01 -04:00
Spacefish 2af36f957f
Add pillow to layoutlmv3 example requirements.txt (#19663) 2022-10-17 08:41:57 -04:00
Duong A. Nguyen 4212bb0d60
[Re-submit] Compute true loss Flax examples (#19504) 
* Compute true loss

* fixup

* final

* final

* final

* Update examples/flax/language-modeling/run_bart_dlm_flax.py

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

* jax.tree_map => jax.tree_util.tree_map

* Compute true loss

* final

* fixup

* final

* final

* Update examples/flax/language-modeling/run_bart_dlm_flax.py

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>

* jax.tree_map => jax.tree_util.tree_map

Co-authored-by: Sanchit Gandhi <93869735+sanchit-gandhi@users.noreply.github.com>
 2022-10-13 11:33:36 +01:00
Sanchit Gandhi eefcecaa35
[Examples] Fix typos in run speech recognition seq2seq (#19514) 2022-10-12 15:33:22 +01:00
FilipposVentirozos 4ed0fa3676
Fix pytorch seq2seq qa (#19258) 
* fixed typo for SQuAD

* Fixed the preprocess_validation_function function for the labels to reflect the remaining truncated instances

* Rolled back the trainer_seq2seq_qa.py for UnboundLocalError: local variable 'metrics' referenced before assignment

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2022-10-12 08:33:44 -04:00
regisss bb2cfd1824
Add multi-node conditions in trainer_qa.py and trainer_seq2seq.py (#19502) 
* Add multi-node conditions in trainer_qa.py and trainer_seq2seq.py

* Code improvement
 2022-10-11 22:48:56 -04:00
Lysandre 10100979ed Dev version 2022-10-10 17:25:40 -04:00
Kaiyu Yang e150c4e2fe
Fix the error message in run_t5_mlm_flax.py (#19282) 2022-10-10 14:51:11 +01:00
wei zhao 7d5ce6802e
Fix typo in image-classification/README.md (#19424) 
Fix link typo of the following content.
PyTorch version, Trainer
PyTorch version, no Trainer
 2022-10-10 09:16:58 -04:00
ddobokki fa4bcd5274
edit: cast attention_mask to long in DataCollatorCTCWithPadding (#19369) 
* edit: casting attention_mask to long in DataCollatorCTCWithPadding

* edit: casting attention_mask to long in DataCollatorCTCWithPadding
 2022-10-07 10:05:48 -04:00
Ilaygoldman ce2620194b
Change link of repojacking vulnerable link (#19393) 
The link to https://github.com/vasudevgupta7/bigbird is vulnerable to repojacking (it redirects to the orignial project that changed name), you should change the link to the current name of the project. if you won't change the link, an attacker can open the linked repository and attacks users that trust your links
 2022-10-06 23:06:39 +02:00