dnrops
/
transformers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,891 Commits 481 Branches 161 Tags 1.2 GiB
Commit Graph

12 Commits

Author SHA1 Message Date
Sylvain Gugger acc3bd9d2a
Enforce string-formatting with f-strings (#10980) 
* First third

* Styling and fix mistake

* Quality

* All the rest

* Treat %s and %d

* typo

* Missing )

* Apply suggestions from code review

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-03-31 10:00:27 -04:00
Sylvain Gugger a735f727cc
Fix test_trainer_distributed (#10875) 2021-03-23 19:03:06 -04:00
Sylvain Gugger 008672e6e5
Fix distributed evaluation (#10795) 
* Fix distributed evaluation

* Use logger
 2021-03-18 13:12:04 -04:00
Sylvain Gugger 00aa9dbca2
Copyright (#8970) 
* Add copyright everywhere missing

* Style
 2020-12-07 18:36:34 -05:00
Stas Bekman 02bdfc0251
using multi_gpu consistently (#8446) 
* s|multiple_gpu|multi_gpu|g; s|multigpu|multi_gpu|g'

* doc
 2020-11-10 13:23:58 -05:00
Stas Bekman 5423f2a9d4
[testing] port test_trainer_distributed to distributed pytest + TestCasePlus enhancements (#8107) 
* move the helper code into testing_utils

* port test_trainer_distributed to work with pytest

* improve docs

* simplify notes

* doc

* doc

* style

* doc

* further improvements

* torch might not be available

* real fix

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2020-10-28 11:51:32 -04:00
Sylvain Gugger a1d1b332d0
Add predict step accumulation (#7767) 
* Add eval_accumulation_step and clean distributed eval

* Add TPU test

* Add TPU stuff

* Fix arg name

* Fix Seq2SeqTrainer

* Fix total_size

* Update src/transformers/trainer_pt_utils.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Doc and add test to TPU

* Add unit test

* Adapt name

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2020-10-14 11:41:45 -04:00
Sylvain Gugger 573bdb0a5d
Add tests to Trainer (#6605) 
* Add tests to Trainer

* Test if removing long breaks everything

* Remove ugly hack

* Fix distributed test

* Use float for number of epochs
 2020-08-20 11:13:50 -04:00
Thomas Wolf 27cf1d97f0
[Tokenization] Fix #5181 - make #5155 more explicit - move back the default logging level in tests to WARNING (#5252) 
* fix-5181

Padding to max sequence length while truncation to another length was wrong on slow tokenizers

* clean up and fix #5155

* fix XLM test

* Fix tests for Transfo-XL

* logging only above WARNING in tests

* switch slow tokenizers tests in @slow

* fix Marian truncation tokenization test

* style and quality

* make the test a lot faster by limiting the sequence length used in tests
 2020-06-25 17:24:28 +02:00
Sylvain Gugger 1affde2f10
Make DataCollator a callable (#5015) 
* Make DataCollator a callable

* Update src/transformers/data/data_collator.py

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-06-15 11:58:33 -04:00
Julien Chaumond a3af8e86cb Update test_trainer_distributed.py 2020-05-20 18:26:51 -04:00
Julien Chaumond 5e7fe8b585
Distributed eval: SequentialDistributedSampler + gather all results (#4243) 
* Distributed eval: SequentialDistributedSampler + gather all results

* For consistency only write to disk from world_master

Close https://github.com/huggingface/transformers/issues/4272

* Working distributed eval

* Hook into scripts

* Fix #3721 again

* TPU.mesh_reduce: stay in tensor space

Thanks @jysohn23

* Just a small comment

* whitespace

* torch.hub: pip install packaging

* Add test scenarii
 2020-05-18 22:02:39 -04:00