bf7f79cd57
Optional layers ( #8961 )
...
* Apply on BERT and ALBERT
* Update TF Bart
* Add input processing to TF BART
* Add input processing for TF CTRL
* Add input processing to TF Distilbert
* Add input processing to TF DPR
* Add input processing to TF Electra
* Add deprecated arguments
* Add input processing to TF XLM
* remove unused imports
* Add input processing to TF Funnel
* Add input processing to TF GPT2
* Add input processing to TF Longformer
* Add input processing to TF Lxmert
* Apply style
* Add input processing to TF Mobilebert
* Add input processing to TF GPT
* Add input processing to TF Roberta
* Add input processing to TF T5
* Add input processing to TF TransfoXL
* Apply style
* Rebase on master
* Fix wrong model name
* Fix BART
* Apply style
* Put the deprecated warnings in the input processing function
* Remove the unused imports
* Raise an error when len(kwargs)>0
* test ModelOutput instead of TFBaseModelOutput
* Address Patrick's comments
* Address Patrick's comments
* Add boolean processing for the inputs
* Take into account the optional layers
* Add missing/unexpected weights in the other models
* Apply style
* rename parameters
* Apply style
* Remove useless
* Remove useless
* Remove useless
* Update num parameters
* Fix tests
* Address Patrick's comment
* Remove useless attribute
2020-12-08 09:14:09 -05:00
00aa9dbca2
Copyright ( #8970 )
...
* Add copyright everywhere missing
* Style
2020-12-07 18:36:34 -05:00
c89bdfbe72
Reorganize repo ( #8580 )
...
* Put models in subfolders
* Styling
* Fix imports in tests
* More fixes in test imports
* Sneaky hidden imports
* Fix imports in doc files
* More sneaky imports
* Finish fixing tests
* Fix examples
* Fix path for copies
* More fixes for examples
* Fix dummy files
* More fixes for example
* More model import fixes
* Is this why you're unhappy GitHub?
* Fix imports in conver command
2020-11-16 21:43:42 -05:00
a573777901
Update repo to isort v5 ( #6686 )
...
* Run new isort
* More changes
* Update CI, CONTRIBUTING and benchmarks
2020-08-24 11:03:01 -04:00
4dc65591b5
[Almost all TF models] TF clean up: add missing CLM / MLM loss; fix T5 naming and keras compile ( #5395 )
...
* add first version of clm tf
* make style
* add more tests for bert
* update tf clm loss
* fix tests
* correct tf ner script
* add mlm loss
* delete bogus file
* clean tf auto model + add tests
* finish adding clm loss everywhere
* fix training in distilbert
* fix flake8
* save intermediate
* fix tf t5 naming
* remove prints
* finish up
* up
* fix tf gpt2
* fix new test utils import
* fix flake8
* keep backward compatibility
* Update src/transformers/modeling_tf_albert.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/modeling_tf_auto.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/modeling_tf_electra.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/modeling_tf_roberta.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/modeling_tf_mobilebert.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/modeling_tf_auto.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/modeling_tf_bert.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* Update src/transformers/modeling_tf_distilbert.py
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
* apply sylvains suggestions
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2020-07-07 18:15:53 +02:00
13deb95a40
Move tests/utils.py -> transformers/testing_utils.py ( #5350 )
2020-07-01 10:31:17 -04:00
27cf1d97f0
[Tokenization] Fix #5181 - make #5155 more explicit - move back the default logging level in tests to WARNING ( #5252 )
...
* fix-5181
Padding to max sequence length while truncation to another length was wrong on slow tokenizers
* clean up and fix #5155
* fix XLM test
* Fix tests for Transfo-XL
* logging only above WARNING in tests
* switch slow tokenizers tests in @slow
* fix Marian truncation tokenization test
* style and quality
* make the test a lot faster by limiting the sequence length used in tests
2020-06-25 17:24:28 +02:00
d4c2cb402d
Kill model archive maps ( #4636 )
...
* Kill model archive maps
* Fixup
* Also kill model_archive_map for MaskedBertPreTrainedModel
* Unhook config_archive_map
* Tokenizers: align with model id changes
* make style && make quality
* Fix CI
2020-06-02 09:39:33 -04:00
0e31e06a75
Add AutoModelForPreTraining
2020-01-27 14:27:07 -05:00
cf8a70bf68
More AutoConfig tests
2020-01-11 03:43:57 +00:00
84c0aa1868
num_parameters helper
2020-01-10 17:40:02 +00:00
81d6841b4b
GPU text generation: mMoved the encoded_prompt to correct device
2020-01-06 15:11:12 +01:00
dd4df80f0b
Moved the encoded_prompts to correct device
2020-01-06 15:11:12 +01:00
c824d15aa1
Remove __future__ imports.
2019-12-22 17:47:54 +01:00
7e98e211f0
Remove unittest.main() in test modules.
...
This construct isn't used anymore these days.
Running python tests/test_foo.py puts the tests/ directory on
PYTHONPATH, which isn't representative of how we run tests.
Use python -m unittest tests/test_foo.py instead.
2019-12-22 14:42:03 +01:00
ced0a94204
Switch test files to the standard test_*.py scheme.
2019-12-22 14:15:13 +01:00
Julien Plu