dnrops
/
transformers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
transformers/tests
History
RafaelWO b99ad457f4
Added feature to move added tokens in vocabulary for Transformer-XL (#4953) 
* Fixed resize_token_embeddings for transfo_xl model

* Fixed resize_token_embeddings for transfo_xl.

Added custom methods to TransfoXLPreTrainedModel for resizing layers of
the AdaptiveEmbedding.

* Updated docstring

* Fixed resizinhg cutoffs; added check for new size of embedding layer.

* Added test for resize_token_embeddings

* Fixed code quality

* Fixed unchanged cutoffs in model.config

* Added feature to move added tokens in tokenizer.

* Fixed code quality

* Added feature to move added tokens in tokenizer.

* Fixed code quality

* Fixed docstring, renamed sym to 	oken.

Co-authored-by: Rafael Weingartner <rweingartner.its-b2015@fh-salzburg.ac.at>
 		2020-06-22 15:40:52 +02:00
..
fixtures BIG Reorganize examples (#4213) 2020-05-07 13:48:44 -04:00
__init__.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
test_activations.py Reformer (#3351) 2020-05-07 10:17:01 +02:00
test_benchmark.py Benchmarks (#4912) 2020-06-22 12:06:56 +02:00
test_benchmark_tf.py Benchmarks (#4912) 2020-06-22 12:06:56 +02:00
test_configuration_auto.py Map configs to models and tokenizers 2020-01-13 23:11:44 +00:00
test_configuration_common.py Pass kwargs to configuration (#3147) 2020-03-05 17:16:57 -05:00
test_doc_samples.py make style 2020-03-04 20:18:07 -05:00
test_hf_api.py [cli] {login, upload, s3} display more helpful error messages 2020-04-30 12:51:06 -04:00
test_hf_argparser.py [Benchmark] Memory benchmark utils (#4198) 2020-05-27 23:22:16 +02:00
test_model_card.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
test_modeling_albert.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_auto.py [AutoModel] Split AutoModelWithLMHead into clm, mlm, encoder-decoder (#4933) 2020-06-12 10:01:49 +02:00
test_modeling_bart.py AutoTokenizer supports mbart-large-en-ro (#5121) 2020-06-18 20:47:37 -04:00
test_modeling_bert.py [AutoModel] Split AutoModelWithLMHead into clm, mlm, encoder-decoder (#4933) 2020-06-12 10:01:49 +02:00
test_modeling_camembert.py Fix slow gpu tests lysandre (#4487) 2020-05-20 11:59:45 -04:00
test_modeling_common.py [Bart] Question Answering Model is added to tests (#5024) 2020-06-15 22:50:09 +02:00
test_modeling_ctrl.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_distilbert.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_electra.py ElectraForMultipleChoice (#4954) 2020-06-18 14:59:35 -04:00
test_modeling_encoder_decoder.py Split LMBert model in two (#4874) 2020-06-10 18:26:42 -04:00
test_modeling_flaubert.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_gpt2.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_longformer.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_marian.py [marian tests ] pass device to pipeline (#4815) 2020-06-06 00:52:17 -04:00
test_modeling_mobilebert.py Add MobileBert (#4901) 2020-06-19 16:38:36 -04:00
test_modeling_openai.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_reformer.py [Reformer] Improved memory if input is shorter than chunk length (#4720) 2020-06-02 23:08:39 +02:00
test_modeling_roberta.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_t5.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_tf_albert.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_tf_auto.py Kill model archive maps (#4636) 2020-06-02 09:39:33 -04:00
test_modeling_tf_bert.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_tf_camembert.py Add camembert integration tests (#3375) 2020-03-24 10:18:37 +01:00
test_modeling_tf_common.py tf add resize_token_embeddings method (#4351) 2020-06-18 18:41:26 -04:00
test_modeling_tf_ctrl.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_tf_distilbert.py tf add resize_token_embeddings method (#4351) 2020-06-18 18:41:26 -04:00
test_modeling_tf_electra.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_tf_flaubert.py Tensorflow improvements (#4530) 2020-06-04 19:45:53 -04:00
test_modeling_tf_gpt2.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_tf_mobilebert.py Add MobileBert (#4901) 2020-06-19 16:38:36 -04:00
test_modeling_tf_openai_gpt.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_tf_roberta.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_tf_t5.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_tf_transfo_xl.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_tf_xlm.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_tf_xlm_roberta.py Tensorflow improvements (#4530) 2020-06-04 19:45:53 -04:00
test_modeling_tf_xlnet.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_transfo_xl.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_xlm.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_modeling_xlm_roberta.py [ci] Re-run integration ground truth from fairseq 2020-03-03 15:31:40 -05:00
test_modeling_xlnet.py [cleanup] Hoist ModelTester objects to top level (#4939) 2020-06-16 08:03:43 -04:00
test_onnx.py Fix onnx export input names order (#4641) 2020-06-01 16:12:48 +02:00
test_optimization.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
test_optimization_tf.py Tensorflow improvements (#4530) 2020-06-04 19:45:53 -04:00
test_pipelines.py [test_pipelines] Mark tests > 10s @slow, small speedups (#4421) 2020-05-18 12:23:21 -04:00
test_tokenization_albert.py [HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests (#4510) 2020-06-15 17:12:51 -04:00
test_tokenization_auto.py Test correct tokenizers after default switch (#3003) 2020-02-24 18:45:53 -05:00
test_tokenization_bert.py [HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests (#4510) 2020-06-15 17:12:51 -04:00
test_tokenization_bert_japanese.py [HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests (#4510) 2020-06-15 17:12:51 -04:00
test_tokenization_common.py [HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests (#4510) 2020-06-15 17:12:51 -04:00
test_tokenization_ctrl.py [HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests (#4510) 2020-06-15 17:12:51 -04:00
test_tokenization_distilbert.py [cleanup] test_tokenization_common.py (#4390) 2020-05-19 10:46:55 -04:00
test_tokenization_fast.py [HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests (#4510) 2020-06-15 17:12:51 -04:00
test_tokenization_gpt2.py [HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests (#4510) 2020-06-15 17:12:51 -04:00
test_tokenization_marian.py Fix marian tokenizer save pretrained (#5043) 2020-06-16 09:48:19 -04:00
test_tokenization_openai.py [HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests (#4510) 2020-06-15 17:12:51 -04:00
test_tokenization_roberta.py [HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests (#4510) 2020-06-15 17:12:51 -04:00
test_tokenization_t5.py [cleanup] test_tokenization_common.py (#4390) 2020-05-19 10:46:55 -04:00
test_tokenization_transfo_xl.py Added feature to move added tokens in vocabulary for Transformer-XL (#4953) 2020-06-22 15:40:52 +02:00
test_tokenization_utils.py Ability to pickle/unpickle BatchEncoding pickle (reimport) (#5039) 2020-06-16 09:25:25 +02:00
test_tokenization_xlm.py [HUGE] Refactoring tokenizers backend - padding - truncation - pre-tokenized pipeline - fast tokenizers - tests (#4510) 2020-06-15 17:12:51 -04:00
test_tokenization_xlm_roberta.py [cleanup] test_tokenization_common.py (#4390) 2020-05-19 10:46:55 -04:00
test_tokenization_xlnet.py [cleanup] test_tokenization_common.py (#4390) 2020-05-19 10:46:55 -04:00
test_trainer.py Fix #5114 (#5122) 2020-06-18 19:20:04 -04:00
test_trainer_distributed.py Make DataCollator a callable (#5015) 2020-06-15 11:58:33 -04:00
utils.py Fix nn.DataParallel compatibility in PyTorch 1.5 (#4300) 2020-05-18 20:34:50 -04:00