transformers/tests
Sylvain Gugger cd56f3fe7e
Merge trainers (#10975)
* Replace is_sagemaker_distributed_available

* Merge SageMakerTrainer into Trainer

* Test with shorter condition

* Put back deleted line

* Deprecate SageMakerTrainer and SageMakerTrainingArguments

* Apply suggestions from code review

Co-authored-by: Philipp Schmid <32632186+philschmid@users.noreply.github.com>

Co-authored-by: Philipp Schmid <32632186+philschmid@users.noreply.github.com>
2021-03-31 10:01:30 -04:00
..
fixtures Conversion from slow to fast for BPE spm vocabs contained an error. (#10120) 2021-02-13 08:24:53 -05:00
sagemaker Merge trainers (#10975) 2021-03-31 10:01:30 -04:00
__init__.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
conftest.py Flax testing should not run the full torch test suite (#10725) 2021-03-16 08:05:37 +03:00
test_activations.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_activations_tf.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_benchmark.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_benchmark_tf.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_cli.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_configuration_auto.py Reorganize repo (#8580) 2020-11-16 21:43:42 -05:00
test_configuration_common.py [PretrainedConfig] Fix save pretrained config for edge case (#7943) 2020-10-22 15:39:01 +02:00
test_data_collator.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_doc_samples.py Fix ignore list behavior in doctests (#8213) 2020-11-02 08:47:37 -05:00
test_feature_extraction_common.py [FeatureExtractorSavingUtils] Refactor PretrainedFeatureExtractor (#10594) 2021-03-09 12:16:59 +03:00
test_feature_extraction_speech_to_text.py Speech2TextTransformer (#10175) 2021-03-10 21:42:04 +05:30
test_feature_extraction_wav2vec2.py W2v2 test require torch (#10665) 2021-03-11 12:56:12 -05:00
test_file_utils.py Patches the full import failure and adds a test (#10750) 2021-03-16 15:37:52 -04:00
test_flax_auto.py [ci, flax] non-existing models are unlikely to pass tests (#10409) 2021-02-26 12:35:36 +03:00
test_generation_beam_search.py Add flags to return scores, hidden states and / or attention weights in GenerationMixin (#9150) 2021-01-06 17:11:42 +01:00
test_generation_logits_process.py [Generate] Add save mode logits processor to remove nans and infs if necessary (#10769) 2021-03-23 01:00:05 +03:00
test_generation_stopping_criteria.py Adding new parameter to `generate`: `max_time`. (#9846) 2021-03-12 10:11:50 +01:00
test_generation_utils.py [Generate] Add save mode logits processor to remove nans and infs if necessary (#10769) 2021-03-23 01:00:05 +03:00
test_hf_api.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_hf_argparser.py Adding required flags to non-default arguments in hf_argparser (#10688) 2021-03-15 09:27:55 -04:00
test_image_utils.py Add ImageFeatureExtractionMixin (#10905) 2021-03-26 11:23:56 -04:00
test_logging.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_model_card.py GPU text generation: mMoved the encoded_prompt to correct device 2020-01-06 15:11:12 +01:00
test_model_output.py Add tests and fix various bugs in ModelOutput (#7073) 2020-09-11 12:01:33 -04:00
test_modeling_albert.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_auto.py LayoutLM Config (#9539) 2021-01-12 10:03:50 -05:00
test_modeling_bart.py fix bart tests (#10060) 2021-02-08 13:25:09 +03:00
test_modeling_bert.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_bert_generation.py Implementing the test integration of BertGeneration (#9990) 2021-02-08 08:22:19 -05:00
test_modeling_big_bird.py fix big bird gpu test (#10967) 2021-03-30 17:03:48 +03:00
test_modeling_blenderbot.py Hotfixing tests (blenderbot decoderonly tests, also need to remove (#10003) 2021-02-04 11:41:34 -05:00
test_modeling_blenderbot_small.py BartForCausalLM analogs to `ProphetNetForCausalLM` (#9128) 2021-02-04 11:56:12 +03:00
test_modeling_bort.py ADD BORT (#9813) 2021-01-27 21:25:11 +03:00
test_modeling_camembert.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_common.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_modeling_convbert.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_ctrl.py Ctrl for sequence classification (#8812) 2020-12-01 09:49:27 +01:00
test_modeling_deberta.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_deberta_v2.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_distilbert.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_dpr.py Fix slow dpr test (#10059) 2021-02-08 04:43:25 -05:00
test_modeling_electra.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_encoder_decoder.py BartForCausalLM analogs to `ProphetNetForCausalLM` (#9128) 2021-02-04 11:56:12 +03:00
test_modeling_flaubert.py Integration test for FlauBert (#10022) 2021-02-08 04:36:50 -05:00
test_modeling_flax_bert.py [Flax] Add other BERT classes (#10977) 2021-03-31 09:45:58 +03:00
test_modeling_flax_common.py [Flax] Add other BERT classes (#10977) 2021-03-31 09:45:58 +03:00
test_modeling_flax_roberta.py [WIP][Flax] Add general conversion script (#10809) 2021-03-30 12:13:59 +03:00
test_modeling_fsmt.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_modeling_funnel.py Switch `return_dict` to `True` by default. (#8530) 2020-11-16 11:43:00 -05:00
test_modeling_gpt2.py GPT2DoubleHeadsModel made parallelizable (#10658) 2021-03-15 09:10:44 -04:00
test_modeling_gpt_neo.py GPT Neo few fixes (#10968) 2021-03-30 11:15:55 -04:00
test_modeling_ibert.py I-BERT model support (#10153) 2021-02-25 10:06:42 -05:00
test_modeling_layoutlm.py Fix slow tests v4.2.0 (#9561) 2021-01-13 09:55:48 -05:00
test_modeling_led.py Add head_mask and decoder_head_mask to PyTorch LED (#9856) 2021-02-02 11:06:52 -08:00
test_modeling_longformer.py Add head_mask and decoder_head_mask to PyTorch LED (#9856) 2021-02-02 11:06:52 -08:00
test_modeling_lxmert.py Remove redundant `test_head_masking = True` flags in test files (#9858) 2021-01-28 10:09:13 -05:00
test_modeling_m2m_100.py layerdrop 0 (#10604) 2021-03-09 17:35:07 +03:00
test_modeling_marian.py Conversion to tensors requires padding (#10661) 2021-03-11 12:58:15 -05:00
test_modeling_mbart.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_mobilebert.py Removed unused `encoder_hidden_states` and `encoder_attention_mask` (#8972) 2020-12-08 12:04:34 -05:00
test_modeling_mpnet.py Add MP Net 2 (#9004) 2020-12-09 10:32:43 -05:00
test_modeling_mt5.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_openai.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_pegasus.py BartForCausalLM analogs to `ProphetNetForCausalLM` (#9128) 2021-02-04 11:56:12 +03:00
test_modeling_prophetnet.py up (#10771) 2021-03-17 16:15:14 +03:00
test_modeling_rag.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_modeling_reformer.py Add flags to return scores, hidden states and / or attention weights in GenerationMixin (#9150) 2021-01-06 17:11:42 +01:00
test_modeling_roberta.py Add caching mechanism to BERT, RoBERTa (#9183) 2020-12-23 23:01:32 +05:30
test_modeling_speech_to_text.py Speech2TextTransformer (#10175) 2021-03-10 21:42:04 +05:30
test_modeling_squeezebert.py Fix integration slow tests (#10670) 2021-03-11 13:43:53 -05:00
test_modeling_t5.py Fix torch 1.8.0 segmentation fault (#10546) 2021-03-05 12:10:19 -05:00
test_modeling_tapas.py Remove tolerance + drop_rows_to_fit by default (#9507) 2021-01-11 08:02:41 -05:00
test_modeling_tf_albert.py Add AMP for Albert (#10141) 2021-02-15 17:18:33 +01:00
test_modeling_tf_auto.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_modeling_tf_bart.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_modeling_tf_bert.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
test_modeling_tf_blenderbot.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_modeling_tf_blenderbot_small.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_modeling_tf_bort.py ADD BORT (#9813) 2021-01-27 21:25:11 +03:00
test_modeling_tf_camembert.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_tf_common.py Tests run on Docker (#10681) 2021-03-15 17:28:01 -04:00
test_modeling_tf_convbert.py ConvBERT fix torch <> tf weights conversion (#10314) 2021-02-24 14:55:34 +03:00
test_modeling_tf_ctrl.py Make TF CTRL compliant with XLA and AMP (#10209) 2021-02-17 18:54:15 +01:00
test_modeling_tf_distilbert.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
test_modeling_tf_dpr.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
test_modeling_tf_electra.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
test_modeling_tf_flaubert.py Making TF XLM-like models XLA and AMP compliant (#10211) 2021-02-17 18:02:48 +01:00
test_modeling_tf_funnel.py Fix AMP (#10216) 2021-02-18 06:29:43 -05:00
test_modeling_tf_gpt2.py Making TF GPT2 compliant with XLA and AMP (#10230) 2021-02-18 09:36:01 +01:00
test_modeling_tf_layoutlm.py Layout lm tf 2 (#10636) 2021-03-25 12:32:38 -04:00
test_modeling_tf_led.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_modeling_tf_longformer.py Making TF Longformer-like models compliant with AMP (#10233) 2021-02-22 15:41:56 +01:00
test_modeling_tf_lxmert.py Making TF Lxmert model compliant with AMP (#10257) 2021-02-19 06:54:14 -05:00
test_modeling_tf_marian.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_modeling_tf_mbart.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_modeling_tf_mobilebert.py Making TF MobileBert model compliant with AMP (#10259) 2021-02-19 06:55:25 -05:00
test_modeling_tf_mpnet.py Making TF MPNet model compliant with XLA (#10260) 2021-02-19 06:56:41 -05:00
test_modeling_tf_mt5.py Adjust loss difference (#10669) 2021-03-12 09:09:46 +03:00
test_modeling_tf_openai.py Making TF OpenAI GPT model compliant with AMP and XLA (#10261) 2021-02-19 09:33:25 -05:00
test_modeling_tf_pegasus.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_modeling_tf_pytorch.py Optional layers (#8961) 2020-12-08 09:14:09 -05:00
test_modeling_tf_rag.py make rag tests smaller (#10679) 2021-03-15 10:07:12 +03:00
test_modeling_tf_roberta.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
test_modeling_tf_t5.py Fix XLA and AMP (#10262) 2021-02-19 06:57:16 -05:00
test_modeling_tf_transfo_xl.py Making TF TransfoXL model compliant with AMP (#10264) 2021-02-19 06:58:07 -05:00
test_modeling_tf_xlm.py Making TF XLM-like models XLA and AMP compliant (#10211) 2021-02-17 18:02:48 +01:00
test_modeling_tf_xlm_roberta.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_tf_xlnet.py Check TF ops for ONNX compliance (#10025) 2021-02-15 07:55:10 -05:00
test_modeling_transfo_xl.py Add flags to return scores, hidden states and / or attention weights in GenerationMixin (#9150) 2021-01-06 17:11:42 +01:00
test_modeling_wav2vec2.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_modeling_xlm.py Add flags to return scores, hidden states and / or attention weights in GenerationMixin (#9150) 2021-01-06 17:11:42 +01:00
test_modeling_xlm_prophetnet.py Ci test tf super slow (#8007) 2020-10-30 10:25:48 -04:00
test_modeling_xlm_roberta.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_modeling_xlnet.py Add flags to return scores, hidden states and / or attention weights in GenerationMixin (#9150) 2021-01-06 17:11:42 +01:00
test_offline.py offline mode for firewalled envs (part 2) (#10569) 2021-03-08 08:52:20 -08:00
test_onnx.py Onnx fix test (#10663) 2021-03-11 13:38:29 -05:00
test_optimization.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_optimization_tf.py Use stable functions (#9369) 2021-01-05 03:58:26 -05:00
test_pipelines_common.py [PretrainedFeatureExtractor] + Wav2Vec2FeatureExtractor, Wav2Vec2Processor, Wav2Vec2Tokenizer (#10324) 2021-02-25 17:42:46 +03:00
test_pipelines_conversational.py Fixing conversation test for torch 1.8 (#10545) 2021-03-05 09:24:14 -05:00
test_pipelines_feature_extraction.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_pipelines_fill_mask.py Adding `skip_special_tokens=True` to FillMaskPipeline (#9783) 2021-01-26 10:06:28 +01:00
test_pipelines_ner.py Fixing NER pipeline for list inputs. (#10184) 2021-02-15 06:22:45 -05:00
test_pipelines_question_answering.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_pipelines_sentiment_analysis.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_pipelines_summarization.py Fix torch 1.8.0 segmentation fault (#10546) 2021-03-05 12:10:19 -05:00
test_pipelines_table_question_answering.py Adding a test to prevent late failure in the Table question answering (#9808) 2021-01-27 04:10:53 -05:00
test_pipelines_text2text_generation.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_pipelines_text_generation.py Adding a new `return_full_text` parameter to TextGenerationPipeline. (#9852) 2021-01-29 10:27:32 +01:00
test_pipelines_translation.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_pipelines_zero_shot.py zero-shot pipeline multi_class -> multi_label (#10727) 2021-03-15 16:02:46 -06:00
test_processor_speech_to_text.py Speech2TextTransformer (#10175) 2021-03-10 21:42:04 +05:30
test_processor_wav2vec2.py [PretrainedFeatureExtractor] + Wav2Vec2FeatureExtractor, Wav2Vec2Processor, Wav2Vec2Tokenizer (#10324) 2021-02-25 17:42:46 +03:00
test_retrieval_rag.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_sequence_feature_extraction_common.py Speech2TextTransformer (#10175) 2021-03-10 21:42:04 +05:30
test_skip_decorators.py [testing] rename skip targets + docs (#7863) 2020-10-20 04:39:13 -04:00
test_tokenization_albert.py ALBERT Tokenizer integration test (#9943) 2021-02-02 04:39:33 -05:00
test_tokenization_auto.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_tokenization_bart.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_tokenization_barthez.py Speedup tf tests (#10601) 2021-03-08 21:44:07 -05:00
test_tokenization_bert.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_tokenization_bert_generation.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_bert_japanese.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_bertweet.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_tokenization_big_bird.py BigBird (#10183) 2021-03-30 08:51:34 +03:00
test_tokenization_blenderbot.py [PyTorch Bart] Split Bart into different models (#9343) 2021-01-05 22:00:05 +01:00
test_tokenization_camembert.py Conversion from slow to fast for BPE spm vocabs contained an error. (#10120) 2021-02-13 08:24:53 -05:00
test_tokenization_common.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_tokenization_ctrl.py Refactor `prepare_seq2seq_batch` (#9524) 2021-01-12 18:19:38 -05:00
test_tokenization_deberta.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_deberta_v2.py Integrate DeBERTa v2(the 1.5B model surpassed human performance on Su… (#10018) 2021-02-19 18:34:44 -05:00
test_tokenization_distilbert.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_dpr.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_fsmt.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_funnel.py Reorganize repo (#8580) 2020-11-16 21:43:42 -05:00
test_tokenization_gpt2.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_tokenization_herbert.py Reorganize repo (#8580) 2020-11-16 21:43:42 -05:00
test_tokenization_layoutlm.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_lxmert.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_m2m_100.py Add m2m100 (#10236) 2021-03-06 22:14:16 +05:30
test_tokenization_marian.py Flax testing should not run the full torch test suite (#10725) 2021-03-16 08:05:37 +03:00
test_tokenization_mbart.py Make Barthez tokenizer tests a bit faster (#10399) 2021-02-25 11:42:25 -05:00
test_tokenization_mbart50.py Make Barthez tokenizer tests a bit faster (#10399) 2021-02-25 11:42:25 -05:00
test_tokenization_mpnet.py [MPNet] Add slow to fast tokenizer converter (#9233) 2020-12-21 15:41:34 +01:00
test_tokenization_openai.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_tokenization_pegasus.py Deprecate prepare_seq2seq_batch (#10287) 2021-02-22 12:36:16 -05:00
test_tokenization_phobert.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_tokenization_prophetnet.py Deprecate prepare_seq2seq_batch (#10287) 2021-02-22 12:36:16 -05:00
test_tokenization_rag.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_reformer.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_tokenization_roberta.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_tokenization_small_blenderbot.py [PyTorch Bart] Split Bart into different models (#9343) 2021-01-05 22:00:05 +01:00
test_tokenization_speech_to_text.py Speech2TextTransformer (#10175) 2021-03-10 21:42:04 +05:30
test_tokenization_squeezebert.py [Dependencies|tokenizers] Make both SentencePiece and Tokenizers optional dependencies (#7659) 2020-10-18 20:51:24 +02:00
test_tokenization_t5.py Flax testing should not run the full torch test suite (#10725) 2021-03-16 08:05:37 +03:00
test_tokenization_tapas.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_tokenization_transfo_xl.py Refactor `prepare_seq2seq_batch` (#9524) 2021-01-12 18:19:38 -05:00
test_tokenization_utils.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_wav2vec2.py W2v2 test require torch (#10665) 2021-03-11 12:56:12 -05:00
test_tokenization_xlm.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_xlm_prophetnet.py Reorganize repo (#8580) 2020-11-16 21:43:42 -05:00
test_tokenization_xlm_roberta.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_tokenization_xlnet.py Copyright (#8970) 2020-12-07 18:36:34 -05:00
test_trainer.py make failure to find a resume checkpoint fatal + tests (#10777) 2021-03-17 11:16:37 -07:00
test_trainer_callback.py Fix tests of TrainerCallback (#10615) 2021-03-09 16:25:32 -05:00
test_trainer_distributed.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_trainer_seq2seq.py fix bert2bert test (#10063) 2021-02-08 16:04:28 +03:00
test_trainer_tpu.py Enforce string-formatting with f-strings (#10980) 2021-03-31 10:00:27 -04:00
test_trainer_utils.py Smmp batch not divisible by microbatches fix (#10778) 2021-03-17 19:18:11 -04:00
test_utils_check_copies.py Check copies blackify (#10775) 2021-03-17 18:11:20 -04:00
test_versions_utils.py Copyright (#8970) 2020-12-07 18:36:34 -05:00