dnrops
/
transformers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
transformers/setup.cfg

54 lines
874 B
INI
Raw Normal View History

Add black-compatible isort configuration. lines_after_imports = 2 is a matter of taste; I like it.
2019-12-21 22:56:44 +08:00
[isort]
Update repo to isort v5 (#6686) * Run new isort * More changes * Update CI, CONTRIBUTING and benchmarks
2020-08-24 23:03:01 +08:00
default_section = FIRSTPARTY
Add black-compatible isort configuration. lines_after_imports = 2 is a matter of taste; I like it.
2019-12-21 22:56:44 +08:00
ensure_newline_before_comments = True
force_grid_wrap = 0
include_trailing_comma = True
known_first_party = transformers
Sort imports for optional third-party libraries. These libraries aren't always installed in the virtual environment where isort is running. Declaring them properly avoids mixing these third-party imports with local imports.
2019-12-22 18:17:48 +08:00
known_third_party =
[isort] declare more third-parties in case no tf install
2020-01-11 04:35:45 +08:00
absl
Add POS tagging and Phrase chunking token classification examples (#6457) * Add more token classification examples * POS tagging example * Phrase chunking example * PR review fixes * Add conllu to third party list (used in token classification examples)
2020-08-14 00:09:51 +08:00
conllu
Fix CI with change of name of nlp (#7054) * nlp -> datasets * More nlp -> datasets * Woopsie * More nlp -> datasets * One last
2020-09-11 02:51:08 +08:00
datasets
Benchmarks (#4912) * finish benchmark * fix isort * fix setup cfg * retab * fix time measuring of tf graph mode * fix tf cuda * clean code * better error message
2020-06-22 18:06:56 +08:00
elasticsearch
Sort imports for optional third-party libraries. These libraries aren't always installed in the virtual environment where isort is running. Declaring them properly avoids mixing these third-party imports with local imports.
2019-12-22 18:17:48 +08:00
fairseq
RAG (#6813) * added rag WIP * path fix * Formatting / renaming prior to actual work * added rag WIP * path fix * Formatting / renaming prior to actual work * added rag WIP * path fix * Formatting / renaming prior to actual work * added rag WIP * Formatting / renaming prior to actual work * First commit * improve comments * Retrieval evaluation scripts * refactor to include modeling outputs + MPI retriever * Fix rag-token model + refactor * Various fixes + finetuning logic * use_bos fix * Retrieval refactor * Finetuning refactoring and cleanup * Add documentation and cleanup * Remove set_up_rag_env.sh file * Fix retrieval wit HF index * Fix import errors * Fix quality errors * Refactor as per suggestions in https://github.com/huggingface/transformers/pull/6813#issuecomment-687208867 * fix quality * Fix RAG Sequence generation * minor cleanup plus initial tests * fix test * fix tests 2 * Comments fix * post-merge fixes * Improve readme + post-rebase refactor * Extra dependencied for tests * Fix tests * Fix tests 2 * Refactor test requirements * Fix tests 3 * Post-rebase refactor * rename nlp->datasets * RAG integration tests * add tokenizer to slow integration test and allow retriever to run on cpu * add tests; fix position ids warning * change structure * change structure * add from encoder generator * save working solution * make all integration tests pass * add RagTokenizer.save/from_pretrained and RagRetriever.save/from_pretrained * don't save paths * delete unnecessary imports * pass config to AutoTokenizer.from_pretrained for Rag tokenizers * init wiki_dpr only once * hardcode legacy index and passages paths (todo: add the right urls) * finalize config * finalize retriver api and config api * LegacyIndex index download refactor * add dpr to autotokenizer * make from pretrained more flexible * fix ragfortokengeneration * small name changes in tokenizer * add labels to models * change default index name * add retrieval tests * finish token generate * align test with previous version and make all tests pass * add tests * finalize tests * implement thoms suggestions * add first version of test * make first tests work * make retriever platform agnostic * naming * style * add legacy index URL * docstrings + simple retrieval test for distributed * clean model api * add doc_ids to retriever's outputs * fix retrieval tests * finish model outputs * finalize model api * fix generate problem for rag * fix generate for other modles * fix some tests * save intermediate * set generate to default * big refactor generate * delete rag_api * correct pip faiss install * fix auto tokenization test * fix faiss install * fix test * move the distributed logic to examples * model page * docs * finish tests * fix dependencies * fix import in __init__ * Refactor eval_rag and finetune scripts * start docstring * add psutil to test * fix tf test * move require torch to top * fix retrieval test * align naming * finish automodel * fix repo consistency * test ragtokenizer save/load * add rag model output docs * fix ragtokenizer save/load from pretrained * fix tokenizer dir * remove torch in retrieval * fix docs * fixe finetune scripts * finish model docs * finish docs * remove auto model for now * add require torch * remove solved todos * integrate sylvains suggestions * sams comments * correct mistake on purpose * improve README * Add generation test cases * fix rag token * clean token generate * fix test * add note to test * fix attention mask * add t5 test for rag * Fix handling prefix in finetune.py * don't overwrite index_name Co-authored-by: Patrick Lewis <plewis@fb.com> Co-authored-by: Aleksandra Piktus <piktus@devfair0141.h2.fair> Co-authored-by: Aleksandra Piktus <piktus@learnfair5102.h2.fair> Co-authored-by: Aleksandra Piktus <piktus@learnfair5067.h2.fair> Co-authored-by: Your Name <you@example.com> Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> Co-authored-by: Quentin Lhoest <lhoest.q@gmail.com>
2020-09-23 00:29:58 +08:00
faiss-cpu
Sort imports for optional third-party libraries. These libraries aren't always installed in the virtual environment where isort is running. Declaring them properly avoids mixing these third-party imports with local imports.
2019-12-22 18:17:48 +08:00
fastprogress
Add fire to setup.cfg to make isort happy (#6066)
2020-07-28 03:17:33 +08:00
fire
Replace mecab-python3 with fugashi for Japanese tokenization (#6086) * Replace mecab-python3 with fugashi This replaces mecab-python3 with fugashi for Japanese tokenization. I am the maintainer of both projects. Both projects are MeCab wrappers, so the underlying C++ code is the same. fugashi is the newer wrapper and doesn't use SWIG, so for basic use of the MeCab API it's easier to use. This code insures the use of a version of ipadic installed via pip, which should make versioning and tracking down issues easier. fugashi has wheels for Windows, OSX, and Linux, which will help with issues with installing old versions of mecab-python3 on Windows. Compared to mecab-python3, because fugashi doesn't use SWIG, it doesn't require a C++ runtime to be installed on Windows. In adding this change I removed some code dealing with `cursor`, `token_start`, and `token_end` variables. These variables didn't seem to be used for anything, it is unclear to me why they were there. I ran the tests and they passed, though I couldn't figure out how to run the slow tests (`--runslow` gave an error) and didn't try testing with Tensorflow. * Style fix * Remove unused variable Forgot to delete this... * Adapt doc with install instructions * Fix typo Co-authored-by: sgugger <sylvain.gugger@gmail.com> Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2020-07-31 16:41:14 +08:00
fugashi
Sort imports for optional third-party libraries. These libraries aren't always installed in the virtual environment where isort is running. Declaring them properly avoids mixing these third-party imports with local imports.
2019-12-22 18:17:48 +08:00
git
[isort] declare more third-parties in case no tf install
2020-01-11 04:35:45 +08:00
h5py
[isort] add matplotlib to known 3rd party dependencies (#4800)
2020-06-06 05:27:31 +08:00
matplotlib
Sort imports for optional third-party libraries. These libraries aren't always installed in the virtual environment where isort is running. Declaring them properly avoids mixing these third-party imports with local imports.
2019-12-22 18:17:48 +08:00
nltk
[isort] declare more third-parties in case no tf install
2020-01-11 04:35:45 +08:00
numpy
Sort imports for optional third-party libraries. These libraries aren't always installed in the virtual environment where isort is running. Declaring them properly avoids mixing these third-party imports with local imports.
2019-12-22 18:17:48 +08:00
packaging
add pandas to setup.cfg (#5093)
2020-06-18 04:39:17 +08:00
pandas
Sort imports for optional third-party libraries. These libraries aren't always installed in the virtual environment where isort is running. Declaring them properly avoids mixing these third-party imports with local imports.
2019-12-22 18:17:48 +08:00
PIL
psutil
examples/seq2seq supports translation (#5202)
2020-06-25 11:58:11 +08:00
pytest
Support for torch-lightning in NER examples (#2890) * initial pytorch lightning commit * tested multigpu * Fix learning rate schedule * black formatting * fix flake8 * isort * isort * . Co-authored-by: Check your git settings! <chris@chris-laptop>
2020-02-21 00:50:05 +08:00
pytorch_lightning
[isort] add known 3rd party to setup.cfg (#4053) * add known 3rd party to setup.cfg * comment * Update CONTRIBUTING.md Co-authored-by: Julien Chaumond <chaumond@gmail.com>
2020-04-29 05:12:00 +08:00
rouge_score
sacrebleu
Sort imports for optional third-party libraries. These libraries aren't always installed in the virtual environment where isort is running. Declaring them properly avoids mixing these third-party imports with local imports.
2019-12-22 18:17:48 +08:00
seqeval
sklearn
Add mbart-large-cc25, support translation finetuning (#5129) improve unittests for finetuning, especially w.r.t testing frozen parameters fix freeze_embeds for T5 add streamlit setup.cfg
2020-07-08 01:23:01 +08:00
streamlit
Sort imports for optional third-party libraries. These libraries aren't always installed in the virtual environment where isort is running. Declaring them properly avoids mixing these third-party imports with local imports.
2019-12-22 18:17:48 +08:00
tensorboardX
keep list sorted
2020-01-11 04:36:46 +08:00
tensorflow
Sort imports for optional third-party libraries. These libraries aren't always installed in the virtual environment where isort is running. Declaring them properly avoids mixing these third-party imports with local imports.
2019-12-22 18:17:48 +08:00
tensorflow_datasets
[testing] add timeout_decorator (#3543)
2020-05-01 21:05:47 +08:00
timeout_decorator
Py35 doesn't like inline variable types
2020-01-14 04:44:33 +08:00
torch
Speech2TextTransformer (#10175) * s2t * fix config * conversion script * fix import * add tokenizer * fix tok init * fix tokenizer * first version working * fix embeds * fix lm head * remove extra heads * fix convert script * handle encoder attn mask * style * better enc attn mask * override _prepare_attention_mask_for_generation * handle attn_maks in encoder and decoder * input_ids => input_features * enable use_cache * remove old code * expand embeddings if needed * remove logits bias * masked_lm_loss => loss * hack tokenizer to support feature processing * fix model_input_names * style * fix error message * doc * remove inputs_embeds * remove input_embeds * remove unnecessary docstring * quality * SpeechToText => Speech2Text * style * remove shared_embeds * subsample => conv * remove Speech2TextTransformerDecoderWrapper * update output_lengths formula * fix table * remove max_position_embeddings * update conversion scripts * add possibility to do upper case for now * add FeatureExtractor and Processor * add tests for extractor * require_torch_audio => require_torchaudio * add processor test * update import * remove classification head * attention mask is now 1D * update docstrings * attention mask should be of type long * handle attention mask from generate * alwyas return attention_mask * fix test * style * doc * Speech2TextTransformer => Speech2Text * Speech2TextTransformerConfig => Speech2TextConfig * remove dummy_inputs * nit * style * multilinguial tok * fix tokenizer * add tgt_lang setter * save lang_codes * fix tokenizer * add forced_bos_token_id to tokenizer * apply review suggestions * add torchaudio to extra deps * add speech deps to CI * fix dep * add libsndfile to ci * libsndfile1 * add speech to extras all * libsndfile1 -> libsndfile1 * libsndfile * libsndfile1-dev * apt update * add sudo to install * update deps table * install libsndfile1-dev on CI * tuple to list * init conv layer * add model tests * quality * add integration tests * skip_special_tokens * add speech_to_text_transformer in toctree * fix tokenizer * fix fp16 tests * add tokenizer tests * fix copyright * input_values => input_features * doc * add model in readme * doc * change checkpoint names * fix copyright * fix code example * add max_model_input_sizes in tokenizer * fix integration tests * add do_lower_case to tokenizer * remove clamp trick * fix "Add modeling imports here" * fix copyrights * fix tests * SpeechToTextTransformer => SpeechToText * fix naming * fix table formatting * fix typo * style * fix typos * remove speech dep from extras[testing] * fix copies * rename doc file, * put imports under is_torch_available * run feat extract tests when torch is available * dummy objects for processor and extractor * fix imports in tests * fix import in modeling test * fxi imports * fix torch import * fix imports again * fix positional embeddings * fix typo in import * adapt new extractor refactor * style * fix torchscript test * doc * doc * Apply suggestions from code review Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com> Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> * fix docs, copied from, style * fix docstring * handle imports * remove speech from all extra deps * remove s2t from seq2seq lm mapping * better names * skip training tests * add install instructions * List => Tuple * doc * fix conversion script * fix urls * add instruction for libsndfile * fix fp16 test Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com> Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
2021-03-11 00:12:04 +08:00
torchaudio
Sort imports for optional third-party libraries. These libraries aren't always installed in the virtual environment where isort is running. Declaring them properly avoids mixing these third-party imports with local imports.
2019-12-22 18:17:48 +08:00
torchtext
torchvision
Support for torch-lightning in NER examples (#2890) * initial pytorch lightning commit * tested multigpu * Fix learning rate schedule * black formatting * fix flake8 * isort * isort * . Co-authored-by: Check your git settings! <chris@chris-laptop>
2020-02-21 00:50:05 +08:00
torch_xla
MarianMTModel.from_pretrained('Helsinki-NLP/opus-marian-en-de') (#3908) Co-Authored-By: Stefan Schweter <stefan@schweter.it>
2020-04-29 06:22:37 +08:00
tqdm
Sort imports for optional third-party libraries. These libraries aren't always installed in the virtual environment where isort is running. Declaring them properly avoids mixing these third-party imports with local imports.
2019-12-22 18:17:48 +08:00
Add black-compatible isort configuration. lines_after_imports = 2 is a matter of taste; I like it.
2019-12-21 22:56:44 +08:00
line_length = 119
lines_after_imports = 2
multi_line_output = 3
use_parentheses = True
Add black-compatible flake8 configuration.
2019-12-22 00:06:41 +08:00
[flake8]
Model utils doc (#6005) * Document TF modeling utils * Document all model utils
2020-07-24 21:16:28 +08:00
ignore = E203, E501, E741, W503, W605
Add black-compatible flake8 configuration.
2019-12-22 00:06:41 +08:00
max-line-length = 119
[Doctest] Setup, quicktour and task_summary (#13078) * Fix doctests for quicktour * Adapt causal LM exemple * Remove space * Fix until summarization * End of task summary * Style * With last changes in quicktour
2021-08-11 19:45:25 +08:00
[tool:pytest]
doctest_optionflags=NUMBER NORMALIZE_WHITESPACE ELLIPSIS