dnrops
/
transformers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,902 Commits 481 Branches 161 Tags 1.2 GiB
Commit Graph

6902 Commits

Author SHA1 Message Date
Stas Bekman 3c27d246e5
[vulnerability] fix dependency (#10914) 
this PR fixes https://github.com/huggingface/transformers/security/dependabot/examples/research_projects/lxmert/requirements.txt/PyYAML/open
 2021-03-26 09:06:11 -04:00
Tomy Hsieh 4b2b50aa7b
Rename NLP library to Datasets library (#10920) 
* Rename NLP library to Datasets library

* Update github template

* Fix styling
 2021-03-26 08:07:59 -04:00
lexhuismans 86c6f8a8b1
Fix comment (#10886) 2021-03-25 21:23:56 +03:00
Sylvain Gugger 9856c9213d Reorder init imports 2021-03-25 12:51:43 -04:00
Sylvain Gugger e70068a719 Fix typo 2021-03-25 12:40:25 -04:00
Sylvain Gugger f183a7a3c3 Sort init imports 2021-03-25 12:38:54 -04:00
Amir Tahmasbi 4684bfc757
Layout lm tf 2 (#10636) 
* Added embeddings layer

* Added layoutlm layers, main model, maskedlm and token classification classes

* Added model classes to tf auto models

* Added model to PT to TF conversion script

* Added model to doc README

* Added tests

* Removed unused imports

* Added layoutlm model, test, and doc for sequence classification, and fix imports in __init__.py

* Made tests pass!

* Fixed typos in imports and docs

* Fixed a typo in embeddings layer

* Removed imports

* Fixed formatting issues, imports, tests

* Added layoutlm layers, main model, maskedlm and token classification classes

* Added model classes to tf auto models

* Added model to PT to TF conversion script

* Removed unused imports

* Added layoutlm model, test, and doc for sequence classification, and fix imports in __init__.py

* Made tests pass!

* Fixed typos in imports and docs

* Removed imports

* Fixed small formatting issues

* Removed duplicates import from main __init__.py

* Chnaged deafult arg to true for adding  pooling layer to tf layoutlm

* Fixed formatting issues

* Style

* Added copied from to classes copied from bert

* Fixed doc strings examples to work with layoutlm inputs

* Removed PyTorch reference in doc strings example

* Added integration tests

* Cleaned up initialization file

* Updated model checkpoint identifiers

* Fixed imports

Co-authored-by: Amir Tahmasbi <amir@ehsai.ca>
Co-authored-by: Lysandre <lysandre.debut@reseau.eseo.fr>
 2021-03-25 12:32:38 -04:00
Philipp Schmid 1a3e0c4fe6
make local setup more clearer and added missing links (#10899) 2021-03-25 09:01:31 -04:00
Jethro Kuan 5f1491d3b3
run_glue_no_trainer: datasets -> raw_datasets (#10898) 
Use the correct variable (raw_datasets) instead of the module (datasets)
where appropriate.
 2021-03-25 08:28:17 -04:00
Sidd Karamcheti 1c06240e1b
Update training args ignore_skip_data -> ignore_data_skip (#10891) 2021-03-24 16:44:51 -04:00
Sylvain Gugger 3b20e910b4
Remove version warning in pretrained BART models (#10890) 
* Remove version warning in pretrained BART models

* Put it at the base model
 2021-03-24 15:21:40 -04:00
Lysandre Debut 3c12e3c1c4
Fix overflowing bad word ids (#10889) 
* Removes overflowing bad word IDs

* Raise warning
 2021-03-24 15:13:56 -04:00
Eliza Szczechla 1f5ea9e04a
Add notebook on fine-tuning Bart (#10883) 
Co-authored-by: Eliza <eliza@habanero.tiger.com.pl>
 2021-03-24 11:03:37 -04:00
imzhengzx f81077fcf3
error type of tokenizer in __init__ definition (#10879) 
the orignal code in line 246 is
```
tokenizer: Optional["PreTrainedTokenizerBase"] = None,
```

it should be
```
tokenizer: Optional[PreTrainedTokenizerBase] = None,
```
 2021-03-24 11:00:14 -04:00
Sylvain Gugger 1aed2b908e
Add new notebook links in the docs (#10876) 2021-03-24 09:45:08 -04:00
Sylvain Gugger a735f727cc
Fix test_trainer_distributed (#10875) 2021-03-23 19:03:06 -04:00
Philipp Schmid 8c297cdb30
Sm trainer smp init fix (#10870) 
* rewrote is_sagemaker_model_parallel_available

* added is_sagemaker_model_parallel_available to SageMakerTrainer

* removed unnecessary mp_parameters as TrainingArguments

* make style happy

* added mp_parameters again to parse mp-specific args.
 2021-03-23 20:07:55 +01:00
RafaelWO d4d4447d53
fixed prefix_allowed_tokens_fn docstring in generate() (#10862) 2021-03-23 13:48:22 -04:00
Bhadresh Savani 7ef40120a0
[Examples] Added predict stage and Updated Example Template (#10868) 
* added predict stage

* added test keyword in exception message

* removed example specific saving predictions

* fixed f-string error

* removed extra line

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
 2021-03-23 10:37:59 -07:00
Stas Bekman fb2b89840b
[file_utils] import refactor (#10859) 
* import refactor

* fix the fallback
 2021-03-23 09:41:41 -07:00
Lysandre 3f48b2bc3e Update stable docs 2021-03-23 11:01:16 -04:00
Philipp Schmid 77ffd5edd5
Amazon SageMaker Documentation (#10867) 
* added finished documentation

* changed version from 1.6 to 1.6.0 for distributed

* updated versions

* updated urls
 2021-03-23 10:56:44 -04:00
Sylvain Gugger bf1f43fbd7
Update the example template for a no Trainer option (#10865) 2021-03-23 10:02:39 -04:00
Marta Maślankowska 2eb596f085
Fix p_mask cls token masking in qa pipeline (#10863) 2021-03-23 09:08:39 -04:00
Bhadresh Savani eb330e8904
fixed typo (#10861) 2021-03-23 08:15:28 -04:00
Stas Bekman e21f89f64c
fix nan in full-fp16 label_smoothing eval (#10815) 2021-03-22 19:23:24 -07:00
Sylvain Gugger b5b957a65c
Make convert_to_onnx runable as script again (#10857) 2021-03-22 22:16:39 -04:00
Patrick von Platen 77bf3fe787
[Generate] Add save mode logits processor to remove nans and infs if necessary (#10769) 
* push

* finish

* finish

* make fix copies

* change name
 2021-03-23 01:00:05 +03:00
Eliza Szczechla 9f8fa4e973
Use DataCollatorForSeq2Seq in run_summarization in all cases (#10856) 
Co-authored-by: Eliza <eliza@habanero.tiger.com.pl>
 2021-03-22 15:05:39 -04:00
Ruan Chaves a8d4d6776d
Modify the Trainer class to handle simultaneous execution of Ray Tune and Weights & Biases (#10823) 
* Modify the _hp_search_setup method on the Trainer class to handle the wandb argument passed by Ray Tune to model config.

* Reformat single quotes as double quotes.
 2021-03-22 14:04:51 -04:00
Boris Dayma 125ccead71
feat(wandb): logging and configuration improvements (#10826) 
* feat: ensure unique artifact id

* feat: allow manual init

* fix: simplify reinit logic

* fix: no dropped value + immediate commits

* fix: wandb use in sagemaker

* docs: improve documenation and formatting

* fix: typos

* docs: improve formatting
 2021-03-22 10:45:17 -04:00
Sidd Karamcheti b230181d41
Add simple one character fix so that on_step_begin and on_step_end are called at the right times (#10839) 2021-03-22 09:15:39 -04:00
Stas Bekman 24ab5b08a3
[makefile] autogenerate target (#10814) 
* autogenerate target

* clarify comment
 2021-03-22 09:14:22 -04:00
Sebastian Olsson 2c6684239f
Correct AutoConfig call docstrings (#10822) 2021-03-22 09:12:44 -04:00
Stas Bekman 8fb4671811
[vulnerability] in example deps fix (#10817) 
Takes care of:
https://github.com/huggingface/transformers/security/dependabot/examples/research_projects/lxmert/requirements.txt/jinja2/open

@LysandreJik

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-03-22 09:05:24 -04:00
dependabot[bot] dbfe379514
Bump jinja2 from 2.11.2 to 2.11.3 in /examples/research_projects/lxmert (#10818) 
Bumps [jinja2](https://github.com/pallets/jinja) from 2.11.2 to 2.11.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/master/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/2.11.2...2.11.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2021-03-22 08:54:50 -04:00
Qiushi Pan 29904a967b
Update FINE_TUNE_XLSR_WAV2VEC2.md (#10849) 
Fix typo.
 2021-03-22 07:58:59 -04:00
Patrick von Platen 0f226f78ce
push (#10846) 2021-03-22 10:32:21 +03:00
Suraj Patil 82b8d8c7b0
Update FINE_TUNE_XLSR_WAV2VEC2.md 2021-03-21 22:47:09 +05:30
Patrick von Platen af6125ffdb
Update FINE_TUNE_XLSR_WAV2VEC2.md 2021-03-21 12:31:33 +03:00
Patrick von Platen 5aaf6e1460
small improvements for wav2vec2 info script (#10829) 2021-03-21 11:41:44 +03:00
Eric Lam be87b84276
Add new community notebook - wav2vec2 with GPT (#10794) 
* Add new community notebook - wav2vec2 with GPT

* Update:community.md, new nb add
* feat: notebook of wav2vec xlsr ctc decoding with gpt logit adjustment
* Update: Wav2vec2 CTC decoding with gpt2 adjustment

* Update docs/source/community.md

Co-authored-by: Suraj Patil <surajp815@gmail.com>
 2021-03-21 13:29:53 +05:30
Suraj Patil 68b55885ed
add doc for Local machine (#10828) 2021-03-21 13:25:34 +05:30
Sylvain Gugger 21e86f99e6
Sort init import (#10801) 
* Initial script

* Add script to properly sort imports in init.

* Add to the CI

* Update utils/custom_init_isort.py

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* Separate scripts that change content from quality

* Move class_mapping_update to style_checks

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2021-03-19 16:17:13 -04:00
Julien Chaumond 1438c487df
wav2vec doc tweaks (#10808) 
* wording/typos tweaks

* Make model upload instructions simpler
 2021-03-19 12:48:54 -04:00
Patrick von Platen b9570a813c
Update FINE_TUNE_XLSR_WAV2VEC2.md 2021-03-19 19:45:28 +03:00
Philipp Schmid f2b744f690
Add transformers id to hub requests (#10811) 
* add uuid.hext to user_agent

* add log

* changed order of it

* renamed as session id

* renamed variable

* reverted naming of the const
 2021-03-19 16:26:32 +01:00
Sylvain Gugger 946400fb68
Expand a bit the presentation of examples (#10799) 
* Expand a bit the presentation of examples

* Apply suggestions from code review

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>

* Address review comments

Co-authored-by: Stas Bekman <stas00@users.noreply.github.com>
 2021-03-19 10:06:08 -04:00
Bhadresh Savani fd1d9f1ab8
[Example] Updating Question Answering examples for Predict Stage (#10792) 
* added prediction stage and eval fix

* style correction

* removed extra lines
 2021-03-19 09:42:17 -04:00
Patrick von Platen e8968bd03a
[XLSR-Wav2Vec2 Info doc] Add a couple of lines (#10806) 
* finish

* fix

* fix

* fix

* fix
 2021-03-19 12:52:54 +03:00