dnrops
/
transformers
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,355 Commits 481 Branches 161 Tags 1.2 GiB
Commit Graph

423 Commits

Author SHA1 Message Date
Marc Sun f54d82cace
[CI] Quantization workflow (#29046) 
* [CI] Quantization workflow

* build dockerfile

* fix dockerfile

* update self-cheduled.yml

* test build dockerfile on push

* fix torch install

* udapte to python 3.10

* update aqlm version

* uncomment build dockerfile

* tests if the scheduler works

* fix docker

* do not trigger on psuh again

* add additional runs

* test again

* all good

* style

* Update .github/workflows/self-scheduled.yml

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>

* test build dockerfile with torch 2.2.0

* fix extra

* clean

* revert changes

* Revert "revert changes"

This reverts commit 4cb52b8822.

* revert correct change

---------

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
 2024-02-28 10:09:25 -05:00
Yih-Dar 93f8617afd
Use `DS_DISABLE_NINJA=1` (#29290) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-02-26 17:41:01 +08:00
Yih-Dar c8d98405a8
Use torch 2.2 for daily CI (model tests) (#29208) 
* Use torch 2.2 for daily CI (model tests)

* update

* update

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-02-23 21:37:08 +08:00
Yuki Watanabe ebf3ea2788
Fix a wrong link to CONTRIBUTING.md section in PR template (#28941) 2024-02-09 15:10:47 +00:00
Yih-Dar 4735866141
Split daily CI using 2 level matrix (#28773) 
* update / add new workflow files

* Add comment

* Use env.NUM_SLICES

* use scripts

* use scripts

* use scripts

* Fix

* using one script

* Fix

* remove unused file

* update

* fail-fast: false

* remove unused file

* fix

* fix

* use matrix

* inputs

* style

* update

* fix

* fix

* no model name

* add doc

* allow args

* style

* pass argument

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-01-31 18:04:43 +01:00
Yih-Dar 95346e9dcd
Add artifact name in job step to maintain job / artifact correspondence (#28682) 
* avoid using job name

* apply to other files

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-01-31 15:58:17 +01:00
Rishit Ratna 78d767e3c8
Fixed minor typos (#28489) 2024-01-15 16:45:15 +00:00
Yih-Dar 59cd9de39d
Byebye torch 1.10 (#28207) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-01-11 16:18:27 +01:00
Patrick von Platen cbbe30749b
[Whisper] Fix slow test (#28407) 
* [Whisper] Fix slow test

* update

* update

* update

* update

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2024-01-10 22:35:36 +01:00
Joao Gante ee2482b6f8
CI: limit natten version (#28432) 2024-01-10 12:39:05 +00:00
dependabot[bot] c2d283a64a
Bump tj-actions/changed-files from 22.2 to 41 in /.github/workflows (#28311) 
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 22.2 to 41.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v22.2...v41)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-01-03 09:12:53 +01:00
Yih-Dar dde6c427a1
Fix AMD push CI not triggered (#28029) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-12-14 12:44:00 +01:00
Ella Charlaix 39acfe84ba
Add deepspeed test to amd scheduled CI (#27633) 
* add deepspeed scheduled test for amd

* fix image

* add dockerfile

* add comment

* enable tests

* trigger

* remove trigger for this branch

* trigger

* change runner env to trigger the docker build image test

* use new docker image

* remove test suffix from docker image tag

* replace test docker image with original image

* push new image

* Trigger

* add back amd tests

* fix typo

* add amd tests back

* fix

* comment until docker image build scheduled test fix

* remove deprecated deepspeed build option

* upgrade torch

* update docker & make tests pass

* Update docker/transformers-pytorch-deepspeed-amd-gpu/Dockerfile

* fix

* tmp disable test

* precompile deepspeed to avoid timeout during tests

* fix comment

* trigger deepspeed tests with new image

* comment tests

* trigger

* add sklearn dependency to fix slow tests

* enable back other tests

* final update

---------

Co-authored-by: Felix Marty <felix@hf.co>
Co-authored-by: Félix Marty <9808326+fxmarty@users.noreply.github.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-12-11 16:33:36 +01:00
Yih-Dar 0f59d2f173
Fix AMD scheduled CI not triggered (#27951) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-12-11 16:22:10 +01:00
Yih-Dar 9f1f11a2e7
Show new failing tests in a more clear way in slack report (#27881) 
* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-12-07 15:09:30 +01:00
Maria Khalusova d6392482bd
removed the delete doc workflows (#27852) 2023-12-06 01:30:56 -08:00
Yih-Dar e0d2e69582
restructure AMD scheduled CI (#27743) 
* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-12-04 15:32:05 +01:00
Yih-Dar 62ab32b299
Remove `check_runner_status.yml` (#27767) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-11-30 10:17:25 +01:00
Yih-Dar 7a757bb694
Fix AMD Push CI not triggered (#27732) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-11-28 09:30:21 +01:00
Yih-Dar e11e26df93
Fix Past CI (#27696) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-11-27 09:11:58 +01:00
fxmarty f93c1e9ece
Add RoCm scheduled CI & upgrade RoCm CI to PyTorch 2.1 (#26940) 
* add scheduled ci on amdgpu

* fix likely typo

* more tests, avoid parallelism

* precise comment

* fix report channel

* trigger docker build on this branch

* fix

* fix

* run rocm scheduled ci

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-11-21 14:55:13 +01:00
Yih-Dar d903abfccc
Fix AMD CI not showing GPU (#27555) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-11-17 10:44:37 +01:00
Yih-Dar acb5b4aff5
Disable docker image build job `latest-pytorch-amd` for now (#27541) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-11-16 17:00:46 +01:00
Yih-Dar 64e21ca2a4
Make some jobs run on the GitHub Actions runners (#27512) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-11-15 10:43:16 +01:00
Yih-Dar 00dc856233
At most 2 GPUs for CI (#27435) 
At most 2 GPUs

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-11-10 16:19:06 +01:00
Mert Yanık eb30a49b20
Translate index.md to Turkish (#27093) 
* Add index.md for tukish language

* Fix index.md (huggingface/transformers#27088)

* Add 'tr' to additional files

* Update docs/source/tr/_toctree.yml

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update index.md

---------

Co-authored-by: Mert Yanık <mert.yanik@lcwaikiki.com>
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2023-11-08 08:35:20 -05:00
Yih-Dar eef7ea98c3
Update doctest workflow file (#27306) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-11-06 11:27:48 +01:00
Lysandre bc78fd1274 Dev version 2023-11-02 18:15:36 +01:00
Yih-Dar 9dc4ce9ea7
Disable CI runner check (#27170) 
Disable runner check

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-10-31 11:59:21 +01:00
Aarya Balwadkar a2f55a65cd
Hindi translation of pipeline_tutorial.md (#26837) 
* hindi translation of pipeline_tutorial.md

* Update pipeline_tutorial.md

* Update build_documentation.yml

* Update build_pr_documentation.yml

* Updated build_documentation.yml

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2023-10-25 11:21:49 -07:00
Akhil 093848d3cc
Added Telugu [te] translations (#26828) 
* Create index.md

* Create _toctree.yml

* Updated index.md in telugu

* Update _toctree.yml

* Create quicktour.md

* Update quicktour.md

* Create index.md

* Update quicktour.md

* Update docs/source/te/quicktour.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Delete docs/source/hi/index.md

* Update docs/source/te/quicktour.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/te/quicktour.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/te/quicktour.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/te/quicktour.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/te/quicktour.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/te/quicktour.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/te/quicktour.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/te/quicktour.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update build_documentation.yml

Added telugu [te]

* Update build_pr_documentation.yml

Added Telugu [te]

* Update _toctree.yml

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
 2023-10-20 15:27:55 -07:00
Yih-Dar 12cc123359
Better way to run AMD CI with different flavors (#26634) 
* Enable testing against mi250

* Change BERT to trigger tests

* Revert BERT's change

* AMD CI

* AMD CI

---------

Co-authored-by: Morgan Funtowicz <funtowiczmo@gmail.com>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-10-16 16:24:30 +02:00
Rockerz 9b7668c03a
add japanese documentation (#26138) 
* udpaet

* update

* Update docs/source/ja/autoclass_tutorial.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* add codes workflows/build_pr_documentation.yml

* Create preprocessing.md

* added traning.md

* Create Model_sharing.md

* add quicktour.md

* new

* ll

* Create benchmark.md

* Create Tensorflow_model

* add

* add community.md

* add create_a_model

* create custom_model.md

* create_custom_tools.md

* create fast_tokenizers.md

* create

* add

* Update docs/source/ja/_toctree.yml

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* md

* add

* commit

* add

* h

* Update docs/source/ja/peft.md

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>

* Update docs/source/ja/_toctree.yml

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>

* Update docs/source/ja/_toctree.yml

Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>

* Suggested Update

* add perf_train_gpu_one.md

* added perf based MD files

* Modify toctree.yml and Add transmartion to md codes

* Add `serialization.md` and edit `_toctree.yml`

* add task summary and tasks explained

* Add and Modify files starting from T

* Add testing.md

* Create main_classes files

* delete main_classes folder

* Add toctree.yml

* Update llm_tutorail.md

* Update docs/source/ja/_toctree.yml

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update misspelled filenames

* Update docs/source/ja/_toctree.yml

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/_toctree.yml

* Update docs/source/ja/_toctree.yml

* missplled file names inmrpovements

* Update _toctree.yml

* close tip block

* close another tip block

* Update docs/source/ja/quicktour.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/pipeline_tutorial.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/pipeline_tutorial.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/preprocessing.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/peft.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/add_new_model.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/testing.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/task_summary.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/tasks_explained.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update glossary.md

* Update docs/source/ja/transformers_agents.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/llm_tutorial.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/create_a_model.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/torchscript.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/benchmarks.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/troubleshooting.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/troubleshooting.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/troubleshooting.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update docs/source/ja/add_new_model.md

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>

* Update perf_torch_compile.md

* Update Year to default in en documentation

* Final Update

---------

Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
Co-authored-by: Younes Belkada <49240599+younesbelkada@users.noreply.github.com>
 2023-10-11 10:26:37 -07:00
Funtowicz Morgan 3632fb3c25
[AMD] Add initial version for run_tests_multi_gpu (#26346) 
* Add initial version for run_tests_multi_gpu

* Trigger change in BERT

* fix typo setup -> setup_gpu

* Add tag mi210

* Enable multi-gpu jobs

* One more

* Use dynamic device allocation

* Attempt to fix syntax for docker create

* fix script path

* fix

* temp machine type

* fix label

* Enable multi-gpu tests

* Rename multi-amd-gpu to multi-gpu

* Let's not be lazy dude

* Update rocm-smi output

* Add gpu_flavour in the matrix

* Fix typos

* merge single/multi dispatch into the matrix

* Format.

* Revert BERT's change

---------

Co-authored-by: Guillaume LEGENDRE <glegendre01@gmail.com>
 2023-10-03 11:13:45 +02:00
Yih-Dar 6ae71ec836
Update `runs-on` in workflow files (#26435) 
* update

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-09-27 19:25:52 +02:00
Funtowicz Morgan 2d71307dc0
Integrate AMD GPU in CI/CD environment (#26007) 
* Add a Dockerfile for PyTorch + ROCm based on official AMD released artifact

* Add a new artifact single-amdgpu testing on main

* Attempt to test the workflow without merging.

* Changed BERT to check if things are triggered

* Meet the dependencies graph on workflow

* Revert BERT changes

* Add check_runners_amdgpu to correctly mount and check availability

* Rename setup to setup_gpu for CUDA and add setup_amdgpu for AMD

* Fix all the needs.setup -> needs.setup_[gpu|amdgpu] dependencies

* Fix setup dependency graph to use check_runner_amdgpu

* Let's do the runner status check only on AMDGPU target

* Update the Dockerfile.amd to put ourselves in / rather than /var/lib

* Restore the whole setup for CUDA too.

* Let's redisable them

* Change BERT to trigger tests

* Restore BERT

* Add torchaudio with rocm 5.6 to AMD Dockerfile (#26050)

fix dockerfile

Co-authored-by: Felix Marty <felix@hf.co>

* Place AMD GPU tests in a separate workflow (correct branch) (#26105)

AMDGPU CI lives in an other workflow

* Fix invalid job name is dependencies.

* Remove tests multi-amdgpu for now.

* Use single-amdgpu

* Use --net=host for now.

* Remote host networking.

* Removed duplicated check_runners_amdgpu step

* Let's tag machine-types with mi210 for now.

* Machine type should be only mi210

* Remove unnecessary push.branches item

* Apply review suggestions moving from `x-amdgpu` to `x-gpu` introducing `amd-gpu` and `miXXX` labels.

* Remove amdgpu from step names.

* finalize

* delete

---------

Co-authored-by: fxmarty <9808326+fxmarty@users.noreply.github.com>
Co-authored-by: Felix Marty <felix@hf.co>
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-09-20 14:48:49 +02:00
Ralf Müller-Zimmermann e469be3406
Fix the gitlab user mention in issue templates to the correct user (#26237) 2023-09-19 01:49:03 +02:00
Yih-Dar 9219d1427b
pin pandas==2.0.3 (#25875) 
Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-30 18:10:01 +02:00
Yih-Dar 62399d6f35
Add flax installation in daily doctest workflow (#25860) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-08-30 15:13:50 +02:00
Sylvain Gugger c2123626aa
Update list of persons to tag (#25708) 2023-08-24 10:13:30 +02:00
Lysandre Debut 8657ec68fc
Sets the stalebot to 10 AM CEST (#25678) 
This sets the stale bot trigger time at 10 AM CEST rather than 5 PM CEST as all core maintainers on watch duty are now in the European timezone
 2023-08-23 14:21:07 +02:00
Yih-Dar b413e0610b
Remove `utils/documentation_tests.txt` (#25680) 
* fix

* fix

* fix

* fix

* fix

* fix

* Apply suggestions from code review

Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>
 2023-08-23 11:14:45 +02:00
Sylvain Gugger 2defb6b048
More utils doc (#25457) 
* Document and clean more utils.

* More documentation and fixes

* Switch to Lysandre's token

* Address review comments

* Actually put else
 2023-08-17 07:58:35 +02:00
Sylvain Gugger d561408cc3
Skip Add model like job (#24865) 2023-07-17 15:52:04 -04:00
Yih-Dar 11cb6e0f7e
Unpin DeepSpeed and require DS >= 0.9.3 (#24541) 
* fix

* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-28 14:01:22 +02:00
Yih-Dar e84bf1f734
⚠️ Time to say goodbye to py37 (#24091) 
* fix

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-28 07:22:39 +02:00
Sylvain Gugger eb849f6604
Migrate doc files to Markdown. (#24376) 
* Rename index.mdx to index.md

* With saved modifs

* Address review comment

* Treat all files

* .mdx -> .md

* Remove special char

* Update utils/tests_fetcher.py

Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>

---------

Co-authored-by: Lysandre Debut <lysandre.debut@reseau.eseo.fr>
 2023-06-20 18:07:47 -04:00
Yih-Dar 0b259a3b7e
Clean up disk sapce during docker image build for `transformers-pytorch-gpu` (#24346) 
fix

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-19 12:54:02 +02:00
Yih-Dar 896a58de15
Byebye pytorch 1.9 (#24080) 
byebye

---------

Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
 2023-06-16 16:38:23 +02:00
Mishig ef010071ee
Update delete_doc_comment_trigger.yml (#24084) 
fix base workflow name
 2023-06-07 17:55:48 +02:00