transformers.js/scripts
Joshua Lochner ebd533531a
Add support for cross-encoder models (+fix token type ids) (#501)
* Formatting

* Update ESM pair template

* Fix token type ids

* Update JSDoc

* Cleanup

* Remove unused `prepare_model_inputs` function

* Move pad and truncate logic to helper functions

* Add static padding/truncation unit tests

* Fix padding/truncation

* Remove unused `add_token_types` function

* Reduce duplication

* `let` -> `const` where possible

* Add cross-encoder models
2024-01-04 18:37:26 +02:00
..
extra Add support for cross-encoder models (+fix token type ids) (#501) 2024-01-04 18:37:26 +02:00
convert.py Add `phi-1_5` model (#493) 2024-01-02 23:12:09 +02:00
requirements.txt Use `transformers==4.33.2` in conversion script 2023-09-23 17:06:27 +02:00
supported_models.py Add support for cross-encoder models (+fix token type ids) (#501) 2024-01-04 18:37:26 +02:00