Commit Graph

980 Commits

Author SHA1 Message Date
Joshua Lochner 1914c0784d Fix conversion to grayscale 2023-06-29 23:38:52 +02:00
Julien Chaumond 6eb924b7b1
Add `RobertaForTokenClassification` and an example checkpoint on Hub (#170) 2023-06-29 20:15:58 +02:00
Joshua Lochner 27d7ea489b
Improvements to documentation (#172)
* link to the conversion Space for maximum simplicity

* add some types to script (very optional)

* typo

* no need for trailing slash here

* Node is also a valid option

* Document how to find a compatible checkpoint on the hub

* Update README

* Fix typing

* Update docs index

---------

Co-authored-by: Julien Chaumond <julien@huggingface.co>
2023-06-29 19:32:17 +02:00
Joshua Lochner a5ca113d51
[WIP] New model/tokenizer types (#165)
* Recursively replace tensors with custom class

* Add mobile vit models

* Add example code for `ImageClassificationPipeline`

* Fix example urls

* Add MobileViT models and processors

* Update optimum requirement in conversion script

Previous name is deprecated

* Update supported models

* Update supported_models.py

* Update supported_models.py

* Update tokenizer test generator script

* Add special test case for falcon tokenizers

* Update tokenizer test script

* Add support for `FalconTokenizer`

* Update `BertPreTokenizer` call parameter types

* Add `GPTNeoXTokenizer` tokenizer (mpt)

* Use transformers from source when testing

* Reuse `prepare_model_inputs` function type

Better than using `@see {@link ... }` since it works with intellisense.
2023-06-28 15:14:44 +02:00
Joshua Lochner 8d6622ef9b [version] Update to 2.3.0 2023-06-22 15:36:02 +02:00
Joshua Lochner c491c2661f Do not use browser cache if inaccessible (Fixes #162) 2023-06-22 15:32:52 +02:00
Pushpender Saini 15854f9cd6
Set chunk timestamp to rounded time (#160) 2023-06-22 01:01:06 +02:00
Joshua Lochner f628b841a8
Allow user to set `per_channel` and `reduce_range` quantization params (#156) (#157)
* Allow user to set `per_channel` and `reduce_range` quantization parameters (#156)

Also save quantization options

* Get operators of graph and subgraphs
2023-06-22 00:43:43 +02:00
Joshua Lochner d90f58110a
Add whisper unit tests (#155)
* Only run encoder with required inputs

* Add basic whisper unit tests

* Add newline after heading for docs

* Add unit test for transcribing english with timestamps

* Add multilingual test case
2023-06-21 23:58:16 +02:00
Joshua Lochner 4804171180
Do not use spread operator to concatenate large arrays (Closes #153) (#154)
* Do not use spread operator for merging large arrays (Fix #153)

* Add unit test for encoding long strings
2023-06-21 01:21:14 +02:00
Joshua Lochner 573012b434
[docs] Add tutorial + example app for server-side whisper (#147)
* Update typo in node tutorial

* Create node audio processing tutorial

* Point to tutorial in `read_audio` function

* Rename `.md` to `.mdx`

* Add node audio processing tutorial to table of contents

* Add link to model in tutorial

* Update error message grammar
2023-06-20 23:10:33 +02:00
Joshua Lochner 35b9e21193
Support calling of decoder-only models (Fixes #137) (#149)
* Override `LOAD_FUNCTION` for decoder-only models

* Use object destructuring in `_call` functions

* Allow decoder-only models to be called

* Fix detection of default call function

* Update default `_call` JSDoc

* Mark helper functions as private

* Remove outdated comments

* Fix JSDoc

* Rename functions

* Specify model types

Reduces major code duplication

* Improve model output classes

* Remove `encoder_input_name` from seq2seq forward method

* Extract `validateInputs` helper function from `sessionRun`

* Move `compare` helper function to separate utility file

* Default `model_type` to null

* Reduce duplication when loading models using `.from_pretrained`

* Add unit tests for loading models using `.from_pretrained()`

* Compute attention mask for decoder if not given

* Improve decoder attention computation

* Implement `flatten` and `view` tensor ops

* Add documentation for new tensor ops

* Fix `flatten` input types
2023-06-20 15:24:35 +02:00
Joshua Lochner d279ec3c86 Add question-answering example (Closes #144) 2023-06-10 00:16:09 +02:00
Joshua Lochner 035f69f79a [version] Update to 2.2.0 2023-06-09 15:18:29 +02:00
Joshua Lochner f7ffef7f43 Add whisper web example 2023-06-09 15:15:23 +02:00
Joshua Lochner 8625f4aba3
Add multilingual transcription + translation for whisper models (#87, #95) (#133)
* Align `.generate()` return type with python library

* Add multilingual transcription + translation for whisper models (#87, #95)

* Include `return_timestamps` in calculation of `forced_decoder_ids`

* Only return non-null `forced_decoder_ids`

* Allow user to specify task in any case

* Only set `forced_decoder_ids` when non-empty

* Implement `SuppressTokensAtBeginLogitsProcessor`
2023-06-09 15:09:42 +02:00
Mishig f5f78c4663
[doc build] Use secrets (#139) 2023-06-09 14:42:14 +02:00
Joshua Lochner 3b546b41f5 Fix JSDoc example 2023-06-06 15:56:49 +02:00
Joshua Lochner 0e28f5314e Add JSDoc examples for `RawImage` usage 2023-06-06 15:53:26 +02:00
Joshua Lochner 0ecf9606be
Post-process jsdoc2md marker links (#120)
* Post-process jsdoc2md marker links

* Add `group` class to certain anchor tags
2023-06-05 17:17:21 +02:00
Joshua Lochner acc526fe13 Update module version in README. 2023-06-05 02:30:10 +02:00
Joshua Lochner e5e460bd3b [version] Update to 2.1.1 2023-06-02 02:11:24 +02:00
Joshua Lochner f517807131 Fix caching in the browser 2023-06-02 02:05:22 +02:00
Joshua Lochner 34b0e8b88f [version] Update to v2.1.0 2023-06-01 14:42:27 +02:00
Joshua Lochner 3eea52d43b
Merge pull request #130 from xenova/merged
Improved feature extraction, quantization, and testing (merged PR)
2023-06-01 14:22:58 +02:00
Joshua Lochner 658fbdcdd6 Add JSDoc for `PretrainedConfig` class 2023-06-01 13:57:45 +02:00
Joshua Lochner 54b861a284 Update configs.js JSDoc 2023-06-01 13:46:40 +02:00
Joshua Lochner db3da29376 Implement `fetch` workaround
Should prevent SocketError's from occurring
2023-06-01 01:09:14 +02:00
Joshua Lochner 1520012b43 Update translation test case 2023-05-31 23:58:48 +02:00
Joshua Lochner 25fb4a6f49 Update supported_models.py 2023-05-31 23:38:23 +02:00
Joshua Lochner 7fa7dc6f71 Merge branch 'sentence-transformers' into merged 2023-05-31 21:08:40 +02:00
Joshua Lochner 569f3f820a [docs] Add JSDoc for configs.js 2023-05-31 19:28:44 +02:00
Joshua Lochner 76dc236e3f Fix tests import 2023-05-31 18:48:06 +02:00
Joshua Lochner 861911fa12 Merge branch 'quantization' 2023-05-31 18:45:15 +02:00
Joshua Lochner 6187de17fa Merge branch 'testing' 2023-05-31 18:44:41 +02:00
Joshua Lochner d107ccb51e [demo] Update model sizes 2023-05-31 18:42:14 +02:00
Joshua Lochner 387d0df7e1 Use custom user-agent when running node 2023-05-31 16:25:18 +02:00
Joshua Lochner 52e20f2daf Update conversion script args 2023-05-31 15:13:19 +02:00
Joshua Lochner eed6a05c5f Update supported_models.py 2023-05-31 15:07:54 +02:00
Joshua Lochner 84719a5fa3 Add `--skip_validation` option to conversion script 2023-05-31 11:59:20 +02:00
Joshua Lochner bcaf7dd155 Remove torch_dtype from config 2023-05-31 02:28:16 +02:00
Joshua Lochner 99c771bf48 Run `generate_tests.py` as module 2023-05-31 02:21:15 +02:00
Joshua Lochner 813b2abe77 Remove unnecessary imports 2023-05-31 02:06:34 +02:00
Joshua Lochner e3f73cd17d Add unit tests for config files 2023-05-31 02:04:58 +02:00
Joshua Lochner 8a4dcbf2ef Add JSDoc to imported Tensor class 2023-05-31 02:04:27 +02:00
Joshua Lochner 3d05083c1a Add config auto class 2023-05-31 02:04:07 +02:00
Joshua Lochner 44caca829f Fix JSDoc for tokenizers 2023-05-31 01:43:50 +02:00
Joshua Lochner db2bcdfd55 Fix masked LM unit tests 2023-05-31 01:43:31 +02:00
Joshua Lochner 40149ebb6b Update supported models 2023-05-30 19:37:34 +02:00
Joshua Lochner 5e53f2cf02 Rename tasks script 2023-05-30 17:13:27 +02:00