Joshua Lochner
db9250b3e8
Add sequence post processor ( #771 )
...
* Add `Sequence` PostProcessor
Required by https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/
* Support `return_token_type_ids`
* Add llama3 tokenizer to unit tests
* Add test for allowing user to request for token type ids
* Add JSDoc
* Update generate_tests.py
2024-05-23 11:42:24 +02:00
Joshua Lochner
e50f5686a2
Add support for FastViT ( #749 )
...
* Add support for FastViT
* Comment out list of timm models
2024-05-23 01:47:16 +02:00
Kendell R
da0e9204c4
Auto rotate image ( #737 )
2024-05-23 01:43:56 +02:00
Joshua Lochner
8d166ca642
Optimize FFT ( #766 )
...
* Optimize FFT for real transforms
* Throw error if power is not specified
https://github.com/huggingface/transformers/issues/27772
2024-05-23 01:43:05 +02:00
Joshua Lochner
8963720585
Fix CI - update tests.yml ( #768 )
2024-05-23 01:40:50 +02:00
Joshua Lochner
0af1e2f3c2
Remove old import from `stream/web` for `ReadableStream` ( #752 )
...
Node 16 is no longer supported
2024-05-20 18:05:48 +02:00
Joshua Lochner
bd315529ab
Update tokenizer playground ( #717 )
...
* [tokenizer playground] Add Mistral v3 tokenizer
* Add support for custom tokenizers
* Add llama 3 tokenizer
2024-05-10 23:56:28 +02:00
Hans
880cd3ea65
Support reading data from blob URI ( #645 )
...
* Make blob as valid URL
* Create function to detect the blob URI
* Change to `isValidUrl`
* Remove comment
Co-authored-by: Joshua Lochner <admin@xenova.com>
* Merge `isValidHttpUrl` into `isValidUrl`
* Correct implement
* Update docs
* Add test
* Remove export for `isValidUrl`
* Test read blob via `getFile`
* Use `res.text()` instead `res.body`
---------
Co-authored-by: Joshua Lochner <admin@xenova.com>
2024-05-08 13:14:50 +02:00
Ikko Eltociear Ashimine
8bb8c5a33c
docs: update vanilla-js.md ( #738 )
...
minor fix
2024-05-05 23:59:50 +02:00
Joshua Lochner
7122a0a5a1
Add mobilevitv2 ( #721 )
...
* Add support for MobileViTV2
* Update supported_models.py
* Add support for `do_flip_channel_order`
* Add unit test for `do_flip_channel_order=true`
2024-05-02 11:42:44 +02:00
Joshua Lochner
1a9964fb09
Use ungated version of mistral tokenizer ( #718 )
2024-04-20 16:01:17 +02:00
Joshua Lochner
992f643e2a
[version] Update to 2.17.1
2024-04-18 17:25:49 +02:00
Joshua Lochner
6d5901e652
Add `ignore_merges` option to BPE tokenizers ( #716 )
2024-04-18 13:59:52 +02:00
Joshua Lochner
642743136e
[version] Update to 2.17.0
2024-04-11 01:20:24 +02:00
Joshua Lochner
c2c45cb577
Improve support of conversational models ( #658 )
...
* Add `return_full_text` option for text-generation models
* [wip] Support chat inputs in text-generation pipeline
* Align return type with python version
* Remove conversational task (moved to text-generation)
* Fix typos
2024-04-11 00:58:50 +02:00
Joshua Lochner
aa542cf548
Update dependencies ( #704 )
...
* Update dependencies
* Bump protobufjs from 7.2.4 to 7.2.6 in /examples/semantic-image-search (#705 )
Bumps [protobufjs](https://github.com/protobufjs/protobuf.js ) from 7.2.4 to 7.2.6.
- [Release notes](https://github.com/protobufjs/protobuf.js/releases )
- [Changelog](https://github.com/protobufjs/protobuf.js/blob/master/CHANGELOG.md )
- [Commits](https://github.com/protobufjs/protobuf.js/compare/protobufjs-v7.2.4...protobufjs-v7.2.6 )
---
updated-dependencies:
- dependency-name: protobufjs
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump protobufjs from 7.2.4 to 7.2.6 in /examples/next-client (#706 )
Bumps [protobufjs](https://github.com/protobufjs/protobuf.js ) from 7.2.4 to 7.2.6.
- [Release notes](https://github.com/protobufjs/protobuf.js/releases )
- [Changelog](https://github.com/protobufjs/protobuf.js/blob/master/CHANGELOG.md )
- [Commits](https://github.com/protobufjs/protobuf.js/compare/protobufjs-v7.2.4...protobufjs-v7.2.6 )
---
updated-dependencies:
- dependency-name: protobufjs
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump protobufjs from 7.2.4 to 7.2.6 in /examples/next-server (#707 )
Bumps [protobufjs](https://github.com/protobufjs/protobuf.js ) from 7.2.4 to 7.2.6.
- [Release notes](https://github.com/protobufjs/protobuf.js/releases )
- [Changelog](https://github.com/protobufjs/protobuf.js/blob/master/CHANGELOG.md )
- [Commits](https://github.com/protobufjs/protobuf.js/compare/protobufjs-v7.2.4...protobufjs-v7.2.6 )
---
updated-dependencies:
- dependency-name: protobufjs
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump protobufjs in /examples/semantic-image-search-client (#708 )
Bumps [protobufjs](https://github.com/protobufjs/protobuf.js ) from 7.2.4 to 7.2.6.
- [Release notes](https://github.com/protobufjs/protobuf.js/releases )
- [Changelog](https://github.com/protobufjs/protobuf.js/blob/master/CHANGELOG.md )
- [Commits](https://github.com/protobufjs/protobuf.js/compare/protobufjs-v7.2.4...protobufjs-v7.2.6 )
---
updated-dependencies:
- dependency-name: protobufjs
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-11 00:24:51 +02:00
Varun Patil
15b90b7523
maths: switch dot to faster for loop ( #667 )
...
* maths: switch dot to faster for loop
* Update src/utils/maths.js
---------
Co-authored-by: Joshua Lochner <admin@xenova.com>
2024-04-10 17:01:03 +02:00
Joshua Lochner
cdbc532cf6
Update dependencies ( #702 )
...
* Bump vite from 4.5.2 to 4.5.3 in /examples/tokenizer-playground (#693 )
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite ) from 4.5.2 to 4.5.3.
- [Release notes](https://github.com/vitejs/vite/releases )
- [Changelog](https://github.com/vitejs/vite/blob/v4.5.3/packages/vite/CHANGELOG.md )
- [Commits](https://github.com/vitejs/vite/commits/v4.5.3/packages/vite )
---
updated-dependencies:
- dependency-name: vite
dependency-type: direct:development
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump express from 4.18.2 to 4.19.2 (#666 )
Bumps [express](https://github.com/expressjs/express ) from 4.18.2 to 4.19.2.
- [Release notes](https://github.com/expressjs/express/releases )
- [Changelog](https://github.com/expressjs/express/blob/master/History.md )
- [Commits](https://github.com/expressjs/express/compare/4.18.2...4.19.2 )
---
updated-dependencies:
- dependency-name: express
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump vite from 5.0.12 to 5.0.13 in /examples/remove-background-client (#694 )
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite ) from 5.0.12 to 5.0.13.
- [Release notes](https://github.com/vitejs/vite/releases )
- [Changelog](https://github.com/vitejs/vite/blob/v5.0.13/packages/vite/CHANGELOG.md )
- [Commits](https://github.com/vitejs/vite/commits/v5.0.13/packages/vite )
---
updated-dependencies:
- dependency-name: vite
dependency-type: direct:development
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump vite from 5.0.12 to 5.0.13 in /examples/zero-shot-classification (#695 )
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite ) from 5.0.12 to 5.0.13.
- [Release notes](https://github.com/vitejs/vite/releases )
- [Changelog](https://github.com/vitejs/vite/blob/v5.0.13/packages/vite/CHANGELOG.md )
- [Commits](https://github.com/vitejs/vite/commits/v5.0.13/packages/vite )
---
updated-dependencies:
- dependency-name: vite
dependency-type: direct:development
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump vite from 5.0.12 to 5.0.13 in /examples/depth-anything-client (#696 )
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite ) from 5.0.12 to 5.0.13.
- [Release notes](https://github.com/vitejs/vite/releases )
- [Changelog](https://github.com/vitejs/vite/blob/v5.0.13/packages/vite/CHANGELOG.md )
- [Commits](https://github.com/vitejs/vite/commits/v5.0.13/packages/vite )
---
updated-dependencies:
- dependency-name: vite
dependency-type: direct:development
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump vite from 5.0.12 to 5.0.13 in /examples/semantic-audio-search (#697 )
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite ) from 5.0.12 to 5.0.13.
- [Release notes](https://github.com/vitejs/vite/releases )
- [Changelog](https://github.com/vitejs/vite/blob/v5.0.13/packages/vite/CHANGELOG.md )
- [Commits](https://github.com/vitejs/vite/commits/v5.0.13/packages/vite )
---
updated-dependencies:
- dependency-name: vite
dependency-type: direct:development
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump vite from 4.5.2 to 4.5.3 in /examples/code-completion (#698 )
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite ) from 4.5.2 to 4.5.3.
- [Release notes](https://github.com/vitejs/vite/releases )
- [Changelog](https://github.com/vitejs/vite/blob/v4.5.3/packages/vite/CHANGELOG.md )
- [Commits](https://github.com/vitejs/vite/commits/v4.5.3/packages/vite )
---
updated-dependencies:
- dependency-name: vite
dependency-type: direct:development
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump vite from 5.1.4 to 5.1.7 in /examples/video-object-detection (#699 )
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite ) from 5.1.4 to 5.1.7.
- [Release notes](https://github.com/vitejs/vite/releases )
- [Changelog](https://github.com/vitejs/vite/blob/v5.1.7/packages/vite/CHANGELOG.md )
- [Commits](https://github.com/vitejs/vite/commits/v5.1.7/packages/vite )
---
updated-dependencies:
- dependency-name: vite
dependency-type: direct:development
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump vite from 5.1.1 to 5.1.7 in /examples/cross-encoder (#700 )
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite ) from 5.1.1 to 5.1.7.
- [Release notes](https://github.com/vitejs/vite/releases )
- [Changelog](https://github.com/vitejs/vite/blob/v5.1.7/packages/vite/CHANGELOG.md )
- [Commits](https://github.com/vitejs/vite/commits/v5.1.7/packages/vite )
---
updated-dependencies:
- dependency-name: vite
dependency-type: direct:development
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Bump vite from 5.1.1 to 5.1.7 in /examples/adaptive-retrieval (#701 )
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite ) from 5.1.1 to 5.1.7.
- [Release notes](https://github.com/vitejs/vite/releases )
- [Changelog](https://github.com/vitejs/vite/blob/v5.1.7/packages/vite/CHANGELOG.md )
- [Commits](https://github.com/vitejs/vite/commits/v5.1.7/packages/vite )
---
updated-dependencies:
- dependency-name: vite
dependency-type: direct:development
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-10 16:07:58 +02:00
Jonathan Padilla
f71ee32dd9
Add binary embedding quantization support to FeatureExtraction pipeline ( #691 )
...
* Add binary embedding quantization support to FeatureExtraction pipeline
* Add JSDoc return type
* Apply suggestions from code review
* Update src/utils/tensor.js
* Update example code snippet
* Update src/pipelines.js
---------
Co-authored-by: Joshua Lochner <admin@xenova.com>
2024-04-10 15:13:36 +02:00
dependabot[bot]
8804c36591
Bump ip from 2.0.0 to 2.0.1 in /examples/electron ( #665 )
...
Bumps [ip](https://github.com/indutny/node-ip ) from 2.0.0 to 2.0.1.
- [Commits](https://github.com/indutny/node-ip/compare/v2.0.0...v2.0.1 )
---
updated-dependencies:
- dependency-name: ip
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-26 22:45:55 +02:00
dependabot[bot]
3509fc405b
Bump webpack-dev-middleware from 5.3.3 to 5.3.4 ( #661 )
...
Bumps [webpack-dev-middleware](https://github.com/webpack/webpack-dev-middleware ) from 5.3.3 to 5.3.4.
- [Release notes](https://github.com/webpack/webpack-dev-middleware/releases )
- [Changelog](https://github.com/webpack/webpack-dev-middleware/blob/v5.3.4/CHANGELOG.md )
- [Commits](https://github.com/webpack/webpack-dev-middleware/compare/v5.3.3...v5.3.4 )
---
updated-dependencies:
- dependency-name: webpack-dev-middleware
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-26 15:49:41 +02:00
Joshua Lochner
d50b3193fb
[version] Update to 2.16.1
2024-03-20 16:44:39 +02:00
Joshua Lochner
4cab8eceb6
Add `image-feature-extractor` pipeline ( #650 )
...
* Add `image-feature-extractor` pipeline
* Update "Image Feature Extraction" heading
2024-03-20 15:30:27 +02:00
Joshua Lochner
f0ef2e8eee
Update tokenizer `apply_chat_template` functionality ( #647 )
...
* Allow custom kwargs in `tokenizer.apply_chat_template`
* Update jinja dependency version
* Add `tokenizer_kwargs` options
* Add support for dictionaries of chat templates in the tokenizer config
* Add `CohereTokenizer`
* `apply_chat_template` is no longer async
* Add unit test for multiple chat templates
* Update tokenizers.js
* Also update when `chat_template` is undefined
* Support setting tokenizer and text from URL
* Update Claude tokenizer display name
* Add Cohere Command-R tokenizer to playground
* Add `Grok1Tokenizer`
* Throw error if chat template object is malformed
* Improved error checking
* Remove redundant error check
* `template_dict` can be a null-prototype object
2024-03-20 15:22:01 +02:00
Joshua Lochner
40cdd36597
Fix channel format when padding non-square images for certain models ( #655 )
...
* Add tensor permute unit tests
* Rename transpose -> permute
* Fix padding for non-square images
* Add vitmatte padding unit test
* Create `RawImage.toTensor` helper function
* Add bankers rounding test case
* `.toBe()` -> `.toBeCloseTo()` for floating point numbers
2024-03-20 14:35:30 +02:00
dependabot[bot]
8c731fa54b
Bump follow-redirects from 1.15.4 to 1.15.6 ( #652 )
...
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects ) from 1.15.4 to 1.15.6.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases )
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.15.4...v1.15.6 )
---
updated-dependencies:
- dependency-name: follow-redirects
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-19 00:17:47 +02:00
Joshua Lochner
a51a61be22
Add support for EfficientNet ( #639 )
...
* Add support for efficientnet
* Add `EfficientNetImageProcessor`
2024-03-16 14:49:32 +02:00
Joshua Lochner
314b7f0dc4
[version] Update to 2.16.0
2024-03-07 15:45:22 +02:00
Dave
8eef154b1e
Add WavLM- & Wav2Vec2ForAudioFrameClassification support ( #611 )
...
* Add WavLMForXVector support
* fix model docs
* Add WavLMForAudioFrameClassification
* Add missing wWav2Vec2ForAudioFrameCl.
* Add doc comment
* Add doc string wav2vec2
* update comment
* make example like python
* Update src/models.js
---------
Co-authored-by: Joshua Lochner <admin@xenova.com>
2024-03-07 15:27:49 +02:00
Dave
5bb8d25337
Add StableLM support ( #616 )
...
* add stablelm model impl.
* add stablelm mapping
* Update `StableLMPreTrainedModel` class config
* Update stablelm conversion script quantization settings
* Add StableLm to list of supported models
* Remove StableLMForSequenceClassification
will be added in seperate PR
Co-authored-by: Joshua Lochner <admin@xenova.com>
* Remove SequenceClassifierOutputWithPast
will be added in seperate PR
Co-authored-by: Joshua Lochner <admin@xenova.com>
* Remove StableLMForSequenceClassification mapping
will be added in seperate pr
Co-authored-by: Joshua Lochner <admin@xenova.com>
* Update src/models.js
* Naming nit
* Naming nits
* Update scripts/convert.py
* Update src/models.js
---------
Co-authored-by: Joshua Lochner <admin@xenova.com>
2024-03-07 03:05:27 +02:00
Joshua Lochner
dab1b8139f
Remove GH-specific redirect logic ( #630 )
2024-03-07 02:47:32 +02:00
Joshua Lochner
8694a856ee
Bump `@huggingface/jinja` ( #629 )
2024-03-07 01:59:06 +02:00
Joshua Lochner
382ff3af35
Create cross-encoder demo ( #617 )
2024-03-06 17:31:57 +02:00
Joshua Lochner
e072ee6b70
Add support for starcoder2 models ( #622 )
2024-03-06 13:09:04 +02:00
Joshua Lochner
5f572241b4
Add support for `UniSpeech` and `UniSpeechSat` models ( #624 )
...
* Add support for `UniSpeech` and `UniSpeechSat` models
* Remove listed support for unispeech STT models
* Align unispeech w/ existing wav2vec logic
2024-03-06 13:08:28 +02:00
Joshua Lochner
2e53f51ce7
Update `@huggingface/jinja` -> 0.2.0 ( #627 )
2024-03-05 17:04:08 +02:00
Joshua Lochner
55a4a69491
Check for existence of `onnx_env.wasm` before updating wasmPaths ( #621 )
2024-03-05 17:03:25 +02:00
Joshua Lochner
d525c8bfd4
Add Claude 3 + Mistral to tokenizer playground
2024-03-05 00:52:12 +02:00
Joshua Lochner
ce4fd62202
Update tokenizer playground dependencies ( #599 )
...
* Update transformers.js version
* Update Token.jsx
2024-03-05 00:42:37 +02:00
Dave
b5a548fc65
Add support for `WavLMForXVector` ( #603 )
...
* Add WavLMForXVector support
* fix model docs
* fix bad naming
* Apply suggestions from code review
* Update default `wavlm` quantization settings
* Update list of supported `wavlm` models
* Update JSDoc
* Fix typo
---------
Co-authored-by: Joshua Lochner <admin@xenova.com>
2024-02-28 21:24:41 +02:00
Joshua Lochner
271c6f1744
Create video object detection demo ( #607 )
2024-02-27 19:35:12 +02:00
Joshua Lochner
7772d1db0a
[version] Update to 2.15.1
2024-02-21 16:26:05 +02:00
Joshua Lochner
68ed7f6cbb
Add Gemma Tokenizer ( #598 )
...
* Fix styling for whitespace tokens
* Add `GemmaTokenizer`
* Update minimum `@huggingface/jinja` version
* Add Gemma to tokenizer playground
* Add Gemma tokenizer unit test
* Update tokenizer names in playground
* Update Gemma tokenizer test
2024-02-21 16:22:22 +02:00
Joshua Lochner
6d2808b571
Create adaptive retrieval demo ( #587 )
...
* Create adaptive retrieval demo
* Remove waiting label
2024-02-21 15:16:51 +02:00
Joshua Lochner
d1eabaeb0b
Update tokenizer playground ( #597 )
...
* Update transformers.js version
* Remove unused images
* Remove link to placeholder image
2024-02-21 14:04:11 +02:00
Joshua Lochner
5ac17bda83
Add support for OWLv2 models ( #579 )
2024-02-19 14:57:06 +02:00
Joshua Lochner
e9092d2337
Add basic 2D `layer_norm` operator ( #588 )
2024-02-19 14:53:06 +02:00
Joshua Lochner
351dbed922
Add support for exporting custom models ( #556 )
...
* Add support for exporting custom models
* Require task to be specified
* Pass `trust_remote_code` to all `from_pretrained` calls.
* Add support for `nomic_bert`
Prevents unsupported model warnings
2024-02-19 14:23:27 +02:00
Joshua Lochner
66d59d79d4
Add remove-background demo ( #576 )
...
* Add remove-background demo
* Open links in new tab
* Add mention of running locally
2024-02-07 14:31:52 +02:00
Joshua Lochner
41f98b761f
[version] Update to 2.15.0
2024-02-06 15:06:50 +02:00