dnrops
/
transformers.js
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,043 Commits 57 Branches 49 Tags 103 MiB
Commit Graph

1043 Commits

Author SHA1 Message Date
Joshua Lochner 314b7f0dc4 [version] Update to 2.16.0 2024-03-07 15:45:22 +02:00
Dave 8eef154b1e
Add WavLM- & Wav2Vec2ForAudioFrameClassification support (#611) 
* Add WavLMForXVector support

* fix model docs

* Add WavLMForAudioFrameClassification

* Add missing wWav2Vec2ForAudioFrameCl.

* Add doc comment

* Add doc string wav2vec2

* update comment

* make example like python

* Update src/models.js

---------

Co-authored-by: Joshua Lochner <admin@xenova.com>
 2024-03-07 15:27:49 +02:00
Dave 5bb8d25337
Add StableLM support (#616) 
* add stablelm model impl.

* add stablelm mapping

* Update `StableLMPreTrainedModel` class config

* Update stablelm conversion script quantization settings

* Add StableLm to list of supported models

* Remove StableLMForSequenceClassification

will be added in seperate PR

Co-authored-by: Joshua Lochner <admin@xenova.com>

* Remove SequenceClassifierOutputWithPast

will be added in seperate PR

Co-authored-by: Joshua Lochner <admin@xenova.com>

* Remove StableLMForSequenceClassification mapping

will be added in seperate pr

Co-authored-by: Joshua Lochner <admin@xenova.com>

* Update src/models.js

* Naming nit

* Naming nits

* Update scripts/convert.py

* Update src/models.js

---------

Co-authored-by: Joshua Lochner <admin@xenova.com>
 2024-03-07 03:05:27 +02:00
Joshua Lochner dab1b8139f
Remove GH-specific redirect logic (#630) 2024-03-07 02:47:32 +02:00
Joshua Lochner 8694a856ee
Bump `@huggingface/jinja` (#629) 2024-03-07 01:59:06 +02:00
Joshua Lochner 382ff3af35
Create cross-encoder demo (#617) 2024-03-06 17:31:57 +02:00
Joshua Lochner e072ee6b70
Add support for starcoder2 models (#622) 2024-03-06 13:09:04 +02:00
Joshua Lochner 5f572241b4
Add support for `UniSpeech` and `UniSpeechSat` models (#624) 
* Add support for `UniSpeech` and `UniSpeechSat` models

* Remove listed support for unispeech STT models

* Align unispeech w/ existing wav2vec logic
 2024-03-06 13:08:28 +02:00
Joshua Lochner 2e53f51ce7
Update `@huggingface/jinja` -> 0.2.0 (#627) 2024-03-05 17:04:08 +02:00
Joshua Lochner 55a4a69491
Check for existence of `onnx_env.wasm` before updating wasmPaths (#621) 2024-03-05 17:03:25 +02:00
Joshua Lochner d525c8bfd4
Add Claude 3 + Mistral to tokenizer playground 2024-03-05 00:52:12 +02:00
Joshua Lochner ce4fd62202
Update tokenizer playground dependencies (#599) 
* Update transformers.js version

* Update Token.jsx
 2024-03-05 00:42:37 +02:00
Dave b5a548fc65
Add support for `WavLMForXVector` (#603) 
* Add WavLMForXVector support

* fix model docs

* fix bad naming

* Apply suggestions from code review

* Update default `wavlm` quantization settings

* Update list of supported `wavlm` models

* Update JSDoc

* Fix typo

---------

Co-authored-by: Joshua Lochner <admin@xenova.com>
 2024-02-28 21:24:41 +02:00
Joshua Lochner 271c6f1744
Create video object detection demo (#607) 2024-02-27 19:35:12 +02:00
Joshua Lochner 7772d1db0a [version] Update to 2.15.1 2024-02-21 16:26:05 +02:00
Joshua Lochner 68ed7f6cbb
Add Gemma Tokenizer (#598) 
* Fix styling for whitespace tokens

* Add `GemmaTokenizer`

* Update minimum `@huggingface/jinja` version

* Add Gemma to tokenizer playground

* Add Gemma tokenizer unit test

* Update tokenizer names in playground

* Update Gemma tokenizer test
 2024-02-21 16:22:22 +02:00
Joshua Lochner 6d2808b571
Create adaptive retrieval demo (#587) 
* Create adaptive retrieval demo

* Remove waiting label
 2024-02-21 15:16:51 +02:00
Joshua Lochner d1eabaeb0b
Update tokenizer playground (#597) 
* Update transformers.js version

* Remove unused images

* Remove link to placeholder image
 2024-02-21 14:04:11 +02:00
Joshua Lochner 5ac17bda83
Add support for OWLv2 models (#579) 2024-02-19 14:57:06 +02:00
Joshua Lochner e9092d2337
Add basic 2D `layer_norm` operator (#588) 2024-02-19 14:53:06 +02:00
Joshua Lochner 351dbed922
Add support for exporting custom models (#556) 
* Add support for exporting custom models

* Require task to be specified

* Pass `trust_remote_code` to all `from_pretrained` calls.

* Add support for `nomic_bert`

Prevents unsupported model warnings
 2024-02-19 14:23:27 +02:00
Joshua Lochner 66d59d79d4
Add remove-background demo (#576) 
* Add remove-background demo

* Open links in new tab

* Add mention of running locally
 2024-02-07 14:31:52 +02:00
Joshua Lochner 41f98b761f [version] Update to 2.15.0 2024-02-06 15:06:50 +02:00
Joshua Lochner dbeb314323
Update `jsdoc-to-markdown` dev dependency (#574) 2024-02-06 14:59:51 +02:00
Joshua Lochner 2bd8c78aec
Support `size_divisbility` after resizing (#569) 2024-02-06 13:10:12 +02:00
Joshua Lochner ae7af0a6c2
Add support for Qwen2 models (#570) 
* Add support for `Qwen2Tokenizer`

* Add qwen-specific tokenizer unit test

* Add support for qwen2 models

* Fix typo
 2024-02-06 12:44:30 +02:00
Joshua Lochner a6f3f88231
Improve unknown model warnings (#554) 
Warn with model type if known
 2024-02-05 17:35:49 +02:00
Joshua Lochner 2b3dcb1863
Use absolute links for example code (#550) 2024-02-01 17:00:56 +02:00
Joshua Lochner 9f877eea95 [version] Update to 2.14.2 2024-01-29 14:02:11 +02:00
dependabot[bot] 9f37036731
Bump vite from 5.0.11 to 5.0.12 in /examples/zero-shot-classification (#548) 
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 5.0.11 to 5.0.12.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v5.0.12/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v5.0.12/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-01-29 13:45:22 +02:00
Joshua Lochner 1edf683e64
Add zero-shot classification demo (#519) 
* Create zero-shot-classification demo

* Update hypothesis template
 2024-01-29 12:29:33 +02:00
Joshua Lochner b1f96a2fc9
Add support for wav2vec2-bert (#544) 
* Add povey window function

* Add `SeamlessM4TFeatureExtractor`

* Add support for wav2vec2-bert models

* Add `SeamlessM4TFeatureExtractor` processor unit tests

* Add pipeline support for `wav2vec2-bert` models

* Update JSDoc
 2024-01-27 17:50:03 +02:00
Joshua Lochner 03f2763a27
Add whitespace pretokenizer (#542) 
* Add `WhitespacePreTokenizer`

* Add unit test for `Whitespace` pretokenizer
 2024-01-27 15:13:20 +02:00
Joshua Lochner 497628ff07
Create depth anything demo (#540) 2024-01-25 17:58:40 +02:00
Joshua Lochner a2fcd110a3 [version] Update to 2.14.1 2024-01-25 15:26:57 +02:00
Joshua Lochner 587adfc193
Add support for Depth Anything (#534) 
* Add support for `DPTImageProcessor`

* Add support for depth anything model

* Update list of `depth_anything` models

* Update processor test model id
 2024-01-25 15:21:51 +02:00
dependabot[bot] 4fb23f2fd1
Bump vite from 4.5.1 to 4.5.2 in /examples/tokenizer-playground (#538) 
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 4.5.1 to 4.5.2.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v4.5.2/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v4.5.2/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-01-25 15:18:54 +02:00
dependabot[bot] 544c2c2f9d
Bump vite from 5.0.6 to 5.0.12 in /examples/semantic-audio-search (#537) 
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 5.0.6 to 5.0.12.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v5.0.12/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v5.0.12/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-01-25 15:18:29 +02:00
dependabot[bot] 9d7fb85167
Bump vite from 4.5.1 to 4.5.2 in /examples/code-completion (#536) 
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 4.5.1 to 4.5.2.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v4.5.2/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v4.5.2/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-01-25 15:18:18 +02:00
Joshua Lochner 4a6addab69
Return empty tokens array if text is empty after normalization (#535) 2024-01-25 14:40:56 +02:00
Joshua Lochner b07336d8f7
Fix typo in tokenizers.js (#518) 2024-01-14 20:39:48 +02:00
Joshua Lochner 935274fd3f
Add SAM demo (#513) 2024-01-11 02:10:38 +02:00
Joshua Lochner 5b5aa4cf6a [version] Update to 2.14.0 2024-01-10 18:30:36 +02:00
dependabot[bot] 3339ed1221
Bump follow-redirects from 1.15.2 to 1.15.4 (#511) 
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.15.2 to 1.15.4.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.15.2...v1.15.4)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-01-10 18:18:39 +02:00
Joshua Lochner cdcbfc125c
Add support for Segment Anything Model (#510) 
* Update SamModel

* Make `AutoModel.from_pretrained` work with SamModel

* Add listed support for SAM (Segment Anything Model)

* Update types of `calculateDimensions`

* Throw error if reading image from tensor with dims.length != 3

* Make SamProcessor input points optional

* Fix type errors

* `let` -> `const`

* `cat` -> `stack`

* Expose `reshape_input_points` in `SamProcessor`

* Add `input_labels` input parameter for SAM

* Add `input_labels` to sam processor

* Update SAM unit tests

* Remove TODOs

* Update JSDoc
 2024-01-10 17:47:21 +02:00
Joshua Lochner 4d1d4d3346
Add support for processing non-square images w/ `ConvNextFeatureExtractor` (#503) 
* Abstract resize function

* Fix tolerance comparison

* Update `ConvNextFeatureExtractor`

* Update ConvNext unit test
 2024-01-10 17:04:21 +02:00
Joshua Lochner f6555dc321
Encode revision in remote URL (#507) 
Required to support accessing PRs on the HF Hub
 2024-01-10 17:03:57 +02:00
Joshua Lochner 07df34ff33 [version] Update to 2.13.4 2024-01-04 19:08:32 +02:00
Joshua Lochner ebd533531a
Add support for cross-encoder models (+fix token type ids) (#501) 
* Formatting

* Update ESM pair template

* Fix token type ids

* Update JSDoc

* Cleanup

* Remove unused `prepare_model_inputs` function

* Move pad and truncate logic to helper functions

* Add static padding/truncation unit tests

* Fix padding/truncation

* Remove unused `add_token_types` function

* Reduce duplication

* `let` -> `const` where possible

* Add cross-encoder models
 2024-01-04 18:37:26 +02:00
Joshua Lochner f3482baa51 [version] Update to 2.13.3 2024-01-04 02:14:10 +02:00