Commit Graph

133 Commits

Author SHA1 Message Date
Joshua Lochner 992f643e2a [version] Update to 2.17.1 2024-04-18 17:25:49 +02:00
Joshua Lochner 642743136e [version] Update to 2.17.0 2024-04-11 01:20:24 +02:00
Joshua Lochner c2c45cb577
Improve support of conversational models (#658)
* Add `return_full_text` option for text-generation models

* [wip] Support chat inputs in text-generation pipeline

* Align return type with python version

* Remove conversational task (moved to text-generation)

* Fix typos
2024-04-11 00:58:50 +02:00
Joshua Lochner d50b3193fb [version] Update to 2.16.1 2024-03-20 16:44:39 +02:00
Joshua Lochner 4cab8eceb6
Add `image-feature-extractor` pipeline (#650)
* Add `image-feature-extractor` pipeline

* Update "Image Feature Extraction" heading
2024-03-20 15:30:27 +02:00
Joshua Lochner a51a61be22
Add support for EfficientNet (#639)
* Add support for efficientnet

* Add `EfficientNetImageProcessor`
2024-03-16 14:49:32 +02:00
Joshua Lochner 314b7f0dc4 [version] Update to 2.16.0 2024-03-07 15:45:22 +02:00
Dave 5bb8d25337
Add StableLM support (#616)
* add stablelm model impl.

* add stablelm mapping

* Update `StableLMPreTrainedModel` class config

* Update stablelm conversion script quantization settings

* Add StableLm to list of supported models

* Remove StableLMForSequenceClassification

will be added in seperate PR

Co-authored-by: Joshua Lochner <admin@xenova.com>

* Remove SequenceClassifierOutputWithPast

will be added in seperate PR

Co-authored-by: Joshua Lochner <admin@xenova.com>

* Remove StableLMForSequenceClassification mapping

will be added in seperate pr

Co-authored-by: Joshua Lochner <admin@xenova.com>

* Update src/models.js

* Naming nit

* Naming nits

* Update scripts/convert.py

* Update src/models.js

---------

Co-authored-by: Joshua Lochner <admin@xenova.com>
2024-03-07 03:05:27 +02:00
Joshua Lochner e072ee6b70
Add support for starcoder2 models (#622) 2024-03-06 13:09:04 +02:00
Joshua Lochner 5f572241b4
Add support for `UniSpeech` and `UniSpeechSat` models (#624)
* Add support for `UniSpeech` and `UniSpeechSat` models

* Remove listed support for unispeech STT models

* Align unispeech w/ existing wav2vec logic
2024-03-06 13:08:28 +02:00
Joshua Lochner 7772d1db0a [version] Update to 2.15.1 2024-02-21 16:26:05 +02:00
Joshua Lochner 5ac17bda83
Add support for OWLv2 models (#579) 2024-02-19 14:57:06 +02:00
Joshua Lochner 41f98b761f [version] Update to 2.15.0 2024-02-06 15:06:50 +02:00
Joshua Lochner ae7af0a6c2
Add support for Qwen2 models (#570)
* Add support for `Qwen2Tokenizer`

* Add qwen-specific tokenizer unit test

* Add support for qwen2 models

* Fix typo
2024-02-06 12:44:30 +02:00
Joshua Lochner 2b3dcb1863
Use absolute links for example code (#550) 2024-02-01 17:00:56 +02:00
Joshua Lochner 9f877eea95 [version] Update to 2.14.2 2024-01-29 14:02:11 +02:00
Joshua Lochner b1f96a2fc9
Add support for wav2vec2-bert (#544)
* Add povey window function

* Add `SeamlessM4TFeatureExtractor`

* Add support for wav2vec2-bert models

* Add `SeamlessM4TFeatureExtractor` processor unit tests

* Add pipeline support for `wav2vec2-bert` models

* Update JSDoc
2024-01-27 17:50:03 +02:00
Joshua Lochner a2fcd110a3 [version] Update to 2.14.1 2024-01-25 15:26:57 +02:00
Joshua Lochner 587adfc193
Add support for Depth Anything (#534)
* Add support for `DPTImageProcessor`

* Add support for depth anything model

* Update list of `depth_anything` models

* Update processor test model id
2024-01-25 15:21:51 +02:00
Joshua Lochner 5b5aa4cf6a [version] Update to 2.14.0 2024-01-10 18:30:36 +02:00
Joshua Lochner cdcbfc125c
Add support for Segment Anything Model (#510)
* Update SamModel

* Make `AutoModel.from_pretrained` work with SamModel

* Add listed support for SAM (Segment Anything Model)

* Update types of `calculateDimensions`

* Throw error if reading image from tensor with dims.length != 3

* Make SamProcessor input points optional

* Fix type errors

* `let` -> `const`

* `cat` -> `stack`

* Expose `reshape_input_points` in `SamProcessor`

* Add `input_labels` input parameter for SAM

* Add `input_labels` to sam processor

* Update SAM unit tests

* Remove TODOs

* Update JSDoc
2024-01-10 17:47:21 +02:00
Joshua Lochner 07df34ff33 [version] Update to 2.13.4 2024-01-04 19:08:32 +02:00
Joshua Lochner f3482baa51 [version] Update to 2.13.3 2024-01-04 02:14:10 +02:00
Joshua Lochner 733f98277d [version] Update to 2.13.2 2024-01-03 16:41:35 +02:00
Joshua Lochner e8d1236c11 [version] Update to 2.13.1 2024-01-03 12:59:13 +02:00
Joshua Lochner 61459e38d8 [version] Update to 2.13.0 2023-12-27 16:27:51 +02:00
Joshua Lochner e2d17b946f
Add support for SigLIP models (#473)
* Add support for SigLIP models

* Skip siglip tokenizer tests

* Move SigLIP-specific zero-shot-image-classification logic to pipeline
2023-12-27 15:49:33 +02:00
Joshua Lochner 9b84d7b4da
Add support for CLIPSeg models (#478)
* Add support for CLIPSeg models

* Update JSDoc
2023-12-27 15:40:17 +02:00
Joshua Lochner 80af1c49bb
Add support for Segformer (#480)
* Add support for Segformer

* Add semantic segmentation unit test

* Update pipelines.test.js
2023-12-27 15:12:07 +02:00
Joshua Lochner 1394f73107
Add support for VITS (multilingual TTS) (#466)
* Add custom VITS tokenizer converter

* Do not decode if expected input_ids is empty

* Update vits tokenizer tests

* Implement `VitsTokenizer`

* Add support for VITS model

* Support VITS through pipeline API

* Update JSDoc

* Add TTS unit test

* Add speecht5 unit test

* Fix typo

* Fix typo

* Update speecht5 model id

* Add note about using quantized speecht5 in unit tests

* Monkey-patch `BigInt64Array` and `BigUint64Array`
2023-12-26 16:34:52 +02:00
Joshua Lochner f5bc758c49
Add support for Table Transformer model (#477) 2023-12-26 15:53:08 +02:00
Joshua Lochner 44b62ed5d4
Add support for DiT models (#474) 2023-12-26 15:51:15 +02:00
Joshua Lochner c91248cf70
Add support for RoFormer models (#464)
* Add `RoFormerTokenizer

* Use `clean_text` in bert normalizer config

* Add control characters test

* Add support for RoFormer models

* Use default label if id2label is not specified

* Update requirements.txt

* Skip roformer tokenizer tests
2023-12-26 15:49:11 +02:00
Joshua Lochner 7636a1c416
Add spaces template link to README (#467) 2023-12-23 19:58:00 +02:00
Joshua Lochner 0bf6e6712f [version] Update to 2.12.1 2023-12-18 23:25:00 +02:00
Joshua Lochner 81aab022ff [version] Update to 2.12.0 2023-12-18 17:04:41 +02:00
Joshua Lochner 6129e45b2b [version] Update to 2.11.0 2023-12-13 15:19:12 +02:00
Joshua Lochner 2de085b6e5
Add support for ChineseCLIP models (#455)
* Update `VitMatteImageProcessor` test comment

* Add support for ChineseCLIP models

* Add chinese-clip to list of supported models

* Sort zero-shot-image-classification results by score (desc)

* Update expected zero-shot image classification output
2023-12-13 14:47:17 +02:00
Joshua Lochner b978ff8ce4
Add support for ViTMatte models (#448)
* Add support for `VitMatte` models

* Add `VitMatteImageProcessor`

* Add `VitMatteImageProcessor` unit test

* Fix typo

* Add example code for `VitMatteForImageMatting`

* Fix JSDoc

* Fix typo
2023-12-13 02:18:36 +02:00
Joshua Lochner 80d22dae7b
Add support for ESM models (#447)
* Add support for ESM models

* Add ESM tokenizer conversion methods

* Add special test cases for ESM tokenizer

* add special tokens in conversion script

* Do not save decoder

* Add special tokens tokenizer test

* Join tokens with space if decoder is null

* Treat all tokens as added tokens

* Use `WhitespaceSplit` pretokenizer

* `<eos>` and `<bos>` are not special tokens

* Update more supported ESM models

* Add `--tokenizer_id` to conversion script

* Add supported models comments
2023-12-13 02:10:27 +02:00
Joshua Lochner 0d2f05def5
Add support for ELECTRA models (#446) 2023-12-12 18:56:36 +02:00
Joshua Lochner 0ffdc8d9ca
Add support for Hubert models (#449) 2023-12-12 18:32:16 +02:00
Joshua Lochner 47b1a873a2
Add support for ConvBERT models (#445)
* Add support for `ConvBERT` models

* Fix `ConvBertTokenizer`

* Fix tokenizer
2023-12-12 17:54:27 +02:00
Joshua Lochner 9308f880c5
Add support for DINOv2 models (#444)
* Add dinov2 models

* Add `BitImageProcessor`

* Update list of supported models
2023-12-12 17:42:48 +02:00
Joshua Lochner 09c760e817
Add support for Phi models (#443) 2023-12-12 17:18:58 +02:00
Joshua Lochner cb8a5961df [version] Update to 2.10.1 2023-12-06 18:48:41 +02:00
Joshua Lochner 374c1052a7
Standardize `HF_ACCESS_TOKEN` -> `HF_TOKEN` (#431) 2023-12-06 18:33:50 +02:00
Joshua Lochner 9a8c664c2c
Documentation improvements (#299)
* Add link to optimum docs for supported architectures

Closes #288

* Refactor `SUPPORTED_MODELS` dict to include task

* Update example model id

* Update list of supported models

* Update generate_tests.py

* Remove requirement of `output_attentions` revision

* Add demo site to examples section (closes #233)

* Fix typo

* Include examples in docs index

* Update github issue templates

* Create config.yml

* Order supported models

* Cleanup

* Update 4_feature-request.yml
2023-12-06 17:01:36 +02:00
Joshua Lochner 57487744e7 [version] Update to 2.10.0 2023-12-05 15:34:53 +02:00
Joshua Lochner c5ed1d70ca
Add support for CLAP (`zero-shot-audio-classification`) and Audio Spectrogram Transformer (`audio-classification`) (#427)
* Add FFT unit tests

* Refactor maths.js and audio.js

* Refactor audio processors

* Add support for AST models

* Add another audio-classification example

* Add audio processing unit tests

* Implement `log_mel='dB'` in `spectrogram` function

* Add `ClapFeatureExtractor`

* Implement `ClapFeatureExtractor` unit tests

* Add support for `CLAP`

* Add `ZeroShotAudioClassificationPipeline`

* Add listed support for  `zero-shot-audio-classification` pipeline tag

* Cleanup

* `let` -> `const`

* Update `mel_filter_bank` unit test

* Add `'Xenova/tiny-random-ClapModel'`

* Add `ClapAudioModelWithProjection` and `ClapTextModelWithProjection`

* Move audio validation to helper function

* Optimize `mel_filter_bank` computation

-30ms

* Update mel filters unit test

* Cleanup

* Optimizations

* Fix jsdoc

* Optimizations

* Add WIP conversion scripts

Will be updated once https://github.com/huggingface/optimum/pull/1552 is merged
2023-12-05 12:17:42 +02:00