* Update transformers.js version
* Use Singleton object in electron tutorial
* Create package-lock.json
* Remove models folder
* Remove step for copying models to local folder
* Add `add_special_tokens` option to tokenizers
* Improve error messages for loading processors
* Add `DonutFeatureExtractor`
* Add `DonutSwinModel` and `MBartForCausalLM` models
* Fix `addPastKeyValues` for `VisionEncoderDecoder` models
* Add `Donut` to list of supported models
* Make encode parameters optional
* Support batched decoder input ids
* Remove unused import
* Add `do_thumbnail` for donut image processing
* Fix `TypeError: decoder_input_ids[i].map is not a function`
* Only pad if width and height specified in size
* Only pad if `pad_size` is defined
* Only cut `decoder_input_ids` if past model output
* Add donut model
* Add example usage to JSDoc for `DonutSwinModel`
* Add support for `DocumentQuestionAnsweringPipeline`
* Add simple document question answering unit test
* Add listed support for document QA pipeline
* Add support for `Blenderbot` models
Closes#37
References #29
* Add support for `BlenderbotTokenizer`
* Add blenderbot to supported models
* Add support for `BlenderbotSmallTokenizer`
* Add custom tests for blenderbot-small
* Add support for `BlenderbotSmall` models
* Update list of supported models
* Improve `addPastKeyValues` function
* Allow skipping of adding encoder past key values
* Cleanup JSDoc
* Store mapping between class and name
* Fix `PretrainedMixin`
* Check seq2seq and vision2seq mappings for possible generate-compatible classes
* Add support for `MinNewTokensLengthLogitsProcessor`
* Add support for `MinLengthLogitsProcessor`
* Fix `generation_config` defaults
* Fix `input_ids_seq_length`
* Add unit tests for generation
* Fix generation parameters test case
* Allow specification of multiple `eos_token_ids`
* Add `CodeLlamaTokenizer`
* Add `codellama` for testing
* Update default quantization settings
* Refactor `PretrainedModel`
* Remove unnecessary error message
* Update llama-code-tokenizer test
* Add support for `GPTNeoX` models
* Fix `GPTNeoXPreTrainedModel` config
* Add support for `GPTJ` models
* Add support for `WavLM` models
* Update list of supported models
- CodeLlama
- GPT NeoX
- GPT-J
- WavLM
* Add support for XLM models
* Add support for `ResNet` models
* Add support for `BeiT` models
* Fix casing of `BeitModel`
* Remove duplicate code
* Update variable name
* Remove `ts-ignore`
* Remove unnecessary duplication
* Update demo model sizes
* [demo] Update default summarization parameters
* Update default quantization parameters for new models
* Remove duplication in mapping
* Update list of supported marian models
* Add support for `CamemBERT` models
* Add support for `MBart` models
* Add support for `OPT` models
* Add `MBartTokenizer` and `MBart50Tokenizer`
* Add example of multilingual translation with MBart models
* Add `CamembertTokenizer`
* Add support for `HerBERT` models
* Add support for `XLMTokenizer`
* Fix `fuse_unk` config
* Do not remove duplicate keys for `Unigram` models
See https://huggingface.co/camembert-base for an example of a Unigram tokenizer that has two tokens with the same value (`<unk>`)
* Update HerBERT supported model text
* Update generate_tests.py
* Update list of supported models
* Use enum object instead of classes for model types
Fixes https://github.com/xenova/transformers.js/issues/283
* Add link to issue
* Update dependencies for unit tests
* Add `sentencepiece` as a testing requirement
* Add `protobuf` to test dependency
* Remove duplicated models to test
* Make // @ts-ignore obsolete for _call overrides by respecting LSP
* oops can't be undefined, back to how it was
* Use `...unused` instead to fix LSP errors
* Add support `DeiT` models
* Add `Swin` models for image classification
* Add support for `yolos` models
* Add `YolosFeatureExtractor`
* Remove unused import
* Update list of supported models
* Remove SAM for now
Move SAM support to next release
* Add example code for zero shot image classification
* Add example code for text classification pipeline
* Fix links to custom usage from pipelines docs
Reported on discord https://discord.com/channels/879548962464493619/1142943169068154950/1142943169068154950
* Fix relative links
* Rename .mdx -> .md
GitHub recently changed how mdx files are displayed, breaking a lot of the formatting. So, we just use .md now (same as transformers)
* Add example code for token classification pipeline
* Add example code for fill-mask pipeline
* Add text2text and summarization pipeline examples
* Add example code for image segmentation pipeline
* Remove redundant `@extends Pipeline`
* Add example code for image-to-text pipeline
* Cleanup example code outputs
* Cleanup JSDoc
* Cleanup pipeline example code
* Update codegen example