Joshua Lochner 060ac830fc Add M2M100 tokenizer (Closes #235) (#250) ... * Add `M2M100Tokenizer` * Allow `added_tokens` list to be empty * Apply hot-fix for issue in HF's `M2M100Tokenizer` * Skip M2M100 tokenizer tests for now TODO: Remove when https://github.com/huggingface/transformers/pull/25478 is merged * Fix `_build_translation_inputs` for `M2M100Tokenizer` * Add example code in JSDoc for `TranslationPipeline` * Update supported_models.py		2023-08-14 17:22:20 +02:00
..
extra	Add support for computing CLIP image and text embeddings separately (Closes #148) (#227)	2023-08-01 14:01:04 +02:00
convert.py	Add support for computing CLIP image and text embeddings separately (Closes #148) (#227)	2023-08-01 14:01:04 +02:00
requirements.txt	Whisper word-level timestamps (#184)	2023-07-09 23:21:43 +02:00
supported_models.py	Add M2M100 tokenizer (Closes #235) (#250)	2023-08-14 17:22:20 +02:00