dnrops
/
transformers.js
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
transformers.js/examples/tokenizer-playground
History
Joshua Lochner bd315529ab
Update tokenizer playground (#717) 
* [tokenizer playground] Add Mistral v3 tokenizer

* Add support for custom tokenizers

* Add llama 3 tokenizer
 		2024-05-10 23:56:28 +02:00
..
src Update tokenizer playground (#717) 2024-05-10 23:56:28 +02:00
.eslintrc.cjs Tokenization improvements (#234) 2023-08-08 12:11:35 +02:00
.gitignore Tokenization improvements (#234) 2023-08-08 12:11:35 +02:00
README.md Tokenization improvements (#234) 2023-08-08 12:11:35 +02:00
index.html Update tokenizer playground (#597) 2024-02-21 14:04:11 +02:00
package-lock.json Update tokenizer playground (#717) 2024-05-10 23:56:28 +02:00
package.json Update tokenizer playground (#717) 2024-05-10 23:56:28 +02:00
postcss.config.js Tokenization improvements (#234) 2023-08-08 12:11:35 +02:00
tailwind.config.js Tokenization improvements (#234) 2023-08-08 12:11:35 +02:00
vite.config.js Tokenization improvements (#234) 2023-08-08 12:11:35 +02:00

README.md

React + Vite

This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.

Currently, two official plugins are available: