Commit Graph

1043 Commits

Author SHA1 Message Date
Joshua Lochner 52e20f2daf Update conversion script args 2023-05-31 15:13:19 +02:00
Joshua Lochner eed6a05c5f Update supported_models.py 2023-05-31 15:07:54 +02:00
Joshua Lochner 84719a5fa3 Add `--skip_validation` option to conversion script 2023-05-31 11:59:20 +02:00
Joshua Lochner bcaf7dd155 Remove torch_dtype from config 2023-05-31 02:28:16 +02:00
Joshua Lochner 99c771bf48 Run `generate_tests.py` as module 2023-05-31 02:21:15 +02:00
Joshua Lochner 813b2abe77 Remove unnecessary imports 2023-05-31 02:06:34 +02:00
Joshua Lochner e3f73cd17d Add unit tests for config files 2023-05-31 02:04:58 +02:00
Joshua Lochner 8a4dcbf2ef Add JSDoc to imported Tensor class 2023-05-31 02:04:27 +02:00
Joshua Lochner 3d05083c1a Add config auto class 2023-05-31 02:04:07 +02:00
Joshua Lochner 44caca829f Fix JSDoc for tokenizers 2023-05-31 01:43:50 +02:00
Joshua Lochner db2bcdfd55 Fix masked LM unit tests 2023-05-31 01:43:31 +02:00
Joshua Lochner 40149ebb6b Update supported models 2023-05-30 19:37:34 +02:00
Joshua Lochner 5e53f2cf02 Rename tasks script 2023-05-30 17:13:27 +02:00
Joshua Lochner ec00d4f540 Update default quantization params in conversion script 2023-05-30 14:57:44 +02:00
Joshua Lochner 55d6ef41b3 [docs] Fix numbering 2023-05-29 17:49:46 +02:00
Joshua Lochner 80d92cc38b Use error mapping instead of switch block 2023-05-29 17:23:39 +02:00
Joshua Lochner 4feb8b4527 Correctly use default module if present 2023-05-29 17:21:19 +02:00
Joshua Lochner 14c02f8760 Add `sentence-transformers` models to supported models/tasks 2023-05-29 17:19:33 +02:00
Joshua Lochner 331c8700ae Replace `Math.max` with custom `max` function 2023-05-29 16:51:38 +02:00
Joshua Lochner dad26f1c0a Update README.md 2023-05-29 15:46:44 +02:00
Joshua Lochner 8b89766034 Update package-lock.json 2023-05-29 12:51:15 +02:00
Joshua Lochner 9bc1c33b39 [version] Update to 2.0.2 2023-05-29 12:50:30 +02:00
Joshua Lochner 2f2092263d Use versioned links (#Closes #114)
Prevents issues where users copy-paste the import code, and then a future update breaks it.

Also ensures that the default wasm files match the target version
2023-05-29 12:50:17 +02:00
Joshua Lochner ac23bc5e12 Freeze onnxruntime dependencies (#124)
Their latest version has a few issues, particularly with webgpu, and also uses .wasm files which are incompatible with their previous versions.

So, while those issues are sorted out, it's best to freeze their packages to the latest stable version.
2023-05-29 12:41:52 +02:00
Joshua Lochner efb45ba392 Improve cache key
- When accessing remote file, use the url as the cache key
- Also use sensible cache key when running locally (with access to file system)
2023-05-28 23:33:46 +02:00
Joshua Lochner d6ca1eaa90 Remove `cos_sim` function from pipeline
The user should use the `cos_sim` or `dot` functions in utils/maths.js
2023-05-28 18:46:34 +02:00
Joshua Lochner 7c041d84b4 Update feature extraction test cases 2023-05-28 18:45:45 +02:00
Joshua Lochner 33b37061c3 Improve feature extraction pipeline
Now works for models that are not specifically used for feature extraction
2023-05-26 12:06:43 +02:00
Joshua Lochner 85f76b3285 Use generated request ID for caching model files
Fixes issue when requesting different revisions or quantizations of models
2023-05-26 10:57:41 +02:00
Joshua Lochner 3edbbdaffb Implement tensor ops
- `sum`
- `norm`
- `normalize`
- `stride`
- `squeeze`
- `unsqueeze`
- `mean_pooling`
2023-05-26 10:49:51 +02:00
Joshua Lochner d7c353d812 [version] Update to 2.0.1 2023-05-20 15:47:07 +02:00
Joshua Lochner 63240244fc [package.json] Update keywords 2023-05-20 15:46:44 +02:00
Joshua Lochner e71a2eaa16 Build demo website after release 2023-05-20 15:37:36 +02:00
Joshua Lochner 1e2d1e2eb4 Add tokenizer test cases with numbers 2023-05-19 10:40:54 +02:00
Joshua Lochner 11ae173957 Update Bert pretokenizer regex
Should match the rust implementation: b4fcc9ce6e/tokenizers/src/pre_tokenizers/bert.rs (L11)

- Removes whitespace
- Splits on unicode punctuation and certain ascii characters
2023-05-19 10:39:08 +02:00
Joshua Lochner eaf8f6ba6c Fix Bert tokenizer regex for numbers 2023-05-19 08:49:50 +02:00
Joshua Lochner 4a6b8ccc41 [demo] Fix token-classification (Closes #116) 2023-05-19 08:04:45 +02:00
Joshua Lochner 8d8f511b34 Update zero-shot image classification test case 2023-05-17 18:40:35 +02:00
Joshua Lochner 112420174b Release version 2.0.0 2023-05-17 15:14:09 +02:00
Joshua Lochner 75ec68ed8b Create example Node.js application 2023-05-17 12:34:48 +02:00
Joshua Lochner 20b6e6d2e4 Update version to 2.0.0-alpha.4 2023-05-17 00:04:04 +02:00
Joshua Lochner a53da6bf4a Rename `CustomImage` to `RawImage` 2023-05-16 23:02:26 +02:00
Joshua Lochner 815a77dd78 Create test suite for utility functions 2023-05-16 22:35:57 +02:00
Joshua Lochner 7ac9dc71c0 Update processors.js 2023-05-16 22:35:04 +02:00
Joshua Lochner acc53dd8b1 Improve `getMelFilters` function
Minor performance +memory improvements (~20%)
2023-05-16 22:28:38 +02:00
Joshua Lochner 32cb92d073 Only calculate mel filters once (during initialization) 2023-05-16 20:36:36 +02:00
Joshua Lochner f424353c61 Merge branch 'pr/112' 2023-05-16 20:06:32 +02:00
Joshua Lochner 0b403ce8e4 Set base path for demo site from environment variable if present 2023-05-16 19:10:02 +02:00
Joshua Lochner be907015e4 Update gh-pages.yml 2023-05-16 19:01:11 +02:00
Joshua Lochner 56e0c06580 Update package-lock.json 2023-05-16 19:00:28 +02:00