Joshua Lochner
52e20f2daf
Update conversion script args
2023-05-31 15:13:19 +02:00
Joshua Lochner
eed6a05c5f
Update supported_models.py
2023-05-31 15:07:54 +02:00
Joshua Lochner
84719a5fa3
Add `--skip_validation` option to conversion script
2023-05-31 11:59:20 +02:00
Joshua Lochner
bcaf7dd155
Remove torch_dtype from config
2023-05-31 02:28:16 +02:00
Joshua Lochner
99c771bf48
Run `generate_tests.py` as module
2023-05-31 02:21:15 +02:00
Joshua Lochner
813b2abe77
Remove unnecessary imports
2023-05-31 02:06:34 +02:00
Joshua Lochner
e3f73cd17d
Add unit tests for config files
2023-05-31 02:04:58 +02:00
Joshua Lochner
8a4dcbf2ef
Add JSDoc to imported Tensor class
2023-05-31 02:04:27 +02:00
Joshua Lochner
3d05083c1a
Add config auto class
2023-05-31 02:04:07 +02:00
Joshua Lochner
44caca829f
Fix JSDoc for tokenizers
2023-05-31 01:43:50 +02:00
Joshua Lochner
db2bcdfd55
Fix masked LM unit tests
2023-05-31 01:43:31 +02:00
Joshua Lochner
40149ebb6b
Update supported models
2023-05-30 19:37:34 +02:00
Joshua Lochner
5e53f2cf02
Rename tasks script
2023-05-30 17:13:27 +02:00
Joshua Lochner
ec00d4f540
Update default quantization params in conversion script
2023-05-30 14:57:44 +02:00
Joshua Lochner
55d6ef41b3
[docs] Fix numbering
2023-05-29 17:49:46 +02:00
Joshua Lochner
80d92cc38b
Use error mapping instead of switch block
2023-05-29 17:23:39 +02:00
Joshua Lochner
4feb8b4527
Correctly use default module if present
2023-05-29 17:21:19 +02:00
Joshua Lochner
14c02f8760
Add `sentence-transformers` models to supported models/tasks
2023-05-29 17:19:33 +02:00
Joshua Lochner
331c8700ae
Replace `Math.max` with custom `max` function
2023-05-29 16:51:38 +02:00
Joshua Lochner
dad26f1c0a
Update README.md
2023-05-29 15:46:44 +02:00
Joshua Lochner
8b89766034
Update package-lock.json
2023-05-29 12:51:15 +02:00
Joshua Lochner
9bc1c33b39
[version] Update to 2.0.2
2023-05-29 12:50:30 +02:00
Joshua Lochner
2f2092263d
Use versioned links (#Closes #114 )
...
Prevents issues where users copy-paste the import code, and then a future update breaks it.
Also ensures that the default wasm files match the target version
2023-05-29 12:50:17 +02:00
Joshua Lochner
ac23bc5e12
Freeze onnxruntime dependencies ( #124 )
...
Their latest version has a few issues, particularly with webgpu, and also uses .wasm files which are incompatible with their previous versions.
So, while those issues are sorted out, it's best to freeze their packages to the latest stable version.
2023-05-29 12:41:52 +02:00
Joshua Lochner
efb45ba392
Improve cache key
...
- When accessing remote file, use the url as the cache key
- Also use sensible cache key when running locally (with access to file system)
2023-05-28 23:33:46 +02:00
Joshua Lochner
d6ca1eaa90
Remove `cos_sim` function from pipeline
...
The user should use the `cos_sim` or `dot` functions in utils/maths.js
2023-05-28 18:46:34 +02:00
Joshua Lochner
7c041d84b4
Update feature extraction test cases
2023-05-28 18:45:45 +02:00
Joshua Lochner
33b37061c3
Improve feature extraction pipeline
...
Now works for models that are not specifically used for feature extraction
2023-05-26 12:06:43 +02:00
Joshua Lochner
85f76b3285
Use generated request ID for caching model files
...
Fixes issue when requesting different revisions or quantizations of models
2023-05-26 10:57:41 +02:00
Joshua Lochner
3edbbdaffb
Implement tensor ops
...
- `sum`
- `norm`
- `normalize`
- `stride`
- `squeeze`
- `unsqueeze`
- `mean_pooling`
2023-05-26 10:49:51 +02:00
Joshua Lochner
d7c353d812
[version] Update to 2.0.1
2023-05-20 15:47:07 +02:00
Joshua Lochner
63240244fc
[package.json] Update keywords
2023-05-20 15:46:44 +02:00
Joshua Lochner
e71a2eaa16
Build demo website after release
2023-05-20 15:37:36 +02:00
Joshua Lochner
1e2d1e2eb4
Add tokenizer test cases with numbers
2023-05-19 10:40:54 +02:00
Joshua Lochner
11ae173957
Update Bert pretokenizer regex
...
Should match the rust implementation: b4fcc9ce6e/tokenizers/src/pre_tokenizers/bert.rs (L11)
- Removes whitespace
- Splits on unicode punctuation and certain ascii characters
2023-05-19 10:39:08 +02:00
Joshua Lochner
eaf8f6ba6c
Fix Bert tokenizer regex for numbers
2023-05-19 08:49:50 +02:00
Joshua Lochner
4a6b8ccc41
[demo] Fix token-classification ( Closes #116 )
2023-05-19 08:04:45 +02:00
Joshua Lochner
8d8f511b34
Update zero-shot image classification test case
2023-05-17 18:40:35 +02:00
Joshua Lochner
112420174b
Release version 2.0.0
2023-05-17 15:14:09 +02:00
Joshua Lochner
75ec68ed8b
Create example Node.js application
2023-05-17 12:34:48 +02:00
Joshua Lochner
20b6e6d2e4
Update version to 2.0.0-alpha.4
2023-05-17 00:04:04 +02:00
Joshua Lochner
a53da6bf4a
Rename `CustomImage` to `RawImage`
2023-05-16 23:02:26 +02:00
Joshua Lochner
815a77dd78
Create test suite for utility functions
2023-05-16 22:35:57 +02:00
Joshua Lochner
7ac9dc71c0
Update processors.js
2023-05-16 22:35:04 +02:00
Joshua Lochner
acc53dd8b1
Improve `getMelFilters` function
...
Minor performance +memory improvements (~20%)
2023-05-16 22:28:38 +02:00
Joshua Lochner
32cb92d073
Only calculate mel filters once (during initialization)
2023-05-16 20:36:36 +02:00
Joshua Lochner
f424353c61
Merge branch 'pr/112'
2023-05-16 20:06:32 +02:00
Joshua Lochner
0b403ce8e4
Set base path for demo site from environment variable if present
2023-05-16 19:10:02 +02:00
Joshua Lochner
be907015e4
Update gh-pages.yml
2023-05-16 19:01:11 +02:00
Joshua Lochner
56e0c06580
Update package-lock.json
2023-05-16 19:00:28 +02:00