51d4411a58
Added two new classes for detokenization tasks:
...
- `AntipromptProcessor` accepts chunks of text and returns a value indicating if any antiprompt has been detected.
- `StreamingTokenDecoder` decodes tokens into text, maintaining some internal state to handle single characters which are encoded as multiple tokens.
Added tests for these classes and updated StatelessExecutor to use them.
Removed most DeTokenize methods, marked the rest as obsolete (should always use a `StreamingTokenDecoder`).
2023-10-23 00:33:50 +01:00
efdf3d630c
- Removed all `TokenToString` methods (it's never correct to use them, because sometimes one single character may be represented by multiple tokens).
...
- Built a new (hacky) `Detokenize` method which handles this
2023-10-22 21:43:36 +01:00
1d0620e634
Created a test that "roundtrips" strings through tokenization. This reveals some flaws with certain characters
2023-10-22 15:28:36 +01:00
e89ca5cc17
Fixed a few minor warnings
2023-10-19 00:43:50 +01:00
d8434ea9d6
Merge pull request #185 from martindevans/wip_major_api_change
...
Major llama.cpp API Change
2023-10-18 20:50:32 +01:00
1f8c94e386
Added in the `special` parameter to the tokenizer (introduced in https://github.com/ggerganov/llama.cpp/pull/3538 )
2023-10-17 23:55:46 +01:00
efb0664df0
- Added new binaries
...
- Fixed stateless executor out-of-context handling
- Fixed token tests
2023-10-17 23:39:41 +01:00
b8f0eff080
- Added `GetCharCountImpl` tests, fixed handling of empty strings
...
- Added ifdef to remove `Deconstruct` extension on everything except `NETSTANDARD2_0`
2023-10-14 00:04:12 +01:00
45118520fa
- Improved coverage of `GBNFGrammarParser` up to 96%
...
- Covered text transforms
- Removed unnecessary non-async transforms
2023-10-13 23:54:01 +01:00
9f694c584c
Further improved grammar parser test coverage (up to 92%)
2023-10-13 02:08:12 +01:00
bff41eef37
Added some more coverage of `GrammarRule`, checking that invalid rules are rejected
2023-10-13 01:36:48 +01:00
2a38808bca
- Added threads to context params, replaced all thread args with `uint?`
...
- Replaced all binaries
2023-10-12 18:49:41 +01:00
669ae47ef7
- Split parameters into two interfaces
...
- params contains a list of loras, instead of just one
2023-09-30 16:21:18 +01:00
9a0a0ae9fe
Removed cloning support
2023-09-30 15:48:26 +01:00
0d40338692
Fixed out-of-context handling in stateless executor
2023-09-29 23:53:07 +01:00
ce1fc51163
Added some more native methods
2023-09-29 16:05:19 +01:00
bca55eace0
Initial changes to match the llama.cpp changes
2023-09-29 01:18:21 +01:00
fe54f6764f
- Added unit tests for extension methods
...
- Removed unused `AddRangeSpan` extension
2023-09-22 16:29:50 +01:00
3f80190f85
Minimal changes required to remove non-async inference.
2023-09-14 21:04:14 +01:00
daf09eae64
Skipping tokenization of empty strings (saves allocating an empty array every time)
2023-09-12 01:03:27 +01:00
bba801f4b7
Added a property to get the KV cache size from a context
2023-09-11 00:10:08 +01:00
d3b8ee988c
Beam Search ( #155 )
...
* Added the low level bindings to beam search.
2023-09-07 19:26:51 +01:00
d0e57a8c92
sealed test class
2023-09-06 20:11:31 +01:00
3f082c6f2c
Fixed naming in tests
2023-09-06 20:09:41 +01:00
614ba40948
- Added a `TokensEndsWithAnyString` extension to `IReadOnlyList<int>` which efficiently checks if a set of tokens ends with one of a set of strings.
...
- Minimal amount of characters converted
- Allocation free
- Added `TokensToSpan` to `SafeLlamaModelHandle` which converts as many tokens as possible into a character span
- Allocation free
2023-09-06 19:44:19 +01:00
821d7f615e
Swapped to llama-7b-chat
2023-09-04 21:26:02 +01:00
21cbecb82d
Disable test parallelism to prevent fix CI
2023-09-03 23:35:53 +01:00
Rinne
4e83e48ad1
Merge pull request #122 from martindevans/gguf
...
Add GGUF support
2023-09-02 11:54:50 +08:00
a70c7170dd
- Created a higher level `Grammar` class which is immutable and contains a list of grammar rules. This is the main "entry point" to the grammar system.
...
- Made all the mechanics of grammar parsing (GBNFGrammarParser, ParseState) internal. Just call `Grammar.Parse("whatever")`.
- Added a `GrammarRule` class which validates elements on construction (this allows constructing grammar without parsing GBNF).
- It should be impossible for a `GrammarRule` to represent an invalid rule.
2023-08-31 00:02:50 +01:00
fb007e5921
Changes to compile in VS Mac + change model to llama2
...
This commit includes changes to compile en VS Mac + changest to use llama2 not codellama.
It includes MacOS binaries in memory and metal
2023-08-30 22:08:29 +02:00
920efaca44
Add original test
2023-08-30 16:48:42 +03:00
2ae1891c13
Bug fixes after running tests.
...
SymbolIds is now SortedDictionary (although I'm not sure it really needs to be) because the test was failing due to expected value being in another order. The C++ data structure if SymbolIds is std::map<std::string, uint32_t> so the items are ordered by key.
2023-08-30 16:18:05 +03:00
0bd495276b
Add initial tests + fix bugs. Still WIP since the test is failing.
2023-08-30 14:10:56 +03:00
ba49ea2991
Removed hardcoded paths from projects, modified Runtime.targets to exclude missing binaries
2023-08-28 19:53:34 +01:00
2022b82947
Added binaries generated by this action: https://github.com/SciSharp/LLamaSharp/actions/runs/6002797872/job/16279896150
...
Based on this version: 6b73ef1201
2023-08-28 19:48:31 +01:00
95dc12dd76
Switched to codellama-7b.gguf in tests (probably temporarily)
2023-08-27 00:15:56 +01:00
0c98ae1955
Passing ctx to `llama_token_nl(_ctx)`
2023-08-27 00:15:55 +01:00
2056078aef
Initial changes required for GGUF support
2023-08-27 00:14:40 +01:00
5a6c6de0dc
Merge pull request #115 from martindevans/model_params_record
...
ModelsParams record class
2023-08-24 22:54:23 +01:00
ebacdb666d
- Moved the lower level state get/set methods onto SafeLLamaContextHandle
...
- Used those methods to add a `Clone` method to SafeLLamaContextHandle
- Simplified `LLamaContext` by using the new methods
- Sealed `LLamaContext` and `LLamaEmbedder`
2023-08-24 17:03:27 +01:00
77aa5fa0d0
Added `JsonConverter` attribute, so System.Text.Json serialization is seamless
2023-08-24 16:17:49 +01:00
634bd3feed
Loosened requirements even more!
2023-08-24 01:55:54 +01:00
9587699a60
loosened requirements even further for CI
2023-08-24 01:48:12 +01:00
bc70358b3e
expanded epsilon enough to allow CI to pass
2023-08-24 01:18:21 +01:00
08501db155
cleaned up basic test a bit
2023-08-24 01:03:03 +01:00
498ed109c1
Fixed dot product based test
2023-08-24 01:03:03 +01:00
35e48691ea
Added a known correct value to the unit tests
2023-08-24 01:03:03 +01:00
dff308e31b
Added a test for the LLamaEmbedder
2023-08-24 01:03:03 +01:00
93f24f8a51
Switched to properly typed `Encoding` property
2023-08-24 00:09:00 +01:00
29df14cd9c
Converted ModelParams into a `record` class. This has several advantages:
...
- Equality, hashing etc all implemented automatically
- Default values are defined in just one place (the properties) instead of the constructor as well
- Added test to ensure that serialization works properly
2023-08-23 00:58:25 +01:00
Martin Evans
Rinne