scisharp
/
LLamaSharp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
LLamaSharp/LLama.Unittest
History
Martin Evans 51d4411a58 Added two new classes for detokenization tasks: 
- `AntipromptProcessor` accepts chunks of text and returns a value indicating if any antiprompt has been detected.
 - `StreamingTokenDecoder` decodes tokens into text, maintaining some internal state to handle single characters which are encoded as multiple tokens.

Added tests for these classes and updated StatelessExecutor to use them.

Removed most DeTokenize methods, marked the rest as obsolete (should always use a `StreamingTokenDecoder`).
 		2023-10-23 00:33:50 +01:00
..
AssemblyAttributes.cs Disable test parallelism to prevent fix CI 2023-09-03 23:35:53 +01:00
BasicTest.cs - Split parameters into two interfaces 2023-09-30 16:21:18 +01:00
BeamTests.cs Added two new classes for detokenization tasks: 2023-10-23 00:33:50 +01:00
Constants.cs Swapped to llama-7b-chat 2023-09-04 21:26:02 +01:00
DictionaryExtensionsTests.cs - Added unit tests for extension methods 2023-09-22 16:29:50 +01:00
EncodingExtensionsTests.cs - Added `GetCharCountImpl` tests, fixed handling of empty strings 2023-10-14 00:04:12 +01:00
GrammarParserTest.cs - Improved coverage of `GBNFGrammarParser` up to 96% 2023-10-13 23:54:01 +01:00
GrammarTest.cs Minimal changes required to remove non-async inference. 2023-09-14 21:04:14 +01:00
IEnumerableExtensionsTests.cs - Added unit tests for extension methods 2023-09-22 16:29:50 +01:00
IReadOnlyListExtensionsTests.cs - Added unit tests for extension methods 2023-09-22 16:29:50 +01:00
KeyValuePairExtensionsTests.cs - Added unit tests for extension methods 2023-09-22 16:29:50 +01:00
LLama.Unittest.csproj Minimal changes required to remove non-async inference. 2023-09-14 21:04:14 +01:00
LLamaContextTests.cs Added in the `special` parameter to the tokenizer (introduced in https://github.com/ggerganov/llama.cpp/pull/3538) 2023-10-17 23:55:46 +01:00
LLamaEmbedderTests.cs Fixed a few minor warnings 2023-10-19 00:43:50 +01:00
ModelsParamsTests.cs Fixed a few minor warnings 2023-10-19 00:43:50 +01:00
StatelessExecutorTest.cs Merge pull request #185 from martindevans/wip_major_api_change 2023-10-18 20:50:32 +01:00
TextTransformTests.cs - Improved coverage of `GBNFGrammarParser` up to 96% 2023-10-13 23:54:01 +01:00
TokenTests.cs Added two new classes for detokenization tasks: 2023-10-23 00:33:50 +01:00
Usings.cs feat: add the framework and basic usages. 2023-05-10 02:13:41 +08:00