a2e29d393c
Swapped `StatelessExecutor` to use `llama_decode`!
...
- Added `logits_i` argument to `Context.ApplyPenalty`
- Added a new exception type for `llama_decode` return code
2024-01-20 21:18:35 +00:00
42be9b136d
Switched form using raw integers, to a `LLamaToken` struct
2024-01-02 20:47:21 +00:00
b34f72a883
- Added `SamplingPipeline` to inference params which overrides all other options with an entirely custom pipeline.
...
- Added a `Sample` method to `LLamaContext` which uses a custom pipeline
- Modified all executors to use the custom pipeline if it exists
2023-12-08 01:02:27 +00:00
48c5039054
Improved test coverage. Discovered some issues:
...
FixedSizeQueue:
- Enqueue would always stop one short of filling the capacity
- Fill would only _replace_ existing items. It was only used in a place where there were not existing items! Removed the method entirely.
LLamaGrammarElement:
- Converted into a `record` struct, removed all of the (now unnecessary) equality stuff.
2023-11-18 02:40:36 +00:00
d743516070
- Added support for the MinP sampler
...
- Cleaned up comments in implementations of `IInferenceParams`
- Removed default values for all parameters in `LLamaContext.Sample` - they're never used and probably _shouldn't_ ever be used
2023-11-12 00:05:18 +00:00
a024d2242e
It works!
...
had to update binary to `b1426`
2023-10-28 21:32:21 +01:00
f1e5a8f995
- Passing the `ILogger` through to every call of `CreateContext`
...
- Passing `ILogger` into executors
2023-10-19 21:09:44 +01:00
4ec9aed47a
Revert LLamasSharp project changes
2023-10-20 08:29:26 +13:00
9b8de007dc
Propagate ILogger
2023-10-04 13:47:08 +13:00
3f80190f85
Minimal changes required to remove non-async inference.
2023-09-14 21:04:14 +01:00
a09aa86324
Merge pull request #153 from martindevans/fix_savestate_OpenOrCreate
...
Changed `OpenOrCreate` to `Create`
2023-09-06 23:03:24 +01:00
f366aa3abe
Changed `OpenOrCreate` to `Create` to fix #151
2023-09-06 22:35:41 +01:00
77bd090150
Simplified `LLamaInteractExecutor` antiprompt matching by using new extension method
2023-09-06 22:26:36 +01:00
33035c82bf
- Removed `LLamaNewlineTokens` from `InteractiveExecutorState`. This is always set in the constructor from the context, so there's no point serializing it.
2023-09-03 18:22:39 +01:00
a5d742b72c
Fix Tokenize of new line, Remove space inserts
2023-08-28 11:57:50 +12:00
31287b5e6e
Rewritten TokenToSpan/TokenToString to better fit the new way it's done in llama.cpp with a few different options:
...
- Just convert it to a `string`, nice and simple
- Write the bytes to a `Span<byte>` no allocations
- Write the chars to a `StringBuilder` potentially no allocations
2023-08-27 00:15:56 +01:00
0c98ae1955
Passing ctx to `llama_token_nl(_ctx)`
2023-08-27 00:15:55 +01:00
829f32b27d
- Added `Obsolete` attributes to the entire `OldVersion` namespace, so they can be removed in the future
...
- Minor changes to cleanup some of the compiler warnings
2023-08-24 00:59:32 +01:00
2830e5755c
- Applied a lot of minor R# code quality suggestions. Lots of unnecessary imports removed.
...
- Deleted `NativeInfo` (internal class, not used anywhere)
2023-08-22 23:20:13 +01:00
759ae26f36
Merge branch 'master' into grammar_basics
2023-08-22 14:06:57 +01:00
4738c26299
- Reduced context size of test, to speed it up
...
- Removed some unnecessary `ToArray` calls
- Initial pass on LLamaStatelessExecutor, the context overflow management is broken but I think I found where it's ported from
2023-08-22 01:28:28 +01:00
64416ca23c
- Created a slightly nicer way to create grammar (from `IReadOnlyList<IReadOnlyList<LLamaGrammarElement>>`)
...
- Integrated grammar into sampling
- Added a test for the grammar sampling
2023-08-17 19:29:15 +01:00
fda7e1c038
Fixed mirostat/mirostate
2023-08-13 01:10:08 +01:00
f3511e390f
WIP demonstrating changes to support multi-context. You can see this in use in `TalkToYourself`, along with notes on what still needs improving.
...
The biggest single change is renaming `LLamaModel` to `LLamaContext`
2023-08-13 01:10:08 +01:00
270c6d55ef
Merge pull request #88 from martindevans/fix_serialization_nan
...
Fix serialization error due to NaN
2023-08-08 14:04:18 +01:00
b5de3ee5aa
Fixed some final mentions of "mirostate" instead of "mirostat"
2023-08-07 21:12:56 +01:00
0e5e00e300
Moved `TokenToString` from Utils into `SafeLLamaContextHandle` (thin wrappers around the same method in `SafeLlamaModelHandle`)
2023-08-07 15:15:34 +01:00
cd3cf2b77d
- Moved tokenization from `Utils.Tokenize` into `SafeLLamaContextHandle.Tokenize`, one less thing in `Utils`.
...
- Also refactored it to return an `int[]` instead of an `IEnumerable<int>`, solving the "multiple enumeration" problems at the source!
2023-08-07 15:13:24 +01:00
bac9cba01a
InferenceParams abstractions
2023-08-06 11:03:45 +12:00
c64507cb41
Correctly passing through mu value to mirostate instead of resetting it every time.
2023-07-30 00:15:52 +01:00
Yaohui Liu
6c400e64c2
docs: publiash documentation 0.4.
2023-06-20 02:38:57 +08:00
Yaohui Liu
2eb2d6df83
test: add 9 examples of the new version.
2023-06-19 22:09:58 +08:00
762fd7c1ae
Fixed a typo in FixedSizeQueue
2023-06-15 22:00:37 +02:00
Yaohui Liu
a3b8186f20
feat: support save and load chat session.
2023-06-12 18:31:37 +08:00
Yaohui Liu
3bf74ec9b9
feat: add chat session for refactored code.
2023-06-12 02:47:25 +08:00
Yaohui Liu
908b79e855
feat: add stateless executor.
2023-06-11 22:39:31 +08:00
Yaohui Liu
e603a09137
fix: state loading and saving not working.
2023-06-11 09:13:30 +08:00
Yaohui Liu
5679e08718
feat: add ILLamaExecutor.InferAsync.
2023-06-11 05:44:21 +08:00
Yaohui Liu
264fb9a706
refactor: LLamaModel and LLamaExecutor.
2023-06-10 18:37:58 +08:00
Martin Evans
Yaohui Liu