70be6c7368
Removed `virtual` method in newly sealed class
2023-08-24 17:08:01 +01:00
ebacdb666d
- Moved the lower level state get/set methods onto SafeLLamaContextHandle
...
- Used those methods to add a `Clone` method to SafeLLamaContextHandle
- Simplified `LLamaContext` by using the new methods
- Sealed `LLamaContext` and `LLamaEmbedder`
2023-08-24 17:03:27 +01:00
93f24f8a51
Switched to properly typed `Encoding` property
2023-08-24 00:09:00 +01:00
759ae26f36
Merge branch 'master' into grammar_basics
2023-08-22 14:06:57 +01:00
a9e6f21ab8
- Creating and destroying contexts in the stateless executor, saving memory. It now uses zero memory when not inferring!
...
- Passing encoding in the `IModelParams`, which reduces how often encoding needs to be passed around
2023-08-22 01:30:13 +01:00
4738c26299
- Reduced context size of test, to speed it up
...
- Removed some unnecessary `ToArray` calls
- Initial pass on LLamaStatelessExecutor, the context overflow management is broken but I think I found where it's ported from
2023-08-22 01:28:28 +01:00
ae8ef17a4a
- Added various convenience overloads to `LLamaContext.Eval`
...
- Converted `SafeLLamaContextHandle` to take a `ReadOnlySpan` for Eval, narrower type better represents what's really needed
2023-08-22 01:28:28 +01:00
64416ca23c
- Created a slightly nicer way to create grammar (from `IReadOnlyList<IReadOnlyList<LLamaGrammarElement>>`)
...
- Integrated grammar into sampling
- Added a test for the grammar sampling
2023-08-17 19:29:15 +01:00
f5a260926f
Renamed `EmbeddingCount` to `EmbeddingSize` in higher level class
2023-08-13 01:10:09 +01:00
479ff57853
Renamed `EmbeddingCount` to `EmbeddingSize`
2023-08-13 01:10:09 +01:00
d0a7a8fcd6
- Cleaned up disposal in LLamaContext
...
- sealed some classes not intended to be extended
2023-08-13 01:10:08 +01:00
4d741d24f2
Marked old `LLamaContext` constructor obsolete
2023-08-13 01:10:08 +01:00
20bdc2ec6f
- Apply LoRA in `LLamaWeights.LoadFromFile`
...
- Sanity checking that weights are not disposed when creating a context from them
- Further simplified `Utils.InitLLamaContextFromModelParams`
2023-08-13 01:10:08 +01:00
e2fe08a9a2
Added a higher level `LLamaWeights` wrapper around `SafeLlamaModelHandle`
2023-08-13 01:10:08 +01:00
f3511e390f
WIP demonstrating changes to support multi-context. You can see this in use in `TalkToYourself`, along with notes on what still needs improving.
...
The biggest single change is renaming `LLamaModel` to `LLamaContext`
2023-08-13 01:10:08 +01:00
Martin Evans