ba49ea2991
Removed hardcoded paths from projects, modified Runtime.targets to exclude missing binaries
2023-08-28 19:53:34 +01:00
2022b82947
Added binaries generated by this action: https://github.com/SciSharp/LLamaSharp/actions/runs/6002797872/job/16279896150
...
Based on this version: 6b73ef1201
2023-08-28 19:48:31 +01:00
a5d742b72c
Fix Tokenize of new line, Remove space inserts
2023-08-28 11:57:50 +12:00
31287b5e6e
Rewritten TokenToSpan/TokenToString to better fit the new way it's done in llama.cpp with a few different options:
...
- Just convert it to a `string`, nice and simple
- Write the bytes to a `Span<byte>` no allocations
- Write the chars to a `StringBuilder` potentially no allocations
2023-08-27 00:15:56 +01:00
0c98ae1955
Passing ctx to `llama_token_nl(_ctx)`
2023-08-27 00:15:55 +01:00
6ffa28f964
Removed `LLAMA_MAX_DEVICES` (not used)
2023-08-27 00:14:40 +01:00
2056078aef
Initial changes required for GGUF support
2023-08-27 00:14:40 +01:00
826c6aaec3
cleaned up higher level code using the sampling API:
...
- Fixed multiple enumeration
- Fixed newline penalisation
2023-08-26 21:47:41 +01:00
cf4754db44
Removed unnecessary parameters from some low level sampler methods
2023-08-26 21:38:24 +01:00
f70525fec2
Two small improvements to the native sampling API:
...
- Modified `llama_sample_token_mirostat` and `llama_sample_token_mirostat_v2` to take `ref float` instead of as a `float*`. Less pointers is always good.
- Modified `llama_sample_repetition_penalty` and `llama_sample_frequency_and_presence_penalties` to take pointers instead of arrays. This allows the use non non allocating types (e.g. Span) instead of arrays
- Modified higher level API to accept `Memory<int>` instead of `int[]`, which can be used to reduce allocations at call sites
2023-08-26 01:25:48 +01:00
a911b77dec
Various minor changes, resolving about 100 ReSharper code quality warnings
2023-08-24 23:15:53 +01:00
5a6c6de0dc
Merge pull request #115 from martindevans/model_params_record
...
ModelsParams record class
2023-08-24 22:54:23 +01:00
70be6c7368
Removed `virtual` method in newly sealed class
2023-08-24 17:08:01 +01:00
ebacdb666d
- Moved the lower level state get/set methods onto SafeLLamaContextHandle
...
- Used those methods to add a `Clone` method to SafeLLamaContextHandle
- Simplified `LLamaContext` by using the new methods
- Sealed `LLamaContext` and `LLamaEmbedder`
2023-08-24 17:03:27 +01:00
77aa5fa0d0
Added `JsonConverter` attribute, so System.Text.Json serialization is seamless
2023-08-24 16:17:49 +01:00
df80ec9161
Merge pull request #97 from martindevans/embedder_tests
...
Embedder Test
2023-08-24 02:08:39 +01:00
058c4e84b1
Rewritten LLamaEmbedder to use `LLamaContext` instead of the lower level handles
2023-08-24 01:14:12 +01:00
829f32b27d
- Added `Obsolete` attributes to the entire `OldVersion` namespace, so they can be removed in the future
...
- Minor changes to cleanup some of the compiler warnings
2023-08-24 00:59:32 +01:00
ee772a2921
added `using` statement instead of full qualification
2023-08-24 00:24:16 +01:00
93f24f8a51
Switched to properly typed `Encoding` property
2023-08-24 00:09:00 +01:00
45b01d5a78
Improved type conversion
...
Type conversion is now done in the property rather than the utils class and uses the System.Convert class to ensure consistency.
2023-08-23 19:36:14 +01:00
29df14cd9c
Converted ModelParams into a `record` class. This has several advantages:
...
- Equality, hashing etc all implemented automatically
- Default values are defined in just one place (the properties) instead of the constructor as well
- Added test to ensure that serialization works properly
2023-08-23 00:58:25 +01:00
2830e5755c
- Applied a lot of minor R# code quality suggestions. Lots of unnecessary imports removed.
...
- Deleted `NativeInfo` (internal class, not used anywhere)
2023-08-22 23:20:13 +01:00
854532c08e
Merge pull request #112 from martindevans/classifier_free_guidance
...
Added native symbol for CFG
2023-08-22 18:35:13 +01:00
4b7d718551
Added native symbol for CFG
2023-08-22 17:11:49 +01:00
8f0b52eb09
Re-renaming some arguments to allow for easy deserialization from appsettings.json.
2023-08-22 09:09:22 -07:00
9fc17f3136
Fixed unit tests
2023-08-22 14:16:20 +01:00
759ae26f36
Merge branch 'master' into grammar_basics
2023-08-22 14:06:57 +01:00
a9e6f21ab8
- Creating and destroying contexts in the stateless executor, saving memory. It now uses zero memory when not inferring!
...
- Passing encoding in the `IModelParams`, which reduces how often encoding needs to be passed around
2023-08-22 01:30:13 +01:00
e7b217f462
Fixed out of context logic
2023-08-22 01:28:28 +01:00
4738c26299
- Reduced context size of test, to speed it up
...
- Removed some unnecessary `ToArray` calls
- Initial pass on LLamaStatelessExecutor, the context overflow management is broken but I think I found where it's ported from
2023-08-22 01:28:28 +01:00
ae8ef17a4a
- Added various convenience overloads to `LLamaContext.Eval`
...
- Converted `SafeLLamaContextHandle` to take a `ReadOnlySpan` for Eval, narrower type better represents what's really needed
2023-08-22 01:28:28 +01:00
592a80840b
renamed some arguments in ModelParams constructor so that classcan be serialized easily
2023-08-19 15:55:19 -07:00
64416ca23c
- Created a slightly nicer way to create grammar (from `IReadOnlyList<IReadOnlyList<LLamaGrammarElement>>`)
...
- Integrated grammar into sampling
- Added a test for the grammar sampling
2023-08-17 19:29:15 +01:00
0294bb1303
Some of the basics of the grammar API
2023-08-17 19:28:17 +01:00
Rinne
62331852bc
Merge pull request #90 from martindevans/proposal_multi_context
...
Multi Context
2023-08-17 21:59:05 +08:00
10f88ebd0e
Potential fix for .Net Framework issues ( #103 )
...
* Added a bool to sbyte Utils convertor
As an attempt to avoid using any MarshalAs attribute for .Net Framework support this Utils method will take in a bool value and return a 1 for true or 0 for false sbyte.
* Changed all bool "MarshalAs" types to sbytes
Changed all previous BOOL types with "MarshalAs" attributes to SBYTEs and changed all the setters of them to use the Utils.BoolToSignedByte() convertor method.
* Fixed Utils bool convertor & added sbyte to bool
Improved the Utils bool convertor just casting an sbyte value to get rid of the unneeded sbyte array and added an sbyte to bool convertor to convert back the way to a C# bool assuming any positive value above 0 is a bool and no bools are packed in the single byte integer.
* bool to & from sbyte conversions via properties
All 1byte bools are now handled where they "sit", via public properties which perform the conversions to keep all external data able to communicate as it did before.
2023-08-16 00:09:52 +01:00
7ebff89f68
Merge pull request #101 from martindevans/llama_sample_classifier_free_guidance
...
llama_sample_classifier_free_guidance
2023-08-13 23:21:21 +01:00
6c84accce8
Added `llama_sample_classifier_free_guidance` method from native API
2023-08-13 23:14:53 +01:00
afe559ef1c
Added comments to `Logger` and fixed some nullability warnings
2023-08-13 01:29:33 +01:00
6473f8d5e5
Temporarily added a `Console.WriteLine` into the test, to print the embedding vector for "cat" in CI
2023-08-13 01:10:09 +01:00
1b35be2e0c
Added some additional basic tests
2023-08-13 01:10:09 +01:00
f5a260926f
Renamed `EmbeddingCount` to `EmbeddingSize` in higher level class
2023-08-13 01:10:09 +01:00
479ff57853
Renamed `EmbeddingCount` to `EmbeddingSize`
2023-08-13 01:10:09 +01:00
d0a7a8fcd6
- Cleaned up disposal in LLamaContext
...
- sealed some classes not intended to be extended
2023-08-13 01:10:08 +01:00
4d741d24f2
Marked old `LLamaContext` constructor obsolete
2023-08-13 01:10:08 +01:00
20bdc2ec6f
- Apply LoRA in `LLamaWeights.LoadFromFile`
...
- Sanity checking that weights are not disposed when creating a context from them
- Further simplified `Utils.InitLLamaContextFromModelParams`
2023-08-13 01:10:08 +01:00
e2fe08a9a2
Added a higher level `LLamaWeights` wrapper around `SafeLlamaModelHandle`
2023-08-13 01:10:08 +01:00
fda7e1c038
Fixed mirostat/mirostate
2023-08-13 01:10:08 +01:00
f3511e390f
WIP demonstrating changes to support multi-context. You can see this in use in `TalkToYourself`, along with notes on what still needs improving.
...
The biggest single change is renaming `LLamaModel` to `LLamaContext`
2023-08-13 01:10:08 +01:00
Martin Evans
Rinne