42be9b136d
Switched form using raw integers, to a `LLamaToken` struct
2024-01-02 20:47:21 +00:00
b868b056f7
Added metadata overrides to `IModelParams`
2023-12-14 02:05:40 +00:00
439d14a061
Updated binaries:
...
- build run: https://github.com/SciSharp/LLamaSharp/actions/runs/7196891440
- commit: 9fb13f9584
2023-12-14 01:23:43 +00:00
b34f72a883
- Added `SamplingPipeline` to inference params which overrides all other options with an entirely custom pipeline.
...
- Added a `Sample` method to `LLamaContext` which uses a custom pipeline
- Modified all executors to use the custom pipeline if it exists
2023-12-08 01:02:27 +00:00
89fef05362
This commit ( 5fe721bdbe) accidentally removed a load of stuff that it shouldn't. Fixed that.
...
Originally from these PRs:
- https://github.com/SciSharp/LLamaSharp/pull/263
- https://github.com/SciSharp/LLamaSharp/pull/259
2023-11-15 01:36:33 +00:00
46ace3ddd7
Add targets in Web project
...
This allow to copy binaries and make the project work.
Update example model in appsettings
2023-11-13 23:26:29 +01:00
e3468d04f0
Merge pull request #277 from martindevans/feature/min_p
...
MinP Sampler
2023-11-13 02:15:52 +00:00
d743516070
- Added support for the MinP sampler
...
- Cleaned up comments in implementations of `IInferenceParams`
- Removed default values for all parameters in `LLamaContext.Sample` - they're never used and probably _shouldn't_ ever be used
2023-11-12 00:05:18 +00:00
97006a214f
Merge remote-tracking branch 'upstream/master' into RuntimeDetection
2023-11-11 20:31:53 +01:00
31244ae691
Merge branch 'master' into YaRN_scaling_parameters
2023-11-11 16:27:42 +00:00
5fe721bdbe
Revert "Merge branch 'pr/268' into RuntimeDetection"
...
This reverts commit 091b8d58b39b2ca9cf8e
2023-11-09 22:13:18 +01:00
db1bc741b0
Modified `ContextSize` in parameters to be nullable. A null value means autodetect from the model.
2023-11-07 19:41:44 +00:00
04ee64a6be
Exposed YaRN scaling parameters in IContextParams
2023-11-06 21:59:18 +00:00
529b06b35b
- Fixed rope frequency/base to use the values in the model by default, instead of always overriding them by default!
2023-10-29 23:59:46 +00:00
c786fb0ec8
Using `IReadOnlyList` instead of `IEnumerable` in `IInferenceParams`
2023-10-28 21:32:23 +01:00
6a4cd506bd
Added a safe `TensorSplitsCollection` to the params which prevents incorrectly setting the `tensor_splits` collection
2023-10-20 14:10:20 +01:00
18b15184ea
Added logger parameter in to LLama.Web context creation
2023-10-19 21:24:02 +01:00
952e77f97b
Remove old parameter
2023-10-20 08:33:27 +13:00
b4b4000342
Merge branch 'master' into upstream_master
...
# Conflicts:
# LLama.Web/Common/ModelOptions.cs
# LLama.Web/Services/ConnectionSessionService.cs
# LLama/LLamaStatelessExecutor.cs
# LLama/LLamaWeights.cs
2023-10-20 08:02:27 +13:00
2a38808bca
- Added threads to context params, replaced all thread args with `uint?`
...
- Replaced all binaries
2023-10-12 18:49:41 +01:00
a8a498dc12
Fix up issues found during testing
2023-10-04 16:32:13 +13:00
9b8de007dc
Propagate ILogger
2023-10-04 13:47:08 +13:00
e2a17d6b6f
Refactor conflicting object name SessionOptions
2023-10-04 13:35:18 +13:00
44f1b91c29
Update Web to support version 0.5.1
2023-10-04 12:57:15 +13:00
c9108f8311
Add service for managing Models and Model Contexts
2023-10-04 10:40:53 +13:00
0f03e8f1a3
Added workaround to LLama.Web and LLama.WebAPI
2023-10-03 21:08:31 +01:00
669ae47ef7
- Split parameters into two interfaces
...
- params contains a list of loras, instead of just one
2023-09-30 16:21:18 +01:00
bca55eace0
Initial changes to match the llama.cpp changes
2023-09-29 01:18:21 +01:00
2056078aef
Initial changes required for GGUF support
2023-08-27 00:14:40 +01:00
93f24f8a51
Switched to properly typed `Encoding` property
2023-08-24 00:09:00 +01:00
759ae26f36
Merge branch 'master' into grammar_basics
2023-08-22 14:06:57 +01:00
a9e6f21ab8
- Creating and destroying contexts in the stateless executor, saving memory. It now uses zero memory when not inferring!
...
- Passing encoding in the `IModelParams`, which reduces how often encoding needs to be passed around
2023-08-22 01:30:13 +01:00
64416ca23c
- Created a slightly nicer way to create grammar (from `IReadOnlyList<IReadOnlyList<LLamaGrammarElement>>`)
...
- Integrated grammar into sampling
- Added a test for the grammar sampling
2023-08-17 19:29:15 +01:00
f3511e390f
WIP demonstrating changes to support multi-context. You can see this in use in `TalkToYourself`, along with notes on what still needs improving.
...
The biggest single change is renaming `LLamaModel` to `LLamaContext`
2023-08-13 01:10:08 +01:00
2c933c57a1
Fixed ModelOptions in Web project
2023-08-06 20:48:35 +01:00
bac9cba01a
InferenceParams abstractions
2023-08-06 11:03:45 +12:00
2a04e31b7d
ModelParams abstraction
2023-08-06 10:44:54 +12:00
3fec7a63c7
Add Instruct and Stateless support
2023-07-23 16:31:28 +12:00
a32a5e4ffe
Decouple connectionId from ModelSession
2023-07-23 12:05:15 +12:00
d9fbd56f10
Strongly type connection status
2023-07-23 11:53:04 +12:00
ef8cf0b283
Add RequestVerificationToken logic fo ajax prefilter, Tidy up js cancel logic
2023-07-23 11:35:39 +12:00
e574d89a40
Send prompt on Enter key
2023-07-23 11:01:07 +12:00
a139423581
Move session management to service, Use ILLamaExecutor in session to make more versatile, scroll bug
2023-07-19 08:55:56 +12:00
1ec59e120a
Move session management to service, Infer cancel support
2023-07-16 21:39:11 +12:00
fd215dce84
Update Readme
2023-07-16 20:05:21 +12:00
21b685649f
Add Readme
2023-07-16 19:55:56 +12:00
0c6b213b0d
Basic ASP.NET Core website example, Interactive started
2023-07-16 19:31:27 +12:00
Martin Evans