scisharp
/
LLamaSharp
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,198 Commits 8 Branches 17 Tags 317 MiB
Commit Graph

139 Commits

Author SHA1 Message Date
SignalRT 4c013aefd4 Disable Llava Embed test on CI 2024-03-26 23:19:32 +01:00
SignalRT e8732efadd Example InteractiveExecutor 
Add an Example and modifications to the interactive executor to enable Llava Models.

Just a preview / demo
 2024-03-26 23:19:32 +01:00
Martin Evans a2d3a847dd
Disabled LLava tests, they're too slow and are crashing CI (#625) 2024-03-25 22:10:41 +00:00
Rinne e3ecc318ff
Merge pull request #612 from xbotter/deps/sk-1.6.2 
Update Semantic Kernel & Kernel Memory Package
 2024-03-19 09:34:28 +08:00
xbotter a019b5cc24
📝 Update LLamaSharpChatCompletion and LLama.Unittest 
- Updated LLamaSharpChatCompletion class in LLama.SemanticKernel/ChatCompletion/LLamaSharpChatCompletion.cs
  - Changed the type of the "_model" field from "StatelessExecutor" to "ILLamaExecutor"
  - Updated the constructor to accept an "ILLamaExecutor" parameter instead of a "StatelessExecutor" parameter
- Updated LLamaSharpChatCompletion class in LLama.SemanticKernel/LLamaSharp.SemanticKernel.csproj

- Updated LLama.Unittest project in LLama.Unittest/LLama.Unittest.csproj
  - Added a "PackageReference" for "Moq" version 4.20.70
- Added ExtensionMethodsTests class in LLama.Unittest/SemanticKernel/ExtensionMethodsTests.cs
  - Added tests for the "ToLLamaSharpChatHistory" and "ToLLamaSharpInferenceParams" extension methods
- Added LLamaSharpChatCompletionTests class in LLama.Unittest/SemanticKernel/LLamaSharpChatCompletionTests.cs
  - Added tests for the LLamaSharpChatCompletion class

ℹ️ The LLamaSharpChatCompletion class in the LLama.SemanticKernel project has been updated to use the ILLamaExecutor interface instead of the StatelessExecutor class. This change allows for better abstraction and flexibility in the implementation of the LLamaSharpChatCompletion class. The LLamaSharpChatCompletion class is responsible for providing chat completion functionality in the LLamaSharp project. The LLama.Unittest project has also been updated to include tests for the LLamaSharpChatCompletion class and the extension methods used by the class.
 2024-03-18 21:49:52 +08:00
jlsantiago 3b2836eac4
Llava api (#563) 
* Add llava_binaries, update all binaries to make the test

* Llava API + LlavaTest

Preliminary

* First prototype of Load + Unit Test

* Temporary run test con branch LlavaAPI

* Disable Embed test to review the rest of the test

* Restore Embedding test

* Use BatchThread to eval image embeddings

Test Threads default value to ensure it doesn´t produce problems.

* Rename test file

* Update action versions

* Test only one method, no release embeddings

* Revert "Test only one method, no release embeddings"

This reverts commit 264e176dccc9cd0be318b800ae5e102a4635d01c.

* Correct API call

* Only test llava related functionality

* Cuda and Cblast binaries

* Restore build policy

* Changes related with code review

* Add SafeHandles

* Set overwrite to upload-artifact@v4

* Revert to upload-artifact@v3

* revert to upload-artifact@v3
 2024-03-13 22:10:44 +00:00
Martin Evans 8ac1634233
Removed `llama_eval`. It is going to be completely removed in the next version of llama.cpp (#553) 2024-02-28 21:41:39 +00:00
Martin Evans a0731db944
Added tests checking that memory is freed properly (#551) 2024-02-28 17:12:24 +00:00
dependabot[bot] 070969b23f
build(deps): bump coverlet.collector from 6.0.0 to 6.0.1 (#540) 
Bumps [coverlet.collector](https://github.com/coverlet-coverage/coverlet) from 6.0.0 to 6.0.1.
- [Release notes](https://github.com/coverlet-coverage/coverlet/releases)
- [Commits](https://github.com/coverlet-coverage/coverlet/compare/v6.0.0...v6.0.1)

---
updated-dependencies:
- dependency-name: coverlet.collector
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-02-26 14:08:27 +00:00
dependabot[bot] 7fbf979fe0
build(deps): bump xunit.runner.visualstudio from 2.5.6 to 2.5.7 
Bumps [xunit.runner.visualstudio](https://github.com/xunit/visualstudio.xunit) from 2.5.6 to 2.5.7.
- [Release notes](https://github.com/xunit/visualstudio.xunit/releases)
- [Commits](https://github.com/xunit/visualstudio.xunit/compare/2.5.6...2.5.7)

---
updated-dependencies:
- dependency-name: xunit.runner.visualstudio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-02-19 13:37:28 +00:00
dependabot[bot] 939d288384
build(deps): bump xunit from 2.6.6 to 2.7.0 
Bumps [xunit](https://github.com/xunit/xunit) from 2.6.6 to 2.7.0.
- [Commits](https://github.com/xunit/xunit/compare/2.6.6...2.7.0)

---
updated-dependencies:
- dependency-name: xunit
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-02-19 06:30:01 +00:00
Martin Evans c7d0dc915a Assorted small changes to clean up some code warnings 2024-02-17 23:07:10 +00:00
Martin Evans d47b6afe4d Normalizing embeddings in `LLamaEmbedder`. As is done in llama.cpp: 2891c8aa9a/examples/embedding/embedding.cpp (L92) 2024-02-13 02:09:35 +00:00
dependabot[bot] 58b6b927c6
build(deps): bump Microsoft.NET.Test.Sdk from 17.8.0 to 17.9.0 
Bumps [Microsoft.NET.Test.Sdk](https://github.com/microsoft/vstest) from 17.8.0 to 17.9.0.
- [Release notes](https://github.com/microsoft/vstest/releases)
- [Changelog](https://github.com/microsoft/vstest/blob/main/docs/releases.md)
- [Commits](https://github.com/microsoft/vstest/compare/v17.8.0...v17.9.0)

---
updated-dependencies:
- dependency-name: Microsoft.NET.Test.Sdk
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-02-12 06:11:00 +00:00
Martin Evans df38d73c79 Switched to `Q3_K_S` for unit test model, instead of `Q4`. This is almost 1gb smaller, and _may_ make the macos tests less flakey. 2024-02-07 02:36:25 +00:00
Martin Evans 15a98b36d8 Updated everything to work with llama.cpp ce32060198b7e2d6a13a9b8e1e1369e3c295ae2a 2024-02-01 16:35:05 +00:00
Martin Evans c9c8cd0d62 - Swapped embeddings generator to use `llama_decode` 
- Modified `GetEmbeddings` method to be async
 2024-01-31 20:28:53 +00:00
Martin Evans 9b995510d6 Removed all setters in `IModelParams` and `IContextParams`, allowing implementations to be immutable. 2024-01-31 17:51:50 +00:00
Martin Evans 96c26c25f5
Merge pull request #445 from martindevans/stateless_executor_llama_decode 
Swapped `StatelessExecutor` to use `llama_decode`!
 2024-01-23 03:02:51 +00:00
dependabot[bot] 3d4c3c5509
build(deps): bump xunit from 2.6.5 to 2.6.6 
Bumps [xunit](https://github.com/xunit/xunit) from 2.6.5 to 2.6.6.
- [Commits](https://github.com/xunit/xunit/compare/2.6.5...2.6.6)

---
updated-dependencies:
- dependency-name: xunit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-01-22 06:10:13 +00:00
Martin Evans a2e29d393c Swapped `StatelessExecutor` to use `llama_decode`! 
- Added `logits_i` argument to `Context.ApplyPenalty`
 - Added a new exception type for `llama_decode` return code
 2024-01-20 21:18:35 +00:00
Martin Evans 99969e538e - Removed some unused `eval` methods. 
- Added a `DecodeAsync` overload which runs the work in a task
 - Replaced some `NativeHandle` usage in `BatchedDecoding` with higher level equivalents.
 - Made the `LLamaBatch` grow when token capacity is exceeded, removing the need to manage token capacity externally.
 2024-01-20 02:38:45 +00:00
Martin Evans 1472704e12 Added a test with examples of troublesome strings from 0.9.1 2024-01-16 15:02:23 +00:00
Martin Evans 2ea2048b78 - Added a test for tokenizing just a new line (reproduce issue https://github.com/SciSharp/LLamaSharp/issues/430) 
- Properly displaying `LLamaToken`
 - Removed all tokenisation code in `SafeLLamaContextHandle` - just pass it all through to the `SafeLlamaModelHandle`
 - Improved `SafeLlamaModelHandle` tokenisation:
   - Renting an array, for one less allocation
   - Not using `&tokens[0]` to take a pointer to an array, this is redundant and doesn't work on empty arrays
 2024-01-12 15:32:59 +00:00
Martin Evans 98635a0d5a Fixed decoding of large tokens (over 16 bytes) in streaming text decoder 2024-01-09 17:18:27 +00:00
Martin Evans 402a110a3a
Merge pull request #404 from martindevans/switched_to_LLamaToken_struct 
LLamaToken Struct
 2024-01-09 01:18:13 +00:00
dependabot[bot] 4a6f07943c
build(deps): bump xunit from 2.6.4 to 2.6.5 
Bumps [xunit](https://github.com/xunit/xunit) from 2.6.4 to 2.6.5.
- [Commits](https://github.com/xunit/xunit/compare/2.6.4...2.6.5)

---
updated-dependencies:
- dependency-name: xunit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-01-08 06:14:06 +00:00
Martin Evans eebe4cb120 Added a new test (commented out for now) which reproduces the issue reported in #394 2024-01-08 00:52:39 +00:00
Martin Evans 82727c4414 Removed collection expressions from test 2024-01-02 21:12:38 +00:00
Martin Evans 42be9b136d Switched form using raw integers, to a `LLamaToken` struct 2024-01-02 20:47:21 +00:00
dependabot[bot] 57fdac493a
build(deps): bump xunit.runner.visualstudio from 2.5.5 to 2.5.6 
Bumps [xunit.runner.visualstudio](https://github.com/xunit/visualstudio.xunit) from 2.5.5 to 2.5.6.
- [Release notes](https://github.com/xunit/visualstudio.xunit/releases)
- [Commits](https://github.com/xunit/visualstudio.xunit/compare/2.5.5...2.5.6)

---
updated-dependencies:
- dependency-name: xunit.runner.visualstudio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-12-25 21:28:44 +00:00
dependabot[bot] 58e5a84d54
build(deps): bump xunit from 2.6.3 to 2.6.4 
Bumps [xunit](https://github.com/xunit/xunit) from 2.6.3 to 2.6.4.
- [Commits](https://github.com/xunit/xunit/compare/2.6.3...2.6.4)

---
updated-dependencies:
- dependency-name: xunit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-12-25 06:18:18 +00:00
Martin Evans f0d7468b22
Merge pull request #356 from xbotter/deps/sk-rc3 
bump sk to 1.0.1 & km to 0.18
 2023-12-21 17:24:19 +00:00
Martin Evans a2bae178fa Added a `Metadata` property to `LLamaWeights` 2023-12-20 15:45:24 +00:00
Martin Evans db7ecf5a43 Added a method to create a clone of a grammar instance 2023-12-15 23:01:05 +00:00
Martin Evans 199b4418c3 Fixed JSON serialization for ModelParams 2023-12-14 20:24:22 +00:00
Martin Evans b868b056f7 Added metadata overrides to `IModelParams` 2023-12-14 02:05:40 +00:00
xbotter 8766fb1b03
Merge branch 'deps/sk-rc3' of https://github.com/xbotter/LLamaSharp into deps/sk-rc3 2023-12-14 09:47:37 +08:00
xbotter 213b4be723
bump sk-1.0.0-rc4 2023-12-14 09:47:32 +08:00
xbotter ce20b30e06
Merge branch 'SciSharp:master' into deps/sk-rc3 2023-12-14 09:28:06 +08:00
Martin Evans e6148c952e Fixed encoding of `Encoding` 2023-12-12 15:50:32 +00:00
xbotter df66d7e0c6
Upgrade unittest target framework to .net8 2023-12-12 19:08:34 +08:00
Martin Evans d87d654a34
Merge pull request #348 from martindevans/new_object_based_sampling_pipeline 
Custom Sampling Pipelines
 2023-12-11 21:40:53 +00:00
dependabot[bot] 85dc43dde0
build(deps): bump xunit from 2.6.2 to 2.6.3 
Bumps [xunit](https://github.com/xunit/xunit) from 2.6.2 to 2.6.3.
- [Commits](https://github.com/xunit/xunit/compare/2.6.2...2.6.3)

---
updated-dependencies:
- dependency-name: xunit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-12-11 14:26:14 +00:00
xbotter 13a312b4ec
update sk to 1.0.0-rc3 & km to 0.18 2023-12-11 19:39:01 +08:00
dependabot[bot] 8fb4476813
build(deps): bump xunit.runner.visualstudio from 2.5.4 to 2.5.5 
Bumps [xunit.runner.visualstudio](https://github.com/xunit/visualstudio.xunit) from 2.5.4 to 2.5.5.
- [Release notes](https://github.com/xunit/visualstudio.xunit/releases)
- [Commits](https://github.com/xunit/visualstudio.xunit/compare/2.5.4...2.5.5)

---
updated-dependencies:
- dependency-name: xunit.runner.visualstudio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-12-11 06:20:24 +00:00
Martin Evans 835958398c - Removed the object wrappers and configurable pipeline, they can be better written in code. 
- Added BaseSamplingPipeline which provides a base impl of `ISamplingPipeline`
 - Added `DefaultSamplingPipeline` which mimics normal llama.cpp sampling
 2023-12-08 16:25:13 +00:00
Martin Evans 3afc007499 - Added "protected" logits, instead of the awkward save/load mechanism 
- Added an example usage to one of the tests
 2023-12-08 01:17:24 +00:00
dependabot[bot] 6d86219d71
build(deps): bump xunit.runner.visualstudio from 2.5.3 to 2.5.4 
Bumps [xunit.runner.visualstudio](https://github.com/xunit/visualstudio.xunit) from 2.5.3 to 2.5.4.
- [Release notes](https://github.com/xunit/visualstudio.xunit/releases)
- [Commits](https://github.com/xunit/visualstudio.xunit/compare/2.5.3...2.5.4)

---
updated-dependencies:
- dependency-name: xunit.runner.visualstudio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-11-27 06:17:47 +00:00
Rinne cf4edeac55
Merge pull request #315 from futzy314/fix-ai-request-settings 
Added a converter similar to the Open AI one
 2023-11-24 22:12:55 +08:00