# SafeLLamaContextHandle
Namespace: LLama.Native
A safe wrapper around a llama_context
```csharp
public sealed class SafeLLamaContextHandle : SafeLLamaHandleBase, System.IDisposable
```
Inheritance [Object](https://docs.microsoft.com/en-us/dotnet/api/system.object) → [CriticalFinalizerObject](https://docs.microsoft.com/en-us/dotnet/api/system.runtime.constrainedexecution.criticalfinalizerobject) → [SafeHandle](https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.safehandle) → [SafeLLamaHandleBase](./llama.native.safellamahandlebase.md) → [SafeLLamaContextHandle](./llama.native.safellamacontexthandle.md)
Implements [IDisposable](https://docs.microsoft.com/en-us/dotnet/api/system.idisposable)
## Properties
### **VocabCount**
Total number of tokens in vocabulary of this model
```csharp
public int VocabCount { get; }
```
#### Property Value
[Int32](https://docs.microsoft.com/en-us/dotnet/api/system.int32)
### **ContextSize**
Total number of tokens in the context
```csharp
public uint ContextSize { get; }
```
#### Property Value
[UInt32](https://docs.microsoft.com/en-us/dotnet/api/system.uint32)
### **EmbeddingSize**
Dimension of embedding vectors
```csharp
public int EmbeddingSize { get; }
```
#### Property Value
[Int32](https://docs.microsoft.com/en-us/dotnet/api/system.int32)
### **BatchSize**
Get the maximum batch size for this context
```csharp
public uint BatchSize { get; }
```
#### Property Value
[UInt32](https://docs.microsoft.com/en-us/dotnet/api/system.uint32)
### **ModelHandle**
Get the model which this context is using
```csharp
public SafeLlamaModelHandle ModelHandle { get; }
```
#### Property Value
[SafeLlamaModelHandle](./llama.native.safellamamodelhandle.md)
### **IsInvalid**
```csharp
public bool IsInvalid { get; }
```
#### Property Value
[Boolean](https://docs.microsoft.com/en-us/dotnet/api/system.boolean)
### **IsClosed**
```csharp
public bool IsClosed { get; }
```
#### Property Value
[Boolean](https://docs.microsoft.com/en-us/dotnet/api/system.boolean)
## Constructors
### **SafeLLamaContextHandle()**
```csharp
public SafeLLamaContextHandle()
```
## Methods
### **ReleaseHandle()**
```csharp
protected bool ReleaseHandle()
```
#### Returns
[Boolean](https://docs.microsoft.com/en-us/dotnet/api/system.boolean)
### **Create(SafeLlamaModelHandle, LLamaContextParams)**
Create a new llama_state for the given model
```csharp
public static SafeLLamaContextHandle Create(SafeLlamaModelHandle model, LLamaContextParams lparams)
```
#### Parameters
`model` [SafeLlamaModelHandle](./llama.native.safellamamodelhandle.md)
`lparams` [LLamaContextParams](./llama.native.llamacontextparams.md)
#### Returns
[SafeLLamaContextHandle](./llama.native.safellamacontexthandle.md)
#### Exceptions
[RuntimeError](./llama.exceptions.runtimeerror.md)
### **GetLogits()**
Token logits obtained from the last call to llama_decode
The logits for the last token are stored in the last row
Can be mutated in order to change the probabilities of the next token.
Rows: n_tokens
Cols: n_vocab
```csharp
public Span GetLogits()
```
#### Returns
[Span<Single>](https://docs.microsoft.com/en-us/dotnet/api/system.span-1)
### **GetLogitsIth(Int32)**
Logits for the ith token. Equivalent to: llama_get_logits(ctx) + i*n_vocab
```csharp
public Span GetLogitsIth(int i)
```
#### Parameters
`i` [Int32](https://docs.microsoft.com/en-us/dotnet/api/system.int32)
#### Returns
[Span<Single>](https://docs.microsoft.com/en-us/dotnet/api/system.span-1)
### **Tokenize(String, Boolean, Boolean, Encoding)**
Convert the given text into tokens
```csharp
public LLamaToken[] Tokenize(string text, bool add_bos, bool special, Encoding encoding)
```
#### Parameters
`text` [String](https://docs.microsoft.com/en-us/dotnet/api/system.string)
The text to tokenize
`add_bos` [Boolean](https://docs.microsoft.com/en-us/dotnet/api/system.boolean)
Whether the "BOS" token should be added
`special` [Boolean](https://docs.microsoft.com/en-us/dotnet/api/system.boolean)
Allow tokenizing special and/or control tokens which otherwise are not exposed and treated as plaintext.
`encoding` [Encoding](https://docs.microsoft.com/en-us/dotnet/api/system.text.encoding)
Encoding to use for the text
#### Returns
[LLamaToken[]](./llama.native.llamatoken.md)
#### Exceptions
[RuntimeError](./llama.exceptions.runtimeerror.md)
### **TokenToSpan(LLamaToken, Span<Byte>)**
Convert a single llama token into bytes
```csharp
public uint TokenToSpan(LLamaToken token, Span dest)
```
#### Parameters
`token` [LLamaToken](./llama.native.llamatoken.md)
Token to decode
`dest` [Span<Byte>](https://docs.microsoft.com/en-us/dotnet/api/system.span-1)
A span to attempt to write into. If this is too small nothing will be written
#### Returns
[UInt32](https://docs.microsoft.com/en-us/dotnet/api/system.uint32)
The size of this token. **nothing will be written** if this is larger than `dest`
### **Decode(LLamaBatch)**
```csharp
public DecodeResult Decode(LLamaBatch batch)
```
#### Parameters
`batch` [LLamaBatch](./llama.native.llamabatch.md)
#### Returns
[DecodeResult](./llama.native.decoderesult.md)
Positive return values does not mean a fatal error, but rather a warning:
- 0: success
- 1: could not find a KV slot for the batch (try reducing the size of the batch or increase the context)
- < 0: error
### **Decode(List<LLamaToken>, LLamaSeqId, LLamaBatch, Int32&)**
Decode a set of tokens in batch-size chunks.
```csharp
internal ValueTuple Decode(List tokens, LLamaSeqId id, LLamaBatch batch, Int32& n_past)
```
#### Parameters
`tokens` [List<LLamaToken>](https://docs.microsoft.com/en-us/dotnet/api/system.collections.generic.list-1)
`id` [LLamaSeqId](./llama.native.llamaseqid.md)
`batch` [LLamaBatch](./llama.native.llamabatch.md)
`n_past` [Int32&](https://docs.microsoft.com/en-us/dotnet/api/system.int32&)
#### Returns
[ValueTuple<DecodeResult, Int32>](https://docs.microsoft.com/en-us/dotnet/api/system.valuetuple-2)
A tuple, containing the decode result and the number of tokens that have not been decoded yet.
### **GetStateSize()**
Get the size of the state, when saved as bytes
```csharp
public ulong GetStateSize()
```
#### Returns
[UInt64](https://docs.microsoft.com/en-us/dotnet/api/system.uint64)
### **GetState(Byte*, UInt64)**
Get the raw state of this context, encoded as bytes. Data is written into the `dest` pointer.
```csharp
public ulong GetState(Byte* dest, ulong size)
```
#### Parameters
`dest` [Byte*](https://docs.microsoft.com/en-us/dotnet/api/system.byte*)
Destination to write to
`size` [UInt64](https://docs.microsoft.com/en-us/dotnet/api/system.uint64)
Number of bytes available to write to in dest (check required size with `GetStateSize()`)
#### Returns
[UInt64](https://docs.microsoft.com/en-us/dotnet/api/system.uint64)
The number of bytes written to dest
#### Exceptions
[ArgumentOutOfRangeException](https://docs.microsoft.com/en-us/dotnet/api/system.argumentoutofrangeexception)
Thrown if dest is too small
### **GetState(IntPtr, UInt64)**
Get the raw state of this context, encoded as bytes. Data is written into the `dest` pointer.
```csharp
public ulong GetState(IntPtr dest, ulong size)
```
#### Parameters
`dest` [IntPtr](https://docs.microsoft.com/en-us/dotnet/api/system.intptr)
Destination to write to
`size` [UInt64](https://docs.microsoft.com/en-us/dotnet/api/system.uint64)
Number of bytes available to write to in dest (check required size with `GetStateSize()`)
#### Returns
[UInt64](https://docs.microsoft.com/en-us/dotnet/api/system.uint64)
The number of bytes written to dest
#### Exceptions
[ArgumentOutOfRangeException](https://docs.microsoft.com/en-us/dotnet/api/system.argumentoutofrangeexception)
Thrown if dest is too small
### **SetState(Byte*)**
Set the raw state of this context
```csharp
public ulong SetState(Byte* src)
```
#### Parameters
`src` [Byte*](https://docs.microsoft.com/en-us/dotnet/api/system.byte*)
The pointer to read the state from
#### Returns
[UInt64](https://docs.microsoft.com/en-us/dotnet/api/system.uint64)
Number of bytes read from the src pointer
### **SetState(IntPtr)**
Set the raw state of this context
```csharp
public ulong SetState(IntPtr src)
```
#### Parameters
`src` [IntPtr](https://docs.microsoft.com/en-us/dotnet/api/system.intptr)
The pointer to read the state from
#### Returns
[UInt64](https://docs.microsoft.com/en-us/dotnet/api/system.uint64)
Number of bytes read from the src pointer
### **SetSeed(UInt32)**
Set the RNG seed
```csharp
public void SetSeed(uint seed)
```
#### Parameters
`seed` [UInt32](https://docs.microsoft.com/en-us/dotnet/api/system.uint32)
### **SetThreads(UInt32, UInt32)**
Set the number of threads used for decoding
```csharp
public void SetThreads(uint threads, uint threadsBatch)
```
#### Parameters
`threads` [UInt32](https://docs.microsoft.com/en-us/dotnet/api/system.uint32)
n_threads is the number of threads used for generation (single token)
`threadsBatch` [UInt32](https://docs.microsoft.com/en-us/dotnet/api/system.uint32)
n_threads_batch is the number of threads used for prompt and batch processing (multiple tokens)
### **KvCacheGetDebugView(Int32)**
Get a new KV cache view that can be used to debug the KV cache
```csharp
public LLamaKvCacheViewSafeHandle KvCacheGetDebugView(int maxSequences)
```
#### Parameters
`maxSequences` [Int32](https://docs.microsoft.com/en-us/dotnet/api/system.int32)
#### Returns
[LLamaKvCacheViewSafeHandle](./llama.native.llamakvcacheviewsafehandle.md)
### **KvCacheCountCells()**
Count the number of used cells in the KV cache (i.e. have at least one sequence assigned to them)
```csharp
public int KvCacheCountCells()
```
#### Returns
[Int32](https://docs.microsoft.com/en-us/dotnet/api/system.int32)
### **KvCacheCountTokens()**
Returns the number of tokens in the KV cache (slow, use only for debug)
If a KV cell has multiple sequences assigned to it, it will be counted multiple times
```csharp
public int KvCacheCountTokens()
```
#### Returns
[Int32](https://docs.microsoft.com/en-us/dotnet/api/system.int32)
### **KvCacheClear()**
Clear the KV cache
```csharp
public void KvCacheClear()
```
### **KvCacheRemove(LLamaSeqId, LLamaPos, LLamaPos)**
Removes all tokens that belong to the specified sequence and have positions in [p0, p1)
```csharp
public void KvCacheRemove(LLamaSeqId seq, LLamaPos p0, LLamaPos p1)
```
#### Parameters
`seq` [LLamaSeqId](./llama.native.llamaseqid.md)
`p0` [LLamaPos](./llama.native.llamapos.md)
`p1` [LLamaPos](./llama.native.llamapos.md)
### **KvCacheSequenceCopy(LLamaSeqId, LLamaSeqId, LLamaPos, LLamaPos)**
Copy all tokens that belong to the specified sequence to another sequence. Note that
this does not allocate extra KV cache memory - it simply assigns the tokens to the
new sequence
```csharp
public void KvCacheSequenceCopy(LLamaSeqId src, LLamaSeqId dest, LLamaPos p0, LLamaPos p1)
```
#### Parameters
`src` [LLamaSeqId](./llama.native.llamaseqid.md)
`dest` [LLamaSeqId](./llama.native.llamaseqid.md)
`p0` [LLamaPos](./llama.native.llamapos.md)
`p1` [LLamaPos](./llama.native.llamapos.md)
### **KvCacheSequenceKeep(LLamaSeqId)**
Removes all tokens that do not belong to the specified sequence
```csharp
public void KvCacheSequenceKeep(LLamaSeqId seq)
```
#### Parameters
`seq` [LLamaSeqId](./llama.native.llamaseqid.md)
### **KvCacheSequenceAdd(LLamaSeqId, LLamaPos, LLamaPos, Int32)**
Adds relative position "delta" to all tokens that belong to the specified sequence
and have positions in [p0, p1. If the KV cache is RoPEd, the KV data is updated
accordingly
```csharp
public void KvCacheSequenceAdd(LLamaSeqId seq, LLamaPos p0, LLamaPos p1, int delta)
```
#### Parameters
`seq` [LLamaSeqId](./llama.native.llamaseqid.md)
`p0` [LLamaPos](./llama.native.llamapos.md)
`p1` [LLamaPos](./llama.native.llamapos.md)
`delta` [Int32](https://docs.microsoft.com/en-us/dotnet/api/system.int32)
### **KvCacheSequenceDivide(LLamaSeqId, LLamaPos, LLamaPos, Int32)**
Integer division of the positions by factor of `d > 1`.
If the KV cache is RoPEd, the KV data is updated accordingly.
p0 < 0 : [0, p1]
p1 < 0 : [p0, inf)
```csharp
public void KvCacheSequenceDivide(LLamaSeqId seq, LLamaPos p0, LLamaPos p1, int divisor)
```
#### Parameters
`seq` [LLamaSeqId](./llama.native.llamaseqid.md)
`p0` [LLamaPos](./llama.native.llamapos.md)
`p1` [LLamaPos](./llama.native.llamapos.md)
`divisor` [Int32](https://docs.microsoft.com/en-us/dotnet/api/system.int32)