Class ModelOptions

This class defines a flexible set of parameters that control how text is generated by a language model. These options are entirely optional, but tuning them allows for precise control over randomness, token filtering, sampling behavior, and performance tuning.

Inheritance

object

ModelOptions

Namespace: Glitch9.AIDevKit

Assembly: .dll

Syntax

public class ModelOptions

Properties

FrequencyPenalty

Penalizes tokens that occur frequently across the generated content. Range: -2.0–2.0 (typical: 0–1.0).

Declaration

public float? FrequencyPenalty { get; set; }

Property Value

Type	Description
float?

LogitBias

Biases specific tokens by ID. Use to influence token selection. Key = token ID (as string), Value = bias (-100 to 100, 0 = no bias).

Declaration

public Dictionary<string, double> LogitBias { get; set; }

Property Value

Type	Description
Dictionary<string, double>

Logprobs

OpenAI only. Whether to return log probabilities of the Output tokens or not. If true, returns the log probabilities of each Output token returned in the content of message. This option is currently not available on the gpt-4-vision-preview model. Defaults to 0.

Declaration

public int? Logprobs { get; set; }

Property Value

Type	Description
int?

LowVram

Ollama only. Use VRAM-optimized loading.

Declaration

public bool? LowVram { get; set; }

Property Value

Type	Description
bool?

MainGpu

Ollama only. ID of the main GPU to prioritize.

Declaration

public int? MainGpu { get; set; }

Property Value

Type	Description
int?

MaxTokens

Optional. Maximum number of tokens (range: [1, context_length)).

Declaration

public int? MaxTokens { get; set; }

Property Value

Type	Description
int?

MinP

Minimum probability for token filtering (less common). Range: 0.0–1.0

Declaration

public float? MinP { get; set; }

Property Value

Type	Description
float?

Mirostat

Ollama only. Enables Mirostat sampling (0: off, 1 or 2: enabled).

Declaration

public int? Mirostat { get; set; }

Property Value

Type	Description
int?

MirostatEta

Ollama only. Controls learning rate in Mirostat sampling. Typical: 0.1

Declaration

public float? MirostatEta { get; set; }

Property Value

Type	Description
float?

MirostatTau

Ollama only. ontrols surprise level in Mirostat sampling. Typical: 5.0

Declaration

public float? MirostatTau { get; set; }

Property Value

Type	Description
float?

NumBatch

Number of tokens to process in a single batch.

Declaration

public int? NumBatch { get; set; }

Property Value

Type	Description
int?

NumCtx

Ollama only. Number of context tokens (max sequence length).

Declaration

public int? NumCtx { get; set; }

Property Value

Type	Description
int?

NumGpu

Ollama only. Number of GPUs to use.

Declaration

public int? NumGpu { get; set; }

Property Value

Type	Description
int?

NumKeep

Ollama only. Number of initial tokens to keep from context when truncating.

Declaration

public int? NumKeep { get; set; }

Property Value

Type	Description
int?

NumPredict

Ollama only. Maximum number of tokens to predict (like max_tokens).

Declaration

public int? NumPredict { get; set; }

Property Value

Type	Description
int?

NumThread

Ollama only. Number of CPU threads to use for inference. Typical: number of physical CPU cores.

Declaration

public int? NumThread { get; set; }

Property Value

Type	Description
int?

Numa

Ollama only. Enable NUMA-aware optimization.

Declaration

public bool? Numa { get; set; }

Property Value

Type	Description
bool?

PenalizeNewline

Ollama only. Whether to apply penalties to newline tokens.

Declaration

public bool? PenalizeNewline { get; set; }

Property Value

Type	Description
bool?

PresencePenalty

Penalizes tokens already present in the generated content. Range: -2.0–2.0 (typical: 0–1.0).

Declaration

public float? PresencePenalty { get; set; }

Property Value

Type	Description
float?

RepeatLastN

Ollama only. Number of previous tokens to consider for repetition penalty. Typical: 64–256

Declaration

public int? RepeatLastN { get; set; }

Property Value

Type	Description
int?

RepeatPenalty

Penalizes repetition of recent tokens. Range: 0.0–2.0 (typical: 1.1).

Declaration

public float? RepeatPenalty { get; set; }

Property Value

Type	Description
float?

Seed

Random seed for reproducibility. Set to the same value for deterministic output.

Declaration

public int? Seed { get; set; }

Property Value

Type	Description
int?

Stop

List of strings that, if generated, will stop further generation.

Declaration

public List<string> Stop { get; set; }

Property Value

Type	Description
List<string>

Temperature

Sampling temperature: controls randomness in output. Lower = deterministic, Higher = creative. Range: 0.0–2.0 (typical: 0.7–1.0).

Declaration

public float? Temperature { get; set; }

Property Value

Type	Description
float?

TopA

Top-A sampling: limits the next token selection to the A most probable tokens. Range: 1–100 (typical: 40).

Declaration

public float? TopA { get; set; }

Property Value

Type	Description
float?

TopK

Top-K sampling: limits the next token selection to the K most probable tokens. Range: 1–100 (typical: 40).

Declaration

public int? TopK { get; set; }

Property Value

Type	Description
int?

TopLogprobs

Number of top log probabilities to return per token. Range: 0–20 (if supported).

Declaration

public int? TopLogprobs { get; set; }

Property Value

Type	Description
int?

TopP

Top-P sampling (nucleus sampling): limits the next token selection to a cumulative probability. Range: 0.0–1.0 (typical: 0.9).

Declaration

public float? TopP { get; set; }

Property Value

Type	Description
float?

TypicalP

Ollama only. Typical sampling, alternative to top_p. Range: 0.0–1.0

Declaration

public float? TypicalP { get; set; }

Property Value

Type	Description
float?

UseMlock

Ollama only. Lock model in RAM.

Declaration

public bool? UseMlock { get; set; }

Property Value

Type	Description
bool?

UseMmap

Ollama only. Use memory-mapped files.

Declaration

public bool? UseMmap { get; set; }

Property Value

Type	Description
bool?

VocabOnly

Ollama only. Only load the vocabulary; do not load full model.

Declaration

public bool? VocabOnly { get; set; }

Property Value

Type	Description
bool?