Class LogitBias

Namespace

Glitch9.AIDevKit

Optional. Modify the likelihood of specified tokens appearing in the completion.

Accepts a JSON object that maps tokens (specified by their token ID in the tokenizer) to an associated bias value from -100 to 100. Mathematically, the bias is added to the logits generated by the model prior to sampling.

The exact effect will vary per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token. Defaults to null.

public class LogitBias

Inheritance

object

LogitBias

Properties

BiasMap

Read-only access to the bias map.

public IReadOnlyDictionary<int, int> BiasMap { get; }

Property Value

IReadOnlyDictionary<int, int>

IsEmpty

Whether any logit biases are defined.

public bool IsEmpty { get; }

Property Value

bool

Methods

Clear()

Clear all token biases.

public void Clear()

Get(int)

Get bias value for a specific token. Returns null if not set.

public int? Get(int tokenId)

Parameters

tokenId int

Returns

int?

Remove(int)

Remove bias for a specific token.

public bool Remove(int tokenId)

Parameters

tokenId int

Returns

bool

Set(int, int)

Add or update a bias for a specific token.

public void Set(int tokenId, int bias)

Parameters

tokenId int

Token ID in the tokenizer.

bias int

Bias value (-100 to 100).