Interface IComputerUseActionHandler

Handler interface for executing computer use actions dispatched by AI agents. Keys are passed as strings from the implementation (e.g., "Ctrl+Shift+P", "Enter", "Meta").

Namespace: Glitch9.AIDevKit.Agents
Assembly: Glitch9.AIDevKit.Agent.dll
Syntax
public interface IComputerUseActionHandler

Properties

| Edit this page View Source

ScreenshotPathType

Declaration
ScreenshotPathType ScreenshotPathType { get; }
Property Value
Type Description
ScreenshotPathType

Methods

| Edit this page View Source

OnKeyboardKeysPressedAsync(List<string>, CancellationToken)

Simulates pressing a sequence of keyboard keys (e.g., "Ctrl", "Shift", "P").

Declaration
UniTask OnKeyboardKeysPressedAsync(List<string> keys, CancellationToken ct)
Parameters
Type Name Description
List<string> keys

List of key names to press in order.
CancellationToken ct

Cancellation token.
Returns
Type Description
UniTask
| Edit this page View Source

OnKeyboardTypedAsync(string, CancellationToken)

Simulates typing the given text string as keyboard input.

Declaration
UniTask OnKeyboardTypedAsync(string text, CancellationToken ct)
Parameters
Type Name Description
string text

The text to type.
CancellationToken ct

Cancellation token.
Returns
Type Description
UniTask
| Edit this page View Source

OnMouseClickAsync(MouseButton, Vector2, CancellationToken)

Simulates a mouse button click at the specified screen position.

Declaration
UniTask OnMouseClickAsync(MouseButton button, Vector2 position, CancellationToken ct)
Parameters
Type Name Description
MouseButton button

The mouse button to click.
Vector2 position

The screen position of the click.
CancellationToken ct

Cancellation token.
Returns
Type Description
UniTask
| Edit this page View Source

OnMouseDoubleClickAsync(Vector2, CancellationToken)

Simulates a mouse double-click at the specified screen position.

Declaration
UniTask OnMouseDoubleClickAsync(Vector2 position, CancellationToken ct)
Parameters
Type Name Description
Vector2 position

The screen position of the double-click.
CancellationToken ct

Cancellation token.
Returns
Type Description
UniTask
| Edit this page View Source

OnMouseDragAsync(List<Vector2>, CancellationToken)

Simulates a mouse drag operation along the specified path of screen positions.

Declaration
UniTask OnMouseDragAsync(List<Vector2> path, CancellationToken ct)
Parameters
Type Name Description
List<Vector2> path

Ordered list of screen positions to drag through.
CancellationToken ct

Cancellation token.
Returns
Type Description
UniTask
| Edit this page View Source

OnMouseMoveAsync(Vector2, CancellationToken)

Moves the mouse cursor to the specified screen position.

Declaration
UniTask OnMouseMoveAsync(Vector2 position, CancellationToken ct)
Parameters
Type Name Description
Vector2 position

The target screen position.
CancellationToken ct

Cancellation token.
Returns
Type Description
UniTask
| Edit this page View Source

OnMouseScrollAsync(Vector2, Vector2, CancellationToken)

Simulates a mouse scroll wheel event at the specified screen position.

Declaration
UniTask OnMouseScrollAsync(Vector2 position, Vector2 scroll, CancellationToken ct)
Parameters
Type Name Description
Vector2 position

The screen position of the scroll event.
Vector2 scroll

The scroll delta in X and Y axes.
CancellationToken ct

Cancellation token.
Returns
Type Description
UniTask
| Edit this page View Source

OnTakeScreenshotAsync(CancellationToken)

Takes a screenshot and returns either an uploaded file ID or a cloud storage URL depending on ScreenshotPathType.

Declaration
UniTask<string> OnTakeScreenshotAsync(CancellationToken ct)
Parameters
Type Name Description
CancellationToken ct

Cancellation token.
Returns
Type Description
UniTask<string>

The uploaded file ID if FileId; otherwise a URL string.
| Edit this page View Source

OnWaitAsync(CancellationToken)

Waits for a platform-defined duration before the next action is executed.

Declaration
UniTask OnWaitAsync(CancellationToken ct)
Parameters
Type Name Description
CancellationToken ct

Cancellation token.
Returns
Type Description
UniTask

Extension Methods