Class RoutedLlm

A BaseLlm implementation that delegates to one of multiple models based on a router function.

Hierarchy (View Summary)

BaseLlm
- RoutedLlm

Constructors

constructor

new RoutedLlm(
    __namedParameters: {
        modelName?: string;
        models: Readonly<Record<string, BaseLlm>> | BaseLlm[];
        router: LlmRouter;
    },
): RoutedLlm
Parameters
- __namedParameters: {
      modelName?: string;
      models: Readonly<Record<string, BaseLlm>> | BaseLlm[];
      router: LlmRouter;
  }
Returns RoutedLlm
Overrides BaseLlm.constructor
- Defined in core/src/models/routed_llm.ts:33

Properties

`Readonly`[BASE_MODEL_SYMBOL]

"[BASE_MODEL_SYMBOL]": true

A unique symbol to identify BaseLlm classes.

`Readonly`model

model: string

`Static` `Readonly`supportedModels

supportedModels: (string | RegExp)[] = []

List of supported models in regex for LlmRegistry.

Accessors

`Protected`trackingHeaders

get trackingHeaders(): Record<string, string>
Returns Record<string, string>
Inherited from BaseLlm.trackingHeaders
- Defined in core/src/models/base_llm.ts:81

Methods

connect

connect(llmRequest: LlmRequest): Promise<BaseLlmConnection>
Creates a live connection to the LLM by delegating to the selected model. This live connection cannot be switched mid-stream, it is tied to the model selected at the time of connection.
Parameters
- llmRequest: LlmRequest
Returns Promise<BaseLlmConnection>
Overrides BaseLlm.connect
- Defined in core/src/models/routed_llm.ts:74

generateContentAsync

generateContentAsync(
llmRequest: LlmRequest,
stream?: boolean,
): AsyncGenerator<LlmResponse, void>
Generates content by delegating to the selected model.
Parameters
- llmRequest: LlmRequest
- Optionalstream: boolean
Returns AsyncGenerator<LlmResponse, void>
Overrides BaseLlm.generateContentAsync
- Defined in core/src/models/routed_llm.ts:59

maybeAppendUserContent

maybeAppendUserContent(llmRequest: LlmRequest): void
Appends a user content, so that model can continue to output.
Parameters
- llmRequest: LlmRequest
  LlmRequest, the request to send to the LLM.
Returns void
Inherited from BaseLlm.maybeAppendUserContent
- Defined in core/src/models/base_llm.ts:95

Class RoutedLlm

Hierarchy (View Summary)

Constructors

constructor

Parameters

Returns RoutedLlm

Properties

Readonly[BASE_MODEL_SYMBOL]

Readonlymodel

Static ReadonlysupportedModels

Accessors

ProtectedtrackingHeaders

Returns Record<string, string>

Methods

connect

Parameters

Returns Promise<BaseLlmConnection>

generateContentAsync

Parameters

Returns AsyncGenerator<LlmResponse, void>

maybeAppendUserContent

Parameters

Returns void

`Readonly`[BASE_MODEL_SYMBOL]

`Readonly`model

`Static` `Readonly`supportedModels

`Protected`trackingHeaders