Skip to content

Agent Platform hosted models for ADK agents

For enterprise-grade scalability, reliability, and integration with Google Cloud's MLOps ecosystem, you can use models deployed to Agent Platform Endpoints. This includes models from Model Garden or your own fine-tuned models.

Integration Method: Pass the full Agent Platform Endpoint resource string (projects/PROJECT_ID/locations/LOCATION/endpoints/ENDPOINT_ID) directly to the model parameter of LlmAgent.

Agent Platform Setup

Ensure your environment is configured for Agent Platform:

  1. Authentication: Use Application Default Credentials (ADC):

    gcloud auth application-default login

  2. Environment Variables: Set your project and location:

    export GOOGLE_CLOUD_PROJECT="YOUR_PROJECT_ID"
export GOOGLE_CLOUD_LOCATION="YOUR_VERTEX_AI_LOCATION" # e.g., us-central1

  3. Enable Agent Platform Backend: Crucially, ensure the google-genai library targets Agent Platform:

    export GOOGLE_GENAI_USE_VERTEXAI=TRUE

Model Garden Deployments

Supported in ADKPython v0.2.0Java v0.1.0

You can deploy various open and proprietary models from the Model Garden to an endpoint.

Example:

from google.adk.agents import LlmAgent
from google.genai import types # For config objects

# --- Example Agent using a Llama 3 model deployed from Model Garden ---

# Replace with your actual Agent Platform Endpoint resource name
llama3_endpoint = "projects/YOUR_PROJECT_ID/locations/us-central1/endpoints/YOUR_LLAMA3_ENDPOINT_ID"

agent_llama3_vertex = LlmAgent(
    model=llama3_endpoint,
    name="llama3_vertex_agent",
    instruction="You are a helpful assistant based on Llama 3, hosted on Agent Platform.",
    generate_content_config=types.GenerateContentConfig(max_output_tokens=2048),
    # ... other agent parameters
)

import com.google.adk.agents.LlmAgent;
import com.google.adk.models.Gemini;
import com.google.genai.types.GenerateContentConfig;

// ...

// Replace with your actual Agent Platform Endpoint resource name
String llama3Endpoint = "projects/YOUR_PROJECT_ID/locations/us-central1/endpoints/YOUR_LLAMA3_ENDPOINT_ID";

LlmAgent agentLlama3Vertex = LlmAgent.builder()
    .model(Gemini.builder()
        .modelName(llama3Endpoint)
        .build())
    .name("llama3_vertex_agent")
    .instruction("You are a helpful assistant based on Llama 3, hosted on Agent Platform.")
    .generateContentConfig(GenerateContentConfig.builder()
        .maxOutputTokens(2048)
        .build())
    // ... other agent parameters
    .build();

Fine-tuned Model Endpoints

Supported in ADKPython v0.2.0Java v0.1.0

Deploying your fine-tuned models (whether based on Gemini or other architectures supported by Agent Platform) results in an endpoint that can be used directly.

Example:

from google.adk.agents import LlmAgent

# --- Example Agent using a fine-tuned Gemini model endpoint ---

# Replace with your fine-tuned model's endpoint resource name
finetuned_gemini_endpoint = "projects/YOUR_PROJECT_ID/locations/us-central1/endpoints/YOUR_FINETUNED_ENDPOINT_ID"

agent_finetuned_gemini = LlmAgent(
    model=finetuned_gemini_endpoint,
    name="finetuned_gemini_agent",
    instruction="You are a specialized assistant trained on specific data.",
    # ... other agent parameters
)

import com.google.adk.agents.LlmAgent;
import com.google.adk.models.Gemini;

// ...

// Replace with your fine-tuned model's endpoint resource name
String finetunedGeminiEndpoint = "projects/YOUR_PROJECT_ID/locations/us-central1/endpoints/YOUR_FINETUNED_ENDPOINT_ID";

LlmAgent agentFinetunedGemini = LlmAgent.builder()
    .model(Gemini.builder()
        .modelName(finetunedGeminiEndpoint)
        .build())
    .name("finetuned_gemini_agent")
    .instruction("You are a specialized assistant trained on specific data.")
    // ... other agent parameters
    .build();

Anthropic Claude on Agent Platform

Supported in ADKPython v0.2.0Java v0.1.0

Some providers, like Anthropic, make their models available directly through Agent Platform.

Example:

Integration Method: Uses the direct model string (e.g., "claude-3-sonnet@20240229"), but requires manual registration within ADK.

Why Registration? ADK's registry automatically recognizes gemini-* strings and standard Agent Platform endpoint strings (projects/.../endpoints/...) and routes them via the google-genai library. For other model types used directly via Agent Platform (like Claude), you must explicitly tell the ADK registry which specific wrapper class (Claude in this case) knows how to handle that model identifier string with the Agent Platform backend.

Setup:

  1. Agent Platform Environment: Ensure the consolidated Agent Platform setup (ADC, Env Vars, GOOGLE_GENAI_USE_VERTEXAI=TRUE) is complete.

  2. Install Provider Library: Install the necessary client library configured for Agent Platform.

    pip install "anthropic[vertex]"

  3. Register Model Class: Add this code near the start of your application, before creating an agent using the Claude model string:

    # Required for using Claude model strings directly via Agent Platform with LlmAgent
from google.adk.models.anthropic_llm import Claude
from google.adk.models.registry import LLMRegistry

LLMRegistry.register(Claude)

from google.adk.agents import LlmAgent
from google.adk.models.anthropic_llm import Claude # Import needed for registration
from google.adk.models.registry import LLMRegistry # Import needed for registration
from google.genai import types

# --- Register Claude class (do this once at startup) ---
LLMRegistry.register(Claude)

# --- Example Agent using Claude 3 Sonnet on Agent Platform ---

# Standard model name for Claude 3 Sonnet on Agent Platform
claude_model_vertexai = "claude-3-sonnet@20240229"

agent_claude_vertexai = LlmAgent(
    model=claude_model_vertexai, # Pass the direct string after registration
    name="claude_vertexai_agent",
    instruction="You are an assistant powered by Claude 3 Sonnet on Agent Platform.",
    generate_content_config=types.GenerateContentConfig(max_output_tokens=4096),
    # ... other agent parameters
)

Integration Method: Directly instantiate the provider-specific model class (e.g., com.google.adk.models.Claude) and configure it with an Agent Platform backend.

Why Direct Instantiation? The Java ADK's LlmRegistry primarily handles Gemini models by default. For third-party models like Claude on Agent Platform, you directly provide an instance of the ADK's wrapper class (e.g., Claude) to the LlmAgent. This wrapper class is responsible for interacting with the model via its specific client library, configured for Agent Platform.

Setup:

  1. Agent Platform Environment:

    • Ensure your Google Cloud project and region are correctly set up.
    • Application Default Credentials (ADC): Make sure ADC is configured correctly in your environment. This is typically done by running gcloud auth application-default login. The Java client libraries use these credentials to authenticate with Agent Platform. Follow the Google Cloud Java documentation on ADC for detailed setup.

  2. Provider Library Dependencies:

    • Third-Party Client Libraries (Often Transitive): The ADK core library often includes the necessary client libraries for common third-party models on Agent Platform (like Anthropic's required classes) as transitive dependencies. This means you might not need to explicitly add a separate dependency for the Anthropic Vertex SDK in your pom.xml or build.gradle.

  3. Instantiate and Configure the Model: When creating your LlmAgent, instantiate the Claude class (or the equivalent for another provider) and configure its VertexBackend.

import com.anthropic.client.AnthropicClient;
import com.anthropic.client.okhttp.AnthropicOkHttpClient;
import com.anthropic.vertex.backends.VertexBackend;
import com.google.adk.agents.LlmAgent;
import com.google.adk.models.Claude; // ADK's wrapper for Claude
import com.google.auth.oauth2.GoogleCredentials;
import java.io.IOException;

// ... other imports

public class ClaudeVertexAiAgent {

    public static LlmAgent createAgent() throws IOException {
        // Model name for Claude 3 Sonnet on Agent Platform (or other versions)
        String claudeModelVertexAi = "claude-3-7-sonnet"; // Or any other Claude model

        // Configure the AnthropicOkHttpClient with the VertexBackend
        AnthropicClient anthropicClient = AnthropicOkHttpClient.builder()
            .backend(
                VertexBackend.builder()
                    .region("us-east5") // Specify your Agent Platform region
                    .project("your-gcp-project-id") // Specify your GCP Project ID
                    .googleCredentials(GoogleCredentials.getApplicationDefault())
                    .build())
            .build();

        // Instantiate LlmAgent with the ADK Claude wrapper
        LlmAgent agentClaudeVertexAi = LlmAgent.builder()
            .model(new Claude(claudeModelVertexAi, anthropicClient)) // Pass the Claude instance
            .name("claude_vertexai_agent")
            .instruction("You are an assistant powered by Claude 3 Sonnet on Agent Platform.")
            // .generateContentConfig(...) // Optional: Add generation config if needed
            // ... other agent parameters
            .build();

        return agentClaudeVertexAi;
    }

    public static void main(String[] args) {
        try {
            LlmAgent agent = createAgent();
            System.out.println("Successfully created agent: " + agent.name());
            // Here you would typically set up a Runner and Session to interact with the agent
        } catch (IOException e) {
            System.err.println("Failed to create agent: " + e.getMessage());
            e.printStackTrace();
        }
    }
}

Open Models on Agent Platform

Supported in ADKPython v0.1.0Java v0.1.0

Agent Platform offers a curated selection of open-source models, such as Meta Llama, through Model-as-a-Service (MaaS). These models are accessible via managed APIs, allowing you to deploy and scale without managing the underlying infrastructure. For a full list of available options, see the Agent Platform open models for MaaS documentation.

You can use the LiteLLM library to access open models like Meta's Llama on Agent Platform MaaS

Integration Method: Use the LiteLlm wrapper class and set it as the model parameter of LlmAgent. Make sure you go through the LiteLLM model connector for ADK agents documentation on how to use LiteLLM in ADK

Setup:

  1. Agent Platform Environment: Ensure the consolidated Agent Platform setup (ADC, Env Vars, GOOGLE_GENAI_USE_VERTEXAI=TRUE) is complete.

  2. Install LiteLLM: 

    pip install litellm

Example:

from google.adk.agents import LlmAgent
from google.adk.models.lite_llm import LiteLlm

# --- Example Agent using Meta's Llama 4 Scout ---
agent_llama_vertexai = LlmAgent(
    model=LiteLlm(model="vertex_ai/meta/llama-4-scout-17b-16e-instruct-maas"), # LiteLLM model string format
    name="llama4_agent",
    instruction="You are a helpful assistant powered by Llama 4 Scout.",
    # ... other agent parameters
)