Gemini Enterprise plugin

The Gemini Enterprise plugin provides access to Gemini Enterprise, offering capabilities for building, scaling, and governing agents alongside model access, grounding, Vector Search, Model Garden, and evaluation metrics.

Note

While this product is Gemini Enterprise, the Genkit plugin packages (@genkit-ai/vertexai, genkit-vertexai), initializers (vertexAI / VertexAI), and model prefixes (vertexai/) retain vertex naming from the prior brand for backward compatibility.

Getting Started

For simple API key access to Google’s AI models, start with the Google AI plugin. This page covers enterprise features available via the Gemini Enterprise API.

Accessing Google GenAI Models via the Gemini Enterprise API

All languages support accessing Google’s generative AI models (Gemini, Imagen, etc.) via the Gemini Enterprise API with enterprise authentication and features.

The unified Google GenAI plugin provides access to models via the Gemini Enterprise API using the vertexAI initializer:

Basic Model Access

Installation

npm i --save @genkit-ai/google-genai

Configuration

import { genkit } from 'genkit';
import { vertexAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [
    vertexAI({ location: 'us-central1' }), // Regional endpoint
    // vertexAI({ location: 'global' }),      // Global endpoint
  ],
});

Authentication Methods:

• Application Default Credentials (ADC): The standard method for most Gemini Enterprise use cases, especially in production. It uses the credentials from the environment (e.g., service account on GCP, user credentials from gcloud auth application-default login locally). This method requires a Google Cloud Project with billing enabled and the Gemini Enterprise API (aiplatform.googleapis.com) enabled.
• Gemini Enterprise Express Mode: A streamlined way to try out many Gemini Enterprise features using just an API key, without needing to set up billing or full project configurations. This is ideal for quick experimentation and has generous free tier quotas. Learn More about Express Mode.

// Using Gemini Enterprise Express Mode (Easy to start, some limitations)
// Get an API key from the Gemini Enterprise Express Mode setup.
vertexAI({ apiKey: process.env.VERTEX_EXPRESS_API_KEY }),

Note: When using Express Mode, you do not provide projectId and location in the plugin config.

Available Models

The following Gemini models are registered for use with the Gemini Enterprise plugin:

Gemini 3 Series - Latest models with state-of-the-art reasoning and multimodal capabilities:

• gemini-3.8-flash - Most intelligent Flash model, engineered for complex reasoning, coding, and agentic workflows
• gemini-3.1-pro-preview - Preview of the most capable model for complex reasoning and problem solving
• gemini-3.5-flash-lite - Fastest, most cost-effective model for high-throughput execution
• gemini-3.1-flash-image - Fast and efficient image generation and editing
• gemini-3-pro-image - State-of-the-art image generation and editing for complex visual tasks

Basic Usage

import { genkit } from 'genkit';
import { vertexAI } from '@genkit-ai/google-genai';

const ai = genkit({
  plugins: [vertexAI({ location: 'us-central1' })],
});

const response = await ai.generate({
  model: vertexAI.model('gemini-3.1-pro-preview'),
  prompt: 'Explain Gemini Enterprise in simple terms.',
});

console.log(response.text);

Model Configuration (PayGo)

You can specify the payGo option to use Flex or Priority routing in Gemini Enterprise.

const response = await ai.generate({
  model: vertexAI.model('gemini-flash-lite-latest'),
  prompt: 'Explain Gemini Enterprise in simple terms.',
  config: {
    payGo: 'priority', // Can be 'priority', 'priority-only', 'flex', or 'flex-only'
  },
});

Multimodal Input

Gemini models can process multimodal inputs, including images and video. When using videos, you can use videoMetadata to specify specific timestamps or sampling rates.

const response = await ai.generate({
  model: vertexAI.model('gemini-flash-latest'),
  prompt: [
    { text: 'transcribe this video' },
    {
      media: {
        url: 'gs://cloud-samples-data/video/animals.mp4',
        contentType: 'video/mp4',
      },
      metadata: {
        videoMetadata: {
          fps: 0.5,
          startOffset: '3.5s',
          endOffset: '10.2s',
        },
      },
    },
  ],
});

Text Embedding

const embeddings = await ai.embed({
  embedder: vertexAI.embedder('text-embedding-005'),
  content: 'Embed this text.',
});

Image Generation (Imagen)

Available Models:

• virtual-try-on-001

// The virtual-try-on model requires two specific media inputs: the person and the product.
const response = await ai.generate({
  model: vertexAI.model('virtual-try-on-001'),
  prompt: [
    {
      media: { url: `data:image/png;base64,${personImageBase64}`, contentType: 'image/png' },
      metadata: { type: 'personImage' },
    },
    {
      media: { url: `data:image/png;base64,${productImageBase64}`, contentType: 'image/png' },
      metadata: { type: 'productImage' },
    },
  ],
});

const generatedImage = response.media;

Video Generation (Veo)

Generate videos from text prompts or manipulate existing images to create dynamic video content.

Available Models:

• veo-3.1-generate-preview
• veo-3.1-fast-generate-preview
• veo-3.1-lite-generate-preview
• veo-3.0-generate-001
• veo-3.0-fast-generate-001
• veo-2.0-generate-001

Usage (Text-to-Video):

let { operation } = await ai.generate({
  model: vertexAI.model('veo-3.1-lite-generate-preview'),
  prompt: 'A majestic dragon soaring over a mystical forest at dawn.',
  config: {
    aspectRatio: '16:9',
    durationSeconds: 8,
    resolution: '1080p',
    personGeneration: 'allow_adult',
  },
});

if (!operation) throw new Error('No operation returned');

while (!operation.done) {
  operation = await ai.checkOperation(operation);
  await new Promise((resolve) => setTimeout(resolve, 5000));
}

const video = operation.output?.message?.content.find((p) => !!p.media);

Video Extension:

You can extend an existing Veo-generated video by providing it as input to another generation request:

let { operation } = await ai.generate({
  model: vertexAI.model('veo-3.1-generate-preview'),
  prompt: [
    { text: 'Track the butterfly into the garden as it lands on a flower.' },
    {
      media: {
        contentType: 'video/mp4',
        url: previousVeoVideo.media.url,
      },
    },
  ],
  config: {
    aspectRatio: '16:9', // Must match the original video
  },
});

Music Generation (Lyria)

Generate high-quality music and audio clips.

Available Models:

• lyria-3-pro-preview
• lyria-3-clip-preview
• lyria-002 (Legacy)

Usage:

const response = await ai.generate({
  model: vertexAI.model('lyria-3-pro-preview'),
  prompt: 'A cheerful acoustic folk song with guitar and harmonica.',
});

const audioMedia = response.media;

Thinking Config

Thinking Level (Gemini 3.0+)

const response = await ai.generate({
  model: vertexAI.model('gemini-3.1-pro-preview'),
  prompt: 'what is heavier, one kilo of steel or one kilo of feathers',
  config: {
    thinkingConfig: {
      thinkingLevel: 'HIGH', // Or 'LOW' or 'MEDIUM'
      includeThoughts: true,
    },
  },
});

Thinking Budget (Gemini 2.5)

const { message } = await ai.generate({
  model: vertexAI.model('gemini-pro-latest'),
  prompt: 'what is heavier, one kilo of steel or one kilo of feathers',
  config: {
    thinkingConfig: {
      thinkingBudget: 1024,
      includeThoughts: true,
    },
  },
});

Grounding (Gemini Enterprise Search & Google Search)

Enable Google Search or Gemini Enterprise data stores to provide answers grounded in verifiable sources.

// Google Search Grounding
const searchResponse = await ai.generate({
  model: vertexAI.model('gemini-flash-latest'),
  prompt: 'What are the top tech news stories this week?',
  config: {
    tools: [{ googleSearch: {} }],
  },
});

// Gemini Enterprise Search Grounding
const vertexResponse = await ai.generate({
  model: vertexAI.model('gemini-flash-latest'),
  prompt: 'Summarize our company policies.',
  config: {
    vertexRetrieval: {
      datastore: {
        projectId: 'your-project-id',
        location: 'us-central1',
        dataStoreId: 'your-data-store-id',
      },
      disableAttribution: false,
    },
  },
});

Enterprise Features (JavaScript Only)

Model Garden Integration

Access third-party models through Model Garden in Gemini Enterprise:

Anthropic (Claude) Models

Available Models:

• claude-opus-4-7
• claude-sonnet-4-6
• claude-opus-4-6
• claude-haiku-4-5@20251001
• claude-sonnet-4-5@20250929
• claude-sonnet-4@20250514
• claude-opus-4-5@20251101
• claude-opus-4-1@20250805
• claude-opus-4@20250514

import { vertexModelGarden } from '@genkit-ai/vertexai/modelgarden';

const ai = genkit({
  plugins: [vertexModelGarden({ location: 'us-central1' })],
});

const response = await ai.generate({
  model: vertexModelGarden.model('claude-sonnet-4-6'),
  prompt: 'What should I do when I visit Melbourne?',
});

Advanced Configuration (Optional):

You can provide configuration options to tailor the model’s behavior, such as enabling extended thinking features.

const response = await ai.generate({
  model: vertexModelGarden.model('claude-sonnet-4-6'),
  prompt: 'What should I do when I visit Melbourne?',
  config: {
    thinking: {
      enabled: true,
      budgetTokens: 2048,
    },
    output_config: {
      effort: 'high', // Can be 'low', 'medium', 'high', or 'xhigh'
    },
  },
});

For the full list of available Claude models see: Available Claude models

Llama (Meta) Models

Available Models:

• meta/llama-4-maverick-17b-128e-instruct-maas
• meta/llama-4-scout-17b-16e-instruct-maas
• meta/llama-3.3-70b-instruct-maas

const ai = genkit({
  plugins: [vertexModelGarden({ location: 'us-central1' })],
});

const response = await ai.generate({
  model: vertexModelGarden.model(
    'meta/llama-4-maverick-17b-128e-instruct-maas',
  ),
  prompt: 'Write a function that adds two numbers together',
});

For the full list of available Llama models see: Fully-managed Llama models

Mistral AI Models

Available Models:

• mistral-medium-3
• mistral-ocr-2505
• mistral-small-2503
• codestral-2

const ai = genkit({
  plugins: [vertexModelGarden({ location: 'us-central1' })],
});

const response = await ai.generate({
  model: vertexModelGarden.model('mistral-medium-3'),
  prompt: 'Write a function that adds two numbers together',
  config: {
    temperature: 0.7,
    maxOutputTokens: 1024,
    topP: 0.9,
    topK: 40,
  },
});

For the full list of available Mistral AI models see: Mistral AI models

Evaluation Metrics

Use the Gemini Enterprise Rapid Evaluation API for model evaluation:

import {
  vertexAIEvaluation,
  VertexAIEvaluationMetricType,
} from '@genkit-ai/vertexai/evaluation';

const ai = genkit({
  plugins: [
    vertexAIEvaluation({
      location: 'us-central1',
      metrics: [
        VertexAIEvaluationMetricType.SAFETY,
        {
          type: VertexAIEvaluationMetricType.ROUGE,
          metricSpec: {
            rougeType: 'rougeLsum',
          },
        },
      ],
    }),
  ],
});

Available metrics:

• BLEU: Translation quality
• ROUGE: Summarization quality
• Fluency: Text fluency
• Safety: Content safety
• Groundedness: Factual accuracy
• Summarization Quality/Helpfulness/Verbosity: Summary evaluation

Run evaluations:

genkit eval:run
genkit eval:flow -e vertexai/safety

Vector Search

Use Vector Search in Gemini Enterprise for enterprise-grade vector operations:

Setup

1. Create a Vector Search index in the Google Cloud Console
2. Configure dimensions based on your embedding model:
   • gemini-embedding-001 / gemini-embedding-2-preview: default 3072 dimensions; you can set output_dimensionality on embed calls (for example 768, 1536, or 3072 per Google). Size the index to the length you actually use.
   • text-embedding-005: 768 dimensions
   • text-multilingual-embedding-002: 768 dimensions
   • multimodalEmbedding001: 128, 256, 512, or 1408 dimensions
3. Deploy the index to a standard endpoint

Configuration

import { vertexAIVectorSearch } from '@genkit-ai/vertexai/vectorsearch';
import {
  getFirestoreDocumentIndexer,
  getFirestoreDocumentRetriever,
} from '@genkit-ai/vertexai/vectorsearch';

const ai = genkit({
  plugins: [
    vertexAIVectorSearch({
      projectId: 'your-project-id',
      location: 'us-central1',
      vectorSearchOptions: [
        {
          indexId: 'your-index-id',
          indexEndpointId: 'your-endpoint-id',
          deployedIndexId: 'your-deployed-index-id',
          publicDomainName: 'your-domain-name',
          documentRetriever: firestoreDocumentRetriever,
          documentIndexer: firestoreDocumentIndexer,
          embedder: vertexAI.embedder('gemini-embedding-001'),
        },
      ],
    }),
  ],
});

Usage

import {
  vertexAiIndexerRef,
  vertexAiRetrieverRef,
} from '@genkit-ai/vertexai/vectorsearch';

// Index documents
await ai.index({
  indexer: vertexAiIndexerRef({
    indexId: 'your-index-id',
  }),
  documents,
});

// Retrieve similar documents
const results = await ai.retrieve({
  retriever: vertexAiRetrieverRef({
    indexId: 'your-index-id',
  }),
  query: queryDocument,
});

Pricing

Vector Search has both ingestion and hosting costs. See Gemini Enterprise pricing for details.

Next Steps

• Learn about generating content to understand how to use these models effectively
• Explore evaluation to leverage Gemini Enterprise evaluation metrics
• See RAG to implement retrieval-augmented generation with Vector Search
• Check out creating flows to build structured AI workflows
• For simple API key access, see the Google AI plugin