OpenAI-compatible plugin
The genkit-openai package connects Genkit to any service that implements the OpenAI Chat Completions or Embeddings HTTP API. Point the OpenAI plugin at the service with base_url and Genkit sends the same requests it would send to OpenAI.
Installation
Section titled “Installation”uv add genkit-openaiConfiguration
Section titled “Configuration”Pointing OpenAI at a compatible endpoint
Section titled “Pointing OpenAI at a compatible endpoint”Pass api_key and base_url to OpenAI. base_url runs up to and including /v1; the OpenAI SDK appends /chat/completions or /embeddings. Left unset, it falls back to OPENAI_BASE_URL, then the public OpenAI API. Other openai.AsyncOpenAI arguments, such as organization, default_headers, timeout, max_retries, or http_client, go in client_options:
import os
from genkit import Genkitfrom genkit_openai import OpenAI
ai = Genkit( plugins=[ OpenAI( api_key=os.environ['COMPAT_API_KEY'], base_url='https://your-custom-endpoint.example.com/v1', client_options={ 'organization': 'your-org-id', 'default_headers': {'X-Custom-Header': 'value'}, 'timeout': 30.0, }, ) ],)A client_options key that AsyncOpenAI does not accept raises TypeError when you construct the plugin, so a typo like orgnization fails at startup.
Model names
Section titled “Model names”Every model goes through the openai/ namespace, whatever base_url points at. Genkit drops the openai/ prefix and sends the rest as the wire model, so 'openai/llama3' sends llama3, and an ID with its own slash, such as a gateway’s vendor/model form, passes through intact.
The plugin still registers its built-in OpenAI catalog at startup, so the Dev UI lists GPT models your endpoint may not serve. Any other ID resolves on demand the first time you call it.
Generation and provider-specific fields
Section titled “Generation and provider-specific fields”Use OpenAIConfig (or a matching dict) for standard settings such as temperature, max_output_tokens, and top_p. Chat Completions has no top_k, so the plugin drops top_k from the config. Send top_k and any other field the server accepts but OpenAIConfig does not declare inside extra, which goes into the JSON request body as-is:
import os
from genkit import Genkitfrom genkit_openai import OpenAI, OpenAIConfig
# 1. Point the plugin at the compatible serverai = Genkit( plugins=[ OpenAI( api_key=os.environ['COMPAT_API_KEY'], base_url='https://your-custom-endpoint.example.com/v1', ) ],)
# 2. Generate with standard settings plus a server-specific fieldresponse = await ai.generate( model='openai/llama3', prompt='Classify this webhook event payload: {"type": "charge.refunded"}', config=OpenAIConfig( temperature=0.2, max_output_tokens=1000, top_p=0.9, extra={'top_k': 40}, ),)
# 3. Read the replyprint(response.text)# => billing.refundThe server receives max_output_tokens as max_tokens (or max_completion_tokens for reasoning model IDs such as o3 or gpt-5), next to temperature, top_p, and top_k. A key in extra replaces a declared field of the same name.
Per-request API keys
Section titled “Per-request API keys”A multi-tenant app can send each caller’s key with the request instead of the plugin’s. The key travels in context, not config, so it stays out of traces:
response = await ai.generate( model='openai/llama3', prompt='Summarize this support ticket.', context={'secrets': {'api_key': tenant_api_key}},)The call keeps base_url, timeouts, retries, and custom headers, and drops the plugin’s organization and project. A plugin built without api_key and without OPENAI_API_KEY serves only requests that bring a key this way.
Embedders
Section titled “Embedders”Embedder IDs resolve only when the ID contains embed, for example 'openai/nomic-embed-text'. ai.embed() takes no context, so embedders always run on the plugin’s key.