Creating Genkit plugins

Genkit’s capabilities are designed to be extended by plugins. Genkit plugins are configurable modules that can provide models, retrievers, trace stores, and more. You’ve already seen plugins in action just by using Genkit:

import (

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/genkit"
    "github.com/firebase/genkit/go/plugins/googlegenai"
    "github.com/firebase/genkit/go/plugins/server"
)

g := genkit.Init(ctx,
    genkit.WithPlugins(
        &googlegenai.GoogleAI{APIKey: ...},
        &googlegenai.VertexAI{ProjectID: "my-project", Location: "us-central1"},
    ),
)

The Gemini Enterprise (vertexAI) plugin takes configuration (such as the user’s Google Cloud project ID) and registers a variety of new models, embedders, and more with the Genkit registry. The registry serves as a lookup service for named actions at runtime, and powers Genkit’s local UI for running and inspecting models, prompts, and more.

Creating a plugin

In Go, a Genkit plugin is a type that implements the api.Plugin interface. A single module can contain several plugins.

Provider ID

Every plugin must have a unique identifier string that distinguishes it from other plugins. Genkit uses this identifier as a namespace for every resource your plugin defines, to prevent naming conflicts with other plugins.

For example, if your plugin has an ID yourplugin and provides a model called text-generator, the full model identifier will be yourplugin/text-generator.

Define the provider ID once and use it consistently. It can stay unexported: if your plugin exports a ModelRef helper, as the example below does, callers never spell the raw string.

package yourplugin

const providerID = "yourplugin"

Every Genkit constructor takes the action name as a plain string. api.NewName(provider, id) returns provider + "/" + id, so api.NewName(providerID, "text-generator") and the literal "yourplugin/text-generator" are the same value. This page uses api.NewName in plugin code and literals in the application-facing asides.

Standard exports

Every plugin exports a struct type that encapsulates all of the configuration options accepted by the plugin, and implements Name() and Init() on it.

For any plugin options that are secret values, such as API keys, you should offer both a config option and a default environment variable to configure it. This lets your plugin take advantage of the secret-management features offered by many hosting providers, such as Cloud Secret Manager, which you can use with Cloud Run.

package myplugin

import (
    "context"
    "os"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/core/api"
)

const providerID = "myprovider"

// MyPlugin holds every option the plugin accepts.
type MyPlugin struct {
    // APIKey is the credential. If empty, MYPROVIDER_API_KEY is consulted.
    APIKey string

    // Models overrides what the plugin knows about a model, keyed by model ID.
    // Fields left at their zero value keep what the plugin resolved.
    Models map[string]ai.ModelOptions

    client *apiClient
}

// Name returns the provider ID.
func (p *MyPlugin) Name() string { return providerID }

// Init creates shared resources and returns the actions to register.
func (p *MyPlugin) Init(ctx context.Context) []api.Action {
    apiKey := p.APIKey
    if apiKey == "" {
        apiKey = os.Getenv("MYPROVIDER_API_KEY")
    }
    if apiKey == "" {
        panic("myprovider requires setting APIKey on the plugin or MYPROVIDER_API_KEY in the environment")
    }
    p.client = newAPIClient(apiKey)

    return []api.Action{
        p.newModel("my-model"),
    }
}

var _ api.Plugin = (*MyPlugin)(nil)

Init is called automatically during genkit.Init() when the user passes the plugin into the WithPlugins() option. In it:

• Confirm that any required configuration values are specified and assign default values to any unspecified optional settings.
• Verify that the given configuration options are valid together.
• Create any shared resources required by the rest of your plugin. For example, create clients for any services your plugin accesses.
• Construct your actions and return them. The framework registers whatever Init returns, so a plugin never reaches into the registry itself.

To the extent possible, the resources provided by your plugin shouldn’t assume that any other plugins have been installed before this one.

Reporting an initialization failure

Init returns no error. Genkit treats a plugin that cannot be configured as a startup failure, so panic. genkit.Init does not recover the panic: it propagates to the caller’s main, which is the intended behavior, because a misconfigured plugin cannot serve traffic. There is no degraded-start mode.

Name the provider, what is missing, and how to supply it. The in-tree plugins set the pattern:

panic("Google AI requires setting GEMINI_API_KEY or GOOGLE_API_KEY in the environment. You can get an API key at https://ai.google.dev")

Resolving actions on demand

A provider that serves more models than are worth registering up front can also implement api.DynamicPlugin, which adds two methods. ListActions advertises everything the plugin could serve, so the Developer UI can list it, and ResolveAction builds one on demand when a name is looked up and nothing is registered under it.

// ListActions advertises every model the plugin could serve.
func (p *MyPlugin) ListActions(ctx context.Context) []api.ActionDesc {
    var descs []api.ActionDesc
    for _, id := range p.client.listModels(ctx) {
        descs = append(descs, p.newModel(id).Desc())
    }
    return descs
}

// ResolveAction builds a model on demand. id arrives bare: the registry has
// already matched the provider prefix to this plugin.
func (p *MyPlugin) ResolveAction(atype api.ActionType, id string) api.Action {
    if atype != api.ActionTypeModel {
        return nil
    }
    return p.newModel(id)
}

Because an unknown ID still resolves, a curated catalog is a source of capability metadata rather than an allowlist.

Lifecycle and concurrency

api.Plugin is two methods, Name() and Init(ctx). There is no teardown hook. That shapes what your plugin may hold:

• Init runs exactly once per genkit.Init call, before any action can run, so the fields it writes need no synchronization.
• A plugin value must not be passed to two genkit.Init calls. Guard it with a flag and panic with "plugin already initialized", as googlegenai does.
• After Init, the same plugin pointer serves ListActions, ResolveAction, and every action call, from many goroutines at once. Anything written after Init has to be goroutine-safe.
• Because there is no shutdown hook, a plugin that owns a flushable resource has to arrange its own flush. See Shutting down cleanly.

A minimal plugin end to end

Everything above fits in one file. This one serves a model that echoes the last user message, so you can run it without a provider account and then swap the generation function for a real API call.

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/core/api"
    "github.com/firebase/genkit/go/genkit"
)

const provider = "myprovider"

// EchoConfig is the model's typed config. Its JSON schema is inferred from it
// and validated on every request.
type EchoConfig struct {
    Prefix string `json:"prefix,omitempty"`
}

// EchoPlugin serves one model.
type EchoPlugin struct{}

func (p *EchoPlugin) Name() string { return provider }

func (p *EchoPlugin) Init(ctx context.Context) []api.Action {
    return []api.Action{
        ai.NewModelAction(api.NewName(provider, "echo-001"),
            &ai.ModelOptions{
                Label:    "Echo",
                Supports: &ai.ModelSupports{Multiturn: true, SystemRole: true},
            },
            func(ctx context.Context, req *ai.ModelRequest, cfg *EchoConfig, cb ai.ModelStreamCallback) (*ai.ModelResponse, error) {
                var text string
                if n := len(req.Messages); n > 0 {
                    text = req.Messages[n-1].Text()
                }
                if cfg != nil {
                    text = cfg.Prefix + text
                }

                if cb != nil {
                    chunk := &ai.ModelResponseChunk{
                        Role:    ai.RoleModel,
                        Content: []*ai.Part{ai.NewTextPart(text)},
                    }
                    if err := cb(ctx, chunk); err != nil {
                        return nil, err
                    }
                }

                return &ai.ModelResponse{
                    Message:      ai.NewModelTextMessage(text),
                    FinishReason: ai.FinishReasonStop,
                    Usage:        &ai.GenerationUsage{OutputTokens: len(text)},
                }, nil
            }),
    }
}

var _ api.Plugin = (*EchoPlugin)(nil)

func main() {
    ctx := context.Background()
    g := genkit.Init(ctx, genkit.WithPlugins(&EchoPlugin{}))

    out, err := genkit.GenerateText(ctx, g,
        ai.WithModelName("myprovider/echo-001"),
        ai.WithPrompt("hello, plugin"),
    )
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(out)
}

Running it prints:

hello, plugin

Run it under genkit start -- go run . and the model shows up in the Developer UI, config sidebar included, with no further work.

Building plugin features

A single plugin can activate many new things within Genkit. For example, the Gemini Enterprise (VertexAI) plugin activates several new models as well as an embedder. Here’s how to build some common plugin types.

Model plugins

Genkit model plugins add one or more generative AI models to the Genkit registry. A model represents any generative model that is capable of receiving a prompt as input and generating text, media, or data as output.

Model definitions

A model definition consists of three components:

1. Metadata declaring the model’s capabilities.
2. A configuration type with any specific parameters supported by the model.
3. A generation function that accepts an ai.ModelRequest and returns an ai.ModelResponse, presumably using an AI model to generate the latter.

ai.NewModelAction binds all three. Its Config type parameter is inferred from the generation function, so the signature you write is the contract:

// MyModelConfig defines the configuration options for your model. Its JSON
// schema is inferred from this type and enforced on every request.
type MyModelConfig struct {
    Temperature *float64 `json:"temperature,omitempty" jsonschema:"minimum=0,maximum=2"`
    MaxTokens   int      `json:"maxTokens,omitempty" jsonschema:"minimum=1"`
}

func (p *MyPlugin) newModel(id string) *ai.ModelAction {
    opts := ai.ModelOptions{
        Label: "My Model", // User-friendly label
        Supports: &ai.ModelSupports{
            Multiturn:  true,  // Does the model support multi-turn chats?
            SystemRole: true,  // Does the model support system messages?
            Media:      false, // Can the model accept media input?
            Tools:      false, // Does the model support function calling (tools)?
        },
        Versions: []string{"my-model-001"}, // List supported versions/aliases
    }
    // An application entry corrects or extends what the plugin resolved.
    opts = opts.Overlay(p.Models[id])

    return ai.NewModelAction(api.NewName(providerID, id), &opts,
        func(ctx context.Context, req *ai.ModelRequest, cfg *MyModelConfig, cb ai.ModelStreamCallback) (*ai.ModelResponse, error) {
            // cfg arrives deserialized and validated. It is nil when the caller
            // sent no config, because Config here is a pointer type.
            if cfg == nil {
                cfg = &MyModelConfig{}
            }

            // Use your custom logic to convert Genkit's ai.ModelRequest into a
            // form usable by the model's native API.
            apiReq, err := apiRequestFromGenkitRequest(req, cfg)
            if err != nil {
                return nil, err
            }

            // Send the request to the model API, using your own code or the
            // model API's client library.
            apiResp, err := p.client.generate(ctx, apiReq)
            if err != nil {
                return nil, wrapAPIError(err)
            }

            // Convert the model's response to Genkit's ai.ModelResponse.
            return genkitResponseFromAPIResponse(req, apiResp)
        })
}

ai.NewModelAction returns an unregistered *ai.ModelAction that satisfies both ai.Model and api.Action, so Init can return it in an []api.Action without a type assertion. ai.ModelOptions.Overlay returns the receiver with every field set in the override replacing it, which is how an application corrects one capability without restating the label and the versions.

Request and response shapes

The generation function receives an *ai.ModelRequest and returns an *ai.ModelResponse. The request carries no model name: the plugin already knows which model it registered.

ai.ModelRequest field	Type	Meaning
Messages	[]*ai.Message	The conversation history.
Config	any	The raw config. The framework has already decoded it into your typed cfg parameter, so read cfg instead.
Tools	[]*ai.ToolDefinition	Tools the model may ask the caller to run.
ToolChoice	ai.ToolChoice	ai.ToolChoiceAuto, ai.ToolChoiceRequired, or ai.ToolChoiceNone.
Output	*ai.ModelOutputConfig	The requested response format.
Docs	[]*ai.Document	Context documents. See Retrieval-augmented generation.

The response is split between the two sides:

ai.ModelResponse field	Type	Who sets it
Message	*ai.Message	Plugin. Required, with Role: ai.RoleModel.
FinishReason	ai.FinishReason	Plugin. Required.
FinishMessage	string	Plugin, when the provider explains why it stopped.
Error	*status.Error	Framework. Leave it nil; the tool loop sets it on a response that comes back beside an error.
Usage	*ai.GenerationUsage	Plugin. Strongly recommended: cost and token accounting read it.
Raw	any	Plugin. The unprocessed provider payload.
Operation	*ai.Operation	Plugin, for long-running background models only.
Request	*ai.ModelRequest	Framework. Leave it nil.
LatencyMs	float64	Framework. Leave it zero.
Custom	any	Deprecated. Use Raw.

ai.FinishReason has eight values:

Constant	Meaning
ai.FinishReasonStop	The model finished normally.
ai.FinishReasonLength	Generation hit the token cap.
ai.FinishReasonBlocked	A safety or policy filter stopped it.
ai.FinishReasonOther	The provider gave a reason none of the others cover.
ai.FinishReasonUnknown	The provider gave no reason.
ai.FinishReasonAborted	The request was cancelled before it completed.
ai.FinishReasonInterrupted	A tool raised an interrupt. The framework sets this in its tool loop, never a plugin.
ai.FinishReasonFailed	Something threw inside the tool loop. The framework sets this on the partial response it returns beside the error, never a plugin.

Map your provider’s reasons onto the first five. Do not invent a value: the field is a string type, and Genkit treats every reason except stop, length, and unknown as an abnormal finish that skips output parsing.

Fill Usage with what the provider billed. Besides InputTokens, OutputTokens, and TotalTokens, ai.GenerationUsage carries CachedContentTokens, ThoughtsTokens, per-modality counts such as InputImages and OutputAudioFiles, and Custom map[string]float64 for provider metrics such as cost. See Generating content for the read side.

A text-only conversion, end to end:

func genkitResponseFromAPIResponse(req *ai.ModelRequest, apiResp *apiResponse) (*ai.ModelResponse, error) {
    reason := ai.FinishReasonUnknown
    switch apiResp.StopReason {
    case "end_turn":
        reason = ai.FinishReasonStop
    case "max_tokens":
        reason = ai.FinishReasonLength
    case "safety":
        reason = ai.FinishReasonBlocked
    }

    return &ai.ModelResponse{
        Message:      ai.NewModelTextMessage(apiResp.Text),
        FinishReason: reason,
        Usage: &ai.GenerationUsage{
            InputTokens:  apiResp.PromptTokens,
            OutputTokens: apiResp.CompletionTokens,
            TotalTokens:  apiResp.TotalTokens,
        },
        Raw: apiResp,
    }, nil
}

Streaming

The fourth parameter of the generation function is an ai.ModelStreamCallback, an alias for func(context.Context, *ai.ModelResponseChunk) error.

The framework passes nil when the caller did not ask for streaming. It never passes a no-op, so branch on cb == nil to pick your non-streaming transport, as the googlegenai plugin does:

func (p *MyPlugin) generate(ctx context.Context, req *ai.ModelRequest, cfg *MyModelConfig, cb ai.ModelStreamCallback) (*ai.ModelResponse, error) {
    apiReq, err := apiRequestFromGenkitRequest(req, cfg)
    if err != nil {
        return nil, err
    }

    if cb == nil {
        apiResp, err := p.client.generate(ctx, apiReq)
        if err != nil {
            return nil, wrapAPIError(err)
        }
        return genkitResponseFromAPIResponse(req, apiResp)
    }

    stream, err := p.client.generateStream(ctx, apiReq)
    if err != nil {
        return nil, wrapAPIError(err)
    }
    defer stream.Close()

    for {
        delta, err := stream.Recv()
        if errors.Is(err, io.EOF) {
            break
        }
        if err != nil {
            return nil, wrapAPIError(err)
        }
        chunk := &ai.ModelResponseChunk{
            Role:    ai.RoleModel,
            Content: []*ai.Part{ai.NewTextPart(delta.Text)},
        }
        // Stop reading the provider stream if the consumer went away.
        if err := cb(ctx, chunk); err != nil {
            return nil, err
        }
    }

    return genkitResponseFromAPIResponse(req, stream.Final())
}

The chunk contract:

• Chunks are incremental deltas. Put only the new parts in Content and leave Aggregated false. Set Aggregated true only if every chunk repeats everything sent so far.
• Index is the index of the message the chunk belongs to, which is the turn index inside a tool loop, not a candidate index. Use 0 for a single-turn call.
• Role is ai.RoleModel.
• No terminating chunk is required.
• cb is not safe for concurrent use. Call it from one goroutine at a time.
• The framework does not aggregate chunks. The *ai.ModelResponse you return is authoritative, so build it from the provider’s final frame, not from what you streamed.

Structured output

req.Output tells you what shape the caller wants back.

ai.ModelOutputConfig field	Type	Meaning
Format	string	The requested format, such as "json" or "text".
ContentType	string	The MIME type of the output.
Schema	map[string]any	A JSON Schema describing the desired response.
Constrained	bool	Whether the provider must enforce Schema natively.

The contract turns on Constrained:

• When it is true, pass Schema to your provider’s native constrained-output mechanism. Genkit is relying on the provider to enforce it.
• When it is false, Genkit has already injected format instructions into the prompt. Send the request unchanged.

Genkit sets Constrained only when the caller asked for a schema and ModelSupports.Constrained says the model can serve it. Claim ai.ConstrainedSupportAll only once you actually wire Schema through: a false claim turns off the prompt-instruction fallback and nothing enforces the schema.

if req.Output != nil && req.Output.Constrained && req.Output.Schema != nil {
    apiReq.ResponseFormat = &apiResponseFormat{
        Type:   "json_schema",
        Schema: req.Output.Schema,
    }
}

Tool calling

req.Tools holds the tools the caller made available. Translate them into your provider’s function-declaration type.

ai.ToolDefinition field	Type	Meaning
Name	string	The tool name the model must echo back.
Description	string	What the tool does and when to use it.
InputSchema	map[string]any	JSON Schema for the tool’s input.
OutputSchema	map[string]any	JSON Schema for the tool’s output.
Metadata	map[string]any	Flags Genkit attaches, such as multipart and strict.
Key	string	Part of the cross-runtime schema. Go tools leave it empty.

for _, t := range req.Tools {
    apiReq.Functions = append(apiReq.Functions, apiFunction{
        Name:        t.Name,
        Description: t.Description,
        Parameters:  t.InputSchema,
    })
}

On the way back, a tool call becomes a part built with ai.NewToolRequestPart:

var parts []*ai.Part
for _, call := range apiResp.FunctionCalls {
    parts = append(parts, ai.NewToolRequestPart(&ai.ToolRequest{
        Name:  call.Name,
        Ref:   call.ID,       // echoed back on the matching ToolResponse
        Input: call.Arguments, // any; typically map[string]any
    }))
}
resp := &ai.ModelResponse{
    Message:      ai.NewModelMessage(parts...),
    FinishReason: ai.FinishReasonStop,
}

ToolRequest.Input is any, not a map type, so decoded JSON arguments go in as-is. Set Partial to true on a tool-call part emitted as a streaming chunk whose arguments are not complete yet.

Per-request credentials

Nothing in ai.ModelRequest carries a credential. A generation function reads request-scoped values from its ctx, so an application that needs a different key per caller puts it there. See Passing information through context. On the OpenAI-compatible transport there is a ready-made path, compat_oai.RequestConfig.APIKey: it is code-only, never serializes into the schema or the trace, and routes that one request through a client built with it.

Defining your model’s config schema

To specify the generation options a model supports, define and export a configuration type and use it as the Config type parameter. jsonschema struct tags become the constraints, and jsonschema_description tags become the field documentation the Developer UI renders in its config sidebar. Genkit has an ai.GenerationCommonConfig type that contains options frequently supported by generative AI model services, which you can embed if your provider matches it.

The framework infers the config JSON schema from Config and validates every request against it before your generation function runs, so the function never needs a type check. Config sent as the exact type, as a pointer to it, or as a map[string]any from a JSON caller such as the Developer UI is converted to the typed value; anything else fails with INVALID_ARGUMENT. When Config is a pointer type, a request that carries no config hands your function a nil pointer, so nil-check it. When Config is a value type, you get the zero value.

A declared config schema is enforced

Set ModelOptions.ConfigSchema only when reflection over your Config type does not match how that type marshals to JSON, which happens with SDK wrapper generics. Because the declared schema is checked on every call, a curated schema narrower than what callers actually send rejects those requests rather than ignoring the extra fields.

Declaring model capabilities

Every model definition must contain, as part of its metadata, an ai.ModelSupports value that declares which features the model supports. Genkit uses this information to determine certain behaviors, such as verifying whether certain inputs are valid for the model. For example, if the model doesn’t support multi-turn interactions, then it’s an error to pass it a message history.

The type has ten fields. A nil Supports claims nothing at all.

Field	Type	Declares	Enforced by rejecting
Multiturn	bool	The model accepts a message history.	A request with more than one message.
SystemRole	bool	The model accepts messages with role system.	A request containing a system message.
Media	bool	The model accepts media in the prompt.	Media parts in the request.
Tools	bool	The model can call tools.	A request that carries tools.
ToolChoice	bool	The caller may force or forbid tool use.	A ToolChoice other than auto.
Context	bool	The model grounds on documents natively.	Nothing. When false, Genkit inlines Docs into the last user message instead of leaving them in req.Docs.
LongRunning	bool	The model returns an ai.Operation to poll.	Nothing; it is advertised metadata.
ContentType	[]string	The MIME types the model can emit.	Nothing; it is advertised metadata.
Output	[]string	The output kinds the model can produce, such as "text" or "json".	Nothing; it is advertised metadata.
Constrained	ai.ConstrainedSupport	Whether the provider enforces an output schema natively.	A direct model call with Output.Constrained set, when the model claims none.

Constrained is an enum, not a bool:

• ai.ConstrainedSupportNone, which is also the zero value, means the provider cannot enforce a schema.
• ai.ConstrainedSupportAll means it always can.
• ai.ConstrainedSupportNoTools means it can, except when the request also carries tools.

Whenever the declaration does not cover the request, ai.Generate leaves req.Output.Constrained false and injects format instructions into the prompt instead, so your model function still receives a request it can serve.

Note that these declarations refer to the capabilities of the model as provided by your plugin, and do not necessarily map one-to-one to the capabilities of the underlying model and model API. For example, even if the model API doesn’t provide a specific way to define system messages, your plugin might still declare support for the system role, and implement it as special logic that inserts system messages into the user prompt.

The two ways to be wrong are not symmetric. Declaring too narrowly is refused by Genkit before the request leaves the process, blocking a model that would have worked, while an over-wide claim reaches the provider and comes back with the real reason.

Transforming requests and responses

The generation function carries out the primary work of a Genkit model plugin: transforming the ai.ModelRequest from Genkit’s common format into a format that is supported by your model’s API, and then transforming the response from your model into the ai.ModelResponse format used by Genkit.

Sometimes, this may require massaging or manipulating data to work around model limitations. For example, if your model does not natively support a system message, you may need to transform a prompt’s system message into a user-model message pair.

Classifying provider errors

Retry and fallback middleware branch on the status an error carries, so classify your SDK’s errors at the boundary rather than returning them raw.

import (
    "errors"

    "github.com/firebase/genkit/go/core/status"
)

// providerSDKError stands in for your SDK's error type; substitute the
// concrete type your client returns.
type providerSDKError struct {
    StatusCode int
}

func (e *providerSDKError) Error() string { return "provider error" }

// wrapAPIError classifies an SDK error so status-aware middleware can tell a
// rate limit from a request the provider rejected.
func wrapAPIError(err error) error {
    var apiErr *providerSDKError
    if !errors.As(err, &apiErr) {
        return err
    }
    return status.Errorf(status.Base(status.FromHTTPCode(apiErr.StatusCode)), "%w", err)
}

status.FromHTTPCode maps an HTTP status code to a canonical status name, and status.Base turns that name into the sentinel status.Errorf classifies with. Recording the cause with %w keeps the original error reachable through errors.Is and errors.As. See Error types for the full status vocabulary and the sentinels Genkit defines.

Leave anything that is not an API error unclassified. The retry middleware retries an unclassified error unconditionally, which is the right answer for a dial timeout, and the fallback middleware leaves it alone, because failing over to a different billed model is worth requiring an explicit classification for. A classified error is retried only when its status is on the middleware’s list, so classifying a 401 as UNAUTHENTICATED is what stops it being reissued until the attempts run out. Middleware covers both from the application side, and the retry and fallback sample is the worked example.

Wrap with %w, never %v. Flattening the error destroys the classification, which silently turns a non-retryable INVALID_ARGUMENT into an unclassified error that is always retried.

Exports

In addition to the resources that all plugins must export, a model plugin should also export the following:

• A generation config type, as discussed earlier.

• A ModelRef function, which creates a model reference paired with its config so the two travel together and the config is typed at the call site:

func ModelRef(id string, config *MyModelConfig) ai.ModelRef {
    return ai.NewModelRef(api.NewName(providerID, id), config)
}

Callers then write ai.WithModel(myplugin.ModelRef("my-model", &myplugin.MyModelConfig{...})). ai.NewRetrieverRef and ai.NewEmbedderRef are the siblings for the other primitives, with the same (name string, config any) signature.

• A Models map[string]ai.ModelOptions field on the plugin struct, so an application can correct a capability your catalog got wrong, or describe a model released after your plugin was. Route every path that describes a model through one method that applies the overlay, so an entry is authoritative whether Init, ListActions, or ResolveAction reaches the model first.

  Prefer this to an exported DefineModel function. Correction is data rather than API surface, and a model that Init has already registered cannot be re-registered.

Retrievers, embedders, and evaluators

The other primitives follow the same shape as models: ai.NewRetrieverAction, ai.NewEmbedderAction, ai.NewEvaluatorAction, ai.NewBatchEvaluatorAction, and ai.NewBackgroundModelAction, each with its own options struct and its own typed Config type parameter, validated the same way. ai.NewBatchEvaluatorAction shares ai.EvaluatorOptions with ai.NewEvaluatorAction; there is no separate batch options type.

The two you are most likely to write are generic over the config type:

func NewRetrieverAction[Config any](name string, opts *RetrieverOptions, fn RetrieverActionFunc[Config]) *RetrieverAction
func NewEmbedderAction[Config any](name string, opts *EmbedderOptions, fn EmbedderActionFunc[Config]) *EmbedderAction

Config is inferred from fn, and its JSON schema is inferred from Config unless the options struct’s ConfigSchema overrides it. Both return an unregistered action, so build them as methods and return them from Init:

type MyRetrieverConfig struct {
    K int `json:"k,omitempty" jsonschema:"minimum=1"`
}

type MyEmbedderConfig struct {
    Dimensions int `json:"dimensions,omitempty" jsonschema:"minimum=1"`
}

// retrieve searches the index. req.Query is an *ai.Document.
func (p *MyPlugin) retrieve(ctx context.Context, req *ai.RetrieverRequest, cfg *MyRetrieverConfig) (*ai.RetrieverResponse, error) {
    k := 10
    if cfg != nil && cfg.K > 0 {
        k = cfg.K
    }
    docs, err := p.client.search(ctx, req.Query, k)
    if err != nil {
        return nil, wrapAPIError(err)
    }
    return &ai.RetrieverResponse{Documents: docs}, nil
}

// embed returns one *ai.Embedding per input document, in the same order.
func (p *MyPlugin) embed(ctx context.Context, req *ai.EmbedRequest, cfg *MyEmbedderConfig) (*ai.EmbedResponse, error) {
    out := make([]*ai.Embedding, 0, len(req.Input))
    for _, d := range req.Input {
        var text string
        for _, part := range d.Content {
            if part.IsText() {
                text += part.Text
            }
        }
        v, err := p.client.embed(ctx, text) // v is []float32
        if err != nil {
            return nil, wrapAPIError(err)
        }
        out = append(out, &ai.Embedding{Embedding: v})
    }
    return &ai.EmbedResponse{Embeddings: out}, nil
}

// Init returns both alongside whatever models the plugin serves.
func (p *MyPlugin) Init(ctx context.Context) []api.Action {
    return []api.Action{
        ai.NewRetrieverAction[*MyRetrieverConfig](api.NewName(providerID, "docs"),
            &ai.RetrieverOptions{Label: "My Docs"}, p.retrieve),
        ai.NewEmbedderAction[*MyEmbedderConfig](api.NewName(providerID, "embed-001"),
            &ai.EmbedderOptions{Label: "My Embedder", Dimensions: 768}, p.embed),
    }
}

ai.EmbedRequest.Input is []*ai.Document, the same document type Retrieval-augmented generation describes, and its Options field carries the raw config that the framework decoded into cfg. ai.EmbedResponse.Embeddings is []*ai.Embedding, where Embedding.Embedding is a []float32 and the optional Embedding.Metadata identifies which part of a document the vector covers.

From an application

An application does not build unregistered actions. It calls the genkit.Define*Action wrappers, which take the same options struct and register immediately:

func defineLocalRetriever(g *genkit.Genkit) {
    genkit.DefineRetrieverAction(g, "myprovider/docs",
        &ai.RetrieverOptions{Label: "My Docs"},
        func(ctx context.Context, req *ai.RetrieverRequest, cfg *MyRetrieverConfig) (*ai.RetrieverResponse, error) {
            return &ai.RetrieverResponse{Documents: search(ctx, req.Query)}, nil
        })
}

Registering JSON schemas

Schema registration lives in package genkit only. Register a schema when a .prompt file, ai.WithOutputSchemaName, or ai.WithInputSchemaName refers to one by name. The call must run before the prompt is rendered.

// Register several Go types at once, each under its type's name.
genkit.DefineSchemasFor(g, JokeRequest{}, Joke{})

// The single-type form.
genkit.DefineSchemaFor[Recipe](g)

// A raw JSON schema needs an explicit name.
genkit.DefineSchema(g, "Rating", map[string]any{
    "type": "integer", "minimum": 1, "maximum": 5,
})

Each name may be registered once per Genkit instance. A second registration of the same name panics, so call these helpers once at startup. DefineSchemasFor and DefineSchemaFor take Go types, and panic on a map, a nil value, or an unnamed type, pointing you at DefineSchema for a raw schema.

Package core keeps the schema utilities that do not register anything. core.InferSchemaMap turns a Go type into a schema map, which is how a plugin fills an options struct’s ConfigSchema:

opts := ai.ModelOptions{ConfigSchema: core.InferSchemaMap(MyModelConfig{})}

Lower-level actions

For anything that is not one of the AI primitives, package core has the generic constructors: core.NewActionOf, core.NewStreamingActionOf, core.NewBidiActionOf, and core.NewBackgroundActionOf. Each takes the action type first, then the name, then an options struct, then the implementation.

action := core.NewActionOf(api.ActionTypeCustom, "myAction",
    &core.ActionOptions{
        Description: "Processes a string",
        Metadata:    map[string]any{"team": "search"},
        // InputSchema, OutputSchema and StreamSchema left nil are inferred
        // from the type parameters.
    },
    func(ctx context.Context, input string) (string, error) {
        return "processed: " + input, nil
    })

Return the action from your plugin’s Init, as in the Init example above. The framework registers whatever Init returns.

A nil options value is legal and infers every schema. core.NewBackgroundActionOf takes a core.BackgroundActionOptions[In, Out], whose Check field is required and whose Cancel field is optional: omitting Cancel means the action does not support cancellation. The framework registers the two as companions named <name>/check and <name>/cancel, the keys the other runtimes and the Developer UI’s background-task panel use, so an operation started in Go can be polled and cancelled from either. ai.NewBackgroundModelAction follows the same layout.

Building on the OpenAI-compatible core

If your provider speaks the OpenAI chat-completions API, build on plugins/compat_oai rather than writing a client from scratch. Point the base plugin at the endpoint and every model resolves by name, taking the OpenAI SDK’s own request type as its config:

import (
    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/genkit"
    "github.com/firebase/genkit/go/plugins/compat_oai"
    "github.com/openai/openai-go"
)

g := genkit.Init(ctx, genkit.WithPlugins(&compat_oai.OpenAICompatible{
    Provider: "myprovider",
    APIKey:   apiKey,
    BaseURL:  "https://api.example.com/v1",
}))

model := ai.NewModelRef("myprovider/some-model", &openai.ChatCompletionNewParams{
    Temperature: openai.Float(0.7),
})

The custom provider sample is the worked example.

To give your provider a config of its own, and a home for the extensions the SDK request type has no room for, embed compat_oai.OpenAICompatible in your plugin struct, declare a config that embeds compat_oai.RequestConfig, implement ApplyToChatCompletion, and build models with compat_oai.NewChatModel[Config]:

type MyPlugin struct {
    APIKey string

    openAICompatible compat_oai.OpenAICompatible
}

type ChatConfig struct {
    compat_oai.RequestConfig

    Temperature  *float64 `json:"temperature,omitempty" jsonschema:"minimum=0,maximum=2"`
    EnableSearch *bool    `json:"enableSearch,omitempty"`
}

func (c ChatConfig) ApplyToChatCompletion(params *openai.ChatCompletionNewParams) {
    c.ApplyVersion(params)
    if c.Temperature != nil {
        params.Temperature = openai.Float(*c.Temperature)
    }
    if c.EnableSearch != nil {
        compat_oai.AddExtraFields(params, map[string]any{"enable_search": *c.EnableSearch})
    }
}

func (p *MyPlugin) newChatModel(id string, opts ai.ModelOptions) *ai.ModelAction {
    return compat_oai.NewChatModel[ChatConfig](&p.openAICompatible, id, opts)
}

Init must populate p.openAICompatible.Provider, APIKey or Opts, and BaseURL before it builds any model. The Config type parameter has to satisfy compat_oai.ChatConfig, which embedding compat_oai.RequestConfig plus your own ApplyToChatCompletion provides. See OpenAI-compatible APIs for openai.Float, openai.ChatCompletionNewParams, and RequestConfig.ApplyVersion.

Embedding RequestConfig gives every config three shared settings: a code-only APIKey for a per-request credential, Version to pin an exact model version, and Extra for undeclared body fields sent verbatim under the provider’s wire names. compat_oai.ListChatActions and compat_oai.ResolveChatAction are the matching DynamicPlugin methods, and compat_oai.ModelOptionsFor applies a caller’s Models entry over what your catalog resolved.

On this path you do not classify SDK errors yourself: the shared request path runs compat_oai.WrapAPIError, which wraps an error the OpenAI SDK returned for an HTTP response in a status.Error carrying the status the server reported, and passes everything else through untouched.

Telemetry plugins

The Genkit libraries are instrumented with OpenTelemetry to support collecting traces, metrics, and logs. Genkit users can export this telemetry data to monitoring and visualization tools by installing a plugin that configures the OpenTelemetry Go SDK to export to a particular OpenTelemetry-capable system.

Genkit includes a plugin that configures OpenTelemetry to export data to Google Cloud Monitoring and Cloud Logging. To support other monitoring systems, you can extend Genkit by writing a telemetry plugin. The telemetry sample exercises one end to end.

Exporters and loggers

The primary job of a telemetry plugin is to configure OpenTelemetry to export data to a particular service. To do so, you need the following:

• An implementation of OpenTelemetry’s SpanExporter interface that exports data to the service of your choice.
• An implementation of OpenTelemetry’s metric.Exporter interface that exports data to the service of your choice.
• Either a slog.Logger or an implementation of the slog.Handler interface, that exports logs to the service of your choice.

Depending on the service you’re interested in exporting to, this might be a relatively minor effort or a large one.

Because OpenTelemetry is an industry standard, many monitoring services already have libraries that implement these interfaces. For example, the googlecloud plugin for Genkit makes use of the opentelemetry-operations-go library, maintained by the Google Cloud team. Similarly, many monitoring services provide libraries that implement the standard slog interfaces.

On the other hand, if no such libraries are available for your service, implementing the necessary interfaces can be a substantial project.

Check the OpenTelemetry registry or the monitoring service’s docs to see if integrations are already available.

If you need to build these integrations yourself, take a look at the source of the official OpenTelemetry exporters and the page A Guide to Writing slog Handlers.

Building the plugin

A telemetry plugin is an ordinary api.Plugin that registers no actions. Keep its configuration as fields on the plugin struct so Init has it in scope, and keep the providers it creates so it can flush them later.

package mytelemetry

import (
    "context"
    "errors"
    "log/slog"
    "os"
    "time"

    "github.com/firebase/genkit/go/core/api"
    "github.com/firebase/genkit/go/core/tracing"
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/sdk/metric"
    sdktrace "go.opentelemetry.io/otel/sdk/trace"
)

// Config holds the settings every telemetry plugin should accept. Most plugins
// add settings for the service they export to, such as an API key or a project.
type Config struct {
    // ForceExport exports even in the dev environment.
    ForceExport bool

    // MetricInterval is how often metric data is exported. Genkit's own
    // exporters use 5s in dev and 5min in production; pick what suits your
    // backend rather than copying a number from here.
    MetricInterval time.Duration

    // LogLevel is the minimum level at which logs are written.
    // Defaults to slog.LevelInfo.
    LogLevel slog.Leveler
}

// Plugin exports Genkit telemetry to your service.
type Plugin struct {
    Config

    meterProvider *metric.MeterProvider
}

// Name returns the provider ID.
func (p *Plugin) Name() string { return "mytelemetry" }

// Init configures the OpenTelemetry SDK. It registers no actions.
func (p *Plugin) Init(ctx context.Context) []api.Action {
    // Stay quiet in the dev environment, such as under `genkit start`, unless
    // the application asked otherwise.
    if !p.ForceExport && os.Getenv("GENKIT_ENV") == "dev" {
        return nil
    }

    // Traces: register a span processor on Genkit's tracer provider.
    exporter := &redactingSpanExporter{SpanExporter: newSpanExporter()}
    tracing.TracerProvider().RegisterSpanProcessor(sdktrace.NewBatchSpanProcessor(exporter))

    // Metrics: a periodic reader on the configured interval.
    interval := p.MetricInterval
    if interval == 0 {
        interval = time.Minute
    }
    reader := metric.NewPeriodicReader(newMetricExporter(), metric.WithInterval(interval))
    p.meterProvider = metric.NewMeterProvider(metric.WithReader(reader))
    otel.SetMeterProvider(p.meterProvider)

    // Logs: your slog handler, honoring the configured minimum level.
    level := p.LogLevel
    if level == nil {
        level = slog.LevelInfo
    }
    slog.SetDefault(slog.New(newLogHandler(&slog.HandlerOptions{Level: level})))

    return nil
}

newSpanExporter, newMetricExporter, and newLogHandler stand in for your service’s implementations of sdktrace.SpanExporter, metric.Exporter, and slog.Handler.

Shutting down cleanly

api.Plugin has no teardown hook, and both a batch span processor and a periodic reader buffer. A process that exits without flushing loses whatever is in those buffers. Expose a Shutdown the application defers:

// Shutdown flushes and stops both pipelines. Call it before the process exits:
//
//    tel := &mytelemetry.Plugin{}
//    g := genkit.Init(ctx, genkit.WithPlugins(tel))
//    defer tel.Shutdown(context.Background())
func (p *Plugin) Shutdown(ctx context.Context) error {
    var errs error
    if p.meterProvider != nil {
        errs = errors.Join(errs, p.meterProvider.Shutdown(ctx))
    }
    return errors.Join(errs, tracing.TracerProvider().Shutdown(ctx))
}

The in-tree googlecloud plugin takes the other route and installs its own SIGINT and SIGTERM handler that flushes and then exits. That works for a server, but it takes the exit path away from the application, so prefer the explicit Shutdown unless you have a reason not to.

PII redaction

Because most generative AI flows begin with user input of some kind, it’s a likely possibility that some flow traces contain personally-identifiable information (PII). To protect your users’ information, you should redact PII from traces before you export them.

If you are building your own span exporter, you can build this functionality into it.

If you’re building your plugin around an existing OpenTelemetry integration, you can wrap the provided span exporter with a custom exporter that carries out this task. For example, the googlecloud plugin removes the genkit:input and genkit:output attributes from every span before exporting them using a wrapper similar to the following:

import (
    "context"

    "go.opentelemetry.io/otel/attribute"
    sdktrace "go.opentelemetry.io/otel/sdk/trace"
)

type redactingSpanExporter struct {
    sdktrace.SpanExporter
}

func (e *redactingSpanExporter) ExportSpans(ctx context.Context, spanData []sdktrace.ReadOnlySpan) error {
    redacted := make([]sdktrace.ReadOnlySpan, 0, len(spanData))
    for _, s := range spanData {
        redacted = append(redacted, redactedSpan{s})
    }
    return e.SpanExporter.ExportSpans(ctx, redacted)
}

func (e *redactingSpanExporter) Shutdown(ctx context.Context) error {
    return e.SpanExporter.Shutdown(ctx)
}

type redactedSpan struct {
    sdktrace.ReadOnlySpan
}

func (s redactedSpan) Attributes() []attribute.KeyValue {
    // Omit input and output, which may contain PII.
    var ts []attribute.KeyValue
    for _, a := range s.ReadOnlySpan.Attributes() {
        if a.Key == "genkit:input" || a.Key == "genkit:output" {
            continue
        }
        ts = append(ts, a)
    }
    return ts
}

The attribute keys are genkit:input and genkit:output, with a colon. Genkit writes them in core/tracing, so a wrapper that filters on any other spelling redacts nothing.

Troubleshooting

If you’re having trouble getting data to show up where you expect, the OpenTelemetry Go documentation covers SDK diagnostics that help locate the source of the problem.

Finding the constructor to call

Every Define* helper that took an api.Registry as its first argument is gone. A registry is unobtainable outside the framework, so those helpers had no reachable callers; the replacement is to construct the action and return it from Plugin.Init, which is what the framework registers. Every plugin-side replacement below is a constructor, so return its result from Plugin.Init. Applications keep reaching the same primitives through the genkit.Define* functions, which construct and register in one call.

Removed	Plugin-side replacement	App-side equivalent
core.DefineAction	core.NewActionOf	None
core.DefineStreamingAction	core.NewStreamingActionOf	None
core.DefineBidiAction	core.NewBidiActionOf	None
core.DefineBackgroundAction	core.NewBackgroundActionOf	None
core.DefineFlow	core.NewFlow	genkit.DefineFlow
core.DefineStreamingFlow	core.NewStreamingFlow	genkit.DefineStreamingFlow
core.DefineSchema	None	genkit.DefineSchema
core.DefineSchemaFor	None	genkit.DefineSchemaFor / genkit.DefineSchemasFor
ai.DefineModel	ai.NewModelAction	genkit.DefineModelAction
ai.DefineBackgroundModel	ai.NewBackgroundModelAction	genkit.DefineBackgroundModelAction
ai.DefineRetriever	ai.NewRetrieverAction	genkit.DefineRetrieverAction
ai.DefineEmbedder	ai.NewEmbedderAction	genkit.DefineEmbedderAction
ai.DefineEvaluator	ai.NewEvaluatorAction	genkit.DefineEvaluatorAction
ai.DefineBatchEvaluator	ai.NewBatchEvaluatorAction	genkit.DefineBatchEvaluatorAction
ai.DefineTool	ai.NewTool	genkit.DefineTool
ai.DefineToolWithInputSchema	ai.NewTool with ai.WithInputSchema	genkit.DefineTool with ai.WithInputSchema
ai.DefineMultipartTool	ai.NewMultipartTool	genkit.DefineMultipartTool
ai.DefineResource	ai.NewResource	genkit.DefineResource
ai.DefineMiddleware	ai.NewMiddleware	genkit.DefineMiddleware
ai.DefineFormat	ai.DefineFormats, which takes the name from each Formatter	genkit.DefineFormats
ai/exp.DefineTool	ai/exp.NewTool	genkit/exp.DefineTool
ai/exp.DefineInterruptibleTool	ai/exp.NewInterruptibleTool	genkit/exp.DefineInterruptibleTool

ai.DefinePrompt, ai.DefineDataPrompt, ai.DefineFormats, and ai.DefineGenerateAction still take a registry: there the registry is a structural input rather than a registration target.

Package core’s own error envelope went with them. core.ReflectionError, core.ToReflectionError, and core.SchemaValidationError were deleted with no exported replacement, and core.NewSchemaValidationError gave way to status.Errorf with a sentinel from core/status.

The flat core.NewAction, NewStreamingAction, NewBidiAction, and NewBackgroundAction constructors still work and are deprecated in favor of their Of counterparts. They take name before atype, so swapping the first two arguments is the likeliest mistake when moving a call across. The untyped ai.NewModel, NewRetriever, NewEmbedder, NewEvaluator, NewBatchEvaluator, and NewBackgroundModel constructors, and the matching genkit.Define* names, are likewise deprecated in favor of the typed-config family: they resolve Config to any, which infers no schema and passes the raw config through untouched.

Publishing a plugin

Genkit plugins can be published as normal Go packages. To increase discoverability, your package should have genkit somewhere in its name so it can be found with a simple search on pkg.go.dev. Any of the following are good choices:

• github.com/yourorg/genkit-plugins/servicename
• github.com/yourorg/your-repo/genkit/servicename

Next steps

• Run your plugin under the Developer UI to check that its models, capability metadata, and config schema look the way you meant.
• Read Error types before you classify your provider’s errors, and Middleware to see what that classification buys the application.
• Read two in-tree plugins as references: googlegenai for a native SDK, and compat_oai for the OpenAI-compatible path.