Generating content with AI models

Genkit provides a unified interface for working with generative AI models from any supported provider. Configure a model plugin once, then call any model through the same API—making it easy to combine multiple models or swap one out as your app evolves.

Before you begin

If you want to run the code examples on this page, first complete the steps in the Get started guide. All of the examples assume that you have already installed Genkit as a dependency in your project.

Loading and configuring model plugins

Before you can use Genkit to start generating content, you need to load and configure a model plugin. If you’re coming from the Get started guide, you’ve already done this. Otherwise, see the Get started guide or the individual plugin’s documentation and follow the steps there before continuing.

Set up a project

The examples on this page assume a Go module with Genkit and the Google AI plugin installed:

go mod init example
go get github.com/firebase/genkit/go
go get github.com/firebase/genkit/go/plugins/googlegenai

&googlegenai.GoogleAI{} reads its credentials from the environment when its APIKey field is empty, consulting GEMINI_API_KEY and then GOOGLE_API_KEY:

export GEMINI_API_KEY=<your key>

The Get started guide covers picking a server framework and wiring Genkit into it.

The genkit.Generate() function

In Genkit, the primary interface through which you interact with generative AI models is the genkit.Generate() function.

The simplest genkit.Generate() call specifies the model you want to use and a text prompt:

package main

import (
    "context"
    "log"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/genkit"
    "github.com/firebase/genkit/go/plugins/googlegenai"
)

func main() {
    ctx := context.Background()

    g := genkit.Init(ctx,
        genkit.WithPlugins(&googlegenai.GoogleAI{}),
        genkit.WithDefaultModel("googleai/gemini-flash-latest"),
    )

    resp, err := genkit.Generate(ctx, g,
        ai.WithPrompt("Invent a menu item for a pirate themed restaurant."),
    )
    if err != nil {
        log.Fatalf("could not generate model response: %v", err)
    }

    log.Println(resp.Text())
}

Convenience function

For simple text generation, you can use genkit.GenerateText() which returns just the text string directly:

text, err := genkit.GenerateText(ctx, g,
    ai.WithPrompt("Invent a menu item for a pirate themed restaurant."),
)

When you run this brief example, it will print out some debugging information followed by the output of the genkit.Generate() call, which will usually be Markdown text as in the following example:

## The Blackheart's Bounty

**A hearty stew of slow-cooked beef, spiced with rum and molasses, served in a
hollowed-out cannonball with a side of crusty bread and a dollop of tangy
pineapple salsa.**

**Description:** This dish is a tribute to the hearty meals enjoyed by pirates
on the high seas. The beef is tender and flavorful, infused with the warm spices
of rum and molasses. The pineapple salsa adds a touch of sweetness and acidity,
balancing the richness of the stew. The cannonball serving vessel adds a fun and
thematic touch, making this dish a perfect choice for any pirate-themed
adventure.

Run the script again and you’ll get a different output.

The preceding code sample sent the generation request to the default model, which you specified when you configured the Genkit instance.

You can also specify a model for a single genkit.Generate() call:

resp, err := genkit.Generate(ctx, g,
    ai.WithModelName("googleai/gemini-pro-latest"),
    ai.WithPrompt("Invent a menu item for a pirate themed restaurant."),
)

A model string identifier looks like providerid/modelid, where the provider ID (in this case, googleai) identifies the plugin, and the model ID is a plugin-specific string identifier for a specific version of a model.

The Google AI and Vertex AI plugins register no models when Genkit starts. Every model ID is resolved the first time you name it, so any model the provider serves works, whether or not the plugin knows about it. The plugin’s curated list decides what the Developer UI offers and what capabilities Genkit assumes for an ID it recognizes. It is a starting point, not a limit.

These examples also illustrate an important point: when you use genkit.Generate() to make generative AI model calls, changing the model you want to use is a matter of passing a different value to the model parameter. By using genkit.Generate() instead of the native model SDKs, you give yourself the flexibility to more easily use several different models in your app and change models in the future.

So far you have only seen examples of the simplest genkit.Generate() calls. However, genkit.Generate() also provides an interface for more advanced interactions with generative models, which you will see in the sections that follow.

Passing options

genkit.Generate() takes ...ai.GenerateOption. ai.CommonGenOption (ai.WithModel(), ai.WithModelName(), ai.WithTools(), ai.WithUse(), ai.WithMaxTurns(), ai.WithMessages(), and the rest) and ai.PromptingOption (ai.WithPrompt(), ai.WithSystem(), and their Parts and Fn variants) both embed ai.GenerateOption, so options of any of these kinds collect into one []ai.GenerateOption and spread into the call:

opts := []ai.GenerateOption{
    ai.WithModelName("googleai/gemini-flash-latest"),
    ai.WithPrompt("Invent a menu item for a pirate themed restaurant."),
    ai.WithStepName("draft"),
}

resp, err := genkit.Generate(ctx, g, opts...)

ai.WithStepName(), ai.WithToolResponses(), and ai.WithToolRestarts() are the three that are ai.GenerateOption and nothing else: they mean nothing to a prompt definition, so genkit.DefinePrompt() does not accept them.

How prompt text is treated

ai.WithPrompt(text string, args ...any) sets the last user message, and ai.WithSystem() has the same shape. Two rules govern the text:

• With no args the text is used verbatim. A % in user input is harmless. Pass args and the text becomes a fmt.Sprintf format string, so ai.WithPrompt("Classify this review: %s", review) is the way to interpolate.
• Handlebars templating applies only under genkit.DefinePrompt(). There the text is compiled as a Dotprompt template against the prompt’s input, so {{field}} resolves. On a plain genkit.Generate() call the braces are sent literally. A {{role}} marker is always an error, because this slot is one message; ai.WithMessagesTemplate(), which genkit.DefinePrompt() takes, is where multi-turn templates belong.

Do not concatenate untrusted text into text. Pass it as a %s argument, or use ai.WithPromptFn() or ai.WithPromptParts(), whose content is never templated.

System prompts

Some models support providing a system prompt, which gives the model instructions as to how you want it to respond to messages from the user. You can use the system prompt to specify characteristics such as a persona you want the model to adopt, the tone of its responses, and the format of its responses.

If the model you’re using supports system prompts, you can provide one with the ai.WithSystem() option:

resp, err := genkit.Generate(ctx, g,
    ai.WithSystem("You are a food industry marketing consultant."),
    ai.WithPrompt("Invent a menu item for a pirate themed restaurant."),
)

For models that don’t support system prompts, ai.WithSystem() simulates it by modifying the request to appear like a system prompt.

Multi-turn conversations with messages

For multi-turn conversations, pass the history with ai.WithMessages(). It fills the slot between the system prompt and the user prompt, so it combines with ai.WithSystem() and ai.WithPrompt() rather than replacing them. Repeating the option appends.

resp, err := genkit.Generate(ctx, g,
    ai.WithSystem("You are a helpful travel assistant."),
    ai.WithMessages(
        ai.NewUserTextMessage("Hello, can you help me plan a trip?"),
        ai.NewModelTextMessage("Of course. Where are you thinking of going?"),
    ),
    ai.WithPrompt("I want to visit Japan for two weeks in spring."),
)

Message text passed this way is used verbatim and is never compiled as a template, so history containing literal braces passes through untouched.

Every message carries an ai.Role, one of four constants:

Constant	Wire value	Meaning
ai.RoleSystem	system	User-independent instructions
ai.RoleUser	user	A turn from the client
ai.RoleModel	model	A turn from the model (Genkit’s name for assistant)
ai.RoleTool	tool	The result of a local tool call

The ai.NewUserTextMessage(), ai.NewModelTextMessage(), and ai.NewSystemTextMessage() constructors set the role for you. ai.NewMessage(role, metadata, parts...) is the general form when you need a different role, metadata, or non-text parts.

For persistent chat sessions with automatic history management, use the Chat API instead of managing the slice yourself.

Model parameters

The genkit.Generate() function takes a ai.WithConfig() option, through which you can specify optional settings that control how the model generates content.

The value you pass is the config type of the model’s own provider SDK, not a Genkit type. For the Google AI and Vertex AI plugins that is *genai.GenerateContentConfig from Google’s GenAI Go SDK, so the snippets below need one more import:

import "google.golang.org/genai"

Anthropic, Ollama, and the OpenAI-compatible plugins each take their own config type; check the plugin’s page for which one.

resp, err := genkit.Generate(ctx, g,
    ai.WithModelName("googleai/gemini-flash-latest"),
    ai.WithPrompt("Invent a menu item for a pirate themed restaurant."),
    ai.WithConfig(&genai.GenerateContentConfig{
        MaxOutputTokens: 500,
        StopSequences:   []string{"<end>", "<fin>"},
        Temperature:     genai.Ptr[float32](0.5),
        TopP:            genai.Ptr[float32](0.4),
        TopK:            genai.Ptr[float32](50),
    }),
)

The exact parameters that are supported depend on the individual model and model API. However, the parameters in the previous example are common to almost every model. The following is an explanation of these parameters:

Parameters that control output length

MaxOutputTokens

LLMs operate on units called tokens. A token usually, but does not necessarily, map to a specific sequence of characters. When you pass a prompt to a model, one of the first steps it takes is to tokenize your prompt string into a sequence of tokens. Then, the LLM generates a sequence of tokens from the tokenized input. Finally, the sequence of tokens gets converted back into text, which is your output.

The maximum output tokens parameter sets a limit on how many tokens to generate using the LLM. Every model potentially uses a different tokenizer, but a good rule of thumb is to consider a single English word to be made of 2 to 4 tokens.

As stated earlier, some tokens might not map to character sequences. One such example is that there is often a token that indicates the end of the sequence: when an LLM generates this token, it stops generating more. Therefore, it’s possible and often the case that an LLM generates fewer tokens than the maximum because it generated the “stop” token.

StopSequences

You can use this parameter to set the tokens or token sequences that, when generated, indicate the end of LLM output. The correct values to use here generally depend on how the model was trained, and are usually set by the model plugin. However, if you have prompted the model to generate another stop sequence, you might specify it here.

Note that you are specifying character sequences, and not tokens per se. In most cases, you will specify a character sequence that the model’s tokenizer maps to a single token.

Parameters that control “creativity”

The temperature, top-p, and top-k parameters together control how “creative” you want the model to be. This section provides very brief explanations of what these parameters mean, but the more important point is this: these parameters are used to adjust the character of an LLM’s output. The optimal values for them depend on your goals and preferences, and are likely to be found only through experimentation.

Temperature

LLMs are fundamentally token-predicting machines. For a given sequence of tokens (such as the prompt) an LLM predicts, for each token in its vocabulary, the likelihood that the token comes next in the sequence. The temperature is a scaling factor by which these predictions are divided before being normalized to a probability between 0 and 1.

Low temperature values—between 0.0 and 1.0—amplify the difference in likelihoods between tokens, with the result that the model will be even less likely to produce a token it already evaluated to be unlikely. This is often perceived as output that is less creative. Although 0.0 is technically not a valid value, many models treat it as a request for greedy decoding: always take the single most likely next token.

High temperature values—those greater than 1.0—compress the differences in likelihoods between tokens, with the result that the model becomes more likely to produce tokens it had previously evaluated to be unlikely. This is often perceived as output that is more creative. Some model APIs impose a maximum temperature, often 2.0.

TopP

Top-p is a value between 0.0 and 1.0 that controls the number of possible tokens you want the model to consider, by specifying the cumulative probability of the tokens. For example, a value of 1.0 means to consider every possible token (but still take into account the probability of each token). A value of 0.4 means to only consider the most likely tokens, whose probabilities add up to 0.4, and to exclude the remaining tokens from consideration.

TopK

Top-k is an integer value that also controls the number of possible tokens you want the model to consider, but this time by explicitly specifying the maximum number of tokens. A value of 1 leaves the model only its most likely token, which is greedy decoding again.

Greedy decoding is not reproducibility

Neither Temperature: 0 nor TopK: 1 makes output reproducible. Providers batch, shard, and update their serving stacks, so the same request at temperature 0 can still come back with different text tomorrow, or on a different replica today. Treat these as controls on style, not as a way to pin an answer.

Seeds

Genkit has no seed option of its own: ai.GenerationCommonConfig, the provider-neutral config struct, covers only API key, max output tokens, stop sequences, temperature, top-k, top-p, and version. A seed is always set through the provider’s config. For the Google plugins:

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Invent a menu item for a pirate themed restaurant."),
    ai.WithConfig(&genai.GenerateContentConfig{
        Temperature: genai.Ptr[float32](0),
        Seed:        genai.Ptr[int32](42),
    }),
)

The Ollama, xAI, DashScope, and OpenRouter configs each carry their own Seed field. Not every provider exposes one. In every case a seed is best effort: it improves the odds that two identical requests agree, and it guarantees nothing.

Experiment with model parameters

You can experiment with the effect of these parameters on the output generated by different model and prompt combinations by using the Developer UI. Start the developer UI with the genkit start command and it will automatically load all of the models defined by the plugins configured in your project. You can quickly try different prompts and configuration values without having to repeatedly make these changes in code.

Pair model with its config

Given that each provider or even a specific model may have its own configuration schema or warrant certain settings, it may be error prone to set separate options using ai.WithModelName() and ai.WithConfig() since the latter is not strongly typed to the former.

To pair a model with its config, you can create a model reference that you can pass into the generate call instead:

model := googlegenai.ModelRef("googleai/gemini-flash-latest", &genai.GenerateContentConfig{
    MaxOutputTokens: 500,
    StopSequences:   []string{"<end>", "<fin>"},
    Temperature:     genai.Ptr[float32](0.5),
    TopP:            genai.Ptr[float32](0.4),
    TopK:            genai.Ptr[float32](50),
})

resp, err := genkit.Generate(ctx, g,
    ai.WithModel(model),
    ai.WithPrompt("Invent a menu item for a pirate themed restaurant."),
)
if err != nil {
    log.Fatal(err)
}

The constructor for the model reference will enforce that the correct config type is provided which may reduce mismatches.

Model versions and aliases

Model aliases like gemini-flash-latest point to the current release of a model. While aliases are convenient during development and prototyping, model providers periodically update which snapshot an alias references, which can affect latency, cost, and output consistency.

For production workloads where consistent behavior is desired, you can specify a dated or specific model version snapshot:

resp, err := genkit.Generate(ctx, g,
    ai.WithModelName("googleai/gemini-3.8-flash"),
    ai.WithPrompt("Invent a menu item for a pirate themed restaurant."),
)

Because plugins like Google AI and Vertex AI resolve model IDs dynamically, specific model versions work directly without requiring updates to the plugin definition. When updating model versions in production, consider testing changes against an evaluation dataset before deploying.

Structured output

When using generative AI as a component in your application, you often want output in a format other than plain text. Even if you’re just generating content to display to the user, you can benefit from structured output simply for the purpose of presenting it more attractively to the user. But for more advanced applications of generative AI, such as programmatic use of the model’s output, or feeding the output of one model into another, structured output is a must.

In Genkit, you can request structured output from a model by specifying an output type when you call genkit.Generate():

type MenuItem struct {
    Name        string   `json:"name"`
    Description string   `json:"description"`
    Calories    int      `json:"calories"`
    Allergens   []string `json:"allergens"`
}

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Invent a menu item for a pirate themed restaurant."),
    ai.WithOutputType(MenuItem{}),
)
if err != nil {
    log.Fatal(err) // One possible error is that the response does not conform to the type.
}

Model output types are specified as JSON schema using the invopop/jsonschema package. This provides runtime type checking, which bridges the gap between static Go types and the unpredictable output of generative AI models. This system lets you write code that can rely on the fact that a successful generate call will always return output that conforms to your Go types.

When you specify an output type in genkit.Generate(), Genkit does several things behind the scenes:

• Augments the prompt with additional guidance about the selected output format. This also has the side effect of specifying to the model what content exactly you want to generate (for example, not only suggest a menu item but also generate a description, a list of allergens, and so on).
• Verifies that the output conforms to the schema.
• Marshals the model output into a Go type.

To get structured output from a successful generate call, call Output() on the model response with an empty value of the type:

var item MenuItem
if err := resp.Output(&item); err != nil {
    log.Fatal(err)
}

log.Printf("%s (%d calories, %d allergens): %s\n",
    item.Name, item.Calories, len(item.Allergens), item.Description)

Alternatively, you can use genkit.GenerateData() for a more succinct call:

item, resp, err := genkit.GenerateData[MenuItem](ctx, g,
    ai.WithPrompt("Invent a menu item for a pirate themed restaurant."),
)
if err != nil {
    log.Fatal(err)
}
if item == nil {
    // The response carried no text to parse. resp says why.
    log.Fatalf("no menu item: finish reason %q, %d interrupts, %d tool requests",
        resp.FinishReason, len(resp.Interrupts()), len(resp.ToolRequests()))
}

log.Printf("%s (%d calories, %d allergens): %s\n",
    item.Name, item.Calories, len(item.Allergens), item.Description)

This function requires the output type parameter but automatically sets the ai.WithOutputType() option and calls ModelResponse.Output() before returning the value.

Check the three results in order: the error, then the value, then the fields. genkit.GenerateData() returns a nil value with a live response and no error whenever the response carried no text to parse, which is what a turn holding a tool request, an interrupt, or media looks like. That is a legitimate answer, not a failure, so it is yours to interpret: read resp.Interrupts(), resp.ToolRequests(), and resp.FinishReason. A refusal is the exception: a blocked finish returns ai.ErrGenerationBlocked with the response beside it, since a nil value with no error would read as success. Check err first either way. resp is nil only when the request failed before the model was called, such as for an unknown model.

The basic-structured sample carries the whole pattern, including the streaming form.

Using registered schemas

For schemas that are shared across your application (such as those used in .prompt files), you can register them with genkit.DefineSchemasFor() and reference them by name. Each value registers a schema under its Go type’s name, so one call covers as many types as your app has:

// Register the schemas once at startup
genkit.DefineSchemasFor(g, MenuItem{}, MenuRequest{})

// Reference by name in generate calls
resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Invent a menu item for a pirate themed restaurant."),
    ai.WithOutputSchemaName("MenuItem"),
)

genkit.DefineSchemaFor[T](g) is the single-type form of the same thing, and genkit.DefineSchema(g, name, schema) registers a schema you wrote by hand under a name of your choosing. All three live in package genkit.

This is particularly useful when working with Dotprompt, as you can define your types once in Go and reference them in .prompt files by name, avoiding duplicate schema definitions.

Handling errors

Note in the prior example that the genkit.Generate() call can result in an error. One possible error can happen when the model fails to generate output that conforms to the schema. The best strategy for dealing with such errors will depend on your exact use case, but here are some general hints:

• Try a different model. For structured output to succeed, the model must be capable of generating output in JSON. The most powerful LLMs like Gemini are versatile enough to do this; however, smaller models, such as some of the local models you would use with Ollama, might not be able to generate structured output reliably unless they have been specifically trained to do so.

• Simplify the schema. LLMs may have trouble generating complex or deeply nested types. Try using clear names, fewer fields, or a flattened structure if you are not able to reliably generate structured data.

• Ask again with the error. A model that produced almost-valid JSON usually fixes it when told what was wrong, so a bounded repair loop beats a plain retry.

A schema failure arrives as status.ErrInvalidOutput, which you match with errors.Is. Feed the validation error back into the next attempt:

import (
    "errors"
    "fmt"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/core/status"
    "github.com/firebase/genkit/go/genkit"
    "github.com/firebase/genkit/go/plugins/middleware"
)

const task = "Invent a menu item for a pirate themed restaurant."

opts := []ai.GenerateOption{ai.WithPrompt(task)}

var item *MenuItem
var err error

for attempt := 0; attempt < 3; attempt++ {
    item, _, err = genkit.GenerateData[MenuItem](ctx, g, opts...)
    if err == nil {
        break
    }
    if !errors.Is(err, status.ErrInvalidOutput) {
        return err // something other than a schema failure
    }
    // Tell the model what was wrong with its last attempt and ask again.
    opts = append(opts, ai.WithPrompt(
        task+" Your previous reply did not match the required schema: %v. "+
            "Reply with valid JSON only, no prose and no code fence.", err))
}
if err != nil {
    return fmt.Errorf("no conforming output after 3 attempts: %w", err)
}

Only one prompt survives: ai.WithPrompt() shares a single slot with the other prompt options, and the last one set wins, so each pass replaces the instruction rather than stacking another.

The response comes back beside the error. A schema failure happens after the model has finished, so genkit.Generate() hands back the response with its original message and finish reason, and resp.Text() holds the output that failed to parse. The loop above feeds the model only the validation error, which names the field that failed; showing it the text it produced is the other option.

Retry middleware does not cover this

middleware.Retry hooks the Model stage only: it retries individual model API calls. Output-schema validation runs in the action layer above it, so the ErrInvalidOutput never reaches the hook, and re-issuing the identical request with no feedback would not fix it anyway. Use the loop above for nonconformance, and the middleware for transport failures (UNAVAILABLE, DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED, ABORTED, INTERNAL):

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Invent a menu item for a pirate themed restaurant."),
    ai.WithUse(&middleware.Retry{MaxRetries: 2}),
)

See Middleware, which also covers middleware.Fallback.

Blocked and interrupted responses

When the model stops for a reason other than finishing its answer, holding it to the schema would report the wrong problem: you would see a parse failure where the real news is that a safety filter fired or a tool paused the turn. So Genkit skips the parsing step that rewrites the message whenever the finish reason is blocked, aborted, interrupted, or the catch-all other. The finish reason survives to you instead. unknown is deliberately not in that set, because plugins map any provider reason they do not recognize to it.

What that means at the call site:

• A refusal is an error from the typed helpers. genkit.GenerateData(), genkit.GenerateDataStream(), and the DataPrompt execute methods return ai.ErrGenerationBlocked when the finish reason is blocked, carrying the provider’s explanation and with the response beside the error, because a zero value with no error would read as success. genkit.Generate() still hands a blocked response back as a value, so read resp.FinishReason there.
• The other abnormal finishes are not failures. genkit.GenerateData() gives you a nil value, the response, and no error when the response carried no text to parse, which is also what a turn holding only tool requests or an interrupt looks like; read resp.Interrupts() for a pause. genkit.GenerateDataStream() ends with a final value whose Output is the zero value of your type, so check val.Response there rather than the output.
• Streamed chunks are provisional. They parse as they arrive, before any finish reason exists, so a generation that streams half a value and then blocks has already yielded a populated chunk. The final value settles it.
• A response that stopped early but still carries conforming text parses as usual, so the skip costs you nothing in the common case.

Output formats

The output format decides two things: how the model is asked to write its answer, and how that answer is parsed back into Go values. Genkit registers exactly five, and any of them can be selected explicitly:

Format	Select with	You get back
text	the default when you set no output type	the raw text, unparsed
json	the default when you set an output type	one value matching the schema
jsonl	ai.WithOutputFormat(ai.OutputFormatJSONL)	a slice, written one item per line
array	ai.WithOutputFormat(ai.OutputFormatArray)	a slice, written as one JSON array
enum	ai.WithOutputEnums("yes", "no")	one string out of a fixed set

jsonl and array need an array schema, so the output type has to be a slice. ai.WithOutputEnums() sets the schema and the format together, so it is the whole of what an enum output needs. Selecting a name that is not registered fails with INVALID_ARGUMENT before the model is ever called.

A format does not replace your schema: the schema still comes from the output type, and only the way the model is asked to write it out changes.

// One item per line instead of one JSON array.
for val, err := range genkit.GenerateDataStream[[]MenuItem](ctx, g,
    ai.WithOutputFormat(ai.OutputFormatJSONL),
    ai.WithPrompt("Invent four menu items for a pirate themed restaurant."),
) {
    if err != nil {
        log.Fatal(err)
    }
    if val.Done {
        log.Printf("%d items\n", len(val.Output))
        break
    }
    for _, item := range val.Chunk {
        log.Println(item.Name)
    }
}

The basic-formats sample puts json, jsonl, and enum side by side over one story premise, a flow for each.

Enum output

ai.WithOutputEnums() is the whole of what a classification needs: it sets the format and the schema together, and the answer comes back as text.

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Classify the sentiment of this review: %s", review),
    ai.WithOutputEnums("positive", "negative", "neutral"),
)
if err != nil {
    log.Fatal(err)
}

sentiment := resp.Text()

The signature is ai.WithOutputEnums[T ~string](values ...T), so your own string type works and keeps the labels in one place:

type Sentiment string

const (
    Positive Sentiment = "positive"
    Negative Sentiment = "negative"
    Neutral  Sentiment = "neutral"
)

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Classify the sentiment of this review: %s", review),
    ai.WithOutputEnums(Positive, Negative, Neutral),
)

A reply outside the set fails with a status.ErrInvalidOutput error reading message <reply> not in list of valid enums: ..., so the value you read is always one of the labels you supplied.

What a chunk means, per format

The format decides what one streamed chunk contains, which is the part that catches people out. Parsing a chunk with chunk.Output() or reading val.Chunk from genkit.GenerateDataStream() gives you:

• text: everything accumulated so far. Use chunk.Text() instead if you want only the text that just arrived.
• json: the whole value so far, filling in field by field. Each chunk supersedes the one before it, so replace what you are holding rather than appending to it. Partial string values are normal mid-stream.
• jsonl: the items that finished since the last chunk, plus the item still being written. That trailing item arrives again, further along, on the next chunk, so a consumer that wants only finished items has to spot the repeat.
• array: only the items that became complete since the last chunk, and never a half-written one. items = append(items, val.Chunk...) is correct, and an empty first chunk is normal.
• enum: the empty string until the whole value has arrived. There is effectively nothing to stream, so do not put a progress indicator on it.

The final response differs too. json, jsonl, and enum validate it against the schema, so a missing required field or a label outside the set is an error. array does not, so a missing field reaches you as a zero Go field rather than as a failure.

Custom formats

Register your own format with genkit.DefineFormats(), then select it by name. The name comes from the formatter’s own Name() method.

A format is two types. ai.Formatter is the registered one, and it is a factory: Genkit calls its Handler() once per request to get an ai.FormatHandler that owns that request’s parsing state.

import (
    "strings"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/genkit"
)

// csvFormatter asks the model for comma-separated values.
type csvFormatter struct{}

var _ ai.Formatter = csvFormatter{}

func (csvFormatter) Name() string { return "csv" }

// schema is the JSON Schema of the requested output type, or nil when the
// request asked for no type. Echo it back in Config().Schema if the model
// should see it.
func (csvFormatter) Handler(schema map[string]any) (ai.FormatHandler, error) {
    return &csvHandler{}, nil
}

type csvHandler struct {
    text   string // text accumulated for the turn being parsed
    index  int    // the index of that turn
    cursor int    // how much of text has already been handed over
}

func (h *csvHandler) Instructions() string {
    return "Output ONLY comma-separated values on a single line. No prose, no code fences."
}

func (h *csvHandler) Config() ai.ModelOutputConfig {
    return ai.ModelOutputConfig{Format: "csv", ContentType: "text/csv"}
}

// ParseMessage is a passthrough: parsing belongs in ParseOutput.
func (h *csvHandler) ParseMessage(m *ai.Message) (*ai.Message, error) { return m, nil }

// ParseOutput parses the final message: every field, in order.
func (h *csvHandler) ParseOutput(m *ai.Message) (any, error) {
    return strings.Split(m.Text(), ","), nil
}

// ParseChunk returns only the fields completed since the previous chunk. The
// handler is reused across turns, so it resets when chunk.Index changes.
func (h *csvHandler) ParseChunk(chunk *ai.ModelResponseChunk) (any, error) {
    if chunk.Index != h.index {
        h.text, h.index, h.cursor = "", chunk.Index, 0
    }
    for _, p := range chunk.Content {
        if p.IsText() {
            h.text += p.Text
        }
    }
    done := strings.LastIndex(h.text, ",") // a field is complete once its comma arrives
    if done < h.cursor {
        return []string{}, nil
    }
    fresh := strings.Split(h.text[h.cursor:done], ",")
    h.cursor = done + 1
    return fresh, nil
}

genkit.DefineFormats(g, csvFormatter{})

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("List three colors."),
    ai.WithOutputFormat("csv"),
)
if err != nil {
    log.Fatal(err)
}

var colors []string
if err := resp.Output(&colors); err != nil {
    log.Fatal(err)
}

Instructions(), Config(), and ParseMessage() make up ai.FormatHandler, the minimum. ParseOutput() and ParseChunk() add ai.StreamingFormatHandler, and without them both resp.Output() and chunk.Output() fail. A name can be claimed only once: registering one that is already taken panics, and that includes the five built-ins, so they cannot be replaced.

The ai.ModelOutputConfig your handler returns from Config() has four fields. Format and ContentType are what the CSV handler above sets. The other two decide how the model learns the shape of the answer: Schema is the JSON Schema to send, normally the map handed to Handler(), and Constrained says the schema can be enforced natively rather than described in prose. Constrained is a request, not a switch. Genkit keeps it only when the caller asked for constrained output, a schema exists, and the model declares support; otherwise it clears the flag, drops Schema, and injects your Instructions() into the prompt instead.

Streaming

When generating large amounts of text, you can improve the experience for your users by presenting the output as it’s generated—streaming the output. A familiar example of streaming in action can be seen in most LLM chat apps: users can read the model’s response to their message as it’s being generated, which improves the perceived responsiveness of the application and enhances the illusion of chatting with an intelligent counterpart.

There are two shapes, and the question that picks between them is whether your code has anything to do with the chunks. If you are only passing them on to your own caller, hand your callback to ai.WithStreaming() and let the chunks travel untouched. If you have to look at them, range over genkit.GenerateStream().

Iterator-based streaming

Use genkit.GenerateStream() when the caller has to act on chunks as they arrive. It returns an iterator you can range over:

stream := genkit.GenerateStream(ctx, g,
    ai.WithPrompt("Suggest a complete menu for a pirate themed restaurant."),
)

for result, err := range stream {
    if err != nil {
        log.Fatal(err)
    }
    if result.Done {
        // Final response is available
        log.Println("Complete response:", result.Response.Text())
        break
    }
    // Just the text that arrived with this chunk
    log.Println(result.Chunk.Text())
}

The iterator yields *ai.ModelStreamValue values, where:

• result.Chunk contains the streamed chunk data
• result.Done indicates whether this is the final result
• result.Response contains the complete response (only available when Done is true)

Text is a delta, Output is a running total

chunk.Text() is only the text that arrived with that chunk, so printing it in a loop reproduces the message once. Parsing the chunk is the opposite: chunk.Output(&v) gives you everything accumulated so far, so printing that reprints the message from the start on every chunk. Exactly what “so far” means depends on the output format, covered under what a chunk means, per format.

Streaming structured output

For streaming structured output with strong typing, use genkit.GenerateDataStream[T]():

type MenuItem struct {
    Name        string `json:"name"`
    Description string `json:"description"`
}

stream := genkit.GenerateDataStream[MenuItem](ctx, g,
    ai.WithPrompt("Invent a menu item for a pirate themed restaurant."),
)

for result, err := range stream {
    if err != nil {
        log.Fatal(err)
    }
    if result.Done {
        // result.Output is strongly typed as MenuItem
        log.Printf("Final: %s - %s\n", result.Output.Name, result.Output.Description)
        break
    }
    // result.Chunk is also strongly typed as MenuItem, holding everything
    // parsed so far
    if result.Chunk.Name != "" {
        log.Printf("Got name: %s\n", result.Chunk.Name)
    }
}

With GenerateDataStream[T], both the streamed chunks and the final output are strongly typed, making your code safer and more predictable.

Ask for the value type, GenerateDataStream[MenuItem], rather than the pointer type. The two behave differently on chunks that parse to nothing, which is what a code fence or a line of prose ahead of the JSON looks like: those chunks are dropped only when the type parameter can be nil, so [*MenuItem] filters them while [MenuItem] delivers a zero-value struct. Reading a half-filled value the same way as one whose fields have not arrived yet is the simpler contract, and it is what basic-structured uses. Guard on a field you care about, as above, rather than assuming every chunk carries something new.

Callback-based streaming

When your code is only handing the chunks onward, pass the callback to ai.WithStreaming() and let genkit.Generate() return the finished response as usual. Inside a streaming flow this is the whole job, because the flow’s own sendChunk is already the callback the option wants:

genkit.DefineStreamingFlow(g, "menuFlow",
    func(ctx context.Context, topic string, sendChunk ai.ModelStreamCallback) (string, error) {
        resp, err := genkit.Generate(ctx, g,
            ai.WithPrompt("Suggest a complete menu for a %s themed restaurant.", topic),
            ai.WithStreaming(sendChunk),
        )
        if err != nil {
            return "", err
        }
        return resp.Text(), nil
    },
)

The basic sample puts that flow next to a non-streaming one, so the pair shows what streaming does and does not change.

The callback is an ordinary function, so use it anywhere you want to process chunks inline or feed callback-based code you already have:

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Suggest a complete menu for a pirate themed restaurant."),
    ai.WithStreaming(func(ctx context.Context, chunk *ai.ModelResponseChunk) error {
        // Process each chunk as it arrives
        log.Println(chunk.Text())
        return nil
    }),
)
if err != nil {
    log.Fatal(err)
}

log.Println(resp.Text())

Multimodal input

The examples you’ve seen so far have used text strings as model prompts. While this remains the most common way to prompt generative AI models, many models can also accept other media as prompts. Media prompts are most often used in conjunction with text prompts that instruct the model to perform some operation on the media, such as to caption an image or transcribe an audio recording.

The ability to accept media input and the types of media you can use are completely dependent on the model and its API. For example, the Gemini 2.5 series of models can accept images, video, and audio as prompts.

To provide a media prompt to a model that supports it, use ai.WithPromptParts() instead of ai.WithPrompt(). It fills the same user prompt slot but takes parts rather than text, so a picture and a question travel together as one turn. This example specifies an image using a publicly accessible HTTPS URL.

resp, err := genkit.Generate(ctx, g,
    ai.WithModelName("googleai/gemini-flash-latest"),
    ai.WithPromptParts(
        ai.NewTextPart("Compose a poem about this image."),
        ai.NewMediaPart("image/jpeg", "https://example.com/photo.jpg"),
    ),
)

You can also pass media data directly by encoding it as a data URL. For example:

image, err := os.ReadFile("photo.jpg")
if err != nil {
    log.Fatal(err)
}

resp, err := genkit.Generate(ctx, g,
    ai.WithModelName("googleai/gemini-flash-latest"),
    ai.WithPromptParts(
        ai.NewTextPart("Compose a poem about this image."),
        ai.NewMediaPart("image/jpeg", "data:image/jpeg;base64,"+base64.StdEncoding.EncodeToString(image)),
    ),
)

All models that support media input support both data URLs and HTTPS URLs. Some model plugins add support for other media sources. For example, the Vertex AI plugin also lets you use Cloud Storage (gs://) URLs.

ai.WithMessages() is still how you supply the turns leading up to the prompt, and those messages can carry media parts of their own. The two options fill different slots, so a request can use both:

resp, err := genkit.Generate(ctx, g,
    ai.WithModelName("googleai/gemini-flash-latest"),
    ai.WithMessages(history...),
    ai.WithPromptParts(
        ai.NewTextPart("Compose a poem about this image."),
        ai.NewMediaPart("image/jpeg", "https://example.com/photo.jpg"),
    ),
)

The basic-media sample covers both ways to attach a picture, and goes on to editing, generating, and animating one.

What a media part holds

ai.NewMediaPart(mimeType, contents) builds a *ai.Part with two fields set: ContentType holds the MIME type, and Text holds the URL, whether that is an https: URL or a data: URI. There is no separate URL field. Read one back with the IsMedia() predicate:

for _, p := range resp.Message.Content {
    if p.IsMedia() {
        log.Printf("%s at %s", p.ContentType, p.Text)
    }
}

IsImage(), IsAudio(), and IsVideo() narrow by MIME type prefix. On a response, resp.MediaParts() returns every media part directly, and resp.Media() returns the URL of the first one as a string, discarding its content type.

Generating media

Image, video, and speech models answer with media parts rather than text, so the same genkit.Generate() call covers them. This example generates an image and writes it to disk:

package main

import (
    "context"
    "encoding/base64"
    "fmt"
    "log"
    "os"
    "strings"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/genkit"
    "github.com/firebase/genkit/go/plugins/googlegenai"
)

func main() {
    ctx := context.Background()
    g := genkit.Init(ctx, genkit.WithPlugins(&googlegenai.GoogleAI{}))

    resp, err := genkit.Generate(ctx, g,
        ai.WithModelName("googleai/imagen-4.0-generate-001"),
        ai.WithPrompt("An illustration of a dog wearing a space suit, photorealistic."),
    )
    if err != nil {
        log.Fatal(err)
    }

    parts := resp.MediaParts()
    if len(parts) == 0 {
        log.Fatalf("no image returned: finish reason %q", resp.FinishReason)
    }

    data, err := mediaBytes(parts[0])
    if err != nil {
        log.Fatal(err)
    }
    if err := os.WriteFile("dog.png", data, 0o644); err != nil {
        log.Fatal(err)
    }
}

// mediaBytes decodes a media part's payload. Generated media normally arrives
// as a "data:" URI, but a provider that stores the result returns an "https:"
// file URI instead, so branch on the prefix.
func mediaBytes(p *ai.Part) ([]byte, error) {
    if !strings.HasPrefix(p.Text, "data:") {
        return nil, fmt.Errorf("media is hosted at %s; fetch it over HTTP", p.Text)
    }
    _, encoded, ok := strings.Cut(p.Text, ",")
    if !ok {
        return nil, fmt.Errorf("malformed data URI")
    }
    return base64.StdEncoding.DecodeString(encoded)
}

Text-to-speech works the same way. The difference is the config: a TTS model needs the audio modality and a voice, which are provider settings rather than Genkit ones.

resp, err := genkit.Generate(ctx, g,
    ai.WithModelName("googleai/gemini-2.5-flash-preview-tts"),
    ai.WithConfig(&genai.GenerateContentConfig{
        ResponseModalities: []string{"AUDIO"},
        SpeechConfig: &genai.SpeechConfig{
            VoiceConfig: &genai.VoiceConfig{
                PrebuiltVoiceConfig: &genai.PrebuiltVoiceConfig{VoiceName: "Algenib"},
            },
        },
    }),
    ai.WithPrompt("Say that Genkit is an amazing AI framework."),
)
if err != nil {
    log.Fatal(err)
}

parts := resp.MediaParts()
if len(parts) == 0 {
    log.Fatalf("no audio returned: finish reason %q", resp.FinishReason)
}
data, err := mediaBytes(parts[0])
if err != nil {
    log.Fatal(err)
}
if err := os.WriteFile("output.wav", data, 0o644); err != nil {
    log.Fatal(err)
}

See the Google GenAI plugin page for the Imagen, Veo, and TTS model IDs each backend serves.

Reasoning

Models that expose their intermediate thinking return it as reasoning parts, separate from the answer. resp.Reasoning() concatenates them:

if thinking := resp.Reasoning(); thinking != "" {
    log.Printf("model reasoning:\n%s", thinking)
}

p.IsReasoning() identifies one part at a time, and ai.NewReasoningPart(text, signature) builds one. The signature is the opaque blob some providers attach to prove the reasoning is theirs; pass nil when there is none. If you replay history yourself rather than using resp.History(), carry the signature back unchanged on the next turn, or the provider may reject the request.

Reasoning tokens are billed and are reported separately in resp.Usage.ThoughtsTokens.

Resources

A resource is content addressed by URI that a prompt can reference by name instead of embedding inline. Define one with genkit.DefineResource():

genkit.DefineResource(g, "company-docs", &ai.ResourceOptions{
    URI:         "file:///docs/handbook.pdf",
    Description: "Company handbook",
}, func(ctx context.Context, in *ai.ResourceInput) (*ai.ResourceOutput, error) {
    content, err := os.ReadFile("/docs/handbook.pdf")
    if err != nil {
        return nil, err
    }
    return &ai.ResourceOutput{
        Content: []*ai.Part{ai.NewTextPart(string(content))},
    }, nil
})

ai.ResourceOptions.URI and ai.ResourceOptions.Template are mutually exclusive. URI matches one exact address. Template is a URI template that matches a family of them, and the captured segments arrive as ai.ResourceInput.Variables, a map[string]string, alongside the full ai.ResourceInput.URI:

genkit.DefineResource(g, "user-profile", &ai.ResourceOptions{
    Template:    "profile://users/{userID}",
    Description: "A user's profile",
}, func(ctx context.Context, in *ai.ResourceInput) (*ai.ResourceOutput, error) {
    profile, err := loadProfile(ctx, in.Variables["userID"])
    if err != nil {
        return nil, err
    }
    return &ai.ResourceOutput{
        Content: []*ai.Part{ai.NewTextPart(profile)},
    }, nil
})

The handler always returns ai.ResourceOutput.Content as []*ai.Part, so a resource can serve media as easily as text.

Reference a resource from a request with ai.NewResourcePart(uri). For a resource you do not want in the registry, build it with ai.NewResource() and attach it to a single call with ai.WithResources(), which appends when repeated:

scratch := ai.NewResource("scratch", &ai.ResourceOptions{
    URI: "mem:///scratch",
}, func(ctx context.Context, in *ai.ResourceInput) (*ai.ResourceOutput, error) {
    return &ai.ResourceOutput{Content: []*ai.Part{ai.NewTextPart(notes)}}, nil
})

resp, err := genkit.Generate(ctx, g,
    ai.WithResources(scratch),
    ai.WithPromptParts(
        ai.NewTextPart("Summarize these notes."),
        ai.NewResourcePart("mem:///scratch"),
    ),
)

Resources attached this way live in a temporary registry for the duration of the request and are discarded afterward.

Token usage and cost

resp.Usage reports what the request consumed. It is a *ai.GenerationUsage and can be nil, because a provider that reports nothing leaves it unset:

if u := resp.Usage; u != nil {
    log.Printf("in=%d out=%d total=%d thoughts=%d cached=%d",
        u.InputTokens, u.OutputTokens, u.TotalTokens,
        u.ThoughtsTokens, u.CachedContentTokens)
}

Field	Type	What it counts
InputTokens, OutputTokens, TotalTokens	int	The usual billing counters
ThoughtsTokens	int	Reasoning tokens, billed but not returned
CachedContentTokens	int	Input tokens served from the provider’s cache
InputCharacters, InputImages, InputVideos, InputAudioFiles	int	Non-token input units some providers bill on
OutputCharacters, OutputImages, OutputVideos, OutputAudioFiles	int	The output twins of those
Custom	map[string]float64	Provider-specific metrics

Every field is omitempty, so a zero means “not reported” as often as it means zero. Read Custom with the two-value map form rather than trusting a zero.

A tool loop reports its last turn only

The *ai.ModelResponse you get back carries the usage of the final model call. Genkit does not accumulate usage across the turns of a tool loop, so billing a whole loop means summing the turns yourself. Attach middleware to see each model call, as in the metering example on the Middleware page.

Caching

Providers discount input tokens they have already processed. A hit shows up as a nonzero resp.Usage.CachedContentTokens, which is the only reliable way to confirm caching is working.

Plugin	Caching in Go
Google AI and Vertex AI	Implicit on Gemini 2.5 and later, on by default with no storage charge. Explicit through WithCacheTTL() on the last message you want cached, which bills a cache resource for its lifetime in exchange for a guaranteed hit.
OpenAI-compatible	Implicit, decided by the provider. xAI adds a PromptCacheKey config field that routes matching prefixes to the same backend.
Anthropic	Not available. The plugin reports CachedContentTokens on a hit but has no way to place a cache breakpoint.

For the Google plugins, mark the boundary on the message:

resp, err := genkit.Generate(ctx, g,
    ai.WithModelName("googleai/gemini-flash-latest"),
    ai.WithMessages(
        ai.NewUserTextMessage(handbook).WithCacheTTL(300), // cache everything up to here for 5 minutes
    ),
    ai.WithPrompt("What is the parental leave policy?"),
)

The marked prefix is uploaded once and referenced by name afterwards rather than resent, so the request carries only the messages after the marker, and a later request that replays the history reuses the same cache. Explicit caching is exclusive with tools and with system prompts: a request that marks a message for caching and also carries either one is rejected with INVALID_ARGUMENT. Not every Gemini model version supports it, so check the provider’s documentation before you rely on it, and see Context caching for reusing a cache by name.

Failed and stopped generations

Once the request has resolved, an error comes back beside a partial *ai.ModelResponse rather than a nil one, so the work the tool loop completed is not lost with the call that ended it:

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Plan the trip."),
    ai.WithTools(searchFlights, bookHotel),
    ai.WithMaxTurns(3),
)
if err != nil && resp != nil {
    transcript := resp.History() // The completed rounds. Send them again to retry.
    cause := resp.Error          // The same failure, classified.
    log.Printf("stopped after %d messages: %s", len(transcript), cause.Status)
}

FinishReason says which kind of stop it was. ai.FinishReasonFailed means something broke, a model call or a tool. ai.FinishReasonAborted means the caller stopped the loop, through a cancelled context, an expired deadline, or a limit it set such as ai.WithMaxTurns. FinishMessage carries the cause as text and resp.Error carries it classified, so a response that travelled as data, in a trace or a persisted turn, still says why it stopped without anyone matching a string.

History() ends at a turn seam: the completed rounds of a model message and the tool message answering it, and nothing from the turn that failed. A failed tool discards its whole round, the model message that requested it and the siblings that succeeded included, because no provider accepts a conversation that ends in an unanswered tool request; Message is nil on such a response. Send the history back with ai.WithMessages() to retry the failed step without repeating the tool calls that already succeeded. Text streamed before the failure reached your callback and the trace, but not the response.

The stream helpers keep the same contract: genkit.GenerateStream() and genkit.GenerateDataStream() yield the error beside a final value that is Done and carries the partial in Response. Errors raised before the request was sent, such as an unknown model or an invalid option, still come with a nil response, so check resp before reading it.

Timeouts and cancellation

Genkit has no per-generation timeout option. The context you pass to genkit.Generate() is the mechanism, and it reaches the provider’s HTTP request:

ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Invent a menu item for a pirate themed restaurant."),
)

Cancelling the context aborts the in-flight provider call and ends the tool loop. The response comes back beside the error, marked ai.FinishReasonAborted and holding the rounds that completed. An expired deadline classifies as status.DeadlineExceeded; an explicit cancel classifies as status.Cancelled. See Error types.

Put the deadline outside Retry, not under it

status.DeadlineExceeded is on middleware.Retry’s default status list, so a deadline that fires inside a retried call looks retryable: the middleware waits out its backoff and tries again against a context that is already dead, until MaxRetries is exhausted. status.Cancelled is not on that list, so an explicit cancel stops immediately. Either derive the deadline in the caller, above the genkit.Generate() that carries ai.WithUse(&middleware.Retry{...}), or drop status.DeadlineExceeded from Retry.Statuses.

Concurrency, cancellation, and lifecycle covers how deadlines propagate through flows, tools, and streams.

Next steps

Learn more about Genkit

• As an app developer, the primary way you influence the output of generative AI models is through prompting. Read Managing prompts with Dotprompt to learn how Genkit helps you develop effective prompts and manage them in your codebase.
• Although genkit.Generate() is the nucleus of every generative AI powered application, real-world applications usually require additional work before and after invoking a generative AI model. To reflect this, Genkit introduces the concept of flows, which are defined like functions but add additional features such as observability and simplified deployment. To learn more, see Defining AI workflows.

Advanced LLM use

There are techniques your app can use to reap even more benefit from LLMs.

• One way to enhance the capabilities of LLMs is to prompt them with a list of ways they can request more information from you, or request you to perform some action. This is known as tool calling or function calling. Models that are trained to support this capability can respond to a prompt with a specially-formatted response, which indicates to the calling application that it should perform some action and send the result back to the LLM along with the original prompt. Genkit has library functions that automate both the prompt generation and the call-response loop elements of a tool calling implementation. See Tool calling to learn more.
• Retrieval-augmented generation (RAG) is a technique used to introduce domain-specific information into a model’s output. This is accomplished by inserting relevant information into a prompt before passing it on to the language model. A complete RAG implementation requires you to bring several technologies together: text embedding generation models, vector databases, and large language models. See Retrieval-augmented generation (RAG) to learn how Genkit simplifies the process of coordinating these various elements.