Tool calling

Tool calling, also known as function calling, is a structured way to give LLMs the ability to make requests back to the application that called it. You define the tools you want to make available to the model, and the model will make tool requests to your app as necessary to fulfill the prompts you give it.

The use cases of tool calling generally fall into a few themes:

Giving an LLM access to information it wasn’t trained with

Frequently changing information, such as a stock price or the current weather.
Information specific to your app domain, such as product information or user profiles.

Note the overlap with retrieval augmented generation (RAG), which is also a way to let an LLM integrate factual information into its generations. RAG is a heavier solution that is most suited when you have a large amount of information or the information that’s most relevant to a prompt is ambiguous. On the other hand, if a function call or database lookup is all that’s necessary for retrieving the information the LLM needs, tool calling is more appropriate.

Introducing a degree of determinism into an LLM workflow

Performing calculations that the LLM cannot reliably complete itself.
Forcing an LLM to generate verbatim text under certain circumstances, such as when responding to a question about an app’s terms of service.

Performing an action when initiated by an LLM

Turning on and off lights in an LLM-powered home assistant
Reserving table reservations in an LLM-powered restaurant agent

Before you begin

If you want to run the code examples on this page, first complete the steps in the Get started guide. All of the examples assume that you have already set up a project with Genkit dependencies installed.

This page discusses one of the advanced features of Genkit model abstraction, so before you dive too deeply, you should be familiar with the content on the Generating content with AI models page. You should also be familiar with Genkit’s system for defining input and output schemas, which is discussed on the Flows page.

Overview of tool calling

At a high level, this is what a typical tool-calling interaction with an LLM looks like:

The calling application prompts the LLM with a request and also includes in the prompt a list of tools the LLM can use to generate a response.
The LLM either generates a complete response or generates a tool call request in a specific format.
If the caller receives a complete response, the request is fulfilled and the interaction ends; but if the caller receives a tool call, it performs whatever logic is appropriate and sends a new request to the LLM containing the original prompt or some variation of it as well as the result of the tool call.
The LLM handles the new prompt as in Step 2.

For this to work, several requirements must be met:

The model must be trained to make tool requests when it’s needed to complete a prompt. Most of the larger models provided through web APIs such as Gemini can do this, but smaller and more specialized models often cannot. Genkit returns an error if you try to provide tools to a model that doesn’t support it.
The calling application must provide tool definitions to the model in the format it expects.
The calling application must prompt the model to generate tool calling requests in the format the application expects.

Tool calling with Genkit

Genkit provides a single interface for tool calling with models that support it. Each model plugin ensures that the last two criteria mentioned in the previous section are met, and the genkit.Generate() function automatically carries out the tool-calling loop described earlier.

Model support

Tool calling support depends on the model, the model API, and the Genkit plugin. Consult the relevant documentation to determine if tool calling is likely to be supported. In addition:

Genkit returns an error if you try to provide tools to a model that doesn’t support it.
If the plugin exports model references, the ModelInfo.Supports.Tools property will indicate if it supports tool calling.

Defining tools

Use the genkit.DefineTool() function to write tool definitions:

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/genkit"
    "github.com/firebase/genkit/go/plugins/googlegenai"
)

// WeatherInput is the tool's input. The schema the model sees is inferred from
// this type, so a jsonschema_description tag is how a field gets described.
type WeatherInput struct {
    Location string `json:"location" jsonschema_description:"The location to get the current weather for."`
}

func main() {
    ctx := context.Background()

    g := genkit.Init(ctx,
        genkit.WithPlugins(&googlegenai.GoogleAI{}),
        genkit.WithDefaultModel("googleai/gemini-flash-latest"),
    )

    getWeather := genkit.DefineTool(g, "getWeather",
        "Gets the current weather in a given location",
        func(ctx *ai.ToolContext, input WeatherInput) (string, error) {
            // Here, we would typically make an API call or database query. For
            // this example, we just return a fixed value.
            return fmt.Sprintf("The current weather in %s is 63°F and sunny.", input.Location), nil
        })

    resp, err := genkit.Generate(ctx, g,
        ai.WithPrompt("What is the weather in San Francisco?"),
        ai.WithTools(getWeather),
    )
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(resp.Text())
}

The syntax looks like genkit.DefineFlow(), with one addition: you must write a description. The name and the description are all the model knows about a tool besides the schemas inferred from the input and output types, so both are prompt and are worth writing as carefully as one.

Describe fields with the jsonschema_description struct tag. The jsonschema tag is a comma-separated keyword list for constraints such as enum= and minimum=, so a description written there is cut off at its first comma with no error; the separate tag has no list to terminate.

The tool context

*ai.ToolContext embeds context.Context, so it satisfies context.Context directly. Pass it unchanged to a flow, an HTTP client, or a database handle, and it carries the caller’s cancellation and deadline with it. This one needs time on top of the imports above:

type RevenueInput struct {
    Region string `json:"region"`
}

reportRevenue := genkit.DefineFlow(g, "reportRevenue",
    func(ctx context.Context, region string) (string, error) {
        // ... look up the numbers ...
        return "revenue for " + region, nil
    })

getRevenue := genkit.DefineTool(g, "getRevenue",
    "Reports revenue for a region over the last quarter.",
    func(ctx *ai.ToolContext, input RevenueInput) (string, error) {
        // ctx is a context.Context, so it goes straight into anything that
        // takes one. Give a slow dependency a deadline of its own.
        callCtx, cancel := context.WithTimeout(ctx, 10*time.Second)
        defer cancel()

        return reportRevenue.Run(callCtx, input.Region)
    })

The struct also carries Resumed map[string]any and OriginalInput any, both set only after an interrupt, plus the methods Interrupt(*ai.InterruptOptions) and IsResumed(). The interrupts guide covers that half.

Tools run concurrently

When a model asks for several tools in one turn, Genkit runs them concurrently, one goroutine per request. A tool function must be safe to call from several goroutines at once: guard shared state with a mutex, and copy a slice or map before returning it. The registry is mutex-protected, so genkit.DefineTool and the other Define* calls are safe from any goroutine, but defining tools during startup is still the pattern to follow.

Schemas known only at runtime

If the input shape is only known at runtime, declare the input parameter as any and pass the schema instead. ai.WithInputSchema(schema map[string]any) takes a plain JSON Schema document as a map[string]any, not a *jsonschema.Schema, not []byte, and not a struct:

schema := map[string]any{
    "type": "object",
    "properties": map[string]any{
        "location": map[string]any{
            "type":        "string",
            "description": "The location to get the current weather for",
        },
    },
    "required": []string{"location"},
}

genkit.DefineTool(g, "getWeather", "Gets the current weather in a given location",
    func(ctx *ai.ToolContext, input any) (string, error) {
        // input arrives as a map[string]any shaped by the schema above.
        return "sunny", nil
    },
    ai.WithInputSchema(schema),
)

ai.WithInputSchemaName references a schema registered with genkit.DefineSchemasFor, and ai.WithOutputSchema(schema map[string]any) and ai.WithOutputSchemaName do the same for the output. An explicit schema stands in for a type parameter, so the parameter it describes must be any.

The basic-tools sample is a runnable program built around one tool, and it is where the deployment example further down comes from.

Using tools

Include defined tools in your prompts to generate content.

Using genkit.Generate():

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("What is the weather in San Francisco?"),
    ai.WithTools(getWeather),
)

ai.WithTools(tools ...ai.ToolRef) accepts anything with a Name() string method: *ai.ToolAction from genkit.DefineTool, *aix.Tool from the in-preview API below, and ai.ToolName("getWeather") when all you have is the name. So a per-request tool set is just a slice you spread. Repeating the option appends, and duplicate names are rejected when the request runs:

adminTools := []ai.ToolRef{getWeather, getRevenue, ai.ToolName("auditLog")}

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Who looked at the San Francisco forecast today?"),
    ai.WithTools(adminTools...),
)

Using genkit.DefinePrompt():

weatherPrompt := genkit.DefinePrompt(g, "weatherPrompt",
    ai.WithPrompt("What is the weather in {{location}}?"),
    ai.WithTools(getWeather),
)

resp, err := weatherPrompt.Execute(ctx,
    ai.WithInput(map[string]any{"location": "San Francisco"}),
)

Using a .prompt file:

Create a file named prompts/weatherPrompt.prompt (assuming default prompt directory):

---
system: "Answer questions using the tools you have."
tools: [getWeather]
input:
  schema:
    location: string
---

What is the weather in {{location}}?

Then execute it in your Go code:

// Assuming prompt file named weatherPrompt.prompt exists in ./prompts dir.
weatherPrompt := genkit.LookupPrompt(g, "weatherPrompt")
if weatherPrompt == nil {
    log.Fatal("no prompt named 'weatherPrompt' found")
}

resp, err := weatherPrompt.Execute(ctx,
    ai.WithInput(map[string]any{"location": "San Francisco"}),
)

Genkit handles the tool call automatically if the LLM needs to use the getWeather tool to answer the prompt.

Controlling tool choice

ai.WithToolChoice() decides whether the model may call a tool, must call one, or must not. ai.ToolChoice is a string type with three values:

Value	Meaning
`ai.ToolChoiceAuto`	The model decides whether to call a tool.
`ai.ToolChoiceRequired`	The model must call at least one tool this turn.
`ai.ToolChoiceNone`	The model must answer without calling a tool.

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("What is the weather in San Francisco?"),
    ai.WithTools(getWeather),
    ai.WithToolChoice(ai.ToolChoiceRequired),
)

Omitting the option leaves the zero value, the empty string, which means unset: Genkit sends nothing and the provider’s own default applies. A model whose ModelInfo.Supports.ToolChoice is false accepts only the unset value and ai.ToolChoiceAuto; anything else fails the request with ai.ErrUnsupportedByModel.

Returning more than one value from a tool

A tool sometimes has more to say than a single value: a chart of what it measured, a screenshot of what it saw, a document it retrieved. Define it with genkit.DefineMultipartTool() and return an *ai.MultipartToolResponse. The value a plain tool would have answered with goes in Output, and whatever is not a value goes in Content:

// Deploy is what the model fills in to call the tool.
type Deploy struct {
    Service     string `json:"service" jsonschema_description:"The service to deploy."`
    Environment string `json:"environment" jsonschema:"enum=staging,enum=production" jsonschema_description:"Where to deploy it."`
}

// Rollout is the tool's answer, and the Output half of its response.
type Rollout struct {
    Service  string  `json:"service"`
    Revision string  `json:"revision"`
    Healthy  bool    `json:"healthy"`
    P95Ms    float64 `json:"p95Ms" jsonschema_description:"The p95 latency after the rollout, in milliseconds."`
}

deployService := genkit.DefineMultipartTool(g, "deployService",
    "Deploys a service to an environment and reports how the rollout went.",
    func(ctx *ai.ToolContext, input Deploy) (*ai.MultipartToolResponse, error) {
        latencies, err := rollOut(input)
        if err != nil {
            return nil, err
        }

        return &ai.MultipartToolResponse{
            // The value a plain tool would have returned.
            Output: &Rollout{
                Service:  input.Service,
                Revision: fmt.Sprintf("%s-00042", input.Service),
                Healthy:  true,
                P95Ms:    latencies[len(latencies)-1],
            },
            // The model receives this as a picture, so it can describe the
            // shape of the rollout rather than only its last number.
            Content: []*ai.Part{ai.NewMediaPart("image/png", barChartPNG(latencies))},
        }, nil
    })

The attached parts reach the model and the client both, so the model can reason about the chart and the Dev UI can render it. They must be media or data parts; a text part is not a valid attachment. Pass a multipart tool to ai.WithTools() like any other. Its advertised output schema is the multipart envelope, not Rollout; pass ai.WithOutputSchema or ai.WithOutputSchemaName to advertise the Output shape instead.

Streaming and tool calling

A tool call takes several turns, so a stream carries the tool’s traffic as well as the model’s text. Use genkit.GenerateStream() when the caller needs to act on that traffic, and switch on the part kind. core.StreamCallback and genkit.DefineStreamingFlow come from the flows API, and status here is github.com/firebase/genkit/go/core/status, not google.golang.org/grpc/status; see Error types:

import (
    "context"
    "fmt"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/core"
    "github.com/firebase/genkit/go/core/status"
    "github.com/firebase/genkit/go/genkit"
)

genkit.DefineStreamingFlow(g, "deployFlow",
    func(ctx context.Context, input string, sendChunk core.StreamCallback[string]) (string, error) {
        for val, err := range genkit.GenerateStream(ctx, g,
            ai.WithPrompt(input),
            ai.WithTools(deployService),
        ) {
            if err != nil {
                return "", fmt.Errorf("could not deploy: %w", err)
            }
            if val.Done {
                return val.Response.Text(), nil
            }
            for _, part := range val.Chunk.Content {
                switch {
                case part.IsText():
                    sendChunk(ctx, part.Text)
                case part.IsToolRequest():
                    sendChunk(ctx, fmt.Sprintf("[calling %s]", part.ToolRequest.Name))
                case part.IsToolResponse():
                    sendChunk(ctx, fmt.Sprintf("[%s answered]", part.ToolResponse.Name))
                }
            }
        }
        return "", status.Errorf(status.ErrInternal, "the stream ended without a final result")
    })

A core.StreamCallback[*ai.ModelResponseChunk] handed to ai.WithStreaming() on an ordinary genkit.Generate() call is shorter and forwards every chunk untouched. The tool traffic still arrives on it, so the caller switches on the part kind the same way.

Limiting tool call iterations with `WithMaxTurns`

When a tool might trigger several sequential calls, ai.WithMaxTurns() caps how many back-and-forth iterations the model gets in one generation. It controls cost, keeps latency bounded, and guards against a loop that never settles. Each turn is one complete cycle of the model calling tools and receiving their responses. The default is 5:

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Research the latest developments in quantum computing."),
    ai.WithTools(webSearch),
    ai.WithMaxTurns(8), // Allow up to 8 tool-calling iterations.
)
if err != nil {
    if errors.Is(err, ai.ErrMaxTurnsExceeded) {
        // The loop hit its limit before the model produced a final answer.
        // resp.History() holds the rounds that completed; send it back with
        // a higher limit to continue rather than start over.
    }
}

When the limit is reached, Genkit stops the loop and returns an error matching ai.ErrMaxTurnsExceeded, whose status is ABORTED, beside a partial response: resp.FinishReason is ai.FinishReasonAborted, since a limit the caller set is a caller stop, and resp.History() carries the completed rounds. Either raise the limit or look for a tool the model keeps retrying.

When a tool fails

A tool that returns a non-nil error is fail-fast. Genkit stops the tool loop as soon as the error arrives, while any siblings in the same round finish on their own, and genkit.Generate() returns the error wrapped as ai.ErrToolFailed, a subtype of status.ErrInternal (HTTP 500). The model never sees it and never retries. The response comes back beside the error with ai.FinishReasonFailed: its History() holds the rounds before the failure and drops the failed round whole, the model’s request message included, because a conversation cannot end on an unanswered tool request. Output that does not match the tool’s declared schema reports the same sentinel, and so does input the model sent that fails the declared input schema: the tool function never runs and the call aborts with ai.ErrToolFailed wrapping status.ErrInvalidInput. A tool that stopped because the call’s context ended is not a tool failure: that error carries CANCELLED, and the response reports ai.FinishReasonAborted.

The tool’s own error is wrapped with %w, so both the framework sentinel and your own cause still match:

// errStationOffline is the tool's own sentinel, returned from its body.
var errStationOffline = errors.New("weather station offline")

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("What is the weather in San Francisco?"),
    ai.WithTools(getWeather),
)
if err != nil {
    if errors.Is(err, ai.ErrToolFailed) && errors.Is(err, errStationOffline) {
        return fmt.Errorf("retry later: %w", err)
    }
    return err
}

There is no soft-fail flag and no ai.ToolError. To let the model recover from a failure instead of aborting the call, return a nil error and put the failure in the tool’s own output type, then say in the description how the model should react:

type Forecast struct {
    Summary     string `json:"summary,omitempty"`
    Unavailable bool   `json:"unavailable" jsonschema_description:"True when the forecast could not be read. Say so, and do not retry."`
    Reason      string `json:"reason,omitempty"`
}

getWeather := genkit.DefineTool(g, "getWeather",
    "Gets the current weather in a given location. If unavailable is true, tell the user rather than guessing.",
    func(ctx *ai.ToolContext, input WeatherInput) (Forecast, error) {
        // readStation is your own call to the weather service.
        summary, err := readStation(ctx, input.Location)
        if err != nil {
            // A nil error, so the loop continues and the model can react.
            return Forecast{Unavailable: true, Reason: err.Error()}, nil
        }
        return Forecast{Summary: summary}, nil
    })

See Error types for the rest of the sentinels generation reports.

Inspecting which tools ran

There is no resp.ToolCalls(). resp.ToolRequests() returns only the requests left unfulfilled by the final turn, which is empty unless ai.WithReturnToolRequests(true) is set or an interrupt fired. The full record of the loop is resp.History(): the request’s messages plus the response, which includes each model message carrying tool requests and each ai.RoleTool message carrying their responses. Pair the two on Ref:

calls := map[string]*ai.ToolRequest{}
for _, msg := range resp.History() {
    for _, part := range msg.Content {
        switch {
        case part.IsToolRequest():
            calls[part.ToolRequest.Ref] = part.ToolRequest
        case part.IsToolResponse():
            req := calls[part.ToolResponse.Ref]
            fmt.Printf("%s(%v) -> %v\n", part.ToolResponse.Name, req.Input, part.ToolResponse.Output)
        }
    }
}

Tools that come from middleware

Not every tool is one you write. Middleware attached with ai.WithUse() can contribute tools to a request, and they join the loop like any other. The Filesystem middleware from github.com/firebase/genkit/go/plugins/middleware adds list_files and read_file, plus write_file and edit_file when AllowWriteAccess is set, all confined to one directory:

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("What is in the config file, and what does it configure?"),
    // Adds list_files and read_file, both confined to ./workspace.
    ai.WithUse(&middleware.Filesystem{RootDir: "./workspace"}),
    // Exploring a directory takes several rounds, so leave the loop room.
    ai.WithMaxTurns(20),
)

The Skills middleware adds a single use_skill tool and lists the available skills in the system prompt, so the heavy instructions stay off the hot path until the model asks for one. Both are worked through in the filesystem and skills samples, and the middleware guide covers the rest of the built-in set.

Pause the tool loop by using interrupts

By default, Genkit repeatedly calls the LLM until every tool call has been resolved. You can conditionally pause execution in situations where you want to, for example:

Ask the user a question or display UI.
Confirm a potentially risky action with the user.
Request out-of-band approval for an action.

Interrupts are special tools that can halt the loop and return control to your code so that you can handle more advanced scenarios. Visit the interrupts guide to learn how to use them.

Explicitly handling tool calls

If you want full control over this tool-calling loop, for example to apply more complicated logic, set the WithReturnToolRequests() option to true. Now it’s your responsibility to ensure all of the tool requests are fulfilled, to carry the tools and the option onto every call, and to stop. One round is not enough: after you answer a tool request the model usually has more to say, and it may ask for another tool:

getWeather := genkit.DefineTool(g, "getWeather",
    "Gets the current weather in a given location",
    func(ctx *ai.ToolContext, input WeatherInput) (string, error) {
        // Tool implementation...
        return "sunny", nil
    })

const maxTurns = 5

messages := []*ai.Message{
    ai.NewUserTextMessage("What is the weather in San Francisco?"),
}

for turn := 0; turn < maxTurns; turn++ {
    resp, err := genkit.Generate(ctx, g,
        ai.WithMessages(messages...),
        // Both options belong on every call, not just the first: without
        // them the model has no tools on the second turn.
        ai.WithTools(getWeather),
        ai.WithReturnToolRequests(true),
    )
    if err != nil {
        log.Fatal(err)
    }

    requests := resp.ToolRequests()
    if len(requests) == 0 {
        fmt.Println(resp.Text())
        return
    }

    // ToolRequests returns the request parts, so the call itself is on
    // part.ToolRequest.
    var parts []*ai.Part
    for _, part := range requests {
        req := part.ToolRequest

        tool := genkit.LookupTool(g, req.Name)
        if tool == nil {
            log.Fatalf("tool %q not found", req.Name)
        }

        output, err := tool.RunRaw(ctx, req.Input)
        if err != nil {
            log.Fatalf("tool %q failed: %v", tool.Name(), err)
        }

        parts = append(parts,
            ai.NewToolResponsePart(&ai.ToolResponse{
                Name:   req.Name,
                Ref:    req.Ref,
                Output: output,
            }))
    }

    // Carry the whole turn forward: the model's request message, then the
    // tool message answering it.
    messages = append(resp.History(), ai.NewMessage(ai.RoleTool, nil, parts...))
}

log.Fatalf("gave up after %d turns", maxTurns)

ai.WithMaxTurns() has nothing to bound in this mode, since Genkit is no longer running the loop, so the turn cap is yours to enforce.

RunRaw returns the tool’s output value. For a multipart tool, use RunRawMultipart instead when you also need the attached content parts.

The tools API in preview

A second tools API is in preview under go/ai/exp and go/genkit/exp. It is slated to replace the stable one in the next major version, and it changes the shape of a tool rather than what a tool can do:

The function takes a plain context.Context rather than an *ai.ToolContext.
It returns its answer directly, so the signature stops having to announce that the tool sometimes has more to say. tool.AttachParts(ctx, parts...) attaches the extra content instead of an *ai.MultipartToolResponse envelope.
tool.SendPartial(ctx, value) streams structured progress while the tool works, so a slow tool does not look like a hang.
tool.SendChunk(ctx, chunk) streams a chunk the tool builds itself, for an update that is a line of prose rather than a value.

Both streaming helpers are best-effort: with a caller that is not streaming they are no-ops, and the returned value is always the authoritative answer. Neither streamed message is written to history, since progress is for showing rather than for the model to read.

The package names collide, so import them deliberately:

import (
    "github.com/firebase/genkit/go/ai"
    aix "github.com/firebase/genkit/go/ai/exp"
    "github.com/firebase/genkit/go/ai/exp/tool"
    "github.com/firebase/genkit/go/genkit"
    genkitx "github.com/firebase/genkit/go/genkit/exp"
)

The constructors panic unless genkit.Init was given genkit.WithExperimental():

// Progress is what the tool streams while it works. It is the tool's own
// shape, not one the API dictates: any value that survives JSON works.
type Progress struct {
    Step    string `json:"step"`
    Percent int    `json:"percent"`
}

g := genkit.Init(ctx,
    genkit.WithPlugins(&googlegenai.GoogleAI{}),
    genkit.WithExperimental(), // Required: the exp constructors panic without it.
)

deployService := genkitx.DefineTool(g, "deployService",
    "Deploys a service to an environment and reports how the rollout went.",
    func(ctx context.Context, input Deploy) (*Rollout, error) {
        // Structured progress, so a slow tool does not look like a hang.
        tool.SendPartial(ctx, Progress{Step: "shifting traffic", Percent: 50})

        latencies, err := rollOut(input)
        if err != nil {
            return nil, err
        }

        revision := fmt.Sprintf("%s-00042", input.Service)

        // An update with no structure worth giving it. RoleTool marks the
        // chunk as the tool's, so the caller can tell it from the model's
        // own text.
        tool.SendChunk(ctx, &ai.ModelResponseChunk{
            Role:    ai.RoleTool,
            Content: []*ai.Part{ai.NewTextPart(revision + " is live")},
        })

        // The chart travels beside the answer instead of inside it.
        tool.AttachParts(ctx, ai.NewMediaPart("image/png", barChartPNG(latencies)))

        return &Rollout{
            Service:  input.Service,
            Revision: revision,
            Healthy:  true,
            P95Ms:    latencies[len(latencies)-1],
        }, nil
    })

The exact signatures are:

func AttachParts(ctx context.Context, parts ...*ai.Part)
func SendPartial(ctx context.Context, output any)
func SendChunk(ctx context.Context, chunk *ai.ModelResponseChunk)

There is no in-preview generate entry point, and none is needed. genkitx.DefineTool[In, Out] returns *aix.Tool[In, Out], the stable genkit.DefineTool[In, Out] returns *ai.ToolAction[In, Out], and both have a Name() string method, so both satisfy ai.ToolRef. Write the type out when a helper hands a tool back, and pass it to the ordinary ai.WithTools() on a stable genkit.Generate():

func newDeployTool(g *genkit.Genkit) *aix.Tool[Deploy, *Rollout] {
    return genkitx.DefineTool(g, "deployService",
        "Deploys a service to an environment and reports how the rollout went.",
        func(ctx context.Context, input Deploy) (*Rollout, error) {
            tool.SendPartial(ctx, Progress{Step: "shifting traffic", Percent: 50})
            // ... roll out, then return the answer ...
            return &Rollout{Service: input.Service, Healthy: true}, nil
        })
}

deployService := newDeployTool(g)

for val, err := range genkit.GenerateStream(ctx, g,
    ai.WithPrompt("Deploy checkout to staging."),
    ai.WithTools(deployService),
) {
    if err != nil {
        log.Fatal(err)
    }
    if val.Done {
        fmt.Println(val.Response.Text())
        break
    }
    for _, part := range val.Chunk.Content {
        // A value from tool.SendPartial lands here as a tool response part
        // flagged partial, named and ref'd like the call it is reporting on.
        // It is progress, not the answer, and never reaches history.
        if part.IsToolResponse() && part.IsPartial() {
            fmt.Printf("[%s] %v\n", part.ToolResponse.Name, part.ToolResponse.Output)
        }
    }
}

genkitx.DefineInterruptibleTool is the interruptible counterpart, which takes a third type parameter for the value that comes back on the resume. The interrupts guide covers it.

The basic-tools-exp sample runs on this in-preview API and is deliberately the same program as basic-tools, written twice, so a diff between the two files is the whole API lesson. Anything under exp may change in any minor release.