Pause generation using interrupts

Interrupts are a special kind of tool that can pause the LLM generation-and-tool-calling loop to return control back to you. When you’re ready, you can then resume generation by sending replies that the LLM processes for further generation.

The most common uses for interrupts fall into a few categories:

• Human-in-the-Loop: Enabling the user of an interactive AI to clarify needed information or confirm the LLM’s action before it is completed, providing a measure of safety and confidence.
• Async Processing: Starting an asynchronous task that can only be completed out-of-band, such as sending an approval notification to a human reviewer or kicking off a long-running background process.
• Exit from an Autonomous Task: Providing the model a way to mark a task as complete, in a workflow that might iterate through a long series of tool calls.

Before you begin

All of the examples documented here assume that you have already set up a project with Genkit dependencies installed. If you want to run the code examples on this page, first complete the steps in the Get started guide.

Before diving too deeply, you should also be familiar with the following concepts:

• Generating content with AI models.
• Genkit’s system for defining input and output schemas.
• General methods of tool-calling.

Overview of interrupts

At a high level, this is what an interrupt looks like when interacting with an LLM:

1. The calling application prompts the LLM with a request. The prompt includes a list of tools, including at least one for an interrupt that the LLM can use to generate a response.
2. The LLM generates either a complete response or a tool call request in a specific format. To the LLM, an interrupt call looks like any other tool call.
3. If the LLM calls an interrupting tool, the Genkit library automatically pauses generation rather than immediately passing responses back to the model for additional processing.
4. The developer checks whether an interrupt call is made, and performs whatever task is needed to collect the information needed for the interrupt response.
5. The developer resumes generation by passing an interrupt response to the model. This action triggers a return to Step 2.

In Go, a tool that can pause is defined with genkit.DefineResumableTool(). It pauses by returning tool.Interrupt() from package github.com/firebase/genkit/go/ai/tool. The caller claims each paused call with the tool’s Interrupted() method, answers it, and resumes with ai.WithResume().

Define manual-response interrupts

The most common kind of interrupt allows the LLM to request clarification from the user, for example by asking a multiple-choice question.

genkit.DefineResumableTool() takes three type parameters: the input the model fills in, the output the tool answers with, and the type the tool is resumed with. The function receives the resume value as a pointer, which is nil on the first call. This tool pauses on every first call, because only the user can answer it:

import (
    "context"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/ai/tool"
    "github.com/firebase/genkit/go/genkit"
)

// QuestionInput is what the model fills in to call the tool.
type QuestionInput struct {
    Question string   `json:"question"`
    Choices  []string `json:"choices" jsonschema_description:"The choices to show the user."`
}

// Answer is what the tool is resumed with when it runs again.
type Answer struct {
    Text string `json:"text"`
}

askQuestion := genkit.DefineResumableTool(g, "askQuestion",
    "use this to ask the user a clarifying question",
    func(ctx context.Context, input QuestionInput, answer *Answer) (string, error) {
        if answer == nil {
            // Pause. The input already holds the question and the choices,
            // so the interrupt carries no data of its own.
            return "", tool.Interrupt(ctx, nil)
        }
        return answer.Text, nil
    },
)

The output type of the tool, string here, is the type of the answer the caller provides when it responds to the interrupt.

Use interrupts

Interrupts are passed into the WithTools() option when generating content, just like other types of tools:

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Ask me a movie trivia question."),
    ai.WithTools(askQuestion),
)

Genkit immediately returns a response on receipt of an interrupt tool call.

Respond to interrupts

A paused generation returns a response with no error. Check the response for interrupts and handle them:

// You can check the finish reason of the response.
paused := resp.FinishReason == ai.FinishReasonInterrupted
// Or you can check whether any interrupts are on the response.
paused = len(resp.Interrupts()) > 0

Each part in resp.Interrupts() is a paused tool call. Claim it with the Interrupted() method of the tool that could have raised it. The claim reports whether the part is an interrupt of that tool and, when it is, returns an *ai.InterruptedCall whose Input field is the model’s input, decoded to the tool’s input type. The call has three verbs, and each one returns the part that resumes generation:

Verb	Effect
call.Respond(output)	Answers the call with output, typed as the tool’s output. The tool does not run again.
call.Restart(resume)	Runs the tool again, with resume as its resume parameter.
call.RestartWithInput(input, resume)	Runs the tool again with revised input.

Send the parts back with ai.WithResume() on a subsequent Generate call, together with the existing history and the same tools. Once resumed, the model re-enters the generation loop, including tool execution, until either it completes or another interrupt is triggered.

A single turn can raise interrupts from more than one tool, so claim each part with every tool that pauses. This example adds a second tool beside askQuestion:

// BudgetInput is what the model fills in to have the user approve a spend.
type BudgetInput struct {
    Dollars float64 `json:"dollars"`
}

// BudgetDecision is what confirmBudget is resumed with.
type BudgetDecision struct {
    Approved bool `json:"approved"`
}

confirmBudget := genkit.DefineResumableTool(g, "confirmBudget",
    "use this to have the user approve a spend before committing to it",
    func(ctx context.Context, input BudgetInput, decision *BudgetDecision) (bool, error) {
        if decision == nil {
            return false, tool.Interrupt(ctx, nil)
        }
        return decision.Approved, nil
    },
)

// The model can keep asking questions, so cap the rounds.
const maxRounds = 5

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Help me plan a backyard BBQ."),
    ai.WithSystem("Ask clarifying questions until you have a complete solution."),
    ai.WithTools(askQuestion, confirmBudget),
)
if err != nil {
    return err
}

for round := 0; round < maxRounds && resp.FinishReason == ai.FinishReasonInterrupted; round++ {
    var parts []*ai.Part

    for _, part := range resp.Interrupts() {
        if call, ok := askQuestion.Interrupted(part); ok {
            // call.Input is a QuestionInput, and Respond takes a string, so
            // the answer cannot drift from what the tool would have returned.
            parts = append(parts, call.Respond(askUser(call.Input.Question, call.Input.Choices)))
            continue
        }
        if call, ok := confirmBudget.Interrupted(part); ok {
            parts = append(parts, call.Respond(userApproves(call.Input.Dollars)))
            continue
        }
        // Every interrupt needs an answer, so fail rather than skip one.
        return fmt.Errorf("no handler for interrupt from tool %q", part.ToolRequest.Name)
    }

    resp, err = genkit.Generate(ctx, g,
        ai.WithMessages(resp.History()...),
        ai.WithTools(askQuestion, confirmBudget),
        ai.WithResume(parts...),
    )
    if err != nil {
        return err
    }
}

if resp.FinishReason == ai.FinishReasonInterrupted {
    return fmt.Errorf("still waiting on the user after %d rounds", maxRounds)
}
fmt.Println(resp.Text())

askUser and userApproves stand for your own UI: askUser shows a question with its choices and returns the user’s answer, and userApproves asks the user to approve a spend and returns their decision.

Interrupted() reports false for an interrupt of another tool, so the claims dispatch the parts without matching names by hand. It also reports false for an interrupt that is already resolved, for a hold that middleware raised (see Answer middleware holds), and for an input that no longer decodes as the tool’s input type. Nothing after a claim can fail, so the verbs return no error.

A resume that pauses again, for example because the restarted tool asks a second question, returns its response with no error and with FinishReason set to ai.FinishReasonInterrupted, like the first pause. Its History() is ready to resume from, so the loop above handles it with no special case.

Answer every interrupt

Every interrupt in a turn must get exactly one response or restart part. Genkit checks the whole resume before any tool runs, so a resume it rejects runs no tool.

• Answer some but not all, and the next Generate fails with ai.ErrUnresolvedToolRequest. The error names the request as name#ref.
• Pass a part that is still paused, such as resp.Interrupts() as is, and the call fails with INVALID_ARGUMENT.
• A response whose output does not match the tool’s output schema, or a restart whose resume value does not match the tool’s resume type, fails the call with INVALID_ARGUMENT.

Tools with restartable interrupts

Another common pattern is the need to confirm an action that the LLM suggests before actually performing it. For example, a payments app might want the user to confirm certain kinds of transfers. The basic-tool-interrupts sample is this whole pattern as a runnable program.

Define a restartable tool

A restartable tool decides when to pause and what to do when it runs again. Its resume parameter tells a first call from a restarted one: it is nil on the first call and holds the caller’s decision on the restart. Letting the tool decide, rather than the caller, keeps the rule it paused on in one place.

tool.Interrupt(ctx, data) can also carry data about the pause, such as why the tool paused. The input already says what the model asked for, so the data holds only what the tool knew. It must serialize to a JSON object, so use a struct or a map:

import (
    "context"

    "github.com/firebase/genkit/go/ai/tool"
    "github.com/firebase/genkit/go/genkit"
)

// Transfers above this need a human to approve them.
const approvalLimit = 100

// accountBalance stands in for a bank.
var accountBalance = 150.00

// TransferInput is what the model fills in to call the tool.
type TransferInput struct {
    ToAccount string  `json:"toAccount"`
    Amount    float64 `json:"amount"`
}

// TransferOutput is what the tool answers with.
type TransferOutput struct {
    Status     string  `json:"status"`
    Message    string  `json:"message,omitempty"`
    NewBalance float64 `json:"newBalance,omitempty"`
}

// TransferInterrupt is what the tool says when it pauses.
type TransferInterrupt struct {
    Reason  string  `json:"reason"` // "insufficient_balance" or "confirm_large"
    Balance float64 `json:"balance"`
}

// Confirmation is what the tool is resumed with.
type Confirmation struct {
    Approved bool `json:"approved"`
}

transferMoney := genkit.DefineResumableTool(g, "transferMoney",
    "Transfers money to another account.",
    func(ctx context.Context, input TransferInput, confirm *Confirmation) (*TransferOutput, error) {
        // More than the account holds: pause and say so.
        if input.Amount > accountBalance {
            return nil, tool.Interrupt(ctx, TransferInterrupt{
                Reason:  "insufficient_balance",
                Balance: accountBalance,
            })
        }

        // A large transfer on the first call: pause for confirmation.
        if confirm == nil && input.Amount > approvalLimit {
            return nil, tool.Interrupt(ctx, TransferInterrupt{
                Reason:  "confirm_large",
                Balance: accountBalance,
            })
        }

        if confirm != nil && !confirm.Approved {
            return &TransferOutput{
                Status:  "declined",
                Message: "The user declined the transfer.",
            }, nil
        }

        accountBalance -= input.Amount
        return &TransferOutput{
            Status:     "completed",
            Message:    "Transfer successful",
            NewBalance: accountBalance,
        }, nil
    },
)

The resume type must be a struct or a map with string keys, so that it serializes to a JSON object. Any other type panics when the tool is defined. A field the caller may leave out needs an omitempty tag, as it does on an input type.

Restart tools after interruption

Restartable tools give you full control over:

1. When an initial tool request should trigger an interrupt.
2. When and whether to resume the generation loop.
3. What additional information to provide to the tool when resuming.

Read the data a tool paused with by using ai.InterruptAs[T](), and the input from the claimed call. Then choose a verb: call.Restart() runs the tool again with the decision, call.RestartWithInput() runs it again with different arguments, and call.Respond() answers the call without running the tool:

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Transfer $200 to account ABC123"),
    ai.WithTools(transferMoney),
)
if err != nil {
    return err
}

for round := 0; round < maxRounds && resp.FinishReason == ai.FinishReasonInterrupted; round++ {
    var parts []*ai.Part

    for _, part := range resp.Interrupts() {
        call, ok := transferMoney.Interrupted(part)
        if !ok {
            return fmt.Errorf("no handler for interrupt from tool %q", part.ToolRequest.Name)
        }
        // The input says what the model asked for; the interrupt data adds
        // what only the tool knew when it paused.
        meta, _ := ai.InterruptAs[TransferInterrupt](part)

        switch meta.Reason {
        case "insufficient_balance":
            if userConfirms("Transfer $%.2f instead?", meta.Balance) {
                // Run the tool again with a smaller amount. The decision is
                // already made, so the tool does not pause again.
                revised := call.Input
                revised.Amount = meta.Balance
                parts = append(parts, call.RestartWithInput(revised, Confirmation{Approved: true}))
            } else {
                // Answer the call outright, without running the tool again.
                parts = append(parts, call.Respond(&TransferOutput{
                    Status:     "cancelled",
                    Message:    "Transfer cancelled by user.",
                    NewBalance: accountBalance,
                }))
            }

        default:
            // The decision arrives as the tool's confirm parameter.
            approved := userConfirms("Confirm transfer of $%.2f to %s?", call.Input.Amount, call.Input.ToAccount)
            parts = append(parts, call.Restart(Confirmation{Approved: approved}))
        }
    }

    resp, err = genkit.Generate(ctx, g,
        ai.WithMessages(resp.History()...),
        ai.WithTools(transferMoney),
        ai.WithResume(parts...),
    )
    if err != nil {
        return err
    }
}

if resp.FinishReason == ai.FinishReasonInterrupted {
    return fmt.Errorf("transfer still waiting after %d rounds", maxRounds)
}
fmt.Println(resp.Text())

userConfirms stands for your own UI: it formats a question as fmt.Sprintf does, asks the user, and returns whether they agreed.

call.Restart() takes the tool’s resume type, so the caller and the tool cannot disagree on its shape. The tool receives the value as the caller built it.

A restart that revises the input changes the request in history to the revised input, so the model sees the call as it ran.

Access original input after replacement

When the caller restarts a tool with call.RestartWithInput(), the tool can read the input it was first called with by using tool.OriginalInput[In]():

transferMoney := genkit.DefineResumableTool(g, "transferMoney",
    "Transfers money to another account.",
    func(ctx context.Context, input TransferInput, confirm *Confirmation) (*TransferOutput, error) {
        // ... interrupt logic ...

        accountBalance -= input.Amount
        message := fmt.Sprintf("Transferred $%.2f to %s", input.Amount, input.ToAccount)

        // Report the adjustment when the caller replaced the input.
        if orig, ok := tool.OriginalInput[TransferInput](ctx); ok {
            message = fmt.Sprintf("Transferred $%.2f to %s (adjusted from $%.2f)",
                input.Amount, input.ToAccount, orig.Amount)
        }

        return &TransferOutput{
            Status:     "completed",
            Message:    message,
            NewBalance: accountBalance,
        }, nil
    },
)

tool.OriginalInput() reports false on a first call and on a restart that kept the input.

Answer middleware holds

Middleware can pause a call too. The ToolApproval middleware from github.com/firebase/genkit/go/plugins/middleware holds every call to a tool that is not on its allowlist, before the tool runs. A hold is the middleware’s interrupt, not the tool’s, so the tool’s Interrupted() reports false for it. Claim it with middleware.ToolApprovalInterrupted() and answer it with a middleware.ToolCallDecision. Approved: true runs the call. Any other restart refuses it: the model receives the refusal, with Reason when you set one, as the tool’s error, and the generation continues.

After a hold is approved, the tool runs as a fresh call and can pause with a question of its own. One claim loop handles both kinds of interrupt:

// getBalance is a read-only tool defined with genkit.DefineTool, so the
// allowlist lets it run without a hold.
// Keep the middleware on every call: the answer to a hold goes to it.
approval := ai.WithUse(&middleware.ToolApproval{AllowedTools: []string{"getBalance"}})

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Transfer $120 to account ABC123"),
    ai.WithTools(getBalance, transferMoney),
    approval,
)
if err != nil {
    return err
}

for round := 0; round < maxRounds && resp.FinishReason == ai.FinishReasonInterrupted; round++ {
    var parts []*ai.Part

    for _, part := range resp.Interrupts() {
        // ToolApproval held the call before the tool ran.
        if call, ok := middleware.ToolApprovalInterrupted(part); ok {
            decision := middleware.ToolCallDecision{Approved: userConfirms("Allow %s?", part.ToolRequest.Name)}
            if !decision.Approved {
                decision.Reason = "the user did not allow this call"
            }
            parts = append(parts, call.Restart(decision))
            continue
        }
        // The tool ran and paused with its own question.
        if call, ok := transferMoney.Interrupted(part); ok {
            approved := userConfirms("Confirm transfer of $%.2f to %s?", call.Input.Amount, call.Input.ToAccount)
            parts = append(parts, call.Restart(Confirmation{Approved: approved}))
            continue
        }
        return fmt.Errorf("no handler for interrupt from tool %q", part.ToolRequest.Name)
    }

    resp, err = genkit.Generate(ctx, g,
        ai.WithMessages(resp.History()...),
        ai.WithTools(getBalance, transferMoney),
        approval,
        ai.WithResume(parts...),
    )
    if err != nil {
        return err
    }
}

The restart that answers the tool’s own question gets through ToolApproval without a second approval, because the paused call records that the middleware let it through. A restart that revises the input is held for approval again. Answer a hold with call.Restart(), not call.Respond(): a response stands in for the tool’s output and must match the tool’s output schema, so it cannot refuse the call.

Other middleware can offer the same claim. A middleware author wraps ai.MiddlewareInterrupted[Res](name, part) in a function that fixes the middleware’s name and its answer type, as ToolApprovalInterrupted does. See the middleware guide for ToolApproval and its judge model.

Resolve a part without the tool value

Code that holds only the part, and not the tool that raised it, builds the same parts with methods on *ai.Part. Examples are a generic handler, a tool found with genkit.LookupTool(), or a hold from middleware that offers no claim function:

Method	Effect
part.ToToolResponse(output)	Answers the call with output.
part.ToToolRestart(resume)	Runs the tool again with resume.
part.ToToolRestartWithInput(input, resume)	Runs the tool again with revised input.

These methods are not typed, so they return an error for a part that is not paused, or for resume data that does not serialize to a JSON object. A resume value that is not of the tool’s resume type is read by its JSON shape: Genkit checks it against the resume schema that the tool advertises in its definition, then decodes it into the resume type, before any tool runs. So a map with the right fields works the same as the typed value:

restart, err := part.ToToolRestart(map[string]any{"approved": true})
if err != nil {
    return err
}

Out-of-band approval

The examples above collect the answer inline, but the point of an interrupt is usually that the answer arrives later, from a different process. ai.Message and ai.Part are plain structs with JSON tags, so a paused run round-trips through json.Marshal. Store resp.History() and resp.Interrupts() under an approval ID, return the ID to the caller, and rebuild the call when the decision comes back. A part decoded from JSON is claimed the same way as one from a live response. transferMoney is the tool from above, held in a package-level variable so both handlers reach the same one:

var transferMoney *ai.ResumableToolAction[TransferInput, *TransferOutput, Confirmation]

// Store is whatever you already run: Firestore, Redis, a table.
type Store interface {
    Put(ctx context.Context, key string, blob []byte) error
    Get(ctx context.Context, key string) ([]byte, error)
}

// A paused run is ordinary JSON.
type pausedRun struct {
    History    []*ai.Message `json:"history"`
    Interrupts []*ai.Part    `json:"interrupts"`
}

// Park the run when generation stops on an interrupt, then hand the caller
// the approval ID and return.
func park(ctx context.Context, store Store, approvalID string, resp *ai.ModelResponse) error {
    blob, err := json.Marshal(pausedRun{
        History:    resp.History(),
        Interrupts: resp.Interrupts(),
    })
    if err != nil {
        return err
    }
    return store.Put(ctx, approvalID, blob)
}

// Later, on a different request and possibly in a different process.
func resume(ctx context.Context, g *genkit.Genkit, store Store, approvalID string, approved bool) (*ai.ModelResponse, error) {
    blob, err := store.Get(ctx, approvalID)
    if err != nil {
        return nil, err
    }
    var run pausedRun
    if err := json.Unmarshal(blob, &run); err != nil {
        return nil, err
    }

    var parts []*ai.Part
    for _, part := range run.Interrupts {
        call, ok := transferMoney.Interrupted(part)
        if !ok {
            return nil, fmt.Errorf("no handler for interrupt from tool %q", part.ToolRequest.Name)
        }
        parts = append(parts, call.Restart(Confirmation{Approved: approved}))
    }

    return genkit.Generate(ctx, g,
        ai.WithMessages(run.History...),
        // Every tool named by a pending request has to be on this call, or
        // Generate fails with ai.ErrToolNotFound.
        ai.WithTools(transferMoney),
        ai.WithResume(parts...),
    )
}

Agent interrupts do this persistence for you: the session store holds the paused history, so the resuming request carries only the decision.

Resuming from an untrusted caller

Generate matches each restart or response part to a pending tool request by tool name and ref only. A restart runs the tool with the input on the restart part, whatever it is, and a part that matches no pending request is ignored. Genkit checks a response against the tool’s output schema, and a restart’s resume value against the tool’s resume schema, and nothing more.

So if the resume parts arrive over the network, re-read the paused history from your own store and check every part against it before calling Generate. aix.ValidateResumeAgainstHistory() does this check. Every part must name a tool request in the last model message, and a restart must carry the input of that request. A restart from RestartWithInput() passes when the original input it records matches, so a server that does not let callers revise inputs rejects those parts itself:

Preview

ValidateResumeAgainstHistory is in the experimental ai/exp package and can change in a minor release.

import (
    "github.com/firebase/genkit/go/ai"
    aix "github.com/firebase/genkit/go/ai/exp"
    "github.com/firebase/genkit/go/core/status"
)

// checkResume rejects resume parts that do not answer the paused run in
// history.
func checkResume(history []*ai.Message, parts []*ai.Part) error {
    resume := &aix.ToolResume{}
    for _, p := range parts {
        if p.IsToolResponse() {
            resume.Respond = append(resume.Respond, p)
            continue
        }
        if p.Restart != nil && p.Restart.OriginalInput != nil {
            return status.Errorf(status.ErrInvalidArgument, "revised tool inputs are not accepted")
        }
        resume.Restart = append(resume.Restart, p)
    }
    return aix.ValidateResumeAgainstHistory(resume, history)
}

The agent runtime runs the same check for you, as described in Agent interrupts.