Pause generation using interrupts

Interrupts are a special kind of tool that can pause the LLM generation-and-tool-calling loop to return control back to you. When you’re ready, you can then resume generation by sending replies that the LLM processes for further generation.

The most common uses for interrupts fall into a few categories:

• Human-in-the-Loop: Enabling the user of an interactive AI to clarify needed information or confirm the LLM’s action before it is completed, providing a measure of safety and confidence.
• Async Processing: Starting an asynchronous task that can only be completed out-of-band, such as sending an approval notification to a human reviewer or kicking off a long-running background process.
• Exit from an Autonomous Task: Providing the model a way to mark a task as complete, in a workflow that might iterate through a long series of tool calls.

Before you begin

All of the examples documented here assume that you have already set up a project with Genkit dependencies installed. If you want to run the code examples on this page, first complete the steps in the Get started guide.

Before diving too deeply, you should also be familiar with the following concepts:

• Generating content with AI models.
• Genkit’s system for defining input and output schemas.
• General methods of tool-calling.

Overview of interrupts

At a high level, this is what an interrupt looks like when interacting with an LLM:

1. The calling application prompts the LLM with a request. The prompt includes a list of tools, including at least one for an interrupt that the LLM can use to generate a response.
2. The LLM generates either a complete response or a tool call request in a specific format. To the LLM, an interrupt call looks like any other tool call.
3. If the LLM calls an interrupting tool, the Genkit library automatically pauses generation rather than immediately passing responses back to the model for additional processing.
4. The developer checks whether an interrupt call is made, and performs whatever task is needed to collect the information needed for the interrupt response.
5. The developer resumes generation by passing an interrupt response to the model. This action triggers a return to Step 2.

Defining tools with interrupts

An interrupting tool is an ordinary tool: pausing is something the tool function does, not something its signature declares. Use genkit.DefineTool() and call ai.InterruptWith() with a struct carrying whatever the person answering needs to know:

// QuestionInput is what the model fills in to call the tool.
type QuestionInput struct {
    Question string   `json:"question"`
    Choices  []string `json:"choices"`
}

// InterruptMetadata carries information about why the tool was interrupted.
type InterruptMetadata struct {
    Reason  string   `json:"reason"`
    Choices []string `json:"choices,omitempty"`
}

askQuestion := genkit.DefineTool(g, "askQuestion",
    "use this to ask the user a clarifying question",
    func(tc *ai.ToolContext, input QuestionInput) (string, error) {
        return "", ai.InterruptWith(tc, InterruptMetadata{
            Reason:  "need_clarification",
            Choices: input.Choices,
        })
    },
)

ai.InterruptWith(tc, meta) is a typed wrapper over tc.Interrupt(&ai.InterruptOptions{Metadata: ...}): it JSON-marshals meta into the same metadata map and calls the same method, so both halt the loop identically. Prefer ai.InterruptWith with a struct, because that is what ai.InterruptAs[T] reads back. Reach for tc.Interrupt only when you already hold a map[string]any.

Use interrupts

Interrupts are passed into the WithTools() option when generating content, just like other types of tools:

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Ask me a movie trivia question."),
    ai.WithTools(askQuestion),
)

Genkit immediately returns a response on receipt of an interrupt tool call.

Respond to interrupts

Check the response for interrupts and handle them. Use ai.InterruptAs() to extract strongly-typed metadata from the interrupt:

// Check if generation was interrupted
if resp.FinishReason == ai.FinishReasonInterrupted {
    for _, interrupt := range resp.Interrupts() {
        if meta, ok := ai.InterruptAs[InterruptMetadata](interrupt); ok {
            fmt.Printf("Interrupt reason: %s\n", meta.Reason)
        }
    }
}

Responding to an interrupt is done using the tool’s RespondWith() method and ai.WithToolResponses() on a subsequent Generate call, passing in the existing message history. RespondWith() answers the paused call outright, so the tool function never runs again.

A single turn can raise interrupts from more than one tool, so dispatch on the tool name rather than assuming which tool paused. This example adds a second interrupting tool beside askQuestion:

type BudgetInput struct {
    Dollars float64 `json:"dollars"`
}

// A second tool that pauses, so the loop below has something to dispatch on.
confirmBudget := genkit.DefineTool(g, "confirmBudget",
    "use this to have the user approve a spend before committing to it",
    func(tc *ai.ToolContext, input BudgetInput) (bool, error) {
        return false, ai.InterruptWith(tc, InterruptMetadata{Reason: "confirm_budget"})
    },
)

const maxRounds = 5

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Help me plan a backyard BBQ."),
    ai.WithSystem("Ask clarifying questions until you have a complete solution."),
    ai.WithTools(askQuestion, confirmBudget),
)
if err != nil {
    return err
}

for round := 0; round < maxRounds && resp.FinishReason == ai.FinishReasonInterrupted; round++ {
    var responses []*ai.Part

    for _, interrupt := range resp.Interrupts() {
        // Every interrupt in this turn needs exactly one response or restart
        // part, so dispatch on the tool that raised it.
        switch interrupt.ToolRequest.Name {
        case askQuestion.Name():
            // RespondWith is typed against the tool's output, so the answer
            // cannot drift from what the tool would have returned.
            part, err := askQuestion.RespondWith(interrupt, getUserAnswer(interrupt))
            if err != nil {
                return err
            }
            responses = append(responses, part)

        case confirmBudget.Name():
            part, err := confirmBudget.RespondWith(interrupt, userApproves(interrupt))
            if err != nil {
                return err
            }
            responses = append(responses, part)

        default:
            return fmt.Errorf("no handler for interrupt from tool %q", interrupt.ToolRequest.Name)
        }
    }

    resp, err = genkit.Generate(ctx, g,
        ai.WithMessages(resp.History()...),
        ai.WithTools(askQuestion, confirmBudget),
        ai.WithToolResponses(responses...),
    )
    if err != nil {
        return err
    }
}

fmt.Println(resp.Text())

RespondWith() and RestartWith() check that the part belongs to the tool you called them on. Handing askQuestion an interrupt raised by confirmBudget returns an INVALID_ARGUMENT error reading tool request is for "confirmBudget", not "askQuestion", so a mismatched branch fails loudly rather than producing a wrong answer.

Answer every interrupt, exactly once

Every interrupt in a turn must get exactly one response or restart part.

• Answer some but not all, and the next Generate fails with ai.ErrUnresolvedToolRequest.
• Answer none of them, and the resume is empty. Genkit treats an empty resume as no resume at all, re-sends the identical history, and a for resp.FinishReason == ai.FinishReasonInterrupted loop never terminates, burning tokens on every pass. That is why the loop above has a round cap and why the default branch returns an error instead of skipping the part.

Out-of-band approval

The examples above collect the answer inline, but the point of an interrupt is usually that the answer arrives later, from a different process. ai.Message and ai.Part are plain structs with JSON tags, so a paused run round-trips through json.Marshal. Store resp.History() and resp.Interrupts() under an approval ID, return the ID to the caller, and rebuild the call when the decision comes back. askQuestion is the tool from above, held in a package-level *ai.ToolAction[QuestionInput, string] so both handlers reach the same one:

// Store is whatever you already run: Firestore, Redis, a table.
type Store interface {
    Put(ctx context.Context, key string, blob []byte) error
    Get(ctx context.Context, key string) ([]byte, error)
}

// A paused run is ordinary JSON.
type pausedRun struct {
    History    []*ai.Message `json:"history"`
    Interrupts []*ai.Part    `json:"interrupts"`
}

// Park the run when generation stops on an interrupt, then hand the caller
// the approval ID and return.
func park(ctx context.Context, store Store, approvalID string, resp *ai.ModelResponse) error {
    blob, err := json.Marshal(pausedRun{
        History:    resp.History(),
        Interrupts: resp.Interrupts(),
    })
    if err != nil {
        return err
    }
    return store.Put(ctx, approvalID, blob)
}

// Later, on a different request and possibly in a different process.
func resume(ctx context.Context, g *genkit.Genkit, store Store, approvalID, answer string) (*ai.ModelResponse, error) {
    blob, err := store.Get(ctx, approvalID)
    if err != nil {
        return nil, err
    }
    var run pausedRun
    if err := json.Unmarshal(blob, &run); err != nil {
        return nil, err
    }

    var responses []*ai.Part
    for _, interrupt := range run.Interrupts {
        if interrupt.ToolRequest.Name != askQuestion.Name() {
            return nil, fmt.Errorf("no handler for interrupt from tool %q", interrupt.ToolRequest.Name)
        }
        part, err := askQuestion.RespondWith(interrupt, answer)
        if err != nil {
            return nil, err
        }
        responses = append(responses, part)
    }

    return genkit.Generate(ctx, g,
        ai.WithMessages(run.History...),
        // Every tool named by a pending request has to be on this call, or
        // Generate fails with ai.ErrToolNotFound.
        ai.WithTools(askQuestion),
        ai.WithToolResponses(responses...),
    )
}

ai.WithToolRestarts() rehydrates the same way when the tool has to run again rather than be answered outright.

Agent interrupts do this persistence for you: the session store holds the paused history, so the resuming request carries only the decision.

Resuming from an untrusted caller

Generate matches each restart or respond part to a pending tool request by tool name and ref only. It does not check that a restarted input matches what the model originally asked for, and a part that matches nothing pending is silently dropped rather than rejected. Respond output is validated against the tool’s output schema and nothing more.

So if the resume payload arrives over the network, re-read the paused history from your own store and verify every part against it before calling Generate. The agent runtime does exactly this with aix.ValidateResumeAgainstHistory(resume, history), described in Agent interrupts.

Tools with restartable interrupts

Another common pattern is the need to confirm an action that the LLM suggests before actually performing it. For example, a payments app might want the user to confirm certain kinds of transfers. The basic-tool-interrupts sample is this whole pattern as a runnable program.

Define a restartable tool

A restartable tool reads two things from its *ai.ToolContext: IsResumed() tells a first call apart from a restarted one, and ai.ResumedValue() reads back the decision the caller attached when restarting it. Letting the tool decide, rather than the caller, keeps the rule it paused on in one place:

type TransferInput struct {
    ToAccount string  `json:"toAccount"`
    Amount    float64 `json:"amount"`
}

type TransferOutput struct {
    Status     string  `json:"status"`
    Message    string  `json:"message,omitempty"`
    NewBalance float64 `json:"newBalance,omitempty"`
}

type TransferInterrupt struct {
    Reason    string  `json:"reason"` // "insufficient_balance" or "confirm_large"
    ToAccount string  `json:"toAccount"`
    Amount    float64 `json:"amount"`
    Balance   float64 `json:"balance,omitempty"`
}

transferMoney := genkit.DefineTool(g, "transferMoney",
    "Transfers money to another account.",
    func(tc *ai.ToolContext, input TransferInput) (TransferOutput, error) {
        // More than the account holds: pause and say so.
        if input.Amount > accountBalance {
            return TransferOutput{}, ai.InterruptWith(tc, TransferInterrupt{
                Reason:    "insufficient_balance",
                ToAccount: input.ToAccount,
                Amount:    input.Amount,
                Balance:   accountBalance,
            })
        }

        // IsResumed is false on the first call and true once the tool has
        // been restarted, which is what tells a fresh large transfer from
        // one that has already been answered.
        if !tc.IsResumed() && input.Amount > 100 {
            return TransferOutput{}, ai.InterruptWith(tc, TransferInterrupt{
                Reason:    "confirm_large",
                ToAccount: input.ToAccount,
                Amount:    input.Amount,
            })
        }

        // The decision arrives as resumed metadata, so it is read back one
        // key at a time.
        if approved, ok := ai.ResumedValue[bool](tc, "approved"); ok && !approved {
            return TransferOutput{
                Status:  "declined",
                Message: "The user declined the transfer.",
            }, nil
        }

        accountBalance -= input.Amount
        return TransferOutput{
            Status:     "completed",
            Message:    "Transfer successful",
            NewBalance: accountBalance,
        }, nil
    },
)

Restart tools after interruption

Use the tool’s RestartWith() method and ai.WithToolRestarts() to run an interrupted tool again. ai.WithResumedMetadata() is how the decision travels back into the tool: whatever you put in that map is what ai.ResumedValue() reads. ai.WithNewInput() restarts the tool with different arguments instead:

resp, err := genkit.Generate(ctx, g,
    ai.WithPrompt("Transfer $200 to account ABC123"),
    ai.WithTools(transferMoney),
)
if err != nil {
    return err
}

for round := 0; round < maxRounds && resp.FinishReason == ai.FinishReasonInterrupted; round++ {
    var restarts, responses []*ai.Part

    for _, interrupt := range resp.Interrupts() {
        meta, ok := ai.InterruptAs[TransferInterrupt](interrupt)
        if !ok {
            // Skipping the part here would leave the interrupt unanswered,
            // so fail instead.
            return fmt.Errorf("interrupt from %q carried no TransferInterrupt metadata",
                interrupt.ToolRequest.Name)
        }

        switch meta.Reason {
        case "confirm_large":
            // The answer travels as resumed metadata. The tool reads it
            // back with ai.ResumedValue and decides what to do.
            approved := userConfirms("Confirm transfer of $%.2f?", meta.Amount)
            part, err := transferMoney.RestartWith(interrupt,
                ai.WithResumedMetadata[TransferInput](map[string]any{"approved": approved}))
            if err != nil {
                return err
            }
            restarts = append(restarts, part)

        case "insufficient_balance":
            if userConfirms("Transfer $%.2f instead?", meta.Balance) {
                // Restart with a different input.
                part, err := transferMoney.RestartWith(interrupt,
                    ai.WithNewInput(TransferInput{ToAccount: meta.ToAccount, Amount: meta.Balance}))
                if err != nil {
                    return err
                }
                restarts = append(restarts, part)
            } else {
                // Answer the call outright, without running the tool again.
                part, err := transferMoney.RespondWith(interrupt, TransferOutput{
                    Status:     "cancelled",
                    Message:    "Transfer cancelled by user.",
                    NewBalance: accountBalance,
                })
                if err != nil {
                    return err
                }
                responses = append(responses, part)
            }

        default:
            // An unrecognized reason still has to be answered.
            part, err := transferMoney.RespondWith(interrupt, TransferOutput{
                Status:  "cancelled",
                Message: fmt.Sprintf("Unhandled interrupt reason %q.", meta.Reason),
            })
            if err != nil {
                return err
            }
            responses = append(responses, part)
        }
    }

    resp, err = genkit.Generate(ctx, g,
        ai.WithMessages(resp.History()...),
        ai.WithTools(transferMoney),
        ai.WithToolRestarts(restarts...),
        ai.WithToolResponses(responses...),
    )
    if err != nil {
        return err
    }
}

fmt.Println(resp.Text())

The type parameter on ai.WithResumedMetadata[TransferInput] is the tool’s input type, and you have to write it out because the option’s only argument is a map[string]any, which gives the compiler nothing to infer from. Its sibling ai.WithNewInput is generic on the same parameter but infers it from the value you pass. Getting it wrong is a compile error rather than a runtime surprise: RestartWith on a *ai.ToolAction[In, Out] accepts only an ai.RestartWithOption[In].

Access original input after replacement

When you use ai.WithNewInput(), you can access the original input inside the tool using ai.OriginalInputAs():

transferMoney := genkit.DefineTool(g, "transferMoney",
    "Transfers money to another account.",
    func(tc *ai.ToolContext, input TransferInput) (TransferOutput, error) {
        // ... interrupt logic ...

        accountBalance -= input.Amount
        message := fmt.Sprintf("Transferred $%.2f to %s", input.Amount, input.ToAccount)

        // Report the adjustment when the caller replaced the input.
        if orig, ok := ai.OriginalInputAs[TransferInput](tc); ok {
            message = fmt.Sprintf("Transferred $%.2f to %s (adjusted from $%.2f)",
                input.Amount, input.ToAccount, orig.Amount)
        }

        return TransferOutput{
            Status:     "completed",
            Message:    message,
            NewBalance: accountBalance,
        }, nil
    },
)

Typed resume payloads

In preview

The genkit/exp tools API is in preview. It may change in any minor release.

With the stable API, the resume payload is a metadata map: the tool reads "approved" with ai.ResumedValue() and the caller writes "approved" with ai.WithResumedMetadata(), and nothing checks that the two agree. The in-preview genkitx.DefineInterruptibleTool() takes a third type parameter for what comes back on the resume, so the payload becomes a real type that both ends share.

The tool function takes a plain context.Context and a pointer to the resume type. The pointer is nil on the first call and set when the tool is resumed, which replaces IsResumed(). tool.Interrupt() pauses with a typed value in place of ai.InterruptWith(), and the tool’s Resume() method carries the typed answer in place of RestartWith() plus a metadata map. Everything else, including ai.WithToolRestarts() and the two-turn shape, is unchanged:

import (
    "context"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/ai/exp/tool"
    "github.com/firebase/genkit/go/genkit"
    genkitx "github.com/firebase/genkit/go/genkit/exp"
)

// Approval is the answer carried back into the tool when it is resumed. It is
// the tool's third type parameter, so both ends share one type instead of
// agreeing on metadata keys.
type Approval struct {
    Approved bool `json:"approved"`
}

// The in-preview tool constructors panic without this option.
g := genkit.Init(ctx, genkit.WithExperimental())

transferMoney := genkitx.DefineInterruptibleTool(g, "transferMoney",
    "Transfers money to another account.",
    func(ctx context.Context, input TransferInput, approval *Approval) (*TransferOutput, error) {
        // approval is nil on the first call and set when the tool is
        // resumed, which is what tells a fresh large transfer from one
        // that has already been answered.
        if approval == nil && input.Amount > 100 {
            return nil, tool.Interrupt(TransferInterrupt{
                ToAccount: input.ToAccount,
                Amount:    input.Amount,
            })
        }
        if approval != nil && !approval.Approved {
            return &TransferOutput{Status: "declined", NewBalance: accountBalance}, nil
        }

        accountBalance -= input.Amount
        return &TransferOutput{Status: "completed", NewBalance: accountBalance}, nil
    })

Resuming reads the interrupt with tool.InterruptAs() and answers it with the tool’s Resume() method:

for resp.FinishReason == ai.FinishReasonInterrupted {
    var restarts []*ai.Part

    for _, interrupt := range resp.Interrupts() {
        meta, ok := tool.InterruptAs[TransferInterrupt](interrupt)
        if !ok {
            continue
        }
        // Resume carries a typed Approval, so neither end has to agree on
        // a metadata key.
        part, err := transferMoney.Resume(interrupt, Approval{
            Approved: userConfirms("Confirm transfer of $%.2f?", meta.Amount),
        })
        if err != nil {
            return err
        }
        restarts = append(restarts, part)
    }

    resp, err = genkit.Generate(ctx, g,
        ai.WithMessages(resp.History()...),
        ai.WithTools(transferMoney),
        ai.WithToolRestarts(restarts...),
    )
    if err != nil {
        return err
    }
}

Both the interrupt value and the resume value must serialize to a JSON object, so use a struct or a map. A scalar or a slice fails at runtime, not at compile time. The tool’s Respond() method is the in-preview counterpart of RespondWith(), answering a paused call without running the tool again.

The basic-tool-interrupts-exp sample is the same program as basic-tool-interrupts written against this in-preview API, so reading the two side by side shows exactly what changes.