Creating persistent chat sessions

The examples on this page use these imports:

import (
    "context"
    "fmt"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/genkit"
)

// ChatTurn is one question plus the conversation it continues.
type ChatTurn struct {
    Question string        `json:"question"`
    History  []*ai.Message `json:"history,omitempty"`
}

Note

Genkit Go has no dedicated Chat type. A conversation is either a plain multi-turn Generate call whose history the caller carries, shown on this page, or an agent, which owns the history, persists it in a session store, and streams each turn.

Many of your users will have interacted with large language models for the first time through chatbots. Although LLMs are capable of much more than simulating conversations, it remains a familiar and useful style of interaction. Even when your users will not be interacting directly with the model in this way, the conversational style of prompting is a powerful way to influence the output generated by an AI model.

Genkit Go gives you two ways to hold a conversation: pass the history along with each request, or let an agent keep it for you.

Before you begin

Before reading this page, you should be familiar with the content covered on the Generating content with AI models page.

If you want to run the code examples on this page, first complete the steps in the Getting started guide. All of the examples assume that you have already installed Genkit as a dependency in your project.

Multi-turn conversations

Every response carries the whole conversation that produced it. resp.History() returns the messages you sent plus the model’s reply, so a flow can hand that back to its caller and accept it again on the next turn:

// ChatReply hands the updated conversation back to the caller.
type ChatReply struct {
    Answer  string        `json:"answer"`
    History []*ai.Message `json:"history"`
}

genkit.DefineFlow(g, "chat", func(ctx context.Context, in ChatTurn) (ChatReply, error) {
    resp, err := genkit.Generate(ctx, g,
        ai.WithModelName("googleai/gemini-flash-latest"),
        ai.WithSystem("You are a helpful assistant. Keep your answers short."),
        ai.WithMessages(in.History...),
        ai.WithPrompt(in.Question),
    )
    if err != nil {
        return ChatReply{}, fmt.Errorf("could not answer the question: %w", err)
    }

    // History is everything the model saw plus what it just said, so the
    // caller sends it straight back with the next question.
    return ChatReply{Answer: resp.Text(), History: resp.History()}, nil
})

ai.WithMessages places the history between the system instruction and the new user message. A prompt can claim that position for itself by marking it with {{history}}, which is how a template scripts an opening exchange ahead of the real conversation. The basic-prompts sample defines the same chat flow both ways, once inline in code and once as a .prompt file, so the pair shows exactly what moves.

Persistent sessions

Carrying the history in and out of a flow works while the caller can hold it. When the conversation has to survive across processes, or carries state beyond the transcript, define an agent instead. An agent owns its history, writes a snapshot to a session store after every turn, and exposes its typed state to its own prompt as {{@state.fieldName}}, so a tool can update the state and the next turn’s instruction reflects it.

The basic-agents sample defines seven agents in seven styles behind one CLI, including a typed-state agent whose tools read and write the session. The agent APIs are in preview, so initialize Genkit with genkit.WithExperimental() to use them.

Next steps

• Learn about tool calling to let the model act during a conversation
• Explore prompts to move a conversation’s wording out of code
• See flows for structuring the application around it