Ollama plugin

The Ollama plugin provides interfaces to any of the local LLMs supported by Ollama.

Prerequisites

This plugin requires that you first install and run the Ollama server. You can follow the instructions on the Download Ollama page.

Use the Ollama CLI to download the models you are interested in. For example:

ollama pull gemma3

For development, you can run Ollama on your development machine. Deployed apps usually run Ollama on a GPU-accelerated machine that is different from the one hosting the app backend running Genkit.

Configuration

To use this plugin, pass ollama.Ollama to WithPlugins() in the Genkit initializer, specifying the address of your Ollama server and the response timeout (defaulted to 30 seconds):

import (
    "context"
    "log"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/genkit"
    "github.com/firebase/genkit/go/plugins/ollama"
)

func main() {
  ctx := context.Background()

  ollamaPlugin := &ollama.Ollama{
      ServerAddress: "http://127.0.0.1:11434",
      Timeout:       60, // Optional field, adjust accordingly
    }

  g := genkit.Init(ctx, genkit.WithPlugins(ollamaPlugin))
}

Timeout bounds each generate request, the streamed body included, as a deadline on the request’s context.

Hosted and proxied servers

A server behind authentication takes two more fields. Headers are sent with every request the plugin makes, and win over the plugin’s own. When Headers has no Authorization header, the plugin sends the OLLAMA_API_KEY environment variable as a bearer token, as Ollama’s own clients do, so a hosted server needs no code at all. (The snippets in this section add net/http, os, and golang.org/x/oauth2 to the imports.)

// OLLAMA_API_KEY is set in the environment.
g := genkit.Init(ctx, genkit.WithPlugins(&ollama.Ollama{
    ServerAddress: "https://ollama.com",
}))

g := genkit.Init(ctx, genkit.WithPlugins(&ollama.Ollama{
    ServerAddress: "https://ollama.example.com",
    Headers:       http.Header{"X-Api-Key": []string{os.Getenv("PROXY_KEY")}},
}))

A credential that changes per request belongs in the transport. HTTPClient replaces the client the plugin sends every request with, so a client from oauth2.NewClient or idtoken.NewClient refreshes its own token, and the same field covers proxies, TLS, and instrumentation:

g := genkit.Init(ctx, genkit.WithPlugins(&ollama.Ollama{
    ServerAddress: "https://ollama.example.com",
    HTTPClient:    oauth2.NewClient(ctx, tokenSource),
}))

Timeout still applies to a client you supply, on top of any timeout the client sets itself.

Usage

Name any model your Ollama server has pulled. The plugin resolves it on demand, so no registration call is needed:

resp, err := genkit.Generate(ctx,
  g,
  ai.WithModelName("ollama/gemma3"),
  ai.WithPrompt("Tell me a joke."),
)
if err != nil {
    return err
}

log.Println(resp.Text())

The tag rides through to Ollama as written, so ollama/gemma3 and ollama/gemma3:latest both reach the same model.

You can also register a model explicitly and pass the reference around:

model := ollamaPlugin.DefineModel(
  g,
  ollama.ModelDefinition{
    Name: "gemma3",
    Type: "chat", // "chat" or "generate"
  },
  nil, // Take the capabilities the plugin knows about.
)

resp, err := genkit.Generate(ctx,
  g,
  ai.WithModel(model),
  ai.WithPrompt("Tell me a joke."),
)
if err != nil {
    return err
}

log.Println(resp.Text())

Model capabilities

Local models differ widely in what they accept, so the plugin asks the server instead of guessing from the model name. Detection runs during model discovery, which is what the Dev UI’s model list triggers: the plugin lists the installed models with GET /api/tags, drops any name containing embed, then calls POST /api/show for each of the rest, at most four at a time, each call bounded by a five second timeout or by Timeout seconds when that is shorter. The capabilities the server reports become the model’s ai.ModelSupports: tools sets Tools, vision sets Media, and Multiturn and SystemRole are always true. The ollama-tools and ollama-vision samples exercise one capability each.

Results are cached in the process and keyed by model name, checked against the digest /api/tags reports so that a re-pulled model is detected again. A successful detection is kept for the life of the process; a failed one is retried after 30 seconds. When detection fails, the model falls back to claiming every capability and the plugin logs a warning.

Resolving a model by name reads that cache and never queries the server, so a model discovery has not covered yet gets the same permissive fallback.

Overriding detected capabilities

Pass a non-nil *ai.ModelOptions to DefineModel to state the capabilities yourself:

model := ollamaPlugin.DefineModel(
  g,
  ollama.ModelDefinition{Name: "gemma3", Type: "chat"},
  &ai.ModelOptions{
    Supports: &ai.ModelSupports{
      Multiturn:  true,
      SystemRole: true,
      Tools:      true,
      Media:      true,
    },
  },
)

With a nil opts, DefineModel reuses a successful detection for that model when one is already cached, and falls back to a built-in name list otherwise. It then turns Tools off unless Type is "chat", because only the chat endpoint accepts tools.

That is the only override the plugin offers: there is no map of capability overrides on the plugin struct. It describes the exact name you define, so ollama/gemma3 and ollama/gemma3:latest are two names and only the one you defined carries your options. It also leaves discovery alone, so the Dev UI list still reports what the server said.

See Generating content for more information.