Deploy to any platform

You can deploy Genkit flows as web services using any service that can host a Go binary. This page, as an example, walks you through the general process of deploying the default sample flow, and points out where you must take provider-specific actions.

1. Set up your project

Create a directory for the Genkit sample project:

mkdir -p ~/tmp/genkit-cloud-project

cd ~/tmp/genkit-cloud-project

If you’re going to use an IDE, open it to this directory.

Initialize a Go module in your project directory:

go mod init example/cloudrun

go get github.com/firebase/genkit/go

2. Configure your Genkit app

package main

import (
    "context"
    "fmt"
    "log"
    "net/http"
    "os"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/genkit"
    "github.com/firebase/genkit/go/plugins/googlegenai"
    "github.com/firebase/genkit/go/plugins/server"
)

func main() {
    ctx := context.Background()

    // Initialize Genkit with the Google AI plugin and the latest Gemini Flash model.
    // Alternatively, use &googlegenai.VertexAI{} and "vertexai/gemini-flash-latest"
    // to use Vertex AI as the provider instead.
    g := genkit.Init(ctx,
        genkit.WithPlugins(&googlegenai.GoogleAI{}),
        genkit.WithDefaultModel("googleai/gemini-flash-latest"),
    )

    flow := genkit.DefineFlow(g, "jokesFlow", func(ctx context.Context, topic string) (string, error) {
        resp, err := genkit.Generate(ctx, g,
            ai.WithPrompt(`Tell a short joke about %s. Be creative!`, topic),
        )
        if err != nil {
            return "", fmt.Errorf("failed to generate joke: %w", err)
        }

        return resp.Text(), nil
    })

    mux := http.NewServeMux()
    mux.HandleFunc("POST /jokesFlow", genkit.Handler(flow))

    // Bind 0.0.0.0, not 127.0.0.1: inside a container, a loopback-only
    // listener is unreachable from the platform's health check and from any
    // other container. Fall back to a port so `go run .` works locally.
    port := os.Getenv("PORT")
    if port == "" {
        port = "8080"
    }
    // server.Start traps SIGINT and SIGTERM and drains in-flight requests
    // before returning, which is what most platforms expect on a redeploy.
    log.Fatal(server.Start(ctx, "0.0.0.0:"+port, mux))
}

3. Gate access to the flow

Implement some form of authentication and authorization before you deploy. Because most generative AI services are metered, you most likely do not want to allow open access to any endpoint that calls them, and genkit.Handler performs no check of its own.

Some hosting services provide an authentication layer as a frontend to apps deployed on them, which you can use for this purpose. For a check inside your own binary, see Securing your deployment.

4. Make API credentials available

Do one of the following, depending on the model provider you chose.

Gemini (Google AI)

1. Make sure Google AI is available in your region.

2. Generate an API key for the Gemini API using Google AI Studio.

3. Make the API key available in the deployed environment.

Most app hosts provide some system for securely handling secrets such as API keys. Often, these secrets are available to your app in the form of environment variables. If you can assign your API key to the GEMINI_API_KEY variable, Genkit will use it automatically. Otherwise, you need to modify the googlegenai.GoogleAI plugin struct to explicitly set the key. (But don’t embed the key directly in code! Use the secret management facilities provided by your hosting provider.)

Gemini (Vertex AI)

1. In the Cloud console, Enable the Vertex AI API for your project.

2. On the IAM page, create a service account for accessing the Vertex AI API if you don’t already have one.

   Grant the account the Vertex AI User role.

3. Set up Application Default Credentials in your hosting environment.

4. Configure the plugin with your Google Cloud project ID and the Vertex AI API location you want to use. You can do so either by setting the GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_LOCATION environment variables in your hosting environment, or in your googlegenai.VertexAI{} constructor.

The only secret you need to set up for this tutorial is for the model provider, but in general, you must do something similar for each service your flow uses.

Optional: Try your flow in the developer UI

1. Set up your local environment for the model provider you chose.

   Gemini (Google AI)

export GEMINI_API_KEY=<your API key>

Gemini (Vertex AI)

export GOOGLE_CLOUD_PROJECT=<your project ID>

export GOOGLE_CLOUD_LOCATION=us-central1

gcloud auth application-default login

2. Start the UI:

genkit start -- go run .

3. In the developer UI (http://localhost:4000/), click jokesFlow.

4. On the Input JSON tab, provide a subject for the model:

"bananas"

5. Click Run.

5. Build and deploy

If everything’s working as expected so far, you can build and deploy the flow using your provider’s tools.

Running in production

What server.Start already does

server.Start is not a development-only helper. It installs a signal.NotifyContext for os.Interrupt and SIGTERM, and on either signal it calls http.Server.Shutdown, which stops accepting connections and waits for in-flight requests to finish. It also stops intercepting signals at that point, so a second interrupt kills the process immediately instead of hanging.

The drain window is fixed at five seconds. Requests still running when it expires are cut off.

When five seconds is not enough

A single model turn frequently runs longer than five seconds, and a tool loop runs much longer. If your turns are long, run your own http.Server with a deadline you choose. This is also where you add health and readiness endpoints, which Genkit does not register for you:

package main

import (
    "context"
    "errors"
    "log"
    "net/http"
    "os"
    "os/signal"
    "sync/atomic"
    "syscall"
    "time"

    "github.com/firebase/genkit/go/genkit"
)

func main() {
    ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
    defer stop()

    g := genkit.Init(ctx)
    flow := genkit.DefineFlow(g, "jokesFlow", func(ctx context.Context, topic string) (string, error) {
        return topic, nil // Replace with the flow body from step 2.
    })

    // ready flips to false the moment shutdown begins, so the load balancer
    // stops sending new requests while the old ones drain.
    var ready atomic.Bool
    ready.Store(true)

    mux := http.NewServeMux()
    mux.HandleFunc("POST /jokesFlow", genkit.Handler(flow))
    mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) {
        w.WriteHeader(http.StatusOK)
    })
    mux.HandleFunc("GET /readyz", func(w http.ResponseWriter, r *http.Request) {
        if !ready.Load() {
            http.Error(w, "shutting down", http.StatusServiceUnavailable)
            return
        }
        w.WriteHeader(http.StatusOK)
    })

    port := os.Getenv("PORT")
    if port == "" {
        port = "8080"
    }
    srv := &http.Server{Addr: "0.0.0.0:" + port, Handler: mux}

    go func() {
        if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
            log.Fatalf("server error: %v", err)
        }
    }()

    <-ctx.Done()
    stop() // A second interrupt now kills the process immediately.
    ready.Store(false)

    shutdownCtx, cancel := context.WithTimeout(context.Background(), 9*time.Second)
    defer cancel()
    if err := srv.Shutdown(shutdownCtx); err != nil {
        log.Printf("graceful shutdown failed: %v", err)
    }
}

Keep the deadline inside your platform’s SIGTERM grace period. Cloud Run kills the container ten seconds after SIGTERM by default, so a 25-second drain there buys you nothing.

The reflection server is not exposed

genkit.Init starts the Dev UI reflection server on port 3100 only when GENKIT_ENV is set to dev. In production, leave GENKIT_ENV unset and no extra listener is opened.