Deploy with Cloud Run

You can deploy Genkit flows as web services using Cloud Run. This page, as an example, walks you through the process of deploying the default sample flow.

Before you begin

• Install the Google Cloud CLI if you haven’t already.
• You should be familiar with Genkit’s concept of flows.
• It would be helpful, but not required, if you’ve already used Google Cloud and Cloud Run before.

1. Set up a Google Cloud project

Create a new Google Cloud project using the Cloud console or choose an existing one. The project must be linked to a billing account.

After you create or choose a project, configure the Google Cloud CLI to use it:

gcloud auth login

gcloud init

2. Prepare your Go project for deployment

Create the project directory

mkdir -p ~/tmp/genkit-cloud-project

cd ~/tmp/genkit-cloud-project

If you’re going to use an IDE, open it to this directory.

Initialize a Go module in your project directory:

go mod init example/cloudrun

go get github.com/firebase/genkit/go

Add code to configure and start the flow server

package main

import (
    "context"
    "fmt"
    "log"
    "net/http"
    "os"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/genkit"
    "github.com/firebase/genkit/go/plugins/googlegenai"
    "github.com/firebase/genkit/go/plugins/server"
)

func main() {
    ctx := context.Background()

    // Initialize Genkit with the Google AI plugin and the latest Gemini Flash model.
    // Alternatively, use &googlegenai.VertexAI{} and "vertexai/gemini-flash-latest"
    // to use Vertex AI as the provider instead.
    g := genkit.Init(ctx,
        genkit.WithPlugins(&googlegenai.GoogleAI{}),
        genkit.WithDefaultModel("googleai/gemini-flash-latest"),
    )

    flow := genkit.DefineFlow(g, "jokesFlow", func(ctx context.Context, topic string) (string, error) {
        resp, err := genkit.Generate(ctx, g,
            ai.WithPrompt(`Tell a short joke about %s. Be creative!`, topic),
        )
        if err != nil {
            return "", fmt.Errorf("failed to generate joke: %w", err)
        }

        return resp.Text(), nil
    })

    mux := http.NewServeMux()
    mux.HandleFunc("POST /jokesFlow", genkit.Handler(flow))

    port := os.Getenv("PORT")
    if port == "" {
        port = "8080" // Cloud Run always sets PORT; this keeps `go run .` working.
    }
    // server.Start traps SIGINT and SIGTERM and drains in-flight requests for
    // up to five seconds, which is the shutdown behavior Cloud Run expects.
    log.Fatal(server.Start(ctx, "0.0.0.0:"+port, mux))
}

Define an authorization policy

genkit.Handler does no authentication. Anything you register on the mux is callable by anyone who can reach the service URL, and every call spends model quota. Decide now which of the two options you want:

• Cloud IAM. Answer N in the deploy step below so Cloud Run rejects unauthenticated callers before the request reaches your process.
• A check in your own code. Wrap each flow handler in middleware that validates the caller. See Securing your deployment for the pattern.

Make API credentials available to deployed flows

Choose which credentials you need based on your choice in the sample above.

Gemini (Google AI)

1. Make sure Google AI is available in your region.

2. Generate an API key for the Gemini API using Google AI Studio.

3. Make the API key available in the Cloud Run environment:

   1. In the Cloud console, enable the Secret Manager API.
   2. On the Secret Manager page, create a new secret containing your API key.
   3. After you create the secret, on the same page, grant your default compute service account access to the secret with the Secret Manager Secret Accessor role. (You can look up the name of the default compute service account on the IAM page.)

   In a later step, when you deploy your service, you will need to reference the name of this secret.

Gemini (Vertex AI)

1. In the Cloud console, Enable the Vertex AI API for your project.

2. On the IAM page, ensure that the Default compute service account is granted the Vertex AI User role.

The only secret you need to set up for this tutorial is for the model provider, but in general, you must do something similar for each service your flow uses.

Optional: Try your flow in the developer UI

1. Set up your local environment for the model provider you chose.

   Gemini (Google AI)

export GEMINI_API_KEY=<your API key>

Gemini (Vertex AI)

export GOOGLE_CLOUD_PROJECT=<your project ID>

export GOOGLE_CLOUD_LOCATION=us-central1

gcloud auth application-default login

2. Start the UI:

genkit start -- go run .

3. In the developer UI (http://localhost:4000/), click jokesFlow.

4. On the Input JSON tab, provide a subject for the model:

"bananas"

5. Click Run.

3. Deploy to Cloud Run

If everything’s working as expected so far, you can build and deploy the flow.

Gemini (Google AI)

gcloud run deploy --port 3400 \
  --update-secrets=GEMINI_API_KEY=<your-secret-name>:latest

Gemini (Vertex AI)

gcloud run deploy --port 3400 \
  --set-env-vars GOOGLE_CLOUD_PROJECT=<your-gcloud-project> \
  --set-env-vars GOOGLE_CLOUD_LOCATION=us-central1

(GOOGLE_CLOUD_LOCATION configures the Vertex API region you want to use.)

Choose N when asked if you want to allow unauthenticated invocations. Answering N will configure your service to require IAM credentials. See Authentication in the Cloud Run docs for information on providing these credentials.

Optional: Try the deployed flow

After deployment finishes, the tool will print the service URL. You can test it with curl:

curl -X POST https://<service-url>/jokesFlow \
  -H "Authorization: Bearer $(gcloud auth print-identity-token)" \
  -H "Content-Type: application/json" -d '{"data": "bananas"}'