OpenAI-compatible plugin

The OpenAI-Compatible API package (compat_oai) provides a unified interface for accessing multiple AI providers that implement OpenAI’s API specification. This includes OpenAI, Anthropic, and other compatible services.

Overview

The compat_oai package serves as a foundation for building plugins that work with OpenAI-compatible APIs. It includes:

• Base Implementation: Common functionality for OpenAI-compatible APIs
• OpenAI Plugin: Direct access to OpenAI’s models and embeddings
• Anthropic Plugin: Access to Claude models through OpenAI-compatible endpoints
• Extensible Framework: Build custom plugins for other compatible providers

Prerequisites

Depending on which provider you use, you’ll need:

• OpenAI: API key from OpenAI API Keys page
• Anthropic: API key from Anthropic Console
• Other providers: API keys from the respective services

Configuration

Use with Compatible Providers

import (
    "context"
    "github.com/firebase/genkit/go/genkit"
    "github.com/firebase/genkit/go/plugins/compat_oai"
    "github.com/openai/openai-go/option"
)

ctx := context.Background()

// Initialize Genkit with the OpenAICompatible plugin
g := genkit.Init(ctx,
    genkit.WithPlugins(&compat_oai.OpenAICompatible{
        Provider: "myprovider",
        APIKey:   "YOUR_API_KEY",
        BaseURL:  "https://your-custom-endpoint.com/v1",
        Opts: []option.RequestOption{
            option.WithOrganization("your-org-id"),
            option.WithHeader("Custom-Header", "value"),
        },
    }),
)

Common Configuration

OpenAI-compatible configuration options are supported:

import (
    "context"
    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/genkit"
    "github.com/openai/openai-go"
)

ctx := context.Background()

config := &openai.ChatCompletionNewParams{
  Temperature:         openai.Float(0.7),
  MaxCompletionTokens: openai.Int(1000),
  TopP:                openai.Float(0.9),
}

resp, err := genkit.Generate(ctx, g,
  ai.WithModelName("myprovider/model-name"),
  ai.WithPrompt("Your prompt here"),
  ai.WithConfig(config),
)