Pinecone vector database

The Pinecone plugin provides retriever implementatons that use the Pinecone cloud vector database.

Pinecone is a cloud-native vector database that provides fast, scalable similarity search for AI applications. It offers managed infrastructure with automatic scaling and high availability.

Configuration

To use this plugin, import the pinecone package and initialize it via Genkit’s WithPlugins:

import "github.com/firebase/genkit/go/plugins/pinecone"

g := genkit.Init(ctx, genkit.WithPlugins(&pinecone.Pinecone{}))

The plugin requires your Pinecone API key. Configure the plugin to use your API key by doing one of the following:

• Set the PINECONE_API_KEY environment variable to your API key.

• Specify the API key when you initialize the plugin:

g := genkit.Init(ctx, genkit.WithPlugins(&pinecone.Pinecone{
    APIKey: pineconeAPIKey,
}))

However, don’t embed your API key directly in code! Use this feature only in conjunction with a service like Cloud Secret Manager or similar.

Usage

To retrieve documents from an index or index new documents, first create a retriever definition which returns a Docstore (ds) and an ai.Retriever:

ds, menuRetriever, err := pinecone.DefineRetriever(ctx, g, pinecone.Config{
    IndexID:  "menu_data",                                           // Your Pinecone index
    Embedder: googlegenai.GoogleAIEmbedder(g, "text-embedding-004"), // Embedding model of your choice
}, nil)
if err != nil {
    log.Fatal(err)
}

Index your documents in Pinecone. An example of indexing is provided within the Pinecone plugin as shown below. This functionality should be customized by the user according to their use case.

err = pinecone.Index(ctx, docChunks, ds, "")
if err != nil {
    log.Fatal(err)
}

Then, call Genkit’s Retrieve() method, passing it the retriever and a text query:

resp, err := genkit.Retrieve(ctx, g, ai.WithRetriever(menuRetriever), ai.WithTextDocs(userInput))
if err != nil {
    log.Fatal(err)
}
menuInfo := resp.Documents

See the Retrieval-augmented generation page for a general discussion on using retrievers for RAG.