Pinecone Vector Database

The Pinecone plugin provides retriever implementatons that use the Pinecone cloud vector database.

Pinecone is a cloud-native vector database that provides fast, scalable similarity search for AI applications. It offers managed infrastructure with automatic scaling and high availability.

Configuration

To use this plugin, import the pinecone package and call pinecone.Init():

import "github.com/firebase/genkit/go/plugins/pinecone"

if err := (&pinecone.Pinecone{}).Init(ctx, g); err != nil {
    log.Fatal(err)
}

The plugin requires your Pinecone API key. Configure the plugin to use your API key by doing one of the following:

• Set the PINECONE_API_KEY environment variable to your API key.

• Specify the API key when you initialize the plugin:

  if err := (&pinecone.Pinecone{APIKey: pineconeAPIKey}).Init(ctx, g); err != nil {
      log.Fatal(err)
  }

However, don’t embed your API key directly in code! Use this feature only in conjunction with a service like Cloud Secret Manager or similar.

Usage

Index your documents in pinecone. An example of indexing is provided within the Pinecone plugin as shown below. This functionality should be customized by the user according to their use case.

err := pinecone.Index(ctx, docChunks, ds, "")
if err != nil {
    log.Fatal(err)
}

To retrieve documents from an index, first create a retriever definition:

menuRetriever, err := pinecone.DefineRetriever(ctx, g, pinecone.Config{
    IndexID:  "menu_data",                                           // Your Pinecone index
    Embedder: googlegenai.GoogleAIEmbedder(g, "text-embedding-004"), // Embedding model of your choice
})
if err != nil {
    log.Fatal(err)
}

Then, call the retriever’s Retrieve() method, passing it a text query:

resp, err := genkit.Retrieve(ctx, g, ai.WithRetriever(menuRetriever), ai.WithTextDocs(userInput))
if err != nil {
    log.Fatal(err)
}
menuInfo := resp.Documents

See the Retrieval-augmented generation page for a general discussion on using retrievers for RAG.