Skip to content

AWS Bedrock plugin

The genkit-amazon-bedrock package provides access to models hosted on Amazon Bedrock through the Genkit framework. It supports text generation with Bedrock-hosted models (Anthropic Claude, Amazon Nova, Meta Llama, Mistral, Cohere, and others) through the Bedrock Converse and ConverseStream APIs. Embeddings and Stability image generation go through InvokeModel.

Terminal window
uv add genkit-amazon-bedrock
from genkit import Genkit
from genkit_amazon_bedrock import Bedrock
ai = Genkit(
plugins=[
Bedrock(
region='us-east-1',
models=['us.anthropic.claude-sonnet-4-5-20250929-v1:0'],
)
],
model=Bedrock.text('us.anthropic.claude-sonnet-4-5-20250929-v1:0'),
)

The region comes from region= or the AWS SDK chain (AWS_REGION, AWS_DEFAULT_REGION, ~/.aws/config). There is no default, so initialization fails if none resolves. Bedrock.text(), Bedrock.stability(), and Bedrock.embedder() attach typed config; 'bedrock/<model-id>' strings behave the same at runtime.

Other plugin parameters:

  • embedders: embedding model IDs to list in the Dev UI.
  • session: a boto3.session.Session for custom credentials.
  • max_retries, connect_timeout, read_timeout, max_pool_connections: unset by default, so your AWS config applies. read_timeout resets on every byte, so it limits silence, not call length.
  • total_timeout: deadline for a whole non-streaming call, retries included. Defaults to 3600 seconds.

The plugin README lists every fallback.

Model access is granted per AWS account and per region, in the Bedrock console under Model access. Most models cannot be called until access is granted. A grant in us-east-1 says nothing about us-west-2, so a working setup can break purely by changing region.

The Anthropic models additionally need the account’s one-time use-case agreement (Bedrock console, Model access, Anthropic use case details). Until the agreement is accepted, Claude calls fail with ResourceNotFoundException.

The minimal policy covering everything this plugin does:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream"
],
"Resource": [
"arn:aws:bedrock:*::foundation-model/*",
"arn:aws:bedrock:*:*:inference-profile/*"
]
}
]
}

Converse is authorized by bedrock:InvokeModel and ConverseStream by bedrock:InvokeModelWithResponseStream; there is no separate Converse action to grant. Embeddings and image generation go through InvokeModel, so they need only bedrock:InvokeModel.

Cross-region profile IDs such as us.anthropic.claude-sonnet-4-5-20250929-v1:0 are inference-profile ARNs rather than foundation-model ARNs, so a policy limited to foundation-model/* refuses them with AccessDeniedException even when model access is granted.

Credentials resolve through the standard AWS SDK chain:

  • environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN for temporary credentials)
  • the shared config and credentials files under ~/.aws, selected with AWS_PROFILE
  • IAM roles attached to EC2, ECS, or Lambda, supplied by the platform at runtime
  • SSO profiles, after aws sso login

Anything the chain does not cover goes through session=, which takes a pre-configured boto3.session.Session.

Listing models in models= is optional; it only adds them to the Dev UI. Any routable ID resolves on demand, including inference profiles and ARNs. Prefixes such as us., eu., or global. mark cross-region inference profiles. Several newer models are callable only through a profile, so the prefixed form is the normal one.

If a model is not offered in the region the call went to, the call fails with a ValidationException reading “The provided model identifier is invalid”. Check the region before doubting the ID.

response = await ai.generate(
prompt='Write a haiku about coding.',
)
print(response.text)

The examples on this page use await, so they assume an async context. Run them inside a flow or an async def main() driven by asyncio.run().

from pydantic import BaseModel
class Dish(BaseModel):
name: str
cuisine: str
allergens: list[str]
response = await ai.generate(
prompt='Suggest one dish for a summer menu.',
output_schema=Dish,
)
print(response.output)
# => name='Chilled Cucumber Soup' cuisine='Mediterranean' allergens=['dairy']

Bedrock models declare no constrained-decoding support, so Genkit appends the JSON schema to the prompt as format instructions and parses the reply into Dish. Pass output_instructions=False to drop the appended text, or a string to replace it.

from pydantic import BaseModel, Field
class CityInput(BaseModel):
city: str = Field(description='City to look up')
@ai.tool()
async def current_weather(city_input: CityInput) -> str:
"""Return the current weather for a city."""
return f'The weather in {city_input.city} is 31C and humid.'
response = await ai.generate(
prompt='What is the weather in San Francisco? Use the tool, then answer in one sentence.',
tools=[current_weather],
tool_choice='auto',
)
print(response.text)
# => It is 31C and humid in San Francisco right now.
from genkit import FlowContext
@ai.flow()
async def haiku_stream(topic: str, ctx: FlowContext[str]) -> str:
stream_response = ai.generate_stream(
prompt=f'Write a haiku about {topic}.',
)
async for chunk in stream_response.stream:
ctx.send_chunk(chunk.text)
return (await stream_response.response).text
flow_response = haiku_stream.stream('summer')
async for text in flow_response.stream:
print(text, end='', flush=True)
haiku = await flow_response.response

.stream yields each value the flow passes to ctx.send_chunk(), and .response resolves to the flow’s return value. When you don’t need chunks, call await haiku_stream('summer').

import base64
from pathlib import Path
from genkit import Part
image_data_url = 'data:image/png;base64,' + base64.b64encode(Path('photo.png').read_bytes()).decode('ascii')
response = await ai.generate(
prompt=[
Part.from_media(url=image_data_url, content_type='image/png'),
Part.from_text('What does this image look like? Answer in one sentence.'),
],
)
print(response.text)

Media travels as data URLs; remote http(s) URLs are refused rather than fetched. A media part whose MIME type is a document type (PDF, DOCX, CSV, and so on) becomes a Converse document block through the same code path. Bedrock parses the document server-side and requires accompanying text in the message.

Pass per-request options with BedrockConfig, which rejects unknown top-level keys:

from genkit_amazon_bedrock import BedrockConfig
response = await ai.generate(
prompt='Explain generative AI in one paragraph.',
config=BedrockConfig(max_output_tokens=512, temperature=0.7),
)

top_k and version are accepted on BedrockConfig for cross-provider portability and dropped before calling Converse, which has no first-class fields for them.

Model-specific Converse parameters—such as Claude’s extended thinking or a model’s native top_k—go inside BedrockConfig(extra={...}), which Genkit forwards as Converse’s additionalModelRequestFields:

from genkit_amazon_bedrock import Bedrock, BedrockConfig
response = await ai.generate(
model=Bedrock.text('us.anthropic.claude-sonnet-4-5-20250929-v1:0'),
prompt='What is 17 * 23? Think it through, then state the answer.',
config=BedrockConfig(
max_output_tokens=4096,
extra={'thinking': {'type': 'enabled', 'budget_tokens': 1024}},
),
)

budget_tokens must be at least 1024 and strictly less than max_output_tokens.

Bedrock can cache a prompt prefix and reuse it across calls. cache_point_part() marks where the cacheable prefix ends, so place it immediately after the content it should cache:

from pathlib import Path
from genkit import Part
from genkit_amazon_bedrock import cache_point_part
# 1. A large static prompt, reused verbatim on every call
MENU_POLICY = Path('menu_policy.md').read_text()
system = [Part.from_text(MENU_POLICY), cache_point_part()]
# 2. The second call reads the prefix the first one wrote
await ai.generate(system=system, prompt='Which dishes contain tree nuts?')
second = await ai.generate(system=system, prompt='Which dishes are vegan?')
print(second.usage.cached_content_tokens)
# => 2048.0

Cache reads surface on usage.cached_content_tokens. Bedrock counts cached tokens outside inputTokens, so usage.input_tokens is only the uncached remainder and stays small on a hit. A prefix below the model’s minimum cacheable size (roughly 1,000 tokens for Claude Sonnet, higher for smaller models) is not cached, and no error says so.

The plugin covers two more surfaces, each described in the plugin README:

  • Embedders: List embedding model IDs in Bedrock(embedders=['amazon.titan-embed-text-v2:0']) (or resolve on demand with Bedrock.embedder(...)) and call ai.embed(embedder=..., content=...). Amazon Titan, Cohere, and Amazon Nova embedding models are supported. Per-request embedder options are ignored.
  • Image generation: Call Stability image models with Bedrock.stability('stability.sd3-5-large-v1:0') and StabilityImageConfig(aspect_ratio='16:9', seed=42, output_format='jpeg'). Stability text-to-image models on Bedrock are served in us-west-2 only.