Bedrock

Introduction

Bedrock is a fully managed service provided by Amazon Web Services (AWS) that makes foundation models from various LLM providers accessible via an API.

LocalStack allows you to use the Bedrock APIs to test and develop AI-powered applications in your local environment.

The supported APIs are available on the API coverage section for Bedrock and Bedrock Runtime, which provides information on the extent of Bedrock’s integration with LocalStack.

Getting started

This guide is designed for users new to AWS Bedrock and assumes basic knowledge of the AWS CLI and our lstk aws command.

Start your LocalStack container using your preferred method with or without pre-warming the Bedrock engine. We will demonstrate how to use Bedrock by following these steps:

Listing available foundation models
Invoking a model for inference
Using the conversation API
Using batch processing

Pre-warming the Bedrock engine

The startup of the Bedrock engine can take some time. Per default, we only start it once you send a request to one of the bedrock-runtime APIs. However, if you want to start the engine when localstack starts to avoid long wait times on your first request you can set the flag BEDROCK_PREWARM.

On startup, the DEFAULT_BEDROCK_MODEL is pulled from the Ollama library and loaded into memory. However, you can define an additional list of models in BEDROCK_PULL_MODELS to pull additional models when the Bedrock engine starts up. This way you avoid long wait times when switching between models on demand with requests.

List available foundation models

You can view all available foundation models using the ListFoundationModels API. This will show you which models are available on AWS Bedrock.

Run the following command:

lstk aws bedrock list-foundation-models

Invoke a model

You can use the InvokeModel API to send requests to a specific model. In this example, we selected the Llama 3 model to process a simple prompt. However, the actual model will be defined by the DEFAULT_BEDROCK_MODEL environment variable.

Run the following command:

lstk aws bedrock-runtime invoke-model \
    --model-id "meta.llama3-8b-instruct-v1:0" \
    --body '{
        "prompt": "<|begin_of_text|><|start_header_id|>user<|end_header_id|>\nSay Hello!\n<|eot_id|>\n<|start_header_id|>assistant<|end_header_id|>",
        "max_gen_len": 2,
        "temperature": 0.9
    }' --cli-binary-format raw-in-base64-out outfile.txt

The output will be available in the outfile.txt.

Use the conversation API

Bedrock provides a higher-level conversation API that makes it easier to maintain context in a chat-like interaction using the Converse API. You can specify both system prompts and user messages.

Run the following command:

lstk aws bedrock-runtime converse \
    --model-id "meta.llama3-8b-instruct-v1:0" \
    --messages '[{
        "role": "user",
        "content": [{
            "text": "Say Hello!"
        }]
    }]' \
    --system '[{
        "text": "You'\''re a chatbot that can only say '\''Hello!'\''"
    }]'

Model Invocation Batch Processing

Bedrock offers the feature to handle large batches of model invocation requests defined in S3 buckets using the CreateModelInvocationJob API.

First, you need to create a JSONL file named batch_input.jsonl that contains all your prompts:

{"prompt": "Tell me a quick fact about Vienna.", "max_tokens": 50, "temperature": 0.5}
{"prompt": "Tell me a quick fact about Zurich.", "max_tokens": 50, "temperature": 0.5}
{"prompt": "Tell me a quick fact about Las Vegas.", "max_tokens": 50, "temperature": 0.5}

Then, you need to define buckets for the input as well as the output and upload the file in the input bucket:

lstk aws s3 mb s3://in-bucket
lstk aws s3 cp batch_input.jsonl s3://in-bucket
lstk aws s3 mb s3://out-bucket

Afterwards you can run the invocation job like this:

lstk aws bedrock create-model-invocation-job \
  --job-name "my-batch-job" \
  --model-id "mistral.mistral-small-2402-v1:0" \
  --role-arn "arn:aws:iam::123456789012:role/MyBatchInferenceRole" \
  --input-data-config '{"s3InputDataConfig": {"s3Uri": "s3://in-bucket"}}' \
  --output-data-config '{"s3OutputDataConfig": {"s3Uri": "s3://out-bucket"}}'

{
    "jobArn": "arn:aws:bedrock:us-east-1:000000000000:model-invocation-job/12345678"
}

The results will be at the S3 URL s3://out-bucket/12345678/batch_input.jsonl.out

Available models

LocalStack’s Bedrock emulation supports models from the Ollama Models library.

To use a model, retrieve its ID from Ollama and set DEFAULT_BEDROCK_MODEL to that ID. LocalStack will pull the model from Ollama and use it for emulation.

For example, to use the Mistral model, set the environment variable while starting LocalStack:

LOCALSTACK_DEFAULT_BEDROCK_MODEL=mistral lstk start

You can also define models directly in the request, by setting the model-id parameter to ollama.<ollama-model-id>. For example, if you want to access deepseek-r1, you can do it like this:

lstk aws bedrock-runtime converse \
    --model-id "ollama.deepseek-r1" \
    --messages '[{
        "role": "user",
        "content": [{
            "text": "Say Hello!"
        }]
    }]'

Troubleshooting

Users of Docker Desktop on macOS or Windows might run into the issue of Bedrock becoming unresponsive after some usage. A common reason for that is insufficient storage or memory space in the Docker Desktop VM. To resolve this issue you can increase those amounts directly in Docker Desktop or clean up unused artifacts with the Docker CLI like this

docker system prune

You could also try to use a model with lower requirements. To achieve that you can search for models in the Ollama Models library with a low parameter count or smaller size.

Limitations

At this point, we have only tested text-based models in LocalStack. Other models available with Ollama might also work, but are not officially supported by the Bedrock implementation.
Currently, GPU models are not supported by the LocalStack Bedrock implementation.

API Coverage

6 of 108 operations implemented

Available from the Ultimate plan. Licensing details

Find an API

Search the full operation list, then sort the table to compare current support.

Loading operations…

Search by operation name

		Verified on Kubernetes
Loading operations…

Complete static API list All 108 operations and their current support status

Operation	Status
BatchDeleteAdvancedPromptOptimizationJob	Not implemented
BatchDeleteEvaluationJob	Not implemented
CancelAutomatedReasoningPolicyBuildWorkflow	Not implemented
CreateAdvancedPromptOptimizationJob	Not implemented
CreateAutomatedReasoningPolicy	Not implemented
CreateAutomatedReasoningPolicyTestCase	Not implemented
CreateAutomatedReasoningPolicyVersion	Not implemented
CreateCustomModel	Not implemented
CreateCustomModelDeployment	Not implemented
CreateEvaluationJob	Not implemented
CreateFoundationModelAgreement	Not implemented
CreateGuardrail	Not implemented
CreateGuardrailVersion	Not implemented
CreateInferenceProfile	Not implemented
CreateMarketplaceModelEndpoint	Not implemented
CreateModelCopyJob	Not implemented
CreateModelCustomizationJob	Not implemented
CreateModelImportJob	Not implemented
CreateModelInvocationJob	Implemented
CreatePromptRouter	Not implemented
CreateProvisionedModelThroughput	Not implemented
DeleteAutomatedReasoningPolicy	Not implemented
DeleteAutomatedReasoningPolicyBuildWorkflow	Not implemented
DeleteAutomatedReasoningPolicyTestCase	Not implemented
DeleteCustomModel	Not implemented
DeleteCustomModelDeployment	Not implemented
DeleteEnforcedGuardrailConfiguration	Not implemented
DeleteFoundationModelAgreement	Not implemented
DeleteGuardrail	Not implemented
DeleteImportedModel	Not implemented
DeleteInferenceProfile	Not implemented
DeleteMarketplaceModelEndpoint	Not implemented
DeleteModelInvocationLoggingConfiguration	Not implemented
DeletePromptRouter	Not implemented
DeleteProvisionedModelThroughput	Not implemented
DeleteResourcePolicy	Not implemented
DeregisterMarketplaceModelEndpoint	Not implemented
ExportAutomatedReasoningPolicyVersion	Not implemented
GetAccountDataRetention	Not implemented
GetAdvancedPromptOptimizationJob	Not implemented
GetAutomatedReasoningPolicy	Not implemented
GetAutomatedReasoningPolicyAnnotations	Not implemented
GetAutomatedReasoningPolicyBuildWorkflow	Not implemented
GetAutomatedReasoningPolicyBuildWorkflowResultAssets	Not implemented
GetAutomatedReasoningPolicyNextScenario	Not implemented
GetAutomatedReasoningPolicyTestCase	Not implemented
GetAutomatedReasoningPolicyTestResult	Not implemented
GetCustomModel	Not implemented
GetCustomModelDeployment	Not implemented
GetEvaluationJob	Not implemented
GetFoundationModel	Implemented
GetFoundationModelAvailability	Not implemented
GetGuardrail	Not implemented
GetImportedModel	Not implemented
GetInferenceProfile	Not implemented
GetMarketplaceModelEndpoint	Not implemented
GetModelCopyJob	Not implemented
GetModelCustomizationJob	Not implemented
GetModelImportJob	Not implemented
GetModelInvocationJob	Implemented
GetModelInvocationLoggingConfiguration	Not implemented
GetPromptRouter	Not implemented
GetProvisionedModelThroughput	Not implemented
GetResourcePolicy	Not implemented
GetUseCaseForModelAccess	Not implemented
ListAdvancedPromptOptimizationJobs	Not implemented
ListAutomatedReasoningPolicies	Not implemented
ListAutomatedReasoningPolicyBuildWorkflows	Not implemented
ListAutomatedReasoningPolicyTestCases	Not implemented
ListAutomatedReasoningPolicyTestResults	Not implemented
ListCustomModelDeployments	Not implemented
ListCustomModels	Not implemented
ListEnforcedGuardrailsConfiguration	Not implemented
ListEvaluationJobs	Not implemented
ListFoundationModelAgreementOffers	Not implemented
ListFoundationModels	Implemented
ListGuardrails	Not implemented
ListImportedModels	Not implemented
ListInferenceProfiles	Not implemented
ListMarketplaceModelEndpoints	Not implemented
ListModelCopyJobs	Not implemented
ListModelCustomizationJobs	Not implemented
ListModelImportJobs	Not implemented
ListModelInvocationJobs	Implemented
ListPromptRouters	Not implemented
ListProvisionedModelThroughputs	Not implemented
ListTagsForResource	Not implemented
PutAccountDataRetention	Not implemented
PutEnforcedGuardrailConfiguration	Not implemented
PutModelInvocationLoggingConfiguration	Not implemented
PutResourcePolicy	Not implemented
PutUseCaseForModelAccess	Not implemented
RegisterMarketplaceModelEndpoint	Not implemented
StartAutomatedReasoningPolicyBuildWorkflow	Not implemented
StartAutomatedReasoningPolicyTestWorkflow	Not implemented
StopAdvancedPromptOptimizationJob	Not implemented
StopEvaluationJob	Not implemented
StopModelCustomizationJob	Not implemented
StopModelInvocationJob	Implemented
TagResource	Not implemented
UntagResource	Not implemented
UpdateAutomatedReasoningPolicy	Not implemented
UpdateAutomatedReasoningPolicyAnnotations	Not implemented
UpdateAutomatedReasoningPolicyTestCase	Not implemented
UpdateCustomModelDeployment	Not implemented
UpdateGuardrail	Not implemented
UpdateMarketplaceModelEndpoint	Not implemented
UpdateProvisionedModelThroughput	Not implemented

API Coverage (Bedrock Runtime)

3 of 11 operations implemented

Available from the Ultimate plan. Licensing details

Find an API

Search the full operation list, then sort the table to compare current support.

Loading operations…

Search by operation name

		Verified on Kubernetes
Loading operations…

Complete static API list All 11 operations and their current support status

Operation	Status
ApplyGuardrail	Not implemented
Converse	Implemented
ConverseStream	Not implemented
CountTokens	Not implemented
GetAsyncInvoke	Not implemented
InvokeGuardrailChecks	Not implemented
InvokeModel	Implemented
InvokeModelWithBidirectionalStream	Implemented
InvokeModelWithResponseStream	Not implemented
ListAsyncInvokes	Not implemented
StartAsyncInvoke	Not implemented

Was this page helpful?