This is the full developer documentation for LocalStack # Welcome to LocalStack Docs > Welcome to LocalStack Docs import { HeroSection } from '../../components/HeroSection'; import { ProductCards } from '../../components/ProductCards'; # Welcome to LocalStack for AWS Docs > Get started with LocalStack for AWS Docs. import { OverviewCards, HeroCards } from '../../../components/OverviewCards'; // Import SVGs from assets folder import rocketIcon from '../../../assets/images/GettingStarted_Color.svg'; import wrenchIcon from '../../../assets/images/Tooling_color.svg'; import cubeIcon from '../../../assets/images/LSAWS_Color.svg'; import starburstIcon from '../../../assets/images/Capabilities_Color.svg'; import plugIcon from '../../../assets/images/Connecting_Color.svg'; import pipelineIcon from '../../../assets/images/CIPipelines_Color.svg'; import usersIcon from '../../../assets/images/OrganizationsAdmin_Color.svg'; import bookIcon from '../../../assets/images/Tutorials_Color.svg'; ## What would you like to do today? # Changelog > This page lists new features, highlights, and bug fixes for official LocalStack releases. ## Introduction Browse release notes for official LocalStack major and minor releases since LocalStack v1.0.0. If you are looking for information about nightly releases, preview features, or experimental features, pull the [latest Docker image](https://hub.docker.com/r/localstack/localstack). Our changelog is updated with every release. Updates that affect only LocalStack Web Application or features in preview or limited release may not be reflected. ## Features under Development LocalStack uses the following terminology to communicate features under development: * **Preview** refers to a feature under development that usually evolves into becoming a stable feature. We encourage you to try out these features and report bugs, missing functionality, or general feedback via [GitHub Discussion](https://github.com/orgs/localstack/discussions/new/choose) or our [support channels](/aws/help-support/get-help/). * **Experimental** refers to a feature prototype that demonstrates initial feasibility and usually comes with many limitations. Please share your feedback to shape such features and express your interest to get them prioritized. Disclaimer: Features under development (i.e., labelled as preview/beta, experimental/alpha, or similar terms) are subject to changes (e.g., behavior, pricing tiers) or even complete removal. ## Official Releases Starting with the end-of-March 2026 release, LocalStack for AWS follows [calendar versioning](https://calver.org/) using the `YYYY.MM.patch` format. For example, `2026.03.0` is the initial March 2026 release and patch releases in the same month increment the last segment (`2026.03.1`, `2026.03.2`, and so on). Releases up to and including `v4.14.0` continue to use [Semantic Versioning](https://semver.org/). | Version | Release Date | Release Notes | |-----------|--------------------|---------------------------------------------------------------------------------------------------| | `v2026.08`| August 26, 2026 | [v2026.08](https://blog.localstack.cloud/localstack-for-aws-release-2026-08-0/) | | `v2026.07`| July 22, 2026 | [v2026.07](https://blog.localstack.cloud/localstack-for-aws-release-2026-07-0/) | | `v2026.06`| June 25, 2026 | [v2026.06](https://blog.localstack.cloud/localstack-for-aws-release-2026-06-0/) | | `v2026.05`| May 20, 2026 | [v2026.05](https://blog.localstack.cloud/localstack-for-aws-release-2026-05-0/) | | `v2026.04`| April 29, 2026 | [v2026.04](https://blog.localstack.cloud/localstack-for-aws-release-2026-04-0/) | | `v2026.03`| March 23, 2026 | [v2026.03](https://blog.localstack.cloud/localstack-for-aws-release-2026-03-0/) | | `v4.14.0` | February 26, 2026 | [v4.14.0](https://blog.localstack.cloud/localstack-for-aws-release-v-4-14-0/) | | `v4.13.0` | January 29, 2026 | [v4.13.0](https://blog.localstack.cloud/localstack-for-aws-release-v-4-13-0/) | | `v4.12.0` | December 11, 2025 | [v4.12.0](https://blog.localstack.cloud/localstack-for-aws-release-v-4-12-0/) | | `v4.11.0` | November 26, 2025 | [v4.11.0](https://blog.localstack.cloud/localstack-for-aws-release-v-4-11-0/) | | `v4.10.0` | October 30, 2025 | [v4.10.0](https://blog.localstack.cloud/localstack-for-aws-release-v-4-10-0/) | | `v4.9.0` | October 2, 2025 | [v4.9.0](https://blog.localstack.cloud/localstack-for-aws-release-v-4-9-0/) | | `v4.8.0` | September 11, 2025 | [v4.8.0](https://blog.localstack.cloud/localstack-for-aws-release-v-4-8-0/) | | `v4.7.0` | July 31, 2025 | [v4.7.0](https://blog.localstack.cloud/localstack-for-aws-release-v-4-7-0/) | | `v4.6.0` | July 3, 2025 | [v4.6.0](https://blog.localstack.cloud/localstack-for-aws-release-v-4-6-0/) | | `v4.5.0` | June 5, 2025 | [v4.5.0](https://blog.localstack.cloud/localstack-release-v-4-5-0/) | | `v4.4.0` | May 8, 2025 | [v4.4.0](https://blog.localstack.cloud/localstack-release-v-4-4-0/) | | `v4.3.0` | March 27, 2025 | [v4.3.0](https://blog.localstack.cloud/localstack-release-v-4-3-0/) | | `v4.2.0` | February 27, 2025 | [v4.2.0](https://blog.localstack.cloud/localstack-release-v-4-2-0/) | | `v4.1.0` | January 30, 2025 | [v4.1.0](https://blog.localstack.cloud/localstack-release-v-4-1-0/) | | `v4.0.0` | November 21, 2024 | [v4.0.0](https://blog.localstack.cloud/announcing-localstack-40-general-availability/) | | `v3.8.0` | October 3, 2024 | [v3.8.0](https://blog.localstack.cloud/localstack-release-v-3-8-0/) | | `v3.7.0` | August 29, 2024 | [v3.7.0](https://blog.localstack.cloud/2024-08-29-localstack-release-v-3-7-0/) | | `v3.6.0` | July 25, 2024 | [v3.6.0](https://discuss.localstack.cloud/t/localstack-release-v3-6-0/997) | | `v3.5.0` | June 13, 2024 | [v3.5.0](https://discuss.localstack.cloud/t/localstack-release-v3-5-0/947) | | `v3.4.0` | April 25, 2024 | [v3.4.0](https://discuss.localstack.cloud/t/localstack-release-v3-4-0/871) | | `v3.3.0` | March 28, 2024 | [v3.3.0](https://discuss.localstack.cloud/t/localstack-release-v3-3-0/828) | | `v3.2.0` | February 29, 2024 | [v3.2.0](https://discuss.localstack.cloud/t/localstack-release-v3-2-0/782/) | | `v3.1.0` | January 25, 2024 | [v3.1.0](https://discuss.localstack.cloud/t/localstack-release-v3-1-0/713/) | | `v3.0.0` | November 16, 2023 | [v3.0.0](https://blog.localstack.cloud/2023-11-16-announcing-localstack-30-general-availability/) | | `v2.3.0` | September 29, 2023 | [v2.3.0](https://discuss.localstack.cloud/t/localstack-release-v2-3-0/533) | | `v2.2.0` | July 20, 2023 | [v2.2.0](https://discuss.localstack.cloud/t/localstack-release-v2-2-0/424) | | `v2.1.0` | May 25, 2023 | [v2.1.0](https://discuss.localstack.cloud/t/localstack-release-v2-1-0/357) | | `v2.0.0` | March 30, 2023 | [v2.0.0](https://github.com/localstack/localstack/releases/tag/v2.0.0) | | `v1.4.0` | February 13, 2023 | [v1.4.0](https://github.com/localstack/localstack/releases/tag/v1.4.0) | | `v1.3.0` | December 1, 2022 | [v1.3.0](https://discuss.localstack.cloud/t/localstack-release-v1-3-0/170) | | `v1.2.0` | October 8, 2022 | [v1.2.0](https://discuss.localstack.cloud/t/localstack-release-v1-2-0/109) | | `v1.1.0` | September 3, 2022 | [v1.1.0](https://discuss.localstack.cloud/t/localstack-release-v1-1-0/89) | | `v1.0.0` | July 13, 2022 | [v1.0.0](https://blog.localstack.cloud/2022-07-13-announcing-localstack-v1-general-availability/) | # Overview > Use LocalStack in your CI environment to run tests against your AWS infrastructure in a high-fidelity cloud emulator. import SectionCards from '../../../../components/SectionCards.astro'; Running integration tests against real AWS in CI means maintaining cloud accounts, waiting on slow provisioning, and sharing a staging environment with every other pipeline. LocalStack replaces all of that with an emulator that runs inside the CI job itself. Deploy with the same Infrastructure as Code you already use in production, run your test suite against emulated AWS APIs, and discard the environment when the job ends. Every run gets a fresh emulator instance, and no test ever requires a real AWS account. ## CI workflow overview The following diagram illustrates a typical CI workflow, using LocalStack instead of the AWS cloud: ![An example CI/CD workflow using LocalStack](/images/aws/localstack-in-ci.svg) A CI build is triggered when you push source code to your version control repository (such as GitHub). The CI runner checks out the source code, then runs a sequence of build and test steps. Where those tests depend on AWS services, they run against the LocalStack emulator rather than the real AWS cloud. You create the resources they need with standard tools such as Terraform, or load them from a [Cloud Pod](/aws/developer-tools/snapshots/cloud-pods/) to avoid redeploying your infrastructure on every run. If the tests pass, your CD pipeline takes over and deploys the application to real AWS infrastructure. A deployment therefore only ever starts from a build that has already been validated against emulated AWS, which increases your confidence in the code you ship. ## CI integrations The steps required to run LocalStack in CI are largely the same, no matter which CI provider you use (such as GitHub Actions, GitLab CI, or CircleCI): 1. Expose a CI Auth Token 2. Install the `lstk` CLI and related tools 3. Start the emulator 4. Seed the emulator with the resources you need 5. Run your tests 6. Collect the output logs Start with the [CI Best Practices](/aws/ci-pipelines/best-practices/) page for the commands behind each step, then select your CI provider below for its own syntax and features. # CI Best Practices > Commands and general practices for running LocalStack in any CI system, from authentication and tool installation to seeding state and collecting logs. import { Tabs, TabItem } from '@astrojs/starlight/components'; Every CI system has its own configuration syntax, runner model, and feature set. Consult your CI provider's own documentation for how to declare jobs, secrets, caches, and artifacts. This guide covers the parts that are the same everywhere, such as the LocalStack-specific commands you run, and the best practices for running a LocalStack job. Whatever the provider, a CI job follows the same steps: 1. Expose your CI Auth Token to the job as `LOCALSTACK_AUTH_TOKEN`. 2. Install `lstk` and any tools your tests need, such as the AWS CLI or Terraform. 3. Configure and start the emulator with `lstk start`. 4. Deploy your infrastructure, using an Infrastructure as Code tool. 5. Alternatively, seed state from a snapshot with `lstk load`. 6. Run your tests. 7. Collect the emulator logs as a build artifact. ## Set your Auth Token Every LocalStack CI run needs a [CI Auth Token](https://app.localstack.cloud/workspace/auth-tokens), rather than a personal Developer Auth Token. Store the token in your CI system as `LOCALSTACK_AUTH_TOKEN`. Every CI provider offers somewhere to keep sensitive values, and most distinguish secrets from plain environment variables. Secrets are masked in job logs and withheld from forked-repository builds. Never commit a token to your repository or paste it into a pipeline definition. The `lstk` CLI tool automatically passes the `LOCALSTACK_AUTH_TOKEN` value into the emulator container when it starts. There is no need to invoke `lstk login`, which is only useful in an interactive session. ## Install the tools Your job needs the `lstk` CLI, plus whichever AWS tooling your tests use. Many hosted runners already ship Docker, the AWS CLI, and Terraform, so check your runner image before adding an install step. ### `lstk` [`lstk`](/aws/developer-tools/running-localstack/lstk/) is the recommended way to run and manage LocalStack. It is a single binary, so installing it in CI is quick with tools such as `npm` or `brew`. ```bash npm install -g @localstack/lstk ``` ```bash brew install localstack/tap/lstk ``` See the [`lstk` installation guide](/aws/developer-tools/running-localstack/lstk/#installation) for all installation methods. `lstk` also needs a working Docker daemon on the runner, with access to a Docker socket so the emulator can spawn its own containers for services such as Lambda and ECS. ### AWS CLI `lstk aws` proxies your host `aws` binary with the LocalStack endpoint, credentials, and region already configured, so the AWS CLI must be installed separately (if not already installed in your CI system). Refer to the [AWS CLI installation instructions](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) for details, and to the [AWS CLI guide](/aws/connecting/aws-cli/) for using it against LocalStack. ### Terraform `lstk terraform` drives the real `terraform` binary, so Terraform itself must be on the job's `PATH`. Install it with your provider's setup step where one exists (for example `hashicorp/setup-terraform` on GitHub Actions), or install it directly. Refer to the [Terraform installation instructions](https://developer.hashicorp.com/terraform/install) for details, and to the [Terraform guide](/aws/connecting/infrastructure-as-code/terraform/) for using it against LocalStack. ## Configure the emulator The `lstk` CLI tool uses a `config.toml` file to discover the required configuration parameters when starting the emulator. Commit a `.lstk/config.toml` to your repository, and both your developers and your CI jobs get the same emulator configuration, with no environment variables to duplicate across pipeline files. `lstk` picks up `./.lstk/config.toml` automatically when it is run from the root of your source tree: ```toml # .lstk/config.toml [[containers]] type = "aws" # Emulator type: "aws", "snowflake", or "azure" tag = "2026.4" # Pin the image tag for reproducible builds port = "4566" env = ["ci"] # Apply the [env.ci] profile below [env.ci] DEBUG = "1" ``` See the [configuration reference](/aws/developer-tools/running-localstack/lstk/configuration/) for every available field. Keep a single `.lstk/config.toml` for local development and CI where you can. However, if a CI job needs different settings, pass an alternative file with `lstk --config ./ci/lstk.toml start`. ## Start the emulator Start LocalStack with a single command: ```bash lstk start ``` `lstk start` brings the LocalStack emulator all the way to a ready state. It pulls the container image if needed, validates your license, starts the container, and returns only once the emulator is ready, so there is no need for a separate wait or health-check step. If startup fails, the command exits with a non-zero return code, causing your CI job to fail. For machine-readable output, add the global `--json` flag to any command. See [structured output](/aws/developer-tools/running-localstack/lstk/automation/#structured-output) and [exit codes](/aws/developer-tools/running-localstack/lstk/automation/#exit-codes) if your pipeline needs to inspect results programmatically. ## Seed state from Infrastructure as Code Most CI pipelines create the resources their tests need by applying the same Infrastructure as Code they use for production. The `lstk` proxies automatically point those tools at the emulator, without any explicit configuration. For example, with Terraform, run your usual commands through `lstk terraform` (or its `lstk tf` alias): ```bash lstk terraform init lstk terraform apply -auto-approve ``` The [`lstk cdk`](/aws/connecting/infrastructure-as-code/aws-cdk/) and [`lstk sam`](/aws/connecting/infrastructure-as-code/aws-sam/) proxies work the same way, and [other IaC tools](/aws/connecting/infrastructure-as-code/) can target the emulator through its endpoint directly. ## Seed state from a snapshot Rather than deploying your whole infrastructure on every run, you can seed the emulator from a [snapshot](/aws/developer-tools/snapshots/) captured earlier, either from a Cloud Pod or from a local snapshot file: ```bash # Load a Cloud Pod (requires LOCALSTACK_AUTH_TOKEN) lstk load pod:my-baseline # Load a snapshot file produced by an earlier job lstk load ./baseline.snapshot ``` `lstk load` starts the emulator first, if it is not already running, so it can replace a separate `lstk start` step. Alternatively, name the snapshot in your config, and `lstk start` loads it for you on every fresh start: ```toml [[containers]] type = "aws" port = "4566" snapshot = "pod:my-baseline" ``` Override the configured snapshot for a single run with `lstk start --snapshot pod:other-baseline`, or skip auto-loading entirely with `lstk start --no-snapshot`. To produce the snapshot in the first place, see [Cloud Pods](/aws/developer-tools/snapshots/cloud-pods/) and [saving snapshots locally](/aws/developer-tools/snapshots/saving-snapshots-locally/). ## Run your tests Once the emulator is running, point your tooling at it. For Infrastructure as Code tools (such as Terraform), use the `lstk` proxy version of the tool, such as `lstk terraform`. For test suites and SDK-based code, either set the endpoint and test credentials in the job's environment: ```bash export AWS_ENDPOINT_URL=http://localhost.localstack.cloud:4566 export AWS_ACCESS_KEY_ID=test export AWS_SECRET_ACCESS_KEY=test ``` Or create a `localstack` AWS profile and select it: ```bash lstk setup aws export AWS_PROFILE=localstack ``` See [connecting to LocalStack](/aws/connecting/) for the full set of options. ## Collect logs The emulator container disappears when the job ends, so consider exporting the logs before the test terminates, then store them as a build artifact: ```bash lstk logs --verbose > localstack.log ``` Run this step even when the tests fail, so you capture the logs regardless of success or failure. To make failures easier to diagnose in the first place, set `DEBUG = "1"` in your CI environment profile. See [logging](/aws/customization/logging/) for the available log levels. # BitBucket > Use LocalStack in BitBucket Pipelines. ## Introduction [BitBucket Pipeline](https://bitbucket.org/product/features/pipelines) is a CI/CD tool that allows you to build, test, and deploy your code directly from BitBucket. This guide will show you how to use LocalStack in BitBucket Pipelines. BitBucket runs your build and the Docker daemon in separate containers, and does not support mounting volumes. This guide therefore starts the LocalStack container directly with `docker run`, so the pipeline controls the port mappings and the Docker connection itself, and then uses the [`lstk`](/aws/developer-tools/running-localstack/lstk/) tool proxies to interact with it. On CI systems without those constraints, `lstk` can manage the container lifecycle as well; see [CI Best Practices](/aws/ci-pipelines/best-practices/). ## Setting up the BitBucket Pipeline When you want to integrate LocalStack into your job configuration, you just have to execute the following steps: - Specify the Docker Socket to allow the LocalStack container to access the Docker daemon. - Pass your CI Auth Token to the container, which is required to start the emulator. - Export the `LSTK_ENDPOINT_URL` environment variable to point `lstk` at the LocalStack endpoint. - Install the AWS CLI and `lstk` to interact with LocalStack's emulated services. - Start the LocalStack container in detached mode by specifying the Docker Socket and Docker Host. - Wait for the emulator to become ready before using it. The following example BitBucket Pipeline configuration (`bitbucket-pipelines.yaml`) executes these steps, creates a new S3 bucket, and queries the list of S3 buckets: ```yaml showshowLineNumbers image: node:22 definitions: services: docker: memory: 2048 pipelines: default: - step: name: Test Localstack services: - docker script: - export DOCKER_SOCK=$DOCKER_HOST - export LSTK_ENDPOINT_URL="http://localhost.localstack.cloud:4566" - echo "${BITBUCKET_DOCKER_HOST_INTERNAL} localhost.localstack.cloud " >> /etc/hosts - apt-get update && apt-get install -y awscli - npm install -g @localstack/lstk - docker run -d --rm -p 4566:4566 -p 4510-4559:4510-4559 -e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} -e DEBUG=1 -e DOCKER_SOCK=tcp://${BITBUCKET_DOCKER_HOST_INTERNAL}:2375 -e DOCKER_HOST=tcp://${BITBUCKET_DOCKER_HOST_INTERNAL}:2375 --name localstack-aws localstack/localstack-pro - | for _ in $(seq 1 60); do curl -sf "${LSTK_ENDPOINT_URL}/_localstack/health" > /dev/null && break sleep 2 done - lstk aws s3 mb s3://test-bucket - lstk aws s3 ls ``` ## Configuring a CI Auth Token For the configuration above to work, add your CI Auth Token to the project's environment variables. The LocalStack container will automatically pick it up and activate your LocalStack license. Go to the [CI Auth Token page](https://app.localstack.cloud/workspace/auth-tokens) and copy your CI Auth Token. To add a CI Auth Token to your BitBucket Pipeline: - Select a workspace from the BitBucket dashboard. - Select the **Settings** on the top navigation bar. - Select **Workspace settings** from the **Settings dropdown** menu. - On the left-hand menu, navigate to **Pipelines** and click on **Workspace variables**. - Add a new variable with the name `LOCALSTACK_AUTH_TOKEN` and the value of your CI Auth Token, and mark it as **Secured**. ## Current Limitations ### Mounting Volumes BitBucket Pipelines does not support mounting volumes, so you cannot mount a volume to the LocalStack container. This limitation prevents you from mounting the Docker Socket to the LocalStack container, which is required to create compute resources, such as Lambda functions or ECS tasks. # CircleCI > Use LocalStack in CircleCI. ## Introduction [CircleCI](https://circleci.com) is a continuous integration and continuous delivery (CI/CD) platform which uses a configuration file (usually named `.circleci/config.yml`) to define the build, test, and deployment workflows. This guide shows how to run LocalStack in CircleCI using the [`lstk` CLI](/aws/developer-tools/running-localstack/lstk/). ## Snippets ### Start up LocalStack ```yaml showshowLineNumbers version: '2.1' jobs: localstack-test: machine: image: ubuntu-2204:current steps: - checkout # LOCALSTACK_AUTH_TOKEN comes from the project's environment variables - run: name: Install lstk command: npm install -g @localstack/lstk - run: name: Configure the AWS profile command: lstk setup aws - run: name: Start LocalStack command: lstk start - run: name: Test LocalStack command: | lstk aws s3 mb s3://test-bucket lstk aws s3 ls workflows: localstack-test: jobs: - localstack-test ``` `lstk start` pulls the image, validates your license, and returns only once the emulator is ready, so no separate wait step is needed. `lstk aws` proxies the `aws` binary with LocalStack's endpoint and credentials applied, so the AWS CLI must be available on the runner; add an install step if your image does not provide it. `lstk setup aws` writes a `localstack` AWS profile for that binary to use. It is optional, but without it `lstk` notes on every call that no profile was found. ### Configuration To configure LocalStack use the `environment` key on the job level or a shell command, where the latter takes higher precedence. `lstk start` forwards host environment variables prefixed with `LOCALSTACK_` into the container, which strips the prefix, so set `LOCALSTACK_DEBUG` to control the container's `DEBUG` option. Read more about the [configuration options](/aws/customization/configuration-options) of LocalStack. #### Job level ```yaml showshowLineNumbers ... jobs: localstack-test: machine: image: ubuntu-2204:current environment: LOCALSTACK_DEBUG: "1" LOCALSTACK_LS_LOG: "trace" steps: ... - run: lstk start ... ``` #### Shell command ```yaml showshowLineNumbers ... jobs: localstack-test: machine: image: ubuntu-2204:current steps: - run: name: Configure LocalStack command: | echo 'export LOCALSTACK_DEBUG=1' >> "$BASH_ENV" echo 'export LOCALSTACK_LS_LOG=trace' >> "$BASH_ENV" ... ``` ### Configuring a CI Auth Token To enable LocalStack for AWS, you need to add your LocalStack CI Auth Token to the project's environment variables. `lstk` will automatically pick it up and activate the licensed features. Go to the [CI Auth Token page](https://app.localstack.cloud/workspace/auth-tokens) and copy your CI Auth Token. To add the CI Auth Token to your CircleCI project, follow these steps: - Click on **Project Settings**. - Select **Environment Variables** from the left side menu. - Click **Add Environment Variable**. - Name your environment variable `LOCALSTACK_AUTH_TOKEN`. - Paste your CI Auth Token into the input field. After adding the variable, CircleCI injects `LOCALSTACK_AUTH_TOKEN` into your job environment. ### Dump LocalStack logs ```yaml showshowLineNumbers ... jobs: localstack-test: machine: image: ubuntu-2204:current steps: ... - run: name: Dump LocalStack logs when: always command: lstk logs --verbose | tee localstack.log - store_artifacts: path: localstack.log name: localstack-logs ... ``` ### Store LocalStack state You can preserve your AWS infrastructure with LocalStack in various ways. To be able to use any of the below samples, you must [set a valid CI Auth Token](#configuring-a-ci-auth-token). _Note: For best result we recommend to use a combination of the below techniques, and you should familiarize yourself with CircleCI's data persistence approach, see their [official documentation](https://circleci.com/docs/persist-data/)._ #### Cloud Pods Cloud Pods providing an easy solution to persist LocalStack's state, even between workflows or projects. Find more information about [Cloud Pods](/aws/developer-tools/snapshots/cloud-pods). ##### Multiple projects Update or create the Cloud Pod in it's own project (ie in a separate Infrastructure as Code repo), this would create a base Cloud Pod, which you can use in the future without any configuration or deployment. _Note: If there is a previously created Cloud Pod which doesn't need updating this step can be skipped._ ```yaml showshowLineNumbers ... jobs: localstack-update-cloud-pod: machine: image: ubuntu-2204:current steps: - run: npm install -g @localstack/lstk - run: lstk start ... - run: name: Load state if exists command: lstk load pod: || true ... # Deploy infrastructure changes ... - run: name: Save Cloud Pod command: lstk save pod: workflows: localstack-build: jobs: - localstack-update-cloud-pod ``` In a separate project use the previously created base Cloud Pod as below: ```yaml showshowLineNumbers ... jobs: localstack-use-cloud-pod: machine: image: ubuntu-2204:current steps: - run: npm install -g @localstack/lstk - run: lstk start ... - run: name: Load Cloud Pod command: lstk load pod: ... # Run some tests workflows: localstack-build: jobs: - localstack-use-cloud-pod ``` ##### Same project To use a dynamically updated Cloud Pod in multiple workflows but in the same project, you must eliminate the race conditions between the update workflow and the others. Before you are able to use any stored artifacts in your pipeline, you must provide either a valid [project API token](https://circleci.com/docs/managing-api-tokens/#creating-a-project-api-token) or a [personal API token](https://circleci.com/docs/managing-api-tokens/#creating-a-personal-api-token) to CircleCI. ```yaml showshowLineNumbers ... parameters: run_workflow_build: default: true type: boolean run_workflow_test1: default: false type: boolean run_workflow_test2: default: false type: boolean ... jobs: localstack-update-state: machine: image: ubuntu-2204:current steps: - run: npm install -g @localstack/lstk - run: lstk start ... - run: name: Load Cloud Pod command: lstk load pod: || true ... # Deploy infrastructure ... - run: name: Save Cloud Pod command: lstk save pod: - run: name: Trigger other workflows # Replace placeholders with right values command: | curl --request POST \ --url https://circleci.com/api/v2/project////pipeline --header 'Circle-Token: $CIRCLECI_TOKEN' \ --header 'content-type: application/json' \ --data '{"parameters":{"run_workflow_build":false, "run_workflow_test1":true, "run_workflow_test2":true}}' localstack-use-state: machine: image: ubuntu-2204:current steps: - run: npm install -g @localstack/lstk - run: lstk start ... - run: name: Load state if exists command: lstk load pod: || true ... # Example workflows workflows: localstack-build: when: << pipeline.parameters.run_workflow_build >> jobs: - localstack-update-state localstack-test1: when: << pipeline.parameters.run_workflow_test1 >> - localstack-use-state ... localstack-test2: when: << pipeline.parameters.run_workflow_test2 >> jobs: - localstack-use-state ... ``` #### Workspace This strategy persist LocalStack's state between jobs for the current workflow. ```yaml showshowLineNumbers ... jobs: localstack-save-state: machine: image: ubuntu-2204:current steps: - run: npm install -g @localstack/lstk - run: lstk start ... # LocalStack already running and deployed infrastructure - run: name: Save a snapshot command: lstk save ./ls-state.snapshot - persist_to_workspace: paths: - ls-state.snapshot # Store state as artifact for local debugging - store_artifacts: path: ls-state.snapshot name: ls-state ... localstack-load-state: machine: image: ubuntu-2204:current steps: - run: npm install -g @localstack/lstk - run: lstk start ... # LocalStack already running - attach_workspace: at: . - run: name: Load the snapshot command: | if [ -f ls-state.snapshot ]; then lstk load ./ls-state.snapshot --merge=overwrite fi ... workflows: localstack-build: jobs: - localstack-save-state - localstack-load-state ``` More information about Localstack's [snapshots](/aws/developer-tools/snapshots/saving-snapshots-locally). #### Cache To preserve state between workflow runs, you can take leverage of CircleCI's caching too. This strategy will persist LocalStack's state for every workflow re-runs, but not for different workflows. ```yaml showshowLineNumbers ... jobs: localstack-update-state: machine: image: ubuntu-2204:current steps: - run: npm install -g @localstack/lstk - run: lstk start ... # LocalStack already running # Let's restore previous workflow run's LocalStack state - restore_cache: # Use latest "ls-state" prefixed cache key: ls-state- - run: name: Load the snapshot command: | if [ -f ls-state.snapshot ]; then lstk load ./ls-state.snapshot --merge=overwrite fi ... # Infrastructure had been updated # Let's update cached LocalStack state - run: name: Save a snapshot command: lstk save ./ls-state.snapshot - save_cache: key: ls-state-{{checksum ls-state.snapshot}} paths: ls-state.snapshot ... localstack-do-work: machine: image: ubuntu-2204:current steps: - run: npm install -g @localstack/lstk - run: lstk start # LocalStack already running - restore_cache: # Use latest "ls-state" prefixed cache key: ls-state- - run: name: Load the snapshot command: | if [ -f ls-state.snapshot ]; then lstk load ./ls-state.snapshot --merge=overwrite fi ... # Example workflows workflows: localstack-build: jobs: - localstack-update-state - localstack-do-work ... ``` More information about [snapshots](/aws/developer-tools/snapshots/saving-snapshots-locally). # CodeBuild > Use LocalStack in CodeBuild. ## Introduction [AWS CodeBuild](https://docs.aws.amazon.com/codebuild/latest/userguide/welcome.html) is a managed AWS service for the build and testing phases of software development. CodeBuild allows you to define your build project, set the source code location, and handles the building and testing, while supporting various programming languages, build tools, and runtime environments. This guide shows how to run LocalStack in CodeBuild using the [`lstk` CLI](/aws/developer-tools/running-localstack/lstk/). The CodeBuild standard images already provide Docker, Node.js, and the AWS CLI, so `lstk` is the only part that needs installing. :::note LocalStack depends on the Docker socket to emulate your infrastructure. To enable it, update your project by ticking **Environment > Additional Configuration > Privileged > Enable this flag if you want to build Docker Images or want your builds to get elevated privileges**. ::: ## Snippets ### Start up LocalStack LocalStack requires a CI Auth Token to run. Go to the [CI Auth Token page](https://app.localstack.cloud/workspace/auth-tokens) and copy your CI Auth Token, then add it to the project's environment variables: - Navigate to your project dashboard, click **Edit** to open the dropdown, and select **Environment**. - Click on **Additional configuration** and navigate to the **Environment variables** section. - Specify **Name** as `LOCALSTACK_AUTH_TOKEN` and **Value** as your CI Auth Token. Specify **Type** as per your requirement. - Click on **Update environment** to save your environment variables. `lstk` automatically recognizes the token and activates the licensed features. You can then install `lstk` and start the emulator in your buildspec file: ```yml showshowLineNumbers version: 0.2 phases: install: runtime-versions: nodejs: 22 commands: - npm install -g @localstack/lstk pre_build: commands: # LOCALSTACK_AUTH_TOKEN comes from the project's environment variables - lstk setup aws - lstk start build: commands: - lstk aws s3 mb s3://test-bucket - lstk aws s3 ls ``` `lstk start` pulls the image, validates your license, and returns only once the emulator is ready, so no separate wait step is needed. `lstk aws` proxies the runner's `aws` binary with LocalStack's endpoint and credentials applied. `lstk setup aws` writes a `localstack` AWS profile for that binary to use. ### Configuration To set LocalStack configuration options, pass them as `LOCALSTACK_`-prefixed environment variables. `lstk start` forwards those into the container which strips the prefix, so `LOCALSTACK_DEBUG` sets the container's `DEBUG` option. ```yml showshowLineNumbers version: 0.2 env: variables: LOCALSTACK_DEBUG: "1" LOCALSTACK_LS_LOG: "trace" ... phases: ... ``` Settings that apply to every run belong in an [`[env.*]` profile](/aws/developer-tools/running-localstack/lstk/configuration/) in a `.lstk/config.toml` committed to your repository, which also lets you pin the image tag. Read more about the [configuration options](/aws/customization/configuration-options) of LocalStack. ### Dump LocalStack logs ```yaml showshowLineNumbers ... phases: pre_build: commands: # Starts up LocalStack ... build: commands: # Run some commands which might fail ... post_build: commands: # Dump logs on build fail - '[ ${CODEBUILD_BUILD_SUCCEEDING:-0} -eq 0 ] && (lstk logs --verbose | tee localstack.log) || true' ... # Optionally store dumped logs as artifact artifacts: files: - localstack.log ``` ### Store LocalStack state You can preserve your AWS infrastructure with LocalStack in various ways. #### Cloud Pods Find more information about Cloud Pods [here](/aws/developer-tools/snapshots/cloud-pods). ```yml showshowLineNumbers ... phases: pre_build: commands: ... # LocalStack is up and running already # Allow the load to fail as the pod does not exist at first run - lstk load pod: || true ... - lstk save pod: ... ``` #### Artifact Instead of the LocalStack platform, you can keep the state as a local snapshot file and move it between builds with CodeBuild's own artifact storage. Find out more about [snapshots](/aws/developer-tools/snapshots/saving-snapshots-locally/). ```yml showshowLineNumbers ... phases: pre_build: commands: # LocalStack is up and running already - | if [ -f ls-state.snapshot ]; then lstk load ./ls-state.snapshot --merge=overwrite fi ... - lstk save ./ls-state.snapshot ... artifacts: files: - ls-state.snapshot ``` Alternatively save as a secondary artifact: ```yml showshowLineNumbers ... artifacts: ... secondary-artifacts: ls-state: files: - ls-state.snapshot ... ``` To use previously stored artifacts as inputs, set them as a source in the project. #### Cache ```yml showshowLineNumbers ... phases: pre_build: commands: # LocalStack is up and running already - | if [ -f ls-state.snapshot ]; then lstk load ./ls-state.snapshot --merge=overwrite fi ... - lstk save ./ls-state.snapshot ... cache: paths: - 'ls-state.snapshot' ``` ## Current Limitations - `lstk` pulls the emulator image from Docker Hub by default, where you may run into the following error: ```bash toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit ``` To resolve this, either use your Docker Hub account credentials to pull the image, or point `lstk` at LocalStack's public ECR mirror with the [`image` field](/aws/developer-tools/running-localstack/lstk/configuration/#custom-container-image) in `.lstk/config.toml`: ```toml [[containers]] type = "aws" port = "4566" image = "public.ecr.aws/localstack/localstack-pro" tag = "latest" ``` - LocalStack depends on the Docker socket to emulate your infrastructure. To enable it, update your project by ticking **Environment > Additional Configuration > Privileged > Enable this flag if you want to build Docker Images or want your builds to get elevated privileges**. For further information see the official CodeBuild [documentation](https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html). # GitHub Actions > Use LocalStack in GitHub Actions. This page contains easily customizable snippets to show you how to manage LocalStack in a GitHub Actions pipeline. The GitHub-hosted `ubuntu-latest` runner already provides Docker, Node.js, and the AWS CLI, so `lstk` is the only part that needs installing. On a self-hosted runner, add install steps for whichever of those are missing. :::caution The [`LocalStack/setup-localstack`](https://github.com/localstack/setup-localstack) action is no longer supported with `lstk`. Install and drive [`lstk`](/aws/developer-tools/running-localstack/lstk/) directly in a `run` step, as shown in the snippets below. ::: ## Snippets ### Start up Localstack To enable LocalStack for AWS, you need to add your LocalStack CI Auth Token to the project's environment variables. `lstk` will automatically pick it up and activate the licensed features. Go to the [CI Auth Token page](https://app.localstack.cloud/workspace/auth-tokens) and copy your CI Auth Token. To add the CI Auth Token to your GitHub project, follow these steps: - Navigate to your repository **Settings > Secrets** and press **New repository secret**. - Enter `LOCALSTACK_AUTH_TOKEN` as the name of the secret and paste your CI Auth Token as the value. Click **Add secret** to save your secret. You can then install `lstk` and start the emulator, passing the secret to the step: ```yaml showshowLineNumbers - name: Install lstk run: npm install -g @localstack/lstk - name: Configure the AWS profile run: lstk setup aws - name: Start LocalStack run: lstk start env: LOCALSTACK_AUTH_TOKEN: ${{ secrets.LOCALSTACK_AUTH_TOKEN }} ``` `lstk start` pulls the image, validates your license, and returns only once the emulator is ready, so no separate wait step is needed. To pin the image tag, commit a [`.lstk/config.toml`](/aws/developer-tools/running-localstack/lstk/configuration/) to your repository rather than passing it on the command line. Where several steps run `lstk`, set `LOCALSTACK_AUTH_TOKEN` once at the job level instead of repeating it on every step. `lstk setup aws` writes a `localstack` AWS profile for the runner's `aws` binary to use. It is optional, but without it `lstk` notes on every call that no profile was found. ### Configuration To set LocalStack configuration options, pass them as `LOCALSTACK_`-prefixed environment variables. `lstk start` forwards those into the container, which strips the prefix, so `LOCALSTACK_DEBUG` sets the container's `DEBUG` option. For example: ```yml showshowLineNumbers - name: Start LocalStack run: lstk start env: LOCALSTACK_AUTH_TOKEN: ${{ secrets.LOCALSTACK_AUTH_TOKEN }} LOCALSTACK_DEBUG: "1" ``` You can add extra configuration options as further `LOCALSTACK_`-prefixed variables. Settings that apply to every run belong in an [`[env.*]` profile](/aws/developer-tools/running-localstack/lstk/configuration/) in `.lstk/config.toml` instead. ### Dump Localstack logs ```yaml showshowLineNumbers - name: Show localstack logs if: always() run: | lstk logs --verbose | tee localstack.log ``` `if: always()` makes the step run even after a failing test, which is when the logs matter most. ### Store Localstack state You can preserve your AWS infrastructure with Localstack in various ways. #### Cloud Pods ```yaml showshowLineNumbers ... # Localstack is up and running already - name: Load the Cloud Pod continue-on-error: true # Allow it to fail as pod does not exist at first run run: lstk load pod: env: LOCALSTACK_AUTH_TOKEN: ${{ secrets.LOCALSTACK_AUTH_TOKEN }} ... - name: Save the Cloud Pod run: lstk save pod: env: LOCALSTACK_AUTH_TOKEN: ${{ secrets.LOCALSTACK_AUTH_TOKEN }} ... ``` Find more information about cloud pods [here](/aws/developer-tools/snapshots/cloud-pods). #### Artifact Instead of the LocalStack platform, you can keep the state as a local snapshot file and move it between runs with GitHub's own artifact storage. ```yaml showshowLineNumbers ... - name: Download the previous state continue-on-error: true # Allow it to fail as the artifact does not exist at first run uses: actions/download-artifact@v4 with: name: my-ls-state - name: Start LocalStack and load the state run: | lstk start if [ -f ls-state.snapshot ]; then lstk load ./ls-state.snapshot --merge=overwrite fi env: LOCALSTACK_AUTH_TOKEN: ${{ secrets.LOCALSTACK_AUTH_TOKEN }} ... - name: Save the state run: lstk save ./ls-state.snapshot - name: Upload the state uses: actions/upload-artifact@v4 with: name: my-ls-state path: ls-state.snapshot ... ``` More information about [snapshots](/aws/developer-tools/snapshots/saving-snapshots-locally/). ## Current Limitations ### Running Lambdas targeting the `arm64` architecture Deploying Lambdas targeting the `arm64` architecture on GitHub Actions can pose challenges. While the [`LAMBDA_IGNORE_ARCHITECTURE` configuration](https://docs.localstack.cloud/references/configuration/#lambda) is an option for cross-architecture compatible Lambdas, it may not be suitable for statically compiled Lambdas. To address this, users are recommended to leverage Docker's [`setup-qemu-action`](https://github.com/docker/setup-qemu-action) to enable emulation for the `arm64` architecture. It's important to note that using this approach may result in significantly slower build times. ### Running LocalStack on Windows runners LocalStack requires Docker to run, which is not natively supported on Windows runners. Windows runners don't support Docker natively due to licensing restrictions. It is currently not possible to run LocalStack on Windows runners. # GitLab CI > Use LocalStack in GitLab CI. This page contains easily customizable snippets to show you how to manage LocalStack in a GitLab CI pipeline with the [`lstk` CLI](/aws/developer-tools/running-localstack/lstk/). GitLab runs your job in one container and the Docker daemon in another, so every snippet below pairs the job with a Docker-in-Docker (`dind`) service. :::tip While working with a Docker-in-Docker (`dind`) setup, the Docker runner requires `privileged` mode. You must always use `privileged = true` in your GitLab CI's `config.toml` file while setting up LocalStack in GitLab CI runners. For more information, see [GitLab CI Docker-in-Docker](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-in-docker-executor) documentation. ::: ## Snippets ### Start up LocalStack LocalStack requires a [CI Auth Token](https://app.localstack.cloud/workspace/auth-tokens), which you must add to the repository's environment variables as `LOCALSTACK_AUTH_TOKEN`. Go to your project's **Settings > CI/CD** and expand the **Variables** section. Select the **Add Variable** button and fill in the necessary details with `LOCALSTACK_AUTH_TOKEN` as the key and your CI Auth Token as the value. After you create the variable, you can use it in the `.gitlab-ci.yml` file. However, variables set in the GitLab UI are not automatically passed down to service containers. You need to assign them as variables in the UI, and then re-assign them in your `.gitlab-ci.yml`. #### Container In this setup, `lstk` owns the emulator's lifecycle: `DOCKER_HOST` points it at the `dind` daemon, and `lstk start` runs the emulator container there. ```yaml showshowLineNumbers image: node:22 stages: - job job: stage: job variables: DOCKER_HOST: tcp://docker:2375 DOCKER_TLS_CERTDIR: "" LOCALSTACK_AUTH_TOKEN: $LOCALSTACK_AUTH_TOKEN LOCALSTACK_HOST: localhost.localstack.cloud:4566 services: - name: docker:dind alias: docker command: ["--tls=false"] before_script: - npm install -g @localstack/lstk - apt-get update && apt-get install -y awscli - dind_ip="$(getent hosts docker | cut -d' ' -f1)" - echo "${dind_ip} localhost.localstack.cloud" >> /etc/hosts - lstk setup aws script: - lstk start - lstk aws s3 mb s3://test-bucket - lstk aws s3 ls ``` `lstk start` pulls the image, validates your license, and returns only once the emulator is ready, so no separate wait step is needed. Because the emulator runs on the `dind` daemon, its ports are published on the `docker` service rather than on the job container. The `/etc/hosts` entry and `LOCALSTACK_HOST` are what let `lstk` and your tests reach it at `localhost.localstack.cloud:4566`; without them `lstk` falls back to `127.0.0.1`, where nothing is listening. :::note `lstk` bind-mounts the Docker socket into the emulator, and sets the emulator's own `DOCKER_HOST`, only when it reaches the daemon over a Unix socket. A TCP `dind` daemon has no socket to mount, so services that spawn their own containers (Lambda, ECS, EKS) need the daemon address passed in explicitly. `lstk start` forwards `LOCALSTACK_`-prefixed variables to the emulator, which strips the prefix, so set `LOCALSTACK_DOCKER_HOST` to the `dind` daemon as seen from inside the `dind` network (its bridge gateway, usually `tcp://172.17.0.1:2375`). ::: #### Service Alternatively, run LocalStack as a GitLab service container and use `lstk` purely as a client, pointing it at the service with `LSTK_ENDPOINT_URL`. GitLab passes the job's `variables` to service containers too, so the emulator picks up both the auth token and the Docker connection directly, with no prefixing required. ```yaml showshowLineNumbers image: node:22 stages: - job job: stage: job variables: DOCKER_SOCK: tcp://docker:2375 DOCKER_HOST: tcp://docker:2375 DOCKER_TLS_CERTDIR: "" LOCALSTACK_AUTH_TOKEN: $LOCALSTACK_AUTH_TOKEN LSTK_ENDPOINT_URL: http://localstack:4566 services: - name: localstack/localstack-pro:latest alias: localstack - name: docker:dind alias: docker command: ["--tls=false"] before_script: - npm install -g @localstack/lstk - apt-get update && apt-get install -y awscli curl - | for _ in $(seq 1 60); do curl -sf "${LSTK_ENDPOINT_URL}/_localstack/health" > /dev/null && break sleep 2 done script: - lstk aws s3 mb s3://test-bucket - lstk aws s3 ls ``` GitLab starts service containers before the job's first command, but does not wait for them to become ready, hence the health poll. ### Dump LocalStack logs ```yaml showshowLineNumbers ... job: script: - set +e - ; status=$? - lstk logs --verbose | tee localstack.log - exit $status artifacts: when: always paths: - localstack.log ... ``` Collect the logs as the last `script` step rather than in `after_script`, where the emulator container is no longer reachable. Capturing the test command's exit code keeps the job's result intact while still writing the logs after a failing test, which is when they matter most. In the [Service](#service) setup, `lstk logs` is not available, because `lstk` does not manage the service container. Set `CI_DEBUG_SERVICES: "true"` to have GitLab stream the service container's logs into the job log instead. ### Store LocalStack state You can preserve your AWS infrastructure with LocalStack in various ways. #### Artifact ```yaml showshowLineNumbers ... job: before_script: - (test -f ./ls-state.snapshot && lstk load ./ls-state.snapshot --merge=overwrite) || true script: ... - lstk save ./ls-state.snapshot ... artifacts: paths: - $CI_PROJECT_DIR/ls-state.snapshot ... ``` More info about LocalStack's snapshots [here](/aws/developer-tools/snapshots/saving-snapshots-locally/). #### Cache ```yaml showshowLineNumbers ... job: before_script: - (test -f ./ls-state.snapshot && lstk load ./ls-state.snapshot --merge=overwrite) || true script: ... - lstk save ./ls-state.snapshot ... cache: key: untracked: true files: - $CI_PROJECT_DIR/ls-state.snapshot paths: - $CI_PROJECT_DIR/ls-state.snapshot ... ``` Additional information about snapshots [here](/aws/developer-tools/snapshots/saving-snapshots-locally/). #### Cloud Pod ```yaml showshowLineNumbers ... job: before_script: - lstk load pod: || true script: ... - lstk save pod: ... ``` Find more information about Cloud Pods [here](/aws/developer-tools/snapshots/cloud-pods). ## Current Limitations - LocalStack must be able to reach a Docker socket to provision containers for certain services, such as Lambda, EKS, and ECS. - The runner must be able to resolve the LocalStack domain (by default _localhost.localstack.cloud_); see the sample pipelines for a possible solution. - To separate steps into their own jobs, you must preserve LocalStack's state, since GitLab does not preserve job-related containers or services across a pipeline. - Docker tooling is necessary to start up LocalStack in GitLab CI. - When LocalStack runs as a container, it is not accessible during the `after_script` phase. # Travis CI > Use LocalStack in Travis CI. This guide shows how to start and use LocalStack in your Travis CI jobs, managed with the [`lstk` CLI](/aws/developer-tools/running-localstack/lstk/). ## Configuring a CI Auth Token `lstk` validates your LocalStack license before it starts the emulator, so a [CI Auth Token](https://app.localstack.cloud/workspace/auth-tokens) is required rather than a personal Developer Auth Token. To configure this in Travis CI, go to the project settings (`More options` → `Settings`), scroll down to the `Environment Variables` section, and add your CI Auth Token as `LOCALSTACK_AUTH_TOKEN`. Travis CI exposes the variable to the build, and `lstk` picks it up from the environment and passes it to the emulator container. Keep `Display value in build log` switched off so the token is not printed. ## Setting up the Travis CI job When you want to integrate LocalStack into your job configuration, you just have to execute the following steps: - Install `lstk`, along with the AWS CLI that `lstk aws` proxies. - Generate the `localstack` AWS profile with `lstk setup aws`. - Use `lstk` to start LocalStack. There is no need to pull the image or wait for the container: `lstk start` pulls the image if needed and returns only once the emulator is ready. The following example Travis CI job config (`.travis.yaml`) executes these steps, creates a new S3 bucket, and prints a nice message in the end: ```yaml showshowLineNumbers language: node_js node_js: - "22" services: - docker before_install: # Install lstk - npm install -g @localstack/lstk # Install the AWS CLI, which `lstk aws` runs under the hood - curl -sSL "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o awscliv2.zip - unzip -q awscliv2.zip && sudo ./aws/install # Write the localstack AWS profile, so lstk does not warn that it's missing - lstk setup aws # Start LocalStack; LOCALSTACK_AUTH_TOKEN comes from the project's environment variables - lstk start script: # Test LocalStack by creating a new S3 bucket (and verify that it has been created by listing all buckets) - lstk aws s3 mb s3://test - lstk aws s3 ls - echo "Execute your tests here :)" ``` Travis CI images vary by language and distribution, so drop either install step if your image already provides the tool. # Overview > Connect to LocalStack using your favorite AWS tools, such as CLI, SDKs, IaC, the web console, and IDE integrations. import SectionCards from '../../../../components/SectionCards.astro'; If you already run an application on AWS, you almost certainly connect to it through familiar tools, such as the AWS CLI, an AWS SDK, an Infrastructure as Code tool such as Terraform or AWS CDK, or an IDE plugin. Running that same application against LocalStack means continuing to use those tools; you only need to point them at LocalStack instead of AWS. # AWS CLI > Use AWS Command Line Interface (CLI) to create local AWS resources with LocalStack. ## Introduction The [AWS Command Line Interface (CLI)](https://aws.amazon.com/cli/) is the standard tool from Amazon for creating and managing AWS services via a command line interface. Due to LocalStack's compatibility with the AWS APIs, this tool can also access LocalStack's emulated services. You can use the AWS CLI with LocalStack using one or more of the following approaches: - [AWS CLI](#aws-cli) - Use the standard `aws` command with hand-crafted configuration options necessary to communicate with LocalStack. - [LocalStack AWS CLI](#localstack-aws-cli-lstk-aws) - Use the `lstk aws` command to set the configuration options for you. - [Using AWS CLI from a pre-built container](#using-aws-cli-from-a-pre-built-container) - Use Amazon's pre-built AWS CLI container image instead of installing `aws` locally. :::note `lstk aws` supersedes the older [`awslocal` wrapper script](/aws/connecting/infrastructure-as-code/deprecated-wrapper-scripts#awslocal), which is deprecated but still available if you need it. ::: ## AWS CLI If you don't already have `aws` (version 2) installed, follow the [official AWS CLI installation instructions](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). Once installed, you can configure the AWS CLI to redirect AWS API requests to LocalStack using two approaches: - [Configuring an endpoint URL](#configuring-an-endpoint-url) - [Configuring a custom profile](#configuring-a-custom-profile) ### Configuring an endpoint URL You can use AWS CLI with an endpoint URL by configuring environment variables and including the `--endpoint-url=` flag in your `aws` CLI commands. For example: ```bash export AWS_ACCESS_KEY_ID="test" export AWS_SECRET_ACCESS_KEY="test" export AWS_DEFAULT_REGION="us-east-1" aws --endpoint-url=http://localhost.localstack.cloud:4566 kinesis list-streams ``` :::note Pre-signed URLs for S3 are generated with the credentials configured on the client, such as the default `test`/`test` pair shown above. For LocalStack to be able to validate a pre-signed URL, it must be generated with valid credentials. More details at [S3 signature validation](/aws/services/s3/#signature-validation). ::: ### Configuring a custom profile You can configure a custom profile to use with LocalStack. Add the following profile to your AWS configuration file (by default, this file is at `~/.aws/config`): ```bash [profile localstack] region=us-east-1 output=json endpoint_url = http://localhost.localstack.cloud:4566 ``` Add the following profile to your AWS credentials file (by default, this file is at `~/.aws/credentials`): ```bash [localstack] aws_access_key_id=test aws_secret_access_key=test ``` You can now use the `localstack` profile with the `aws` CLI: ```bash aws s3 mb s3://test --profile localstack aws s3 ls --profile localstack ``` :::tip Alternatively, you can also set the `AWS_PROFILE=localstack` environment variable, in which case the `--profile localstack` parameter can be omitted in the commands above. ::: ## LocalStack AWS CLI (`lstk aws`) `lstk aws` serves as a thin wrapper and a substitute for the standard `aws` command, enabling you to run AWS CLI commands within the LocalStack environment without specifying the `--endpoint-url` parameter or a profile. ### Installation To make use of `lstk aws`, you must install both the `lstk` CLI and the standard `aws` command from Amazon. 1. To install `lstk`, follow the [`lstk` installation instructions](/aws/developer-tools/running-localstack/lstk/#installation). 2. To install `aws`, follow the [official AWS CLI installation instructions](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). ### Usage The `lstk aws` command shares identical usage with the standard `aws` command. For comprehensive usage instructions, refer to the manual pages by running `lstk aws help`. ```bash lstk aws kinesis list-streams ``` ## Using AWS CLI from a pre-built container As an alternative to installing the `aws` command directly on your local machine, Amazon provides a [pre-built container image with the AWS CLI pre-installed](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-docker.html). This approach is most suitable when working in multi-container environments, rather than single-machine installations. If you take this approach, an extra step is required for communication with LocalStack, also running inside a container. By default, the AWS CLI container is isolated from `0.0.0.0:4566` on the host machine, which means the AWS CLI cannot reach LocalStack. To ensure the two docker containers can communicate create a network on the docker engine: ```bash docker network create localstack 0c9cb3d37b0ea1bfeb6b77ade0ce5525e33c7929d69f49c3e5ed0af457bdf123 ``` Then modify the `docker-compose.yml` specifying the network to use: ```yaml networks: default: external: name: 'localstack' ``` Run the AWS CLI v2 docker container using this network (example): ```bash docker run --network localstack --rm -it amazon/aws-cli --endpoint-url=http://localstack:4566 lambda list-functions { "Functions": [] } ``` If you use AWS CLI v2 from a docker container often, create an alias: ```bash alias laws='docker run --network localstack --rm -it amazon/aws-cli --endpoint-url=http://localstack:4566' ``` So you can type: ```bash laws lambda list-functions { "Functions": [] } ``` # Overview > Use LocalStack with AWS SDKs to manage your AWS resources locally. import SectionCards from '../../../../../components/SectionCards.astro'; ## Introduction LocalStack integrates with official AWS Software Development Kits (SDKs) so you can connect to LocalStack services using the same SDKs you use for AWS services. This lets you develop and test your applications locally without connecting to the cloud. ## How to connect with AWS SDKs? To connect to LocalStack services using AWS SDKs, you can use one of the following methods: - **Manual configuration:** Manually configure the SDK to connect to LocalStack services by setting the endpoint URL to `http://localhost.localstack.cloud:4566` or `localhost:4566`. This can also be specified using a [profile or an environment variable](https://docs.aws.amazon.com/sdkref/latest/guide/feature-ss-endpoints.html). - **Transparent endpoint injection (recommended):** Connect to LocalStack services without modifying your application code. Transparent endpoint injection uses the integrated DNS server to resolve AWS API calls to target LocalStack. Refer to the [Transparent Endpoint Injection](/aws/customization/networking/transparent-endpoint-injection/) guide for more information. ## Supported SDKs # AWS SDK for C++ > How to use the C++ AWS SDK with LocalStack. ## Overview The [AWS SDK for C++](https://docs.aws.amazon.com/sdk-for-cpp), like other AWS SDKs, lets you set the endpoint when creating resource clients, which is the preferred way of integrating the C++ SDK with LocalStack. ## Example Consider the following example, which creates an SQS queue, sends a message to it, then receives the same message via the SDK: ```cpp showshowLineNumbers #include #include #include #include #include #include #include #include using namespace Aws; int main() { // initialize AWS SDK SDKOptions options; options.loggingOptions.logLevel = Utils::Logging::LogLevel::Debug; InitAPI(options); // create SQS client with local endpoint configuration Aws::Client::ClientConfiguration clientConfig; clientConfig.endpointOverride = "http://localhost.localstack.cloud:4566"; SQS::SQSClient client = SQS::SQSClient(clientConfig); // create queue std::cout << "Creating queue ..." << std::endl; Aws::SQS::Model::CreateQueueRequest cqReq; cqReq.SetQueueName("test-queue"); auto cqRes = client.CreateQueue(cqReq); auto queueUrl = cqRes.GetResult().GetQueueUrl(); // send message std::cout << "Sending message ..." << std::endl; Aws::SQS::Model::SendMessageRequest smReq; smReq.SetQueueUrl(queueUrl); smReq.SetMessageBody("test message 123"); client.SendMessage(smReq); // receive message std::cout << "Receiving message ..." << std::endl; Aws::SQS::Model::ReceiveMessageRequest rmReq; rmReq.SetQueueUrl(queueUrl); auto rmRes = client.ReceiveMessage(rmReq); const auto& messages = rmRes.GetResult().GetMessages(); std::cout << "Received " << messages.size() << " messages from queue " << queueUrl << std::endl; // print message const auto& message = messages[0]; std::cout << "Received message:" << std::endl; std::cout << " MessageId: " << message.GetMessageId() << std::endl; std::cout << " ReceiptHandle: " << message.GetReceiptHandle() << std::endl; std::cout << " Body: " << message.GetBody() << std::endl; // delete message std::cout << "Deleting message ..." << std::endl; Aws::SQS::Model::DeleteMessageRequest dmReq; dmReq.SetQueueUrl(queueUrl); dmReq.SetReceiptHandle(message.GetReceiptHandle()); auto dmRes = client.DeleteMessage(dmReq); std::cout << "Delete message success result: " << dmRes.IsSuccess() << std::endl; // shut down the SDK ShutdownAPI(options); return 0; } ``` Once compiled, we'll see the following output when running the application: ```sh Creating queue ... Sending message ... Receiving message ... Received 1 messages from queue http://localhost.localstack.cloud:4566/000000000000/test-queue Received message: MessageId: 4731b327-49b2-4410-a8da-2c479e7bde04 ReceiptHandle: ZTQ1M2Q1ZjYtMjBkZS00ODQxLTlkYzQtMjBlMGQ4MDNkODVkIGFybjphd3M6c3FzOnVzLWVhc3QtMTowMDAwMDAwMDAwMDA6dGVzdC1xdWV1ZSA0NzMxYjMyNy00OWIyLTQ0MTAtYThkYS0yYzQ3OWU3YmRlMDQgMTY3ODIxMjExMS42ODk1NTE= Body: test message 123 Deleting message ... Delete message success result: 1 ``` ## Resources - [AWS SDK for C++](https://docs.aws.amazon.com/sdk-for-cpp) - [Getting Started Guide](https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/getting-started.html) # AWS SDK for .NET > How to use the .NET AWS SDK with LocalStack. ## Overview The [AWS SDK for .NET](https://aws.amazon.com/sdk-for-net/), like other AWS SDKs, lets you set the endpoint when creating resource clients, which is the preferred way of integrating the .NET SDK with LocalStack. ## Example Here is an example of how to create an `LambdaClient` with the endpoint set to LocalStack. ```csharp var lambdaClient = new AmazonLambdaClient(new AmazonLambdaConfig( { ServiceURL = "http://localhost.localstack.cloud:4566" } ); ``` If you want to specify a region and credentials when creating the client, please set them as `AuthenticationRegion` and `BasicAWSCredentials`, like in this example: ```csharp var lambdaClient = new AmazonLambdaClient(new BasicAWSCredentials("test", "test"), new AmazonLambdaConfig( { ServiceURL = "http://localhost.localstack.cloud:4566", AuthenticationRegion = "eu-west-1" } ); ``` :::note Make sure you are setting the `AuthenticationRegion` and not the `RegionEndpoint`. Setting the `RegionEndpoint` to a constant like `RegionEndpoint.EUWest1` will override the ServiceURL, and your request will end up against AWS. ::: ### S3 specific endpoint Here is another example, this time with an `S3Client` and its specific endpoint. ```csharp var config = new AmazonS3Config({ ServiceURL = "http://s3.localhost.localstack.cloud:4566" }); var s3client = new AmazonS3Client(config); ``` :::note In case of issues resolving this DNS record, we can fallback to [http://localhost:4566](http://localhost:4566) in combination with the provider setting `ForcePathStyle = true`. The S3 service endpoint is slightly different from the other service endpoints, because AWS is deprecating path-style based access for hosting buckets. ::: ```csharp var config = new AmazonS3Config( { ServiceURL = "http://localhost.localstack.cloud:4566", ForcePathStyle = true } ); var s3client = new AmazonS3Client(config); ``` ## Alternative: Using LocalStack.NET As an alternative to manual endpoint configuration, you can use LocalStack.NET, an easy-to-use .NET client for LocalStack. ### Overview `LocalStack.NET` provides a thin wrapper around the official [aws-sdk-net](https://github.com/aws/aws-sdk-net) (AWS SDK for .NET). It automatically configures the target endpoints to use LocalStack for your local cloud application development. When LocalStack is disabled in configuration, LocalStack.NET automatically uses the official AWS SDK clients, allowing your application to target your real AWS account with no code changes. **LocalStack.NET Documentation:** Comprehensive guide and examples [here](https://github.com/localstack-dotnet/localstack-dotnet-client). ### How it Works Instead of manually setting the endpoint configurations when initializing a client, `LocalStack.NET` offers methods that handle these details. The library aims to reduce the boilerplate required to set up LocalStack clients in .NET. ### Example Usage #### Dependency Injection Approach ```csharp showshowLineNumbers public void ConfigureServices(IServiceCollection services) { // Add framework services. services.AddMvc(); services.AddLocalStack(Configuration) services.AddDefaultAWSOptions(Configuration.GetAWSOptions()); services.AddAwsService(); } ... var amazonS3Client = serviceProvider.GetRequiredService(); ``` #### Standalone Approach ```csharp showshowLineNumbers var sessionOptions = new SessionOptions(); var configOptions = new ConfigOptions(); ISession session = SessionStandalone.Init() .WithSessionOptions(sessionOptions) .WithConfigurationOptions(configOptions).Create(); var amazonS3Client = session.CreateClientByImplementation(); ``` ### Benefits - **Consistent Client Configuration:** `LocalStack.NET` provides a standardized approach to initialize clients, eliminating the need for manual endpoint configurations. - **Tailored for .NET Developers:** The library offers functionalities specifically developed to streamline integration of LocalStack with .NET applications. - **Adaptable Environment Transition:** Switching between LocalStack and actual AWS services can be achieved with minimal configuration changes when leveraging `LocalStack.NET`. - **Versatile .NET Compatibility:** Supports a broad spectrum of .NET versions, from .NET Framework 4.6.1 and .NET Standard 2.0, up to recent .NET iterations such as .NET 10. ### Considerations - Both the standard AWS SDK method and `LocalStack.NET` provide ways to integrate with LocalStack using .NET. The choice depends on developer preferences and specific project needs. - `LocalStack.NET` works alongside the AWS SDK, using it as a base and providing a more focused API for LocalStack interactions. ## Aspire Integration If you are building cloud-native applications with [Aspire](https://aspire.dev/), LocalStack provides first-class integration through the Aspire orchestration framework. The [`LocalStack.Aspire.Hosting`](https://github.com/localstack-dotnet/dotnet-aspire-for-localstack) package enables seamless local development with automatic container lifecycle management, service discovery, and observability integration. For detailed guidance on using LocalStack with Aspire, including configuration options and example projects, see the [Aspire integration guide](/aws/customization/integrations/app-frameworks/aspire). ## Resources - [AWS SDK for .NET](https://aws.amazon.com/sdk-for-net/) - [Official repository of the AWS SDK for .NET](https://github.com/aws/aws-sdk-net) - [LocalStack.NET Documentation](https://github.com/localstack-dotnet/localstack-dotnet-client) - [LocalStack.Aspire.Hosting Documentation](https://github.com/localstack-dotnet/dotnet-aspire-for-localstack) # AWS SDK for Go > How to use the Go AWS SDK with LocalStack. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Overview The [AWS SDK for Go](https://aws.amazon.com/sdk-for-go/), like other AWS SDKs, lets you set the endpoint when creating resource clients, which is the preferred way of integrating the Go SDK with LocalStack. The Go SDK has two major versions, each with their own way of specifying the LocalStack endpoint: * [aws-sdk-go](https://github.com/aws/aws-sdk-go) * [aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) ## Examples Here is an example of how to create an S3 Client from a Session with the endpoint set to LocalStack. Full examples for both SDK versions can be found [in our samples repository](https://github.com/localstack/localstack-aws-sdk-examples/tree/main/go). ```go showshowLineNumbers package main import ( "github.com/aws/aws-sdk-go/aws" "github.com/aws/aws-sdk-go/aws/session" "github.com/aws/aws-sdk-go/service/s3" "github.com/aws/aws-sdk-go/aws/credentials" ) func main() { // Initialize a session sess, _ := session.NewSession(&aws.Config{ Region: aws.String("us-east-1"), Credentials: credentials.NewStaticCredentials("test", "test", ""), S3ForcePathStyle: aws.Bool(true), Endpoint: aws.String("http://localhost:4566"), }) // Create S3 service client client := s3.New(sess) // ... } ``` ```go showshowLineNumbers package main import ( "context" "log" "github.com/aws/aws-sdk-go-v2/aws" "github.com/aws/aws-sdk-go-v2/config" "github.com/aws/aws-sdk-go-v2/service/s3" ) func main() { awsEndpoint := "http://localhost:4566" awsRegion := "us-east-1" awsCfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion(awsRegion), ) if err != nil { log.Fatalf("Cannot load the AWS configs: %s", err) } // Create the resource client client := s3.NewFromConfig(awsCfg, func(o *s3.Options) { o.UsePathStyle = true o.BaseEndpoint = aws.String(awsEndpoint) }) // ... } ``` ## Resources * [localstack-aws-sdk-examples for Go](https://github.com/localstack/localstack-aws-sdk-examples/tree/main/go) * [AWS SDK for Go](https://aws.amazon.com/sdk-for-go/) * [Official repository of the AWS SDK for Go (v1)](https://github.com/aws/aws-sdk-go) * [Official repository of the AWS SDK for Go (v2)](https://github.com/aws/aws-sdk-go-v2) # AWS SDK for Java > How to use the Java AWS SDK with LocalStack. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Overview The AWS SDK for Java provides a Java API for AWS services. Using the SDK, your Java application can interact with LocalStack services the same way it does with Amazon services. Support for new services is regularly added to the SDK. For a list of the supported services and their API versions that are included with each release of the SDK, view the [release notes](https://github.com/aws/aws-sdk-java#release-notes) for the version that you’re working with. The Java SDK currently supports two major versions: * [AWS SDK for Java v1](https://github.com/aws/aws-sdk-java) * [AWS SDK for Java v2](https://github.com/aws/aws-sdk-java-v2) ## Examples Full examples for both SDK versions can be found in the [example repository](https://github.com/localstack/localstack-aws-sdk-examples/tree/main/java). This includes proper exception handling and all the necessary Maven dependencies. The scripts to create the AWS services on LocalStack can be found under the `src/main/resources` folder of each module in the repository. ### S3 Service Below you'll find an example of how to create an S3 client with the endpoint configured for LocalStack. The client can be used to upload a file to an existing bucket and then retrieve it. #### Configuring the S3 Client ```java showshowLineNumbers // Credentials that can be replaced with real AWS values. (To be handled properly and not hardcoded.) // These can be skipped altogether for LocalStack, but we generally want to avoid discrepancies with production code. final String ACCESS_KEY = "test"; final String SECRET_KEY = "test"; // S3 Client with configured credentials, endpoint directing to LocalStack and desired region. AmazonS3 s3Client = AmazonS3ClientBuilder.standard() .withCredentials(new AWSStaticCredentialsProvider(credentials)) .withEndpointConfiguration(new EndpointConfiguration("s3.localhost.localstack.cloud:4566", Regions.US_EAST_1.getName())) .build(); ``` ```java showshowLineNumbers // Credentials that can be replaced with real AWS values. (To be handled properly and not hardcoded.) // These can be skipped altogether for LocalStack, but we generally want to avoid discrepancies with production code. final String ACCESS_KEY = "test"; final String SECRET_KEY = "test"; // Desired region. Region region = Region.US_EAST_1; // S3 Client with configured credentials, endpoint directing to LocalStack and desired region. S3Client s3Client = S3Client.builder() .endpointOverride(URI.create("https://s3.localhost.localstack.cloud:4566")) .credentialsProvider(StaticCredentialsProvider.create( AwsBasicCredentials.create(ACCESS_KEY, SECRET_KEY))) .region(region) .build(); ``` #### Interacting with S3 ```java showshowLineNumbers // Existing bucket name. final String BUCKET_NAME = "records"; // The key of the object in the bucket, usually the name of the file. final String key = "hello-v1.txt"; // Creating a File object and FileInputStream. File file = new File(filePathOnDisk); InputStream fileInputStream = new FileInputStream(file); // Creating an ObjectMetadata to specify the content type and content length. ObjectMetadata metadata = new ObjectMetadata(); metadata.setContentType("text/plain"); metadata.setContentLength(file.length()); // Put the file into the S3 bucket s3Client.putObject(new PutObjectRequest(BUCKET_NAME, key, fileInputStream, metadata)); // Retrieving the object from the bucket. S3Object s3Object = s3Client.getObject(BUCKET_NAME, key); // Read the text content of the file using a BufferedReader. S3ObjectInputStream objectInputStream = s3Object.getObjectContent(); BufferedReader reader = new BufferedReader(new InputStreamReader(objectInputStream)); ``` ```java showshowLineNumbers // Existing bucket name. final String BUCKET_NAME = "records"; // The key of the object in the bucket, usually the name of the file. final String objectKey = "hello-v2x.txt"; // Creating the PUT request with all the relevant information. PutObjectRequest putObjectRequest = PutObjectRequest.builder() .bucket(BUCKET_NAME) .key(objectKey) .build(); // Put the file into the S3 bucket PutObjectResponse response = s3Client.putObject(putObjectRequest, Paths.get(filePath)); // Creating the GET request with all the relevant information. GetObjectRequest getObjectRequest = GetObjectRequest.builder() .bucket(BUCKET_NAME) .key(objectKey) .build(); // Retrieving the object from the bucket. ResponseInputStream response = s3Client.getObject(getObjectRequest); ``` ### DynamoDB Service Another interesting case is interacting with the DynamoDB service. Here we can see code snippets of a DynamoDB client inserting an entity of type `Person` into a table with the same name. Once the object is in the database, we would like to retrieve it as well. Just like the example before, the scripts to create the AWS services on LocalStack can be found under the `src/main/resources` folder of each module in the repository. Pay particular attention to the handling of the data model in the v2 example. As part of improvements, some boilerplate code can be abstracted with the help of specific annotations, which help label the Java bean, the partition key and even specify converters for certain data types. Unfortunately, the enhanced mapping in v2 does not support Date type, but a handwritten converter is enough to cater to the application's needs. The full list of supported converters can be found [here](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/enhanced/dynamodb/internal/converter/attribute/package-summary.html). #### Configuring the DynamoDB Client ```java showshowLineNumbers // Credentials that can be replaced with real AWS values. (To be handled properly and not hardcoded.) // These can be skipped altogether for LocalStack, but we generally want to avoid discrepancies with production code. final String ACCESS_KEY = "test"; final String SECRET_KEY = "test"; // Building a basic credentials object. BasicAWSCredentials credentials = new BasicAWSCredentials(ACCESS_KEY, SECRET_KEY); // Creating the DynamoDB client using the credentials, specific region and an endpoint configured for LocalStack. private static AmazonDynamoDB dynamoDBClient = AmazonDynamoDBClientBuilder.standard() .withCredentials(new AWSStaticCredentialsProvider(credentials)) .withEndpointConfiguration( new EndpointConfiguration("localhost.localstack.cloud:4566", Regions.US_EAST_1.getName())) .build(); ``` ```java showshowLineNumbers // Credentials that can be replaced with real AWS values. (To be handled properly and not hardcoded.) // These can be skipped altogether for LocalStack, but we generally want to avoid discrepancies with production code. final String ACCESS_KEY = "test"; final String SECRET_KEY = "test"; // Creating the AWS Credentials provider, using the above access and secret keys. AwsCredentialsProvider credentials = StaticCredentialsProvider.create( AwsBasicCredentials.create(ACCESS_KEY, SECRET_KEY)); // Selected region. Region region = Region.US_EAST_1; // Creating the dynamoDB client using the credentials, the specific region and a LocalStack endpoint. DynamoDbClient dynamoDbClient = DynamoDbClient.builder() .region(region) .credentialsProvider( credentials) .endpointOverride(URI.create("https://localhost.localstack.cloud:4566")) .build(); // Creating an enhanced client, which provides additional actions to the plain client. DynamoDbEnhancedClient enhancedClient = DynamoDbEnhancedClient.builder() .dynamoDbClient(dynamoDbClient) .build(); ``` #### Interacting with DynamoDB ```java showshowLineNumbers // Existing table name String TABLE_NAME = "person"; // Creating a DynamoDB instance DynamoDB dynamoDB = new DynamoDB(dynamoDBClient); // Creating a Table object that will be an interface in interacting with the DB table. Table table = dynamoDB.getTable(TABLE_NAME); //Creating a Person object. Person person = new Person(); person.setId(personID); person.setName("John Doe"); person.setBirthdateFromString("1984-10-12"); // Creating an Item to represent the Person object. Item item = new Item() .withPrimaryKey("id", person.getId()) .withString("name", person.getName()) .withString("birthdate", person.getBirthdateAsString()); // Adding the item to the DynamoDB table. table.putItem(item); // The "id" field was defined as partition key. String partitionKey = "id"; // Getting the item from the DynamoDB table using the primary key. Item item = table.getItem(partitionKey, personId); // Mapping the DynamoDB item to a Person object. Person person = new Person(); person.setId(item.getString("id")); person.setName(item.getString("name")); person.setBirthdateFromString(item.getString("birthdate")); ``` ```java showshowLineNumbers // Existing table name. String TABLE_NAME = "person"; // Creating the Person object. Person person = new Person(); person.setId(personID); person.setName("John Doe"); person.setBirthdateFromString("1979-01-01"); // Using the enhanced client to interact with the table. DynamoDbTable table = enhancedClient.table(TABLE_NAME, TableSchema.fromBean(Person.class)); // Adding new entry to the table. table.putItem(person); // Person's id. String personId = "000012356"; // Retrieving the entity based on id. Person person = table.getItem(Key.builder().partitionValue(personId).build()); ``` ## Resources * [AWS SDK for Java](https://aws.amazon.com/sdk-for-java/) * [Official repository of the AWS SDK for Java (v1)](https://github.com/aws/aws-sdk-java) * [Official repository of the AWS SDK for Java (v2)](https://github.com/aws/aws-sdk-java-v2) * [localstack-aws-sdk-examples for Java](https://github.com/localstack/localstack-aws-sdk-examples/tree/main/java) # AWS SDK for JavaScript > How to use the JavaScript AWS SDK with LocalStack. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Overview The [AWS SDK for JavaScript](https://aws.amazon.com/sdk-for-javascript/), like other AWS SDKs, lets you set the endpoint when creating resource clients, which is the preferred way of integrating the JavaScript SDK with LocalStack. The JavaScript SDK has two major versions, each with their own way of specifying the LocalStack endpoint: * [aws-sdk-js](https://github.com/aws/aws-sdk-js) * [aws-sdk-js-v3](https://github.com/aws/aws-sdk-js-v3) ## Examples Here is an example of how to create a Lambda client and an S3 client with the endpoint set to LocalStack. ```javascript showshowLineNumbers const AWS = require('aws-sdk'); // Configure the AWS SDK to use the LocalStack endpoint and credentials const lambda = new AWS.Lambda({ endpoint: 'http://localhost:4566', accessKeyId: 'test', secretAccessKey: 'test', region: 'us-east-1', }); // List the Lambda functions using the LocalStack endpoint lambda.listFunctions({}, (err, data) => { if (err) { console.error(err); } else { console.log(data); } }); // Now, we create an S3 client, which has a special endpoint // You can read the S3 documentation to learn more about the different endpoints. const s3 = new AWS.S3({ endpoint: 'http://s3.localhost.localstack.cloud:4566', s3ForcePathStyle: true, // If you want to use virtual host addressing of buckets, you can remove `s3ForcePathStyle: true`. accessKeyId: 'test', secretAccessKey: 'test', region: 'us-east-1', }); // If your region is `us-east-1`, you will need to override the globalEndpoint of the client // due to an issue in the SDK with `createBucket`. // you will need to set it to the hostname of your endpoint specified right above // If your region is different than `us-east-1`, you can skip that line s3.api.globalEndpoint = 's3.localhost.localstack.cloud'; // Call an S3 API using the LocalStack endpoint s3.listBuckets((err, data) => { if (err) { console.error(err); } else { console.log(data); } }); ``` ```javascript showshowLineNumbers const { LambdaClient, ListFunctionsCommand } = require('@aws-sdk/client-lambda'); const { S3Client, ListBucketsCommand } = require('@aws-sdk/client-s3'); // Configure the AWS SDK to use the LocalStack endpoint and credentials const lambda = new LambdaClient({ endpoint: 'http://localhost:4566', region: 'us-east-1', credentials: { accessKeyId: 'test', secretAccessKey: 'test', }, }); // Call a Lambda API using the LocalStack endpoint lambda.send(new ListFunctionsCommand({})) .then((data) => console.log(data)) .catch((error) => console.error(error)); // By default, @aws-sdk/client-s3 will using virtual host addressing: // -> http://.s3.localhost.localstack.cloud:4566/ // To allow those requests to be directed to LocalStack, you need to set a specific endpoint. // If this is not possible, you can set the special S3 configuration flag to use path // addressing instead: // -> http://s3.localhost.localstack.cloud:4566// // You can read the S3 documentation to learn more about the different endpoints. const s3 = new S3Client({ region: 'us-east-1', forcePathStyle: true, // If you want to use virtual host addressing of buckets, you can remove `forcePathStyle: true`. endpoint: 'http://s3.localhost.localstack.cloud:4566', credentials: { accessKeyId: 'test', secretAccessKey: 'test', }, }); // Call an S3 API using the LocalStack endpoint s3.send(new ListBucketsCommand({})) .then((data) => console.log(data)) .catch((error) => console.error(error)); ``` :::note In case of issues resolving S3 DNS record, we can fallback to `http://localhost:4566` in combination with the provider setting `forcePathStyle: true` (see the specific way of setting this parameter for each SDK above). The S3 service endpoint is slightly different from the other service endpoints, because AWS is deprecating path-style based access for hosting buckets. See [S3 documentation](/aws/services/s3) about endpoints. ::: ## Resources * [AWS SDK for JavaScript](https://aws.amazon.com/sdk-for-javascript/) * [Official repository of the AWS SDK for JavaScript (v2)](https://github.com/aws/aws-sdk-js) * [Official repository of the AWS SDK for JavaScript (v3)](https://github.com/aws/aws-sdk-js-v3) # AWS SDK for PHP > How to use the PHP AWS SDK with LocalStack. ## Overview The [AWS SDK for PHP](https://aws.amazon.com/sdk-for-php/), like other AWS SDKs, lets you set the endpoint when creating resource clients, which is the preferred way of integrating the PHP SDK with LocalStack. ## Example Here is an example of how to create an `S3Client` with the endpoint set to LocalStack. ```php showshowLineNumbers use Aws\S3\S3Client; use Aws\Exception\AwsException; // Configuring S3 Client with virtual-hosted-style addressing (recommended) $s3 = new Aws\S3\S3Client([ 'version' => '2006-03-01', 'region' => 'us-east-1', 'endpoint' => 'http://s3.localhost.localstack.cloud:4566', ]); ``` This configuration uses virtual-hosted-style addressing, which AWS recommends and some regions require. If you need to use path-style addressing (for non-DNS-compliant bucket names or other specific requirements), enable it explicitly: ```php showshowLineNumbers // Only use path-style if you have a specific requirement for it $s3 = new Aws\S3\S3Client([ 'version' => '2006-03-01', 'region' => 'us-east-1', 'use_path_style_endpoint' => true, 'endpoint' => 'http://localhost:4566', // Use non-S3-prefixed endpoint with path-style ]); ``` A full example can be found [in our samples repository](https://github.com/localstack/localstack-aws-sdk-examples/tree/main/php). ## Resources * [localstack-aws-sdk-examples for PHP](https://github.com/localstack/localstack-aws-sdk-examples/tree/main/php) * [AWS SDK for PHP](https://aws.amazon.com/sdk-for-php/) * [Official repository of the AWS SDK for PHP](https://github.com/aws/aws-sdk-php) # AWS SDK for Python (Boto3) > How to use the Python Boto3 AWS SDK with LocalStack. [Boto3](https://github.com/boto/boto3) is the Amazon Web Services (AWS) Software Development Kit (SDK) for Python, which allows Python developers to write software that makes use of AWS services. You can easily create a `boto3` client that interacts with your LocalStack instance. The example below creates a `boto3` client that lists all available Lambda functions: ```python showshowLineNumbers import boto3 endpoint_url = "http://localhost.localstack.cloud:4566" # alternatively, to use HTTPS endpoint on port 443: # endpoint_url = "https://localhost.localstack.cloud" def main(): client = boto3.client("lambda", endpoint_url=endpoint_url) result = client.list_functions() print(result) if __name__ == "__main__": main() ``` :::note If you're connecting from within a Python **Lambda function** handler in LocalStack, you can create a default client without configuring the `endpoint_url` - LocalStack will automatically forward the invocations to the local API endpoints (available in Pro, see [here](/aws/customization/networking/transparent-endpoint-injection) for more details). ::: ```python client = boto3.client("lambda") ... ``` Alternatively, if you prefer to (or need to) set the endpoints directly, you can use the environment variable `AWS_ENDPOINT_URL`, which is available when executing user code (e.g., Lambda functions) in LocalStack: ```python import os client = boto3.client("lambda", endpoint_url=os.getenv("AWS_ENDPOINT_URL")) ... ``` ### Further Material * [localstack-python-client](https://github.com/localstack/localstack-python-client): small Python library with additional utils for interacting with LocalStack # AWS SDK for Ruby > How to use the Ruby AWS SDK with LocalStack. ## Overview The [AWS SDK for Ruby](https://aws.amazon.com/sdk-for-ruby/), like other AWS SDKs, lets you set the endpoint when creating resource clients, which is the preferred way of integrating the Ruby SDK with LocalStack. ## Example Here is an example of how to create a S3 bucket with the AWS configuration endpoint set to LocalStack: ```ruby showshowLineNumbers require "aws-sdk-s3" # Wraps Amazon S3 bucket actions. class BucketCreateWrapper attr_reader :bucket # @param bucket [Aws::S3::Bucket] An Amazon S3 bucket initialized with a name. def initialize(bucket) @bucket = bucket end # Creates an Amazon S3 bucket in the specified AWS Region. # # @param region [String] The Region where the bucket is created. # @return [Boolean] True when the bucket is created; otherwise, false. def create?(region) @bucket.create(create_bucket_configuration: { location_constraint: region }) true rescue Aws::Errors::ServiceError => e puts "Couldn't create bucket. Here's why: #{e.message}" false end # Gets the Region where the bucket is located. # # @return [String] The location of the bucket. def location if @bucket.nil? "None. You must create a bucket before you can get its location!" else @bucket.client.get_bucket_location(bucket: @bucket.name).location_constraint end rescue Aws::Errors::ServiceError => e "Couldn't get the location of #{@bucket.name}. Here's why: #{e.message}" end end def run_demo region = "us-east-2" Aws.config.update( endpoint: 'http://s3.localhost.localstack.cloud:4566', # update with localstack endpoint access_key_id: 'test', # update with localstack credentials secret_access_key: 'test', # update with localstack credentials region: region, force_path_style: true, # Enable 'force_path_style' => true, if bucket name is non DNS compliant ) bucket_name = "doc-example-bucket-#{SecureRandom.uuid}" wrapper = BucketCreateWrapper.new(Aws::S3::Bucket.new(bucket_name)) return unless wrapper.create?(region) puts "Created bucket #{wrapper.bucket.name}." puts "Your bucket's region is: #{wrapper.location}" end run_demo if $PROGRAM_NAME == __FILE__ ``` You can run the example by saving it to a file, for example `localstack.rb`, and then running it with: ```bash ruby ./localstack.rb Created bucket doc-example-bucket-b911f85f-4dd3-4668-a32e-3f69aa4e37dc. Your bucket's region is: us-east-2 ``` :::note The endpoint we configure for the S3 and virtual host bucket is `http://s3.localhost.localstack.cloud`. In case of issues resolving the DNS record, we can fall back to `http://localhost:4566` in combination with the provider setting `force_path_style: true`. The S3 service endpoint differs slightly from the other service endpoints because AWS deprecates path-style-based access for hosting buckets. ::: For alternative AWS services, you can use the following configuration: ```ruby showshowLineNumbers region = "us-east-2" Aws.config.update( endpoint: 'http://localhost.localstack.cloud:4566', # update with localstack endpoint access_key_id: 'test', # update with localstack credentials secret_access_key: 'test', # update with localstack credentials region: region, ) ``` ## Resources - [AWS SDK for Ruby](https://aws.amazon.com/sdk-for-ruby/) - [Official repository of the AWS SDK for Ruby](https://github.com/aws/aws-sdk-ruby) # Overview > Use the LocalStack web app as a local equivalent of the AWS Console to inspect and manage resources running on LocalStack. import SectionCards from '../../../../../components/SectionCards.astro'; If you use the AWS Console today to inspect resources, navigate stacks, or check on your environment, the LocalStack web app gives you the same kind of view for the resources running on your LocalStack instance. Sign in once, and you can browse and manage your local AWS resources without leaving the browser. # Instance Management > Instance Management allows you to view and manage your LocalStack instances through the LocalStack Web Application alongside other auxiliary features. ## Introduction Once your LocalStack instance is running, the [LocalStack Web Application](https://app.localstack.cloud/) is the primary way to access and inspect it from your browser, the local equivalent of opening the AWS Console. **Instance Management** is where you first connect to a running instance and see everything it exposes. You can access it through the [**LocalStack Instances**](https://app.localstack.cloud/instances) section in the sidebar of the LocalStack Web Application. From here you can view and manage your LocalStack instances while you build and test your cloud applications locally. Instance Management offers these features: - **Overview**: Shows the stack details of your LocalStack instances. - **Status**: Shows the status of the services running in the LocalStack container. - **Resource Browser**: Lets you view and manage your local AWS resources. - **State**: Allows you to export and import the state of your LocalStack instances. - **IAM Policy Stream**: Provides a stream of IAM policies corresponding to the AWS API calls. - **App Inspector**: Provides insight into the state of your running application, including data payloads and IAM policy evaluation. - **Chaos Engineering**: Allows you to inject failures & simulate outages in your LocalStack instance. - **Extensions**: Provides extra integrations to improve your LocalStack experience. ![LocalStack Web Application's Instance Management page](/images/aws/instance-management.png) ## Instance Bookmark Instance Bookmark lets users save references to instances without directly creating or managing them. To create an Instance Bookmark, do the following: - Click on the **Add Bookmark** button on the Instance Management page. - Enter a name for the bookmark, specify the endpoint, and add a description. - Click on the **Save Bookmark** button. ![Instance Bookmark](/images/aws/new-instance-bookmark.png) ## Connect to an instance on a different machine You can use the Instance Bookmark feature to connect the LocalStack Web Application to a LocalStack instance running on a different machine. To connect the Web Application with your running LocalStack instance, you need to ensure the endpoint URL’s server SSL certificate corresponds to the hostname/IP address of the URL. This is necessary when the endpoint URL is set as something like `https://myhost:4566` or uses an IP address like `https://1.2.3.4:4566`. Sites with an `https://...` URL must use HTTPS for requests, and the SSL certificate must match the hostname (e.g., localhost.localstack.cloud). To address this, consider setting up a local TCP proxy server that listens on `127.0.0.1:4566` and forwards all requests to the endpoint where your LocalStack instance runs. In the Web user interface, you can keep the default setting, `https://localhost.localstack.cloud:4566`. Tools like [simpleproxy](https://manpages.ubuntu.com/manpages/trusty/man1/simpleproxy.1.html) or [proxy.py](https://github.com/abhinavsingh/proxy.py) can help set this up. Alternatively, you can direct `localhost.localstack.cloud` to your target machine's IP address by modifying the `/etc/hosts` file, which is useful if you’re using the LocalStack Web UI on a macOS or Linux-based machine. :::note To bind to a custom IP address and port, configure the ['GATEWAY_LISTEN' configuration variable](/aws/customization/configuration-options/#core). For troubleshooting, refer to the [network troubleshooting docs](/aws/customization/networking/). ::: # Resource Browser > The Resource Browser allows you to view and manage your local AWS resources through the LocalStack Web Application. ## Introduction The LocalStack Resource Browser allow you to view, manage, and deploy AWS resources locally while building & testing their cloud applications locally. It provides an internal, integrated experience, similar to the AWS Management Console, to manage the ephemeral resources in a LocalStack container on your local machine. ![LocalStack Web Application's Resource Browsers outlining various local AWS services](/images/aws/resource-browser.png) The Resource Browser provide an experience similar to the AWS Management Console. However, the Resource Browser is not a replacement for the AWS Management Console and only replicate some of the features of the AWS Management Console. We recommend using our [integrations](/aws/customization/integrations/) to create your resources, with the Resource Browser being used for quick viewing and management of your resources. The LocalStack Web Application connects to your LocalStack container and retrieves the information about your local resources directly via `localhost` without using the internet. None of the information is sent to the internet, or stored on any external servers maintained by LocalStack. :::tip An AWS region dropdown menu in the dashboard is located on the top right of the page. You can select your desired region to ensure that you can view your resources. If you cannot view resources that you have recently created, you should verify that you are checking the resources in the correct region. ::: ## Supported services The Resource Browser supports the following AWS services: | Resource Group | Service | |------------------------------|-------------------------------------------------------------------------------------------------------| | **App Integration** | [API Gateway](https://app.localstack.cloud/inst/default/resources/apigateway) | | | [Amazon MQ](https://app.localstack.cloud/inst/default/resources/mq/brokers) | | | [Amazon MWAA](https://app.localstack.cloud/inst/default/resources/mwaa/environments) | | | [Amazon SNS](https://app.localstack.cloud/inst/default/resources/sns) | | | [Amazon SQS](https://app.localstack.cloud/inst/default/resources/sqs) | | | [Application Auto Scaling](https://app.localstack.cloud/inst/default/resources/application-autoscaling) | | | [AWS Step Functions](https://app.localstack.cloud/inst/default/resources/stepfunctions) | | **Compute** | [Amazon EC2](https://app.localstack.cloud/inst/default/resources/ec2) | | | [Amazon ECS](https://app.localstack.cloud/inst/default/resources/ecs) | | | [Amazon ECR](https://app.localstack.cloud/inst/default/resources/ecr/repositories) | | | [Amazon EKS](https://app.localstack.cloud/inst/default/resources/eks/clusters) | | | [AWS Lambda](https://app.localstack.cloud/inst/default/resources/lambda/functions) | | **Management/Governance** | [AWS Account](https://app.localstack.cloud/inst/default/resources/account/contactinfo) | | | [AWS CloudFormation](https://app.localstack.cloud/inst/default/resources/cloudformation) | | | [Amazon CloudWatch](https://app.localstack.cloud/inst/default/resources/cloudwatch) | | | [Amazon CloudTrail](https://app.localstack.cloud/inst/default/resources/cloudtrail/events) | | | [Amazon EventBridge (CloudWatch Events)](https://app.localstack.cloud/inst/default/resources/events) | | | [AWS Systems Manager (SSM)](https://app.localstack.cloud/inst/default/resources/ssm) | | **Business Applications** | [Amazon SES](https://app.localstack.cloud/inst/default/resources/ses) | | **Developer Tools** | [AWS AppConfig](https://app.localstack.cloud/inst/default/resources/appconfig/applications) | | | [AWS CodeCommit](https://app.localstack.cloud/inst/default/resources/codecommit/repositories) | | **Front-end Web & Mobile** | [AWS Amplify](https://app.localstack.cloud/inst/default/resources/amplify/apps) | | | [AWS AppSync](https://app.localstack.cloud/inst/default/resources/appsync) | | **Security Identity Compliance** | [AWS ACM (Certificate Manager)](https://app.localstack.cloud/inst/default/resources/acm/certificates) | | | [Amazon Cognito Identity](https://app.localstack.cloud/inst/default/resources/cognito-idp) | | | [AWS IAM (Identity and Access Management)](https://app.localstack.cloud/inst/default/resources/iam) | | | [AWS Key Management Service (KMS)](https://app.localstack.cloud/inst/default/resources/kms) | | | [AWS Secrets Manager](https://app.localstack.cloud/inst/default/resources/secretsmanager) | | **Storage** | [Amazon S3](https://app.localstack.cloud/inst/default/resources/s3) | | | [AWS Backup](https://app.localstack.cloud/inst/default/resources/backup/plans) | | **Machine Learning** | [Amazon SageMaker](https://app.localstack.cloud/inst/default/resources/sagemaker/models) | | | [Amazon Transcribe](https://app.localstack.cloud/inst/default/resources/transcribe/transcriptionjobs) | | **Database** | [Amazon DynamoDB](https://app.localstack.cloud/inst/default/resources/dynamodb) | | | [Amazon RDS](https://app.localstack.cloud/inst/default/resources/rds) | | | [Amazon ElastiCache](https://app.localstack.cloud/inst/default/resources/elasticache) | | | [Amazon DocumentDB](https://app.localstack.cloud/inst/default/resources/docdb/clusters) | | | [Amazon Neptune](https://app.localstack.cloud/inst/default/resources/neptune/clusters) | | | [Amazon Timestream](https://app.localstack.cloud/inst/default/resources/timestream-write) | | | [Amazon Redshift](https://app.localstack.cloud/inst/default/resources/redshift/clusters) | | **Analytics** | [Amazon Athena](https://app.localstack.cloud/inst/default/resources/athena/databases) | | | [Amazon Kinesis](https://app.localstack.cloud/inst/default/resources/kinesis) | | | [Amazon MSK (Managed Streaming for Kafka)](https://app.localstack.cloud/inst/default/resources/kafka) | | | [AWS Glue](https://app.localstack.cloud/inst/default/resources/glue) | | | [Amazon Route 53](https://app.localstack.cloud/inst/default/resources/route53) | | | [Amazon CloudFront](https://app.localstack.cloud/inst/default/resources/cloudfront/distributions) | | | [Amazon OpenSearch Service](https://app.localstack.cloud/inst/default/resources/opensearch/domains) | | **Cloud Financial Management** | [AWS Cost Explorer](https://app.localstack.cloud/inst/default/resources/ce/costcategorydefinitions) | | **Migration & Transfer** | [Database Migration Service](https://app.localstack.cloud/inst/default/resources/dms/endpoints) | ## Troubleshooting If you encounter a `Network Failure` error message while accessing the Resource Browser, it is likely that the LocalStack container is not running or the instance is not reachable at the endpoint specified in the instance bookmark. # Stack Overview > Stack Overview reflects the current state of your LocalStack environment. ## Introduction The Stack Overview provides a summary of deployed resources, categorized services with configurations, and quick access to resource details like identifiers and endpoints. You can access the Stack Overview in the [LocalStack Web Application](https://app.localstack.cloud/inst/default/overview). Alternatively, go to your LocalStack Instance and click on **Overview** to see a high-level visualization of your locally running cloud app architecture. ![Stack Overview](/images/aws/stack-overview.png) :::note Stack Overview is offered as a **preview** feature and is under active development. ::: ## Supported Resources The following resources are supported by Stack Overview: 1. [`AWS::ApiGateway::RestApi`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-apigateway-restapi.html) 2. [`AWS::CloudFormation::Stack`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudformation-stack.html) 3. [`AWS::DynamoDB::Table`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-dynamodb-table.html) 4. [`AWS::EC2::VPC`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpc.html) 5. [`AWS::Events::EventBus`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-events-eventbus.html) 6. [`AWS::IAM::Group`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-group.html) 7. [`AWS::IAM::Role`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-role.html) 8. [`AWS::IAM::User`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-user.html) 9. [`AWS::Lambda::Function`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-function.html) 10. [`AWS::S3::Bucket`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-s3-bucket.html) 11. [`AWS::SES::EmailIdentity`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ses-emailidentity.html) 12. [`AWS::SNS::Topic`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-sns-topic.html) 13. [`AWS::SQS::Queue`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-sqs-queue.html) 14. [`AWS::SSM::Parameter`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ssm-parameter.html) 15. [`AWS::SecretsManager::Secret`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-secretsmanager-secret.html) 16. [`AWS::StepFunctions::StateMachine`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-stepfunctions-statemachine.html) 17. [`AWS::CloudFront::Distribution`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudfront-distribution.html) 18. [`AWS::Pipes::Pipe`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-pipes-pipe.html) # Credentials > Credentials for accessing LocalStack AWS API Like AWS, LocalStack requires AWS credentials to be supplied in all API operations. ## Access Key ID For root accounts, the choice of access key ID affects [multi-account namespacing](/aws/customization/advanced/multi-account-setups). Access key IDs can be one of following patterns: ### Accounts IDs You can specify a 12-digit number which will be taken by LocalStack as the account ID. For example, `112233445566`. ### Structured access key ID You can specify a structured key like `LSIAQAAAAAAVNCBMPNSG` (which translates to account ID `000000000042`). This must be at least 20 characters in length and must be decodable to an account ID. By default, LocalStack will only accept access keys that start with the `LSIA...` or `LKIA...` prefix. If keys with `ASIA...`/`AKIA...` prefix are provided, these are rejected and the fallback account ID `000000000000` is used. This is a safeguard to prevent misuse of production AWS access key IDs. To disable this safeguard, set the `PARITY_AWS_ACCESS_KEY_ID` configuration variable. :::danger Disabling the access key safeguard and using production access key IDs may cause accidental connections to AWS. We strongly recommend leaving it on. ::: Please refer to the [IAM docs](/aws/services/iam) to learn how to create access keys in LocalStack. ### Alphanumeric string You can also specify an arbitrary alphanumeric access key ID like `test` or `foobar123`. In all such cases, the account ID is evaluated to `000000000000`. ## Secret Access Key The value of the secret access key is generally ignored by LocalStack. We recommend using the same value as access key ID or `test`. S3 can optionally validate request signatures, in which case the secret access key matters: it must be the secret the access key ID was issued with, or `test` for access key IDs not issued by LocalStack. See [S3 signature validation](/aws/services/s3/#signature-validation) for details. # Overview > Connect to LocalStack from your IDE. Install, configure, run, and inspect LocalStack without leaving your editor. import SectionCards from '../../../../../components/SectionCards.astro'; Use IDE extensions to install, configure, run, and inspect LocalStack alongside the code you are developing, without leaving your editor. The LocalStack Toolkit manages the LocalStack lifecycle from VS Code, and the AWS Toolkit for VS Code lets you browse and operate on the resources running on your LocalStack instance from a familiar AWS-style explorer. # AWS Toolkit for VS Code > Use the AWS Toolkit for VS Code to browse and operate on AWS resources running on LocalStack. ## Introduction The [AWS Toolkit for VS Code](https://docs.aws.amazon.com/toolkit-for-vscode/) is an Amazon-maintained extension for Visual Studio Code that lets you interact with AWS resources directly from the editor. When configured with a `localstack` profile, the AWS Toolkit operates against your local LocalStack instance instead of a real AWS account, giving you the same AWS-style resource browser experience you already use against AWS. ## AWS Explorer Once installed, the AWS Toolkit adds an **AWS Explorer** view to the VS Code activity bar. Selecting the `localstack` profile points the Explorer at your local LocalStack instance, where you can browse the resources you have provisioned. ![AWS Explorer view in the AWS Toolkit for VS Code](/images/aws/lambda-remote-debugging/explorer.png) For installation, profile setup, and the full set of features the AWS Toolkit provides, see the [AWS Toolkit for VS Code User Guide](https://docs.aws.amazon.com/toolkit-for-vscode/). # LocalStack Toolkit for VS Code > Install, configure, and run LocalStack without leaving VS Code. ## Introduction The [LocalStack Toolkit for VS Code](https://github.com/localstack/localstack-toolkit-vscode) enables you to install, configure, and run LocalStack without leaving VS Code. ## Prerequisites - [VS Code](https://code.visualstudio.com/) ## Install and configure LocalStack The setup wizard ensures LocalStack is installed and configured for a seamless integration with AWS tools, like the AWS CLI, SDKs, AWS SAM or the AWS CDK. LocalStack can be installed either locally for the current user or globally for all users. You can [start using LocalStack for free by signing up for a free account](https://www.localstack.cloud/pricing) or signing into an existing one. The setup wizard facilitates this process and configures your authentication token required to start LocalStack. The LocalStack Toolkit integrates seamlessly with AWS tools like the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html). It automatically configures a dedicated `localstack` AWS profile in your `.aws/config` and `.aws/credentials` files, if one is not already present. ![Installing LocalStack Toolkit](/images/aws/localstack-toolkit/starting-localstack.png) ## Run LocalStack The LocalStack button in the VS Code status bar provides an instant view of LocalStack's runtime status, such as `stopped` or `running`. ![LocalStack Toolkit Running](/images/aws/localstack-toolkit/running.png) The status bar button provides access to `Start` and `Stop` LocalStack commands. The status button turns red if LocalStack is not found or misconfigured. You can also open the LocalStack log view from here. ![Stop LocalStack Toolkit](/images/aws/localstack-toolkit/stop-localstack.png) ## Viewing LocalStack logs You can see LocalStack logs in the VS Code Output panel. Simply select LocalStack from the drop-down menu. ![LocalStack Toolkit Logs](/images/aws/localstack-toolkit/logs.png) ## `localstack` AWS profile Once the profile is configured you can use it from your favorite AWS tools like AWS CLI, SDKs, CDK to deploy to and interact with LocalStack. For example, to list SQS queues using the AWS CLI and your `localstack` profile: ```bash aws --profile localstack sqs list-queues ``` ## LocalStack Commands Table | ID | Title | Menu Contexts | | --------------------------------- | ---------------------------------- | ---------------- | | `localstack.configureAwsProfiles` | Configure AWS Profile "localstack" | `commandPalette` | | `localstack.setup` | Run Setup Wizard | `commandPalette` | | `localstack.start` | Start LocalStack | `commandPalette` | | `localstack.stop` | Stop LocalStack | `commandPalette` | | `localstack.viewLogs` | View Logs | `commandPalette` | :::note The AWS Toolkit for VS Code, a separate VS Code extension available from Amazon, now provides the ability to connect with LocalStack. This automates much of the existing manual setup required to debug Lambda functions (https://docs.localstack.cloud/aws/developer-tools/lambda-tools/remote-debugging/). ::: ## Contributing [Read our contributing guidelines](https://github.com/localstack/localstack-toolkit-vscode/blob/main/CONTRIBUTING.md) to learn how you can help. ### LocalStack Toolkit for VS Code extension support Please provide feedback or report an issue on the LocalStack Toolkit for VS Code by using our [GitHub Discussions board](https://github.com/orgs/localstack/discussions) page. ### LocalStack general support For LocalStack-related questions, feedback, and contributions, you can visit our [help & support page](/aws/help-support/get-help/) to learn about the different support channels and report issues on our [GitHub Discussions board](https://github.com/orgs/localstack/discussions/new/choose). # Overview > Deploy resources to LocalStack using the same Infrastructure as Code tools you already use against AWS. import SectionCards from '../../../../../components/SectionCards.astro'; If you provision AWS resources with an Infrastructure as Code (IaC) tool today, you can keep using the same templates, modules, and stacks against LocalStack. Each integration shows how to point your IaC tool at LocalStack so the resources are created locally instead of in your AWS account. # AWS CDK > Use the AWS CDK (Cloud Development Kit) with LocalStack. ![AWS CDK](public/images/aws/aws-cdk-logo.svg) ## Overview The AWS Cloud Development Kit (CDK) is an Infrastructure-as-Code (IaC) tool using general-purpose programming languages such as TypeScript/JavaScript, Python, Java, and .NET to programmatically define your cloud architecture on AWS. ## AWS CDK CLI for LocalStack [`lstk cdk`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#cdk) proxies the [AWS CDK](https://github.com/aws/aws-cdk) library against local APIs provided by LocalStack. It requires the AWS CDK CLI version `2.177.0` or newer on your `PATH`. :::note `lstk cdk` supersedes the older [`cdklocal` wrapper script](/aws/connecting/infrastructure-as-code/deprecated-wrapper-scripts#cdklocal), which is deprecated but still available if you need it. ::: ### Installation Install `lstk` by following the [`lstk` installation instructions](/aws/developer-tools/running-localstack/lstk/#installation). You'll also need the AWS CDK CLI installed separately: ```bash # Install globally npm install -g aws-cdk # Verify it installed correctly cdk --version ``` ### Usage `lstk cdk` can be used as a drop-in replacement of where you would otherwise use `cdk` when targeting the AWS Cloud. ```bash lstk cdk --help ``` ### Configuration `lstk cdk`'s only lstk-specific flag (before the CDK action) is `--region ` (default `us-east-1`); CDK always targets the default LocalStack account `000000000000`, so there is no `--account` flag. The following environment variables can be configured: - `AWS_ENDPOINT_URL`: The endpoint URL (i.e., protocol, host, and port) to connect to LocalStack (default: `http://localhost.localstack.cloud:4566`) - `AWS_ENDPOINT_URL_S3`: The S3-specific endpoint URL, required alongside `AWS_ENDPOINT_URL` and must include `.s3.` to correctly identify S3 API calls - `LSTK_CDK_CMD`: Binary to invoke (default `cdk`) - `AWS_REGION`: Fallback for `--region` ### Example Make sure that LocalStack is installed and successfully started with the required services before running the example ```bash curl http://localhost.localstack.cloud:4566/_localstack/health ``` The CDK command line ships with a sample app generator to run a quick test for getting started. ```bash # create sample app mkdir /tmp/test; cd /tmp/test lstk cdk init sample-app --language=javascript # bootstrap localstack environment lstk cdk bootstrap # deploy the sample app lstk cdk deploy > Do you wish to deploy these changes (y/n)? y ``` Once the deployment is done, you can inspect the created resources via the [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command line ```bash lstk aws sns list-topics { "Topics": [ { "TopicArn": "arn:aws:sns:us-east-1:000000000000:TestStack-TestTopic339EC197-79F43WWCCS4Z" } ] } ``` ## Current Limitations ### Updating CDK stacks Updating CDK stacks may result in deployment failures and inconsistent state within LocalStack. It is advisable to prioritize re-creating (deleting and re-deploying) over updating stacks. :::note CDK Asset deployment (e.g., Lambda code, S3 content) requires a LocalStack paid plan. This process relies on the `AWS::CloudFormation::CustomResource` API. If deployments hang or fail silently, check the LocalStack logs for `CustomResource` errors. ::: ### Stacks with validated certificates By default, stacks with validated certificates may not be deployed using the `local` lambda executor. This originates from the way how CDK ensures the certificate is ready - it creates a single-file lambda function with a single dependency on `aws-sdk` which is usually preinstalled and available globally in lambda runtime. When this lambda is executed locally from the `/tmp` folder, the package can not be discovered by Node due to the way how Node package resolution works. ## CDK Version Compatibility `lstk cdk` requires AWS CDK CLI `2.177.0` or newer, as noted in [Installation](#installation). If you're using an older `aws-cdk` version, or the legacy `cdklocal` wrapper script, see [CDK Version Compatibility](/aws/connecting/infrastructure-as-code/deprecated-wrapper-scripts#cdklocal) on the deprecated wrapper scripts page for the relevant environment-variable workarounds. # AWS Chalice > Use AWS Chalice with LocalStack. import { FileTree } from '@astrojs/starlight/components'; [AWS Chalice](https://aws.github.io/chalice/) is a serverless micro framework used to develop and deploy your serverless applications on AWS resources. Chalice provides integrated functionality with most of the AWS Toolings like S3 Storage, Simple Queue Service, API Gateway and more. It offers a handy CLI interface that allows you to easily create, develop & deploy your serverless applications. LocalStack offers an [AWS Chalice client](https://github.com/localstack/chalice-local) that allows you to interact with your Chalice applications locally. Using LocalStack, you can kick-start your development process, create a new Chalice application, and test it application locally. ## Creating a new Chalice project Start LocalStack inside a Docker container by running: ```bash lstk start ``` Install the `chalice-local` package by running: ```bash pip install chalice-local ``` You can now create a new Chalice project by running: ```bash chalice-local new-project ``` You will be prompted with an interactive menu where you can choose the name of your project and the project type. In this example, we are using `localstack-test` as the project name and `REST API` as the project type: ```sh ___ _ _ _ _ ___ ___ ___ / __|| || | /_\ | | |_ _|/ __|| __| | (__ | __ | / _ \ | |__ | || (__ | _| \___||_||_|/_/ \_\|____||___|\___||___| The python serverless microframework for AWS allows you to quickly create and deploy applications using Amazon API Gateway and AWS Lambda. Please enter the project name [?] Enter the project name: localstack-test [?] Select your project type: REST API > REST API S3 Event Handler Lambda Functions only Legacy REST API Template [CDK] Rest API with a DynamoDB table Your project has been generated in ./localstack-test ``` Let's take a look inside the project structure: - app.py - chalicelib - \_\_init\_\_.py - requirements-dev.txt - requirements.txt - tests - \_\_init\_\_.py - test_app.py The `app.py` is our main API file. It has only one Route that would assign the URL of the application to the function. The decorators here primarily "wrap" functions here which makes it easy to write Code Logic by breaking them down into separate routes. For now, our Application is serving only a JSON Message which is `{'hello': 'world'}`. ## Testing the Chalice API Just as with AWS, you can now test your API using `chalice-local local`: ```bash chalice-local local Serving on http://127.0.0.1:8000 ``` You can also do a `curl` to test the API: ```bash curl -X GET http://127.0.0.1:8000 {"hello":"world"} ``` ## Deploying the Chalice API You can use `chalice-local deploy` to deploy the REST API now: ```bash chalice-local deploy Creating deployment package. Creating IAM role: localstack-test-dev Creating lambda function: localstack-test-dev Creating Rest API Resources deployed: - Lambda ARN: arn:aws:lambda:us-east-1:000000000000:function:localstack-test-dev - Rest API URL: https://y5iuni004m.execute-api.us-east-1.amazonaws.com/api/ ``` We now have our Chalice Application deployed on a Lambda Amazon Resource Name (ARN) along with a REST API URL. # AWS SAM > Use the AWS SAM (Serverless Application Model) with LocalStack. ## Introduction The AWS Serverless Application Model (SAM) is an open-source framework for developing serverless applications. It uses a simplified syntax to define functions, APIs, databases, and event source mappings. When you deploy, SAM converts its syntax into AWS CloudFormation syntax, helping you create serverless applications more quickly. LocalStack can work with SAM using [`lstk sam`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#sam), which lets you deploy SAM applications on LocalStack. This guide explains how to set up local AWS resources using `lstk sam`. ## `lstk sam` `lstk sam` proxies the `sam` command line interface, facilitating the use of the SAM framework with LocalStack. When executing deployment commands like `lstk sam [ build | deploy | validate | package ]`, it configures the SAM settings for LocalStack and runs the specified SAM command. :::note `lstk sam` supersedes the older [`samlocal` wrapper script](/aws/connecting/infrastructure-as-code/deprecated-wrapper-scripts#samlocal), which is deprecated but still available if you need it. ::: ### Install lstk To use `lstk sam`, install `lstk` by following the [`lstk` installation instructions](/aws/developer-tools/running-localstack/lstk/#installation). You'll also need the [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html) (version `1.95.0` or newer) installed and on your `PATH`. ### Create a new SAM project You can initialize a new SAM project using the following command: ```bash lstk sam init ``` Select `1` to create a new SAM application using an AWS Quick Start template. The SAM CLI will ask you for the project name and the runtime for the Lambda function. For this example, select `1` for the Hello World example. Choose the Python runtime and `zip` for the packaging type. Optionally, you can enable X-Ray tracing, monitoring, and structured JSON logging. Then, enter the project name and press `Enter`. ### Deploy the SAM application After initializing the SAM project, enter the project directory and deploy the application using the following command: ```bash lstk sam deploy --guided ``` Enter the default values for the deployment, such as the stack name, region, and confirm the changes. `lstk sam` will package and deploy the application to LocalStack. ### Configuration The `lstk sam` command supports several options and environment variables beyond what is available with the standard `sam` command. | Option | Default | Description | |----------------------|-----------------|--------------| | `--region ` | `us-east-1` | Deployment region | | `--account ` | `000000000000` | Target AWS account id (12 digits) | `lstk sam`-specific flags must appear **before** the SAM action (for example, `lstk sam --region us-west-2 build`) | Environment Variable | Default value | Description | |-------------------------|----------------|--------------| | `AWS_ENDPOINT_URL` | - | Override the auto-resolved LocalStack endpoint | | `AWS_ENDPOINT_URL_S3` | - | Override the auto-resolved LocalStack S3 endpoint | | `LSTK_SAM_CMD` | `sam` | Binary to invoke | | `AWS_REGION` | - | Fallback for `--region` | | `AWS_ACCESS_KEY_ID` | - | Fallback for `--account` | ## Debugging on VS Code To debug your Lambda functions in VS Code while using the SAM CLI's `sam local` command alongside other services provided by LocalStack, set up a launch configuration in the `.vscode/launch.json` file. Insert the following settings into the file: ```json showshowLineNumbers { "type": "aws-sam", "request": "direct-invoke", "name": "Job dispatcher lambda", "invokeTarget": { "target": "code", "projectRoot": "${workspaceFolder}", "lambdaHandler": "lambda/lambda.handler" }, "lambda": { "runtime": "python3.8", "payload": {}, "environmentVariables": { "ENDPOINT_URL": "http://localstack:4566/", "S3_ENDPOINT_URL": "http://s3.localhost.localstack.cloud:4566/", "AWS_ACCESS_KEY_ID": "test", "AWS_SECRET_ACCESS_KEY": "test", "AWS_SESSION_TOKEN": "test", "AWS_REGION": "us-east-1", "MAIN_DOCKER_NETWORK": "localstack-network" } }, "sam": { "dockerNetwork": "localstack-network" } } ``` The `dockerNetwork` property is essential as it allows the LocalStack container to use the `sam invoke` commands within the same network as the LocalStack container itself. Adjust the Lambda function handler and environment variables as needed. # Cloud Custodian > Use Cloud Custodian with LocalStack ## Introduction Cloud Custodian is an open-source rules engine and cloud management tool designed to help organizations maintain security and compliance across their cloud environments. Cloud Custodian's YAML DSL allows definition of rules to filter and tag resources, and then apply actions to those resources. Cloud Custodian can be used to manage local AWS resources in LocalStack, resembling the live AWS environment, allowing you to test and validate your security policies locally. You can use Cloud Custodian with LocalStack by just specifying the Cloud Custodian package to use the LocalStack profile configured with your AWS CLI. ## Getting started This guide is designed for users who are new to Cloud Custodian and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how you can spin up an EC2 instance and tag it with the key `Custodian`, and then use Cloud Custodian to stop the instance. ### Install Cloud Custodian To install Cloud Custodian, run the following command: ```bash pip install c7n ``` After installing Cloud Custodian, you can configure a [custom LocalStack profile](http://docs.localstack.cloud/user-guide/integrations/aws-cli/#configuring-a-custom-profile) in your AWS CLI configuration file. ### Create an EC2 instance You can create an EC2 instance using `lstk aws`. You can use the [`RunInstances`](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html) API to create an EC2 instance. The following example creates an EC2 instance with the tag `Custodian` (any value): ```bash lstk aws ec2 run-instances \ --image-id ami-ff0fea8310f3 \ --count 1 \ --instance-type t3.nano \ --tag-specifications "ResourceType=instance,Tags=[{Key=Custodian,Value=AnyValue}]" ``` You can navigate to the LocalStack logs to verify that the EC2 instance was created successfully: ```bash 2023-10-16T15:27:35.479 INFO --- [ asgi_gw_0] l.u.container_networking : Determined main container network: bridge 2023-10-16T15:27:35.504 INFO --- [ asgi_gw_0] l.u.container_networking : Determined main container target IP: 172.17.0.2 2023-10-16T15:27:36.154 INFO --- [ asgi_gw_0] l.s.ec2.vmmanager.docker : Instance i-d87f1ab75e95ab0d2 will be accessible via SSH at: 127.0.0.1:22, 172.17.0.3:22 2023-10-16T15:27:36.155 INFO --- [ asgi_gw_0] l.s.ec2.vmmanager.docker : Instance i-d87f1ab75e95ab0d2 port mappings (container -> host): {'22/tcp': 22} 2023-10-16T15:27:36.218 INFO --- [ asgi_gw_0] localstack.request.aws : AWS ec2.RunInstances => 200 ``` ### Create a Cloud Custodian policy You can now create a Cloud Custodian policy to stop the EC2 instances with the tag `Custodian`. Create a file named `custodian.yml` and add the following content: ```yaml showshowLineNumbers policies: - name: my-first-policy resource: aws.ec2 filters: - "tag:Custodian": present actions: - stop ``` The above policy specifies the following: - `name`: The name of the policy. - `resource`: The AWS resource to apply the policy to. - `filters`: The filters to apply to the resource. In this case, the filter is `tag:Custodian` and the value is `present`. - `actions`: The actions to apply to the resource. In this case, the action is `stop`. ### Run the Cloud Custodian policy You can now run the Cloud Custodian policy using the following command: ```bash custodian run \ --output-dir=. custodian.yml \ --profile localstack ``` :::tip Alternatively, you can also set the `AWS_PROFILE=localstack` environment variable, in which case the `--profile localstack` parameter can be omitted in the commands above. ::: You should see the following output: ```sh 2023-10-16 21:06:13,853: custodian.policy:INFO policy:my-first-policy resource:aws.ec2 region:us-east-1 count:1 time:0.20 2023-10-16 21:06:13,961: custodian.policy:INFO policy:my-first-policy action:stop resources:1 execution_time:0.10 ``` You can then navigate to the LocalStack logs to verify that the EC2 instance was stopped successfully: ```bash 2023-10-16T15:36:13.583 INFO --- [ asgi_gw_0] localstack.request.aws : AWS sts.GetCallerIdentity => 200 2023-10-16T15:36:13.851 INFO --- [ asgi_gw_1] localstack.request.aws : AWS ec2.DescribeInstances => 200 2023-10-16T15:36:13.960 INFO --- [ asgi_gw_0] localstack.request.aws : AWS ec2.StopInstances => 200 ``` ### Create CloudWatch metrics for Cloud Custodian Cloud Custodian creates CloudWatch metrics for each policy. These metrics show how many resources met the filters, the time taken to gather and filter those resources, and the time required to perform actions. Certain filters and actions might produce their own metrics. To activate metric output, you must set the `metrics` flag running Cloud Custodian. ```bash custodian run -s . \ --metrics aws custodian.yml \ --profile localstack ``` You can access the CloudWatch metrics in the [LocalStack Web Application](https://app.localstack.cloud/inst/default/resources/cloudwatch). ## Further reading - [Example Policies](https://cloudcustodian.io/docs/aws/examples/index.html) for practical examples of specific policies for individual AWS modules. - [Cloud Custodian documentation](https://cloudcustodian.io/docs/quickstart/index.html) to learn more about Cloud Custodian. - [Integrations](https://cloudcustodian.io/docs/aws/topics/index.html) with Config, Security Hub, Systems Manager, and X-Ray. - [Monitoring the environment](https://cloudcustodian.io/docs/aws/usage.html) by generating CloudWatch metrics. # Crossplane > Use the Crossplane cloud-native control plane framework with LocalStack. ## Overview [Crossplane](https://www.crossplane.io) is a cloud-native control plane framework, which offers an extensible backend that enables orchestrating applications and infrastructure via declarative APIs and resource definitions. Crossplane offers a native [AWS provider](https://github.com/upbound/provider-aws) which can be used to create and manage AWS cloud resources via the Crossplane platform. For example, it can be used to create S3 buckets, SQS queues, Lambda functions, among many other resources. Crossplane AWS provider supports a comprehensive set of some [900+ resource types](https://marketplace.upbound.io/providers/upbound/provider-aws). ## Getting started In the following, we provide a step-by-step guide for installing Crossplane in a local test environment, and creating AWS resources (S3 bucket, SQS queue) in LocalStack via Crossplane. ### Prerequisites - LocalStack running in local Docker - A local Kubernetes cluster: - We can use the [embedded Kubernetes cluster](https://docs.docker.com/desktop/kubernetes) that ships with modern versions of Docker Desktop (can be easily enabled in the Docker settings) - Alternatively, you can [create a local EKS cluster](/aws/services/eks/#create-an-embedded-kubernetes-cluster) in LocalStack directly, which will spin up a light-weight embedded `k3d` Kubernetes cluster in your Docker environment - The [`helm`](https://helm.sh) and [`kubectl`](https://kubernetes.io/docs/tasks/tools/#kubectl) command-line clients installed ## Installing Crossplane in local Kubernetes Once your `kubectl` is configured to point to the local Kubernetes cluster, we first install Crossplane via `helm`: ```bash helm repo add crossplane-stable https://charts.crossplane.io/stable helm repo update helm install crossplane crossplane-stable/crossplane --namespace crossplane-system --create-namespace ``` The installation may take a few minutes. In parallel, we can install the `crossplane` command-line tool. ```bash curl -sL https://raw.githubusercontent.com/crossplane/crossplane/master/install.sh | bash sudo mv crossplane /usr/local/bin ``` To confirm that the installation was successful, we can run these commands, which should yield output similar to the following: ```bash crossplane version ``` ```bash title="Output" Client Version: v1.17.0 Server Version: v1.17.0 ``` ```bash kubectl get crds | grep crossplane ``` ```bash title="Output" compositions.apiextensions.crossplane.io 2023-09-03T11:30:36Z configurations.pkg.crossplane.io 2023-09-03T11:30:36Z ``` ### Installing the Crossplane AWS Provider Once the basic Crossplane installation is running properly, we can proceed with installing the AWS provider. Newer versions of Crossplane promote the use of [provider families](https://docs.upbound.io/providers/provider-families), which are collections of providers for different groups of resources. For example, there is a separate provider for each individual AWS service (like S3, SQS, Lambda, etc), and in addition provider family provides shared resources for common configuration of all services (e.g., credentials, etc). In the following, we first install the AWS provider for S3. Note that you can copy/paste the entire multi-line command below into your terminal: ```bash cat < Reference for the legacy awslocal, tflocal, samlocal, and cdklocal wrapper scripts, superseded by lstk. ## Introduction The following (now deprecated) wrapper scripts were used in conjunction with LocalStack, to direct the `aws`, `terraform`, `sam`, and `cdk` commands to LocalStack endpoints. - [`awslocal`](#awslocal) - Runs AWS CLI commands against LocalStack. - [`tflocal`](#tflocal) - Runs Terraform against LocalStack. - [`samlocal`](#samlocal) - Runs AWS SAM CLI commands against LocalStack. - [`cdklocal`](#cdklocal) - Runs AWS CDK commands against LocalStack. :::caution `awslocal`, `tflocal`, `samlocal`, and `cdklocal` are **deprecated** in favor of [`lstk`](/aws/developer-tools/running-localstack/lstk/), which provides equivalent functionality via `lstk aws`, `lstk terraform` (alias `lstk tf`), `lstk sam`, and `lstk cdk`. These wrapper scripts remain available and continue to work. This page documents them for anyone still relying on them. ::: See the [AWS CLI](/aws/connecting/aws-cli), [Terraform](/aws/connecting/infrastructure-as-code/terraform), [AWS SAM](/aws/connecting/infrastructure-as-code/aws-sam), and [AWS CDK](/aws/connecting/infrastructure-as-code/aws-cdk) pages for the current `lstk`-based workflow. ## awslocal `awslocal` serves as a thin wrapper and a substitute for the standard `aws` command, enabling you to run AWS CLI commands within the LocalStack environment without specifying the `--endpoint-url` parameter or a profile. ### Installation Install the `awslocal` command using the following command: ```bash pip install awscli-local[ver1] ``` :::tip The above command installs the most recent version of the underlying AWS CLI version 1 (`awscli`) package. If you would rather manage your own `awscli` version (e.g., `v1` or `v2`) and only install the wrapper script, you can use the following command: ```bash pip install awscli-local ``` ::: :::note Automatic installation of AWS CLI version 2 is not supported (at the time of writing there is no official pypi package for `v2` available), but the `awslocal` technically also works with AWS CLI v2 (see [Current Limitations](#current-limitations) for more details). ::: ### Usage The `awslocal` command shares identical usage with the standard `aws` command. For comprehensive usage instructions, refer to the manual pages by running `awslocal help`. ```bash awslocal kinesis list-streams ``` ### Configuration | Variable Name | Description | | ---------------- | ----------------------------------------------------------------------------------- | | AWS_ENDPOINT_URL | The endpoint URL to connect to (takes precedence over USE_SSL/LOCALSTACK_HOST) | | LOCALSTACK_HOST | A variable defining where to find LocalStack (default: `localhost:4566`) | | USE_SSL | Whether to use SSL when connecting to LocalStack (default: False) | ### Current Limitations Please note that there is a known limitation for using the `cloudformation package ...` command with the AWS CLI v2. The problem is that the AWS CLI v2 is [not available as a package on pypi.org](https://github.com/aws/aws-cli/issues/4947), but is instead shipped as a binary package that cannot be easily patched from `awslocal`. To work around this issue, you have 2 options: - Downgrade to the v1 AWS CLI (this is the recommended approach) - There is an unofficial way to install AWS CLI v2 from sources. We do not recommend this, but it is technically possible. Also, you should install these libraries in a Python virtualenv, to avoid version clashes with other libraries on your system: ```bash virtualenv .venv . .venv/bin/activate pip install https://github.com/boto/botocore/archive/v2.zip https://github.com/aws/aws-cli/archive/v2.zip ``` Please also note there is a known limitation for issuing requests using `--no-sign-request` with the AWS CLI. LocalStack's routing mechanism depends on the signature of each request to identify the correct service for the request. Thus, adding the flag `--no-sign-requests` provokes your request to reach the wrong service. One possible way to address this is to use the `awslocal` CLI instead of AWS CLI. ## tflocal `tflocal` is a small wrapper script to run Terraform against LocalStack. It uses the [Terraform Override mechanism](https://www.terraform.io/language/files/override) and creates a temporary file `localstack_providers_override.tf` to configure the endpoints for the AWS `provider` section. The endpoints for all services are configured to point to the LocalStack API (`http://localhost:4566` by default). ### Installation To install the `tflocal` command, you can use `pip` (assuming you have a local Python installation): ```bash pip install terraform-local ``` After installation, you can use the `tflocal` command, which has the same interface as the `terraform` command line. ```bash tflocal --help ``` ### Usage ```bash tflocal init tflocal apply ``` ### Configuration | Environment Variable | Default value | Description | | ------------------------ | -------------------------------- | ----------- | | `TF_CMD` | `terraform` | Terraform command to call | | `AWS_ENDPOINT_URL` | - | Hostname and port of the target LocalStack instance | | `LOCALSTACK_HOSTNAME` | `localhost` | Host name of the target LocalStack instance | | `EDGE_PORT` | `4566` | Port number of the target LocalStack instance | | `S3_HOSTNAME` | `s3.localhost.localstack.cloud` | Special hostname to be used to connect to LocalStack S3 | | `USE_EXEC` | - | Whether to use `os.exec` instead of `subprocess.Popen` (try using this in case of I/O issues) | | `_ENDPOINT` | - | Setting a custom service endpoint, e.g., `COGNITO_IDP_ENDPOINT=http://example.com` | | `AWS_DEFAULT_REGION` | `us-east-1` | The AWS region to use (determined from local credentials if `boto3` is installed) | | `CUSTOMIZE_ACCESS_KEY` | - | Enables you to override the static AWS Access Key ID | | `AWS_ACCESS_KEY_ID` | `test` (`accountId`: 000000000000) | AWS Access Key ID to use for multi-account setups | :::note While using `CUSTOMIZE_ACCESS_KEY`, following cases are taking precedence over each other from top to bottom: 1. If the `AWS_ACCESS_KEY_ID` environment variable is set. 2. If `access_key` is configured in the Terraform AWS provider. 3. If the `AWS_PROFILE` environment variable is set and properly configured. 4. If the `AWS_DEFAULT_PROFILE` environment variable is set and configured. 5. If credentials for the `default` profile are configured. 6. If none of the above settings are present, it falls back to using the default `AWS_ACCESS_KEY_ID` mock value. ::: ### OpenTofu You can use the `TF_CMD` environment variable with `tflocal` to specify the `tofu` binary to call: ```bash TF_CMD=tofu tflocal --help ``` ## samlocal `samlocal` is a wrapper for the `sam` command line interface, facilitating the use of the SAM framework with LocalStack. When executing deployment commands like `samlocal [ build | deploy | validate | package ]`, the script configures the SAM settings for LocalStack and runs the specified SAM command. :::note `samlocal` supports image/container-based Lambda (ECR) deploys and nested CloudFormation stacks, which `lstk sam` does not (yet) support - this is the main reason to keep using `samlocal` today. ::: ### Installation You can install the `samlocal` wrapper script by running the following command: ```bash pip install aws-sam-cli-local ``` ### Usage ```bash samlocal init samlocal deploy --guided ``` ### Configuration | Environment Variable | Default value | Description | |------------------------|--------------------------------------------------|-------------------------------------------------------------------------| | AWS_ENDPOINT_URL | `http://localhost.localstack.cloud:4566` | URL at which the `boto3` client can reach LocalStack | | EDGE_PORT | `4566` | Port number under which the LocalStack edge service is available | | LOCALSTACK_HOSTNAME | `localhost` | Host under which the LocalStack edge service is available ## cdklocal `cdklocal` is a thin wrapper script for using the [AWS CDK](https://github.com/aws/aws-cdk) library against local APIs provided by LocalStack. ### Installation The `cdklocal` command line is published as an [npm library](https://www.npmjs.com/package/aws-cdk-local): ```bash # Install globally npm install -g aws-cdk-local aws-cdk # Verify it installed correctly cdklocal --version # e.g. 1.65.5 ``` :::note Using `cdklocal` locally (e.g. within the `node_modules` of your repo instead of globally installed) does not work at the moment for some setups, so make sure you install both `aws-cdk` and `aws-cdk-local` with the `-G` flag. ::: ### Usage `cdklocal` can be used as a drop-in replacement of where you would otherwise use `cdk` when targeting the AWS Cloud. ```bash cdklocal --help # create sample app mkdir /tmp/test; cd /tmp/test cdklocal init sample-app --language=javascript # bootstrap localstack environment cdklocal bootstrap # deploy the sample app cdklocal deploy ``` ### Configuration The following environment variables can be configured: - `AWS_ENDPOINT_URL`: The endpoint URL (i.e., protocol, host, and port) to connect to LocalStack (default: `http://localhost.localstack.cloud:4566`) - `LAMBDA_MOUNT_CODE`: Whether to use local Lambda code mounting (via setting `hot-reload` S3 bucket name) ### CDK Version Compatibility `cdklocal` works with all installed versions of the Node.js `aws-cdk` package. However, issues exist for `aws-cdk >= 2.177.0`. For these versions: - We unset AWS-related environment variables like `AWS_PROFILE` before calling `cdk`. - We explicitly set `AWS_ENDPOINT_URL` and `AWS_ENDPOINT_URL_S3` to point to LocalStack. Some environment variables may cause conflicting config, such as wrong region or accidental deploys to real AWS. To allow specific variables (e.g., `AWS_REGION`), use `AWS_ENVAR_ALLOWLIST`: ```bash AWS_ENVAR_ALLOWLIST=AWS_REGION,AWS_DEFAULT_REGION AWS_DEFAULT_REGION=eu-central-1 AWS_REGION=eu-central-1 cdklocal ... ``` If you manually set `AWS_ENDPOINT_URL`, it will be used. You must also set `AWS_ENDPOINT_URL_S3`, and it must include `.s3.` to correctly identify S3 API calls. See full configuration details [on our configuration docs](https://github.com/localstack/aws-cdk-local?tab=readme-ov-file#configurations). # Former2 > Use Former2 to generate Infrastructure-as-Code outputs from existing resources with LocalStack. ## Introduction [Former2](https://github.com/iann0036/former2) allows you to generate Infrastructure-as-Code (IaC) outputs using your pre-existing AWS resources. It uses the AWS JavaScript SDK to make relevant API calls, scans your infrastructure, and provides you with a resource list. You can then select the resources for which you want to generate IaC outputs. Former2 currently supports the following outputs: - [CloudFormation](https://aws.amazon.com/cloudformation/) - [Terraform](https://www.terraform.io/) - [Troposphere](https://github.com/cloudtools/troposphere) - [CDK V1 (Cfn Primitives) & CDK V2 (Cfn Primitives)](https://docs.aws.amazon.com/cdk/latest/guide/getting_started.html) (TypeScript, Python, Java, C#) - [CDK for Terraform](https://developer.hashicorp.com/terraform/cdktf) (TypeScript) - [Pulumi](https://www.pulumi.com/docs/get-started/aws/) (TypeScript) - [Diagrams](https://diagrams.mingrammer.com/) With Former2, you can scan the resources within your LocalStack instance and produce Infrastructure-as-Code (IaC) outputs. These outputs enable you to redeploy your resources while spinning a new LocalStack instance or deploy them to a live Amazon Web Services (AWS) environment. ## Getting started This guide is designed for users new to Former2 and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. We will demonstrate how you can create local AWS resources using LocalStack, and import a CloudFormation output via Former2. ### Install Former2 You can use the publicly hosted [Former2 Web Application](https://former2.com/) or a [self-hosted version](https://github.com/iann0036/former2/blob/master/HOSTING.md) to generate IaC outputs. For this guide, we will use the publicly hosted version. You would also need a Former2 Helper extension/add-on for your preferred web browser: - [Google Chrome](https://chrome.google.com/webstore/detail/former2-helper/fhejmeojlbhfhjndnkkleooeejklmigi) - [Mozilla Firefox](https://addons.mozilla.org/en-US/firefox/addon/former2-helper/) - [Microsoft Edge](https://microsoftedge.microsoft.com/addons/detail/okkjnfohglnomdbpimkcdkiojbeiedof) Alternatively, you can [download and install](https://github.com/iann0036/former2-helper) the extension yourself. ### Create local resources Start your LocalStack container using your preferred method with the following environment variables, depending on the browser you are using: - **Google Chrome**: `EXTRA_CORS_ALLOWED_ORIGINS=chrome-extension://fhejmeojlbhfhjndnkkleooeejklmigi` - **Mozilla Firefox**: `EXTRA_CORS_ALLOWED_ORIGINS=moz-extension://853c673f-1bd8-4226-a5ff-f1473f7b3d90` - **Microsoft Edge**: `EXTRA_CORS_ALLOWED_ORIGINS=extension://okkjnfohglnomdbpimkcdkiojbeiedof` You can create local AWS resources using the AWS CLI and `lstk aws`. For example, you can create a new S3 bucket, SQS queue, and DynamoDB table using the following commands: ```bash lstk aws s3 mb s3://my-bucket lstk aws sqs create-queue --queue-name my-queue lstk aws dynamodb create-table \ --table-name my-table \ --attribute-definitions AttributeName=id,AttributeType=S \ --key-schema AttributeName=id,KeyType=HASH \ --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5 ``` You can verify that the resources were created successfully by running the following command: ```bash lstk logs ``` ```bash title="Output" emulator | 2023-10-14T15:31:08.852 INFO --- [ asgi_gw_0] localstack.request.aws : AWS s3.CreateBucket => 200 emulator | 2023-10-14T15:31:09.356 INFO --- [ asgi_gw_0] localstack.request.aws : AWS sqs.CreateQueue => 200 emulator | 2023-10-14T15:31:12.920 INFO --- [ asgi_gw_0] botocore.credentials : Found credentials in environment variables. emulator | 2023-10-14T15:31:13.332 INFO --- [ asgi_gw_0] localstack.utils.bootstrap : Execution of "require" took 2028.25ms emulator | 2023-10-14T15:31:13.712 INFO --- [ asgi_gw_0] localstack.request.aws : AWS dynamodb.CreateTable => 200 ``` ```bash lstk aws s3 ls ``` ```bash title="Output" 2023-10-14 21:01:08 my-bucket ``` ```bash lstk aws sqs list-queues ``` ```bash title="Output" { "QueueUrls": [ "http://localhost.localstack.cloud:4566/000000000000/my-queue" ] } ``` ```bash lstk aws dynamodb list-tables ``` ```bash title="Output" { "TableNames": [ "my-table" ] } ``` ### Configure Former2 Navigate to the Former2 setup dashboard. Open the [**Credentials**](https://former2.com/#section-setup-credentials) tab and enter your IAM credentials. For LocalStack, you can just configure the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables as `test` and `test`, respectively. ![Enter test credentials on Former2 Dashboard](/images/aws/former2-credentials.png) Click on [**Continue to Parameters**](https://former2.com/#section-setup-parameters) and include your own CloudFormation stack parameters by adding them below. Click on [**Continue to Settings**](https://former2.com/#section-setup-settings) and navigate to **Custom Endpoints**. Toggle the **Use LocalStack Endpoint** switch to enable the LocalStack endpoint URL (`http://localhost:4566`). Click on [**Go to Dashboard**](https://former2.com/#section-dashboard) to complete the setup. ![LocalStack endpoint toggle on Former2 Dashboard](/images/aws/former2-localstack-endpoint.png) You can now click on **Scan Account** button on the top-right corner of the dashboard to scan your LocalStack instance for resources. Once the scan is complete, you can select the resources you want to generate IaC outputs for. ### Generate IaC output Navigate to [S3](https://former2.com/#section-storage-s3), [DynamoDB](https://former2.com/#section-database-dynamodb), and [SQS](https://former2.com/#section-applicationintegration-sqs) to verify that the resources you created earlier are listed. ![S3 Console on Former2 Dashboard](/images/aws/former2-s3.png) You can select the resources you want to generate IaC outputs for and click on **Add Selected**. Finally, you can click on **Generate** on the top-left corner of the dashboard to generate the IaC outputs. ![CloudFormation Output on Former2 Dashboard](/images/aws/former2-cloudformation-output.png) You can also choose to generate the IaC outputs in a different format by clicking on the various options available on the left-hand side of the dashboard. # Pulumi > Use the Pulumi Infrastructure as Code framework with LocalStack. import { FileTree } from "@astrojs/starlight/components"; ## Introduction Pulumi's SDK for infrastructure-as-code allows you to create, deploy, and manage AWS containers, serverless functions, and other infrastructure using popular programming languages. It supports a range of cloud providers, including AWS, Azure, Google Cloud, and Kubernetes. LocalStack can integrate with Pulumi through the Pulumi configuration environment. There are two main methods to configure Pulumi for use with LocalStack: - Using the `pulumilocal` wrapper script which automatically configures service endpoints. - Manually setting up the service endpoints in your Pulumi configuration, which requires ongoing maintenance. This guide will show you how to set up local AWS resources using both the `pulumilocal` wrapper and manual configuration. ## `pulumilocal` wrapper script :::note `pulumi-local` currently does not support the `aws-native` package as it relies on the AWS Cloud Control API. ::: `pulumilocal` is a wrapper for the `pulumi` command line interface, facilitating the use of Pulumi with LocalStack. When executing deployment commands like `pulumilocal ["up", "destroy", "preview", "cancel"]`, the script configures the Pulumi settings for LocalStack and runs the specified Pulumi command. The endpoints are set to point to the LocalStack API (`http://localhost:4566`). This setup simplifies the deployment of Pulumi stacks against LocalStack. ### Configure the Local Backend Optionally, you can set environment variables to store state locally, avoiding cloud storage. ```bash export PULUMI_CONFIG_PASSPHRASE=lsdevtest export PULUMI_BACKEND_URL=file://`pwd`/myproj ``` :::note For further options please consult the official documentation on available [environment variables](https://www.pulumi.com/docs/cli/environment-variables/) and [local backend](https://www.pulumi.com/docs/concepts/state/#local-filesystem). ::: ### Install the `pulumilocal` wrapper script You can install the `pulumilocal` wrapper script by running the following command: ```bash pip install pulumi-local ``` You can now use the `pulumilocal` command to interact with your Pulumi project. ```bash pulumilocal --help ``` ```bash title="Output" Pulumi - Modern Infrastructure as Code ... ``` ### Create a new Pulumi project To start a new project, use these commands: ```bash mkdir myproj pulumilocal new aws-typescript -y -s lsdev --cwd myproj ``` :::tip The `--cwd` option is unnecessary if you're already in the project directory. ::: ### Deploy the Pulumi stack Create and select the `lsdev` stack with: ```bash pulumilocal stack select -c lsdev --cwd myproj ``` If you've just run the `new typescript` command, the stack is already selected. Deploy it with: ```bash pulumilocal up --cwd myproj ``` ### Configuration | Environment Variable | Default value | Description | |-----------------------|-------------------|---------------------------------------------------------------| | `AWS_ENDPOINT_URL` | - | Target LocalStack instance hostname and port | | `LOCALSTACK_HOSTNAME` | `localhost` | *(Deprecated)* Target host for connecting to LocalStack | | `EDGE_PORT` | `4566` | *(Deprecated)* Target port for connecting to LocalStack | | `PULUMI_CMD` | `pulumi` | Name of the executable Pulumi command on the system PATH | ## Manual configuration Alternatively, you can manually configure local service endpoints and credentials. The following section will provide detailed steps for this manual configuration, assuming you have [Pulumi](https://www.pulumi.com/docs/install/) installed. ### Create a new Pulumi stack Start a new project with: ```bash mkdir quickstart && cd quickstart pulumi new aws-typescript ``` We use the default configuration values: ```plaintext This command will walk you through creating a new Pulumi project. Enter a value or leave blank to accept the (default), and press . Press ^C at any time to quit. project name: (quickstart) project description: (A minimal AWS TypeScript Pulumi program) Created project 'quickstart' Please enter your desired stack name. To create a stack in an organization, use the format / (e.g. `acmecorp/dev`). stack name: (dev) Created stack 'dev' aws:region: The AWS region to deploy into: (us-east-1) Saved config Installing dependencies... ``` This will create the following directory structure. - index.ts - node_modules - package.json - package-lock.json - Pulumi.dev.yaml - Pulumi.yaml - tsconfig.json ### Configure the stack Modify your stack configuration in `Pulumi.dev.yaml` to include endpoints for AWS services pointing to `http://localhost:4566`. However, these endpoints may change depending on the AWS plugin version you are using. ```yaml showshowLineNumbers config: aws:accessKey: test aws:s3UsePathStyle: "true" aws:secretKey: test aws:skipCredentialsValidation: "true" aws:skipRequestingAccountId: "true" aws:endpoints: - accessanalyzer: http://localhost:4566 - account: http://localhost:4566 - acm: http://localhost:4566 - acmpca: http://localhost:4566 - amg: http://localhost:4566 - amp: http://localhost:4566 - amplify: http://localhost:4566 - apigateway: http://localhost:4566 - apigatewayv2: http://localhost:4566 - appautoscaling: http://localhost:4566 - appconfig: http://localhost:4566 - appfabric: http://localhost:4566 - appflow: http://localhost:4566 - appintegrations: http://localhost:4566 - appintegrationsservice: http://localhost:4566 - applicationautoscaling: http://localhost:4566 - applicationinsights: http://localhost:4566 - appmesh: http://localhost:4566 - appregistry: http://localhost:4566 - apprunner: http://localhost:4566 - appstream: http://localhost:4566 - appsync: http://localhost:4566 - athena: http://localhost:4566 - auditmanager: http://localhost:4566 - autoscaling: http://localhost:4566 - autoscalingplans: http://localhost:4566 - backup: http://localhost:4566 - batch: http://localhost:4566 - beanstalk: http://localhost:4566 - bedrock: http://localhost:4566 - bedrockagent: http://localhost:4566 - budgets: http://localhost:4566 - ce: http://localhost:4566 - chime: http://localhost:4566 - chimesdkmediapipelines: http://localhost:4566 - chimesdkvoice: http://localhost:4566 - cleanrooms: http://localhost:4566 - cloud9: http://localhost:4566 - cloudcontrol: http://localhost:4566 - cloudcontrolapi: http://localhost:4566 - cloudformation: http://localhost:4566 - cloudfront: http://localhost:4566 - cloudfrontkeyvaluestore: http://localhost:4566 - cloudhsm: http://localhost:4566 - cloudhsmv2: http://localhost:4566 - cloudsearch: http://localhost:4566 - cloudtrail: http://localhost:4566 - cloudwatch: http://localhost:4566 - cloudwatchevents: http://localhost:4566 - cloudwatchevidently: http://localhost:4566 - cloudwatchlog: http://localhost:4566 - cloudwatchlogs: http://localhost:4566 - cloudwatchobservabilityaccessmanager: http://localhost:4566 - cloudwatchrum: http://localhost:4566 - codeartifact: http://localhost:4566 - codebuild: http://localhost:4566 - codecatalyst: http://localhost:4566 - codecommit: http://localhost:4566 - codedeploy: http://localhost:4566 - codeguruprofiler: http://localhost:4566 - codegurureviewer: http://localhost:4566 - codepipeline: http://localhost:4566 - codestarconnections: http://localhost:4566 - codestarnotifications: http://localhost:4566 - cognitoidentity: http://localhost:4566 - cognitoidentityprovider: http://localhost:4566 - cognitoidp: http://localhost:4566 - comprehend: http://localhost:4566 - computeoptimizer: http://localhost:4566 - config: http://localhost:4566 - configservice: http://localhost:4566 - connect: http://localhost:4566 - connectcases: http://localhost:4566 - controltower: http://localhost:4566 - costandusagereportservice: http://localhost:4566 - costexplorer: http://localhost:4566 - costoptimizationhub: http://localhost:4566 - cur: http://localhost:4566 - customerprofiles: http://localhost:4566 - databasemigration: http://localhost:4566 - databasemigrationservice: http://localhost:4566 - dataexchange: http://localhost:4566 - datapipeline: http://localhost:4566 - datasync: http://localhost:4566 - dax: http://localhost:4566 - deploy: http://localhost:4566 - detective: http://localhost:4566 - devicefarm: http://localhost:4566 - directconnect: http://localhost:4566 - directoryservice: http://localhost:4566 - dlm: http://localhost:4566 - dms: http://localhost:4566 - docdb: http://localhost:4566 - docdbelastic: http://localhost:4566 - ds: http://localhost:4566 - dynamodb: http://localhost:4566 - ec2: http://localhost:4566 - ecr: http://localhost:4566 - ecrpublic: http://localhost:4566 - ecs: http://localhost:4566 - efs: http://localhost:4566 - eks: http://localhost:4566 - elasticache: http://localhost:4566 - elasticbeanstalk: http://localhost:4566 - elasticloadbalancing: http://localhost:4566 - elasticloadbalancingv2: http://localhost:4566 - elasticsearch: http://localhost:4566 - elasticsearchservice: http://localhost:4566 - elb: http://localhost:4566 - elbv2: http://localhost:4566 - emr: http://localhost:4566 - emrcontainers: http://localhost:4566 - emrserverless: http://localhost:4566 - es: http://localhost:4566 - eventbridge: http://localhost:4566 - events: http://localhost:4566 - evidently: http://localhost:4566 - finspace: http://localhost:4566 - firehose: http://localhost:4566 - fis: http://localhost:4566 - fms: http://localhost:4566 - fsx: http://localhost:4566 - gamelift: http://localhost:4566 - glacier: http://localhost:4566 - globalaccelerator: http://localhost:4566 - glue: http://localhost:4566 - grafana: http://localhost:4566 - greengrass: http://localhost:4566 - groundstation: http://localhost:4566 - guardduty: http://localhost:4566 - healthlake: http://localhost:4566 - iam: http://localhost:4566 - identitystore: http://localhost:4566 - imagebuilder: http://localhost:4566 - inspector: http://localhost:4566 - inspector2: http://localhost:4566 - inspectorv2: http://localhost:4566 - internetmonitor: http://localhost:4566 - iot: http://localhost:4566 - iotevents: http://localhost:4566 - ivs: http://localhost:4566 - ivschat: http://localhost:4566 - kafka: http://localhost:4566 - kafkaconnect: http://localhost:4566 - kendra: http://localhost:4566 - keyspaces: http://localhost:4566 - kinesis: http://localhost:4566 - kinesisanalyticsv2: http://localhost:4566 - kinesisvideo: http://localhost:4566 - kms: http://localhost:4566 - lakeformation: http://localhost:4566 - lambda: http://localhost:4566 - launchwizard: http://localhost:4566 - lex: http://localhost:4566 - lexmodelbuilding: http://localhost:4566 - lexmodelbuildingservice: http://localhost:4566 - lexmodels: http://localhost:4566 - lexmodelsv2: http://localhost:4566 - lexv2models: http://localhost:4566 - licensemanager: http://localhost:4566 - lightsail: http://localhost:4566 - location: http://localhost:4566 - locationservice: http://localhost:4566 - logs: http://localhost:4566 - lookoutmetrics: http://localhost:4566 - m2: http://localhost:4566 - macie2: http://localhost:4566 - managedgrafana: http://localhost:4566 - mediaconnect: http://localhost:4566 - mediaconvert: http://localhost:4566 - medialive: http://localhost:4566 - mediapackage: http://localhost:4566 - mediapackagev2: http://localhost:4566 - memorydb: http://localhost:4566 - mq: http://localhost:4566 - msk: http://localhost:4566 - mwaa: http://localhost:4566 - neptune: http://localhost:4566 - networkfirewall: http://localhost:4566 - networkmanager: http://localhost:4566 - oam: http://localhost:4566 - opensearch: http://localhost:4566 - opensearchingestion: http://localhost:4566 - opensearchserverless: http://localhost:4566 - opensearchservice: http://localhost:4566 - opsworks: http://localhost:4566 - organizations: http://localhost:4566 - osis: http://localhost:4566 - outposts: http://localhost:4566 - pcaconnectorad: http://localhost:4566 - pinpoint: http://localhost:4566 - pipes: http://localhost:4566 - polly: http://localhost:4566 - pricing: http://localhost:4566 - prometheus: http://localhost:4566 - prometheusservice: http://localhost:4566 - qbusiness: http://localhost:4566 - quicksight: http://localhost:4566 - ram: http://localhost:4566 - rbin: http://localhost:4566 - rds: http://localhost:4566 - recyclebin: http://localhost:4566 - redshift: http://localhost:4566 - redshiftdata: http://localhost:4566 - redshiftdataapiservice: http://localhost:4566 - redshiftserverless: http://localhost:4566 - rekognition: http://localhost:4566 - resourceexplorer2: http://localhost:4566 - resourcegroups: http://localhost:4566 - resourcegroupstagging: http://localhost:4566 - resourcegroupstaggingapi: http://localhost:4566 - rolesanywhere: http://localhost:4566 - route53: http://localhost:4566 - route53domains: http://localhost:4566 - route53recoverycontrolconfig: http://localhost:4566 - route53recoveryreadiness: http://localhost:4566 - route53resolver: http://localhost:4566 - rum: http://localhost:4566 - s3: http://localhost:4566 - s3api: http://localhost:4566 - s3control: http://localhost:4566 - s3outposts: http://localhost:4566 - sagemaker: http://localhost:4566 - scheduler: http://localhost:4566 - schemas: http://localhost:4566 - sdb: http://localhost:4566 - secretsmanager: http://localhost:4566 - securityhub: http://localhost:4566 - securitylake: http://localhost:4566 - serverlessapplicationrepository: http://localhost:4566 - serverlessapprepo: http://localhost:4566 - serverlessrepo: http://localhost:4566 - servicecatalog: http://localhost:4566 - servicecatalogappregistry: http://localhost:4566 - servicediscovery: http://localhost:4566 - servicequotas: http://localhost:4566 - ses: http://localhost:4566 - sesv2: http://localhost:4566 - sfn: http://localhost:4566 - shield: http://localhost:4566 - signer: http://localhost:4566 - simpledb: http://localhost:4566 - sns: http://localhost:4566 - sqs: http://localhost:4566 - ssm: http://localhost:4566 - ssmcontacts: http://localhost:4566 - ssmincidents: http://localhost:4566 - ssmsap: http://localhost:4566 - sso: http://localhost:4566 - ssoadmin: http://localhost:4566 - stepfunctions: http://localhost:4566 - storagegateway: http://localhost:4566 - sts: http://localhost:4566 - swf: http://localhost:4566 - synthetics: http://localhost:4566 - timestreamwrite: http://localhost:4566 - transcribe: http://localhost:4566 - transcribeservice: http://localhost:4566 - transfer: http://localhost:4566 - verifiedpermissions: http://localhost:4566 - vpclattice: http://localhost:4566 - waf: http://localhost:4566 - wafregional: http://localhost:4566 - wafv2: http://localhost:4566 - wellarchitected: http://localhost:4566 - worklink: http://localhost:4566 - workspaces: http://localhost:4566 - xray: http://localhost:4566 ``` ### Deploy the stack to LocalStack To deploy, ensure the S3 service is included in your configuration, start LocalStack, and run: ```bash pulumi up ``` After the update, check the S3 buckets with: ```bash lstk aws s3 ls ``` You should see output similar to: ```bash 2021-09-30 11:50:59 my-bucket-6c21027 ``` # Serverless Framework > Use the Serverless Framework with LocalStack. ## Overview This guide explains how to integrate LocalStack with the [Serverless Framework](https://www.serverless.com/). Although it probably requires a few code changes, integrating LocalStack with the Serverless Framework is fairly straightforward. In particular, the setup consists of the following two steps. 1. Installing and configuring the [Serverless-LocalStack plugin](https://github.com/localstack/serverless-localstack). 2. Adjusting AWS endpoints in Lambda functions. ## Prerequisites This guide assumes that you have the following tools installed. - The `lstk` CLI ([Install](/aws/developer-tools/running-localstack/lstk/#installation)) - Serverless ([Install](https://www.serverless.com/framework/docs/getting-started/)) It also assumes that you already have a Serverless app set up consisting of a couple of Lambda functions and a `serverless.yml` file similar to the following. An example Serverless app integrated with LocalStack can be found here: Simple REST API using the Serverless Framework and LocalStack ```yaml showshowLineNumbers service: my-service frameworkVersion: ">=1.1.0 <=2.50.0" provider: name: aws runtime: python3.8 environment: DYNAMODB_TABLE: ${self:service}-${opt:stage, self:provider.stage} iamRoleStatements: - Effect: Allow Action: - dynamodb:Query - ... Resource: "arn:aws:dynamodb:${opt:region, self:provider.region}:*:table/${self:provider.environment.DYNAMODB_TABLE}" functions: create: handler: todos/create.create events: - http: path: todos method: post cors: true ... resources: Resources: TodosDynamoDbTable: Type: 'AWS::DynamoDB::Table' DeletionPolicy: Retain Properties: ... TableName: ${self:provider.environment.DYNAMODB_TABLE} ``` ## Install and configure Serverless-LocalStack Plugin To install the plugin, execute the following command in the root of your project. ```bash npm install -D serverless-localstack ``` Next, set up the plugin by adding the following properties to `serverless.yml`. ```yaml ... plugins: - serverless-localstack custom: localstack: stages: - local ``` This sets up Serverless to use the LocalStack plugin but only for the stage "local". Next, you need make minor adjustments to your function code in order to make your application work no matter if it is deployed on AWS or LocalStack. ## Adjust AWS endpoints in Lambda functions You are likely using an AWS SDK (such as [Boto3](https://github.com/boto/boto3) for Python) in your Lambda functions to interact with other AWS services such as DynamoDB. For example, in Python, your code to set up a connection to DynamoDB may look like this: ```python ... dynamodb = boto3.resource('dynamodb') ... ``` By default, this call attempts to create a connection via the usual AWS endpoints. However, when running services in LocalStack, we need to make sure, our applications creates a connection via the LocalStack endpoint instead. Usually, all of LocalStack's services are available via a specific port on localhost (e.g. `localhost.localstack.cloud:4566`). However, this endpoint only works when accessing LocalStack from outside its Docker runtime. Since the Lambda functions execute within the LocalStack Docker container, Lambda functions cannot access other services via the usual localhost endpoint. Instead, LocalStack provides a special environment variable `AWS_ENDPOINT_URL` which contains the internal endpoint of the LocalStack services from within its runtime environment. Hence, you need to configure the Lambda functions to use the `AWS_ENDPOINT_URL` endpoint when accessing other AWS services in LocalStack. In Python, this may look something like. The code detects if it is running in LocalStack by checking if the `AWS_ENDPOINT_URL` variable exists and then configures the endpoint URL accordingly. ```python showshowLineNumbers ... if 'AWS_ENDPOINT_URL' in os.environ: dynamodb = boto3.resource('dynamodb', endpoint_url=os.environ['AWS_ENDPOINT_URL']) else: dynamodb = boto3.resource('dynamodb') ... ``` In LocalStack for AWS, no code changes are required using our [Transparent Endpoint Injection](/aws/customization/networking/transparent-endpoint-injection). ## Deploying to LocalStack You can now deploy your Serverless service to LocalStack. First, start LocalStack by running ```bash lstk start ``` Then deploy the endpoint by running ```bash serverless deploy --stage local ``` The expected result should be similar to: ```bash Serverless: Packaging service... Serverless: Excluding development dependencies... Serverless: Creating Stack... Serverless: Checking Stack create progress... ........ Serverless: Stack create finished... Serverless: Uploading CloudFormation file to S3... Serverless: Uploading artifacts... Serverless: Uploading service my-service.zip file to S3 (38.3 KB)... Serverless: Validating template... Serverless: Skipping template validation: Unsupported in Localstack Serverless: Updating Stack... Serverless: Checking Stack update progress... ..................................... Serverless: Stack update finished... Service Information service: my-service stage: local region: us-east-1 stack: my-service-local resources: 35 api keys: None endpoints: http://localhost.localstack.cloud:4566/restapis/XXXXXXXXXX/local/_user_request_ functions: ... layers: None ``` Use the displayed endpoint `http://localhost.localstack.cloud:4566/restapis/XXXXXXXXXX/local/_user_request_/my/custom/endpoint` to make requests to the deployed service. ## Advanced topics ### Local code mounting for lambda functions serverless-localstack supports a feature for lambda functions that allows local code mounting: ```yaml showshowLineNumbers # serverless.yml custom: localstack: # ... lambda: mountCode: True ``` When this flag is set, the lambda code will be mounted into the container running the function directly from your local directory instead of packaging and uploading it. ## Ran into trouble? If you run into any issues or problems while integrating LocalStack with your Serverless app, please [submit an issue](https://github.com/localstack/serverless-localstack/issues). # Terraform > Use the Terraform Infrastructure as Code framework with LocalStack. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction [Terraform](https://terraform.io/) is an Infrastructure-as-Code (IaC) framework developed by HashiCorp. It enables users to define and provision infrastructure using a high-level configuration language. Terraform uses HashiCorp Configuration Language (HCL) as its configuration syntax. HCL is a domain-specific language designed for writing configurations that define infrastructure elements and their relationships. LocalStack supports Terraform via the [AWS provider](https://registry.terraform.io/providers/hashicorp/aws/latest/docs) through [custom service endpoints](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/custom-service-endpoints#localstack). You can configure Terraform to use LocalStack in two ways: - Using [`lstk terraform`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#terraform) (alias `lstk tf`) to automatically configure the service endpoints for you. - Manually configuring the service endpoints in your Terraform configuration with additional maintenance. In this guide, we will demonstrate how you can create local AWS resources using Terraform and LocalStack, by using `lstk terraform` and a manual configuration example. ## `lstk terraform` `lstk terraform` (alias `lstk tf`) runs Terraform against LocalStack. It uses the [Terraform Override mechanism](https://www.terraform.io/language/files/override) and creates a temporary file `localstack_providers_override.tf` to configure the endpoints for the AWS `provider` section, then forwards your arguments to the real `terraform` binary. The endpoints for all services are configured to point to the LocalStack API (`http://localhost:4566` by default). It allows you to easily deploy your unmodified Terraform scripts against LocalStack. :::note `lstk terraform` supersedes the older [`tflocal` wrapper script](/aws/connecting/infrastructure-as-code/deprecated-wrapper-scripts#tflocal), which is deprecated but still available if you need it. ::: ### Create a Terraform configuration Create a new file named `main.tf` and add a minimal S3 bucket configuration to it. The following contents should be added in the `main.tf` file: ```hcl resource "aws_s3_bucket" "test-bucket" { bucket = "my-bucket" } ``` ### Install lstk To use `lstk terraform`, install `lstk` by following the [`lstk` installation instructions](/aws/developer-tools/running-localstack/lstk/#installation). You'll also need the [`terraform` CLI](https://developer.hashicorp.com/terraform/install) itself installed and on your `PATH`. Once installed, you can use `lstk terraform`, which has the same interface as the `terraform` command line. ```bash lstk terraform --help ``` ```bash title="Output" Usage: terraform [global options] [args] ... ``` ### Deploy the Terraform configuration Start your LocalStack container using your preferred method. Initialize Terraform using the following command: ```bash lstk terraform init ``` You can now provision the S3 bucket specified in the configuration: ```bash lstk terraform apply ``` ### Configuration The `lstk terraform` command supports several options and environment variables beyond what is available with standard `terraform`. Flags that are specific to `lstk terraform` must appear **before** the Terraform action: | Option | Description | | ---------------------- | ----------- | | `--region ` | Deployment region | | `--account ` | Target AWS account id (12 digits) | | Environment Variable | Default value | Description | | ------------------------------ | ------------------------------------- | ----------- | | `AWS_ENDPOINT_URL` | - | Override the auto-resolved LocalStack endpoint | | `LSTK_TF_CMD` | `terraform` | Binary to invoke, e.g. `tofu` | | `LSTK_TF_OVERRIDE_FILE_NAME` | `localstack_providers_override.tf` | Override file name | | `LSTK_TF_DRY_RUN` | - | When set, generate the override file but do not run Terraform | | `AWS_REGION` | - | Fallback for `--region` | | `AWS_ACCESS_KEY_ID` | - | Fallback for `--account` | ## Manual Configuration Instead of using `lstk terraform`, you have the option to manually configure the local service endpoints and credentials. The following sections will provide detailed steps for this manual configuration. ### General Configuration To begin, you need to define mock credentials for the AWS provider. Specify the following in your `main.tf` file: ```hcl showshowLineNumbers provider "aws" { access_key = "test" secret_key = "test" region = "us-east-1" } ``` ### Request Management Next, to prevent routing and authentication issues (which are unnecessary in this context), you should provide some general parameters: ```hcl showshowLineNumbers provider "aws" { access_key = "test" secret_key = "test" region = "us-east-1" skip_credentials_validation = true skip_metadata_api_check = true } ``` ### Services Furthermore, it's necessary to configure the individual services to use LocalStack. #### Using AWS_ENDPOINT_URL_S3 (Recommended for S3) With terraform-provider-aws version 5.x and later, you can use the `AWS_ENDPOINT_URL_S3` environment variable instead of configuring the endpoint in your Terraform code: ```bash export AWS_ENDPOINT_URL_S3=http://s3.localhost.localstack.cloud:4566 ``` This allows your Terraform configuration to work unchanged against both LocalStack and real AWS. No `endpoints` block is needed for S3, and virtual-hosted-style addressing is used by default. #### Configuring Endpoints in Terraform Alternatively, you can configure service endpoints directly in your provider block. For S3, use the virtual hosted-style endpoint: ```hcl showshowLineNumbers endpoints { s3 = "http://s3.localhost.localstack.cloud:4566" } ``` :::note The S3 endpoint uses the `s3.localhost.localstack.cloud` hostname to support virtual-hosted-style addressing, which AWS recommends and some regions require. If you cannot resolve this DNS record, you can use `http://localhost:4566` as a fallback and enable path-style addressing by adding `s3_use_path_style = true` to the provider configuration. Only use path-style if you have a specific requirement for it. ::: ### Final Configuration The final minimal configuration for deploying an S3 bucket via a `main.tf` file should resemble the following: ```hcl showshowLineNumbers provider "aws" { access_key = "mock_access_key" secret_key = "mock_secret_key" region = "us-east-1" skip_credentials_validation = true skip_metadata_api_check = true endpoints { s3 = "http://s3.localhost.localstack.cloud:4566" } } resource "aws_s3_bucket" "test-bucket" { bucket = "my-bucket" } ``` :::tip With terraform-provider-aws >= 5.x, you can simplify this further by removing the `endpoints` block and using the environment variable instead: ```bash export AWS_ENDPOINT_URL_S3=http://s3.localhost.localstack.cloud:4566 ``` Then your provider configuration only needs the credentials and skip parameters. ::: ### Endpoint Configuration Here's a configuration example with additional service endpoints. Please note that these provider configurations may not be necessary if you use `lstk terraform` (as described above). You can save the following configuration in a file named `provider.tf` and include it in your Terraform configuration. ```hcl showshowLineNumbers provider "aws" { access_key = "test" secret_key = "test" region = "us-east-1" skip_credentials_validation = true skip_metadata_api_check = true endpoints { apigateway = "http://localhost:4566" apigatewayv2 = "http://localhost:4566" cloudformation = "http://localhost:4566" cloudwatch = "http://localhost:4566" dynamodb = "http://localhost:4566" ec2 = "http://localhost:4566" es = "http://localhost:4566" elasticache = "http://localhost:4566" firehose = "http://localhost:4566" iam = "http://localhost:4566" kinesis = "http://localhost:4566" lambda = "http://localhost:4566" rds = "http://localhost:4566" redshift = "http://localhost:4566" route53 = "http://localhost:4566" s3 = "http://s3.localhost.localstack.cloud:4566" secretsmanager = "http://localhost:4566" ses = "http://localhost:4566" sns = "http://localhost:4566" sqs = "http://localhost:4566" ssm = "http://localhost:4566" stepfunctions = "http://localhost:4566" sts = "http://localhost:4566" } } ``` :::note The S3 endpoint uses `s3.localhost.localstack.cloud` to support virtual-hosted-style addressing (AWS recommended). If you need to use path-style addressing instead, change the S3 endpoint to `http://localhost:4566` and add `s3_use_path_style = true` to the provider block. ::: :::note To heuristically detect whether your Terraform configuration should be deployed against LocalStack, you can use the following snippet: ```hcl showshowLineNumbers data "aws_caller_identity" "current" {} output "is_localstack" { value = data.aws_caller_identity.current.id == "000000000000" } ``` It will detect whether the AWS account ID is `000000000000`, which is the default value for LocalStack. If you use a different account ID within LocalStack, you can customize the snippet accordingly. ::: ## OpenTofu OpenTofu is an open-source fork of Terraform acting as a drop-in replacement for Terraform, as it's compatible with Terraform versions 1.5.x and most of 1.6.x. You can use OpenTofu with LocalStack to create and manage your AWS resources with your pre-existing Terraform configurations. You can use the `LSTK_TF_CMD` environment variable with `lstk terraform` to specify the `tofu` binary to call, or setup a manual configuration to point the individual services to LocalStack. ```bash LSTK_TF_CMD=tofu lstk terraform --help ``` ```bash title="Output" Usage: tofu [global options] [args] The available commands for execution are listed below. The primary workflow commands are given first, followed by less common or more advanced commands. ... ``` ## Terragrunt Terragrunt is an open-source wrapper for Terraform that provides extra tools for keeping your configurations DRY, working with multiple Terraform modules, and managing remote state. You can use Terragrunt with LocalStack to create and manage your AWS resources with your pre-existing Terraform configurations. ### Configuration A sample `terragrunt.hcl` configuration file to use with LocalStack is shown below: ```hcl showshowLineNumbers generate "provider" { path = "provider.tf" if_exists = "overwrite_terragrunt" contents = < Customize how the LocalStack emulator behaves, including configuration options, logging, Kubernetes, other installations, integrations, networking, and advanced features. import SectionCards from '../../../../components/SectionCards.astro'; LocalStack provides a wide range of options for customizing how the emulator behaves and where it runs. Most users are well served by the defaults, but as your needs grow you can fine-tune everything from logging verbosity and networking to running on Kubernetes, alternative installation methods, and third-party integrations. # Overview > Tailor LocalStack for complex setups using power-user features. import SectionCards from '../../../../../components/SectionCards.astro'; LocalStack offers power-user features for tailoring the emulator for complex environments. Most users will not need to understand these features, but they are useful when you have complex requirements or are integrating LocalStack into a larger system. # ARM64 Support > Running LocalStack on ARM64 CPUs ## Introduction Since [version 0.13](https://github.com/localstack/localstack/releases/tag/v0.13.0), LocalStack officially publishes a [multi-architecture Docker manifest](https://hub.docker.com/r/localstack/localstack). This manifest contains links to a Linux AMD64 as well as a Linux ARM64 image. ## Pulling the LocalStack image With the multi-arch Docker manifest, your Docker client (and therefore [`lstk`](/aws/developer-tools/running-localstack/lstk)) now automatically selects the image according to your platform: ```bash docker pull localstack/localstack ``` You can check the architecture of the pulled image by using `docker inspect`: ```bash docker inspect localstack/localstack | jq '.[0].Architecture' ``` ```bash title="Output" "arm64" ``` ## Lambda multi-architecture support Since LocalStack 2.0, Lambda functions execute in Docker containers with the target platform `linux/amd64` or `linux/arm64` depending on the [instruction set architecture](https://docs.aws.amazon.com/lambda/latest/dg/foundation-arch.html) configured for the function (`x86_64` by default or `arm64`). This behavior can lead to errors if the host system, the Docker image, or the code/layer of the function do not support the target architecture. If you prefer to execute Lambda functions on your native platform architecture, you can set the [Lambda configuration](https://docs.localstack.cloud/aws/customization/configuration-options/#lambda) variable to `LAMBDA_IGNORE_ARCHITECTURE=1`. Example scenario: I have an amd64 machine and want to run a an arm64 Lambda function. I know that the Lambda function runs on both architectures (i.e., no architecture specific code or dependencies). Host systems with [multi-architecture support](https://docs.docker.com/build/building/multi-platform/) can run containers for different Linux architectures using emulation. For example, an Apple Silicon MacBook can execute `linux/arm64` (`arm64`) Lambda functions natively or emulate them for `linux/arm64` (`x86_64`). However, emulation through qemu is only best-effort and certain features such as [ptrace](https://github.com/docker/for-mac/issues/5191#issuecomment-834154431) for debugging might not work. You can check the supported architectures on your host system with: ```bash docker run --privileged --rm tonistiigi/binfmt ``` ```json title="Output" { "supported": [ "linux/amd64", "linux/arm64", "linux/386" ], "emulators": [ "jar", "llvm-12-runtime.binfmt", "python3.10", "python3.9", "qemu-aarch64" ] } ``` If you want to execute Docker Lambda functions or binaries which have not been built for your architecture, you might need to configure cross-platform emulation on your system. You can do so by installing the `bin_fmt` emulator with the following command: :::danger The following command installs additional emulators on your host system. ::: ```bash docker run --privileged --rm tonistiigi/binfmt --install amd64 ``` ## Troubleshooting ### Container exits with SIGILL on Apple Silicon On newer Apple Silicon hardware (for example Apple M4), running under Colima or Podman with an older Linux guest kernel can cause the container to crash immediately after license activation with exit code `252` (SIGILL). See [Why does the LocalStack container exit immediately with SIGILL (exit code 252) on Apple Silicon?](/aws/getting-started/faq/#why-does-the-localstack-container-exit-immediately-with-sigill-exit-code-252-on-apple-silicon) for the workaround and permanent fix. ### Pulling images for other architectures :::note Please be aware that this workaround is not supported by LocalStack at all. ::: If you want to use a LocalStack image which has been built for another architecture than yours, you can instruct Docker to use another platform by setting the `DOCKER_DEFAULT_PLATFORM` environment variable: ```bash export DOCKER_DEFAULT_PLATFORM=linux/amd64 ``` When using Docker Compose, you can use the `platform` element [as described in the specification](https://github.com/compose-spec/compose-spec/blob/master/spec.md#platform). ### Emulating AMD64 in host mode on Apple Silicon :::note Please be aware that this workaround is not supported by LocalStack at all. ::: This advanced workaround is running the open source version of LocalStack in host mode (i.e., developer mode) using AMD64 emulation on an ARM64 machine. First, you should enable "Rosetta" on your preferred terminal. This way you'll be installing packages for `x86_64` platform. ![Rosetta](/images/aws/m1-trouble-1.png) What we will be doing now is installing Java and Python executables using Homebrew, it should automatically resolve packages to proper architecture versions. ```bash # Install Homebrew /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" # Install java11 and follow instructions brew install java11 # Install jenv and follow instructions brew install jenv # Add Java11 to jenv and use it globally jenv add /Library/Java/JavaVirtualMachines/openjdk-11.jdk/Contents/Home/ jenv global 11 # Install pyenv and follow instructions brew install pyenv # Install python and enable it globally (check localstack/.python-version) pyenv install 3.11.9 pyenv global 3.11.9 ``` Then clone LocalStack to your machine, run `make install` and then `make start`. ### Raspberry Pi If you want to run LocalStack on your Raspberry Pi, make sure to use a 64bit operating system. In our experience, it works best on a Raspberry Pi 4 8GB with [Ubuntu Server 20.04 64Bit for Raspberry Pi](https://ubuntu.com/download/raspberry-pi). You can check if Docker is running and your architecture is ARM64 / aarch64 by using `docker info`: ```bash docker info ``` ```bash title="Output" Client: ... Server: ... Operating System: Ubuntu 20.04 OSType: linux Architecture: aarch64 ... ``` # Cross-Account and Cross-Region Access > Accessing resources in another account or region ## Introduction LocalStack automatically namespaces all resources based on the account ID and, in some cases, the region. However, there are certain resource types that can be accessed across multiple accounts or regions. This document provides information to help design such setups. :::note Cross-account support in LocalStack is being actively developed. Please report any issues to [LocalStack Support](/aws/help-support/get-help). ::: Cross-account/cross-region access happens when a client attempts to access a resource in another account or region than what it is configured with: The examples below select the account with the `--account` flag of `lstk aws`. You can also set the account ID through the `AWS_ACCESS_KEY_ID` environment variable, for example `AWS_ACCESS_KEY_ID=111111111111 lstk aws ...`. ```bash # Create a queue in one account and region lstk aws --account 111111111111 sqs create-queue \ --queue-name my-queue \ --region ap-south-1 ``` ```json title="Output" { "QueueUrl": "http://sqs.ap-south-1.localhost.localstack.cloud:443/111111111111/my-queue" } ``` ```bash # Set some attributes lstk aws --account 111111111111 sqs set-queue-attributes \ --attributes VisibilityTimeout=60 \ --queue-url http://sqs.ap-south-1.localhost.localstack.cloud:443/111111111111/my-queue \ --region ap-south-1 # Retrieve the queue attribute from another account and region # The required information for LocalStack to locate the queue is available in the queue URL lstk aws --account 222222222222 sqs get-queue-attributes \ --attribute-names VisibilityTimeout \ --region eu-central-1 \ --queue-url http://sqs.ap-south-1.localhost.localstack.cloud:443/111111111111/my-queue ``` ```json title="Output" { "Attributes": { "VisibilityTimeout": "60" } } ``` ## Cross-Account Resources that can be accessed across accounts are identified by their Amazon Resource Names (ARNs) or other schemes such as SQS Queue URLs. The full list of resources and operations that allow cross-account access are listed below. :::tip LocalStack does not enforce IAM for cross-account access by default. Use the `ENFORCE_IAM` [configuration](/aws/customization/configuration-options#iam) option to enable it. ::: ### EC2 Peering It is possible to create peered VPCs and transit gateway peering attachments that are in a different region or account than the requester. Ensure that the `PeerRegion` and `PeerOwnerId` arguments are correctly set when creating these resources. ### KMS keys - `CreateGrant` - `Decrypt` - `DescribeKey` - `Encrypt` - `GenerateDataKey` - `GenerateDataKeyPair` - `GenerateDataKeyPairWithoutPlaintext` - `GenerateDataKeyWithoutPlaintext` - `GenerateMac` - `GetKeyRotationStatus` - `GetPublicKey` - `ListGrants` - `RetireGrant` - `RevokeGrant` - `Sign` - `Verify` - `VerifyMac` ### Lambda functions and layers - `AddLayerVersionPermission` - `CreateAlias` - `DeleteAlias` - `DeleteFunction` - `DeleteFunctionConcurrency` - `DeleteLayerVersion` - `GetAlias` - `GetFunction` - `GetFunctionConfiguration` - `GetLayerVersion` - `GetLayerVersionByArn` - `GetLayerVersionPolicy` - `GetPolicy` - `Invoke` - `ListAliases` - `ListLayerVersions` - `ListTags` - `ListVersionsByFunction` - `PublishVersion` - `PutFunctionConcurrency` - `RemoveLayerVersionPermission` - `TagResource` - `UntagResource` - `UpdateAlias` - `UpdateFunctionCode` ### S3 buckets Like AWS, LocalStack S3 has a bucket namespace which is shared by all accounts. This means that the bucket name has to be globally unique. - `GetObject` - `ListObjects` - `PutObject` ### Secrets Manager - `DeleteSecret` - `DescribeSecret` - `GetResourcePolicy` - `GetSecretValue` - `ListSecretVersionIds` - `PutResourcePolicy` - `PutSecretValue` - `RestoreSecret` - `TagResource` - `UntagResource` - `UpdateSecret` ### SNS topics - `AddPermission` - `DeleteTopic` - `GetTopicAttributes` - `ListSubscriptionByTopic` - `Publish` - `RemovePermission` - `SetTopicAttributes` - `Subscribe` ### SQS queues On AWS, all operations except `AddPermission`, `CreateQueue`, `DeleteQueue`, `ListQueues`, `ListQueueTags`, `RemovePermission`, `SetQueueAttributes`, `TagQueue` and `UntagQueue` allow cross-account access. On LocalStack, all operations allow cross-account access. ## Cross-Region AWS provides individual API endpoints for each region, and typically, resources can only be accessed within their respective regions. On the other hand, LocalStack operates on a unified API endpoint, allowing interactions with services across regions. # Filesystem Layout > Understanding the runtime directory layout LocalStack uses internally import { FileTree } from '@astrojs/starlight/components'; This page describes the filesystem directory layout used internally by LocalStack. :::note This filesystem layout was introduced in LocalStack v1 and can be disabled by setting `LEGACY_DIRECTORIES` to `1`. ::: LocalStack uses following directory layout when running within a container. - etc - localstack - init - usr - lib - localstack - var - lib - **localstack** the LocalStack volume directory root - cache - lib - logs - state - tmp ## Directory contents ### LocalStack volume directory - `/var/lib/localstack`: the [LocalStack volume](#localstack-volume) directory root - `/var/lib/localstack/lib`: variable packages (like extensions or lazy-loaded third-party dependencies) - `/var/lib/localstack/logs`: logs for recent LocalStack runs - `/var/lib/localstack/state`: contains the state of services if persistence is enabled (such as OpenSearch cluster data) - `/var/lib/localstack/tmp`: temporary data that is not expected to survive LocalStack runs (may be cleared when LocalStack starts or stops) - `/var/lib/localstack/cache`: temporary data that is expected to survive LocalStack runs (is not cleared when LocalStack starts or stops) ### Configuration - `/etc/localstack`: configuration directory - `/etc/localstack/init`: root directory for [initialization hooks](/aws/customization/advanced/initialization-hooks) ### Static libraries - `/usr/lib/localstack`: static third-party packages installed into the container images :::note Previously, directories were individually configurable, e.g., via `DATA_DIR` or `HOST_TMP_DIR`. These have been deprecated since LocalStack v1, since we now follow a directory convention. `DATA_DIR` implicitly points to `/var/lib/localstack/state` if persistence is enabled. Use `PERSISTENCE=1` to enable persistence. If `DATA_DIR` is set, its value is ignored, a warning is logged and `PERSISTENCE` is set to `1`. `HOST_TMP_FOLDER` is determined by inspecting the volume mounts and using the source of the bind mount to `/var/lib/localstack`. ::: ## LocalStack volume For LocalStack to function correctly, the LocalStack volume must be mounted from the host into the container at `/var/lib/localstack`. ### Using docker-compose When using Docker Compose, this can be achieved using following: ```yaml volumes: - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" ``` `${LOCALSTACK_VOLUME_DIR}` could be an arbitrary location on the host, e.g., `./volume`. In this case, the effective layout would be something like: - localstack - cache - machine.json - server.test.pem - server.test.pem.crt - server.test.pem.key - lib - opensearch - 1.1.0 - logs - localstack_infra.err - localstack_infra.log - state - startup_info.json - tmp - zipfile.4986fb95 ### Using lstk When using [`lstk`](/aws/developer-tools/running-localstack/lstk) to start LocalStack, the volume directory is configured with the `volume` field on a container block. It should point to a directory on the host which is then automatically mounted into `/var/lib/localstack`: ```toml # .lstk/config.toml [[containers]] type = "aws" volume = "./volume" ``` If `volume` is not set, `lstk` defaults to `/lstk/volume/`. Run [`lstk volume path`](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/#volume) to print the resolved directory. # Initialization Hooks > Writing shell or Python scripts to customize or initialize your LocalStack instance. import { Tabs, TabItem, FileTree } from '@astrojs/starlight/components'; ## Lifecycle Stages and Hooks LocalStack has four well-known lifecycle phases or stages: * `BOOT`: the container is running but the LocalStack runtime has not been started * `START`: the Python process is running and the LocalStack runtime is starting * `READY`: LocalStack is ready to serve requests * `SHUTDOWN`: LocalStack is shutting down You can hook into each of these lifecycle phases using custom shell or Python scripts. Each lifecycle phase has its own directory in `/etc/localstack/init`. You can mount individual files, stage directories, or the entire init directory from your host into the container. - etc - localstack - init - boot.d executed in the container before localstack starts - ready.d executed when localstack becomes ready - shutdown.d executed when localstack shuts down - start.d executed when localstack starts up In these directories, you can put either executable shell scripts or Python programs, which will be executed from within a Python process. All except `boot.d` will be run in the same Python interpreter as LocalStack, which gives additional ways of configuring/extending LocalStack. You can also use subdirectories to organize your init scripts. Currently, known script extensions are `.sh` and `.py`. Additionally, with the installation of the `localstack-extension-terraform-init` [extension](/aws/customization/integrations/extensions/), `.tf` files can also be supported. Shell scripts have to be executable, and have to have a [shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) (usually `#!/bin/bash`). A script can be in one of four states: `UNKNOWN`, `RUNNING`, `SUCCESSFUL`, `ERROR`. Scripts are by default in the `UNKNOWN` state once they have been discovered. The remaining states should be self-explanatory. ### Execution Order and Script Failures Scripts are sorted and executed in alphanumerical order. If you use subdirectories, scripts in parent folders are executed first, and then the directories are traversed in alphabetical order, depth first. If an init script fails, the remaining scripts will still be executed in order. A script is considered in `ERROR` state if it is a shell script and returns with a non-zero exit code, or if a Python script raises an exception during its execution. ## Status Endpoint There is an additional endpoint at `localhost:4566/_localstack/init` which returns the state of the initialization procedure. Boot scripts (scripts placed in `boot.d`) are currently always in the `UNKNOWN` state, since they are executed outside the LocalStack process and we don't know whether they have been successfully executed or not. ```bash curl -s localhost:4566/_localstack/init | jq . ``` ```json { "completed": { "BOOT": false, "START": true, "READY": true, "SHUTDOWN": false }, "scripts": [ { "stage": "BOOT", "name": "booting.sh", "state": "UNKNOWN" }, { "stage": "READY", "name": "pre_seed.py", "state": "SUCCESSFUL" } ] } ``` ### Querying Stages You can also query a specific stage at `localhost:4566/_localstack/init/`: ```bash curl -s localhost:4566/_localstack/init/ready | jq . ``` ```json { "completed": true, "scripts": [ { "stage": "READY", "name": "pre_seed.py", "state": "SUCCESSFUL" } ] } ``` To check whether a given stage has been completed you can now run, for example: ```bash curl -s localhost:4566/_localstack/init/ready | jq .completed ``` which returns either `true` or `false`. ## Example A common use case for init hooks is pre-seeding LocalStack with custom state. For example if you want to have a certain S3 bucket or DynamoDB table created when starting LocalStack, init hooks can be very useful. :::tip If you have more complex states, [Cloud Pods](/aws/developer-tools/snapshots/cloud-pods) and [how to auto-load them on startup](/aws/developer-tools/snapshots/cloud-pods#auto-loading-from-cloud-pods) may be a good option to look into! ::: To execute aws cli commands when LocalStack becomes ready, simply create a script `init-aws.sh` and mount it into `/etc/localstack/init/ready.d/`. Make sure the script is executable: run `chmod +x init-aws.sh` on the file first. You can use anything available inside the container, and can install new software: ```bash #!/bin/bash npm install -g @localstack/lstk lstk setup aws lstk aws s3 mb s3://my-bucket lstk aws sqs create-queue --queue-name my-queue ``` Start Localstack: ```yaml services: localstack: container_name: "${LOCALSTACK_DOCKER_NAME:-localstack-main}" image: localstack/localstack ports: - "127.0.0.1:4566:4566" environment: - DEBUG=1 volumes: - "/path/to/init-aws.sh:/etc/localstack/init/ready.d/init-aws.sh" # ready hook - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" ``` Declare the bind mount and the `DEBUG` profile in your config file, then start LocalStack: ```toml # .lstk/config.toml [[containers]] type = "aws" env = ["debug"] volumes = ["/path/to/init-aws.sh:/etc/localstack/init/ready.d/init-aws.sh"] [env.debug] DEBUG = "1" ``` ```bash lstk start ``` Another use for init hooks can be seen when [adding custom TLS certificates to LocalStack](/aws/developer-tools/security-testing/custom-tls-certificates#custom-tls-certificates-with-init-hooks). ### Terraform Files as Init Hooks Running Terraform configuration files as init hooks requires the installation of a special extension. For more information on how to manage [LocalStack extensions](/aws/customization/integrations/extensions/), please refer to the dedicated documentation page, and for more details on running init hooks in development mode, you can check out the [extension repository description](https://github.com/localstack/localstack-extensions/tree/main/terraform-init). Start LocalStack with **`EXTENSION_AUTO_INSTALL="localstack-extension-terraform-init"`**. Mount a **`main.tf`** file into **`/etc/localstack/init/ready.d`** When LocalStack starts up, it will install the extension, which in turn installs Terraform into the container. If one of the init stage directories contain a `main.tf` file, the extension will run `terraform init` and `terraform apply` on that directory. ```terraform # main.tf resource "aws_s3_bucket" "example" { bucket = "my-tf-test-bucket" tags = { Name = "My bucket" Environment = "Dev" } } ``` Start LocalStack for AWS with mounted `main.tf`: ```yaml services: localstack: container_name: "localstack-main" image: localstack/localstack-pro ports: - "127.0.0.1:4566:4566" # LocalStack Gateway environment: # Activate LocalStack for AWS: https://docs.localstack.cloud/getting-started/auth-token/ - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} - EXTENSION_AUTO_INSTALL=localstack-extension-terraform-init volumes: # you could also place your main.tf in `./ready.d` and set "./ready.d:/etc/localstack/init/ready.d" - "./main.tf:/etc/localstack/init/ready.d/main.tf" - "./volume:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" ``` ```toml # .lstk/config.toml [[containers]] type = "aws" env = ["terraform-init"] volumes = ["./main.tf:/etc/localstack/init/ready.d/main.tf"] [env.terraform-init] EXTENSION_AUTO_INSTALL = "localstack-extension-terraform-init" ``` ```bash lstk start ``` You can wait for LocalStack to complete the startup process, and then print the created S3 bucket: ```bash lstk aws s3 ls ``` The logs should show something like: ```bash 2024-06-26T20:36:19.946 INFO --- [ady_monitor)] l.extension : Applying terraform project from file /etc/localstack/init/ready.d/main.tf 2024-06-26T20:36:19.946 DEBUG --- [ady_monitor)] localstack.utils.run : Executing command: ['tflocal', '-chdir=/etc/localstack/init/ready.d', 'init', '-input=false'] 2024-06-26T20:36:26.864 DEBUG --- [ady_monitor)] localstack.utils.run : Executing command: ['tflocal', '-chdir=/etc/localstack/init/ready.d', 'apply', '-auto-approve'] ``` For a more complex demo project, on how to use Terraform init hooks for your testing environments, you can check out [this example](/aws/tutorials/using-terraform-with-testcontainers-and-localstack/) in the Tutorials section. ## Troubleshooting If you are having issues with your initialization hooks not being executed, please perform the following checks: * Do the scripts have a known file extensions (`.sh` or `.py`)? * If not, rename the files to the matching file extension. * Is the script file configured to use LF endings instead of CRLF endings? * If not, switch the file to LF mode as it is utilized in the Unix environment within a container. * Is the script marked as executable? * If not, set the executable flag on the file (`chmod +x ...`). * If it's a shell script, does it have a shebang (e.g., `#!/bin/bash`) as the first line in the file? * If not, add the shebang header (usually `#!/bin/bash`) on top of your script file. * Is the script being listed in the logs when running LocalStack with `DEBUG=1`? * The detected scripts are logged like this: ```bash ... Init scripts discovered: {BOOT: [], START: [], READY: [Script(path='/etc/localstack/init/ready.d/setup.sh', stage=READY, state=UNKNOWN)], SHUTDOWN: []} ... Running READY script /etc/localstack/init/ready.d/setup.sh ... ``` * If your script does not show up in the list of discovered init scripts, please check your Docker volume mount. Most likely the scripts are not properly mounted into the Docker container. * Are resources not being created? * Ensure that AWS [credentials](/aws/connecting/credentials) are set, e.g. through `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables. # Multi-Account Setups > Using LocalStack in multi-tenant setups :::note Please note that multi-accounts may not work for use-cases that have cross-account and cross-service access. Please contact [LocalStack Support](/aws/help-support/get-help) to request support for specific use-cases. ::: LocalStack ships with multi-account support which allows namespacing based on AWS account ID. LocalStack uses the value in the AWS Access Key ID field for the purpose of namespacing over account ID. For more information, see [Credentials](/aws/connecting/credentials). The Access Key ID field can be configured in the AWS CLI in multiple ways: please refer to [AWS CLI documentation](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html#cli-configure-quickstart-precedence). ## Examples In the following examples, we select the account ID with the `--account` flag of `lstk aws`. ```bash lstk aws --account 000000000001 ec2 create-key-pair --key-name green-hospital lstk aws --account 000000000002 ec2 create-key-pair --key-name red-medicine lstk aws --account 000000000001 ec2 describe-key-pairs { "KeyPairs": [ { "KeyFingerprint": "6b:e3:a3:41:4b:60:f3:6d:7b:84:3e:17:e3:ad:d0:15", "KeyName": "green-hospital" } ] } lstk aws --account 000000000002 ec2 describe-key-pairs { "KeyPairs": [ { "KeyFingerprint": "16:4c:64:13:36:41:7c:75:d0:51:f0:db:ed:d7:c8:95", "KeyName": "red-medicine" } ] } ``` Alternatively, you can set the account ID through the `AWS_ACCESS_KEY_ID` environment variable: ```bash AWS_ACCESS_KEY_ID=000000000001 lstk aws ec2 describe-key-pairs ``` If no explicit Account ID is set, LocalStack falls back to default. In this example, no resources are returned. ```bash lstk aws ec2 describe-key-pairs { "KeyPairs": [] } ``` # Regions Coverage > Understanding the Region Coverage for LocalStack's emulation of AWS services. ## Introduction LocalStack ships with multi-region support, enabling users to emulate different AWS regions and namespace resources based on the region. LocalStack supports a wide range of AWS regions, including commercial, government, and China regions, providing a realistic environment for development and testing across multiple regions on your local machine. ## Supported Regions | Region Code | Supported | Notes | |-----------------|------------|--------------------------------------------| | us-east-1 | ✔️ | Commonly used as the default region | | us-east-2 | ✔️ | | | us-west-1 | ✔️ | | | us-west-2 | ✔️ | | | ca-central-1 | ✔️ | | | ca-west-1 | ✔️ | Available in LocalStack Enterprise only | | eu-north-1 | ✔️ | | | eu-west-1 | ✔️ | | | eu-west-2 | ✔️ | | | eu-west-3 | ✔️ | | | eu-central-1 | ✔️ | | | eu-south-1 | ✔️ | | | eu-south-2 | ✔️ | | | eu-central-2 | ✔️ | | | ap-south-1 | ✔️ | | | ap-south-2 | ✔️ | | | ap-northeast-1 | ✔️ | | | ap-northeast-2 | ✔️ | | | ap-northeast-3 | ✔️ | | | ap-southeast-1 | ✔️ | | | ap-southeast-2 | ✔️ | | | ap-southeast-3 | ✔️ | | | ap-southeast-4 | ✔️ | | | ap-southeast-5 | ✔️ | Available in LocalStack Enterprise only | | ap-east-1 | ✔️ | | | sa-east-1 | ✔️ | | | af-south-1 | ✔️ | | | me-south-1 | ✔️ | | | me-central-1 | ✔️ | | | cn-north-1 | ✔️ | Available in LocalStack Enterprise only | | cn-northwest-1 | ✔️ | Available in LocalStack Enterprise only | | us-gov-east-1 | ✔️ | Available in LocalStack Enterprise only | | us-gov-west-1 | ✔️ | Available in LocalStack Enterprise only | | il-central-1 | ✔️ | | # Usage Tracking > Understanding what data LocalStack collects and how you can opt out of usage tracking ## Overview For license activations, we track the timestamp and the licensing credentials. It is tracked regardless of whether the user disables event tracking since we collect this in the backend, not the client. ## LocalStack usage statistics For Pro users, most of the information is collected to populate the [Stack Insights](/aws/organizations-admin/stack-insights) dashboard. Collecting basic anonymized usage of AWS services helps us better direct engineering efforts to services that are used the most or cause the most issues. ### Session information The current usage event collection on the client side includes: - A randomly generated ID pertaining to the session - The Auth Token - A randomly generated machine ID is kept throughout the session but deleted once the LocalStack cache directory is removed - The operating system (mostly Linux since LocalStack typically runs in our Debian container) - The LocalStack version being used - Whether LocalStack is running in a CI environment - Whether LocalStack is running in Docker - Whether this is an internal test run (LocalStack development flag) Here is an example of a usage event: ```json { "session_id": "f41119fd-af20-4d48-af92-2b8d69f8cf7e", "machine_id": "1b6a2f12", "api_key": "0123456789", "system": "Linux", "version": "1.0.5.dev", "is_ci": false, "is_docker": true, "is_testing": false } ``` ### AWS API call metadata The AWS API call metadata includes: - The service being called (like `s3` or `lambda`) - The operation being called (like `PutObject`, `CreateQueue`, `DeleteQueue`) - The HTTP status code of the response - If it is a 400 error, we collect the error type and message. If it is a 500 error (internal LocalStack error), and `DEBUG=1` is enabled, we may also collect the stack trace to help us identify LocalStack bugs - Whether the call originated from inside LocalStack - The region user made the call to - The dummy account ID user made the request - The user agent the request was made with (`aws-cli`, `terraform`) Here is an example of AWS API call metadata: ```json { "name": "aws_request", "metadata": { "session_id": "64517be1-7dd0-479b-a8ec-88d3e544afd9", "client_time": "2022-08-30 12:27:25.556001" }, "payload": { "service": "sqs", "operation": "DeleteQueue", "status_code": 400, "err_type": "AWS.SimpleQueueService.NonExistentQueue", "err_msg": "The specified queue does not exist for this wsdl version.", "is_internal": false, "region": "us-east-1", "account_id": "000000000000", "user_agent": "aws-cli/1.25.52 Python/3.10.4 Linux/5.13.0-28-generic awscrt/0.14.0 botocore/1.27.52" } } ``` ### CLI invocations We collect an anonymized event if a CLI command was invoked, but do not collect any of the parameter values. This event is not connected to the session or the Auth Token. Here is an example of a CLI invocation event: ```json { "name": "cli_cmd", "metadata": { "client_time": "2022-08-30 14:46:54.116457" }, "payload": { "cmd": "lstk start", "params": [ "file" ] } } ``` ### Feature usage We collect the usage of particular features in an anonymized and aggregated way. - If you use init scripts, we collect the stage, how many scripts are being executed, and how long they took - Nothing else at the moment, but we may track additional features ## What we are not collecting? - Specific LocalStack configuration values - Content or file names of files being uploaded to S3 - More generally, we don't collect any parameters of AWS API Calls. We do not track S3 bucket names, Lambda function names, EC2 configurations, or anything similar - Any sensitive information about the request (like credentials and URL parameters) ## Configuration You can disable event reporting in your LocalStack instance by setting the environment variable `DISABLE_EVENTS=1`. # Configuration > Set the environment variables and flags that change how LocalStack starts and runs. LocalStack exposes various configuration options to control its behaviour. With `lstk`, these options can be passed as `LOCALSTACK_`-prefixed environment variables when starting the container: ```bash LOCALSTACK_DEBUG=1 lstk start ``` Alternatively, set them as named environment profiles in your config file and reference them from the container block: ```toml # .lstk/config.toml [[containers]] type = "aws" env = ["debug"] [env.debug] DEBUG = "1" ``` ```bash lstk start ``` See [Passing environment variables to the container](/aws/developer-tools/running-localstack/lstk/configuration/#passing-environment-variables-to-the-container) for details. To facilitate interoperability, configuration variables can be prefixed with `LOCALSTACK_` in docker. For instance, setting `LOCALSTACK_PERSISTENCE=1` is equivalent to `PERSISTENCE=1`. Configurations marked as **Deprecated** will be removed in the next major version. You can find previously removed configuration variables under [Legacy](#legacy). ## Core Options that affect the core LocalStack system. | Variable | Example Values | Description | | - | - | - | | `DEBUG` | `0` (default) \|`1`| Flag to increase log level and print more verbose logs (useful for troubleshooting issues)| | `IMAGE_NAME`| `localstack/localstack` (default), `localstack/localstack:0.11.0` | Specific name and tag of LocalStack Docker image to use.| | `LOCALSTACK_HOST`| `localhost.localstack.cloud:4566` (default) | This is interpolated into URLs and addresses that are returned by LocalStack. It has the form `:`. | | `GATEWAY_LISTEN` | `0.0.0.0:4566,0.0.0.0:443` (default in Docker mode), `127.0.0.1:4566,127.0.0.1:443` (default in host mode) | Configures the bind addresses of LocalStack. It has the form `:(,:)*`. When unset, LocalStack listens on both the HTTP edge port (`4566`) and HTTPS (`443`). | | `USE_SSL` | `0` (default) | Whether to return URLs using HTTP (`0`) or HTTPS (`1`). Changed with 3.0.0. In earlier versions this was toggling SSL support on or off. | | `PERSISTENCE` | `0` (default) | Enable persistence. See [Persistence Mechanism](/aws/developer-tools/snapshots/persistence) and [Filesystem Layout](/aws/customization/advanced/filesystem). | | `MAIN_CONTAINER_NAME` | `localstack-main` (default) | Specify the main docker container name | | `LS_LOG` | `trace`, `trace-internal`, `debug`, `info`, `warn`, `error`, `warning`| Specify the log level. Currently overrides the `DEBUG` configuration. `trace` for detailed request/response, `trace-internal` for internal calls, too. | | `EXTERNAL_SERVICE_PORTS_START` | `4510` (default) | Start of the [External Service Port Range](/aws/customization/networking/external-port-range) (inclusive). | | `EXTERNAL_SERVICE_PORTS_END` | `4560` (default) | End of the [External Service Port Range](/aws/customization/networking/external-port-range) (exclusive). | | `EAGER_SERVICE_LOADING` | `0` (default) \|`1` | Boolean that toggles lazy loading of services. If eager loading is enabled, services are started at LocalStack startup rather than their first use. Be aware that eager loading increases the LocalStack startup time. | | `SERVICES`| `s3,sqs` | A comma-delimited string of services. Check the [internal health endpoint](/aws/customization/networking/internal-endpoints#localstack-endpoints) `/_localstack/health` for valid service names. If `SERVICES` is set LocalStack will only load the listed services. All other services will be disabled and cannot be used. | | `ALLOW_NONSTANDARD_REGIONS` | `0` (default) | Allows the use of non-standard AWS regions. By default, LocalStack only accepts [standard AWS regions](https://docs.aws.amazon.com/general/latest/gr/rande.html). | | `PARITY_AWS_ACCESS_KEY_ID` | `0` (default) | Enables the use production-like access key IDs. By default, LocalStack issues keys with `LSIA...` and `LKIA...` prefix, and will reject keys that start with `ASIA...` or `AKIA...`. | | `LOCALSTACK_RESPONSE_HEADER_ENABLED` | `1` (default) \| `0` | Whether LocalStack adds the [`x-localstack` response header](/aws/customization/networking/internal-endpoints#x-localstack-response-header) to every AWS API response. The header value is the LocalStack version and lets client tools detect LocalStack and its version. | ## CLI `lstk` is configured through its config file rather than through environment variables. See [Configuration](/aws/developer-tools/running-localstack/lstk/configuration/) on the `lstk` page for the config file search order, the field reference, and how to define named environment profiles. ## Docker Options to configure how LocalStack interacts with Docker. | Variable | Example Values | Description | | - | - | - | | `LOCALSTACK_VOLUME_DIR` | `~/.cache/localstack/volume` (on Linux) | The location on the host of the LocalStack volume directory mount. See [Filesystem Layout](/aws/customization/advanced/filesystem) | | `DOCKER_FLAGS` | | Allows to pass custom flags (e.g., volume mounts) to "docker run" when running LocalStack in Docker. | | `DOCKER_SOCK` | `/var/run/docker.sock` | Path to local Docker UNIX domain socket | | `DOCKER_BRIDGE_IP` | `172.17.0.1` | IP of the docker bridge used to enable access between containers | | `LEGACY_DOCKER_CLIENT` | `0`\|`1` | Whether LocalStack should use the command-line Docker client and subprocess execution to run Docker commands, rather than the Docker SDK. | | `DOCKER_CMD` | `docker` (default), `sudo docker`| Shell command used to run Docker containers (only used in combination with `LEGACY_DOCKER_CLIENT`) | | `FORCE_NONINTERACTIVE` | | When running with Docker, disables the `--interactive` and `--tty` flags. Useful when running headless. | ## Local AWS Services This section covers configuration options that are specific to certain AWS services. ### API Gateway | Variable | Example Values | Description | | - | - | - | | `PROVIDER_OVERRIDE_APIGATEWAY` | `legacy`\|`next_gen` (default)| The [new API Gateway implementation](/aws/services/apigateway#new-api-gateway-implementation) is active by default since LocalStack 4.0. | ### AppSync | Variable | Example Values | Description | | - | - | - | | `GRAPHQL_ENDPOINT_STRATEGY` | `legacy`\|`domain`\|`path` | Governs how AppSync endpoints are created to access a GraphQL API (see [AppSync Endpoints](/aws/services/appsync/#configuring-graphql-endpoints)) | | `APPSYNC_JS_LIBS_VERSION` | `latest`(default) \|`refresh`\|`` | Control the version of the `@aws-appsync/utils` package to use. `latest` means fetch the latest but only if not present. `refresh` means always fetch the latest version. `` means use a specific git reference. | ### Batch | Variable | Example Values | Description | | - | - | - | | `BATCH_DOCKER_FLAGS` | `-e TEST_ENV=1337` | Additional flags provided to the batch container. Same restrictions as `LAMBDA_DOCKER_FLAGS`. | ### Bedrock | Variable | Example Values | Description | | - | - | - | | `BEDROCK_PREWARM` | `0` (default) \| `1` | Pre-warm the Bedrock engine directly on LocalStack startup instead of on demand. | | `DEFAULT_BEDROCK_MODEL` | `smollm2:360m` (default) | The model that is used initially to handle text model invocations in Bedrock. Any text-based model available for Ollama is usable. | | `BEDROCK_PULL_MODELS` | `deepseek-r1,mistral` \' '' (default) | A list of models that should get pulled into the model cache on startup. `DEFAULT_BEDROCK_MODEL` is automatically in there | | `BEDROCK_DOCKER_FLAGS` | `-v /certs:/certs` | Additional flags passed to Docker when creating Bedrock containers. Same restrictions as `LAMBDA_DOCKER_FLAGS`. | ### BigData (EMR, Athena, Glue) | Variable | Example Values | Description | | - | - | - | | `BIGDATA_DOCKER_NETWORK` | | Network the bigdata should be connected to. The LocalStack container has to be connected to that network as well. Per default, the bigdata container will be connected to a network LocalStack is also connected to. | `BIGDATA_DOCKER_FLAGS` | | Additional flags for the bigdata container. Same restrictions as `LAMBDA_DOCKER_FLAGS`. ### CloudFormation | Variable | Example Values | Description | | - | - | - | | `CFN_PER_RESOURCE_TIMEOUT` | `300` (default) | Set the timeout to deploy each individual CloudFormation resource. | `CFN_VERBOSE_ERRORS` | `0` (default) \|`1` | Show exceptions for CloudFormation deploy errors. | `CFN_STRING_REPLACEMENT_DENY_LIST` | `""` (default) \|`https://api-1.execute-api.us-east-2.amazonaws.com/test-resource,https://api-2.execute-api.us-east-2.amazonaws.com/test-resource` | Comma-separated list of AWS URLs that should not be modified to point to Localstack. For example, when deploying a CloudFormation template we might want to leave certain resources pointing to actual AWS URLs, or even leave environment variables with URLs like that untouched. ### CloudFront | Variable | Example Values | Description | | - | - | - | | `CLOUDFRONT_LAMBDA_EDGE` | `0` (default) \| `1` | Enable Lambda@Edge support for CloudFront distributions. | ### CloudWatch | Variable | Example Values | Description | | - | - | - | | `PROVIDER_OVERRIDE_CLOUDWATCH` | `v1` | Use the old CloudWatch provider. | ### CodeBuild | Variable | Example Values | Description | | - | - | - | | `CODEBUILD_REMOVE_CONTAINERS` | `0`\|`1` (default) | Remove Docker containers associated with a CodeBuild build tasks after execution. Disabling this and dumping container logs might help with troubleshooting failing builds. | | `CODEBUILD_ENABLE_CUSTOM_IMAGES` | `0` (default) \|`1` | Enable the usage of arbitrary CodeBuild build images. By default, all the builds are executed in a Amazon Linux 2023 container. | | `CODEBUILD_DOCKER_FLAGS` | `-v /certs:/certs` | Additional flags passed to Docker when creating CodeBuild build containers. Same restrictions as `LAMBDA_DOCKER_FLAGS`. | ### CodePipeline | Variable | Example Values | Description | | - | - | - | | `CODEPIPELINE_GH_TOKEN` | | GitHub Personal Access Token to used by CodeConnections Source action to access private repositories on GitHub. | ### DMS | Variable | Example Values | Description | | - | - | - | | `DMS_SERVERLESS_DEPROVISIONING_DELAY` | `60` (default), any positive integer | Delay the deprovisioning of serverless by defined seconds. Once deprovisioned the statistics will be reset. | | `DMS_SERVERLESS_STATUS_CHANGE_WAITING_TIME` | `0` (default), any positive integer | Simulates a waiting time (in seconds) between status changes when the serverless replication starts. The waiting time will be applied for each status change (there are 6 status changes before the task is running). | ### DocumentDB | Variable | Example Values | Description | | - | - | - | | `DOCDB_PROXY_CONTAINER` | `0` (default) \|`1` | Whether the DocumentDB starts the MongoDB container proxied over LocalStack container. When enabled lambda functions can use the `Endpoint` configuration of the DocDB cluster or instance to connect to the DocumentDB. By default the container starts without proxy as standalone container. | | `DOCDB_DOCKER_FLAGS` | `-v /certs:/certs` | Additional flags passed to Docker when creating DocumentDB containers. Same restrictions as `LAMBDA_DOCKER_FLAGS`. | ### DynamoDB | Variable | Example Values | Description | | - | - | - | | `DYNAMODB_ERROR_PROBABILITY` | Decimal value between `0.0`(default) and `1.0` | Randomly inject `ProvisionedThroughputExceededException` errors into DynamoDB API responses. | | `DYNAMODB_HEAP_SIZE` | `256m` (default), `1G` | Sets the JAVA EE maximum memory size for DynamoDB; full table scans require more memory | | `DYNAMODB_SHARE_DB` | `0`\|`1` | When activated, DynamodDB will use a single database instead of separate databases for each credential and region. | | `DYNAMODB_IN_MEMORY` | `0` (default) \|`1` | When activated, DynamodDB will start in in-memory mode, which can have a faster throughput. If you use this options, both persistence and cloud pods will not work for DynamoDB | | `DYNAMODB_OPTIMIZE_DB_BEFORE_STARTUP` | `0`\|`1` | Optimize the database tables in the store before starting | | `DYNAMODB_DELAY_TRANSIENT_STATUSES` | `0`\|`1` | When activated, DynamoDB will introduce artificial delays in resource creation to simulate the actual cloud service more closely. Currently works only for CREATING and DELETING online index statuses. | | `DYNAMODB_CORS` | `*` | Enable CORS support for specific allow-list list the domains separated by `,` use `*` for public access (default is `*`) | | `DYNAMODB_REMOVE_EXPIRED_ITEMS` | `0`\|`1` | Enables [Time to Live (TTL)](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/TTL.html) feature | ### ECR | Variable | Example Values | Description | | - | - | - | | `ECR_ENDPOINT_STRATEGY` | `domain` (default)\|`off`\| | Governs how the default ECR endpoints are returned | ### ECS | Variable | Example Values | Description | | - | - | - | | `ECS_REMOVE_CONTAINERS` | `0`\|`1` (default) | Remove Docker containers associated with ECS tasks after execution. Disabling this and dumping container logs might help with troubleshooting failing ECS tasks. | | `ECS_DOCKER_FLAGS` | `--privileged`, `--dns 1.2.3.4` | Additional flags passed to Docker when creating ECS task containers. Same restrictions as `LAMBDA_DOCKER_FLAGS`. | | `ECS_DISABLE_AWS_ENDPOINT_URL` | `0` (default) \| `1` | Whether to disable injecting the environment variable `AWS_ENDPOINT_URL`, which automatically configures [supported AWS SDKs](https://docs.aws.amazon.com/sdkref/latest/guide/feature-ss-endpoints.html). | | `ECS_TASK_EXECUTOR` | `kubernetes` | Whether to run ECS tasks when LocalStack is deployed on Kubernetes. Tasks are added to ELB load balancer target groups. | ### EC2 | Variable | Example Values | Description | | - | - | - | | `EC2_DOCKER_FLAGS` | `--privileged` | Additional flags passed to Docker when launching containerized instances. Same restrictions as `LAMBDA_DOCKER_FLAGS`. | | `EC2_DOCKER_INIT` | `0`\|`1` (default) | Start container instances with docker-init system, learn more [here](https://docs.docker.com/reference/cli/docker/container/run/#init). Disable this if you want to use a custom init system. | | `EC2_DOWNLOAD_DEFAULT_IMAGES` | `0`\|`1` (default) | At startup, LocalStack for AWS downloads latest Ubuntu images from Docker Hub for use as AMIs. This can be disabled for security reasons. | | `EC2_EBS_MAX_VOLUME_SIZE` | `1000` (default) | Maximum size (in MiBs) of user-specified EBS block devices mounted into EC2 container instances. | | `EC2_MOUNT_BLOCK_DEVICES` | `1`\|`0` (default) | Whether to create and mount user-specified EBS block devices into EC2 container instances. | | `EC2_REMOVE_CONTAINERS` | `0`\|`1` (default) | Controls whether created Docker containers are removed at instance termination or LocalStack shuts down. Disable this if there is a need to examine the container filesystem for debugging. | | `EC2_VM_MANAGER` | `docker` (default) \| `kubernetes` (Enterprise) \| `mock` | Emulation method to use in LocalStack for AWS. The `kubernetes` value runs instances as Kubernetes pods. | ### EKS | Variable | Example Values | Description | | - | - | - | | `K3S_FLAGS` | | Customize the `k3s` cluster created by LocalStack to emulate EKS clusters. (formerly `EKS_K3S_FLAGS`, still accepted as a deprecated alias) | | `EKS_LOADBALANCER_PORT` | `8081` (default) | Local port on which the Kubernetes load balancer is exposed on the host. | | `K3S_IMAGE_TAG` | `v1.31.5-k3s1` (default) | Custom tag of the `rancher/k3s` image used to spin up Kubernetes clusters locally. (formerly `EKS_K3S_IMAGE_TAG`, still accepted as a deprecated alias) | | `MANAGED_K8S_PROVIDER` | `k3s` (default)\|`local` | The k8s provider which should be used to start the k8s cluster backing EKS. For more information on the providers, please see [Elastic Kubernetes Service (EKS)](/aws/services/eks) (formerly `EKS_K8S_PROVIDER`, still accepted as a deprecated alias) | | `K3S_IMAGE_REPOSITORY` | `rancher/k3s` (default) | Custom repository of the `rancher/k3s` image used to spin up Kubernetes clusters locally. (formerly `EKS_K3S_IMAGE_REPOSITORY`, still accepted as a deprecated alias) | | `K3D_START_LB_INGRESS` | `0` (default) | Whether to start the k3d load balancer and Traefik ingress controller automatically when creating an EKS cluster. Set to `1` to enable. (formerly `EKS_START_K3D_LB_INGRESS`, still accepted as a deprecated alias) | | `EKS_PERSIST_CLUSTER_CONTENTS` | `0` (default) | When Persistence is enabled or when saving/loading Cloud Pods, this flag can be used to control whether the content deployed to EKS clusters will be persisted. Set to `1` to enable. | | `K3D_CLUSTER_TOKEN` | `localstack-k3d-cluster-token` (default) | Token used to authenticate agent nodes joining a k3d-backed EKS cluster. Setting an explicit token ensures consistent node authentication across k3d versions, which is required for dynamic agent assignment. (formerly `EKS_K3D_CLUSTER_TOKEN`, still accepted as a deprecated alias) | | `K3D_VERSION` | `v5.9.0` (default) | Overrides the k3d CLI version used to manage EKS clusters. (formerly `EKS_K3D_VERSION`, still accepted as a deprecated alias) | :::note The EKS configuration variables were renamed to cloud-agnostic names since they're shared across cloud emulators (AWS EKS / Azure AKS). The previous `EKS_*`, `LOCALSTACK_K8S_*`, and `LAMBDA_K8S_*` names still work as deprecated aliases and will be removed in a future release. If you configure these options through the LocalStack CLI **v1**, keep the `LOCALSTACK_` prefix on the new names (e.g. `LOCALSTACK_MANAGED_K8S_PROVIDER`). The CLI v1 only forwards host environment variables prefixed with `LOCALSTACK_` into the container, where the prefix is stripped to yield the in-container variable (`MANAGED_K8S_PROVIDER`). ::: ### ElastiCache | Variable | Example Values | Description | | - | - | - | | `PROVIDER_OVERRIDE_ELASTICACHE` | `legacy` | Use the legacy ElastiCache provider. | | `REDIS_CONTAINER_MODE` | `1`\|`0` (default) | Start ElastiCache cache nodes in separate containers instead of in the LocalStack container | | `ELASTICACHE_DOCKER_FLAGS` | `-v /certs:/certs` | Additional flags passed to Docker when creating ElastiCache containers. Same restrictions as `LAMBDA_DOCKER_FLAGS`. | ### ElasticSearch :::note Also see [OpenSearch configuration variables](#opensearch) which are used to manage both OpenSearch and ElasticSearch clusters. ::: | Variable | Example Values | Description | | - | - | - | | `IGNORE_ES_DOWNLOAD_ERRORS` | `0`\|`1` | Whether to ignore errors (e.g., network/SSL) when downloading ElasticSearch plugins | ### Glue | Variable | Example Values | Description | | - | - | - | | `GLUE_JOB_EXECUTOR` | `docker` (default) \| `kubernetes` | Whether to run Glue jobs when LocalStack is deployed on Kubernetes. Jobs are run as pods in the Kubernetes cluster. | | `DOCKER_GLOBAL_IMAGE_PREFIX` | | Specify custom images for Glue jobs by configuring their custom image repository. | | `GLUE_DOCKER_FLAGS` | `-v /certs:/certs` | Additional flags passed to Docker when creating Glue job containers. Same restrictions as `LAMBDA_DOCKER_FLAGS`. | ### IAM | Variable | Example Values | Description | | - | - | - | | `ENFORCE_IAM` (pro) | `0` (default)\|`1` | Enable IAM policy evaluation and enforcement. If this is disabled (the default), IAM policies will have no effect to your requests. | | `IAM_SOFT_MODE` (pro) | `0` (default)\|`1` | Enable IAM soft mode. This leads to policy evaluation without actually denying access. Needs `ENFORCE_IAM` enabled as well. For more information, see [Identity and Access Management](/aws/services/iam).| ### Kinesis | Variable | Example Values | Description | | - | - | - | | `KINESIS_ERROR_PROBABILITY` | Decimal value between `0.0`(default) and `1.0` | Randomly inject `ProvisionedThroughputExceededException` errors into Kinesis API responses. | | `KINESIS_SHARD_LIMIT` | `100` (default), `Infinity` (to disable) | Integer value , causing the Kinesis API to start throwing exceptions to mimic the default shard limit. | | `KINESIS_ON_DEMAND_STREAM_COUNT_LIMIT` | `10` (default), `Infinity` (to disable) | Integer value , causing the Kinesis API to start throwing exceptions to mimic the default on demand stream count limit. | | `KINESIS_LATENCY` | `500` (default), `0` (to disable)| Integer value of milliseconds, causing the Kinesis API to delay returning a response in order to mimic latency from a live AWS call. | | `KINESIS_MOCK_PROVIDER_ENGINE` | `node` (default) \| `scala` | String value of `node` (default) or `scala` that determines the underlying build of Kinesis Mock. | | `KINESIS_MOCK_MAXIMUM_HEAP_SIZE` | `512m` (default) | JVM memory format string that sets the maximum memory size for the Kinesis Mock Scala server, corresponds to the JVM `-Xmx` flag. | | `KINESIS_MOCK_INITIAL_HEAP_SIZE` | `256m` (default) | JVM memory format string that sets the initial memory size for the Kinesis Mock Scala server, corresponds to the JVM `-Xms` flag. | ### Kinesis Analytics | Variable | Example Values | Description | | - | - | - | | `KINESISANALYTICSV2_DOCKER_FLAGS` | `-v /certs:/certs` | Additional flags passed to Docker when creating Kinesis Analytics v2 (Flink) containers. Same restrictions as `LAMBDA_DOCKER_FLAGS`. | ### Kafka | Variable | Example Values | Description | | - | - | - | | `KAFKA_DOCKER_FLAGS` | `-v /certs:/certs` | Additional flags passed to Docker when creating Kafka/MSK containers. Same restrictions as `LAMBDA_DOCKER_FLAGS`. | ### Lambda :::note The legacy [Lambda](/aws/services/lambda) implementation has been removed since LocalStack 3.0 (Docker `latest` since 2023-11-09). Please consult the [migration guide](/aws/services/lambda#migrating-to-lambda-v2) for more information. ::: | Variable| Example Values | Description | | - | - | - | | `BUCKET_MARKER_LOCAL` | `hot-reload` (default) | Magic S3 bucket name for [Hot Reloading](/aws/developer-tools/lambda-tools/hot-reloading). The S3Key points to the source code on the local file system. | | `HOSTNAME_FROM_LAMBDA` | `localstack` | Endpoint host under which APIs are accessible from Lambda containers (optional). This can be useful in docker-compose stacks to use the local container hostname if neither IP address nor container name of the main container are available (e.g., in CI). Often used in combination with `LAMBDA_DOCKER_NETWORK`.| | `LAMBDA_DISABLE_AWS_ENDPOINT_URL` | `0` (default) \| `1` | Whether to disable injecting the environment variable `AWS_ENDPOINT_URL`, which automatically configures [supported AWS SDKs](https://docs.aws.amazon.com/sdkref/latest/guide/feature-ss-endpoints.html). | | `LAMBDA_DISABLE_JAVA_SDK_V2_CERTIFICATE_VALIDATION` | `1` (default) | Whether to disable the certificate name validation for [AWS Java SDK v2](https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/home.html) calls when using [transparent endpoint injection](/aws/customization/networking/transparent-endpoint-injection).| | `LAMBDA_DOCKER_DNS` | `""` (default) | Optional custom DNS server for the container running your Lambda function. Overwrites the default LocalStack [DNS Server](/aws/customization/networking/dns-server). Hence, resolving `localhost.localstack.cloud` requires additional configuration. | | `LAMBDA_DOCKER_FLAGS` | `-e KEY=VALUE`, `-v host:container`, `-p host:container`, `--add-host domain:ip` | Additional flags passed to Docker `run`\|`create` commands. Supports environment variables (also with `--env-file`, but the file has to be mounted into the LocalStack container), ports, volume mounts, extra hosts, networks, DNS servers, labels, ulimits, user, platform, and privileged mode. The `--env-file` argument for Docker `run` and Docker Compose have different feature sets. To provide both, we support the `--env-file` for environment files with the docker run syntax, while `--compose-env-file` supports the full docker compose features, like placeholders with `${}`, replacing quotes, etc. | | `LAMBDA_DOCKER_NETWORK` | `bridge` (Docker default) | [Docker network driver](https://docs.docker.com/network/) for the Lambda and ECS containers. Needs to be set to the network the LocalStack container is connected to. Limitation: `host` mode currently not supported. | | `LAMBDA_DOWNLOAD_AWS_LAYERS` | `1` (default, pro) | Whether to download public Lambda layers from AWS through a LocalStack proxy when creating or updating functions. | | `LAMBDA_EVENT_SOURCE_MAPPING_SQS_POLLER_COUNT` | `5` (default) | Number of concurrent pollers spawned per SQS event source mapping for standard queues, each running in its own thread. Capped by `ScalingConfig.MaximumConcurrency` when set on the event source mapping. FIFO queues always use a single poller to preserve message-group ordering. | | `LAMBDA_IGNORE_ARCHITECTURE` | `0` (default) | Whether to ignore the AWS architectures (x86_64 or arm64) configured for the lambda function. Set to `1` to run cross-platform compatible lambda functions natively (i.e., Docker selects architecture). | | `LAMBDA_K8S_IMAGE_PREFIX` | `amazon/aws-lambda-` (default, enterprise) | Prefix for images that will be used to execute Lambda functions in Kubernetes. | | `LAMBDA_K8S_INIT_IMAGE` | | Specify the image for downloading the init binary from LocalStack. The image must include the `curl` and `chmod` commands. This is only relevant for container-based Lambdas on Kubernetes | | `LAMBDA_KEEPALIVE_MS` | `600000` (default 10min) | Time in milliseconds until lambda shuts down the execution environment after the last invocation has been processed. Set to `0` to immediately shut down the execution environment after an invocation. | | `LAMBDA_LIMITS_CONCURRENT_EXECUTIONS` | `1000` (default) | The maximum number of events that functions can process simultaneously in the current Region. See [AWS service quotas](https://docs.aws.amazon.com/general/latest/gr/lambda-service.html) | | `LAMBDA_LIMITS_CODE_SIZE_ZIPPED` | `52428800` (default) | The maximum zip file size in bytes for the [CreateFunction](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunction.html) operation. Raising this limit enables the creation of larger Lambda functions without the need to upload the code to an S3 deployment bucket. | | `LAMBDA_LIMITS_CREATE_FUNCTION_REQUEST_SIZE` | `70167211` (default) | The maximum HTTP request size in bytes for the [CreateFunction](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunction.html) operation. Raising this limit enables larger HTTP requests including zipped file size. | | `LAMBDA_LIMITS_MAX_FUNCTION_ENVVAR_SIZE_BYTES` | `4096` (default) | The maximum size of the environment variables that you can use to configure your function. | | `LAMBDA_PREBUILD_IMAGES` | `0` (default) | Prebuild images before execution which increases the cold start time but reduces the time until the Lambda function is `ACTIVE`. (preview) | | `LAMBDA_REMOVE_CONTAINERS` | `1` (default) | Whether to remove any Lambda Docker containers. | | `LAMBDA_RUNTIME_ENVIRONMENT_TIMEOUT` | `20` (default) | How many seconds Lambda will wait for the runtime environment to start up. Increase this timeout if I/O is slow or your Lambda deployments are large or contain many files. | | `LAMBDA_RUNTIME_EXECUTOR` | `docker` (default) | Where Lambdas will be executed. | | | `kubernetes` (enterprise) | Execute lambdas in a Kubernetes cluster. | | `LAMBDA_RUNTIME_IMAGE_MAPPING` | [base images for Lambda](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-images.html) (default) | Customize the Docker image of Lambda runtimes, either by:
a) pattern with `` placeholder, e.g. `custom-repo/lambda-:2022`
b) json dict mapping the `` to an image, e.g. `{"python3.9": "custom-repo/lambda-py:thon3.9"}` | | `LAMBDA_RUNTIME_VALIDATION` | `0` (default) | Set to `1` to enforce strict [AWS parity](https://blog.localstack.cloud/2022-08-04-parity-explained/) by raising an exception when using a deprecated [Lambda runtime](https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html) for the API operation [CreateFunction](https://docs.aws.amazon.com/lambda/latest/api/API_CreateFunction.html). Deprecated Lambda runtimes (e.g., `nodejs14.x`) can be used with disabled validation (current default). | | `LAMBDA_SYNCHRONOUS_CREATE` | `0` (default) | Set to `1` to create lambda functions synchronously (not recommended). | | `LAMBDA_TRUNCATE_STDOUT` | `2000` (default) | Allows increasing the default char limit for truncation of lambda log lines when printed in the console. This does not affect the logs processing in CloudWatch. | ### MemoryDB | Variable | Example Values | Description | | - | - | - | | `REDIS_CONTAINER_MODE` | `1`\|`0` (default) | Start MemoryDB cluster nodes in separate containers instead of in the LocalStack container | | `MEMORYDB_DOCKER_FLAGS` | `-v /certs:/certs` | Additional flags passed to Docker when creating MemoryDB containers. Same restrictions as `LAMBDA_DOCKER_FLAGS`. | ### MWAA | Variable | Example Values | Description | | - | - | - | | `MWAA_PIP_TRUSTED_HOSTS` | `pypi.org,files.pythonhosted.org` | Comma-separated list of hosts for which SSL verification is not performed when installing Python dependencies for MWAA environment. | | `MWAA_S3_POLL_INTERVAL` | `30` (default) | Interval in seconds with which MWAA polls S3 bucket to check for new or updated assets. | | `MWAA_DOCKER_FLAGS` | `-e TEST_ENV=1337` | Additional flags provided to the Airflow container. Same restrictions as `LAMBDA_DOCKER_FLAGS`. | ### MQ | Variable | Example Values | Description | | - | - | - | | `MQ_DOCKER_FLAGS` | `-v /certs:/certs` | Additional flags passed to Docker when creating Amazon MQ broker containers. Same restrictions as `LAMBDA_DOCKER_FLAGS`. | ### Neptune | Variable | Example Values | Description | | - | - | - | | `NEPTUNE_DB_TYPE` | `neo4j`\|`tinkerpop` (default) | Starts Neptune DB as traditional netpune with Tinkerpop/Gremlin (default) or in Neo4J mode. | | `NEPTUNE_ENABLE_TRANSACTION` | `1`\|`0` (default) | Enables Gremlin transaction. This is an experimental feature, [see notes](/aws/services/neptune#gremlin-transactions) | | `NEPTUNE_GREMLIN_DEBUG` | `1`\|`0` (default) | Enable Gremlin logs | | `NEPTUNE_USE_SSL` | `1`\|`0` (default) | Whether to start the Neptune server with SSL configuration, which will enable wss protocol. This setting is only valid for Tinkerpop/Gremlin. By default SSL is not enabled. | ### OpenSearch | Variable | Example Values | Description | | - | - | - | | `OPENSEARCH_CUSTOM_BACKEND` | `http://opensearch:9200` | URL to a custom OpenSearch backend cluster. If this is set to a valid URL, then LocalStack will not create OpenSearch cluster instances, but instead forward all domains to the given backend (see [Custom Opensearch Backends](/aws/services/opensearch#custom-opensearch-backends). | | `OPENSEARCH_MULTI_CLUSTER` | `1`\| `0` | When activated, LocalStack will spawn one OpenSearch cluster per domain. Otherwise all domains will share a single cluster instance. This is ignored if `OPENSEARCH_CUSTOM_BACKEND` is set. | | `OPENSEARCH_ENDPOINT_STRATEGY` | `path`\|`domain`\|`port` | Governs how domain endpoints are created to access a cluster (see [Opensearch Endpoints](/aws/services/opensearch#domain-endpoints)). | | `SKIP_INFRA_DOWNLOADS` | `1` \| `0` (default) | **Deprecated since 1.3.0** Whether to skip downloading additional infrastructure components (e.g., specific Elasticsearch versions) | | `IGNORE_OS_DOWNLOAD_ERRORS` | `0`\|`1` | Whether to ignore errors (e.g., network/SSL) when downloading OpenSearch plugins | ### RDS | Variable | Example Values | Description | | - | - | - | | `RDS_CLUSTER_ENDPOINT_HOST_ONLY` | `1` (default) \| `0` | Whether the cluster endpoint returns the host only (which is AWS parity). If set to `0` it will return `:`. | | `RDS_PG_CUSTOM_VERSIONS` | `0` \| `1` (default) | Whether to install and use custom Postgres versions for RDS (or alternatively, use default version 15). | | `RDS_MYSQL_DOCKER` | `1` (default) \| `0` | Whether to disable MySQL engines (and use MariaDB instead). MySQL engine for cluster/instances will start in a new docker container. If you have troubles running MySQL in docker, you can disable the feature. | | `MYSQL_IMAGE` | `mysql:8.0` | Defines a specific MySQL image that should be used when spinning up the MySQL engine. Only available if `RDS_MYSQL_DOCKER` is enabled. | | `MSSQL_IMAGE` | `mcr.microsoft.com/mssql/server:2022-latest` | Defines a specific image that should be used when spinning up a SQL server engine. | | `MSSQL_ACCEPT_EULA` | `Y` | Set to `Y` if you accept the [EULA from MSSQL](https://hub.docker.com/_/microsoft-mssql-server). | | `RDS_PG_MAX_CONNECTIONS` | `0` (default) | Sets the maximum number of connections for Postgres RDS instances. | | `RDS_DOCKER_FLAGS` | `-v /certs:/certs` | Additional flags passed to Docker when creating RDS containers. Same restrictions as `LAMBDA_DOCKER_FLAGS`. | ### S3 | Variable | Example Values | Description | | - | - | - | | `S3_SKIP_SIGNATURE_VALIDATION`| `0` \| `1` (default) | Used to toggle validation of S3 pre-signed URLs. Set to `0` to validate their signature and expiration. See [Signature validation](/aws/services/s3/#signature-validation) for the credentials accepted by the validation. | | `S3_VALIDATE_SIGNATURES` | `0` (default) \| `1` | Used to toggle SigV4 signature validation of regular (non-pre-signed) S3 requests. Set to `1` to validate the request signature and the payload integrity. See [Signature validation](/aws/services/s3/#signature-validation) for the credentials accepted by the validation. | | `S3_SKIP_KMS_KEY_VALIDATION` | `0` \| `1` (default) | Used to toggle validation of provided KMS key in S3 operations. | ### SageMaker | Variable | Example Values | Description | | - | - | - | | `SAGEMAKER_DOCKER_FLAGS` | `-v /certs:/certs` | Additional flags passed to Docker when creating SageMaker training and endpoint containers. Same restrictions as `LAMBDA_DOCKER_FLAGS`. | ### SNS | Variable | Example Values | Description | | - | - | - | | `SNS_SES_SENDER_ADDRESS` | `no-reply@sns.localstack.cloud` \| `admin@localstack.com` (default) | The email address used to send SNS `email` and `email-json` notifications. | ### SQS | Variable | Example Values | Description | | - | - | - | | `SQS_DELAY_PURGE_RETRY` | `0` (default) | Used to toggle PurgeQueueInProgress errors when making more than one PurgeQueue call within 60 seconds. | | `SQS_DELAY_RECENTLY_DELETED` | `0` (default) | Used to toggle QueueDeletedRecently errors when re-creating a queue within 60 seconds of deleting it. | | `SQS_ENABLE_MESSAGE_RETENTION_PERIOD`| `0` (default) \| `1` | Used to toggle the MessageRetentionPeriod feature (see [Enabling `MessageRetentionPeriod`](/aws/services/sqs/#enabling-messageretentionperiod) | | `SQS_ENDPOINT_STRATEGY`| `standard` (default) \| `domain` \| `path` \| `off` | Configures the format of Queue URLs (see [SQS Queue URLs](/aws/services/sqs/#queue-urls) | | `SQS_DISABLE_CLOUDWATCH_METRICS` | `0` (default) | Disables the CloudWatch Metrics for SQS when set to `1` | | `SQS_CLOUDWATCH_METRICS_REPORT_INTERVAL` | `60` (default) | Configures the report interval (in seconds) for `Approximate*` metrics that are sent to CloudWatch periodically. Sending will be disabled if `SQS_DISABLE_CLOUDWATCH_METRICS=1` | ### Step Functions | Variable | Example Values | Description | | - | - | - | | `SFN_MOCK_CONFIG` | `/tmp/MockConfigFile.json` | Specifies the file path to the mock configuration file that defines mock service integrations for Step Functions. | ### Verified Permissions | Variable | Example Values | Description | | - | - | - | | `VERIFIEDPERMISSIONS_DISABLE_JWT_VERIFICATION` | `0` (default) \| `1` | Disables JWT signature verification for OIDC identity sources. When enabled, LocalStack will decode tokens without validating signatures against the issuer's JWKS, allowing use of unreachable or self-signed OIDC providers in local development. | ## Security :::danger Please be aware that the following options may have severe security implications. ::: | Variable| Example Values | Description | | - | - | - | | `DISABLE_CORS_HEADERS` | `0` (default) | Whether to disable the returning of default CORS headers in API responses (disables access from https://app.localstack.cloud). | | `DISABLE_CORS_CHECKS` | `0` (default) | Whether to disable all CSRF (server-side) mitigations. | | `DISABLE_CUSTOM_CORS_S3` | `0` (default) | Whether to disable CORS override by S3. | | `DISABLE_CUSTOM_CORS_APIGATEWAY` | `0` (default)| Whether to disable CORS override by apigateway. | | `EXTRA_CORS_ALLOWED_ORIGINS` | | Comma-separated list of origins that are allowed to communicate with localstack. | | `EXTRA_CORS_ALLOWED_HEADERS` | | Comma-separated list of header names to be be added to Access-Control-Allow-Headers CORS header. | | `EXTRA_CORS_EXPOSE_HEADERS` | | Comma-separated list of header names to be be added to Access-Control-Expose-Headers CORS header. | | `ENABLE_CONFIG_UPDATES` | `0` (default) | Whether to enable dynamic configuration updates at runtime. | ## Emails Please check with your SMTP email service provider for the following settings. | Variable | Example Values | Description | | - | - | - | | `SMTP_HOST` | `localhost:1025` | Hostname (and optionally the port) of the SMTP server. The port defaults to 25. | | `SMTP_USER` | | Login username for the SMTP server if required. | | `SMTP_PASS` | | Login password for the SMTP server if required. | | `SMTP_EMAIL` | `sender@example.com` | Origin email address. Required for Cognito only. | ## Persistence To learn more about these configuration options, see [Persistence](/aws/developer-tools/snapshots/persistence). | Variable | Valid options | Description | | - | - | - | | `SNAPSHOT_SAVE_STRATEGY` | `ON_SHUTDOWN`\|`ON_REQUEST`\|`SCHEDULED`\|`MANUAL` | Strategy that governs when LocalStack should make state snapshots | | `SNAPSHOT_LOAD_STRATEGY` | `ON_STARTUP`\|`ON_REQUEST`\|`MANUAL` | Strategy that governs when LocalStack restores state snapshots | | `SNAPSHOT_FLUSH_INTERVAL` | 15 (default) | The interval (in seconds) between persistence snapshots. It only applies to a `SCHEDULED` save strategy (see [Persistence Mechanism](/aws/developer-tools/snapshots/persistence))| | `DISABLE_COMPATIBILITY_RULES` | `0` (default) \| `1` | Disable the [snapshot compatibility rules](/aws/developer-tools/snapshots/service-coverage#snapshot-compatibility) that prevent loading incompatible state into LocalStack. Applies to both snapshot persistence and Cloud Pods. | ## Cloud Pods To learn more about these configuration options, see [Cloud Pods](/aws/developer-tools/snapshots/cloud-pods). | Variable | Valid options | Description | | - | - | - | | `AUTO_LOAD_POD` | | Comma-separated list of Cloud Pods to be automatically loaded at startup time. This feature is disabled when snapshot persistence is set via the `PERSISTENCE` variable. | | `POD_LOAD_CLI_TIMEOUT` | 60 (default) | Timeout in seconds to wait before returning from load operations on the Cloud Pods CLI | | `POD_ENCRYPTION` | `0` (default) \| `1` | Whether to encrypt the Cloud Pods artifacts at rest. | | `ENABLE_POD_RESOURCES=1` | `0` (default) \| `1` | Whether to save a detailed Stack Overview including available resources for the Cloud Pod | | `MERGE_STRATEGY` | `account-region-merge` (default) \| `service-merge` \| `overwrite` | The merge strategy to apply when loading a Cloud Pod into LocalStack (see [merging snapshots](/aws/developer-tools/snapshots/merging-snapshots/)) | ## Extensions | Variable | Example Values | Description | | - | - | - | | `EXTENSION_AUTO_INSTALL` | | Install a list of extensions automatically at startup. Comma-separated list of extensions directives which will be installed automatically at startup (see [managing extensions](/aws/customization/integrations/extensions/managing-extensions#automating-extensions-installation))| ## Miscellaneous | Variable | Example Values | Description | | - | - | - | | `SKIP_SSL_CERT_DOWNLOAD` | | Whether to skip downloading the SSL certificate for localhost.localstack.cloud | | `CUSTOM_SSL_CERT_PATH` | `/var/lib/localstack/custom/server.test.pem` | Defines the absolute path to a custom SSL certificate for localhost.localstack.cloud | | `OVERRIDE_IN_DOCKER` | | Overrides the check whether LocalStack is executed within a docker container. If set to `true`, LocalStack assumes it runs in a docker container. Should not be set unless necessary. | | `DISABLE_EVENTS` | `1` | Whether to disable publishing LocalStack events | | `OUTBOUND_HTTP_PROXY` | `http://10.10.1.3` | HTTP Proxy used for downloads of runtime dependencies and connections outside LocalStack itself | | `OUTBOUND_HTTPS_PROXY` | `https://10.10.1.3` | HTTPS Proxy used for downloads of runtime dependencies and connections outside LocalStack itself | | `REQUESTS_CA_BUNDLE` | `/var/lib/localstack/lib/ca_bundle.pem` | CA Bundle to be used to verify HTTPS requests made by LocalStack | | `DOCKER_HOST` | `unix:///var/run/docker.sock` (default) | Daemon socket to connect Docker. Used by the LocalStack dependency [Docker](https://docs.docker.com/engine/reference/commandline/cli/#environment-variables). | | `SSL_NO_VERIFY` | `0` \| `1` | Disables TLS verification when making requests to the LocalStack Cloud, including the license server, and when fetching the certificate for localhost.localstack.cloud. | ## Debugging | Variable | Example Values | Description | | - | - | - | | `DEVELOP` | | Starts a debugpy server before starting LocalStack services | `DEVELOP_PORT` | | Port number for debugpy server | `WAIT_FOR_DEBUGGER` | | Forces LocalStack to wait for a debugger to start the services ## DNS To learn more about these configuration options, see [DNS Server](/aws/customization/networking/dns-server). | Variable | Example Values | Description | | - | - | - | | `DNS_ADDRESS` | `0.0.0.0` (default) | Address the LocalStack should bind the DNS server on (port 53 tcp/udp). Value `0` to disable. | `DNS_SERVER` | Default upstream DNS or `8.8.8.8` (default) | Fallback DNS server for queries not handled by LocalStack. | `DNS_RESOLVE_IP` | `127.0.0.1` (default) | IP address the DNS server should return as A record for queries handled by LocalStack. If customized, this value will be returned in preference to the DNS server response. | `DNS_NAME_PATTERNS_TO_RESOLVE_UPSTREAM` | `([^.]+\.)*(ecr\|lambda)\.[^.]+\.amazonaws\.com` (example) | List of domain names that should *NOT* be redirected by the LocalStack DNS to the LocalStack container, but instead always forwarded to the upstream resolver. This will *NOT* redirect requests made to LocalStack due to manual endpoint configuration. Comma-separated list of Python-flavored regex patterns. See [the DNS server documentation](/aws/customization/networking/dns-server#skip-localstack-dns-resolution) for more details. | `DNS_LOCAL_NAME_PATTERNS` | `([^.]+\.)*(ecr\|lambda)\.[^.]+\.amazonaws\.com` (example) | **Deprecated since 3.0.2** List of domain names that should *NOT* be redirected by the LocalStack DNS to the LocalStack container, but instead always forwarded to the upstream resolver. This will *NOT* redirect requests made to LocalStack due to manual endpoint configuration. Comma-separated list of Python-flavored regex patterns. **Renamed to `DNS_NAME_PATTERNS_TO_RESOLVE_UPSTREAM`** ## Transparent Endpoint Injection | Variable | Example Values | Description | | - | - | - | | `DISABLE_TRANSPARENT_ENDPOINT_INJECTION` | `0` (default in Pro) \| `1` | Whether to disable DNS resolution of AWS hostnames to the LocalStack container. Pro feature. (see [Transparent Endpoint Injection](/aws/customization/networking/transparent-endpoint-injection)) ## LocalStack for AWS | Variable | Example Values | Description | |----------------------|----------------|-------------| | `LOCALSTACK_AUTH_TOKEN` | | [Auth token](/aws/getting-started/auth-token) to activate LocalStack for AWS. | | `LOCALSTACK_API_KEY` | | **Deprecated since 3.0.0** API key to activate LocalStack for AWS.
**Use the `LOCALSTACK_AUTH_TOKEN` instead (except for [CI environments](/aws/ci-pipelines/)).** | | `LOG_LICENSE_ISSUES` | `0` \| `1` (default) | Whether to log issues with the license activation to the console. | ## Legacy These configurations have already been removed and **won't have any effect** on newer versions of LocalStack. **Please remove them from your configuration.** | Variable | Removed in | Example Values | Description | | - | - | - | - | | `ACTIVATE_PRO` | 2026.7.0 | `0` \| `1` (default) | Whether Pro should be activated or not. Previously allowed starting LocalStack without Pro features when set to `0`. The flag had no remaining effect and was removed; use the community image (`localstack/localstack`) if you do not need Pro features. | | `EVENT_RULE_ENGINE` | 4.1.0 | `python` (default) \| `java` (deprecated) | Engine for [event pattern matching](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-event-patterns-content-based-filtering.html) used in EventBridge, EventBridge Pipes, and Lambda Event Source Mapping. Our new `python` implementation introduced with version `4.0.3` makes the `java` engine (previously in preview) using AWS [event-ruler](https://github.com/aws/event-ruler) obsolete. | | `LAMBDA_EVENT_SOURCE_MAPPING` | 4.0.0 | `v2` (default since [3.8.0](https://blog.localstack.cloud/localstack-release-v-3-8-0/#new-default-lambda-event-source-mapping-implementation)) \| `v1` | Feature flag to switch Lambda Event Source Mapping (ESM) implementations. | | `PROVIDER_OVERRIDE_STEPFUNCTIONS` | 4.0.0 | `v2` (default) \| `legacy` | The new LocalStack-native StepFunctions provider (v2) is active by default since LocalStack 3.0. | | `PROVIDER_OVERRIDE_EVENTS` | 2026.04.0 | `v2` (default) \| `legacy` | The new EventBridge provider is active by default since LocalStack 4.0. Remove this variable from your configuration, as any value now points to an invalid provider configuration and prevents the `events` provider from loading. | | `STEPFUNCTIONS_LAMBDA_ENDPOINT` | 4.0.0 | `default` | This is only supported for the `legacy` provider. URL to use as the Lambda service endpoint in Step Functions. By default this is the LocalStack Lambda endpoint. Use default to select the original AWS Lambda endpoint. | | `LOCAL_PORT_STEPFUNCTIONS` | 4.0.0 | `8083` (default) | This is only supported for the legacy provider. It defines the local port to which Step Functions traffic is redirected. By default, LocalStack routes Step Functions traffic to its internal runtime. Use this variable only if you need to redirect traffic to a different local Step Functions runtime. | | `S3_DIR` | 4.0.0 | `/path/to/root` | This was only supported for the `legacy_v2` provider. Configure a global parent directory that contains all buckets as sub-directories (`S3_DIR=/path/to/root`) or an individual directory that will get mounted as special bucket names (`S3_DIR=/path/to/root/bucket1:bucket1`). Only available for LocalStack for AWS. | `_BACKEND` | 3.0.0 | `http://localhost:7577` | Custom endpoint URL to use for a specific service, where `` is the uppercase service name. | | `_PORT_EXTERNAL` | 3.0.0 | `4567` | Port number to expose a specific service externally . `SQS_PORT_EXTERNAL`, e.g. , is used when returning queue URLs from the SQS service to the client. | | `ACTIVATE_NEW_POD_CLIENT` | 3.0.0 | `0`\|`1` (default) | Whether to use the new Cloud Pods client leveraging LocalStack container's APIs. | | `BIGDATA_MONO_CONTAINER` | 3.0.0 |`0`\|`1` (default) | Whether to spin Big Data services inside the LocalStack main container. Glue jobs breaks when using `BIGDATA_MONO_CONTAINER=0`. | | `DEFAULT_REGION` | 3.0.0 | `us-east-1` (default) | AWS region to use when talking to the API (needs to be activated via `USE_SINGLE_REGION=1`). LocalStack now has full multi-region support. | | `EDGE_BIND_HOST` | 3.0.0 | `127.0.0.1` (default), `0.0.0.0` (docker)| Address the edge service binds to. Use `GATEWAY_LISTEN` instead. | | `EDGE_FORWARD_URL` | 3.0.0 | `http://10.0.10.5678` | Optional target URL to forward all edge requests to (e.g., for distributed deployments) | | `EDGE_PORT` | 3.0.0 | `4566` (default)| Port number for the edge service, the main entry point for all API invocations. | | `EDGE_PORT_HTTP` | 3.0.0 | `4566` (default)| Port number for the edge service, the main entry point for all API invocations. | | `ES_CUSTOM_BACKEND` | 3.0.0 | `http://elasticsearch:9200` | Use [`OPENSEARCH_CUSTOM_BACKEND`](#opensearch) instead. URL to a custom elasticsearch backend cluster. If this is set to a valid URL, then LocalStack will not create elasticsearch cluster instances, but instead forward all domains to the given backend (see [Custom Elasticsearch Backends](/aws/services/es#custom-elasticsearch-backends)). | | `ES_ENDPOINT_STRATEGY` | 3.0.0 | `path`\|`domain`\|`port` (formerly `off`) | Use [`OPENSEARCH_ENDPOINT_STRATEGY`](#opensearch) instead. Governs how domain endpoints are created to access a cluster (see [Elasticsearch Endpoints](/aws/services/es#endpoints)) | | `ES_MULTI_CLUSTER` | 3.0.0 | `0`\|`1` | Use [`OPENSEARCH_MULTI_CLUSTER`](#opensearch) instead. When activated, LocalStack will spawn one Elasticsearch cluster per domain. Otherwise all domains will share a single cluster instance. This is ignored if `ES_CUSTOM_BACKEND` is set. | | `HOSTNAME_EXTERNAL` | 3.0.0 | `localhost` (default) | Name of the host to expose the services externally. This host is used, e.g., when returning queue URLs from the SQS service to the client. Use `LOCALSTACK_HOST` instead. | | `KINESIS_INITIALIZE_STREAMS` | 3.0.0 | `"my-first-stream:1,my-other-stream:2:us-west-2,my-last-stream:1"` | A comma-delimited string of stream names, its corresponding shard count and an optional region to initialize during startup. If the region is not provided, the default region is used. Only works with the `kinesis-mock` `KINESIS_PROVIDER`. | | `KINESIS_PROVIDER` | 3.0.0 | `kinesis-mock` (default) and `kinesalite` | | | `KMS_PROVIDER` | 3.0.0 | `moto` (default), `local-kms` | `local-kms` has been removed. | | `LAMBDA_CODE_EXTRACT_TIME` | 3.0.0 | `25` (default) | Time in seconds to wait at max while extracting Lambda code. By default, it is 25 seconds for limiting the execution time to avoid client/network timeout issues.
**Removed in new provider because function creation happens asynchronously.**| | `LAMBDA_CONTAINER_REGISTRY` | 3.0.0 | `lambci/lambda` (default) | An alternative docker registry from where to pull lambda execution containers.
**Replaced by `LAMBDA_RUNTIME_IMAGE_MAPPING` in new provider.** | | `LAMBDA_EXECUTOR` | 3.0.0 | | Method to use for executing Lambda functions. For `docker` and `docker-reuse`, if LocalStack itself is started inside Docker, then the `docker` command needs to be available inside the container (usually requires to run the container in privileged mode). More information in Lambda Executor Modes.
**Removed in new provider. Mount the Docker socket or see [migration guide](/aws/services/lambda).** | | | | `docker` (default) | Run each function invocation in a separate Docker container. | | | | `local` (fallback) | Run Lambda functions in a temporary directory on the local machine. | | | | `docker-reuse` | Create one Docker container per function and reuse it across invocations. | | `LAMBDA_FALLBACK_URL` | 3.0.0 | | Fallback URL to use when a non-existing Lambda is invoked. Either records invocations in DynamoDB (value `dynamodb://`) or forwards invocations as a POST request (value `http(s)://...`).
**Removed in new provider.** | | `LAMBDA_FORWARD_URL` | 3.0.0 | | URL used to forward all Lambda invocations (useful to run Lambdas via an external service).
**Removed in new provider.** | | `LAMBDA_JAVA_OPTS` | 3.0.0 | `-Xmx512M` | Allow passing custom JVM options to Java Lambdas executed in Docker. Use `_debug_port_` placeholder to configure the debug port, e.g., `-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=_debug_port_`.
**Currently not supported in new provider but possible via custom entrypoint.** | | `LAMBDA_REMOTE_DOCKER` | 3.0.0 | | determines whether Lambda code is copied or mounted into containers.
**Removed in new provider because zip file copying is used by default and hot reloading automatically configures mounting.** | | | | `true` (default) | your Lambda function definitions will be passed to the container by copying the zip file (potentially slower). It allows for remote execution, where the host and the client are not on the same machine.| | | | `false` | your Lambda function definitions will be passed to the container by mounting a volume (potentially faster). This requires to have the Docker client and the Docker host on the same machine. | | `LAMBDA_STAY_OPEN_MODE` | 3.0.0 | `1` (default) | Usage of the stay-open mode of Lambda containers. Only applicable if `LAMBDA_EXECUTOR=docker-reuse`. Set to `0` if you want to use [Hot Reloading](/aws/developer-tools/lambda-tools/hot-reloading).
**Removed in new provider because stay-open mode is the default behavior. `LAMBDA_KEEPALIVE_MS` can be used to configure how long containers should be kept running in-between invocations.** | | `LAMBDA_XRAY_INIT` | 3.0.0 | `1` \| `0` (default) | Whether to fully initialize XRay daemon for Lambda containers (may increase Lambda startup times).
**the X-Ray daemon is now always initialized.** | | `LEGACY_EDGE_PROXY` | 3.0.0 | `1` \| `0` (default) | Whether to use the legacy edge proxy or the newer Gateway/HandlerChain framework. | | `LOCALSTACK_HOSTNAME` | 3.0.0 | `http://${LOCALSTACK_HOSTNAME}:4566` | Name of the host where LocalStack services are available. Use this hostname as endpoint in order to access the services from within your Lambda functions (e.g., to store an item to DynamoDB or S3 from a Lambda). This option is read-only. Use `LOCALSTACK_HOST` instead. | | `MOCK_UNIMPLEMENTED` | 3.0.0 | `1` \| `0` (default) | Whether to return mocked success responses (instead of 501 errors) for currently unimplemented API methods | | `PERSIST_ALL` | 3.0.0 | `true` (default) | Whether to persist all resources (including user code like Lambda functions), or only "light-weight" resources (e.g., SQS queues, or Cognito users). Can be set to `false` to reduce storage size of `DATA_DIR` folders or Cloud Pods. | | `SYNCHRONOUS_KINESIS_EVENTS` | 3.0.0 | `1` (default) \| `0` | Whether or not to handle Kinesis Lambda event sources as synchronous invocations. | | `USE_SINGLE_REGION` | 3.0.0 | `1` \| `0` (default) | Whether to use the legacy single-region mode, defined via `DEFAULT_REGION`. | | `DATA_DIR`| 2.0.0 | blank (disabled/default), `/tmp/localstack/data` | Local directory for saving persistent data. Use `PERSISTENCE` instead. | | `DISABLE_TERM_HANDLER` | 2.0.0 | `""` (default) \| `1` | Whether to disable signal passing to LocalStack when running in docker. Enabling this will prevent an orderly shutdown when running inside LS in docker. Setting this to anything else than an empty string will disable it. | `HOST_TMP_FOLDER` | 2.0.0 | `/some/path` | Temporary folder on the host that gets mounted as `$TMPDIR/localstack` into the LocalStack container. Required only for Lambda volume mounts when using `LAMBDA_REMOTE_DOCKER=false.` | | `INIT_SCRIPTS_PATH` | 2.0.0 | `/some/path` | Before 1.0, this was used to configure the path to the initializing files with extensions `.sh` that were found in `/docker-entrypoint-initaws.d`. This has been replaced by the [init-hook system](/aws/customization/advanced/initialization-hooks/). | | `LEGACY_DIRECTORIES` | 2.0.0 | `0` (default) | Use legacy method of managing internal filesystem layout. See [Filesystem Layout](/aws/customization/advanced/filesystem). | | `LEGACY_INIT_DIR` | 2.0.0 | `1` \| `0`(default) | Used with `INIT_SCRIPTS_PATH`. This has been replaced by the [init-hook system](/aws/customization/advanced/initialization-hooks). | | `MULTI_ACCOUNTS` | 2.0.0 | `0` (default) | Enable multi-accounts (preview) | | `SQS_PROVIDER` | 2.0.0 | `moto` (default) and `elasticmq` | | | `SYNCHRONOUS_API_GATEWAY_EVENTS` | 2.0.0 | `1` (default) \| `0` | Whether or not to handle API Gateway Lambda event sources as synchronous invocations. | | `SYNCHRONOUS_DYNAMODB_EVENTS` | 2.0.0 | `1` (default) \| `0` | Whether or not to handle DynamoDB Lambda event sources as synchronous invocations. | | `SYNCHRONOUS_SQS_EVENTS` | 2.0.0 | `1` \| `0` (default) | Whether or not to handle SQS Lambda event sources as synchronous invocations. | | `SYNCHRONOUS_SNS_EVENTS` | 2.0.0 | `1` \| `0` (default) | Whether or not to handle SNS Lambda event sources as synchronous invocations. | | `TMPDIR`| 2.0.0 | `/tmp` (default) | Temporary folder on the host running the CLI and inside the LocalStack container .| | `USE_LIGHT_IMAGE` | 2.0.0 | `1` (default) | Whether to use the light-weight Docker image. Overwritten by `IMAGE_NAME`.| | `PORT_WEB_UI` | 0.12.8 | `8080` (default) | Port for the legacy Web UI. Replaced by our [Web Application](https://app.localstack.cloud) | # Overview > Integrate LocalStack with supported extensions, SDKs, application frameworks, and testing tools. import SectionCards from '../../../../../components/SectionCards.astro'; LocalStack integrates with a wide range of third-party software from the cloud development ecosystem. This section is mostly useful to platform teams who need to wire LocalStack into existing development and testing workflows. ## The Cloud Development Ecosystem Whether you’re installing a supported extension, driving LocalStack from the Java or Python SDKs, or wiring it into an application framework, this section shows how to integrate LocalStack with the tools you already use. ![Sample of supported integrations](/images/aws/integrations-overview.png) ## Integrations We strive to make integrating LocalStack into your workflow as seamless as possible. Explore LocalStack Extensions, the LocalStack SDKs, supported app frameworks, and testing integrations: # Overview > Use LocalStack with your application frameworks to develop and test your applications locally. LocalStack supports popular application frameworks that simplify building and deploying serverless and event-driven applications. With these integrations, you can develop locally using the same tools and abstractions you rely on in production, while also benefiting from faster feedback loops and no cloud dependencies. This section covers how to use LocalStack with: - Serverless Framework - Architect (ARC) - Quarkus - Spring Cloud Function - Aspire Each guide shows how to configure the framework to work seamlessly with LocalStack for local development and testing. # Architect > Use the Architect Infrastructure as Code framework with LocalStack. ## Overview Architect enables you to quickly build large serverless apps without worrying about the underlying infrastructure. On this page we discuss how Architect and LocalStack can be used together. If you are adapting an existing configuration, you might be able to skip certain steps at your own discretion. ## Example ### Setup To use Architect in conjunction with LocalStack, simply install the `arclocal` command (sources can be found [here](https://github.com/localstack/architect-local)). ```bash npm install -g architect-local @architect/architect aws-sdk ``` The `arclocal` command has the same usage as the `arc` command, so you can start right away. Create a test directory ```bash mkdir architect_quickstart && cd architect_quickstart ``` then create an architect project ```bash arclocal init ``` ### Deployment Now you need to start LocalStack. After LocalStack has started you can deploy your Architect setup via: ```bash arclocal deploy ``` ## Further reading For more architect examples, you can take a look at the [official architect docs](https://arc.codes). # Aspire > Use the Aspire framework with LocalStack ## Introduction [Aspire](https://aspire.dev/) is an opinionated, cloud-ready stack for building observable, production-ready distributed applications. It provides a consistent approach to service discovery, configuration, telemetry, and health checks across cloud-native applications. With Aspire, developers can orchestrate cloud-native applications locally using the same AWS resources they deploy in production. By combining Aspire with LocalStack, teams can emulate their full cloud environment—including Lambda, SQS, S3, and DynamoDB—with minimal configuration and no AWS costs. LocalStack integrates with Aspire through the [`LocalStack.Aspire.Hosting`](https://github.com/localstack-dotnet/dotnet-aspire-for-localstack) package, enabling seamless local development and testing of AWS-powered applications within the Aspire orchestration framework. This package extends the official [AWS integrations for .NET Aspire](https://github.com/aws/integrations-on-dotnet-aspire-for-aws) to provide LocalStack-specific functionality. ## Getting started This guide demonstrates how to integrate LocalStack into Aspire projects for local AWS service emulation. ### Prerequisites - [.NET 8 SDK](https://dotnet.microsoft.com/download/dotnet/8.0) or later - [Docker Desktop](https://docs.docker.com/get-docker/) or compatible container runtime - [Node.js](https://nodejs.org/) (for AWS CDK infrastructure provisioning) - Basic familiarity with [Aspire concepts](https://aspire.dev/get-started/welcome/) ### Installation Add the LocalStack Aspire integration to your App Host project: ```bash dotnet add package LocalStack.Aspire.Hosting ``` For projects that need to interact with AWS services, add the LocalStack.NET client: ```bash dotnet add package LocalStack.Client ``` ## Usage Configure LocalStack integration in your Aspire AppHost project using auto-configuration: ```csharp var builder = DistributedApplication.CreateBuilder(args); // 1. Set up AWS SDK configuration (optional) var awsConfig = builder.AddAWSSDKConfig() .WithProfile("default") .WithRegion(RegionEndpoint.USWest2); // 2. Add LocalStack container var localstack = builder .AddLocalStack(awsConfig: awsConfig, configureContainer: container => { container.Lifetime = ContainerLifetime.Session; container.DebugLevel = 1; container.LogLevel = LocalStackLogLevel.Debug; }); // 3. Add your AWS resources as usual var awsResources = builder.AddAWSCloudFormationTemplate("resources", "template.yaml") .WithReference(awsConfig); var project = builder.AddProject("api") .WithReference(awsResources); // 4. Auto-configure LocalStack for all AWS resources builder.UseLocalStack(localstack); builder.Build().Run(); ``` The `UseLocalStack()` method automatically: - Detects all AWS resources (CloudFormation, CDK stacks) - Configures LocalStack endpoints for all AWS services and project resources - Sets up proper dependency ordering and CDK bootstrap if needed - Transfers LocalStack configuration to service projects via environment variables ## AWS SDK Configuration When using the AWS SDK for .NET with LocalStack in an Aspire context, the SDK clients need to be configured to point to the LocalStack endpoint. ### Using LocalStack.NET The [`LocalStack.Client`](https://github.com/localstack-dotnet/localstack-dotnet-client) library simplifies AWS SDK configuration: ```csharp services.AddLocalStack(configuration); services.AddDefaultAWSOptions(configuration.GetAWSOptions()); services.AddAwsService(); services.AddAwsService(); ``` This automatically configures AWS service clients to use the LocalStack endpoint when running locally. See the [.NET](/aws/connecting/aws-sdks/dotnet/) guide for more information. ## Infrastructure Provisioning LocalStack integrates well with Infrastructure as Code tools within the Aspire orchestration model. ### AWS CDK Integration You can provision AWS resources using AWS CDK during application startup: ```csharp var awsConfig = builder.AddAWSSDKConfig() .WithProfile("default") .WithRegion(RegionEndpoint.USWest2); var localstack = builder.AddLocalStack("localstack"); var customStack = builder .AddAWSCDKStack("custom", scope => new CustomStack(scope, "Aspire-custom")) .WithReference(awsConfig); ``` You can use AWS CDK Stack classes to define and deploy resources: ```csharp // An excerpt of a CDK Stack class internal sealed class CustomStack : Stack { public CustomStack(Construct scope, string id) : base(scope, id) { // Example resources var bucket = new Bucket(this, "Bucket"); var topic = new Topic(this, "ChatTopic"); var queue = new Queue(this, "ChatMessagesQueue", new QueueProps { VisibilityTimeout = Duration.Seconds(30), }); topic.AddSubscription(new SqsSubscription(queue)); // ... (rest of the stack) } } ``` :::note For detailed AWS CDK integration patterns with LocalStack and Aspire, refer to the [provisioning playground example](https://github.com/localstack-dotnet/dotnet-aspire-for-localstack/tree/master/playground/provisioning). ::: ## Configuration Reference For comprehensive configuration options, including environment variables, container settings, and advanced scenarios, refer to the [Configuration Guide](https://github.com/localstack-dotnet/dotnet-aspire-for-localstack/blob/master/docs/CONFIGURATION.md). ## Sample Projects ### Playground Examples The [playground examples](https://github.com/localstack-dotnet/dotnet-aspire-for-localstack/tree/master/playground) include Lambda development patterns and infrastructure provisioning with AWS CDK. ### LocalStack Serverless .NET Demo A reference implementation demonstrating serverless applications with Lambda functions, S3, DynamoDB, SQS, and CDK provisioning: [localstack-serverless-dotnet-demo](https://github.com/localstack-dotnet/localstack-serverless-dotnet-demo) ### OpenTelemetry with Aspire and LocalStack An event registration system showcasing distributed tracing and observability patterns with Lambda and SQS: [dotnet-otel-aspire-localstack-demo](https://github.com/Blind-Striker/dotnet-otel-aspire-localstack-demo) ## Resources - [Aspire Documentation](https://aspire.dev/) - [LocalStack.Aspire.Hosting on GitHub](https://github.com/localstack-dotnet/dotnet-aspire-for-localstack) - [LocalStack.Client on GitHub](https://github.com/localstack-dotnet/localstack-dotnet-client) - [AWS Aspire Integration](https://github.com/aws/integrations-on-dotnet-aspire-for-aws) - [AWS SDK for .NET Documentation](https://docs.aws.amazon.com/sdk-for-net/) - [LocalStack Serverless .NET Demo](https://github.com/localstack-dotnet/localstack-serverless-dotnet-demo) - [OpenTelemetry with Aspire and LocalStack Demo](https://github.com/Blind-Striker/dotnet-otel-aspire-localstack-demo) # Quarkus > Use the Quarkus framework with LocalStack ## Introduction Quarkus is a Java framework optimized for cloud, serverless, and containerized environments. Quarkus leverages a Kubernetes Native Java stack tailored for GraalVM & OpenJDK HotSpot, which further builds on various Java libraries and standards. Localstack is supported by Quarkus as a Dev service for Amazon Services. Quarkus Amazon Services automatically starts a LocalStack container in development mode and when running tests, and the extension client is configured automatically. ## Getting started In this guide, we will demonstrate how you can create a service client for creating and managing Lambdas on LocalStack. The Lambda extension is based on [AWS Java SDK 2.x](https://docs.aws.amazon.com/sdk-for-java/v2/developer-guide/welcome.html). ### Prerequisites - [LocalStack](/aws/getting-started/installation/) installed and running - [JDK 17+](https://www.oracle.com/java/technologies/javase/jdk17-archive-downloads.html) with `JAVA_HOME` configured properly - [Maven 3.8.1+](https://maven.apache.org/download.cgi) - [Docker](https://docs.docker.com/get-docker/) ### Create a Maven project Create a new project with the following command: ```bash showshowLineNumbers mvn io.quarkus.platform:quarkus-maven-plugin:3.6.3:create \ -DprojectGroupId=org.acme \ -DprojectArtifactId=amazon-lambda-quickstart \ -DclassName="org.acme.lambda.QuarkusLambdaSyncResource" \ -Dpath="/sync" \ -Dextensions="resteasy-reactive-jackson,amazon-lambda" cd amazon-lambda-quickstart ``` The above command generates a Maven project structure with imports for RESTEasy Reactive/JAX-RS and Amazon Lambda Client extensions. ### Configure Lambda Client Both Lambda clients (sync and async) can be configured through the `application.properties` file, which should be located in the `src/main/resources` directory. Additionally, ensure that a suitable implementation of the sync client is added to the `classpath`. By default, the extension employs the URL connection HTTP client, so it's necessary to include a URL connection client dependency in the `pom.xml` file: ```xml software.amazon.awssdk url-connection-client ``` If you want to use Apache HTTP client instead, configure it as follows in `application.properties`: ```xml quarkus.lambda.sync-client.type=apache ``` Add the following dependencies to the `pom.xml` file: ```xml software.amazon.awssdk apache-client ``` To configure LocalStack, add the following properties to the `application.properties` file: ```bash showshowLineNumbers quarkus.lambda.endpoint-override=http://localhost.localstack.cloud:4566 quarkus.lambda.aws.region=us-east-1 quarkus.lambda.aws.credentials.type=static quarkus.lambda.aws.credentials.static-provider.access-key-id=test-key quarkus.lambda.aws.credentials.static-provider.secret-access-key=test-secret ``` ### Package the application You can package the application with the following command: ```bash ./mvnw clean package ``` You can further run the application in dev mode with the following command: ```bash java -Dparameters.path=/quarkus/is/awesome/ -jar target/quarkus-app/quarkus-run.jar ``` :::tip With GraalVM installed, you can also create a native executable binary using the following command: ```bash ./mvnw clean package -Dnative. ``` ::: :::note Dev Services for Amazon Services is automatically enabled for each extension added to the `pom.xml`, except in the following scenarios: - When `quarkus.devservices.enabled` is set to false. - When `devservices.enabled` is set to false per extension (e.g., `quarkus.s3.devservices.enabled=false`). - When the `endpoint-override` is configured (e.g., `quarkus.s3.endpoint-override=http://localhost.localstack.cloud:4566`). ::: ## Supported extensions - [Lambda](https://docs.quarkiverse.io/quarkus-amazon-services/dev/amazon-lambda.html) - [S3](https://docs.quarkiverse.io/quarkus-amazon-services/dev/amazon-s3.html) - [SSM](https://docs.quarkiverse.io/quarkus-amazon-services/dev/amazon-ssm.html) - [SQS](https://docs.quarkiverse.io/quarkus-amazon-services/dev/amazon-sqs.html) - [SNS](https://docs.quarkiverse.io/quarkus-amazon-services/dev/amazon-sns.html) - [SES](https://docs.quarkiverse.io/quarkus-amazon-services/dev/amazon-ses.html) - [Secrets Manager](https://docs.quarkiverse.io/quarkus-amazon-services/dev/amazon-secretsmanager.html) - [KMS](https://docs.quarkiverse.io/quarkus-amazon-services/dev/amazon-kms.html) - [IAM](https://docs.quarkiverse.io/quarkus-amazon-services/dev/amazon-iam.html) ## Configuration The following configuration properties are fixed at build time. All the other configuration properties can be overridden at runtime. | Property | Type | Default | | --------------------------------------------------------------------------------------------------- | -------------------- | ----------------------------- | | `quarkus.aws.devservices.localstack.image-name` | string | `localstack/localstack:3.0.1` | | `quarkus.aws.devservices.localstack.init-scripts-folder` | string | | | `quarkus.aws.devservices.localstack.init-scripts-classpath` | string | | | `quarkus.aws.devservices.localstack.init-completion-msg` | string | | | `quarkus.aws.devservices.localstack.container-properties` | `Map` | | | `quarkus.aws.devservices.localstack.additional-services."additional-services".enabled` | boolean | | | `quarkus.aws.devservices.localstack.additional-services."additional-services".shared` | boolean | `false` | | `quarkus.aws.devservices.localstack.additional-services."additional-services".service-name` | string | `localstack` | | `quarkus.aws.devservices.localstack.additional-services."additional-services".container-properties` | `Map` | | :::note - If `quarkus.aws.devservices.localstack.additional-services."additional-services".enabled` is set to `true` and the endpoint-override is not configured, LocalStack will be started and utilized instead of the provided configuration. For all services excluding Cognito, LocalStack will function as the core cloud emulator. In the case of Cognito, the emulation/mocking will be done by Moto. - The `quarkus.aws.devservices.localstack.additional-services."additional-services".shared` indicates whether the LocalStack container managed by Dev Services is shared. In shared mode, Quarkus utilizes label-based service discovery, specifically the `quarkus-dev-service-localstack` label, to identify running containers. If a matching container is found, it is used. Otherwise, Dev Services initiates a new container. It's important to note that sharing is not supported for the Cognito extension. - In `quarkus.aws.devservices.localstack.additional-services."additional-services".service-name`, the value of the `quarkus-dev-service-localstack` label is attached to the initiated container. In dev mode, when the shared flag is true, Dev Services checks for a container with the `quarkus-dev-service-localstack` label set to the configured value before starting a new one. If found, it utilizes the existing container. Otherwise, it creates a new container with the `quarkus-dev-service-localstack` label set to the specified value. In test mode, Dev Services groups services with the same service-name value into a single container instance. This property is useful when there's a requirement for multiple shared LocalStack instances. ::: ### Specific configuration Dev Services can support specific configurations passed to the LocalStack container. These configurations can be globally applied to all containers or specified individually per service. ```bash quarkus.aws.devservices.localstack.image-name=localstack/localstack:3.0.3 quarkus.dynamodb.devservices.container-properties.DYNAMODB_HEAP_SIZE=1G ``` ### Additional services To start additional services for which a Quarkus extension does not exist or is not imported in the project, use the `additional-services` property: ```bash quarkus.aws.devservices.localstack.additional-services."kinesis".enabled=true quarkus.aws.devservices.localstack.additional-services."redshift".enabled=true ``` # Self-managed Kafka cluster > Using LocalStack Lambda with a self-managed Kafka cluster. LocalStack for AWS supports [AWS Managed Streaming for Kafka (MSK)](/aws/services/kafka) and you can create Kafka clusters directly through the MSK API that will run in LocalStack. In some cases, you may want to run your own self-managed Kafka cluster and integrate it with your applications, like triggering Lambdas from a Kafka stream running in your own cluster. ## Running self-managed Kafka You can find the [example Docker Compose](https://github.com/localstack/localstack-docs/blob/master/src/content/docs/aws/customization/integrations/app-frameworks/docker-compose.yml) file which contains a single-noded ZooKeeper and a Kafka cluster and a simple LocalStack setup as well as [Kowl](https://github.com/cloudhut/kowl), an Apache Kafka Web UI. 1. Run Docker Compose: ```bash showshowLineNumbers docker-compose up -d ``` 2. Create the Lambda function: ```bash showshowLineNumbers lstk aws lambda create-function \ --function-name fun1 \ --handler lambda.handler \ --runtime python3.8 \ --role arn:aws:iam::000000000000:role/lambda-role \ --zip-file fileb://lambda.zip { "FunctionName": "fun1", "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:fun1", "Runtime": "python3.8", "Role": "arn:aws:iam::000000000000:role/lambda-role", "Handler": "lambda.handler", "CodeSize": 294, "Description": "", "Timeout": 3, "LastModified": "2021-05-19T02:01:06.617+0000", "CodeSha256": "/GPsiNXaq4tBA4QpxPCwgpeVfP7j+1tTH6zdkJ3jiU4=", "Version": "$LATEST", "VpcConfig": {}, "TracingConfig": { "Mode": "PassThrough" }, "RevisionId": "d85469d2-8558-4d75-bc0e-5926f373e12c", "State": "Active", "LastUpdateStatus": "Successful", "PackageType": "Zip" } ``` 3. Create an example secret: ```bash showshowLineNumbers lstk aws secretsmanager create-secret --name localstack { "ARN": "arn:aws:secretsmanager:us-east-1:000000000000:secret:localstack-TDIuI", "Name": "localstack", "VersionId": "32bbb8e2-46ee-4322-b3d5-b6459d54513b" } ``` 4. Create an example Kafka topic: ```bash showshowLineNumbers docker exec -ti kafka kafka-topics --zookeeper zookeeper:2181 --create --replication-factor 1 --partitions 1 --topic t1 Created topic t1. ``` 5. Create the event source mapping to your local kafka cluster: ```bash showshowLineNumbers lstk aws lambda create-event-source-mapping \ --topics t1 \ --source-access-configuration Type=SASL_SCRAM_512_AUTH,URI=arn:aws:secretsmanager:us-east-1:000000000000:secret:localstack-TDIuI \ --function-name arn:aws:lambda:us-east-1:000000000000:function:fun1 \ --self-managed-event-source '{"Endpoints":{"KAFKA_BOOTSTRAP_SERVERS":["localhost:9092"]}}' { "UUID": "4a2b0ea6-960c-4847-8684-465876dd6dbd", "BatchSize": 100, "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:fun1", "LastModified": "2021-05-19T04:02:49+02:00", "LastProcessingResult": "OK", "State": "Enabled", "StateTransitionReason": "User action", "Topics": [ "t1" ], "SourceAccessConfigurations": [ { "Type": "SASL_SCRAM_512_AUTH", "URI": "arn:aws:secretsmanager:us-east-1:000000000000:secret:localstack-TDIuI" } ], "SelfManagedEventSource": { "Endpoints": { "KAFKA_BOOTSTRAP_SERVERS": [ "localhost:9092" ] } } } ``` 6. Additionally visit `http://localhost:8080` for Kowl's UI. # Spring Cloud Function > Use the Spring Cloud Function framework with LocalStack. import { Tabs, TabItem } from '@astrojs/starlight/components'; import { Badge } from '@astrojs/starlight/components'; ## Overview In this guide, you will learn how to use LocalStack to test your serverless applications powered by Spring Cloud Function framework. :::note Some features and services described in this document may not work properly on aarch64, including Apple's M1 silicon. ::: ## Covered Topics We will create a new Rest API application that will route requests to a Cloud Function using `functionRouter` and routing expressions. The primary language for the application is Kotlin powered by [Gradle](https://gradle.org) build tool, but the described concepts would work for any other JVM setup. - [Overview](#overview) - [Covered Topics](#covered-topics) - [Current Limitations](#current-limitations) - [Setting up an Application](#setting-up-an-application) - [Starting a new Project](#starting-a-new-project) - [Project Settings](#project-settings) - [Configure Log4J2 for AWS Lambda](#configure-log4j2-for-aws-lambda) - [Configure Spring Cloud Function for Rest API](#configure-spring-cloud-function-for-rest-api) - [Define an Application class](#define-an-application-class) - [Configure Jackson](#configure-jackson) - [Define Logging Utility](#define-logging-utility) - [Add Request/Response utilities](#add-requestresponse-utilities) - [Creating a sample Model / DTO](#creating-a-sample-model--dto) - [Creating Rest API endpoints](#creating-rest-api-endpoints) - [Cold Start and Warmup](#cold-start-and-warmup-) - [Creating other lambda Handlers](#creating-other-lambda-handlers) - [Setting up Deployment](#setting-up-deployment) - [Testing, Debugging and Hot Reloading](#testing-debugging-and-hot-reloading) - [Useful Links](#useful-links) ### Current Limitations This document demonstrates the usage of the Spring Cloud Function framework together with LocalStack. It does not cover some of the application-specific topics, like 404 error handling, or parametrized routing, that you need to consider when building production-ready applications. ## Setting up an Application We recommend using [jenv](https://github.com/jenv/jenv) to manage multiple Java runtimes. ### Starting a new Project Please follow the instructions from the official [website](https://gradle.org) to install the Gradle build tool on your machine. Then run the following command to initialize a new Gradle project ```bash gradle init ``` Running the command below will run the gradle wrapper task ```bash gradle wrapper ``` After running the wrapper task, you will find the Gradle wrapper script `gradlew`. From now on, we will use the wrapper instead of the globally installed Gradle binary: ```bash ./gradlew ``` ### Project Settings Let's give our project a name: open `settings.gradle`, and adjust the autogenerated name to something meaningful. ```groovy rootProject.name = 'localstack-sampleproject' ``` Now we need to define our dependencies. Here's a list of what we will be using in our project. Gradle plugins: - [java](https://docs.gradle.org/current/userguide/java_plugin.html) - [kotlin jvm](https://kotlinlang.org/docs/gradle.html#targeting-the-jvm) - [kotlin spring plugin](https://kotlinlang.org/docs/all-open-plugin.html#spring-support) - [spring boot plugin](https://docs.spring.io/spring-boot/docs/current/gradle-plugin/reference/htmlsingle/) - [spring dependency management plugin](https://docs.spring.io/spring-boot/docs/current/gradle-plugin/reference/htmlsingle/#managing-dependencies) - [shadow plugin](https://github.com/johnrengelman/shadow) Dependencies: - [kotlin stdlib](https://mvnrepository.com/artifact/org.jetbrains.kotlin/kotlin-stdlib) - [spring cloud starter function web](https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-function-web) - [spring cloud function adapter for aws](https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-function-adapter-aws) - [lambda log4j2](https://mvnrepository.com/artifact/com.amazonaws/aws-lambda-java-log4j2) - [lambda events](https://mvnrepository.com/artifact/com.amazonaws/aws-lambda-java-events) - [jackson core](https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-core) - [jackson databind](https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-databind) - [jackson annotations](https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-annotations) - [jackson module kotlin](https://mvnrepository.com/artifact/com.fasterxml.jackson.module/jackson-module-kotlin) In order to deploy our application to AWS, we need to build so-called "fat jar" which contains all application dependencies. To that end, we use the "Shadow Jar" plugin. Here's our final `build.gradle`: ```groovy showshowLineNumbers=true title="build.gradle" plugins { id "java" id "org.jetbrains.kotlin.jvm" version '1.5.31' id "org.jetbrains.kotlin.plugin.spring" version '1.5.31' id 'org.springframework.boot' version '2.5.5' id "io.spring.dependency-management" version '1.0.11.RELEASE' id "com.github.johnrengelman.shadow" version '7.0.0' } group = 'org.localstack.sampleproject' sourceCompatibility = 11 tasks.withType(JavaCompile) { options.encoding = 'UTF-8' } repositories { mavenCentral() maven { url "https://plugins.gradle.org/m2/" } } ext { springCloudVersion = "3.1.4" awsLambdaLog4jVersion = "1.2.0" awsLambdaJavaEventsVersion = "3.10.0" jacksonVersion = "2.12.5" } dependencies { implementation "org.jetbrains.kotlin:kotlin-stdlib" implementation "org.springframework.cloud:spring-cloud-starter-function-web:$springCloudVersion" implementation "org.springframework.cloud:spring-cloud-function-adapter-aws:$springCloudVersion" implementation "com.amazonaws:aws-lambda-java-log4j2:$awsLambdaLog4jVersion" implementation "com.amazonaws:aws-lambda-java-events:$awsLambdaJavaEventsVersion" implementation "com.fasterxml.jackson.core:jackson-core:$jacksonVersion" implementation "com.fasterxml.jackson.core:jackson-databind:$jacksonVersion" implementation "com.fasterxml.jackson.core:jackson-annotations:$jacksonVersion" implementation "com.fasterxml.jackson.module:jackson-module-kotlin:$jacksonVersion" } import com.github.jengelman.gradle.plugins.shadow.transformers.* // Configure the main class jar { manifest { attributes 'Start-Class': 'org.localstack.sampleproject.Application' } } // Build a fatjar (with dependencies) for aws lambda shadowJar { transform(Log4j2PluginsCacheFileTransformer) dependencies { exclude( dependency("org.springframework.cloud:spring-cloud-function-web:${springCloudVersion}") ) } // Required for Spring mergeServiceFiles() append 'META-INF/spring.handlers' append 'META-INF/spring.schemas' append 'META-INF/spring.tooling' transform(PropertiesFileTransformer) { paths = ['META-INF/spring.factories'] mergeStrategy = "append" } } assemble.dependsOn shadowJar ``` Please note that we will be using `org.localstack.sampleproject` as a working namespace, and `org.localstack.sampleproject.Application` as an entry class for our application. You can adjust it for your needs, but don't forget to change your package names accordingly. ### Configure Log4J2 for AWS Lambda Spring framework comes with Log4J logger, so all we need to do is to configure it for AWS Lambda. In this project, we are following [official documentation](https://docs.aws.amazon.com/lambda/latest/dg/java-logging.html#java-wt-logging-using-log4j2.8) to setup up `src/main/resources/log4j2.xml` content. ```xml title="log4j2.xml" showshowLineNumbers ?xml version="1.0" encoding="UTF-8"?> %d{yyyy-MM-dd HH:mm:ss} %X{AWSRequestId} %-5p %c{1}:%L - %m%n ``` ### Configure Spring Cloud Function for Rest API Spring Function comes with `functionRouter` that can route requests to different `Beans` based on predefined routing expressions. Let's configure it to lookup our function Beans by HTTP method and path, create a new `application.properties` file under `src/main/resources/application.properties` with the following content: ```dotenv spring.main.banner-mode=off spring.cloud.function.definition=functionRouter spring.cloud.function.routing-expression=headers['httpMethod'].concat(' ').concat(headers['path']) spring.cloud.function.scan.packages=org.localstack.sampleproject.api ``` Once configured, you can use `FunctionInvoker` as a handler for your Rest API lambda function. It will automatically pick up the configuration we have just set. ```java title="FunctionInvoker.kt" org.springframework.cloud.function.adapter.aws.FunctionInvoker::handleRequest ``` ### Define an Application class Now our application needs an entry-class, the one we referenced earlier. Let's add it under `src/main/kotlin/org/localstack/sampleproject/Application.kt`. ```kotlin showshowLineNumbers package org.localstack.sampleproject import org.springframework.boot.autoconfigure.SpringBootApplication @SpringBootApplication class Application fun main(args: Array) { // Do nothing unless you use a custom runtime } ``` ### Configure Jackson In our sample project we are using a JSON format for requests and responses. The easiest way to get started with JSON is to use the Jackson library. Let's configure it by creating a new configuration class `JacksonConfiguration.kt` under `src/main/kotlin/org/localstack/sampleproject/config`: ```kotlin title="JacksonConfiguration.kt" showshowLineNumbers package org.localstack.sampleproject.config import com.fasterxml.jackson.annotation.JsonInclude import com.fasterxml.jackson.databind.* import org.springframework.context.annotation.Bean import org.springframework.context.annotation.Configuration import org.springframework.context.annotation.Primary import org.springframework.http.converter.json.Jackson2ObjectMapperBuilder import java.text.DateFormat @Configuration class JacksonConfiguration { @Bean fun jacksonBuilder() = Jackson2ObjectMapperBuilder() .dateFormat(DateFormat.getDateInstance(DateFormat.FULL)) @Bean @Primary fun objectMapper(): ObjectMapper = ObjectMapper().apply { configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false) configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false) configure(SerializationFeature.WRITE_ENUMS_USING_TO_STRING, true) configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false) configure(SerializationFeature.ORDER_MAP_ENTRIES_BY_KEYS, true) configure(MapperFeature.SORT_PROPERTIES_ALPHABETICALLY, true) setSerializationInclusion(JsonInclude.Include.NON_NULL) findAndRegisterModules() } } ``` In applications where you need support for multiple formats or a format different from JSON (for example, SOAP/XML applications) simply use multiple beans with corresponding [ObjectMapper](https://mvnrepository.com/artifact/com.fasterxml.jackson.dataformat/jackson-dataformat-xml) implementations. ### Define Logging Utility Let's create a small logging utility to simplify interactions with the logger ```kotlin title="Logger.kt" showshowLineNumbers package org.localstack.sampleproject.util import org.apache.logging.log4j.LogManager import org.apache.logging.log4j.Logger open class Logger { val LOGGER: Logger = LogManager.getLogger(javaClass.enclosingClass) } ``` ### Add Request/Response utilities To reduce the amount of boilerplate code, we are going to introduce three utility functions for our Rest API communications: - to build regular json response - to build error json response - to parse request payload using ObjectMapper. Note that ObjectMapper does not necessarily need to be a JSON only. It could also be XML or any other Mapper extended from standard ObjectMapper. Your application may even support multiple protocols with different request/response formats at once. Let's define utility functions to to build API gateway responses: ```kotlin showshowLineNumbers package org.localstack.sampleproject.util import org.springframework.messaging.Message import org.springframework.messaging.support.MessageBuilder data class ResponseError( val message: String, ) fun buildJsonResponse(data: T, code: Int = 200): Message { return MessageBuilder .withPayload(data) .setHeader("Content-Type", "application/json") .setHeader("Access-Control-Allow-Origin", "*") .setHeader("Access-Control-Allow-Methods", "OPTIONS,POST,GET") .setHeader("statusCode", code) .build() } fun buildJsonErrorResponse(message: String, code: Int = 500) = buildJsonResponse(ResponseError(message), code) ``` And now a utility function to process API Gateway requests: ```kotlin showshowLineNumbers package org.localstack.sampleproject.util import com.amazonaws.services.lambda.runtime.events.APIGatewayProxyRequestEvent import com.fasterxml.jackson.databind.ObjectMapper import org.springframework.messaging.Message import java.util.function.Function fun apiGatewayFunction( objectMapper: ObjectMapper, callable: (message: Message, context: APIGatewayProxyRequestEvent) -> Message<*> ): Function, Message<*>> = Function { input -> try { val context = objectMapper.readValue( objectMapper.writeValueAsString(input.headers), APIGatewayProxyRequestEvent::class.java ) return@Function callable(input, context) } catch (e: Throwable) { val message = e.message?.replace("\n", "")?.replace("\"", "'") return@Function buildJsonErrorResponse(message ?: "", 500) } } ``` ### Creating a sample Model / DTO To transfer data from requests into something more meaningful than JSON strings (and back) you will be using a lot of Models and Data Transfer Objects (DTOs). It's time to define our first one. ```kotlin showshowLineNumbers title="SampleModel.kt" package org.localstack.sampleproject.model import com.fasterxml.jackson.annotation.JsonIgnore data class SampleModel( val id: Int, val name: String, @JsonIgnore val jsonIgnoredProperty: String? = null, ) ``` ### Creating Rest API endpoints Let's add our first endpoints to simulate CRUD operations on previously defined `SampleModel`: ```kotlin showshowLineNumbers title="SampleApi.kt" package org.localstack.sampleproject.api import com.fasterxml.jackson.databind.ObjectMapper import org.localstack.sampleproject.model.SampleModel import org.localstack.sampleproject.util.Logger import org.localstack.sampleproject.util.apiGatewayFunction import org.localstack.sampleproject.util.buildJsonResponse import org.springframework.context.annotation.Bean import org.springframework.stereotype.Component private val SAMPLE_RESPONSE = mutableListOf( SampleModel(id = 1, name = "Sample #1"), SampleModel(id = 2, name = "Sample #2"), ) @Component class SampleApi(private val objectMapper: ObjectMapper) { companion object : Logger() @Bean("POST /v1/entities") fun createSampleEntity() = apiGatewayFunction(objectMapper) { input, context -> LOGGER.info("calling POST /v1/entities") SAMPLE_RESPONSE.add(input.payload) buildJsonResponse(input.payload, code = 201) } @Bean("GET /v1/entities") fun listSampleEntities() = apiGatewayFunction(objectMapper) { input, context -> LOGGER.info("calling GET /v1/entities") buildJsonResponse("hello world") } @Bean("GET /v1/entities/get") fun getSampleEntity() = apiGatewayFunction(objectMapper) { input, context -> LOGGER.info("calling GET /v1/entities/get") val desiredId = context.queryStringParameters["id"]!!.toInt() buildJsonResponse(SAMPLE_RESPONSE.find { it.id == desiredId }) } } ``` Note how we used Spring's dependency injection to inject `ObjectMapper` Bean we configured earlier. #### Cold Start and Warmup We know Java's cold start is always a pain. To minimize this pain, we will try to define a pre-warming endpoint within the Rest API. By invoking this function every 5-10 mins we can make sure Rest API lambda is always kept in a pre-warmed state. ```kotlin showshowLineNumbers title="ScheduleApi.kt" package org.localstack.sampleproject.api import com.fasterxml.jackson.databind.ObjectMapper import org.localstack.sampleproject.util.apiGatewayFunction import org.localstack.sampleproject.util.buildJsonResponse import org.springframework.context.annotation.Bean import org.springframework.stereotype.Component @Component class ScheduleApi(private val objectMapper: ObjectMapper) { @Bean("SCHEDULE warmup") fun warmup() = apiGatewayFunction(objectMapper) { input, context -> // execute scheduled events buildJsonResponse("OK") } } ``` Now you can add a scheduled event to the Rest API lambda function with the following synthetic payload (to simulate API gateway request). This way, you can define any other scheduled events, but we recommend using pure lambda functions. ```json { "httpMethod": "SCHEDULE", "path": "warmup" } ``` As you may have guessed, this input will get mapped to the `SCHEDULE warmup` Bean. > For more information, please read the "Setting up Deployment" section. ### Creating other lambda Handlers HTTP requests are not the only thing our Spring Function-powered lambdas can do. We can still define pure lambda functions, DynamoDB stream handlers, and so on. Below you can find a little example of few lambda functions grouped in `LambdaApi` class. ```kotlin showshowLineNumbers title="LambdaApi.kt" package org.localstack.sampleproject.api import com.amazonaws.services.lambda.runtime.events.DynamodbEvent import org.localstack.sampleproject.model.SampleModel import org.localstack.sampleproject.util.Logger import org.springframework.cloud.function.adapter.aws.SpringBootStreamHandler import org.springframework.context.annotation.Bean import org.springframework.stereotype.Component import java.util.function.Function @Component class LambdaApi : SpringBootStreamHandler() { companion object : Logger() @Bean fun functionOne(): Function { return Function { LOGGER.info("calling function one") return@Function "ONE"; } } @Bean fun functionTwo(): Function { return Function { LOGGER.info("calling function two") return@Function it; } } @Bean fun dynamoDbStreamHandlerExample(): Function { return Function { LOGGER.info("handling DynamoDB stream event") } } } ``` As you can see from the example above, we are using `SpringBootStreamHandler` class as a base that takes care of the application bootstrapping process and AWS requests transformation. Now `org.localstack.sampleproject.api.LambdaApi` can be used as a handler for your lambda function along with `FUNCTION_NAME` environmental variable with the function bean name. You may have noticed we used `DynamodbEvent` in the last example. The `Lambda-Events` package comes with a set of predefined wrappers that you can use to handle different lifecycle events from AWS. ## Setting up Deployment Check our [sample project](https://github.com/localstack/localstack-pro-samples/tree/master/sample-archive/spring-cloud-function-microservice) for usage examples. ```yaml showshowLineNumbers service: localstack-sampleproject-serverless provider: name: aws runtime: java11 stage: ${opt:stage} region: us-west-1 lambdaHashingVersion: 20201221 deploymentBucket: name: deployment-bucket package: artifact: build/libs/localstack-sampleproject-all.jar plugins: - serverless-localstack - serverless-deployment-bucket custom: localstack: stages: - local functions: http_proxy: timeout: 30 handler: org.springframework.cloud.function.adapter.aws.FunctionInvoker::handleRequest events: - http: path: /{proxy+} method: ANY cors: true # Please, note that events are a LocalStack for AWS feature - schedule: rate: rate(10 minutes) enabled: true input: httpMethod: SCHEDULE path: warmup lambda_helloOne: timeout: 30 handler: org.localstack.sampleproject.api.LambdaApi environment: FUNCTION_NAME: functionOne lambda_helloTwo: timeout: 30 handler: org.localstack.sampleproject.api.LambdaApi environment: FUNCTION_NAME: functionTwo ```` ```java title="ApplicationStack.kt" showshowLineNumbers package org.localstack.cdkstack import java.util.UUID import software.amazon.awscdk.core.Construct import software.amazon.awscdk.core.Duration import software.amazon.awscdk.core.Stack import software.amazon.awscdk.services.apigateway.CorsOptions import software.amazon.awscdk.services.apigateway.LambdaRestApi import software.amazon.awscdk.services.apigateway.StageOptions import software.amazon.awscdk.services.events.Rule import software.amazon.awscdk.services.events.RuleTargetInput import software.amazon.awscdk.services.events.Schedule import software.amazon.awscdk.services.events.targets.LambdaFunction import software.amazon.awscdk.services.lambda.* import software.amazon.awscdk.services.lambda.Function import software.amazon.awscdk.services.s3.Bucket private val STAGE = System.getenv("STAGE") ?: "local" private const val JAR_PATH = "../../build/libs/localstack-sampleproject-all.jar" class ApplicationStack(parent: Construct, name: String) : Stack(parent, name) { init { val restApiLambda = Function.Builder.create(this, "RestApiFunction") .code(Code.fromAsset(JAR_PATH)) .handler("org.springframework.cloud.function.adapter.aws.FunctionInvoker") .timeout(Duration.seconds(30)) .runtime(Runtime.JAVA_11) .tracing(Tracing.ACTIVE) .build() val corsOptions = CorsOptions.builder().allowOrigins(listOf("*")).allowMethods(listOf("*")).build() LambdaRestApi.Builder.create(this, "ExampleRestApi") .proxy(true) .restApiName("ExampleRestApi") .defaultCorsPreflightOptions(corsOptions) .deployOptions(StageOptions.Builder().stageName(STAGE).build()) .handler(restApiLambda) .build() val warmupRule = Rule.Builder.create(this, "WarmupRule") .schedule(Schedule.rate(Duration.minutes(10))) .build() val warmupTarget = LambdaFunction.Builder.create(restApiLambda) .event(RuleTargetInput.fromObject(mapOf("httpMethod" to "SCHEDULE", "path" to "warmup"))) .build() warmupRule.addTarget(warmupTarget) SingletonFunction.Builder.create(this, "ExampleFunctionOne") .code(Code.fromAsset(JAR_PATH)) .handler("org.localstack.sampleproject.api.LambdaApi") .environment(mapOf("FUNCTION_NAME" to "functionOne")) .timeout(Duration.seconds(30)) .runtime(Runtime.JAVA_11) .uuid(UUID.randomUUID().toString()) .build() SingletonFunction.Builder.create(this, "ExampleFunctionTwo") .code(Code.fromAsset(JAR_PATH)) .handler("org.localstack.sampleproject.api.LambdaApi") .environment(mapOf("FUNCTION_NAME" to "functionTwo")) .timeout(Duration.seconds(30)) .runtime(Runtime.JAVA_11) .uuid(UUID.randomUUID().toString()) .build() } } ```` ```hcl title="variables.tf" showshowLineNumbers variable "STAGE" { type = string default = "local" } variable "AWS_REGION" { type = string default = "us-east-1" } variable "JAR_PATH" { type = string default = "build/libs/localstack-sampleproject-all.jar" } provider "aws" { access_key = "test_access_key" secret_key = "test_secret_key" region = var.AWS_REGION s3_force_path_style = true skip_credentials_validation = true skip_metadata_api_check = true endpoints { apigateway = var.STAGE == "local" ? "http://localhost:4566" : null cloudformation = var.STAGE == "local" ? "http://localhost:4566" : null cloudwatch = var.STAGE == "local" ? "http://localhost:4566" : null cloudwatchevents = var.STAGE == "local" ? "http://localhost:4566" : null iam = var.STAGE == "local" ? "http://localhost:4566" : null lambda = var.STAGE == "local" ? "http://localhost:4566" : null s3 = var.STAGE == "local" ? "http://localhost:4566" : null } } resource "aws_iam_role" "lambda-execution-role" { name = "lambda-execution-role" assume_role_policy = < ## Testing, Debugging and Hot Reloading Please read our [Lambda Tools](/aws/developer-tools/lambda-tools) documentation to learn more about testing, debugging, and hot reloading for JVM Lambda functions. ## Useful Links * [Spring Cloud Function on LocalStack (Kotlin JVM)](https://github.com/localstack/localstack-pro-samples/tree/master/sample-archive/spring-cloud-function-microservice) ``` # Overview > Use LocalStack Extensions to customize and extend your local development experience. LocalStack Extensions add services and integrations to your LocalStack container. This feature is available across all LocalStack plans. You can find supported extensions in the [Official Extensions Library](https://app.localstack.cloud/extensions/library). :::tip Want to try out a common LocalStack extension? Our [MailHog tutorial](/aws/customization/integrations/extensions/mailhog) teaches you how to install and use the official MailHog extension. It’s a quick way to explore how extensions work in LocalStack. ::: :::note The feature and the API are currently in preview stage and may be subject to change. Please report any issues or feature requests on [LocalStack Extension's GitHub repository](https://github.com/localstack/localstack-extensions). The new CLI experience, `lstk`, does not support LocalStack Extensions. There is no `lstk extensions` command suite. Continue using the legacy [LocalStack CLI](/aws/developer-tools/running-localstack/localstack-cli/) to install and manage extensions. ::: # Extensions Library > Extend LocalStack by adding new services and features as extensions. ## Introduction LocalStack extensions allows you to extend and customize LocalStack. A LocalStack extension is a Python application that runs together with LocalStack within the LocalStack container. LocalStack extensions are available across all plans, and the list of available extensions can be found in the [Extensions Library](https://app.localstack.cloud/extensions/library). ![LocalStack Extensions Library](/images/aws/extensions-library-ui.png) ## Installing an Extension To install an extension using the LocalStack Extensions Library, you can navigate to the [**app.localstack.cloud/extensions/library**](https://app.localstack.cloud/extensions/library) and click on the **Go to Instance** button to open the list of available instances. If you are running your LocalStack instance locally, you can click on the **Default** option. You will be redirected to the LocalStack instance page, where you can directly click the **Install** button to install the Extension. The installation process will take a few seconds, and **will restart your LocalStack instance**. Click **Continue** to proceed. ## Managing Extensions You can further manage the installed extensions by navigating to the **Extensions** tab in the LocalStack Instance page. You can remove an Extension by clicking the **Remove** button. ![Installed LocalStack Extensions Library](/images/aws/extensions-library-management.png) # MailHog > Learn how to install and use the official MailHog extension. ## Introduction MailHog is an open source email testing tool for developers. It provides a simple SMTP server and web interface that allows developers to easily catch and inspect emails sent from their application during development. In this guide, you will install and use the [official MailHog extension for LocalStack](https://github.com/localstack/localstack-extensions/tree/main/mailhog) and send an email through SES, while inspecting it in MailHog. ## Prerequisites - [LocalStack Account](https://app.localstack.cloud/) - [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html) ## Installation To get started, start your LocalStack instance with your `LOCALSTACK_AUTH_TOKEN`. Access our [Extension Manager](https://app.localstack.cloud/inst/default/extensions/manage), and click the **Install** button for the MailHog extension. ![Extensions Manager](/images/aws/install-extensions.png) You'll receive a confirmation prompt indicating that LocalStack container will restart, after which the extension will become accessible. Check your LocalStack logs for MailHog extension output, where you should see relevant logging information: ```bash 2023-10-11T19:10:54.708 INFO --- [ MainThread] l.extensions.platform : loaded 1 extensions 2023-10-11T19:10:54.709 INFO --- [ MainThread] mailhog.extension : starting mailhog server 2023-10-11T19:10:54.709 INFO --- [ MainThread] mailhog.extension : configuring SMTP host to internal mailhog smtp: localhost:25 ... 2023-10-11T19:10:55.023 INFO --- [ MainThread] mailhog.extension : serving mailhog extension on host: http://mailhog.localhost.localstack.cloud:4566 2023-10-11T19:10:55.023 INFO --- [ MainThread] mailhog.extension : serving mailhog extension on path: http://localhost:4566/mailhog/ ``` ## Usage MailHog enables you to conduct end-to-end testing of applications that utilize SES (Simple Email Service) for sending emails. To test this, let's use the AWS CLI to send an email. ### Send an Email You can use the [`VerifyEmailIdentity`](https://docs.aws.amazon.com/cli/latest/reference/ses/verify-email-identity.html) API to verify an email address with SES. This is a required step before you can send emails from SES. Run the following command to verify an email address: ```bash aws --endpoint-url=http://localhost.localstack.cloud:4566 \ ses verify-email-identity --email-address user1@yourdomain.com ``` You can further send an email using the [`SendEmail`](https://docs.aws.amazon.com/cli/latest/reference/ses/send-email.html) API. Run the following command to send an email: ```bash aws --endpoint-url=http://localhost.localstack.cloud:4566 \ ses send-email \ --from user1@yourdomain.com \ --message 'Body={Text={Data="Hello from LocalStack to MailHog"}},Subject={Data=Test Email}' \ --destination 'ToAddresses=recipient1@example.com' ``` ### Navigate to Extension UI Navigate in your browser to the [MailHog UI in LocalStack](http://mailhog.localhost.localstack.cloud:4566/). You should see the email you sent in the MailHog UI. ![Mailhog UI](/images/aws/mailhog.png) ## Next steps - Explore our collection of official extensions, along with a growing ecosystem of third-party extensions, in our [Extensions Library](https://app.localstack.cloud/extensions/library). - Learn about the various methods for extension management and automating their installation when using LocalStack in a CI environment. Get detailed insights from our [Managing Extensions](/aws/customization/integrations/extensions/managing-extensions) guide. # Managing extensions > How to manage LocalStack extensions in your LocalStack environment. import { FileTree } from '@astrojs/starlight/components'; You have different options to install and manage your LocalStack extensions depending on your environment and work style. Extensions are managed through the LocalStack container, but stored in the [LocalStack volume](/aws/customization/advanced/filesystem) on your host. The next time you start up LocalStack, your extensions will still be there! ## Using the extensions manager in our App The easiest way to manage official extensions is through our webapp and our [Extension Manager App](https://app.localstack.cloud/inst/default/extensions/manage). Simply install and remove extensions from your specific LocalStack instance directly from the App. If you have multiple instances of LocalStack, each instance has its own set of extensions, and our App allows you to manage extensions for each instance individually. :::tip When you install or uninstall extensions, LocalStack needs to be restarted. LocalStack will do this automatically for you! It re-starts the process inside the running container, not the container itself. However, you may lose LocalStack state if you do not use persistence. ::: ![Extensions Manager](/images/aws/extensions-manager.png) ## Using the extensions CLI :::note The new CLI experience, `lstk`, does not support LocalStack Extensions. There is no `lstk extensions` command suite. Continue using the legacy [LocalStack CLI](/aws/developer-tools/running-localstack/localstack-cli/) for the commands in this section. ::: If you use LocalStack with the CLI, you can also use our `localstack extensions` CLI command suite. To get a list of all available commands in LocalStack Extensions, run: ```bash localstack extensions --help Usage: localstack extensions [OPTIONS] COMMAND [ARGS]... Manage LocalStack extensions (preview) Options: -v, --verbose Print more output -h, --help Show this message and exit Commands: init Initialize the LocalStack extensions environment install Install a LocalStack extension list List installed extension uninstall Remove a LocalStack extension ``` To install an extension, specify the name of the `pip` dependency that contains the extension. For example, for the official Stripe extension, you can either use the package distributed on PyPI: ```bash localstack extensions install localstack-extension-httpbin ``` Extensions are just Python pip packages, and the `install` command will accept anything that resolves to a valid pip package. For example, you can install the latest development version directly from our Git repository using the `git+https` directive. ```bash localstack extensions install "git+https://github.com/localstack/localstack-extensions/#egg=localstack-extension-httpbin&subdirectory=httpbin" ``` If you have Python distribution as files on your local machine, for instance pip wheels or source distributions, then you can also install those via the `file://` directive The file will be automatically mounted into the container and installed from there. ```bash pip install file://./my-extensions/dist/my-extension-0.0.1.dev0.tar.gz ``` ### Specify the `LOCALSTACK_VOLUME_DIR` Extensions should be installed in the `LOCALSTACK_VOLUME_DIR`. The default directory on your host is currently `~/.cache/localstack`. If you decide to mount a different directory to `/var/lib/localstack` in your docker-compose file, as shown below, you must specify the `LOCALSTACK_VOLUME_DIR` before installing extensions. ```yaml volumes: - '/tmp/volume:/var/lib/localstack' # LOCALSTACK_VOLUME_DIR mount - '/var/run/docker.sock:/var/run/docker.sock' ``` Here's how you can use `LOCALSTACK_VOLUME_DIR` in your commands: ```bash export LOCALSTACK_VOLUME_DIR=/tmp/volume localstack extensions install file:///tmp/my_files/my-extension-1.0.0.tar.gz ``` ## Automating extensions installation When you are working in CI or with docker-compose, you may want to automate extension management. LocalStack provides two ways to do this: ### Environment variable The `EXTENSION_AUTO_INSTALL` variable is interpreted by LocalStack at startup, to ensure that the extensions set in the variable value are installed when the container starts up. The value is a comma-separated list of extensions directives that can also be specified in `localstack extensions install`. If you want to use the `file://` directive, the distribution file needs to be mounted into the container. In a docker-compose file, this would look something like: ```yaml showLineNumbers services: localstack: container_name: 'localstack-main' image: localstack/localstack-pro ports: - '127.0.0.1:4566:4566' - '127.0.0.1:4510-4559:4510-4559' environment: - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} - DEBUG=1 - EXTENSION_AUTO_INSTALL=localstack-extension-mailhog,localstack-extension-httpbin volumes: - './volume:/var/lib/localstack' - '/var/run/docker.sock:/var/run/docker.sock' ``` ### Configuration file LocalStack also supports configuration files to automatically install extensions. Inside the container, LocalStack will resolve the file `/etc/localstack/conf.d/extensions.txt`, and will install all extensions defined in this list. Since LocalStack extensions are essentially just Python pip packages, the `extensions.txt` has the same format as a [pip requirements file](https://pip.pypa.io/en/stable/reference/requirements-file-format/). An example project could look something like this: - `extensions.txt` ```text localstack-extension-mailhog git+https://github.com/localstack/localstack-extensions/#egg=localstack-extension-aws-replicator&subdirectory=aws-replicator ``` - Project layout: - extension-install - conf.d - extensions.txt - docker-compose.yml - `docker-compose.yaml` ```yaml showLineNumbers services: localstack: ... volumes: - "./volume:/var/lib/localstack" - "conf.d:/etc/localstack/conf.d" - "/var/run/docker.sock:/var/run/docker.sock" ``` When LocalStack starts up, you should see it tries to install the extensions and all their dependencies. ## Extension Management within LocalStack Extensions in LocalStack are Python distributions that operate within their dedicated virtual environment, residing in the [LocalStack Volume](/aws/customization/advanced/filesystem). This involves the creation of a"variable packages folder `/var/lib/localstack/lib`," where the volume management system establishes both an `extensions` folder and a virtual environment named `python_venv`. Within this environment, all extensions and their dependencies are managed. LocalStack integrates its virtual environment, ensuring the resolution of all transitive dependencies associated with extensions. Here's an example what the default LocalStack volume looks like after installing the MailHog extension: ```bash pwd ~/.cache/localstack/volume tree -L 4 ``` - cache - ... - lib - extensions - python_venv - bin - include - lib - lib64 -> lib - pyvenv.cfg - mailhog - v1.0.1 - MailHog_linux_amd64 - logs - tmp - state ## Troubleshooting ### Fixing `"ModuleNotFoundError: No module named 'flask'"` After a recent update in our packaging, you may see this error in your logs when starting LocalStack with an Extension: ```bash ModuleNotFoundError: No module named 'flask' ``` To resolve this, follow these steps: - Clear the directory mounted to the LocalStack container at `/var/lib/localstack`. - Pull the `latest` image of LocalStack (`localstack/localstack-pro:latest`). - Restart the LocalStack container. - Reinstall any extensions you were using, unless you have `EXTENSION_AUTO_INSTALL` enabled. # Official Extensions > Browse the official and community LocalStack Extensions available on the marketplace. ## Introduction The tables below list the extensions currently available on the [LocalStack marketplace](https://app.localstack.cloud/extensions/library), and are kept up to date automatically. You can install any of the extensions below with the LocalStack CLI: ```bash localstack extensions install ``` See [Managing extensions](/aws/customization/integrations/extensions/managing-extensions/) for more details on installing, listing, and removing extensions. :::note This page is auto-generated from the LocalStack marketplace API. ::: ## Official Extensions Extensions built and maintained by the LocalStack team. | Extension | Description | Author | Install | |-----------|-------------|--------|---------| | AWS Proxy | Proxy requests from your LocalStack instance to real AWS resources | LocalStack | `localstack extensions install localstack-extension-aws-proxy` | | Diagnosis Viewer | View the diagnostics endpoint directly in localstack | LocalStack | `localstack extensions install localstack-extension-diagnosis-viewer` | | Hello World | A minimal LocalStack extension | LocalStack | `localstack extensions install localstack-extension-hello-world` | | httpbin | A simple HTTP Request & Response Service directly in LocalStack | LocalStack | `localstack extensions install localstack-extension-httpbin` | | MailHog | Web and API based SMTP testing directly in LocalStack using MailHog | LocalStack | `localstack extensions install localstack-extension-mailhog` | | Miniflare | This extension makes Miniflare (dev environment for Cloudflare workers) available directly in LocalStack | LocalStack | `localstack extensions install localstack-extension-miniflare` | | Resource Graph | Altimeter based LocalStack extension that allows you to create and import into neptune a graph of the resources in your LocalStack instance | LocalStack | `localstack extensions install localstack-extension-resource-graph` | | Stripe | A LocalStack extension that provides a mocked version of Stripe as a service | LocalStack | `localstack extensions install localstack-extension-stripe` | | Terraform Init Hooks | Use Terraform files as initialization hooks to pre-seed your LocalStack instance automatically | Thomas Rausch | `localstack extensions install localstack-extension-terraform-init` | ## Community Extensions Extensions contributed and maintained by the LocalStack community and partners. | Extension | Description | Author | Install | |-----------|-------------|--------|---------| | Authress | Add authentication, permissions, and access control to LocalStack | Authress | `localstack extensions install localstack-extension-authress` | | Claude (Anthropic API) | LocalStack Extension for testing Anthropic Claude API integrations locally | LocalStack Team | `localstack extensions install localstack-claude` | | Keycloak | LocalStack Extension for developing Keycloak-secured apps locally | LocalStack Team | `localstack extensions install localstack-keycloak` | | ParadeDB | LocalStack Extension for running ParadeDB search databases locally | LocalStack & ParadeDB | `localstack extensions install localstack-paradedb` | | TypeDB | LocalStack Extension that facilitates developing TypeDB-based applications locally. | LocalStack & TypeDB | `localstack extensions install localstack-extension-typedb` | | WireMock | LocalStack Extension that facilitates running WireMock mock APIs locally. | LocalStack & WireMock | `localstack extensions install localstack-wiremock` | | Xero | LocalStack Extension for developing against the Xero Finance API locally | LocalStack Team | `localstack extensions install localstack-xero` | # Overview > LocalStack offers various developer endpoints and SDKs provides a programmatic and easy way to interact with them. LocalStack SDKs provide a programmatic interface for interacting with your local cloud environment. These SDKs expose developer endpoints that make it easier to automate workflows, integrate LocalStack into your tools, and build advanced use cases around state, services, or CI automation. This section introduces the available SDKs and explains how to use them to enhance your local cloud development experience. # Java > Use the LocalStack SDK for Java. ## Introduction You can use the LocalStack SDK for Java to develop Java applications that interact with the LocalStack platform and internal developer endpoints. The SDK extends the REST API, offering an object-oriented interface for easier use. The LocalStack SDK for Java currently supports these features: - Save, list, load, and delete Cloud Pods. - Manage fault configurations for the Chaos API. :::note This SDK is still in a preview phase, and will be subject to fast and breaking changes. ::: ## Installation The best way to use the LocalStack SDK for Java in your project is to consume it from Maven Central. You can use Maven to import the entire SDK into your project. ```xml showLineNumbers cloud.localstack localstack-sdk 0.0.1 ``` Similarly, you can copy the following line in the dependencies block of your `build.gradle(.kts)` file, if you are using Gradle as a build tool. ```kotlin implementation("cloud.localstack:localstack-sdk:0.0.1") ``` ## Quick Start Currently, the LocalStack SDK for Java only supports Chaos and Cloud Pods APIs. Both these features have a client that can be instantiated from the `cloud.localstack.sdk.chaos` and `cloud.localstack.sdk.pods` package, respectively. The clients accept requests built by using the [builder pattern](https://en.wikipedia.org/wiki/Builder_pattern). For instance, let us imagine the case in which you want to add a fault rule for S3 on the `us-east-1` region. You first need to use the `FaultRuleRequest` class to build a fault rule request. Then, you need to pass such a request object to the `addFaultRules` method of a created `ChaosClient`. ```java showLineNumbers import cloud.localstack.sdk.chaos.ChaosClient; import cloud.localstack.sdk.chaos.requests.FaultRuleRequest; var client = new ChaosClient(); var faultRuleRequest = new FaultRuleRequest.Builder().faultRule("s3", "us-east-1").build(); var addedRules = client.addFaultRules(request); ``` As a second example, let us look at the necessary code to save and load a Cloud Pod. Similarly to the `ChaosClient`, the `PodsClient` exposes two functions, `savePod` and `loadPod`, which expect a `SavePodRequest` and a `LoadPodRequest`, respectively. The resulting code is the following: ```java showLineNumbers import cloud.localstack.sdk.pods.PodsClient; import cloud.localstack.sdk.pods.requests.LoadPodRequest; import cloud.localstack.sdk.pods.requests.SavePodRequest; var podsClient = new PodsClient(); // save a cloud pod var saveRequest = new SavePodRequest.Builder().podName(POD_NAME).build(); podsClient.savePod(saveRequest); //load a cloud pod var loadRequest = new LoadPodRequest.Builder().podName(POD_NAME).build(); podsClient.loadPod(loadRequest); ``` # Python > Use the LocalStack SDK for Python. ## Introduction You can use the LocalStack SDK for Python to develop Python applications that interact with the LocalStack platform and internal developer endpoints. The SDK extends the REST API, offering an object-oriented interface for easier use. The LocalStack SDK for Python supports these features: - Save, list, load, and delete Cloud Pods. - Manage fault configurations for the Chaos API. - Automatically reset service states. - List SQS queue messages without causing side effects. - Retrieve and delete sent SES messages. :::note This SDK is still in a preview phase, and will be subject to fast and breaking changes. ::: ## Installation Install the latest `localstack-sdk-python` release via pip: ```bash pip install --upgrade localstack-sdk-python ``` ## Basic Concepts LocalStack SDK for Python organizes functionality into specific modules like `aws`, `state`, `pods`, and `chaos`. For example, the `aws` module allows developers to initialize clients for various AWS services. Using the SDK in Python is straightforward: developers can import the relevant modules and initialize specific clients (e.g., `AWSClient`, `StateClient`, `PodsClient`, `ChaosClient`) to perform actions on local AWS services. ### AWS endpoints #### SQS The following code snippet shows how to set up an SQS client, create a queue, send messages, and retrieve them to test local SQS interactions using LocalStack. ```python showLineNumbers import json import boto3 import localstack.sdk.aws # Initialize LocalStack AWS client client = localstack.sdk.aws.AWSClient() # Set up SQS client using the LocalStack AWS client configuration sqs_client = boto3.client( "sqs", endpoint_url=client.configuration.host, region_name="us-east-1", aws_access_key_id="test", aws_secret_access_key="test", ) # Create an SQS queue queue_name = "test-queue" sqs_client.create_queue(QueueName=queue_name) queue_url = sqs_client.get_queue_url(QueueName=queue_name)["QueueUrl"] # Send messages to the queue for i in range(5): sqs_client.send_message( QueueUrl=queue_url, MessageBody=json.dumps( {"event": f"event-{i}", "message": f"message-{i}"} ), ) # Retrieve messages from the queue response = sqs_client.receive_message( QueueUrl=queue_url, MaxNumberOfMessages=5 ) # Print each message body messages = response.get("Messages", []) for msg in messages: print("Message Body:", msg.get("Body")) ``` ```bash showLineNumbers Message Body: {"event": "event-0", "message": "message-0"} Message Body: {"event": "event-1", "message": "message-1"} Message Body: {"event": "event-2", "message": "message-2"} Message Body: {"event": "event-3", "message": "message-3"} Message Body: {"event": "event-4", "message": "message-4"} ``` #### SES The following code snippet verifies an email address, sends a raw email, retrieves the message ID, and discards all SES messages afterward. ```python showLineNumbers import boto3 import localstack.sdk.aws client = localstack.sdk.aws.AWSClient() # Initialize SES client ses_client = boto3.client( "ses", endpoint_url=client.configuration.host, region_name="us-east-1", aws_access_key_id="test", aws_secret_access_key="test", ) # Verify email address email = "user@example.com" ses_client.verify_email_address(EmailAddress=email) # Send a raw email raw_message_data = f"From: {email}\nTo: recipient@example.com\nSubject: test\n\nThis is the message body.\n\n" ses_client.send_raw_email(RawMessage={"Data": raw_message_data}) # Get and print SES message IDs messages = client.get_ses_messages() for msg in messages: print("Message ID:", msg.id) # Discard all SES messages client.discard_ses_messages() ``` ```bash Message ID: khqzljuixhpnpejl-mnlhgajk-ebch-zfxq-orit-qgexxjlrkipo-ywgvwr ``` ### State Management LocalStack provides various features for managing local state, including Cloud Pods, which allow for saving, loading, and deleting cloud states. ### Cloud Pods Cloud Pods is a feature that enables storing and managing snapshots of the current state. This code snippet shows listing available pods, saving a new pod, loading it, and then deleting it. You need to set your `LOCALSTACK_AUTH_TOKEN` in your terminal session before running the snippet. ```python showLineNumbers from localstack.sdk.pods import PodsClient POD_NAME = "ls-cloud-pod" client = PodsClient() # List all pods pods = client.list_pods() print("Pods:", pods) # Save Cloud Pod client.save_pod(pod_name=POD_NAME) print(f"Pod '{POD_NAME}' saved.") # Load Cloud Pod client.load_pod(pod_name=POD_NAME) print(f"Pod '{POD_NAME}' loaded.") # Delete Cloud Pod client.delete_pod(pod_name=POD_NAME) print(f"Pod '{POD_NAME}' deleted.") ``` ```bash Pods: cloudpods=[PodListCloudpodsInner(max_version=1, pod_name='check-pod', last_change=None)] Pod 'ls-cloud-pod' saved. Pod 'ls-cloud-pod' loaded. Pod 'ls-cloud-pod' deleted. ``` ### State Reset The following example demonstrates how to reset the current cloud state using LocalStack’s `StateClient`. ```python showLineNumbers import boto3 from localstack.sdk.state import StateClient # Initialize StateClient and SQS client client = StateClient() sqs_client = boto3.client( "sqs", endpoint_url=client.configuration.host, region_name="us-east-1", aws_access_key_id="test", aws_secret_access_key="test", ) # Create SQS queue sqs_client.create_queue(QueueName="test-queue") url = sqs_client.get_queue_url(QueueName="test-queue")["QueueUrl"] print("Queue URL before reset:", url) # Reset state client.reset_state() print("State reset.") # Try to retrieve the queue URL after state reset try: sqs_client.get_queue_url(QueueName="test-queue") except Exception as exc: error_code = exc.response["Error"]["Code"] print("Error after state reset:", error_code) ``` ```bash Queue URL before reset: http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/test-queue State reset. Error after state reset: AWS.SimpleQueueService.NonExistentQueue ``` ### Chaos API LocalStack’s Chaos API enables fault injection to simulate issues in AWS services. This example shows how to add a fault rule for the S3 service, retrieve and display the rule, and finally delete it to return to normal operations. ```python showLineNumbers import localstack.sdk.chaos from localstack.sdk.models import FaultRule # Initialize ChaosClient client = localstack.sdk.chaos.ChaosClient() # Add a fault rule for S3 rule = FaultRule(region="us-east-1", service="s3") rules = client.add_fault_rules(fault_rules=[rule]) print("Added S3 rule:", [(r.region, r.service) for r in rules]) # Retrieve and display current fault rules rules = client.get_fault_rules() print("Current rules:", [(r.region, r.service) for r in rules]) # Delete the S3 fault rule rules = client.delete_fault_rules(fault_rules=[rule]) print("Rules after deleting S3 rule:", [(r.region, r.service) for r in rules]) ``` ```bash Added S3 rule: [('us-east-1', 's3')] Current rules: [('us-east-1', 's3')] Rules after deleting S3 rule: [] ``` # Testing Utils > Tools to simplify application testing on LocalStack. ## Introduction LocalStack provides a set of tools to simplify application testing on LocalStack. These tools are available for Python and can be used to integrate with various unit testing frameworks and simplify the setup of AWS clients with LocalStack. ## Python This [Python Testing Utils](https://github.com/localstack/localstack-python-utils) streamlines the integration of Localstack with your unit tests. ### Installation ```bash pip install localstack-utils ``` ### Usage ```python showLineNumbers import time import boto3 import unittest from localstack_utils.localstack import startup_localstack, stop_localstack class TestKinesis(unittest.TestCase): def setUp(self): startup_localstack() def tearDown(self): stop_localstack() return super().tearDown() def test_create_stream(self): kinesis = boto3.client( service_name="kinesis", aws_access_key_id="test", aws_secret_access_key="test", endpoint_url="http://localhost.localstack.cloud:4566", ) kinesis.create_stream(StreamName="test", ShardCount=1) time.sleep(1) response = kinesis.list_streams() self.assertGreater(len(response.get("StreamNames", [])), 0) ``` # Overview > Use LocalStack with testing tools & utilities to test your application infrastructure locally. LocalStack integrates with popular testing frameworks and platforms to help you automate and scale cloud application testing. LocalStack makes it easier to run containers with isolated test environments or execute parallel tests in CI pipelines. This section covers how to use LocalStack with: - **Testcontainers**: spin up reproducible, containerized test environments - **LambdaTest HyperExecute**: fast, scalable execution of cloud-native test suites # LambdaTest HyperExecute > Executing LocalStack tests on LambdaTest's HyperExecute. [HyperExecute](https://www.lambdatest.com/hyperexecute) is a test orchestration platform designed to optimize the execution of automated tests in the cloud. It supports a wide range of testing frameworks and integrates seamlessly with CI/CD pipelines, such as GitHub Actions. You can use HyperExecute to run your LocalStack tests on your local machine or in the CI pipeline using a single configuration file. :::note LambdaTest provides specialized runners for LocalStack. The default runners don't provide a Docker socket, which is required for LocalStack to work properly. If you want to use LocalStack with HyperExecute, you need to get in touch with the LambdaTest team to get access to the specialized runners. ::: ## Getting started To get started with HyperExecute, you need to fulfill the following prerequisites: - [HyperExecute CLI](https://www.lambdatest.com/support/docs/hyperexecute-cli-run-tests-on-hyperexecute-grid/) - [GitHub Personal Access Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) - [LambdaTest Account](https://hyperexecute.lambdatest.com/) and Access Key for HyperExecute (`he`) ### Configuring the HyperExecute environment Create a new file named `he.yml` in the root directory of your project and add the following content: ```yaml showshowLineNumbers version: "0.1" runson: linux autosplit: true parallelism: 2 concurrency: 2 scenarioCommandStatusOnly: true runtime: - language: python version: '3.10' - language: node version: '18' pre: - npm install -g @localstack/lstk - pip install awscli - LOCALSTACK_AUTH_TOKEN=${{ .secrets.LOCALSTACK_AUTH_TOKEN }} lstk start --non-interactive --timeout 60s - lstk aws s3 mb s3://test-bucket - lstk aws sqs create-queue --queue-name test-queue - lstk aws sns create-topic --name test-topic ``` The above minimal configuration file starts LocalStack and creates an S3 bucket, SQS queue, and SNS topic. :::note `lstk` requires a LocalStack Auth Token, and CI environments need a [CI Auth Token](/aws/getting-started/auth-token/) rather than a personal Developer Token. Add it to your HyperExecute Portal as a secret named `LOCALSTACK_AUTH_TOKEN` so `${{ .secrets.LOCALSTACK_AUTH_TOKEN }}` resolves at runtime. ::: ### Enabling test execution on HyperExecute To enable test execution on HyperExecute, you need to add the following content to your GitHub Actions workflow file: ```yaml showshowLineNumbers version: "0.1" runson: linux ... pre: ... - bin/deploy.sh testDiscovery: type: raw mode: dynamic command: pytest --co -q tests | sed '$d' testRunnerCommand: pytest $test sourcePayload: platform: git link: https://github.com/localstack-samples/sample-serverless-image-resizer-s3-lambda ref: main accessToken: ${{ .secrets.PAT }} ``` Before running the tests, add your Personal Access Token (PAT) to your HyperExecute Portal as a secret. In this minimal configuration, you will set up our [`Serverless image resizer`](https://github.com/localstack-samples/sample-serverless-image-resizer-s3-lambda) application and run the tests using `pytest`. The `bin/deploy.sh` script is responsible for deploying the application to LocalStack. HyperExecute will automatically detect the tests and run them in parallel. ### Running the tests locally You can run the tests locally using the following command: ```bash hyperexecute --user '' --key '' --config he.yaml ``` Swap `` and `` with your HyperExecute username and access key. You can find your access key in the HyperExecute Portal. ### Running the tests in the CI pipeline In this example, we will use GitHub Actions to run the tests in the CI pipeline. To do so, you need to add the following content to your GitHub Actions workflow file in `.github/workflows/main.yml`: ```yaml showshowLineNumbers name: Running tests on HyperExecute on: push: branches: - main pull_request: branches: - main jobs: HE: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - run: | wget https://downloads.lambdatest.com/hyperexecute/linux/hyperexecute chmod +x hyperexecute ./hyperexecute \ --user {{ secrets.username }} \ --key ${{ secrets.HE }} \ --config he.yaml ``` Add your username and access key to your GitHub repository secrets. You can find your access key in the HyperExecute Portal. You need to add your LocalStack Auth Token to your GitHub repository secrets. # Testcontainers > Use Testcontainers with LocalStack. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Overview [Testcontainers](https://www.testcontainers.com/) is a library that helps you to run your tests against real dependencies. In this guide, you will learn how to use [Testcontainers](https://www.testcontainers.com/) with LocalStack. ## Covered Topics - [Overview](#overview) - [Covered Topics](#covered-topics) - [Installing the Localstack module](#installing-the-localstack-module) - [Obtaining a LocalStack container](#obtaining-a-localstack-container) - [Configuring the AWS client](#configuring-the-aws-client) - [Special Setup for using RDS](#special-setup-for-using-rds) - [Useful Links](#useful-links) ### Installing the Localstack module ```sh dotnet add package Testcontainers.LocalStack --version 3.0.0 ``` ```go go get github.com/testcontainers/testcontainers-go/modules/localstack ``` ```java showshowLineNumbers org.testcontainers localstack 1.18.0 test ``` ```java testImplementation 'org.testcontainers:localstack:1.18.0' ``` ```javascript npm i @testcontainers/localstack ``` ### Obtaining a LocalStack container ```csharp showshowLineNumbers var localStackContainer = new LocalStackBuilder().Build(); await localStackContainer.StartAsync() .ConfigureAwait(false); ``` ```go container, err := localstack.StartContainer(ctx, localstack.NoopOverrideContainerRequest) ``` ```java LocalStackContainer localstack = new LocalStackContainer(DockerImageName.parse("localstack/localstack:3")); ``` ```javascript const localstack = new LocalstackContainer("localstack/localstack:3").start() ``` ## Configuring the AWS client ```csharp showshowLineNumbers var config = new AmazonS3Config(); config.ServiceURL = localStackContainer.GetConnectionString(); using var client = new AmazonS3Client(config); ``` ```go showshowLineNumbers func s3Client(ctx context.Context, l *localstack.LocalStackContainer) (*s3.Client, error) { // the Testcontainers Docker provider is used to get the host of the Docker daemon provider, err := testcontainers.NewDockerProvider() if err != nil { return nil, err } host, err := provider.DaemonHost(ctx) if err != nil { return nil, err } mappedPort, err := l.MappedPort(ctx, nat.Port("4566/tcp")) if err != nil { return nil, err } customResolver := aws.EndpointResolverWithOptionsFunc( func(service, region string, opts ...interface{}) (aws.Endpoint, error) { return aws.Endpoint{ PartitionID: "aws", URL: fmt.Sprintf("http://%s:%d", host, mappedPort.Int()), SigningRegion: region, }, nil }) awsCfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion(region), config.WithEndpointResolverWithOptions(customResolver), config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(accesskey, secretkey, token)), ) if err != nil { return nil, err } client := s3.NewFromConfig(awsCfg, func(o *s3.Options) { o.UsePathStyle = true }) return client, nil } ``` ```java showshowLineNumbers S3Client s3 = S3Client.builder() .endpointOverride(localstack.getEndpoint()) .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create(localstack.getAccessKey(), localstack.getSecretKey()))) .region(Region.of(localstack.getRegion())) .build(); ``` ```typescript showshowLineNumbers const awsConfig = { endpoint: localstack.getConnectionUri(), credentials: { accessKeyId: "test", secretAccessKey: "test", }, region: "eu-central-1", }; const s3 = S3Client(awsConfig); ``` ## Special Setup for using RDS Some services like RDS require additional setup so that the correct port is exposed and accessible for the tests. The reserved ports on LocalStack are between `4510-4559`, depending on your use case you might need to expose several ports using `witExposedPorts`. Check the [pro-sample on how to use RDS with Testcontainers for Java](https://github.com/localstack/localstack-pro-samples/tree/master/testcontainers-java-sample). The Testcontainer can be created like this: ```java /** * Start LocalStackContainer with exposed Ports. Those ports are used by services like RDS, where several databases can be started, running on different ports. * In this sample we only map 5 ports, however, depending on your use case you may need to map ports up to 4559 */ @Rule public LocalStackContainer localstack = new LocalStackContainer(DockerImageName("localstack/localstack:2.0.0")) .withExposedPorts(4510, 4511, 4512, 4513, 4514) // the port can have any value between 4510-4559, but LS starts from 4510 .withEnv("LOCALSTACK_AUTH_TOKEN", auth_token); // add your Auth Token here ``` To find the exposed port which you can use to connect to the instance: ```java // identify the port localstack provides for the instance int localstack_port = response.dbInstance().endpoint().port(); // get the port it was mapped to, e.g. the one we can reach from host/the test int mapped_port = localstack.getMappedPort(localstack_port); ``` ## Useful Links * https://www.testcontainers.com (Java, .NET, Go, Python, Ruby, Node.js) * https://www.testcontainers.org (Java) * https://www.testcontainers.org/modules/localstack (Java) * https://golang.testcontainers.org (Go) * https://golang.testcontainers.org/modules/localstack (Go) * https://node.testcontainers.org (NodeJs) * https://node.testcontainers.org/modules/localstack (NodeJs) # Overview > Run LocalStack on Kubernetes, a common approach for enterprises to centrally manage their containers. LocalStack is a local AWS cloud environment that emulates core AWS services for development and testing. When LocalStack is deployed on Kubernetes and configured to use the Kubernetes-native executor (by setting `CONTAINER_RUNTIME=kubernetes`), services that would normally spawn Docker containers (ie., Lambda, ECS, or RDS) instead create Kubernetes pods within the cluster. This enables dynamic scaling, isolation, and native Kubernetes orchestration. Supported cases: - Local Development Environments: Provide isolated, consistent environments for individual developers or small teams. - Hosted Development Environments: Provide scalable and isolated development environments for teams. - CI/CD Pipeline Testing: Run end-to-end integration tests in a reproducible, cloud-like environment during CI/CD workflows. ## Requirements: - K8s Cluster (such as k3d, minikube, EKS) - [kubectl](https://kubernetes.io/docs/reference/kubectl/) - (Optional) [Helm](https://helm.sh/) - (Optional) LS helm chart - (Optional) LS operator ## Deployment methods LocalStack can be deployed into a Kubernetes cluster using multiple methods: * using the LocalStack Operator * using the LocalStack helm chart * by manually creating Kubernetes manifests The table below compares these methods. | Deployment approach | Pros | Cons | |---------------------|------|------| | **Operator** | · Declarative, self-managed control plane
· Built-in validation, defaulting, and reconciliation logic | · Requires in-cluster component (controller) | | **Helm chart** | · Simplifies deployment using templates and `values.yaml`
· Supports versioning, upgrades, and rollbacks
· Supports LocalStack for AWS under all Licences | · Customization is limited to chart values and overrides | | **DIY (YAML manifests)** | · Full control over Kubernetes configuration and resources | · Time-consuming to set up and maintain
· Manual updates and lifecycle management | ## Limitations | Service | Supported | Explanation | | ------- | --------- | ----------- | | `Sagemaker` | No | Requires spawning Docker containers at runtime, which is unavailable when LocalStack runs inside a Kubernetes pod. | | `Bedrock` | No | Requires spawning Docker containers at runtime, which is unavailable when LocalStack runs inside a Kubernetes pod. | | `EKS` | No | Uses k3d, which requires Docker to create clusters. A Docker-free local mode is available but unsupported. | | `CodeBuild` | No | Requires spawning Docker containers at runtime, which is unavailable when LocalStack runs inside a Kubernetes pod. | | `SSM` | Partial | Most SSM functionality is supported. Session Manager (`ssm:StartSession`) for exec-ing into EC2 instances is not supported. | | `ECS` | Partial | Core task execution is supported. FireLens log routing, volume mounts, port exposure from tasks, and private registry credentials via `RepositoryCredentials` are not supported. | | `EC2` | Partial | Core EC2 functionality is supported. User data scripts may not behave identically to AWS — full parity is not guaranteed. | | `RDS` | Partial | Most database engines are supported. MySQL 5.7 is not supported. Persistence across pod restarts is not supported, except for PostgreSQL and MariaDB. | ## Help & Support LocalStack Enterprise (or additional purchase of the Kubernetes pack add-on) is the only version that provides: - Official support and integration for Kubernetes environments. - Dynamic pod creation by services like Lambda, ECS, RDS, etc. # Concepts & Architecture > Concepts & Architecture This conceptual guide explains how LocalStack runs inside a Kubernetes cluster, how workloads are executed, and how networking and DNS behave in a Kubernetes-based deployment. ## How the LocalStack pod works The LocalStack pod runs the LocalStack runtime and acts as the central coordinator for all emulated AWS services within the cluster. Its primary responsibilities include: * Exposing the LocalStack edge endpoint and AWS service API ports * Receiving and routing incoming AWS API requests * Orchestrating services that require additional compute (for example Lambda, Glue, ECS, and EC2) * Managing the lifecycle of compute workloads spawned on behalf of AWS services From a Kubernetes perspective, the LocalStack pod is a standard pod that fully participates in cluster networking. It is typically exposed through a Kubernetes `Service`, and all AWS API interactions —whether from inside or outside the cluster— are routed through this pod. ![How the Localstack Pod works](/images/aws/k8s-concepts.png) ## Execution modes LocalStack supports two execution modes for running compute workloads: * Kubernetes-native executor * Docker executor ### Kubernetes-native executor The Kubernetes-native executor runs workloads as Kubernetes pods. In this mode, LocalStack communicates directly with the Kubernetes API to create, manage, and clean up pods on demand. This execution mode provides stronger isolation, better security, and full integration with Kubernetes scheduling, resource limits, and lifecycle management. The execution mode is configured using the `CONTAINER_RUNTIME` environment variable. ### Docker executor The Docker executor runs workloads as containers started via a Docker runtime that is accessible from the LocalStack pod. This provides a simple, self-contained execution model without Kubernetes-level scheduling. However, Kubernetes does not provide a Docker daemon inside pods by default. To use the Docker executor in Kubernetes, the LocalStack pod must be given access to a Docker-compatible runtime (commonly via a Docker-in-Docker sidecar), which adds complexity and security concerns. ## Child pods For compute-oriented AWS services, LocalStack can execute workloads either within the LocalStack pod itself or as separate Kubernetes pods. When the Kubernetes-native executor is enabled, LocalStack launches compute workloads as dedicated Kubernetes pods (referred to here as *child pods*). These include: * Lambda function invocations * Glue jobs * ECS tasks and Batch jobs * EC2 instances * RDS databases * Apache Airflow workflows * Amazon Managed Service for Apache Flink * Amazon DocumentDB databases * Redis instances * CodeBuild containers For example, each Glue job run or ECS task invocation results in a new pod created from the workload’s configured runtime image and resource requirements. These child pods execute independently of the LocalStack pod. Kubernetes is responsible for scheduling them, enforcing resource limits, and managing their lifecycle. Most child pods are short-lived and terminate once the workload completes, though some services (such as Lambda) may keep pods running for longer periods. ## Networking model LocalStack runs as a standard Kubernetes pod and is accessed through a Kubernetes `Service` that exposes the edge API endpoint and any additional service ports. Other pods within the cluster communicate with LocalStack through this Service using normal Kubernetes DNS resolution and cluster networking. When the Kubernetes-native executor is enabled, child pods communicate with LocalStack in the same way, by sending API requests over the cluster network to the LocalStack Service. ## DNS behavior LocalStack includes a DNS server capable of resolving AWS-style service endpoints. In a Kubernetes deployment: * The DNS server can be exposed through the same Kubernetes Service as the LocalStack API ports. * This allows transparent resolution of AWS service hostnames and `localhost.localstack.cloud` to LocalStack endpoints from within the cluster. * If a custom domain is used to refer to the LocalStack Kubernetes service (via `LOCALSTACK_HOST`) then this name and subdomains of this name are also resolved by the LocalStack DNS server This enables applications running in Kubernetes to interact with LocalStack using standard AWS SDK endpoint resolution without additional configuration. ## Storage LocalStack can store data that persists across sessions, such as data that can be used for [local persistence](https://docs.localstack.cloud/aws/developer-tools/snapshots/persistence/) or caching downloaded resources between sessions. See the [LocalStack volume](/aws/customization/advanced/filesystem/#localstack-volume) page for more information. This volume directory can be created in Kubernetes using a [Persistent Volume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) and associated [Persistent Volume Claim](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims). This volume should be mounted into the pod at `/var/lib/localstack` to persist LocalStack state. See the [Operator](/aws/customization/kubernetes/kubernetes-operator/) and [Helm Chart](/aws/customization/kubernetes/deploy-helm-chart) documentation for specific details. ## Choose execution mode The Kubernetes-native executor should be used when LocalStack is deployed inside a Kubernetes cluster and workloads must run reliably and securely. It is the recommended execution mode for nearly all Kubernetes deployments, because Kubernetes does not include a Docker daemon inside pods and does not provide native Docker access. The Kubernetes-native executor aligns with Kubernetes’ workload model, enabling pod-level isolation, scheduling, and resource governance. The Docker executor is not supported for use inside Kubernetes clusters. While it may function in environments that have been explicitly configured to expose a Docker-compatible runtime to the LocalStack pod, such setups are uncommon and may introduce security or operational complexity. For Kubernetes-based deployments, the Kubernetes-native executor is the supported and recommended execution mode. # Kubernetes Configuration > Kubernetes configuration reference for LocalStack running on Kubernetes When LocalStack runs on Kubernetes with the Kubernetes executor enabled, a set of configuration variables controls how child pods are created and managed. These variables apply to pods spawned by services such as Lambda, ECS, and RDS. ### Namespace By default, LocalStack creates child pods in the `default` namespace. Use `K8S_NAMESPACE` to deploy them into a different namespace. ```bash K8S_NAMESPACE=localstack-workloads ``` The namespace must already exist in your cluster before starting LocalStack. ### Labels and annotations You can attach custom Kubernetes labels and annotations to all child pods created by LocalStack. This is useful for integrating with cluster tooling such as monitoring agents, network policies, or admission controllers. Both variables accept a comma-separated list of `key=value` pairs: ```bash K8S_LABELS=env=dev,team=platform K8S_ANNOTATIONS=prometheus.io/scrape=true,prometheus.io/port=8080 ``` ### Pod configuration `K8S_POD_CONFIG` configures Kubernetes metadata, scheduling, and resource settings for child pods created by supported services such as Lambda and ECS. `LOCALSTACK_K8S_POD_CONFIG` configures Kubernetes metadata, scheduling, and resource settings for child pods created by supported services, including Lambda, ECS, CodeBuild, DocumentDB, ElastiCache, Kafka, Kinesis Data Analytics, MWAA, RDS (MySQL and SQL Server engines), Glue, and EC2. Use it to define reusable pod profiles with fields such as `nodeSelector`, `tolerations`, `affinity`, `resources`, `labels`, and `annotations`. The value must be valid JSON. ```bash K8S_POD_CONFIG='{"profiles":{"default":{"nodeSelector":{"pool":"general"}}}}' ``` For the full JSON schema, profile resolution order, and examples, see [Pod Configuration](/aws/customization/kubernetes/pod-configuration/). ### Container security context `K8S_CONTAINER_SECURITY_CONTEXT` sets the [container security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) applied to child pods created by LocalStack. The value should be a JSON object matching the Kubernetes `SecurityContext` spec. This is useful when your cluster enforces pod security policies or security admission controls that require specific security context fields to be set. ```bash K8S_CONTAINER_SECURITY_CONTEXT='{"runAsNonRoot": true, "runAsUser": 1000, "allowPrivilegeEscalation": false}' ``` ### Init images LocalStack uses init containers in some child pods to perform setup tasks before the main container starts. The following variables let you override the default images used for these init containers: - `K8S_CURL_INIT_IMAGE` — the image used for the curl-based init container, typically responsible for waiting on network dependencies. - `LAMBDA_K8S_INIT_IMAGE` — the image used for the init container in Lambda pods specifically. You may need to override these if your cluster cannot pull from the default registry, for example when working in an air-gapped environment or when images must be sourced from a private registry. ```bash K8S_CURL_INIT_IMAGE=my-registry.example.com/curl-init:latest LAMBDA_K8S_INIT_IMAGE=my-registry.example.com/lambda-init:latest ``` ### Lambda image prefix `LAMBDA_K8S_IMAGE_PREFIX` sets a prefix applied to all Lambda runtime image names when pulling them in the Kubernetes executor. Use this to redirect image pulls to a private registry or mirror. ```bash LAMBDA_K8S_IMAGE_PREFIX=my-registry.example.com/lambda-images/ ``` ### Readiness timeouts LocalStack waits for child pods, deployments, and services to become ready before considering them available. The following variables control how long LocalStack waits before timing out: - `K8S_WAIT_FOR_POD_READY_TIMEOUT` — maximum time to wait for a pod to reach the `Ready` state - `K8S_WAIT_FOR_DEPLOYMENT_READY_TIMEOUT` — maximum time to wait for a deployment to become available - `K8S_WAIT_FOR_SERVICE_READY_TIMEOUT` — maximum time to wait for a service endpoint to be ready ```bash K8S_WAIT_FOR_POD_READY_TIMEOUT=120 K8S_WAIT_FOR_DEPLOYMENT_READY_TIMEOUT=180 K8S_WAIT_FOR_SERVICE_READY_TIMEOUT=60 ``` Increase these values if your cluster is under heavy load or if image pulls are slow. ### Configuration reference | Variable | Description | |---|---| | `K8S_NAMESPACE` | Kubernetes namespace for child pods | | `K8S_LABELS` | Comma-separated `key=value` labels applied to child pods | | `K8S_ANNOTATIONS` | Comma-separated `key=value` annotations applied to child pods | | `K8S_POD_CONFIG` | JSON pod configuration for supported child pods. See [Pod Configuration](/aws/customization/kubernetes/pod-configuration/) | | `K8S_CONTAINER_SECURITY_CONTEXT` | JSON security context applied to child pod containers | | `K8S_CURL_INIT_IMAGE` | Init container image used for network readiness checks | | `LAMBDA_K8S_INIT_IMAGE` | Init container image used in Lambda pods | | `LAMBDA_K8S_IMAGE_PREFIX` | Image name prefix for Lambda runtime images | | `K8S_WAIT_FOR_POD_READY_TIMEOUT` | Timeout waiting for pod readiness | | `K8S_WAIT_FOR_DEPLOYMENT_READY_TIMEOUT` | Timeout waiting for deployment readiness | | `K8S_WAIT_FOR_SERVICE_READY_TIMEOUT` | Timeout waiting for service readiness | # Deploy with Helm > Install and run LocalStack on Kubernetes using the official Helm chart. A Helm chart is a package that bundles Kubernetes manifests into a reusable, configurable deployment unit. It makes applications easier to install, upgrade, and manage. Using the LocalStack Helm chart lets you deploy LocalStack to Kubernetes with set defaults while still customizing resources, persistence, networking, and environment variables through a single `values.yaml`. This approach is especially useful for teams running LocalStack in shared clusters or CI environments where repeatable, versioned deployments matter. ## Getting Started This guide shows you how to install and run LocalStack on Kubernetes using the official Helm chart. It walks you through adding the Helm repository, installing and configuring LocalStack, and verifying that your deployment is running and accessible in your cluster. ## Prerequisites * **Kubernetes** 1.19 or newer * **Helm** 3.2.0 or newer * A working Kubernetes cluster (self-hosted, managed, or local) * `kubectl` installed and configured for your cluster * Helm CLI installed and available in your shell `PATH` :::note **Namespace note:** All commands in this guide assume installation into the **`default`** namespace. If you’re using a different namespace: * Add `--namespace ` (and `--create-namespace` on first install) to Helm commands * Add `-n ` to `kubectl` commands ::: ## Install ### 1) Add Helm repo ```bash helm repo add localstack https://localstack.github.io/helm-charts helm repo update ``` ### 2) Install with default configuration ```bash helm install localstack localstack/localstack ``` This creates the LocalStack resources in your cluster using the chart defaults. ### Install LocalStack for AWS If you want to use the `localstack-pro` image, create a `values.yaml` file: ```yaml image: repository: localstack/localstack-pro extraEnvVars: - name: LOCALSTACK_AUTH_TOKEN value: "" ``` Then install using your custom values: ```bash helm install localstack localstack/localstack -f values.yaml ``` #### Auth token from a Kubernetes Secret If your auth token is stored in a Kubernetes Secret, you can reference it using `valueFrom`: ```yaml extraEnvVars: - name: LOCALSTACK_AUTH_TOKEN valueFrom: secretKeyRef: name: key: ``` ## Configure chart The chart ships with sensible defaults, but most production setups will want a small `values.yaml` to customize behavior. ### View all default values ```bash helm show values localstack/localstack ``` ### Override values with a custom `values.yaml` Create a `values.yaml` and apply it during install/upgrade: ```bash helm upgrade --install localstack localstack/localstack -f values.yaml ``` ## Verify ### 1) Check the Pod status ```bash kubectl get pods ``` After a short time, you should see the LocalStack Pod in `Running` status: ```text NAME READY STATUS RESTARTS AGE localstack-7f78c7d9cd-w4ncw 1/1 Running 0 1m9s ``` ### 2) Optional: Port-forward to access LocalStack from localhost If you’re running a **local cluster** (for example, k3d) and LocalStack is not exposed externally, port-forward the service: ```bash kubectl port-forward svc/localstack 4566:4566 ``` Now verify connectivity with the AWS CLI: ```bash aws sts get-caller-identity --endpoint-url "http://0.0.0.0:4566" ``` Example response: ```json { "UserId": "AKIAIOSFODNN7EXAMPLE", "Account": "000000000000", "Arn": "arn:aws:iam::000000000000:root" } ``` ## Common customizations ### Enable persistence If you want state to survive Pod restarts, enable PVC-backed persistence: * Set: `persistence.enabled = true` Example `values.yaml`: ```yaml persistence: enabled: true ``` :::note This is especially useful for workflows where you seed resources or rely on state across restarts. ::: This performs these two changes: * sets `PERSISTENCE=1`, and * creates a psrsistent volume claim for the (customizable) storage class. ### Set Pod resource requests and limits Some environments (notably **EKS on Fargate**) may terminate the LocalStack pod if not configured with reasonable requests/limits: ```yaml resources: requests: cpu: 1 memory: 1Gi limits: cpu: 2 memory: 2Gi ``` ### Add environment variables and startup scripts You can inject environment variables or run a startup script to: * pre-configure LocalStack * seed AWS resources * tweak LocalStack behavior Use: * `extraEnvVars` for environment variables * `startupScriptContent` for startup scripts Example pattern: ```yaml extraEnvVars: - name: DEBUG value: "1" startupScriptContent: | echo "Starting up..." # add your initialization logic here ``` ### Install into a different namespace Use `--namespace` and create it on first install: ```bash helm install localstack localstack/localstack --namespace localstack --create-namespace ``` Then include the namespace on kubectl commands: ```bash kubectl get pods -n localstack ``` ### Update installation ```bash helm repo update helm upgrade localstack localstack/localstack ``` If you use a `values.yaml`: ```bash helm upgrade localstack localstack/localstack -f values.yaml ``` ### Helm chart options Run: ```bash helm show values localstack/localstack ``` Keep the parameter tables on this page for quick reference (especially for common settings like persistence, resources, env vars, and service exposure). # eksctl > Running `eksctl` on LocalStack to create EKS clusters. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction [eksctl](https://eksctl.io/) is a CLI tool for creating and managing EKS clusters, Amazon's managed Kubernetes service. LocalStack supports running `eksctl` on LocalStack to create EKS clusters locally. LocalStack's EKS spin up embedded Kubernetes clusters using [K3s](https://github.com/k3s-io/k3s) to allow you to use the EKS APIs in your local environment. :::note The support for `eksctl` is currently experimental and may not work in all cases. We are working on improving the support for `eksctl` in LocalStack. ::: ## Getting started This guide is designed for users new to `eksctl` and running EKS clusters with LocalStack. Start LocalStack using your preferred method. We will demonstrate how you can create a local EKS cluster using `eksctl` and fetch the nodes in the cluster. ### Pre-requisites - LocalStack with `LOCALSTACK_AUTH_KEY` configured - [Docker](https://www.docker.com/) - [`eksctl`](https://eksctl.io/) (version 0.167.0 or higher) - [`kubectl`](https://kubernetes.io/docs/tasks/tools/#kubectl) ### Create a cluster To create a cluster, you can use the `eksctl create cluster` command. You can use the `--profile` flag to [specify the LocalStack profile](https://docs.localstack.cloud/user-guide/integrations/aws-cli/#configuring-a-custom-profile) to use for the cluster. Run the following command to create a cluster: {/* TODO: change labels so they are formatted like **** in markdown */} ```bash eksctl create cluster --nodes 1 --profile localstack ``` ```bash export AWS_CLOUDFORMATION_ENDPOINT=http://localhost.localstack.cloud:4566 export AWS_EC2_ENDPOINT=http://localhost.localstack.cloud:4566 export AWS_EKS_ENDPOINT=http://localhost.localstack.cloud:4566 export AWS_ELB_ENDPOINT=http://localhost.localstack.cloud:4566 export AWS_ELBV2_ENDPOINT=http://localhost.localstack.cloud:4566 export AWS_IAM_ENDPOINT=http://localhost.localstack.cloud:4566 export AWS_STS_ENDPOINT=http://localhost.localstack.cloud:4566 eksctl create cluster --nodes 1 ``` ### Get the nodes You can use the `kubectl` command to get the nodes in the cluster: ```bash kubectl get nodes ``` # FAQ > FAQ This section covers common issues when running LocalStack on Kubernetes and how to diagnose them. ## The LocalStack Pod won’t start ### Potential issues * The auth token is invalid. If license activation fails, the Pod will terminate immediately. * The container image cannot be pulled or takes a long time to pull. * The Pod cannot be scheduled because no nodes have sufficient capacity or nodes are tainted. * **When using the Operator:** the Operator cannot validate the LocalStack license and refuses to create the Pod. ### Potential fixes 1. Check the Pod status List all Pods in the cluster and locate the LocalStack Pod: ```bash kubectl get pods -A ``` 2. Describe the Pod If the Pod exists, inspect it for scheduling or startup errors: ```bash kubectl describe pod -n ``` Errors are typically visible in the `Events:` section. Example: ```text Back-off restarting failed container localstack in pod ``` This indicates that the LocalStack container crashed after startup. 3. Check Pod logs If the Pod started but then crashed: ```bash kubectl logs -n ``` Startup errors are usually found near the end of the log output. Debugging is similar to running LocalStack in Docker—there is nothing Kubernetes-specific about most startup failures. 4. If the Pod was never created * **Helm:** check the output of `helm install` * **Operator:** check the Operator logs: ```bash kubectl logs -n localstack-operator-system deployments/localstack-operator-system ``` ## Services are unavailable If an AWS service fails to start or respond: * Verify that your LocalStack license includes the service you are trying to use. * Check the LocalStack Pod logs for license-related or service-specific errors. There are typically no additional Kubernetes-level actions required for this issue. ## DNS resolution failures ### DNS timeouts If you see errors such as: ```text Unable to get DNS result from upstream server for domain . The DNS operation timed out. ``` * Check whether the upstream DNS server was detected correctly. * Look for a log line like: ```text Determined fallback dns: ``` * If the detected DNS server is not valid for your cluster, set it explicitly using the `DNS_SERVER` configuration option. ### `localhost.localstack.cloud` does not resolve * **Inside LocalStack-spawned compute Pods** * Ensure `DNS_ADDRESS` is **not** set to `0`. * **Inside other Pods in the cluster** * Configure the Pod DNS settings to use the LocalStack Service IP. * Set `dnsPolicy: None` and define a custom DNS config and search domains. * See: [https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-dns-config](https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-dns-config) * **Alternative** * Configure your cluster DNS (for example CoreDNS) to forward requests ending in `localhost.localstack.cloud` to the LocalStack DNS server. * This is done automatically when using the LocalStack Operator. ## Permission / RBAC issues ### Docker permission errors ```text Creating Docker SDK client failed ``` This is expected in Kubernetes. LocalStack attempts to connect to a Docker socket, which is typically unavailable in Pods. Docker is **not required** for running LocalStack on Kubernetes. ### Filesystem permission errors ```text PermissionError: [Errno 13] Permission denied: '/etc/resolv.conf' ``` This is expected when running LocalStack as a non-root user. Transparent endpoint injection may not work for init scripts or extensions in this case. ### Kubernetes API permission errors ```text kubernetes.client.exceptions.ApiException: (403) Reason: Forbidden ``` This indicates missing or incorrect RBAC permissions for the LocalStack Pod’s ServiceAccount. * If using the official Helm chart or Operator, update to the latest version: * Helm: `helm repo update localstack` * Operator: re-apply the latest controller manifest * If deploying LocalStack manually (not recommended), ensure the ServiceAccount role includes the required permissions: [https://github.com/localstack/helm-charts/blob/main/charts/localstack/templates/role.yaml](https://github.com/localstack/helm-charts/blob/main/charts/localstack/templates/role.yaml) ## Networking problems ### LocalStack or spawned Pods cannot connect to other cluster resources * Ensure the LocalStack Pod uses: ```yaml dnsPolicy: ClusterFirst ``` ### LocalStack cannot connect to real AWS You may want LocalStack to connect to real AWS when testing hybrid workflows, forwarding specific requests, or accessing resources that are not fully emulated locally. Common causes for connection failure: * **Transparent Endpoint Injection** * Review: [https://docs.localstack.cloud/aws/customization/networking/transparent-endpoint-injection/](https://docs.localstack.cloud/aws/customization/networking/transparent-endpoint-injection/) * **Egress restrictions** * Ensure cluster network policies allow outbound internet access. ### Spawned Pods cannot connect to LocalStack * Spawned compute workloads should connect to LocalStack using: ```text localhost.localstack.cloud ``` * Using the Kubernetes Service name is possible, but host-based access (for example S3 virtual-host addressing) will not work. * Verify that the LocalStack DNS server is running (see DNS resolution failures). ### Other Pods cannot connect to LocalStack * Use the Kubernetes Service name created by Helm or the Operator. * Confirm the LocalStack Pod is running. :::note API responses returned by LocalStack include the `localhost.localstack.cloud` domain. You can override this by setting: ```text LOCALSTACK_HOST= ``` ::: ### Cannot connect from the host machine * For local clusters (k3d, kind): * Ensure ports are forwarded correctly. * Port `4566` (and service ports `4510–4559`) must be exposed. * See: * [https://k3d.io/v5.3.0/usage/exposing_services/](https://k3d.io/v5.3.0/usage/exposing_services/) * [https://kind.sigs.k8s.io/docs/user/configuration/#extra-port-mappings](https://kind.sigs.k8s.io/docs/user/configuration/#extra-port-mappings) * Alternatively, use port-forwarding: ```bash kubectl port-forward -n 4566 ``` ## Child containers are not spawning ### Certificate issues when spawning child pods If you experience the following error when creating child pods (i.e., Lambda pod failure), then your proxy settings might be applied to cluster internal communication: ``` localstack.services.lambda_.invocation.assignment.AssignmentException: Could not start new environment: MaxRetryError:MyHTTPSConnectionPool(host='192.168.0.1', port=443): Max retries exceeded with url: /api/v1/namespaces/ns-perf-a39e28bf-c600-498d-9ecc-41419eca1007/pods/lambda-pod-52c280f8dd194dc72bced60e190db6ef/log (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1032)'))) ``` If you are using `HTTP_PROXY` or `HTTPS_PROXY` environment variables to configure a TLS terminating proxy server (i.e., corporate environments), then you may need to add the Kubernetes API server IP address to the `NO_PROXY` environment variable. In the example above, add `NO_PROXY=192.168.0.1` to your pod environment variables. For Lambda child pods, LocalStack automatically extends the Lambda execution environment's `no_proxy` value with the configured `LOCALSTACK_HOST` host, such as `localhost.localstack.cloud`. This ensures Lambda runtime traffic to LocalStack bypasses corporate proxies. If your cluster uses an admission controller or webhook to inject proxy variables into pods, ensure that it does not overwrite this value, or configure it to include the LocalStack host in `no_proxy`/`NO_PROXY`. ### Docker runtime errors ```text DockerNotAvailable: Docker not available ``` * Ensure `CONTAINER_RUNTIME=kubernetes` is set. * Verify your license includes Kubernetes support. * When using Helm, ensure `lambda.executor: kubernetes` is not overriding the runtime unintentionally. ### Image pull failures ```text ErrImagePull ``` This usually means the cluster restricts which images can be pulled. * Allow the LocalStack images in your cluster * If you must use a custom image name or pull-through cache, see: [https://docs.localstack.cloud/aws/customization/configuration-options/](https://docs.localstack.cloud/aws/customization/configuration-options/) ### Admission or security policy failures Errors such as: ```text kubernetes.utils.create_from_yaml.FailToCreateError ``` * Ensure you are running the latest LocalStack and Helm chart / Operator version. * Check for: * Validating admission webhooks [https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#validatingadmissionwebhook](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#validatingadmissionwebhook) * Pod Security Admission restrictions [https://kubernetes.io/docs/concepts/security/pod-security-admission/](https://kubernetes.io/docs/concepts/security/pod-security-admission/) ### Child Pod timeouts If a child Pod is created but times out: * This is commonly caused by slow or blocked image pulls. * Inspect the child Pod status and events, similar to diagnosing a LocalStack Pod startup failure. ## Logs to check * Check LocalStack Pod logs as described in **The LocalStack Pod won’t start** * If using the Operator, also check the Operator logs: ```bash kubectl logs -n localstack-operator-system deployments/localstack-operator-system ``` # Kubernetes Executor > Configuring Kubernetes Executor for compute services in LocalStack Enterprise. LocalStack Enterprise provides a Kubernetes executor for various emulated services. It allows you to run these services as Kubernetes pods in your Kubernetes clusters. By default, LocalStack uses the `docker` backend for these services. You can use either service-specific configuration variables or the generic `CONTAINER_RUNTIME` variable set to `kubernetes` to enable the Kubernetes executor. ## EC2 Kubernetes Executor The LocalStack Enterprise image allows you to run EC2 instances on Kubernetes. You can do so by setting the `EC2_VM_MANAGER` environment variable to `kubernetes` in the LocalStack container. Each EC2 instance in the Kubernetes VM manager is backed by a Pod. The following operations are supported: | Operation | Notes | | :------------------- | :------------------------------------- | | `DescribeInstances` | Returns all EC2 instances | | `RunInstances` | Defines and starts an EC2 instance | | `StartInstances` | Starts an already defined EC2 instance | | `StopInstances` | Stops a running EC2 instance | | `TerminateInstances` | Stops and undefines a EC2 instance | The current implementation is in preview and does not support volumes, custom AMIs, or networking features available in other VM managers. ## ECS Kubernetes Executor The LocalStack Enterprise image allows you to run ECS tasks on Kubernetes. The tasks are added to ELB load balancer target groups. You can do so by setting the `ECS_TASK_EXECUTOR` environment variable to `kubernetes` in the LocalStack container. ## Lambda Kubernetes Executor The LocalStack Enterprise image allows you to execute Lambda functions as Kubernetes pods. You can do so by setting `LAMBDA_RUNTIME_EXECUTOR` ( or `lambda.executor` when using the Helm configuration) to `kubernetes`. For more information, see the [Helm Chart configuration](https://github.com/localstack/helm-charts/blob/ce47b1590605901650ab788556bc871efbd78b8d/charts/localstack/values.yaml#L178-L208). - Kubernetes Lambda Executor in LocalStack scales Lambda execution by spawning new environments (running in pods) during concurrent invocations. Inactive environments shut down after 10 minutes (configurable via `LAMBDA_KEEPALIVE_MS`). - Executor schedules multiple Lambda functions according to Kubernetes cluster defaults without specifying node affinity. Users can assign labels to lambda pods using the `K8S_LABELS` variable (e.g., `K8S_LABELS=key=value,key2=value2`). - Timeout configurations similar to AWS are enforced using the `Timeout` function parameter. No intrinsic limits on the number of Lambdas; default limit on concurrent executions is 1000 (`LAMBDA_LIMITS_CONCURRENT_EXECUTIONS`). - Custom DNS configuration for Lambda on Kubernetes can be set through the `LAMBDA_DOCKER_DNS` configuration variable. - Users can customize Lambda runtime behavior by building custom images, pushing them to their registry, and specifying these images using the `LAMBDA_RUNTIME_IMAGE_MAPPING` configuration variable. - Lambda on Kubernetes supports Warm Start and Persistence. Persistence must be configured for the LocalStack pod. The `/var/lib/localstack` directory should be persisted over LocalStack runs, typically in a volume. Lambda hot reloading & remote debugging are not supported in the Kubernetes executor as the bind mounting into pods cannot be done at runtime. ## Other services You can run the following services on Kubernetes clusters using the LocalStack Enterprise image: - [DocumentDB](/aws/services/docdb) - [MWAA](/aws/services/docdb) - [Glue](/aws/services/glue) - [RDS](/aws/services/rds) ([MySQL](/aws/services/rds/#mysql-engine) & [MSSQL](/aws/services/rds/#microsoft-sql-server-engine)) To use Kubernetes as the runtime backend, set the `CONTAINER_RUNTIME` configuration variable to `kubernetes`. Note that there are no service-specific configuration variables for these services. # Deploy LocalStack Operator > Deploy and manage LocalStack in a Kubernetes cluster using the LocalStack Operator. import OperatorPermissionsTable from '../../../../../components/kubernetes-operator/OperatorPermissionsTable'; The LocalStack Operator is our Kubernetes-native way to deploy and manage LocalStack instances. It abstracts Kubernetes-specific configuration and automates operational tasks, making LocalStack deployments more consistent and easier to maintain. It can manage multiple LocalStack instances within a cluster to provide isolated local clouds for multiple users. The Operator manages the full lifecycle of LocalStack resources and enables advanced Kubernetes integrations that are difficult to configure manually. ## Getting started This guide explains how to deploy and manage LocalStack in a Kubernetes cluster using the LocalStack Operator. ### Advanced features The Operator supports the following advanced capabilities: * Opt-in DNS and endpoint injection for Kubernetes workloads * Cluster DNS configuration to resolve AWS-style subdomains in the same namespace * Automatic loading of Cloud Pods on startup * Support for initialization hooks * Simplified logging configuration * Automatic mounting of a PersistentVolumeClaim (PVC) for the LocalStack data directory, enabling artifact caching and persistence ### Prerequisites Before installing the LocalStack Operator, ensure you have: * A running Kubernetes cluster * A LocalStack license that includes Kubernetes features * An authentication token for that license :::note A separate auth token must be used for each LocalStack instance. ::: ### Deploy Operator The easiest way to install the Operator controller is to apply the published manifests directly from GitHub. ```bash # Install the latest version kubectl apply -f https://github.com/localstack/localstack-operator/releases/latest/download/controller.yaml ``` To install a specific version: ```bash # Example: install v0.4.0 kubectl apply -f https://github.com/localstack/localstack-operator/releases/v0.4.0/download/controller.yaml ``` See the [Operator releases page](https://github.com/localstack/localstack-operator/releases) for all available versions. ### Deploy LocalStack instance Once the Operator is running, you can deploy a LocalStack instance by creating a `LocalStack` custom resource. A minimal example looks like this: ```yaml apiVersion: api.localstack.cloud/v1alpha1 kind: LocalStack metadata: name: localstack namespace: my-namespace spec: image: localstack/localstack-pro:latest dnsProvider: coredns dnsConfigName: coredns dnsConfigNamespace: kube-system envFrom: - secretRef: name: localstack-auth-token ``` In this example, the LocalStack auth token is read from a Kubernetes Secret named `localstack-auth-token`. You can create this Secret with: ```bash kubectl create secret generic localstack-auth-token \ --from-literal=LOCALSTACK_AUTH_TOKEN="$LOCALSTACK_AUTH_TOKEN" ``` With this example, the auth token must be available in the `LOCALSTACK_AUTH_TOKEN` environment variable when creating the Secret. :::note The Secret must be created in the **same namespace** as the `LocalStack` resource. In the example above, that is `my-namespace`. ::: :::note More advanced examples are available in the LocalStack Operator GitHub repository. ::: ## Accessing LocalStack By default, the Operator creates a `ClusterIP` Service named: ```text localstack- ``` For the example above (`name: localstack`), the Service name is: ```text localstack-localstack ``` This Service exposes: * The LocalStack gateway port (`4566`) * AWS service ports * Port `53` for DNS Using standard Kubernetes DNS resolution, the Service can be reached at: * `localstack-localstack` (same namespace) * `localstack-localstack.my-namespace` * `localstack-localstack.my-namespace.svc.cluster.local` When `dnsProvider: coredns` is configured, LocalStack can also be reached through **any subdomain** of these service names. ## Inject LocalStack endpoints into workloads :::note Transparent endpoint injection requires LocalStack Operator version 0.4.13 or later. ::: The Operator can configure individual workloads to use the LocalStack instance in their namespace without changing cluster-wide DNS. It uses an admission webhook to update opted-in Pods when Kubernetes creates them. Pods without the opt-in label are not affected. ### Opt in a workload Add the `localstack.cloud/inject-dns: "true"` label to the Pod template of each Deployment, StatefulSet, Job, or other workload that needs to access LocalStack: ```yaml spec: template: metadata: labels: localstack.cloud/inject-dns: "true" ``` Add the same label directly under `metadata.labels` when you create a standalone Pod. ### Injected configuration For an opted-in Pod, the Operator: * Configures the Pod's DNS to send requests to LocalStack first, before falling back to the cluster DNS resolver. LocalStack only responds to domain names it owns, that is, those ending in `localhost.localstack.cloud` or your configured `$LOCALSTACK_HOST`, so requests such as `my-bucket.s3.localhost.localstack.cloud` resolve to LocalStack while every other DNS request, including in-cluster Service names, is forwarded to the cluster resolver as normal. * Sets `AWS_ENDPOINT_URL` to `http://localstack-.:4566` in every regular, init, and ephemeral container. [AWS SDKs and tools that support this setting](https://docs.aws.amazon.com/sdkref/latest/guide/feature-ss-endpoints.html#ss-endpoints-support) use the LocalStack endpoint without application-specific endpoint configuration. If a container already defines `AWS_ENDPOINT_URL`, the Operator replaces its value with the LocalStack endpoint. An endpoint configured directly in application code, with the AWS CLI `--endpoint-url` option, or with a service-specific endpoint environment variable takes precedence over `AWS_ENDPOINT_URL`. :::note The injected DNS search domains assume that the cluster uses the default `cluster.local` domain. Clusters configured with a different cluster domain require additional DNS configuration. ::: ### Injection requirements and behavior The namespace must contain exactly one `LocalStack` resource. If the namespace contains no `LocalStack` resources or more than one, the Operator creates the Pod without injecting DNS or the endpoint. The Operator also skips injection when the LocalStack Service does not have a usable ClusterIP. Injection occurs only when Kubernetes creates a Pod. After adding the label to an existing workload, recreate its Pods to apply the configuration. The webhook does not block Pod creation. If the Operator or LocalStack Service is unavailable, Kubernetes creates the Pod without the injected configuration. ## Manage webhook certificates The endpoint injection webhook uses TLS. Choose how the Operator manages its serving certificate when you install the Operator. ### Self-managed certificates The default `self-managed` mode requires no additional cluster components. The Operator creates and rotates the certificate and keeps the Kubernetes API server's trust configuration up to date. Existing installations continue to use this mode without configuration changes. ### cert-manager certificates Use `cert-manager` mode to delegate certificate issuance and rotation to [cert-manager](https://cert-manager.io/docs/). Install cert-manager and create an `Issuer` or `ClusterIssuer` before starting the Operator in this mode. The Operator reports an error at startup if the cert-manager `Certificate` resource is unavailable. Configure the Operator manager with the following flags or equivalent environment variables: | Flag | Environment variable | Default | Description | | --- | --- | --- | --- | | `--certificate-mode` | `CERTIFICATE_MODE` | `self-managed` | Set to `cert-manager` to delegate certificate management. | | `--certificate-issuer-name` | `CERTIFICATE_ISSUER_NAME` | Unset | Name of the issuer. Required in `cert-manager` mode. | | `--certificate-issuer-kind` | `CERTIFICATE_ISSUER_KIND` | `ClusterIssuer` | Set to `Issuer` or `ClusterIssuer`. | When you install the Operator from the published `controller.yaml`, add the environment variables to the Operator's ConfigMap and restart the manager: ```bash kubectl patch configmap localstack-operator-controller-manager-config \ --namespace localstack-operator-system \ --type merge \ --patch '{"data":{"CERTIFICATE_MODE":"cert-manager","CERTIFICATE_ISSUER_NAME":"","CERTIFICATE_ISSUER_KIND":"ClusterIssuer"}}' kubectl rollout restart deployment/localstack-operator-controller-manager \ --namespace localstack-operator-system ``` Replace `` with the name of your issuer. Set `CERTIFICATE_ISSUER_KIND` to `Issuer` if you use a namespaced issuer. Use a cert-manager [`CA` issuer](https://cert-manager.io/docs/configuration/ca/) rather than a `SelfSigned` issuer. :::caution Before switching from `cert-manager` back to `self-managed`, delete the Operator's `webhook-server-cert` `Certificate` resource. Otherwise, cert-manager and the Operator both update the same certificate Secret. ::: The webhook does not block Pod creation while you switch certificate modes. ## CRDs The LocalStack Operator introduces a `LocalStack` Custom Resource Definition (CRD) that controls how LocalStack instances are deployed and configured. :::Note CRD documentation is currently maintained manually. For a full reference of available fields, see: [https://github.com/localstack/localstack-operator/blob/main/api-docs.md](https://github.com/localstack/localstack-operator/blob/main/api-docs.md) ::: ## Permissions The Operator manifest creates all required `Roles`, `ClusterRoles`, and `bindings`. ### Manager ClusterRole This ClusterRole allows the Operator to manage LocalStack resources and related Kubernetes objects. Resources include: * Pods (including exec and logs) * Services * Secrets * Deployments * ServiceAccounts * LocalStack CRDs (`localstacks`, `status`, `finalizers`) * RBAC roles and role bindings Verbs include: ```text create, delete, get, list, watch, patch, update ``` ### Metrics and proxy ClusterRoles Additional ClusterRoles are created for: * Reading metrics (`/metrics`) * Authentication and authorization reviews (`tokenreviews`, `subjectaccessreviews`) ## DNS handling With `dnsProvider: coredns`, the LocalStack Operator configures cluster DNS to forward AWS-style subdomain requests to the LocalStack DNS server. This enables features such as: * S3 virtual-host–style addressing * API Gateway domain name resolution Example from another pod in the cluster: ```bash aws apigatewayv2 create-api \ --name testGatewayProxy \ --protocol-type HTTP \ --target "https://httpbin.org" ``` Example response: ```json { "ApiEndpoint": "http://1d4b6907.execute-api.localstack-localstack.my-namespace:4566", "ApiId": "1d4b6907" } ``` Calling the API: ```bash curl http://1d4b6907.execute-api.localstack-localstack.my-namespace:4566/json ``` This works without additional DNS configuration in client applications. ### EKS Auto Mode Amazon EKS Auto Mode does not expose the CoreDNS configuration for the Operator to modify. Set `dnsProvider: eksauto` on the `LocalStack` resource to prevent the Operator from trying to update CoreDNS: ```yaml spec: # ... other fields dnsProvider: eksauto dnsConfigName: coredns dnsConfigNamespace: kube-system ``` The `dnsConfigName` and `dnsConfigNamespace` fields remain required by the `LocalStack` resource schema, but the Operator does not use their values in `eksauto` mode. Use opt-in [endpoint injection](#inject-localstack-endpoints-into-workloads) to configure workloads that need to access LocalStack. ## Storage :::note Requires Operator version 0.4.0 or later. ::: To persist the [LocalStack volume](/aws/customization/advanced/filesystem/#localstack-volume), use the `spec.pvcName` to specify the PVC you want to mount. This automatically mounts the PVC at `/var/lib/localstack`. For example: ```yaml # pvc definition apiVersion: v1 kind: PersistentVolumeClaim metadata: name: myPvc spec: # ... # LocalStack instance definition apiVersion: api.localstack.cloud/v1alpha1 kind: LocalStack metadata: name: localstack namespace: my-namespace spec: # ... pvcName: myPvc ``` ## Update The Operator can be upgraded by applying a newer controller manifest. Example: ```bash # Install an older version kubectl apply -f https://github.com/localstack/localstack-operator/releases/download/v0.3.3/controller.yaml # Upgrade to a newer version kubectl apply -f https://github.com/localstack/localstack-operator/releases/download/v0.4.1/controller.yaml ``` Kubernetes will handle rolling updates of the Operator deployment. ## Verify To verify that the Operator and LocalStack instance are running: ```bash kubectl get pods -n my-namespace kubectl get localstacks -n my-namespace ``` Ensure that: * The Operator controller pod is running * The LocalStack resource reports a healthy status * The LocalStack Service is created :::note * The LocalStack Operator is available **only with licenses that include Kubernetes features** * Each LocalStack instance requires its **own auth token** * DNS integration and Cloud Pod automation are Operator-exclusive features ::: # Limitations > Known limitations when running LocalStack on Kubernetes Some LocalStack services have limited or no support when running on Kubernetes. :::note We are continually working on improving parity between Docker and Kubernetes so there will be fewer limitations in the future. ::: ## Unsupported services The following services require the use of Docker, and are not supported when running LocalStack on Kubernetes: - Sagemaker - Bedrock - EKS - CodeBuild ## Limitations of partially supported services ### SSM - Exec into EC2 instances is not supported. ### ECS - Firelens is not supported. - Volumes are not supported. - Exposing ports from tasks is not supported. - [`RepositoryCredentials`](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_RepositoryCredentials.html) is not supported. ### EC2 - Userdata does not have full parity with AWS. ### ElastiCache/MemoryDB - Requires using `REDIS_CONTAINER_MODE=1`. ### RDS - MySQL 5.7 is not supported on Kubernetes. - Persistence is not supported. ### Neptune - Persistence is not supported. # OpenShift > Use the OpenShift managed Kubernetes cluster to deploy LocalStack. ## Introduction OpenShift is a container orchestration platform as a service designed to simplify the deployment, scaling, and management of containerized applications. Built on Kubernetes, OpenShift provides a comprehensive set of tools and features that facilitate the orchestration, automation, and monitoring of containerized workloads. With OpenShift, you can deploy LocalStack on a managed Kubernetes cluster, as a cloud sandbox that emulates various AWS services & APIs. This guide demonstrates how you can deploy LocalStack on OpenShift using Devfile. You can use the deployed LocalStack container to create AWS resources that you can use for local development and testing purposes. :::danger Creating shared/hosted LocalStack instances may have some licensing implications. For example, a valid license might be necessary for each user who interacts with the instance. If you have any questions or uncertainties regarding the licensing implications, we encourage you to [contact us](https://localstack.cloud/contact) for further details. ::: ## Getting started This guide is designed for users new to LocalStack and assumes basic knowledge of the AWS CLI and our [`lstk aws` AWS CLI proxy](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws). As a general prerequisite, you should have access to the [OpenShift Web Console](https://docs.openshift.com/container-platform/4.14/web_console/web-console-overview.html). We will demonstrate how you can create local AWS resources using LocalStack using the AWS CLI. Instead of running LocalStack locally, you will deploy it on OpenShift and use the exposed endpoint to interact with the LocalStack container. ### Setting up LocalStack on OpenShift You can deploy LocalStack via the **Developer** perspective in the OpenShift Web Console. Navigate to the **+Add** view to deploy LocalStack using a Devfile. ![OpenShift Developer perspective](/images/aws/openshift-developer-view.png) To deploy LocalStack on OpenShift, click on **Import from Git** in the **Git Repository** tile. In the Git section, enter the following Git repository URL to import the Devfile and Helm charts which contains the configuration for LocalStack: [**https://github.com/localstack/localstack-dev-spaces**](https://github.com/localstack/localstack-dev-spaces). OpenShift Web Console will automatically detect the Devfile and display the import strategy. A unique application name will be generated to the application grouping to label your resources. A unique name will also be provided to the component that will be used to name associated resources. You can edit these values if you want. Click on **Create** to deploy LocalStack on OpenShift. ### Viewing the LocalStack deployment You can see the build status of the LocalStack deployment in the **Topology** view. ![OpenShift Topology view](/images/aws/openshift-topology-view.png) After successful deployment, you can see the **localstack-dev-spaces** pod in the **Topology** view. Click on the pod to view the details. You will be able to see the following details: - Running pods along with the status and logs. - Builds for your existing pods and an option to create new builds. - Exposed services along with the service port and the pod port. - Exposed routes for your deployed pods on the cluster. ![LocalStack Dev Spaces Deployment](/images/aws/localstack-dev-spaces.png) ### Creating AWS resources on OpenShift Click on the **localstack-dev-spaces** pod to view the details. You will be able to see the exposed route for the LocalStack container. Copy the route URL and use it to interact with the LocalStack container. Since LocalStack is running on the cluster rather than on your machine, point `lstk` at the exposed route instead of its default local endpoint. Set `LSTK_ENDPOINT_URL` once for the whole session: ```bash export LSTK_ENDPOINT_URL='' lstk aws s3 mb s3://my-bucket lstk aws sqs create-queue --queue-name my-queue ``` Alternatively, pass the global `--endpoint-url` flag per command: ```bash lstk --endpoint-url '' aws s3 mb s3://my-bucket lstk --endpoint-url '' aws sqs create-queue --queue-name my-queue ``` In the above commands, replace `` with the route URL of the LocalStack container. :::note By default, `lstk aws` targets the emulator on your local machine. Since we are running LocalStack on OpenShift, we need to specify the route URL of the LocalStack container, using either `LSTK_ENDPOINT_URL` or `--endpoint-url`. The flag takes precedence over the environment variable. ::: You can further use the other `lstk` tool proxies, such as [`lstk cdk`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#cdk), [`lstk sam`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#sam), and [`lstk terraform`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#terraform), to interact with the deployment. As with `lstk aws`, set `LSTK_ENDPOINT_URL` or pass `--endpoint-url` to point these at the route URL instead of a local emulator. ### Deleting the LocalStack deployment To delete the LocalStack deployment, click on the **localstack-dev-spaces** pod in the **Topology** view. Click on the **Actions** menu and select **Delete Deployment**. # Pod Configuration > Configure Kubernetes pods launched by LocalStack services. ## Introduction When LocalStack runs inside Kubernetes with the Kubernetes executor enabled, some services create child pods for workloads such as Lambda invocations and ECS tasks. In heterogeneous clusters, these child pods may need additional Kubernetes configuration so they are scheduled onto the right node pools, carry the right resource requests, or integrate with cluster policies. Use the `K8S_POD_CONFIG` environment variable to configure Kubernetes metadata, scheduling, and resource settings for LocalStack-spawned pods. The variable accepts a JSON object with reusable `profiles` and optional per-service mappings. The value must be valid JSON. LocalStack validates this configuration at startup and refuses to start if the value is not valid JSON or contains unknown fields, so misconfigurations surface before any child pods are created. ## Supported services `LOCALSTACK_K8S_POD_CONFIG` applies to child pods created by the following services: - Lambda (`lambda`) - ECS (`ecs`) - CodeBuild (`codebuild`) - DocumentDB (`docdb`) - ElastiCache (`elasticache`) - Kafka (`kafka`) - Kinesis Data Analytics (`kinesisanalyticsv2`) - MWAA (`mwaa`) - RDS (`rds`), for the MySQL and SQL Server (MSSQL) engines only - Glue (`glue`) - EC2 (`ec2`) The value in parentheses is the key to use for that service under the `services` block. :::note RDS pod configuration only applies to the MySQL and SQL Server engines. The PostgreSQL and MariaDB engines run in-process inside the main LocalStack pod rather than spawning a separate child pod, so `LOCALSTACK_K8S_POD_CONFIG` doesn't apply to them. ::: ## Supported fields Each profile can define the following fields: | Field | Description | |---|---| | `tolerations` | Kubernetes tolerations applied to the pod spec | | `nodeSelector` | Node selector applied to the pod spec | | `affinity` | Kubernetes affinity configuration applied to the pod spec | | `topologySpreadConstraints` | Topology spread constraints applied to the pod spec | | `priorityClassName` | Priority class name applied to the pod spec | | `resources` | Resource requests and limits applied to every non-init container in the pod | | `labels` | Labels merged into the pod metadata | | `annotations` | Annotations merged into the pod metadata | Scheduling fields such as `nodeSelector`, `tolerations`, and `affinity` replace the corresponding values on the generated pod spec. They are not merged with existing spec values. Resource settings are applied to the pod's application containers; init containers keep their generated defaults. ## Configure default profiles The simplest configuration is to define default profiles. LocalStack uses the architecture-specific defaults for Lambda and ECS when the workload architecture is known, and falls back to `default` otherwise. ```json { "profiles": { "default": { "nodeSelector": { "pool": "general" } }, "defaultArm64": { "nodeSelector": { "pool": "arm-nodes" } }, "defaultAmd64": { "nodeSelector": { "pool": "amd-nodes" } } } } ``` With this configuration: - Lambda and ECS ARM workloads use the `defaultArm64` profile. - Lambda and ECS x86 workloads use the `defaultAmd64` profile. - Workloads without architecture information use the `default` profile. To use this with the Helm chart, pass the JSON as an environment variable in your `values.yaml`: ```yaml extraEnvVars: - name: K8S_POD_CONFIG value: | { "profiles": { "default": { "nodeSelector": { "pool": "general" } }, "defaultArm64": { "nodeSelector": { "pool": "arm-nodes" } }, "defaultAmd64": { "nodeSelector": { "pool": "amd-nodes" } } } } ``` If you deploy LocalStack with the [LocalStack Operator](/aws/customization/kubernetes/kubernetes-operator/), set the same configuration as structured YAML under `spec.podSchedulingConfig` in your `LocalStack` resource instead of passing JSON: ```yaml apiVersion: api.localstack.cloud/v1alpha1 kind: LocalStack spec: # other fields podSchedulingConfig: profiles: default: nodeSelector: pool: general defaultArm64: nodeSelector: pool: arm-nodes defaultAmd64: nodeSelector: pool: amd-nodes ``` ## Configure per-service profiles Use the `services` block when a service needs a dedicated profile. A service can reference a single profile with `profile`, or it can map `arm64` and `amd64` workloads to different profiles. ```json { "profiles": { "default": { "nodeSelector": { "pool": "general" } }, "defaultAmd64": { "nodeSelector": { "pool": "amd-nodes" } }, "lambda-arm": { "nodeSelector": { "pool": "lambda-arm-nodes" }, "resources": { "requests": { "cpu": "500m", "memory": "512Mi" }, "limits": { "cpu": "2", "memory": "2Gi" } } }, "ecs-tasks": { "nodeSelector": { "pool": "ecs-nodes" } }, "database": { "nodeSelector": { "pool": "db-nodes" } } }, "services": { "lambda": { "arm64": "lambda-arm" }, "ecs": { "profile": "ecs-tasks" }, "rds": { "profile": "database" } } } ``` In this example, ARM Lambda pods use `lambda-arm`. AMD64 Lambda pods use `defaultAmd64`. ECS pods always use `ecs-tasks`, and RDS pods always use `database`, regardless of architecture, because `services..profile` bypasses architecture-based routing. Workloads without architecture information use `default`. ## Profile resolution LocalStack resolves the profile for a child pod in this order: 1. `services..profile` 2. `services..arm64` or `services..amd64` 3. `profiles.defaultArm64` or `profiles.defaultAmd64` 4. `profiles.default` 5. No scheduling, resource, or metadata profile is applied. System labels are still added. `defaultArm64`, `defaultAmd64`, and `default` are reserved profile names. Use them only for global defaults. Architecture values are normalized before lookup. For example, `ARM64` is treated as `arm64`, and `x86_64` or `X86_64` are treated as `amd64`. The keys in `K8S_POD_CONFIG` should still be `arm64` and `amd64`. If a service references a profile that does not exist, LocalStack logs a warning and applies no pod configuration for that request. It does not silently fall back to `default`. ## Labels and annotations `labels` and `annotations` in `K8S_POD_CONFIG` are merged into the metadata of the generated child pod. Profile labels override labels set through `K8S_LABELS`, and profile annotations override annotations set through `K8S_ANNOTATIONS`. LocalStack also injects the following system labels: ```yaml app.kubernetes.io/managed-by: localstack localstack.cloud/service: ``` System labels are applied last and cannot be overridden. You can use these labels to target LocalStack-spawned pods from Kubernetes tooling such as network policies, admission controllers, or monitoring agents. ## Example with scheduling and metadata The following example schedules Lambda pods on dedicated nodes, adds tolerations, sets resource limits, and attaches metadata used by platform tooling: ```json { "profiles": { "lambda": { "nodeSelector": { "workload": "localstack-lambda" }, "tolerations": [ { "key": "dedicated", "operator": "Equal", "value": "localstack", "effect": "NoSchedule" } ], "resources": { "requests": { "cpu": "250m", "memory": "256Mi" }, "limits": { "cpu": "1", "memory": "1Gi" } }, "labels": { "team": "platform" }, "annotations": { "prometheus.io/scrape": "true" } } }, "services": { "lambda": { "profile": "lambda" } } } ``` ## Related configuration - Use `K8S_NAMESPACE` to choose the namespace for child pods. - Use `K8S_LABELS` and `K8S_ANNOTATIONS` for simple labels and annotations that apply to all child pods. - Use `K8S_CONTAINER_SECURITY_CONTEXT` to configure the security context for child pod containers. For the complete list of Kubernetes executor configuration variables, see the [Kubernetes configuration reference](/aws/customization/kubernetes/configuration/). # Logging > Control LocalStack log output, verbosity, and error reporting. LocalStack supports logging output and error reporting through the `lstk` CLI or a Docker/Docker Compose based setup. LocalStack's logging setup allows you to: - Discover errors in your code during development & testing. - Get visibility into how and why your API calls are failing. - Figure out unexpected errors such as Lambda timeouts and more! With LocalStack logging, you can easily retrieve additional detail around errors using various configuration variables to specify the verbosity and the log level. ## Log Level You can explicitly set a log level via two configuration variables: `DEBUG` and `LS_LOG`. You can configure them while starting the LocalStack container, either with the CLI or a Docker/Docker-Compose setup. `DEBUG` can be either `0` or `1` (`0` is the default). With `DEBUG`, you can print more verbose logs, useful for troubleshooting issues. With `DEBUG=1`, errors inside LocalStack are reported to the client in full, and these stack traces can help you better triage your issues. `LS_LOG` supports the following values: - `trace` - `trace-internal` - `debug` - `info` - `warn` - `error` - `warning` The `LS_LOG` affects the log handlers level directly. If `LS_LOG` is configured as `trace` or `trace-internal`, it will automatically set `DEBUG=1`. To retrieve the debug information, it is recommended to set `DEBUG=1`. While configuring `LS_LOG` as `trace` or `trace-internal`, the LocalStack container will report the same log format but append the request and response objects and the HTTP headers to the log line. ## Error reporting AWS requests are logged uniformly in the `INFO` log level (set by default or when `DEBUG=0`). The shape is `AWS . => ()`. Requests to HTTP endpoints are logged in a similar way. ```bash 2022-07-12T10:12:03.250 INFO --- [ asgi_gw_0] localstack.request.aws : AWS s3.PutObject => 404 (NoSuchBucket) 2022-07-12T10:12:11.295 INFO --- [ asgi_gw_0] localstack.request.aws : AWS s3.CreateBucket => 200 2022-07-12T10:12:13.159 INFO --- [ asgi_gw_1] localstack.request.aws : AWS s3.PutObject => 200 2022-07-12T10:12:28.761 INFO --- [ asgi_gw_0] localstack.request.http : GET /_localstack/health => 200 ``` ## Log inspection You can inspect the logs of the LocalStack container using [`lstk`](/aws/developer-tools/running-localstack/lstk) or your Docker/Docker Compose setup. With `lstk`, you can run the following command to inspect the logs of the LocalStack container: ```bash lstk logs ``` By default this prints the currently available logs, with noisy internal lines filtered out. Add `--follow` to stream logs in real time, and `--verbose` to show every line unfiltered: ```bash # Stream filtered logs in real-time lstk logs --follow # Stream all logs without filtering lstk logs --follow --verbose ``` With Docker/Docker-Compose, you can run `docker ps` to get the container ID of the LocalStack container and then run `docker logs ` to inspect the logs. To view the logs via a user interface, you can use the following options: - [LocalStack Desktop](/aws/developer-tools/running-localstack/localstack-desktop/) - [LocalStack Docker Extension](/aws/customization/other-installations/localstack-docker-extension/) # Overview > Expose endpoints, resolve DNS, and integrate LocalStack with your local network. import CardGridLayout from '../../../../../components/CardGridLayout.astro'; If you're having trouble connecting your application to LocalStack, you're likely running into a networking mismatch. This section helps you identify the right troubleshooting path based on where your code is running. ## Networking Troubleshooting Choose the scenario below that best describes your networking layout. Whether you're running code on your host machine, in a container, or across multiple hosts, simply follow the corresponding guide to resolve common connection issues. :::tip LocalStack only binds to IPv4 addresses (e.g. `127.0.0.1`). Make sure you're not trying to access LocalStack over IPv6. ::: ## [Using the endpoint URL](/aws/customization/networking/accessing-endpoint-url) For example, setting the `endpoint_url` parameter with an [AWS SDK](/aws/connecting/aws-sdks/). :::note TLS certificates for `localhost.localstack.cloud` support only certain AWS regions. See [TLS Certificate Coverage](/aws/customization/networking/https-tls-support) for details. ::: ## [Using transparent endpoint injection](/aws/customization/networking/transparent-endpoint-injection) For example, you have a Lambda function that needs to access LocalStack resources. ## [Accessing a resource created by LocalStack](/aws/customization/networking/accessing-resources-created) For example, you have created an OpenSearch cluster and are trying to access that resource by its URL. # Accessing LocalStack via the endpoint URL > This documentation provides step-by-step guidance on how to access LocalStack services via the endpoint URL and troubleshoot common issues. import { Tabs, TabItem } from '@astrojs/starlight/components'; This documentation provides step-by-step guidance on how to access LocalStack services via the endpoint URL and troubleshoot common issues. ## From the same computer ![Code communicating with LocalStack via an endpoint](/images/aws/1.svg) Suppose you have LocalStack installed on your machine and want to access it using the AWS CLI. To connect, you must expose port 4566 from your LocalStack instance and connect to `localhost` or a domain name that points to `localhost`. While [`lstk`](/aws/developer-tools/running-localstack/lstk) does this automatically, when running the Docker container directly or with docker compose, you must configure it manually. Check out the [getting started documentation](/aws/getting-started/installation) for more information. :::tip If you bind a domain name to `localhost`, ensure that you are not subject to [DNS rebind protection](/aws/customization/networking/dns-server#dns-rebind-protection). ::: You can also use the `GATEWAY_LISTEN` [configuration variable](/aws/customization/configuration-options) to change the exposed port if necessary. ## From a container LocalStack created ![An ECS container communicating with LocalStack via an endpoint](/images/aws/4.svg) Suppose your code is running inside an ECS container that LocalStack has created. The LocalStack instance is available at the domain `localhost.localstack.cloud`. All subdomains of `localhost.localstack.cloud` also resolve to the LocalStack instance, e.g. API Gateway default URLs.
For LocalStack versions before 2.3.0 To enable access to the LocalStack instance, it's advisable to start LocalStack in a [user-defined network](https://docs.docker.com/network/bridge/), and then set the `MAIN_DOCKER_NETWORK` environment variable to this network's name. This allows the code running inside the container to access the LocalStack instance using its hostname. For example: ```bash # create the network docker network create my-network # launch localstack MAIN_DOCKER_NETWORK=my-network DOCKER_FLAGS="--network my-network" localstack start # then your code can access localstack at its container name (by default: localstack-main) aws --endpoint-url http://localstack-main:4566 s3api list-buckets ``` ```bash # create the network docker network create my-network # launch localstack docker run --rm -it --network my-network -e MAIN_DOCKER_NETWORK=my-network localstack/localstack[-pro] # then your code can access localstack at its container name (by default: localstack-main) aws --endpoint-url http://localstack-main:4566 s3api list-buckets ``` ```yaml services: localstack: # other configuration here environment: MAIN_DOCKER_NETWORK=ls networks: - ls networks: ls: name: ls # Your application code can then use # http://localstack:4566 for the # endpoint url ```
## From your container ![A docker container communicating with LocalStack via an endpoint](/images/aws/7.svg) Suppose you're accessing AWS resources such as S3 in LocalStack by running your application code in a container. Your application container should be configured to use LocalStack as its DNS server. Once this is done, the domain name `localhost.localstack.cloud` will resolve to the LocalStack container. All subdomains of `localhost.localstack.cloud` will also resolve to the LocalStack instance, e.g. API Gateway default URLs. To configure your application container: * either determine your LocalStack container IP, or configure your LocalStack container to have a fixed known IP address; * set the DNS server of your application container to the IP address of the LocalStack container. ```bash # start localstack lstk start # get the ip address of the LocalStack container docker inspect localstack-aws | \ jq -r '.[0].NetworkSettings.Networks | to_entries | .[].value.IPAddress' # prints 172.27.0.2 # run your application container docker run --rm -it --dns 172.27.0.2 ``` ```bash # start localstack docker network create ls docker run --rm -it --network ls --name localstack-main localstack/localstack[-pro] # get the ip address of the LocalStack container docker inspect localstack-main | \ jq -r '.[0].NetworkSettings.Networks | to_entries | .[].value.IPAddress' # prints 172.27.0.2 # run your application container docker run --rm -it --dns 172.27.0.2 --network ls ``` ```yaml showshowLineNumbers services: localstack: container_name: "${LOCALSTACK_DOCKER_NAME:-localstack-main}" image: localstack/localstack ports: # Now only required if you need to access LocalStack from the host - "127.0.0.1:4566:4566" # Now only required if you need to access LocalStack from the host - "127.0.0.1:4510-4559:4510-4559" environment: - DEBUG=${DEBUG:-0} volumes: - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" networks: ls: # Set the container IP address in the 10.0.2.0/24 subnet ipv4_address: 10.0.2.20 application: image: ghcr.io/localstack/localstack-docker-debug:main entrypoint: "" command: ["sleep", "infinity"] dns: # Set the DNS server to be the LocalStack container - 10.0.2.20 networks: - ls networks: ls: ipam: config: # Specify the subnet range for IP address allocation - subnet: 10.0.2.0/24 ```
For LocalStack versions before 2.3.0 To facilitate access to LocalStack from within the container, it's recommended to start LocalStack in a user-defined network and set the MAIN_DOCKER_NETWORK environment variable to the network's name. Doing so enables the containerized code to connect to the LocalStack instance using its hostname. For instance: ```bash # create the network docker network create my-network # launch localstack DOCKER_FLAGS="--network my-network" localstack start # launch your container docker run --rm it --network my-network # then your code can access localstack at its container name (by default: localstack-main) ``` ```bash # create the network docker network create my-network # launch localstack docker run --rm -it --network my-network localstack/localstack[-pro] # launch your container docker run --rm it --network my-network # then your code can access localstack at its container name (by default: localstack-main) ``` ```yaml showshowLineNumbers services: localstack: # other configuration here networks: - ls your_container: # other configuration here networks: - ls networks: ls: name: ls # Your application code can then use # http://localstack:4566 for the # endpoint url ``` ### Wildcard DNS access LocalStack newer than version 2.3.0 supports wildcard DNS access by default. Please update your LocalStack container and see the [instructions](#from-your-container).
## From a separate host ![A separate host communicating with LocalStack via an endpoint](/images/aws/10.svg) LocalStack must listen to the address of the host, or `0.0.0.0`. ```toml # .lstk/config.toml [[containers]] type = "aws" env = ["listen-all"] [env.listen-all] GATEWAY_LISTEN = "0.0.0.0" ``` ```bash lstk start ``` ```bash # this command exposes ports on all interfaces by default docker run --rm -it -p 4566:4566 localstack ``` ```yaml services: localstack: # other configuration here ports: - "4566:4566" # other ports ``` Check out our [FAQ article on accessing LocalStack from another computer](/aws/getting-started/faq#how-can-i-access-localstack-from-an-alternative-computer). # Accessing a resource created by LocalStack > This guide will explore different scenarios and provide detailed instructions on accessing resources created by LocalStack under different scenarios. If you have created a resource using LocalStack, such as an OpenSearch cluster or RDS database, you may need to access it from your application. Typically, these resources are accessible through a URL or a hostname provided by LocalStack. By default, LocalStack returns the hostname `localhost.localstack.cloud`, which resolves to LocalStack using DNS. For special environments (e.g., proxies), the [configuration](/aws/customization/configuration-options) `LOCALSTACK_HOST` customizes the URLs returned by LocalStack. This guide will explore different scenarios and provide detailed instructions on accessing resources created by LocalStack under different scenarios. ## From your host ![Accessing a resource created by LocalStack](/images/aws/3.svg) For example, suppose you have created an OpenSearch cluster using LocalStack and want to access it from the same computer. In such a case, you can set the `LOCALSTACK_HOST` environment variable to specify the desired hostname and port that will be returned. Check out the [service-specific documentation](/aws/services) for more details. ## From a container LocalStack created ![Accessing a resource created by LocalStack from a container created by LocalStack](/images/aws/6.svg) Check out our documentation while [using the endpoint URL](/aws/customization/networking/accessing-endpoint-url).
For LocalStack versions before 2.3.0 The Lambda service in LocalStack also supports the HOSTNAME_FROM_LAMBDA environment variable, which can be handy if LocalStack is reachable through a specific hostname. Suppose you're running LocalStack in a user-defined network using Docker, where the LocalStack container can be accessed from other containers in the network using its service name. In that case, you can set the HOSTNAME_FROM_LAMBDA environment variable to this value to help resolve any issues with lambda functions accessing resources created by LocalStack.
## From your container ![Accessing a resource created by LocalStack from a Docker container](/images/aws/9.svg) Check out our documentation [on using the endpoint URL](/aws/customization/networking/accessing-endpoint-url#from-your-container). ## From a separate host ![Accessing a resource created by LocalStack from a separate host](/images/aws/12.svg) LocalStack must listen to the address of the host, or `0.0.0.0`. Check out our [FAQ article on accessing LocalStack from another computer](/aws/getting-started/faq#how-can-i-access-localstack-from-an-alternative-computer). # DNS Server > Use LocalStack as DNS server to resolve AWS queries to LocalStack. LocalStack includes a DNS server that enables seamless connectivity to LocalStack from different environments using `localhost.localstack.cloud`. The DNS server is available on all IPv4 addresses within the LocalStack container (i.e., listening to `0.0.0.0`) and resolves `localhost.localstack.cloud` to the LocalStack container. Therefore, containers that are configured to use the DNS server can reach LocalStack using `localhost.localstack.cloud`. This configuration happens automatically for containers created by LocalStack, including compute resources such as Lambda, ECS, and EC2. Your container can be configured to use the DNS server as demonstrated in the [Network Troubleshooting guide](/aws/customization/networking/accessing-endpoint-url/#from-the-same-computer). If you wish to use the DNS server on your host system, follow the instructions under [System DNS configuration](#system-dns-configuration). LocalStack for AWS additionally offers [Transparent Endpoint Injection](/aws/customization/networking/transparent-endpoint-injection/) (active by default), which enables seamless connectivity to LocalStack without changing your application code targeting AWS. The DNS server resolves AWS domains such as `*.amazonaws.com` including subdomains to the LocalStack container. Therefore, your application seamlessly accesses the LocalStack APIs instead of the real AWS APIs. :::note On your host machine, `localhost.localstack.cloud` and any subdomains such as `mybucket.s3.localhost.localstack.cloud` resolve to `localhost` using a public DNS entry by LocalStack unless your router has [DNS rebind protection](#dns-rebind-protection) enabled. ::: ### Fallback DNS server If you want to use another upstream DNS resolver than your default system DNS resolver or Google DNS (`8.8.8.8` fallback if detection fails), specify the fallback DNS server where all non-redirected queries (i.e., not matching `DNS_NAME_PATTERNS_TO_RESOLVE_UPSTREAM`) will be forwarded to: ```bash DNS_SERVER=1.1.1.1 ``` By default, LocalStack attempts to detect the default system DNS resolver upon startup. If this detection fails, LocalStack uses Google DNS `8.8.8.8` as a fallback. ### Skip LocalStack DNS resolution If you want to resolve certain AWS URLs to AWS instead of LocalStack, specify a comma-separated list of skip patterns using Python-flavored regex such as: ```bash DNS_NAME_PATTERNS_TO_RESOLVE_UPSTREAM='.*(ecr|lambda).*.amazonaws.com' ``` Using this configuration, the LocalStack DNS server resolves all AWS domains to LocalStack _except_ ECR and Lambda domains which will be resolved via the `DNS_SERVER` (i.e., the real DNS entry by default). For example, `https://123456789012.dkr.ecr.us-west-2.amazonaws.com` will be forwarded to the upstream DNS resolver and reach real AWS. This can be used for hybrid setups, where certain API calls (e.g., ECR, Lambda) target AWS, whereas other services will target LocalStack. The regex pattern follows Python flavored-regex and can be tested at [regex101.com](https://regex101.com/r/OzIsQa/1). It redirects to the main page if the saved example would not work.]: # :::danger Use this configuration with caution because we generally do not recommend connecting to real AWS from within LocalStack. ::: ### DNS Server bind address If you experience problems when running LocalStack and the DNS server is the issue, you can disable the DNS server using: ```bash DNS_ADDRESS=0 ``` :::danger We do not recommend disabling the DNS server since this disables resolving `localhost.localstack.cloud` to the LocalStack container. ::: ### LocalStack endpoints If you operate behind an enterprise proxy and wish to customize the domain name returned by LocalStack services (e.g., SQS queue URL), check out the [Configuration](/aws/customization/configuration-options#core) `LOCALSTACK_HOST`. If you wish to customize internal LocalStack DNS routing of `localhost.localstack.cloud`, refer to the instructions in the [Route53 documentation](/aws/services/route53#customizing-internal-endpoint-resolution). ## DNS rebind protection If you rely on your local network's DNS, your router/DNS server might block requests due to the DNS Rebind Protection. This feature is enabled by default in pfSense, OPNSense, OpenWRT, AVM FritzBox, and potentially also other devices. Some of the vendors might allow upstream responses in the 127.0.0.0/8 range (like OpenWRT). ```bash If you are using the LocalStack DNS server, DNS rebind protection should not cause any issues. ``` You can check if your DNS setup works correctly by resolving a subdomain of `localhost.localstack.cloud`: ```bash {16} dig test.localhost.localstack.cloud ; <<>> DiG 9.16.8-Ubuntu <<>> test.localhost.localstack.cloud ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45150 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 65494 ;; QUESTION SECTION: ;test.localhost.localstack.cloud. IN A ;; ANSWER SECTION: test.localhost.localstack.cloud. 10786 IN CNAME localhost.localstack.cloud. localhost.localstack.cloud. 389 IN A 127.0.0.1 ;; Query time: 16 msec ;; SERVER: 127.0.0.53#53(127.0.0.53) ;; WHEN: Fr Jän 14 11:23:12 CET 2022 ;; MSG SIZE rcvd: 90 ``` If the DNS resolves the subdomain to your localhost (127.0.0.1), your setup is working. If not, please check the configuration of your router / DNS if the Rebind Protection is active or [enable the LocalStack DNS on your system](#system-dns-configuration). ## System DNS configuration If you wish to use the DNS server on your host system, you need to expose the LocalStack DNS server and configure your operating system. This is necessary if you want to test unmodified application code directly on your system against LocalStack and cannot configure the endpoint URL. :::danger Please be careful when changing the network configuration on your system, as this may have undesired side effects. Remember to save the default configuration and restore it after testing. ::: 1. Expose the LocalStack DNS server: a) `lstk` does not publish port `53` on the host by default. Add `expose_ports = [53]` to the container block in your `config.toml` to expose it: ```toml # .lstk/config.toml [[containers]] type = "aws" expose_ports = [53] ``` b) For Docker Compose, add the following port mappings to your `docker-compose.yml`: ```yaml ports: - "127.0.0.1:53:53" # Expose DNS server to host - "127.0.0.1:53:53/udp" # Expose DNS server to host ``` :::note If port 53 is already bound, `docker-compose up` fails with the error: ```plain Error response from daemon: Ports are not available: exposing port UDP 127.0.0.1:53 -> 0.0.0.0:0: command failed ``` To find out if a program is listening on a port, run the following command: ```bash # sudo is required if the port is < 1024 # [sudo] lsof -P -i : | grep LISTEN sudo lsof -P -i :53 | grep LISTEN ``` In macOS, a common process that listens on port 53 is `mDNSResponder`. Docker for Mac 4.24 has a [known issue](https://docs.docker.com/desktop/release-notes/#4240) and suggests the following workaround: > Deactivate network acceleration by adding `"kernelForUDP": false`, in the `settings.json` file located at `~/Library/Group Containers/group.com.docker/settings.json`. Additionally, ensure that "Internet Sharing" is disabled in the system preferences as suggested in [this GitHub issue](https://github.com/docker/for-mac/issues/7008#issuecomment-1748344545). ::: 2. Configure LocalStack to use a `DNS_SERVER` other than the host, for example using [CloudFlare DNS](https://www.cloudflare.com/learning/dns/what-is-1.1.1.1/) `DNS_SERVER=1.1.1.1`. 3. Configure your system to use the LocalStack DNS depending on your operating system: ### macOS Search for "DNS servers" in the system preferences and add a new DNS server with the IP `127.0.0.1`. Updates in the system settings are automatically reflected in `/etc/resolv.conf` and should add such an entry such as `nameserver 127.0.0.1`. ![macOS DNS server configuration](/images/aws/macos-dns-server-configuration.png) ### Linux In Linux, the configuration depends on your network manager/DNS configuration. #### systemd-resolved [//]: # (TODO: fix docs for Linux) On many modern systemd-based distributions, like Ubuntu, systemd-resolved is used for name resolution. LocalStack provides a CLI command for exactly this scenario. To use systemd-resolved and the LocalStack domain resolution, try the following steps. Start LocalStack for AWS with `DNS_ADDRESS=127.0.0.1` as environment variable. This makes LocalStack bind port 53 on 127.0.0.1, whereas systemd-resolved binds its stub resolver to 127.0.0.53:53, which prevents a conflict. Once LocalStack is started, you can test the DNS server using `dig @127.0.0.1 s3.amazonaws.com` versus `dig @127.0.0.53 s3.amazonaws.com`, the former should return an A record `127.0.0.1`, the latter the real AWS DNS result. :::caution The `dns` command is only available in the [deprecated LocalStack CLI](/aws/developer-tools/running-localstack/localstack-cli#dns-systemd-resolved). `lstk` has no equivalent command, so these steps require the legacy `localstack` CLI. ::: Run: ```bash localstack dns systemd-resolved ``` To revert, please run: ```bash localstack dns systemd-resolved --revert ``` :::note You need sudo privileges to execute this command. ::: This command sets the DNS server of the bridge interface of the docker network LocalStack currently runs in to the LocalStack container's IP address. (The command does not work with host networking or without LocalStack running for this reason.) Also, it configures the DNS route to exclusively (and only) route the following DNS names (and its subdomains) to the LocalStack DNS: ```text ~amazonaws.com ~aws.amazon.com ~cloudfront.net ~localhost.localstack.cloud ``` If you want to perform this action manually, please do the following steps: 1. Find out the bridge interface and container IP of your LocalStack container. Use `docker inspect localstack-main` to get the IP address and network, then `docker inspect network` to get the interface name. If the interface name is not mentioned, it is usually the first 12 characters of the network ID prefixed with `br-`, like `br-0ae393d3345e`. If you use the default bridge network, it is usually `docker0`. 1. Configure the DNS resolver for the bridge network: ```bash # resolvectl dns ``` 3. Set the DNS route to route only the above mentioned domain names (and subdomains) to LocalStack: ```bash # resolvectl domain ~amazonaws.com ~aws.amazon.com ~cloudfront.net ~localhost.localstack.cloud ``` In both cases, you can use `resolvectl query s3.amazonaws.com` or `resolvectl query example.com` to check which interface your DNS request is routed through, to confirm only the above mentioned domains (and its subdomains) are routed to LocalStack. When correctly configured, either using the LocalStack CLI command or manually, only the requests for the mentioned domain names are routed to LocalStack, all other queries will resolve as usual. #### Other resolution settings Depending on your Linux distribution, the settings to set a DNS server can be quite different. In some systems, directly editing `/etc/resolv.conf` is possible, like described in [macOS](#macos). If your `/etc/resolv.conf` is overwritten by some service, it might be possible to install and enable/start `resolvconf` and specify the nameserver in `/etc/resolvconf/resolv.conf.d/head` with `nameserver 127.0.0.1`. This will prepend this line in the resolv.conf file even after changes. :::note Using these options, every DNS request is forwarded to LocalStack, which will forward queries it does not need to modify (in essence all but certain AWS domains). LocalStack does not share or store any forwarded DNS requests, except for local exception logging in debug mode. ::: # External Service Port Range > The range of ports used by services not directly provided by LocalStack ## Introduction LocalStack provides local cloud services, such as [OpenSearch](/aws/services/opensearch) or [Elasticsearch](/aws/services/es), which might utilize external software bound to specific ports. This documentation discusses two approaches to access these external services within LocalStack and explores the concept of an _external service port range_. ## Proxy Functionality for External Services LocalStack offers a proxy functionality to access external services indirectly. In this approach, LocalStack assigns local domains to the external services based on the individual service's configuration. For instance, if OpenSearch is configured to use the [`OPENSEARCH_ENDPOINT_STRATEGY=domain`](/aws/services/opensearch#domain-endpoints) setting, a cluster can be reached using the domain name `...localhost.localstack.cloud`. Incoming messages to these domains are relayed to servers running on ports that do not require external accessibility. ## Direct Access with External Service Port Range An alternative approach to accessing external services is by utilizing the _external service port range_. This method, applicable to services like OpenSearch, is activated using the [`OPENSEARCH_ENDPOINT_STRATEGY=port`](/aws/services/opensearch#domain-endpoints) configuration. The external service port range is pre-defined and set to `4510-4559` by default. When a LocalStack service starts an external service, it automatically selects an available port from within the specified range. The primary advantage of this approach is that these ports are accessible from outside the Docker container, allowing direct access to the external service without the need for LocalStack to act as a proxy. ## Configuring the External Service Port Range To configure the external service port range, you can make use of the environment variables `EXTERNAL_SERVICE_PORTS_START` and `EXTERNAL_SERVICE_PORTS_END`. The range is defined as `(EXTERNAL_SERVICE_PORTS_START, EXTERNAL_SERVICE_PORTS_END]`, wherein the `EXTERNAL_SERVICE_PORTS_END` value is not included in the range. By adjusting these environment variables, you can customize the port range according to your requirements, granting you greater flexibility in managing external service access. ## Running multiple LocalStack containers with Custom Port Mapping If you wish to run multiple instances of LocalStack simultaneously, it is essential to ensure that the edge port (default: `4566`) and external service ports are mapped to non-overlapping ranges. Here's how you can achieve this using docker-compose to start your LocalStack instances: ```yaml showshowLineNumbers services: localstack-main-1: container_name: localstack-main-1 image: localstack/localstack ports: - "4566:4566" # LocalStack Gateway - "4510-4559:4510-4559" # external services port range environment: - GATEWAY_LISTEN=0.0.0.0:4566 - EXTERNAL_SERVICE_PORTS_START=4510 - EXTERNAL_SERVICE_PORTS_END=4559 - MAIN_CONTAINER_NAME=localstack-main-1 volumes: - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" localstack-main-2: container_name: localstack-main-2 image: localstack/localstack ports: - "4666:4666" # LocalStack Gateway - "4610-4659:4610-4659" # external services port range environment: - GATEWAY_LISTEN=0.0.0.0:4666 - EXTERNAL_SERVICE_PORTS_START=4610 - EXTERNAL_SERVICE_PORTS_END=4659 - MAIN_CONTAINER_NAME=localstack-main-2 volumes: - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" localstack-main-3: container_name: localstack-main-3 image: localstack/localstack ports: - "4766:4766" # LocalStack Gateway - "4710-4759:4710-4759" # external services port range environment: - GATEWAY_LISTEN=0.0.0.0:4766 - EXTERNAL_SERVICE_PORTS_START=4710 - EXTERNAL_SERVICE_PORTS_END=4759 - MAIN_CONTAINER_NAME=localstack-main-3 volumes: - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" ``` By customizing the `GATEWAY_LISTEN` and `EXTERNAL_SERVICE_PORTS_START`/`EXTERNAL_SERVICE_PORTS_END` values for each instance, you can ensure that they operate on distinct port ranges, preventing any conflicts and enabling smooth execution of multiple LocalStack instances. Each instance is given a distinct `MAIN_CONTAINER_NAME` so that it can be addressed individually. # HTTPS/TLS Support > Overview of TLS certificate coverage for the `localhost.localstack.cloud` domain and supported AWS regions for secure HTTPS access to LocalStack service endpoints. ## Introduction LocalStack provides TLS certificates for the `localhost.localstack.cloud` domain, which allows secure HTTPS access to service endpoints using region-specific hostnames such as: ```arduino https://s3.us-east-1.localhost.localstack.cloud:4566 ``` These certificates enable proper hostname validation for supported AWS regions when using HTTPS with SDKs, the AWS CLI, browsers, and other tools. If LocalStack runs behind a corporate proxy that interferes with fetching the public certificate, you can set [`SSL_NO_VERIFY=1`](/aws/customization/configuration-options#miscellaneous) to disable TLS verification when downloading the certificate from `localhost.localstack.cloud`. ### Supported Regions Due to certificate authority and infrastructure limitations, TLS certificates are currently only issued for a subset of AWS regions. If you attempt to use an unsupported region, you may encounter TLS errors such as: ```vbnet SSL: CERTIFICATE_VERIFY_FAILED hostname mismatch x509: certificate is not valid for any names ``` The full list of supported regions is available here: - `us-east-1` - `us-east-2` - `us-west-1` - `us-west-2` - `eu-central-1` - `eu-west-1` ### Why this limitation exists TLS certificates must explicitly include supported hostnames. Because each region requires hostname coverage, and certificate authorities impose size and validation constraints, it is currently not possible to include all AWS regions in the LocalStack certificate. We are actively working to expand coverage where technically feasible. # Internal Endpoints > Overview of LocalStack and AWS specific internal endpoints for local development and testing LocalStack provides several internal endpoints for various local AWS services and LocalStack-specific features. These endpoints are not part of the official AWS API and are available in the `/_localstack` and `/_aws` paths. You can use [curl](https://curl.se/) or your favourite HTTP REST client to access endpoints. You can start your LocalStack instance and go to [http://localhost.localstack.cloud:4566/\_localstack/swagger](http://localhost.localstack.cloud:4566/_localstack/swagger) to browse the Swagger UI, visualize and interact with all the API's resources implemented in LocalStack. ### LocalStack endpoints The API path for the LocalStack internal resources is `/_localstack`. Several endpoints are available under this path. For instance, `/_localstack/health` checks the available and running AWS services in LocalStack while `/_localstack/diagnose` (enable with the `DEBUG=1` configuration variable), reports extensive and sensitive data from the LocalStack instance. :::tip You can use the `/_localstack/health` endpoint to restart or kill the services. You can use [curl](https://curl.se/) or your HTTP REST client to access the endpoint: ```bash curl -v --request POST --header "Content-Type: application/json" --data '{"action":"restart"}' http://localhost.localstack.cloud:4566/_localstack/health curl -v --request POST --header "Content-Type: application/json" --data '{"action":"kill"}' http://localhost.localstack.cloud:4566/_localstack/health ``` ::: ### AWS endpoints The API path for the AWS internal resources is `/_aws`. These endpoints offer LocalStack-specific features in addition to the ones offered by the AWS services. For instance, `/aws/services/sqs/messages` conveniently access all messages within a SQS queue, without deleting them. ### `x-localstack` response header LocalStack adds an `x-localstack` HTTP header to every response served by its AWS gateway. The header value is the LocalStack version string (for example, `2026.3.1.dev65`), so client tools can detect both that they are talking to LocalStack and which version is running in a single round-trip. ```bash curl -s -i http://localhost.localstack.cloud:4566/_localstack/health | grep -i x-localstack # x-localstack: 2026.3.1.dev65 ``` :::note Before LocalStack `v2026.04`, the header value was the static string `true`. Starting with `v2026.04`, it returns the LocalStack version instead. Clients that only check for the *presence* of the header remain compatible. ::: The header is enabled by default and can be disabled by setting [`LOCALSTACK_RESPONSE_HEADER_ENABLED`](/aws/customization/configuration-options#core) to `0`. # Transparent endpoint injection > Transparently resolve your AWS calls to LocalStack ## Introduction LocalStack provides Transparent Endpoint Injection, which enables seamless connectivity to LocalStack without modifying your application code targeting AWS. The [DNS Server](/aws/customization/networking/dns-server) resolves AWS domains such as `*.amazonaws.com` including subdomains to the LocalStack container. Therefore, your application seamlessly accesses the LocalStack APIs instead of the real AWS APIs. For local testing, you might need to disable SSL validation as explained under [Self-signed certificates](#self-signed-certificates). :::note This feature is enabled when the LocalStack DNS server is used. If you wish to use Transparent Endpoint Injection, do not set `DNS_ADDRESS=0` when configuring LocalStack. ::: :::danger Transparent endpoint injection is required when using some tooling, for example AWS CDK custom resources. These resources invoke lambda functions, which execute code written by the CDK authors. They cannot be configured to make requests against LocalStack, so Transparent Endpoint Injection is used to redirect requests made against AWS to target LocalStack. ::: ## Motivation Previously, your application code targeting AWS needs to be modified to target LocalStack. For example, the AWS SDK client for Python called boto3 needs to be configured using the environment variable `AWS_ENDPOINT_URL`, which is available within Lambda functions in LocalStack: ```python client = boto3.client("lambda", endpoint_url=os.environ['AWS_ENDPOINT_URL']) ``` For [supported AWS SDKs](https://docs.aws.amazon.com/sdkref/latest/guide/feature-ss-endpoints.html#ss-endpoints-sdk-compat) (including boto3 since [1.28.0](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst#L892)), this configuration happens automatically without any custom code changes. Currently, no application code changes are required to let your application connect to local cloud APIs because Transparent Endpoint Injection uses the integrated [DNS Server](/aws/customization/networking/dns-server) to resolve AWS API calls to target LocalStack. ## Configuration This section explains the most important configuration options summarized under [Configuration](/aws/customization/configuration-options#dns). ### Disable transparent endpoint injection If you do not wish to use Transparent Endpoint Injection in LocalStack for AWS, opt out using: ```bash DISABLE_TRANSPARENT_ENDPOINT_INJECTION=1 ``` This option disables DNS resolution of AWS domains to the LocalStack container and prevents Lambda from disabling SSL validation. If Transparent Endpoint Injection is _not_ used, the AWS SDK within Lambda functions might connect to the real AWS API. Transparent Endpoint Injection is only available in LocalStack for AWS. Alternatively, specific AWS endpoints can be resolved to AWS while continuing to use Transparent Endpoint Injection. Refer to the [DNS server configuration](/aws/customization/networking/dns-server#system-dns-configuration) for skipping selected domain name patterns. :::danger Use this configuration with caution because we generally do not recommend connecting to real AWS from within LocalStack. ::: ## Self-signed certificates In LocalStack for AWS and Lambda, Transparent Endpoint Injection automatically disables SSL certificate validation of the AWS SDK for the most common Lambda runtimes including Python, Node.js, and Java (SDK v1). :::note Python Lambdas that bundle botocore 1.43.54 or newer (which added validation that rejects an empty `AWS_CA_BUNDLE`) work by default under Transparent Endpoint Injection: LocalStack no longer sets an empty `AWS_CA_BUNDLE` when `AWS_ENDPOINT_URL` injection is active, since that mode reaches LocalStack over plain HTTP and needs no certificate handling. If you instead run with `LAMBDA_DISABLE_AWS_ENDPOINT_URL=1` (DNS-based injection), LocalStack still sets an empty `AWS_CA_BUNDLE` to disable certificate verification, because the SDK connects to real AWS hostnames that resolve to LocalStack over TLS with a non-matching certificate. This is incompatible with a bundled botocore 1.43.54 or newer, which rejects the empty value with an `InvalidConfigError`. If you hit this combination, set `DISABLE_TRANSPARENT_ENDPOINT_INJECTION=1` instead. ::: For other services and unsupported Lambda runtimes, you may have to configure your AWS clients to accept self-signed certificates because we are repointing AWS domain names (e.g., `*.amazonaws.com`) to `localhost.localstack.cloud`. For example, the following command fails with an SSL error: ```bash aws kinesis list-streams SSL validation failed for https://kinesis.us-east-1.amazonaws.com/ [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1076) ``` whereas the following command works: ```bash PYTHONWARNINGS=ignore aws --no-verify-ssl kinesis list-streams { "StreamNames": [] } ``` Disabling SSL validation depends on the programming language and version of the AWS SDK used. For example, the [`boto3` AWS SDK for Python](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html#boto3.session.Session.client) provides a parameter `verify=False` to disable SSL verification. Similar parameters are available for most other [AWS SDKs](https://docs.aws.amazon.com/sdkref/latest/guide/version-support-matrix.html). For Node.js, you can set this environment variable in your application, to allow the AWS SDK to talk to the local APIs via SSL: ```javascript process.env.NODE_TLS_REJECT_UNAUTHORIZED = "0" ``` If you are using the Java AWS SDK v2 in Lambda, LocalStack will per default use bytecode instrumentation to disable certificate validation, so the endpoint injection can work. You can opt out of this behavior by setting `LAMBDA_DISABLE_JAVA_SDK_V2_CERTIFICATE_VALIDATION=0`. Opting out will lead to certificate errors when using the AWS SDK without manually overriding the endpoint url to point to LocalStack. :::danger Disabling SSL validation may have undesired side effects and security implications. Make sure to use this only for local testing, and never in production. ::: ## Current Limitations - The mechanism to disable certificate validation for these requests is not currently functional with Go Lambdas. To work around this issue, you'll need to manually set your endpoint when creating your AWS SDK client, as detailed in our documentation on the [Go AWS SDK](/aws/connecting/aws-sdks/go). - Transparent Endpoint Injection does not work when code runs inside the LocalStack container. If you need to connect to LocalStack from within the container, here are a couple of alternative approaches: - Set the AWS_ENDPOINT_URL environment variable: Set `AWS_ENDPOINT_URL=http://localhost.localstack.cloud:4566`. This is the recommended approach as it directly points your AWS client to the LocalStack endpoint. - Disable certificate validation (not recommended): If the first option isn't feasible, you can disable certificate validation by exporting an empty AWS_CA_BUNDLE variable(`export AWS_CA_BUNDLE=""`). However, note that this will cause a warning to be raised for every command. You can suppress these warnings by setting the `PYTHONWARNINGS=ignore` environment variable. This will only work for the `boto3` AWS SDK. - Transparent endpoint injection involves a combination redirecting requests using DNS and disabling certificate validation for these requests (to avoid issues when using https). Disabling certificate validation only works for processes LocalStack controls, for example Lambda (managed runtimes) and processes LocalStack starts within the LocalStack container. This means that, even in cases where DNS properly redirects the requests both inside the main LocalStack container and any spawned containers, you may still encounter certificate issues for processes not spawned directly by LocalStack. To avoid this issue, use `AWS_ENDPOINT_URL=http://localhost.localstack.cloud:4566` as an alternative. ## Troubleshooting Suppose you're attempting to access LocalStack, but you're relying on transparent endpoint injection to redirect AWS (`*.amazonaws.com`) requests. In such cases, there are different approaches you can take depending on your setup. ### From your host ![AWS SDK connecting to a Docker host](/images/aws/2.svg) If you're using LocalStack with an [Auth Token](/aws/getting-started/auth-token), then you can utilize the [DNS server](/aws/customization/networking/dns-server) to perform requests to LocalStack as if it were AWS. You need to make two changes: * Publish port 53 from the LocalStack docker container to your host. * Configure your host to use the LocalStack DNS server by default. For more details, see your [DNS server documentation](/aws/customization/networking/dns-server). Note that in both cases, SSL verification must be disabled. ### From a lambda function If you're using LocalStack with a Lambda function, run it inside the same Docker network as LocalStack. The lambda function uses the AWS SDK to communicate directly with LocalStack over the internal Docker network, without leaving the container environment. Because both the Lambda function and LocalStack are co-located within the Docker network, AWS service requests are resolved internally and routed directly to LocalStack. This does not require transparent endpoint injection or host-level DNS configuration, as networking is handled entirely within Docker. ![A Lambda function communicating with LocalStack within a Docker container](/images/aws/5.svg) # Overview > Run LocalStack in non-default container environments, from alternative images to different container runtimes. import SectionCards from '../../../../../components/SectionCards.astro'; The [Getting Started](/aws/getting-started/installation) section installs LocalStack using Docker Desktop, but many teams prefer not to use that approach. This section covers alternative installation methods, different images, and container runtimes, so you can run LocalStack in the environment that best fits your needs. For running LocalStack on Kubernetes, see the dedicated [Kubernetes](/aws/customization/kubernetes) section. # DevContainers > Add LocalStack to a reproducible, containerized DevContainer development environment using LocalStack templates. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Overview [DevContainers](https://containers.dev/) is a local tool to create a self-contained, reproducible and containerized development environment that you can setup to encapsulate your project with all its libraries and dependencies. In this guide, you will learn how to use [DevContainers](https://containers.dev/) with LocalStack. You can use the following two approaches to set up LocalStack with DevContainers: * [LocalStack templates](#localstack-templates) * [LocalStack feature](#localstack-feature) ## LocalStack Templates :::note The LocalStack DevContainer templates and Feature install the legacy `localstack` CLI, not the new CLI experience, `lstk`. The commands and configuration keys on this page therefore refer to the [legacy LocalStack CLI](/aws/developer-tools/running-localstack/localstack-cli/), which is what is available inside the DevContainer. ::: LocalStack provides two different approaches for [Templates](https://github.com/localstack/devcontainer-template) which can be used via [supporting tools](https://containers.dev/supporting). | **Type** | **Advantages** | **Disadvantages** | |------------------------------|----------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | **Docker-in-Docker** | • Strict separation from host Docker service
• Control LocalStack with LocalStack CLI
• All-in-one container | • Resources are limited as all resources spawned by LocalStack are encapsulated within the container
• LocalStack volume directory must exist beforehand
• Larger container size
• Cannot use existing images on host system | | **Docker-outside-of-Docker** | • Easy addition of external services managed by Docker Compose
• DNS service for custom domains | • Host's Docker socket mounted into containers, raising security concerns
• Limited LocalStack CLI usage
• LocalStack volume directory must exist beforehand | ### Docker-in-Docker * [Dev Container CLI](#dev-container-cli) * [VS Code](#vscode) * [Reference file](#reference-file) #### Dev Container CLI You can use the DevContainer CLI to create a `devcontainer.json` file from the LocalStack template. Before you start, ensure that you have the [DevContainer CLI](https://code.visualstudio.com/docs/devcontainers/devcontainer-cli) installed. Create a JSON file called `options.json` with the desired options in it. ```json showshowLineNumbers { "imageVariant": "bullseye", "awslocal": "true", "logLevel": "debug", "debug": "true", "startup": "true" } ``` Use the command below to generate your `devcontainer.json` from the template. Include additional features with the `--features` option if needed. ```bash devcontainer templates apply \ --template-id ghcr.io/localstack/devcontainer-template/localstack-dind \ --template-args "$(cat ./options.json)" \ --features '[{"id":"ghcr.io/devcontainers/features/aws-cli:1"}]' ``` Start your container using the following command. ```bash devcontainer up --id-label project=localstack --workspace-folder . ``` Connect to it using the `id-label`. ```bash devcontainer exec --id-label project=localstack /bin/bash ``` Check that the LocalStack CLI is installed by executing: ```bash localstack --version ``` To remove the container, run this cleanup script since the Dev Container CLI cannot currently do it. ```bash for container in $(docker ps -q); do \ [[ "$(docker inspect --format '{{ index .Config.Labels "project"}}' $container)" = "localstack" ]] && \ docker rm -f $container; \ done ``` #### VSCode :::note The DevContainer extension is currently reporting issues & bugs. Follow the [issue](https://github.com/microsoft/vscode-remote-release/issues/10180) for details. ::: To get started with LocalStack and DevContainers in VS Code, follow these steps: * Open VS Code with the DevContainers extension installed. * From the Command Palette, select **Dev Containers: Add Dev Container configuration file**. ![Add Dev Container configuration file](public/images/aws/01_add_devcontainer_conf.png) * Choose **Add configuration to workspace**; alternatively, select **Add configuration to user data folder** for general usage. ![Add configuration to workspace](public/images/aws/02_add_conf_workspace.png) * Select **Show All Definitions...** to view community templates. ![Show all Template definitions](public/images/aws/03_show_all_definitions.png) * Filter by typing "localstack" in the search bar and select the **LocalStack Docker-in-Docker** template. [Select official LocalStack Template (DinD)](/aws/customization/other-installations/devcontainers/#localstack-templates) * Proceed through the configuration by selecting or entering values. Pressing **Enter** through the options will apply default settings, which include: * Select the image variant (only Debian-based images are supported). ![Image variant option](public/images/aws/05_option_1.png) * Select the log level. ![Log level option](public/images/aws/06_option_2.png) * Select the LocalStack version. ![LocalStack version option](public/images/aws/07_option_3.png) * Relative paths are acceptable for the volume path, but the specified mount folder must be created prior to building the container. ![Volume path option](public/images/aws/08_volume_option.png) ![Volume folder exists](public/images/aws/09_volume_folder.png) * Select various tools and configuration options from the checklist. For local tools, either select the appropriate SDK or tool feature, or install it manually. The template and LocalStack CLI feature do not manage these installations. ![List of options (DinD)](public/images/aws/10a_options_list_dind.png) * You can also add additional features. ![Additional Features](public/images/aws/11_additional_features.png) * This results in the following folder structure in your workspace. ![Generated folder structure (DinD)](public/images/aws/12a_folder_structure_dind.png) #### Reference file The `devcontainer.json` will look similar to the following: ```json showshowLineNumbers { "name": "LocalStack DinD setup", "image": "mcr.microsoft.com/devcontainers/base:bullseye", "remoteEnv": { // Activate LocalStack for AWS: https://docs.localstack.cloud/getting-started/auth-token/ "LOCALSTACK_AUTH_TOKEN": "${localEnv:LOCALSTACK_AUTH_TOKEN}", // required for Pro, not processed via template due to security reasons "LOCALSTACK_API_KEY": "${localEnv:LOCALSTACK_API_KEY}", // LocalStack configuration: https://docs.localstack.cloud/references/configuration/ "DEBUG": true, "LS_LOG": "debug", "PERSISTENCE": false, "AWS_ENDPOINT_URL": "http://localhost.localstack.cloud:4566", "AUTO_LOAD_POD": " ", "ENFORCE_IAM": false, "AWS_REGION": "us-east-1", "AWS_DEFAULT_REGION": "us-east-1", "IMAGE_NAME": "localstack/localstack-pro:latest", "LOCALSTACK_VOLUME_DIR": "/data" }, // 👇 Features to add to the Dev Container. // More info: https://containers.dev/implementors/features. "features": { "ghcr.io/devcontainers/features/docker-in-docker:2": {}, "ghcr.io/localstack/devcontainer-feature/localstack-cli:latest": { "version": "latest", "awslocal": true, // if true, add in features manually: ghcr.io/devcontainers/features/aws-cli "cdklocal": false, // if true, add in features manually: ghcr.io/devcontainers-contrib/features/aws-cdk "pulumilocal": false, // if true, add in features manually: ghcr.io/devcontainers-contrib/features/pulumi "samlocal": false, // if true, add in features manually: ghcr.io/customink/codespaces-features/sam-cli "tflocal": false // if true, add in features manually: ghcr.io/devcontainers-contrib/features/terraform-asdf }, "ghcr.io/devcontainers/features/aws-cli:1": {} }, // 👇 Use 'postCreateCommand' to run commands after the container is created. "postCreateCommand": "type localstack; true && localstack start -d || true", "mounts": [ { // to persist build data and images "source": "dind-var-lib-docker", "target": "/var/lib/docker", "type": "volume" }, { "source": "./.volume", "target": "/data", "type": "bind", "consistency": "cached" } ] } ``` ### Docker-outside-of-Docker * [Dev Container CLI](#dev-container-cli) * [VS Code](#vscode) * [Reference files](#reference-files) #### Dev Container CLI You can use the DevContainer CLI to create a `devcontainer.json` file from the LocalStack template. Before you start, ensure that you have the [DevContainer CLI](https://code.visualstudio.com/docs/devcontainers/devcontainer-cli) installed. Create a JSON file called `options.json` with the desired options in it. ```json showshowLineNumbers { "imageVariant": "bookworm", "awslocal": "true", "logLevel": "debug", "debug": "true", "networkName": "localstack-network", "networkCidr": "192.168.9.0/24", "ipAddress": "192.168.9.13" } ``` Use the command below to generate your `devcontainer.json` from the template. Include additional features with the `--features` option if needed. ```bash devcontainer templates apply \ --template-id ghcr.io/localstack/devcontainer-template/localstack-dood \ --template-args "$(cat ./options.json)" \ --features '[{"id":"ghcr.io/devcontainers/features/aws-cli:1"}]' ``` Start your container using the following command. ```bash devcontainer up --id-label project=localstack --workspace-folder . ``` Connect to it using the `id-label`. ```bash devcontainer exec --id-label project=localstack /bin/bash ``` Check that the LocalStack CLI is installed by executing: ```bash localstack --version ``` To remove the container, run this cleanup script since the Dev Container CLI cannot currently do it. ```bash docker compose \ --project-name "$(basename $PWD)_devcontainer" \ -f ./.devcontainer/docker-compose.yml down ``` #### VSCode :::note The DevContainer extension is currently reporting issues & bugs. Follow the [issue](https://github.com/microsoft/vscode-remote-release/issues/10180) for details. ::: To get started with LocalStack and DevContainers in VS Code, follow these steps: * Open VSCode with the DevContainers extension installed. * From the Command Palette, choose **Dev Containers: Add Dev Container configuration file**. ![Add Dev Container configuration file](public/images/aws/01_add_devcontainer_conf.png) * Choose the **Add configuration to workspace** option; alternatively, select **Add configuration to user data folder** for general usage. ![Add configuration to workspace](public/images/aws/02_add_conf_workspace.png) * Select **Show All Definitions...** to view community templates. ![Show all Template definitions](public/images/aws/03_show_all_definitions.png) * Start typing "localstack" in the search bar to filter the official LocalStack templates and choose **LocalStack Docker-outside-of-Docker**. ![Select official LocalStack Template (DooD)](public/images/aws/04b_select_template_dood.png) * Navigate through the configuration inputs by either selecting or typing in values. The defaults provided in the template are sufficient; navigating through the options by hitting Enter will result in a valid configuration. These options include: * The image variant (currently only Debian-based images are supported). ![Image variant option](public/images/aws/05_option_1.png) * The log level. ![Log level option](public/images/aws/06_option_2.png) * The LocalStack version. ![LocalStack version option](public/images/aws/07_option_3.png) * Note that LocalStack's IP address must be within the defined CIDR range. The network CIDR defaults to `10.0.2.0/24`, with the container IP set to `10.0.2.20`. * For the volume path, relative paths are accepted, but you must create the specified mount's folder before successfully building the container. The default is `./.volume`. ![Volume path option](public/images/aws/08_volume_option.png) ![Volume folder exists](public/images/aws/09_volume_folder.png) * Select multiple tools and configuration options from the checklist. For local tools, you must select the appropriate SDK or tool feature, or install it manually. The template and the underlying LocalStack CLI Feature do not manage these installations. ![List of options (DooD)](public/images/aws/10b_options_list_dood.png) * You can also add additional features. ![Additional Features](public/images/aws/11_additional_features.png) * As a result, you will end up with the folder structure shown below. ![Folder structure (DooD)](public/images/aws/12b_folder_structure_dood.png) ###### Reference files ```json showshowLineNumbers { "name": "LocalStack DooD setup", "dockerComposeFile": "docker-compose.yml", "service": "app", "workspaceFolder": "/workspaces/${localWorkspaceFolderBasename}", // 👇 Features to add to the Dev Container. // More info: https://containers.dev/implementors/features. "features": { "ghcr.io/devcontainers/features/docker-outside-of-docker:1": {}, "ghcr.io/localstack/devcontainer-feature/localstack-cli:latest": { "version": "latest", "awslocal": true, // if true, add in features manually: ghcr.io/devcontainers/features/aws-cli "cdklocal": false, // if true, add in features manually: ghcr.io/devcontainers-contrib/features/aws-cdk "pulumilocal": false, // if true, add in features manually: ghcr.io/devcontainers-contrib/features/pulumi "samlocal": false, // if true, add in features manually: ghcr.io/customink/codespaces-features/sam-cli "tflocal": false // if true, add in features manually: ghcr.io/devcontainers-contrib/features/terraform-asdf }, "ghcr.io/devcontainers/features/aws-cli:1": {} } } ``` ```yml showshowLineNumbers services: localstack: container_name: "localstack-main" image: localstack/localstack-pro:latest # required for Pro ports: - "127.0.0.1:4566:4566" # LocalStack Gateway - "127.0.0.1:4510-4559:4510-4559" # external services port range - "127.0.0.1:443:443" # LocalStack HTTPS Gateway (Pro) env_file: - .env volumes: - "/var/run/docker.sock:/var/run/docker.sock" - "./.volume:/var/lib/localstack" networks: ls: # Set the container IP address in the 10.0.2.0/24 subnet ipv4_address: 10.0.2.20 app: build: context: . dockerfile: Dockerfile volumes: - ../..:/workspaces:cached # Overrides default command so things don't shut down after the process ends. command: sleep infinity init: true env_file: - .env dns: # Set the DNS server to be the LocalStack container - 10.0.2.20 networks: - ls networks: ls: ipam: config: # Specify the subnet range for IP address allocation - subnet: 10.0.2.0/24 ``` ```dockerfile FROM mcr.microsoft.com/devcontainers/base:bookworm ``` ```bash showshowLineNumbers # Activate LocalStack for AWS: https://docs.localstack.cloud/getting-started/auth-token/ LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:-} # required for Pro, not processed via template due to security reasons LOCALSTACK_API_KEY=${LOCALSTACK_API_KEY:-} # LocalStack configuration: https://docs.localstack.cloud/references/configuration/ DEBUG=true LS_LOG=debug PERSISTENCE=false AWS_ENDPOINT_URL=http://localhost.localstack.cloud:4566 LOCALSTACK_HOST=localhost.localstack.cloud:4566 AUTO_LOAD_POD= ENFORCE_IAM=false AWS_REGION=us-east-1 AWS_DEFAULT_REGION=us-east-1 IMAGE_NAME=localstack/localstack-pro:latest ``` ## LocalStack Feature Add the following minimal [Feature](https://github.com/localstack/devcontainer-feature) snippet to your DevContainer config. ```json ... "features": { "ghcr.io/localstack/devcontainer-feature/localstack-cli:latest": {} } ... ``` That's it. By building your container the LocalStack CLI and any of the enabled local-tools (currently these are `awslocal`, `cdklocal`, `pulumilocal`, `samlocal` and `tflocal`) will be installed. :::note The LocalStack Feature does not manage the installation of underlying tools (e.g., for awslocal, aws-cli is not installed). For more information on dependencies, please refer to the [Feature documentation](https://github.com/localstack/devcontainer-feature). ::: # Docker Images > Overview of LocalStack Docker images and their purpose, their tags, and when to use each. LocalStack functions as a local “mini-cloud” operating system that runs inside a Docker container. LocalStack has multiple components, which include process management, file system abstraction, event processing, schedulers, and more. Running inside a Docker container, LocalStack exposes external network ports for integrations, SDKs, or CLI interfaces to connect to LocalStack APIs. The LocalStack & LocalStack for AWS Docker images have been downloaded over 130+ million times and provide a multi-arch build compatible with AMD/x86 and ARM-based CPU architectures. This section will cover the different Docker images available for LocalStack and how to use them. ## LocalStack for AWS image LocalStack for AWS contains various advanced extensions to the LocalStack base platform. With LocalStack for AWS image, you can access all the emulated AWS cloud services running entirely on your local machine. To use the LocalStack for AWS image, you can pull the image from Docker Hub: ```bash docker pull localstack/localstack:latest ``` To use the LocalStack for AWS image, you must configure an environment variable named `LOCALSTACK_AUTH_TOKEN` to contain your Auth Token. The LocalStack for AWS image will display a warning if you do not set an Auth Token (or if the license is invalid/expired) and will not activate the Pro features. LocalStack for AWS gives you access to the complete set of LocalStack features, including the [LocalStack Web Application](https://app.localstack.cloud) and [dedicated customer support](/aws/help-support/get-help/). You can use the LocalStack for AWS image to start your LocalStack container using various [installation methods](/aws/getting-started/installation/). While configuring to run LocalStack with Docker or Docker Compose, run the `localstack/localstack-pro` image with the appropriate tag you have pulled (if not `latest`). ## Image tags Starting with the end-of-March 2026 release, LocalStack version tags follow [calendar versioning](https://calver.org/) in the `YYYY.MM.patch` format (for example, `2026.03.0`). Releases up to and including `v4.14.0` use [semantic versioning](https://semver.org). The following tags are available for the LocalStack Docker image: | Tag | Updated when | Recommended for | |---|---|---| | `latest` / `stable` | Tagged releases only (e.g. `2026.05.0`) | Most users — stable, release-quality builds | | `dev` | Every merged commit on `main` | Users who need the latest unreleased changes | | `YYYY.MM` (e.g. `2026.05`) | Each patch release within that month | Users who want bugfixes but want to avoid feature changes | | `YYYY.MM.patch` (e.g. `2026.05.0`) | Never (pinned) | Fully reproducible environments where no changes are acceptable | :::note As of May 2026, `latest` was changed to mirror `stable` and is only updated on official tagged releases. If you previously relied on `latest` for the most recent unreleased changes, switch to the `dev` tag. The `nightly` tag is no longer published for LocalStack for AWS. Use the `dev` tag to track untagged changes from `main`. ::: Visit the [LocalStack for AWS tag](https://hub.docker.com/r/localstack/localstack-pro/tags?page=1&ordering=last_updated) page on Docker Hub. # Enterprise Image > Custom LocalStack Enterprise image for offline or air-gapped environments with preferred configurations and packages. ## Introduction LocalStack offers an Enterprise image that allows offline usage and includes a customer-specific configuration. This offline functionality is enabled by: - Pre-installed packages required for running specific services that are usually downloaded on demand (such as `opensearch` or `dynamodb-local`). - A certificate keypair for `localhost.localstack.cloud` to resolve to the LocalStack container via our DNS server. - An embedded decryption key in the image, eliminating the need to contact the license server to operate LocalStack. ## Why use Enterprise Image? - **Airgapped environments**: The Enterprise image is ideal for customers who operate in airgapped environments where internet access is restricted. - **Security Fixes**: The Enterprise image is updated with the latest security fixes and patches including container image scans on a priority basis. - **Custom Configuration**: The Enterprise image can be customized to include specific packages and configurations required by the customer. - **CI Usage**: The Enterprise image can be used in CI/CD pipelines to ensure that the same image is used across all environments. ## How to use the image? - After the image is pushed to the customer-specific ECR repository, the customer can pull and push it to their internal Docker registry. - Developers within the customer’s network can then pull the image from this registry. - To use the image from the command line interface (CLI), set the `image` field on the container block to the name of the Enterprise image: ```toml # .lstk/config.toml [[containers]] type = "aws" image = "localstack-enterprise" ``` ```bash lstk start ``` See [Custom container image](/aws/developer-tools/running-localstack/lstk/configuration/#custom-container-image) for details, including how `lstk` falls back to a locally present image when a pull fails. ## "Online" vs "Offline" image This section compares the standard [LocalStack for AWS Docker image](/aws/customization/other-installations/docker-images) ("online") with the customer-specific Enterprise image ("offline"). ### Key differences | Area | Standard image | Enterprise image | |---|---|---| | Internet requirement for core startup | Requires network access for normal [license activation](/aws/getting-started/auth-token). | Designed to run without internet access in air-gapped environments. | | License behavior | Activates via LocalStack licensing endpoints. If unreachable, LocalStack attempts offline activation and requires re-activation every 24 hours. | Includes an embedded keypair/decryption key so LocalStack can run without contacting the license server. | | Service dependencies | Some services may download dependencies on demand during runtime. | Service dependencies are pre-baked into the image for offline usage. | | Cloud Pods | Platform remote integration can sync state with your LocalStack account. | LocalStack Platform remotes are typically unavailable in fully air-gapped setups. Use self-managed remotes (for example S3 or ORAS) when available in your environment. | | Ephemeral instances | Available via Web App/CLI as cloud-hosted LocalStack runtimes. | Not available in air-gapped/offline deployments because they run on LocalStack Cloud infrastructure. | | Telemetry | Can send usage events for features such as [Stack Insights](/aws/organizations-admin/stack-insights). | Keep event reporting disabled (`DISABLE_EVENTS=1`) for strict offline setups. | ### What communicates with LocalStack Cloud? The main integrations are: - **License activation**: The standard image performs online activation using your `LOCALSTACK_AUTH_TOKEN`. See [Auth Token](/aws/getting-started/auth-token) for activation behavior and fallbacks. - **Event reporting (telemetry)**: Used for Stack Insights and related usage analytics. You can disable this via `DISABLE_EVENTS=1`. - **Cloud Pods (platform remote)**: Saving/loading pods against the default platform remote uses LocalStack-managed infrastructure. See [Cloud Pods](/aws/developer-tools/snapshots/cloud-pods/) for where that data is held. For stricter data residency, consider other [remote storage options](/aws/developer-tools/snapshots/saving-snapshots-to-s3). - **Ephemeral instances**: These are managed cloud instances and therefore require connectivity to LocalStack Cloud services. ### Recommended setup for offline environments - Use the **offline Enterprise image** when no outbound connectivity is permitted. - Keep `DISABLE_EVENTS=1` to prevent event reporting. - Prefer local persistence or self-managed Cloud Pod remotes instead of platform remotes. - Do not rely on Ephemeral Instances in fully isolated networks; run LocalStack directly in your controlled environment instead. # LocalStack Docker Extension > Manage your LocalStack container directly from Docker Desktop with the LocalStack Extension. ## Introduction The LocalStack Extension for Docker Desktop enables developers working with LocalStack to operate their LocalStack container via Docker Desktop, including checking service status, container logs, and configuring profiles. To install the LocalStack Extension for Docker Desktop, you need to have [Docker Desktop installed on your machine](https://www.docker.com/products/docker-desktop). ![LocalStack Extension for Docker Desktop](/images/aws/localstack-docker-extension.png) ## Installation To utilize LocalStack's Docker Extension, it is necessary to have a recent version of Docker Desktop (v4.8 or higher) installed on the local machine. To enable the extension, access the **Extensions** tab and select the **Enable Docker Extensions** and **Show Docker Extensions system containers** option. ![Enable Docker Extensions in the Preferences within the Extensions tab](/images/aws/localstack-docker-extension-preferences.png) The LocalStack Extension for Docker Desktop has been validated and can be accessed on the Extensions Marketplace. To begin using it, navigate to the **Extensions Marketplace**, search for **LocalStack**, and click the **Install** button to proceed with the installation. ![Discover the LocalStack Extension on the Docker Desktop Marketplace and install it!](/images/aws/localstack-docker-extension-marketplace.png) An alternative method for installing the LocalStack's Extension for Docker Desktop is pulling the [public Docker image](https://hub.docker.com/r/localstack/localstack-docker-desktop) from Docker Hub and installing it! ```bash docker extension install localstack/localstack-docker-desktop:latest ``` After installation, you can access the LocalStack Extension for Docker Desktop from the **Extensions** tab. Upon the initial launch of the extension, a prompt to select a mount point for the LocalStack container will appear. Select your username from the drop-down menu. Furthermore, you can modify this setting later by navigating to the **Configurations** tab and choosing a different mount point. Select the mount point upon the launch of LocalStack's Docker extension. ![Select the mount point upon the launch of LocalStack's Docker extension](/images/aws/localstack-docker-extension-mount-point.png) ## Features LocalStack's Docker Extension helps users to manage their LocalStack container with a simple and intuitive user interface through Docker Desktop. The extension includes container management, configuration profile management, service status, and container logs! ### Container management You can start, stop, and restart LocalStack from the Docker Desktop. You can also see the current status of your LocalStack container and navigate to LocalStack Web Application. ![Start and Stop your LocalStack container with a single click of a button with LocalStack's extension](/images/aws/localstack-docker-extension-start.png) ### Container logs You can see the log information of the LocalStack container and all the available services and their status on the service page. ![Check the logs of your running LocalStack container through LocalStack's Docker extension](/images/aws/localstack-docker-extension-logs.png) ### Configuration management You can manage and use your profiles via configurations and create new configurations for your LocalStack container. ![Create your configuration profiles within LocalStack's Extension to affect the state of LocalStack](/images/aws/localstack-docker-extension-configuration-profile.png) ## Configure an Auth Token To configure an Auth Token for the LocalStack Docker Extension, you need to create a new configuration profile. Navigate to the **Configurations** tab and click the **New +** button. Enter the configuration name and add the `LOCALSTACK_AUTH_TOKEN` environment variable with the desired value. To start the LocalStack for AWS container with the Auth Token, select the configuration profile from the drop-down menu and click the **Start** button. # Podman > Run LocalStack inside Podman, a Docker-compatible container engine. ## Introduction By default, the LocalStack CLI starts the LocalStack runtime inside a Docker container. Docker may not be available on your system, and a popular alternative is [Podman](https://podman.io/get-started) which you can use to run LocalStack. Podman support is still experimental, and the following docs give you an overview of the current state. :::note The new CLI experience, `lstk`, does not yet fully support Podman. The commands and environment variables on this page refer to the legacy [LocalStack CLI](/aws/developer-tools/running-localstack/localstack-cli/), which you should continue to use for Podman setups. ::: From the Podman docs: > Podman is a daemonless, open source, Linux native tool designed to make it easy to find, run, build, share and deploy applications using Open Containers Initiative (OCI) Containers and Container Images. > Podman provides a command line interface (CLI) familiar to anyone who has used the Docker Container Engine. > Most users can simply alias Docker to Podman (`alias docker=podman`) without any problems. ## Options To run `localstack`, simply aliasing `alias docker=podman` is not enough, for the following reasons: - `localstack` is using [docker-py](https://pypi.org/project/docker/) which requires a connection to `/var/run/docker.sock` - Lambda requires mounting the Docker socket `/var/run/docker.sock` into the container (see [Lambda providers](/aws/services/lambda)). Here are several options on running LocalStack using podman: ### podman-docker The package `podman-docker` emulates the Docker CLI using podman. It creates the following links: - `/usr/bin/docker -> /usr/bin/podman` - `/var/run/docker.sock -> /run/podman/podman.sock` This package is available for some distros: - https://archlinux.org/packages/extra/x86_64/podman-docker/ - https://packages.ubuntu.com/oracular/podman-docker - https://packages.debian.org/sid/podman-docker ### Rootfull Podman with podman-docker The simplest option is to run `localstack` using `podman` by having `podman-docker` and running `localstack start` as root ```bash # you have to start the podman socket first sudo systemctl start podman # then sudo sh -c 'DEBUG=1 localstack start --network podman' ``` ### Rootfull Podman without podman-docker ```sh # you still have to start the podman socket first sudo systemctl start podman # you have to pass a bunch of env variables sudo sh -c 'DEBUG=1 DOCKER_CMD=podman DOCKER_HOST=unix://run/podman/podman.sock DOCKER_SOCK=/run/podman/podman.sock localstack start --network podman' ``` ### Rootless Podman You have to prepare your environment first: - https://wiki.archlinux.org/title/Podman#Rootless_Podman - https://github.com/containers/podman/blob/main/docs/tutorials/rootless_tutorial.md - https://www.redhat.com/sysadmin/rootless-podman ```bash # again, you have to start the podman socket first systemctl --user start podman.service # and then localstack DEBUG=1 DOCKER_CMD="podman" DOCKER_SOCK=$XDG_RUNTIME_DIR/podman/podman.sock DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock localstack start --network podman ``` If you have problems with [subuid and subgid](https://wiki.archlinux.org/title/Podman#Set_subuid_and_subgid), you could try to use [overlay.ignore_chown_errors option](https://www.redhat.com/sysadmin/controlling-access-rootless-podman-users) ```bash DEBUG=1 DOCKER_CMD="podman --storage-opt overlay.ignore_chown_errors=true" DOCKER_SOCK=$XDG_RUNTIME_DIR/podman/podman.sock DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock localstack start --network podman ``` ### Podman on Windows You can run Podman on Windows using [WSLv2](https://learn.microsoft.com/en-us/windows/wsl/about#what-is-wsl-2). In the guide, we use a Docker Compose setup to run LocalStack. Initialize and start Podman: ```bash podman machine init podman machine start ``` At this stage, Podman operates in rootless mode, where exposing port 443 on Windows is not possible. To enable this, switch Podman to rootful mode using the following command: ```bash podman machine set --rootful ``` For the Docker Compose setup, use the following configuration. When running in rootless mode, ensure to comment out the HTTPS gateway port, as it is unable to bind to privileged ports below 1024. ```yaml showshowLineNumbers services: localstack: container_name: "${LOCALSTACK_DOCKER_NAME:-localstack-main}" image: localstack/localstack-pro ports: - "127.0.0.1:4566:4566" - "127.0.0.1:4510-4559:4510-4559" - "0.0.0.0:443:443" networks: - podman security_opt: - "label=disable" environment: - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} - DEBUG=${DEBUG:-0} - PERSISTENCE=${PERSISTENCE:-0} volumes: - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" ``` The docker socket `/var/run/docker.sock` is correctly linked by default in a Podman setup. To start the services, use `docker compose up` or `podman compose up`, depending on the availability of docker-compose. # Rancher Desktop > Run LocalStack on the local container runtime provided by Rancher Desktop, a Docker Desktop alternative. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction Rancher Desktop is a desktop application that provides a Kubernetes cluster on your local machine. Rancher Desktop allows you to run Docker containers and Kubernetes clusters without relying on remote or cloud-based systems. It utilizes `containerd` and `dockerd`, enabling users to easily switch between container runtimes. By default, the LocalStack CLI launches the LocalStack runtime inside a Docker container. However, if Docker is not available on your system, you can use Rancher Desktop as a popular alternative to run LocalStack. :::note The new CLI experience, `lstk`, does not yet fully support Rancher Desktop. The commands and environment variables on this page refer to the legacy [LocalStack CLI](/aws/developer-tools/running-localstack/localstack-cli/), which you should continue to use for Rancher Desktop setups. ::: ## Getting started To run LocalStack using Rancher Desktop, simply aliasing Docker commands to Rancher Desktop's `dockerd` service may not be sufficient for these reasons: 1. LocalStack depends on `docker-py`, which needs to connect to `/var/run/docker.sock`. 2. Lambda services in LocalStack require the Docker socket at `/var/run/docker.sock` to be mounted into the container. Depending on your operating system, you may need to make additional configurations to ensure LocalStack runs smoothly with Rancher Desktop. - 1. Make sure there is no existing socket at /var/run/docker.sock - 2. Adjust the path if your Rancher Desktop socket is in a different location - [Rancher Desktop with containerd](#rancher-desktop-with-containerd) - [Windows](#windows) These setups enable LocalStack to run smoothly with Rancher Desktop across various operating systems, ensuring compatibility with Docker-based workflows. ### Linux/macOS :::note ### Recommended Settings for Rancher Desktop on macOS If you're using Rancher Desktop on macOS, particularly on Apple Silicon (M1, M2, etc.), it's crucial to adjust both the emulation engine and the volume-sharing method. It is recommended to switch to the VZ virtualization engine and use VirtioFS for optimal performance and to avoid permission issues. Without these adjustments, you may encounter permission issues with volume mounts in LocalStack. #### Switching Emulation from QEMU to VZ (Apple Virtualization Framework) For macOS users, Rancher Desktop allows switching from QEMU to the Apple Virtualization Framework (VZ) for virtualization. Using VZ can enhance performance and resolve permission issues with volume mounts when running LocalStack. Here’s how to switch from QEMU to VZ: 1. Open Rancher Desktop and navigate to **Settings**. 2. Go to the **Virtual Machine** section. 3. Find the **Virtualization Engine** option, which is set to QEMU by default. 4. Change the setting from `QEMU` to `VZ (Apple Virtualization)`. 5. Restart Rancher Desktop to apply the changes. #### Changing Volume from Reverse-SSHFS to VirtioFS By default, Rancher Desktop uses `reverse-sshfs` for mounting volumes inside the virtual machine. However, you can switch to `VirtioFS` to improve performance and address permission issues. VirtioFS is a faster and more reliable volume sharing method on macOS. To switch the volume sharing method from reverse-SSHFS to VirtioFS: 1. Open Rancher Desktop and access the **Settings**. 2. Proceed to the **Virtual Machine** section, where you'll find the volume mount options. 3. Select the **File Sharing** setting and change it from `reverse-sshfs` to `VirtioFS`. 4. Restart Rancher Desktop to implement the changes. ::: #### Rancher Desktop with dockerd The simplest way to run LocalStack using Rancher Desktop involves making sure that Rancher Desktop's `dockerd` is active and properly configured. Rancher Desktop typically places its Docker socket file somewhere under your user directory, such as `~/.rancher-desktop`. LocalStack, however, expects the Docker socket to be at `/var/run/docker.sock`. In this scenario, you need to create a symlink from the Rancher Desktop socket to the expected location. Start Rancher Desktop and verify it is set to use the Docker runtime. Link the Docker socket with the following command: ```bash 1. Make sure there is no existing socket at /var/run/docker.sock sudo rm -f /var/run/docker.sock 2. Adjust the path if your Rancher Desktop socket is in a different location sudo ln -s /var/run/rancher-desktop-lima/docker.sock /var/run/docker.sock ``` Start LocalStack using this command: ```bash DEBUG=1 localstack start --network rancher ``` #### Rancher Desktop with containerd If you are using the `containerd` runtime in Rancher Desktop, you'll need to make some additional configurations. Ensure that the `docker` command is available through Rancher Desktop's setup, or alternatively, use the [`nerdctl` command line interface](https://github.com/containerd/nerdctl). To start LocalStack with the `containerd` environment, use the following command: ```bash DEBUG=1 DOCKER_CMD=nerdctl localstack start --network rancher ``` ### Windows You can run Rancher Desktop on Windows using WSL2 (Windows Subsystem for Linux) with a Docker Compose setup for LocalStack. Ensure Rancher Desktop is configured to use `dockerd`, and that the Docker socket is accessible in WSL2: ```bash rancher-desktop settings set --docker ``` Initialize and start Rancher Desktop: ```bash rancher-desktop --start ``` Modify your Docker Compose configuration to work with Rancher Desktop: ```yml showshowLineNumbers services: localstack: container_name: "${LOCALSTACK_DOCKER_NAME:-localstack-main}" image: localstack/localstack-pro ports: - "127.0.0.1:4566:4566" - "127.0.0.1:4510-4559:4510-4559" - "0.0.0.0:443:443" networks: - rancher environment: - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} - DEBUG=${DEBUG:-0} - PERSISTENCE=${PERSISTENCE:-0} volumes: - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" ``` Finally, start the services using `docker compose up` or `nerdctl compose up`, depending on your configuration. This will launch your LocalStack instance configured to interact with Rancher Desktop. # Overview > LocalStack's developer tools build on the emulator to give developers additional ways to increase their velocity in day-to-day cloud development. import SectionCards from '../../../../components/SectionCards.astro'; LocalStack's developer tools build upon the emulator by providing additional ways for developers to increase their velocity. They go beyond running cloud services locally, helping you start and manage LocalStack, inspect what your application is doing, manage state, test resilience and security, and replicate resources from real AWS environments. In this documentation, developers will be shown how they can incorporate these tools into their day to day development activities. # App Inspector > App Inspector is a LocalStack tool designed to enhance system observability by enabling users to view, collect, and inspect data exchanges, including event payloads and metadata, between AWS services. ## Introduction App Inspector allows users to view, collect, and inspect data exchanges, including payloads and metadata, between AWS services. It enhances system observability by displaying the data exchanged at every stage, facilitating clear understanding of operation flows. In addition, it serves as a single point of truth to understand potential errors, service configuration mismatches, and IAM permission issues. With App Inspector, you can: - Observe and understand the flow of operations through your system. - Identify errors and obtain detailed information for corrections. - Get immediate feedback on any misconfigurations in your services. - Gain insights into IAM policies and detect missing permissions. ## Requirements App Inspector requires **LocalStack 2026.04.0** or later. ## Use Cases **Understanding service-to-service information flow** When a service call doesn't reach its intended target, App Inspector shows exactly where it stopped and why — tracing the flow of information between services at every hop. You get a precise answer without deploying to AWS or adding instrumentation. **Verifying service-to-service payloads** Inspect the exact data passed between services at every hop of your workflow. For example, confirming that a Lambda function is passing the correct payload to SNS, or that an SQS message body matches what the downstream consumer expects — before it becomes a production issue. **Catching IAM misconfigurations early** App Inspector runs against the LocalStack emulator, which monitors use of IAM policies. Missing or overly restrictive permissions show up immediately — in your local environment, where they're fast and cheap to fix, rather than at deployment time in the cloud. ## LocalStack Toolkit for VS Code App Inspector is built into the [LocalStack Toolkit for VS Code](/aws/connecting/ides/vscode-extension), so you can trace operation flows and inspect payloads without leaving your editor. Once LocalStack is running, open the App Inspector panel directly in the LocalStack Toolkit extension to view operations and drill into service interactions. If you haven't set up the toolkit yet, see [LocalStack Toolkit for VS Code](/aws/connecting/ides/vscode-extension) to get started. ![App Inspector panel in the LocalStack Toolkit for VS Code showing a live event stream](/images/aws/app-inspector/app-inspector-vscode.png) ## Web Application You can access App Inspector directly in the [LocalStack Web Application](https://app.localstack.cloud/inst/default/appinspector/spans). Navigate to **App Inspector** in your browser to view operations captured from your running LocalStack instance, inspect payloads, and trace service connections. :::note Operations generated during resource creation will appear in App Inspector alongside your application operations. To keep your trace clean, click **Clear Operations** before sending requests so only the relevant activity is captured. ::: ### Features ### List Operations You can view a detailed list of operations in your application, including the **Producer** (the service that initiated the call), **Consumer** (the target service), **Action** (the specific API call made), and **Timestamp**. Operations that encountered errors are flagged inline, so you can identify failures at a glance. Click **View Graph** on any operation to switch to the graph view and see how services connect. The interface enables you to trace the flow of operations, identify relationships between services, and analyze patterns for debugging or optimization. ![App Inspector event list showing a fanout pipeline across EventBridge, Lambda, SQS, and SNS](/images/aws/app-inspector/app-inspector-fanout-pipeline.png) Use the search box above the operations list to filter by any text fragment, such as a resource name, an ARN segment, a status message, or a payload snippet. Search is case-insensitive, matches substrings anywhere in the field, and covers the full result set rather than just the operations currently loaded on screen. ### Graph View and Operation Details Click **View Detail** to visualize the relationships between AWS services in your application as an interactive graph. Each node represents a service, and each edge represents a call between them, making it easy to follow the path of a request across your architecture. By clicking on an operation in the graph or list, you can drill down into the specifics of each interaction. App Inspector displays detailed payloads, metadata, and status for each operation, and highlights errors, warnings, and potential IAM permission issues, enabling precise debugging and troubleshooting. ![App Inspector operation details panel showing service, resource, payloads, and status for a Lambda invocation](/images/aws/app-inspector/app-inspector-operation-details.png) ## Supported Services :::note Visit our [Licensing & Tiers](https://docs.localstack.cloud/aws/licensing/) doc to explore usage with App Inspector. ::: All service operations appear in App Inspector. The following services have been optimized for improved visual layout in the graph view: - [S3](/aws/services/s3) - [SQS](/aws/services/sqs/) - [SNS](/aws/services/sns/) - [DynamoDB](/aws/services/dynamodb/) - [Lambda](/aws/services/lambda/) - [EventBridge](/aws/services/events/) - [Step Functions](/aws/services/stepfunctions) - [EventBridge Pipes](/aws/services/pipes/) - [Kinesis Data Streams](/aws/services/kinesis/) - [API Gateway V1](/aws/services/apigateway/) # AWS Replicator > AWS Replicator makes it easier to use LocalStack in shared AWS environments by copying resources into LocalStack. import { Tabs, TabItem } from '@astrojs/starlight/components'; import ReplicatorCoverage from '@/components/replicator-coverage/ReplicatorCoverage'; ## Introduction Applications deployed on AWS often depend on shared resources defined outside their own stack, for example a VPC managed by another team. Reproducing this kind of setup in LocalStack is hard: the dependencies may not live in IaC you have access to, and some resources are referenced by ARN, which is partly random, so simply recreating the resource in LocalStack produces a different ARN. The AWS Replicator solves this by creating identical copies of existing AWS resources directly inside a running LocalStack instance. This lets you replicate external dependencies before deploying your application, without changing existing stacks or building custom bootstrap infrastructure. :::note The AWS Replicator is in a preview state, supporting only [selected resources](#supported-resources). The new CLI experience, `lstk`, does not yet support the AWS Replicator. Continue using the legacy [LocalStack CLI](/aws/developer-tools/running-localstack/localstack-cli/) version 4.2.0 or newer. ::: ## Getting started A valid `LOCALSTACK_AUTH_TOKEN` must be configured to start the LocalStack for AWS image. ### Retrieve credentials to access AWS The AWS Replicator needs read access to your AWS account and performs a limited set of read-only operations on supported resources. These operations can be limited by creating a minimal IAM role with just the policy actions required for replication, and providing credentials to assume this role. See the [supported resources section](#supported-resources) for details of what policy actions are required for each resource. Replication should be triggered from a shell that has access to AWS. Here are some options for providing credentials: If you have the AWS CLI v2 installed, the CLI will read credentials from your configured `AWS_PROFILE`. ```bash export AWS_PROFILE=my-aws-profile localstack replicator ... ``` If you have the AWS CLI v1 installed or no installation of the AWS CLI, the following environment variables must be set: - `AWS_ACCESS_KEY_ID` - `AWS_SECRET_ACCESS_KEY` - `AWS_SESSION_TOKEN` (optional) - `AWS_DEFAULT_REGION` ### Trigger a replication job Replication jobs can be triggered using the LocalStack CLI or an HTTP API. Both methods have two steps: 1. Submit a replication job. 2. Check the job status. #### Replication strategies The Replicator supports different strategies depending on how many resources you want to copy and whether their related resources should be included. The [supported resources](#supported-resources) table shows which strategies are available for each resource type, along with the IAM actions required for each. - **Single** (`SINGLE_RESOURCE`): replicate a single resource identified by its identifier or ARN. This is the default. - **Batch** (`BATCH`): discover and replicate every matching resource in a single job, for example all SSM parameters under a path prefix or all S3 buckets matching a prefix. Only some resource types support batch discovery, check the **Batch** badge in the [supported resources](#supported-resources) table. - **Tree** (`TREE` explore strategy): starting from a single resource, also replicate its related child resources. For example, replicating an `AWS::Organizations::Organization` also replicates its organizational units, accounts, and policies. `replication_type` (single vs. batch discovery) and `explore_strategy` (`SIMPLE` or `TREE`, whether related resources are followed) are independent settings and can be combined. Two options apply regardless of strategy: - **Cross-region source discovery**: if the resource lives in a different AWS region than your credentials' default region (or its ARN has no region component, as with S3 buckets), set a source region explicitly. - **Skip existing resources**: by default a job fails if the target resource already exists. Set [`ignore_already_existing`](#skip-existing-resources) to skip it instead and continue the job. #### Using the LocalStack CLI The Replicator CLI is part of the LocalStack CLI. Follow the [installation instructions](/aws/developer-tools/running-localstack/localstack-cli/#installation) to set it up. Trigger a job with `localstack replicator start`, identifying the resource by its ARN: ```bash showLineNumbers export LOCALSTACK_AUTH_TOKEN= export AWS_DEFAULT_REGION=... # if required # export AWS_ACCESS_KEY_ID= # export AWS_SECRET_ACCESS_KEY= localstack replicator start --resource-arn ``` Or, identify the resource by its CloudControl type and identifier instead, using `--resource-type` and `--resource-identifier`: ```bash localstack replicator start \ --resource-type \ --resource-identifier ``` The command outputs the new job, including its `job_id`: ```json showLineNumbers { "job_id": "50005865-1589-4f6d-a720-c86f5a5dd021", "state": "TESTING_CONNECTION", "resources": {"succeeded": [], "failed": [], "skipped": []}, "error_message": null, "type": "SINGLE_RESOURCE", "explore_strategy": "SIMPLE" } ``` **Batch replication**: pass `--replication-type BATCH` to discover and replicate every matching resource in a single job, instead of one resource at a time. Only some resource types support batch discovery, check the **Batch** badge in the [supported resources](#supported-resources) table. For example, to replicate all SSM parameters under `/dev/`: ```bash localstack replicator start \ --replication-type BATCH \ --resource-type AWS::SSM::Parameter \ --resource-identifier /dev/ ``` `--resource-identifier` means something different for each resource type in batch mode: for `AWS::SSM::Parameter` it's a path prefix (not a wildcard or glob). Check the [supported resources](#supported-resources) table for the identifier format each resource type expects. **Targeting**: - `--target-account-id` sets the destination LocalStack account. Defaults to `000000000000`. - `--target-region-name` sets the destination region. Defaults to the source region. - `--source-region-name` sets the AWS region to read the resource from. Use it when the resource lives in a different region than your credentials' default region, or when its ARN has no region component, as with S3 buckets. **Resource-specific configuration**: some resource types accept parameters that aren't covered by the flags above. Pass them with `--extra-config KEY=VALUE` (repeat the flag for multiple entries). For example, replicating an `AWS::RDS::DBCluster` or `AWS::RDS::DBInstance` accepts a `master_user_password`: ```bash localstack replicator start \ --resource-type AWS::RDS::DBCluster \ --resource-identifier my-cluster \ --extra-config master_user_password= ``` Check the [supported resources](#supported-resources) table for which resource types accept extra configuration and which keys they support. ##### Skip existing resources By default, a replication job fails when creating a resource that already exists in LocalStack. To keep the existing resource unchanged and continue the job, set `ignore_already_existing` to `true`: ```bash localstack replicator start \ --resource-type AWS::SSM::Parameter \ --resource-identifier myparam \ --extra-config ignore_already_existing=true ``` ```json title="Output" { "job_id": "50005865-1589-4f6d-a720-c86f5a5dd021", "state": "TESTING_CONNECTION", "resources": {"succeeded": [], "failed": [], "skipped": []}, "error_message": null, "type": "SINGLE_RESOURCE", "explore_strategy": "SIMPLE" } ``` Only conflicts caused by an existing target resource are skipped. The Replicator does not overwrite or synchronize the existing resource, and other errors still fail the job. After the job completes, the identifier appears in `resources.skipped` and the job state is `SUCCEEDED`: ```json { "job_id": "50005865-1589-4f6d-a720-c86f5a5dd021", "state": "SUCCEEDED", "resources": {"succeeded": [], "failed": [], "skipped": ["myparam"]}, "error_message": null, "type": "SINGLE_RESOURCE", "explore_strategy": "SIMPLE" } ``` The option applies to single, batch, and tree replication jobs. Tree replication also automatically skips some shared related resources, such as an IAM managed policy referenced by multiple roles. #### Using the HTTP API To trigger replication via the HTTP API, send a `POST` request to: ```bash http://localhost.localstack.cloud:4566/_localstack/replicator/jobs ``` with a payload identifying the resource, the replication strategy, and AWS credentials to read it: ```json showLineNumbers { "replication_type": "SINGLE_RESOURCE", "replication_job_config": { "resource_type": "", "resource_identifier": "" }, "source_aws_config": { "aws_access_key_id": "...", "aws_secret_access_key": "...", "aws_session_token": "...", // optional "region_name": "...", "endpoint_url": "..." // optional }, "target_aws_config": {} // optional, same shape as `source_aws_config` } ``` Unlike the CLI, the HTTP API always requires `aws_access_key_id` and `aws_secret_access_key` in `source_aws_config` — it cannot fall back to a local AWS profile. For example, to replicate an SSM parameter named `myparam` using credentials whose default region is `eu-central-1`: ```json showLineNumbers { "replication_type": "SINGLE_RESOURCE", "replication_job_config": { "resource_type": "AWS::SSM::Parameter", "resource_identifier": "myparam" }, "source_aws_config": { "aws_access_key_id": "...", "aws_secret_access_key": "...", "region_name": "eu-central-1" } } ``` Use `replication_type: "BATCH"` to discover and replicate every matching resource instead of one. For example, to replicate all SSM parameters under `/dev/`: ```json { "replication_type": "BATCH", "replication_job_config": { "resource_type": "AWS::SSM::Parameter", "resource_identifier": "/dev/" }, "source_aws_config": { "aws_access_key_id": "...", "aws_secret_access_key": "...", "region_name": "eu-central-1" } } ``` Set `explore_strategy` to `TREE` to also replicate a resource's related resources. For example, to replicate an entire organization tree: ```json { "replication_type": "SINGLE_RESOURCE", "explore_strategy": "TREE", "replication_job_config": { "resource_type": "AWS::Organizations::Organization", "resource_identifier": "o-exampleorgid" }, "source_aws_config": { "aws_access_key_id": "...", "aws_secret_access_key": "...", "region_name": "..." } } ``` When omitted, `explore_strategy` defaults to `SIMPLE`, which replicates only the requested resource. The related resources replicated by `TREE` are listed in the [supported resources](#supported-resources) table, along with the additional IAM actions they require. `replication_job_config` also accepts `source_region_name`, any resource-specific extra configuration such as `master_user_password`, and `ignore_already_existing: true` to use the [skip existing resources](#skip-existing-resources) behavior. To list every replication job instead of one, send a `GET` request to the same `/_localstack/replicator/jobs` endpoint without a job ID. ### Check Replication Job Status Replication jobs run asynchronously, so you need to poll their status to check when they finish. #### Using the LocalStack CLI When creating a replication job, the response includes a `job_id`. Use this ID to check the job status: ```bash export LOCALSTACK_AUTH_TOKEN= localstack replicator status ``` This command returns the job status in JSON format. For example, here's a single-resource replication job: ```json showLineNumbers { "job_id": "50005865-1589-4f6d-a720-c86f5a5dd021", "state": "SUCCEEDED", "resources": {"succeeded": ["myParameter"], "failed": [], "skipped": []}, "error_message": null, "type": "SINGLE_RESOURCE", "explore_strategy": "SIMPLE" } ``` `state` is one of `TESTING_CONNECTION`, `RUNNING`, `SUCCEEDED`, or `ERROR`. `resources` lists the identifiers of resources that succeeded, failed, or were [skipped](#skip-existing-resources) — for a single-resource job these lists have at most one entry, for a batch job they can have many: ```json { "job_id": "9acdc850-f71b-4474-b138-1668eb8b8396", "state": "SUCCEEDED", "resources": { "succeeded": ["/dev/param1", "/dev/param2"], "failed": [], "skipped": [] }, "error_message": null, "type": "BATCH", "explore_strategy": "SIMPLE" } ``` For long-running jobs, the CLI can poll the status until the job reaches a terminal state. To wait for the job to finish, use the `--follow` flag. #### Using the HTTP API To check the status of a replication job via the HTTP API, send a `GET` request to `http://localhost.localstack.cloud:4566/_localstack/replicator/jobs/`. :::tip If the replication state is `SUCCEEDED` but the resource is missing, check in account `000000000000`. ::: ## Quickstart This quickstart example creates an SSM parameter in AWS and replicates it to LocalStack. To start, create the parameter in AWS. This example uses an SSO profile named `ls-sandbox` for AWS configuration, and replicates resources from the `eu-central-1` region. ```bash AWS_PROFILE=ls-sandbox aws ssm put-parameter\ --name myparam \ --type String \ --value abc123 ``` ```json { "Version": 1, "Tier": "Standard" } ``` ```bash AWS_PROFILE=ls-sandbox aws ssm get-parameters --names myparam ``` ```json showLineNumbers { "Parameters": [ { "Name": "myparam", "Type": "String", "Value": "abc123", "Version": 1, "LastModifiedDate": "2025-02-07T13:36:56.240000+00:00", "ARN": "arn:aws:ssm:eu-central-1::parameter/myparam", "DataType": "text" } ], "InvalidParameters": [] } ``` The SSM parameter has the ARN: `arn:aws:ssm:eu-central-1::parameter/myparam`. Next, we can check that the parameter is not present in LocalStack using `awslocal`: ```bash awslocal ssm get-parameters --name myparam ``` ```json showLineNumbers { "Parameters": [], "InvalidParameters": [ "myparam" ] } ``` Next, trigger replication from AWS to LocalStack, using the same `ls-sandbox` profile: ```bash LOCALSTACK_AUTH_TOKEN= \ AWS_PROFILE=ls-sandbox \ localstack replicator start \ --resource-type AWS::SSM::Parameter \ --resource-identifier myparam ``` ```json showLineNumbers { "job_id": "9acdc850-f71b-4474-b138-1668eb8b8396", "state": "TESTING_CONNECTION", "resources": {"succeeded": [], "failed": [], "skipped": []}, "error_message": null, "type": "SINGLE_RESOURCE", "explore_strategy": "SIMPLE" } ``` You can check the replication job status using the `job_id`: ```bash LOCALSTACK_AUTH_TOKEN= \ localstack replicator status 9acdc850-f71b-4474-b138-1668eb8b8396 ``` ```json showLineNumbers { "job_id": "9acdc850-f71b-4474-b138-1668eb8b8396", "state": "SUCCEEDED", "resources": {"succeeded": ["myparam"], "failed": [], "skipped": []}, "error_message": null, "type": "SINGLE_RESOURCE", "explore_strategy": "SIMPLE" } ``` The state is `SUCCEEDED`, indicating the replication job completed successfully. The SSM parameter is now accessible. ```bash awslocal ssm get-parameters --name myparam --region eu-central-1 ``` ```json showLineNumbers { "Parameters": [ { "Name": "myparam", "Type": "String", "Value": "abc123", "Version": 1, "LastModifiedDate": 1738935663.08, "ARN": "arn:aws:ssm:eu-central-1:000000000000:parameter/myparam", "DataType": "text" } ], "InvalidParameters": [] } ``` The resource is replicated into the same AWS region by default. Use the `--target-region-name` flag to change it. By default, replication occurs in LocalStack account `000000000000`. Use the `--target-account-id` flag to specify a different account. ## Supported Resources We welcome feedback and bug reports. Please open a [new GitHub Discussion](https://github.com/orgs/localstack/discussions/new/choose) to request and upvote for support for new resources. :::tip To ensure support for all resources, use the latest LocalStack Docker image. ::: The table below lists every supported resource type and the [replication strategies](#replication-strategies) available for each. Select a row to expand it and view the resource identifier, the IAM actions required for each strategy, and any related resources replicated by the Tree strategy. # Overview > Chaos Engineering with LocalStack enables you to build resilient systems early on in the development phase. Chaos engineering in LocalStack helps you build more resilient systems by deliberately introducing controlled disruptions into your cloud environment. Simulating failures early in the development process helps teams proactively uncover weaknesses, improve error handling, and validate system behavior under stress. Different teams benefit from chaos engineering in different ways: - Software Developers test application logic and error response behavior - Architects evaluate the robustness of system design - Operations teams investigate infrastructure reliability under adverse conditions LocalStack supports the following chaos engineering features: - **Application behavior and error management**: AWS Fault Injection Service (FIS) simulates errors and latency - **Robust architecture**: Failover testing and resilience validation via the Chaos API - **Consistent infrastructure setup**: Disruption-tolerant provisioning to verify consistent infrastructure under unstable conditions The best way to get started is by practicing running experiments yourself. Check out our [chaos engineering tutorials](/aws/tutorials). # AWS Fault Injection Service > Use Fault Injection Service to simulate faults in your infrastructure and test its fault tolerance. The [Fault Injection Service (FIS)](https://aws.amazon.com/fis/) is a fully managed service by AWS designed to help you improve the resilience of your applications by simulating real-world outages and operational issues. This service allows you to conduct controlled experiments on your AWS infrastructure, injecting faults and observing how your system responds under various conditions. By using the Fault Injection Service, you can identify weaknesses, test recovery procedures, and ensure that your applications can withstand unexpected disruptions. This proactive approach to reliability engineering enables you to enhance system robustness, minimize downtime, and maintain a high level of service availability for your users. :::note Fault Injection Service emulation is available as part of the LocalStack Enterprise plan. If you'd like to try it out, please [contact us](https://www.localstack.cloud/demo) to request access. ::: :::tip For more information, please refer to the [FIS service docs](/aws/services/fis). ::: Some of the most important concepts associated with a FIS experiment are: **1. Experiment Templates**: Experiment templates define the actions, targets, and any stop conditions for your experiment. They serve as blueprints for conducting fault injection experiments, allowing you to specify what resources are targeted, what faults are injected, and under what conditions the experiment should automatically stop. **2. Actions**: Actions are the specific fault injection operations that the experiment performs on the target resources. These can be injecting latency or throttling to API requests, completely blocking access to instances, etc. Actions define the type of fault, parameters for the fault injection, and the targets affected. **3. Targets**: Targets are the AWS resources on which the experiment actions will be applied. To make things even more fine-grained, a specific operation of the service can be targeted. **4. Stop Conditions**: Stop conditions are criteria that, when met, will automatically stop the experiment. **5. IAM Roles and Permissions**: To run experiments, AWS FIS requires specific IAM roles and permissions. These are necessary for AWS FIS to perform actions on your behalf, like injecting faults into your resources. **6. Experiment Execution**: When you start an experiment, AWS FIS executes the actions defined in the experiment template against the specified targets, adhering to any defined stop conditions. The execution process is logged, and detailed information about the experiment's progress and outcome is provided. # Chaos API > Simulate outages and network failures to test the resiliency of your infrastructure ## Introduction LocalStack Chaos API allows you to mimic outages across any AWS region or service. Intentionally triggering service outages and monitoring the system's response in situations where the infrastructure is compromised offers a powerful way to test. This strategy helps gauge the effectiveness of the system's deployment procedures and its resilience against infrastructure disruptions, which is a key element of chaos engineering. You can use LocalStack Chaos API to cause API failures for any combination of the following: - Service - Region - Operation You can customize the HTTP error code and message that LocalStack responds with. If required, you can make the failures occur probabilistically. Furthermore, the Chaos API can also be configured to add a network latency for all calls. :::note Chaos API is available as part of the LocalStack Enterprise plan. If you'd like to try it out, please [contact us](https://www.localstack.cloud/demo) to request access. ::: ## Prerequisites The prerequisites for this guide are: - LocalStack for AWS and [`lstk`](/aws/developer-tools/running-localstack/lstk/#installation) - [LocalStack Auth Token](/aws/getting-started/auth-token/) - [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) - [Python](https://www.python.org/downloads/) ## Configuration The disruption types supported by Chaos API are broadly categorised into two groups. **Service Faults** lead to an application-level HTTP error in an AWS service, and **Network Effects** introduce network-level effects to all connections. ### Service Faults Service faults can be configured using the endpoint at `/_localstack/chaos/faults`. The configuration schema consists of an array of one or more rules, where each rule specifies the conditions for the fault to occur. When active, rules are evaluated sequentially on every request to LocalStack until the first match. The schema for the configuration is as follows. ```json showshowLineNumbers [ { "region": "(str) Region name, e.g. 'ap-south-1'. If omitted, all regions are affected.", "service": "(str) Name of the service, e.g. 'kinesis'. If omitted, all services are affected.", "operation": "(str) Name of the operation, e.g. 'PutRecord'. If omitted, all operations are affected.", "probability": "(num) Probability of invoking this rule, e.g. 0.5. If omitted, 1 is used.", "error": { "statusCode": "(int) HTTP status code to use in response, e.g. 503. If omitted, 503 is used.", "code": "(str) Descriptive error code used in response. If omitted, 'ServiceUnavailable' is used." } }, ... ] ``` The endpoint allows the following operations: - `GET`: Get current configuration - `POST`: Add new configuration - `PATCH`: Add a rule - `DELETE`: Delete a rule An empty array `[]` disables the faults entirely, while an empty rule in the array `[{}]` causes all AWS operations to lead to faults. ### Network Effects Network effects are configured using the endpoint `/_localstack/chaos/effects`. Currently the Chaos API only supports a latency factor. ```json { "latency": "(int) Network latency in milliseconds. By default, 0 is used." } ``` This endpoint allows the following operations: - `GET`: Get current configuration - `POST`: Add new configuration ## Examples To cause faults, make a POST request as follows: ```bash curl --location --request POST 'http://localhost.localstack.cloud:4566/_localstack/chaos/faults' \ --header 'Content-Type: application/json' \ --data ' [ { "service": "s3", "region": "us-east-1" }, { "service": "s3", "region": "ap-south-1" }, { "service": "lambda" } ]' ``` In this example, S3 is affected in `us-east-1` and `ap-south-1,` and Lambda is affected in all regions. All calls to these services in these regions will return a 503 Service Unavailable error. To see this in action, try to create an S3 bucket in `us-east-1`: ```bash lstk aws s3 mb s3://test-bucket --region us-east-1 ``` ```bash make_bucket failed: s3://test-bucket An error occurred (ServiceUnavailableException) when calling the CreateBucket operation (reached max retries: 4): Service 's3' not accessible due to an outage ``` However, the same operation, when run in `eu-central-1` will work as expected. ```bash lstk aws s3 mb s3://test-bucket --region eu-central-1 ``` ```bash make_bucket: test-bucket ``` Faults can be disabled by setting an empty rule list in the configuration. The following request will clear the current configuration: ```bash curl --location --request POST 'http://localhost.localstack.cloud:4566/_localstack/chaos/faults' \ --header 'Content-Type: application/json' \ --data '[]' ``` To retrieve the current configuration, make the following GET call: ```bash curl --location --request GET 'http://localhost.localstack.cloud:4566/_localstack/chaos/faults' ``` To add a new rule to the current configuration, make a PATCH call as follows: ```bash curl --location --request PATCH 'http://localhost.localstack.cloud:4566/_localstack/chaos/faults' \ --header 'Content-Type: application/json' \ --data ' [ { "service": "kinesis", "operation": "PutRecord", "probability": 0.3, "error": { "statusCode": 400, "code": "ProvisionedThroughputExceededException" } } ]' ``` This new rule will cause probabilistic failures for Kinesis PutRecord operation. Here, the returned error is also customised to be HTTP 400 ProvisionedThroughputExceededException. To remove a rule from the configuration, make a DELETE call as follows: ```bash curl --location --request DELETE 'http://localhost.localstack.cloud:4566/_localstack/chaos/faults' \ --header 'Content-Type: application/json' \ --data '[{"service": "lambda"}]' ``` The rule to be removed must be exactly the same as in the existing configuration. ## Comparison with Fault Injection Service AWS [Fault Injection Service (FIS)](/aws/services/fis) also allows controlled chaos engineering experiments on infrastructure. While similar in purpose, there are notable differences between FIS and LocalStack Chaos API. This table highlights those differences, offering a detailed comparison of how each service approaches chaos engineering, their capabilities, and their integration options. | **Aspect** | **AWS Fault Injection Service (FIS)** | **LocalStack Chaos API** | |-----------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------| | **Fault Types** | • EC2 Stop/Terminate Instances
• RDS Reboot Instances
• SSM Send Command
• Inject API errors (e.g., `aws:fis:inject-api-internal-error` for EC2 only) | • API failures (HTTP error codes and messages for any service)
• Network effects (latency)
• Can be probabilistic and customized. | | **Procedural vs Declarative** | • Capable of running procedural experiments where it invokes API actions affecting AWS resources (e.g., `aws:ec2:stop-instances`). | • Focuses on declarative effects impacting the AWS API, such as returning errors or adding latency, without invoking AWS resource actions. | | **Experiment Execution** | • Requires creating and running controlled experiments with predefined templates. Systems are restored after disruption duration. | • Faults are applied dynamically based on configuration rules. Can inject faults on-the-fly without predefining experiments. | | **Customization** | • Limited to predefined actions (e.g., stopping EC2 instances, inducing specific errors like InternalError for EC2). | • Highly customizable, including probabilistic failures, custom error codes, HTTP status codes, and errors for any AWS operation. | | **Service Coverage** | • Covers specific AWS services such as EC2, RDS, and SSM. | • Covers all AWS services and operations (e.g., S3, Lambda, Kinesis) with no service-specific restrictions. | | **Network Effects** | • Not supported. | • Supports adding network latency to simulate slow network conditions. | | **API Interaction** | • `create-experiment-template` to create templates
• `start-experiment` to begin experiments
| • `POST` to `/chaos/faults` to configure faults
• `POST` to `/chaos/effects` to introduce network effects.
| | **Probabilistic Failure Injection** | • Not available. | • Supports probabilistic failure injection, introducing partial failures, which mimic intermittent outages. | | **Broader Fault Injection** | • Limited to predefined actions (e.g., stop instances, reboot databases, inject errors for specific services). | • Broader fault injection for any AWS operation (e.g., PutObject for S3, Invoke for Lambda). | # Chaos Engineering Dashboard > Chaos Engineering Dashboard allows users to run chaos experiments within their application stack to test the system's resilience. ## Introduction The Chaos Engineering Dashboard in LocalStack offers streamlined testing for cloud applications, enabling you to simulate server errors, service outages, regional disruptions, and network latency with ease, ensuring your app is ready for real-world challenges. The dashboard uses [LocalStack Chaos API](/aws/developer-tools/chaos-engineering/chaos-api) under the hood to offer a set of customizable templates that can be seamlessly integrated into any automation workflows. ![chaos engineering dashboard](/images/aws/chaos-engineering-dashboard.png) You can find this feature in the LocalStack Web Application by navigating to [**app.localstack.cloud/chaos-engineering**](https://app.localstack.cloud/chaos-engineering). :::note Chaos Engineering Dashboard is offered as a **preview** feature and is under active development. ::: ## Features The dashboard offers the following features: * **DynamoDB Error**: Randomly inject `ProvisionedThroughputExceededException` errors into DynamoDB API responses. * **Kinesis Error**: Randomly inject `ProvisionedThroughputExceededException` errors into Kinesis API responses. * **500 Internal Error**: Randomly terminate incoming requests, returning an `Internal Server Error` with a response code of 500. * **Service Unavailable**: Cause a specified percentage of service API calls to receive a 503 `Service Unavailable` response. * **AWS Region Unavailable**: Simulate regional outages and failovers by disabling entire AWS regions. * **Latency**: Introduce specified latency to every API call, useful for simulating network latency or degraded network performance. # Overview > You can run a LocalStack instance as a Cloud Sandbox and access it from your local machine. ## Introduction LocalStack Cloud Sandbox lets you deploy a fully functional LocalStack instance in the cloud instead of running it locally. This enables new workflows for testing, previewing, and collaboration across teams. Key use cases include: - Running ephemeral instances locally for dev/test loops in CI - Generating on-demand preview environments by enabling preview-per-PR type workflows - Facilitate collaboration by sharing a consistent testing environment for all developers :::note The Cloud Sandbox feature is currently in **preview** and under active development. ::: # Application Preview > Create an Application Preview to deploy your application changes in an Ephemeral Instance. ## Introduction Application Preview generates a preview environment from GitHub Actions workflows. For example, you can create a preview URL for every GitHub Pull Request (PRs). It allows temporary deployment of AWS powered applications on a LocalStack Ephemeral Instance to preview changes. This feature is currently only available for GitHub repositories that use GitHub Actions. :::note Application Preview is offered as a **preview** feature and is under active development. ::: ## Getting started This guide is designed for users new to Application Preview and assumes basic knowledge of GitHub Actions. We will configure a CI pipeline that runs on pull requests using GitHub Actions. ### Prerequisites - [LocalStack Account](https://app.localstack.cloud/) - [GitHub Account](https://github.com) ### Create the Application Preview To create an Application Preview, use the [`LocalStack/setup-localstack` action](https://github.com/localstack/setup-localstack). Create a file named `preview-pipeline.yml` in the `.github/workflows` directory of your custom repository. The example below contains the details that can be used for a step in an existing CI pipeline that activates on every pull request. The pipeline deploys the application to a LocalStack Ephemeral Instance. The steps necessary for building and deploying the application to LocalStack should be listed within the `preview-cmd` property. The example below aggregates these steps into a single deployment script called `deploy.sh`. A comment containing the preview link is automatically added to a Pull Request when created. This preview is available for 30 minutes ```yaml uses: LocalStack/setup-localstack@v0.2.3 with: github-token: ${{ secrets.GITHUB_TOKEN }} state-backend: ephemeral state-action: start # Adding this option prevents Ephemeral Instance to be stopped after the `preview-cmd` run skip-ephemeral-stop: 'true' # Optional script/command to run preview-cmd: deploy.sh env: LOCALSTACK_API_KEY: ${{ secrets.LOCALSTACK_API_KEY }} ``` The `LOCALSTACK_API_KEY` must be set as a repository secret. You can create a [CI key](https://app.localstack.cloud/workspace/ci-keys) via the LocalStack Web Application (note that you may need permission from your administrator to access this capability). To add this secret to a GitHub repository, go to the "Settings" tab and then on the left-hand navigation choose "Secrets and variables" and then "Actions". The `GITHUB_TOKEN` is automatically generated by GitHub and requires no further configuration. ### Stop the Application Preview It's important to understand that the ephemeral instance that powers the application preview is created _before_ the steps in the `preview-cmd` are run. If the `preview-cmd` fails to complete successfully, the ephemeral instance will still be running on your account. To address this, you can create an additional workflow step that shuts down the ephemeral instance if the workflow run fails (see the [GitHub Actions documentation](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#running-a-workflow-based-on-the-conclusion-of-another-workflow) for help in determining whether a workflow completed successfully). To stop the Application Preview, you can configure the `state-action` to `stop`. ```yaml uses: LocalStack/setup-localstack@v0.2.2 with: github-token: ${{ secrets.GITHUB_TOKEN }} state-backend: ephemeral state-action: stop env: LOCALSTACK_API_KEY: ${{ secrets.LOCALSTACK_API_KEY }} ``` ## Configuration | Input | Description | Default | |------------------------------|---------------------------------------------------------------------------|--------------| | `auto-load-pod` | Specifies which Cloud Pod to load during LocalStack startup | `None` | | `extension-auto-install` | Defines which extensions to install during LocalStack startup for Application Previews | `None` | | `lifetime` | Duration an Application Preview remains active | 30 | | `state-backend` | Starts an Application Preview, used with `state-action` to manage state | `ephemeral` | | `state-action` | Commands `start`/`stop` for managing Application Previews | | | `skip-ephemeral-stop` | Option to bypass stopping the Application Preview | `false` | | `preview-cmd` | Commands to generate the application Preview of the PR (supports `$AWS_ENDPOINT_URL`) | | ## Overriding the Application Preview URL The Application Preview URL is automatically generated and added as a comment to the Pull Request. However, if your application is served on a different URL, you can override the URL using the `LS_PREVIEW_URL`. It is beneficial if you are using a CloudFront distribution or a custom domain. Here is an example of how to override the URL: ```yaml preview-cmd: | make build; make bootstrap; make deploy; make build-frontend; make deploy-frontend; distributionId=$(lstk aws cloudfront list-distributions | jq -r '.DistributionList.Items[0].Id'); echo LS_PREVIEW_URL=$AWS_ENDPOINT_URL/cloudfront/$distributionId/ >> $GITHUB_ENV; ``` ## Examples - [Creating ephemeral application previews with LocalStack and GitHub Actions](/aws/tutorials/ephemeral-application-previews/) and the [example repository](https://github.com/localstack-samples/sample-notes-app-dynamodb-lambda-apigateway) # Ephemeral Instances > Create an Ephemeral Instance in the cloud using the LocalStack Web Application ## Introduction Ephemeral Instances allows you to run a LocalStack instance in the cloud. You can interact with these instances via the LocalStack Web Application, or by configuring your integrations and developer tools with the endpoint URL of the ephemeral instance. :::note Ephemeral Instances is offered as a **preview** feature. `lstk` does not yet support Ephemeral Instances. Continue using the legacy [LocalStack CLI](/aws/developer-tools/running-localstack/localstack-cli/) for this feature. ::: ## Getting started This guide is designed for users new to Ephemeral Instance and assumes basic knowledge of the LocalStack Web Application. In this guide, we will create an Ephemeral Instance and interact with it via the LocalStack Web Application and the AWS CLI. ### Create a new Ephemeral Instance Navigate to the [**LocalStack Ephemeral Instance Management**](https://app.localstack.cloud/instances/ephemeral) page. In the form, enter the name of the new Ephemeral Instance, select the lifetime of the instance by dragging the slider, and click on **Launch**. ![Creating an Ephemeral Instance](/images/aws/ephemeral-instance-creation.png) Optionally, you can specify a LocalStack Extension to be installed or loaded in the Ephemeral Instance. You can select the extension from the **Extension settings** dropdown list before launching the Ephemeral Instance. In case you have access to Cloud Pods and a pod you want to start your instance with, you can also choose it from the **Cloud Pod Settings** dropdown. ### Interact with the Ephemeral Instance After the Ephemeral Instance is created, you will be able to see the instance in the **LocalStack Instance Management** page. You will also be able to access the following features with your Ephemeral Instance: - Status Page - Resource Browser - State Management - Extensions - Logs ![LocalStack Ephemeral Instance](/images/aws/localstack-ephemeral-instance.png) ### Access the Ephemeral Instance via AWS CLI You can access the Ephemeral Instance via the AWS CLI by configuring the AWS CLI with the endpoint URL of the Ephemeral Instance. You can find the endpoint URL of the Ephemeral Instance in the **LocalStack Instance Management** page. Copy the endpoint URL and set it as the `--endpoint-url` parameter in the AWS CLI command. To create an S3 bucket in the Ephemeral Instance, run the following command: ```bash aws --endpoint-url= s3 mb s3:// ``` You can replace `` with the endpoint URL of the Ephemeral Instance and `` with the name of the S3 bucket you want to create. To query the list of S3 buckets in the Ephemeral Instance, run the following command: ```bash aws --endpoint-url= s3 ls ``` You can also use integrations, such as [CDK](/aws/connecting/infrastructure-as-code/aws-cdk/), [SAM CLI](/aws/connecting/infrastructure-as-code/aws-sam/), and [Terraform](/aws/connecting/infrastructure-as-code/terraform/), to interact with the Ephemeral Instance. In these integrations, you can change the `AWS_ENDPOINT_URL` environment variable to the endpoint URL of the Ephemeral Instance. ### View the Logs of the Ephemeral Instance You can view the logs of the Ephemeral Instance by navigating to the **Logs** tab in the **LocalStack Instance Management** page. ![Ephemeral Instance Logs](/images/aws/ephemeral-instance-logs.png) ### Shut Down the Ephemeral Instance Open the Ephemeral Instance in the LocalStack Web Application. Click the three-dot menu next to the instance endpoint in the upper-left corner, then select **Shut down**. ![Shut down the LocalStack Ephemeral Instance](/images/aws/shutdown-ephemeral-instance.png) :::danger Ephemeral Instances, by default, are created with the latest version of LocalStack. If you have created a Cloud Pod from an older version of LocalStack, you need to update the Cloud Pod to the latest version before loading it into an Ephemeral Instance. ::: ## Ephemeral Instances CLI The Ephemeral Instances CLI is included in the [LocalStack CLI](/aws/developer-tools/running-localstack/localstack-cli/#installation), so no additional installations are needed to start using it. If you're a licensed user, setting the `LOCALSTACK_AUTH_TOKEN` as an environment variable is recommended to access all features of the Ephemeral Instances CLI. Access the Ephemeral Instances CLI by running the `localstack ephemeral` command from your terminal. ```bash localstack ephemeral --help ``` ```bash Usage: localstack ephemeral [OPTIONS] COMMAND [ARGS]... (Preview) Manage ephemeral LocalStack instances in the cloud. This command group allows you to create, list, and delete ephemeral LocalStack instances. Ephemeral instances are temporary cloud instances that can be used for testing and development. Options: -h, --help Show this message and exit. Commands: create Create a new ephemeral instance delete Delete an ephemeral instance list List all ephemeral instances logs Fetch logs from an ephemeral instance ``` To start an Ephemeral Instance, run the following command: ```bash localstack ephemeral create --name my-instance-123 ``` The output of the command should look like this: ```bash { "creation_time": 1731347416, "endpoint_url": "https://ls-ji9gajoqrveou.sandbox.localstack.cloud", "expiry_time": 1731351016, "id": "ji9gajoqrveou", "image": { "image_name": "localstack/localstack-pro", "tag": "latest" }, "instance_name": "my-instance-123", "labels": { "image-name": "localstack/localstack-pro", "image-tag": "latest", "instance-name": "my-instance-123", "requestor": "4e60f2cb" }, "requestor": "4e60f2cb", "shape": { "memory_megabytes": 2048, "virtual_cpus": 1 }, "status": "running" } ``` List your available running Ephemeral Instances with: ```bash localstack ephemeral list ``` Retrieve your Ephemeral Instance logs with: ```bash localstack ephemeral logs --name my-instance-123 ``` The logs output will look like this: ```bash LocalStack version: 3.8.2.dev98 LocalStack build date: 2024-11-11 LocalStack build git hash: c624ee66 2024-11-11T17:50:42.373 INFO --- [ MainThread] l.p.c.b.licensingv2 : Successfully requested and activated new license 636c4b55-b09c-4a93-bef6-2f6d024f7d8a:enterprise 🔑✅ 2024-11-11T17:50:43.504 INFO --- [ MainThread] l.p.c.extensions.platform : loaded 0 extensions Ready. ``` Finally, delete your Ephemeral Instance with: ```bash localstack ephemeral delete --name my-instance-123 ``` ```bash Successfully deleted instance: my-instance-123 ✅ ``` ## Credit Consumption Ephemeral Instances consume credits based on the resources used and the duration of the instance. You can view the credit consumption of the Ephemeral Instance in the **Ephemeral Instance** page. Currently, for every 1 credit, you can run an Ephemeral Instance for 1 minute. You can view the available minutes under the **Lifetime in minutes** slider when creating an Ephemeral Instance. You can also see the credit consumption in the **Credit Consumption** section of the Ephemeral Instance page. ![Credit Consumption](/images/aws/credit-consumption.png) # Overview > Develop your Lambdas more efficiently. LocalStack’s Lambda Tools offer a set of utilities to streamline the local development, testing, and debugging of AWS Lambda functions. Emulating Lambda behavior on your machine lets you skip cloud deployments and iterate faster in a fully local environment. These tools are designed to shorten feedback loops and improve the developer experience with features like: - **IDE debugging**: Attach a debugger to your running Lambda function, set breakpoints, inspect variables, and step through code. - **Hot reload**: Automatically apply code changes without needing to redeploy the function, enabling rapid iteration. # Hot Reloading > Hot code reloading continuously applies code changes to Lambda functions. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction Hot reloading (formerly known as hot swapping) continuously applies code changes to Lambda functions without manual redeployment. Quickly iterating over Lambda function code can be quite cumbersome, as you need to deploy your function on every change. LocalStack enables fast feedback cycles during development by automatically reloading your function code. Pro users can also hot-reload Lambda layers. :::note The magic S3 bucket name changed from `__local__` to `hot-reload` in LocalStack v2.0. Please change your deployment configuration accordingly because the old value is an invalid bucket name. The configuration `BUCKET_MARKER_LOCAL` is still supported. More information about the new Lambda provider is available under [Lambda providers](/aws/services/lambda). ::: ## Hot Reloading Behavior **Delay in code change detection:** It can take up to 700ms to detect code changes. In the meantime, invocations still execute the former code. Hot reloading triggers after 500ms without changes, and it can take up to an additional 200ms until the reloaded code is in effect. **Runtime restart after each code change:** The runtime inside the container is restarted after every code change. During the runtime restart, the handler function re-executes any initialization code *outside* your handler function. The container itself is not restarted. Therefore, filesystem changes persist between code changes for invocations dispatched to the same container. **File sharing permissions with Docker Desktop on macOS:** If using Docker Desktop on macOS, you might need to allow [file sharing](https://docs.docker.com/desktop/settings/mac/#file-sharing) for your target folders. MacOS may prompt you to grant Docker access to your target folders. **Layer limit with hot reloading for layers:** When hot reloading is active for a Lambda layer (Pro), the function can use at most one layer. :::note Configuring the file sharing mechanism in Rancher Desktop or Colima distributions is necessary to enable hot reloading for Lambda. * For Rancher Desktop it is required to set the configuration `LAMBDA_DOCKER_FLAGS=-e LOCALSTACK_FILE_WATCHER_STRATEGY=polling`. * For Colima, it is required to start with the Virtiofs mount type: `colima start --vm-type vz --mount-type virtiofs`. ::: :::note **WSL2-compatible paths required with Rancher Desktop on Windows:** Make sure your Lambda handler paths are specified using WSL2-compatible paths. For example, instead of using a Windows-style path such as: ```bash C:\Users\myuser\projects\lambda\handler.py ``` Use the corresponding WSL-style path: ```bash /mnt/c/Users/myuser/projects/lambda/handler.py ``` This ensures that LocalStack can properly mount and watch your Lambda code inside the container when running under WSL2. ::: ## Application Configuration Examples ### Hot reloading for JVM Lambdas Since lambda containers lifetime is usually limited, regular hot code reloading techniques are not applicable here. In our implementation, we will be watching for fs changes under the project folder, then build a `FatJar`, unzip it, and mount it into the Lambda Docker Container. We assume you already have: * [watchman](https://facebook.github.io/watchman/) * configured JVM project capable of building FatJars using your preferred build tool First, create a watchman wrapper by using [one of our examples](https://github.com/localstack/localstack-pro-samples/tree/master/sample-archive/spring-cloud-function-microservice/bin/watchman.sh) Don't forget to adjust permissions: ```bash chmod +x bin/watchman.sh ``` Now configure your build tool to unzip the FatJar to some folder, which will be then mounted to LocalStack. We are using `Gradle` build tool to unpack the `FatJar` into the `build/hot` folder: ```groovy // We assume you are using something like `Shadow` plugin that comes with `shadowJar` task task buildHot(type: Copy) { from zipTree("${project.buildDir}/libs/${project.name}-all.jar") into "${project.buildDir}/hot" } buildHot.dependsOn shadowJar ``` Now run the following command to start watching your project in a hot-reloading mode: ```bash bin/watchman.sh src "./gradlew buildHot" ``` Please note that you still need to configure your deployment tool to use local code mounting. Read the [Deployment Configuration Examples](#deployment-configuration-examples) for more information. ### Hot reloading for Python Lambdas We will show you how you can do this with a simple example function, taken directly from the [AWS Lambda developer guide](https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/python/example_code/lambda/lambda_handler_basic.py). You can check out that code, or use your own lambda functions to follow along. To use the example just do: ```bash cd /tmp git clone git@github.com:awsdocs/aws-doc-sdk-examples.git ``` #### Creating the Lambda Function To create the Lambda function, you just need to take care of two things: 1. Deploy via an S3 Bucket. You need to use the magic variable `hot-reload` as the bucket. 2. Set the S3 key to the path of the directory your lambda function resides in. The handler is then referenced by the filename of your lambda code and the function in that code that needs to be invoked. So, using the AWS example, this would be: ```bash lstk aws lambda create-function --function-name my-cool-local-function \ --code S3Bucket="hot-reload",S3Key="/tmp/aws-doc-sdk-examples/python/example_code/lambda" \ --handler lambda_handler_basic.lambda_handler \ --runtime python3.8 \ --role arn:aws:iam::000000000000:role/lambda-role ``` You can also check out some of our [Deployment Configuration Examples](#deployment-configuration-examples). We can also quickly make sure that it works by invoking it with a simple payload: ```bash lstk aws lambda invoke --function-name my-cool-local-function \ --cli-binary-format raw-in-base64-out \ --payload '{"action": "increment", "number": 3}' \ output.txt ``` The invocation returns itself returns: ```bash title="Output" { "StatusCode": 200, "LogResult": "", "ExecutedVersion": "$LATEST" } ``` and `output.txt` contains: ```text {"result":4} ``` #### Changing things up Now, that we got everything up and running, the fun begins. Because the function is now mounted as a file in the executing container, any change that we save on the file will be there in an instant. For example, we can now make a minor change to the API and replace the response in [line 36](https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/python/example_code/lambda/lambda_handler_basic.py#L36) with the following: ```python response = {'math_result': result} ``` Without redeploying or updating the function, the result of the previous request will look like this: ```json {"math_result":4} ``` Cool! #### Usage with Virtualenv For [virtualenv](https://virtualenv.pypa.io)-driven projects, all dependencies should be made available to the Python interpreter at runtime. There are different ways to achieve that, including: * expanding the Python module search path in your Lambda handler * creating a watchman script to copy the libraries ##### Expanding the module search path in your Lambda handler The easiest approach is to expand the module search path (`sys.path`) and add the `site-packages` folder inside the virtualenv. We can add the following two lines of code at the top of the Lambda handler script: ```python import sys, glob sys.path.insert(0, glob.glob(".venv/lib/python*/site-packages")[0]) ... import some_lib_from_virtualenv # import your own modules here ``` This way you can easily import modules from your virtualenv, without having to change the file system layout. Note: As an alternative to modifying `sys.path`, you could also set the `PYTHONPATH` environment variable when creating your Lambda function, to add the additional path. ##### Using a watchman script to copy libraries Another alternative is to implement a watchman script that will be preparing a special folder for hot code reloading. In our example, we are using `build/hot` folder as a mounting point for our Lambdas. First, create a watchman wrapper by using [one of our examples](https://github.com/localstack/localstack-pro-samples/tree/master/sample-archive/spring-cloud-function-microservice/bin/watchman.sh) After that, you can use the following `Makefile` snippet, or implement another shell script to prepare the codebase for hot reloading: ```make showLineNumbers BUILD_FOLDER ?= build PROJECT_MODULE_NAME = my_project_module build-hot: rm -rf $(BUILD_FOLDER)/hot && mkdir -p $(BUILD_FOLDER)/hot cp -r $(VENV_DIR)/lib/python$(shell python --version | grep -oE '[0-9]\.[0-9]')/site-packages/* $(BUILD_FOLDER)/hot/ cp -r $(PROJECT_MODULE_NAME) $(BUILD_FOLDER)/hot/$(PROJECT_MODULE_NAME) cp *.toml $(BUILD_FOLDER)/hot watch: bin/watchman.sh $(PROJECT_MODULE_NAME) "make build-hot" .PHONY: build-hot watch ``` To run the example above, run `make watch`. The script is copying the project module `PROJECT_MODULE_NAME` along with all dependencies into the `build/hot` folder, which is then mounted into LocalStack's Lambda container. ### Hot reloading for TypeScript Lambdas You can hot-reload your [TypeScript Lambda functions](https://docs.aws.amazon.com/lambda/latest/dg/lambda-typescript.html). You can use the following options to build your TypeScript code: * ESbuild * Webpack #### ESbuild We will check-out a simple example to create a simple `Hello World!` Lambda function using TypeScript and ESbuild. ##### Setting up the Lambda function Create a new Node.js project with `npm` or an alternative package manager: ```bash npm init -y ``` Install the the [@types/aws-lambda](https://www.npmjs.com/package/@types/aws-lambda) and [esbuild](https://esbuild.github.io/) packages in your Node.js project: ```bash npm install -D @types/aws-lambda esbuild ``` Create a new file named `index.ts`. Add the following code to the new file: ```ts showLineNumbers import { Context, APIGatewayProxyResult, APIGatewayEvent } from 'aws-lambda'; export const handler = async (event: APIGatewayEvent, context: Context): Promise => { console.log(`Event: ${JSON.stringify(event, null, 2)}`); console.log(`Context: ${JSON.stringify(context, null, 2)}`); return { statusCode: 200, body: JSON.stringify({ message: 'Hello World!', }), }; }; ``` Add a build script to your `package.json` file: ```json title="package.json" "scripts": { "build": "esbuild index.ts --bundle --minify --sourcemap --platform=node --target=es2020 --outfile=dist/index.js --watch" }, ``` The build script will use `esbuild` to bundle and minify the TypeScript code into a single JavaScript file, which will be placed in the `dist` folder. You can now run the build script to create the `dist/index.js` file: ```bash npm run build ``` ##### Creating the Lambda Function with ESbuild To create the Lambda function, you need to take care of two things: * Deploy via an S3 Bucket. You need to use the magic variable `hot-reload` as the bucket. * Set the S3 key to the path of the directory your lambda function resides in. The handler is then referenced by the filename of your lambda code and the function in that code that needs to be invoked. Create the Lambda Function using `lstk aws`: ```bash lstk aws lambda create-function \ --function-name hello-world \ --runtime "nodejs16.x" \ --role arn:aws:iam::123456789012:role/lambda-ex \ --code S3Bucket="hot-reload",S3Key="/absolute/path/to/dist" \ --handler index.handler ``` You can quickly make sure that it works by invoking it with a simple payload: ```bash lstk aws lambda invoke \ --function-name hello-world \ --cli-binary-format raw-in-base64-out \ --payload '{"action": "test"}' \ output.txt ``` The invocation returns itself returns: ```bash title="Output" { "StatusCode": 200, "ExecutedVersion": "$LATEST" } ``` The `output.txt` file contains the following: ```text {"statusCode":200,"body":"{\"message\":\"Hello World!\"}"} ``` ##### Changing the Lambda Function The Lambda function is now mounted as a file in the executing container, hence any change that we save on the file will be there in an instant. Change the `Hello World!` message to `Hello LocalStack!` and run `npm run build`. Trigger the Lambda once again. You will see the following in the `output.txt` file: ```text {"statusCode":200,"body":"{\"message\":\"Hello LocalStack!\"}"} ``` #### Webpack In this example, you can use our public [Webpack example](https://github.com/localstack-samples/localstack-pro-samples/tree/master/lambda-hot-reloading/lambda-typescript-webpack) to create a simple Lambda function using TypeScript and Webpack. To use the example, run the following commands: ```bash cd /tmp git clone https://github.com/localstack-samples/localstack-pro-samples.git cd lambda-hot-reloading/lambda-typescript-webpack ``` ##### Setting up the build Before you can build the Lambda function, you need to install the dependencies: ```bash yarn install ``` Next, you can build the Lambda function: ```bash yarn run build ``` The `build` script in the `package.json` file uses Nodemon to watch for changes in the `src` directory and rebuild the Lambda. This is enabled using the [`nodemon-webpack-plugin`](https://www.npmjs.com/package/nodemon-webpack-plugin) plugin, which has been pre-configured in the `webpack.config.js` file. ##### Creating the Lambda Function with Webpack You can now create the Lambda function using `lstk aws`: ```bash lstk aws lambda create-function \ --function-name localstack-example \ --runtime nodejs18.x \ --role arn:aws:iam::000000000000:role/lambda-ex \ --code S3Bucket="hot-reload",S3Key="/absolute/path/to/dist" \ --handler api.default ``` Additionally, you can create a Lambda Function URL with the following command: ```bash function_url=$(lstk aws lambda create-function-url-config \ --function-name localstack-example \ --auth-type NONE | jq -r '.FunctionUrl') ``` ##### Trigger the Hot Reload Before triggering the Lambda function, you can check the current response by running the following command: ```bash curl -X GET "$function_url" ``` ```bash title="Output" {"error":"Only JSON payloads are accepted"} ``` Go to `src/api.ts` and make the `errorResponse` function return `"Only JSON payload is accepted"` instead of `"Only JSON payloads are accepted"`. Save the file and run the last `curl` command again. The output should now be: ```bash title="Output" {"error":"Only JSON payload is accepted"} ``` You can now see that the changes are applied without redeploying the Lambda function. ## Deployment Configuration Examples ```yaml showLineNumbers custom: localstack: ... lambda: mountCode: true # or if you need to enable code mounting only for specific stages custom: stages: local: mountCode: true testing: mountCode: false localstack: stages: - local - testing lambda: mountCode: ${self:custom.stages.${opt:stage}.mountCode} ``` ```kotlin showLineNumbers package org.localstack.cdkstack import java.util.UUID import software.amazon.awscdk.core.Construct import software.amazon.awscdk.core.Duration import software.amazon.awscdk.core.Stack import software.amazon.awscdk.services.lambda.* import software.amazon.awscdk.services.s3.Bucket private val STAGE = System.getenv("STAGE") ?: "local" private val LAMBDA_MOUNT_CWD = System.getenv("LAMBDA_MOUNT_CWD") ?: "" private const val JAR_PATH = "build/libs/localstack-sampleproject-all.jar" class ApplicationStack(parent: Construct, name: String) : Stack(parent, name) { init { val lambdaCodeSource = this.buildCodeSource() SingletonFunction.Builder.create(this, "ExampleFunctionOne") .code(lambdaCodeSource) .handler("org.localstack.sampleproject.api.LambdaApi") .environment(mapOf("FUNCTION_NAME" to "functionOne")) .timeout(Duration.seconds(30)) .runtime(Runtime.JAVA_11) .uuid(UUID.randomUUID().toString()) .build() } /** * Mount code for hot-reloading when STAGE=local */ private fun buildCodeSource(): Code { if (STAGE == "local") { val bucket = Bucket.fromBucketName(this, "HotReloadingBucket", "hot-reload") return Code.fromBucket(bucket, LAMBDA_MOUNT_CWD) } return Code.fromAsset(JAR_PATH) } } ``` ```hcl showLineNumbers variable "STAGE" { type = string default = "local" } variable "AWS_REGION" { type = string default = "us-east-1" } variable "JAR_PATH" { type = string default = "build/libs/localstack-sampleproject-all.jar" } variable "LAMBDA_MOUNT_CWD" { type = string } provider "aws" { access_key = "test_access_key" secret_key = "test_secret_key" region = var.AWS_REGION s3_force_path_style = true skip_credentials_validation = true skip_metadata_api_check = true endpoints { apigateway = var.STAGE == "local" ? "http://localhost:4566" : null cloudformation = var.STAGE == "local" ? "http://localhost:4566" : null cloudwatch = var.STAGE == "local" ? "http://localhost:4566" : null cloudwatchevents = var.STAGE == "local" ? "http://localhost:4566" : null iam = var.STAGE == "local" ? "http://localhost:4566" : null lambda = var.STAGE == "local" ? "http://localhost:4566" : null s3 = var.STAGE == "local" ? "http://localhost:4566" : null } } resource "aws_iam_role" "lambda-execution-role" { name = "lambda-execution-role" assume_role_policy = < You can then pass `LAMBDA_MOUNT_CWD` as an environment variable to your deployment tool. ```bash LAMBDA_MOUNT_CWD=$(pwd)/build/hot serverless deploy --stage local ``` ```bash STAGE=local && LAMBDA_MOUNT_CWD=$(pwd)/build/hot && lstk cdk bootstrap aws://000000000000/$(AWS_REGION) && \ lstk cdk deploy ``` ```bash terraform init && \ terraform apply -var "STAGE=local" -var "LAMBDA_MOUNT_CWD=$(pwd)/build/hot" ``` ## Share deployment configuration between different machines The paths provided for hot reloading have to be absolute paths on the host running the LocalStack container. This, however makes sharing the same configuration between multiple machines difficult, whether using [Cloud Pods](/aws/developer-tools/snapshots/cloud-pods) or sharing IaC templates between different developers. In order to remove the need for manual adjustments for your hot-reloading paths specified in the `S3Key` field, you can use placeholders for environment variables inside the path. The placeholders use the same format as you would use for shell parameter expansion, namely `$ENV_VAR` or `${ENV_VAR}`. These used environment variables have to be set inside the LocalStack container. Please note that the final path, after substituting the placeholders for their values, has to be an absolute path. :::note Please make sure the placeholder is not substituted by your shell before being sent to LocalStack. This is mostly relevant when using the AWS CLI to create a function. Please use string quotation marks which prevent parameter expansion in your shell. For bash, please use single quotes `'` instead of double quotes `"` to make sure the placeholder does not get expanded before being sent to LocalStack. ::: ### Example In order to make use of the environment variable placeholders, you can inject them into the LocalStack container, for example using the following `docker-compose.yml` file. ```yaml showLineNumbers services: localstack: container_name: "${LOCALSTACK_DOCKER_NAME:-localstack-main}" image: localstack/localstack ports: - "127.0.0.1:4566:4566" # LocalStack Gateway - "127.0.0.1:4510-4559:4510-4559" # external services port range environment: # LocalStack configuration: https://docs.localstack.cloud/references/configuration/ - DEBUG=${DEBUG:-0} - HOST_LAMBDA_DIR=${PWD} volumes: - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" ``` This will set a `HOST_LAMBDA_DIR` environment variable to the current working directory when creating the Docker Compose stack. Please note that this environment variable name is arbitrary - you can use any you want, but need to refer to that variable in your templates or commands to deploy your function correctly. You can then deploy a hot-reloading function with the following command: ```bash lstk aws lambda create-function \ --function-name test-function \ --code S3Bucket=hot-reload,S3Key='$HOST_LAMBDA_DIR/src' \ --handler handler.handler \ --runtime python3.12 \ --role 'arn:aws:iam::000000000000:role/lambda-ex' ``` Please note the single quotes `'` which prevent our shell to replace `$HOST_LAMBDA_DIR` before the function is created. With the above example, you can make hot-reloading paths sharable between machines, as long as there is a point on the host to which the relative paths will stay the same. One example for this are checked out git repositories, where the code is located in the same structure - the absolute location of the checked out repository on the machine might however differ. If the chosen variable always points to the checked out directory, you can set the path using the placeholder in the checked out IaC template, or can share a Cloud Pod between machines. ## Hot Reloading on LocalStack Web Application You can use the LocalStack Web Application to hot reload your Lambda functions. The [Lambda Resource Browser](https://app.localstack.cloud/inst/default/resources/lambda/functions) allows you to update your Lambda function and specify the file path for your Lambda code and dependencies. To set up Lambda Hot Reloading via the LocalStack Web Application: 1. Navigate to the [Lambda Resource Browser](https://app.localstack.cloud/inst/default/resources/lambda/functions). 2. Click **Create** to create a new Lambda function, or select **Update Function Code** for an existing Lambda function. 3. In the **Code Source** section, choose **Hot Reload**. 4. Enter the path to the directory containing your Lambda code for hot reloading. 5. Click **Submit** to save the configuration. LocalStack will automatically set up the magic S3 bucket and the S3 key pointing to your specified file path. Changes to your Lambda code locally will be reflected immediately upon saving. ![Setting Hot Reload on Web App](/images/aws/hot-reload-lambda-web-app.png) ## Examples - [Lambda Hot Reloading](https://github.com/localstack/localstack-pro-samples/tree/master/lambda-hot-reloading) # Remote Debugging > Attach a debugger to your Lambda functions from within your IDE. import { Tabs, TabItem } from '@astrojs/starlight/components'; import { Badge } from '@astrojs/starlight/components'; # Overview Lambda Remote Debugging lets you use breakpoints, inspect variables, and step through your Lambda function code locally within VS Code. It is supported for Python, Node.js, and Java, and works with SAM-based or standalone Lambda projects. :::note For examples and sample apps, visit the [LocalStack Samples Repository](https://github.com/localstack-samples/localstack-pro-samples): - [Debug your Python Lambda Function](https://github.com/localstack-samples/localstack-pro-samples/tree/master/lambda-debugging-sam-python) - [Debug your JavaScript Lambda Function](https://github.com/localstack-samples/localstack-pro-samples/tree/master/lambda-debugging-sam-javascript) - [Debug your TypeScript Lambda Function](https://github.com/localstack-samples/localstack-pro-samples/tree/master/lambda-debugging-sam-typescript) - [Debug your Java Lambda Function](https://github.com/localstack-samples/localstack-pro-samples/tree/master/lambda-debugging-sam-java) ::: ## Lambda Remote Debugging with AWS Toolkit for VS Code This guide describes how to use the AWS Toolkit for VS Code to debug Lambda functions running in LocalStack. This new integration enables interactive, IDE-native debugging for Python, Node.js, and Java Lambda functions with minimal setup. ### Key Benefits * One-click remote automatic debugger configuration and injection via the AWS Toolkit for VS Code * Automatic Timeout Management * Support for Python, Node.js, and Java runtimes ## Getting Started with AWS Toolkit (VS Code) ### Prerequisites * Install the [`lstk` CLI](/aws/developer-tools/running-localstack/lstk) * [VS Code](https://code.visualstudio.com/) (>= v1.83.0) * [AWS Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.aws-toolkit-vscode) (>= v3.74) * [LocalStack Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=LocalStack.localstack) (>= v1.2.0) * [Docker](https://www.docker.com/) * A valid LocalStack Auth Token. Sign up for a [free LocalStack account](https://www.localstack.cloud/pricing). * A valid **auth token** * VS Code running on the **same machine** as LocalStack (container-based setups like Kubernetes are not yet supported). ### Setup Steps The following setup creates the required `~/.aws/config` and `~/.aws/credentials` entries and ensures LocalStack runs with your active license. 1. Install [AWS Toolkit](https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.aws-toolkit-vscode) and [LocalStack Toolkit](https://marketplace.visualstudio.com/items?itemName=LocalStack.localstack) from the VS Code Marketplace. 2. Install the latest version of the [AWS SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html) for using the AWS Application builder functionality in the AWS Toolkit. 3. Run the `LocalStack: Run Setup Wizard` from the Command Palette. (This sets your auth token and configures the `localstack` AWS profile.) ![LocalStack: Run Setup Wizard](/images/aws/lambda-remote-debugging/wizard.png) 4. Start LocalStack using the status bar or the command `Start LocalStack`. 5. Switch your AWS profile in the status bar to `profile:localstack`. ![LocalStack status bar](/images/aws/lambda-remote-debugging/status-bar.png) ### Debugging Node.js, Python, Java (with Examples) You can debug either your own Lambda function or our sample project. ### Use AWS Toolkit Sample 1. Open the **AWS Explorer** in VS Code. ![AWS Explorer](/images/aws/lambda-remote-debugging/explorer.png) 2. To install the sample application, select the `...` menu in the AWS Explorer view and choose *Create application with Serverless template*. ![Create app](/images/aws/lambda-remote-debugging/create-app.png) 3. Choose the **"Process SQS Records with Lambda"** sample. ![Process SQS Records with Lambda](/images/aws/lambda-remote-debugging/choose-sqs-records-lambda.png) 4. Select a runtime (Python, Node.js, or Java). 5. Complete the wizard to generate the project. :::note If you want to use your own Lambda function, ensure your function and SAM template are compatible with LocalStack. If using our sample app, you may need to change the Lambda architecture from `arm64` to `x86_64` in `template.yaml` if running on an Intel machine. ::: ### Deploy and Debug 1. Deploy the function: * Go to Application Builder view * Click **Deploy SAM Application** (cloud icon) ![Deploy SAM Application](/images/aws/lambda-remote-debugging/deploy-sam.png) * Follow the wizard to deploy to LocalStack 2. Open Remote Invoke Configuration: * In **AWS Explorer**, right-click your deployed Lambda function * Select **Invoke Remotely** 3. Enable Remote Debugging: * In the configuration view, check **Remote debugging** ![Remote Invoke Config](/images/aws/lambda-remote-debugging/remote-invoke-config.png) ![Expanded Remote Debugging](/images/aws/lambda-remote-debugging/expanded-remote-debugging.png) * If prompted, provide the event input (e.g., SQS sample event) 4. Set Breakpoints and Debug: * Open your Lambda handler source file * Set breakpoints in the margin * Click **Remote Invoke** Your Lambda function will execute in LocalStack and pause at your breakpoint. You can inspect variables, step through code, and continue execution. ### Debugging TypeScript with Source Maps To enable source-level debugging for TypeScript: 1. Build your TypeScript Lambda with source maps enabled: ```bash tsc --sourceMap true ``` 2. Ensure the compiled `.js` and `.map` files are present in the deployment package. 3. Open the `.ts` source file in VS Code, set breakpoints, and proceed as with other runtimes. ## Troubleshooting ### `UnsupportedLocalStackVersion` Error If you receive the following error message: ``` UnsupportedLocalStackVersion: Your current LocalStack version does not support Lambda remote debugging. Update LocalStack and check your license. ``` Here's how to fix it: 1. Make sure you are running **LocalStack for AWS >= v4.8.0** ```bash docker pull localstack/localstack-pro:latest ``` 2. Ensure your **auth token is valid** and has an **assigned license**. * You can re-run the **LocalStack Setup Wizard** in VS Code to verify. * Or log into [app.localstack.cloud](https://app.localstack.cloud) and check your token and [license status](https://docs.localstack.cloud/aws/licensing/). ### DNS Rebind Protection Issues Should downloading the Lambda function code fail, here's how to fix it: * Disable DNS rebind protection (see [LocalStack DNS docs](https://docs.localstack.cloud/aws/customization/networking/dns-server/#dns-rebind-protection)) * Use `127.0.0.1` instead of `localhost.localstack.cloud` (**Automatic** setup) * Run the `LocalStack: Configure AWS Profile "localstack"` command from the VS Code Command Palette which will auto-configure the correct endpoint. * Use `127.0.0.1` instead of `localhost.localstack.cloud` (**Manual** setup) * Set `LOCALSTACK_HOST=127.0.0.1:4566` in `~/.localstack/default.env` * Update your `endpoint_url` in your `localstack` profile located in `~/.aws/config` ## Lambda Debug Mode (Preview) Lambda Debug Mode is a preview feature in LocalStack designed to enhance your Lambda debugging workflows. This feature provides an optimized environment for debugging Lambda functions, ensuring that you have the necessary tools and flexibility to troubleshoot effectively. ### Key Features * **Automatic Timeout Management**: Integrates with API Gateway to prevent Lambda function timeouts, giving developers ample time to connect remote debuggers and inspect the function's behavior. * **Multi-Function Debugging**: Supports debugging multiple Lambda functions concurrently. ### Enabling Lambda Debug Mode To enable Lambda Debug Mode, set the `LAMBDA_DEBUG_MODE` environment variable as shown below: ```toml # .lstk/config.toml [[containers]] type = "aws" env = ["lambda-debug"] [env.lambda-debug] LAMBDA_DEBUG_MODE = "1" LAMBDA_DOCKER_FLAGS = "-p 19891:19891" ``` ```bash lstk start ``` When enabled, Lambda Debug Mode automatically adjusts timeouts to accommodate debugging needs: * **Lambda Container Startup Timeout**: Provides additional time for debugger connection during container creation. * **Lambda Execution Timeout**: Extends the execution window, allowing for in-depth remote debugging. * **API Gateway-Lambda Integration Timeout**: Increases timeout settings to avoid premature terminations. ### Advanced Configuration For further customization, you can use a configuration file. Specify the path to this file with the `LAMBDA_DEBUG_MODE_CONFIG_PATH` environment variable, ensuring the file is mounted into the LocalStack container. Manually setting `LAMBDA_DOCKER_FLAGS` is unnecessary when using this configuration. Here is an example of mounting a `debug_config.yaml` in your LocalStack container to start your Debug Mode: ```toml # .lstk/config.toml [[containers]] type = "aws" env = ["lambda-debug-advanced"] volumes = ["/path/to/debug-config.yaml:/tmp/lambda_debug_mode_config.yaml"] [env.lambda-debug-advanced] LAMBDA_DEBUG_MODE = "1" LAMBDA_DEBUG_MODE_CONFIG_PATH = "/tmp/debug_config.yaml" ``` ```bash lstk start ``` ```yaml showLineNumbers services: localstack: container_name: "${LOCALSTACK_DOCKER_NAME:-localstack-main}" image: localstack/localstack-pro ports: - "127.0.0.1:4566:4566" # LocalStack Gateway - "127.0.0.1:4510-4559:4510-4559" # external services port range - "127.0.0.1:443:443" # LocalStack HTTPS Gateway (Pro) environment: # LocalStack configuration: https://docs.localstack.cloud/references/configuration/ - DEBUG=${DEBUG:-0} - LAMBDA_DEBUG_MODE=1 - LAMBDA_DEBUG_MODE_CONFIG_PATH=/tmp/debug_config.yaml volumes: - "./debug_config.yaml:/tmp/debug_config.yaml" - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" ``` Any change to the configuration file on your local filesystem would be automatically picked by the LocalStack container. After debugging a Lambda function, its associated container will automatically stop. The configuration file should contain a `functions` block where you can define debug settings for each specific Lambda function ARN. #### Example: Basic Debugging Configuration This example configures Lambda Debug Mode to use port 19891 for the remote debugger. ```yaml showLineNumbers functions: arn:aws:lambda:eu-central-1:000000000000:function:func-one: debug-port: 19891 ``` #### Example: Disabling Automatic Timeout Handling In this example, the automatic timeout handling feature is disabled for the specified Lambda function, enforcing the predefined timeouts instead. ```yaml showLineNumbers functions: arn:aws:lambda:eu-central-1:000000000000:function:func-one: debug-port: 19891 enforce-timeouts: true ``` ### Handling Unqualified ARNs Specifying an unqualified Lambda ARN in the configuration is equivalent to specifying the ARN with the `$LATEST` version qualifier. ```yaml showLineNumbers functions: arn:aws:lambda:eu-central-1:000000000000:function:func-one:$LATEST: debug-port: 19891 ``` ### Debugging Multiple Functions To debug multiple Lambda functions simultaneously, assign a different debug port to each function. Note that this configuration affects the container's internal debugger port as well, so the debugger port must be set accordingly. ```yaml showLineNumbers functions: arn:aws:lambda:eu-central-1:000000000000:function:func-one: debug-port: 19891 arn:aws:lambda:eu-central-1:000000000000:function:func-two: debug-port: 19892 ``` ### Debugging Different Versions You can also debug different versions of the same Lambda function by assigning unique ports to each version. ```yaml showLineNumbers functions: arn:aws:lambda:eu-central-1:000000000000:function:func-one:1: debug-port: 19891 arn:aws:lambda:eu-central-1:000000000000:function:func-two:2: debug-port: 19892 ``` ## Manual Lambda Debugging (Legacy) * [Debugging Python Lambda functions](#debugging-python-lambda-functions) * [Debugging JVM Lambda functions](#debugging-jvm-lambda-functions) * [Debugging Node.js Lambda functions](#debugging-nodejs-lambda-functions) :::note Due to the ports published by the Lambda container for the debugger, it is currently only possible to debug one Lambda function at a time. This limitation only applies to advanced manual debugging scenarios in legacy mode, such as those requiring multiple ports. ::: ### Debugging Python Lambda functions Lambda functions debugging used to be a difficult task. LocalStack changes that with the same local code mounting functionality that also helps you to [iterate quickly over your function code](/aws/developer-tools/lambda-tools/hot-reloading). ### Debugging a Python Lambda in Visual Studio Code #### Configure LocalStack for VS Code remote Python debugging First, make sure that LocalStack is started with the following configuration (see the [Configuration docs](/aws/customization/configuration-options#lambda) for more information): ```toml # .lstk/config.toml [[containers]] type = "aws" env = ["lambda-debug"] [env.lambda-debug] LAMBDA_DOCKER_FLAGS = "-p 19891:19891" ``` ```bash lstk start ``` #### Preparing your code For providing the debug server, we use [`debugpy`](https://github.com/microsoft/debugpy) inside the Lambda function code. In general, all you need is the following code fragment placed inside your handler code: ```python showLineNumbers import debugpy debugpy.listen(("0.0.0.0", 19891)) debugpy.wait_for_client() # blocks execution until client is attached ``` For extra convenience, you can use the `wait_for_debug_client` function from our example. It implements the above-mentioned start of the debug server and also adds an automatic cancellation of the wait task if the debug client (i.e. VSCode) doesn't connect. ```python showLineNumbers def wait_for_debug_client(timeout=15): """Utility function to enable debugging with Visual Studio Code""" import time, threading import sys, glob sys.path.append(glob.glob(".venv/lib/python*/site-packages")[0]) import debugpy debugpy.listen(("0.0.0.0", 19891)) class T(threading.Thread): daemon = True def run(self): time.sleep(timeout) print("Canceling debug wait task ...") debugpy.wait_for_client.cancel() T().start() print("Waiting for client to attach debugger ...") debugpy.wait_for_client() ``` #### Configuring Visual Studio Code for remote Python debugging For attaching the debug server from Visual Studio Code, you need to add a run configuration. ```json showLineNumbers { "version": "0.2.0", "configurations": [ { "name": "Python: Remote Attach", "type": "python", "request": "attach", "connect": { "host": "localhost", "port": 19891 }, "pathMappings": [ { "localRoot": "${workspaceFolder}", "remoteRoot": "." } ] } ] } ``` In the next step we create our function. In order to debug the function in Visual Studio Code, run the preconfigured remote debugger, which will wait about 15 seconds as defined above, and then invoke the function. Make sure to set a breakpoint in the Lambda handler code first, which can then later be inspected. The screenshot below shows the triggered breakpoint with our `'Hello from LocalStack!'` in the variable inspection view: ![Visual Studio Code debugging](/images/aws/vscode-debugging-py-1.png) #### Current Limitations Due to the ports published by the lambda container for the debugger, you can currently only debug one Lambda at a time. Due to the port publishing, multiple concurrently running lambda environments are not supported. ### Debugging a Python Lambda in PyCharm Professional Please be aware that [remote debugging in PyCharm](https://www.jetbrains.com/help/pycharm/remote-debugging-with-product.html) is only available in the Professional version. You do not need to change the `LAMBDA_DOCKER_FLAGS` when debugging with PyCharm Professional. #### Configuring PyCharm for remote Python debugging You can [follow the steps in the official docs](https://www.jetbrains.com/help/pycharm/remote-debugging-with-product.html#remote-debug-config), which will come down to: * Create a debug configuration with the IDE host name `localhost` and the debug port `19891`. * Add path mapping with your project files on the host and map it to the remote directory `/var/task`. * Copy the `pip install` command, and make sure to install the correct `pydevd-pycharm` version for your PyCharm IDE. ![PyCharm Professional Remote Debugging Configuration](/images/aws/pycharm_remote_debugging.png) #### Preparing your code PyCharm provides its own debugging package, called `pydevd-pycharm`. Essentially, you will add the following code to your lambda: ```python import pydevd_pycharm pydevd_pycharm.settrace('host.docker.internal', port=19891, stdoutToServer=True, stderrToServer=True) ``` The `host.docker.internal` is a [special DNS name by Docker](https://docs.docker.com/desktop/networking/#use-cases-and-workarounds-for-all-platforms) and will make sure that the lambda running in the docker can connect to PyCharm running on your Localhost. You can use the `wait_for_debug_client` and add it to your lambda (please adapt the path to your `venv` directory if necessary): ```python def wait_for_debug_client(): """Utility function to enable debugging with PyCharm""" import sys, glob # enter the correct path here to your venv (where pydev_pycharm is installed my_venv = "venv/lib/python*/site-packages" sys.path.insert(0, glob.glob(my_venv)[0]) import pydevd_pycharm # host.docker.internal should resolve to the host # see also: https://docs.docker.com/desktop/networking#use-cases-and-workarounds-for-all-platforms pydevd_pycharm.settrace('host.docker.internal', port=19891, stdoutToServer=True, stderrToServer=True) ``` In the next step we create our function. In order to debug the function in PyCharm set a breakpoint in your function, run the Remote Debug configuration and then invoke the function. ![PyCharm Professional debugging](/images/aws/pycharm_lambda_debugging.png) ### Creating the Lambda function To create the Lambda function, you just need to take care of two things: 1. Deploy the function via an S3 Bucket. You need to use the magic variable `hot-reload` as the bucket name. 2. Set the S3 key to the path of the directory your lambda function resides in. The handler is then referenced by the filename of your lambda code and the function in that code that should be invoked. Using the AWS CLI, this would be: ```bash lstk aws lambda create-function --function-name my-cool-local-function \ --code S3Bucket="hot-reload",S3Key="$(pwd)/" \ --handler handler.handler \ --runtime python3.13 \ --timeout 150 \ --role arn:aws:iam::000000000000:role/lambda-role ``` We can quickly verify that it works by invoking it with a simple payload: ```bash lstk aws lambda invoke --function-name my-cool-local-function \ --cli-binary-format raw-in-base64-out \ --payload '{"message": "Hello from LocalStack!"}' \ output.txt ``` ### Debugging JVM Lambda functions ### Configure LocalStack and your Lambda function for remote JVM debugging Set `LAMBDA_DOCKER_FLAGS` to export the `5050` (you can use any other port of your choice) port which your IDE debugger will connect to. ```yaml #docker-compose.yml services: localstack: ... environment: ... - LAMBDA_DOCKER_FLAGS=-p 127.0.0.1:5050:5050 ``` When creating your Lambda function, set the `_JAVA_OPTIONS` environment variable like so: ```bash lstk aws lambda create-function --function-name debugfunc \ --zip-file fileb://java-handler.zip \ --handler myindex.handler \ --runtime java8.al2 \ --timeout 150 \ --role arn:aws:iam::000000000000:role/lambda-role \ --environment '{"Variables": {"_JAVA_OPTIONS": "-Xshare:off -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=0.0.0.0:5050"}}' ``` Note the `suspend=y` option here, it will delay code execution until the debugger is attached to the debugger server. If you want to change that, simply switch to `suspend=n`. By default the runtime environment for Java will set `-Xshare: on`, so we'll have to disable it here again. Your IDE might show you the listen address as `*:5050`, but please note that this only works for Java 9+. ### Configuring IntelliJ IDEA for remote JVM debugging Open the `Run/Debug Configurations` window and create a new `Shell Script` with the following content: ```shell while [[ -z $(docker ps | grep :5050) ]]; do sleep 1; done ``` ![Run/Debug Configurations](/images/aws/inteliji-debugging-jvm-1.png) This shell script should simplify the process a bit since the debugger server is not immediately available (only once Lambda container is up). Then create a new `Remote JVM Debug` configuration and use the script from above as a `Before launch` target: ![Run/Debug Configurations](/images/aws/inteliji-debugging-jvm-2.png) Now to debug your Lambda function, simply click on the `Debug` icon with `Remote JVM on LS Debug` configuration selected, and then invoke your Lambda function. ### Alternative setup for IntelliJ IDEA The debugger can also act as a server by changing the drop-down "Debugger mode" to "Listen to remote JVM". In this case you should not set `LAMBDA_DOCKER_FLAGS` since the port will be exposed on your host instead of the Lambda container. Compared to the previous setup the "Wait Remote Debugger Server" run configuration should also be removed and instead tick the mark at "Auto restart" after switching to the "Listen to remote JVM" mode. For the Lambda function you will have to adjust the environment variable to `"_JAVA_OPTIONS": "-Xshare:off -agentlib:jdwp=transport=dt_socket,server=n,address=172.17.0.1:5050,suspend=y,onuncaught=n"`. Notice the `address=172.17.0.1:5050`. Here we tell the Lambda function to connect to port 5050 on 172.17.0.1. When using Docker desktop you might have to set this to `address=host.docker.internal:5050` instead. ### Configuring Visual Studio Code for remote JVM debugging Make sure you installed the following extensions: * [Language Support for Java(TM) by Red Hat](https://marketplace.visualstudio.com/items?itemName=redhat.java) * [Debugger for Java](https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-java-debug) Add a new task by creating/modifying the `.vscode/tasks.json` file: ```json showLineNumbers { "version": "2.0.0", "tasks": [ { "label": "Wait Remote Debugger Server", "type": "shell", "command": "while [[ -z $(docker ps | grep :5050) ]]; do sleep 1; done; sleep 1;" } ] } ``` Create a new `launch.json` file or edit an existing one from the `Run and Debug` tab, then add the following configuration: ```json showLineNumbers { "version": "0.2.0", "configurations": [ { "type": "java", "name": "Remote JVM on LS Debug", "projectRoot": "${workspaceFolder}", "request": "attach", "hostName": "localhost", "preLaunchTask": "Wait Remote Debugger Server", "port": 5050 } ] } ``` Now to debug your lambda function, click on the `Debug` icon with `Remote JVM on LS Debug` configuration selected, and then invoke your lambda function. ### Debugging Node.js Lambda functions ### Configure LocalStack for remote Node.js debugging Set the `LAMBDA_DOCKER_FLAGS` to enable the debugger using `NODE_OPTIONS`: ```yaml showLineNumbers #docker-compose.yml services: localstack: ... environment: ... - LAMBDA_DOCKER_FLAGS=-e NODE_OPTIONS=--inspect-brk=0.0.0.0:9229 -p 9229:9229 ``` ### Configuring Visual Studio Code for remote Node.js debugging Add a new task by creating/modifying the `.vscode/tasks.json` file: ```json showLineNumbers { "version": "2.0.0", "tasks": [ { "label": "Wait Remote Debugger Server", "type": "shell", "command": "while [[ -z $(docker ps | grep :9229) ]]; do sleep 1; done; sleep 1;" } ] } ``` Create a new `launch.json` file or edit an existing one from the `Run and Debug` tab, then add the following configuration: ```json showLineNumbers { "version": "0.2.0", "configurations": [ { "address": "127.0.0.1", "localRoot": "${workspaceFolder}", "name": "Attach to Remote Node.js", "port": 9229, "remoteRoot": "/var/task/", "request": "attach", "type": "node", "preLaunchTask": "Wait Remote Debugger Server" }, ] } ``` A simple example of a Node.js lambda, `myindex.js` could look like this: ```js showLineNumbers exports.handler = async (event) => { console.log(event); const response = { statusCode: 200, body: "ok", }; return response; }; ``` Create the lambda function using: ```bash lstk aws lambda create-function --function-name func1 \ --code S3Bucket="hot-reload",S3Key="$(pwd)/" \ --handler myindex.handler \ --runtime nodejs14.x \ --timeout 150 \ --role arn:aws:iam::000000000000:role/lambda-role ``` Now to debug your lambda function, click on the `Debug` icon with `Attach to Remote Node.js` configuration selected, and then invoke your lambda function: ```bash lstk aws lambda invoke --function-name func1 \ --cli-binary-format raw-in-base64-out \ --payload '{"hello":"world"}' \ output.txt ``` ## Examples - [Debug your Python Lambda Function](https://github.com/localstack-samples/localstack-pro-samples/tree/master/lambda-debugging-sam-python) - [Debug your JavaScript Lambda Function](https://github.com/localstack-samples/localstack-pro-samples/tree/master/lambda-debugging-sam-javascript) - [Debug your TypeScript Lambda Function](https://github.com/localstack-samples/localstack-pro-samples/tree/master/lambda-debugging-sam-typescript) - [Debug your Java Lambda Function](https://github.com/localstack-samples/localstack-pro-samples/tree/master/lambda-debugging-sam-java) # Overview > Tools for starting, stopping, and managing your LocalStack emulator. import SectionCards from '../../../../../components/SectionCards.astro'; Several tools are available to install, start, and manage the LocalStack emulator. Your starting point should be `lstk`, the modern CLI for managing LocalStack. The MCP server is also available for AI-driven management, giving you complete control over the lifecycle of your local cloud environment. Note that the previous `localstack` CLI is now deprecated, but can [still be used](/aws/developer-tools/running-localstack/localstack-cli/) if necessary. # Deprecated LocalStack CLI > Reference guide for the deprecated LocalStack CLI commands, options, and usage. We recommend using lstk instead. import { Tabs, TabItem } from '@astrojs/starlight/components'; import { LinkButton, Code } from '@astrojs/starlight/components'; import { LOCALSTACK_AWS_VERSION } from 'astro:env/server'; ## Introduction The LocalStack Command Line Interface (CLI) is a tool for starting, managing, and configuring your LocalStack container. It provides convenience features to interact with LocalStack features like Cloud Pods, Extensions, State Management, and more. :::caution The LocalStack CLI is deprecated and will be removed in a future version. Please use [lstk](/aws/developer-tools/running-localstack/lstk/) instead. ::: ## Installation You can download the pre-built binary for your architecture using the link below: {' '} x86-64 ARM64 or use the curl commands below: For x86-64: For ARM64: Then extract the LocalStack CLI from the terminal:
Alternative: Homebrew on Linux If you are using [Homebrew for Linux](https://docs.brew.sh/Homebrew-on-Linux), you can install the LocalStack CLI directly from our official LocalStack tap: ```bash brew install localstack/tap/localstack-cli ```
:::caution macOS x86_64 (Intel) is no longer officially supported for the legacy LocalStack CLI. The `cryptography` package used by the CLI has dropped support for macOS x86_64, so the CLI now pins an older, unsupported version of it as a stopgap to keep working on Intel Macs. If you are on an Intel Mac, we recommend switching to [`lstk`](/aws/developer-tools/running-localstack/lstk/) instead. ::: You can install the LocalStack CLI using Brew directly from our official LocalStack tap: ```bash brew install localstack/tap/localstack-cli ``` You can download the pre-built binary for your architecture using the link below: {' '} Intel (AMD64) Then extract the archive and execute the binary in Powershell. If you cannot use the binary releases of LocalStack, you can install the Python distribution. Please make sure to install the following before moving ahead: - [Python](https://docs.python.org/3/using/index.html) - [pip](https://pip.pypa.io/en/stable/installation/) Next install the LocalStack CLI in your Python environment by running: ```bash python3 -m pip install --upgrade localstack ``` :::note To download a specific version of LocalStack, replace `` with the required version from [changelog page](/aws/changelog). ```bash python3 -m pip install localstack== ``` ::: :::tip[MacOS Sierra?] If you have problems with permissions in MacOS X Sierra, install with: ```bash python3 -m pip install --user localstack ``` ::: :::danger Do not use `sudo` or the `root` user when starting LocalStack. It should be installed and started entirely under a local non-root user. :::
### Starting LocalStack To verify that the LocalStack CLI was installed correctly, you can check the version in your terminal: You are all set! :::note To start LocalStack, you must first [set up your auth token](/aws/getting-started/auth-token). ::: Once you've set up your auth token, you can start LocalStack with the following command: ```bash localstack start # start localstack in background with -d flag ``` {/_ prettier-ignore _/} ### Updating LocalStack CLI The LocalStack CLI allows you to easily update the different components of LocalStack. To check the various options available for updating, run: ```bash localstack update --help ``` ```bash Usage: localstack update [OPTIONS] COMMAND [ARGS]... Update different LocalStack components. Options: -h, --help Show this message and exit. Commands: all Update all LocalStack components docker-images Update docker images LocalStack depends on localstack-cli Update LocalStack CLI ``` :::note Updating the LocalStack CLI using `localstack update localstack-cli` and `localstack update all` will work only if it was installed from the Python distribution. If it was installed using the pre-built binary or via Brew, please run the installation steps again to update to the latest version. ::: :::note This documentation below was auto-generated from LocalStack CLI version `LocalStack CLI 2026.4.0`. [`lstk`](/aws/developer-tools/running-localstack/lstk/) is our new Go-based CLI with an interactive terminal UI for lifecycle (`start`, `stop`), monitoring (`status`, `logs`), storage (`snapshot`), and more. ::: ## Global Options The following global options are available for the `localstack` CLI: | Option | Description | | ---------------------- | ----------------------------- | | `-v`, `--version` | Show the version and exit | | `-d`, `--debug` | Enable CLI debugging mode | | `-p`, `--profile TEXT` | Set the configuration profile | | `-h`, `--help` | Show help message and exit | ## Commands The following commands are available for managing your LocalStack instance. ### `auth` Authenticate with your LocalStack account ```bash Usage: localstack auth [OPTIONS] COMMAND [ARGS]... Authenticate with your LocalStack account. Manage your credentials and authenticate with your LocalStack account. Options: -h, --help Show this message and exit. Commands: clear-token Clear any existing LocalStack auth token from your environment set-token Set your Localstack auth token to allow you to start LocalStack Pro show-token Show the auth token in your configuration Deprecated: login Login to the your LocalStack account (DEPRECATED) logout Log out from your LocalStack account (DEPRECATED) ```
Subcommands for localstack auth #### `auth clear-token` Clear any existing LocalStack auth token from your environment ```bash Usage: localstack auth clear-token [OPTIONS] Options: -h, --help Show this message and exit. ``` #### `auth set-token` Set your Localstack auth token to allow you to start LocalStack ```bash Usage: localstack auth set-token [OPTIONS] AUTH_TOKEN Configure your auth token. Your auth token is used the license activation to activate LocalStack Pro. This is different from `localstack auth login` which enables platform features such as pushing cloud pods to your webapp account. The auth token you configure here will be passed to the `LOCALSTACK_AUTH_TOKEN` environment variable of the LocalStack container when you run `localstack start`. AUTH_TOKEN: Your Localstack auth token that you can find in https://app.localstack.cloud. Options: -h, --help Show this message and exit. ``` #### `auth show-token` Show the auth token in your configuration ```bash Usage: localstack auth show-token [OPTIONS] Show the token that LocalStack picks up from your environment. This can either be the auth token set via `localstack auth set-token`, or the value of `LOCALSTACK_AUTH_TOKEN`. Options: --plain Setting this flag will output only the value of the token in plain text, so it can be used as input to other programs, like `LOCALSTACK_AUTH_TOKEN=$(localstack auth show-token --plain)`. -h, --help Show this message and exit. ``` #### `auth login` (Deprecated) Login to the your LocalStack account (DEPRECATED) ```bash Usage: localstack auth login [OPTIONS] Login to the LocalStack Platform. This command performs a login to your LocalStack account, giving you access to features that require platform permissions, such as uploading cloud pods to your account. This command is deprecated and it will be removed soon. To use LocalStack features that requires authentication to the LocalStack platform (e.g., Cloud Pods), please run `localstack auth set-token `, or set the environment variable `LOCALSTACK_AUTH_TOKEN` to a valid auth token. (DEPRECATED) Options: -u, --username USER Username (email address) for login [required] -p, --password PWD Password for login [required] -h, --help Show this message and exit. ``` #### `auth logout` (Deprecated) Log out from your LocalStack account (DEPRECATED) ```bash Usage: localstack auth logout [OPTIONS] Log out from the LocalStack Platform. This command performs a logout from the LocalStack platform and deletes all session information on your machine. (DEPRECATED) Options: -h, --help Show this message and exit. ```
### `completion` CLI shell completion ```bash Usage: localstack completion [OPTIONS] {bash|zsh|fish} Options: -h, --help Show this message and exit. ``` ### `config` Manage your LocalStack config ```bash Usage: localstack config [OPTIONS] COMMAND [ARGS]... Options: -h, --help Show this message and exit. Commands: show Show your config validate Validate your config ```
Subcommands for localstack config #### `config show` Show your config ```bash Usage: localstack config show [OPTIONS] Options: -f, --format [table|plain|dict|json] The formatting style for the command output. [default: table] -h, --help Show this message and exit. ``` #### `config validate` Validate your config ```bash Usage: localstack config validate [OPTIONS] Options: -f, --file PATH Path to compose file [default: docker-compose.yml] -h, --help Show this message and exit. ```
### `logs` Show LocalStack logs ```bash Usage: localstack logs [OPTIONS] Options: -f, --follow Block the terminal and follow the log output -n, --tail N Print only the last lines of the log output -h, --help Show this message and exit. ``` ### `restart` Restart LocalStack ```bash Usage: localstack restart [OPTIONS] Options: -h, --help Show this message and exit. ``` ### `ssh` Obtain a shell in LocalStack ```bash Usage: localstack ssh [OPTIONS] Options: -h, --help Show this message and exit. ``` ### `start` Start LocalStack ```bash Usage: localstack start [OPTIONS] Options: --docker Start LocalStack in a docker container [default] --host Start LocalStack directly on the host (DEPRECATED) --no-banner Disable LocalStack banner -d, --detached Start LocalStack in the background --network TEXT The container network the LocalStack container should be started in. By default, the default docker bridge network is used. -e, --env TEXT Additional environment variables that are passed to the LocalStack container -p, --publish TEXT Additional port mappings that are passed to the LocalStack container -v, --volume TEXT Additional volume mounts that are passed to the LocalStack container --host-dns Expose the LocalStack DNS server to the host using port bindings. -s, --stack TEXT Use a specific stack with optional version. Examples: [localstack:4.5, snowflake] -h, --help Show this message and exit. ``` ### `status` Query status info ```bash Usage: localstack status [OPTIONS] COMMAND [ARGS]... Options: -h, --help Show this message and exit. Commands: docker Query LocalStack Docker status services Query LocalStack services status ```
Subcommands for localstack status #### `status docker` Query LocalStack Docker status ```bash Usage: localstack status docker [OPTIONS] Options: -f, --format [table|plain|dict|json] The formatting style for the command output. [default: table] -h, --help Show this message and exit. ``` #### `status services` Query LocalStack services status ```bash Usage: localstack status services [OPTIONS] Options: -f, --format [table|plain|dict|json] The formatting style for the command output. [default: table] -h, --help Show this message and exit. ```
### `stop` Stop LocalStack ```bash Usage: localstack stop [OPTIONS] Options: -h, --help Show this message and exit. ``` ### `update` Update LocalStack ```bash Usage: localstack update [OPTIONS] COMMAND [ARGS]... Options: -h, --help Show this message and exit. Commands: all Update all LocalStack components docker-images Update docker images LocalStack depends on localstack-cli Update LocalStack CLI ```
Subcommands for localstack update #### `update all` Update all LocalStack components ```bash Usage: localstack update all [OPTIONS] Options: -h, --help Show this message and exit. ``` #### `update docker-images` Update docker images LocalStack depends on ```bash Usage: localstack update docker-images [OPTIONS] Options: -h, --help Show this message and exit. ``` #### `update localstack-cli` Update LocalStack CLI ```bash Usage: localstack update localstack-cli [OPTIONS] Options: -h, --help Show this message and exit. ```
### `wait` Wait for LocalStack ```bash Usage: localstack wait [OPTIONS] Options: -t, --timeout N Only wait for seconds before raising a timeout error -h, --help Show this message and exit. ``` ## Advanced Commands The following advanced commands provide additional functionality for power users. ### `aws` Access additional functionality on LocalStack AWS Services ```bash Usage: localstack aws [OPTIONS] COMMAND [ARGS]... Accesses additional functionality on LocalStack emulated AWS services. This command provides tools to enhance your experience with certain emulated AWS services. Options: -h, --help Show this message and exit. Commands: iam (Preview) Access LocalStack IAM features ```
Subcommands for localstack aws #### `aws iam` (Preview) Access LocalStack IAM features ```bash Usage: localstack aws iam [OPTIONS] COMMAND [ARGS]... Access LocalStack IAM features. This command provides tools to make it easier to write IAM policies for your cloud application. Options: -h, --help Show this message and exit. Commands: stream Stream policies for all requests enforced on LocalStack summary Summary of policies for all requests enforced on LocalStack ```
### `dns` Manage LocalStack DNS host config ```bash Usage: localstack dns [OPTIONS] COMMAND [ARGS]... Manage the usage of the LocalStack DNS on your host. This command provides tools to configure your the DNS on your host machine to use the LocalStack DNS on your host machine. The LocalStack DNS is used for certain Pro features (like the transparent endpoint injection). Visit https://docs.localstack.cloud/user-guide/tools/transparent-endpoint-injection/dns-server/ for more information on the LocalStack DNS and how it is used. Options: -h, --help Show this message and exit. Commands: systemd-resolved Manage LocalStack DNS in systemd-resolved ```
Subcommands for localstack dns #### `dns systemd-resolved` Manage LocalStack DNS in systemd-resolved ```bash Usage: localstack dns systemd-resolved [OPTIONS] Manage the LocalStack DNS configuration using systemd-resolved (Ubuntu, Debian, etc.). This command sets (or reverts) the LocalStack DNS, running in the current LocalStack runtime, in systemd-resolved for the docker network interface. Most current Linux systems - like Ubuntu, Debian, or Fedora - use systemd- resolved for the network name resolution. Options: -s, --set / -r, --revert Set or revert DNS settings [default: set] -h, --help Show this message and exit. ```
### `ephemeral` Manage ephemeral LocalStack instances ```bash Usage: localstack ephemeral [OPTIONS] COMMAND [ARGS]... Manage ephemeral LocalStack instances in the cloud. This command group allows you to create, list, and delete ephemeral LocalStack instances. Ephemeral instances are temporary cloud instances that can be used for testing and development. Options: -h, --help Show this message and exit. Commands: create Create a new ephemeral instance delete Delete an ephemeral instance describe Describe an ephemeral instance list List all ephemeral instances logs Fetch logs from an ephemeral instance ```
Subcommands for localstack ephemeral #### `ephemeral create` Create a new ephemeral instance ```bash Usage: localstack ephemeral create [OPTIONS] Create a new ephemeral LocalStack instance in the cloud. Specify an instance name and optional parameters like lifetime and environment variables. The instance will be created with the specified configuration and its connection details will be returned. Examples: localstack ephemeral create --name my-test-instance localstack ephemeral create --name my-instance --lifetime 60 localstack ephemeral create --name my-instance --env DEBUG=1 Options: --name TEXT Name of the ephemeral instance [required] --lifetime INTEGER Lifetime of the instance in minutes -e, --env TEXT Additional environment variables that are passed to the LocalStack instance -h, --help Show this message and exit. ``` #### `ephemeral delete` Delete an ephemeral instance ```bash Usage: localstack ephemeral delete [OPTIONS] Delete a specific ephemeral LocalStack instance. Specify the name of the instance you want to delete. Once deleted, the instance cannot be recovered. Example: localstack ephemeral delete --name my-test-instance Options: --name TEXT Name of the ephemeral instance to delete [required] --wait Wait until the instance is fully deleted --timeout INTEGER Maximum seconds to wait for deletion when --wait is set [default: 300] -h, --help Show this message and exit. ``` #### `ephemeral describe` Describe an ephemeral instance ```bash Usage: localstack ephemeral describe [OPTIONS] Describe a specific ephemeral LocalStack instance and show its current state. Retrieve the full details and current state of an ephemeral instance by specifying its name. Example: localstack ephemeral describe --name my-test-instance Options: --name TEXT Name of the ephemeral instance to describe [required] -h, --help Show this message and exit. ``` #### `ephemeral list` List all ephemeral instances ```bash Usage: localstack ephemeral list [OPTIONS] List all available ephemeral LocalStack instances. This command shows all ephemeral instances associated with your account, including their names, status, and other relevant details. Examples: localstack ephemeral list Options: -h, --help Show this message and exit. ``` #### `ephemeral logs` Fetch logs from an ephemeral instance ```bash Usage: localstack ephemeral logs [OPTIONS] Fetch logs from a specific ephemeral LocalStack instance. Retrieve the logs of a running ephemeral instance by specifying its name. The logs are returned in chronological order. Example: localstack ephemeral logs --name my-test-instance Options: --name TEXT Name of the ephemeral instance to fetch logs from [required] -h, --help Show this message and exit. ```
### `extensions` (Preview) Manage LocalStack extensions ```bash Usage: localstack extensions [OPTIONS] COMMAND [ARGS]... (Preview) Manage LocalStack extensions. LocalStack Extensions allow developers to extend and customize LocalStack. The feature and the API are currently in a preview stage and may be subject to change. If you are using LocalStack extensions with docker-compose, you can use the CLI by pointing the `LOCALSTACK_VOLUME_DIR=` variable to localstack volume directory on your host. By default, the volume on your host is located in `~/.cache/localstack` on Linux, and `~/Library/Caches` on Mac. Visit https://docs.localstack.cloud/references/localstack-extensions/ for more information on LocalStack Extensions. Options: -v, --verbose Print more output -h, --help Show this message and exit. Commands: dev init Initialize the LocalStack extensions environment. install Install a LocalStack extension. list List installed extension. uninstall Remove a LocalStack extension. ```
Subcommands for localstack extensions #### `extensions dev` ```bash Usage: localstack extensions dev [OPTIONS] COMMAND [ARGS]... Options: -h, --help Show this message and exit. Commands: disable Disables an extension on the host for developer mode. enable Enables an extension on the host for developer mode. list List LocalStack extensions for which dev mode is enabled. new Create a new LocalStack extension from the official extension... ``` #### `extensions init` Initialize the LocalStack extensions environment. ```bash Usage: localstack extensions init [OPTIONS] Initialize the LocalStack extensions environment. The environment variable `LOCALSTACK_VOLUME_DIR` currently defaults to ~/.cache/localstack, where the extension environment will be installed into ./lib/extensions/ Options: -h, --help Show this message and exit. ``` #### `extensions install` Install a LocalStack extension. ```bash Usage: localstack extensions install [OPTIONS] NAME Install a LocalStack extension. This command installs a LocalStack extension, where the name can be any valid pip dependency identifier. Additionally, we support the installation of distribution files from disk, which you can indicate by a ``file://`` prefix in the name Example invocations: localstack extensions install localstack-extension-stripe localstack extensions install "git+https://github.com/localstack/localstack-stripe.git#egg=localstack-stripe" localstack extensions install file://./dist/localstack-extension-hello-world-0.1.0.tar.gz localstack extensions install file://. # assumes the current directory is a source distribution Options: -h, --help Show this message and exit. ``` #### `extensions list` List installed extension. ```bash Usage: localstack extensions list [OPTIONS] List installed extension. The environment variable `LOCALSTACK_VOLUME_DIR` currently defaults to ~/.cache/localstack, where the extension environment will be installed into ./lib/extensions/ Options: -h, --help Show this message and exit. ``` #### `extensions uninstall` Remove a LocalStack extension. ```bash Usage: localstack extensions uninstall [OPTIONS] NAME Remove a LocalStack extension. This command removes a previously installed LocalStack extension, where the name can be any valid package name. Example invocations: localstack extensions uninstall localstack-extension-stripe Options: -h, --help Show this message and exit. ```
### `license` (Preview) Manage and verify your LocalStack license ```bash Usage: localstack license [OPTIONS] COMMAND [ARGS]... (Preview) Manage and verify your LocalStack license. Your LocalStack license allows you to use advanced features of LocalStack. Options: -h, --help Show this message and exit. Commands: activate info ```
Subcommands for localstack license #### `license activate` ```bash Usage: localstack license activate [OPTIONS] Options: -h, --help Show this message and exit. ``` #### `license info` ```bash Usage: localstack license info [OPTIONS] Options: -h, --help Show this message and exit. ```
### `pod` Manage the state of your instance via Cloud Pods. ```bash Usage: localstack pod [OPTIONS] COMMAND [ARGS]... Manage the state of your instance via Cloud Pods. Options: -h, --help Show this message and exit. Commands: delete Delete a Cloud Pod list List all available Cloud Pods load Load the state of a Cloud Pod into the application runtime. remote Manage cloud pod remotes save Create a new Cloud Pod versions List all available versions for a Cloud Pod ```
Subcommands for localstack pod #### `pod delete` Delete a Cloud Pod ```bash Usage: localstack pod delete [OPTIONS] NAME [REMOTE] Delete a Cloud Pod registered on a remote (by default, the LocalStack platform). This command will remove all the versions of a Cloud Pod, and the operation is not reversible. Options: -h, --help Show this message and exit. ``` #### `pod list` List all available Cloud Pods ```bash Usage: localstack pod list [OPTIONS] [REMOTE] List all the Cloud Pods available for a single user, or for an entire organization, if the user is part of one. With the --public flag, it lists the all the available public Cloud Pods. A public Cloud Pod is available across the boundary of a user and/or organization. In other words, any public Cloud Pod can be injected by any other user holding a LocalStack Pro (or above) license. Options: -p, --public List all the available public Cloud Pods -m, --mine List only the Cloud Pods created by the current user -f, --format [table|json] The formatting style for the list pods command output. [default: table] -h, --help Show this message and exit. ``` #### `pod load` Load the state of a Cloud Pod into the application runtime. ```bash Usage: localstack pod load [OPTIONS] NAME [REMOTE] Load the state of a Cloud Pod into the application runtime. Users can import Cloud Pods from different remotes, with the LocalStack platform being the default one. Users can also load a specific version by appending a version number to the pod name after a colon (e.g., `localstack pod load my-pod:3`). If not specified, the latest version will be loaded. Use the `localstack pod versions` to list all the available versions. Loading the state of a Cloud Pod into LocalStack might cause some conflicts with the current state of the container. LocalStack will attempt a best- effort merging strategy between the current state and the one from the Cloud Pod. For a service X present in both the current state and the Cloud Pod, we will attempt to merge states across different accounts and regions. If the service X has a state for the same account and region both in the running container and the Cloud Pod, the latter will be used. If a service Y is present in the running container but not in the Cloud Pod, it will be left untouched. This is the default merge strategy which is activated by either the `--strategy account-region-merge` option or by omitting the `--strategy` option at all. In addition to the default one, LocalStack provides two more strategies: - overwrite, in which the state of LocalStack is completely reset before loading the state from the Cloud Pod. This strategy is activated with the `--strategy overwrite` option . - service-merge: in which LocalStack merges the state of a service under the same account and region when there is no resource overlap. In such a case, the loaded resources are preferred. This option is activated with the `--strategy service-merge` option. To load a local copy of a LocalStack state, you can use the 'localstack state import' command. Options: -s, --secret TEXT Secret for the Cloud Pod encryption. Encryption is an Enterprise only feature. --strategy [overwrite|account-region-merge|service-merge] The merge strategy to adopt when loading the Cloud Pod. [default: account-region-merge] -y, --yes Automatic yes to prompts. Assume a positive answer to all prompts and run non- interactively. --dry-run Checks the resources added or modified in the application runtime by loading a Cloud Pod. -h, --help Show this message and exit. ``` #### `pod remote` Manage cloud pod remotes ```bash Usage: localstack pod remote [OPTIONS] COMMAND [ARGS]... Manage cloud pod remotes Options: -h, --help Show this message and exit. Commands: add Add a remote delete Delete a remote list Lists the available remotes ``` #### `pod save` Create a new Cloud Pod ```bash Usage: localstack pod save [OPTIONS] NAME [REMOTE] Save the current state of the LocalStack container in a Cloud Pod. A Cloud Pod can be registered and saved with different storage options, called remotes. By default, Cloud Pods are hosted in the LocalStack platform. However, users can decide to store their Cloud Pods in other remotes, such as AWS S3 buckets or ORAS registries. An optional message can be attached to any Cloud Pod. Furthermore, one could decide to export only a subset of services with the optional --services option. To use the LocalStack platform for storage, the desired Cloud Pod's name will suffice, e.g.: localstack pod save Please be aware that each following save invocation with the same name will result in a new version being created. To save a local copy of your state, you can use the 'localstack state export' command. Options: -m, --message TEXT Add a comment describing this Cloud Pod's version -s, --services TEXT Comma-delimited list of services to push in the Cloud Pod (all by default) --visibility [public|private] Set the visibility of the Cloud Pod [`public` or `private`]. Does not create a new version -S, --secret TEXT Secret for the Cloud Pod encryption. Encryption is an Enterprise only feature. -f, --format [json] The formatting style for the save command output. -h, --help Show this message and exit. ``` #### `pod versions` List all available versions for a Cloud Pod ```bash Usage: localstack pod versions [OPTIONS] NAME List all available versions for a Cloud Pod This command lists the versions available for a Cloud Pod. Each invocation of the save command is going to create a new version for a named Cloud Pod, if a Pod with such name already does exist in the LocalStack platform. Options: -f, --format [table|json] The formatting style for the version command output. [default: table] -h, --help Show this message and exit. ```
### `replicator` (Preview) Start a replication job or check its status ```bash Usage: localstack replicator [OPTIONS] COMMAND [ARGS]... *** Preview Feature *** This feature is currently in preview mode in our Teams offering and it's availability may change in future releases. The replicator command group allows you to replicate AWS resources into LocalStack. Options: -h, --help Show this message and exit. Commands: resources List supported resources start Replicate an AWS resource status Check replication status ```
Subcommands for localstack replicator #### `replicator resources` List supported resources ```bash Usage: localstack replicator resources [OPTIONS] Options: -h, --help Show this message and exit. *** Preview Feature *** This feature is currently in preview mode in our Teams offering and it's availability may change in future releases. ``` #### `replicator start` Replicate an AWS resource ```bash Usage: localstack replicator start [OPTIONS] Starts a job to replicate an AWS resource into localstack. You must have credentials with sufficient read access to the resource trying to replicate. At the moment only environment variables are recognized. `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_DEFAULT_REGION` must be set. `AWS_ENDPOINT_URL` and `AWS_SESSION_TOKEN` are optional. Options: --replication-type [SINGLE_RESOURCE|BATCH] Type of replication job: SINGLE_RESOURCE, BATCH [default: SINGLE_RESOURCE] --explore-strategy [SIMPLE|TREE] How we explore the resource tree. SIMPLE only replicates the resource requested. [default: SIMPLE] --resource-arn TEXT ARN of the resource to recreate. Optional for SINGLE_RESOURCE replication --resource-type TEXT CloudControl type of the resource to recreate. Optional for SINGLE_RESOURCE replication --resource-identifier TEXT CloudControl identifier of the resource to recreate. Mandatory if --resource-type is used --target-account-id TEXT Localstack account ID where the resources will be replicated. Defaults to 000000000000. See to enable same account replication --target-region-name TEXT Localstack region where the resources will be replicated. Only provide if different than source AWS account. --delay TEXT Delay for the MOCK replication work --extra-config KEY=VALUE Additional entries to merge into replication_job_config as key=value pairs. Repeat the flag for multiple entries (--extra-config k1=v1 --extra-config k2=v2) Values are sent as strings. -h, --help Show this message and exit. *** Preview Feature *** This feature is currently in preview mode in our Teams offering and it's availability may change in future releases. ``` #### `replicator status` Check replication status ```bash Usage: localstack replicator status [OPTIONS] JOB_ID Check the status of a replication job using its Job ID. Use the --follow flag to continuously check the status until the job is completed. Options: --follow Follow the status until completed --delay INTEGER Delay between calls [default: 5] -h, --help Show this message and exit. *** Preview Feature *** This feature is currently in preview mode in our Teams offering and it's availability may change in future releases. ```
### `state` (Preview) Export, restore, and reset LocalStack state. ```bash Usage: localstack state [OPTIONS] COMMAND [ARGS]... (Preview) Manage and manipulate the localstack state. The state command group allows you to interact with LocalStack's state backend. Read more: https://docs.localstack.cloud/references/persistence- mechanism/#snapshot-based-persistence Options: -h, --help Show this message and exit. Commands: export Export the state of LocalStack services import Import the state of LocalStack services inspect Inspect the state of LocalStack services reset Reset the state of LocalStack services ```
Subcommands for localstack state #### `state export` Export the state of LocalStack services ```bash Usage: localstack state export [OPTIONS] [DESTINATION] Save the current state of the LocalStack container to a file on the local disk. This file can be restored at any point in time using the `localstack state import` command. Please be aware that this might not be possible when importing the state with a different version of LocalStack. If you are looking for a managed solution to handle the state of your LocalStack container, please check out the Cloud Pods feature: https://docs.localstack.cloud/user-guide/tools/cloud-pods/. Use the DESTINATION argument to specify an absolute or relative path for the exported file. If no destination is specified, a file named `ls-state- export` will be saved in the current working directory. Examples: localstack state export my-state localstack state export ../parent-dir/my-state localstack state export /home/johndoe/my-state You can also specify a subset of services to export with the `--services` option. For example: localstack state export my-state --services s3,lambda By default, the state of all running services is exported. Options: -s, --services TEXT Comma-delimited list of services to reset. By default, the state of all running services is exported. -f, --format [json] The formatting style for the save command output. -h, --help Show this message and exit. ``` #### `state import` Import the state of LocalStack services ```bash Usage: localstack state import [OPTIONS] SOURCE Load the state of LocalStack from a file into the running container. The SOURCE argument is the absolute or relative path to the file containing the state to import. This file must have been generated from a previous `localstack state export` command. Please be aware that it might not be possible to import a state generated from a different version of LocalStack. Examples: localstack state import my-state localstack state import ../parent-dir/my-state localstack state import /home/johndoe/my-state Options: -h, --help Show this message and exit. ``` #### `state inspect` Inspect the state of LocalStack services ```bash Usage: localstack state inspect [OPTIONS] Inspect the state of the Localstack Container. By default, it starts a curses interface which allows an interactive inspection of the contents of the LocalStack running instance. Options: -f, --format [curses|rich|json] The formatting style for the inspect command output. [default: curses] -h, --help Show this message and exit. ``` #### `state reset` Reset the state of LocalStack services ```bash Usage: localstack state reset [OPTIONS] Reset the service states of the current LocalStack runtime. This command invokes a reset of services in the currently running LocalStack container. By default, all services are rest. The `services` options allows to select a subset of services which should be reset. This command tries to automatically discover the running LocalStack instance. If LocalStack has not been started with `localstack start` (and is not automatically discoverable), please set `LOCALSTACK_HOST`. Options: -s, --services TEXT Comma-delimited list of services to reset. By default, the state of all running services is reset. -h, --help Show this message and exit. ```
# LocalStack Desktop > Getting started with the LocalStack Desktop application. LocalStack Desktop is a desktop client that allows users to easily control and interact with their LocalStack instance. Using LocalStack Desktop, users can start and stop their LocalStack instance with a single click, create a new container, view logs, interact with LocalStack container via cli and use our resource browser. :::note LocalStack Desktop replaces the previous LocalStack Cockpit application. Cockpit isn't available or maintained anymore and we recommend you to use LocalStack Desktop instead. ::: ## Installation You can download LocalStack Desktop from our [web application](https://app.localstack.cloud/download). To install LocalStack Desktop, **Docker** is the only prerequisite. ## Features LocalStack Desktop helps users to interact with their LocalStack instance with a simple and intuitive UI. Some of the features of LocalStack Desktop includes the ability to: Control LocalStack, Interact with LocalStack, get LocalStack insights and use the Resource browser. ### Control LocalStack Using our Desktop application you will be able to start, stop, delete and create new containers with just a click. It also allows to set up a custom URL if you are using LocalStack outside of Docker or in Kubernetes. ![LocalStack Desktop container creation](/images/aws/localstack-desktop-containers.png) ### Interact with LocalStack You can run commands within the LocalStack container by using our CLI ![LocalStack Desktop cli interaction](/images/aws/localstack-desktop-terminal.png) ### LocalStack Insights LocalStack Desktop provides quick access to your LocalStack logs for instant insights. See what's happening in details from the Logs tab. ![LocalStack Desktop Logs tab](/images/aws/localstack-desktop-logs.png) ### Resource browser You can also create, modify, delete and read all of your resources from the Resource Browser tab, having the same experience that you would have using it in our [web application](https://app.localstack.cloud/inst/default/resources) ![LocalStack Desktop Resource Browser](/images/aws/localstack-desktop-resource-browser.png) # lstk CLI > Overview, installation, and quick start for lstk, the modern CLI for managing LocalStack. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction `lstk` is a high-performance command-line interface for LocalStack, built in Go. It provides a built-in terminal UI (TUI) for interactive use and plain text output for CI/CD pipelines and scripting. `lstk` handles the full emulator lifecycle: authentication, pulling the Docker image, starting, stopping, and restarting the container, streaming logs, and checking status. It can also save and load emulator state (as local snapshots or Cloud Pods) reset running state, run AWS CLI commands against the emulator, and manage the on-disk volume. Running `lstk` with no arguments takes you through the entire startup flow automatically. `lstk` also proxies developer tools so they run directly against LocalStack: the AWS CLI (`lstk aws`), the Azure CLI (`lstk az`), Terraform (`lstk terraform`), the AWS CDK (`lstk cdk`), and the AWS SAM CLI (`lstk sam`). :::tip[Recommended] `lstk` is the recommended way to run and manage LocalStack. The [legacy LocalStack CLI](/aws/developer-tools/running-localstack/localstack-cli/) is deprecated. ::: This section is split into focused pages: - **Overview** (this page): installation, quick start, global options, and shell completions. - [Authentication](/aws/developer-tools/running-localstack/lstk/authentication/): logging in and out, and how `lstk` resolves your auth token. - [Configuration](/aws/developer-tools/running-localstack/lstk/configuration/): the `config.toml` file, emulator types, environment variables, and volumes. - [Lifecycle commands](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/): `start`, `stop`, `restart`, `status`, `logs`, `reset`, `volume`. - [Cloud & IaC commands](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/): `aws`, `az`, `terraform`, `cdk`, `sam`. - [Snapshots](/aws/developer-tools/running-localstack/lstk/snapshots/): save and load emulator state with `snapshot save`/`load`/`list`/`remove`/`show`/`versions`. - [Automation & CI](/aws/developer-tools/running-localstack/lstk/automation/): non-interactive mode, structured output, targeting an external emulator, and environment variables. - [Setup & maintenance](/aws/developer-tools/running-localstack/lstk/setup-and-maintenance/): `setup`, `config`, `update`, and offline/enterprise environments. - [FAQ & Troubleshooting](/aws/developer-tools/running-localstack/lstk/faq-and-troubleshooting/). ## Prerequisites - [Docker](https://docs.docker.com/get-docker/) installed and running. - A [LocalStack account](https://www.localstack.cloud/pricing) with a [license](/aws/getting-started/auth-token/#license-assignment), and `lstk` handles authentication for you (see [Authentication](/aws/developer-tools/running-localstack/lstk/authentication/)). ## Installation ```bash brew install localstack/tap/lstk ``` Homebrew also installs shell completions for bash, zsh, and fish automatically. ```bash npm install -g @localstack/lstk ``` Download the binary for your platform from [GitHub Releases](https://github.com/localstack/lstk/releases), extract it, and place it on your `PATH`. Verify the installation: ```bash lstk --version ``` ### Updating `lstk` can update itself. It detects how it was originally installed (Homebrew, npm, or binary) and uses the matching update method: ```bash # Check for updates without installing lstk update --check # Update to the latest version lstk update ``` See the [`update`](/aws/developer-tools/running-localstack/lstk/setup-and-maintenance/#update) command for details, including the start-time update notification. ## Quick start ```bash lstk ``` Running `lstk` without arguments performs the full startup sequence: authenticates you automatically, pulls the latest image if needed, and starts the LocalStack container. In an interactive terminal it launches the TUI; in a non-interactive environment it prints plain text output. On the very first interactive run, `lstk` prompts you to pick which emulator to run (AWS, Snowflake, or Azure) and writes your choice to `config.toml`. See [Emulator types](/aws/developer-tools/running-localstack/lstk/configuration/#emulator-types) for the available options. For CI or headless environments, set `LOCALSTACK_AUTH_TOKEN` and use `--non-interactive`: ```bash LOCALSTACK_AUTH_TOKEN= lstk --non-interactive ``` CI environments require a CI Auth Token; a personal Developer Auth Token cannot be used there. ## Global options These options are available for all commands: | Option | Description | |:--------------------|:------------------------------------------------------------------------------| | `--config ` | Path to a specific TOML config file | | `--endpoint-url ` | Target an existing, externally-managed emulator at this URL instead of discovering one via local Docker. See [Targeting an external emulator](/aws/developer-tools/running-localstack/lstk/automation/#targeting-an-external-emulator). | | `--non-interactive` | Disable the interactive TUI, use plain output | | `--json` | Emit a single machine-readable JSON envelope on stdout instead of human-oriented output. Supported by `start`, `stop`, `status`, `reset`, and `update`; any other command rejects it. See [Structured output](/aws/developer-tools/running-localstack/lstk/automation/#structured-output). | | `--persist` | Persist emulator state across restarts (on `start`/bare `lstk` and `restart`) | | `--type `, `-t ` | Emulator type to start: `aws`, `snowflake`, or `azure` (on `start`/bare `lstk`; records the choice in config). See [Selecting the emulator with `--type`](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/#selecting-the-emulator-with---type). | | `--snapshot ` | Snapshot REF to auto-load after start (on `start`/bare `lstk`; overrides config for one run) | | `--no-snapshot` | Skip auto-loading the configured snapshot (on `start`/bare `lstk`) | | `--timeout ` | Startup readiness deadline for `start`/bare `lstk`, as a Go duration; overrides `LSTK_STARTUP_TIMEOUT` for one run. See [`start`](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/#start). | | `-v`, `--version` | Print the version and exit | | `-h`, `--help` | Print help and exit | These apply to both interactive and non-interactive (scripted/CI) use, see [Automation & CI](/aws/developer-tools/running-localstack/lstk/automation/) for the details behind `--non-interactive`, `--json`, and `--endpoint-url`. ## Shell completions `lstk` includes completion scripts for bash, zsh, fish, and powershell. If you installed via Homebrew, completions are set up automatically. For manual setup: ```bash # Load in current session eval "$(lstk completion bash)" # Persist (Linux) lstk completion bash > /etc/bash_completion.d/lstk # Persist (macOS with Homebrew) lstk completion bash > $(brew --prefix)/etc/bash_completion.d/lstk ``` :::note Use `eval "$(lstk completion bash)"` rather than `source <(lstk completion bash)`. The `lstk` script works with or without the `bash-completion` package (it bundles a fallback for stock macOS bash 3.2), but `source <(...)` is a silent no-op on that shell. ::: ```bash # Load in current session source <(lstk completion zsh) # Persist (Linux) lstk completion zsh > "${fpath[1]}/_lstk" # Persist (macOS with Homebrew) lstk completion zsh > $(brew --prefix)/share/zsh/site-functions/_lstk ``` ```bash # Load in current session lstk completion fish | source # Persist lstk completion fish > ~/.config/fish/completions/lstk.fish ``` Restart your shell after persisting completions. ### `completion` Generate shell completion scripts. ```bash lstk completion [bash|zsh|fish|powershell] ``` See [Shell completions](#shell-completions) above for setup instructions. # lstk Authentication > How lstk resolves your auth token, and the login and logout commands. `lstk` resolves your auth token in the following order: 1. **`LOCALSTACK_AUTH_TOKEN` environment variable**: takes precedence over a stored token. 2. **System keyring**: a token stored by a previous `lstk login`, used when the environment variable is not set. 3. **Browser login**: triggered automatically in interactive mode when neither of the above provides a token. :::note `LOCALSTACK_AUTH_TOKEN` takes precedence over a token in the keyring. A per-invocation token (a CI secret, or `LOCALSTACK_AUTH_TOKEN=... lstk start` for a second account) therefore overrides a previous `lstk login` without needing `lstk logout` first. To go back to the stored token, unset the environment variable. ::: ## Logging in ```bash lstk login ``` Opens a browser window for authentication and stores the resulting token in your system keyring. This command requires an interactive terminal. See the [`login`](#login) command below for the full flow and the endpoints it uses. ## Logging out ```bash lstk logout ``` Removes the stored credentials from the system keyring and the file-based fallback, and clears the cached license. `logout` cannot clear a token supplied via `LOCALSTACK_AUTH_TOKEN`; if you authenticated that way, unset the variable instead. See the [`logout`](#logout) command below for the full behavior. ## File-based token storage On systems where the system keyring is unavailable, `lstk` automatically falls back to storing the token in a file (`/auth-token`, mode `0600`). You can force file-based storage by setting: ```bash export LSTK_KEYRING=file ``` ## `login` Authenticate with LocalStack via a browser-based device authorization flow and store the resulting credential in your system keyring. This command requires an interactive terminal. ```bash lstk login ``` `lstk` opens your default browser to the LocalStack Web Application, shows a one-time code, and waits for you to approve the request. If the browser cannot open automatically, `lstk` prints the URL to visit manually. On success it stores the **license token** returned by the platform (not the raw browser bearer token). If you are already authenticated — either `LOCALSTACK_AUTH_TOKEN` is set or a token already exists in storage — `login` prints `You're already logged in` and exits without starting a new flow. In non-interactive mode (piped output, CI, or `--non-interactive`), `login` fails with `login requires an interactive terminal`. The `--config ` flag selects which `config.toml` is loaded, which affects `keyring`, `web_app_url`, and `api_endpoint` resolution. :::note If you approve the request in the browser only *after* pressing a key in the terminal, `lstk` reports `auth request not confirmed - please complete the authentication in your browser`. Re-run `lstk login` and approve in the browser before continuing. ::: The credential is written to the system keyring (service `lstk`, key `lstk.auth-token`). When the keyring is unavailable — or `LSTK_KEYRING=file` is set — `lstk` stores it in a file at `/auth-token` (mode `0600`) instead. Endpoints used by the flow can be overridden via config or environment: | Config key | Env var | Default | Description | |:---------------|:--------------------|:-------------------------------|:-----------------------------------------------------------------------------| | `keyring` | `LSTK_KEYRING` | (system keyring) | Set to `file` to force file-based token storage instead of the OS keyring. | | `web_app_url` | `LSTK_WEB_APP_URL` | `https://app.localstack.cloud` | Base URL used to build the browser authorization link. | | `api_endpoint` | `LSTK_API_ENDPOINT` | `https://api.localstack.cloud` | LocalStack platform API endpoint used for the device flow and license token. | ```bash # Force file-based token storage during login LSTK_KEYRING=file lstk login # Use a specific config file lstk --config ./.lstk/config.toml login ``` ## `logout` Remove stored authentication credentials. ```bash lstk logout lstk logout --non-interactive ``` `logout` deletes the auth token from your system keyring (falling back to the file-based token at `/auth-token` when the keyring is unavailable or `LSTK_KEYRING=file` is set) and removes the cached license file. On success it prints `Logged out successfully`. The outcome depends on how you are authenticated: | Situation | Behavior | |:----------|:---------| | A token is stored (from `lstk login`) | The token is deleted from the keyring and file fallback, the cached license is removed, and `lstk` prints `Logged out successfully`. | | No stored token, but `LOCALSTACK_AUTH_TOKEN` is set | Nothing is deleted. `lstk` prints a note that you are authenticated via the environment variable and to unset it to log out. | | No stored token and no `LOCALSTACK_AUTH_TOKEN` | `lstk` prints `Not currently logged in` and exits successfully. | :::note `logout` never clears the `LOCALSTACK_AUTH_TOKEN` environment variable, and it does not stop running emulators. If a LocalStack emulator is still running after logout, `lstk` prints a note reminding you it is running in the background; run `lstk stop` to stop it. ::: # lstk Automation & CI > Non-interactive mode, structured JSON output, targeting an external emulator, environment variables, tracing, and logging for scripting lstk. ## Interactive and non-interactive mode `lstk` automatically selects its output mode: - **Interactive mode** (TUI): used when both stdin and stdout are connected to a terminal. Commands like `start`, `stop`, `restart`, `status`, `login`, `update`, and the confirmation prompts of `reset`/`volume clear` display a Bubble Tea-powered terminal UI. - **Non-interactive mode** (plain text): used when the output is piped, redirected, or running in CI. Force this in a TTY with `--non-interactive`. ```bash # Force plain output even in an interactive terminal lstk --non-interactive start ``` :::note `lstk login` requires an interactive terminal; if you need to authenticate in CI, set `LOCALSTACK_AUTH_TOKEN` instead. Commands that mutate state without prompting in CI (`reset`, `volume clear`) require `--force`. `lstk setup aws` works non-interactively — it writes the profile with defaults and needs `--force` only to overwrite a conflicting `localstack` profile. ::: ## Targeting an external emulator By default `lstk` discovers the emulator it manages through local Docker. The `--endpoint-url ` global flag (or the `LSTK_ENDPOINT_URL` environment variable) instead points a command at an emulator `lstk` did not start — a Docker Compose or host-network deployment, one running in CI or on another machine, or a LocalStack cloud-hosted ephemeral instance. ```bash # Run against an emulator reachable at a custom URL lstk aws --endpoint-url http://localhost:4566 s3 ls # Equivalent via the environment LSTK_ENDPOINT_URL=https://my-ephemeral-instance.localstack.cloud lstk status ``` The endpoint is resolved from, in order of precedence: the `--endpoint-url` flag, `LSTK_ENDPOINT_URL`, then `AWS_ENDPOINT_URL` (a full synonym for `LSTK_ENDPOINT_URL`, one tier lower). Both `http://` and `https://` URLs are accepted (any other scheme is rejected), and the scheme is preserved end-to-end, so `https://` ephemeral instances work. The commands that accept an external endpoint are the ones that only *talk to* an already-running emulator: [`aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws), [`az`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#az), [`terraform`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#terraform)/`tf`, [`cdk`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#cdk), [`sam`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#sam), [`status`](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/#status), [`reset`](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/#reset), and the [`snapshot`](/aws/developer-tools/running-localstack/lstk/snapshots/) `save`/`load`/`remove` subcommands (including the `lstk save`/`lstk load` aliases) and `list s3://…`. Commands that manage the emulator's lifecycle or on-disk state have no remote equivalent and **reject** any endpoint source: `start`, the bare `lstk`, `stop`, `restart`, `logs`, and `volume`. The emulator's type (AWS, Azure, or Snowflake) is auto-detected by probing the endpoint's health API — there is no override flag or config setting, and an inconclusive probe is a hard failure. The AWS-only tools (`terraform`, `cdk`, `sam`) reject an endpoint whose detected type is not AWS. ## Structured output The global `--json` flag makes a command emit a single, machine-readable JSON object on stdout instead of human-oriented text, for scripting and CI. JSON support is available per command: `start`, `stop`, `status`, `reset`, and `update` accept `--json`. Any other command rejects it with an error envelope (`error.code: NOT_JSON_CAPABLE`) rather than silently printing plain text. Every JSON-capable command writes **exactly one** JSON object with the following envelope shape: ```json { "schemaVersion": 1, "command": "stop", "status": "ok", "data": { "emulators": [ { "type": "aws", "name": "localstack-aws", "wasRunning": true } ] }, "warnings": [], "error": null } ``` | Field | Type | Description | |:----------------|:-----------------|:--------------------------------------------------------------------------------------------------------| | `schemaVersion` | integer | Wire-format version of the envelope, always `1` for this schema. Check it once before parsing. | | `command` | string | The command that produced the envelope (e.g. `"stop"`, `"reset"`). | | `status` | string | `"ok"` or `"error"` — branch on this first. | | `data` | object or `null`| Command-specific result. Non-null when `status` is `"ok"`, `null` when it is `"error"`. | | `warnings` | array | Non-fatal notices, always present (empty array when there are none). Each entry is `{ "code", "message" }`. | | `error` | object or `null`| The machine-readable failure. Non-null when `status` is `"error"`, `null` otherwise. | When `status` is `"error"`, the `error` object carries a stable `code` (e.g. `EMULATOR_NOT_RUNNING`, `CONFIRMATION_REQUIRED`, `RUNTIME_UNAVAILABLE`), a coarse `category`, a human-readable `message` (informational only — branch on `code`, not `message`), and a `retryable` boolean: ```json { "schemaVersion": 1, "command": "reset", "status": "error", "data": null, "warnings": [], "error": { "code": "CONFIRMATION_REQUIRED", "category": "USAGE", "message": "reset requires confirmation; use --force to skip in non-interactive mode", "retryable": false } } ``` ### Exit codes For a full enumeration, read `error.code` from the envelope; the process exit code carries only the two most common, mechanically-remediable failures: | Exit code | Meaning | |:----------|:-------------------------------------------------------------------------------------------| | `0` | `status: "ok"`. | | `1` | `status: "error"` for any code other than the two below. | | `2` | A Cobra-level usage error that occurred before `--json` could be recognized (plain-text error on stderr, not an envelope). | | `3` | `error.code == "CONFIRMATION_REQUIRED"` (re-run with `--force`). | | `4` | `error.code == "AUTH_REQUIRED"` (run `lstk login` or set `LOCALSTACK_AUTH_TOKEN`). | :::note `--json` implies non-interactive behavior: no TUI and no prompts. Combining it with a destructive command that would otherwise prompt (`reset`) still requires `--force`, which surfaces as `CONFIRMATION_REQUIRED` (exit code `3`) when omitted. ::: ## Environment variables The following environment variables configure `lstk` itself (not the LocalStack container): | Variable | Description | |:-------------------------------|:---------------------------------------------------------------------------------------------------------------------| | `LOCALSTACK_AUTH_TOKEN` | Auth token for non-interactive runs or to skip browser login. Takes precedence over a token stored in the keyring. | | `LSTK_ENDPOINT_URL` | Target an existing, externally-managed emulator at this URL (equivalent to `--endpoint-url`). `AWS_ENDPOINT_URL` is a lower-precedence synonym. See [Targeting an external emulator](#targeting-an-external-emulator). | | `LOCALSTACK_HOST` | Override the host (and optional port) used when resolving and printing the emulator endpoint, and when writing the AWS CLI profile. Bypasses the `localhost.localstack.cloud` DNS probe. | | `LOCALSTACK_DISABLE_EVENTS` | Set to `1` to disable anonymous telemetry event reporting. | | `DOCKER_HOST` | Override the Docker daemon socket (e.g. `unix:///home/user/.colima/default/docker.sock`). | | `LSTK_KEYRING` | Set to `file` to force file-based token storage instead of the system keyring. | | `LSTK_STARTUP_TIMEOUT` | Startup readiness deadline for `lstk start`, as a Go duration (e.g. `90s`, `2m`). Zero/unset uses the per-mode default (20s interactive, 60s non-interactive). See [`start`](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/#start). | | `LSTK_MERGE_STRATEGY` | Default merge strategy for `snapshot load` / `load` (`account-region-merge`, `overwrite`, or `service-merge`) when `--merge` is not passed. An explicit `--merge` always wins. | | `LSTK_OTEL` | Set to `1` to enable OpenTelemetry trace export (disabled by default). See [OpenTelemetry tracing](#opentelemetry-tracing). | | `LSTK_GITHUB_TOKEN` | Optional GitHub token used when checking for or downloading `lstk` updates (raises GitHub API rate limits). | | `LSTK_API_ENDPOINT` | Override the LocalStack platform API base URL. Default: `https://api.localstack.cloud`. | | `LSTK_WEB_APP_URL` | Override the LocalStack Web Application URL used for browser login. Default: `https://app.localstack.cloud`. | When `LSTK_OTEL` is enabled, the standard `OTEL_EXPORTER_OTLP_*` environment variables are honored by the OpenTelemetry SDK. ### Container runtime discovery `lstk` talks to a Docker-compatible runtime and works with Docker Desktop, Rancher Desktop, Colima, OrbStack, Lima, and Podman. When `DOCKER_HOST` is not set, it resolves the daemon endpoint in this order: 1. **`DOCKER_HOST`**, if set, always wins. 2. **`DOCKER_CONTEXT`** or the active Docker CLI context, when it is non-default and reachable (a stale or unreachable context is skipped rather than failing). 3. On **Linux**, a live `/var/run/docker.sock` — a running Docker daemon is preferred over a co-installed runtime such as Podman. 4. A probe of known runtime sockets (Docker Desktop, Rancher Desktop, Colima, OrbStack, Podman, Lima). Each candidate is dialed, not just checked for existence, so a leftover socket file never shadows a live daemon. 5. The Docker SDK's own default. If no runtime is reachable, the error tailors its suggested start command (`rdctl start`, `colima start`, `podman machine start`, …) to the runtime it detects. Set `DOCKER_HOST` to point at a specific socket to bypass discovery entirely. ### Container-injected variables `lstk` injects several environment variables into the LocalStack container on every start, in addition to any profiles you configure: | Variable | Default value | Description | |:-----------------------------|:-------------------------------------------------|:---------------------------------------------| | `LOCALSTACK_AUTH_TOKEN` | (your resolved token) | Passed from the CLI to activate the license. | | `GATEWAY_LISTEN` | `:4566,:443` | Ports the emulator binds inside the container. | | `MAIN_CONTAINER_NAME` | `localstack-aws` | Container name for internal references. | | `LOCALSTACK_HOST` | `localhost.localstack.cloud:` | Hostname/port the emulator advertises. | | `LOCALSTACK_PERSISTENCE` | `1` (only with `--persist`) | Enables state persistence across restarts. | | `LOCALSTACK_CLIENT_NAME` | `lstk` | Identifies the client that started the emulator. | | `LOCALSTACK_CLIENT_VERSION`| (the `lstk` version) | Version of the client that started the emulator. | When a Docker socket is detected it is bind-mounted into the container and `DOCKER_HOST=unix:///var/run/docker.sock` is injected so the emulator can spawn its own containers. `lstk` also forwards host environment variables matching `CI` and `LOCALSTACK_*` (the host `LOCALSTACK_AUTH_TOKEN` is dropped so it cannot override the token resolved by `lstk`). The container also gets port mappings for `4566`, `443`, and the service port range `4510-4559`. :::note `GATEWAY_LISTEN` is read from the container's resolved environment (set it via an `[env.*]` profile), not hardcoded. Beyond controlling which ports the emulator binds, its host part sets the host publish IP for all published ports: a value like `GATEWAY_LISTEN = "0.0.0.0:4566,0.0.0.0:443"` exposes the emulator beyond loopback (e.g. on a remote host), whereas the default binds to `127.0.0.1` only. ::: ## OpenTelemetry tracing `lstk` can export traces of its own command execution over OTLP/HTTP. Tracing is **disabled by default**. Enable it with: ```bash LSTK_OTEL=1 lstk start ``` When enabled, every command is wrapped in a span (e.g. `lstk.start`) recording the exit code and any error. `lstk` does not hardcode an export target, so the OpenTelemetry Go SDK reads the standard `OTEL_EXPORTER_OTLP_*` environment variables automatically (default target: OTLP/HTTP at `localhost:4318`). You need an OTLP-compatible backend running to receive the traces. ## Logging `lstk` writes its own diagnostic logs to `lstk.log` in the same directory as the active config file. This is separate from the LocalStack container logs (which you view with [`lstk logs`](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/#logs)). - The log file is created automatically and appended to across runs. - When the file exceeds **1 MB**, it is cleared on the next run. - Use `lstk config path` to find the config directory; `lstk.log` sits alongside `config.toml`. # lstk Cloud & IaC Commands > The aws, az, terraform, cdk, and sam commands that proxy cloud and infrastructure-as-code tools against LocalStack. `lstk` proxies developer tools so they run directly against LocalStack. :::note Like `lstk aws`, the `az`, `terraform`, `cdk`, and `sam` proxies do not start the emulator — start it first with [`lstk start`](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/#start). Each requires the corresponding third-party CLI to be installed and on your `PATH`. To run any of them against an emulator `lstk` did not start, pass [`--endpoint-url`](/aws/developer-tools/running-localstack/lstk/automation/#targeting-an-external-emulator) (or set `LSTK_ENDPOINT_URL`). ::: :::note When you interrupt a proxied tool (for example Ctrl+C or `kill` during `lstk terraform apply`), `lstk` forwards the termination signal to the wrapped tool and waits for it to shut down cleanly rather than killing it outright, so operations like releasing a Terraform state lock can complete. The wrapped tool's real exit code is passed through unchanged. ::: ## `aws` Run AWS CLI commands against the running LocalStack emulator. `lstk aws` proxies your host `aws` CLI with the endpoint, credentials, and region pre-configured, so you don't have to pass `--endpoint-url` or set test credentials yourself. ```bash lstk aws s3 ls lstk aws sqs list-queues lstk aws s3 mb s3://my-bucket ``` It is equivalent to running: ```bash aws --endpoint-url http://localhost:4566 ``` with `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_DEFAULT_REGION` set automatically. Everything after `lstk aws` is forwarded verbatim to the host `aws` binary, including AWS CLI flags such as `--region` or `--output`. The exit code and `stdout`/`stderr` of the underlying `aws` process are passed through unchanged, so piping and interactive subcommands work as expected. | Option | Description | |:--------------------|:--------------------------------------------------------------------------------------------------| | `--account ` | Target a specific 12-digit LocalStack account (default `000000000000`). Must appear immediately after `lstk aws`, before the AWS CLI's own action. Falls back to a 12-digit `AWS_ACCESS_KEY_ID`. See [Selecting the account](#selecting-the-account). | | `--non-interactive` | Suppress the loading spinner. Unlike other commands, this flag is stripped before invoking `aws` (not forwarded). | :::note `lstk aws` does not start the emulator. The AWS emulator must already be running (`lstk start`), Docker must be healthy, and the host `aws` CLI must be installed and on your `PATH`. ::: ### Credentials and region `lstk aws` injects credentials in one of two ways: - **Profile mode**: if a complete `localstack` profile exists in both `~/.aws/config` and `~/.aws/credentials`, `lstk` appends `--profile localstack` and lets `aws` read the region, credentials, and endpoint from that profile. - **Profile-less mode**: if the profile is not present, `lstk` runs `aws` with `AWS_ACCESS_KEY_ID=test`, `AWS_SECRET_ACCESS_KEY=test`, and `AWS_DEFAULT_REGION=us-east-1` injected only when those variables are not already set in your environment. In this mode it also prints an informational note: `No AWS profile found, run 'lstk setup aws'`. Run [`lstk setup aws`](/aws/developer-tools/running-localstack/lstk/setup-and-maintenance/#setup-aws) to create the `localstack` profile for use with the AWS CLI and SDKs. ### Endpoint resolution By default, `lstk` probes whether `localhost.localstack.cloud` resolves to `127.0.0.1` and uses `localhost.localstack.cloud:` if so, otherwise it falls back to `127.0.0.1:`. Set [`LOCALSTACK_HOST`](/aws/developer-tools/running-localstack/lstk/automation/#environment-variables) to override the host:port used to reach LocalStack and skip the DNS probe. The port comes from the AWS container's `port` in `config.toml` (default `4566`). ### Selecting the account LocalStack derives the AWS account from the access key id it receives, so `lstk aws --account ` targets a specific 12-digit LocalStack account by controlling the credentials `aws` runs with (a neutral, real-looking `AKIA…`/`ASIA…` key never reaches the emulator): ```bash lstk aws --account 111111111111 s3 mb s3://my-bucket ``` The flag must appear immediately after `lstk aws`, before the AWS CLI's own action (placing it before `lstk aws` is a placement error; placing it after the action is not caught — `lstk` silently forwards it to the `aws` CLI, which then rejects it). When it is omitted, `lstk` falls back to a 12-digit `AWS_ACCESS_KEY_ID` if one is set, then to the default account `000000000000`. The same leading-flag account selection is available on [`lstk terraform`](#terraform) and [`lstk sam`](#sam); `lstk cdk` does not support it. ### Tab completion `lstk aws ` completes AWS services, operations, and parameters using the AWS CLI's own completer. It is enabled together with the rest of `lstk`'s completion — see [Shell completions](/aws/developer-tools/running-localstack/lstk/#shell-completions). ## `az` Run Azure CLI commands against the running LocalStack Azure emulator. `lstk az` runs `az` with an isolated `AZURE_CONFIG_DIR` in which a custom Azure cloud is registered against LocalStack's endpoints, so your global `~/.azure` configuration is left untouched and plain `az` keeps talking to real Azure. Run [`lstk setup azure`](/aws/developer-tools/running-localstack/lstk/setup-and-maintenance/#setup-azure) once before using this mode. Arguments are forwarded to the host `az` binary, and its exit code and output are passed through unchanged. `lstk`'s own flags (`--non-interactive`, `--config`) are consumed by `lstk` rather than forwarded — for example `lstk az --non-interactive …` suppresses the loading spinner instead of passing the flag to `az`. ```bash lstk az group list lstk az storage account list ``` The Azure CLI has no `--endpoint-url`/`--profile` equivalent, so the isolation relies entirely on the dedicated config directory prepared by `setup azure`. ### Global interception (optional) If a script must invoke plain `az` (not `lstk az`), you can redirect your **global** `~/.azure` to LocalStack instead: ```bash # Point global 'az' at the LocalStack Azure emulator lstk az start-interception # Switch back to real Azure lstk az stop-interception ``` `start-interception` registers and activates the `LocalStack` cloud in your global Azure configuration so every `az` invocation targets LocalStack until you stop it. `stop-interception` switches the active cloud back to `AzureCloud` (override with `--cloud `) and re-enables instance discovery, but only when `LocalStack` is still the active cloud, to avoid clobbering an unrelated selection. :::caution Interception changes global state that affects every `az` command in any terminal. Use the isolated `lstk az ` mode unless you specifically need plain `az` to target LocalStack. ::: ## `terraform` Run Terraform against LocalStack, using LocalStack endpoints as AWS provider overrides. `lstk terraform` (alias `lstk tf`) generates a provider-override file and forwards your arguments to the real `terraform` binary. :::note `lstk terraform` targets the AWS emulator. To use Terraform with the other emulators, see the relevant emulator docs. ::: ```bash lstk terraform init lstk terraform --region us-west-2 plan lstk tf apply ``` lstk-specific flags must appear **before** the Terraform action: | Option | Default | Description | |:------------------|:---------------------|:---------------------------------------| | `--region ` | `us-east-1` | Deployment region. | | `--account ` | `test` | Target AWS account id (12 digits). | Relevant environment variables: `AWS_ENDPOINT_URL` (override the auto-resolved endpoint), `LSTK_TF_CMD` (binary to invoke, e.g. `tofu`; default `terraform`), `LSTK_TF_OVERRIDE_FILE_NAME` (override file name; default `localstack_providers_override.tf`), `LSTK_TF_DRY_RUN` (generate the override file but do not run Terraform), `AWS_REGION` (fallback for `--region`), and `AWS_ACCESS_KEY_ID` (fallback for `--account`). ## `cdk` Run the AWS CDK against LocalStack. Requires the AWS CDK CLI version `2.177.0` or newer on your `PATH`. ```bash lstk cdk bootstrap lstk cdk --region us-west-2 deploy lstk cdk synth ``` The only lstk-specific flag (before the CDK action) is `--region ` (default `us-east-1`); CDK always targets the default LocalStack account `000000000000`, so there is no `--account` flag. Relevant environment variables: `AWS_ENDPOINT_URL`, `AWS_ENDPOINT_URL_S3`, `LSTK_CDK_CMD` (default `cdk`), and `AWS_REGION`. ## `sam` Run the AWS SAM CLI against LocalStack. Requires the AWS SAM CLI version `1.95.0` or newer on your `PATH` (older versions ignore `AWS_ENDPOINT_URL` and would target real AWS). ```bash lstk sam build lstk sam --region us-west-2 deploy lstk sam validate ``` lstk-specific flags (before the SAM action): `--region ` (default `us-east-1`) and `--account ` (12 digits, default `000000000000`). Relevant environment variables: `AWS_ENDPOINT_URL`, `AWS_ENDPOINT_URL_S3`, `LSTK_SAM_CMD` (default `sam`), `AWS_REGION` (fallback for `--region`), and `AWS_ACCESS_KEY_ID` (fallback for `--account`). :::note Compared with `samlocal`, image/container-based Lambda (ECR) deploys and nested CloudFormation stacks are not supported; use `samlocal` for those workflows. ::: # lstk Configuration > The lstk config.toml file, emulator types, environment variables, custom images, and volume mounts. `lstk` uses a TOML configuration file, created automatically on first run. ## Config file search order `lstk` uses the first `config.toml` it finds in this order: 1. `./.lstk/config.toml`: project-local config in the current directory. 2. `$HOME/.config/lstk/config.toml`: user config (created here if `$HOME/.config/` exists). 3. OS default: - **macOS**: `$HOME/Library/Application Support/lstk/config.toml` - **Windows**: `%AppData%\lstk\config.toml` - **Linux**: `$XDG_CONFIG_HOME/lstk/config.toml` or `$HOME/.config/lstk/config.toml` On first run, the config is created at path #2 if `$HOME/.config/` already exists; otherwise at the OS default (#3). To see the active config file path: ```bash lstk config path ``` To use a specific config file: ```bash lstk --config /path/to/config.toml start ``` ## Default configuration The default `config.toml` created on first run. The `type` field reflects whichever emulator you chose at first run (see [Emulator types](#emulator-types)); the example below shows the `aws` default: ```toml [[containers]] type = "aws" # Emulator type. Supported: "aws", "snowflake", "azure" tag = "latest" # Docker image tag, e.g. "latest", "2026.4" port = "4566" # Host port the emulator will be accessible on # container_name = "" # Override the derived container name (also MAIN_CONTAINER_NAME) # image = "" # Full image override (e.g. an internal mirror or offline image) # expose_ports = [] # Extra container ports to publish, e.g. [53] for the DNS server # volume = "" # Host directory for persistent state (default: OS cache dir) # volumes = [] # Docker-style "host:container[:ro]" bind mounts (see Volumes) # env = [] # Named environment profiles to apply (see [env.*] sections below) # snapshot = "" # Snapshot REF to auto-load after start (AWS only) ``` ## Config field reference | Field | Type | Default | Description | |:-----------|:---------|:-----------|:-----------------------------------------------------------------------------------------------------| | `type` | string | `"aws"` | Emulator type. One of `"aws"`, `"snowflake"`, `"azure"`. Run a single `[[containers]]` block at a time. See [Emulator types](#emulator-types). | | `tag` | string | `"latest"` | Docker image tag (`"latest"`, `"2026.4"`, etc.). Useful for pinning a specific version. Zero-padded months (`"2026.04"`) are normalized to `"2026.4"`. | | `port` | string | `"4566"` | Host port the emulator listens on (1–65535). The in-container port is always `4566`. | | `container_name` | string | (derived) | Override the derived container name (`localstack-`, plus `-` when `tag` is not `"latest"`). This is also what the emulator reports as `MAIN_CONTAINER_NAME`. Set it when something outside `lstk` addresses the emulator by a fixed name, e.g. a sidecar proxy on a CI agent. | | `image` | string | (default) | Full image reference that overrides the default Docker Hub image, e.g. an internal-registry mirror or a locally loaded offline image. If it already carries a tag, `tag` is ignored; otherwise `tag` (or `latest`) is appended. | | `expose_ports` | (int \| string)[] | `[]` | Publish additional container ports on the host, beyond the gateway and service ports `lstk` publishes by default. Each entry is a bare port number (published on the same host port) or a Docker-style `"[host:]container[/proto]"` string — e.g. `expose_ports = [53]` to use the emulator's DNS server as the host's resolver, or `expose_ports = ["5354:5353/udp"]`. | | `volume` | string | (OS cache) | Host directory for persistent emulator state. Defaults to `/lstk/volume/`. See also `volumes`. | | `volumes` | string[] | `[]` | Docker-style `"host:container[:ro]"` bind mounts (e.g. init hooks). May also carry the persistence mount (target `/var/lib/localstack`). See [Volume mounts](#volume-mounts). | | `env` | string[] | `[]` | List of named environment profiles to inject into the container (see below). | | `snapshot` | string | `""` | Snapshot REF (e.g. `pod:my-baseline` or a local path) to auto-load after the emulator starts. AWS emulator only. See [Auto-loading a snapshot on start](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/#auto-loading-a-snapshot-on-start). | :::note There is no `update_prompt` config key. `lstk` always checks for available updates on startup. Once you choose to skip a version, `lstk` records it under the `[cli]` table as `update_skipped_version` and stops prompting for that version. This value is written automatically and is not meant to be hand-edited (see [`update`](/aws/developer-tools/running-localstack/lstk/setup-and-maintenance/#update)). ::: ## Emulator types `lstk` can run more than one kind of emulator. The `type` field in your `config.toml` selects which one: | Type | Docker image | Description | |:------------|:------------------------------|:-------------------------------------| | `aws` | `localstack/localstack-pro` | LocalStack AWS emulator (default). | | `snowflake` | `localstack/snowflake` | LocalStack Snowflake emulator. | | `azure` | `localstack/localstack-azure` | LocalStack Azure emulator. | On the first interactive run, `lstk` prompts you to pick an emulator (`a` for AWS, `s` for Snowflake, `z` for Azure) and writes your choice to `config.toml`. In non-interactive mode the default `aws` emulator is used if no config file is found. Lifecycle commands operate on the emulators defined in your `config.toml`. Run a single `[[containers]]` block at a time; the AWS-specific commands (`status` resources, `aws`, `reset`, `setup aws`) require an `aws` emulator to be configured. :::note The AWS emulator's license is validated by `lstk` before the container starts. The Snowflake and Azure emulators validate their own license inside the container at startup, so `lstk` skips its pre-flight license check for them. If your license does not include the selected emulator, the container exits and `lstk` reports the missing entitlement. ::: ## Passing environment variables to the container Define reusable environment profiles under `[env.]` and reference them in your container config: ```toml [[containers]] type = "aws" tag = "latest" port = "4566" env = ["debug", "ci"] [env.debug] DEBUG = "1" ENFORCE_IAM = "1" PERSISTENCE = "1" [env.ci] SERVICES = "s3,sqs" EAGER_SERVICE_LOADING = "1" ``` When `lstk start` runs, the key-value pairs from each referenced profile are injected as environment variables into the LocalStack container. Keys are uppercased automatically. :::note If you reference an `env` profile name that doesn't exist in your config, `lstk` returns an error: `environment "..." referenced in container config not found`. ::: In addition to your custom profiles, `lstk` always injects several variables into the container. See [Container-injected variables](/aws/developer-tools/running-localstack/lstk/automation/#container-injected-variables) for the full list. ## Custom container image By default the emulator image is pulled from Docker Hub (`localstack/localstack-pro`, `localstack/snowflake`, or `localstack/localstack-azure` depending on `type`). Set `image` on a container block to override it — for example, to pull from an internal-registry mirror or to run a locally loaded image in an air-gapped environment: ```toml [[containers]] type = "aws" image = "registry.internal.example.com/localstack/localstack-pro" tag = "2026.4" ``` If `image` already carries a tag (e.g. `...:2026.4`), the separate `tag` field is ignored; otherwise `tag` (or `latest`) is appended. See [Offline and enterprise environments](/aws/developer-tools/running-localstack/lstk/setup-and-maintenance/#offline-and-enterprise-environments) for how `lstk` falls back to a locally present image when a pull fails. ## Volume mounts Beyond the single persistence directory set by `volume`, a container block can declare arbitrary Docker-style bind mounts with `volumes`. Each entry is a `"host:container[:ro]"` spec — useful, for example, for mounting a [Snowflake init hook](/snowflake/capabilities/init-hooks/) script into `/etc/localstack/init/{boot,start,ready,shutdown}.d`: ```toml [[containers]] type = "snowflake" port = "4566" volumes = [ "./test.sf.sql:/etc/localstack/init/ready.d/test.sf.sql", "./data:/var/lib/localstack", ] ``` - A `volumes` entry whose container target is `/var/lib/localstack` sets the persistence directory (the same mount `volume` configures); this is what [`lstk volume path`](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/#volume) and [`lstk volume clear`](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/#volume) resolve. - Relative host sources and a leading `~/` are resolved against the config file's directory. This differs from the legacy `volume` field, whose value is passed to Docker verbatim. - Setting the persistence directory through both `volume` and a `volumes` entry with a different source is a validation error. `volume` and `volumes` overlap only for the persistence mount: `volume` can *only* set the persistence directory, while `volumes` is a superset that can also express init hooks and other mounts. ## Using a project-local config Place a `.lstk/config.toml` in your project directory. When you run `lstk` from that directory, the local config takes precedence over the global one. This lets each project pin its own emulator type, image tag, and environment profiles. For example, a project that targets the Snowflake emulator can keep its own config: ```toml # .lstk/config.toml [[containers]] type = "snowflake" port = "4566" ``` An AWS project might instead pin a specific image tag and enable a debug profile: ```toml # .lstk/config.toml [[containers]] type = "aws" tag = "2026.4" port = "4566" env = ["dev"] [env.dev] DEBUG = "1" PERSISTENCE = "1" ``` # lstk FAQ & Troubleshooting > Frequently asked questions and common issues when using lstk. ## FAQ ### Can I use `lstk` with Docker Compose? Yes, for the commands that talk to an already-running emulator. `lstk start`, `lstk stop`, and the other lifecycle commands manage their own Docker container and are not meant to drive a Compose-managed instance, so don't point `lstk stop` at one. But if you run LocalStack from a `docker-compose.yml`, you can still use `lstk`'s emulator-facing commands against it — `aws`, `az`, `terraform`/`cdk`/`sam`, `status`, `reset`, and `snapshot` — by passing `--endpoint-url ` (or setting `LSTK_ENDPOINT_URL`) to target the Compose deployment. See [Targeting an external emulator](/aws/developer-tools/running-localstack/lstk/automation/#targeting-an-external-emulator) for the commands that accept an endpoint, and the [Docker Compose installation guide](/aws/getting-started/installation/#docker-compose) for the Compose setup itself. ### Which Docker image does `lstk` use? It depends on the emulator type configured in your `config.toml`. The AWS emulator uses `localstack/localstack-pro`, the Snowflake emulator uses `localstack/snowflake`, and the Azure emulator uses `localstack/localstack-azure`. All require a valid auth token (including the free Hobby tier). See [Emulator types](/aws/developer-tools/running-localstack/lstk/configuration/#emulator-types). ### How do I pass configuration options like `DEBUG` or `PERSISTENCE` to the container? Use environment profiles in your `config.toml`. Define the variables under an `[env.]` section and reference that name in the `env` list of your container config. See [Passing environment variables to the container](/aws/developer-tools/running-localstack/lstk/configuration/#passing-environment-variables-to-the-container) for details. ### How do I save and restore emulator state? Use [`lstk snapshot save`](/aws/developer-tools/running-localstack/lstk/snapshots/#snapshot-save) to capture the running AWS emulator's state to a local file or a Cloud Pod, and [`lstk snapshot load`](/aws/developer-tools/running-localstack/lstk/snapshots/#snapshot-load) (or the `lstk save` / `lstk load` aliases) to restore it. To drop in-memory state without writing a snapshot, use [`lstk reset`](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/#reset) (AWS emulator only). ### How do I pin a specific LocalStack version? Set the `tag` field in your `config.toml` to a specific version tag: ```toml [[containers]] type = "aws" tag = "2026.4" port = "4566" ``` ## Troubleshooting ### Port 443 already in use By default, LocalStack publishes both port `4566` and port `443` (controlled by the `GATEWAY_LISTEN` variable). On some systems port 443 is already taken — Windows with Hyper-V, IIS, or VPN software, or an ingress proxy such as Rancher Desktop's Traefik. Because port 443 comes from the **default** `GATEWAY_LISTEN`, a busy 443 is **not fatal**: `lstk` drops that publication with a warning and starts anyway, and HTTPS is still served on the edge port `4566`. You only need to act if you want to silence the warning or bind 443 elsewhere. To skip port 443 entirely, override `GATEWAY_LISTEN` to bind only to `4566`: ```toml [[containers]] type = "aws" tag = "latest" port = "4566" env = ["nossl"] [env.nossl] GATEWAY_LISTEN = "0.0.0.0:4566" ``` :::note A port you list **explicitly** in a custom `GATEWAY_LISTEN` is treated as a hard requirement, so a busy one there fails the start rather than being dropped. Only the `443` from the default value is best-effort. ::: ### Docker is not running `lstk` requires a running Docker daemon. If Docker is not reachable, you will see an error like: ```text Error: runtime not healthy ``` **Fix:** Start your container runtime. `lstk` works with Docker Desktop, Rancher Desktop, Colima, OrbStack, Lima, and Podman — start the Docker daemon (`sudo systemctl start docker` on Linux) or the relevant VM (`rdctl start`, `colima start`, `podman machine start`, …). When the runtime is unavailable, `lstk`'s error tailors its suggested start command to whichever runtime it detects. You can also point `lstk` at a specific socket with `DOCKER_HOST`. See [Container runtime discovery](/aws/developer-tools/running-localstack/lstk/automation/#container-runtime-discovery) for how the daemon is located. ### Authentication required in non-interactive mode When running without a TTY (e.g. in CI), `lstk` cannot open a browser for login. If no token is found in the keyring or environment, it fails: ```text authentication required: set LOCALSTACK_AUTH_TOKEN or run in interactive mode ``` **Fix:** Set the `LOCALSTACK_AUTH_TOKEN` environment variable before running `lstk`: ```bash export LOCALSTACK_AUTH_TOKEN= lstk --non-interactive start ``` You can find your auth token on the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens). ### License validation failed If your auth token is invalid, expired, or not linked to an active license, the LocalStack container exits with a license error: ```text The license activation failed for the following reason: No credentials were found in the environment. ``` **Fix:** - Verify your token is valid at the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens). - Make sure the token is set correctly, either via `lstk login` or the `LOCALSTACK_AUTH_TOKEN` environment variable. - A stale token or cached license no longer requires a manual `lstk logout`: when the platform definitively rejects it, `lstk` drops the cached license and, in an interactive terminal, prompts you to log in again and retries automatically. In non-interactive mode, run `lstk logout && lstk login` (or set a valid `LOCALSTACK_AUTH_TOKEN`) and re-run. ### Image pull failed If `lstk` cannot pull the Docker image, check your network connection and Docker configuration. On corporate networks, you may need to configure Docker's proxy settings, see [How do I configure LocalStack to use my corporate HTTP and HTTPS proxy?](/aws/getting-started/faq/#how-do-i-configure-localstack-to-use-my-corporate-http-and-https-proxy). ### Unknown environment profile If your container config references an `env` profile that doesn't exist, `lstk` returns: ```text environment "myprofile" referenced in container config not found ``` **Fix:** Make sure the profile name in the `env` list matches an `[env.]` section in your `config.toml`: ```toml [[containers]] type = "aws" env = ["myprofile"] # must match the section name below [env.myprofile] DEBUG = "1" ``` ### Getting help If the steps above don't resolve your issue, see [Get Help](/aws/help-support/get-help/) for the available support channels, including the support email and in-app chat. # lstk Lifecycle Commands > The start, stop, restart, status, logs, reset, and volume commands for managing the LocalStack emulator with lstk. `lstk` uses a flat command structure. Running `lstk` with no command is equivalent to `lstk start`. ## `start` Start the LocalStack emulator. Launches the TUI in interactive terminals and prints plain output otherwise. `lstk start` launches the emulator defined in the first `[[containers]]` entry of the resolved `config.toml` (not necessarily AWS). ```bash lstk start lstk start --persist lstk start --non-interactive ``` | Option | Description | |:--------------------|:-----------------------------------------------------------------------------| | `--persist` | Persist emulator state across restarts (sets `LOCALSTACK_PERSISTENCE=1` in the container) | | `--type `, `-t ` | Select the emulator to start (`aws`, `snowflake`, or `azure`) non-interactively, recording the choice in `config.toml`. See [Selecting the emulator with `--type`](#selecting-the-emulator-with---type). | | `--snapshot ` | Auto-load this snapshot after the emulator starts, overriding the configured `snapshot` for one run (AWS only) | | `--no-snapshot` | Skip auto-loading the configured `snapshot` for this run | | `--timeout ` | Maximum time to wait for the emulator to become ready, as a Go duration (e.g. `90s`, `2m`). Overrides `LSTK_STARTUP_TIMEOUT` for this run; `0` uses the per-mode default. | | `--non-interactive` | Disable the interactive TUI and use plain output | `lstk start` forwards host environment variables prefixed with `LOCALSTACK_` to the emulator (the host `LOCALSTACK_AUTH_TOKEN` is dropped so it cannot override the token `lstk` resolved). See [Container-injected variables](/aws/developer-tools/running-localstack/lstk/automation/#container-injected-variables). `lstk` applies a readiness deadline while waiting for the emulator to come up (a crash during startup is detected instantly, with its exit code, and does not wait for the deadline). In an interactive terminal the deadline defaults to 20 seconds and is only a recoverable prompt — you can keep waiting or stop; in non-interactive mode it defaults to 60 seconds and is fatal, leaving the container running for inspection. Override the deadline for a single run with `--timeout` (a Go duration such as `90s` or `2m`), or for every run with [`LSTK_STARTUP_TIMEOUT`](/aws/developer-tools/running-localstack/lstk/automation/#environment-variables); an explicit `--timeout` wins over the environment variable, and `--timeout 0` falls back to the per-mode default. The flag is available on `start` and the bare `lstk` command only — `restart` and the snapshot auto-start path do not expose it. By default the emulator starts with a fresh state on every run. Pass `--persist` to keep data across restarts: `lstk` injects `LOCALSTACK_PERSISTENCE=1` into the container so state is written to the mounted [`volume`](/aws/developer-tools/running-localstack/lstk/configuration/#config-field-reference) and reloaded on the next start. When persistence is active, the AWS emulator's startup summary includes a `• Persistence: Enabled` line. ```bash # Start with persistent state lstk start --persist ``` :::note `--persist` is a flag on `start` (and the bare `lstk` command) and on [`restart`](#restart). For finer-grained control, you can also set `PERSISTENCE = "1"` in an environment profile (see [Passing environment variables to the container](/aws/developer-tools/running-localstack/lstk/configuration/#passing-environment-variables-to-the-container)). ::: `start` supports [`--json`](/aws/developer-tools/running-localstack/lstk/automation/#structured-output) (as does the bare `lstk` command, which reports `"command": "start"`): the `data` payload is a flat object describing the started emulator — its `emulator` type, `container` name, `endpoint`, `version`, whether it was `alreadyRunning`, and whether `persistence` is enabled. ### Selecting the emulator with `--type` `--type` (shorthand `-t`, also available on the bare `lstk` command) is the non-interactive answer to the first-run emulator picker. It selects which emulator to start (`aws`, `snowflake`, or `azure`) and **records the choice in `config.toml`**, so lifecycle commands (`stop`, `status`, `logs`, `volume`, snapshot auto-load) stay in sync with what you started. ```bash # Start the Snowflake emulator, recording the choice in config lstk start --type snowflake # Shorthand lstk start -t azure ``` - On first run, the config is created with the selected type. - If the configured type already matches, `--type` is a no-op. - If it differs, `lstk` rewrites the `type` line in place (comments and formatting preserved) and prints a note naming the config file. When switching an existing config to a different type: - A custom `image` is a **hard error** — it pins a specific product that cannot be reinterpreted under a new emulator type. Use a separate config (`--config`) for that profile instead. - A non-`latest` `tag` and any `volume`/`volumes` mounts are kept, but `lstk` warns that they may be product-specific. - `port`, `env`, and `snapshot` are kept silently. `--type` is a flag only; passing the emulator as a positional (`lstk start azure`) is rejected with a hint pointing at `--type`. ### Auto-loading a snapshot on start For the **AWS emulator**, you can have `lstk` load a snapshot automatically every time it starts the emulator. Set the `snapshot` field on the container block to any load REF (a `pod:` Cloud Pod or a local path): ```toml [[containers]] type = "aws" port = "4566" snapshot = "pod:my-baseline" ``` The snapshot is loaded only when the emulator is **freshly started** this run; if it is already running, the auto-load is skipped. Override it for a single run with `--snapshot REF`, or skip it entirely with `--no-snapshot`: ```bash # Start and load a different snapshot for this run only lstk start --snapshot pod:other-baseline # Start without loading the configured snapshot lstk start --no-snapshot ``` The `snapshot` field is only read on start; [`snapshot save`](/aws/developer-tools/running-localstack/lstk/snapshots/#snapshot-save) never writes it back into your config. ## `stop` Stop the running LocalStack emulator. Stops every emulator container defined in the resolved `config.toml` (the `[[containers]]` entries), with a 30-second stop timeout per container. ```bash lstk stop lstk stop --non-interactive ``` `stop` fails fast if the Docker runtime is not healthy (for example, Docker is not running), or if a configured emulator is not currently running (`LocalStack is not running`). In an interactive terminal it shows an animated "Stopping LocalStack..." spinner and a styled confirmation; in non-interactive mode it prints the same progress and result as plain text. `stop` supports [`--json`](/aws/developer-tools/running-localstack/lstk/automation/#structured-output): the `data` payload lists each configured emulator and whether it `wasRunning`. ## `restart` Stop and restart the LocalStack emulator. Performs a stop of the running emulator followed by a fresh start, using the same auth, config, and Docker settings as [`start`](#start). Launches the TUI in interactive terminals and prints plain output otherwise. ```bash lstk restart lstk restart --persist ``` | Option | Description | |:-------------|:-------------------------------------------| | `--persist` | Persist emulator state across the restart | By default, emulator state is **not** retained across the restart and the container starts clean. Pass `--persist` to keep the emulator's state so it survives the restart. ## `status` Show the status of a running emulator and its deployed resources. Before contacting the emulator, `lstk` checks that the Docker runtime is healthy; if it is not, the command reports `runtime not healthy` and exits with a non-zero status. ```bash lstk status lstk --non-interactive status ``` For each emulator configured in your `config.toml` (the `[[containers]]` entries), `status` reports whether it is running and, if so, prints an instance summary: ```text LocalStack AWS Emulator is running • Endpoint: localhost:4566 • Persistence: Enabled • Container: localstack-aws • Version: 4.0.0 • Uptime: 1h 12m 4s ``` - **Endpoint** is the live `host:port`, queried from Docker, so it stays correct even if the configured `port` was changed while the container kept running. - **Persistence** appears only for the AWS emulator and only when persistence is enabled. - **Uptime** is computed from the container's start time and is omitted if it cannot be determined. If an emulator is not running, `status` prints an error and exits non-zero without checking the remaining emulators: ```text LocalStack AWS Emulator is not running Start LocalStack: lstk See help: lstk -h ``` For the **AWS emulator**, `status` additionally lists deployed resources. When resources exist it prints a summary line followed by a table; when none exist it prints `No resources deployed`. ```text ~ 3 resources · 2 services Service Resource Region Account S3 my-bucket us-east-1 000000000000 SQS my-queue us-east-1 000000000000 ``` In an interactive terminal the output is rendered through the TUI; in non-interactive mode (or with `--non-interactive`) the same content is printed as plain text, with the resource table shown at full width when stdout is not a TTY. The Snowflake and Azure emulators show the instance summary only and never report resources. `status` supports [`--json`](/aws/developer-tools/running-localstack/lstk/automation/#structured-output): the `data` payload lists one entry per configured emulator with its running state, health, version, host, and (for the AWS emulator) a `resourceSummary` and the deployed `resources`. Pass `--no-resources` to omit the resource details for a faster response when polling. `--json` also honors [`--endpoint-url`](/aws/developer-tools/running-localstack/lstk/automation/#targeting-an-external-emulator) to report on an emulator `lstk` did not start. ## `logs` Show or stream emulator logs. ```bash lstk logs [options] ``` | Option | Description | |:------------|:-----------------------------------------| | `--follow`, `-f` | Stream logs in real-time. Without this flag, `lstk` prints the currently available logs and exits. | | `--verbose`, `-v` | Show all logs without filtering. By default, `lstk` drops noisy lines (internal request logs, provider chatter); `--verbose` shows every line verbatim. | | `--tail `, `-n ` | Show only the last `N` lines from the end of the logs. Accepts a non-negative integer or `all` (the default, showing all available lines). | By default, `lstk logs` reads from the first configured emulator container and applies a noise filter. In an interactive terminal, lines are color-coded by log level (`DEBUG`, `INFO`, `WARN`, `ERROR`); in non-interactive mode, raw log lines are written to stdout. Example: ```bash # Print current filtered logs and exit lstk logs # Stream filtered logs in real-time lstk logs --follow # Show only the last 100 lines lstk logs --tail 100 # Stream all logs without filtering lstk logs --follow --verbose ``` ## `reset` Discard the running AWS emulator's in-memory state (all created resources such as S3 buckets and Lambda functions are dropped). The emulator **keeps running**; only its state is cleared. `reset` is **AWS-only** and errors out with `reset is only supported for the AWS emulator` for the Snowflake and Azure emulators. ```bash lstk reset lstk reset --force ``` | Option | Description | |:----------|:------------------------------------------------------------------| | `--force` | Skip the confirmation prompt. Required in non-interactive mode. | In interactive mode, `reset` prompts for confirmation before clearing state. In non-interactive mode it fails unless `--force` is passed: ```text reset requires confirmation; use --force to skip in non-interactive mode ``` `reset` supports [`--json`](/aws/developer-tools/running-localstack/lstk/automation/#structured-output): on success the `data` payload reports the reset emulator and `"reset": true`. :::note `reset` clears in-memory state only. It does **not** wipe the on-disk volume (certificates, persistence data, cached tools). To clear that, stop the emulator and run [`lstk volume clear`](#volume-clear). ::: ## `volume` Manage the emulator volume: the host directory that holds persistent state such as certificates, downloaded tools, and persistence data. ```bash lstk volume path lstk volume clear [options] ``` ### `volume path` Prints the resolved volume directory for every emulator in your config, one per line. With the default config (a single `aws` emulator) it prints one path. Each path is the container's configured `volume` value, or the default OS cache location if `volume` is unset (`~/Library/Caches/lstk/volume/localstack-aws` on macOS, `~/.cache/lstk/volume/localstack-aws` on Linux). ```bash # Print the volume directory for each configured emulator lstk volume path ``` ### `volume clear` Removes all data from the emulator volume directory, resetting cached state. It operates on all configured emulators by default, or a single one with `--type`. Before clearing, it lists each target as `: ()`. | Option | Description | |:----------------|:-----------------------------------------| | `--force` | Skip the confirmation prompt | | `--type ` | Clear only the emulator of this type | ```bash # Clear all configured emulator volumes (prompts for confirmation) lstk volume clear # Clear only the AWS emulator volume lstk volume clear --type aws # Skip the confirmation prompt lstk volume clear --force # Clear without prompting in a non-interactive environment lstk volume clear --type snowflake --force ``` In an interactive terminal, `lstk volume clear` prompts `Clear volume data? This cannot be undone` before deleting anything; choosing **NO** or pressing Ctrl+C cancels with no changes. In non-interactive mode, `--force` is required, otherwise the command fails with `volume clear requires confirmation; use --force to skip in non-interactive mode`. :::caution If the volume contains files owned by `root` (created by Docker), clearing fails with a permission error. Re-run with elevated privileges: ```bash sudo lstk volume clear ``` ::: # lstk Migration Guide > Migration guide for lstk, the new command-line interface for LocalStack. import { Tabs, TabItem } from '@astrojs/starlight/components'; # Migrating from `localstack` to `lstk` ## Introduction `lstk` is the new command-line interface for LocalStack. It is a single, self-contained binary that starts and manages the emulator, runs the AWS CLI and your infrastructure-as-code tools, and saves and restores emulator state. In other words, it replaces both the Python-based `localstack` CLI and the family of wrapper scripts that many projects installed alongside it. Those scripts are `awslocal`, `tflocal`, `samlocal`, and `cdklocal`. There are three good reasons to start using `lstk`: 1. Installation is simpler, because there is no Python environment to manage, and no separate wrapper script to install for each tool you use. 2. `lstk` works with every LocalStack emulator, not just AWS. Snowflake and Azure are supported today, and future emulators will be available too. 3. `lstk` is the place for new CLI functionality to be added from now on. The `localstack` CLI is deprecated, and will no longer be supported. What does not change is LocalStack itself. The emulator is the same Docker image with the same behavior and the same configuration variables (`DEBUG`, `SERVICES`, `PERSISTENCE`, and the rest). You are changing the tool you drive LocalStack with, not the LocalStack emulator itself. For most teams the migration is a short exercise in translating a handful of commands in shell history, scripts, and CI pipelines. :::note The legacy CLI and the wrapper scripts continue to work, so you can migrate gradually rather than all at once. A small number of features are still only available with the legacy `localstack` CLI. Those are listed under [Unsupported features](#unsupported-features). ::: ## Installation `lstk` is distributed through the LocalStack Homebrew tap and the `@localstack/lstk` npm package. Pre-built binaries for Linux, macOS, and Windows are published on GitHub Releases. Pick whichever fits your environment. ```bash brew install localstack/tap/lstk ``` ```bash npm install -g @localstack/lstk ``` Check for correct installation by invoking `lstk --version`. Docker must be installed and running, exactly as before. Homebrew installs shell completions automatically. With the other methods you'll need generate them yourself, using `lstk completion bash|zsh|fish|powershell`. Full instructions are in the [lstk documentation](https://docs.localstack.cloud/aws/developer-tools/running-localstack/lstk/). There is no need to uninstall the `localstack` CLI, as the two can exist side by side. This is useful while you migrate, and necessary if you rely on a feature that is not yet available with `lstk`. However, we recommend against using them at the same time. Each CLI starts and manages its own LocalStack container, so stop whichever is running before you start the other. ## Logging in Both CLIs need a LocalStack license to run the emulator, but they ask for it in different ways. With the `localstack` CLI you copied an auth token out of the web application and stored it with `localstack auth set-token`, or exported `LOCALSTACK_AUTH_TOKEN` yourself. `lstk` replaces that with a browser-based login triggered automatically: simply running `lstk` or `lstk start` takes you through the login flow the first time, opening a browser window for you to approve the request. The credential is then stored securely on your machine. There is no token to copy, and none to keep in your shell profile. Run `lstk login` directly if you want to authenticate ahead of time, or to switch accounts: ```bash lstk login ``` Similarly, use `lstk logout` to remove the auth token from your machine. Continuous integration has no browser, so auth tokens remain the right approach there. Set `LOCALSTACK_AUTH_TOKEN` as a secret in your pipeline, and `lstk` will use it without any login step. Use a CI Auth Token rather than a personal developer token, as described in the [Auth Token documentation](https://docs.localstack.cloud/aws/getting-started/auth-token/#ci-environments). ## The `config.toml` file `lstk` keeps its settings in a TOML file named `config.toml`. This is the central place for describing the LocalStack container you want to run: which emulator to start, which image tag to use, which port to publish, and which environment variables to pass through. Where the `localstack` CLI took all of this as flags and environment variables at start-up, `lstk` reads it from this file on every run. You do not need to write the file by hand. `lstk` creates a default `config.toml` the first time you run it, and `lstk config path` prints the location of the file currently in effect: ```bash lstk config path ``` A bare-bones file looks like this: ```toml [[containers]] type = "aws" # aws, snowflake, or azure tag = "latest" # image tag to run port = "4566" # host port to publish ``` `lstk` looks for `.lstk/config.toml` in the current directory first, and falls back to a user-level file in your personal home directory (such as `/Users/maureen/.config/lstk/config.toml`). A project can therefore carry its own settings in version control, while your personal defaults apply outside of that project. To use a file somewhere else entirely, pass `lstk --config `. The [Configuration parameters](#configuration-parameters) section covers what else you can put in it. ## Starting, stopping, restarting, and upgrading Day-to-day lifecycle management is where the two CLIs line up most closely, and the commands you already know have direct counterparts. The most noticeable difference is that `lstk start` completes only once the emulator is ready to serve requests, so the familiar pattern of starting in the background and then waiting is no longer necessary. Running `lstk` with no arguments does the same thing as `lstk start`. The other difference is presentation. In a terminal, `lstk` renders a compact interactive view of what it is doing. When its output is piped, redirected, or running in CI, it prints plain text instead. You can force the plain output at any time with `--non-interactive`. | With `localstack` | With `lstk` | | --- | --- | | `localstack start`, `localstack start -d` | `lstk start`, or simply `lstk` | | `localstack wait` | Not needed, as `lstk` waits until the emulator is ready. | | `localstack stop` | `lstk stop` | | `localstack restart` | `lstk restart` | | `localstack status docker` | `lstk status` | | `localstack logs -f -n 100` | `lstk logs --follow --tail 100` | | `localstack start -s snowflake` | `lstk start -t snowflake` | | `localstack update localstack-cli` | `lstk update` | `lstk status` is worth a second look for AWS users. Alongside the endpoint, container, version, and uptime, it lists the resources currently deployed in the AWS emulator. That makes it a quick way to confirm that a script or a snapshot did what you expected. Upgrading now involves two separate things. First, `lstk update` upgrades the CLI itself, using whichever method you installed it with, and it also offers the upgrade when you start the emulator. Second, the emulator image is upgraded independently, by choosing an image tag in your `config.toml` file. Use `latest` to track the newest monthly emulator release, or pin a specific version such as `2026.4`. See the command reference for the full set of options. ## Configuration parameters LocalStack's own configuration variables are unchanged. `DEBUG`, `SERVICES`, `PERSISTENCE`, `EXTENSION_AUTO_INSTALL` and everything else in the configuration reference mean exactly what they meant before. What changes is how you get them into the container. For a one-off run, pass the variable on the command line as you always have, with a `LOCALSTACK_` prefix so that `lstk` knows to forward it to the emulator: ```bash LOCALSTACK_DEBUG=1 lstk start ``` For anything you use more than once, put it in the `config.toml` file instead. The file describes the container you want to run, and groups environment variables into named profiles that you can switch on and off: ```toml [[containers]] type = "aws" tag = "latest" port = "4566" env = ["dev"] [env.dev] DEBUG = "1" SERVICES = "s3,sqs" ``` With that in place, `lstk start` is the whole command. The configuration section of the docs describes every available field. ## Infrastructure as code If you deploy infrastructure into LocalStack, you have almost certainly been using the wrapper scripts: `awslocal` for the AWS CLI, and `tflocal`, `cdklocal`, or `samlocal` for Terraform, the CDK, and SAM. Each script existed to point its underlying tool at LocalStack instead of AWS. `lstk` folds all four into subcommands, so there is nothing extra to install and one less thing to keep up to date. | Wrapper script | With `lstk` | | --- | --- | | `awslocal s3 ls` | `lstk aws s3 ls` | | `tflocal init`, `tflocal apply` | `lstk terraform init`, `lstk tf apply` | | `cdklocal bootstrap`, `cdklocal deploy` | `lstk cdk bootstrap`, `lstk cdk deploy` | | `samlocal build`, `samlocal deploy` | `lstk sam build`, `lstk sam deploy` | These subcommands are simply wrappers around the standard commands. You must still install the AWS CLI, Terraform, the CDK, or SAM yourself, and `lstk` runs them with the endpoint, credentials, and region configured to point to LocalStack. Everything you type after the subcommand is passed through untouched, and the output and exit code come back unchanged. In practice, migrating a script means prefixing each of these commands with `lstk`, such as `lstk aws`. ## Snapshots Saving and restoring emulator state is no longer split across two command groups that behave differently. Previously, `localstack state` wrote to a local file, while `localstack pod` published Cloud Pods to the LocalStack platform. `lstk` merges them into a single `snapshot` command group where the destination decides where the snapshot is saved. Use a path for a local file, a `pod:` reference for a Cloud Pod, or an `s3://` location for your own bucket. ```bash lstk snapshot save ./my-state # a local file lstk snapshot save pod:my-baseline # a Cloud Pod lstk snapshot load pod:my-baseline # restore it, starting the emulator if needed ``` Because saving and loading are such common operations, `lstk save` and `lstk load` are available as shorthands. | With `localstack` | With `lstk` | | --- | --- | | `localstack state export`, `localstack state import` | `lstk snapshot save `, `lstk snapshot load ` | | `localstack pod save`, `localstack pod load` | `lstk snapshot save pod:`, `lstk snapshot load pod:` | | `localstack pod list`, `localstack pod versions` | `lstk snapshot list`, `lstk snapshot versions pod:` | | `localstack pod inspect`, `localstack pod delete` | `lstk snapshot show pod:`, `lstk snapshot remove pod:` | | `localstack state reset` | `lstk reset` | The underlying concepts are unchanged. Every save to an existing Cloud Pod creates a new version, and you can load an earlier version by appending the version number to the reference, as in `pod:my-baseline:3`. The merge strategies that control how a loaded snapshot combines with running state are unchanged, and are selected with `--merge`. Snapshots can be restricted to a subset of services with `--services`. Two related features are also available. The first is automatic persistence, where the emulator saves and restores its own state across restarts. It is enabled with `lstk start --persist`, the equivalent of the `PERSISTENCE` variable. The second is auto-loading. A snapshot can be loaded every time the emulator starts, by naming it in your configuration file. That replaces the old auto-load behavior. The snapshots documentation covers local snapshots, Cloud Pods, S3 storage, merging, and persistence in detail. ## Accessing remote emulators Not every LocalStack instance is started by `lstk`. You might instead use `docker-compose.yml`, or run it on a remote machine, or share a single deployment with others in your team. `lstk` works with these too. The global `--endpoint-url` option instructs `lstk` to communicate with an emulator that wasn't started locally by `lstk`: ```bash lstk --endpoint-url http://localhost:4566 status lstk --endpoint-url https://localstack.example.com aws s3 ls ``` If you target the same instance repeatedly, set `LSTK_ENDPOINT_URL` in your environment: ```bash export LSTK_ENDPOINT_URL=https://localstack.example.com lstk aws s3 ls lstk snapshot save pod:my-baseline ``` Every `lstk` subcommand that communicates with a running emulator works as expected, including `lstk status`, the AWS CLI and infrastructure-as-code wrappers, and the snapshot commands. However, commands that manage the container itself, such as starting or stopping it or clearing its volume, do not support the `--endpoint-url` option. This is because `lstk` did not create the container and does not control its lifecycle. ## Using `lstk` with Continuous Integration Continuous integration is where LocalStack does much of its work, and the shape of a pipeline does not change when you move to `lstk`. You still install a CLI, start the emulator, run your tests against it, and let the job tear everything down at the end. The steps are simply shorter than they used to be. There is no dedicated GitHub Action for `lstk`. The existing setup-localstack Action installs the legacy CLI, so for now you install `lstk` in a step of your own and call it directly. The npm package is often the most convenient option on a hosted runner. The pre-built binaries suit images where Node.js is not available. Authentication is the only part that differs from your personal machine. `lstk login` needs a browser, so a CI pipeline must supply a CI Auth Token through the `LOCALSTACK_AUTH_TOKEN` environment variable instead, normally from your CI system secret store. No login step is required. A GitHub Actions job then looks like this: ```yaml - name: Install lstk run: npm install -g @localstack/lstk - name: Start LocalStack env: LOCALSTACK_AUTH_TOKEN: ${{ secrets.LOCALSTACK_AUTH_TOKEN }} run: lstk start - name: Run tests run: | lstk aws s3 mb s3://test-bucket make test - name: Reset the emulator between test suites run: lstk reset --force - name: Run integration tests run: make integration-test ``` A couple of practices from the legacy `localstack` CLI are being dropped here. `lstk start` returns only once the emulator is ready, removing the need to explicitly wait for it to become ready. Additionally, subcommand output will be in plain text (not interactive) when running in CI, so commands that would normally ask for confirmation, such as `lstk reset` and `lstk volume clear`, require an additional `--force` option, as in the reset step above. If your pipeline seeds LocalStack with fixtures or infrastructure before the tests run, snapshots are worth a look. Saving a snapshot once and loading it at the start of each job with `lstk snapshot load` is usually much faster than re-running Terraform or a long list of AWS CLI commands. The same three steps apply to GitLab CI, CircleCI, Jenkins, and the others. Only the syntax around them changes. See the CI/CD documentation for the general guidance, and the CI pipelines section for per-platform examples. ## Migrating your existing configuration Much of the work in a migration is not learning new commands but relocating settings. Start by gathering everything that currently configures LocalStack: flags on your `localstack start` command line, environment variables set in a shell profile, alias, or Makefile, any `~/.localstack` profile files, and the environment section of a `docker-compose.yml` file if you have one. Nearly all of it maps onto fields in the `lstk` configuration file. | Previously | In `config.toml` | | --- | --- | | `localstack start -e DEBUG=1 -e SERVICES=s3,sqs` | An `[env.]` profile, referenced by `env` on the container | | `localstack start -v ./init.sh:/etc/localstack/init/ready.d/init.sh` | `volumes = ["./init.sh:/etc/localstack/init/ready.d/init.sh"]` | | `IMAGE_NAME=localstack-enterprise` | `image = "localstack-enterprise"` | | A pinned LocalStack version | `tag = "2026.4"` | | `LOCALSTACK_VOLUME_DIR=./volume` | `volume = "./volume"` | | `localstack start -s snowflake` | `type = "snowflake"` | | `localstack start --host-dns` | `expose_ports = [53]` | | `AUTO_LOAD_POD=my-baseline` | `snapshot = "pod:my-baseline"` | To illustrate, the following before-and-after shows the mapping. Where you previously ran: ```bash DEBUG=1 PERSISTENCE=1 localstack start -d \ -e SERVICES=s3,sqs \ -v ./init.sh:/etc/localstack/init/ready.d/init.sh localstack wait ``` you would now write a `.lstk/config.toml` in the project. ```toml [[containers]] type = "aws" port = "4566" env = ["dev"] volumes = ["./init.sh:/etc/localstack/init/ready.d/init.sh"] [env.dev] DEBUG = "1" SERVICES = "s3,sqs" PERSISTENCE = "1" ``` then start it with `lstk start`. Because the `config.toml` file lives in the repository, everyone on the team gets the same environment. Your CI jobs also pick up the same file. The existing `CONFIG_PROFILE` mechanism in the `localstack` CLI and its `~/.localstack/*.env` files have no direct equivalent with `lstk`. The recommended replacement is a per-project `.lstk/config.toml`. If you need to switch between several configurations for the same project, keep them as separate files and choose between them with `lstk --config `. ## Unsupported features A handful of capabilities have not moved to `lstk`, although they may do so in a future release. In most cases the recommendation is to keep the `localstack` CLI installed for that one task, but migrate the rest of your workflow to `lstk`. The table below is deliberately high-level, and you should follow the linked documentation for more detail. If you are an active user of one of these unsupported features, please contact LocalStack Support. | Not supported in `lstk` | Recommended alternative | | --- | --- | | [Ephemeral Instances](https://docs.localstack.cloud/aws/developer-tools/cloud-sandbox/ephemeral-instances/) | Use the `localstack` CLI or the LocalStack Console. Ephemeral Instances are a preview feature and are not yet available in `lstk`. | | [AWS Replicator](https://docs.localstack.cloud/aws/developer-tools/aws-replicator/) | Use the `localstack` CLI, version 4.2.0 or newer. Also a preview feature. | | [IAM Policy Stream](https://docs.localstack.cloud/aws/developer-tools/security-testing/iam-policy-stream/) | Use the `localstack` CLI, or the dashboard in the LocalStack Console. IAM enforcement itself needs no CLI support and can be switched on with configuration variables, which `lstk` passes through as usual. | | [Extensions](https://docs.localstack.cloud/aws/customization/integrations/extensions/) | Use the `localstack` CLI to install, manage, and develop extensions. To install one at start-up without the CLI, set `EXTENSION_AUTO_INSTALL` in your configuration. | | [Cloud Pod](https://docs.localstack.cloud/aws/developer-tools/snapshots/cloud-pods/) publishing, snapshot encryption, and version messages | These features remain available only in the `localstack` CLI. `lstk` covers the everyday save, load, list, and inspect operations. | | [Cloud Pod](https://docs.localstack.cloud/aws/developer-tools/snapshots/cloud-pods/) remotes other than S3 | Use the `localstack` CLI. `lstk` supports the LocalStack platform and your own S3 bucket, with the location passed inline rather than registered in advance. | | [Host DNS setup](https://docs.localstack.cloud/aws/customization/networking/dns-server/) on Linux | Use the `localstack` CLI for the one-off resolver configuration. Publishing the DNS port itself is supported through the `expose_ports` setting. | | Arbitrary Docker options, such as custom networks | Use `docker-compose.yml` when you need full control of the container and the many Docker configuration options. The `lstk` configuration file covers only the basic features, such as the image, tag, port, exposed ports, volumes, and environment variables. | | Shell access to the container (`localstack ssh`) | Use `docker exec -it localstack-aws bash`. Note that `lstk` names its container `localstack-[aws,azure,snowflake]`. | | [GitHub Action](https://github.com/localstack/setup-localstack) | There is no GitHub Action for `lstk` yet, and the existing `setup-localstack` Action installs the `localstack` CLI. Install `lstk` in a step of your own instead, as shown in [Using lstk with Continuous Integration](#using-lstk-with-continuous-integration). | ## Next steps - The lstk reference documents every command, flag, and configuration field. - The configuration options page lists the LocalStack variables themselves, which are unchanged. - The snapshots section covers local snapshots, Cloud Pods, persistence, and merging. - The deprecated wrapper scripts and legacy CLI pages remain available for as long as you need them. # lstk Setup & Maintenance > The setup, config, and update commands, and running lstk in offline or enterprise environments. ## `setup` Set up CLI integration for an emulator type. `lstk setup` is a grouping command with no action of its own; the work is done by its subcommands, `setup aws` and `setup azure`. ```bash lstk setup aws lstk setup azure ``` ### `setup aws` Create or update a `localstack` profile in `~/.aws/config` and `~/.aws/credentials` so the AWS CLI and SDKs can target LocalStack. ```bash lstk setup aws lstk setup aws --force ``` | Option | Description | |:----------|:-----------------------------------------------------------------------------------------| | `--force` | Overwrite an existing `localstack` profile whose values differ, and skip the confirmation prompt. | On an interactive terminal it prompts (Y/n) before making changes. In non-interactive mode (piped output, CI, or `--non-interactive`) it writes the profile with defaults without prompting and exits `0`; a failed write or check returns a non-zero exit code so automation notices. Overwriting an existing `localstack` profile whose values differ requires `--force` (which also skips the interactive prompt); creating a fresh profile, completing a partial one, or leaving an already-correct profile in place never needs it. It writes the following profile (existing unrelated profiles are preserved): ```ini # ~/.aws/config [profile localstack] region = us-east-1 output = json endpoint_url = http://localhost.localstack.cloud:4566 # ~/.aws/credentials [localstack] aws_access_key_id = test aws_secret_access_key = test ``` Afterwards, target LocalStack by passing `--profile localstack` or exporting `AWS_PROFILE`: ```bash export AWS_PROFILE=localstack aws s3 ls ``` The endpoint host is resolved the same way as for [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#endpoint-resolution) (probing `localhost.localstack.cloud` and falling back to `127.0.0.1`), and [`LOCALSTACK_HOST`](/aws/developer-tools/running-localstack/lstk/automation/#environment-variables) overrides the host and port written into the profile. The port comes from your AWS emulator's configured `port` (default `4566`); if no `aws` emulator is configured, the command fails with `no aws emulator configured`. If the `localstack` profile is already configured correctly, `lstk` reports `LocalStack AWS profile is already configured.` and makes no changes. :::note The former `lstk config profile` command has been removed; use `lstk setup aws`. ::: ### `setup azure` Prepare an isolated Azure CLI configuration directory (under the `lstk` config dir, via `AZURE_CONFIG_DIR`) that routes [`lstk az`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#az) commands to the LocalStack Azure emulator. Your global `~/.azure` configuration is left untouched. ```bash lstk setup azure # alias: lstk setup az ``` `setup azure` registers a custom Azure cloud (`LocalStack`) whose endpoints point at the LocalStack Azure emulator, activates it, disables Azure CLI instance discovery and telemetry, and performs a one-time dummy service-principal login — all inside a dedicated config directory under the `lstk` config dir (via `AZURE_CONFIG_DIR`). It requires the `az` CLI to be installed and a running LocalStack Azure emulator. Run this once; afterwards use `lstk az ` to run Azure CLI commands against LocalStack. To instead redirect your **global** `az` (so existing scripts run unmodified against LocalStack), see [`lstk az start-interception`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#global-interception-optional). ## `config` Manage CLI configuration. `config` has no behavior of its own; run it with a subcommand. ### `config path` Print the resolved path to the active `config.toml`. ```bash lstk config path ``` This subcommand is read-only: it never creates or initializes a config file. If `--config ` is set, it prints that path verbatim. Otherwise it prints the already-loaded config path, the first existing config in the search order, or the path where a config would be created on first run. ## `update` Check for and apply updates to the `lstk` CLI itself. `lstk` auto-detects how it was installed (Homebrew, npm, or direct binary) and updates using that same method. Development builds (version `dev`) are skipped, and updates are checked against the latest [GitHub release](https://github.com/localstack/lstk/releases/latest). ```bash lstk update [options] ``` | Option | Description | |:--------------------|:------------------------------------------------------------| | `--check` | Check for updates without installing them | | `--non-interactive` | Use plain output instead of the TUI (update logic unchanged) | | `--json` | Emit the result as a JSON envelope (see [Structured output](/aws/developer-tools/running-localstack/lstk/automation/#structured-output)). With `--check`, `data` reports `currentVersion`/`latestVersion`/`updateAvailable`; after an applied update, `updatedVersion`/`updated`/`method`. | Examples: ```bash # Check for updates without installing lstk update --check # Update to the latest version lstk update # Update with plain (non-TUI) output lstk update --non-interactive ``` By install method: - **Homebrew** (binary under a `Caskroom` path): runs `brew upgrade localstack/tap/lstk`. - **npm** (binary under `node_modules`): runs `npm install -g @localstack/lstk@latest`. - **Binary** (anything else): downloads the release asset for your OS/arch from GitHub, verifies its SHA-256 against the release's `checksums.txt` (a missing, malformed, or mismatched checksum aborts the update), extracts it, and replaces the running executable in place. With `--check`, `lstk` only reports whether a newer version is available and exits without downloading or installing anything. :::note Set `LSTK_GITHUB_TOKEN` to send an authenticated GitHub request and avoid API rate limits during update checks. It is optional; updates also work unauthenticated. ::: If more than one `lstk` installation is found on your `PATH` (for example a Homebrew binary and an npm one), `lstk update` and the start-time update notification print a warning listing each location, its install method, and which one is currently running, so you can tell which binary an update will actually replace. ### Update notification on start Separately from `lstk update`, `lstk` checks for a newer version when you run `lstk start` (the default command), using a short timeout that fails silently if GitHub is unreachable. In an interactive terminal, when an update is available `lstk` prints the new version and a release-notes link, then prompts: ```text Update lstk to latest version? > Update now [U] Remind me next time [R] Skip this version [S] ``` - **Update now [U]**: downloads and applies the update, then asks you to re-run your command. - **Remind me next time [R]**: does nothing; you are reminded on the next run. - **Skip this version [S]**: records the version in `config.toml` so you are not prompted about it again. In non-interactive mode the notification is not a prompt — `lstk` emits a single note (`Update available: (run lstk update)`) and continues. When you choose **Skip this version**, `lstk` writes the skipped version under a `[cli]` table: ```toml [cli] update_skipped_version = "0.5.0" ``` While this value matches the latest available version, the start-time update notification for that version is suppressed. This key is managed automatically and is not intended to be edited by hand. ## Offline and enterprise environments There is no `--offline` flag. Instead, `lstk` degrades gracefully when common enterprise blockers (Docker Hub unreachable, a proxy/TLS interceptor, or an unreachable license server) prevent an internet request: - **Image pull**: if the image pull fails but the image is already present locally, `lstk` warns and uses the local image instead of failing. In interactive mode you can also press Esc to abort an in-progress pull and fall back to the local image. - **License pre-flight**: when the pinned image is already present locally, `lstk` skips its pre-flight license check so a fully offline start is not blocked; the emulator validates the license itself once it starts. When a check does run, a transport-level failure (offline, proxy, or certificate error) is treated as non-fatal and the emulator validates the license instead. A definitive server rejection (HTTP 400/401/403) is handled differently: `lstk` drops the cached license and, in an interactive terminal, offers to log in again and retries the start once with the refreshed credentials (a rejected token often just predates a license purchase or plan change); in non-interactive mode it fails with an error pointing at `lstk logout && lstk login` or a valid `LOCALSTACK_AUTH_TOKEN`. The pre-flight is also skipped — with a warning — when the license server does not recognize the image *tag format* (for example a `dev` nightly or a custom internal-mirror tag): that is not a verdict on the license, so `lstk` defers to the emulator's own startup check rather than blocking the start. - **Telemetry and update checks** are best-effort and fail silently when offline. Pair this behavior with a custom [`image`](/aws/developer-tools/running-localstack/lstk/configuration/#custom-container-image) that points at an internal-registry mirror or a locally loaded image to run `lstk` in an air-gapped environment. # lstk Snapshots > Save, load, list, remove, and show emulator snapshots with lstk, including S3 remotes. ## `snapshot` Manage emulator snapshots. A snapshot captures the running emulator's state, either as a local file on disk, as a Cloud Pod on the LocalStack platform, or in your own S3 bucket. The `snapshot` command groups six subcommands — `save`, `load`, `list`, `remove`, `show`, and `versions`. The first two are also exposed as the top-level aliases `lstk save` and `lstk load`. :::note Snapshots are best supported on the **AWS emulator**. `snapshot save`/`load` (and the `save`/`load` aliases) also work for the Snowflake emulator, but its snapshot support is experimental and not fully tested — `lstk` prints a warning such as `Snapshot support for the snowflake emulator is experimental and not fully tested.` The Azure emulator does not support snapshots. ::: ## `snapshot save` Save a snapshot of the running emulator's state. The emulator must already be running; this command does **not** auto-start it. ```bash # Auto-named snapshot file in the current directory lstk snapshot save # Save to a specific local path lstk snapshot save ./my-snapshot # Save to a Cloud Pod on the LocalStack platform (requires auth) lstk snapshot save pod:my-baseline # Save to your own S3 bucket (pod name is auto-generated if omitted) lstk snapshot save my-pod s3://my-bucket/prefix # Limit the snapshot to a subset of services lstk snapshot save --services s3,lambda ``` The optional `[destination]` argument takes one of these forms: | Destination | Description | |:---------------------------------|:--------------------------------------------------------------------------------------------------| | (omitted) | Auto-generates a timestamped snapshot file in the current directory (`./snapshot--.snapshot`). | | local path | Writes a snapshot archive to that path. The `.snapshot` extension is forced. | | `pod:` | Saves a Cloud Pod to the LocalStack platform. Requires authentication. | | ` s3://bucket/prefix` | Saves to your own S3 bucket. The pod name is a separate positional (auto-generated when omitted). See [S3 remotes](#s3-remotes). | Pod operations require an auth token (`LOCALSTACK_AUTH_TOKEN` or a prior `lstk login`); local-file snapshots do not. Every save to an existing `pod:` snapshot creates a new **version** rather than replacing it; use [`snapshot versions`](#snapshot-versions) to list them and [`snapshot load`](#snapshot-load)/[`snapshot show`](#snapshot-show) with a `pod::` ref to act on a specific one. `save` itself rejects a version suffix (you cannot save "as version 3"). By default a snapshot captures every service's state. Pass `-s`/`--services` with a comma-separated list to limit it to a subset; this applies uniformly to local files, `pod:` Cloud Pods, and `s3://` remotes. | Option | Description | |:--------------------|:------------------------------------------------------------------------------------------------| | `--services `, `-s ` | Comma-separated list of services to include in the snapshot (all services by default). Applies to local, `pod:`, and `s3://` destinations. | | `--profile ` | AWS profile to read S3 credentials from (used only for `s3://` destinations). Defaults to `AWS_*` env vars, then `AWS_PROFILE`. | ## `snapshot load` Load a snapshot into the emulator, **auto-starting it first** if it is not already running. ```bash # Load a local snapshot by path or name lstk snapshot load my-baseline lstk snapshot load ./checkpoint # Load from a Cloud Pod (requires auth; latest version) lstk snapshot load pod:my-baseline # Load a specific version of a Cloud Pod lstk snapshot load pod:my-baseline:3 # Load from your own S3 bucket (pod name is required) lstk snapshot load my-pod s3://my-bucket/prefix # Control how the snapshot merges with running state lstk snapshot load pod:my-baseline --merge=overwrite # Preview what a Cloud Pod load would change, without applying it lstk snapshot load pod:my-baseline --dry-run ``` The `REF` argument is required and identifies a local path/name or a `pod:` Cloud Pod. For a Cloud Pod you can append a version (`pod::`) to load an older version; the latest is used when no version is given. To load from S3, pass the pod name followed by an `s3://bucket/prefix` location (see [S3 remotes](#s3-remotes)). | Option | Description | |:---------------------|:------------------------------------------------------------------------------------------------------------| | `--merge ` | How the loaded state combines with running state. One of `account-region-merge` (default), `overwrite`, `service-merge`. | | `--dry-run` | Preview the resource additions and modifications the load would produce, per service, without changing any state. Supported for `pod:` refs only; requires a running emulator (it does not auto-start one). | | `--profile ` | AWS profile to read S3 credentials from (used only for `s3://` sources). Defaults to `AWS_*` env vars, then `AWS_PROFILE`. | - `account-region-merge` (default): the snapshot wins on any `(service, account, region)` overlap. - `overwrite`: running state is reset first, then the snapshot is imported onto a clean state. - `service-merge`: the snapshot wins per resource; non-overlapping resources are combined. Set [`LSTK_MERGE_STRATEGY`](/aws/developer-tools/running-localstack/lstk/automation/#environment-variables) to change the default strategy used when `--merge` is not passed; an explicit `--merge` always wins. Pass `--dry-run` with a `pod:` ref to preview a load before committing to it: `lstk` queries the platform and prints, per service, how many resources the snapshot would add or modify under the chosen merge strategy, without touching running state. It is supported for `pod:` refs only (other refs are rejected) and requires the emulator to already be running, since it does not auto-start one. ### `save`/`load` aliases `snapshot save` and `snapshot load` are also exposed as the top-level aliases `lstk save` and `lstk load`. The aliases behave identically: ```bash lstk save pod:my-baseline lstk load ./checkpoint ``` ## `snapshot list` List the Cloud Pod snapshots available on the LocalStack platform. By default, only snapshots you created are listed; pass `--all` to include every snapshot in your organization. This subcommand operates on Cloud Pods, so it requires authentication. ```bash # Snapshots you created lstk snapshot list # Every snapshot in your organization lstk snapshot list --all # List snapshots in your own S3 bucket (requires a running emulator) lstk snapshot list s3://my-bucket/prefix ``` Passing an `s3://bucket/prefix` location lists snapshots stored in your own S3 bucket instead of the platform (see [S3 remotes](#s3-remotes)). Unlike the platform listing, this queries the emulator, so it requires a running emulator. | Option | Description | |:--------------------|:--------------------------------------------------------------| | `--all` | List all snapshots in your organization, not just your own. | | `--profile ` | AWS profile to read S3 credentials from (used only with an `s3://` location). Defaults to `AWS_*` env vars, then `AWS_PROFILE`. | ## `snapshot remove` Delete a Cloud Pod snapshot from the LocalStack platform. Only cloud snapshots (the `pod:` prefix) can be removed; local snapshots are plain files you delete yourself. This operation cannot be undone. ```bash lstk snapshot remove pod:my-baseline # Skip the confirmation prompt (required in non-interactive mode) lstk snapshot remove pod:my-baseline --force ``` The required `REF` argument must be a `pod:` Cloud Pod reference. | Option | Description | |:----------|:-------------------------------------------------------------------------| | `--force` | Skip the confirmation prompt. Required when running non-interactively. | ## `snapshot show` Show metadata for a single Cloud Pod snapshot on the LocalStack platform: its name, created date, size, LocalStack version, message, the services it contains, and per-service resource counts (resource counts render only when the platform has them for that snapshot). This subcommand is cloud-only and requires authentication. ```bash # Latest version lstk snapshot show pod:my-baseline # A specific version lstk snapshot show pod:my-baseline:3 ``` The required `REF` argument must be a `pod:` Cloud Pod reference. It defaults to the latest version; append `:` to inspect an older one. Use [`snapshot versions`](#snapshot-versions) to see which versions exist. ## `snapshot versions` List the version history of a Cloud Pod on the LocalStack platform. Every save to an existing pod adds a new version; this prints each version's number, created date, LocalStack version, and services. This subcommand is cloud-only and requires authentication. ```bash lstk snapshot versions pod:my-baseline ``` The required `REF` argument must be a `pod:` Cloud Pod reference. Only Cloud Pods have versions — local files and `s3://` remotes do not, and passing a version suffix to `versions` is rejected. Act on a specific version elsewhere by appending it to the ref, e.g. `lstk snapshot load pod:my-baseline:3` or `lstk snapshot show pod:my-baseline:3`. ## S3 remotes `snapshot save`, `load`, and `list` can target a snapshot stored in your **own S3 bucket** by passing an `s3://bucket/prefix` location. The pod name (the snapshot's identity within the bucket) is a positional separate from the `s3://` location — required for `load`, auto-generated for `save` when omitted, and unused for `list`. ```bash lstk snapshot save my-pod s3://my-bucket/prefix lstk snapshot load my-pod s3://my-bucket/prefix lstk snapshot list s3://my-bucket/prefix ``` Credentials follow AWS CLI precedence: `--profile ` wins, otherwise the static `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` (plus optional `AWS_SESSION_TOKEN`) environment variables, otherwise the profile named by `AWS_PROFILE`. Only static credentials are supported (no SSO, assume-role, or `credential_process`), and credentials must never be embedded in the URL. `lstk` runs a pre-flight check that the target bucket exists and errors out rather than letting the emulator auto-create a bucket on a typo. Because the transfer is performed by the emulator (not the CLI), S3 remotes require a **running emulator**, and `list s3://…` in particular queries the emulator rather than the platform API. :::note `remove` and `show` do not support S3; they operate on Cloud Pods only. ::: # LocalStack MCP Server > Use the LocalStack MCP Server to manage your local cloud development environment through AI-powered MCP clients. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction The [LocalStack MCP Server](https://github.com/localstack/localstack-mcp-server) is a [Model Context Protocol](https://modelcontextprotocol.io/) (MCP) server that connects MCP-compatible clients to your LocalStack environment. It enables AI agents to manage the full local cloud development lifecycle: starting containers, deploying infrastructure, analyzing logs, injecting chaos faults, managing state snapshots, and running AWS CLI commands, all through natural language prompts. ## Prerequisites Before configuring the MCP server, ensure the following are installed and available on your system `PATH`: - [Node.js](https://nodejs.org/en/download/) (v22.x or later) to run `npx`. - [Docker](https://docs.docker.com/get-docker/) to manage the LocalStack container. - A [LocalStack Auth Token](/aws/getting-started/auth-token/) configured as `LOCALSTACK_AUTH_TOKEN`. All MCP server tools require a valid Auth Token. - [`cdklocal`](https://github.com/localstack/aws-cdk-local), [`tflocal`](https://github.com/localstack/terraform-local), or [`samlocal`](https://github.com/localstack/aws-sam-cli-local) if you plan to use the infrastructure deployment tool. (**optional**) - [Snowflake CLI](https://docs.snowflake.com/en/developer-guide/snowflake-cli/index) (`snow`) if you plan to use the Snowflake client tool. (**optional**) :::note The MCP server currently manages LocalStack via the legacy `localstack` CLI and the `awslocal`, `cdklocal`, `tflocal`, and `samlocal` wrapper scripts. `lstk` support is not yet available for the MCP server. ::: ## Installation The LocalStack MCP Server is published on npm as [`@localstack/localstack-mcp-server`](https://www.npmjs.com/package/@localstack/localstack-mcp-server). The recommended approach is to let your MCP client run the server via `npx`, which downloads and caches the package automatically. :::note All MCP server tools require `LOCALSTACK_AUTH_TOKEN` to be set. You must include it in the `env` block of your configuration. You can get your Auth Token from the [LocalStack Web App](https://app.localstack.cloud). ::: The quickest way to get started with the MCP server is to use the interactive setup wizard: ```bash npx -y @localstack/localstack-mcp-server init ``` The wizard detects your installed clients, asks how you want to run the server, and writes the configuration for you. You need a valid [Auth Token](/aws/getting-started/auth-token/) to configure the server. For manual setup of the MCP server, choose your MCP client below for setup instructions. Click the button below to install the LocalStack MCP Server in Cursor automatically: [![Install MCP Server](https://cursor.com/deeplink/mcp-install-dark.svg)](https://cursor.com/en/install-mcp?name=localstack-mcp-server&config=eyJjb21tYW5kIjoibnB4IC15IEBsb2NhbHN0YWNrL2xvY2Fsc3RhY2stbWNwLXNlcnZlciJ9) After installing, open `~/.cursor/mcp.json` and add the `env` block with your Auth Token: ```json { "mcpServers": { "localstack-mcp-server": { "command": "npx", "args": ["-y", "@localstack/localstack-mcp-server"], "env": { "LOCALSTACK_AUTH_TOKEN": "" } } } } ``` Restart Cursor after saving. The server appears in **Cursor Settings > MCP** once detected. Add the following to your VS Code user settings JSON (`settings.json`) or workspace `.vscode/mcp.json`: ```json { "mcp": { "servers": { "localstack-mcp-server": { "command": "npx", "args": ["-y", "@localstack/localstack-mcp-server"], "env": { "LOCALSTACK_AUTH_TOKEN": "" } } } } } ``` You can also run the command **MCP: Add Server** from the Command Palette and paste the server command. MCP support requires the [GitHub Copilot Chat](https://marketplace.visualstudio.com/items?itemName=GitHub.copilot-chat) extension. Run the following command in your terminal: ```bash claude mcp add localstack-mcp-server \ -e LOCALSTACK_AUTH_TOKEN= \ -- npx -y @localstack/localstack-mcp-server ``` Claude Code stores this in its project-level configuration and starts the server automatically when you open a conversation. Add the following to your OpenCode configuration file (`opencode.json`): ```json { "mcpServers": { "localstack-mcp-server": { "command": "npx", "args": ["-y", "@localstack/localstack-mcp-server"], "env": { "LOCALSTACK_AUTH_TOKEN": "" } } } } ``` Add the following to your Amazon Q MCP configuration at `~/.aws/amazonq/mcp.json`: ```json { "mcpServers": { "localstack-mcp-server": { "command": "npx", "args": ["-y", "@localstack/localstack-mcp-server"], "env": { "LOCALSTACK_AUTH_TOKEN": "" } } } } ``` For any MCP-compatible client, configure a stdio server with: - **Command:** `npx` - **Arguments:** `["-y", "@localstack/localstack-mcp-server"]` - **Environment:** `LOCALSTACK_AUTH_TOKEN` set to your token If your client uses a JSON configuration file, the entry follows this format: ```json { "mcpServers": { "localstack-mcp-server": { "command": "npx", "args": ["-y", "@localstack/localstack-mcp-server"], "env": { "LOCALSTACK_AUTH_TOKEN": "" } } } } ``` Refer to your client's documentation for the exact location of its MCP configuration file. ### Connecting to a custom LocalStack endpoint By default the MCP server connects to `http://localhost:4566`. If your LocalStack instance runs on a different host or port, set the following environment variables in the `env` block: | Variable | Default | Description | | --------------------- | ----------- | ----------------------------------- | | `LOCALSTACK_HOSTNAME` | `localhost` | Hostname of the LocalStack instance | | `LOCALSTACK_PORT` | `4566` | Port of the LocalStack instance | You can also pass any [LocalStack configuration variable](/aws/customization/configuration-options/) through the `env` block. These are forwarded to the container when the `localstack-management` tool starts it. ```json { "mcpServers": { "localstack-mcp-server": { "command": "npx", "args": ["-y", "@localstack/localstack-mcp-server"], "env": { "LOCALSTACK_AUTH_TOKEN": "", "LOCALSTACK_HOSTNAME": "my-host", "LOCALSTACK_PORT": "4566" } } } } ``` ## Tools The MCP server exposes the following tools that your AI agent can call. Each tool runs pre-flight checks (verifying the CLI is available, the container is running, and the Auth Token is present) and returns structured responses. ### `localstack-management` Manage the LocalStack runtime lifecycle for both the AWS emulator and the Snowflake emulator. | Parameter | Type | Required | Description | | --------- | ------------------------------------------ | -------- | --------------------------------------------- | | `action` | `start` \| `stop` \| `restart` \| `status` | Yes | The operation to perform | | `service` | `aws` \| `snowflake` | No | The stack to manage (default: `aws`) | | `envVars` | `Record` | No | Extra environment variables passed on `start` | **Example prompts:** - "Start my LocalStack container." - "What's the current status of LocalStack?" - "Start the Snowflake emulator." ### `localstack-deployer` Deploy or destroy infrastructure on LocalStack using CDK, Terraform, SAM, or CloudFormation. | Parameter | Type | Required | Description | | -------------- | --------------------------------------------------------- | --------------------------------------- | -------------------------------------------------------------------------------------------------- | | `action` | `deploy` \| `destroy` \| `create-stack` \| `delete-stack` | Yes | The deployment operation | | `projectType` | `cdk` \| `terraform` \| `sam` \| `auto` | No | Framework to use (default: `auto`, detected from project files) | | `directory` | `string` | Yes (for `deploy`/`destroy`) | Path to the project directory | | `variables` | `Record` | No | Variables passed as Terraform `-var` flags, CDK `--context` values, or SAM `--parameter-overrides` | | `stackName` | `string` | Yes (for `create-stack`/`delete-stack`) | CloudFormation/SAM stack name | | `templatePath` | `string` | No | Path to a CloudFormation/SAM template | | `s3Bucket` | `string` | No | S3 bucket for SAM deployments (if omitted, SAM uses `--resolve-s3`) | | `resolveS3` | `boolean` | No | For SAM deployments, whether to use `--resolve-s3` when no `s3Bucket` is provided | | `saveParams` | `boolean` | No | For SAM deployments, whether to persist resolved parameters to `samconfig.toml` | **Example prompts:** - "Deploy my CDK project in the `infra/` directory on LocalStack." - "Destroy the Terraform deployment in `./terraform`." - "Deploy my SAM application in `./sam-app`." - "Create a CloudFormation stack named `my-stack` from `template.yaml`." :::note The `deploy` and `destroy` actions require [`cdklocal`](https://github.com/localstack/aws-cdk-local), [`tflocal`](https://github.com/localstack/terraform-local), or [`samlocal`](https://github.com/localstack/aws-sam-cli-local) to be installed on your system `PATH`, depending on the project type. The `create-stack` and `delete-stack` actions run `awslocal` inside the LocalStack container and require the container to be running. ::: ### `localstack-logs-analysis` Analyze LocalStack logs to find errors, summarize API activity, or inspect raw output. | Parameter | Type | Required | Description | | -------------- | --------------------------------------------- | -------- | ---------------------------------------------------------------- | | `analysisType` | `summary` \| `errors` \| `requests` \| `logs` | No | Type of analysis (default: `summary`) | | `lines` | `number` | No | Number of log lines to fetch (default: `2000`) | | `service` | `string` | No | Filter by AWS service name | | `operation` | `string` | No | Filter by API operation (used with `service` in `requests` mode) | | `filter` | `string` | No | Keyword filter (used with `logs` mode only) | **Example prompts:** - "Analyze LocalStack logs for any errors." - "Show me the S3 API requests from the last 500 log lines." - "Get the raw LocalStack logs filtered by 'lambda'." ### `localstack-iam-policy-analyzer` Configure IAM enforcement and generate IAM policies from access denials in the logs. | Parameter | Type | Required | Description | | --------- | ------------------------------------------------ | -------------------- | ------------------------ | | `action` | `set-mode` \| `analyze-policies` \| `get-status` | Yes | The operation to perform | | `mode` | `ENFORCED` \| `SOFT_MODE` \| `DISABLED` | Yes (for `set-mode`) | IAM enforcement level | **Example prompts:** - "Enable IAM enforcement in soft mode on LocalStack." - "Analyze the logs for IAM policy violations and generate the required policies." - "What's the current IAM enforcement status?" ### `localstack-chaos-injector` Inject faults and network latency into LocalStack services to test application resilience. | Parameter | Type | Required | Description | | ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------- | ---------------------------------------------- | | `action` | `inject-faults` \| `add-fault-rule` \| `remove-fault-rule` \| `get-faults` \| `clear-all-faults` \| `inject-latency` \| `get-latency` \| `clear-latency` | Yes | The chaos operation | | `rules` | `Array` | Yes (for `inject-faults`, `add-fault-rule`, `remove-fault-rule`) | Fault rules to inject/modify | | `latency_ms` | `number` | Yes (for `inject-latency`) | Latency in milliseconds to add to all requests | Each **fault rule** can include: | Field | Type | Description | | ------------- | ---------------------------------------- | ----------------------------------------- | | `service` | `string` | Target AWS service (e.g., `s3`, `lambda`) | | `region` | `string` | Target region (e.g., `us-east-1`) | | `operation` | `string` | Target API operation (e.g., `PutObject`) | | `probability` | `number` (0-1) | Probability of the fault triggering | | `error` | `{ statusCode?: number, code?: string }` | Error response to return | **Example prompts:** - "Inject a 500 error for all Lambda Invoke calls in us-east-1 with 100% probability." - "Add 2000ms of network latency to all LocalStack requests." - "Clear all chaos faults." ### `localstack-cloud-pods` Save, load, and manage LocalStack state snapshots using [Cloud Pods](/aws/developer-tools/snapshots/cloud-pods/). | Parameter | Type | Required | Description | | ---------- | --------------------------------------- | -------------------------------- | ------------------------------------------------------------------------------------ | | `action` | `save` \| `load` \| `delete` \| `reset` | Yes | The state management operation | | `pod_name` | `string` | Yes (for `save`/`load`/`delete`) | Name of the Cloud Pod (alphanumeric, dots, hyphens, underscores; max 128 characters) | **Example prompts:** - "Save the current LocalStack state as a Cloud Pod named `my-app-state`." - "Load the Cloud Pod `my-app-state`." - "Reset the LocalStack state completely." ### `localstack-aws-client` Execute AWS CLI commands inside the running LocalStack container via `awslocal`. | Parameter | Type | Required | Description | | --------- | -------- | -------- | ------------------------------------------------------------------- | | `command` | `string` | Yes | The AWS CLI command to run (without the `aws` or `awslocal` prefix) | **Example prompts:** - "List all S3 buckets in LocalStack." - "Describe my Lambda functions." - "Query the DynamoDB table `users` for all items." The tool sanitizes input to prevent shell injection (pipes, redirects, and command chaining are blocked). If a command fails due to a service not being emulated, the tool returns a link to the relevant [service coverage documentation](/aws/services/). ### `localstack-extensions` Install, uninstall, list, and discover [LocalStack Extensions](/aws/customization/integrations/extensions/) from the marketplace. | Parameter | Type | Required | Description | | --------- | ------------------------------------------------- | ------------------------------- | ------------------------------------------------------------ | | `action` | `list` \| `install` \| `uninstall` \| `available` | Yes | The extensions operation | | `name` | `string` | Yes (for `install`/`uninstall`) | Extension package name (e.g., `localstack-extension-typedb`) | | `source` | `string` | No | Git URL to install from (alternative to `name`) | **Example prompts:** - "List my installed LocalStack extensions." - "Browse the available extensions in the marketplace." - "Install the `localstack-extension-typedb` extension." - "Uninstall the `localstack-extension-stripe` extension." :::note After installing or uninstalling an extension, the tool automatically restarts LocalStack to apply the changes. ::: ### `localstack-ephemeral-instances` Manage cloud-hosted [Ephemeral Instances](/aws/developer-tools/cloud-sandbox/ephemeral-instances/) for remote LocalStack testing workflows. | Parameter | Type | Required | Description | | ----------- | ---------------------------------------- | ---------------------------------- | ---------------------------------------------------------------- | | `action` | `create` \| `list` \| `logs` \| `delete` | Yes | The ephemeral instance operation | | `name` | `string` | Yes (for `create`/`logs`/`delete`) | Instance name | | `lifetime` | `number` | No | Lifetime in minutes for the instance (only for `create`) | | `extension` | `string` | No | Extension package to preload on the instance (only for `create`) | | `cloudPod` | `string` | No | Cloud Pod name to initialize state from (only for `create`) | | `envVars` | `Record` | No | Extra environment variables for the instance (only for `create`) | **Example prompts:** - "Create an ephemeral LocalStack instance named `test-env` with a 60-minute lifetime." - "List all my ephemeral instances." - "Get the logs from the `test-env` ephemeral instance." - "Delete the ephemeral instance named `test-env`." ### `localstack-docs` Search the LocalStack documentation to find guides, API references, and configuration details. | Parameter | Type | Required | Description | | --------- | -------- | -------- | ------------------------------------------------------------- | | `query` | `string` | Yes | The search query | | `limit` | `number` | No | Maximum number of results to return (default: `5`, max: `10`) | **Example prompts:** - "Search the LocalStack docs for how to configure S3." - "Find the LocalStack documentation on Cloud Pods." ### `localstack-snowflake-client` Execute SQL queries and commands against the [LocalStack Snowflake emulator](/snowflake/) using the Snowflake CLI (`snow`). | Parameter | Type | Required | Description | | ----------- | ------------------------------- | ------------------------------------------------ | ----------------------------------------- | | `action` | `execute` \| `check-connection` | Yes | The operation to perform | | `query` | `string` | Yes (for `execute`, if `file_path` not provided) | SQL query to execute | | `file_path` | `string` | Yes (for `execute`, if `query` not provided) | Absolute path to a `.sql` file to execute | | `database` | `string` | No | Snowflake database context | | `schema` | `string` | No | Snowflake schema context | | `warehouse` | `string` | No | Snowflake warehouse to use | | `role` | `string` | No | Snowflake role to use | **Example prompts:** - "Check the connection to the LocalStack Snowflake emulator." - "Run `SELECT * FROM my_table` on the Snowflake emulator." - "Execute the SQL file at `/path/to/setup.sql` on Snowflake." :::note This tool requires the [Snowflake CLI](https://docs.snowflake.com/en/developer-guide/snowflake-cli/index) (`snow`) to be installed on your system `PATH`. The tool automatically configures a `localstack` connection profile pointing to the Snowflake emulator. ::: ## Quickstart Once your MCP client is configured, verify the setup by opening a conversation with your AI agent. **1. Start LocalStack** > _"Start my LocalStack container."_ The agent uses the `localstack-management` tool to start the container and confirms the status. **2. Deploy infrastructure** > _"Deploy my CDK project in the `./my-app` directory."_ The agent detects the framework, runs `cdklocal bootstrap` and `cdklocal deploy`, and returns the stack outputs. **3. Verify resources** > _"List the Lambda functions and DynamoDB tables that were created."_ The agent runs `awslocal` commands inside the container and returns the results. **4. Analyze logs** > _"Check the LocalStack logs for any errors."_ The agent fetches recent logs and highlights any errors or warnings. **5. Save state** > _"Save a Cloud Pod named `my-checkpoint`."_ The agent persists the current LocalStack state so you can restore it later. ## Configuration reference The following environment variables can be set in the `env` block of your MCP configuration: | Variable | Default | Description | | -------------------------------------- | ----------------- | -------------------------------------------------------------- | | `LOCALSTACK_AUTH_TOKEN` (**required**) | None | Your LocalStack Auth Token. Required for all MCP server tools. | | `LOCALSTACK_HOSTNAME` | `localhost` | Hostname of the LocalStack instance | | `LOCALSTACK_PORT` | `4566` | Port of the LocalStack instance | | `MAIN_CONTAINER_NAME` | `localstack-main` | Name of the LocalStack Docker container | | `MCP_ANALYTICS_DISABLED` | `0` | Set to `1` to disable MCP analytics | Any [LocalStack configuration variable](/aws/customization/configuration-options/) can also be passed through the `env` block. These are forwarded to the container when the `localstack-management` tool starts it. For example, to enable debug logging and persistence: ```json { "mcpServers": { "localstack-mcp-server": { "command": "npx", "args": ["-y", "@localstack/localstack-mcp-server"], "env": { "LOCALSTACK_AUTH_TOKEN": "", "DEBUG": "1", "PERSISTENCE": "1" } } } } ``` ## Troubleshooting ### LocalStack container fails to start **Symptoms:** The `localstack-management` tool reports a timeout after 120 seconds. **Solutions:** - Ensure Docker is running on your machine. - Verify the LocalStack CLI is installed and on your `PATH` by running `localstack --version`. - Check Docker resource limits. LocalStack needs at least 2 GB of memory. - If you are using a custom `LOCALSTACK_HOSTNAME`, ensure the host is reachable. ### MCP server not detected by the client **Symptoms:** The server does not appear in your client's MCP server list. **Solutions:** - Verify Node.js (v22+) is installed by running `node --version`. - Run `npx -y @localstack/localstack-mcp-server` manually in a terminal to check for errors. - Ensure the JSON in your MCP configuration file is valid (no trailing commas, correct key names). - Restart your MCP client after saving configuration changes. ### Tools return "Auth Token Required" **Symptoms:** Any tool call fails with an "Auth Token Required" error. **Solutions:** - Confirm your `LOCALSTACK_AUTH_TOKEN` is set in the `env` block of your MCP configuration. - Verify the token is valid by running `localstack auth show-token`. - Ensure there are no extra spaces or quotes around the token value in your configuration file. # Overview > Security Testing in LocalStack allows you to test your IAM policies and permissions locally resembling the AWS environment. Security Testing in LocalStack allows you to enforce and validate IAM policies in a local environment that closely mirrors real AWS behavior. This helps you catch misconfigurations, uncover missing permissions, and confidently test access control logic. LocalStack supports the following security testing features: - Enforce IAM policies to simulate realistic permission boundaries in your application - Retrieve Policy engine logs for debugging and understanding how policies are evaluated - Apply IAM policy streams to discover required permissions and resolve access issues efficiently - Simulate IAM policies, including Service Control Policies, to check access decisions before making live requests # Custom TLS certificates > Using custom TLS certificates with LocalStack import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Background LocalStack sometimes performs on-demand fetching of resources from the public internet. This requires that LocalStack is able to access public URLs. If there is a proxy server in your network that uses a non-standard TLS certificate, LocalStack will not be able to download any files on demand. You may see errors in the logs relating to TLS such as "unable to get local issuer certificate". There are two options when running LocalStack: 1. [creating a custom Docker image](#creating-a-custom-docker-image) or 2. [using init hooks](#custom-tls-certificates-with-init-hooks) They all can be summarised as: 1. get your proxy's custom certificate into the system certificate store, and 2. configure [`requests`](https://pypi.python.org/pypi/requests) to use the custom certificate, 3. configure [`curl`](https://curl.se/) to use the custom certificate, and 4. configure [`node.js`](https://nodejs.org/) to use the custom certificate. ## Creating a custom docker image If you run LocalStack in a docker container (which includes using [`lstk`](/aws/developer-tools/running-localstack/lstk/), [docker](/aws/getting-started/installation/#docker-cli), [docker-compose](/aws/getting-started/installation/#docker-compose), or [helm](/aws/customization/kubernetes/deploy-helm-chart)), to include a custom TLS root certificate a new docker image should be created. Create a `Dockerfile` containing the following commands: ```yaml showshowLineNumbers FROM localstack/localstack:latest # or if using the pro image: FROM localstack/localstack-pro:latest COPY /usr/local/share/ca-certificates/cert-bundle.crt RUN update-ca-certificates ENV CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt ENV REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt ENV NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-certificates.crt ``` and build the image: ```bash docker build -t . ``` :::tip Certificate files must end in `.crt` to be included in the system certificate store. If your certificate file ends with `.pem`, you can rename it to end in `.crt`. ::: ### Starting LocalStack with the custom image LocalStack now needs to be configured to use this custom image. The workflow is different depending on how you start localstack. ```toml # .lstk/config.toml [[containers]] type = "aws" image = "" ``` ```bash lstk start ``` ```bash docker run ``` ```yaml showshowLineNumbers services: localstack: image: # the rest of your configuration ``` ## Custom TLS certificates with init hooks It is recommended to create a `boot` init hook. Create a directory on your local system that includes - the certificate you wish to copy, and - the following shell script: ```bash #!/bin/bash set -euo pipefail cp /etc/localstack/init/boot.d/.crt /usr/local/share/ca-certificates update-ca-certificates ``` Then run LocalStack with the environment variables - `REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt`, and - `CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt`, and - `NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-certificates.crt` and follow the instructions fn the [init hooks documentation](/aws/customization/advanced/initialization-hooks) for configuring LocalStack to use the hook directory as a `boot` hook. ## Disabling TLS verification for LocalStack Cloud If your proxy intercepts traffic to LocalStack cloud services (e.g., license server), you can disable TLS verification for these specific requests using the `SSL_NO_VERIFY` [configuration variable](/aws/customization/configuration-options#security) (or `LOCALSTACK_SSL_NO_VERIFY` in Docker). ```toml # .lstk/config.toml [[containers]] type = "aws" env = ["tls"] [env.tls] SSL_NO_VERIFY = "1" ``` ```bash lstk start ``` :::caution This approach disables certificate verification rather than trusting your proxy's certificate. Use custom certificates (as described above) when you need to maintain proper TLS verification for all traffic. ::: # Explainable IAM > Discover IAM Policy Engine logs related to failed policy evaluation. ## Introduction When IAM enforcement denies a request, the IAM Policy Engine returns a descriptive denial message in the API response and records the same information in the LocalStack logs. These messages identify the action that was denied, the policy type responsible (identity-based policy, resource-based policy, permissions boundary, or service control policy), the specific policy document involved, and whether the denial was explicit or implicit. This helps you pinpoint the additional policies required for your request to succeed. Enable `DEBUG=1` to surface the full log output. ## Getting started This guide is designed for users new to Explainable IAM and assumes basic knowledge of the AWS CLI and our [`lstk aws` AWS CLI proxy](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws). Start your LocalStack container with the `DEBUG=1` and `ENFORCE_IAM=1` environment variables set: ```toml # .lstk/config.toml [[containers]] type = "aws" env = ["iam-enforcement"] [env.iam-enforcement] DEBUG = "1" ENFORCE_IAM = "1" ``` ```bash lstk start ``` In this guide, we will create a policy for creating Lambda functions by only allowing the `lambda:CreateFunction` permission. However we have not included the `iam:PassRole` permission, and we will use the Policy Engine's log to point out adding the necessary permission. ### Create a new user Create a policy document named `policy_1.json` and add the following content: ```json showshowLineNumbers { "Version": "2012-10-17", "Statement": [ { "Sid": "FirstStatement", "Effect": "Allow", "Action": "lambda:CreateFunction", "Resource": "*" } ] } ``` You can now create a new user named `test-user`, and put the policy in place by executing the following commands: ```bash lstk aws iam create-user --user-name test-user ``` ```bash { "User": { "Path": "/", "UserName": "test-user", "UserId": "x8a2eu4mc91yqtjazvhp", "Arn": "arn:aws:iam::000000000000:user/test-user", "CreateDate": "2022-07-05T16:08:25.741000+00:00" } } ``` ```bash lstk aws iam put-user-policy --user-name test-user --policy-name policy1 --policy-document file://policy_1.json ``` You can further create an access key for the user by executing the following command: ```bash lstk aws iam create-access-key --user-name test-user ``` Export the access key and secret key as environment variables: ```bash export AWS_ACCESS_KEY_ID=... export AWS_SECRET_ACCESS_KEY=... ``` ### Attempt to create a Lambda function You can now attempt to create a Lambda function using the newly created user's credentials: ```bash lstk aws lambda create-function \ --function-name test-function \ --role arn:aws:iam::000000000000:role/lambda-role \ --runtime python3.8 \ --handler handler.handler \ --zip-file fileb://function.zip ``` The request is denied, and the error response names the exact action that was missing: ```bash An error occurred (AccessDeniedException) when calling the CreateFunction operation: User: arn:aws:iam::000000000000:user/test-user is not authorized to perform: iam:PassRole on resource: arn:aws:iam::000000000000:role/lambda-role because no identity-based policy allows the iam:PassRole action ``` The same information is written to the LocalStack logs. Inspect the logs to see the corresponding Policy Engine entry: ```bash 2026-06-23T15:48:09.650 INFO --- [PoolThread-twisted.internet.reactor-2] localstack.pro.core.services.iam.policy_engine.handler : User: arn:aws:iam::000000000000:user/test-user is not authorized to perform: iam:PassRole on resource: arn:aws:iam::000000000000:role/lambda-role because no identity-based policy allows the iam:PassRole action ``` The message tells you that `iam:PassRole` is *implicitly* denied for your user on the resource `arn:aws:iam::000000000000:role/lambda-role`. This means there is no explicit deny statement in the relevant policies, but there is also no allow statement, so the action is denied by default. You can incorporate this action into the policy. ### Incorporate the action into the policy For illustrative purposes, we will keep the example straightforward, using the same wildcard resource. Edit the `policy_1.json` file to include the `iam:PassRole` action: ```json showshowLineNumbers { "Version": "2012-10-17", "Statement": [ { "Sid": "FirstStatement", "Effect": "Allow", "Action": ["lambda:CreateFunction", "iam:PassRole"], "Resource": "*" } ] } ``` Re-run the Lambda [`CreateFunction`](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunction.html) API. You will notice that the request is now successful, and the function is created. ## Reading denial messages Every denial message follows the same structure: the **principal** that made the request, the **action** it attempted, the **resource** it targeted, and the **cause** of the denial. The cause tells you both the policy type and whether the denial was explicit or implicit: - An **explicit deny** names the policy type and the specific policy document, for example `with an explicit deny in a service control policy: arn:aws:organizations::000000000000:policy/p-abc123`. - An **implicit deny** indicates that no policy allowed the action, for example `because no identity-based policy allows the s3:ListBucket action`. The Policy Engine evaluates identity-based policies, resource-based policies, permissions boundaries, and service control policies, and the denial message identifies whichever one is responsible. ### Inline policies The logs go one step further for inline policies. On AWS, inline policies are reported anonymously, which makes denials caused by them difficult to trace. LocalStack instead names the specific inline policy responsible for the denial: ```bash 2026-06-23T15:57:15.197 INFO --- [PoolThread-twisted.internet.reactor-2] localstack.pro.core.services.iam.policy_engine.handler : User: arn:aws:sts::000000000000:assumed-role/role-881f9fff/TestSession is not authorized to perform: kms:DescribeKey on resource: arn:aws:kms:us-east-1:000000000000:key/ad764663-5e3d-4325-81de-c7e0964f7b7f with an explicit deny in a role inline policy: policy-fc33c780 ``` Here, the request was blocked by an explicit deny in the inline policy `policy-fc33c780` attached to the assumed role, letting you go straight to the policy that needs changing. ## Soft Mode Enabling `IAM_SOFT_MODE=1` allows you to review the logs and assess whether your requests would have been denied or granted while executing your entire stack without disruptions. Using this, you can avoid the need for redeployment to address each missing permission individually, streamlining the debugging process and enhancing the efficiency of your IAM configurations. # IAM Coverage > This page lists the IAM Enforcement Feature Coverage for LocalStack's emulation of AWS services. ## Supported Services In principle, LocalStack supports all operations. However, not all services and their operations have been tested yet. The table below lists all IAM services and operations that have been tested, noting if they were ever denied or allowed during testing. It only includes operations performed with a principal, not as root, so test setups are excluded. |Name |operation |Access denied|Access allowed| |--------------|----------------------------|-------------|--------------| |acm |ListCertificates |Yes |Yes | |apigateway |DeleteRestApi |No |Yes | |apigateway |CreateRestApi |Yes |Yes | |backup |DescribeBackupVault |Yes |Yes | |batch |CreateComputeEnvironment |No |Yes | |cloudformation|ListStacks |Yes |Yes | |cloudwatch |PutMetricData |Yes |Yes | |dynamodb |DescribeTable |No |Yes | |dynamodb |CreateTable |Yes |Yes | |dynamodb |DeleteTable |No |Yes | |ecr |DescribeImages |Yes |No | |efs |DescribeFileSystems |Yes |Yes | |es |DescribeElasticsearchDomains|Yes |Yes | |events |DeleteEventBus |No |Yes | |events |PutEvents |Yes |Yes | |events |CreateEventBus |Yes |Yes | |kinesis |CreateStream |Yes |Yes | |kinesis |DeleteStream |No |Yes | |kms |CreateKey |Yes |Yes | |kms |DescribeKey |Yes |Yes | |lambda |DeleteFunction |No |Yes | |lambda |Invoke |Yes |Yes | |lambda |GetLayerVersion |Yes |Yes | |lambda |CreateFunction |Yes |Yes | |logs |CreateLogGroup |Yes |Yes | |logs |PutLogEvents |No |Yes | |logs |CreateLogStream |No |Yes | |logs |DeleteLogGroup |No |Yes | |redshift |DescribeClusters |Yes |Yes | |redshift-data |ListDatabases |Yes |Yes | |s3 |UploadPart |No |Yes | |s3 |GetObject |Yes |Yes | |s3 |DeleteBucket |No |Yes | |s3 |CreateBucket |Yes |Yes | |s3 |ListBuckets |Yes |Yes | |s3 |CreateMultipartUpload |Yes |Yes | |s3 |CompleteMultipartUpload |No |Yes | |s3 |DeleteObject |No |Yes | |s3 |ListObjects |Yes |Yes | |s3 |PutObject |Yes |Yes | |secretsmanager|CreateSecret |Yes |Yes | |secretsmanager|GetSecretValue |Yes |Yes | |secretsmanager|DeleteSecret |No |Yes | |sns |Publish |No |Yes | |sqs |GetQueueAttributes |Yes |No | |sqs |CreateQueue |Yes |Yes | |sqs |SendMessage |Yes |Yes | |sqs |ReceiveMessage |Yes |Yes | |sqs |DeleteQueue |No |Yes | |stepfunctions |DeleteStateMachine |No |Yes | |stepfunctions |CreateStateMachine |Yes |Yes | |sts |GetCallerIdentity |No |Yes | ## Inter Service Enforcement |Source Service|Target Service|Feature |Operation |Implemented|Tested| |--------------|--------------|-----------------------|------------------------------------------------------------|-----------|------| |sns |sqs |SNS subscription |sqs.SendMessage |Yes |Yes | |sns |lambda |SNS subscription |lambda.Invoke |Yes |Yes | |lambda |sqs |Event destinations |sqs.SendMessage |Yes |Yes | |lambda |logs |Storing Lambda logs |logs.CreateLogGroup, logs.CreateLogStream, logs.PutLogEvents|Yes |No | |lambda |sns |Event destinations |sns.Publish |Yes |No | |lambda |sqs |Event source mapping | |Yes |Yes | |lambda |kinesis |Event source mapping | |Yes |Yes | |lambda |dynamodb |Event source mapping | |Yes |Yes | |lambda |kafka |Event source mapping | |No |No | |events |lambda |Event rule target | |Yes |Yes | |sns |ses |SNS subscription | |Yes |Yes | |sns |firehose |SNS subscription | |Yes |Yes | |events |sns |Event rule target | |Yes |Yes | |events |sqs |Event rule target | |Yes |Yes | |events |logs |Event rule target | |Yes |Yes | |events |firehose |Event rule target | |Yes |Yes | |events |events |Event rule target | |Yes |Yes | |events |kinesis |Event rule target | |Yes |Yes | |events |stepfunctions |Event rule target | |Yes |Yes | |apigateway |lambda |API integration | |Yes |Yes | |apigateway |dynamodb |API integration | |Yes |Yes | |apigateway |kinesis |API integration | |Yes |Yes | |apigateway |s3 |API integration | |No |No | |apigateway |sns |API integration | |No |Yes | |apigateway |sqs |API integration | |Yes |Yes | |apigateway |stepfunctions |API integration | |No |No | |apigateway |appsync |API integration | |No |No | |cloudformation|* |Resource Modification | |No |No | |lambda |sts |Assuming execution role| |Yes |Yes | |s3 |sqs |Bucket notification | |Yes |Yes | |s3 |sns |Bucket notification | |Yes |Yes | ## Supported Policy Types | Permission Type | Details | |-----------------------------|-----------------------------------------------------| | **Identity Based Permissions** | | | | - Roles | | | - Users | | **Resource Based Permissions** | | | | - Lambda | | | - ECR (Elastic Container Registry) | | | - EFS (Elastic File System) | | | - SQS (Simple Queue Service) | | | - SNS (Simple Notification Service) | | | - KMS (Key Management Service) | | | - S3 (Simple Storage Service) | | | - Backup | | | - Events | | | - Secrets Manager | | | - IAM/STS (Identity and Access Management/Security Token Service) | | **Permission Boundaries** | | | | - Roles | | | - Users | | **Service Control Policies (SCPs)** | | | | - Enforced across the organization hierarchy (root, OU, account) | | | - Enforced for cross-account access | | | - Evaluated by the IAM Policy Simulator | ## Supported Policy Features | Category | Description | |----------------|--------------------------------------------------------------------------------------| | **Version** | Not evaluated, but only `"2012-10-17"` supported/tested. | | **Id** | The policy ID is currently ignored. | | **Statements** | Supported with the following policy elements: | | **Effect** | Fully supported. Allow + Deny | | **Sid** | Currently ignored | | **Action, NotAction** | Supported including placeholder `*` | | **Principal, NotPrincipal** | Supported principals: | | | - Service | | | - (Assumed) role (ARN only) | | | - User (ARN only) | | | Organizations, Federated, CanonicalUsers etc. are currently _not_ supported | | **Resource, NotResource** | In general supported, including placeholders `*` and `?`. | | | No support for policy variables | | **Condition** | Supported condition operators: | | | - Null | | | - Bool | | | - StringEquals | | | - StringEqualsIgnoreCase | | | - StringLike | | | - ArnLike/ArnEquals | | | Supported [global condition keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_condition-keys.html): | | | - aws:RequestedRegion | | | - aws:PrincipalArn | | | - aws:SourceArn | | | Supported condition tags: | | | - aws:ResourceTag | | | - aws:RequestTag | | | - aws:PrincipalTag | ## Service-Specific Condition Keys In addition to the global condition keys above, some services support their own condition keys, matching AWS's [per-service condition key reference](https://docs.aws.amazon.com/service-authorization/latest/reference/reference_policies_actions-resources-contextkeys.html): - [EC2](/aws/services/ec2/#iam-condition-keys): `ec2:MetadataHttpTokens`, `ec2:Attribute/` - [RAM](/aws/services/ram/#iam-condition-keys): `ram:RequestedAllowsExternalPrincipals` ## Current Limitations - CloudFormation stack permissions do not work as expected. # IAM Policy Enforcement > Enforce IAM policies in LocalStack to test your policies. ## Introduction IAM Policy Enforcement feature can be used to test your security policies and create a more realistic environment that more closely resembles real AWS. The environment configuration `ENFORCE_IAM=1` is required while starting LocalStack to enable this feature. Per default, IAM enforcement is disabled, and all APIs can be accessed without authentication. When enabled, LocalStack evaluates identity-based policies, resource-based policies, permissions boundaries, and [service control policies](#service-control-policies) together to decide whether a request is allowed. When a request is denied, LocalStack returns a descriptive error that identifies the denied action and the policy responsible for the denial. See [Explainable IAM](/aws/developer-tools/security-testing/explainable-iam/) for a detailed look at these messages. ## Getting started This guide is designed for users new to IAM Policy Enforcement and assumes basic knowledge of the AWS CLI and our `lstk aws` AWS CLI proxy. Start your LocalStack container with the `DEBUG=1` and `ENFORCE_IAM=1` environment variables set: ```toml # .lstk/config.toml [[containers]] type = "aws" env = ["iam-enforcement"] [env.iam-enforcement] DEBUG = "1" ENFORCE_IAM = "1" ``` ```bash lstk start ``` We will demonstrate IAM Policy Enforcement, by creating a user and obtaining the access/secret keys. We will make an attempt to create a bucket using the user’s credentials, which inevitably fails due to insufficient permissions. Lastly, a policy is attached to the user, granting the necessary `s3:CreateBucket` permission, thereby enabling the successful creation of the bucket. ### Create a user To follow this guide, open two separate terminal sessions: **Terminal 1** for the administrative IAM commands, which will utilize the default root IAM user, and **Terminal 2** for executing the commands under the test IAM user you are about to create. This way, we can demonstrate the differentiation in access permissions between the administrative and test users in real-time. In **Terminal 1**, execute the following commands to create a `test` user and obtain the access/secret keys: ```bash lstk aws iam create-user --user-name test ``` ```bash { "User": { "Path": "/", "UserName": "test", "UserId": "d7ryukg7bls4rq1ihq1d", "Arn": "arn:aws:iam::000000000000:user/test", "CreateDate": "2023-11-03T12:20:12.332000Z" } } ``` ```bash lstk aws iam create-access-key --user-name test ``` ```bash { "AccessKey": { "UserName": "test", "AccessKeyId": "LKIAQAAAAAAAHFR7QTN3", "Status": "Active", "SecretAccessKey": "EYUHpIol7bRJpKd/28c/LI2C4bbEnp82LJCRwXRV", "CreateDate": "2023-11-03T12:20:27Z" } } ``` ### Attempt to create a bucket Navigate to **Terminal 2**, where we will configure the access keys for the user `test` in the environment. Once the access keys are set, you will attempt to create an S3 bucket using these credentials. ```bash export AWS_ACCESS_KEY_ID=LKIAQAAAAAAAHFR7QTN3 AWS_SECRET_ACCESS_KEY=EYUHpIol7bRJpKd/28c/LI2C4bbEnp82LJCRwXRV lstk aws s3 mb s3://mybucket ``` ```bash make_bucket failed: s3://mybucket An error occurred (AccessDeniedException) when calling the CreateBucket operation: User: arn:aws:iam::000000000000:user/test is not authorized to perform: s3:CreateBucket on resource: arn:aws:s3:::mybucket because no identity-based policy allows the s3:CreateBucket action ``` As anticipated, the attempt to create the bucket fails with an `AccessDeniedException` error, confirming that user `test` lacks the necessary permissions for this action. The error message names the denied action (`s3:CreateBucket`) and explains that no identity-based policy allows it. You can view the LocalStack logs to validate the policy enforcement: ```bash 2023-11-03T12:21:10.971 INFO --- [ asgi_gw_1] localstack.pro.core.services.iam.policy_engine.handler : User: arn:aws:iam::000000000000:user/test is not authorized to perform: s3:CreateBucket on resource: arn:aws:s3:::mybucket because no identity-based policy allows the s3:CreateBucket action 2023-11-03T12:21:10.972 INFO --- [ asgi_gw_1] localstack.request.aws : AWS s3.CreateBucket => 403 (AccessDenied) ``` ### Attach a policy to the user Let's now return to **Terminal 1** and execute the following commands to attach a policy to the user `test`: ```bash lstk aws iam create-policy --policy-name p1 --policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":"s3:CreateBucket","Resource":"*"}]}' lstk aws iam attach-user-policy --user-name test --policy-arn arn:aws:iam::000000000000:policy/p1 ``` ### Create a bucket Now, let's switch back to **Terminal 2** and observe how the bucket creation succeeds with the `test` IAM user: ```bash lstk aws s3 mb s3://mybucket ``` ```bash make_bucket: mybucket ``` The bucket creation succeeds, confirming that the user `test` now has the necessary permissions to perform this action. You can view the LocalStack logs to validate the policy enforcement: ```bash 2023-11-03T12:23:11.469 INFO --- [ asgi_gw_1] localstack.request.aws : AWS iam.CreatePolicy => 200 2023-11-03T12:23:15.753 INFO --- [ asgi_gw_1] localstack.request.aws : AWS iam.AttachUserPolicy => 200 2023-11-03T12:23:22.795 INFO --- [ asgi_gw_2] localstack.request.aws : AWS s3.CreateBucket => 200 ``` You can further use the IAM Policy Enforcement feature to test your Infrastructure as Code (IaC) deployments and ensure that your policies are correctly enforced. If the IAM policies are not correctly enforced, you will get an unsuccessful response from the API call, and the LocalStack logs will provide you with the necessary information to debug the issue. ## Service Control Policies Service Control Policies (SCPs) are a policy type managed by [AWS Organizations](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps.html). Unlike identity-based and resource-based policies, SCPs do not grant permissions on their own — they act as guardrails that define the maximum permissions available to the accounts they apply to. If an SCP does not allow an action (or explicitly denies it), no `Allow` in an identity-based policy can override that result. With `ENFORCE_IAM=1`, LocalStack evaluates SCPs alongside identity-based policies, resource-based policies, and permissions boundaries. This applies both to single-account access and to cross-account access, where a principal in one account acts on a resource owned by another. The steps below extend the walkthrough above: user `test` already has an identity-based policy that allows `s3:CreateBucket`. We will add an SCP that denies the action and confirm that the request is blocked despite the identity-based `Allow`. In **Terminal 1**, create an organization and a service control policy that denies `s3:CreateBucket`: ```bash awslocal organizations create-organization --feature-set ALL ``` ```bash awslocal organizations create-policy \ --name deny-create-bucket \ --type SERVICE_CONTROL_POLICY \ --description "Deny S3 bucket creation" \ --content '{"Version":"2012-10-17","Statement":[{"Effect":"Deny","Action":"s3:CreateBucket","Resource":"*"}]}' ``` Attach the SCP to the target account: ```bash awslocal organizations attach-policy \ --policy-id \ --target-id ``` Back in **Terminal 2**, attempt to create the bucket again as user `test`: ```bash awslocal s3 mb s3://mybucket ``` Even though the user's identity-based policy allows `s3:CreateBucket`, the SCP guardrail blocks the request, and the denial message names the responsible SCP: ```bash make_bucket failed: s3://mybucket An error occurred (AccessDeniedException) when calling the CreateBucket operation: User: arn:aws:iam::000000000000:user/test is not authorized to perform: s3:CreateBucket on resource: arn:aws:s3:::mybucket with an explicit deny in a service control policy: arn:aws:organizations::000000000000:policy/ ``` The LocalStack logs record the same information: ```bash 2023-11-03T12:30:44.512 INFO --- [ asgi_gw_1] localstack.pro.core.services.iam.policy_engine.handler : User: arn:aws:iam::000000000000:user/test is not authorized to perform: s3:CreateBucket on resource: arn:aws:s3:::mybucket with an explicit deny in a service control policy: arn:aws:organizations::000000000000:policy/ 2023-11-03T12:30:44.513 INFO --- [ asgi_gw_1] localstack.request.aws : AWS s3.CreateBucket => 403 (AccessDenied) ``` This confirms that the SCP overrides the identity-based `Allow`, matching the AWS evaluation order in which an SCP guardrail takes precedence. ## Feature coverage The feature coverage is documented in the [IAM coverage documentation](/aws/developer-tools/security-testing/iam-coverage/). # IAM Policy Simulator > Test IAM policies and Service Control Policies before applying them using the IAM Policy Simulator. ## Introduction The [IAM Policy Simulator](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_testing-policies.html) lets you test the effect of IAM policies attached to a user, group, or role, without making a real request against your resources. It evaluates the policies attached to a principal (and any policies you pass in) and reports whether each requested action would be `allowed`, `explicitDeny`, or `implicitDeny`. LocalStack implements the [`SimulatePrincipalPolicy`](https://docs.aws.amazon.com/IAM/latest/APIReference/API_SimulatePrincipalPolicy.html) operation, which simulates the policies already attached to an existing IAM user, group, or role. [`SimulateCustomPolicy`](https://docs.aws.amazon.com/IAM/latest/APIReference/API_SimulateCustomPolicy.html), which simulates policy documents that aren't attached to any principal, is not yet supported. See the [IAM coverage documentation](/aws/developer-tools/security-testing/iam-coverage/) for the full list of supported operations. :::tip Unlike [IAM Policy Enforcement](/aws/developer-tools/security-testing/iam-policy-enforcement/), the Policy Simulator doesn't require `ENFORCE_IAM=1`. `SimulatePrincipalPolicy` only evaluates your policies; it never performs the underlying AWS operation, so it works the same whether or not enforcement is turned on. ::: ## Getting started This guide is designed for users new to the IAM Policy Simulator and assumes basic knowledge of the AWS CLI and our [`awslocal`](https://github.com/localstack/awscli-local) wrapper script. Start your LocalStack container using your preferred method. ### Create a user with a limited policy Create a user and attach a policy that only allows `s3:CreateBucket`: ```bash awslocal iam create-user --user-name test-user ``` ```bash awslocal iam create-policy \ --policy-name allow-create-bucket \ --policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":"s3:CreateBucket","Resource":"*"}]}' ``` ```bash awslocal iam attach-user-policy \ --user-name test-user \ --policy-arn arn:aws:iam::000000000000:policy/allow-create-bucket ``` ### Simulate the policy Use `simulate-principal-policy` to check whether `test-user` can create and delete an S3 bucket, without actually calling S3: ```bash awslocal iam simulate-principal-policy \ --policy-source-arn arn:aws:iam::000000000000:user/test-user \ --action-names s3:CreateBucket s3:DeleteBucket \ --resource-arns "*" ``` ```bash title="Output" { "EvaluationResults": [ { "EvalActionName": "s3:CreateBucket", "EvalResourceName": "*", "EvalDecision": "allowed", "OrganizationsDecisionDetail": { "AllowedByOrganizations": true } }, { "EvalActionName": "s3:DeleteBucket", "EvalResourceName": "*", "EvalDecision": "implicitDeny", "OrganizationsDecisionDetail": { "AllowedByOrganizations": true } } ], "IsTruncated": false } ``` `s3:CreateBucket` is `allowed` because of the attached policy, and `s3:DeleteBucket` is `implicitDeny` because no statement grants it. ## SCP evaluation If the principal's account is part of an organization, the Policy Simulator also evaluates [Service Control Policies (SCPs)](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps.html) covering that account, in addition to the principal's identity-based policies. This lets you validate SCP behavior with `simulate-principal-policy` before making live requests. Testing SCPs with the Policy Simulator requires [AWS Organizations](/aws/services/organizations/), available on the Ultimate plan and above. The `OrganizationsDecisionDetail.AllowedByOrganizations` field indicates whether the final decision was caused by an SCP: ```bash title="Output" { "EvaluationResults": [ { "EvalActionName": "s3:ListAllMyBuckets", "EvalResourceName": "*", "EvalDecision": "implicitDeny", "OrganizationsDecisionDetail": { "AllowedByOrganizations": false } } ], "IsTruncated": false } ``` For a full walkthrough of SCP enforcement, including cross-account access, see the [Service Control Policy enforcement](/aws/services/organizations/#service-control-policy-enforcement) section of the Organizations documentation. :::note LocalStack's Policy Simulator shares the same evaluation engine as its IAM enforcement, so it reflects real AWS IAM behavior rather than the AWS Policy Simulator, which differs in a few ways: - AWS ignores SCPs that contain conditions during simulation. LocalStack evaluates SCPs with conditions. - AWS applies SCPs to the organization's management account during simulation. LocalStack does not apply SCPs to the management account, matching the real behavior of AWS Organizations. - AWS reports an explicit `Deny` from an SCP as an implicit deny. LocalStack reports it as an explicit deny, which is the expected outcome. ::: ## Limitations - Only `SimulatePrincipalPolicy` is implemented. `SimulateCustomPolicy`, `GetContextKeysForPrincipalPolicy`, and `GetContextKeysForCustomPolicy` are not yet supported, so you need to know which context keys your policies reference and supply them yourself via `--context-entries`. - The response only includes `EvalActionName`, `EvalResourceName`, `EvalDecision`, and `OrganizationsDecisionDetail`. Fields such as `MatchedStatements`, `ResourceSpecificResults`, `EvalDecisionDetails`, and `PermissionsBoundaryDecisionDetail` are not populated, so the response doesn't identify which specific statement caused a decision. ## Feature coverage The feature coverage is documented in the [IAM coverage documentation](/aws/developer-tools/security-testing/iam-coverage/). # IAM Policy Stream > Generate a stream of IAM policies as requests are coming into LocalStack using IAM Policy Stream. ## Introduction The IAM Policy Stream generates a steady stream of policies along with their corresponding principals or resources. When a request is made, it initially displays the principal or resource to which the policy will be attached. This is typically a service resource for resource-based policies, or an IAM principal for other cases. Subsequently, it displays the suggested policy. This feature aids in identifying the correct permissions for cloud applications and can help spot logical errors, such as unexpected actions in a policy. :::note IAM Policy Stream is offered as a **preview** feature and is under active development. `lstk` does not yet support IAM Policy Stream. Continue using the legacy [LocalStack CLI](/aws/developer-tools/running-localstack/localstack-cli/) for this feature. ::: ## Getting started This guide is designed for users who are new to the IAM Policy Stream. It assumes you have basic knowledge of the AWS CLI (and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) AWS CLI proxy). ### Start your LocalStack container To experiment with the IAM Policy Stream, initiate LocalStack using these flags: 1. Enable debugging: `DEBUG=1` 2. Set your LocalStack Auth Token: `LOCALSTACK_AUTH_TOKEN=` 3. Set the IAM Soft Mode: `IAM_SOFT_MODE=1` You can execute the following command in your terminal to start your LocalStack container: ```bash DEBUG=1 IAM_SOFT_MODE=1 localstack start ``` ### Enable IAM Policy Stream To enable the IAM Policy Stream, open a new terminal window or tab and run the following command: ```bash localstack aws iam stream ``` ### Create AWS Resources In a separate terminal tab, we will create AWS resources to observe the necessary policies for them. In this example, we are creating an SNS topic using the following command: ```bash lstk aws sns create-topic --name test-topic ``` In the other tab, the required policy will be generated. This policy can then be attached to an IAM role, enabling it to create the resource. ```bash showshowLineNumbers Attached to identity: "arn:aws:iam::000000000000:root" Policy: { "Version": "2012-10-17", "Statement": [ { "Sid": "Test3a92fb6c", "Effect": "Allow", "Action": "sns:CreateTopic", "Resource": "arn:aws:sns:us-east-1:000000000000:test-topic" } ] } ``` ## Web Application The LocalStack Web Application includes an IAM Policy Stream dashboard, which allows you to discover the necessary permissions for AWS API calls. The Web Application provides the following features: 1. Provides a live display of API calls and the specific policies each call generates. 2. Offers a real-time summary policy, merging all individual policies into one consolidated policy. 3. Includes a feature to activate or deactivate this functionality on-the-fly for performance tuning. 4. Presents an option to reset the stream, facilitating a clean slate to generate new policies. :::tip You don't need to set additional configuration variables, such as `DEBUG=1` or `IAM_SOFT_MODE=1`, when using the IAM Policy Stream with Web Application. However, it won't enforce policies or print IAM-related logs in the LocalStack container. ::: To use this feature, open the LocalStack Web Application in your browser, go to the IAM Policy Stream section, and click on **Enable** to view the **Summary Policy** and **Output**. ![IAM Policy Stream UI](/images/aws/live-policy-stream-enable.png) Run the following command in your terminal to generate a corresponding policy in the IAM Policy Stream dashboard: ```bash lstk aws sns create-topic --name test-topic ``` You will see the following output in the IAM Policy Stream dashboard: ![IAM Policy Stream UI](/images/aws/policy-generate.png) # Overview > Snapshots in LocalStack allow you to save and load the state of your LocalStack instance. import { SectionCards } from '../../../../../components/SectionCards.tsx'; LocalStack is designed to be ephemeral by default, meaning all state is lost when the container stops. The _Snapshot_ feature provides tools to persist, reuse, and share the state of your LocalStack instance across sessions or teams. This is useful for preloading test data, debugging workflows, or collaborating with teammates.
Overview of the LocalStack snapshot lifecycle
Snapshots enhance your development workflow in the following ways: * **Faster loading** - Snapshots can be loaded into your instance within a few seconds. Use this to avoid lengthy redeploys of your infrastructure as code, such as Terraform or CDK, each time the emulator is started. * **Team sharing** - Use a repository, such as _Cloud Pods_, to share snapshots with your team. Snapshots provide a curated set of resources for team members to use as a starting point for their work. * **Automatic durability** - Enable the _Persistence_ feature to gain the same durability semantics you expect from the AWS cloud. A snapshot is taken automatically when the instance is shut down, or at user-defined intervals during operation, then reloaded when the instance is restarted. * **Application preview** - During the code review process, share a snapshot so reviewers can see the software in action without redeploying it for themselves. * **Debugging failures** - Save the state of an instance after a failure has occurred to allow debugging at a later time or by a different team member. For more detail, see the following sections: :::caution Not all LocalStack services support snapshots. If you encounter a limitation, please [contact support](/aws/help-support/get-help/). ::: # Saving snapshots to Cloud Pods > Using LocalStack's Cloud Pods repository to share snapshots with your team. import { Tabs, TabItem } from '@astrojs/starlight/components'; In [the previous section](/aws/developer-tools/snapshots/saving-snapshots-locally/) you learned how to save a snapshot of the emulator's state to a local file, then reload the snapshot into a different emulator instance. When working in a team environment, it's important to have a standard mechanism for sharing snapshot files among your team, and for managing updates when new versions are published.
Cloud Pods workflows
LocalStack provides the web-based _Cloud Pods_ repository for exactly this purpose, accessible only to the users in your organization. Snapshots are generated by an emulator instance, then automatically published to your Cloud Pods repository. From there, snapshots can be loaded back into an instance, either in a desktop environment or a CI environment.
Cloud Pods Web UI
Each organization has its own private Cloud Pods repository, securely managed in LocalStack's cloud. These repositories are backed by dedicated, isolated Amazon S3 buckets. The LocalStack CLI utilizes secure S3 presigned URLs to directly interface with the S3 bucket, bypassing the need to transmit the snapshot files through LocalStack's Platform APIs. :::note[Data residency] Cloud Pods are stored on LocalStack-managed infrastructure in the AWS `eu-central-1` region. This is the only part of the LocalStack platform that retains your data long-term. If your organization requires snapshots to remain within infrastructure you control, save them to [your own S3 bucket](/aws/developer-tools/snapshots/saving-snapshots-to-s3/) instead. ::: ## Using the `lstk` CLI You can save and load snapshots to or from your Cloud Pods repository using the [`lstk snapshot`](/aws/developer-tools/running-localstack/lstk/snapshots/#snapshot) command. ```bash lstk snapshot --help ``` ```bash title="Output" Manage emulator snapshots Usage: lstk snapshot [flags] Commands: list List Cloud Pod snapshots available on the LocalStack platform load Load a snapshot into the running emulator remove Delete a cloud snapshot from the LocalStack platform save Save a snapshot of the emulator state show Show metadata for a cloud snapshot versions List the version history of a cloud snapshot Options: -h, --help help for snapshot Global Options: --config string Path to config file --endpoint-url string Target an existing, externally-managed emulator at this URL --json Output in JSON format (only supported by some commands) --non-interactive Disable interactive mode ``` ### Saving a snapshot to Cloud Pods The following examples build up a Cloud Pod over three versions, adding one service at a time. Start an emulator instance and create a single S3 bucket: ```bash lstk start lstk aws s3 mb s3://bucket1 ``` The command for saving a snapshot to Cloud Pods is similar to the one for saving locally, but instead of a file name, provide the `pod:` prefix followed by a valid Cloud Pod name. ```bash lstk snapshot save pod:sample-application ``` ```bash title="Output" ✔︎ Snapshot saved to pod:sample-application • Version: 1 • Services: s3 • Size: 64.2 KB ``` You can list the available Cloud Pods, for both you and your organization, using the `lstk snapshot list` command: ```bash lstk snapshot list ``` ```bash title="Output" ~ 1 snapshot NAME VERSION LAST CHANGED sample-application 1 2026-08-04 01:00 UTC ``` With the `save` command, you can create multiple versions of a Cloud Pod. For instance, to create an SQS queue and save a second version of `sample-application`: ```bash lstk aws sqs create-queue --queue-name queue-1 lstk snapshot save pod:sample-application ``` ```bash title="Output" ✔︎ Snapshot saved to pod:sample-application • Version: 2 • Services: sqs, s3 • Size: 76.0 KB ``` Adding an SNS topic and saving once more produces a third version: ```bash lstk aws sns create-topic --name topic1 lstk snapshot save pod:sample-application ``` ```bash title="Output" ✔︎ Snapshot saved to pod:sample-application • Version: 3 • Services: sqs, sns, s3 • Size: 85.4 KB ``` To review the version history of a Cloud Pod, use the `versions` command. Note how each version covers the services that existed in the emulator at the time it was saved: ```bash lstk snapshot versions pod:sample-application ``` ```bash title="Output" ~ 3 versions VERSION CREATED LOCALSTACK SERVICES 3 2026-08-04 01:01 UTC 2026.8.0 sqs, sns, s3 2 2026-08-04 01:00 UTC 2026.8.0 sqs, s3 1 2026-08-04 01:00 UTC 2026.8.0 s3 ``` :::note Permissions on Cloud Pods are assigned at the organization level. This means that every individual in the organization can view, load, and delete snapshots created by other team members. Similarly, everyone can save a new version on top of a snapshot originally created by someone else. ::: ### Loading snapshots from a Cloud Pod To load a snapshot from a Cloud Pod into a running emulator, use the `lstk snapshot load` command: ```bash lstk restart lstk snapshot load pod:sample-application ``` ```bash title="Output" ✔︎ Snapshot loaded from pod:sample-application • Services: s3, sns, sqs ``` You can examine the loaded resources with the `lstk status` command: ```bash lstk status ``` ```bash title="Output" ✔︎ LocalStack AWS Emulator is running • Endpoint: localhost.localstack.cloud:4566 • Container: localstack-aws-dev • Version: 2026.8.0 • Uptime: 59s ~ 3 resources · 3 services SERVICE RESOURCE REGION ACCOUNT S3 bucket1 global 000000000000 SNS topic1 us-east-1 000000000000 SQS http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/queue-1 us-east-1 000000000000 ``` By default, `load` fetches the most recent version of the Cloud Pod. To work with an earlier version, append `:` to the Cloud Pod name. Before loading, you can inspect the metadata for a specific version with the `show` command: ```bash lstk snapshot show pod:sample-application:1 ``` ```bash title="Output" Name sample-application Version 1 Created 2026-08-04 01:00 UTC Size 64.2 KB LocalStack 2026.8.0 Services s3 ``` As expected, version 1 covers only S3, since the SQS queue and SNS topic were created later. To load this older version into the running emulator, use the same `:` suffix with the `load` command: ```bash lstk restart lstk snapshot load pod:sample-application:1 ``` ```bash title="Output" ✔︎ Snapshot loaded from pod:sample-application:1 • Services: s3 ``` Comprehensive instructions on using the `lstk snapshot` CLI command are found in the [`lstk` CLI Guide](/aws/developer-tools/running-localstack/lstk/snapshots/#snapshot). :::note The snapshots stored in a Cloud Pod may not remain compatible if used with a different version of LocalStack. LocalStack applies [snapshot compatibility rules](/aws/developer-tools/snapshots/service-coverage#snapshot-compatibility) to block loading snapshots known to be incompatible with the running LocalStack version. ::: ## Using the LocalStack Console The LocalStack Console enables you to: - Browse your Cloud Pods and access your snapshot version history. - Save and load snapshots to and from Cloud Pods. - View snapshot metadata, resources, regions, and version history. ### Browse Cloud Pods The [Cloud Pods Browser](https://app.localstack.cloud/pods) allows you to view, manage, and explore your snapshots through the LocalStack Console. With Cloud Pods, you can have individual or shared ownership of snapshots. ![LocalStack Web Application's Cloud Pods Browser outlining various saved snapshots](/images/aws/cloud-pods-browser.png "Cloud Pods Browser") The Cloud Pods Browser provides the following functionality: - **View Cloud Pods**: View all snapshots saved by you or your organization. - **View Versions**: View the version history of a Cloud Pod and access previous snapshots by clicking on the Cloud Pod's name. - **View Snapshot Details**: View the details of a specific version by clicking on the version number. - **View Cloud Pod Storage**: View both the organization-wide and user-specific storage usage. - **Delete Cloud Pod**: Delete a Cloud Pod by selecting the name and navigating to the **Actions** button, followed by **Delete**. ### View snapshot metadata You can view snapshot metadata by selecting any snapshot in the [Cloud Pods Browser](https://app.localstack.cloud/pods). The metadata includes details such as: - The user who created the snapshot - The creation timestamp - The LocalStack version used to create the snapshot - The size of the snapshot - The service resources contained in the snapshot You can view detailed information within a snapshot, including available resources, categorized services with configurations, and quick access to resource identifiers and endpoints—all without loading the snapshot into your LocalStack runtime. ![Cloud Pods details](/images/aws/cloud-pod-details.png) :::note To save a snapshot with enhanced detail for each of the resources, start your LocalStack instance with the `ENABLE_POD_RESOURCES=1` option. ::: ### Save and load snapshots to or from Cloud Pods You can save and load snapshots using the LocalStack Console. This is useful when you prefer a user-friendly interface without the need to interact with the CLI. ![LocalStack Save/Load Snapshot Cloud Pod Mode](/images/aws/export-import-state-cloud-pod.png) #### Save the snapshot To save a snapshot, follow these steps: 1. Navigate to the **Cloud** tab within the [State](https://app.localstack.cloud/inst/default/state) page. 2. Create AWS resources locally as needed. 3. Enter the Cloud Pod name and toggle between the **New Pod** and **Existing Pod** options. 4. Enter the services to save resources for. By default, all available service resources are saved. 5. Click on **Create New Pod**. A new Cloud Pod will be created, or an existing Cloud Pod will be updated. The snapshot is immediately available for loading into another LocalStack instance. You can view the list of available Cloud Pods on the [Cloud Pods](https://app.localstack.cloud/pods) page. #### Load the snapshot To load a snapshot, follow these steps: 1. Navigate to the **Cloud** tab within the [State](https://app.localstack.cloud/inst/default/state) page. 2. Choose the relevant Cloud Pod from the drop-down list. 3. Click on **Load State From Pod**. To confirm the successful injection of the container state, visit the respective [Resource Browser](https://app.localstack.cloud/inst/default/resources) for the services and verify the resources. ## Auto-loading from Cloud Pods In addition to loading snapshots through the Command-Line Interface (CLI) or the Console, you can configure the automatic loading of one or more Cloud Pods upon the startup of the LocalStack instance. The recommended way to automatically load a snapshot from a Cloud Pod at startup is to set the `snapshot` field in your `lstk` [`config.toml`](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/#auto-loading-a-snapshot-on-start). However, if you run the LocalStack container directly, for example via Docker Compose or `docker run`, you can instead use the `AUTO_LOAD_POD` [configuration variable](/aws/customization/configuration-options/). `AUTO_LOAD_POD` can accept multiple Cloud Pod names separated by commas. To autoload multiple Cloud Pods, such as `foo-pod` and `bar-pod`, use: `AUTO_LOAD_POD=foo-pod,bar-pod`. The order of Cloud Pods in `AUTO_LOAD_POD` dictates their loading sequence. When autoloading multiple Cloud Pods, later pods might overwrite the state of earlier ones if they share the same service, account, and region. Set the `snapshot` field on the container block in your [`config.toml`](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/#auto-loading-a-snapshot-on-start) to the Cloud Pod's `pod:` REF: ```toml [[containers]] type = "aws" port = "4566" snapshot = "pod:foo-pod" ``` ```bash lstk start ``` ```yaml showLineNumbers services: localstack: container_name: "localstack-main" image: localstack/localstack-pro ports: - "127.0.0.1:4566:4566" - "127.0.0.1:4510-4559:4510-4559" environment: - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} - DEBUG=1 - AUTO_LOAD_POD=foo-pod,bar-pod volumes: - "./volume:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" ``` ```bash docker run \ -e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} \ -e AUTO_LOAD_POD=foo-pod \ -v ./volume:/var/lib/localstack \ -p 4566:4566 \ localstack/localstack-pro ``` # Merging snapshots > Merge strategies for loading multiple snapshots into the same emulator instance. LocalStack's snapshot mechanism allows multiple snapshot files to be merged into the same emulator instance. This is useful when several teams collaborate to build a single running emulator image. For example, a Platform team may create a snapshot containing VPCs, subnets, S3 buckets, and SSM parameters. An Application team then produces their own snapshot (building on the first) containing Lambda functions, S3 buckets, ECS images, and other application-level resources. It's therefore important to load multiple snapshots, one on top of the other. LocalStack supports several _merge strategies_ for loading a snapshot into an existing emulator instance. You can think of this as loading two or more snapshots into the same emulator instance, one after the other. The chosen strategy can be passed to `lstk snapshot load` either by using the `--merge` option, or by setting the `LSTK_MERGE_STRATEGY` environment variable. ```bash lstk snapshot load --merge= LSTK_MERGE_STRATEGY= lstk snapshot load ``` ## `overwrite` strategy This strategy completely resets the state of the instance before loading each new snapshot. This results in the instance containing the new snapshot's content, with resources from the older snapshot being discarded. ![Merging snapshots using `--merge=overwrite`](/images/aws/snapshot-merge-overwrite.png) For this merge strategy, use the following: ```bash lstk snapshot load snapshot1 lstk snapshot load --merge=overwrite snapshot2 lstk status ``` ```bash title="Output" [...] SNS topic3 us-east-1 000000000000 SQS http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/queue-3 us-east-1 000000000000 ``` ## `account-region-merge` strategy (**default**) This strategy merges snapshots at the service level, for any given account and region. For example, if the first snapshot contains an SQS queue (`queue-1`) in the `000000000000/us-east-1` account and region, and the second snapshot contains a different SQS queue (`queue-3`), also in the `000000000000/us-east-1` account and region, the first snapshot's SQS resources are discarded. This strategy does not consider whether the SQS queues have different names, since _all_ SQS resources in that account/region are discarded. ![Merging snapshots using `--merge=account-region-merge`](/images/aws/snapshot-merge-account-region.png) For this merge strategy, use the following: ```bash lstk snapshot load snapshot1 lstk snapshot load --merge=account-region-merge snapshot2 lstk status ``` ```bash title="Output" [...] S3 bucket1 global 000000000000 SNS topic2 ap-southeast-2 000000000000 SNS topic3 us-east-1 000000000000 SQS http://sqs.ap-southeast-2.localhost.localstack.cloud:4566/000000000000/queue-2 ap-southeast-2 000000000000 SQS http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/queue-3 us-east-1 000000000000 ``` ## `service-merge` strategy This strategy performs fine-grained merging, similar to the `account-region-merge` strategy, but also considers the names of resources. For example, if each snapshot contains an SQS queue, but the queues have different names (`queue-1` vs `queue-3`), the merge contains both queues. If the names are the same, the resource from the newer snapshot is kept. This is the same behavior you'd expect if you applied two infrastructure-as-code stacks, one on top of the other. ![Merging snapshots using `--merge=service-merge`](/images/aws/snapshot-merge-service.png) For this merge strategy, use the following: ```bash lstk snapshot load snapshot1 lstk snapshot load --merge=service-merge snapshot2 lstk status ``` ```bash title="Output" [...] S3 bucket1 global 000000000000 SNS topic1 us-east-1 000000000000 SNS topic2 ap-southeast-2 000000000000 SNS topic3 us-east-1 000000000000 SQS http://sqs.ap-southeast-2.localhost.localstack.cloud:4566/000000000000/queue-2 ap-southeast-2 000000000000 SQS http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/queue-1 us-east-1 000000000000 SQS http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/queue-3 us-east-1 000000000000 ``` :::note Merge strategies are not currently supported for file-based snapshots when using the LocalStack Console. ::: # Persistence > Enabling automatic persistence of data, providing enhanced durability. import { Tabs, TabItem } from '@astrojs/starlight/components'; LocalStack's _Persistence_ mechanism uses snapshots to provide an enhanced level of durability, bringing it closer to the behavior you'd expect from a cloud-based service. By default, LocalStack's internal state is ephemeral, resetting when the emulator is shut down or exits unexpectedly. When the Persistence feature is enabled, LocalStack takes periodic snapshots of your emulator, then restores it upon restart. This reduces the likelihood of unexpected data loss. ## Configuration To start snapshot-based persistence, launch LocalStack with the `--persist` command-line option, or the configuration option `PERSISTENCE=1`. This instructs LocalStack to periodically generate a snapshot, storing it within LocalStack's internal volume directory. There is no visible snapshot file (or Cloud Pod) created, as the snapshot is managed internally by LocalStack. Upon restarting LocalStack, the last successful snapshot is automatically reloaded, so you can resume your activities exactly where you left off. ```bash lstk start --persist ``` ```yaml showLineNumbers image: localstack/localstack-pro environment: - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} - PERSISTENCE=1 volumes: - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" ``` ```bash docker run \ -e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} \ -e PERSISTENCE=1 \ -v ./volume:/var/lib/localstack \ -p 4566:4566 \ localstack/localstack-pro ``` :::note Snapshots (stored in LocalStack's volume) may not remain compatible if you upgrade your version of LocalStack. LocalStack applies [snapshot compatibility rules](/aws/developer-tools/snapshots/service-coverage#snapshot-compatibility) to block loading snapshots known to be incompatible with the running LocalStack version. ::: ### Save strategies LocalStack generates periodic snapshots of the running emulator. There are four strategies you can choose from to govern when these snapshots are taken. You can select a particular save strategy by setting `SNAPSHOT_SAVE_STRATEGY=`. * **`ON_REQUEST`**: On every AWS API call that potentially makes a modification, LocalStack saves the state of that service. This strategy minimizes the chance of data loss, but also has significant performance implications. The service must be locked during snapshotting, with any requests to the particular AWS service being blocked until the snapshot is complete. In many cases this is just a few milliseconds, but can become significant in some services. * **`ON_SHUTDOWN`**: The state of all services is saved during the shutdown phase of LocalStack. This strategy has negligible performance impact, but is not good for minimizing the chance of data loss. Should LocalStack for some reason not shut down properly, or be terminated before it can finalize the snapshot, you may be left with an incomplete state on disk. * **`SCHEDULED`** (**default**): Saves the state of all services at regular intervals, as long as the state has been modified since the last snapshot. By default, the flush interval is 15 seconds. It can be configured via the `SNAPSHOT_FLUSH_INTERVAL` configuration variable. This is a compromise between `ON_REQUEST` and `ON_SHUTDOWN` in terms of performance and reliability. * **`MANUAL`**: Turns off automatic snapshotting and gives you control through the internal state endpoints. ### Load strategies Similarly, you can configure when LocalStack should restore the state snapshots, by using `SNAPSHOT_LOAD_STRATEGY=`. * **`ON_REQUEST`**: (**default**) The state is loaded lazily when the service is first used (that is, the first API call to that service). This maintains LocalStack's lazy-loading behavior for AWS services. * **`ON_STARTUP`**: The state of all services in the snapshot is restored when LocalStack starts up, before any of the services are accessed. This reduces the cost of lazy-loading when the data is eventually accessed, but does cause an upfront delay to pre-load everything. * **`MANUAL`**: Turns off automatic loading of snapshots and gives you control through the internal state endpoints. ### Endpoints With the `MANUAL` save or load strategy, you can trigger snapshotting manually when it best suits your application flow. * `POST /_localstack/state//save` take a snapshot of the given service * `POST /_localstack/state//load` load the most recent snapshot of the given service For example, a snapshot for a particular service (e.g., `s3`) can be triggered by running the following command. The service name refers to the AWS service code. ```bash curl -X POST http://localhost:4566/_localstack/state/s3/save ``` It is also possible to take and load a snapshot of all the services at once. We provide the following endpoints: * `POST /_localstack/state/save` * `POST /_localstack/state/load` The response is a service-by-service JSON stream, showing the service that has been saved/loaded and the status of the operation. ```bash curl -X POST localhost:4566/_localstack/state/save ``` ```bash title="Output" {"service": "sqs", "status": "ok"} {"service": "s3", "status": "ok"} ``` # Saving snapshots locally > Saving and loading snapshots from local files. With Snapshots, you can save the state of your LocalStack instance to a file on disk, then load it back at a later time. This concept is similar to that of desktop-based word processors, spreadsheets, or practically any software that allows saving and loading the program state. Loading a snapshot is significantly faster, and far more convenient, than re-creating the same state with infrastructure-as-code tools such as Terraform or CDK, or re-running deployment scripts by hand. Additionally, the dynamic state of those resources (such as database content) is automatically captured, avoiding the need for data-seeding scripts to populate them. ## Using the `lstk` CLI The [`lstk` CLI](/aws/developer-tools/running-localstack/lstk/snapshots/#snapshot) lets you save your instance's state to a local file and load it back into another instance at a later time. For example, starting from an empty emulator instance, create an S3 bucket, an SNS topic, and an SQS queue: ```bash lstk start lstk aws s3 mb s3://bucket1 lstk aws sns create-topic --name topic1 lstk aws sqs create-queue --queue-name queue-1 ``` The `lstk status` command confirms that the three resources are deployed: ```bash lstk status ``` ```bash title="Output" ✔︎ LocalStack AWS Emulator is running • Endpoint: localhost.localstack.cloud:4566 • Container: localstack-aws-dev • Version: 2026.8.0 • Uptime: 17s ~ 3 resources · 3 services SERVICE RESOURCE REGION ACCOUNT S3 bucket1 global 000000000000 SNS topic1 us-east-1 000000000000 SQS http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/queue-1 us-east-1 000000000000 ``` To save the state to a local file, run: ```bash lstk snapshot save my-snapshot ``` ```bash title="Output" ✔︎ Snapshot saved to ./my-snapshot.snapshot • Services: sns, sqs, s3 • Size: 85.4 KB ``` The destination argument is optional. If you omit it, `lstk` auto-generates a timestamped snapshot file in the current directory: ```bash lstk snapshot save ``` ```bash title="Output" ✔︎ Snapshot saved to ./snapshot-2026-08-04T20-16-49-5e3.snapshot • Services: sns, sqs, s3 • Size: 85.4 KB ``` Since saving is a common operation, the `lstk save` abbreviation is also available: ```bash lstk save ``` ```bash title="Output" ✔︎ Snapshot saved to ./snapshot-2026-08-04T20-16-49-db4.snapshot • Services: sns, sqs, s3 • Size: 85.4 KB ``` Restarting the emulator discards all of its state, so `lstk status` now reports that no resources are deployed: ```bash lstk restart lstk status ``` ```bash title="Output" ✔︎ LocalStack AWS Emulator is running • Endpoint: localhost.localstack.cloud:4566 • Container: localstack-aws-dev • Version: 2026.8.0 • Uptime: 5s > Note: No resources deployed ``` To load a previously saved snapshot, run: ```bash lstk snapshot load my-snapshot ``` ```bash title="Output" ✔︎ Snapshot loaded from ./my-snapshot.snapshot ``` Alternatively, the `lstk load` command is also available: ```bash lstk load my-snapshot ``` ```bash title="Output" ✔︎ Snapshot loaded from ./my-snapshot.snapshot ``` Running `lstk status` once more confirms that the bucket, topic, and queue have returned: ```bash lstk status ``` ```bash title="Output" ✔︎ LocalStack AWS Emulator is running • Endpoint: localhost.localstack.cloud:4566 • Container: localstack-aws-dev • Version: 2026.8.0 • Uptime: 6s ~ 3 resources · 3 services SERVICE RESOURCE REGION ACCOUNT S3 bucket1 global 000000000000 SNS topic1 us-east-1 000000000000 SQS http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/queue-1 us-east-1 000000000000 ``` ## Using the LocalStack Console The LocalStack Console allows saving a snapshot to a file, then loading it into another LocalStack instance. ![LocalStack Export/Import State Local Mode](/images/aws/export-import-state-local.png) To save the snapshot, follow these steps: 1. Create AWS resources locally as needed. 2. Navigate to the **Local** tab within the [Export/Import State](https://app.localstack.cloud/inst/default/state) page. 3. Click on the **Export State** button. This action will initiate the download of a ZIP file. The downloaded ZIP file contains your container state, which can be injected into another LocalStack instance for further use. To load an existing snapshot, follow these steps: 1. Navigate to the **Local** tab within the [Export/Import State](https://app.localstack.cloud/inst/default/state) page. 2. Upload the ZIP file that contains your container state. This action will restore your previously saved AWS resources. To confirm the successful injection of the container state, visit the respective [Resource Browser](https://app.localstack.cloud/inst/default/resources) for the services and verify the resources. # Saving snapshots to S3 > Save snapshots directly to an Amazon S3 bucket, as an alternative to Cloud Pods or local storage. By default, Cloud Pod artifacts are stored on the LocalStack platform. However, if your organization's data regulations or sovereignty requirements prohibit storing snapshots in LocalStack's managed storage, saving directly to your own Amazon S3 bucket is the recommended solution for keeping full control over where that data lives. When saving, loading, or listing snapshots in your own S3 bucket, `lstk` uses pre-signed S3 URLs to transfer the data directly between the emulator and your bucket, without proxying it through LocalStack's platform. ## Using the `lstk` CLI The [`lstk snapshot`](/aws/developer-tools/running-localstack/lstk/snapshots/#snapshot) command lets you save, load, and list snapshots stored in your own S3 bucket by passing an `s3://bucket/prefix` location alongside a snapshot name. The initial step is to export the necessary AWS credentials in your terminal session. ```bash export AWS_ACCESS_KEY_ID=... export AWS_SECRET_ACCESS_KEY=... ``` To obtain credentials automatically, use [AWS SSO CLI](https://github.com/synfinatic/aws-sso-cli). Alternatively, set `AWS_PROFILE` or pass `--profile ` to have `lstk` read credentials from a named AWS profile instead of the environment. To save a snapshot to your S3 bucket, provide a snapshot name followed by the `s3://` location: ```bash lstk snapshot save my-snapshot s3://ls-s3-bucket-example ``` ```bash title="Output" ✔︎ Snapshot saved to s3://ls-s3-bucket-example as "my-snapshot" • Version: 1 • Size: 74.6 KB ``` :::note On LocalStack `v2026.08` or later, snapshots transfer to a real Amazon S3 bucket with [transparent endpoint injection](/aws/customization/networking/transparent-endpoint-injection) left enabled, as it is by default. No extra configuration is needed. On versions before `v2026.08`, transparent endpoint injection must be disabled. Otherwise LocalStack's [DNS server](/aws/customization/networking/dns-server) resolves the AWS domains back to the emulator, and the transfer never reaches your bucket — typically surfacing as a TLS certificate validation error. Disable this feature by setting [`DNS_ADDRESS=0`](/aws/customization/configuration-options/) when starting the emulator, which turns off transparent endpoint injection application-wide. On the command line, use `LOCALSTACK_DNS_ADDRESS=0 lstk start` — host variables prefixed with `LOCALSTACK_` are forwarded to the emulator. In a `config.toml` [environment profile](/aws/developer-tools/running-localstack/lstk/configuration/#passing-environment-variables-to-the-container), use the unprefixed form `DNS_ADDRESS = "0"`. ::: Once the snapshot has been saved, you can confirm the presence of the snapshot artifacts in the S3 bucket by running: ```bash aws s3 ls s3://ls-s3-bucket-example ``` ```bash title="Output" 2026-08-05 10:08:55 76390 localstack-pod-my-snapshot-state-1.zip ``` You can then use `lstk snapshot load` to load the previously saved snapshot: ```bash lstk snapshot load my-snapshot s3://ls-s3-bucket-example ``` ```bash title="Output" ✔︎ Snapshot loaded from s3://ls-s3-bucket-example (my-snapshot) ``` Similarly, you can list the snapshots stored in this bucket with `lstk snapshot list`: ```bash lstk snapshot list s3://ls-s3-bucket-example ``` ```bash title="Output" ~ 1 snapshot NAME VERSION my-snapshot 1 ``` :::note Because data transfer is performed by the emulator rather than the CLI, saving, loading, and listing snapshots in your own S3 bucket require a **running emulator**. ::: Comprehensive instructions on using the `lstk snapshot` CLI command, including credential resolution order, are found in the [`lstk` CLI Guide](/aws/developer-tools/running-localstack/lstk/snapshots/#s3-remotes). # Service coverage > Snapshot compatibility rules and service coverage of LocalStack's snapshot mechanism. import { Tabs, TabItem } from '@astrojs/starlight/components'; import PersistenceCoverage from '../../../../../components/persistence-coverage/PersistenceCoverage.tsx'; This page covers two topics: the compatibility rules LocalStack enforces when loading a snapshot, and the current level of snapshot support across AWS services. ## Snapshot compatibility The internal data structures inside a LocalStack emulator change over time as services evolve. To prevent silently loading state into an incompatible runtime, LocalStack ships a set of compatibility rules that compare the LocalStack version recorded in the snapshot with the version of the running emulator. These rules apply to all snapshots, whether saved locally to a file, saved to a Cloud Pod, or implicitly saved when persistence is enabled. If a rule rejects the snapshot, LocalStack does not load it, and logs the reason. The rules currently enforced are: | Rule | Behavior | | - | - | | Forward compatibility | Reject loading a snapshot into a LocalStack emulator older than the one that produced it. | | First CalVer release (`v2026.03`) | Reject loading a snapshot saved before `v2026.03` into LocalStack `v2026.03` or later. The snapshot mechanism was rewritten for several services in the first calendar-versioned release. | Loading a snapshot saved with `v2026.03` or later into a newer LocalStack version of the same series remains supported. For example, a snapshot saved with `v2026.03` can be loaded into `v2026.03.1` or `v2026.04`. ### Disable compatibility checks If you understand the risks and want LocalStack to load a snapshot regardless of these rules, start the container with `DISABLE_COMPATIBILITY_RULES=1`. This bypasses every compatibility rule and lets LocalStack attempt to load the state as-is. ```bash LOCALSTACK_DISABLE_COMPATIBILITY_RULES=1 lstk start --persist ``` ```yaml showLineNumbers image: localstack/localstack-pro environment: - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} - PERSISTENCE=1 - DISABLE_COMPATIBILITY_RULES=1 volumes: - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" ``` ```bash docker run \ -e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} \ -e PERSISTENCE=1 \ -e DISABLE_COMPATIBILITY_RULES=1 \ -v ./volume:/var/lib/localstack \ -p 4566:4566 \ localstack/localstack-pro ``` :::caution Disabling compatibility rules can leave LocalStack in an inconsistent state. Use this option only for debugging or when migrating data with a tested workaround in place. ::: ## Service coverage Although we are working to support snapshots for all AWS services, there are some common issues, known limitations, and services that are not well tested for snapshot support. See the [snapshot coverage overview](#snapshot-coverage-overview) below for the current state of support. For example, when using LocalStack's snapshot feature, ports assigned to certain services (such as RDS or ElastiCache) may not be preserved when reloading that snapshot. If you start new services *before* restoring the snapshot, these new instances may use ports originally used by the saved services. As a result, restored resources may point to invalid or unintended ports. We suggest restoring services in the same order they were initially deployed, though this is not always reliable. If you encounter a limitation or bug with snapshot support, please [contact support](/aws/help-support/get-help/). ## Snapshot coverage overview The **Persistence Test Suite** column indicates whether a service is covered by LocalStack's internal persistence test suite. We first record API responses when querying the resources in the LocalStack emulator, then reset the emulator and restore the snapshotted state. Finally, we verify that the API responses match those recorded earlier. # Overview > Introduction to LocalStack for AWS, covering core use cases, local cloud capabilities, and deployment options for development and testing. import { SectionCards } from '../../../../components/SectionCards.tsx'; LocalStack provides a cloud service emulator that runs within a single container on your local machine or CI environment. It delivers a functional AWS environment including Lambda, DynamoDB, S3, SQS, and [80+ supported services](/aws/services/), enabling development and testing without an AWS account or cloud-related costs. ### Core Use Cases - **Accelerate development loops**: Test changes against local AWS services instantly to bypass deployment wait times. - **Automate integration testing**: Execute integration tests against local AWS infrastructure within pull requests to identify regressions before production. - **Validate IaC**: Deploy Terraform, CDK, or CloudFormation templates to LocalStack to verify infrastructure logic before applying changes to a cloud environment. - **Experimental sandbox**: Explore new AWS services and architectures in a risk-free environment. LocalStack also provides advanced features for team collaboration and security, including [Cloud Pods](/aws/developer-tools/snapshots/cloud-pods/) for state management, [IAM policy enforcement](/aws/developer-tools/security-testing/iam-policy-enforcement/), and [Chaos Engineering](/aws/developer-tools/chaos-engineering/). ## Start with the basics :::note **Enterprise Kubernetes Deployment:** LocalStack also supports execution within Kubernetes clusters via the Operator or Helm charts. This model enables dynamic scaling, environment isolation, and native orchestration. See our [Kubernetes Deployment guide](/aws/customization/kubernetes/) for more information. ::: # AI & Agent Workflows > Use LocalStack with AI coding assistants, MCP clients, and agent-driven infrastructure workflows. ## Introduction LocalStack gives AI coding assistants a local AWS-compatible environment to work against. Instead of letting an agent experiment in a real AWS account, you can ask it to create infrastructure, deploy code, inspect logs, and test resources in LocalStack first. This is useful when you want to: - Prototype AWS applications & infrastructure code from natural language prompts. - Validate AI-generated Terraform, CDK, or AWS CLI commands before using a cloud account. - Give an AI assistant a safe place to inspect resources, debug logs, and iterate on deployments. - Use reusable agent instructions for LocalStack-aware infrastructure workflows. ## Common workflows There are three common ways to use LocalStack in AI-assisted development: - Use the [LocalStack MCP Server](/aws/developer-tools/running-localstack/mcp-server/) when your AI assistant supports MCP clients such as Cursor, Claude, Codex, or OpenCode. - Use [LocalStack Skills](https://github.com/localstack/skills) when you want reusable agent instructions for deploying and testing AWS architectures against LocalStack. - Use LocalStack with `tflocal`, `cdklocal`, or `awslocal` when you want the agent to generate infrastructure code or commands that you review and run locally. You do not need all three approaches to get started. If your editor supports MCP, start with the LocalStack MCP Server. Or, you can use Skills if you want reusable agent instructions. If not, ask your assistant to generate Terraform, CDK, or AWS CLI steps and run them with LocalStack's local wrappers. ## Quick Setup LocalStack provides an [`agents.md`](https://docs.localstack.cloud/agents.md) file with the full instructions your AI agent needs to get started with LocalStack, including how to configure the MCP server and LocalStack Skills. You can give the file directly to your agent or copy and paste the prompt below. ```text Fetch https://docs.localstack.cloud/agents.md and follow the instructions to set up LocalStack on my machine. ``` For manual setup of the MCP server and skills, you can follow the steps below. ## Connect an MCP client The LocalStack MCP Server connects MCP-compatible clients to your LocalStack environment. Once configured, your AI assistant can use LocalStack tools to start the container, deploy infrastructure, run AWS CLI commands, inspect logs, manage state, and query resources. Start the MCP server with an interactive setup wizard: ```bash npx -y @localstack/localstack-mcp-server init ``` :::note The MCP server runs locally and talks to a LocalStack instance. Your AI assistant is the MCP client. For full installation instructions, detailed setup, and the full tool reference, see the [LocalStack MCP Server guide](/aws/developer-tools/running-localstack/mcp-server/). You need a valid [Auth Token](/aws/getting-started/auth-token/) to configure the server. ::: ## Use agent skills [LocalStack Skills](https://github.com/localstack/skills) provide reusable instructions for AI agents working with LocalStack. They help agents follow LocalStack-specific conventions when creating infrastructure, deploying resources, running tests, and inspecting local cloud state. Skills are most useful when you want the assistant to follow a repeatable workflow, for example: - Scaffold a local AWS application and deploy it to LocalStack. - Convert an AWS architecture idea into Terraform or CDK that targets LocalStack first. - Debug a failing local deployment by checking resources, logs, and configuration. - Save or restore LocalStack state as part of an iterative development loop. Refer to the [LocalStack Skills repository](https://github.com/localstack/skills) for available skills and setup instructions. ## Example prompt sequence After LocalStack and your preferred AI tooling are configured, you can use a sequence like this: ```text Create a Terraform application with an S3 bucket, a Lambda function, and a DynamoDB table. Make it deployable to LocalStack with tflocal. ``` ```text Deploy the application to LocalStack and fix any errors from the deployment. ``` ```text Invoke the Lambda function locally, inspect the DynamoDB table, and summarize what resources were created. ``` ```text Add an integration test that verifies the Lambda writes an item to DynamoDB. Run the test against LocalStack. ``` This keeps the feedback loop local while still giving the assistant a realistic AWS-compatible target. ## Review before applying to AWS AI-generated infrastructure still needs review. Treat LocalStack as the first validation step, not as a replacement for code review, tests, or production deployment controls. Before applying changes to AWS, check that: - The generated infrastructure matches your intended architecture. - Resource names, IAM policies, and environment variables are appropriate for your project. - Tests pass against LocalStack. - You understand any changes the assistant made to application code or deployment configuration. ## Next steps - Configure the [LocalStack MCP Server](/aws/developer-tools/running-localstack/mcp-server/) if your AI assistant supports MCP. - Review [LocalStack Skills](https://github.com/localstack/skills) for reusable agent workflows. - Browse the [LocalStack for AWS services](/aws/services/) reference, or check the [Getting Started FAQ](/aws/getting-started/faq/) for common setup questions. # Auth Token > Configure and manage your LocalStack Auth Token to activate LocalStack and access licensed features. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## What is an Auth Token? An Auth Token is a mandatory credential required to start the LocalStack container and activate licensed features. It links your running LocalStack instance to your workspace license and unlocks the services and capabilities available to your account. You can manage Auth Tokens from the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens) in the LocalStack Web Application. :::danger[Credential security] Auth Tokens provide access to your license and workspace. Do not commit tokens to version control. If a token is exposed, rotate it immediately in the LocalStack Web Application. ::: ## Token types | Token Type | Scope | Use Case | | :--- | :--- | :--- | | **Developer Token** | Individual | Local development workstations. Managed per user. | | **CI Auth Token** | Workspace | Automated pipelines and shared runners. Managed by workspace admins. | ## Configure your token Authentication requirements vary based on your chosen execution method. ### lstk The `lstk` CLI automates the authentication lifecycle. On initial execution, it triggers a browser-based OAuth flow and stores the resulting token in your system keyring. No manual environment variable configuration is required. ```bash lstk start ``` ### LocalStack CLI If you use the LocalStack CLI, set your token with the `auth` command. This stores the token in your local configuration. ```bash localstack auth set-token localstack start ``` :::note You can alternatively set the `LOCALSTACK_AUTH_TOKEN` environment variable in your shell session. The `localstack auth set-token` command is only available for the `localstack` CLI and cannot be used with a Docker or Docker Compose setup. ::: ### Docker and Docker Compose For direct container execution, inject the token as an environment variable. For complete startup examples, see the [Docker Compose](/aws/getting-started/installation/#docker-compose) and [Docker CLI](/aws/getting-started/installation/#docker-cli) installation options. **Docker CLI:** ```bash -e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN} ``` **Docker Compose:** ```yaml environment: - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN} ``` ### CI environments CI environments should use a dedicated CI Auth Token stored in your CI provider's secret manager. For complete examples, see the [CI/CD guide](/aws/getting-started/ci-cd/). ## Verify activation Verify the activation status by querying the LocalStack info endpoint: ```bash curl http://localhost:4566/_localstack/info | jq ``` ```powershell Invoke-WebRequest -Uri http://localhost:4566/_localstack/info | ConvertFrom-Json ``` ```json title="Output" { "edition": "pro", "is_license_activated": true } ``` The `edition` field should be `pro`, and `is_license_activated` should be `true`. ## License assignment An Auth Token can only activate licensed features if a license is assigned to the associated user or workspace. 1. Navigate to the [Users & Licenses page](https://app.localstack.cloud/workspace/members). 2. Identify the target user in **Workspace Members**. 3. Select the appropriate **Member Role**. 4. Save the configuration to activate the license for that identity. :::note LocalStack cannot activate licensed features unless the token belongs to a user or workspace with an assigned license. ::: ## Rotate a token Rotate an Auth Token if it has been exposed, shared accidentally, or stored in a place where it should not be. Go to the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens) and select the reset option for the affected token. After rotation, update every local shell, container configuration, or CI secret that used the old token. ## Troubleshooting LocalStack requires successful license activation during startup. If activation fails, LocalStack exits and displays an error message: ```bash =============================================== License activation failed! 🔑❌ Reason: The credentials defined in your environment are invalid. Please make sure to either set the LOCALSTACK_AUTH_TOKEN variable to a valid auth token, or the LOCALSTACK_API_KEY variable to a valid LocalStack API key. You can find your Auth Token or API key in the LocalStack web app https://app.localstack.cloud. Due to this error, Localstack has quit. LocalStack pro features can only be used with a valid license. - Please check that your credentials are set up correctly and that you have an active license. You can find your credentials in our webapp at https://app.localstack.cloud. ``` Activation may fail for several reasons, and the most common ones are listed below. ### Missing credentials You need to provide an Auth Token to start the LocalStack for AWS image successfully. You can find your Auth Token on the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens) in the LocalStack Web Application. If you are using the `localstack` CLI, you can set the `LOCALSTACK_AUTH_TOKEN` environment variable to your Auth Token or use the following command to set it up: ```bash localstack auth set-token ``` ### Invalid license The issue may occur if there is no valid license linked to your account due to expiration or if the license has not been assigned. You can check your license status in the LocalStack Web Application on the [My License page](https://app.localstack.cloud/workspace/my-license). ### License server unreachable LocalStack initiates offline activation when the license server is unreachable, requiring re-activation every 24 hours. Log output may indicate issues with your machine resolving the LocalStack API domain, which can be verified using a tool like `dig`: ```bash dig api.localstack.cloud ``` If the result shows a status other than `status: NOERROR`, your machine is unable to resolve this domain. Certain corporate DNS servers may filter requests to specific domains. Kindly reach out to your network administrator to safelist `localstack.cloud` domain. If you have any further problems concerning your license activation, or if the steps do not help, don't hesitate to [contact us](https://localstack.cloud/contact/). ## Next steps After configuring your Auth Token, continue to the [Local Development guide](/aws/getting-started/local-development/) to start LocalStack and deploy a local serverless API. # CI Integration > Use LocalStack in CI pipelines to run integration tests against local AWS infrastructure. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction LocalStack helps you run integration tests in CI against emulated AWS infrastructure. Your pipeline starts LocalStack inside the CI job, deploys or prepares the resources your application needs, runs tests against the local endpoint, and then discards the environment when the job ends. ## How LocalStack works in CI A typical CI job with LocalStack follows this flow: 1. Check out your application code. 2. Start LocalStack in the CI runner. 3. Configure a CI Auth Token through the CI provider's secret manager. 4. Deploy test infrastructure with tools such as `awslocal`, `tflocal`, `cdklocal`, or your application's test harness. 5. Run integration tests against the LocalStack endpoint. 6. Collect logs, test reports, and artifacts from the job. This gives every pipeline run a fresh AWS-compatible environment without creating cloud resources in an AWS account. ## What changes from local development CI runs are usually more constrained than local development: - Use a dedicated **CI Auth Token** instead of a personal Developer Token. - Store `LOCALSTACK_AUTH_TOKEN` as a protected CI secret. - Start LocalStack non-interactively as part of the job. - Treat the LocalStack container as ephemeral unless your workflow explicitly saves state. - Export logs and test reports before the runner shuts down. Docker and Docker Compose are still common ways to run containers inside CI runners, but they are not CI tools by themselves. For container startup details, see the [Installation guide](/aws/getting-started/installation/#container-and-orchestration-tools). For provider-specific CI setup, use the integration guides below. ## Choose your CI provider Start with the CI system you use. These snippets show the basic LocalStack startup shape for each provider; the linked guides include authentication, configuration, logs, state management, and provider-specific caveats. :::note For brevity, these snippets show only the LocalStack startup shape. Apart from the GitHub Actions example, they assume your CI Auth Token is already exposed to the job as the `LOCALSTACK_AUTH_TOKEN` environment variable. Store it as a secret in your CI provider before running them, and see [Authentication in CI](#authentication-in-ci) below. ::: ```yaml - name: Start LocalStack uses: LocalStack/setup-localstack@main with: image-tag: 'latest' install-awslocal: 'true' env: LOCALSTACK_AUTH_TOKEN: ${{ secrets.LOCALSTACK_AUTH_TOKEN }} ``` See the [GitHub Actions guide](/aws/ci-pipelines/github-actions/) for the full setup. ```yaml version: '2.1' orbs: python: circleci/python@4.0.0 jobs: localstack-test: machine: image: ubuntu-2204:current steps: - checkout - run: name: Install lstk and awslocal command: | python3 -m pip install --user --upgrade pip python3 -m pip install --user localstack awscli-local[ver1] echo 'export PATH=$HOME/.local/bin:$PATH' >> "$BASH_ENV" - run: name: Start LocalStack command: | source "$BASH_ENV" docker pull localstack/localstack:latest localstack start -d localstack wait -t 60 ``` See the [CircleCI guide](/aws/ci-pipelines/circleci/) for the full setup. ```yaml image: python:3.9 definitions: services: docker: memory: 2048 pipelines: default: - step: name: Test LocalStack services: - docker script: - export DOCKER_SOCK=$DOCKER_HOST - export AWS_ENDPOINT_URL="http://localhost.localstack.cloud:4566" - echo "${BITBUCKET_DOCKER_HOST_INTERNAL} localhost.localstack.cloud " >> /etc/hosts - pip install localstack awscli-local - docker run -d --rm -p 4566:4566 -p 4510-4559:4510-4559 -e DOCKER_SOCK=tcp://${BITBUCKET_DOCKER_HOST_INTERNAL}:2375 -e DOCKER_HOST=tcp://${BITBUCKET_DOCKER_HOST_INTERNAL}:2375 --name localstack-main localstack/localstack - localstack wait -t 60 ``` See the [Bitbucket Pipelines guide](/aws/ci-pipelines/bitbucket/) for the full setup. ```yaml version: 0.2 phases: pre_build: commands: - pip3 install localstack awscli - docker pull public.ecr.aws/localstack/localstack:latest - localstack start -d - localstack wait -t 30 ``` See the [CodeBuild guide](/aws/ci-pipelines/codebuild/) for the full setup. ```yaml stages: - test variables: DOCKER_HOST: tcp://docker:2375 DOCKER_TLS_CERTDIR: "" LOCALSTACK_HOST: "localstack:4566" services: - name: localstack/localstack:latest alias: localstack - name: docker:dind alias: docker command: ["--tls=false"] localstack-test: stage: test image: python:3.11 script: - pip install awscli-local - awslocal s3 mb s3://test-bucket ``` See the [GitLab CI guide](/aws/ci-pipelines/gitlab-ci/) for the full setup. ```yaml language: python services: - docker python: - "3.8" before_install: - python -m pip install localstack awscli-local[ver1] - docker pull localstack/localstack - localstack start -d - localstack wait -t 30 ``` See the [Travis CI guide](/aws/ci-pipelines/travis-ci/) for the full setup. You can also start from the [CI integrations overview](/aws/ci-pipelines/) if you want a broader explanation of the CI workflow. ## Authentication in CI CI environments should use a CI Auth Token. Create one from the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens), then store it as `LOCALSTACK_AUTH_TOKEN` in your CI provider's secret manager. Do not commit tokens to your repository or write them directly into workflow files. For more details on token types and rotation, see the [Auth Token guide](/aws/getting-started/auth-token/). ## State in CI Most CI jobs should start with a clean LocalStack instance. A fresh instance makes test runs reproducible and avoids hidden dependencies between jobs. If your pipeline needs state across jobs or workflow stages, use one of the state management options documented outside this getting started page: - [Cloud Pods](/aws/developer-tools/snapshots/cloud-pods/) to save and restore named LocalStack state snapshots. - [State export and import](/aws/developer-tools/snapshots/saving-snapshots-locally/) to move state through artifacts or caches. - [Persistence](/aws/developer-tools/snapshots/persistence/) when the same runner keeps a mounted LocalStack volume. ## Next steps After choosing your CI provider, continue to [AI & Agent Workflows](/aws/getting-started/ai-workflows/) to learn how AI coding assistants can help generate, deploy, and test LocalStack-backed AWS applications. # FAQ > Frequently asked questions about LocalStack for AWS. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## LocalStack Core FAQs ### How do I resolve SSL issues due to revoked local certificate for `localhost.localstack.cloud`? To resolve the issue follow the steps: 1. **Update to the latest LocalStack version:** To resolve the SSL issues due to revoked certificate, we strongly recommend updating to the latest LocalStack version (v3.7.0 and above)for the most reliable and seamless experience. 2. **Clear the cached certificate:** It's important to clear the cached certificate if you continue to experience the issue when updating to the latest LS version. This can be done by deleting the cached certificate file. For example, on Linux systems, you can locate and remove the file at `~/.cache/localstack/volume/cache/server.test.pem`. The exact path may differ depending on your operating system and how you've started LocalStack. Please refer to our [documentation](/aws/customization/advanced/filesystem/#localstack-volume-directory) for specific instructions. **Workarounds for older (<v3.7.0) LocalStack versions:** 1. **Disable Certificate Download**: To prevent downloading a revoked certificate, set the environment variable `SKIP_SSL_CERT_DOWNLOAD=1`. This will cause LocalStack to use a self-signed SSL certificate. Additionally, it's important to clear the cached certificate from your host machine as mentioned above. 2. **Use HTTP Instead of HTTPS**: Where possible, use `http://` instead of `https://` to avoid issues related to the revoked certificates. This workaround works with most browsers. However, Safari requires additional steps: 2.1. **Safari Users**: To make this work, you'll need to first navigate to the page in a new tab and accept the security warning. To do this, make sure that LocalStack is started with `SKIP_SSL_CERT_DOWNLOAD=1` and that you have cleared the cached certificate as mentioned above. Once you've accepted the warning, you should be able to proceed. For other SSL-related issues encountered during startup — such as Python `CERTIFICATE_VERIFY_FAILED` tracebacks or corporate TLS interception — see [How do I diagnose if my SSL traffic is being intercepted by a corporate proxy?](#how-do-i-diagnose-if-my-ssl-traffic-is-being-intercepted-by-a-corporate-proxy). ### Is using `localhost.localstack.cloud:4566` to set as the endpoint for AWS services recommended? `localhost.localstack.cloud` is the recommended endpoint - especially for S3, in order to enable host-based bucket endpoints. - When using this domain within LocalStack compute environments like Lambda, ECS or EC2, this domain name resolves to the LocalStack container via our DNS server available in version 2.3. - By configuring your environment, your applications can also use `localhost.localstack.cloud` to resolve to the LocalStack container via our DNS server. - In addition, we also publish an SSL certificate that is automatically used inside LocalStack, in order to enable HTTPS endpoints with valid certificates. Across our docs, we use `localhost.localstack.cloud:4566` instead of `localhost:4566`, as this is the recommended endpoint. However, we still provide `localhost:4566` as a fallback option to users, especially for users who are behind a corporate firewall or an internet service provider that does not allow resolving `localhost.localstack.cloud` properly. ### How should I use the latest LocalStack Docker images? :::note As of May 2026, the `latest` tag mirrors `stable` and is only updated on official releases. If you need the most recent unreleased changes, pull `localstack/localstack:dev` instead. ::: To use the latest LocalStack Docker images, you either run `docker pull localstack/localstack:latest` or use the `docker-compose pull` if the image is set to `localstack/localstack:latest`. You can also specify a particular digest to make sure you are using the correct image, like this: `localstack/localstack:latest@sha256:f803cc657843c6c7acf2631d15600783c3593e496fba418415afc87680d9d5bc`. You can also use the our diagnose endpoint (`http://localhost:4566/_localstack/diagnose`) to get the specific image hashes and compare them with the current (latest) images on [Docker Hub](https://hub.docker.com/r/localstack/). The diagnose endpoint is only available if you run LocalStack with `DEBUG=1`. ### What do the tags of the LocalStack Docker images mean? We publish a set of image tags with different semantics, updated on different occasions: - **`latest`**: Updated only on official tagged releases (e.g. `2026.05.0`, `2026.05.1`). Equivalent to `stable`. Recommended for most users who want a stable, release-quality image. As of May 2026, this tag no longer tracks untagged commits on `main`, use `dev` for that behavior. - **`stable`**: Same as `latest`. Updated with every official release. - **`dev`**: Contains all merged, untagged commits from the `main` branch. Use this if you want the latest unreleased changes. - **`YYYY.MM`** (e.g. `2026.05`): Updated with each patch release within that month. Use this to get the latest security fixes and dependency updates. - **`YYYY.MM.patch`** (e.g. `2026.05.0`): Pinned to an exact release and never updated. Use this for fully reproducible environments where even minor bugfix changes are undesirable. Starting with the end-of-March 2026 release, LocalStack follows [calendar versioning](https://calver.org/) for official releases. For releases up to and including `v4.14.0`, tags follow Semantic Versioning. Starting with the end-of-March 2026 release, LocalStack follows [calendar versioning](https://calver.org/) for official releases. For releases up to and including `v4.14.0`, tags follow Semantic Versioning. ### How can I access LocalStack from an alternative computer? You can access LocalStack from an alternative computer, by exposing port `4566` to the public network interface (`0.0.0.0` instead of `127.0.0.1`) in your `docker-compose.yml` configuration. However, we do not recommend using this setup - for security reasons, as it exposes your local computer to potential attacks from the outside world. ### How to resolve Git Bash issues with LocalStack? If you're using Git Bash with LocalStack, you might encounter some issues. This is due to the automatic conversion of POSIX paths to Windows paths when command-line options start with a slash. For instance, `"/usr/bin/bash.exe"` would be converted to `"C:\Program Files\Git\usr\bin\bash.exe"`. This conversion can cause problems when it's not needed, such as with `"--name /test/parameter/new"`. To prevent this, you can temporarily set the `MSYS_NO_PATHCONV` environment variable. Another workaround is to double the first slash in your command to prevent the POSIX-to-Windows path conversion. This will lead to issues with Git Bash ```bash aws ssm get-parameter --name "/test/parameter/new" ``` Option 1: Set the environment variable ```bash MSYS_NO_PATHCONV=1 aws ssm put-parameter --name "/test/parameter/new" --type String --value "test" ``` Option 2: Double the first slash ```bash aws ssm put-parameter --name "//test/parameter/new" --type String --value "test" ``` For additional known issues related to Git Bash, you can refer to the following link: [Git Bash Known Issues](https://github.com/git-for-windows/build-extra/blob/main/ReleaseNotes.md#known-issues) ### How do I resolve connection issues with proxy blocking access to LocalStack's BigData image? A company proxy can lead to connection issues. To allow access to the `localstack/bigdata` image, use the following Docker configuration in your `docker-compose.yml` file: ```yaml ... environment: - HTTP_PROXY= - NO_PROXY=.s3.localhost.localstack.cloud,127.0.0.1,*.localhost ... ``` For the broader corporate-proxy story (HTTPS proxy, outbound proxy variables, Zscaler-style TLS interception, and DNS), see [How do I configure LocalStack to use my corporate HTTP and HTTPS proxy?](#how-do-i-configure-localstack-to-use-my-corporate-http-and-https-proxy). ### Why is it that LocalStack is unable to connect to internet? You might be able to connect to the internet, but your Docker container can't connect. This can affect start of LocalStack. Please ensure that you are not using the `none` network driver when starting your docker container. More details about the default bridge network can be found on [official docker documentation](https://docs.docker.com/network/bridge). Please also ensure that the docker container has an assigned IP address, by running: ```bash docker inspect | jq -r '.[0].NetworkSettings.Networks | to_entries | .[].value.IPAddress' ``` At least one IP address should be returned. If you are using Linux, ensure that you have enabled IP forwarding: ```bash sudo sysctl -w net.ipv4.ip_forward=1 ``` If the container can reach the internet generally but not LocalStack endpoints specifically, the issue is more likely a corporate proxy, DNS, or TLS interception. Continue with [How do I verify outbound connectivity from inside the LocalStack container?](#how-do-i-verify-outbound-connectivity-from-inside-the-localstack-container). ### Why can't my other Docker containers reach LocalStack? Using LocalStack inside a Docker network with multiple other containers can lead to connectivity issues from/to those containers. For example, a container which attempts to deploy a stack and interact with the services directly, from within the same Docker network. Refer to our [network troubleshooting guide](/aws/customization/networking/) covering several scenarios. ### Why are some containers left behind after I stop LocalStack with Docker Compose? When LocalStack shuts down, it cleans up the auxiliary containers it started for services such as Lambda, ECS, RDS or EKS. If it is stopped before that cleanup finishes, those containers are left running. By default, LocalStack gets only a few seconds in which to shut down. Docker Compose sends `SIGTERM` to the container and follows up with `SIGKILL` once `stop_grace_period` expires (10 seconds by default), and LocalStack stops waiting for its own cleanup after `SHUTDOWN_TIMEOUT` (5 seconds by default). Whichever window runs out first, LocalStack is stopped mid-cleanup and the containers survive as orphans. Cleanup that takes longer than a few seconds is the most likely to be cut short. To give LocalStack enough time to shut down cleanly, increase `stop_grace_period` on the LocalStack service in your `docker-compose.yml` (for example, `3m`): ```yaml services: localstack: image: localstack/localstack-pro:latest stop_grace_period: 180s environment: - SHUTDOWN_TIMEOUT=180 # ... ``` Increase both settings. If you increase only `stop_grace_period`, LocalStack still gives up on its cleanup after 5 seconds and the container then stays up until the grace period runs out, so every `docker compose stop` waits the full three minutes. If a previous run already left containers behind, run `docker ps` and remove the ones LocalStack started before starting it again. This applies to Docker Compose specifically. Starting LocalStack with `lstk` is not affected, as `lstk` waits for LocalStack to finish shutting down. ### How to resolve the pull rate limit issue for LocalStack's Docker image? If you receive `ERROR: toomanyrequests: Too Many Requests.` when pulling the LocalStack Docker image, you have reached your pull rate limit. You may increase the limit by [authenticating and upgrading](https://www.docker.com/increase-rate-limits). Set your DockerHub credentials: ```bash (sudo) docker login --username=yourUsername ``` You can add in the volume `~/.docker/config.json:/config.json` where the `config.json` is saved and point the `DOCKER_CONFIG=/config.json` variable to the JSON file in the Docker image. ```yaml ... environment: - DOCKER_CONFIG=/config.json volumes: - ~/.docker/config.json:/config.json ... ``` If you have an active AWS account, you can use the public AWS ECR image. You can use the following command to pull the image: ```shell docker pull public.ecr.aws/localstack/localstack-pro:latest ``` ### How to increase IO performance for LocalStack's Docker image under Windows? :::note Some options that are not part of the standard configuration may have unintended consequences for AWS services that operate within LocalStack. For example, these options may interfere with the functionality of AppSync function executor, RDS MySQL persistence and SageMaker. We advise you to exercise caution. ::: You can change the LocalStack `volume` folder to use the WSL Linux file system instead of the Windows host folder. To do so, you need to change the [`docker-compose.yml`](https://github.com/localstack/localstack/blob/main/docker-compose-pro.yml) file and add the following lines: ```yaml showshowLineNumbers volumes: - "/var/run/docker.sock:/var/run/docker.sock" - "\\\\wsl$\\\\home\\\\volume:/var/lib/localstack" # mount volume in WSL2 Linux file system ``` --- As an alternative, you can set the volume as `- "~/volume:/var/lib/localstack"` then start Docker using command `wsl docker compose -f docker-compose.yml up`. ```yaml showshowLineNumbers volumes: - "/var/run/docker.sock:/var/run/docker.sock" - "localstack_data:/var/lib/localstack" # mount Docker volume volumes: localstack_data: ``` For more details visit [Docker WSL documentation](https://docs.docker.com/desktop/wsl), [Docker WSL best practices](https://docs.docker.com/desktop/wsl/best-practices) and [Docker Volumes documentation](https://docs.docker.com/storage/volumes/). ### Why does LocalStack fail to start with "enhanced container isolation: Docker socket mount denied"? This error occurs when Docker Desktop's [Enhanced Container Isolation](https://docs.docker.com/desktop/hardened-desktop/enhanced-container-isolation/) (ECI) feature is enabled, typically on Docker Business accounts, and LocalStack has not been added to the Docker socket mount allowlist. To fix this, ask your Docker Desktop administrator to add `localstack/localstack` and `localstack/localstack-pro` to the allowlist in your organisation's Settings Management policy. ``` json { "enhancedContainerIsolation": { "dockerSocketMount": { "imageList": { "images": [ "docker.io/localstack/localstack-pro:**", "docker.io/localstack/localstack:**" ], "allowDerivedImages": true } } } } ``` ## Startup Troubleshooting FAQs LocalStack startup failures most commonly come from one of three areas: **license activation**, **CA / SSL certificate validation**, or **outbound network access** (corporate proxies, Zscaler, restricted DNS). The FAQs below are ordered by topic — debug logging first, then license activation, SSL certificates, corporate proxy and DNS, air-gapped environments, and finally less common startup errors. If LocalStack exits with `exit code 55` or never finishes starting, start with [enabling debug logs](#how-do-i-enable-verbose-debug-logs-for-localstack-startup) and read the last lines of the log to identify which sub-case applies. ### How do I enable verbose debug logs for LocalStack startup? Almost every startup ticket can be resolved within minutes once the full debug log is available. LocalStack exposes two log-related environment variables: | Variable | Values | What it does | | :-- | :-- | :-- | | `DEBUG` | `0` (default), `1` | Verbose application logs and full Python stack traces on error | | `LS_LOG` | `warning`, `info` (default), `debug`, `trace`, `trace-internal` | Sets the log handler level. `trace` and `trace-internal` also imply `DEBUG=1` and add request/response bodies | For a startup issue, set both: ```shell DEBUG=1 LS_LOG=trace localstack start ``` Or via Docker Compose: ```yaml services: localstack: image: localstack/localstack-pro:latest environment: - DEBUG=1 - LS_LOG=trace - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN} ``` The CLI also supports a `--debug` flag that prints host-side debug output (host preparation, Docker command construction, license cache checks) on top of the container logs: ```shell localstack --debug start ``` See the [Logging reference](/aws/customization/logging/) for the full list of log-related options. ### How do I capture and share LocalStack container logs for troubleshooting? If LocalStack runs in Docker and exits or becomes unhealthy, capture the full container log starting from container start: ```shell docker ps -a | grep localstack docker logs docker logs > localstack.log 2>&1 ``` If LocalStack reached the point of serving HTTP and you started it with `DEBUG=1`, you can also pull a compressed diagnose bundle: ```shell curl -s localhost:4566/_localstack/diagnose | gzip -cf > diagnose.json.gz ``` The most useful lines for support are the ones immediately before `exit code 55` or the final traceback — please attach the full log rather than the last few lines. ### What hostnames must LocalStack be able to reach during startup? LocalStack must be able to make outbound HTTPS requests to the following hostnames over TCP/443 during startup. If any required hostname is blocked, startup will fail. | Hostname | Purpose | Required | | :-- | :-- | :-- | | `api.localstack.cloud` | License activation and per-org TLS cert download | Yes | | `assets.localstack.cloud` | Local TLS server cert and other static assets | Yes | | `localstack-pro-artifacts.s3.amazonaws.com` (and related S3 buckets) | Optional on-demand service packages (Glue, RDS engines, Tinkerpop, Flink, etc.) | When using those services | | `analytics.localstack.cloud` | Anonymous usage telemetry | No, can be disabled | A quick connectivity check from the host that runs LocalStack: ```shell curl -v https://api.localstack.cloud/v1/health dig api.localstack.cloud ``` You expect HTTP `200` and DNS `status: NOERROR`. If either fails, jump to [How do I verify outbound connectivity from inside the LocalStack container?](#how-do-i-verify-outbound-connectivity-from-inside-the-localstack-container). ### What does "Could not reach the LocalStack licensing server" mean and how do I fix it? When LocalStack prints `Could not reach the LocalStack licensing server… outbound HTTPS traffic is allowed` and exits with code 55, this is a **network problem, not a license problem**. The LocalStack container cannot reach `api.localstack.cloud:443`. Work through these steps in order: 1. [Verify outbound connectivity from inside the container](#how-do-i-verify-outbound-connectivity-from-inside-the-localstack-container). 2. [Configure LocalStack to use your corporate HTTP/HTTPS proxy](#how-do-i-configure-localstack-to-use-my-corporate-http-and-https-proxy). 3. [Fix DNS resolution inside the container](#how-do-i-fix-dns-resolution-issues-inside-the-localstack-container). If the network path is fine but the TLS handshake fails, the issue is corporate TLS interception. See [How do I trust my corporate TLS interceptor certificate inside LocalStack?](#how-do-i-trust-my-corporate-tls-interceptor-certificate-zscaler-netskope-and-similar-inside-localstack). ### Why does my license show as EXPIRED even though my subscription is active? If LocalStack exits with `Expected license to be ACTIVE, was EXPIRED`, the license LocalStack received from the server is in `EXPIRED` state. Common causes: 1. **Trial period ended.** Upgrade or extend the trial via the [LocalStack web app](https://app.localstack.cloud/account/subscriptions). 2. **Subscription renewed but the cached license file is stale.** Clear the cache so LocalStack requests a fresh license: ```shell # Host CLI cache (Linux) rm -f ~/.cache/localstack/license.json # macOS rm -f ~/Library/Caches/localstack-cli/license.json # Inside the container the cache lives at: # /var/lib/localstack/cache/license.json # If you mount a persistence volume, also clear it there. ``` 3. **The billing system has not yet activated your renewal.** Check the subscription status at [app.localstack.cloud/account/subscriptions](https://app.localstack.cloud/account/subscriptions). If the subscription is paid but still shows `EXPIRED`, open a support ticket with your workspace name and the last four characters of the auth token. 4. **Wrong license type assigned.** Sometimes a user is on an expired Trial or Hobby license. Unassign the old license and assign a paid one — see [Managing users and licenses](/aws/organizations-admin/managing-users-licenses/#managing-licenses). ### Why does my license show as SUSPENDED? `Expected license to be ACTIVE, was SUSPENDED` means the license is server-side suspended. This is almost always a billing or admin action — payment failure, plan downgrade, or a workspace admin pausing access. Resolve via the web app billing page, or contact support. ### What does `licensing.license.not_assigned` mean? Your auth token is valid, but the licensing server cannot match it to an assigned seat. Either: - A workspace admin needs to assign you a seat at [app.localstack.cloud/workspace/members](https://app.localstack.cloud/workspace/members), or - You purchased a license but haven't assigned it to yourself yet (common in single-engineer setups where the billing page shows `0 / 1 used`). ### What does `licensing.license.not_enough_credits` mean? You have more active engineers than purchased seats. Either un-assign a user or buy more seats on the billing page. ### What does `licensing.license.product_error` or "your LocalStack license requires version X.Y.Z or higher" mean? You are either using a feature your license tier does not include, or running an outdated LocalStack image. Two fixes: - Pin a current image, e.g. `localstack/localstack-pro:latest` (or the explicit version your license requires). - Confirm your subscription tier matches the feature you are using — some Pro features such as Snowflake, Chaos, or Enterprise extensions require specific entitlements. ### What does "The credentials defined in your environment are invalid" mean? The auth token is malformed, missing, or you accidentally pasted the old-style API key into the new `LOCALSTACK_AUTH_TOKEN` variable. Verify the token locally first: ```shell localstack auth show-token # Should print: Valid: True ``` Then check: - The variable name is exactly `LOCALSTACK_AUTH_TOKEN`. - The token includes the `ls-…` prefix. - There is no trailing whitespace or quote character. - The token has not been committed to source control — if it has, regenerate it immediately at [app.localstack.cloud/workspace/auth-tokens](https://app.localstack.cloud/workspace/auth-tokens). ### What does "User exists in different workspace" mean? You signed up earlier with the same email and got assigned to a different workspace. Sign in to the web app with that account, leave the old workspace, then accept the new invitation. ### How does LocalStack cache its license for offline or intermittent connectivity? LocalStack caches the last validated license at `/var/lib/localstack/cache/license.json` inside the container. As long as the cache is valid, LocalStack can start without reaching `api.localstack.cloud` on every boot. If your environment has intermittent connectivity, mount this directory as a persistent volume so the cached license survives container restarts: ```yaml volumes: - "./localstack-volume:/var/lib/localstack" ``` For fully air-gapped environments, see [How do I run LocalStack in a fully air-gapped environment?](#how-do-i-run-localstack-in-a-fully-air-gapped-environment). ### How do I diagnose if my SSL traffic is being intercepted by a corporate proxy? SSL errors during startup usually show up as Python tracebacks ending in one of: - `[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate` - `[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain` - `[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Basic Constraints of CA cert not marked critical` All three mean Python (inside the LocalStack container) refused to trust the certificate presented by the server. Before fixing, diagnose the root cause. From inside the LocalStack container — or any host on the same network — run: ```shell echo | openssl s_client -connect api.localstack.cloud:443 -servername api.localstack.cloud 2>/dev/null \ | openssl x509 -noout -issuer -subject ``` - If the issuer is **a public CA** (`ZeroSSL`, `Let's Encrypt`, `DigiCert`, …) the certificate is fine and your container just doesn't trust the public CA bundle. See [How do I provide a corporate or updated CA bundle to LocalStack?](#how-do-i-provide-a-corporate-or-updated-ca-bundle-to-localstack). - If the issuer is **your company's CA** (`Zscaler`, `Netskope`, an internal corporate CA), outbound TLS is being intercepted. See [How do I trust my corporate TLS interceptor certificate inside LocalStack?](#how-do-i-trust-my-corporate-tls-interceptor-certificate-zscaler-netskope-and-similar-inside-localstack). ### How do I trust my corporate TLS interceptor certificate (Zscaler, Netskope, and similar) inside LocalStack? This is by far the most common issue for corporate Zscaler / Netskope / Palo Alto / Cisco Umbrella users. The interceptor terminates TLS and presents its own cert signed by a CA your laptop trusts but the LocalStack container does not. You must inject that CA into the container. The cleanest fix is to build a thin image on top of LocalStack and bake the CA in: ```dockerfile FROM localstack/localstack-pro:latest # Replace the URL with your organisation's CA bundle, or COPY it from your build context ADD https://mobile.zscaler.net/downloads/zscaler2048_sha256.crt \ /usr/local/share/ca-certificates/zscaler.crt RUN update-ca-certificates ENV CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt \ REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt \ NODE_EXTRA_CA_CERTS=/etc/ssl/certs/ca-certificates.crt ``` Then use this image instead of the upstream one in your `docker-compose.yml`. If you can't build a custom image, mount the CA bundle at runtime and point Python, Node, and curl at it: ```yaml environment: - REQUESTS_CA_BUNDLE=/etc/ssl/certs/corp-ca.crt - CURL_CA_BUNDLE=/etc/ssl/certs/corp-ca.crt - NODE_EXTRA_CA_CERTS=/etc/ssl/certs/corp-ca.crt volumes: - "/path/to/your/corp-ca.crt:/etc/ssl/certs/corp-ca.crt:ro" ``` `REQUESTS_CA_BUNDLE` is one of a small set of host environment variables that LocalStack forwards into the container automatically. If you see a CLI warning that this variable is being forwarded, that's expected — but prefer setting `LOCALSTACK_REQUESTS_CA_BUNDLE` to make the intent explicit. ### How do I provide a corporate or updated CA bundle to LocalStack? If the issuer on the server cert is a public CA but you still get `unable to get local issuer certificate`, the container's bundled CA store is missing or stale. This typically happens when LocalStack ships a newer Python that enforces stricter validation — for example, after the base image moved to Python 3.13, which enforces the `Basic Constraints critical` check on intermediate CAs. Two options: 1. Upgrade to the latest LocalStack image — these issues have been progressively fixed upstream. 2. Mount an updated CA bundle from your host (e.g. `/etc/ssl/certs/ca-certificates.crt` on Debian/Ubuntu) using the same pattern as the [corporate TLS interceptor FAQ](#how-do-i-trust-my-corporate-tls-interceptor-certificate-zscaler-netskope-and-similar-inside-localstack) above. ### How do I skip the per-organization TLS certificate download on startup? On startup LocalStack tries to download a per-org TLS cert from `api.localstack.cloud/v1/proxy/localstack.cert.key`. If that one specific request is blocked but everything else works, set: ```shell SKIP_SSL_CERT_DOWNLOAD=1 ``` LocalStack falls back to its bundled self-signed certificate. This **only** suppresses the download step; it does not disable license activation, which still requires reaching `api.localstack.cloud`. The same flag is also useful when working around a revoked `localhost.localstack.cloud` certificate — see [How do I resolve SSL issues due to revoked local certificate for `localhost.localstack.cloud`?](#how-do-i-resolve-ssl-issues-due-to-revoked-local-certificate-for-localhostlocalstackcloud). ### How do I disable TLS verification entirely (last resort)? ```shell SSL_NO_VERIFY=1 ``` This disables outbound TLS verification entirely. Acceptable for short-term debugging on a single developer machine; **not recommended for shared CI environments** because it hides real misconfigurations and downgrades your security posture. ### How do I verify outbound connectivity from inside the LocalStack container? If the host can reach `api.localstack.cloud` but LocalStack can't, the container is missing proxy or DNS settings. Exec into the container and run the same checks you would on the host: ```shell docker exec -it sh # DNS nslookup api.localstack.cloud # TCP + TLS curl -v https://api.localstack.cloud/v1/health ``` If the host succeeds and the container fails, jump to the [proxy FAQ](#how-do-i-configure-localstack-to-use-my-corporate-http-and-https-proxy) or [DNS FAQ](#how-do-i-fix-dns-resolution-issues-inside-the-localstack-container) depending on which step failed. ### How do I configure LocalStack to use my corporate HTTP and HTTPS proxy? LocalStack honours both the standard and LocalStack-prefixed proxy variables. Setting all of them is safe: ```yaml environment: # Standard Docker / Linux variables - HTTP_PROXY=http://proxy.corp.example.com:8080 - HTTPS_PROXY=http://proxy.corp.example.com:8080 - NO_PROXY=localhost,127.0.0.1,.localstack.cloud,169.254.169.254 # LocalStack-specific, used for outbound calls from inside the container - OUTBOUND_HTTP_PROXY=http://proxy.corp.example.com:8080 - OUTBOUND_HTTPS_PROXY=http://proxy.corp.example.com:8080 ``` `NO_PROXY` should include: - `localhost` and `127.0.0.1` - Internal corporate hostnames you do not want routed through the proxy - `169.254.169.254` (the AWS-style metadata endpoint LocalStack emulates) - Your Docker network's hostnames if you run multi-container setups If TLS handshake errors appear after the proxy is configured, your proxy is probably intercepting TLS — see [How do I trust my corporate TLS interceptor certificate inside LocalStack?](#how-do-i-trust-my-corporate-tls-interceptor-certificate-zscaler-netskope-and-similar-inside-localstack). ### How do I fix DNS resolution issues inside the LocalStack container? If `nslookup api.localstack.cloud` fails inside the container, the container is inheriting a DNS server it can't reach — common with corporate split-horizon DNS. Two ways to fix: **Option A — tell LocalStack which DNS to use:** ```shell DNS_ADDRESS=0 DNS_SERVER=8.8.8.8 # or your corporate-approved public resolver, e.g. 1.1.1.1 ``` `DNS_ADDRESS=0` tells LocalStack's embedded DNS server not to bind to the container interface; `DNS_SERVER` is the upstream resolver LocalStack uses for any name it does not own. **Option B — configure Docker daemon DNS** in Docker Desktop → Settings → Docker Engine, add: ```json { "dns": ["10.95.161.250", "8.8.8.8"] } ``` Replace `10.95.161.250` with your corporate DNS server and leave a public resolver as a fallback. Apply & Restart. If `dig api.localstack.cloud` returns `NXDOMAIN` or `SERVFAIL`, some corporate DNS servers filter the `localstack.cloud` zone — ask your network administrator to safelist `localstack.cloud` domains. ### What hostnames should I ask my network team to allowlist for LocalStack? If your security team manages an explicit allow-list, request the following hostnames over TCP/443: - `api.localstack.cloud` - `assets.localstack.cloud` - `analytics.localstack.cloud` *(optional — telemetry only)* - `localstack-pro-artifacts.s3.amazonaws.com` *(if you use Glue, RDS engines beyond the default, Tinkerpop, Flink, etc.)* The allow-list must cover the entire **TLS handshake**, not just the URL pattern — Zscaler / Netskope policies sometimes break by inspecting and rewriting the cert. If that happens, see [How do I trust my corporate TLS interceptor certificate inside LocalStack?](#how-do-i-trust-my-corporate-tls-interceptor-certificate-zscaler-netskope-and-similar-inside-localstack). ### How do I run LocalStack in a fully air-gapped environment? Offline images for airgapped environments are available on our enterprise tier, please reach out to our sales team. ### Why does LocalStack fail with "ports are not available: exposing port TCP 127.0.0.1:443"? Another process is bound to port `443` — often a previous Docker run that didn't clean up, or a local web server. Identify the conflict and either stop it or remove the port from the Compose file if you don't need the HTTPS edge: ```shell lsof -i :443 # or netstat -anv | grep 443 ``` ### Why does the LocalStack container exit immediately with SIGILL (exit code 252) on Apple Silicon? On newer Apple Silicon hardware (for example Apple M4), the container can crash immediately after license activation with exit code `252` (SIGILL, illegal instruction, reported as `-4` by some supervisors). This is a known incompatibility between the `cryptography` library (version 47.0.0 and newer) and older Linux guest kernels running under Apple's Virtualization Framework: the library tries to use ARM CPU instructions the kernel does not fully support. It mainly affects Colima or Podman with an older guest kernel (for example `6.8.0-39`); a fully updated Docker Desktop is generally not affected. The supervisor decodes and logs a fatal signal like this unconditionally, without needing `DEBUG=1`: ```text localstack process (PID 24) was terminated by signal 4 (SIGILL) (exit code 252). If there is no traceback above, the crash happened in native code, ... ``` It also enables Python's `faulthandler` in the `localstack` process by default (`PYTHONFAULTHANDLER=1`), so a native crash such as this one prints a Python and C stack trace before the process dies, for example pointing directly at the `cryptography` import. Set `PYTHONFAULTHANDLER=0` yourself if you need to opt out. Exit-code semantics are unchanged, the container still exits with the crashed process's status. **Workaround:** disable the faulty ARM capability detection by setting `OPENSSL_armcap=0` on the container: ```yaml services: localstack: image: localstack/localstack-pro:latest environment: - OPENSSL_armcap=0 ``` With `lstk`, set it as a `LOCALSTACK_`-prefixed host variable so `lstk start` forwards it into the container: ```bash LOCALSTACK_OPENSSL_armcap=0 lstk start ``` **Permanent fix:** update your VM provider (for example Colima) or its underlying Linux guest kernel to a newer version (for example `7.0.0`, or a patched `6.8.0`). ### Why do I see a warning about non-prefixed `REQUESTS_CA_BUNDLE` being forwarded? The CLI prints `Non-prefixed environment variable REQUESTS_CA_BUNDLE is forwarded…` when it auto-forwards a host environment variable into the container. The warning is informational — to silence it, use the prefixed form `LOCALSTACK_REQUESTS_CA_BUNDLE` instead. ### Why do my AWS SDK or CDK clients fail with "x509: certificate is valid for *.localhost.localstack.cloud, … not …amazonaws.com"? A client (CDK, the AWS Load Balancer Controller, an AWS SDK) is targeting real AWS hostnames (e.g. `s3.amazonaws.com`) but the request is actually being routed to LocalStack, which presents its local TLS cert. Pick one fix depending on the client: - Point the client at LocalStack explicitly with `AWS_ENDPOINT_URL=http://localhost:4566` (or `http://localhost.localstack.cloud:4566`), preferably over HTTP for local dev. - Or disable cert validation for the client during local development. ### What information should I include when contacting LocalStack support about a startup issue? To speed up your ticket, attach: 1. The full container log captured with `DEBUG=1 LS_LOG=trace`, from container start until exit. 2. The compressed diagnose bundle if LocalStack reached the point of serving HTTP: ```shell curl -s localhost:4566/_localstack/diagnose | gzip -cf > diagnose.json.gz ``` 3. LocalStack image tag — `docker inspect --format '{{.Config.Image}}'`. 4. CLI version — `localstack --version`. 5. Host OS, architecture, and Docker runtime (Docker Desktop, Colima, Rancher, Podman, Linux native). 6. Output of: ```shell curl -v https://api.localstack.cloud/v1/health dig api.localstack.cloud ``` 7. Whether you are behind a corporate proxy, Zscaler, Netskope, or similar — mention the product name explicitly. Half of all "outbound HTTPS allowed" tickets resolve to a known Zscaler interception pattern. Open a ticket by emailing `support@localstack.cloud`. ### What are the most useful environment variables for LocalStack startup? | Variable | Purpose | | :-- | :-- | | `LOCALSTACK_AUTH_TOKEN` | Personal or workspace auth token used for license activation | | `DEBUG` | `1` enables verbose logs and full Python stack traces | | `LS_LOG` | `debug`, `trace`, or `trace-internal` for progressively more detail | | `HTTP_PROXY` / `HTTPS_PROXY` / `NO_PROXY` | Standard proxy variables | | `OUTBOUND_HTTP_PROXY` / `OUTBOUND_HTTPS_PROXY` | LocalStack-specific outbound proxy variables used inside the container | | `DNS_SERVER` | Upstream DNS used by LocalStack's embedded resolver | | `DNS_ADDRESS` | Set to `0` to disable LocalStack's DNS server binding | | `REQUESTS_CA_BUNDLE` / `CURL_CA_BUNDLE` / `NODE_EXTRA_CA_CERTS` | CA bundle path for Python, curl, and Node respectively | | `LOCALSTACK_REQUESTS_CA_BUNDLE` | Prefixed form of `REQUESTS_CA_BUNDLE` that avoids the CLI auto-forward warning | | `SKIP_SSL_CERT_DOWNLOAD` | `1` to skip the per-org TLS cert download on startup | | `SSL_NO_VERIFY` | `1` to disable outbound TLS verification (debug only) | For the full configuration reference, see the [Configuration reference](/aws/customization/configuration-options/) and the broader [Networking documentation](/aws/customization/networking/). ## LocalStack Platform FAQs ### Where can I check the status of LocalStack's services? LocalStack will provide the current status of it's services and any relevant details regarding any outages or incidents at [status.localstack.cloud](https://status.localstack.cloud). ### Where are my Cloud Pods stored? LocalStack provides a secure storage mechanism to store Cloud Pods on the Web Application. When you push a Cloud Pod, it is stored securely in our storage backend in AWS, with each user/organization receiving a dedicated, isolated S3 bucket. Pushing and pulling a Cloud Pod from our Web Application is facilitated by using secure S3 pre-signed URLs for the Cloud Pods CLI to interact directly with the S3 bucket, rather than piping the state files through our LocalStack Platform APIs. ### How do I check if my license is valid and activated? The easiest way to check if LocalStack for AWS is activated is to check the health endpoint of LocalStack for a list of the running services: ```bash curl localhost:4566/_localstack/health | jq ``` If a service like [XRay](/aws/services/xray) is running, LocalStack for AWS has started successfully. If your Auth Token is invalid, you will see an error message like this in the logs of LocalStack: ```bash license activation failed! Reason: ... ``` If this error occurs, something is wrong with your Auth Token or license. Make sure your Auth Token is set correctly (check for typos!) and your license is valid. If the Auth Token still does not work, please [contact us](https://localstack.cloud/contact/). The `Reason:` text in the log line tells you which specific failure you're hitting. The most common reasons each have their own FAQ: - [`Could not reach the LocalStack licensing server`](#what-does-could-not-reach-the-localstack-licensing-server-mean-and-how-do-i-fix-it) - [`Expected license to be ACTIVE, was EXPIRED`](#why-does-my-license-show-as-expired-even-though-my-subscription-is-active) - [`Expected license to be ACTIVE, was SUSPENDED`](#why-does-my-license-show-as-suspended) - [`licensing.license.not_assigned`](#what-does-licensinglicensenot_assigned-mean) - [`licensing.license.not_enough_credits`](#what-does-licensinglicensenot_enough_credits-mean) - [`licensing.license.product_error`](#what-does-licensinglicenseproduct_error-or-your-localstack-license-requires-version-xyz-or-higher-mean) - [`The credentials defined in your environment are invalid`](#what-does-the-credentials-defined-in-your-environment-are-invalid-mean) ### What should I do if I cannot connect to LocalStack API? If your log output contains lines like: ```shell WARNING:localstack_ext.bootstrap.licensing: Error activating API key "abc..."(10): ... ConnectionRefusedError: [Errno 111] Connection refused ``` LocalStack cannot contact our API to perform the license activation. Confirm with your network administrator that no policies block the connection to our backend. Before opening a ticket, work through the connectivity FAQs in order — they cover the great majority of "outbound HTTPS allowed but LocalStack still can't reach us" cases: 1. [Enable verbose debug logs](#how-do-i-enable-verbose-debug-logs-for-localstack-startup) and capture the full traceback. 2. [Verify outbound connectivity from inside the container](#how-do-i-verify-outbound-connectivity-from-inside-the-localstack-container). 3. [Configure the corporate HTTP/HTTPS proxy](#how-do-i-configure-localstack-to-use-my-corporate-http-and-https-proxy) if there is one. 4. [Trust the corporate TLS interceptor certificate](#how-do-i-trust-my-corporate-tls-interceptor-certificate-zscaler-netskope-and-similar-inside-localstack) if the proxy intercepts TLS. ### What should I do if I cannot resolve `api.localstack.cloud`? Log output like the following indicates that your machine cannot resolve the domain of the LocalStack API. ```shell WARNING:localstack_ext.bootstrap.licensing: Error activating API key "abc..."(10): ... socket.gaierror: [Errno -3] Temporary failure in name resolution ``` Confirm this by using a tool like `dig`: ```bash dig api.localstack.cloud ``` If the result has some other status than `status: NOERROR,` your machine cannot resolve this domain. Some corporate DNS servers filter requests to certain domains — contact your network administrator to safelist `localstack.cloud` domains. If the host can resolve the domain but the container can't, the container is inheriting a DNS server it can't reach. Two ways to fix: **Option A — tell LocalStack which DNS to use:** ```shell DNS_ADDRESS=0 DNS_SERVER=8.8.8.8 # or your corporate-approved public resolver, e.g. 1.1.1.1 ``` **Option B — configure Docker daemon DNS** in Docker Desktop → Settings → Docker Engine: ```json { "dns": ["10.95.161.250", "8.8.8.8"] } ``` Replace `10.95.161.250` with your corporate DNS server. Apply & Restart. For the full DNS troubleshooting flow, see [How do I fix DNS resolution issues inside the LocalStack container?](#how-do-i-fix-dns-resolution-issues-inside-the-localstack-container). ### How does LocalStack for AWS handle security patches and bug fixes? We take security seriously and respond to any emergency vulnerabilities as soon as possible. Our cloud provider (AWS) handles most of the infrastructure maintenance for us. We also use Infrastructure-as-Code scripts to ensure that our infrastructure configuration is consistent and recoverable in case of a disaster. ### How does LocalStack ensure the security of its containers and images? Our software assets are regularly checked for vulnerabilities, such as code issues and outdated dependencies. We use Dependabot to scan our GitHub repositories, and Trivy as well as Snyk (among other security tools) to scan our Docker images. ### Does LocalStack provide offline capabilities? Yes, the LocalStack image does provide limited offline capabilities. To use a fully-fledged offline mode, you may use LocalStack Enterprise, which can be used in air-gapped environments. The regular LocalStack Docker images may need to download additional dependencies for specific services (e.g., Elasticsearch, Big Data services) at runtime, while the offline image bakes all dependencies into the image, along with any other configuration that you might need. For more details, please take a look at our [Enterprise offering](https://localstack.cloud/pricing). For the licensing side of offline / restricted environments — including how to cache the license across container restarts and how to use an offline `license.json` issued by support — see [How does LocalStack cache its license for offline or intermittent connectivity?](#how-does-localstack-cache-its-license-for-offline-or-intermittent-connectivity) and [How do I run LocalStack in a fully air-gapped environment?](#how-do-i-run-localstack-in-a-fully-air-gapped-environment). ### How does the LocalStack Web Application communicate with the LocalStack container? The LocalStack Web Application connects to your LocalStack container running on your local machine and retrieves the information directly via the `localhost` without using the internet. Features such as Resource Browsers, IAM Policy Stream, Chaos Engineering dashboard, and others communicate directly with the LocalStack container using your browser. None of the information is sent to the internet, or stored on any external servers maintained by LocalStack. ### Why can't I access my LocalStack instance in the Web Application when using Chrome? If you are using Google Chrome and encounter an error accessing your LocalStack instance (e.g., at `localhost.localstack.cloud:4566`) from the [Web Application](https://app.localstack.cloud), it is likely due to Chrome's recent security changes regarding [**Private Network Access**](https://developer.chrome.com/blog/local-network-access). This change requires you to explicitly grant the LocalStack Web Application permission to communicate with your local network: 1. In your Chrome browser, navigate to the LocalStack Web Application: `https://app.localstack.cloud`. 2. Click the **lock icon** located to the left of the URL. 3. Select **Site settings** (or **Settings** if shown directly). 4. Scroll down to the **Local network access** setting. 5. Change the setting to **Allow**. 6. Refresh the Web App page. This resolves the issue by allowing the public-facing Web Application to access your LocalStack instance running on your local machine. # Installation > Install LocalStack with lstk, Docker, Docker Compose, or Helm. import { Code, LinkButton, Tabs, TabItem } from '@astrojs/starlight/components'; import { LOCALSTACK_AWS_VERSION } from 'astro:env/server'; ## Introduction LocalStack provides multiple installation paths depending on your development environment and requirements. We recommend a CLI-based installation for the most consistent local startup experience. Use [`lstk`](#lstk) to install, authenticate, and start LocalStack with minimal setup. LocalStack for AWS features require an [Auth Token](/aws/getting-started/auth-token/) to activate your running instance. `lstk` handles authentication through a browser-based login flow, while Docker and CI workflows can use `LOCALSTACK_AUTH_TOKEN`. ## lstk `lstk` is a lightweight CLI for LocalStack that manages the authentication and container lifecycle in a single workflow. **Requirement:** You must have a working [Docker installation](https://docs.docker.com/get-docker/) before proceeding. ### Install lstk {/* prettier-ignore-start */} ```bash brew install localstack/tap/lstk ``` ```bash npm install -g @localstack/lstk ``` Download the binary for your platform from the [GitHub Releases](https://github.com/localstack/lstk/releases) and add it to your `PATH`. {/* prettier-ignore-end */} ### Start lstk ```bash lstk start ``` The first execution initiates a browser-based login flow. Subsequent starts use credentials stored in your system keyring. ### Update lstk ```bash lstk update ``` For more details, see the [lstk documentation](/aws/developer-tools/running-localstack/lstk/). ## Container and orchestration tools Use these methods when you need explicit container configuration, want to run LocalStack alongside other services, or deploy LocalStack in CI and Kubernetes environments. For everyday local development, `lstk` is usually simpler. ### Docker Compose Use Docker Compose when you want a reusable configuration file that can be shared across a team or checked into a project repository. Create a `docker-compose.yml` with the following configuration: ```yaml showLineNumbers services: localstack: container_name: '${LOCALSTACK_DOCKER_NAME:-localstack-main}' image: localstack/localstack ports: - '127.0.0.1:4566:4566' # LocalStack Gateway - '127.0.0.1:4510-4559:4510-4559' # external services port range - '127.0.0.1:443:443' # LocalStack HTTPS Gateway environment: # Activate LocalStack for AWS: https://docs.localstack.cloud/getting-started/auth-token/ - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} # LocalStack configuration: https://docs.localstack.cloud/references/configuration/ - DEBUG=${DEBUG:-0} - PERSISTENCE=${PERSISTENCE:-0} volumes: - '${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack' - '/var/run/docker.sock:/var/run/docker.sock' ``` Execute `docker compose up` to start. ### Docker CLI Use the Docker CLI for one-off starts or when you want to test a container configuration before moving it into Compose: ```bash docker run \ --rm -it \ -p 127.0.0.1:4566:4566 \ -p 127.0.0.1:4510-4559:4510-4559 \ -p 127.0.0.1:443:443 \ -e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} \ -v /var/run/docker.sock:/var/run/docker.sock \ localstack/localstack ``` :::note The Docker Compose and Docker CLI examples above use the same runtime settings: - The `4566` port exposes the LocalStack Gateway. - The `4510-4559` range exposes external service ports used by services that bind additional endpoints. - The `443` port exposes the LocalStack HTTPS Gateway. - The Docker socket mount is required for services that start additional containers, such as Lambda. - Docker reuses a local image if one already exists. Pull explicitly or pin an image tag, such as `localstack/localstack:`, when you need reproducible CI or team environments. - If you use Docker bridge networking, container name resolution may not work as expected from other containers. Prefer the default LocalStack networking setup unless you have a specific reason to customize it. - Configuration variables can be prefixed with `LOCALSTACK_` in Docker. For instance, setting `LOCALSTACK_PERSISTENCE=1` is equivalent to `PERSISTENCE=1`. For more details, see the [Docker images](/aws/customization/other-installations/docker-images/), [configuration](/aws/customization/configuration-options/), and [networking](/aws/customization/networking/) documentation. ::: ### Helm (Kubernetes) Deploy LocalStack to a Kubernetes cluster: ```bash helm repo add localstack-repo https://helm.localstack.cloud helm upgrade --install localstack localstack-repo/localstack ``` ## Graphical user interfaces (GUIs) ### LocalStack Desktop Manage local instances via a standalone desktop application. [Download here](https://app.localstack.cloud/download). Install the [official extension](https://hub.docker.com/extensions/localstack/localstack-docker-desktop) to manage LocalStack directly from the Docker Desktop. ## Troubleshooting Installation issues typically fall into one of three areas: getting your chosen install method working, activating your license, or reaching LocalStack over the network. This section will guide you to the appropriate guide that contains detailed fixes based on your installation method. ### lstk If you installed via [`lstk`](#lstk) and LocalStack fails to start, authenticate, or pull its image, see [lstk troubleshooting](/aws/developer-tools/running-localstack/lstk/faq-and-troubleshooting/#troubleshooting). For first-run authentication (browser login, keyring tokens, or `LOCALSTACK_AUTH_TOKEN` in CI), refer to [Authentication](/aws/developer-tools/running-localstack/lstk/authentication/) and the [Auth Token guide](/aws/getting-started/auth-token/). ### Docker Compose and Docker CLI If you started LocalStack with [Docker Compose](#docker-compose) or the [Docker CLI](#docker-cli): - **License or credential errors**: see [Auth Token troubleshooting](/aws/getting-started/auth-token/#troubleshooting). - **Container exits during startup, proxy/DNS/TLS issues, or port conflicts**: see [Startup Troubleshooting FAQs](/aws/getting-started/faq/#startup-troubleshooting-faqs). - **DNS server or port conflicts**: see the [DNS Server guide](/aws/customization/networking/dns-server/). Ensure you have exported `LOCALSTACK_AUTH_TOKEN` in your shell before running `docker compose up` or `docker run`. ### View logs Stream container logs using the command that matches your install method: ```bash lstk logs ``` For `lstk` CLI diagnostics (separate from container logs), see [Logging](/aws/developer-tools/running-localstack/lstk/automation/#logging). ```bash docker compose logs -f localstack ``` ```bash docker logs -f localstack-main ``` Use the container name from your `docker run --name` flag if you set one. ```bash kubectl logs -f deployment/localstack ``` The deployment name follows your Helm release name (default: `localstack`). To enable verbose startup logging, capture logs from a failed container, or share a diagnose bundle with support, see [How do I capture and share LocalStack container logs for troubleshooting?](/aws/getting-started/faq/#how-do-i-capture-and-share-localstack-container-logs-for-troubleshooting). ### Network connectivity If your application cannot reach LocalStack after installation, see the [networking documentation](/aws/customization/networking/). ## Next steps Now that you've completed installation, proceed to the [Auth Token guide](/aws/getting-started/auth-token/) to activate LocalStack and prepare your environment for local development. # Local Development > Deploy an AWS serverless API locally using Lambda and DynamoDB on LocalStack. import { Code, Tabs, TabItem, Steps } from '@astrojs/starlight/components'; import { LOCALSTACK_AWS_VERSION } from 'astro:env/server'; ## Introduction This guide walks you through starting LocalStack and deploying a serverless API consisting of a Lambda function and a DynamoDB table. You will perform the entire deployment on your local machine without an AWS account. A successful deployment results in a: - **Serverless API:** A Lambda function with a configured function URL. - **Persistence Layer:** A DynamoDB table for message storage. - **Local Cloud Environment:** A fully functional local sandbox that emulates AWS services. Choose your preferred deployment method: **Terraform** or **AWS CLI**. ## Prerequisites - [Docker](https://docs.docker.com/get-docker/) engine installed and running. - A [LocalStack account](https://app.localstack.cloud/sign-up) and a valid [LocalStack Auth Token](/aws/getting-started/auth-token/). - Either [Terraform CLI](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) or [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) installed, depending on your preferred deployment method. If you haven't installed LocalStack yet, follow the [installation guide](/aws/getting-started/installation/) to get started. ## Step 1: Install and start LocalStack Start LocalStack: ```bash lstk start ``` The first run triggers a browser-based authentication flow. After authentication, the CLI pulls the LocalStack image and initializes the container. When the container is ready, you will see the following logs: ```text ✔︎ LocalStack ready (containerId: 400b3e61f3c6) • Endpoint: localhost.localstack.cloud:4566 • Web app: https://app.localstack.cloud ``` ## Step 2: Deploy the serverless API You can deploy the Lambda function and DynamoDB table using either our AWS CLI wrapper `lstk aws` or our Terraform wrapper `tflocal`. These tools automatically route AWS API calls to your LocalStack container, so you do not need AWS account credentials for this guide. 1. Create the Lambda function source. Execute the following to create a project directory, a function file and a Python handler: ```bash mkdir -p /tmp/localstack-demo cat > /tmp/localstack-demo/handler.py << 'EOF' import json, boto3, os, uuid def handler(event, context): table = boto3.resource('dynamodb').Table(os.environ['TABLE_NAME']) method = event.get('requestContext', {}).get('http', {}).get('method', 'GET') # Function URL POST, or direct invoke (e.g. Resource Browser) with a message if method == 'POST' or 'message' in event: data = json.loads(event.get('body', '{}')) if method == 'POST' else event item = {'id': str(uuid.uuid4()), **data} table.put_item(Item=item) return {'statusCode': 200, 'body': json.dumps(item)} result = table.scan() return {'statusCode': 200, 'body': json.dumps(result['Items'])} EOF cd /tmp/localstack-demo && zip handler.zip handler.py ``` 2. Create the DynamoDB table: ```bash lstk aws dynamodb create-table \ --table-name Messages \ --attribute-definitions AttributeName=id,AttributeType=S \ --key-schema AttributeName=id,KeyType=HASH \ --billing-mode PAY_PER_REQUEST ``` 3. Deploy the Lambda function: ```bash lstk aws lambda create-function \ --function-name messages-api \ --runtime python3.12 \ --handler handler.handler \ --zip-file fileb:///tmp/localstack-demo/handler.zip \ --role arn:aws:iam::000000000000:role/lambda-role \ --environment Variables={TABLE_NAME=Messages} lstk aws lambda wait function-active --function-name messages-api ``` 4. Configure a function URL and retrieve the endpoint: ```bash lstk aws lambda create-function-url-config \ --function-name messages-api \ --auth-type NONE LAMBDA_URL=$(lstk aws lambda list-function-url-configs \ --function-name messages-api \ --query 'FunctionUrlConfigs[0].FunctionUrl' \ --output text) echo $LAMBDA_URL ``` 1. Create a project directory and a `main.tf` file: ```bash mkdir -p /tmp/localstack-demo cat > /tmp/localstack-demo/main.tf << 'TF' terraform { required_providers { aws = { source = "hashicorp/aws" } archive = { source = "hashicorp/archive" } } } resource "aws_dynamodb_table" "messages" { name = "Messages" billing_mode = "PAY_PER_REQUEST" hash_key = "id" attribute { name = "id" type = "S" } } data "archive_file" "lambda" { type = "zip" output_path = "${path.module}/handler.zip" source { filename = "handler.py" content = <<-EOF import json, boto3, os, uuid def handler(event, context): table = boto3.resource('dynamodb').Table(os.environ['TABLE_NAME']) method = event.get('requestContext', {}).get('http', {}).get('method', 'GET') # Function URL POST, or direct invoke (e.g. Resource Browser) with a message if method == 'POST' or 'message' in event: data = json.loads(event.get('body', '{}')) if method == 'POST' else event item = {'id': str(uuid.uuid4()), **data} table.put_item(Item=item) return {'statusCode': 200, 'body': json.dumps(item)} result = table.scan() return {'statusCode': 200, 'body': json.dumps(result['Items'])} EOF } } resource "aws_iam_role" "lambda_role" { name = "lambda-role" assume_role_policy = jsonencode({ Version = "2012-10-17" Statement = [{ Action = "sts:AssumeRole", Effect = "Allow", Principal = { Service = "lambda.amazonaws.com" } }] }) } resource "aws_lambda_function" "messages_api" { function_name = "messages-api" runtime = "python3.12" handler = "handler.handler" filename = data.archive_file.lambda.output_path source_code_hash = data.archive_file.lambda.output_base64sha256 role = aws_iam_role.lambda_role.arn environment { variables = { TABLE_NAME = aws_dynamodb_table.messages.name } } } resource "aws_lambda_function_url" "messages_api" { function_name = aws_lambda_function.messages_api.function_name authorization_type = "NONE" } output "function_url" { value = aws_lambda_function_url.messages_api.function_url } TF cd /tmp/localstack-demo ``` 2. Initialize and apply the configuration: ```bash lstk terraform init && lstk terraform apply -auto-approve ``` 3. Retrieve the endpoint: ```bash LAMBDA_URL=$(lstk terraform output -raw function_url) echo $LAMBDA_URL ``` ## Step 3: Test the API Send a POST request to store a message in the locally emulated DynamoDB table: ```bash curl -X POST "$LAMBDA_URL" \ -H "Content-Type: application/json" \ -d '{"message": "Hello, LocalStack!"}' ``` You will get back a response: ```json title="Output" { "id": "3e1b5cae-4386-447b-8567-f0615fdb0fff", "message": "Hello, LocalStack!" } ``` Retrieve all your messages: ```bash curl "$LAMBDA_URL" ``` The Lambda function executes within the local environment and interacts with the locally emulated DynamoDB service. Because no actual cloud resources are created, you won't incur any real AWS cloud costs or infrastructure changes. ## Step 4: Inspect resources View the state of your local infrastructure via the [LocalStack Web Application](https://app.localstack.cloud/). Navigate to the [Stack Overview](https://app.localstack.cloud/inst/default/overview) to inspect your running resources, which are the Lambda function and DynamoDB table you just deployed. You can expand each service to see the details of the deployed resources. ![Inspect Resources using LocalStack Web Application](/images/aws/stack-overview.jpg) ## Step 5 (Optional): `Application Inspection & Tracing` Quickstart There's a lot more you can do with LocalStack than just emulate AWS services. To learn more about how LocalStack can help you inspect, manage, snapshot, and debug your AWS project, check out the [**Application Inspection & Tracing** quickstart](/aws/quickstart-library/application-inspection-tracing/) to continue this tutorial (Make sure to **skip Step 6**, the cleanup step, below). ## Step 6: Clean up Stop your LocalStack container to remove all emulated resources. LocalStack is ephemeral by default; stopping the instance clears the state. {/* prettier-ignore-start */} ```bash lstk stop ``` {/* prettier-ignore-end */} To persist resource state, like S3 buckets or DynamoDB tables, across restarts, check out our [state management tools](/aws/developer-tools/snapshots/). Remove the local files you created in this guide: ```bash rm -rf /tmp/localstack-demo ``` ## Next steps You have successfully deployed and tested a serverless API on your local workstation. Proceed to the [CI/CD guide](/aws/getting-started/ci-cd/) to learn how to integrate LocalStack into your automated continuous integration (CI) pipelines across a wide range of providers and platforms. # Overview > How to get Help and Support for LocalStack for AWS. LocalStack for AWS provides multiple support options to help you troubleshoot issues, understand features, and integrate the platform into your workflows. The level of support available depends on your subscription plan. Our support team can assist with: - Troubleshooting LocalStack-specific issues - Understanding LocalStack features and functionality - Integration guidance for LocalStack in your application - Best practices for working with LocalStack services For non-technical inquiries, such as billing or account-related questions, contact us at [support@localstack.cloud](mailto:support@localstack.cloud) or via the [LocalStack Web Application](https://app.localstack.cloud/) chat. ## Support options LocalStack offers different support plans with varying levels of access, response times, and communication channels. Use the following sections to find the support option that best fits your needs: - [Get Help](/aws/help-support/get-help/): Learn which support channel to use based on your situation. - [Support Offerings](/aws/help-support/support-offerings/): Compare available plans and included features. - [Enterprise Support](/aws/help-support/enterprise-support/): Explore dedicated support options for enterprise customers. :::note Support is currently provided in `English` only. As an international team, this ensures clear and consistent communication across all regions. ::: # Enterprise Support > How to request Enterprise Support for LocalStack for AWS. ## Introduction Enterprise support offers organizations personalized resources, direct communication channels with the LocalStack team, and flexible service level agreements (SLAs) to meet specific business requirements. The key components of our enterprise support offering include: - **Direct Slack Connect or Teams Channel**: A dedicated Slack Connect or Teams channel is available to maintain a direct communication link with the LocalStack engineering team. This setup ensures quick issue resolution and streamlined collaboration, improving overall service efficiency. - **Dedicated Customer Success Manager (CSM) and Technical Account Manager (TAM)**: Enterprise customers are assigned a CSM and SA. The CSM acts as a strategic advisor to help fully utilize LocalStack's offerings, while the SA provides expert technical assistance in designing and optimizing solutions tailored to your needs. - **Custom Service Level Agreements (SLAs)**: Tailor your service levels and response times to meet your organization's requirements. Custom SLAs can be negotiated to align with your business objectives and ensure optimal system performance. - **Support Ticketing Portal**: Access the [support ticketing portal](https://support.localstack.cloud/portal) to view, create, and respond to support tickets, ensuring organized tracking of all queries. - **Real-time Chat Support**: Real-time Level 1 (L1) chat support is available during support operating hours. While immediate resolutions are prioritized, complex issues may require additional time and resources for thorough handling. ## Customer portal A customer portal is a home behind a login where customers can view, open, and reply to their support tickets. Currently, the **customer portal** is only **available to Enterprise customers**. You can find the customer portal here: [https://support.localstack.cloud/portal](https://support.localstack.cloud/portal). ![Customer portal for enterprise support](/images/aws/customer-portal.png) ## Signing up for Enterprise Support If you are a member of an organization with an enterprise LocalStack subscription, you will receive an invitation to create an account and join the LocalStack Support Portal via email. Follow the instructions in the email and set up your account by clicking on the **Sign up** button. You will be asked to create a password. Once you do so, you will be able to log in and start using the customer portal to create, view, and engage with tickets. ## Creating a Support Ticket You can open a new ticket with LocalStack support by going to the **Create a Support Ticket** link. You will be redirected to a form where you will have to provide certain information to file a new support ticket. ![Filing a support ticket](/images/aws/file-a-support-ticket.png) The form consists of two parts. One is basic information, which is mandatory to fill out, and additional information, which adds more context to your issue but is not mandatory. Once all the mandatory fields are filled out, you can create a new support ticket by clicking on the Submit button. When the ticket is submitted, it's reported to LocalStack support, who will get back to you on that query as soon as possible. A ticket will show up in the ticket list as soon as it’s submitted. ### Basic Information You need to fill out the following fields, which are mandatory to open a new ticket: - **Type**: Choose the type of your query from the following options: - **Issue**: Select this when you are facing an issue using LocalStack. - **General inquiry**: Select this when you have a general question regarding LocalStack. - **Feature request**: Select this when you are looking for a feature that is not yet implemented in LocalStack. - **Ticket name**: Provide a descriptive name for the ticket that summarizes your inquiry. - **Description**: Provide a comprehensive description of your inquiry, explaining all the details that will help us understand your query. ### Additional Information - **CI Issue?** If the query is related to a CI issue, select the one that best fits your query from the dropdown. - **Operating system**: From the dropdown, select the operating system you are using. - **Affected Services**: From the dropdown, select the AWS service that is affected in your query. - **File upload**: Here you can provide any additional files that you believe would be helpful for LocalStack support (e.g., screenshots, log files, etc.). # Get Help > Choose the right support channel for your LocalStack issue or question. If you need help with LocalStack for AWS, choosing the right support channel can help you get a faster and more effective response. This guide explains when to use each available support option. ## Choose the right support channel ### Community Slack Use the [LocalStack Slack Community](https://localstack.cloud/slack) for: - Quick questions - General guidance - Discussions with other users and maintainers Best for: - Early-stage troubleshooting - Learning from others’ experiences :::note Community support is provided on a best-effort basis and is not guaranteed. ::: ### Support email Contact LocalStack Support via email at [support@localstack.cloud](mailto:support@localstack.cloud). ### Web application chat To create a support request using the [LocalStack Web Application](https://app.localstack.cloud/) chat: 1. Open the LocalStack Web Application 2. Click the chat icon in the bottom right corner 3. Select **Technical Issue** to request technical support or **Account/Billing Issue** to request account related support. 4. Enter your details and submit. ### Enterprise support channels Enterprise customers have access to additional support options, including: - Dedicated Slack or Teams channel - Support ticketing portal - Real-time chat support For more details, see [Enterprise Support](/aws/help-support/enterprise-support/). ## What to include To help us troubleshoot your issue efficiently, please include the following information: - **Logs** aws emulator container logs with the environment variables `SF_LOG=trace` and `DEBUG=1` enabled - **Query (if applicable)** The query that triggered the issue - **Client details** Client tool or driver used - **Connection parameters** Excluding sensitive information - **Additional logs (if available)** Client tool or driver logs Providing detailed information upfront helps reduce back-and-forth and speeds up resolution time. :::note In many scenarios, we ask our customers to use the diagnostics endpoint to provide additional information. To use LocalStack’s diagnostics endpoint: - Set the environment variable `LS_LOG=trace` - Start LocalStack - Run the affected task(s) - Call the diagnostic endpoint (the endpoint URL depends on your configuration): ```bash title="Collect diagnostics" curl -s localhost.localstack.cloud:4566/_localstack/diagnose > diagnose.json && zip diagnose.zip diagnose.json && rm diagnose.json ``` - Once you have the `diagnose.zip` file, please send it to our support team via our email at [support@localstack.cloud](mailto:support@localstack.cloud), or via your existing support ticket. ::: :::danger Ensure that you avoid sending the diagnostic output to public channels or forums, as it may contain sensitive information. ::: ## Before you reach out Before contacting support, we recommend: - Reviewing the documentation and FAQs - Verifying your configuration settings - Checking logs for errors or warnings - Ensuring your setup meets system requirements Providing clear and complete information helps us respond more quickly and effectively. # Support Offerings > Compare LocalStack support plans, features, and response expectations. LocalStack offers multiple support plans with different levels of access, response times, and communication channels. Choose the plan that best fits your needs based on the level of support and responsiveness required. ## Plans overview | **Plan** | **Tier** | | --- | --- | | Hobby (Free) | Basic Support | | Trial | Standard Support | | Base | Standard Support | | Ultimate | Priority Support | | Enterprise | Enterprise Support | | Student | Basic Support | ## Legacy Plans Support Coverage | **Plan** | **Tier** | | --- | --- | | Starter | Standard Support | | Teams | Priority Support | ## Feature comparison | **Features** | **Basic** | **Standard** | **Priority** | **Enterprise** | | --- | --- | --- | --- | --- | | Documentation access | ✅ | ✅ | ✅ | ✅ | | Community support | ✅ | ✅ | ✅ | ✅ | | Operational support | ✅ | ✅ | ✅ | ✅ | | 1:1 technical support | | ✅ | ✅ | ✅ | | Screen sharing sessions | | | ✅ | ✅ | | Third-party tools support | | | Limited | Limited | | Faster response times | | | ✅ | ✅ | | Real-time chat support | | | | ✅ | | Support ticketing portal | | | | ✅ | | Service Level Agreements (SLAs) | | | | ✅ | | Direct Slack/Teams channel | | | | ✅ | | Dedicated CSM & TAM | | | | ✅ | ## Response expectations ### Standard support - Best-effort support - No guaranteed response times - Typical response time: **24–48 hours** during business hours ### Priority support - **First response:** within 24 hours - **Follow-up responses:** within 24 hours - Responses provided during business hours ### Enterprise support Enterprise support includes custom SLAs and dedicated communication channels. For full details, see [Enterprise Support](/aws/help-support/enterprise-support/). ## Support scope LocalStack support focuses on helping you use and integrate LocalStack effectively. ### Included - Troubleshooting LocalStack-specific issues - Guidance on features and functionality - Integration support for LocalStack workflows - Best practices for using LocalStack services ### Limitations Support does **not** include: - **Customer-specific code** Debugging or modifying custom applications, scripts, or workflows - **Third-party tools (Standard plan)** External tools, plugins, or development environments - **Advanced third-party troubleshooting (Priority plan)** Only basic integration guidance is provided for officially supported tools - **AWS in production** Support is limited to LocalStack’s emulated services ## Support channels by plan | **Channel** | **Basic** | **Standard** | **Priority** | **Enterprise** | | --- | --- | --- | --- | --- | | Slack community | ✅ | ✅ | ✅ | ✅ | | GitHub Discussions | ✅ | ✅ | ✅ | ✅ | | Support email | | ✅ | ✅ | ✅ | | Web application chat | | ✅ | ✅ | ✅ | | Support ticketing portal | | | | ✅ | | Dedicated Slack/Teams channel | | | | ✅ | ## Support business hours Support is available: - Monday to Friday - 6:00 AM – 9:00 PM UTC Excludes: - January 1 - May 1 - November 1 - December 24, 25, and 31 # Third Party Software Tools > This page documents the third-party software tools that we use in our software development. We build on a number of third-party software tools, including the following: Third-Party software | License --------------------------|----------------------- **Python/pip modules:** | airspeed | BSD License amazon_kclpy | Amazon Software License boto3 | Apache License 2.0 coverage | Apache License 2.0 docopt | MIT License flask | BSD License flask_swagger | MIT License jsonpath-rw | Apache License 2.0 moto | Apache License 2.0 requests | Apache License 2.0 subprocess32 | PSF License **Other tools:** | Elasticsearch | Apache License 2.0 kinesis-mock | MIT License # LocalStack Plans > Service availability and licensing details across LocalStack for AWS plans. import LicensingCoverage from "../../../components/licensing-coverage/LicensingCoverage"; import LegacyLicensingCoverage from "../../../components/licensing-coverage/LegacyLicensingCoverage"; ## Introduction This document outlines the features, emulated AWS services, and enhancements included in each LocalStack for AWS plan. It also clarifies how licensing works across workspaces and users. As of **March 23rd, 2026**, LocalStack for AWS offers the following subscriptions that provide licenses for commercial use:: - Base - Ultimate - Enterprise Customers looking to purchase LocalStack for use primarily in automated environments or through shared infrastructure, such as an internal developer platform are encouraged to reach out to our sales team. For more information, please refer to our fair use policy. We provide the following subscription for non-commercial use: - Hobby We offer special subscriptions for select segments: - Student, requires a verified GitHub Education student account - OSS project sponsorship, requires approval from LocalStack. [Applications can be submitted here]( https://www.localstack.cloud/localstack-open-source). If you purchased a LocalStack license **before May 8, 2025**, [click here to learn about your available features and legacy entitlements](#legacy-plans). ### Licensing & Access Rules Each **workspace** can only be assigned a single pricing plan. You cannot mix and match (e.g., Base and Ultimate) within the same workspace. Licenses must be assigned to individual users. This generates an authentication token that enables access to the emulator and any enhancements included in the plan. Not sure which plan fits your use case? Explore our [pricing page](https://www.localstack.cloud/pricing). For unique licensing needs across teams or environments, please contact Sales. ### Usage Allocation Per Workspace All paid plans include a fixed allocation of: - Cloud Sandbox (Ephemeral Instance) minutes (monthly pool) - State Management (Cloud Pod) storage (per contract, shared across all users) ### Service Coverage Clarification The table below shows which AWS services are available in each pricing plan. It does not indicate the level of API coverage or feature availability. To learn more about how a service behaves in LocalStack, refer to that individual service page or contact Support. ## Legacy Plans As of **May 8, 2025**, the following plans are no longer available for new purchases. If you’re an existing customer on one of these plans, your subscription remains active and unchanged. You’ll continue receiving all regular version updates and will not experience any downgrade or loss of access. If you have questions or concerns, please contact Support. ### Subscription Continuity You may continue purchasing new licenses under your current legacy plan for the duration of your active subscription. However, if your subscription lapses, we may not be able to restore access to these legacy plans. # Overview > Manage your LocalStack accounts, workspaces, users, licenses, and single sign-on through the LocalStack Web Application. import SectionCards from '../../../../components/SectionCards.astro'; Organizations & Admin covers the administrative side of LocalStack: managing your account, organizing your workspace, and provisioning users, licenses, and permissions through the [LocalStack Web Application](https://app.localstack.cloud/). These are generally one-time setup tasks performed by an IT administrator (or by an individual setting up their own account), rather than day-to-day development work. # Accounts > A LocalStack account is required to access features in the Web Application, and to access any of our offerings. ## Introduction A user account on the LocalStack Web Application is required to access the following features: - Advanced AWS services - Resource Browsers - Cloud Pods - Extensions Library - Stack Insights - Ephemeral Instances - IAM Policy Stream - Chaos Engineering To create an Auth Token for your LocalStack account, you need to sign up for an account on the LocalStack Web Application. This token is used to authenticate your requests to the LocalStack platform and access the features mentioned above. ## Creating an Account To create an account for LocalStack for AWS, please visit our [pricing page](https://www.localstack.cloud/pricing) and click on "Get Started for Free". Follow the prompts to fill out your information and remember to verify your email to continue. All new users get their first month free (a 30-day trial of our Ultimate tier) with no commitment. At the end of your free trial period, you may select your preferred plan. > Terms: [localstack.cloud/legal/tos](https://www.localstack.cloud/legal/tos) > Privacy Policy: [localstack.cloud/legal/privacy-policy](https://www.localstack.cloud/legal/privacy-policy) ![Sign-up screen](/images/aws/account-signup-form.png) ## Logging In Once your account is activated, log in at [**app.localstack.cloud**](https://app.localstack.cloud) using your selected sign-in method. Supported login options: - GitHub - SSO (if configured) - Email-based authentication ## Updating Account Settings To update your profile or change account settings: 1. Click your name or organization's name in the top-left corner. 2. Select **Settings** from the dropdown. 3. Navigate to **Profile** to update your name, company, job title, phone number, or GitHub username. ![Account settings in sidebar](/images/aws/account-settings.png) Changes are saved automatically once submitted. # Users and Licenses > Invite new members and manage a member's license ## Introduction The **Users & Licenses** page in the LocalStack Web Application allows workspace administrators to manage workspace memberships and assign licenses. To access this page: 1. Click your name or organization's name in the top-left corner of the dashboard. 2. Go to **Settings** → **Users & Licenses** under the **Administration** section. ![Users & Licenses management screen](/images/aws/webapp-managing-users-licenses.png) ## Member Roles Each member of a workspace is either a **Workspace Admin** or a **Workspace Member**. The role determines which parts of the workspace they can access and manage. The table below summarizes what each role can do: | Action | Workspace Admin | Workspace Member | | ---------------------------------------------- | :-------------: | :--------------: | | Invite new members | ✅ | ❌ | | Remove members from workspace | ✅ | ❌ | | Assign or unassign licenses | ✅ | ❌ | | Change member roles (e.g., promote to Admin) | ✅ | ❌ | | Configure advanced permissions | ✅ | ❌ | | Access Auth Tokens | ✅ | ✅ | | Use assigned LocalStack licenses | ✅ | ✅ | :::note Admins manage the overall pool of licenses for the workspace. Each member is still responsible for configuring their own local environment to use their Auth Token. ::: ### Key differences - **Administrative control:** Only Admins can open the **Administration** section of Settings to manage workspace membership and the license pool. - **License management:** Admins distribute available licenses from the subscription plan to specific members through the **Users & Licenses** dashboard. - **Role management:** Admins can switch any member between **Admin** and **Member**. ## Managing Members ### Inviting Members To invite someone to your workspace: - Click **Invite Members**. - Enter the email of the person you want to invite. - Check the option to automatically assign a license (optional). - Send the invite. If the invitee does not have a LocalStack account, they will receive an email to create one. :::note Only workspace admins can invite members and manage license assignments. ::: ### Removing Members To remove a member from the workspace: - Click the **⋯** menu at the right of their row. - Select the option to remove them from the workspace. You can re-invite them anytime. ### Managing Roles and Permissions Use the **⋯** menu at the right of a member's row to view and edit their role. - Set them as **Admin** or **Member** - Configure advanced permissions if available ## Managing Licenses Licenses are part of subscription plans and are shown in the **License** column of the members list. - To **assign** or **change** a license: Click directly on the license name for a member to open the license dropdown. - To **unassign** a license: Select the no-license option from the same dropdown. - A license can be reassigned at any time. Changes apply immediately and don’t require user action. # Single-Sign On > Configuring Custom Single-Sign On (SSO) Providers in LocalStack Web Application. Custom Single-Sign On (SSO) Identity providers, can be enabled to facilitate the process of quickly onboarding team members from your organization. In order to configure SSO access, first sign in to the LocalStack Web application under [app.localstack.cloud](https://app.localstack.cloud/). In your profile settings, navigate to the Single Sign-on tab which will list existing SSO Identity Providers (if any exist). ![Adding SSO Identity providers in LocalStack Settings](/images/aws/localstack-setting-sso.png) Next, click the button to create a new identity provider (IdP), where you can choose between the two leading industry standards: - OpenID Connect (OIDC): [openid.net/connect](https://openid.net/connect/) - SAML: [saml.xml.org/saml-specifications](http://saml.xml.org/saml-specifications) ## Configuring SSO using OpenID Connect (OIDC) In the form illustrated below, you can then enter the main information for the new IdP (using OpenID Connect): - Name of your identity provider - Client ID, Client Secret, Attributes request method, OIDC issues, Authorize scopes, and more. - You should be able to find these attributes in your OIDC IdP configuration. ![Configuring SSO using OpenID Connect (OIDC)](/images/aws/oidc-sso.png) ## Configuring SSO using SAML When configuring SSO using SAML, you can configure the settings of the Identity Provider via a standard SAML metadata file (see illustration below). The SAML metadata file can be specified either via URL or via a file upload. Select **Enable IdP sign out flow** if you want your users to be logged out from our app and your SAML IdP when they log out from your our Web Application. ![Configuring SSO using SAML](/images/aws/saml-sso.png) ## Configuring SSO with Okta This section provides a reference configuration for setting up SAML-based SSO with **Okta**. The steps below mirror the fields required in the LocalStack UI and can be used as a template when configuring your Okta application. ### 1. Create a SAML 2.0 App in Okta In your Okta Admin Dashboard, create a new application under: > **Applications → Create App Integration → SAML 2.0** During setup, Okta will ask for: * **Single sign-on URL** * **Audience URI (SP Entity ID)** You can copy these values directly from your LocalStack SSO provider creation screen. Example mapping: | LocalStack name | Okta field name | | ---------------------- | --------------------------- | | Callback URL | Single sign-on URL | | Identifier (Entity Id) | Audience URI (SP Entity ID) | ### 2. Configure SAML Attribute Statements LocalStack supports mapping the following user attributes: * **email** * **firstName** * **lastName** In Okta, add these under **Attribute Statements (optional)**: | Name | Name format | Value | | --------- | ----------- | ---------------- | | email | Unspecified | `user.email` | | firstName | Unspecified | `user.firstName` | | lastName | Unspecified | `user.lastName` | > **Note:** In some setups, Okta may not always populate `firstName` or `lastName` during signup. This is usually a configuration mismatch on the IdP side. Users can still manually enter these fields during signup if needed. ![Configuring SSO using Okta with SAML Attribute Statements](/images/aws/sso-okta-attribute-statements.png) ![Configuring SSO using Okta with SAML Attribute Statements](/images/aws/sso-okta-attribute-statements-2.png) ### 3. Retrieve the Okta Metadata URL Once the application is created, navigate to: > **Applications → Sign On → SAML 2.0 → Metadata URL** Copy this URL. ![Retrieve Okta Metadata URL](/images/aws/retrieve-okta-metadata-url.png) This URL should be used in the LocalStack UI under: > **Metadata File → URL** LocalStack will automatically import the SAML metadata and map the endpoints required for SSO. ### 4. Configure LocalStack Identity Provider In the LocalStack SSO configuration screen: * Select **Provider type: SAML** * Enter an **Identity provider name** (e.g., “Okta”) * Paste the **Metadata URL** from Okta * Fill in attribute mappings: | Your attributes (from Okta) | LocalStack attributes | | --------------------------- | --------------------- | | email | Email | | firstName | First Name | | lastName | Last Name | Once completed, LocalStack will display: * **Callback URL** * **Identifier (Entity Id)** * **Sign Up Portal URL** These values are used in the Okta app configuration and for distributing the signup link to end-users. ![Place Okta Metadata URL in LocalStack UI](/images/aws/import-metadata-file.png) ### 5. Assign Users to the Okta Application Ensure that the correct users and groups have access to the Okta SAML app. Only assigned users will be able to authenticate into LocalStack via SSO. ## SSO for JumpCloud This example outlines the required configuration when using **JumpCloud** as a SAML Identity Provider for LocalStack. ### 1. Create a Custom SAML Application In the JumpCloud Admin Portal: 1. Go to **SSO Applications → Add New Application** 2. Select **Custom Application** 3. Open **Manage Single Sign-On (SSO)** and choose **Configure SSO with SAML** ![JumpCloud Admin Portal Custom Application](/images/aws/jumpcloud-step1.jpg) ### 2. Map Required Fields Copy the fields from the LocalStack SSO configuration screen into the corresponding JumpCloud fields. | JumpCloud field | LocalStack value | | ----------------- | ---------------------- | | **IdP Entity ID** | Identity provider name | | **SP Entity ID** | Identifier (Entity Id) | | **ACS URLs** | Callback URL | | **Login URL** | Sign Up Portal | ![JumpCloud Map Required Fields](/images/aws/jumpcloud-step2.png) ### 3. Attribute Mapping Add the following user attributes: | Service Provider Attribute | JumpCloud Attribute | | -------------------------- | ------------------- | | email | email | | firstname | firstname | | lastname | lastname | ### 4. Required Options Ensure the following options are enabled: * **Declare Redirect Endpoint** * **Include Group Attribute** with the name: ``` memberOf ``` ![JumpCloud Map Required Fields](/images/aws/jumpcloud-step4.png) ### 5. Assign Users Save the application and assign users or groups who should access LocalStack via SSO. ## SSO for Google Workspace This example outlines the required configuration when using **Google Workspace** as a SAML Identity Provider for LocalStack. ### 1. Create a custom SAML app In the Google Workspace Admin Console, navigate to **Apps → Web and mobile apps** in the left side menu. ![Navigate to Web and mobile apps in Google Workspace Admin Console](/images/aws/google-sso/google-sso-1.png) Select **Add app**, then **Add custom SAML app**. ![Add a custom SAML app in Google Workspace](/images/aws/google-sso/google-sso-2.png) Fill out a name for your custom app (e.g., "LocalStack"), then continue to the next page and download the IdP metadata file. You'll upload this to LocalStack in a later step. ![Download the IdP metadata file from Google Workspace](/images/aws/google-sso/google-sso-3.png) ### 2. Configure service provider details Navigate to our web application, or follow this [link](https://app.localstack.cloud/workspace/sso), and create a new Identity provider to retrieve your **Callback URL** and **Identifier (Entity Id)**. ![Callback URL and Identifier (Entity Id) in the LocalStack Web Application](/images/aws/google-sso/google-sso-4.png) Back in the Google SAML app wizard, on the **Service provider details** step: * Paste the Callback URL into **ACS URL** * Paste the Identifier (Entity Id) into **Entity ID** * Set **Name ID format** to `EMAIL` * Set **Name ID** to `Basic Information > Primary email` ![Google Workspace Service provider details, including ACS URL, Entity ID, and Name ID](/images/aws/google-sso/google-sso-5.png) Example mapping: | LocalStack name | Google field name | | ----------------------- | ------------------ | | Callback URL | ACS URL | | Identifier (Entity Id) | Entity ID | ### 3. Configure SAML attribute mapping On the **Attribute mapping** step, map the following Google Directory attributes to service provider attributes: | Google Directory attribute | App attribute | | ---------------------------------- | -------------- | | Basic Information > First name | `firstName` | | Basic Information > Last name | `lastName` | | Basic Information > Primary email | `email` | ![Google Workspace Attribute mapping](/images/aws/google-sso/google-sso-6.png) ### 4. Configure LocalStack Identity Provider In the LocalStack SSO configuration screen: * Select **Provider type: SAML** * Enter an **Identity provider name** (e.g., "Google-Workspace") * Upload the metadata file you downloaded from Google in step 1 ![Uploading the Google Workspace metadata file in the LocalStack Web Application](/images/aws/google-sso/google-sso-7.png) Then fill in the attribute mappings: | Your attributes (from Google) | LocalStack attributes | | ------------------------------ | ----------------------- | | email | Email | | firstName | First Name | | lastName | Last Name | ![Attribute mapping in the LocalStack Web Application](/images/aws/google-sso/google-sso-8.png) ### 5. Assign Users to the Google Workspace Application Ensure the correct users and groups have access to the custom SAML app in Google Workspace. Only assigned users will be able to authenticate into LocalStack via SSO. ## Attribute mapping These attributes can be defined to automatically map attributes of user entities in your internal IdP to user attributes in the LocalStack platform. The following user attribute mappings can currently be configured: - Email - First name - Last name The Email should be configured to ensure correct functionality. ![Attribute Mapping](/images/aws/attribute-mapping.png) ## Callback URL, Sign Up Portal URL and Identifier (Entity Id) After configuring the base details for your Identity Provider (IdP), the following additional information can be copied from the UI: - **Callback URL**: The Callback URL that you may need to configure in the settings of your IdP. - **Identifier (Entity Id)**: The Identifier (Entity Id) that you may need to configure in the settings of your IdP. - **Sign Up Portal URL**: This is the URL that can be shared with your users to start the SSO signup flow for the LocalStack Web Application. The format of this endpoint is `https://app.localstack.cloud/auth/sso//` ![Callback URL, Sign Up Portal URL, and Identifier (Entity Id)](/images/aws/additional-information-page.png) ## Strict SSO Mode Strict SSO Mode is an optional security enhancement that requires all members of your organization to authenticate exclusively through the configured Identity Provider (IdP). Once enabled, standard username/password login is disabled for your organization and the configured IdP becomes the only permitted way to sign in. This provides two key security benefits: - **Leaked credential protection**: Even if a user's LocalStack password is compromised, attackers cannot log in without going through your IdP. - **Revocation enforcement**: When an employee's account is removed or suspended in your IdP, they immediately lose access to LocalStack. ### Enabling Strict SSO Mode To enable strict mode, open the identity provider configuration in your LocalStack Web Application profile settings under **Single Sign-on**, and toggle the **Enable Strict SSO Mode** checkbox in the identity provider settings. :::caution Before enabling strict mode, ensure all team members have linked their accounts to the configured Identity Provider. Once strict mode is active, any user who has not completed SSO setup will be unable to sign in via password. ::: ## User Roles and Permissions For each new member that joins your org, you can specify user roles and permissions that should be assigned to them. - **Default User Role**: The Role that should be assigned to users of your organization signing up via SSO. In most cases, this should be a Member. - **Default User Permissions**: Use this to define which permissions should be assigned to users of your organization signing up via SSO. - Tip: In order to enable self-serve licences (i.e., allowing your users to allocate themselves their own license), make sure to select the **Allow member to issue a license for themselves** permission. ![User Roles and Permissions](/images/aws/roles-permissions.png) # SSO for Azure AD > Configuring Azure AD for Single Sign-on in LocalStack Enterprise To configure SSO with an Azure AD Enterprise application, we provide a simple step-by-step solution below: 1. Navigate to "Set up single sign on" in your Azure AD Enterprise application. ![Azure AD First Configuration Step](/images/aws/azure-step-1.png) 2. In the Basic SAML Configuration, ensure that the settings match the following details ![Azure AD Second Configuration Step](/images/aws/azure-step-2.png) Take the correct values for Identifier (Entity ID) and Reply URL from the Identity Provider configuration page. 3. In the Attributes & Claims section, add a group claim with the following configuration and save it. ![Azure AD Third Configuration Step](/images/aws/azure-step-3.png) 4. In the SAML Certificates section, copy the App Federation Metadata Url ![Azure AD Fourth Configuration Step](/images/aws/azure-step-4.png) 5. Navigate to our web application, or follow this [link](https://app.localstack.cloud/workspace/sso), and: * Create a new Identity provider * Enter a name for you Identity provider, and choose SAML as the provider type. * Select URL for the Metadata file and paste the link that you copied previously in step 4. * For the attribute mapping, provide the following value for the Email attribute: `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name` - (This should match the Claim name of user.userprincipalname in your Attributes & Claims) * Leave First name attribute and Last name attribute blank. 6. Let your team members sign up to your LocalStack Organization via the Sign Up Portal Link. # SCIM > Automating user provisioning, role assignment, and license assignment in LocalStack using SCIM (System for Cross-domain Identity Management). SCIM (System for Cross-domain Identity Management) allows you to automate user provisioning, deprovisioning, role assignment, and license assignment in LocalStack through your identity provider (IdP). LocalStack's SCIM implementation follows the SCIM v2.0 specification and has been developed and tested with both the **Okta** and **Microsoft Entra ID** SCIM clients. SCIM is a sub-feature of SSO and requires an active SSO configuration with at least one Identity Provider already set up. See the [Single Sign-On](/aws/organizations-admin/sso/) documentation before proceeding. All integration details - including the SCIM Base Connector URL, Bearer Auth Token, and group names per subscription - are available in the LocalStack web app under Settings → Single Sign-On. For IdP-specific setup instructions, see: - [SCIM with Okta](/aws/organizations-admin/sso/scim/okta/) - [SCIM with Microsoft Entra ID](/aws/organizations-admin/sso/scim/entra/) ## Prerequisites - An active Enterprise subscription with the SCIM feature enabled - A configured SSO Identity Provider (OIDC or SAML) - Admin access to your organization in the LocalStack web app ## Enabling SCIM In the LocalStack web app, navigate to **Settings → Single Sign-On**. For each configured Identity Provider, you will see a **SCIM User Provisioning** toggle. Enable it for the IdP you want to use for SCIM provisioning. :::note Only one Identity Provider can have SCIM active at a time. ::: Once enabled, click **View SCIM Configuration** to access the SCIM Base Connector URL and Bearer Auth Token needed to configure your IdP. ## Setup and Configuration The settings contain the **SCIM API Base Connector URL** and the **Bearer Auth Token** as shown in the image below. You can copy these values to configure your SCIM client. ![SCIM connection details](/images/aws/SCIM-configuration.png) SCIM clients authenticate using a long-lived bearer token. The token starts with `scim-` and is displayed (masked) in the SCIM configuration panel. Use the copy icon to copy it to your clipboard. You can regenerate the token at any time using the refresh icon. Regenerating the token immediately invalidates the previous one - update your IdP configuration with the new token to avoid interruptions. Once you have the Base Connector URL and Bearer Token, continue with the IdP-specific setup: - [SCIM with Okta](/aws/organizations-admin/sso/scim/okta/) - [SCIM with Microsoft Entra ID](/aws/organizations-admin/sso/scim/entra/) ## Web App Roles and Permissions There are two ways roles and permissions are applied to SCIM-provisioned users: - **Default presets at provisioning time** - LocalStack lets you configure a default role and permissions that are applied when a user is first provisioned via SCIM (for example, to grant CI credentials by default). These presets are inherited from the SSO settings and apply to every newly provisioned user. ![SCIM user role and permission settings](/images/aws/SCIM-permissions.png) - **Role assignment via SCIM role groups** - workspace roles (**admin** / **member**) can be assigned and changed directly from your IdP by syncing role groups. See **Role Management** for [Okta](/aws/organizations-admin/sso/scim/okta/#role-management) or [Microsoft Entra ID](/aws/organizations-admin/sso/scim/entra/#role-management). - **License assignment via SCIM license groups** - licenses for a subscription can be assigned and revoked directly from your IdP by syncing license groups. See **License Management** for [Okta](/aws/organizations-admin/sso/scim/okta/#license-management) or [Microsoft Entra ID](/aws/organizations-admin/sso/scim/entra/#license-management). Granular permissions beyond the workspace role (e.g. specific CI credential grants) are not individually assignable via SCIM - they are controlled by the provisioning-time presets above or managed directly in the LocalStack web app. ## Limitations - **One license group per user:** Each user can be assigned to only one license group (subscription) per organization. - **One SCIM provider at a time:** Only one Identity Provider can have SCIM enabled at a time. - **Provisioning is one-way:** SCIM sync goes from your IdP to LocalStack only. There is no synchronization from LocalStack back to your IdP. - **LocalStack UI does not block manual edits:** The LocalStack web app does not prevent you from manually editing SCIM-provisioned users or their license assignments. It is strongly recommended to manage SCIM-provisioned users exclusively through your IdP to avoid inconsistencies. - **Re-provisioning removed users requires re-invitation:** If a user was provisioned via SCIM and later removed, they cannot be re-provisioned via SCIM directly. They must be re-invited through the LocalStack **Users & Licenses** page and accept the invitation before being reassigned. ## API Reference LocalStack's SCIM API is available at `/scim/v2` and implements the SCIM v2.0 specification (RFC 7644). ### User Endpoints (`/scim/v2/Users`) | Method | Endpoint | Description | | ------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `POST` | `/scim/v2/Users` | Create a SCIM user, or idempotently return an existing member when the email matches. Enforces global email uniqueness and `userName` uniqueness per org and IdP. | | `GET` | `/scim/v2/Users` | List active SCIM-provisioned users. Supports `filter=userName eq "..."`, `startIndex`, and `count` for pagination. | | `GET` | `/scim/v2/Users/{id}` | Retrieve a SCIM user only if they are SCIM-provisioned and active in the org; returns `404` otherwise. | | `PATCH` | `/scim/v2/Users/{id}` | RFC 7644 PatchOp for selected fields (`name`, `emails`) and deactivation via `active:false`. Reactivation via SCIM is not supported. Patching `userName` or `externalId` is not supported. | | `PUT` | `/scim/v2/Users/{id}` | Full replace of mutable fields (name, email) with support for deactivation via `active:false`. Reactivation via SCIM is ignored. | ### Group Endpoints (`/scim/v2/Groups`) | Method | Endpoint | Description | | -------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `POST` | `/scim/v2/Groups` | Bind an existing subscription as a SCIM group via `displayName` (format: `{PLAN}-{EMULATOR}-{subscription_id}`). Optionally assign members. Validates membership and enforces one-group-per-user per org. Returns `201` on success; `409` for insufficient seats or conflicts. | | `GET` | `/scim/v2/Groups` | List groups (subscriptions) with their SCIM members. Supports `filter=displayName eq "..."`, `startIndex`, and `count` (max 1000). | | `GET` | `/scim/v2/Groups/{id}` | Retrieve a group by its subscription ID with SCIM members. Returns `404` if not found. | | `PATCH` | `/scim/v2/Groups/{id}` | RFC 7644 PatchOp (`add`, `remove`, `replace`) for members. Supports capacity checks and rollback on partial failures. | | `PUT` | `/scim/v2/Groups/{id}` | Full replace of group membership. Omitting or passing an empty `members` array clears all members. Supports rollback on errors. | | `DELETE` | `/scim/v2/Groups/{id}` | Delete the group binding and unassign SCIM members. Non-SCIM assignments are unaffected. Returns `204` on success. | ### Metadata Endpoints | Method | Endpoint | Description | | ------ | -------------------------------- | ----------------------------------------------------------------- | | `GET` | `/scim/v2/ResourceTypes` | List all supported SCIM resource types (`User`, `Group`). | | `GET` | `/scim/v2/Schemas` | List supported SCIM schemas for user and group resources. | | `GET` | `/scim/v2/ServiceProviderConfig` | Return service provider configuration and supported capabilities. | # SCIM with Entra ID > Configuring Microsoft Entra ID as the SCIM client for LocalStack user and license provisioning. This page covers configuring **Microsoft Entra ID** as your SCIM client to provision users, groups, and licenses into LocalStack. Before starting, make sure you've completed the steps in the [SCIM overview](/aws/organizations-admin/sso/scim/) to enable SCIM and obtain the **SCIM Base Connector URL** and **Bearer Auth Token** from the LocalStack web app. ## Configuring SCIM with Microsoft Entra ID Use the following steps to configure SCIM provisioning from a Microsoft Entra ID Enterprise Application. 1. **Select or create your Enterprise Application** - In the Microsoft Entra admin center, go to **Identity → Applications → Enterprise applications** and select the application you want to enable SCIM provisioning for. If you don't have one yet, create a new non-gallery application. 2. **Navigate to Provisioning** - In the application's side menu, open **Manage → Provisioning**. On first setup, click **Get started** and set the **Provisioning Mode** to **Automatic**. 3. **Enter the SCIM connection details** under the **Connectivity** section (or **Admin Credentials** in the legacy view): - **Authentication method:** Select **Bearer authentication**. - **Tenant URL:** Paste the SCIM Base Connector URL from the LocalStack SCIM configuration panel. - **Secret Token:** Paste the SCIM bearer token from the LocalStack SCIM configuration panel. ![Entra ID SCIM connectivity configuration](/images/aws/SCIM_entra_connectivity.png) 4. **Test the connection and save** - Click **Test connection** to confirm Entra can reach LocalStack, then save the settings. 5. **(Recommended) Set scope** - Under **Provisioning → Settings → Scope**, select **Sync only assigned users and groups** to limit provisioning to users and groups you explicitly assign to the application. 6. **Start provisioning** - Return to the Provisioning overview and click **Start provisioning**. Entra will sync user and group changes to LocalStack every ~40 minutes; for an immediate sync of a specific user, use **Provision on Demand** from the Provisioning blade. :::caution Do **NOT** enable the `aadOptscim062020` feature flag on the Entra provisioning configuration. This flag changes Entra's outbound `PATCH /Groups` semantics in a way that can cause destructive single-user member replacements. The default behavior (flag off) is what LocalStack expects. ::: ### User Management #### Provisioning Individual Users LocalStack supports full provisioning and deprovisioning of individual user accounts via SCIM. 1. **Create the user in Entra** (if not already present) - In **Microsoft Entra ID → Users**, click **+ New user → Create new user** and fill in the basic details (User principal name, Display name, etc). ![Creating a new user in Entra ID](/images/aws/SCIM_entra_create_new_user.png) 2. **Assign the user to the LocalStack application** - Open your Enterprise Application and go to **Manage → Users and groups**. Click **+ Add user/group**, search for the user, select them, and click **Select**. ![Selecting users to assign to the application](/images/aws/SCIM_entra_add_members_search.png) 3. **Wait for sync** - On the next provisioning cycle (or via **Provision on Demand**), Entra will send a SCIM request to LocalStack to create the user account. :::tip Legacy users (existing LocalStack accounts) can also be assigned to the Entra application, provided their email address matches the one they used to register with the LocalStack web app. ::: :::note For security reasons, SCIM can only provision user accounts for users who do not already exist in the LocalStack web app. If a user was originally created via SCIM and later removed from your workspace, you must invite them again through the LocalStack Users & Licenses. The user will receive an email invitation and must explicitly accept it to rejoin the workspace. ::: #### Updating User Accounts Changes to user attributes (first name, last name, email) in Entra are automatically pushed to LocalStack via SCIM while the integration is active. #### Deprovisioning Users 1. In Entra, open the LocalStack Enterprise Application and go to **Manage → Users and groups**. 2. Find the user you want to remove and click **Remove**. 3. Confirm the action. Entra will send a SCIM deprovisioning request and the user will be removed from LocalStack on the next sync cycle. Disabling the user in the Entra directory itself (`accountEnabled = false`) has the same effect. :::caution LocalStack will not let you deprovision the last remaining workspace admin. If the user being removed is the only admin, the request fails with `409 Cannot remove the last workspace admin`. Assign another admin in LocalStack first, then retry the deprovisioning. ::: #### Provisioning Groups of Users Groups in Microsoft Entra ID can be used to provision multiple users to LocalStack at once. To enable group provisioning, ensure the **Provision Microsoft Entra ID Groups** mapping is enabled in **Provisioning → Mappings**. 1. **Create a security group** - In **Microsoft Entra ID → Groups → All groups**, click **+ New group**. Choose **Security** as the group type, set the **Membership type** to **Assigned**, give the group a name, and (optionally) a description. ![Creating a new security group in Entra ID](/images/aws/SCIM_entra_new_group.png) 2. **Add members to the group** - In the same form (or after creation, via the group's **Members** tab), select the users you want to provision. ![Adding members to a group in Entra ID](/images/aws/SCIM_entra_create_group_member_popup.png) 3. **Assign the group to the application** - Open your Enterprise Application, go to **Manage → Users and groups**, click **+ Add user/group**, select the group, and confirm. 4. **Wait for sync** - Entra will send SCIM requests to LocalStack to provision each member on the next sync cycle. Changes to a group's membership in Entra are automatically pushed to LocalStack via SCIM on subsequent sync cycles. #### Deprovisioning Groups of Users 1. In Entra, open the LocalStack Enterprise Application and go to **Manage → Users and groups**. 2. Find the group and click **Remove**. 3. Confirm the action. Entra will send SCIM requests to remove the group's users from LocalStack. Users who were provisioned solely through this group assignment will also be deprovisioned. :::tip Any changes in Entra (user/group attribute changes, group memberships, etc.) are automatically synchronized with LocalStack on subsequent sync cycles as long as the SCIM integration is active. ::: #### Migrating an Existing Enterprise Application If you enable SCIM provisioning on an Entra Enterprise Application that was already used for SSO, the users and groups already assigned to it are brought under SCIM management automatically - there's no separate migration step. 1. Enable provisioning on the existing application by following the [Configuring SCIM with Microsoft Entra ID](#configuring-scim-with-microsoft-entra-id) steps above. 2. The users and groups already assigned to the application are provisioned to LocalStack on the next sync cycle. 3. To sync them immediately rather than waiting for the cycle, use **Provision on Demand** from the Provisioning blade for each user. ### Role Management LocalStack workspace roles (**admin** and **member**) are assigned to users by syncing SCIM groups whose name identifies the target role. The role groups themselves do not need to exist in LocalStack before the sync - they are synthetic SCIM groups keyed off the `displayName`. :::caution Each user can only be in **one** role group at a time. Attempting to add a user who is already in the admin role group to the member role group (or vice versa) returns a `409` conflict. Remove the user from the previous role group first, then add to the new one. ::: #### Group Name Convention Role groups are matched by `displayName` using a case-insensitive substring check: - Any group whose name contains `admin` → admin role group - Any group whose name contains `member` → member role group All of the following are valid names for the admin role group: - `LocalStack-Admin` - `LocalStack-Admins-Prod` - `ABC-ABC-AB1000_AuthLocalstackAdmin` The first time you sync a role group from Entra, LocalStack persists that `displayName` so subsequent GET responses to your IdP reflect the name you sent. You can also rename the group later via SCIM and LocalStack will track the rename. #### Creating a Role Group in Microsoft Entra ID 1. In **Microsoft Entra ID → Groups → All groups**, click **+ New group**. Create a **Security** group with **Membership type: Assigned** whose name contains either `Admin` (for the admin role) or `Member` (for the member role). ![Creating a role group in Entra ID](/images/aws/SCIM_entra_role_group.png) 2. Add users to the group (users must already be assigned to the LocalStack Enterprise Application). 3. Assign the group to the LocalStack Enterprise Application via **Manage → Users and groups**. 4. Confirm that **Provision Microsoft Entra ID Groups** is enabled under **Provisioning → Mappings**. 5. On the next provisioning cycle (or via **Provision on Demand**), Entra will sync the group to LocalStack and assign the corresponding role to all members. #### Moving a User Between Roles To change a user's role from member to admin (or vice versa), apply the two changes **in sequence**, letting the removal sync to LocalStack before adding the new group: 1. Remove the user from their current role group in Entra. 2. **Wait for the removal to sync to LocalStack** - either the next provisioning cycle, or use **Provision on Demand** on that user to commit it immediately. 3. Add the user to the target role group (and let it sync as in step 2). :::caution Don't change both groups at once. Entra may send the *add* and *remove* operations in any order within a single sync cycle. If the *add* reaches LocalStack before the *removal* has been committed, the user still carries their old role-group marker and the request is rejected with a `409` conflict. Committing the removal first guarantees the marker is cleared before the new role is applied. The `409` is transient - Entra retries the failed operation on the next cycle, and the move eventually converges - but sequencing the changes avoids the error and the temporary inconsistency entirely. ::: #### Last-Admin Protection LocalStack will reject any SCIM request that would leave the workspace without an admin. If you attempt to remove the only admin from the admin role group, the request fails with `409 Cannot remove the last workspace admin`. Assign another admin in LocalStack first, then retry the removal. ### License Management Licenses are assigned to users by syncing specifically named SCIM groups that correspond to your LocalStack subscriptions. :::caution Each user can only be a member of one license group (subscription) per organization. Assigning a user to multiple license groups will result in an error and provisioning will fail for that user. ::: #### Group Name Format License group names follow this format: ```text {PLAN}-{EMULATOR}-{SUBSCRIPTION_ID} ``` For example: `Enterprise Plan-AWS-sub_1RqpMYGCs0LNOzY9UszOGJkL` The exact group name for each subscription is displayed in the SCIM configuration panel in the LocalStack web app. Use the subscription dropdown to select the plan you want to manage, and the correct group name will be shown for you to copy. :::tip Legacy users can be added to a license assignment group in Entra, provided their email address matches their LocalStack registration email, they have been assigned to the LocalStack Enterprise Application, and the group name matches the correct subscription. ::: #### Creating a License Group in Microsoft Entra ID 1. In **Microsoft Entra ID → Groups → All groups**, click **+ New group**. Create a **Security** group with **Membership type: Assigned** named exactly as shown in the LocalStack SCIM configuration panel. 2. Add users to the group (users must already be assigned to the LocalStack Enterprise Application). 3. Assign the group to the LocalStack Enterprise Application via **Manage → Users and groups**. 4. Confirm that **Provision Microsoft Entra ID Groups** is enabled under **Provisioning → Mappings**. 5. On the next provisioning cycle (or via **Provision on Demand**), Entra will sync the group to LocalStack and assign the corresponding license to all members. #### Migrating Users with Existing Licenses If your organization already has users with assigned licenses and you want to manage them through SCIM: 1. Create a license group in Entra with the correct name. 2. Assign it to the LocalStack Enterprise Application via **Manage → Users and groups**. 3. Add the existing licensed users to that group. Once synced - either on the next provisioning cycle, or immediately via **Provision on Demand** - they will be managed through SCIM going forward. # SCIM with Okta > Configuring Okta as the SCIM client for LocalStack user and license provisioning. This page covers configuring **Okta** as your SCIM client to provision users, groups, and licenses into LocalStack. Before starting, make sure you've completed the steps in the [SCIM overview](/aws/organizations-admin/sso/scim/) to enable SCIM and obtain the **SCIM Base Connector URL** and **Bearer Auth Token** from the LocalStack web app. ## Configuring SCIM with Okta Use the following steps to configure SCIM Base Connector URL and Bearer Auth Token: 1. **Select your application** - Go to **Applications → Applications** and select the application you want to enable SCIM provisioning for. 2. **Navigate to Provisioning settings** - In the application settings, go to the **Provisioning** tab and click **Integration** or **Edit** (wording may vary). 3. **Enter the SCIM connection details:** - **SCIM connector base URL:** Paste the SCIM Base Connector URL from the LocalStack SCIM configuration panel. - **Authentication Mode:** Select **HTTP Header**. - **Bearer Token:** Paste the SCIM bearer token from the LocalStack SCIM configuration panel. 4. **Test the connection** - Click **Test Connector Configuration** to confirm Okta can connect successfully. 5. **Enable provisioning features** (optional) - Once the connection succeeds, enable the desired provisioning actions (Create Users, Update User Attributes, Deactivate Users) under the **To App** settings tab. There is no need to enable Sync Password, as SSO does not require a password. 6. **Save** - Save and apply the integration settings. :::note The exact menu names may vary depending on Okta's UI version and app type, but the key settings are always the SCIM Base Connector URL and Bearer Auth Token under the provisioning or integration section. ::: ### User Management #### Provisioning Individual Users LocalStack supports full provisioning and deprovisioning of individual user accounts via SCIM. 1. In the Okta Admin Console, go to your application and click the **Assignments** tab. 2. Select **Assign → Assign to People**. 3. Search for and select the users you want to provision, then click **Assign** and **Done**. 4. Okta will automatically send a SCIM request to LocalStack to create the user account. The user will be visible in LocalStack and their account details will sync from Okta. :::tip Legacy users (existing LocalStack accounts) can also be assigned to the Okta application, provided their email address matches the one they used to register with the LocalStack web app. ::: :::note For security reasons, SCIM can only provision user accounts for users who do not already exist in the LocalStack web app. If a user was originally created via SCIM and later removed from your workspace, you must invite them again through the LocalStack Users & Licenses. The user will receive an email invitation and must explicitly accept it to rejoin the workspace. ::: #### Updating User Accounts Changes to user attributes (first name, last name, email) in Okta are automatically pushed to LocalStack via SCIM while the integration is active. #### Deprovisioning Users 1. In Okta, go to your application's **Assignments** tab. 2. Find the user you want to remove and click **Remove** next to their name. 3. Confirm the action. Okta will send a SCIM deprovisioning request and the user will be removed from LocalStack. :::caution LocalStack will not let you deprovision the last remaining workspace admin. If the user being removed is the only admin, the request fails with `409 Cannot remove the last workspace admin`. Assign another admin in LocalStack first, then retry the deprovisioning. ::: #### Provisioning Groups of Users Groups in Okta can be used to provision multiple users to LocalStack at once. 1. In the Okta Admin Console, go to your application and click the **Assignments** tab. 2. Select **Assign → Assign to Groups**. 3. Search for and select the groups you want to provision, then click **Assign** and **Done**. Okta will send a SCIM request to LocalStack to create a user account for each member of the group. Changes to a group's membership in Okta are automatically pushed to LocalStack via SCIM. #### Deprovisioning Groups of Users 1. In Okta, return to your application's **Assignments** tab. 2. Find the group and click **Remove** next to its name. 3. Confirm the action. Okta will send a SCIM request to remove the group's users from LocalStack. Users who were provisioned solely through this group assignment will also be deprovisioned. :::tip Any changes in Okta (user/group attribute changes, group memberships, etc.) are automatically synchronized with LocalStack as long as the SCIM integration is active. ::: #### Migrating an Existing OpenID Connect or SAML Application If you have an existing OIDC or SAML app in Okta that already has SSO users assigned, follow these steps to add SCIM provisioning: 1. On the **General** tab of your Okta application, set **Provisioning** to **SCIM**. ![Enabling SCIM provisioning on an Okta application](/images/aws/SCIM_okta_enable_scim.png) 2. Go to the **Provisioning** tab and click **Edit** to configure the SCIM connection: - **SCIM connector base URL:** Paste the URL from LocalStack. - **Unique identifier field for users:** Enter `userName` (the Okta default). - **Supported provisioning actions:** Enable all available options. ![Configuring the SCIM connection settings in Okta](/images/aws/SCIM_okta_connection_settings.png) 3. Select **HTTP Header** as the Authentication Mode and paste the Bearer token from the LocalStack SCIM configuration panel. Click **Save**. ![Adding the Bearer token in Okta](/images/aws/SCIM_okta_bearer_token.png) 4. After a successful connection test, go to the **To App** tab, click **Edit**, and enable **Create Users**, **Update User Attributes**, and **Deactivate Users**. Save your changes. ![Enabling provisioning actions in Okta](/images/aws/SCIM_okta_provisioning_actions.png) 5. Click the **Assignments** tab. Okta will show error messages for users who were assigned before provisioning was enabled. Click **Provision User** and confirm the action to sync all existing users. If the task fails, you can retry it under **Dashboard → Tasks**. ![Provisioning existing users in Okta](/images/aws/SCIM_okta_provision_existing_users.png) 6. After syncing completes, refresh the page - the error messages should be gone and all users will be fully managed via Okta SCIM. ### Role Management LocalStack workspace roles (**admin** and **member**) are assigned to users by pushing SCIM groups whose name identifies the target role. The role groups themselves do not need to exist in LocalStack before the push - they are synthetic SCIM groups keyed off the `displayName`. :::caution Each user can only be in **one** role group at a time. Attempting to add a user who is already in the admin role group to the member role group (or vice versa) returns a `409` conflict. Remove the user from the previous role group first, then add to the new one. ::: #### Group Name Convention Role groups are matched by `displayName` using a case-insensitive substring check: - Any group whose name contains `admin` → admin role group - Any group whose name contains `member` → member role group All of the following are valid names for the admin role group: - `LocalStack-Admin` - `LocalStack-Admins-Prod` - `ABC-ABC-AB1000_AuthLocalstackAdmin` The first time you push a role group from Okta, LocalStack persists that `displayName` so subsequent GET responses to your IdP reflect the name you sent. You can also rename the group later via SCIM and LocalStack will track the rename. #### Creating and Pushing a Role Group in Okta 1. Create a new Okta group whose name contains either `Admin` (for the admin role) or `Member` (for the member role). 2. Add users to the group (users must already be assigned to the LocalStack SCIM application). 3. In your application, go to the **Push Groups** tab. 4. Push the group to LocalStack via SCIM. 5. Once synced, LocalStack will assign the corresponding role to all members of the group. ![Pushing a role group from Okta](/images/aws/SCIM_okta_role_group.png) #### Moving a User Between Roles To change a user's role from member to admin (or vice versa), apply the two changes **in sequence**, letting the removal sync to LocalStack before adding the new group: 1. Remove the user from their current role group in Okta. 2. Push the change - if it isn't synced automatically, push the old role group from the **Push Groups** tab so the removal is committed. 3. Add the user to the target role group. 4. Push the change again for the new role group (or wait for the automatic sync) to apply the new role. :::caution Don't change both groups at once. If the *add* reaches LocalStack before the *removal* has been committed, the user still carries their old role-group marker and the request is rejected with a `409` conflict. Committing the removal first - by pushing the old group from the **Push Groups** tab - guarantees the marker is cleared before the new role is applied. The `409` is transient - the move eventually converges once the removal syncs - but sequencing the changes avoids the error and the temporary inconsistency entirely. ::: #### Last-Admin Protection LocalStack will reject any SCIM request that would leave the workspace without an admin. If you attempt to remove the only admin from the admin role group, the request fails with `409 Cannot remove the last workspace admin`. Assign another admin in LocalStack first, then retry the removal. ### License Management Licenses are assigned to users by pushing specifically named SCIM groups that correspond to your LocalStack subscriptions. :::caution Each user can only be a member of one license group (subscription) per organization. Assigning a user to multiple license groups will result in an error and provisioning will fail for that user. ::: #### Group Name Format License group names follow this format: ```text {PLAN}-{EMULATOR}-{SUBSCRIPTION_ID} ``` For example: `Enterprise Plan-AWS-sub_1RqpMYGCs0LNOzY9UszOGJkL` The exact group name for each subscription is displayed in the SCIM configuration panel in the LocalStack web app. Use the subscription dropdown to select the plan you want to manage, and the correct group name will be shown for you to copy. :::tip Legacy users can be added to a license assignment group in Okta, provided their email address matches their LocalStack registration email, they have been assigned to the Okta application, and the group name matches the correct subscription. ::: #### Creating and Pushing a License Group in Okta 1. Create a new Okta group named exactly as shown in the LocalStack SCIM configuration panel. 2. Add users to the group (users must already be assigned to the LocalStack SCIM application). 3. In your application, go to the **Push Groups** tab. 4. Push the group to LocalStack via SCIM. 5. Once synced, LocalStack will recognize the group and assign the corresponding license to all members. :::danger[License revocation risk] The Okta group's membership is the source of truth for license assignments on this subscription. Any change to this group in Okta (adding users, removing users, or syncing it) will reconcile the subscription's licenses to match the group exactly. Users who are licensed on this subscription but not in the Okta group will have their licenses revoked, regardless of how the license was originally assigned (manually or via SCIM). This means: - If you sync an **empty group**, every license on this subscription will be revoked. - If you sync a **partial group** (for example, 2 users in Okta but 5 currently licensed), the 3 users not in the group will lose their licenses. If you are enabling SCIM on a subscription that already has licensed users, follow the [Migrating Users with Existing Licenses](#migrating-users-with-existing-licenses) steps below **before** any sync occurs. Once SCIM is enabled, manage license assignments exclusively through Okta. ::: #### Migrating Users with Existing Licenses If your organization already has users with assigned licenses and you want to manage them through SCIM: 1. Create a license group in Okta with the correct name. 2. Add it to the application via the **Push Groups** tab. 3. Add the existing licensed users to that group through the application. Once added, they will be automatically synced (Push Status becomes **Active**) and managed through SCIM going forward. # Stack Insights > Stack Insights enable users to report AWS API usage telemetry of LocalStack runs to their LocalStack account. ## Introduction LocalStack collects execution events to provide usage analytics and insights into development and testing. Stack Insights let users report AWS API usage telemetry to their LocalStack account. Stack Insights show which APIs are used, which clients or integrations use specific services and API operations, and which services cause the most API errors. :::note Your privacy matters to us! We only collect anonymized and sanitized data. No sensitive information about your application is ever collected or exposed. The data is only used to provide you with insights into the usage of LocalStack and to help us improve the product. ::: ## Getting started ![Stack Insights](/images/aws/stack-insights-getting-started.png) To start using this feature, log in to your [LocalStack account](https://app.localstack.cloud/) and start a [LocalStack instance on your local machine](/aws/getting-started/auth-token). The system will start making your events accessible on the [Stack Insights dashboard](https://app.localstack.cloud/stacks). Click on the Stack widget to see: - Number of API calls - Services used - Runtime duration for each instance - Timestamps for each instance - Number of successful and failed API calls Click on an individual stack for more details, such as: - Number of API calls - Service invocations - User agent (e.g., `aws-cli`, `terraform`) - Specific services called during the instance - Use the slide toggle to select a time period to view specific API calls Stack insights are collected only if the session runs for less than 24 hours. View the list of events during the entire Stack lifetime, including: - Service - Operation - Status code - Server time - User agent ## Configuration You can disable event reporting on your LocalStack client by setting the environment variable `DISABLE_EVENTS=1`. :::tip Brave blocks `localhost` requests due to security by default via shields. While some sites need access to `localhost` / `127.0.0.1` to work correctly, an easy option to allow a user to enable this is manually enabling via the site via `brave://settings/content/insecureContent`. ::: # Workspace > A Workspace is the base organizational unit in the LocalStack Web Application. ## Introduction In LocalStack, a **Workspace** is the foundational unit for organizing users, resources, and billing. It enables collaboration across teams and provides access to shared capabilities. ## Workspace Features These include: - **Cloud Pods** : Manage and share your state snapshots. - **Stack Insights**: Monitor and observe your LocalStack usage. - **Extensions**: Access the Extensions Library. You can find these under the **Workspace** section in the left sidebar. User workspace features section in sidebar ## Workspace Settings Workspace settings can be accessed by clicking your or your organization's name in the top-left corner → **Settings** → **Workspace**. Here, administrators can configure and manage: - Workspace name - Company account status - Country and Tax ID - User and license management - Authentication tokens - Subscriptions and billing These options are available under the **Administration** section. ![Administration settings for workspace](/images/aws/workspace-admin-settings.png) # Overview > LocalStack Quickstart Library. import { SectionCards } from '../../../../components/SectionCards.tsx'; Our Quickstart Library is a growing collection of quickstart guides that help you get started with LocalStack immediately. Each quickstart walks you through a specific use case or feature of LocalStack. # App Inspection & Tracing > This Quickstart uses the LocalStack Web Application to inspect, browse, snapshot, and trace a deployed serverless API. import { Steps } from '@astrojs/starlight/components'; ## Introduction This quickstart picks up where the **Getting Started** guide left off and introduces the **LocalStack Web Application**, which you can leverage to inspect, manage, snapshot, and debug your LocalStack instance. In the [Getting Started](/aws/getting-started/local-development/), you deployed a Lambda function and a DynamoDB table entirely from the command line. We will be using that exact deployment for this quickstart. Now, you're going to step away from the terminal and use the web application to visually inspect, browse, snapshot, and trace that serverless API. Specifically, you will use the [LocalStack Web Application](https://app.localstack.cloud/) to: 1. **Confirm** your local instance is running and find it in the browser. 2. **Get a visual summary** of your deployed resources with **Stack Overview**. 3. **Trigger** your Lambda function and browse the resulting DynamoDB rows with the **Resource Browser**. 4. **Snapshot** the entire application state, wipe it, and bring it back with **Cloud Pods**. 5. **Trace** the request flow between your Lambda function and DynamoDB table with **App Inspector**. ## Prerequisites - Complete the [Getting Started's 'Local Development' section](/aws/getting-started/local-development/), but **skip Step 6**. You don't want to run the cleanup step because your `messages-api` Lambda function and `Messages` DynamoDB table should still be deployed. - The LocalStack container from the Getting Started must still be running. If you closed your terminal, the container keeps running in the background. Verify by running: ```bash lstk status ``` If it reports that the AWS emulator is not running, start it again with `lstk start`. Since LocalStack is ephemeral by default, this gives you a fresh, empty instance. Redeploy the Lambda function and DynamoDB table from the Getting Started before continuing. - Sign in to the [LocalStack Web Application](https://app.localstack.cloud/). ## Step by Step ### Step 1: Find your running instance Before opening the browser, confirm LocalStack is actually up and check what it currently has deployed: ```bash lstk status ``` ```text title="Output" LocalStack AWS Emulator is running • Endpoint: localhost:4566 • Container: localstack-main • Version: 4.9.1 • Uptime: 12m 3s ~ 2 resources · 2 services Service Resource Region Account Lambda messages-api us-east-1 000000000000 DynamoDB Messages us-east-1 000000000000 ``` 1. Open the [LocalStack Web Application](https://app.localstack.cloud/) and sign in. 2. In the sidebar, under **LocalStack Instances**, click on your running instance (e.g. `localhost.localstack.cloud`). The instance should display a green **running** badge. ![LocalStack Web Application sidebar showing a running instance](/images/aws/running-instance-sidebar.jpg) **Expected result:** the instance card shows a green **running** badge. Clicking into it opens the instance dashboard with tabs for **Overview**, **Status**, **Resource Browser**, **State**, and **Extensions**. The default tab is **Resource Browser**, which displays a list of all the resources you can inspect via the web application. If you don't see a running instance, see [Troubleshooting](#troubleshooting) before continuing. ### Step 2: Inspect your app with Stack Overview Stack Overview gives you a summary of everything deployed on the instance. 1. From the instance dashboard, click the [**Overview** tab](https://app.localstack.cloud/inst/default/overview). 2. Look for the **Lambda Function** and **DynamoDB Table** items in the list. 3. Click the arrow (`>`) next to **Lambda Function** to expand it and confirm `messages-api` is listed. Do the same for **DynamoDB Table** to confirm the `Messages` table is deployed. ![Stack Overview showing deployed resource types with counts](/images/aws/stack-overview.jpg) :::note Stack Overview is a **preview** feature. It only tracks a defined set of [supported resource types](/aws/connecting/console/stack-overview/#supported-resources). Lambda functions and DynamoDB tables are both covered, but if you extend this app with a service that isn't on the list, it won't show up here even though it's running fine. ::: **Expected result:** you can confirm, at a glance, that both the Lambda function and the DynamoDB table are deployed without needing to use the CLI. ### Step 3: Trigger the Lambda and inspect data with the Resource Browsers Many supported services in LocalStack for AWS come with a resource browser that allows you to view configuration details and manage individual resources. Use it here to check the Lambda function's configuration, trigger the function, and then validate that a new row lands in DynamoDB. 1. Navigate to the [**Status** tab](https://app.localstack.cloud/inst/default/status). It displays a list of the running services at the top as well as additional services that are available to use within the LocalStack emulator. 2. Under **Running** services, click **Lambda**, then, within the **Functions** tab, click the `messages-api` function. From here you can review the function's details including the ARN, runtime, handler, and the `TABLE_NAME` environment variable pointing at `Messages`. You can also update the function code, invoke the function, and view the function logs. ![Lambda Resource Browser showing the details of a deployed Lambda function](/images/aws/lambda-resource-browser.jpg) 3. Click **Invoke** to open the invoke dialog and paste the following JSON payload into the input field: ```json { "message": "Checked out from the Resource Browser" } ``` 4. Click **Submit** to trigger the function. You will see the function invoked and the response returned. The log result of the response should look like this: ```json title="Output" { "statusCode": 200, "body": { "id": "faa4dc54-f0b7-4b6b-9e52-39702af78d73", "message": "Checked out from the Resource Browser" } } ``` With the Lambda invoked, switch to the DynamoDB side to see what it wrote: 1. Back in the [**Status** tab](https://app.localstack.cloud/inst/default/status), click on **DynamoDB** and then, from the **Tables** tab, click the `Messages` table. This will open the table details page where you can view the table's details including the table name, key schema, and the number of items in the table. 2. Click the **Items** tab to view the table's contents. You should see the message you posted _and_ the new one you just sent. ![DynamoDB Resource Browser showing the Items list](/images/aws/dynamodb-resource-browser.jpg) **Expected result:** the Items view for the DynamoDB "Messages" table lists the message you posted _and_ the new one you just sent confirming that the Lambda function is writing to the table. :::tip If the table doesn't appear, check the **region** dropdown in the top-right corner of the Resource Browser. It must match the region your resources were created in (`us-east-1` by default in the Getting Started). ::: ### Step 4: Save and restore state with Cloud Pods [Cloud Pods](/aws/developer-tools/snapshots/cloud-pods/) let you snapshot your entire LocalStack state, including resources _and_ their data, and restore it later or share it with your team. You can save a Cloud Pod and view saved pods using the web application or CLI. Try this by saving your current app, wiping it, and restoring it via the web application. 1. In your instance dashboard, click the **State** tab and then click the **Cloud** view. 2. Under **Save State to Cloud Pod**, enter `messages-api-demo` for the pod name. Leave the other options as default and click **Create New Pod**. ![Cloud Pods Save State to Cloud Pod dialog](/images/aws/export-cloud-pod-web-app.jpg) 3. Verify it in the web application by navigating to **Cloud Pods** in the sidebar (`https://app.localstack.cloud/pods`). You should see `messages-api-demo` listed with a version `1`. ![Cloud Pods Browser listing saved Cloud Pods](/images/aws/verify-cloud-pod-save.jpg) Now let's clear the instance and confirm it's actually gone and then restore it: 1. Reset the LocalStack instance: ```bash lstk reset ``` 2. Back in the [**Stack Overview** tab](https://app.localstack.cloud/inst/default/overview), confirm the `messages-api` function and `Messages` table are no longer listed. LocalStack is ephemeral by default, so restarting the container discards everything. 3. Switch back to the [**State** tab](https://app.localstack.cloud/inst/default/state) and click the **Cloud** view. Under **Load State from Cloud Pod**, select `messages-api-demo` from the dropdown. You should see details about the pod displayed. Leave the default merge strategy (to learn more about merge strategies, see the [Cloud Pods documentation](https://docs.localstack.cloud/aws/developer-tools/snapshots/cloud-pods/#state-merging)) and click **Load State from Pod**. ![Cloud Pods Load State from Cloud Pod dialog](/images/aws/load-state-from-pod.jpg) **Expected result:** Return to the [**Stack Overview** tab](https://app.localstack.cloud/inst/default/overview) again. You should see the `messages-api` and `Messages` tables are back. Opening the DynamoDB table in the resource browser and clicking the **Items** tab will show the items you saved before the restart are back as well. Cloud Pods restored the _data_ in the table, not just the table definition. :::note You can do the same save/restore flow from `lstk` instead of the web application using the `lstk snapshot save` and `lstk snapshot load` commands. For more details, see the [lstk documentation](https://docs.localstack.cloud/aws/developer-tools/running-localstack/lstk/snapshots/#snapshot). ::: ### Step 5: Trace the request flow with App Inspector [App Inspector](/aws/developer-tools/app-inspector/) records every call your application makes to LocalStack, so you can see the flow between services, inspect the exact payloads exchanged, and catch IAM or configuration issues before they become deployment surprises. 1. Navigate to [**App Inspector** tab](https://app.localstack.cloud/inst/default/appinspector/spans?region=us-east-1) for your instance. 2. App Inspector hasn't been enabled on this instance yet, so click the **Enable App Inspector Now** button to enable it. You won't see any operations listed yet. 3. Fetch a fresh function URL, then trigger the Lambda: ```bash LAMBDA_URL=$(lstk aws lambda list-function-url-configs \ --function-name messages-api \ --query 'FunctionUrlConfigs[0].FunctionUrl' \ --output text) echo $LAMBDA_URL curl -X POST "$LAMBDA_URL" \ -H "Content-Type: application/json" \ -d '{"message": "Hello, App Inspector!"}' ``` 4. Return to the App Inspector tab and you should see the operations listed. This is a simple demo, so you'll only see two operations: one to get the function URL and one to invoke the function. ![App Inspector event list showing a request flow across services](/images/aws/app-inspector-operations.jpg) **Expected result:** App Inspector correctly captures the two operations you triggered. Clicking on an operation will open the operation details panel. This will show a diagram of the request flow and the service, action, resource ARN, duration, status, and the exact request/response payload. In this case, the operations details are extremely simple and don't show any errors, but this kind of detail is invaluable for debugging and understanding the flow of your application. :::note App Inspector requires LocalStack **2026.04.0** or later. If you don't see it in the sidebar, check your version with `lstk status` and update your image. ::: ## Troubleshooting ### Instance shows as "not running" in Instance Management Run `lstk status` from your terminal to confirm the container's actual state, and check that Docker is running. Start it again with `lstk start` if needed — remember this gives you an empty instance unless you load a Cloud Pod afterwards. ### Resource Browser shows a "Network Failure" error The LocalStack container isn't running, or isn't reachable at the endpoint configured for the instance. Confirm the endpoint in Instance Management matches your running container (`https://localhost.localstack.cloud:4566` by default). Ensure that you have enabled **Private Network Access** if you are using Google Chrome (see [Troubleshooting](https://docs.localstack.cloud/aws/getting-started/faq/#why-cant-i-access-my-localstack-instance-in-the-web-application-when-using-chrome) for more details). ### Lambda function or DynamoDB table don't appear anywhere in the web app Check the region dropdown in the top-right of the Resource Browser. If it's set to a region other than the one you deployed to (`us-east-1` by default in the Getting Started), switch it. ### Resources are missing after a restart, even though you didn't mean to wipe them LocalStack does not persist state across restarts by default. Either avoid stopping the container, enable [persistence](https://docs.localstack.cloud/aws/developer-tools/snapshots/persistence/), or restore your last Cloud Pod with `lstk snapshot load pod:`. ### `lstk snapshot save` fails with an authentication or license error Cloud Pods require a valid LocalStack account. Run `lstk login`, or confirm `LOCALSTACK_AUTH_TOKEN` is set correctly, and try again. ### `lstk snapshot load` warns about a version mismatch This happens when the LocalStack version that saved the pod differs from the one running now. It's safe to proceed for this demo; in real projects, pin your LocalStack version to avoid state incompatibilities. ## Next steps You've now used all four core areas of the LocalStack Web Application to inspect, manage, snapshot, and debug a running application. To dig deeper on any of these features, you can explore: - [Instance Management](https://docs.localstack.cloud/aws/connecting/console/instance-management/): bookmarking instances and connecting to a LocalStack instance running on another machine. - [Stack Overview](https://docs.localstack.cloud/aws/connecting/console/stack-overview/): the full list of supported resource types. - [Resource Browser](https://docs.localstack.cloud/aws/connecting/console/resource-browser/): the complete list of supported services beyond Lambda and DynamoDB. - [Cloud Pods](https://docs.localstack.cloud/aws/developer-tools/snapshots/cloud-pods/): merge strategies, remotes, auto-loading pods on startup, and sharing state with your team. - [App Inspector](https://docs.localstack.cloud/aws/developer-tools/app-inspector/): using it from the [LocalStack Toolkit for VS Code](https://docs.localstack.cloud/aws/connecting/ides/vscode-extension/) without leaving your editor. - [`lstk` CLI reference](https://docs.localstack.cloud/aws/developer-tools/running-localstack/lstk/): the full set of `snapshot`, `status`, and `reset` commands used in this guide. Once you're comfortable with the local workflow, continue to the [CI/CD guide](https://docs.localstack.cloud/aws/getting-started/ci-cd/) to bring LocalStack into your automated pipelines. # Deploy LocalStack on Kubernetes > Deploy LocalStack into a Kubernetes cluster and run a sample Lambda + RDS application in under 5 minutes. ## Introduction This quickstart spins up LocalStack in a local Kubernetes cluster and deploys a sample application in 5 minutes. You'll run an AWS Lambda function that queries an RDS MySQL database, with both services executing as pods managed by LocalStack. LocalStack's Kubernetes integration is available as part of the [Enterprise plan](https://localstack.cloud/pricing). ## Prerequisites Before starting, make sure you have the following: - A [LocalStack Auth Token](https://docs.localstack.cloud/getting-started/auth-token/) exported as `LOCALSTACK_AUTH_TOKEN` - [Docker](https://docs.docker.com/get-docker/) - [`kind`](https://kind.sigs.k8s.io/) - [Terraform](https://www.terraform.io/downloads) (v1.11.1 or later) with the [`tflocal`](https://docs.localstack.cloud/user-guide/integrations/terraform/) wrapper - [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) with the [`awslocal`](https://docs.localstack.cloud/user-guide/integrations/aws-cli/#localstack-aws-cli-awslocal) wrapper - [`kubectl`](https://kubernetes.io/docs/reference/kubectl/) - [`jq`](https://jqlang.github.io/jq/download/) - [`k9s`](https://k9scli.io/) (optional, for visual cluster monitoring) ## Step by Step ### Step 1: Clone the sample repository ```bash git clone https://github.com/localstack-samples/localstack-k8s-demo.git cd localstack-k8s-demo ``` The repository contains: - `main.tf`: Terraform configuration that provisions an RDS MySQL database and a Lambda function on LocalStack - `lambda-src/`: Python Lambda function source code with `pymysql` as a dependency - `localstack-instance.yml`: Custom resource definition for the LocalStack deployment - `scripts/`: Helper scripts for managing the auth token secret and port forwarding - `Makefile`: Convenience targets for the full workflow ### Step 2: Create the Kubernetes cluster If you want to monitor the cluster visually, open a separate terminal and run `k9s`. The interface starts empty but populates as pods come up. Create a local Kubernetes cluster using `kind`: ```bash kind create cluster --name ls-k8s-demo ``` Verify the cluster is running: ```bash kubectl cluster-info ``` ### Step 3: Deploy the LocalStack Operator The [LocalStack Operator](https://github.com/localstack/localstack-operator) manages the LocalStack deployment and configures cluster DNS so that AWS-style hostnames resolve correctly inside the cluster. :::note The LocalStack Operator and Kubernetes executor are part of the [Enterprise plan](https://localstack.cloud/pricing) and are not enabled by default on a trial license. If your Auth Token doesn't have access, contact your LocalStack account team or [support](/aws/help-support/get-help/) to get the Kubernetes pack enabled on your trial. ::: ```bash kubectl apply -f https://github.com/localstack/localstack-operator/releases/latest/download/controller.yaml ``` Wait for the operator pod to reach a `Running` state: ```bash kubectl get pods -n localstack-operator-system ``` ``` NAME READY STATUS RESTARTS AGE localstack-operator-controller-manager-78dcf78855-xxxxx 1/1 Running 0 30s ``` ### Step 4: Deploy LocalStack into the cluster Create a namespace and a secret containing your Auth Token: ```bash kubectl create namespace workspace kubectl create secret -n workspace generic localstack-auth-token \ --from-literal=LOCALSTACK_AUTH_TOKEN=$LOCALSTACK_AUTH_TOKEN ``` Deploy the LocalStack instance: ```bash kubectl apply --server-side -f ./localstack-instance.yml ``` Wait for the LocalStack pod to be ready (this may take a minute or two while the image is pulled): ```bash kubectl get pods -n workspace -w ``` Proceed once the pod shows `1/1 Running`. ### Step 5: Set up port forwarding Forward `port 4566` so you can run AWS commands against LocalStack from your local machine: ```bash kubectl port-forward -n workspace svc/localstack-env-1 4566 ``` This runs in the foreground. Open a new terminal for the remaining steps. Verify LocalStack is accessible: ```bash awslocal sts get-caller-identity ``` You can also confirm connectivity using the [LocalStack Web Application](https://app.localstack.cloud/inst/default/overview). With the port forward active, open the [Stack Overview](https://app.localstack.cloud/inst/default/overview) in your browser. It should load and show your LocalStack instance as connected, which is a quick way to confirm your cluster setup before deploying the sample application. ### Step 6: Deploy the sample application with Terraform The Terraform configuration provisions the following resources on LocalStack: - A VPC - An RDS MySQL database (`k8sdb`) - A Lambda function (`myfunction`) that connects to and queries the database Both the database and Lambda function run as separate pods in the cluster, managed by LocalStack's Kubernetes executor. ```bash tflocal init -upgrade tflocal apply -auto-approve ``` The deployment takes a few minutes as the MySQL pod needs to start up. Monitor progress with `k9s` or: ```bash kubectl get pods -A -w ``` :::note The Lambda module is configured for ARM64 by default. If you are on an Intel/AMD machine, update `main.tf` to remove the `docker_additional_options` block and change `architectures` to `["x86_64"]`. ::: ### Step 7: Invoke the Lambda function ```bash awslocal lambda invoke \ --function-name myfunction \ --payload '{}' /dev/stdout | jq . ``` The first invocation takes about 30 seconds as the Lambda pod starts up. You should see output like: ```json { "results": [ [1, "test"], [2, "another"] ] } ``` ## Validation Confirm that all three pods are running in the `workspace` namespace: ```bash kubectl get pods -n workspace ``` ``` NAME READY STATUS AGE lambda-myfunction-xxxxx 1/1 Running 36s localstack-env-1-xxxxx 1/1 Running 11m ls-mysql-xxxxx 1/1 Running 4m ``` You should see the LocalStack pod (`localstack-*`), the MySQL database pod (`ls-mysql-*`), and the Lambda function pod (`lambda-myfunction-*`) all running. ## Cleanup To tear down all resources: ```bash tflocal apply -destroy -auto-approve kubectl delete -f ./localstack-instance.yml kubectl delete secret -n workspace localstack-auth-token ``` ## Troubleshooting ### LocalStack pod is stuck in `Pending` or `ImagePullBackOff` Verify that your Auth Token secret was created correctly and that your cluster nodes can pull from the LocalStack registry. Check pod events with `kubectl describe pod -n workspace `. ### `awslocal sts get-caller-identity` times out Confirm that port forwarding is still running in a separate terminal. If it dropped, restart it with `kubectl port-forward -n workspace svc/localstack-env-1 4566`. ### Lambda invocation returns an error after the first call The first invocation takes up to 30 seconds for the Lambda pod to start. Wait and retry. ### Terraform apply fails with a connection error Ensure port forwarding is active before running `tflocal apply`. LocalStack must be accessible on `localhost:4566`. ### MySQL pod does not start Check cluster resource availability. The MySQL pod requires sufficient CPU and memory. Run `kubectl describe pod -n workspace ` to inspect scheduling events. ## Next Steps This quickstart covers a minimal deployment. For production-ready configuration options (including persistent storage, advanced networking, scaling, and monitoring), see the full [Kubernetes Enterprise guide](https://docs.localstack.cloud/aws/enterprise/kubernetes/). # Sample Apps > Sample Apps to help LocalStack users adopt real-world scenarios to rapidly and conveniently create, configure, and deploy applications locally. import ApplicationsShowcase from "../../../components/ApplicationsShowcase.astro"; # Local AWS Services > Browse LocalStack's implemented AWS services and explore their capabilities import SearchableAwsServices from '../../../../components/SearchableAwsServices.astro'; # Account Management > Get started with AWS Account Management on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Account service provides APIs to manage your AWS account. You can use the Account APIs to retrieve information about your account, manage your contact information and alternate contacts. Additionally, you can use the Account APIs to enable or disable a region for your account, and delete alternate contacts in your account. LocalStack allows you to use the Account API to retrieve information about your account. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Account's integration with LocalStack. :::note LocalStack's Account provider is mock-only and does not support connecting to any real AWS account. The Account APIs are only intended to demonstrate how you can use and mock the AWS Account APIs in your local environment. It's important to note that LocalStack doesn't offer a programmatic interface to manage your AWS or your LocalStack account. ::: ## Getting started This guide is designed for users who are new to Account and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to put contact information, fetch account details, and attach an alternate contact to your account. ### Put contact information You can use the [`PutContactInformation`](https://docs.aws.amazon.com/accounts/latest/reference/API_PutContactInformation.html) API to add or update the contact information for your AWS account. Run the following command to add contact information to your account: ```bash showshowLineNumbers lstk aws account put-contact-information \ --contact-information '{ "FullName": "Jane Doe", "PhoneNumber": "+XXXXXXXXX", "AddressLine1": "XXXX Main St", "City": "XXXX", "PostalCode": "XXXXX", "CountryCode": "US", "StateOrRegion": "WA" }' ``` ### Fetch account details You can use the [`GetContactInformation`](https://docs.aws.amazon.com/accounts/latest/reference/API_GetContactInformation.html) API to retrieve the contact information for your AWS account. Run the following command to fetch the contact information for your account: ```bash lstk aws account get-contact-information ``` ```bash title="Output" showshowLineNumbers { "ContactInformation": { "AddressLine1": "XXXX Main St", "City": "XXXX", "CountryCode": "US", "FullName": "Jane Doe", "PhoneNumber": "+XXXXXXXXX", "PostalCode": "XXXXX", "StateOrRegion": "WA" } } ``` ### Attach alternate contact You can attach an alternate contact using [`PutAlternateContact`](https://docs.aws.amazon.com/accounts/latest/reference/API_PutAlternateContact.html) API. Run the following command to attach an alternate contact to your account: ```bash showshowLineNumbers lstk aws account put-alternate-contact \ --alternate-contact-type "BILLING" \ --email-address "bill@ing.com" \ --name "Bill Ing" \ --phone-number "+1 555-555-5555" \ --title "Billing" ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing contact information & alternate accounts for the Account service. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the Resources section, and then clicking on **Account** under the **Management & Governance** section. ![Account Resource Browser](/images/aws/account-resource-browser.png) The Resource Browser allows you to perform the following actions: * **Create Contact Information**: Add the contact information for your mocked AWS account by clicking on the **Create** button in the contact information section. * **Create Alternate Contact**: Add an alternate contact for your mocked AWS account by clicking on the **Create** button in the alternate contacts section. * **View Contact Information**: View the contact information for your mocked AWS account by clicking on the contact information. * **Update Contact Information**: Update the contact information for your mocked AWS account by clicking on the contact information. * **Filter**: Filter the contact information and alternate contacts by types, such as `BILLING`, `OPERATIONS`, and `SECURITY`. ## API Coverage # Certificate Manager (ACM) > Get started with AWS Certificate Manager (ACM) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction [AWS Certificate Manager (ACM)](https://aws.amazon.com/certificate-manager/) is a service that enables you to create and manage SSL/TLS certificates that can be used to secure your applications and resources in AWS. You can use ACM to provision and deploy public or private certificates trusted by browsers and other clients. ACM supports securing multiple domain names and subdomains and can create wildcard SSL certificates to protect an entire domain and its subdomains. You can also use ACM to import certificates from third-party certificate authorities or to generate private certificates for internal use. LocalStack allows you to use the ACM APIs to create, list, and delete certificates. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of ACM's integration with LocalStack. ## Getting started This guide is designed for users who are new to ACM and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. ### Request a public certificate Start your LocalStack container using your preferred method, then use the [RequestCertificate API](https://docs.aws.amazon.com/acm/latest/APIReference/API_RequestCertificate.html) to request a new public ACM certificate. Specify the domain name you want to request the certificate for, and any additional options you need. Here's an example command: ```bash showshowLineNumbers lstk aws acm request-certificate \ --domain-name www.example.com \ --validation-method DNS \ --idempotency-token 1234 \ --options CertificateTransparencyLoggingPreference=DISABLED ``` This command will return the Amazon Resource Name (ARN) of the new certificate, which you can use in other ACM commands. ```bash title="Output" { "CertificateArn": "arn:aws:acm::000000000000:certificate/" } ``` ### List the certificates Use the [`ListCertificates` API](https://docs.aws.amazon.com/acm/latest/APIReference/API_ListCertificates.html) to list all the certificates. This command returns a list of the ARNs of all the certificates that have been requested or imported into ACM. Here's an example command: ```bash lstk aws acm list-certificates --max-items 10 ``` ### Describe the certificate Use the [`DescribeCertificate` API](https://docs.aws.amazon.com/acm/latest/APIReference/API_DescribeCertificate.html) to view the details of a specific certificate. Provide the ARN of the certificate you want to view, and this command will return information about the certificate's status, domain name, and other attributes. Here's an example command: ```bash lstk aws acm describe-certificate --certificate-arn arn:aws:acm::account:certificate/ ``` ### Delete the certificate Finally you can use the [`DeleteCertificate` API](https://docs.aws.amazon.com/acm/latest/APIReference/API_DeleteCertificate.html) to delete a certificate from ACM, by passing the ARN of the certificate you want to delete. Here's an example command: ```bash lstk aws acm delete-certificate --certificate-arn arn:aws:acm::account:certificate/ ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing ACM Certificates. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resource Browser** section, and then clicking on **Certificate Manager** under the **Security Identity Compliance** section. ![ACM Resource Browser](/images/aws/acm-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Certificate**: Create a new ACM certificate by clicking **Create Certificate** and providing the required information. - **View Certificate**: View the details of a specific certificate by clicking on the domain name. - **Delete Certificate**: Delete a certificate by selecting the certificate, followed by clicking **Actions** and then **Remove Selected**. ## Examples The following code snippets and sample applications provide practical examples of how to use ACM in LocalStack for various use cases: - [API Gateway with Custom Domains](https://github.com/localstack/localstack-pro-samples/tree/master/apigw-custom-domain) - [Generating an ACM certificate via Terraform](https://github.com/localstack/localstack-terraform-samples/tree/master/acm-route53) ## API Coverage # Private Certificate Authority (ACM PCA) > Get started with Private Certificate Authority (ACM PCA) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction AWS Private Certificate Authority (ACM PCA) is a managed private Certificate Authority (CA) service that manages the lifecycle of your private certificates. ACM PCA extends ACM's certificate management capabilities to private certificates, enabling you to manage public and private certificates centrally. LocalStack allows you to use the ACM PCA APIs to create, list, and delete private certificates. You can creating, describing, tagging, and listing tags for a CA using ACM PCA. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of ACM PCA's integration with LocalStack. ## Getting started This guide is designed for users who are new to ACM PCA and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. We will follow the procedure to create and install a certificate for a single-level hierarchy CA hosted by ACM PCA. ### Create a CA Start by creating a new Certificate Authority with ACM PCA using the [`CreateCertificateAuthority`](https://docs.aws.amazon.com/privateca/latest/APIReference/API_CreateCertificateAuthority.html) API. This command sets up a new CA with specified configurations for key algorithm, signing algorithm, and subject information. ```bash lstk aws acm-pca create-certificate-authority \ --certificate-authority-configuration '{ "KeyAlgorithm":"RSA_2048", "SigningAlgorithm":"SHA256WITHRSA", "Subject":{ "Country":"CH", "Organization":"LocalStack", "OrganizationalUnit":"Engineering", "CommonName":"test.localstack.cloud" } }' \ --certificate-authority-type "ROOT" ``` ```bash title="Output" { "CertificateAuthorityArn": "arn:aws:acm-pca:eu-central-1:000000000000:certificate-authority/0b20353f-ce7a-4de4-9b82-e06903a893ff" } ``` Note the `CertificateAuthorityArn` from the output as it will be needed for subsequent commands. To retrieve the detailed information about the created Certificate Authority, use the [`DescribeCertificateAuthority`](https://docs.aws.amazon.com/privateca/latest/APIReference/API_DescribeCertificateAuthority.html) API. This command returns the detailed information about the CA, including the CA's ARN, status, and configuration. ```bash lstk aws acm-pca describe-certificate-authority \ --certificate-authority-arn arn:aws:acm-pca:eu-central-1:000000000000:certificate-authority/0b20353f-ce7a-4de4-9b82-e06903a893ff ``` ```bash title="Output" { "CertificateAuthority": { "Arn": "arn:aws:acm-pca:eu-central-1:000000000000:certificate-authority/0b20353f-ce7a-4de4-9b82-e06903a893ff", "OwnerAccount": "000000000000", "CreatedAt": "2024-08-08T10:45:58.065504+05:30", "Type": "ROOT", "Status": "PENDING_CERTIFICATE", "CertificateAuthorityConfiguration": { "KeyAlgorithm": "RSA_2048", "SigningAlgorithm": "SHA256WITHRSA", "Subject": { "Country": "CH", "Organization": "LocalStack", "OrganizationalUnit": "Engineering", "CommonName": "test.localstack.cloud" } }, "RevocationConfiguration": { "CrlConfiguration": { "Enabled": false } }, "KeyStorageSecurityStandard": "FIPS_140_2_LEVEL_3_OR_HIGHER", "UsageMode": "SHORT_LIVED_CERTIFICATE" } } ``` Note the `PENDING_CERTIFICATE` status. In the following steps, we will create and attach a certificate for this CA. ### Issue CA Certificate Use the [`GetCertificateAuthorityCsr`](https://docs.aws.amazon.com/privateca/latest/APIReference/API_GetCertificateAuthorityCsr.html) operation to obtain the Certificate Signing Request (CSR) for the CA. ```bash lstk aws acm-pca get-certificate-authority-csr \ --certificate-authority-arn arn:aws:acm-pca:eu-central-1:000000000000:certificate-authority/0b20353f-ce7a-4de4-9b82-e06903a893ff \ --output text | tee ca.csr ``` Next, issue the certificate for the CA using this CSR. ```bash lstk aws acm-pca issue-certificate \ --csr fileb://ca.csr \ --signing-algorithm SHA256WITHRSA \ --template-arn arn:aws:acm-pca:::template/RootCACertificate/V1 \ --validity Value=10,Type=YEARS \ --certificate-authority-arn arn:aws:acm-pca:eu-central-1:000000000000:certificate-authority/0b20353f-ce7a-4de4-9b82-e06903a893ff ``` ```bash title="Output" { "CertificateArn": "arn:aws:acm-pca:eu-central-1:000000000000:certificate-authority/0b20353f-ce7a-4de4-9b82-e06903a893ff/certificate/17ef7bbf3cc6471ba3ef0707119b8392" } ``` The CA certificate is now created and its ARN is indicated by the `CertificateArn` parameter. ### Import CA Certificate Finally, we retrieve the signed certificate with [`GetCertificate`](https://docs.aws.amazon.com/privateca/latest/APIReference/API_GetCertificate.html) and import it using [`ImportCertificateAuthorityCertificate`](https://docs.aws.amazon.com/privateca/latest/APIReference/API_ImportCertificateAuthorityCertificate.html). ```bash lstk aws acm-pca get-certificate \ --certificate-authority-arn arn:aws:acm-pca:eu-central-1:000000000000:certificate-authority/0b20353f-ce7a-4de4-9b82-e06903a893ff \ --certificate-arn arn:aws:acm-pca:eu-central-1:000000000000:certificate-authority/0b20353f-ce7a-4de4-9b82-e06903a893ff/certificate/17ef7bbf3cc6471ba3ef0707119b8392 \ --output text | tee cert.pem ``` ```bash lstk aws acm-pca import-certificate-authority-certificate \ --certificate-authority-arn arn:aws:acm-pca:eu-central-1:000000000000:certificate-authority/0b20353f-ce7a-4de4-9b82-e06903a893ff \ --certificate fileb://cert.pem ``` The CA is now ready for use. You can verify this by checking its status: ```bash lstk aws acm-pca describe-certificate-authority \ --certificate-authority-arn arn:aws:acm-pca:eu-central-1:000000000000:certificate-authority/0b20353f-ce7a-4de4-9b82-e06903a893ff \ --query CertificateAuthority.Status \ --output text ``` ```bash title="Output" ACTIVE ``` The CA certificate can be retrieved at a later point using [`GetCertificateAuthorityCertificate`](https://docs.aws.amazon.com/privateca/latest/APIReference/API_GetCertificateAuthorityCertificate.html). In general, this operation returns both the certificate and the certificate chain. In this case however, only the certificate will be returned, because we used a single-level CA hierarchy and the certificate chain is null. For production setups, you must use a [multi-level CA hierarchy](https://docs.aws.amazon.com/privateca/latest/userguide/ca-hierarchy.html) for best security. ### Issue End-entity Certificates With the private CA set up, you can now issue end-entity certificates. Using [OpenSSL](https://openssl-library.org/), create a CSR and the private key: ```bash openssl req -out local-csr.pem -new -newkey rsa:2048 -nodes -keyout local-pkey.pem ``` You may inspect the CSR using the following command. It should resemble the illustrated output. ```bash openssl req -in local-csr.pem -text -noout ``` ```bash title="Output" Certificate Request: Data: Version: 1 (0x0) Subject: C = IN, ST = GA, O = EvilCorp, OU = Engineering, CN = evilcorp.com Subject Public Key Info: Public Key Algorithm: rsaEncryption Public-Key: (2048 bit) Modulus: 00:a3:1d:5d:50:00:5c:4e:5d:79:a8:9a:d4:10:f4: ... Exponent: 65537 (0x10001) Attributes: (none) Requested Extensions: Signature Algorithm: sha256WithRSAEncryption Signature Value: 3e:23:12:26:45:af:39:35:5d:d7:b4:40:fb:1a:08:c7:16:c3: ... ``` Next, using [`IssueCertificate`](https://docs.aws.amazon.com/privateca/latest/APIReference/API_IssueCertificate.html) you can generate the end-entity certificate. Note that there is no [certificate template](https://docs.aws.amazon.com/privateca/latest/userguide/UsingTemplates.html) specified which causes the end-entity certificate to be issued by default. ```bash lstk aws acm-pca issue-certificate \ --certificate-authority-arn arn:aws:acm-pca:eu-central-1:000000000000:certificate-authority/0b20353f-ce7a-4de4-9b82-e06903a893ff \ --csr fileb://local-csr.pem \ --signing-algorithm "SHA256WITHRSA" \ --validity Value=365,Type="DAYS" ``` The output will be similar to the following: ```json { "CertificateArn": "arn:aws:acm-pca:eu-central-1:000000000000:certificate-authority/0b20353f-ce7a-4de4-9b82-e06903a893ff/certificate/079d0a13daf943f6802d365dd83658c7" } ``` ### Verify Certificates Using OpenSSL, you can verify that the end-entity certificate was indeed signed by the CA. In the following command, `local-cert.pem` refers to the end-entity certificate and `cert.pem` refers to the CA certificate. ```bash openssl verify -CAfile cert.pem local-cert.pem ``` ```bash title="Output" local-cert.pem: OK ``` ### Tag the Certificate Authority Tagging resources in AWS helps in managing and identifying them. Use the [`TagCertificateAuthority`](https://docs.aws.amazon.com/privateca/latest/APIReference/API_TagCertificateAuthority.html) API to tag the created Certificate Authority. This command adds the specified tags to the specified CA. ```bash lstk aws acm-pca tag-certificate-authority \ --certificate-authority-arn arn:aws:acm-pca:us-east-1:000000000000:certificate-authority/f38ee966-bc23-40f8-8143-e981aee73600 \ --tags Key=Admin,Value=Alice ``` After tagging your Certificate Authority, you may want to view these tags. You can use the [`ListTags`](https://docs.aws.amazon.com/privateca/latest/APIReference/API_ListTags.html) API to list all the tags associated with the specified CA. ```bash lstk aws acm-pca list-tags \ --certificate-authority-arn arn:aws:acm-pca:us-east-1:000000000000:certificate-authority/f38ee966-bc23-40f8-8143-e981aee73600 \ --max-results 10 ``` ```bash title="Output" { "Tags": [ { "Key": "Name", "Value": "MyPCA" }, { "Key": "Admin", "Value": "Alice" } ] } ``` ## API Coverage # Amplify > Get started with Amplify on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Amplify is a JavaScript-based development framework with libraries, UI components, and a standard CLI interface for building and deploying web and mobile applications. With Amplify, developers can build and host static websites, single-page applications, and full-stack serverless web applications using an abstraction layer over popular AWS services like DynamoDB, Cognito, AppSync, Lambda, S3, and more. LocalStack allows you to use the Amplify APIs to build and test their Amplify applications locally. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Amplify's integration with LocalStack. :::note The `amplifylocal` CLI and the Amplify JS library have been deprecated and are no longer supported. We recommend using the Amplify CLI with the Amplify LocalStack Plugin instead. ::: ## Amplify LocalStack Plugin [Amplify LocalStack Plugin](https://github.com/localstack/amplify-localstack) allows the `amplify` CLI tool to create resources on your local machine instead of AWS. It achieves this by redirecting any requests to AWS to a LocalStack container running locally on your machine. ### Installation To install the Amplify LocalStack Plugin, install the [amplify-localstack](https://www.npmjs.com/package/amplify-localstack) package from the npm registry and add the plugin to your Amplify setup: ```bash npm install -g amplify-localstack amplify plugin add amplify-localstack ``` ### Configuration You can configure the following environment variables to customize LocalStack's behaviour: - `EDGE_PORT`: The port number under which the LocalStack edge service is accessible. The default value is `4566`. - `LOCALSTACK_HOSTNAME`: It specifies the target host under which the LocalStack edge service is accessible. The default value is `localhost.localstack.cloud`. - `LOCALSTACK_ENDPOINT`: It allows you to set a custom endpoint directly. If set, it overrides the values set for `EDGE_PORT` and `LOCALSTACK_HOSTNAME`. The default value is `https://localhost.localstack.cloud:4566`. ### Usage After installing the plugin, you can deploy your resources to LocalStack using the `amplify init` or `amplify push` commands. The console will prompt you to select whether to deploy to LocalStack or AWS. You can also add the parameter `--use-localstack true` to your commands to avoid being prompted and automatically use LocalStack. Here is an example: ```bash amplify init --use-localstack true amplify add api amplify push --use-localstack true ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing Amplify applications. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resource Browser** section, and then clicking on **Amplify** under the **Front-end Web & Mobile** section. ![Amplify Resource Browser](/images/aws/amplify-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create new Amplify applications**: Create new Amplify applications by clicking **Create App** and filling in the required details. - **View Amplify applications**: View the list of Amplify applications created in LocalStack by clicking on the application ID. - **Edit Amplify applications**: Edit the configuration of an existing Amplify application by clicking on the application ID and then clicking **Edit App**. - **Delete Amplify applications**: Delete an existing Amplify application by selecting the application, followed by clicking **Actions** and then **Remove Selected**. ## Current Limitations LocalStack's Amplify API coverage does not include all deployment APIs required for full Amplify Hosting publish flows. If your workflow depends on `amplify publish` to upload static frontend artifacts (HTML/CSS/JS), run the frontend with your local framework/dev server while using LocalStack for backend services. ## API Coverage # API Gateway > Get started with API Gateway on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; import { Badge } from '@astrojs/starlight/components'; ## Introduction API Gateway is a managed service that enables developers to create, deploy, and manage APIs (Application Programming Interfaces). It allows easy creation of REST, HTTP, and WebSocket APIs to securely access data, business logic, or functionality from backend services like AWS Lambda functions or EC2 instances. API Gateway supports standard HTTP methods such as `GET`, `POST`, `PUT`, `PATCH`, and `DELETE` and integrates with various AWS services, including Lambda, Cognito, CloudWatch, and X-Ray. LocalStack supports API Gateway V2 (HTTP, Management and WebSocket API) in the Base plan. LocalStack allows you to use the API Gateway APIs to create, deploy, and manage APIs on your local machine to invoke those exposed API endpoints. The supported APIs are available on the API coverage section for [API Gateway V1](#api-coverage-v1), [API Gateway V2](#api-coverage-v2), and [API Gateway Management](#api-coverage-api-gateway-management), which provides information on the extent of API Gateway's integration with LocalStack. ## Getting started This guide is designed for users new to API Gateway and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will use the Lambda proxy integration to integrate an API method with a Lambda function. The Lambda function will be invoked with a `GET` request and return a response with a status code of `200` and a body containing the string `Hello from Lambda!`. ### Create a Lambda function Create a new file named `lambda.js` with the following contents: ```javascript showshowLineNumbers 'use strict' const apiHandler = (payload, context, callback) => { callback(null, { statusCode: 200, body: JSON.stringify({ message: 'Hello from Lambda' }), }); } module.exports = { apiHandler, } ``` The above code defines a function named `apiHandler` that returns a response with a status code of `200` and a body containing the string `Hello from Lambda`. Zip the file and upload it to LocalStack using the `lstk aws` command. Run the following command: ```bash showshowLineNumbers zip function.zip lambda.js lstk aws lambda create-function \ --function-name apigw-lambda \ --runtime nodejs16.x \ --handler lambda.apiHandler \ --memory-size 128 \ --zip-file fileb://function.zip \ --role arn:aws:iam::111111111111:role/apigw ``` This creates a new Lambda function named `apigw-lambda` with the code you specified. ### Create a REST API We will use the API Gateway's [`CreateRestApi`](https://docs.aws.amazon.com/apigateway/latest/api/API_CreateRestApi.html) API to create a new REST API. Here's an example command: ```bash lstk aws apigateway create-rest-api --name 'API Gateway Lambda integration' ``` This creates a new REST API named `API Gateway Lambda integration`. ```bash title="Output" { "id": "cor3o5oeci", "name": "API Gateway Lambda integration", "createdDate": "2023-04-27T16:08:46+05:30", "apiKeySource": "HEADER", "endpointConfiguration": { "types": [ "EDGE" ] }, "disableExecuteApiEndpoint": false } ``` Note the REST API ID returned in the response. You'll need this ID for the next step. ### Fetch the Resources Use the REST API ID generated in the previous step to fetch the resources for the API, using the [`GetResources`](https://docs.aws.amazon.com/apigateway/latest/api/API_GetResources.html) API: ```bash lstk aws apigateway get-resources --rest-api-id ``` ```bash title="Output" { "items": [ { "id": "u53af9hm83", "path": "/" } ] } ``` Note the ID of the root resource returned in the response. You'll need this ID for the next step. ### Create a resource Create a new resource for the API using the [`CreateResource`](https://docs.aws.amazon.com/apigateway/latest/api/API_CreateResource.html) API. Use the ID of the resource returned in the previous step as the parent ID: ```bash showshowLineNumbers lstk aws apigateway create-resource \ --rest-api-id \ --parent-id \ --path-part "{somethingId}" ``` ```bash title="Output" { "id": "zzcvcf56ar", "parentId": "u53af9hm83", "pathPart": "{somethingId}", "path": "/{somethingId}" } ``` Note the ID of the root resource returned in the response. You'll need this Resource ID for the next step. ### Add a method and integration Add a `GET` method to the resource using the [`PutMethod`](https://docs.aws.amazon.com/apigateway/latest/api/API_PutMethod.html) API. Use the ID of the resource returned in the previous step as the Resource ID: ```bash showshowLineNumbers lstk aws apigateway put-method \ --rest-api-id \ --resource-id \ --http-method GET \ --request-parameters "method.request.path.somethingId=true" \ --authorization-type "NONE" ``` ```bash title="Output" { "httpMethod": "GET", "authorizationType": "NONE", "apiKeyRequired": false, "requestParameters": { "method.request.path.somethingId": true } } ``` Now, create a new integration for the method using the [`PutIntegration`](https://docs.aws.amazon.com/apigateway/latest/api/API_PutIntegration.html) API. ```bash showshowLineNumbers lstk aws apigateway put-integration \ --rest-api-id \ --resource-id \ --http-method GET \ --type AWS_PROXY \ --integration-http-method POST \ --uri arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:000000000000:function:apigw-lambda/invocations \ --passthrough-behavior WHEN_NO_MATCH ``` The above command integrates the `GET` method with the Lambda function created in the first step. We can now proceed with the deployment before invoking the API. ### Create a deployment Create a new deployment for the API using the [`CreateDeployment`](https://docs.aws.amazon.com/apigateway/latest/api/API_CreateDeployment.html) API: ```bash lstk aws apigateway create-deployment \ --rest-api-id \ --stage-name dev ``` Your API is now ready to be invoked. You can use [curl](https://curl.se/) or any HTTP REST client to invoke the API endpoint: ```bash curl -X GET http://.execute-api.localhost.localstack.cloud:4566/dev/test ``` ```bash title="Output" {"message":"Hello World"} ``` You can also use our [alternative URL format](#alternative-url-format) in case of DNS issues: ```bash curl -X GET http://localhost:4566/_aws/execute-api//dev/test ``` ```bash title="Output" {"message":"Hello World"} ``` ## New API Gateway implementation :::note The new API Gateway implementation for both v1 (REST API) and v2 (HTTP API), introduced in [LocalStack 3.8.0](https://blog.localstack.cloud/localstack-release-v-3-8-0/#new-api-gateway-provider), is now the default in 4.0. If you were using the `PROVIDER_OVERRIDE_APIGATEWAY=next_gen` flag, please remove it as it is no longer required. The legacy provider (`PROVIDER_OVERRIDE_APIGATEWAY=legacy`) is temporarily available but deprecated and will be removed in the next major release. We strongly recommend migrating to the new implementation. ::: We're entirely reworked how REST and HTTP APIs are invoked, to closely match the behavior on AWS. This new implementation has improved parity on several key areas: - for [REST APIs](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-rest-api.html): - properly applying the [request and response data mappings](https://docs.aws.amazon.com/apigateway/latest/developerguide/request-response-data-mappings.html) for all integrations - better parity for VTL template rendering ([Mapping Templates](https://docs.aws.amazon.com/apigateway/latest/developerguide/models-mappings.html)) for the integrations supporting it (`AWS`, `HTTP` and `MOCK`) - properly supporting [Mapping Templates overrides](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-override-request-response-parameters.html) - better parity for `AWS_PROXY` integration payloads - out of the box support for most of `AWS` integrations - support for [Gateway Responses](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-gatewayResponse-definition.html) - we currently only support overriding the Status Code and returning the proper exception, and do not apply mapping template (response body) or parameter mappings (response headers) - for [HTTP APIs](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api.html): - better validation and parity for most API operations related to HTTP APIs - better parity and properly applying [request and response Parameter Mappings](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-parameter-mapping.html) for all integrations - we've properly implemented the `AWS_PROXY` [Lambda integration](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-integrations-lambda.html) and `REQUEST` [Lambda Authorizer](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-lambda-authorizer.html) payloads to be fully on parity with AWS - better [routing](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-routes.html) handling - better [CORS](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-cors.html) handling, especially around automatic `OPTIONS` responses - support for [automatic deployments](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-stages.html) of your stages - For both REST and HTTP APIs: - support Stage and Deployments, meaning you can now have different stages pointing to different deployments like in AWS - better logging on the different steps in the LocalStack logs Currently, [WebSockets APIs](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-websocket-api.html) are still using the default implementation. As we're closely following AWS, for REST and HTTP APIs, you now need to create a deployment in order for your API to be reachable. Thanks to this improvement, you can now create different stages point to different deployments of your API (for example, `dev` and `production`) with different settings and stage variables, and those will be reflected in LocalStack. ## LocalStack features LocalStack provides additional features and functionality on top of the official AWS APIs, to help you develop, debug, and test your local API Gateway APIs. ### Accessing HTTP APIs via Local Domain Name To demonstrate how to access APIs through LocalStack's local domain name, consider the following Serverless configuration that shows two Lambda functions (`serviceV1` and `serviceV2`) that are connected to an API Gateway v1 (`http` event) and an API Gateway v2 endpoint (`httpApi` event), respectively: ```yaml showshowLineNumbers ... plugins: - serverless-localstack custom: localstack: stages: [local] functions: serviceV1: handler: handler.handler events: - http: # for API GW v1 integration method: POST path: /my/path1 serviceV2: handler: handler.handler events: - httpApi: # for API GW v2 integration method: POST path: /my/path2 ``` After you deploy the Lambda functions and API Gateway endpoints, you can access them using the LocalStack edge port (`4566` by default). There are two alternative URL formats to access these endpoints. #### Recommended URL format The recommended URL format for accessing APIs is to use the following URL syntax with an `execute-api` hostname: ```shell http://.execute-api.localhost.localstack.cloud:4566// ``` Here's an example of how you would access the HTTP/REST API with an ID of `0v1p6q6`: ```shell http://0v1p6q6.execute-api.localhost.localstack.cloud:4566/local/my/path2 ``` Note that the local stage ID is added in this example. Adding the stage ID is required for API Gateway V1 APIs, but optional for API Gateway V2 APIs (in case a `$default` stage is created). For v2 APIs, the following URL should also work: ```shell http://0v1p6q6.execute-api.localhost.localstack.cloud:4566/my/path1 ``` #### Alternative URL format The alternative URL format is an endpoint with the predefined base path `/_aws/execute-api`: ```shell http://localhost:4566/_aws/execute-api/// ``` For the example above, the URL would be: ```shell http://localhost:4566/_aws/execute-api/0v1p6q6/local/my/path1 ``` This format is sometimes used in case of local DNS issues. :::note If you are using LocalStack 4.0, the following `_user_request_` format is deprecated, and you should use the format above. ```shell http://localhost:4566/restapis///_user_request_/ ``` ::: ### WebSocket APIs WebSocket APIs provide real-time communication channels between a client and a server. To use WebSockets in LocalStack, you can define a WebSocket route in your Serverless configuration: ```yaml showshowLineNumbers ... plugins: - serverless-localstack functions: actionHandler: handler: handler.handler events: - websocket: route: test-action ``` Upon deployment of the Serverless project, LocalStack creates a new API Gateway V2 endpoint. To retrieve the list of APIs and verify the WebSocket endpoint, you can use the `lstk aws` CLI: ```bash lstk aws apigatewayv2 get-apis ``` ```bash title="Output" { "Items": [{ "ApiEndpoint": "ws://localhost:4510", "ApiId": "129ca37e", ... }] } ``` In the above example, the WebSocket endpoint is `ws://localhost:4510`. Assuming your Serverless project contains a simple Lambda `handler.js` like this: ```javascript module.exports.handler = function(event, context, callback) { callback(null, event); }; ``` You can send a message to the WebSocket at `ws://localhost:4510` and the same message will be returned as a response on the same WebSocket. To push data from a backend service to the WebSocket connection, you can use the [Amazon API Gateway Management API](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/apigatewaymanagementapi/index.html). In LocalStack, use the following CLI command (replace `` with your WebSocket connection ID): ```bash lstk aws apigatewaymanagementapi \ post-to-connection \ --connection-id '' \ --data '{"msg": "Hi"}' ``` ## Custom IDs for API Gateway resources via tags You can assign custom IDs to API Gateway REST and HTTP APIs using the `_custom_id_` tag during resource creation. This can be useful to ensure a static endpoint URL for your API, simplifying testing and integration with other services. To assign a custom ID to an API Gateway REST API, use the `create-rest-api` command with the `tags={"_custom_id_":"myid123"}` parameter. The following example assigns the custom ID `"myid123"` to the API: ```bash lstk aws apigateway create-rest-api --name my-api --tags '{"_custom_id_":"myid123"}' ``` ```bash title="Output" { "id": "myid123", .... } ``` You can also configure the protocol type, the possible values being `HTTP` and `WEBSOCKET`: ```bash showshowLineNumbers lstk aws apigatewayv2 create-api \ --name=my-api \ --protocol-type=HTTP --tags="_custom_id_=my-api" { "ApiEndpoint": "my-api.execute-api.localhost.localstack.cloud:4566", "ApiId": "my-api", "Name": "my-api", "ProtocolType": "HTTP", "Tags": { "_custom_id_": "my-api" } } ``` :::note Setting the API Gateway ID via `_custom_id_` works only on the creation of the resource, but not on update in LocalStack. Ensure that you set the `_custom_id_` tag on creation of the resource. ::: ## Custom Domain Names with API Gateway You can use custom domain names with API Gateway [REST APIs](https://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-custom-domains.html) and [HTTP APIs](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-custom-domain-names.html). To use custom domains, you will need to set up an API Gateway Domain Name and create an API Mapping linked to your API. Assuming your custom domain is set up as `test.example.com` to point to your REST API with a base path mapping `base-path` linked to your stage named `dev`, the following command will be directed to your REST API on the `dev` stage. You should include the `Host` header with the custom domain name in your request, so you don't need to set up any custom DNS to resolve to LocalStack. ```bash curl -H 'Host: test.example.com' http://localhost:4566/base-path ``` The request above will be equivalent to the following request: ```bash curl http://.execute-api.localhost.localstack.cloud:4566/dev/ ``` ## API Gateway Resource Browser The LocalStack Web Application provides a Resource Browser for managing API Gateway resources. You can access the Resource Browser by opening the LocalStack Web Application in your browser and navigating to the **Resources** section, then clicking on **API Gateway** under the **App Integration** section. The Resource Browser displays [API Gateway V1](https://app.localstack.cloud/resources/gateway/v1) and [API Gateway V2](https://app.localstack.cloud/resources/gateway/v2) resources. You can click on individual resources to view their details. ![API Gateway Resource Browser](/images/aws/api-gateway-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create API**: Create a new API ([`V1`](https://app.localstack.cloud/resources/gateway/v1/new)/[`V2`](https://app.localstack.cloud/resources/gateway/v2/new)) by clicking on **Create API** button on top-right and creating a new configuration by clicking on **Submit** button. - **Edit API**: Edit the API configuration (`V1`/`V2`) by clicking on **Edit API** button on top-right and saving the new configuration by clicking on **Submit** button. - **Check the Resources**: Click on **Resources** tab to view the resources associated with the API, along with their details, such as `Id`, `ParentId`, `Path Part`, and `Path` and their `HTTP` method. - **Navigate the Stages**: Click on **Stages** tab to view the stages associated with the API, along with their details, such as `Deployment Id`, `Stage Name`, `Client Certificate Id`, and more. - **Delete API**: Delete the API configuration (`V1`/`V2`) by selecting the resource, clicking on **Remove Selected** button on top-right and confirming the deletion by clicking on **Continue** button. You can also use the Resource Browser to check out the **Authorizers**, **Models**, **Request Validators**, **API Keys**, and **Usage Plans**. ## Examples The following code snippets and sample applications provide practical examples of how to use API Gateway in LocalStack for various use cases: - [API Gateway with Custom Domains](https://github.com/localstack/localstack-pro-samples/tree/master/apigw-custom-domain) - [Websockets via API Gateway V2](https://github.com/localstack/localstack-pro-samples/tree/master/serverless-websockets) - [Serverless Container-based APIs with Amazon ECS and Amazon API Gateway](https://github.com/localstack/serverless-api-ecs-apigateway-sample) - [Step-up Authentication using Amazon Cognito, DynamoDB, API Gateway Lambda Authorizer, and Lambda functions](https://github.com/localstack/step-up-auth-sample) - [Serverless Microservices with Amazon API Gateway, DynamoDB, SQS, and Lambda](https://github.com/localstack/microservices-apigateway-lambda-dynamodb-sqs-sample) - [Note-Taking application using AWS SDK for JavaScript, Amazon DynamoDB, Lambda, Cognito, API Gateway, and S3](https://github.com/localstack/aws-sdk-js-notes-app) - For Terraform samples, check out the [LocalStack Terraform examples](https://github.com/localstack/localstack-terraform-samples) repository ## API Coverage (V1) ## API Coverage (V2) ## API Coverage (API Gateway Management) # AppConfig > Get started with AppConfig on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; AppConfig is a service provided by Amazon Web Services (AWS) that simplifies the process of managing and deploying application configurations. AppConfig offers centralized management of configuration data and the ability to create, manage, and deploy configuration changes separately. It allows you to avoid deploying the service repeatedly for smaller changes, enables controlled deployments to applications and includes built-in validation checks & monitoring. LocalStack allows you to use the AppConfig APIs in your local environment to define configurations for different environments and deploy them to your applications as needed. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of AppConfig's integration with LocalStack. ## Getting started This guide is designed for users new to AppConfig and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create an AppConfig application, environment, configuration profiles & feature flags, and deploy the configuration with the AWS CLI. ### Create an AppConfig application and environment You can create an AppConfig application using the [`CreateApplication`](https://docs.aws.amazon.com/appconfig/latest/APIReference/API_CreateApplication.html) API. The application is a folder/directory that contains the configuration data for your specific application. The following command creates an application named `my-app`: ```bash lstk aws appconfig create-application \ --name my-app \ --description "My application" ``` The following output would be retrieved: ```bash { "Id": "400c285", "Name": "my-app", "Description": "My application" } ``` You can now create an AppConfig environment for your application using the [`CreateEnvironment`](https://docs.aws.amazon.com/appconfig/latest/APIReference/API_CreateEnvironment.html) API. An environment consists of the deployment group of your AppConfig applications. The following command creates an environment named `my-app-env`: ```bash lstk aws appconfig create-environment \ --application-id 400c285 \ --name my-app-env \ --description "My application environment" ``` Replace the `application-id` with the ID of the application you created in the previous step. The following output would be retrieved: ```bash { "ApplicationId": "400c285", "Id": "3695ea3", "Name": "my-app-env", "Description": "My application environment", "State": "ReadyForDeployment" } ``` ### Create configuration profiles and feature flags You can create an AppConfig configuration profile using the [`CreateConfigurationProfile`](https://docs.aws.amazon.com/appconfig/latest/APIReference/API_CreateConfigurationProfile.html) API. A configuration profile contains for the configurations of your AppConfig applications. The following command creates a configuration profile named `my-app-config`: ```bash lstk aws appconfig create-configuration-profile \ --application-id 400c285 \ --name my-app-config \ --location-uri hosted \ --type AWS.AppConfig.FeatureFlags ``` The following output would be retrieved: ```bash { "ApplicationId": "400c285", "Id": "7d748f9", "Name": "my-app-config", "LocationUri": "hosted", "Type": "AWS.AppConfig.FeatureFlags" } ``` You can now create a JSON file to add your feature flag configuration data. Create a file named `feature-flag-config.json` with the following content: ```json showshowLineNumbers { "allow_mobile_payments": { "enabled": false }, "default_payments_per_region": { "enabled": true } } ``` You can now use the [`CreateHostedConfigurationVersion`](https://docs.aws.amazon.com/appconfig/latest/APIReference/API_CreateHostedConfigurationVersion.html) API to save your feature flag configuration data to AppConfig. The following command creates a hosted configuration version for the configuration profile you created in the previous step: ```bash lstk aws appconfig create-hosted-configuration-version \ --application-id 400c285 \ --configuration-profile-id 7d748f9 \ --content-type "application/json" \ --content file://feature-flag-config.json \ configuration-data.json ``` ```bash title="Output" { "ApplicationId": "400c285", "ConfigurationProfileId": "7d748f9", "VersionNumber": 1, "ContentType": "application/json" } ``` ### Create an AppConfig deployment You can now create an AppConfig deployment strategy using the [`CreateDeploymentStrategy`](https://docs.aws.amazon.com/appconfig/latest/APIReference/API_CreateDeploymentStrategy.html) API. A deployment strategy defines important criteria for rolling out your configuration to the target environment. The following command creates a deployment strategy named `my-app-deployment-strategy`: ```bash lstk aws appconfig create-deployment-strategy \ --name my-app-deployment-strategy \ --description "My application deployment strategy" \ --deployment-duration-in-minutes 10 \ --growth-factor 1.0 ``` ```bash title="Output" { "Id": "f2f2225", "Name": "my-app-deployment-strategy", "Description": "My application deployment strategy", "DeploymentDurationInMinutes": 10, "GrowthFactor": 1.0 } ``` You can now use the [`StartDeployment`](https://docs.aws.amazon.com/appconfig/latest/APIReference/API_StartDeployment.html) API to deploy the configuration. The following command deploys the configuration to the environment you created in the previous step: ```bash lstk aws appconfig start-deployment \ --application-id 400c285 \ --environment-id 3695ea3 \ --deployment-strategy-id f2f2225 \ --configuration-profile-id 7d748f9 \ --configuration-version 1 \ --description "My application deployment" ``` ```bash title="Output" { "ApplicationId": "400c285", "EnvironmentId": "3695ea3", "DeploymentStrategyId": "f2f2225", "ConfigurationProfileId": "7d748f9", "DeploymentNumber": 1, "ConfigurationName": "my-app-config", "ConfigurationLocationUri": "hosted", "ConfigurationVersion": "1", "Description": "My application deployment", "DeploymentDurationInMinutes": 0, "GrowthFactor": 1.0, "State": "BAKING", "EventLog": [ { "EventType": "DEPLOYMENT_STARTED", "TriggeredBy": "USER", "Description": "Deployment started", "OccurredAt": "2023-08-28T11:18:43.273250Z" } ], "PercentageComplete": 0.0, "StartedAt": "2023-08-28T11:18:43.273250Z", "AppliedExtensions": [] } ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing AppConfig applications. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resource Browser** section, and then clicking on **AppConfig** under the **Developer Tools** section. ![AppConfig Resource Browser](/images/aws/appconfig-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create new AppConfig applications**: Create new AppConfig applications by clicking **Create Application** and filling in the required details. - **View AppConfig applications**: View the list of AppConfig applications created in LocalStack by clicking on the application ID. - **Edit AppConfig applications**: Edit the configuration of an existing AppConfig application by clicking on the application ID and then clicking **Edit Application**. - **Delete AppConfig applications**: Delete an existing AppConfig application by selecting the application, followed by clicking **Actions** and then **Remove Selected**. ## API Coverage ## API Coverage (AppConfig Data) # Application Auto Scaling > Get started with Application Auto Scaling on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Application Auto Scaling is a centralized solution for managing automatic scaling by defining scaling policies based on specific metrics. Based on CPU utilization or request rates, it automatically adjusts capacity in response to changes in workload. With Application Auto Scaling, you can configure automatic scaling for services such as DynamoDB, ECS, Lambda, ElastiCache, and more. Auto scaling uses CloudWatch under the hood to configure scalable targets which a service namespace, resource ID, and scalable dimension can uniquely identify. LocalStack allows you to use the Application Auto Scaling APIs in your local environment to scale different resources based on scaling policies and scheduled scaling. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Application Auto Scaling's integration with LocalStack. ## Getting Started This guide is designed for users new to Application Auto Scaling and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how you can configure auto scaling to handle a heavy workload for your Lambda function. ### Create a Lambda Function To create a new Lambda function, create a new file called `index.js` with the following code: ```js showshowLineNumbers exports.handler = async (event, context) => { console.log('Hello from Lambda!'); return { statusCode: 200, body: 'Hello, World!' }; }; ``` Run the following command to create a new Lambda function using the [`CreateFunction`](https://docs.aws.amazon.com/cli/latest/reference/lambda/create-function.html) API: ```bash zip function.zip index.js lstk aws lambda create-function \ --function-name autoscaling-example \ --runtime nodejs18.x \ --zip-file fileb://function.zip \ --handler index.handler \ --role arn:aws:iam::000000000000:role/cool-stacklifter ``` ### Create a version and alias for your Lambda function Next, you can create a version for your Lambda function and publish an alias. We will use the [`PublishVersion`](https://docs.aws.amazon.com/cli/latest/reference/lambda/publish-version.html) and [`CreateAlias`](https://docs.aws.amazon.com/cli/latest/reference/lambda/create-alias.html) APIs for this. Run the following commands: ```bash lstk aws lambda publish-version --function-name autoscaling-example lstk aws lambda create-alias \ --function-name autoscaling-example \ --description "alias for blue version of function" \ --function-version 1 \ --name BLUE ``` ### Register the Lambda function as a scalable target To register the Lambda function as a scalable target, you can use the [`RegisterScalableTarget`](https://docs.aws.amazon.com/cli/latest/reference/application-autoscaling/register-scalable-target.html) API. We will specify the `--service-namespace` as `lambda`, `--scalable-dimension` as `lambda:function:ProvisionedConcurrency`, and `--resource-id` as `function:autoscaling-example:BLUE`. Run the following command to register the scalable target: ```bash lstk aws application-autoscaling register-scalable-target \ --service-namespace lambda \ --scalable-dimension lambda:function:ProvisionedConcurrency \ --resource-id function:autoscaling-example:BLUE \ --min-capacity 0 --max-capacity 0 ``` ### Setting up a scheduled action You can create a scheduled action that scales out by specifying the `--schedule` parameter with a recurring schedule specified as a cron job. Run the following command to create a scheduled action using the [`PutScheduledAction`](https://docs.aws.amazon.com/cli/latest/reference/application-autoscaling/put-scheduled-action.html) API: ```bash lstk aws application-autoscaling put-scheduled-action \ --service-namespace lambda \ --scalable-dimension lambda:function:ProvisionedConcurrency \ --resource-id function:autoscaling-example:BLUE \ --scheduled-action-name lambda-action \ --schedule "cron(*/2* ** *)" \ --scalable-target-action MinCapacity=1,MaxCapacity=5 ``` You can confirm if the scheduled action exists using [`DescribeScheduledActions`](https://docs.aws.amazon.com/cli/latest/reference/application-autoscaling/describe-scheduled-actions.html) API: ```bash lstk aws application-autoscaling describe-scheduled-actions \ --service-namespace lambda ``` ### Setting up a target tracking scaling policy You can now set up a target tracking scaling policy to scale based on current resource utilization. You can use the [`PutScalingPolicy`](https://docs.aws.amazon.com/cli/latest/reference/application-autoscaling/put-scaling-policy.html) API to create a target tracking scaling policy after ensuring that your predefined metric expects the target value. When metrics lack data due to minimal application load, Application Auto Scaling does not adjust capacity. Run the following command to create a target-tracking scaling policy: ```bash lstk aws application-autoscaling put-scaling-policy \ --service-namespace lambda \ --scalable-dimension lambda:function:ProvisionedConcurrency \ --resource-id function:events-example:BLUE \ --policy-name scaling-policy --policy-type TargetTrackingScaling \ --target-tracking-scaling-policy-configuration '{ "TargetValue": 50.0, "PredefinedMetricSpecification": { "PredefinedMetricType": "predefinedmetric" }}' ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing AppConfig applications. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resource Browser** section, and then clicking on **Application Auto Scaling** under the **App Integration** section. ![Application Auto Scaling Resource Browser](/images/aws/application-auto-scaling-resource-browser.png) The Resource Browser allows you to perform the following actions: * **Create scalable target**: Create a new scalable target by clicking **Create Scalable Target** and providing the required details. * **Filter services**: Filter services by service namespace to view only the services you are interested in, by choosing from the dropdown list. * **Delete**: Delete a scalable target by selecting the target, followed by clicking **Actions** and then **Remove Selected**. The following service namespaces are currently supported: * Elastic Container Service (ECS) * Elastic MapReduce (EMR) * Elastic Compute Cloud (EC2) * AppStream * Lambda * DynamoDB * RDS * Sagemaker * Kafka * Cassandra * Comprenhend * Custom Resource ## API Coverage # AppSync > Get started with AppSync on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction AWS AppSync is a fully managed API management service that connects applications to events, data, and AI models. LocalStack allows you to use the AppSync APIs in your local environment to connect your applications and services to data and events. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of AppSync's integration with LocalStack. This guide is designed for users new to **AppSync** in LocalStack, and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. LocalStack supports two primary ways to work with AppSync, GraphQL and Events API. Start your LocalStack container using your preferred method, then jump into the section that matches your use case: * [GraphQL API](#graphql-api): build query-based APIs with schema-first design. * [Events API](#events-api): build with publish/subscribe style, real-time messaging. ## GraphQL API Use schemas and resolvers to interact with data sources like DynamoDB. ### Getting Started Create serverless GraphQL APIs to query databases, microservices, and other APIs. AppSync allows you to define your data models and business logic using a declarative approach, and connect to various data sources, including other AWS services, relational databases, and custom data sources. This guide is designed for users new to AppSync and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create an AppSync API with a DynamoDB data source using the AWS CLI. #### 1. Create a DynamoDB table You can create a DynamoDB table using the [`CreateTable`](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_CreateTable.html) API. Execute the following command to create a table named `DynamoDBNotesTable` with a primary key named `NoteId`: ```bash lstk aws dynamodb create-table \ --table-name DynamoDBNotesTable \ --attribute-definitions AttributeName=NoteId,AttributeType=S \ --key-schema AttributeName=NoteId,KeyType=HASH \ --billing-mode PAY_PER_REQUEST ``` After the table is created, you can use the [`ListTables`](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_ListTables.html) API. Run the following command to list all tables in your running LocalStack container: ```bash lstk aws dynamodb list-tables ``` ```bash title="Output" { "TableNames": [ "DynamoDBNotesTable" ] } ``` #### 2. Create a GraphQL API You can create a GraphQL API using the [`CreateGraphqlApi`](https://docs.aws.amazon.com/appsync/latest/APIReference/API_CreateGraphqlApi.html) API. Execute the following command to create a GraphQL API named `NotesApi`: ```bash lstk aws appsync create-graphql-api \ --name NotesApi \ --authentication-type API_KEY ``` ```bash title="Output" { "graphqlApi": { "name": "NotesApi", "apiId": "014d18d0c2b149ee8b66f39173", "authenticationType": "API_KEY", "arn": "arn:aws:appsync:us-east-1:000000000000:apis/014d18d0c2b149ee8b66f39173", "uris": { "GRAPHQL": "http://localhost:4566/graphql/014d18d0c2b149ee8b66f39173", "REALTIME": "ws://localhost:4510/graphql/014d18d0c2b149ee8b66f39173" }, "tags": {}, "xrayEnabled": false } } ``` You can now create an API key for your GraphQL API using the [`CreateApiKey`](https://docs.aws.amazon.com/appsync/latest/APIReference/API_CreateApiKey.html) API. Execute the following command to create an API key for your GraphQL API: ```bash lstk aws appsync create-api-key \ --api-id 014d18d0c2b149ee8b66f39173 ``` ```bash title="Output" { "apiKey": { "id": "31d94a05", "expires": 1693551600 } } ``` #### 3. Create a GraphQL schema Create a file named `schema.graphql` with the following content: ```graphql showshowLineNumbers type Note { NoteId: ID! title: String content: String } type PaginatedNotes { notes: [Note!]! nextToken: String } type Query { allNotes(limit: Int, nextToken: String): PaginatedNotes! getNote(NoteId: ID!): Note } type Mutation { saveNote(NoteId: ID!, title: String!, content: String!): Note deleteNote(NoteId: ID!): Note } type Schema { query: Query mutation: Mutation } ``` You can start the schema creation process using the [`StartSchemaCreation`](https://docs.aws.amazon.com/appsync/latest/APIReference/API_StartSchemaCreation.html) API. Execute the following command to start the schema creation process: ```bash lstk aws appsync start-schema-creation \ --api-id 014d18d0c2b149ee8b66f39173 \ --definition file://schema.graphql ``` ```bash title="Output" { "status": "ACTIVE" } ``` #### 4. Create a data source You can create a data source using the [`CreateDataSource`](https://docs.aws.amazon.com/appsync/latest/APIReference/API_CreateDataSource.html) API. Execute the following command to create a data source named `DynamoDBNotesTable`: ```bash lstk aws appsync create-data-source \ --name AppSyncDB \ --api-id 014d18d0c2b149ee8b66f39173 \ --type AMAZON_DYNAMODB \ --dynamodb-config tableName=DynamoDBNotesTable,awsRegion=us-east-1 ``` ```bash title="Output" { "dataSource": { "dataSourceArn": "arn:aws:appsync:us-east-1:000000000000:apis/014d18d0c2b149ee8b66f39173/datasources/AppSyncDB", "name": "AppSyncDB", "type": "AMAZON_DYNAMODB", "dynamodbConfig": { "tableName": "DynamoDBNotesTable", "awsRegion": "us-east-1" } } } ``` #### 5. Create a resolver You can create a resolver using the [`CreateResolver`](https://github.com/localstack/docs/pull/782) API. You can create a custom `request-mapping-template.vtl` and `response-mapping-template.vtl` file to use as a mapping template to use for requests and responses respectively. Execute the following command to create a VTL resolver attached to the `PaginatedNotes.notes` field: ```bash lstk aws appsync create-resolver \ --api-id 014d18d0c2b149ee8b66f39173 \ --type Query \ --field PaginatedNotes.notes \ --data-source-name AppSyncDB \ --request-mapping-template file://request-mapping-template.vtl \ --response-mapping-template file://response-mapping-template.vtl ``` ### GraphQL resolvers LocalStack's AppSync offers support for both unit and pipeline resolvers, as detailed in the [AWS resolvers documentation](https://docs.aws.amazon.com/appsync/latest/devguide/resolver-components.html). Unit resolvers consist of request and response mapping templates, facilitating the transformation of requests to and from data sources. Pipeline resolvers, on the other hand, invoke AppSync functions that wraps the AppSync data sources. Unit resolvers are written in the Velocity templating language (VTL), while pipeline resolvers can be written in either VTL or JavaScript. ### WebSocket Subscriptions LocalStack supports real-time GraphQL subscriptions over WebSocket for AWS AppSync APIs. Clients can subscribe to mutation-triggered events and receive updates in real time via the AppSync WebSocket endpoint. LocalStack supports the GraphQL `@aws_subscribe` directive, and core subscription message flow. Use this feature to power live updates in apps such as chat, dashboards, or collaborative editors. There's no need to poll for changes. You can set up a GraphQL subscription in your schema, connect to the WebSocket endpoint, and receive live updates triggered by a mutation. #### 1. Extend Your GraphQL Schema First, ensure your schema includes a `subscription` type. Use the `@aws_subscribe` directive to link each subscription to a corresponding mutation. ```graphql type Message { id: ID! content: String! } type Mutation { postMessage(id: ID!, content: String!): Message! } type Subscription { onMessagePosted: Message @aws_subscribe(mutations: ["postMessage"]) } schema { query: Query mutation: Mutation subscription: Subscription } ``` #### 2. Connect to the WebSocket Endpoint LocalStack exposes a WebSocket endpoint for each GraphQL API: ```json "REALTIME": "ws://localhost:4510/graphql/" ``` Use a WebSocket client like `wscat` to connect: ```bash npm install -g wscat export API_ID= wscat \ -s "graphql-ws" \ -c "ws://localhost:4510/graphql/$API_ID" ``` #### 3. Initialize the WebSocket Connection After connecting, send a `connection_init` message: ```json { "type": "connection_init" } ``` You should receive a `connection_ack` in response. #### 4. Start a Subscription Use a `start` message to register your subscription: ```json { "id": "1", "type": "start", "payload": { "data": "{\"query\":\"subscription { onMessagePosted { id content } }\"}" } } ``` #### 5. Trigger the Subscription Now, trigger the matching mutation: ```bash curl -X POST http://${API_ID}.appsync-api.localhost.localstack.cloud:4566/graphql \ -H "content-type: application/json" \ -H "x-api-key: ${API_KEY}" \ -d '{"query": "mutation { postMessage(id: \"1\", content: \"Hello world!\") { id content } }" }' ``` You should see a live message from the WebSocket server like: ```json title="Output" { "type": "data", "id": "1", "payload": { "data": { "onMessagePosted": { "id": "1", "content": "Hello world!" } } } } ``` #### 6. Stop the Subscription To unregister, send a `stop` message: ```json { "type": "stop", "id": "1" } ``` You'll receive a `complete` message when successful. ### Supported WebSocket Message Types LocalStack supports the following WebSocket message types: | Type | Description | | ----------------- | ----------------------------------------------- | | `connection_init` | Client initiates connection | | `connection_ack` | Server acknowledges the connection | | `start` | Starts a subscription | | `start_ack` | Acknowledges a successful subscription | | `data` | Sends a message when a matching mutation occurs | | `stop` | Client unsubscribes from a subscription | | `complete` | Server confirms unsubscription | ### Configuring GraphQL endpoints There are three configurable strategies that govern how GraphQL API endpoints are created. The strategy can be configured via the `GRAPHQL_ENDPOINT_STRATEGY` environment variable. | Value | Format | Description | |----------|--------------------------------------------------------|-----------------------------------------------------------------------------------------------------| | `domain` | `.appsync-api.localhost.localstack.cloud:4566` | This strategy, slated to be the future default, uses the `localhost.localstack.cloud` domain to route to your localhost. | | `path` | `localhost:4566/appsync-api//graphql` | An alternative strategy that can be beneficial if you're unable to resolve LocalStack's `localhost` domain. | | `legacy` | `localhost:4566/graphql/` | This strategy represents the old endpoint format, which is currently the default but will eventually be phased out. | In addition to the GraphQL HTTP endpoint, each AppSync API also provides a WebSocket endpoint for GraphQL subscriptions. The WebSocket server runs on a separate port that may change between deployments. To ensure correct connectivity, always use the `REALTIME` URL returned in the `uris` field of the `create-graphql-api` or `get-graphql-api` responses. Example: ```json "uris": { "GRAPHQL": "http://localhost:4566/graphql/", "REALTIME": "ws://localhost:4510/graphql/" } ``` :::note The `REALTIME` endpoint will ignore the `GRAPHQL_ENDPOINT_STRATEGY` and always use the `legacy` strategy. ::: ### GraphQL Resource Browser The LocalStack Web Application provides a Resource Browser for managing AppSync APIs, Data Sources, Schema, Query, Types, Resolvers, Functions and API keys. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **AppSync** under the **App Integration** section. ![AppSync Resource Browser](/images/aws/appsync-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create API**: Create a new GraphQL API by clicking **Create API** and providing a name for the API, Authentication Type, and optional tags among other parameters. - **Edit API**: Click on the GraphQL API name and click **Edit API** to edit the GraphQL API, by updating the parameters before clicking **Submit**. - **Create Data Source**: Click on the GraphQL API name and click **Data Source**. Click on **Create Data Source** to create a new data source for the GraphQL API, by providing a name for the data source, data source type, and Service Role ARN before clicking **Submit**. - **Edit Data Source**: Click on the GraphQL API name and click **Data Source**. Click on the data source name and click **Edit Data Source** to edit the data source, by updating the parameters before clicking **Submit**. - **Create Types**: Click on the GraphQL API name and click **Types**. Click on **Create Type** to create a type definition, in GraphQL Schema Definition Language (SDL) format, before clicking **Submit**. - **Create API Key**: Click on the GraphQL API name and click **API Keys**. Click on **Create API Key** to create an API key for the GraphQL API, by providing a description for the API key and its expiration time before clicking **Submit**. - **View and edit Schema**: Click on the GraphQL API name and click **Schema**. You can view the GraphQL schema, and edit the GraphQL schema, in GraphQL Schema Definition Language (SDL) format, before clicking **Update**. - **Query**: Click on the GraphQL API name and click **Query**. You can query the GraphQL API by providing the GraphQL query and variables, including the operation and API key, before clicking **Execute**. - **Attach Resolver**: Click on the GraphQL API name and click **Resolvers**. Click on **Attach Resolver** to attach a resolver to a field, by providing the field name, data source name, Request Mapping Template, Response Mapping Template, among other parameters, before clicking **Submit**. - **Create Function**: Click on the GraphQL API name and click **Functions**. Click on **Create Function** to create a function, by providing a name for the function, data source name, and Function Version, Request Mapping Template, Response Mapping Template, among other parameters, before clicking **Submit**. ## Events API Enable sending real-time event data to subscribed clients. ### Getting Started Create a fully managed WebSocket API that lets you subscribe to channels and broadcast events to subscribers. #### 1. Create an Events API You can create an Events API using the [CreateApi](https://docs.aws.amazon.com/appsync/latest/APIReference/API_CreateApi.html) API. The following command will create an Events API with the name `my-api`. Note the `apiId`, `dns.REALTIME` and `dns.HTTP` in the outputs as it will be reused as ``, `` and `` for the remainder of this example. ```bash lstk aws appsync create-api \ --name my-api \ --event-config '{ "authProviders":[{"authType": "API_KEY"}], "connectionAuthModes":[{"authType": "API_KEY"}], "defaultSubscribeAuthModes":[{"authType": "API_KEY"}], "defaultPublishAuthModes":[{"authType": "API_KEY"}] }' ``` ```bash title="Output" { "api": { "apiId": "", "name": "my-api", "tags": {}, "dns": { "REALTIME": ".appsync-realtime-api.ca-west-1.amazonaws.com", "HTTP": ".appsync-api.ca-west-1.amazonaws.com" }, "apiArn": "arn:aws:appsync:us-east-1:000000000000:apis/", "created": "2025-07-14T11:38:32.594000-06:00", "eventConfig": {...} } } ``` #### 2. Create a `channelNamespace` You can create an `channelNamespace` using the [CreateChannelNamespace](https://docs.aws.amazon.com/appsync/latest/APIReference/API_CreateChannelNamespace.html) API. At least one `channelNamespace` is required in order to subscribe and publish to it. ```bash lstk aws appsync create-channel-namespace \ --api-id \ --name "default" ``` ```bash title="Output" { "channelNamespace": { "apiId": "", "name": "default", "tags": {}, "channelNamespaceArn": "arn:aws:appsync:us-east-1:000000000000:apis//channelNamespace/default", "created": "2025-07-14T11:39:44.554000-06:00", "lastModified": "2025-07-14T11:39:44.554000-06:00", "handlerConfigs": {} } } ``` #### 3. Create an API Key You can create an Api Key using the [CreateApiKey](https://docs.aws.amazon.com/appsync/latest/APIReference/API_CreateApiKey.html) API. The Api Key is used to authenticate Websocket connections and authorize `publish` and `subscribe` events. Note you Api Key id as it will be referenced as `` in this example. ```bash lstk aws appsync create-api-key --api-id ``` ```bash title="Output" { "apiKey": { "id": "", "expires": 1753117200, "deletes": 1758301200 } } ``` #### 4. Connect via WebSocket This example will use `wscat` to create a WebSocket connection. From there, we will create a subscription and show an example of how to publish via WebSocket and HTTP. If you do not have it installed, it can be installed as a `npm` global package with the following command. ```bash npm install -g wscat ``` Export to your shell all of the required configuration. Note that we are base64 encoding the headers in a url safe manner. ```bash export HTTP_DOMAIN= export REALTIME_DOMAIN= export API_KEY= export HEADER="{\"host\":\"$HTTP_DOMAIN\", \"x-api-key\":\"$API_KEY\"}" export HEADER=`echo "$HEADER" | base64 | tr '+/' '-_' | tr -d '\n='` ``` Using `wscat` you can now connect to your api. Note that we are sending the base64 encoded header map as subprotocol. ```bash wscat \ -s "header-$HEADER" \ -s "aws-appsync-event-ws" \ -c "ws://$REALTIME_DOMAIN/event/realtime" ``` ```bash title="Output" Connected (press CTRL+C to quit) > | ``` #### 5. Subscribe to a channel Once connected a `subscribe` event containing the following keys can be sent: `type`, `id`, `channel` and `authorization`. The connection `id` is later reused when receiving events from this subscription and when sending an `unsubscribe` event. ``` {"type": "subscribe", "channel": "/default/*", "id": "1234567890", "authorization": { "x-api-key": ""}} ``` ```bash title="Output" < {"id":"1234567890","type":"subscribe_success"} ``` #### 6. Publish via WebSocket Once subscribed, a `publish` event can be sent. Note that the subscription create to listen to every sub-channel of `default` via `/default/*` so publish event sent to `/default/race` or `/default/race/formulaOne` would be received by the subscriber, but events published to root `/default`, in this case would not be received. A `subscribe` event contains the following keys: `type`, `channel`, `events` and `authorization`. Note that `events` must be a list of JSON string. ``` {"type": "publish", "channel": "/default/race", "events": ["{\"team\": \"McLaren\"}"], "authorization": {"x-api-key": ""}, "id": "000"} ``` The `publish_success` event is received first followed by the published event ```bash title="Output" < {"id":"000","type":"publish_success","successful":[{"identifier":"7e77f331-bf1b-4e42-9c9e-6e7c34dcf7f2","index":0}],"failed":[]} < {"id":"1234567890","type":"data","event":"{\"team\": \"McLaren\"}"} ``` #### 7. Publish via HTTP Create a file named `publishEvent.json` with and paste within the following content. ```json { "channel": "default/channel", "events": [ "{\"event_1\":\"data_1\"}", "{\"event_2\":\"data_2\"}" ] } ``` In a separate terminal from your WebSocket connection, make a `POST` request containing a `publish` event can be sent to the http endpoint. ```bash curl -X POST "http://${HTTP_DOMAIN}/event" \ -d @publishEvent.json \ -H "x-api-key: ${API_KEY}" \ -H "content-type: application/json" ``` ```bash title="Output" { "failed": [], "successful": [ { "identifier": "1260cde1-a50c-410d-9eee-9932eb32511d", "index": 0 }, { "identifier": "5f4e1897-9295-4ac0-a6fd-185736e48744", "index": 1 } ] } ``` In your terminal with the WebSocket Connection, you can now see the 2 new events ```bash title="Output" < {"id":"1234567890","type":"data","event":"{\"event_1\":\"data_1\"}"} < {"id":"1234567890","type":"data","event":"{\"event_2\":\"data_2\"}"} ``` ### Event code handlers LocalStack supports configuring [code handlers](https://docs.aws.amazon.com/appsync/latest/eventapi/runtime-reference-overview.html) for your [channel namespace](#2-create-a-channelnamespace). Code handlers can be configured with or without [data sources](#data-sources). ### Configuring Event endpoints We are registering 2 type of endpoints that you can use to target you Events API. The examples represent the HTTP endpoint with `appsync-api` but the same is true for REALTIME endpoint with `appsync-realtime-api`. Note that unlike for GraphQL APIs, Events API endpoint is isn't the same as the API ID. | Value | Format | Description | |----------|--------------------------------------------------------|-----------------------------------------------------------------------------------------------------| | `domain` | `.appsync-api.localhost.localstack.cloud:4566` | This strategy, uses the `localhost.localstack.cloud` domain to route to your localhost. | | `path` | `localhost:4566/_aws/appsync-api/` | An alternative strategy that can be beneficial if you're unable to resolve LocalStack's `localhost` domain. | ### Events Resource Browser :::note Resource Browser support for Events APIs is not available yet. ::: ## Shared configurations ### Custom API IDs You can employ a pre-defined ID during the creation of AppSync APIs by utilizing the special tag `_custom_id_`. For example, the following command will create a GraphQL API with the ID `faceb00c`. `--tags` can also be passed when creating an Events API, and both the API id and the endpoint id will use the provided id. ```bash lstk aws appsync create-graphql-api \ --name my-api \ --authentication-type API_KEY \ --tags _custom_id_=faceb00c ``` ```bash title="Output" { "graphqlApi": { "name": "my-api", "apiId": "faceb00c", "authenticationType": "API_KEY", "arn": "arn:aws:appsync:us-east-1:000000000000:apis/my-api", "uris": { "GRAPHQL": "http://localhost:4566/graphql/faceb00c", "REALTIME": "ws://localhost:4510/graphql/faceb00c" }, "tags": { "_custom_id_": "faceb00c" } } } ``` ### Data sources LocalStack supports the following data source types types and services: | Resolver Type | Description | | ------------------------- | ---------------------------------------------------------------------- | | `AMAZON_DYNAMODB` | Provides access to DynamoDB tables. | | `RELATIONAL_DATABASE` | Provides access to RDS database tables. | | `AWS_LAMBDA` | Allows retrieval of data from Lambda function invocations. | | `HTTP` | Enables calling HTTP endpoints to fetch data. | | `NONE` (GraphQL API Only) | Used for pass-through resolver mapping templates returning input data. | ## Evaluation endpoints LocalStack supports code evaluation endpoints: [`EvaluateCode`](https://docs.aws.amazon.com/appsync/latest/APIReference/API_EvaluateCode.html) and [`EvaluateMappingTemplate`](https://docs.aws.amazon.com/appsync/latest/APIReference/API_EvaluateMappingTemplate.html). Code can be either passed in as a string, or from a file with the `file://` prefix for the `--template/--code` arguments. See the AWS documentation for [`evaluate-mapping-template`](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/appsync/evaluate-mapping-template.html) and [`evaluate-code`](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/appsync/evaluate-code.html) for more details. ### VTL template evaluation ```bash lstk aws appsync evaluate-mapping-template \ --template '$ctx.result' \ --context '{"result":"ok"}' ``` ```bash title="Output" { "evaluationResult": "ok", "logs": [] } ``` ### JavaScript code evaluation ```bash lstk aws appsync evaluate-code \ --runtime name=APPSYNC_JS,runtimeVersion=1.0.0 \ --function request \ --code 'export function request(ctx) { return ctx.result; } export function response(ctx) {}' \ --context '{"result": "ok"}' ``` ```bash title="Output" { "evaluationResult": "ok", "logs": [] } ``` ## Examples The following code snippets and sample applications provide practical examples of how to use AppSync in LocalStack for various use cases: - [AppSync GraphQL APIs for DynamoDB and RDS Aurora PostgreSQL](https://github.com/localstack/appsync-graphql-api-sample) ## API Coverage # Athena > Get started with Athena on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction Athena is an interactive query service provided by Amazon Web Services (AWS) that enables you to analyze data stored in S3 using standard SQL queries. Athena allows users to create ad-hoc queries to perform data analysis, filter, aggregate, and join datasets stored in S3. It supports various file formats, such as JSON, Parquet, and CSV, making it compatible with a wide range of data sources. LocalStack allows you to configure the Athena APIs with a Hive metastore that can connect to the S3 API and query your data directly in your local environment. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Athena's integration with LocalStack. ## Getting started This guide is designed for users new to Athena and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create an Athena table and run a query against it in addition to reading the results with the AWS CLI. :::note To utilize the Athena API, LocalStack will download additional dependencies. This involves getting a Docker image of around 1.5GB, containing Presto, Hive, and other tools. These components are retrieved automatically when you initiate the service. To ensure a smooth initial setup, ensure you're connected to a stable internet connection while fetching these components for the first time. ::: ### Create an S3 bucket You can create an S3 bucket using the [`mb`](https://docs.aws.amazon.com/cli/latest/reference/s3/mb.html) command. Run the following command to create a bucket named `athena-bucket`: ```bash lstk aws s3 mb s3://athena-bucket ``` You can create some sample data using the following commands: ```bash echo "Name,Service" > data.csv echo "LocalStack,Athena" >> data.csv ``` You can upload the data to your bucket using the [`cp`](https://docs.aws.amazon.com/cli/latest/reference/s3/cp.html) command: ```bash lstk aws s3 cp data.csv s3://athena-bucket/data/ ``` ### Create an Athena table You can create an Athena table using the [`CreateTable`](https://docs.aws.amazon.com/athena/latest/APIReference/API_CreateTable.html) API Run the following command to create a table named `athena_table`: ```bash lstk aws athena start-query-execution \ --query-string "create external table tbl01 (name STRING, surname STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LOCATION 's3://athena-bucket/data/';" --result-configuration "OutputLocation=s3://athena-bucket/output/" ``` ```bash title="Output" { "QueryExecutionId": "593acab7" } ``` You can retrieve information about the query execution using the [`GetQueryExecution`](https://docs.aws.amazon.com/athena/latest/APIReference/API_GetQueryExecution.html) API. Run the following command: ```bash lstk aws athena get-query-execution --query-execution-id 593acab7 ``` Replace `593acab7` with the `QueryExecutionId` returned by the [`StartQueryExecution`](https://docs.aws.amazon.com/athena/latest/APIReference/API_StartQueryExecution.html) API. ### Get output of the query You can get the output of the query using the [`GetQueryResults`](https://docs.aws.amazon.com/athena/latest/APIReference/API_GetQueryResults.html) API. Run the following command: ```bash lstk aws athena get-query-results --query-execution-id 593acab7 ``` You can now read the data from the `tbl01` table and retrieve the data from S3 that was mentioned in your table creation statement. Run the following command: ```bash lstk aws athena start-query-execution \ --query-string "select * from tbl01;" --result-configuration "OutputLocation=s3://athena-bucket/output/" ``` You can retrieve the execution details similarly using the [`GetQueryExecution`](https://docs.aws.amazon.com/athena/latest/APIReference/API_GetQueryExecution.html) API using the `QueryExecutionId` returned by the previous step. You can copy the `ResultConfiguration` from the output and use it to retrieve the results of the query. Run the following command: ```bash lstk aws s3 cp s3://athena-bucket/output/593acab7.csv . cat 593acab7.csv ``` Replace `593acab7.csv` with the path to the file that was present in the `ResultConfiguration` of the previous step. You can also use the [`GetQueryResults`](https://docs.aws.amazon.com/athena/latest/APIReference/API_GetQueryResults.html) API to retrieve the results of the query. ## Delta Lake Tables LocalStack Athena supports [Delta Lake](https://delta.io), an open-source storage framework that extends Parquet data files with a file-based transaction log for ACID transactions and scalable metadata handling. To illustrate this feature, we take a sample published in the [AWS blog](https://aws.amazon.com/blogs/big-data/crawl-delta-lake-tables-using-aws-glue-crawlers). The Delta Lake files used in this sample are available in a public S3 bucket under `s3://aws-bigdata-blog/artifacts/delta-lake-crawler/sample_delta_table`. For your convenience, we have prepared the test files in a downloadable ZIP file [here](https://localstack-assets.s3.amazonaws.com/aws-sample-athena-delta-lake.zip). We start by downloading and extracting this ZIP file: ```bash mkdir /tmp/delta-lake-sample; cd /tmp/delta-lake-sample wget https://localstack-assets.s3.amazonaws.com/aws-sample-athena-delta-lake.zip unzip aws-sample-athena-delta-lake.zip; rm aws-sample-athena-delta-lake.zip ``` We can then create an S3 bucket in LocalStack using the [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command line, and upload the files to the bucket: ```bash lstk aws s3 mb s3://test lstk aws s3 sync /tmp/delta-lake-sample s3://test ``` Next, we create the table definitions in Athena: ```bash lstk aws athena start-query-execution \ --query-string "CREATE EXTERNAL TABLE test (product_id string, product_name string, \ price bigint, currency string, category string, updated_at double) \ LOCATION 's3://test/' TBLPROPERTIES ('table_type'='DELTA')" ``` Please note that this query may take some time to finish executing. You can observe the output in the LocalStack container (ideally with `DEBUG=1` enabled) to follow the steps of the query execution. Finally, we can now run a `SELECT` query to extract data from the Delta Lake table we've just created. To query Delta Lake tables, specify `Catalog=deltalake` in the `QueryExecutionContext`: ```bash queryId=$(lstk aws athena start-query-execution \ --query-string "SELECT * FROM test" \ --query-execution-context "Database=default,Catalog=deltalake" \ --result-configuration "OutputLocation=s3://test/output/" | jq -r .QueryExecutionId) lstk aws athena get-query-results --query-execution-id $queryId ``` The query should yield a result similar to the output below: ```bash title="Output" ... "Rows": [ { "Data": [ { "VarCharValue": "product_id" }, { "VarCharValue": "product_name" }, { "VarCharValue": "price" }, { "VarCharValue": "currency" }, { "VarCharValue": "category" }, { "VarCharValue": "updated_at" } ] }, { "Data": [ { "VarCharValue": "00005" }, { "VarCharValue": "USB charger" }, { "VarCharValue": "50" }, { "VarCharValue": "INR" }, { "VarCharValue": "Electronics" }, { "VarCharValue": "1653462374.9975588" } ] }, ... ... ``` ## Iceberg Tables The LocalStack Athena implementation also supports [Iceberg tables](https://docs.aws.amazon.com/athena/latest/ug/querying-iceberg-creating-tables.html). You can define an Iceberg table in Athena using the `CREATE TABLE` statement, as shown in the example below: ```sql CREATE TABLE mytable (c1 integer, c2 string, c3 double) LOCATION 's3://mybucket/prefix/' TBLPROPERTIES ( 'table_type' = 'ICEBERG' ) ``` To query Iceberg tables, specify `Catalog=iceberg` in the `QueryExecutionContext`: ```bash lstk aws athena start-query-execution \ --query-string "SELECT * FROM mytable" \ --query-execution-context "Database=default,Catalog=iceberg" \ --result-configuration "OutputLocation=s3://mybucket/output/" ``` Once the table has been created and data inserted into it, you can see the Iceberg metadata and data files being created in S3: ```bash s3://mybucket/_tmp.prefix/ s3://mybucket/prefix/data/00000-0-user1_20230212221600_cd8f8cbd-4dcc-4c3f-96a2-f08d4104d6fb-job_local1695603329_0001-00001.parquet s3://mybucket/prefix/data/00000-0-user1_20230212221606_eef1fd88-8ff1-467a-a15b-7a24be7bc52b-job_local1976884152_0002-00001.parquet s3://mybucket/prefix/metadata/00000-06706bea-e09d-4ff1-b366-353705634f3a.metadata.json s3://mybucket/prefix/metadata/00001-3df6a04e-070d-447c-a213-644fe6633759.metadata.json s3://mybucket/prefix/metadata/00002-5dcd5d07-a9ed-4757-a6bc-9e87fcd671d5.metadata.json s3://mybucket/prefix/metadata/2f8d3628-bb13-4081-b5a9-30f2e81b7226-m0.avro s3://mybucket/prefix/metadata/70de28f7-6507-44ae-b505-618d734174b9-m0.avro s3://mybucket/prefix/metadata/snap-8425363304532374388-1-70de28f7-6507-44ae-b505-618d734174b9.avro s3://mybucket/prefix/metadata/snap-9068645333036463050-1-2f8d3628-bb13-4081-b5a9-30f2e81b7226.avro s3://mybucket/prefix/temp/ ``` ## S3 Tables LocalStack Athena can query [S3 Tables](/aws/services/s3tables/) through Glue federated catalogs, mirroring the AWS workflow that bridges S3 Tables, Glue, and Athena into a single query path. This lets you point Athena at a table bucket and run SQL against the Iceberg tables it manages without copying data into a separate warehouse. The flow is the same as on AWS: 1. Create a table bucket and namespaces in S3 Tables. 2. Register a Glue federated catalog (conventionally named `s3tablescatalog`) that delegates metadata to S3 Tables. 3. Register an Athena data catalog with `Type=GLUE` whose `catalog-id` parameter points to a specific table bucket via the federated catalog (`s3tablescatalog/`). 4. Reference the Athena data catalog in `QueryExecutionContext` when running queries. ### Create S3 Tables resources Create a table bucket and a namespace in S3 Tables. The bucket holds your Iceberg tables and the namespace organizes them. ```bash lstk aws s3tables create-table-bucket --name athena-doc-bucket ``` ```bash title="Output" { "arn": "arn:aws:s3tables:us-east-1:000000000000:bucket/athena-doc-bucket" } ``` ```bash lstk aws s3tables create-namespace \ --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/athena-doc-bucket \ --namespace sales ``` ```bash title="Output" { "tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/athena-doc-bucket", "namespace": [ "sales" ] } ``` ### Register a Glue federated catalog Register a Glue catalog that federates to S3 Tables using the [`CreateCatalog`](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-catalog-Catalogs.html#aws-glue-api-catalog-CreateCatalog) API. The catalog name `s3tablescatalog` matches the AWS convention used by Athena, EMR, and Redshift. ```bash lstk aws glue create-catalog \ --name s3tablescatalog \ --catalog-input '{ "FederatedCatalog": { "Identifier": "arn:aws:s3tables:us-east-1:000000000000:bucket/*", "ConnectionName": "aws:s3tables" } }' ``` You can verify the federated catalog with: ```bash lstk aws glue get-catalogs ``` ### Register an Athena data catalog Register an Athena data catalog that points at a specific table bucket using the [`CreateDataCatalog`](https://docs.aws.amazon.com/athena/latest/APIReference/API_CreateDataCatalog.html) API. The `catalog-id` parameter follows the format `s3tablescatalog/` so that Athena routes queries through the federated catalog path. ```bash lstk aws athena create-data-catalog \ --name s3tables-catalog \ --type GLUE \ --parameters "catalog-id=s3tablescatalog/athena-doc-bucket" ``` Confirm the data catalog status: ```bash lstk aws athena get-data-catalog --name s3tables-catalog ``` ```bash title="Output" { "DataCatalog": { "Name": "s3tables-catalog", "Type": "GLUE", "Parameters": { "catalog-id": "s3tablescatalog/athena-doc-bucket" }, "Status": "CREATE_COMPLETE" } } ``` ### Resolve metadata through the catalog Once the data catalog is registered, Athena resolves S3 Tables namespaces as databases and S3 Tables as tables. List the databases exposed by the federated catalog: ```bash lstk aws athena list-databases --catalog-name s3tables-catalog ``` ```bash title="Output" { "DatabaseList": [ { "Name": "sales", "Parameters": { "createdBy": "000000000000", "ownerAccountId": "000000000000" } } ] } ``` You can also describe a single namespace with [`GetDatabase`](https://docs.aws.amazon.com/athena/latest/APIReference/API_GetDatabase.html): ```bash lstk aws athena get-database \ --catalog-name s3tables-catalog \ --database-name sales ``` ### Run queries via the federated catalog To query S3 Tables data from Athena, reference the data catalog name in the `QueryExecutionContext`. The `Catalog` field maps to the Athena data catalog you registered, and `Database` maps to the S3 Tables namespace: ```bash lstk aws athena start-query-execution \ --query-string "CREATE TABLE orders (id int, customer string, amount double) TBLPROPERTIES ('table_type' = 'ICEBERG')" \ --query-execution-context "Catalog=s3tables-catalog,Database=sales" \ --result-configuration "OutputLocation=s3://athena-doc-output/results/" ``` Insert and read data using the same `QueryExecutionContext`: ```bash lstk aws athena start-query-execution \ --query-string "INSERT INTO orders VALUES (1, 'alice', 100.0), (2, 'bob', 250.5)" \ --query-execution-context "Catalog=s3tables-catalog,Database=sales" \ --result-configuration "OutputLocation=s3://athena-doc-output/results/" ``` ```bash lstk aws athena start-query-execution \ --query-string "SELECT * FROM orders ORDER BY id" \ --query-execution-context "Catalog=s3tables-catalog,Database=sales" \ --result-configuration "OutputLocation=s3://athena-doc-output/results/" ``` You can also use the catalog-id reference (`s3tablescatalog/`) directly in `QueryExecutionContext.Catalog` if you prefer not to register a named Athena data catalog. :::note Query execution against the federated catalog routes through Trino's Iceberg connector inside the LocalStack bigdata container. The first query may take several minutes while LocalStack downloads and starts the bigdata dependencies. Subsequent queries reuse the running services. ::: ## Client configuration You can configure the Athena service in LocalStack with various clients, such as [PyAthena](https://github.com/laughingman7743/PyAthena/), [awswrangler](https://github.com/aws/aws-sdk-pandas), among others! Here are small snippets to get you started: ```python from pyathena import connect conn = connect( s3_staging_dir="s3://s3-results-bucket/output/", region_name="us-east-1", endpoint_url="http://localhost:4566", ) cursor = conn.cursor() cursor.execute("SELECT 1,2,3 AS test") print(cursor.fetchall()) ``` ```python showshowLineNumbers import awswrangler as wr import pandas as pd ENDPOINT = "http://localhost:4566" DATABASE = "testdb" S3_BUCKET = "s3://s3-results-bucket/output/" wr.config.athena_endpoint_url = ENDPOINT wr.config.glue_endpoint_url = ENDPOINT wr.config.s3_endpoint_url = ENDPOINT wr.catalog.create_database(DATABASE) df = wr.athena.read_sql_query("SELECT 1 AS col1, 2 AS col2, 3 AS col3", database=DATABASE) print(df) ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for Athena query execution, writing SQL queries, and visualizing query results. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **Athena** under the **Analytics** section. ![Athena Resource Browser](/images/aws/athena-resource-browser.png) The Resource Browser allows you to perform the following actions: - **View Databases**: View the databases available in your Athena instance by clicking on the **Databases** tab. - **View Catalogs**: View the catalogs available in your Athena instance by clicking on the **Catalogs** tab. - **Edit Catalogs**: Edit the catalogs available in your Athena instance by clicking on the **Catalog name**, editing the catalog, and then clicking on the **Submit** button. - **Create Catalogs**: Create a new catalog by clicking on the **Create Catalog** button, entering the catalog details, and then clicking on the **Submit** button. - **Run SQL Queries**: Run SQL queries by clicking on the **SQL** button, entering the query, and then clicking on the **Execute** button. ## Examples The following code snippets and sample applications provide practical examples of how to use Athena in LocalStack for various use cases: - [Query data in S3 Bucket with Amazon Athena, Glue Catalog & CloudFormation](https://github.com/localstack/query-data-s3-athena-glue-sample) ## API Coverage # Auto Scaling > Get started with Auto Scaling on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Auto Scaling helps you maintain application availability and allows you to automatically add or remove EC2 instances according to the demand. You can use Auto Scaling to ensure that you are running your desired number of instances. LocalStack allows you to use the Auto Scaling APIs locally to create and manage Auto Scaling groups locally. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Auto Scaling's integration with LocalStack. ## Getting started This guide is designed for users new to Auto Scaling and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how you can create a launch template, an Auto Scaling group, and attach an instance to the Auto Scaling group using the AWS CLI. ### Create a launch template You can create a launch template that defines the launch configuration for the instances in the Auto Scaling group using the [`CreateLaunchTemplate`](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateLaunchTemplate.html) API. Run the following command to create a launch template: ```bash lstk aws ec2 create-launch-template \ --launch-template-name my-template-for-auto-scaling \ --version-description version1 \ --launch-template-data '{"ImageId":"ami-ff0fea8310f3","InstanceType":"t2.micro"}' ``` ```bash title="Output" { "LaunchTemplate": { "LaunchTemplateId": "lt-5ccdf1a84f178ba44", "LaunchTemplateName": "my-template-for-auto-scaling", "CreateTime": "2024-07-12T07:59:08+00:00", "CreatedBy": "arn:aws:iam::000000000000:root", "DefaultVersionNumber": 1, "LatestVersionNumber": 1, "Tags": [] } } ``` ### Create an Auto Scaling group Before creating an Auto Scaling group, you need to fetch the subnet ID. Run the following command to describe the subnets: ```bash lstk aws ec2 describe-subnets --output text --query Subnets[0].SubnetId ``` Copy the subnet ID from the output and use it to create the Auto Scaling group. Run the following command to create an Auto Scaling group using the [`CreateAutoScalingGroup`](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_CreateAutoScalingGroup.html) API: ```bash lstk aws autoscaling create-auto-scaling-group \ --auto-scaling-group-name my-asg \ --launch-template LaunchTemplateId=lt-5ccdf1a84f178ba44 \ --min-size 1 \ --max-size 5 \ --vpc-zone-identifier 'subnet-d4d16268' ``` ### Describe the Auto Scaling group You can describe the Auto Scaling group using the [`DescribeAutoScalingGroups`](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_DescribeAutoScalingGroups.html) API. Run the following command to describe the Auto Scaling group: ```bash lstk aws autoscaling describe-auto-scaling-groups ``` ```bash title="Output" { "AutoScalingGroups": [ { "AutoScalingGroupName": "my-asg", "AutoScalingGroupARN": "arn:aws:autoscaling:us-east-1:000000000000:autoScalingGroup:74b4ffac-4588-4a7c-86b1-9fa992f49c8e:autoScalingGroupName/my-asg", "LaunchTemplate": { "LaunchTemplateId": "lt-5ccdf1a84f178ba44", "LaunchTemplateName": "my-template-for-auto-scaling" }, "MinSize": 1, "MaxSize": 5, ... "Instances": [ { "InstanceId": "i-fc01551d496fc363f", "InstanceType": "t2.micro", "AvailabilityZone": "us-east-1a", ... } ], ... "TerminationPolicies": [ "Default" ], ... "CapacityRebalance": false } ] } ``` ### Attach an instance to the Auto Scaling group You can attach an instance to the Auto Scaling group using the [`AttachInstances`](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_AttachInstances.html) API. Before that, create an EC2 instance using the [`RunInstances`](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html) API. Run the following command to create an EC2 instance locally: ```bash lstk aws ec2 run-instances \ --image-id ami-ff0fea8310f3 --count 1 ``` Fetch the instance ID from the output and use it to attach the instance to the Auto Scaling group. Run the following command to attach the instance to the Auto Scaling group: ```bash lstk aws autoscaling attach-instances \ --instance-ids i-0d678c4ecf6018dde \ --auto-scaling-group-name my-asg ``` Replace `i-0d678c4ecf6018dde` with the instance ID that you fetched from the output. ## Current Limitations LocalStack does not support the `docker` [VM manager for EC2](/aws/services/ec2/#vm-managers). It only works with the `mock` VM manager. ## API Coverage # Backup > Get started with Backup on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Backup is a centralized backup service provided by Amazon Web Services. It simplifies the process of backing up and restoring your data across various AWS services and resources. Backup supports a wide range of AWS resources, including Elastic Block Store (EBS) volumes, Relational Database Service (RDS) databases, DynamoDB tables, Elastic File System (EFS) file systems, and more. Backup enables you to set backup retention policies, allowing you to specify how long you want to retain your backup copies. LocalStack allows you to use the Backup APIs in your local environment to manage backup plans, create scheduled or on-demand backups of certain resource types. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Backup's integration with LocalStack. ## Getting started This guide is designed for users new to Backup and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a backup job and specify a set of resources to the backup plan name and backup rules with the AWS CLI. ### Create a backup vault You can create a backup vault which acts as a logical container where backups are stored using the [`CreateBackupVault`](https://docs.aws.amazon.com/aws-backup/latest/devguide/API_CreateBackupVault.html) API. Run the following command to create a backup vault named `my-vault`: ```bash lstk aws backup create-backup-vault \ --backup-vault-name primary ``` ```bash title="Output" { "BackupVaultName": "primary", "BackupVaultArn": "arn:aws:backup:us-east-1:000000000000:backup-vault:primary", "CreationDate": 1693286432.432258 } ``` ### Create a backup plan You can create a backup plan which specifies the backup vault to store the backups in and the schedule for creating backups. You can specify the backup plan in a `backup-plan.json` file: ```json showshowLineNumbers { "BackupPlanName": "testplan", "Rules": [{ "RuleName": "HalfDayBackups", "TargetBackupVaultName": "primary", "ScheduleExpression": "cron(0 5/12 ? * * *)", "StartWindowMinutes": 480, "CompletionWindowMinutes": 10080, "Lifecycle": { "DeleteAfterDays": 30 }, "CopyActions": [{ "DestinationBackupVaultArn": "arn:aws:backup:us-east-1:000000000000:backup-vault:secondary", "Lifecycle": { "DeleteAfterDays": 30 } }] }] } ``` You can use the [`CreateBackupPlan`](https://docs.aws.amazon.com/aws-backup/latest/devguide/API_CreateBackupPlan.html) API to create a backup plan. Run the following command to create a backup plan: ```bash lstk aws backup create-backup-plan \ --backup-plan file://backup-plan.json ``` ```bash title="Output" { "BackupPlanId": "9337aba3", "BackupPlanArn": "arn:aws:backup:us-east-1:000000000000:backup-plan:testplan", "CreationDate": 1693286644.0, "VersionId": "9dc2cb60" } ``` ### Create a backup selection You can create a backup selection which specifies the resources to backup and the backup plan to associate with. You can specify the backup selection in a `backup-selection.json` file: ```json showshowLineNumbers { "SelectionName": "Myselection", "IamRoleArn": "arn:aws:iam::000000000000:role/service-role/AWSBackupDefaultServiceRole", "Resources": ["arn:aws:ec2:us-east-1:000000000000:volume/vol-0abcdef1234"], "ListOfTags": [{ "ConditionType": "STRINGEQUALS", "ConditionKey": "backup", "ConditionValue": "yes" }] } ``` You can use the [`CreateBackupSelection`](https://docs.aws.amazon.com/aws-backup/latest/devguide/API_CreateBackupSelection.html) API to create a backup selection. Run the following command to create a backup selection: ```bash lstk aws backup create-backup-selection \ --backup-plan-id 9337aba3 \ --backup-selection file://backup-plan-resources.json ``` Replace the `--backup-plan-id` value with the `BackupPlanId` value from the output of the previous command. ```bash title="Output" { "SelectionId": "91ce25f8", "BackupPlanId": "9337aba3", "CreationDate": 1693287607.209043 } ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing backup plans and vaults. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **Backup** under the **Storage** section. ![Backup Resource Browser](/images/aws/backup-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Backup Plan**: Create a backup plan by clicking the **Create** button in the **Backup Plans** tab and specifying the backup plan details, including the plan name, rules, backup setting, and more in the modal dialog. - **Create Backup Vault**: Create a backup vault by clicking the **Create** button in the **Backup Vault** tab and specifying the vault name, tags, and other parameters in the modal dialog. - **Create Backup**: Create a backup by clicking the **Backup Vault** and then clicking the **Actions** button followed by clicking the **Create Backup** button in the modal dialog. Specify the backup name, backup vault, and other parameters in the modal dialog. - **Assign Resources**: Click the backup plan and then click the **Actions** button followed by clicking the **Assign Resources** button in the modal dialog. Specify the backup plan ID and resources to assign in the modal dialog, and click **Submit** to assign the resources to the backup plan. - **Delete Vault**: Delete a backup vault by clicking the **Backup Vault** or selecting multiple vaults. Click the **Actions** button followed by clicking the **Delete Vault** button or **Remove Selected** to delete an individual vault or multiple vaults respectively in the modal dialog. - **Delete Backup Plan**: Delete a backup plan by clicking the **Backup Plan** or selecting multiple plans. Click the **Actions** button followed by clicking the **Delete Backup Plan** button or **Remove Selected** to delete an individual plan or multiple plans respectively in the modal dialog. ## API Coverage # Batch > Get started with Batch on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Batch is a cloud-based service provided by Amazon Web Services (AWS) that simplifies the process of running batch computing workloads on the AWS cloud infrastructure. Batch allows you to efficiently process large volumes of data and run batch jobs without the need to manage and provision underlying compute resources. Under the hood, the local Docker engine is used to run the containers that simulate your Batch jobs. LocalStack allows you to use the Batch APIs to automate and scale computational tasks in your local environment while handling batch workloads. Batch jobs are executed using the ECS runtime, allowing for support of managed compute environments and improved service compatibility. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Batch integration with LocalStack. ## Getting started This guide is designed for users new to AWS Batch and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how you create and run a Batch job by following these steps: 1. Creating a service role for the compute environment. 2. Creating the compute environment. 3. Creating a job queue using the compute environment. 4. Creating a job definition. 5. Submitting a job to the job queue. ### Create a service role You can create a role using the [`CreateRole`](https://docs.aws.amazon.com/cli/latest/reference/iam/create-role.html) API. LocalStack requires the role to exist with a valid trust policy. When [enforcing IAM policies](/aws/developer-tools/security-testing/iam-policy-enforcement), ensure that the policy is valid and the role is properly attached. Run the following command to create a role for ECS task execution: ```bash lstk aws iam create-role \ --role-name myrole \ --assume-role-policy-document '{ "Version": "2025-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "ecs-tasks.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }' ``` Then attach the ECS task execution policy: ```bash lstk aws iam attach-role-policy \ --role-name myrole \ --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy ``` ### Create the compute environment You can use the [`CreateComputeEnvironment`](https://docs.aws.amazon.com/cli/latest/reference/batch/create-compute-environment.html) API to create a compute environment. Run the following command using the role ARN above (arn:aws:iam::000000000000:role/myrole) to create a managed compute environment with FARGATE: ```bash lstk aws batch create-compute-environment \ --compute-environment-name myenv \ --type MANAGED \ --state ENABLED \ --compute-resources type=FARGATE,maxvCpus=128,subnets=subnet-12345678,securityGroupIds=sg-12345678 \ --service-role arn:aws:iam::000000000000:role/myrole ``` :::note While networking resources such as subnets and security groups are required as input, LocalStack does not create real cloud infrastructure. These values must still be present for the compute environment to be created. ::: ### Create a job queue You can fetch the ARN using the [`DescribeComputeEnvironments`](https://docs.aws.amazon.com/cli/latest/reference/batch/describe-compute-environments.html) API. Run the following command to fetch the ARN of the compute environment: ```bash lstk aws batch describe-compute-environments --compute-environments myenv ``` ```bash title="Output" { "computeEnvironments": [ { "computeEnvironmentName": "myenv", "computeEnvironmentArn": "arn:aws:batch:us-east-1:000000000000:compute-environment/myenv", "ecsClusterArn": "arn:aws:ecs:us-east-1:000000000000:cluster/OnDemand_Batch_abc123", "type": "MANAGED", "status": "VALID", "statusReason": "Compute environment is available", "serviceRole": "arn:aws:iam::000000000000:role/myrole" } ] } ``` You can use the ARN to create the job queue using [`CreateJobQueue`](https://docs.aws.amazon.com/cli/latest/reference/batch/create-job-queue.html) API. Run the following command to create the job queue: ```bash lstk aws batch create-job-queue \ --job-queue-name myqueue \ --priority 1 \ --compute-environment-order order=0,computeEnvironment=arn:aws:batch:us-east-1:000000000000:compute-environment/myenv \ --state ENABLED ``` ### Create a job definition Now, you can define what occurs during a job run. In this example, you can execute the 'busybox' container from DockerHub and initiate the command: 'sleep 30'. It's important to note you can override this command when submitting the job. Run the following command to create the job definition using the [`RegisterJobDefinition`](https://docs.aws.amazon.com/cli/latest/reference/batch/register-job-definition.html) API: ```bash lstk aws batch register-job-definition \ --job-definition-name myjobdefn \ --type container \ --platform-capabilities FARGATE \ --container-properties '{ "image": "busybox", "resourceRequirements": [ {"type": "VCPU", "value": "0.25"}, {"type": "MEMORY", "value": "512"} ], "command": ["sleep", "30"], "networkConfiguration": { "assignPublicIp": "ENABLED" }, "executionRoleArn": "arn:aws:iam::000000000000:role/myrole" }' ``` If you want to pass arguments to the command as [parameters](https://docs.aws.amazon.com/batch/latest/userguide/job_definition_parameters.html#parameters), you can use the `Ref::` declaration to set placeholders for parameter substitution. This allows the dynamic passing of values at runtime for specific job definitions. ```bash lstk aws batch register-job-definition \ --job-definition-name myjobdefn \ --type container \ --parameters '{"time":"10"}' \ --platform-capabilities FARGATE \ --container-properties '{ "image": "busybox", "resourceRequirements": [ {"type": "VCPU", "value": "0.25"}, {"type": "MEMORY", "value": "512"} ], "command": ["sleep", "Ref::time"], "networkConfiguration": { "assignPublicIp": "ENABLED" }, "executionRoleArn": "arn:aws:iam::000000000000:role/myrole" }' ``` ### Submit a job to the job queue You can now run a compute job. This command runs a job on the queue that you have set up previously, overriding the container command to run: `sh -c "sleep 5; pwd"`. This command simulates work being done in the container. Run the following command to submit a job to the job queue using the [`SubmitJob`](https://docs.aws.amazon.com/cli/latest/reference/batch/submit-job.html) API: ```bash lstk aws batch submit-job \ --job-name myjob \ --job-queue myqueue \ --job-definition myjobdefn \ --container-overrides '{"command":["sh", "-c", "sleep 5; pwd"]}' ``` ## Multi-node parallel jobs LocalStack supports [AWS Batch multi-node parallel (MNP) jobs](https://docs.aws.amazon.com/batch/latest/userguide/multi-node-parallel-jobs.html), which run a single job across a main node and one or more worker nodes. The main node starts first, and the workers follow once it is running. Each worker receives the main node's private IP so the nodes can communicate. MNP jobs run on EC2-backed compute environments only. Fargate is not supported. To run one, register a job definition with `--type multinode` and a `nodeProperties` object that sets the main node, the number of nodes, and a container per node range: ```bash lstk aws batch register-job-definition \ --job-definition-name mnp-jobdefn \ --type multinode \ --node-properties '{ "mainNode": 0, "numNodes": 2, "nodeRangeProperties": [ { "targetNodes": "0:1", "container": { "image": "busybox", "command": ["sh", "-c", "echo node $AWS_BATCH_JOB_NODE_INDEX; sleep 10"], "resourceRequirements": [ {"type": "MEMORY", "value": "512"}, {"type": "VCPU", "value": "1"} ] } } ] }' ``` Then submit it to an EC2-backed queue: ```bash lstk aws batch submit-job \ --job-name mnp-job \ --job-queue mnp-queue \ --job-definition mnp-jobdefn ``` The submitted job is the parent. Each node is addressable as a child job using the `#` notation, which you can inspect with `describe-jobs`: ```bash lstk aws batch describe-jobs --jobs "#0" "#1" ``` Each node also receives additional [environment variables](#environment-variables), such as `AWS_BATCH_JOB_NODE_INDEX` and `AWS_BATCH_JOB_MAIN_NODE_PRIVATE_IPV4_ADDRESS`, that let the nodes coordinate. ## Environment variables LocalStack injects a subset of the Batch environment variables into each job container: - `AWS_BATCH_CE_NAME` - `AWS_BATCH_JOB_ARRAY_INDEX` - `AWS_BATCH_JOB_ARRAY_SIZE` - `AWS_BATCH_JOB_ATTEMPT` - `AWS_BATCH_JOB_ID` - `AWS_BATCH_JQ_NAME` [Multi-node parallel jobs](#multi-node-parallel-jobs) receive the following additional variables on each node: - `AWS_BATCH_JOB_NODE_INDEX` — the index of the current node. - `AWS_BATCH_JOB_NUM_NODES` — the total number of nodes in the job. - `AWS_BATCH_JOB_MAIN_NODE_INDEX` — the index of the main node. - `AWS_BATCH_JOB_MAIN_NODE_PRIVATE_IPV4_ADDRESS` — the private IP of the main node, set on worker nodes so they can connect back to the main node. ## Current Limitations LocalStack simulates the execution of ECS-based AWS Batch jobs using the local ECS runtime. No real infrastructure is created or managed. Array jobs are supported in sequential mode only. The configuration variable `ECS_DOCKER_FLAGS` can be used to pass additional Docker flags to the container runtime. Setting `ECS_TASK_EXECUTOR=kubernetes` is supported as an alternative backend, though Kubernetes execution is experimental and may not support all features. ## API Coverage # Bedrock > Get started with Bedrock on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Bedrock is a fully managed service provided by Amazon Web Services (AWS) that makes foundation models from various LLM providers accessible via an API. LocalStack allows you to use the Bedrock APIs to test and develop AI-powered applications in your local environment. The supported APIs are available on the API coverage section for [Bedrock](#api-coverage) and [Bedrock Runtime](#api-coverage-bedrock-runtime), which provides information on the extent of Bedrock's integration with LocalStack. ## Getting started This guide is designed for users new to AWS Bedrock and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method with or without pre-warming the Bedrock engine. We will demonstrate how to use Bedrock by following these steps: 1. Listing available foundation models 2. Invoking a model for inference 3. Using the conversation API 4. Using batch processing ### Pre-warming the Bedrock engine The startup of the Bedrock engine can take some time. Per default, we only start it once you send a request to one of the `bedrock-runtime` APIs. However, if you want to start the engine when localstack starts to avoid long wait times on your first request you can set the flag `BEDROCK_PREWARM`. On startup, the `DEFAULT_BEDROCK_MODEL` is pulled from the Ollama library and loaded into memory. However, you can define an additional list of models in `BEDROCK_PULL_MODELS` to pull additional models when the Bedrock engine starts up. This way you avoid long wait times when switching between models on demand with requests. ### List available foundation models You can view all available foundation models using the [`ListFoundationModels`](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_ListFoundationModels.html) API. This will show you which models are available on AWS Bedrock. :::note The actual model that will be used for emulation will differ from the ones defined in this list. You can define the used model with `DEFAULT_BEDROCK_MODEL` ::: Run the following command: ```bash lstk aws bedrock list-foundation-models ``` ### Invoke a model You can use the [`InvokeModel`](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) API to send requests to a specific model. In this example, we selected the Llama 3 model to process a simple prompt. However, the actual model will be defined by the `DEFAULT_BEDROCK_MODEL` environment variable. Run the following command: ```bash lstk aws bedrock-runtime invoke-model \ --model-id "meta.llama3-8b-instruct-v1:0" \ --body '{ "prompt": "<|begin_of_text|><|start_header_id|>user<|end_header_id|>\nSay Hello!\n<|eot_id|>\n<|start_header_id|>assistant<|end_header_id|>", "max_gen_len": 2, "temperature": 0.9 }' --cli-binary-format raw-in-base64-out outfile.txt ``` The output will be available in the `outfile.txt`. ### Use the conversation API Bedrock provides a higher-level conversation API that makes it easier to maintain context in a chat-like interaction using the [`Converse`](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) API. You can specify both system prompts and user messages. Run the following command: ```bash lstk aws bedrock-runtime converse \ --model-id "meta.llama3-8b-instruct-v1:0" \ --messages '[{ "role": "user", "content": [{ "text": "Say Hello!" }] }]' \ --system '[{ "text": "You'\''re a chatbot that can only say '\''Hello!'\''" }]' ``` ### Model Invocation Batch Processing Bedrock offers the feature to handle large batches of model invocation requests defined in S3 buckets using the [`CreateModelInvocationJob`](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_CreateModelInvocationJob.html) API. First, you need to create a `JSONL` file named `batch_input.jsonl` that contains all your prompts: ```json {"prompt": "Tell me a quick fact about Vienna.", "max_tokens": 50, "temperature": 0.5} {"prompt": "Tell me a quick fact about Zurich.", "max_tokens": 50, "temperature": 0.5} {"prompt": "Tell me a quick fact about Las Vegas.", "max_tokens": 50, "temperature": 0.5} ``` Then, you need to define buckets for the input as well as the output and upload the file in the input bucket: ```bash lstk aws s3 mb s3://in-bucket lstk aws s3 cp batch_input.jsonl s3://in-bucket lstk aws s3 mb s3://out-bucket ``` Afterwards you can run the invocation job like this: ```bash lstk aws bedrock create-model-invocation-job \ --job-name "my-batch-job" \ --model-id "mistral.mistral-small-2402-v1:0" \ --role-arn "arn:aws:iam::123456789012:role/MyBatchInferenceRole" \ --input-data-config '{"s3InputDataConfig": {"s3Uri": "s3://in-bucket"}}' \ --output-data-config '{"s3OutputDataConfig": {"s3Uri": "s3://out-bucket"}}' ``` ```bash title="Output" { "jobArn": "arn:aws:bedrock:us-east-1:000000000000:model-invocation-job/12345678" } ``` The results will be at the S3 URL `s3://out-bucket/12345678/batch_input.jsonl.out` ## Available models LocalStack's Bedrock emulation supports models from the [Ollama Models library](https://ollama.com/search). To use a model, retrieve its ID from Ollama and set `DEFAULT_BEDROCK_MODEL` to that ID. LocalStack will pull the model from Ollama and use it for emulation. For example, to use the Mistral model, set the environment variable while starting LocalStack: ```bash LOCALSTACK_DEFAULT_BEDROCK_MODEL=mistral lstk start ``` You can also define models directly in the request, by setting the `model-id` parameter to `ollama.`. For example, if you want to access `deepseek-r1`, you can do it like this: ```bash lstk aws bedrock-runtime converse \ --model-id "ollama.deepseek-r1" \ --messages '[{ "role": "user", "content": [{ "text": "Say Hello!" }] }]' ``` ## Troubleshooting Users of Docker Desktop on macOS or Windows might run into the issue of Bedrock becoming unresponsive after some usage. A common reason for that is insufficient storage or memory space in the Docker Desktop VM. To resolve this issue you can increase those amounts directly in Docker Desktop or clean up unused artifacts with the Docker CLI like this ```bash docker system prune ``` You could also try to use a model with lower requirements. To achieve that you can search for models in the [Ollama Models library](https://ollama.com/search) with a low parameter count or smaller size. ## Limitations * At this point, we have only tested text-based models in LocalStack. Other models available with Ollama might also work, but are not officially supported by the Bedrock implementation. * Currently, GPU models are not supported by the LocalStack Bedrock implementation. ## API Coverage ## API Coverage (Bedrock Runtime) # Cost Explorer > Get started with Cost Explorer on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Cost Explorer is a service provided by Amazon Web Services (AWS) that enables you to visualize, analyze, and manage your AWS spending and usage. Cost Explorer offers options to filter and group data by dimensions such as service, region, instance type, and more. With Cost Explorer, you can forecast costs, track budget progress, and set up alerts to receive notifications when spending exceeds predefined thresholds. LocalStack allows you to use the Cost Explorer APIs in your local environment to create and manage cost category definition, cost anomaly monitors & subscriptions. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Cost Explorer's integration with LocalStack. ## Getting started This guide is designed for users new to Cost Explorer and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to mock the Cost Explorer APIs with the AWS CLI. ### Create a Cost Category definition You can create a Cost Category definition using the [`CreateCostCategoryDefinition`](https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/API_CreateCostCategoryDefinition.html)) API. The following example creates a Cost Category definition using an empty rule condition of type "REGULAR": ```bash lstk aws ce create-cost-category-definition --name test \ --rule-version "CostCategoryExpression.v1" --rules '[{"Value": "test", "Rule": {}, "Type": "REGULAR"}]' ``` ```bash title="Output" { "CostCategoryArn": "arn:aws:ce::000000000000:costcategory/test" } ``` You can describe the Cost Category definition using the [`DescribeCostCategoryDefinition`](https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/API_DescribeCostCategoryDefinition.html) API. Run the following command: ```bash lstk aws ce describe-cost-category-definition \ --cost-category-arn arn:aws:ce::000000000000:costcategory/test ``` ```bash title="Output" { "CostCategory": { "CostCategoryArn": "arn:aws:ce::000000000000:costcategory/test", "Name": "test", "RuleVersion": "CostCategoryExpression.v1", "Rules": [ { "Value": "test", "Rule": {}, "Type": "REGULAR" } ] } } ``` ### Create a cost anomaly subscription You can add an alert subscription to a cost anomaly detection monitor to define subscribers using the [`CreateAnomalySubscription`](https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/API_CreateAnomalySubscription.html) API. The following example creates a cost anomaly subscription: ```bash lstk aws ce create-anomaly-subscription --anomaly-subscription '{ "AccountId": "12345", "SubscriptionName": "sub1", "Frequency": "DAILY", "MonitorArnList": [], "Subscribers": [], "Threshold": 111 }' ``` ```bash title="Output" { "SubscriptionArn": "arn:aws:ce::000000000000:anomalysubscription/70644961" } ``` You can retrieve the cost anomaly subscriptions using the [`GetAnomalySubscriptions`](https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/API_GetAnomalySubscriptions.html) API. Run the following command: ```bash lstk aws ce get-anomaly-subscriptions ``` ```bash title="Output" { "AnomalySubscriptions": [ { "SubscriptionArn": "arn:aws:ce::000000000000:anomalysubscription/70644961", "AccountId": "12345", "MonitorArnList": [], "Subscribers": [], "Threshold": 111.0, "Frequency": "DAILY", "SubscriptionName": "sub1" } ] } ``` ### Create a cost anomaly monitor You can create a new cost anomaly detection subscription with the requested type and monitor specification using the [`CreateAnomalyMonitor`](https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/API_CreateAnomalyMonitor.html) API. The following example creates a cost anomaly monitor: ```bash lstk aws ce create-anomaly-monitor --anomaly-monitor '{ "MonitorName": "mon5463", "MonitorType": "DIMENSIONAL" }' ``` ```bash title="Output" { "MonitorArn": "arn:aws:ce::000000000000:anomalymonitor/22570ff3" } ``` You can retrieve the cost anomaly monitors using the [`GetAnomalyMonitors`](https://docs.aws.amazon.com/aws-cost-management/latest/APIReference/API_GetAnomalyMonitors.html) API. Run the following command: ```bash lstk aws ce get-anomaly-monitors ``` ```bash title="Output" { "AnomalyMonitors": [ { "MonitorArn": "arn:aws:ce::000000000000:anomalymonitor/22570ff3", "MonitorName": "mon5463", "MonitorType": "DIMENSIONAL" } ] } ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing cost category definitions for the Cost Explorer service. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the Resources section, and then clicking on **Cost Explorer** under the **Cloud Financial Management** section. ![Cost Explorer Resource Browser](/images/aws/cost-explorer-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Cost Category definition**: Create a new Cost Category definition by clicking on the **Create** button and providing the required details. - **View Cost Category definition**: View the details of a Cost Category definition by clicking on the Cost Category definition. - **Delete Cost Category definition**: Delete a Cost Category definition by selecting on the Cost Category definition, and then clicking on the **Actions** button followed by **Remove Selected**. ## Current Limitations LocalStack's Cost Explorer implementation cannot programmatically query your cost and usage data, or provide aggregated data such as total monthly costs or total daily usage. However, you can use the integrations to mock the Cost Explorer APIs and test your workflow locally. ## API Coverage # Cloud Control > Get started with Cloud Control on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Cloud Control API allows you to create, read, update, delete, and list (CRUD-L) resources in AWS. Cloud Control API provides a standardized interface provision and access these resources in a programmatic way. LocalStack allows you to use the Cloud Control API in your local environment to interact with your emulated resources. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Cloud Control's integration with LocalStack. ## Getting started This guide is designed for users new to Cloud Control and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to get and list resources using the Cloud Control API. ### List resources You can list resources using the [`ListResources`](https://docs.aws.amazon.com/cloudcontrolapi/latest/APIReference/API_ListResources.html) API. Create an S3 bucket using the following command: ```bash lstk aws s3 mb s3://my-bucket ``` List the resources using the following command: ```bash lstk aws cloudcontrol list-resources --type-name AWS::S3::Bucket ``` You should see the S3 bucket in the output. ```bash title="Output" { "ResourceDescriptions": [ { "Identifier": "my-bucket", "Properties": "{\"BucketName\":\"my-bucket\"}" } ], "TypeName": "AWS::S3::Bucket" } ``` ### Get a resource You can get a resource using the [`GetResource`](https://docs.aws.amazon.com/cloudcontrolapi/latest/APIReference/API_GetResource.html) API. ```bash lstk aws cloudcontrol get-resource --type-name AWS::S3::Bucket --identifier my-bucket ``` ```bash title="Output" { "TypeName": "AWS::S3::Bucket", "ResourceDescription": { "Identifier": "my-bucket", "Properties": "{\"BucketName\":\"my-bucket\"}" } } ``` ## Supported Resources The following resources are supported by Cloud Control: 1. [`AWS::ApiGateway::RestApi`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-apigateway-restapi.html) 2. [`AWS::CloudFormation::Stack`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudformation-stack.html) 3. [`AWS::DynamoDB::Table`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-dynamodb-table.html) 4. [`AWS::EC2::VPC`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-vpc.html) 5. [`AWS::Events::EventBus`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-events-eventbus.html) 6. [`AWS::IAM::Group`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-group.html) 7. [`AWS::IAM::Role`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-role.html) 8. [`AWS::IAM::User`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-user.html) 9. [`AWS::Lambda::Function`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-function.html) 10. [`AWS::S3::Bucket`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-s3-bucket.html) 11. [`AWS::SES::EmailIdentity`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ses-emailidentity.html) 12. [`AWS::SNS::Topic`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-sns-topic.html) 13. [`AWS::SQS::Queue`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-sqs-queue.html) 14. [`AWS::SSM::Parameter`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ssm-parameter.html) 15. [`AWS::SecretsManager::Secret`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-secretsmanager-secret.html) 16. [`AWS::StepFunctions::StateMachine`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-stepfunctions-statemachine.html) 17. [`AWS::CloudFront::Distribution`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudfront-distribution.html) 18. [`AWS::Pipes::Pipe`](https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-resource-pipes-pipe.html) ## API Coverage # CloudFormation > Get started with Cloudformation on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; import CloudFormationCoverage from "../../../../components/cloudformation-coverage/CloudFormationCoverage"; import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction :::note With LocalStack version 4.8.0 (and above) we've introduced a **new CloudFormation engine** with Change Sets at its core, which would allow proper update and rollback support in the near future. This includes internal changes that may affect existing stacks or deployment behavior. Most users will benefit from the new behavior automatically, but there are a few important notes to be aware of: - **Persistence is not backwards-compatible:** If you use persistent state, your stacks may not load correctly between the new and old engines. - **Default behavior has changed:** If your deployment logic depends on specific legacy quirks or unsupported update behavior, you may encounter issues. - **New features and improvements:** are now available under the new engine. If you encounter problems or regressions you can **revert to the legacy engine** by setting: ```bash PROVIDER_OVERRIDE_CLOUDFORMATION=engine-legacy ``` ::: :::note ## Upcoming Change in Handling Unsupported Resource Types In a future LocalStack release, the behavior of the CloudFormation engine will change when stacks contain **unsupported AWS resource types**. **Currently**, unsupported resources are silently ignored or mocked so that the rest of the stack can proceed. **With the upcoming change**, CloudFormation will instead **fail the deployment** if the template includes unsupported resource types. To keep the current behavior and prepare for this breaking change ahead, you can enable it manually: ```bash CFN_IGNORE_UNSUPPORTED_RESOURCE_TYPES=1 ``` ::: CloudFormation is a service provided by Amazon Web Services (AWS) that allows you to define and provision infrastructure as code. It enables you to create, update, and manage resources in a repeatable and automated manner using declarative templates. With CloudFormation, you can use JSON or YAML templates to define your desired infrastructure state. You can specify resources, their configurations, dependencies, and relationships in these templates. LocalStack supports CloudFormation, allowing you to use the CloudFormation APIs in your local environment to declaratively define your architecture on the AWS, including resources such as S3 Buckets, Lambda Functions, and much more. The [API Coverage section](#api-coverage) and [feature coverage](#feature-coverage) provides information on the extent of CloudFormation's integration with LocalStack. ## Getting started This guide is designed for users new to CloudFormation and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to deploy a simple CloudFormation stack consisting of a single S3 Bucket with the AWS CLI. ### Create a CloudFormation Stack CloudFormation stack is a collection of AWS resources that you can create, update, or delete as a single unit. Stacks are defined using JSON or YAML templates. Use the following code snippet and save the content in either `cfn-quickstart-stack.yaml` or `cfn-quickstart-stack.json`, depending on your preferred format. ```yaml showshowLineNumbers Resources: LocalBucket: Type: AWS::S3::Bucket Properties: BucketName: cfn-quickstart-bucket ``` ```json showshowLineNumbers { "Resources": { "LocalBucket": { "Type": "AWS::S3::Bucket", "Properties": { "BucketName": "cfn-quickstart-bucket" } } } } ``` ### Deploy the CloudFormation Stack You can deploy the CloudFormation stack using the AWS CLI with the [`deploy`](https://docs.aws.amazon.com/cli/latest/reference/cloudformation/deploy/index.html) command. The `deploy` command creates and updates CloudFormation stacks. Run the following command to deploy the stack: ```bash lstk aws cloudformation deploy \ --stack-name cfn-quickstart-stack \ --template-file "./cfn-quickstart-stack.yaml" ``` You can verify that the stack was created successfully by listing the S3 buckets in your LocalStack container using the [`ListBucket` API](https://docs.aws.amazon.com/cli/latest/reference/s3api/list-buckets.html). Run the following command to list the buckets: ```bash lstk aws s3api list-buckets ``` ### Delete the CloudFormation Stack You can delete the CloudFormation stack using the [`delete-stack`](https://docs.aws.amazon.com/cli/latest/reference/cloudformation/delete-stack.html) command. Run the following command to delete the stack along with all the resources created by the stack: ```bash lstk aws cloudformation delete-stack \ --stack-name cfn-quickstart-stack ``` ## Registry Extensions LocalStack supports the execution of private CloudFormation registry extensions — custom resource types packaged with the [CloudFormation CLI](https://docs.aws.amazon.com/cloudformation-cli/latest/userguide/what-is-cloudformation-cli.html) and registered in your account's CloudFormation registry. Registry extensions work similarly to [custom resources](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html), with one key difference: the Lambda function that handles their lifecycle is not directly managed by the user. When a private extension is activated, LocalStack deploys and invokes the embedded handler Lambda internally, giving you full local emulation of the extension's Create, Read, Update, Delete, and List (CRUDL) lifecycle. ### Registering and using a private extension Build and package your extension using the CloudFormation CLI, then upload the package to S3 and register the type: ```bash lstk aws cloudformation register-type \ --type RESOURCE \ --type-name MyOrg::MyService::MyResource \ --schema-handler-package s3://my-bucket/my-extension.zip ``` You can then reference the registered type in a template like any built-in resource type: ```yaml Resources: MyCustomResource: Type: MyOrg::MyService::MyResource Properties: SomeProperty: value ``` When the stack is deployed, LocalStack routes each lifecycle operation to the handler Lambda that was deployed from the extension package. ### Supported package formats LocalStack currently resolves the handler artifact from the following formats inside the extension ZIP package: | Format | Description | |:-------|:------------| | `ResourceProvider.zip` | Python or Node.js handler produced by the CloudFormation CLI | | Single JAR file | Java-based resource provider handler | Support for additional payload formats will be added in future releases. :::note Extension packages must target a currently supported Lambda runtime. Python 3.9 is no longer supported; use Python 3.12 or another supported runtime when building your extension. ::: ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing CloudFormation stacks to manage your AWS resources locally. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **CloudFormation** under the **Management/Governance** section. ![CloudFormation Resource Browser](/images/aws/cloudformation-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Stack**: Create a new CloudFormation stack by clicking on **Create Stack** and provide a template file or URL, including the stack name and parameters. - **Edit Stack**: Edit an existing CloudFormation stack by clicking on **Edit Stack** and editing the stack name and parameters and clicking on **Submit**. - **View Stack**: View an existing CloudFormation stack by clicking on the Stack Name and viewing the stack details, including the stack name, status, and resources. - **Delete Stack**: Delete an existing CloudFormation stack by clicking on the Stack Name and clicking on **Actions** and then **Remove Selected**. ## Examples The following code snippets and sample applications provide practical examples of how to use CloudFormation in LocalStack for various use cases: - [Serverless Container-based APIs with Amazon ECS & API Gateway](https://github.com/localstack/serverless-api-ecs-apigateway-sample) - [Deploying containers on ECS clusters using ECR and Fargate](/aws/tutorials/ecs-ecr-container-app/) - [Messaging Processing application with SQS, DynamoDB, and Fargate](https://github.com/localstack/sqs-fargate-ddb-cdk-go) - [CloudFormation Registry Extension demo](https://github.com/localstack-samples/cloudformation-registry-demo) ## Best practices CloudFormation templates that target both real AWS and LocalStack should avoid hardcoded values that differ between the two environments. Using [pseudo parameters](#pseudo-parameters) and [intrinsic functions](#intrinsic-functions) keeps a single template portable without conditional logic or environment-specific parameter overrides. ### Use `AWS::URLSuffix` for service domain names Hardcoding `amazonaws.com` (or conversely `localhost.localstack.cloud`) when building service URLs is one of the most common causes of templates that deploy on AWS but fail on LocalStack, or vice versa. This typically shows up in API Gateway invoke URLs, Step Functions API integration targets, and other places where a template constructs a fully qualified endpoint. The `AWS::URLSuffix` pseudo parameter resolves to `amazonaws.com` on AWS (or `amazonaws.com.cn` in China Regions) and to the configured [`LOCALSTACK_HOST`](/aws/customization/configuration-options/) on LocalStack, which defaults to `localhost.localstack.cloud`. Referencing it lets the same template produce a valid URL in either environment. The following snippet shows the anti-pattern to avoid, where `amazonaws.com` is hardcoded into the output URL. A template written this way deploys on AWS but produces a non-resolvable URL on LocalStack: ```yaml Outputs: ApiUrl: Value: !Sub "https://${MyApi}.execute-api.${AWS::Region}.amazonaws.com/${StageName}" ``` Reference `AWS::URLSuffix` instead so the same template resolves to `amazonaws.com` on AWS and to the LocalStack host locally: ```yaml Outputs: ApiUrl: Value: !Sub "https://${MyApi}.execute-api.${AWS::Region}.${AWS::URLSuffix}/${StageName}" ``` The same pattern applies when wiring an API Gateway stage into a Step Functions task, when building a WebSocket invoke URL, or any other integration `Uri` that embeds a service domain. The LocalStack team contributed this practice upstream to the [AWS SAM application templates](https://github.com/aws/aws-sam-cli-app-templates/pull/525) and to the [AWS serverless patterns collection](https://github.com/aws-samples/serverless-patterns) that backs [serverlessland.com/patterns](https://serverlessland.com/patterns) and the VS Code Application Builder. :::caution `AWS::URLSuffix` is the right tool for endpoints your template constructs, such as API Gateway URLs or service domain joins. Avoid substituting it into URIs that AWS resolves to fixed production hostnames, for example: - ECR image URIs such as `.dkr.ecr..amazonaws.com/` - SageMaker built-in image URIs - AppSync `HttpConfig` endpoints for Bedrock or Step Functions data sources In those cases, the `amazonaws.com` suffix is part of a registry or service endpoint that is not served by LocalStack, so rewriting it can break the template on AWS without making it work on LocalStack. ::: ### Use `AWS::Partition` when building ARNs Prefer composing ARNs with `AWS::Partition`, `AWS::Region`, and `AWS::AccountId` rather than embedding a literal `arn:aws:...` prefix. The resulting template also works on AWS GovCloud and AWS China without changes: ```yaml ManagedPolicyArns: - !Sub "arn:${AWS::Partition}:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs" ``` ### Reference resources with `!Ref` and `Fn::GetAtt` When one resource needs the address of another, read it from the resource itself with `!Ref` or `!GetAtt` rather than constructing the URL from service domains. For example, use `!GetAtt MyQueue.QueueUrl` or `!GetAtt MyBucket.DomainName` so LocalStack returns the local endpoint while AWS returns the real one. ## Feature coverage :::tip We are continually enhancing our CloudFormation feature coverage by consistently introducing new resource types. Your feature requests assist us in determining the priority of resource additions. Feel free to contribute by [creating a new GitHub Discussion](https://github.com/orgs/localstack/discussions/new/choose). ::: ### Features | Feature | Support | |:--------------------|:------------------------------------------------| | Parameters | Partial | | Dynamic References | **Full** | | Rules | - | | Mappings | **Full** | | Conditions | **Full** | | Transform | **Full** | | Outputs | **Full** | | Custom resources | Partial | | Drift detection | - | | Importing Resources | - | | Change sets | **Full** | | Nested stacks | Partial | | StackSets | Partial | | Intrinsic Functions | Partial | | Registry extension execution | Partial | :::note Currently, support for `UPDATE` operations on resources is limited. Prefer stack re-creation over stack update at this time. ::: :::note Currently, support for `NoEcho` parameters is limited. Parameters will be masked only in the `Parameters` section of responses to `DescribeStacks` and `DescribeChangeSets` requests. This might expose sensitive information. Please exercise caution when using parameters with `NoEcho`. ::: ### Intrinsic Functions | Intrinsic Function | Supported | Explanation | | ------------------ | --------- | ------------------------------------------------------------ | | `Fn::And` | Yes | Performs a logical AND operation on two or more expressions. | | `Fn::Or` | Yes | Performs a logical OR operation on two or more expressions. | | `Fn::Base64` | Yes | Converts a binary string to a Base64-encoded string. | | `Fn::Sub` | Yes | Performs a string substitution operation. | | `Fn::Split` | Yes | Splits a string into an array of strings. | | `Fn::Length` | Yes | Returns the length of a string. | | `Fn::Join` | Yes | Joins an array of strings into a single string. | | `Fn::FindInMap` | Yes | Finds a value in a map. | | `Fn::Ref` | Yes | References a resource in the template. | | `Fn::GetAtt` | Yes | Gets an attribute from a resource. | | `Fn::If` | Yes | Performs a conditional evaluation. | | `Fn::Import` | Yes | Imports a value from another template. | | `Fn::ToJsonString` | No | Converts an object or map into a json string. | | `Fn::Cidr` | No | Generates a CIDR block from the inputs. | | `Fn::GetAZs` | No | Returns a list of the Availability Zones of a region. | ### Pseudo Parameters [Pseudo parameters](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html) are built-in variables that CloudFormation resolves at deployment time. You can reference them with the `Ref` intrinsic function (for example, `!Ref AWS::Region`) or with `Fn::Sub` (for example, `!Sub "${AWS::Region}"`). LocalStack resolves each pseudo parameter to the equivalent value for the local environment, which lets the same template deploy against both AWS and LocalStack. | Pseudo Parameter | Supported | Value in LocalStack | Value in AWS | | ----------------------- | --------- | ------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- | | `AWS::AccountId` | Yes | The account ID used by the stack (default: `000000000000`) | The AWS account ID of the account deploying the stack | | `AWS::NotificationARNs` | Partial | Empty list | The list of SNS topic ARNs passed to the stack via `--notification-arns` | | `AWS::NoValue` | Yes | Removes the corresponding property when used as a return value in `Fn::If` | Same | | `AWS::Partition` | Yes | `aws` | `aws`, `aws-cn`, or `aws-us-gov` depending on the Region | | `AWS::Region` | Yes | The Region of the encompassing resource | Same | | `AWS::StackId` | Yes | The ARN of the stack | Same | | `AWS::StackName` | Yes | The name of the stack | Same | | `AWS::URLSuffix` | Yes | The configured [`LOCALSTACK_HOST`](/aws/customization/configuration-options/) (default: `localhost.localstack.cloud`) | `amazonaws.com`, or `amazonaws.com.cn` in China Regions | :::tip Reach for `AWS::URLSuffix` and `AWS::Partition` instead of hardcoding `amazonaws.com` or `arn:aws:...` in templates. See [Best practices](#best-practices) for details. ::: ### Resources ## API Coverage # CloudFront > Get started with CloudFront on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; import { Badge } from '@astrojs/starlight/components'; ## Introduction CloudFront is a content delivery network (CDN) service provided by Amazon Web Services (AWS). CloudFront distributes its web content, videos, applications, and APIs with low latency and high data transfer speeds. CloudFront APIs allow you to configure distributions, customize cache behavior, secure content with access controls, and monitor the CDN's performance through real-time metrics. LocalStack allows you to use the CloudFront APIs in your local environment to create local CloudFront distributions to transparently access your applications and file artifacts. LocalStack also runs [CloudFront Functions](#cloudfront-functions) at request time and emulates [CloudFront KeyValueStore](#keyvaluestore-), so you can develop edge logic such as a tenant pre-router locally instead of validating it against live AWS. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of CloudFront's integration with LocalStack. ## Getting started This guide is intended for users who wish to get more acquainted with CloudFront over LocalStack. It assumes you have basic knowledge of the AWS CLI (and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command). Start your LocalStack container using your preferred method. We will demonstrate how you can create an S3 bucket, put a text file named `hello.txt` to the bucket, and then create a CloudFront distribution which makes the file accessible via a `https://abc123.cloudfront.net/hello.txt` proxy URL (where `abc123` is a placeholder for the real distribution ID). To get started, create an S3 bucket using the `mb` command: ```bash lstk aws s3 mb s3://abc123 ``` You can now go ahead, create a new text file named `hello.txt` and upload it to the bucket: ```bash echo 'Hello World' > /tmp/hello.txt lstk aws s3 cp /tmp/hello.txt s3://abc123/hello.txt --acl public-read ``` After uploading the file to S3, you can create a CloudFront distribution using the [`CreateDistribution`](https://docs.aws.amazon.com/cloudfront/latest/APIReference/API_CreateDistribution.html) API call. Run the following command to create a distribution with the default settings: ```bash domain=$(lstk aws cloudfront create-distribution \ --origin-domain-name abc123.s3.amazonaws.com | jq -r '.Distribution.DomainName') curl -k https://$domain/hello.txt ``` :::tip If you wish to use CloudFront on system host, ensure your local DNS setup is correctly configured. Refer to the section on [System DNS configuration](/aws/customization/networking/dns-server#system-dns-configuration) for details. ::: In the example provided above, be aware that the final command (`curl https://$domain/hello.txt`) might encounter a temporary failure accompanied by a warning message `Could not resolve host`. This can occur because different operating systems adopt diverse DNS caching strategies, causing a delay in the availability of the CloudFront distribution's DNS name (e.g., `abc123.cloudfront.net`) within the system. Typically, after a few retries, the command should succeed. It's worth noting that similar behavior can be observed in the actual AWS environment, where CloudFront DNS names may take up to 10-15 minutes to propagate across the network. ## CloudFront Functions [CloudFront Functions](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cloudfront-functions.html) are lightweight JavaScript functions that run at the edge to inspect and rewrite requests. LocalStack executes `viewer-request` functions at request time, so you can create a function, validate it with [`TestFunction`](https://docs.aws.amazon.com/cloudfront/latest/APIReference/API_TestFunction.html), publish it, attach it to a distribution, and observe its effect on a live request. ### Create a function Write the function code to a file. The handler must be a top-level function named `handler`: ```javascript title="stamp-env.js" import cf from 'cloudfront'; function handler(event) { var request = event.request; request.headers['x-erp-env'] = { value: 'prod' }; return request; } ``` Create the function with [`CreateFunction`](https://docs.aws.amazon.com/cloudfront/latest/APIReference/API_CreateFunction.html): ```bash lstk aws cloudfront create-function \ --name stamp-env \ --function-code fileb://stamp-env.js \ --function-config 'Comment=stamp the environment,Runtime=cloudfront-js-2.0' ``` ```bash title="Output" { "Location": "TODO", "ETag": "54ddd071", "FunctionSummary": { "Name": "stamp-env", "Status": "UNPUBLISHED", "FunctionConfig": { "Comment": "stamp the environment", "Runtime": "cloudfront-js-2.0" }, "FunctionMetadata": { "FunctionARN": "arn:aws:cloudfront::000000000000:function/stamp-env", "Stage": "DEVELOPMENT", "CreatedTime": "2026-08-20T15:28:49.477222+00:00", "LastModifiedTime": "2026-08-20T15:28:49.477226+00:00" } } } ``` ### Test a function `TestFunction` runs the function against a sample event and returns the computed output. This is how you validate the logic without sending a request through a distribution. Write the event object to a file: ```json title="event.json" showLineNumbers { "version": "1.0", "context": { "eventType": "viewer-request" }, "viewer": { "ip": "1.2.3.4" }, "request": { "method": "GET", "uri": "/index.html", "querystring": {}, "headers": { "host": { "value": "tenant-b.example.com" } }, "cookies": {} } } ``` Pass the ETag returned by `create-function` as `--if-match`: ```bash lstk aws cloudfront test-function \ --name stamp-env \ --if-match 54ddd071 \ --event-object fileb://event.json \ --query 'TestResult.{Output:FunctionOutput,Logs:FunctionExecutionLogs,Error:FunctionErrorMessage}' ``` ```bash title="Output" { "Output": "{\"request\": {\"method\": \"GET\", \"uri\": \"/index.html\", \"querystring\": {}, \"headers\": {\"host\": {\"value\": \"tenant-b.example.com\"}, \"x-erp-env\": {\"value\": \"prod\"}}, \"cookies\": {}}}", "Logs": [], "Error": "" } ``` `FunctionOutput` is wrapped in `request` when the function returns a request, and in `response` when it returns a response object. Anything the function writes with `console.log`, `console.error` or the other `console` methods is collected in `FunctionExecutionLogs`: ```bash title="Output" [ "routing tenant-b.example.com", "uri /index.html" ] ``` A function that raises at runtime does not fail the API call. `TestFunction` returns `200` with the error in `FunctionErrorMessage` and `FunctionOutput` set to `{}`. :::note The text of `FunctionErrorMessage` is a raw JavaScript stack trace from the local runtime and does not match the wording AWS returns. Use it for debugging, but do not assert on it in tests. ::: ### Publish a function [`PublishFunction`](https://docs.aws.amazon.com/cloudfront/latest/APIReference/API_PublishFunction.html) marks the function ready to associate with a distribution: ```bash lstk aws cloudfront publish-function --name stamp-env --if-match 54ddd071 ``` ```bash title="Output" { "FunctionSummary": { "Name": "stamp-env", "Status": "UNASSOCIATED", "FunctionConfig": { "Comment": "stamp the environment", "Runtime": "cloudfront-js-2.0" }, "FunctionMetadata": { "FunctionARN": "arn:aws:cloudfront::000000000000:function/stamp-env", "Stage": "DEVELOPMENT", "CreatedTime": "2026-08-20T15:28:49.477222+00:00", "LastModifiedTime": "2026-08-20T15:28:49.477226+00:00" } } } ``` ### Attach the function to a distribution Add the function ARN to `FunctionAssociations` on the `DefaultCacheBehavior` of your distribution config: ```json title="distribution-config.json (excerpt)" "DefaultCacheBehavior": { "TargetOriginId": "erp-origin", "ViewerProtocolPolicy": "allow-all", "ForwardedValues": { "QueryString": false, "Cookies": { "Forward": "none" } }, "MinTTL": 0, "FunctionAssociations": { "Quantity": 1, "Items": [ { "EventType": "viewer-request", "FunctionARN": "arn:aws:cloudfront::000000000000:function/stamp-env" } ] } } ``` Every request through the distribution now runs the function before the origin is contacted. See [Tenant routing at the edge](#tenant-routing-at-the-edge) for a complete, working configuration. ### Returning a response directly A function can end the request without contacting the origin by returning an object with a `statusCode`. LocalStack applies the status code, headers, cookies and body: ```javascript title="block-tenant.js" import cf from 'cloudfront'; function handler(event) { return { statusCode: 403, headers: { 'x-blocked-tenant': { value: 'acme' } }, cookies: { blocked: { value: '1', attributes: 'Path=/; Secure' }, trace: { value: 'abc' } }, body: { encoding: 'text', data: 'tenant blocked' } }; } ``` `body.encoding` accepts `text` and `base64`. Each entry in `cookies` becomes a `Set-Cookie` header, with `attributes` appended verbatim and `multiValue` entries emitted as additional headers of the same name. If the function raises at request time, the distribution responds with `500` and the body `The CloudFront function associated with the distribution failed to execute.` ### Current limitations - Only `viewer-request` associations execute. `viewer-response` associations are stored but never run. - Only associations on the `DefaultCacheBehavior` execute. Associations on other cache behaviors are stored but never run. - Only `uri` and `headers` from the returned request are applied. Changes to `querystring`, `cookies` and `method` are discarded. - `cf.kvs()` is the only runtime helper. `cf.crypto`, `cf.querystring` and `cf.updateRequestOrigin()` are not available. - `statusDescription` is not propagated when a function returns a response directly. The reason phrase is regenerated from the status code. - Publishing is not enforced at request time: an unpublished function attached to a distribution still runs. `PublishFunction` updates `Status` but `Stage` remains `DEVELOPMENT`. - One code blob is stored per function, so the `DEVELOPMENT` and `LIVE` stages resolve to the same code and the `--stage` option of `test-function` has no effect. - `ComputeUtilization` is always `"0"`. - `Location` in the `CreateFunction` response is the placeholder string `TODO` instead of a URL. - Functions run on Node.js rather than the restricted CloudFront JavaScript runtime. Code that uses Node.js globals or `fetch` works locally and fails on AWS. Conversely, only `import` statements that reference `cloudfront` are removed before execution, so any other import, such as `crypto`, raises a `SyntaxError` locally even though AWS supports it. - Function executions are serialized on a single Node.js process, which limits throughput under concurrent requests. :::note AWS requires `import cf from 'cloudfront'` in a `cloudfront-js-2.0` function. LocalStack removes that line before execution and also exposes `cf` as a global, so a function that omits the import runs locally and fails on AWS. Always write the import. ::: ## KeyValueStore A [CloudFront KeyValueStore](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/kvs-with-functions.html) holds key-value data that a CloudFront Function reads at request time, which lets you change the data a function acts on without republishing it. It is split across two APIs: the stores themselves are managed through the `cloudfront` control plane, and their contents are read and written through the separate `cloudfront-keyvaluestore` data plane. :::note The two planes are licensed separately. The control-plane operations belong to CloudFront, while the `cloudfront-keyvaluestore` data plane requires a plan that includes it. On a plan without the data plane you can create a store and associate it with a function, but you cannot write keys, and `cf.kvs().get()` then fails at request time. ::: ### Create a key value store Create a store with [`CreateKeyValueStore`](https://docs.aws.amazon.com/cloudfront/latest/APIReference/API_CreateKeyValueStore.html): ```bash lstk aws cloudfront create-key-value-store \ --name tenant-map \ --comment "tenant to environment" ``` ```bash title="Output" { "ETag": "02CDEC9C", "Location": "arn:aws:cloudfront::000000000000:key-value-store/d1fa734b-f440-4ebe-b477-8a12c8383488", "KeyValueStore": { "Name": "tenant-map", "Id": "d1fa734b-f440-4ebe-b477-8a12c8383488", "Comment": "tenant to environment", "ARN": "arn:aws:cloudfront::000000000000:key-value-store/d1fa734b-f440-4ebe-b477-8a12c8383488", "Status": "READY", "LastModifiedTime": "2026-08-20T15:28:15.621304+00:00" } } ``` The remaining control-plane operations are [`DescribeKeyValueStore`](https://docs.aws.amazon.com/cloudfront/latest/APIReference/API_DescribeKeyValueStore.html), [`ListKeyValueStores`](https://docs.aws.amazon.com/cloudfront/latest/APIReference/API_ListKeyValueStores.html), [`UpdateKeyValueStore`](https://docs.aws.amazon.com/cloudfront/latest/APIReference/API_UpdateKeyValueStore.html) and [`DeleteKeyValueStore`](https://docs.aws.amazon.com/cloudfront/latest/APIReference/API_DeleteKeyValueStore.html), all addressing the store by `--name`. ### Read and write keys Keys live behind the `cloudfront-keyvaluestore` service, which addresses a store by ARN rather than by name. :::tip Address the data plane through `localhost.localstack.cloud`, not `localhost` or an IP. These endpoint rules prepend the account ID from `--kvs-arn` to the host, so the request goes to `000000000000.`. An IP address can never serve that name, and `000000000000.localhost` resolves on some clients but not others. Terraform fails with `no such host`. Only `localhost.localstack.cloud` works everywhere, through its wildcard DNS entry. Control-plane commands are unaffected. ::: :::tip When you use `lstk aws`, install the AWS Common Runtime once with `pip install 'botocore[crt]'`. These requests are signed with SigV4A, which a `pip`-installed AWS CLI cannot sign without it. ::: The remaining examples in this section assume you have exported the store ARN and the endpoint host: ```bash export KVS_ARN=arn:aws:cloudfront::000000000000:key-value-store/d1fa734b-f440-4ebe-b477-8a12c8383488 export LOCALSTACK_HOST=localhost.localstack.cloud ``` Writes require the current ETag in `--if-match`. Read it from the data plane with [`DescribeKeyValueStore`](https://docs.aws.amazon.com/cloudfront-keyvaluestore/latest/APIReference/API_DescribeKeyValueStore.html): ```bash lstk aws cloudfront-keyvaluestore describe-key-value-store --kvs-arn "$KVS_ARN" ``` ```bash title="Output" { "ETag": "02CDEC9C", "ItemCount": 0, "TotalSizeInBytes": 0, "KvsARN": "arn:aws:cloudfront::000000000000:key-value-store/d1fa734b-f440-4ebe-b477-8a12c8383488", "Created": "2026-08-20T17:28:15.621304+02:00", "LastModified": "2026-08-20T17:28:15.621304+02:00", "Status": "READY" } ``` Write several keys at once with [`UpdateKeys`](https://docs.aws.amazon.com/cloudfront-keyvaluestore/latest/APIReference/API_UpdateKeys.html), which also accepts `--deletes`: ```bash lstk aws cloudfront-keyvaluestore update-keys \ --kvs-arn "$KVS_ARN" \ --if-match 02CDEC9C \ --puts 'Key=tenant-a.example.com,Value=prod' 'Key=tenant-b.example.com,Value=prod-sand' ``` ```bash title="Output" { "ETag": "6B15D4E0", "ItemCount": 2, "TotalSizeInBytes": 53 } ``` `TotalSizeInBytes` is the combined UTF-8 length of every key and value in the store. List the contents with [`ListKeys`](https://docs.aws.amazon.com/cloudfront-keyvaluestore/latest/APIReference/API_ListKeys.html): ```bash lstk aws cloudfront-keyvaluestore list-keys --kvs-arn "$KVS_ARN" ``` ```bash title="Output" { "Items": [ { "Key": "tenant-a.example.com", "Value": "prod" }, { "Key": "tenant-b.example.com", "Value": "prod-sand" } ] } ``` Single keys are handled with [`PutKey`](https://docs.aws.amazon.com/cloudfront-keyvaluestore/latest/APIReference/API_PutKey.html), [`GetKey`](https://docs.aws.amazon.com/cloudfront-keyvaluestore/latest/APIReference/API_GetKey.html) and [`DeleteKey`](https://docs.aws.amazon.com/cloudfront-keyvaluestore/latest/APIReference/API_DeleteKey.html): ```bash lstk aws cloudfront-keyvaluestore get-key --kvs-arn "$KVS_ARN" --key tenant-a.example.com ``` ```bash title="Output" { "Key": "tenant-a.example.com", "Value": "prod", "ItemCount": 2, "TotalSizeInBytes": 53 } ``` LocalStack implements the whole `cloudfront-keyvaluestore` API: | Operation | Implemented | | - | - | | `DescribeKeyValueStore` | ✅ | | `GetKey` | ✅ | | `PutKey` | ✅ | | `DeleteKey` | ✅ | | `UpdateKeys` | ✅ | | `ListKeys` | ✅ | ### ETag handling Every write rotates the store's ETag, so a write invalidates the ETag any earlier response gave you. Read the current ETag from the same plane you are about to call: `cloudfront describe-key-value-store --name` for a control-plane update or delete, and `cloudfront-keyvaluestore describe-key-value-store --kvs-arn` for a data-plane write. :::note On AWS the control plane and the data plane keep independent ETags. LocalStack keeps a single ETag shared by both planes, so an ETag from one plane is accepted by the other and a data-plane write also invalidates the control-plane ETag. Sourcing each ETag from the plane you are calling keeps your scripts portable to AWS. ::: Concurrency and lookup failures surface as follows: | Situation | Error code | Message | | - | - | - | | Stale or empty `--if-match` on a data-plane write | `ValidationException` | `Pre-Condition failed during update of Key-Value-Store` | | Stale or empty `--if-match` on a control-plane update or delete | `InvalidIfMatchVersion` | `The If-Match version is missing or not valid for the resource.` | | Store name not found | `EntityNotFound` | `The specified KeyValueStore does not exist.` | | Store ARN not found | `ResourceNotFoundException` | `The Key Value Store was not found.` | | Key not found in `get-key` | `ResourceNotFoundException` | `The Key was not found.` | | Store name already taken | `EntityAlreadyExists` | `The Key Value Store already exists.` | | Deleting a store a function is associated with | `CannotDeleteEntityWhileInUse` | `Cannot delete KeyValueStore tenant-map because it is associated with a function` | ### Reading a store from a function Associate the store when you create the function, through `KeyValueStoreAssociations`: ```bash lstk aws cloudfront create-function \ --name tenant-router \ --function-code fileb://tenant-router.js \ --function-config "Comment=tenant pre-router,Runtime=cloudfront-js-2.0,KeyValueStoreAssociations={Quantity=1,Items=[{KeyValueStoreARN=$KVS_ARN}]}" ``` The function reads the associated store through `cf.kvs()`: | Call | Returns | | - | - | | `await cf.kvs().get(key)` | the value as a string | | `await cf.kvs().get(key, { format: 'json' })` | the value parsed as JSON | | `await cf.kvs().exists(key)` | `true` or `false` | | `await cf.kvs().meta()` | `{ keyCount: }` | `get` raises `KeyValueStore key not found: ` for a key that is absent, and an unhandled error becomes a `500` response, so guard lookups that can miss with `exists`. Calling `cf.kvs()` in a function with no associated store raises `Function is not associated with a KeyValueStore`. ### Managing a key value store with Terraform The `hashicorp/aws` provider manages stores with `aws_cloudfront_key_value_store` and their contents with `aws_cloudfrontkeyvaluestore_keys_exclusive`, both of which work against LocalStack from version 5.100 onwards. :::tip Give `cloudfrontkeyvaluestore` its own `endpoints` entry pointing at `localhost.localstack.cloud`. With `localhost` the store still creates and only the keys fail, so the error reads as a missing store rather than a bad endpoint: ```bash title="Output" Error: reading AWS CloudFront KeyValueStore Key Value Store (arn:aws:cloudfront::000000000000:key-value-store/15a5...): operation error CloudFront KeyValueStore: DescribeKeyValueStore, https response error StatusCode: 0, RequestID: , request send failed, Get "http://000000000000.localhost:4566/key-value-stores/arn%3Aaws%3A...": dial tcp: lookup 000000000000.localhost: no such host ``` ::: ```hcl title="main.tf" showLineNumbers provider "aws" { region = "us-east-1" access_key = "test" secret_key = "test" skip_credentials_validation = true skip_metadata_api_check = true skip_requesting_account_id = true endpoints { cloudfront = "http://localhost:4566" cloudfrontkeyvaluestore = "http://localhost.localstack.cloud:4566" } } resource "aws_cloudfront_key_value_store" "tenant_map" { name = "tenant-map" comment = "tenant to environment" } resource "aws_cloudfrontkeyvaluestore_keys_exclusive" "tenant_map" { key_value_store_arn = aws_cloudfront_key_value_store.tenant_map.arn max_batch_size = 50 resource_key_value_pair { key = "tenant-a.example.com" value = "prod" } resource_key_value_pair { key = "tenant-b.example.com" value = "prod-sand" } } ``` ```bash title="Output" aws_cloudfront_key_value_store.tenant_map: Creation complete after 0s [id=15a50515-9931-4d2e-87bd-df5b3bf312e6] aws_cloudfrontkeyvaluestore_keys_exclusive.tenant_map: Creation complete after 0s Apply complete! Resources: 2 added, 0 changed, 0 destroyed. ``` ### Current limitations - `ImportSource` and `Tags` on `create-key-value-store` are accepted and ignored. A store is not seeded from S3 and cannot be tagged. - `Status` is always `READY`. There is no `PROVISIONING` state and no propagation delay, so a write is visible to the next request immediately rather than eventually. - `Location` in the `create-key-value-store` response is the store ARN instead of a URL. - Pagination is not implemented. `list-keys` ignores `--max-results` and `--next-token`, and `list-key-value-stores` ignores `--marker`, `--max-items` and the status filter. - AWS quotas, such as the 5 MB store and 1 KB key limits, are not enforced. - Only the first entry of `KeyValueStoreAssociations` is used. - The store contents are copied into the function at the start of an execution, so a write made during an execution is not visible to it. - The data plane resolves a store purely by ARN and does not check the caller's account, so any credentials can read and write any store. - Keys are persisted as part of the `cloudfront` service state rather than the `cloudfront-keyvaluestore` service. A [Cloud Pod](/aws/developer-tools/snapshots/cloud-pods) or state export limited to `cloudfront-keyvaluestore` therefore contains no keys, and resetting `cloudfront` discards them. - There is no CloudFormation resource provider for `AWS::CloudFront::KeyValueStore`. ## Tenant routing at the edge A common use of Functions with a KeyValueStore is a tenant pre-router. The function maps the incoming tenant to an environment and passes the decision to the origin, so routing data can change without redeploying the function. This example maps a tenant hostname to an environment name and stamps it on the request as `x-erp-env`, which is the form that behaves the same on AWS. Create the store and seed it as shown in [KeyValueStore](#create-a-key-value-store), then write the function: ```javascript title="tenant-router.js" import cf from 'cloudfront'; async function handler(event) { var request = event.request; var tenant = request.headers.host.value; var kvs = cf.kvs(); var env = (await kvs.exists(tenant)) ? await kvs.get(tenant) : 'prod'; request.headers['x-erp-env'] = { value: env }; return request; } ``` Create the function with the store associated, then publish it: ```bash export FUNCTION_ETAG=$(lstk aws cloudfront create-function \ --name tenant-router \ --function-code fileb://tenant-router.js \ --function-config "Comment=tenant pre-router,Runtime=cloudfront-js-2.0,KeyValueStoreAssociations={Quantity=1,Items=[{KeyValueStoreARN=$KVS_ARN}]}" \ --query ETag --output text) lstk aws cloudfront publish-function --name tenant-router --if-match "$FUNCTION_ETAG" ``` Confirm the routing decision with `test-function` before wiring up a distribution. With `host` set to `tenant-b.example.com` in `event.json`, the function resolves the tenant through the store: ```bash lstk aws cloudfront test-function \ --name tenant-router \ --if-match "$FUNCTION_ETAG" \ --event-object fileb://event.json \ --query 'TestResult.FunctionOutput' ``` ```bash title="Output" "{\"request\": {\"method\": \"GET\", \"uri\": \"/index.html\", \"querystring\": {}, \"headers\": {\"host\": {\"value\": \"tenant-b.example.com\"}, \"x-erp-env\": {\"value\": \"prod-sand\"}}, \"cookies\": {}}}" ``` To exercise the same path over a real request, create a distribution that lists the tenant hostnames in `Aliases` and attaches the function to its `DefaultCacheBehavior`. `DomainName` is the address of your origin as seen from the LocalStack container: ```json title="distribution-config.json" showLineNumbers { "CallerReference": "tenant-router-demo", "Comment": "", "Enabled": true, "Aliases": { "Quantity": 2, "Items": ["tenant-a.example.com", "tenant-b.example.com"] }, "Origins": { "Quantity": 1, "Items": [ { "Id": "erp-origin", "DomainName": "", "CustomOriginConfig": { "HTTPPort": 80, "HTTPSPort": 443, "OriginProtocolPolicy": "http-only" } } ] }, "DefaultCacheBehavior": { "TargetOriginId": "erp-origin", "ViewerProtocolPolicy": "allow-all", "ForwardedValues": { "QueryString": false, "Cookies": { "Forward": "none" } }, "MinTTL": 0, "FunctionAssociations": { "Quantity": 1, "Items": [ { "EventType": "viewer-request", "FunctionARN": "arn:aws:cloudfront::000000000000:function/tenant-router" } ] } } } ``` ```bash lstk aws cloudfront create-distribution \ --distribution-config file://distribution-config.json \ --query '{Id:Distribution.Id,DomainName:Distribution.DomainName}' ``` ```bash title="Output" { "Id": "56a90d1e", "DomainName": "56a90d1e.cloudfront.localhost.localstack.cloud" } ``` Because the tenant hostnames are registered as aliases, you can address the distribution with the tenant's `Host` header and the function receives the same hostname it would see on AWS: ```bash curl -s -H "Host: tenant-a.example.com" http://localhost.localstack.cloud:4566/index.html curl -s -H "Host: tenant-b.example.com" http://localhost.localstack.cloud:4566/index.html ``` With an origin that echoes the request headers, the two requests reach it carrying different environments: ```bash title="Output" "x-erp-env": "prod" "x-erp-env": "prod-sand" ``` A hostname that is not listed in `Aliases` does not match the distribution and returns `404`. A hostname that is listed but has no key in the store falls through to the `prod` default in the function. ### Routing by URI rewrite A function can also select the origin itself, by rewriting `request.uri` to a prefix that a cache behavior matches: ```javascript title="uri-router.js" import cf from 'cloudfront'; async function handler(event) { var request = event.request; var env = await cf.kvs().get(request.headers.tenant.value); request.uri = '/' + env + request.uri; return request; } ``` With `CacheBehaviors` entries for the path patterns `/prod/*` and `/nonprod/*`, each pointing at a different origin, the rewritten path selects the origin. :::caution This pattern is specific to LocalStack. LocalStack runs the `viewer-request` function before it matches the cache behavior, so a rewritten URI selects a different origin. Real CloudFront matches the cache behavior against the original request URI before the function runs and does not re-evaluate it afterwards, so the same configuration does not change the origin on AWS. Use it to exercise a rewrite locally, and prefer passing the decision to the origin, as in the example above, for logic you intend to deploy. ::: ## Lambda@Edge :::note We’re introducing an early, incomplete, and experimental feature that emulates AWS CloudFront Lambda@Edge, starting with version 4.3.0. It enables running Lambda functions at simulated edge locations. This allows you to locally test and develop request/response modifications, security enhancements and more. This feature is still under development, and functionality is limited. ::: You can enable this feature by setting `CLOUDFRONT_LAMBDA_EDGE=1` in your LocalStack configuration. ### Current features - Support for [`CreateDistribution`](https://docs.aws.amazon.com/cloudfront/latest/APIReference/API_CreateDistribution.html) API to set up CloudFront distributions with Lambda@Edge. - Support for modifying request and response headers dynamically. - Support for [`IncludeBody`](https://docs.aws.amazon.com/cloudfront/latest/APIReference/API_LambdaFunctionAssociation.html#cloudfront-Type-LambdaFunctionAssociation-IncludeBody) parameter. - Support for Node.js & Python 3.x runtime. ### Current limitations - The [`UpdateDistribution`](https://docs.aws.amazon.com/cloudfront/latest/APIReference/API_UpdateDistribution.html), [`DeleteDistribution`](https://docs.aws.amazon.com/cloudfront/latest/APIReference/API_DeleteDistribution.html), and [Persistence Restore](/aws/developer-tools/snapshots/persistence) features are not yet supported for Lambda@Edge. - The `origin-request` and `origin-response` event types currently trigger for each request because caching is not implemented in CloudFront. ## Using custom URLs LocalStack for AWS supports using an alternate domain name, also referred to as a `CNAME` or custom domain name, to access your applications and file artifacts instead of relying on the domain name generated by CloudFront for your distribution. To set up the custom domain name, you must configure it in your local DNS server. Once that is done, you can designate the desired domain name as an alias for the target distribution. To achieve this, you'll need to provide the `Aliases` field in the `--distribution-config` option when creating or updating a distribution. The format of this structure is similar to the one used in [AWS CloudFront options](https://docs.aws.amazon.com/cli/latest/reference/cloudfront/create-distribution.html#options). In the given example, two domains are specified as `Aliases` for a distribution. Please note that a complete configuration would entail additional values relevant to the distribution, which have been omitted here for brevity. ```bash --distribution-config {...'Aliases':'{'Quantity':2, 'Items': ['custom.domain.one', 'customDomain.two']}'...} ``` ## Custom IDs for CloudFront Distributions via tags Each CloudFront distribution is created with a random unique identifier automatically assigned by AWS. Given that the distribution ID is part of the generated domain name, it can be useful to have the possibility to create distributions with a deterministic ID (e.g., to simplify testing or integration with other AWS services). LocalStack offers this possibility by using the `_custom_id_` tag when creating a distribution with the [`CreateDistributionWithTags`](https://docs.aws.amazon.com/cloudfront/latest/APIReference/API_CreateDistributionWithTags.html) operation. ## Resource Browser The LocalStack Web Application provides a Resource Browser for CloudFront, which allows you to view and manage your CloudFront distributions. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resource Browser** section, and then clicking on **CloudFront** under the **Analytics** section. ![CloudFront Resource Browser](/images/aws/cloudfront-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Distribution**: Create a new CloudFront distribution by specifying the **Origins** and other settings. - **List Distributions**: View a list of all CloudFront distributions. - **Edit Distribution**: Modify the settings of an existing CloudFront distribution by opening the distribution's details page and clicking on the **Edit Distribution** button. - **Delete Distribution**: Delete an existing CloudFront distribution by selecting the distribution, click on **Actions**, and then click on **Remove Selected**. ## Examples The following code snippets and sample applications provide practical examples of how to use CloudFront in LocalStack for various use cases: - [Step-up Authentication using Amazon Cognito](https://github.com/localstack/step-up-auth-sample) ## API Coverage # CloudTrail > Get started with CloudTrail on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction CloudTrail is a service provided by Amazon Web Services (AWS) that enables you to track and monitor all activities and events within your AWS environment. It records API calls and actions made on your AWS resources, offering an audit trail that helps you understand changes, diagnose issues, and maintain compliance. LocalStack allows you to use the CloudTrail APIs in your local environment to create and manage Event history and trails. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of CloudTrail's integration with LocalStack. ## Getting started This guide is designed for users new to CloudTrail and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to enable S3 object logging to CloudTrail using AWS CLI. ### Create a bucket Before you create a trail, you need to create an S3 bucket where CloudTrail can deliver the log data. You can use the [`mb`](https://docs.aws.amazon.com/cli/latest/reference/s3/mb.html) command to create a bucket: ```bash lstk aws s3 mb s3://my-bucket ``` ### Create a trail You can create a trail which would allow the delivery of events to the S3 bucket we created earlier. You can use the [`CreateTrail`](https://docs.aws.amazon.com/awscloudtrail/latest/APIReference/API_CreateTrail.html) API to create a trail. Run the following command to create a trail: ```bash lstk aws cloudtrail create-trail \ --name MyTrail \ --s3-bucket-name my-bucket ``` ### Enable logging and configure event selectors You can now enable logging for your trail. You can use the [`StartLogging`](https://docs.aws.amazon.com/awscloudtrail/latest/APIReference/API_StartLogging.html) API to enable logging for your trail. Run the following command to enable logging: ```bash lstk aws cloudtrail start-logging --name MyTrail ``` You can further configure event selectors for the trail. In this example, we will configure the trail to log all S3 object level events. You can use the [`PutEventSelectors`](https://docs.aws.amazon.com/awscloudtrail/latest/APIReference/API_PutEventSelectors.html) API to configure event selectors for your trail. Run the following command to configure event selectors: ```bash lstk aws cloudtrail put-event-selectors \ --trail-name MyTrail \ --event-selectors '[{"ReadWriteType": "All", "IncludeManagementEvents":true, "DataResources": [{"Type": "AWS::S3::Object", "Values": ["arn:aws:s3:::my-bucket/"]}]}]' ``` You can verify if your configuration is correct by using the [`GetEventSelectors`](https://docs.aws.amazon.com/awscloudtrail/latest/APIReference/API_GetEventSelectors.html) API. Run the following command to verify your configuration: ```bash lstk aws cloudtrail get-event-selectors \ --trail-name MyTrail ``` ```bash title="Output" { "TrailARN": "arn:aws:cloudtrail:us-east-1:000000000000:trail/MyTrail", "EventSelectors": [ { "ReadWriteType": "All", "IncludeManagementEvents": true, "DataResources": [ { "Type": "AWS::S3::Object", "Values": [ "arn:aws:s3:::my-bucket/" ] } ] } ] } ``` ### Test the configuration You can now test the configuration by creating an object in the S3 bucket. You can use the [`cp`](https://docs.aws.amazon.com/cli/latest/reference/s3/cp.html) command to copy an object in the S3 bucket: ```bash echo "hello world" > /tmp/hello-world lstk aws s3 cp /tmp/hello-world s3://my-bucket/hello-world lstk aws s3 ls s3://my-bucket ``` You can verify that the object was created in the S3 bucket. You can also verify that the object level event was logged by CloudTrail using the [`LookupEvents`](https://docs.aws.amazon.com/awscloudtrail/latest/APIReference/API_LookupEvents.html) API. Run the following command to verify the event: ```bash lstk aws cloudtrail lookup-events \ --lookup-attributes AttributeKey=EventName,AttributeValue=PutObject \ --max-results 1 ``` ```bash title="Output" { "Events": [{ "EventId": "218785bf-3ec4-4bdd-a055-57eca773294f", "EventName": "PutObject", "ReadOnly": "false", ... "CloudTrailEvent": "{\"eventVersion\": \"1.08\", ... {\"bucketName\": \"my-bucket\", \"key\": \"hello-world\"} ...}" }] } ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing CloudTrail's Event History & Trails. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **CloudTrail** under the **Management/Governance** section. ![CloudTrail Resource Browser](/images/aws/cloudtrail-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Trail**: Create a new CloudTrail trail, by specifying the name of the trail, the S3 bucket where the logs should be stored, and other optional parameters. - **View Trail**: View the details of a CloudTrail trail, including the name, ARN, S3 bucket, and other parameters. - **View Event History**: View the event history of a CloudTrail trail, including the Event Id, Event time, Event source, and other parameters. ## API Coverage # CloudWatch > Get started with AWS CloudWatch on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction CloudWatch is a comprehensive monitoring and observability service that Amazon Web Services (AWS) provides. It allows you to collect and track metrics, collect and monitor log files, and set alarms. CloudWatch provides valuable insights into your AWS resources, applications, and services, enabling you to troubleshoot issues, optimize performance, and make informed decisions. LocalStack allows you to use CloudWatch APIs on your local machine to create and manage CloudWatch resources, such as custom metrics, alarms, and log groups, for local development and testing purposes. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of CloudWatch's integration with LocalStack. ## Getting started This guide is designed for users new to CloudWatch and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method and deploy your Lambda functions that will generate some logs. You can get the name for your Lambda Functions using the [`ListFunctions`](https://docs.aws.amazon.com/lambda/latest/dg/API_ListFunctions.html) API. Fetch the Log Groups using the [`DescribeLogGroups`](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_DescribeLogGroups.html) API. Run the following command to get the Log Group name: ```bash lstk aws logs describe-log-groups ``` ```bash title="Output" { "logGroups": [ { "logGroupName": "/aws/lambda/serverless-local-hello", "creationTime": 1683009865348, "metricFilterCount": 0, "arn": "arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/serverless-local-hello:*", "storedBytes": 262 }, { "logGroupName": "/aws/lambda/serverless-local-hello2", "creationTime": 1683009865420, "metricFilterCount": 0, "arn": "arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/serverless-local-hello2:*", "storedBytes": 262 } ] } ``` Get the log streams for the Log Group using the [`DescribeLogStreams`](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_DescribeLogStreams.html) API. Run the following command to get the Log Stream name: ```bash lstk aws logs describe-log-streams \ --log-group-name /aws/lambda/serverless-local-hello ``` ```bash title="Output" { "logStreams": [ { "logStreamName": "2023/05/02/[$LATEST]853a59d0767cfaf10d6b29a6790d8b03", "creationTime": 1683009968958, "firstEventTimestamp": 1683009968920, "lastEventTimestamp": 1683009968945, "lastIngestionTime": 1683009968979, "uploadSequenceToken": "1", "arn": "arn:aws:logs:us-east-1:000000000000:log-group:/aws/lambda/serverless-local-hello:log-stream:2023/05/02/[$LATEST]853a59d0767cfaf10d6b29a6790d8b03", "storedBytes": 262 } ] } ``` You can now fetch the log events using the [`GetLogEvents`](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_GetLogEvents.html) API. Run the following command to get the logs: ```bash lstk aws logs get-log-events \ --log-group-name '/aws/lambda/serverless-local-hello' --log-stream-name '2023/05/02/[$LATEST]853a59d0767cfaf10d6b29a6790d8b03' ``` ```bash title="Output" { "events": [ { "timestamp": 1683009968920, "message": "START RequestId: 71712856-9f41-4d22-827c-e3883f799f25 Version: $LATEST", "ingestionTime": 1683009968979 }, { "timestamp": 1683009968932, "message": "END RequestId: 71712856-9f41-4d22-827c-e3883f799f25", "ingestionTime": 1683009968979 }, { "timestamp": 1683009968945, "message": "REPORT RequestId: 71712856-9f41-4d22-827c-e3883f799f25\tDuration: 1.27 ms\tBilled Duration: 2 ms\tMemory Size: 1024 MB\tMax Memory Used: 1024 MB\t", "ingestionTime": 1683009968979 } ], "nextForwardToken": "f/00000000000000000000000000000000000000000000000000000002", "nextBackwardToken": "b/00000000000000000000000000000000000000000000000000000000" } ``` :::tip You can use [filters](https://docs.aws.amazon.com/cli/latest/reference/logs/filter-log-events.html) or [queries](https://docs.aws.amazon.com/cli/latest/reference/logs/get-query-results.html) with a licensed LocalStack edition to refine your results. ::: ## Metric Alarms Alarms in CloudWatch are crucial in monitoring specific data thresholds and automating actions based on those thresholds. To learn more about how alarms are evaluated in general, please refer to the [AWS documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html#alarm-evaluation). In LocalStack, you can use metric-alarm evaluation, explicitly utilizing the statistic and comparison-operator functionalities. These features enable you to define and evaluate alarms based on various statistical calculations and comparison operators. ### Metric Alarm Examples Metric alarms in CloudWatch allow you to evaluate the state of a metric by analyzing its data points over a specified period. With metric alarms, you can create customized thresholds and define actions based on the metric's behavior. To get started with creating an alarm in LocalStack using the `lstk aws` integration, use the following command: ```bash lstk aws cloudwatch put-metric-alarm \ --alarm-name my-alarm \ --metric-name Orders \ --namespace test \ --threshold 1 \ --comparison-operator LessThanThreshold \ --evaluation-periods 1 \ --period 30 \ --statistic Minimum \ --treat-missing notBreaching ``` To monitor the status of the alarm, open a separate terminal and execute the following command: ```bash watch "lstk aws cloudwatch describe-alarms --alarm-names my-alarm | jq '.MetricAlarms[0].StateValue'" ``` Afterward, you can add some data that will cause a breach and set the `metric-alarm` state to **ALARM** using the following command: ```bash lstk aws cloudwatch put-metric-data \ --namespace test \ --metric-data '[{"MetricName": "Orders", "Value": -1}]' ``` Within a few seconds, the alarm state should change to **ALARM**, and eventually, it will go back to **OK** as we configured it to treat missing data points as `not breaching`. This allows you to observe how the alarm behaves in response to the provided data. #### Metric Alarm with Action When the state of an alarm changes, actions can be triggered accordingly. In LocalStack, you can configure `alarm-actions`, `ok-actions`, and `insufficient-data-actions` to specify the actions to be taken. Currently, only SNS Topics are supported as the target for these actions, and it's important to note that the topic must be created beforehand. Here's an example demonstrating how to set up an alarm that sends a message to the specified topic when entering the **ALARM** state. Make sure to replace `` with the valid ARN of an existing SNS topic. ```bash lstk aws cloudwatch put-metric-alarm \ --alarm-name my-alarm \ --metric-name Orders \ --namespace test \ --threshold 50 \ --comparison-operator GreaterThanThreshold \ --evaluation-periods 1 \ --period 300 \ --statistic Maximum \ --treat-missing notBreaching \ --alarm-actions ``` By executing this command, you'll create an alarm named `my-alarm` that monitors the `Orders` metric in the `test` namespace. If the metric value exceeds the threshold of 50 (using the `GreaterThanThreshold` operator) during a single evaluation period of 300 seconds, the alarm will trigger the specified action on the provided SNS topic. :::danger Please be aware of the following known limitations in LocalStack: - Anomaly detection and extended statistics are not supported. - The `unit` values specified in the alarm are ignored. - Composite alarms are not evaluated. - Metric streams are not supported. ::: ## Current Limitations The following CloudWatch Metrics are not supported: - Anomaly detection - Metric streams - Extended statistics In addition, the `unit` values specified in the alarm are ignored, and Composite alarms are not evaluated. ## Supported Service Integrations LocalStack supports the following AWS services for integration with CloudWatch metrics: - **SQS**: Supports `Approximate*` metrics, `NumberOfMessagesSent`, and other metrics triggered by events such as message received or sending. - **Lambda**: Supports `Invocations` and `Errors` metrics. ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing CloudWatch logs. You can access the Resource Browser by opening the LocalStack Web Application in your browser and navigating to the Resources section, then clicking on [**CloudWatch Logs**](https://app.localstack.cloud/resources/cloudwatch/groups) and [**CloudWatch Metrics**](https://app.localstack.cloud/resources/monitoring) under the **Management/Governance** section. The Resource Browser allows you to perform the following actions: ![CloudWatch Logs Resource Browser](/images/aws/cloudwatch-log-groups-resource-browser.png) ![CloudWatch Metrics Resource Browser](/images/aws/cloudwatch-metrics-resource-browser.png) - **Create Log Group**: Create a new log group by specifying the `Log Group Name`, `KMS Key ID`, and `Tags`. - **Put metric**: Create a new metric by specifying the `Namespace` and `Metric Data`. - **Put Alarm**: Create a new alarm by specifying the `Alarm Name`, `Alarm Description`, `Actions Enabled`, `Metric Name`, `Namespace`, `Statistic`, `Comparison Operator`, `Threshold`, `Evaluation Periods`, `Period`, `Unit`, `Treat Missing Data`, `Tags`, and `Alarm Actions`. - **Check the Resources**: View and manage existing log groups, metrics, and alarms and perform actions such as `Delete`, `View`, and `Edit`. ## Examples The following code snippets and sample applications provide practical examples of how to use CloudWatch in LocalStack for various use cases: - [Creating Cloudwatch metric alarms](https://github.com/localstack/localstack-pro-samples/tree/master/cloudwatch-metrics-aws) to demonstrate a simple example for creating CloudWatch metric alarm based on the metrics of a failing Lambda function. - [Event-driven architecture with Amazon SNS FIFO, DynamoDB, Lambda, and S3](https://github.com/localstack/event-driven-architecture-with-amazon-sns-fifo) to deploy a recruiting agency application with a job listings website and view the CloudWatch logs. ## API Coverage # CodeArtifact > Get started with CodeArtifact on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction CodeArtifact is a fully managed artifact repository service that makes it easy to securely store, publish, and share software packages used in your development process. On AWS, CodeArtifact supports popular package formats such as Maven, npm, Python (pip), NuGet, etc. You can configure it to work with public repositories or use it to store your private packages. LocalStack provides mocking support for several CodeArtifact API operations. You can find supported operations on the [API coverage page](#api-coverage). It also has full support to create and use NPM repositories. ## Getting Started This guide will help you create a domain, repository, and manage package publishing workflows using the `lstk aws` command. Basic knowledge of the AWS CLI and the [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command is expected. Start LocalStack using your preferred method. ### Domains Domains are the top-level containers for repositories in CodeArtifact. Create a domain with the [`CreateDomain`](https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_CreateDomain.html) API. ```bash lstk aws codeartifact create-domain --domain demo-domain ``` ```json title="Output" { "domain": { "name": "demo-domain", "owner": "000000000000", "arn": "arn:aws:codeartifact:eu-central-1:000000000000:domain/demo-domain", "status": "Active", "createdTime": "2025-05-20T11:30:52.073202+02:00", "repositoryCount": 0, "assetSizeBytes": 0 } } ``` You can use the [`DescribeDomain`](https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_DescribeDomain.html), [`UpdateDomain`](https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_UpdateDomain.html), and [`DeleteDomain`](https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_DeleteDomain.html) APIs for domain management. ```bash lstk aws codeartifact describe-domain --domain demo-domain ``` ```json title="Output" { "domain": { "name": "demo-domain", "owner": "000000000000", "arn": "arn:aws:codeartifact:eu-central-1:000000000000:domain/demo-domain", "status": "Active", "createdTime": "2025-05-20T11:30:52.073202+02:00", "repositoryCount": 0, "assetSizeBytes": 0 } } ``` You can list all domains using the [`ListDomains`](https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_ListDomains.html) API. ```bash lstk aws codeartifact list-domains ``` ```json title="Output" { "domains": [ { "name": "demo-domain", "owner": "000000000000", "arn": "arn:aws:codeartifact:eu-central-1:000000000000:domain/demo-domain", "status": "Active", "createdTime": "2025-05-20T11:30:52.073202+02:00" } ] } ``` ### Repositories Repositories store packages and are associated with a domain. Create a repository using the [`CreateRepository`](https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_CreateRepository.html) API. ```bash lstk aws codeartifact create-repository --domain demo-domain \ --repository demo-repo ``` ```json title="Output" { "repository": { "name": "demo-repo", "administratorAccount": "000000000000", "domainName": "demo-domain", "domainOwner": "000000000000", "arn": "arn:aws:codeartifact:eu-central-1:000000000000:repository/demo-domain/demo-repo", "upstreams": [], "externalConnections": [], "createdTime": "2025-05-20T11:34:27.712367+02:00" } } ``` You can use the [`DescribeRepository`](https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_DescribeRepository.html), [`UpdateRepository`](https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_UpdateRepository.html), and [`DeleteRepository`](https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_DeleteRepository.html) APIs to manage repositories. ```bash lstk aws codeartifact describe-repository --domain demo-domain \ --repository demo-repo ``` ```json title="Output" { "repository": { "name": "demo-repo", "administratorAccount": "000000000000", "domainName": "demo-domain", "domainOwner": "000000000000", "arn": "arn:aws:codeartifact:eu-central-1:000000000000:repository/demo-domain/demo-repo", "upstreams": [], "externalConnections": [], "createdTime": "2025-05-20T11:34:27.712367+02:00" } } ``` Use the [`ListRepositories`](https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_ListRepositories.html) API to view all of the repositories. ```bash lstk aws codeartifact list-repositories ``` ```json title="Output" { "repositories": [ { "name": "demo-repo", "administratorAccount": "000000000000", "domainName": "demo-domain", "domainOwner": "000000000000", "arn": "arn:aws:codeartifact:eu-central-1:000000000000:repository/demo-domain/demo-repo", "createdTime": "2025-05-20T11:34:27.712367+02:00" } ] } ``` Otherwise, list repositories in a specific domain using the [`ListRepositoriesInDomain`](https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_ListRepositoriesInDomain.html) API. ```bash lstk aws codeartifact list-repositories-in-domain --domain demo-domain ``` ```json title="Output" { "repositories": [ { "name": "demo-repo", "administratorAccount": "000000000000", "domainName": "demo-domain", "domainOwner": "000000000000", "arn": "arn:aws:codeartifact:eu-central-1:000000000000:repository/demo-domain/demo-repo", "createdTime": "2025-05-20T11:34:27.712367+02:00" } ] } ``` ### Upstream Repositories and External Connections A repository can have other CodeArtifact repositories as upstream repositories. This enables a package manager client to access the packages that are contained in more than one repository using a single repository endpoint. Furthermore, you can add a external connection between a CodeArtifact repository and an external, public repository such as [https://npmjs.com](https://npmjs.com). Then, when you request a package from the CodeArtifact repository that's not already present in the repository, the package can be fetched from the external connection. This makes it possible to consume open-source dependencies used by your application. Repositories can be associated with external connections using [AssociateExternalConnection](https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_AssociateExternalConnection.html) and [DisassociateExternalConnection](https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_DisassociateExternalConnection.html) APIs. ```bash lstk aws codeartifact associate-external-connection --domain demo-domain \ --repository demo-repo \ --external-connection "public:npmjs" ``` ```json title="Output" { "repository": { "name": "demo-repo", "administratorAccount": "000000000000", "domainName": "demo-domain", "domainOwner": "000000000000", "arn": "arn:aws:codeartifact:eu-central-1:000000000000:repository/demo-domain/demo-repo", "upstreams": [], "externalConnections": [ { "externalConnectionName": "public:npmjs", "packageFormat": "npm", "status": "AVAILABLE" } ], "createdTime": "2025-05-20T14:03:27.539994+02:00" } } ``` Alternatively, repositories can be configured with upstream repositories using the `upstreams` property of [CreateRepository](https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_CreateRepository.html) and [UpdateRepository](https://docs.aws.amazon.com/codeartifact/latest/APIReference/API_UpdateRepository.html). ```bash lstk aws codeartifact create-repository --domain demo-domain \ --repository demo-repo2 \ --upstreams repositoryName=demo-repo ``` ```bash title="Output" { "repository": { "name": "demo-repo2", "administratorAccount": "000000000000", "domainName": "demo-domain", "domainOwner": "000000000000", "arn": "arn:aws:codeartifact:eu-central-1:000000000000:repository/demo-domain/demo-repo2", "upstreams": [ { "repositoryName": "demo-repo" } ], "externalConnections": [], "createdTime": "2025-05-20T14:07:56.741333+02:00" } } ``` :::note Please note, a repository can have one or more upstream repositories, or an external connection. ::: ## Using CodeArtifact with npm ### Configuring npm with the login command Use the `lstk aws codeartifact login` command to fetch credentials for use with npm. ```bash lstk aws codeartifact login --tool npm --domain demo-domain --repository demo-repo ``` This command makes the following changes to your `~/.npmrc` file: - Adds an authorization token after fetching it from CodeArtifact using your AWS credentials. - Sets the npm registry to the repository specified by the `--repository` option. - **For npm 6 and lower:** Adds `"always-auth=true"` so the authorization token is sent for every npm command. The default authorization period after calling login is 12 hours, and login must be called to periodically refresh the token. For more information about the authorization token created with the login command, see [Tokens created with the login command](https://docs.aws.amazon.com/codeartifact/latest/ug/tokens-authentication.html#auth-token-login). ### Configuring npm manually You can configure npm with your CodeArtifact repository without the `lstk aws codeartifact login` command by manually updating the npm configuration. 1. In a command line, fetch a CodeArtifact authorization token and store it in an environment variable. npm will use this token to authenticate with your CodeArtifact repository. ```bash export CODEARTIFACT_AUTH_TOKEN=$(lstk aws codeartifact get-authorization-token --domain demo-domain --query authorizationToken --output text) ``` 2. Get your CodeArtifact repository's endpoint by running the following command. Your repository endpoint is used to point npm to your repository to install or publish packages. ```bash lstk aws codeartifact get-repository-endpoint --domain demo-domain --repository demo-repo --format npm --output text ``` The following URL is an example repository endpoint. ```text http://demo-domain-000000000000.d.codeartifact.eu-central-1.localhost.localstack.cloud:4566/npm/demo-repo/ ``` 3. Use the `npm config set` command to set the registry to your CodeArtifact repository. Replace the URL with the repository endpoint URL from the previous step. ```bash npm config set registry http://demo-domain-000000000000.d.codeartifact.eu-central-1.localhost.localstack.cloud:4566/npm/demo-repo/ ``` 4. Use the `npm config set` command to add your authorization token to your npm configuration. ```bash npm config set //demo-domain-000000000000.d.codeartifact.eu-central-1.localhost.localstack.cloud:4566/:_authToken=${CODEARTIFACT_AUTH_TOKEN} ``` :::note **For npm 6 or lower:** To make npm always pass the auth token to CodeArtifact, even for GET requests, set the always-auth configuration variable with npm config set. ```bash npm config set //demo-domain-000000000000.d.codeartifact.eu-central-1.localhost.localstack.cloud:4566/:always-auth=true ``` ::: ### Example npm configuration file The following is an example `.npmrc` file after following the preceding instructions to set the CodeArtifact registry endpoint, add an authentication token, and configure `always-auth`. ```text registry=http://demo-domain-000000000000.d.codeartifact.eu-central-1.localhost.localstack.cloud:4566/npm/demo-repo/ //demo-domain-000000000000.d.codeartifact.eu-central-1.localhost.localstack.cloud:4566/:_authToken=eyJ2ZX... //demo-domain-000000000000.d.codeartifact.eu-central-1.localhost.localstack.cloud:4566/:always-auth=true ``` ## Current Limitations LocalStack does not support the following features yet: - Domain owners are ignored - Copying package versions is not supported yet - Domain and repository permission policies are not supported yet - Package groups are not supported yet - Only supports the `npm` format ## API Coverage # CodeBuild > Get started with CodeBuild on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; import { FileTree } from '@astrojs/starlight/components'; ## Introduction AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces software packages that are ready to deploy. It is part of the [AWS Developer Tools suite](https://aws.amazon.com/products/developer-tools/) and integrates with other AWS services to provide an end-to-end development pipeline. LocalStack supports the emulation of most of the CodeBuild operations. The supported operations are listed on the [API Coverage section](#api-coverage). AWS CodeBuild emulation is powered by the [AWS CodeBuild agent](https://docs.aws.amazon.com/codebuild/latest/userguide/use-codebuild-agent.html). ## Getting Started This tutorial will show you how to use AWS CodeBuild to test and build a deployable version of a Java executable. It assumes basic knowledge of the [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command, Apache Maven, and Java. ### Create the source code In the first step, we have to create the project that we want to build with AWS CodeBuild. In an empty directory, we need to re-create the following structure: - root-directory-name - pom.xml - src - main - java - MessageUtil.java - test - java - TestMessageUtil.java Let us walk through these files. `MessageUtil.java` contains the entire logic of this small application. It does nothing more than print a salutation message. Create a `MessageUtil.java` file and save it into the `src/main/java` directory. ```java showshowLineNumbers public class MessageUtil { private String message; public MessageUtil(String message) { this.message = message; } public String printMessage() { System.out.println(message); return message; } public String salutationMessage() { message = "Hi!" + message; System.out.println(message); return message; } } ``` Every build needs to be tested. Therefore, create the `TestMessageUtil.java` file in the `src/test/java` directory. ```java showshowLineNumbers import org.junit.Test; import org.junit.Ignore; import static org.junit.Assert.assertEquals; public class TestMessageUtil { String message = "Robert"; MessageUtil messageUtil = new MessageUtil(message); @Test public void testPrintMessage() { System.out.println("Inside testPrintMessage()"); assertEquals(message,messageUtil.printMessage()); } @Test public void testSalutationMessage() { System.out.println("Inside testSalutationMessage()"); message = "Hi!" + "Robert"; assertEquals(message,messageUtil.salutationMessage()); } } ``` This small suite simply verifies that the greeting message is built correctly. Finally, we need a `pom.xml` file to instruct Maven about what to build and which artifact needs to be produced. Create this file at the root of your directory. ```xml showshowLineNumbers 4.0.0 org.example messageUtil 1.0 jar Message Utility Java Sample App junit junit 4.11 test org.apache.maven.plugins maven-compiler-plugin 3.8.0 ``` With the following configuration, Maven will compile the `java` files into a executable jar and run the specified tests. ### Create the buildspec file Now that we have our project set up, we need to create a `buildspec` file. A `buildspec` file is a collection of settings and commands, specified in YAML format, that tells AWS CodeBuild how to run a build. Create this `buildspec.yml` file in the root directory. ```yaml showshowLineNumbers version: 0.2 phases: install: runtime-versions: java: corretto11 pre_build: commands: - echo Nothing to do in the pre_build phase... build: commands: - echo Build started on `date` - mvn install post_build: commands: - echo Build completed on `date` artifacts: files: - target/messageUtil-1.0.jar ``` In this file we can observe how the build will be executed. First, we define a runtime version. Then, we run a `mvn install` command in the build phase which does both the compilation and the testing. The pre and post build phases do not do much in this example, but can be used for various things, like install some software needed for the build itself. A full specification of a `buildspec` file can be found in the [AWS CodeBuild docs](https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html). ### Create input and output buckets Now we have to create two S3 buckets: - one bucket that stores the source we just created, that will be the source of the AWS CodeBuild build - one bucket where the output of the build, i.e., the JAR file, will be stored. Create the buckets with the following commands: ```bash lstk aws s3 mb s3://codebuild-demo-input lstk aws s3 mb s3://codebuild-demo-output ``` Finally, zip the content of the source code directory and upload it to the created source bucket. With a UNIX system, you can simply use the `zip` utility: ```bash zip -r MessageUtil.zip ``` Then, upload `MessageUtil.zip` to the `codebuild-demo-input` bucket with the following command: ```bash lstk aws s3 cp MessageUtil.zip s3://codebuild-demo-input ``` ### Configuring IAM To properly work, AWS CodeBuild needs access to other AWS services, e.g., to retrieve the source code from a S3 bucket. Create a `create-role.json` file with following content: ```json showshowLineNumbers { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "codebuild.amazonaws.com" }, "Action": "sts:AssumeRole" } ] } ``` Then, run the following command to create the necessary IAM role: ```bash lstk aws iam create-role --role-name CodeBuildServiceRole --assume-role-policy-document file://create-role.json ``` From the command's response, keep note of the role ARN: it will be needed to create the CodeBuild project later on. Let us now define a policy for the created role. Create a `put-role-policy.json` file with the following content: ```json showshowLineNumbers { "Version": "2012-10-17", "Statement": [ { "Sid": "CloudWatchLogsPolicy", "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "*" }, { "Sid": "CodeCommitPolicy", "Effect": "Allow", "Action": [ "codecommit:GitPull" ], "Resource": "*" }, { "Sid": "S3GetObjectPolicy", "Effect": "Allow", "Action": [ "s3:GetObject", "s3:GetObjectVersion" ], "Resource": "*" }, { "Sid": "S3PutObjectPolicy", "Effect": "Allow", "Action": [ "s3:PutObject" ], "Resource": "*" }, { "Sid": "S3BucketIdentity", "Effect": "Allow", "Action": [ "s3:GetBucketAcl", "s3:GetBucketLocation" ], "Resource": "*" } ] } ``` Finally, assign the policy to the role with the following command: ```bash lstk aws put-role-policy \ --role-name CodeBuildServiceRole \ --policy-name CodeBuildServiceRolePolicy \ --policy-document file://put-role-policy.json ``` ### Create the build project We now need to create a build project, containing all the information about how to run a build, where to get the source code, and where to place the output. You can use the CLI to generate the skeleton of the `CreateBuild` request, which you can later modify. Save the output of the following command to a file named `create-project.json`. ```bash lstk aws codebuild create-project --generate-cli-skeleton ``` From the generated file, change the source and the artifact location to match the S3 bucket names you just created. Similarly, fill in the ARN of the CodeBuild service role. ```json {hl_lines=[5,9,16]} showshowLineNumbers { "name": "codebuild-demo-project", "source": { "type": "S3", "location": "codebuild-demo-input" }, "artifacts": { "type": "S3", "location": "codebuild-demo-output" }, "environment": { "type": "LINUX_CONTAINER", "image": "aws/codebuild/standard:5.0", "computeType": "BUILD_GENERAL1_SMALL" }, "serviceRole": "service-role-arn" } ``` Now create the project with the following command: ```bash lstk aws codebuild create-project --cli-input-json file://create-project.json ``` You have now created a CodeBuild project called `codebuild-demo-project` that uses the S3 buckets you just created as source and artifact. :::note By default, LocalStack runs the all the builds in a Amazon Linux Container, ignoring the image provided in the `environment` parameter. See the [Build Environments](#build-environments) section for more details. ::: ### Run the build In this final step, you can now execute your build with the following command: ```bash lstk aws codebuild start-build --project-name codebuild-demo-project ``` Make note of the `id` information given in output, since it can be used to query the status of the build. If you inspect the running containers (e.g., with the `docker ps -a` command), you will notice a container with the `localstack-codebuild` prefix (followed by the build ID), which CodeBuild started to execute the build. This container will be responsible to start a Docker compose stack that executes the actual build. As said, you can inspect the status of the build with the following command: ```bash lstk aws codebuild batch-get-builds --ids ``` The command returns a list of builds. A build has a `buildStatus` attribute that will be set to `SUCCEEDED` if the build correctly terminates. :::note Each build goes through different phases, each of them having a start and end time, as well as a status. LocalStack does not provided such granular information. Currently, it reports only the final status of the build. ::: Once the build is completed, you can verify that the JAR artifact has been uploaded to the correct S3 bucket with the following command: ```bash lstk aws s3 ls s3://codebuild-demo-output ``` ## Build Environments LocalStack does not offer out-of-the-box all the build environments provided by AWS CodeBuild. By default, all the builds are executed in a Amazon Linux 2023 image (`public.ecr.aws/codebuild/amazonlinux-x86_64-standard:5.0` and `public.ecr.aws/codebuild/amazonlinux-aarch64-standard:3.0` for x86 and ARM, respectively). You can overcome this limitation by activating the `CODEBUILD_ENABLE_CUSTOM_IMAGES` environment variable. AWS shares the Dockerfiles of official AWS CodeBuild curated Docker images in a dedicated [GitHub repository](https://github.com/aws/aws-codebuild-docker-images). For instance, let us assume you want to run your builds on the Ubuntu `7.0` standard image. First, you have to build the image as follows: ```bash git clone https://github.com/aws/aws-codebuild-docker-images.git cd aws-codebuild-docker-images cd ubuntu/standard/7.0 docker build -t aws/codebuild/standard:7.0 . ``` Then, start LocalStack with `CODEBUILD_ENABLE_CUSTOM_IMAGES=1`. Finally, you can use the create image name, i.e., `aws/codebuild/standard:7.0` in the environment reference when you create you CodeBuild project. ## Limitations - CodeBuild currently only supports S3, NO_SOURCE, and CODEPIPELINE as [project source](https://docs.aws.amazon.com/codebuild/latest/APIReference/API_ProjectSource.html). - Custom build environments needs to have `bash` installed to properly work in LocalStack. - Environment variables in the `buildspec` are currently not supported. ## API Coverage # CodeCommit > Get started with CodeCommit on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction CodeCommit is a managed source control service by AWS that enables developers to store and collaborate on their code repositories. With CodeCommit, you can host private Git repositories with integrations to other AWS services. You can also use standard Git commands or CodeCommit APIs (using AWS CLI or SDKs) to manage your repositories. CodeCommit also uses identity-based policies, which can be attached to IAM users, groups, and roles, ensuring secure and granular access control. LocalStack allows you to use the CodeCommit APIs in your local environment to create new repositories, push your commits, and manage the repositories. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of CodeCommit's integration with LocalStack. ## Getting started This guide is designed for users new to CodeCommit and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how you can create a CodeCommit repository, clone a repository, and push a commit to the repository. ### Create a repository You can use the [`CreateRepository`](https://docs.aws.amazon.com/codecommit/latest/APIReference/API_CreateRepository.html) API to create a repository. You need to specify the repository name, repository description, and tags. Run the following command to create a new repository named `localstack-repo`: ```bash lstk aws codecommit create-repository \ --repository-name localstack-repo \ --repository-description "A demo repository to showcase LocalStack's CodeCommit" \ --tags Team=LocalStack ``` ```bash title="Output" { "repositoryMetadata": { "repositoryId": "", "repositoryName": "localstack-repo", "repositoryDescription": "A demo repository to showcase LocalStack's CodeCommit", "lastModifiedDate": "", "creationDate": "", "cloneUrlHttp": "git://localhost:4510/localstack-repo", "cloneUrlSsh": "git://localhost:4510/localstack-repo", "Arn": "arn:aws:codecommit:us-east-1:000000000000:localstack-repo" } } ``` ### Clone a repository Next, you can clone the CodeCommit repository to a local directory. To do so, you can use the [`git clone`](https://git-scm.com/docs/git-clone) command. The repository URL is the `cloneUrlHttp` value returned by the `CreateRepository` API. Run the following command to clone the repository to a local directory named `localstack-repo`: ```bash git clone git://localhost:4510/localstack-repo ``` You will notice that the repository is empty. This is because we have not pushed any commits to the repository yet. ### Push a commit Create a new file named `README.md` in the `localstack-repo` directory. Add some content to the file and save it. You can use [`git add`](https://git-scm.com/docs/git-add) to add the file to the staging area, followed by [`git commit`](https://git-scm.com/docs/git-commit) with a commit message, to commit the file to the repository. Then, you can use [`git commit`](https://git-scm.com/docs/git-commit) to commit the file to the repository. Run the following command to push the file to the repository: ```bash git add README.md git commit -m "Add README.md" git push ``` ```bash title="Output" ... To git://localhost:4510/localstack-repo * [new branch] main -> main ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing CodeCommit repositories. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resource Browser** section, and then clicking on **CodeCommit** under the **Developer Tools** section. ![CodeCommit Resource Browser](/images/aws/codecommit-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Repository**: Create a new CodeCommit repository by specifying the repository name and description, along with optional tags and KMS key ID. - **View Repository**: View the details of a CodeCommit repository, including the repository name, description, ARN, and clone URLs. - **Delete Repository**: Delete a CodeCommit repository by selecting the repository from the list and clicking the **Actions** dropdown menu followed by **Delete**. ## Examples You can find a sample application illustrating the usage of the CodeCommit APIs locally in the [`localstack-pro-samples` repository](https://github.com/localstack/localstack-pro-samples/tree/master/codecommit-git-repo). ## API Coverage # CodeConnections > Get started with CodeConnections on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; :::tip CodeConnections was formerly known as CodeStar Connections. ::: ## Introduction CodeConnections is a service provided by Amazon Web Services (AWS) that enables you to connect your code repositories to your AWS resources. It allows you to create connections to your code repositories and integrate them with supported AWS services. LocalStack provides a mock implementation of the CodeConnections API that allows you to create and manage connections to your code repositories. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of CodeConnections's integration with LocalStack. ## Getting started This guide is designed for users new to CodeConnections and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a connection to a code repository using the CodeConnections API. ### Create a connection You can create a connection to a code repository using the [`CreateConnection`](https://docs.aws.amazon.com/codeconnections/latest/APIReference/API_CreateConnection.html) API. ```bash lstk aws codeconnections create-connection \ --connection-name my-connection ``` You should see the connection in the output. ```bash title="Output" { "ConnectionArn": "arn:aws:codeconnections:eu-central-1:000000000000:connection/7c9f0d0b" } ### List connections You can list connections using the [`ListConnections`](https://docs.aws.amazon.com/codeconnections/latest/APIReference/API_ListConnections.html) API. ```bash lstk aws codeconnections list-connections ``` ```bash title="Output" { "Connections": [ { "ConnectionName": "my-connection", "ConnectionArn": "arn:aws:codeconnections:us-east-1:000000000000:connection/023ff7e3", "ProviderType": "GitHub", "OwnerAccountId": "000000000000", "ConnectionStatus": "AVAILABLE" } ] } ``` ### Get a connection You can get a connection using the [`GetConnection`](https://docs.aws.amazon.com/codeconnections/latest/APIReference/API_GetConnection.html) API. ```bash lstk aws codeconnections get-connection --connection-arn arn:aws:codeconnections:us-east-1:000000000000:connection/023ff7e3 ``` Replace the `connection-arn` with the ARN of the connection you want to get. ```bash title="Output" { "Connection": { "ConnectionName": "my-connection", "ConnectionArn": "arn:aws:codeconnections:us-east-1:000000000000:connection/023ff7e3", "ProviderType": "GitHub", "OwnerAccountId": "000000000000", "ConnectionStatus": "AVAILABLE" } } ``` ### Delete a connection You can delete a connection using the [`DeleteConnection`](https://docs.aws.amazon.com/codeconnections/latest/APIReference/API_DeleteConnection.html) API. ```bash lstk aws codeconnections delete-connection --connection-arn arn:aws:codeconnections:us-east-1:000000000000:connection/023ff7e3 ``` Replace the `connection-arn` with the ARN of the connection you want to delete. ## API Coverage # CodeDeploy > Get started with CodeDeploy on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction CodeDeploy is a service that automates application deployments. On AWS, it supports deployments to Amazon EC2 instances, on-premises instances, serverless Lambda functions, or Amazon ECS services. Furthermore, based on the target it is also possible to use an in-place deployment or a blue/green deployment. LocalStack supports a mocking of CodeDeploy API operations. The supported operations are listed on the [API Coverage section](#api-coverage). ## Getting Started This guide will walk through the process of creating CodeDeploy applications, deployment configuration, deployment groups, and deployments. Basic knowledge of the AWS CLI and the [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command is expected. Start LocalStack using your preferred method. ### Applications An application is a CodeDeploy construct that uniquely identifies your targetted application. Create an application with the [CreateApplication](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_CreateApplication.html) operation: ```bash lstk aws deploy create-application --application-name hello --compute-platform Server ``` ```bash title="Output" { "applicationId": "063714b6-f438-4b90-bacb-ce04af7f5e83" } ``` Make note of the application name, which can be used with other operations such as [GetApplication](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_GetApplication.html), [UpdateApplication](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_UpdateApplication.html) and [DeleteApplication](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_DeleteApplication.html). ```bash lstk aws deploy get-application --application-name hello ``` ```bash title="Output" { "application": { "applicationId": "063714b6-f438-4b90-bacb-ce04af7f5e83", "applicationName": "hello", "createTime": 1747663397.271634, "computePlatform": "Server" } } ``` You can list all application using [ListApplications](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_ListApplications.html). ```bash lstk aws deploy list-applications ``` ```bash title="Output" { "applications": [ "hello" ] } ``` ### Deployment configuration A deployment configuration consists of rules for deployment along with success and failure criteria. Create a deployment configuration using [CreateDeploymentConfig](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_CreateDeploymentConfig.html): ```bash lstk aws deploy create-deployment-config --deployment-config-name hello-conf \ --compute-platform Server \ --minimum-healthy-hosts '{"type": "HOST_COUNT", "value": 1}' ``` ```bash title="Output" { "deploymentConfigId": "0327ce0a-4637-4884-8899-49af7b9423b6" } ``` [ListDeploymentConfigs](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_ListDeploymentConfigs.html) can be used to list all available configs: ```bash lstk aws deploy list-deployment-configs ``` ```bash title="Output" { "deploymentConfigsList": [ "hello-conf" ] } ``` Use [GetDeploymentConfig](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_GetDeploymentConfig.html) and [DeleteDeploymentConfig](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_DeleteDeploymentConfig.html) to manage deployment configurations. ```bash lstk aws deploy get-deployment-config --deployment-config-name hello-conf ``` ```bash title="Output" { "deploymentConfigInfo": { "deploymentConfigId": "0327ce0a-4637-4884-8899-49af7b9423b6", "deploymentConfigName": "hello-conf", "minimumHealthyHosts": { "type": "HOST_COUNT", "value": 1 }, "createTime": 1747663716.208291, "computePlatform": "Server" } } ``` ### Deployment groups Deployment groups can be managed with: - [CreateDeploymentGroup](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_CreateDeploymentGroup.html) - [ListDeploymentGroups](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_ListDeploymentGroups.html) - [UpdateDeploymentGroup](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_UpdateDeploymentGroup.html) - [GetDeploymentGroup](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_GetDeploymentGroup.html) - [DeleteDeploymentGroup](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_DeleteDeploymentGroup.html) Create a deployment group with [CreateDeploymentGroup](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_CreateDeploymentGroup.html): ```bash lstk aws deploy create-deployment-group \ --application-name hello \ --service-role-arn arn:aws:iam::000000000000:role/role \ --deployment-group-name hello-group ``` ```bash title="Output" { "deploymentGroupId": "09506586-9ba9-4005-a1be-840407abb39d" } ``` List all deployment groups for an application with [ListDeploymentGroups](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_ListDeploymentGroups.html): ```bash lstk aws deploy list-deployment-groups --application-name hello ``` ```bash title="Output" { "deploymentGroups": [ "hello-group" ] } ``` Get a deployment group with [GetDeploymentGroup](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_GetDeploymentGroup.html): ```bash lstk aws deploy get-deployment-group --application-name hello \ --deployment-group-name hello-group ``` ```bash title="Output" { "deploymentGroupInfo": { "applicationName": "hello", "deploymentGroupId": "09506586-9ba9-4005-a1be-840407abb39d", "deploymentGroupName": "hello-group", "deploymentConfigName": "CodeDeployDefault.OneAtATime", "autoScalingGroups": [], "serviceRoleArn": "arn:aws:iam::000000000000:role/role", "triggerConfigurations": [], "deploymentStyle": { "deploymentType": "IN_PLACE", "deploymentOption": "WITHOUT_TRAFFIC_CONTROL" }, "outdatedInstancesStrategy": "UPDATE", "computePlatform": "Server", "terminationHookEnabled": false } } ``` ### Deployments Operations related to deployment management are: - [CreateDeployment](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_CreateDeployment.html) - [GetDeployment](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_GetDeployment.html) - [ListDeployments](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_ListDeployments.html) Create a deployment with [CreateDeployment](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_CreateDeployment.html): ```bash lstk aws deploy create-deployment \ --application-name hello \ --deployment-group-name hello-group \ --revision '{"revisionType": "S3", "s3Location": {"bucket": "placeholder", "key": "placeholder", "bundleType": "tar"}}' ``` ```bash title="Output" { "deploymentId": "d-TU3TNCSTO" } ``` List all deployments for an application with [ListDeployments](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_ListDeployments.html): ```bash lstk aws deploy list-deployments ``` ```bash title="Output" { "deployments": [ "d-TU3TNCSTO" ] } ``` Get a deployment with [GetDeployment](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_GetDeployment.html): ```bash lstk aws deploy get-deployment --deployment-id d-TU3TNCSTO ``` ```bash title="Output" { "deploymentInfo": { "applicationName": "hello", "deploymentGroupName": "hello-group", "deploymentConfigName": "CodeDeployDefault.OneAtATime", "deploymentId": "d-TU3TNCSTO", "revision": { "revisionType": "S3", "s3Location": { "bucket": "placeholder", "key": "placeholder", "bundleType": "tar" } }, "status": "Created", "createTime": 1747750522.133381, "creator": "user", "ignoreApplicationStopFailures": false, "updateOutdatedInstancesOnly": false, "deploymentStyle": { "deploymentType": "IN_PLACE", "deploymentOption": "WITHOUT_TRAFFIC_CONTROL" }, "instanceTerminationWaitTimeStarted": false, "fileExistsBehavior": "DISALLOW", "deploymentStatusMessages": [], "computePlatform": "Server" } } ``` Furthermore, [ContinueDeployment](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_StopDeployment.html) and [StopDeployment](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_StopDeployment.html) can be used to control the deployment flows: Continue a deployment with [ContinueDeployment](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_StopDeployment.html): ```bash lstk aws deploy continue-deployment --deployment-id d-TU3TNCSTO ``` Stop a deployment with [StopDeployment](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_StopDeployment.html): ```bash lstk aws deploy stop-deployment --deployment-id d-TU3TNCSTO ``` ```bash title="Output" { "status": "Succeeded", "statusMessage": "Mock deployment stopped" } ``` ## Limitations All CodeDeploy operations are currently mocked. ## API Coverage # CodePipeline > Get started with CodePipeline on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction CodePipeline is a continuous integration/continuous delivery (CI/CD) service offered by AWS. CodePipeline can be used to create automated pipelines that handle the build, test and deployment of software. LocalStack comes with a bespoke execution engine that can be used to create, manage, and execute pipelines. It supports a variety of actions that integrate with S3, CodeBuild, CodeConnections, and more. The available operations can be found on the [API coverage](#api-coverage) page. ## Getting started In this guide, we will create a simple pipeline that fetches an object from an S3 bucket and uploads it to a different S3 bucket. It is for users that are new to CodePipeline and have a basic knowledge of the AWS CLI and the [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start LocalStack using your preferred method. ### Create prerequisite buckets Begin by creating the S3 buckets that will serve as the source and target. ```bash lstk aws s3 mb s3://source-bucket lstk aws s3 mb s3://target-bucket ``` It is important to note the CodePipeline requires source S3 buckets to have versioning enabled. This can be done using the S3 [`PutBucketVersioning`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketVersioning.html) operation. ```bash lstk aws s3api put-bucket-versioning \ --bucket source-bucket \ --versioning-configuration Status=Enabled ``` Now create a placeholder file that will flow through the pipeline and upload it to the source bucket. ```bash echo "Hello LocalStack!" > file lstk aws s3 cp file s3://source-bucket ``` Pipelines also require an artifact store, which is also an S3 bucket that is used as intermediate storage. ```bash lstk aws s3 mb s3://artifact-store-bucket ``` ### Configure IAM Depending on the specifics of the declaration, CodePipeline pipelines need access other AWS services. In this case we want our pipeline to retrieve and upload files to S3. This requires a properly configured IAM role that our pipeline can assume. Create the role and make note of the role ARN: ```json showshowLineNumbers # role.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "codepipeline.amazonaws.com" }, "Action": "sts:AssumeRole" } ] } ``` Create the role with the following command: ```bash lstk aws iam create-role --role-name role --assume-role-policy-document file://role.json | jq .Role.Arn ``` Now add a permissions policy to this role that permits read and write access to S3. ```json showshowLineNumbers # policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:*" ], "Resource": "*" } ] } ``` The permissions in the above example policy are relatively broad. You might want to use a more focused policy for better security on production systems. ```bash lstk aws iam put-role-policy --role-name role --policy-name policy --policy-document file://policy.json ``` ### Create pipeline Now we can turn our attention to the pipeline declaration. A pipeline declaration is used to define the structure of actions and stages to be performed. The following pipeline defines two stages with one action each. There is a source action which retrieves a file from an S3 bucket and marks it as the output. The output is placed in the intermediate bucket until it is picked up by the action in the second stage. This is a deploy action which uploads the file to the target bucket. Pay special attention to `roleArn`, `artifactStore.location` as well as `S3Bucket`, `S3ObjectKey`, and `BucketName`. These correspond to the resources we created earlier. ```json {hl_lines=[6,9,26,27,52]} showshowLineNumbers # declaration.json { "name": "pipeline", "executionMode": "SUPERSEDED", "pipelineType": "V1", "roleArn": "arn:aws:iam::000000000000:role/role", "artifactStore": { "type": "S3", "location": "artifact-store-bucket" }, "version": 1, "stages": [ { "name": "stage1", "actions": [ { "name": "action1", "actionTypeId": { "category": "Source", "owner": "AWS", "provider": "S3", "version": "1" }, "runOrder": 1, "configuration": { "S3Bucket": "source-bucket", "S3ObjectKey": "file", "PollForSourceChanges": "false" }, "outputArtifacts": [ { "name": "intermediate-file" } ], "inputArtifacts": [] } ] }, { "name": "stage2", "actions": [ { "name": "action1", "actionTypeId": { "category": "Deploy", "owner": "AWS", "provider": "S3", "version": "1" }, "runOrder": 1, "configuration": { "BucketName": "target-bucket", "Extract": "false", "ObjectKey": "output-file" }, "inputArtifacts": [ { "name": "intermediate-file" } ], "outputArtifacts": [] } ] } ] } ``` Create the pipeline using the following command: ```bash lstk aws codepipeline create-pipeline --pipeline file://./declaration.json ``` ### Verify pipeline execution A 'pipeline execution' is an instance of a pipeline in a running or finished state. The [`CreatePipeline`](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_CreatePipeline.html) operation we ran earlier started a pipeline execution. This can be confirmed using: ```bash lstk aws codepipeline list-pipeline-executions --pipeline-name pipeline ``` ```bash title="Output" { "pipelineExecutionSummaries": [ { "pipelineExecutionId": "37e8eb2e-0ed9-447a-a016-8dbbd796bfe7", "status": "Succeeded", "startTime": 1745486647.138571, "lastUpdateTime": 1745486648.290341, "trigger": { "triggerType": "CreatePipeline" }, "executionMode": "SUPERSEDED" } ] } ``` Note the `trigger.triggerType` field specifies what initiated the pipeline execution. Currently in LocalStack, only two triggers are implemented: `CreatePipeline` and `StartPipelineExecution`. The above pipeline execution was successful. This means that we can retrieve the `output-file` object from the `target-bucket` S3 bucket. ```bash lstk aws s3 cp s3://target-bucket/output-file output-file ``` To verify that it is the same file as the original input: ```bash cat output-file ``` The output will be: ```text Hello LocalStack! ``` ### Examine action executions Using the [`ListActionExecutions`](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_ListPipelineExecutions.html), detailed information about each action execution such as inputs and outputs can be retrieved. This is useful when debugging the pipeline. ```bash lstk aws codepipeline list-action-executions --pipeline-name pipeline ``` ```bash title="Output" { "actionExecutionDetails": [ { "pipelineExecutionId": "37e8eb2e-0ed9-447a-a016-8dbbd796bfe7", "actionExecutionId": "e38716df-645e-43ce-9597-104735c7f92c", "pipelineVersion": 1, "stageName": "stage2", "actionName": "action1", "startTime": 1745486647.269867, "lastUpdateTime": 1745486647.289813, "status": "Succeeded", "input": { "actionTypeId": { "category": "Deploy", "owner": "AWS", "provider": "S3", "version": "1" }, "configuration": { "BucketName": "target-bucket", "Extract": "false", "ObjectKey": "output-file" }, "resolvedConfiguration": { "BucketName": "target-bucket", "Extract": "false", "ObjectKey": "output-file" }, "region": "eu-central-1", "inputArtifacts": [ { "name": "intermediate-file", "s3location": { "bucket": "artifact-store-bucket", "key": "pipeline/intermediate-file/01410aa4.zip" } } ] }, "output": { "outputArtifacts": [], "executionResult": { "externalExecutionId": "bcff0781", "externalExecutionSummary": "Deployment Succeeded" }, "outputVariables": {} } }, { "pipelineExecutionId": "37e8eb2e-0ed9-447a-a016-8dbbd796bfe7", "actionExecutionId": "ae99095a-1d43-46ee-8a48-c72b6d60021e", "pipelineVersion": 1, "stageName": "stage1", "actionName": "action1", ... ``` :::note LocalStack does not use the same logic to generate external execution IDs as AWS so there may be minor discrepancies. The same is true for status and error messages produced by actions. ::: ## Pipelines The operations [`CreatePipeline`](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_CreatePipeline.html), [`GetPipeline`](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_GetPipeline.html), [`UpdatePipeline`](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_UpdatePipeline.html), [`ListPipelines`](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_ListPipelines.html), [`DeletePipeline`](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_DeletePipeline.html) are used to manage pipeline declarations. LocalStack supports emulation for V1 pipelines. V2 pipelines are only created as mocks. :::tip Emulation for V2 pipelines is not supported. Make sure that the pipeline type is explicitly set in the declaration. ::: Pipeline executions can be managed with: - [`StartPipelineExecution`](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_StartPipelineExecution.html) - [`GetPipelineExecution`](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_GetPipelineExecution.html) - [`ListPipelineExecutions`](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_ListPipelineExecutions.html) - [`StopPipelineExecutions`](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_StopPipelineExecution.html) When stopping pipeline executions with `StopPipelineExecution`, the stop and abandon method is not supported. Setting the `abandon` flag will have no impact. This is because LocalStack uses threads as the underlying mechanism to simulate pipelines, and threads cannot be cleanly preempted. Action executions can be inspected using the [`ListActionExecutions`](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_ListPipelineExecutions.html) operation. ### Tagging pipelines Pipelines resources can be [tagged](https://docs.aws.amazon.com/codepipeline/latest/userguide/pipelines-tag.html) using the following operations: - [`TagResource`](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_TagResource.html) - [`UntagResource`](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_UntagResource.html) - [`ListTagsForResource`](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_ListTagsForResource.html) Tag the pipeline with the following command: ```bash lstk aws codepipeline tag-resource \ --resource-arn arn:aws:codepipeline:eu-central-1:000000000000:pipeline \ --tags key=purpose,value=tutorial lstk aws codepipeline list-tags-for-resource \ --resource-arn arn:aws:codepipeline:eu-central-1:000000000000:pipeline ``` ```bash title="Output" { "tags": [ { "key": "purpose", "value": "tutorial" } ] } ``` Untag the pipeline with the following command: ```bash lstk aws codepipeline untag-resource \ --resource-arn arn:aws:codepipeline:eu-central-1:000000000000:pipeline \ --tag-keys purpose ``` ## Variables CodePipeline on LocalStack supports [variables](https://docs.aws.amazon.com/codepipeline/latest/userguide/reference-variables.html) which allow dynamic configuration of pipeline actions. Actions produce output variables which can be referenced in the configuration of subsequent actions. Make note that only when the action defines a namespace, its output variables are availabe to downstream actions. :::tip If an action does not use a namespace, its output variables are not available to downstream actions. ::: CodePipeline's variable placeholder syntax is as follows: ```text #{namespace.variable} ``` As with AWS, LocalStack only makes the `codepipeline.PipelineExecutionId` variable available by default in a pipeline. ## Actions You can use [`runOrder`](https://docs.aws.amazon.com/codepipeline/latest/userguide/action-requirements.html#action.runOrder) to control parallel or sequential order of execution of actions. The supported actions in LocalStack CodePipeline are listed below. Using an unsupported action will make the pipeline fail. If you would like support for more actions, please [raise a feature request on GitHub Discussion](https://github.com/orgs/localstack/discussions/new/choose). ### CloudFormation Deploy The [CloudFormation Deploy](https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-CloudFormation.html) action executes a CloudFormation stack. It supports the following modes: `CREATE_UPDATE`, `CHANGE_SET_REPLACE`, `CHANGE_SET_EXECUTE` ### CodeBuild Source and Test The [CodeBuild Source and Test](https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-CodeBuild.html) action can be used to start a CodeBuild container and run the given buildspec. ### CodeConnections Source The [CodeConnections Source](https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-CodestarConnectionSource.html) action is used to specify a VCS repo as the input to the pipeline. LocalStack supports integration only with [GitHub](https://github.com/) at this time. Please set the environment configuration option `CODEPIPELINE_GH_TOKEN` with the GitHub Personal Access Token to be able to fetch private repositories. ### ECR Source The [ECR Source](https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-ECR.html) action is used to specify an Elastic Container Registry image as a source artifact. ### ECS CodeDeploy Blue/Green The [ECS CodeDeply Blue/Green](https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-ECSbluegreen.html) action is used to deploy container application using a blue/green deployment. LocalStack does not accurately emulate a blue/green deployment due to limitations in ELB and ECS. It will only update the running ECS service with a new task definition and wait for the service to be stable. ### ECS Deploy The [ECS Deploy](https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-ECS.html) action creates a revision of a task definition based on an already deployed ECS service. ### Lambda Invoke The [Lambda Invoke](https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-Lambda.html) action is used to execute a Lambda function in a pipeline. ### Manual Approval The Manual Approval action can be included in the pipeline declaration but it will only function as a no-op. ### S3 Deploy The [S3 Deploy](https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-S3Deploy.html) action is used to upload artifacts to a given S3 bucket as the output of the pipeline. ### S3 Source The [S3 Source](https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-S3.html) action is used to specify an S3 bucket object as input to the pipeline. ## Limitations - Emulation for [V2 pipeline types](https://docs.aws.amazon.com/codepipeline/latest/userguide/pipeline-types-planning.html) is not supported. They will be created as mocks only. - [Rollbacks and stage retries](https://docs.aws.amazon.com/codepipeline/latest/userguide/pipelines-stages.html) are not available. - [Custom actions](https://docs.aws.amazon.com/codepipeline/latest/userguide/actions-create-custom-action.html) and associated operations (AcknowledgeJob, GetJobDetails, PollForJobs, etc.) are not supported. - [Triggers](https://docs.aws.amazon.com/codepipeline/latest/userguide/pipelines-triggers.html) are not implemented. Pipelines are executed only when [CreatePipeline](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_CreatePipeline.html) and [StartPipelineExecution](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_StartPipelineExecution.html) are invoked. - [Execution mode behaviours](https://docs.aws.amazon.com/codepipeline/latest/userguide/concepts-how-it-works.html#concepts-how-it-works-executions) are not implemented. Parallel pipeline executions will not lead to stage locks and waits. - [Stage transition controls](https://docs.aws.amazon.com/codepipeline/latest/userguide/transitions.html) are not implemented. - [Manual approval action](https://docs.aws.amazon.com/codepipeline/latest/userguide/approvals-action-add.html) and [PutApprovalResult](https://docs.aws.amazon.com/codepipeline/latest/APIReference/API_PutApprovalResult.html) operations are not available. ## API Coverage # Cognito > Get started with Cognito on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Cognito is a managed identity service provided by AWS that is used for securing user authentication, authorization, and managing user identities in web and mobile applications. Cognito enables developers to add user sign-up, sign-in, and access control functionalities to their applications. Cognito supports various authentication methods, including social identity providers, SAML-based identity providers, and custom authentication flows. LocalStack allows you to use the Cognito APIs in your local environment to manage authentication and access control for your local application and resources. The supported APIs are available on our [Cognito Identity coverage section](#api-coverage-cognito-identity-pools) and [Cognito User Pools coverage section](#api-coverage-cognito-user-pools), which provides information on the extent of Cognito's integration with LocalStack. ## Getting started This guide is designed for users new to Cognito and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how you can create a Cognito user pool and client, and then sign up and authenticate a new user in the pool. ### Creating a User Pool To create a user pool, you can use the [`CreateUserPool`](https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_CreateUserPool.html) API call. The following command creates a user pool named `test`: ```bash lstk aws cognito-idp create-user-pool --pool-name test ``` ```bash title="Output" "UserPool": { "Id": "us-east-1_fd924693e9b04f549f989283123a29c2", "Name": "test", "Policies": { "PasswordPolicy": { "MinimumLength": 8, "RequireUppercase": true, "RequireLowercase": true, "RequireNumbers": true, "RequireSymbols": true, "TemporaryPasswordValidityDays": 7 } }, "LastModifiedDate": "2021-10-06T11:57:21.883Z", "CreationDate": "2021-10-06T11:57:21.883Z", "SchemaAttributes": [], "VerificationMessageTemplate": { "DefaultEmailOption": "CONFIRM_WITH_CODE" }, "EmailConfiguration": { "EmailSendingAccount": "COGNITO_DEFAULT" }, "AdminCreateUserConfig": { "AllowAdminCreateUserOnly": false }, "Arn": "arn:aws:cognito-idp:us-east-1:000000000000:userpool/us-east-1_fd924693e9b04f549f989283123a29c2" } ``` You will need the user pool's `id` for further operations. Save it in a `pool_id` variable: ```bash pool_id= ``` Alternatively, you can use JSON processor like [`jq`](https://stedolan.github.io/jq/) to extract the essential information right from the outset when creating a pool. ```bash pool_id=$(lstk aws cognito-idp create-user-pool --pool-name test | jq -rc ".UserPool.Id") ``` ### Adding a Client You can proceed with adding a client to the pool we just created. You will require the ID of the newly created client for the subsequent steps. You can use the [`CreateUserPoolClient`](https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_CreateUserPoolClient.html) for both client creation and extraction of the corresponding ID. Run the following command: ```bash client_id=$(lstk aws cognito-idp create-user-pool-client --user-pool-id $pool_id --client-name test-client | jq -rc ".UserPoolClient.ClientId") ``` ### Using Predefined IDs for Pool Creation When creating Cognito user or identity pools, you have the flexibility to utilize a predefined ID by setting the tag `_custom_id_`. This feature proves particularly useful during the testing of authentication flows, especially when dealing with scenarios involving frequent restarts of LocalStack and the recreation of resources. Please note that a valid custom id must be in the format `_`. Run the following command to create a user pool with a predefined ID: ```bash lstk aws cognito-idp create-user-pool --pool-name p1 --user-pool-tags "_custom_id_=us-east-1_myid123" ``` ```bash title="Output" { "UserPool": { "Id": "myid123", "Name": "p1", ... ``` You also have the possibility to create a Cognito user pool client with a predefined ID by specifying a `ClientName` with the specific format: `_custom_id_:`. ```bash lstk aws cognito-idp create-user-pool-client --user-pool-id us-east-1_myid123 --client-name _custom_id_:myclient123 ``` ```bash title="Output" { "UserPoolClient": { "UserPoolId": "us-east-1_myid123", "ClientName": "_custom_id_:myclient123", "ClientId": "myclient123", ... ``` ### Signing up and confirming a user You can now use the [`SignUp`](https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_SignUp.html) API to sign up a user. Run the following command: ```bash lstk aws cognito-idp sign-up \ --client-id $client_id \ --username example_user \ --password 12345678Aa! \ --user-attributes Name=email,Value= ``` ```bash title="Output" { "UserConfirmed": false, "UserSub": "5fdbe1d5-7901-4fee-9d1d-518103789c94" } ``` Once the user is successfully created, a confirmation code will be generated. This code can be found in the LocalStack container logs (as shown below). Additionally, if you have [SMTP configured](/aws/customization/configuration-options/#emails), the confirmation code can be optionally sent via email for enhanced convenience and user experience. ```bash INFO:localstack_ext.services.cognito.cognito_idp_api: Confirmation code for Cognito user example_user: 125796 DEBUG:localstack_ext.bootstrap.email_utils: Sending confirmation code via email to "your.email@address.com" ``` You can confirm the user with the activation code, using the [`ConfirmSignUp`](https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_ConfirmSignUp.html) API. Execute the following command: ```bash lstk aws cognito-idp confirm-sign-up \ --client-id $client_id \ --username example_user \ --confirmation-code ``` Since the above command does not provide a direct response, we need to verify the success of the request by checking the pool. Run the following command to use the [`ListUsers`](https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_ListUsers.html) API to list the users in the pool: ```bash lstk aws cognito-idp list-users --user-pool-id $pool_id ``` ```bash title="Output" { "Users": [ { "Username": "example_user", "Attributes": [ { "Name": "email", "Value": "your.email@address.com" }, { "Name": "sub", "Value": "5fdbe1d5-7901-4fee-9d1d-518103789c94" }, { "Name": "cognito:username", "Value": "example_user" } ], "Enabled": true, "UserStatus": "CONFIRMED" } ] } ``` ## Multi-factor authentication (MFA) LocalStack challenges for multi-factor authentication during the sign-in flow. The following MFA factors are supported: - **SMS text message** (`SMS_MFA`) - **Software token / TOTP** (`SOFTWARE_TOKEN_MFA`), such as Google Authenticator or Authy - **Email** (`EMAIL_OTP`) MFA is controlled at two levels. You configure the available factors and the enforcement mode at the pool level with [`SetUserPoolMfaConfig`](https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_SetUserPoolMfaConfig.html), and you enable specific factors per user with [`AdminSetUserMFAPreference`](https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_AdminSetUserMFAPreference.html) or [`SetUserMFAPreference`](https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_SetUserMFAPreference.html). The pool's `MfaConfiguration` determines whether a challenge is issued: - `OFF`: no MFA challenge is ever issued. - `OPTIONAL`: a challenge is issued only when the user has an MFA factor enabled in their preferences. - `ON`: an MFA challenge is always required. When a user has multiple factors enabled, the one marked as `PreferredMfa` is selected. ### Software token (TOTP) MFA First, enable software token MFA at the pool level. Setting `MfaConfiguration` to `OPTIONAL` enforces MFA per-user based on their preferences: ```bash lstk aws cognito-idp set-user-pool-mfa-config \ --user-pool-id $pool_id \ --software-token-mfa-configuration Enabled=true \ --mfa-configuration OPTIONAL ``` Sign in once to obtain an access token, then associate a software token for the user. This returns a `SecretCode` that you register in your authenticator app: ```bash lstk aws cognito-idp associate-software-token --access-token ``` ```bash title="Output" { "SecretCode": "QDWSDFGRVASRWQRRWE..." } ``` Verify the token by submitting a code generated from the secret, then set the software token as the user's preferred MFA factor: ```bash lstk aws cognito-idp verify-software-token \ --access-token \ --user-code lstk aws cognito-idp set-user-mfa-preference \ --access-token \ --software-token-mfa-settings Enabled=true,PreferredMfa=true ``` On the next sign-in, the authentication flow now returns a `SOFTWARE_TOKEN_MFA` challenge instead of issuing tokens directly: ```bash lstk aws cognito-idp initiate-auth \ --client-id $client_id \ --auth-flow USER_PASSWORD_AUTH \ --auth-parameters USERNAME=example_user,PASSWORD=12345678Aa! ``` ```bash title="Output" { "ChallengeName": "SOFTWARE_TOKEN_MFA", "Session": "abcd1234...", "ChallengeParameters": { "USER_ID_FOR_SRP": "example_user" } } ``` Respond to the challenge with a fresh TOTP code to complete authentication: ```bash lstk aws cognito-idp respond-to-auth-challenge \ --client-id $client_id \ --challenge-name SOFTWARE_TOKEN_MFA \ --session \ --challenge-responses USERNAME=example_user,SOFTWARE_TOKEN_MFA_CODE= ``` An invalid code is rejected with a `CodeMismatchException`. ### Email MFA Enable email MFA at the pool level by providing an `EmailMfaConfiguration`. The configuration is persisted and returned by [`GetUserPoolMfaConfig`](https://docs.aws.amazon.com/cognito-user-identity-pools/latest/APIReference/API_GetUserPoolMfaConfig.html): ```bash lstk aws cognito-idp set-user-pool-mfa-config \ --user-pool-id $pool_id \ --email-mfa-configuration 'Message="Your code is {####}",Subject="Your verification code"' \ --mfa-configuration OPTIONAL ``` Set email as the user's preferred MFA factor. The user must have a verified `email` attribute: ```bash lstk aws cognito-idp admin-set-user-mfa-preference \ --user-pool-id $pool_id \ --username example_user \ --email-mfa-settings Enabled=true,PreferredMfa=true ``` `AdminGetUser` now surfaces `EMAIL_OTP` in the user's MFA settings: ```bash lstk aws cognito-idp admin-get-user --user-pool-id $pool_id --username example_user ``` ```bash title="Output" { ... "UserMFASettingList": [ "EMAIL_OTP" ], "PreferredMfaSetting": "EMAIL_OTP" } ``` On the next sign-in, the flow returns an `EMAIL_OTP` challenge: ```bash title="Output" { "ChallengeName": "EMAIL_OTP", "Session": "abcd1234...", "ChallengeParameters": { "CODE_DELIVERY_DELIVERY_MEDIUM": "EMAIL", "CODE_DELIVERY_DESTINATION": "e***@example.com" } } ``` The one-time code is printed to the LocalStack container logs (and sent via email if [SMTP is configured](/aws/customization/configuration-options/#emails)): ```bash INFO --- [et.reactor-0] l.p.c.s.c.auth_flows : Code verification sent via email: 123456 ``` Respond to the challenge with the emailed code to complete authentication: ```bash lstk aws cognito-idp respond-to-auth-challenge \ --client-id $client_id \ --challenge-name EMAIL_OTP \ --session \ --challenge-responses USERNAME=example_user,EMAIL_OTP_CODE= ``` As with TOTP, an invalid code is rejected with a `CodeMismatchException`. :::note LocalStack cannot deliver or verify real SMS messages locally. For `SMS_MFA` challenges, the `SMS_MFA_CODE` parameter must be present in the challenge response but its value is not validated. ::: ## JWT Token Issuer and JSON Web Key Sets (JWKS) endpoints When Cognito creates JWT tokens, they include an issuer (`iss`) attribute that specifies the endpoint of the corresponding user pool. Generally, the issuer endpoint follows this format, with `` being the ID of the Cognito user pool: ```bash http://localhost:4566/ ``` However, depending on your specific configurations, there might be slight variations in the issuer URL, such as: ```bash https://cognito-idp.localhost.localstack.cloud/ ``` To access the JSON Web Key Sets (JWKS) configuration for each user pool, you can use the standardized well-known URL below: ```bash curl 'http://localhost:4566//.well-known/jwks.json' ``` ```bash title="Output" {"keys": [{"kty": "RSA", "alg": "RS256", "use": "sig", "kid": "test-key", "n": "k6lrbEH..."]} ``` Moreover, you can retrieve the global region-specific public keys for Cognito Identity Pools using the following endpoint: ```bash curl http://localhost:4566/.well-known/jwks_uri ``` The output will be similar to the following: ```bash {"keys": [{"kty": "RSA", "alg": "RS512", "use": "sig", "kid": "ap-northeast-11", "n": "AI7mc1assO5..."]} ``` ## Cognito Lambda Triggers Cognito offers a variety of lifecycle hooks called Cognito Lambda triggers, which allow you to react to different lifecycle events and customize the behavior of user signup, confirmation, migration, and more. To illustrate, suppose you wish to define a _user migration_ Lambda trigger in order to migrate users from your existing user directory into Amazon Cognito user pools at sign-in. In this case, you can start by creating a Lambda function, let's say named `"migrate_users"`, responsible for performing the migration by creating a new file `index.js` with the following code: ```javascript showshowLineNumbers const validUsers = { belladonna: { password: "12345678Aa!", emailAddress: "bella@example.com" }, }; // Replace this mock with a call to a real authentication service. const authenticateUser = (username, password) => { if (validUsers[username] && validUsers[username].password === password) { return validUsers[username]; } else { return null; } }; const lookupUser = (username) => { const user = validUsers[username]; if (user) { return { emailAddress: user.emailAddress }; } else { return null; } }; exports.handler = async (event) => { if (event.triggerSource == "UserMigration_Authentication") { // Authenticate the user with your existing user directory service const user = authenticateUser(event.userName, event.request.password); if (user) { event.response.userAttributes = { email: user.emailAddress, email_verified: "true", }; event.response.finalUserStatus = "CONFIRMED"; event.response.messageAction = "SUPPRESS"; } } else if (event.triggerSource == "UserMigration_ForgotPassword") { // Look up the user in your existing user directory service const user = lookupUser(event.userName); if (user) { event.response.userAttributes = { email: user.emailAddress, // Required to enable password-reset code to be sent to user email_verified: "true", }; event.response.messageAction = "SUPPRESS"; } } return event; }; ``` Enter the following commands to create the Lambda function: ```bash zip function.zip index.js lstk aws lambda create-function \ --function-name migrate_users \ --runtime nodejs18.x \ --zip-file fileb://function.zip \ --handler index.handler \ --role arn:aws:iam::000000000000:role/lambda-role ``` Subsequently, you can define the corresponding `--lambda-config` when creating the user pool to link it with the Lambda function: ```bash lstk aws cognito-idp create-user-pool \ --pool-name test2 \ --lambda-config '{"UserMigration":"arn:aws:lambda:us-east-1:000000000000:function:migrate_users"}' ``` Upon successful authentication of a non-registered user, Cognito will automatically trigger the migration Lambda function, allowing the user to be added to the pool after migration. ## OAuth Flows via Cognito Login Form You can access the local [Cognito login form](https://docs.aws.amazon.com/cognito/latest/developerguide/login-endpoint.html) by entering the following URL in your web browser: ```bash https://localhost.localstack.cloud/_aws/cognito-idp/login?response_type=code&client_id=&redirect_uri= ``` Replace `` with the ID of your existing user pool client (for example, `example_user`), and `` with the redirect URI specific to your application (e.g., `http://example.com`). The login form should look similar to the screenshot below: ![Cognito Login Form](/images/aws/cognitoLogin.png) Upon successful login, the page will automatically redirect to the designated ``, with an appended path parameter `?code=`. For instance, the redirect URL might look like `http://example.com?code=test123`. To obtain a token, you need to submit the received code using `grant_type=authorization_code` to LocalStack's implementation of the Cognito OAuth2 TOKEN Endpoint, which is documented [on the AWS Cognito Token endpoint page](https://docs.aws.amazon.com/cognito/latest/developerguide/token-endpoint.html). Note that the value of the `redirect_uri` parameter in your token request must match the value provided during the login process. Ensuring this match is crucial for the proper functioning of the authentication flow. ```bash curl \ --data-urlencode 'grant_type=authorization_code' \ --data-urlencode 'redirect_uri=http://example.com' \ --data-urlencode "client_id=${client_id}" \ --data-urlencode 'code=test123' \ 'http://localhost:4566/_aws/cognito-idp/oauth2/token' ``` ```bash title="Output" {"access_token": "eyJ0eXAi…lKaHx44Q", "expires_in": 86400, "token_type": "Bearer", "refresh_token": "e3f08304", "id_token": "eyJ0eXAi…ADTXv5mA"} ``` If your use case requires it, you can enforce https in URLs used by the auth flow by setting the environment variable `USE_SSL=1` ### Client credentials grant The client credentials grant is designed for machine-to-machine (M2M) communication. The Client Credentials Grant allows the machine (client) to authenticate itself directly with the authorization server using its credentials, such as a client ID and client secret. The client credentials grant allows for scope-based authorization from a non-interactive system to an API. Your app can directly request client credentials from the token endpoint to receive an access token. To request the token from the LocalStack URL, use the following endpoint: `://cognito-idp.localhost.localstack.cloud:4566/_aws/cognito-idp/oauth2/token`. For additional information on our endpoints, refer to our [Internal Endpoints](/aws/customization/networking/internal-endpoints/) documentation. If there are multiple user pools, LocalStack identifies the appropriate one by examining the `clientid` of the request. To get started, follow the example below: ```bash #Create client user pool with a client. export client_id=$(lstk aws cognito-idp create-user-pool-client --user-pool-id $pool_id --client-name test-client --generate-secret | jq -rc ".UserPoolClient.ClientId") #Retrieve secret. export client_secret=$(lstk aws cognito-idp describe-user-pool-client --user-pool-id $pool_id --client-id $client_id | jq -r '.UserPoolClient.ClientSecret') #Create resource server lstk aws cognito-idp create-resource-server \ --user-pool-id $pool_id \ --identifier "api-client-organizations" \ --name "Resource Server Name" \ --scopes '[{"ScopeName":"read","ScopeDescription":"Read access to Organizations"}]' ``` You can retrieve the token from your application using the specified endpoint: `http://cognito-idp.localhost.localstack.cloud:4566/_aws/cognito-idp/oauth2/token`. ```javascript showshowLineNumbers require('dotenv').config(); const axios = require('axios'); async function getAccessTokenWithSecret() { const clientId = process.env.client_id; const clientSecret = process.env.client_secret; const scope = 'api-client-organizations/read'; const url = 'http://cognito-idp.localhost.localstack.cloud:4566/_aws/cognito-idp/oauth2/token'; const authHeader = Buffer.from(`${clientId}:${clientSecret}`).toString('base64'); const headers = { 'Content-Type': 'application/x-www-form-urlencoded', 'Authorization': `Basic ${authHeader}` }; const payload = new URLSearchParams({ grant_type: 'client_credentials', client_id: clientId, scope: scope }); try { const response = await axios.post(url, payload, { headers }); console.log(response.data); } catch (error) { console.error('Error fetching access token:', error.response ? error.response.data : error.message); } } getAccessTokenWithSecret(); ``` ## Serverless and Cognito Furthermore, you have the option to combine Cognito and LocalStack seamlessly with the [Serverless framework](https://www.serverless.com/). For instance, consider this snippet from a `serverless.yml` configuration: ```yaml showshowLineNumbers service: test plugins: - serverless-deployment-bucket - serverless-pseudo-parameters - serverless-localstack custom: localstack: stages: [local] functions: http_request: handler: http.request events: - http: path: v1/request authorizer: arn: arn:aws:cognito-idp:us-east-1:#{AWS::AccountId}:userpool/ExampleUserPool resources: Resources: UserPool: Type: AWS::Cognito::UserPool Properties: ... ``` After configuring the Serverless setup, you can deploy it using `serverless deploy --stage local`. The provided example includes a Lambda function called `http_request` that's linked to an API Gateway endpoint. Once deployed, the `v1/request` API Gateway endpoint will be protected by the Cognito user pool named "`ExampleUserPool`". As a result, you can register users against the local pool using the same API calls as you would with AWS. To send requests to the secured API Gateway endpoint, you need to fetch identity credentials from the local Cognito API. These credentials can then be included as `Authentication` HTTP headers (where `test-1234567` represents the name of the access key ID generated by Cognito): ```bash Authentication: AWS4-HMAC-SHA256 Credential=test-1234567/20190821/us-east-1/cognito-idp/aws4_request ... ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing Cognito User Pools, and more. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **Cognito** under the **Security Identity Compliance** section. ![Cognito Resource Browser](/images/aws/cognito-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create User Pool**: Create a new Cognito User Pool, by specifying the pool name, policies, and other settings. - **View User Pools**: View a list of all existing Cognito User Pools, including their **Details**, **Groups**, and **Users**. - **Edit User Pool**: Edit an existing Cognito User Pool, by adding additional configurations, policies, and more. - **Create Group**: Add a new Group to an existing Cognito User Pool, by specifying the group name, description, Role Arn, and Precedence. - **Create User**: Add a new User to an existing Cognito User Pool, by specifying the user name, user attributes, and more. - **Remove Selected**: Remove the selected User Pool, Group, or User from the list of existing Cognito resources. ## Examples The following code snippets and sample applications provide practical examples of how to use Cognito in LocalStack for various use cases: - [Running Cognito authentication and user pools locally](https://github.com/localstack/localstack-pro-samples/tree/master/cognito-jwt) - [Serverless Container-based APIs with ECS & API Gateway](https://github.com/localstack/serverless-api-ecs-apigateway-sample) - [Step-up Authentication using Cognito](https://github.com/localstack/step-up-auth-sample) ## Current Limitations By default, LocalStack's Cognito does not send actual email messages. However, if you wish to enable this feature, you will need to provide an email address and configure the corresponding SMTP settings. The instructions on configuring the connection parameters of your SMTP server can be found in the [Configuration](/aws/customization/configuration-options/#emails) guide to allow your local Cognito environment to send email notifications. ## API Coverage (Cognito Identity Pools) ## API Coverage (Cognito User Pools) # Config > Get started with Config on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction AWS Config is a service provided by Amazon Web Services (AWS) that enables you to assess, audit, and manage the configuration state of your AWS resources. Config provides a comprehensive view of the resource configuration across your AWS environment, helping you ensure compliance with security policies, track changes, and troubleshoot operational issues. Config continuously records configuration changes and allows you to retain a historical record of these changes. LocalStack allows you to use the Config APIs in your local environment to assesses resource configurations and notifies you of any non-compliant items to mitigate potential security risks. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Config's integration with LocalStack. ## Getting started This guide is designed for users new to Config and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to specify the resource types you want Config to record and grant it the needful permissions to access an S3 bucket and SNS topic with the AWS CLI. ### Create an S3 bucket and SNS topic The S3 bucket will be used to receive a configuration snapshot on request and configuration history. The SNS topic will be used to notify you when a configuration snapshot is available. You can create a new S3 bucket and SNS topic using the AWS CLI: ```bash lstk aws s3 mb s3://config-test lstk aws sns create-topic --name config-test-topic ``` ### Create a new configuration recorder You can now create a new configuration recorder to record configuration changes for specified resource types, using the [`PutConfigurationRecorder`](https://docs.aws.amazon.com/config/latest/APIReference/API_PutConfigurationRecorder.html) API. Run the following command to create a new configuration recorder: ```bash lstk aws configservice put-configuration-recorder \ --configuration-recorder name=default,roleARN=arn:aws:iam::000000000000:role/config-role ``` We have specified the `roleARN` parameter to grant the configuration recorder the needful permissions to access the S3 bucket and SNS topic. In LocalStack, IAM roles are not enforced, so you can specify any role ARN you like. The `name` parameter has been set to `default`, and you can optionally specify a `recordingGroup` parameter to specify the resource types you want to record. ### Create a delivery channel You can now create a delivery channel object to deliver configuration information to an S3 bucket and an SNS topic. You have already created the S3 bucket and SNS topic, so you can now create the delivery channel object using the [`PutDeliveryChannel`](https://docs.aws.amazon.com/config/latest/APIReference/API_PutDeliveryChannel.html) API. We're going to create a delivery channel with the following configuration. You can inline the JSON into the `lstk aws` command. ```json { "name": "default", "s3BucketName": "config-test", "snsTopicARN": "arn:aws:sns:us-east-1:000000000000", "configSnapshotDeliveryProperties": { "deliveryFrequency": "Twelve_Hours" } } ``` Run the following command to create the delivery channel: ```bash lstk aws configservice put-delivery-channel \ --delivery-channel '{ "name": "default", "s3BucketName": "config-test", "snsTopicARN": "arn:aws:sns:us-east-1:000000000000", "configSnapshotDeliveryProperties": { "deliveryFrequency": "Twelve_Hours" } }' ``` ### Start the configuration recorder You can now start recording configurations of the local AWS resources you have selected to record in your running LocalStack container. You can use the [`StartConfigurationRecorder`](https://docs.aws.amazon.com/config/latest/APIReference/API_StartConfigurationRecorder.html) API to start the configuration recorder. Run the following command to start the configuration recorder: ```bash lstk aws configservice start-configuration-recorder \ --configuration-recorder-name default ``` You can list the delivery channels and configuration recorders using the [`DescribeDeliveryChannels`](https://docs.aws.amazon.com/config/latest/APIReference/API_DescribeDeliveryChannels.html) and [`DescribeConfigurationRecorderStatus`](https://docs.aws.amazon.com/config/latest/APIReference/API_DescribeConfigurationRecorderStatus.html) APIs respectively. ```bash lstk aws configservice describe-delivery-channels lstk aws configservice describe-configuration-recorder-status ``` ## Current Limitations AWS Config is currently mocked in LocalStack. You can create, read, update, and delete AWS Config resources (like delivery channels or configuration recorders), but LocalStack will currently not record any configuration changes to service resources. If you need this feature, please consider opening a [feature request on GitHub Discussion](https://github.com/orgs/localstack/discussions/new/choose). ## API Coverage # Database Migration Service (DMS) > Get started with Database Migration Service (DMS) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction AWS Database Migration Service provides migration solution from databases, data warehouses, and other type of data stores (e.g. S3, SAP). The migration can be homogeneous (source and target have the same type), but often times is heterogeneous as it supports migration from various sources to various targets (self-hosted and AWS services). LocalStack only supports selected use cases for DMS at the moment. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of DMS integration with LocalStack. :::note DMS is in a preview state, supporting only [selected use cases](#supported-use-cases). You need to set the env `ENABLE_DMS=1` in order to activate it. ::: ## Getting started You can run a DMS sample showcasing MariaDB source and Kinesis target from our [GitHub repository](https://github.com/localstack-samples/sample-dms-kinesis-rds-mariadb/). * The sample is using CDK to setup the infrastructure. * It setups two databases: one external MariaDB (starting in a docker container) and one RDS MariaDB. * It creates two `cdc` replication tasks, with different table mappings, that will run against the RDS database, * and two `full-load` replication tasks with different table mappings, running against the hosted (containerized) MariaDB. To follow the sample, simply clone the repository: ```bash git clone https://github.com/localstack-samples/sample-dms-kinesis-rds-mariadb.git ``` Next, start LocalStack (there is a docker-compose included, setting the `ENABLE_DMS=1` flag): ```bash export LOCALSTACK_AUTH_TOKEN= # this must be a enterprise license token docker-compose up ``` Now you can install the dependencies, deploy the resources, and run the tests: ```bash # install dependencies make install # deploys cdk stack with all required resources (replication instances, tasks, endpoints) make deploy # starts the tasks make run ``` You will then see some log output, indicating the status of the ongoing replication: ```bash ************ STARTING FULL LOAD FLOW ************ db endpoint: localhost:3306 Cleaning tables Creating tables Inserting data Added the following authors [{'first_name': 'John', 'last_name': 'Doe'}] Added the following accounts [{'account_balance': Decimal('1500.00'), 'name': 'Alice'}] Added the following novels [{'author_id': 1, 'title': 'The Great Adventure'}, {'author_id': 1, 'title': 'Journey to the Stars'}] ****Full Task 1**** Starting Full load task 1 a% Replication Task arn:aws:dms:us-east-1:000000000000:task:FQWFF7YIZ4VGQHBIXCLI9FJTUUS17NSECIM0UR7 status: starting Waiting for task status stopped task='arn:aws:dms:us-east-1:000000000000:task:FQWFF7YIZ4VGQHBIXCLI9FJTUUS17NSECIM0UR7' status='starting' task='arn:aws:dms:us-east-1:000000000000:task:FQWFF7YIZ4VGQHBIXCLI9FJTUUS17NSECIM0UR7' status='stopped' Kinesis events fetching Kinesis event Received: 6 events [{'control': {}, 'metadata': {'operation': 'drop-table', 'partition-key-type': 'task-id', 'partition-key-value': 'FQWFF7YIZ4VGQHBIXCLI9FJTUUS17NSECIM0UR7', 'record-type': 'control', 'schema-name': 'dms_sample', 'table-name': 'accounts', 'timestamp': '2024-05-23T19:17:33.126Z'}, 'partition_key': 'FQWFF7YIZ4VGQHBIXCLI9FJTUUS17NSECIM0UR7.dms_sample.accounts'}, {'control': {}, 'metadata': {'operation': 'drop-table', 'partition-key-type': 'task-id', 'partition-key-value': 'FQWFF7YIZ4VGQHBIXCLI9FJTUUS17NSECIM0UR7', 'record-type': 'control', 'schema-name': 'dms_sample', 'table-name': 'authors', 'timestamp': '2024-05-23T19:17:33.128Z'}, ... ... ... ``` ## Supported Use Cases DMS is in a preview state on LocalStack and only supports some selected use cases: | Source | Target | Migration Types | Serverless Support | | - | - | - | - | | MariaDB (external) | Kinesis | full-load, cdc | Yes | | MySQL (external) | Kinesis | full-load, cdc | Yes | | RDS MariaDB | Kinesis | full-load, cdc | Yes | | RDS MySQL | Kinesis | full-load, cdc | Yes | | S3 | Kinesis | full-load, cdc | Not supported by AWS | | Aurora PostgreSQL | Kinesis | full-load, cdc | No | | RDS PostgreSQL | Kinesis | full-load, cdc | No | | PostgreSQL (external) | Kinesis | full-load, cdc | No | ## Serverless [DMS Serverless](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Serverless.html) can be used in Localstack for the above mentioned supported use cases that are [officially supported by AWS](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Serverless.Components.html#CHAP_Serverless.SupportedVersions). In order to simulate the different states that the replication config goes through when provisioning, you can set the env `DMS_SERVERLESS_STATUS_CHANGE_WAITING_TIME`, which will cause the state-change to wait the configured seconds. The waiting time is applied for every status change before the replication is actually in `running`. See also the [official docs for explanation about the different states](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Serverless.Components.html). Be aware that the replication table statistics on AWS is deleted automatically once the replication finished, and the replication configuration deprovisioned. For parity reasons, this is also true on LocalStack. In order to delay the deprovisioning, you can use the env `DMS_SERVERLESS_DEPROVISIONING_DELAY`, which by default is set to 60 seconds. ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing: * [Replication Instances](https://app.localstack.cloud/inst/default/resources/dms/replication-instances) * [Endpoints](https://app.localstack.cloud/inst/default/resources/dms/endpoints) * [Replication Tasks](https://app.localstack.cloud/inst/default/resources/dms/replication-tasks) You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **Database Migration Service** under the **Migration and transfer** section. ![DMS Resource Browser](/images/aws/dms-resource-browser.png) The Resource Browser supports CRD (Create, Read, Delete) operations on DMS resources. ### Replication Instances * **Create Replication Instance**: To create a new replication instance, click the **Create Replication Instance** button and enter details such as the Replication Instance Identifier and Replication Instance class. * **View Replication Instance**: To view details of a replication instance, click on its ARN. * **Delete Replication Instance**: To delete a replication instance, select it, go to **Actions**, and choose **Remove Selected**. ### Endpoints * **Create Endpoint**: To create a new endpoint, click on the **Create Endpoint** button and fill in necessary details such as the Endpoint Identifier, Endpoint Type, and Engine Name. * **View Endpoint**: To see the details of an endpoint, click on its ARN. You can further click **Connections** and test a connection by specifying the Replication Instance ARN. * **Delete Endpoint**: To remove an endpoint, select it, navigate to **Actions**, and click **Remove Selected**. ### Replication Tasks * **Create Replication Task**: To create a new replication task, press the **Create Replication Task** button and specify the Task Identifier, Source Endpoint Identifier, and Target Endpoint Identifier, among other settings. * **View Replication Task**: To review a replication task, click on the task identifier. * **Delete Replication Task**: To delete a replication task, choose the task, click on **Actions**, and select **Remove Selected**. ## Current Limitations For RDS MariaDB and RDS MySQL it is not yet possible to set custom db-parameters. In order to make those databases work with `cdc` migration for DMS, some default db-parameters are changed upon start if the `ENABLE_DMS=1` flag is set: ```bash binlog_checksum=NONE binlog_row_image=FULL binlog_format=ROW server_id=1 log_bin=mysqld-bin ``` For S3 as a source, only the first 1000 files of a table in a bucket are considered for migration. For PostgreSQL as a source, the `ReplicationTaskSettings.BeforeImageSettings` parameter is not supported. ### Enum Values for CDC data events To support Enum values for CDC data events, you need to enable the database setting `BINLOG_ROW_METADATA=FULL` ### Migration Type A replication task on LocalStack does currently only support `full-load` (migrate existing data) or `cdc` (replicate data changes only). On AWS there is also a combination for those, which is not yet implemented on LocalStack. ### ReplicationTaskSettings The `ReplicationTaskSettings` for a [replication task](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.html) only considers `BeforeImageSettings`, `FullLoadSettings.CommitRate` and `FullLoadSettings.TargetTablePrepMode` ### Other Limitations * [Data Validation](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Validating.html#CHAP_Validating.TaskStatistics) is not supported * [Reload](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.ReloadTables.html) of tables is not supported * [Task Logs](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Monitoring.html#CHAP_Monitoring.ManagingLogs), specifically CloudWatch, and CloudTrail are not supported (table statistics are supported) * [Time Travel](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.TimeTravel.html) is not supported * [Target Metadata Settings](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.TargetMetadata.html): `ParallelLoadThreads` is not supported * [Transformation](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Transformations.html): `"rule-type": "transformation"` is not supported * [AWS DMS Schema Conversion Tool](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_SchemaConversion.html) is not supported * [AWS DMS Fleet Advisor](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_FleetAdvisor.html) is not supported ## API Coverage # DocumentDB (DocDB) > Get started with AWS DocumentDB on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction DocumentDB is a fully managed, non-relational database service that supports MongoDB workloads. DocumentDB is compatible with MongoDB, meaning you can use the same MongoDB drivers, applications, and tools to run, manage, and scale workloads on DocumentDB without having to worry about managing the underlying infrastructure. LocalStack allows you to use the DocumentDB APIs to create and manage DocumentDB clusters and instances. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of DocumentDB's integration with LocalStack. ## Getting started To create a new DocumentDB cluster we use the `create-db-cluster` command as follows: ```bash lstk aws docdb create-db-cluster \ --db-cluster-identifier test-docdb-cluster \ --engine docdb ``` ```bash title="Output" { "DBCluster": { "DBClusterIdentifier": "test-docdb-cluster", "DBClusterParameterGroup": "default.docdb", "Status": "available", "Endpoint": "localhost.localstack.cloud", "MultiAZ": false, "Engine": "docdb", "Port": 39045, "MasterUsername": "test", "DBClusterMembers": [], "VpcSecurityGroups": [ { "VpcSecurityGroupId": "sg-a30edea1f7da6ff90", "Status": "active" } ], "StorageEncrypted": false, "DBClusterArn": "arn:aws:rds:us-east-1:000000000000:cluster:test-docdb-cluster" } } ``` If we break down the previous command, we can identify: - `docdb`: The command related to Amazon DocumentDB for the `AWS CLI`. - `create-db-cluster`: The command to create an Amazon DocumentDB cluster. - `--db-cluster-identifier test-docdb-cluster`: Specifies the unique identifier for the DocumentDB cluster. In this case, it is set to `test-docdb-cluster`. You can customize this identifier to a name of your choice. - `--engine docdb`: Specifies the database engine. Here, it is set to `docdb`, indicating the use of Amazon DocumentDB. Notice in the `DBClusterMembers` field of the cluster description that there are no other databases created. As we did not specify a `MasterUsername` or `MasterUserPassword` for the creation of the database, the mongo-db will not set any credentials when starting the docker container. To create a new database, we can use the `create-db-instance` command, like in this example: ```bash lstk aws docdb create-db-instance \ --db-instance-identifier test-company \ --db-instance-class db.r5.large \ --engine docdb \ --db-cluster-identifier test-docdb-cluster ``` ```bash title="Output" { "DBInstance": { "DBInstanceIdentifier": "test-docdb-instance", "DBInstanceClass": "db.r5.large", "Engine": "docdb", "DBInstanceStatus": "creating", "Endpoint": { "Address": "localhost.localstack.cloud", "Port": 50761 }, "InstanceCreateTime": "2022-10-28T04:27:35.917000+00:00", "PreferredBackupWindow": "03:50-04:20", "BackupRetentionPeriod": 1, "VpcSecurityGroups": [ , "AvailabilityZone": "us-east-1a", "PreferredMaintenanceWindow": "wed:06:38-wed:07:08", "EngineVersion": "12.34", "AutoMinorVersionUpgrade": false, "PubliclyAccessible": false, "StatusInfos": [], "DBClusterIdentifier": "test-docdb-cluster", "StorageEncrypted": false, "DbiResourceId": "db-M5ENSHXFPU6XHZ4G4ZEI5QIO2U", "CopyTagsToSnapshot": false, "DBInstanceArn": "arn:aws:rds:us-east-1:000000000000:db:test-docdb-instance", "EnabledCloudwatchLogsExports": [] } } ``` Some noticeable fields: - `--db-instance-identifier test-company`: Represents the unique identifier of the newly created database. - `--db-instance-class db.r5.large`: Is the type or class of the Amazon DocumentDB instance. It determines the compute and memory capacity allocated to the instance. `db.r5.large` refers to a specific instance type in the R5 family. Although the flag is required for database creation, LocalStack will only mock the `DBInstanceClass` attribute. You can find out more about instance classes in the [AWS documentation](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.DBInstanceClass.html) . To obtain detailed information about the cluster, we use the `describe-db-cluster` command: ```bash lstk aws docdb describe-db-clusters \ --db-cluster-identifier test-docdb-cluster ``` ```bash title="Output" { "DBClusters": [ { "DBClusterIdentifier": "test-docdb-cluster", "DBClusterParameterGroup": "default.docdb", "Status": "available", "Endpoint": "localhost.localstack.cloud", "MultiAZ": false, "Engine": "docdb", "Port": 39045, "MasterUsername": "test", "DBClusterMembers": [ { "DBInstanceIdentifier": "test-company", "IsClusterWriter": true, "DBClusterParameterGroupStatus": "in-sync", "PromotionTier": 1 } ], "VpcSecurityGroups": [ { "VpcSecurityGroupId": "sg-a30edea1f7da6ff90", "Status": "active" } ], "StorageEncrypted": false, "DBClusterArn": "arn:aws:rds:us-east-1:000000000000:cluster:test-docdb-cluster" } ] } ``` ### Connect to DocumentDB using mongosh Interacting with the databases is done using `mongosh`, which is an official command-line shell and [interactive MongoDB shell provided by MongoDB](https://www.mongodb.com/docs/mongodb-shell/). It is designed to provide a modern and enhanced user experience for interacting with MongoDB databases. ```bash mongosh mongodb://localhost:39045 ``` ```bash title="Output" Current Mongosh Log ID: 64a70b795697bcd4865e1b9a Connecting to: mongodb://localhost: 39045/?directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+1.10.1 Using MongoDB: 6.0.7 Using Mongosh: 1.10.1 ``` This command will default to accessing the `test` database that was created with the cluster. Notice the port, `39045`, which is the cluster port that appears in the aforementioned description. To work with a specific database, the command is: ```bash mongosh mongodb://localhost:39045/test-company ``` From here on we can manipulate collections using [the JavaScript methods provided](https://www.mongodb.com/docs/manual/reference/method/) by `mongosh`: ```bash test-company> db.createCollection("employees") { ok: 1 } test-company> db.createCollection("customers") { ok: 1 } test-company> show collections customers employees test-company> exit ``` For more information on how to use MongoDB with `mongosh` please refer to the [MongoDB documentation](https://www.mongodb.com/docs/). ### Connect to DocumentDB using Node.js Lambda :::note You need to set `DOCDB_PROXY_CONTAINER=1` when starting LocalStack to be able to use the returned `Endpoint`, which will be correctly resolved automatically. The flag `DOCDB_PROXY_CONTAINER=1` changes the default behavior and the container will be started as proxied container. Meaning a port from the [pre-defined port](/aws/customization/networking/external-port-range/) range will be chosen, and when using lambda, you can use `localhost.localstack.cloud` to connect to the instance. ::: In this sample we will use a Node.js lambda function to connect to a DocumentDB. For the mongo-db connection we will use the `mongodb` lib. Please note, that this sample is only for demo purpose, e.g., we will set the credentials as environment variables to the lambda function. In a best-practise sample you would use a secret instead. We included a snippet at the very end. #### Create the DocDB Cluster with a username and password We assume you have a `MasterUsername` and `MasterUserPassword` set for DocDB e.g: ```bash lstk aws docdb create-db-cluster \ --db-cluster-identifier test-docdb \ --engine docdb \ --master-user-password S3cretPwd! \ --master-username someuser ``` #### Prepare the lambda function First, we create the zip required for the lambda function with the mongodb dependency. You will need [`npm`](https://docs.npmjs.com/) in order to install the dependencies. In your terminal run: ```bash mkdir resources cd resources mkdir node_modules npm install mongodb@6.3.0 ``` Next, copy the following code into a new file named `index.js` in the `resources` folder: ```javascript showshowLineNumbers const AWS = require('aws-sdk'); const RDS = AWS.RDS; const { MongoClient } = require('mongodb'); const docdb_client = new RDS(); const docdb_id = process.env.DOCDB_CLUSTER_ID; const pwd = process.env.DOCDB_SECRET; exports.handler = async (event) => { try { // Get endpoint details using rds/docdb client: const cluster_result = await docdb_client.describeDBClusters({DBClusterIdentifier: docdb_id}).promise(); const cluster = cluster_result.DBClusters[0]; const host = cluster.Endpoint; const port = cluster.Port; const user = cluster.MasterUsername; // Connection URI const dbname = "mydb"; // retryWrites is by default true, but not supported by AWS DocumentDB const uri = `mongodb://${user}:${pwd}@${host}:${port}/?retryWrites=false`; // Connect to DocumentDB const client = await MongoClient.connect(uri); const db = client.db(dbname); // Insert data const collection = db.collection('your_collection'); await collection.insertOne({ key: 'value' }); // Query data const result = await collection.findOne({ key: 'value' }); await client.close(); // Return result return { statusCode: 200, body: JSON.stringify(result), }; } catch (error) { return { statusCode: 500, body: JSON.stringify({ error: error.message }), }; } }; ``` Now, you can zip the entire. Make sure you are inside `resources` directory and run: ```bash zip -r function.zip . ``` Finally, we can create the `lambda` function using `lstk aws`: ```bash lstk aws lambda create-function \ --function-name MyNodeLambda \ --runtime nodejs16.x \ --role arn:aws:iam::000000000000:role/lambda-role \ --handler index.handler \ --zip-file fileb://function.zip \ --environment Variables="{DOCDB_CLUSTER_ID=test-docdb,DOCDB_SECRET=S3cretPwd!}" ``` You can invoke the lambda by calling: ```bash lstk aws lambda invoke \ --function-name MyNodeLambda \ outfile ``` The `outfile` contains the returned value, e.g.: ```bash title="Output" {"statusCode":200,"body":"{\"_id\":\"6560a21ca7771a02ef128c72\",\"key\":\"value\"}"} ``` #### Use Secret To Connect to DocDB The best-practise for accessing databases is by using secrets. Secrets follow a [well-defined pattern](https://docs.aws.amazon.com/secretsmanager/latest/userguide/create_database_secret.html). For the lambda function, you can pass the secret arn as `SECRET_NAME`. In the lambda, you can then retrieve the secret details like this: ```javascript showshowLineNumbers const AWS = require('aws-sdk'); const { MongoClient } = require('mongodb'); const secretsManager = new AWS.SecretsManager(); const secretName = process.env.SECRET_NAME; function customURIEncode(str) { // encode also characters that encodeURIComponent does not encode return encodeURIComponent(str) .replace(/!/g, '%21') .replace(/~/g, '%7E') .replace(/\*/g, '%2A') .replace(/'/g, '%27') .replace(/\(/g, '%28') .replace(/\)/g, '%29'); } exports.handler = async (event) => { try { // Retrieve secret const secretValue = await secretsManager.getSecretValue({ SecretId: secretName }).promise(); const { username, password, host, port } = JSON.parse(secretValue.SecretString); // make sure username and password are correctly encoded for the URI const user = customURIEncode(username); const pwd = customURIEncode(password); // retryWrites is by default true, but not supported by AWS DocumentDB const uri = `mongodb://${user}:${pwd}@${host_name}:${port}/?retryWrites=false`; // Connect to DocumentDB const client = await MongoClient.connect(uri); // ... interact with the mongo-db ... return { statusCode: 200 }; } catch (error) { console.error('Error: ', error); return { statusCode: 500, body: JSON.stringify({ error: error.message }), }; } }; ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing DocumentDB instances and clusters. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **DocumentDB** under the **Database** section. ![DocumentDB Resource Browser](/images/aws/docdb-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Cluster**: Create a new DocumentDB cluster by specifying the DBCluster Identifier, Availability Zone, and other parameters. - **Create Instance**: Create a new DocumentDB instance by specifying the database class, engine, DBInstance Identifier, and other parameters. - **View Instance & Cluster**: View an existing DocumentDB instance or cluster by clicking the instance/cluster name. - **Edit Instance & Cluster**: Edit an existing DocumentDB instance or cluster by clicking the instance/cluster name and clicking the **Edit Instance** or **Edit Cluster** button. - **Remove Instance & Cluster**: Remove an existing DocumentDB instance or cluster by clicking the instance/cluster name and clicking the **Actions** followed by **Remove Selected** button. ## Current Limitations Under the hood, LocalStack starts a MongoDB server, to handle DocumentDB storage, in a separate Docker container and adds port-mapping so that it can be accessed from `localhost`. When defining a port to access the container, an available port on the host machine will be selected, that means there is no pre-defined port range by default. Because LocalStack utilizes a MongoDB container to provide DocumentDB storage, LocalStack may not have exact feature parity with Amazon DocumentDB. The database engine may support additional features that DocumentDB does not and vice versa. DocumentDB currently uses the default configuration of the latest [MongoDB Docker image](https://hub.docker.com/_/mongo). When the `MasterUsername` and `MasterUserPassword` are set for the creation for the DocumentDB cluster or instance, the container will be started with the corresponding ENVs `MONGO_INITDB_ROOT_USERNAME` respectively `MONGO_INITDB_ROOT_PASSWORD`. ## API Coverage # Aurora DSQL > Get started with Aurora DSQL on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Aurora DSQL is a serverless, distributed, PostgreSQL-compatible database service provided by AWS. It offers active-active high availability and is designed for transactional workloads that require scalability without the operational overhead of managing database infrastructure. LocalStack allows you to use the Aurora DSQL APIs to create and manage clusters, tags, resource policies, and streams in your local environment. The data plane is backed by an embedded PostgreSQL instance, so you can connect to a cluster and run SQL against it, including DSQL-specific dialect such as `CREATE INDEX ASYNC` and the `sys.jobs` system view. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Aurora DSQL's integration with LocalStack. ## Getting started This guide is designed for users new to Aurora DSQL and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a cluster, inspect it, and clean it up using the AWS CLI. ### Create a cluster You can create a cluster using the [`CreateCluster`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_CreateCluster.html) API. Run the following command to create a cluster: ```bash lstk aws dsql create-cluster ``` ```bash title="Output" { "identifier": "8a71d298-c086-4fb4-a698-d7b4eeb657e6", "arn": "arn:aws:dsql:us-east-1:000000000000:cluster/8a71d298-c086-4fb4-a698-d7b4eeb657e6", "status": "CREATING", "creationTime": 1782306284.339124, "deletionProtectionEnabled": true, "encryptionDetails": { "encryptionType": "AWS_OWNED_KMS_KEY", "encryptionStatus": "ENABLED" }, "endpoint": "localhost.localstack.cloud:4513" } ``` The cluster is returned with a `CREATING` status and transitions to `ACTIVE` shortly after. Note that `deletionProtectionEnabled` defaults to `true`, matching AWS behaviour. To use a customer-managed KMS key, pass `--kms-encryption-key `; the `encryptionDetails` will then report `CUSTOMER_MANAGED_KMS_KEY` and echo the key ARN. ### Inspect a cluster You can retrieve the details of a cluster using the [`GetCluster`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_GetCluster.html) API. Replace the identifier with the one returned in the previous step: ```bash lstk aws dsql get-cluster --identifier 8a71d298-c086-4fb4-a698-d7b4eeb657e6 ``` ```bash title="Output" { "identifier": "8a71d298-c086-4fb4-a698-d7b4eeb657e6", "arn": "arn:aws:dsql:us-east-1:000000000000:cluster/8a71d298-c086-4fb4-a698-d7b4eeb657e6", "status": "ACTIVE", "creationTime": 1782306284.339124, "deletionProtectionEnabled": true, "tags": {}, "encryptionDetails": { "encryptionType": "AWS_OWNED_KMS_KEY", "encryptionStatus": "ENABLED" }, "endpoint": "localhost.localstack.cloud:4513" } ``` You can list all clusters in the current account and region using the [`ListClusters`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_ListClusters.html) API: ```bash lstk aws dsql list-clusters ``` ```bash title="Output" { "clusters": [ { "identifier": "8a71d298-c086-4fb4-a698-d7b4eeb657e6", "arn": "arn:aws:dsql:us-east-1:000000000000:cluster/8a71d298-c086-4fb4-a698-d7b4eeb657e6" } ] } ``` ### Connect to the cluster The cluster `endpoint` returned by `GetCluster` points at an embedded PostgreSQL instance, so you can connect to it with any PostgreSQL client. The endpoint uses the `host:port` format; split it to obtain the host and port for your client. :::note Data-plane connectivity requires LocalStack to run inside Docker, so that the embedded PostgreSQL backend is available and its port is reachable. Locally, connections use plain PostgreSQL credentials (database `test`, user `test`, password `test`). The IAM authentication-token flow that AWS Aurora DSQL requires is not used against LocalStack. ::: Using `psql`, connect to the database and run some SQL: ```bash psql -d test -U test -h localhost.localstack.cloud -p 4513 -W ``` ```sql CREATE TABLE employees (id integer, name text); INSERT INTO employees (id, name) VALUES (1, 'Alice'); SELECT id, name FROM employees; ``` ```bash title="Output" id | name ----+------- 1 | Alice (1 row) ``` ### Delete a cluster Because clusters are created with deletion protection enabled, you must first disable it using the [`UpdateCluster`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_UpdateCluster.html) API. Attempting to delete a protected cluster returns a `ValidationException`. ```bash lstk aws dsql update-cluster \ --identifier 8a71d298-c086-4fb4-a698-d7b4eeb657e6 \ --no-deletion-protection-enabled ``` You can then delete the cluster using the [`DeleteCluster`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_DeleteCluster.html) API: ```bash lstk aws dsql delete-cluster --identifier 8a71d298-c086-4fb4-a698-d7b4eeb657e6 ``` ```bash title="Output" { "identifier": "8a71d298-c086-4fb4-a698-d7b4eeb657e6", "arn": "arn:aws:dsql:us-east-1:000000000000:cluster/8a71d298-c086-4fb4-a698-d7b4eeb657e6", "status": "DELETING", "creationTime": 1782306284.339124 } ``` ## Tags You can attach tags at creation time with `--tags`, and manage them afterwards using the [`TagResource`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_TagResource.html), [`UntagResource`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_UntagResource.html), and [`ListTagsForResource`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_ListTagsForResource.html) APIs. ```bash lstk aws dsql create-cluster --tags Name=my-cluster,Env=dev ``` Add or update tags on an existing cluster: ```bash lstk aws dsql tag-resource \ --resource-arn arn:aws:dsql:us-east-1:000000000000:cluster/8a71d298-c086-4fb4-a698-d7b4eeb657e6 \ --tags Team=platform ``` List the tags on a resource: ```bash lstk aws dsql list-tags-for-resource \ --resource-arn arn:aws:dsql:us-east-1:000000000000:cluster/8a71d298-c086-4fb4-a698-d7b4eeb657e6 ``` ```bash title="Output" { "tags": { "Name": "my-cluster", "Env": "dev", "Team": "platform" } } ``` Remove tags by key: ```bash lstk aws dsql untag-resource \ --resource-arn arn:aws:dsql:us-east-1:000000000000:cluster/8a71d298-c086-4fb4-a698-d7b4eeb657e6 \ --tag-keys Env ``` ## Cluster policies You can attach a resource-based policy to a cluster using the [`PutClusterPolicy`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_PutClusterPolicy.html) API, then read and remove it with [`GetClusterPolicy`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_GetClusterPolicy.html) and [`DeleteClusterPolicy`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_DeleteClusterPolicy.html). ```bash lstk aws dsql put-cluster-policy \ --identifier 8a71d298-c086-4fb4-a698-d7b4eeb657e6 \ --policy '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":"arn:aws:iam::000000000000:root"},"Action":"dsql:DbConnect","Resource":"*"}]}' ``` ```bash title="Output" { "policyVersion": "a1b2c3d4" } ``` Retrieve the stored policy: ```bash lstk aws dsql get-cluster-policy --identifier 8a71d298-c086-4fb4-a698-d7b4eeb657e6 ``` :::note Cluster policies are stored and returned as-is but are not enforced by LocalStack. ::: ## Streams You can manage stream metadata using the [`CreateStream`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_CreateStream.html), [`GetStream`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_GetStream.html), [`ListStreams`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_ListStreams.html), and [`DeleteStream`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_DeleteStream.html) APIs. ```bash lstk aws dsql create-stream \ --cluster-identifier 8a71d298-c086-4fb4-a698-d7b4eeb657e6 \ --target-definition '{"kinesis":{"streamArn":"arn:aws:kinesis:us-east-1:000000000000:stream/my-stream","roleArn":"arn:aws:iam::000000000000:role/dsql-stream-role"}}' \ --ordering UNORDERED \ --format JSON ``` ```bash title="Output" { "clusterIdentifier": "8a71d298-c086-4fb4-a698-d7b4eeb657e6", "streamIdentifier": "3506a484-f6b2-4610-b04e-5cb0eae4405a", "arn": "arn:aws:dsql:us-east-1:000000000000:cluster/8a71d298-c086-4fb4-a698-d7b4eeb657e6/stream/3506a484-f6b2-4610-b04e-5cb0eae4405a", "status": "CREATING", "creationTime": 1782306345.637581, "ordering": "UNORDERED", "format": "JSON" } ``` List the streams of a cluster: ```bash lstk aws dsql list-streams --cluster-identifier 8a71d298-c086-4fb4-a698-d7b4eeb657e6 ``` :::note Streams are backed as metadata only; no change-data-capture (CDC) records are emitted. ::: ## VPC endpoint You can retrieve the synthesised VPC endpoint service name for a cluster using the [`GetVpcEndpointServiceName`](https://docs.aws.amazon.com/aurora-dsql/latest/APIReference/API_GetVpcEndpointServiceName.html) API: ```bash lstk aws dsql get-vpc-endpoint-service-name --identifier 8a71d298-c086-4fb4-a698-d7b4eeb657e6 ``` ```bash title="Output" { "serviceName": "com.amazonaws.us-east-1.dsql", "clusterVpcEndpoint": "vpce-local.8a71d298-c086-4fb4-a698-d7b4eeb657e6.dsql.us-east-1.vpce.amazonaws.com" } ``` ## SQL dialect Aurora DSQL is PostgreSQL wire-compatible but adds dialect-specific DDL that vanilla Postgres rejects. LocalStack accepts the DSQL-specific statements that applications commonly rely on so DSQL-targeted apps can run against the emulator. ### Asynchronous indexes On real Aurora DSQL, [`CREATE INDEX ASYNC`](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/working-with-create-index-async.html) (and `CREATE UNIQUE INDEX ASYNC`) submits a background index build, returns a `job_id` immediately, and records the job in the [`sys.jobs`](https://docs.aws.amazon.com/aurora-dsql/latest/userguide/working-with-systems-tables.html) system view for the application to poll. LocalStack accepts the same syntax. The index is built synchronously under the hood; the observable result matches AWS: a `job_id` is returned, the index exists afterwards, and `sys.jobs` reports the job as `completed` with `job_type` `INDEX_BUILD`. Connect to the cluster as shown in [Connect to the cluster](#connect-to-the-cluster), then run: ```sql CREATE TABLE departments (name varchar(255) PRIMARY KEY, manager varchar(255)); INSERT INTO departments (name, manager) VALUES ('HR', 'John Doe'); CREATE INDEX ASYNC idx_dept_name_manager ON departments (name, manager); ``` ```bash title="Output" job_id ------------------------------------ a1b2c3d4e5f6g7h8i9j0k1l2m3 (1 row) ``` Query `sys.jobs` to inspect the job, or call `sys.wait_for_job` to wait until it reports completion: ```sql SELECT job_id, status, job_type, object_name FROM sys.jobs; SELECT sys.wait_for_job('a1b2c3d4e5f6g7h8i9j0k1l2m3'); ``` ```bash title="Output" job_id | status | job_type | object_name ----------------------------------+-----------+-------------+--------------------- a1b2c3d4e5f6g7h8i9j0k1l2m3 | completed | INDEX_BUILD | idx_dept_name_manager (1 row) wait_for_job -------------- t (1 row) ``` `CREATE UNIQUE INDEX ASYNC` works the same way and enforces uniqueness once the index exists: ```sql CREATE UNIQUE INDEX ASYNC uidx_dept_manager ON departments (manager); ``` :::note Index builds run synchronously under the hood. Async timing (background processing, `submitted`/`processing` statuses) and the AWS 30-minute auto-cleanup of completed or failed jobs from `sys.jobs` are not modelled. ::: ## Current Limitations - CloudFormation is not yet supported for Aurora DSQL resources. - The data plane is backed by a standard embedded PostgreSQL instance rather than the real Aurora DSQL distributed engine. - LocalStack accepts DSQL-specific `CREATE [UNIQUE] INDEX ASYNC` and exposes `sys.jobs` / `sys.wait_for_job`, but broader DSQL dialect restrictions (for example rejecting foreign keys or `TRUNCATE`) are not enforced, so behaviour may differ from AWS for unsupported statements. - Asynchronous index builds complete synchronously; timing semantics and automatic cleanup of old `sys.jobs` rows are not modelled. - Multi-region clusters are tracked at the control-plane level only. Peering metadata is recorded, but there is no real cross-region replication. - Data-plane data is not persisted yet. Cluster metadata survives restarts when persistence is enabled, but the data written through the embedded PostgreSQL backend (tables, rows) is not retained. - Streams support metadata CRUD only; no change-data-capture record flow is produced. - Cluster policies are stored as opaque JSON and are not enforced. - `GetVpcEndpointServiceName` returns a cosmetic, synthesised endpoint name. - KMS encryption is reflected in metadata only; no actual encryption is performed. - Data-plane connectivity requires running LocalStack inside Docker and uses plain PostgreSQL credentials. The AWS IAM authentication-token flow is not used locally. ## API Coverage # DynamoDB > Get started with DynamoDB on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; DynamoDB is a fully managed NoSQL database service provided by AWS. It offers a flexible and highly scalable way to store and retrieve data, making it suitable for a wide range of applications. DynamoDB provides a fast and scalable key-value datastore with support for replication, automatic scaling, data encryption at rest, and on-demand backup, among other capabilities. LocalStack allows you to use the DynamoDB APIs in your local environment to manage key-value and document data models. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of DynamoDB's integration with LocalStack. DynamoDB emulation is powered by [DynamoDB Local](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html). ## Getting started This guide is designed for users new to DynamoDB and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create DynamoDB table, along with its replicas, and put an item into the table using the AWS CLI. ### Create a DynamoDB table You can create a DynamoDB table using the [`CreateTable`](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_CreateTable.html) API. Execute the following command to create a table named `global01` with a primary key `id`: ```bash lstk aws dynamodb create-table \ --table-name global01 \ --key-schema AttributeName=id,KeyType=HASH \ --attribute-definitions AttributeName=id,AttributeType=S \ --billing-mode PAY_PER_REQUEST \ --region ap-south-1 ``` ```bash title="Output" { "TableDescription": { "AttributeDefinitions": [ { "AttributeName": "id", "AttributeType": "S" } ], "TableName": "global01", "KeySchema": [ { "AttributeName": "id", "KeyType": "HASH" } ], "TableStatus": "ACTIVE", "CreationDateTime": 1693244562.147, ... "TableArn": "arn:aws:dynamodb:ap-south-1:000000000000:table/global01", "TableId": "6bc6dd46-98d8-486a-aed8-6ef66a35df7c", ... } } } ``` ### Create replicas You can create replicas of a DynamoDB table using the [`UpdateTable`](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_UpdateTable.html) API. Execute the following command to create replicas in `ap-south-1` and `us-west-1` regions: ```bash lstk aws dynamodb update-table \ --table-name global01 \ --replica-updates '[{"Create": {"RegionName": "eu-central-1"}}, {"Create": {"RegionName": "us-west-1"}}]' \ --region ap-south-1 ``` ```bash title="Output" { "TableDescription": { "AttributeDefinitions": [ { "AttributeName": "id", "AttributeType": "S" } ], ... "Replicas": [ { "RegionName": "eu-central-1", "ReplicaStatus": "ACTIVE" }, { "RegionName": "us-west-1", "ReplicaStatus": "ACTIVE" } ] } } ``` You can now operate on the table in the replicated regions as well. You can use the [`ListTables`](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_ListTables.html) API to list the tables in the replicated regions. Run the following command to list the tables in the `eu-central-1` region: ```bash lstk aws dynamodb list-tables \ --region eu-central-1 ``` ```bash title="Output" { "TableNames": [ "global01" ] } ``` ### Insert an item You can insert an item into a DynamoDB table using the [`PutItem`](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PutItem.html) API. Execute the following command to insert an item into the `global01` table: ```bash lstk aws dynamodb put-item \ --table-name global01 \ --item '{"id":{"S":"foo"}}' \ --region eu-central-1 ``` You can now query the number of items in the table using the [`DescribeTable`](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_DescribeTable.html) API. Run the following command to query the number of items in the `global01` table from a different region: ```bash lstk aws dynamodb describe-table \ --table-name global01 \ --query 'Table.ItemCount' \ --region ap-south-1 ``` ```bash title="Output" 1 ``` :::note You can run DynamoDB in memory, which can greatly improve the performance of your database operations. However, this also means that the data will not be possible to persist on disk and will be lost even though persistence is enabled in LocalStack. To enable this feature, you need to set the environment variable `DYNAMODB_IN_MEMORY=1` while starting LocalStack. ::: ### Time To Live LocalStack supports [Time to Live (TTL)](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/TTL.html) in DynamoDB. To enable this feature, you need to set the environment variable `DYNAMODB_REMOVE_EXPIRED_ITEMS` to 1. This enables a worker running every 60 minutes that scans all the tables and deletes the expired items. In addition, to programmatically trigger the worker at convenience, we provide the following endpoint: - `DELETE /_aws/dynamodb/expired` The response returns the number of deleted items: ```bash curl -X DELETE localhost:4566/_aws/dynamodb/expired ``` ```bash title="Output" {"ExpiredItems": 3} ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing DynamoDB tables and items. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **DynamoDB** under the **Database** section. ![DynamoDB Resource Browser](/images/aws/dynamodb-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Table**: Create a new DynamoDB table by clicking on the **Create Table** button. You can specify the table name, table class, key schema and other attributes of the table. - **Edit Table**: Edit an existing DynamoDB table by clicking on the **Edit Table** button. You can modify the table name, key schema and other attributes of the table. - **View items**: View the items in a DynamoDB table by clicking on the **Items** button. You can also add, edit and delete items in the table. You can also switch to scan or query mode to view the items in the table. - **Run PartiQL**: Run a PartiQL query against a DynamoDB table by clicking on the **PartiQL** button. You can add your query in the editor and click on the **Execute** button to execute the query. - **Delete Table**: Delete an existing DynamoDB table by selecting the DynamoDB table and clicking **Actions** and then **Remove Selected**. ## Examples The following code snippets and sample applications provide practical examples of how to use IAM in LocalStack for various use cases: - [Serverless Container-based APIs with Amazon ECS & API Gateway](https://github.com/localstack/serverless-api-ecs-apigateway-sample) - [Full-Stack application with AWS Lambda, DynamoDB & S3 for shipment validation](https://github.com/localstack/shipment-list-demo) - [Step-up Authentication using Amazon Cognito](https://github.com/localstack/step-up-auth-sample) - [Serverless microservices with Amazon API Gateway, DynamoDB, SQS, and Lambda](https://github.com/localstack/microservices-apigateway-lambda-dynamodb-sqs-sample) - [Event-driven architecture with Amazon SNS FIFO, DynamoDB, Lambda, and S3](https://github.com/localstack/event-driven-architecture-with-amazon-sns-fifo) - [Note-Taking application using AWS SDK for JavaScript](https://github.com/localstack/aws-sdk-js-notes-app) - [AppSync GraphQL APIs for DynamoDB and RDS Aurora PostgreSQL](https://github.com/localstack/appsync-graphql-api-sample) - [Loan Broker application with AWS Step Functions, DynamoDB, Lambda, SQS, and SNS](https://github.com/localstack/loan-broker-stepfunctions-lambda-app) - [Messaging Processing application with SQS, DynamoDB, and Fargate](https://github.com/localstack/sqs-fargate-ddb-cdk-go) ## Current Limitations ### Global tables LocalStack provides support for global tables (Version 2019), which are tables that exist within the same account and are replicated across various regions. However, legacy global tables (Version 2017) are not supported by LocalStack. Operations such as `CreateGlobalTable`, `UpdateGlobalTable`, and `DescribeGlobalTable` will not replicate globally. ### Replication - Removing the original table region from the replication set while retaining the replicas is currently not feasible. Deleting the original table will result in the removal of all replicas as well. - DynamoDB Streams are exclusively supported for original tables and not for replicated ones. - Batch operations such as `BatchWriteItem`, `BatchGetItem`, etc. are currently not supported for replicated tables. ## API Coverage # DynamoDB Streams > Get started with DynamoDB Streams on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction DynamoDB Streams captures data modification events in a DynamoDB table. The stream records are written to a DynamoDB stream, which is an ordered flow of information about changes to items in a table. DynamoDB Streams records data in near-real time, enabling you to develop workflows that process these streams and respond based on their contents. LocalStack supports DynamoDB Streams, allowing you to create and manage streams in a local environment. The supported APIs are available on our [API coverage section](#api-coverage), which provides information on the extent of DynamoDB Streams integration with LocalStack. ## Getting started This guide is designed for users new to DynamoDB Streams and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate the following process using LocalStack: - A user adds an entry to a DynamoDB table. - A new stream record is generated in DynamoDB Streams when an entry is added. - This stream record triggers a Lambda function. - If the record indicates a new entry in the DynamoDB table, the Lambda function extracts the data. ### Create a DynamoDB table You can create a DynamoDB table named `BarkTable` using the [`CreateTable`](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_CreateTable.html) API. Run the following command to create the table: ```bash lstk aws dynamodb create-table \ --table-name BarkTable \ --attribute-definitions AttributeName=Username,AttributeType=S AttributeName=Timestamp,AttributeType=S \ --key-schema AttributeName=Username,KeyType=HASH AttributeName=Timestamp,KeyType=RANGE \ --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5 \ --stream-specification StreamEnabled=true,StreamViewType=NEW_AND_OLD_IMAGES ``` The `BarkTable` has a stream enabled which you can trigger by associating a Lambda function with the stream. You can notice that in the `LatestStreamArn` field of the response: ```bash title="Output" ... "LatestStreamArn": "arn:aws:dynamodb:000000000000:us-east-1:table/BarkTable/stream/timestamp ... ``` ### Create a Lambda function You can now create a Lambda function (`publishNewBark`) to process stream records from `BarkTable`. Create a new file named `index.js` with the following code: ```javascript showshowLineNumbers 'use strict'; var AWS = require("aws-sdk"); exports.handler = (event, context, callback) => { event.Records.forEach((record) => { console.log('Stream record: ', JSON.stringify(record, null, 2)); if (record.eventName == 'INSERT') { var who = JSON.stringify(record.dynamodb.NewImage.Username.S); var when = JSON.stringify(record.dynamodb.NewImage.Timestamp.S); var what = JSON.stringify(record.dynamodb.NewImage.Message.S); var params = { Subject: 'A new bark from ' + who, Message: 'Woofer user ' + who + ' barked the following at ' + when + ':\n\n ' + what, }; } }); callback(null, `Successfully processed ${event.Records.length} records.`); }; ``` You can now create a Lambda function using the [`CreateFunction`](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunction.html) API. Run the following command to create the Lambda function: ```bash zip index.zip index.js lstk aws lambda create-function \ --function-name publishNewBark \ --zip-file fileb://index.zip \ --role roleARN \ --handler index.handler \ --timeout 50 \ --runtime nodejs16.x \ --role arn:aws:iam::000000000000:role/lambda-role ``` ### Invoke the Lambda function To test the Lambda function, you can invoke it using the [`Invoke`](https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html) API. Create a new file named `payload.json` with the following content: ```json showshowLineNumbers { "Records": [ { "eventID": "7de3041dd709b024af6f29e4fa13d34c", "eventName": "INSERT", "eventVersion": "1.1", "eventSource": "aws:dynamodb", "awsRegion": "us-east-1", "dynamodb": { "ApproximateCreationDateTime": 1479499740, "Keys": { "Timestamp": { "S": "2016-11-18:12:09:36" }, "Username": { "S": "John Doe" } }, "NewImage": { "Timestamp": { "S": "2016-11-18:12:09:36" }, "Message": { "S": "This is a bark from the Woofer social network" }, "Username": { "S": "John Doe" } }, "SequenceNumber": "13021600000000001596893679", "SizeBytes": 112, "StreamViewType": "NEW_IMAGE" }, "eventSourceARN": "arn:aws:dynamodb:000000000000:us-east-1 ID:table/BarkTable/stream/2016-11-16T20:42:48.104" } ] } ``` Run the following command to invoke the Lambda function: ```bash lstk aws lambda invoke \ --function-name publishNewBark \ --payload file://payload.json \ --cli-binary-format raw-in-base64-out output.txt ``` In the `output.txt` file, you should see the following output: ```bash title="Output" "Successfully processed 1 records." ``` ### Add event source mapping To add the DynamoDB stream as an event source for the Lambda function, you need the stream ARN. You can get the stream ARN using the [`DescribeTable`](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_DescribeTable.html) API. Run the following command to get the stream ARN: ```bash lstk aws dynamodb describe-table --table-name BarkTable ``` You can now create an event source mapping using the [`CreateEventSourceMapping`](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateEventSourceMapping.html) API. Run the following command to create the event source mapping: ```bash lstk aws lambda create-event-source-mapping \ --function-name publishNewBark \ --event-source arn:aws:dynamodb:us-east-1:000000000000:table/BarkTable/stream/2024-07-12T06:18:37.101 \ --batch-size 1 \ --starting-position TRIM_HORIZON ``` Make sure to replace the `event-source` value with the stream ARN you obtained from the previous command. You should see the following output: ```bash title="Output" { "UUID": "7ae3426a-eda6-4c10-a596-100c59bd6787", ... "EventSourceArn": "arn:aws:dynamodb:us-east-1:000000000000:table/BarkTable/stream/2024-07-12T06:18:37.101", "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:publishNewBark", ... "FunctionResponseTypes": [] } ``` You can now test the event source mapping by adding an item to the `BarkTable` table using the [`PutItem`](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PutItem.html) API. Run the following command to add an item to the table: ```bash lstk aws dynamodb put-item \ --table-name BarkTable \ --item Username={S="Jane Doe"},Timestamp={S="2016-11-18:14:32:17"},Message={S="Testing...1...2...3"} ``` You can find Lambda function being triggered in the LocalStack logs. ### Inspect the stream You can list the streams using the [`ListStreams`](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_ListStreams.html) API. Run the following command to list the streams: ```bash lstk aws dynamodbstreams list-streams ``` The following output shows the list of streams: ```bash title="Output" { "Streams": [ { "StreamArn": "arn:aws:dynamodb:us-east-1:000000000000:table/BarkTable/stream/2024-07-12T06:18:37.101", "TableName": "BarkTable", "StreamLabel": "2024-07-12T06:18:37.101" } ] } ``` You can also describe the stream using the [`DescribeStream`](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_DescribeStream.html) API. Run the following command to describe the stream: ```bash lstk aws dynamodbstreams describe-stream \ --stream-arn arn:aws:dynamodb:us-east-1:000000000000:table/BarkTable/stream/2024-07-12T06:18:37.101 ``` Replace the `stream-arn` value with the stream ARN you obtained from the previous command. ## API Coverage # Elastic Compute Cloud (EC2) > Get started with Amazon Elastic Compute Cloud (EC2) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction Elastic Compute Cloud (EC2) is a core service within Amazon Web Services (AWS) that provides scalable and flexible virtual computing resources. EC2 enables users to launch and manage virtual machines, referred to as instances. LocalStack allows you to use the EC2 APIs in your local environment to create and manage EC2 instances and related resources such as VPCs, EBS volumes, etc. The list of supported APIs can be found on the [API Coverage section](#api-coverage). ## Getting started This guide is designed for users new to EC2 and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. We will demonstrate how to create an EC2 instance that runs a simple Python web server. LocalStack for AWS running on a Linux host is required as network access to containers is not possible on macOS. Start your LocalStack container using your preferred method. ### Create or import a key pair Key pairs are SSH public key/private key combinations that are used to log in to created instances. To create a key pair, you can use the [`CreateKeyPair`](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateKeyPair.html) API. Run the following command to create the key pair and pipe the output to a file named `key.pem`: ```bash lstk aws ec2 create-key-pair \ --key-name my-key \ --query 'KeyMaterial' \ --output text | tee key.pem ``` You may need to assign necessary permissions to the key files for security reasons. This can be done using the following commands: ```bash chmod 400 key.pem ``` ```bash $acl = Get-Acl -Path "key.pem" $fileSystemAccessRule = New-Object System.Security.AccessControl.FileSystemAccessRule("$env:username", "Read", "Allow") $acl.SetAccessRule($fileSystemAccessRule) $acl.SetAccessRuleProtection($true, $false) Set-Acl -Path "key.pem" -AclObject $acl ``` ```bash icacls.exe key.pem /reset icacls.exe key.pem /grant:r "$($env:username):(r)" icacls.exe key.pem /inheritance:r ``` If you already have an SSH public key that you wish to use, such as the one located in your home directory at `~/.ssh/id_rsa.pub`, you can import it instead. ```bash lstk aws ec2 import-key-pair --key-name my-key --public-key-material "$(cat ~/.ssh/id_rsa.pub)" ``` If you only have the SSH private key, a public key can be generated using the following command, and then imported: ```bash ssh-keygen -y -f id_rsa > id_rsa.pub ``` ### Add rules to your security group Currently, LocalStack only supports the `default` security group. You can add rules to the security group using the [`AuthorizeSecurityGroupIngress`](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_AuthorizeSecurityGroupIngress.html) API. Run the following command to add a rule to allow inbound traffic on port 8000: ```bash lstk aws ec2 authorize-security-group-ingress \ --group-id default \ --protocol tcp \ --port 8000 \ --cidr 0.0.0.0/0 ``` The above command will enable rules in the security group to allow incoming traffic from your local machine on port 8000 of an emulated EC2 instance. ### Run an EC2 instance You can fetch the Security Group ID using the [`DescribeSecurityGroups`](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeSecurityGroups.html) API. Run the following command to fetch the Security Group ID: ```bash lstk aws ec2 describe-security-groups ``` ```bash title="Output" { "SecurityGroups": [ { "Description": "default VPC security group", "GroupName": "default", ... "OwnerId": "000000000000", "GroupId": "sg-0372ee3c519883079", ... } ] } ``` To start your Python Web Server in your locally emulated EC2 instance, you can use the following user script by saving it to a file named `user_script.sh`: ```bash #!/bin/bash -xeu apt update apt install python3 -y python3 -m http.server 8000 ``` You can now run an EC2 instance using the [`RunInstances`](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_RunInstances.html) API. Run the following command to run an EC2 instance by adding the appropriate Security Group ID that we fetched in the previous step: ```bash lstk aws ec2 run-instances \ --image-id ami-df5de72bdb3b \ --count 1 \ --instance-type t3.nano \ --key-name my-key \ --security-group-ids '' \ --user-data file://./user_script.sh ``` ### Test the Python web server You can now open the LocalStack logs to find the IP address of the locally emulated EC2 instance. Run the following command to open the LocalStack logs: ```bash lstk logs ``` ```bash title="Output" emulator | 2023-08-16T17:18:29.702 INFO --- [ asgi_gw_0] l.s.ec2.vmmanager.docker : Instance i-b07acefd77a3c415f will be accessible via SSH at: 127.0.0.1:12862, 172.17.0.4:22 emulator | 2023-08-16T17:18:29.702 INFO --- [ asgi_gw_0] l.s.ec2.vmmanager.docker : Instance i-b07acefd77a3c415f port mappings (container -> host): {'8000/tcp': 29043, '22/tcp': 12862} ``` You can now use the IP address to test the Python Web Server. Run the following command to test the Python Web Server: ```bash curl 172.17.0.4:8000 # Or, you can run curl 127.0.0.1:29043 ``` ```bash title="Output" Directory listing for / ... ``` :::note Similar to the setup in production AWS, the user data content is stored at `/var/lib/cloud/instances//` within the instance. Any execution of this data is recorded in the `/var/log/cloud-init-output.log` file. ::: ### Connecting via SSH You can also set up an SSH connection to the locally emulated EC2 instance using the instance IP address. This section assumes that you have created or imported an SSH key pair named `my-key`. When running the EC2 instance, make sure to pass the `--key-name` parameter to the command: ```bash lstk aws ec2 run-instances --key-name my-key ... ``` Once the instance is up and running, we can use the `ssh` command to set up an SSH connection. Assuming the instance is available under `127.0.0.1:12862` (as per the LocalStack log output), use this command: ```bash ssh -p 12862 -i key.pem root@127.0.0.1 ``` :::tip If the `ssh` command throws an error like "Identity file not accessible" or "bad permissions", make sure that the key file has a restrictive `0400` permission as illustrated above. ::: ## VM Managers LocalStack EC2 supports multiple methods to simulate the EC2 service. All tiers support the mock/CRUD capability. For advanced setups, LocalStack for AWS comes with emulation capability for certain resource types so that they behave more closely like AWS. The underlying method for this can be controlled using the [`EC2_VM_MANAGER`](/aws/customization/configuration-options#ec2) configuration option. You may choose between plain mocked resources, containerized emulation, or the [Kubernetes executor](/aws/customization/kubernetes/kubernetes-executor/#ec2-kubernetes-executor). :::note The Libvirt VM manager (`EC2_VM_MANAGER=libvirt`) has been retired and is no longer available. If you were using it to launch fully virtualized EC2 instances, switch to the [Docker VM manager](#docker-vm-manager). ::: ## Mock VM Manager With the Mock VM manager, all resources are stored as in-memory representation. This only offers the CRUD capability. To use this VM manager in LocalStack for AWS, set [`EC2_VM_MANAGER`](/aws/customization/configuration-options#ec2) to `mock`. This serves as the fallback manager if an operation is not implemented in other VM managers. ## Docker VM Manager LocalStack for AWS supports the Docker VM manager which uses the [Docker Engine](https://docs.docker.com/engine/) to emulate EC2 instances. This VM manager requires the Docker socket from the host machine to be mounted inside the LocalStack container at `/var/run/docker.sock`. This is the default VM manager in LocalStack for AWS. You may set [`EC2_VM_MANAGER`](/aws/customization/configuration-options#ec2) to `docker` to explicitly use this VM manager. All launched EC2 instances have the Docker socket mounted inside them at `/var/run/docker.sock` to make Docker-in-Docker usecases possible. All limitations associated with containers are also applicable to EC2 instances managed by the Docker manager. These restrictions include things like root access and networking. Please note that this VM manager does not fully support persistence. While the records of resources will be persisted, the instances or AMIs themselves (i.e. Docker containers and Docker images) will not be persisted. ### AMIs Docker base images which are tagged with the scheme `localstack-ec2/:` are recognized as Amazon Machine Images (AMIs). These can be used to launch EC2 instances which are in fact Docker containers. You can mark any Docker base image as AMI using the below command: ```bash docker tag ubuntu:focal localstack-ec2/ubuntu-focal-ami:ami-000001 ``` The above example will make LocalStack treat the `ubuntu:focal` Docker image as an AMI with name `ubuntu-focal-ami` and ID `ami-000001`. At startup, LocalStack downloads the following AMIs that can be used to launch Dockerized instances. - Ubuntu 26.04: `ami-61ad6e59d7b0` - Amazon Linux 2023: `ami-024f768332f0` :::note The auto download of Docker images for default AMIs can be disabled using the `EC2_DOWNLOAD_DEFAULT_IMAGES=0` configuration variable. ::: :::note Amazon Linux 2 reached its [AWS end-of-life](https://aws.amazon.com/amazon-linux-2/faqs/) on June 30, 2026, so LocalStack no longer pre-loads the `Amazon Linux 2` AMI (`ami-07b643b5e45e`) by default. You can still use it by manually registering it as a Docker-backed AMI: ```bash docker pull amazonlinux:2 docker tag amazonlinux:2 localstack-ec2/amazonlinux-2-ami:ami-07b643b5e45e ``` ::: All LocalStack-managed Docker AMIs bear the resource tag `ec2_vm_manager:docker`. These can be listed using: ```bash lstk aws ec2 describe-images \ --filters Name=tag:ec2_vm_manager,Values=docker ``` :::note If an AMI does not have the `ec2_vm_manager:docker` tag, it means that it is mocked. Attempting to launch Dockerized instances using these AMIs will result in an `InvalidAMIID.NotFound` error. See [Mock VM manager](#mock-vm-manager). ::: AWS does not provide an API to download AMIs which prevents the use of real AWS AMIs on LocalStack. However, in certain cases it may be possible to tweak your workflow to make it work with Localstack. For example, you can use [Packer](https://packer.io/) to customise the Amazon Linux AMI on AWS. Packer can be made to use the [Docker builder](https://developer.hashicorp.com/packer/integrations/hashicorp/docker/latest/components/builder/docker) instead of the Amazon builder and add the customisations on top of the Amazon Linux [Docker base image](https://hub.docker.com/_/amazonlinux/). The final image then can be used by LocalStack EC2 as illustrated above. ### Instances When `RunInstances` is invoked, LocalStack creates an underlying Docker container to simulate an instance. Docker containers that back EC2 instances have the naming scheme `localstack-ec2.`. LocalStack EC2 supports execution of user data scripts when the instance starts. A shell script can be passed to the `UserData` argument of `RunInstances`. Alternatively, the user data may also be added using the `ModifyInstanceAttribute` operation. The user data is placed at `/var/lib/cloud/instances//` in the container. The execution log is generated at `/var/log/cloud-init-output.log` in the container. ### Networking :::note Network access from host to EC2 instance containers is not possible on macOS. This is because Docker Desktop on macOS does not expose the bridge network to the host system. See [Docker Desktop Known Limitations](https://docs.docker.com/desktop/networking/#known-limitations). ::: Network addresses for Dockerized instances are allocated by the Docker daemon and can be obtained from the `PublicIpAddress` attribute. These addresses are also printed in the logs while the instance is being initialized. ```bash 2022-03-21T14:46:49.540 INFO Instance i-1d6327abf04e31be6 will be accessible via SSH at: 127.0.0.1:55705 ``` When instances are launched, LocalStack attempts to start SSH server `/usr/sbin/sshd` in the Docker base image. If not found, it installs and starts the [Dropbear](https://github.com/mkj/dropbear) SSH server. To be able to access the instance at additional ports from the host system, you can modify the default security group and include the required ingress ports. :::note Security group ingress rules are applied only during the creation of the Dockerized instance. Modifying a security group will not open any ports for a running instance. ::: The system supports up to 32 ingress ports. This constraint is in place to prevent exhausting free ports on the host. ```bash lstk aws ec2 authorize-security-group-ingress \ --group-id default \ --protocol tcp \ --port 8080 lstk aws ec2 describe-security-groups --group-names default ``` The port mapping details are provided in the logs when the instance starts up. ```bash title="Output" 2022-12-20T19:43:44.544 INFO Instance i-1d6327abf04e31be6 port mappings (container -> host): {'8080/tcp': 51747, '22/tcp': 55705} ``` ### Elastic Block Store A common use case is to attach an EBS block device to an EC2 instance, which can then be used to create a custom filesystem for additional storage. This section illustrates how this functionality can be achieved with EC2 Docker instances in LocalStack. :::note This feature is disabled by default. Please set the [`EC2_MOUNT_BLOCK_DEVICES`](/aws/customization/configuration-options#ec2) configuration option to enable it. ::: First, we create a user data script `init.sh` which creates an ext3 file system on the block device `/ebs-dev/sda1` and mounts it under `/ebs-mounted`: ```bash cat > init.sh <22/tcp localstack-ec2... ``` You can then list the contents of the mounted filesystem `/ebs-mounted`, which should contain our test file named `my-test-file`: ```bash docker exec 5c60cf72d84a ls /ebs-mounted ``` ```bash title="Output" my-test-file ``` ### Instance Metadata Service The Docker VM manager supports the [Instance Metadata Service](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html) which provides information about the running instance. Both IMDSv1 and IMDSv2 can be used. LocalStack does not strictly enforce either versions. If the `X-aws-ec2-metadata-token` header is present, LocalStack will use IMDSv2, otherwise it will fall back to IMDSv1. To create an IMDSv2 token, run the following inside the EC2 container: ```bash curl -X PUT "http://169.254.169.254/latest/api/token" -H "x-aws-ec2-metadata-token-ttl-seconds: 300" ``` The token can be used in subsequent requests like so: ```bash curl -H "x-aws-ec2-metadata-token: " -v http://169.254.169.254/latest/meta-data/ ``` You can use the [`ModifyInstanceMetadataOptions`](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_ModifyInstanceMetadataOptions.html) API to change the metadata options of a running instance, for example to require IMDSv2. Parameters that are omitted from the request retain their current value, matching AWS behavior. :::note IMDS IPv6 endpoint is currently not supported. ::: #### Metadata Categories Currently a limited set of [metadata categories](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html#instancedata-data-categories) are implemented. They are: - `ami-id` - `ami-launch-index` - `instance-id` - `instance-type` - `local-hostname` - `local-ipv4` - `public-hostname` - `public-ipv4` If you would like support for more metadata categories, please make a feature request on [GitHub Discussion](https://github.com/orgs/localstack/discussions/new/choose). ### Configuration You can use the [`EC2_DOCKER_FLAGS`](/aws/customization/configuration-options#ec2) LocalStack configuration variable to pass supplementary flags to Docker during the initiation of containerized instances. This allows for fine-tuned behaviours, for example, running containers in privileged mode using `--privileged` or specifying an alternate CPU platform with `--platform`. Keep in mind that this will apply to all instances that are launched in the LocalStack session. ### Operations The following table explains the emulated action for various API operations. Any operation not listed below will use the mock VM manager. | Operation | Notes | |:----------------------|:---------------------------------------------------------------------------------------------| | `CreateImage` | Uses Docker commit to capture a snapshot of a running instance into a new AMI | | `DescribeImages` | Retrieves a list of Docker images that can be used as AMIs | | `DescribeInstances` | Describes both mocked and Docker-backed instances. Docker-backed instances are marked with the resource tag `ec2_vm_manager:docker` | | `RunInstances` | Creates and runs Docker containers that back instances | | `StopInstances` | Pauses the Docker containers that back instances | | `StartInstances` | Resumes the Docker containers that back instances | | `TerminateInstances` | Stops the Docker containers that back instances | | `CreateFleet` | Spawns Docker containers or Kubernetes pods to fulfill fleet capacity requests. Supports On-Demand, Spot, and mixed fleets. | | `DeleteFleets` | Stops and removes the underlying containers or pods when `TerminateInstances` is set to `true`. | ## IAM Condition Keys When [IAM Policy Enforcement](/aws/developer-tools/security-testing/iam-policy-enforcement/) is enabled, LocalStack supports the following EC2-specific condition keys, matching the behavior described in the [AWS condition keys reference](https://docs.aws.amazon.com/service-authorization/latest/reference/list_amazonec2.html#amazonec2-policy-keys): - `ec2:MetadataHttpTokens` — the `HttpTokens` value of an instance's [metadata options](#instance-metadata-service), useful for enforcing IMDSv2. - `ec2:Attribute/` — exposes request parameters (e.g. `HttpTokens` on `ModifyInstanceMetadataOptions`) as condition keys. For example, the following policy statement only allows launching instances when IMDSv2 is required: ```json { "Effect": "Allow", "Action": "ec2:RunInstances", "Resource": "arn:aws:ec2:*:*:instance/*", "Condition": { "StringEquals": { "ec2:MetadataHttpTokens": "required" } } } ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing EC2 instances. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **EC2** under the **Compute** section. ![EC2 Resource Browser](/images/aws/ec2-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Instance**: Create a new EC2 instance by clicking the **Launch Instance** button and specifying the AMI ID, instance type, and other parameters. - **View Instance**: View the details of an EC2 instance by clicking on the Instance ID. - **Terminate Instance**: Terminate an EC2 instance by selecting the Instance ID, and clicking on the **ACTIONS** button followed by clicking on **Terminate Selected**. - **Start Instance**: Start a stopped EC2 instance by selecting the Instance ID, and clicking on the **ACTIONS** button followed by clicking on **Start Selected**. - **Stop Instance**: Stop a running EC2 instance by selecting the Instance ID, and clicking on the **ACTIONS** button followed by clicking on **Stop Selected**. ## API Coverage # Elastic Container Registry (ECR) > Get started with Elastic Container Registry (ECR) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Elastic Container Registry (ECR) is a fully managed container registry service provided by Amazon Web Services. ECR enables you to store, manage, and deploy Docker container images to build, store, and deploy containerized applications. ECR integrates with other AWS services, such as Lambda, ECS, and EKS. LocalStack allows you to use the ECR APIs in your local environment to build & push Docker images to a local ECR registry. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of ECR's integration with LocalStack. ## Getting started This guide is designed for users new to Elastic Container Registry and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to build and push a Docker image to a local ECR repository. ### Create a Docker image To get started, create a Docker image for a simple web application that can be used in an ECS task definition. Create a new file named `Dockerfile` (with no file extension) in your project directory. This file will contain the instructions for building the Docker image. Add the following content to the file: ```Dockerfile FROM public.ecr.aws/docker/library/ubuntu:18.04 # Install dependencies RUN apt-get update && \ apt-get -y install apache2 # Install apache and write hello world message RUN echo 'Hello World!' > /var/www/html/index.html # Configure apache RUN echo '. /etc/apache2/envvars' > /root/run_apache.sh && \ echo 'mkdir -p /var/run/apache2' >> /root/run_apache.sh && \ echo 'mkdir -p /var/lock/apache2' >> /root/run_apache.sh && \ echo '/usr/sbin/apache2 -D FOREGROUND' >> /root/run_apache.sh && \ chmod 755 /root/run_apache.sh EXPOSE 80 CMD /root/run_apache.sh ``` You can now build the Docker image from the `Dockerfile` using the `docker CLI: ```bash docker build -t localstack-ecr-image . ``` You can run the following command to verify that the image was built successfully: ```bash docker images ``` ```bash title="Output" REPOSITORY TAG IMAGE ID CREATED SIZE .. localstack-ecr-image latest 38883941b8fa 1 minute ago 185MB ``` ### Create an ECR repository To push the Docker image to ECR, you first need to create a repository. You can create an ECR repository using the [`CreateRepository`](https://docs.aws.amazon.com/AmazonECR/latest/APIReference/API_CreateRepository.html) API. Run the following command to create a repository named `localstack-ecr-repository`: ```bash lstk aws ecr create-repository \ --repository-name localstack-ecr-repository \ --image-scanning-configuration scanOnPush=true ``` ```bash title="Output" { "repository": { "repositoryArn": "arn:aws:ecr:us-east-1:000000000000:repository/localstack-ecr-repository", "registryId": "000000000000", "repositoryName": "localstack-ecr-repository", "repositoryUri": "000000000000.dkr.ecr.us-east-1.localhost.localstack.cloud:4566/localstack-ecr-repository", "createdAt": "2023-07-24T16:58:36+05:30", "imageTagMutability": "MUTABLE", "imageScanningConfiguration": { "scanOnPush": true }, "encryptionConfiguration": { "encryptionType": "AES256" } } } ``` You will need the `repositoryUri` value to push the Docker image to the repository. ### Push the Docker image to the repository To push the Docker image to the repository, you first need to tag the image with the `repositoryUri`. Run the following command to tag the image: ```bash docker tag localstack-ecr-image 000000000000.dkr.ecr.us-east-1.localhost.localstack.cloud:4566/localstack-ecr-repository ``` You can now push the image to the repository using the `docker` CLI: ```bash docker push 000000000000.dkr.ecr.us-east-1.localhost.localstack.cloud:4566/localstack-ecr-repository ``` The image will take a few seconds to push to the repository. You can run the following command to verify that the image was pushed successfully: ```bash lstk aws ecr list-images --repository-name localstack-ecr-repository ``` ```bash title="Output" { "imageIds": [ { "imageDigest": "sha256:1cbc853c42983362817b5eecac80b1389c0a5cf9cfd1e711d9d0a1f5a7a36d43", "imageTag": "latest" } ] } ``` ## Endpoint Strategy The `ECR_ENDPOINT_STRATEGY` configuration variable in LocalStack determines how the endpoints for ECR repositories are constructed. The different values for `ECR_ENDPOINT_STRATEGY` are: - `ECR_ENDPOINT_STRATEGY=domain`: Uses a domain-based approach for the repository URIs. In this case, the URI includes the AWS account ID and the region, similar to how it would appear in a real AWS environment. For example: ```bash "repositoryUri": "000000000000.dkr.ecr.us-east-1.localhost.localstack.cloud:4566/lambda-ecr-repo" ``` This mimics the structure of real AWS ECR URIs, which can help ensure compatibility with tools and scripts that expect this format. - `ECR_ENDPOINT_STRATEGY=off`: This strategy disables the domain-based structure and uses a simpler, direct approach. The repository URI is more straightforward and does not include the account ID or region. For example: ```bash "repositoryUri": "localhost.localstack.cloud:4510/lambda-ecr-repo" ``` This can simplify configurations and reduce potential issues with domain resolution. If you are experiencing issues with domain name resolution or formatting, switching from `domain` to `off` will likely resolve the problem by simplifying the repository URI and avoiding potential complications. ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing ECR repositories and images. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **ECR** under the **Compute** section. ![ECR Resource Browser](/images/aws/ecr-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create repository**: Create a new ECR repository by clicking the **Create** button, and specify the **Registry Id**, **Repository Name**, **Tags**, and other options. - **View repository**: View the details of an ECR repository by clicking on the repository name. You can also view the push commands to push an image to the repository by clicking the **View Push Commands** button. - **Delete repository**: Delete an ECR repository by selecting the ECR repository, clicking the **Actions** button, and then clicking **Remove Selected**. ## Examples The following code snippets and sample applications provide practical examples of how to use ECR in LocalStack for various use cases: - [Amazon RDS initialization using CDK, Lambda, ECR, and Secrets Manager](https://github.com/localstack/amazon-rds-init-cdk) - [Lambda Container Images with ECR](https://github.com/localstack/localstack-pro-samples/tree/master/lambda-container-image) - [Pushing Docker images to ECR and running them locally on ECS](https://github.com/localstack/localstack-pro-samples/tree/master/ecs-ecr-container-app) ## API Coverage # Elastic Container Service (ECS) > Get started with Elastic Container Service (ECS) on LocalStack import { Badge } from '@astrojs/starlight/components'; import FeatureCoverage from '../../../../components/feature-coverage/FeatureCoverage'; ## Introduction Amazon Elastic Container Service (Amazon ECS) is a fully managed container orchestration service provided by Amazon Web Services (AWS). It allows you to run, stop, and manage Docker containers on a cluster. ECS eliminates the need for you to install, operate, and scale your own cluster management infrastructure. LocalStack allows you to use the ECS APIs in your local environment to create & manage ECS clusters, tasks, and services. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of ECS's integration with LocalStack. ## Getting Started This guide is designed for users new to ECS and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create an ECS service using the AWS CLI ### Create a cluster :::note By default, the **ECS Fargate** launch type is assumed, i.e., the local Docker engine is used for deployment of applications, and there is no need to create and manage EC2 virtual machines to run the containers. ::: ECS tasks and services run on a cluster. Execute the following command to create an ECS cluster named `mycluster`: ```bash lstk aws ecs create-cluster --cluster-name mycluster ``` ```bash title="Output" { "cluster": { "clusterArn": "arn:aws:ecs:us-east-1:000000000000:cluster/mycluster", "clusterName": "mycluster", "status": "ACTIVE", "registeredContainerInstancesCount": 0, "runningTasksCount": 0, "pendingTasksCount": 0, "activeServicesCount": 0, "settings": [ { "name": "containerInsights", "value": "disabled" } ] } } ``` ### Create a task definition Containers within tasks are defined by a task definition that is managed outside of the context of a cluster. To create a task definition that runs an `ubuntu` container forever (by running an infinite loop printing "Running" on startup), create the following file as `task_definition.json`: ```json showshowLineNumbers { "containerDefinitions": [ { "name": "server", "image": "ubuntu", "cpu": 10, "memory": 10, "command": ["sh", "-c", "while true; do echo running; sleep 1; done"], "essential": true, "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-create-group": "true", "awslogs-group": "myloggroup", "awslogs-stream-prefix": "myprefix", "awslogs-region": "us-east-1" } } } ], "family": "myfamily" } ``` and then run the following command: ```bash lstk aws ecs register-task-definition --cli-input-json file://task_definition.json ``` ```bash title="Output" { "taskDefinition": { "taskDefinitionArn": "arn:aws:ecs:us-east-1:000000000000:task-definition/myfamily:1", "containerDefinitions": [ { "name": "server", "image": "ubuntu", "cpu": 10, "memory": 10, "portMappings": [], "essential": true, "command": [ "sh", "-c", "while true; do echo running; sleep 1; done" ], "environment": [], "mountPoints": [], "volumesFrom": [], "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-create-group": "true", "awslogs-group": "myloggroup", "awslogs-stream-prefix": "myprefix", "awslogs-region": "us-east-1" } } } ], "family": "myfamily", "networkMode": "bridge", "revision": 1, "volumes": [], "status": "ACTIVE", "placementConstraints": [], "compatibilities": [ "EXTERNAL", "EC2" ], "registeredAt": 1713364207.068659 } } ``` Task definitions are immutable, and are identified by their `family` field, and calling `register-task-definition` again with the same `family` value creates a new _version_ of a task definition. This task definition creates a CloudWatch Logs log group and log stream for the container so you can view the service logs. ### Launch a service Finally we launch an ECS service using the task definition above. This will create a number of containers in replica mode meaning they are distributed over the nodes of the cluster, or in the case of Fargate, over availability zones within the region of the cluster. To create a service, execute the following command: ```bash lstk aws ecs create-service --service-name myservice --cluster mycluster --task-definition myfamily --desired-count 1 ``` ```bash title="Output" { "service": { "serviceArn": "arn:aws:ecs:us-east-1:000000000000:service/mycluster/myservice", "serviceName": "myservice", "clusterArn": "arn:aws:ecs:us-east-1:000000000000:cluster/mycluster", "loadBalancers": [], "serviceRegistries": [], "status": "ACTIVE", "desiredCount": 1, "runningCount": 1, "pendingCount": 0, "launchType": "EC2", "taskDefinition": "arn:aws:ecs:us-east-1:000000000000:task-definition/myfamily:1", "deploymentConfiguration": { "deploymentCircuitBreaker": { "enable": false, "rollback": false }, "maximumPercent": 200, "minimumHealthyPercent": 100 }, "deployments": [ { "id": "ecs-svc/49976591540684372", "status": "PRIMARY", "taskDefinition": "arn:aws:ecs:us-east-1:000000000000:task-definition/myfamily:1", "desiredCount": 1, "pendingCount": 0, "runningCount": 1, "failedTasks": 0, "createdAt": 1709242525.05109, "updatedAt": 1709242525.051093, "launchType": "EC2", "rolloutState": "IN_PROGRESS", "rolloutStateReason": "ECS deployment ecs-svc/49976591540684372 in progress." } ], "events": [], "createdAt": 1709242525.051096, "placementStrategy": [], "schedulingStrategy": "REPLICA", "createdBy": "arn:aws:iam::000000000000:user/test" } } ``` You should see a new docker container has been created, using the `ubuntu:latest` image, and running the infinite loop command: ```bash docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 5dfeb9376391 ubuntu "sh -c 'while true; …" 3 minutes ago Up 3 minutes ls-ecs-mycluster-75f0515e-0364-4ee5-9828-19026140c91a-0-a1afaa9d 9967fe5300cc localstack/localstack-pro "docker-entrypoint.sh" 5 minutes ago Up 5 minutes (healthy) 0.0.0.0:443->443/tcp, 0.0.0.0:4510-4560->4510-4560/tcp, 53/tcp, 5678/tcp, 0.0.0.0:4566->4566/tcp localstack-main ``` ### Collect container logs To access the generated logs from the container, run the following command: ```bash lstk aws logs filter-log-events --log-group-name myloggroup --query 'events[].message' ``` ```bash title="Output" { "events": [ { "logStreamName": "myprefix/ls-ecs-mycluster-75f0515e-0364-4ee5-9828-19026140c91a-0-a1afaa9d/75f0515e-0364-4ee5-9828-19026140c91a", "timestamp": 1713364216375, "message": "running", "ingestionTime": 1713364216704, "eventId": "0" }, { "logStreamName": "myprefix/ls-ecs-mycluster-75f0515e-0364-4ee5-9828-19026140c91a-0-a1afaa9d/75f0515e-0364-4ee5-9828-19026140c91a", "timestamp": 1713364216440, "message": "running", "ingestionTime": 1713364216704, "eventId": "1" }, { "logStreamName": "myprefix/ls-ecs-mycluster-75f0515e-0364-4ee5-9828-19026140c91a-0-a1afaa9d/75f0515e-0364-4ee5-9828-19026140c91a", "timestamp": 1713364216505, "message": "running", ``` See our [CloudWatch Logs user guide](/aws/services/logs) for more details. ## LocalStack ECS behavior You can use the configuration option `MAIN_DOCKER_NETWORK` to specify the network the ECS containers are started in. Otherwise, your ECS containers will be created in the same Docker network that LocalStack is in. If your ECS containers depend on LocalStack services, your ECS task network should be the same as the LocalStack container network. If you are running LocalStack through a `docker run` command, do not forget to enable the communication from the container to the Docker Engine API. You can provide the access by adding the following option `-v /var/run/docker.sock:/var/run/docker.sock`. For more information regarding the configuration of LocalStack, please check the [LocalStack configuration](/aws/customization/configuration-options) section. ## Remote debugging To enable a remote debugging port for your ECS tasks, set the environment variable `ECS_DOCKER_FLAGS="-p 0:"` to expose your debugger on a random port on your host. You can then use this port to remote attach your debugger. Or if you are working with a single container, you can set `ECS_DOCKER_FLAGS="-p :"` to expose the debugger port to your host system. ## Mounting local directories for ECS tasks In some cases, it can be useful to mount code from the host filesystem into the ECS container. For example, to enable a quick debugging loop where you can test changes without having to build and redeploy the task's Docker image each time - similar to the [Lambda Hot Reloading](/aws/developer-tools/lambda-tools/hot-reloading) feature in LocalStack. In order to leverage code mounting, we can use the ECS bind mounts feature, which is covered in the [AWS Bind mounts documentation](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/bind-mounts.html). ### Boto3 example The Python sample code below registers a task definition, mounting a host path `/host/path` into the container under `/container/path`: ```bash ecs_client = boto3.client("ecs", endpoint_url="http://localhost:4566") ... ecs_client.register_task_definition( family="...", containerDefinitions=[ { "name": "...", "image": "alpine", "command": ["..."], "mountPoints": [ {"containerPath": "/container/path", "sourceVolume": "test-volume"} ], } ], volumes=[{"host": {"sourcePath": "/host/path"}, "name": "test-volume"}], ) ``` ### CDK example The same functionality can be achieved with the AWS CDK following this (Python) example: ```python showshowLineNumbers task_definition = ecs.TaskDefinition( ... volumes=[ ecs.Volume(name="test-volume", host=ecs.Host(source_path="/host/path")) ] ) container = task_def.add_container(...) container.add_mount_points( ecs.MountPoint( container_path="/container/path", source_volume="test-volume", ), ) ``` ## Private registry authentication To download images from a private registry using LocalStack, you must provide your credentials. LocalStack (as of 4.13.0) supports the `repositoryCredentials` parameter in an ECS task definition allowing ECS to pull images from registries that require authentication. This is currently only implemented for Docker executor, with support for the Kubernetes executor forthcoming. Below is a minimal example demonstrating the use of `repositoryCredentials` in an ECS task definition: ```json showshowLineNumbers { "family": "...", "containerDefinitions": [ { "name": "...", "image": "private-registry.example.com/my-image:latest", "repositoryCredentials": { "credentialsParameter": "arn:aws:secretsmanager:us-east-1:000000000000:secret:my-registry-credentials" } } ] } ``` The `credentialsParameter` value is the ARN of a Secrets Manager secret containing the registry credentials. ## Firelens for ECS Tasks LocalStack's ECS emulation supports custom log routing via FireLens. FireLens allows the ECS service to manage the configuration of the logging driver of application containers, and to create the proper configuration for the `fluentbit`/`fluentd` logging layer. However, you cannot use ECS on Kubernetes with FireLens. ### Custom Config Support LocalStack ECS FireLens now supports the provision of custom configurations: - You can supply a custom Fluent Bit configuration through the ECS task definition. The config is included in the default configuration via the [`@INCLUDE` directive](https://docs.fluentbit.io/manual/administration/configuring-fluent-bit/classic-mode/configuration-file#config-include-file). - Custom config files can be provisioned via S3 (for EC2 tasks) or be baked into the image (supported for both Fargate and EC2 tasks). - LocalStack now also supports the `s3` fluentbit plugin for extended log routing use-cases. For usage details and configuration patterns, refer to the [AWS ECS FireLens custom config documentation](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/firelens-taskdef.html#firelens-taskdef-customconfig). ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing ECS clusters & task definitions. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resource Browser** section, and then clicking on **ECS** under the **Compute** section. ![ECS Resource Browser](/images/aws/ecs-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Cluster**: Create a new ECS cluster by clicking on the **Create Cluster** button in the **Clusters** tab and providing the cluster name among other details. - **Register Task Definition**: Register a new task definition by clicking on the **Register Task Definition** button in the **Task Definitions** tab and providing the task definition details. - **View Cluster Details**: Click on a cluster in the **Clusters** tab to view the cluster details, including the cluster ARN, status, and other information. - **View Task Definition Details**: Click on a task definition in the **Task Definitions** tab to view the task definition details, including the task definition ARN, family, and other information. - **Edit Cluster**: Click on the **Edit Cluster** button while you are viewing a cluster to edit the cluster details. - **Edit Task Definition**: Click on the **Edit Task Definition** button while you are viewing a task definition to edit the task definition details. - **Delete Cluster**: Select the cluster name in the **Clusters** tab and click on the **Actions** button followed by **Remove Selected** button. - **Delete Task Definition**: Select the task definition name in the **Task Definitions** tab and click on the **Actions** button followed by **Remove Selected** button. ## API Coverage # Elastic File System (EFS) > Get started with Elastic File System (EFS) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Elastic File System (EFS) is a fully managed file storage service provided by Amazon Web Services (AWS). EFS offers scalable and shared file storage that can be accessed by multiple EC2 instances and on-premises servers simultaneously. EFS utilizes the Network File System protocol to allow it to be used as a data source for various applications and workloads. LocalStack allows you to use the EFS APIs in your local environment to create local file systems, lifecycle configurations, and file system policies. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of EFS's integration with LocalStack. ## Getting started This guide is designed for users new to Elastic File System and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a file system, apply an IAM resource-based policy, and create a lifecycle configuration using the AWS CLI. ### Create a filesystem To create a new, empty file system you can use the [`CreateFileSystem`](https://docs.aws.amazon.com/goto/WebAPI/elasticfilesystem-2015-02-01/CreateFileSystem) API. Run the following command to create a new file system: ```bash lstk aws efs create-file-system \ --performance-mode generalPurpose \ --throughput-mode bursting \ --encrypted \ --tags Key=Name,Value=my-file-system ``` ```bash title="Output" { "CreationToken": "53465731-0032-4cef-92f5-8aefe7c7b91e", "FileSystemId": "fs-34feac549e66b814", "FileSystemArn": "arn:aws:elasticfilesystem:us-east-1:000000000000:file-system/fs-34feac549e66b814", "CreationTime": 1692808338.424, "LifeCycleState": "available", "PerformanceMode": "generalPurpose", "Encrypted": true, "ThroughputMode": "bursting", "Tags": [ { "Key": "Name", "Value": "my-file-system" } ] } ``` You can also describe the locally available file systems using the [`DescribeFileSystems`](https://docs.aws.amazon.com/efs/latest/ug/API_DescribeFileSystems.html) API. Run the following command to describe the local file systems available: ```bash lstk aws efs describe-file-systems ``` You can alternatively pass the `--file-system-id` parameter to the `describe-file-system` command to retrieve information about a specific file system in AWS CLI. ### Put file system policy You can apply an EFS `FileSystemPolicy` to an EFS file system using the [`PutFileSystemPolicy`](https://docs.aws.amazon.com/efs/latest/ug/API_PutFileSystemPolicy.html) API. Run the following command to apply a policy to the file system created in the previous step: ```bash lstk aws efs put-file-system-policy \ --file-system-id \ --policy "{\"Version\":\"2012-10-17\",\"Id\":\"ExamplePolicy01\",\"Statement\":[{\"Sid\":\"ExampleStatement01\",\"Effect\":\"Allow\",\"Principal\":{\"AWS\":\"*\"},\"Action\":[\"elasticfilesystem:ClientMount\",\"elasticfilesystem:ClientWrite\"],\"Resource\":\"arn:aws:elasticfilesystem:us-east-1:000000000000:file-system/fs-34feac549e66b814\"}]}" ``` You can list the file system policies using the [`DescribeFileSystemPolicy`](https://docs.aws.amazon.com/efs/latest/ug/API_DescribeFileSystemPolicy.html) API. Run the following command to list the file system policies: ```bash lstk aws efs describe-file-system-policy \ --file-system-id ``` Replace `` with the ID of the file system you want to list the policies for. The output will return the `FileSystemPolicy` for the specified EFS file system. ### Create a lifecycle configuration You can create a lifecycle configuration for an EFS file system using the [`PutLifecycleConfiguration`](https://docs.aws.amazon.com/efs/latest/ug/API_PutLifecycleConfiguration.html) API. Run the following command to create a lifecycle configuration for the file system created in the previous step: ```bash lstk aws efs put-lifecycle-configuration \ --file-system-id \ --lifecycle-policies "{\"TransitionToIA\":\"AFTER_30_DAYS\"}" ``` ```bash title="Output" { "LifecyclePolicies": [ { "TransitionToIA": "AFTER_30_DAYS" } ] } ``` ## Current Limitations LocalStack uses Moto to emulate the EFS APIs. The legacy [`CreateTags`](https://docs.aws.amazon.com/efs/latest/ug/API_CreateTags.html), [`DeleteTags`](https://docs.aws.amazon.com/efs/latest/ug/API_DeleteTags.html), and [`DescribeTags`](https://docs.aws.amazon.com/efs/latest/ug/API_DescribeTags.html) operations are not supported; use [`TagResource`](https://docs.aws.amazon.com/efs/latest/ug/API_TagResource.html), [`UntagResource`](https://docs.aws.amazon.com/efs/latest/ug/API_UntagResource.html), and [`ListTagsForResource`](https://docs.aws.amazon.com/efs/latest/ug/API_ListTagsForResource.html) instead. ## API Coverage # Elastic Kubernetes Service (EKS) > Get started with Elastic Kubernetes Service (EKS) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; import { Tabs, TabItem, Steps } from '@astrojs/starlight/components'; ## Introduction Elastic Kubernetes Service (EKS) is a managed Kubernetes service that makes it easy to run Kubernetes on AWS without installing, operating, and maintaining your own Kubernetes control plane or worker nodes. Kubernetes is an open-source system for automating containerized applications' deployment, scaling, and management. LocalStack allows you to use the EKS APIs in your local environment to spin up embedded Kubernetes clusters in your local Docker engine or use an existing Kubernetes installation you can access from your local machine (defined in `$HOME/.kube/config`). The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of EKS's integration with LocalStack. ## Getting started This guide is designed for users new to Elastic Kubernetes Service and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. To interact with the Kubernetes cluster, you should also install [`kubectl`](https://kubernetes.io/docs/tasks/tools/). Start your LocalStack container using your preferred method. We will demonstrate how you can auto-install an embedded Kubernetes cluster, configure ingress, and deploy a sample service with ECR. ### Deploy the necessary networking components First we need to create a VPC for the EKS cluster. You can create a new VPC using the [`CreateVpc` API](https://docs.aws.amazon.com/vpc/latest/APIReference/API_CreateVpc.html). Run the following command: ```bash title="Create VPC" lstk aws ec2 create-vpc --cidr-block 10.0.0.0/16 ``` ```bash title="Output" { "Vpc": { ... "CidrBlock": "10.0.0.0/16", "VpcId": "", ... } } ``` Next, we need to create a subnet in the VPC. You can create a 2 subnets using the [`CreateSubnet` API](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateSubnet.html). Some extra tags might be required for specific Controllers to work properly. Please refer to their specific documentation for more details. Run the following command: ```bash title="Create Subnet 1" lstk aws ec2 create-subnet \ --vpc-id \ --cidr-block 10.0.1.0/24 \ --availability-zone us-east-1a ``` ```bash title="Output" { "Subnet": { ... "SubnetId": "", "VpcId": "", "CidrBlock": "10.0.1.0/24" ... } } ``` ```bash title="Create Subnet 2" lstk aws ec2 create-subnet \ --vpc-id \ --cidr-block 10.0.2.0/24 \ --availability-zone us-east-1b ``` ```bash title="Output" { "Subnet": { ... "SubnetId": "", "VpcId": "", "CidrBlock": "10.0.2.0/24" ... } } ``` ### Create an embedded Kubernetes cluster The default approach for creating Kubernetes clusters using the local EKS API is by setting up an embedded [k3d](https://k3d.io/) kube cluster within Docker. LocalStack seamlessly manages the download and installation process, making it hassle-free for users. In most cases, the installation is automatic, eliminating the need for any manual customizations. :::note The Traefik ingress controller and the default k3d load balancer containers are no longer started automatically when creating an EKS cluster. To restore the previous behavior, set the following configuration variable: ```bash K3D_START_LB_INGRESS=1 ``` ::: :::note If you run LocalStack with Docker Compose, the k3d containers backing an EKS cluster can be left behind as orphans if LocalStack is killed before it finishes shutting down. See [Why are some containers left behind after I stop LocalStack with Docker Compose?](/aws/getting-started/faq/#why-are-some-containers-left-behind-after-i-stop-localstack-with-docker-compose) for the cause and how to configure `stop_grace_period` and `SHUTDOWN_TIMEOUT` to avoid it. ::: You can create a new cluster using the [`CreateCluster` API](https://docs.aws.amazon.com/eks/latest/APIReference/API_CreateCluster.html). Run the following command: ```bash title="Create Cluster" lstk aws eks create-cluster \ --name cluster1 \ --role-arn "arn:aws:iam::000000000000:role/eks-role" \ --resources-vpc-config '{"subnetIds":["", ""]}' ``` ```bash title="Output" { "cluster": { "name": "cluster1", "arn": "arn:aws:eks:us-east-1:000000000000:cluster/cluster1", "createdAt": "2022-04-13T16:38:24.850000+02:00", "roleArn": "arn:aws:iam::000000000000:role/eks-role", "resourcesVpcConfig": { "subnetIds": [ "", "" ] }, "identity": { "oidc": { "issuer": "https://localhost.localstack.cloud/eks-oidc" } }, "status": "CREATING", "clientRequestToken": "cbdf2bb6-fd3b-42b1-afe0-3c70980b5959" } } ``` The cluster creation process may take a few moments as LocalStack sets up the necessary components. Avoid attempting to access the cluster until the status changes to `ACTIVE`. Run the following command to wait for the cluster status to become `ACTIVE`: ```bash title="Wait for Cluster" lstk aws eks wait cluster-active --name cluster1 ``` :::note When setting up a local EKS cluster, if you encounter a `"status": "FAILED"` in the command output and see `Unable to start EKS cluster` in LocalStack logs, remove or rename the `~/.kube/config` file on your machine and retry. The CLI mounts this file automatically for CLI versions before `3.7`, leading EKS to assume you intend to use the specified cluster, a feature that has specific requirements. ::: You can use the `docker` CLI to check that some containers have been created: ```bash docker ps ``` ```bash title="Output" CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ... b335f7f089e4 rancher/k3d-proxy:5.0.1-rc.1 "/bin/sh -c nginx-pr…" 1 minute ago Up 1 minute 0.0.0.0:8081->80/tcp, 0.0.0.0:44959->6443/tcp k3d-cluster1-serverlb f05770ec8523 rancher/k3s:v1.21.5-k3s2 "/bin/k3s server --t…" 1 minute ago Up 1 minute ... ``` ### Creating a managed node group The EKS cluster created in the previous step does not include any worker nodes by default. While you can inspect the server node, it is [tainted](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/), and workloads cannot be scheduled on it. To run workloads on the cluster, you must add at least one worker node. One way to do this is by creating a managed node group. When you create a managed node group, LocalStack automatically provisions a Docker container, joins it to the cluster, and provisions a mocked EC2 instance. You can create a managed node group for your EKS cluster using the [`CreateNodegroup` API](https://docs.aws.amazon.com/eks/latest/APIReference/API_CreateNodegroup.html). Run the following command: ```bash title="Create Node Group" lstk aws eks create-nodegroup \ --cluster-name cluster1 \ --nodegroup-name nodegroup1 \ --node-role arn:aws:iam::000000000000:role/eks-nodegroup-role \ --subnets \ --scaling-config desiredSize=1 ``` ```bash title="Output" { "nodegroup": { "nodegroupName": "nodegroup1", "nodegroupArn": "arn:aws:eks:us-east-1:000000000000:nodegroup/cluster1/nodegroup1/xxx", "clusterName": "cluster1", "version": "1.21", "releaseVersion": "1.21.7-20220114", "createdAt": "2022-04-13T17:25:45.821000+02:00", "status": "CREATING", "capacityType": "ON_DEMAND", "scalingConfig": { "desiredSize": 1 }, "subnets": [ "", "" ], "nodeRole": "arn:aws:iam::000000000000:role/eks-nodegroup-role", "labels": {}, "health": { "issues": [] }, "updateConfig": { "maxUnavailable": 1 } } } ``` The node group creation process may take a few moments as LocalStack sets up the necessary components. You can wait for the node group status to become `ACTIVE` by running the following command: ```bash title="Wait for Node Group" lstk aws eks wait nodegroup-active --cluster-name cluster1 --nodegroup-name nodegroup1 ``` At this point, your EKS cluster is fully operational and ready to deploy workloads. ### Utilizing ECR Images within EKS You can now use ECR (Elastic Container Registry) images within your EKS environment. #### Initial configuration To modify the return value of resource URIs for most services, including ECR, you can utilize the `LOCALSTACK_HOST` variable in the [configuration](/aws/customization/configuration-options). By default, ECR returns a `repositoryUri` starting with `localhost.localstack.cloud`, such as: `localhost.localstack.cloud:/`. :::note In this section, we assume that `localhost.localstack.cloud` resolves in your environment, and LocalStack is connected to a non-default bridge network. For more information, refer to the article about [DNS rebind protection](/aws/customization/networking/dns-server#dns-rebind-protection). If the domain `localhost.localstack.cloud` does not resolve on your host, you can still proceed by setting `LOCALSTACK_HOST=localhost` (not recommended). LocalStack will take care of the DNS resolution of `localhost.localstack.cloud` within ECR itself, allowing you to use the `localhost:/` URI for tagging and pushing the image on your host. ::: Once you have configured this correctly, you can seamlessly use your ECR image within EKS as expected. #### Deploying a sample application from an ECR image To showcase this behavior, let's go through a concise step-by-step guide that will lead us to the successful pulling of an image from local ECR. For the purpose of this guide, we will retag the `nginx` image to be pushed to a local ECR repository under a different name, and then utilize it for a pod configuration. You can create a new ECR repository using the [`CreateRepository` API](https://docs.aws.amazon.com/AmazonECR/latest/APIReference/API_CreateRepository.html). Run the following command: ```bash lstk aws ecr create-repository --repository-name "fancier-nginx" ``` ```bash title="Output" { "repository": { "repositoryArn": "arn:aws:ecr:us-east-1:000000000000:repository/fancier-nginx", "registryId": "c75fd0e2", "repositoryName": "fancier-nginx", "repositoryUri": "000000000000.dkr.ecr.us-east-1.localhost.localstack.cloud:4566/fancier-nginx", "createdAt": "2022-04-13T14:22:47+02:00", "imageTagMutability": "MUTABLE", "imageScanningConfiguration": { "scanOnPush": false }, "encryptionConfiguration": { "encryptionType": "AES256" } } } ``` You can now pull the `nginx` image from Docker Hub using the `docker` CLI: ```bash docker pull nginx ``` You can further tag the image to be pushed to ECR: ```bash docker tag nginx 000000000000.dkr.ecr.us-east-1.localhost.localstack.cloud:4566/fancier-nginx ``` Finally, you can push the image to local ECR: ```bash docker push 000000000000.dkr.ecr.us-east-1.localhost.localstack.cloud:4566/fancier-nginx ``` Now, let us set up the EKS cluster using the image pushed to local ECR. Next, we can configure `kubectl` to use the EKS cluster, using the [`UpdateKubeconfig` API](https://docs.aws.amazon.com/eks/latest/APIReference/API_UpdateClusterConfig.html). Run the following command: ```bash lstk aws eks update-kubeconfig --name cluster1 && \ kubectl config use-context arn:aws:eks:us-east-1:000000000000:cluster/cluster1 ``` ```bash title="Output" ... Added new context arn:aws:eks:us-east-1:000000000000:cluster/cluster1 to /home/localstack/.kube/config Switched to context "arn:aws:eks:us-east-1:000000000000:cluster/cluster1". ... ``` You can now go ahead and add a deployment configuration for the `fancier-nginx` image. ```bash cat <\.s3.*\.amazonaws\.com` in your [configuration](/aws/customization/configuration-options). ::: ### Configuring an Ingress for your services To make an EKS service externally accessible, it is necessary to create an Ingress configuration, which exposes the service on a specific path to the load balancer. For our sample deployment, we can create an `nginx` Kubernetes service by applying the following configuration: ```bash cat < ...
nginx/1.21.6
... ``` :::tip You can customize the Load Balancer port by configuring `EKS_LOADBALANCER_PORT` in your environment. ::: ### Enabling HTTPS with local SSL/TLS certificate for the Ingress To enable HTTPS for your endpoints, you can configure Kubernetes to use SSL/TLS with the [certificate for local domain names](https://github.com/localstack/localstack-artifacts/blob/master/local-certs/server.key) `*.localhost.localstack.cloud`. The local EKS cluster comes pre-configured with a secret named `ls-secret-tls`, which can be conveniently utilized to define the `tls` section in the ingress configuration: ```yaml showshowLineNumbers apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: test-ingress annotations: ingress.kubernetes.io/ssl-redirect: "false" traefik.ingress.kubernetes.io/router.entrypoints: web,websecure traefik.ingress.kubernetes.io/router.tls: "true" spec: tls: - secretName: ls-secret-tls hosts: - myservice.localhost.localstack.cloud ... ``` Once you have deployed your service using the mentioned ingress configuration, it will be accessible via the HTTPS endpoint `https://myservice.localhost.localstack.cloud`. Remember that the ingress controller does not support HTTP/HTTPS multiplexing within the same Ingress. Consequently, if you want your service to be accessible via HTTP and HTTPS, you must create two separate Ingress definitions — one Ingress for HTTP and another for HTTPS. :::note The `ls-secret-tls` secret is created in the `default` namespace. If your ingress and services are residing in a custom namespace, it is essential to copy the secret to that custom namespace to make use of it. ::: ## Self-managed nodes and Karpenter In addition to [managed node groups](#creating-a-managed-node-group), LocalStack supports self-managed worker nodes: EC2 instances that join an EKS cluster on their own using the standard EKS bootstrap user-data. This is the same mechanism that [Karpenter](https://karpenter.sh/) relies on to provision capacity, so you can run the full Karpenter node lifecycle (SSM AMI lookup, EC2 Fleet provisioning, node bootstrap, and scale-up/scale-down) against a local cluster without any LocalStack-specific configuration. When an EC2 instance is launched with EKS bootstrap user-data, LocalStack parses it, connects the instance container to the cluster's internal network, and starts a [k3s](https://k3s.io/) agent inside it. The instance then registers as a worker node, exactly as a real EKS node would appear to the control plane. Two AMI families are supported: - **Amazon Linux 2023 (AL2023)**: configuration is carried as a multipart MIME `application/node.eks.aws` NodeConfig document (used by `nodeadm` and Karpenter). - **Bottlerocket**: configuration is carried as a plain TOML document under `[settings.kubernetes]`. LocalStack reads the same set of fields from both formats and propagates them to the registered node, including node labels, taints, topology metadata (region, zone, instance type), and the provider ID. This matches what real EKS nodes look like from the cluster's perspective. :::note Self-managed nodes use the embedded k3d-backed provider (`MANAGED_K8S_PROVIDER=k3s`), which is the default. The walkthrough below runs entirely between Docker containers, so it also works on macOS where direct host-to-instance networking is not available. ::: ### Resolving a node AMI Self-managed nodes must be launched from an EKS-optimized AMI. LocalStack resolves the standard EKS AMI [SSM public parameters](https://docs.aws.amazon.com/eks/latest/userguide/retrieve-ami-id.html) to a k3s-backed image, so you can look them up the same way Karpenter does. ```bash title="Resolve AL2023 AMI" lstk aws ssm get-parameter \ --name /aws/service/eks/optimized-ami/1.35/amazon-linux-2023/x86_64/standard/recommended/image_id \ --query 'Parameter.Value' --output text ``` ```bash title="Output" ami-eks-k3d-1.35-amd64-standard ``` ```bash title="Resolve Bottlerocket AMI" lstk aws ssm get-parameter \ --name /aws/service/bottlerocket/aws-k8s-1.35/x86_64/latest/image_id \ --query 'Parameter.Value' --output text ``` ```bash title="Output" ami-eks-k3d-1.35-amd64 ``` The walkthrough below assumes a cluster named `cluster1` that is already `ACTIVE` (see [Create an embedded Kubernetes cluster](#create-an-embedded-kubernetes-cluster)). ### Joining a self-managed node 1. Create a user-data file containing a `NodeConfig` document. At minimum, `spec.cluster.name` must match the name of your EKS cluster, which LocalStack uses to resolve which cluster the node should join. Optionally, you can set kubelet configuration and node labels: ```text title="al2023-userdata.txt" MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="//" --// Content-Type: application/node.eks.aws apiVersion: node.eks.aws/v1alpha1 kind: NodeConfig spec: cluster: name: cluster1 kubelet: config: maxPods: 20 registerWithTaints: - key: dedicated value: gpu effect: NoSchedule flags: - '--node-labels=role=worker,env=demo' --//-- ``` 2. Launch an EC2 instance using the resolved AL2023 AMI and the user-data file: ```bash title="Launch AL2023 node" lstk aws ec2 run-instances \ --image-id ami-eks-k3d-1.35-amd64-standard \ --count 1 \ --instance-type t3.medium \ --user-data file://./al2023-userdata.txt ``` 1. Bottlerocket carries its bootstrap configuration as a plain TOML document. Create a user-data file with a `[settings.kubernetes]` section, setting `cluster-name` to your cluster: ```toml title="bottlerocket-userdata.toml" [settings.kubernetes] cluster-name = "cluster1" max-pods = 30 [settings.kubernetes.node-labels] "role" = "worker" "env" = "demo" [settings.kubernetes.node-taints] "dedicated" = ["gpu:NoSchedule"] ``` 2. Launch an EC2 instance using the resolved Bottlerocket AMI: ```bash title="Launch Bottlerocket node" lstk aws ec2 run-instances \ --image-id ami-eks-k3d-1.35-amd64 \ --count 1 \ --instance-type m5.large \ --user-data file://./bottlerocket-userdata.toml ``` ### Verifying the node Point `kubectl` at the cluster and list the nodes: ```bash lstk aws eks update-kubeconfig --name cluster1 && \ kubectl config use-context arn:aws:eks:us-east-1:000000000000:cluster/cluster1 ``` After a few seconds, the self-managed instance registers and becomes `Ready` alongside the cluster's control-plane node. The node name is the EC2 instance ID: ```bash kubectl get nodes ``` ```bash title="Output" NAME STATUS ROLES AGE VERSION i-8a2eb615ddf838df7 Ready 33s v1.35.5+k3s1 k3d-cluster1-c2fad0d2-server-0 Ready control-plane 2m v1.35.5+k3s1 ``` You can confirm that the configuration from the user-data was applied to the node: ```bash kubectl get node i-8a2eb615ddf838df7 \ -o jsonpath='{.spec.providerID}{"\n"}{.spec.taints}{"\n"}{.status.allocatable.pods}{"\n"}' ``` ```bash title="Output" aws:///us-east-1a/i-8a2eb615ddf838df7 [{"effect":"NoSchedule","key":"dedicated","value":"gpu"}] 20 ``` ### Supported configuration fields LocalStack reads the following fields from the node bootstrap user-data and reflects them on the registered Kubernetes node. The AL2023 and Bottlerocket columns show where each value comes from in the respective format: | Behaviour | AL2023 (`NodeConfig`) | Bottlerocket (TOML) | |:-----------------------------------|:---------------------------------------|:------------------------------------------| | Target cluster | `spec.cluster.name` | `cluster-name` | | Cluster DNS | `kubelet.config.clusterDNS` | `cluster-dns-ip` | | Max pods | `kubelet.config.maxPods` | `max-pods` | | Eviction thresholds | `kubelet.config.evictionHard` | `[settings.kubernetes.eviction-hard]` | | Node taints | `kubelet.config.registerWithTaints` | `[settings.kubernetes.node-taints]` | | Node labels | `kubelet.flags` (`--node-labels`) | `[settings.kubernetes.node-labels]` | In addition, LocalStack always emits the topology labels `topology.kubernetes.io/region`, `topology.kubernetes.io/zone`, and `node.kubernetes.io/instance-type` (derived from the instance's placement and type), as well as the provider ID in the form `aws:////`. :::note Fields such as `apiServerEndpoint` and `certificateAuthority` are required when bootstrapping against real EKS, but are resolved internally by LocalStack and can be omitted from the user-data. ::: ### Using Karpenter Because self-managed nodes rely only on standard EC2 and EKS bootstrap behaviour, [Karpenter](https://karpenter.sh/) can drive node provisioning against a local cluster without any LocalStack-specific changes. Karpenter looks up node AMIs through the SSM parameters described above, launches instances via EC2 Fleet with the appropriate NodeConfig or Bottlerocket user-data, and the new instances join the cluster as described. To try this out, follow the upstream [Getting started with Karpenter](https://karpenter.sh/docs/getting-started/getting-started-with-karpenter/) guide, pointing the AWS endpoints at LocalStack. ## Use an existing Kubernetes installation You can also access the EKS API using your existing local Kubernetes installation. This can be achieved by setting the configuration variable `MANAGED_K8S_PROVIDER=local` and mounting the `$HOME/.kube/config` file into the LocalStack container. When using a `docker-compose.yml` file, you need to add a bind mount like this: ```yaml volumes: - "${HOME}/.kube/config:/root/.kube/config" ``` When using `lstk`, add the mount and provider variable to your `config.toml`: ```toml [[containers]] type = "aws" volumes = ["~/.kube/config:/root/.kube/config"] env = ["k8s-provider"] [env.k8s-provider] MANAGED_K8S_PROVIDER = "local" ``` Then start LocalStack: ```bash lstk start ``` :::note Using an existing Kubernetes installation is currently only possible when the authentication with the cluster uses X509 client certificates: https://kubernetes.io/docs/reference/access-authn-authz/authentication/#x509-client-certificates ::: In recent versions of Docker, you can enable Kubernetes as an embedded service running inside Docker. The picture below illustrates the Kubernetes settings in Docker for macOS (similar configurations apply for Linux/Windows). By default, the Kubernetes API is assumed to run on the local TCP port `6443`. ![Kubernetes in Docker](/images/aws/kubernetes.png) You can create an EKS Cluster configuration using the following command: ```bash lstk aws eks create-cluster --name cluster1 --role-arn arn:aws:iam::000000000000:role/eks-role --resources-vpc-config '{}' ``` ```bash title="Output" { "cluster": { "name": "cluster1", "arn": "arn:aws:eks:eu-central-1:000000000000:cluster/cluster1", "createdAt": "Sat, 05 Oct 2019 12:29:26 GMT", "endpoint": "https://172.17.0.1:6443", "status": "ACTIVE", ... } } ``` And check that it was created with: ```bash lstk aws eks list-clusters ``` ```bash title="Output" { "clusters": [ "cluster1" ] } ``` To interact with your Kubernetes cluster, configure your Kubernetes client (such as `kubectl` or other SDKs) to point to the `endpoint` provided in the `create-cluster` output mentioned earlier. However, depending on whether you're calling the Kubernetes API from your local machine or from within a Lambda function, you might need to use different endpoint URLs. For local machine interactions, use `https://localhost:6443` as the endpoint URL. If you are accessing the Kubernetes API from within a Lambda function, you should use `https://172.17.0.1:6443` as the endpoint URL, assuming that `172.17.0.1` is the IP address of the Docker network bridge. By using the appropriate endpoint URL based on your context, you can effectively communicate with your Kubernetes cluster and manage your resources as needed. ## Customizing the Kubernetes Load Balancer Ports By default, the Kubernetes load balancer (LB) is exposed on port `8081`. If you need to customize the port or expose the load balancer on multiple ports, you can utilize the special tag name `_lb_ports_` during the cluster creation process. For instance, if you want to expose the load balancer on ports 8085 and 8086, you can use the following tag definition when creating the cluster: ```bash lstk aws eks create-cluster \ --name cluster1 \ --role-arn arn:aws:iam::000000000000:role/eks-role \ --resources-vpc-config '{}' --tags '{"_lb_ports_":"8085,8086"}' ``` ## Routing Traffic to Services on Different Endpoints When working with EKS, a common scenario is to access multiple Kubernetes services behind different endpoints. For instance, you might have multiple microservices, each following a common path versioning scheme, such as API request paths starting with `/v1/...`. In such cases, path-based routing may not be ideal if you need the services to be accessible in a uniform manner. To address this requirement, we recommend utilizing host-based routing rules, as demonstrated in the example below: ```bash showshowLineNumbers cat < # ElastiCache > Get started with Amazon ElastiCache on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Amazon ElastiCache is a managed in-memory caching service provided by Amazon Web Services (AWS). It facilitates the deployment and operation of in-memory caches within the AWS cloud environment. ElastiCache is designed to improve application performance and scalability by alleviating the workload on backend databases. Amazon ElastiCache supports popular open-source caching engines like Redis, Valkey, and Memcached. LocalStack currently supports Redis and Valkey, enabling developers to simulate ElastiCache behavior locally for efficient, low-latency data caching. LocalStack supports ElastiCache via the Pro offering, allowing you to use the ElastiCache APIs in your local environment. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of ElastiCache integration with LocalStack. ## Getting started This guide is designed for users new to ElastiCache and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. ### Single cache cluster After starting LocalStack for AWS, you can create a cluster with the following command. ```bash lstk aws elasticache create-cache-cluster \ --cache-cluster-id my-redis-cluster \ --cache-node-type cache.t2.micro \ --engine redis \ --num-cache-nodes 1 ``` Wait for it to be available, then you can use the cluster endpoint for Redis operations. ```bash lstk aws elasticache describe-cache-clusters --show-cache-node-info --query "CacheClusters[0].CacheNodes[0].Endpoint" ``` ```bash title="Output" { "Address": "localhost.localstack.cloud", "Port": 4510 } ``` The cache cluster uses a random port of the [external service port range](/aws/customization/networking/external-port-range/). Use this port number to connect to the Redis instance like so: ```bash redis-cli -p 4510 ping PONG redis-cli -p 4510 set foo bar OK redis-cli -p 4510 get foo "bar" ``` ### Replication groups in non-cluster mode ```bash lstk aws elasticache create-replication-group \ --replication-group-id my-redis-replication-group \ --replication-group-description 'my replication group' \ --engine redis \ --cache-node-type cache.t2.micro \ --num-cache-clusters 3 ``` Wait for it to be available. When running the following command, you should see one node group when running: ```bash lstk aws elasticache describe-replication-groups --replication-group-id my-redis-replication-group ``` To retrieve the primary endpoint: ```bash lstk aws elasticache describe-replication-groups --replication-group-id my-redis-replication-group \ --query "ReplicationGroups[0].NodeGroups[0].PrimaryEndpoint" ``` ### Replication groups in cluster mode The cluster mode is enabled by using `--num-node-groups` and `--replicas-per-node-group`: ```bash lstk aws elasticache create-replication-group \ --engine redis \ --replication-group-id my-clustered-redis-replication-group \ --replication-group-description 'my clustered replication group' \ --cache-node-type cache.t2.micro \ --num-node-groups 2 \ --replicas-per-node-group 2 ``` Note that the group nodes do not have a primary endpoint. Instead they have a `ConfigurationEndpoint`, which you can connect to using `redis-cli -c` where `-c` is for cluster mode. ```bash lstk aws elasticache describe-replication-groups --replication-group-id my-clustered-redis-replication-group \ --query "ReplicationGroups[0].ConfigurationEndpoint" ``` ## Container mode In order to start Redis clusters of a specific version, you need to use the container mode for Redis-based services. This instructs LocalStack to start Redis instances in a separate container using the specified image tag. Another reason you might want to use the container mode is to check the logs of every Redis instance separately. To do this, you can set the `REDIS_CONTAINER_MODE` configuration variable to `1`. ## Valkey Engine LocalStack offers the additional option to use Valkey as an alternative to Redis in Amazon ElastiCache. To enable full Valkey emulation: 1. Start LocalStack with Valkey support enabled by setting the environment variable, `REDIS_CONTAINER_MODE=1` 2. Create a cluster with the Valkey engine by including the `--engine valkey` flag in your API call: ```bash lstk aws elasticache create-replication-group \ --replication-group-id my-valkey-group \ --replication-group-description "Valkey test group" \ --engine valkey \ --cache-node-type cache.t4g.small \ --num-node-groups 1 \ --replicas-per-node-group 1 \ --automatic-failover-enabled ``` Valkey support includes: - The ability to specify `valkey` as the engine when creating Amazon ElastiCache replication groups. - Automatic mapping of each engine to a default supported version (Redis `7.2.10`, Valkey `7.2.10`), ensuring DockerHub compatibility. - Support for the `Engine` and `EngineVersion` fields in the `CreateReplicationGroup` API, which now recognize and handle `valkey`. :::note A Valkey replication group can only be started when [container mode](/aws/services/elasticache/#container-mode) is enabled. ::: ## Resource browser The LocalStack Web Application provides a Resource Browser for managing ElastiCache resources. You can access the Resource Browser by opening the LocalStack Web Application in your browser and navigating to the Resources section, then clicking on ElastiCache. In the ElastiCache resource browser you can: * List and remove existing cache clusters ![List existing cache clusters](/images/aws/elasticache-resource-browser-list.png) * View details of cache clusters ![View details of cache clusters](/images/aws/elasticache-resource-browser-show.png) * Create new cache clusters ![Create a ElastiCache cluster in the resource browser](/images/aws/elasticache-resource-browser-create.png) ## Current Limitations LocalStack currently supports Redis single-node and cluster mode, but not memcached. Moreover, LocalStack emulation support for ElastiCache is mostly centered around starting/stopping Redis servers. Resources necessary to operate a cluster, like parameter groups, security groups, subnets groups, etc. are mocked, but have no effect on the functioning of the Redis servers. LocalStack currently doesn't support ElastiCache snapshots, failovers, users/passwords, service updates, replication scaling, SSL, migrations, service integration (like CloudWatch/Kinesis log delivery, SNS notifications) or tests. ## API Coverage # Elastic Beanstalk > Get started with Elastic Beanstalk (EB) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Elastic Beanstalk (EB) is a managed platform-as-a-service (PaaS) provided by Amazon Web Services (AWS) that simplifies the process of deploying, managing, and scaling web applications and services. Elastic Beanstalk orchestrates various AWS services, including EC2, S3, SNS, and Elastic Load Balancers. Elastic Beanstalk also supports various application environments, such as Java, .NET, Node.js, PHP, Python, Ruby, Go, and Docker. LocalStack allows you to use the Elastic Beanstalk APIs in your local environment to create and manage applications, environments and versions. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Elastic Beanstalk's integration with LocalStack. ## Getting started This guide is designed for users new to Elastic Beanstalk and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create an Elastic Beanstalk application and environment with the AWS CLI. ### Create an application To create an Elastic Beanstalk application, you can use the [`CreateApplication`](https://docs.aws.amazon.com/elasticbeanstalk/latest/api/API_CreateApplication.html) API. Run the following command to create an application named `my-app`: ```bash lstk aws elasticbeanstalk create-application \ --application-name my-app ``` ```bash title="Output" { "Application": { "ApplicationArn": "arn:aws:elasticbeanstalk:us-east-1:000000000000:application/my-app", "ApplicationName": "my-app", "DateCreated": "2023-08-24T05:55:57.603443Z" } } ``` You can also use the [`DescribeApplications`](https://docs.aws.amazon.com/elasticbeanstalk/latest/api/API_DescribeApplications.html) API to retrieve information about your application. Run the following command to retrieve information about the `my-app` application, we created earlier: ```bash lstk aws elasticbeanstalk describe-applications \ --application-names my-app ``` ### Create an environment To create an Elastic Beanstalk environment, you can use the [`CreateEnvironment`](https://docs.aws.amazon.com/elasticbeanstalk/latest/api/API_CreateEnvironment.html) API. Run the following command to create an environment named `my-environment`: ```bash lstk aws elasticbeanstalk create-environment \ --application-name my-app \ --environment-name my-environment ``` ```bash title="Output" { "EnvironmentName": "my-environment", "EnvironmentId": "4fcae3fb", "ApplicationName": "my-app", "DateCreated": "2023-08-24T05:57:59.889966Z", "EnvironmentArn": "arn:aws:elasticbeanstalk:us-east-1:000000000000:applicationversion/my-app/version" } ``` You can also use the [`DescribeEnvironments`](https://docs.aws.amazon.com/elasticbeanstalk/latest/api/API_DescribeEnvironments.html) API to retrieve information about your environment. Run the following command to retrieve information about the `my-environment` environment, we created earlier: ```bash lstk aws elasticbeanstalk describe-environments \ --environment-names my-environment ``` ### Create an application version To create an Elastic Beanstalk application version, you can use the [`CreateApplicationVersion`](https://docs.aws.amazon.com/elasticbeanstalk/latest/api/API_CreateApplicationVersion.html) API. Run the following command to create an application version named `v1`: ```bash lstk aws elasticbeanstalk create-application-version \ --application-name my-app \ --version-label v1 ``` ```bash title="Output" { "ApplicationVersion": { "ApplicationVersionArn": "arn:aws:elasticbeanstalk:us-east-1:000000000000:applicationversion/my-app/v1", "ApplicationName": "my-app", "VersionLabel": "v1", "DateCreated": "2023-08-24T05:59:58.166021Z" } } ``` You can also use the [`DescribeApplicationVersions`](https://docs.aws.amazon.com/elasticbeanstalk/latest/api/API_DescribeApplicationVersions.html) API to retrieve information about your application version. Run the following command to retrieve information about the `v1` application version, we created earlier: ```bash lstk aws elasticbeanstalk describe-application-versions \ --application-name my-app ``` ## Current Limitations LocalStack's Elastic Beanstalk implementation is limited and lacks support for installing application and running it in a local Elastic Beanstalk environment. LocalStack also does not support the [`eb`](https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb-cli3.html) CLI tool. However, you can use other integrations, such as AWS CLI & Terraform, to mock the Elastic Beanstalk APIs and test your workflow locally. ## API Coverage # Elastic Load Balancing (ELB) > Get started with Elastic Load Balancing (ELB) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Elastic Load Balancing (ELB) is a service that allows users to distribute incoming traffic across multiple targets, such as EC2 instances, containers, IP addresses, and lambda functions and automatically scales its request handling capacity in response to incoming traffic. It also monitors the health of its registered targets and ensures that it routes traffic only to healthy targets. You can check [the official AWS documentation](https://docs.aws.amazon.com/elasticloadbalancing/latest/userguide/what-is-load-balancing.html) to understand the basic terms and concepts used in the ELB. Localstack allows you to use the Elastic Load Balancing APIs in your local environment to create, edit, and view load balancers, target groups, listeners, and rules. The supported APIs are available on the API coverage section for [ELBv1](#api-coverage-elbv1) and [ELBv2](#api-coverage-elbv2), which provides information on the extent of ELB's integration with LocalStack. ## Getting started This guide is designed for users new to Elastic Load Balancing and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create an Application Load Balancer, along with its target group, listener, and rule, and forward requests to an IP target. ### Start a target server Launch an HTTP server which will serve as the target for our load balancer. ```bash docker run --rm -itd -p 5678:80 ealen/echo-server ``` ### Create a load balancer To specify the subnet and VPC in which the load balancer will be created, you can use the [`DescribeSubnets`](https://docs.aws.amazon.com/elasticloadbalancing/latest/APIReference/API_DescribeSubnets.html) API to retrieve the subnet ID and VPC ID. In this example, we will use the subnet and VPC in the `us-east-1f` availability zone. ```bash subnet_info=$(lstk aws ec2 describe-subnets --filters Name=availability-zone,Values=us-east-1f \ | jq -r '.Subnets[] | select(.AvailabilityZone == "us-east-1f") | {SubnetId: .SubnetId, VpcId: .VpcId}') subnet_id=$(echo $subnet_info | jq -r '.SubnetId') vpc_id=$(echo $subnet_info | jq -r '.VpcId') ``` To create a load balancer, you can use the [`CreateLoadBalancer`](https://docs.aws.amazon.com/elasticloadbalancing/latest/APIReference/API_CreateLoadBalancer.html) API. The following command creates an Application Load Balancer named `example-lb`: ```bash loadBalancer=$(lstk aws elbv2 create-load-balancer --name example-lb \ --subnets $subnet_id | jq -r '.LoadBalancers[]|.LoadBalancerArn') ``` ### Create a target group To create a target group, you can use the [`CreateTargetGroup`](https://docs.aws.amazon.com/elasticloadbalancing/latest/APIReference/API_CreateTargetGroup.html) API. The following command creates a target group named `example-target-group`: ```bash targetGroup=$(lstk aws elbv2 create-target-group --name example-target-group \ --protocol HTTP --target-type ip --port 80 --vpc-id $vpc_id \ | jq -r '.TargetGroups[].TargetGroupArn') ``` ### Register a target To register a target, you can use the [`RegisterTargets`](https://docs.aws.amazon.com/elasticloadbalancing/latest/APIReference/API_RegisterTargets.html) API. The following command registers the target with the target group created in the previous step: ```bash lstk aws elbv2 register-targets --targets Id=127.0.0.1,Port=5678,AvailabilityZone=all \ --target-group-arn $targetGroup ``` :::note Note that in some cases the `targets` parameter `Id` can be the `Gateway` address of the docker container. You can find the gateway address by running `docker inspect `. ::: ### Create a listener and a rule We create a listener for the load balancer using the [`CreateListener`](https://docs.aws.amazon.com/elasticloadbalancing/latest/APIReference/API_CreateListener.html) API. The following command creates a listener for the load balancer created in the previous step: ```bash listenerArn=$(lstk aws elbv2 create-listener \ --protocol HTTP \ --port 80 \ --default-actions '{"Type":"forward","TargetGroupArn":"'$targetGroup'","ForwardConfig":{"TargetGroups":[{"TargetGroupArn":"'$targetGroup'","Weight":11}]}}' \ --load-balancer-arn $loadBalancer | jq -r '.Listeners[]|.ListenerArn') ``` To create a rule for the listener, you can use the [`CreateRule`](https://docs.aws.amazon.com/elasticloadbalancing/latest/APIReference/API_CreateRule.html) API. The following command creates a rule for the listener created above: ```bash listenerRule=$(lstk aws elbv2 create-rule \ --conditions Field=path-pattern,Values=/ \ --priority 1 \ --actions '{"Type":"forward","TargetGroupArn":"'$targetGroup'","ForwardConfig":{"TargetGroups":[{"TargetGroupArn":"'$targetGroup'","Weight":11}]}}' \ --listener-arn $listenerArn \ | jq -r '.Rules[].RuleArn') ``` ### Send a request to the load balancer Finally, you can issue an HTTP request to the `DNSName` parameter of `CreateLoadBalancer` operation, and `Port` parameter of `CreateListener` command with the following command: ```bash curl example-lb.elb.localhost.localstack.cloud:4566 ``` ```bash title="Output" { "host": { "hostname": "example-lb.elb.localhost.localstack.cloud", "ip": "::ffff:172.17.0.1", "ips": [] }, "http": { "method": "GET", "baseUrl": "", "originalUrl": "/", "protocol": "http" }, "request": { "params": { "0": "/" }, "query": {}, "cookies": {}, "body": {}, "headers": { "accept-encoding": "identity", "host": "example-lb.elb.localhost.localstack.cloud:4566", "user-agent": "curl/7.88.1", "accept": "*/*" } }, "environment": { "PATH": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "HOSTNAME": "bee08b83d633", "TERM": "xterm", "NODE_VERSION": "18.17.1", "YARN_VERSION": "1.22.19", "HOME": "/root" } } ``` #### Alternative URL structure If a request cannot be made to a subdomain of `localhost.localstack.cloud`, an alternative URL structure is available, however it is not returned by AWS management API methods. To make a request against an ELB with id ``, use the URL: ```bash http(s)://localhost.localstack.cloud:4566/_aws/elb// ``` Here's an example of how you would access the load balancer with a name of `example-lb` with the subdomain-based URL format: ```bash http(s)://example-lb.elb.localhost.localstack.cloud:4566/test/path ``` With the alternative URL structure: ```bash http(s)://localhost.localstack.cloud:4566/_aws/elb/example-lb/test/path ``` ## Multiple Listeners and Port-Based Routing An Application Load Balancer can have multiple listeners, each bound to a different port. In LocalStack, same-scheme listeners (for example, two HTTP listeners on ports 80 and 8080) are routed by matching the request's arrival port to the listener's configured port. ### Configuring LocalStack for multiple listener ports To reach two same-scheme listeners on distinct ports, both ports must be published in [`GATEWAY_LISTEN`](/aws/customization/configuration-options/#core) when starting LocalStack: ```bash GATEWAY_LISTEN=0.0.0.0:4566,0.0.0.0:80,0.0.0.0:8080 localstack start ``` ### Creating multiple listeners With LocalStack running and both ports published, create a load balancer and two HTTP listeners on different ports. The following example uses the `subnet_id` variable set in the [Getting started](#getting-started) steps above, and creates two listeners with distinct `fixed-response` default actions so you can verify that each port routes to the correct listener: ```bash # Create the load balancer loadBalancer=$(lstk aws elbv2 create-load-balancer \ --name multi-listener-lb \ --subnets $subnet_id | jq -r '.LoadBalancers[].LoadBalancerArn') # Listener on port 80 lstk aws elbv2 create-listener \ --load-balancer-arn $loadBalancer \ --protocol HTTP \ --port 80 \ --default-actions '{"Type":"fixed-response","FixedResponseConfig":{"StatusCode":"200","MessageBody":"Listener 80","ContentType":"text/plain"}}' # Listener on port 8080 lstk aws elbv2 create-listener \ --load-balancer-arn $loadBalancer \ --protocol HTTP \ --port 8080 \ --default-actions '{"Type":"fixed-response","FixedResponseConfig":{"StatusCode":"200","MessageBody":"Listener 8080","ContentType":"text/plain"}}' ``` A request to port 80 is handled by the listener bound to that port: ```bash curl multi-listener-lb.elb.localhost.localstack.cloud:80 ``` ```bash title="Output" Listener 80 ``` A request to port 8080 is handled by the listener bound to that port: ```bash curl multi-listener-lb.elb.localhost.localstack.cloud:8080 ``` ```bash title="Output" Listener 8080 ``` ### By-design limitation: shared gateway port When a request arrives on the shared `:4566` gateway port, LocalStack cannot determine which same-scheme listener was intended and falls back to the first-created listener: ```bash curl multi-listener-lb.elb.localhost.localstack.cloud:4566 ``` ```bash title="Output" Listener 80 ``` :::note If your setup uses only the default `:4566` gateway port and you need multiple listeners, consolidate to a single listener per scheme. Same-scheme listeners on distinct ports can only be told apart when those ports are added to `GATEWAY_LISTEN` and targeted directly. ::: ## Examples The following code snippets and sample applications provide practical examples of how to use ELB in LocalStack for various use cases: - [Setting up Elastic Load Balancing (ELB) Application Load Balancers using LocalStack, deployed via the Serverless framework](/aws/tutorials/elb-load-balancing/) ## Current Limitations - The Application Load Balancer currently supports only the `forward`, `redirect` and `fixed-response` action types. - When opting for Route53 CNAMEs to direct requests towards the ALBs, it's important to remember that explicit configuration of the `Host` header to match the resource record might be necessary while making calls. ## API Coverage (ELBv1) ## API Coverage (ELBv2) # Elastic MapReduce (EMR) > Get started with Elastic MapReduce (EMR) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Amazon Elastic MapReduce (EMR) is a fully managed big data processing service that allows developers to effortlessly create, deploy, and manage big data applications. EMR supports various big data processing frameworks, including Hadoop MapReduce, Apache Spark, Apache Hive, and Apache Pig. Developers can leverage these frameworks and their rich ecosystem of tools and libraries to perform complex data transformations, machine learning tasks, and real-time data processing. LocalStack supports EMR and allows developers to run data analytics workloads locally. EMR utilizes various tools in the [Hadoop](https://hadoop.apache.org/) and [Spark](https://spark.apache.org) ecosystem, and your EMR instance is automatically configured to connect seamlessly to LocalStack's S3 API. LocalStack also supports EMR Serverless to create applications and job runs, to run your Spark/PySpark jobs locally. The supported APIs are available on the API coverage section for [EMR](#api-coverage) and [EMR Serverless](#api-coverage-emr-serverless), which provides information on the extent of EMR's integration with LocalStack. :::note To utilize the EMR API, certain additional dependencies need to be downloaded from the network (including Hadoop, Hive, Spark, etc). These dependencies are fetched automatically during service startup, hence it is important to ensure a reliable internet connection when retrieving the dependencies for the first time. Alternatively, you can use one of our `*-bigdata` Docker image tags which already ship with the required libraries baked in and may provide better stability (see [here](/aws/customization/other-installations/docker-images/) for more details). ::: ## Getting started This guide is designed for users new to EMR and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will create a virtual EMR cluster using the AWS CLI. To create an EMR cluster, run the following command: ```bash lstk aws emr create-cluster \ --release-label emr-5.9.0 \ --instance-groups InstanceGroupType=MASTER,InstanceCount=1,InstanceType=m4.large InstanceGroupType=CORE,InstanceCount=1,InstanceType=m4.large ``` You will see a response similar to the following: ```bash title="Output" { "ClusterId": "j-A2KF3EKLAOWRI" } ``` You can also specify startup commands using the `--steps=...` command line argument to the `CreateCluster` API. ## Examples The following code snippets and sample applications provide practical examples of how to use EMR in LocalStack for various use cases: - [Running data analytics jobs using EMR](https://github.com/localstack/localstack-pro-samples/tree/master/sample-archive/emr-hadoop-spark-jobs) - [Running EMR Serverless Jobs with Java](https://github.com/localstack/localstack-pro-samples/tree/master/emr-serverless-sample) ## API Coverage ## API Coverage (EMR Serverless) # Elasticsearch Service > Get started with Amazon Elasticsearch Service (ES) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction The Elasticsearch Service in LocalStack lets you create one or more single-node Elasticsearch/OpenSearch cluster that behaves like the [Amazon Elasticsearch Service](https://aws.amazon.com/opensearch-service/the-elk-stack/what-is-elasticsearch/). This service is, like its AWS counterpart, heavily linked with the [OpenSearch Service](/aws/services/opensearch/). Any cluster created with the Elasticsearch Service will show up in the OpenSearch Service and vice versa. ## Creating an Elasticsearch cluster You can go ahead and use [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) to create a new elasticsearch domain via the `lstk aws es create-elasticsearch-domain` command. :::note Unless you use the Elasticsearch default version, the first time you create a cluster with a specific version, the Elasticsearch binary is downloaded, which may take a while to download. ::: ```bash lstk aws es create-elasticsearch-domain --domain-name my-domain ``` ```bash title="Output" { "DomainStatus": { "DomainId": "000000000000/my-domain", "DomainName": "my-domain", "ARN": "arn:aws:es:us-east-1:000000000000:domain/my-domain", "Created": true, "Deleted": false, "Endpoint": "my-domain.us-east-1.es.localhost.localstack.cloud:4566", "Processing": true, "ElasticsearchVersion": "7.10.0", "ElasticsearchClusterConfig": { "InstanceType": "m3.medium.elasticsearch", "InstanceCount": 1, "DedicatedMasterEnabled": true, "ZoneAwarenessEnabled": false, "DedicatedMasterType": "m3.medium.elasticsearch", "DedicatedMasterCount": 1 }, "EBSOptions": { "EBSEnabled": true, "VolumeType": "gp2", "VolumeSize": 10, "Iops": 0 }, "CognitoOptions": { "Enabled": false } } } ``` In the LocalStack log you will see something like the following, where you can see the cluster starting up in the background. ```bash 2021-11-08T16:29:28:INFO:localstack.services.es.cluster: starting elasticsearch: /opt/code/localstack/localstack/localstack/infra/elasticsearch/bin/elasticsearch -E http.port=57705 -E http.publish_port=57705 -E transport.port=0 -E network.host=127.0.0.1 -E http.compression=false -E path.data="/var/lib/localstack/lib//elasticsearch/arn:aws:es:us-east-1:000000000000:domain/my-domain/data" -E path.repo="/var/lib/localstack/lib//elasticsearch/arn:aws:es:us-east-1:000000000000:domain/my-domain/backup" -E xpack.ml.enabled=false with env {'ES_JAVA_OPTS': '-Xms200m -Xmx600m', 'ES_TMPDIR': '/var/lib/localstack/lib//elasticsearch/arn:aws:es:us-east-1:000000000000:domain/my-domain/tmp'} 2021-11-08T16:29:28:INFO:localstack.services.es.cluster: registering an endpoint proxy for http://my-domain.us-east-1.es.localhost.localstack.cloud:4566 => http://127.0.0.1:57705 2021-11-08T16:29:30:INFO:localstack.services.es.cluster: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. 2021-11-08T16:29:32:INFO:localstack.services.es.cluster: [2021-11-08T16:29:32,502][INFO ][o.e.n.Node ] [noctua] version[7.10.0], pid[22403], build[default/tar/51e9d6f22758d0374a0f3f5c6e8f3a7997850f96/2020-11-09T21:30:33.964949Z], OS[Linux/5.4.0-89-generic/amd64], JVM[Ubuntu/OpenJDK 64-Bit Server VM/11.0.11/11.0.11+9-Ubuntu-0ubuntu2.20.04] 2021-11-08T16:29:32:INFO:localstack.services.es.cluster: [2021-11-08T16:29:32,510][INFO ][o.e.n.Node ] [noctua] JVM home [/usr/lib/jvm/java-11-openjdk-amd64], using bundled JDK [false] 2021-11-08T16:29:32:INFO:localstack.services.es.cluster: [2021-11-08T16:29:32,511][INFO ][o.e.n.Node ] [noctua] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Djava.io.tmpdir=/var/lib/localstack/lib//elasticsearch/arn:aws:es:us-east-1:000000000000:domain/my-domain/tmp, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Xms200m, -Xmx600m, -XX:MaxDirectMemorySize=314572800, -Des.path.home=/opt/code/localstack/localstack/localstack/infra/elasticsearch, -Des.path.conf=/opt/code/localstack/localstack/localstack/infra/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=tar, -Des.bundled_jdk=true] 2021-11-08T16:29:36:INFO:localstack.services.es.cluster: [2021-11-08T16:29:36,258][INFO ][o.e.p.PluginsService ] [noctua] loaded module [aggs-matrix-stats] 2021-11-08T16:29:36:INFO:localstack.services.es.cluster: [2021-11-08T16:29:36,259][INFO ][o.e.p.PluginsService ] [noctua] loaded module [analysis-common] 2021-11-08T16:29:36:INFO:localstack.services.es.cluster: [2021-11-08T16:29:36,260][INFO ][o.e.p.PluginsService ] [noctua] loaded module [constant-keyword] ... ``` and after some time, you should see that the `Processing` state of the domain is set to `false`: ```bash lstk aws es describe-elasticsearch-domain --domain-name my-domain | jq ".DomainStatus.Processing" ``` ```bash title="Output" false ``` ## Interact with the cluster You can now interact with the cluster at the cluster API endpoint for the domain, in this case `http://my-domain.us-east-1.es.localhost.localstack.cloud:4566`. For example: ```bash curl http://my-domain.us-east-1.es.localhost.localstack.cloud:4566 ``` ```bash title="Output" { "name" : "localstack", "cluster_name" : "elasticsearch", "cluster_uuid" : "IC7E9daNSiepRBB9Ksul7w", "version" : { "number" : "7.10.0", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "51e9d6f22758d0374a0f3f5c6e8f3a7997850f96", "build_date" : "2020-11-09T21:30:33.964949Z", "build_snapshot" : false, "lucene_version" : "8.7.0", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" } ``` Or the health endpoint: ```bash curl -s http://my-domain.us-east-1.es.localhost.localstack.cloud:4566/_cluster/health | jq . ``` ```bash title="Output" { "cluster_name": "elasticsearch", "status": "green", "timed_out": false, "number_of_nodes": 1, "number_of_data_nodes": 1, "active_primary_shards": 0, "active_shards": 0, "relocating_shards": 0, "initializing_shards": 0, "unassigned_shards": 0, "delayed_unassigned_shards": 0, "number_of_pending_tasks": 0, "number_of_in_flight_fetch": 0, "task_max_waiting_in_queue_millis": 0, "active_shards_percent_as_number": 100 } ``` ## Advanced topics ### Endpoints There are three configurable strategies that govern how domain endpoints are created, and can be configured via the `OPENSEARCH_ENDPOINT_STRATEGY` (previously `ES_ENDPOINT_STRATEGY`) environment variable. | Value | Format | Description | | - | - | - | | `domain` | `..es.localhost.localstack.cloud:4566` | This is the default strategy that uses the `localhost.localstack.cloud` domain to route to your localhost | | `path` | `localhost:4566/es//` | An alternative that can be useful if you cannot resolve LocalStack's localhost domain | | `port` | `localhost:` | Exposes the cluster(s) directly with ports from the [external service port range](/aws/customization/networking/external-port-range/)| | `off` | | *Deprecated*. This value now reverts to the `port` setting, using a port from the given range instead of `4571` | Regardless of the service from which the clusters were created, the domain of the cluster always corresponds to the engine type (OpenSearch or Elasticsearch) of the cluster. OpenSearch cluster therefore have `opensearch` in their domain (e.g. `my-domain.us-east-1.opensearch.localhost.localstack.cloud:4566`) and Elasticsearch clusters have `es` in their domain (e.g. `my-domain.us-east-1.es.localhost.localstack.cloud:4566`) #### Custom Endpoints LocalStack allows you to set arbitrary custom endpoints for your clusters in the domain endpoint options. This can be used to overwrite the behavior of the endpoint strategies described above. You can also choose custom domains, however it is important to add the edge port (`80`/`443` or by default `4566`). ```bash lstk aws es create-elasticsearch-domain --domain-name my-domain \ --elasticsearch-version 7.10 \ --domain-endpoint-options '{ "CustomEndpoint": "http://localhost:4566/my-custom-endpoint", "CustomEndpointEnabled": true }' ``` Once the domain processing is complete, you can access the cluster: ```bash title="Output" curl http://localhost:4566/my-custom-endpoint/_cluster/health ``` ### Re-using a single cluster instance In some cases, you may not want to create a new cluster instance for each domain, for example when you are only interested in testing API interactions instead of actual Elasticsearch functionality. In this case, you can set `OPENSEARCH_MULTI_CLUSTER=0` (previously `ES_MULTI_CLUSTER`). This will multiplex all domains to the same cluster, or return the same port every time when using the `port` endpoint strategy. This can however lead to unexpected behavior when persisting data into Elasticsearch, or creating clusters with different versions, so we do not recommend it. ### Storage Layout Elasticsearch will be organized in your state directory as follows: ```plaintext localstack@machine % tree -L 4 volume/state . ├── elasticsearch │ └── arn:aws:es:us-east-1:000000000000:domain │ ├── my-cluster-1 │ │ ├── backup │ │ ├── data │ │ └── tmp │ ├── my-cluster-2 │ │ ├── backup │ │ ├── data │ │ └── tmp ``` ### Advanced Security Options Since LocalStack 1.4.0, the OpenSearch and ElasticSearch services support "Advanced Security Options". This feature is currently only supported for OpenSearch domains (which can also be created by the elasticsearch service). More info can be found on [the OpenSearch Service docs page](/aws/services/opensearch/#advanced-security-options). ## Custom Elasticsearch backends LocalStack downloads elasticsearch asynchronously the first time you run the `aws es create-elasticsearch-domain`, so you will get the response from localstack first and then (after download/install) you will have your elasticsearch cluster running locally. You may not want this, and instead use your already running elasticsearch cluster. This can also be useful when you want to run a cluster with a custom configuration that localstack does not support. To customize the elasticsearch backend, you can your own elasticsearch cluster locally and point localstack to it using the `OPENSEARCH_CUSTOM_BACKEND` (previously `ES_CUSTOM_BACKEND`) environment variable. Note that only a single backend can be configured, meaning that you will get a similar behavior as when you [re-use a single cluster instance](#re-using-a-single-cluster-instance). ### Example The following shows a sample docker-compose file that contains a single-noded elasticsearch cluster and a basic localstack setp. ```yaml showshowLineNumbers services: elasticsearch: container_name: elasticsearch image: docker.elastic.co/elasticsearch/elasticsearch:7.10.2 environment: - node.name=elasticsearch - cluster.name=es-docker-cluster - discovery.type=single-node - bootstrap.memory_lock=true - "ES_JAVA_OPTS=-Xms512m -Xmx512m" ports: - "9200:9200" ulimits: memlock: soft: -1 hard: -1 volumes: - data01:/usr/share/elasticsearch/data localstack: container_name: "${LOCALSTACK_DOCKER_NAME:-localstack-main}" image: localstack/localstack ports: - "4566:4566" depends_on: - elasticsearch environment: - ES_CUSTOM_BACKEND=http://elasticsearch:9200 - DEBUG=${DEBUG:-0} volumes: - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" volumes: data01: driver: local ``` 1. Run docker compose: ```bash docker-compose up -d ``` 2. Create the Elasticsearch domain: ```bash lstk aws es create-elasticsearch-domain \ --domain-name mylogs-2 \ --elasticsearch-version 7.10 \ --elasticsearch-cluster-config '{ "InstanceType": "m3.xlarge.elasticsearch", "InstanceCount": 4, "DedicatedMasterEnabled": true, "ZoneAwarenessEnabled": true, "DedicatedMasterType": "m3.xlarge.elasticsearch", "DedicatedMasterCount": 3}' ``` ```bash title="Output" { "DomainStatus": { "DomainId": "000000000000/mylogs-2", "DomainName": "mylogs-2", "ARN": "arn:aws:es:us-east-1:000000000000:domain/mylogs-2", "Created": true, "Deleted": false, "Endpoint": "mylogs-2.us-east-1.es.localhost.localstack.cloud:4566", "Processing": true, "ElasticsearchVersion": "7.10", "ElasticsearchClusterConfig": { "InstanceType": "m3.xlarge.elasticsearch", "InstanceCount": 4, "DedicatedMasterEnabled": true, "ZoneAwarenessEnabled": true, "DedicatedMasterType": "m3.xlarge.elasticsearch", "DedicatedMasterCount": 3 }, "EBSOptions": { "EBSEnabled": true, "VolumeType": "gp2", "VolumeSize": 10, "Iops": 0 }, "CognitoOptions": { "Enabled": false } } } ``` 3. If the `Processing` status is true, it means that the cluster is not yet healthy. You can run `describe-elasticsearch-domain` to receive the status: ```bash lstk aws es describe-elasticsearch-domain --domain-name mylogs-2 ``` 4. Check the cluster health endpoint and create indices: ```bash curl mylogs-2.us-east-1.es.localhost.localstack.cloud:4566/_cluster/health ``` ```bash title="Output" {"cluster_name":"es-docker-cluster","status":"green","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":0,"active_shards":0,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":100.0}[~] ``` 5. Create an example index: ```bash curl -X PUT mylogs-2.us-east-1.es.localhost.localstack.cloud:4566/my-index ``` ```bash title="Output" {"acknowledged":true,"shards_acknowledged":true,"index":"my-index"} ``` ## Differences to AWS * By default, AWS only sets the `Endpoint` attribute of the cluster status once the cluster is up. LocalStack will return the endpoint immediately, but keep `Processing = "true"` until the cluster has been started. * The `CustomEndpointOptions` allows arbitrary endpoint URLs, which is not allowed in AWS ## Current Limitations The default Elasticsearch version used is 7.10.0. This is a slight deviation from the default version used in AWS (Elasticsearch 1.5), which is not supported in LocalStack. ## API Coverage # EventBridge > Get started with EventBridge on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction EventBridge provides a centralized mechanism to discover and communicate events across various AWS services and applications. EventBridge allows you to register, track, and resolve events, which indicates a change in the environment and then applies a rule to route the event to a target. EventBridge rules are tied to an Event Bus to manage event-driven workflows. You can use either identity-based or resource-based policies to control access to EventBridge resources, where the former can be attached to IAM users, groups, and roles, and the latter can be attached to specific AWS resources. LocalStack allows you to use the EventBridge APIs in your local environment to create rules that route events to a target. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of EventBridge's integration with LocalStack. For information on EventBridge Pipes, please refer to the [EventBridge Pipes](/aws/services/pipes/) documentation. :::note LocalStack 2026.04.0 removes the legacy v1 EventBridge provider. If your configuration sets `PROVIDER_OVERRIDE_EVENTS`, remove it from your configuration profiles, Docker Compose files, CI settings, and other startup environments. Any value for this variable now points to an invalid provider configuration and prevents the `events` provider from loading. ::: ## Getting Started This guide is designed for users new to EventBridge and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate creating an EventBridge rule to run a Lambda function when a custom event is published to an event bus. ### Create an EventBridge Bus First, create a custom EventBridge bus using the [`CreateEventBus`](https://docs.aws.amazon.com/cli/latest/reference/events/create-event-bus.html) API: ```bash lstk aws events create-event-bus \ --name my-custom-bus ``` While you can always use the default event bridge bus automatically available and configured for your account in any region, custom event buses are much more commonly used in practice and provide better organization for your events. ### Create a Lambda Function To create a new Lambda function, create a new file called `index.js` with the following code: ```js showLineNumbers 'use strict'; exports.handler = (event, context, callback) => { console.log('LogEventBridgeEvent'); console.log('Received event:', JSON.stringify(event, null, 2)); callback(null, 'Finished'); }; ``` Run the following command to create a new Lambda function using the [`CreateFunction`](https://docs.aws.amazon.com/cli/latest/reference/lambda/create-function.html) API: ```bash zip function.zip index.js lstk aws lambda create-function \ --function-name events-example \ --runtime nodejs16.x \ --zip-file fileb://function.zip \ --handler index.handler \ --role arn:aws:iam::000000000000:role/cool-stacklifter ``` The output will consist of the `FunctionArn`, which you will need to add the Lambda function to the EventBridge target. ### Create an EventBridge Rule Run the following command to create a new EventBridge rule using the [`PutRule`](https://docs.aws.amazon.com/cli/latest/reference/events/put-rule.html) API: ```bash lstk aws events put-rule \ --name my-custom-rule \ --event-bus-name my-custom-bus \ --event-pattern '{"source":["my-source"],"detail-type":["my-detail-type"]}' \ --state ENABLED ``` In the above command, we have specified an event pattern that will match events with: - `source` field equal to `my-source` - `detail-type` field equal to `my-detail-type` This rule will trigger whenever an event matching this pattern is published to the custom event bus. Next, grant the EventBridge service principal (`events.amazonaws.com`) permission to run the rule, using the [`AddPermission`](https://docs.aws.amazon.com/cli/latest/reference/lambda/add-permission.html) API: ```bash lstk aws lambda add-permission \ --function-name events-example \ --statement-id my-custom-event \ --action 'lambda:InvokeFunction' \ --principal events.amazonaws.com \ --source-arn arn:aws:events:us-east-1:000000000000:rule/my-custom-bus/my-custom-rule ``` ### Add the Lambda Function as a Target Create a file named `targets.json` with the following content: ```json [ { "Id": "1", "Arn": "arn:aws:lambda:us-east-1:000000000000:function:events-example" } ] ``` Finally, add the Lambda function as a target to the EventBridge rule using the [`PutTargets`](https://docs.aws.amazon.com/cli/latest/reference/events/put-targets.html) API: ```bash lstk aws events put-targets \ --rule my-custom-rule \ --event-bus-name my-custom-bus \ --targets file://targets.json ``` ### Send an Event to Trigger the Lambda Now, send an event that matches the rule pattern to trigger the Lambda function using the [`PutEvents`](https://docs.aws.amazon.com/cli/latest/reference/events/put-events.html) API: ```bash lstk aws events put-events \ --entries '[{"Source": "my-source", "DetailType": "my-detail-type", "Detail": "{\"key\": \"value\"}", "EventBusName": "my-custom-bus"}]' ``` This event will match the pattern we defined in the rule and should trigger the Lambda function immediately. ### Verify the Lambda invocation You can verify the Lambda invocation by checking the CloudWatch logs. Run the following command to list the CloudWatch log groups: ```bash lstk aws logs describe-log-groups ``` The output will contain the log group name, which you can use to list the log streams: ```bash lstk aws logs describe-log-streams \ --log-group-name /aws/lambda/events-example ``` Alternatively, you can fetch LocalStack logs to verify the Lambda invocation: ```bash title="Output" lstk logs ... emulator | 2023-07-17T09:37:52.028 INFO --- [ asgi_gw_0] localstack.request.aws : AWS lambda.Invoke => 202 emulator | 2023-07-17T09:37:52.106 INFO --- [ asgi_gw_0] localstack.request.http : POST /_localstack_lambda/97e08ac50c18930f131d9dd9744b8df4/invocations/ecb744d0-b3f2-400f-9e49-c85cf12b1e00/logs => 202 emulator | 2023-07-17T09:37:52.114 INFO --- [ asgi_gw_0] localstack.request.http : POST /_localstack_lambda/97e08ac50c18930f131d9dd9744b8df4/invocations/ecb744d0-b3f2-400f-9e49-c85cf12b1e00/response => 202 ... ``` ## Supported target types At this time LocalStack supports the following [target types](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-targets.html#eb-console-targets) for EventBridge rules: - Lambda function - SNS Topic - SQS queue - StepFunctions StateMachine - Firehose - Event bus - API destination - Kinesis - CloudWatch log group - API Gateway ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing EventBridge Buses. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **EventBridge** under the **App Integration** section. ![EventBridge Resource Browser](/images/aws/eventbridge-resource-browser.png) The Resource Browser allows you to perform the following actions: - **View the Event Buses**: You can view the list of EventBridge Buses running locally, alongside their Amazon Resource Names (ARNs) and Policies. - **Create Event Rule**: You can create a new Event Rule by specifying **Name**, **Description**, **Event Pattern**, **Schedule Expressions**, **State**, **Role ARN**, and **Tags**. - **Trigger Event**: You can trigger an Event by specifying the **Entries** and **Endpoint Id**. While creating an Entry, you must specify **Source**, **Event Bus Name**, **Detail**, **Resources**, **Detail Type**, and **Trace Header**. - **Remove Selected**: You can remove the selected EventBridge Bus. ## API Coverage # Data Firehose > Get started with Data Firehose on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; :::note This service was formerly called as 'Kinesis Data Firehose'. ::: ## Introduction Data Firehose is a service provided by AWS that allows you to extract, transform and load streaming data into various destinations, such as Amazon S3, Amazon Redshift, and Elasticsearch. With Data Firehose, you can ingest and deliver real-time data from different sources as it automates data delivery, handles buffering and compression, and scales according to the data volume. LocalStack allows you to use the Data Firehose APIs in your local environment to load and transform real-time data. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Data Firehose's integration with LocalStack. ## Getting started This guide is designed for users new to Data Firehose and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to use Firehose to load Kinesis data into Elasticsearch with S3 Backup with the AWS CLI. ### Create an Elasticsearch domain You can create an Elasticsearch domain using the [`create-elasticsearch-domain`](https://docs.aws.amazon.com/cli/latest/reference/es/create-elasticsearch-domain.html) command. Execute the following command to create a domain named `es-local`: ```bash lstk aws es create-elasticsearch-domain --domain-name es-local ``` Save the value of the `Endpoint` field from the response, as it will be required further down to confirm the setup. ### Create the source Kinensis stream Now let us create our target S3 bucket and our source Kinesis stream: Before creating the stream, we need to create an S3 bucket to store our backup data. You can do this using the [`mb`](https://docs.aws.amazon.com/cli/latest/reference/s3/mb.html) command: ```bash lstk aws s3 mb s3://kinesis-activity-backup-local ``` You can now use the [`CreateStream`](https://docs.aws.amazon.com/kinesis/latest/APIReference/API_CreateStream.html) API to create a Kinesis stream named `kinesis-es-local-stream` with two shards: ```bash lstk aws kinesis create-stream \ --stream-name kinesis-es-local-stream \ --shard-count 2 ``` ### Create a Firehose delivery stream You can now create the Firehose delivery stream. In this configuration, Elasticsearch serves as the destination, while S3 serves as the repository for our AllDocuments backup. Within the `kinesis-stream-source-configuration`, it is required to specify the ARN of our Kinesis stream and the role that will allow you the access to the stream. The `elasticsearch-destination-configuration` sets vital parameters, which includes the access role, `DomainARN` of the Elasticsearch domain where you wish to publish, and the settings including the `IndexName` and `TypeName` for the Elasticsearch setup. Additionally to backup all documents to S3, the `S3BackupMode` parameter is set to `AllDocuments`, which is accompanied by `S3Configuration`. :::note Within LocalStack's default configuration, IAM roles remain unverified and no strict validation is applied on ARNs. However, when operating within the AWS environment, you need to check the access rights of the specified role for the task. ::: You can use the [`CreateDeliveryStream`](https://docs.aws.amazon.com/firehose/latest/APIReference/API_CreateDeliveryStream.html) API to create a Firehose delivery stream named `activity-to-elasticsearch-local`: ```bash lstk aws firehose create-delivery-stream \ --delivery-stream-name activity-to-elasticsearch-local \ --delivery-stream-type KinesisStreamAsSource \ --kinesis-stream-source-configuration "KinesisStreamARN=arn:aws:kinesis:us-east-1:000000000000:stream/kinesis-es-local-stream,RoleARN=arn:aws:iam::000000000000:role/Firehose-Reader-Role" \ --elasticsearch-destination-configuration "RoleARN=arn:aws:iam::000000000000:role/Firehose-Reader-Role,DomainARN=arn:aws:es:us-east-1:000000000000:domain/es-local,IndexName=activity,TypeName=activity,S3BackupMode=AllDocuments,S3Configuration={RoleARN=arn:aws:iam::000000000000:role/Firehose-Reader-Role,BucketARN=arn:aws:s3:::kinesis-activity-backup-local}" ``` On successful execution, the command will return the `DeliveryStreamARN` of the created delivery stream: ```bash title="Output" { "DeliveryStreamARN": "arn:aws:firehose:us-east-1:000000000000:deliverystream/activity-to-elasticsearch-local" } ``` ### Testing the setup Before testing the integration, it's necessary to confirm if the local Elasticsearch cluster is up. You can use the [`describe-elasticsearch-domain`](https://docs.aws.amazon.com/cli/latest/reference/es/describe-elasticsearch-domain.html) command to check the status of the Elasticsearch cluster. Run the following command: ```bash lstk aws es describe-elasticsearch-domain \ --domain-name es-local | jq ".DomainStatus.Processing" ``` Once the command returns `false`, you can move forward with data ingestion. The data can be added to the source Kinesis stream or directly to the Firehose delivery stream. You can add data to the Kinesis stream using the [`PutRecord`](https://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecord.html) API. The following command adds a record to the stream: ```bash lstk aws kinesis put-record \ --stream-name kinesis-es-local-stream \ --data '{ "target": "barry" }' \ --partition-key partition ``` :::tip For users using AWS CLI v2, consider adding `--cli-binary-format raw-in-base64-out` to the command mentioned above. ::: You can use the [`PutRecord`](https://docs.aws.amazon.com/firehose/latest/APIReference/API_PutRecord.html) API to add data to the Firehose delivery stream. The following command adds a record to the stream: ```bash lstk aws firehose put-record \ --delivery-stream-name activity-to-elasticsearch-local \ --record '{ "Data": "eyJ0YXJnZXQiOiAiSGVsbG8gd29ybGQifQ==" }' ``` To review the entries in Elasticsearch, you can employ [curl](https://curl.se/) for simplicity. Remember to replace the URL with the `Endpoint` field from the initial `create-elasticsearch-domain` operation. ```bash curl -s http://es-local.us-east-1.es.localhost.localstack.cloud:443/activity/_search | jq '.hits.hits' ``` You will get an output similar to the following: ```bash title="Output" [ { "_index": "activity", "_type": "activity", "_id": "f38e2c49-d101-46aa-9ce2-0d2ea8fcd133", "_score": 1, "_source": { "target": "Hello world" } }, { "_index": "activity", "_type": "activity", "_id": "d2f1c125-b3b0-4c7c-ba90-8acf4075a682", "_score": 1, "_source": { "target": "barry" } } ] ``` If you receive a comparable output, your Firehose delivery stream setup is accurate! Additionally, take a look at the designated S3 bucket to ensure the backup process is functioning correctly. ## Examples The following code snippets and sample applications provide practical examples of how to use Data Firehose in LocalStack for various use cases: - [Search application with Lambda, Kinesis, Firehose, ElasticSearch, S3](https://github.com/localstack/sample-fuzzy-movie-search-lambda-kinesis-elasticsearch) - [Streaming Data Pipeline with Kinesis, Tinybird, CloudWatch, Lambda](https://github.com/localstack/serverless-streaming-data-pipeline) ## API Coverage # Fault Injection Service (FIS) > Get started with Fault Injection Service (FIS) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Fault Injection Service (FIS) is a service provided by Amazon Web Services that enables you to test the resilience of your applications and infrastructure by injecting faults and failures into your AWS resources. FIS simulates faults such as resource unavailability and service errors to assess the impact on your application's performance and availability. The full list of such possible fault injections is available in the [AWS docs](https://docs.aws.amazon.com/fis/latest/userguide/fis-actions-reference.html). LocalStack allows you to use the FIS APIs in your local environment to introduce faults in other services, in order to check how your setup behaves when parts of it stop working locally. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of FIS API's integration with LocalStack. :::tip LocalStack also features its own powerful chaos engineering tool, [Chaos API](/aws/developer-tools/chaos-engineering/chaos-api). ::: ## Concepts FIS defines the following elements: 1. Action: Type of fault to introduce 1. Target: Resources to be impacted 1. Duration of the disruption. Together this is termed as an Experiment. After the designated time, running experiments restore systems to their original state and cease introducing faults. :::note FIS experiment emulation is part of LocalStack Enterprise. If you'd like to try it out, please [contact us](https://www.localstack.cloud/demo). ::: FIS actions can be categorized into two main types: 1. One-time events: For example, the `aws:ec2:stop-instances` FIS action, which sends a [`StopInstances`](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_StopInstances.html) API to specific EC2 instances. Some of these events can automatically be undone after a defined time, such as sending a [`StartInstances`](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_StartInstances.html) command to the affected instances. 1. Probabilistic API errors: For instance, using `aws:fis:inject-api-unavailable-error` to introduce an HTTP 503 error. ## Getting started This guide is designed for users new to FIS and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create an experiment that stops EC2 instances. ### Creating an experiment Create a new file named `create-experiment.json`. This file should contain a JSON configuration that will be utilized during the subsequent invocation of the [`CreateExperimentTemplate`](https://docs.aws.amazon.com/fis/latest/APIReference/API_CreateExperimentTemplate.html) API. ```json showshowLineNumbers { "actions": { "StopInstance": { "actionId": "aws:ec2:stop-instances", "targets": { "Instances": "InstancesToStop" }, "description": "stop instances" } }, "targets": { "InstancesToStop": { "resourceType": "aws:ec2:instance", "resourceTags": { "foo": "bar" }, "selectionMode": "COUNT(1)" } }, "description": "template for a test action", "stopConditions": [ { "source": "none" } ], "roleArn": "arn:aws:iam:123456789012:role/ExperimentRole" } ``` This configuration will result in EC2 [`StopInstances`](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_StopInstances.html) operation being invoked against EC2 instances that have the resource tags `Key=foo Value=bar`. Settings pertaining to `stopConditions` and `roleArn` hold no significance for in LocalStack FIS emulation. Nonetheless, they are obligatory fields according to AWS specifications and must be included. Run the following command to create an FIS experiment template using the configuration file we just created: ```bash lstk aws fis create-experiment-template --cli-input-json file://create-experiment.json ``` The following output would be retrieved: ```bash title="Output" { "experimentTemplate": { "id": "ad16589a-4a91-4aee-88df-c33446605882", "description": "template for a test action", "targets": { "InstancesToStop": { "resourceType": "aws:ec2:instance", "resourceTags": { "foo": "bar" }, "selectionMode": "COUNT(1)" } }, "actions": { "StopInstance": { "actionId": "aws:ec2:stop-instances", "description": "stop instances", "targets": { "Instances": "InstancesToStop" } } }, "stopConditions": [ { "source": "none" } ], "creationTime": 1718268196.305881, "lastUpdateTime": 1718268196.305881, "roleArn": "arn:aws:iam:123456789012:role/ExperimentRole" } } ``` You can list all the templates you have created using the [`ListExperimentTemplates`](https://docs.aws.amazon.com/fis/latest/APIReference/API_ListExperimentTemplates.html): ```bash lstk aws fis list-experiment-templates ``` ### Starting the experiment Now let us start an EC2 instance that will match the criteria we specified in the experiment template. ```bash lstk aws ec2 run-instances \ --image-id ami-024f768332f0 \ --count 1 \ --tag-specifications '{"ResourceType": "instance", "Tags": [{"Key": "foo", "Value": "bar"}]}' ``` You can start the experiment using the [`StartExperiment`](https://docs.aws.amazon.com/fis/latest/APIReference/API_StartExperiment.html). Run the following command and specify the ID of the experiment template you created earlier: ```bash lstk aws fis start-experiment --experiment-template-id ad16589a-4a91-4aee-88df-c33446605882 ``` ```bash title="Output" { "experiment": { "id": "efee7c02-8733-4d7c-9628-1b60bbec9759", "experimentTemplateId": "ad16589a-4a91-4aee-88df-c33446605882", "roleArn": "arn:aws:iam:123456789012:role/ExperimentRole", "state": { "status": "running" }, "targets": { "InstancesToStop": { "resourceType": "aws:ec2:instance", "resourceTags": { "foo": "bar" }, "selectionMode": "COUNT(1)" } }, "actions": { "StopInstance": { "actionId": "aws:ec2:stop-instances", "description": "stop instances", "targets": { "Instances": "InstancesToStop" } } }, "stopConditions": [ { "source": "none" } ], "creationTime": 1718268311.209798, "startTime": 1718268311.209798 } } ``` You can use the [`ListExperiments`](https://docs.aws.amazon.com/fis/latest/APIReference/API_ListExperiments.html) to check the status of your experiment. Run the following command: ```bash lstk aws fis list-experiments ``` You can fetch the details of your experiment using the [`GetExperiment`](https://docs.aws.amazon.com/fis/latest/APIReference/API_GetExperiment.html) API. Run the following command and specify the ID of the experiment you created earlier: ```bash lstk aws fis get-experiment --id efee7c02-8733-4d7c-9628-1b60bbec9759 ``` ### Verifying the outcome You can now test that the experiment is working as expected by trying to obtain the state of the EC2 instance using [`DescribeInstanceStatus`](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstanceStatus.html). Run the following command: ```bash lstk aws ec2 describe-instance-status \ --instance-ids i-3c40b52ab72f99c63 \ --output json \ --query InstanceStatuses[0].InstanceState ``` If everything happened as expected, the following output would be retrieved: ```json { "Code": 80, "Name": "stopped" } ``` ## Supported Actions LocalStack FIS currently supports the following actions: - **`aws:ec2:stop-instances`**: Runs EC2 StopInstances on the target EC2 instances. - **`aws:ec2:terminate-instances`**: Runs EC2 TerminateInstances on the target EC2 instances. - **`aws:rds:reboot-db-instances`**: Runs EC2 RebootInstances on the target EC2 instances. - **`aws:ssm:send-command`**: Runs the Systems Manager SendCommand on the target EC2 instances. - **`aws:ecs:stop-task`**: Runs ECS StopTask on the target ECS tasks. If you would like support for more FIS actions, please make a feature request on [GitHub Discussion](https://github.com/orgs/localstack/discussions/new/choose). ## Current Limitations - LocalStack does not implement the [selection mode](https://docs.aws.amazon.com/fis/latest/userguide/targets.html#target-selection-mode) mechanism available on AWS. - LocalStack ignores [`RoleARN`](https://docs.aws.amazon.com/fis/latest/APIReference/API_ExperimentTemplate.html#fis-Type-ExperimentTemplate-roleArn). On AWS, FIS executes actions based on permissions granted by the specified `RoleARN`. ## API Coverage # Glacier > Get started with S3 Glacier on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Glacier is a data storage service provided by Amazon Web Services to suit the long-term storage of archives and backup of infrequently accessed data. It offers various retrieval options, different levels of retrieval speed, and more. Glacier uses a Vault container to store your data, similar to how S3 stores data in Buckets. A Vault further holds the data in an Archive, which can contain text, images, video, and audio files. Glacier uses Jobs to retrieve the data in an Archive or list the inventory of a Vault. LocalStack allows you to use the Glacier APIs in your local environment to manage Vaults and Archives. You can use the Glacier API to configure and set up vaults where you can store archives and manage them. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Glacier's integration with LocalStack. ## Getting started This guide is designed for users new to Glacier and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a vault, upload an archive, initiate a job to get an inventory details or download an archive, and delete the archive and vault with the AWS CLI. ### Create a vault You can create a vault using the [`CreateVault`](https://docs.aws.amazon.com/amazonglacier/latest/dev/api-vault-put.html) API. Run the follow command to create a Glacier Vault named `sample-vault`. ```bash lstk aws glacier create-vault --vault-name sample-vault --account-id - ``` You can get the details from your vault using the [`DescribeVault`](https://docs.aws.amazon.com/amazonglacier/latest/dev/api-vault-get.html) API. Run the following command to describe your vault. ```bash lstk aws glacier describe-vault --vault-name sample-vault --account-id - ``` ```bash title="Output" { "VaultARN": "arn:aws:glacier:us-east-1:000000000000:vaults/sample-vault", "VaultName": "sample-vault", "CreationDate": "2023-09-11T15:07:28.000Z", "LastInventoryDate": "2023-09-11T15:07:28.000Z", "NumberOfArchives": 0, "SizeInBytes": 0 } ``` ### Upload an archive to a vault You can upload an archive or an individual file to a vault using the [`UploadArchive`](https://docs.aws.amazon.com/amazonglacier/latest/dev/api-archive-post.html) API. Download a random image from the internet and save it as `image.jpg`. Run the following command to upload the file to your Glacier vault: ```bash lstk aws glacier upload-archive --vault-name sample-vault --account-id - --body image.jpg ``` ```bash title="Output" { "location": "/000000000000/vaults/sample-vault/archives/d41d8cd98f00b204e9800998ecf8427e", "checksum": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", "archiveId": "d41d8cd98f00b204e9800998ecf8427e" } ``` ### Initiate the retrieval of an archive from a vault You can initiate the retrieval of an archive from a vault using the [`InitiateJob`](https://docs.aws.amazon.com/amazonglacier/latest/dev/api-initiate-job-post.html) API. To download an archive, you will need to initiate an `archive-retrieval` job first to make the Archive available for download. ```bash lstk aws glacier initiate-job \ --vault-name sample-vault \ --account-id - \ --job-parameters '{"Type":"archive-retrieval","ArchiveId":"d41d8cd98f00b204e9800998ecf8427e"}' ``` ```bash title="Output" { "location": "//vaults/sample-vault/jobs/25CEOTJ7ZUR5Q7YY0B1O55AE4C3L1502EOHWMNY10IIYEBWEQB73D23S8BVYO9RTRTPLRK2LJLUCCRM52GDV87C9A4JW", "jobId": "25CEOTJ7ZUR5Q7YY0B1O55AE4C3L1502EOHWMNY10IIYEBWEQB73D23S8BVYO9RTRTPLRK2LJLUCCRM52GDV87C9A4JW" } ``` ### List the jobs You can list the current and previous processes, called Jobs, to monitor the requests sent to the Glacier API using the [`ListJobs`](https://docs.aws.amazon.com/amazonglacier/latest/dev/api-jobs-get.html) API. ```bash lstk aws glacier list-jobs --vault-name sample-vault --account-id - ``` ```bash title="Output" { "JobList": [ { "JobId": "25CEOTJ7ZUR5Q7YY0B1O55AE4C3L1502EOHWMNY10IIYEBWEQB73D23S8BVYO9RTRTPLRK2LJLUCCRM52GDV87C9A4JW", "Action": "ArchiveRetrieval", "ArchiveId": "d41d8cd98f00b204e9800998ecf8427e", "VaultARN": "arn:aws:glacier:us-east-1:000000000000:vaults/sample-vault", "CreationDate": "2023-09-11T15:25:54.000Z", "Completed": true, "StatusCode": "Succeeded", "ArchiveSizeInBytes": 0, "InventorySizeInBytes": 10000, "CompletionDate": "2023-09-11T15:25:59.000Z", "Tier": "Standard" } ] } ``` ### Download the result of an archive retrieval You can download the output of an `ArchiveRetrieval` job with the [`GetJobOutput`](https://docs.aws.amazon.com/amazonglacier/latest/dev/api-job-output-get.html) API. The data download process can be verified through the previous `ListJobs` call to check progress. Once the `ArchiveRetrieval` Job is complete, the data can be downloaded. You can use the `JobId` of the Job to download your archive with the following command: ```bash lstk aws glacier get-job-output \ --vault-name sample-vault \ --account-id - \ --job-id 25CEOTJ7ZUR5Q7YY0B1O55AE4C3L1502EOHWMNY10IIYEBWEQB73D23S8BVYO9RTRTPLRK2LJLUCCRM52GDV87C9A4JW \ my-archive.jpg ``` :::danger Please not that currently, this operation is only mocked, and will create an empty file named `my-archive.jpg`, not containing the contents of your archive. ::: ### Retrieve the inventory information You can also initiate the retrieval of the inventory of a vault using the same [`InitiateJob`](https://docs.aws.amazon.com/amazonglacier/latest/dev/api-initiate-job-post.html) API. Initiate a job of the specified type to get the details of the individual inventory items inside a Vault using the `initiate-job` command: ```bash lstk aws glacier initiate-job \ --vault-name sample-vault \ --account-id - \ --job-parameters '{"Type":"inventory-retrieval","ArchiveId":"d41d8cd98f00b204e9800998ecf8427e"}' ``` ```bash title="Output" { "location": "//vaults/sample-vault/jobs/P5972CSWFR803BHX48OD1A7JWNBFJUMYVWCMZWY55ZJPIJMG1XWFV9ISZPZH1X3LBF0UV3UG6ORETM0EHE5R86Z47B1F", "jobId": "P5972CSWFR803BHX48OD1A7JWNBFJUMYVWCMZWY55ZJPIJMG1XWFV9ISZPZH1X3LBF0UV3UG6ORETM0EHE5R86Z47B1F" } ``` In the same fashion as the archive retrieval, you can now download the result of the inventory retrieval job using `GetJobOutput` using the `JobId` from the result of the previous command: ```bash lstk aws glacier get-job-output \ --vault-name sample-vault \ --account-id - \ --job-id P5972CSWFR803BHX48OD1A7JWNBFJUMYVWCMZWY55ZJPIJMG1XWFV9ISZPZH1X3LBF0UV3UG6ORETM0EHE5R86Z47B1F \ inventory.json ``` Inspecting the content of the `inventory.json` file, we can find an inventory of the vault: ```json title="inventory.json" { "VaultARN": "arn:aws:glacier:us-east-1:000000000000:vaults/sample-vault", "InventoryDate": "2023-09-11T17:20:48.000Z", "ArchiveList": [ { "ArchiveId": "d41d8cd98f00b204e9800998ecf8427e", "ArchiveDescription": "", "CreationDate": "2023-09-11T15:13:41.000Z", "Size": 0, "SHA256TreeHash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" } ] } ``` ### Delete an archive You can delete a Glacier archive using the [`DeleteArchive`](https://docs.aws.amazon.com/amazonglacier/latest/dev/api-archive-delete.html) API. Run the following command to delete the previously created archive: ```bash lstk aws glacier delete-archive \ --vault-name sample-vault \ --account-id - \ --archive-id d41d8cd98f00b204e9800998ecf8427e ``` ### Delete a vault You can delete a Glacier vault with the [`DeleteVault`](https://docs.aws.amazon.com/amazonglacier/latest/dev/api-vault-delete.html) API. Run the following command to delete the vault: ```bash lstk aws glacier delete-vault \ --vault-name sample-vault \ --account-id - ``` ## API Coverage # Glue > Get started with Glue on LocalStack import FeatureCoverage from '../../../../components/feature-coverage/FeatureCoverage'; ## Introduction The Glue API in LocalStack for AWS allows you to run ETL (Extract-Transform-Load) jobs locally, maintaining table metadata in the local Glue data catalog, and using the Spark ecosystem (PySpark/Scala) to run data processing workflows. LocalStack allows you to use the Glue APIs in your local environment. LocalStack uses a container-based Glue job executor, running Glue jobs within a Docker environment (or as pods when deployed on Kubernetes). The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Glue's integration with LocalStack. ## Getting started This guide is designed for users new to Glue and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create databases and table metadata in Glue, run Glue ETL jobs, import databases from Athena, and run Glue Crawlers with the AWS CLI. :::note In order to run Glue jobs, some additional dependencies have to be fetched from the network, including a Docker image of approximately 1.5GB which includes Spark, Presto, Hive and other tools. These dependencies are automatically fetched when you start up the service, so please make sure you're on a decent internet connection when pulling the dependencies for the first time. ::: ### Creating Databases and Table Metadata The commands below illustrate the creation of some very basic entries (databases, tables) in the Glue data catalog: ```bash lstk aws glue create-database --database-input '{"Name":"db1"}' lstk aws glue create-table --database db1 --table-input '{"Name":"table1"}' lstk aws glue get-tables --database db1 ``` ```bash title="Output" { "TableList": [ { "Name": "table1", "DatabaseName": "db1" } ] } ``` ### Running Scripts with Scala and PySpark Create a new PySpark script named `job.py` with the following code: ```python showshowLineNumbers from pyspark.sql import SparkSession def init_spark(): spark = SparkSession.builder.appName("HelloWorld").getOrCreate() sc = spark.sparkContext return spark,sc def main(): spark,sc = init_spark() nums = sc.parallelize([1,2,3,4]) print(nums.map(lambda x: x*x).collect()) if __name__ == '__main__': main() ``` You can now copy the script to an S3 bucket: ```bash lstk aws s3 mb s3://glue-test lstk aws s3 cp job.py s3://glue-test/job.py ``` Next, you can create a job definition: ```bash lstk aws glue create-job \ --name job1 \ --role arn:aws:iam::000000000000:role/glue-role \ --command '{"Name": "pythonshell", "ScriptLocation": "s3://glue-test/job.py"}' ``` You can finally start the job execution: ```bash lstk aws glue start-job-run --job-name job1 ``` The returned `JobRunId` can be used to query the status job the job execution, until it becomes `SUCCEEDED`: ```bash lstk aws glue get-job-run --job-name job1 --run-id ``` ```bash title="Output" { "JobRun": { "Id": "733b76d0", "Attempt": 1, "JobRunState": "SUCCEEDED" } } ``` For a more detailed example illustrating how to run a local Glue PySpark job, please refer to this [sample repository](https://github.com/localstack/localstack-pro-samples/tree/master/glue-etl-jobs). ### Importing Athena Tables into Glue Data Catalog The Glue data catalog is integrated with Athena, and the database/table definitions can be imported via the `import-catalog-to-glue` API. Assume you are running the following Athena queries to create databases and table definitions: ```sql CREATE DATABASE db2 CREATE EXTERNAL TABLE db2.table1 (a1 Date, a2 STRING, a3 INT) LOCATION 's3://test/table1' CREATE EXTERNAL TABLE db2.table2 (a1 Date, a2 STRING, a3 INT) LOCATION 's3://test/table2' ``` Then this command will import these DB/table definitions into the Glue data catalog: ```bash lstk aws glue import-catalog-to-glue ``` Afterwards, the databases and tables will be available in Glue. You can query the databases with the `get-databases` operation: ```bash lstk aws glue get-databases ``` ```bash title="Output" { "DatabaseList": [ ... { "Name": "db2", "Description": "Database db2 imported from Athena", "TargetDatabase": { "CatalogId": "000000000000", "DatabaseName": "db2" } } ] } ``` And you can query the databases with the `get-databases` operation: ```bash lstk aws glue get-tables --database-name db2 ``` ```bash title="Output" { "TableList": [ { "Name": "table1", "DatabaseName": "db2", "Description": "Table db2.table1 imported from Athena", "CreateTime": ... }, { "Name": "table2", "DatabaseName": "db2", "Description": "Table db2.table2 imported from Athena", "CreateTime": ... } ] } ``` ### Crawlers Glue crawlers allow extracting metadata from structured data sources. LocalStack Glue currently supports S3 targets (configurable via `S3Targets`), as well as JDBC targets (configurable via `JdbcTargets`). Support for other target types is in our pipeline and will be added soon. #### S3 Crawler Example The example below illustrates crawling tables and partition metadata from S3 buckets. You can first create an S3 bucket with a couple of items: ```bash lstk aws s3 mb s3://test printf "1, 2, 3, 4\n5, 6, 7, 8" > /tmp/file.csv lstk aws s3 cp /tmp/file.csv s3://test/table1/year=2021/month=Jan/day=1/file.csv lstk aws s3 cp /tmp/file.csv s3://test/table1/year=2021/month=Jan/day=2/file.csv lstk aws s3 cp /tmp/file.csv s3://test/table1/year=2021/month=Feb/day=1/file.csv lstk aws s3 cp /tmp/file.csv s3://test/table1/year=2021/month=Feb/day=2/file.csv ``` You can then create and trigger the crawler: ```bash lstk aws glue create-database --database-input '{"Name":"db1"}' lstk aws glue create-crawler \ --name c1 \ --database-name db1 \ --role arn:aws:iam::000000000000:role/glue-role \ --targets '{"S3Targets": [{"Path": "s3://test/table1"}]}' lstk aws glue start-crawler --name c1 ``` Finally, you can query the table metadata that has been created by the crawler: ```bash lstk aws glue get-tables --database-name db1 ``` ```bash title="Output" { "TableList": [{ "Name": "table1", "DatabaseName": "db1", "PartitionKeys": [ ... ] ... ``` You can also query the created table partitions: ```bash lstk aws glue get-partitions --database-name db1 --table-name table1 ``` ```bash title="Output" { "Partitions": [{ "Values": ["2021", "Jan", "1"], "DatabaseName": "db1", "TableName": "table1", ... ``` #### JDBC Crawler Example When using JDBC crawlers, you can point your crawler towards a Redshift database created in LocalStack. Below is a rough outline of the steps required to get the integration for the JDBC crawler working. You can first create the local Redshift cluster via: ```bash lstk aws redshift create-cluster \ --cluster-identifier c1 \ --node-type dc1.large \ --master-username test \ --master-user-password test \ --db-name db1 ``` The output of this command contains the endpoint address of the created Redshift database: ```bash title="Output" ... "Endpoint": { "Address": "localhost.localstack.cloud", "Port": 4510 }, ... ``` Then you can use any JDBC or Postgres client to create a table `mytable1` in the Redshift database, and fill the table with some data. Next, you're creating the Glue database, the JDBC connection, as well as the crawler: ```bash lstk aws glue create-database --database-input '{"Name":"gluedb1"}' lstk aws glue create-connection --connection-input \ {"Name":"conn1","ConnectionType":"JDBC","ConnectionProperties":{"USERNAME":"test","PASSWORD":"test","JDBC_CONNECTION_URL":"jdbc:redshift://localhost.localstack.cloud:4510/db1"}}' lstk aws glue create-crawler \ --name c1 \ --database-name gluedb1 \ --role arn:aws:iam::000000000000:role/glue-role \ --targets '{"JdbcTargets":[{"ConnectionName":"conn1","Path":"db1/%/mytable1"}]}' lstk aws glue start-crawler --name c1 ``` Once the crawler has started, you have to wait until the `State` turns to `READY` when querying the current state: ```bash lstk aws glue get-crawler --name c1 ``` Once the crawler has finished running and is back in `READY` state, the Glue table within the `gluedb1` DB should have been populated and can be queried via the API. ### Schema Registry The Glue Schema Registry allows you to centrally discover, control, and evolve data stream schemas. With the Schema Registry, you can manage and enforce schemas and schema compatibilities in your streaming applications. It integrates nicely with [Managed Streaming for Kafka (MSK)](/aws/services/kafka/). :::note Currently, LocalStack supports the AVRO dataformat for the Glue Schema Registry. Support for other dataformats will be added in the future. ::: You can create a schema registry with the following command: ```bash lstk aws glue create-registry --registry-name demo-registry ``` You can create a schema in the newly created registry with the `create-schema` command: ```bash lstk aws glue create-schema --schema-name demo-schema \ --registry-id RegistryName=demo-registry \ --data-format AVRO \ --compatibility FORWARD \ --schema-definition '{"type":"record","namespace":"Demo","name":"Person","fields":[{"name":"Name","type":"string"}]}' ``` ```bash title="Output" { "RegistryName": "demo-registry", "RegistryArn": "arn:aws:glue:us-east-1:000000000000:file-registry/demo-registry", "SchemaName": "demo-schema", "SchemaArn": "arn:aws:glue:us-east-1:000000000000:schema/demo-registry/demo-schema", "DataFormat": "AVRO", "Compatibility": "FORWARD", "SchemaCheckpoint": 1, "LatestSchemaVersion": 1, "NextSchemaVersion": 2, "SchemaStatus": "AVAILABLE", "SchemaVersionId": "546d3220-6ab8-452c-bb28-0f1f075f90dd", "SchemaVersionStatus": "AVAILABLE" } ``` Once the schema has been created, you can create a new version: ```bash lstk aws glue register-schema-version \ --schema-id SchemaName=demo-schema,RegistryName=demo-registry \ --schema-definition '{"type":"record","namespace":"Demo","name":"Person","fields":[{"name":"Name","type":"string"}, {"name":"Address","type":"string"}]}' ``` ```bash title="Output" { "SchemaVersionId": "ee38732b-b299-430d-a88b-4c429d9e1208", "VersionNumber": 2, "Status": "AVAILABLE" } ``` You can find a more advanced sample in our [localstack-pro-samples repository on GitHub](https://github.com/localstack/localstack-pro-samples/tree/master/glue-msk-schema-registry), which showcases the integration with AWS MSK and automatic schema registrations (including schema rejections based on the compatibilities). ### Delta Lake Tables LocalStack Glue supports [Delta Lake](https://delta.io), an open-source storage framework that extends Parquet data files with a file-based transaction log for ACID transactions and scalable metadata handling. :::note Please note that Delta Lake tables are only [supported for Glue versions `3.0` and `4.0`](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-format-delta-lake.html). ::: To illustrate this feature, we take a closer look at a Glue sample job that creates a Delta Lake table, puts some data into it, and then queries data from the table. First, we define the PySpark job in a file named `job.py` (see below). The job first creates a database `db1` and table `table1`, then inserts data into the table via both a dataframe and an `INSERT INTO` query, and finally fetches the inserted rows via a `SELECT` query: ```python from awsglue.context import GlueContext from pyspark import SparkContext, SparkConf conf = SparkConf() conf.set("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension") conf.set("spark.sql.catalog.spark_catalog", "org.apache.spark.sql.delta.catalog.DeltaCatalog") glue_context = GlueContext(SparkContext.getOrCreate(conf=conf)) spark = glue_context.spark_session # create database and table spark.sql("CREATE DATABASE db1") spark.sql("CREATE TABLE db1.table1 (name string, key long) USING delta PARTITIONED BY (key) LOCATION 's3a://test/data/'") # create dataframe and write to table in S3 df = spark.createDataFrame([("test1", 123)], ["name", "key"]) df.write.format("delta").options(path="s3a://test/data/") \ .mode("append").partitionBy("key").saveAsTable("db1.table1") # insert data via 'INSERT' query spark.sql("INSERT INTO db1.table1 (name, key) VALUES ('test2', 456)") # get and print results, to run assertions further below result = spark.sql("SELECT * FROM db1.table1") print("SQL result:", result.toJSON().collect()) ``` You can now run the following commands to create and start the Glue job: ```bash lstk aws s3 mb s3://test lstk aws s3 cp job.py s3://test/job.py lstk aws glue create-job --name job1 --role arn:aws:iam::000000000000:role/test \ --glue-version 4.0 \ --command '{"Name": "pythonshell", "ScriptLocation": "s3://test/job.py"}' lstk aws glue start-job-run --job-name job1 ``` Retrieve the job run ID from the output of the `start-job-run` command. The execution of the Glue job can take a few moments - once the job has finished executing, you should see a log line with the query results in the LocalStack container logs, similar to the output below: ```bash title="Output" 2023-10-17 12:59:20,088 INFO scheduler.DAGScheduler: Job 15 finished: collect at /private/tmp/script-90e5371e.py:28, took 0,158257 s SQL result: ['{"name":"test1","key":123}', '{"name":"test2","key":456}'] ``` In order to see the logs above, make sure to enable `DEBUG=1` in the LocalStack container environment. Alternatively, you can also retrieve the job logs programmatically via the CloudWatch Logs API - for example, using the job run ID from the above command. ```bash lstk aws logs get-log-events \ --log-group-name /aws-glue/jobs/logs-v2 \ --log-stream-name ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for Glue. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **Glue** under the **Analytics** section. ![Glue Resource Browser](/images/aws/glue-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Manage Databases**: Create, view, and delete databases in your Glue catalog **Databases** tab. - **Manage Tables**: Create, view, edit, and delete tables in a database in your Glue catalog clicking on the **Tables** tab. - **Manage Connections**: Create, view, and delete Connections in your Glue catalog by clicking on the **Connections** tab. - **Manage Crawlers**: Create, view, and delete Crawlers in your Glue catalog by clicking on the **Crawlers** tab. - **Manage Jobs**: Create, view, and delete Jobs in your Glue catalog by clicking on the **Jobs** tab. - **Manage Schema Registries**: Create, view, and delete Schema Registries in your Glue catalog by clicking on the **Schema Registries** tab. - **Manage Schemas**: Create, view, and delete Schemas in your Glue catalog by clicking on the **Schemas** tab. ## Examples The following code snippets and sample applications provide practical examples of how to use Glue in LocalStack for various use cases: - [localstack-pro-samples/glue-etl-jobs](https://github.com/localstack/localstack-pro-samples/tree/master/glue-etl-jobs) - Simple demo application illustrating the use of the Glue API to run local ETL jobs using LocalStack. - [localstack-pro-samples/glue-redshift-crawler](https://github.com/localstack/localstack-pro-samples/tree/master/glue-redshift-crawler) - Simple demo application illustrating the use of AWS Glue Crawler to populate the Glue metastore from a Redshift database. ## Further Reading The AWS Glue API is a fairly comprehensive service - more details can be found in the official [AWS Glue Developer Guide](https://docs.aws.amazon.com/glue/latest/dg/what-is-glue.html). ## Current Limitations Support for triggers is currently limited - the basic API endpoints are implemented, but triggers are currently still under development (more details coming soon). ## API Coverage # Identity and Access Management (IAM) > Get started with AWS Identity and Access Management (IAM) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Identity and Access Management (IAM) is a web service provided by Amazon Web Services (AWS) that enables users to control access to AWS resources securely. IAM allows organizations to create and manage AWS users, groups, and roles, defining granular permissions to access specific AWS services and resources. By centralizing access control, administrators can enforce the principle of least privilege, ensuring users have only the necessary permissions for their tasks. LocalStack allows you to use the IAM APIs in your local environment to create and manage users, groups, and roles, granting permissions that adhere to the principle of least privilege. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of IAM's integration with LocalStack. The policy coverage is documented in the [IAM coverage documentation](/aws/developer-tools/security-testing/iam-coverage/). ## Getting started This guide is designed for users new to IAM and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how you can create a new user named `test`, create an access key pair for the user, and assert that the user is recognized after the access keys are configured in the environment. By default, in the absence of custom credentials configuration, all requests to LocalStack run under the administrative root user. Run the following command to use the [`GetCallerIdentity`](https://docs.aws.amazon.com/cli/latest/reference/sts/get-caller-identity.html) API to confirm that the request is running under the root user: ```bash lstk aws sts get-caller-identity ``` ```bash title="Output" { "UserId": "AKIAIOSFODNN7EXAMPLE", "Account": "000000000000", "Arn": "arn:aws:iam::000000000000:root" } ``` You can now create a new user named `test` using the [`CreateUser`](https://docs.aws.amazon.com/cli/latest/reference/iam/create-user.html) API. Run the following command: ```bash lstk aws iam create-user --user-name test ``` You can now create an access key pair for the user using the [`CreateAccessKey`](https://docs.aws.amazon.com/cli/latest/reference/iam/create-access-key.html) API. Run the following command: ```bash lstk aws iam create-access-key --user-name test ``` ```bash title="Output" { "AccessKey": { "UserName": "test", "AccessKeyId": "LKIAQAAAAAAAGFWKCM5F", "Status": "Active", "SecretAccessKey": "DUulXk2N2yD6rgoBBR9A/5iXa6dBcLyDknr925Q5", "CreateDate": "2023-07-25T09:36:51+00:00" } } ... ``` You can save the `AccessKeyId` and `SecretAccessKey` values, and export them in the environment to run commands under the `test` user. Run the following command: ```bash export AWS_ACCESS_KEY_ID=LKIAQAAAAAAAGFWKCM5F AWS_SECRET_ACCESS_KEY=DUulXk2N2yD6rgoBBR9A/5iXa6dBcLyDknr925Q5 lstk aws sts get-caller-identity ``` ```bash title="Output" { "UserId": "b2yxf5g824zklfx5ry8o", "Account": "000000000000", "Arn": "arn:aws:iam::000000000000:user/test" } ``` You can see that the request is now running under the `test` user. ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing IAM users, groups, and roles. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **IAM** under the **Security Identity Compliance** section. ![IAM Resource Browser](/images/aws/iam-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create User, Group, Role, and Policy**: Create a new IAM user, group, or role by clicking the top-level **Create** button and filling out the form. - **View User, Group, Role, and Policy Details**: Click on any of the listed resources to view its details by clicking on the desired User, Group, Role, or Policy. - **Edit User, Group, Role, and Policy Details**: Click on any listed resources to edit its details by clicking on the desired User, Group, Role, or Policy. - **Delete User, Group, Role, and Policy**: Select any listed resources to delete them by clicking the **Actions** button and selecting **Remove Selected**. ## Special Tools LocalStack provides various tools to help you generate, test, and enforce IAM policies more efficiently. - **IAM Policy Stream**: IAM Policy Stream provides a real-time view of API calls and the corresponding IAM policies they generate, simplifying permission management and ensuring correct permissions are assigned. Learn more in the [IAM Policy Stream documentation](/aws/developer-tools/security-testing/iam-policy-stream). - **IAM Policy Enforcement**: This configuration enforces IAM policies when interacting with local cloud APIs, simulating a real AWS environment. For additional information, refer to the [IAM Policy Enforcement documentation](/aws/developer-tools/security-testing/iam-policy-enforcement). - **Explainable IAM**: Explainable IAM logs outputs related to failed policy evaluations directly to LocalStack logs, aiding in the identification of necessary policies for successful requests. More details are available in the [Explainable IAM documentation](/aws/developer-tools/security-testing/explainable-iam). - **IAM Policy Simulator**: Test the effect of a principal's policies, including Service Control Policies, without making a real request against your resources. Learn more in the [IAM Policy Simulator documentation](/aws/developer-tools/security-testing/iam-policy-simulator). ## Examples The following code snippets and sample applications provide practical examples of how to use IAM in LocalStack for various use cases: - [Serverless Container-based APIs with Amazon ECS & API Gateway](https://github.com/localstack/serverless-api-ecs-apigateway-sample) - [Event-driven architecture with Amazon SNS FIFO, DynamoDB, Lambda, and S3](https://github.com/localstack/event-driven-architecture-with-amazon-sns-fifo) - [Full-Stack application with AWS Lambda, DynamoDB & S3 for shipment validation](https://github.com/localstack/shipment-list-demo) - [Enforcement of IAM policies when working with local cloud APIs](https://github.com/localstack/localstack-pro-samples/tree/master/iam-policy-enforcement) ## API Coverage # Identity Store > Get started with Identity Store on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Identity Store is a managed service that enables the creation and management of groups within your AWS environment. Groups are used to manage access to AWS resources, and Identity Store provides a central location to create and manage groups across your AWS accounts. LocalStack allows you to use the Identity Store APIs to create and manage groups in your local environment. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Identity Store integration with LocalStack. ## Getting started This guide is aimed at users who are familiar with the AWS CLI and [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. It will walk you through the basics of setting up and managing groups within the AWS Identity Store using LocalStack. Start your LocalStack container using your preferred method. This guide will demonstrate how to create a group within Identity Store, list all groups, and describe a specific group. ### Create a Group in Identity Store You can create a new group in the Identity Store using the [`CreateGroup`](https://docs.aws.amazon.com/singlesignon/latest/IdentityStoreAPIReference/API_CreateGroup.html) API. Execute the following command to create a group with an identity store ID of `testls`: ```bash lstk aws identitystore create-group --identity-store-id testls ``` ```bash title="Output" { "GroupId": "38cec731-de22-45bf-9af7-b74457bba884", "IdentityStoreId": "testls" } ``` Copy the `GroupId` value from the output, as it will be needed in subsequent steps. ### List all Groups in Identity Store After creating groups, you might want to list all groups within the Identity Store to manage or review them. Run the following command to list all groups using the [`ListGroups`](https://docs.aws.amazon.com/singlesignon/latest/IdentityStoreAPIReference/API_ListGroups.html) API: ```bash lstk aws identitystore list-groups --identity-store-id testls ``` ```bash title="Output" { "Groups": [ { "GroupId": "38cec731-de22-45bf-9af7-b74457bba884", "ExternalIds": [], "IdentityStoreId": "testls" } ] } ``` This command returns a list of all groups, including the group you created in the previous step. ### Describe a Group in Identity Store To view details about a specific group, use the [`DescribeGroup`](https://docs.aws.amazon.com/singlesignon/latest/IdentityStoreAPIReference/API_DescribeGroup.html) API. Run the following command to describe the group you created in the previous step: ```bash lstk aws describe-group --identity-store-id testls --group-id 38cec731-de22-45bf-9af7-b74457bba884 ``` ```bash title="Output" { "GroupId": "38cec731-de22-45bf-9af7-b74457bba884", "ExternalIds": [], "IdentityStoreId": "testls" } ``` This command provides detailed information about the specific group, including its ID and any external IDs associated with it. ## API Coverage # IoT > Get started with AWS IoT on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction AWS IoT provides cloud services to manage IoT devices and integrate them with other AWS services. LocalStack supports IoT Core, IoT Data, IoT Analytics. Common operations for creating and updating things, groups, policies, certificates and other entities are implemented with full CloudFormation support. The supported APIs are available on our [API Coverage section](#api-coverage). LocalStack ships a [Message Queuing Telemetry Transport (MQTT)](https://mqtt.org/) broker powered by [Eclipse Mosquitto](https://mosquitto.org/) which supports both pure MQTT and MQTT-over-WSS (WebSockets Secure) protocols. ## Getting Started This guide is for users that are new to IoT and assumes a basic knowledge of the AWS CLI and LocalStack [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start LocalStack using your preferred method. To retrieve the MQTT endpoint, use the [`DescribeEndpoint`](https://docs.aws.amazon.com/iot/latest/apireference/API_DescribeEndpoint.html) operation. ```bash lstk aws iot describe-endpoint ``` ```bash title="Output" { "endpointAddress": "000000000000.iot.eu-central-1.localhost.localstack.cloud:4510" } ``` :::tip LocalStack lazy-loads services by default. The MQTT broker may not be automatically available on a fresh launch of LocalStack. You can make a `DescribeEndpoint` call to start the broker and identify the port. ::: This endpoint can then be used with any MQTT client to publish and subscribe to topics. In this example, we will use the [Hive MQTT CLI](https://hivemq.github.io/mqtt-cli/docs/installation/). Run the following command to subscribe to an MQTT topic. ```bash mqtt subscribe \ --host 000000000000.iot.eu-central-1.localhost.localstack.cloud \ --port 4510 \ --topic climate ``` In a separate terminal session, publish a message to this topic. ```bash mqtt publish \ --host 000000000000.iot.eu-central-1.localhost.localstack.cloud \ --port 4510 \ --topic climate \ -m "temperature=30°C;humidity=60%" ``` This message will be pushed to all subscribers of this topic, including the one in the first terminal session. ## Authentication LocalStack IoT maintains its own root certificate authority which is regenerated at every run. The root CA certificate can be retrieved from [`http://localhost.localstack.cloud:4566/_aws/iot/LocalStackIoTRootCA.pem`](http://localhost.localstack.cloud:4566/_aws/iot/LocalStackIoTRootCA.pem). :::tip AWS provides its root CA certificate at [`https://www.amazontrust.com/repository/AmazonRootCA1.pem`](https://www.amazontrust.com/repository/AmazonRootCA1.pem). [This section](https://docs.aws.amazon.com/iot/latest/developerguide/server-authentication.html#server-authentication-certs) contains information about CA certificates. ::: When connecting to the endpoints, you will need to provide this root CA certificate for authentication. This is illustrated below with Python [AWS IoT SDK](https://docs.aws.amazon.com/iot/latest/developerguide/iot-sdks.html), ```py showshowLineNumbers import awscrt import boto3 from awsiot import mqtt_connection_builder region = 'eu-central-1' iot_client = boto3.client('iot', region=region) endpoint = aws_client.iot.describe_endpoint()["endpointAddress"] endpoint, port = endpoint.split(':') event_loop_group = io.EventLoopGroup(1) host_resolver = io.DefaultHostResolver(event_loop_group) client_bootstrap = io.ClientBootstrap(event_loop_group, host_resolver) credentials_provider = awscrt.auth.AwsCredentialsProvider.new_static( access_key_id='...', secret_access_key='...', ) client_id = 'example-client' # Path to root CA certificate downloaded from `/_aws/iot/LocalStackIoTRootCA.pem` ca_filepath = '...' mqtt_over_wss = mqtt_connection_builder.websockets_with_default_aws_signing( region=region, credentials_provider=credentials_provider, client_bootstrap=client_bootstrap, client_id=client_id, endpoint=endpoint, port=port, ca_filepath=ca_filepath, ) mqtt_over_wss.connect().result() mqtt_over_wss.subscribe(...) ``` If you are using pure MQTT, you also need to set the client-side X509 certificates and Application Layer Protocol Negotiation (ALPN) for a successful mutual TLS (mTLS) authentication. This is not required for MQTT-over-WSS since it does not use mTLS. AWS IoT SDKs automatically set the ALPN when the endpoint port is 443. However, because LocalStack does not use this port, this must be done manually. For details on how ALPN works with AWS, see [this page](https://docs.aws.amazon.com/iot/latest/developerguide/protocols.html). The client certificate and key can be retrieved using `CreateKeysAndCertificate` operation. The certificate is signed by the LocalStack root CA. ```py showshowLineNumbers result = iot_client.create_keys_and_certificate(setAsActive=True) # Path to file with saved content `result["certificatePem"]` cert_file = '...' # Path to file with saved content `result["keyPair"]["PrivateKey"]` priv_key_file = '...' tls_ctx_options = awscrt.io.TlsContextOptions.create_client_with_mtls_from_path( cert_file, priv_key_file ) tls_ctx_options.alpn_list = ["x-amzn-mqtt-ca"] mqtt = mqtt_connection_builder._builder( tls_ctx_options, cert_filepath=cert_file, pri_key_filepath=priv_key_file, client_bootstrap=client_bootstrap, client_id=client_id, endpoint=endpoint, port=port, ca_filepath=ca_filepath, ) mqtt.connect().result() mqtt.subscribe(...) ``` ## Lifecycle Events LocalStack publishes the [lifecycle events](https://docs.aws.amazon.com/iot/latest/developerguide/life-cycle-events.html) to the standard endpoints. - `$aws/events/presence/connected/clientId`: when a client connects - `$aws/events/presence/disconnected/clientId`: when a client disconnects - `$aws/events/subscriptions/subscribed/clientId`: when a client subscribes to a topic - `$aws/events/subscriptions/unsubscribed/clientId`: when a client unsubscribes from a topic Currently the `principalIdentifier` and `sessionIdentifier` fields in event payload contain dummy values. ## Registry Events LocalStack can publish the [registry events](https://docs.aws.amazon.com/iot/latest/developerguide/registry-events.html), if [you enable it](https://docs.aws.amazon.com/iot/latest/developerguide/iot-events.html#iot-events-enable). ```bash lstk aws iot update-event-configurations \ --event-configurations '{"THING":{"Enabled": true}}' ``` You can then subscribe or use topic rules on the follow topics: - `$aws/events/thing//created`: when a new thing is created - `$aws/events/thing//updated`: when a thing is updated - `$aws/events/thing//deleted`: when a thing is deleted ## Topic Rules It is possible to use actions with SQL queries for IoT Topic Rules. For example, you can use the [`CreateTopicRule`](https://docs.aws.amazon.com/iot/latest/apireference/API_CreateTopicRule.html) operation to define a topic rule with a SQL query `SELECT * FROM 'my/topic' where attr=123` which will execute a trigger whenever a message with attribute `attr=123` is received on the MQTT topic `my/topic`. The following actions are supported: - [Lambda](https://docs.aws.amazon.com/iot/latest/developerguide/lambda-rule-action.html) - [SQS](https://docs.aws.amazon.com/iot/latest/developerguide/sqs-rule-action.html) - [Kinesis](https://docs.aws.amazon.com/iot/latest/developerguide/kinesis-rule-action.html) - [Firehose](https://docs.aws.amazon.com/iot/latest/developerguide/kinesis-firehose-rule-action.html) - [DynamoDBv2](https://docs.aws.amazon.com/iot/latest/developerguide/dynamodb-v2-rule-action.html) - [HTTP](https://docs.aws.amazon.com/iot/latest/developerguide/https-rule-action.html) (URL confirmation and substitution templating is not implemented) ## Troubleshooting ### Node.js `aws-iot-device-sdk` Connection Issues When using the [`aws-iot-device-sdk`](https://github.com/aws/aws-iot-device-sdk-js) library, you may encounter SSL certificate errors because Node.js rejects self-signed certificates by default. **Solution:** Set the environment variable to disable certificate validation: ```bash export NODE_TLS_REJECT_UNAUTHORIZED=0 ``` **For Lambda functions**, you also need to explicitly set the `region` parameter in the device configuration: ```js const device = new iot.device({ protocol: 'wss', host: endpoint, region: process.env.AWS_REGION, // Required for LocalStack // ... other options }); ``` And configure the Lambda environment: ```bash lstk aws lambda update-function-configuration \ --function-name your-function-name \ --environment "Variables={NODE_TLS_REJECT_UNAUTHORIZED=0}" ``` :::caution Only use `NODE_TLS_REJECT_UNAUTHORIZED=0` in development environments. As an alternative, consider using standard MQTT libraries like [MQTT.js](https://github.com/mqttjs/MQTT.js). ::: ## API Coverage # IoT Data > Get started with IoT Data on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction IoT Data provides secure, bi-directional communication between Internet-connected things, such as sensors, actuators, embedded devices, or smart appliances, and the AWS Cloud. It allows you to connect your devices to the cloud and interact with them using the AWS Management Console, AWS CLI, or AWS SDKs. LocalStack allows you to use the IoT Data APIs to update, get, and delete the shadow of a thing in your local environment. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of IoT Data integration with LocalStack. ## Getting started This guide is designed for users new to IoT Data and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a thing, update its shadow, get its shadow, and delete its shadow using IoT Data. ### Update the shadow You can update the shadow of a thing using the [`UpdateThingShadow`](https://docs.aws.amazon.com/iot/latest/apireference/API_UpdateThingShadow.html) API. Run the following command to update the shadow of a thing named `MyRPi`: ```bash lstk aws iot-data update-thing-shadow \ --thing-name "MyRPi" \ --payload "{\"state\":{\"reported\":{\"moisture\":\"okay\"}}}" \ output.txt --cli-binary-format raw-in-base64-out ``` The `output.txt` file contains the following output: ```text title="output.txt" { "state": { "reported": { "moisture": "okay" } }, "metadata": { "reported": { "moisture": { "timestamp": 1724226109 } } }, "version": 1, "timestamp": 1724226109 } ``` ### Get the shadow You can get the shadow of a thing using the [`GetThingShadow`](https://docs.aws.amazon.com/iot/latest/apireference/API_GetThingShadow.html) API. Run the following command to get the shadow: ```bash lstk aws iot-data get-thing-shadow \ --thing-name "MyRPi" \ output.txt ``` The `output.txt` will contain the same output as the previous command. ### Delete the shadow You can delete the shadow of a thing using the [`DeleteThingShadow`](https://docs.aws.amazon.com/iot/latest/apireference/API_DeleteThingShadow.html) API. Run the following command to delete the shadow: ```bash lstk aws iot-data delete-thing-shadow \ --thing-name "MyRPi" \ output.txt ``` The `output.txt` will contain the following output: ```text title="output.txt" { "version": 1, "timestamp": 1724226371 } ``` ## Device Shadows LocalStack supports both unnamed (classic) and named device shadows. You can use AWS CLI and [MQTT topics](https://docs.aws.amazon.com/iot/latest/developerguide/device-shadow-mqtt.html) to get, update or delete device shadow state information. The endpoint as returned by `DescribeEndpoint` currently does not support the [device shadow REST API](https://docs.aws.amazon.com/iot/latest/developerguide/device-shadow-rest-api.html#API_GetThingShadow) ## API Coverage # IoT Wireless > Get started with IoT Wireless on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction AWS IoT Wireless is a managed service that enables customers to connect and manage wireless devices. The service provides a set of APIs to manage wireless devices, gateways, and destinations. LocalStack allows you to use the IoT Wireless APIs in your local environment from creating wireless devices and gateways. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of IoT Wireless's integration with LocalStack. ## Getting started This guide is designed for users new to IoT Wireless and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to use IoT Wireless to create wireless devices and gateways with the AWS CLI. ### Create a Wireless Device You can create a wireless device using the [`CreateWirelessDevice`](https://docs.aws.amazon.com/iot-wireless/2020-11-22/API_CreateWirelessDevice.html) API. Run the following command to create a wireless device: ```bash lstk aws iotwireless create-device-profile ``` The following output would be retrieved: ```bash title="Output" { "Id": "b8a8e3a8" } ``` You can list the device profiles using the [`ListDeviceProfiles`](https://docs.aws.amazon.com/iot-wireless/2020-11-22/API_ListDeviceProfiles.html) API. Run the following command to list the device profiles: ```bash lstk aws iotwireless list-device-profiles ``` The following output would be retrieved: ```bash title="Output" { "DeviceProfileList": [ { "Id": "b8a8e3a8" } ] } ``` ### Create a Wireless device You can create a wireless device using the [`CreateWirelessDevice`](https://docs.aws.amazon.com/iot-wireless/2020-11-22/API_CreateWirelessDevice.html) API. Run the following command to create a wireless device: ```bash lstk aws iotwireless create-wireless-device \ --cli-input-json file://input.json ``` The `input.json` file contains the following content: ```json title="input.json" showshowLineNumbers { "Description": "My LoRaWAN wireless device", "DestinationName": "IoTWirelessDestination", "LoRaWAN": { "DeviceProfileId": "ab0c23d3-b001-45ef-6a01-2bc3de4f5333", "ServiceProfileId": "fe98dc76-cd12-001e-2d34-5550432da100", "OtaaV1_1": { "AppKey": "3f4ca100e2fc675ea123f4eb12c4a012", "JoinEui": "b4c231a359bc2e3d", "NwkKey": "01c3f004a2d6efffe32c4eda14bcd2b4" }, "DevEui": "ac12efc654d23fc2" }, "Name": "SampleIoTWirelessThing", "Type": "LoRaWAN" } ``` You can list the wireless devices using the [`ListWirelessDevices`](https://docs.aws.amazon.com/iot-wireless/2020-11-22/API_ListWirelessDevices.html) API. Run the following command to list the wireless devices: ```bash lstk aws iotwireless list-wireless-devices ``` The following output would be retrieved: ```bash title="Output" { "WirelessDeviceList": [ { "Id": "0bca2fe2", "Type": "LoRaWAN", "Name": "SampleIoTWirelessThing", "DestinationName": "IoTWirelessDestination", "LoRaWAN": { "DevEui": "ac12efc654d23fc2" } } ] } ``` ### Create a Wireless Gateway You can create a wireless gateway using the [`CreateWirelessGateway`](https://docs.aws.amazon.com/iot-wireless/2020-11-22/API_CreateWirelessGateway.html) API. Run the following command to create a wireless gateway: ```bash lstk aws iotwireless create-wireless-gateway \ --lorawan GatewayEui="a1b2c3d4567890ab",RfRegion="US915" \ --name "myFirstLoRaWANGateway" \ --description "Using my first LoRaWAN gateway" ``` The following output would be retrieved: ```bash title="Output" { "Id": "e519dc4e" } ``` You can list the wireless gateways using the [`ListWirelessGateways`](https://docs.aws.amazon.com/iot-wireless/2020-11-22/API_ListWirelessGateways.html) API. Run the following command to list the wireless gateways: ```bash lstk aws iotwireless list-wireless-gateways ``` The following output would be retrieved: ```bash title="Output" { "WirelessGatewayList": [ { "Id": "e519dc4e", "Name": "myFirstLoRaWANGateway", "Description": "Using my first LoRaWAN gateway", "LoRaWAN": { "GatewayEui": "a1b2c3d4567890ab", "RfRegion": "US915" } } ] } ``` ## API Coverage # Managed Streaming for Kafka (MSK) > Get started with Managed Streaming for Kafka (MSK) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Managed Streaming for Apache Kafka (MSK) is a fully managed Apache Kafka service that allows you to build and run applications that process streaming data. MSK offers a centralized platform to facilitate seamless communication between various AWS services and applications through event-driven architectures, facilitating data ingestion, processing, and analytics for various applications. MSK also features automatic scaling and built-in monitoring, allowing users to build robust, high-throughput data pipelines. LocalStack allows you to use the MSK APIs in your local environment to spin up Kafka clusters on the local machine, create topics for exchanging messages, and define event source mappings that trigger Lambda functions when messages are received on a certain topic. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of MSK's integration with LocalStack. ## Getting started This guide is designed for users new to Managed Streaming for Kafka and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to configure an MSK Cluster locally, create a Kafka topic, and produce and consume messages. ### Create a local MSK Cluster To set up a local MSK (Managed Streaming for Apache Kafka) cluster, you can use the [`CreateCluster`](https://docs.aws.amazon.com/msk/1.0/apireference/clusters.html#CreateCluster) API to create a cluster named `EventsCluster` with three broker nodes. In this process, you'll need a JSON file named `brokernodegroupinfo.json` which specifies the three subnets where you want your local Amazon MSK to distribute the broker nodes. Create the file and add the following content to it: ```bash title="Output" { "InstanceType": "kafka.m5.xlarge", "BrokerAZDistribution": "DEFAULT", "ClientSubnets": [ "subnet-0123456789111abcd", "subnet-0123456789222abcd", "subnet-0123456789333abcd" ] } ``` Run the following command to create the cluster: ```bash lstk aws kafka create-cluster \ --cluster-name "EventsCluster" \ --broker-node-group-info file://brokernodegroupinfo.json \ --kafka-version "2.8.0" \ --number-of-broker-nodes 3 ``` ```bash title="Output" { "ClusterArn": "arn:aws:kafka:us-east-1:000000000000:cluster/EventsCluster/b154d18a-8ecb-4691-96b2-50348357fc2f-25", "ClusterName": "EventsCluster", "State": "CREATING" } ``` The cluster creation process might take a few minutes. You can describe the cluster using the [`DescribeCluster`](https://docs.aws.amazon.com/msk/1.0/apireference/clusters.html#DescribeCluster) API. Run the following command, replacing `ClusterArn` with the Amazon Resource Name (ARN) you obtained above when you created cluster. ```bash lstk aws kafka describe-cluster \ --cluster-arn "arn:aws:kafka:us-east-1:000000000000:cluster/EventsCluster/b154d18a-8ecb-4691-96b2-50348357fc2f-25" ``` ```bash title="Output" { "ClusterInfo": { "BrokerNodeGroupInfo": { "BrokerAZDistribution": "DEFAULT", "ClientSubnets": [ "subnet-01", "subnet-02", "subnet-03" ], "InstanceType": "kafka.m5.xlarge" }, "ClusterArn": "arn:aws:kafka:us-east-1:000000000000:cluster/EventsCluster/b154d18a-8ecb-4691-96b2-50348357fc2f-25", "ClusterName": "EventsCluster", "CreationTime": "2022-06-29T02:45:16.848000Z", "CurrentBrokerSoftwareInfo": { "KafkaVersion": "2.5.0" }, "CurrentVersion": "K5OWSPKW0IK7LM", "NumberOfBrokerNodes": 3, "State": "ACTIVE", "ZookeeperConnectString": "localhost:4510" } } ``` ### Create a Kafka topic To use LocalStack MSK, you can download and utilize the Kafka command line interface (CLI) to create a topic for producing and consuming data. To download Apache Kafka, execute the following commands. ```bash wget https://archive.apache.org/dist/kafka/2.8.0/kafka_2.12-2.8.0.tgz tar -xzf kafka_2.12-2.8.0.tgz ``` Navigate to the **kafka_2.12-2.8.0** directory. Execute the following command, replacing `ZookeeperConnectString` with the value you saved after running the [`DescribeCluster`](https://docs.aws.amazon.com/msk/1.0/apireference/clusters.html#DescribeCluster) API: ```bash bin/kafka-topics.sh \ --create \ --zookeeper localhost:4510 \ --replication-factor 1 \ --partitions 1 \ --topic LocalMSKTopic ``` ```bash title="Output" Created topic LocalMSKTopic. ``` ### Interacting with the topic You can now utilize the JVM truststore to establish communication with the MSK cluster. Create a folder named `/tmp` on the client machine, and navigate to the bin folder of the Apache Kafka installation. Run the following command, replacing `java_home` with the path of your `java_home`. For this instance, the java_home path is `/Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin/Contents/Home`. :::note The following step is optional and may not be required, depending on the operating system environment being used. ::: ```bash cp java_home/lib/security/cacerts /tmp/kafka.client.truststore.jks ``` While you are still in the `bin` folder of the Apache Kafka installation on the client machine, create a text file named `client.properties` with the following contents: ```txt ssl.truststore.location=/tmp/kafka.client.truststore.jks ``` Run the following command, replacing `ClusterArn` with the Amazon Resource Name (ARN) you have. ```bash lstk aws kafka get-bootstrap-brokers \ --cluster-arn ClusterArn ``` To proceed with the following commands, save the value associated with the string named `BootstrapBrokerStringTls` from the JSON result obtained from the previous command. It should look like this: ```bash { "BootstrapBrokerString": "localhost:4511" } ``` Now, navigate to the bin folder and run the next command, replacing `BootstrapBrokerStringTls` with the value you obtained: ```bash ./kafka-console-producer.sh \ --broker-list BootstrapBrokerStringTls \ --producer.config client.properties \ --topic LocalMSKTopic ``` To send messages to your Apache Kafka cluster, enter any desired message and press Enter. You can repeat this process twice or thrice, sending each line as a separate message to the Kafka cluster. Keep the connection to the client machine open, and open a separate connection to the same machine in a new window. In this new connection, navigate to the `bin` folder and run a command, replacing `BootstrapBrokerStringTls` with the value you saved earlier. This command will allow you to interact with the Apache Kafka cluster using the saved value for secure communication. ```bash ./kafka-console-consumer.sh \ --bootstrap-server BootstrapBrokerStringTls \ --consumer.config client.properties \ --topic LocalMSKTopic \ --from-beginning ``` You should start seeing the messages you entered earlier when you used the console producer command. These messages are TLS encrypted in transit. Enter more messages in the producer window, and watch them appear in the consumer window. ### Adding a local MSK trigger You can add a Lambda Event Source Mapping API to create a mapping between a Lambda function, named `my-kafka-function`, and a Kafka topic called `LocalMSKTopic`. The configuration for this mapping sets the starting position of the topic to `LATEST`. Run the following command to use the [`CreateEventSourceMapping`](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateEventSourceMapping.html) API by specifying the Event Source ARN, the topic name, the starting position, and the Lambda function name. ```bash lstk aws lambda create-event-source-mapping \ --event-source-arn arn:aws:kafka:us-east-1:000000000000:cluster/EventsCluster \ --topics LocalMSKTopic \ --starting-position LATEST \ --function-name my-kafka-function ``` Upon successful completion of the operation to create the Lambda Event Source Mapping, you can expect the following response: ```bash title="Output" { "UUID": "9c353a2b-bc1a-48b5-95a6-04baf67f01e4", "StartingPosition": "LATEST", "BatchSize": 100, "ParallelizationFactor": 1, "EventSourceArn": "arn:aws:kafka:us-east-1:000000000000:cluster/EventsCluster", "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:my-kafka-function", "LastModified": "2021-11-21T20:55:49.438914+01:00", "LastProcessingResult": "OK", "State": "Enabled", "StateTransitionReason": "User action", "Topics": [ "LocalMSKTopic" ] } ``` With the event source mapping feature, LocalStack offers an automated process for spawning Lambda functions whenever a message is published to the designated Kafka topic. You can use the `kafka-console-producer.sh` client script to publish messages to the topic. By doing so, you can closely monitor the execution of Lambda functions within Docker containers as new messages arrive by simply observing the LocalStack log output. ## Delete the local MSK cluster You can delete the local MSK cluster using the [`DeleteCluster`](https://docs.aws.amazon.com/cli/latest/reference/kafka/delete-cluster.html) API. To do so, you must first obtain the ARN of the cluster you want to delete. Run the following command to list all the clusters in the region: ```bash lstk aws kafka list-clusters --region us-east-1 ``` To initiate the deletion of a cluster, select the corresponding `ClusterARN` from the list of clusters, and then execute the following command: ```bash lstk aws kafka delete-cluster --cluster-arn ClusterArn ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing MSK clusters. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **Kafka** under the **Analytics** section. ![MSK Resource Browser](/images/aws/msk-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Cluster**: Create a new MSK cluster by clicking on the **Create Cluster** button and specifying the required parameters. - **View Cluster**: View the details of an existing MSK cluster by clicking on the cluster name. - **Edit Cluster**: Edit the configuration of an existing MSK cluster by clicking on the **Edit** button in the cluster details page. - **Delete Cluster**: Delete an existing MSK cluster by selecting the cluster name and clicking on the **Actions** dropdown menu, then selecting **Remove Selected**. ## API Coverage # Kinesis Data Streams > Get started with Kinesis Data Streams on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Kinesis Data Streams is an AWS service for ingesting, buffering, and processing data in high throughput data streams. It is used for applications that require real-time processing and deriving insights from data streams such as logs, metrics, user interactions, and sensor readings. LocalStack allows you to use the Kinesis Data Streams APIs in your local environment from setting up data streams and configuring data processing to building real-time applications. The supported APIs are available on our [API Coverage section](#api-coverage). Emulation for Kinesis is powered by [Kinesis Mock](https://github.com/etspaceman/kinesis-mock). ## Getting started This guide is designed for users new to Kinesis Data Streams and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a Lambda function to consume events from a Kinesis stream with the AWS CLI. ### Create a Lambda function You need to create a Lambda function that receives a Kinesis event input and processes the messages that it contains. Create a file named `index.mjs` with the following content: ```javascript showshowLineNumbers console.log('Loading function'); export const handler = (event, context) => { event.Records.forEach(record => { let payload = Buffer.from(record.kinesis.data, 'base64').toString('ascii'); console.log('Decoded payload:', payload); }); }; ``` You can create a Lambda function using the [`CreateFunction`](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunction.html) API. Run the following command to create a Lambda function named `ProcessKinesisRecords`: ```bash zip function.zip index.mjs lstk aws lambda create-function \ --function-name ProcessKinesisRecords \ --zip-file fileb://function.zip \ --handler index.handler \ --runtime nodejs18.x \ --role arn:aws:iam::000000000000:role/lambda-kinesis-role ``` ```bash title="Output" { "FunctionName": "ProcessKinesisRecords", "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:ProcessKinesisRecords", "Runtime": "nodejs18.x", "Role": "arn:aws:iam::000000000000:role/lambda-kinesis-role", "Handler": "index.handler", ... } ``` ### Invoke the Lambda function Create a file named `input.txt` with the following JSON content: ```text { "Records": [ { "kinesis": { "kinesisSchemaVersion": "1.0", "partitionKey": "1", "sequenceNumber": "49590338271490256608559692538361571095921575989136588898", "data": "SGVsbG8sIHRoaXMgaXMgYSB0ZXN0Lg==", "approximateArrivalTimestamp": 1545084650.987 }, "eventSource": "aws:kinesis", "eventVersion": "1.0", "eventID": "shardId-000000000006:49590338271490256608559692538361571095921575989136588898", "eventName": "aws:kinesis:record", "invokeIdentityArn": "arn:aws:iam::000000000000:role/lambda-kinesis-role", "awsRegion": "us-east-1", "eventSourceARN": "arn:aws:kinesis:us-east-1:000000000000:stream/lambda-stream" } ] } ``` The JSON contains a sample Kinesis event. You can use the [`Invoke`](https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html) API to invoke the Lambda function with the Kinesis event as input. Execute the following command: ```bash lstk aws lambda invoke \ --function-name ProcessKinesisRecords \ --payload file://input.txt outputfile.txt ``` ### Create a Kinesis Stream You can create a Kinesis Stream using the [`CreateStream`](https://docs.aws.amazon.com/kinesis/latest/APIReference/API_CreateStream.html) API. Run the following command to create a Kinesis Stream named `lambda-stream`: ```bash lstk aws kinesis create-stream \ --stream-name lambda-stream \ --shard-count 1 ``` You can retrieve the Stream ARN using the [`DescribeStream`](https://docs.aws.amazon.com/kinesis/latest/APIReference/API_DescribeStream.html) API. Execute the following command: ```bash lstk aws kinesis describe-stream \ --stream-name lambda-stream ``` ```bash title="Output" { "StreamDescription": { "Shards": [ { "ShardId": "shardId-000000000000", "HashKeyRange": { "StartingHashKey": "0", "EndingHashKey": "340282366920938463463374607431768211455" ... } ], "StreamARN": "arn:aws:kinesis:us-east-1:000000000000:stream/lambda-stream", "StreamName": "lambda-stream", "StreamStatus": "ACTIVE", ... } ``` You can save the `StreamARN` value for later use. ### Add an Event Source in Lambda You can add an Event Source to your Lambda function using the [`CreateEventSourceMapping`](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateEventSourceMapping.html) API. Run the following command to add the Kinesis Stream as an Event Source to your Lambda function: ```bash lstk aws lambda create-event-source-mapping \ --function-name ProcessKinesisRecords \ --event-source arn:aws:kinesis:us-east-1:000000000000:stream/lambda-stream \ --batch-size 100 \ --starting-position LATEST ``` ### Test the Event Source mapping You can test the event source mapping by adding a record to the Kinesis Stream using the [`PutRecord`](https://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecord.html) API. Run the following command to add a record to the Kinesis Stream: ```bash lstk aws kinesis put-record \ --stream-name lambda-stream \ --partition-key 1 \ --data "Hello, this is a test." ``` You can fetch the CloudWatch logs for your Lambda function reading records from the stream, using AWS CLI or LocalStack Resource Browser. ### Performance Tuning For high-volume workloads or large payloads, we recommend switching to the Scala engine via the `KINESIS_MOCK_PROVIDER_ENGINE=scala` flag, delivering up to 10x better performance compared to the default Node.js engine. Additionally, the following parameters can be tuned: - Increase `KINESIS_MOCK_MAXIMUM_HEAP_SIZE` beyond the default `512m` to reduce JVM memory pressure. - Increase `KINESIS_MOCK_INITIAL_HEAP_SIZE` beyond the default `256m` to pre-allocate more JVM heap memory. - Reduce `KINESIS_LATENCY` artificial response delays from the default `500` milliseconds (or disable entirely with `0`). Refer to our [Kinesis configuration documentation](https://docs.localstack.cloud/references/configuration/#kinesis) for more details on these parameters. :::note `KINESIS_MOCK_MAXIMUM_HEAP_SIZE` and `KINESIS_MOCK_INITIAL_HEAP_SIZE` are only applicable when using the Scala engine. Future versions of LocalStack will likely default to using the `scala` engine over the less-performant `node` version currently in use. ::: ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing Kinesis Streams & Kafka Clusters. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **Kinesis** under the **Analytics** section. ![Kinesis Resource Browser](/images/aws/kinesis-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Stream**: Create a Kinesis Stream by specifying the **Stream Name**, **Shard Count**, and **Stream Mode**. - **Create Cluster**: Create a Kafka Cluster by specifying the **Cluster Name**, **Kafka Version**, **Number Of Broker Nodes**, **Instance Type**, and more. - **View Streams & Clusters**: Click on any of the listed resources to view its details by clicking on the desired Stream & Cluster. - **Edit Streams & Clusters**: Click on any listed resources to edit its details by clicking on the desired Stream & Cluster. - **Delete Streams & Clusters**: Select any listed resources to delete them by clicking the **Actions** button and selecting **Remove Selected**. ## Examples The following code snippets and sample applications provide practical examples of how to use Kinesis in LocalStack for various use cases: - [Search application with Lambda, Kinesis, Firehose, ElasticSearch, S3](https://github.com/localstack/sample-fuzzy-movie-search-lambda-kinesis-elasticsearch) - [Streaming Data Pipeline with Kinesis, Tinybird, CloudWatch, Lambda](https://github.com/localstack/serverless-streaming-data-pipeline) ## Limitations In multi-account setups, each AWS account launches a separate instance of Kinesis Mock, which is very resource intensive when a large number of AWS accounts are used. [This Kinesis Mock issue](https://github.com/etspaceman/kinesis-mock/issues/377) is being used to keep track of this feature. ## API Coverage # Managed Service for Apache Flink > Get started with Managed Service for Apache Flink on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; :::note This service was formerly known as 'Kinesis Data Analytics for Apache Flink'. ::: ## Introduction [Apache Flink](https://flink.apache.org/) is a framework for building applications that process and analyze streaming data. [Managed Service for Apache Flink (MSF)](https://docs.aws.amazon.com/managed-flink/latest/java/what-is.html) is an AWS service that provides the underlying infrastructure and a hosted Apache Flink cluster that can run Apache Flink applications. LocalStack lets you to run Flink applications locally and implements several [AWS-compatible API operations](#api-coverage). A separate Apache Flink cluster is started in [application mode](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/deployment/overview/#application-mode) for every Managed Flink application created. Flink cluster deployment on LocalStack consists of two separate containers for [JobManager](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/concepts/flink-architecture/#jobmanager) and [TaskManager](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/concepts/flink-architecture/#taskmanagers). ## Deployment Considerations The default container runtime for MSF on LocalStack is Docker. LocalStack creates Flink JobManager and TaskManager containers on the Docker network specified by `MAIN_DOCKER_NETWORK` (defaults to `bridge`). **Running LocalStack inside a Kubernetes cluster with the Docker executor is not supported.** Mounting the host Docker socket (`/var/run/docker.sock`) into a LocalStack pod is not sufficient — the Docker executor cannot create Flink containers in this topology and applications will remain stuck in `STARTING` indefinitely. If you are running LocalStack outside of Kubernetes (for example, with Docker Compose or the `lstk` CLI), no additional configuration is required and the Docker executor is used automatically. ## Getting Started This guide builds a demo Flink application and deploys it to LocalStack. The application generates synthetic records, processes them and sends the output to an S3 bucket. Start the LocalStack container using your preferred method. ### Build Application Code Begin by cloning the AWS sample repository. We will use the [S3 Sink](https://github.com/localstack-samples/amazon-managed-service-for-apache-flink-examples/tree/main/java/S3Sink) application in this example. ```bash git clone https://github.com/localstack-samples/amazon-managed-service-for-apache-flink-examples.git cd java/S3Sink ``` Next, use [Maven](https://maven.apache.org/) to compile and package the Flink application into a jar. ```bash mvn package ``` The Flink application jar file will be placed in the `./target/flink-kds-s3.jar` directory. ### Upload Application Code MSF requires that all application code resides in S3. Create an S3 bucket and upload the compiled Flink application jar. ```bash lstk aws s3api create-bucket --bucket flink-bucket lstk aws s3api put-object --bucket flink-bucket --key job.jar --body ./target/flink-kds-s3.jar ``` ### Output Sink As mentioned earlier, this Flink application writes the output to an S3 bucket. Create the S3 bucket that will serve as the sink. ```bash lstk aws s3api create-bucket --bucket sink-bucket ``` ### Permissions MSF requires a service execution role which allows it to connect to other services. Without the proper permissions policy and role, this example application will not be able to connect to S3 sink bucket to output the result. Create an IAM role for the running MSF application to assume. ```json showshowLineNumbers # role.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": {"Service": "kinesisanalytics.amazonaws.com"}, "Action": "sts:AssumeRole" } ] } ``` ```bash lstk aws iam create-role --role-name msaf-role --assume-role-policy-document file://role.json ``` Next create add a permissions policy to this role that permits read and write access to S3. ```json showshowLineNumbers # policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": ["s3:GetObject", "s3:GetObjectVersion", "s3:PutObject"], "Resource": "*" } ] } ``` ```bash lstk aws iam put-role-policy --role-name msaf-role --policy-name msaf-policy --policy-document file://policy.json ``` Now, when the running MSF application assumes this role, it will have the necessary permissions to write to the S3 sink. ### Deploy Application With all prerequisite resources in place, the Flink application can now be created and started. ```bash showshowLineNumbers lstk aws kinesisanalyticsv2 create-application \ --application-name msaf-app \ --runtime-environment FLINK-1_20 \ --application-mode STREAMING \ --service-execution-role arn:aws:iam::000000000000:role/msaf-role \ --application-configuration '{ "ApplicationCodeConfiguration": { "CodeContent": { "S3ContentLocation": { "BucketARN": "arn:aws:s3:::flink-bucket", "FileKey": "job.jar" } }, "CodeContentType": "ZIPFILE" }, "EnvironmentProperties": { "PropertyGroups": [{ "PropertyGroupId": "bucket", "PropertyMap": {"name": "sink-bucket"} }] } }' lstk aws kinesisanalyticsv2 start-application --application-name msaf-app ``` Once the Flink cluster is up and running, the application will stream the results to the sink S3 bucket. You can verify this with: ```bash lstk aws s3api list-objects --bucket sink-bucket ``` ## CloudWatch Logging LocalStack MSF supports [CloudWatch Logs integration](https://docs.aws.amazon.com/managed-flink/latest/java/cloudwatch-logs.html) to help monitor the Flink cluster for application events or configuration problems. The logging option can be added at the time of creating the Flink application using the [CreateApplication](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_CreateApplication.html) operation. Logging options can also be managed at a later point using the [AddApplicationCloudWatchLoggingOption](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_AddApplicationCloudWatchLoggingOption.html) and [DeleteApplicationCloudWatchLoggingOption](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_DeleteApplicationCloudWatchLoggingOption.html) operations. There are following prerequisites for CloudWatch Logs integration: - You must create the application's log group and log stream. Flink will not create it for you. - You must add the permissions your application needs to write to the log stream to the service execution role. Generally the following IAM actions are sufficient: `logs:DescribeLogGroups`, `logs:DescribeLogStreams` and `logs:PutLogEvents` To add a logging option: ```bash showshowLineNumbers lstk aws kinesisanalyticsv2 add-application-cloud-watch-logging-option \ --application-name msaf-app \ --cloud-watch-logging-option '{"LogStreamARN": "arn:aws:logs:us-east-1:000000000000:log-group:msaf-log-group:log-stream:msaf-log-stream"}' ``` ```bash title="Output" { "ApplicationARN": "arn:aws:kinesisanalytics:us-east-1:000000000000:application/msaf-app", "ApplicationVersionId": 2, "CloudWatchLoggingOptionDescriptions": [ { "CloudWatchLoggingOptionId": "1.1", "LogStreamARN": "arn:aws:logs:us-east-1:000000000000:log-group:msaf-log-group:log-stream:msaf-log-stream" } ] } ``` :::note Enabling CloudWatch Logs integration has a significant performance hit. ::: Configured logging options can be retrieved using [DescribeApplication](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_DescribeApplication.html): ```bash lstk aws kinesisanalyticsv2 describe-application --application-name msaf-app | jq .ApplicationDetail.CloudWatchLoggingOptionDescriptions ``` ```bash title="Output" [ { "CloudWatchLoggingOptionId": "1.1", "LogStreamARN": "arn:aws:logs:us-east-1:000000000000:log-group:msaf-log-group:log-stream:msaf-log-stream" } ] ``` :::note Logs events are reported to CloudWatch every 10 seconds. ::: Log events can be retrieved from CloudWatch Logs using the appropriate operation. To retrieve all events: ```bash lstk aws logs get-log-events --log-group-name msaf-log-group --log-stream-name msaf-log-stream ``` LocalStack reports both Flink application and Flink framework logs to CloudWatch. However, certain extended information such as stack traces may be missing. You may obtain this information by execing into the Flink Docker container created by LocalStack and inspecting `/opt/flink/log`. ## Resource Tagging You can manage [resource tags](https://docs.aws.amazon.com/managed-flink/latest/java/how-tagging.html) using [TagResource](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_TagResource.html), [UntagResource](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_UntagResource.html) and [ListTagsForResource](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_ListTagsForResource.html). Tags can also be specified when creating the Flink application using the [CreateApplication](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_CreateApplication.html) operation. ```bash lstk aws kinesisanalyticsv2 tag-resource \ --resource-arn arn:aws:kinesisanalytics:us-east-1:000000000000:application/msaf-app \ --tags Key=country,Value=SE lstk aws kinesisanalyticsv2 list-tags-for-resource \ --resource-arn arn:aws:kinesisanalytics:us-east-1:000000000000:application/msaf-app ``` ```bash title="Output" { "Tags": [ { "Key": "country", "Value": "SE" } ] } ``` You can also untag the resource: ```bash lstk aws kinesisanalyticsv2 untag-resource \ --resource-arn arn:aws:kinesisanalytics:us-east-1:000000000000:application/msaf-app \ --tag-keys country ``` ## Supported Flink Versions | Flink version | Supported by LocalStack | Supported by Apache | |:---:|:---:|:---:| | 1.20.0 | yes | yes | | 1.19.1 | yes | yes | | 1.18.1 | yes | yes | | 1.15.2 | yes | no | ## Troubleshooting ### Application stuck in `STARTING` If `describe-application` returns `STARTING` indefinitely and no Flink containers appear, the most likely cause is that LocalStack cannot reach the Docker daemon or Kubernetes API to create the Flink JobManager and TaskManager. When this occurs, LocalStack logs an internal error after `CLUSTER_READY_WAIT_TIMEOUT` elapses: ``` Exception: Error submitting job: Flink cluster is not running ``` The application does not automatically transition to `FAILED` — it remains in `STARTING`. This means IaC tools that use state waiters (such as the Terraform AWS provider or Crossplane) will block until their own timeout expires. **Common causes and fixes:** - **Running LocalStack inside Kubernetes with the Docker executor** — the Docker executor is not supported in this topology. See [Deployment Considerations](#deployment-considerations) for details. - **Wrong or missing Docker network** — if LocalStack is running in a non-default Docker network, set `MAIN_DOCKER_NETWORK` to the name of that network so Flink containers are attached to the correct network. - **Docker socket not accessible** — confirm that the Docker socket is mounted and functional. You can verify by listing containers from inside the LocalStack container: `curl --unix-socket /var/run/docker.sock http://localhost/containers/json`. ## Limitations - Application versions are not maintained - Only S3 zipfile code is supported - Values of 20,000 ms for `execution.checkpointing.interval` and 5,000 ms for `execution.checkpointing.min-pause` are used for checkpointing. They can not be overridden. - In-place [version upgrades](https://docs.aws.amazon.com/managed-flink/latest/java/how-in-place-version-upgrades.html) and [roll-backs](https://docs.aws.amazon.com/managed-flink/latest/java/how-system-rollbacks.html) are not supported - [Snapshot/savepoint management](https://docs.aws.amazon.com/managed-flink/latest/java/how-snapshots.html) is not implemented - CloudTrail integration and CloudWatch metrics is not implemented. The application logging level defaults to `INFO` and can not be overridden. - Parallelism is limited to the default value of 1, with one TaskManager that has one [Task Slot](https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/concepts/flink-architecture/#task-slots-and-resources) allocated. [Parallelism configuration](https://docs.aws.amazon.com/managed-flink/latest/apiv2/API_FlinkApplicationConfiguration.html#APIReference-Type-FlinkApplicationConfiguration-ParallelismConfiguration) provided on Flink application creation or update is ignored. - When a Flink cluster fails to start, the application remains in `STARTING` rather than transitioning to `FAILED`. Check LocalStack logs and see [Troubleshooting](#troubleshooting) for guidance. - The Docker executor is not supported when LocalStack runs as a pod inside a Kubernetes cluster. ## API Coverage # Key Management Service (KMS) > Get started with Key Management Service (KMS) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Key Management Service (KMS) is a managed service that allows users to handle encryption keys within the Amazon Web Services ecosystem. KMS allows users to create, control, and utilize keys to encrypt and decrypt data, as well as to sign and verify messages. KMS allows you to create, delete, list, and update aliases, friendly names for your KMS keys, and tag them for identification and automation. You can check [the official AWS documentation](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html) to understand the basic terms and concepts used in the KMS. LocalStack allows you to use the KMS APIs in your local environment to create, edit, and view symmetric and asymmetric KMS keys, including HMAC keys. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of KMS's integration with LocalStack. ## Getting started This guide is designed for users new to KMS and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a simple symmetric encryption key and use it to encrypt/decrypt data. ### Create a key To generate a new key within the KMS, you can use the [`CreateKey`](https://docs.aws.amazon.com/kms/latest/APIReference/API_CreateKey.html) API. Execute the following command to create a new key: ```bash lstk aws kms create-key ``` By default, this command generates a symmetric encryption key, eliminating the need for any additional arguments. You can take a look at the `KeyId` of the freshly generated key in the output, and save it for future use. In case the key ID is misplaced, it is possible to retrieve a comprehensive list of IDs and [Amazon Resource Names](https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html) (ARNs) for all available keys through the following command: ```bash lstk aws kms list-keys ``` Additionally, if needed, you can obtain extensive details about a specific key by providing its key ID or ARN using the subsequent command: ```bash lstk aws kms describe-key --key-id ``` ### Encrypt the data You can now leverage the generated key for encryption purposes. For instance, let's consider encrypting "_some important stuff_". To do so, you can use the [`Encrypt`](https://docs.aws.amazon.com/kms/latest/APIReference/API_Encrypt.html) API. Execute the following command to encrypt the data: ```bash lstk aws kms encrypt \ --key-id 010a4301-4205-4df8-ae52-4c2895d47326 \ --plaintext "some important stuff" \ --output text \ --query CiphertextBlob \ | base64 --decode > my_encrypted_data ``` You will notice that a new file named `my_encrypted_data` has been created in your current directory. This file contains the encrypted data, which can be decrypted using the same key. ### Decrypt the data To decrypt the data, you can use the [`Decrypt`](https://docs.aws.amazon.com/kms/latest/APIReference/API_Decrypt.html) API. You don't need to specify the `KEY_ID` while decrypting the file, since AWS includes the Key ID into the encrypted data. However, with asymmetric keys the `KEY_ID` has to be specified. Execute the following command to decrypt the data: ```bash lstk aws kms decrypt \ --ciphertext-blob fileb://my_encrypted_data \ --output text \ --query Plaintext \ | base64 --decode ``` Similar to the previous `Encrypt` operation, to retrieve the actual data, it's necessary to decode the Base64-encoded output. To achieve this, employ the `output` and `query` parameters along with the `base64` tool as before. Upon successful execution, the output will correspond to our original text: ```sh some important stuff ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing KMS keys. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **KMS** under the **Security Identity Compliance** section. ![KMS Resource Browser](/images/aws/kms-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Key**: Create a new KMS key by specifying the **Policy**, **Key Usage**, **Tags**, **Multi Region**, **Customer Master Key Spec**, and more. - **Edit Key**: Edit an existing KMS key by specifying the **Description**, after clicking the key in the list and clicking **EDIT KEY**. - **View Key**: View the details of an existing KMS key by clicking the key in the list. - **Enable & Disable Key**: Select any listed keys to enable or disable them by clicking the **Actions** button and select **Enable Selected** or **Disable Selected**. - **Delete Key**: Select any listed keys to delete them by clicking the **Actions** button and selecting **Schedule Deletion**. ## Custom IDs for KMS keys via tags You can assign custom IDs to KMS keys using the `_custom_id_` tag during key creation. This can be useful to pre-seed a test environment and use a static `KeyId` for your keys. Below is a simple example to create a key with a custom `KeyId` (note that the `KeyId` should have the format of a UUID): ```bash lstk aws kms create-key --tags '[{"TagKey":"_custom_id_","TagValue":"00000000-0000-0000-0000-000000000001"}]' ``` The following output will be displayed: ```bash title="Output" { "KeyMetadata": { "AWSAccountId": "000000000000", "KeyId": "00000000-0000-0000-0000-000000000001", .... } ``` ## Custom Key Material for KMS Keys via Tags You can seed a KMS key with custom key material using the `_custom_key_material_` tag during creation. This can be useful to pre-seed a development environment so values encrypted with KMS can be decrypted later. Here is an example of using custom key material with the value being base64 encoded: ```bash echo 'dGhpc2lzYXNlY3VyZWtleQ==' | base64 -d ``` The following output will be displayed: ```bash title="Output" thisisasecurekey ``` You can create a key with custom key material using the following command: ```bash lstk aws kms create-key --tags '[{"TagKey":"_custom_key_material_","TagValue":"dGhpc2lzYXNlY3VyZWtleQ=="}]' ``` The following output will be displayed: ```bash title="Output" { "KeyMetadata": { "AWSAccountId": "000000000000", "KeyId": "00000000-0000-0000-0000-000000000001", .... } ``` ## Current Limitations ### Encryption data format In LocalStack's KMS implementation, the encryption process is uniformly symmetric, even when an asymmetric key is requested. Furthermore, LocalStack utilizes an encrypted data format distinct from that employed by AWS. This could lead to decryption failures if a key is manually generated outside the local KMS environment, imported to KMS using the [ImportKeyMaterial](https://docs.aws.amazon.com/kms/latest/APIReference/API_ImportKeyMaterial.html) API, utilized for encryption within local KMS, and later decryption is attempted externally using the self-generated key. However, conventional setups are likely to function seamlessly. ### Key states In AWS KMS, cryptographic keys exhibit [multiple states](https://docs.aws.amazon.com/kms/latest/developerguide/key-state.html). However, LocalStack's KMS implementation provides only a subset of these states - `Enabled` - `Disabled` - `Creating` - `PendingImport` - `PendingDeletion` ### Multi-region keys LocalStack's KMS implementation is equipped to facilitate [multi-region keys](https://docs.aws.amazon.com/kms/latest/developerguide/multi-region-keys-overview.html), but there's a distinct behavior compared to AWS KMS. Unlike AWS KMS, the replication of multi-region key replicas in LocalStack KMS isn't automatically synchronized with their corresponding primary key. Consequently, adjustments made to the primary key's settings won't propagate automatically to the replica. ### Key aliases While AWS KMS conveniently establishes [aliases](https://docs.aws.amazon.com/kms/latest/developerguide/kms-alias.html), LocalStack follows suit by supporting these pre-configured aliases. However, it's important to note that in LocalStack, these aliases come into picture after the initial access attempt. Until that point, they are not visible. ### Key specs In AWS KMS, [SM2](https://docs.aws.amazon.com/kms/latest/developerguide/asymmetric-key-specs.html#key-spec-sm:~:text=the%20message%20digest.-,SM2%20key%20spec%20(China%20Regions%20only),-The%20SM2%20key) is a supported key spec for asymmetric keys. However, LocalStack's KMS implementation doesn't support this key spec. ## API Coverage # Lake Formation > Get started with Lake Formation on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Lake Formation is a managed service that allows users to build, secure, and manage data lakes. Lake Formation allows users to define and enforce fine-grained access controls, manage metadata, and discover and share data across multiple data sources. LocalStack allows you to use the Lake Formation APIs in your local environment to register resources, grant permissions, and list resources and permissions. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Lake Formation's integration with LocalStack. ## Getting started This guide is designed for users new to Lake Formation and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to register an S3 bucket as a resource in Lake Formation, grant permissions to a user, and list the resources and permissions. ### Register the resource Create a new S3 bucket named `test-bucket` using the `mb` command: ```bash lstk aws s3 mb s3://test-bucket ``` You can now register the S3 bucket as a resource in Lake Formation using the [`RegisterResource`](https://docs.aws.amazon.com/lake-formation/latest/dg/API_RegisterResource.html) API. Create a file named `input.json` with the following content: ```json { "ResourceArn": "arn:aws:s3:::test-bucket", "UseServiceLinkedRole": true } ``` Run the following command to register the resource: ```bash lstk aws lakeformation register-resource \ --cli-input-json file://input.json ``` ### List resources You can list the registered resources using the [`ListResources`](https://docs.aws.amazon.com/lake-formation/latest/dg/API_ListResources.html) API. Execute the following command to list the resources: ```bash lstk aws lakeformation list-resources ``` ```bash title="Output" { "ResourceInfoList": [ { "ResourceArn": "arn:aws:s3:::test-bucket", "LastModified": "2024-07-11T23:27:30.699312+05:30" } ] } ``` ### Grant permissions You can grant permissions to a user or group using the [`GrantPermissions`](https://docs.aws.amazon.com/lake-formation/latest/dg/API_GrantPermissions.html) API. Create a file named `permissions.json` with the following content: ```json showshowLineNumbers { "CatalogId": "000000000000", "Principal": { "DataLakePrincipalIdentifier": "arn:aws:iam::000000000000:user/lf-developer" }, "Resource": { "Table": { "CatalogId": "000000000000", "DatabaseName": "tpc", "TableWildcard": {} } }, "Permissions": [ "SELECT" ], "PermissionsWithGrantOption": [] } ``` Run the following command to grant permissions: ```bash lstk aws lakeformation grant-permissions \ --cli-input-json file://check.json ``` ### List permissions You can list the permissions granted to a user or group using the [`ListPermissions`](https://docs.aws.amazon.com/lake-formation/latest/dg/API_ListPermissions.html) API. Execute the following command to list the permissions: ```bash lstk aws lakeformation list-permissions ``` ## API Coverage # Lambda > Get started with Lambda on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; import { Badge } from '@astrojs/starlight/components'; import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction AWS Lambda is a Serverless Function as a Service (FaaS) platform that lets you run code in your preferred programming language on the AWS ecosystem. AWS Lambda automatically scales your code to meet demand and handles server provisioning, management, and maintenance. AWS Lambda allows you to break down your application into smaller, independent functions that integrate seamlessly with AWS services. LocalStack allows you to use the Lambda APIs to create, deploy, and test your Lambda functions. The supported APIs are available on our [API coverage section](#api-coverage), which provides information on the extent of Lambda's integration with LocalStack. ## Getting started This guide is designed for users new to Lambda and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a Lambda function with a Function URL. With the Function URL property, you can call a Lambda Function via an HTTP API call. ### Create a Lambda function To create a new Lambda function, create a new file called `index.js` with the following code: ```javascript showshowLineNumbers exports.handler = async (event) => { let body = JSON.parse(event.body) const product = body.num1 * body.num2; const response = { statusCode: 200, body: "The product of " + body.num1 + " and " + body.num2 + " is " + product, }; return response; }; ``` Enter the following command to create a new Lambda function: ```bash zip function.zip index.js lstk aws lambda create-function \ --function-name localstack-lambda-url-example \ --runtime nodejs22.x \ --zip-file fileb://function.zip \ --handler index.handler \ --role arn:aws:iam::000000000000:role/lambda-role ``` :::note To create a predictable URL for the function, you can assign a custom ID by specifying the `_custom_id_` tag on the function itself. ```bash lstk aws lambda create-function \ --function-name localstack-lambda-url-example \ --runtime nodejs22.x \ --zip-file fileb://function.zip \ --handler index.handler \ --role arn:aws:iam::000000000000:role/lambda-role \ --tags '{"_custom_id_":"my-custom-subdomain"}' ``` You must specify the `_custom_id_` tag **before** creating a Function URL. After the URL configuration is set up, any modifications to the tag will not affect it. LocalStack supports assigning custom IDs to both the `$LATEST` version of the function or to an existing version alias. ::: :::note In the old Lambda provider, you could create a function with any arbitrary string as the role, such as `r1`. However, the new provider requires the role ARN to be in the format `arn:aws:iam::000000000000:role/lambda-role` and validates it using an appropriate regex. However, it currently does not check whether the role exists. ::: ### Invoke the Function To invoke the Lambda function, you can use the [`Invoke` API](https://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html). Run the following command to invoke the function: ```bash lstk aws lambda invoke --function-name localstack-lambda-url-example \ --payload '{"body": "{\"num1\": \"10\", \"num2\": \"10\"}" }' output.txt ``` ```bash lstk aws lambda invoke --function-name localstack-lambda-url-example \ --cli-binary-format raw-in-base64-out \ --payload '{"body": "{\"num1\": \"10\", \"num2\": \"10\"}" }' output.txt ``` ### Create a Function URL :::note [Response streaming](https://docs.aws.amazon.com/lambda/latest/dg/configuration-response-streaming.html) is currently not supported, so it will still return a synchronous/full response instead. ::: With the Function URL property, there is now a new way to call a Lambda Function via HTTP API call using the [`CreateFunctionURLConfig` API](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunctionUrlConfig.html). To create a URL for invoking the function, run the following command: ```bash lstk aws lambda create-function-url-config \ --function-name localstack-lambda-url-example \ --auth-type NONE ``` This will generate a HTTP URL that can be used to invoke the Lambda function. The URL will be in the format `http://.lambda-url.us-east-1.localhost.localstack.cloud:4566`. :::note As previously mentioned, when a Lambda Function has a `_custom_id_` tag, LocalStack sets this tag's value as the subdomain in the Function's URL. ```bash lstk aws lambda create-function-url-config \ --function-name localstack-lambda-url-example \ --auth-type NONE ``` ```bash title="Output" { "FunctionUrl": "http://my-custom-subdomain.lambda-url....", .... } ``` In addition, if you pass an existing version alias as a `Qualifier` to the request, the created URL will combine the custom ID and the alias in the form `-`. ```bash lstk aws lambda create-function-url-config \ --function-name localstack-lambda-url-example \ --auth-type NONE --qualifier test-alias ``` ```bash title="Output" { "FunctionUrl": "http://my-custom-subdomain-test-alias.lambda-url....", .... } ``` ::: ### Trigger the Lambda function URL You can now trigger the Lambda function by sending a HTTP POST request to the URL using [curl](https://curl.se/) or your REST HTTP client: ```bash curl -X POST \ 'http://.lambda-url.us-east-1.localhost.localstack.cloud:4566/' \ -H 'Content-Type: application/json' \ -d '{"num1": "10", "num2": "10"}' ``` ```bash title="Output" The product of 10 and 10 is 100% ``` ## Lambda Event Source Mappings [Lambda event source mappings](https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventsourcemapping.html) allows you to connect Lambda functions to other AWS services. The following event sources are supported in LocalStack: - [Simple Queue Service (SQS)](https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html) - [DynamoDB](https://docs.aws.amazon.com/lambda/latest/dg/with-ddb.html) - [Kinesis](https://docs.aws.amazon.com/lambda/latest/dg/with-kinesis.html) - [Managed Streaming for Apache Kafka (MSK)](https://docs.aws.amazon.com/lambda/latest/dg/with-msk.html) ⭐️ - [Self-Managed Apache Kafka](https://docs.aws.amazon.com/lambda/latest/dg/with-kafka.html) ⭐️ ### Behaviour Coverage The table below shows feature coverage for all supported event sources for the latest version of LocalStack. Unlike [API operation coverage](#api-coverage), this table illustrates the **functional and behavioural coverage** of LocalStack's Lambda Event Source Mapping implementation. Where necessary, footnotes are used to provide additional context. :::note Feature availability and coverage is categorized with the following system: - ⭐️ Only Available in LocalStack licensed editions - 🟢 Fully Implemented - 🟡 Partially Implemented - 🟠 Not Implemented - ➖ Not Applicable (Not Supported by AWS) ::: import { Table, TableHeader, TableBody, TableHead, TableRow, TableCell } from '@/components/ui/table'; Parameter Description SQS Stream Kafka ⭐️ Standard FIFO Kinesis DynamoDB Amazon MSK Self-Managed BatchSize Batching events by count. 🟢 🟢 🟢 🟢 🟢 🟢 Not Configurable Batch when ≥ 6 MB limit. 🟠 🟠 🟠 🟠 🟢 🟢 MaximumBatchingWindowInSeconds Batch by Time Window. 🟢 🟢 🟢 🟢 🟢 🟢 MaximumRetryAttempts Discard after N retries. 🟢 🟢 MaximumRecordAgeInSeconds Discard records older than time `t`. 🟢 🟢 Enabled Enabling/Disabling. 🟢 🟢 🟢 🟢 🟢 🟢 FilterCriteria Filter pattern evaluating. [^1] [^2] 🟢 🟢 🟢 🟢 🟢 🟢 FunctionResponseTypes Enabling ReportBatchItemFailures. 🟢 🟢 🟢 🟢 🟠 🟠 BisectBatchOnFunctionError Bisect a batch on error and retry. 🟠 🟠 ScalingConfig The scaling configuration for the event source. [^3] 🟢 🟡 ParallelizationFactor Parallel batch processing by shard. 🟠 🟠 DestinationConfig.OnFailure SQS Failure Destination. 🟢 🟢 🟠 🟠 SNS Failure Destination. 🟢 🟢 🟠 🟠 S3 Failure Destination. 🟢 🟢 🟠 🟠 DestinationConfig.OnSuccess Success Destinations. MetricsConfig CloudWatch metrics. 🟠 🟠 🟠 🟠 🟠 🟠 ProvisionedPollerConfig Control throughput via min-max limits. [^4] 🟡 🟡 🟠 🟠 StartingPosition Position to start reading from. 🟢 🟢 🟢 🟢 StartingPositionTimestamp Timestamp to start reading from. 🟢 🟢 🟢 TumblingWindowInSeconds Duration (seconds) of a processing window. 🟠 🟠 Topics ⭐️ Kafka topics to read from. 🟢 🟢
[^1]: Read more at [Control which events Lambda sends to your function](https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventfiltering.html) [^2]: The available Metadata properties may not have full parity with AWS depending on the event source (read more at [Understanding event filtering basics](https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventfiltering.html#filtering-basics)). [^3]: For SQS Standard, LocalStack spawns multiple concurrent pollers per event source mapping (`LAMBDA_EVENT_SOURCE_MAPPING_SQS_POLLER_COUNT`, default `5`), and `ScalingConfig.MaximumConcurrency` caps the poller count when set. For SQS FIFO, a single poller is always used to preserve message-group ordering, so the scaling configuration is accepted but has no effect. [^4]: For SQS Standard, only `MinimumPollers` is honored (it sets the poller count); `MaximumPollers` has no effect. For SQS FIFO, the single-poller model means the configuration is accepted but ignored. Create a [GitHub Discussion](https://github.com/orgs/localstack/discussions/new/choose) or reach out to [LocalStack Support](/aws/help-support/get-help/) if you experience any challenges. ## Lambda Layers [Lambda layers](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html) let you include additional code and dependencies in your Lambda functions. With a valid LocalStack license, you can deploy Lambda Layers locally to streamline your development and testing process. ### Creating and using a Lambda Layer Locally To create a Lambda Layer locally, you can use the [`PublishLayerVersion` API](https://docs.aws.amazon.com/lambda/latest/dg/API_PublishLayerVersion.html) in LocalStack. Here's a simple example using Python: ```bash mkdir -p /tmp/python/ echo 'def util():' > /tmp/python/testlayer.py echo ' print("Output from Lambda layer util function")' >> /tmp/python/testlayer.py (cd /tmp; zip -r testlayer.zip python) LAYER_ARN=$(lstk aws lambda publish-layer-version --layer-name layer1 --zip-file fileb:///tmp/testlayer.zip | jq -r .LayerVersionArn) ``` Next, define a Lambda function that uses our layer: ```bash echo 'def handler(*args, **kwargs):' > /tmp/testlambda.py echo ' import testlayer; testlayer.util()' >> /tmp/testlambda.py echo ' print("Debug output from Lambda function")' >> /tmp/testlambda.py (cd /tmp; zip testlambda.zip testlambda.py) lstk aws lambda create-function \ --function-name func1 \ --runtime python3.8 \ --role arn:aws:iam::000000000000:role/lambda-role \ --handler testlambda.handler \ --timeout 30 \ --zip-file fileb:///tmp/testlambda.zip \ --layers $LAYER_ARN ``` Here, we've defined a Lambda function called `handler()` that imports the `util()` function from our `layer1` Lambda Layer. We then used the [`CreateFunction` API](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunction.html) to create this Lambda function in LocalStack, specifying the `layer1` Lambda Layer as a dependency. To test our Lambda function and see the output from the Lambda Layer, we can invoke the function and check the logs (with `DEBUG=1` enabled). Here's an example: ```bash title="Output" > START RequestId: a8bc4ce6-e2e8-189e-cf58-c2eb72827c23 Version: $LATEST > Output from Lambda layer util function > Debug output from Lambda function > END RequestId: a8bc4ce6-e2e8-189e-cf58-c2eb72827c23 ``` ### Referencing Lambda layers from AWS If your Lambda function references a layer in real AWS, you can integrate it into your local dev environment by making it accessible to the `886468871268` AWS account ID. This account is managed by LocalStack on AWS. To grant access to your layer, run the following command: ```bash aws lambda add-layer-version-permission \ --layer-name test-layer \ --version-number 1 \ --statement-id layerAccessFromLocalStack \ --principal 886468871268 \ --action lambda:GetLayerVersion ``` Replace `test-layer` and `1` with the name and version number of your layer, respectively. After granting access, the next time you reference the layer in one of your local Lambda functions using the AWS Lambda layer ARN, the layer will be automatically pulled down and integrated into your local dev environment. ## Lambda Managed Instances LocalStack provides local testing support for Lambda Managed Instances. We support all of the new APIs, as well as the CloudFormation and Terraform resource types. All the same configuration is therefore possible on your local machine, without needing to modify your infrastructure as code (IaC). :::note This is an evolving feature and current support is scoped as follows: - We **do not spin up any additional EC2 instances**. LocalStack runs Lambda functions in containers on your local machine. - **Multi-concurrency support is not yet available.** This feature is planned for a future release (target: 2026). Currently, each parallel Lambda invocation is served by a separate container. - **IAM Permissions are not enforced.** A Capacity Provider is configured with an Operator Role, but the permissions are not enforced by LocalStack. It’s important you check the AWS documentation to ensure you’ve configured this role correctly. ::: ## Lambda Durable Functions [Lambda Durable Functions](https://docs.aws.amazon.com/lambda/latest/dg/durable-functions.html) run checkpointed, replayable workflows that can continue for up to one year. LocalStack supports durable executions for Node.js 22 and 24, Python 3.13 and 3.14, Java 17, 21, and 25, and .NET 8 and 10. ### Create and invoke a durable function Install the JavaScript durable execution SDK and include it with the function's deployment package: ```bash npm install --silent @aws/durable-execution-sdk-js zip -qr function.zip index.mjs node_modules package.json package-lock.json ``` Create `index.mjs` with a checkpointed step followed by a one-second durable wait: ```javascript title="index.mjs" showLineNumbers import { withDurableExecution } from "@aws/durable-execution-sdk-js"; export const handler = withDurableExecution(async (event, context) => { const result = await context.step(async () => ({ message: `Hello, ${event.name}!`, })); await context.wait({ seconds: 1 }); return result; }); ``` Create the function with a five-minute execution timeout and seven-day retention period: ```bash lstk aws lambda create-function \ --function-name durable-hello \ --runtime nodejs22.x \ --handler index.handler \ --role arn:aws:iam::000000000000:role/lambda-role \ --zip-file fileb://function.zip \ --timeout 30 \ --durable-config '{"ExecutionTimeout":300,"RetentionPeriodInDays":7}' ``` ```bash title="Output" { "FunctionName": "durable-hello", "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:durable-hello", "Runtime": "nodejs22.x", "State": "Pending", "DurableConfig": { "RetentionPeriodInDays": 7, "ExecutionTimeout": 300 } } ``` Wait for the function to become active: ```bash lstk aws lambda wait function-active-v2 --function-name durable-hello ``` Create `event.json`: ```json title="event.json" { "name": "LocalStack" } ``` Start an asynchronous durable execution. Use a qualified function target and a durable execution name, which acts as an idempotency key: ```bash EXECUTION_ARN=$(lstk aws lambda invoke \ --function-name durable-hello \ --qualifier '$LATEST' \ --invocation-type Event \ --durable-execution-name hello-docs \ --payload fileb://event.json \ response.json \ | jq -r '.DurableExecutionArn') echo "$EXECUTION_ARN" ``` ```text title="Output" arn:aws:lambda:us-east-1:000000000000:function:durable-hello:$LATEST/durable-execution/hello-docs/ ``` After the execution finishes, retrieve its status and result: ```bash lstk aws lambda get-durable-execution \ --durable-execution-arn "$EXECUTION_ARN" \ --query '{DurableExecutionName:DurableExecutionName,Status:Status,Result:Result,DurableConfig:DurableConfig}' ``` ```bash title="Output" { "DurableExecutionName": "hello-docs", "Status": "SUCCEEDED", "Result": "{\"message\":\"Hello, LocalStack!\"}", "DurableConfig": { "RetentionPeriodInDays": 7, "ExecutionTimeout": 300 } } ``` Inspect the execution history to see the checkpoint, suspension, replay, and completion: ```bash lstk aws lambda get-durable-execution-history \ --durable-execution-arn "$EXECUTION_ARN" \ --query 'Events[].EventType' ``` ```bash title="Output" [ "ExecutionStarted", "StepStarted", "StepSucceeded", "WaitStarted", "InvocationCompleted", "WaitSucceeded", "InvocationCompleted", "ExecutionSucceeded" ] ``` ### Supported behaviors LocalStack supports the following durable workflow behaviors: - Synchronous and asynchronous durable invocations with named, idempotent executions. - Checkpointed steps, waits, condition polling, and step retries. - Callbacks with heartbeats, success and failure responses, and restart-safe timeouts. - Chained Lambda invocations, parallel branches, maps, and child contexts. - Execution history, stopping and draining executions, retention, persistence, and asynchronous dead-letter queue delivery. - Function URL and event source mapping dispatch to durable functions, subject to the AWS execution-time constraints. ### Current Limitations :::note LocalStack Lambda runtime images do not include the durable execution SDK. Package the SDK and its dependencies with every durable function deployment artifact. This differs from AWS managed Node.js and Python runtimes, which include the SDK for testing and development. ::: LocalStack currently has the following limitations: - `KMSKeyArn` is validated, stored, merged, and returned, but LocalStack does not use the key to encrypt durable execution data. - LocalStack does not emit durable execution monitoring events to CloudWatch or EventBridge. - LocalStack does not enforce API request-per-second throttling or the account-level running-executions quota. - LocalStack enforces the checkpoint protocol limits, including 3,000 operations and 100 MB of written state per execution. [Lambda Debug Mode](/aws/developer-tools/lambda-tools/remote-debugging/#lambda-debug-mode-preview-) supports durable executions. Replays use the debug environment pinned to the execution, and LocalStack defers `ExecutionTimeout` while execution is paused at a breakpoint. ## LocalStack Lambda Runtime Interface Emulator (RIE) LocalStack uses a [custom implementation](https://github.com/localstack/lambda-runtime-init/) of the [AWS Lambda Runtime Interface Emulator](https://github.com/aws/aws-lambda-runtime-interface-emulator) to match the behavior of AWS Lambda as closely as possible while providing additional features such as [hot reloading](/aws/developer-tools/lambda-tools/hot-reloading). We ship our custom implementation as a Golang binary, which gets copied into each Lambda container under `/var/rapid/init`. This init binary is used as the entry point for every Lambda container. Our custom implementation offers additional configuration options, but these configurations are primarily intended for LocalStack developers and could change in the future. The LocalStack [configuration](/aws/customization/configuration-options) `LAMBDA_DOCKER_FLAGS` can be used to configure all Lambda containers, for example `LAMBDA_DOCKER_FLAGS=-e LOCALSTACK_INIT_LOG_LEVEL=debug`. Some noteworthy configurations include: - `LOCALSTACK_INIT_LOG_LEVEL` defines the log level of the Golang binary. Values: `trace`, `debug`, `info`, `warn` (default), `error`, `fatal`, `panic` - `LOCALSTACK_USER` defines the system user executing the Lambda runtime. Values: `sbx_user1051` (default), `root` (skip dropping root privileges) The full list of configurations is defined in the Golang function [InitLsOpts](https://github.com/localstack/lambda-runtime-init/blob/localstack/cmd/localstack/main.go#L43). ### Proxy configuration Lambda execution environments communicate back to the LocalStack container for runtime APIs, credentials, logs, and calls to emulated AWS services. If your host, Docker daemon, or Kubernetes cluster injects `http_proxy` or `https_proxy` into Lambda containers, that internal traffic must bypass the proxy. LocalStack automatically adds the configured `LOCALSTACK_HOST` host, and the runtime `LOCALSTACK_HOSTNAME` when present, to the Lambda execution environment's `no_proxy` variable. If your function already defines `no_proxy`, LocalStack preserves the existing entries and prepends the LocalStack hosts. In Kubernetes deployments, make sure admission controllers or proxy-injection policies do not overwrite the `no_proxy` value after LocalStack creates Lambda pods. If your cluster-level policy manages `no_proxy`, include the LocalStack host used by your deployment, for example `localhost.localstack.cloud` or the host configured through `LOCALSTACK_HOST`. ## Special Tools LocalStack provides various tools to help you develop, debug, and test your AWS Lambda functions more efficiently. - **Hot reloading**: With Lambda hot reloading, you can continuously apply code changes to your Lambda functions without needing to redeploy them manually. To learn more about how to use hot reloading with LocalStack, check out our [hot reloading documentation](/aws/developer-tools/lambda-tools/hot-reloading). - **Remote debugging**: LocalStack's remote debugging functionality allows you to attach a debugger to your Lambda function using your preferred IDE. To get started with remote debugging in LocalStack, see our [debugging documentation](/aws/developer-tools/lambda-tools/remote-debugging). - **Lambda VS Code Extension**: LocalStack's Lambda VS Code Extension supports deploying and invoking Python Lambda functions through AWS SAM or AWS CloudFormation. To get started with the Lambda VS Code Extension, see our [Lambda VS Code Extension documentation](/aws/connecting/ides/vscode-extension). - **API for querying Lambda runtimes**: LocalStack offers a metadata API to query the list of Lambda runtimes via `GET http://localhost.localstack.cloud:4566/_aws/lambda/runtimes`. It returns the [Supported Runtimes](https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html) matching AWS parity (i.e., excluding deprecated runtimes) and offers additional filters for `deprecated` runtimes and `all` runtimes (`GET /_aws/lambda/runtimes?filter=all`). ## Resource Browser The LocalStack Web Application provides a [Resource Browser](/aws/connecting/console/resource-browser) for managing Lambda resources. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **Lambda** under the **Compute** section. The Resource Browser displays Functions and Layers resources. You can click on individual resources to view their details. ![Lambda Resource Browser](/images/aws/lambda-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Functions & Layers**: Create a new Lambda function or a new Lambda Layer by clicking on **Create API** button on top-right and creating a new configuration by clicking on **Submit** button. - **View Function & Layer Details**: Click on any function or layer to view detailed information such as the resource's name, ARN, runtime, handler, and more. You can also navigate across different versions of the resource. - **Delete Functions & Layers**: To delete a function or layer, select the resource from the Resource Browser, click on the **Remove Selected** button at the top-right of the screen, and confirm the deletion by clicking on the **Continue** button. ## Migrating to Lambda v2 :::note The legacy Lambda implementation has been removed since LocalStack 3.0 (Docker `latest` since 2023-11-09). ::: As part of the [LocalStack 2.0 release](https://discuss.localstack.cloud/t/new-lambda-implementation-in-localstack-2-0/258), the Lambda provider has been migrated to `v2` (formerly known as `asf`). With the new implementation, the following changes have been introduced: - To run Lambda functions in LocalStack, mount the Docker socket into the LocalStack container. Add the following Docker volume mount to your LocalStack startup configuration: `/var/run/docker.sock:/var/run/docker.sock`. You can find an example of this configuration in our official [`docker-compose.yml` file](/aws/getting-started/installation/#docker-compose). - The `v2` provider discontinues Lambda Executor Modes such as `LAMBDA_EXECUTOR=local`. Previously, this mode was used as a fallback when the Docker socket was unavailable in the LocalStack container, but many users unintentionally used it instead of the configured `LAMBDA_EXECUTOR=docker`. The new provider now behaves similarly to the old `docker-reuse` executor and does not require such configuration. - The Lambda containers are now reused between invocations. The changes made to the filesystem (such as in `/tmp`) will persist between subsequent invocations if the function is dispatched to the same container. This is known as a **warm start** (see [Operating Lambda](https://aws.amazon.com/blogs/compute/operating-lambda-performance-optimization-part-1/) for more information). To ensure that each invocation starts with a fresh container, you can set the `LAMBDA_KEEPALIVE_MS` configuration option to 0 milliseconds, to force **cold starts**. - The platform uses [official Docker base images](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-images.html) pulled from `public.ecr.aws/lambda/`, instead of `lambci`, and supports both `arm64` and `x86_64` architectures. The Lambda functions filesystem now matches the AWS Lambda production environment. The ARM containers for compatible runtimes are based on Amazon Linux 2, and ARM-compatible hosts can create functions with the `arm64` architecture. - Lambda functions in LocalStack resolve AWS domains, such as `s3.amazonaws.com`, to the LocalStack container. This domain resolution is DNS-based and can be disabled by setting `DNS_ADDRESS=0`. For more information, refer to [Transparent Endpoint Injection](/aws/customization/networking/transparent-endpoint-injection). Previously, LocalStack provided patched AWS SDKs to redirect AWS API calls transparently to LocalStack. - The new provider may generate more exceptions due to invalid input. For instance, while the old provider accepted arbitrary strings (such as `r1`) as Lambda roles when creating a function, the new provider validates role ARNs using a regular expression that requires them to be in the format `arn:aws:iam::000000000000:role/lambda-role`. However, it currently does not verify whether the role actually exists. - The new Lambda provider now follows the [AWS Lambda state model](https://aws.amazon.com/blogs/compute/tracking-the-state-of-lambda-functions/), while creating and updating Lambda functions, which allows for asynchronous processing. Functions are always created in the `Pending state` and move to `Active` once they are ready to accept invocations. Previously, the functions were created synchronously by blocking until the function state was active. The configuration `LAMBDA_SYNCHRONOUS_CREATE=1` can force synchronous function creation, but it is not recommended. - LocalStack's Lambda implementation, allows you to customize the Lambda execution environment using the [Lambda Extensions API](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-extensions-api.html). This API allows for advanced monitoring, observability, or developer tooling, providing greater control and flexibility over your Lambda functions. Lambda functions can also be run on hosts with [multi-architecture support](/aws/customization/advanced/arm64-support/), allowing you to leverage LocalStack's Lambda API to develop and test Lambda functions with high parity. The following configuration options from the old provider are discontinued in the new provider: - The `LAMBDA_EXECUTOR` and specifically, the `LAMBDA_EXECUTOR=local` options are no longer supported. - The `LAMBDA_STAY_OPEN_MODE` is now the default behavior and can be removed. Instead, use the `LAMBDA_KEEPALIVE_MS` option to configure how long containers should be kept running in between invocations. - The `LAMBDA_REMOTE_DOCKER` option is not used anymore since the new provider automatically copies zip files and configures hot reloading. - The `LAMBDA_CODE_EXTRACT_TIME` option is no longer used because function creation is now asynchronous. - The `LAMBDA_FALLBACK_URL`, `SYNCHRONOUS_KINESIS_EVENTS`, `SYNCHRONOUS_SNS_EVENTS` and `LAMBDA_FORWARD_URL` options are currently not supported. - The `LAMBDA_CONTAINER_REGISTRY` option is not used anymore. Instead, use the more flexible `LAMBDA_RUNTIME_IMAGE_MAPPING` option to customize individual runtimes. - The `LAMBDA_XRAY_INIT` option is no longer needed because the X-Ray daemon is always initialized. However, the new provider still supports the following configuration options: - The `BUCKET_MARKER_LOCAL` option has a new default value, `hot-reload`. The former default value `__local__` is an invalid bucket name. - The `LAMBDA_TRUNCATE_STDOUT` option. - The `LAMBDA_DOCKER_NETWORK` option. - The `LAMBDA_DOCKER_FLAGS` option. - The `LAMBDA_REMOVE_CONTAINERS` option. - The `LAMBDA_DOCKER_DNS` option since LocalStack 2.2. - The `HOSTNAME_FROM_LAMBDA` option since LocalStack 3.0. ## Examples The following code snippets and sample applications provide practical examples of how to use Lambda in LocalStack for various use cases: - [Lambda Debugging](/aws/developer-tools/lambda-tools/remote-debugging) demonstrates how to remotely debug a Lambda function from within your IDE. - [Debug your Python Lambda Function](https://github.com/localstack-samples/localstack-pro-samples/tree/master/lambda-debugging-sam-python) - [Debug your JavaScript Lambda Function](https://github.com/localstack-samples/localstack-pro-samples/tree/master/lambda-debugging-sam-javascript) - [Debug your TypeScript Lambda Function](https://github.com/localstack-samples/localstack-pro-samples/tree/master/lambda-debugging-sam-typescript) - [Debug your Java Lambda Function](https://github.com/localstack-samples/localstack-pro-samples/tree/master/lambda-debugging-sam-java) - [Lambda Hot Reloading](https://github.com/localstack/localstack-pro-samples/tree/master/lambda-hot-reloading) shows how to use hot reloading to update function code and layers without having to redeploy them. - [Lambda Function URL](https://github.com/localstack-samples/localstack-pro-samples/tree/master/lambda-function-urls-javascript) shows how to use HTTP to invoke a Lambda function via its Function URL. - [Lambda Layers](https://github.com/localstack/localstack-pro-samples/blob/master/serverless-lambda-layers) demonstrates how to use Lambda layers, which are reusable packages of code that can be shared across multiple functions. - [Lambda PHP/Bref](https://github.com/localstack/localstack-pro-samples/tree/master/lambda-php-bref-cdk-app) shows how to use PHP/Bref with and without fpm, using the Serverless framework and AWS CDK. - [Lambda Container Images](https://github.com/localstack/localstack-pro-samples/tree/master/lambda-container-image) demonstrates how to use Lambda functions packaged as container images, which can be built using Docker and pushed to a local ECR registry. - [Lambda X-Ray](https://github.com/localstack/localstack-pro-samples/tree/master/lambda-xray) shows how to instrument Lambda functions for X-Ray using Powertools and the X-Ray SDK. ## Troubleshooting ### Docker not available In the old Lambda provider, Lambda functions were executed within the LocalStack container using the local executor mode. This mode was used as a fallback if the Docker socket was unavailable in the LocalStack container. However, many users inadvertently used the local executor mode instead of the intended Docker executor mode, which caused unexpected behavior. If you encounter the following error message, you may be using the local executor mode: ```bash Lambda 'arn:aws:lambda:us-east-1:000000000000:function:my-function:$LATEST' changed to failed. Reason: Docker not available ... raise DockerNotAvailable("Docker not available") ``` ```bash An error occurred (ResourceConflictException) when calling the Invoke operation (reached max retries: 0): The operation cannot be performed at this time. The function is currently in the following state: Failed ``` ```bash Error: Failed to create/update the stack: sam-app, Waiter StackCreateComplete failed: Waiter encountered a terminal failure state: For expression "Stacks[].StackStatus" we matched expected path: "CREATE_FAILED" at least once ``` To fix this issue, add the Docker volume mount `/var/run/docker.sock:/var/run/docker.sock` to your LocalStack startup. Refer to our [sample `docker-compose.yml` file](https://github.com/localstack/localstack/blob/main/docker-compose.yml) as an example. ### Function in Pending state If you receive a `ResourceConflictException` when trying to invoke a function, it is currently in a `Pending` state and cannot be executed yet. ```bash lstk aws lambda get-function --function-name my-function ``` ```bash title="Output" An error occurred (ResourceConflictException) when calling the Invoke operation (reached max retries: 0): The operation cannot be performed at this time. The function is currently in the following state: Pending ``` To wait until the function becomes `active`, you can use the following command: ```bash lstk aws lambda wait function-active-v2 --function-name my-function ``` Alternatively, you can check the function state using the [`GetFunction` API](https://docs.aws.amazon.com/lambda/latest/dg/API_GetFunction.html): ```bash lstk aws lambda get-function --function-name my-function ``` ```bash title="Output" { "Configuration": { ... "RevisionId": "c61d6139-1441-4ad5-983a-5a1cec7a1847", "State": "Pending", "StateReason": "The function is being created.", "StateReasonCode": "Creating", ... } } ``` When the function is active, the output will be similar to the following: ```bash lstk aws lambda get-function --function-name my-function ``` ```bash title="Output" { "Configuration": { ... "RevisionId": "c6633a28-b8d2-40f7-b8e1-02f6f32e8473", "State": "Active", "LastUpdateStatus": "Successful", ... } } ``` If the function is still in the `Pending` state, the output will include a `"State": "Pending"` field and a `"StateReason": "The function is being created."` message. Once the function is active, the `"State"` field will change to `"Active"` and the `"LastUpdateStatus"` field will indicate the status of the last update. ### Not implemented error If you are using LocalStack versions prior to 2.0, and encounter a `NotImplementedError` in the LocalStack logs and an `InternalFailure (501) error` in the client while creating a Lambda function using the [`CreateFunction` API](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunction.html), check your `PROVIDER_OVERRIDE_LAMBDA` configuration. You might encounter this error if it is set to `legacy`. ## API Coverage # CloudWatch Logs > Get started with AWS CloudWatch Logs on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; import { Badge } from '@astrojs/starlight/components'; ## Introduction [CloudWatch Logs](https://docs.aws.amazon.com/cloudwatch/index.html) allows to store and retrieve logs. While some services automatically create and write logs (e.g. Lambda), logs can also be added manually. :::note We've introduced a **new CloudWatch Logs provider (`v2`)** backed by a persistent SQLite database. It is an optional alternative to the default provider and is planned to become the default in an upcoming release. The main benefit is a significant reduction in memory usage for long-running instances that generate high volumes of log events, which addresses a memory leak present in the default provider. You can opt in to the new provider by setting: ```bash PROVIDER_OVERRIDE_LOGS=v2 ``` **Known limitation:** Switching to the `v2` provider on an existing instance results in an empty log event history, since log event data cannot be migrated from the default provider's in-memory store. Metadata such as log groups, log streams, subscription filters, and tags is migrated correctly. ::: ## Subscription Filters Subscription filters can be used to forward logs to certain services, e.g. Kinesis, Lambda, and Kinesis Data Firehose. You can read upon details in the [official AWS docs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html). ### Subscription Filters with Kinesis Example In the following we setup a little example on how to use subscription filters with kinesis. First, we setup the required resources. Therefore, we create a kinesis stream, a log group and log stream. Then we can configure the subscription filter. ```bash lstk aws kinesis create-stream --stream-name "logtest" --shard-count 1 kinesis_arn=$(lstk aws kinesis describe-stream --stream-name "logtest" | jq -r .StreamDescription.StreamARN) lstk aws logs create-log-group --log-group-name test lstk aws logs create-log-stream \ --log-group-name test \ --log-stream-name test lstk aws logs put-subscription-filter \ --log-group-name "test" \ --filter-name "kinesis_test" \ --filter-pattern "" \ --destination-arn $kinesis_arn \ --role-arn "arn:aws:iam::000000000000:role/kinesis_role" ``` Next, we can add a log event, that will be forwarded to Kinesis. ```bash timestamp=$(($(date +'%s * 1000 + %-N / 1000000'))) lstk aws logs put-log-events --log-group-name test --log-stream-name test --log-events "[{\"timestamp\": ${timestamp} , \"message\": \"hello from cloudwatch\"}]" ``` Now we can retrieve the data. In our example, there will only be one record. The data record is base64 encoded and compressed in gzip format: ```bash shard_iterator=$(lstk aws kinesis get-shard-iterator --stream-name logtest --shard-id shardId-000000000000 --shard-iterator-type TRIM_HORIZON | jq -r .ShardIterator) record=$(lstk aws kinesis get-records --limit 10 --shard-iterator $shard_iterator | jq -r '.Records[0].Data') echo $record | base64 -d | zcat ``` ## Filter Pattern [Filter patterns](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html) can be used to select certain logs only. LocalStack currently supports simple json-property filter. ### Metric Filter Example Metric filters can be used to automatically create CloudWatch metrics. In the following example we are interested in logs that include a key-value pair `"foo": "bar"` and create a metric filter. ```bash lstk aws logs create-log-group --log-group-name test-filter lstk aws logs create-log-stream \ --log-group-name test-filter \ --log-stream-name test-filter-stream lstk aws logs put-metric-filter \ --log-group-name test-filter \ --filter-name my-filter \ --filter-pattern "{$.foo = \"bar\"}" \ --metric-transformations \ metricName=MyMetric,metricNamespace=MyNamespace,metricValue=1,defaultValue=0 ``` Next, we can insert some values: ```bash timestamp=$(($(date +'%s * 1000 + %-N / 1000000'))) lstk aws logs put-log-events --log-group-name test-filter \ --log-stream-name test-filter-stream \ --log-events \ timestamp=$timestamp,message='"{\"foo\":\"bar\", \"hello\": \"world\"}"' \ timestamp=$timestamp,message="my test event" \ timestamp=$timestamp,message='"{\"foo\":\"nomatch\"}"' ``` Now we can check that the metric was indeed created: ```bash end=$(date +%s) lstk aws cloudwatch get-metric-statistics --namespace MyNamespace \ --metric-name MyMetric --statistics Sum --period 3600 \ --start-time 1659621274 --end-time $end ``` ### Filter Log Events Similarly, you can use filter-pattern to filter logs with different kinds of patterns as described by [AWS](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html). #### JSON Filter Pattern For purely JSON structured log messages, you can use JSON filter patterns to traverse the JSON object. Enclose your pattern in curly braces, like this: ```bash lstk aws logs filter-log-events --log-group-name test-filter --filter-pattern "{$.foo = \"bar\"}" ``` This returns all events whose top level "foo" key has the "bar" value. #### Regular Expression Filter Pattern You can use a simplified regex syntax for regular expression matching. Enclose your pattern in percentage signs like this: ```bash lstk aws logs filter-log-events --log-group-name test-filter --filter-pattern "\%[fF]oo\%" ``` This returns all events containing "Foo" or "foo". For a complete set of the supported syntax, check [the official AWS documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html#regex-expressions) #### Unstructured Filter Pattern If not specified otherwise in the pattern, we look for a match in the whole event message: ```bash lstk aws logs filter-log-events --log-group-name test-filter --filter-pattern "foo" ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for exploring CloudWatch Logs. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **CloudWatch Logs** under the **Management/Governance** section. ![CloudWatch Logs Resource Browser](/images/aws/logs-resource-browser.png) The Resource Browser allows you to perform the following actions: * **Create Log Group**: Create a new log group by clicking on the **Create Log Group** button followed by entering the details in the dialog box. * **Create Log Stream**: Create a new log stream by clicking on the **Create Log Stream** button in the log group detail followed by entering the details in the dialog box. * **Filter Log Events**: Filter log events by clicking the log stream name followed by entering the filter pattern and clicking **Apply**. * **Delete Log Group**: Delete a log group by selecting the log group name and clicking on the **Actions** dropdown menu, then selecting **Remove Selected**. * **Delete Log Stream**: Delete a log stream by selecting the log stream name and clicking on the **Actions** dropdown menu, then selecting **Remove Selected**. ## API Coverage # Managed Blockchain (AMB) > Get started with Managed Blockchain (AMB) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Managed Blockchain (AMB) is a managed service that enables the creation and management of blockchain networks, such as Hyperledger Fabric, Bitcoin, Polygon and Ethereum. Blockchain enables the development of applications in which multiple entities can conduct transactions and exchange data securely and transparently, eliminating the requirement for a central, trusted authority. LocalStack allows you to use the AMB APIs to develop and deploy decentralized applications in your local environment. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of AMB integration with LocalStack. ## Getting started This guide is designed for users new to AMB and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a blockchain network, a node, and a proposal. ### Create a blockchain network You can create a blockchain network using the [`CreateNetwork`](https://docs.aws.amazon.com/managed-blockchain/latest/APIReference/API_CreateNetwork.html) API. Run the following command to create a network named `OurBlockchainNet` which uses the Hyperledger Fabric with the following configuration: ```bash showshowLineNumbers lstk aws managedblockchain create-network \ --cli-input-json '{ "Name": "OurBlockchainNet", "Description": "OurBlockchainNetDesc", "Framework": "HYPERLEDGER_FABRIC", "FrameworkVersion": "1.2", "FrameworkConfiguration": { "Fabric": { "Edition": "STARTER" } }, "VotingPolicy": { "ApprovalThresholdPolicy": { "ThresholdPercentage": 50, "ProposalDurationInHours": 24, "ThresholdComparator": "GREATER_THAN" } }, "MemberConfiguration": { "Name": "org1", "Description": "Org1 first member of network", "FrameworkConfiguration": { "Fabric": { "AdminUsername": "MyAdminUser", "AdminPassword": "Password123" } }, "LogPublishingConfiguration": { "Fabric": { "CaLogs": { "Cloudwatch": { "Enabled": true } } } } } }' ``` ```bash title="Output" { "NetworkId": "n-X24AF1AK2GC6MDW11HYW5I5DQC", "MemberId": "m-6VWBWHP2Y15F7TQ2DS093RTCW2" } ``` Copy the `NetworkId` and `MemberId` values from the output of the above command, as we will need them in the next step. ### Create a node You can create a node using the [`CreateNode`](https://docs.aws.amazon.com/managed-blockchain/latest/APIReference/API_CreateNode.html) API. Run the following command to create a node with the following configuration: ```bash showshowLineNumbers lstk aws managedblockchain create-node \ --node-configuration '{ "InstanceType": "bc.t3.small", "AvailabilityZone": "us-east-1a", "LogPublishingConfiguration": { "Fabric": { "ChaincodeLogs": { "Cloudwatch": { "Enabled": true } }, "PeerLogs": { "Cloudwatch": { "Enabled": true } } } } }' \ --network-id n-X24AF1AK2GC6MDW11HYW5I5DQC \ --member-id m-6VWBWHP2Y15F7TQ2DS093RTCW2 ``` ```bash title="Output" { "NodeId": "nd-77K8AI0O5BEQD1IW4L8OGKMXV7" } ``` Replace the `NetworkId` and `MemberId` values in the above command with the values you copied in the previous step. ### Create a proposal You can create a proposal using the [`CreateProposal`](https://docs.aws.amazon.com/managed-blockchain/latest/APIReference/API_CreateProposal.html) API. Run the following command to create a proposal with the following configuration: ```bash lstk aws managedblockchain create-proposal \ --actions "Invitations=[{Principal=000000000000}]" \ --network-id n-X24AF1AK2GC6MDW11HYW5I5DQC \ --member-id m-6VWBWHP2Y15F7TQ2DS093RTCW2 ``` ```bash title="Output" { "ProposalId": "p-NK0PSLDPETJQX01Q4OLBRHP8CZ" } ``` Replace the `NetworkId` and `MemberId` values in the above command with the values you copied in the previous step. ## API Coverage # Elemental MediaConvert > Get started with Elemental MediaConvert on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Elemental MediaConvert is a file-based video transcoding service with broadcast-grade features. It enables you to easily create high-quality video streams for broadcast and multiscreen delivery. LocalStack allows you to mock the MediaConvert APIs in your local environment. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of MediaConvert's integration with LocalStack. :::note Elemental MediaConvert is in a preview state. ::: ## Getting started This guide is designed for users new to Elemental MediaConvert and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a MediaConvert job, list jobs, create a queue, and list all queues using the AWS CLI. ### Create a job Create a new file named `job.json` on your local directory: ```json showshowLineNumbers { "Role": "arn:aws:iam::000000000000:role/MediaConvert_Default_Role", "Settings": { "Inputs": [ { "VideoSelector": {}, "AudioSelectors": { "Audio Selector 1": { "DefaultSelection": "DEFAULT" } }, "TimecodeSource": "ZEROBASED", "FileInput": "s3://testbucket/input.mp4" } ], "OutputGroups": [ { "Name": "File Group", "OutputGroupSettings": { "Type": "FILE_GROUP_SETTINGS", "FileGroupSettings": { "Destination": "s3://testbucket/output.mp4" } }, "Outputs": [ { "VideoDescription": { "CodecSettings": { "Codec": "H_264", "H264Settings": { "RateControlMode": "QVBR", "SceneChangeDetect": "TRANSITION_DETECTION", "MaxBitrate": 5000000 } } }, "AudioDescriptions": [ { "CodecSettings": { "Codec": "AAC", "AacSettings": { "Bitrate": 96000, "CodingMode": "CODING_MODE_2_0", "SampleRate": 48000 } }, "AudioSourceName": "Audio Selector 1" } ], "ContainerSettings": { "Container": "MP4", "Mp4Settings": {} } } ], "CustomName": "output" } ], "TimecodeConfig": { "Source": "ZEROBASED" }, "FollowSource": 1 } } ``` You can create a MediaConvert job using the [`CreateJob`](https://docs.aws.amazon.com/goto/WebAPI/mediaconvert-2017-08-29/CreateJob) API. Execute the following command to create a job using a `job.json` file: ```bash lstk aws mediaconvert create-job --cli-input-json file://job.json ``` ```bash title="Output" { "Job": { "AccelerationSettings": { "Mode": "DISABLED" }, "AccelerationStatus": "NOT_APPLICABLE", "Arn": "arn:aws:mediaconvert:us-east-1:000000000000:jobs/1727963943858-7bdace", ... "Role": "arn:aws:iam::123456789012:role/MediaConvert_Default_Role", "Settings": { "FollowSource": 1, "Inputs": [ { "AudioSelectors": { "Audio Selector 1": { "DefaultSelection": "DEFAULT" } }, ... } ], "OutputGroups": [ { "CustomName": "output", "Name": "File Group", ... } ], "TimecodeConfig": { "Source": "ZEROBASED" } }, "Status": "SUBMITTED", ... } } ``` ### List the jobs You can list all MediaConvert jobs using the [`ListJobs`](https://docs.aws.amazon.com/mediaconvert/latest/apireference/jobs.html#jobsget) API. Execute the following command to list all jobs: ```bash lstk aws mediaconvert list-jobs ``` ### Create a queue You can create a MediaConvert queue using the [`CreateQueue`](https://docs.aws.amazon.com/mediaconvert/latest/apireference/queues.html#queuespost) API. Execute the following command to create a queue named `MyQueue`: ```bash lstk aws mediaconvert create-queue --name MyQueue --description "High priority queue for video encoding" ``` ```bash title="Output" { "Queue": { "Arn": "arn:aws:mediaconvert:us-east-1:000000000000:queues/MyQueue", "CreatedAt": "2024-10-03T19:30:04.015501+05:30", "Description": "High priority queue for video encoding", "LastUpdated": "2024-10-03T19:30:04.015501+05:30", "Name": "MyQueue", "PricingPlan": "ON_DEMAND", "ProgressingJobsCount": 0, "Status": "ACTIVE", "SubmittedJobsCount": 0, "Type": "CUSTOM" } } ``` ### List the queues You can list all MediaConvert queues using the [`ListQueues`](https://docs.aws.amazon.com/mediaconvert/latest/apireference/queues.html#queuesget) API. Execute the following command to list all queues: ```bash lstk aws mediaconvert list-queues ``` ## Current Limitations Currently, the service mocks the submission of encoding jobs to either the default queue or a custom-created queue. While actual transcoding is not performed, job completion is emulated. Job status progresses after a brief wait, and EventBridge events are emitted when the job state changes, allowing users to determine if a job has finished. This delay can be disabled by setting `MEDIACONVERT_DISABLE_JOB_DURATION=1`, which causes processing jobs to complete almost instantly. ## API Coverage # MemoryDB > Get started with MemoryDB on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Amazon MemoryDB is a fully managed, in-memory database service designed for ultra-fast, primary database use cases. Compatible with both Valkey and Redis OSS, it improves performance and durability. Built for the AWS cloud environment, MemoryDB simplifies the deployment and operation of in-memory databases, acting as a replacement for using a cache in front of a database. LocalStack provides support for the main MemoryDB APIs surrounding cluster creation, allowing developers to utilize the MemoryDB functionalities in their local development environment. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of MemoryDB's integration with LocalStack. ## Getting started This guide is designed for users new to MemoryDB and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how you can create a MemoryDB cluster and connect to it. ### Basic cluster creation You can create a MemoryDB cluster using the [`CreateCluster`](https://docs.aws.amazon.com/memorydb/latest/APIReference/API_CreateCluster.html) API. Run the following command to create a cluster: ```bash lstk aws memorydb create-cluster \ --cluster-name my-redis-cluster \ --node-type db.t4g.small \ --acl-name open-access ``` Once it becomes available, you will be able to use the cluster endpoint for Redis operations. Run the following command to retrieve the cluster endpoint using the [`DescribeClusters`](https://docs.aws.amazon.com/memorydb/latest/APIReference/API_DescribeClusters.html) API: ```bash lstk aws memorydb describe-clusters --query "Clusters[0].ClusterEndpoint" ``` ```bash title="Output" { "Address": "127.0.0.1", "Port": 36739 } ``` The cache cluster uses a random port of the [external service port range](/aws/customization/networking/external-port-range/) in regular execution and a port between 36739 and 46738 in container mode. Use this port number to connect to the Redis instance using the `redis-cli` command line tool: ```bash redis-cli -p 4510 ping PONG redis-cli -p 4510 set foo bar OK redis-cli -p 4510 get foo "bar" ``` You can also check the cluster configuration using the [`cluster nodes`](https://redis.io/commands/cluster-nodes) command: ```bash redis-cli -c -p 4510 cluster nodes ``` ## Container mode To start Redis clusters of a specific version, enable container mode for Redis-based services in LocalStack. This approach directs LocalStack to launch Redis instances in distinct containers, utilizing your chosen image tag. Additionally, container mode is beneficial for independently examining the logs of each Redis instance. To activate this, set the `REDIS_CONTAINER_MODE` configuration variable to `1`. ## Valkey Engine LocalStack offers the additional option to use Valkey as an alternative to Redis in Amazon MemoryDB. To enable full Valkey emulation: 1. Start LocalStack with Valkey support enabled by setting the environment variable, `REDIS_CONTAINER_MODE=1` 2. Create a cluster with the Valkey engine by including the `--engine valkey` flag in your API call: ```bash lstk aws memorydb create-cluster \ --cluster-name my-valkey-cluster \ --node-type db.t4g.small \ --acl-name open-access \ --engine valkey ``` Valkey support includes: - The ability to specify `valkey` as the engine when creating Amazon MemoryDB clusters. - Automatic mapping of each engine to a default supported version (Redis `7.2.10`, Valkey `7.2.10`), ensuring DockerHub compatibility. - Support for the `Engine` and `EngineVersion` fields in the `CreateCluster` API, which now recognize and handle `valkey`. :::note A Valkey cluster can only be started when [container mode](/aws/services/memorydb/#container-mode) is enabled. ::: ## Current Limitations LocalStack's emulation support for MemoryDB primarily focuses on the creation and termination of Redis servers in cluster mode. Essential resources for running a cluster, such as parameter groups, security groups, and subnet groups, are mocked but have no effect on the Redis servers' operation. LocalStack currently doesn't support MemoryDB snapshots, failovers, users/passwords, service updates, replication scaling, SSL, migrations, service integration (like CloudWatch/Kinesis log delivery, SNS notifications) or tests. At present, LocalStack does not support features such as: - MemoryDB snapshots - Failovers - User/password management - Service updates - Replication scaling - SSL - Migrations - Service integration (e.g., CloudWatch/Kinesis log delivery, SNS notifications) or facilitate related testing. ## API Coverage # MQ > Get started with MQ on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction MQ is a managed message broker service offered by Amazon Web Services (AWS). It facilitates the exchange of messages between various components of distributed applications, enabling reliable and scalable communication. AWS MQ supports popular messaging protocols like MQTT, AMQP, and STOMP, making it suitable for a wide range of messaging use cases. LocalStack allows you to use the MQ APIs to implement pub/sub messaging, request/response patterns, or distributed event-driven architectures in your local environment. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of MQ integration with LocalStack. ## Getting started This guide is designed for users new to MQ and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create an MQ broker and send a message to a sample queue. ### Create a broker You can create a broker using the [`CreateBroker`](https://docs.aws.amazon.com/amazon-mq/latest/api-reference/brokers.html#brokerspost) API. Run the following command to create a broker named `test-broker` with the following configuration: ```bash lstk aws mq create-broker \ --broker-name test-broker \ --deployment-mode SINGLE_INSTANCE \ --engine-type ACTIVEMQ \ --engine-version='5.18' \ --host-instance-type 'mq.t3.micro' \ --auto-minor-version-upgrade \ --publicly-accessible \ --users='{"ConsoleAccess": true, "Groups": ["testgroup"],"Password": "QXwV*$iUM9USHnVv&!^7s3c@", "Username": "admin"}' ``` ```bash title="Output" { "BrokerArn": "arn:aws:mq:us-east-1:000000000000:broker:test-broker:b-f503abb7-66bc-47fb-b1a9-8d8c51ef6545", "BrokerId": "b-f503abb7-66bc-47fb-b1a9-8d8c51ef6545" } ``` ### Describe the broker You can use the [`DescribeBroker`](https://docs.aws.amazon.com/amazon-mq/latest/api-reference/brokers.html#brokersget) API to get more detailed information about the broker. Run the following command to get information about the broker we created above: ```bash lstk aws mq describe-broker --broker-id b-f503abb7-66bc-47fb-b1a9-8d8c51ef6545 ``` ```bash title="Output" { "BrokerArn": "arn:aws:mq:us-east-1:000000000000:broker:test-broker:b-f503abb7-66bc-47fb-b1a9-8d8c51ef6545", "BrokerId": "b-f503abb7-66bc-47fb-b1a9-8d8c51ef6545", "BrokerInstances": [ { "ConsoleURL": "http://localhost:4513", "Endpoints": [ "stomp://localhost:4515", "tcp://localhost:4514" ] } ], "BrokerName": "test-broker", "BrokerState": "RUNNING", "Created": "2022-10-17T07:14:21.065527Z", "DeploymentMode": "SINGLE_INSTANCE", "EngineType": "ACTIVEMQ", "HostInstanceType": "mq.t3.micro", "Tags": {} } ``` ### Send a message Now that the broker is actively listening, we can use curl to send a message to a sample queue. Run the following command to send a message to the `orders.input` queue: ```bash curl -XPOST -d "body=message" http://admin:admin@localhost:4513/api/message\?destination\=queue://orders.input ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing MQ brokers. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resource Browser** section, and then clicking on **MQ** under the **App Integration** section. ![MQ Resource Browser](/images/aws/mq-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Broker**: Create a new MQ broker by clicking on the **Create Broker** button and providing the required parameters. - **View Broker**: View details of an existing MQ broker by clicking on the broker name. - **Delete Broker**: Select the broker name and click on the **Actions** button followed by **Remove Selected** button. ## Examples The following code snippets and sample applications provide practical examples of how to use MQ in LocalStack for various use cases: - [Demo application illustrating the use of MQ with LocalStack](https://github.com/localstack/localstack-pro-samples/tree/master/mq-broker) ## Current Limitations Currently, our MQ emulation offers only fundamental capabilities, and it comes with certain limitations: - **ActiveMQ Version Limitation:** LocalStack pins Apache MQ 5.18 to the patched version 5.18.7. Since June 2025, [5.18 is the only supported version on AWS](https://docs.aws.amazon.com/amazon-mq/latest/developer-guide/activemq-version-management.html). RabbitMQ is not supported at this time. - **IAM User Management:** IAM Users are not actively enforced, although they are necessary for making correct calls within the system. - **Configuration Enforcement:** While it is feasible to create configurations, they are not actively enforced within the broker. - **Persistence and Cloud Pods:** LocalStack does not provide support for Persistence and Cloud Pods at this time. - **API Coverage:** Please note that there is limited API coverage available as part of the current emulation capabilities. ## API Coverage # Managed Workflows for Apache Airflow (MWAA) > Get started with Managed Workflows for Apache Airflow (MWAA) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Managed Workflows for Apache Airflow (MWAA) is a fully managed service by AWS that simplifies the deployment, management, and scaling of [Apache Airflow](https://airflow.apache.org/) workflows in the cloud. MWAA leverages the familiar Airflow features and integrations while integrating with S3, Glue, Redshift, Lambda, and other AWS services to build data pipelines and orchestrate data processing workflows in the cloud. LocalStack allows you to use the MWAA APIs in your local environment to allow the setup and operation of data pipelines. The supported APIs are available on the [API Coverage section](#api-coverage). ## Getting started This guide is designed for users new to MWAA and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create an Airflow environment and access the Airflow UI. ### Create a S3 bucket Create a S3 bucket that will be used for Airflow resources. Run the following command to create a bucket using the [`mb`](https://docs.aws.amazon.com/cli/latest/reference/s3/mb.html) command. ```bash lstk aws s3 mb s3://my-mwaa-bucket ``` ### Create an Airflow environment You can now create an Airflow environment, using the [`CreateEnvironment`](https://docs.aws.amazon.com/mwaa/latest/API/API_CreateEnvironment.html) API. Run the following command, by specifying the bucket ARN we created earlier: ```bash lstk aws mwaa create-environment --dag-s3-path /dags \ --execution-role-arn arn:aws:iam::000000000000:role/airflow-role \ --network-configuration {} \ --source-bucket-arn arn:aws:s3:::my-mwaa-bucket \ --airflow-version 2.10.3 \ --airflow-configuration-options agent.code=007,agent.name=bond \ --name my-mwaa-env ``` ### Access the Airflow UI The Airflow UI can be accessed via the URL in the `WebserverUrl` attribute of the response of the `GetEnvironment` operation. The username and password are always set to `localstack`. ```bash lstk aws mwaa get-environment \ --name my-mwaa-env \ --query Environment.WebserverUrl \ "http://localhost.localstack.cloud:4510" ``` LocalStack also prints this information in the logs: ```bash title="Output" 2024-03-06T14:54:47.070 INFO --- [functhread10] l.services.mwaa.provider : Airflow environment 'my-mwaa-env' available at http://localhost.localstack.cloud:4510 with username 'localstack' and password 'localstack' ``` ## Airflow versions LocalStack supports the following versions of Apache Airflow: - `2.7.2` - `2.8.1` - `2.9.2` - `2.10.1` - `2.10.3` - `3.0.6` (default) :::note Deprecated versions will be removed in line with [AWS's end of support dates](https://docs.aws.amazon.com/mwaa/latest/userguide/airflow-versions.html#airflow-versions-official). ::: ## Airflow configuration options To configure Airflow environments effectively, you can utilize the `AirflowConfigurationOptions` argument. These options are transformed into corresponding environment variables and passed to Airflow. For instance: - `agent.code`:`007` is transformed into `AIRFLOW__AGENT__CODE:007`. - `agent.name`:`bond` is transformed into `AIRFLOW__AGENT__NAME:bond`. This transformation process ensures that your configuration settings are easily applied within the Airflow environment. ## Adding or updating DAGs When it comes to adding or updating DAGs in Airflow, the process is simple and efficient. Just upload your DAGs to the designated S3 bucket path, configured by the `DagS3Path` argument. For example, the command below uploads a sample DAG named `sample_dag.py` to your S3 bucket named `my-mwaa-bucket`: ```bash lstk aws s3 cp sample_dag.py s3://my-mwaa-bucket/dags ``` LocalStack syncs new and changed objects in the S3 bucket to the Airflow container every 30 seconds. The polling interval can be changed using the [`MWAA_S3_POLL_INTERVAL`](/aws/customization/configuration-options/#mwaa) config option. ## Installing custom plugins You can extend the capabilities of Airflow by incorporating custom plugins, which introduce new operators, interfaces, or hooks. LocalStack seamlessly supports plugins packaged according to [AWS specifications](https://docs.aws.amazon.com/mwaa/latest/userguide/configuring-dag-import-plugins.html#configuring-dag-plugins-test-create). To integrate your custom plugins into the MWAA environment, upload the packaged `plugins.zip` file to the designated S3 bucket path: ```bash lstk aws s3 cp plugins.zip s3://my-mwaa-bucket/plugins.zip ``` ## Installing Python dependencies LocalStack streamlines the process of installing Python dependencies for Apache Airflow within your environments. To get started, create a `requirements.txt` file that lists the required dependencies. For example: ```txt boto3==1.17.54 boto==2.49.0 botocore==1.20.54 ``` Once you have your `requirements.txt` file ready, upload it to the designated S3 bucket, configured for use by the MWAA environment. Make sure to upload the file to `/requirements.txt` in the bucket: ```bash lstk aws s3 cp requirements.txt s3://my-mwaa-bucket/requirements.txt ``` After the upload, the environment will be automatically updated, and your Apache Airflow setup will be equipped with the new dependencies. It is important to note that, unlike [AWS](https://docs.aws.amazon.com/mwaa/latest/userguide/connections-packages.html), LocalStack does not install any provider packages by default. Therefore, you must follow the above steps to install any required provider packages. ## Connections When incorporating connections to other AWS services within your DAGs, it is crucial to specify either the internal Docker IP address of the LocalStack container or utilize `host.docker.internal`. LocalStack currently does not use the credentials and region from `aws_conn_id`. This information must be explicitly passed in operators, hooks, and sensors. ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing MWAA Environments. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resource Browser** section, and then clicking on **MWAA** under the **App Integration** section. ![MWAA Resource Browser](/images/aws/mwaa-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Environment**: Create a new MWAA environment by clicking on the **Create Environment** button and providing the required parameters. - **View Environment**: View details of an existing MWAA environment by clicking on the environment name. - **Edit Environment**: Edit an existing MWAA environment by clicking on the **Edit** button after clicking on the environment name. - **Delete Environment**: Select the environment name and click on the **Actions** button followed by **Remove Selected** button. ## Current Limitations - LocalStack MWAA does not support [startup scripts](https://docs.aws.amazon.com/mwaa/latest/userguide/using-startup-script.html) ## API Coverage # Neptune > Get started with Neptune on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Neptune is a fully managed, highly available, and scalable graph database service offered by AWS. It is designed for storing and querying highly connected data for applications that require complex relationship modeling, such as social networks, recommendation engines, and fraud detection. Neptune supports popular graph query languages like Gremlin and SPARQL, making it compatible with a wide range of graph applications and tools. LocalStack allows you to use the Neptune APIs in your local environment to support both property graph and RDF graph models. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Neptune's integration with LocalStack. The following versions of Neptune engine are supported by LocalStack: | Engine Version | Tinkerpop Version | |-----------------|---------------------| | `1.1.0.0` | `3.4.11` | | `1.1.1.0` | `3.5.2` | | `1.2.0.0` | `3.5.2` | | `1.2.0.1` | `3.5.2` | | `1.2.0.2` | `3.5.2` | | `1.2.1.0` | `3.6.2` | | `1.2.1.1` | `3.6.2` | | `1.3.0.0` | `3.6.2` | | `1.3.1.0` | `3.6.2` | | `1.3.2.0` | `3.7.2` | | `1.3.2.1` | `3.7.2` | | `1.3.4.0` | `3.7.2` | | `1.4.0.0` | `3.7.2` | | `1.4.1.0` | `3.7.2` | | `1.4.2.0` | `3.7.2` | | `1.4.3.0` | `3.7.2` | ## Getting started This guide is designed for users new to Neptune and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate the following with AWS CLI & Python: - Creating a Neptune cluster. - Starting a connection to the Neptune cluster. - Running a Python script to create nodes and edges and query the graph database. ### Create a Neptune cluster To create a Neptune cluster you can use the [`CreateDBCluster`](https://docs.aws.amazon.com/neptune/latest/userguide/api-clusters.html#CreateDBCluster) API. Run the following command to create a Neptune cluster: ```bash lstk aws neptune create-db-cluster \ --engine neptune \ --db-cluster-identifier my-neptune-db ``` ```bash title="Output" { "DBCluster": { ... "Endpoint": "localhost", "Port": 4510, # may vary "DBClusterArn": "arn:aws:rds:us-east-1:000000000000:cluster:my-neptune-db", ... } } ``` ### Add an instance to the cluster To add an instance you can use the [`CreateDBInstance`](https://docs.aws.amazon.com/neptune/latest/userguide/api-instances.html#CreateDBInstance) API. Run the following command to create a Neptune instance: ```bash lstk aws neptune create-db-instance \ --db-cluster-identifier my-neptune-db \ --db-instance-identifier my-neptune-instance \ --engine neptune \ --db-instance-class db.t3.medium ``` In LocalStack the `Endpoint` for the `DBCluster` and the `Endpoint.Address` of the `DBInstance` will be the same and can be used to connect to the graph database. ### Start a connection To start a connection you have to use the `ws` protocol. Here is an example that uses Python and [`gremlinpython`](https://pypi.org/project/gremlinpython/) to connect to the database: ```python showshowLineNumbers from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection from gremlin_python.process.anonymous_traversal import traversal from gremlin_python.process.traversal import Bindings, T, gt ENDPOINT = "localhost:4510" # TODO change to your endpoint DATABASE_URL = f"ws://{ENDPOINT}/gremlin" if __name__ == '__main__': conn = DriverRemoteConnection( DATABASE_URL, "g", pool_size=1, ) g = traversal().withRemote(conn) # add some nodes v1 = g.addV("person").property(T.id, "1").property("name", "marko").property("age", 29).next() v2 = g.addV("person").property(T.id, "2").property("name", "stephen").property("age", 33).next() v3 = g.addV("person").property(T.id, "3").property("name", "mia").property("age", 30).next() # add edges/relation g.V(Bindings.of("id", v1)).addE("knows").to(v2).property("weight", 0.75).iterate() g.V(Bindings.of("id", v1)).addE("knows").to(v3).property("weight", 0.85).iterate() # retrieve all names names = g.V().values("name").to_list() # list all names of persons that know "marko" marko_knows = g.V("1").outE("knows").inV().values("name").order().to_list() # all persons that "marko" know that are older than 30 marko_knows_older_30 = g.V("1").out("knows").has("age", gt(30)).values("name").to_list() # reset everything g.V().drop().iterate() result = { "names": names, "marko_knows": marko_knows, "marko_knows_older_30": marko_knows_older_30, } print(result) ``` ## IAM Enforcement for Gremlin Queries Amazon Neptune resources with IAM DB authentication enabled require all requests to use AWS Signature Version 4. When LocalStack starts with [IAM enforcement enabled](/aws/developer-tools/security-testing/iam-policy-enforcement), the Neptune database checks user permissions before granting access. The following Gremlin query actions are available for database engine versions `1.3.2.0` and higher: ```json { "Action": [ "neptune-db:ReadDataViaQuery", "neptune-db:WriteDataViaQuery", "neptune-db:DeleteDataViaQuery" ] } ``` Start LocalStack with `LOCALSTACK_ENFORCE_IAM=1` to create a Neptune cluster with IAM DB authentication enabled. ```bash LOCALSTACK_ENFORCE_IAM=1 lstk start ``` You can then create a cluster. ```bash lstk aws neptune create-db-cluster \ --engine neptune \ --db-cluster-identifier myneptune-db \ --enable-iam-database-authentication ``` After the cluster is deployed, the Gremlin server will reject unsigned queries. ```bash curl "https://localhost.localstack.cloud:4510/gremlin?gremlin=g.V()" -v ``` The output will be similar to the following: ```bash - Request completely sent off < HTTP/1.1 403 Forbidden - no chunk, no close, no size. Assume close to signal end ... ``` Use the Python package [awscurl](https://pypi.org/project/awscurl/) to make your first signed query. ```bash awscurl "https://localhost.localstack.cloud:4510/gremlin?gremlin=g.V().count()" -H "Accept: application/json" | jq . ``` ```bash title="Output" { "requestId": "729c3e7b-50b3-4df7-b0b6-d1123c4e81df", "status": { "message": "", "code": 200, "attributes": { "@type": "g:Map", "@value": [] } }, "result": { "data": { "@type": "g:List", "@value": [ { "@type": "g:Int64", "@value": 0 } ] }, "meta": { "@type": "g:Map", "@value": [] } } } ``` :::note If Gremlin Server is installed in your LocalStack environment, you must delete it and restart LocalStack. You can find your LocalStack volume location on the [LocalStack filesystem documentation](/aws/customization/advanced/filesystem/#localstack-volume). ```bash rm -rf /lib/tinkerpop ``` ::: ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing Neptune databases and clusters. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **Neptune** under the **Database** section. ![Neptune Resource Browser](/images/aws/neptune-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Cluster**: Create a new Neptune cluster by clicking on **Create Cluster** under the **Clusters** tab and providing the required parameters. - **List Clusters**: View a list of all Neptune clusters in your LocalStack environment by clicking on the **Clusters** tab. - **View Cluster Details**: Click on a cluster name to view detailed information about the cluster, including its status, endpoint, and other configuration details. - **Graph Browser**: Access the Neptune Graph Browser by clicking on the **Graph Browser** tab in the cluster details. The Graph Browser allows you to interactively query and visualize the graph data stored in your Neptune cluster. - **Quick Actions**: Perform quick actions on the cluster, such as adding a new Node, modifying an existing one or creating a new Edge between 2 nodes. You can access the **Quick Actions** by clicking in the respective tab from the cluster details page. - **Create instance**: Create a new Neptune database by clicking on **Create Instance** under the **Instances** tab and providing the required parameters. - **List Instances**: View a list of all Neptune databases in your LocalStack environment by clicking on the **Instances** tab. - **View Instance Details**: Click on a database name to view detailed information about the database, including its status, endpoint, and other configuration details. - **Edit Instance**: Edit the configuration of a Neptune database by clicking on the **Edit Instance** button in the instance details. ## Examples The following code snippets and sample applications provide practical examples of how to use Neptune in LocalStack for various use cases: - [Neptune Graph Database Demo](https://github.com/localstack/localstack-pro-samples/tree/master/neptune-graph-db) ## Preview Features ### Gremlin Transactions Gremlin transactions can be enabled by setting the environment `NEPTUNE_ENABLE_TRANSACTION=1`. Be aware that the `engine_version` provided when creating your cluster will be ignored and LocalStack will use `3.7.2` Gremlin Server. This feature is in beta and any feedback is appreciated. ## Current Limitations ### Fixed ID - If you create a vertex with an ID inside a transaction and then delete it, creating another vertex with the same ID will fail. ### Serializer - You can connect using older Gremlin Language Variants, but `GraphBinarySerializersV1` has breaking changes. - To fix this, either use the serializer version that matches your Gremlin variant, or switch to `GraphSONSerializersV3d0`, which works. - If using Neptune version `1.2.0.2` or earlier, the Gryo serializer is no longer supported. This only affects users who explicitly use it. Here is an example of how to use the `GraphSONSerializersV3d0` serializer with `gremlinpython==3.6.2`: ```python showshowLineNumbers from gremlin_python.driver import serializer from gremlin_python.driver.driver_remote_connection import DriverRemoteConnection from gremlin_python.process.anonymous_traversal import traversal ENDPOINT = "localhost:4510" # TODO change to your endpoint DATABASE_URL = f"ws://{ENDPOINT}/gremlin" if __name__ == '__main__': conn = DriverRemoteConnection( DATABASE_URL, "g", # Note, the serializer is only required if using gremplin_python < 3.7.0 message_serializer=serializer.GraphSONSerializersV3d0(), ) g = traversal().withRemote(conn) tx = g.tx() gtx = tx.begin() try: v1 = gtx.addV("person").property("name", "Mark").next() v2 = gtx.addV("person").property("name", "Jane").next() tx.commit() except Exception: tx.rollback() nodes = g.V().valueMap().fold().next() print(nodes) ``` ## API Coverage # OpenSearch Service > Get started with OpenSearch Service on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction OpenSearch Service is an open-source search and analytics engine, offering developers and organizations advanced search capabilities, robust data analysis, and insightful visualizations. OpenSearch Service also offers log analytics, real-time application monitoring, and clickstream analysis. LocalStack allows you to use the OpenSearch Service APIs in your local environment to create, manage, and operate the OpenSearch clusters. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of OpenSearch's integration with LocalStack. The following versions of OpenSearch Service are supported by LocalStack: - 1.0 - 1.1 - 1.2 - 1.3 - 2.3 - 2.7 - 2.9 - 2.11 - 2.13 - 2.15 - 2.17 - 2.19 - 3.1 - 3.3 - 3.5 - 3.7 (**default**) OpenSearch is closely coupled with the [Elasticsearch Service](/aws/services/es/). Clusters generated through the OpenSearch Service will be visible within the Elasticsearch Service interface, and vice versa. You can select an Elasticsearch version with the `--engine-version` parameter while creating an OpenSearch Service domain. ## Getting started This guide is designed for users new to OpenSearch Service and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a new OpenSearch Service cluster and interact with it, using the AWS CLI. ### Creating an OpenSearch cluster To create an OpenSearch Service cluster, you can use the [`CreateDomain`](https://docs.aws.amazon.com/opensearch-service/latest/APIReference/API_CreateDomain.html) API. OpenSearch Service domain is synonymous with an OpenSearch cluster. Execute the following command to create a new OpenSearch domain: ```bash lstk aws opensearch create-domain --domain-name my-domain ``` Each time you establish a cluster using a new version of OpenSearch, the corresponding OpenSearch binary must be downloaded, a process that might require some time to complete. In the LocalStack log you will see something like, where you can see the cluster starting up in the background. You can open the LocalStack logs, to see that the OpenSearch Service cluster is being created in the background. You can use the [`DescribeDomain`](https://docs.aws.amazon.com/opensearch-service/latest/APIReference/API_DescribeDomain.html) API to check the status of the cluster: ```bash lstk aws opensearch describe-domain \ --domain-name my-domain | jq ".DomainStatus.Processing" ``` The `Processing` attribute will be `false` once the cluster is up and running. Once the cluster is up, you can interact with the cluster. ### Interact with the cluster You can now interact with the cluster at the cluster API endpoint for the domain, in this case `http://my-domain.us-east-1.opensearch.localhost.localstack.cloud:4566`. Run the following command to get the cluster health: ```bash curl http://my-domain.us-east-1.opensearch.localhost.localstack.cloud:4566 ``` You can verify that the cluster is up and running by checking the cluster health: ```bash curl -s http://my-domain.us-east-1.opensearch.localhost.localstack.cloud:4566/_cluster/health | jq . ``` ```bash title="Output" { "cluster_name": "opensearch", "status": "green", "timed_out": false, "number_of_nodes": 1, "number_of_data_nodes": 1, "discovered_master": true, "active_primary_shards": 0, "active_shards": 0, "relocating_shards": 0, "initializing_shards": 0, "unassigned_shards": 0, "delayed_unassigned_shards": 0, "number_of_pending_tasks": 0, "number_of_in_flight_fetch": 0, "task_max_waiting_in_queue_millis": 0, "active_shards_percent_as_number": 100 } ``` ## Domain Endpoints There are two configurable strategies that govern how domain endpoints are created. The strategy can be configured via the `OPENSEARCH_ENDPOINT_STRATEGY` environment variable. | Value | Format | Description | | ------- | ----------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | | `domain` | `...localhost.localstack.cloud:4566` | The default strategy employing the `localhost.localstack.cloud` domain for routing to localhost. | | `path` | `localhost:4566///` | An alternative strategy useful if resolving LocalStack's localhost domain poses difficulties. | | `port` | `localhost:` | Directly exposes cluster(s) via ports from [the external service port range](/aws/customization/networking/external-port-range/). | Irrespective of the originating service for the clusters, the domain of each cluster consistently aligns with its engine type, be it OpenSearch or Elasticsearch. Consequently, OpenSearch clusters incorporate `opensearch` within their domains (e.g., `my-domain.us-east-1.opensearch.localhost.localstack.cloud:4566`), while Elasticsearch clusters feature `es` in their domains (e.g., `my-domain.us-east-1.es.localhost.localstack.cloud:4566`). ## Custom Endpoints LocalStack allows you to define arbitrary endpoints for your clusters within the domain endpoint options. This functionality can be used to overwrite the behavior of the aforementioned endpoint strategies. Moreover, you can opt for custom domains, though it's important to incorporate the edge port (80/443, or the default 4566). Run the following command to create a new OpenSearch domain with a custom endpoint: ```bash lstk aws opensearch create-domain --domain-name my-domain \ --domain-endpoint-options '{ "CustomEndpoint": "http://localhost:4566/my-custom-endpoint", "CustomEndpointEnabled": true }' ``` After the domain processing is complete, you can access the cluster using the custom endpoint: ```bash curl http://localhost:4566/my-custom-endpoint/_cluster/health ``` ## Re-using a single cluster instance In certain scenarios, creating a distinct cluster instance for each domain might not align with your use-case. For example, if your focus is solely on testing API interactions rather than actual OpenSearch functionality, individual clusters might be excessive. In such situations, the option to set `OPENSEARCH_MULTI_CLUSTER=0` exists, allowing all domains to be funneled into a single cluster instance. However, it's important to be aware that it can introduce unexpected complications. This is particularly true when dealing with data persistence within OpenSearch or when working with clusters of varying versions. As a result, we advise caution when considering this approach and generally recommend against it. ## Storage Layout OpenSearch will be organized in your state directory as follows: import { Tabs, TabItem, FileTree } from '@astrojs/starlight/components'; - volume - state - opensearch - arn:aws:es:us-east-1:000000000000:domain - my-cluster-1 - backup - data - tmp - my-cluster-2 - backup - data - tmp ## Advanced Security Options Both OpenSearch and Elasticsearch services offer **Advanced Security Options**. Presently, OpenSearch domains are equipped with support for an internal user database. However, Elasticsearch domains are not currently covered, whether through the OpenSearch or the Elasticsearch service. IAM support is also not yet available. A secure OpenSearch domain can be spawned with this example CLI input. Save it in a file named `opensearch_domain.json`. ```json title="opensearch_domain.json" showshowLineNumbers { "DomainName": "secure-domain", "ClusterConfig": { "InstanceType": "r5.large.search", "InstanceCount": 1, "DedicatedMasterEnabled": false, "ZoneAwarenessEnabled": false, "WarmEnabled": false }, "EBSOptions": { "EBSEnabled": true, "VolumeType": "gp2", "VolumeSize": 10 }, "EncryptionAtRestOptions": { "Enabled": true }, "NodeToNodeEncryptionOptions": { "Enabled": true }, "DomainEndpointOptions": { "EnforceHTTPS": true }, "AdvancedSecurityOptions": { "Enabled": true, "InternalUserDatabaseEnabled": true, "MasterUserOptions": { "MasterUserName": "admin", "MasterUserPassword": "really-secure-passwordAa!1" } } } ``` To provision it, use the following `lstk aws` CLI command, assuming the aforementioned CLI input has been stored in a file named `opensearch_domain.json`: ```bash lstk aws opensearch create-domain --cli-input-json file://./opensearch_domain.json ``` Once the domain setup is complete (`Processing: false`), the cluster can only be accessed with the given master user credentials, via HTTP basic authentication: ```bash curl -u 'admin:really-secure-passwordAa!1' http://secure-domain.us-east-1.opensearch.localhost.localstack.cloud:4566/_cluster/health ``` ```bash title="Output" {"cluster_name":"opensearch","status":"green",...} ``` It's important to note that any unauthorized requests will yield an HTTP response with a status code of 401 (`Unauthorized`). ## OpenSearch Dashboards [OpenSearch Dashboards](https://opensearch.org/docs/latest/dashboards/) is a great tool to analyze and visualize the data in your OpenSearch domain. And you can directly use the official OpenSearch Dashboards Docker image to analyze data in your OpenSearch domain within LocalStack! When using OpenSearch Dashboards with LocalStack, you need to make sure to: - Enable the [advanced security options](#advanced-security-options) and set a username and a password. This is required by OpenSearch Dashboards. - Ensure that the OpenSearch Dashboards Docker container uses the LocalStack DNS. You can find more information on how to connect your Docker container to Localstack in our [Network Troubleshooting guide](/aws/customization/networking/). First, you need to create a dedicated Docker network and start LocalStack within it (note: custom networks are not supported by the `lstk` CLI) ```bash docker network create ls docker run \ -d --rm \ --name localstack-main \ --network ls \ -p 127.0.0.1:4566:4566 \ -p 127.0.0.1:4510-4559:4510-4559 \ -v /var/run/docker.sock:/var/run/docker.sock \ localstack/localstack ``` Now you can provision a new OpenSearch domain. Make sure to enable the [advanced security options](#advanced-security-options): ```bash lstk aws opensearch create-domain --cli-input-json file://./opensearch_domain.json ``` Now you can start another container for OpenSearch Dashboards, which is configured such that: - The port for OpenSearch Dashboards is mapped (`5601`). - The container is in the same network as LocalStack. - The container uses the LocalStack DNS. - The OpenSearch Domain is set. - The OpenSearch credentials are set. - The version of OpenSearch Dashboards is the same as the OpenSearch domain. ```bash LS_IP=$(docker inspect localstack-main | \ jq -r '.[0].NetworkSettings.Networks | to_entries | .[].value.IPAddress') LS_NETWORK=$(docker inspect localstack-main | jq -r '.[0].NetworkSettings.Networks | to_entries | .[].key') docker run --rm -p 5601:5601 \ --network $LS_NETWORK \ --dns $LS_IP \ -e "OPENSEARCH_HOSTS=http://secure-domain.us-east-1.opensearch.localhost.localstack.cloud:4566" \ -e "OPENSEARCH_USERNAME=admin" -e 'OPENSEARCH_PASSWORD=really-secure-passwordAa!1' \ opensearchproject/opensearch-dashboards:3.1.0 ``` Once the container is running, you can reach OpenSearch Dashboards at `http://localhost:5601` and you can log in with your OpenSearch domain credentials. ## Custom OpenSearch backends LocalStack employs an asynchronous approach to download OpenSearch the first time you create an OpenSearch cluster. Consequently, you'll receive a prompt response from LocalStack initially, followed by the setup of your local OpenSearch cluster once the download and installation are completed. However, there might be scenarios where this behavior is not desirable. For instance, you may prefer to use an existing OpenSearch cluster that is already up and running. This approach can also prove beneficial when you require a cluster with a customized configuration that isn't supported by LocalStack. To tailor the OpenSearch backend according to your needs, you can initiate your own local OpenSearch cluster and then direct LocalStack to utilize it through the `OPENSEARCH_CUSTOM_BACKEND` environment variable. It's important to bear in mind that only a single backend configuration is possible, resulting in behavior akin to the approach of [re-using a single cluster instance](#re-using-a-single-cluster-instance). Here is a sample `docker-compose.yaml` file that contains a single-node OpenSearch cluster and a basic LocalStack setup. ```yaml showshowLineNumbers services: opensearch: container_name: opensearch image: opensearchproject/opensearch:1.1.0 environment: - node.name=opensearch - cluster.name=opensearch-docker-cluster - discovery.type=single-node - bootstrap.memory_lock=true - "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" - "DISABLE_SECURITY_PLUGIN=true" ports: - "9200:9200" ulimits: memlock: soft: -1 hard: -1 volumes: - data01:/usr/share/opensearch/data localstack: container_name: "${LOCALSTACK_DOCKER_NAME:-localstack-main}" image: localstack/localstack ports: - "127.0.0.1:4566:4566" # LocalStack Gateway - "127.0.0.1:4510-4559:4510-4559" # external services port range depends_on: - opensearch environment: - OPENSEARCH_CUSTOM_BACKEND=http://opensearch:9200 - DEBUG=${DEBUG:-0} volumes: - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" volumes: data01: driver: local ``` You can start the Docker Compose environment using the following command: ```bash docker-compose up -d ``` You can now create an OpenSearch cluster using the `lstk aws` CLI: ```bash lstk aws opensearch create-domain --domain-name my-domain ``` If the `Processing` status shows as `true`, the cluster isn't fully operational yet. You can use the `describe-domain` command to retrieve the current status: ```bash lstk aws opensearch describe-domain --domain-name my-domain ``` You can now verify cluster health and set up indices: ```bash curl my-domain.us-east-1.opensearch.localhost.localstack.cloud:4566/_cluster/health | jq ``` The output will provide insights into the cluster's health and version information. Finally create an example index using the following command: ```bash curl -X PUT my-domain.us-east-1.opensearch.localhost.localstack.cloud:4566/my-index ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing OpenSearch domains. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **OpenSearch Service** under the **Analytics** section. ![OpenSearch Resource Browser](/images/aws/opensearch-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Domain**: Create a new OpenSearch domain by clicking on the **Create Domain** button and providing the required details. - **View Domain Details**: Click on a domain to view its details, such as the domain name, status, endpoint, and configuration. - **Edit Domain**: Edit the configuration of a domain by clicking on domain name and then clicking on the **Edit Domain** button. - **Delete Domain**: Delete a domain by selecting the domain name and clicking on the **Actions** dropdown menu, then selecting **Remove Selected**. ## Current Limitations Internally, LocalStack makes use of the [OpenSearch Python client 2.x](https://github.com/opensearch-project/opensearch-py). The functionalities marked as deprecated in OpenSearch 1.x and subsequently removed in OpenSearch 2.x may not operate reliably when interacting with OpenSearch 1.x clusters through LocalStack. You can refer to the [compatibility documentation](https://github.com/opensearch-project/opensearch-py/blob/main/COMPATIBILITY.md) provided by the [OpenSearch Python client repository](https://github.com/opensearch-project/opensearch-py). AWS typically populates the `Endpoint` attribute of the cluster status only after the cluster is fully operational. In contrast, LocalStack provides the endpoint information immediately but retains `Processing = "true"` until the cluster initialization is complete. The `CustomEndpointOptions` in LocalStack offers the flexibility to utilize arbitrary endpoint URLs, a feature that diverges from the constraints imposed by AWS. ## Troubleshooting If you encounter difficulties resolving subdomains while employing the `OPENSEARCH_ENDPOINT_STRATEGY=domain` (the default setting), it's advisable to investigate whether your DNS configuration might be obstructing rebind queries. For further insights on addressing this issue, refer to the section on [DNS rebind protection](/aws/customization/networking/dns-server#dns-rebind-protection). ## API Coverage # Organizations > Get started with AWS Organizations on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Amazon Web Services Organizations is an account management service that allows you to consolidate multiple different AWS accounts into an organization. It allows you to manage different accounts in a single organization and consolidate billing. With Organizations, you can also attach different policies to your organizational units (OUs) or individual accounts in your organization. Organizations is available over LocalStack for AWS and the supported APIs are available over our [configuration page](/aws/customization/configuration-options). ## Getting started In this getting started guide, you'll learn how to create your local AWS Organization and configure it with member accounts. This guide is intended for users who wish to get more acquainted with Organizations, and assumes you have basic knowledge of the AWS CLI (and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command). To get started, start your LocalStack instance using your preferred method: 1. Create a new local AWS Organization with the feature set flag set to `ALL`: ```bash lstk aws organizations create-organization --feature-set ALL ``` 2. You can now run the `describe-organization` command to see the details of your organization: ```bash lstk aws organizations describe-organization ``` 3. You can now create an AWS account that would be a member of your organization: ```bash lstk aws organizations create-account \ --email example@example.com \ --account-name "Test Account" ``` Since LocalStack essentially mocks AWS, the account creation is instantaneous. You can now run the `list-accounts` command to see the details of your organization: ```bash lstk aws organizations list-accounts ``` 4. You can also remove a member account from your organization: ```bash lstk aws organizations remove-account-from-organization --account-id ``` 5. To close an account in your organization, you can run the `close-account` command: ```bash lstk aws organizations close-account --account-id 000000000000 ``` 6. You can use organizational units (OUs) to group accounts together to administer as a single unit. To create an OU, you can run: ```bash lstk aws organizations list-roots lstk aws organizations list-children \ --parent-id \ --child-type ORGANIZATIONAL_UNIT lstk aws organizations create-organizational-unit \ --parent-id \ --name New-Child-OU ``` 7. Before you can create and attach a policy to your organization, you must enable a policy type. To enable a policy type, you can run: ```bash lstk aws organizations enable-policy-type \ --root-id \ --policy-type SERVICE_CONTROL_POLICY ``` To disable a policy type, you can run: ```bash lstk aws organizations disable-policy-type \ --root-id \ --policy-type SERVICE_CONTROL_POLICY ``` 8. To view the policies that are attached to your organization, you can run: ```bash lstk aws organizations list-policies --filter SERVICE_CONTROL_POLICY ``` 9. To delete an organization, you can run: ```bash lstk aws organizations delete-organization ``` ## Service Control Policy enforcement [Service Control Policies (SCPs)](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps.html) set the maximum permissions for accounts in your organization. When IAM enforcement is enabled, LocalStack checks SCPs together with other applicable policies. A request goes through only if both the principal's policies, resource's policies and the SCPs covering its account allow the action on the resource. To turn on SCP enforcement, start LocalStack with [`ENFORCE_IAM=1`](/aws/developer-tools/security-testing/iam-policy-enforcement) and enable the `SERVICE_CONTROL_POLICY` policy type on your organization root (see the [getting started](#getting-started) steps above). LocalStack evaluates SCPs at each level of the organization hierarchy: root, organizational unit, and account. An action must be allowed by an SCP at every level between the root and the account. If any level lacks an `Allow`, the result is an implicit deny, and an explicit `Deny` overrides any `Allow`. :::note The organization's management (master) account is exempt from SCPs. Principals in the management account are never restricted by SCPs, even with an explicit `Deny` SCP attached. ::: ### Cross-account access LocalStack enforces SCPs for [cross-account access](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_evaluation-logic-cross-account.html), where a principal in one account uses a resource owned by another account. For a cross-account request, LocalStack checks the SCPs of the source account (the account making the request). A deny in those SCPs blocks the request even when the target resource's policy grants access. Consider a member account that lists the objects of an S3 bucket in another account of the same organization, with a bucket policy that grants the member account access: ```bash # Run as the member (source) account lstk aws s3api list-objects-v2 --bucket cross-account-bucket ``` The default `FullAWSAccess` SCP lets the request succeed on the bucket policy. Attach an SCP that denies `s3:ListBucket` to the member account, and the request fails: ```bash title="Output" An error occurred (AccessDenied) when calling the ListObjectsV2 operation: User: arn:aws:iam::111111111111:user/test is not authorized to perform: s3:ListBucket on resource: "arn:aws:s3:::cross-account-bucket" with an explicit deny in a service control policy ``` An SCP that allows only unrelated actions (for example, an `ec2:*`-only SCP) produces an implicit deny, since no SCP allows `s3:ListBucket`: ```bash title="Output" An error occurred (AccessDenied) when calling the ListObjectsV2 operation: User: arn:aws:iam::111111111111:user/test is not authorized to perform: s3:ListBucket on resource: "arn:aws:s3:::cross-account-bucket" because no service control policy allows the s3:ListBucket action ``` The management account stays exempt from SCPs for cross-account requests too. ### Testing SCPs with the IAM Policy Simulator You can use the [IAM Policy Simulator](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_testing-policies.html) to check whether a principal's request would be allowed or denied without running it against your resources. LocalStack evaluates SCPs during policy simulation, so you can validate SCP behavior with [`SimulatePrincipalPolicy`](https://docs.aws.amazon.com/IAM/latest/APIReference/API_SimulatePrincipalPolicy.html) before making live requests. See the [IAM Policy Simulator documentation](/aws/developer-tools/security-testing/iam-policy-simulator/) for general usage and response limitations; this section focuses on SCP-specific behavior. ```bash lstk aws iam simulate-principal-policy \ --policy-source-arn arn:aws:iam::111111111111:user/test \ --action-names s3:ListBucket \ --resource-arns arn:aws:s3:::cross-account-bucket ``` When SCPs affect the decision, LocalStack populates the `OrganizationsDecisionDetail` field of the [`EvaluationResult`](https://docs.aws.amazon.com/IAM/latest/APIReference/API_EvaluationResult.html), so you can see whether an SCP allowed the action. :::note LocalStack's Policy Simulator and its IAM enforcement engine share the same underlying evaluation logic. As a result, the simulator reflects the real AWS IAM behavior rather than the behavior of the AWS Policy Simulator, which differs in a few ways: - AWS ignores SCPs that contain conditions during simulation. LocalStack evaluates SCPs with conditions. - AWS applies SCPs to the organization's management account during simulation. LocalStack does not apply SCPs to the management account, matching the real behavior of AWS Organizations. - AWS reports an explicit `Deny` from an SCP as an implicit deny. LocalStack reports it as an explicit deny, which is the expected outcome. ::: ## Service Control Policy validation When you create or update a Service Control Policy (SCP) with `CreatePolicy` or `UpdatePolicy`, LocalStack validates the policy document against the syntax rules that SCPs must follow. A policy that violates any of these rules is rejected with a `MalformedPolicyDocumentException` or a `ConstraintViolationException`, matching the behavior you would see on AWS. The following constraints are enforced for SCPs: - **No `Principal` or `NotPrincipal` elements**: unlike identity-based or resource-based IAM policies, SCPs cannot specify a `Principal` or `NotPrincipal` key inside a `Statement`. A policy that includes either key is rejected with a `MalformedPolicyDocumentException`. - **Maximum policy size of 10,240 characters**: a policy document larger than 10,240 characters is rejected with a `ConstraintViolationException` (reason `POLICY_CONTENT_LIMIT_EXCEEDED`). This matches the limit enforced by AWS in practice. - **A single policy object**: the document must be a single JSON object. Passing a JSON array of policy objects is rejected. - **A single `Statement` key**: the document may contain only one `Statement` key. Duplicate `Statement` keys (or any other duplicate keys) cause the policy to be rejected as malformed. - **Resources must be present**: each statement must contain a `Resource` element. Specific resource ARNs (not just the `*` wildcard) are accepted. :::note The AWS documentation states that the [maximum SCP size is 5,120 characters](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_reference_limits.html) and that SCPs [only support the `*` wildcard for resources](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_scps_syntax.html#scp-syntax-resource). LocalStack instead mirrors the behavior observed against real AWS: the enforced size limit is 10,240 characters, and specific resource ARNs are accepted. ::: For example, the following policy is rejected because it includes a `Principal` element, which is not permitted in an SCP: ```bash lstk aws organizations create-policy \ --name "InvalidSCP" \ --description "SCP with a Principal element" \ --type SERVICE_CONTROL_POLICY \ --content '{ "Version": "2012-10-17", "Statement": [ { "Effect": "Deny", "Principal": "*", "Action": "s3:*", "Resource": "*" } ] }' ``` ## API Coverage # Pinpoint > Get started with Pinpoint on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; :::caution Amazon Pinpoint will be [retired on 30 October 2026](https://docs.aws.amazon.com/pinpoint/latest/userguide/migrate.html) and will be removed from LocalStack soon after this date. ::: ## Introduction Pinpoint is a customer engagement service to facilitate communication across multiple channels, including email, SMS, and push notifications. Pinpoint allows developers to create and manage customer segments based on various attributes, such as user behavior and demographics, while integrating with other AWS services to send targeted messages to customers. LocalStack allows you to mock the Pinpoint APIs in your local environment. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Pinpoint's integration with LocalStack. ## Getting started This guide is designed for users new to Pinpoint and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a Pinpoint application, retrieve all applications, and list tags for the resource. ### Create an application Create a Pinpoint application using the [`CreateApp`](https://docs.aws.amazon.com/pinpoint/latest/apireference/apps-application-id.html) API. Execute the following command: ```bash lstk aws pinpoint create-app \ --create-application-request Name=ExampleCorp,tags={"Stack"="Test"} ``` ```bash title="Output" { "ApplicationResponse": { "Arn": "arn:aws:mobiletargeting:us-east-1:000000000000:apps/4487a55ac6fb4a2699a1b90727c978e7", "Id": "4487a55ac6fb4a2699a1b90727c978e7", "Name": "ExampleCorp", "CreationDate": 1706609789.906863 } } ``` ### List applications You can list all applications using the [`GetApps`](https://docs.aws.amazon.com/pinpoint/latest/apireference/apps.html) API. Execute the following command: ```bash lstk aws pinpoint get-apps ``` ```bash title="Output" { "ApplicationsResponse": { "Item": [ { "Arn": "arn:aws:mobiletargeting:us-east-1:000000000000:apps/4487a55ac6fb4a2699a1b90727c978e7", "Id": "4487a55ac6fb4a2699a1b90727c978e7", "Name": "ExampleCorp", "CreationDate": 1706609789.906863 } ] } } ``` ### List tags for the application You can list all tags for the application using the [`GetApp`](https://docs.aws.amazon.com/pinpoint/latest/apireference/apps-application-id.html) API. Execute the following command: ```bash lstk aws pinpoint list-tags-for-resource \ --resource-arn arn:aws:mobiletargeting:us-east-1:000000000000:apps/4487a55ac6fb4a2699a1b90727c978e7 ``` Replace the `resource-arn` with the ARN of the application you created earlier. ```bash title="Output" { "TagsModel": { "tags": { "Stack": "Test" } } } ``` ### OTP verification The operations [`SendOTPMessage`](https://docs.aws.amazon.com/pinpoint/latest/apireference/apps-application-id-otp.html#SendOTPMessage) and [`VerifyOTPMessage`](https://docs.aws.amazon.com/pinpoint/latest/apireference/apps-application-id-verify-otp.html#VerifyOTPMessage) are used for one-time password (OTP) verification. On production AWS, `SendOTPMessage` sends an SMS text message with the OTP code. The OTP can then be verified against the reference ID using `VerifyOTPMessage` LocalStack however can not send real SMS text messages. Instead it provides alternative ways to retrieve the actual OTP code as illustrated below. Begin by making a OTP request: ```bash lstk aws pinpoint send-otp-message \ --application-id fff5a801e01643c18a13a763e22a8fbf \ --send-otp-message-request-parameters '{ "BrandName": "LocalStack Pro", "Channel": "SMS", "DestinationIdentity": "+1224364860", "ReferenceId": "liftoffcampaign", "OriginationIdentity": "+1123581321", "CodeLength": 6, "AllowedAttempts": 3, "ValidityPeriod": 2 }' ``` ```bash title="Output" { "MessageResponse": { "ApplicationId": "fff5a801e01643c18a13a763e22a8fbf" } } ``` You can use the debug endpoint `/_aws/pinpoint//` to retrieve the OTP message details: ```bash curl http://localhost:4566/_aws/pinpoint/fff5a801e01643c18a13a763e22a8fbf/liftoffcampaign | jq . ``` ```bash title="Output" { "AllowedAttempts": 3, "BrandName": "LocalStack Pro", "CodeLength": 6, "DestinationIdentity": "+1224364860", "OriginationIdentity": "+1123581321", "ReferenceId": "liftoffcampaign", "ValidityPeriod": 2, "Attempts": 0, "ApplicationId": "fff5a801e01643c18a13a763e22a8fbf", "CreatedTimestamp": "2024-10-17T05:38:24.070Z", "Code": "655745" } ``` The OTP code is also printed in an `INFO` level message in the LocalStack log output: ```bash title="Output" 2024-10-17T11:08:24.044 INFO : OTP for application ID fff5a801e01643c18a13a763e22a8fbf reference ID liftoffcampaign: 655745 ``` Finally, the OTP code can be verified using: ```bash lstk aws pinpoint verify-otp-message \ --application-id fff5a801e01643c18a13a763e22a8fbf \ --verify-otp-message-request-parameters '{ "ReferenceId": "liftoffcampaign", "DestinationIdentity": "+1224364860", "Otp": "655745" }' ``` ```bash title="Output" { "VerificationResponse": { "Valid": true } } ``` When validating OTP codes, LocalStack checks for the number of allowed attempts and the validity period. Unlike AWS, there is no lower limit for validity period. ## API Coverage # EventBridge Pipes > Get started with EventBridge Pipes on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction EventBridge Pipes allows users to create point-to-point integrations between event producers and consumers with transform, filter and enrichment steps. Pipes are particularly useful for scenarios involving real-time data processing, application integration, and automated workflows, while simplifying the process of routing events between AWS services. Pipes offer a point-to-point connection from one source to one target (one-to-one). In contrast, EventBridge Event Bus offers a one-to-many integration where an event router delivers one event to zero or more destinations. LocalStack allows you to use the Pipes APIs in your local environment to create Pipes with SQS queues and Kinesis streams as source and target. You can also filter events using EventBridge event patterns and enrich events using Lambda. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Pipe's integration with LocalStack. :::note The implementation of EventBridge Pipes is currently in **preview** stage and under active development. If you would like support for more APIs or report bugs, please make a request on [GitHub Discussion](https://github.com/orgs/localstack/discussions/new/choose). ::: ## Getting started This guide is designed for users new to EventBridge Pipes and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a Pipe with SQS queues as source and target, and send events to the source queue which will be routed to the target queue. ### Create an SQS queue Create two SQS queues that will be used as source and target for the Pipe. Run the following command to create a queue using the [`CreateQueue`](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_CreateQueue.html) API: ```bash lstk aws sqs create-queue --queue-name source-queue lstk aws sqs create-queue --queue-name target-queue ``` You can fetch their queue ARNs using the [`GetQueueAttributes`](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_GetQueueAttributes.html) API: ```bash SOURCE_QUEUE_ARN=$(lstk aws sqs get-queue-attributes --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/source-queue --attribute-names QueueArn --output text) TARGET_QUEUE_ARN=$(lstk aws sqs get-queue-attributes --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/target-queue --attribute-names QueueArn --output text) ``` ### Create a Pipe You can now create a Pipe, using the [`CreatePipe`](https://docs.aws.amazon.com/eventbridge/latest/APIReference/API_CreatePipe.html) API. Run the following command, by specifying the source and target queue ARNs we created earlier: ```bash lstk aws pipes create-pipe --name sample-pipe \ --source $SOURCE_QUEUE_ARN \ --target $TARGET_QUEUE_ARN \ --role-arn arn:aws:iam::000000000000:role/pipes-role ``` ```bash title="Output" { "Arn": "arn:aws:pipes:us-east-1:000000000000:pipe/sample-pipe", "CreationTime": "2024-01-26T11:55:27.069088+05:30", "CurrentState": "CREATING", "DesiredState": "RUNNING", "LastModifiedTime": "2024-01-26T11:55:27.069088+05:30", "Name": "sample-pipe" } ``` ### Describe the Pipe You can use the [`DescribePipe`](https://docs.aws.amazon.com/eventbridge/latest/APIReference/API_DescribePipe.html) API to get information about the Pipe: ```bash lstk aws pipes describe-pipe --name sample-pipe ``` ```bash title="Output" { "Arn": "arn:aws:pipes:us-east-1:000000000000:pipe/sample-pipe", "CreationTime": "2024-01-26T11:55:27.069088+05:30", "CurrentState": "RUNNING", "DesiredState": "RUNNING", "EnrichmentParameters": {}, "LastModifiedTime": "2024-01-26T11:55:27.069088+05:30", "Name": "sample-pipe", "RoleArn": "arn:aws:iam::000000000000:role/pipe-role", "Source": "arn:aws:sqs:us-east-1:000000000000:source-queue", "SourceParameters": { "SqsQueueParameters": { "BatchSize": 10 } }, "StateReason": "USER_INITIATED", "Tags": {}, "Target": "arn:aws:sqs:us-east-1:000000000000:target-queue", "TargetParameters": {} } ``` ### Send events to the source queue You can now send events to the source queue, which will be routed to the target queue. Run the following command to send an event to the source queue: ```bash lstk aws sqs send-message \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/source-queue \ --message-body "message-1" ``` ### Receive events from the target queue You can fetch the message from the target queue using the [`ReceiveMessage`](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_ReceiveMessage.html) API: ```bash lstk aws sqs receive-message \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/target-queue ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing EventBridge Pipes. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resource Browser** section, and then clicking on **EventBridge Pipes** under the **App Integration** section. ![EventBridge Pipes Resource Browser](/images/aws/pipes-resource-browser.png) The Resource Browser for EventBridge Pipes in LocalStack allows you to perform the following actions: 1. **Create a Pipe**: Click on the **Create Pipe** button to set up a new pipe with a source and target service, filter criteria, and more. 2. **View Pipe Details**: Click on the pipe name to view detailed information, including source, target, batch size, state, and more. 3. **Delete a Pipe**: Select a pipe and click on the **Actions** dropdown menu, followed by **Remove Selected**, to delete the pipe. ## Supported sources LocalStack supports the following [sources](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-event-source.html) for Pipes: * Amazon DynamoDB stream * Amazon Kinesis stream * Amazon SQS queue Please create a feature request on [GitHub](https://github.com/orgs/localstack/discussions) if you miss support for Amazon MQ broker, Amazon MSK stream, or Apache Kafka stream. ## Supported enrichments LocalStack supports the following [enrichments](https://docs.aws.amazon.com/eventbridge/latest/userguide/pipes-enrichment.html) for Pipes: * Lambda function Please create a feature request on [GitHub](https://github.com/orgs/localstack/discussions) if you miss support for API destination, Amazon API Gateway, or Step Functions state machine ## Supported targets LocalStack supports the following [targets](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-event-target.html) for Pipes: * EventBride bus * Kinesis stream * Lambda function (SYNC or ASYNC) * Amazon SNS topic * Amazon SQS queue * Step Functions state machine * Standard workflows (ASYNC) Please create a feature request on [GitHub](https://github.com/orgs/localstack/discussions) if you miss support for API destination, API Gateway, Batch job queue, CloudWatch log group, ECS task, Firehose delivery stream, Inspector assessment template, Redshift cluster data API queries, SageMaker Pipeline, Step Functions state machine: Express workflows (SYNC or ASYNC), or Timestream for LiveAnalytics table. ## Input transformation LocalStack supports target and enrichment input transformation for Pipes using JSONPath. Wildcards (*) are also supported. ## Supported log destinations LocalStack supports the following [log destinations](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-logs.html) for detailed Pipes logging: * CloudWatch Logs Please create a feature request on [GitHub](https://github.com/orgs/localstack/discussions) if you miss support for Firehose stream logs, or Amazon S3 logs. ## Current Limitations The EventBridge Pipes implementation in LocalStack is currently in preview stage and has the following limitations: - Lack of concurrency support (i.e., ParallelizationFactor), resulting in slower processing in high-throughput scenarios. - Lack of lifecycle management for pipe states (i.e., missing tests for state transitions). - Lack of re-sharding support when polling from Kinesis and DynamoDB streams. - Batch handling behavior may have parity issues (e.g., batch flushing rules by size, length, time, etc. are not implemented). ## API Coverage # Resource Access Manager (RAM) > Get started with RAM on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Resource Access Manager (RAM) helps resources to be shared across AWS accounts, within or across organizations. On AWS, RAM is an abstraction on top of AWS Identity and Access Management (IAM) which can manage resource-based policies to supported resource types. The API operations supported by LocalStack can be found on the [API Coverage section](#api-coverage), which provides information on the extent of RAM's integration with LocalStack. ## Getting started Start the LocalStack container using your preferred method. This section will illustrate how to create permissions and resource shares using the AWS CLI. ### Create a permission ```bash lstk aws ram create-permission \ --name example \ --resource-type appsync:apis \ --policy-template '{"Effect": "Allow", "Action": "appsync:SourceGraphQL"}' ``` ### Create a resource share ```bash lstk aws ram create-resource-share \ --name example-resource-share \ --principals arn:aws:organizations::000000000000:organization/o-truopwybwi \ --resource-arn arn:aws:appsync:eu-central-1:000000000000:apis/wcgmjril5wuyvhmpildatuaat3 ``` ## Current Limitations LocalStack RAM supports emulated sharing for EC2 Subnets only. Only specified account principals are granted access to the shared subnets, and associated VPC and route tables. Furthermore, only the sharing aspect is implemented at this time. No IAM policies are created or attached, and no permission enforcement takes place. For all other resource types, the functionality is limited to mocking. ## IAM Condition Keys When [IAM Policy Enforcement](/aws/developer-tools/security-testing/iam-policy-enforcement/) is enabled, LocalStack supports the following RAM-specific condition key, matching the behavior described in the [AWS condition keys reference](https://docs.aws.amazon.com/service-authorization/latest/reference/list_awsresourceaccessmanager.html#awsresourceaccessmanager-policy-keys): - `ram:RequestedAllowsExternalPrincipals` — the `allowExternalPrincipals` value of a `CreateResourceShare` or `UpdateResourceShare` request, useful for restricting resource shares to principals within your organization. For example, the following policy statement only allows creating or updating a resource share when it does not allow external principals: ```json { "Effect": "Allow", "Action": ["ram:CreateResourceShare", "ram:UpdateResourceShare"], "Resource": "*", "Condition": { "Bool": { "ram:RequestedAllowsExternalPrincipals": "false" } } } ``` ## API Coverage # Relational Database Service (RDS) > Get started with Relational Database Service (RDS) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Relational Database Service (RDS) is a managed database service provided by Amazon Web Services (AWS) that allows users to setup, operate, and scale relational databases in the cloud. RDS allows you to deploy and manage various relational database engines like MySQL, PostgreSQL, MariaDB, and Microsoft SQL Server. RDS handles routine database tasks such as provisioning, patching, backup, recovery, and scaling. LocalStack allows you to use the RDS APIs in your local environment to create and manage RDS clusters and instances for testing & integration purposes. The supported APIs are available on the API coverage section for [RDS](#api-coverage) and [RDS Data](#api-coverage-rds-data), which provides information on the extent of RDS's integration with LocalStack. :::note We’ve introduced a new native RDS provider in LocalStack and made it the default. This replaces Moto-based CRUD operations with a more reliable setup. RDS state created in version 4.3 or earlier using Cloud Pods or standard persistence will not be compatible with the new provider introduced in version 4.4. Recreating the RDS state is recommended for compatibility. ::: ## Getting started This guide is designed for users new to RDS and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate the following with the AWS CLI: 1. Creating an RDS cluster. 2. Generating a `SecretsManager` secret containing the database password. 3. Executing a basic `SELECT 123 query` through the RDS Data API. LocalStack's RDS implementation also supports the [RDS Data API](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/data-api.html), which allows executing data queries against RDS clusters over a JSON/REST interface. ### Create an RDS cluster To create an RDS cluster, you can use the [`CreateDBCluster`](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_CreateDBCluster.html) API. The following command creates a new cluster with the name `db1` and the engine `aurora-postgresql`. Instances for the cluster must be added manually. ```bash lstk aws rds create-db-cluster \ --db-cluster-identifier db1 \ --engine aurora-postgresql \ --database-name test \ --master-username myuser \ --master-user-password mypassword ``` ```bash title="Output" { "DBCluster": { ... "Endpoint": "localhost", "Port": 4510, # may vary "DBClusterArn": "arn:aws:rds:us-east-1:000000000000:cluster:db1", ... } } ``` To add an instance you can run the following command: ```bash lstk aws rds create-db-instance \ --db-instance-identifier db1-instance \ --db-cluster-identifier db1 \ --engine aurora-postgresql \ --db-instance-class db.t3.large ``` ### Create a SecretsManager secret To create a `SecretsManager` secret, you can use the [`CreateSecret`](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_CreateSecret.html) API. Before creating the secret, you need to create a JSON file containing the credentials for the database. The following command creates a file called `mycreds.json` with the credentials for the database. ```bash cat << 'EOF' > mycreds.json { "engine": "aurora-postgresql", "username": "myuser", "password": "mypassword", "host": "localhost", "dbname": "test", "port": "4510" } EOF ``` Run the following command to create the secret: ```bash lstk aws secretsmanager create-secret \ --name dbpass \ --secret-string file://mycreds.json ``` ```bash title="Output" { "ARN": "arn:aws:secretsmanager:us-east-1:000000000000:secret:dbpass-cfnAX", "Name": "dbpass", "VersionId": "fffa1f4a-2381-4a2b-a977-4869d59a16c0" } ``` ### Execute a query To execute a query, you can use the [`ExecuteStatement`](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_ExecuteStatement.html) API. Make sure to replace the `secret-arn` with the ARN from the secret you just created in the previous step, and check that the `resource-arn` matches the `cluster-arn` that you have created before. The following command executes a query against the database. The query returns the value `123`. ```bash lstk aws rds-data execute-statement \ --database test \ --resource-arn arn:aws:rds:us-east-1:000000000000:cluster:db1 \ --secret-arn arn:aws:secretsmanager:us-east-1:000000000000:secret:dbpass-cfnAX \ --include-result-metadata --sql 'SELECT 123' ``` ```bash title="Output" { "columnMetadata": [ { "arrayBaseColumnType": 0, "isAutoIncrement": false, "isCaseSensitive": false, "isCurrency": false, "isSigned": true, "label": "?column?", "name": "?column?", "nullable": 0, "precision": 10, "scale": 0, "schemaName": "", "tableName": "", "type": 4, "typeName": "int4" } ], "numberOfRecordsUpdated": 0, "records": [ [ { "longValue": 123 } ] ] } ``` Alternative clients, such as `psql`, can also be employed to interact with the database. You can retrieve the hostname and port of your created instance either from the preceding output or by using the [`DescribeDbInstances`](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_DescribeDBInstances.html) API. ```bash psql -d test -U test -p 4513 -h localhost -W ``` ## Supported DB engines Presently, you can spin up PostgreSQL, MariaDB, MySQL, and MSSQL (SQL Server) databases directly on your local machine, using LocalStack's RDS implementation. However, certain configurations of RDS clusters and instances currently offer only CRUD functionality. For instance, the `storage-encrypted` flag is returned as configured, but active support for actual storage encryption is not yet available. ### PostgreSQL Engine When you establish an RDS DB cluster or instance using the `postgres`/`aurora-postgresql` DB engine along with a specified `EngineVersion`, LocalStack will dynamically install and configure the corresponding PostgreSQL version as required. Presently, you have the option to choose major versions ranging from 13 to 17. If you select a major version beyond this range, the system will automatically default to version 17. It's important to note that the selection of minor versions is not available. The latest major version will be installed within the Docker environment. If you wish to prevent the installation of customized versions, adjusting the `RDS_PG_CUSTOM_VERSIONS` environment variable to `0` will enforce the use of the default PostgreSQL version 17. :::note PostgreSQL 11 and 12 are no longer available in LocalStack following the move to a Debian `sid` base image. Both versions remain in [Amazon RDS Extended Support](https://docs.aws.amazon.com/AmazonRDS/latest/PostgreSQLReleaseNotes/postgresql-release-calendar.html), but LocalStack no longer ships them. If you require one of these older versions, use a LocalStack image released before v2026.05.0. ::: :::note While the [`DescribeDbCluster`](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_DescribeDBClusters.html) and [`DescribeDbInstances`](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_DescribeDBInstances.html) APIs will still reflect the initially defined `engine-version`, the actual installed PostgreSQL engine might differ. This can have implications, particularly when employing a Terraform configuration, where unexpected changes should be avoided. ::: Instances and clusters with the PostgreSQL engine have the capability to both create and restore snapshots. ### MariaDB Engine MariaDB will be set up as an operating system package within LocalStack. However, currently, the option to choose a particular version is not available. As of now, snapshots are not supported for MariaDB. ### MySQL Engine A MySQL server will be launched in a new Docker container upon requesting the MySQL engine. The `engine-version` will serve as the tag for the Docker image, allowing you to freely select the desired MySQL version from those available on the [official MySQL Docker Hub](https://hub.docker.com/_/mysql). If you have a specific image in mind, you can also use the environment variable `MYSQL_IMAGE=`. :::note The `arm64` MySQL images are limited to newer versions. For more information about availability, check the [MySQL Docker Hub repository](https://hub.docker.com/_/mysql). ::: It's essential to understand that the `MasterUserPassword` you define for the database cluster/instance will be used as the `MYSQL_ROOT_PASSWORD` environment variable for the `root` user within the MySQL container. The user specified in `MasterUserName` will use the same password and will have complete access to the database. As of now, snapshots are not supported for MySQL. ### Microsoft SQL Server Engine To utilize MSSQL databases, it's necessary to expressly agree to the terms of the [Microsoft SQL Server End-User Licensing Agreement (EULA)](https://hub.docker.com/_/microsoft-mssql-server) by configuring `MSSQL_ACCEPT_EULA=Y` within the LocalStack container environment. The `arm64` architecture is not currently officially supported for MSSQL. For the MSSQL engine, the database server is initiated in a fresh Docker container using the `latest` image. As of now, snapshots are not supported for MSSQL. ## Default Usernames and Passwords The following details concern default usernames, passwords, and database names for local RDS clusters created by LocalStack: - The default values for `master-username` and `db-name` are both **test**. For the `master-user-password`, the default is **test**, except for MSSQL databases, which employ **Test123!** as the default master password. - When setting up a new RDS instance, you have the flexibility to utilize any `master-username`, with the exception of **postgres**. The system will automatically generate the user. - It's important to remember that the username **postgres** has special significance, preventing the creation of a new RDS instance under this particular name. - Using the `db-name` **postgres** might lead to issues for older versions of LocalStack, please try to avoid using it. ## IAM Authentication Support IAM authentication tokens can be employed to establish connections with RDS. As of now, this functionality is supported for PostgreSQL within LocalStack. However, IAM authentication is not yet validated at this stage. Consequently, any database user assigned the `rds_iam` role will obtain a valid token, thereby gaining the ability to connect to the database. In this example, you will be able to verify the IAM authentication process for RDS Postgres: 1. Establish a database instance and obtain the corresponding host and port information. 2. Connect to the database using the master username and password. Subsequently, generate a new user and assign the `rds_iam` role as follows: - `CREATE USER WITH LOGIN` - `GRANT rds_iam TO ` 3. Create a token for the `` using the `generate-db-auth-token` command. 4. Connect to the database utilizing the user you generated and the token obtained in the previous step as the password. ### Create a database instance The following command creates a new database instance with the name `mydb` and the engine `postgres`. The database will be created with a single instance, which will be used as the master instance. ```bash MASTER_USER=hello MASTER_PW='MyPassw0rd!' DB_NAME=test lstk aws rds create-db-instance \ --master-username $MASTER_USER \ --master-user-password $MASTER_PW \ --db-instance-identifier mydb \ --engine postgres \ --db-name $DB_NAME \ --enable-iam-database-authentication \ --db-instance-class db.t3.small ``` ### Connect to the database You can retrieve the hostname and port of your created instance either from the preceding output or by using the [`DescribeDbInstances`](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_DescribeDBInstances.html) API. Run the following command to retrieve the host and port of the instance: ```bash PORT=$(lstk aws rds describe-db-instances --db-instance-identifier mydb | jq -r ".DBInstances[0].Endpoint.Port") HOST=$(lstk aws rds describe-db-instances --db-instance-identifier mydb | jq -r ".DBInstances[0].Endpoint.Address") ``` Next, you can connect to the database using the master username and password: ```bash PGPASSWORD=$MASTER_PW psql -d $DB_NAME -U $MASTER_USER -p $PORT -h $HOST -w -c 'CREATE USER myiam WITH LOGIN' PGPASSWORD=$MASTER_PW psql -d $DB_NAME -U $MASTER_USER -p $PORT -h $HOST -w -c 'GRANT rds_iam TO myiam' ``` ### Create a token You can create a token for the user you generated using the [`generate-db-auth-token`](https://docs.aws.amazon.com/cli/latest/reference/rds/generate-db-auth-token.html) command: ```bash TOKEN=$(lstk aws rds generate-db-auth-token --username myiam --hostname $HOST --port $PORT) ``` You can now connect to the database utilizing the user you generated and the token obtained in the previous step as the password: ```bash PGPASSWORD=$TOKEN psql -d $DB_NAME -U myiam -w -p $PORT -h $HOST ``` ## SSL/TLS Support LocalStack's RDS PostgreSQL emulation supports SSL/TLS-encrypted client connections, so you can test applications that require `sslmode=require`. SSL/TLS support is currently available for the `postgres` engine. ### Connect using SSL Once your DB instance is running, request an encrypted connection from any PostgreSQL client by passing the `sslmode` parameter. With `psql`: ```bash PGPASSWORD=$MASTER_PW psql "host=$HOST port=$PORT dbname=$DB_NAME user=$MASTER_USER sslmode=require" ``` Certificate verification with `sslmode=verify-ca` or `sslmode=verify-full` is not currently supported. ### Limitations LocalStack currently enables SSL/TLS connections for PostgreSQL DB instances, but does not enforce SSL-only connections. The `rds.force_ssl` parameter is accepted for compatibility, but it is not enforced. Clients can still connect without SSL. :::note The PostgreSQL `pg_stat_ssl` view always reports `ssl = false`, even when the client connection is encrypted. ::: ## Global Database Support LocalStack extends support for [Aurora Global Database](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) with certain limitations: - Creating a global database will result in the generation of a single local database. All clusters and instances associated with the global database will share a common endpoint. - It's important to note that clusters removed from a global database lose their ability to function as standalone clusters, differing from their intended behavior on AWS. - At present, the capability for persistence within global databases is not available. ## RDS PostgreSQL Extensions for AWS Service Integrations LocalStack supports certain extensions and functions that are provided in RDS to interact with other AWS services. At the moment, primarily extension functions for the PostgreSQL engine are supported. ### `aws_lambda` extension The [`aws_lambda` extension](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/PostgreSQL-Lambda.html) can be used in local RDS PostgreSQL databases to interact with the Lambda API. For example, in the SQL code snippet below, we are loading the `aws_lambda` extension, then generate a full ARN from a function name, and finally invoke the Lambda function directly from the SQL query: ```sql CREATE EXTENSION IF NOT EXISTS aws_lambda CASCADE; -- create a Lambda function ARN SELECT aws_commons.create_lambda_function_arn('my_function'); -- invoke a Lambda function directly from a SQL query SELECT aws_lambda.invoke('my_function', '{\"body\": \"Hello!\"}'::json); ``` ### `aws_s3` extension The [`aws_s3` extension](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/postgresql-s3-export.html) can be used in local RDS PostgreSQL databases to interact with the S3 API. In the SQL code snippet below, we are loading the `aws_s3` extension, then use the `table_import_from_s3(..)` function to populate the data in a table `table1` from a CSV file `test.csv` stored in a local S3 bucket `mybucket1`: ```sql showLineNumbers CREATE EXTENSION IF NOT EXISTS aws_s3 CASCADE; SELECT aws_s3.table_import_from_s3( 'table1', 'c1, c2, c3', '(format csv)', aws_commons.create_s3_uri('mybucket1', 'test.csv', 'us-east-1') ) ``` Analogously, we can use the `query_export_to_s3(..)` extension function to export data from a table `table2` into a CSV file `test.csv` in local S3 bucket `mybucket2`: ```sql showLineNumbers CREATE EXTENSION IF NOT EXISTS aws_s3 CASCADE; SELECT aws_s3.query_export_to_s3( 'SELECT * FROM table2', aws_commons.create_s3_uri('mybucket2', 'test.csv', 'us-east-1'), options := 'FORMAT csv' ) ``` ### Additional extensions In addition to the `aws_*` extensions described in the sections above, LocalStack RDS supports the following PostgreSQL extensions (some of which are bundled with the [`PostGIS` extension](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.PostGIS.html)): - `address_standardizer_data_us` - `fuzzystrmatch` - `postgis` - `postgis_raster` - `postgis_tiger_geocoder` - `postgis_topology` - `pgvector` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing RDS instances and clusters. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **RDS** under the **Database** section. ![RDS Resource Browser](/images/aws/rds-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Instance**: Create a new RDS instance by specifying the instance name, engine, DBInstance Class & Identifier, and other parameters. - **Create Cluster**: Create a new RDS cluster by specifying the database name, engine, DBCluster Identifier, and other parameters. - **View Instance & Cluster**: View an existing RDS instance or cluster by clicking the instance/cluster name. - **Edit Instance & Cluster**: Edit an existing RDS instance or cluster by clicking the instance/cluster name and clicking the **EDIT INSTANCE** or **EDIT CLUSTER** button. - **Remove Instance & Cluster**: Remove an existing RDS instance or cluster by clicking the instance/cluster name and clicking the **ACTIONS** followed by **Remove Selected** button. ## Examples The following code snippets and sample applications provide practical examples of how to use RDS in LocalStack for various use cases: - [AppSync GraphQL APIs for DynamoDB and RDS Aurora PostgreSQL](https://github.com/localstack/appsync-graphql-api-sample) - [Amazon RDS initialization using CDK, Lambda, ECR, and Secrets Manager](https://github.com/localstack/amazon-rds-init-cdk) - [Serverless RDS Proxy with API Gateway, Lambda, and Aurora RDS](https://github.com/localstack-samples/sample-serverless-rds-proxy-demo/) - [Running queries against an RDS database](https://github.com/localstack/localstack-pro-samples/tree/master/rds-db-queries) - [Running cloud integration tests against LocalStack's RDS with Testcontainers](https://github.com/localstack/localstack-pro-samples/tree/master/testcontainers-java-sample) ## API Coverage ## API Coverage (RDS Data) # Redshift > Get started with Redshift on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction RedShift is a cloud-based data warehouse solution which allows end users to aggregate huge volumes of data and parallel processing of data. RedShift is fully managed by AWS and serves as a petabyte-scale service which allows users to create visualization reports and critically analyze collected data. The query results can be saved to an S3 Data Lake while additional analytics can be provided by Athena or SageMaker. LocalStack allows you to use the RedShift APIs in your local environment to analyze structured and semi-structured data across local data warehouses and data lakes. The supported APIs are available on the API coverage section for [Redshift](#api-coverage) and [Redshift Data](#api-coverage-redshift-data), which provides information on the extent of RedShift's integration with LocalStack. :::note For advanced features like Redshift Data API and other emulation capabilities, please refer to the Ultimate plan. ::: ## Getting started This guide is designed for users new to RedShift and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a RedShift cluster and database while using a Glue Crawler to populate the metadata store with the schema of the RedShift database tables using the AWS CLI. ### Define the variables First, we will define the variables we will use throughout this guide. Export the following variables in your shell: ```bash REDSHIFT_CLUSTER_IDENTIFIER="redshiftcluster" REDSHIFT_SCHEMA_NAME="public" REDSHIFT_DATABASE_NAME="db1" REDSHIFT_TABLE_NAME="sales" REDSHIFT_USERNAME="crawlertestredshiftusername" REDSHIFT_PASSWORD="crawlertestredshiftpassword" GLUE_DATABASE_NAME="gluedb" GLUE_CONNECTION_NAME="glueconnection" GLUE_CRAWLER_NAME="gluecrawler" ``` The above variables will be used to create a RedShift cluster, database, table, and user. You will also create a Glue database, connection, and crawler to populate the Glue Data Catalog with the schema of the RedShift database tables. ### Create a RedShift cluster and database You can create a RedShift cluster using the [`CreateCluster`](https://docs.aws.amazon.com/redshift/latest/APIReference/API_CreateCluster.html) API. The following command will create a RedShift cluster with the variables defined above: ```bash lstk aws redshift create-cluster \ --cluster-identifier $REDSHIFT_CLUSTER_IDENTIFIER \ --db-name $REDSHIFT_DATABASE_NAME \ --master-username $REDSHIFT_USERNAME \ --master-user-password $REDSHIFT_PASSWORD \ --node-type n1 ``` You can fetch the status of the cluster using the [`DescribeClusters`](https://docs.aws.amazon.com/redshift/latest/APIReference/API_DescribeClusters.html) API. Run the following command to extract the URL of the cluster: ```bash REDSHIFT_URL=$(lstk aws redshift describe-clusters \ --cluster-identifier $REDSHIFT_CLUSTER_IDENTIFIER | jq -r '(.Clusters[0].Endpoint.Address) + ":" + (.Clusters[0].Endpoint.Port|tostring)') ``` ### Create a Glue database, connection, and crawler You can create a Glue database using the [`CreateDatabase`](https://docs.aws.amazon.com/glue/latest/webapi/API_CreateDatabase.html) API. The following command will create a Glue database: ```bash lstk aws glue create-database \ --database-input "{\"Name\": \"$GLUE_DATABASE_NAME\"}" ``` You can create a connection to the RedShift cluster using the [`CreateConnection`](https://docs.aws.amazon.com/glue/latest/webapi/API_CreateConnection.html) API. The following command will create a Glue connection with the RedShift cluster: ```bash lstk aws glue create-connection \ --connection-input "{\"Name\":\"$GLUE_CONNECTION_NAME\", \"ConnectionType\": \"JDBC\", \"ConnectionProperties\": {\"USERNAME\": \"$REDSHIFT_USERNAME\", \"PASSWORD\": \"$REDSHIFT_PASSWORD\", \"JDBC_CONNECTION_URL\": \"jdbc:redshift://$REDSHIFT_URL/$REDSHIFT_DATABASE_NAME\"}}" ``` Finally, you can create a Glue crawler using the [`CreateCrawler`](https://docs.aws.amazon.com/glue/latest/webapi/API_CreateCrawler.html) API. The following command will create a Glue crawler: ```bash lstk aws glue create-crawler \ --name $GLUE_CRAWLER_NAME \ --database-name $GLUE_DATABASE_NAME \ --targets "{\"JdbcTargets\": [{\"ConnectionName\": \"$GLUE_CONNECTION_NAME\", \"Path\": \"$REDSHIFT_DATABASE_NAME/%/$REDSHIFT_TABLE_NAME\"}]}" \ --role r1 ``` ### Create table in RedShift You can create a table in RedShift using the [`CreateTable`](https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_TABLE_NEW.html) API. The following command will create a table in RedShift: ```bash REDSHIFT_STATEMENT_ID=$(lstk aws redshift-data execute-statement \ --cluster-identifier $REDSHIFT_CLUSTER_IDENTIFIER \ --database $REDSHIFT_DATABASE_NAME \ --sql \ "create table $REDSHIFT_TABLE_NAME(salesid integer not null, listid integer not null, sellerid integer not null, buyerid integer not null, eventid integer not null, dateid smallint not null, qtysold smallint not null, pricepaid decimal(8,2), commission decimal(8,2), saletime timestamp)" | jq -r .Id) ``` You can check the status of the statement using the [`DescribeStatement`](https://docs.aws.amazon.com/redshift-data/latest/APIReference/API_DescribeStatement.html) API. The following command will check the status of the statement: ```bash wait "lstk aws redshift-data describe-statement \ --id $REDSHIFT_STATEMENT_ID" ".Status" "FINISHED" ``` ### Run the crawler You can run the crawler using the [`StartCrawler`](https://docs.aws.amazon.com/glue/latest/webapi/API_StartCrawler.html) API. The following command will run the crawler: ```bash lstk aws glue start-crawler \ --name $GLUE_CRAWLER_NAME ``` You can wait for the crawler to finish using the [`GetCrawler`](https://docs.aws.amazon.com/glue/latest/webapi/API_GetCrawler.html) API. The following command will wait for the crawler to finish: ```bash wait "lstk aws glue get-crawler \ --name $GLUE_CRAWLER_NAME" ".Crawler.State" "READY" ``` You can finally retrieve the schema of the table using the [`GetTable`](https://docs.aws.amazon.com/glue/latest/webapi/API_GetTable.html) API. The following command will retrieve the schema of the table: ```bash lstk aws glue get-table \ --database-name $GLUE_DATABASE_NAME \ --name "${REDSHIFT_DATABASE_NAME}_${REDSHIFT_SCHEMA_NAME}_${REDSHIFT_TABLE_NAME}" ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing RedShift clusters. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **RedShift** under the **Analytics** section. ![RedShift Resource Browser](/images/aws/redshift-resource-browser.png) The Resource Browser allows you to perform the following actions: * **Create Cluster**: Create a new RedShift cluster by specifying the cluster identifier, database name, master username, master password, and node type. * **View Cluster**: View the details of a RedShift cluster, including the cluster identifier, database name, master username, master password, node type, and endpoint. * **Edit Cluster**: Edit an existing RedShift cluster by clicking the cluster name and clicking the **EDIT CLUSTER** button. * **Remove Cluster**: Remove an existing Redshift cluster by selecting it from the table and clicking the **ACTIONS** followed by **Remove Selected** button. ## API Coverage ## API Coverage (Redshift Data) # Resource Groups > Get started with Resource Groups on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Resource Groups allow developers to organize and manage their AWS resources more efficiently. Resource Groups allow for a unified view of their resources allowing developers to perform specific actions, such as resource tagging, access control, and policy enforcement across multiple resources simultaneously. Resource Groups in AWS provide two types of queries that developers can use to build groups: Tag-based queries and CloudFormation stack-based queries. With Tag-based queries, developers can organize resources based on common attributes or characteristics, while CloudFormation stack-based queries allow developers to group resources that are deployed together as part of a CloudFormation stack. LocalStack allows you to use the Resource Groups APIs in your local environment to group and categorize resources based on criteria such as tags, resource types, regions, or custom attributes. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Resource Group's integration with LocalStack. For tag-centric operations across AWS resources, see the [Resource Groups Tagging API](/aws/services/resource-groups-tagging-api/) guide. ## Getting Started This guide is designed for users new to Resource Groups and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a Resource Group using the AWS CLI. We will use tag-based query to create a resource group. However, you can also use CloudFormation stack-based queries to create a resource group. ### Create a Resource Group Resource Groups in AWS are built around the concept of queries, which serve as a fundamental component. The tag-based queries list the resource types in the format `AWS::::` (e.g. `AWS::Lambda::Function` along with specified tags. A tag-based group is created based on a query of type `TAG_FILTERS_1_0`. Use the [`CreateGroup`](https://docs.aws.amazon.com/resource-groups/latest/APIReference/API_CreateGroup.html) API to create a Resource Group. Run the following command to create a Resource Group named `my-resource-group`: ```bash lstk aws resource-groups create-group \ --name my-resource-group \ --resource-query '{"Type":"TAG_FILTERS_1_0","Query":"{\"ResourceTypeFilters\":[\"AWS::EC2::Instance\"],\"TagFilters\":[{\"Key\":\"Stage\",\"Values\":[\"Test\"]}]}"}' ``` You can also specify `AWS::AllSupported` as the `ResourceTypeFilters` value to include all supported resource types in the group. ### Update a Resource Group To update a Resource Group, use the [`UpdateGroup`](https://docs.aws.amazon.com/resource-groups/latest/APIReference/API_UpdateGroup.html) API. Execute the following command to update the Resource Group `my-resource-group`: ```bash lstk aws resource-groups update-group \ --group-name my-resource-group \ --description "EC2 S3 buckets and RDS DBs that we are using for the test stage" ``` Furthermore, you can also update the query and tags associated with a Resource Group using the [`UpdateGroup`](https://docs.aws.amazon.com/resource-groups/latest/APIReference/API_UpdateGroup.html) API. Run the following command to update the query and tags of the Resource Group `my-resource-group`: ```bash lstk aws resource-groups update-group-query \ --group-name my-resource-group \ --resource-query '{"Type":"TAG_FILTERS_1_0","Query":"{\"ResourceTypeFilters\":[\"AWS::EC2::Instance\",\"AWS::S3::Bucket\",\"AWS::RDS::DBInstance\"],\"TagFilters\":[{\"Key\":\"Stage\",\"Values\":[\"Test\"]}]}"}' ``` ### Delete a Resource Group To delete a Resource Group, use the [`DeleteGroup`](https://docs.aws.amazon.com/resource-groups/latest/APIReference/API_DeleteGroup.html) API. Run the following command to delete the Resource Group `my-resource-group`: ```bash lstk aws resource-groups delete-group \ --group-name my-resource-group ``` ## API Coverage # Resource Groups Tagging API > Get started with Resource Groups Tagging API on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Resource Groups Tagging API helps you centrally manage tags across AWS resources. You can apply and remove tags for multiple resources at once, and query resources by tag keys and values. You can also query tagged resources by AWS service, by resource type within a service, or directly by specific resource ARNs. LocalStack allows you to use the Resource Groups Tagging API in your local environment to test tagging workflows end-to-end. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Resource Groups Tagging API's integration with LocalStack. ## Getting started This guide is designed for users new to Resource Groups Tagging API and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create resources, tag them, and query them using the Resource Groups Tagging API. ### Create resources to tag Create an S3 bucket and an SQS queue: ```bash BUCKET_NAME="rg-tagging-bucket" QUEUE_NAME="rg-tagging-queue" lstk aws s3api create-bucket --bucket "$BUCKET_NAME" QUEUE_URL=$(lstk aws sqs create-queue --queue-name "$QUEUE_NAME" | jq -r '.QueueUrl') ``` Retrieve the resource ARNs: ```bash BUCKET_ARN="arn:aws:s3:::$BUCKET_NAME" QUEUE_ARN=$(lstk aws sqs get-queue-attributes \ --queue-url "$QUEUE_URL" \ --attribute-names QueueArn | jq -r '.Attributes.QueueArn') ``` ### Tag multiple resources Use the [`TagResources`](https://docs.aws.amazon.com/resourcegroupstagging/latest/APIReference/API_TagResources.html) API to apply tags: ```bash lstk aws resourcegroupstaggingapi tag-resources \ --resource-arn-list "$BUCKET_ARN" "$QUEUE_ARN" \ --tags '{"Environment":"dev","Team":"platform"}' ``` ### Query resources by tags Use the [`GetResources`](https://docs.aws.amazon.com/resourcegroupstagging/latest/APIReference/API_GetResources.html) API to list resources with a specific tag: ```bash lstk aws resourcegroupstaggingapi get-resources \ --tag-filters Key=Environment,Values=dev ``` You can also inspect available keys and values: ```bash lstk aws resourcegroupstaggingapi get-tag-keys lstk aws resourcegroupstaggingapi get-tag-values --key Environment ``` ### Remove tags from resources Use the [`UntagResources`](https://docs.aws.amazon.com/resourcegroupstagging/latest/APIReference/API_UntagResources.html) API to remove one or more tag keys: ```bash lstk aws resourcegroupstaggingapi untag-resources \ --resource-arn-list "$BUCKET_ARN" "$QUEUE_ARN" \ --tag-keys Team ``` ## Supported services LocalStack's Resource Groups Tagging API supports tagging for the following services: - [AWS Key Management Service (KMS)](/aws/services/kms/) - [Amazon S3](/aws/services/s3/) - [AWS Lambda](/aws/services/lambda/) - [Amazon Route 53](/aws/services/route53/) - [Amazon SNS](/aws/services/sns/) - [Amazon SQS](/aws/services/sqs/) - [Amazon OpenSearch Service](/aws/services/opensearch/) - [AWS Elastic Beanstalk](/aws/services/elasticbeanstalk/) - [Amazon EC2](/aws/services/ec2/) ## API Coverage # Route 53 > Get started with Route 53 on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Route 53 is a highly scalable and reliable domain name system (DNS) web service provided by Amazon Web Services. Route 53 allows you to register domain names, and associate them with IP addresses or other resources. In addition to basic DNS functionality, Route 53 offers advanced features like health checks and DNS failover. Route 53 integrates seamlessly with other AWS services, such as route traffic to CloudFront distributions, S3 buckets configured for static website hosting, Elastic Load Balancers, EC2 instances, API Gateway custom domain names, and more. LocalStack allows you to use the Route53 APIs in your local environment to create hosted zones and to manage DNS entries. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Route53's integration with LocalStack. LocalStack supports routing traffic to various AWS resources including [S3 static websites](#routing-traffic-to-s3-static-websites) and [Elastic Load Balancers](#routing-traffic-to-elastic-load-balancers) using alias records. LocalStack also integrates with its DNS server to respond to DNS queries with these domains. :::note `lstk` does not publish port `53` on the host by default. Add `expose_ports = [53]` to the container block in your `config.toml` to expose it: ```toml # .lstk/config.toml [[containers]] type = "aws" expose_ports = [53] ``` This is required if you want to reach out to Route53 domain names from your host machine, using the LocalStack DNS server. See [System DNS configuration](/aws/customization/networking/dns-server#system-dns-configuration) for details. ::: ## Getting started This guide is designed for users new to Route53 and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a hosted zone and query the DNS record with the AWS CLI. ### Create a hosted zone You can created a hosted zone for `example.com` using the [`CreateHostedZone`](https://docs.aws.amazon.com/Route53/latest/APIReference/API_CreateHostedZone.html) API. Run the following command: ```bash zone_id=$(lstk aws route53 create-hosted-zone \ --name example.com \ --caller-reference r1 | jq -r '.HostedZone.Id') echo $zone_id ``` ```bash title="Output" /hostedzone/WBCZ6F10CWV9J1G ``` ### Change resource record sets You can now change the resource record sets for the hosted zone `example.com` using the [`ChangeResourceRecordSets`](https://docs.aws.amazon.com/Route53/latest/APIReference/API_ChangeResourceRecordSets.html) API. Run the following command: ```bash lstk aws route53 change-resource-record-sets \ --hosted-zone-id $zone_id \ --change-batch 'Changes=[{Action=CREATE,ResourceRecordSet={Name=test.example.com,Type=A,ResourceRecords=[{Value=1.2.3.4}]}}]' ``` ```bash title="Output" { "ChangeInfo": { "Id": "/change/C2682N5HXP0BZ4", "Status": "INSYNC", "SubmittedAt": "2010-09-10T01:36:41.958000Z" } } ``` ## Routing traffic to S3 static websites You can route traffic from a Route53 domain to an S3 bucket configured for static website hosting using alias records. This is useful when you want to serve a static website with a custom domain name. ### Create an S3 bucket with website hosting First, create an S3 bucket and configure it for static website hosting. Run the following commands: ```bash DOMAIN="example.com" BUCKET_NAME="$DOMAIN" # Create the bucket lstk aws s3api create-bucket --bucket "$BUCKET_NAME" # Upload your website files lstk aws s3 cp index.html s3://$BUCKET_NAME/ lstk aws s3 cp error.html s3://$BUCKET_NAME/ # Configure the bucket for website hosting lstk aws s3 website s3://"$BUCKET_NAME"/ \ --index-document index.html \ --error-document error.html # Set bucket policy to allow public read access lstk aws s3api put-bucket-policy \ --bucket $BUCKET_NAME \ --policy '{ "Version": "2012-10-17", "Statement": [{ "Sid": "PublicReadGetObject", "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::'$BUCKET_NAME'/*" }] }' ``` ### Create a Route53 alias record Now create a hosted zone and an alias record that points to the S3 website endpoint: ```bash # Create the hosted zone HOSTED_ZONE_ID=$(lstk aws route53 create-hosted-zone \ --name "$DOMAIN" \ --caller-reference "$(date +%s)" \ --output text \ --query 'HostedZone.Id' | cut -d'/' -f3) echo "Hosted Zone created with ID: $HOSTED_ZONE_ID" # Create an alias record pointing to the S3 website endpoint lstk aws route53 change-resource-record-sets \ --hosted-zone-id "$HOSTED_ZONE_ID" \ --change-batch '{ "Comment": "Create alias record for S3 static website", "Changes": [{ "Action": "CREATE", "ResourceRecordSet": { "Name": "'$DOMAIN'", "Type": "A", "AliasTarget": { "HostedZoneId": "'$HOSTED_ZONE_ID'", "DNSName": "'$BUCKET_NAME'.s3-website.localhost.localstack.cloud", "EvaluateTargetHealth": false } } }] }' ``` The key points for S3 website alias records are: - The `DNSName` follows the format: `.s3-website.localhost.localstack.cloud` - The `Type` must be `A` (for IPv4) or `AAAA` (for IPv6) - Set `EvaluateTargetHealth` to `false` for S3 website endpoints ### Verify DNS resolution You can verify that your domain resolves to the S3 website using [DNS resolution](#dns-resolution) or by making HTTP requests: ```bash # Using dig to verify DNS resolution dig @localhost $DOMAIN # Using curl to access the website curl http://$DOMAIN:4566/ ``` ## Routing traffic to Elastic Load Balancers You can also route traffic to Elastic Load Balancers (ELB) using alias records. This is commonly used for distributing traffic across multiple instances. After creating your load balancer, you can create an alias record that points to it: ```bash # Assuming you have an ELB with DNS name ELB_DNS_NAME="my-load-balancer-123456.elb.localhost.localstack.cloud" # Create an alias record pointing to the ELB lstk aws route53 change-resource-record-sets \ --hosted-zone-id "$HOSTED_ZONE_ID" \ --change-batch '{ "Comment": "Create alias record for ELB", "Changes": [{ "Action": "CREATE", "ResourceRecordSet": { "Name": "app.example.com", "Type": "A", "AliasTarget": { "HostedZoneId": "'$HOSTED_ZONE_ID'", "DNSName": "'$ELB_DNS_NAME'", "EvaluateTargetHealth": true } } }] }' ``` For ELB alias records: - Use the load balancer's DNS name as the `DNSName` value - You can set `EvaluateTargetHealth` to `true` to enable health checks - The `Type` should be `A` for IPv4 addresses ## DNS resolution LocalStack for AWS supports the ability to respond to DNS queries for your Route53 domain names, with our [integrated DNS server](/aws/customization/networking/dns-server). :::note To follow the example below you must [configure your system DNS to use the LocalStack DNS server](/aws/customization/networking/dns-server#system-dns-configuration). ::: ### Query a DNS record You can query the DNS record using `dig` via the built-in DNS server by running the following command: ```bash dig @localhost test.example.com ``` ```bash title="Output" ;; QUESTION SECTION: ;test.example.com. IN A ;; ANSWER SECTION: test.example.com. 300 IN A 1.2.3.4 ``` ### Customizing internal endpoint resolution The DNS name `localhost.localstack.cloud`, along with its subdomains like `mybucket.s3.localhost.localstack.cloud`, serves an internal routing purpose within LocalStack. It facilitates communication between a LocalStack compute environment (such as a Lambda function) and the LocalStack APIs, as well as your containerised applications with the LocalStack APIs. For example configurations, see the [Network Troubleshooting guide](/aws/customization/networking/). For most use-cases, the default configuration of the internal LocalStack DNS name requires no modification. It functions seamlessly in typical scenarios. However, there are instances where adjusting the external resolution of this DNS name becomes necessary. For instance, this might be required when your LocalStack instance operates on a distinct Docker network compared to your application code or even on a separate machine. Suppose you intend to achieve a scenario in which all subdomains in the format `*.localhost.localstack.cloud` resolve to the IP address `5.6.7.8`. This IP signifies the accessibility of your LocalStack instance. This can be accomplished using Route53. Create a hosted zone for the domain `localhost.localstack.cloud` using the [`CreateHostedZone` API](https://docs.aws.amazon.com/Route53/latest/APIReference/API_CreateHostedZone.html) API. Run the following command: ```bash zone_id=$(lstk aws route53 create-hosted-zone \ --name localhost.localstack.cloud \ --caller-reference r1 | jq -r .HostedZone.Id) echo $zone_id ``` ```bash title="Output" /hostedzone/3NF6SEGOB5EBHS1 ``` You can now use the [`ChangeResourceRecordSets`](https://docs.aws.amazon.com/Route53/latest/APIReference/API_ChangeResourceRecordSets.html) API to create a record set for the domain `localhost.localstack.cloud` using the `zone_id` retrieved in the previous step. Run the following command to accomplish this: ```bash lstk aws route53 change-resource-record-sets \ --hosted-zone-id $zone_id \ --change-batch '{"Changes":[{"Action":"CREATE","ResourceRecordSet":{"Name":"localhost.localstack.cloud","Type":"A","ResourceRecords":[{"Value":"5.6.7.8"}]}},{"Action":"CREATE","ResourceRecordSet":{"Name":"*.localhost.localstack.cloud","Type":"A","ResourceRecords":[{"Value":"5.6.7.8"}]}}]}' ``` ```bash title="Output" { "ChangeInfo": { "Id": "/change/C2682N5HXP0BZ4", "Status": "INSYNC", "SubmittedAt": "2010-09-10T01:36:41.958000Z" } } ``` You can now verify that the DNS name `localhost.localstack.cloud` and its subdomains resolve to the IP address: ```bash dig @127.0.0.1 bucket1.s3.localhost.localstack.cloud dig @127.0.0.1 localhost.localstack.cloud ``` ```bash title="Output" ... ;; ANSWER SECTION: bucket1.s3.localhost.localstack.cloud. 300 IN A 127.0.0.1 bucket1.s3.localhost.localstack.cloud. 300 IN A 5.6.7.8 ... ;; QUESTION SECTION: ;localhost.localstack.cloud. IN A ;; ANSWER SECTION: localhost.localstack.cloud. 300 IN A 5.6.7.8 ``` ## Resource Browser The LocalStack Web Application provides a Route53 for creating hosted zones and to manage DNS entries. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **Route53** under the **Analytics** section. ![Route53 Resource Browser](/images/aws/route53-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Hosted Zone**: Create a hosted zone for a domain name by clicking on the **Create Hosted Zone** button. This will open a modal where you can enter the name, VPC, and other parameters and click on the **Submit** button to create the hosted zone. - **View Hosted Zone**: View the details of a hosted zone by clicking on the specific hosted zone name. This will open a modal where you can view the hosted zone details. - **Create Record**: Click on the **Records** button on the individual hosted zone page, followed by clicking **Create Record** to create a record for the hosted zone. This will open a modal where you can enter the name, type, and other parameters and click on the **Submit** button to create the record. - **Edit Record**: Click on the **Records** button on the individual hosted zone page, followed by clicking **Edit** on the specific record to edit the record. This will open a modal where you can edit the record details and click on the **Submit** button to save the changes.s - **View Records**: Click on the **Records** button on the individual hosted zone page, followed by clicking on the specific record to view the record details. This will open a modal where you can view the record details. - **Delete Hosted Zone**: Select the hosted zones you want to delete by clicking on the checkbox next to the hosted zone name, followed by clicking on the **Actions** button and then clicking on **Remove Selected**. - **Delete Record**: Click on the **Records** button on the individual hosted zone page, followed by clicking on checkbox next to the specific record, and then clicking on the **Actions** button and then clicking on **Remove Selected**. ## Examples The following code snippets and sample applications provide practical examples of how to use Route53 in LocalStack for various use cases: - [DNS Failover with Route53 on LocalStack](https://github.com/localstack/localstack-pro-samples/tree/master/route53-dns-failover) ## API Coverage # Route 53 Resolver > Get started with Route 53 Resolver on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Route 53 Resolver allows you to route DNS queries between your virtual private cloud (VPC) and your network. Route 53 Resolver forwards DNS queries for domain names to the appropriate DNS service based on the configuration you set up. Route 53 Resolver can be used to resolve domain names between your VPC and your network, and to resolve domain names between your VPCs. LocalStack allows you to use the Route 53 Resolver endpoints in your local environment. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Route 53 Resolver's integration with LocalStack. ## Getting started This guide is designed for users new to Route53 Resolver and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a resolver endpoint, list the endpoints, and delete the endpoint with the AWS CLI. ### Fetch the IP addresses & Security Group ID Fetch the default VPC ID using the following command: ```bash VPC_ID=$(lstk aws ec2 describe-vpcs --query 'Vpcs[?IsDefault==`true`].VpcId' --output text) ``` Fetch the default VPC's security group ID using the following command: ```bash lstk aws ec2 describe-subnets --filters Name=vpc-id,Values=$VPC_ID --query 'Subnets[].SubnetId' ``` ```bash title="Output" [ "subnet-bdd58a47", "subnet-957d6ba6", "subnet-3f8669d3", "subnet-ec2a41c6", "subnet-3d583924", "subnet-8c1b0af8" ] ``` Choose two subnets from the list above and fetch the CIDR block of the subnets which tells you the range of IP addresses within it. Let's fetch the CIDR block of the subnet `subnet-957d6ba6`: ```bash lstk aws ec2 describe-subnets --subnet-ids subnet-957d6ba6 --query 'Subnets[*].CidrBlock' ``` ```bash title="Output" [ "172.31.16.0/20" ] ``` Similarly, fetch the CIDR block of the subnet `subnet-bdd58a47`: ```bash lstk aws ec2 describe-subnets --subnet-ids subnet-bdd58a47 --query 'Subnets[*].CidrBlock' ``` ```bash title="Output" [ "172.31.0.0/20" ] ``` Save the CIDR blocks of the subnets as you will need them later. Lastly fetch the security group ID of the default VPC: ```bash lstk aws ec2 describe-security-groups \ --filters Name=vpc-id,Values=$VPC_ID \ --query 'SecurityGroups[0].GroupId' ``` ```bash title="Output" sg-39936e572e797b360 ``` Save the security group ID as you will need it later. ### Create a resolver endpoint Create a new file named `create-outbound-resolver-endpoint.json` and add the following content: ```json { "CreatorRequestId": "2020-01-01-18:47", "Direction": "OUTBOUND", "IpAddresses": [ { "Ip": "172.31.0.0", "SubnetId": "subnet-bdd58a47" }, { "Ip": "172.31.16.0", "SubnetId": "subnet-957d6ba6" } ], "Name": "my-outbound-endpoint", "SecurityGroupIds": [ "sg-39936e572e797b360" ], "Tags": [ { "Key": "purpose", "Value": "test" } ] } ``` Replace the `Ip` and `SubnetId` values with the CIDR blocks and subnet IDs you fetched earlier. You can now use the [`CreateResolverEndpoint`](https://docs.aws.amazon.com/Route53/latest/APIReference/API_route53resolver_CreateResolverEndpoint.html) API to create an outbound resolver endpoint. Run the following command: ```bash lstk aws route53resolver create-resolver-endpoint \ --cli-input-json file://create-outbound-resolver-endpoint.json ``` ```bash title="Output" { "ResolverEndpoint": { "Id": "rslvr-out-5d61abaff9de06b99", "CreatorRequestId": "2020-01-01-18:47", "Arn": "arn:aws:route53resolver:us-east-1:000000000000:resolver-endpoint/rslvr-out-5d61abaff9de06b99", "Name": "my-outbound-endpoint", "SecurityGroupIds": [ "sg-39936e572e797b360" ], "Direction": "OUTBOUND", "IpAddressCount": 2, "HostVPCId": "vpc-d78cf7bb", "Status": "CREATING", "StatusMessage": "[Trace id: 1-bf9fe209-b90acae7cbcefe68a98b2882] Successfully created Resolver Endpoint", "CreationTime": "2024-05-02T15:03:17.266471+00:00", "ModificationTime": "2024-05-02T15:03:17.266491+00:00" } } ``` ### List the resolver endpoints You can list the resolver endpoints using the [`ListResolverEndpoints`](https://docs.aws.amazon.com/Route53/latest/APIReference/API_route53resolver_ListResolverEndpoints.html) API. Run the following command: ```bash lstk aws route53resolver list-resolver-endpoints ``` ```bash title="Output" { "ResolverEndpoints": [ { "Id": "rslvr-out-5d61abaff9de06b99", "CreatorRequestId": "2020-01-01-18:47", "Arn": "arn:aws:route53resolver:us-east-1:000000000000:resolver-endpoint/rslvr-out-5d61abaff9de06b99", "Name": "my-outbound-endpoint", "SecurityGroupIds": [ "sg-39936e572e797b360" ], "Direction": "OUTBOUND", "IpAddressCount": 2, "HostVPCId": "vpc-d78cf7bb", "Status": "OPERATIONAL", "StatusMessage": "[Trace id: 1-bf9fe209-b90acae7cbcefe68a98b2882] Successfully created Resolver Endpoint", "CreationTime": "2024-05-02T15:03:17.266471+00:00", "ModificationTime": "2024-05-02T15:03:17.266491+00:00" } ], "MaxResults": 10 } ``` ### Delete the resolver endpoint You can delete the resolver endpoint using the [`DeleteResolverEndpoint`](https://docs.aws.amazon.com/Route53/latest/APIReference/API_route53resolver_DeleteResolverEndpoint.html) API. Run the following command: ```bash lstk aws route53resolver delete-resolver-endpoint \ --resolver-endpoint-id rslvr-out-5d61abaff9de06b99 ``` Replace `rslvr-out-5d61abaff9de06b99` with the ID of the resolver endpoint you want to delete. ## Resource Browser The LocalStack Web Application provides a Route53 Resolver for creating and managing resolver endpoints. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resource Browser** section, and then clicking on **Route53** under the **Analytics** section. Navigate to the **Resolver Endpoints** tab to view the resolver endpoints. ![Route53Resolver Resource Browser](/images/aws/route53-resolver-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create resolver endpoint**: Create a resolver endpoint by clicking on the **Create Endpoint** button. This will open a modal where you can enter the name, VPC, and other parameters and click on the **Submit** button to create the resolver endpoint. - **View resolver endpoint**: View the details of a resolver endpoint by clicking on the specific resolver endpoint name. This will open a modal where you can view the resolver endpoint details. - **Edit resolver endpoint**: Edit the details of a resolver endpoint by clicking on the **Edit Endpoint** button in the specific resolver endpoint page. This will open a modal where you can edit the resolver endpoint details. - **Delete resolver endpoint**: Select the resolver endpoints you want to delete by clicking on the checkbox next to the resolver endpoint name, followed by clicking on the **Actions** button and then clicking on **Remove Selected**. ## API Coverage # Simple Storage Service (S3) > Get started with Amazon S3 on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Simple Storage Service (S3) is an object storage service that provides a highly scalable and durable solution for storing and retrieving data. In S3, a bucket represents a directory, while an object corresponds to a file. Each object or file within S3 encompasses essential attributes such as a unique key denoting its name, the actual content it holds, a version ID for versioning support, and accompanying metadata. S3 can store unlimited objects, allowing you to store, retrieve, and manage your data in a highly adaptable and reliable manner. LocalStack allows you to use the S3 APIs in your local environment to create new buckets, manage your S3 objects, and test your S3 configurations locally. The supported APIs are available on the API coverage section for [S3](#api-coverage) and [S3 Control](#api-coverage-s3-control), which provides information on the extent of S3's integration with LocalStack. ## Getting started This guide is designed for users new to S3 and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how you can create an S3 bucket, manage S3 objects, and generate pre-signed URLs for S3 objects. ### Create an S3 bucket You can create an S3 bucket using the [`CreateBucket`](https://docs.aws.amazon.com/cli/latest/reference/s3api/create-bucket.html) API. Run the following command to create an S3 bucket named `sample-bucket`: ```bash lstk aws s3api create-bucket --bucket sample-bucket ``` You can list your S3 buckets using the [`ListBuckets`](https://docs.aws.amazon.com/cli/latest/reference/s3api/list-buckets.html) API. Run the following command to list your S3 buckets: ```bash lstk aws s3api list-buckets ``` ```bash title="Output" { "Buckets": [ { "Name": "sample-bucket", "CreationDate": "2023-07-18T06:36:25+00:00" } ], "Owner": { "DisplayName": "webfile", "ID": "75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a" } } ``` ### Managing S3 objects To upload a file to your S3 bucket, you can use the [`PutObject`](https://docs.aws.amazon.com/cli/latest/reference/s3api/put-object.html) API. Download a random image from the internet and save it as `image.jpg`. Run the following command to upload the file to your S3 bucket: ```bash lstk aws s3api put-object \ --bucket sample-bucket \ --key image.jpg \ --body image.jpg ``` You can list the objects in your S3 bucket using the [`ListObjects`](https://docs.aws.amazon.com/cli/latest/reference/s3api/list-objects.html) API. Run the following command to list the objects in your S3 bucket: ```bash lstk aws s3api list-objects \ --bucket sample-bucket ``` If your image has been uploaded successfully, you will see the following output: ```bash title="Output" { "Contents": [ { "Key": "image.jpg", "LastModified": "2023-07-18T06:40:07+00:00", "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"", "Size": 0, "StorageClass": "STANDARD", "Owner": { "DisplayName": "webfile", "ID": "75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a" } } ] } ``` Run the following command to upload a file named `index.html` to your S3 bucket: ```bash lstk aws s3api put-object --bucket sample-bucket --key index.html --body index.html ``` ```bash title="Output" { "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"" } ``` ### Generate a pre-signed URL for S3 object You can generate a pre-signed URL for your S3 object using the [`presign`](https://docs.aws.amazon.com/cli/latest/reference/s3/presign.html) command. Pre-signed URL allows anyone to retrieve the S3 object with an HTTP GET request. Run the following command to generate a pre-signed URL for your S3 object: ```bash lstk aws s3 presign s3://sample-bucket/image.jpg ``` You will see a generated pre-signed URL for your S3 object. You can use [curl](https://curl.se/) or [`wget`](https://www.gnu.org/software/wget/) to retrieve the S3 object using the pre-signed URL. By default, LocalStack does not validate the signature or the expiration of pre-signed URLs. Check out the [Signature validation](#signature-validation) section to enable it. ## Configuring S3 Endpoint LocalStack supports both [Virtual-Hosted style and Path style](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html) S3 requests. AWS recommends Virtual-Hosted style addressing, and some AWS regions do not support path-style requests at all. LocalStack follows this recommendation: **Virtual-Hosted style is the default and recommended approach**. ### Recommended: Using AWS_ENDPOINT_URL_S3 The simplest way to configure your application to use LocalStack's S3 endpoint is with the `AWS_ENDPOINT_URL_S3` environment variable. This approach works with modern AWS SDKs and tools without requiring code changes: ```bash export AWS_ENDPOINT_URL_S3=http://s3.localhost.localstack.cloud:4566 ``` This environment variable is supported by: - **AWS CLI v2**: All `aws s3` and `aws s3api` commands automatically use this endpoint - **boto3** (Python SDK) with botocore >= 1.29: `boto3.client("s3")` resolves the endpoint automatically - **Terraform** with terraform-provider-aws >= 5.x: No `endpoints {}` block needed in your configuration See AWS's [service-specific endpoint SDK compatibility matrix](https://docs.aws.amazon.com/sdkref/latest/guide/feature-ss-endpoints.html#ss-endpoints-sdk-compat) for the complete, up-to-date list of SDKs and minimum versions that support this variable. With this variable set, your application code remains unchanged and can run against both LocalStack and real AWS by simply changing the environment. ### Virtual-Hosted Style Requests A **Virtual-Hosted style** request includes the bucket name as part of the `Host` header. For LocalStack to parse the bucket name correctly, your endpoint must be prefixed with `s3.`, like `s3.localhost.localstack.cloud`: ```bash http://.s3.localhost.localstack.cloud:4566/ ``` This is the format that `AWS_ENDPOINT_URL_S3` uses, and it's what most modern AWS SDKs use by default. ### Path Style Requests (Fallback) **Path style** requests include the bucket as part of the URL path instead of the hostname: ```bash http://s3.localhost.localstack.cloud:4566// ``` You can combine path-style addressing with the `s3.`-prefixed endpoint, as shown above, or with a bare endpoint (`http://localhost:4566//`). You should only use path-style requests if you have a specific reason: - Your bucket names contain periods. A period is a valid, DNS-compliant character, but the TLS certificate used for virtual-hosted-style buckets doesn't cover names with an extra dot in them, so virtual-hosted-style addressing over HTTPS breaks. See the [bucket naming rules](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html) for the full list of naming restrictions. - You're using an older SDK or tool that doesn't support virtual-hosted style. - You're connecting to LocalStack from another container, for example in **Docker Compose**, using the LocalStack service's container name (such as `http://localstack:4566`). Wildcard subdomains like `.localstack` are not resolvable in that case, so path-style addressing is required. This is one of the most common reasons users need path-style requests. :::tip To enable path-style requests in [AWS SDKs](https://aws.amazon.com/developer/tools/#SDKs), set the `ForcePathStyle` parameter to `true` in your S3 client configuration. The parameter name varies by SDK: - Python (boto3): `s3={'addressing_style': 'path'}` in the Session config, or `S3ForcePathStyle=true` in the Config - JavaScript: `forcePathStyle: true` - Go: `WithS3ForcePathStyle(true)` - Terraform: `s3_use_path_style = true` - PHP: `use_path_style_endpoint => true` Check our [SDK documentation](/aws/connecting/aws-sdks/) for language-specific examples. ::: ### Endpoint URL Formats LocalStack recognizes the following endpoint formats: ```bash # Virtual-Hosted style (recommended) http://.s3.localhost.localstack.cloud:4566/ http://.s3..localhost.localstack.cloud:4566/ # Path style (fallback) http://s3.localhost.localstack.cloud:4566// http://s3..localhost.localstack.cloud:4566// http://localhost:4566// ``` For detailed configuration instructions for specific SDKs and tools, see our [SDK documentation](/aws/customization/integrations/localstack-sdks/) and [Infrastructure as Code guides](/aws/connecting/infrastructure-as-code/). ## Signature validation Like AWS, S3 in LocalStack can validate the signature of incoming requests and reject requests signed with invalid credentials. Signature validation is disabled by default, so that S3 accepts requests signed with any credentials. Two independent configuration options control signature validation: - [`S3_SKIP_SIGNATURE_VALIDATION=0`](/aws/customization/configuration-options/#s3) validates [pre-signed URLs](#pre-signed-urls). - [`S3_VALIDATE_SIGNATURES=1`](/aws/customization/configuration-options/#s3) validates regular, [SigV4-signed requests](#sigv4-validation). ### Credentials When signature validation is enabled, requests must be signed with credentials that are valid in LocalStack. The following credentials pass validation: - **Default credentials**: the default `test` access key ID with the `test` secret access key, which works out of the box. - **IAM user credentials**: access keys created for an IAM user with [`CreateAccessKey`](https://docs.aws.amazon.com/IAM/latest/APIReference/API_CreateAccessKey.html). Check out the [IAM documentation](/aws/services/iam/#getting-started) to learn how to create a user and its access keys. - **Temporary credentials**: credentials returned by the STS [`AssumeRole`](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) or [`GetSessionToken`](https://docs.aws.amazon.com/STS/latest/APIReference/API_GetSessionToken.html) APIs, used together with their session token. For example, after [creating a role](/aws/services/sts/#create-an-iam-role), you can retrieve temporary credentials for it using the `AssumeRole` API: ```bash lstk aws sts assume-role \ --role-arn arn:aws:iam::000000000000:role/localstack-role \ --role-session-name localstack-session ``` ```bash title="Output" { "Credentials": { "AccessKeyId": "ACCESS_KEY_ID", "SecretAccessKey": "SECRET_ACCESS_KEY", "SessionToken": "SESSION_TOKEN", "Expiration": "TIMESTAMP" }, ... } ``` Export the returned credentials and use the AWS CLI or your SDK as usual — requests are now signed with the temporary credentials and pass validation: ```bash export AWS_ACCESS_KEY_ID=ACCESS_KEY_ID export AWS_SECRET_ACCESS_KEY=SECRET_ACCESS_KEY export AWS_SESSION_TOKEN=SESSION_TOKEN lstk aws s3api list-buckets ``` :::note An access key ID that was not issued by LocalStack through IAM or STS is expected to be paired with the `test` secret access key. Signing a request with such an access key ID and a different secret access key results in a `SignatureDoesNotMatch` error. One exception is a 12-digit account ID used as access key ID for [multi-account namespacing](/aws/customization/advanced/multi-account-setups/), which cannot pass signature validation. To send signed requests to another account than the default, use the credentials of an IAM user created in that account, or temporary credentials of a role assumed in it. ::: Signature validation authenticates a request, but does not authorize it: by default, LocalStack does not check whether the credentials are allowed to perform the operation. Authorization is handled by [IAM policy enforcement](/aws/developer-tools/security-testing/iam-policy-enforcement/), which can be enabled independently and combined with signature validation for the closest behavior to AWS. ### Pre-signed URLs A pre-signed URL grants time-limited access to an S3 object: anyone with the URL can access the object without providing credentials. You can generate a pre-signed URL as shown in the [Getting started](#generate-a-pre-signed-url-for-s3-object) section. Presigning is a purely client-side operation: the SDK or the CLI computes the pre-signed URL locally with the credentials it is configured with, without contacting LocalStack. The signature is only checked when the pre-signed URL is used. For the validation to pass, the URL must therefore be generated with valid [credentials](#credentials). By default, LocalStack accepts pre-signed URLs with an invalid signature or an expired date. Start LocalStack with `S3_SKIP_SIGNATURE_VALIDATION=0` to validate pre-signed URLs like AWS does: - The signature must match the request. Both SigV2 and SigV4 pre-signed URLs are supported. - The URL must not be expired. - All `x-amz-*` headers sent with the request must be signed in the URL. Requests that fail validation are rejected with a `403` error, such as `SignatureDoesNotMatch` for an invalid signature, or `AccessDenied` for an expired URL. ### SigV4 validation By default, LocalStack accepts regular S3 requests signed with any credentials. Start LocalStack with `S3_VALIDATE_SIGNATURES=1` to validate [SigV4-signed requests](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html) like AWS does. LocalStack validates the signature in the `Authorization` header, as well as the integrity of the payload declared in the `x-amz-content-sha256` header, including streamed `aws-chunked` uploads. If your SDK or CLI is configured with valid [credentials](#credentials), validation is fully transparent and does not require any change. Requests that fail validation are rejected with the same errors as AWS, such as `SignatureDoesNotMatch` for a signature computed with the wrong secret access key, or `XAmzContentSHA256Mismatch` for a payload that does not match its declared checksum. :::note `S3_VALIDATE_SIGNATURES` only applies to SigV4-signed requests. Anonymous requests, such as requests to public buckets or static S3 websites, and CORS preflight requests are not affected. Pre-signed URL validation is exclusively controlled by `S3_SKIP_SIGNATURE_VALIDATION`. ::: ## Configuring Cross-Origin Resource Sharing on S3 You can configure Cross-Origin Resource Sharing (CORS) on a LocalStack S3 bucket using AWS Command Line Interface (CLI). It would allow your local application to communicate directly with an S3 bucket in LocalStack. By default, LocalStack will apply specific CORS rules to all requests to allow you to display and access your resources through [LocalStack Web Application](https://app.localstack.cloud). If no CORS rules are configured for your S3 bucket, LocalStack will apply default rules unless specified otherwise. To configure CORS rules for your S3 bucket, you can use the `lstk aws` command. Optionally, you can run a local web application on [localhost:3000](http://localhost:3000). You can emulate the same behaviour with an AWS SDK or an integration you use. Follow this step-by-step guide to configure CORS rules on your S3 bucket. Run the following command on your terminal to create your S3 bucket: ```bash lstk aws s3api create-bucket --bucket cors-bucket ``` ```bash title="Output" { "Location": "/cors-bucket" } ``` Next, create a JSON file with the CORS configuration. The file should have the following format: ```json title="cors-config.json" showLineNumbers { "CORSRules": [ { "AllowedHeaders": ["*"], "AllowedMethods": ["GET", "POST", "PUT"], "AllowedOrigins": ["http://localhost:3000"], "ExposeHeaders": ["ETag"] } ] } ``` :::note Note that this configuration is a sample, and you can tailor it to fit your needs better, for example, restricting the **AllowedHeaders** to specific ones. ::: Save the file locally with a name of your choice, for example, `cors-config.json`. Run the following command to apply the CORS configuration to your S3 bucket: ```bash lstk aws s3api put-bucket-cors --bucket cors-bucket --cors-configuration file://cors-config.json ``` You can further verify that the CORS configuration was applied successfully by running the following command: ```bash lstk aws s3api get-bucket-cors --bucket cors-bucket ``` On applying the configuration successfully, you should see the same JSON configuration file you created earlier. Your S3 bucket is configured to allow cross-origin resource sharing, and if you try to send requests from your local application running on [localhost:3000](http://localhost:3000), they should be successful. However, if you try to access your bucket from [LocalStack Web Application](https://app.localstack.cloud), you'll see errors, and your bucket won't be accessible anymore. We can edit the JSON file `cors-config.json` you created earlier with the following configuration and save it: ```json title="cors-config.json" showLineNumbers { "CORSRules": [ { "AllowedHeaders": ["*"], "AllowedMethods": ["GET", "POST", "PUT", "HEAD", "DELETE"], "AllowedOrigins": [ "http://localhost:3000", "https://app.localstack.cloud", "http://app.localstack.cloud" ], "ExposeHeaders": ["ETag"] } ] } ``` You can now run the same steps as before to update the CORS configuration and verify if it is applied correctly: ```bash lstk aws s3api put-bucket-cors --bucket cors-bucket --cors-configuration file://cors-config.json lstk aws s3api get-bucket-cors --bucket cors-bucket ``` You can try again to upload files in your bucket from the [LocalStack Web Application](https://app.localstack.cloud) and it should work. ## SSE-C Encryption SSE-C (Server-Side Encryption with Customer-Provided Keys) is an Amazon S3 encryption method where customers provide their own encryption keys for securing objects. AWS handles the encryption and decryption, but the keys are managed entirely by the customer. LocalStack supports SSE-C parameter validation for the following S3 APIs: - [`PutObject`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html) - [`GetObject`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html) - [`HeadObject`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html) - [`GetObjectAttributes`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectAttributes.html) - [`CopyObject`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html) - [`CreateMultipartUpload`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateMultipartUpload.html) - [`UploadPart`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_UploadPart.html) However, LocalStack does not support the actual encryption and decryption of objects using SSE-C. ## S3 Replication S3 Replication allows you to automatically copy objects from a source bucket to one or more destination buckets. Replication can occur within the same region or across regions, and across different accounts. LocalStack supports the following replication configurations: - **One-way replication**: Objects are replicated from a source bucket to a destination bucket. You can scope replication using prefix-based or tag-based filtering, and optionally override the storage class for objects written to the destination bucket. - **Two-way replication**: Both buckets are configured as source and destination for each other, and replication is configured to work in both directions. ### IAM enforcement LocalStack supports IAM enforcement for S3 replication. IAM permissions are evaluated in the context of each replication task using the IAM engine directly, which mirrors how AWS itself handles replication permissions. ### Metadata replication LocalStack supports replication of object metadata, specifically tags and Object Lock settings. Metadata replication operates in two modes: - **Default metadata replication**: When a source object's metadata is modified, those changes are automatically propagated to all of its replicas. This behavior is enabled by default and requires no additional configuration. - **Replica metadata synchronization**: When enabled on the destination bucket, metadata changes made directly to a replica are synced back to the source object. This applies only when two-way replication is configured. See [Replication for metadata changes](https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication-for-metadata-changes.html) in the AWS documentation for more details. ### ReplicationStatus Replicated objects are assigned a `ReplicationStatus` field, which you can inspect with `GetObject` or `HeadObject`. The possible values follow AWS semantics: | Status | Meaning | |---|---| | `PENDING` | Replication has been queued but not yet completed | | `COMPLETED` | Object was successfully replicated to the destination | | `FAILED` | Replication could not be completed | | `REPLICA` | This object is itself a copy created by replication | :::note The following replication features are not yet supported in LocalStack and will be available in a future release: - **`s3:ReplicateTags` deny evaluation**: Explicitly denying `s3:ReplicateTags` will not cause replication to be denied if the object has tags. - **KMS-encrypted object replication**: Objects encrypted with customer-provided KMS keys are not replicated, even when replication of KMS-encrypted objects is explicitly configured. See [Replicating objects created with server-side encryption using AWS KMS keys](https://docs.aws.amazon.com/AmazonS3/latest/userguide/replication-config-for-kms-objects.html#replications) in the AWS documentation for more details. - **ACL replication**: Replication of Access Control Lists is not currently supported. ::: ## Resource Browser The LocalStack Web Application provides a [Resource Browser](/aws/connecting/console/resource-browser) for managing S3 buckets & configurations. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **S3** under the **Storage** section. ![S3 Resource Browser](/images/aws/s3-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Bucket**: Create a new S3 bucket by specifying a **Bucket Name**, **Bucket Configuration**, **ACL**, **Object Ownership**, and more. - **Objects & Permissions**: View, upload, download, and delete objects in your S3 buckets. You can also view and edit the permissions, like the CORS Configuration for the bucket. - **Create Folder**: Create a new folder in your S3 bucket by clicking on the **Create Folder** button and specifying a **Folder Name**. - **Delete Bucket**: Delete an S3 bucket by selecting the S3 bucket and clicking on **Actions** button and clicking on **Remove Selected**. ## Examples The following code snippets and sample applications provide practical examples of how to use S3 in LocalStack for various use cases: - [Full-Stack application with Lambda, DynamoDB & S3 for shipment validation](https://github.com/localstack-samples/sample-shipment-list-demo-lambda-dynamodb-s3). - [Serverless Transcription application using Transcribe, S3, Lambda, SQS, and SES](https://github.com/localstack/sample-transcribe-app) - [Query data in S3 Bucket with Amazon Athena, Glue Catalog & CloudFormation](https://github.com/localstack/query-data-s3-athena-glue-sample) - [Serverless Image Resizer with Lambda, S3, SNS, and SES](https://github.com/localstack/serverless-image-resizer) - [Host a static website locally using Simple Storage Service (S3) and Terraform with LocalStack](https://docs.localstack.cloud/aws/tutorials/s3-static-website-terraform/) ## API Coverage ## API Coverage (S3 Control) # S3 Tables > Get started with Amazon S3 Tables on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Amazon S3 Tables is a managed Apache Iceberg table catalog that uses S3 storage. It acts as a catalog that transparently manages the underlying S3 buckets for you, providing built-in maintenance features like automatic compaction and snapshot management. It is designed for analytics workloads that need high read/write throughput and simplified table operations without having to directly manage S3 bucket infrastructure. LocalStack lets you use the S3 Tables API locally to create table buckets, organize tables in namespaces, and manage table metadata locations. The supported APIs are available on the [API coverage section](#api-coverage), which provides information on the extent of S3 Tables' integration with LocalStack. ## Getting started This guide is designed for users new to S3 Tables and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a table bucket, a namespace, a table, and how to retrieve table details and metadata location with the AWS CLI. ### Create a table bucket You can create a table bucket to store S3 Tables using the [`CreateTableBucket`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_s3Buckets_CreateTableBucket.html) API. Run the following command to create a table bucket named `my-table-bucket`: ```bash lstk aws s3tables create-table-bucket --name my-table-bucket ``` ```bash title="Output" { "arn": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket" } ``` ### Create a namespace Namespaces help organize tables within a table bucket. You can create a namespace within the table bucket using the [`CreateNamespace`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_s3Buckets_CreateNamespace.html) API. Run the following command to create a namespace named `my_namespace` within the table bucket `my-table-bucket`: ```bash lstk aws s3tables create-namespace \ --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket \ --namespace my_namespace ``` ```bash title="Output" { "tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket", "namespace": [ "my_namespace" ] } ``` ### Create a table You can also create a table within the namespace with the [`CreateTable`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_s3Tables_CreateTable.html) API. Run the following command to create a table named `my_table` within the namespace `my_namespace`: ```bash lstk aws s3tables create-table \ --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket \ --namespace my_namespace \ --name my_table \ --format ICEBERG ``` ```bash title="Output" { "tableARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket/table/my_table", "versionToken": "0c0c1509" } ``` ### Retrieve table information You can describe the table to view details such as ARN, namespace, format, and warehouse location using the [`GetTable`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_s3Tables_GetTable.html) API. Run the following command to describe the table `my_table`: ```bash lstk aws s3tables get-table \ --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket \ --namespace my_namespace \ --name my_table ``` ```bash title="Output" { "name": "my_table", "type": "customer", "tableARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket/table/my_table", "namespace": [ "my_namespace" ], "namespaceId": "380d99d1-abbf-4121-8e2c-c9a06e2def06", "versionToken": "0c0c1509", "warehouseLocation": "s3://hqpdve6ni1lb7w5bdn24lruswomtsh5bdrw66oip--table-s3", "createdAt": "2025-10-23T15:34:59.193399Z", "createdBy": "000000000000", "modifiedAt": "2025-10-23T15:34:59.193400Z", "ownerAccountId": "000000000000", "format": "ICEBERG", "tableBucketId": "bead5f2e-405f-4c66-b8f3-545f89ee1058" } ``` ### Retrieve table metadata location You can fetch the warehouse location used for table metadata using the [`GetTableMetadataLocation`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_s3Tables_GetTableMetadataLocation.html) API. Run the following command to fetch the warehouse location for the table `my_table`: ```bash lstk aws s3tables get-table-metadata-location \ --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket \ --namespace my_namespace \ --name my_table ``` ```bash title="Output" { "versionToken": "0c0c1509", "metadataLocation": "s3://hqpdve6ni1lb7w5bdn24lruswomtsh5bdrw66oip--table-s3/metadata/00000-b6d96c57-403a-4387-ac59-ec55ac2e646b.metadata.json", "warehouseLocation": "s3://hqpdve6ni1lb7w5bdn24lruswomtsh5bdrw66oip--table-s3" } ``` ### List tables in a namespace You can list tables in the `my_namespace` namespace using the [`ListTables`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_s3Tables_ListTables.html) API. Run the following command to list tables in the namespace `my_namespace`: ```bash lstk aws s3tables list-tables \ --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket \ --namespace my_namespace ``` ```bash title="Output" { "tables": [ { "namespace": [ "my_namespace" ], "name": "my_table", "type": "customer", "tableARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket/table/my_table", "createdAt": "2025-10-23T15:34:59.193399Z", "modifiedAt": "2025-10-23T15:34:59.193400Z" } ] } ``` ## Querying S3 Tables from Athena LocalStack [Athena](/aws/services/athena/) can query S3 Tables data through a Glue federated catalog. Once you register a federated `s3tablescatalog` in Glue and add a matching Athena data catalog, you can run SQL against your S3 Tables namespaces and tables directly from Athena. See [S3 Tables in the Athena documentation](/aws/services/athena/#s3-tables) for the full workflow. ## API Coverage # SageMaker > Get started with SageMaker on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Amazon SageMaker is a fully managed service provided by Amazon Web Services (AWS) that provides the tools to build, train, and deploy machine-learning models in the cloud for predictive analytics applications. It streamlines the machine learning development process, reduces the time and effort required to build and deploy models, and offers the scalability and flexibility needed for large-scale machine learning projects in the AWS cloud. LocalStack provides a local version of the SageMaker API, which allows running jobs to create machine learning models (e.g., using PyTorch) and to deploy them. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Sagemaker's integration with LocalStack. :::note LocalStack supports custom-built models in SageMaker. You can push your Docker image to LocalStack's Elastic Container Registry (ECR) and use it in SageMaker. LocalStack will use the local ECR image to create a SageMaker model. ::: ## Getting started This guide is designed for users new to SageMaker and assumes basic knowledge of Python3 and [AWS SDK for Python (Boto3)](https://aws.amazon.com/sdk-for-python/). We will demonstrate an application illustrating running a machine learning job using the SageMaker API locally that perform the following: - Set up an MNIST model in SageMaker using LocalStack. - Creates a SageMaker Endpoint for accessing the model - Invokes the endpoint directly on the container via Boto3 :::note SageMaker is a fairly comprehensive API for now. Currently a subset of the functionality is provided locally, but new features are being added on a regular basis. ::: ### Download the sample application You can download the sample application from [GitHub](https://github.com/localstack/localstack-pro-samples/tree/master/sagemaker-inference) or by running the following commands: ```bash mkdir localstack-samples && cd localstack-samples git init git remote add origin -f git@github.com:localstack/localstack-pro-samples.git git config core.sparseCheckout true echo sagemaker-inference >> .git/info/sparse-checkout git pull origin master ``` ### Set up the environment After downloading the sample application, you can set up your Docker Client to pull the AWS Deep Learning images by running the following command: ```bash aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 763104351884.dkr.ecr.us-east-1.amazonaws.com ``` Since the images are quite large (several gigabytes), it's a good idea to pull the images using Docker in advance. ```bash docker pull 763104351884.dkr.ecr.us-east-1.amazonaws.com/pytorch-inference:1.5.0-cpu-py3 ``` ### Run the sample application Start your LocalStack container using your preferred method. Run the sample application by executing the following command: ```bash python3 main.py ``` ```bash title="Output" Creating bucket... Uploading model data to bucket... Creating model in SageMaker... Adding endpoint configuration... Creating endpoint... Checking endpoint status... Endpoint not ready - waiting... Checking endpoint status... Endpoint ready! Invoking via boto... Predicted digits: [7, 3] Invoking endpoint directly... Predicted digits: [2, 6] ``` You can also invoke a serverless endpoint, by navigating to `main.py` and uncommenting the [`run_serverless`](https://github.com/localstack/localstack-pro-samples/blob/cca7a59e0b2b46a18a3db226c31d44401b68447e/sagemaker-inference/main.py#L134) function call. ## Resource Browser The LocalStack Web Application provides a [Resource Browser](/aws/connecting/console/resource-browser) for managing Sagemaker resources. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **Sagemaker** under the **Compute** section. The Resource Browser displays Models, Endpoint Configurations and Endpoint. You can click on individual resources to view their details. ![Sagemaker Resource Browser](/images/aws/sagemaker-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create and Remove Models**: You can remove existing model and create a new model with the required configuration ![Sagemaker Create Model](/images/aws/sagemaker-create-model.png) - **Endpoint Configurations & Endpoints**: You can create endpoints from the resource browser that hosts your deployed machine learning model. You can also create endpoint configuration that specifies the type and number of instances that will be used to serve your model on an endpoint. ## Examples The following code snippets and sample applications provide practical examples of how to use Sagemaker in LocalStack for various use cases: - [MNIST handwritten digit recognition model](https://github.com/localstack-samples/sample-mnist-digit-recognition-sagemaker) demonstrates a web application that allows users to draw a digit and submit it to a locally running SageMaker endpoint. ## Limitations Currently, GPU models are not supported by the LocalStack SageMaker implementation. ## API Coverage # EventBridge Scheduler > Get started with EventBridge Scheduler on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction EventBridge Scheduler is a service that enables you to schedule the execution of your AWS Lambda functions, Amazon ECS tasks, and Amazon Batch jobs. You can use EventBridge Scheduler to create schedules that run at a specific time or at regular intervals. You can also use EventBridge Scheduler to create schedules that run within a flexible time window. LocalStack allows you to use the Scheduler APIs in your local environment to create and run schedules. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of EventBridge Scheduler's integration with LocalStack. ## Getting started This guide is designed for users new to EventBridge Scheduler and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how you can create a new schedule, list all schedules, and tag a schedule using the EventBridge Scheduler APIs. ### Create a new SQS queue You can create a new SQS queue using the [`CreateQueue`](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_CreateQueue.html) API. Run the following command to create a new SQS queue: ```bash lstk aws sqs create-queue --queue-name local-notifications ``` You can fetch the Queue ARN using the [`GetQueueAttributes`](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_GetQueueAttributes.html) API. Run the following command to fetch the Queue ARN by specifying the Queue URL: ```bash lstk aws sqs get-queue-attributes \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/local-notifications \ --attribute-names All ``` Save the Queue ARN for later use. ### Create a new schedule You can create a new schedule using the [`CreateSchedule`](https://docs.aws.amazon.com/eventbridge/latest/APIReference/API_CreateSchedule.html) API. Run the following command to create a new schedule: ```bash lstk aws scheduler create-schedule \ --name sqs-templated-schedule \ --schedule-expression 'rate(5 minutes)' \ --target '{"RoleArn": "arn:aws:iam::000000000000:role/schedule-role", "Arn":"arn:aws:sqs:us-east-1:000000000000:local-notifications", "Input": "test" }' \ --flexible-time-window '{ "Mode": "OFF"}' ``` ```bash title="Output" { "ScheduleArn": "arn:aws:scheduler:us-east-1:000000000000:schedule/default/sqs-templated-schedule" } ``` ### List all schedules You can list all schedules using the [`ListSchedules`](https://docs.aws.amazon.com/eventbridge/latest/APIReference/API_ListSchedules.html) API. Run the following command to list all schedules: ```bash lstk aws scheduler list-schedules ``` ```bash title="Output" { "Schedules": [ { "Arn": "arn:aws:scheduler:us-east-1:000000000000:schedule/default/sqs-templated-schedule", "CreationDate": "2024-07-11T23:13:15.296906+05:30", "GroupName": "default", "LastModificationDate": "2024-07-11T23:13:15.296906+05:30", "Name": "sqs-templated-schedule", "State": "ENABLED", "Target": { "Arn": "arn:aws:sqs:us-east-1:000000000000:local-notifications" } } ] } ``` ### Tag a schedule You can tag a schedule using the [`TagResource`](https://docs.aws.amazon.com/eventbridge/latest/APIReference/API_TagResource.html) API. Run the following command to tag a schedule: ```bash lstk aws scheduler tag-resource \ --resource-arn arn:aws:scheduler:us-east-1:000000000000:schedule/default/sqs-templated-schedule \ --tags Key=Name,Value=Test ``` You can view the tags associated with a schedule using the [`ListTagsForResource`](https://docs.aws.amazon.com/eventbridge/latest/APIReference/API_ListTagsForResource.html) API. Run the following command to list the tags associated with a schedule: ```bash lstk aws scheduler list-tags-for-resource \ --resource-arn arn:aws:scheduler:us-east-1:00000000 ```bash title="Output" { "Tags": [ { "Key": "Name", "Value": "Test" } ] } ``` ## Current Limitations EventBridge Scheduler in LocalStack only provides mocked functionality. It does not emulate actual features such as schedule execution or target triggering for Lambda functions or SQS queues. ## API Coverage # Secrets Manager > Get started with Secrets Manager on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Secrets Manager is a service provided by Amazon Web Services (AWS) that enables you to securely store, manage, and retrieve sensitive information such as passwords, API keys, and other credentials. Secrets Manager integrates seamlessly with AWS services, making it easier to manage secrets used by various applications and services. Secrets Manager supports automatic secret rotation, replacing long-term secrets with short-term ones to mitigate the risk of compromise without requiring application updates. LocalStack allows you to use the Secrets Manager APIs in your local environment to manage, retrieve, and rotate secrets. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Secrets Manager's integration with LocalStack. ## Getting started This guide is designed for users new to Secrets Manager and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a secret, get the secret value, and rotate the secret using the AWS CLI. ### Create a secret Before your create a secret, create a file named `secrets.json` and add the following content: ```bash touch secrets.json cat > secrets.json << EOF { "username": "admin", "password": "password" } EOF ``` You can now create a secret using the [`CreateSecret`](https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_CreateSecret.html) API. Execute the following command to create a secret named `test-secret`: ```bash lstk aws secretsmanager create-secret \ --name test-secret \ --description "LocalStack Secret" \ --secret-string file://secrets.json ``` Upon successful execution, the output will provide you with the ARN of the newly created secret. This identifier will be useful for further operations or integrations. ```bash title="Output" { "ARN": "arn:aws:secretsmanager:us-east-1:000000000000:secret:test-secret-pyfjVP", "Name": "test-secret", "VersionId": "a50c6752-3343-4eb0-acf3-35c74f00f707" } ``` ### Describe the secret To retrieve the details of the secret you created earlier, you can use the [`DescribeSecret`](https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_DescribeSecret.html) API. Execute the following command: ```bash lstk aws secretsmanager describe-secret \ --secret-id test-secret ``` ```bash title="Output" { "ARN": "arn:aws:secretsmanager:us-east-1:000000000000:secret:test-secret-pyfjVP", "Name": "test-secret", "Description": "LocalStack Secret", "LastChangedDate": 1692882479.857329, "VersionIdsToStages": { "a50c6752-3343-4eb0-acf3-35c74f00f707": [ "AWSCURRENT" ] }, "CreatedDate": 1692882479.857329 } ``` You can also get a list of the secrets available in your local environment that have **Secret** in the name using the [`ListSecrets`](https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_ListSecrets.html) API. Execute the following command: ```bash lstk aws secretsmanager list-secrets \ --filters Key=name,Values=Secret ``` ### Get the secret value To retrieve the value of the secret you created earlier, you can use the [`GetSecretValue`](https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_GetSecretValue.html) API. Execute the following command: ```bash lstk aws secretsmanager get-secret-value \ --secret-id test-secret ``` ```bash title="Output" { "ARN": "arn:aws:secretsmanager:us-east-1:000000000000:secret:test-secret-pyfjVP", "Name": "test-secret", "VersionId": "a50c6752-3343-4eb0-acf3-35c74f00f707", "SecretString": "{\n \"username\": \"admin\",\n \"password\": \"password\"\n}\n", "VersionStages": [ "AWSCURRENT" ], "CreatedDate": 1692882479.857329 } ``` You can tag your secret using the [`TagResource`](https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_TagResource.html) API. Execute the following command: ```bash lstk aws secretsmanager tag-resource \ --secret-id test-secret \ --tags Key=Environment,Value=Development ``` ### Rotate the secret To rotate a secret, you need a Lambda function that can rotate the secret. You can copy the code from a [Secrets Manager template](https://docs.aws.amazon.com/secretsmanager/latest/userguide/reference_available-rotation-templates.html) or you can use a [generic Lambda function](https://github.com/aws-samples/aws-secrets-manager-rotation-lambdas/blob/master/SecretsManagerRotationTemplate/lambda_function.py) that rotates the secret. Zip the Lambda function and create a Lambda function using the [`CreateFunction`](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunction.html) API. Execute the following command: ```bash zip my-function.zip lambda_function.py lstk aws lambda create-function \ --function-name my-rotation-function \ --runtime python3.9 \ --zip-file fileb://my-function.zip \ --handler my-handler \ --role arn:aws:iam::000000000000:role/service-role/rotation-lambda-role ``` You can now set a resource policy on the Lambda function to allow Secrets Manager to invoke it using [`AddPermission`](https://docs.aws.amazon.com/lambda/latest/dg/API_AddPermission.html) API. Please note that this is not required with the default LocalStack settings, since IAM permission enforcement is disabled by default. Execute the following command: ```bash lstk aws lambda add-permission \ --function-name my-rotation-function \ --action lambda:InvokeFunction \ --statement-id SecretsManager \ --principal secretsmanager.amazonaws.com ``` You can now create a rotation schedule for the secret using the [`RotateSecret`](https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_RotateSecret.html) API. Execute the following command: ```bash lstk aws secretsmanager rotate-secret \ --secret-id MySecret \ --rotation-lambda-arn arn:aws:lambda:us-east-1:000000000000:function:my-rotation-function \ --rotation-rules "{\"ScheduleExpression\": \"cron(0 16 1,15 *?*)\", \"Duration\": \"2h\"}" ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing secrets in your local environment. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **Secrets Manager** under the **Security Identity Compliance** section. ![Secrets Manager Resource Browser](/images/aws/secrets-manager-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Secret**: Create a new secret by clicking **Add a Secret** and providing the required details, such as Name, Tags, Kms Key Id, Secret String, and more. - **View Secrets**: View the details of a secret by clicking on the secret name. You can also see the secret value by clicking on **Display Secret**. - **Edit Secret**: Edit the details of a secret by clicking on the secret name and then clicking **Edit Secret** and adding the new secret value. - **Delete Secret**: Delete a secret by clicking on the secret name and then clicking **Actions** and then **Remove Selected**. ## Examples The following code snippets and sample applications provide practical examples of how to use Secrets Manager in LocalStack for various use cases: - [Amazon RDS initialization using CDK, Lambda, ECR, and Secrets Manager](https://github.com/localstack/amazon-rds-init-cdk) ## API Coverage # Serverless Application Repository > Get started with Serverless Application Repository on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction [Serverless Application Repository](https://aws.amazon.com/serverless/serverlessrepo/) allows developers to discover, deploy, and share serverless applications and components. Using Serverless Application Repository, developers can build & publish applications and components once and share them across the community and organizations, making them accessible to others. Serverless Application Repository provides a user-friendly interface to search, filter, and browse through a diverse catalog of serverless applications. LocalStack allows you to use the Serverless Application Repository APIs in your local environment to create, update, delete, and list serverless applications and components. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Serverless Application Repository's integration with LocalStack. ## Getting started This guide is designed for users new to Serverless Application Repository and assumes basic knowledge of the SAM CLI and our [`lstk sam`](/aws/developer-tools/running-localstack/lstk/) command. Start your LocalStack container using your preferred method, such as via docker-compose. We will demonstrate how to create a SAM application that comprises a Hello World serverless application with a simple API backend using the SAM CLI and then publish it to the Serverless Application Repository by defining it using a SAM template. ### Setup the SAM application To create a sample SAM application using the `lstk sam` command, execute the following: ```bash lstk sam init --runtime python3.9 ``` This command downloads a sample SAM application template and generates a `template.yml` file in the current directory. The template includes a Lambda function and an API Gateway endpoint that supports a `GET` operation. ### Package the SAM application Next, we can use the `lstk sam` command to create a deployment package and a packaged SAM template. Add a Metadata section to your SAM template file (`template.yaml`), and specify the following properties: To create a deployment package and a packaged SAM template using the `lstk sam` command, add a Metadata section to your SAM template file (`template.yaml`) and specify the desired properties: ```yaml title="template.yaml" Metadata: AWS::ServerlessRepo::Application: Name: helloworld Description: hello world Author: author SpdxLicenseId: Apache-2.0 Labels: ['tests'] SemanticVersion: 0.0.1 ``` Once the Metadata section is added, run the following command to create the Lambda function deployment package and the packaged SAM template: ```bash lstk sam package \ --template-file template.yaml \ --output-template-file packaged.yaml ``` This command generates a `packaged.yaml` file in the current directory containing the packaged SAM template. The packaged template will be similar to the original template file, but it will now include a `CodeUri` property for the Lambda function, as shown in the example below: ```yaml title="packaged.yaml" Resources: HelloWorldFunction: Type: AWS::Serverless::Function Properties: CodeUri: s3://aws-sam-cli-managed-default-samclisourcebucket-b6325dc3/c6ce8fa8b5a97dd022ecd006536eb5a4 ``` ### Retrieve the Application ID To retrieve the Application ID for your SAM application, you can utilize [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) by running the following command: ```bash lstk aws serverlessrepo list-applications ``` In the output, you will observe the `ApplicationId` property in the output, which is the Application ID for your SAM application, along with other properties such as the `Author`, `Description`, `Name`, `SpdxLicenseId`, and `Version` providing further details about your application. ### Publish the SAM application To publish your application to the Serverless Application Repository, execute the following command: ```bash lstk sam publish \ --template packaged.yaml \ --region us-east-1 ``` ### Delete the SAM application To remove a SAM application from the Serverless Application Repository, you can use the following command: ```bash lstk aws serverlessrepo delete-application \ --application-id ``` Replace `` with the Application ID of your SAM application that you retrieved in the previous step. You can also create a CloudFormation changeset using the [`CreateCloudFormationChangeSet`](https://docs.aws.amazon.com/serverlessrepo/latest/devguide/serverlessrepo-how-to-publish.html) API, and then execute the changeset to deploy the SAM application using the [`ExecuteChangeSet`](https://docs.aws.amazon.com/AWSCloudFormation/latest/APIReference/API_ExecuteChangeSet.html) API. ## Current Limitations - Keep in mind, since the application is only registered in your individual localstack instance, you won't be able to share them with other developers. - Currently LocalStack only supports one AWS-hosted application (`"arn:aws:serverlessrepo:us-east-1:297356227824:applications/SecretsManagerRDSPostgreSQLRotationMultiUser`). ## API Coverage # Service Discovery > Get started with Service Discovery on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Service Discovery simplifies the management and discovery of services by locating and connecting to the components and resources that make up their applications. Service Discovery allows for a centralized mechanism for dynamically registering, tracking, and resolving service instances, allowing seamless communication between services. Service discovery uses Cloud Map API actions to manage HTTP and DNS namespaces for services, enabling automatic registration and discovery of services running in the cluster. LocalStack allows you to use the Service Discovery APIs in your local environment to monitor and manage your services across various environments and network topologies. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Service Discovery's integration with LocalStack. ## Getting Started This guide is designed for users new to Service Discovery and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create an ECS service containing a Fargate task that uses Service Discovery with the AWS CLI. ### Create a Cloud Map service discovery namespace To set up a private Cloud Map service discovery namespace, you can utilize the [`CreatePrivateDnsNamespace`](https://docs.aws.amazon.com/cloud-map/latest/api/API_CreatePrivateDnsNamespace.html) API. This API allows you to define a custom name for your namespace and specify the VPC ID where your services will be locatedBefore proceeding, make sure to create the required VPC. To create the private Cloud Map service discovery namespace, execute the following command: ```bash lstk aws servicediscovery create-private-dns-namespace \ --name tutorial \ --vpc ``` Ensure that you replace `` with the actual ID of the VPC you intend to use for the namespace. Upon running this command, you will receive an output containing an `OperationId`. This identifier can be used to check the status of the operation. To verify the status of the operation, execute the following command: ```bash lstk aws servicediscovery get-operation \ --operation-id ``` The output will consist of a `NAMESPACE` ID, which you will need to create a service within the namespace. ### Create a Cloud Map service After creating the private Cloud Map service discovery namespace, you can proceed to create a service within that namespace using the [`CreateService`](https://docs.aws.amazon.com/cloud-map/latest/api/API_CreateService.html) API This service represents a specific component or resource in your application. To create a service within the namespace, execute the following command: ```bash lstk aws servicediscovery create-service \ --name myapplication \ --dns-config "NamespaceId="",DnsRecords=[{Type="A",TTL="300"}]" \ --health-check-custom-config FailureThreshold=1 ``` Upon successful execution, the output will provide you with the Service ID and the Amazon Resource Name (ARN) of the newly created service. These identifiers will be useful for further operations or integrations. ### Create an ECS cluster To integrate the service you created earlier with an ECS (Elastic Container Service) service, you can follow the steps below. Start by creating an ECS cluster using the [`CreateCluster`](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_CreateCluster.html) API. Execute the following command: ```bash lstk aws ecs create-cluster \ --cluster-name tutorial ``` ### Register a task definition Next, you will register a task definition that's compatible with Fargate. Create a file named `fargate-task.json` and add the following content: ```json title="fargate-task.json" showLineNumbers { "family": "tutorial-task-def", "networkMode": "awsvpc", "containerDefinitions": [ { "name": "sample-app", "image": "httpd:2.4", "portMappings": [ { "containerPort": 80, "hostPort": 80, "protocol": "tcp" } ], "essential": true, "entryPoint": [ "sh", "-c" ], "command": [ "/bin/sh", "-c", "echo ' Amazon ECS Sample App

Amazon ECS Sample App

Congratulations!

Your application is now running on a container in Amazon ECS.

' > /usr/local/apache2/htdocs/index.html && httpd-foreground" ] } ], "requiresCompatibilities": [ "FARGATE" ], "cpu": "256", "memory": "512" } ``` Register the task definition using the [`RegisterTaskDefinition`](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_RegisterTaskDefinition.html) API. Execute the following command: ```bash lstk aws ecs register-task-definition \ --cli-input-json file://fargate-task.json ``` ### Create an ECS service To create an ECS service, you will need to retrieve the `securityGroups` and `subnets` associated with the VPC used to create the Cloud Map namespace. You can obtain this information by using the [`DescribeVpcs`](https://docs.aws.amazon.com/vpc/latest/APIReference/API_DescribeVpcs.html) API. Execute the following command to retrieve the details of all VPCs: ```bash lstk aws ec2 describe-vpcs ``` The output will include a list of VPCs. Locate the VPC that was used to create the Cloud Map namespace and make a note of its `VpcId` value. Next, execute the following commands to retrieve the `securityGroups` and `subnets` associated with the VPC: ```bash lstk aws ec2 describe-security-groups \ --filters Name=vpc-id,Values=vpc- \ --query 'SecurityGroups[*].[GroupId, GroupName]' \ --output text lstk aws ec2 describe-subnets \ --filters Name=vpc-id,Values=vpc- \ --query 'Subnets[*].[SubnetId, CidrBlock]' \ --output text ``` Replace `` with the actual VpcId value of the VPC you identified earlier. Make a note of the `GroupId` and `SubnetId` values. Create a new file named `ecs-service-discovery.json` and add the following content to it: ```json title="ecs-service-discovery.json" showLineNumbers { "cluster": "tutorial", "serviceName": "ecs-service-discovery", "taskDefinition": "tutorial-task-def", "serviceRegistries": [ { "registryArn": } ], "launchType": "FARGATE", "platformVersion": "LATEST", "networkConfiguration": { "awsvpcConfiguration": { "assignPublicIp": "ENABLED", "securityGroups": [ "sg-*" ], // Add the security group IDs here "subnets": [ "subnet-*" ] // Add the subnet IDs here } }, "desiredCount": 1 } ``` Create your ECS service using the [`CreateService`](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_CreateService.html) API. Execute the following command: ```bash lstk aws ecs create-service \ --cli-input-json file://ecs-service-discovery.json ``` ### Verify the service You can use the Service Discovery service ID to verify that the service was created successfully. Execute the following command: ```bash lstk aws servicediscovery list-instances \ --service-id ``` The output will consist of the resource ID, and you can further use the [`DiscoverInstances`](https://docs.aws.amazon.com/cloud-map/latest/api/API_DiscoverInstances.html) API. This API allows you to query the DNS records associated with the service and perform various operations. To explore the DNS records of your service and perform other operations, refer to the [AWS CLI documentation](https://docs.aws.amazon.com/cli/latest/reference/servicediscovery/index.html) for comprehensive instructions and examples. ### Using filters Filters can be used to narrow down the results of a list operation. Filters are supported for the following operations: - [`list-namespaces`](https://docs.aws.amazon.com/cli/latest/reference/servicediscovery/list-namespaces.html) - [`list-services`](https://docs.aws.amazon.com/cli/latest/reference/ecs/list-services.html) - [`discover-instances`](https://docs.aws.amazon.com/cli/latest/reference/servicediscovery/discover-instances.html) Using `list-namespaces` you can filter for the parameters `TYPE`, `NAME`, `HTTP_NAME`. Using `list-services` it is only possible to filter for `NAMESPACE_ID`. Both `list-services` and `list-namespaces` support `EQ` (default condition if not specified) and `BEGINS_WITH` as conditions. Both conditions and only support a single value to match by. The following examples demonstrate how to use filters with these operations: ```bash lstk aws servicediscovery list-namespaces \ --filters "Name=HTTP_NAME,Values=['example-namespace'],Condition=EQ" lstk aws servicediscovery list-services \ --filters "Name=NAMESPACE_ID,Values=['id_to_match']" ``` The command `discover-instance` supports parameters and optional parameters as filter criteria. Conditions in parameters must match return values, while if one ore more conditions in optional parameters match, the subset is returned, if no conditions in optional parameters match, all unfiltered results are returned. This command will only return instances where the parameter `env` is equal to `fuu`: ```bash lstk aws servicediscovery discover-instances \ --namespace-name example-namespace \ --service-name example-service \ --query-parameters "env"="fuu" ``` This command instead will return all instances where the optional parameter `env` is equal to `bar`, but if no instances match, all instances are returned: ```bash lstk aws servicediscovery discover-instances \ --namespace-name example-namespace \ --service-name example-service \ --optional-parameters "env"="bar" ``` ## API Coverage # Simple Email Service (SES) > Get started with Amazon Simple Email Service (SES) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Simple Email Service (SES) is an emailing service that can be integrated with other cloud-based services. It provides API to facilitate email templating, sending bulk emails and more. The supported APIs are available on the API coverage page for [SESv1](#api-coverage-sesv1) and [SESv2](#api-coverage-sesv2). :::note For advanced features like SMTP integration and other emulation capabilities, please refer to the Ultimate plan. ::: ## Getting Started This is an introductory guide to get started with SES. Basic knowledge of the AWS CLI and LocalStack [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command is assumed. Start LocalStack using your preferred method. To be able to send emails, we need to create a verified identity. A verified identity appears as part of the 'From' field in the sent email. A singular email identity can be added using the `VerifyEmailIdentity` operation. ```bash lstk aws ses verify-email-identity --email hello@example.com lstk aws ses list-identities ``` ```bash title="Output" { "Identities": [ "hello@example.com" ] } ``` :::note On AWS, verifying email identities or domain identities require additional steps like changing DNS configuration or clicking verification links respectively. In LocalStack, identities are automatically verified. ::: Next, emails can be sent using the `SendEmail` operation. ```bash lstk aws ses send-email \ --from "hello@example.com" \ --message 'Body={Text={Data="This is the email body"}},Subject={Data="This is the email subject"}' \ --destination 'ToAddresses=jeff@aws.com' ``` ```bash title="Output" { "MessageId": "labpqxukegeaftfh-ymaouvvy-ribr-qeoy-izfp-kxaxbfcfsgbh-wpewvd" } ``` :::note In LocalStack for AWS, it is possible to send real emails via an SMTP server. ::: ## Retrieve Sent Emails LocalStack keeps track of all sent emails for retrospection. Sent messages can be retrieved in following ways: - **API endpoint:** LocalStack provides a service endpoint (`/_aws/ses`) which can be used to return in-memory saved messages. A `GET` call returns all messages. Query parameters `id` and `email` can be used to filter by message ID and message source respectively. ```bash curl --silent localhost.localstack.cloud:4566/_aws/ses?email=hello@example.com | jq . ``` ```bash title="Output" { "messages": [ { "Id": "dqxhhgoutkmylpbc-ffuqlkjs-ljld-fckp-hcph-wcsrkmxhhldk-pvadjc", "Region": "eu-central-1", "Destination": { "ToAddresses": [ "jeff@aws.com" ] }, "Source": "hello@example.com", "Subject": "This is the email subject", "Body": { "text_part": "This is the email body", "html_part": null }, "Timestamp": "2023-09-11T08:37:13" } ] } ``` A `DELETE` call clears all messages from the memory. The query parameter `id` can be used to delete only a specific message. ```bash curl -X DELETE localhost.localstack.cloud:4566/_aws/ses?id=dqxhhgoutkmylpbc-ffuqlkjs-ljld-fckp-hcph-wcsrkmxhhldk-pvadjc ``` - **Filesystem:** All messages are saved to the state directory (see [filesystem layout](/aws/customization/advanced/filesystem)). The files are saved as JSON in the `ses/` subdirectory and named by the message ID. ## SMTP Integration LocalStack for AWS supports sending emails via an SMTP server. To enable this, set the connections parameters and access credentials for the server in the configuration. Refer to the [Configuration](/aws/customization/configuration-options/#emails) guide for details. :::tip If you do not have access to a live SMTP server, you can use tools like [MailDev](https://github.com/maildev/maildev) or [smtp4dev](https://github.com/rnwood/smtp4dev). These run as Docker containers on your local machine. Make sure they run in the same Docker network as the LocalStack container. ::: ## Resource Browser LocalStack Web Application provides a resource browser for managing email identities and introspecing sent emails. ![SES Resource Browser](/images/aws/ses-resource-browser.png) The Resource Browser allows you to perform following actions: - **Create Email Identity**: Create an email identity by clicking **Create Identity** and specifying the email address. - **View Sent Emails**: View all sent emails from an email identity by clicking the email address. You can the view the details of a sent email by selecting them from the list. - **Send Emails**: On selecting an email identity, click **Send Message** and specify destination fields (To, CC and BCC addresses) and the body (Plaintext, HTML) to send an email. ## Current Limitations - It is currently not possible to [receive emails via SES](https://docs.aws.amazon.com/ses/latest/dg/receiving-email.html) in LocalStack. - All operations related to Receipt Rules are mocked. ## API Coverage (SESv1) ## API Coverage (SESv2) # Shield > Get started with Shield on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Shield is a managed Distributed Denial of Service (DDoS) protection service that safeguards applications running on AWS. Shield provides always-on detection and inline mitigations that minimize application downtime and latency, by protecting users from L4, L7 and most common L3, L4 network and transport layer DDoS attacks. Shield detection and mitigation is designed to protect against threats, including ones that are not known to the service at the time of detection. LocalStack allows you to use the Shield APIs in your local environment, and provides a simple way to mock and test the Shield service locally. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Shield's integration with LocalStack. ## Getting Started This guide is designed for users new to Shield and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a Shield protection, list all protections, and delete a protection with the AWS CLI. ### Create a Shield Protection To create a Shield protection, use the [`CreateProtection`](https://docs.aws.amazon.com/cli/latest/reference/shield/create-protection.html) API. The following command creates a Shield protection for a resource: ```bash lstk aws shield create-protection \ --name "my-protection" \ --resource-arn "arn:aws:elasticloadbalancing:us-east-1:000000000000:loadbalancer/app/my-alb/1234567890" ``` ```bash title="Output" { "ProtectionId": "67908d33-16c0-443d-820a-31c02c4d5976" } ``` ### List all Protections To list all Shield protections, use the [`ListProtections`](https://docs.aws.amazon.com/cli/latest/reference/shield/list-protections.html) API. The following command lists all Shield protections: ```bash lstk aws shield list-protections ``` ```bash title="Output" { "Protections": [ { "Id": "67908d33-16c0-443d-820a-31c02c4d5976", "Name": "my-protection", "ResourceArn": "arn:aws:elasticloadbalancing:us-east-1:000000000000:loadbalancer/app/my-alb/1234567890", "ProtectionArn": "arn:aws:shield::000000000000:protection/67908d33-16c0-443d-820a-31c02c4d5976" } ] } ``` ### Describe a Protection To describe a Shield protection, use the [`DescribeProtection`](https://docs.aws.amazon.com/cli/latest/reference/shield/describe-protection.html) API. The following command describes a Shield protection: ```bash lstk aws shield describe-protection \ --protection-id "67908d33-16c0-443d-820a-31c02c4d5976" ``` Replace the protection ID with the ID of the protection you want to describe. ```bash title="Output" { "Protection": { "Id": "67908d33-16c0-443d-820a-31c02c4d5976", "Name": "my-protection", "ResourceArn": "arn:aws:elasticloadbalancing:us-east-1:000000000000:loadbalancer/app/my-alb/1234567890", "ProtectionArn": "arn:aws:shield::000000000000:protection/67908d33-16c0-443d-820a-31c02c4d5976" } } ``` ### Delete a Protection To delete a Shield protection, use the [`DeleteProtection`](https://docs.aws.amazon.com/cli/latest/reference/shield/delete-protection.html) API. The following command deletes a Shield protection: ```bash lstk aws shield delete-protection \ --protection-id "67908d33-16c0-443d-820a-31c02c4d5976" ``` ## Current Limitations Shield Config is currently mocked in LocalStack. You can create, read, update, and delete Shield protections & subscriptions, but the actual protection or subscription is not applied to any resources. If you need this feature, please consider opening a [feature request on GitHub Discussion](https://github.com/orgs/localstack/discussions/new/choose). ## API Coverage # Simple Notification Service (SNS) > Get started with Simple Notification Service (SNS) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Simple Notification Service (SNS) is a serverless messaging service that can distribute a massive number of messages to multiple subscribers and can be used to send messages to mobile devices, email addresses, and HTTP(s) endpoints. SNS employs the Publish/Subscribe, an asynchronous messaging pattern that decouples services that produce events from services that process events. LocalStack allows you to use the SNS APIs in your local environment to coordinate the delivery of messages to subscribing endpoints or clients. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of SNS's integration with LocalStack. ## Getting started This guide is intended for users who wish to get more acquainted with SNS over LocalStack. It assumes you have basic knowledge of the AWS CLI (and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command). Start your LocalStack container using your preferred method. We will demonstrate how to create an SNS topic, publish messages, and subscribe to the topic. ### Create an SNS topic To create an SNS topic, use the [`CreateTopic`](https://docs.aws.amazon.com/sns/latest/api/API_CreateTopic.html) API. Run the following command to create a topic named `localstack-topic`: ```bash lstk aws sns create-topic --name localstack-topic ``` You can set the SNS topic attribute using the SNS topic you created previously by using the [`SetTopicAttributes`](https://docs.aws.amazon.com/sns/latest/api/API_SetTopicAttributes.html) API. Run the following command to set the `DisplayName` attribute for the topic: ```bash lstk aws sns set-topic-attributes \ --topic-arn arn:aws:sns:us-east-1:000000000000:localstack-topic \ --attribute-name DisplayName \ --attribute-value MyTopicDisplayName ``` You can list all the SNS topics using the [`ListTopics`](https://docs.aws.amazon.com/sns/latest/api/API_ListTopics.html) API. Run the following command to list all the SNS topics: ```bash lstk aws sns list-topics ``` ### Get attributes and publish messages to SNS topic You can get attributes for a single SNS topic using the [`GetTopicAttributes`](https://docs.aws.amazon.com/sns/latest/api/API_GetTopicAttributes.html) API. Run the following command to get the attributes for the SNS topic: ```bash lstk aws sns get-topic-attributes \ --topic-arn arn:aws:sns:us-east-1:000000000000:localstack-topic ``` You can change the `topic-arn` to the ARN of the SNS topic you created previously. To publish messages to the SNS topic, create a new file named `messages.txt` in your current directory and add some content. Run the following command to publish messages to the SNS topic using the [`Publish`](https://docs.aws.amazon.com/sns/latest/api/API_Publish.html) API: ```bash lstk aws sns publish \ --topic-arn "arn:aws:sns:us-east-1:000000000000:localstack-topic" \ --message file://message.txt ``` ### Subscribing to SNS topics and setting subscription attributes You can subscribe to the SNS topic using the [`Subscribe`](https://docs.aws.amazon.com/sns/latest/api/API_Subscribe.html) API. Run the following command to subscribe to the SNS topic: ```bash lstk aws sns subscribe \ --topic-arn arn:aws:sns:us-east-1:000000000000:localstack-topic \ --protocol email \ --notification-endpoint test@gmail.com ``` You can configure the SNS Subscription attributes, using the `SubscriptionArn` returned by the previous step. For example, run the following command to set the `RawMessageDelivery` attribute for the subscription: ```bash lstk aws sns set-subscription-attributes \ --subscription-arn arn:aws:sns:us-east-1:000000000000:test-topic:b6f5e924-dbb3-41c9-aa3b-589dbae0cfff \ --attribute-name RawMessageDelivery --attribute-value true ``` ### Working with SQS subscriptions for SNS The getting started covers email subscription, but SNS can integrate with many AWS technologies as seen in the [aws-cli docs](https://docs.aws.amazon.com/cli/latest/reference/sns/subscribe.html). A Common technology to integrate with is SQS. First we need to ensure we create an SQS queue named `my-queue`: ```bash lstk aws sqs create-queue --queue-name my-queue ``` ```bash title="Output" { "QueueUrl": "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/my-queue" } ``` Subscribe the SQS queue to the topic we created previously: ```bash lstk aws sns subscribe \ --topic-arn "arn:aws:sns:us-east-1:000000000000:localstack-topic" \ --protocol sqs \ --notification-endpoint "arn:aws:sqs:us-east-1:000000000000:my-queue" ``` ```bash title="Output" { "SubscriptionArn": "arn:aws:sns:us-east-1:000000000000:localstack-topic:636e2a73-0dda-4e09-9fdf-77f113d0edd8" } ``` Sending a message to the queue, via the topic ```bash lstk aws sns publish --topic-arn "arn:aws:sns:us-east-1:000000000000:localstack-topic" --message "hello" { "MessageId": "5a1593ce-411b-44dc-861d-907daa05353b" } ``` Check that our message has arrived: ```bash lstk aws sqs receive-message \ --queue-url "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/my-queue" ``` ```bash title="Output" { "Messages": [ { "MessageId": "72a15a17-5652-45ab-b4db-937f60f0c6d8", "ReceiptHandle": "YjQ0YjgzMjAtNTk2NC00ZDk0LWE4ZGYtNjljMTViOTkwOTFmIGFybjphd3M6c3FzOnVzLWVhc3QtMTowMDAwMDAwMDAwMDA6bXktcXVldWUgNzJhMTVhMTctNTY1Mi00NWFiLWI0ZGItOTM3ZjYwZjBjNmQ4IDE3MDM3MDQxMTEuNTI2MzEwNA==", "MD5OfBody": "2664b540fb6ce6fd7467cd8fb071c30f", "Body": "{\"Type\": \"Notification\", \"MessageId\": \"5a1593ce-411b-44dc-861d-907daa05353b\", \"TopicArn\": \"arn:aws:sns:us-east-1:000000000000:localstack-topic\", \"Message\": \"hello\", \"Timestamp\": \"2023-12-27T19:07:55.341Z\", \"SignatureVersion\": \"1\", \"Signature\": \"EXAMPLEpH+..\", \"SigningCertURL\": \"...\", \"UnsubscribeURL\": \"http://localhost.localstack.cloud:4566/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:000000000000:localstack-topic:636e2a73-0dda-4e09-9fdf-77f113d0edd8\"}" } ] } ``` To remove the subscription you need the subscription ARN which you can find by listing the subscriptions. You can list all the SNS subscriptions using the [`ListSubscriptions`](https://docs.aws.amazon.com/sns/latest/api/API_ListSubscriptions.html) API. Run the following command to list all the SNS subscriptions: ```bash lstk aws sns list-subscriptions ``` ```bash title="Output" { "Subscriptions": [ { "SubscriptionArn": "arn:aws:sns:us-east-1:000000000000:localstack-topic:636e2a73-0dda-4e09-9fdf-77f113d0edd8", "Owner": "000000000000", "Protocol": "sqs", "Endpoint": "arn:aws:sqs:us-east-1:000000000000:my-queue", "TopicArn": "arn:aws:sns:us-east-1:000000000000:localstack-topic" } ] } ``` Then, use the ARN to unsubscribe ```bash lstk aws sns unsubscribe \ --subscription-arn "arn:aws:sns:us-east-1:000000000000:localstack-topic:636e2a73-0dda-4e09-9fdf-77f113d0edd8" ``` ## Developer endpoints LocalStack’s SNS implementation offers additional endpoints for developers located at `/_aws/sns`. These endpoints provide the ability to access different SNS internals, like Platform Endpoint messages which are not sent to those platforms, or Subscription Tokens which you might not be able to retrieve otherwise. #### Query parameters | Parameter | Required | Description | | ----------- | -------- | ------------------------------------------------------------------------------------ | | phoneNumber | Yes | Phone number to add to the SNS opt-out list. | | accountId | No | AWS Account ID under which the opt-out should be stored. Defaults to `000000000000`. | The opt-out list is account-wide and stored under the default region `us-east-1`. #### Create phone opt-out list Add a phone number to create the opt-out list: ```bash curl -X POST "http://localhost:4566/_aws/sns/phone-opt-outs" \ -H "Content-Type: application/json" \ -d '{ "phoneNumber": "+123123123", "accountId": "000000000000" }' ``` :::note * This endpoint is intended for **local testing only**. * The opt-out list is stored per account. * Adding the same phone number multiple times is idempotent (performing the same operation once has the same effect as performing it multiple times). * Once opted out, SNS operations that respect the opt-out list (for example, `ListPhoneNumbersOptedOut`) will include the number. ::: ### Platform Endpoint messages For testing purposes, LocalStack retains all messages published to a platform endpoint in memory, making it easy to retrieve them. To learn more about SNS mobile push notifications, refer to the [AWS documentation on SNS mobile push notifications](https://docs.aws.amazon.com/sns/latest/dg/sns-mobile-application-as-subscriber.html). You can access these messages in JSON format through `GET /_aws/sns/platform-endpoint-messages`. To retrieve specific messages, you can use query parameters to filter by `accountId`, `region`, and `endpointArn`. You can also call `DELETE /_aws/sns/platform-endpoint-messages` to clear the messages. #### Query parameters | Parameter | Required | Description | | - | - | - | | `accountId` | No | The AWS Account ID from which the messages have been published. If not specified, it will use the default `000000000000` | | `region` | No | The AWS region from which the messages have been published. If not specified, it will use the default `us-east-1` | | `endpointArn` | No | The target `EndpointArn` to which the messages have been published. If specified, the response will contain only messages sent to this target. Otherwise, it will return all endpoints with their messages. | #### Response format and attributes | Attribute | Description | | - | - | | `platform_endpoint_messages` | Contains endpoints ARN as field names. Each endpoint will have its messages in an Array. | | `region` | The region of the endpoints and messages. | In this example, we will create a platform endpoint in SNS and publish a message to it. Run the following commands to create a platform endpoint: ```bash lstk aws sns create-platform-application \ --name app-test \ --platform APNS \ --attributes {} ``` ```bash title="Output" { "PlatformApplicationArn": "arn:aws:sns:us-east-1:000000000000:app/APNS/app-test" } ``` Using the `PlatformApplicationArn` from the previous call: ```bash lstk aws sns create-platform-endpoint \ --platform-application-arn "arn:aws:sns:us-east-1:000000000000:app/APNS/app-test" \ --token my-fake-token ``` ```bash title="Output" { "EndpointArn": "arn:aws:sns:us-east-1:000000000000:endpoint/APNS/app-test/c25f353e-856b-4b02-a725-6bde35e6e944" } ``` Publish a message to the platform endpoint: ```bash lstk aws sns publish \ --target-arn "arn:aws:sns:us-east-1:000000000000:endpoint/APNS/app-test/c25f353e-856b-4b02-a725-6bde35e6e944" \ --message '{"APNS_PLATFORM": "{\"aps\": {\"content-available\": 1}}"}' \ --message-structure json ``` ```bash title="Output" { "MessageId": "ed501a7a-caab-45aa-a941-2fcc64b5c227" } ``` Retrieve the messages published to the platform endpoint using [curl](https://curl.se/): ```bash curl "http://localhost:4566/_aws/sns/platform-endpoint-messages" | jq . ``` ```bash title="Output" { "platform_endpoint_messages": { "arn:aws:sns:us-east-1:000000000000:endpoint/APNS/app-test/c25f353e-856b-4b02-a725-6bde35e6e944": [ { "TargetArn": "arn:aws:sns:us-east-1:000000000000:endpoint/APNS/app-test/c25f353e-856b-4b02-a725-6bde35e6e944", "Message": "{\"APNS_PLATFORM\": \"{\\\"aps\\\": {\\\"content-available\\\": 1}}\"}", "MessageAttributes": null, "MessageStructure": "json", "Subject": null } ] }, "region": "us-east-1" } ``` With those same filters, you can reset the saved messages at `DELETE /_aws/sns/platform-endpoint-messages`. Run the following command to reset the saved messages: ```bash curl -X "DELETE" "http://localhost:4566/_aws/sns/platform-endpoint-messages" ``` We can now check that the messages have been properly deleted: ```bash curl "http://localhost:4566/_aws/sns/platform-endpoint-messages" | jq . ``` ```bash title="Output" { "platform_endpoint_messages": {}, "region": "us-east-1" } ``` ### SMS messages For testing purposes, LocalStack also retains all SMS messages published to a phone number in memory, making it easy to retrieve them. To learn more about SNS SMS notifications, refer to the [AWS documentation on SNS mobile text messaging (SMS)](https://docs.aws.amazon.com/sns/latest/dg/sns-mobile-phone-number-as-subscriber.html). You can access these messages in JSON format through `GET /_aws/sns/sms-messages`. To retrieve specific messages, you can use query parameters to filter by `accountId`, `region`, and `phoneNumber`. You can also call `DELETE /_aws/sns/sms-messages` to clear the messages. #### Query parameters | Parameter | Required | Description | | - | - | - | | `accountId` | No | The AWS Account ID from which the messages have been published. If not specified, it will use the default `000000000000` | | `region` | No | The AWS region from which the messages have been published. If not specified, it will use the default `us-east-1` | | `phoneNumber` | No | The `phoneNumber` to which the messages have been published. If specified, the response will contain only messages sent to this number. Otherwise, it will return all phone numbers with their messages. | #### Response format and attributes | Attribute | Description | | - | - | | `sms_messages` | Contains phone numbers as field names. Each phone number will have its messages in an Array. | | `region` | The region from where the messages were sent. | In this example, we will publish a message to a phone number and retrieve it: Publish a message to a phone number: ```bash lstk aws sns publish \ --phone-number "" \ --message "Hello World!" ``` ```bash title="Output" { "MessageId": "9ce56934-dcc4-45f5-ba40-13691329fc67" } ``` Retrieve the message published using [curl](https://curl.se/) and [jq](https://jqlang.github.io/jq/): ```bash curl "http://localhost:4566/_aws/sns/sms-messages" | jq . ``` ```bash title="Output" { "sms_messages": { "+123123123": [ { "PhoneNumber": "+123123123", "TopicArn": null, "SubscriptionArn": null, "MessageId": "9ce56934-dcc4-45f5-ba40-13691329fc67", "Message": "Hello World", "MessageAttributes": {}, "MessageStructure": null, "Subject": null } ] }, "region": "us-east-1" } ``` You can reset the saved messages at `DELETE /_aws/sns/sms-messages`. Using the query parameters, you can also selectively reset messages only in one region or from one phone number. Run the following command to reset the saved messages: ```bash curl -X "DELETE" "http://localhost:4566/_aws/sns/sms-messages" ``` We can now check that the messages have been properly deleted: ```bash curl "http://localhost:4566/_aws/sns/sms-messages" | jq . ``` ```bash title="Output" { "sms_messages": {}, "region": "us-east-1" } ``` ### Subscription Tokens In case of email and HTTP(S) subscriptions, a special message is sent to the subscriber with a link to confirm the subscription so that it will be able to receive the messages afterwards. SNS does not send messages to endpoints pending confirmation. However, when working with external integrations, the link sent will most probably point to your local environment, which won't be accessible from the external integration to confirm. To still be able to test your external integrations, we expose the subscription tokens so that you can manually confirm the subscription. The subscription tokens are never deleted from memory, because they can be re-used. To manually confirm the subscription, you will use [`ConfirmSubscription`](https://docs.aws.amazon.com/sns/latest/api/API_ConfirmSubscription.html). To learn more about confirming subscriptions, refer to the [AWS documentation](https://docs.aws.amazon.com/sns/latest/dg/SendMessageToHttp.confirm.html). You can access the subscription tokens in JSON format through `GET /_aws/sns/subscription-tokens/`. #### Path parameters | Parameter | Required | Description | | - | - | - | | `subscription-arn` | Yes | The SNS Subscription ARN for which you would like to fetch the tokens | #### Response format and attributes | Attribute | Description | | - | - | | `subscription_token` | The Subscription token to be used with `ConfirmSubscription`. | | `subscription_arn` | The Subscription ARN provided. | In this example, we will subscribe to an external SNS integration not confirming the subscription, retrieve the subscription token and manually confirm it: Create an SNS topic, and create a subscription to a external HTTP SNS integration: ```bash lstk aws sns create-topic --name "test-external-integration" ``` ```bash title="Output" { "TopicArn": "arn:aws:sns:us-east-1:000000000000:test-external-integration" } ``` We now create an HTTP SNS subscription to an external endpoint: ```bash lstk aws sns subscribe \ --topic-arn "arn:aws:sns:us-east-1:000000000000:test-external-integration" \ --protocol https \ --notification-endpoint "https://api.opsgenie.com/v1/json/amazonsns?apiKey=b13fd59a-9" \ --return-subscription-arn ``` ```bash title="Output" { "SubscriptionArn": "arn:aws:sns:us-east-1:000000000000:test-external-integration:c3ab47f3-b964-461d-84eb-903d8765b0c8" } ``` Now, we can check the `PendingConfirmation` status of our subscription, showing our endpoint did not confirm the subscription. You will need to use the `SubscriptionArn` from the response of your subscribe call: ```bash lstk aws sns get-subscription-attributes \ --subscription-arn "arn:aws:sns:us-east-1:000000000000:test-external-integration:c3ab47f3-b964-461d-84eb-903d8765b0c8" ``` ```bash title="Output" { "Attributes": { "TopicArn": "arn:aws:sns:us-east-1:000000000000:test-external-integration", "Endpoint": "https://api.opsgenie.com/v1/json/amazonsns?apiKey=b13fd59a-9", "Protocol": "https", "SubscriptionArn": "arn:aws:sns:us-east-1:000000000000:test-external-integration:c3ab47f3-b964-461d-84eb-903d8765b0c8", "PendingConfirmation": "true", "Owner": "000000000000", "RawMessageDelivery": "false", "SubscriptionPrincipal": "arn:aws:iam::000000000000:user/DummySNSPrincipal" } } ``` To manually confirm the subscription, we will fetch its token with our developer endpoint: ```bash curl "http://localhost:4566/_aws/sns/subscription-tokens/arn:aws:sns:us-east-1:000000000000:test-external-integration:c3ab47f3-b964-461d-84eb-903d8765b0c8" | jq . ``` ```json { "subscription_token": "75732d656173742d312f3b875fb03b875fb03b875fb03b875fb03b875fb03b87", "subscription_arn": "arn:aws:sns:us-east-1:000000000000:test-external-integration:c3ab47f3-b964-461d-84eb-903d8765b0c8" } ``` We can now use this token to manually confirm the subscription: ```bash lstk aws sns confirm-subscription \ --topic-arn "arn:aws:sns:us-east-1:000000000000:test-external-integration" \ --token 75732d656173742d312f3b875fb03b875fb03b875fb03b875fb03b875fb03b87 ``` ```bash title="Output" { "SubscriptionArn": "arn:aws:sns:us-east-1:000000000000:test-external-integration:c3ab47f3-b964-461d-84eb-903d8765b0c8" } ``` We can now finally verify the subscription has been confirmed: ```bash lstk aws sns get-subscription-attributes \ --subscription-arn "arn:aws:sns:us-east-1:000000000000:test-external-integration:c3ab47f3-b964-461d-84eb-903d8765b0c8" ``` ```bash title="Output" { "Attributes": { "TopicArn": "arn:aws:sns:us-east-1:000000000000:test-external-integration", "Endpoint": "https://api.opsgenie.com/v1/json/amazonsns?apiKey=b13fd59a-9", "Protocol": "https", "SubscriptionArn": "arn:aws:sns:us-east-1:000000000000:test-external-integration:c3ab47f3-b964-461d-84eb-903d8765b0c8", "PendingConfirmation": "false", "Owner": "000000000000", "RawMessageDelivery": "false", "SubscriptionPrincipal": "arn:aws:iam::000000000000:user/DummySNSPrincipal", "ConfirmationWasAuthenticated": "true" } } ``` SNS will now publish messages to your HTTP endpoint, even if it did not confirm itself the subscription. ### Phone opt-outs For testing purposes, LocalStack provides an internal developer endpoint to add phone numbers to the SNS opt-out list. In AWS, phone number opt-outs are typically handled via inbound SMS keywords (for example, `STOP`) or managed through Amazon Pinpoint. Since inbound SMS handling and Pinpoint integration are outside the scope of LocalStack’s SNS emulation, this endpoint allows you to opt-out phone numbers directly for local testing. This is a LocalStack internal endpoint, not part of the AWS SNS public API. ``` POST /_aws/sns/phone-opt-outs ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing SNS topics. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **SNS** under the **App Integration** section. ![SNS Resource Browser](/images/aws/sns-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Topic**: Create a new SNS topic by specifying a topic name, attributes, and tags. - **View Details and Subscription**: View details and subscription of an SNS topic by selecting the topic name and navigating to the **Details** and **Subscriptions** tabs. - **Create Subscription**: Create a new subscription for an SNS topic by selecting the topic name, navigating to the **Subscriptions** tab, and clicking the **Create Subscription** button. Fill in the required details such as protocol, endpoint, and attributes, delivery policy, return subscription ARN, and click **Create**. - **Delete Topic**: Delete an SNS topic by selecting the topic name and clicking the **Action** button, followed by **Delete Selected**. ## Examples The following code snippets and sample applications provide practical examples of how to use SNS in LocalStack for various use cases: - [Full-Stack application with AWS Lambda, DynamoDB & S3 for shipment validation](https://github.com/localstack/shipment-list-demo) - [Event-driven architecture with Amazon SNS FIFO, DynamoDB, Lambda, and S3](https://github.com/localstack/event-driven-architecture-with-amazon-sns-fifo) - [Loan Broker application with AWS Step Functions, DynamoDB, Lambda, SQS, and SNS](https://github.com/localstack/loan-broker-stepfunctions-lambda-app) - [Serverless Image Resizer with AWS Lambda, S3, SNS, and SES](https://github.com/localstack/serverless-image-resizer) ## Current Limitations - LocalStack does not support the `cidr` operator for filter policies. However, [other policies](https://docs.aws.amazon.com/sns/latest/dg/sns-subscription-filter-policies.html) are supported. ## API Coverage # Simple Queue Service (SQS) > Get started with Simple Queue Service (SQS) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Simple Queue Service (SQS) is a managed messaging service offered by AWS. It allows you to decouple different components of your applications by enabling asynchronous communication through message queues. SQS allows you to reliably send, store, and receive messages with support for standard and FIFO queues. LocalStack allows you to use the SQS APIs in your local environment to integrate and decouple distributed systems via hosted queues. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of SQS's integration with LocalStack. ## Getting started This guide is designed for users new to SQS and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create an SQS queue, retrieve queue attributes and URLs, and receive and delete messages from the queue. ### Create a queue To create an SQS queue, use the [`CreateQueue`](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_CreateQueue.html) API. Run the following command to create a queue named `localstack-queue`: ```bash lstk aws sqs create-queue --queue-name localstack-queue ``` You can list all queues in your account using the [`ListQueues`](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_ListQueues.html) API. Run the following command to list all queues in your account: ```bash lstk aws sqs list-queues ``` ```bash title="Output" { "QueueUrls": [ "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/localstack-queue" ] } ``` You can query queue attributes with the [`GetQueueAttributes`](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_GetQueueAttributes.html) API. You need to pass the `queue-url` and `attribute-names` parameters. Run the following command to retrieve the queue attributes: ```bash lstk aws sqs get-queue-attributes \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/localstack-queue \ --attribute-names All ``` To create a [FIFO queue](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-fifo-queue-message-identifiers.html), the queue name must end with the `.fifo` suffix in addition to the `FifoQueue=true` attribute set: ```bash lstk aws sqs create-queue --queue-name localstack-queue.fifo --attributes "FifoQueue=true" { "QueueUrl": "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/localstack-queue.fifo" } ``` ### Sending and receiving messages from the queue You can send a message to the SQS queue which will be queued and a consumer can pick it up. To send a message to a SQS queue, you can use the [`SendMessage`](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessage.html) API. Run the following command to send a message to the queue: ```bash lstk aws sqs send-message \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/localstack-queue \ --message-body "Hello World" ``` It will return the MD5 hash of the Message Body and a Message ID. ```bash title="Output" { "MD5OfMessageBody": "b10a8db164e0754105b7a99be72e3fe5", "MessageId": "92612c02-4879-47db-92f6-40bf2b341c07" } ``` You can receive messages from the queue using the [`ReceiveMessage`](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_ReceiveMessage.html) API. Run the following command to receive messages from the queue: ```bash lstk aws sqs receive-message \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/localstack-queue ``` You will see the Message ID, MD5 hash of the Message Body, Receipt Handle, and the Message Body in the output. ### Delete a message from the queue To delete a message from the queue, you can use the [`DeleteMessage`](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_DeleteMessage.html) API. You need to pass the `queue-url` and `receipt-handle` parameters. Run the following command to delete a message from the queue: ```bash lstk aws sqs delete-message \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/localstack-queue \ --receipt-handle ``` Replace `` with the receipt handle you received in the previous step. If you have sent multiple messages to the queue, you can purge the queue using the [`PurgeQueue`](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_PurgeQueue.html) API. Run the following command to purge the queue: ```bash lstk aws sqs purge-queue \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/localstack-queue ``` ## Dead-letter queue testing LocalStack's SQS implementation supports both regular [dead-letter queues (DLQ)](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html) and [DLQ redrive](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-configure-dead-letter-queue-redrive.html) via move message tasks. Here's an end-to-end example of how to use message move tasks to test DLQ redrive. First, create three queues. One will serve as original input queue, one as DLQ, and the third as target for DLQ redrive. ```bash lstk aws sqs create-queue --queue-name input-queue lstk aws sqs create-queue --queue-name dead-letter-queue lstk aws sqs create-queue --queue-name recovery-queue ``` ```bash title="Output" { "QueueUrl": "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/input-queue" } { "QueueUrl": "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/dead-letter-queue" } { "QueueUrl": "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/recovery-queue" } ``` Configure `dead-letter-queue` to be a DLQ for `input-queue`: ```bash lstk aws sqs set-queue-attributes \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/input-queue \ --attributes '{ "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:000000000000:dead-letter-queue\",\"maxReceiveCount\":\"1\"}" }' ``` Send a message to the input queue: ```bash lstk aws sqs send-message \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/input-queue \ --message-body '{"hello": "world"}' ``` Receive the message twice to provoke a move into the dead-letter queue: ```bash lstk aws sqs receive-message \ --visibility-timeout 0 \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/input-queue lstk aws sqs receive-message \ --visibility-timeout 0 \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/input-queue ``` In the localstack logs you should see something like the following line, indicating the message was moved to the DLQ: ```bash 2024-01-24T13:51:16.824 DEBUG --- [ asgi_gw_1] l.services.sqs.models : message SqsMessage(id=5be95a04-93f0-4b9d-8bd5-6695f34758cf,group=None) has been received 2 times, marking it for DLQ ``` Now, start a message move task to asynchronously move the messages from the DLQ into the recovery queue: ```bash lstk aws sqs start-message-move-task \ --source-arn arn:aws:sqs:us-east-1:000000000000:dead-letter-queue \ --destination-arn arn:aws:sqs:us-east-1:000000000000:recovery-queue ``` Listing the message move tasks should yield something like ```bash lstk aws sqs list-message-move-tasks \ --source-arn arn:aws:sqs:us-east-1:000000000000:dead-letter-queue ``` ```bash title="Output" { "Results": [ { "Status": "COMPLETED", "SourceArn": "arn:aws:sqs:us-east-1:000000000000:dead-letter-queue", "DestinationArn": "arn:aws:sqs:us-east-1:000000000000:recovery-queue", "ApproximateNumberOfMessagesMoved": 1, "ApproximateNumberOfMessagesToMove": 1, "StartedTimestamp": 1706097183866 } ] } ``` Receiving messages from the recovery queue should now show us the original message: ```bash lstk aws sqs receive-message \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/recovery-queue ``` ```bash title="Output" { "Messages": [ { "MessageId": "5be95a04-93f0-4b9d-8bd5-6695f34758cf", "ReceiptHandle": "NzkwMWJiZDYtMzgyNy00Nzc3LTlkODMtMmEzYTNjYjlhZWQwIGFybjphd3M6c3FzOnV...", "MD5OfBody": "49dfdd54b01cbcd2d2ab5e9e5ee6b9b9", "Body": "{\"hello\": \"world\"}" } ] } ``` ## SQS Query API The [SQS Query API](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-making-api-requests.html), provides SQS Queue URLs as endpoints, enabling direct HTTP requests to the queues. LocalStack extends support for the Query API. With LocalStack, you can conveniently test SQS Query API calls without the need to sign or include `AUTHPARAMS` in your HTTP requests. For instance, you can use a basic [curl](https://curl.se/) command to send a `SendMessage` command along with a MessageBody attribute: ```bash curl "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/localstack-queue?Action=SendMessage&MessageBody=hello%2Fworld" ``` ```xml title="Output" c6be4e95a26409675447367b3e79f663 466144ab-1d03-4ec5-8d70-97535b2957fb JU40AF5GORK0WSR75MOY3VNQ1KZ3TAI7S5KAJYGK9C5P4W4XKMGF ``` Adding the `Accept: application/json` header will make the server return JSON: To receive JSON responses from the server, include the `Accept: application/json` header in your request. Here's an example using the [curl](https://curl.se/) command: ```bash curl -H "Accept: application/json" "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/localstack-queue?Action=SendMessage&MessageBody=hello%2Fworld" ``` The response will be in JSON format: ```bash title="Output" { "SendMessageResponse": { "SendMessageResult": { "MD5OfMessageBody": "c6be4e95a26409675447367b3e79f663", "MessageId": "748297f2-4abd-4ec2-afc0-4d1a497fe604" }, "ResponseMetadata": { "RequestId": "XEA5L5AX16RTPET25U3TIRIASN6KNIT820WIT3EY7RCH7164W68T" } } } ``` ## Configuration ### Queue URLs You can control the format of the generated Queue URLs by setting the environment variable `SQS_ENDPOINT_STRATEGY` when starting LocalStack to one of the following values. | Value | URL format | Description | |------------|----------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `standard` | `sqs..localhost.localstack.cloud:4566//` | Default. This strategy resembles AWS the closest (see [Identifiers for Amazon SQS](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-queue-message-identifiers.html#sqs-general-identifiers)) and comes with full multi-account and multi-region support. | | `domain` | `.queue.localhost.localstack.cloud:4566//` | This strategy behaves like the [SQS legacy service endpoints](https://docs.aws.amazon.com/general/latest/gr/sqs-service.html#sqs_region), and uses `localhost.localstack.cloud` to resolve to localhost. While using the `us-east-1` region, the `.` prefix is omitted. | | `path` | `localhost:4566/queue///` | An alternative that can be useful if you cannot resolve LocalStack's `localhost` domain. | | `dynamic` | either of the above, using the hostname used from the request | Based on the format of the hostname used by the client to call localstack, the URL will be constructed accordingly. The URL will also use the hostname specified in the request to make sure the client will be able to reach the URl. | | `off` | `localhost:4566//` | It is the current default for maintaining backward compatibility. However, this format does not encode the region information. As a result, you will encounter limitations when querying queues with the same name that exist in different regions. | ### Enabling `PurgeQueue` errors In AWS, there is a restriction that allows only one call to the `PurgeQueue` operation every 60 seconds. You can refer to the [`PurgeQueue` API Reference](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_PurgeQueue.html) for more details. By default, LocalStack disables this behavior. However, if you want to enable the retry delay for `PurgeQueue` in LocalStack, you can start it with the `SQS_DELAY_PURGE_RETRY=1` environment variable. ### Enabling `QueueDeletedRecently` errors In AWS, there is a restriction that prevents the creation of a queue with the same name within 60 seconds after it has been deleted. You can find more information about this behavior in the [`DeleteQueue` API Reference](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_DeleteQueue.html). By default, LocalStack disables this behavior. However, if you want to enable the delay for creating a recently deleted queue in LocalStack, you can start it with the `SQS_DELAY_RECENTLY_DELETED=1` environment variable. ### Enabling `MessageRetentionPeriod` In AWS, you can set the `MessageRetentionPeriod` to control the length of time, in seconds, for which Amazon SQS retains a message. You can find more details in the [`SetQueueAttributes` API reference](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SetQueueAttributes.html#API_SetQueueAttributes_RequestParameters). You can enable this behavior in LocalStack by setting the `SQS_ENABLE_MESSAGE_RETENTION_PERIOD=1` environment variable. In AWS, valid values for message retention range from 60 (1 minute) to 1,209,600 (14 days). In LocalStack, we do not put constraints on the value which can be helpful for test scenarios. :::note Note that, if you enable this option, [persistence](/aws/developer-tools/snapshots/persistence) or [cloud pods](/aws/developer-tools/snapshots/cloud-pods) for SQS may not work as expected. The reason is that, LocalStack does not adjust timestamps when restoring a state, so time appears to pass between LocalStack runs. Consequently, when you restart LocalStack after a period that is longer than the message retention period, LocalStack will remove all those messages when SQS starts. ::: ### Disable CloudWatch Metrics Reporting When working with SQS messages, actions like sending, receiving, and deleting them will automatically trigger CloudWatch metrics. This feature, known as [CloudWatch metrics for Amazon SQS](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-available-cloudwatch-metrics.html), is enabled by default but can be deactivated if needed. Disabling CloudWatch metrics can enhance the performance of SQS message operations. However, it's important to note that deactivation will also disable any integration with CloudWatch, including the triggering of alarms based on metrics. By default, metrics related to `Approximate*` messages are sent to CloudWatch once every minute. You can customize the reporting interval (in seconds) by setting the `SQS_CLOUDWATCH_METRICS_REPORT_INTERVAL` variable to the desired value, such as `SQS_CLOUDWATCH_METRICS_REPORT_INTERVAL=120`. If you wish to disable all CloudWatch metrics for SQS, including the `Approximate*` metrics, you can set the `SQS_DISABLE_CLOUDWATCH_METRICS` variable to `1`. ## Accessing queues from Lambdas or other containers Using the SQS Query API, Queue URLs act as accessible endpoints via HTTP. Several SDKs, such as the Java SDK, leverage the SQS Query API for SQS interaction. By default, Queue URLs are configured to point to `http://localhost:4566`. This configuration can pose problems when Lambdas or other containers attempt to make direct calls to these queue URLs. These issues arise due to the fact that a Lambda function operates within a separate Docker container, and LocalStack is not accessible at the `localhost` address within that container. For instance, users of the Java SDK often encounter the following error when trying to access an SQS queue from their Lambda functions: ```bash 2023-07-28 15:04:00 Unable to execute HTTP request: Connect to localhost:4566 [localhost/127.0.0.1] failed: Connection refused (Connection refused): com.amazonaws.SdkClientException 2023-07-28 15:04:00 com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to localhost:4566 [localhost/127.0.0.1] failed: Connection refused (Connection refused) ... ``` To address this issue, you can consider the steps documented below. ### Lambda When utilizing the SQS Query API in Lambdas, we suggest configuring `SQS_ENDPOINT_STRATEGY=domain`. This configuration results in queue URLs using `*.queue.localhost.localstack.cloud` as their domain names. Our Lambda implementation automatically resolves these URLs to the LocalStack container, ensuring smooth interaction between your code and the SQS service. ### Other containers When your code run within different containers like ECS tasks or your custom ones, it's advisable to establish your Docker network setup. You can follow these steps: 1. Override the `LOCALSTACK_HOST` variable as outlined in our [network troubleshooting guide](/aws/customization/networking/). 2. Ensure that your containers can resolve `LOCALSTACK_HOST` to the LocalStack container within the Docker network. 3. We recommend employing `SQS_ENDPOINT_STRATEGY=path`, which generates queue URLs in the format `http:///queue/...`. ## Developer endpoints LocalStack's SQS implementation offers additional endpoints for developers located at `/_aws/sqs`. These endpoints provide the ability to inspect queues without causing any side effects. This can be particularly useful when you need to examine the content of queues without executing a `ReceiveMessage` operation, which would normally remove messages from the queue. ### Peeking into queues The `/_aws/sqs/messages` endpoint provides access to all messages within a queue without triggering the visibility timeout or modifying access metrics. This endpoint is particularly useful in scenarios such as tests, where you need to wait until a specific message arrives in the queue. The `/_aws/sqs/messages` endpoint is fully compatible with the `ReceiveMessage` operation from the SQS API. By default, it returns all messages in the queue along with their attributes and system attributes. The endpoint ignores any additional parameters from the `ReceiveMessage` operation, except for the `QueueUrl`. You can call the `/_aws/sqs/messages` endpoint in two different ways: 1. Using the query argument `QueueUrl`, like this: ```bash http://localhost.localstack.cloud:4566/_aws/sqs/messages?QueueUrl=http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/my-queue ``` 2. Utilizing the path-based endpoint, as shown in this example: ```bash http://localhost.localstack.cloud:4566/_aws/sqs/messages/us-east-1/000000000000/my-queue ``` #### XML response You can directly call the endpoint to obtain the raw AWS XML response. import { Tabs, TabItem } from '@astrojs/starlight/components'; ```bash curl "http://localhost.localstack.cloud:4566/_aws/sqs/messages?QueueUrl=http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/my-queue" ``` ```python showLineNumbers import requests response = requests.get( url="http://localhost.localstack.cloud:4566/_aws/sqs/messages", params={"QueueUrl": "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/my-queue"}, ) print(response.text) # outputs the response XML ``` An example response is shown below: ```xml title="Output" 6a736e5d-4997-4895-8c96-b65a2d7dd600 5d41402abc4b2a76b9719d911017c592 hello SenderId 000000000000 SentTimestamp 1672853965675 ApproximateReceiveCount 0 ApproximateFirstReceiveTimestamp 1672855121076 SQS/BACKDOOR/ACCESS 173c5aee-503a-4249-90be-159e0d427b48 7d793037a0760186574b0282f2f435e7 world SenderId 000000000000 SentTimestamp 1672853968176 ApproximateReceiveCount 0 ApproximateFirstReceiveTimestamp 1672855121076 SQS/BACKDOOR/ACCESS KR3H1IN3JQ4LO1592IMGK2JLH8HW3J0Y4LRY1TVW2SAFGZFVXJGI ``` #### JSON response You can include the `Accept: application/json` header in your request if you prefer a JSON response. ```bash curl -H "Accept: application/json" \ "http://localhost.localstack.cloud:4566/_aws/sqs/messages?QueueUrl=http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/my-queue" ``` ```python import requests response = requests.get( url="http://localhost.localstack.cloud:4566/_aws/sqs/messages", params={"QueueUrl": "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/my-queue"}, ) print(response.text) # outputs the response XML ``` An example response is shown below: ```bash title="Output" { "ReceiveMessageResponse": { "ReceiveMessageResult": { "Message": [ { "MessageId": "6a736e5d-4997-4895-8c96-b65a2d7dd600", "MD5OfBody": "5d41402abc4b2a76b9719d911017c592", "Body": "hello", "Attribute": [ { "Name": "SenderId", "Value": "000000000000" }, { "Name": "SentTimestamp", "Value": "1672853965675" }, { "Name": "ApproximateReceiveCount", "Value": "0" }, { "Name": "ApproximateFirstReceiveTimestamp", "Value": "1672855535794" } ], "ReceiptHandle": "SQS/BACKDOOR/ACCESS" }, { "MessageId": "173c5aee-503a-4249-90be-159e0d427b48", "MD5OfBody": "7d793037a0760186574b0282f2f435e7", "Body": "world", "Attribute": [ { "Name": "SenderId", "Value": "000000000000" }, { "Name": "SentTimestamp", "Value": "1672853968176" }, { "Name": "ApproximateReceiveCount", "Value": "0" }, { "Name": "ApproximateFirstReceiveTimestamp", "Value": "1672855535794" } ], "ReceiptHandle": "SQS/BACKDOOR/ACCESS" } ] }, "ResponseMetadata": { "RequestId": "TF87187MUBXJHA39J4Y6OVQG57J51OEEMX62UWYBUQJKC8YVID3P" } } } ``` #### AWS Client Since the `/_aws/sqs/messages` endpoint is compatible with the SQS `ReceiveMessage` operation, you can use the endpoint as the endpoint URL parameter in your AWS client call. ```bash aws --endpoint-url=http://localhost.localstack.cloud:4566/_aws/sqs/messages sqs receive-message \ --queue-url=http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/my-queue ``` ```python showLineNumbers import boto3 sqs = boto3.client("sqs", endpoint_url="http://localhost.localstack.cloud:4566/_aws/sqs/messages") response = sqs.receive_message(QueueUrl="http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/my-queue") print(response) ``` An example response is shown below: ```bash title="Output" { "Messages": [ { "MessageId": "6a736e5d-4997-4895-8c96-b65a2d7dd600", "ReceiptHandle": "SQS/BACKDOOR/ACCESS", "MD5OfBody": "5d41402abc4b2a76b9719d911017c592", "Body": "hello", "Attributes": { "SenderId": "000000000000", "SentTimestamp": "1672853965675", "ApproximateReceiveCount": "0", "ApproximateFirstReceiveTimestamp": "1672854900237" } }, { "MessageId": "173c5aee-503a-4249-90be-159e0d427b48", "ReceiptHandle": "SQS/BACKDOOR/ACCESS", "MD5OfBody": "7d793037a0760186574b0282f2f435e7", "Body": "world", "Attributes": { "SenderId": "000000000000", "SentTimestamp": "1672853968176", "ApproximateReceiveCount": "0", "ApproximateFirstReceiveTimestamp": "1672854900237" } } ] } ``` #### Show invisible or delayed messages The developer endpoint also supports showing invisible and delayed messages via the query arguments `ShowInvisible` and `ShowDelayed`. ```bash curl -H "Accept: application/json" \ "http://localhost.localstack.cloud:4566/_aws/sqs/messages?ShowInvisible=true&ShowDelayed=true&QueueUrl=http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/my-queue ``` ```python showLineNumbers import requests response = requests.get( "http://localhost.localstack.cloud:4566/_aws/sqs/messages", params={"QueueUrl": queue_url, "ShowInvisible": True, "ShowDelayed": True}, headers={"Accept": "application/json"}, ) print(response.text) ``` This will also include messages that currently have an active visibility timeout or were delayed and are not actually in the queue yet. Here's an example: ```bash title="Output" [ { "MessageId": "1c4187cc-f2c9-4f1c-9702-4a3bfaaa4817", "MD5OfBody": "a06498de7fb4bd539c8895748f03175d", "Body": "message-3", "Attribute": [ {"Name": "SenderId", "Value": "000000000000"}, {"Name": "SentTimestamp", "Value": "1697494407799"}, {"Name": "ApproximateReceiveCount", "Value": "0"}, {"Name": "ApproximateFirstReceiveTimestamp", "Value": "0"}, {"Name": "IsVisible", "Value": "true"}, <-- {"Name": "IsDelayed", "Value": "false"}, <-- ], "ReceiptHandle": "SQS/BACKDOOR/ACCESS", }, ... ] ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing SQS queues. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **SQS** under the **App Integration** section. ![SQS Resource Browser](/images/aws/sqs-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Queue**: Create a new SQS queue by specifying a queue name, optional attributes, and tags. - **Send Message**: Send a message to an SQS queue by specifying the queue name, message body, delay seconds, optional message attributes, and more. - **View Details and Messages**: View details and messages of an SQS queue by selecting the queue name and navigating to the **Details** and **Messages** tabs. - **Delete Queue**: Delete an SQS queue by selecting the queue name and clicking the **Action** button, followed by **Remove Selected**. ## Examples The following code snippets and sample applications provide practical examples of how to use SQS in LocalStack for various use cases: - [Serverless microservices with Amazon API Gateway, DynamoDB, SQS, and Lambda](https://github.com/localstack/microservices-apigateway-lambda-dynamodb-sqs-sample) - [Loan Broker application with AWS Step Functions, DynamoDB, Lambda, SQS, and SNS](https://github.com/localstack/loan-broker-stepfunctions-lambda-app) - [Messaging Processing application with SQS, DynamoDB, and Fargate](https://github.com/localstack/sqs-fargate-ddb-cdk-go) - [Serverless Transcription application using Transcribe, S3, Lambda, SQS, and SES](https://github.com/localstack/sample-transcribe-app) ## Current Limitations - Updating a queue's `MessageRetentionPeriod` currently has no effect on existing messages ## API Coverage # Systems Manager (SSM) > Get started with Systems Manager (SSM) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Systems Manager (SSM) is a management service provided by Amazon Web Services that helps you effectively manage and control your infrastructure resources. SSM simplifies tasks related to system and application management, patching, configuration, and automation, allowing you to maintain the health and compliance of your environment. LocalStack allows you to use the SSM APIs in your local environment to run operational tasks on the Dockerized instances. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of SSM's integration with LocalStack. ## Getting started This guide is designed for users new to Systems Manager (SSM) and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method with an additional `EC2_VM_MANAGER=docker` configuration variable. We will demonstrate how to use EC2 and SSM functionalities when using the Docker backend with LocalStack with the AWS CLI. ### Create an EC2 instance To get started, pull the `ubuntu:focal` image from Docker Hub and tag it as `localstack-ec2/ubuntu-focal-docker-ami:ami-00a001`. LocalStack uses a naming scheme to recognise and manage the containers and images associated with it. The container are named `localstack-ec2.`, while images are tagged `localstack-ec2/:`. ```bash docker pull ubuntu:focal docker tag ubuntu:focal localstack-ec2/ubuntu-focal-docker-ami:ami-00a001 ``` LocalStack's Docker backend treats Docker images with the above naming scheme as AMIs. The AMI ID is the last part of the image tag, `ami-00a001` in this case. You can run an EC2 instance using the [`RunInstances`](https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_RunInstances.html) API. Execute the following command to create an EC2 instance using the `ami-00a001` AMI. ```bash lstk aws ec2 run-instances \ --image-id ami-00a001 --count 1 ``` ```bash title="Output" { ... "Instances": [ { ... "InstanceId": "i-abf6920789a06dd84", "InstanceType": "m1.small", ... "SecurityGroups": [], "SourceDestCheck": true, "Tags": [], "VirtualizationType": "paravirtual" } ], "OwnerId": "000000000000", "ReservationId": "r-e9b21a68" ... ``` You can copy the `InstanceId` value and use it in the following commands. ### Send command using SSM You can use the [`SendCommand`](https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_SendCommand.html) API to send a command to the EC2 instance. The following command sends a `cat lsb-release` command in the `/etc` directory to the EC2 instance. ```bash lstk aws ssm send-command --document-name "AWS-RunShellScript" \ --document-version "1" \ --instance-ids i-abf6920789a06dd84 \ --parameters "commands='cat lsb-release',workingDirectory=/etc" ``` ```bash title="Output" { "Command": { "CommandId": "23547a9b-6993-4967-9446-f96b9b5dac70", "DocumentName": "AWS-RunShellScript", "DocumentVersion": "1", "InstanceIds": [ "i-abf6920789a06dd84" ], "Status": "InProgress" } } ``` You can copy the `CommandId` value and use it in the following commands. ### Retrieve the command output You can use the [`GetCommandInvocation`](https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_GetCommandInvocation.html) API to retrieve the command output. The following command retrieves the output of the command sent in the previous step. ```bash lstk aws ssm get-command-invocation \ --command-id 23547a9b-6993-4967-9446-f96b9b5dac70 \ --instance-id i-abf6920789a06dd84 ``` Change the `CommandId` and `InstanceId` values to the ones you received in the previous step. ```bash title="Output" { "CommandId": "23547a9b-6993-4967-9446-f96b9b5dac70", "InstanceId": "i-abf6920789a06dd84", "DocumentName": "AWS-RunShellScript", "DocumentVersion": "1", "Status": "Success", "StandardOutputContent": "DISTRIB_ID=Ubuntu\nDISTRIB_RELEASE=20.04\nDISTRIB_CODENAME=focal\nDISTRIB_DESCRIPTION=\"Ubuntu 20.04.6 LTS\"\n", "StandardErrorContent": "" } ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing SSM System Parameters. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resource Browser** section, and then clicking on **Simple Systems Manager (SSM)** under the **Management/Governance** section. ![SSM Resource Browser](/images/aws/ssm-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create System Parameter**: Create a new System Parameter by clicking on the **Create Parameter** button and providing the required details. - **View the System Parameter**: View the details of a System Parameter, such as its value, by clicking on the parameter name. - **Delete the System Parameter**: Delete a System Parameter by selecting the parameter and clicking on the **Actions** dropdown menu followed by **Remove Selected**. ## Current Limitations The following table highlights some differences between LocalStack SSM and AWS SSM. | LocalStack | AWS | | ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | | Automated SSM registration for instances | Manual instance registration using [`CreateActivation`](https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_CreateActivation.html) | | Operations performed through Docker exec | Operations facilitated by [Amazon SSM Agent](https://github.com/aws/amazon-ssm-agent) | | Instance IDs prefixed with `i-` | Instance IDs prefixed with `mi-` | The other limitations of LocalStack SSM are: - **Document Support**: LocalStack supports both AWS-managed documents (such as `AWS-RunShellScript`) and custom documents with [`SendCommand`](https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_SendCommand.html) API. - Parameter substitution in documents using the `{{ parameter-name }}` syntax is supported. - Only documents using the `aws:runShellScript` plugin are fully supported for execution in Dockerized instances. - Documents using other plugins will fall back to the Moto implementation, which may not function correctly. - Commands returning non-zero codes won't capture standard output or error streams, leaving them empty. - Shell constructs such as job controls (`&&`, `||`), and redirection (`>`) are not supported. ## API Coverage # SSO Admin > Get started with SSO Admin on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction SSO Admin is a service provided by Amazon Web Services (AWS) that enables you to manage your AWS Single Sign-On (AWS SSO) resources. It allows you to create, update, and delete AWS SSO resources such as directories, groups, and users. LocalStack provides a mock implementation of the SSO Admin API that allows you to create and manage your AWS SSO resources. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of SSO Admin's integration with LocalStack. ## Getting started This guide is designed for users new to SSO Admin and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a permission set, add tags to a permission set, and list permission sets. ### Create a permission set You can create a permission set using the [`CreatePermissionSet`](https://docs.aws.amazon.com/sso-admin/latest/APIReference/API_CreatePermissionSet.html) API. ```bash lstk aws sso-admin create-permission-set \ --name my-permission-set \ --description "My permission set" \ --instance-arn arn:aws:sso:::instance/d-1234567890 \ --tags Key=Name,Value=my-permission-set ``` ```bash title="Output" { "PermissionSet": { "CreatedDate": "2025-07-02T12:15:33.352631+05:30", "Description": "My permission set", "Name": "my-permission-set", "PermissionSetArn": "arn:aws:sso:::instance/d-1234567890/ps-lm0rshcjz3tikab8", "SessionDuration": 3600 } } ``` ### List permission sets You can list permission sets using the [`ListPermissionSets`](https://docs.aws.amazon.com/sso-admin/latest/APIReference/API_ListPermissionSets.html) API. ```bash lstk aws sso-admin list-permission-sets --instance-arn arn:aws:sso:::instance/d-1234567890 ``` ```bash title="Output" { "PermissionSets": [ "arn:aws:sso:::instance/d-1234567890/ps-lm0rshcjz3tikab8" ] } ``` ### List tags for a permission set You can list tags for a permission set using the [`ListTagsForResource`](https://docs.aws.amazon.com/sso-admin/latest/APIReference/API_ListTagsForResource.html) API. ```bash lstk aws sso-admin list-tags-for-resource --resource-arn arn:aws:sso:::instance/d-1234567890/ps-lm0rshcjz3tikab8 --instance-arn arn:aws:sso:::instance/d-1234567890 ``` ```bash title="Output" { "Tags": [ { "Key": "Name", "Value": "my-permission-set" } ] } ``` ## API Coverage # Step Functions > Get started with Step Functions on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Step Functions is a serverless workflow engine that enables the orchestrating of multiple AWS services. It provides a JSON-based structured language called Amazon States Language (ASL) which allows to specify how to manage a sequence of tasks and actions that compose the application's workflow. Thus making it easier to build and maintain complex and distributed applications. LocalStack allows you to use the Step Functions APIs in your local environment to create, execute, update, and delete state machines locally. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Step Function's integration with LocalStack. ## Getting started This guide is designed for users new to Step Functions and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how you can create a state machine, execute it, and check the status of the execution. ### Create a state machine You can create a state machine using the [`CreateStateMachine`](https://docs.aws.amazon.com/step-functions/latest/apireference/API_CreateStateMachine.html) API. The API requires the name of the state machine, the state machine definition, and the role ARN that the state machine will assume to call AWS services. Run the following command to create a state machine: ```bash showLineNumbers lstk aws stepfunctions create-state-machine \ --name "CreateAndListBuckets" \ --definition '{ "Comment": "Create bucket and list buckets", "StartAt": "CreateBucket", "States": { "CreateBucket": { "Type": "Task", "Resource": "arn:aws:states:::aws-sdk:s3:createBucket", "Parameters": { "Bucket": "new-sfn-bucket" }, "Next": "ListBuckets" }, "ListBuckets": { "Type": "Task", "Resource": "arn:aws:states:::aws-sdk:s3:listBuckets", "End": true } } }' \ --role-arn "arn:aws:iam::000000000000:role/stepfunctions-role" ``` ```bash title="Output" { "stateMachineArn": "arn:aws:states:us-east-1:000000000000:stateMachine:CreateAndListBuckets", "creationDate": 1714643996.18017 } ``` ### Execute the state machine You can execute the state machine using the [`StartExecution`](https://docs.aws.amazon.com/step-functions/latest/apireference/API_StartExecution.html) API. The API requires the state machine's ARN and the state machine's input. Run the following command to execute the state machine: ```bash lstk aws stepfunctions start-execution \ --state-machine-arn "arn:aws:states:us-east-1:000000000000:stateMachine:CreateAndListBuckets" ``` ```bash title="Output" { "executionArn": "arn:aws:states:us-east-1:000000000000:execution:CreateAndListBuckets:bf7d2138-e96f-42d1-b1f9-41f0c1c7bc3e", "startDate": 1714644089.748442 } ``` ### Check the execution status To check the status of the execution, you can use the [`DescribeExecution`](https://docs.aws.amazon.com/step-functions/latest/apireference/API_DescribeExecution.html) API. Run the following command to describe the execution: ```bash lstk aws stepfunctions describe-execution \ --execution-arn "arn:aws:states:us-east-1:000000000000:execution:CreateAndListBuckets:bf7d2138-e96f-42d1-b1f9-41f0c1c7bc3e" ``` Replace the `execution-arn` with the ARN of the execution you want to describe. ```bash title="Output" { "executionArn": "arn:aws:states:us-east-1:000000000000:execution:CreateAndListBuckets:bf7d2138-e96f-42d1-b1f9-41f0c1c7bc3e", "stateMachineArn": "arn:aws:states:us-east-1:000000000000:stateMachine:CreateAndListBuckets", "name": "bf7d2138-e96f-42d1-b1f9-41f0c1c7bc3e", "status": "SUCCEEDED", "startDate": 1714644089.748442, "stopDate": 1714644089.907964, "input": "{}", "inputDetails": { "included": true }, "output": "{\"Buckets\":[{\"Name\":\"cdk-hnb659fds-assets-000000000000-us-east-1\",\"CreationDate\":\"2024-05-02T09:53:54+00:00\"},{\"Name\":\"new-sfn-bucket\",\"CreationDate\":\"2024-05-02T10:01:29+00:00\"}],\"Owner\":{\"DisplayName\":\"webfile\",\"Id\":\"75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a\"}}", "outputDetails": { "included": true } } ``` ## Supported services and operations Step Functions integrates with AWS services, allowing you to invoke API actions for each service within your workflow. LocalStack's Step Functions emulation supports the following AWS services: | Supported service integrations | Service | Request Response | Run a Job (.sync) | Run a Job (.sync2) | Wait for Callback (.waitForTaskToken) | |--------------------------------|-------------------------|:---: |:---: |:---: |:---: | | Optimized integrations | Lambda | ✓ | | | ✓ | | | DynamoDB | ✓ | | | | | | Amazon ECS/AWS Fargate | ✓ | ✓ | | ✓ | | | Amazon SNS  | ✓ | | | ✓ | | | Amazon SQS | ✓ | | | ✓ | | | API Gateway | ✓ | | | ✓ | | | Amazon EventBridge | ✓ | | | ✓ | | | AWS Glue | ✓ | ✓ | | | | | AWS Step Functions | ✓ | ✓ | ✓ | ✓ | | | AWS Batch | ✓ | ✓ | | | | AWS SDK integrations | All LocalStack services | ✓ | | | ✓ | ## Mocked Service Integrations Mocked service integrations let you test AWS Step Functions without invoking LocalStack’s emulated AWS services. Instead, Task states return predefined outputs from a mock configuration file. The key components are: - **Mocked service integrations**: Task states that return predefined responses instead of calling local AWS services. - **Mocked responses**: Static payloads linked to mocked Task states. - **Test cases**: Executions of your state machine that use mocked responses. - **Mock configuration file**: A JSON file that defines test cases, mocked states, and their response payloads. During execution, each Task state listed in the mock file returns its associated mocked response. States not included in the file continue to invoke the corresponding emulated services, allowing a mix of mocked and real interactions. You can define one or more mocked payloads per Task state. Supported integration patterns include `.sync`, `.sync2`, and `.waitForTaskToken`. Both success and failure scenarios can be simulated. ### Compatibility with AWS Step Functions Local LocalStack can also serve as a drop-in replacement for [AWS Step Functions Local testing with mocked service integrations](https://docs.aws.amazon.com/step-functions/latest/dg/sfn-local-test-sm-exec.html). It supports test cases with mocked Task states and maintains compatibility with existing Step Functions Local configurations. This functionality is extended in LocalStack by providing access to the latest Step Functions features such as [JSONata and Variables](https://blog.localstack.cloud/aws-step-functions-made-easy/), as well as the ability to enable both mocked and emulated service interactions emulated by LocalStack. :::note LocalStack does not validate response formats. Ensure the payload structure in the mocked responses matches what the real service expects. ::: ### Identify a State Machine for Mocked Integrations Mocked service integrations apply to specific state machine definitions. The first step is to select the state machine where mocked responses should be applied. In this example, we'll use a state machine named `LambdaSQSIntegration`, defined as follows: ```json title="LambdaSQSIntegration.json" showLineNumbers { "Comment": "This state machine is called: LambdaSQSIntegration", "QueryLanguage": "JSONata", "StartAt": "LambdaState", "States": { "LambdaState": { "Type": "Task", "Resource": "arn:aws:states:::lambda:invoke", "Arguments": { "FunctionName": "GreetingsFunction", "Payload": { "fullname": "{% $states.input.name & ' ' & $states.input.surname %}" } }, "Retry": [ { "ErrorEquals": [ "States.ALL" ], "IntervalSeconds": 2, "MaxAttempts": 4, "BackoffRate": 2 } ], "Assign": { "greeting": "{% $states.result.Payload.greeting %}" }, "Next": "SQSState" }, "SQSState": { "Type": "Task", "Resource": "arn:aws:states:::sqs:sendMessage", "Arguments": { "QueueUrl": "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/localstack-queue", "MessageBody": "{% $greeting %}" }, "End": true } } } ``` ## Define Mock Integrations in a Configuration File Mock integrations are defined in a JSON file that follows the `RawMockConfig` schema. This file contains two top-level sections: - **StateMachines** – Maps each state machine to its test cases, specifying which states use which mocked responses. - **MockedResponses** – Defines reusable mock payloads, each identified by a `ResponseID`, which test cases can reference. #### `StateMachines` This section specifies the Step Functions state machines to mock, along with their corresponding test cases. Each test case maps state names to `ResponseID`s defined in the `MockedResponses` section. ```json showLineNumbers "StateMachines": { "": { "TestCases": { "": { "": "", ... } } } } ``` In the example above: - **`StateMachineName`**: Must exactly match the name used when the state machine was created in LocalStack. - **`TestCases`**: Named scenarios that define mocked behavior for specific `Task` states. Each test case maps `Task` states to mock responses that define their expected behavior. At runtime, if a test case is selected, the state uses the mocked response (if defined); otherwise, it falls back to calling the emulated service. Below is a complete example of the `StateMachines` section: ```json showLineNumbers "LambdaSQSIntegration": { "TestCases": { "LambdaRetryCase": { "LambdaState": "MockedLambdaStateRetry", "SQSState": "MockedSQSStateSuccess" } } } ``` #### `MockedResponses` This section defines mocked responses for Task states. Each `ResponseID` includes one or more step keys and defines either a `Return` value or a `Throw` error. ```json showLineNumbers "MockedResponses": { "": { "": { "Return": ... }, "": { "Throw": ... } } } ``` In the example above: - `ResponseID`: A unique identifier used in test cases to reference a specific mock response. - `step-key`: Indicates the attempt number. For example, `"0"` refers to the first try, while `"1-2"` covers a range of attempts. - `Return`: Simulates a successful response by returning a predefined payload. - `Throw`: Simulates a failure by returning an `Error` and an optional `Cause`. :::note Each entry must have **either** `Return` or `Throw`, but cannot have both. ::: Here is a complete example of the `MockedResponses` section: ```json showLineNumbers "MockedLambdaStateRetry": { "0": { "Throw": { "Error": "Lambda.ServiceException", "Cause": "An internal service error occurred." } }, "1-2": { "Throw": { "Error": "Lambda.TooManyRequestsException", "Cause": "Invocation rate limit exceeded." } }, "3": { "Return": { "StatusCode": 200, "Payload": { "greeting": "Hello John Smith, you’re now testing mocked integrations with LocalStack!" } } } } ``` The `MockConfigFile.json` below is used to test the `LambdaSQSIntegration` state machine defined earlier. ```json showLineNumbers { "StateMachines":{ "LambdaSQSIntegration":{ "TestCases":{ "BaseCase":{ "LambdaState":"MockedLambdaStateSuccess", "SQSState":"MockedSQSStateSuccess" }, "LambdaRetryCase":{ "LambdaState":"MockedLambdaStateRetry", "SQSState":"MockedSQSStateSuccess" }, "HybridCase":{ "LambdaState":"MockedLambdaSuccess" } } } }, "MockedResponses":{ "MockedLambdaStateSuccess":{ "0":{ "Return":{ "StatusCode":200, "Payload":{ "greeting":"Hello John Smith, you’re now testing mocked integrations with LocalStack!" } } } }, "MockedSQSStateSuccess":{ "0":{ "Return":{ "MD5OfMessageBody":"3661896f-1287-45a3-8f89-53bd7b25a9a6", "MessageId":"7c9ef661-c455-4779-a9c2-278531e231c2" } } }, "MockedLambdaStateRetry":{ "0":{ "Throw":{ "Error":"Lambda.ServiceException", "Cause":"An internal service error occurred." } }, "1-2":{ "Throw":{ "Error":"Lambda.TooManyRequestsException", "Cause":"Invocation rate limit exceeded." } }, "3":{ "Return":{ "StatusCode":200, "Payload":{ "greeting":"Hello John Smith, you’re now testing mocked integrations with LocalStack!" } } } } } } ``` ### Provide the Mock Configuration to LocalStack Set the `SFN_MOCK_CONFIG` environment variable to the path of your mock configuration file. If you're running LocalStack in Docker, mount the file and pass the variable as shown below: import { Tabs, TabItem } from '@astrojs/starlight/components'; Add the mount to your `config.toml`: ```toml [[containers]] type = "aws" volumes = ["/path/to/MockConfigFile.json:/tmp/MockConfigFile.json"] ``` Then set the environment variable and start LocalStack: ```bash LOCALSTACK_SFN_MOCK_CONFIG=/tmp/MockConfigFile.json lstk start ``` ```yaml showLineNumbers services: localstack: container_name: "${LOCALSTACK_DOCKER_NAME:-localstack-main}" image: localstack/localstack ports: - "127.0.0.1:4566:4566" # LocalStack Gateway - "127.0.0.1:4510-4559:4510-4559" # external services port range environment: # LocalStack configuration: https://docs.localstack.cloud/references/configuration/ - DEBUG=${DEBUG:-0} - SFN_MOCK_CONFIG=/tmp/MockConfigFile.json volumes: - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" - "./MockConfigFile.json:/tmp/MockConfigFile.json" ``` ### Run Test Cases with Mocked Integrations Create the state machine to match the name defined in the mock configuration file. In this example, create the `LambdaSQSIntegration` state machine using: ```bash lstk aws stepfunctions create-state-machine \ --definition file://LambdaSQSIntegration.json \ --name "LambdaSQSIntegration" \ --role-arn "arn:aws:iam::000000000000:role/service-role/testrole" ``` After the state machine is created and correctly named, you can run test cases defined in the mock configuration file using the [`StartExecution`](https://docs.aws.amazon.com/step-functions/latest/apireference/API_StartExecution.html) API. To execute a test case, append the test case name to the state machine ARN using `#`. This tells LocalStack to apply the corresponding mocked responses from the configuration file. For example, to run the `BaseCase` test case: ```bash lstk aws stepfunctions start-execution \ --state-machine arn:aws:states:us-east-1:000000000000:stateMachine:LambdaSQSIntegration#BaseCase \ --input '{"name": "John", "surname": "smith"}' \ --name "MockExecutionBaseCase" ``` During execution, any state mapped in the mock config will use the predefined response. States without mock entries invoke the actual emulated service as usual. You can inspect the execution using the [`DescribeExecution`](https://docs.aws.amazon.com/step-functions/latest/apireference/API_DescribeExecution.html) API: ```bash lstk aws stepfunctions describe-execution \ --execution-arn "arn:aws:states:us-east-1:000000000000:execution:LambdaSQSIntegration:MockExecutionBaseCase" ``` ```json showLineNumbers { "executionArn": "arn:aws:states:us-east-1:000000000000:execution:LambdaSQSIntegration:MockExecutionBaseCase", "stateMachineArn": "arn:aws:states:us-east-1:000000000000:stateMachine:LambdaSQSIntegration", "name": "MockExecutionBaseCase", "status": "SUCCEEDED", "startDate": "...", "stopDate": "...", "input": "{\"name\":\"John\",\"surname\":\"smith\"}", "inputDetails": { "included": true }, "output": "{\"MessageId\":\"7c9ef661-c455-4779-a9c2-278531e231c2\",\"MD5OfMessageBody\":\"3661896f-1287-45a3-8f89-53bd7b25a9a6\"}", "outputDetails": { "included": true } } ``` You can also use the [`GetExecutionHistory`](https://docs.aws.amazon.com/step-functions/latest/apireference/API_GetExecutionHistory.html) API to retrieve the execution history, including the events and their details. ```bash lstk aws stepfunctions get-execution-history \ --execution-arn "arn:aws:states:us-east-1:000000000000:execution:LambdaSQSIntegration:MockExecutionBaseCase" ``` This will return the full execution history, including entries that indicate how mocked responses were applied to Lambda and SQS states. ```bash title="Output" ... { "timestamp": "...", "type": "TaskSucceeded", "id": 5, "previousEventId": 4, "taskSucceededEventDetails": { "resourceType": "lambda", "resource": "invoke", "output": "{\"StatusCode\": 200, \"Payload\": {\"greeting\": \"Hello John Smith, you\\u2019re now testing mocked integrations with LocalStack!\"}}", "outputDetails": { "truncated": false } } } ... { "timestamp": "...", "type": "TaskSucceeded", "id": 10, "previousEventId": 9, "taskSucceededEventDetails": { "resourceType": "sqs", "resource": "sendMessage", "output": "{\"MessageId\": \"7c9ef661-c455-4779-a9c2-278531e231c2\", \"MD5OfMessageBody\": \"3661896f-1287-45a3-8f89-53bd7b25a9a6\"}", "outputDetails": { "truncated": false } } } ... ``` ## Resource Browser The LocalStack Web Application includes a **Resource Browser** for managing Step Functions state machines. To access it, open the LocalStack Web UI in your browser, navigate to the **Resource Browser** section, and click **Step Functions** under **App Integration**. ![Step Functions Resource Browser](/images/aws/stepfunctions-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create state machine**: Create a new state machine by clicking on the **Create state machine** button and providing the required information. - **View state machine details**: Click on a state machine to view its details, including the state executions, definition details, such as the schema and flowchart, and the state machine's ARN. - **Start execution**: Start a new execution of the state machine by clicking on the **Start Execution** button and providing the input data. - **Delete state machine**: Delete a state machine by selecting it and clicking on the **Actions** button followed by **Remove Selected** button. ## Examples The following code snippets and sample applications provide practical examples of how to use Step Functions in LocalStack for various use cases: - [Loan Broker application with AWS Step Functions, DynamoDB, Lambda, SQS, and SNS](https://github.com/localstack/loan-broker-stepfunctions-lambda-app) - [Integrating Step Functions with local Lambda functions on LocalStack](https://github.com/localstack/localstack-pro-samples/tree/master/stepfunctions-lambda) ## API Coverage # Security Token Service (STS) > Get started with Security Token Service on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Security Token Service (STS) is a service provided by Amazon Web Services (AWS) that enables you to grant temporary, limited-privilege credentials to users and applications. STS implements fine-grained access control and reduce the exposure of your long-term credentials. The temporary credentials, known as security tokens, can be used to access AWS services and resources based on the permissions specified in the associated policies. LocalStack allows you to use the STS APIs in your local environment to request security tokens, manage permissions, integrate with identity providers, and more. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of STS's integration with LocalStack. ## Getting started This guide is designed for users new to STS and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to assume an IAM Role and assume the role as well as creating an IAM user and getting using the STS with the AWS CLI. ### Create an IAM User and get temporary Credentials You can create an IAM User and Role using the [`CreateUser`](https://docs.aws.amazon.com/STS/latest/APIReference/API_CreateUser.html) API. The IAM User will be used to assume the IAM Role. Run the following command to create an IAM User, named `localstack-user`: ```bash lstk aws iam create-user \ --user-name localstack-user ``` You can generate long-term access keys for the IAM user using the [`CreateAccessKey`](https://docs.aws.amazon.com/STS/latest/APIReference/API_CreateAccessKey.html) API. Run the following command to create an access key for the IAM user: ```bash lstk aws iam create-access-key \ --user-name localstack-user ``` ```bash title="Output" { "AccessKey": { "UserName": "localstack-user", "AccessKeyId": "ACCESS_KEY_ID", "Status": "Active", "SecretAccessKey": "SECRET_ACCESS_KEY", "CreateDate": "2023-08-24T17:16:16Z" } } ``` Using STS, you can also fetch temporary credentials for this user using the [`GetSessionToken`](https://docs.aws.amazon.com/STS/latest/APIReference/API_GetSessionToken.html) API. Run the following command using your long-term credentials to get your temporary credentials: ```bash lstk aws sts get-session-token ``` ```bash title="Output" { "Credentials": { "AccessKeyId": "ACCESS_KEY_ID", "SecretAccessKey": "SECRET_ACCESS_KEY", "SessionToken": "SESSION_TOKEN", "Expiration": "TIMESTAMP" } } ``` ### Create an IAM Role You can now create an IAM Role, named `localstack-role`, using the [`CreateRole`](https://docs.aws.amazon.com/STS/latest/APIReference/API_CreateRole.html) API. Run the following command to create the IAM Role: ```bash lstk aws iam create-role \ --role-name localstack-role \ --assume-role-policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"AWS":"arn:aws:iam::000000000000:root"},"Action":"sts:AssumeRole"}]}' ``` ```bash title="Output" { "Role": { "Path": "/", "RoleName": "localstack-role", "RoleId": "AROAQAAAAAAAEDP262HSR", "Arn": "arn:aws:iam::000000000000:role/localstack-role", "CreateDate": "2023-08-24T17:17:13.632000Z", "AssumeRolePolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::000000000000:root" }, "Action": "sts:AssumeRole" } ] } } } ``` You can attach the policy to the IAM role using the [`AttachRolePolicy`](https://docs.aws.amazon.com/STS/latest/APIReference/API_AttachRolePolicy.html) API. Run the following command to attach the policy to the IAM role: ```bash lstk aws iam attach-role-policy \ --role-name localstack-role \ --policy-arn arn:aws:iam::aws:policy/AdministratorAccess ``` ### Assume an IAM Role You can assume an IAM Role using the [`AssumeRole`](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html) API. Run the following command to assume the IAM Role: ```bash lstk aws sts assume-role \ --role-arn arn:aws:iam::000000000000:role/localstack-role \ --role-session-name localstack-session ``` ```bash title="Output" { "Credentials": { "AccessKeyId": "ACCESS_KEY_ID", "SecretAccessKey": "SECRET_ACCESS_KEY", "SessionToken": "SESSION_TOKEN", "Expiration": "TIMESTAMP", }, "AssumedRoleUser": { "AssumedRoleId": "AROAQAAAAAAAEDP262HSR:localstack-session", "Arn": "arn:aws:sts::000000000000:assumed-role/localstack-role/localstack-session" }, "PackedPolicySize": 6 } ``` You can use the temporary credentials in your applications for temporary access. ### Get caller identity You can get the caller identity to identify the principal your current credentials are valid for using the [`GetCallerIdentity`](https://docs.aws.amazon.com/STS/latest/APIReference/API_GetCallerIdentity.html) API. Run the following command to get the caller identity for the credentials set in your environment: ```bash lstk aws sts get-caller-identity ``` ```bash title="Output" { "UserId": "AKIAIOSFODNN7EXAMPLE", "Account": "000000000000", "Arn": "arn:aws:iam::000000000000:root" } ``` ## API Coverage # Support > Get started with Support on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction AWS Support is a service provided by Amazon Web Services (AWS) that offers technical assistance and resources to help you optimize your AWS environment, troubleshoot issues, and maintain operational efficiency. Support APIs provide programmatic access to AWS Support services, including the ability to create and manage support cases programmatically. You can further automate your support workflow using various AWS services, such as Lambda, CloudWatch, and EventBridge. LocalStack allows you to use the Support APIs in your local environment to create and manage new cases, while testing your configurations locally. LocalStack provides a mock implementation via a mock Support Center provided by [Moto](https://docs.getmoto.org/en/latest/docs/services/support.html), and does not create real cases in the AWS. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Support API's integration with LocalStack. :::note For technical support with LocalStack, you can reach out through our [support channels](/aws/help-support/get-help/). It's important to note that LocalStack doesn't offer a programmatic interface to create support cases, and this documentation is only intended to demonstrate how you can use and mock the AWS Support APIs in your local environment. ::: ## Getting started This guide is designed for users new to Support and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how you can create a case in the mock Support Center using the AWS CLI. ### Create a support case To create a support case, you can use the [`CreateCase`](https://docs.aws.amazon.com/goto/WebAPI/support-2013-04-15/CreateCase) API. The following example creates a case with the subject "Test case" and the description "This is a test case" in the category "General guidance". ```bash lstk aws support create-case \ --subject "Test case" \ --service-code "general-guidance" \ --category-code "general-guidance" \ --communication-body "This is a test case" ``` ```bash title="Output" { "caseId": "case-12345678910-2020-kEa16f90bJE766J4" } ``` ### List support cases To list all support cases, you can use the [`DescribeCases`](https://docs.aws.amazon.com/awssupport/latest/APIReference/API_DescribeCases.html) API. The following example lists all cases in the category "General guidance". ```bash lstk aws support describe-cases ``` ```bash title="Output" { "cases": [ { "caseId": "case-12345678910-2020-kEa16f90bJE766J4", ... "submittedBy": "moto@moto.com", "timeCreated": "2023-08-24T18:03:08.895247", "recentCommunications": { "communications": [ { "caseId": "case-12345678910-2020-kEa16f90bJE766J4", "body": "This is a test case", "submittedBy": "moto@moto.com", ... } ], "nextToken": "foo_next_token" } } ] } ``` ### Resolve a support case To resolve a support case, you can use the [`ResolveCase`](https://docs.aws.amazon.com/goto/WebAPI/support-2013-04-15/ResolveCase) API. The following example resolves the case created in the previous step. ```bash lstk aws support resolve-case \ --case-id "case-12345678910-2020-kEa16f90bJE766J4" ``` Replace the case ID with the ID of the case you want to resolve. ```bash title="Output" { "initialCaseStatus": "resolved", "finalCaseStatus": "resolved" } ``` You can also use the [`DescribeCases`](https://docs.aws.amazon.com/awssupport/latest/APIReference/API_DescribeCases.html) API to verify that the case has been resolved. ## API Coverage # Simple Workflow Service (SWF) > Get started with Simple Workflow Service (SWF) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Simple Workflow Service (SWF) is a fully managed service offered by Amazon Web Services (AWS) that enables you to build and manage applications with distributed components and complex workflows. SWF allows you to define workflows in a way that's separate from the actual application code, making it easier to modify and adapt workflows without changing the application logic. SWF also provides a programming framework to design, coordinate, and execute workflows that involve multiple tasks, steps, and decision points. LocalStack allows you to use the SWF APIs in your local environment to monitor and manage workflow design, task coordination, activity implementation, and error handling. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of SWF's integration with LocalStack. ## Getting started This guide is designed for users new to Simple Workflow Service and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to register an SWF domain and workflow using the AWS CLI. ### Registering a domain You can register an SWF domain using the [`RegisterDomain`](https://docs.aws.amazon.com/amazonswf/latest/apireference/API_RegisterDomain.html) API. Execute the following command to register a domain named `test-domain`: ```bash lstk aws swf register-domain \ --name test-domain \ --workflow-execution-retention-period-in-days 1 ``` You can use the [`DescribeDomain`](https://docs.aws.amazon.com/amazonswf/latest/apireference/API_DescribeDomain.html) API to verify that the domain was registered successfully. Run the following command to describe the `test-domain` domain: ```bash lstk aws swf describe-domain \ --name test-domain ``` ```bash title="Output" { "domainInfo": { "name": "test-domain", "status": "REGISTERED", "arn": "arn:aws:swf:us-east-1:000000000000:/domain/test-domain" }, "configuration": { "workflowExecutionRetentionPeriodInDays": "1" } } ``` ### List the domains You can list all registered domains using the [`ListDomains`](https://docs.aws.amazon.com/amazonswf/latest/apireference/API_ListDomains.html) API. Run the following command to list all registered domains: ```bash lstk aws swf list-domains --registration-status REGISTERED ``` To deprecate a domain, use the [`DeprecateDomain`](https://docs.aws.amazon.com/amazonswf/latest/apireference/API_DeprecateDomain.html) API. Run the following command to deprecate the `test-domain` domain: ```bash lstk aws swf deprecate-domain \ --name test-domain ``` You can now list the deprecated domains using the `--registration-status DEPRECATED` flag: ```bash lstk aws swf list-domains --registration-status DEPRECATED ``` ### Registering a workflow You can register a workflow using the [`RegisterWorkflowType`](https://docs.aws.amazon.com/amazonswf/latest/apireference/API_RegisterWorkflowType.html) API. Execute the following command to register a workflow named `test-workflow`: ```bash showLineNumbers lstk aws swf register-workflow-type \ --domain test-domain \ --name test-workflow \ --default-task-list name=test-task-list \ --default-task-start-to-close-timeout 30 \ --default-execution-start-to-close-timeout 60 \ --default-child-policy TERMINATE \ --workflow-version "1.0" ``` You can use the [`DescribeWorkflowType`](https://docs.aws.amazon.com/amazonswf/latest/apireference/API_DescribeWorkflowType.html) API to verify that the workflow was registered successfully. Run the following command to describe the `test-workflow` workflow: ```bash lstk aws swf describe-workflow-type \ --domain test-domain \ --workflow-type name=test-workflow,version=1.0 ``` The following output would be retrieved: ```bash title="Output" { "typeInfo": { "workflowType": { "name": "test-workflow", "version": "1.0" }, "status": "REGISTERED", "creationDate": 1420066800.0 }, "configuration": { "defaultTaskStartToCloseTimeout": "30", "defaultExecutionStartToCloseTimeout": "60", "defaultTaskList": { "name": "test-task-list" }, "defaultChildPolicy": "TERMINATE" } } ``` ### Registering an activity You can register an activity using the [`RegisterActivityType`](https://docs.aws.amazon.com/amazonswf/latest/apireference/API_RegisterActivityType.html) API. Execute the following command to register an activity named `test-activity`: ```bash showLineNumbers lstk aws swf register-activity-type \ --domain test-domain \ --name test-activity \ --default-task-list name=test-task-list \ --default-task-start-to-close-timeout 30 \ --default-task-heartbeat-timeout 30 \ --default-task-schedule-to-start-timeout 30 \ --default-task-schedule-to-close-timeout 30 \ --activity-version "1.0" ``` You can use the [`DescribeActivityType`](https://docs.aws.amazon.com/amazonswf/latest/apireference/API_DescribeActivityType.html) API to verify that the activity was registered successfully. Run the following command to describe the `test-activity` activity: ```bash showLineNumbers lstk aws swf describe-activity-type \ --domain test-domain \ --activity-type name=test-activity,version=1.0 ``` The following output would be retrieved: ```bash title="Output" { "typeInfo": { "activityType": { "name": "test-activity", "version": "1.0" }, "status": "REGISTERED", "creationDate": 1420066800.0 }, "configuration": { "defaultTaskStartToCloseTimeout": "30", "defaultTaskHeartbeatTimeout": "30", "defaultTaskList": { "name": "test-task-list" }, "defaultTaskScheduleToStartTimeout": "30", "defaultTaskScheduleToCloseTimeout": "30" } } ``` ### Starting a workflow execution You can start a workflow execution using the [`StartWorkflowExecution`](https://docs.aws.amazon.com/amazonswf/latest/apireference/API_StartWorkflowExecution.html) API. Execute the following command to start a workflow execution for the `test-workflow` workflow: ```bash showLineNumbers lstk aws swf start-workflow-execution \ --domain test-domain \ --workflow-type name=test-workflow,version=1.0 \ --workflow-id test-workflow-id \ --task-list name=test-task-list \ --input '{"foo": "bar"}' ``` The following output would be retrieved: ```bash title="Output" { "runId": "0602601afc71403abb934d8094c51668" } ``` ## API Coverage # Textract > Get started with Textract on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; Textract is a machine learning service that automatically extracts text, forms, and tables from scanned documents. It simplifies the process of extracting valuable information from a variety of document types, enabling applications to quickly analyze and understand document content. LocalStack allows you to mock Textract APIs in your local environment. The supported APIs are available on our [API Coverage section](#api-coverage), providing details on the extent of Textract's integration with LocalStack. ## Getting started This guide is tailored for users new to Textract and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to perform basic Textract operations, such as mocking text detection in a document. ### Detect document text You can use the [`DetectDocumentText`](https://docs.aws.amazon.com/textract/latest/dg/API_DetectDocumentText.html) API to identify and extract text from a document. Execute the following command: ```bash lstk aws textract detect-document-text \ --document '{"S3Object":{"Bucket":"your-bucket","Name":"your-document"}}' ``` ```bash title="Output" { "DocumentMetadata": { "Pages": { "Pages": 389 } }, "Blocks": [], "DetectDocumentTextModelVersion": "1.0" } ``` ### Start document text detection job You can use the [`StartDocumentTextDetection`](https://docs.aws.amazon.com/textract/latest/dg/API_StartDocumentTextDetection.html) API to asynchronously detect text in a document. Execute the following command: ```bash lstk aws textract start-document-text-detection \ --document-location '{"S3Object":{"Bucket":"bucket","Name":"document"}}' ``` ```bash title="Output" { "JobId": "501d7251-1249-41e0-a0b3-898064bfc506" } ``` Save the `JobId` value to use in the next command. ### Get document text detection job You can use the [`GetDocumentTextDetection`](https://docs.aws.amazon.com/textract/latest/dg/API_GetDocumentTextDetection.html) API to retrieve the results of a document text detection job. Execute the following command: ```bash lstk aws textract get-document-text-detection \ --job-id "501d7251-1249-41e0-a0b3-898064bfc506" ``` Replace `501d7251-1249-41e0-a0b3-898064bfc506` with the `JobId` value retrieved from the previous command. ```bash title="Output" { "DocumentMetadata": { "Pages": { "Pages": 389 } }, "JobStatus": "SUCCEEDED", "Blocks": [], "DetectDocumentTextModelVersion": "1.0" } ``` ## API Coverage # Timestream > Get started with Timestream on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction LocalStack contains basic support for Timestream time series databases, including these operations: * Creating databases * Creating tables * Writing records to tables * Querying timeseries data from tables The supported APIs are available on our API Coverage Page ([Timestream Query](#api-coverage-timestream-query)/[Timestream Write](#api-coverage-timestream-write), which provides information on the extent of Timestream integration with LocalStack. ## Getting Started The following example illustrates the basic operations, using the [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command line. First, we create a test database and table: ```bash lstk aws timestream-write create-database --database-name testDB lstk aws timestream-write create-table --database-name testDB --table-name testTable ``` We can then add a few records with a timestamp, measure name, and value to the table: ```bash lstk aws timestream-write write-records --database-name testDB --table-name testTable --records '[{"MeasureName":"cpu","MeasureValue":"60","TimeUnit":"SECONDS","Time":"1636986409"}]' lstk aws timestream-write write-records --database-name testDB --table-name testTable --records '[{"MeasureName":"cpu","MeasureValue":"80","TimeUnit":"SECONDS","Time":"1636986412"}]' lstk aws timestream-write write-records --database-name testDB --table-name testTable --records '[{"MeasureName":"cpu","MeasureValue":"70","TimeUnit":"SECONDS","Time":"1636986414"}]' ``` Finally, we can run a query to retrieve the timeseries data (or aggregate values) from the table: ```bash lstk aws timestream-query query --query-string "SELECT CREATE_TIME_SERIES(time, measure_value::double) as cpu FROM testDB.timeStreamTable WHERE measure_name='cpu'" ``` ```bash title="Output" { "Rows": [{ "Data": [{ "TimeSeriesValue": [{ "Time": "2021-11-15T14:26:49", "Value": { "ScalarValue": 60 } }, ... ``` ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing Timestream databases. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resources** section, and then clicking on **Timestream** under the **Database** section. ![Timestream Resource Browser](/images/aws/timestream-resource-browser.png) The Resource Browser allows you to perform the following actions: * **Create Database**: Create a new Timestream database by clicking on the **Create Database** button and providing a name for the database among other optional details. * **Create Table**: Create a new Timestream table by clicking on the **Create Table** button in the database view and providing a name for the table among other optional details. * **Run Query**: Run a Timestream query by clicking on the **Run Query** button in the table view and providing a query string. * **View Database/Table Details**: Click on a database or table to view its details, including the schema, retention policy, and other metadata. * **Delete Database/Table**: Delete the Timestream database/table by selecting it and clicking on the **Actions** button followed by **Remove Selected** button. ## Current Limitations LocalStack's Timestream implementation is under active development and only supports a limited set of operations, please refer to the API Coverage pages for an up-to-date list of implemented and tested functions within [Timestream-Query](#api-coverage-timestream-query) and [Timestream-Write](#api-coverage-timestream-write). If you have a usecase that uses Timestream but doesn't work with our implementation yet, we encourage you to [get in touch](https://localstack.cloud/contact/), so we can streamline any operations you rely on. ## API Coverage (Timestream Query) ## API Coverage (Timestream Write) # Transcribe > Get started with Amazon Transcribe on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Transcribe is a service provided by AWS that offers automatic speech recognition (ASR) capabilities. It enables developers to convert spoken language into written text, making it valuable for a wide range of applications, from transcription services to voice analytics. LocalStack allows you to use the Transcribe APIs for offline speech-to-text jobs in your local environment. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Transcribe integration with LocalStack. LocalStack Transcribe uses an offline speech-to-text library called [Vosk](https://alphacephei.com/vosk/). It requires an active internet connection to download the language model. Once the language model is downloaded, subsequent transcriptions for the same language can be performed offline. Language models typically have a size of around 50 MiB and are saved in the cache directory (see [Filesystem Layout](/aws/customization/advanced/filesystem)). ## Getting Started This guide is designed for users new to Transcribe and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a transcription job and view the transcript in an S3 bucket using the AWS CLI. ### Create an S3 bucket You can create an S3 bucket using the [`mb`](https://docs.aws.amazon.com/cli/latest/reference/s3/mb.html) command. Run the following command to create a bucket named `foo` to upload a sample audio file named `example.wav`: ```bash lstk aws s3 mb s3://foo lstk aws s3 cp ~/example.wav s3://foo/example.wav ``` ### Create a transcription job You can create a transcription job using the [`StartTranscriptionJob`](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_StartTranscriptionJob.html) API. Run the following command to create a transcription job named `example` for the audio file `example.wav`: ```bash lstk aws transcribe start-transcription-job \ --transcription-job-name example \ --media MediaFileUri=s3://foo/example.wav \ --language-code en-IN ``` You can list the transcription jobs using the [`ListTranscriptionJobs`](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_ListTranscriptionJobs.html) API. Run the following command to list the transcription jobs: ```bash lstk aws transcribe list-transcription-jobs ``` The following output would be retrieved: ```bash title="Output" { "TranscriptionJobSummaries": [ { "TranscriptionJobName": "example", "CreationTime": "2022-08-17T14:04:39.277000+05:30", "StartTime": "2022-08-17T14:04:39.308000+05:30", "LanguageCode": "en-IN", "TranscriptionJobStatus": "IN_PROGRESS" } ] } ``` ### View the transcript After the job is complete, the transcript can be retrieved from the S3 bucket using the [`GetTranscriptionJob`](https://docs.aws.amazon.com/transcribe/latest/APIReference/API_GetTranscriptionJob.html) API. Run the following command to get the transcript: ```bash lstk aws transcribe get-transcription-job --transcription-job example ``` ```bash title="Output" { "TranscriptionJob": { "TranscriptionJobName": "example", "TranscriptionJobStatus": "COMPLETED", "LanguageCode": "en-IN", "MediaFormat": "wav", "Media": { "MediaFileUri": "s3://foo/example.wav" }, "Transcript": { "TranscriptFileUri": "s3://foo/7844aaa5.json" }, "CreationTime": "2022-08-17T14:04:39.277000+05:30", "StartTime": "2022-08-17T14:04:39.308000+05:30", "CompletionTime": "2022-08-17T14:04:57.400000+05:30", } } ``` You can then view the transcript by running the following command: ```bash lstk aws s3 cp s3://foo/7844aaa5.json . jq .results.transcripts[0].transcript 7844aaa5.json ``` The following output would be retrieved: ```bash title="Output" "it is just a question of getting rid of the illusion that we are separate from nature" ``` ## Audio Formats The following input media formats are supported: - Adaptive Multi-Rate (AMR) - Free Lossless Audio Codec (FLAC) - MPEG-1 Audio Layer-3 (MP3) - MPEG-4 Part 14 (MP4) - OGG - Matroska Video files (MKV) - Waveform Audio File Format (WAV) ## Supported Languages The following languages and dialects are supported: | Language | Language Code | | ---------------- | ------------- | | Catalan | `ca-ES` | | Czech | `cs-CZ` | | German | `de-DE` | | English, British | `en-GB` | | English, Indian | `en-IN` | | English, US | `en-US` | | Spanish | `es-ES` | | Farsi | `fa-IR` | | French | `fr-FR` | | Gujarati | `gu-IN` | | Hindi | `hi-IN` | | Italian | `it-IT` | | Japan | `ja-JP` | | Kazakh | `kk-KZ` | | Korean | `ko-KR` | | Dutch | `nl-NL` | | Polish | `pl-PL` | | Portuguese | `pt-BR` | | Russian | `ru-RU` | | Telugu | `te-IN` | | Turkish | `tr-TR` | | Ukrainian | `uk-UA` | | Uzbek | `uz-UZ` | | Vietnamese | `vi-VN` | | Chinese | `zh-CN` | ## Resource Browser The LocalStack Web Application provides a Resource Browser for managing Transcribe Transcription Jobs. You can access the Resource Browser by opening the LocalStack Web Application in your browser, navigating to the **Resource Browser** section, and then clicking on **Transcribe Service** under the **Machine Learning** section. ![Transcribe Resource Browser](/images/aws/transcribe-resource-browser.png) The Resource Browser allows you to perform the following actions: - **Create Transcription Job**: Create a new transcription job by clicking on the **Create Transcription Job** button, and then providing the required details. - **View Transcription Job**: View the details of a specific transcription job by clicking on the job in the list. - **Delete Transcription Job**: Delete the transcription job by clicking on the **Actions** button followed by **Remove Selected** button. ## Examples The following code snippets and sample applications provide practical examples of how to use Transcribe in LocalStack for various use cases: - [Serverless Transcription App using Transcribe, S3, Lambda, SQS, SES](https://github.com/localstack-samples/sample-serverless-transcribe) ## Limitations Transcribe does not support speaker diarization and does not produce certain other numerical information, like confidence levels. ## API Coverage # Transfer > Get started with Transfer on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction The AWS Transfer API is a powerful tool that empowers users to establish FTP(S) servers with ease. These servers serve as gateways, allowing direct access to files residing in Amazon S3 buckets. This functionality streamlines file management processes, making it simpler and more efficient to handle data stored in S3 by providing a familiar FTP interface for users to interact with their files securely. Whether you're looking to facilitate file transfers or enhance your data access capabilities, the AWS Transfer API simplifies the process and extends the versatility of your cloud storage infrastructure. ## Getting started This Python code demonstrates a basic workflow for transferring a file between a local machine and AWS S3 using the AWS Transfer Family service and FTP (File Transfer Protocol). ```python showLineNumbers import io import time import uuid import boto3 from ftplib import FTP, FTP_TLS EDGE_URL = 'http://localhost:4566' USERNAME = 'user_123' BUCKET = 'transfer-files' S3_FILENAME = 'test-file-aws-transfer.txt' FTP_USER_DEFAULT_PASSWD = '12345' FILE_CONTENT = b'title "Test" \nfile content!!' # create bucket s3_client = boto3.client('s3', endpoint_url=EDGE_URL) s3_client.create_bucket(Bucket=BUCKET) transfer_client = boto3.client('transfer', endpoint_url=EDGE_URL) # create transfer server rs = transfer_client.create_server( EndpointType='PUBLIC', IdentityProviderType='SERVICE_MANAGED', Protocols=['FTP'] ) time.sleep(1) server_id = rs['ServerId'] port = int(server_id[-4:]) transfer_client.create_user( ServerId=server_id, HomeDirectory=BUCKET, HomeDirectoryType='PATH', Role='arn:aws:iam::testrole', UserName=USERNAME ) # upload file through AWS Transfer ftp = FTP() ftp.connect('localhost', port=port) result = ftp.login(USERNAME, FTP_USER_DEFAULT_PASSWD) assert 'Login successful.' in result ftp.storbinary(cmd='STOR %s' % S3_FILENAME, fp=io.BytesIO(FILE_CONTENT)) ftp.quit() # download file through AWS S3 rs = s3_client.get_object(Bucket=BUCKET, Key=S3_FILENAME) assert rs['Body'].read() == FILE_CONTENT ``` Please note that this code is a simplified example for demonstration purposes. In a production environment, you should use more secure practices, including setting proper IAM roles and handling sensitive credentials securely. Additionally, error handling and cleanup code may be needed to ensure the script behaves robustly in all scenarios. ## Current Limitations The Transfer API does not provide a way to return the endpoint URL of created FTP servers. Hence, in order to determine the server endpoint, the local port is encoded as a suffix within the `ServerId` attribute, constituting the only numeric digits within the ID string. For example, assume the following is the response from the `CreateServer` API call, then the FTP server is accessible on port `4511` (i.e., `ftp://localhost:4511`): ```bash title="Output" { "ServerId": "s-afcedbffaecca4511" } ``` ## API Coverage # Verified Permissions > Get started with Verified Permissions on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Amazon Verified Permissions is a scalable service for managing fine-grained permissions and authorization in custom applications. It helps secure applications by moving authorization logic outside the app and managing policies in one place, using the [Cedar policy language](https://docs.cedarpolicy.com/) to define access rules. It checks if a principal can take an action on a resource in a specific context in your application. LocalStack allows you to use the Verified Permissions APIs in your local environment to test your authorization logic, with integrations with other AWS services like Cognito and support for custom OIDC identity providers. LocalStack uses the Cedar engine to evaluate permissions, ensuring authorization testing closely matches AWS Verified Permissions behavior. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Verified Permissions' integration with LocalStack. ## Getting started This guide is designed for users new to Verified Permissions and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how to create a Verified Permissions Policy Store, add a policy to it, and authorize a request with the AWS CLI. ### Create a Policy Store To create a Verified Permissions Policy Store, use the [`CreatePolicyStore`](https://docs.aws.amazon.com/verifiedpermissions/latest/apireference/API_CreatePolicyStore.html) API. Run the following command to create a Policy Store with Schema validation settings set to `OFF`: ```bash lstk aws verifiedpermissions create-policy-store \ --validation-settings mode=OFF \ --description "A local Policy Store" ``` ```bash title="Output" { "policyStoreId": "q5PCScu9qo4aswMVc0owNN", "arn": "arn:aws:verifiedpermissions::000000000000:policy-store/q5PCScu9qo4aswMVc0owNN", "createdDate": "2025-04-22T19:24:11.175557Z", "lastUpdatedDate": "2025-04-22T19:24:11.175557Z" } ``` You can list all the Verified Permissions policy stores using the [`ListPolicyStores`](https://docs.aws.amazon.com/verifiedpermissions/latest/apireference/API_ListPolicyStores.html) API. Run the following command to list all the Verified Permissions policy stores: ```bash lstk aws verifiedpermissions list-policy-stores ``` ### Create a Policy To create a Verified Permissions Policy, use the [`CreatePolicy`](https://docs.aws.amazon.com/verifiedpermissions/latest/apireference/API_CreatePolicy.html) API. Create a JSON file named `static_policy.json` with the following content: ```json showLineNumbers { "static": { "description": "Grant the User alice access to view the trip Album", "statement": "permit(principal == User::\"alice\", action == Action::\"view\", resource == Album::\"trip\");" } } ``` You can then run this command to create the policy: ```bash lstk aws verifiedpermissions create-policy \ --definition file://static_policy.json \ --policy-store-id q5PCScu9qo4aswMVc0owNN ``` Replace the policy store ID with the ID of the policy store you created previously. You should see the following output: ```bash title="Output" { "policyStoreId": "q5PCScu9qo4aswMVc0owNN", "policyId": "MfsIseJDeZsr5WUm3tB4FX", "policyType": "STATIC", "principal": { "entityType": "User", "entityId": "alice" }, "resource": { "entityType": "Album", "entityId": "trip" }, "actions": [ { "actionType": "Action", "actionId": "view" } ], "createdDate": "2025-04-22T19:25:25.161652Z", "lastUpdatedDate": "2025-04-22T19:25:25.161652Z", "effect": "Permit" } ``` ### Authorize a request We can now make use of the Policy Store and the Policy to start authorizing requests. To authorize a request using Verified Permissions, use the [`IsAuthorized`](https://docs.aws.amazon.com/verifiedpermissions/latest/apireference/API_IsAuthorized.html) API. ```bash title="Output" lstk aws verifiedpermissions is-authorized \ --policy-store-id q5PCScu9qo4aswMVc0owNN \ --principal entityType=User,entityId=alice \ --action actionType=Action,actionId=view \ --resource entityType=Album,entityId=trip ``` You should get the following output, indicating that your request was allowed: ```bash title="Output" { "decision": "ALLOW", "determiningPolicies": [ { "policyId": "MfsIseJDeZsr5WUm3tB4FX" } ], "errors": [] } ``` ## Identity Sources LocalStack supports both Cognito User Pools and custom OIDC (OpenID Connect) identity providers as identity sources for Verified Permissions. When you create an identity source with an [`OpenIdConnectConfiguration`](https://docs.aws.amazon.com/verifiedpermissions/latest/apireference/API_OpenIdConnectConfiguration.html), LocalStack: - Fetches the OIDC discovery document and JWKS (JSON Web Key Set) from the configured issuer - Validates JWT signatures against the issuer's public keys - Enforces token expiration (`exp` claim) - Extracts principal information and group memberships from token claims Once the identity source is configured, you can evaluate authorization requests using tokens from that provider with [`IsAuthorizedWithToken`](https://docs.aws.amazon.com/verifiedpermissions/latest/apireference/API_IsAuthorizedWithToken.html) and [`BatchIsAuthorizedWithToken`](https://docs.aws.amazon.com/verifiedpermissions/latest/apireference/API_BatchIsAuthorizedWithToken.html). :::note For local development scenarios where the OIDC issuer may not be reachable or uses a self-signed certificate, you can disable JWT signature verification by setting the `VERIFIEDPERMISSIONS_DISABLE_JWT_VERIFICATION=1` environment variable. See the [Configuration reference](/aws/customization/configuration-options/#verified-permissions) for details. ::: ## Current limitations - No Schema validation when creating a new schema using `PutSchema`, and no Policy validation using said schema when creating policies and template policies. ## API Coverage # Web Application Firewall (WAF) > Get started with Web Application Firewall (WAF) on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction Web Application Firewall (WAF) is a service provided by Amazon Web Services (AWS) that helps protect your web applications from common web exploits that could affect application availability, compromise security, or consume excessive resources. WAFv2 is the latest version of WAF, and it allows you to specify a single set of rules to protect your web applications, APIs, and mobile applications from common attack patterns, such as SQL injection and cross-site scripting. LocalStack allows you to use the WAFv2 APIs for offline web application firewall jobs in your local environment. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of WAFv2 integration with LocalStack. ## Getting started This guide is for users who are familiar with the AWS CLI and [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will walk you through creating, listing, tagging, and viewing tags for Web Access Control Lists (WebACLs) using the Web Application Firewall (WAF) service in a LocalStack environment using the AWS CLI. ### Create a WebACL Start by creating a Web Access Control List (WebACL) using the [`CreateWebACL`](https://docs.aws.amazon.com/waf/latest/APIReference/API_CreateWebACL.html) API. Run the following command to create a WebACL named `TestWebAcl`: ```bash showLineNumbers lstk aws wafv2 create-web-acl \ --name TestWebAcl \ --scope REGIONAL \ --default-action Allow={} \ --visibility-config SampledRequestsEnabled=true,CloudWatchMetricsEnabled=true,MetricName=TestWebAclMetrics ``` ```bash title="Output" { "Summary": { "Name": "TestWebAcl", "Id": "f94fd5bc-e4d4-4280-9f53-51e9441ad51d", "Description": "", "ARN": "arn:aws:wafv2:us-east-1:000000000000:regional/webacl/TestWebAcl/f94fd5bc-e4d4-4280-9f53-51e9441ad51d" } } ``` Note the `Id` and `ARN` from the output, as they will be needed for subsequent commands. ### List WebACLs To view all the WebACLs you have created, use the [`ListWebACLs`](https://docs.aws.amazon.com/waf/latest/APIReference/API_ListWebACLs.html) API. Run the following command to list the WebACLs: ```bash lstk aws wafv2 list-web-acls --scope REGIONAL ``` ```bash title="Output" { "NextMarker": "Not Implemented", "WebACLs": [ { "Name": "TestWebAcl", "Id": "f94fd5bc-e4d4-4280-9f53-51e9441ad51d", "Description": "", "ARN": "arn:aws:wafv2:us-east-1:000000000000:regional/webacl/TestWebAcl/f94fd5bc-e4d4-4280-9f53-51e9441ad51d" } ] } ``` ### Tag a WebACL Tagging resources in AWS WAF helps you manage and identify them. Use the [`TagResource`](https://docs.aws.amazon.com/waf/latest/APIReference/API_TagResource.html) API to add tags to a WebACL. Run the following command to add a tag to the WebACL created in the previous step: ```bash lstk aws wafv2 tag-resource \ --resource-arn arn:aws:wafv2:us-east-1:000000000000:regional/webacl/TestWebAcl/f94fd5bc-e4d4-4280-9f53-51e9441ad51d \ --tags Key=Name,Value=AWSWAF ``` After tagging your resources, you may want to view these tags. Use the [`ListTagsForResource`](https://docs.aws.amazon.com/waf/latest/APIReference/API_ListTagsForResource.html) API to list the tags for a WebACL. Run the following command to list the tags for the WebACL created in the previous step: ```bash lstk aws wafv2 list-tags-for-resource \ --resource-arn arn:aws:wafv2:us-east-1:000000000000:regional/webacl/TestWebAcl/f94fd5bc-e4d4-4280-9f53-51e9441ad51d ``` ```bash title="Output" { "TagInfoForResource": { "ResourceARN": "arn:aws:wafv2:us-east-1:000000000000:regional/webacl/TestWebAcl/f94fd5bc-e4d4-4280-9f53-51e9441ad51d", "TagList": [ { "Key": "Name", "Value": "AWSWAF" } ] } } ``` ## API Coverage # X-Ray > Get started with X-Ray on LocalStack import FeatureCoverage from "../../../../components/feature-coverage/FeatureCoverage"; ## Introduction [X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/aws-xray.html) is a distributed tracing service that helps to understand cross-service interactions and facilitates debugging of performance bottlenecks. Instrumented applications generate trace data by recording trace segments with information about the work tasks of an application, such as timestamps, tasks names, or metadata. X-Ray supports different ways of [instrumenting your application](https://docs.aws.amazon.com/xray/latest/devguide/xray-instrumenting-your-app.html) including the [AWS X-Ray SDK](https://docs.aws.amazon.com/xray/latest/devguide/xray-instrumenting-your-app.html#xray-instrumenting-xray-sdk) and the [AWS Distro for OpenTelemetry (ADOT)](https://docs.aws.amazon.com/xray/latest/devguide/xray-instrumenting-your-app.html#xray-instrumenting-opentel). [X-Ray daemon](https://docs.aws.amazon.com/xray/latest/devguide/xray-daemon.html) is an application that gathers raw trace segment data from the X-Ray SDK and relays it to the AWS X-Ray API. The X-Ray API can then be used to retrieve traces originating from different application components. LocalStack allows you to use the X-Ray APIs to send and retrieve trace segments in your local environment. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of X-Ray integration with LocalStack. ## Getting started This guide is designed for users new to X-Ray and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command. Start your LocalStack container using your preferred method. We will demonstrate how you can create a minimal [trace segment](https://docs.aws.amazon.com/xray/latest/devguide/xray-api-segmentdocuments.html#api-segmentdocuments-fields) and manually send it to the X-Ray API. Notice that this trace ingestion typically happens in the background, for example by the X-Ray SDK and X-Ray daemon. [PutTraceSegments](https://docs.aws.amazon.com/xray/latest/api/API_PutTraceSegments.html). ### Sending trace segments You can generates a unique trace ID and constructs a JSON document with trace information. It then sends this trace segment to the AWS X-Ray API using the [PutTraceSegments](https://docs.aws.amazon.com/xray/latest/api/API_PutTraceSegments.html) API. Run the following commands in your terminal: ```bash showLineNumbers START_TIME=$(date +%s) HEX_TIME=$(printf '%x\n' $START_TIME) GUID=$(dd if=/dev/random bs=12 count=1 2>/dev/null | od -An -tx1 | tr -d ' \t\n') TRACE_ID="1-$HEX_TIME-$GUID" END_TIME=$(($START_TIME+3)) DOC=$(cat < # Tutorials > These tutorials enhance your comprehension of LocalStack's functionality by providing detailed information on how it works for specific use cases using diverse resources. import DynamicTutorials from '../../../../components/DynamicTutorials.astro'; # Replicating cloud resources locally with LocalStack's AWS Cloud Proxy extension > Learn how you can replicate cloud resources in your local environment using the LocalStack's AWS Cloud Proxy extension. This tutorial provides step-by-step guidance on setting up and leveraging the AWS Cloud Proxy extension to mirror cloud services locally, enabling efficient hybrid development and testing workflows without maintaining additional configurations. ## Introduction LocalStack's core cloud emulator enables you to emulate various cloud services on your own local machine. This allows you to work on and test your cloud-based solutions without needing to connect to a remote cloud. However, sometimes you might need to smoothly switch between your local setup and actual cloud resources, especially in hybrid scenarios. This could be useful, for example, if you want to share a database with your local Lambda function, or if you require access to S3 files stored remotely while running a Glue ETL job locally. With the [AWS Cloud Proxy extension](https://github.com/localstack/localstack-extensions/tree/main/aws-proxy), you can: - Enable your local environment to mirror AWS cloud resources at the API level, allowing for direct interaction with cloud services. - Facilitate the forwarding of specific requests from LocalStack to AWS without the need for complex proxy setups. - Support scenarios that require a combination of local and cloud resources, such as testing cloud services with local databases or functions. In this tutorial, you will learn how to install the AWS Cloud Proxy extension and utilize its different modes to seamlessly work in a hybrid environment. ## Prerequisites - [`lstk`](/aws/getting-started/installation#lstk) with [`LOCALSTACK_AUTH_TOKEN`](/aws/getting-started/auth-token) - [Docker](https://docs.docker.com/) - [AWS CLI](https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-install.html) with [`lstk aws`](/aws/connecting/aws-cli#localstack-aws-cli-lstk-aws) - [LocalStack account](https://www.localstack.cloud/pricing) - [AWS Account](https://aws.amazon.com/) with an [`AWS_ACCESS_KEY_ID` & `AWS_SECRET_ACCESS_KEY`](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html#Using_CreateAccessKey) ## Install the AWS Cloud Proxy extension To install the AWS Cloud Proxy Extension, follow these steps: 1. Launch your LocalStack container using the `lstk` CLI, ensuring that `LOCALSTACK_AUTH_TOKEN` is available in the environment. 2. Visit the [Extensions library](https://app.localstack.cloud/extensions/library) page on the LocalStack Web Application. ![Extensions Library](/images/aws/aws-proxy-tutorial/extensions-library.png) 3. Scroll down to find the **AWS Cloud Proxy** card, then click on the **Install on Instance** button. ![Installing AWS Cloud Proxy extension](/images/aws/aws-proxy-tutorial/installing-aws-proxy-extensions.png) Once the installation is complete, you will notice that your LocalStack container has restarted with the AWS Cloud Proxy extension successfully installed. ## Tutorial: Working with the AWS Cloud Proxy Extension In this tutorial, you will set up a basic example consisting of: - A Lambda function named `func1` that prints a simple statement when invoked. - An SQS queue named `test-queue` where messages are sent. - An event source mapping that triggers the Lambda function when a message is sent to the SQS queue. The basic architecture for the scenario is outlined in the figure below. It shows the relationship between the resources deployed in the LocalStack container, the LocalStack AWS Proxy, and the remote AWS account. ![AWS Cloud Proxy sample use case](/images/aws/aws-proxy-sqs-lambda-sample.png) In the following sections, you will create the SQS queue on your local machine and the remote cloud to showcase how you can switch between the two with the AWS Cloud Proxy extension. ### Create the Lambda function Begin by running your LocalStack container with the following configuration: ```bash LOCALSTACK_EXTRA_CORS_ALLOWED_ORIGINS=https://aws-proxy.localhost.localstack.cloud:4566 \ LOCALSTACK_DEBUG=1 \ lstk start ``` In the above command: - The `LOCALSTACK_EXTRA_CORS_ALLOWED_ORIGINS` variable allows the AWS Cloud Proxy extension's web interface to connect with the LocalStack container. - The `LOCALSTACK_DEBUG` variable enables verbose logging allowing you to see the printed statements from the Lambda function. Next, create a file named `testlambda.py` and add the following Python code to it: ```python def handler(*args, **kwargs): print("Debug output from Lambda function") ``` Execute the following commands to create the local Lambda function: ```bash (zip testlambda.zip testlambda.py) lstk aws lambda create-function \ --function-name func1 \ --runtime python3.8 \ --role arn:aws:iam::000000000000:role/r1 --handler testlambda.handler \ --timeout 30 \ --zip-file fileb://./testlambda.zip ``` ```bash title="Output" { "FunctionName": "func1", "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:func1", "Runtime": "python3.8", "Role": "arn:aws:iam::000000000000:role/r1", "Handler": "testlambda.handler", "CodeSize": 250, ... } ``` ### Create the SQS queue You can create the local SQS queue named `test-queue` by executing the following command: ```bash lstk aws sqs create-queue --queue-name test-queue ``` ```bash title="Output" { "QueueUrl": "http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/test-queue" } ``` Additionally, you can create the remote SQS queue on the real AWS cloud to test invocation after starting the AWS Cloud Proxy extension. Use the following command to set up the SQS queue on AWS: ```bash aws sqs create-queue --queue-name test-queue ``` ### Invoke the Lambda function Before invoking, set up an event source mapping between the SQS queue and the Lambda function. Configure the queue for Lambda using the following command: ```bash lstk aws lambda create-event-source-mapping \ --function-name func1 \ --batch-size 1 \ --event-source-arn arn:aws:sqs:us-east-1:000000000000:test-queue ``` ```bash title="Output" { ... "MaximumBatchingWindowInSeconds": 0, "EventSourceArn": "arn:aws:sqs:us-east-1:000000000000:test-queue", "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:func1", .. "FunctionResponseTypes": [] } ``` You can then send a message to the SQS queue to trigger the local Lambda function: ```bash lstk aws sqs send-message \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/test-queue \ --message-body '{}' ``` Upon successful execution, you will receive a message ID and MD5 hash of the message body. ```bash title="Output" { "MD5OfMessageBody": "99914b932bd37a50b983c5e7c90ae93b", "MessageId": "64e8297c-f0b2-4b68-a482-6cd3317f5096" } ``` In the LocalStack logs, you will see confirmation of the Lambda function invocation along with any debug messages. ```bash title="Output" 2024-03-26T07:23:47.842 DEBUG --- [5119b27cdf1e] l.s.l.i.version_manager : [func1-381c6f7c-3ad8-4c79-aad8-5119b27cdf1e] START RequestId: 381c6f7c-3ad8-4c79-aad8-5119b27cdf1e Version: $LATEST 2024-03-26T07:23:47.842 DEBUG --- [5119b27cdf1e] l.s.l.i.version_manager : [func1-381c6f7c-3ad8-4c79-aad8-5119b27cdf1e] Debug output from Lambda function 2024-03-26T07:23:47.842 DEBUG --- [5119b27cdf1e] l.s.l.i.version_manager : [func1-381c6f7c-3ad8-4c79-aad8-5119b27cdf1e] END RequestId: 381c6f7c-3ad8-4c79-aad8-5119b27cdf1e ``` ### Run the AWS Cloud Proxy extension To run the AWS Cloud Proxy extension: - Access [`https://aws-proxy.localhost.localstack.cloud:4566`](https://aws-proxy.localhost.localstack.cloud:4566/) via your web browser. - Provide your AWS Credentials: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and optionally `AWS_SESSION_TOKEN`. - Add a new YAML-based Proxy configuration to proxy requests for specific resources to AWS. For this scenario, configure it to proxy requests for the SQS queue created earlier. ```yaml services: sqs: resources: - '.*:test-queue' ``` - Save the configuration to enable the AWS Cloud Proxy extension. Once enabled, you will see the proxy status as **enabled**. To invoke the local Lambda function with the remote SQS queue: - Navigate to your AWS Management Console and access **Simple Queue Service**. - Select the **test-queue** queue. - Send a message with a body (e.g., `Hello LocalStack`) by clicking **Send Message**. You will observe the local Lambda function being invoked once again, with corresponding debug messages visible in the logs. ```bash title="Output" 2024-03-26T07:45:16.524 DEBUG --- [db58fad602e5] l.s.l.i.version_manager : [func1-ed938bb0-e1ee-41fb-a844-db58fad602e5] START RequestId: ed938bb0-e1ee-41fb-a844-db58fad602e5 Version: $LATEST 2024-03-26T07:45:16.524 DEBUG --- [db58fad602e5] l.s.l.i.version_manager : [func1-ed938bb0-e1ee-41fb-a844-db58fad602e5] Debug output from Lambda function 2024-03-26T07:45:16.524 DEBUG --- [db58fad602e5] l.s.l.i.version_manager : [func1-ed938bb0-e1ee-41fb-a844-db58fad602e5] END RequestId: ed938bb0-e1ee-41fb-a844-db58fad602e5 ``` You can even run the standard `lstk aws` commands in your terminal that would query the remote cloud resources, instead of the local ones. Upon completion, you can click **Disable** on the AWS Cloud Proxy extension web interface to deactivate the proxy configuration. Additionally, you can delete the remote SQS queue to avoid AWS billing for long-running resources. To remove local resources, stop the LocalStack container to clear the local Lambda function and SQS queue. ## Conclusion In this tutorial, you've discovered how the AWS Cloud Proxy extension bridges the gap between local and remote cloud resources by mirroring resources from real AWS accounts into your LocalStack instance. You can explore additional use-cases with the AWS Cloud Proxy extension, such as: - Developing a local Lambda function that interacts with a remote DynamoDB table - Executing a local Athena SQL query in LocalStack, accessing files in a real S3 bucket on AWS - Testing a local Terraform script with SSM parameters from a real AWS account - And many more! # How To: Collaborative AWS local development with LocalStack's Cloud Pods > Replicating development environments ensures that all developers, regardless of their local machine configurations or operating systems, work within an environment that closely mirrors production. This consistency helps identify and solve environment-specific issues early in the development cycle, reducing the "it works on my machine" problem where code behaves differently on different developers' machines. # **Introduction** By replicating environments, teams can share the exact conditions under which a bug occurs. For developing AWS applications locally, the tool of choice is LocalStack, which can sustain a full-blown comprehensive stack. However, when issues appear, and engineers need a second opinion from a colleague, recreating the environment from scratch can leave details slipping through the cracks. This is where Cloud Pods come in, to encapsulate the state of the LocalStack instance and allow for seamless collaboration. While databases have snapshots, similarly, LocalStack uses Cloud Pods for reproducing state and data. In this tutorial, we will explore a common situation where a basic IAM misconfiguration causes unnecessary delays in finding the right solution. We will also discuss the best practices to prevent this and review some options for configuring Cloud Pod storage. The full sample application can be found [on GitHub](https://github.com/localstack-samples/cloud-pods-collaboration-demo) to clone, for following along more easily. ### **Prerequisites** - [`lstk`](/aws/getting-started/installation#lstk) - [Docker](https://docs.docker.com/engine/install/) - [Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) or [OpenTofu](https://opentofu.org/docs/intro/install/) and [`lstk terraform`](/aws/connecting/infrastructure-as-code/terraform#lstk-terraform) - Optional for Lambda build & editing: [Maven 3.9.4](https://maven.apache.org/install.html) & [Java 21](https://www.java.com/en/download/help/download_options.html) - Basic knowledge of AWS services (API Gateway, Lambda, DynamoDB, IAM) - Basic understanding of Terraform for provisioning AWS resources In this demo scenario, a new colleague, Bob, joins the company, clones the application repository, and starts working on the Lambda code. He will add the necessary resources in the Terraform configuration file and some IAM policies that the functions need in order to access the database. He is following good practice rules, where the resource has only the necessary permissions. However, Bob encounters an error despite this. ### Architecture Overview The stack consists of an API Gateway that exposes endpoints and integrates with two Lambda functions responsible for adding and fetching products from a DynamoDB database. IAM policies are enforced to ensure compliance with the **[principle of least privilege](https://en.wikipedia.org/wiki/Principle_of_least_privilege)**, and the logs will be sent to the CloudWatch service. ### Note This demo application is suitable for AWS and behaves the same as on LocalStack. You can try this out by running the Terraform configuration file against the AWS platform. ![Application Diagram](/images/aws/cloud-pod-collab.png) ### Starting LocalStack In the root directory, there is a `docker-compose.yml` file that will spin up version 3.3.0 of LocalStack, with an important configuration flag, `ENFORCE_IAM=1`, which will facilitate IAM policy evaluation and enforcement. For this example, a `LOCALSTACK_AUTH_TOKEN` is needed, which you can find in the LocalStack web app on the [Getting Started](https://app.localstack.cloud/getting-started) page. ```bash export LOCALSTACK_AUTH_TOKEN= docker compose up ``` Given that you've started LocalStack via `docker-compose`, you'll need to configure the `lstk` CLI to contact your container: ```bash export LSTK_ENDPOINT_URL=http://localhost.localstack.cloud:4566 ``` ### The Terraform Configuration File The entire Terraform configuration file for setting up the application stack is available in the same repository at https://github.com/localstack-samples/cloud-pods-collaboration-demo/blob/main/terraform/main.tf. To deploy all the resources on LocalStack, navigate to the project's root folder and use the following commands: ```bash cd terraform lstk terraform init lstk terraform plan lstk terraform apply --auto-approve ``` `lstk terraform` runs Terraform against LocalStack, using LocalStack endpoints as AWS provider overrides. The endpoints for all services are configured to point to the LocalStack API, which allows you to deploy your unmodified Terraform scripts against LocalStack. - **`init`**: This command initializes the Terraform working directory, installs any necessary plugins, and sets up the backend. - **`plan`**: Creates an execution plan, which allows you to review the actions Terraform will take to change your infrastructure. - **`apply`**: Finally, the **`apply`** command applies the changes required to reach the desired state of the configuration. If **`-auto-approve`** is used, it bypasses the interactive approval step normally required. As mentioned previously, there is something missing from this configuration, and that is the **`GetItem`** operation permission for one of the Lambda functions: ```hcl resource "aws_iam_policy" "lambda_dynamodb_policy" { name = "LambdaDynamoDBAccess" description = "IAM policy for accessing DynamoDB from Lambda" policy = jsonencode({ Version = "2012-10-17" Statement = [ { Action = [ "dynamodb:Scan", "dynamodb:Query", "dynamodb:UpdateItem", "dynamodb:PutItem", ] Effect = "Allow" Resource = "*" }, ] }) } ``` Bob has mistakenly used `dynamodb:Scan` and `dynamodb:Query`, but missed adding the `dynamodb:GetItem` action to the policy document above. ## Testing the application ### Reproducing the issue locally Let's test out the current state of the application. The Terraform configuration file outputs the REST API ID of the API Gateway. We can capture that value and use it further to invoke the **`add-product`** Lambda: ```bash export rest_api_id=$(cd terraform; lstk terraform output --raw rest_api_id) ``` The endpoint for the API Gateway is constructed similarly to the one on AWS: **` https://.execute-api.localhost.localstack.cloud:4566// `** So adding two products to the database is straightforward using `curl`: ```bash curl --location "http://$rest_api_id.execute-api.localhost.localstack.cloud:4566/dev/productApi" \ --header 'Content-Type: application/json' \ --data '{ "id": "34534", "name": "EcoFriendly Water Bottle", "description": "A durable, eco-friendly water bottle designed to keep your drinks cold for up to 24 hours and hot for up to 12 hou s. Made from high-quality, food-grade stainless steel, it'\''s perfect for your daily hydration needs.", "price": "29.99" }' curl --location "http://$rest_api_id.execute-api.localhost.localstack.cloud:4566/dev/productApi?id=82736" \ --header 'Content-Type: application/json' \ --data '{ "id": "82736", "name": "Sustainable Hydration Flask", "description": "This sustainable hydration flask is engineered to maintain your beverages at the ideal temperature—cold for 24 hours and hot for 12 hours. Constructed with premium, food-grade stainless steel, it offers an environmentally friendly solution to stay hydrated throughout the day.", "price": "31.50" }' ``` The response is the one that we expect: `Product added/updated successfully.` However, retrieving one of the products does not return the desired result: ```bash curl --location "http://$rest_api_id.execute-api.localhost.localstack.cloud:4566/dev/productApi?id=34534" ``` ```bash title="Output" Internal server error⏎ ``` An `Internal server error⏎` does not give out too much information. Bob does not know for sure what could be causing this. The Lambda code and the configurations look fine to him. ## Using Cloud Pods for collaborative debugging ### Creating a Cloud Pod To share this exact environment and issue with Alice, a more experienced colleague, Bob only needs to run a simple `lstk snapshot save` command: ```bash lstk snapshot save pod:cloud-pod-product-app ``` ```bash title="Output" Cloud Pod `cloud-pod-product-app` successfully created ✅ Version: 1 Remote: platform Services: sts,iam,apigateway,dynamodb,lambda,s3,cloudwatch,logs ``` LocalStack provides a remote storage backend that can be used to store the state of your application and share it with your team members. Cloud Pods are managed through the `snapshot` command, included in the `lstk` CLI installation, so there's no need for additional plugins to begin using it. The `LOCALSTACK_AUTH_TOKEN` needs to be set as an environment variable. Additionally, there are other `snapshot` subcommands for managing Cloud Pods: - `lstk snapshot save` (alias `lstk save`) — create a new Cloud Pod - `lstk snapshot load` (alias `lstk load`) — load the state of a Cloud Pod into the application runtime - `lstk snapshot list` — list all available Cloud Pods - `lstk snapshot remove` — delete a Cloud Pod - `lstk snapshot show` — show metadata for a Cloud Pod ### Pulling and Loading the Cloud Pod The workflow between Alice and Bob is incredibly easy: ![Bob and Alice Collab](/images/aws/bob-alice-cloud-pod-collab.png) Now, in a fresh LocalStack instance, Alice can immediately load the Cloud Pod, because she's part of the same organization: ```bash lstk snapshot load pod:cloud-pod-product-app ``` ```bash title="Output" Cloud Pod cloud-pod-product-app successfully loaded ``` ### Debugging and Resolving the Issue Not only can Alice easily reproduce the bug now, but she also has access to the state and data of the services involved, meaning that the Lambda logs are still in the CloudWatch log groups. ![CloudWatch Logs](/images/aws/cloudwatch-logs.png) By spotting the error message, there's an instant starting point for checking the source of the problem. The error message displayed in the logs is very specific: `"Error: User: arn:aws:sts::000000000000:assumed-role/productRole/get-product is not authorized to perform: dynamodb:GetItem on resource: arn:aws:dynamodb:us-east-1:000000000000:table/Products because no identity-based policy allows the dynamodb:GetItem action (Service: DynamoDb, Status Code: 400, Request ID: d50e9dad-a01a-4860-8c21-e844a930ba7d)"` ### Identifying the Misconfiguration The error points to a permissions issue related to accessing DynamoDB. The action **`dynamodb:GetItem`** is not authorized for the role, preventing the retrieval of a product by its ID. This kind of error was not foreseen as one of the exceptions to be handled in the application. IAM policies are not always easy and straightforward, so it's a well known fact that these configurations are prone to mistakes. To confirm the finding, Alice now has the exact same environment to reproduces the error in. There are no machine specific configurations and no other manual changes. This leads to the next step in troubleshooting: **inspecting the Terraform configuration file** responsible for defining the permissions attached to the Lambda role for interacting with DynamoDB. ### Fixing the Terraform Configuration Upon review, Alice discovers that the Terraform configuration does not include the necessary permission **`dynamodb:GetItem`** in the policy attached to the Lambda role. This oversight explains the error message. The Terraform configuration file acts as a blueprint for AWS resource permissions, and any missing action can lead to errors related to authorization. This scenario underscores the importance of thorough review and testing of IAM roles and policies when working with AWS resources. It's easy to overlook a single action in a policy, but as we've seen, such an omission can significantly impact application functionality. By carefully checking the Terraform configuration files and ensuring that all necessary permissions are included, developers can avoid similar issues and ensure a smoother, error-free interaction with AWS services. The action list should now look like this: ```bash resource "aws_iam_policy" "lambda_dynamodb_policy" { name = "LambdaDynamoDBAccess" description = "IAM policy for accessing DynamoDB from Lambda" policy = jsonencode({ Version = "2012-10-17" Statement = [ { Action = [ "dynamodb:GetItem", "dynamodb:UpdateItem", "dynamodb:PutItem", ] Effect = "Allow" Resource = "*" }, ] }) } ``` To double-check, Alice creates the stack on AWS, and observes that the issue is the same, related to policy misconfiguration: ![AWS CloudWatch Logs](/images/aws/aws-cloudwatch-logs.png) ### Impact on the team Alice has updated the infrastructure and deployed a new version of the Cloud Pod with the necessary fixes. Bob will access the updated infrastructure and proceed with his tasks. Meanwhile, Carol is developing integration tests for the CI pipeline. She will use the stable version of the infrastructure to ensure that the workflows function effectively from start to finish. ![Carol writes tests](/images/aws/carol-bob-alice-cloud-pod-collab.png) ### Other Remote Options For organizations with specific data regulations, LocalStack offers an Amazon S3 storage option, allowing full control with on-premises storage if needed. That way, Bob, Alice and Carol could collaborate using an S3 bucket for remote storage. The `lstk` command-line interface enables users to manage this storage with ease, by following the instructions in the [documentation](/aws/developer-tools/snapshots/saving-snapshots-to-s3). ## Conclusion Cloud Pods play a crucial role in team collaboration, significantly speeding up development processes. The multiple and versatile options for remote storage can support different business requirements for companies that prefer using the environments they control. Cloud Pods are not just for teamwork; they also excel in other areas, such as creating resources in Continuous Integration (CI) for ultra-fast testing pipelines. This tutorial demonstrated how Cloud Pods enable seamless collaboration between team members by: - Capturing exact environment states for easy sharing - Providing immediate access to service data and logs for debugging - Maintaining consistency across different developer machines - Supporting multiple storage backends for compliance needs The workflow from Bob capturing the problematic state, to Alice debugging and fixing it, to Carol using the stable version for testing showcases how Cloud Pods streamline development cycles and eliminate environment-specific issues. ## Additional resources - [Cloud Pods documentation](/aws/developer-tools/snapshots/cloud-pods) - [Terraform for AWS](https://developer.hashicorp.com/terraform/tutorials/aws-get-started) # Deploying containers on Elastic Container Service (ECS) clusters using Elastic Container Registry (ECR) and AWS Fargate, with LocalStack > Set up an NGINX web server via Elastic Container Service (ECS) and Elastic Container Registry (ECR) to serve a static website using LocalStack. Learn how you can use CloudFormation templates to declaratively define, create, and deploy your architecture locally with LocalStack's `lstk aws` CLI. [Amazon Elastic Container Service (ECS)](https://aws.amazon.com/ecs/) is a fully-managed container orchestration service that simplifies the deployment, management, and scaling of Docker containers on AWS. With support for two [launch types](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/launch_types.html), EC2 and Fargate, ECS allows you to run containers on your cluster of EC2 instances or have AWS manage your underlying infrastructure with Fargate. The Fargate launch type provides a serverless-like experience for running containers, allowing you to focus on your applications instead of infrastructure. [Amazon Elastic Container Registry (ECR)](https://aws.amazon.com/ecr/) is a fully-managed service that allows you to store, manage, and deploy Docker container images. It is tightly integrated with other AWS services such as ECS, EKS, and Lambda, enabling you to quickly deploy your container images to these services. With ECR, you can version, tag, and manage your container images’ lifecycles independently of your applications, making it easy to maintain and deploy your containers. ECS tasks can pull container images from ECR repositories and are customizable using task definitions to specify settings such as CPU and memory limits, environment variables, and networking configurations. [LocalStack for AWS](https://localstack.cloud/) allows creating ECR registries, repositories, and ECS clusters and tasks on your local machine. This tutorial will showcase using LocalStack to set up an NGINX web server to serve a static website using CloudFormation templates in a local AWS environment. ## Prerequisites - [LocalStack for AWS](https://localstack.cloud/pricing/) - [`lstk aws`](/aws/connecting/aws-cli#localstack-aws-cli-lstk-aws) - [Docker](https://docker.io/) - [curl](https://curl.se/download.html) ## Creating the Docker image To start setting up an NGINX web server on an ECS cluster, we need to create a Docker image that can be pushed to an ECR repository. We'll begin by creating a `Dockerfile` that defines the configuration for our NGINX web server. ```dockerfile FROM nginx ENV foo=bar ``` The `Dockerfile` uses the official `nginx` image from Docker Hub, which allows us to serve the default index page. Before building our Docker image, we need to start LocalStack and create an ECR repository to push our Docker image. To start LocalStack, run the following command: ```bash lstk start ``` Next, we will create an ECR repository to push our Docker image. We will use the `lstk aws` CLI to create the repository. ```bash lstk aws ecr create-repository --repository-name sample-ecr-repo ``` The output of this command will contain the `repositoryUri` value that we'll need in the next step: ```bash title="Output" { "repository": { "repositoryArn": "arn:aws:ecr:us-east-1:000000000000:repository/sample-ecr-repo", "registryId": "000000000000", "repositoryName": "sample-ecr-repo", "repositoryUri": "localhost.localstack.cloud:4510/sample-ecr-repo", "createdAt": "", "imageTagMutability": "MUTABLE", "imageScanningConfiguration": { "scanOnPush": false }, "encryptionConfiguration": { "encryptionType": "AES256" } } } ``` Copy the `repositoryUri` value from the output and replace `` in the following command: ```bash docker build -t . ``` This command will build the Docker image for our NGINX web server. After the build is complete, we'll push the Docker image to the ECR repository we created earlier using the following command: ```bash docker push ``` After a few seconds, the Docker image will be pushed to the local ECR repository. We can now create an ECS cluster and deploy our NGINX web server. ## Creating the local ECS infrastructure LocalStack enables the deployment of ECS task definitions, services, and tasks, allowing us to deploy our ECR containers via the ECS Fargate launch type, which uses the local Docker engine to deploy containers locally. To create the necessary ECS infrastructure on our local machine before deploying our NGINX web server, we will use a CloudFormation template. You can create a new file named `ecs.infra.yml` inside a new `templates` directory, using a [publicly available CloudFormation template as a starting point](https://github.com/aws-cloudformation/aws-cloudformation-templates/blob/main/ECS/FargateLaunchType/services/public-service.yaml). To begin, we'll add the `Mappings` section and configure the subnet mask values, which define the range of internal IP addresses that can be assigned. ```yaml AWSTemplateFormatVersion: '2010-09-09' Description: A stack for deploying containerized applications in AWS Fargate. This stack runs containers in a public VPC subnet, and includes a public facing load balancer to register the services in. Mappings: SubnetConfig: VPC: CIDR: '10.0.0.0/16' PublicOne: CIDR: '10.0.2.0/24' PublicTwo: CIDR: '10.0.3.0/24' ``` Let us now declaratively create the VPC, subnets, ECS cluster, and more: ```yaml Resources: VPC: Type: AWS::EC2::VPC Properties: EnableDnsSupport: true EnableDnsHostnames: true CidrBlock: !FindInMap ['SubnetConfig', 'VPC', 'CIDR'] PublicSubnetOne: Type: AWS::EC2::Subnet Properties: AvailabilityZone: us-east-1a VpcId: !Ref 'VPC' CidrBlock: !FindInMap ['SubnetConfig', 'PublicOne', 'CIDR'] MapPublicIpOnLaunch: true PublicSubnetTwo: Type: AWS::EC2::Subnet Properties: AvailabilityZone: us-east-1b VpcId: !Ref 'VPC' CidrBlock: !FindInMap ['SubnetConfig', 'PublicTwo', 'CIDR'] MapPublicIpOnLaunch: true InternetGateway: Type: AWS::EC2::InternetGateway GatewayAttachment: Type: AWS::EC2::VPCGatewayAttachment Properties: VpcId: !Ref 'VPC' InternetGatewayId: !Ref 'InternetGateway' PublicRouteTable: Type: AWS::EC2::RouteTable Properties: VpcId: !Ref 'VPC' PublicRoute: Type: AWS::EC2::Route DependsOn: GatewayAttachment Properties: RouteTableId: !Ref 'PublicRouteTable' DestinationCidrBlock: '0.0.0.0/0' GatewayId: !Ref 'InternetGateway' PublicSubnetOneRouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: SubnetId: !Ref PublicSubnetOne RouteTableId: !Ref PublicRouteTable PublicSubnetTwoRouteTableAssociation: Type: AWS::EC2::SubnetRouteTableAssociation Properties: SubnetId: !Ref PublicSubnetTwo RouteTableId: !Ref PublicRouteTable ECSCluster: Type: AWS::ECS::Cluster FargateContainerSecurityGroup: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Access to the Fargate containers VpcId: !Ref 'VPC' EcsSecurityGroupIngressFromPublicALB: Type: AWS::EC2::SecurityGroupIngress Properties: Description: Ingress from the public ALB GroupId: !Ref 'FargateContainerSecurityGroup' IpProtocol: -1 SourceSecurityGroupId: !Ref 'PublicLoadBalancerSG' EcsSecurityGroupIngressFromSelf: Type: AWS::EC2::SecurityGroupIngress Properties: Description: Ingress from other containers in the same security group GroupId: !Ref 'FargateContainerSecurityGroup' IpProtocol: -1 SourceSecurityGroupId: !Ref 'FargateContainerSecurityGroup' PublicLoadBalancerSG: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Access to the public facing load balancer VpcId: !Ref 'VPC' SecurityGroupIngress: # Allow access to ALB from anywhere on the internet - CidrIp: 0.0.0.0/0 IpProtocol: -1 FromPort: 9000 ToPort: 9010 PublicLoadBalancer: Type: AWS::ElasticLoadBalancingV2::LoadBalancer Properties: Scheme: internet-facing LoadBalancerAttributes: - Key: idle_timeout.timeout_seconds Value: '30' Subnets: - !Ref PublicSubnetOne - !Ref PublicSubnetTwo SecurityGroups: [!Ref 'PublicLoadBalancerSG'] DummyTargetGroupPublic: Type: AWS::ElasticLoadBalancingV2::TargetGroup Properties: HealthCheckIntervalSeconds: 6 HealthCheckPath: / HealthCheckProtocol: HTTP HealthCheckTimeoutSeconds: 5 HealthyThresholdCount: 2 Name: !Join ['-', [!Ref 'AWS::StackName', 'drop-1']] Port: 80 Protocol: HTTP UnhealthyThresholdCount: 2 VpcId: !Ref 'VPC' PublicLoadBalancerListener: Type: AWS::ElasticLoadBalancingV2::Listener DependsOn: - PublicLoadBalancer Properties: DefaultActions: - TargetGroupArn: !Ref 'DummyTargetGroupPublic' Type: 'forward' LoadBalancerArn: !Ref 'PublicLoadBalancer' Port: 80 Protocol: HTTP ECSRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Effect: Allow Principal: Service: [ecs.amazonaws.com] Action: ['sts:AssumeRole'] Path: / Policies: - PolicyName: ecs-service PolicyDocument: Statement: - Effect: Allow Action: - 'ec2:AttachNetworkInterface' - 'ec2:CreateNetworkInterface' - 'ec2:CreateNetworkInterfacePermission' - 'ec2:DeleteNetworkInterface' - 'ec2:DeleteNetworkInterfacePermission' - 'ec2:Describe*' - 'ec2:DetachNetworkInterface' - 'elasticloadbalancing:DeregisterInstancesFromLoadBalancer' - 'elasticloadbalancing:DeregisterTargets' - 'elasticloadbalancing:Describe*' - 'elasticloadbalancing:RegisterInstancesWithLoadBalancer' - 'elasticloadbalancing:RegisterTargets' Resource: '*' ECSTaskExecutionRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Statement: - Effect: Allow Principal: Service: [ecs-tasks.amazonaws.com] Action: ['sts:AssumeRole'] Path: / Policies: - PolicyName: AmazonECSTaskExecutionRolePolicy PolicyDocument: Statement: - Effect: Allow Action: - 'ecr:GetAuthorizationToken' - 'ecr:BatchCheckLayerAvailability' - 'ecr:GetDownloadUrlForLayer' - 'ecr:BatchGetImage' - 'logs:CreateLogStream' - 'logs:PutLogEvents' Resource: '*' ``` So far, we have set up the VPC where the containers will be networked and created networking resources for the public subnets. We have also added a security group for the container running in Fargate and an IAM role that authorizes ECS to manage resources in the VPC. Next, we can configure the outputs generated by the CloudFormation template. These outputs are values generated during the creation of the CloudFormation stack and can be used by other resources or scripts in your application. To export the values as CloudFormation outputs, we can add the following to the end of our `ecs.infra.yml` file: ```yaml Outputs: ClusterName: Description: The name of the ECS cluster Value: !Ref 'ECSCluster' Export: Name: !Join [ ':', [ !Ref 'AWS::StackName', 'ClusterName' ] ] ExternalUrl: Description: The url of the external load balancer Value: !Join ['', ['http://', !GetAtt 'PublicLoadBalancer.DNSName']] Export: Name: !Join [ ':', [ !Ref 'AWS::StackName', 'ExternalUrl' ] ] ECSRole: Description: The ARN of the ECS role Value: !GetAtt 'ECSRole.Arn' Export: Name: !Join [ ':', [ !Ref 'AWS::StackName', 'ECSRole' ] ] ECSTaskExecutionRole: Description: The ARN of the ECS role Value: !GetAtt 'ECSTaskExecutionRole.Arn' Export: Name: !Join [ ':', [ !Ref 'AWS::StackName', 'ECSTaskExecutionRole' ] ] PublicListener: Description: The ARN of the public load balancer's Listener Value: !Ref PublicLoadBalancerListener Export: Name: !Join [ ':', [ !Ref 'AWS::StackName', 'PublicListener' ] ] VPCId: Description: The ID of the VPC that this stack is deployed in Value: !Ref 'VPC' Export: Name: !Join [ ':', [ !Ref 'AWS::StackName', 'VPCId' ] ] PublicSubnetOne: Description: Public subnet one Value: !Ref 'PublicSubnetOne' Export: Name: !Join [ ':', [ !Ref 'AWS::StackName', 'PublicSubnetOne' ] ] PublicSubnetTwo: Description: Public subnet two Value: !Ref 'PublicSubnetTwo' Export: Name: !Join [ ':', [ !Ref 'AWS::StackName', 'PublicSubnetTwo' ] ] FargateContainerSecurityGroup: Description: A security group used to allow Fargate containers to receive traffic Value: !Ref 'FargateContainerSecurityGroup' Export: Name: !Join [ ':', [ !Ref 'AWS::StackName', 'FargateContainerSecurityGroup' ] ] ``` To deploy the CloudFormation template we created earlier, use the following command: ```bash lstk aws cloudformation create-stack --stack-name infra --template-body file://templates/ecs.infra.yml ``` Wait until the stack status changes to `CREATE_COMPLETE` by running the following command: ```bash lstk aws cloudformation wait stack-create-complete --stack-name infra ``` You can also check your deployed stack on the LocalStack Web Application by navigating to the [CloudFormation resource browser](https://app.localstack.cloud/resources/cloudformation/stacks). With the ECS infrastructure now in place, we can proceed to deploy our NGINX web server. ## Deploying the ECS service To deploy the ECS service, we'll use another CloudFormation template. You can create a new file named `ecs.sample.yml` in the `templates` directory, based on the [publicly available CloudFormation template](https://github.com/awslabs/aws-cloudformation-templates/blob/master/aws/services/ECS/FargateLaunchType/services/public-service.yml). This template will deploy the ECS service on AWS Fargate and expose it via a public load balancer. Before we proceed, let's declare the parameters for the CloudFormation template: ```yaml AWSTemplateFormatVersion: '2010-09-09' Description: Deploy a service on AWS Fargate, hosted in a public subnet, and accessible via a public load balancer. Parameters: StackName: Type: String Default: infra Description: The name of the parent Fargate networking stack that you created. Necessary to locate and reference resources created by that stack. ServiceName: Type: String Default: nginx Description: A name for the service ImageUrl: Type: String Default: nginx Description: The url of a docker image that contains the application process that will handle the traffic for this service ContainerPort: Type: Number Default: 80 Description: What port number the application inside the docker container is binding to HostPort: Type: Number Default: 45139 Description: What port number the application on the host is binding to ContainerCpu: Type: Number Default: 256 Description: How much CPU to give the container. 1024 is 1 CPU ContainerMemory: Type: Number Default: 512 Description: How much memory in megabytes to give the container Path: Type: String Default: "*" Description: A path on the public load balancer that this service should be connected to. Use * to send all load balancer traffic to this service. Priority: Type: Number Default: 1 Description: The priority for the routing rule added to the load balancer. This only applies if your have multiple services which have been assigned to different paths on the load balancer. DesiredCount: Type: Number Default: 2 Description: How many copies of the service task to run Role: Type: String Default: "" Description: (Optional) An IAM role to give the service's containers if the code within needs to access other AWS resources like S3 buckets, DynamoDB tables, etc Conditions: HasCustomRole: !Not [ !Equals [!Ref 'Role', ''] ] ``` Next, we can define the resources, which includes our task, service, target group, and load balancer rule: ```yaml Resources: TaskDefinition: Type: AWS::ECS::TaskDefinition Properties: Family: !Ref ServiceName Cpu: !Ref ContainerCpu Memory: !Ref ContainerMemory NetworkMode: awsvpc RequiresCompatibilities: - FARGATE ExecutionRoleArn: Fn::ImportValue: !Join [':', [!Ref 'StackName', 'ECSTaskExecutionRole']] TaskRoleArn: Fn::If: - HasCustomRole - !Ref Role - !Ref AWS::NoValue ContainerDefinitions: - Name: !Ref ServiceName Cpu: !Ref ContainerCpu Memory: !Ref ContainerMemory Image: !Ref ImageUrl PortMappings: - ContainerPort: !Ref ContainerPort HostPort: !Ref HostPort Service: Type: AWS::ECS::Service DependsOn: LoadBalancerRule Properties: ServiceName: !Ref 'ServiceName' Cluster: Fn::ImportValue: !Join [':', [!Ref 'StackName', 'ClusterName']] LaunchType: FARGATE DeploymentConfiguration: MaximumPercent: 200 MinimumHealthyPercent: 75 DesiredCount: !Ref 'DesiredCount' NetworkConfiguration: AwsvpcConfiguration: AssignPublicIp: ENABLED SecurityGroups: - Fn::ImportValue: !Join [':', [!Ref 'StackName', 'FargateContainerSecurityGroup']] Subnets: - Fn::ImportValue: !Join [':', [!Ref 'StackName', 'PublicSubnetOne']] - Fn::ImportValue: !Join [':', [!Ref 'StackName', 'PublicSubnetTwo']] TaskDefinition: !Ref 'TaskDefinition' LoadBalancers: - ContainerName: !Ref 'ServiceName' ContainerPort: !Ref 'ContainerPort' TargetGroupArn: !Ref 'TargetGroup' TargetGroup: Type: AWS::ElasticLoadBalancingV2::TargetGroup Properties: HealthCheckIntervalSeconds: 6 HealthCheckPath: / HealthCheckProtocol: HTTP HealthCheckTimeoutSeconds: 5 HealthyThresholdCount: 2 TargetType: ip Name: !Ref 'ServiceName' Port: !Ref 'ContainerPort' Protocol: HTTP UnhealthyThresholdCount: 2 VpcId: Fn::ImportValue: !Join [':', [!Ref 'StackName', 'VPCId']] LoadBalancerRule: Type: AWS::ElasticLoadBalancingV2::ListenerRule Properties: Actions: - TargetGroupArn: !Ref 'TargetGroup' Type: 'forward' Conditions: - Field: path-pattern Values: [!Ref 'Path'] ListenerArn: Fn::ImportValue: !Join [':', [!Ref 'StackName', 'PublicListener']] Priority: !Ref 'Priority' ``` Next, let's deploy the CloudFormation template by running the following command: ```bash lstk aws cloudformation create-stack --stack-name ecs --template-body file://templates/ecs.sample.yml --parameters ParameterKey=ImageUrl,ParameterValue= ``` Replace `` with the URI of the Docker image that you want to deploy. Wait for the stack to be created by running the following command: ```bash lstk aws cloudformation wait stack-create-complete --stack-name ecs ``` Now that the ECS service has been deployed successfully, let's access the application endpoint. First, let's list all the ECS clusters we have deployed in our local environment by running the following command to retrieve the cluster ARN: ```bash lstk aws ecs list-clusters | jq -r '.clusterArns[0]' ``` Save the output of the above command as `CLUSTER_ARN`, as we will use it to list the tasks running in the cluster. Next, run the following command to list the task ARN: ```bash lstk aws ecs list-tasks --cluster | jq -r '.taskArns[0]' ``` Save the task ARN as `TASK_ARN`. Let us now list the port number on which the application is running. Run the following command: ```bash lstk aws ecs describe-tasks --cluster --tasks | jq -r '.tasks[0].containers[0].networkBindings[0].hostPort' ``` Earlier, we configured the application to run on port `45139`, in our `HostPort` parameter. Let us now access the application endpoint. Run the following command to get the public IP address of the host: ```bash curl localhost:45139 ``` Alternatively, in the address bar of your web browser, you can navigate to [`localhost:45139`](https://localhost:45139/). You should see the default index page of the NGINX web server. ## Conclusion In this tutorial, we have demonstrated how to deploy a containerized service locally using Amazon ECS, ECR, and LocalStack. We have also shown how you can use CloudFormation templates with the `lstk aws` CLI to deploy your local AWS infrastructure. With LocalStack, you can easily mount code from your host filesystem into the ECS container, allowing for a quicker debugging loop that doesn't require rebuilding and redeploying the task's Docker image for each change. To try out this tutorial for yourself, you can find the code in our [`localstack-pro-samples` repository](https://github.com/localstack/localstack-pro-samples/tree/master/ecs-ecr-container-app) over GitHub, including a `Makefile` to execute each step of the process. # Setting up Elastic Load Balancing (ELB) Application Load Balancers using LocalStack, deployed via the Serverless framework > Learn how to configure Elastic Load Balancing (ELB) Application Load Balancers and set up Node.js Lambda functions as targets. This tutorial demonstrates how to forward requests to the target group for your Lambda function using the Serverless Framework and the `serverless-localstack` plugin to effortlessly deploy and manage your infrastructure locally with LocalStack. ## Introduction [Elastic Load Balancer (ELB)](https://aws.amazon.com/elasticloadbalancing/)is a service that distributes incoming application traffic across multiple targets, such as EC2 instances, containers, IP addresses, and Lambda functions. ELBs can be physical hardware or virtual software components. They accept incoming traffic and distribute it across multiple targets in one or more Availability Zones. Using ELB, you can quickly scale your load balancer to accommodate changes in traffic over time, ensuring optimal performance for your application and workloads running on the AWS infrastructure. ELB provides four types of load balancers: - **[Application Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/introduction.html)**: Manages HTTP/HTTPS traffic, offering advanced routing features at the application layer. - **[Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html)**: Handles TCP traffic with high performance and low latency at the transport layer. - **[Gateway Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/gateway/introduction.html)**: Deploys, scales, and manages third-party virtual appliances with a transparent network gateway. - **[Classic Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/introduction.html)**: Provides basic load balancing for both HTTP/HTTPS and TCP traffic. In this tutorial, we focus on the Application Load Balancer (ALB), which operates at Layer 7 (Application layer) of the OSI model and is specifically designed for load balancing HTTP and HTTPS traffic for web applications. ALB works at the request level, allowing advanced load-balancing features for HTTP and HTTPS requests. It also enables you to register Lambda functions as targets. You can configure a listener rule that forwards requests to a target group for your Lambda function, triggering its execution to process the request. [LocalStack for AWS](https://localstack.cloud) extends support for ELB Application Load Balancers and the configuration of target groups, including Lambda functions. This tutorial will guide you through setting up an ELB Application Load Balancer to configure Node.js Lambda functions as targets. We will utilize the [Serverless framework](http://serverless.com/) along with the [`serverless-localstack` plugin](https://www.serverless.com/plugins/serverless-localstack) to simplify the setup. Additionally, we will demonstrate how to set up ELB endpoints to efficiently forward requests to the target group associated with your Lambda functions. ## Prerequisites - LocalStack for AWS - [Serverless framework](https://www.serverless.com/framework/docs/getting-started/) - [Node.js & `npm`](https://nodejs.org/en/download/) - [`lstk aws`](/aws/connecting/aws-cli#localstack-aws-cli-lstk-aws) - [curl](https://curl.se/) and [jq](https://jqlang.github.io/jq/) ## Architecture The architecture emulates a scalable AWS setup locally: Clients send HTTP/HTTPS requests to the ALB's DNS endpoint. The ALB listener (e.g., on port 80) routes traffic based on path rules to target groups, which forward to registered Lambda functions. These functions process requests and return responses. The setup runs within a VPC and subnet for networking isolation, all emulated in LocalStack. ### Key components - Clients/Users: Initiate traffic via browsers or tools like curl. - ALB Listener: Receives and routes based on rules (e.g., /hello1 → hello1 Lambda). - Target Group: Manages health checks and forwards to Lambda targets. - Lambda Functions: Handle business logic (e.g., return "Hello 1"). - VPC/Subnet: Provides network boundaries. ![Architecture diagram for ELB Application Load Balancer with Lambda targets](/src/assets/images/aws/tutorials/elb-load-balancing-architecture-image.png) ## Setup a Serverless project Serverless is an open-source framework that enables you to build, package, and deploy serverless applications seamlessly across various cloud providers and platforms. With the Serverless framework, you can easily set up your serverless development environment, define your applications as functions and events, and deploy your entire infrastructure to the cloud using a single command. To start using the Serverless framework, install the Serverless framework globally by executing the following command using `npm`: ```bash npm install -g serverless ``` The above command installs the Serverless framework globally on your machine. After the installation is complete, you can verify it by running the following command: ```bash serverless --version Framework Core: 3.24.1 Plugin: 6.2.2 SDK: 4.3.2 ``` This command displays the version numbers of the Serverless framework's core, plugins, and SDK you installed. Now, let's proceed with creating a new Serverless project using the `serverless` command: ```bash serverless create --template aws-nodejs --path serverless-elb ``` In this example, we use the `aws-nodejs` template to create our Serverless project. This template includes a simple Node.js Lambda function that returns a message when invoked. It also generates a `serverless.yml` file that contains the project's configuration. The `serverless.yml` file is where you configure your project. It includes information such as the service name, the provider (AWS in this case), the functions, and example events that trigger those functions. If you prefer to set up your project using a different template, refer to the [Serverless templates documentation](https://www.serverless.com/framework/docs/providers/aws/cli-reference/create/) for more options. Now that we have created our Serverless project, we can proceed to configure it to use LocalStack. ## Configure Serverless project to use LocalStack To configure your Serverless project to use LocalStack, you need to install the `serverless-localstack` plugin. Before that, let's initialize the project and install some dependencies: ```bash npm init -y npm install -D serverless serverless-localstack serverless-deployment-bucket ``` In the above commands, we use `npm init -y` to initialize a new Node.js project with default settings and then install the necessary dependencies, including `serverless`, `serverless-localstack`, and `serverless-deployment-bucket`, as dev dependencies. The `serverless-localstack` plugin enables your Serverless project to redirect AWS API calls to LocalStack, while the `serverless-deployment-bucket` plugin creates a deployment bucket in LocalStack. This bucket is responsible for storing the deployment artifacts and ensuring that old deployment buckets are properly cleaned up after each deployment. We have a `serverless.yml` file in the directory to define our Serverless project's configuration, which includes information such as the service name, the provider (AWS in this case), the functions, and example events that trigger those functions. To set up the plugins we installed earlier, you need to add the following properties to your `serverless.yml` file: ```yaml showLineNumbers service: serverless-elb frameworkVersion: '3' provider: name: aws runtime: nodejs12.x functions: hello: handler: handler.hello plugins: - serverless-deployment-bucket - serverless-localstack custom: localstack: stages: - local ``` To configure Serverless to use the LocalStack plugin specifically for the `local` stage and ensure that your Serverless project only deploys to LocalStack instead of the real AWS Cloud, you need to set the `--stage` flag when using the `serverless deploy` command and specify the flag variable as `local`. Configure a `deploy` script in your `package.json` file to simplify the deployment process. It lets you run the `serverless deploy` command directly over your local infrastructure. Update your `package.json` file to include the following: ```json showLineNumbers { "name": "serverless-elb", "version": "1.0.0", "description": "", "main": "handler.js", "scripts": { "deploy": "sls deploy --stage local" }, "keywords": [], "author": "", "license": "ISC", "devDependencies": { "serverless": "^3.25.0", "serverless-deployment-bucket": "^1.6.0", "serverless-localstack": "^1.0.1" } } ``` With this configuration, you can now run the deployment script using: ```bash npm run deploy ``` This will execute the `serverless deploy --stage local` command, deploying your Serverless project to LocalStack. ## Create Lambda functions & ELB Application Load Balancers Now, let's create two Lambda functions named `hello1` and `hello2` that will run on the Node.js 12.x runtime. Open the `handler.js` file and replace the existing code with the following: ```js showLineNumbers 'use strict'; module.exports.hello1 = async (event) => { console.log(event); return { "isBase64Encoded": false, "statusCode": 200, "statusDescription": "200 OK", "headers": { "Content-Type": "text/plain" }, "body": "Hello 1" }; }; module.exports.hello2 = async (event) => { console.log(event); return { "isBase64Encoded": false, "statusCode": 200, "statusDescription": "200 OK", "headers": { "Content-Type": "text/plain" }, "body": "Hello 2" }; }; ``` We have defined the `hello1` and `hello2` Lambda functions in the updated code. Each function receives an event parameter and logs it to the console. The function then returns a response with a status code of 200 and a plain text body containing the respective `"Hello"` message. It's important to note that the `isBase64Encoded` property is not required for plain text responses. It is typically used when you need to include binary content in the response body and want to indicate that the content is Base64 encoded. Let us now configure the `serverless.yml` file to create an Application Load Balancer (ALB) and attach the Lambda functions to it. ```yaml showLineNumbers service: serverless-elb provider: name: aws runtime: nodejs12.x deploymentBucket: name: testbucket functions: hello1: handler: handler.hello1 events: - alb: listenerArn: !Ref HTTPListener priority: 1 conditions: path: /hello1 hello2: handler: handler.hello2 events: - alb: listenerArn: !Ref HTTPListener priority: 2 conditions: path: /hello2 plugins: - serverless-deployment-bucket - serverless-localstack custom: localstack: stages: - local ``` In the above configuration, we specify the service name (`serverless-elb` in this case) and set the provider to AWS with the Node.js 12.x runtime. We include the necessary plugins, `serverless-localstack` and `serverless-deployment-bucket`, for LocalStack support and deployment bucket management. Next, we define the `hello1` and `hello2` functions with their respective handlers and event triggers. In this example, both functions are triggered by HTTP GET requests to the `/hello1` and `/hello2` paths. Lastly, let's create a VPC, a subnet, an Application Load Balancer, and an HTTP listener on the load balancer that redirects traffic to the target group. To do this, add the following resources to your `serverless.yml` file: ```yaml showLineNumbers ... resources: Resources: LoadBalancer: Type: AWS::ElasticLoadBalancingV2::LoadBalancer Properties: Name: lb-test-1 Subnets: - !Ref Subnet HTTPListener: Type: AWS::ElasticLoadBalancingV2::Listener Properties: DefaultActions: - Type: redirect RedirectConfig: Protocol: HTTPS Port: 443 Host: "#{host}" LoadBalancerArn: !Ref LoadBalancer Protocol: HTTP Subnet: Type: AWS::EC2::Subnet Properties: VpcId: !Ref VPC CidrBlock: 12.2.1.0/24 AvailabilityZone: !Select - 0 - Fn::GetAZs: !Ref "AWS::Region" VPC: Type: AWS::EC2::VPC Properties: EnableDnsSupport: "true" EnableDnsHostnames: "true" CidrBlock: 12.2.1.0/24 ``` You have completed the configuration of your Serverless project! Now you can create your local AWS infrastructure on LocalStack and deploy your Application Load Balancers with the two Lambda functions as targets. ## Creating the infrastructure on LocalStack Now that we have completed the initial setup, let's run LocalStack's AWS emulation on our local machine. Start LocalStack by running the following command: ```bash lstk start ``` This command launches LocalStack in the background, enabling you to use the AWS services locally. Now, let's deploy our Serverless project and verify the resources created in LocalStack. Run the following command: ```bash npm run deploy ``` This command deploys your Serverless project using the "local" stage. The output will resemble the following: ```bash > serverless-elb@1.0.0 deploy > sls deploy --stage local Using serverless-localstack Deploying test-elb-load-balancing to stage local (us-east-1) Creating deployment bucket 'testbucket'... Using deployment bucket 'testbucket' Skipping template validation: Unsupported in Localstack ✔ Service deployed to stack test-elb-load-balancing-local (15s) functions: hello1: test-elb-load-balancing-local-hello1 (157 kB) hello2: test-elb-load-balancing-local-hello2 (157 kB) ``` This output confirms the successful deployment of your Serverless service to the `local` stage in LocalStack. It also displays information about the deployed Lambda functions (`hello1` and `hello2`). You can run the following command to verify that the functions and the load balancers have been deployed: ```bash showLineNumbers lstk aws lambda list-functions { "Functions": [ { "FunctionName": "test-elb-load-balancing-local-hello1", "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:test-elb-load-balancing-local-hello1", "Runtime": "nodejs12.x", "Role": "arn:aws:iam::000000000000:role/test-elb-load-balancing-local-us-east-1-lambdaRole", "Handler": "handler.hello1", ... }, { "FunctionName": "test-elb-load-balancing-local-hello2", "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:test-elb-load-balancing-local-hello2", "Runtime": "nodejs12.x", "Role": "arn:aws:iam::000000000000:role/test-elb-load-balancing-local-us-east-1-lambdaRole", "Handler": "handler.hello2", ... } ] } lstk aws elbv2 describe-load-balancers { "LoadBalancers": [ { "LoadBalancerArn": "arn:aws:elasticloadbalancing:us-east-1:000000000000:loadbalancer/app/lb-test-1/", "DNSName": "lb-test-1.elb.localhost.localstack.cloud", "CanonicalHostedZoneId": "", "CreatedTime": "", "LoadBalancerName": "lb-test-1", "Scheme": "None", ... } ] } ``` The ALB endpoints for the two Lambda functions, hello1 and hello2, are accessible at the following URLs: - [`http://lb-test-1.elb.localhost.localstack.cloud:4566/hello1`](http://lb-test-1.elb.localhost.localstack.cloud:4566/hello1) - [`http://lb-test-1.elb.localhost.localstack.cloud:4566/hello2`](http://lb-test-1.elb.localhost.localstack.cloud:4566/hello2) ## Testing Here in the testing phase we will test endpoints, do a validation check which includes health check and error handling. To test these endpoints, you can use the curl command along with the jq tool for better formatting. 1. **Verify Deployment:** Use the commands `lstk aws lambda list-functions` and `lstk aws elbv2 describe-load-balancers` respectively to confirm the existince of Lambda and ALB respectively 2. **Test Endpoints:** Run the following commands: ```bash curl http://lb-test-1.elb.localhost.localstack.cloud:4566/hello1 | jq "Hello 1" curl http://lb-test-1.elb.localhost.localstack.cloud:4566/hello2 | jq "Hello 2" ``` Both commands send an HTTP GET request to the endpoints and uses `jq` to format the response. The expected outputs are `Hello 1` & `Hello 2`, representing the Lambda functions' response. 3. **Health Checks:** Describe target health: ```bash lstk aws elbv2 describe-target-health --target-group-arn $(lstk aws elbv2 describe-target-groups --load-balancer-arn $(lstk aws elbv2 describe-load-balancers --names lb-test-1 --query 'LoadBalancers[0].LoadBalancerArn' --output text) --query 'TargetGroups[0].TargetGroupArn' --output text) ``` 4. **Invalid Path:** Test fallback/redirect: ```bash curl -I http://lb-test-1.elb.localhost.localstack.cloud:4566/invalid ``` 5. **Logs Validation:** Check Lambda logs for invocations: ```bash lstk aws logs describe-log-groups --query 'logGroups[].logGroupName' | jq -r '.[] | select(contains("hello1"))' | xargs -I {} lstk aws logs tail {} --follow ``` If tests fail, you can ensure LocalStack is healthy (`lstk status`), check ports (default `4566`), and restart if needed. ## Conclusion In this tutorial, we have learned how to create an Application Load Balancer (ALB) with two Lambda functions as targets using LocalStack. We have also explored creating, configuring, and deploying a Serverless project with LocalStack, enabling developers to develop and test Cloud and Serverless applications locally without AWS costs—accelerating iteration for cloud-native workloads. LocalStack for AWS offers integrations with various popular tools such as Terraform, Pulumi, Serverless Application Model (SAM), and more. For more information about integrations, you can refer to our [Integrations documentation](https://docs.localstack.cloud/aws/customization/integrations). To further explore and experiment with the concepts covered in this tutorial, you can access the code and resources on our [`localstack-pro-samples` repository on GitHub](https://github.com/localstack/localstack-pro-samples/tree/master/elb-load-balancing) along with a Makefile for step-by-step execution. # Creating ephemeral application previews with LocalStack and GitHub Actions > Learn how to use LocalStack's Ephemeral Instances to generate application previews for your cloud applications using GitHub Actions. This tutorial will guide you through deploying a full-stack serverless application using Lambda, DynamoDB, API Gateway, S3, and CloudFront, and serving it on a short-lived, encapsulated deployment generated for every new pull request. ## Introduction LocalStack's core cloud emulator allows you to set up your cloud infrastructure on your local machine. You can access databases, queues, and other managed services without needing to connect to a remote cloud provider. This speeds up your Software Development Life Cycle (SDLC) by making development and testing more efficient. Despite this, you still need a staging environment to do final acceptance tests before deploying your application to production. In many cases, staging environments are costly and deploying changes to them takes a lot of time. Also, teams can only use one staging environment at a time, which makes it difficult to test changes quickly. With LocalStack's [Ephemeral Instances](/aws/developer-tools/cloud-sandbox/ephemeral-instances/), you can create short-lived, self-contained deployments of LocalStack in the cloud. These Ephemeral Instances also let you deploy your application on a remote LocalStack container, creating an [Application Preview](/aws/developer-tools/cloud-sandbox/app-preview/). This allows you to run end-to-end tests, preview features, and collaborate within your team or across teams asynchronously. This tutorial will show you how to use LocalStack's Ephemeral Instance feature to generate an Application Preview automatically for every new Pull Request (PR) using a GitHub Action workflow. :::note Ephemeral Instances are not supported by [`lstk`](/aws/developer-tools/running-localstack/lstk/). This tutorial instead uses the [legacy LocalStack CLI](/aws/developer-tools/running-localstack/localstack-cli/). ::: ## Architecture diagram of the preview flow ![Ephemeral Previews Flow](/images/aws/empheral_previews_flow.png) This diagram illustrates the ephemeral preview flow, where a GitHub Pull Request triggers a GitHub Actions workflow that automatically deploys the backend and frontend resources to a temporary LocalStack instance. The ephemeral instance generates a shareable application preview URL, allowing team members to test and validate the application in a production like environment. Once the Pull Request is closed or merged, the ephemeral instance is automatically shut down, ensuring no unnecessary costs are incurred. ## Prerequisites - [LocalStack Web Application account](https://app.localstack.cloud/) - [GitHub Account](https://github.com/join) & [`gh` CLI](https://github.com/cli/cli?tab=readme-ov-file#installation) (optional) ## Tutorial: Setting up Application Previews for your cloud application This tutorial uses a [public LocalStack sample](https://github.com/localstack-samples/sample-notes-app-dynamodb-lambda-apigateway) to showcase a simple note-taking application using the modular AWS SDK for JavaScript. The example application deploys several AWS resources including DynamoDB, Lambda, API Gateway, S3, Cognito, and CloudFront, functioning as follows: - Five Lambda functions handle basic CRUD functionality around note entities. - The frontend is built with React and served via Cloudfront and an S3 bucket. - DynamoDB is used as a persistence layer to store the notes. - API Gateway exposes the Lambda functions through HTTP APIs. - A Cognito User Pool is used for Authentication and Authorization. This tutorial guides you through setting up a GitHub Action workflow to create an Application Preview of the sample application by deploying it on an ephemeral instance. ### Create the GitHub Action workflow GitHub Actions serves as a continuous integration and continuous delivery (CI/CD) platform, automating software development workflows directly from GitHub. It allows customization of actions and automation throughout the software development lifecycle. In this tutorial, you'll implement a workflow that: - Checks out the repository from GitHub. - Installs necessary dependencies. - Deploys the application on a ephemeral LocalStack Instance using a GitHub Action Runner to generate a sharable application preview. To begin, fork the [LocalStack sample repository](https://github.com/localstack-samples/sample-notes-app-dynamodb-lambda-apigateway) on GitHub. If you're using GitHub's `gh` CLI, fork and clone the repository with this command: ```bash gh repo fork https://github.com/localstack-samples/sample-notes-app-dynamodb-lambda-apigateway ``` After forking and cloning, navigate to the `.github/workflows` directory in your forked repository and open the `preview.yml` file. This file will contain the GitHub Action workflow configuration. Now you're set to create your GitHub Action workflow, which will deploy your cloud application on an ephemeral instance using LocalStack. ### Set Up the Actions & dependencies To achieve the goal, you can utilize a few prebuilt Actions: - [`actions/checkout`](https://github.com/actions/checkout): Checkout the application code with Git. - [`setup-localstack/ephemeral/startup`](https://github.com/localstack/setup-localstack): Configure the workflow to generate the application preview. - [`LocalStack/setup-localstack/finish`](https://github.com/localstack/setup-localstack): Add a comment to the PR, which includes a URL to the application preview. You will find the following content to the `preview.yml` file that you opened earlier: ```yaml name: Create PR Preview on: pull_request: types: [opened, synchronize, reopened] ``` This configuration ensures that every time a pull request is raised, the action is triggered. A new job named `preview` specifies the GitHub-hosted runner to execute our workflow steps, while also checking out the code we need to deploy to the application preview instance: ```yaml jobs: preview: permissions: write-all runs-on: ubuntu-latest timeout-minutes: 15 steps: - name: Checkout uses: actions/checkout@v4 ``` ### Deploy the application preview To deploy the application preview, you can utilize the `LocalStack/setup-localstack/ephemeral/startup` action, which requires the following parameters: - `github-token`: Automatically configured on the GitHub Action runner. - `localstack-api-key`: Configuration of a LocalStack [CI key](https://app.localstack.cloud/workspace/ci-keys) (`LOCALSTACK_API_KEY`) to activate licensed features in LocalStack (Note: You may need administrator permission to access creating new CI keys). - `preview-cmd`: The set of commands necessary to deploy the application, including its infrastructure, on LocalStack. The following step sets up the dependencies and deploys the application preview on an ephemeral LocalStack instance: ```yaml - name: Deploy Preview uses: LocalStack/setup-localstack/ephemeral/startup@v0.2.2 with: github-token: ${{ secrets.GITHUB_TOKEN }} localstack-api-key: ${{ secrets.LOCALSTACK_API_KEY }} preview-cmd: | npm install -g aws-cdk-local aws-cdk pip install awscli-local[ver1] make build make bootstrap make deploy make prepare-frontend-local make build-frontend make bootstrap-frontend make deploy-frontend distributionId=$(awslocal cloudfront list-distributions | jq -r '.DistributionList.Items[0].Id') echo LS_PREVIEW_URL=$AWS_ENDPOINT_URL/cloudfront/$distributionId/ >> $GITHUB_ENV ``` In the provided workflow: - Dependencies such as `awslocal`, AWS CDK library, and the `cdklocal` wrapper are installed. - `Makefile` targets are employed to build the application, bootstrap the CDK stack, and deploy it. - Additionally, the frontend application is built and deployed on an S3 bucket served via a CloudFront distribution. - The application preview URL is provided by querying the CloudFront distribution ID using `awslocal`. To complete the process, the last step attaches the application preview URL to the Pull Request (PR) as a comment. This allows for quick access to the deployed URL for validating features or enhancements pushed to your application. ```yaml - name: Finalize PR comment uses: LocalStack/setup-localstack/finish@v0.2.2 with: github-token: ${{ secrets.GITHUB_TOKEN }} include-preview: true preview-url: ${{ env.PREVIEW_URL }} ``` ### Configure a CI key for GitHub Actions Before triggering your workflow, set up a continuous integration (CI) key for LocalStack. LocalStack requires a CI Key for usage in CI or similar automated environments to activate licensed features. Follow these steps to add your LocalStack CI key to your forked GitHub repository: - Navigate to the [LocalStack Web Application](https://app.localstack.cloud/) and access the [CI Keys](https://app.localstack.cloud/workspace/ci-keys) page. - Scroll down to the **Generate CI Key** card, where you can provide a name, and click **Generate CI Key** to receive a new key. - In your [GitHub repository secrets](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions), set the **Name** as `LOCALSTACK_API_KEY` and the **Secret** as the CI Key. If you do not have access to creating CI keys, contact your LocalStack account administrator. Now, you can commit and push your workflow to your forked GitHub repository. ### Run the GitHub Action workflow Now that the GitHub Action Workflow is set up, each pull request in your cloud application will undergo building, deployment, and packaging as an application preview running within an ephemeral instance. The workflow will automatically update the application preview whenever new commits are pushed to the pull request. ![PR preview comment for every pull request](/images/aws/github-action-pr-preview-comment.png) In case your deployment encounters issues and fails on LocalStack, you can troubleshoot by incorporating additional steps to generate a diagnostics report. After downloading, you can visualize logs and environment variables using a tool like [`diapretty`](https://github.com/silv-io/diapretty): ```yaml - name: Generate a Diagnostic Report if: failure() run: curl -s localhost:4566/_localstack/diagnose | gzip -cf > diagnose.json.gz - name: Upload the Diagnostic Report if: failure() uses: actions/upload-artifact@v4 with: name: diagnose.json.gz path: ./diagnose.json.gz ``` ## **Testing the application** Once the Application Preview is successfully deployed on a LocalStack Ephemeral Instance, you can validate that your cloud application is functioning as expected before merging your Pull Request. Follow the checklist below to verify the preview environment: ### **Testing the application checklist** - **Preview URL Reachability:** Ensure the preview URL added as a comment on your Pull Request is accessible. Open the link to verify that the frontend application loads successfully in your browser. - **Smoke Tests Execution:** Perform basic smoke tests to validate the core functionality of your application. For example, verify that API endpoints respond correctly, CRUD operations succeed, and authentication flows (if applicable) function as intended. - **Backend Resource Validation:** Confirm that key AWS-like resources, such as DynamoDB tables, Lambda functions, and API Gateway endpoints, are deployed and operational within the LocalStack environment. You can use the `awslocal` CLI to inspect resources, for example: ```bash awslocal dynamodb list-tables awslocal lambda list-functions awslocal apigatewayv2 get-apis ``` - **Frontend Verification:** Test that the deployed frontend connects correctly to the backend APIs exposed via LocalStack and that any user interactions (e.g., creating or retrieving data) work as expected. After completing these checks and confirming the application behaves as expected, your preview is considered validated and ready for review or merge. ## Conclusion In this tutorial, you've learned how to utilize LocalStack's Ephemeral Instances to generate application previews for your cloud applications. You can explore additional use cases with Ephemeral Instances, including: - Injecting a pre-defined Cloud Pod into an ephemeral instance to rapidly spin up infrastructure. - Running your automated end-to-end (E2E) test suite to conduct thorough testing before deploying to production. - Enabling collaboration across different teams by offering a pre-production environment for collaborative work. # End-to-End Testing in Gitlab CI with Testcontainers and LocalStack: Understanding Runners and Docker in Docker > In this tutorial, we'll walk through the process of setting up end-to-end testing for a backend application using Testcontainers and LocalStack within GitLab. We'll understand the types of GitLab Runners available for CI pipelines and how the concept of Docker-in-Docker plays a crucial role in this environment. ## Introduction: Testcontainers & LocalStack Testcontainers is an open-source framework that provides lightweight APIs for bootstrapping local development and test dependencies with real services wrapped in Docker containers. Running tests with Testcontainers and LocalStack is crucial for AWS-powered applications because it ensures each test runs in a clean, isolated environment, providing consistency across all development and CI machines. LocalStack avoids AWS costs by emulating services locally, preventing exceeding AWS free tier limits, and eliminates reliance on potentially unstable external AWS services. This allows for the simulation of difficult-to-reproduce scenarios, edge cases, and enables testing of the entire application stack in an integrated manner. Testing with LocalStack and Testcontainers also integrates seamlessly with CI/CD pipelines like GitLab CI or GitHub Actions, allowing developers to run automated tests without requiring AWS credentials or services. ## Prerequisites For this tutorial, you will need: - [LocalStack for AWS](/aws/getting-started/auth-token) to emulate the AWS services. If you don't have a subscription yet, you can just get a trial license for free. - [Docker](https://docker.io/) - [A GitLab account](https://gitlab.com/) ## GitLab overview GitLab is striving to be a complete tool for DevOps practices, offering not just source code management and continuous integration, but also features for monitoring, security, planning, deploying and more. By having your code and CI on the same platform, workflows are simplified and collaboration is enhanced. While Jenkins is still a very prominent CI/CD tool in the industry, it is up to the user to figure out where to host it and focuses solely on CI/CD features. ## GitLab architecture ![GitLab architecture](/images/aws/gitlab-architecture.png) As users, we only interact directly with a GitLab instance which is responsible for hosting the application code and all the needed configurations, including the ones for pipelines. The instance is then in charge of running the pipelines and assigning runners to execute the defined jobs. When running CI pipelines, you can choose to use [**GitLab-hosted runners**](https://docs.gitlab.com/ee/ci/runners/index.html), or provision and register [**self-managed runners**](https://docs.gitlab.com/runner/install/docker.html). This tutorial will cover both. ### Runners hosted by GitLab The GitLab documentation highlights some key aspects about the provided runners: - They can run on Linux, Windows (beta) and MacOS (beta). - They are enabled by default for all projects, with no configuration required. - Each job is executed by a newly provisioned VM. - Job runs have `sudo` access without a password. - VMs are isolated between job executions. - Their storage is shared by the operating system, the image with pre-installed software, and a copy of your cloned repository, meaning that the remaining disk space for jobs will be reduced. - The runners are configured to run in privileged mode to support Docker in Docker to build images natively or run multiple containers within each job. ### Self-hosted runners Essentially, the architecture does not change, except the runners will be executing the jobs on a local machine. For developing locally, this approach is very convenient and there are several benefits: - **Customization**: you can configure the runners to suit your specific needs and environment. - **Performance**: improved performance and faster builds by leveraging your own hardware. - **Security**: enhanced control over your data and build environment, reducing exposure to external threats. - **Resource Management**: better management and allocation of resources to meet your project's demands. - **Cost Efficiency**: depending on your alternatives, you can avoid usage fees associated with cloud-hosted runners. ## Application Overview Our sample backend application stores information about different types of coffee in files, with descriptions stored in an S3 bucket. It utilizes two Lambda functions to create/update and retrieve these descriptions, all accessible through an API Gateway. While we won't delve into the details of creating these AWS resources, we'll use AWS CLI to initialize them during container startup using init hooks. You can find the whole setup in the [init-resources.sh](https://gitlab.com/tinyg210/coffee-backend-localstack/-/blob/main/src/test/resources/init-resources.sh?ref_type=heads) file. The following diagram visually explains the simple workflows that we want to check in our automated test in CI, using Testcontainers. We'll need to make sure that the files are correctly created and named, that the validations and exceptions happen as expected. ![Coffee app diagram](/images/aws/coffee-app-diagram.png) ## CI Pipeline Using GitLab Runners ### Configuring the test container To follow along, make changes to the code or run your own pipelines, you may fork the repository from the [coffee-backend-localstack sample](https://gitlab.com/tinyg210/coffee-backend-localstack). The application is developed, built and tested locally, the next step is to establish a quality gate in the pipeline, to make sure nothing breaks. The basis for the container used for testing looks like this: ```java @Testcontainers public class LocalStackConfig { @Container protected static LocalStackContainer localStack = new LocalStackContainer(DockerImageName.parse("localstack/localstack-pro:3.4.0")) .withEnv("LOCALSTACK_AUTH_TOKEN", System.getenv("LOCALSTACK_AUTH_TOKEN")) .withCopyFileToContainer( MountableFile.forHostPath("target/lambda.jar", 0777), "/etc/localstack/init/ready.d/target/lambda.jar") .withCopyFileToContainer( MountableFile.forClasspathResource("init-resources.sh", 0777), "/etc/localstack/init/ready.d/init-resources.sh") .withEnv("DEBUG", "1") .withNetworkAliases("localstack") .withEnv("LAMBDA_DOCKER_FLAGS", testcontainersLabels()) .withEnv("LAMBDA_RUNTIME_ENVIRONMENT_TIMEOUT", "90") .waitingFor(Wait.forLogMessage(".*Finished creating resources.*\\n", 1)); ...... ``` Here's a breakdown of what's important: - The `@Testcontainers` marks the test class to use the Testcontainers library. - The `@Container` annotation indicates that the field is a Testcontainers managed container. - The image used for the test LocalStack instance is set to the latest Pro version (at the time of writing). - In order to use the Pro image, a `LOCALSTACK_AUTH_TOKEN` variable needs to be set and read from the environment. - There are two files copied to the container before startup: the JAR file for the Lambda functions and the script for provisioning all the necessary AWS resources. Both files are copied with read/write/execute permissions. - `DEBUG=1` enables a more verbose logging of LocalStack. - `LAMBDA_DOCKER_FLAGS` sets specific Testcontainers labels to the Lambda containers, as a solution to be correctly managed by Ryuk. Since the compute containers are created by LocalStack and not the Testcontainers framework, they do not receive the necessary tags. - `LAMBDA_RUNTIME_ENVIRONMENT_TIMEOUT` sets an environment variable to configure the Lambda runtime environment timeout to 90 seconds, for slower environments. - The last line `.waitingFor(Wait.forLogMessage(...))` configures the container to wait until the specified log message appears, exactly once, indicating that resource creation is complete. :::note Ryuk is a component of Testcontainers that helps manage and clean up Docker resources created during testing. Specifically, Ryuk ensures that any Docker containers, networks, volumes, and other resources are properly removed when they are no longer needed. This prevents resource leaks and ensures that the testing environment remains clean and consistent between test runs. When Testcontainers starts, it typically launches a Ryuk container in the background. This container continuously monitors the Docker resources created by Testcontainers and removes them once the test execution is complete or if they are no longer in use. ::: The tests are set up in the `CoffeeAppTests` class, validating the workflows for creating a coffee description files, retrieving them, and exception throwing when needed. For this tutorial you don't really need to dive into the specifics of the tests, but you're more than welcome to. ### Setting up the pipeline configuration The `.gitlab-ci.yml` file is a configuration file for defining GitLab CI/CD pipelines, which automate the process of building, testing, and deploying applications. It specifies stages (such as build, test, and deploy) and the jobs within each stage, detailing the commands to be executed. Jobs can define dependencies, artifacts, and environment variables. Pipelines are triggered by events like code pushes, merge requests, or schedules, and they are executed by runners. This file enables automated, consistent, and repeatable workflows for software development and deployment. In this example we will focus on just the building and testing parts. Let's break down the `.gitlab-ci.yml` for this project: ```yaml image: ubuntu:latest before_script: # install necessary dependencies - apt-get update && apt-get install -y curl tar bash maven docker.io - curl -L https://download.oracle.com/java/21/latest/jdk-21_linux-x64_bin.tar.gz -o /tmp/openjdk.tar.gz - tar -xzf /tmp/openjdk.tar.gz -C /opt - mv /opt/jdk-21* /opt/jdk-21 - export JAVA_HOME="/opt/jdk-21" - export PATH="$JAVA_HOME/bin:$PATH" stages: - build - test cache: paths: - .m2/repository - target build_job: stage: build script: - echo "Running build..." - mvn clean package -DskipTests artifacts: paths: - target/ test_job: stage: test services: - docker:26.1.2-dind variables: DOCKER_HOST: tcp://docker:2375 DOCKER_TLS_CERTDIR: "" DOCKER_DRIVER: overlay2 LOCALSTACK_AUTH_TOKEN: $LOCALSTACK_AUTH_TOKEN script: - echo "Running tests..." - mvn test allow_failure: false ``` - `image: ubuntu:latest` - This specifies the base Docker image used for all jobs in the pipeline. `ubuntu:latest` is a popular and easy choice because it's a well-known, stable, and widely-supported Linux distribution. It ensures a consistent environment across all pipeline stages. Each job can define its own image (for example `maven` or `docker` images), but in this case a generic image with the necessary dependencies (curl, Java, maven, docker) installed covers the needs for both stages. - `before_script` - these commands are run before any job script in the pipeline, on top of the Ubuntu image. - The two stages are defined at the top: `build` and `test`. - `cache` - caches the Maven dependencies to speed up subsequent pipeline runs. - `.m2/repository` - this is the default location where Maven stores its local repository of dependencies. - The `script` section - specifies the scripts that run for each job. - `artifacts` - specifies the build artifacts (e.g., JAR files) to be preserved and passed to the next stages (the `target` folder). - The build job runs only on the `main` branch. - `docker:26.1.2-dind` - specifies the service necessary to use Docker-in-Docker to run Docker commands inside the pipeline job. This is useful for integration testing with Docker containers. - Variables: - `DOCKER_HOST: tcp://docker:2375` - sets the Docker host to communicate with the Docker daemon inside the dind service. - `DOCKER_TLS_CERTDIR: ""` - we'll disable TLS to simplify the setup in a testing environment. - `DOCKER_DRIVER: overlay2` - specifies the storage driver for Docker, ensuring better performance and compatibility. - The last line ensures that the pipeline fails if the tests fail. ### Executors We mentioned in the beginning that each job runs in a newly provisioned VM. You can also notice that the pipeline configuration mentions a docker image, which is a template that contains instructions for creating a container. This might look confusing, but a runner is responsible for the execution of one job. This runner is installed on a machine and implements a certain [executor](https://docs.gitlab.com/runner/executors/). The executor determines the environment in which the job runs. By default, the GitLab-managed runners use a Docker Machine executor. Some other available executor options are: SSH, Shell, Parallels, VirtualBox, Docker, Docker Autoscaler, Kubernetes. Sometimes visualizing the components of a pipeline can be tricky, so let's simplify this into a diagram: ![GitLab CI diagram](/images/aws/gitlab-ci-diagram.png) Basically, the `service` is an additional container that starts at the same time as the one running the `test_job`. The job container has a Docker client, and it communicates with the Docker daemon, running in the service container, in order to spin up more containers, in this case for the Lambda functions. Don't forget to add your `LOCALSTACK_AUTH_TOKEN` as a masked variable in your CI/CD settings. ```vue Settings -> CI/CD -> Expand the Variables section -> Add variable ``` ![CI variable](/images/aws/ci-variable.png) In the web interface, under the Jobs section, you can see the jobs that ran, and you can also filter them based on their status. ![Pipeline run](/images/aws/pipeline-run.png) ## CI Pipeline Using Self-hosted Runners There are some cases when you want to run your pipelines locally and GitLab can provide that functionality. If you're new to the GitLab ecosystem, you need to be careful in configuring this setup, because it's easy to overlook an important field which can hinder your job runs. Let's get started by using the web interface. In your GitLab project, in the left-hand side panel, follow the path: ```vue Settings -> CI/CD -> Expand the Runners section -> Project runners -> New project runner ``` ![Project runners section](/images/aws/project-runners-section.png) Adding a tag, will allow you in the future to select a particular subset of runners to execute pipelines that require specific attributes. You can go ahead an tick the `Run untagged jobs` checkbox to be able to use this runner for all jobs that don't have a tag defined. The following fields are optional, the description of the runner and the maximum job timeout. ![Create runner 1](/images/aws/create-runner-1.png) This dashboard may suffer changes and improvements over time, but the attributes should essentially remain the same. ![Create runner 2](/images/aws/create-runner-2.png) After selecting the Linux machine you're done with defining the runner. Now you need a place to execute this runner, which will be your local computer. Notice the token in the first step command and save it for later. Runner authentication tokens have the prefix `glrt-`. For simplicity, we'll use a GitLab Runner Docker image. The GitLab Runner Docker images are designed as wrappers around the standard `gitlab-runner` command, like if GitLab Runner was installed directly on the host. You can read more about it in the [GitLab documentation](https://docs.gitlab.com/runner/install/docker.html). Make sure you have Docker installed. To verify your setup you can run the `docker info` command. Now, you need to create a volume on the disk that holds the configuration for the runner. You can have different volumes that can be used for different runners. ```bash docker volume create gitlab-runner-config ```bash title="Output" gitlab-runner-config ``` The agent that will interact with the system to create the runner needs to be run: ```bash docker run -d --name gitlab-runner \ --restart always \ -v /var/run/docker.sock:/var/run/docker.sock \ -v gitlab-runner-config:/etc/gitlab-runner/ \ gitlab/gitlab-runner:latest ``` ```bash title="Output" 79ad150847bd78fc567b08cafd96fc6bfec8f1946feb2beea89d7c6c395c01c4 ``` The breakdown for this command: - The container is named `gitlab-runner`. - It is configured to always restart. - It has access to the Docker socket on the host machine, allowing it to manage Docker containers - which is very important here. - It uses the named volume previously defined for persistent configuration storage. - It uses the latest GitLab Runner image. The next step is to register the runner: ```bash docker run --rm -it \ -v gitlab-runner-config:/etc/gitlab-runner/ \ gitlab/gitlab-runner:latest register ``` Follow the instructions that are prompted: ![Runner config](/images/aws/runner-config.png) In the container logs you should see this: ```commandline Configuration loaded builds=0 max_builds=1 ``` Let's look at the `config.toml` file and make the final adjustment before successfully running the pipeline. For running a job that does not require any additional containers to be created, you can stop here. However, since we need to run Docker commands in our CI/CD jobs, we must configure GitLab Runner to support those commands. This method requires `privileged` mode. Let's use the current running container to do that. Run the following: ```bash docker exec -it gitlab-runner bin/bash ``` Inside the container, let's run: ```bash cd etc/gitlab-runner ls -al ``` ```bash title="Output" total 24 drwx------ 3 root root 4096 May 16 19:58 . drwxr-xr-x 1 root root 4096 May 16 19:56 .. -rw------- 1 root root 14 May 16 19:50 .runner_system_id drwx------ 2 root root 4096 May 3 17:36 certs -rwx------ 1 root root 781 May 16 19:52 config.toml ``` You can see the file that contains all the runner configurations. ```bash apt update && apt install nano nano config.toml ``` The `privileged` field needs to be changed to `true`. Now the configurations should look like this: ```toml connection_max_age = "15m0s" shutdown_timeout = 0 [session_server] session_timeout = 1800 [[runners]] name = "localstack-testcontainers-runner" url = "https://gitlab.com" id = 36509569 token = "glrt-RUNNER_AUTHENTICATION_TOKEN" token_obtained_at = 2024-05-16T19:51:27Z token_expires_at = 0001-01-01T00:00:00Z executor = "docker" [runners.custom_build_dir] [runners.cache] MaxUploadedArchiveSize = 0 [runners.cache.s3] [runners.cache.gcs] [runners.cache.azure] [runners.docker] tls_verify = false image = "docker:26.1.2-dind" privileged = true disable_entrypoint_overwrite = false oom_kill_disable = false disable_cache = false volumes = ["/cache"] shm_size = 0 network_mtu = 0 ``` `[CTRL] + [X]` to save and exit the file. The runner is ready to use. You can now run your pipeline by pushing changes to your project or from the dashboard, by going to `Build -> Pipelines` and using the `Run pipeline` button. ## Conclusion In this tutorial, we've covered setting up a CI pipeline with GitLab runners and configuring a local Docker container to run the pipeline using a self-configured GitLab runner. Overall, the GitLab platform is an intricate system that can be used for highly complex projects to serve a multitude of purposes. With the steps learnt in this article, you can efficiently run end-to-end tests for your application using Testcontainers and LocalStack. # Generate IAM Policies with LocalStack IAM Policy Stream > Learn how to generate IAM policies for your AWS API requests on your local machine using LocalStack's IAM Policy Stream. ## Introduction When you're developing cloud and serverless applications, you need to grant access to various AWS resources like S3 buckets and RDS databases. To handle this, you create IAM roles and assign permissions through policies. However, configuring these policies can be challenging, especially if you want to ensure minimal access of all principals to your resources. [LocalStack IAM Policy Stream](https://app.localstack.cloud/inst/default/policy-stream) automates the generation of IAM policies for your AWS API requests on your local machine. This stream helps you identify the necessary permissions for your cloud application and allows you to detect logical errors, such as unexpected actions in your policies. This tutorial will guide you through setting up IAM Policy Stream for a locally running AWS application. We'll use a basic example involving an S3 bucket, an SQS queue, and a bucket notification configuration. You'll generate the policy for the bucket notification configuration and insert it into the SQS queue. ## Why use IAM Policy Stream? LocalStack enables you to create and enforce local IAM roles and policies using the [`ENFORCE_IAM` feature](/aws/developer-tools/security-testing/iam-policy-enforcement). However, users often struggle to figure out the necessary permissions for different actions. It's important to find a balance, avoiding giving too many permissions while making sure the right ones are granted. This challenge becomes more complex when dealing with AWS services that make requests not directly visible to users. For instance, if an SNS topic sends a message to an SQS queue and the underlying call fails, there might be no clear error message, causing confusion, especially for those less familiar with the services. IAM Policy Stream simplifies this by automatically generating the needed policies and showing them to users. This makes it easier to integrate with resources, roles, and users, streamlining the development process. Additionally, it serves as a useful learning tool, helping users understand the permissions linked to various AWS calls and improving the onboarding experience for newcomers to AWS. ## Prerequisites - [`lstk`](/aws/getting-started/installation#lstk) with [`LOCALSTACK_AUTH_TOKEN`](/aws/getting-started/auth-token) - [Docker](https://docs.docker.com/get-docker/) - [Terraform](https://developer.hashicorp.com/terraform/install) & [`lstk terraform`](/aws/connecting/infrastructure-as-code/terraform#lstk-terraform) - [AWS](https://docs.aws.amazon.com/cli/v1/userguide/cli-chap-install.html) CLI with [`lstk aws`](/aws/connecting/aws-cli#localstack-aws-cli-lstk-aws) - [LocalStack account](https://www.localstack.cloud/pricing) - [`jq`](https://jqlang.github.io/jq/download/) ## Architecture diagram The following diagram illustrates the architecture of this tutorial: ![LocalStack Environment Architecture](/images/aws/iam-policy-stream-architecture.png) In this architecture: 1. An **S3 bucket** is configured to send event notifications when objects are created 2. An **SQS queue** receives these notifications 3. The **IAM Enforcement Engine** intercepts API calls and checks for proper permissions 4. The **IAM Policy Stream Dashboard** captures all API requests and generates the necessary IAM policies 5. When a file is uploaded to the S3 bucket, S3 attempts to send a message to SQS, which initially fails due to missing permissions 6. The IAM Policy Stream automatically generates the required policy, which can then be applied to resolve the violation ## Tutorial: Configure an S3 bucket for event notifications using SQS In this tutorial, you will configure a LocalStack S3 bucket to send event notifications to an SQS queue. You will then use IAM Policy Stream to generate the necessary IAM policy for the SQS queue. You will use Terraform to create the resources and the AWS CLI to interact with them. With LocalStack's IAM enforcement enabled, you can thoroughly test your policy and ensure that the development setup mirrors the production environment. ### Start your LocalStack container Launch the LocalStack container on your local machine using the specified command: ```bash LOCALSTACK_DEBUG=1 LOCALSTACK_IAM_SOFT_MODE=1 lstk start ``` In the above command: - `LOCALSTACK_DEBUG=1` turns on detailed logging to check API calls and IAM violations. - `LOCALSTACK_IAM_SOFT_MODE=1` lets you test IAM enforcement by logging violations without stopping the API calls. ### Create the Terraform configuration Create a new file called `main.tf` for the Terraform setup of an S3 bucket and an SQS queue. Start by using the `aws_sqs_queue` resource to create an SQS queue named `s3-event-notification-queue`. ```hcl resource "aws_sqs_queue" "queue" { name = "s3-event-notification-queue" } ``` Next, use the `aws_s3_bucket` resource to create an S3 bucket called `s3-event-notification-bucket`. ```hcl resource "aws_s3_bucket" "bucket" { bucket = "s3-event-notification-bucket" } ``` Finally, use the `aws_s3_bucket_notification` resource to link the S3 bucket with the SQS queue for sending notifications: ```hcl resource "aws_s3_bucket_notification" "bucket_notification" { bucket = aws_s3_bucket.bucket.id queue { queue_arn = aws_sqs_queue.queue.arn events = ["s3:ObjectCreated:*"] } } ``` ### Deploy the Terraform configuration You can use `lstk terraform` to deploy your Terraform configuration within the LocalStack environment. Run the following commands to initialize and apply the Terraform configuration: ```bash lstk terraform init lstk terraform apply ``` You will be prompted to confirm the changes. Type `yes` to continue. Since LocalStack is used, no real AWS resources are created. LocalStack will emulate ephemeral development resources that will be removed automatically once you stop the LocalStack container. After applying the Terraform configuration, the output will appear similar to this: ```shell aws_sqs_queue.queue: Creating... aws_s3_bucket.bucket: Creating... aws_s3_bucket.bucket: Creation complete after 1s [id=s3-event-notification-bucket] aws_sqs_queue.queue: Still creating... [10s elapsed] aws_sqs_queue.queue: Still creating... [20s elapsed] aws_sqs_queue.queue: Creation complete after 26s [id=http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/s3-event-notification-queue] aws_s3_bucket_notification.bucket_notification: Creating... aws_s3_bucket_notification.bucket_notification: Creation complete after 0s [id=s3-event-notification-bucket] Apply complete! Resources: 3 added, 0 changed, 0 destroyed. ``` ### Start the IAM Policy Stream Access the [LocalStack Web Application](https://app.localstack.cloud/) and go to the [IAM Policy Stream dashboard](https://app.localstack.cloud/inst/default/policy-stream). This feature enables you to directly examine the generated policies, displaying the precise permissions required for each API call. ![IAM Policy Stream dashboard](/images/aws/iam-policy-stream-dashboard.png) You'll observe the Stream active status icon, indicating that making any local AWS API request will trigger the generation of an IAM Policy. Now, let's proceed to upload a file to the S3 bucket to trigger the event notification and generate the IAM policy. ### Trigger the event notification Create a new file named `some-log-file.log` and upload it to the S3 bucket using the AWS CLI: ```bash echo "Hello, LocalStack" > some-log-file.log lstk aws s3 cp some-log-file.log s3://s3-event-notification-bucket/ ``` Uploading a file will activate an event notification, sending a message to the SQS queue. However, since the SQS queue lacks the necessary permissions, an IAM violation will appear in the [IAM Policy Stream dashboard](https://app.localstack.cloud/inst/default/policy-stream). ![IAM Policy Stream showcasing an IAM violation](/images/aws/iam-policy-stream-violation.png) You can also navigate to the LocalStack logs and observe the IAM violation message: ```shell 2024-07-09T05:30:33.583 INFO --- [et.reactor-4] l.s.i.p.handler : Request for service 'sqs' by principal 's3.amazonaws.com' for operation 'SendMessage' denied. 2024-07-09T05:30:33.583 DEBUG --- [et.reactor-4] l.s.i.p.handler : Necessary permissions for this action: ["Action 'sqs:SendMessage' for 'arn:aws:sqs:us-east-1:000000000000:s3-event-notification-queue'"] 2024-07-09T05:30:33.583 DEBUG --- [et.reactor-4] l.s.i.p.handler : 0 permissions have been explicitly denied: [] 2024-07-09T05:30:33.583 DEBUG --- [et.reactor-4] l.s.i.p.handler : 0 permissions have been explicitly allowed: [] 2024-07-09T05:30:33.583 DEBUG --- [et.reactor-4] l.s.i.p.handler : 1 permissions have been implicitly denied: ["Action 'sqs:SendMessage' for 'arn:aws:sqs:us-east-1:000000000000:s3-event-notification-queue'"] ``` ### Generate the IAM policy Go to the IAM Policy Stream dashboard and review the API calls such as `PutObject`, `SendMessage`, and `ReceiveMessage`. Notice that the `SendMessage` call was denied due to an IAM violation. Click on the **SQS.SendMessage** action to see the suggested IAM policy. ![IAM Policy Stream showcasing the required SQS policy](/images/aws/iam-policy-stream-sqs-policy.png) LocalStack automatically recommends a resource-based policy for the SQS queue `arn:aws:sqs:us-east-1:000000000000:s3-event-notification-queue`. Copy this policy and incorporate it into your Terraform configuration under the `aws_sqs_queue` resource by adding the `policy` attribute: ```hcl resource "aws_sqs_queue" "queue" { name = "s3-event-notification-queue" policy = < test-file.log lstk aws s3 cp test-file.log s3://s3-event-notification-bucket/ ``` **Expected output - IAM Violation in LocalStack logs:** ```shell 2024-07-09T05:30:33.583 INFO --- [et.reactor-4] l.s.i.p.handler : Request for service 'sqs' by principal 's3.amazonaws.com' for operation 'SendMessage' denied. 2024-07-09T05:30:33.583 DEBUG --- [et.reactor-4] l.s.i.p.handler : Necessary permissions for this action: ["Action 'sqs:SendMessage' for 'arn:aws:sqs:us-east-1:000000000000:s3-event-notification-queue'"] 2024-07-09T05:30:33.583 DEBUG --- [et.reactor-4] l.s.i.p.handler : 0 permissions have been explicitly denied: [] 2024-07-09T05:30:33.583 DEBUG --- [et.reactor-4] l.s.i.p.handler : 0 permissions have been explicitly allowed: [] 2024-07-09T05:30:33.583 DEBUG --- [et.reactor-4] l.s.i.p.handler : 1 permissions have been implicitly denied: ["Action 'sqs:SendMessage' for 'arn:aws:sqs:us-east-1:000000000000:s3-event-notification-queue'"] ``` **IAM Policy Stream Dashboard showing the violation:** ![IAM Policy Stream showcasing an IAM violation](/images/aws/iam-policy-stream-violation.png) The dashboard clearly shows: - **Action**: `SQS.SendMessage` - **Status**: `Denied` (shown in red) - **Principal**: `s3.amazonaws.com` - **Resource**: `arn:aws:sqs:us-east-1:000000000000:s3-event-notification-queue` **Attempting to receive messages from the queue:** ```bash lstk aws sqs receive-message \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/s3-event-notification-queue ``` **Expected output - the message is still delivered, because soft mode does not block the call:** ```json { "Messages": [ { "MessageId": "5da627c5-5b4b-4202-a499-510222727e43", "ReceiptHandle": "NWMyZDA1MDEtM2NlYi00MzBjLWIyNjQtYjM2ZjNmYmQxZTAyIGFybjphd3M6c3FzOnVzLWVhc3QtMTowMDAwMDAwMDAwMDA6czMtZXZlbnQtbm90aWZpY2F0aW9uLXF1ZXVlIDVkYTYyN2M1LTViNGItNDIwMi1hNDk5LTUxMDIyMjcyN2U0MyAxNzg2NjY1NzQ3LjQ3ODI1MTI=", "MD5OfBody": "dee5cf145a0678a0ac02e3b38aa302f7", "Body": "{\"Records\": [{\"eventVersion\": \"2.1\", \"eventSource\": \"aws:s3\", \"awsRegion\": \"us-east-1\", \"eventName\": \"ObjectCreated:Put\", \"s3\": {\"bucket\": {\"name\": \"s3-event-notification-bucket\"}, \"object\": {\"key\": \"test-file.log\", \"size\": 18}}}]}" } ] } ``` :::note In a production environment, where the same policy gap would be enforced by real AWS IAM, this `SendMessage` call would fail and the message would never arrive. ::: ### Testing Scenario 2: Allow (With IAM Policy) After applying the IAM policy generated by the Policy Stream to your SQS queue, the S3 service will be granted permission to send messages. **The required policy (already applied via Terraform):** ```json { "Version": "2012-10-17", "Statement": [ { "Sid": "Test22bf6867", "Effect": "Allow", "Action": "sqs:SendMessage", "Resource": "arn:aws:sqs:us-east-1:000000000000:s3-event-notification-queue", "Principal": { "Service": [ "s3.amazonaws.com" ] }, "Condition": { "ArnEquals": { "aws:SourceArn": "arn:aws:s3:::s3-event-notification-bucket" } } } ] } ``` **Upload another test file:** ```bash echo "Test file with policy" > test-file-2.log lstk aws s3 cp test-file-2.log s3://s3-event-notification-bucket/ ``` **Expected output - Success (no IAM violation):** ```shell upload: ./test-file-2.log to s3://s3-event-notification-bucket/test-file-2.log ``` **LocalStack logs:** An allowed request produces no output from the IAM policy handler — only denials are logged. ```bash lstk logs | grep "i.p.handler" ``` Only the entries from the earlier, un-permitted upload should remain. **IAM Policy Stream Dashboard showing no violations:** ![IAM Policy Stream showcasing no violations](/images/aws/iam-policy-stream-no-violations.png) The dashboard shows all actions with green checkmarks, indicating successful execution. **Receive the message from the queue:** ```bash lstk aws sqs receive-message \ --queue-url http://sqs.us-east-1.localhost.localstack.cloud:4566/000000000000/s3-event-notification-queue ``` **Expected output - Message successfully received:** ```json { "Messages": [ { "MessageId": "7c9d6b22-cb35-4a66-98dc-6f48dfc78f33", "ReceiptHandle": "MTM4ZTg2NTYtMGIwNC00ZWE2LWIyM2EtNWNlZTIyOTZmOGE1IGFybjphd3M6c3FzOnVzLWVhc3QtMTowMDAwMDAwMDAwMDA6czMtZXZlbnQtbm90aWZpY2F0aW9uLXF1ZXVlIDdjOWQ2YjIyLWNiMzUtNGE2Ni05OGRjLTZmNDhkZmM3OGYzMyAxNzIwNTAzNjEyLjU2NDEyOTQ=", "MD5OfBody": "10eacb105ec11badc56f7e0198e0c4ad", "Body": "{\"Service\": \"Amazon S3\", \"Event\": \"s3:TestEvent\", \"Time\": \"2024-07-09T05:29:55.923Z\", \"Bucket\": \"s3-event-notification-bucket\", \"RequestId\": \"bfa882c0-a3b0-4549-b4c5-ac34167b3076\", \"HostId\": \"eftixk72aD6Ap51TnqcoF8eFidJG9Z/2\"}" } ] } ``` The message body contains the S3 event notification with details about the uploaded file, confirming that the IAM policy is working correctly. ### Verification Checklist To ensure your IAM policies are correctly configured: - **No IAM violations** appear in the IAM Policy Stream dashboard - **Messages are successfully delivered** to the SQS queue - **No new violation entries** are logged for the `SendMessage` operation after the policy is applied - **All API calls display green checkmarks** in the Policy Stream dashboard ## Conclusion IAM Policy Stream streamlines your development process by minimizing the manual creation of policies and confirming the necessity of granted permissions. However, it is advisable to manually confirm that your policy aligns with your intended actions. Your code may unintentionally make requests, and LocalStack considers all requests made during policy generation as valid. A practical scenario is automating tests, such as integration or end-to-end testing, against your application using LocalStack. This setup allows LocalStack to automatically generate policies with the required permissions. However, it's important to note that these generated policies may not cover all possible requests, as only the requests made during testing are included. You can then review and customize the policies to meet your needs, ensuring that overly permissive policies don't find their way into production environments. # Building a Java Notification app using AWS Java SDK, Simple Email Service (SES), and CloudFormation > Build a Java Spring Boot application to configure Simple Email Service (SES) to send messages using AWS Java SDK in LocalStack. Learn how to configure Simple Queue Service (SQS) & Simple Notification Service (SNS) using CloudFormation templates deployed locally. ## Introduction Java is a popular platform for cloud applications that use Amazon Web Services. With the AWS Java SDK, Java developers can build applications that work with various AWS services, like Simple Email Service (SES), Simple Queue Service (SQS), Simple Notification Service (SNS), and more. Simple Email Service (SES) is a cloud-based email-sending service that enables developers to integrate email functionality into their applications running on AWS. SES allows developers to work without an on-prem Simple Mail Transfer Protocol (SMTP) system and send bulk emails to many recipients. [LocalStack for AWS](https://app.localstack.cloud/) supports SES along with a simple user interface to inspect email accounts and sent messages. LocalStack also supports sending SES messages through an actual SMTP email server. We will use SQS and SNS to process the emails. We would further employ a CloudFormation stack to configure the infrastructure and configure SNS & SQS subscriptions. AWS Java SDK would be employed to receive these SQS messages and to send these messages through SES further. In this tutorial, we will build a Java Spring Boot application that uses locally emulated AWS infrastructure on LocalStack provisioned by CloudFormation, and that uses the Java AWS SDK to send SES, SQS, and SNS messages. We will further use [MailHog](https://github.com/mailhog/MailHog), a local SMTP server, to inspect the emails sent through SES via an intuitive user interface. ## Prerequisites For this tutorial, you will need: - [LocalStack for AWS](https://localstack.cloud/pricing/) to emulate the AWS services (SNS, SQS, SES, etc) locally - Don't worry, if you don't have a subscription yet, you can just get a trial license for free. - [`lstk aws`](/aws/connecting/aws-cli#localstack-aws-cli-lstk-aws) - [Docker](https://docker.io/) - Java 11+ - Maven 3+ ## Project setup To get started, we will set up our Spring Boot project by implementing a single module named `example` that will house our application code. The module will contain the code required to set up our AWS configuration, notification service, and message application. We will have another directory called `resources` that will house our CloudFormation stack required to set up an SNS topic and an SQS queue. The project directory would look like this: ```bash ├── pom.xml ├── src │ └── main │ ├── java │ │ └── com │ │ └── example │ │ ├── AwsConfiguration.java │ │ ├── MessageApplication.java │ │ ├── Notification.java │ │ ├── NotificationController.java │ │ └── ReceiveSendNotifications.java │ └── resources │ └── email-infra.yml ``` In our root POM configuration, we will add the following dependencies: ```xml 4.0.0 cloud.localstack.samples java-notification-app 1.0-SNAPSHOT org.springframework.boot spring-boot-starter-parent 2.2.5.RELEASE 11 2.17.189 software.amazon.awssdk bom 2.17.189 pom import software.amazon.awssdk ses software.amazon.awssdk sns software.amazon.awssdk sqs software.amazon.awssdk cloudformation org.springframework.boot spring-boot-starter-web org.springframework.boot spring-boot-starter-test test org.junit.vintage junit-vintage-engine org.springframework.boot spring-boot-maven-plugin ``` In the above POM file, we have added the AWS Java SDK dependencies for SES, SNS, SQS, and CloudFormation. We have also added the Spring Boot dependencies for our application. We can move on to the next step with the initial setup complete. ## Setting up AWS configuration To get started, we will setup the AWS configuration, to be defined in `AwsConfiguration.java`, required for our Spring Boot application. We will create a configuration class to use the Spring Bean annotation to create two beans: `SesClient` and a `SqsClient`, to connect to the SES and SQS clients respectively. We will then create a bean to retrieve the `queueUrl` for the `email-notification-queue`: ```java package com.example; import java.net.URI; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import software.amazon.awssdk.auth.credentials.EnvironmentVariableCredentialsProvider; import software.amazon.awssdk.regions.Region; import software.amazon.awssdk.services.ses.SesClient; import software.amazon.awssdk.services.sqs.SqsClient; @Configuration public class AwsConfiguration { private static final String ENDPOINT_URL = "http://localhost:4566"; private static final Region DEFAULT_REGION = Region.US_EAST_1; @Bean public SqsClient sqsClient() { return SqsClient.builder() .region(DEFAULT_REGION) .credentialsProvider(EnvironmentVariableCredentialsProvider.create()) .applyMutation(builder -> { builder.endpointOverride(URI.create(ENDPOINT_URL)); }) .build(); } @Bean public SesClient sesClient() { return SesClient.builder() .region(DEFAULT_REGION) .credentialsProvider(EnvironmentVariableCredentialsProvider.create()) .applyMutation(builder -> { builder.endpointOverride(URI.create(ENDPOINT_URL)); }) .build(); } @Bean @Autowired public String notificationQueueUrl(SqsClient sqsClient) { return sqsClient.getQueueUrl(builder -> { builder.queueName("email-notification-queue"); }).queueUrl(); } } ``` In the above code, we have used the `@Autowired` annotation to autowire the dependencies that are required for the application (`SqsClient` `SesClient`, and `notificationQueueUrl` in this case). Now that we have got the URL of the queue created in the previous step, we can move on to the next step. :::note You can also use the pre-defined clients from the [localstack-utils](https://mvnrepository.com/artifact/cloud.localstack/localstack-utils) Maven project, as an alternative to creating the AWS SDK clients with endpoint overrides manually. ::: ## Creating a Notification Service To get started with creating a Notification Service, we would need to create a `Notification` class to define the structure of the notification that we would be sending to the SQS queue. We will create a `Notification` class in the `Notification.java` file: ```java package com.example; public class Notification { private String address; private String subject; private String body; public String getAddress() { return address; } public void setAddress(String address) { this.address = address; } public String getSubject() { return subject; } public void setSubject(String subject) { this.subject = subject; } public String getBody() { return body; } public void setBody(String body) { this.body = body; } } ``` In the above code, we have defined three instance variables: `address`, `subject`, and `body`. We have also defined the getters and setters for the instance variables. Let's now create a `@Component` class to listen to a queue, receive and transform the notifications into emails, and send the emails transactionally: ```java package com.example; import java.util.ArrayList; import java.util.Collections; import java.util.HashMap; import java.util.List; import java.util.stream.Collectors; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Component; import com.fasterxml.jackson.core.JsonProcessingException; import com.fasterxml.jackson.databind.ObjectMapper; import software.amazon.awssdk.services.ses.SesClient; import software.amazon.awssdk.services.ses.model.SendEmailRequest; import software.amazon.awssdk.services.sqs.SqsClient; import software.amazon.awssdk.services.sqs.model.Message; import software.amazon.awssdk.services.sqs.model.ReceiveMessageRequest; import software.amazon.awssdk.services.sqs.model.ReceiveMessageResponse; @Component public class ReceiveSendNotifications { private static final Logger LOG = LoggerFactory.getLogger(ReceiveSendNotifications.class); private static final String SOURCE_EMAIL = "no-reply@localstack.cloud"; @Autowired private SqsClient sqsClient; @Autowired private SesClient sesClient; @Autowired private String notificationQueueUrl; private final ObjectMapper objectMapper = new ObjectMapper(); public List processNotifications() { // receive messages from queue ReceiveMessageResponse receiveMessageResponse = sqsClient.receiveMessage( request -> request.queueUrl(notificationQueueUrl).maxNumberOfMessages(10) ); if (!receiveMessageResponse.hasMessages()) { return Collections.emptyList(); } // transform notifications List messages = receiveMessageResponse.messages(); List notificationsToSend = new ArrayList<>(messages.size()); List notificationReceipts = new ArrayList<>(messages.size()); for (Message message : messages) { String body = message.body(); try { // extract SNS event HashMap snsEvent = objectMapper.readValue(body, HashMap.class); LOG.info("processing snsEvent {}", snsEvent); // Notification is expected to be wrapped in the SNS message body String notificationString = snsEvent.get("Message").toString(); Notification notification = objectMapper.readValue(notificationString, Notification.class); notificationsToSend.add(notification); notificationReceipts.add(message.receiptHandle()); } catch (JsonProcessingException e) { LOG.error("error processing message body {}", body, e); } } // send notifications transactional List sentMessages = new ArrayList<>(); for (int i = 0; i < notificationsToSend.size(); i++) { Notification notification = notificationsToSend.get(i); String receiptHandle = notificationReceipts.get(i); try { String messageId = sendNotificationAsEmail(notification); LOG.info("successfully sent notification as email, message id = {}", messageId); sentMessages.add(messageId); } catch (Exception e) { LOG.error("could not send notification as email {}", notification, e); continue; } sqsClient.deleteMessage(builder -> { builder.queueUrl(notificationQueueUrl).receiptHandle(receiptHandle); }); } return sentMessages; } public String sendNotificationAsEmail(Notification notification) { return sesClient.sendEmail(notificationToEmail(notification)).messageId(); } public SendEmailRequest notificationToEmail(Notification notification) { return SendEmailRequest.builder().applyMutation(email -> { email.message(msg -> { msg.body(body -> { body.text(text -> { text.data(notification.getBody()); }); }).subject(subject -> { subject.data(notification.getSubject()); }); }).destination(dest -> { dest.toAddresses(notification.getAddress()); }).source(SOURCE_EMAIL); }).build(); } public List> listMessages() { ReceiveMessageRequest receiveRequest = ReceiveMessageRequest.builder() .queueUrl(notificationQueueUrl) .visibilityTimeout(0) .maxNumberOfMessages(10) .build(); ReceiveMessageResponse receiveMessageResponse = sqsClient.receiveMessage(receiveRequest); if (!receiveMessageResponse.hasMessages()) { return Collections.emptyList(); } return receiveMessageResponse.messages().stream().map(Message::body).map(str -> { try { return (HashMap) objectMapper.readValue(str, HashMap.class); } catch (JsonProcessingException e) { LOG.error("error processing message body {}", str, e); HashMap map = new HashMap<>(); map.put("body", str); return map; } }).collect(Collectors.toList()); } public void purgeQueue() { sqsClient.purgeQueue(builder -> { builder.queueUrl(notificationQueueUrl); }); } } ``` Let us now create a Notification Controller to: - Send emails from all parseable notifications in the queue (using the `/process` endpoint) - List all the message bodies (using the `/list` endpoint) - Purge the messages from the queue (using the `/purge` endpoint) Let's create a controller class to define the endpoints for the Notification Service: ```java package com.example; import java.util.HashMap; import java.util.List; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Controller; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMethod; import org.springframework.web.bind.annotation.ResponseBody; @Controller public class NotificationController { @Autowired ReceiveSendNotifications msgService; // Send emails for all parseable notifications @RequestMapping(value = "/process", method = RequestMethod.GET) @ResponseBody List processNotifications(HttpServletRequest request, HttpServletResponse response) { return msgService.processNotifications(); } // Lists all message bodies @RequestMapping(value = "/list", method = RequestMethod.GET) @ResponseBody List> listMessages(HttpServletRequest request, HttpServletResponse response) { return msgService.listMessages(); } // Purge the message queue @RequestMapping(value = "/purge", method = RequestMethod.GET) @ResponseBody void purgeQueue(HttpServletRequest request, HttpServletResponse response) { msgService.purgeQueue(); } } ``` ## Setup the Spring Boot application & infrastructure Now that we have the code ready, let us setup the Spring Boot application using the `SpringApplication` Class to bootstrap and launch our Spring application from the `main` method. ```java package com.example; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; @SpringBootApplication public class MessageApplication { public static void main(String[] args) { SpringApplication.run(MessageApplication.class, args); } } ``` You can now build the application using the following command: ```bash mvn clean install ``` If the build is successful, you will notice a `BUILD SUCCESS` message. Now that we have the application ready, let us setup the infrastructure using CloudFormation. Create a new file in ``src/main/resources` called `email-infra.yml` and add the following content: ```yaml AWSTemplateFormatVersion: 2010-09-09 Resources: EmailQueue: Type: AWS::SQS::Queue Properties: QueueName: email-notification-queue EmailTopic: Type: AWS::SNS::Topic Properties: TopicName: email-notifications SnsSubscription: Type: AWS::SNS::Subscription Properties: Protocol: sqs Endpoint: !GetAtt EmailQueue.Arn TopicArn: !GetAtt EmailTopic.TopicArn ``` In the above code, we have created a queue called `email-notification-queue` and a topic called `email-notifications`. We have also created a subscription between the queue and the topic, allowing any message published to the topic to be sent to the queue. ## Creating the infrastructure Now that the initial coding is done, we can give it a try. Let's start LocalStack using a custom `docker-compose` setup, which includes MailHog to capture the emails sent by SES: ```yaml services: localstack: container_name: "${LOCALSTACK_DOCKER_NAME:-localstack-main}" image: localstack/localstack ports: - "127.0.0.1:4510-4559:4510-4559" # external service port range - "127.0.0.1:4566:4566" # LocalStack Edge Proxy environment: - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} - DEBUG=1 - HOST_TMP_FOLDER=${TMPDIR:-/tmp/}localstack - SMTP_HOST=smtp:1025 volumes: - "${TMPDIR:-/tmp}/localstack:/tmp/localstack" - "/var/run/docker.sock:/var/run/docker.sock" smtp: image: mailhog/mailhog ports: - "1025" - "8025:8025" ``` The above `docker-compose` file will start LocalStack and pull the MailHog image to start the SMTP server (if it doesn't exist yet!) on port `8025`. You can start LocalStack using the following command: ```bash LOCALSTACK_AUTH_TOKEN= docker-compose up -d ``` Given that you've started LocalStack via `docker-compose`, you'll need to configure the `lstk` CLI to contact your container: ```bash export LSTK_ENDPOINT_URL=http://localhost.localstack.cloud:4566 ``` Once LocalStack is started, we can deploy the CloudFormation stack (which might take a few moments): ```bash lstk aws cloudformation deploy \ --template-file src/main/resources/email-infra.yml \ --stack-name email-infra ``` With our infrastructure ready, we can now start the Spring Boot application. We will set dummy AWS access credentials as environment variables in the command: ```bash AWS_ACCESS_KEY_ID=test AWS_SECRET_ACCESS_KEY=test mvn spring-boot:run ``` ## Testing the application To get started, we will an add email address to the list of identities for our mocked SES account to verify the email address: ```bash lstk aws ses verify-email-identity --email-address no-reply@localstack.cloud ``` Let us now send a message to the topic: ```bash lstk aws sns publish \ --topic arn:aws:sns:us-east-1:000000000000:email-notifications \ --message '{"subject":"hello", "address": "alice@example.com", "body": "hello world"}' ``` In the above command, we have published a message to the topic `email-notifications` with a generic message body. The output of the command should look like this: ```json { "MessageId": "" } ``` You can now use [curl](https://curl.se/) to send a request to the `/list` endpoint for the queued messages: ```bash curl -s localhost:8080/list | jq . ``` You will see an output similar to the following: ```json [ { "SignatureVersion": "1", "Type": "Notification", "TopicArn": "arn:aws:sns:us-east-1:000000000000:email-notifications", "Message": "{\"subject\":\"hello\", \"address\": \"alice@example.com\", \"body\": \"hello world\"}", "UnsubscribeURL": "http://localhost:4566/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:000000000000:email-notifications:", "Signature": "EXAMPLEpH+..", "Timestamp": "", "SigningCertURL": "https://sns.us-east-1.amazonaws.com/SimpleNotificationService-0000000000000000000000.pem", "MessageId": "", } ] ``` You can now run the `/process` endpoint to send the queued notifications as emails: ```bash curl -s localhost:8080/process ``` To check whether the email has been sent, you can query the LocalStack internal SES endpoint using the following command: ```bash curl -s localhost:4566/_aws/ses | jq . ``` ```bash title="Output" { "messages": [ { "Id": "", "Timestamp": "", "Region": "us-east-1", "Source": "no-reply@localstack.cloud", "Destination": { "ToAddresses": [ "alice@example.com" ] }, "Subject": "hello", "Body": { "text_part": "hello world", "html_part": null } } ] } ``` You can also navigate to the MailHog via the user-interface: [`localhost:8025`](http://localhost:8025/) to check out the email. ## Conclusion In this tutorial, we have demonstrated, how you can: - Use CloudFormation to provision infrastructure for SNS & SQS subscriptions on LocalStack - Use the AWS Java SDK and Spring Boot to build an application that sends SQS and SES messages. Using [LocalStack for AWS](https://app.localstack.cloud), you can use our Web user interface to view the email messages sent by SES. The code for this tutorial can be found in our [LocalStack for AWS samples over GitHub](https://github.com/localstack/localstack-pro-samples/tree/master/java-notification-app). # Deploying Lambda container image locally with Elastic Container Registry (ECR) using LocalStack > Learn how to create and deploy Lambda functions using container images in LocalStack. This tutorial guides you through packaging your code and dependencies into a Docker image, creating a local Elastic Container Registry (ECR) in LocalStack, and deploying the Lambda container image. ## Introduction [Lambda](https://aws.amazon.com/lambda/) is a powerful serverless compute system that enables you to break down your application into smaller, independent functions. These functions can be deployed as individual units within the AWS ecosystem. Lambda offers seamless integration with various AWS services and supports multiple programming languages for different runtime environments. To deploy Lambda functions programmatically, you have two options: [uploading a ZIP file containing your code and dependencies](https://docs.aws.amazon.com/lambda/latest/dg/configuration-function-zip.html) or [packaging your code in a container image](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-images.html) and deploying it through Elastic Container Registry (ECR). [ECR](https://aws.amazon.com/ecr/) is an AWS-managed registry that facilitates the storage and distribution of containerized software. With ECR, you can effectively manage your image lifecycles, versioning, and tagging, separate from your application. It seamlessly integrates with other AWS services like ECS, EKS, and Lambda, enabling you to deploy your container images effortlessly. Creating container images for your Lambda functions involves using Docker and implementing the Lambda Runtime API according to the Open Container Initiative (OCI) specifications. [LocalStack for AWS](https://localstack.cloud) extends support for Lambda functions using container images through ECR. It enables you to deploy your Lambda functions locally using LocalStack. In this tutorial, we will explore creating a Lambda function using a container image and deploying it locally with the help of LocalStack. ## Prerequisites Before diving into this tutorial, make sure you have the following prerequisites: - [LocalStack for AWS](https://localstack.cloud/pricing/) - [`lstk aws`](/aws/connecting/aws-cli#localstack-aws-cli-lstk-aws) - [Python](https://www.python.org/downloads/) - [Docker](https://docker.io/) ## Architecture Diagram The following diagram shows the architecture that we are going to follow : ![lambda-ecr-container-images-architecture](/images/aws/lambda-ecr-container-images.png) ## Creating a Lambda function To package and deploy a Lambda function as a container image, we'll create a Lambda function containing our code and a Dockerfile. Create a new directory for your lambda function and navigate to it: ```bash mkdir -p lambda-container-image cd lambda-container-image ``` Initialize the directory by creating two files: `handler.py` and `Dockerfile`. Use the following commands to create the files: ```bash touch handler.py Dockerfile ``` Open the `handler.py` file and add the following Python code, which represents a simple Lambda function that returns the message `'Hello from LocalStack Lambda container image!'`: ```python def handler(event, context): print('Hello from LocalStack Lambda container image!') ``` In the code above, the `handler` function is executed by the Lambda service whenever a trigger event occurs. It serves as the entry point for the Lambda function within the runtime environment and accepts `event` and `context` as parameters, providing information about the event and invocation properties, respectively. Following these steps, you have created the foundation for your Lambda function and defined its behaviour using Python code. In the following sections, we will package this code and its dependencies into a container image using the `Dockerfile`. ## Building the image To package our Lambda function as a container image, we must create a Dockerfile containing the necessary instructions for building the image. Open the Dockerfile and add the following content. This Dockerfile uses the `python:3.8` base image provided by AWS for Lambda and copies the `handler.py` file into the image. It also specifies the function handler as `handler.handler` to ensure the Lambda runtime can locate it where the Lambda handler is available. ```Dockerfile FROM public.ecr.aws/lambda/python:3.8 COPY ./handler.py ./ CMD [ "handler.handler" ] ``` :::note If your Lambda function has additional dependencies, create a file named `requirements.txt` in the same directory as the Dockerfile. List the required libraries in this file. You can install these dependencies in the `Dockerfile` under the `${LAMBDA_TASK_ROOT}` directory. ::: With the Dockerfile prepared, you can now build the container image using the following command, to check if everything works as intended: ```bash docker build . ``` By executing these steps, you have defined the Dockerfile that instructs Docker on how to build the container image for your Lambda function. The resulting image will contain your function code and any specified dependencies. ## Publishing the image to ECR Now that the initial setup is complete let's explore how to leverage LocalStack's AWS emulation by pushing our image to ECR and deploying the Lambda container image. Start LocalStack by executing the following command. ```bash LOCALSTACK_ECR_ENDPOINT_STRATEGY=off LOCALSTACK_DEBUG=1 lstk start ``` Once the LocalStack container is running, we can create a new ECR repository to store our container image. Use the `lstk aws` CLI to achieve this. Run the following command to create the repository, replacing `localstack-lambda-container-image` with the desired name for your repository: ```bash lstk aws ecr create-repository --repository-name localstack-lambda-container-image ``` ```bash title="Output" { "repository": { "repositoryArn": "arn:aws:ecr:us-east-1:000000000000:repository/localstack-lambda-container-image", "registryId": "000000000000", "repositoryName": "localstack-lambda-container-image", "repositoryUri": "localhost.localstack.cloud:4510/localstack-lambda-container-image", "createdAt": , "imageTagMutability": "MUTABLE", "imageScanningConfiguration": { "scanOnPush": false }, "encryptionConfiguration": { "encryptionType": "AES256" } } } ``` :::note To further customize the ECR repository, you can pass additional flags to the `create-repository` command. For more details on the available options, refer to the [AWS CLI documentation](https://docs.aws.amazon.com/cli/latest/reference/ecr/create-repository.html). ::: Next, build the image and push it to the ECR repository. Execute the following commands: ```bash docker build -t localhost:4510/localstack-lambda-container-image . docker push localhost:4510/localstack-lambda-container-image ``` In the above commands, we specify the `repositoryUri` as the image name to push the image to the ECR repository. After executing these commands, you can verify that the image is successfully pushed to the repository by using the `describe-images` command: ```bash lstk aws ecr describe-images --repository-name localstack-lambda-container-image ``` ```bash title="Output" { "imageDetails": [ { "registryId": "000000000000", "repositoryName": "localstack-lambda-container-image", "imageDigest": "sha256:459fce12258ff1048925e0f4e7fb039d8b54111a8e3cca5db4acb434a9e8af37", "imageTags": [ "latest" ], "imageSizeInBytes": 184217147, "imagePushedAt": , "imageManifestMediaType": "application/vnd.docker.distribution.manifest.v2+json", "artifactMediaType": "application/vnd.docker.container.image.v1+json" } ] } ``` By running this command, you can confirm that the image is now in the ECR repository. It ensures it is ready for deployment as a Lambda function using LocalStack's AWS emulation capabilities. ## Deploying the Lambda function To deploy the container image as a Lambda function, we will create a new Lambda function using the `create-function` command. Run the following command to create the function: :::note Before creating the lambda function, please double check under which architecture you have built your image. If your image is built as arm64, you need to specify the lambda architecture when deploying or set `LAMBDA_IGNORE_ARCHITECTURE=1` when starting LocalStack. More information can be found [in our documentation regarding ARM support.](/aws/customization/advanced/arm64-support) ::: ```bash lstk aws lambda create-function \ --function-name localstack-lambda-container-image \ --package-type Image \ --code ImageUri="localhost.localstack.cloud:4510/localstack-lambda-container-image" \ --role arn:aws:iam::000000000000:role/lambda-role \ --handler handler.handler ``` ```bash title="Output" { "FunctionName": "localstack-lambda-container-image", "FunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:localstack-lambda-container-image", "Role": "arn:aws:iam::000000000000:role/lambda-role", "Handler": "handler.handler", "CodeSize": 0, "Description": "", "Timeout": 3, "MemorySize": 128, "LastModified": , "CodeSha256": "9be73524cd5aa70fbcee3fc8d7aac4eb7e2a644e9ef2b13031719077a65c0031", "Version": "$LATEST", "TracingConfig": { "Mode": "PassThrough" }, "RevisionId": "cab4268c-2d56-4591-821a-9154e157b984", "State": "Pending", "StateReason": "The function is being created.", "StateReasonCode": "Creating", "PackageType": "Image", "Architectures": [ "x86_64" ], "EphemeralStorage": { "Size": 512 }, "SnapStart": { "ApplyOn": "None", "OptimizationStatus": "Off" } } ``` The command provided includes several flags to create the Lambda function. Here's an explanation of each flag: - `ImageUri`: Specifies the image URI of the container image you pushed to the ECR repository (`localhost.localstack.cloud:4510/localstack-lambda-container-image` in this case. Use the return `repositoryUri` from the create-repository command). - `package-type`: Sets the package type to Image to indicate that the Lambda function will be created using a container image. - `function-name`: Specifies the name of the Lambda function you want to create. - `runtime`: Defines the runtime environment for the Lambda function. In this case, it's specified as provided, indicating that the container image will provide the runtime. - `role`: Sets the IAM role ARN that the Lambda function should assume. In the example, a mock role ARN is used. For an actual role, please refer to the [IAM documentation](/aws/services/iam). ## Testing the application To invoke the Lambda function, you can use the `invoke` command: ```bash lstk aws lambda invoke --function-name localstack-lambda-container-image /tmp/lambda.out ``` ```bash title="Output" { "StatusCode": 200, "ExecutedVersion": "$LATEST" } ``` The command above will execute the Lambda function locally within the LocalStack environment. The response will include the StatusCode and ExecutedVersion. You can find the logs of the Lambda invocation in the Lambda container output: ```bash title="Output" Hello from LocalStack Lambda container image! ``` ## Conclusion In conclusion, the Lambda container image support enables you to use Docker to package your custom code and dependencies for Lambda functions. With the help of LocalStack, you can seamlessly package, deploy, and invoke Lambda functions locally. It empowers you to develop, debug, and test your Lambda functions with a wide range of AWS services. For more advanced usage patterns, you can explore features like [Lambda Hot Reloading](/aws/developer-tools/lambda-tools/hot-reloading) and [Lambda Debugging](/aws/developer-tools/lambda-tools/remote-debugging). To further explore and experiment with the concepts covered in this tutorial, you can access the code and accompanying `Makefile` on our [`localstack-pro-samples` repository on GitHub](https://github.com/localstack/localstack-pro-samples/tree/master/lambda-container-image). # Initializing an RDS Database with AWS CDK and LocalStack > Learn how to provision and initialize Amazon RDS databases locally using AWS CDK and LocalStack. This tutorial demonstrates database schema creation, data seeding, and testing with Cloud Pods for reproducible development environments. ## Introduction Database initialization is a critical aspect of application development and testing. Setting up databases with proper schemas and seed data consistently across development, testing, and CI environments can be challenging. Amazon RDS provides managed database services, but testing database initialization scripts and configurations requires a reliable local development environment. [LocalStack for AWS](https://app.localstack.cloud/) enables you to emulate Amazon RDS, Lambda, and Secrets Manager locally, allowing you to develop and test database initialization workflows without connecting to AWS. This approach accelerates development cycles, reduces costs, and ensures consistent environments across different stages of your development pipeline. In this tutorial, we will demonstrate how to provision and initialize an Amazon RDS database using AWS CDK and LocalStack. We'll create a Lambda function that executes custom SQL scripts to set up database schemas and seed data. Additionally, we'll explore how Cloud Pods can streamline CI workflows by providing pre-seeded database environments. ## Prerequisites For this tutorial, you will need: - [LocalStack for AWS](https://localstack.cloud/pricing/) with a valid auth token - [AWS CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) with the [`lstk aws`](/aws/connecting/aws-cli#localstack-aws-cli-lstk-aws) command - [AWS CDK](https://docs.localstack.cloud/user-guide/integrations/aws-cdk/) with the [`lstk cdk`](/aws/connecting/infrastructure-as-code/aws-cdk#aws-cdk-cli-for-localstack) command - [Node.js](https://nodejs.org/en/download/) (version 16 or later) - [Docker](https://docker.io/) - MySQL or PostgreSQL client (for testing database connections) - [`make`](https://www.gnu.org/software/make/) (optional, but recommended) ## Architecture The following diagram shows the architecture that this sample application builds and deploys: ![Architecture Diagram demonstrating Amazon RDS initialization using CDK](/images/aws/rds-database-initialization-architecture.png) The architecture consists of: - **Amazon RDS**: The central database instance that will be initialized and pre-filled with data - **AWS Lambda**: A Node.js function that executes SQL scripts to initialize the database schema and seed data - **AWS Secrets Manager**: Stores database credentials and connection details securely - **CloudFormation Custom Resource**: Triggers the Lambda function during deployment to perform database initialization The initialization process works as follows: 1. CDK deploys the RDS instance and related resources 2. A CloudFormation Custom Resource triggers the Lambda function 3. The Lambda function retrieves database credentials from Secrets Manager 4. The function connects to the RDS instance and executes initialization SQL scripts 5. Database tables are created and populated with seed data ## Getting Started ### Project Setup First, clone the sample repository and install dependencies: ```bash git clone https://github.com/localstack-samples/sample-cdk-rds-database-initialization.git cd sample-cdk-rds-database-initialization ``` Install the project dependencies: ```bash npm install # or if you prefer using make make install ``` ### Configure LocalStack Start LocalStack with your auth token: ```bash lstk start ``` > **Note**: By default, LocalStack uses the MariaDB engine for RDS (see [RDS documentation](https://docs.localstack.cloud/user-guide/aws/rds/#mysql-engine)). To use the real MySQL engine in a separate Docker container, set the environment variable `RDS_MYSQL_DOCKER=1`. ## Deployment Deploy the sample application using CDK: ```bash make deploy # or manually: lstk cdk deploy ``` The deployment process will: 1. Create an RDS MySQL instance 2. Set up a Secrets Manager secret with database credentials 3. Deploy a Lambda function with database initialization code 4. Execute the initialization script via CloudFormation Custom Resource After successful deployment, you'll see output similar to: ```bash Outputs: RdsInitExample.RdsInitFnResponse = {"status":"OK","results":[/*...SQL operations...*/]} RdsInitExample.functionName = my-lambda-rds-query-helper RdsInitExample.secretName = /rdsinitexample/rds/creds/mysql-01 Stack ARN: arn:aws:cloudformation:us-east-1:000000000000:stack/RdsInitExample/3f53b7bd ✨ Total time: 80.21s CDK deployed successfully. ``` The outputs include: - `RdsInitFnResponse`: Results from executing the database initialization script - `functionName`: Lambda function name for running test queries - `secretName`: Secrets Manager secret containing database connection details ## Testing the Application The sample application creates a database with tables and sample data. Let's verify the initialization was successful by running queries against the database. ### Querying the Database via Lambda The deployed Lambda function `my-lambda-rds-query-helper` can execute SQL queries against the initialized database. The function requires two parameters: - `sqlQuery`: The SQL command to execute - `secretName`: The Secrets Manager secret containing database credentials **For AWS CLI v1:** ```bash lstk aws lambda invoke \ --function-name my-lambda-rds-query-helper \ --payload '{"sqlQuery": "select Author from books", "secretName":"/rdsinitexample/rds/creds/mysql-01"}' \ output ``` **For AWS CLI v2:** ```bash lstk aws lambda invoke \ --cli-binary-format raw-in-base64-out \ --function-name my-lambda-rds-query-helper \ --payload '{"sqlQuery": "select Author from books", "secretName":"/rdsinitexample/rds/creds/mysql-01"}' \ output ``` View the results: ```bash cat output ``` Expected output: ```json { "status": "SUCCESS", "results": [ {"Author": "Jane Doe"}, {"Author": "Jane Doe"}, {"Author": "LocalStack"} ] } ``` You can also run more detailed queries to explore the data: **Query all book details:** ```bash lstk aws lambda invoke \ --cli-binary-format raw-in-base64-out \ --function-name my-lambda-rds-query-helper \ --payload '{"sqlQuery": "SELECT * FROM books LIMIT 5", "secretName":"/rdsinitexample/rds/creds/mysql-01"}' \ output && cat output ``` ### Testing Different SQL Operations Test various database operations to verify the initialization: **Check table structure:** ```bash lstk aws lambda invoke \ --cli-binary-format raw-in-base64-out \ --function-name my-lambda-rds-query-helper \ --payload '{"sqlQuery": "DESCRIBE books", "secretName":"/rdsinitexample/rds/creds/mysql-01"}' \ output && cat output ``` **Count records:** ```bash lstk aws lambda invoke \ --cli-binary-format raw-in-base64-out \ --function-name my-lambda-rds-query-helper \ --payload '{"sqlQuery": "SELECT COUNT(*) as total_books FROM books", "secretName":"/rdsinitexample/rds/creds/mysql-01"}' \ output && cat output ``` **Filter by author:** ```bash lstk aws lambda invoke \ --cli-binary-format raw-in-base64-out \ --function-name my-lambda-rds-query-helper \ --payload '{"sqlQuery": "SELECT title, published_year FROM books WHERE author = \"George Orwell\"", "secretName":"/rdsinitexample/rds/creds/mysql-01"}' \ output && cat output ``` ### Connecting Directly to the Database For more comprehensive testing, you can connect directly to the RDS instance using a MySQL client. First, retrieve the database connection details: ```bash # Get the database endpoint lstk aws rds describe-db-instances --query 'DBInstances[0].Endpoint.Address' --output text # Get credentials from Secrets Manager lstk aws secretsmanager get-secret-value --secret-id /rdsinitexample/rds/creds/mysql-01 --query SecretString --output text ``` Connect using the MySQL command-line client: ```bash mysql -h -P 4510 -u -p ``` Once connected, you can run SQL queries directly: ```sql USE your_database_name; SHOW TABLES; SELECT * FROM books; ``` ### Running Integration Tests Execute the complete test suite to validate all functionality: ```bash make test ``` This will run end-to-end tests that verify: - Database connectivity - Schema creation - Data seeding - Query operations - Error handling ## Conclusion This tutorial demonstrated how to provision and initialize an Amazon RDS database locally using AWS CDK and LocalStack. You learned how to: - **Set up LocalStack for AWS** for local AWS service emulation - **Deploy RDS infrastructure** using AWS CDK and CloudFormation - **Initialize database schemas and data** via Lambda functions during deployment - **Test the initialized database** using both Lambda queries and direct MySQL connections - **Create repeatable database setups** for development and testing environments This approach provides several key benefits for database development: - **Consistent Environments**: Reproducible database setup across development, testing, and CI environments - **Faster Development Cycles**: Test database initialization scripts locally without AWS dependencies - **Cost-Effective Testing**: No AWS charges during development and testing phases - **Reliable CI/CD**: Automated database setup ensures consistent test environments The patterns demonstrated in this tutorial provide a solid foundation for managing database initialization in your LocalStack-based development workflow, enabling you to develop and test database-driven applications more efficiently and reliably. # Creating reproducible machine learning applications using Cloud Pods for persistent state snapshots > With LocalStack Cloud Pods, you can create persistent state snapshots to enable next-generation state management and team collaboration features for your local development environment. Learn how you can create reproducible machine learning applications & samples using Cloud Pods in LocalStack. ## Introduction [LocalStack Cloud Pods](/aws/developer-tools/snapshots/cloud-pods) enable you to create persistent state snapshots of your LocalStack instance, which can then be versioned, shared, and restored. It allows next-generation state management and team collaboration for your local cloud development environment, which you can utilize to create persistent shareable cloud sandboxes. Cloud Pods works directly with the [`lstk`](/aws/developer-tools/running-localstack/lstk/) CLI to save, merge, and restore snapshots of your LocalStack state. You can always tear down your LocalStack instance and restore it from a snapshot at any point in time. The `lstk snapshot` commands allow you to inspect your Cloud Pods, version them, and push them to the LocalStack platform for storage and collaboration. In this tutorial, we will use [LocalStack for AWS](/aws/getting-started/auth-token) to train a simple machine-learning model that recognizes handwritten digits on an image. We will rely on Cloud Pods to create a reproducible sample by using: - S3 to create a bucket to host our training data - Lambda to create a function to train and save the model to an S3 bucket - Lambda layer to host the dependencies for our training code - Lambda to create a secondary function to download and run some predictions with the saved model We will then create a Cloud Pod to save the state of our LocalStack instance and restore it from the Cloud Pod to share it with our team. ![Reproducible machine-learning applications with LocalStack Cloud Pods](/images/aws/reproducible_ml_application.png) ## Prerequisites For this tutorial, you will need the following: - [LocalStack for AWS](https://localstack.cloud/pricing/) - [`lstk aws`](/aws/connecting/aws-cli#localstack-aws-cli-lstk-aws) - [Optical recognition of handwritten digits dataset](https://github.com/localstack-samples/localstack-pro-samples/raw/refs/heads/master/reproducible-ml/digits.csv.gz) ([Source](https://archive.ics.uci.edu/ml/datasets/Optical+Recognition+of+Handwritten+Digits)) If you don't have a subscription to LocalStack for AWS, you can request a trial license upon sign-up. For this tutorial to work, you must have [`lstk`](/aws/getting-started/installation#lstk) installed. ## Training the machine learning model We will use the [Optical Recognition of Handwritten Digits Data Set](https://archive.ics.uci.edu/ml/datasets/Optical+Recognition+of+Handwritten+Digits) to train a simple machine-learning model to recognise handwritten texts. It contains images of individual digits, represented as arrays of pixel values, along with their corresponding labels, indicating the correct digit that each image represents. You can download the dataset from UCI's Machine Learning Repository (linked above) or from our [samples repository](https://github.com/localstack/localstack-pro-samples/tree/master/reproducible-ml). To train our model, we will upload our dataset on a local S3 bucket and use a Lambda function to train the model. Create a new file named `train.py` and import the required libraries: ```python import os import boto3 import numpy from sklearn import datasets, svm, metrics from sklearn.utils import Bunch from sklearn.model_selection import train_test_split from joblib import dump, load import io ``` We will now create a separate function named `load_digits` to load the dataset from the S3 bucket and return it as a `Bunch` object. The `Bunch` object is a container object that allows us to access the dataset's attributes as dictionary keys. It is similar to a Python dictionary but provides attribute-style access and can be used to store the dataset and its attributes. ```python def load_digits(*, n_class=10, return_X_y=False, as_frame=False): # download files from S3 s3_client = boto3.client("s3") s3_client.download_file(Bucket="reproducible-ml", Key="digits.csv.gz", Filename="/tmp/digits.csv.gz") data = numpy.loadtxt('/tmp/digits.csv.gz', delimiter=',') target = data[:, -1].astype(numpy.int, copy=False) flat_data = data[:, :-1] images = flat_data.view() images.shape = (-1, 8, 8) if n_class < 10: idx = target < n_class flat_data, target = flat_data[idx], target[idx] images = images[idx] feature_names = ['pixel_{}_{}'.format(row_idx, col_idx) for row_idx in range(8) for col_idx in range(8)] frame = None target_columns = ['target', ] if as_frame: frame, flat_data, target = datasets._convert_data_dataframe( "load_digits", flat_data, target, feature_names, target_columns) if return_X_y: return flat_data, target return Bunch(data=flat_data, target=target, frame=frame, feature_names=feature_names, target_names=numpy.arange(10), images=images) ``` The above code uses the `boto3` library to download the data file from an S3 bucket. The file is then loaded into a NumPy array using the `numpy.loadtxt` function, and the target values (i.e. the labels corresponding to each image) are extracted from the last column of the array. The images are then reshaped into 2-dimensional arrays, and the function has been configured to return only a subset of the available classes by filtering the target values. Finally, the function returns an object containing the data, target values, and metadata. Let us now define a `handler` function that would be executed by the Lambda every time a trigger event occurs. In this case, we would like to use the above function to load the dataset and train a model using the [Support Vector Machine (SVM)](https://scikit-learn.org/stable/modules/svm.html) algorithm. ```python def handler(event, context): digits = load_digits() # flatten the images n_samples = len(digits.images) data = digits.images.reshape((n_samples, -1)) # Create a classifier: a support vector classifier clf = svm.SVC(gamma=0.001) # Split data into 50% train and 50% test subsets X_train, X_test, y_train, y_test = train_test_split( data, digits.target, test_size=0.5, shuffle=False ) # Learn the digits on the train subset clf.fit(X_train, y_train) # Dump the trained model to S3 s3_client = boto3.client("s3") buffer = io.BytesIO() dump(clf, buffer) s3_client.put_object(Body=buffer.getvalue(), Bucket="reproducible-ml", Key="model.joblib") # Save the test-set to the S3 bucket numpy.save('/tmp/test-set.npy', X_test) with open('/tmp/test-set.npy', 'rb') as f: s3_client.put_object(Body=f, Bucket="reproducible-ml", Key="test-set.npy") ``` First, we loaded the images and flattened them into 1-dimensional arrays. Then, we created a training and a test set using the `train_test_split` function from the `sklearn.model_selection` module. We trained an SVM classifier on the training set using the `fit` method. Finally, we uploaded the trained model, together with the test set, to an S3 bucket for later usage. ## Perform predictions with the model Now, we will create a new file called `infer.py` which will contain a second handler function. This function will be used to perform predictions on new data with the model we trained previously. ```python import boto3 import numpy from joblib import load def handler(event, context): # download the model and the test set from S3 s3_client = boto3.client("s3") s3_client.download_file(Bucket="reproducible-ml", Key="test-set.npy", Filename="/tmp/test-set.npy") s3_client.download_file(Bucket="reproducible-ml", Key="model.joblib", Filename="/tmp/model.joblib") with open("/tmp/test-set.npy", "rb") as f: X_test = numpy.load(f) clf = load("/tmp/model.joblib") predicted = clf.predict(X_test) print("--> prediction result:", predicted) ``` To perform inference on the test set, we will download both the trained SVN model and the test set that we previously uploaded to the S3 bucket. Using these resources, we will predict the values of the digits in the test set. ## Deploying the Lambda functions Before creating our Lambda functions, let us start LocalStack to use emulated S3 and Lambda services to deploy and train our model. Let's start LocalStack: ```bash LOCALSTACK_DEBUG=1 lstk start ``` We have specified `LOCALSTACK_DEBUG=1` so that the logs from our Lambda invocations are recorded by LocalStack. Since `lstk start` runs the emulator in the background, we will read those logs with `lstk logs`. We can now create an S3 bucket to upload our Lambda functions and the dataset: ```bash zip lambda.zip train.py zip infer.zip infer.py lstk aws s3 mb s3://reproducible-ml lstk aws s3 cp lambda.zip s3://reproducible-ml/lambda.zip lstk aws s3 cp infer.zip s3://reproducible-ml/infer.zip lstk aws s3 cp digits.csv.gz s3://reproducible-ml/digits.csv.gz ``` In the above commands, we first create two zip files for our Lambda functions: lambda.zip and infer.zip. These zip files contain the code for training the machine learning model and do predictions with it, respectively. Next, we create an S3 bucket called `reproducible-ml` and upload the zip files and the dataset to it. Finally, we use the `lstk aws` CLI to create the two Lambda functions ```bash lstk aws lambda create-function --function-name ml-train \ --runtime python3.8 \ --role arn:aws:iam::000000000000:role/lambda-role \ --handler train.handler \ --timeout 600 \ --code '{"S3Bucket":"reproducible-ml","S3Key":"lambda.zip"}' \ --layers arn:aws:lambda:us-east-1:446751924810:layer:python-3-8-scikit-learn-0-23-1:2 ``` ```bash lstk aws lambda create-function --function-name ml-predict \ --runtime python3.8 \ --role arn:aws:iam::000000000000:role/lambda-role \ --handler infer.handler \ --timeout 600 \ --code '{"S3Bucket":"reproducible-ml","S3Key":"infer.zip"}' \ --layers arn:aws:lambda:us-east-1:446751924810:layer:python-3-8-scikit-learn-0-23-1:2 ``` For each function, we provide the function name, runtime (`python3.8`), handler function (`train.handler` and `infer.handler`, respectively), and the location of the `zip` files in the S3 bucket. We have also specified the `python-3-8-scikit-learn-0-23-1` layer to be used by the Lambda function. This layer includes the scikit-learn library and its dependencies. We can now invoke the first Lambda function using the `lstk aws` CLI: ```bash lstk aws lambda invoke --function-name ml-train /tmp/test.tmp ``` The first Lambda function will train the model and upload it to the S3 bucket. Finally, we can invoke the second Lambda function to do predictions with the model. ```bash lstk aws lambda invoke --function-name ml-predict /tmp/test.tmp ``` Each `invoke` call writes the function's return value to `/tmp/test.tmp` and prints the invocation status: ```bash title="Output" { "StatusCode": 200, "ExecutedVersion": "$LATEST" } ``` The prediction output itself is logged by LocalStack (with `LOCALSTACK_DEBUG=1` enabled). Retrieve it with the `logs` command: ```bash lstk logs ``` ```bash title="Output" 2026-08-18T20:06:25.174 DEBUG --- [et.reactor-2] l.p.c.s.l.i.version_manage : [ml-predict-f5e813df-cc00-41ef-bb88-5075f4d05ff7] START RequestId: f5e813df-cc00-41ef-bb88-5075f4d05ff7 Version: $LATEST 2026-08-18T20:06:25.174 DEBUG --- [et.reactor-2] l.p.c.s.l.i.version_manage : [ml-predict-f5e813df-cc00-41ef-bb88-5075f4d05ff7] --> prediction result: [8 8 4 9 0 8 9 8 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 9 6 7 8 9 ... 2026-08-18T20:06:25.177 DEBUG --- [et.reactor-2] l.p.c.s.l.i.version_manage : [ml-predict-f5e813df-cc00-41ef-bb88-5075f4d05ff7] 9 5 4 8 8 4 9 0 8 9 8] 2026-08-18T20:06:25.178 DEBUG --- [et.reactor-2] l.p.c.s.l.i.version_manage : [ml-predict-f5e813df-cc00-41ef-bb88-5075f4d05ff7] END RequestId: f5e813df-cc00-41ef-bb88-5075f4d05ff7 ``` You can also stream the logs with `lstk logs --follow`. ## Creating a Cloud Pod After deploying the Lambda functions, we can create a Cloud Pod to share our local infrastructure and instance state with other LocalStack users in the organization. To save the current state of our LocalStack instance, we can use the `save` command: ```bash lstk snapshot save pod:reproducible-ml ``` ```bash title="Output" Saving snapshot to pod "reproducible-ml"...... ✔︎ Snapshot saved to pod:reproducible-ml • Version: 1 • Services: lambda, cloudwatch, logs, s3, sts • Size: 40.8 MB ``` :::note You can also save a snapshot locally by specifying a plain path as an argument, instead of a `pod:` destination. To export on a local path, run the following command: ```bash lstk snapshot save / ``` The output of the above command will be a `.snapshot` file in the specified directory. We can restore it at any time with the `load` command. ::: To list available the Cloud Pods you can use the `list` command: ```bash lstk snapshot list ``` ```bash title="Output" Fetching snapshots... ~ 1 snapshots NAME VERSION LAST CHANGED reproducible-ml 1 2026-08-18 20:08 UTC ``` You can also inspect the contents of a Cloud Pod using the `show` command: ```bash lstk snapshot show pod:reproducible-ml ``` While you save a Cloud Pod, it is automatically published on the LocalStack platform and can be shared with other users in your organization. While saving an already existing Cloud Pod, we would create a new version, which is eventually uploaded to the LocalStack platform. You can check all the Cloud Pods in your organization over the [LocalStack Web Application](https://app.localstack.cloud/pods). Now that we have created a Cloud Pod, we can ask one of our team members to start LocalStack and load the Cloud Pod using the `load` command. ```bash lstk snapshot load pod:reproducible-ml ``` The `load` command will retrieve the content of our Cloud Pod named `reproducible-ml` from the LocalStack platform and inject it into our running LocalStack instance. Upon successfully loading the Cloud Pod, the Lambda function can be invoked again, and the log output should be the same as before. LocalStack Cloud Pods also feature different [merge strategies](/aws/developer-tools/snapshots/merging-snapshots/) to merge the state of a Cloud Pod with the current LocalStack instance. You can use the `--merge` flag to specify the merge strategy. The available merge strategies are: - **`account-region-merge`**: This is the default merge strategy. The state of the Cloud Pod wins wherever it overlaps with the running state on a (service, account, region) combination. - **`overwrite`**: This merge strategy wipes the running state, then loads the state of the Cloud Pod into the current LocalStack instance. - **`service-merge`**: This merge strategy combines non-overlapping resources, and the state of the Cloud Pod wins on a per-resource basis. ![State Merge mechanisms with LocalStack Cloud Pods](/images/aws/cloud-pods-state-merge-mechanisms.png) ## Testing the Application After deploying and invoking the Lambdas, first verify the end-to-end ML workflow via the data loading, training, and inference. After successfully running the application and saving a Cloud Pod, re-running the application after Pod restore should yield identical results. ### Expected Outputs from Training Invoke `ml-train` with: `lstk aws lambda invoke --function-name ml-train /tmp/test.tmp` - Logs show dataset load (1797 samples), training on 50% split, and S3 uploads for `model.joblib` and `test-set.npy`. - No explicit accuracy during training (focus is on savings), but the SVM classifier fits successfully. ### Expected Outputs from Inference (ml-predict Invocation) Invoke `ml-predict` with: `lstk aws lambda invoke --function-name ml-predict /tmp/test.tmp` - Downloads model and test set from S3. - Runs predictions on the test set (898 samples). - **Sample prediction result** (first 20): `[8 8 4 9 0 8 9 8 1 2 3 4 5 6 7 8 9 0 1 2]` - **Expected accuracy**: ~96.9% (calculated as `accuracy_score(y_test, predicted)`—e.g., 870/898 correct). Full logs in LocalStack output (with `LOCALSTACK_DEBUG=1`): --> prediction result: [8 8 4 9 0 8 9 8 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 9 6 7 8 9 ... 9 5 4 8 8 4 9 0 8 9 8] To compute accuracy locally (optional extension): Add to `infer.py` after predictions: ```python from sklearn.metrics import accuracy_score # Assuming y_test saved similarly y_test = np.load('y-test.npy') # You'd need to save this during training accuracy = accuracy_score(y_test, predicted) print(f"Model accuracy: {accuracy:.4f}") ``` Expected Model accuracy: 0.9689 ### Validation After Pod Restore - Save Pod: `lstk snapshot save pod:reproducible-ml` - (In a new instance) Load: `lstk snapshot load pod:reproducible-ml` - Re-invoke `ml-predict`: Outputs should match exactly, proving state persistence (S3 objects, Lambdas intact). If a mismatch occurs, check the Pod's merge strategy `(default: account-region-merge)` or logs for S3/Lambda errors. ## Conclusion In conclusion, LocalStack Cloud Pods facilitate collaboration and debugging among team members by allowing the sharing of local cloud infrastructure and instance state. These Cloud Pods can be used to create reproducible environments for various purposes, including machine learning. By using Cloud Pods, teams can work together to create a reproducible environment for their application and share it with other team members. Additionally, Cloud Pods can be used to pre-seed continuous integration (CI) pipelines with the necessary instance state to bootstrap testing environments or to troubleshoot failures in the CI pipeline. For more information about LocalStack Cloud Pods, refer to the documentation provided. The code for this tutorial, including a Makefile to execute it step-by-step, is available in the [`localstack-pro-samples` repository on GitHub](https://github.com/localstack/localstack-pro-samples/tree/master/reproducible-ml) on GitHub. # Chaos Engineering: Route53 Failover > Set up Route 53 failover to create a resilient, self-repairing infrastructure, which manages traffic effectively during simulated disruptions. ## Introduction LocalStack allows you to integrate and test [Chaos API](/aws/developer-tools/chaos-engineering/chaos-api) with [Route53](/aws/services/route53) to automatically divert users to a healthy secondary zone if the primary region fails, ensuring system availability and responsiveness. Route53's health checks and traffic redirection enhance architecture resilience and ensure service continuity during regional outages, crucial for uninterrupted user experiences. :::note Route53 Failover and Chaos API is currently available as part of the Ultimate plan. If you'd like to try it out, please [contact us](https://www.localstack.cloud/demo) to request access. ::: ## Getting started This tutorial is designed for users new to the Route53 and LocalStack Chaos API. In this example, there's an active-primary and passive-standby configuration. Route53 routes traffic to the primary region, which processes product-related requests through API Gateway and Lambda functions, with data stored in DynamoDB. If the primary region fails, Route53 redirects to the standby region, maintained in sync by a replication Lambda function. For this particular example, we'll be using a [sample application repository](https://github.com/localstack-samples/sample-chaos-serverless-multi-region-failover). Clone the repository, and follow the instructions below to get started. ### Prerequisites The general prerequisites for this guide are: - LocalStack for AWS with [LocalStack Auth Token](/aws/getting-started/auth-token) - [AWS CLI](/aws/connecting/aws-cli) with the [`lstk aws`](/aws/connecting/aws-cli#localstack-aws-cli-lstk-aws) command - [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) - [Python-3](https://www.python.org/downloads/) - `dig` Start LocalStack by using the `docker-compose.yml` file from the repository. Ensure to set your Auth Token as an environment variable during this process. ```bash LOCALSTACK_AUTH_TOKEN= docker compose up ``` Given that you've started LocalStack via `docker-compose`, you'll need to configure the `lstk` CLI to contact your container: ```bash export LSTK_ENDPOINT_URL=http://localhost.localstack.cloud:4566 ``` ### Architecture The following diagram shows the architecture that this application builds and deploys: ![Route53 Failover 1](/images/aws/route53-failover-1.png) ### Creating the resources To begin, deploy the same services in both `us-west-1` and `us-east-1` regions. The resources specified in the `init-resources.sh` file will be created when the LocalStack container starts, using [Initialization Hooks](/aws/customization/advanced/initialization-hooks) and the `awslocal` CLI tool. The objective is to have a backup system in case of a regional outage in the primary availability zone (`us-west-1`). We'll focus on this region to examine the existing resilience mechanisms. ![Route53 Failover 2](/images/aws/route53-failover-2.png) - The primary API Gateway includes a health check endpoint that returns a 200 HTTP status code, serving as a basic check for its availability. - Data synchronization across regions can be achieved with AWS-native tools like DynamoDB Streams and AWS Lambda. Here, any changes to the primary table trigger a Lambda function, replicating these changes to a secondary table. This configuration is essential for high availability and disaster recovery. ### Configuring a Route53 hosted zone Let's begin by setting up a hosted zone in Route53 named `hello-localstack.com` and retrieved the hosted zone ID: ```bash HOSTED_ZONE_NAME=hello-localstack.com HOSTED_ZONE_ID=$(lstk aws route53 create-hosted-zone --name $HOSTED_ZONE_NAME --caller-reference foo | jq -r .HostedZone.Id) ``` Then, define the health check ID for the API Gateway available in the `us-west-1` region: ```bash HEALTH_CHECK_ID=$( lstk aws route53 create-health-check \ --caller-reference foobar \ --health-check-config '{ "FullyQualifiedDomainName": "12345.execute-api.localhost.localstack.cloud", "Port": 4566, "ResourcePath": "/dev/healthcheck", "Type": "HTTP", "RequestInterval": 10 }' | jq -r .HealthCheck.Id ) ``` This command creates a Route 53 health check for an HTTP endpoint (`12345.execute-api.localhost.localstack.cloud:4566/dev/healthcheck`) with a 10-second request interval and captures the health check's ID. The caller reference identifier in AWS resource creation or updates prevents accidental duplication if requests are repeated. To update DNS records in the specified Route53 hosted zone (`$HOSTED_ZONE_ID`), add two CNAME records: `12345.$HOSTED_ZONE_NAME` pointing to `12345.execute-api.localhost.localstack.cloud`, and `67890.$HOSTED_ZONE_NAME` pointing to `67890.execute-api.localhost.localstack.cloud`. Set a TTL (Time to Live) of 60 seconds for these records. ```bash lstk aws route53 change-resource-record-sets \ --hosted-zone $HOSTED_ZONE_ID \ --change-batch '{ "Changes": [ { "Action": "CREATE", "ResourceRecordSet": { "Name": "12345.'$HOSTED_ZONE_NAME'", "Type": "CNAME", "TTL": 60, "ResourceRecords": [ {"Value": "12345.execute-api.localhost.localstack.cloud"} ] } }, { "Action": "CREATE", "ResourceRecordSet": { "Name": "67890.'$HOSTED_ZONE_NAME'", "Type": "CNAME", "TTL": 60, "ResourceRecords": [ {"Value": "67890.execute-api.localhost.localstack.cloud"} ] } } ] }' ``` Finally, we'll update the DNS records in the Route53 hosted zone identified by `$HOSTED_ZONE_ID`. We're adding two CNAME records for the subdomain `test.$HOSTED_ZONE_NAME`. The first record points to `12345.$HOSTED_ZONE_NAME` and is linked with the earlier created health check, designated as the primary failover target. The second record points to `67890.$HOSTED_ZONE_NAME` and is set as the secondary failover target. ```bash lstk aws route53 change-resource-record-sets \ --hosted-zone-id $HOSTED_ZONE_ID \ --change-batch '{ "Changes": [ { "Action": "CREATE", "ResourceRecordSet": { "Name": "test.'$HOSTED_ZONE_NAME'", "Type": "CNAME", "SetIdentifier": "12345", "AliasTarget": { "HostedZoneId": "'$HOSTED_ZONE_ID'", "DNSName": "12345.'$HOSTED_ZONE_NAME'", "EvaluateTargetHealth": true }, "HealthCheckId": "'$HEALTH_CHECK_ID'", "Failover": "PRIMARY" } }, { "Action": "CREATE", "ResourceRecordSet": { "Name": "test.'$HOSTED_ZONE_NAME'", "Type": "CNAME", "SetIdentifier": "67890", "AliasTarget": { "HostedZoneId": "'$HOSTED_ZONE_ID'", "DNSName": "67890.'$HOSTED_ZONE_NAME'", "EvaluateTargetHealth": true }, "Failover": "SECONDARY" } } ] }' ``` This setup represents the basic failover configuration where traffic is redirected to different endpoints based on their health check status. To confirm that the CNAME record for `test.hello-localstack.com` points to `12345.execute-api.localhost.localstack.cloud`, you can use the following `dig` command: ```bash dig @localhost test.hello-localstack.com CNAME ..... ;; QUESTION SECTION: ;test.hello-localstack.com. IN CNAME ;; ANSWER SECTION: test.hello-localstack.com. 300 IN CNAME 12345.execute-api.localhost.localstack.cloud. ..... ``` ### Testing the application Our setup is now complete and ready for testing. To mimic a regional outage in the `us-west-1` region, we'll configure the [Chaos API](/aws/developer-tools/chaos-engineering/chaos-api) to halt all service invocations in this region, including the health check function. Once the primary region becomes non-functional, Route 53's health checks will fail. This failure will activate the failover policy, redirecting traffic to the corresponding services in the secondary region, thus maintaining service continuity. ```bash curl -L -X POST 'http://localhost.localstack.cloud:4566/_localstack/chaos/faults' \ -H 'Content-Type: application/json' \ -d ' [ { "region": "us-west-1" } ]' ``` This will cause all services to fail in the `us-west-1` region with a 503 Service Unavailable error. Because of this, Route 53's health checks will detect the failure and redirect traffic to the standby region as per the failover setup. Confirm this redirection with the following command. Notice that the secondary endpoint is returned in the CNAME answer. ```bash dig @localhost test.hello-localstack.com CNAME ``` ```bash title="Output" ..... ;; QUESTION SECTION: ;test.hello-localstack.com. IN CNAME ;; ANSWER SECTION: test.hello-localstack.com. 300 IN CNAME 67890.execute-api.localhost.localstack.cloud. ..... ``` This indicates that the hosted zone name now points to the secondary API Gateway, and `us-east-1` services are in use. A Python script can simulate backend handling of this switch: ```python import dns.resolver import requests # Set the Route53 DNS resolver to use dns_resolver_ip = '127.0.0.1' # Domain to resolve domain_to_resolve = 'test.hello-localstack.com' # Resolve the CNAME record using the specified DNS server resolver = dns.resolver.Resolver(configure=False) resolver.nameservers = [dns_resolver_ip] try: cname_record = resolver.query(domain_to_resolve, rdtype=dns.rdatatype.CNAME) resolved_domain = str(cname_record[0].target) # Construct the full URL with the resolved domain resolved_url = f'http://{resolved_domain}:4566/dev/productApi?id=prod-1088' # Make an HTTP request to the resolved URL response = requests.get(resolved_url) # Print the response print(response.text) except dns.resolver.NXDOMAIN: print(f"CNAME record not found for {domain_to_resolve}") except Exception as e: print(f"Error: {e}") ``` Running the script will resolve the CNAME record for 'test.hello-localstack.com', make an HTTP request to the resolved URL, and print the response, which fetches a Product object from DynamoDB in the `us-east-1` region. ```bash python3 dns-resolver.py ``` ```bash title="Output" {"price":"29.99","name":"Super Widget","description":"A versatile widget that can be used for a variety of purposes. Durable, reliable, and affordable.","id":"prod-1088"} ``` The LocalStack logs will confirm which API Gateway was called based on the resolved domain. ```bash 2023-11-07T11:59:28.292 DEBUG --- [ asgi_gw_9] l.s.l.i.version_manager : > {resource: /productApi,path: /productApi,httpMethod: GET,headers: {Host=67890.execute-api.localhost.localstack.cloud:4566, User-Agent=python-requests/2.31.0, accept-encoding=gzip, deflate, accept=*/*, Connection=keep-alive, x-localstack-tgt-api=apigateway .... ``` ### Conclusion This tutorial demonstrated how to build a resilient, self-healing infrastructure using Route53 failover routing in combination with LocalStack's Chaos Engineering capabilities. Key takeaways include: - **Automatic Failover**: Route53 health checks continuously monitor endpoint health and automatically redirect traffic to standby regions when primary regions become unavailable. - **Data Resilience**: Cross-region data replication ensures business continuity during regional outages. - **Testing in Production-like Environments**: LocalStack's Chaos API enables safe testing of failure scenarios without impacting production systems. - **Reduced Recovery Time**: Automated failover mechanisms significantly reduce recovery time objectives (RTO) compared to manual intervention. - **Cost-Effective Testing**: LocalStack provides a cost-effective platform for validating disaster recovery procedures and ensuring organizational readiness for actual outage scenarios. Implementing this architecture helps organizations achieve higher availability SLAs and ensure uninterrupted service delivery even during regional disruptions. The combination of Route53 for intelligent traffic routing and cross-region replication for data redundancy creates a robust foundation for mission-critical applications. # Host a static website locally using Simple Storage Service (S3) and Terraform with LocalStack > Host a static website using a Simple Storage Service (S3) bucket to serve static content by provisioning the infrastructure using Terraform in LocalStack. Learn how to configure S3 buckets locally for testing and integration, and make use of LocalStack's S3 API & `lstk terraform` CLI to provision infrastructure locally. ## Introduction [AWS Simple Storage Service (S3)](https://aws.amazon.com/s3/) is a proprietary object storage solution that can store an unlimited number of objects for many use cases. S3 is a highly scalable, durable and reliable service that we can use for any use case involving file-based storage: hosting a static site, handling big data analytics, managing application logs, storing web assets and much more! With S3, objects are stored in buckets. A bucket is effectively a directory, while an object is a file. Every object (file) stores the name of the file (key), the contents (value), a version ID and the associated metadata. You can also use S3 to host to server static content as a static website. The static content might include HTML, CSS, JavaScript, images, and other assets that make up your website. LocalStack supports the S3 API, which means you can use the same API calls to interact with S3 in LocalStack as you would with AWS. Using LocalStack, you can create and manage S3 buckets and objects locally, use AWS SDKs and third-party integrations to work with S3, and test your applications without making any significant alterations. LocalStack also supports the creation of S3 buckets with static website hosting enabled. In this tutorial, we will deploy a static website using an S3 bucket over a locally emulated AWS infrastructure on LocalStack. We will use Terraform to automate the creation & management of AWS resources by declaring them in the HashiCorp Configuration Language (HCL). We will also learn about `lstk terraform`, part of the `lstk` CLI, that allows you to run Terraform locally against LocalStack. ## Prerequisites For this tutorial, you will need: - [LocalStack for AWS](https://www.localstack.cloud/localstack-for-aws) - [Terraform](https://www.terraform.io/downloads.html) - [`lstk aws`](/aws/connecting/aws-cli#localstack-aws-cli-lstk-aws) ## Architecture The following diagram illustrates the architecture of the static website hosting setup using S3 and Terraform: ![Architecture](/images/aws/s3-static-website-terraform-diagram.png) In this architecture: - A browser makes an HTTP request to the S3 website endpoint - LocalStack's S3 service serves the static content from the configured bucket - The bucket contains HTML files and optional assets - Terraform provisions and configures all resources locally ## Creating a static website We will create a simple static website using plain HTML to get started. To create a static website deployed over S3, we need to create an index document and a custom error document. We will name our index document `index.html` and our error document `error.html`. Let's create a directory named `s3-static-website-localstack` where we'll store our project files, with a `www/` subdirectory that holds the website content: ```bash mkdir -p s3-static-website-localstack/www cd s3-static-website-localstack ``` Inside `www/`, create an `index.html` file with the following content: ```html showLineNumbers Static Website

Static Website deployed locally over S3 using LocalStack

``` S3 will serve this file when a user visits the root URL of your static website, serving as the default page. In a similar fashion, you can configure a custom error document that contains a user-friendly error message. Create a file named `error.html` next to `index.html` inside `www/` and add the following code: ```html showLineNumbers 404

Something is amiss.

``` S3 will return the above file content only for HTTP 4XX error codes. Some browsers might choose to display their custom error message if a user tries to access a resource that does not exist. In this case, browsers might ignore the above error document. With the initial setup complete, we can now move on to creating a static website using S3 via `lstk aws`, LocalStack's wrapper for the AWS CLI. ## Hosting a static website using S3 To create a static website using S3, we need to create a bucket, enable static website hosting, and upload the files to the bucket. We will use the `lstk aws` CLI for these operations. Navigate to the root directory of the project and create a bucket named `testwebsite` using LocalStack's S3 API: ```bash lstk aws s3api create-bucket --bucket testwebsite ``` With the bucket created, we can now attach a policy to it to allow public access and its contents. Let's create a file named `bucket_policy.json` in the project root (next to the `www/` folder) and add the following code: ```json showLineNumbers { "Version": "2012-10-17", "Statement": [ { "Sid": "PublicReadGetObject", "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::testwebsite/*" } ] } ``` Let's now attach the policy to the bucket: ```bash lstk aws s3api put-bucket-policy --bucket testwebsite --policy file://bucket_policy.json ``` With the policy attached, we can now sync the contents of our `www/` directory to the bucket: ```bash lstk aws s3 sync ./www/ s3://testwebsite ``` We'll now enable static website hosting on the bucket and configure the index and error documents: ```bash lstk aws s3 website s3://testwebsite/ --index-document index.html --error-document error.html ``` If you are deploying a static website using S3 on real AWS cloud, your S3 website endpoint will follow one of these two formats: - `http://.s3-website-.amazonaws.com` - `http://.s3-website..amazonaws.com` In LocalStack, the S3 website endpoint follows the following format: `http://.s3-website.localhost.localstack.cloud:4566`. You can navigate to [`http://testwebsite.s3-website.localhost.localstack.cloud:4566/`](http://testwebsite.s3-website.localhost.localstack.cloud:4566/) to view your static website. ## Orchestrating infrastructure using Terraform You can automate the above process by orchestrating your AWS infrastructure using Terraform. Terraform is an infrastructure as code (IaC) tool that allows you to create, manage, and version your infrastructure. Terraform uses a declarative configuration language called HashiCorp Configuration Language (HCL) to describe your infrastructure. Before that, we would need to manually configure the local service endpoints and credentials for Terraform to integrate with LocalStack. We will use the [AWS Provider for Terraform](https://registry.terraform.io/providers/hashicorp/aws/latest/docs) to interact with the many resources supported by AWS in LocalStack. Create a new file named `provider.tf` and specify mock credentials for the AWS provider: ```hcl showLineNumbers provider "aws" { region = "us-east-1" access_key = "fake" secret_key = "fake" } ``` We would also need to avoid issues with routing and authentication (as we do not need it). Therefore we need to supply some general parameters. Additionally, we have to point the individual services to LocalStack. We can do this by specifying the `endpoints` parameter for each service that we intend to use. Our `provider.tf` file should look like this: ```hcl showLineNumbers provider "aws" { access_key = "test" secret_key = "test" region = "us-east-1" # only required for non virtual hosted-style endpoint use case. # https://registry.terraform.io/providers/hashicorp/aws/latest/docs#s3_use_path_style s3_use_path_style = false skip_credentials_validation = true skip_metadata_api_check = true endpoints { s3 = "http://s3.localhost.localstack.cloud:4566" s3control = "http://localhost.localstack.cloud:4566" } } ``` :::note We use `localhost.localstack.cloud` as the recommended endpoint for the S3 to enable host-based bucket endpoints. Users can rely on the `localhost.localstack.cloud` domain to be publicly resolvable. We also publish an SSL certificate which is automatically used inside LocalStack to enable HTTPS endpoints with valid certificates. For most of the other services, it is fine to use `localhost:4566`. ::: :::note The `s3control` endpoint is required because recent versions of the AWS Terraform provider use the S3 Control API to read bucket tags. We point it at `localhost.localstack.cloud` so the account-id-prefixed hostname (`.localhost.localstack.cloud`) resolves to LocalStack. ::: With the provider configured, we can now configure the variables for our S3 bucket. Create a new file named `variables.tf` and add the following code: ```hcl showLineNumbers variable "bucket_name" { description = "Name of the s3 bucket. Must be unique." type = string } variable "tags" { description = "Tags to set on the bucket." type = map(string) default = {} } ``` We take a user input for the bucket name and tags. Next, we will define the output variables for our Terraform configuration. Create a new file named `outputs.tf` and add the following code: ```hcl showLineNumbers output "arn" { description = "ARN of the bucket" value = aws_s3_bucket.s3_bucket.arn } output "name" { description = "Name (id) of the bucket" value = aws_s3_bucket.s3_bucket.id } output "domain" { description = "Domain name of the bucket" value = "s3-website.localhost.localstack.cloud:4566" } output "website_endpoint" { description = "Website endpoint URL" value = "http://${aws_s3_bucket.s3_bucket.id}.s3-website.localhost.localstack.cloud:4566" } ``` The output variables are the ARN, name, LocalStack S3 website domain, and the full website endpoint URL of the bucket. We hardcode the `domain` and `website_endpoint` values to point at LocalStack so that the outputs surface a URL you can open directly. The native `aws_s3_bucket_website_configuration` attributes return the AWS-formatted endpoint (`.s3-website-.amazonaws.com`), which would be misleading in a LocalStack-only setup. With all the configuration files in place, we can now create the S3 bucket. Create a new file named `main.tf` and create the S3 bucket using the following code: ```hcl showLineNumbers resource "aws_s3_bucket" "s3_bucket" { bucket = var.bucket_name tags = var.tags } ``` To configure the static website hosting, we will use the `aws_s3_bucket_website_configuration` resource. Add the following code to the `main.tf` file: ```hcl showLineNumbers resource "aws_s3_bucket_website_configuration" "s3_bucket" { bucket = aws_s3_bucket.s3_bucket.id index_document { suffix = "index.html" } error_document { key = "error.html" } } ``` To set the bucket policy, we will use the `aws_s3_bucket_policy` resource. Add the following code to the `main.tf` file: ```hcl showLineNumbers resource "aws_s3_bucket_acl" "s3_bucket" { bucket = aws_s3_bucket.s3_bucket.id acl = "public-read" } resource "aws_s3_bucket_policy" "s3_bucket" { bucket = aws_s3_bucket.s3_bucket.id policy = jsonencode({ Version = "2012-10-17" Statement = [ { Sid = "PublicReadGetObject" Effect = "Allow" Principal = "*" Action = "s3:GetObject" Resource = [ aws_s3_bucket.s3_bucket.arn, "${aws_s3_bucket.s3_bucket.arn}/*", ] }, ] }) } ``` In the above code, we are setting the ACL of the bucket to `public-read` and setting the bucket policy to allow public access to the bucket. Pick up an appropriate policy based on your use case. Let's use the `aws_s3_object` resource to upload the files to the bucket. Add the following code to the `main.tf` file: ```hcl showLineNumbers resource "aws_s3_object" "object_www" { depends_on = [aws_s3_bucket.s3_bucket] for_each = fileset("${path.root}", "www/*.html") bucket = var.bucket_name key = basename(each.value) source = each.value etag = filemd5("${each.value}") content_type = "text/html" acl = "public-read" } ``` The above code uploads every `.html` file under `www/` to the bucket and sets each object's ACL to `public-read`. With all the configuration files in place, we can now initialize the Terraform configuration. Run the following command to initialize the Terraform configuration: ```bash terraform init ... Terraform has been successfully initialized! ... ``` We can create an execution plan based on our Terraform configuration for the AWS resources. Run the following command to create an execution plan: ```bash terraform plan ``` Finally, we can apply the Terraform configuration to create the AWS resources. Run the following command to apply the Terraform configuration: ```bash terraform apply var.bucket_name Name of the s3 bucket. Must be unique. Enter a value: testwebsite ... arn = "arn:aws:s3:::testwebsite" domain = "s3-website.localhost.localstack.cloud:4566" name = "testwebsite" website_endpoint = "http://testwebsite.s3-website.localhost.localstack.cloud:4566" ``` In the above command, we specified `testwebsite` as the bucket name to keep it consistent with the `lstk aws` flow above and the testing commands further down. You can specify any bucket name since LocalStack is ephemeral, and stopping your LocalStack container will delete all the created resources. The above command output includes the ARN, name, LocalStack website domain, and the website endpoint URL of the bucket. You can navigate directly to the printed `website_endpoint` to view your site, since the endpoint uses `localhost.localstack.cloud`, no real AWS resources have been created. You can optionally use the `lstk terraform` command as a drop-in replacement for the official Terraform CLI. `lstk terraform` uses the Terraform Override mechanism to create a temporary `localstack_providers_override.tf` file, which is deleted after the infrastructure is created. It mitigates the need to create the `provider.tf` file manually. You can use `lstk terraform` to create the infrastructure by running the following commands: ```bash lstk terraform init lstk terraform plan lstk terraform apply ``` ## Testing the application After deploying your static website, it's important to verify that everything is working correctly. Here are several ways to test your S3-hosted static website: ### Accessing the website Navigate to the LocalStack S3 website endpoint in your browser: ``` http://testwebsite.s3-website.localhost.localstack.cloud:4566/ ``` You should see your `index.html` content displayed, which in our case shows: "Static Website deployed locally over S3 using LocalStack". ### Testing with curl You can also test the website using `curl` from your terminal: ```bash curl http://testwebsite.s3-website.localhost.localstack.cloud:4566/ ``` This should return the HTML content of your `index.html` file. ### Verifying the error page To test the custom error document, try accessing a non-existent page: ```bash curl http://testwebsite.s3-website.localhost.localstack.cloud:4566/nonexistent.html ``` You should receive the content from your `error.html` file: "Something is amiss." with an appropriate HTTP 4XX status code. ### Checking bucket configuration You can verify the bucket's website configuration using `lstk aws`: ```bash lstk aws s3api get-bucket-website --bucket testwebsite ``` This command should return the index and error document configuration for your bucket. ### Listing bucket contents To confirm all your files were uploaded correctly: ```bash lstk aws s3 ls s3://testwebsite/ ``` This will display all the files in your bucket, including `index.html`, `error.html`, and any additional assets. ## Conclusion In this tutorial, we have seen how to use LocalStack to create an S3 bucket and configure it to serve a static website. We have also seen how you can use Terraform to provision AWS infrastructure in an emulated local environment using LocalStack. You can use the [LocalStack App](https://app.localstack.cloud) to view the created buckets and files on the LocalStack Resource dashboard for S3 and upload more files or perform other operations on the bucket. Using LocalStack, you can perform various operations using emulated S3 buckets and other AWS services without creating any real AWS resources. The code for this tutorial can be found in our [LocalStack Terraform samples over GitHub](https://github.com/localstack/localstack-terraform-samples/tree/master/s3-static-website). Please make sure to adjust the paths for the HTML files in `main.tf`. Further documentation for S3 is available on our [S3 documentation](/aws/services/s3). # Schema Evolution with Glue Schema Registry and Managed Streaming for Kafka (MSK) using LocalStack > Find incompatibilities early or even avoid them altogether when developing Kafka producers or consumers! Learn how to test data schema evolution by using Managed Streaming for Kafka (MSK) with the Glue Schema Registry in LocalStack. ## Introduction [Apache Kafka](https://kafka.apache.org/) is an open-source distributed event store and stream-processing platform. It is used to capture data generated by producers and distribute it among its consumers. Kafka is known for its scalability, with reports of production environments scaling to [trillions of messages per day](https://engineering.linkedin.com/blog/2019/apache-kafka-trillion-messages). With [Amazon Managed Streaming for Apache Kafka (MSK)](https://aws.amazon.com/msk/), AWS provides a service to provision Apache Kafka clusters easily. [LocalStack for AWS](https://app.localstack.cloud/) supports [Amazon Managed Streaming for Kafka (MSK)](/aws/services/kafka/), which enables you to spin up Kafka clusters on your local machine and test the integration of your applications with Amazon MSK. Kafka clusters are often used as the central messaging infrastructure in complex microservice environments. However, the continuous and independent development of the individual microservices - the data producers and consumers - can make it hard to coordinate and evolve data schemas over time without introducing application failures due to incompatibilities. A common solution to this problem is to use a schema registry which provides for the validation of schema changes, preventing any unsafe changes and subsequent application failures. [AWS Glue Schema Registry](https://docs.aws.amazon.com/glue/latest/dg/schema-registry.html) can be used as such a schema registry, enabling you to validate and evolve streaming data using Apache Avro schemas. It can be easily integrated into Java applications for Apache Kafka with [AWS's official open-source serializers and deserializers](https://github.com/awslabs/aws-glue-schema-registry).
The following chart shows the integration of producers and consumers with Amazon MSK and the AWS Glue Schema Registry: ![Workflow: Glue Schema Registry with MSK](/images/aws/schema-evolution-glue-msk-flow.svg) 1. Before sending a record, the producer validates that the schema it is using to serialize its records is valid. We can configure the producer to register a new schema version if the schema is not yet registered. - When registering the new schema version, the schema registry validates if the schema is compatible. - If the registry detects an incompatibility, the registration is rejected. This ensures that a producer fails early and cannot publish incompatible records in the first place. 2. Once the schema is valid, the producer serializes and compresses the record and sends it to the Kafka cluster. 3. The consumer reads the serialized and compressed record. 4. The consumer requests the schema from the schema registry (if it is not already cached) and uses the schema to decompress and deserialize the record. [AWS Glue Schema Registry](/aws/services/glue) is supported by LocalStack for AWS as well, ultimately allowing you to test the evolution of your data streaming application completely on your local machine. It allows you develop and test your application's data schema evolution locally. The code for this tutorial (including a script to execute it step-by-step) can be found in our [`localstack-pro-samples` repository on GitHub](https://github.com/localstack/localstack-pro-samples/tree/master/glue-msk-schema-registry). # Prerequisites For this tutorial you will need: - [LocalStack for AWS](https://localstack.cloud/pricing/) to emulate Amazon MSK and AWS Glue Schema Registry locally - Don't worry, if you don't have a subscription yet, you can just get a trial license for free. - [`lstk aws`](/aws/connecting/aws-cli#localstack-aws-cli-lstk-aws) - Java 11+ - Maven 3 ## Initial schema At first, we will define our schema, set up our Java project, and generate the Java data classes using the schema. In our Apache Avro data schema we describe a request to ride a unicorn, including the necessary addresses, a fare, a duration, some preferences, and a customer record: ```json { "type": "record", "name": "UnicornRideRequest", "namespace": "cloud.localstack.demos.gluemsk.schema", "fields": [ {"name": "request_id", "type": "int", "doc": "customer request id"}, {"name": "pickup_address","type": "string","doc": "customer pickup address"}, {"name": "destination_address","type": "string","doc": "customer destination address"}, {"name": "ride_fare","type": "float","doc": "ride fare amount (USD)"}, {"name": "ride_duration","type": "int","doc": "ride duration in minutes"}, {"name": "preferred_unicorn_color","type": {"type": "enum","name": "UnicornPreferredColor","symbols": ["WHITE","BLACK","RED","BLUE","GREY"]}, "default": "WHITE"}, { "name": "recommended_unicorn", "type": { "type": "record", "name": "RecommendedUnicorn", "fields": [ {"name": "unicorn_id","type": "int", "doc": "recommended unicorn id"}, {"name": "color","type": {"type": "enum","name": "unicorn_color","symbols": ["WHITE","RED","BLUE"]}}, {"name": "stars_rating", "type": ["null", "int"], "default": null, "doc": "unicorn star ratings based on customers feedback"} ] } }, { "name": "customer", "type": { "type": "record", "name": "Customer", "fields": [ {"name": "customer_account_no","type": "int", "doc": "customer account number"}, {"name": "first_name","type": "string"}, {"name": "middle_name","type": ["null","string"], "default": null}, {"name": "last_name","type": "string"}, {"name": "email_addresses","type": ["null", {"type":"array", "items":"string" }]}, {"name": "customer_address","type": "string","doc": "customer address"}, {"name": "mode_of_payment","type": {"type": "enum","name": "ModeOfPayment","symbols": ["CARD","CASH"]}, "default": "CARD"}, {"name": "customer_rating", "type": ["null", "int"], "default": null} ] } } ] } ``` ## Project setup Now we can set up our Java project with one module for the `producer` and another module for the `consumer`. Both modules have their schema in the `src/main/resources` folder. ```bash . ├── consumer │ ├── pom.xml │ └── src │ └── main │ └── resources │ └── avro │ └── unicorn_ride_request_v1.avsc ├── producer │ ├── pom.xml │ └── src │ └── main │ └── resources │ └── avro │ └── unicorn_ride_request_v1.avsc └── pom.xml ``` In our root pom, we configure the `producer` and the `consumer` module, some shared dependencies (most notably `software.amazon.glue:schema-registry-serde`), and the `avro-maven-plugin` to generate Java classes for the schema: ```xml 4.0.0 cloud.localstack.demos.gluemsk root-pom 1.0-SNAPSHOT pom Glue MSK Demo producer consumer UTF-8 11 11 software.amazon.glue schema-registry-serde 1.1.10 org.slf4j slf4j-api 1.7.36 org.slf4j slf4j-reload4j 1.7.36 org.apache.avro avro 1.11.0 com.beust jcommander 1.82 org.apache.avro avro-maven-plugin 1.11.0 generate-sources schema ${project.basedir}/src/main/resources/avro/ ${project.basedir}/src/main/java/ ``` While the root pom is a bit lengthy, the `pom.xml` files of the two modules are quite simple. They only reference the root pom, activate the `avro-maven-plugin`, and define a main class (we'll go into detail on the actual Java code in [the section below](#implementing-a-producer-and-consumer)). Here is what the producer's `pom.xml` looks like: ```xml 4.0.0 cloud.localstack.demos.gluemsk root-pom 1.0-SNAPSHOT ../pom.xml producer 1.0-SNAPSHOT Glue MSK Demo Producer cloud.localstack.demos.gluemsk.producer.Producer org.apache.avro avro-maven-plugin ``` And similarly the consumer's `pom.xml` looks like: ```xml 4.0.0 cloud.localstack.demos.gluemsk root-pom 1.0-SNAPSHOT ../pom.xml consumer 1.0-SNAPSHOT Glue MSK Demo Consumer cloud.localstack.demos.gluemsk.consumer.Consumer org.apache.avro avro-maven-plugin ``` Now the project is all set up and we can already generate our schema classes from the AVRO schema using the `avro-maven-plugin`: ```bash mvn clean generate-sources ``` After the maven plugin is done, we have all types generated for both the producer and the consumer: ```plaintext . ├── consumer │ └── src │ └── main │ ├── java │ └── cloud │ └── localstack │ └── demos │ └── gluemsk │ └── schema │ ├── Customer.java │ ├── ModeOfPayment.java │ ├── RecommendedUnicorn.java │ ├── unicorn_color.java │ ├── UnicornPreferredColor.java │ └── UnicornRideRequest.java └── producer └── src └── main └── java └── cloud └── localstack └── demos └── gluemsk └── schema ├── Customer.java ├── ModeOfPayment.java ├── RecommendedUnicorn.java ├── unicorn_color.java ├── UnicornPreferredColor.java └── UnicornRideRequest.java ``` Since we want to log the sent and received messages, we need to configure the logging by adding a `log4j.properties` to the `src/main/resources` folder of the two modules: ```properties log4j.rootLogger=DEBUG, CONSOLE log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout log4j.appender.CONSOLE.layout.ConversionPattern=[%c{1}][%-5p] %m%n ``` ## Implementing a Producer and Consumer Now, all the boilerplate is done: - We have a proper project setup. - The logging is properly configured. - We generated our schema classes which we'll use as a typesafe interface for the records. ### The Producer The next step is to implement our producer. The complete module can be found on our [samples repository (along with the rest of the code of this tutorial)](https://github.com/localstack/localstack-pro-samples/blob/cd023a84a3b473984e9c34053d4feb7de8e038c1/glue-msk-schema-registry/producer/). We create a new class called `Producer` in `producer/src/main/java/cloud/localstack/demos/gluemsk/producer/`. The `Producer` contains a `main` method which uses [`jcommander`](https://jcommander.org/) to create a simple CLI interface: ```java package cloud.localstack.demos.gluemsk.producer; import com.beust.jcommander.JCommander; import com.beust.jcommander.Parameter; import org.slf4j.Logger; import org.slf4j.LoggerFactory; public class Producer { private final static Logger LOGGER = LoggerFactory.getLogger(org.apache.kafka.clients.producer.Producer.class.getName()); @Parameter(names = {"--help", "-h"}, help = true) protected boolean help = false; @Parameter(names = {"--bootstrap-servers", "-bs"}, description = "Kafka bootstrap servers endpoint to connect to.") protected String bootstrapServers = "localhost:4511"; @Parameter(names = {"--aws-endpoint-servers", "-ae"}, description = "AWS endpoint to use.") protected String awsEndpoint = "https://localhost.localstack.cloud:4566"; @Parameter(names = {"--region", "-reg"}, description = "AWS Region to use.") protected String regionName = "us-east-1"; @Parameter(names = {"--topic-name", "-topic"}, description = "Kafka topic name where you send the data records. Default is unicorn-ride-request-topic.") protected String topic = "unicorn-ride-request-topic"; @Parameter(names = {"--num-messages", "-nm"}, description = "Number of messages you want producer to send. Default is 100.") protected String str_numOfMessages = "100"; public static void main(String[] args) { Producer producer = new Producer(); JCommander jc = JCommander.newBuilder().addObject(producer).build(); jc.parse(args); if (producer.help) { jc.usage(); return; } producer.startProducer(); } public void startProducer() { // TODO: // - Create the producer // - Send the UnicornRideRequest records to the topic } } ``` Now we can add a method to configure our producer: ```java private Properties getProducerConfig() { Properties props = new Properties(); // use the "--bootstrap-servers" argument to define the Kafka bootstrap address to connect to props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, this.bootstrapServers); props.put(ProducerConfig.ACKS_CONFIG, "-1"); props.put(ProducerConfig.CLIENT_ID_CONFIG, "glue-msk-demo-producer"); props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); // use the GlueSchemaRegistryKafkaSerializer from software.amazon.glue:schema-registry-serde props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, GlueSchemaRegistryKafkaSerializer.class.getName()); // configure the GlueSchemaRegistryKafkaSerializer (data format, region, Glue registry and schema,...) props.put(AWSSchemaRegistryConstants.DATA_FORMAT, DataFormat.AVRO.name()); props.put(AWSSchemaRegistryConstants.AWS_REGION, regionName); props.put(AWSSchemaRegistryConstants.REGISTRY_NAME, "unicorn-ride-request-registry"); props.put(AWSSchemaRegistryConstants.SCHEMA_NAME, "unicorn-ride-request-schema-avro"); props.put(AWSSchemaRegistryConstants.AVRO_RECORD_TYPE, AvroRecordType.SPECIFIC_RECORD.getName()); // define the endpoint to use - by default we use LocalStack (localhost.localstack.cloud) props.put(AWSSchemaRegistryConstants.AWS_ENDPOINT, this.awsEndpoint); // enable compression of records props.put(AWSSchemaRegistryConstants.COMPRESSION_TYPE, AWSSchemaRegistryConstants.COMPRESSION.ZLIB.name()); return props; } ``` In addition, we create a method to generate new dummy `UnicornRideRequests` and a `Callback` class logging the producer metadata: ```java public UnicornRideRequest getRecord(int requestId) { /* Initialise UnicornRideRequest object of class that is generated from AVRO Schema */ UnicornRideRequest rideRequest = UnicornRideRequest.newBuilder() .setRequestId(requestId) .setPickupAddress("Melbourne, Victoria, Australia") .setDestinationAddress("Sydney, NSW, Aus") .setRideFare(1200.50F) .setRideDuration(120) .setPreferredUnicornColor(UnicornPreferredColor.WHITE) .setRecommendedUnicorn(RecommendedUnicorn.newBuilder() .setUnicornId(requestId * 2) .setColor(unicorn_color.WHITE) .setStarsRating(5).build()) .setCustomer(Customer.newBuilder() .setCustomerAccountNo(1001) .setFirstName("Dummy") .setLastName("User") .setEmailAddresses(List.of("demo@example.com")) .setCustomerAddress("Flinders Street Station") .setModeOfPayment(ModeOfPayment.CARD) .setCustomerRating(5).build()).build(); LOGGER.info(rideRequest.toString()); return rideRequest; } private static class ProducerCallback implements Callback { @Override public void onCompletion(RecordMetadata recordMetaData, Exception e) { if (e == null) { LOGGER.info("Received new metadata. \t" + "Topic:" + recordMetaData.topic() + "\t" + "Partition: " + recordMetaData.partition() + "\t" + "Offset: " + recordMetaData.offset() + "\t" + "Timestamp: " + recordMetaData.timestamp()); } else { LOGGER.info("There's been an error from the Producer side"); e.printStackTrace(); } } } ``` And finally, we can implement our `startProducer` method which creates the producer using `getProducerConfig` and sends records generated by `getRecord`: ```java public void startProducer() { try (KafkaProducer producer = new KafkaProducer<>(getProducerConfig())) { int numberOfMessages = Integer.parseInt(str_numOfMessages); LOGGER.info("Starting to send records..."); for (int i = 0; i < numberOfMessages; i++) { UnicornRideRequest rideRequest = getRecord(i); String key = "key-" + i; ProducerRecord record = new ProducerRecord<>(topic, key, rideRequest); producer.send(record, new ProducerCallback()); } } } ``` ### The Consumer Now that we have a `Producer`, we need a component which reads the data from the Kafka cluster. The complete module can be found on our [samples repository (along with the rest of the code of this tutorial).](https://github.com/localstack/localstack-pro-samples/blob/cd023a84a3b473984e9c34053d4feb7de8e038c1/glue-msk-schema-registry/consumer/) We create a new class called `Consumer` in `consumer/src/main/java/cloud/localstack/demos/gluemsk/consumer/`. Analogous to the `Producer`, the `Consumer` contains a `main` method which uses [`jcommander`](https://jcommander.org/) to create a simple CLI interface: ```java package cloud.localstack.demos.gluemsk.consumer; import com.beust.jcommander.JCommander; import com.beust.jcommander.Parameter; import org.slf4j.Logger; import org.slf4j.LoggerFactory; public class Consumer { private final static Logger LOGGER = LoggerFactory.getLogger(java.util.function.Consumer.class.getName()); @Parameter(names = {"--help", "-h"}, help = true) protected boolean help = false; @Parameter(names = {"--bootstrap-servers", "-bs"}, description = "kafka bootstrap servers endpoint") protected String bootstrapServers = "localhost:4511"; @Parameter(names = {"--aws-endpoint-servers", "-ae"}, description = "AWS endpoint") protected String awsEndpoint = "https://localhost.localstack.cloud:4566"; @Parameter(names = {"--region", "-reg"}, description = "AWS Region to use.") protected String regionName = "us-east-1"; @Parameter(names = {"--topic-name", "-topic"}, description = "Kafka topic name where you send the data records. Default is unicorn-ride-request-topic") protected String topic = "unicorn-ride-request-topic"; @Parameter(names = {"--num-messages", "-nm"}, description = "Number of messages you want consumer to wait for until it stops. Default is 100, use 0 if you want it to run indefinitely.") protected String str_numOfMessages = "100"; public static void main(String[] args) { Consumer consumer = new Consumer(); JCommander jc = JCommander.newBuilder().addObject(consumer).build(); jc.parse(args); if (consumer.help) { jc.usage(); return; } consumer.startConsumer(); } public void startConsumer() { // TODO: // - Create the Kafka Consumer // - Subscribe to the topic // - Process the incoming records } } ``` Similar to the producer, we need to configure our consumer: ```java private Properties getConsumerConfig() { Properties props = new Properties(); // use the "--bootstrap-servers" argument to define the Kafka bootstrap address to connect to props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, this.bootstrapServers); props.put(ConsumerConfig.GROUP_ID_CONFIG, "unicorn.riderequest.consumer"); props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest"); props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); // use the GlueSchemaRegistryKafkaDeserializer from software.amazon.glue:schema-registry-serde props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, GlueSchemaRegistryKafkaDeserializer.class.getName()); props.put(AWSSchemaRegistryConstants.AWS_REGION, this.regionName); props.put(AWSSchemaRegistryConstants.AVRO_RECORD_TYPE, AvroRecordType.SPECIFIC_RECORD.getName()); // define the endpoint to use - by default we use LocalStack (localhost.localstack.cloud) props.put(AWSSchemaRegistryConstants.AWS_ENDPOINT, this.awsEndpoint); return props; } ``` And finally, we can implement our `startConsumer` method which creates the consumer using `getConsumerConfig`, reads the records from the topic, and logs them: ```java public void startConsumer() { LOGGER.info("Starting consumer..."); try (KafkaConsumer consumer = new KafkaConsumer<>(getConsumerConfig())) { consumer.subscribe(Collections.singletonList(topic)); int outstandingMessages = Integer.parseInt(str_numOfMessages); boolean runIndefinitely = outstandingMessages == 0; while (outstandingMessages > 0 || runIndefinitely) { // a real consumer would probably run in an endless loop waiting for new records here final ConsumerRecords records = consumer.poll(Duration.ofMillis(10)); for (final ConsumerRecord record : records) { final UnicornRideRequest rideRequest = record.value(); LOGGER.info(String.valueOf(rideRequest.getRequestId())); LOGGER.info(rideRequest.toString()); outstandingMessages--; } } } LOGGER.info("Stopping consumer..."); } ``` ## Setting up the infrastructure Now that the initial coding is done, we can give it a try. Let's start LocalStack: ```bash lstk start ``` Once LocalStack is started, we can create a new Kafka cluster using `lstk aws`: ```bash lstk aws kafka create-cluster \ --cluster-name "unicorn-ride-cluster" \ --kafka-version "2.2.1" \ --number-of-broker-nodes 1 \ --broker-node-group-info "{\"ClientSubnets\": [], \"InstanceType\":\"kafka.m5.xlarge\"}" ``` ```bash title="Output" { "ClusterArn": "arn:aws:kafka:us-east-1:000000000000:cluster/unicorn-ride-cluster/f9b16124-baf3-459b-8507-ec6c605b7a0a-25", "ClusterName": "unicorn-ride-cluster", "State": "CREATING" } ``` The `ClusterArn` is created dynamically and will be different for your run. Make sure to use your `ClusterArn` for the commands below. It takes some time for the cluster to get up and running. We can monitor the state with `describe-cluster`: ```bash lstk aws kafka describe-cluster --cluster-arn "arn:aws:kafka:us-east-1:000000000000:cluster/unicorn-ride-cluster/f9b16124-baf3-459b-8507-ec6c605b7a0a-25" ``` ```bash title="Output" { "ClusterInfo": { "BrokerNodeGroupInfo": { "ClientSubnets": [], "InstanceType": "kafka.m5.xlarge" }, "ClusterArn": "arn:aws:kafka:us-east-1:000000000000:cluster/unicorn-ride-cluster/f9b16124-baf3-459b-8507-ec6c605b7a0a-25", "ClusterName": "unicorn-ride-cluster", "CreationTime": "2022-07-21T14:41:07.897000Z", "CurrentBrokerSoftwareInfo": { "KafkaVersion": "2.5.0" }, "CurrentVersion": "KFABVNKTBGF6HX", "NumberOfBrokerNodes": 1, "State": "ACTIVE", "ZookeeperConnectString": "localhost:4510" } } ``` Once the `State` is `ACTIVE`, the cluster is ready to be used. Now it's time to create our Glue Schema Registry: ```bash lstk aws glue create-registry --registry-name unicorn-ride-request-registry ``` ```bash title="Output" { "RegistryArn": "arn:aws:glue:us-east-1:000000000000:file-registry/unicorn-ride-request-registry", "RegistryName": "unicorn-ride-request-registry" } ``` In the newly created registry, we can now add our initial `UnicornRideRequest` schema: ```bash lstk aws glue create-schema \ --registry-id RegistryName="unicorn-ride-request-registry" \ --schema-name unicorn-ride-request-schema-avro \ --compatibility BACKWARD \ --data-format AVRO \ --schema-definition "file://producer/src/main/resources/avro/unicorn_ride_request_v1.avsc" ``` ```bash title="Output" { "RegistryName": "unicorn-ride-request-registry", "RegistryArn": "arn:aws:glue:us-east-1:000000000000:file-registry/unicorn-ride-request-registry", "SchemaName": "unicorn-ride-request-schema-avro", "SchemaArn": "arn:aws:glue:us-east-1:000000000000:schema/unicorn-ride-request-registry/unicorn-ride-request-schema-avro", "DataFormat": "AVRO", "Compatibility": "BACKWARD", "SchemaCheckpoint": 1, "LatestSchemaVersion": 1, "NextSchemaVersion": 2, "SchemaStatus": "AVAILABLE", "SchemaVersionId": "925c868c-ff56-4992-9f24-6df5933c15cc", "SchemaVersionStatus": "AVAILABLE" } ``` For the schema, we just defined the compatibility mode `BACKWARD`. This means that the consumers using the new schema can also read data produced with the last schema. For example, this would allow the deletion of fields, or the introduction of new optional fields. You can find a thorough description of the different compatibility modes in the [AWS docs on Schema Versioning and Compatibility](https://docs.aws.amazon.com/glue/latest/dg/schema-registry.html#schema-registry-compatibility). The Glue Schema Registry will ensure that newly registered schemas fulfill the constraints defined by the compatibility mode. ## Running the Producer and Consumer Finally, everything is ready to start our `Producer` and `Consumer`. First, we need to get the bootstrap server address from the Kafka cluster: ```bash lstk aws kafka get-bootstrap-brokers --cluster-arn "arn:aws:kafka:us-east-1:000000000000:cluster/unicorn-ride-cluster/f9b16124-baf3-459b-8507-ec6c605b7a0a-25" ``` ```bash title="Output" { "BootstrapBrokerString": "localhost:4511" } ``` Like the `ClusterArn`, the `BootstrapBrokerString` is dynamic and can be different. Please make sure to use your bootstrap server address for the runs below. Now let's start the `Producer`. By default, our producer will just send 100 records and shut down. ```bash # Compile the Java packages mvn clean install # Run the producer mvn -pl producer exec:java -Dexec.args="--bootstrap-servers localhost:4511" ``` Once the producer is running, we can observe the different [steps of the producer as described in the intro](#integration-description): 1. Before sending a record, the producer validates that the schema, which it is using to serialize its records, is valid: ```plaintext ... [GlueSchemaRegistrySerializerFactory][DEBUG] Returning Avro serializer instance from GlueSchemaRegistrySerializerFactory [AWSSchemaRegistryClient][DEBUG] Getting Schema Version Id for : schemaDefinition = {"type":"record","name":"UnicornRideRequest","namespace":"cloud.localstack.demos.gluemsk.schema","fields":[{"name":"request_id","type":"int","doc":"customer request id"},{"name":"pickup_address","type":"string","doc":"customer pickup address"},{"name":"destination_address","type":"string","doc":"customer destination address"},{"name":"ride_fare","type":"float","doc":"ride fare amount (USD)"},{"name":"ride_duration","type":"int","doc":"ride duration in minutes"},{"name":"preferred_unicorn_color","type":{"type":"enum","name":"UnicornPreferredColor","symbols":["WHITE","BLACK","RED","BLUE","GREY"]},"default":"WHITE"},{"name":"recommended_unicorn","type":{"type":"record","name":"RecommendedUnicorn","fields":[{"name":"unicorn_id","type":"int","doc":"recommended unicorn id"},{"name":"color","type":{"type":"enum","name":"unicorn_color","symbols":["WHITE","RED","BLUE"]}},{"name":"stars_rating","type":["null","int"],"doc":"unicorn star ratings based on customers feedback","default":null}]}},{"name":"customer","type":{"type":"record","name":"Customer","fields":[{"name":"customer_account_no","type":"int","doc":"customer account number"},{"name":"first_name","type":"string"},{"name":"middle_name","type":["null","string"],"default":null},{"name":"last_name","type":"string"},{"name":"email_addresses","type":["null",{"type":"array","items":"string"}]},{"name":"customer_address","type":"string","doc":"customer address"},{"name":"mode_of_payment","type":{"type":"enum","name":"ModeOfPayment","symbols":["CARD","CASH"]},"default":"CARD"},{"name":"customer_rating","type":["null","int"],"default":null}]}}]}, schemaName = unicorn-ride-request-schema-avro, dataFormat = AVRO ... ``` 2. Once the schema is known to be valid, the producer serializes the record, compresses it, and sends it to the Kafka cluster. ```plaintext ... [GlueSchemaRegistryKafkaSerializer][DEBUG] Schema Version Id received from the from schema registry: f95edc4b-778d-4f65-b23e-7de41c5b4e53 [GlueSchemaRegistrySerializerFactory][DEBUG] Returning Avro serializer instance from GlueSchemaRegistrySerializerFactory [GlueSchemaRegistryDefaultCompression][DEBUG] Compression :: record length: 0KB [GlueSchemaRegistryDefaultCompression][DEBUG] Compression :: record length after compression: 0KB [Producer][INFO ] {"request_id": 1, "pickup_address": "Melbourne, Victoria, Australia", "destination_address": "Sydney, NSW, Aus", "ride_fare": 1200.5, "ride_duration": 120, "preferred_unicorn_color": "WHITE", "recommended_unicorn": {"unicorn_id": 2, "color": "WHITE", "stars_rating": 5}, "customer": {"customer_account_no": 1001, "first_name": "Dummy", "middle_name": null, "last_name": "User", "email_addresses": ["demo@example.com"], "customer_address": "Flinders Street Station", "mode_of_payment": "CARD", "customer_rating": 5}} ... ``` Now, the records sent by the producer are managed by the Kafka cluster and are waiting for a consumer to pick them up. We can start the consumer with the same bootstrap server address as the producer: ```bash mvn -pl consumer exec:java -Dexec.args="--bootstrap-servers localhost:4511" ``` In the logs of the consumer, we can now observe the [steps of the consumer as described in the intro](#integration-description): 3. The serialized and compressed record is read by the consumers: ```plaintext [NetworkClient][DEBUG] [Consumer clientId=consumer-unicorn.riderequest.consumer-1, groupId=unicorn.riderequest.consumer] Sending FETCH request with header RequestHeader(apiKey=FETCH, apiVersion=12, clientId=consumer-unicorn.riderequest.consumer-1, correlationId=10) and timeout 30000 to node 0: FetchRequestData(clusterId=null, replicaId=-1, maxWaitMs=500, minBytes=1, maxBytes=52428800, isolationLevel=0, sessionId=0, sessionEpoch=0, topics=[FetchTopic(topic='unicorn-ride-request-topic', partitions=[FetchPartition(partition=0, currentLeaderEpoch=0, fetchOffset=900, lastFetchedEpoch=-1, logStartOffset=-1, partitionMaxBytes=1048576)])], forgottenTopicsData=[], rackId='') [NetworkClient][DEBUG] [Consumer clientId=consumer-unicorn.riderequest.consumer-1, groupId=unicorn.riderequest.consumer] Received FETCH response from node 0 for request with header RequestHeader(apiKey=FETCH, apiVersion=12, clientId=consumer-unicorn.riderequest.consumer-1, correlationId=10): FetchResponseData(throttleTimeMs=0, errorCode=0, sessionId=2057133662, responses=[FetchableTopicResponse(topic='unicorn-ride-request-topic', partitionResponses=[FetchablePartitionResponse(partition=0, errorCode=0, highWatermark=1000, lastStableOffset=1000, logStartOffset=0, divergingEpoch=EpochEndOffset(epoch=-1, endOffset=-1), currentLeader=LeaderIdAndEpoch(leaderId=-1, leaderEpoch=-1), snapshotId=SnapshotId(endOffset=-1, epoch=-1), abortedTransactions=null, preferredReadReplica=-1, recordSet=MemoryRecords(size=17141, buffer=java.nio.HeapByteBuffer[pos=0 lim=17141 cap=17144]))])]) ``` 4. The consumer requests the schema from the schema registry, and uses the schema to decompress and deserialize the record. ```plaintext [request][DEBUG] Sending Request: DefaultSdkHttpFullRequest(httpMethod=POST, protocol=https, host=localhost.localstack.cloud, port=4566, encodedPath=/, headers=[amz-sdk-invocation-id, Content-Length, Content-Type, User-Agent, X-Amz-Target], queryParameters=[]) ... [request][DEBUG] Received successful response: 200, Request ID: 1EDT4G1DBSDS0XCD2N7FM9L4SRROEEML1FPDOYSANII78CXDR4F2, Extended Request ID: not available [GlueSchemaRegistryDeserializerFactory][DEBUG] Returning Avro de-serializer instance from GlueSchemaRegistryDeserializerFactory [GlueSchemaRegistryDefaultCompression][DEBUG] Decompression :: Compressed record length: 0KB [GlueSchemaRegistryDefaultCompression][DEBUG] Decompression :: Decompressed record length: 0KB [AvroDeserializer][DEBUG] Length of actual message: 121 [DatumReaderInstance][DEBUG] Using SpecificDatumReader for de-serializing Avro message, schema: {"type":"record","name":"UnicornRideRequest","namespace":"cloud.localstack.demos.gluemsk.schema","fields":[{"name":"request_id","type":"int","doc":"customer request id"},{"name":"pickup_address","type":"string","doc":"customer pickup address"},{"name":"destination_address","type":"string","doc":"customer destination address"},{"name":"ride_fare","type":"float","doc":"ride fare amount (USD)"},{"name":"ride_duration","type":"int","doc":"ride duration in minutes"},{"name":"preferred_unicorn_color","type":{"type":"enum","name":"UnicornPreferredColor","symbols":["WHITE","BLACK","RED","BLUE","GREY"]},"default":"WHITE"},{"name":"recommended_unicorn","type":{"type":"record","name":"RecommendedUnicorn","fields":[{"name":"unicorn_id","type":"int","doc":"recommended unicorn id"},{"name":"color","type":{"type":"enum","name":"unicorn_color","symbols":["WHITE","RED","BLUE"]}},{"name":"stars_rating","type":["null","int"],"doc":"unicorn star ratings based on customers feedback","default":null}]}},{"name":"customer","type":{"type":"record","name":"Customer","fields":[{"name":"customer_account_no","type":"int","doc":"customer account number"},{"name":"first_name","type":"string"},{"name":"middle_name","type":["null","string"],"default":null},{"name":"last_name","type":"string"},{"name":"email_addresses","type":["null",{"type":"array","items":"string"}]},{"name":"customer_address","type":"string","doc":"customer address"},{"name":"mode_of_payment","type":{"type":"enum","name":"ModeOfPayment","symbols":["CARD","CASH"]},"default":"CARD"},{"name":"customer_rating","type":["null","int"],"default":null}]}}]}) [AvroDeserializer][DEBUG] Finished de-serializing Avro message [GlueSchemaRegistryDeserializerFactory][DEBUG] Returning Avro de-serializer instance from GlueSchemaRegistryDeserializerFactory [GlueSchemaRegistryDefaultCompression][DEBUG] Decompression :: Compressed record length: 0KB [GlueSchemaRegistryDefaultCompression][DEBUG] Decompression :: Decompressed record length: 0KB [AvroDeserializer][DEBUG] Length of actual message: 121 [AvroDeserializer][DEBUG] Finished de-serializing Avro message ``` ## Schema Evolution In the course of this tutorial, we have implemented a Kafka producer and a consumer which integrate with the Glue Schema Registry. But the full potential of the Glue Schema Registry is unlocked when performing a schema evolution, i.e., when running producers and consumers with a new version of an already registered schema. Therefore, we will run a few more interesting scenarios to illustrate the benefits of the Schema Registry: 1. Running a producer which automatically registers a new, compatible schema version. 2. Running a producer which is rejected when trying to register a new, incompatible schema version. 3. Running an old consumer which is rejected since it is not compatible with the newly registered schema version. 4. Running an updated consumer which can consume the new schema / records. ### Producer registering a new schema version In this step, we will create a new producer which uses a new, `BACKWARD` compatible schema version. The complete module can be found in our [samples repository (along with the rest of the code of this tutorial).](https://github.com/localstack/localstack-pro-samples/blob/cd023a84a3b473984e9c34053d4feb7de8e038c1/glue-msk-schema-registry/producer-2/) The producer should register the new schema version automatically on its own. We create the new producer by executing the following steps: - Copy the `producer` directory and rename it to `producer-2`. - Set a new artifact ID in the `pom.xml` of the module: ```xml ... producer-2 ... ``` - Add the new module to the root `pom.xml`: ```xml producer producer-2 consumer ``` - Create a new version of the schema: - Rename the schema to `unicorn_ride_request_v2.avsc`. - In the schema, remove the previously required field `customer`: ```bash diff -u producer/src/main/resources/avro/unicorn_ride_request_v1.avsc producer-2/src/main/resources/avro/unicorn_ride_request_v2.avsc --- producer/src/main/resources/avro/unicorn_ride_request_v1.avsc 2022-05-13 08:27:08.219354922 +0200 +++ producer-2/src/main/resources/avro/unicorn_ride_request_v2.avsc 2022-05-13 08:27:08.219354922 +0200 @@ -20,23 +20,6 @@ {"name": "stars_rating", "type": ["null", "int"], "default": null, "doc": "unicorn star ratings based on customers feedback"} ] } - }, - { - "name": "customer", - "type": { - "type": "record", - "name": "Customer", - "fields": [ - {"name": "customer_account_no","type": "int", "doc": "customer account number"}, - {"name": "first_name","type": "string"}, - {"name": "middle_name","type": ["null","string"], "default": null}, - {"name": "last_name","type": "string"}, - {"name": "email_addresses","type": ["null", {"type":"array", "items":"string" }]}, - {"name": "customer_address","type": "string","doc": "customer address"}, - {"name": "mode_of_payment","type": {"type": "enum","name": "ModeOfPayment","symbols": ["CARD","CASH"]}, "default": "CARD"}, - {"name": "customer_rating", "type": ["null", "int"], "default": null} - ] - } } ] } ``` This change is `BACKWARD` compatible, because an updated consumer can read records for both - the current and the previous - records (new consumers don't need the customer data, they don't care if it's present or not). - Re-generate the Java classes for the schema: ```bash mvn clean generate-sources ``` - Once the classes have been generated, the producer code needs to be adjusted (remove the usage of `setCustomer` in the producer's `getRecord`, since the method does not exist anymore). - Configure the producer to automatically register its schema version in case it's not yet registered by setting the additional property `AWSSchemaRegistryConstants.SCHEMA_AUTO_REGISTRATION_SETTING` to `true`: ```java ... props.put(AWSSchemaRegistryConstants.COMPRESSION_TYPE, AWSSchemaRegistryConstants.COMPRESSION.ZLIB.name()); // Automatically register the new schema version of this producer! props.put(AWSSchemaRegistryConstants.SCHEMA_AUTO_REGISTRATION_SETTING, true); return props; } ... ``` Now we can run the new producer with the same bootstrap server address as before: ```bash mvn clean install mvn -pl producer-2 exec:java -Dexec.args="--bootstrap-servers localhost:4511" ``` In the logs we can see that the producer registered a new schema version before successfully publishing the new records: ```plaintext ... [AWSSchemaRegistryClient][INFO ] Registered the schema version with schema version id = 02922438-aa47-41e0-80e0-42238d07565f and with version number = 2 and status AVAILABLE ... ``` ### Producer trying to register an incompatible schema version In our next scenario we will create a new producer which wants to register a schema which is _not_ compatible to the schema in the registry. The producer will be rejected right when trying to register the new schema version, before even sending a record. The complete module can be found in our [samples repository (along with the rest of the code of this tutorial).](https://github.com/localstack/localstack-pro-samples/blob/cd023a84a3b473984e9c34053d4feb7de8e038c1/glue-msk-schema-registry/producer-3/) Similar to the previous scenario, we create a new producer by executing the following steps: - Copy the `producer-2` directory and rename it to `producer-3`. - Set a new artifact ID in the `pom.xml` of the module: ```xml ... producer-3 ... ``` - Add the new module to the root `pom.xml`: ```xml producer producer-2 producer-3 consumer ``` - Create a new version of the schema: - Rename the schema to `unicorn_ride_request_v3.avsc`. - In the schema, add a new required field `unicorn_food`: ```bash diff -u producer-2/src/main/resources/avro/unicorn_ride_request_v2.avsc producer-3/src/main/resources/avro/unicorn_ride_request_v3.avsc --- producer-2/src/main/resources/avro/unicorn_ride_request_v2.avsc 2022-05-13 08:27:08.219354922 +0200 +++ producer-3/src/main/resources/avro/unicorn_ride_request_v3.avsc 2022-05-13 08:27:08.219354922 +0200 @@ -20,6 +20,17 @@ {"name": "stars_rating", "type": ["null", "int"], "default": null, "doc": "unicorn star ratings based on customers feedback"} ] } + }, + { + "name": "unicorn_food", + "type": { + "type": "record", + "name": "Food", + "fields": [ + {"name": "price","type": "float","doc": "price per pound of food (USD)"}, + {"name": "name","type": "string"} + ] + } } ] } ``` This change is _not_ `BACKWARD` compatible, because an updated consumer cannot read records for both - the current and the previous - records (new consumers would expect the `unicorn_food`, which is not present in old records). - Re-generate the Java classes for the schema: ```bash mvn clean generate-sources ``` - Once the classes have been generated, the producer code needs to be adjusted (set the new required `unicorn_food` in the producer's `getRecord`): ```java ... .setStarsRating(5).build()) // we removed the (previously required) customer data here in the new version of the producer (v1) // and added a new field without a default (which isn't backward compatible) .setUnicornFood(Food.newBuilder().setPrice(133.7f).setName("Rainbow").build()) .build(); ... ``` We can run the new producer with the same bootstrap server address as before: ```bash mvn clean install mvn -pl producer-3 exec:java -Dexec.args="--bootstrap-servers localhost:4511" ``` In the logs we can see that the producer fails when trying to register its new, but incompatible, version of the schema: ```plaintext ... [AWSSchemaRegistryClient][INFO ] Registered the schema version with schema version id = 8f625284-07b4-442e-83ae-395cd7853746 and with version number = 3 and status FAILURE ... [WARNING] com.amazonaws.services.schemaregistry.exception.AWSSchemaRegistryException: Register schema :: Call failed when registering the schema with the schema registry for schema name = unicorn-ride-request-schema-avro at com.amazonaws.services.schemaregistry.common.AWSSchemaRegistryClient.registerSchemaVersion (AWSSchemaRegistryClient.java:310) ... ``` ### Outdated consumers We've seen how the producer's schema evolution works in the previous scenarios. Now, we'll take a closer look at our consumer. In our [first schema evolution scenario](#producer-registering-a-new-schema-version), the producer registered a new version of the schema and afterwards published records with that schema. The `BACKWARD` schema compatibility guarantees that updated consumers can read older records, i.e., the consumers need to be updated before the producers. Therefore, our old consumer will fail in consuming these records, because it is not compatible with the new schema registered by the new producer yet: ```bash mvn -pl consumer exec:java -Dexec.args="--bootstrap-servers localhost:4511" ``` In the logs, we can see that the consumer fails because the AVRO schema used by the consumer is expecting a required field (`customer`): ```plaintext [request][DEBUG] Sending Request: DefaultSdkHttpFullRequest(httpMethod=POST, protocol=https, host=localhost.localstack.cloud, port=4566, encodedPath=/, headers=[amz-sdk-invocation-id, Content-Length, Content-Type, User-Agent, X-Amz-Target], queryParameters=[]) ... [DatumReaderInstance][DEBUG] Using SpecificDatumReader for de-serializing Avro message, schema: {"type":"record","name":"UnicornRideRequest","namespace":"cloud.localstack.demos.gluemsk.schema","fields":[{"name":"request_id","type":"int","doc":"customer request id"},{"name":"pickup_address","type":"string","doc":"customer pickup address"},{"name":"destination_address","type":"string","doc":"customer destination address"},{"name":"ride_fare","type":"float","doc":"ride fare amount (USD)"},{"name":"ride_duration","type":"int","doc":"ride duration in minutes"},{"name":"preferred_unicorn_color","type":{"type":"enum","name":"UnicornPreferredColor","symbols":["WHITE","BLACK","RED","BLUE","GREY"]},"default":"WHITE"},{"name":"recommended_unicorn","type":{"type":"record","name":"RecommendedUnicorn","fields":[{"name":"unicorn_id","type":"int","doc":"recommended unicorn id"},{"name":"color","type":{"type":"enum","name":"unicorn_color","symbols":["WHITE","RED","BLUE"]}},{"name":"stars_rating","type":["null","int"],"doc":"unicorn star ratings based on customers feedback","default":null}]}},{"name":"customer","type":{"type":"record","name":"Customer","fields":[{"name":"customer_account_no","type":"int","doc":"customer account number"},{"name":"first_name","type":"string"},{"name":"middle_name","type":["null","string"],"default":null},{"name":"last_name","type":"string"},{"name":"email_addresses","type":["null",{"type":"array","items":"string"}]},{"name":"customer_address","type":"string","doc":"customer address"},{"name":"mode_of_payment","type":{"type":"enum","name":"ModeOfPayment","symbols":["CARD","CASH"]},"default":"CARD"},{"name":"customer_rating","type":["null","int"],"default":null}]}}]}) ... [WARNING] org.apache.kafka.common.errors.SerializationException: Error deserializing key/value for partition unicorn-ride-request-topic-0 at offset 100. If needed, please seek past the record to continue consumption. Caused by: com.amazonaws.services.schemaregistry.exception.AWSSchemaRegistryException: Exception occurred while de-serializing Avro message at com.amazonaws.services.schemaregistry.deserializers.avro.AvroDeserializer.deserialize (AvroDeserializer.java:103) ... Caused by: org.apache.avro.AvroTypeException: Found cloud.localstack.demos.gluemsk.schema.UnicornRideRequest, expecting cloud.localstack.demos.gluemsk.schema.UnicornRideRequest, missing required field customer at org.apache.avro.io.ResolvingDecoder.doAction (ResolvingDecoder.java:308) ... ``` ### Updated consumers Finally, we will update our consumer such that it is compatible to the new version of the schema. The complete module can be found in our [samples repository (along with the rest of the code of this tutorial).](https://github.com/localstack/localstack-pro-samples/blob/cd023a84a3b473984e9c34053d4feb7de8e038c1/glue-msk-schema-registry/consumer-2/) Similar to the previous producer scenarios, we create a new consumer by executing the following steps: - Copy the `consumer` directory and rename it to `consumer-2`. - Set a new artifact ID in the `pom.xml` of the module: ```xml ... consumer-2 ... ``` - Add the new module to the root `pom.xml`: ```xml producer producer-2 producer-3 consumer consumer-2 ``` - Replace the `unicorn_ride_request_v1.avsc` with the new version used by `producer-2` (`unicorn_ride_request_v2.avsc`). - Re-generate the Java classes for the schema: ```bash mvn clean generate-sources ``` Our new consumer, based on the latest version of the schema, will be able to successfully consume the records published by the new producer: ```bash mvn -pl consumer-2 exec:java -Dexec.args="--bootstrap-servers localhost:4511" ``` ## Conclusion Apache Kafka is used as the core messaging system in complex environments, with independent producers and consumers. The individual development of these components make it hard to coordinate and evolve data schemas over time. Using the AWS Glue Schema Registry can help you to prevent the usage of incompatible schemas. With LocalStack, emulating Amazon Managed Streaming for Kafka and AWS Glue Schema Registry, you can develop and test the next evolution of your data schema locally on your own machine. # Building a Serverless Quiz Application with LocalStack > Build an interactive serverless quiz application using AWS Lambda, DynamoDB, and API Gateway. Learn how to create, deploy, and test a complete serverless architecture locally with LocalStack, featuring quiz creation, submission handling, and scoring mechanisms. ## Introduction Interactive quiz applications are popular for education, training, and engagement platforms. Building them with serverless architecture provides scalability, cost-effectiveness, and simplified maintenance. In this tutorial, we'll create a complete serverless quiz application using AWS Lambda, DynamoDB, and API Gateway. Our quiz application will allow users to: - Create new quizzes with multiple-choice questions - Submit quiz responses and receive immediate scoring - View quiz results and leaderboards Using LocalStack, we can develop and test this entire serverless infrastructure locally before deploying to AWS, enabling rapid development cycles and cost-effective testing. ## Prerequisites For this tutorial, you will need: - [LocalStack for AWS](https://localstack.cloud/pricing/) with a valid auth token - [AWS CLI](https://docs.localstack.cloud/user-guide/integrations/aws-cli/) with [`lstk aws`](/aws/connecting/aws-cli#localstack-aws-cli-lstk-aws) - [AWS CDK](https://docs.localstack.cloud/user-guide/integrations/aws-cdk/) with [`lstk cdk`](/aws/connecting/infrastructure-as-code/aws-cdk#aws-cdk-cli-for-localstack) (**optional**) - [Python 3.11+](https://www.python.org/downloads/) and `pip` - [curl](https://curl.se/) for testing API endpoints - [`make`](https://www.gnu.org/software/make/) (**optional**, but recommended for running the sample application) ## Architecture The following diagram shows the serverless architecture we'll build: ![Application Architecture](/images/aws/serverless-quiz-app-architecture.png) The architecture consists of: - **API Gateway**: REST API endpoints for quiz operations with Lambda integrations - **Lambda Functions**: Serverless functions handling quiz operations (create, submit, score, retrieve) - **DynamoDB Tables**: NoSQL database storing quiz metadata (`Quizzes`) and user submissions (`UserSubmissions`) - **CloudFront Distribution**: Global delivery of frontend assets with caching - **S3 Bucket**: Static website hosting for the quiz frontend interface - **SQS**: Managing asynchronous submissions with Dead Letter Queue for failed processing - **SNS Topics**: Alert notifications and system integration - **Step Functions**: Email notification workflows - **IAM Roles and Policies**: Least-privilege access control for all services ### Request Flow 1. Client sends HTTP requests to API Gateway endpoints 2. API Gateway triggers corresponding Lambda functions 3. Lambda functions interact with DynamoDB for data persistence 4. Responses are returned through API Gateway to the client 5. Static frontend is served from S3 bucket via CloudFront distribution ## Getting Started ### Clone the Repository First, clone the sample repository and navigate to the project directory: ```bash git clone https://github.com/localstack-samples/sample-serverless-quiz-app.git cd sample-serverless-quiz-app ``` ### Set up the Environment Create a Python virtual environment and install dependencies: ```bash python -m venv .venv source .venv/bin/activate # On Windows: .venv\Scripts\activate pip install -r tests/requirements-dev.txt ``` ## Deploying the Application ### Start LocalStack First, start LocalStack with your auth token: ```bash lstk start ``` ### Deploy the Infrastructure You can deploy the application using either AWS CLI or CDK: #### Option 1: AWS CLI Deployment (Recommended) Deploy the complete serverless infrastructure using the provided script: ```bash bin/deploy.sh ``` #### Option 2: CDK Deployment Alternatively, deploy using AWS CDK with LocalStack: ```bash cd cdk lstk cdk bootstrap AWS_CMD="lstk aws" CDK_CMD="lstk cdk" bash bin/deploy_cdk.sh ``` Both deployment methods will: 1. Create DynamoDB tables for quizzes and submissions 2. Deploy Lambda functions for quiz operations 3. Set up API Gateway endpoints 4. Configure S3 bucket for static hosting 5. Seed sample quiz data The deployment output will show: ```bash CloudFront URL: https://1e372b81.cloudfront.localhost.localstack.cloud API Gateway Endpoint: http://localhost:4566/_aws/execute-api/4xu5emxibf/test ``` ## Testing the Application The application includes comprehensive testing capabilities across multiple dimensions: ### Manual Testing Navigate to the CloudFront URL from the deployment output to interact with the quiz application. The interface allows you to: - Create new quizzes with multiple choice questions - Submit quiz responses and receive immediate scoring - View leaderboards with top performers - Test email notifications through the MailHog extension **Note**: If you have deployed the application using AWS CLI, sample quiz data would have been seeded to make local testing easier. ### End-to-End Integration Testing Run the complete test suite to validate quiz creation, submission, and scoring: ```bash export AWS_DEFAULT_REGION=us-east-1 export AWS_ACCESS_KEY_ID=test export AWS_SECRET_ACCESS_KEY=test pytest tests/test_infra.py ``` The automated tests utilize the AWS SDK for Python (boto3) and the `requests` library to interact with the quiz application API. ## Advanced Features with LocalStack for AWS ### Resource Browser Use the LocalStack Web Application to inspect your deployed resources: - [DynamoDB Tables](https://app.localstack.cloud/inst/default/resources/dynamodb): View table data and query operations - [Lambda Functions](https://app.localstack.cloud/inst/default/resources/lambda/functions): Monitor function invocations and logs - [API Gateway](https://app.localstack.cloud/inst/default/resources/apigateway): Inspect API endpoints and request routing ### Cloud Pods for Quick Setup Skip the deployment step by loading a pre-configured environment: ```bash lstk restart lstk snapshot load pod:serverless-quiz-app ``` This instantly loads the complete application infrastructure from a saved state. ## Conclusion In this tutorial, we've built a complete serverless quiz application demonstrating key serverless patterns: - **Event-driven architecture** with API Gateway triggering Lambda functions - **NoSQL data persistence** using DynamoDB for scalable storage - **Stateless function design** enabling automatic scaling - **RESTful API design** for clean client-server communication The application showcases how LocalStack enables rapid serverless development by providing a local AWS environment for testing and iteration. This approach allows developers to: - Test serverless applications without cloud costs - Develop offline with full AWS service emulation - Validate application logic before production deployment - Iterate quickly during development cycles For production deployment, the same code and configuration can be deployed to AWS with minimal changes, demonstrating the power of LocalStack for serverless development workflows. # Chaos Engineering: Simulating Outages using Chaos API > Use the Chaos API to simulate service disruptions and assess how well your infrastructure can deploy and recover from unexpected situations. ## Introduction [LocalStack Chaos API](/aws/developer-tools/chaos-engineering/chaos-api) is capable of simulating infrastructure faults to allow conducting controlled chaos engineering tests on AWS infrastructure. Its purpose is to uncover vulnerabilities and improve system robustness. Chaos API offers a means to deliberately introduce failures and observe their impacts, helping developers to better equip their systems against actual outages. ## Getting started In this tutorial we study the effects of outages on a sample AWS application. We use the Chaos API to simulate the outage and design a mitigation to make the application resilient against database outages. This tutorial is designed for users new to the Chaos API and assumes basic knowledge of the AWS CLI and our [`lstk aws`](/aws/connecting/aws-cli#localstack-aws-cli-lstk-aws) command. In this example, we will use the Chaos API to create controlled outages in a DynamoDB database. The aim is to test the software's behavior and error handling capabilities. For this particular example, we'll be using a [sample application repository](https://github.com/localstack-samples/sample-chaos-api-serverless). Clone the repository, and follow the instructions below to get started. ### Prerequisites The general prerequisites for this guide are: - LocalStack for AWS with [LocalStack Auth Token](/aws/getting-started/auth-token) - [AWS CLI](/aws/connecting/aws-cli) with the [`lstk aws`](/aws/connecting/aws-cli#localstack-aws-cli-lstk-aws) command - [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) Start LocalStack by using the `docker-compose.yml` file from the repository. Ensure to set your Auth Token as an environment variable during this process. The cloud resources will be automatically created upon the LocalStack start. ```bash LOCALSTACK_AUTH_TOKEN= docker compose up ``` Given that you've started LocalStack via `docker-compose`, you'll need to configure the `lstk` CLI to contact your container: ```bash export LSTK_ENDPOINT_URL=http://localhost.localstack.cloud:4566 ``` ### Architecture The following diagram shows the architecture that this application builds and deploys: ![Architecture](/images/aws/arch-1.png) ## Testing the application Before simulating outages, verify that the application is working as expected: ```bash curl --location 'http://12345.execute-api.localhost.localstack.cloud:4566/dev/productApi' \ --header 'Content-Type: application/json' \ --data '{ "id": "prod-2004", "name": "Ultimate Gadget", "price": "49.99", "description": "The Ultimate Gadget is the perfect tool for tech enthusiasts looking for the next level in gadgetry. Compact, powerful, and loaded with features." }' ``` Expected output: ```bash title="Output" Product added/updated successfully. ``` After ending the outage, confirm that previously failed items are stored successfully: ```bash lstk aws dynamodb scan --table-name Products ``` Expected output: ```json { "Items": [ { "name": { "S": "Super Widget" }, "description": { "S": "A versatile widget that can be used for a variety of purposes. Durable, reliable, and affordable." }, "id": { "S": "prod-1003" }, "price": { "N": "29.99" } }, { "name": { "S": "Ultimate Gadget" }, "description": { "S": "The Ultimate Gadget is the perfect tool for tech enthusiasts looking for the next level in gadgetry. Compact, powerful, and loaded with features." }, "id": { "S": "prod-2004" }, "price": { "N": "49.99" } } ], "Count": 2, "ScannedCount": 2, "ConsumedCapacity": null } ``` ### Simulating the outage Next, we will configure the Chaos API to target all DynamoDB operations. The Chaos API is powerful enough to refine outages to particular operations like `PutItem` or `GetItem`, but the objective here is to simulate a failure of entire service. The following configuration will cause all API calls to fail with a 80% failure rate, each resulting in an HTTP 500 status code and a `SomethingWentWrong` error. ```bash curl --location --request PATCH 'http://localhost.localstack.cloud:4566/_localstack/chaos/faults' \ --header 'Content-Type: application/json' \ --data ' [ { "service": "dynamodb", "probability": 0.8, "error": { "statusCode": 500, "code": "SomethingWentWrong" } } ]' ``` This makes the database inaccessible. No external client or a LocalStack service can retrieve or add new products, resulting in the API Gateway returning an Internal Server Error. Downtime and data loss are critical issues to avoid in enterprise applications. Fortunately, encountering this issue early in the development phase allows developers to implement effective error handling and develop mechanisms to prevent data loss during a database outage. ### Designing a more resilient system ![Architecture](/images/aws/arch-2.png) A possible solution involves setting up an SNS topic, an SQS queue, and a Lambda function. The Lambda function will be responsible for retrieving queued items and attempting to re-execute the `PutItem` operation on the database. If DynamoDB remains unavailable, the item will be placed back in the queue for a later retry. ```bash curl --location 'http://12345.execute-api.localhost.localstack.cloud:4566/dev/productApi' \ --header 'Content-Type: application/json' \ --data '{ "id": "prod-1003", "name": "Super Widget", "price": "29.99", "description": "A versatile widget that can be used for a variety of purposes. Durable, reliable, and affordable." }' ``` ```bash title="Output" A DynamoDB error occurred. Message sent to queue. ``` If we review the logs, it will show that the `DynamoDbException` has been managed effectively. ```text 2023-11-06T22:21:40.789 INFO --- [ asgi_gw_2] localstack.request.aws : AWS dynamodb.PutItem => 500 (DynamoDbException) 2023-11-06T22:21:40.834 DEBUG --- [ asgi_gw_4] l.services.sns.publisher : Topic 'arn:aws:sns:us-east-1:000000000000:ProductEventsTopic' publishing '5520d37a-fc21-4a73-b1bf-f9b9afce5908' to subscribed 'arn:aws:sqs:us-east-1:000000000000:ProductEventsQueue' with protocol 'sqs' (subscription 'arn:aws:sns:us-east-1:000000000000:ProductEventsTopic:0a4abf8c-744a-404a-9ff9-f132e25d1b30') ``` This element will remain in the queue until the outage is resolved. ### Ending the outage To stop the outage, use the following configuration: ```bash curl --location --request POST 'http://localhost.localstack.cloud:4566/_localstack/chaos/faults' \ --header 'Content-Type: application/json' \ --data '[]' ``` With the outage now ended, the Product that initially failed to reach the database to finally be stored successfully. This can be confirmed by scanning the database. ```bash lstk aws dynamodb scan --table-name Products ``` ```bash title="Output" { "Items": [ { "name": { "S": "Super Widget" }, "description": { "S": "A versatile widget that can be used for a variety of purposes. Durable, reliable, and affordable." }, "id": { "S": "prod-1003" }, "price": { "N": "29.99" } }, { "name": { "S": "Ultimate Gadget" }, "description": { "S": "The Ultimate Gadget is the perfect tool for tech enthusiasts looking for the next level in gadgetry. Compact, powerful, and loaded with features." }, "id": { "S": "prod-2004" }, "price": { "N": "49.99" } } ], "Count": 2, "ScannedCount": 2, "ConsumedCapacity": null } ``` ## Conclusion Simulating outages with the Chaos API helps uncover weaknesses in application error handling and data durability. To mitigate outages: - Implement retry logic for failed operations - Use queues (e.g., SQS) to buffer writes during downtime - Employ Lambda functions to process and retry queued items - Monitor system health and automate recovery actions By proactively testing and designing for failure, you can build resilient cloud applications that gracefully handle disruptions and minimize data loss. --- ### Introducing network latency The LocalStack Chaos API can also introduce a network latency for all connections. This can be done with the following configuration: ```bash curl --location --request POST 'http://localhost.localstack.cloud:4566/_localstack/chaos/effects' \ --header 'Content-Type: application/json' \ --data '{ "latency": 5000 }' ``` With this configured, you can use the same sample stack to observe and understand the effects of a 5-second delay on each service call. ```bash curl --location 'http://12345.execute-api.localhost.localstack.cloud:4566/dev/productApi' \ --max-time 2 \ --header 'Content-Type: application/json' \ --data '{ "id": "prod-1088", "name": "Super Widget", "price": "29.99", "description": "A versatile widget that can be used for a variety of purposes. Durable, reliable, and affordable." }' ``` ```bash title="Output" An error occurred (InternalError) when calling the GetResources operation (reached max retries: 4) ``` # Tutorial: Terraform Fullstack Serverless Shipment App > Deploy a full-stack shipment tracking application locally using Terraform and LocalStack. [LocalStack](https://localstack.cloud) enables you to develop and test cloud applications locally by emulating AWS services on your machine. In this tutorial, you will deploy a full-stack serverless shipment tracking application using Terraform and LocalStack. This sample app consists of a React frontend and a Spring Boot backend, integrating with key AWS services like API Gateway, Lambda, DynamoDB, S3, SNS, and SQS. The infrastructure is managed entirely using Terraform, demonstrating Infrastructure as Code (IaC) workflows. ![Terraform Fullstack Serverless Shipment App Architecture](https://github.com/localstack-samples/sample-terraform-fullstack-serverless-shipment-app/raw/master/sample-pictures/architecture.png) ## Prerequisites Make sure the following tools and dependencies are installed and configured on your local machine before proceeding: - **LocalStack for AWS** - **Terraform CLI** - **AWS CLI** with the [`lstk aws`](https://docs.localstack.cloud/aws/connecting/aws-cli/#localstack-aws-cli-lstk-aws) command for LocalStack - **Maven 3.8.5+** and **Java 17** for Spring Boot backend - **Node.js** and **npm** for React frontend - **make** (optional, but recommended for simplified commands) ## Installation Clone the sample repository and install dependencies: ``` git clone https://github.com/localstack-samples/sample-terraform-fullstack-serverless-shipment-app.git cd sample-terraform-fullstack-serverless-shipment-app make install ``` This command builds the Lambda validator JAR and installs frontend Node.js packages. ## Deployment Start LocalStack in the background with your authorization token configured: ``` lstk start ``` Use the provided Makefile to deploy all infrastructure components: ``` make deploy ``` This creates and configures: - S3 buckets for shipment images and Lambda code - DynamoDB tables preloaded with sample shipments - Lambda functions for image validation and processing - SNS topics and SQS queues for event messaging - Required IAM roles and permissions ## Running the Application ### Start React Frontend ``` cd shipment-list-frontend npm start ``` Access the UI at [http://localhost:3000](http://localhost:3000). ### Start Spring Boot Backend In a separate terminal, run: ``` mvn spring-boot:run -Dspring-boot.run.profiles=dev ``` The backend API will be available at [http://localhost:8081](http://localhost:8081). ## Testing Run full end-to-end tests with: ``` make test ``` ## Using the Application - View shipment list on the React frontend. - Upload shipment images; valid ones are watermarked by the Lambda function. - Invalid files are automatically replaced. - Real-time updates are delivered via Server-Sent Events. - Create, update, or delete shipments through provided UI and API endpoints. ## Summary and Use Cases This project illustrates: - Deploying AWS resources (S3, Lambda, DynamoDB, SNS, SQS) with Terraform. - Serverless image processing and validation using Lambda. - Reactive messaging using SNS and SQS. - Seamless switching between AWS and LocalStack via Spring Profiles. - Integration testing using Testcontainers. - Using LocalStack's `lstk aws` and `lstk terraform` commands for streamlined local development. - Infrastructure as Code testing enabling consistent, repeatable environment setups. --- By completing this tutorial, you can confidently develop and test complex serverless applications locally with LocalStack and Terraform, accelerating your cloud-native development cycles. # How To: Terraform Init Hooks for Automation & Production-Identical Test Environments > This tutorial guides you through using LocalStack's new extension that supports Terraform configuration files as initialization hooks. You'll learn how to leverage this feature and integrate it with Testcontainers to simplify testing cycles, making the process more efficient and closely aligned with real AWS infrastructure practices. ## Introduction: The importance of integration testing and how to streamline it LocalStack is a robust tool that emulates a local AWS cloud stack, allowing engineers to test and develop apps using AWS services directly on their local environments. This tool is essential for enhancing developer experience, reducing development costs and increasing efficiency. In LocalStack, [**initialization hooks**](/aws/customization/advanced/initialization-hooks) are scripts that customize or initialize your LocalStack instance at different stages of its lifecycle. Up until now, the supported hooks could be shell or Python scripts executed at predefined lifecycle phases — BOOT, START, READY, and SHUTDOWN. By placing scripts in the respective directories (`/etc/localstack/init/{stage}.d`), developers can automate tasks like setting up initial states, configuring services, or performing clean-up activities. [Terraform](https://www.terraform.io/), is one of the most widely adopted tools for provisioning AWS infrastructure, so naturally, enabling Terraform configuration files to be used directly as initialization hooks boosts LocalStack's utility. The direct use of Terraform scripts as init hooks allows developers to replicate production environments accurately and automate integration tests more effectively. This capability ensures that the test environment mirrors the production setup as closely as possible. This tutorial guides you through using LocalStack's [**new extension**](https://github.com/localstack/localstack-extensions/tree/main/terraform-init) that supports Terraform configuration files as initialization hooks, and will show you how to leverage this new feature, and integrate it with Testcontainers for seamless testing. This approach simplifies the development and testing cycle, making it more efficient and closely aligned with real AWS infrastructure practices. ## Prerequisites For this tutorial, you will need: - [LocalStack for AWS](/aws/getting-started/auth-token) to emulate the AWS services and to use LocalStack Extensions. If you don't have LocalStack for AWS yet, you can sign up on our [webapp](https://app.localstack.cloud) to get a trial license for free. - [Docker](https://docker.io/) - [`lstk`](/aws/getting-started/installation#lstk) - [AWS CLI](https://aws.amazon.com/cli/) - Optional for building the Lambda functions: [Java 17](https://openjdk.org/install/) - Optional for building the Lambda functions: [Apache Maven 3.9.8](https://maven.apache.org/install.html) - Optional: [Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) and [`lstk terraform`](/aws/connecting/infrastructure-as-code/terraform#lstk-terraform) ## Project overview You can get hands-on with this setup by cloning the [**demo repository**](https://github.com/localstack-samples/terraform-init-hooks-demo). The diagram below illustrates how everything within the project connects. The application is simple, yet it reflects a realistic scenario: there's an API Gateway that directs requests to two Lambda functions. One Lambda function fetches product details by ID, and the other saves new products to a DynamoDB database. A CloudWatch Logs instance is used to store and access the Lambda log files. ![architecture-diagram](/images/aws/architecture-diagram.png) ## Using Terraform init hooks ### Using init hooks directly Let's first take a look at how you can use Terraform init hooks to create AWS resources automatically when LocalStack starts up. After establishing this foundation, we will proceed to integrate this feature with Testcontainers to further enhance our development and testing workflow. :::note If you're new to Terraform, you can quickly familiarize yourself with the basic commands by reading the [getting started tutorials](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/aws-change) on their official documentation page. ::: #### LocalStack CLI `lstk` takes container environment variables and bind mounts from its [`config.toml`](/aws/developer-tools/running-localstack/lstk/configuration/) rather than from command-line flags. In the root folder of the demo project, create a project-local `.lstk/config.toml`: ```toml title=".lstk/config.toml" [[containers]] type = "aws" tag = "latest" port = "4566" env = ["terraform-init"] volumes = [ "../terraform:/etc/localstack/init/ready.d", ] [env.terraform-init] EXTENSION_AUTO_INSTALL = "localstack-extension-terraform-init" ``` :::note Relative host paths in `volumes` are resolved against the directory that holds `config.toml` — here `.lstk/` — which is why the mount is written as `../terraform` rather than `./terraform`. ::: `main.tf` refers to the Lambda JAR as `target/product-lambda.jar`, relative to the Terraform working directory. Build it and stage it inside `terraform/`, so that the single mount above carries both the configuration and the JAR into the container: ```bash mvn clean package -DskipTests mkdir -p terraform/target cp target/product-lambda.jar terraform/target/product-lambda.jar ``` Then start LocalStack from the project root: ```bash lstk start ``` This is the easiest way to quickly spin up the desired services at startup. The [`env` profile](/aws/developer-tools/running-localstack/lstk/configuration/#passing-environment-variables-to-the-container) tells LocalStack to automatically install the **`localstack-extension-terraform-init`** [extension](/aws/customization/integrations/extensions/), and the [`volumes` entry](/aws/developer-tools/running-localstack/lstk/configuration/#volume-mounts) mounts the Terraform configuration and the Lambda JAR into the container. The extension will install both `terraform` and `tflocal` into your LocalStack container, and enable the init hook runners to detect Terraform files. You can also organize your Terraform files into subdirectories if you want. Since the initialization hook runs `terraform init`, the AWS Terraform provider would be downloaded in the container on every start. Mounting the whole `terraform` directory, as above, avoids this: any Terraform state including the `.terraform` folder that contains the provider will be cached on your host directory, however it may require `sudo` permissions to modify or delete, as it is created by the container. Once the `Ready.` message appears, you can list what the init hook created: ```bash lstk status ``` ```bash title="Output" ~ 5 resources · 4 services SERVICE RESOURCE REGION ACCOUNT ApiGateway nq7sycvcbw us-east-1 000000000000 DynamoDB Products us-east-1 000000000000 IAM productRole global 000000000000 Lambda add-product us-east-1 000000000000 Lambda get-product us-east-1 000000000000 ``` #### Docker compose Another way of starting LocalStack with the desired services is using `docker compose`. In the root folder, you'll find the essential configs in the `docker-compose.yml` file: ```yaml services: localstack: container_name: "${LOCALSTACK_DOCKER_NAME:-localstack-main}" image: localstack/localstack-pro:latest # required for Pro ports: - "127.0.0.1:4566:4566" # LocalStack Gateway - "127.0.0.1:4510-4559:4510-4559" # external services port range - "127.0.0.1:443:443" # LocalStack HTTPS Gateway (Pro) environment: # Activate LocalStack for AWS: https://docs.localstack.cloud/getting-started/auth-token/ - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} # required for Pro # LocalStack configuration: https://docs.localstack.cloud/references/configuration/ - DEBUG=1 - PERSISTENCE=${PERSISTENCE:-0} - EXTENSION_AUTO_INSTALL=localstack-extension-terraform-init volumes: - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" - "./terraform:/etc/localstack/init/ready.d" - "./target/product-lambda.jar:/etc/localstack/init/ready.d/target/product-lambda.jar" ``` Environment Variables: - **LOCALSTACK_AUTH_TOKEN**: Required for using LocalStack for AWS. - **DEBUG**: Set to 1 to enable verbose logging of the container. - **EXTENSION_AUTO_INSTALL**: Automatically installs specified LocalStack [extensions](/aws/customization/integrations/extensions/), in this case, `localstack-extension-terraform-init` which allows Terraform files to be directly used as init hooks. Volumes: - Docker Socket: Mounts the Docker socket `/var/run/docker.sock` from the host into the container. This allows LocalStack to manage Docker containers directly, facilitating functionalities like spinning up Lambda containers. - Terraform Configuration: Mounts a directory containing Terraform files (./terraform) from the host to `/etc/localstack/init/ready.d` in the container. This enables the use of init hooks, as well as the AWS provider (plugins and modules) which is downloaded once and reused in subsequent startups. - Lambda Function JAR: Places the `product-lambda.jar` file from the host into the `/etc/localstack/init/ready.d/target` directory in the container, making it available for use, as described in `main.tf`. After running `docker compose up`, we should keep an eye on the container logs until the `Ready.` message appears. Given that you've started LocalStack via `docker-compose`, you'll need to configure the `lstk` CLI to contact your container: ```bash export LSTK_ENDPOINT_URL=http://localhost.localstack.cloud:4566 ``` Now we can test the functionality of our stack by running the following commands: ```bash lstk aws apigateway get-rest-apis \ --query 'items[?name==`product-api-gateway`].id' ``` ```bash title="Output" [ "ixqd52qrip" ] ``` This will get us the ID of the API Gateway, which is necessary to build the URL: ```bash curl --location "http://ixqd52qrip.execute-api.localhost.localstack.cloud:4566/dev/productApi" \ --header 'Content-Type: application/json' \ --data '{ "id": "34534", "name": "EcoFriendly Water Bottle", "description": "A durable, eco-friendly water bottle designed to keep your drinks cold for up to 24 hours.", "price": "29.99" }' ``` ```bash title="Output" Product added/updated successfully. ``` To check if the product object has been persisted to the database, we can fire a GET request against the same URL, using the product ID as a query param: ```bash curl --location "http://ixqd52qrip.execute-api.localhost.localstack.cloud:4566/dev/productApi?id=34534" ``` ```bash title="Output" {"price":"29.99","name":"EcoFriendly Water Bottle","description":"A durable, eco-friendly water bottle designed to keep your drinks cold for up to 24 hours.","id":"34534"} ``` ### Integrating with Testcontainers #### The setup Now that we've established how seamlessly LocalStack integrates with Terraform using initialization hooks, let's explore how we can leverage this feature to enhance our testing processes using [Testcontainers](https://testcontainers.com/). This demo is a Java project, but the framework supports multiple other programming languages. We can now automate and streamline our LocalStack initialization, ensuring that every test suite includes a fresh, fully configured AWS environment. This helps users build confidence in moving on to deploy to the AWS platform, as the IaC files remain unchanged. To get started with Testcontainers, you need to include a few dependencies in the Maven `pom.xml` file: ```xml org.testcontainers testcontainers org.testcontainers junit-jupiter test org.testcontainers localstack test . . . org.testcontainers testcontainers-bom 1.19.8 pom import ``` If you intend to make changes to the code, you may build the project by running the following command and the JAR file will be recreated in the `target` folder. However, you don't need to, as the built file is already provided. ```bash mvn clean package ``` In the provided code snippet, we configure a LocalStackContainer object using Testcontainers. Don't forget to set the `LOCALSTACK_AUTH_TOKEN` as an environment variable. This configuration is abstracted in a superclass to be reusable across different test cases. ```java @Container protected static LocalStackContainer localStack = new LocalStackContainer(DockerImageName.parse("localstack/localstack-pro:latest")) .withEnv("LAMBDA_REMOVE_CONTAINERS", "1") .withEnv("EXTENSION_AUTO_INSTALL", "localstack-extension-terraform-init") .withEnv("LOCALSTACK_AUTH_TOKEN", System.getenv("LOCALSTACK_AUTH_TOKEN")) .withFileSystemBind("./target/product-lambda.jar", "/etc/localstack/init/ready.d/target/product-lambda.jar") .withFileSystemBind("./terraform", "/etc/localstack/init/ready.d") .withEnv("DEBUG", "1") .withStartupTimeout(Duration.of(2, ChronoUnit.MINUTES)); ``` Here's what each configuration line does: - **LAMBDA_REMOVE_CONTAINERS="1"**: Ensures that Lambda containers are removed after execution to free up resources and avoid clutter. - **EXTENSION_AUTO_INSTALL="localstack-extension-terraform-init"**: Automatically installs the Terraform init hooks extension. - **LOCALSTACK_AUTH_TOKEN**: Fetches the LocalStack Auth Token from environment variables. - **DEBUG="1"**: Enables verbose logging for troubleshooting and ensuring detailed logs are available for debugging. The `withFileSystemBind` commands mount the `product-lambda.jar` and the directory containing the Terraform files from the host machine into the appropriate init hook directory within the LocalStack container. The last line specifies a timeout for the container startup, set to 2 minutes. This ensures that the container has enough time to initialize all services. Normally, the process runs a lot faster, but this prevents a worse case scenario that could include any delays cause by hardware resources or network issues. This is very similar to the `docker-compose.yml` file we've seen before. #### The tests The test suite in the `ProductAppTests` class is checking three scenarios: - Product Persistence: Tests the ability to successfully save a new product to DynamoDB via a Lambda function, confirming the POST request and the response. - Product Retrieval: Ensures the system can accurately fetch a product by its ID from DynamoDB through a GET request. - Non-Existent Product Handling: Validates the system's response to a request for a non-existent product, ensuring the Lambda function properly returns the appropriate error message "Product not found". ![architecture-diagram](/images/aws/architecture-diagram-test.png) Since the app runs entirely inside the LocalStack container, an HTTP client is used to make calls against the service. ```java @Test @Order(1) void testSuccessfulPostAction() { var postUrl = localStackEndpoint + "/restapis/" + apiGWId + "/dev/_user_request_/productApi"; var expectedResponse = "Product added/updated successfully."; try (CloseableHttpClient httpClient = HttpClients.createDefault()) { // add headers to a POST request var httpPost = new HttpPost(postUrl); httpPost.setHeader(new BasicHeader("Content-Type", "application/json")); // create the JSON request body var jsonRequestBody = "{\n" + " \"id\": \"34534\",\n" + " \"name\": \"EcoFriendly Water Bottle\",\n" + " \"description\": \"A durable, eco-friendly water bottle.\",\n" + " \"price\": \"29.99\"\n" + "}"; // set the request body var entity = new StringEntity(jsonRequestBody); httpPost.setEntity(entity); // execute the request try (CloseableHttpResponse response = httpClient.execute(httpPost)) { String responseBody = EntityUtils.toString(response.getEntity()); //assert 200 OK status & response message Assertions.assertEquals(HttpStatus.SC_OK, response.getStatusLine().getStatusCode()); Assertions.assertEquals(expectedResponse, responseBody); } } catch (IOException e) { throw new RuntimeException(e); } } ``` It is now incredibly straightforward to utilize our Terraform configuration file to construct the exact, production-ready environment needed for effective testing. ### About owner permissions It's important to note that this extension is still new and primarily intended for straightforward Terraform configurations. It is subject to change and improvements in the future. We already mentioned that if you mount a directory instead of a single file, the AWS Terraform provider will not be downloaded each time the `init` command runs. Any state files created will be in your host directory, potentially requiring `sudo` to modify or delete. The reason for this is that the container user that creates these files is `root`, and on Linux systems this will propagate to your local files. MacOS, on the other hand, will not allow this to happen and your locally created files will belong to your user. Here's how it looks like: ##### In the LocalStack container ```bash drwxr-xr-x 8 root root 256 Jul 10 15:26 . drwxr-xr-x 1 root root 4096 Jul 10 15:24 .. drwxr-xr-x 3 root root 96 Jul 10 07:28 .terraform -rw-r--r-- 1 root root 1406 Jul 10 07:28 .terraform.lock.hcl -rwxrwxrwx 1 root root 5563 Jul 7 06:51 main.tf drwxr-xr-x 3 root root 96 Jul 10 07:28 target -rw-r--r-- 1 root root 23620 Jul 10 15:26 terraform.tfstate -rw-r--r-- 1 root root 23620 Jul 10 15:25 terraform.tfstate.backup ``` ##### On MacOS ```bash drwxr-xr-x@ 9 user staff 288 Jul 10 00:28 ./ drwxr-xr-x@ 19 user staff 608 Jul 10 00:28 ../ drwxr-xr-x 3 user staff 96 Jul 10 00:28 .terraform/ -rw-r--r-- 1 user staff 1406 Jul 10 00:28 .terraform.lock.hcl -rw------- 1 user staff 202 Jul 10 00:28 .terraform.tfstate.lock.info -rw-r--r-- 1 user staff 3798 Jul 10 00:28 localstack_providers_override.tf -rwxrwxrwx@ 1 user staff 5563 Jul 6 23:51 main.tf drwxr-xr-x 3 user staff 96 Jul 10 00:28 target/ -rw-r--r-- 1 user staff 17338 Jul 10 00:29 terraform.tfstate ``` ### Using multiple TF files Organizing multiple Terraform files into subfolders can be a highly effective strategy. This approach allows you to manage multiple Terraform projects within a single structure efficiently. The scripts are executed using a preorder traversal method, where each level of the directory hierarchy is processed in alphabetical order. This ensures a consistent and predictable execution sequence. For example, consider the following directory structure: ```bash ready.d/myscript.sh ready.d/a/script_0.sh ready.d/a/aa/script_0.sh ready.d/a/aa/script_2.sh ready.d/b/script_0.sh ``` This alphabetical and hierarchical execution strategy helps maintain an organized and logical flow, making it easier to manage and execute complex Terraform projects. ## Conclusion Terraform init hooks will not only allow us to replicate our production infrastructure within our testing environments, but will also bring great value in terms of automation - configurations being automatically applied, self-contained tests, and reproducibility - we can easily reproduce the setup every time. This is crucial for maintaining the integrity and reliability of our systems, as it enables thorough testing under conditions that closely mirror the actual deployment scenario. By preserving this production-ready setup throughout the testing phase, we can confidently validate changes and catch potential issues early, enhancing our deployment quality and operational stability. # Welcome to LocalStack for Azure Docs > Get started with LocalStack for Azure docs. import { OverviewCards, HeroCards } from '../../../components/OverviewCards'; import rocketIcon from '../../../assets/images/GettingStarted_Color.svg'; import connectionsIcon from '../../../assets/images/Integrations_Color.svg'; import cubeIcon from '../../../assets/images/LSAWS_Color.svg'; import fileIcon from '../../../assets/images/SampleApps_Color.svg'; import pipelineIcon from '../../../assets/images/CIPipelines_Color.svg'; ## What would you like to do today? # Changelog > Changelog for LocalStack for Azure. This changelog tracks updates to LocalStack for Azure support, including new services, enhancements, and compatibility fixes. Starting with the end-of-March 2026 release, LocalStack for Azure follows [calendar versioning](https://calver.org/) in the `YYYY.MM.patch` format. For example, `2026.03.0` is the initial March 2026 release. LocalStack for Azure 0.1.0 supports the following services: - [Azure API Management](https://azure.microsoft.com/en-us/products/api-management/) - [Azure App Service](https://azure.microsoft.com/en-us/products/app-service/) - [Azure RBAC](https://learn.microsoft.com/en-us/azure/role-based-access-control/) - [Azure Container Registry](https://azure.microsoft.com/en-us/products/container-registry/) - [Azure Kubernetes Service](https://azure.microsoft.com/en-us/products/kubernetes-service/) - [Azure Database for PostgreSQL](https://azure.microsoft.com/en-us/products/postgresql/) - [Azure Key Vault](https://azure.microsoft.com/en-us/products/key-vault/) - [Azure Resource Manager](https://azure.microsoft.com/en-us/get-started/azure-portal/resource-manager/) - [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs/) - [Azure Storage](https://azure.microsoft.com/en-us/products/category/storage/) - [Azure SQL](https://azure.microsoft.com/en-us/products/azure-sql/database/) # lstk CLI > Reference guide for lstk, the modern CLI for managing LocalStack, with installation, configuration, commands, and troubleshooting. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction `lstk` is a high-performance command-line interface for LocalStack, built in Go. It provides a built-in terminal UI (TUI) for interactive use and plain text output for CI/CD pipelines and scripting. `lstk` handles the full emulator lifecycle: authentication, pulling the Docker image, starting, stopping, and restarting the container, streaming logs, and checking status. It can also save and load emulator state (as local snapshots or Cloud Pods) reset running state, run AWS CLI commands against the emulator, and manage the on-disk volume. Running `lstk` with no arguments takes you through the entire startup flow automatically. `lstk` also proxies developer tools so they run directly against LocalStack: the AWS CLI (`lstk aws`), the Azure CLI (`lstk az`), Terraform (`lstk terraform`), the AWS CDK (`lstk cdk`), and the AWS SAM CLI (`lstk sam`). ## Prerequisites - [Docker](https://docs.docker.com/get-docker/) installed and running. - A [LocalStack account](https://www.localstack.cloud/pricing) with a [license](/azure/getting-started/auth-token/#managing-your-license), and `lstk` handles authentication for you (see [Authentication](#authentication)). ## Installation ```bash brew install localstack/tap/lstk ``` Homebrew also installs shell completions for bash, zsh, and fish automatically. ```bash npm install -g @localstack/lstk ``` Download the binary for your platform from [GitHub Releases](https://github.com/localstack/lstk/releases), extract it, and place it on your `PATH`. Verify the installation: ```bash lstk --version ``` ### Updating `lstk` can update itself. It detects how it was originally installed (Homebrew, npm, or binary) and uses the matching update method: ```bash # Check for updates without installing lstk update --check # Update to the latest version lstk update ``` See the [`update`](#update) command for details, including the start-time update notification. ## Quick start ```bash lstk ``` Running `lstk` without arguments performs the full startup sequence: authenticates you automatically, pulls the latest image if needed, and starts the LocalStack container. In an interactive terminal it launches the TUI; in a non-interactive environment it prints plain text output. On the very first interactive run, `lstk` prompts you to pick which emulator to run (AWS, Snowflake, or Azure) and writes your choice to `config.toml`. See [Emulator types](#emulator-types) for the available options. For CI or headless environments, set `LOCALSTACK_AUTH_TOKEN` and use `--non-interactive`: ```bash LOCALSTACK_AUTH_TOKEN= lstk --non-interactive ``` CI environments require a CI Auth Token; a personal Developer Auth Token cannot be used there. ## Authentication `lstk` resolves your auth token in the following order: 1. **`LOCALSTACK_AUTH_TOKEN` environment variable**: takes precedence over a stored token. 2. **System keyring**: a token stored by a previous `lstk login`, used when the environment variable is not set. 3. **Browser login**: triggered automatically in interactive mode when neither of the above provides a token. :::note `LOCALSTACK_AUTH_TOKEN` takes precedence over a token in the keyring. A per-invocation token (a CI secret, or `LOCALSTACK_AUTH_TOKEN=... lstk start` for a second account) therefore overrides a previous `lstk login` without needing `lstk logout` first. To go back to the stored token, unset the environment variable. ::: ### Logging in ```bash lstk login ``` Opens a browser window for authentication and stores the resulting token in your system keyring. This command requires an interactive terminal. See the [`login`](#login) command for the full flow and the endpoints it uses. ### Logging out ```bash lstk logout ``` Removes the stored credentials from the system keyring and the file-based fallback, and clears the cached license. `logout` cannot clear a token supplied via `LOCALSTACK_AUTH_TOKEN`; if you authenticated that way, unset the variable instead. See the [`logout`](#logout) command for the full behavior. ### File-based token storage On systems where the system keyring is unavailable, `lstk` automatically falls back to storing the token in a file (`/auth-token`, mode `0600`). You can force file-based storage by setting: ```bash export LSTK_KEYRING=file ``` ## Configuration `lstk` uses a TOML configuration file, created automatically on first run. ### Config file search order `lstk` uses the first `config.toml` it finds in this order: 1. `./.lstk/config.toml`: project-local config in the current directory. 2. `$HOME/.config/lstk/config.toml`: user config (created here if `$HOME/.config/` exists). 3. OS default: - **macOS**: `$HOME/Library/Application Support/lstk/config.toml` - **Windows**: `%AppData%\lstk\config.toml` - **Linux**: `$XDG_CONFIG_HOME/lstk/config.toml` or `$HOME/.config/lstk/config.toml` On first run, the config is created at path #2 if `$HOME/.config/` already exists; otherwise at the OS default (#3). To see the active config file path: ```bash lstk config path ``` To use a specific config file: ```bash lstk --config /path/to/config.toml start ``` ### Default configuration The default `config.toml` created on first run: ```toml [[containers]] type = "aws" # Emulator type. Supported: "aws", "snowflake", "azure" tag = "latest" # Docker image tag, e.g. "latest", "2026.4" port = "4566" # Host port the emulator will be accessible on # container_name = "" # Override the derived container name (also MAIN_CONTAINER_NAME) # image = "" # Full image override (e.g. an internal mirror or offline image) # expose_ports = [] # Extra container ports to publish, e.g. [53] for the DNS server # volume = "" # Host directory for persistent state (default: OS cache dir) # volumes = [] # Docker-style "host:container[:ro]" bind mounts (see Volumes) # env = [] # Named environment profiles to apply (see [env.*] sections below) # snapshot = "" # Snapshot REF to auto-load after start (AWS only) ``` ### Config field reference | Field | Type | Default | Description | |:-----------|:---------|:-----------|:-----------------------------------------------------------------------------------------------------| | `type` | string | `"aws"` | Emulator type. One of `"aws"`, `"snowflake"`, `"azure"`. Run a single `[[containers]]` block at a time. See [Emulator types](#emulator-types). | | `tag` | string | `"latest"` | Docker image tag (`"latest"`, `"2026.4"`, etc.). Useful for pinning a specific version. Zero-padded months (`"2026.04"`) are normalized to `"2026.4"`. | | `port` | string | `"4566"` | Host port the emulator listens on (1–65535). The in-container port is always `4566`. | | `container_name` | string | (derived) | Override the derived container name (`localstack-`, plus `-` when `tag` is not `"latest"`). This is also what the emulator reports as `MAIN_CONTAINER_NAME`. Set it when something outside `lstk` addresses the emulator by a fixed name, e.g. a sidecar proxy on a CI agent. | | `image` | string | (default) | Full image reference that overrides the default Docker Hub image, e.g. an internal-registry mirror or a locally loaded offline image. If it already carries a tag, `tag` is ignored; otherwise `tag` (or `latest`) is appended. | | `expose_ports` | (int \| string)[] | `[]` | Publish additional container ports on the host, beyond the gateway and service ports `lstk` publishes by default. Each entry is a bare port number (published on the same host port) or a Docker-style `"[host:]container[/proto]"` string — e.g. `expose_ports = [53]` to use the emulator's DNS server as the host's resolver, or `expose_ports = ["5354:5353/udp"]`. | | `volume` | string | (OS cache) | Host directory for persistent emulator state. Defaults to `/lstk/volume/`. See also `volumes`. | | `volumes` | string[] | `[]` | Docker-style `"host:container[:ro]"` bind mounts (e.g. init hooks). May also carry the persistence mount (target `/var/lib/localstack`). See [Volume mounts](#volume-mounts). | | `env` | string[] | `[]` | List of named environment profiles to inject into the container (see below). | | `snapshot` | string | `""` | Snapshot REF (e.g. `pod:my-baseline` or a local path) to auto-load after the emulator starts. AWS emulator only. See [Auto-loading a snapshot on start](#auto-loading-a-snapshot-on-start). | :::note There is no `update_prompt` config key. `lstk` always checks for available updates on startup. Once you choose to skip a version, `lstk` records it under the `[cli]` table as `update_skipped_version` and stops prompting for that version. This value is written automatically and is not meant to be hand-edited (see [`update`](#update)). ::: ### Emulator types `lstk` can run more than one kind of emulator. The `type` field in your `config.toml` selects which one: | Type | Docker image | Description | |:------------|:------------------------------|:-------------------------------------| | `aws` | `localstack/localstack-pro` | LocalStack AWS emulator (default). | | `snowflake` | `localstack/snowflake` | LocalStack Snowflake emulator. | | `azure` | `localstack/localstack-azure` | LocalStack Azure emulator. | On the first interactive run, `lstk` prompts you to pick an emulator (`a` for AWS, `s` for Snowflake, `z` for Azure) and writes your choice to `config.toml`. In non-interactive mode the default `aws` emulator is used if no config file is found. Lifecycle commands operate on the emulators defined in your `config.toml`. Run a single `[[containers]]` block at a time; the AWS-specific commands (`status` resources, `aws`, `reset`, `setup aws`) require an `aws` emulator to be configured. :::note The AWS emulator's license is validated by `lstk` before the container starts. The Snowflake and Azure emulators validate their own license inside the container at startup, so `lstk` skips its pre-flight license check for them. If your license does not include the selected emulator, the container exits and `lstk` reports the missing entitlement. ::: ### Passing environment variables to the container Define reusable environment profiles under `[env.]` and reference them in your container config: ```toml [[containers]] type = "aws" tag = "latest" port = "4566" env = ["debug", "ci"] [env.debug] DEBUG = "1" ENFORCE_IAM = "1" PERSISTENCE = "1" [env.ci] SERVICES = "s3,sqs" EAGER_SERVICE_LOADING = "1" ``` When `lstk start` runs, the key-value pairs from each referenced profile are injected as environment variables into the LocalStack container. Keys are uppercased automatically. :::note If you reference an `env` profile name that doesn't exist in your config, `lstk` returns an error: `environment "..." referenced in container config not found`. ::: In addition to your custom profiles, `lstk` always injects several variables into the container. See [Container-injected variables](#container-injected-variables) for the full list. ### Custom container image By default the emulator image is pulled from Docker Hub (`localstack/localstack-pro`, `localstack/snowflake`, or `localstack/localstack-azure` depending on `type`). Set `image` on a container block to override it — for example, to pull from an internal-registry mirror or to run a locally loaded image in an air-gapped environment: ```toml [[containers]] type = "aws" image = "registry.internal.example.com/localstack/localstack-pro" tag = "2026.4" ``` If `image` already carries a tag (e.g. `...:2026.4`), the separate `tag` field is ignored; otherwise `tag` (or `latest`) is appended. See [Offline and enterprise environments](#offline-and-enterprise-environments) for how `lstk` falls back to a locally present image when a pull fails. ### Volume mounts Beyond the single persistence directory set by `volume`, a container block can declare arbitrary Docker-style bind mounts with `volumes`. Each entry is a `"host:container[:ro]"` spec — useful, for example, for mounting an initialization script into `/etc/localstack/init/{boot,start,ready,shutdown}.d`, or a directory to persist state across restarts: ```toml [[containers]] type = "azure" port = "4566" volumes = [ "./init-ready.sh:/etc/localstack/init/ready.d/init-ready.sh", "./data:/var/lib/localstack", ] ``` - A `volumes` entry whose container target is `/var/lib/localstack` sets the persistence directory (the same mount `volume` configures); this is what [`lstk volume path`](#volume) and [`lstk volume clear`](#volume) resolve. - Relative host sources and a leading `~/` are resolved against the config file's directory. This differs from the legacy `volume` field, whose value is passed to Docker verbatim. - Setting the persistence directory through both `volume` and a `volumes` entry with a different source is a validation error. `volume` and `volumes` overlap only for the persistence mount: `volume` can *only* set the persistence directory, while `volumes` is a superset that can also express init hooks and other mounts. ### Using a project-local config Place a `.lstk/config.toml` in your project directory. When you run `lstk` from that directory, the local config takes precedence over the global one. This lets each project pin its own emulator type, image tag, and environment profiles. For example, a project that targets the Snowflake emulator can keep its own config: ```toml # .lstk/config.toml [[containers]] type = "snowflake" port = "4566" ``` An AWS project might instead pin a specific image tag and enable a debug profile: ```toml # .lstk/config.toml [[containers]] type = "aws" tag = "2026.4" port = "4566" env = ["dev"] [env.dev] DEBUG = "1" PERSISTENCE = "1" ``` ## Commands `lstk` uses a flat command structure. Running `lstk` with no command is equivalent to `lstk start`. ### `start` Start the LocalStack emulator. Launches the TUI in interactive terminals and prints plain output otherwise. `lstk start` launches the emulator defined in the first `[[containers]]` entry of the resolved `config.toml` (not necessarily AWS). ```bash lstk start lstk start --persist lstk start --non-interactive ``` | Option | Description | |:--------------------|:-----------------------------------------------------------------------------| | `--persist` | Persist emulator state across restarts (sets `LOCALSTACK_PERSISTENCE=1` in the container) | | `--type `, `-t ` | Select the emulator to start (`aws`, `snowflake`, or `azure`) non-interactively, recording the choice in `config.toml`. See [Selecting the emulator with `--type`](#selecting-the-emulator-with---type). | | `--snapshot ` | Auto-load this snapshot after the emulator starts, overriding the configured `snapshot` for one run (AWS only) | | `--no-snapshot` | Skip auto-loading the configured `snapshot` for this run | | `--timeout ` | Maximum time to wait for the emulator to become ready, as a Go duration (e.g. `90s`, `2m`). Overrides `LSTK_STARTUP_TIMEOUT` for this run; `0` uses the per-mode default. | | `--non-interactive` | Disable the interactive TUI and use plain output | `lstk start` forwards host environment variables prefixed with `LOCALSTACK_` to the emulator (the host `LOCALSTACK_AUTH_TOKEN` is dropped so it cannot override the token `lstk` resolved). See [Container-injected variables](#container-injected-variables). `lstk` applies a readiness deadline while waiting for the emulator to come up (a crash during startup is detected instantly, with its exit code, and does not wait for the deadline). In an interactive terminal the deadline defaults to 20 seconds and is only a recoverable prompt — you can keep waiting or stop; in non-interactive mode it defaults to 60 seconds and is fatal, leaving the container running for inspection. Override the deadline for a single run with `--timeout` (a Go duration such as `90s` or `2m`), or for every run with [`LSTK_STARTUP_TIMEOUT`](#environment-variables); an explicit `--timeout` wins over the environment variable, and `--timeout 0` falls back to the per-mode default. The flag is available on `start` and the bare `lstk` command only — `restart` and the snapshot auto-start path do not expose it. By default the emulator starts with a fresh state on every run. Pass `--persist` to keep data across restarts: `lstk` injects `LOCALSTACK_PERSISTENCE=1` into the container so state is written to the mounted [`volume`](#config-field-reference) and reloaded on the next start. When persistence is active, the AWS emulator's startup summary includes a `• Persistence: Enabled` line. ```bash # Start with persistent state lstk start --persist ``` :::note `--persist` is a flag on `start` (and the bare `lstk` command) and on [`restart`](#restart). For finer-grained control, you can also set `PERSISTENCE = "1"` in an environment profile (see [Passing environment variables to the container](#passing-environment-variables-to-the-container)). ::: `start` supports [`--json`](#structured-output) (as does the bare `lstk` command, which reports `"command": "start"`): the `data` payload is a flat object describing the started emulator — its `emulator` type, `container` name, `endpoint`, `version`, whether it was `alreadyRunning`, and whether `persistence` is enabled. #### Selecting the emulator with `--type` `--type` (shorthand `-t`, also available on the bare `lstk` command) is the non-interactive answer to the first-run emulator picker. It selects which emulator to start (`aws`, `snowflake`, or `azure`) and **records the choice in `config.toml`**, so lifecycle commands (`stop`, `status`, `logs`, `volume`, snapshot auto-load) stay in sync with what you started. ```bash # Start the Snowflake emulator, recording the choice in config lstk start --type snowflake # Shorthand lstk start -t azure ``` - On first run, the config is created with the selected type. - If the configured type already matches, `--type` is a no-op. - If it differs, `lstk` rewrites the `type` line in place (comments and formatting preserved) and prints a note naming the config file. When switching an existing config to a different type: - A custom `image` is a **hard error** — it pins a specific product that cannot be reinterpreted under a new emulator type. Use a separate config (`--config`) for that profile instead. - A non-`latest` `tag` and any `volume`/`volumes` mounts are kept, but `lstk` warns that they may be product-specific. - `port`, `env`, and `snapshot` are kept silently. `--type` is a flag only; passing the emulator as a positional (`lstk start azure`) is rejected with a hint pointing at `--type`. #### Auto-loading a snapshot on start For the **AWS emulator**, you can have `lstk` load a snapshot automatically every time it starts the emulator. Set the `snapshot` field on the container block to any load REF (a `pod:` Cloud Pod or a local path): ```toml [[containers]] type = "aws" port = "4566" snapshot = "pod:my-baseline" ``` The snapshot is loaded only when the emulator is **freshly started** this run; if it is already running, the auto-load is skipped. Override it for a single run with `--snapshot REF`, or skip it entirely with `--no-snapshot`: ```bash # Start and load a different snapshot for this run only lstk start --snapshot pod:other-baseline # Start without loading the configured snapshot lstk start --no-snapshot ``` The `snapshot` field is only read on start; [`snapshot save`](#snapshot-save) never writes it back into your config. ### `stop` Stop the running LocalStack emulator. Stops every emulator container defined in the resolved `config.toml` (the `[[containers]]` entries), with a 30-second stop timeout per container. ```bash lstk stop lstk stop --non-interactive ``` `stop` fails fast if the Docker runtime is not healthy (for example, Docker is not running), or if a configured emulator is not currently running (`LocalStack is not running`). In an interactive terminal it shows an animated "Stopping LocalStack..." spinner and a styled confirmation; in non-interactive mode it prints the same progress and result as plain text. `stop` supports [`--json`](#structured-output): the `data` payload lists each configured emulator and whether it `wasRunning`. ### `restart` Stop and restart the LocalStack emulator. Performs a stop of the running emulator followed by a fresh start, using the same auth, config, and Docker settings as [`start`](#start). Launches the TUI in interactive terminals and prints plain output otherwise. ```bash lstk restart lstk restart --persist ``` | Option | Description | |:-------------|:-------------------------------------------| | `--persist` | Persist emulator state across the restart | By default, emulator state is **not** retained across the restart and the container starts clean. Pass `--persist` to keep the emulator's state so it survives the restart. ### `status` Show the status of a running emulator and its deployed resources. Before contacting the emulator, `lstk` checks that the Docker runtime is healthy; if it is not, the command reports `runtime not healthy` and exits with a non-zero status. ```bash lstk status lstk --non-interactive status ``` For each emulator configured in your `config.toml` (the `[[containers]]` entries), `status` reports whether it is running and, if so, prints an instance summary: ```text LocalStack AWS Emulator is running • Endpoint: localhost:4566 • Persistence: Enabled • Container: localstack-aws • Version: 4.0.0 • Uptime: 1h 12m 4s ``` - **Endpoint** is the live `host:port`, queried from Docker, so it stays correct even if the configured `port` was changed while the container kept running. - **Persistence** appears only for the AWS emulator and only when persistence is enabled. - **Uptime** is computed from the container's start time and is omitted if it cannot be determined. If an emulator is not running, `status` prints an error and exits non-zero without checking the remaining emulators: ```text LocalStack AWS Emulator is not running Start LocalStack: lstk See help: lstk -h ``` For the **AWS emulator**, `status` additionally lists deployed resources. When resources exist it prints a summary line followed by a table; when none exist it prints `No resources deployed`. ```text ~ 3 resources · 2 services Service Resource Region Account S3 my-bucket us-east-1 000000000000 SQS my-queue us-east-1 000000000000 ``` In an interactive terminal the output is rendered through the TUI; in non-interactive mode (or with `--non-interactive`) the same content is printed as plain text, with the resource table shown at full width when stdout is not a TTY. The Snowflake and Azure emulators show the instance summary only and never report resources. `status` supports [`--json`](#structured-output): the `data` payload lists one entry per configured emulator with its running state, health, version, and host. For the AWS emulator it also includes a `resourceSummary` and the deployed `resources`, which `--no-resources` omits for a faster response when polling. `--json` also honors [`--endpoint-url`](#targeting-an-external-emulator) to report on an emulator `lstk` did not start. ### `logs` Show or stream emulator logs. ```bash lstk logs [options] ``` | Option | Description | |:------------|:-----------------------------------------| | `--follow`, `-f` | Stream logs in real-time. Without this flag, `lstk` prints the currently available logs and exits. | | `--verbose`, `-v` | Show all logs without filtering. By default, `lstk` drops noisy lines (internal request logs, provider chatter); `--verbose` shows every line verbatim. | | `--tail `, `-n ` | Show only the last `N` lines from the end of the logs. Accepts a non-negative integer or `all` (the default, showing all available lines). | By default, `lstk logs` reads from the first configured emulator container and applies a noise filter. In an interactive terminal, lines are color-coded by log level (`DEBUG`, `INFO`, `WARN`, `ERROR`); in non-interactive mode, raw log lines are written to stdout. Example: ```bash # Print current filtered logs and exit lstk logs # Stream filtered logs in real-time lstk logs --follow # Show only the last 100 lines lstk logs --tail 100 # Stream all logs without filtering lstk logs --follow --verbose ``` ### `aws` Run AWS CLI commands against the running LocalStack emulator. `lstk aws` proxies your host `aws` CLI with the endpoint, credentials, and region pre-configured, so you don't have to pass `--endpoint-url` or set test credentials yourself. ```bash lstk aws s3 ls lstk aws sqs list-queues lstk aws s3 mb s3://my-bucket ``` It is equivalent to running: ```bash aws --endpoint-url http://localhost:4566 ``` with `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_DEFAULT_REGION` set automatically. Everything after `lstk aws` is forwarded verbatim to the host `aws` binary, including AWS CLI flags such as `--region` or `--output`. The exit code and `stdout`/`stderr` of the underlying `aws` process are passed through unchanged, so piping and interactive subcommands work as expected. | Option | Description | |:--------------------|:--------------------------------------------------------------------------------------------------| | `--account ` | Target a specific 12-digit LocalStack account (default `000000000000`). Must appear immediately after `lstk aws`, before the AWS CLI's own action. Falls back to a 12-digit `AWS_ACCESS_KEY_ID`. See [Selecting the account](#selecting-the-account). | | `--non-interactive` | Suppress the loading spinner. Unlike other commands, this flag is stripped before invoking `aws` (not forwarded). | :::note `lstk aws` does not start the emulator. The AWS emulator must already be running (`lstk start`), Docker must be healthy, and the host `aws` CLI must be installed and on your `PATH`. ::: #### Credentials and region `lstk aws` injects credentials in one of two ways: - **Profile mode**: if a complete `localstack` profile exists in both `~/.aws/config` and `~/.aws/credentials`, `lstk` appends `--profile localstack` and lets `aws` read the region, credentials, and endpoint from that profile. - **Profile-less mode**: if the profile is not present, `lstk` runs `aws` with `AWS_ACCESS_KEY_ID=test`, `AWS_SECRET_ACCESS_KEY=test`, and `AWS_DEFAULT_REGION=us-east-1` injected only when those variables are not already set in your environment. In this mode it also prints an informational note: `No AWS profile found, run 'lstk setup aws'`. Run [`lstk setup aws`](#setup) to create the `localstack` profile for use with the AWS CLI and SDKs. #### Endpoint resolution By default, `lstk` probes whether `localhost.localstack.cloud` resolves to `127.0.0.1` and uses `localhost.localstack.cloud:` if so, otherwise it falls back to `127.0.0.1:`. Set [`LOCALSTACK_HOST`](#environment-variables) to override the host:port used to reach LocalStack and skip the DNS probe. The port comes from the AWS container's `port` in `config.toml` (default `4566`). #### Selecting the account LocalStack derives the AWS account from the access key id it receives, so `lstk aws --account ` targets a specific 12-digit LocalStack account by controlling the credentials `aws` runs with (a neutral, real-looking `AKIA…`/`ASIA…` key never reaches the emulator): ```bash lstk aws --account 111111111111 s3 mb s3://my-bucket ``` The flag must appear immediately after `lstk aws`, before the AWS CLI's own action (placing it before `lstk aws` is a placement error; placing it after the action is not caught — `lstk` silently forwards it to the `aws` CLI, which then rejects it). When it is omitted, `lstk` falls back to a 12-digit `AWS_ACCESS_KEY_ID` if one is set, then to the default account `000000000000`. The same leading-flag account selection is available on [`lstk terraform`](#terraform) and [`lstk sam`](#sam); `lstk cdk` does not support it. ### `az` Run Azure CLI commands against the running LocalStack Azure emulator. `lstk az` runs `az` with an isolated `AZURE_CONFIG_DIR` in which a custom Azure cloud is registered against LocalStack's endpoints, so your global `~/.azure` configuration is left untouched and plain `az` keeps talking to real Azure. Run [`lstk setup azure`](#setup-azure) once before using this mode. Arguments are forwarded to the host `az` binary, and its exit code and output are passed through unchanged. `lstk`'s own flags (`--non-interactive`, `--config`) are consumed by `lstk` rather than forwarded — for example `lstk az --non-interactive …` suppresses the loading spinner instead of passing the flag to `az`. ```bash lstk az group list lstk az storage account list ``` The Azure CLI has no `--endpoint-url`/`--profile` equivalent, so the isolation relies entirely on the dedicated config directory prepared by `setup azure`. #### Global interception (optional) If a script must invoke plain `az` (not `lstk az`), you can redirect your **global** `~/.azure` to LocalStack instead: ```bash # Point global 'az' at the LocalStack Azure emulator lstk az start-interception # Switch back to real Azure lstk az stop-interception ``` `start-interception` registers and activates the `LocalStack` cloud in your global Azure configuration so every `az` invocation targets LocalStack until you stop it. `stop-interception` switches the active cloud back to `AzureCloud` (override with `--cloud `) and re-enables instance discovery, but only when `LocalStack` is still the active cloud, to avoid clobbering an unrelated selection. :::caution Interception changes global state that affects every `az` command in any terminal. Use the isolated `lstk az ` mode unless you specifically need plain `az` to target LocalStack. ::: ### `terraform` Run Terraform against LocalStack, using LocalStack endpoints as AWS provider overrides. `lstk terraform` (alias `lstk tf`) generates a provider-override file and forwards your arguments to the real `terraform` binary. :::note `lstk terraform` targets the AWS emulator. To use Terraform with the other emulators, see the relevant emulator docs. ::: ```bash lstk terraform init lstk terraform --region us-west-2 plan lstk tf apply ``` lstk-specific flags must appear **before** the Terraform action: | Option | Default | Description | |:------------------|:---------------------|:---------------------------------------| | `--region ` | `us-east-1` | Deployment region. | | `--account ` | `test` | Target AWS account id (12 digits). | Relevant environment variables: `AWS_ENDPOINT_URL` (override the auto-resolved endpoint), `LSTK_TF_CMD` (binary to invoke, e.g. `tofu`; default `terraform`), `LSTK_TF_OVERRIDE_FILE_NAME` (override file name; default `localstack_providers_override.tf`), `LSTK_TF_DRY_RUN` (generate the override file but do not run Terraform), `AWS_REGION` (fallback for `--region`), and `AWS_ACCESS_KEY_ID` (fallback for `--account`). ### `cdk` Run the AWS CDK against LocalStack. Requires the AWS CDK CLI version `2.177.0` or newer on your `PATH`. ```bash lstk cdk bootstrap lstk cdk --region us-west-2 deploy lstk cdk synth ``` The only lstk-specific flag (before the CDK action) is `--region ` (default `us-east-1`); CDK always targets the default LocalStack account `000000000000`, so there is no `--account` flag. Relevant environment variables: `AWS_ENDPOINT_URL`, `AWS_ENDPOINT_URL_S3`, `LSTK_CDK_CMD` (default `cdk`), and `AWS_REGION`. ### `sam` Run the AWS SAM CLI against LocalStack. Requires the AWS SAM CLI version `1.95.0` or newer on your `PATH` (older versions ignore `AWS_ENDPOINT_URL` and would target real AWS). ```bash lstk sam build lstk sam --region us-west-2 deploy lstk sam validate ``` lstk-specific flags (before the SAM action): `--region ` (default `us-east-1`) and `--account ` (12 digits, default `000000000000`). Relevant environment variables: `AWS_ENDPOINT_URL`, `AWS_ENDPOINT_URL_S3`, `LSTK_SAM_CMD` (default `sam`), `AWS_REGION` (fallback for `--region`), and `AWS_ACCESS_KEY_ID` (fallback for `--account`). :::note Compared with `samlocal`, image/container-based Lambda (ECR) deploys and nested CloudFormation stacks are not supported; use `samlocal` for those workflows. ::: :::note Like `lstk aws`, the `az`, `terraform`, `cdk`, and `sam` proxies do not start the emulator — start it first with `lstk start`. Each requires the corresponding third-party CLI to be installed and on your `PATH`. ::: :::note When you interrupt a proxied tool (for example Ctrl+C or `kill` during `lstk terraform apply`), `lstk` forwards the termination signal to the wrapped tool and waits for it to shut down cleanly rather than killing it outright, so operations like releasing a Terraform state lock can complete. The wrapped tool's real exit code is passed through unchanged. ::: ### `snapshot` Manage emulator snapshots. A snapshot captures the running emulator's state, either as a local file on disk, as a Cloud Pod on the LocalStack platform, or in your own S3 bucket. The `snapshot` command groups six subcommands — `save`, `load`, `list`, `remove`, `show`, and `versions`. The first two are also exposed as the top-level aliases `lstk save` and `lstk load`. :::note Snapshots are best supported on the **AWS emulator**. `snapshot save`/`load` (and the `save`/`load` aliases) also work for the Snowflake emulator, but its snapshot support is experimental and not fully tested — `lstk` prints a warning such as `Snapshot support for the snowflake emulator is experimental and not fully tested.` The Azure emulator does not support snapshots. [`reset`](#reset) remains **AWS-only** and errors out with `reset is only supported for the AWS emulator` otherwise. ::: #### `snapshot save` Save a snapshot of the running emulator's state. The emulator must already be running; this command does **not** auto-start it. ```bash # Auto-named snapshot file in the current directory lstk snapshot save # Save to a specific local path lstk snapshot save ./my-snapshot # Save to a Cloud Pod on the LocalStack platform (requires auth) lstk snapshot save pod:my-baseline # Save to your own S3 bucket (pod name is auto-generated if omitted) lstk snapshot save my-pod s3://my-bucket/prefix # Limit the snapshot to a subset of services lstk snapshot save --services s3,lambda ``` The optional `[destination]` argument takes one of these forms: | Destination | Description | |:--------------------------------|:------------------------------------------------------------------------------------------------| | (omitted) | Auto-generates a timestamped snapshot file in the current directory (`./snapshot--.snapshot`). | | local path | Writes a snapshot archive to that path. The `.snapshot` extension is forced. | | `pod:` | Saves a Cloud Pod to the LocalStack platform. Requires authentication. | | ` s3://bucket/prefix` | Saves to your own S3 bucket. The pod name is a separate positional (auto-generated when omitted). See [S3 remotes](#s3-remotes). | Pod operations require an auth token (`LOCALSTACK_AUTH_TOKEN` or a prior `lstk login`); local-file snapshots do not. Every save to an existing `pod:` snapshot creates a new **version** rather than replacing it; use [`snapshot versions`](#snapshot-versions) to list them and [`snapshot load`](#snapshot-load)/[`snapshot show`](#snapshot-show) with a `pod::` ref to act on a specific one. `save` itself rejects a version suffix (you cannot save "as version 3"). By default a snapshot captures every service's state. Pass `-s`/`--services` with a comma-separated list to limit it to a subset; this applies uniformly to local files, `pod:` Cloud Pods, and `s3://` remotes. | Option | Description | |:--------------------|:----------------------------------------------------------------------------------------------| | `--services `, `-s ` | Comma-separated list of services to include in the snapshot (all services by default). Applies to local, `pod:`, and `s3://` destinations. | | `--profile ` | AWS profile to read S3 credentials from (used only for `s3://` destinations). Defaults to `AWS_*` env vars, then `AWS_PROFILE`. | #### `snapshot load` Load a snapshot into the emulator, **auto-starting it first** if it is not already running. ```bash # Load a local snapshot by path or name lstk snapshot load my-baseline lstk snapshot load ./checkpoint # Load from a Cloud Pod (requires auth; latest version) lstk snapshot load pod:my-baseline # Load a specific version of a Cloud Pod lstk snapshot load pod:my-baseline:3 # Load from your own S3 bucket (pod name is required) lstk snapshot load my-pod s3://my-bucket/prefix # Control how the snapshot merges with running state lstk snapshot load pod:my-baseline --merge=overwrite # Preview what a Cloud Pod load would change, without applying it lstk snapshot load pod:my-baseline --dry-run ``` The `REF` argument is required and identifies a local path/name or a `pod:` Cloud Pod. For a Cloud Pod you can append a version (`pod::`) to load an older version; the latest is used when no version is given. To load from S3, pass the pod name followed by an `s3://bucket/prefix` location (see [S3 remotes](#s3-remotes)). | Option | Description | |:---------------------|:--------------------------------------------------------------------------------------------------------| | `--merge ` | How the loaded state combines with running state. One of `account-region-merge` (default), `overwrite`, `service-merge`. | | `--dry-run` | Preview the resource additions and modifications the load would produce, per service, without changing any state. Supported for `pod:` refs only; requires a running emulator (it does not auto-start one). | | `--profile ` | AWS profile to read S3 credentials from (used only for `s3://` sources). Defaults to `AWS_*` env vars, then `AWS_PROFILE`. | - `account-region-merge` (default): the snapshot wins on any `(service, account, region)` overlap. - `overwrite`: running state is reset first, then the snapshot is imported onto a clean state. - `service-merge`: the snapshot wins per resource; non-overlapping resources are combined. Set [`LSTK_MERGE_STRATEGY`](#environment-variables) to change the default strategy used when `--merge` is not passed; an explicit `--merge` always wins. Pass `--dry-run` with a `pod:` ref to preview a load before committing to it: `lstk` queries the platform and prints, per service, how many resources the snapshot would add or modify under the chosen merge strategy, without touching running state. It is supported for `pod:` refs only (other refs are rejected) and requires the emulator to already be running, since it does not auto-start one. The aliases behave identically: ```bash lstk save pod:my-baseline lstk load ./checkpoint ``` #### `snapshot list` List the Cloud Pod snapshots available on the LocalStack platform. By default, only snapshots you created are listed; pass `--all` to include every snapshot in your organization. This subcommand operates on Cloud Pods, so it requires authentication. ```bash # Snapshots you created lstk snapshot list # Every snapshot in your organization lstk snapshot list --all # List snapshots in your own S3 bucket (requires a running emulator) lstk snapshot list s3://my-bucket/prefix ``` Passing an `s3://bucket/prefix` location lists snapshots stored in your own S3 bucket instead of the platform (see [S3 remotes](#s3-remotes)). Unlike the platform listing, this queries the emulator, so it requires a running emulator. | Option | Description | |:-------------------|:-------------------------------------------------------------| | `--all` | List all snapshots in your organization, not just your own. | | `--profile ` | AWS profile to read S3 credentials from (used only with an `s3://` location). Defaults to `AWS_*` env vars, then `AWS_PROFILE`. | #### `snapshot remove` Delete a Cloud Pod snapshot from the LocalStack platform. Only cloud snapshots (the `pod:` prefix) can be removed; local snapshots are plain files you delete yourself. This operation cannot be undone. ```bash lstk snapshot remove pod:my-baseline # Skip the confirmation prompt (required in non-interactive mode) lstk snapshot remove pod:my-baseline --force ``` The required `REF` argument must be a `pod:` Cloud Pod reference. | Option | Description | |:----------|:----------------------------------------------------------------------| | `--force` | Skip the confirmation prompt. Required when running non-interactively. | #### `snapshot show` Show metadata for a single Cloud Pod snapshot on the LocalStack platform: its name, created date, size, LocalStack version, message, the services it contains, and per-service resource counts (resource counts render only when the platform has them for that snapshot). This subcommand is cloud-only and requires authentication. ```bash # Latest version lstk snapshot show pod:my-baseline # A specific version lstk snapshot show pod:my-baseline:3 ``` The required `REF` argument must be a `pod:` Cloud Pod reference. It defaults to the latest version; append `:` to inspect an older one. Use [`snapshot versions`](#snapshot-versions) to see which versions exist. #### `snapshot versions` List the version history of a Cloud Pod on the LocalStack platform. Every save to an existing pod adds a new version; this prints each version's number, created date, LocalStack version, and services. This subcommand is cloud-only and requires authentication. ```bash lstk snapshot versions pod:my-baseline ``` The required `REF` argument must be a `pod:` Cloud Pod reference. Only Cloud Pods have versions — local files and `s3://` remotes do not, and passing a version suffix to `versions` is rejected. Act on a specific version elsewhere by appending it to the ref, e.g. `lstk snapshot load pod:my-baseline:3` or `lstk snapshot show pod:my-baseline:3`. #### S3 remotes `snapshot save`, `load`, and `list` can target a snapshot stored in your **own S3 bucket** by passing an `s3://bucket/prefix` location. The pod name (the snapshot's identity within the bucket) is a positional separate from the `s3://` location — required for `load`, auto-generated for `save` when omitted, and unused for `list`. ```bash lstk snapshot save my-pod s3://my-bucket/prefix lstk snapshot load my-pod s3://my-bucket/prefix lstk snapshot list s3://my-bucket/prefix ``` Credentials follow AWS CLI precedence: `--profile ` wins, otherwise the static `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` (plus optional `AWS_SESSION_TOKEN`) environment variables, otherwise the profile named by `AWS_PROFILE`. Only static credentials are supported (no SSO, assume-role, or `credential_process`), and credentials must never be embedded in the URL. `lstk` runs a pre-flight check that the target bucket exists and errors out rather than letting the emulator auto-create a bucket on a typo. Because the transfer is performed by the emulator (not the CLI), S3 remotes require a **running emulator**, and `list s3://…` in particular queries the emulator rather than the platform API. :::note `remove` and `show` do not support S3; they operate on Cloud Pods only. ::: ### `reset` Discard the running AWS emulator's in-memory state (all created resources such as S3 buckets and Lambda functions are dropped). The emulator **keeps running**; only its state is cleared. ```bash lstk reset lstk reset --force ``` | Option | Description | |:----------|:----------------------------------------------------------------| | `--force` | Skip the confirmation prompt. Required in non-interactive mode. | In interactive mode, `reset` prompts for confirmation before clearing state. In non-interactive mode it fails unless `--force` is passed: ```text reset requires confirmation; use --force to skip in non-interactive mode ``` `reset` supports [`--json`](#structured-output): on success the `data` payload reports the reset emulator and `"reset": true`. :::note `reset` clears in-memory state only. It does **not** wipe the on-disk volume (certificates, persistence data, cached tools). To clear that, stop the emulator and run [`lstk volume clear`](#volume). ::: ### `volume` Manage the emulator volume: the host directory that holds persistent state such as certificates, downloaded tools, and persistence data. ```bash lstk volume path lstk volume clear [options] ``` #### `volume path` Prints the resolved volume directory for every emulator in your config, one per line. With the default config (a single `aws` emulator) it prints one path. Each path is the container's configured `volume` value, or the default OS cache location if `volume` is unset (`~/Library/Caches/lstk/volume/localstack-aws` on macOS, `~/.cache/lstk/volume/localstack-aws` on Linux). ```bash # Print the volume directory for each configured emulator lstk volume path ``` #### `volume clear` Removes all data from the emulator volume directory, resetting cached state. It operates on all configured emulators by default, or a single one with `--type`. Before clearing, it lists each target as `: ()`. | Option | Description | |:----------------|:-----------------------------------------| | `--force` | Skip the confirmation prompt | | `--type ` | Clear only the emulator of this type | ```bash # Clear all configured emulator volumes (prompts for confirmation) lstk volume clear # Clear only the AWS emulator volume lstk volume clear --type aws # Skip the confirmation prompt lstk volume clear --force # Clear without prompting in a non-interactive environment lstk volume clear --type snowflake --force ``` In an interactive terminal, `lstk volume clear` prompts `Clear volume data? This cannot be undone` before deleting anything; choosing **NO** or pressing Ctrl+C cancels with no changes. In non-interactive mode, `--force` is required, otherwise the command fails with `volume clear requires confirmation; use --force to skip in non-interactive mode`. :::caution If the volume contains files owned by `root` (created by Docker), clearing fails with a permission error. Re-run with elevated privileges: ```bash sudo lstk volume clear ``` ::: ### `login` Authenticate with LocalStack via a browser-based device authorization flow and store the resulting credential in your system keyring. This command requires an interactive terminal. ```bash lstk login ``` `lstk` opens your default browser to the LocalStack Web Application, shows a one-time code, and waits for you to approve the request. If the browser cannot open automatically, `lstk` prints the URL to visit manually. On success it stores the **license token** returned by the platform (not the raw browser bearer token). If you are already authenticated — either `LOCALSTACK_AUTH_TOKEN` is set or a token already exists in storage — `login` prints `You're already logged in` and exits without starting a new flow. In non-interactive mode (piped output, CI, or `--non-interactive`), `login` fails with `login requires an interactive terminal`. The `--config ` flag selects which `config.toml` is loaded, which affects `keyring`, `web_app_url`, and `api_endpoint` resolution. :::note If you approve the request in the browser only *after* pressing a key in the terminal, `lstk` reports `auth request not confirmed - please complete the authentication in your browser`. Re-run `lstk login` and approve in the browser before continuing. ::: The credential is written to the system keyring (service `lstk`, key `lstk.auth-token`). When the keyring is unavailable — or `LSTK_KEYRING=file` is set — `lstk` stores it in a file at `/auth-token` (mode `0600`) instead. Endpoints used by the flow can be overridden via config or environment: | Config key | Env var | Default | Description | |:---------------|:--------------------|:-------------------------------|:-----------------------------------------------------------------------------| | `keyring` | `LSTK_KEYRING` | (system keyring) | Set to `file` to force file-based token storage instead of the OS keyring. | | `web_app_url` | `LSTK_WEB_APP_URL` | `https://app.localstack.cloud` | Base URL used to build the browser authorization link. | | `api_endpoint` | `LSTK_API_ENDPOINT` | `https://api.localstack.cloud` | LocalStack platform API endpoint used for the device flow and license token. | ```bash # Force file-based token storage during login LSTK_KEYRING=file lstk login # Use a specific config file lstk --config ./.lstk/config.toml login ``` ### `logout` Remove stored authentication credentials. ```bash lstk logout lstk logout --non-interactive ``` `logout` deletes the auth token from your system keyring (falling back to the file-based token at `/auth-token` when the keyring is unavailable or `LSTK_KEYRING=file` is set) and removes the cached license file. On success it prints `Logged out successfully`. The outcome depends on how you are authenticated: | Situation | Behavior | |:----------|:---------| | A token is stored (from `lstk login`) | The token is deleted from the keyring and file fallback, the cached license is removed, and `lstk` prints `Logged out successfully`. | | No stored token, but `LOCALSTACK_AUTH_TOKEN` is set | Nothing is deleted. `lstk` prints a note that you are authenticated via the environment variable and to unset it to log out. | | No stored token and no `LOCALSTACK_AUTH_TOKEN` | `lstk` prints `Not currently logged in` and exits successfully. | :::note `logout` never clears the `LOCALSTACK_AUTH_TOKEN` environment variable, and it does not stop running emulators. If a LocalStack emulator is still running after logout, `lstk` prints a note reminding you it is running in the background; run `lstk stop` to stop it. ::: ### `setup` Set up CLI integration for an emulator type. `lstk setup` is a grouping command with no action of its own; the work is done by its subcommands, `setup aws` and `setup azure`. ```bash lstk setup aws lstk setup azure ``` #### `setup aws` Create or update a `localstack` profile in `~/.aws/config` and `~/.aws/credentials` so the AWS CLI and SDKs can target LocalStack. ```bash lstk setup aws lstk setup aws --force ``` | Option | Description | |:----------|:-----------------------------------------------------------------------------------------| | `--force` | Overwrite an existing `localstack` profile whose values differ, and skip the confirmation prompt. | On an interactive terminal it prompts (Y/n) before making changes. In non-interactive mode (piped output, CI, or `--non-interactive`) it writes the profile with defaults without prompting and exits `0`; a failed write or check returns a non-zero exit code so automation notices. Overwriting an existing `localstack` profile whose values differ requires `--force` (which also skips the interactive prompt); creating a fresh profile, completing a partial one, or leaving an already-correct profile in place never needs it. It writes the following profile (existing unrelated profiles are preserved): ```ini # ~/.aws/config [profile localstack] region = us-east-1 output = json endpoint_url = http://localhost.localstack.cloud:4566 # ~/.aws/credentials [localstack] aws_access_key_id = test aws_secret_access_key = test ``` Afterwards, target LocalStack by passing `--profile localstack` or exporting `AWS_PROFILE`: ```bash export AWS_PROFILE=localstack aws s3 ls ``` The endpoint host is resolved the same way as for [`lstk aws`](#endpoint-resolution) (probing `localhost.localstack.cloud` and falling back to `127.0.0.1`), and [`LOCALSTACK_HOST`](#environment-variables) overrides the host and port written into the profile. The port comes from your AWS emulator's configured `port` (default `4566`); if no `aws` emulator is configured, the command fails with `no aws emulator configured`. If the `localstack` profile is already configured correctly, `lstk` reports `LocalStack AWS profile is already configured.` and makes no changes. :::note The former `lstk config profile` command has been removed; use `lstk setup aws`. ::: #### `setup azure` Prepare an isolated Azure CLI configuration directory that routes [`lstk az`](#az) commands to the LocalStack Azure emulator. Your global `~/.azure` configuration is left untouched. ```bash lstk setup azure # alias: lstk setup az ``` `setup azure` registers a custom Azure cloud (`LocalStack`) whose endpoints point at the LocalStack Azure emulator, activates it, disables Azure CLI instance discovery and telemetry, and performs a one-time dummy service-principal login — all inside a dedicated config directory under the `lstk` config dir (via `AZURE_CONFIG_DIR`). It requires the `az` CLI to be installed and a running LocalStack Azure emulator. To instead redirect your **global** `az` (so existing scripts run unmodified against LocalStack), see [`lstk az start-interception`](#az). ### `config` Manage CLI configuration. `config` has no behavior of its own; run it with a subcommand. #### `config path` Print the resolved path to the active `config.toml`. ```bash lstk config path ``` This subcommand is read-only: it never creates or initializes a config file. If `--config ` is set, it prints that path verbatim. Otherwise it prints the already-loaded config path, the first existing config in the search order, or the path where a config would be created on first run. ### `update` Check for and apply updates to the `lstk` CLI itself. `lstk` auto-detects how it was installed (Homebrew, npm, or direct binary) and updates using that same method. Development builds (version `dev`) are skipped, and updates are checked against the latest [GitHub release](https://github.com/localstack/lstk/releases/latest). ```bash lstk update [options] ``` | Option | Description | |:--------------------|:-------------------------------------------------------------| | `--check` | Check for updates without installing them | | `--non-interactive` | Use plain output instead of the TUI (update logic unchanged) | | `--json` | Emit the result as a JSON envelope (see [Structured output](#structured-output)). With `--check`, `data` reports `currentVersion`/`latestVersion`/`updateAvailable`; after an applied update, `updatedVersion`/`updated`/`method`. | Examples: ```bash # Check for updates without installing lstk update --check # Update to the latest version lstk update # Update with plain (non-TUI) output lstk update --non-interactive ``` By install method: - **Homebrew** (binary under a `Caskroom` path): runs `brew upgrade localstack/tap/lstk`. - **npm** (binary under `node_modules`): runs `npm install -g @localstack/lstk@latest`. - **Binary** (anything else): downloads the release asset for your OS/arch from GitHub, verifies its SHA-256 against the release's `checksums.txt` (a missing, malformed, or mismatched checksum aborts the update), extracts it, and replaces the running executable in place. With `--check`, `lstk` only reports whether a newer version is available and exits without downloading or installing anything. :::note Set `LSTK_GITHUB_TOKEN` to send an authenticated GitHub request and avoid API rate limits during update checks. It is optional; updates also work unauthenticated. ::: If more than one `lstk` installation is found on your `PATH` (for example a Homebrew binary and an npm one), `lstk update` and the start-time update notification print a warning listing each location, its install method, and which one is currently running, so you can tell which binary an update will actually replace. #### Update notification on start Separately from `lstk update`, `lstk` checks for a newer version when you run `lstk start` (the default command), using a short timeout that fails silently if GitHub is unreachable. In an interactive terminal, when an update is available `lstk` prints the new version and a release-notes link, then prompts: ```text Update lstk to latest version? > Update now [U] Remind me next time [R] Skip this version [S] ``` - **Update now [U]**: downloads and applies the update, then asks you to re-run your command. - **Remind me next time [R]**: does nothing; you are reminded on the next run. - **Skip this version [S]**: records the version in `config.toml` so you are not prompted about it again. In non-interactive mode the notification is not a prompt — `lstk` emits a single note (`Update available: (run lstk update)`) and continues. When you choose **Skip this version**, `lstk` writes the skipped version under a `[cli]` table: ```toml [cli] update_skipped_version = "0.5.0" ``` While this value matches the latest available version, the start-time update notification for that version is suppressed. This key is managed automatically and is not intended to be edited by hand. ### `completion` Generate shell completion scripts. ```bash lstk completion [bash|zsh|fish|powershell] ``` See [Shell completions](#shell-completions) for setup instructions. ## Global options These options are available for all commands: | Option | Description | |:--------------------|:---------------------------------------------------------------------------| | `--config ` | Path to a specific TOML config file | | `--endpoint-url ` | Target an existing, externally-managed emulator at this URL instead of discovering one via local Docker. See [Targeting an external emulator](#targeting-an-external-emulator). | | `--non-interactive` | Disable the interactive TUI, use plain output | | `--json` | Emit a single machine-readable JSON envelope on stdout instead of human-oriented output. Supported by `start`, `stop`, `status`, `reset`, and `update`; any other command rejects it. See [Structured output](#structured-output). | | `--persist` | Persist emulator state across restarts (on `start`/bare `lstk` and `restart`) | | `--type `, `-t ` | Emulator type to start: `aws`, `snowflake`, or `azure` (on `start`/bare `lstk`; records the choice in config). See [Selecting the emulator with `--type`](#selecting-the-emulator-with---type). | | `--snapshot ` | Snapshot REF to auto-load after start (on `start`/bare `lstk`; overrides config for one run) | | `--no-snapshot` | Skip auto-loading the configured snapshot (on `start`/bare `lstk`) | | `--timeout ` | Startup readiness deadline for `start`/bare `lstk`, as a Go duration; overrides `LSTK_STARTUP_TIMEOUT` for one run. See [`start`](#start). | | `-v`, `--version` | Print the version and exit | | `-h`, `--help` | Print help and exit | ## Interactive and non-interactive mode `lstk` automatically selects its output mode: - **Interactive mode** (TUI): used when both stdin and stdout are connected to a terminal. Commands like `start`, `stop`, `restart`, `status`, `login`, `update`, and the confirmation prompts of `reset`/`volume clear` display a Bubble Tea-powered terminal UI. - **Non-interactive mode** (plain text): used when the output is piped, redirected, or running in CI. Force this in a TTY with `--non-interactive`. ```bash # Force plain output even in an interactive terminal lstk --non-interactive start ``` :::note `lstk login` requires an interactive terminal; if you need to authenticate in CI, set `LOCALSTACK_AUTH_TOKEN` instead. Commands that mutate state without prompting in CI (`reset`, `volume clear`) require `--force`. `lstk setup aws` works non-interactively — it writes the profile with defaults and needs `--force` only to overwrite a conflicting `localstack` profile. ::: ## Targeting an external emulator By default `lstk` discovers the emulator it manages through local Docker. The `--endpoint-url ` global flag (or the `LSTK_ENDPOINT_URL` environment variable) instead points a command at an emulator `lstk` did not start — a Docker Compose or host-network deployment, one running in CI or on another machine, or a LocalStack cloud-hosted ephemeral instance. ```bash # Run against an emulator reachable at a custom URL lstk az group list --endpoint-url http://localhost:4566 # Equivalent via the environment LSTK_ENDPOINT_URL=https://my-ephemeral-instance.localstack.cloud lstk status ``` The endpoint is resolved from, in order of precedence: the `--endpoint-url` flag, `LSTK_ENDPOINT_URL`, then `AWS_ENDPOINT_URL` (a full synonym for `LSTK_ENDPOINT_URL`, one tier lower). Both `http://` and `https://` URLs are accepted (any other scheme is rejected), and the scheme is preserved end-to-end, so `https://` ephemeral instances work. The commands that accept an external endpoint are the ones that only *talk to* an already-running emulator: `aws`, `az`, `terraform`/`tf`, `cdk`, `sam`, `status`, `reset`, and the `snapshot` `save`/`load`/`remove` subcommands (including the `lstk save`/`lstk load` aliases) and `list s3://…`. Commands that manage the emulator's lifecycle or on-disk state have no remote equivalent and **reject** any endpoint source: `start`, the bare `lstk`, `stop`, `restart`, `logs`, and `volume`. The emulator's type (AWS, Azure, or Snowflake) is auto-detected by probing the endpoint's health API — there is no override flag or config setting, and an inconclusive probe is a hard failure. The AWS-only tools (`terraform`, `cdk`, `sam`) reject an endpoint whose detected type is not AWS. ## Structured output The global `--json` flag makes a command emit a single, machine-readable JSON object on stdout instead of human-oriented text, for scripting and CI. JSON support is available per command: `start`, `stop`, `status`, `reset`, and `update` accept `--json`. Any other command rejects it with an error envelope (`error.code: NOT_JSON_CAPABLE`) rather than silently printing plain text. Every JSON-capable command writes **exactly one** JSON object with the following envelope shape: ```json { "schemaVersion": 1, "command": "stop", "status": "ok", "data": { "emulators": [ { "type": "aws", "name": "localstack-aws", "wasRunning": true } ] }, "warnings": [], "error": null } ``` | Field | Type | Description | |:----------------|:----------------|:-----------------------------------------------------------------------------------------------------| | `schemaVersion` | integer | Wire-format version of the envelope, always `1` for this schema. Check it once before parsing. | | `command` | string | The command that produced the envelope (e.g. `"stop"`, `"reset"`). | | `status` | string | `"ok"` or `"error"` — branch on this first. | | `data` | object or `null`| Command-specific result. Non-null when `status` is `"ok"`, `null` when it is `"error"`. | | `warnings` | array | Non-fatal notices, always present (empty array when there are none). Each entry is `{ "code", "message" }`. | | `error` | object or `null`| The machine-readable failure. Non-null when `status` is `"error"`, `null` otherwise. | When `status` is `"error"`, the `error` object carries a stable `code` (e.g. `EMULATOR_NOT_RUNNING`, `CONFIRMATION_REQUIRED`, `RUNTIME_UNAVAILABLE`), a coarse `category`, a human-readable `message` (informational only — branch on `code`, not `message`), and a `retryable` boolean: ```json { "schemaVersion": 1, "command": "reset", "status": "error", "data": null, "warnings": [], "error": { "code": "CONFIRMATION_REQUIRED", "category": "USAGE", "message": "reset requires confirmation; use --force to skip in non-interactive mode", "retryable": false } } ``` ### Exit codes For a full enumeration, read `error.code` from the envelope; the process exit code carries only the two most common, mechanically-remediable failures: | Exit code | Meaning | |:----------|:--------------------------------------------------------------------------------------------| | `0` | `status: "ok"`. | | `1` | `status: "error"` for any code other than the two below. | | `2` | A Cobra-level usage error that occurred before `--json` could be recognized (plain-text error on stderr, not an envelope). | | `3` | `error.code == "CONFIRMATION_REQUIRED"` (re-run with `--force`). | | `4` | `error.code == "AUTH_REQUIRED"` (run `lstk login` or set `LOCALSTACK_AUTH_TOKEN`). | :::note `--json` implies non-interactive behavior: no TUI and no prompts. Combining it with a destructive command that would otherwise prompt (`reset`) still requires `--force`, which surfaces as `CONFIRMATION_REQUIRED` (exit code `3`) when omitted. ::: ## Environment variables The following environment variables configure `lstk` itself (not the LocalStack container): | Variable | Description | |:-----------------------------|:-----------------------------------------------------------------------------------------------------------------| | `LOCALSTACK_AUTH_TOKEN` | Auth token for non-interactive runs or to skip browser login. Takes precedence over a token stored in the keyring. | | `LSTK_ENDPOINT_URL` | Target an existing, externally-managed emulator at this URL (equivalent to `--endpoint-url`). `AWS_ENDPOINT_URL` is a lower-precedence synonym. See [Targeting an external emulator](#targeting-an-external-emulator). | | `LOCALSTACK_HOST` | Override the host (and optional port) used when resolving and printing the emulator endpoint, and when writing the AWS CLI profile. Bypasses the `localhost.localstack.cloud` DNS probe. | | `LOCALSTACK_DISABLE_EVENTS` | Set to `1` to disable anonymous telemetry event reporting. | | `DOCKER_HOST` | Override the Docker daemon socket (e.g. `unix:///home/user/.colima/default/docker.sock`). | | `LSTK_KEYRING` | Set to `file` to force file-based token storage instead of the system keyring. | | `LSTK_STARTUP_TIMEOUT` | Startup readiness deadline for `lstk start`, as a Go duration (e.g. `90s`, `2m`). Zero/unset uses the per-mode default (20s interactive, 60s non-interactive). See [`start`](#start). | | `LSTK_MERGE_STRATEGY` | Default merge strategy for `snapshot load` / `load` (`account-region-merge`, `overwrite`, or `service-merge`) when `--merge` is not passed. An explicit `--merge` always wins. | | `LSTK_OTEL` | Set to `1` to enable OpenTelemetry trace export (disabled by default). See [OpenTelemetry tracing](#opentelemetry-tracing). | | `LSTK_GITHUB_TOKEN` | Optional GitHub token used when checking for or downloading `lstk` updates (raises GitHub API rate limits). | | `LSTK_API_ENDPOINT` | Override the LocalStack platform API base URL. Default: `https://api.localstack.cloud`. | | `LSTK_WEB_APP_URL` | Override the LocalStack Web Application URL used for browser login. Default: `https://app.localstack.cloud`. | When `LSTK_OTEL` is enabled, the standard `OTEL_EXPORTER_OTLP_*` environment variables are honored by the OpenTelemetry SDK. ### Container runtime discovery `lstk` talks to a Docker-compatible runtime and works with Docker Desktop, Rancher Desktop, Colima, OrbStack, Lima, and Podman. When `DOCKER_HOST` is not set, it resolves the daemon endpoint in this order: 1. **`DOCKER_HOST`**, if set, always wins. 2. **`DOCKER_CONTEXT`** or the active Docker CLI context, when it is non-default and reachable (a stale or unreachable context is skipped rather than failing). 3. On **Linux**, a live `/var/run/docker.sock` — a running Docker daemon is preferred over a co-installed runtime such as Podman. 4. A probe of known runtime sockets (Docker Desktop, Rancher Desktop, Colima, OrbStack, Podman, Lima). Each candidate is dialed, not just checked for existence, so a leftover socket file never shadows a live daemon. 5. The Docker SDK's own default. If no runtime is reachable, the error tailors its suggested start command (`rdctl start`, `colima start`, `podman machine start`, …) to the runtime it detects. Set `DOCKER_HOST` to point at a specific socket to bypass discovery entirely. ### Container-injected variables `lstk` injects several environment variables into the LocalStack container on every start, in addition to any profiles you configure: | Variable | Default value | Description | |:---------------------------|:---------------------------------------------|:---------------------------------------------| | `LOCALSTACK_AUTH_TOKEN` | (your resolved token) | Passed from the CLI to activate the license. | | `GATEWAY_LISTEN` | `:4566,:443` | Ports the emulator binds inside the container. | | `MAIN_CONTAINER_NAME` | `localstack-aws` | Container name for internal references. | | `LOCALSTACK_HOST` | `localhost.localstack.cloud:` | Hostname/port the emulator advertises. | | `LOCALSTACK_PERSISTENCE` | `1` (only with `--persist`) | Enables state persistence across restarts. | | `LOCALSTACK_CLIENT_NAME` | `lstk` | Identifies the client that started the emulator. | | `LOCALSTACK_CLIENT_VERSION`| (the `lstk` version) | Version of the client that started the emulator. | When a Docker socket is detected it is bind-mounted into the container and `DOCKER_HOST=unix:///var/run/docker.sock` is injected so the emulator can spawn its own containers. `lstk` also forwards host environment variables matching `CI` and `LOCALSTACK_*` (the host `LOCALSTACK_AUTH_TOKEN` is dropped so it cannot override the token resolved by `lstk`). The container also gets port mappings for `4566`, `443`, and the service port range `4510-4559`. :::note `GATEWAY_LISTEN` is read from the container's resolved environment (set it via an `[env.*]` profile), not hardcoded. Beyond controlling which ports the emulator binds, its host part sets the host publish IP for all published ports: a value like `GATEWAY_LISTEN = "0.0.0.0:4566,0.0.0.0:443"` exposes the emulator beyond loopback (e.g. on a remote host), whereas the default binds to `127.0.0.1` only. ::: ## OpenTelemetry tracing `lstk` can export traces of its own command execution over OTLP/HTTP. Tracing is **disabled by default**. Enable it with: ```bash LSTK_OTEL=1 lstk start ``` When enabled, every command is wrapped in a span (e.g. `lstk.start`) recording the exit code and any error. `lstk` does not hardcode an export target, so the OpenTelemetry Go SDK reads the standard `OTEL_EXPORTER_OTLP_*` environment variables automatically (default target: OTLP/HTTP at `localhost:4318`). You need an OTLP-compatible backend running to receive the traces. ## Logging `lstk` writes its own diagnostic logs to `lstk.log` in the same directory as the active config file. This is separate from the LocalStack container logs (which you view with `lstk logs`). - The log file is created automatically and appended to across runs. - When the file exceeds **1 MB**, it is cleared on the next run. - Use `lstk config path` to find the config directory; `lstk.log` sits alongside `config.toml`. ## Offline and enterprise environments There is no `--offline` flag. Instead, `lstk` degrades gracefully when common enterprise blockers (Docker Hub unreachable, a proxy/TLS interceptor, or an unreachable license server) prevent an internet request: - **Image pull**: if the image pull fails but the image is already present locally, `lstk` warns and uses the local image instead of failing. In interactive mode you can also press Esc to abort an in-progress pull and fall back to the local image. - **License pre-flight**: when the pinned image is already present locally, `lstk` skips its pre-flight license check so a fully offline start is not blocked; the emulator validates the license itself once it starts. When a check does run, a transport-level failure (offline, proxy, or certificate error) is treated as non-fatal and the emulator validates the license instead. A definitive server rejection (HTTP 400/401/403) is handled differently: `lstk` drops the cached license and, in an interactive terminal, offers to log in again and retries the start once with the refreshed credentials (a rejected token often just predates a license purchase or plan change); in non-interactive mode it fails with an error pointing at `lstk logout && lstk login` or a valid `LOCALSTACK_AUTH_TOKEN`. The pre-flight is also skipped — with a warning — when the license server does not recognize the image *tag format* (for example a `dev` nightly or a custom internal-mirror tag): that is not a verdict on the license, so `lstk` defers to the emulator's own startup check rather than blocking the start. - **Telemetry and update checks** are best-effort and fail silently when offline. Pair this behavior with a custom [`image`](#custom-container-image) that points at an internal-registry mirror or a locally loaded image to run `lstk` in an air-gapped environment. ## Shell completions `lstk` includes completion scripts for bash, zsh, fish, and powershell. If you installed via Homebrew, completions are set up automatically. Once completion is enabled, `lstk aws ` also completes AWS services, operations, and parameters using the AWS CLI's own completer. For manual setup: ```bash # Load in current session eval "$(lstk completion bash)" # Persist (Linux) lstk completion bash > /etc/bash_completion.d/lstk # Persist (macOS with Homebrew) lstk completion bash > $(brew --prefix)/etc/bash_completion.d/lstk ``` :::note Use `eval "$(lstk completion bash)"` rather than `source <(lstk completion bash)`. The `lstk` script works with or without the `bash-completion` package (it bundles a fallback for stock macOS bash 3.2), but `source <(...)` is a silent no-op on that shell. ::: ```bash # Load in current session source <(lstk completion zsh) # Persist (Linux) lstk completion zsh > "${fpath[1]}/_lstk" # Persist (macOS with Homebrew) lstk completion zsh > $(brew --prefix)/share/zsh/site-functions/_lstk ``` ```bash # Load in current session lstk completion fish | source # Persist lstk completion fish > ~/.config/fish/completions/lstk.fish ``` Restart your shell after persisting completions. ## FAQ ### Can I use `lstk` with Docker Compose? Yes, for the commands that talk to an already-running emulator. `lstk start`, `lstk stop`, and the other lifecycle commands manage their own Docker container and are not meant to drive a Compose-managed instance, so don't point `lstk stop` at one. But if you run LocalStack from a `docker-compose.yml`, you can still use `lstk`'s emulator-facing commands against it — `aws`, `az`, `terraform`/`cdk`/`sam`, `status`, `reset`, and `snapshot` — by passing `--endpoint-url ` (or setting `LSTK_ENDPOINT_URL`) to target the Compose deployment. See [Targeting an external emulator](#targeting-an-external-emulator) for the commands that accept an endpoint, and the [Docker Compose installation guide](/aws/getting-started/installation/#docker-compose) for the Compose setup itself. ### Which Docker image does `lstk` use? It depends on the emulator type configured in your `config.toml`. The AWS emulator uses `localstack/localstack-pro`, the Snowflake emulator uses `localstack/snowflake`, and the Azure emulator uses `localstack/localstack-azure`. All require a valid auth token (including the free Hobby tier). See [Emulator types](#emulator-types). ### How do I pass configuration options like `DEBUG` or `PERSISTENCE` to the container? Use environment profiles in your `config.toml`. Define the variables under an `[env.]` section and reference that name in the `env` list of your container config. See [Passing environment variables to the container](#passing-environment-variables-to-the-container) for details. ### How do I save and restore emulator state? Use [`lstk snapshot save`](#snapshot) to capture the running AWS emulator's state to a local file or a Cloud Pod, and [`lstk snapshot load`](#snapshot) (or the `lstk save` / `lstk load` aliases) to restore it. To drop in-memory state without writing a snapshot, use [`lstk reset`](#reset). ### How do I pin a specific LocalStack version? Set the `tag` field in your `config.toml` to a specific version tag: ```toml [[containers]] type = "aws" tag = "2026.4" port = "4566" ``` ## Troubleshooting ### Port 443 already in use By default, LocalStack publishes both port `4566` and port `443` (controlled by the `GATEWAY_LISTEN` variable). On some systems port 443 is already taken — Windows with Hyper-V, IIS, or VPN software, or an ingress proxy such as Rancher Desktop's Traefik. Because port 443 comes from the **default** `GATEWAY_LISTEN`, a busy 443 is **not fatal**: `lstk` drops that publication with a warning and starts anyway, and HTTPS is still served on the edge port `4566`. You only need to act if you want to silence the warning or bind 443 elsewhere. To skip port 443 entirely, override `GATEWAY_LISTEN` to bind only to `4566`: ```toml [[containers]] type = "azure" tag = "latest" port = "4566" env = ["nossl"] [env.nossl] GATEWAY_LISTEN = "0.0.0.0:4566" ``` :::note A port you list **explicitly** in a custom `GATEWAY_LISTEN` is treated as a hard requirement, so a busy one there fails the start rather than being dropped. Only the `443` from the default value is best-effort. ::: ### Docker is not running `lstk` requires a running Docker daemon. If Docker is not reachable, you will see an error like: ```text Error: runtime not healthy ``` **Fix:** Start your container runtime. `lstk` works with Docker Desktop, Rancher Desktop, Colima, OrbStack, Lima, and Podman — start the Docker daemon (`sudo systemctl start docker` on Linux) or the relevant VM (`rdctl start`, `colima start`, `podman machine start`, …). When the runtime is unavailable, `lstk`'s error tailors its suggested start command to whichever runtime it detects. You can also point `lstk` at a specific socket with `DOCKER_HOST`. See [Container runtime discovery](#container-runtime-discovery) for how the daemon is located. ### Authentication required in non-interactive mode When running without a TTY (e.g. in CI), `lstk` cannot open a browser for login. If no token is found in the keyring or environment, it fails: ```text authentication required: set LOCALSTACK_AUTH_TOKEN or run in interactive mode ``` **Fix:** Set the `LOCALSTACK_AUTH_TOKEN` environment variable before running `lstk`: ```bash export LOCALSTACK_AUTH_TOKEN= lstk --non-interactive start ``` You can find your auth token on the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens). ### License validation failed If your auth token is invalid, expired, or not linked to an active license, the LocalStack container exits with a license error: ```text The license activation failed for the following reason: No credentials were found in the environment. ``` **Fix:** - Verify your token is valid at the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens). - Make sure the token is set correctly, either via `lstk login` or the `LOCALSTACK_AUTH_TOKEN` environment variable. - A stale token or cached license no longer requires a manual `lstk logout`: when the platform definitively rejects it, `lstk` drops the cached license and, in an interactive terminal, prompts you to log in again and retries automatically. In non-interactive mode, run `lstk logout && lstk login` (or set a valid `LOCALSTACK_AUTH_TOKEN`) and re-run. ### Image pull failed If `lstk` cannot pull the Docker image, check your network connection and Docker configuration. On corporate networks, you may need to configure Docker's proxy settings, see [How do I configure LocalStack to use my corporate HTTP and HTTPS proxy?](/aws/getting-started/faq/#how-do-i-configure-localstack-to-use-my-corporate-http-and-https-proxy). ### Unknown environment profile If your container config references an `env` profile that doesn't exist, `lstk` returns: ```text environment "myprofile" referenced in container config not found ``` **Fix:** Make sure the profile name in the `env` list matches an `[env.]` section in your `config.toml`: ```toml [[containers]] type = "aws" env = ["myprofile"] # must match the section name below [env.myprofile] DEBUG = "1" ``` ### Getting help If the steps above don't resolve your issue, see [Get Help](/aws/help-support/get-help/) for the available support channels, including the support email and in-app chat. # Azure Portal (emulated) > A local Azure Portal served by the emulator itself, letting you browse, create, and manage your emulated Azure resources visually with no extra installation. ## Introduction The LocalStack Azure emulator can serve an emulated version of the Azure Portal directly from its own edge port. It gives you a visual way to work with your emulated resources: browse and filter everything in your local subscription, create resources through guided wizards, inspect blobs and Key Vault secrets, and invoke any ARM operation the emulator implements, all without installing anything beyond the emulator you already run. :::caution The emulated portal is a **preview feature** and is disabled by default. It is not affiliated with or connected to the real Azure Portal: everything it shows and everything it does stays inside your local emulator. ::: ## Enabling the portal Set `LS_AZURE_PORTAL=1` on the emulator container and open: ``` http://localhost:4566/_localstack/portal/ ``` With `lstk`, add the flag to an environment profile in your config: ```toml [[containers]] type = "azure" tag = "latest" port = "4566" env = ["portal"] [env.portal] LS_AZURE_PORTAL = "1" ``` Or with plain Docker: ```bash docker run -d -p 4566:4566 \ -e LOCALSTACK_AUTH_TOKEN=$LOCALSTACK_AUTH_TOKEN \ -e LS_AZURE_PORTAL=1 \ -v /var/run/docker.sock:/var/run/docker.sock \ localstack/localstack-azure ``` When the flag is not set, the portal is fully inactive: the URL returns 404 and no portal code is loaded. There is no separate port, container, or install step: the portal is served on the same edge port as the emulator's API, so it works wherever the emulator works. ## What you can do - **Browse resources.** All resource groups and resources in your emulated subscription, with filtering, sorting, and configurable columns. - **Create resources.** Guided create wizards for supported resource types. - **Work with data.** A storage browser for blob containers (create, upload, download, delete), and Key Vault secrets and certificates. - **Invoke any implemented operation.** The API operations drawer lists every ARM operation your emulator implements and lets you run it with your own parameters and request body. - **See real coverage.** Actions the emulator does not implement are greyed out with a reason, rather than failing unexpectedly. ## Always in sync with your emulator The portal computes its capability catalog at runtime from the emulator it is running inside. It never claims an operation your emulator version does not support, and it picks up newly implemented operations automatically. There is no separate portal version to keep in step with the emulator. ## Identity and sign-in The portal's sign-in screen is a **mock**: one click signs you in, and no credentials are collected. Inside the emulator, the portal acts as the default operator principal, the same identity used by the `az` CLI integration, SDKs, and Terraform. If you enable RBAC enforcement (`LS_AZURE_ENFORCE_RBAC=1`), portal requests are evaluated like any other operator traffic. ## Things to know :::note - **Local only.** The portal manages emulated resources in your local emulator. Nothing it does touches a real Azure subscription, and no data leaves your machine. - **Same trust model as the emulator API.** Anyone who can reach port 4566 can use the portal, just as they can use the emulator's REST API. Do not expose the edge port to untrusted networks. - **State follows the emulator.** Resources created in the portal live in the emulator's state; without persistence configured, they are gone after a restart. - **A subset of the real portal.** The emulated portal covers the resource types and operations the emulator implements; it is not a re-implementation of every Azure Portal blade. - **Telemetry.** Portal-originated requests are not counted in the emulator's usage analytics. ::: ## Troubleshooting - **404 at `/_localstack/portal/`.** The `LS_AZURE_PORTAL` flag is not set on the container. - **Page loads but shows errors.** Check `http://localhost:4566/_localstack/portal/api/meta/health`; it reports the emulator edition and the identity the portal is acting as. - **A resource action is greyed out.** The emulator does not implement that operation yet; the tooltip names the gap. # Installation > Installation guide to get started with LocalStack for Azure. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction LocalStack provides multiple installation paths depending on your development environment and requirements. We recommend a CLI-based installation for the most consistent local startup experience. Use [`lstk`](#lstk) to install, authenticate, and start the Azure emulator with minimal setup. LocalStack for Azure features require an [Auth Token](/azure/getting-started/auth-token/) to activate your running instance. `lstk` handles authentication through a browser-based login flow, while Docker and CI workflows can use `LOCALSTACK_AUTH_TOKEN`. Alternatively, you can set up the Azure emulator directly using the LocalStack for Azure Docker image, [`localstack/localstack-azure`](https://hub.docker.com/r/localstack/localstack-azure), with the [`docker` CLI](#docker-cli) or [Docker Compose](#docker-compose). :::note The Azure emulator image was previously published as `localstack/localstack-azure-alpha`. That name is no longer updated, and the Docker Hub repository is scheduled for removal at the end of August 2026. If your setup still references `localstack/localstack-azure-alpha`, switch to `localstack/localstack-azure` before then. ::: ## lstk `lstk` is a lightweight CLI for LocalStack that manages the authentication and container lifecycle for the AWS, Azure, and Snowflake emulators. **Requirement:** You must have a working [Docker installation](https://docs.docker.com/get-docker/) before proceeding. ### Install lstk ```bash brew install localstack/tap/lstk ``` ```bash npm install -g @localstack/lstk ``` Download the binary for your platform from the [GitHub Releases](https://github.com/localstack/lstk/releases) and add it to your `PATH`. ### Start lstk ```bash lstk start ``` The first execution initiates a browser-based login flow and, in an interactive terminal, prompts you to pick which emulator to run — choose `z` for Azure. Your choice is written to `config.toml` and used as the default on subsequent runs. Subsequent starts use credentials stored in your system keyring. ### Update lstk ```bash lstk update ``` For more details, see the [lstk documentation](/aws/developer-tools/running-localstack/lstk/). ### Already using lstk with a different default emulator? If your global `config.toml` already defaults to a different emulator (for example, AWS), target Azure for a specific project instead by creating a project-local `.lstk/config.toml`: ```toml # .lstk/config.toml [[containers]] type = "azure" port = "4566" ``` ## Container and orchestration tools Use these methods when you need explicit container configuration or want to run LocalStack alongside other services. For everyday local development, `lstk` is usually simpler. ### `docker` CLI To start the Azure emulator using the `docker` CLI, execute the following command: ``` $ docker run \ --rm -it \ -p 4566:4566 \ -v /var/run/docker.sock:/var/run/docker.sock \ -v ~/.localstack/volume:/var/lib/localstack \ -e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} \ localstack/localstack-azure ``` ### Docker Compose Create a `docker-compose.yml` file with the specified content: ```yaml version: "3.8" services: localstack: container_name: "localstack-main" image: localstack/localstack-azure ports: - "127.0.0.1:4566:4566" environment: - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} volumes: - "./volume:/var/lib/localstack" ``` Start the Azure emulator with the following command: ``` $ docker-compose up ``` ### Updating the Docker image To update the Azure Docker container, pull the latest image and restart the container. The following tags are available for the LocalStack for Azure Docker image: | Tag | Updated when | Recommended for | |---|---|---| | `latest` | Tagged releases only (e.g. `2026.05.0`) | Most users, stable, release-quality builds | | `dev` | Every merged commit on `main` | Users who need the latest unreleased changes | Starting with the end-of-March 2026 release, versioned Azure image tags follow [calendar versioning](https://calver.org/) in the `YYYY.MM.patch` format (for example, `2026.03.0`). Refer to the available [tags on Docker Hub](https://hub.docker.com/r/localstack/localstack-azure/tags) for the latest releases. # Auth Token > Configure your Auth Token to access and activate LocalStack for Azure. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction An Auth Token is required to activate the LocalStack for Azure emulator. It identifies and authenticates users outside the LocalStack Web Application, granting access to your workspace and to advanced features such as the Azure emulator image. Auth Tokens are issued at the workspace level in [app.localstack.cloud](https://app.localstack.cloud) and are not specific to any single LocalStack product. The same token works across every LocalStack product your account has access to, including LocalStack for AWS, Azure, and Snowflake. Auth Tokens come in two types: a **Developer Auth Token** and a **CI Auth Token**: - The **Developer Auth Token** uniquely identifies a user within a workspace. Every user has their own Auth Token. It cannot be deleted but can be rotated for security reasons if needed. - The **CI Auth Token** uniquely identifies a subscription rather than a specific user. It is designed for use in CI environments and other non-developer contexts, and is stored in the workspace where it can be managed by members with appropriate permissions. In both cases, the Auth Token grants access to whatever product(s) the associated user or subscription is entitled to. Both the **Developer Auth Token** and **CI Auth Token** can be managed on the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens). :::danger - It's crucial to keep your Auth Token confidential. Do not include it in source code management systems, such as Git repositories. - Be aware that if an Auth Token is committed to a public repository, it is at risk of exposure and could remain in the repository's history, even if attempts are made to rewrite it. - In case your Auth Token is accidentally published, immediately rotate it on the [Auth Token page](https://app.localstack.cloud/workspace/auth-tokens). ::: ## Managing your License LocalStack for Azure is currently in private preview. To access it, you need a LocalStack account with **any active subscription**. This can be a paid subscription, a trial, or the Hobby subscription (terms apply). :::note During the private preview, Azure access is enabled manually and cannot be self-served: - If you already have a LocalStack subscription (any type), [contact support](https://localstack.cloud/contact/) to have Azure access added. - If you don't have a LocalStack subscription yet, choose a [Hobby subscription](https://www.localstack.cloud/pricing) and then [contact support](https://localstack.cloud/contact/) to have Azure access added. Once Azure access is enabled on your subscription, you can use the emulator with your Developer Auth Token or CI Auth Token, just like any other LocalStack product. ::: After your subscription has Azure access, assign the license to a user by following these steps: - Visit the [Users & Licenses page](https://app.localstack.cloud/workspace/members). - Select a user in the **Workspace Members** section for license assignment. - Define the user's role via the **Member Role** dropdown. Single users automatically receive the **Admin** role. - Toggle **Advanced Permissions** to set specific permissions. Single users automatically receive full permissions. - Click **Save** to complete the assignment. Single users assign licenses to themselves. If you have joined a workspace, you need to be assigned a license by the workspace administrator. When switching workspaces or licenses, make sure you are assigned to the correct license. :::note If you do not assign a license, the Azure emulator will not start even if you have a valid Auth Token. ::: To view your own assigned license, visit the [My License page](https://app.localstack.cloud/workspace/my-license). For more details on inviting users, assigning licenses, or managing roles, see [Users and Licenses](/aws/organizations-admin/managing-users-licenses/). ## Configuring your Auth Token The Azure emulator reads the Auth Token from the `LOCALSTACK_AUTH_TOKEN` environment variable. You can configure the Auth Token in several ways, depending on your setup. The following sections describe the various methods of providing your Auth Token to the Azure container. :::danger - It's crucial to keep your Auth Token confidential. Do not include it in source code management systems, such as Git repositories. - Be aware that if an Auth Token is committed to a public repository, it is at risk of exposure and could remain in the repository's history, even if attempts are made to rewrite it. - In case your Auth Token is accidentally published, immediately rotate it on the [Auth Token page](https://app.localstack.cloud/workspace/auth-tokens). ::: ### lstk `lstk` handles authentication for you — just run `lstk` or `lstk start` and it takes care of the rest. See the [lstk documentation](/aws/developer-tools/running-localstack/lstk/) for details on how it resolves your Auth Token. On first run, `lstk` prompts you to pick an emulator and remembers your choice. If your `config.toml` already defaults to a different emulator, see [Already using lstk with a different default emulator?](/azure/getting-started/#already-using-lstk-with-a-different-default-emulator) to target Azure instead. For CI environments, see [CI Environments](#ci-environments) below. ### Docker To start the Azure emulator via Docker, provide the Auth Token using the `-e` flag: ```bash {5} docker run \ --rm -it \ -p 4566:4566 \ -v /var/run/docker.sock:/var/run/docker.sock \ -e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:- } \ localstack/localstack-azure ``` For more information about starting the Azure emulator with Docker, take a look at our [Azure installation guide](/azure/getting-started/#docker-cli). ### Docker Compose To start the Azure emulator using `docker compose`, include the `LOCALSTACK_AUTH_TOKEN` environment variable in your `docker-compose.yml` file: ```yaml environment: - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} ``` You can manually set the Auth Token, or use the `export` command to establish the Auth Token in your current shell session. This ensures the Auth Token is transmitted to the Azure container, enabling license activation. ### CI Environments CI environments require a CI Auth Token. Developer Auth Tokens cannot be used in CI. CI Auth Tokens are available on the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens) and are configured similarly to Developer Auth Tokens. To set the CI Auth Token, add the Auth Token value in the `LOCALSTACK_AUTH_TOKEN` environment variable of your CI provider, and reference it when starting the Azure emulator in your CI workflow. The same patterns used for [LocalStack in CI](/aws/ci-pipelines/) apply to Azure — swap the image for `localstack/localstack-azure`. ## Rotating the Auth Token Your personal Auth Token provides full access to your workspace and LocalStack license. Treat it as confidential and avoid sharing or storing it in source control management systems (SCMs) like Git. If you believe your Auth Token has been compromised or becomes known to someone else, reset it without delay. When you reset a token, the old one is immediately deactivated and can no longer access your license or workspace. Previous tokens cannot be restored. To rotate your Auth Token, go to the [Auth Token page](https://app.localstack.cloud/workspace/auth-tokens) and select the **Reset Auth Token** option. ## Verifying activation The simplest way to verify that the Azure emulator activated successfully is to query the health endpoint: ```bash curl http://localhost:4566/_localstack/info | jq ``` ```bash Invoke-WebRequest -Uri http://localhost:4566/_localstack/info | ConvertFrom-Json ``` A successful activation returns `"is_license_activated": true`. You can also check the container logs for a message indicating successful license activation: ```bash [...] Successfully activated license ``` Otherwise, check the [Troubleshooting](#troubleshooting) section below. ## Troubleshooting The Azure emulator requires a successful license activation to start. If activation fails, the container exits and prints an error message similar to: ```bash =============================================== License activation failed! Reason: The credentials defined in your environment are invalid. Please make sure to set the LOCALSTACK_AUTH_TOKEN variable to a valid auth token. You can find your Auth Token in the LocalStack web app https://app.localstack.cloud. Due to this error, LocalStack has quit. The Azure emulator can only be used with a valid license. ``` The most common causes are listed below. ### Missing credentials You need to provide an Auth Token to start the Azure emulator. You can find your Auth Token on the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens) in the LocalStack Web Application. If you are using `lstk`, run `lstk login` to authenticate through a browser-based flow. In CI, set the `LOCALSTACK_AUTH_TOKEN` environment variable instead; see [CI Environments](#ci-environments). ### Invalid license The issue may occur if there is no valid license linked to your account (for example, because it has expired), or if the license has not been assigned to your user. You can check your license status in the LocalStack Web Application on the [My License page](https://app.localstack.cloud/workspace/my-license). If your license does not grant access to the Azure emulator, [contact us](https://localstack.cloud/contact/) to upgrade. ### License server unreachable LocalStack initiates offline activation when the license server is unreachable, requiring re-activation every 24 hours. Log output may indicate issues with your machine resolving the LocalStack API domain, which can be verified using a tool like `dig`: ```bash dig api.localstack.cloud ``` If the result shows a status other than `status: NOERROR`, your machine is unable to resolve this domain. Certain corporate DNS servers may filter requests to specific domains. Kindly reach out to your network administrator to safelist the `localstack.cloud` domain. If you continue to have problems with license activation, or if the steps above do not help, do not hesitate to [contact us](https://localstack.cloud/contact/). # Quickstart > Get started with LocalStack for Azure in a few simple steps. ## Introduction This guide explains how to set up the Azure emulator and interact with it using the [`az` CLI](https://learn.microsoft.com/en-us/cli/azure/). In this guide, you will run some basic Azure CLI commands to manage resource groups in an local Azure development environment without connecting to the real cloud services. ## Prerequisites - [`lstk`](/azure/getting-started/#lstk) - [Azure CLI (`az`)](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) - A LocalStack account with a license that covers Azure usage — `lstk` handles authentication for you (see [Authentication](/azure/getting-started/auth-token/)) ## Instructions Start the Azure emulator: ``` $ lstk start ``` For more installation details, see the [installation instructions](/azure/getting-started/). ### Set up the `az` CLI integration To make sure the `az` tool sends requests to the Azure Emulator REST API, run the following command: ``` $ lstk az start-interception ``` ### Create a resource group To create a resource group, you can now run the same `az` command as you would normally: ``` $ az group create --name myResourceGroup --location westeurope ``` The following output would be displayed: ```bash { "id": "/subscriptions/some-generated-id/resourceGroups/myResourceGroup", "location": "westeurope", "managedBy": null, "name": "myResourceGroup", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Check & list resource groups To check the resource group details, run the following command: ``` $ az group show --name myResourceGroup ``` To list all the resource groups, run the following command: ``` $ az group list ``` ### Delete the resource group To delete the resource group, run the following command: ``` $ az group delete --name myResourceGroup --yes ``` ### Teardown When you're done, disable interception and stop the emulator: ``` $ lstk az stop-interception $ lstk stop ``` ### Alternative: prefixed commands Instead of interception, you can prefix each `az` command with `lstk az` individually, without changing your global `~/.azure` configuration. Run this once to prepare the integration: ``` $ lstk setup azure ``` Then prefix every command: ``` $ lstk az group create --name myResourceGroup --location westeurope $ lstk az group show --name myResourceGroup $ lstk az group list $ lstk az group delete --name myResourceGroup --yes ``` # Azure Integrations > Integrate LocalStack for Azure with your preferred tools and frameworks. Integrate LocalStack for Azure with your preferred tools and frameworks. # az > Get started with the az tool on LocalStack for Azure. ## Introduction The Azure CLI tool (`az`) is a tool that allows you to manually create and mange Azure resources. This guide will show you how to use it to interact with LocalStack. ## Getting started This guide is designed for users who are new to LocalStack for Azure emulator and assumes basic knowledge of how the Azure CLI works. We will demonstrate how to create, show and delete an Azure resource group. ### Prerequisites This guide uses [`lstk`](/aws/developer-tools/running-localstack/lstk/) to point the `az` CLI at the Azure emulator. To make sure the `az` tool sends requests to the Azure Emulator REST API, run the following command: ``` $ lstk az start-interception ``` ### Create and manage the Resource Group Run the following command to create a resource group in the Emulator: ``` $ az group create --name MyResourceGroup --location westeurope ``` To check the resource group details, run the following command: ``` $ az group show --name MyResourceGroup ``` To delete the resource group, run the following command: ``` $ az group delete --name MyResourceGroup --yes ``` ### Teardown When you're done using the Azure Emulator, you can run the following command: ``` $ lstk az stop-interception ``` The `az` CLI tool will now communicate with the Azure cloud on future invocations. ### Alternative: prefixed commands Instead of interception, you can prefix each `az` command with `lstk az` individually, without changing your global `~/.azure` configuration. Run this once to prepare the integration: ``` $ lstk setup azure ``` Then prefix every command: ``` $ lstk az group create --name MyResourceGroup --location westeurope $ lstk az group show --name MyResourceGroup $ lstk az group delete --name MyResourceGroup --yes ``` # azd > Get started with the azd tool on LocalStack for Azure. ## Introduction The Azure Developer tool (`azd`) is a tool that allows you to provision Azure resources. This guide will show you how to use it to interact with LocalStack. ## Getting started This guide is designed for users who are new to LocalStack for Azure emulator and assumes basic knowledge of how ARM/Bicep templates work in Azure. We will demonstrate how to create an Azure resource group using a Bicep template. ### Install the packages Run the following command to install the required packages: ``` $ pip install azlocal ``` You now have access to the following LocalStack tools: | CLI tool | LocalStack tool | Purpose | |-----------|-----------------|-------------------------------| | az | azlocal | Interact with Azure resources | | azd | azdlocal | Deploy ARM/Bicep templates | The LocalStack variants are wrappers around the existing tools, so you keep the full functionality of the original tool. It will just redirect all commands to the running LocalStack Emulator. ### Create a template You can now use `azdlocal` to provision infrastructure in LocalStack, just like you would use `azd` to do this in Azure. Create the following three files: A configuration file called `azure.yaml`: ```shell azure.yaml # yaml-language-server: $schema=https://raw.githubusercontent.com/Azure/azure-dev/main/schemas/v1.0/azure.yaml.json name: simple-template ``` A templated in `infra/main.bicep`: ```shell targetScope = 'subscription' @minLength(1) @maxLength(64) @description('Name of the the environment which is used to generate a short unique hash used in all resources.') param name string var tags = { 'azd-env-name': name } resource resourceGroup 'Microsoft.Resources/resourceGroups@2021-04-01' = { name: '${name}-rg' location: 'westeurope' tags: tags } ``` And a parameter-file in `infra/main.parameters.json`: ```shell { "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", "contentVersion": "1.0.0.0", "parameters": { "name": { "value": "${AZURE_ENV_NAME}" } } } ``` ### Deploy the template You can now deploy the template using `azdlocal`: ``` $ azdlocal up ``` The `azd` tool will ask a few questions about the environment name, subscription and location that you want your resources deployed in - just like you would see in Azure. When the deployment has finished, you should see the following: ```bash SUCCESS: Your up workflow to provision and deploy to Azure completed in 33 seconds. ``` You can now verify that the resource exist by using the `azlocal` tool: ```shell azlocal login azlocal group list ``` # Python > Get started with Azure libraries (SDK) for Python on LocalStack ## Introduction Azure SDK for Python is a set of libraries that allow you to interact with Azure services using Python. This guide will show you how to use the Azure SDK for Python to interact with LocalStack. ## Getting started This guide is designed for users who are new to LocalStack for Azure emulator and assumes basic knowledge of the Azure SDK for Python. We will demonstrate how to create an Azure resource group and update it with tags using Python. ### Install the packages Run the following command to install the required packages: ``` $ pip install azlocal $ pip install azure-mgmt-resource azure-identity ``` The `azlocal` package is provided by LocalStack to simplify the process of interacting with the Azure Emulator. The other two packages are packages provided by Azure to interact with Azure services in Python. :::note Elsewhere in these docs, `az` CLI examples use [`lstk az`](/aws/developer-tools/running-localstack/lstk/) instead of `azlocal`. `lstk` proxies the `az` CLI, but does not provide a Python SDK interception helper — for Python, `azlocal`'s `PythonLocalSdk` (used below) remains the supported approach. ::: ### Create a Python file You can now use the Azure SDK for Python to interact with LocalStack. Create a Python file named `provision_rg.py`. The code will perform the following actions: * Specifies mock credentials for the Azure SDK. * Creates a resource group. * Updates the resource group with tags. Paste the following code into the file: ```python from azlocal.python_local_sdk import PythonLocalSdk from azure.mgmt.resource import ResourceManagementClient from azure.identity import ClientSecretCredential def get_credentials(): return ClientSecretCredential(tenant_id="tenant-id", client_id="client_id", client_secret="client_secret") def create_or_update_resource_group(resource_client, group_name, location, tags=None): # Provision or update the resource group rg_result = resource_client.resource_groups.create_or_update( group_name, {"location": location, "tags": tags if tags else {}} ) # Logging the action taken action = "Updated" if tags else "Provisioned" print(f"{action} resource group {rg_result.name} in the {rg_result.location} region with tags {tags}") # Intercept all Azure requests python_local_sdk = PythonLocalSdk() python_local_sdk.start_interception() # Setup credentials and client credential = get_credentials() subscription_id = "sub-id" resource_client = ResourceManagementClient(credential, subscription_id) # Create or update resource groups create_or_update_resource_group(resource_client, "PythonAzureExample-rg", "centralus") create_or_update_resource_group(resource_client, "PythonAzureExample-rg", "centralus", {"environment": "test", "department": "tech"}) # Stop interception of Azure requests python_local_sdk.stop_interception() ``` ### Run the Python file You can now run the Python file. ``` python3 provision_rg.py ``` The following output will be displayed: ```bash Provisioned resource group PythonAzureExample-rg in the centralus region with tags None Updated resource group PythonAzureExample-rg in the centralus region with tags {'environment': 'test', 'department': 'tech'} ``` # Terraform > Get started with the `terraform` tool on LocalStack. ## Introduction Terraform is an Infrastructure-as-Code (IaC) tool that can deploy your full infrastructure with a few simple commands, in a reproducible manner. This guide will show you how to use it with LocalStack. ## Getting started This guide is designed for users who are new to LocalStack for Azure emulator and assumes basic knowledge of how Terraform works. We will demonstrate how to create an Azure resource group using Terraform. ### Terraform configuration Create the following two files: A Terraform template with the provider information called `provider.tf`: ```shell provider.tf terraform { required_providers { azurerm = { source = "hashicorp/azurerm" version = "= 5.1.0" } # only needed for the `random_uuid` resource random = { source = "hashicorp/random" version = "= 3.9.0" } } } provider "azurerm" { features {} metadata_host = "azure.localhost.localstack.cloud:4566" subscription_id = "00000000-0000-0000-0000-000000000000" } ``` Please note the `metadata_host` attribute! This is essential to ensure that the infrastructure is deployed to the LocalStack Emulator, instead of to the real cloud. A Terraform file that contains the resource group information called `main.tf`: ```shell main.tf resource "random_uuid" "uuid" {} resource "azurerm_resource_group" "rg" { name = "rg-hello-tf-${random_uuid.uuid.result}" location = "westeurope" } ``` ### Deploy Terraform You can now use Terraform like you would normally. First initialize the repository, and download the specified provider: ``` terraform init ``` Second, apply (create) the specified resources: ``` terraform apply ``` The `terraform` tool will now create the resource group. If you've followed our guide on how to configure the `az` CLI tool to point to LocalStack, you can verify that the resource exist by using the `az` tool: ```shell az group list ``` # Sample Apps > Sample apps for LocalStack for Azure. # Local Azure Services > Browse LocalStack's implemented Azure services and explore their capabilities. import SearchableAzureServices from '../../../../components/SearchableAzureServices.astro'; # Action Group > Get started with Azure Monitor Action Groups on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Monitor Action Groups define a collection of notification preferences and actions to execute when an alert fires. Action Groups are referenced by alert rules such as metric alerts and activity log alerts. They support a variety of notification channels including email, SMS, voice calls, webhooks, and Azure Functions, making them the central dispatch mechanism for Azure Monitor alerts. For more information, see [Create and manage action groups](https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/action-groups). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Monitor Action Groups. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Action Groups' integration with LocalStack. ## Getting started This guide walks you through creating an Action Group with an email receiver and managing it with the Azure CLI (`az monitor action-group`). You can attach the action group to alert rules in your application or infrastructure code the same way you would against Azure. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create --name rg-monit-demo --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-monit-demo", "location": "westeurope", "managedBy": null, "name": "rg-monit-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create an action group Create an action group with an email receiver as the notification endpoint: ```bash az monitor action-group create \ --name my-action-group \ --resource-group rg-monit-demo \ --short-name myag \ --action email myemail admin@example.com ``` ```bash title="Output" { "emailReceivers": [ { "emailAddress": "admin@example.com", "name": "myemail", "useCommonAlertSchema": false } ], "groupShortName": "myag", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-monit-demo/providers/microsoft.insights/actionGroups/my-action-group", "name": "my-action-group", "resourceGroup": "rg-monit-demo", "type": "Microsoft.Insights/actionGroups" ... } ``` ### Show and list action groups Retrieve the details of the action group, then list all action groups in the resource group: ```bash az monitor action-group show \ --name my-action-group \ --resource-group rg-monit-demo ``` ```bash title="Output" { "emailReceivers": [ { "emailAddress": "admin@example.com", "name": "myemail", "useCommonAlertSchema": false } ], "groupShortName": "myag", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-monit-demo/providers/microsoft.insights/actionGroups/my-action-group", "name": "my-action-group", "resourceGroup": "rg-monit-demo", "type": "Microsoft.Insights/actionGroups" ... } ``` Then list all action groups in the resource group to confirm it appears: ```bash az monitor action-group list \ --resource-group rg-monit-demo ``` ```bash title="Output" [ { "emailReceivers": [ { "emailAddress": "admin@example.com", "name": "myemail", "useCommonAlertSchema": false } ], "groupShortName": "myag", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-monit-demo/providers/microsoft.insights/actionGroups/my-action-group", "name": "my-action-group", "resourceGroup": "rg-monit-demo", "type": "Microsoft.Insights/actionGroups" } ] ``` ### Update an action group Update the action group to add a second email receiver: ```bash az monitor action-group update \ --name my-action-group \ --resource-group rg-monit-demo \ --add-action email newcontact ops@example.com ``` ```bash title="Output" { "emailReceivers": [ { "emailAddress": "admin@example.com", "name": "myemail", "useCommonAlertSchema": false }, { "emailAddress": "ops@example.com", "name": "newcontact", "useCommonAlertSchema": false } ], "groupShortName": "myag", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-monit-demo/providers/microsoft.insights/actionGroups/my-action-group", "name": "my-action-group", "resourceGroup": "rg-monit-demo", "type": "Microsoft.Insights/actionGroups" ... } ``` ### Delete and verify Delete the resource and confirm it no longer appears in the list: ```bash az monitor action-group delete \ --name my-action-group \ --resource-group rg-monit-demo ``` Then list all action groups to confirm the resource group is now empty: ```bash az monitor action-group list --resource-group rg-monit-demo ``` ```bash title="Output" [] ``` ## Features - **Action group lifecycle:** Create, read, list, update, and delete action groups. - **Email receivers:** Define one or more email receiver addresses. - **SMS receivers:** Define SMS receiver phone numbers. - **Webhook receivers:** Define webhook receiver URLs (stored, not invoked). - **Azure app push receivers:** Define Azure app push notification receivers. - **Voice receivers:** Define voice notification receivers. - **Logic app receivers:** Define Logic App action receivers. - **Azure function receivers:** Define Azure Function receivers. - **Event hub receivers:** Define Event Hub receivers. - **Short name support:** Each action group has a short name (max 12 characters) for SMS and push notifications. ## Limitations - **No notifications sent:** Emails, SMS messages, voice calls, and push notifications are not dispatched when an alert is triggered. - **No webhook invocation:** Webhook URLs are stored but not called. - **No Logic App or Azure Function invocation:** Logic App and Azure Function actions are stored but not executed. - **No test notifications:** The Azure CLI commands for sample notifications, [`az monitor action-group test-notifications`](https://learn.microsoft.com/cli/azure/monitor/action-group/test-notifications) and `az monitor action-group test-notifications create`, are not supported. ## Samples Explore end-to-end examples in the [LocalStack for Azure Samples](https://github.com/localstack/localstack-azure-samples) repository. ## API Coverage # Azure Kubernetes Service (AKS) > Get started with Azure Kubernetes Service on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Kubernetes Service (AKS) is Azure's managed Kubernetes offering. Azure operates the control plane while you manage node pools of worker machines that run your workloads. For more information, see [What is Azure Kubernetes Service?](https://learn.microsoft.com/en-us/azure/aks/what-is-aks). LocalStack for Azure creates real, working Kubernetes clusters on your machine. `az aks create` produces a cluster backed by [k3d](https://k3d.io/) that you can reach with `kubectl`, so manifests, Helm charts, and operators behave as they would against a cluster in the cloud. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of AKS's integration with LocalStack. ## Getting started This guide is designed for users new to AKS and assumes basic knowledge of the Azure CLI, `kubectl`, and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold the cluster: ```bash az group create \ --name rg-aks-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-aks-demo", "location": "westeurope", "managedBy": null, "name": "rg-aks-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a cluster Create a cluster with a single node in its system node pool: ```bash az aks create \ --resource-group rg-aks-demo \ --name aks-demo \ --node-count 1 \ --generate-ssh-keys ``` The command returns when the cluster is ready to use. Locally that takes a couple of minutes: the emulator provisions a k3d cluster, so what you get back is a live API server, not a mock. ```bash title="Output" { "currentKubernetesVersion": "1.34.4", "fqdn": "aks-demo-rg-aks-demo-000000-oq7mpqgx.hcp.westeurope.azmk8s.io", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/rg-aks-demo/providers/Microsoft.ContainerService/managedClusters/aks-demo", "kubernetesVersion": "1.34", "location": "westeurope", "name": "aks-demo", "nodeResourceGroup": "MC_rg-aks-demo_aks-demo_westeurope", "powerState": { "code": "Running" }, "provisioningState": "Succeeded", ... } ``` ### Show and list clusters Retrieve the details of a single cluster: ```bash az aks show \ --resource-group rg-aks-demo \ --name aks-demo ``` ```bash title="Output" { "currentKubernetesVersion": "1.34.4", "dnsPrefix": "aks-demo-rg-aks-demo-000000", "kubernetesVersion": "1.34", "location": "westeurope", "name": "aks-demo", "nodeResourceGroup": "MC_rg-aks-demo_aks-demo_westeurope", "provisioningState": "Succeeded", ... } ``` List the clusters in a resource group: ```bash az aks list \ --resource-group rg-aks-demo \ --output table ``` ```bash title="Output" Name Location ResourceGroup KubernetesVersion CurrentKubernetesVersion ProvisioningState Fqdn -------- ---------- --------------- ------------------- -------------------------- ------------------- ------------------------------------------------------------- aks-demo westeurope rg-aks-demo 1.34 1.34.4 Succeeded aks-demo-rg-aks-demo-000000-oq7mpqgx.hcp.westeurope.azmk8s.io ``` ### Update a cluster `az aks update` changes the properties of an existing cluster. The following example sets resource tags: ```bash az aks update \ --resource-group rg-aks-demo \ --name aks-demo \ --tags environment=local team=platform ``` ```bash title="Output" { "name": "aks-demo", "provisioningState": "Succeeded", "tags": { "environment": "local", "team": "platform" }, ... } ``` ### Connect with kubectl Merge the cluster credentials into your local kubeconfig: ```bash az aks get-credentials \ --resource-group rg-aks-demo \ --name aks-demo \ --overwrite-existing ``` ```bash title="Output" Merged "aks-demo" as current context in /home/user/.kube/config ``` Query the nodes: ```bash kubectl get nodes ``` ```bash title="Output" NAME STATUS ROLES AGE VERSION aks-nodepool1-5829393-vmss000000 Ready 99s v1.36.2+k3s1 k3d-aks-demo-7da4c24d-server-0 Ready control-plane 2m1s v1.36.2+k3s1 ``` :::note Unlike in the cloud, where the control plane is hidden, the local cluster also lists its k3d control-plane node. Agent nodes carry the same `aks--...-vmss` naming scheme as real AKS nodes, and the `VERSION` column reflects the underlying k3s runtime rather than the cluster's `kubernetesVersion`. ::: ### Manage node pools Add a user node pool with two nodes: ```bash az aks nodepool add \ --resource-group rg-aks-demo \ --cluster-name aks-demo \ --name workers \ --mode User \ --node-count 2 ``` ```bash title="Output" { "count": 2, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/rg-aks-demo/providers/Microsoft.ContainerService/managedClusters/aks-demo/agentPools/workers", "mode": "User", "name": "workers", "orchestratorVersion": "1.34", "osType": "Linux", "provisioningState": "Succeeded", ... } ``` List the node pools of the cluster: ```bash az aks nodepool list \ --resource-group rg-aks-demo \ --cluster-name aks-demo \ --output table ``` ```bash title="Output" Name OsType VmSize Count MaxPods ProvisioningState Mode --------- -------- -------- ------- --------- ------------------- ------ nodepool1 Linux 1 250 Succeeded System workers Linux 2 250 Succeeded User ``` Inspect a single node pool: ```bash az aks nodepool show \ --resource-group rg-aks-demo \ --cluster-name aks-demo \ --name workers ``` ```bash title="Output" { "count": 2, "mode": "User", "name": "workers", "orchestratorVersion": "1.34", "osType": "Linux", "powerState": { "code": "Running" }, "provisioningState": "Succeeded", ... } ``` Delete the node pool when you no longer need it: ```bash az aks nodepool delete \ --resource-group rg-aks-demo \ --cluster-name aks-demo \ --name workers ``` ### Stop, start, and delete Stop the cluster to free local resources while preserving its state: ```bash az aks stop \ --resource-group rg-aks-demo \ --name aks-demo ``` Verify that the cluster has stopped by confirming that `powerState` reports `Stopped`: ```bash az aks show \ --resource-group rg-aks-demo \ --name aks-demo \ --query powerState.code \ --output tsv ``` Start the cluster again. It restarts with the previous control plane state and number of agent nodes: ```bash az aks start \ --resource-group rg-aks-demo \ --name aks-demo ``` Delete the cluster once you are done. A deleted cluster cannot be recovered: ```bash az aks delete \ --resource-group rg-aks-demo \ --name aks-demo \ --yes ``` ### Teardown Remove the resource group and any resources it still contains: ```bash az group delete \ --name rg-aks-demo \ --yes ``` Disable Azure CLI interception to point the `az` CLI back to the official Azure management REST API: ```bash lstk az stop-interception ``` ## Features The local control plane implements the following capabilities: - **Networking**: Azure CNI overlay with the Cilium data plane, Cilium and Calico network policies, Hubble observability, and the managed Gateway API add-on with NGINX Gateway Fabric as its implementation. - **Storage**: The Secrets Store CSI driver for Azure Key Vault and the Azure Files CSI driver. The Azure Disk CSI driver is in progress. - **Scaling**: The cluster autoscaler, node auto-provisioning based on the AKS Karpenter provider, the Kubernetes Event-driven Autoscaling (KEDA) add-on, and the Vertical Pod Autoscaler. - **Identity**: Microsoft Entra Workload ID with a working OIDC issuer, so pods can exchange service account tokens for Azure credentials without secrets. - **Operations**: Multiple node pools with tags, labels, and taints; the Azure cloud controller manager reconciling `LoadBalancer` services; and cluster stop and start. - **Tooling**: The same clusters can be provisioned with the Azure CLI, Terraform, or Bicep. ## Cluster-creation scripts The [aks-samples](https://github.com/localstack-samples/aks-samples) repository provides two interchangeable scripts that provision a production-shaped cluster, complete with a virtual network, a container registry, a Log Analytics workspace, and system and user node pools. Both scripts run unchanged against real Azure and the emulator; they differ only in the cluster identity: | Script | Cluster identity | When to use | | ------ | ---------------- | ----------- | | [01-system-assigned-managed-identity.sh](https://github.com/localstack-samples/aks-samples/blob/main/scripts/01-system-assigned-managed-identity.sh) | System-assigned managed identity | Simplest option: Azure creates and manages the identity lifecycle together with the cluster. | | [01-user-assigned-managed-identity.sh](https://github.com/localstack-samples/aks-samples/blob/main/scripts/01-user-assigned-managed-identity.sh) | User-assigned managed identity | Use when you need a stable, pre-created identity that can be reused across resources and granted role assignments ahead of time. | Both scripts are idempotent and safe to re-run. The same folder also contains optional add-on installers for [Prometheus](https://github.com/localstack-samples/aks-samples/blob/main/scripts/02-install-prometheus.sh), the [NGINX ingress controller](https://github.com/localstack-samples/aks-samples/blob/main/scripts/03-install-nginx-ingress-controller.sh), the [Gateway API CRDs](https://github.com/localstack-samples/aks-samples/blob/main/scripts/04-install-gateway-api.sh), [NGINX Gateway Fabric](https://github.com/localstack-samples/aks-samples/blob/main/scripts/05-install-nginx-gateway-fabric.sh), and [cert-manager](https://github.com/localstack-samples/aks-samples/blob/main/scripts/06-install-cert-manager.sh). ## Samples Every sample deploys the same Vacation Planner web application, a small Python [Flask](https://flask.palletsprojects.com/) single-page app. Only the data service, its provisioning, and the way the app authenticates to it change from one sample to the next. | Sample | Description | | ------ | ----------- | | [web-app-sql-database](https://github.com/localstack-samples/aks-samples/tree/main/samples/web-app-sql-database) | Stores activities in an Azure SQL Database, connecting with a SQL login over TDS. | | [web-app-mysql-flexible-server](https://github.com/localstack-samples/aks-samples/tree/main/samples/web-app-mysql-flexible-server) | Stores activities in an Azure Database for MySQL flexible server. | | [web-app-postgresql-flexible-server](https://github.com/localstack-samples/aks-samples/tree/main/samples/web-app-postgresql-flexible-server) | Stores activities in an Azure Database for PostgreSQL flexible server. | | [web-app-in-cluster-postgresql](https://github.com/localstack-samples/aks-samples/tree/main/samples/web-app-in-cluster-postgresql) | Stores activities in an in-cluster PostgreSQL database deployed as a Kubernetes StatefulSet, with a primary and two streaming replicas. | | [web-app-cosmosdb-mongodb-api](https://github.com/localstack-samples/aks-samples/tree/main/samples/web-app-cosmosdb-mongodb-api) | Stores activities in a collection of an Azure Cosmos DB for MongoDB account. | | [web-app-cosmosdb-nosql-api](https://github.com/localstack-samples/aks-samples/tree/main/samples/web-app-cosmosdb-nosql-api) | Stores activities in a container of an Azure Cosmos DB for NoSQL account. | | [web-app-blob-storage](https://github.com/localstack-samples/aks-samples/tree/main/samples/web-app-blob-storage) | Stores activities in an Azure Blob Storage container, using a connection string. | | [web-app-file-storage](https://github.com/localstack-samples/aks-samples/tree/main/samples/web-app-file-storage) | Stores activities as text files on an Azure Files share mounted by the Azure Files CSI driver, over either SMB or NFS. | | [web-app-managed-identity](https://github.com/localstack-samples/aks-samples/tree/main/samples/web-app-managed-identity) | Stores activities in Azure Blob Storage, authenticating with Microsoft Entra Workload ID instead of a secret, and optionally exposes the app through the Gateway API with a managed TLS certificate. | ## Tutorials The same repository includes standalone tutorials that exercise individual AKS capabilities. Unlike the samples, they do not deploy the web application: | Tutorial | Description | | -------- | ----------- | | [policies](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/policies) | Kubernetes network policies that enforce zero-trust traffic control with Calico and Cilium: cluster-wide default-deny, DNS-aware egress, and L3/L4/L7 ingress. | | [ccm](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/ccm) | Exercises the Azure cloud controller manager: public and internal `LoadBalancer` services, source ranges, the nodeIP backend-pool variant, and an NGINX ingress controller. | | [gateway-api](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/gateway-api) | Enables the managed Gateway API CRDs, installs NGINX Gateway Fabric, and routes traffic to a backend through a `Gateway` and an `HTTPRoute`. | | [keda](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/keda) | Event-driven autoscaling with the KEDA add-on: a producer creates a backlog on an Azure event source, and a `ScaledObject` scales a consumer from zero to four replicas and back. | | [keda/service-bus](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/keda/service-bus) | Scales a consumer on an Azure Service Bus queue with the `azure-servicebus` scaler, authenticating with Microsoft Entra Workload ID. Start here if you are new to KEDA. | | [keda/queue-storage](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/keda/queue-storage) | Scales a consumer on an Azure Storage queue with the `azure-queue` scaler. Workload identity is used end to end, so no data-plane secret exists anywhere. | | [keda/event-hubs](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/keda/event-hubs) | Scales a consumer on an Azure Event Hubs hub with the `azure-eventhub` scaler, whose backlog is the distance between the last enqueued event and the consumer group's blob checkpoints. | | [key-vault-csi-driver](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/key-vault-csi-driver) | Mounts secrets from Azure Key Vault into a pod with the Secrets Store CSI driver, in both the workload identity and the user-assigned managed identity access modes. | | [terraform/tags-labels-taints](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/terraform/tags-labels-taints) | Deploys a modular, feature-rich AKS stack with Terraform, steering workloads across agent pools with Azure resource tags, node labels, and taints, then validates them through both the ARM and Kubernetes APIs. | | [bicep/tags-labels-taints](https://github.com/localstack-samples/aks-samples/tree/main/tutorials/bicep/tags-labels-taints) | The same modular AKS stack built with Bicep, with parameters and outputs that mirror the Terraform tutorial one-to-one. | ## API Coverage # API Management > Get started with Azure API Management on LocalStack import AzureFeatureCoverage from '../../../../components/feature-coverage/AzureFeatureCoverage'; ## Introduction Azure API Management (APIM) is a managed service for publishing, securing, and analyzing APIs at scale. It acts as a gateway between clients and backend services, providing features such as rate limiting, policy enforcement, authentication, and developer portal integration. APIM is commonly used to expose internal services as managed APIs, implement API versioning, and monitor API usage across organizations. For more information, see [Azure API Management overview](https://learn.microsoft.com/en-us/azure/api-management/api-management-key-concepts). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure API Management. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of API Management's integration with LocalStack. ## Getting started This guide walks you through creating an API Management service, adding an API, and defining and updating an operation. It is designed for users new to API Management and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group for your API Management resources: ```bash az group create \ --name rg-apim-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-apim-demo", "location": "westeurope", "name": "rg-apim-demo", "properties": { "provisioningState": "Succeeded" }, ... } ``` ### Create an API Management service instance Create an API Management service in the resource group: ```bash az apim create \ --name apimdoc86 \ --resource-group rg-apim-demo \ --location westeurope \ --sku-name Consumption \ --publisher-name "LocalStack" \ --publisher-email "dev@localstack.cloud" ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-apim-demo/providers/Microsoft.ApiManagement/service/apimdoc86", "name": "apimdoc86", "location": "West Europe", "provisioningState": "Succeeded", "publisherName": "LocalStack", "publisherEmail": "dev@localstack.cloud", "gatewayUrl": "https://apimdoc86.azure-api.net", "sku": { "capacity": 0, "name": "Consumption" }, ... } ``` Get and list API Management services: ```bash az apim show \ --name apimdoc86 \ --resource-group rg-apim-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-apim-demo/providers/Microsoft.ApiManagement/service/apimdoc86", "location": "West Europe", "name": "apimdoc86", "provisioningState": "Succeeded", "publisherEmail": "dev@localstack.cloud", "publisherName": "LocalStack", "resourceGroup": "rg-apim-demo", "sku": { "capacity": 0, "name": "Consumption" }, "type": "Microsoft.ApiManagement/service" ... } ``` ```bash az apim list \ --resource-group rg-apim-demo ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-apim-demo/providers/Microsoft.ApiManagement/service/apimdoc86", "name": "apimdoc86", "provisioningState": "Succeeded", "resourceGroup": "rg-apim-demo", "sku": { "capacity": 0, "name": "Consumption" }, "type": "Microsoft.ApiManagement/service" } ] ``` ### Check service name availability Check whether the service name is globally available before creating: ```bash az apim check-name --name apimdoc86 ``` ```bash title="Output" { "message": "", "nameAvailable": true, "reason": "Valid" } ``` ### Create and inspect an API Create an API in API Management: ```bash az apim api create \ --resource-group rg-apim-demo \ --service-name apimdoc86 \ --api-id orders-api \ --path orders \ --display-name "Orders API" \ --protocols https ``` ```bash title="Output" { "displayName": "Orders API", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-apim-demo/providers/Microsoft.ApiManagement/service/apimdoc86/apis/orders-api", "name": "orders-api", "path": "orders", "protocols": [ "https" ], "subscriptionRequired": true, ... } ``` Get the API: ```bash az apim api show \ --resource-group rg-apim-demo \ --service-name apimdoc86 \ --api-id orders-api ``` ```bash title="Output" { "displayName": "Orders API", "name": "orders-api", "path": "orders", "protocols": [ "https" ], ... } ``` ### Create and update an API operation Create an operation on the API: ```bash az apim api operation create \ --resource-group rg-apim-demo \ --service-name apimdoc86 \ --api-id orders-api \ --operation-id get-orders \ --display-name "Get orders" \ --method GET \ --url-template "/orders" ``` ```bash title="Output" { "displayName": "Get orders", "method": "GET", "name": "get-orders", "type": "Microsoft.ApiManagement/service/apis/operations", "urlTemplate": "/orders", ... } ``` Update the operation and verify the change: ```bash az apim api operation update \ --resource-group rg-apim-demo \ --service-name apimdoc86 \ --api-id orders-api \ --operation-id get-orders \ --display-name "Get all orders" az apim api operation show \ --resource-group rg-apim-demo \ --service-name apimdoc86 \ --api-id orders-api \ --operation-id get-orders ``` ```bash title="Output" { "displayName": "Get all orders", "method": "GET", "name": "get-orders", "urlTemplate": "/orders", ... } ``` ## Features - **Full CRUD lifecycle:** Create, read, update, and delete APIM service instances. - **API management:** Create, read, and delete API definitions within a service. - **API operations:** Register and retrieve API operation definitions. - **Backend management:** Define and manage backend service configurations. - **API gateway management:** Create and manage self-hosted API gateways. - **API policies:** Attach XML policy documents to APIs (stored but not evaluated). - **Name availability check:** Validate service name uniqueness via the `checkNameAvailability` action. - **Service listing:** List all APIM services in a subscription or resource group. ## Limitations - **No gateway proxy:** Incoming API calls are not proxied through LocalStack. The APIM gateway does not process requests, apply policies, or forward traffic to backends. - **No policy evaluation:** Inbound, outbound, and error policies are stored in the ARM model but are not executed. - **No developer portal:** The APIM developer portal and its OAuth/subscription flows are not emulated. - **No subscription keys:** API subscriptions and key-based authentication are not enforced. - **No rate limiting or quotas:** Throttling, quota, and cache policies have no effect. - **Consumption plan only for creation:** SKU differences between Developer, Basic, Standard, Premium, and Consumption tiers are not emulated. ## Samples Explore end-to-end examples in the [LocalStack for Azure Samples](https://github.com/localstack/localstack-azure-samples) repository. ## API Coverage # Application Insights > Get started with Azure Application Insights on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Application Insights is an application performance management (APM) service for monitoring live applications. It automatically collects request rates, response times, failure rates, and dependency traces, surfacing them in a unified monitoring experience. Application Insights is commonly used to diagnose production issues, track custom business metrics, and set up availability alerts for distributed applications. For more information, see [What is Application Insights?](https://learn.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Application Insights. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Application Insights' integration with LocalStack. ## Getting started This guide walks you through creating an Application Insights component and retrieving its instrumentation key. The Azure CLI commands below use the [`application-insights` extension](https://learn.microsoft.com/en-us/cli/azure/azure-cli-extensions-overview); it is installed automatically the first time you run an `az monitor app-insights` command (Azure CLI 2.71.0 or later). Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create --name rg-insights-demo --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-insights-demo", "location": "westeurope", "name": "rg-insights-demo", "properties": { "provisioningState": "Succeeded" }, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create an Application Insights component Create an Application Insights component in the resource group where you intend to emit telemetry ([`--workspace`](https://learn.microsoft.com/en-us/cli/azure/monitor/app-insights/component?view=azure-cli-latest#optional-parameters), for a workspace-based component linked to Log Analytics, is optional and omitted here). ```bash az monitor app-insights component create \ --app my-app-insights \ --resource-group rg-insights-demo \ --location westeurope \ --kind web \ --application-type web ``` ```bash title="Output" { "appId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "connectionString": "InstrumentationKey=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx;IngestionEndpoint=https://westeurope.in.applicationinsights.azure.com/", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-insights-demo/providers/Microsoft.Insights/components/my-app-insights", "instrumentationKey": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "kind": "web", "location": "westeurope", "name": "my-app-insights", "provisioningState": "Succeeded", "resourceGroup": "rg-insights-demo", "type": "Microsoft.Insights/components" ... } ``` ### Show and list components There is no `az monitor app-insights component list` command in the Azure CLI. Retrieve one component with `component show`, and list all Application Insights components in the resource group with [`az resource list`](https://learn.microsoft.com/en-us/cli/azure/resource?view=azure-cli-latest#az-resource-list) and the `Microsoft.Insights/components` resource type: ```bash az monitor app-insights component show \ --app my-app-insights \ --resource-group rg-insights-demo ``` ```bash title="Output" { "appId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-insights-demo/providers/Microsoft.Insights/components/my-app-insights", "instrumentationKey": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "kind": "web", "location": "westeurope", "name": "my-app-insights", "provisioningState": "Succeeded", "resourceGroup": "rg-insights-demo", "type": "Microsoft.Insights/components" ... } ``` ```bash az resource list \ --resource-group rg-insights-demo \ --resource-type Microsoft.Insights/components ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-insights-demo/providers/Microsoft.Insights/components/my-app-insights", "name": "my-app-insights", "resourceGroup": "rg-insights-demo", "location": "westeurope", "type": "Microsoft.Insights/components" } ] ``` ### Retrieve billing features Retrieve the current billing features and daily data volume cap for the component: ```bash az monitor app-insights component billing show \ --app my-app-insights \ --resource-group rg-insights-demo ``` ```bash title="Output" { "currentBillingFeatures": ["Basic"], "dataVolumeCap": { "cap": 100.0, "maxHistoryCap": 1000.0, "resetTime": 0, "warningThreshold": 90 } } ``` ### Update billing features Update the daily data volume cap to limit ingestion costs: ```bash az monitor app-insights component billing update \ --app my-app-insights \ --resource-group rg-insights-demo \ --cap 100 ``` ```bash title="Output" { "currentBillingFeatures": ["Basic"], "dataVolumeCap": { "cap": 100.0, "maxHistoryCap": 1000.0, "resetTime": 0, "warningThreshold": 90 } } ``` ### Delete and verify Delete the resource and confirm it no longer appears when listing `Microsoft.Insights/components` in the resource group: ```bash az monitor app-insights component delete \ --app my-app-insights \ --resource-group rg-insights-demo ``` ```bash az resource list \ --resource-group rg-insights-demo \ --resource-type Microsoft.Insights/components ``` ```bash title="Output" [] ``` ## Features - **Component lifecycle:** Create, show, and delete Application Insights components; discover instances in a resource group with `az resource list` and `--resource-type Microsoft.Insights/components` (see [Azure CLI — `az monitor app-insights component`](https://learn.microsoft.com/en-us/cli/azure/monitor/app-insights/component?view=azure-cli-latest)). - **Instrumentation key generation:** Each component is assigned an instrumentation key returned on creation. - **App ID assignment:** Each component is assigned a unique application ID (for example [`properties.AppId`](https://learn.microsoft.com/en-us/rest/api/application-insights/components/create-or-update) / `appId` in CLI output). - **Billing feature configuration:** Get and update billing features such as daily data volume caps (`currentBillingFeatures`, `dataVolumeCap`); see [Azure CLI — `component billing`](https://learn.microsoft.com/en-us/cli/azure/monitor/app-insights/component/billing?view=azure-cli-latest). - **Application type (`--application-type` / `Application_Type`):** Accepted values documented for the CLI are **`web`** and **`other`** (default `web`). - **Kind (`--kind`):** Typical values **`web`**, **`ios`**, **`other`**, **`store`**, **`java`**, **`phone`** (free-form string for UI customization; see [Azure CLI — `component create`](https://learn.microsoft.com/en-us/cli/azure/monitor/app-insights/component?view=azure-cli-latest#az-monitor-app-insights-component-create)). ## Limitations - **No telemetry ingestion:** The Application Insights SDK endpoint (`/v2/track`) is not emulated. Telemetry sent from instrumented applications is not stored or queryable. - **No Live Metrics stream:** The Live Metrics (QuickPulse) endpoint is not emulated. - **No Logs (KQL) queries:** Running queries via [`az monitor app-insights query`](https://learn.microsoft.com/en-us/cli/azure/monitor/app-insights?view=azure-cli-latest#az-monitor-app-insights-query) (KQL over stored application data) is not supported. - **No transaction search:** Individual telemetry records are not stored or searchable. - **No availability tests via this component:** Availability tests are separate **[`Microsoft.Insights/webtests`](https://learn.microsoft.com/en-us/azure/azure-monitor/app/monitor-web-app-availability)** resources (CLI group [`web-test`](https://learn.microsoft.com/en-us/cli/azure/monitor/app-insights/web-test?view=azure-cli-latest)), not capabilities of an Application Insights component alone. - **No continuous export:** Continuous export to Storage is not supported. ## Samples The following sample demonstrates how to use Azure Application Insights with LocalStack for Azure: - [Function App and Service Bus](https://github.com/localstack/localstack-azure-samples/samples/function-app-service-bus/dotnet/README.md) ## API Coverage # Autoscale Setting > Get started with Azure Monitor Autoscale Settings on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Monitor Autoscale Settings manage automatic scaling of compute resources based on schedule or metric-driven rules. Autoscale settings can target Virtual Machine Scale Sets, App Service plans, and other scalable resources. They are commonly used to adjust capacity in response to traffic patterns, reducing costs during off-peak periods while maintaining performance under load. For more information, see [Overview of autoscale in Azure](https://learn.microsoft.com/en-us/azure/azure-monitor/autoscale/autoscale-overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Monitor Autoscale Settings. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Autoscale Settings' integration with LocalStack. ## Getting started This guide walks you through creating an autoscale setting targeting an App Service plan. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create --name rg-autoscale-demo --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-autoscale-demo", "location": "westeurope", "name": "rg-autoscale-demo", "properties": { "provisioningState": "Succeeded" }, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create an App Service plan Create an App Service plan to use as the autoscale target: ```bash az appservice plan create \ --name my-plan \ --resource-group rg-autoscale-demo \ --sku S1 \ --is-linux ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-autoscale-demo/providers/Microsoft.Web/serverfarms/my-plan", "kind": "linux", "location": "westeurope", "name": "my-plan", "resourceGroup": "rg-autoscale-demo", "sku": { "capacity": 1, "name": "S1", "tier": "Standard" }, "type": "Microsoft.Web/serverfarms" ... } ``` ### Create an autoscale setting Create an autoscale setting linked to the App Service plan: ```bash az monitor autoscale create \ --name my-autoscale \ --resource-group rg-autoscale-demo \ --resource my-plan \ --resource-type Microsoft.Web/serverfarms \ --min-count 1 \ --max-count 5 \ --count 2 ``` ```bash title="Output" { "enabled": true, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-autoscale-demo/providers/microsoft.insights/autoscalesettings/my-autoscale", "name": "my-autoscale", "profiles": [ { "capacity": { "default": "2", "maximum": "5", "minimum": "1" }, "name": "default", "rules": [] } ], "resourceGroup": "rg-autoscale-demo", "targetResourceUri": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-autoscale-demo/providers/Microsoft.Web/serverfarms/my-plan", "type": "Microsoft.Insights/autoscaleSettings" ... } ``` ### Add a scale-out rule Add a scale-out rule that increases the instance count when average CPU usage rises above 70%. For App Service plans, the metric name is **CpuPercentage** ([common autoscale metrics](https://learn.microsoft.com/en-us/azure/azure-monitor/autoscale/autoscale-common-metrics#web-apps-metrics)); VM and Virtual Machine Scale Sets often use **Percentage CPU** in rule conditions instead. ```bash az monitor autoscale rule create \ --autoscale-name my-autoscale \ --resource-group rg-autoscale-demo \ --scale out 1 \ --condition "CpuPercentage > 70 avg 5m" ``` ```bash title="Output" { "metricTrigger": { "metricName": "CpuPercentage", "operator": "GreaterThan", "statistic": "Average", "threshold": 70.0, "timeAggregation": "Average", "timeGrain": "PT1M", "timeWindow": "PT5M" }, "scaleAction": { "cooldown": "PT5M", "direction": "Increase", "type": "ChangeCount", "value": "1" } } ``` ### Show and list autoscale settings Retrieve the details of the autoscale setting and list all settings in the resource group: ```bash az monitor autoscale show \ --name my-autoscale \ --resource-group rg-autoscale-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-autoscale-demo/providers/microsoft.insights/autoscalesettings/my-autoscale", "name": "my-autoscale", "profiles": [ { "capacity": { "default": "2", "maximum": "5", "minimum": "1" }, "name": "default", "rules": [ { "metricTrigger": { "metricName": "CpuPercentage", "operator": "GreaterThan", "threshold": 70.0, ... }, "scaleAction": { "direction": "Increase", "type": "ChangeCount", "value": "1" } } ] } ], "resourceGroup": "rg-autoscale-demo", "targetResourceUri": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-autoscale-demo/providers/Microsoft.Web/serverfarms/my-plan", "type": "Microsoft.Insights/autoscaleSettings" ... } ``` Then list all autoscale settings in the resource group: ```bash az monitor autoscale list \ --resource-group rg-autoscale-demo ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-autoscale-demo/providers/microsoft.insights/autoscalesettings/my-autoscale", "name": "my-autoscale", "resourceGroup": "rg-autoscale-demo", "type": "Microsoft.Insights/autoscaleSettings" } ] ``` ### Delete and verify Delete the resource and confirm it no longer appears in the list: ```bash az monitor autoscale delete \ --name my-autoscale \ --resource-group rg-autoscale-demo ``` Then list all autoscale settings to confirm the resource group is now empty: ```bash az monitor autoscale list --resource-group rg-autoscale-demo ``` ```bash title="Output" [] ``` ## Features - **Autoscale setting lifecycle:** Create, read, list, update, and delete autoscale settings. - **Profile configuration:** Define default, fixed date, and recurring schedule profiles. - **Scale rules:** Add metric-based scale-out and scale-in rules per profile. - **Capacity bounds:** Define minimum, maximum, and default instance counts. - **Notification configuration:** Attach email and webhook notifications on scale events (stored, not dispatched). - **Resource targeting:** Point autoscale settings at the resource ID of a [supported scalable resource](https://learn.microsoft.com/en-us/azure/azure-monitor/autoscale/autoscale-overview#supported-services-for-autoscale). ## Limitations - **No scaling actions:** Instance counts are not changed by LocalStack. Scale-out and scale-in rules are stored but never executed. - **No metric evaluation:** CPU, memory, and custom metric conditions in scale rules are not evaluated. - **No scale history:** Autoscale run history (for example the portal run history view, diagnostic logs, or PowerShell [`Get-AzAutoscaleHistory`](https://learn.microsoft.com/en-us/powershell/module/az.monitor/get-azautoscalehistory)) is not available. - **No notifications dispatched:** Email and webhook scale notifications are stored but not sent. ## Samples Explore end-to-end examples in the [LocalStack for Azure Samples](https://github.com/localstack/localstack-azure-samples) repository. ## API Coverage # Bastion Host > Get started with Azure Bastion Host on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Bastion Host provides secure and seamless RDP and SSH connectivity to virtual machines directly through the Azure portal over TLS. Bastion is deployed in a virtual network and eliminates the need for a public IP address on the VM, protecting against port scanning and other external threats. It is the recommended approach for securely accessing Azure VMs without exposing them to the public internet. For more information, see [What is Azure Bastion?](https://learn.microsoft.com/en-us/azure/bastion/bastion-overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Bastion Host. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Bastion Host's integration with LocalStack. ## Getting started This guide is designed for users new to Bastion Host and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group and virtual network For dedicated Bastion SKUs on Azure (Basic, Standard, or Premium), you need a subnet named exactly `AzureBastionSubnet` (with a prefix of /26 or larger) and a Standard SKU public IP address with static allocation. Create the prerequisites first: ```bash az group create \ --name rg-bastion-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-bastion-demo", "location": "westeurope", "managedBy": null, "name": "rg-bastion-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` Create a virtual network with a /16 address space to host the bastion subnet: ```bash az network vnet create \ --name vnet-bastion-demo \ --resource-group rg-bastion-demo \ --location westeurope \ --address-prefixes 10.0.0.0/16 ``` ```bash title="Output" { "newVNet": { "addressSpace": { "addressPrefixes": [ "10.0.0.0/16" ] }, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-bastion-demo/providers/Microsoft.Network/virtualNetworks/vnet-bastion-demo", "location": "westeurope", "name": "vnet-bastion-demo", "provisioningState": "Succeeded", "resourceGroup": "rg-bastion-demo", "subnets": [], "type": "Microsoft.Network/virtualNetworks", ... } } ``` Create the `AzureBastionSubnet` subnet, which is required by Azure Bastion for dedicated deployments (Basic, Standard, or Premium SKUs). Microsoft requires this subnet to be **/26 or larger** (/25, /24, and so on); smaller prefixes such as /27 are not valid for Bastion on Azure. See [Choose the right Azure Bastion SKU](https://learn.microsoft.com/en-us/azure/bastion/bastion-sku-comparison) and [Bastion configuration settings](https://learn.microsoft.com/en-us/azure/bastion/configuration-settings#subnet). ```bash az network vnet subnet create \ --name AzureBastionSubnet \ --resource-group rg-bastion-demo \ --vnet-name vnet-bastion-demo \ --address-prefixes 10.0.255.0/26 ``` ```bash title="Output" { "addressPrefix": "10.0.255.0/26", "delegations": [], "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-bastion-demo/providers/Microsoft.Network/virtualNetworks/vnet-bastion-demo/subnets/AzureBastionSubnet", "name": "AzureBastionSubnet", "privateEndpointNetworkPolicies": "Disabled", "privateLinkServiceNetworkPolicies": "Enabled", "provisioningState": "Succeeded", "resourceGroup": "rg-bastion-demo", "type": "Microsoft.Network/virtualNetworks/subnets" ... } ``` ### Create a Standard public IP address Create a Standard SKU static public IP address required by the Bastion host: ```bash az network public-ip create \ --name pip-bastion \ --resource-group rg-bastion-demo \ --location westeurope \ --sku Standard \ --allocation-method Static ``` ```bash title="Output" { "publicIp": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-bastion-demo/providers/Microsoft.Network/publicIPAddresses/pip-bastion", "idleTimeoutInMinutes": 4, "ipAddress": "20.56.137.218", "location": "westeurope", "name": "pip-bastion", "provisioningState": "Succeeded", "publicIPAllocationMethod": "Static", "resourceGroup": "rg-bastion-demo", "sku": { "name": "Standard", "tier": "Regional" }, "type": "Microsoft.Network/publicIPAddresses", ... } } ``` ### Create a bastion host Create the Bastion host, linking it to the virtual network and the public IP address: ```bash az network bastion create \ --name bastion-demo \ --resource-group rg-bastion-demo \ --location westeurope \ --vnet-name vnet-bastion-demo \ --public-ip-address pip-bastion ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-bastion-demo/providers/Microsoft.Network/bastionHosts/bastion-demo", "ipConfigurations": [ { "name": "bastion_ip_config", "provisioningState": "Succeeded", "publicIPAddress": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-bastion-demo/providers/Microsoft.Network/publicIPAddresses/pip-bastion", "resourceGroup": "rg-bastion-demo" }, "subnet": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-bastion-demo/providers/Microsoft.Network/virtualNetworks/vnet-bastion-demo/subnets/AzureBastionSubnet", "resourceGroup": "rg-bastion-demo" } } ], "location": "westeurope", "name": "bastion-demo", "provisioningState": "Succeeded", "resourceGroup": "rg-bastion-demo", "scaleUnits": 2, "sku": { "name": "Standard" }, "zones": [] } ``` ### Get and list bastion hosts Retrieve the details of the Bastion host and list all Bastion instances in the resource group: ```bash az network bastion show \ --name bastion-demo \ --resource-group rg-bastion-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-bastion-demo/providers/Microsoft.Network/bastionHosts/bastion-demo", "ipConfigurations": [ { "name": "bastion_ip_config", "provisioningState": "Succeeded", "publicIPAddress": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-bastion-demo/providers/Microsoft.Network/publicIPAddresses/pip-bastion", "resourceGroup": "rg-bastion-demo" }, "subnet": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-bastion-demo/providers/Microsoft.Network/virtualNetworks/vnet-bastion-demo/subnets/AzureBastionSubnet", "resourceGroup": "rg-bastion-demo" } } ], "location": "westeurope", "name": "bastion-demo", "provisioningState": "Succeeded", "resourceGroup": "rg-bastion-demo", "scaleUnits": 2, "sku": { "name": "Standard" }, "zones": [] } ``` Then list all Bastion instances in the resource group: ```bash az network bastion list --resource-group rg-bastion-demo ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-bastion-demo/providers/Microsoft.Network/bastionHosts/bastion-demo", "ipConfigurations": [ { "name": "bastion_ip_config", "provisioningState": "Succeeded", "publicIPAddress": { "id": "...pip-bastion...", "resourceGroup": "rg-bastion-demo" }, "subnet": { "id": "...AzureBastionSubnet...", "resourceGroup": "rg-bastion-demo" } } ], "location": "westeurope", "name": "bastion-demo", "provisioningState": "Succeeded", "resourceGroup": "rg-bastion-demo", "scaleUnits": 2, "sku": { "name": "Standard" }, "zones": [] } ] ``` ### Delete the bastion host Delete the Bastion host and verify it no longer appears in the list: ```bash az network bastion delete \ --name bastion-demo \ --resource-group rg-bastion-demo \ --yes ``` Then list all bastion hosts in the resource group to confirm none remain: ```bash az network bastion list --resource-group rg-bastion-demo ``` ```bash title="Output" [] ``` ## Features The Bastion Host emulator supports the following features: - **Create and manage bastion hosts**: Full lifecycle management including create, get, update, list, and delete. - **Subnet name validation**: Enforces that the target subnet is named `AzureBastionSubnet`. On Azure, dedicated Bastion deployments (Basic, Standard, or Premium SKUs) also require this subnet to be `/26` or larger—see [Bastion subnet requirements](https://learn.microsoft.com/en-us/azure/bastion/configuration-settings#subnet). - **Standard SKU public IP validation**: Enforces that the associated public IP address uses the Standard SKU and Static allocation. - **IP configuration storage**: Records and returns the subnet and public IP address associations in `ipConfigurations`. - **Tags**: Apply and update resource tags on bastion host resources. - **SKU tiers**: Support for `Basic` and `Standard` bastion SKUs. Azure also provides `Developer` and `Premium` SKUs; see [Azure Bastion SKUs](https://learn.microsoft.com/en-us/azure/bastion/bastion-sku-comparison). ## Limitations - **No RDP or SSH connectivity**: Bastion Host is a mock implementation. No actual remote desktop or SSH sessions are established through the emulator. - **No TLS tunnel**: The secure TLS tunneling behavior of Azure Bastion is not simulated. - **No VM connectivity verification**: The emulator does not verify that the target VM exists or is reachable. - **No data persistence**: Bastion Host resources are not persisted and are lost when the emulator is stopped or restarted. ## Samples Explore end-to-end examples in the [LocalStack for Azure Samples](https://github.com/localstack/localstack-azure-samples) repository. ## API Coverage # Blob Storage > Get started with Azure Blob Storage on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Blob Storage is a highly scalable object storage solution optimized for storing massive volumes of unstructured data, such as text and binary content. It supports block blobs, append blobs, and page blobs, and is commonly used for serving documents, images, and streaming media. For more information, see [Introduction to Azure Blob Storage](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Blob Storage. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Blob Storage's integration with LocalStack. ## Getting started This guide is designed for users new to Blob Storage and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` ### Create a resource group Create a resource group to contain your storage resources: ```bash az group create \ --name rg-blob-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-blob-demo", "location": "westeurope", "managedBy": null, "name": "rg-blob-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a storage account Create a storage account in the resource group: ```bash az storage account create \ --name stblobdemols \ --resource-group rg-blob-demo \ --location westeurope \ --sku Standard_LRS \ --only-show-errors ``` ```bash title="Output" { ... "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-blob-demo/providers/Microsoft.Storage/storageAccounts/stblobdemols", ... "name": "stblobdemols", ... "placement": null, "primaryEndpoints": { "blob": "https://stblobdemols.blob.core.azure.localhost.localstack.cloud:456", ... }, .... } ``` ### Authentication There are three ways to authenticate storage container commands against the emulator: #### Storage account key Retrieve the account key and pass it with `--account-name` and `--account-key`: ```bash ACCOUNT_KEY=$(az storage account keys list \ --account-name stblobdemols \ --resource-group rg-blob-demo \ --query "[0].value" \ --output tsv) az storage container list \ --account-name stblobdemols \ --account-key "$ACCOUNT_KEY" ``` #### Login credentials Use `--auth-mode login` to authenticate with the current session credentials: ```bash az storage container list \ --account-name stblobdemols \ --auth-mode login ``` #### Connection string Bundle the account name and key into a single value: ```bash CONNECTION_STRING=$(az storage account show-connection-string \ --name stblobdemols \ --resource-group rg-blob-demo \ --query connectionString -o tsv) az storage container list \ --connection-string "$CONNECTION_STRING" ``` The remaining examples in this guide use connection strings for brevity. ### Create and inspect a blob container Create a container in the storage account: ```bash az storage container create \ --name documents \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" { "created": true } ``` Verify the container exists: ```bash az storage container exists \ --name documents \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" { "exists": true } ``` List containers in the storage account: ```bash az storage container list \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" [ { ... "name": "documents", "properties": { ... "lease": { ... }, ... }, ... } ] ``` ### Upload, list, and download blobs Upload a local file as a block blob: ```bash echo "Hello from LocalStack" > /tmp/hello.txt az storage blob upload \ --container-name documents \ --name hello.txt \ --file /tmp/hello.txt \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" { "client_request_id": "...", "content_md5": "...", "date": "...", "etag": "... ... } ``` List blobs in the container: ```bash az storage blob list \ --container-name documents \ --connection-string "$CONNECTION_STRING" \ --output table ``` Download the blob to a local file: ```bash az storage blob download \ --container-name documents \ --name hello.txt \ --file /tmp/hello-downloaded.txt \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" Finished[#############################################################] 100.0000% { "container": "documents", ... } ``` Delete the blob: ```bash az storage blob delete \ --container-name documents \ --name hello.txt \ --connection-string "$CONNECTION_STRING" ``` ## Features The Blob Storage emulator supports the following features: - **Data plane REST API**: Blob CRUD, message operations (put, peek, get, delete), container metadata, stored access policies, and SAS token generation. - **Control plane REST API**: Create, update, delete, and get containers, get and set container service properties via Azure Resource Manager. - **Multiple authentication modes**: Storage account key, login credentials, and connection strings. ## Limitations - **No data persistence across restarts**: Blob data is not persisted and is lost when the LocalStack emulator is stopped or restarted. - **Blob service properties**: `set_service_properties` is a no-op and `get_service_properties` returns empty defaults, unlike Azure where CORS, logging, and metrics settings are persisted and applied. - **Storage account keys**: Keys are emulator-generated rather than managed by Azure. - **Header validation**: Unsupported request headers or parameters are silently accepted instead of being rejected. - **API version enforcement**: The emulator does not validate the `x-ms-version` header; all API versions are accepted. - **RBAC enforcement is opt-in**: By default, data-plane operations succeed regardless of role assignments. Set `LS_AZURE_ENFORCE_RBAC` to require the caller to hold a role such as `Storage Blob Data Contributor`; see [Role Assignment: Enabling RBAC enforcement](/azure/services/role-assignment/#enabling-rbac-enforcement). ## Samples The following samples demonstrate how to use Azure Blob Storage with LocalStack for Azure: - [Azure Functions Sample with LocalStack for Azure](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-storage-http/dotnet) - [Azure Functions App with Managed Identity](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-managed-identity/python) - [Function App and Service Bus](https://github.com/localstack/localstack-azure-samples/samples/function-app-service-bus/dotnet/README.md) - [Azure Web App with Managed Identity](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-managed-identity/python) ## API Coverage # Container Apps > Get started with Azure Container Apps on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Container Apps is a serverless container platform for running containerized applications and microservices without managing Kubernetes infrastructure. Applications are deployed into a managed environment, receive an HTTPS ingress endpoint, and are versioned through revisions, while background and scheduled work runs as jobs. For more information, see [Azure Container Apps overview](https://learn.microsoft.com/en-us/azure/container-apps/overview). LocalStack for Azure provides a local environment for building and testing applications that use Azure Container Apps. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Container Apps' integration with LocalStack. ## Getting started This guide is designed for users new to Container Apps and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group that will contain your Container Apps resources: ```bash az group create \ --name rg-aca-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-aca-demo", "location": "westeurope", "managedBy": null, "name": "rg-aca-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a Container Apps environment Create a managed environment that will host your container apps and jobs: ```bash az containerapp env create \ --name my-environment \ --resource-group rg-aca-demo \ --location westeurope \ --logs-destination none ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-aca-demo/providers/Microsoft.App/managedEnvironments/my-environment", "location": "westeurope", "name": "my-environment", "properties": { "appLogsConfiguration": { "destination": null }, "defaultDomain": "nicesmoke-4f9d21-westeurope.aca.azure.localhost.localstack.cloud", "provisioningState": "Succeeded" }, "type": "Microsoft.App/managedEnvironments" ... } ``` Each environment receives a `defaultDomain` under `aca.azure.localhost.localstack.cloud`. This domain resolves to `127.0.0.1`, so the ingress endpoints of apps in the environment are directly reachable from your machine. ### Create a container app Create a container app with external HTTP ingress: ```bash az containerapp create \ --name quickstart \ --resource-group rg-aca-demo \ --environment my-environment \ --image mcr.microsoft.com/k8se/quickstart:latest \ --ingress external \ --target-port 80 \ --cpu 0.5 --memory 1Gi \ --min-replicas 1 --max-replicas 3 \ --revision-suffix v1 ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-aca-demo/providers/Microsoft.App/containerapps/quickstart", "location": "westeurope", "name": "quickstart", "properties": { "configuration": { "activeRevisionsMode": "Single", "ingress": { "allowInsecure": false, "external": true, "fqdn": "quickstart--nicesmoke-4f9d21-westeurope.aca.azure.localhost.localstack.cloud", "targetPort": 80, "transport": "Auto" } }, "latestReadyRevisionName": "quickstart--v1", "latestRevisionName": "quickstart--v1", "provisioningState": "Succeeded", "runningStatus": "Running", "template": { "containers": [ { "image": "mcr.microsoft.com/k8se/quickstart:latest", "name": "quickstart", "resources": { "cpu": 0.5, "memory": "1Gi" } } ], "revisionSuffix": "v1", "scale": { "maxReplicas": 3, "minReplicas": 1 } } }, "type": "Microsoft.App/containerApps" ... } ``` :::note The first container app or job in an environment provisions a local Kubernetes (k3d) cluster backing that environment, so the first create can take a few minutes. Subsequent deployments into the same environment are much faster. ::: ### Invoke the container app Retrieve the ingress FQDN and send a request to the running app. The FQDN is served by the LocalStack gateway on port `4566` with a valid TLS certificate: ```bash FQDN=$(az containerapp show \ --name quickstart \ --resource-group rg-aca-demo \ --query "properties.configuration.ingress.fqdn" \ --output tsv) curl -s -o /dev/null -w "%{http_code}\n" "https://$FQDN:4566/" ``` ```bash title="Output" 200 ``` You can also open `https://$FQDN:4566/` in your browser to see the welcome page of the quickstart image. ### Manage secrets Add a secret to the container app: ```bash az containerapp secret set \ --name quickstart \ --resource-group rg-aca-demo \ --secrets api-key=top-secret ``` List the secrets, including their values: ```bash az containerapp secret list \ --name quickstart \ --resource-group rg-aca-demo \ --show-values ``` ```bash title="Output" [ { "identity": null, "keyVaultUrl": null, "name": "api-key", "value": "top-secret" } ] ``` Secrets can be referenced from environment variables via `secretref:`, mounted as secret volumes, and defined as Key Vault references that are resolved from the emulated Key Vault. ### Update the app and work with revisions Update the container app with a new environment variable. Every change to the app template mints a new revision: ```bash az containerapp update \ --name quickstart \ --resource-group rg-aca-demo \ --revision-suffix v2 \ --set-env-vars GREETING=hello ``` List the revisions of the app: ```bash az containerapp revision list \ --name quickstart \ --resource-group rg-aca-demo \ --query "[].name" \ --output tsv ``` ```bash title="Output" quickstart--v1 quickstart--v2 ``` In the default `Single` revisions mode, the latest ready revision serves all traffic and older revisions are deactivated automatically. In `Multiple` mode, revisions stay active and can be deactivated and re-activated with `az containerapp revision deactivate` and `az containerapp revision activate`. ### Run a job Create a manually triggered job in the same environment: ```bash az containerapp job create \ --name my-job \ --resource-group rg-aca-demo \ --environment my-environment \ --trigger-type Manual \ --replica-timeout 1800 \ --image mcr.microsoft.com/k8se/quickstart-jobs:latest \ --cpu 0.25 --memory 0.5Gi ``` Start an execution of the job. The command returns as soon as the execution is accepted, so the execution is first reported as `Running`: ```bash az containerapp job start \ --name my-job \ --resource-group rg-aca-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-aca-demo/providers/Microsoft.App/jobs/my-job/executions/my-job-8b31fc2", "name": "my-job-8b31fc2" } ``` List the executions of the job to inspect their status: ```bash az containerapp job execution list \ --name my-job \ --resource-group rg-aca-demo \ --query "[].{name:name, status:properties.status}" ``` ```bash title="Output" [ { "name": "my-job-8b31fc2", "status": "Running" } ] ``` Re-run the command until the status reaches a terminal state, such as `Succeeded` or `Failed`: ```bash title="Output" [ { "name": "my-job-8b31fc2", "status": "Succeeded" } ] ``` ### Delete and verify Delete the container app and the job, then delete the environment: ```bash az containerapp delete \ --name quickstart \ --resource-group rg-aca-demo \ --yes az containerapp job delete \ --name my-job \ --resource-group rg-aca-demo \ --yes az containerapp env delete \ --name my-environment \ --resource-group rg-aca-demo \ --yes ``` Deleting a container app removes its containers from the local cluster. An environment can only be deleted once all apps, jobs, and managed certificates in it have been removed. Verify the resource group is now empty: ```bash az containerapp list \ --resource-group rg-aca-demo ``` ```bash title="Output" [] ``` ## Features - **Real container execution:** Container apps and job executions run as real containers on a local Kubernetes (k3d) cluster that LocalStack provisions per managed environment. Set `LS_AZURE_CONTAINER_APPS_RUNTIME=0` to manage Container Apps resources in control-plane-only mode without starting containers. - **Live HTTPS ingress:** Every app with ingress gets an FQDN that resolves to `127.0.0.1` and is served with a valid TLS certificate. CORS policies, IP security restrictions, HTTPS redirects, and session affinity are enforced at the ingress. - **Revisions:** Both `Single` and `Multiple` revision modes are supported, including revision minting on template changes, activation and deactivation, and per-revision FQDNs. - **Secrets:** Inline secrets and Key Vault references are resolved and injected into containers as environment variables or secret volume mounts. - **Private registries:** Registry credentials with a password secret reference are used to pull images, including images hosted in the emulated Azure Container Registry. - **Health probes:** Liveness, readiness, and startup probes (HTTP and TCP) are enforced by the local cluster. - **Container logs:** `az containerapp logs show` streams logs directly from the running container via each replica's log stream endpoint. - **Jobs:** Manually started job executions run to completion and report `Succeeded` or `Failed`; parallelism and replica completion count are honored. - **Auxiliary resources:** Dapr components, environment storages, managed certificates, and HTTP route configs support full CRUD with validation. HTTP route configs perform real path-based routing, including exact and prefix matches and prefix rewrites. ## Limitations - **No autoscaling:** KEDA scale rules are stored and echoed back but not evaluated, and scale-to-zero is not supported. Apps run with a fixed replica count derived from `minReplicas` (at least 1, capped by `maxReplicas`). - **No traffic splitting:** Traffic weights across revisions are stored but not enforced at the data plane; the latest ready revision serves all requests. - **No automatic job triggers:** Scheduled (cron) and event-driven job triggers are stored but never fire; `az containerapp job start` is the only way to create an execution. - **Azure Files storages are metadata-only:** Environment storages can be managed via CRUD, but `AzureFile` and `NfsAzureFile` volumes are skipped at deploy time and the container starts without the mount. - **No Dapr sidecar:** Dapr components and app-level Dapr configuration are stored and validated, but no Dapr sidecar is injected into running containers. - **No real certificates or domain validation:** Managed certificates skip certificate issuance and DNS validation, custom domains are stored without verification, and custom hostname analysis always reports the domain verification as failed. - **No interactive log streaming or exec:** `az containerapp logs show --follow`, system logs (`--type system`), and `az containerapp exec` are not supported. ## Samples - [Guestbook on Azure Container Apps with Blob Storage and Container Registry](https://github.com/localstack/localstack-azure-samples/tree/main/samples/container-apps-blob-storage/python/) ## API Coverage # Container Instances > Get started with Azure Container Instances on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Container Instances is a serverless container service that lets you run Docker containers on demand without managing any underlying virtual machines or orchestration infrastructure. Each deployment unit is called a container group and can host one or more containers that share a network and lifecycle. Container Instances is well-suited for short-lived tasks, CI workloads, and event-driven processing. For more information, see [About Azure Container Instances](https://learn.microsoft.com/en-us/azure/container-instances/container-instances-overview). LocalStack for Azure provides a local environment for building and testing applications that use Azure Container Instances. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Container Instances' integration with LocalStack. ## Getting started This guide is designed for users new to Container Instances and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group that will contain your Container Instances resources: ```bash az group create \ --name rg-aci-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-aci-demo", "location": "westeurope", "managedBy": null, "name": "rg-aci-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a container group Create a container group with a single container, a public IP address, and port 80 exposed: ```bash az container create \ --name mycontainer \ --resource-group rg-aci-demo \ --image mcr.microsoft.com/cbl-mariner/base/core:2.0 \ --cpu 1 --memory 1 \ --ports 80 \ --ip-address Public \ --restart-policy Never \ --command-line "/bin/sh -c 'echo hello && sleep 3600'" ``` ```bash title="Output" { "containers": [ { "command": [ "/bin/sh", "-c", "echo hello && sleep 3600" ], "environmentVariables": [], "image": "mcr.microsoft.com/cbl-mariner/base/core:2.0", "instanceView": { "currentState": { "detailStatus": "", "exitCode": null, "finishTime": null, "startTime": "2026-04-15T13:05:28+00:00", "state": "Running" }, "events": [], "previousState": null, "restartCount": 0 }, "name": "mycontainer", "ports": [ { "port": 80, "protocol": null } ], "resources": { "limits": null, "requests": { "cpu": 1.0, "memoryInGb": 1.0 } } } ], "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-aci-demo/providers/Microsoft.ContainerInstance/containerGroups/mycontainer", "instanceView": { "events": [], "state": "Running" }, "ipAddress": { "ip": "10.0.0.1", "ports": [ { "port": 80, "protocol": "TCP" } ], "type": "Public" }, "location": "westeurope", "name": "mycontainer", "osType": "Linux", "provisioningState": "Succeeded", "restartPolicy": "Never", "sku": "Standard", "type": "Microsoft.ContainerInstance/containerGroups" ... } ``` ### Show and list container groups Retrieve the full details of the container group to inspect its current state: ```bash az container show \ --name mycontainer \ --resource-group rg-aci-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-aci-demo/providers/Microsoft.ContainerInstance/containerGroups/mycontainer", "instanceView": { "events": [], "state": "Running" }, "ipAddress": { "ip": "10.0.0.1", "ports": [ { "port": 80, "protocol": "TCP" } ], "type": "Public" }, "location": "westeurope", "name": "mycontainer", "osType": "Linux", "provisioningState": "Succeeded", "restartPolicy": "Never", "type": "Microsoft.ContainerInstance/containerGroups" ... } ``` List all container groups within the resource group to see a summary of each one: ```bash az container list \ --resource-group rg-aci-demo ``` ```bash title="Output" [ { "ipAddress": { "ip": "10.0.0.1", "ports": [ { "port": 80, "protocol": "TCP" } ], "type": "Public" }, "location": "westeurope", "name": "mycontainer", "osType": "Linux", "provisioningState": "Succeeded" } ] ``` ### Retrieve container logs Fetch the standard output from a running container to verify it started correctly: ```bash az container logs \ --name mycontainer \ --resource-group rg-aci-demo \ --container-name mycontainer ``` ```bash title="Output" hello ``` ### Stop, start, and restart Stop a running container group to release its compute resources without deleting the group: ```bash az container stop \ --name mycontainer \ --resource-group rg-aci-demo ``` Verify that the container group state has changed to `Stopped`: ```bash az container show \ --name mycontainer \ --resource-group rg-aci-demo \ --query "{name:name, provisioningState:provisioningState, instanceView:instanceView}" ``` ```bash title="Output" { "instanceView": { "events": [], "state": "Stopped" }, "name": "mycontainer", "provisioningState": "Succeeded" } ``` Start the container group again to resume the containers from their stopped state: ```bash az container start \ --name mycontainer \ --resource-group rg-aci-demo ``` Verify the container group is running again: ```bash az container show \ --name mycontainer \ --resource-group rg-aci-demo \ --query "{name:name, provisioningState:provisioningState, instanceView:instanceView}" ``` ```bash title="Output" { "instanceView": { "events": [], "state": "Running" }, "name": "mycontainer", "provisioningState": "Succeeded" } ``` Restart all containers in the group without re-creating the group itself: ```bash az container restart \ --name mycontainer \ --resource-group rg-aci-demo ``` Confirm the group is running after the restart: ```bash az container show \ --name mycontainer \ --resource-group rg-aci-demo \ --query "{name:name, provisioningState:provisioningState, instanceView:instanceView}" ``` ```bash title="Output" { "instanceView": { "events": [], "state": "Running" }, "name": "mycontainer", "provisioningState": "Succeeded" } ``` ### Delete and verify Delete the container group to remove all associated containers and Docker resources from the emulator: ```bash az container delete \ --name mycontainer \ --resource-group rg-aci-demo \ --yes ``` Verify the resource group is now empty: ```bash az container list \ --resource-group rg-aci-demo ``` ```bash title="Output" [] ``` ## Features - **Real Docker execution:** Container groups are backed by real Docker containers running on the local Docker engine. - **Full container group lifecycle:** Stop, start, and restart operations are supported. - **Container logs:** Logs are streamed directly from the Docker container and returned via the `containers__list_logs` API. - **Volume types:** Both `emptyDir` and `secret` volume types are implemented using bind mounts from temporary directories on the host. - **Private registries:** Image registry credentials passed via `--registry-login-server`, `--registry-username`, and `--registry-password` are used to pull images from private registries. - **Init containers:** Init containers run sequentially before main containers and must exit with code 0 for group creation to succeed. ## Limitations - **No interactive exec sessions:** `az container exec` and the `containers__execute_command` API return a stub WebSocket URI but do not provide a real terminal session. - **No interactive attach:** `containers__attach` returns a stub WebSocket URI and does not stream container output interactively. - **Linux containers only:** Windows containers are not supported. - **No VNet integration:** Subnet-based networking is not emulated; all container groups are assigned an IP address from a local address pool. - **No `az container update`:** Tag updates must be performed via `az rest PATCH`. ## Samples - [Quickstart: Deploy a container instance with Azure Container Instances](https://learn.microsoft.com/en-us/azure/container-instances/container-instances-quickstart) - [Enable a managed identity in Azure Container Instances to access Key Vault](https://learn.microsoft.com/en-us/azure/container-instances/container-instances-managed-identity) ## API Coverage # Container Registry > Get started with Azure Container Registry on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Container Registry (ACR) is a managed, private OCI-compatible registry for storing container images and Helm charts. It integrates natively with Azure Kubernetes Service, Azure Container Instances, and Azure App Service to streamline container-based application deployments. ACR is commonly used to build, store, and manage container images as part of a continuous integration and deployment pipeline. For more information, see [What is Azure Container Registry?](https://learn.microsoft.com/en-us/azure/container-registry/container-registry-intro). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Container Registry. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Container Registry's integration with LocalStack. ## Getting started This guide walks you through creating a registry, logging in with Docker, pushing an image, and listing repositories. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create --name rg-acr-demo --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-acr-demo", "location": "westeurope", "name": "rg-acr-demo", "properties": { "provisioningState": "Succeeded" }, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a container registry Create a Basic-tier Azure Container Registry (Azure also offers Standard and Premium; limits differ by tier, as described in [Azure Container Registry SKU features and limits](https://learn.microsoft.com/en-us/azure/container-registry/container-registry-skus)): ```bash az acr create \ --name myacrdemo \ --resource-group rg-acr-demo \ --sku Basic \ --admin-enabled true ``` ```bash title="Output" { "adminUserEnabled": true, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-acr-demo/providers/Microsoft.ContainerRegistry/registries/myacrdemo", "location": "westeurope", "loginServer": "myacrdemo.azurecr.azure.localhost.localstack.cloud:4566", "name": "myacrdemo", "provisioningState": "Succeeded", "resourceGroup": "rg-acr-demo", "sku": { "name": "Basic", "tier": "Basic" }, "type": "Microsoft.ContainerRegistry/registries" ... } ``` ### Log in with Docker Authenticate the local Docker daemon to the registry using the Azure CLI: ```bash az acr login --name myacrdemo ``` With Azure, `az acr login` is the documented way to authenticate Docker to your registry using your signed-in Microsoft Entra identity ([Push your first image to your Azure container registry using the Docker CLI](https://learn.microsoft.com/en-us/azure/container-registry/container-registry-get-started-docker-cli?tabs=azure-cli); for other auth options see [Authenticate with an Azure container registry](https://learn.microsoft.com/en-us/azure/container-registry/container-registry-authentication)). The emulator does not reproduce the full Azure identity flow, so the CLI may print an informational warning before confirming `Login Succeeded`. That is expected for LocalStack. ### Build and push an image Create a minimal `Dockerfile` for the demo image: ```bash cat > Dockerfile <<'EOF' FROM alpine:latest CMD ["echo", "Hello from LocalStack ACR!"] EOF ``` Capture the registry login server returned by the emulator, then build and push the image using that address: ```bash LOGIN_SERVER=$(az acr show --name myacrdemo --resource-group rg-acr-demo --query loginServer -o tsv) docker build -t $LOGIN_SERVER/hello:v1 . docker push $LOGIN_SERVER/hello:v1 ``` Alternatively build directly within the emulated registry: ```bash az acr build \ --image hello:v1 \ --registry myacrdemo \ . ``` ### Pull an image Pull the image back from the registry to confirm it was pushed correctly: ```bash docker pull $LOGIN_SERVER/hello:v1 ``` ### List repositories List all image repositories stored in the registry: ```bash az acr repository list --name myacrdemo ``` ```bash title="Output" [ "hello" ] ``` ### Check name availability Check registry name availability against the management API. The following example assumes you already created `myacrdemo` above, so the response matches a name that is not available: ```bash az acr check-name --name myacrdemo ``` ```bash title="Output" { "message": "The registry myacrdemo is already in use.", "nameAvailable": false, "reason": "AlreadyExists" } ``` ### Update registry settings Disable the registry admin account (`az acr update` is the same command Azure documents for changing registry settings; see [`az acr update`](https://learn.microsoft.com/en-us/cli/azure/acr#az-acr-update)): ```bash az acr update --name myacrdemo --resource-group rg-acr-demo --admin-enabled false ``` ```bash title="Output" { "adminUserEnabled": false, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-acr-demo/providers/Microsoft.ContainerRegistry/registries/myacrdemo", "loginServer": "myacrdemo.azurecr.azure.localhost.localstack.cloud:4566", "name": "myacrdemo", "provisioningState": "Succeeded", "resourceGroup": "rg-acr-demo", "sku": { "name": "Basic", "tier": "Basic" }, "type": "Microsoft.ContainerRegistry/registries" ... } ``` ### Show registry usage Show current storage usage statistics for the registry: ```bash az acr show-usage --name myacrdemo --resource-group rg-acr-demo ``` ```bash title="Output" { "value": [ { "currentValue": 0, "limit": 10737418240, "name": "Size", "unit": "Bytes" }, { "currentValue": 0, "limit": 2, "name": "Webhooks", "unit": "Count" }, { "currentValue": 0, "limit": 100, "name": "ScopeMaps", "unit": "Count" }, { "currentValue": 0, "limit": 100, "name": "Tokens", "unit": "Count" } ] } ``` ### Delete and verify Delete the registry and remove the demo `Dockerfile` created earlier: ```bash az acr delete --name myacrdemo --resource-group rg-acr-demo --yes rm -f Dockerfile ``` ## Features - **Full CRUD lifecycle:** Create, read, update, and delete registry resources using the Azure CLI or ARM API. - **Admin user management:** Enable or disable admin user access and retrieve admin credentials. - **Name availability check:** Validate registry name uniqueness via `az acr check-name`. - **Image push and pull:** Push and pull OCI-compliant container images using the standard Docker CLI. - **In-registry builds:** Build images directly in the emulated registry using `az acr build`. - **Repository listing:** List repositories and image tags stored in the registry. - **Registry usage reporting:** Retrieve storage and limit usage via `az acr show-usage`. - **Registry update:** Modify registry properties such as admin enabled and SKU. - **Multiple SKUs accepted:** Basic, Standard, and Premium SKU names are accepted (all backed by the same local registry). ## Limitations - **Geo-replication not supported:** Multi-region registry replication is not emulated. - **ACR Tasks beyond basic build:** Task scheduling, triggers, and multi-step task workflows are mocked at the ARM level but not executed. - **Private endpoints for ACR:** Private Link–based network isolation is not supported. - **Webhook notifications:** Registry webhooks defined via the ARM API are stored but not fired on push events. - **Content trust and quarantine:** Image signing and quarantine policies are not enforced. - **Delete is not persisted:** `az acr delete` returns HTTP 200 and exits cleanly, but the registry record is not removed from the in-memory store in the current emulator version. ## Samples The following sample demonstrates how to use Azure Container Registry with LocalStack for Azure: - [Azure Container Instances, Key Vault, and Storage](https://github.com/localstack/localstack-azure-samples/blob/main/samples/aci-blob-storage/python/README.md) ## API Coverage # Cosmos DB > Get started with Azure Cosmos DB on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Cosmos DB is a globally distributed, multi-model NoSQL database service designed for high availability and low latency. It supports multiple APIs including SQL (NoSQL), MongoDB, Cassandra, Gremlin, and Table, enabling teams to choose the data model that best fits their application. Cosmos DB is commonly used for real-time applications, globally replicated datasets, and multi-tenant SaaS platforms that require predictable performance at any scale. For more information, see [Introduction to Azure Cosmos DB](https://learn.microsoft.com/en-us/azure/cosmos-db/introduction). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Cosmos DB. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Cosmos DB's integration with LocalStack. ## Getting started This guide walks you through creating Cosmos DB accounts, databases, and containers using the SQL and MongoDB APIs. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### SQL (NoSQL) API This section walks through creating a Cosmos DB account with the SQL API, creating a database and a container, and listing resources. #### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create --name rg-cosmos-demo --location eastus ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-cosmos-demo", "location": "eastus", "name": "rg-cosmos-demo", "properties": { "provisioningState": "Succeeded" }, "type": "Microsoft.Resources/resourceGroups" } ``` #### Create a Cosmos DB account Create a Cosmos DB account configured for the SQL API: ```bash az cosmosdb create \ --name mycosmosaccount \ --resource-group rg-cosmos-demo \ --locations regionName=eastus ``` ```bash title="Output" { "databaseAccountOfferType": "Standard", "documentEndpoint": "https://mycosmosaccount.documents.azure.com:8081/", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-cosmos-demo/providers/Microsoft.DocumentDB/databaseAccounts/mycosmosaccount", "kind": "GlobalDocumentDB", "location": "East US", "name": "mycosmosaccount", "provisioningState": "Succeeded", "resourceGroup": "rg-cosmos-demo", "type": "Microsoft.DocumentDB/databaseAccounts", ... } ``` #### List account keys Retrieve the primary and secondary access keys for the account: ```bash az cosmosdb keys list \ --name mycosmosaccount \ --resource-group rg-cosmos-demo ``` ```bash title="Output" { "primaryMasterKey": "C2y6yDjf5/R+ob0N8A7C...", "primaryReadonlyMasterKey": "C2y6yDjf5/R+ob0N8A7C...", "secondaryMasterKey": "C2y6yDjf5/R+ob0N8A7C...", "secondaryReadonlyMasterKey": "C2y6yDjf5/R+ob0N8A7C..." } ``` Retrieve the connection string for the account: ```bash az cosmosdb keys list \ --type connection-strings \ --name mycosmosaccount \ --resource-group rg-cosmos-demo \ --query "connectionStrings[0].connectionString" ``` ```bash title="Output" "AccountEndpoint=https://mycosmosaccount.documents.azure.com:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==;" ``` #### Create a SQL database Create a SQL API database within the Cosmos DB account: ```bash az cosmosdb sql database create \ --account-name mycosmosaccount \ --resource-group rg-cosmos-demo \ --name mydb ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-cosmos-demo/providers/Microsoft.DocumentDB/databaseAccounts/mycosmosaccount/sqlDatabases/mydb", "name": "mydb", "resourceGroup": "rg-cosmos-demo", "type": "Microsoft.DocumentDB/databaseAccounts/sqlDatabases" ... } ``` #### Create a SQL container Create a SQL container with a partition key path of `/id`: ```bash az cosmosdb sql container create \ --account-name mycosmosaccount \ --resource-group rg-cosmos-demo \ --database-name mydb \ --name mycontainer \ --partition-key-path /id ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-cosmos-demo/providers/Microsoft.DocumentDB/databaseAccounts/mycosmosaccount/sqlDatabases/mydb/containers/mycontainer", "name": "mycontainer", "resourceGroup": "rg-cosmos-demo", "type": "Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers" ... } ``` #### List databases and containers List all SQL databases in the account and all containers within the database: ```bash az cosmosdb sql database list \ --account-name mycosmosaccount \ --resource-group rg-cosmos-demo ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-cosmos-demo/providers/Microsoft.DocumentDB/databaseAccounts/mycosmosaccount/sqlDatabases/mydb", "name": "mydb", "resourceGroup": "rg-cosmos-demo", "type": "Microsoft.DocumentDB/databaseAccounts/sqlDatabases" } ] ``` Then list all containers in the database: ```bash az cosmosdb sql container list \ --account-name mycosmosaccount \ --resource-group rg-cosmos-demo \ --database-name mydb ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-cosmos-demo/providers/Microsoft.DocumentDB/databaseAccounts/mycosmosaccount/sqlDatabases/mydb/containers/mycontainer", "name": "mycontainer", "resourceGroup": "rg-cosmos-demo", "type": "Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers" } ] ``` ### MongoDB API This section walks through creating a Cosmos DB account with the MongoDB API, creating a database, and connecting with a MongoDB client. #### Create a MongoDB Cosmos DB account Create a second Cosmos DB account configured for the MongoDB API: ```bash az cosmosdb create \ --name mymongoaccount \ --resource-group rg-cosmos-demo \ --locations regionName=eastus \ --kind MongoDB ``` ```bash title="Output" { "databaseAccountOfferType": "Standard", "documentEndpoint": "https://mymongoaccount.documents.azure.com:443/", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-cosmos-demo/providers/Microsoft.DocumentDB/databaseAccounts/mymongoaccount", "kind": "MongoDB", "location": "East US", "name": "mymongoaccount", "provisioningState": "Succeeded", "resourceGroup": "rg-cosmos-demo", "type": "Microsoft.DocumentDB/databaseAccounts", ... } ``` #### Retrieve connection string Retrieve the MongoDB-compatible connection string for the account: ```bash az cosmosdb keys list \ --name mymongoaccount \ --resource-group rg-cosmos-demo \ --type connection-strings \ --query "connectionStrings[0].connectionString" \ --output tsv ``` ```bash title="Output" mongodb://primary:@172.17.0.10:27017/ ``` #### Create a MongoDB database Create a MongoDB database within the account: ```bash az cosmosdb mongodb database create \ --account-name mymongoaccount \ --resource-group rg-cosmos-demo \ --name mymongodb ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-cosmos-demo/providers/Microsoft.DocumentDB/databaseAccounts/mymongoaccount/mongodbDatabases/mymongodb", "name": "mymongodb", "resourceGroup": "rg-cosmos-demo", "type": "Microsoft.DocumentDB/databaseAccounts/mongodbDatabases" ... } ``` #### Create a MongoDB collection Create a MongoDB collection inside the database with a shard key: ```bash az cosmosdb mongodb collection create \ --account-name mymongoaccount \ --resource-group rg-cosmos-demo \ --database-name mymongodb \ --name mycollection \ --shard /userId \ --throughput 400 ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-cosmos-demo/providers/Microsoft.DocumentDB/databaseAccounts/mymongoaccount/mongodbDatabases/mymongodb/collections/mycollection", "name": "mycollection", "resourceGroup": "rg-cosmos-demo", "type": "Microsoft.DocumentDB/databaseAccounts/mongodbDatabases/collections" ... } ``` #### Connect with the MongoDB client Extract the connection string and connect to the MongoDB database using `mongosh`: ```bash CONN_STR=$(az cosmosdb keys list \ --name mymongoaccount \ --resource-group rg-cosmos-demo \ --type connection-strings \ --query "connectionStrings[0].connectionString" \ --output tsv) mongosh "$CONN_STR" ``` ### Delete and verify Delete the Cosmos DB account and confirm it no longer appears in the list: ```bash az cosmosdb delete \ --name mycosmosaccount \ --resource-group rg-cosmos-demo \ --yes az cosmosdb list --resource-group rg-cosmos-demo ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-cosmos-demo/providers/Microsoft.DocumentDB/databaseAccounts/mymongoaccount", "kind": "MongoDB", "name": "mymongoaccount", "provisioningState": "Succeeded", "resourceGroup": "rg-cosmos-demo", "type": "Microsoft.DocumentDB/databaseAccounts" } ] ``` Delete the second Cosmos DB account configured for the MongoDB API: ```bash az cosmosdb delete \ --name mymongoaccount \ --resource-group rg-cosmos-demo \ --yes ``` Then list all Cosmos DB accounts to confirm the resource group is now empty: ```bash az cosmosdb list --resource-group rg-cosmos-demo ``` ```bash title="Output" [] ``` ## Features - **Account lifecycle:** Create, read, list, and delete Cosmos DB accounts for both SQL and MongoDB API kinds. - **Account key management:** Retrieve primary and secondary master keys and read-only keys. - **Connection strings:** Retrieve connection strings for the API for NoSQL (SQL) and for MongoDB via `az cosmosdb keys list --type connection-strings`. - **Name availability check:** Validate account name uniqueness. - **SQL databases:** Create, read, list, and delete SQL (NoSQL) databases within an account. - **SQL containers:** Create, read, list, and delete SQL containers with partition key configuration. - **SQL indexing policy:** Create containers with custom indexing policies. - **MongoDB databases:** Create, read, list, and delete MongoDB databases within an account. - **MongoDB collections:** Create, read, list, and delete MongoDB collections with shard key and throughput settings. - **Native MongoDB access:** Connect directly to the local MongoDB backend using a standard MongoDB client. ## Limitations - **PostgreSQL API not supported:** The Azure Cosmos DB for PostgreSQL (formerly Citus) API is not emulated. - **Table API not supported:** The Cosmos DB Table API is not emulated. - **Cassandra API not supported:** The Cosmos DB Cassandra API is not emulated. - **Gremlin API not supported:** The Cosmos DB Gremlin (graph) API is not emulated. - **Global distribution not emulated:** Multi-region replication and conflict resolution are accepted at the model level but not executed. - **Change feed and triggers:** Change feed processing and Cosmos DB triggers for Azure Functions are not emulated. - **Throughput and RU metering:** Request unit (RU) consumption is not tracked or enforced. - **RBAC for data plane:** Data plane RBAC (built-in Cosmos DB roles) is not enforced. ## Samples The following sample demonstrates how to use Azure Cosmos DB with LocalStack for Azure: - [Web App and Cosmos DB for MongoDB API ](https://github.com/localstack/localstack-azure-samples/samples/web-app-cosmosdb-mongodb-api/python/README.md) - [Web App and Cosmos DB for NoSQL API ](https://github.com/localstack/localstack-azure-samples/samples/web-app-cosmosdb-nosql-api/python/README.md) ## API Coverage # Data Collection Rules > Get started with Azure Monitor Data Collection Rules on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Monitor Data Collection Rules (DCR) define the data to collect, how to transform it, and where to send it. Data Collection Endpoints (DCE) expose the endpoints used for configuration access and for ingesting data in DCR-based pipelines, while Data Collection Rule Associations (DCRA) link a DCR to a specific monitored resource. Together, DCRs, DCEs, and DCRAs form the foundation of the Azure Monitor Logs ingestion pipeline. For more information, see [Data collection rules in Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/data-collection-rule-overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Monitor Data Collection Rules. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Data Collection Rules' integration with LocalStack. ## Getting started This guide walks you through creating a Data Collection Endpoint, a Data Collection Rule, and associating the rule with a virtual machine. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create --name rg-dcr-demo --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dcr-demo", "location": "westeurope", "name": "rg-dcr-demo", "properties": { "provisioningState": "Succeeded" }, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a data collection endpoint Create a data collection endpoint (DCE) to serve as the ingestion target: ```bash az monitor data-collection endpoint create \ --name my-dce \ --resource-group rg-dcr-demo \ --location westeurope \ --public-network-access Enabled ``` ```bash title="Output" { "configurationAccess": { "endpoint": "https://my-dce-a1b2.westeurope-1.control.monitor.azure.com" }, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dcr-demo/providers/Microsoft.Insights/dataCollectionEndpoints/my-dce", "immutableId": "dce-82f06343382a4872ba2270b5dba2eee7", "location": "westeurope", "logsIngestion": { "endpoint": "https://my-dce-a1b2.westeurope-1.ingest.monitor.azure.com" }, "name": "my-dce", "networkAcls": { "publicNetworkAccess": "Enabled" }, "provisioningState": "Succeeded", "resourceGroup": "rg-dcr-demo", "type": "Microsoft.Insights/dataCollectionEndpoints" ... } ``` ### Create a data collection rule Save the following JSON to `my-dcr.json`: ```json title="my-dcr.json" { "location": "westeurope", "properties": { "dataSources": { "performanceCounters": [ { "name": "perfCounterDataSource", "samplingFrequencyInSeconds": 60, "counterSpecifiers": ["\\Processor(_Total)\\% Processor Time"], "streams": ["Microsoft-Perf"] } ] }, "destinations": { "logAnalytics": [ { "workspaceResourceId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dcr-demo/providers/Microsoft.OperationalInsights/workspaces/my-workspace", "name": "myWorkspace" } ] }, "dataFlows": [ { "streams": ["Microsoft-Perf"], "destinations": ["myWorkspace"] } ] } } ``` Create the data collection rule from the configuration file: ```bash az monitor data-collection rule create \ --name my-dcr \ --resource-group rg-dcr-demo \ --location westeurope \ --rule-file my-dcr.json ``` ```bash title="Output" { "dataFlows": [ { "destinations": ["myWorkspace"], "streams": ["Microsoft-Perf"] } ], "dataSources": { "performanceCounters": [ { "counterSpecifiers": ["\\Processor(_Total)\\% Processor Time"], "name": "perfCounterDataSource", "samplingFrequencyInSeconds": 60, "streams": ["Microsoft-Perf"] } ] }, "destinations": { "logAnalytics": [ { "name": "myWorkspace", "workspaceResourceId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dcr-demo/providers/Microsoft.OperationalInsights/workspaces/my-workspace" } ] }, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dcr-demo/providers/Microsoft.Insights/dataCollectionRules/my-dcr", "immutableId": "dcr-d7f9c291105845b8b470d40631e5c883", "location": "westeurope", "name": "my-dcr", "provisioningState": "Succeeded", "resourceGroup": "rg-dcr-demo", "type": "Microsoft.Insights/dataCollectionRules" ... } ``` ### List data collection rules List the data collection rules in the resource group: ```bash az monitor data-collection rule list --resource-group rg-dcr-demo ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dcr-demo/providers/Microsoft.Insights/dataCollectionRules/my-dcr", "location": "westeurope", "name": "my-dcr", "provisioningState": "Succeeded", "resourceGroup": "rg-dcr-demo", "type": "Microsoft.Insights/dataCollectionRules", ... } ] ``` ### Create a data collection rule association Associate the rule with a virtual machine: ```bash VM_ID="/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dcr-demo/providers/Microsoft.Compute/virtualMachines/my-vm" az monitor data-collection rule association create \ --name my-dcra \ --resource "$VM_ID" \ --rule-id "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dcr-demo/providers/Microsoft.Insights/dataCollectionRules/my-dcr" ``` ```bash title="Output" { "dataCollectionRuleId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dcr-demo/providers/Microsoft.Insights/dataCollectionRules/my-dcr", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dcr-demo/providers/Microsoft.Compute/virtualMachines/my-vm/providers/Microsoft.Insights/dataCollectionRuleAssociations/my-dcra", "name": "my-dcra", "resourceGroup": "rg-dcr-demo", "type": "Microsoft.Insights/dataCollectionRuleAssociations" ... } ``` ### Delete and verify Delete the data collection rule and confirm it no longer appears in the list: ```bash az monitor data-collection rule delete \ --name my-dcr \ --resource-group rg-dcr-demo \ --yes ``` Then list data collection rules again to confirm none remain in the resource group: ```bash az monitor data-collection rule list --resource-group rg-dcr-demo ``` ```bash title="Output" [] ``` ## Features - **Data Collection Rule lifecycle:** Create, read, list, update, and delete DCRs. - **Data Collection Endpoint lifecycle:** Create, read, list, and delete DCEs. - **Data Collection Rule Association lifecycle:** Create, read, list, and delete DCRAs linking a DCR to a resource. - **Data source configuration:** Accept performance counter, Windows event log, Syslog, and custom log data sources. - **Destination configuration:** Accept Log Analytics workspace and storage account destinations. - **Data flow configuration:** Define stream-to-destination routing in the data flow section. ## Limitations - **No data ingestion:** Data sent to a DCE ingestion URL is not processed or stored. - **No transformation:** KQL-based data transformations defined in DCRs are not executed. - **No agent-managed collection:** The Azure Monitor Agent (AMA) interacting with LocalStack does not collect or forward real metrics or logs. ## Samples Explore end-to-end examples in the [LocalStack for Azure Samples](https://github.com/localstack/localstack-azure-samples) repository. ## API Coverage # Database for PostgreSQL Flexible Server > Get started with Azure Database for PostgreSQL flexible server on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Database for PostgreSQL flexible server is a fully managed relational database service that provides granular control over database configuration and tuning. It supports PostgreSQL community versions and offers built-in high availability, intelligent performance monitoring, and flexible scaling across Burstable, General Purpose, and Memory Optimized compute tiers. Common use cases include web applications, microservices backends, and analytics workloads that benefit from PostgreSQL's extensibility and standards compliance. For more information, see [What is Azure Database for PostgreSQL - Flexible Server?](https://learn.microsoft.com/azure/postgresql/flexible-server/overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure DB for PostgreSQL. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of DB for PostgreSQL integration with LocalStack. ## Getting started This guide is designed for users new to Azure DB for PostgreSQL and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create \ --name rg-postgres-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-postgres-demo", "location": "westeurope", "name": "rg-postgres-demo", "properties": { "provisioningState": "Succeeded" }, ... } ``` ### Create and inspect a PostgreSQL flexible server Create a Burstable-tier PostgreSQL 16 flexible server with 32 GB of storage: ```bash az postgres flexible-server create \ --name postgres-demo \ --resource-group rg-postgres-demo \ --location westeurope \ --admin-user pgadmin \ --admin-password 'P@ssw0rd2024!' \ --sku-name Standard_B1ms \ --tier Burstable \ --version 16 \ --storage-size 32 \ --yes ``` ```bash title="Output" { "connectionString": "postgresql://pgadmin:P%40ssw0rd2024%21@postgres-demo.postgres.database.azure.com/postgres?sslmode=require", "host": "postgres-demo.postgres.database.azure.com", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-postgres-demo/providers/Microsoft.DBforPostgreSQL/flexibleServers/postgres-demo", "location": "westeurope", "password": "P@ssw0rd2024!", "resourceGroup": "rg-postgres-demo", "skuname": "Standard_B1ms", "username": "pgadmin", "version": "16" } ``` The command waits for the server to reach the `Ready` state before returning. ### Show and list flexible servers Retrieve the details of the flexible server: ```bash az postgres flexible-server show \ --name postgres-demo \ --resource-group rg-postgres-demo ``` ```bash title="Output" { "administratorLogin": "pgadmin", "authConfig": { "activeDirectoryAuth": "Disabled", "passwordAuth": "Enabled", "tenantId": null }, "backup": { "backupRetentionDays": 7, "geoRedundantBackup": "Disabled" }, "fullyQualifiedDomainName": "postgres-demo.postgres.database.azure.com", "highAvailability": { "mode": "Disabled", "state": "NotEnabled" }, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-postgres-demo/providers/Microsoft.DBforPostgreSQL/flexibleServers/postgres-demo", "location": "westeurope", "name": "postgres-demo", "network": { "publicNetworkAccess": "Enabled" }, "resourceGroup": "rg-postgres-demo", "sku": { "name": "Standard_B1ms", "tier": "Burstable" }, "state": "Ready", "storage": { "autoGrow": "Disabled", "storageSizeGb": 32, "tier": "P4", "type": "Premium_LRS" }, "type": "Microsoft.DBforPostgreSQL/flexibleServers", "version": "16", ... } ``` Then list all flexible servers in the resource group: ```bash az postgres flexible-server list \ --resource-group rg-postgres-demo ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-postgres-demo/providers/Microsoft.DBforPostgreSQL/flexibleServers/postgres-demo", "location": "westeurope", "name": "postgres-demo", "sku": { "name": "Standard_B1ms", "tier": "Burstable" }, "state": "Ready", "version": "16", ... } ] ``` ### Update a flexible server Update the server SKU and storage size: ```bash az postgres flexible-server update \ --name postgres-demo \ --resource-group rg-postgres-demo \ --sku-name Standard_B2s \ --tier Burstable \ --storage-size 64 ``` ```bash title="Output" { "administratorLogin": "pgadmin", "authConfig": { "activeDirectoryAuth": "Disabled", "passwordAuth": "Enabled", "tenantId": null }, "backup": { "backupRetentionDays": 7, "geoRedundantBackup": "Disabled" }, "fullyQualifiedDomainName": "postgres-demo.postgres.database.azure.com", "highAvailability": { "mode": "Disabled", "state": "NotEnabled" }, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-postgres-demo/providers/Microsoft.DBforPostgreSQL/flexibleServers/postgres-demo", "location": "westeurope", "name": "postgres-demo", "network": { "publicNetworkAccess": "Enabled" }, "resourceGroup": "rg-postgres-demo", "sku": { "name": "Standard_B2s", "tier": "Burstable" }, "state": "Ready", "storage": { "autoGrow": "Disabled", "storageSizeGb": 64, "tier": "P6", "type": "Premium_LRS" }, "type": "Microsoft.DBforPostgreSQL/flexibleServers", "version": "16", ... } ``` ### Create a database Create a database on the flexible server: ```bash az postgres flexible-server db create \ --server-name postgres-demo \ --resource-group rg-postgres-demo \ --database-name sampledb \ --charset UTF8 \ --collation en_US.utf8 ``` ```bash title="Output" { "charset": "UTF8", "collation": "en_US.utf8", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-postgres-demo/providers/Microsoft.DBforPostgreSQL/flexibleServers/postgres-demo/databases/sampledb", "name": "sampledb", "resourceGroup": "rg-postgres-demo", "type": "Microsoft.DBforPostgreSQL/flexibleServers/databases" } ``` Get and list databases: ```bash az postgres flexible-server db show \ --resource-group rg-postgres-demo \ --server-name postgres-demo \ --database-name sampledb az postgres flexible-server db list \ --resource-group rg-postgres-demo \ --server-name postgres-demo ``` ```bash title="Output" { "name": "sampledb", "charset": "utf8", "collation": "en_US.utf8", ... } [ { "name": "postgres", ... }, { "name": "azure_sys", ... }, { "name": "azure_maintenance", ... }, { "name": "sampledb", ... } ] ``` ### Create and inspect a firewall rule Create a firewall rule to allow connections from a specific IP address: ```bash az postgres flexible-server firewall-rule create \ --resource-group rg-postgres-demo \ --name postgres-demo \ --rule-name allow-myip \ --start-ip-address 203.0.113.10 \ --end-ip-address 203.0.113.10 ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-postgres-demo/providers/Microsoft.DBforPostgreSQL/flexibleServers/postgres-demo/firewallRules/allow-myip", "name": "allow-myip", "resourceGroup": "rg-postgres-demo", "startIpAddress": "203.0.113.10", "endIpAddress": "203.0.113.10", "type": "Microsoft.DBforPostgreSQL/flexibleServers/firewallRules" } ``` Get and list firewall rules: ```bash az postgres flexible-server firewall-rule show \ --resource-group rg-postgres-demo \ --name postgres-demo \ --rule-name allow-myip az postgres flexible-server firewall-rule list \ --resource-group rg-postgres-demo \ --name postgres-demo ``` ```bash title="Output" { "name": "allow-myip", "startIpAddress": "203.0.113.10", "endIpAddress": "203.0.113.10", ... } [ { "name": "allow-myip", "startIpAddress": "203.0.113.10", "endIpAddress": "203.0.113.10", ... } ] ``` ### View and update server configuration View the current value of the `max_connections` configuration parameter: ```bash az postgres flexible-server parameter show \ --resource-group rg-postgres-demo \ --server-name postgres-demo \ --name max_connections ``` ```bash title="Output" { "allowedValues": "25-5000", "dataType": "Integer", "defaultValue": "100", "description": "Sets the maximum number of concurrent connections.", "documentationLink": "https://www.postgresql.org/docs/16/runtime-config-connection.html#GUC-MAX-CONNECTIONS", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-postgres-demo/providers/Microsoft.DBforPostgreSQL/flexibleServers/postgres-demo/configurations/max_connections", "isConfigPendingRestart": false, "isDynamicConfig": false, "isReadOnly": false, "name": "max_connections", "resourceGroup": "rg-postgres-demo", "source": "system-default", "type": "Microsoft.DBforPostgreSQL/flexibleServers/configurations", "unit": "", "value": "100" } ``` Update `max_connections` to 200: ```bash az postgres flexible-server parameter set \ --resource-group rg-postgres-demo \ --server-name postgres-demo \ --name max_connections \ --value 200 \ --source user-override ``` ```bash title="Output" { "allowedValues": "25-5000", "dataType": "Integer", "defaultValue": "100", "description": "Sets the maximum number of concurrent connections.", "documentationLink": "https://www.postgresql.org/docs/16/runtime-config-connection.html#GUC-MAX-CONNECTIONS", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-postgres-demo/providers/Microsoft.DBforPostgreSQL/flexibleServers/postgres-demo/configurations/max_connections", "isConfigPendingRestart": true, "isDynamicConfig": false, "isReadOnly": false, "name": "max_connections", "resourceGroup": "rg-postgres-demo", "source": "user-override", "type": "Microsoft.DBforPostgreSQL/flexibleServers/configurations", "unit": "", "value": "200" } ``` Non-dynamic parameters such as `max_connections` set `isConfigPendingRestart` to `true` after an update. A server restart applies the change. ### Stop, start, and restart a server Stop the flexible server: ```bash az postgres flexible-server stop \ --resource-group rg-postgres-demo \ --name postgres-demo ``` ```bash title="Output" Server will be automatically started after 7 days if you do not perform a manual start operation ``` Verify the server state is `Stopped`: ```bash az postgres flexible-server show \ --name postgres-demo \ --resource-group rg-postgres-demo \ --query state \ --output tsv ``` ```bash title="Output" Stopped ``` Start the server again: ```bash az postgres flexible-server start \ --resource-group rg-postgres-demo \ --name postgres-demo ``` Restart the server to apply pending configuration changes: ```bash az postgres flexible-server restart \ --resource-group rg-postgres-demo \ --name postgres-demo ``` ### Delete and verify Delete the flexible server: ```bash az postgres flexible-server delete \ --resource-group rg-postgres-demo \ --name postgres-demo \ --yes ``` Then list all flexible servers to confirm the resource group is now empty: ```bash az postgres flexible-server list \ --resource-group rg-postgres-demo ``` ```bash title="Output" [] ``` ## Features The PostgreSQL Flexible Server emulator supports the following features: - **Server lifecycle management**: Create, get, update, list, and delete flexible servers with full ARM resource model support. - **Stop, start, and restart**: Transition servers between Ready and Stopped states with proper state machine enforcement. - **Database management**: Create, get, list, and delete user databases. Each database is backed by a real PostgreSQL instance. - **Firewall rules**: Create, get, update, list, and delete IP-based firewall rules with address range validation. - **Server configuration**: Get, list, and update over 14 PostgreSQL configuration parameters with data-type validation and restart-pending tracking. - **SKU tiers**: Burstable, General Purpose, and Memory Optimized tiers with multiple SKU sizes per tier. - **PostgreSQL versions**: Versions 13, 14, 15, 16, and 17 with in-place major version upgrade support. - **Storage management**: Configurable storage from 32 GB to 16 TB with automatic tier mapping and storage auto-grow support. - **Name availability check**: Validate server name uniqueness across subscriptions before creation. - **Location capabilities**: Query available SKUs, storage sizes, and supported PostgreSQL versions per region. - **Long-running operations**: Server create, delete, start, stop, and restart return `202 Accepted` with async operation tracking headers. - **Bicep and Terraform support**: Deploy flexible servers, databases, and firewall rules using infrastructure-as-code templates. ## Limitations - **Backup and restore**: Backup retention days and geo-redundant backup settings are stored but no actual backups are performed. Point-in-time restore is not supported. - **Read replicas**: Replica properties are returned in the server response but replica creation and management are not implemented. - **Microsoft Entra authentication**: The `authConfig.activeDirectoryAuth` property is stored but Active Directory authentication is not enforced. - **Virtual network integration**: Delegated subnet and private DNS zone properties are accepted but VNet injection is not performed. - **Private endpoint connections**: The property is returned in the response but private endpoint connectivity is not implemented. - **High availability failover**: High availability mode and standby availability zone are stored but no failover mechanism is active. - **Firewall rule enforcement**: Firewall rules are stored and queryable but are not enforced on actual database connections. - **Storage auto-grow enforcement**: The auto-grow setting is stored but automatic storage expansion does not occur. ## Samples Explore end-to-end examples in the [LocalStack for Azure Samples](https://github.com/localstack/localstack-azure-samples) repository. ## API Coverage # Diagnostic Setting > Get started with Azure Monitor Diagnostic Settings on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Monitor Diagnostic Settings configure where a resource sends its platform logs and metrics. Supported destinations include Log Analytics Workspaces, Storage Accounts, Event Hubs, and partner solutions. Diagnostic settings are commonly used to enable centralized log collection and compliance auditing across Azure deployments. For more information, see [Diagnostic settings in Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/diagnostic-settings). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Monitor Diagnostic Settings. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Diagnostic Settings' integration with LocalStack. ## Getting started This guide uses the existing `monitor.mdx` article workflow as a reference. See also the [Monitor](/azure/services/monitor) page for a broader overview of diagnostic settings alongside activity log examples. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create --name rg-diag-demo --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-diag-demo", "location": "westeurope", "name": "rg-diag-demo", "properties": { "provisioningState": "Succeeded" }, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a storage account as the destination Create a storage account to serve as the export destination for the logs and metrics: ```bash az storage account create \ --name sadiagdemo \ --resource-group rg-diag-demo \ --location westeurope \ --sku Standard_LRS ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-diag-demo/providers/Microsoft.Storage/storageAccounts/sadiagdemo", "kind": "StorageV2", "location": "westeurope", "name": "sadiagdemo", "resourceGroup": "rg-diag-demo", "sku": { "name": "Standard_LRS", "tier": "Standard" }, "type": "Microsoft.Storage/storageAccounts" ... } ``` ### Create a diagnostic setting on a resource The following example creates a diagnostic setting on a storage account’s default blob service (`Microsoft.Storage/storageAccounts/.../blobServices/default`) that enables the `StorageRead` resource log category and the `Transaction` metric category for export, with a storage account as the destination. Those categories are defined for blob services in Azure’s [Blob Storage monitoring data reference](https://learn.microsoft.com/en-us/azure/storage/blobs/monitor-blob-storage-reference#resource-logs). :::note In Azure, you must not use the **same** storage account as both the monitored blob service and the diagnostic setting’s storage destination—doing so would create recursive logging. That restriction is described under *Destination limitations* in [Monitor Azure Blob Storage](https://learn.microsoft.com/en-us/azure/storage/blobs/monitor-blob-storage). The commands below use one account for brevity; because LocalStack does not route or ingest logs for this feature (see [Limitations](#limitations)), this does not imply real Azure behavior. ::: ```bash RESOURCE_ID="/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-diag-demo/providers/Microsoft.Storage/storageAccounts/sadiagdemo/blobServices/default" DEST_ID="/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-diag-demo/providers/Microsoft.Storage/storageAccounts/sadiagdemo" az monitor diagnostic-settings create \ --name my-diag-setting \ --resource "$RESOURCE_ID" \ --storage-account "$DEST_ID" \ --logs '[{"category": "StorageRead", "enabled": true}]' \ --metrics '[{"category": "Transaction", "enabled": true}]' ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-diag-demo/providers/Microsoft.Storage/storageAccounts/sadiagdemo/blobServices/default/providers/microsoft.insights/diagnosticSettings/my-diag-setting", "logs": [ { "category": "StorageRead", "enabled": true } ], "metrics": [ { "category": "Transaction", "enabled": true } ], "name": "my-diag-setting", "storageAccountId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-diag-demo/providers/Microsoft.Storage/storageAccounts/sadiagdemo", "type": "Microsoft.Insights/diagnosticSettings" ... } ``` ### List diagnostic settings List all diagnostic settings attached to the target resource: ```bash az monitor diagnostic-settings list --resource "$RESOURCE_ID" ``` ```bash title="Output" { "value": [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-diag-demo/providers/Microsoft.Storage/storageAccounts/sadiagdemo/blobServices/default/providers/microsoft.insights/diagnosticSettings/my-diag-setting", "logs": [ { "category": "StorageRead", "enabled": true } ], "metrics": [ { "category": "Transaction", "enabled": true } ], "name": "my-diag-setting", "storageAccountId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-diag-demo/providers/Microsoft.Storage/storageAccounts/sadiagdemo", "type": "Microsoft.Insights/diagnosticSettings" } ] ... } ``` ### Show a diagnostic setting Retrieve the full configuration of the diagnostic setting: ```bash az monitor diagnostic-settings show \ --name my-diag-setting \ --resource "$RESOURCE_ID" ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-diag-demo/providers/Microsoft.Storage/storageAccounts/sadiagdemo/blobServices/default/providers/microsoft.insights/diagnosticSettings/my-diag-setting", "logs": [ { "category": "StorageRead", "enabled": true } ], "metrics": [ { "category": "Transaction", "enabled": true } ], "name": "my-diag-setting", "storageAccountId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-diag-demo/providers/Microsoft.Storage/storageAccounts/sadiagdemo", "type": "Microsoft.Insights/diagnosticSettings" ... } ``` ### Update a diagnostic setting Update the diagnostic setting to disable the `StorageRead` log category while leaving metrics and the storage destination unchanged: ```bash az monitor diagnostic-settings update \ --name my-diag-setting \ --resource "$RESOURCE_ID" \ --logs '[{"category": "StorageRead", "enabled": false}]' ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-diag-demo/providers/Microsoft.Storage/storageAccounts/sadiagdemo/blobServices/default/providers/microsoft.insights/diagnosticSettings/my-diag-setting", "logs": [ { "category": "StorageRead", "enabled": false } ], "metrics": [ { "category": "Transaction", "enabled": true } ], "name": "my-diag-setting", "storageAccountId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-diag-demo/providers/Microsoft.Storage/storageAccounts/sadiagdemo", "type": "Microsoft.Insights/diagnosticSettings" ... } ``` ### Delete and verify Delete the diagnostic setting: ```bash az monitor diagnostic-settings delete \ --name my-diag-setting \ --resource "$RESOURCE_ID" ``` Then list diagnostic settings again to confirm the setting was removed: ```bash az monitor diagnostic-settings list --resource "$RESOURCE_ID" ``` ```bash title="Output" { "value": [] } ``` ## Features - **Diagnostic setting lifecycle:** Create, read, list, update, and delete diagnostic settings on any resource. - **Multiple destinations:** Accept Storage Account, Log Analytics Workspace, and Event Hub as destinations. - **Log category configuration:** Enable or disable individual log categories per setting. - **Metric category configuration:** Enable or disable individual metric categories per setting. - **Retention policy support:** Accept retention policy fields in log and metric settings (some Azure resource and destination combinations disallow retention on the diagnostic setting itself—see [Monitor Azure Blob Storage](https://learn.microsoft.com/en-us/azure/storage/blobs/monitor-blob-storage)). - **Resource-scoped settings:** Settings are scoped to a specific resource (by resource ID). ## Limitations - **No data routing:** Logs and metrics are not routed to the configured Storage Account, Log Analytics Workspace, or Event Hub. The setting is stored in the emulator only. - **No log ingestion:** Platform logs emitted by Azure services within LocalStack are not captured or forwarded. - **No subscription diagnostic settings:** The subscription-level diagnostic settings API (`/subscriptions/{subscriptionId}/providers/Microsoft.Insights/diagnosticSettings`) described in [Subscription Diagnostic Settings](https://learn.microsoft.com/en-us/rest/api/monitor/subscription-diagnostic-settings) is not currently supported. ## Samples The following sample demonstrates how to use Azure Diagnostic Settings with LocalStack for Azure: - [Function App and Service Bus](https://github.com/localstack/localstack-azure-samples/samples/function-app-service-bus/dotnet/README.md) - [Web App and Cosmos DB for MongoDB API ](https://github.com/localstack/localstack-azure-samples/samples/web-app-cosmosdb-mongodb-api/python/README.md) ## API Coverage # Event Grid > API coverage for Microsoft.EventGrid in LocalStack for Azure. import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## API Coverage # Event Grid Data Plane > API coverage for Microsoft.EventGrid.DataPlane in LocalStack for Azure. import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## API Coverage # Event Hubs > Get started with Azure Event Hubs on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Event Hubs is a real-time data streaming platform that ingests and processes millions of events per second. Producers append events to partitioned logs, and consumers read those logs independently at their own pace, which makes Event Hubs a common backbone for telemetry pipelines, event sourcing, and stream analytics. It exposes its data plane over AMQP 1.0, HTTPS, and a Kafka-compatible endpoint. For more information, see [What is Azure Event Hubs?](https://learn.microsoft.com/azure/event-hubs/event-hubs-about) LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Event Hubs. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Event Hubs' integration with LocalStack. ## Getting started This guide is designed for users new to Event Hubs and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to contain your Event Hubs resources: ```bash az group create \ --name rg-eventhubs-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-eventhubs-demo", "location": "westeurope", "managedBy": null, "name": "rg-eventhubs-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create an Event Hubs namespace Create an Event Hubs namespace in the resource group: ```bash az eventhubs namespace create \ --resource-group rg-eventhubs-demo \ --name ehnsdoc42 \ --location westeurope \ --sku Standard ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-eventhubs-demo/providers/Microsoft.EventHub/namespaces/ehnsdoc42", "name": "ehnsdoc42", "location": "westeurope", "kafkaEnabled": true, "provisioningState": "Succeeded", "serviceBusEndpoint": "https://ehnsdoc42.servicebus.azure.localhost.localstack.cloud:4511", "sku": { "capacity": 1, "name": "Standard", "tier": "Standard" }, "status": "Active", ... } ``` :::note The namespace endpoint embeds a dynamic port allocated by the emulator, so the port in your output may differ from the one shown here. That endpoint serves AMQP, AMQP-over-WebSockets, and the Kafka protocol on the same port. ::: ### Create an event hub Create an event hub in the namespace: ```bash az eventhubs eventhub create \ --resource-group rg-eventhubs-demo \ --namespace-name ehnsdoc42 \ --name orders-hub \ --partition-count 4 \ --cleanup-policy Delete \ --retention-time 168 ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-eventhubs-demo/providers/Microsoft.EventHub/namespaces/ehnsdoc42/eventhubs/orders-hub", "name": "orders-hub", "location": "westeurope", "messageRetentionInDays": 7, "partitionCount": 4, "partitionIds": [ "0", "1", "2", "3" ], "retentionDescription": { "cleanupPolicy": "Delete", "retentionTimeInHours": 168 }, "status": "Active", "type": "Microsoft.EventHub/namespaces/eventhubs", ... } ``` :::note As in Azure, the partition count cannot be changed after the event hub is created, and tier limits are enforced: retention is capped at 1 day on Basic and 7 days on Standard, and each tier limits how many event hubs a namespace can hold. ::: ### Create a consumer group Create a consumer group for the event hub: ```bash az eventhubs eventhub consumer-group create \ --resource-group rg-eventhubs-demo \ --namespace-name ehnsdoc42 \ --eventhub-name orders-hub \ --name analytics ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-eventhubs-demo/providers/Microsoft.EventHub/namespaces/ehnsdoc42/eventhubs/orders-hub/consumergroups/analytics", "name": "analytics", "type": "Microsoft.EventHub/namespaces/eventhubs/consumergroups", ... } ``` List the consumer groups of the event hub: ```bash az eventhubs eventhub consumer-group list \ --resource-group rg-eventhubs-demo \ --namespace-name ehnsdoc42 \ --eventhub-name orders-hub \ --query "[].name" -o json ``` ```bash title="Output" [ "$Default", "analytics" ] ``` :::note The `$Default` consumer group is created automatically with the event hub and cannot be deleted, matching Azure. On the Basic tier, custom consumer groups are rejected as in Azure. ::: ### Manage authorization rules and connection strings Create a send-only authorization rule on the event hub: ```bash az eventhubs eventhub authorization-rule create \ --resource-group rg-eventhubs-demo \ --namespace-name ehnsdoc42 \ --eventhub-name orders-hub \ --name send-policy \ --rights Send ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-eventhubs-demo/providers/Microsoft.EventHub/namespaces/ehnsdoc42/eventhubs/orders-hub/authorizationRules/send-policy", "name": "send-policy", "rights": [ "Send" ], "type": "Microsoft.EventHub/namespaces/eventhubs/authorizationrules" } ``` List the connection strings for the rule: ```bash az eventhubs eventhub authorization-rule keys list \ --resource-group rg-eventhubs-demo \ --namespace-name ehnsdoc42 \ --eventhub-name orders-hub \ --name send-policy ``` ```bash title="Output" { "keyName": "send-policy", "primaryConnectionString": "Endpoint=sb://ehnsdoc42.servicebus.azure.localhost.localstack.cloud:4511/;SharedAccessKeyName=send-policy;SharedAccessKey=...;UseDevelopmentEmulator=true;EntityPath=orders-hub", "primaryKey": "...", "secondaryConnectionString": "Endpoint=sb://ehnsdoc42.servicebus.azure.localhost.localstack.cloud:4511/;SharedAccessKeyName=send-policy;SharedAccessKey=...;UseDevelopmentEmulator=true;EntityPath=orders-hub", "secondaryKey": "..." } ``` The namespace-level `RootManageSharedAccessKey` rule is created automatically with the namespace: ```bash az eventhubs namespace authorization-rule keys list \ --resource-group rg-eventhubs-demo \ --namespace-name ehnsdoc42 \ --name RootManageSharedAccessKey ``` ```bash title="Output" { "keyName": "RootManageSharedAccessKey", "primaryConnectionString": "Endpoint=sb://ehnsdoc42.servicebus.azure.localhost.localstack.cloud:4511/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=...;UseDevelopmentEmulator=true", ... } ``` :::note Use the connection string exactly as returned. The endpoint carries the emulator's dynamic port, and `UseDevelopmentEmulator=true` stops the Azure SDKs from forcing TLS on port 5671, so the official `azure-eventhub` SDKs work against the emulator without any code changes. ::: ### Send events over HTTP The Event Hubs runtime REST API accepts events on the namespace endpoint. Send a single event to the event hub: ```bash curl -sk -i -X POST \ "https://ehnsdoc42.servicebus.azure.localhost.localstack.cloud:4566/orders-hub/messages" \ -H "Authorization: SharedAccessSignature sr=...&sig=...&se=...&skn=RootManageSharedAccessKey" \ -H "Content-Type: application/json" \ -d '{"orderId": 1001, "status": "created"}' ``` ```bash title="Output" HTTP/2 201 server: TwistedWeb/26.4.0 content-type: application/xml; charset=utf-8 content-length: 0 ``` Route an event to a specific partition by partition key using the `BrokerProperties` header: ```bash curl -sk -i -X POST \ "https://ehnsdoc42.servicebus.azure.localhost.localstack.cloud:4566/orders-hub/messages" \ -H "Authorization: SharedAccessSignature sr=...&sig=...&se=...&skn=RootManageSharedAccessKey" \ -H 'BrokerProperties: {"PartitionKey": "customer-42"}' \ -H "Content-Type: application/json" \ -d '{"orderId": 1002, "status": "created"}' ``` ```bash title="Output" HTTP/2 201 server: TwistedWeb/26.4.0 content-type: application/xml; charset=utf-8 content-length: 0 ``` :::note The `Authorization` header is required, but the emulator does not verify the SAS signature itself, so any well-formed value is accepted. The HTTP API is send-only, as in Azure; to receive events, use the `azure-eventhub` SDKs or the Kafka endpoint with the connection string from the previous step. ::: ### Capture events to Blob Storage Event Hubs Capture archives the event stream into a storage account as Avro files. Create a storage account for the archives: ```bash az storage account create \ --resource-group rg-eventhubs-demo \ --name stehcapturels \ --location westeurope \ --sku Standard_LRS ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-eventhubs-demo/providers/Microsoft.Storage/storageAccounts/stehcapturels", "name": "stehcapturels", "kind": "StorageV2", "location": "westeurope", "primaryEndpoints": { "blob": "https://stehcapturels.blob.core.azure.localhost.localstack.cloud:4566", ... }, "provisioningState": "Succeeded", ... } ``` Enable Capture on the event hub: ```bash STORAGE_ID=$(az storage account show \ --resource-group rg-eventhubs-demo \ --name stehcapturels \ --query id -o tsv) az eventhubs eventhub update \ --resource-group rg-eventhubs-demo \ --namespace-name ehnsdoc42 \ --name orders-hub \ --enable-capture true \ --capture-interval 60 \ --capture-size-limit 10485760 \ --destination-name EventHubArchive.AzureBlockBlob \ --storage-account "$STORAGE_ID" \ --blob-container eventhub-capture \ --skip-empty-archives true ``` ```bash title="Output" { "captureDescription": { "destination": { "blobContainer": "eventhub-capture", "name": "EventHubArchive.AzureBlockBlob", "storageAccountResourceId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-eventhubs-demo/providers/Microsoft.Storage/storageAccounts/stehcapturels" }, "enabled": true, "encoding": "Avro", "intervalInSeconds": 60, "sizeLimitInBytes": 10485760, "skipEmptyArchives": true }, "name": "orders-hub", ... } ``` Send a few more events, wait for the capture window to elapse (60 seconds here), and list the archives: ```bash STORAGE_CONNECTION=$(az storage account show-connection-string \ --resource-group rg-eventhubs-demo \ --name stehcapturels \ --query connectionString -o tsv) az storage blob list \ --container-name eventhub-capture \ --connection-string "$STORAGE_CONNECTION" \ --query "[].{name:name, bytes:properties.contentLength}" -o json ``` ```bash title="Output" [ { "bytes": 605, "name": "ehnsdoc42/orders-hub/0/2026/08/09/19/23/32.avro" }, { "bytes": 674, "name": "ehnsdoc42/orders-hub/1/2026/08/09/19/23/32.avro" }, { "bytes": 605, "name": "ehnsdoc42/orders-hub/2/2026/08/09/19/23/32.avro" }, { "bytes": 605, "name": "ehnsdoc42/orders-hub/3/2026/08/09/19/23/32.avro" } ] ``` The archives are Avro object-container files in Azure's `EventData` record schema, one per partition and capture window. After each archive is written, the emulator raises the `Microsoft.EventHub.CaptureFileCreated` event through [Event Grid](/azure/services/event-grid/) system topics, so downstream automation such as an Azure Function can react to new archives. ## Features The emulator includes the following core capabilities: - **Multi-protocol data plane on a single event log**: AMQP 1.0 (what the official `azure-eventhub` SDKs speak, including AMQP-over-WebSockets), a Kafka-compatible endpoint, the HTTP runtime API, and the legacy Atom-XML management API all read and write the same per-partition log, so an event sent over one protocol is readable over the others. - **ARM control plane**: CRUD for namespaces, event hubs, consumer groups, authorization rules and keys, network rule sets, schema groups, application groups, geo-disaster-recovery configurations, and dedicated clusters — with tier quotas enforced as in Azure (event hub counts, retention caps, partition limits, and the Basic-tier consumer-group restriction). - **Event Hubs Capture**: per-partition Avro archives written to emulated Blob Storage on time- or size-based windows, including `skipEmptyArchives`, and the `Microsoft.EventHub.CaptureFileCreated` system event raised through Event Grid. - **Schema Registry**: the data-plane API served on the namespace endpoint, with schema group CRUD, schema registration and versioning, lookup by content, and the tier quota on schema groups. - **Checkpointing**: the SDK's event processor pattern works with the blob-backed checkpoint store against emulated Blob Storage. - **Kafka-compatible endpoint**: topics map to event hubs, Kafka offsets equal Event Hubs sequence numbers, and consumer-group offsets are stored broker-side; verified with `kafka-python` and `confluent-kafka`. - **Emulator-ready connection strings**: `keys list` returns connection strings that the Azure SDKs, Kafka clients, and IaC tools can use verbatim against the emulator. ## Limitations The current version of the emulator does **not** support the following: - **Credential verification**: SAS tokens and JWTs are checked for the existence of the target entity, but signatures are not cryptographically verified. The Kafka endpoint listens in plaintext without SASL, whereas real Azure requires `SASL_SSL`. - **Azure's partition-key hash**: the partition-key-to-partition mapping is a stable hash, so events with the same key land on the same partition, but the concrete partition ids differ from real Azure. - **Kafka namespace isolation**: Kafka topics resolve across all Kafka-enabled namespaces, and Kafka consumer-group offsets and producer ids are held in memory only. - **Enforcement of throughput and network controls**: throughput-unit throttling, application-group policies, network rule sets, private endpoints, and geo-disaster-recovery are stored and echoed as metadata, but not enforced — a geo-DR failover changes state without moving events. - **Dedicated cluster hardware**: clusters are metadata-level, and Azure's 4-hour cluster deletion moratorium is intentionally skipped so local clean-up is immediate. - **Receiving over HTTP**: the HTTP runtime API is send-only, as in Azure; receiving requires AMQP or Kafka. - **Persistence**: entities and event data are held in memory and are lost when the emulator restarts. Recreate resources on startup via the CLI or IaC. ## Samples Explore the following samples to get started with Event Hubs on LocalStack: - [Payment fraud detection pipeline with Event Hubs, Functions, and Capture](https://github.com/localstack/localstack-azure-samples/tree/main/samples/eventhubs/python) - [Cold-path automation with Event Hubs Capture, Event Grid, and Functions](https://github.com/localstack/localstack-azure-samples/tree/main/samples/eventhubs-eventgrid/python) ## API Coverage # Front Door > Get started with Azure Front Door on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Front Door is a global content delivery network and load balancer that routes client traffic to the fastest available origin using Microsoft's global network. It combines HTTP load balancing, SSL offloading, URL-based routing, and traffic acceleration into a single entry point for your web applications. Front Door Standard and Premium profiles are configured through the `Microsoft.Cdn` resource provider and managed using the `az afd` CLI command group. For more information, see [What is Azure Front Door?](https://learn.microsoft.com/en-us/azure/frontdoor/front-door-overview). LocalStack for Azure provides a local environment for building and testing applications that use Azure Front Door. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Front Door's integration with LocalStack. ## Getting started This guide is designed for users new to Azure Front Door and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold your Front Door resources: ```bash az group create \ --name rg-afd-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-afd-demo", "location": "westeurope", "name": "rg-afd-demo", "properties": { "provisioningState": "Succeeded" }, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a Front Door profile Create a Front Door Standard profile to serve as the top-level container for all Front Door resources: ```bash az afd profile create \ --resource-group rg-afd-demo \ --profile-name afd-demo \ --sku Standard_AzureFrontDoor ``` ```bash title="Output" { "frontDoorId": "a3664ef5-d8cb-4930-b7f8-8a762d24acf9", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-afd-demo/providers/Microsoft.Cdn/profiles/afd-demo", "kind": "frontdoor", "location": "Global", "name": "afd-demo", "originResponseTimeoutSeconds": 30, "provisioningState": "Succeeded", "resourceGroup": "rg-afd-demo", "resourceState": "Active", "sku": { "name": "Standard_AzureFrontDoor" }, "tags": {}, "type": "Microsoft.Cdn/profiles" } ``` ### Create an endpoint Create an endpoint to expose a publicly accessible hostname for your Front Door profile: ```bash az afd endpoint create \ --resource-group rg-afd-demo \ --profile-name afd-demo \ --endpoint-name my-endpoint \ --enabled-state Enabled ``` ```bash title="Output" { "deploymentStatus": "Succeeded", "enabledState": "Enabled", "hostName": "my-endpoint-08621c35aa0147f0.z01.azurefd.net", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-afd-demo/providers/Microsoft.Cdn/profiles/afd-demo/afdEndpoints/my-endpoint", "location": "Global", "name": "my-endpoint", "provisioningState": "Succeeded", "resourceGroup": "rg-afd-demo", "tags": {}, "type": "Microsoft.Cdn/profiles/afdEndpoints" } ``` The `hostName` field contains the generated `*.azurefd.net` domain assigned to this endpoint. ### Create an origin group Create an origin group to define the set of backend servers that Front Door will load-balance across, including a health probe configuration: ```bash az afd origin-group create \ --resource-group rg-afd-demo \ --profile-name afd-demo \ --origin-group-name my-origin-group \ --probe-path "/health" \ --probe-request-type GET \ --probe-protocol Http \ --probe-interval-in-seconds 120 \ --sample-size 2 \ --successful-samples-required 2 ``` ```bash title="Output" { "deploymentStatus": "Succeeded", "healthProbeSettings": { "probeIntervalInSeconds": 120, "probePath": "/health", "probeProtocol": "Http", "probeRequestType": "GET" }, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-afd-demo/providers/Microsoft.Cdn/profiles/afd-demo/originGroups/my-origin-group", "loadBalancingSettings": { "additionalLatencyInMilliseconds": 0, "sampleSize": 2, "successfulSamplesRequired": 2 }, "name": "my-origin-group", "provisioningState": "Succeeded", "resourceGroup": "rg-afd-demo", "sessionAffinityState": "Disabled", "type": "Microsoft.Cdn/profiles/originGroups" } ``` ### Add an origin Add an origin to the origin group to specify the backend host that Front Door will forward traffic to: ```bash az afd origin create \ --resource-group rg-afd-demo \ --profile-name afd-demo \ --origin-group-name my-origin-group \ --origin-name my-origin \ --host-name example.com \ --http-port 80 \ --https-port 443 ``` ```bash title="Output" { "deploymentStatus": "Succeeded", "enabledState": "Enabled", "enforceCertificateNameCheck": true, "hostName": "example.com", "httpPort": 80, "httpsPort": 443, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-afd-demo/providers/Microsoft.Cdn/profiles/afd-demo/originGroups/my-origin-group/origins/my-origin", "name": "my-origin", "originGroupName": "my-origin-group", "priority": 1, "provisioningState": "Succeeded", "resourceGroup": "rg-afd-demo", "type": "Microsoft.Cdn/profiles/originGroups/origins", "weight": 50 } ``` ### Create a route Create a route to wire the endpoint to the origin group and define which protocols and path patterns are accepted. Per the [`az afd route create`](https://learn.microsoft.com/en-us/cli/azure/afd/route#az-afd-route-create) reference, `--origin-group` accepts either the origin group resource name within the profile or its full ARM resource ID (this example passes the origin group name, `my-origin-group`). ```bash az afd route create \ --resource-group rg-afd-demo \ --profile-name afd-demo \ --endpoint-name my-endpoint \ --route-name my-route \ --origin-group my-origin-group \ --supported-protocols Http Https \ --link-to-default-domain Enabled \ --https-redirect Disabled ``` ```bash title="Output" { "customDomains": [], "deploymentStatus": "Succeeded", "enabledState": "Enabled", "forwardingProtocol": "MatchRequest", "httpsRedirect": "Disabled", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-afd-demo/providers/Microsoft.Cdn/profiles/afd-demo/afdEndpoints/my-endpoint/routes/my-route", "linkToDefaultDomain": "Enabled", "name": "my-route", "originGroup": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-afd-demo/providers/Microsoft.Cdn/profiles/afd-demo/originGroups/my-origin-group", "resourceGroup": "rg-afd-demo" }, "patternsToMatch": [ "/*" ], "provisioningState": "Succeeded", "resourceGroup": "rg-afd-demo", "ruleSets": [], "supportedProtocols": [ "Http", "Https" ], "type": "Microsoft.Cdn/profiles/afdEndpoints/routes" } ``` ### Work with rule sets and rules Create a rule set to group related routing rules under the profile. Rule set names must be globally unique within the CDN resource provider ([`az afd rule-set create`](https://learn.microsoft.com/en-us/cli/azure/afd/rule-set#az-afd-rule-set-create)). Microsoft's naming overview documents patterns for sibling `Microsoft.Cdn` entities such as [origin groups and routes](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/resource-name-rules#microsoftcdn). ```bash az afd rule-set create \ --resource-group rg-afd-demo \ --profile-name afd-demo \ --rule-set-name myruleset ``` ```bash title="Output" { "deploymentStatus": "Succeeded", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-afd-demo/providers/Microsoft.Cdn/profiles/afd-demo/ruleSets/myruleset", "name": "myruleset", "provisioningState": "Succeeded", "resourceGroup": "rg-afd-demo", "type": "Microsoft.Cdn/profiles/ruleSets" } ``` Add a rule to the rule set with an execution order and match processing behaviour: ```bash az afd rule create \ --resource-group rg-afd-demo \ --profile-name afd-demo \ --rule-set-name myruleset \ --rule-name myrule \ --order 1 \ --match-processing-behavior Continue ``` ```bash title="Output" { "actions": [], "conditions": [], "deploymentStatus": "Succeeded", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-afd-demo/providers/Microsoft.Cdn/profiles/afd-demo/ruleSets/myruleset/rules/myrule", "matchProcessingBehavior": "Continue", "name": "myrule", "order": 1, "provisioningState": "Succeeded", "resourceGroup": "rg-afd-demo", "ruleSetName": "myruleset", "type": "Microsoft.Cdn/profiles/ruleSets/rules" } ``` List all rule sets and rules under the profile to confirm their state: ```bash az afd rule-set list \ --resource-group rg-afd-demo \ --profile-name afd-demo ``` ```bash title="Output" [ { "deploymentStatus": "Succeeded", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-afd-demo/providers/Microsoft.Cdn/profiles/afd-demo/ruleSets/myruleset", "name": "myruleset", "provisioningState": "Succeeded", "resourceGroup": "rg-afd-demo", "type": "Microsoft.Cdn/profiles/ruleSets" } ] ``` ```bash az afd rule list \ --resource-group rg-afd-demo \ --profile-name afd-demo \ --rule-set-name myruleset ``` ```bash title="Output" [ { "actions": [], "conditions": [], "deploymentStatus": "Succeeded", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-afd-demo/providers/Microsoft.Cdn/profiles/afd-demo/ruleSets/myruleset/rules/myrule", "matchProcessingBehavior": "Continue", "name": "myrule", "order": 1, "provisioningState": "Succeeded", "resourceGroup": "rg-afd-demo", "ruleSetName": "myruleset", "type": "Microsoft.Cdn/profiles/ruleSets/rules" } ] ``` ### Show and list Show the Front Door profile to inspect its current state: ```bash az afd profile show \ --resource-group rg-afd-demo \ --profile-name afd-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-afd-demo/providers/Microsoft.Cdn/profiles/afd-demo", "kind": "frontdoor", "location": "Global", "name": "afd-demo", "provisioningState": "Succeeded", "resourceState": "Active", "sku": { "name": "Standard_AzureFrontDoor" }, "type": "Microsoft.Cdn/profiles" } ``` List all profiles in the resource group to verify the profile appears in results: ```bash az afd profile list --resource-group rg-afd-demo ``` ```bash title="Output" [ { "frontDoorId": "083e606f-cec4-481b-b1d2-781b4428e52c", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-afd-demo/providers/Microsoft.Cdn/profiles/afd-demo", "kind": "frontdoor", "location": "Global", "name": "afd-demo", "originResponseTimeoutSeconds": 30, "provisioningState": "Succeeded", "resourceGroup": "rg-afd-demo", "resourceState": "Active", "sku": { "name": "Standard_AzureFrontDoor" }, "tags": {}, "type": "Microsoft.Cdn/profiles" } ] ``` ### Delete and verify Delete the Front Door profile to remove the profile and all child resources from the emulator: ```bash az afd profile delete \ --resource-group rg-afd-demo \ --profile-name afd-demo ``` Verify the resource group no longer contains any profiles: ```bash az afd profile list --resource-group rg-afd-demo ``` ```bash title="Output" [] ``` ## Features - **Full resource hierarchy:** Profiles, endpoints, origin groups, origins, routes, rule sets, and rules can all be created, updated, listed, and deleted under `Microsoft.Cdn/profiles`. - **Hostname generation:** A realistic `*.azurefd.net` hostname is generated for each endpoint, matching the format used by the real Azure service. - **Health probe settings:** Origin group health probe configuration — path, protocol, request type, and probe interval — is stored and returned on all get and list operations. - **Load balancing settings:** `sampleSize`, `successfulSamplesRequired`, and `additionalLatencyInMilliseconds` are stored and returned correctly. - **Route configuration:** Pattern matching, supported protocols, HTTPS redirect behaviour, and origin group linkage are fully supported. - **Rule sets and rules:** Match processing behaviour and ordering are stored and returned. - **Endpoint name availability:** **`checkEndpointNameAvailability`** (`POST .../Microsoft.Cdn/profiles/{profileName}/checkEndpointNameAvailability`); see Microsoft's [REST reference](https://learn.microsoft.com/en-us/rest/api/frontdoor/azurefrontdoorstandardpremium/afd-profiles/check-endpoint-name-availability). ## Limitations - **Rules not enforced at data plane:** Rule conditions and actions are stored by the management plane but are not evaluated; rules do not affect how requests are proxied. - **No custom domain validation:** Custom domain binding is stored but DNS validation and certificate provisioning are not performed. - **No HTTPS certificate management:** Managed certificates and bring-your-own-certificate flows are not emulated. - **Classic CDN data plane not emulated:** Classic CDN profiles (`az cdn`) can be created under the same `Microsoft.Cdn` namespace, but traffic acceleration behaviour is not emulated. - **Deployment status always `Succeeded`:** There is no asynchronous deployment pipeline behind profile or endpoint state changes. - **No WAF policy support:** WAF policy association is not supported. ## Samples The following samples demonstrate how to use Azure Front Door with LocalStack for Azure: - [Function App and Front Door](https://github.com/localstack/localstack-azure-samples/samples/function-app-front-door/python/README.md) ## API Coverage # Function Apps > Get started with Azure Function Apps on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Function Apps are serverless compute resources that host and execute Azure Functions in response to events. They support multiple runtime environments including Python, Node.js, and .NET, and integrate with a wide range of trigger sources such as HTTP requests, queues, and timers. Function Apps are commonly used to build event-driven microservices, scheduled background jobs, and lightweight API backends. For more information, see [Introduction to Azure Functions](https://learn.microsoft.com/en-us/azure/azure-functions/functions-overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Function Apps. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Function Apps' integration with LocalStack. ## Getting started This guide walks you through creating a Function App backed by a Storage Account, deploying a Python HTTP-trigger function, and invoking it. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create --name rg-func-demo --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-func-demo", "location": "westeurope", "name": "rg-func-demo", "properties": { "provisioningState": "Succeeded" }, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a storage account Function Apps require a Storage Account for internal bookkeeping. ```bash az storage account create \ --name safuncdemo \ --resource-group rg-func-demo \ --location westeurope \ --sku Standard_LRS ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-func-demo/providers/Microsoft.Storage/storageAccounts/safuncdemo", "kind": "StorageV2", "location": "westeurope", "name": "safuncdemo", "provisioningState": "Succeeded", "resourceGroup": "rg-func-demo", "sku": { "name": "Standard_LRS", "tier": "Standard" }, "type": "Microsoft.Storage/storageAccounts", ... } ``` ### Create a function app Create a function app and associate it with the storage account and App Service plan: ```bash az functionapp create \ --name my-func-app \ --resource-group rg-func-demo \ --consumption-plan-location westeurope \ --runtime python \ --runtime-version 3.13 \ --functions-version 4 \ --os-type linux \ --storage-account safuncdemo ``` ```bash title="Output" { "defaultHostName": "my-func-app.azurewebsites.azure.localhost.localstack.cloud:4566", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-func-demo/providers/Microsoft.Web/sites/my-func-app", "kind": "functionapp,linux", "location": "westeurope", "name": "my-func-app", "resourceGroup": "rg-func-demo", "state": "Running", "type": "Microsoft.Web/sites", ... } ``` ### Configure app settings Add environment variables as application settings on the function app: ```bash az functionapp config appsettings set \ --name my-func-app \ --resource-group rg-func-demo \ --settings \ FUNCTIONS_WORKER_RUNTIME=python \ MY_CUSTOM_SETTING=hello-world ``` ```bash title="Output" [ { "name": "FUNCTIONS_WORKER_RUNTIME", "slotSetting": false, "value": "python" }, { "name": "MY_CUSTOM_SETTING", "slotSetting": false, "value": "hello-world" }, { "name": "AzureWebJobsStorage", "slotSetting": false, "value": "DefaultEndpointsProtocol=https;..." }, ... ] ``` ### List application settings List all application settings currently configured on the function app: ```bash az functionapp config appsettings list \ --name my-func-app \ --resource-group rg-func-demo ``` ```bash title="Output" [ { "name": "FUNCTIONS_WORKER_RUNTIME", "slotSetting": false, "value": "python" }, { "name": "FUNCTIONS_EXTENSION_VERSION", "slotSetting": false, "value": "~4" }, { "name": "AzureWebJobsStorage", "slotSetting": false, "value": "DefaultEndpointsProtocol=https;..." }, { "name": "MY_CUSTOM_SETTING", "slotSetting": false, "value": "hello-world" }, ... ] ``` ### Create the function package Create a directory for the function source and add the required files: ```bash mkdir my_func && cd my_func ``` Create `function_app.py` with an HTTP-triggered function: ```python title="function_app.py" import azure.functions as func app = func.FunctionApp() @app.function_name(name="HelloWorld") @app.route(route="public", methods=["GET"], auth_level=func.AuthLevel.ANONYMOUS) def public(req: func.HttpRequest) -> func.HttpResponse: name = req.params.get("name", "World") return func.HttpResponse(f"Hello, {name}!") ``` Create `host.json`: ```json title="host.json" { "version": "2.0" } ``` Create `requirements.txt`: ```text title="requirements.txt" azure-functions ``` Package everything into a zip archive and return to the parent directory: ```bash zip my_func.zip function_app.py host.json requirements.txt cd .. ``` ### Deploy function code Deploy the zip package to the function app: ```bash az functionapp deploy \ --resource-group rg-func-demo \ --name my-func-app \ --src-path ./my_func/my_func.zip \ --type zip ``` ### Invoke an HTTP-trigger function After deployment, invoke the function via its default hostname: ```bash curl "http://my-func-app.azurewebsites.azure.localhost.localstack.cloud:4566/api/public?name=LocalStack" ``` ```bash title="Output" Hello, LocalStack! ``` ### List and inspect function apps List all function apps in the resource group in table format and retrieve the full configuration of the app: ```bash az functionapp list --resource-group rg-func-demo --output table ``` ```bash title="Output" Name Location State ResourceGroup DefaultHostName AppServicePlan ----------- ----------- ------- --------------- ---------------------------------------------------------------- ---------------- my-func-app West Europe Running rg-func-demo my-func-app.azurewebsites.azure.localhost.localstack.cloud:4566 Default1pn ``` Then retrieve the full configuration of the function app: ```bash az functionapp show --name my-func-app --resource-group rg-func-demo ``` ```bash title="Output" { "defaultHostName": "my-func-app.azurewebsites.azure.localhost.localstack.cloud:4566", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-func-demo/providers/Microsoft.Web/sites/my-func-app", "kind": "functionapp,linux", "location": "westeurope", "name": "my-func-app", "resourceGroup": "rg-func-demo", "state": "Running", "type": "Microsoft.Web/sites" ... } ``` ### Delete and verify Delete the resource and confirm it no longer appears in the list: ```bash az functionapp delete --name my-func-app --resource-group rg-func-demo ``` Then list all function apps to confirm the resource group is now empty: ```bash az functionapp list --resource-group rg-func-demo ``` ```bash title="Output" [] ``` ## Features - **Full CRUD lifecycle:** Create, read, update, and delete Function App resources using the Azure CLI or ARM API. - **App settings management:** Set and list application settings via `az functionapp config appsettings`. - **Site configuration:** Configure site properties including `linuxFxVersion`, `use32BitWorkerProcess`, and `alwaysOn`. - **Publishing credentials:** Retrieve deployment credentials via the `listPublishingCredentials` action. - **Publishing profile:** Download publish profiles via the `listPublishingProfileXmlWithSecrets` action. - **SCM access control:** Get and configure source control manager (SCM) access policy. - **Diagnostic logs:** Configure logging via `az webapp log config`. - **Consumption plan auto-provisioning:** An App Service Plan is automatically created for the Function App when using a consumption plan location. - **Docker container execution:** Actual function execution is backed by a Docker container spawned on deploy. - **HTTP trigger invocation:** HTTP-triggered functions are accessible via the default host name after deployment. - **Multiple runtimes:** Python, Node.js, and .NET function runtimes are supported for execution. ## Limitations - **Docker required for execution:** Function code execution requires Docker to be running. Without Docker, the Function App resource can be created and managed, but code will not run. - **Durable Functions not supported:** Stateful orchestration via Durable Functions is not emulated. - **Timer and non-HTTP triggers:** Timer, Blob, Queue, Service Bus, Event Hub, Event Grid, and Cosmos DB triggers are not automatically fired by LocalStack. They must be invoked manually or by external tooling. - **Deployment slots:** Staging slots and slot swaps are not supported. - **Log streaming:** Live log streaming via `az webapp log tail` is not supported. - **Managed identities for functions:** Assigning a managed identity to a Function App is accepted at the ARM level but authentication tokens are not issued for bindings. ## Samples The following sample demonstrates how to use Azure Function Apps with LocalStack for Azure: - [Function App and Storage](https://github.com/localstack/localstack-azure-samples/samples/function-app-storage-http/dotnet/README.md) - [Function App and Front Door](https://github.com/localstack/localstack-azure-samples/samples/function-app-front-door/python/README.md) - [Function App and Managed Identities](https://github.com/localstack/localstack-azure-samples/samples/function-app-managed-identity/python/README.md) - [Function App and Service Bus](https://github.com/localstack/localstack-azure-samples/samples/function-app-service-bus/dotnet/README.md) ## API Coverage # Key Vault > Get started with Azure Key Vault on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Key Vault is a managed service for securely storing and accessing secrets, keys, and certificates. It helps centralize sensitive configuration and credentials for your applications and services. Key Vault also supports secure key management and certificate lifecycle operations. For more information, see [About Azure Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Key Vault. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Key Vault's integration with LocalStack. ## Getting started This guide is designed for users new to Key Vault and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group that will contain your Key Vault resources: ```bash az group create \ --name rg-keyvault-demo \ --location westeurope ``` ```bash title="Output" { "name": "rg-keyvault-demo", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-keyvault-demo", "location": "westeurope", "properties": { "provisioningState": "Succeeded" } } ``` ### Create a Key Vault Create a Key Vault in your resource group: ```bash az keyvault create \ --name kv-demo-localstack \ --resource-group rg-keyvault-demo \ --location westeurope ``` ```bash title="Output" { "name": "kv-demo-localstack", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-keyvault-demo/providers/Microsoft.KeyVault/vaults/kv-demo-localstack", "location": "westeurope", "properties": { "provisioningState": "Succeeded", "vaultUri": "https://kv-demo-localstack.localhost.localstack.cloud:4566" } ... } ``` ### Add and read a secret Create a secret in the vault: ```bash az keyvault secret set \ --vault-name kv-demo-localstack \ --name app-secret \ --value "super-secret-value" ``` ```bash title="Output" { "name": "app-secret", "id": "https://kv-demo-localstack.localhost.localstack.cloud:4566/secrets/app-secret/d8a709f96aee4bea901bd8825f28a281", "attributes": { "enabled": true }, "value": "super-secret-value" ... } ``` Read the secret value: ```bash az keyvault secret show \ --vault-name kv-demo-localstack \ --name app-secret ``` ```bash title="Output" { "name": "app-secret", "id": "https://kv-demo-localstack.localhost.localstack.cloud:4566/secrets/app-secret/d8a709f96aee4bea901bd8825f28a281", "attributes": { "enabled": true }, "value": "super-secret-value" ... } ``` List all secrets in the vault: ```bash az keyvault secret list \ --vault-name kv-demo-localstack ``` ```bash title="Output" [ { "name": "app-secret", "id": "https://kv-demo-localstack.localhost.localstack.cloud:4566/secrets/app-secret" ... } ] ``` ## Limitations - **Keys and HSM not supported:** Key Vault keys, HSM-related operations, and getting a real certificate from an official CA are not supported. - **RBAC enforcement is opt-in and covers secrets/certificates only:** By default, data-plane operations succeed regardless of role assignments. Set `LS_AZURE_ENFORCE_RBAC` to enforce roles such as `Key Vault Secrets User` or `Key Vault Certificates Officer` on vaults created with `enableRbacAuthorization=true`; keys are not covered. See [Role Assignment: Enabling RBAC enforcement](/azure/services/role-assignment/#enabling-rbac-enforcement). ## Samples The following sample demonstrates how to use Azure Key Vault with LocalStack for Azure: - [Azure Container Instances, Key Vault, and Storage](https://learn.microsoft.com/en-us/azure/container-instances/container-instances-quickstart) - [Azure Web App with Azure SQL Database and Azure Key Vault](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-sql-database/python) ## API Coverage # Log Analytics > Get started with Azure Log Analytics on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Log Analytics Workspaces are the primary data store for Azure Monitor log data. They collect, index, and query log data from Azure resources, virtual machines, and custom sources. For more information, see [Log Analytics workspace overview](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/log-analytics-workspace-overview). They are commonly used as the central destination for diagnostic settings, Azure Monitor agents, and security audit logs in enterprise monitoring architectures. LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Log Analytics Workspaces. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Log Analytics' integration with LocalStack. ## Getting started This guide walks you through creating a Log Analytics Workspace, retrieving its shared keys, and deleting the workspace. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create --name rg-laws-demo --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-laws-demo", "location": "westeurope", "name": "rg-laws-demo", "properties": { "provisioningState": "Succeeded" }, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a Log Analytics Workspace Create a Log Analytics workspace with a 30-day data retention period (matching the default for `--retention-time` in the [Azure CLI workspace create](https://learn.microsoft.com/en-us/cli/azure/monitor/log-analytics/workspace?view=azure-cli-latest#az-monitor-log-analytics-workspace-create) reference): ```bash az monitor log-analytics workspace create \ --name my-workspace \ --resource-group rg-laws-demo \ --location westeurope \ --retention-time 30 ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-laws-demo/providers/Microsoft.OperationalInsights/workspaces/my-workspace", "location": "westeurope", "name": "my-workspace", "resourceGroup": "rg-laws-demo", "type": "Microsoft.OperationalInsights/workspaces", "properties": { "customerId": "a1b2c3d4-e5f6-4789-a012-3456789abcd0", "provisioningState": "Succeeded", "publicNetworkAccessForIngestion": "Enabled", "publicNetworkAccessForQuery": "Enabled", "retentionInDays": 30, "sku": { "name": "PerGB2018" } } } ``` ### Retrieve workspace shared keys Retrieve the primary and secondary shared keys used to send logs directly to the workspace: ```bash az monitor log-analytics workspace get-shared-keys \ --workspace-name my-workspace \ --resource-group rg-laws-demo ``` ```bash title="Output" { "primarySharedKey": "ZW5jb2RlZFNoYXJlZEtleUV4YW1wbGUxMjM0NTY3OA==", "secondarySharedKey": "c2Vjb25kYXJ5U2hhcmVkS2V5RXhhbXBsZTk4NzY1NDMyMQ==" } ``` ### List workspaces List all Log Analytics workspaces in the resource group: ```bash az monitor log-analytics workspace list \ --resource-group rg-laws-demo ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-laws-demo/providers/Microsoft.OperationalInsights/workspaces/my-workspace", "location": "westeurope", "name": "my-workspace", "resourceGroup": "rg-laws-demo", "type": "Microsoft.OperationalInsights/workspaces", "properties": { "customerId": "a1b2c3d4-e5f6-4789-a012-3456789abcd0", "provisioningState": "Succeeded", "retentionInDays": 30, "sku": { "name": "PerGB2018" } } } ] ``` ### Show a workspace Retrieve the full details of the workspace, including its unique customer ID: ```bash az monitor log-analytics workspace show \ --workspace-name my-workspace \ --resource-group rg-laws-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-laws-demo/providers/Microsoft.OperationalInsights/workspaces/my-workspace", "location": "westeurope", "name": "my-workspace", "resourceGroup": "rg-laws-demo", "type": "Microsoft.OperationalInsights/workspaces", "properties": { "customerId": "a1b2c3d4-e5f6-4789-a012-3456789abcd0", "provisioningState": "Succeeded", "publicNetworkAccessForIngestion": "Enabled", "publicNetworkAccessForQuery": "Enabled", "retentionInDays": 30, "sku": { "name": "PerGB2018" } } } ``` ### Delete and verify Delete the resource and confirm it no longer appears in the list: ```bash az monitor log-analytics workspace delete \ --workspace-name my-workspace \ --resource-group rg-laws-demo \ --yes ``` Then list all workspaces to confirm the resource group is now empty: ```bash az monitor log-analytics workspace list \ --resource-group rg-laws-demo ``` ```bash title="Output" [] ``` ## Features - **Workspace lifecycle:** Create, read, list, update, and delete Log Analytics Workspaces. - **Shared key retrieval:** Retrieve primary and secondary shared keys via `get-shared-keys`. - **SKU configuration:** Accept `PerGB2018`, `Free`, `Standard`, `Premium`, `PerNode`, and `Standalone` SKUs (Azure's REST API also defines `CapacityReservation` and `LACluster`; see [WorkspaceSkuNameEnum](https://learn.microsoft.com/en-us/rest/api/loganalytics/workspaces/create-or-update#workspaceskunameenum) in the Azure REST reference). - **Retention configuration:** Configure log retention period in days. - **Activity Logs:** Activity log events generated by LocalStack operations are fully emulated and queryable via the Activity Log API. ## Limitations - **No log ingestion:** Data sent to the legacy [HTTP Data Collector API](https://learn.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api) or other ingestion endpoints is not stored. - **No KQL query execution:** Running `az monitor log-analytics query` is not supported. - **No table or schema management:** Custom tables, table schemas, and retention policies per table are not managed. - **No saved searches:** Saved queries and search functions are not supported. - **No linked services:** Linking Automation accounts or [Microsoft Defender for Cloud](https://learn.microsoft.com/en-us/azure/defender-for-cloud/defender-for-cloud-introduction) to a workspace is not emulated. - **No Microsoft Sentinel:** Sentinel and related SIEM workspace features are not emulated. ## Samples The following sample demonstrates how to use Azure Log Analytics with LocalStack for Azure: - [Function App and Service Bus](https://github.com/localstack/localstack-azure-samples/samples/function-app-service-bus/dotnet/README.md) - [Web App and Cosmos DB for MongoDB API](https://github.com/localstack/localstack-azure-samples/samples/web-app-cosmosdb-mongodb-api/python/README.md) ## API Coverage # Managed Identity > Get started with Azure Managed Identity on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Managed Identity provides identities for Azure resources so applications can authenticate without storing credentials in code. The Azure platform supports two types of identities: - **System-assigned**: Tied directly to the lifecycle of a specific resource; when the resource is deleted, Azure automatically cleans up the identity. - **User-assigned**: Created as a standalone Azure resource that can be assigned to one or more instances, making it ideal for shared workloads and scale sets. Managed identities are commonly used to access Azure services securely from apps and automation workflows. For more information, see [What are managed identities for Azure resources?](https://learn.microsoft.com/entra/identity/managed-identities-azure-resources/overview). LocalStack for Azure allows you to build and emulate applications that make use of system-assigned or user-assigned Managed Identities directly in your local environment. This enables you to validate your secret-less authentication logic with high fidelity, ensuring your code is production-ready without needing to provision live cloud resources. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Managed Identity's integration with LocalStack. ## Getting started This guide is designed for users new to Managed Identity and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group for the identity resources: ```bash az group create \ --name rg-managedidentity-demo \ --location westeurope ``` ```bash title="Output" { "name": "rg-managedidentity-demo", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-managedidentity-demo", "location": "westeurope", "properties": { "provisioningState": "Succeeded" }, ... } ``` ### User-assigned managed identity Create a user-assigned managed identity: ```bash az identity create \ --name mi-doc77 \ --resource-group rg-managedidentity-demo \ --location westeurope \ --tags environment=test ``` ```bash title="Output" { "name": "mi-doc77", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-managedidentity-demo/providers/Microsoft.ManagedIdentity/userAssignedIdentities/mi-doc77", "location": "westeurope", "principalId": "a55f8986-0187-48fd-ac82-e87db6b80376", "clientId": "216de8da-baf0-4403-925d-ac69c6ad67e3", "tenantId": "00000000-0000-0000-0000-000000000000", "tags": { "environment": "test" }, ... } ``` Get the new user-assigned managed identity: ```bash az identity show \ --name mi-doc77 \ --resource-group rg-managedidentity-demo ``` ```bash title="Output" { "name": "mi-doc77", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-managedidentity-demo/providers/Microsoft.ManagedIdentity/userAssignedIdentities/mi-doc77", "principalId": "a55f8986-0187-48fd-ac82-e87db6b80376", "clientId": "216de8da-baf0-4403-925d-ac69c6ad67e3", "tags": { "environment": "test" }, ... } ``` List user-assigned managed identities by resource group: ```bash az identity list --resource-group rg-managedidentity-demo ``` ```bash title="Output" [ { "name": "mi-doc77", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-managedidentity-demo/providers/Microsoft.ManagedIdentity/userAssignedIdentities/mi-doc77", "resourceGroup": "rg-managedidentity-demo", "tags": {"environment": "test"}, ... } ] ``` List identities by subscription: ```bash az identity list ``` ```bash title="Output" [ { "name": "mi-doc77", "type": "Microsoft.ManagedIdentity/userAssignedIdentities", "resourceGroup": "rg-managedidentity-demo", ... } ] ``` Update identity tags: ```bash az identity update \ --name mi-doc77 \ --resource-group rg-managedidentity-demo \ --tags environment=dev ``` ```bash title="Output" { "name": "mi-doc77", "tags": { "environment": "dev" }, ... } ``` Delete the identity and verify it no longer appears in the resource group: ```bash az identity delete --name mi-doc77 --resource-group rg-managedidentity-demo az identity list --resource-group rg-managedidentity-demo ``` ```bash title="Output" [] ``` ### System-assigned managed identity Create an app service plan and a web app: ```bash az appservice plan create \ --name asp-doc77 \ --resource-group rg-managedidentity-demo \ --location westeurope \ --sku F1 az webapp create \ --name ls-app-doc77 \ --resource-group rg-managedidentity-demo \ --plan asp-doc77 \ --runtime "PYTHON:3.11" ``` ```bash title="Output" { "name": "asp-doc77", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-managedidentity-demo/providers/Microsoft.Web/serverfarms/asp-doc77", "location": "westeurope", "provisioningState": "Succeeded", ... } { "name": "ls-app-doc77", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-managedidentity-demo/providers/Microsoft.Web/sites/ls-app-doc77", "type": "Microsoft.Web/sites", "location": "westeurope", ... } ``` Enable the system-assigned managed identity on the web app ```bash az webapp identity assign \ --name ls-app-doc77 \ --resource-group rg-managedidentity-demo ``` ```bash title="Output" { "type": "SystemAssigned", "principalId": "78b44418-f917-4f3a-ac29-a9821d3d8e7c", "tenantId": "00000000-0000-0000-0000-000000000000", ... } ``` Retrieve the system-assigned managed identity by scope: ```bash az webapp identity show \ --name ls-app-doc77 \ --resource-group rg-managedidentity-demo ``` ```bash title="Output" { "type": "SystemAssigned", "principalId": "78b44418-f917-4f3a-ac29-a9821d3d8e7c", "tenantId": "00000000-0000-0000-0000-000000000000", ... } ``` You can also retrieve the system-assigned managed identity of a web app by calling the control plane REST API as follows: ```bash SITE_ID=$(az webapp show --name ls-app-doc77 --resource-group rg-managedidentity-demo --query id -o tsv) az rest --method get \ --url "http://management.localhost.localstack.cloud:4566${SITE_ID}/providers/Microsoft.ManagedIdentity/identities/default?api-version=2024-11-30" ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-managedidentity-demo/providers/microsoft.web/sites/ls-app-doc77", "name": "ls-app-doc77", "type": "microsoft.web/sites", "location": "westeurope", "properties": { "principalId": "78b44418-f917-4f3a-ac29-a9821d3d8e7c", "clientId": "4364940c-ede7-43d8-8043-3dbad79377ee", "tenantId": "00000000-0000-0000-0000-000000000000", ... } } ``` ## Features The Managed Identity emulator supports the following features: - **User-assigned identity lifecycle**: Full create, read, update, and delete operations for user-assigned managed identities, including tag management and cross-region relocation. - **System-assigned identity retrieval**: Retrieve the system-assigned identity of any resource by scope, returning the associated principal ID, client ID, and tenant ID. - **Service principal auto-provisioning**: When a managed identity is created, a corresponding service principal is automatically registered in the Microsoft Graph store, mirroring Azure's built-in identity-to-directory integration. - **Role assignments**: Create, retrieve, delete, and list role assignments at subscription and scope levels. Scope-based filtering matches assignments by resource hierarchy. - **Role definitions**: Create and manage custom role definitions with granular permissions and assignable scopes. Over 549 builtin Azure role definitions are preloaded and available for immediate use. - **Management locks**: Create, delete, retrieve, and list management locks at the resource group level. Supported lock levels are `CanNotDelete` and `ReadOnly`. - **Microsoft Graph service principal queries**: List, create, and delete service principals through the Microsoft Graph `/v1.0/servicePrincipals` endpoint with OData query support including `$filter`, `$select`, `$top`, `$count`, and `$orderby`. - **Directory object lookups**: Resolve multiple directory objects by ID through the `/v1.0/directoryObjects/getByIds` endpoint. - **Principal propagation**: Tokens minted for a managed identity via the Instance Metadata Service (IMDS) or AKS Workload Identity carry that identity's principal ID, which is what [RBAC enforcement](/azure/services/role-assignment/#enabling-rbac-enforcement) evaluates role assignments against when enabled. ## Limitations The Managed Identity emulator has the following limitations: - **Federated identity credentials**: Federated identity credential operations (create, get, delete, list) are not yet implemented. - **No token issuance**: The emulator does not issue actual OAuth 2.0 tokens or enforce authentication. Identity objects are created and stored, but no real credential exchange occurs. - **Management locks scope**: Management locks are supported only at the resource group level. Subscription-level and individual-resource-level locks are not implemented. - **Microsoft Graph pagination**: The `@odata.nextLink` pagination mechanism is not implemented. Large result sets are returned in a single response. - **No data persistence across restarts**: Identity, role assignment, role definition, and service principal data is held in memory and is lost when the emulator is stopped or restarted. ## Samples The following samples demonstrate how to use Azure Managed Identity with LocalStack for Azure: - [Functions App with Managed Identity](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-managed-identity/python) - [Function App and Service Bus](https://github.com/localstack/localstack-azure-samples/samples/function-app-service-bus/dotnet/README.md) - [Web App and Cosmos DB for MongoDB API ](https://github.com/localstack/localstack-azure-samples/samples/web-app-cosmosdb-mongodb-api/python/README.md) - [Web App with Managed Identity](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-managed-identity/python) ## API Coverage # Metric Alert > Get started with Azure Monitor Metric Alerts on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Monitor Metric Alerts trigger notifications when a monitored metric crosses a defined threshold. They evaluate metric data at configurable intervals and route notifications through Action Groups when conditions are met. Metric Alerts are commonly used to detect anomalies in CPU usage, memory pressure, request latency, and other resource metrics across Azure workloads. For more information, see [Overview of metric alerts in Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-metric-overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Monitor Metric Alerts. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Metric Alerts' integration with LocalStack. ## Getting started This guide walks you through creating a metric alert rule referencing an action group. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create --name rg-alert-demo --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-alert-demo", "location": "westeurope", "name": "rg-alert-demo", "properties": { "provisioningState": "Succeeded" }, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create an action group Create an action group to use as the notification target for the metric alert: ```bash az monitor action-group create \ --name my-ag \ --resource-group rg-alert-demo \ --short-name myag \ --action email admin admin@example.com ``` ```bash title="Output" { "groupShortName": "myag", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-alert-demo/providers/microsoft.insights/actionGroups/my-ag", "name": "my-ag", "resourceGroup": "rg-alert-demo", "type": "Microsoft.Insights/ActionGroups" ... } ``` ### Create a metric alert The following alert fires when CPU percentage on a virtual machine exceeds 80%: ```bash SCOPE="/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-alert-demo/providers/Microsoft.Compute/virtualMachines/my-vm" AG_ID="/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-alert-demo/providers/microsoft.insights/actionGroups/my-ag" az monitor metrics alert create \ --name cpu-alert \ --resource-group rg-alert-demo \ --scopes "$SCOPE" \ --condition "avg Percentage CPU > 80" \ --action "$AG_ID" \ --description "Alert when CPU > 80%" ``` ```bash title="Output" { "criteria": { "allOf": [ { "criterionType": "StaticThresholdCriterion", "metricName": "Percentage CPU", "name": "cond0", "operator": "GreaterThan", "threshold": 80.0, "timeAggregation": "Average" } ], "odata.type": "Microsoft.Azure.Monitor.SingleResourceMultipleMetricCriteria" }, "description": "Alert when CPU > 80%", "enabled": true, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-alert-demo/providers/Microsoft.Insights/metricAlerts/cpu-alert", "name": "cpu-alert", "resourceGroup": "rg-alert-demo", "severity": 2, "type": "Microsoft.Insights/metricAlerts", ... } ``` ### Show and list metric alerts Retrieve the details of the metric alert and list all alerts in the resource group: ```bash az monitor metrics alert show \ --name cpu-alert \ --resource-group rg-alert-demo ``` ```bash title="Output" { "criteria": { "allOf": [ { "metricName": "Percentage CPU", "operator": "GreaterThan", "threshold": 80.0, "timeAggregation": "Average", ... } ], "odata.type": "Microsoft.Azure.Monitor.SingleResourceMultipleMetricCriteria" }, "description": "Alert when CPU > 80%", "enabled": true, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-alert-demo/providers/Microsoft.Insights/metricAlerts/cpu-alert", "name": "cpu-alert", "resourceGroup": "rg-alert-demo", "severity": 2, "type": "Microsoft.Insights/metricAlerts" ... } ``` Then list all metric alerts in the resource group: ```bash az monitor metrics alert list \ --resource-group rg-alert-demo ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-alert-demo/providers/Microsoft.Insights/metricAlerts/cpu-alert", "name": "cpu-alert", "resourceGroup": "rg-alert-demo", "severity": 2, "type": "Microsoft.Insights/metricAlerts" } ] ``` ### Create a service health activity log alert Create an [activity log alert for Service Health](https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-activity-log) events. The condition `category=ServiceHealth` matches the default pattern documented for [`az monitor activity-log alert create`](https://learn.microsoft.com/en-us/cli/azure/monitor/activity-log/alert#az-monitor-activity-log-alert-create). That is different from alerting on administrative operations such as deleting a virtual machine, which uses other activity log fields (for example `category` and `operationName` for the delete operation). ```bash az monitor activity-log alert create \ --name service-health-alert \ --resource-group rg-alert-demo \ --scope "/subscriptions/00000000-0000-0000-0000-000000000000" \ --condition category=ServiceHealth \ --action-group "$AG_ID" ``` ```bash title="Output" { "actions": { "actionGroups": [ { "actionGroupId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-alert-demo/providers/microsoft.insights/actionGroups/my-ag" } ] }, "condition": { "allOf": [ { "equals": "ServiceHealth", "field": "category" } ] }, "enabled": true, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-alert-demo/providers/Microsoft.Insights/activityLogAlerts/service-health-alert", "name": "service-health-alert", "resourceGroup": "rg-alert-demo", "scopes": ["/subscriptions/00000000-0000-0000-0000-000000000000"], "type": "Microsoft.Insights/ActivityLogAlerts" ... } ``` ### Delete and verify Delete the metric alert rule and confirm it no longer appears in the metric alert list: ```bash az monitor metrics alert delete \ --name cpu-alert \ --resource-group rg-alert-demo ``` Then list metric alerts in the resource group to confirm the rule was removed: ```bash az monitor metrics alert list --resource-group rg-alert-demo ``` ```bash title="Output" [] ``` ## Features - **Metric alert lifecycle:** Create, read, list, update, and delete metric alert rules. - **Activity log alert lifecycle:** Create, read, list, and delete activity log alert rules. - **Single and multi-resource scopes:** Define alerts scoped to a single resource or multiple resources. - **Action group references:** Associate action groups with alert rules. - **Dynamic threshold support:** Accept dynamic threshold criteria in the alert condition. - **Severity configuration:** Set alert severity from 0 (Critical) to 4 (Verbose). - **Auto-mitigation:** Configure whether alerts auto-resolve when the condition clears. - **Frequency and window size:** Configure evaluation frequency and aggregation window size. ## Limitations - **No alert evaluation:** Metric conditions are not evaluated against real or simulated metric data. Alerts never fire. - **No state transitions:** Alert state (Fired, Resolved) is not tracked or updated. - **No email or webhook dispatch:** Even if an alert were to fire, no notifications would be sent. - **Metrics not ingested:** Metric data is not ingested, stored, or queryable from LocalStack. ## Samples Explore end-to-end examples in the [LocalStack for Azure Samples](https://github.com/localstack/localstack-azure-samples) repository. ## API Coverage # Monitor > Get started with Azure Monitor on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Monitor is a platform service for collecting, analyzing, and acting on telemetry from Azure resources and applications. It helps you inspect activity logs and configure diagnostic settings for operational visibility. These capabilities are useful for troubleshooting, auditing, and observability workflows. For more information, see [Azure Monitor overview](https://learn.microsoft.com/azure/azure-monitor/fundamentals/overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Monitor. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Monitor's integration with LocalStack. ## Getting started This guide is designed for users new to Azure Monitor and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to contain your Monitor demo resources: ```bash az group create \ --name rg-monitor-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-monitor-demo", "location": "westeurope", "managedBy": null, "name": "rg-monitor-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a storage account Create a storage account to use as a diagnostic settings destination: ```bash az storage account create \ --name mystore \ --resource-group rg-monitor-demo \ --location westeurope \ --sku Standard_LRS ``` ```bash title="Output" { ... "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-monitor-demo/providers/Microsoft.Storage/storageAccounts/mystore", ... "name": "stmonitordoc79", ... "primaryEndpoints": { "blob": "https://mystore.blob.core.azure.localhost.localstack.cloud:4566", ... "table": "https://mystore.table.core.azure.localhost.localstack.cloud:4566", ... }, ... } ``` ### List activity logs List recent activity logs from the subscription: ```bash az monitor activity-log list --max-events 5 ``` ```bash title="Output" [ { "caller": "00000000-0000-0000-0000-000000000000", "category": { "value": "Administrative", ... }, "eventName": { "value": "EndRequest", ... }, "eventTimestamp": "2026-03-17T07:34:43.230050", "resourceGroupName": "rg-monitor-demo", "resourceProviderName": { "value": "Microsoft.Resources", ... }, ... }, ... ] ``` ### Create and inspect diagnostic settings Get the resource ID of the storage account: ```bash RESOURCE_ID=$(az storage account show \ --name mystore \ --resource-group rg-monitor-demo \ --query id \ --output tsv) ``` Create a diagnostic setting for the bob service of the storage account. For more information, see [Diagnostic settings in Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/platform/diagnostic-settings): ```bash az monitor diagnostic-settings create \ --name rg-monitor-demo \ --resource "${RESOURCE_ID}/blobServices/default" \ --storage-account mystore \ --logs '[{"category":"StorageRead","enabled":true},{"category":"StorageWrite","enabled":true}]' \ --metrics '[{"category":"Transaction","enabled":true},{"category":"Capacity","enabled":true}]' ``` ```bash title="Output" { "id": "subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-monitor-demo/providers/microsoft.insights/diagnosticSettings/rg-monitor-demo", "name": "rg-monitor-demo", "logs": [ { "category": "Administrative", "enabled": true } ], "metrics": [], "storageAccountId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-monitor-demo/providers/microsoft.Storage/storageAccounts/stmonitordoc79", "type": "microsoft.insights/diagnosticSettings" ... } ``` Get the diagnostic setting: ```bash az monitor diagnostic-settings show \ --name rg-monitor-demo \ --resource "${RESOURCE_ID}/blobServices/default" ``` ```bash title="Output" { "id": "subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-monitor-demo/providers/microsoft.insights/diagnosticSettings/rg-monitor-demo", "name": "rg-monitor-demo", "logs": [ { "category": "StorageRead", "enabled": true }, { "category": "StorageWrite", "enabled": true } ], "metrics": [ { "category": "Transaction", "enabled": true, "retentionPolicy": { "days": 0, "enabled": false } }, { "category": "Capacity", "enabled": true, "retentionPolicy": { "days": 0, "enabled": false } } ], ... } ``` ### Update and delete diagnostic settings Update diagnostic settings to include an additional category: ```bash az monitor diagnostic-settings update \ --name rg-monitor-demo \ --resource"${RESOURCE_ID}/blobServices/default" \ --logs '[{"category":"StorageRead","enabled":true},{"category":"StorageWrite","enabled":true},,{"category":"StorageDelete","enabled":true}]' \ --metrics '[{"category":"Transaction","enabled":true},{"category":"Capacity","enabled":true}]' ``` ```bash title="Output" { "name": "rg-monitor-demo", "logs": [ { "category": "StorageRead", "enabled": true, "retentionPolicy": { "days": 0, "enabled": false } }, { "category": "StorageWrite", "enabled": true, "retentionPolicy": { "days": 0, "enabled": false } }, { "category": "StorageDelete", "enabled": false, "retentionPolicy": { "days": 0, "enabled": false } } ], "metrics": [ { "category": "Transaction", "enabled": true, "retentionPolicy": { "days": 0, "enabled": false } }, { "category": "Capacity", "enabled": true, "retentionPolicy": { "days": 0, "enabled": false } } ], ... } ``` Delete the diagnostic setting: ```bash az monitor diagnostic-settings delete \ --name rg-monitor-demo \ --resource "${RESOURCE_ID}/blobServices/default" ``` ## Features The Azure Monitor emulator supports the following features: - **Activity logs**: List activity log entries for a subscription, with optional filtering by resource ID and time range. - **Diagnostic settings**: Create, get, update, and delete diagnostic settings on any ARM resource. - **Application Insights components**: Create, get, update tags, list, delete, and purge Application Insights components. Billing features (get and update) are also supported. - **Action groups**: Create, get, update, list (by resource group or subscription), and delete action groups. - **Metric alerts**: Create, get, update, list (by resource group or subscription), and delete metric alert rules. - **Activity log alerts**: Create, get, update, list (by resource group or subscription), and delete activity log alert rules. - **Autoscale settings**: Create, get, update, list (by resource group or subscription), and delete autoscale settings. - **Scheduled query rules**: Create, get, update, list (by resource group or subscription), and delete scheduled query rules. - **Data collection rules and endpoints**: Create, get, update, list (by resource group or subscription), and delete data collection rules and data collection endpoints. Create, get, list (by resource, by rule, or by endpoint), and delete data collection rule associations. - **Web tests**: Create, get, update tags, list (by resource group, subscription, or component), and delete availability web tests. - **Workbooks and workbook templates**: Create, get, update, list (by resource group or subscription), and delete workbooks. Create, get, update, list (by resource group), and delete workbook templates. Workbook revisions (get and list) are also supported. - **Telemetry ingestion (data plane)**: Accept Application Insights SDK telemetry payloads (`track`), custom metrics publish, and live metrics subscription checks. - **Query API (data plane)**: Execute and get log analytics queries, retrieve metrics, and list events. ## Limitations - **No data persistence across restarts**: Activity logs, diagnostic settings, and all resource state are held in memory and lost when the emulator is stopped or restarted. - **Activity log recording**: Only non-read (non-GET) control plane operations are recorded. Data plane operations and read requests do not produce activity log entries. - **Diagnostic settings are not enforced**: Diagnostic settings are stored but do not route logs or metrics to the specified destination (storage account, Log Analytics workspace, or event hub). - **Telemetry ingestion is a no-op**: The data plane telemetry endpoints (`track`, `publish`, `isSubscribed`) accept payloads and return success responses, but telemetry data is not stored or queryable. - **Query API returns empty results**: Log analytics queries, metrics, and event queries return structurally valid but empty responses. - **Autoscale rules are not evaluated**: Autoscale settings are stored but scaling actions are never triggered. - **Alert rule evaluation**: Metric alerts, activity log alerts, and scheduled query rules are stored as resources but are never evaluated or fired. - **Workbook revision history**: Revisions always return the current version of a workbook; historical revision tracking is not implemented. - **Components purge**: The purge endpoint accepts requests and returns an operation ID, but no data is actually purged. ## Samples Explore end-to-end examples in the [LocalStack for Azure Samples](https://github.com/localstack/localstack-azure-samples) repository. ## API Coverage # NAT Gateway > Get started with Azure NAT Gateway on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure NAT Gateway provides outbound connectivity for virtual machines and other resources in a virtual network. It enables all resources in a subnet to share one or more static public IP addresses or public IP prefixes for outbound internet connections. NAT Gateway is commonly used to give private workloads consistent and predictable outbound IP addresses without exposing individual resources to the internet. For more information, see [What is Azure NAT Gateway?](https://learn.microsoft.com/en-us/azure/nat-gateway/nat-overview). LocalStack for Azure provides a local environment for building and testing applications that make use of NAT Gateway. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of NAT Gateway's integration with LocalStack. ## Getting started This guide is designed for users new to NAT Gateway and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create \ --name rg-nat-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nat-demo", "location": "westeurope", "managedBy": null, "name": "rg-nat-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a public IP prefix NAT Gateway requires a public IP address or public IP prefix to route outbound traffic. Create a public IP prefix: ```bash az network public-ip prefix create \ --name pip-prefix-nat \ --resource-group rg-nat-demo \ --location westeurope \ --length 29 ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nat-demo/providers/Microsoft.Network/publicIPPrefixes/pip-prefix-nat", "ipPrefix": "20.163.121.0/29", "ipTags": [], "location": "westeurope", "name": "pip-prefix-nat", "prefixLength": 29, "provisioningState": "Succeeded", "publicIPAddressVersion": "IPv4", "resourceGroup": "rg-nat-demo", "sku": { "name": "Standard", "tier": "Regional" }, "type": "Microsoft.Network/publicIPPrefixes", "zones": [] ... } ``` ### Create a NAT gateway Create a NAT gateway attached to the public IP prefix: ```bash az network nat gateway create \ --name nat-gw-demo \ --resource-group rg-nat-demo \ --location westeurope \ --public-ip-prefixes pip-prefix-nat \ --idle-timeout 4 ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nat-demo/providers/Microsoft.Network/natGateways/nat-gw-demo", "idleTimeoutInMinutes": 4, "location": "westeurope", "name": "nat-gw-demo", "provisioningState": "Succeeded", "publicIpPrefixes": [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nat-demo/providers/Microsoft.Network/publicIPPrefixes/pip-prefix-nat", "resourceGroup": "rg-nat-demo" } ], "resourceGroup": "rg-nat-demo", "sku": { "name": "Standard" }, "type": "Microsoft.Network/natGateways" ... } ``` ### Get and list NAT gateways Retrieve the details of the NAT gateway and list all NAT gateways in the resource group: ```bash az network nat gateway show \ --name nat-gw-demo \ --resource-group rg-nat-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nat-demo/providers/Microsoft.Network/natGateways/nat-gw-demo", "idleTimeoutInMinutes": 4, "location": "westeurope", "name": "nat-gw-demo", "provisioningState": "Succeeded", "publicIpPrefixes": [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nat-demo/providers/Microsoft.Network/publicIPPrefixes/pip-prefix-nat", "resourceGroup": "rg-nat-demo" } ], "resourceGroup": "rg-nat-demo", "sku": { "name": "Standard" }, "type": "Microsoft.Network/natGateways" ... } ``` Then list all NAT gateways in the resource group: ```bash az network nat gateway list \ --resource-group rg-nat-demo ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nat-demo/providers/Microsoft.Network/natGateways/nat-gw-demo", "idleTimeoutInMinutes": 4, "location": "westeurope", "name": "nat-gw-demo", "provisioningState": "Succeeded", "publicIpPrefixes": [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nat-demo/providers/Microsoft.Network/publicIPPrefixes/pip-prefix-nat", "resourceGroup": "rg-nat-demo" } ], "resourceGroup": "rg-nat-demo", "sku": { "name": "Standard" }, "type": "Microsoft.Network/natGateways" } ] ``` ### Delete the NAT gateway Delete the NAT gateway and verify it no longer appears in the list: ```bash az network nat gateway delete \ --name nat-gw-demo \ --resource-group rg-nat-demo ``` Then list all NAT gateways in the resource group to confirm the NAT gateway was removed: ```bash az network nat gateway list \ --resource-group rg-nat-demo ``` ```bash title="Output" [] ``` ## Features The NAT Gateway emulator supports the following features: - **Create and manage NAT gateways**: Full lifecycle management including create, get, update, list, and delete. - **Public IP and prefix associations**: Attach public IP addresses or public IP prefixes to a NAT gateway at creation or update time. - **Idle timeout configuration**: Set the TCP idle timeout (in minutes) for outbound connections. - **Tags**: Apply and update resource tags on NAT Gateway resources. - **Subscription-scoped listing**: List all NAT gateways across a subscription using `az network nat gateway list`. ## Limitations - **No outbound traffic routing**: NAT Gateway is a mock implementation. State is persisted in memory and returned faithfully, but no outbound network traffic is routed through the gateway. - **No data persistence**: NAT Gateway resources are not persisted and are lost when the emulator is stopped or restarted. - **No subnet association enforcement**: Associating a NAT gateway with a subnet is accepted but not enforced at the network level. ## Samples The following samples demonstrate how to use Azure NAT Gateway with LocalStack for Azure: - [Function App and Service Bus](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-service-bus/dotnet/) - [Web App and Cosmos DB for MongoDB API](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-cosmosdb-mongodb-api/python) ## API Coverage # Network Interface > Get started with Azure Network Interface on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Network Interface (NIC) is the interconnection between a virtual machine and a virtual network. A NIC enables an Azure VM to communicate with the internet, Azure, and on-premises resources. Each NIC can have one or more IP configurations, an associated subnet, optional network security group (NSG), and optional public IP address. For more information, see [Network interfaces](https://learn.microsoft.com/en-us/azure/virtual-network/virtual-network-network-interface). LocalStack for Azure provides a local environment for building and testing applications that make use of Network Interfaces. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Network Interface's integration with LocalStack. ## Getting started This guide is designed for users new to Network Interfaces and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group and virtual network A network interface must be associated with a subnet. Create the prerequisite resources first: ```bash az group create \ --name rg-nic-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nic-demo", "location": "westeurope", "managedBy": null, "name": "rg-nic-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` Create a virtual network for the network interfaces: ```bash az network vnet create \ --name vnet-nic-demo \ --resource-group rg-nic-demo \ --location westeurope \ --address-prefixes 10.0.0.0/16 ``` ```bash title="Output" { "newVNet": { "addressSpace": { "addressPrefixes": [ "10.0.0.0/16" ] }, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nic-demo/providers/Microsoft.Network/virtualNetworks/vnet-nic-demo", "location": "westeurope", "name": "vnet-nic-demo", "provisioningState": "Succeeded", "resourceGroup": "rg-nic-demo", "subnets": [], "type": "Microsoft.Network/virtualNetworks", ... } } ``` Create a subnet within the virtual network to attach the NICs to: ```bash az network vnet subnet create \ --name subnet-nic \ --resource-group rg-nic-demo \ --vnet-name vnet-nic-demo \ --address-prefixes 10.0.1.0/24 ``` ```bash title="Output" { "addressPrefix": "10.0.1.0/24", "delegations": [], "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nic-demo/providers/Microsoft.Network/virtualNetworks/vnet-nic-demo/subnets/subnet-nic", "name": "subnet-nic", "privateEndpointNetworkPolicies": "Disabled", "privateLinkServiceNetworkPolicies": "Enabled", "provisioningState": "Succeeded", "resourceGroup": "rg-nic-demo", "type": "Microsoft.Network/virtualNetworks/subnets" ... } ``` ### Create a network interface Create a NIC attached to the subnet: ```bash az network nic create \ --name nic-demo \ --resource-group rg-nic-demo \ --location westeurope \ --vnet-name vnet-nic-demo \ --subnet subnet-nic ``` ```bash title="Output" { "NewNIC": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nic-demo/providers/Microsoft.Network/networkInterfaces/nic-demo", "ipConfigurations": [ { "name": "ipconfig1", "primary": true, "privateIPAddress": "10.0.1.4", "privateIPAddressVersion": "IPv4", "privateIPAllocationMethod": "Dynamic", "provisioningState": "Succeeded", "resourceGroup": "rg-nic-demo", "subnet": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nic-demo/providers/Microsoft.Network/virtualNetworks/vnet-nic-demo/subnets/subnet-nic", "resourceGroup": "rg-nic-demo" }, ... } ], "location": "westeurope", "name": "nic-demo", "provisioningState": "Succeeded", "resourceGroup": "rg-nic-demo", "type": "Microsoft.Network/networkInterfaces", ... } } ``` ### Create a NIC with a static private IP Create a second network interface and assign a static private IP address from the subnet: ```bash az network nic create \ --name nic-static \ --resource-group rg-nic-demo \ --location westeurope \ --vnet-name vnet-nic-demo \ --subnet subnet-nic \ --private-ip-address 10.0.1.10 ``` ```bash title="Output" { "NewNIC": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nic-demo/providers/Microsoft.Network/networkInterfaces/nic-static", "ipConfigurations": [ { "name": "ipconfig1", "primary": true, "privateIPAddress": "10.0.1.10", "privateIPAddressVersion": "IPv4", "privateIPAllocationMethod": "Static", "provisioningState": "Succeeded", "subnet": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nic-demo/providers/Microsoft.Network/virtualNetworks/vnet-nic-demo/subnets/subnet-nic", "resourceGroup": "rg-nic-demo" }, ... } ], "location": "westeurope", "name": "nic-static", "provisioningState": "Succeeded", "resourceGroup": "rg-nic-demo", "type": "Microsoft.Network/networkInterfaces", ... } } ``` ### Get and list network interfaces Retrieve the details of a network interface and list all NICs in the resource group: ```bash az network nic show \ --name nic-demo \ --resource-group rg-nic-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nic-demo/providers/Microsoft.Network/networkInterfaces/nic-demo", "ipConfigurations": [ { "name": "ipconfig1", "primary": true, "privateIPAddress": "10.0.1.4", "privateIPAddressVersion": "IPv4", "privateIPAllocationMethod": "Dynamic", "provisioningState": "Succeeded", "subnet": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nic-demo/providers/Microsoft.Network/virtualNetworks/vnet-nic-demo/subnets/subnet-nic", "resourceGroup": "rg-nic-demo" }, ... } ], "location": "westeurope", "name": "nic-demo", "provisioningState": "Succeeded", "resourceGroup": "rg-nic-demo", "type": "Microsoft.Network/networkInterfaces", ... } ``` Then list all network interfaces in the resource group: ```bash az network nic list \ --resource-group rg-nic-demo ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nic-demo/providers/Microsoft.Network/networkInterfaces/nic-demo", "ipConfigurations": [ { "name": "ipconfig1", "privateIPAllocationMethod": "Dynamic", ... } ], "location": "westeurope", "name": "nic-demo", "provisioningState": "Succeeded", "type": "Microsoft.Network/networkInterfaces", ... }, { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-nic-demo/providers/Microsoft.Network/networkInterfaces/nic-static", "ipConfigurations": [ { "name": "ipconfig1", "privateIPAddress": "10.0.1.10", "privateIPAllocationMethod": "Static", ... } ], "location": "westeurope", "name": "nic-static", "provisioningState": "Succeeded", "type": "Microsoft.Network/networkInterfaces", ... } ] ``` ### Delete a network interface Delete the network interface and verify it no longer appears in the list: ```bash az network nic delete \ --name nic-demo \ --resource-group rg-nic-demo ``` ## Features The Network Interface emulator supports the following features: - **Create and manage NICs**: Full lifecycle management including create, get, update, list, and delete. - **Dynamic IP allocation**: Automatically assigns a private IP address from the associated subnet address space. - **Static IP configuration**: Assign a fixed private IP address from the subnet range. - **Public IP association**: Attach a public IP address resource to a NIC IP configuration. - **NSG association**: Associate a network security group with a NIC. - **IP forwarding**: Configure IP forwarding on the NIC for routing scenarios. - **Accelerated networking**: Store and return the `enableAcceleratedNetworking` flag. - **Tags**: Apply and update resource tags. - **Subscription-scoped listing**: List all NICs across a subscription. ## Limitations - **No actual networking**: Network Interface is a mock implementation. State is persisted in memory and returned faithfully, but no network packets are routed through the interface. - **No VM attachment enforcement**: Associating a NIC with a virtual machine is accepted but VM resources are not implemented. - **No data persistence**: NIC resources are not persisted and are lost when the emulator is stopped or restarted. ## Samples The following samples demonstrate how to use Azure Network Interfaces with LocalStack for Azure: - [Function App and Service Bus](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-service-bus/dotnet/) - [Web App and Cosmos DB for MongoDB API](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-cosmosdb-mongodb-api/python) ## API Coverage # Private DNS Zone > Get started with Azure Private DNS Zone on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Private DNS Zone provides a reliable and secure DNS service to manage and resolve domain names in a virtual network without the need to add a custom DNS solution. Private DNS zones allow you to use your own custom domain names instead of the Azure-provided names, and to resolve DNS names across linked virtual networks. They are commonly paired with Private Endpoints to enable name resolution for privately-accessed PaaS services within a virtual network. For more information, see [What is an Azure Private DNS Zone?](https://learn.microsoft.com/en-us/azure/dns/private-dns-overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Private DNS Zones. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Private DNS Zone's integration with LocalStack. ## Getting started This guide is designed for users new to Private DNS Zone and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group and virtual network Create a resource group and a virtual network to link to the private DNS zone: ```bash az group create \ --name rg-dns-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dns-demo", "location": "westeurope", "managedBy": null, "name": "rg-dns-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` Create a virtual network to link to the private DNS zone: ```bash az network vnet create \ --name vnet-dns-demo \ --resource-group rg-dns-demo \ --location westeurope \ --address-prefixes 10.0.0.0/16 ``` ```bash title="Output" { "newVNet": { "addressSpace": { "addressPrefixes": [ "10.0.0.0/16" ] }, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dns-demo/providers/Microsoft.Network/virtualNetworks/vnet-dns-demo", "location": "westeurope", "name": "vnet-dns-demo", "provisioningState": "Succeeded", "resourceGroup": "rg-dns-demo", "subnets": [], "type": "Microsoft.Network/virtualNetworks", ... } } ``` ### Create a private DNS zone Create a private DNS zone using the recommended name for Blob storage private endpoints, [`privatelink.blob.core.windows.net`](https://learn.microsoft.com/en-us/azure/private-link/private-endpoint-dns), so it can be integrated with a private endpoint later in this guide. Linking the zone to your virtual network is covered in the next section. ```bash az network private-dns zone create \ --name privatelink.blob.core.windows.net \ --resource-group rg-dns-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dns-demo/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net", "location": "global", "name": "privatelink.blob.core.windows.net", "numberOfRecordSets": 1, "numberOfVirtualNetworkLinks": 0, "provisioningState": "Succeeded", "resourceGroup": "rg-dns-demo", "type": "Microsoft.Network/privateDnsZones" ... } ``` ### Link the DNS zone to a virtual network Create a virtual network link to enable DNS resolution from the linked VNet: ```bash VNET_ID=$(az network vnet show \ --name vnet-dns-demo \ --resource-group rg-dns-demo \ --query id \ --output tsv) az network private-dns link vnet create \ --name link-to-vnet-dns-demo \ --resource-group rg-dns-demo \ --zone-name privatelink.blob.core.windows.net \ --virtual-network "$VNET_ID" \ --registration-enabled false ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dns-demo/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net/virtualNetworkLinks/link-to-vnet-dns-demo", "location": "global", "name": "link-to-vnet-dns-demo", "provisioningState": "Succeeded", "registrationEnabled": false, "resourceGroup": "rg-dns-demo", "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", "virtualNetwork": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dns-demo/providers/Microsoft.Network/virtualNetworks/vnet-dns-demo" }, "virtualNetworkLinkState": "Completed" ... } ``` ### Show and list DNS zones Retrieve the details of the DNS zone and list all private DNS zones in the resource group: ```bash az network private-dns zone show \ --name privatelink.blob.core.windows.net \ --resource-group rg-dns-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dns-demo/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net", "location": "global", "name": "privatelink.blob.core.windows.net", "numberOfRecordSets": 1, "numberOfVirtualNetworkLinks": 1, "provisioningState": "Succeeded", "resourceGroup": "rg-dns-demo", "type": "Microsoft.Network/privateDnsZones" ... } ``` Then list all private DNS zones in the resource group: ```bash az network private-dns zone list \ --resource-group rg-dns-demo ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dns-demo/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net", "location": "global", "name": "privatelink.blob.core.windows.net", "numberOfRecordSets": 1, "numberOfVirtualNetworkLinks": 1, "provisioningState": "Succeeded", "resourceGroup": "rg-dns-demo", "type": "Microsoft.Network/privateDnsZones" } ] ``` ### Show and list virtual network links Retrieve the details of the virtual network link and list all links for the zone: ```bash az network private-dns link vnet show \ --name link-to-vnet-dns-demo \ --resource-group rg-dns-demo \ --zone-name privatelink.blob.core.windows.net ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dns-demo/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net/virtualNetworkLinks/link-to-vnet-dns-demo", "location": "global", "name": "link-to-vnet-dns-demo", "provisioningState": "Succeeded", "registrationEnabled": false, "resourceGroup": "rg-dns-demo", "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", "virtualNetwork": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dns-demo/providers/Microsoft.Network/virtualNetworks/vnet-dns-demo" }, "virtualNetworkLinkState": "Completed" ... } ``` Then list all virtual network links for the zone: ```bash az network private-dns link vnet list \ --resource-group rg-dns-demo \ --zone-name privatelink.blob.core.windows.net ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dns-demo/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net/virtualNetworkLinks/link-to-vnet-dns-demo", "location": "global", "name": "link-to-vnet-dns-demo", "provisioningState": "Succeeded", "registrationEnabled": false, "resourceGroup": "rg-dns-demo", "type": "Microsoft.Network/privateDnsZones/virtualNetworkLinks", "virtualNetwork": { "id": "...vnet-dns-demo...", "resourceGroup": "rg-dns-demo" }, "virtualNetworkLinkState": "Completed" } ] ``` ### Integrate with a private endpoint Private DNS zones are most useful when paired with a private endpoint, which automatically registers an A record in the zone when a DNS zone group is created. Create a subnet to host the private endpoint and a storage account as the target resource: ```bash az network vnet subnet create \ --name subnet-pe \ --resource-group rg-dns-demo \ --vnet-name vnet-dns-demo \ --address-prefixes 10.0.1.0/24 ``` ```bash az storage account create \ --name stdnsdemo \ --resource-group rg-dns-demo \ --location westeurope \ --sku Standard_LRS ``` Create a private endpoint for the blob service of the storage account: ```bash STORAGE_ID=$(az storage account show \ --name stdnsdemo \ --resource-group rg-dns-demo \ --query id \ --output tsv) az network private-endpoint create \ --name pe-blob-dns \ --resource-group rg-dns-demo \ --location westeurope \ --vnet-name vnet-dns-demo \ --subnet subnet-pe \ --private-connection-resource-id "$STORAGE_ID" \ --group-id blob \ --connection-name stdnsdemo-blob-connection ``` Create a DNS zone group that links the private endpoint to the private DNS zone. The emulator automatically registers an A record for the storage account name in the zone: ```bash az network private-endpoint dns-zone-group create \ --name default \ --resource-group rg-dns-demo \ --endpoint-name pe-blob-dns \ --private-dns-zone privatelink.blob.core.windows.net \ --zone-name blob-zone ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dns-demo/providers/Microsoft.Network/privateEndpoints/pe-blob-dns/privateDnsZoneGroups/default", "name": "default", "privateDnsZoneConfigs": [ { "name": "blob-zone", "privateDnsZoneId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dns-demo/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net", "recordSets": [ { "fqdn": "stdnsdemo.privatelink.blob.core.windows.net", "ipAddresses": [ "10.0.1.4" ], "provisioningState": "Succeeded", "recordSetName": "stdnsdemo", "recordType": "A", "ttl": 10 } ] } ], "provisioningState": "Succeeded", "resourceGroup": "rg-dns-demo" } ``` ### Verify IP address consistency Confirm that the private endpoint's network interface IP, the DNS zone group's embedded record set (shown above), and the A record in the private DNS zone all report the same address. Retrieve the IP address assigned to the private endpoint's network interface: ```bash NIC_ID=$(az network private-endpoint show \ --name pe-blob-dns \ --resource-group rg-dns-demo \ --query "networkInterfaces[0].id" \ --output tsv) az network nic show \ --ids "$NIC_ID" \ --query "ipConfigurations[0].privateIPAddress" \ --output tsv ``` ```bash title="Output" 10.0.1.4 ``` List the A records in the private DNS zone: ```bash az network private-dns record-set a list \ --resource-group rg-dns-demo \ --zone-name privatelink.blob.core.windows.net ``` ```bash title="Output" [ { "aRecords": [ { "ipv4Address": "10.0.1.4" } ], "fqdn": "stdnsdemo.privatelink.blob.core.windows.net", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-dns-demo/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net/A/stdnsdemo", "name": "stdnsdemo", "ttl": 10, "type": "Microsoft.Network/privateDnsZones/A" } ] ``` The IP address `10.0.1.4` — the first usable address in the `10.0.1.0/24` subnet — is consistent across all three: the network interface IP configuration, the DNS zone group record set, and the A record in the private DNS zone. ### Delete a virtual network link and DNS zone On Azure, a private DNS zone cannot be deleted while a private endpoint still references it through a DNS zone group, and the zone must have no remaining [virtual network links](https://learn.microsoft.com/en-us/cli/azure/network/private-dns/zone#az-network-private-dns-zone-delete). Remove the DNS zone group from the private endpoint first, then delete the link and the zone: ```bash az network private-endpoint dns-zone-group delete \ --name default \ --resource-group rg-dns-demo \ --endpoint-name pe-blob-dns az network private-dns link vnet delete \ --name link-to-vnet-dns-demo \ --resource-group rg-dns-demo \ --zone-name privatelink.blob.core.windows.net az network private-dns zone delete \ --name privatelink.blob.core.windows.net \ --resource-group rg-dns-demo ``` ## Features The Private DNS Zone emulator supports the following features: - **Create and manage private DNS zones**: Full lifecycle management including create, get, list, and delete. - **Virtual network links**: Create, get, list, and delete virtual network links that associate a DNS zone with a virtual network. - **Virtual network link settings**: The `registrationEnabled` flag on virtual network links is stored and returned. Automatic DNS registration from VMs in linked virtual networks is not emulated; see [Limitations](#limitations). - **Zone isolation by resource group**: The same DNS zone name can exist independently in multiple resource groups. - **Tags**: Apply and update resource tags on private DNS zone resources. - **Cross-virtual-network support**: Link a single DNS zone to multiple virtual networks. ## Limitations - **No DNS resolution**: Private DNS Zone is a mock implementation. DNS queries are not resolved using the zone's records; no actual DNS lookup behavior is simulated. - **Limited record set management**: A records are automatically created in a private DNS zone when a private endpoint DNS zone group is created. Manual record set management (AAAA, CNAME, MX, and other record types) is not supported. - **No auto-registration**: The `registrationEnabled` flag is stored and returned, but no automatic DNS record registration from linked VMs is performed. - **No data persistence**: Private DNS zone resources are not persisted and are lost when the emulator is stopped or restarted. ## Samples The following samples demonstrate how to use Azure Private DNS Zones with LocalStack for Azure: - [Function App and Service Bus](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-service-bus/dotnet/) - [Web App and Cosmos DB for MongoDB API](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-cosmosdb-mongodb-api/python/) ## API Coverage # Private Endpoint > Get started with Azure Private Endpoint on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Private Endpoint is a network interface that connects you privately and securely to a service powered by Azure Private Link. Private endpoints use a private IP address from your virtual network, effectively bringing the service into your virtual network and eliminating exposure to the public internet. They are commonly used to access Azure PaaS services, such as Storage or Cosmos DB, from within a private network without internet connectivity. For more information, see [What is Azure Private Endpoint?](https://learn.microsoft.com/en-us/azure/private-link/private-endpoint-overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Private Endpoints. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Private Endpoint's integration with LocalStack. ## Getting started This guide is designed for users new to Private Endpoint and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group, virtual network, and subnet Private endpoints are assigned a private IP from a subnet. Create the prerequisites first: ```bash az group create \ --name rg-pe-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo", "location": "westeurope", "managedBy": null, "name": "rg-pe-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` Create a virtual network for the private endpoint: ```bash az network vnet create \ --name vnet-pe-demo \ --resource-group rg-pe-demo \ --location westeurope \ --address-prefixes 10.0.0.0/16 ``` ```bash title="Output" { "newVNet": { "addressSpace": { "addressPrefixes": [ "10.0.0.0/16" ] }, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Network/virtualNetworks/vnet-pe-demo", "location": "westeurope", "name": "vnet-pe-demo", "provisioningState": "Succeeded", "resourceGroup": "rg-pe-demo", "subnets": [], "type": "Microsoft.Network/virtualNetworks", ... } } ``` Create a subnet within the virtual network to host the private endpoint: ```bash az network vnet subnet create \ --name subnet-pe \ --resource-group rg-pe-demo \ --vnet-name vnet-pe-demo \ --address-prefixes 10.0.1.0/24 ``` ```bash title="Output" { "addressPrefix": "10.0.1.0/24", "delegations": [], "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Network/virtualNetworks/vnet-pe-demo/subnets/subnet-pe", "name": "subnet-pe", "provisioningState": "Succeeded", "resourceGroup": "rg-pe-demo", "type": "Microsoft.Network/virtualNetworks/subnets" ... } ``` ### Create a storage account (target resource) Private endpoints connect to a specific Azure resource. Create a storage account as the target: ```bash az storage account create \ --name stpedemo \ --resource-group rg-pe-demo \ --location westeurope \ --sku Standard_LRS ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Storage/storageAccounts/stpedemo", "kind": "StorageV2", "location": "westeurope", "name": "stpedemo", "primaryEndpoints": { "blob": "https://stpedemo.blob.core.azure.localhost.localstack.cloud:4566", "file": "https://stpedemo.file.core.azure.localhost.localstack.cloud:4566", "queue": "https://stpedemo.queue.core.azure.localhost.localstack.cloud:4566", "table": "https://stpedemo.table.core.azure.localhost.localstack.cloud:4566", ... }, "provisioningState": "Succeeded", "resourceGroup": "rg-pe-demo", "sku": { "name": "Standard_LRS", "tier": "Standard" }, "type": "Microsoft.Storage/storageAccounts", ... } ``` ### Create a private endpoint Retrieve the storage account resource ID and create a private endpoint targeting the blob sub-resource: ```bash STORAGE_ID=$(az storage account show \ --name stpedemo \ --resource-group rg-pe-demo \ --query id \ --output tsv) az network private-endpoint create \ --name pe-storage-blob \ --resource-group rg-pe-demo \ --location westeurope \ --vnet-name vnet-pe-demo \ --subnet subnet-pe \ --private-connection-resource-id "$STORAGE_ID" \ --group-id blob \ --connection-name stpedemo-blob-connection ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Network/privateEndpoints/pe-storage-blob", "location": "westeurope", "name": "pe-storage-blob", "networkInterfaces": [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Network/networkInterfaces/pe-storage-blob.nic.8388c53a-14fc-4a02-a517-5e462aab03be", "resourceGroup": "rg-pe-demo" } ], "privateLinkServiceConnections": [ { "groupIds": [ "blob" ], "name": "stpedemo-blob-connection", "privateLinkServiceConnectionState": { "actionsRequired": "None", "description": "Auto-Approved", "status": "Approved" }, "privateLinkServiceId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Storage/storageAccounts/stpedemo", "provisioningState": "Succeeded" } ], "provisioningState": "Succeeded", "resourceGroup": "rg-pe-demo", "subnet": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Network/virtualNetworks/vnet-pe-demo/subnets/subnet-pe" }, "type": "Microsoft.Network/privateEndpoints" ... } ``` ### Configure a private DNS zone group Link the private endpoint to a private DNS zone for name resolution: ```bash az network private-dns zone create \ --name privatelink.blob.core.windows.net \ --resource-group rg-pe-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net", "location": "global", "name": "privatelink.blob.core.windows.net", "provisioningState": "Succeeded", "resourceGroup": "rg-pe-demo", "type": "Microsoft.Network/privateDnsZones" ... } ``` Create a DNS zone group to associate the private DNS zone with the private endpoint: ```bash az network private-endpoint dns-zone-group create \ --name dns-zone-group-blob \ --resource-group rg-pe-demo \ --endpoint-name pe-storage-blob \ --private-dns-zone privatelink.blob.core.windows.net \ --zone-name blob-zone ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Network/privateEndpoints/pe-storage-blob/privateDnsZoneGroups/dns-zone-group-blob", "name": "dns-zone-group-blob", "privateDnsZoneConfigs": [ { "name": "blob-zone", "privateDnsZoneId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net", "recordSets": [ { "fqdn": "stpedemo.privatelink.blob.core.windows.net", "ipAddresses": [ "10.0.1.4" ], "provisioningState": "Succeeded", "recordSetName": "stpedemo", "recordType": "A", "ttl": 10 } ] } ], "provisioningState": "Succeeded", "resourceGroup": "rg-pe-demo" } ``` ### Show and list private endpoints Retrieve the details of the private endpoint and list all endpoints in the resource group: ```bash az network private-endpoint show \ --name pe-storage-blob \ --resource-group rg-pe-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Network/privateEndpoints/pe-storage-blob", "location": "westeurope", "name": "pe-storage-blob", "privateLinkServiceConnections": [ { "groupIds": [ "blob" ], "name": "stpedemo-blob-connection", "privateLinkServiceConnectionState": { "status": "Approved", ... }, "privateLinkServiceId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Storage/storageAccounts/stpedemo", "provisioningState": "Succeeded" } ], "provisioningState": "Succeeded", "resourceGroup": "rg-pe-demo", "type": "Microsoft.Network/privateEndpoints" ... } ``` Then list all private endpoints in the resource group: ```bash az network private-endpoint list \ --resource-group rg-pe-demo ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Network/privateEndpoints/pe-storage-blob", "location": "westeurope", "name": "pe-storage-blob", "privateLinkServiceConnections": [ { "groupIds": [ "blob" ], "name": "stpedemo-blob-connection", ... } ], "provisioningState": "Succeeded", "resourceGroup": "rg-pe-demo", "type": "Microsoft.Network/privateEndpoints", ... } ] ``` ### List DNS zone groups List the DNS zone groups associated with the private endpoint: ```bash az network private-endpoint dns-zone-group list \ --resource-group rg-pe-demo \ --endpoint-name pe-storage-blob ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Network/privateEndpoints/pe-storage-blob/privateDnsZoneGroups/dns-zone-group-blob", "name": "dns-zone-group-blob", "privateDnsZoneConfigs": [ { "name": "blob-zone", "privateDnsZoneId": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net", "recordSets": [ { "fqdn": "stpedemo.privatelink.blob.core.windows.net", "ipAddresses": [ "10.0.1.4" ], "recordType": "A", ... } ] } ], "provisioningState": "Succeeded" } ] ``` ### Verify IP address consistency When a DNS zone group is created, the emulator automatically registers an A record in the linked private DNS zone using the private IP address assigned to the private endpoint's network interface. The same address appears in the NIC's IP configuration, the DNS zone group's embedded record set (shown in the output above), and the A record in the private DNS zone. Retrieve the IP address from the private endpoint's network interface: ```bash NIC_ID=$(az network private-endpoint show \ --name pe-storage-blob \ --resource-group rg-pe-demo \ --query "networkInterfaces[0].id" \ --output tsv) az network nic show \ --ids "$NIC_ID" \ --query "ipConfigurations[0].privateIPAddress" \ --output tsv ``` ```bash title="Output" 10.0.1.4 ``` List the A records in the private DNS zone: ```bash az network private-dns record-set a list \ --resource-group rg-pe-demo \ --zone-name privatelink.blob.core.windows.net ``` ```bash title="Output" [ { "aRecords": [ { "ipv4Address": "10.0.1.4" } ], "fqdn": "stpedemo.privatelink.blob.core.windows.net", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pe-demo/providers/Microsoft.Network/privateDnsZones/privatelink.blob.core.windows.net/A/stpedemo", "name": "stpedemo", "ttl": 10, "type": "Microsoft.Network/privateDnsZones/A" } ] ``` The IP address `10.0.1.4` is consistent across all three: the network interface IP configuration, the DNS zone group record set, and the A record in the private DNS zone. On Azure, the first four addresses in each subnet are reserved (for `10.0.1.0/24`, that is `10.0.1.0`–`10.0.1.3`), so `10.0.1.4` is typically the first assignable address for resources such as a private endpoint. The emulator surfaces the same value across these APIs for consistency even though it does not enforce real subnet allocation (see [Limitations](#limitations)). ### Delete a private endpoint Delete the private endpoint and verify it no longer appears in the list: ```bash az network private-endpoint delete \ --name pe-storage-blob \ --resource-group rg-pe-demo ``` Then list all private endpoints to confirm the resource group is now empty: ```bash az network private-endpoint list --resource-group rg-pe-demo ``` ```bash title="Output" [] ``` ## Features The Private Endpoint emulator supports the following features: - **Create and manage private endpoints**: Full lifecycle management including create, get, update, list, and delete. - **Private Link service connections**: Store and return `privateLinkServiceConnections` with group IDs and connection names. - **DNS zone group management**: Create, get, list, and delete private DNS zone groups for a private endpoint. - **Group IDs**: Support for any service group ID (e.g., `blob`, `file`, `queue`, `table`, `sqlServer`). - **Resource group-scoped listing**: List all private endpoints in a resource group. - **Tags**: Apply and update resource tags on private endpoint resources. ## Limitations - **DNS resolution not performed**: A records are automatically registered in private DNS zones when a DNS zone group is created. However, no actual DNS resolution is performed; the emulator does not respond to DNS queries. - **No private IP assignment enforcement**: A private IP is stored but no actual address is reserved from the subnet range. - **No target service connectivity**: The emulator does not verify that the target resource or connection is valid. - **No data persistence**: Private endpoint resources are not persisted and are lost when the emulator is stopped or restarted. ## Samples The following samples demonstrate how to use Azure Private Endpoints with LocalStack for Azure: - [Function App and Service Bus](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-service-bus/dotnet/) - [Web App and Cosmos DB for MongoDB API](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-cosmosdb-mongodb-api/python/README.md) ## API Coverage # Public IP Address > Get started with Azure Public IP Address on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Public IP Address is a resource that provides a static or dynamic IP address reachable from the internet. Public IP addresses are assigned to Azure resources such as virtual machines, load balancers, application gateways, VPN gateways, and bastion hosts. They are commonly used when resources need to accept inbound internet connections or require a stable outbound identity. For more information, see [Public IP addresses](https://learn.microsoft.com/en-us/azure/virtual-network/ip-services/public-ip-addresses). LocalStack for Azure provides a local environment for building and testing applications that make use of Public IP Addresses. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Public IP Address's integration with LocalStack. ## Getting started This guide is designed for users new to Public IP Addresses and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create \ --name rg-pip-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pip-demo", "location": "westeurope", "managedBy": null, "name": "rg-pip-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a public IP address Create a static Standard SKU public IP address: ```bash az network public-ip create \ --name pip-demo \ --resource-group rg-pip-demo \ --location westeurope \ --sku Standard \ --allocation-method Static ``` ```bash title="Output" { "publicIp": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pip-demo/providers/Microsoft.Network/publicIPAddresses/pip-demo", "idleTimeoutInMinutes": 4, "ipAddress": "20.115.40.220", "location": "westeurope", "name": "pip-demo", "provisioningState": "Succeeded", "publicIPAddressVersion": "IPv4", "publicIPAllocationMethod": "Static", "resourceGroup": "rg-pip-demo", "sku": { "name": "Standard", "tier": "Regional" }, "type": "Microsoft.Network/publicIPAddresses", ... } } ``` ### Get and list public IP addresses Retrieve the details of the public IP address and list all public IPs in the resource group: ```bash az network public-ip show \ --name pip-demo \ --resource-group rg-pip-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pip-demo/providers/Microsoft.Network/publicIPAddresses/pip-demo", "idleTimeoutInMinutes": 4, "ipAddress": "20.115.40.220", "location": "westeurope", "name": "pip-demo", "provisioningState": "Succeeded", "publicIPAddressVersion": "IPv4", "publicIPAllocationMethod": "Static", "resourceGroup": "rg-pip-demo", "sku": { "name": "Standard", "tier": "Regional" }, "type": "Microsoft.Network/publicIPAddresses", ... } ``` Then list all public IP addresses in the resource group: ```bash az network public-ip list \ --resource-group rg-pip-demo ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pip-demo/providers/Microsoft.Network/publicIPAddresses/pip-demo", "ipAddress": "20.115.40.220", "location": "westeurope", "name": "pip-demo", "provisioningState": "Succeeded", "publicIPAddressVersion": "IPv4", "publicIPAllocationMethod": "Static", "sku": { "name": "Standard", "tier": "Regional" }, "type": "Microsoft.Network/publicIPAddresses", ... } ] ``` ### List all public IPs in a subscription List every public IP address across the entire subscription: ```bash az network public-ip list \ --output table ``` ```bash title="Output" Name ResourceGroup Location Zones Address AddressVersion AllocationMethod IdleTimeoutInMinutes ProvisioningState -------- --------------- ---------- ------- ------------- ---------------- ------------------ ---------------------- ------------------- pip-demo rg-pip-demo westeurope 20.115.40.220 IPv4 Static 4 Succeeded ``` To show the same resources scoped to one resource group: ```bash az network public-ip list \ --resource-group rg-pip-demo \ --output table ``` ```bash title="Output" Name ResourceGroup Location Zones Address AddressVersion AllocationMethod IdleTimeoutInMinutes ProvisioningState -------- --------------- ---------- ------- ------------- ---------------- ------------------ ---------------------- ------------------- pip-demo rg-pip-demo westeurope 20.115.40.220 IPv4 Static 4 Succeeded ``` ### Delete the public IP address Delete the public IP address and verify it no longer appears in the list: ```bash az network public-ip delete \ --name pip-demo \ --resource-group rg-pip-demo ``` Then list all public IP addresses to confirm the resource group is now empty: ```bash az network public-ip list --resource-group rg-pip-demo ``` ```bash title="Output" [] ``` ## Features The Public IP Address emulator supports the following features: - **Create and manage public IPs**: Full lifecycle management including create, get, update, list, and delete. - **Static and dynamic allocation**: Configure `Static` or `Dynamic` IP allocation methods. - **SKU tiers**: Support for `Basic` and `Standard` SKUs. - **IPv4 and IPv6**: Store and return the IP address version. - **DNS name labels**: Associate a DNS name label with a public IP. - **Tags**: Apply and update resource tags on public IP address resources. - **Resource group and subscription-scoped listing**: List public IPs scoped to a resource group or across the entire subscription. - **Zone configuration**: Record and return availability zone assignments. ## Limitations - **No real IP allocation**: Public IP Address is a mock implementation. No actual IP addresses are allocated from Azure's public IP pools, and no public internet connectivity is provided. - **Allocation method not enforced**: Dynamic and Static allocation methods are stored and returned, but no real IP assignment behavior is simulated. - **No data persistence**: Public IP address resources are not persisted and are lost when the emulator is stopped or restarted. ## Samples The following samples demonstrate how to use Azure Public IP Addresses with LocalStack for Azure: - [Function App and Service Bus](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-service-bus/dotnet/) - [Web App and Cosmos DB for MongoDB API](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-cosmosdb-mongodb-api/python/) ## API Coverage # Public IP Prefix > Get started with Azure Public IP Prefix on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Public IP Prefix is a contiguous range of Standard SKU static public IP addresses. When you create a public IP address from a prefix, the address is guaranteed to stay within the prefix range, making it useful for allow-listing IP ranges in external firewalls. Public IP Prefixes are commonly used with NAT Gateway to provide predictable outbound IP addresses for entire subnets. For more information, see [Public IP address prefixes](https://learn.microsoft.com/en-us/azure/virtual-network/ip-services/public-ip-address-prefix). LocalStack for Azure provides a local environment for building and testing applications that make use of Public IP Prefixes. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Public IP Prefix's integration with LocalStack. ## Getting started This guide is designed for users new to Public IP Prefixes and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create \ --name rg-pip-prefix-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pip-prefix-demo", "location": "westeurope", "managedBy": null, "name": "rg-pip-prefix-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a public IP prefix Create a /29 public IP prefix (8 IP addresses): ```bash az network public-ip prefix create \ --name pip-prefix-demo \ --resource-group rg-pip-prefix-demo \ --location westeurope \ --length 29 ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pip-prefix-demo/providers/Microsoft.Network/publicIPPrefixes/pip-prefix-demo", "location": "westeurope", "name": "pip-prefix-demo", "properties": { "ipPrefix": "20.184.13.0/29", "ipTags": [], "prefixLength": 29, "provisioningState": "Succeeded", "publicIPAddressVersion": "IPv4", "resourceGuid": "00000000-0000-0000-0000-000000000000" ... }, "resourceGroup": "rg-pip-prefix-demo", "sku": { "name": "Standard", "tier": "Regional" }, "type": "Microsoft.Network/publicIPPrefixes", "zones": [] ... } ``` ### Get and list public IP prefixes Retrieve the details of the public IP prefix and list all prefixes in the resource group: ```bash az network public-ip prefix show \ --name pip-prefix-demo \ --resource-group rg-pip-prefix-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pip-prefix-demo/providers/Microsoft.Network/publicIPPrefixes/pip-prefix-demo", "location": "westeurope", "name": "pip-prefix-demo", "properties": { "ipPrefix": "20.184.13.0/29", "ipTags": [], "prefixLength": 29, "provisioningState": "Succeeded", "publicIPAddressVersion": "IPv4", "resourceGuid": "00000000-0000-0000-0000-000000000000" ... }, "resourceGroup": "rg-pip-prefix-demo", "sku": { "name": "Standard", "tier": "Regional" }, "type": "Microsoft.Network/publicIPPrefixes", "zones": [] ... } ``` Then list all public IP prefixes in the resource group: ```bash az network public-ip prefix list \ --resource-group rg-pip-prefix-demo ``` ```bash title="Output" [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-pip-prefix-demo/providers/Microsoft.Network/publicIPPrefixes/pip-prefix-demo", "location": "westeurope", "name": "pip-prefix-demo", "properties": { "ipPrefix": "20.184.13.0/29", "ipTags": [], "prefixLength": 29, "provisioningState": "Succeeded", "publicIPAddressVersion": "IPv4" }, "resourceGroup": "rg-pip-prefix-demo", "sku": { "name": "Standard", "tier": "Regional" }, "type": "Microsoft.Network/publicIPPrefixes", "zones": [] } ] ``` ### Delete the public IP prefix Delete the public IP prefix and verify it no longer appears in the list: ```bash az network public-ip prefix delete \ --name pip-prefix-demo \ --resource-group rg-pip-prefix-demo ``` Then list all public IP prefixes to confirm the resource group is now empty: ```bash az network public-ip prefix list --resource-group rg-pip-prefix-demo ``` ```bash title="Output" [] ``` ## Features The Public IP Prefix emulator supports the following features: - **Create and manage prefixes**: Full lifecycle management including create, get, update, list, and delete. - **Configurable prefix length**: For IPv4, set prefix length /28 through /31 to define how many addresses the range contains (/28 = 16, /29 = 8, /30 = 4, /31 = 2 addresses), matching Azure’s [published prefix sizes](https://learn.microsoft.com/en-us/azure/virtual-network/ip-services/public-ip-address-prefix#prefix-sizes). IPv6 uses /124–/127 for the same address counts when you configure an IPv6 prefix in Azure. - **Tags**: Apply and update resource tags on public IP prefix resources. - **Subscription-scoped listing**: List all public IP prefixes across a subscription. - **Multiple prefix lengths**: Create prefixes of different lengths within the same resource group. ## Limitations - **No real IP range allocation**: Public IP Prefix is a mock implementation. No actual IP address ranges are allocated from Azure's public IP pools, and no public internet connectivity is provided. - **No IP address derivation**: Creating individual public IP addresses from a prefix is not enforced; both resources are stored independently. - **No data persistence**: Public IP prefix resources are not persisted and are lost when the emulator is stopped or restarted. ## Samples The following samples demonstrate how to use Azure Public IP Prefixes with LocalStack for Azure: - [Function App and Service Bus](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-service-bus/dotnet/) - [Web App and Cosmos DB for MongoDB API](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-cosmosdb-mongodb-api/python/) ## API Coverage # Queue Storage > Get started with Azure Queue Storage on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Queue Storage is a messaging service for storing large numbers of messages that can be accessed from anywhere over HTTP or HTTPS. It is commonly used to decouple application components and build asynchronous processing workflows. Queue Storage is useful for buffering work items between producers and consumers. For more information, see [What is Azure Queue Storage?](https://learn.microsoft.com/en-us/azure/storage/queues/storage-queues-introduction) LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Queue Storage. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Queue Storage's integration with LocalStack. ## Getting started This guide is designed for users new to Queue Storage and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` ### Create a resource group Create a resource group to contain your storage resources: ```bash az group create \ --name rg-queue-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-queue-demo", "location": "westeurope", "managedBy": null, "name": "rg-queue-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a storage account Create a storage account in the resource group: ```bash az storage account create \ --name stqueuedemols \ --resource-group rg-queue-demo \ --location westeurope \ --sku Standard_LRS ``` ```bash title="Output" { ... "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-queue-demo/providers/Microsoft.Storage/storageAccounts/stqueuedemols", ... "name": "stqueuedemols", ... "placement": null, "primaryEndpoints": { "blob": "https://stqueuedemols.blob.core.azure.localhost.localstack.cloud:4566", ... "queue": "https://stqueuedemols.queue.core.azure.localhost.localstack.cloud:4566", ... }, .... } ``` ### Authentication There are three ways to authenticate storage queue commands against the emulator: #### Storage account key Retrieve the account key and pass it with `--account-name` and `--account-key`: ```bash ACCOUNT_KEY=$(az storage account keys list \ --account-name stqueuedemols \ --resource-group rg-queue-demo \ --query "[0].value" \ --output tsv) az storage queue list \ --account-name stqueuedemols \ --account-key "$ACCOUNT_KEY" ``` #### Login credentials Use `--auth-mode login` to authenticate with the current session credentials: ```bash az storage queue list \ --account-name stqueuedemols \ --auth-mode login ``` #### Connection string Bundle the account name and key into a single value: ```bash CONNECTION_STRING=$(az storage account show-connection-string \ --name stqueuedemols \ --resource-group rg-queue-demo \ --query connectionString -o tsv) az storage queue list \ --connection-string "$CONNECTION_STRING" ``` The remaining examples in this guide use connection strings for brevity. ### Create and inspect a queue Create a queue: ```bash az storage queue create \ --name app-queue \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" { "created": true } ``` Verify the queue exists: ```bash az storage queue exists \ --name app-queue \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" { "exists": true } ``` List queues in the storage account: ```bash az storage queue list \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" [ { "approximateMessageCount": null, "metadata": null, "name": "app-queue" } ] ``` ### Put, peek, and get messages Add a message to the queue: ```bash az storage message put \ --queue-name app-queue \ --content "hello-from-localstack" \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" { "content": "hello-from-localstack", ... "id": "a253ff4a-7b9c-434e-9c33-deae3070193c", ... } ``` Peek at messages without consuming them: ```bash az storage message peek \ --queue-name app-queue \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" [ { "content": "hello-from-localstack", ... "id": "a253ff4a-7b9c-434e-9c33-deae3070193c", "insertionTime": "2026-02-27T07:45:14+00:00", ... } ] ``` Get (dequeue) a message from the queue, which makes it invisible to other consumers for the visibility timeout you set (when you omit `--visibility-timeout`, Azure Queue Storage uses a default of 30 seconds; see [Get Messages](https://learn.microsoft.com/en-us/rest/api/storageservices/get-messages)): ```bash az storage message get \ --queue-name app-queue \ --connection-string "$CONNECTION_STRING" \ --output json ``` ```bash title="Output" [ { "content": "hello-from-localstack", ... "id": "a253ff4a-7b9c-434e-9c33-deae3070193c", "popReceipt": "...", ... } ] ``` ## Features The Queue Storage emulator supports the following features: - **Data plane REST API**: Queue CRUD, message operations (put, peek, get, delete), queue metadata, stored access policies, and SAS token generation. - **Control plane REST API**: Create and get queues, get and set queue service properties via Azure Resource Manager. - **Multiple authentication modes**: Storage account key, login credentials, and connection strings. ## Limitations - **No data persistence across restarts**: Queue data is not persisted and is lost when the LocalStack emulator is stopped or restarted. - **Queue service properties**: `set_service_properties` is a no-op and `get_service_properties` returns empty defaults, unlike Azure where CORS, logging, and metrics settings are persisted and applied. - **Storage account keys**: Keys are emulator-generated rather than managed by Azure. - **Header validation**: Unsupported request headers or parameters are silently accepted instead of being rejected. - **API version enforcement**: The emulator does not validate the `x-ms-version` header; all API versions are accepted. - **RBAC enforcement is opt-in**: By default, data-plane operations succeed regardless of role assignments. Set `LS_AZURE_ENFORCE_RBAC` to require the caller to hold a role such as `Storage Queue Data Contributor`; see [Role Assignment: Enabling RBAC enforcement](/azure/services/role-assignment/#enabling-rbac-enforcement). ## Samples The following sample demonstrates how to use Queue Storage with LocalStack for Azure: - [Azure Functions Sample with LocalStack for Azure](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-storage-http/dotnet) ## API Coverage # Resource Graph > Get started with Azure Resource Graph on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Resource Graph is a service for querying Azure resources at scale using a structured query language. It helps you search, filter, and project resource metadata across subscriptions. Resource Graph is useful for inventory, governance checks, and automated analysis workflows. For more information, see [Azure Resource Graph overview](https://learn.microsoft.com/en-us/azure/governance/resource-graph/overview). LocalStack for Azure enables users to explore resources deployed within the local environment using [Kusto Query Language (KQL)](https://learn.microsoft.com/en-us/azure/governance/resource-graph/concepts/query-language#supported-tabulartop-level-operators). ## Getting started This guide is designed for users new to Resource Graph and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group for the resources you want to query: ```bash az group create \ --name rg-resourcegraph-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resourcegraph-demo", "location": "westeurope", "managedBy": null, "name": "rg-resourcegraph-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create sample web resources Create an App Service plan and a Web App, which we will query using Resource Graph: ```bash az appservice plan create \ --name asp-doc81 \ --resource-group rg-resourcegraph-demo \ --location westeurope \ --sku F1 az webapp create \ --name ls-app-doc81 \ --resource-group rg-resourcegraph-demo \ --plan asp-doc81 \ --runtime "PYTHON:3.11" ``` ```bash title="Output" { "asyncScalingEnabled": false, "elasticScaleEnabled": false, "geoRegion": "West Europe", "hyperV": false, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resourcegraph-demo/providers/Microsoft.Web/serverfarms/asp-doc81", ... "name": "asp-doc81", ... "status": "Ready", "subscription": "00000000-0000-0000-0000-000000000000", ... } { ... "enabledHostNames": [ "ls-app-doc81.azurewebsites.net", "ls-app-doc81.scm.azurewebsites.net" ], ... "hostNameSslStates": [ { "hostType": "Standard", "name": "ls-app-doc81.azurewebsites.net", "sslState": "Disabled", "thumbprint": null, "toUpdate": null, "virtualIp": null }, ... ], "hostNames": [ "ls-app-doc81.azurewebsites.net" ], ... } ``` ### Query resources with Resource Graph The Resource Graph extension must be installed to use `az graph` commands. Refer to the [official installation instructions](https://learn.microsoft.com/en-us/azure/governance/resource-graph/first-query-azurecli#install-the-extension). Use the [az graph query](https://learn.microsoft.com/en-us/cli/azure/graph?view=azure-cli-latest#az-graph-query) command to run Kusto Query language (KQL) queries against the emulator. The following examples demonstrate common query patterns: ```bash az graph query \ --graph-query "Resources | where type =~ 'Microsoft.Web/sites'" ``` ```bash title="Output" { "count": 1, "data": [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resourcegraph-demo/providers/Microsoft.Web/sites/ls-app-doc81", "location": "westeurope", "name": "ls-app-doc81", "properties": {}, "resourceGroup": "rg-resourcegraph-demo", "subscriptionId": "00000000-0000-0000-0000-000000000000", "type": "Microsoft.Web/sites" } ], "skip_token": null, "total_records": 1 } ``` Query a web site by type and name, projecting only the `id` column: ```bash az graph query \ --graph-query "Resources | where type =~ 'Microsoft.Web/sites' and name =~ 'ls-app-doc81' | project id" ``` ```bash title="Output" { "count": 1, "data": [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resourcegraph-demo/providers/Microsoft.Web/sites/ls-app-doc81", "resourceGroup": "rg-resourcegraph-demo" } ], "skip_token": null, "total_records": 1 } ``` Count all resources in a resource group: ```bash az graph query \ --graph-query "Resources | where resourceGroup =~ 'rg-resourcegraph-demo' | count" ``` ```bash title="Output" { "count": 1, "data": [ { "Count": 2 } ], "skip_token": null, "total_records": 1 } ``` List resources sorted by name with a row limit: ```bash az graph query \ --graph-query "Resources | where resourceGroup =~ 'rg-resourcegraph-demo' | project name, type | order by name asc | limit 5" ``` ```bash title="Output" { "count": 2, "data": [ { "name": "asp-doc81", "type": "Microsoft.Web/serverfarms" }, { "name": "ls-app-doc81", "type": "Microsoft.Web/sites" } ], "skip_token": null, "total_records": 2 } ``` Query a resource that does not exist: ```bash az graph query \ --graph-query "Resources | where type =~ 'Microsoft.Web/sites' and name =~ 'doesnotexist'" ``` ```bash title="Output" { "count": 0, "data": [], "skip_token": null, "total_records": 0 } ``` ### Query resources with the REST API An alternative way to invoke Azure Resource Graph is to call its REST API directly using the [`az rest`](https://learn.microsoft.com/en-us/cli/azure/reference-index?view=azure-cli-latest#az-rest) command: ```bash az rest --method post \ --url "http://azure.localhost.localstack.cloud:4566/providers/Microsoft.ResourceGraph/resources?api-version=2024-04-01" \ --headers "Content-Type=application/json" \ --body "{\"subscriptions\":[\"00000000-0000-0000-0000-000000000000\"],\"query\":\"Resources | where type=~'Microsoft.Web/sites'\", \"options\":{\"resultFormat\":\"objectArray\"}}" ``` ```bash title="Output" { "count": 1, "data": [ { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resourcegraph-demo/providers/Microsoft.Web/sites/ls-app-doc81", "location": "westeurope", "name": "ls-app-doc81", "properties": {}, "resourceGroup": "rg-resourcegraph-demo", "subscriptionId": "00000000-0000-0000-0000-000000000000", "type": "Microsoft.Web/sites" } ], "facets": [], "resultTruncated": "false", "totalRecords": 1 } ``` ## Features The Resource Graph emulator supports the following features: - **KQL query engine**: A built-in parser and executor for the Kusto Query Language (KQL) subset used by Azure Resource Graph. - **Tabular and object result formats**: The `resultFormat` option controls whether results are returned as a column/row table or as an array of objects. - **Scalar functions**: Built-in functions including `tolower`, `toupper`, `strlen`, `trim`, `substring`, `strcat`, `isnull`, `isnotnull`, `isempty`, `isnotempty`, `tostring`, `toint`, `tolong`, `todouble`, and `coalesce`. - **Comparison operators**: Full support for `==`, `!=`, `=~`, `!~`, `contains`, `!contains`, `contains_cs`, `!contains_cs`, `startswith`, `!startswith`, `endswith`, `!endswith`, `has`, `!has`, `in`, `!in`, and `matches regex`. - **Aggregate functions**: `count()`, `dcount()`, `countif()`, `sum()`, `sumif()`, `avg()`, `min()`, and `max()` for use in `summarize` stages. ## Limitations - **Single table only**: The emulator queries the `Resources` table. Other Resource Graph tables (such as `ResourceContainers`, `AdvisorResources`, and `SecurityResources`) are not available. - **No data persistence across restarts**: Resource metadata is not persisted and is lost when the LocalStack emulator is stopped or restarted. ### Supported tabular operators The table below lists the KQL tabular operators supported by Azure Resource Graph and their availability in the LocalStack emulator. For the full reference, see [Supported tabular/top-level operators](https://learn.microsoft.com/en-us/azure/governance/resource-graph/concepts/query-language#supported-tabulartop-level-operators). | Operator | Supported | Notes | |---|---|---| | `count` | Yes | Returns a single row with the total number of input rows. | | `distinct` | Yes | Deduplicates rows by the specified columns. | | `extend` | Yes | Adds computed columns to the result set. | | `join` | No | Cross-table joins are not supported. The emulator does not implement `ResourceContainers` or other secondary tables. | | `limit` | Yes | Synonym of `take`. | | `mv-expand` | No | Array expansion into multiple rows is not supported. | | `order` | Yes | Synonym of `sort`. Supports `asc` and `desc` directions. | | `parse` | No | String parsing with pattern matching is not supported. | | `project` | Yes | Supports column selection and aliased expressions. | | `project-away` | Yes | Removes specified columns from the result set. | | `sort` | Yes | Synonym of `order`. | | `summarize` | Yes | Supports aggregate functions with an optional `by` clause. | | `take` | Yes | Synonym of `limit`. | | `top` | Yes | Returns the first N rows sorted by specified columns. | | `union` | No | Combining results from multiple tables is not supported. | | `where` | Yes | Filters rows using comparison, logical, and string operators. | ## API Coverage # Resource Manager > Get started with Azure Resource Manager on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Resource Manager (ARM) is the unified deployment and management layer for Azure resources, providing a consistent control-plane API for resource organization and lifecycle management. It enables idempotent, declarative infrastructure as code (IaC) through JSON-based ARM templates and Bicep modules, allowing for automated resource group orchestration and provider registrations. For more information, see: - [What is Azure Resource Manager?](https://learn.microsoft.com/azure/azure-resource-manager/management/overview)](https://learn.microsoft.com/azure/azure-resource-manager/management/overview) - [What are ARM templates?](https://learn.microsoft.com/azure/azure-resource-manager/templates/overview)](https://learn.microsoft.com/azure/azure-resource-manager/templates/overview) - [What is Bicep?](https://learn.microsoft.com/azure/azure-resource-manager/bicep/overview)](https://learn.microsoft.com/azure/azure-resource-manager/bicep/overview) LocalStack for Azure enables seamless interaction with the emulator’s management REST API via Azure Resource Manager. It also provides native support for Bicep and ARM templates, allowing for standardized Infrastructure as Code (IaC) deployments within your local environment. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Resource Manager's integration with LocalStack. ## Getting started This guide is designed for users new to Resource Manager and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group: ```bash az group create \ --name rg-resources-demo \ --location westeurope ``` ```bash title="Output" { "name": "rg-resources-demo", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resources-demo", "location": "westeurope", "properties": { "provisioningState": "Succeeded" }, "..." } ``` ### Get and list resource groups Get the resource group details: ```bash az group show --name rg-resources-demo ``` ```bash title="Output" { "name": "rg-resources-demo", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resources-demo", "location": "westeurope", "properties": { "provisioningState": "Succeeded" }, "..." } ``` List matching resource groups: ```bash az group list --query "[?name=='rg-resources-demo']" ``` ```bash title="Output" [ { "name": "rg-resources-demo", "location": "westeurope", "..." } ] ``` ### Query resource providers Get a specific provider: ```bash az provider show --namespace Microsoft.Resources ``` ```bash title="Output" { "namespace": "Microsoft.Resources", "registrationState": "Registered", "registrationPolicy": "RegistrationFree", "resourceTypes": [ { "resourceType": "resourceGroups", "apiVersions": ["2023-07-01", "..."], "..." } ... ], "..." } ``` List provider registration state: ```bash az provider list --query "[?namespace=='Microsoft.Resources'].{namespace:namespace,registrationState:registrationState}" ``` ```bash title="Output" [ { "namespace": "Microsoft.Resources", "registrationState": "Registered" } ] ``` ### Deploy a Bicep template Create a Bicep file named `main.bicep` that provisions a storage account inside the resource group: ```bicep param location string = resourceGroup().location param storageAccountName string = 'stbicep${uniqueString(resourceGroup().id)}' resource storageAccount 'Microsoft.Storage/storageAccounts@2023-05-01' = { name: storageAccountName location: location sku: { name: 'Standard_LRS' } kind: 'StorageV2' } output storageAccountId string = storageAccount.id ``` Deploy the template into the resource group: ```bash az deployment group create \ --resource-group rg-resources-demo \ --template-file main.bicep ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resources-demo/providers/Microsoft.Resources/deployments/main", "name": "main", "properties": { "correlationId": "...", "mode": "Incremental", "provisioningState": "Succeeded", "outputResources": [ { ... "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resources-demo/providers/Microsoft.Storage/storageAccounts/..." } ], "outputs": { "storageAccountId": { "type": "String", "value": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resources-demo/providers/Microsoft.Storage/storageAccounts/..." } }, "..." }, "type": "Microsoft.Resources/deployments", "..." } ``` Verify the deployment status: ```bash az deployment group show \ --resource-group rg-resources-demo \ --name main ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-resources-demo/providers/Microsoft.Resources/deployments/main", "name": "main", "properties": { "correlationId": "...", "mode": "Incremental", "provisioningState": "Succeeded", ... }, "type": "Microsoft.Resources/deployments", "..." } ``` ## Features The Resource Manager emulator supports the following features: - **Resource group lifecycle**: Create, update, get, list, delete, and check existence of resource groups. Deletion cascades to all contained resources. - **Resource listing and filtering**: List resources across a subscription or within a resource group, with support for `$filter` and `$expand` query parameters. - **Resource move**: Move top-level resources between resource groups with validation of source and destination groups, resource locks, and resource hierarchy. - **ARM template deployments**: Create or update deployments at the resource group scope and at the subscription scope. Templates are parsed and resources are provisioned in dependency order. - **Bicep support**: Bicep files compiled by the Azure CLI are accepted as ARM JSON templates. The emulator handles Bicep 2.0 symbolic names, `languageVersion`, and `definitions` sections. - **Nested deployments**: Inner-scoped and outer-scoped nested templates (`Microsoft.Resources/deployments`) are supported, including parameter passing between scopes. - **ARM template functions**: Over 60 built-in functions are evaluated locally, including `resourceId`, `reference`, `listKeys`, `concat`, `format`, `uniqueString`, `resourceGroup`, `subscription`, `copyIndex`, `if`, `union`, `createObject`, and others. - **Template validation**: Validate ARM templates at the resource group scope and subscription scope without creating resources. - **Copy loops and conditional resources**: The `copy` element and `condition` property are supported, enabling iterative resource creation and conditional deployment logic. - **Deployment operations**: List deployment operations at the resource group scope and at the subscription scope, including provisioning state and target resource details. - **Deployment outputs**: Template outputs are evaluated after deployment completes, with full ARM expression resolution. - **Subscription management**: List and get subscriptions, and list available Azure locations with metadata. - **Tenant listing**: List tenant identifiers associated with the emulator environment. - **Provider registry**: List, get, register, and unregister resource providers, including resource type details, API versions, and zone mappings. - **Resource group locks**: Lock and unlock resource groups to prevent accidental deletion or modification of contained resources. ## Limitations - **Single subscription**: The emulator exposes a single subscription. Multiple subscriptions are not supported. - **Template validation is a no-op**: The `deployments validate` and `deployments validate at subscription scope` endpoints return a success response without performing syntactic or semantic validation. - **No management group or tenant-scoped deployments**: Deployments are supported only at the resource group and subscription scopes. - **No what-if analysis**: The `az deployment group what-if` operation is not implemented. - **ARM template function coverage**: While over 60 functions are supported, some less common functions or advanced overloads may not be fully implemented. - **No resource tags on generic resource listings**: Tags and extended properties may not be fully populated when listing resources across a subscription. - **RBAC enforcement on resource operations is opt-in**: By default, all API calls succeed without role-based access control checks. Set `LS_AZURE_ENFORCE_RBAC` to enable control-plane checks across ARM resource types; see [Role Assignment: Enabling RBAC enforcement](/azure/services/role-assignment/#enabling-rbac-enforcement). - **Deployment concurrency**: Resources within a single deployment are created sequentially with dependency resolution, not in full parallel as in Azure. - **No deployment cancellation**: Running deployments cannot be cancelled. - **No deployment deletion**: Deployments are retained in memory and cannot be explicitly deleted via the API. ## Samples The following samples demonstrate how to use Azure Resource Manager and Bicep with LocalStack for Azure: - [Function App and Storage](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-storage-http/dotnet/) - [Function App and Front Door](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-front-door/python/) - [Function App and Managed Identities](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-managed-identity/python/) - [Function App and Service Bus](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-service-bus/dotnet/) - [Web App and Cosmos DB for MongoDB API](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-cosmosdb-mongodb-api/python/) - [Web App and Cosmos DB for NoSQL API](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-cosmosdb-nosql-api/python/) - [Web App and Managed Identities](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-managed-identity/python/) - [Web App and SQL Database](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-sql-database/python/) - [ACI and Blob Storage](https://github.com/localstack/localstack-azure-samples/tree/main/samples/aci-blob-storage/python/) - [Azure Service Bus with Spring Boot](https://github.com/localstack/localstack-azure-samples/tree/main/samples/servicebus/java/) ## API Coverage # Role Assignment > Get started with Azure Role Assignments on LocalStack import AzureFeatureCoverage from '../../../../components/feature-coverage/AzureFeatureCoverage'; ## Introduction Azure Role Assignments grant an identity (user, group, or service principal) the permissions defined by a role definition at a specific scope. Together with Role Definitions, Role Assignments form the foundation of Azure RBAC. They are commonly used to grant managed identities access to storage accounts, key vaults, and other Azure resources in infrastructure automation scenarios. For more information, see [Assign Azure roles using the Azure CLI](https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-cli). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Role Assignments. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Role Assignments' integration with LocalStack. ## Getting started This guide walks you through assigning a built-in role to a managed identity, listing assignments, and removing the assignment. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create --name rg-rbac-demo --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rbac-demo", "location": "westeurope", "managedBy": null, "name": "rg-rbac-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a user-assigned managed identity Create a user-assigned managed identity to use as the role assignee: ```bash az identity create \ --name my-identity \ --resource-group rg-rbac-demo ``` ```bash title="Output" { "clientId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rbac-demo/providers/Microsoft.ManagedIdentity/userAssignedIdentities/my-identity", "isolationScope": "None", "location": "westeurope", "name": "my-identity", "principalId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "resourceGroup": "rg-rbac-demo", "systemData": null, "tags": {}, "tenantId": "00000000-0000-0000-0000-000000000000", "type": "Microsoft.ManagedIdentity/userAssignedIdentities" } ``` Capture the identity's principal ID: ```bash PRINCIPAL_ID=$(az identity show \ --name my-identity \ --resource-group rg-rbac-demo \ --query principalId \ --output tsv) ``` ### Assign a built-in role Assign the `Contributor` role to the identity at the resource group scope: ```bash SUBSCRIPTION_ID=$(az account show --query id --output tsv) az role assignment create \ --assignee "$PRINCIPAL_ID" \ --role Contributor \ --scope "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/rg-rbac-demo" ``` ```bash title="Output" { "condition": null, "conditionVersion": null, "createdBy": null, "createdOn": null, "delegatedManagedIdentityResourceId": null, "description": null, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rbac-demo/providers/Microsoft.Authorization/roleAssignments/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "name": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalType": "ServicePrincipal", "roleDefinitionId": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c", "scope": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rbac-demo", "type": "Microsoft.Authorization/roleAssignments", "updatedBy": null, "updatedOn": null } ``` ### List role assignments List all role assignments scoped to the resource group: ```bash az role assignment list \ --scope "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/rg-rbac-demo" ``` ```bash title="Output" [ { "condition": null, "conditionVersion": null, "createdBy": null, "createdOn": null, "delegatedManagedIdentityResourceId": null, "description": null, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rbac-demo/providers/Microsoft.Authorization/roleAssignments/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "name": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalName": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalType": "ServicePrincipal", "roleDefinitionId": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c", "roleDefinitionName": "Contributor", "scope": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rbac-demo", "type": "Microsoft.Authorization/roleAssignments", "updatedBy": null, "updatedOn": null } ] ``` ### Filter by assignee Filter the role assignments to show only assignments for the managed identity's principal ID: ```bash az role assignment list \ --assignee "$PRINCIPAL_ID" \ --all ``` ```bash title="Output" [ { "condition": null, "conditionVersion": null, "createdBy": null, "createdOn": null, "delegatedManagedIdentityResourceId": null, "description": null, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rbac-demo/providers/Microsoft.Authorization/roleAssignments/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "name": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalName": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalType": "ServicePrincipal", "roleDefinitionId": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c", "roleDefinitionName": "Contributor", "scope": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rbac-demo", "type": "Microsoft.Authorization/roleAssignments", "updatedBy": null, "updatedOn": null } ] ``` ### List all role assignments for the subscription List every role assignment across the entire subscription: ```bash az role assignment list --all ``` ```bash title="Output" [ { "condition": null, "conditionVersion": null, "createdBy": null, "createdOn": null, "delegatedManagedIdentityResourceId": null, "description": null, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rbac-demo/providers/Microsoft.Authorization/roleAssignments/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "name": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalName": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalType": "ServicePrincipal", "roleDefinitionId": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c", "roleDefinitionName": "Contributor", "scope": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rbac-demo", "type": "Microsoft.Authorization/roleAssignments", "updatedBy": null, "updatedOn": null } ] ``` ### Assign a Storage Blob Data Owner role on a storage account Create a storage account and assign the `Storage Blob Data Owner` role to the managed identity at the storage account scope. This is a common pattern in infrastructure automation where a function app or container needs full access to a specific storage account. ```bash az storage account create \ --name strblobdataowner \ --resource-group rg-rbac-demo \ --location westeurope \ --sku Standard_LRS ``` Capture the storage account resource ID: ```bash STORAGE_ID=$(az storage account show \ --name strblobdataowner \ --resource-group rg-rbac-demo \ --query id \ --output tsv) ``` Assign `Storage Blob Data Owner` at the storage account scope: ```bash az role assignment create \ --assignee "$PRINCIPAL_ID" \ --role "Storage Blob Data Owner" \ --scope "$STORAGE_ID" ``` ```bash title="Output" { "condition": null, "conditionVersion": null, "createdBy": null, "createdOn": null, "delegatedManagedIdentityResourceId": null, "description": null, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rbac-demo/providers/Microsoft.Storage/storageAccounts/strblobdataowner/providers/Microsoft.Authorization/roleAssignments/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "name": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalType": "ServicePrincipal", "roleDefinitionId": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Authorization/roleDefinitions/b7e6dc6d-f1e8-4753-8033-0f276bb0955b", "scope": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rbac-demo/providers/Microsoft.Storage/storageAccounts/strblobdataowner", "type": "Microsoft.Authorization/roleAssignments", "updatedBy": null, "updatedOn": null } ``` List assignments scoped to the storage account to verify: ```bash az role assignment list --scope "$STORAGE_ID" ``` ```bash title="Output" [ { "condition": null, "conditionVersion": null, "createdBy": null, "createdOn": null, "delegatedManagedIdentityResourceId": null, "description": null, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rbac-demo/providers/Microsoft.Storage/storageAccounts/strblobdataowner/providers/Microsoft.Authorization/roleAssignments/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "name": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalName": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "principalType": "ServicePrincipal", "roleDefinitionId": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Authorization/roleDefinitions/b7e6dc6d-f1e8-4753-8033-0f276bb0955b", "roleDefinitionName": "Storage Blob Data Owner", "scope": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rbac-demo/providers/Microsoft.Storage/storageAccounts/strblobdataowner", "type": "Microsoft.Authorization/roleAssignments", "updatedBy": null, "updatedOn": null } ] ``` ### Delete a role assignment Delete the role assignment and confirm it no longer appears in the list: ```bash az role assignment delete \ --assignee "$PRINCIPAL_ID" \ --role Contributor \ --scope "/subscriptions/$SUBSCRIPTION_ID/resourceGroups/rg-rbac-demo" ``` ## Enabling RBAC enforcement By default, Azure RBAC on LocalStack is **not enforced**: role assignments and role definitions are stored, but every operation succeeds regardless of the assigned roles. Set `LS_AZURE_ENFORCE_RBAC` to `true`, `yes`, or `1` when starting the emulator to turn on enforcement. With enforcement enabled: - **Control plane**: every ARM request, across all Azure services and resource types, is checked against the caller's role assignments at the target scope and denied with a `403` if unauthorized. - **Data plane**: checked for [Blob](/azure/services/blob-storage/), [Queue](/azure/services/queue-storage/), and [Table](/azure/services/table-storage/) Storage, RBAC-mode [Key Vault](/azure/services/key-vault/) vaults (secrets and certificates only), and Event Grid publish/receive. Denials match the shape Azure returns for each service, for example a Storage `AuthorizationPermissionMismatch` XML error or a Key Vault `ForbiddenByRbac` error. - **Not covered yet**, even with enforcement enabled: Storage File, the Service Bus data plane, Cosmos DB's data plane, and Microsoft Entra database authentication for Azure SQL, PostgreSQL, and MySQL flexible servers. Requests to these continue to succeed regardless of role assignments. The default SDK/Terraform service principal and the `az` CLI's `any-app` principal are always treated as a Global Administrator and subscription Owner, and bypass data-plane checks by default too. To observe a deny, use a managed identity or service principal that assumes neither role — for example, the identity created in [Create a user-assigned managed identity](#create-a-user-assigned-managed-identity) without a role assigned at the target scope. ## Features - **Role assignment creation:** Create role assignments by specifying an assignee principal ID, role name or ID, and scope. - **Assignment listing:** List role assignments at subscription scope, resource group scope, or filtered by assignee. - **Assignee filtering:** Filter assignments by principal ID or display name. - **Subscription-wide listing:** Retrieve all role assignments across a subscription via `--all`. - **Role assignment deletion:** Delete assignments by role name, assignee, and scope. - **Custom role support:** Assign custom role definitions alongside built-in roles. ## Limitations - **RBAC enforcement is opt-in:** By default, role assignments are stored but not evaluated, and all operations succeed regardless of assigned roles. Set `LS_AZURE_ENFORCE_RBAC` to enable enforcement. - **Data-plane coverage is partial:** Enforced for Storage (Blob/Queue/Table), Key Vault (secrets and certificates), and Event Grid. Not yet enforced for Storage File, the Service Bus data plane, or Cosmos DB. Azure SQL Database and Azure Database for PostgreSQL/MySQL flexible servers don't use RBAC data actions (their data-plane authorization is Microsoft Entra database authentication), so they're out of scope for RBAC. - **Key Vault keys:** Only the secrets and certificates data planes are enforced; the keys data plane is not yet implemented. - **Condition-based assignments:** Attribute-based access control (ABAC) conditions in assignments are accepted at the model level but are not evaluated. - **Deny assignments:** `Microsoft.Authorization/denyAssignments` are not supported. - **Management group scopes:** Assignments at management group scope are not supported. Subscription, resource group, and resource scopes are supported, including inheritance down the hierarchy — a role assigned at a broader scope applies to narrower scopes beneath it. - **Groups and transitive membership:** A role assigned to a group is not expanded to its members; only assignments made directly to the calling principal are evaluated. ## Samples The following sample demonstrates how to use Azure Role Assignments with LocalStack for Azure: - [Function App and Service Bus](https://github.com/localstack/localstack-azure-samples/samples/function-app-service-bus/dotnet/README.md) - [Web App and Cosmos DB for MongoDB API ](https://github.com/localstack/localstack-azure-samples/samples/web-app-cosmosdb-mongodb-api/python/README.md) ## API Coverage # Role Definition > Get started with Azure Role Definitions on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Role Definitions are the building blocks of Azure role-based access control (RBAC). A role definition is a collection of permissions that can be assigned to identities at a specific scope. They allow organizations to grant least-privilege access to Azure resources by defining precisely which operations an identity is permitted to perform. For more information, see [What is Azure RBAC?](https://learn.microsoft.com/en-us/azure/role-based-access-control/overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Role Definitions. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Role Definitions' integration with LocalStack. ## Getting started This guide walks you through creating a custom role definition, listing role definitions, and deleting the custom role. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### List role definitions Run [`az role definition list`](https://learn.microsoft.com/en-us/cli/azure/role/definition#az-role-definition-list) to list role definitions for the current subscription. The results include built-in roles (such as Owner, Contributor, and Reader) as well as any custom roles: ```bash az role definition list --output table ``` ```bash title="Output" Name Type Description --------------------------------------- --------------------------------------- ----------------------------------------------------------- Contributor Microsoft.Authorization/roleDefinitions Grants full access to manage all resources, but does not allow you to assign roles in Azure RBAC... Owner Microsoft.Authorization/roleDefinitions Grants full access to manage all resources, including assigning roles in Azure RBAC... Reader Microsoft.Authorization/roleDefinitions View all resources, but does not allow you to make any changes. ... ``` ### Create a custom role definition Save the following JSON to `custom-role.json`: ```json title="custom-role.json" { "Name": "Custom Storage Reader", "Description": "Can read storage blobs.", "Actions": [ "Microsoft.Storage/storageAccounts/blobServices/containers/read" ], "NotActions": [], "DataActions": [ "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read" ], "NotDataActions": [], "AssignableScopes": [ "/subscriptions/00000000-0000-0000-0000-000000000000" ] } ``` Then create the role: ```bash az role definition create --role-definition @custom-role.json ``` ```bash title="Output" { "assignableScopes": ["/subscriptions/00000000-0000-0000-0000-000000000000"], "description": "Can read storage blobs.", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Authorization/roleDefinitions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "name": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "permissions": [ { "actions": [ "Microsoft.Storage/storageAccounts/blobServices/containers/read" ], "notActions": [], "dataActions": [ "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read" ], "notDataActions": [] } ], "roleName": "Custom Storage Reader", "roleType": "CustomRole", "type": "Microsoft.Authorization/roleDefinitions" ... } ``` ### List a role definition by name List role definitions that match the display name (`roleName`), as in [Azure’s custom role CLI workflow](https://learn.microsoft.com/en-us/azure/role-based-access-control/custom-roles-cli#list-a-custom-role-definition): ```bash az role definition list --name "Custom Storage Reader" ``` ```bash title="Output" [ { "assignableScopes": ["/subscriptions/00000000-0000-0000-0000-000000000000"], "description": "Can read storage blobs.", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Authorization/roleDefinitions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "name": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "permissions": [ { "actions": [ "Microsoft.Storage/storageAccounts/blobServices/containers/read" ], "notActions": [], "dataActions": [ "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read" ], "notDataActions": [] } ], "roleName": "Custom Storage Reader", "roleType": "CustomRole", "type": "Microsoft.Authorization/roleDefinitions" } ] ``` ### Update a custom role definition Update the custom role definition by passing a modified JSON definition file. As described in [Create or update Azure custom roles using Azure CLI](https://learn.microsoft.com/en-us/azure/role-based-access-control/custom-roles-cli), retrieve the current definition with `az role definition list`, edit the JSON (for example permissions or assignable scopes), then apply the update: ```bash az role definition update --role-definition @custom-role.json ``` ### Delete a custom role definition Delete the custom role definition by name: ```bash az role definition delete --name "Custom Storage Reader" az role definition list --name "Custom Storage Reader" ``` ## Features - **Custom role creation:** Create custom role definitions with `Actions`, `NotActions`, `DataActions`, and `NotDataActions`. - **Built-in roles pre-populated:** Standard Azure built-in roles are available via `az role definition list`. - **Role listing and filtering:** List role definitions by name, scope, or custom flag. - **Role update:** Update existing custom role definitions including permissions and assignable scopes. - **Role deletion:** Delete custom role definitions by name or ID. - **Assignable scopes support:** Roles specify assignable scopes at subscription or resource group level. ## Limitations - **RBAC enforcement is opt-in:** By default, role definitions and assignments are stored but permissions are not enforced, so API calls are not gated the way they are in Azure. Set `LS_AZURE_ENFORCE_RBAC` to enable enforcement; see [Role Assignment: Enabling RBAC enforcement](/azure/services/role-assignment/#enabling-rbac-enforcement) for scope and coverage. - **Management group scopes:** Management group–level assignable scopes are not supported. ## Samples Explore end-to-end examples in the [LocalStack for Azure Samples](https://github.com/localstack/localstack-azure-samples) repository. ## API Coverage # Route Table > Get started with Azure Route Table on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Route Table contains a set of rules (routes) that specify how packets should be routed in a virtual network. You can associate a route table with a subnet to override Azure's default system routes and direct traffic to specific next-hop targets such as virtual appliances, VPN gateways, or other virtual networks. Route tables are commonly used in hub-and-spoke topologies to force traffic through a central network virtual appliance for inspection. For more information, see [Virtual network traffic routing](https://learn.microsoft.com/en-us/azure/virtual-network/virtual-networks-udr-overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Route Tables. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Route Table's integration with LocalStack. ## Getting started This guide is designed for users new to Route Tables and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create \ --name rg-rt-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo", "location": "westeurope", "managedBy": null, "name": "rg-rt-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a route table Create an empty route table to hold the custom routing rules: ```bash az network route-table create \ --name rt-demo \ --resource-group rg-rt-demo \ --location westeurope ``` ```bash title="Output" { "disableBgpRoutePropagation": false, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/routeTables/rt-demo", "location": "westeurope", "name": "rt-demo", "provisioningState": "Succeeded", "resourceGroup": "rg-rt-demo", "routes": [], "type": "Microsoft.Network/routeTables" } ``` ### Create a route table and add routes Routes are managed as separate resources in Azure; create the route table first, then add one or more routes. Create an empty route table: ```bash az network route-table create \ --name rt-with-routes \ --resource-group rg-rt-demo \ --location westeurope ``` ```bash title="Output" { "disableBgpRoutePropagation": false, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/routeTables/rt-with-routes", "location": "westeurope", "name": "rt-with-routes", "provisioningState": "Succeeded", "resourceGroup": "rg-rt-demo", "routes": [], "type": "Microsoft.Network/routeTables" } ``` Add a route that directs internet-bound traffic (`0.0.0.0/0`) to a virtual appliance: ```bash az network route-table route create \ --name route-to-internet \ --route-table-name rt-with-routes \ --resource-group rg-rt-demo \ --address-prefix 0.0.0.0/0 \ --next-hop-type VirtualAppliance \ --next-hop-ip-address 10.0.2.4 ``` ```bash title="Output" { "addressPrefix": "0.0.0.0/0", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/routeTables/rt-with-routes/routes/route-to-internet", "name": "route-to-internet", "nextHopIpAddress": "10.0.2.4", "nextHopType": "VirtualAppliance", "provisioningState": "Succeeded", "resourceGroup": "rg-rt-demo", "type": "Microsoft.Network/routeTables/routes" } ``` ### Manage individual routes Add, show, and delete individual routes on an existing route table: Add a route that directs internet-bound traffic through a virtual appliance: ```bash az network route-table route create \ --name route-to-appliance \ --route-table-name rt-demo \ --resource-group rg-rt-demo \ --address-prefix 0.0.0.0/0 \ --next-hop-type VirtualAppliance \ --next-hop-ip-address 10.0.1.4 ``` ```bash title="Output" { "addressPrefix": "0.0.0.0/0", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/routeTables/rt-demo/routes/route-to-appliance", "name": "route-to-appliance", "nextHopIpAddress": "10.0.1.4", "nextHopType": "VirtualAppliance", "provisioningState": "Succeeded", "resourceGroup": "rg-rt-demo", "type": "Microsoft.Network/routeTables/routes" } ``` Add a second route that keeps spoke traffic local to the virtual network: ```bash az network route-table route create \ --name route-to-spoke \ --route-table-name rt-demo \ --resource-group rg-rt-demo \ --address-prefix 172.16.0.0/16 \ --next-hop-type VnetLocal ``` ```bash title="Output" { "addressPrefix": "172.16.0.0/16", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/routeTables/rt-demo/routes/route-to-spoke", "name": "route-to-spoke", "nextHopType": "VnetLocal", "provisioningState": "Succeeded", "resourceGroup": "rg-rt-demo", "type": "Microsoft.Network/routeTables/routes" } ``` Then retrieve the details of the `route-to-spoke` route within the table: ```bash az network route-table route show \ --name route-to-spoke \ --route-table-name rt-demo \ --resource-group rg-rt-demo ``` ```bash title="Output" { "addressPrefix": "172.16.0.0/16", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/routeTables/rt-demo/routes/route-to-spoke", "name": "route-to-spoke", "nextHopType": "VnetLocal", "provisioningState": "Succeeded", "resourceGroup": "rg-rt-demo", "type": "Microsoft.Network/routeTables/routes" } ``` Delete the `route-to-spoke` route from the route table: ```bash az network route-table route delete \ --name route-to-spoke \ --route-table-name rt-demo \ --resource-group rg-rt-demo ``` ### Get and list route tables Retrieve the details of a route table and list all route tables in the resource group: ```bash az network route-table show \ --name rt-demo \ --resource-group rg-rt-demo ``` ```bash title="Output" { "disableBgpRoutePropagation": false, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/routeTables/rt-demo", "location": "westeurope", "name": "rt-demo", "provisioningState": "Succeeded", "resourceGroup": "rg-rt-demo", "routes": [ { "addressPrefix": "0.0.0.0/0", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/routeTables/rt-demo/routes/route-to-appliance", "name": "route-to-appliance", "nextHopIpAddress": "10.0.1.4", "nextHopType": "VirtualAppliance", "provisioningState": "Succeeded", "type": "Microsoft.Network/routeTables/routes" } ], "type": "Microsoft.Network/routeTables" } ``` Then list all route tables in the resource group: ```bash az network route-table list \ --resource-group rg-rt-demo ``` ```bash title="Output" [ { "disableBgpRoutePropagation": false, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/routeTables/rt-demo", "location": "westeurope", "name": "rt-demo", "provisioningState": "Succeeded", "routes": [ { "name": "route-to-appliance", "nextHopType": "VirtualAppliance", "nextHopIpAddress": "10.0.1.4" } ], "type": "Microsoft.Network/routeTables" }, { "disableBgpRoutePropagation": false, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/routeTables/rt-with-routes", "location": "westeurope", "name": "rt-with-routes", "provisioningState": "Succeeded", "routes": [ { "name": "route-to-internet", "nextHopType": "VirtualAppliance", "nextHopIpAddress": "10.0.2.4" } ], "type": "Microsoft.Network/routeTables" } ] ``` ### Associate a route table with a subnet To apply routing rules, associate the route table with a subnet. First create a virtual network with a subnet: ```bash az network vnet create \ --name vnet-demo \ --resource-group rg-rt-demo \ --location westeurope \ --address-prefix 10.0.0.0/16 \ --subnet-name subnet-demo \ --subnet-prefixes 10.0.1.0/24 ``` ```bash title="Output" { "newVNet": { "addressSpace": { "addressPrefixes": [ "10.0.0.0/16" ] }, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/virtualNetworks/vnet-demo", "location": "westeurope", "name": "vnet-demo", "provisioningState": "Succeeded", "subnets": [ { "addressPrefix": "10.0.1.0/24", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/virtualNetworks/vnet-demo/subnets/subnet-demo", "name": "subnet-demo", "provisioningState": "Succeeded", "type": "Microsoft.Network/virtualNetworks/subnets" } ], "type": "Microsoft.Network/virtualNetworks" } } ``` Then associate `rt-demo` with `subnet-demo`: ```bash az network vnet subnet update \ --name subnet-demo \ --vnet-name vnet-demo \ --resource-group rg-rt-demo \ --route-table rt-demo ``` ```bash title="Output" { "addressPrefix": "10.0.1.0/24", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/virtualNetworks/vnet-demo/subnets/subnet-demo", "name": "subnet-demo", "provisioningState": "Succeeded", "resourceGroup": "rg-rt-demo", "routeTable": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/routeTables/rt-demo", "resourceGroup": "rg-rt-demo" }, "type": "Microsoft.Network/virtualNetworks/subnets" } ``` Verify the association is reflected on the subnet: ```bash az network vnet subnet show \ --name subnet-demo \ --vnet-name vnet-demo \ --resource-group rg-rt-demo ``` ```bash title="Output" { "addressPrefix": "10.0.1.0/24", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/virtualNetworks/vnet-demo/subnets/subnet-demo", "name": "subnet-demo", "provisioningState": "Succeeded", "resourceGroup": "rg-rt-demo", "routeTable": { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-rt-demo/providers/Microsoft.Network/routeTables/rt-demo", "resourceGroup": "rg-rt-demo" }, "type": "Microsoft.Network/virtualNetworks/subnets" } ``` ### Delete the route tables Delete both route tables and verify they no longer appear in the list: ```bash az network route-table delete \ --name rt-demo \ --resource-group rg-rt-demo az network route-table delete \ --name rt-with-routes \ --resource-group rg-rt-demo ``` Then list route tables for the resource group to confirm neither route table remains: ```bash az network route-table list --resource-group rg-rt-demo ``` ```bash title="Output" [] ``` ## Features The Route Table emulator supports the following features: - **Create and manage route tables**: Full lifecycle management including create, get, update, list, and delete. - **Sub-resource route management**: Create, get, update, list, and delete individual routes within a route table. - **Next-hop types**: Support for `VirtualAppliance`, `VnetLocal`, `VirtualNetworkGateway`, `Internet`, and `None`. - **Next-hop IP addresses**: Store and return the next-hop IP address for `VirtualAppliance` routes. - **Address prefix ranges**: Store and return the address prefix for each route. - **Tags**: Apply and update resource tags on route table resources. - **Subscription-scoped listing**: List all route tables across a subscription. ## Limitations - **No traffic routing**: Route Table is a mock implementation. State is persisted in memory and returned faithfully, but no actual network traffic is routed according to the configured routes. - **No subnet association enforcement**: Associating a route table with a subnet is accepted but not enforced at the network level. - **No data persistence**: Route table resources and their routes are not persisted and are lost when the emulator is stopped or restarted. ## Samples Explore end-to-end examples in the [LocalStack for Azure Samples](https://github.com/localstack/localstack-azure-samples) repository. ## API Coverage # Scheduled Query Rules > Get started with Azure Monitor Scheduled Query Rules on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Monitor Scheduled Query Rules (SQR) run KQL log queries on a defined schedule against data in one or more scopes (for example a Log Analytics workspace). When query results meet a configured condition, an alert is fired and routed through an Action Group. Scheduled Query Rules are commonly used to detect patterns in application logs, audit events, and custom metrics that cannot be captured by standard metric alerts. For more information, see [Log alerts in Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/alerts/alerts-log). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Monitor Scheduled Query Rules. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Scheduled Query Rules' integration with LocalStack. ## Getting started This guide walks you through creating a Scheduled Query Rule that targets a Log Analytics workspace. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create --name rg-sqr-demo --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sqr-demo", "location": "westeurope", "name": "rg-sqr-demo", "properties": { "provisioningState": "Succeeded" }, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a Log Analytics workspace Create a Log Analytics workspace to use as the scheduled query target: ```bash az monitor log-analytics workspace create \ --name my-workspace \ --resource-group rg-sqr-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sqr-demo/providers/Microsoft.OperationalInsights/workspaces/my-workspace", "location": "westeurope", "name": "my-workspace", "provisioningState": "Succeeded", "resourceGroup": "rg-sqr-demo", "sku": { "name": "PerGB2018" }, "type": "Microsoft.OperationalInsights/workspaces" } ``` ### Create an action group Create an action group to serve as the notification target when the alert fires: ```bash az monitor action-group create \ --name my-ag \ --resource-group rg-sqr-demo \ --short-name myag \ --action email admin admin@example.com ``` ```bash title="Output" { "emailReceivers": [ { "emailAddress": "admin@example.com", "name": "admin", "useCommonAlertSchema": false } ], "enabled": true, "groupShortName": "myag", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sqr-demo/providers/microsoft.insights/actionGroups/my-ag", "location": "Global", "name": "my-ag", "resourceGroup": "rg-sqr-demo", "type": "Microsoft.Insights/ActionGroups" } ``` ### Create a scheduled query rule Retrieve the workspace and action group resource IDs, then create a scheduled query rule with a count-based condition on the `Heartbeat` table (in Azure, the rule would evaluate this KQL on the schedule you set and fire when the condition is met): ```bash WORKSPACE_ID=$(az monitor log-analytics workspace show \ --workspace-name my-workspace \ --resource-group rg-sqr-demo \ --query id \ --output tsv) AG_ID=$(az monitor action-group show \ --name my-ag \ --resource-group rg-sqr-demo \ --query id \ --output tsv) az monitor scheduled-query create \ --name my-sqr \ --resource-group rg-sqr-demo \ --scopes "$WORKSPACE_ID" \ --condition "count 'Placeholder_1' > 5" \ --condition-query 'Placeholder_1="Heartbeat | where TimeGenerated > ago(5m)"' \ --description "Alert on Heartbeat count" \ --action-groups "$AG_ID" \ --evaluation-frequency 5m \ --window-size 5m \ --severity 2 ``` ```bash title="Output" { "actions": { "actionGroups": [ "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sqr-demo/providers/microsoft.insights/actionGroups/my-ag" ] }, "criteria": { "allOf": [ { "operator": "GreaterThan", "query": "Heartbeat | where TimeGenerated > ago(5m)", "threshold": 5.0, "timeAggregation": "Count" } ] }, "description": "Alert on Heartbeat count", "enabled": true, "evaluationFrequency": "PT5M", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sqr-demo/providers/microsoft.insights/scheduledqueryrules/my-sqr", "location": "westeurope", "name": "my-sqr", "resourceGroup": "rg-sqr-demo", "scopes": [ "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sqr-demo/providers/Microsoft.OperationalInsights/workspaces/my-workspace" ], "severity": 2, "type": "microsoft.insights/scheduledqueryrules", "windowSize": "PT5M" } ``` ### Show and list query rules Retrieve the details of the scheduled query rule and list all rules in the resource group: ```bash az monitor scheduled-query show \ --name my-sqr \ --resource-group rg-sqr-demo ``` ```bash title="Output" { "actions": { "actionGroups": [ "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sqr-demo/providers/microsoft.insights/actionGroups/my-ag" ] }, "criteria": { "allOf": [ { "operator": "GreaterThan", "query": "Heartbeat | where TimeGenerated > ago(5m)", "threshold": 5.0, "timeAggregation": "Count" } ] }, "description": "Alert on Heartbeat count", "enabled": true, "evaluationFrequency": "PT5M", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sqr-demo/providers/microsoft.insights/scheduledqueryrules/my-sqr", "location": "westeurope", "name": "my-sqr", "resourceGroup": "rg-sqr-demo", "scopes": [ "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sqr-demo/providers/Microsoft.OperationalInsights/workspaces/my-workspace" ], "severity": 2, "type": "microsoft.insights/scheduledqueryrules", "windowSize": "PT5M" } ``` Then list all scheduled query rules in the resource group: ```bash az monitor scheduled-query list \ --resource-group rg-sqr-demo ``` ```bash title="Output" [ { "actions": { "actionGroups": [ "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sqr-demo/providers/microsoft.insights/actionGroups/my-ag" ] }, "criteria": { "allOf": [ { "operator": "GreaterThan", "query": "Heartbeat | where TimeGenerated > ago(5m)", "threshold": 5.0, "timeAggregation": "Count" } ] }, "enabled": true, "evaluationFrequency": "PT5M", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sqr-demo/providers/microsoft.insights/scheduledqueryrules/my-sqr", "location": "westeurope", "name": "my-sqr", "resourceGroup": "rg-sqr-demo", "scopes": [ "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sqr-demo/providers/Microsoft.OperationalInsights/workspaces/my-workspace" ], "severity": 2, "type": "microsoft.insights/scheduledqueryrules", "windowSize": "PT5M" } ] ``` ### Delete and verify Delete the resource and confirm it no longer appears in the list: ```bash az monitor scheduled-query delete \ --name my-sqr \ --resource-group rg-sqr-demo \ --yes ``` Then list all scheduled query rules to confirm the resource group is now empty: ```bash az monitor scheduled-query list --resource-group rg-sqr-demo ``` ```bash title="Output" [] ``` ## Features - **Scheduled Query Rule lifecycle:** Create, read, list, update, and delete SQR resources. - **KQL query storage:** Store the KQL query definition within the rule (not executed). - **Condition configuration:** Define threshold, operator, time aggregation, and evaluation period. - **Action group references:** Associate one or more action groups with a query rule. - **Severity levels:** Set alert severity from 0 (critical) to 4 (verbose), consistent with [Azure CLI `az monitor scheduled-query`](https://learn.microsoft.com/en-us/cli/azure/monitor/scheduled-query). - **Evaluation frequency and window size:** Set how often the rule runs and the aggregation window (`5m`-style values in the CLI; the API represents these as ISO 8601 durations such as `PT5M`). See the REST API property reference for [`scheduledQueryRules`](https://learn.microsoft.com/en-us/rest/api/monitor/scheduled-query-rules/create-or-update). - **Multiple scopes:** Provide several scope resource IDs when your scenario requires it (the CLI documents [constraints on `scopes`](https://learn.microsoft.com/en-us/cli/azure/monitor/scheduled-query?view=azure-cli-latest#az-monitor-scheduled-query-create)). ## Limitations - **No KQL execution:** The query defined in the rule is never run against Log Analytics data. - **No alert firing:** Alert thresholds are never evaluated and no alerts are triggered. - **No notifications dispatched:** Action group notifications are not sent (unlike Azure, where a firing rule invokes the configured action groups). - **No alert history:** Alert instance history and state transitions are not recorded. ## Samples Explore end-to-end examples in the [LocalStack for Azure Samples](https://github.com/localstack/localstack-azure-samples) repository. ## API Coverage # Service Bus > Get started with Azure Service Bus on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Service Bus is a fully managed enterprise message broker that supports queues and publish/subscribe topics. It helps decouple distributed systems and build reliable asynchronous messaging workflows. Service Bus is commonly used for command processing, event distribution, and integration between independent services. For more information, see [What is Azure Service Bus?](https://learn.microsoft.com/azure/service-bus-messaging/service-bus-messaging-overview) LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Service Bus. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Service Bus's integration with LocalStack. ## Getting started This guide is designed for users new to Service Bus and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to contain your Service Bus resources: ```bash az group create \ --name rg-servicebus-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-servicebus-demo", "location": "westeurope", "managedBy": null, "name": "rg-servicebus-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a Service Bus namespace Create a Service Bus namespace in the resource group: ```bash az servicebus namespace create \ --resource-group rg-servicebus-demo \ --name sbnsdoc83 \ --location westeurope \ --sku Standard ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-servicebus-demo/providers/Microsoft.ServiceBus/namespaces/sbnsdoc83", "name": "sbnsdoc83", "location": "westeurope", "provisioningState": "Succeeded", "serviceBusEndpoint": "https://sbnsdoc83.localhost.localstack.cloud:4511", "sku": { "name": "Standard", "tier": "Standard" }, ... } ``` Get and list namespaces: ```bash az servicebus namespace show \ --resource-group rg-servicebus-demo \ --name sbnsdoc83 az servicebus namespace list \ --resource-group rg-servicebus-demo ``` ### Create and inspect a queue Create a queue in the namespace: ```bash az servicebus queue create \ --resource-group rg-servicebus-demo \ --namespace-name sbnsdoc83 \ --name orders-queue ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-servicebus-demo/providers/Microsoft.ServiceBus/namespaces/sbnsdoc83/queues/orders-queue", "name": "orders-queue", "location": "westeurope", "status": "Active", "messageCount": 0, "maxSizeInMegabytes": 1024, ... } ``` Get and list queues: ```bash az servicebus queue show \ --resource-group rg-servicebus-demo \ --namespace-name sbnsdoc83 \ --name orders-queue ``` ```bash title="Output" { "accessedAt": "2026-03-18T10:13:18.3906198Z", "autoDeleteOnIdle": "P10675199DT2H48M5.4775807S", "countDetails": { "activeMessageCount": 0, "deadLetterMessageCount": 0, "scheduledMessageCount": 0, "transferDeadLetterMessageCount": 0, "transferMessageCount": 0 }, ... "name": "orders-queue", ... "status": "Active", "type": "Microsoft.ServiceBus/namespaces/queues", ... } ``` ```bash az servicebus queue list \ --resource-group rg-servicebus-demo \ --namespace-name sbnsdoc83 ``` ```bash title="Output" [ { "accessedAt": "2026-03-18T10:14:44.3808099Z", "autoDeleteOnIdle": "P10675199DT2H48M5.4775807S", "countDetails": { "activeMessageCount": 0, "deadLetterMessageCount": 0, "scheduledMessageCount": 0, "transferDeadLetterMessageCount": 0, "transferMessageCount": 0 }, ... "name": "orders-queue", ... "status": "Active", "type": "Microsoft.ServiceBus/namespaces/queues", ... } ] ``` :::note The values under `countDetails` may not be accurate in the emulator. ::: ### Create topic and subscription Create a topic and a subscription: ```bash az servicebus topic create \ --resource-group rg-servicebus-demo \ --namespace-name sbnsdoc83 \ --name orders-topic az servicebus topic subscription create \ --resource-group rg-servicebus-demo \ --namespace-name sbnsdoc83 \ --topic-name orders-topic \ --name orders-sub ``` ```bash title="Output" { "name": "orders-topic", "status": "Active", "subscriptionCount": 0, ... } { "name": "orders-sub", "status": "Active", "messageCount": 0, ... } ``` Get and list subscriptions: ```bash az servicebus topic subscription show \ --resource-group rg-servicebus-demo \ --namespace-name sbnsdoc83 \ --topic-name orders-topic \ --name orders-sub ``` ```bash title="Output" { "accessedAt": "2026-03-18T10:13:18.3906198Z", "autoDeleteOnIdle": "P10675199DT2H48M5.4775807S", "countDetails": { "activeMessageCount": 0, "deadLetterMessageCount": 0, "scheduledMessageCount": 0, "transferDeadLetterMessageCount": 0, "transferMessageCount": 0 }, ... "name": "orders-sub", ... "status": "Active", "type": "Microsoft.ServiceBus/namespaces/topics/subscriptions", ... } ``` ```bash az servicebus topic subscription list \ --resource-group rg-servicebus-demo \ --namespace-name sbnsdoc83 \ --topic-name orders-topic ``` ```bash title="Output" [ { "accessedAt": "2026-03-18T10:14:44.3808099Z", "autoDeleteOnIdle": "P10675199DT2H48M5.4775807S", "countDetails": { "activeMessageCount": 0, "deadLetterMessageCount": 0, "scheduledMessageCount": 0, "transferDeadLetterMessageCount": 0, "transferMessageCount": 0 }, ... "name": "orders-sub", ... "status": "Active", "type": "Microsoft.ServiceBus/namespaces/topics/subscriptions", ... } ] ``` :::note The values under `countDetails` may not be accurate in the emulator. ::: ### Create and list subscription rules Create a SQL filter rule for the subscription: ```bash az servicebus topic subscription rule create \ --resource-group rg-servicebus-demo \ --namespace-name sbnsdoc83 \ --topic-name orders-topic \ --subscription-name orders-sub \ --name high-priority \ --filter-sql-expression "priority = 'high'" ``` ```bash title="Output" { "name": "high-priority", "filterType": "SqlFilter", "sqlFilter": { "sqlExpression": "priority = 'high'", ... }, ... } ``` List rules for the subscription: ```bash az servicebus topic subscription rule list \ --resource-group rg-servicebus-demo \ --namespace-name sbnsdoc83 \ --topic-name orders-topic \ --subscription-name orders-sub ``` ### Create and manage namespace authorization rules Create an authorization rule: ```bash az servicebus namespace authorization-rule create \ --resource-group rg-servicebus-demo \ --namespace-name sbnsdoc83 \ --name app-policy \ --rights Listen Send ``` ```bash title="Output" { "name": "app-policy", "rights": [ "Listen", "Send" ], ... } ``` List authorization rules: ```bash az servicebus namespace authorization-rule list \ --resource-group rg-servicebus-demo \ --namespace-name sbnsdoc83 ``` List and regenerate keys: ```bash az servicebus namespace authorization-rule keys list \ --resource-group rg-servicebus-demo \ --namespace-name sbnsdoc83 \ --name app-policy az servicebus namespace authorization-rule keys renew \ --resource-group rg-servicebus-demo \ --namespace-name sbnsdoc83 \ --name app-policy \ --key PrimaryKey ``` ```bash title="Output" { "keyName": "app-policy", "primaryConnectionString": "Endpoint=https://sbnsdoc83.localhost.localstack.cloud:4511/;SharedAccessKeyName=app-policy;SharedAccessKey=...;UseDevelopmentEmulator=true", "secondaryConnectionString": "Endpoint=https://sbnsdoc83.localhost.localstack.cloud:4511/;SharedAccessKeyName=app-policy;SharedAccessKey=...;UseDevelopmentEmulator=true", ... } { "keyName": "app-policy", "primaryConnectionString": "Endpoint=https://sbnsdoc83.localhost.localstack.cloud:4511/;SharedAccessKeyName=app-policy;SharedAccessKey=...;UseDevelopmentEmulator=true", ... } ``` ## Features The emulator includes the following core capabilities: - **Data Plane REST API**: Supports message-level operations, including Send, Receive, and Peek. - **Control Plane REST API**: Enables CRUD operations for namespaces and messaging entities (queues, topics, and subscriptions) via Azure Resource Manager (ARM). - **Multiple Authentication Modes**: Supports both Connection String and Managed Identity authentication. - **Containerized Deployment**: Runs as a lightweight, Linux-based Docker container. - **Cross-Platform Compatibility**: Fully compatible with Windows, macOS, and Linux environments. - **Flexible Configuration**: Manage Service Bus entities via the Service Bus Administration Client or through JSON-based configuration files. - **Advanced Streaming**: Supports message streaming via the Advanced Message Queuing Protocol (AMQP). ## Limitations The current version of the emulator does **not** support the following: - **Protocols**: JMS protocol streaming and AMQP Web Sockets (AMQP over TCP is the only supported transport). - **Messaging Patterns**: Transactions, auto-forwarding (queue chaining), and message lock renewal. - **Validation**: Enforcements such as maximum entity counts or maximum message sizes. - **Metrics**: Property-based message counts for queues, topics, and subscriptions may be inaccurate. The following Azure-native features are currently unavailable in the emulator: - **Scaling & Resiliency**: Autoscale, Geo-disaster recovery, and Large Message support. - **Monitoring**: Visual metrics, alerts, and telemetry dashboards. ## Samples Explore the following samples to get started with Service Bus on LocalStack: - [Azure Functions App with Service Bus Messaging](https://github.com/localstack/localstack-azure-samples/blob/main/samples/function-app-service-bus/dotnet/) - [Azure Service Bus with Spring Boot](https://github.com/localstack/localstack-azure-samples/tree/main/samples/servicebus/java) ## Features The emulator includes the following core capabilities: - **Data Plane REST API**: Supports message-level operations, including Send, Receive, and Peek. - **Control Plane REST API**: Enables CRUD operations for namespaces and messaging entities (queues, topics, and subscriptions) via Azure Resource Manager (ARM). - **Multiple Authentication Modes**: Supports both Connection String and Managed Identity authentication. - **Containerized Deployment**: Runs as a lightweight, Linux-based Docker container. - **Cross-Platform Compatibility**: Fully compatible with Windows, macOS, and Linux environments. - **Flexible Configuration**: Manage Service Bus entities via the Service Bus Administration Client or through JSON-based configuration files. - **Advanced Streaming**: Supports message streaming via the Advanced Message Queuing Protocol (AMQP). ## Limitations The current version of the emulator does **not** support the following: - **Protocols**: JMS protocol streaming and AMQP Web Sockets (AMQP over TCP is the only supported transport). - **Messaging Patterns**: Transactions, auto-forwarding (queue chaining), and message lock renewal. - **Validation**: Enforcements such as maximum entity counts or maximum message sizes. - **Metrics**: Property-based message counts for queues, topics, and subscriptions may be inaccurate. The following Azure-native features are currently unavailable in the emulator: - **Scaling & Resiliency**: Autoscale, Geo-disaster recovery, and Large Message support. - **Monitoring**: Visual metrics, alerts, and telemetry dashboards. ## Samples Explore the following samples to get started with Service Bus on LocalStack: - [Azure Functions App with Service Bus Messaging](https://github.com/localstack/localstack-azure-samples/blob/main/samples/function-app-service-bus/dotnet/) - [Azure Service Bus with Spring Boot](https://github.com/localstack/localstack-azure-samples/tree/main/samples/servicebus/java) ## API Coverage # Service Bus Data Plane > Get started with Azure Service Bus Data Plane on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Service Bus Data Plane APIs let you operate messaging entities through the namespace endpoint directly. This data plane REST API allows for direct interaction with queues, topics, and subscriptions. It supports core messaging operations including sending, peeking, and receiving messages, as well as batch processing. For more information, see [Azure Service Bus REST API](https://learn.microsoft.com/rest/api/servicebus/service-bus-runtime-rest). In LocalStack, they are useful for validating data-plane behavior without calling Azure cloud endpoints. LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Service Bus Data Plane APIs. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Service Bus Data Plane integration with LocalStack. ## Getting started This guide is designed for users new to Service Bus Data Plane APIs and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group for your Service Bus resources: ```bash az group create \ --name rg-servicebus-dp-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-servicebus-dp-demo", "location": "westeurope", "name": "rg-servicebus-dp-demo", "properties": { "provisioningState": "Succeeded" }, ... } ``` ### Create a Service Bus namespace Create a namespace and capture its data-plane endpoint: ```bash az servicebus namespace create \ --resource-group rg-servicebus-dp-demo \ --name sbnsdoc84 \ --location westeurope \ --sku Standard ``` ```bash title="Output" { "name": "sbnsdoc84", "serviceBusEndpoint": "https://sbnsdoc84.localhost.localstack.cloud:4511", "provisioningState": "Succeeded", ... } ``` Store the HTTP endpoint for data-plane REST calls: ```bash SB_ENDPOINT="http://sbnsdoc84.localhost.localstack.cloud:4511" ``` ### Create and inspect a queue entity Create a queue via the data-plane Entity `Put` API: ```bash curl -s -X PUT "$SB_ENDPOINT/dpqueue?api-version=2017-04" \ -H "Content-Type: application/atom+xml;type=entry;charset=utf-8" \ -d 'dpqueue' ``` ```xml title="Output" dpqueue Active ... ``` Get the queue entity via Entity `Get`: ```bash curl -s -X GET "$SB_ENDPOINT/dpqueue?api-version=2017-04" ``` ```xml title="Output" dpqueue 0 Active ... ``` ### Create and inspect a topic subscription Create a topic entity: ```bash curl -s -X PUT "$SB_ENDPOINT/dptopic?api-version=2017-04" \ -H "Content-Type: application/atom+xml;type=entry;charset=utf-8" \ -d 'dptopic' ``` Create a subscription via Subscription `Put`: ```bash curl -s -X PUT "$SB_ENDPOINT/dptopic/subscriptions/dpsub?api-version=2017-04" \ -H "Content-Type: application/atom+xml;type=entry;charset=utf-8" \ -d 'dpsub' ``` ```xml title="Output" dpsub Active 10 ... ``` Get the subscription via Subscription `Get`: ```bash curl -s -X GET "$SB_ENDPOINT/dptopic/subscriptions/dpsub?api-version=2017-04" ``` ```xml title="Output" dpsub 0 Active ... ``` ### Delete subscription and entities Delete the subscription via Subscription `Delete`: ```bash curl -s -X DELETE "$SB_ENDPOINT/dptopic/subscriptions/dpsub?api-version=2017-04" ``` Delete entities via Entity `Delete`: ```bash curl -s -X DELETE "$SB_ENDPOINT/dptopic?api-version=2017-04" curl -s -X DELETE "$SB_ENDPOINT/dpqueue?api-version=2017-04" ``` ## Features The emulator includes the following core capabilities: - **Data Plane REST API**: Supports message-level operations, including Send, Receive, and Peek. - **Control Plane REST API**: Enables CRUD operations for namespaces and messaging entities (queues, topics, and subscriptions) via Azure Resource Manager (ARM). - **Multiple Authentication Modes**: Supports both Connection String and Managed Identity authentication. - **Containerized Deployment**: Runs as a lightweight, Linux-based Docker container. - **Cross-Platform Compatibility**: Fully compatible with Windows, macOS, and Linux environments. - **Flexible Configuration**: Manage Service Bus entities via the Service Bus Administration Client or through JSON-based configuration files. - **Advanced Streaming**: Supports message streaming via the Advanced Message Queuing Protocol (AMQP). ## Limitations The current version of the emulator does **not** support the following: - **Protocols**: JMS protocol streaming and AMQP Web Sockets (AMQP over TCP is the only supported transport). - **Messaging Patterns**: Transactions, auto-forwarding (queue chaining), and message lock renewal. - **Validation**: Enforcements such as maximum entity counts or maximum message sizes. - **Metrics**: Property-based message counts for queues, topics, and subscriptions may be inaccurate. The following Azure-native features are currently unavailable in the emulator: - **Scaling & Resiliency**: Autoscale, Geo-disaster recovery, and Large Message support. - **Monitoring**: Visual metrics, alerts, and telemetry dashboards. ## Samples Explore the following samples to get started with Service Bus on LocalStack: - [Azure Functions App with Service Bus Messaging](https://github.com/localstack/localstack-azure-samples/blob/main/samples/function-app-service-bus/dotnet/) - [Azure Service Bus with Spring Boot](https://github.com/localstack/localstack-azure-samples/tree/main/samples/servicebus/java) ## API Coverage # SQL Database > Get started with Azure SQL in LocalStack for Azure. import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure SQL is a managed relational database service for building cloud-native applications with familiar SQL Server tooling. It supports creating logical servers, provisioning databases, and configuring operational features such as firewall access and retention policies. This makes it a common choice for transactional workloads and application backends. For more information, see [What is Azure SQL Database?](https://learn.microsoft.com/azure/azure-sql/database/sql-database-paas-overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure SQL Database. The supported APIs are listed in the [API Coverage](#api-coverage) section. ## Getting started This guide is designed for users new to Azure SQL Database and assumes basic knowledge of the Azure CLI and `lstk az`. The following example creates a SQL server and database, configures firewall access, and defines retention and encryption settings. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to contain your SQL resources: ```bash az group create --name rg-sql-demo --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sql-demo", "location": "westeurope", "name": "rg-sql-demo", "properties": { "provisioningState": "Succeeded" }, ... } ``` ### Create and inspect a SQL server Create a logical SQL server to host your databases: ```bash az sql server create \ --name sqlsrvdoc85 \ --resource-group rg-sql-demo \ --location westeurope \ --admin-user lsadmin \ --admin-password "LocalstackSqlPassw0rd" ``` ```bash title="Output" { "administratorLogin": "lsadmin", ... "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sql-demo/providers/Microsoft.Sql/servers/sqlsrvdoc85", ... "location": "westeurope", ... "name": "sqlsrvdoc85", ... "type": "Microsoft.Sql/servers", ... } ``` Get the SQL server details to verify it is ready: ```bash az sql server show --name sqlsrvdoc85 --resource-group rg-sql-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sql-demo/providers/Microsoft.Sql/servers/sqlsrvdoc85", "name": "sqlsrvdoc85", "location": "westeurope", "state": "Ready", "publicNetworkAccess": "Enabled", "type": "Microsoft.Sql/servers", ... } ``` ### Create and query a database Create a database on the SQL server: ```bash az sql db create \ --name sqldbdoc85 \ --resource-group rg-sql-demo \ --server sqlsrvdoc85 \ --service-objective S0 \ --compute-model Provisioned ``` ```bash title="Output" { ... "catalogCollation": "SQL_Latin1_General_CP1_CI_AS", "collation": "SQL_Latin1_General_CP1_CI_AS", "creationDate": "2026-03-24T09:32:54.177434+00:00", "currentBackupStorageRedundancy": "Geo", "currentServiceObjectiveName": "GP_Gen5_2", "currentSku": { "capacity": 2, "family": "Gen5", "name": "S0", "size": null, "tier": "GeneralPurpose" }, "databaseId": "62951a4b-e3b7-41ce-b7e7-1d4860801828", ... "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sql-demo/providers/Microsoft.Sql/servers/sqlsrvdoc85/databases/sqldbdoc85", ... "name": "sqldbdoc85", ... "sku": { "capacity": 2, "family": "Gen5", "name": "S0", "size": null, "tier": "GeneralPurpose" }, ... "status": "Online", ... } ``` Verify the database status to confirm successful creation: ```bash az sql db show \ --name sqldbdoc85 \ --resource-group rg-sql-demo ``` ```bash title="Output" { ... "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sql-demo/providers/Microsoft.Sql/servers/sqlsrvdoc85/databases/sqldbdoc85", ... "name": "sqldbdoc85", ... "status": "Online", ... } ``` List the databases on the SQL server: ```bash az sql db list \ --resource-group rg-sql-demo \ --server sqlsrvdoc85 ``` ```bash title="Output" [ { ... "name": "master", ... }, { ... "name": "sqldbdoc85", ... } ] ``` ### Add a firewall rule Create a firewall rule to allow client access: ```bash az sql server firewall-rule create \ --resource-group rg-sql-demo \ --server sqlsrvdoc85 \ --name AllowLocal \ --start-ip-address 0.0.0.0 \ --end-ip-address 255.255.255.255 ``` ```bash title="Output" { "name": "AllowLocal", "startIpAddress": "0.0.0.0", "endIpAddress": "255.255.255.255", "type": "Microsoft.Sql/servers/firewallRules", ... } ``` ### Configure transparent data encryption Enable transparent data encryption on the database: ```bash az sql db tde set \ --database sqldbdoc85 \ --server sqlsrvdoc85 \ --resource-group rg-sql-demo \ --status Enabled ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sql-demo/providers/Microsoft.Sql/servers/sqlsrvdoc85/databases/sqldbdoc85/transparentDataEncryption/current", "name": "current", "resourceGroup": "rg-sql-demo", ... "state": "Enabled", "type": "Microsoft.Sql/servers/databases/transparentDataEncryption" ... } ``` ### Configure backup retention policies Configure a short-term backup retention policy: ```bash az sql db str-policy set \ --name sqldbdoc85 \ --server sqlsrvdoc85 \ --resource-group rg-sql-demo \ --retention-days 7 \ --diffbackup-hours 24 ``` ```bash title="Output" { "diffBackupIntervalInHours": 24, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sql-demo/providers/Microsoft.Sql/servers/sqlsrvdoc85/databases/sqldbdoc85/backupShortTermRetentionPolicies/default", "name": "default", "resourceGroup": "rg-sql-demo", "retentionDays": 7, "type": "Microsoft.Sql/servers/databases/backupShortTermRetentionPolicies" ... } ``` Configure a long-term backup retention policy: ```bash az sql db ltr-policy set \ --name sqldbdoc85 \ --server sqlsrvdoc85 \ --resource-group rg-sql-demo \ --weekly-retention "P4W" \ --monthly-retention "P12M" \ --yearly-retention "P5Y" \ --week-of-year 16 ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-sql-demo/providers/Microsoft.Sql/servers/sqlsrvdoc85/databases/sqldbdoc85/backupLongTermRetentionPolicies/default", "monthlyRetention": "P12M", "name": "default", "resourceGroup": "rg-sql-demo", "timeBasedImmutability": null, "timeBasedImmutabilityMode": null, "type": "Microsoft.Sql/servers/databases/backupLongTermRetentionPolicies", "weekOfYear": 16, "weeklyRetention": "P4W", "yearlyRetention": "P5Y" ... } ``` ## Features The Azure SQL emulator supports the following features: - **Server lifecycle management**: Create, update, delete, get, and list logical SQL servers. - **Database CRUD**: Create, update, delete, get, list, and rename databases on a logical server. - **Firewall rules**: Create, update, delete, get, and list server-level firewall rules. - **Transparent data encryption (TDE)**: Create or update, get, and list TDE configurations per database. - **Short-term backup retention policies**: Create or update, get, update, and list short-term retention policies per database. - **Long-term backup retention policies**: Create or update, get, and list long-term retention policies per database. - **Database security alert policies**: Create or update, get, and list security alert policies per database. - **Server connection policies**: Create or update, get, and list connection policies per server. - **SQL vulnerability assessments**: Create or update, get, and list vulnerability assessment settings per server. - **Database schema introspection**: Get and list schemas, tables, and columns by querying the live SQL Server instance. - **Server name availability check**: Check whether a server name is available for use. - **Restorable dropped databases**: Get and list restorable dropped databases (stub: always returns an empty list). - **Replication links**: List replication links for a database (stub: always returns an empty list). - **Asynchronous provisioning**: Server and database creation use async operations with polling headers, matching real Azure behavior. ## Limitations - **No data persistence across restarts**: SQL Server containers are ephemeral. All databases, schemas, and data are lost when the LocalStack emulator is stopped or restarted. - **Backup retention policies are metadata-only**: Short-term and long-term retention policies are stored but no actual backup or restore operations are performed. - **Transparent data encryption is metadata-only**: TDE settings are stored but no actual encryption is applied to the database files. - **Security alert policies are metadata-only**: Alert policies are stored but do not trigger notifications or log monitoring. - **Vulnerability assessments are metadata-only**: Assessment settings are stored but no scans are executed. - **Server connection policies are metadata-only**: Connection policies are stored but do not affect network behavior. - **Replication links always empty**: No geo-replication or active replication is supported. - **Elastic pools, failover groups, and geo-replication are not supported**: These features are not implemented. - **MSSQL EULA acceptance required**: The `MSSQL_ACCEPT_EULA` environment variable must be set to `Y` before creating any SQL server. ## Configuration The behavior of the Azure SQL emulator can be customized using the environment variables listed below. | **Variable** | **Description** | **Type** | **Default** | | -------------------------------------- | --------------------------------------------------------------------------------------------------------- | -------- | ----------- | | `MSSQL_ACCEPT_EULA` | Accept the Microsoft SQL Server End-User License Agreement. Must be set to `Y` to create SQL servers. | String | (unset) | | `ALLOW_MULTIPLE_SQL_SERVER_DEPLOYMENTS`| Allow provisioning more than one SQL Server Docker container. Set to `1` or `true` to enable. | Boolean | `0` | ## Samples The following sample demonstrates how to use Azure SQL Database with LocalStack for Azure: - [Web App and SQL Database](https://github.com/localstack/localstack-azure-samples/blob/main/samples/web-app-sql-database/python/) ## API Coverage # Storage Account > Get started with Azure Storage Accounts in LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction An Azure storage account serves as a centralized container for all your data objects, including blobs, files, queues, and tables. It provides a unique, globally accessible namespace reachable via HTTP or HTTPS. For more information, see [Overview of storage accounts](https://learn.microsoft.com/en-us/azure/storage/common/storage-account-overview). LocalStack for Azure provides a local environment for building and testing applications that make use of blobs, queues, and tables. For more information, see: - [Blob Storage](/azure/services/blob-storage) - [Queue Storage](/azure/services/queue-storage) - [Table Storage](/azure/services/table-storage) The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Storage Account's integration with LocalStack. ## Getting started This guide is designed for users new to Azure Storage Accounts and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group for your storage account resources: ```bash az group create \ --name rg-storage-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-storage-demo", "location": "westeurope", "managedBy": null, "name": "rg-storage-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a storage account Create a storage account with the `StorageV2` kind and `Standard_LRS` SKU: ```bash az storage account create \ --name stordoc86acct \ --resource-group rg-storage-demo \ --location westeurope \ --sku Standard_LRS \ --kind StorageV2 ``` ```bash title="Output" { ... "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-storage-demo/providers/Microsoft.Storage/storageAccounts/stordoc86acct", ... "kind": "StorageV2", "location": "westeurope", "name": "stordoc86acct", ... "primaryEndpoints": { "blob": "https://stordoc86acct.blob.core.azure.localhost.localstack.cloud:4566", "queue": "https://stordoc86acct.queue.core.azure.localhost.localstack.cloud:4566", "table": "https://stordoc86acct.table.core.azure.localhost.localstack.cloud:4566", ... }, "provisioningState": "Succeeded", ... } ``` ### Authentication There are three ways to authenticate storage commands against the emulator: #### Storage account key Retrieve the account key and pass it with `--account-name` and `--account-key`: ```bash ACCOUNT_KEY=$(az storage account keys list \ --account-name stordoc86acct \ --resource-group rg-storage-demo \ --query "[0].value" \ --output tsv) az storage container list \ --account-name stordoc86acct \ --account-key "$ACCOUNT_KEY" ``` #### Login credentials Use `--auth-mode login` to authenticate with the current session credentials: ```bash az storage container list \ --account-name stordoc86acct \ --auth-mode login ``` #### Connection string Bundle the account name and key into a single value: ```bash CONNECTION_STRING=$(az storage account show-connection-string \ --name stordoc86acct \ --resource-group rg-storage-demo \ --query connectionString -o tsv) az storage container list \ --connection-string "$CONNECTION_STRING" ``` The remaining examples in this guide use connection strings for brevity. ### Manage account keys and connection string List the storage account access keys: ```bash az storage account keys list \ --account-name stordoc86acct \ --resource-group rg-storage-demo ``` ```bash title="Output" [ { "keyName": "key1", "permissions": "FULL", "value": "MWFjYTgyZjgtYzU0My00NjE0LThmZDctNzlkODg5ZjU4ZTE5", "..." }, { "keyName": "key2", "permissions": "FULL", "value": "NzliNzVhN2EtYTcwZC00ZTg4LWJkMTQtYjg4MWNlMDJjZDcx", "..." } ] ``` Regenerate the primary key: ```bash az storage account keys renew \ --account-name stordoc86acct \ --resource-group rg-storage-demo \ --key key1 ``` Fetch a connection string for data-plane operations: ```bash az storage account show-connection-string \ --name stordoc86acct \ --resource-group rg-storage-demo ``` ```bash title="Output" { "connectionString": "DefaultEndpointsProtocol=https;EndpointSuffix=core.azure.localhost.localstack.cloud:4566;AccountName=stordoc86acct;AccountKey=YWQ5Y2Q2NDYtZTJmOC00ZjU3LWFmOTEtNzk5MjAxNzE1OWQx;BlobEndpoint=https://stordoc86acct.blob.core.azure.localhost.localstack.cloud:4566;FileEndpoint=https://stordoc86acct.file.core.azure.localhost.localstack.cloud:4566;QueueEndpoint=https://stordoc86acct.queue.core.azure.localhost.localstack.cloud:4566;TableEndpoint=https://stordoc86acct.table.core.azure.localhost.localstack.cloud:4566" } ``` ## Features The Storage Account emulator supports the following features: - **Control plane REST API**: Storage account CRUD (create, read, update, delete, list), account key management, and name availability checks via Azure Resource Manager. - **Multiple authentication modes**: Storage account key, login credentials, and connection strings. - **Storage account management**: Create, update, delete, and list storage accounts. Supports `StorageV2`, `BlobStorage`, and `Storage` account kinds with configurable SKU, access tier, and TLS version. - **Account key management**: List and regenerate storage account keys (`key1`/`key2`). - **Connection string generation**: Retrieve ready-to-use connection strings containing all service endpoints (Blob, Queue, Table, File). ## Limitations - **Header validation**: Unsupported request headers or parameters are silently accepted instead of being rejected. - **API version enforcement**: The emulator does not validate the `x-ms-version` header; all API versions are accepted. ## Samples The following samples demonstrate how to use Storage Accounts with LocalStack for Azure: - [Azure Functions Sample with LocalStack for Azure](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-storage-http/dotnet) - [Azure Functions App with Managed Identity](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-managed-identity/python) - [Azure Web App with Managed Identity](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-managed-identity/python) ## API Coverage # Table Storage > Get started with Azure Table Storage in LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Table Storage is a NoSQL key-value store designed for large volumes of semi-structured data, useful for lightweight metadata, lookup records, and simple operational datasets. Data is organized into tables, partitions, and entities addressed by PartitionKey and RowKey. It offers schemaless flexibility at a fraction of the cost of traditional SQL, making it easy to adapt as your application evolves. For more information, see [What is Azure Table storage?](https://learn.microsoft.com/en-us/azure/storage/tables/table-storage-overview) LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Table Storage. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Table Storage's integration with LocalStack. ## Getting started This guide is designed for users new to Table Storage and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to contain your storage resources: ```bash az group create \ --name rg-table-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-table-demo", "location": "westeurope", "managedBy": null, "name": "rg-table-demo", "properties": { "provisioningState": "Succeeded" }, "tags": null, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a storage account Create a storage account in the resource group: ```bash az storage account create \ --name sttabledemols \ --resource-group rg-table-demo \ --location westeurope \ --sku Standard_LRS ``` ```bash title="Output" { ... "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-table-demo/providers/Microsoft.Storage/storageAccounts/sttabledemols", ... "name": "sttabledemols", ... "placement": null, "primaryEndpoints": { "blob": "https://sttabledemols.blob.core.azure.localhost.localstack.cloud:4566", ... "table": "https://sttabledemols.table.core.azure.localhost.localstack.cloud:4566", ... }, .... } ``` ### Authentication There are three ways to authenticate storage table commands against the emulator: #### Storage account key Retrieve the account key and pass it with `--account-name` and `--account-key`: ```bash ACCOUNT_KEY=$(az storage account keys list \ --account-name sttabledemols \ --resource-group rg-table-demo \ --query "[0].value" \ --output tsv) az storage table list \ --account-name sttabledemols \ --account-key "$ACCOUNT_KEY" ``` #### Login credentials Use `--auth-mode login` to authenticate with the current session credentials: ```bash az storage table list \ --account-name sttabledemols \ --auth-mode login ``` #### Connection string Bundle the account name and key into a single value: ```bash CONNECTION_STRING=$(az storage account show-connection-string \ --name sttabledemols \ --resource-group rg-table-demo \ --query connectionString -o tsv) az storage table list \ --connection-string "$CONNECTION_STRING" ``` The remaining examples in this guide use connection strings for brevity. ### Create and inspect a table Create a table: ```bash az storage table create \ --name apptable \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" { "created": true } ``` Verify the table exists: ```bash az storage table exists \ --name apptable \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" { "exists": true } ``` List tables in the storage account: ```bash az storage table list \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" [ { "name": "apptable" } ] ``` ### Insert and query entities Insert an entity into the table: ```bash az storage entity insert \ --table-name apptable \ --entity PartitionKey=demo RowKey=1 name=Alice score=100 \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" { "content": { "PartitionKey": "demo", "RowKey": "1", "name": "Alice", "score": 100, ... }, "etag": "W/\"datetime'...'\"", ... } ``` Retrieve the entity by its partition key and row key: ```bash az storage entity show \ --table-name apptable \ --partition-key demo \ --row-key 1 \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" { "PartitionKey": "demo", "RowKey": "1", "name": "Alice", "score": 100, ... } ``` Query entities by partition key: ```bash az storage entity query \ --table-name apptable \ --filter "PartitionKey eq 'demo'" \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" { "items": [ { "PartitionKey": "demo", "RowKey": "1", "name": "Alice", "score": 100, ... } ], "nextMarker": {} } ``` ### Update, merge, and delete entities Update the entity with a merge operation: ```bash az storage entity merge \ --table-name apptable \ --entity PartitionKey=demo RowKey=1 score=101 \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" { "etag": "W/\"datetime'...'\"", ... } ``` Delete the entity and verify the table is empty: ```bash az storage entity delete \ --table-name apptable \ --partition-key demo \ --row-key 1 \ --connection-string "$CONNECTION_STRING" az storage entity query \ --table-name apptable \ --connection-string "$CONNECTION_STRING" ``` ```bash title="Output" { "deleted": null } { "items": [], "nextMarker": {} } ``` ## Features The Table Storage emulator supports the following features: - **Data plane REST API**: Table CRUD, entity operations (insert, query, merge, replace, delete), OData query filters, and batch/transaction requests. - **Control plane REST API**: Create and get tables, get and set table service properties via Azure Resource Manager. - **Multiple authentication modes**: Storage account key, login credentials, and connection strings. - **Entity operations**: Insert, query, show, merge, replace, and delete entities with schemaless, key-value data addressed by `PartitionKey` and `RowKey`. - **OData query support**: Filter and project entities using OData expressions (e.g., `PartitionKey eq 'demo'`). - **Batch operations**: Entity batch (transaction) requests are proxied with correct URL and authorization rewriting. ## Limitations - **No data persistence across restarts**: Table data is not persisted and is lost when the LocalStack emulator is stopped or restarted. - **Table service properties**: `set_service_properties` is a no-op and `get_service_properties` returns empty defaults, unlike Azure where CORS, logging, and metrics settings are persisted and applied. - **Storage account keys**: Keys are emulator-generated rather than managed by Azure. - **Header validation**: Unsupported request headers or parameters are silently accepted (Azurite runs in loose mode) instead of being rejected. - **API version enforcement**: The emulator does not validate the `x-ms-version` header; all API versions are accepted. - **RBAC enforcement is opt-in**: By default, data-plane operations succeed regardless of role assignments. Set `LS_AZURE_ENFORCE_RBAC` to require the caller to hold a role such as `Storage Table Data Contributor`; see [Role Assignment: Enabling RBAC enforcement](/azure/services/role-assignment/#enabling-rbac-enforcement). ## Samples The following sample demonstrates how to use Table Storage with LocalStack for Azure: - [Azure Functions Sample with LocalStack for Azure](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-storage-http/dotnet) ## API Coverage # Virtual Network > Get started with Azure Virtual Network on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Virtual Network (VNet) is the core networking service for isolating and routing Azure resources in private IP address spaces. It lets you define address ranges, create subnets, and control network behavior for applications. Virtual networks are commonly used to model secure, segmented network topologies in cloud environments. For more information, see [What is Azure Virtual Network?](https://learn.microsoft.com/en-us/azure/virtual-network/virtual-networks-overview). LocalStack for Azure provides a local environment to build and test Azure networking resources, such as virtual networks, private endpoints, and private DNS zones. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Virtual Network's integration with LocalStack. ## Getting started This guide is designed for users new to Virtual Network and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group for your networking resources: ```bash az group create \ --name rg-vnet-demo \ --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-vnet-demo", "location": "westeurope", "managedBy": null, "name": "rg-vnet-demo", "properties": { "provisioningState": "Succeeded" }, ... } ``` ### Create and inspect a virtual network Create a virtual network with a `10.0.0.0/16` address space: ```bash az network vnet create \ --name vnet-doc78 \ --resource-group rg-vnet-demo \ --location westeurope \ --address-prefixes 10.0.0.0/16 ``` ```bash title="Output" { "newVNet": { "name": "vnet-doc78", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-vnet-demo/providers/Microsoft.Network/virtualNetworks/vnet-doc78", "location": "westeurope", "addressSpace": { "addressPrefixes": ["10.0.0.0/16"] }, "provisioningState": "Succeeded", ... } } ``` Get the virtual network (VNet) details: ```bash az network vnet show \ --name vnet-doc78 \ --resource-group rg-vnet-demo ``` ```bash title="Output" { "name": "vnet-doc78", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-vnet-demo/providers/Microsoft.Network/virtualNetworks/vnet-doc78", "location": "westeurope", "addressSpace": { "addressPrefixes": ["10.0.0.0/16"] }, "provisioningState": "Succeeded", ... } ``` ### Create and manage subnets Create a subnet: ```bash az network vnet subnet create \ --name subnet1 \ --resource-group rg-vnet-demo \ --vnet-name vnet-doc78 \ --address-prefixes 10.0.1.0/24 ``` ```bash title="Output" { "name": "subnet1", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-vnet-demo/providers/Microsoft.Network/virtualNetworks/vnet-doc78/subnets/subnet1", "addressPrefix": "10.0.1.0/24", "provisioningState": "Succeeded", ... } ``` Retrieve new subnet details and list all VNet subnets ```bash az network vnet subnet show \ --name subnet1 \ --resource-group rg-vnet-demo \ --vnet-name vnet-doc78 az network vnet subnet list \ --resource-group rg-vnet-demo \ --vnet-name vnet-doc78 ``` ```bash title="Output" { "name": "subnet1", "addressPrefix": "10.0.1.0/24", ... } [ { "name": "subnet1", "addressPrefix": "10.0.1.0/24", ... } ] ``` Add a second subnet to the virtual network, remove the first , and relist all subnets: ```bash az network vnet subnet create \ --name subnet2 \ --resource-group rg-vnet-demo \ --vnet-name vnet-doc78 \ --address-prefixes 10.0.2.0/24 az network vnet subnet delete \ --name subnet1 \ --resource-group rg-vnet-demo \ --vnet-name vnet-doc78 az network vnet subnet list \ --resource-group rg-vnet-demo \ --vnet-name vnet-doc78 ``` ```bash title="Output" { "name": "subnet2", "addressPrefix": "10.0.2.0/24", ... } [ { "name": "subnet2", "addressPrefix": "10.0.2.0/24", ... } ] ``` ### Update virtual network properties Update DNS servers and tags on the VNet: ```bash az network vnet update \ --name vnet-doc78 \ --resource-group rg-vnet-demo \ --dns-servers 8.8.8.8 8.8.4.4 \ --set tags.environment=test tags.project=localstack ``` ```bash title="Output" { "name": "vnet-doc78", "dhcpOptions": { "dnsServers": ["8.8.8.8", "8.8.4.4"] }, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-vnet-demo/providers/Microsoft.Network/virtualNetworks/vnet-doc78", "provisioningState": "Succeeded", "tags": { "environment": "test", "project": "localstack" }, ... } ``` ### Delete and verify Delete the VNet and validate that no virtual networks remain in the resource group: ```bash az network vnet delete \ --name vnet-doc78 \ --resource-group rg-vnet-demo az network vnet list --resource-group rg-vnet-demo ``` ```bash title="Output" [] ``` ## Features The Virtual Network emulator supports the following features: - **Virtual networks**: Create, update, delete, list, and get virtual networks with configurable address spaces, DNS servers, and DDoS protection settings. - **Subnets**: Full lifecycle management of subnets within virtual networks, including address prefix allocation, service endpoint configuration, and NSG/route table associations. - **Network security groups**: Create and manage network security groups with custom security rules. Default rules (AllowVnetInBound, AllowAzureLoadBalancerInBound, DenyAllInBound, AllowVnetOutBound, AllowInternetOutBound, DenyAllOutBound) are automatically provisioned. - **Route tables**: Create and manage route tables with custom route entries supporting next hop types such as VirtualAppliance, VirtualNetworkGateway, Internet, and VnetLocal. - **Public IP addresses**: Create and manage public IP addresses with Static or Dynamic allocation methods, Standard or Basic SKUs, and availability zone configuration. - **Public IP prefixes**: Create and manage public IP prefixes with configurable prefix lengths and SKU settings. - **NAT gateways**: Create and manage NAT gateways with public IP address and public IP prefix associations. - **Network interfaces**: Create and manage network interfaces with IP configurations, dynamic IP allocation from subnets, accelerated networking, and IP forwarding settings. - **Private DNS zones**: Create and manage private DNS zones with virtual network links, registration enablement, and A record sets. - **Private endpoints**: Create and manage private endpoints with automatic network interface provisioning, private link service connections, and private DNS zone group integration. - **Bastion hosts**: Create and manage bastion hosts with IP configuration validation, SKU selection (Basic, Standard, Premium), and scale unit configuration. ## Limitations - **No network traffic routing**: The emulator does not route network traffic or enforce security rules. Resources are stored and returned with correct metadata, but no packet-level behavior is applied. - **IPv6**: IPv6 fields are accepted in requests but are not functional. All IP allocation operates on IPv4 address spaces only. - **Private DNS record types**: Only A record sets are supported in private DNS zones. Other record types (CNAME, MX, TXT, SRV, AAAA) are not available. - **Public IP addresses**: Addresses are locally generated and do not represent routable IPs on the public internet. - **Bastion host connectivity**: Feature flags such as tunneling, file copy, and Kerberos authentication are stored as configuration but do not provide actual connectivity. - **VNet peering**: Virtual network peering is not supported. - **VPN and ExpressRoute gateways**: VPN gateways and ExpressRoute circuits are not implemented. - **Load balancers**: Azure Load Balancer resources are not implemented. - **Application gateways**: Application Gateway resources are not implemented. - **Network watchers**: Network Watcher and flow log resources are not implemented. - **No data persistence**: Network resources are not persisted and are lost when the emulator is stopped or restarted. ## Samples The following samples demonstrate how to use Virtual Network with LocalStack for Azure: - [Function App and Service Bus](https://github.com/localstack/localstack-azure-samples/tree/main/samples/function-app-service-bus/dotnet/) - [Web App and Cosmos DB for MongoDB API](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-cosmosdb-mongodb-api/python/) ## API Coverage # App Services > Get started with Azure App Services on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure App Service is a fully managed platform for hosting web applications, mobile backends, and RESTful APIs without infrastructure overhead. It supports multiple runtimes including .NET, Java, Node.js, Python, and PHP on both Windows and Linux, or as custom containers. For more information, see the [App Service overview](https://learn.microsoft.com/azure/app-service/overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure App Services. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Web App's integration with LocalStack. ## Getting started This guide is designed for users new to Web App and assumes basic knowledge of the Azure CLI and our `lstk az` proxy. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group for your Web App resources: ```bash az group create \ --name rg-web-demo \ --location westeurope ``` ```bash title="Output" { "name": "rg-web-demo", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-web-demo", "location": "westeurope", "properties": { "provisioningState": "Succeeded" }, ... } ``` ### Create an App Service plan Create an App Service plan that will host the web app: ```bash az appservice plan create \ --name asp-web-doc89 \ --resource-group rg-web-demo \ --location westeurope \ --sku B1 ``` ```bash title="Output" { "name": "asp-web-doc89", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-web-demo/providers/Microsoft.Web/serverfarms/asp-web-doc89", "location": "westeurope", "provisioningState": "Succeeded", "sku": { "name": "B1", "tier": "Basic", ... }, ... } ``` ### Create and inspect a Web App Create a web app using a Python runtime: ```bash az webapp create \ --name ls-web-doc89 \ --resource-group rg-web-demo \ --plan asp-web-doc89 \ --runtime "PYTHON:3.11" ``` ```bash title="Output" { "name": "ls-web-doc89", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-web-demo/providers/Microsoft.Web/sites/ls-web-doc89", "type": "Microsoft.Web/sites", "location": "westeurope", "defaultHostName": "ls-web-doc89.azurewebsites.azure.localhost.localstack.cloud:4566", "state": "Running", ... } ``` Get the web app: ```bash az webapp show \ --name ls-web-doc89 \ --resource-group rg-web-demo ``` ### Read and update web app configuration Read the current web app configuration: ```bash az webapp config show \ --name ls-web-doc89 \ --resource-group rg-web-demo ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-web-demo/providers/Microsoft.Web/sites/ls-web-doc89/config/web", "linuxFxVersion": "PYTHON|3.11", "http20Enabled": true, "alwaysOn": false, "ftpsState": "FtpsOnly", ... } ``` Update the web app configuration: ```bash az webapp config set \ --name ls-web-doc89 \ --resource-group rg-web-demo \ --always-on false \ --http20-enabled true ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-web-demo/providers/Microsoft.Web/sites/ls-web-doc89", "http20Enabled": true, "alwaysOn": false, ... } ``` ### Configure application settings Set an application setting that instructs the SCM endpoint to build the application during deployment: ```bash az webapp config appsettings set \ --name ls-web-doc89 \ --resource-group rg-web-demo \ --settings SCM_DO_BUILD_DURING_DEPLOYMENT="true" ``` ```bash title="Output" [ { "name": "SCM_DO_BUILD_DURING_DEPLOYMENT", "slotSetting": false, "value": "true" } ] ``` Verify the setting was applied: ```bash az webapp config appsettings list \ --name ls-web-doc89 \ --resource-group rg-web-demo \ --query "[?name=='SCM_DO_BUILD_DURING_DEPLOYMENT']" ``` ```bash title="Output" [ { "name": "SCM_DO_BUILD_DURING_DEPLOYMENT", "slotSetting": false, "value": "true" } ] ``` ### Deploy a Python application Create a minimal Flask application and package it as a zip archive for deployment: ```bash mkdir python_webapp && cd python_webapp cat > app.py << 'EOF' import json from flask import Flask app = Flask(__name__) @app.route("/") def index(): return json.dumps({"status": "Great Success!"}) if __name__ == "__main__": app.run() EOF cat > requirements.txt << 'EOF' Flask==3.0.3 gunicorn==23.0.0 Werkzeug==3.0.4 EOF zip -r ../python_webapp.zip . cd .. ``` Deploy the zip package to the web app: ```bash az webapp deploy \ --name ls-web-doc89 \ --resource-group rg-web-demo \ --src-path python_webapp.zip \ --type zip \ --async true ``` ```bash title="Output" { "status": 4, "complete": true, "active": true, ... } ``` Verify that the web app is in the `Running` state: ```bash az webapp show \ --name ls-web-doc89 \ --resource-group rg-web-demo \ --query "state" \ --output tsv ``` ```bash title="Output" Running ``` ### Delete and verify Delete the web app and verify it no longer appears: ```bash az webapp delete \ --name ls-web-doc89 \ --resource-group rg-web-demo az webapp list --resource-group rg-web-demo ``` ```bash title="Output" [] ``` ## Features The App Service emulator supports the following features: - **App Service Plans**: Create, get, list (subscription-wide and by resource group), and delete App Service Plans. - **Web Apps**: Create, get, list (subscription-wide and by resource group), and delete web apps. - **Application settings**: List and update application settings via `az webapp config appsettings set` and `az webapp config appsettings list`. - **Site configuration**: Read and update site configuration properties including `http20Enabled`, `alwaysOn`, `ftpsState`, and `linuxFxVersion`. - **Zip deployment**: Deploy application packages via `az webapp deploy --type zip`, triggering an Oryx-based container build. - **SCM deployment endpoints**: Zip deploy and publish endpoints accessible via the SCM subdomain. - **Deployment lifecycle**: Create, get, list, delete, and retrieve logs for individual deployments. - **Deployment status**: Poll production site deployment status for async deployments. - **Source control integration**: Create, get, update, delete, and sync source control configuration. - **Function App support**: List functions, list function keys, sync function triggers, sync functions, and query sync status. - **Publishing credentials**: Retrieve publishing credentials and publishing profile XML. - **Diagnostic logs configuration**: Get and update HTTP and application log settings. - **Azure Storage account mounts**: List and update Azure Storage account configurations on a site. - **Authentication settings**: Read auth settings (v1 and v2) for a site. - **Publishing policy controls**: Get and update FTP and SCM publishing credential policies. - **Connection strings**: List connection strings for a site. - **Managed identity**: System-assigned and user-assigned managed identities for web apps. - **Instance identifiers**: List web app instance identifiers. - **Slot configuration names**: List the names of settings and connection strings that are slot-specific. - **Runtime stack enumeration**: List available web app and Function App runtime stacks for both Linux and Windows. - **Geo region listing**: List available geographic regions for App Service deployments. - **Name availability check**: Validate that a web app name is available within the subscription. ## Limitations - **Deployment slots**: Only the production slot is supported. Staging slots and slot-swap operations are not implemented. - **Custom domains**: Binding custom hostnames to a web app is not supported. Apps are accessible only via the emulator-assigned `*.azurewebsites.azure.localhost.localstack.cloud` hostname. - **Managed TLS certificates**: SSL/TLS certificate provisioning and binding are not implemented. - **VNet integration and private endpoints**: Network isolation features are not enforced. - **Autoscaling**: App Service Plan capacity settings are accepted but not enforced by the emulator. - **Backup and restore**: The backup configuration endpoint returns an empty response. No backup or restore operations are performed. - **EasyAuth (authentication and authorization)**: Auth settings endpoints return fixed defaults. No identity provider flow or token validation is active. - **FTP/S deployments**: FTP publishing is not available. Use `az webapp deploy --type zip` or the SCM zip-deploy endpoint instead. - **Publishing policy persistence**: The `update_ftp_allowed` and `update_scm_allowed` endpoints accept requests but do not persist policy changes. - **No data persistence across restarts**: Web App state is held in memory and is lost when the LocalStack emulator is stopped or restarted. ## Configuration The behavior of the App Service emulator can be customized using the following environment variables. | **Variable** | **Default** | **Type** | **Description** | | --------------------------------------- | ------------------------------- | ----------------- | -------------------------------------------------------------------------- | | `ALLOW_MULTIPLE_ORYX_DEPLOYMENTS` | `"0"` | Boolean | Allow multiple Oryx build containers to run concurrently for a single app. | | `CONTAINER_CREATION_RETRIES` | `"5"` (web) / `"3"` (functions) | Integer | Number of attempts to create a container if the initial attempt fails. | | `DOCKER_PULL_TIMEOUT` | `"120"` | Integer (seconds) | Maximum time to wait for a Docker image pull to complete. | | `PORT_RESERVATION_DURATION` | `"120"` | Integer (seconds) | Duration for which a host port is reserved during container startup. | | `ORYX_BUILD_CONTAINER_IMAGE_TO_USE` | Oryx LTS build image | String | Specific Oryx build image to use when building application containers. | | `USE_LATEST_ORYX_BUILD_CONTAINER_IMAGE` | `"0"` | Boolean | Automatically fetch and use the latest Oryx build container image. | | `AZURE_FUNCTIONS_CORE_TOOLS_VERSION` | `"4.2.2"` | String | Version of Azure Functions Core Tools used for Function App deployments. | ## Samples The following samples demonstrate how to use App Services with LocalStack for Azure: - [Web App and Cosmos DB for MongoDB API](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-cosmosdb-mongodb-api/python/) - [Web App and Cosmos DB for NoSQL API](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-cosmosdb-nosql-api/python/) - [Web App and Managed Identities](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-managed-identity/python/) - [Web App and SQL Database](https://github.com/localstack/localstack-azure-samples/tree/main/samples/web-app-sql-database/python/) ## API Coverage # Web Test > Get started with Azure Monitor Web Tests on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Monitor Web Tests (availability tests) send HTTP probes to a URL from multiple geographic locations and alert when the endpoint is unavailable or slow. Web Tests are associated with an Application Insights component and report availability data alongside application telemetry. They are commonly used to monitor public-facing APIs and web applications for uptime and response time from a global perspective. For more information, see [Application Insights availability tests](https://learn.microsoft.com/en-us/azure/azure-monitor/app/availability-overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Monitor Web Tests. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Web Tests' integration with LocalStack. ## Getting started This guide walks you through creating a web test linked to an Application Insights component. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create --name rg-webtest-demo --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-webtest-demo", "location": "westeurope", "name": "rg-webtest-demo", "properties": { "provisioningState": "Succeeded" }, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create an Application Insights component Create an Application Insights component to attach the web test to: ```bash az monitor app-insights component create \ --app my-app-insights \ --resource-group rg-webtest-demo \ --location westeurope \ --kind web ``` ```bash title="Output" { "appId": "c62300bc-c7ae-5dd1-9f6c-08016bcbfbd9", "applicationType": "web", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-webtest-demo/providers/microsoft.insights/components/my-app-insights", "kind": "web", "location": "westeurope", "name": "my-app-insights", "provisioningState": "Succeeded", "resourceGroup": "rg-webtest-demo", "type": "microsoft.insights/components", ... } ``` ### Create a web test Retrieve the Application Insights resource ID, then create a standard availability test linked to it via a `hidden-link` tag: ```bash AI_ID=$(az monitor app-insights component show \ --app my-app-insights \ --resource-group rg-webtest-demo \ --query id \ --output tsv) az monitor app-insights web-test create \ --name my-web-test \ --resource-group rg-webtest-demo \ --location westeurope \ --defined-web-test-name "My Web Test" \ --web-test-kind standard \ --enabled true \ --frequency 300 \ --timeout 30 \ --locations Id=us-tx-sn1-azr \ --request-url "https://example.com" \ --http-verb GET \ --synthetic-monitor-id my-web-test \ --tags "hidden-link:$AI_ID=Resource" ``` ```bash title="Output" { "enabled": true, "frequency": 300, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-webtest-demo/providers/Microsoft.Insights/webtests/my-web-test", "kind": "standard", "location": "westeurope", "locations": [ { "Id": "us-tx-sn1-azr" } ], "name": "my-web-test", "request": { "httpVerb": "GET", "requestUrl": "https://example.com" }, "tags": { "hidden-link:/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-webtest-demo/providers/microsoft.insights/components/my-app-insights": "Resource" }, "timeout": 30, "type": "Microsoft.Insights/webtests", "webTestKind": "standard", "webTestName": "My Web Test" } ``` ### Show a web test Retrieve the details of a specific web test: ```bash az monitor app-insights web-test show \ --name my-web-test \ --resource-group rg-webtest-demo ``` ```bash title="Output" { "enabled": true, "frequency": 300, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-webtest-demo/providers/Microsoft.Insights/webtests/my-web-test", "kind": "standard", "location": "westeurope", "locations": [ { "Id": "us-tx-sn1-azr" } ], "name": "my-web-test", "request": { "httpVerb": "GET", "requestUrl": "https://example.com" }, "tags": { "hidden-link:/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-webtest-demo/providers/microsoft.insights/components/my-app-insights": "Resource" }, "timeout": 30, "type": "Microsoft.Insights/webtests", "webTestKind": "standard", "webTestName": "My Web Test" } ``` ### List web tests List all web tests in the resource group: ```bash az monitor app-insights web-test list \ --resource-group rg-webtest-demo ``` ```bash title="Output" [ { "enabled": true, "frequency": 300, "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-webtest-demo/providers/Microsoft.Insights/webtests/my-web-test", "kind": "standard", "location": "westeurope", "locations": [ { "Id": "us-tx-sn1-azr" } ], "name": "my-web-test", "request": { "httpVerb": "GET", "requestUrl": "https://example.com" }, "timeout": 30, "type": "Microsoft.Insights/webtests", "webTestKind": "standard", "webTestName": "My Web Test" } ] ``` ### Delete a web test Delete the web test and verify it no longer appears in the list: ```bash az monitor app-insights web-test delete \ --name my-web-test \ --resource-group rg-webtest-demo \ --yes ``` Then list all web tests to confirm the resource group is now empty: ```bash az monitor app-insights web-test list --resource-group rg-webtest-demo ``` ```bash title="Output" [] ``` ## Features - **Web test lifecycle:** Create, read, list, and delete web test resources. - **Classic ping and standard test kinds:** Accept `ping`, `multistep`, and `standard` test kinds. - **Test location configuration:** Accept one or more agent location IDs per test. - **Request configuration:** Define URL, HTTP verb, headers, and body for standard tests. - **Frequency and timeout settings:** Configure probing frequency and response timeout. - **Application Insights linking:** Associate web tests with an Application Insights component via a `hidden-link` tag on the web test resource (see [Web Tests REST API examples](https://learn.microsoft.com/en-us/rest/api/application-insights/web-tests/create-or-update?view=rest-application-insights-2022-06-15)). - **Enable/disable flag:** Enable or disable a web test without deleting it. ## Limitations - **No HTTP probes sent:** LocalStack does not send HTTP requests to the configured URL. - **No availability data collected:** Pass, fail, and response time data is not recorded. - **No availability alerts fired:** Alert rules associated with a web test are not triggered. - **No synthetic transactions:** Multi-step web tests (`multistep` kind), which rely on a recorded XML web test sequence, are not executed; only metadata is emulated locally. ## Samples Explore end-to-end examples in the [LocalStack for Azure Samples](https://github.com/localstack/localstack-azure-samples) repository. ## API Coverage # Workbook > Get started with Azure Monitor Workbooks on LocalStack import AzureFeatureCoverage from "../../../../components/feature-coverage/AzureFeatureCoverage"; ## Introduction Azure Monitor Workbooks are interactive reports that combine text, KQL queries, metrics, and visualizations into a single shareable document. Workbook templates are Azure Resource Manager resources that package reusable definitions and gallery metadata so teams can publish templates to experiences such as Azure Monitor workbooks galleries. They are commonly used to create operational dashboards, cost reports, and compliance summaries that surface data from multiple Azure Monitor sources. For more information, see [Azure Workbooks overview](https://learn.microsoft.com/en-us/azure/azure-monitor/visualize/workbooks-overview). LocalStack for Azure provides a local environment for building and testing applications that make use of Azure Monitor Workbooks. The supported APIs are available on our [API Coverage section](#api-coverage), which provides information on the extent of Workbooks' integration with LocalStack. ## Getting started This guide walks you through creating a workbook and a workbook template. Launch LocalStack using your preferred method. For more information, see [Introduction to LocalStack for Azure](/azure/getting-started/). Once the container is running, enable Azure CLI interception by running: ```bash lstk az start-interception ``` This command points the `az` CLI away from the public Azure management REST API and toward the LocalStack for Azure emulator API. To revert this configuration, run: ```bash lstk az stop-interception ``` This reconfigures the `az` CLI to send commands to the official Azure management REST API. ### Create a resource group Create a resource group to hold all resources created in this guide: ```bash az group create --name rg-workbook-demo --location westeurope ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-workbook-demo", "location": "westeurope", "name": "rg-workbook-demo", "properties": { "provisioningState": "Succeeded" }, "type": "Microsoft.Resources/resourceGroups" } ``` ### Create a workbook Generate a UUID for the workbook name, then create a shared workbook: ```bash WORKBOOK_ID=$(python3 -c "import uuid; print(uuid.uuid4())") az monitor app-insights workbook create \ --name "$WORKBOOK_ID" \ --resource-group rg-workbook-demo \ --display-name "My Workbook" \ --kind shared \ --category workbook \ --version "Notebook/1.0" \ --serialized-data '{"version":"Notebook/1.0","items":[{"type":1,"content":{"json":"## My Workbook\nHello, world!"},"name":"text-0"}],"isLocked":false}' \ --location westeurope ``` ```bash title="Output" { "category": "workbook", "displayName": "My Workbook", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/rg-workbook-demo/providers/Microsoft.Insights/workbooks/de747180-ecea-4b97-bef4-f8376fc72abe", "kind": "shared", "location": "westeurope", "name": "de747180-ecea-4b97-bef4-f8376fc72abe", "serializedData": "{\"version\":\"Notebook/1.0\",\"items\":[{\"type\":1,\"content\":{\"json\":\"## My Workbook\\nHello, world!\"},\"name\":\"text-0\"}],\"isLocked\":false}", "type": "Microsoft.Insights/workbooks", "version": "Notebook/1.0" } ``` ### List workbooks List all workbooks in the resource group filtered by category: ```bash az monitor app-insights workbook list \ --resource-group rg-workbook-demo \ --category workbook ``` ```bash title="Output" [ { "category": "workbook", "displayName": "My Workbook", "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourcegroups/rg-workbook-demo/providers/Microsoft.Insights/workbooks/de747180-ecea-4b97-bef4-f8376fc72abe", "kind": "shared", "location": "westeurope", "name": "de747180-ecea-4b97-bef4-f8376fc72abe", "serializedData": "...", "type": "Microsoft.Insights/workbooks", "version": "Notebook/1.0" } ] ``` ### Create a workbook template Workbook templates are not exposed as a first-party `az monitor app-insights` command group. Use `az rest` against the resource manager endpoint (via your active cloud, which LocalStack updates when interception is enabled): ```bash RM=$(az cloud show --query "endpoints.resourceManager" -o tsv | sed 's|/$||') az rest --method PUT \ --url "${RM}/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-workbook-demo/providers/microsoft.insights/workbookTemplates/my-template?api-version=2020-11-20" \ --body '{ "location": "westeurope", "properties": { "priority": 1, "author": "My Team", "templateData": { "version": "Notebook/1.0", "items": [] }, "galleries": [ { "name": "My Template", "category": "General", "type": "workbook", "order": 100, "resourceType": "Azure Monitor" } ] } }' ``` ```bash title="Output" { "id": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-workbook-demo/providers/Microsoft.Insights/workbookTemplates/my-template", "location": "westeurope", "name": "my-template", "properties": { "author": "My Team", "galleries": [ { "category": "General", "name": "My Template", "order": 100, "resourceType": "Azure Monitor", "type": "workbook" } ], "priority": 1, "templateData": { "version": "Notebook/1.0", "items": [] } }, "type": "Microsoft.Insights/workbookTemplates" } ``` ### Delete resources Delete the workbook and workbook template, then delete the resource group. Use the same shell session as earlier steps so `WORKBOOK_ID` and `RM` are still set. ```bash az monitor app-insights workbook delete \ --name "$WORKBOOK_ID" \ --resource-group rg-workbook-demo \ --yes az rest --method DELETE \ --url "${RM}/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/rg-workbook-demo/providers/microsoft.insights/workbookTemplates/my-template?api-version=2020-11-20" az group delete --name rg-workbook-demo --yes ``` ## Features - **Workbook lifecycle:** Create, read, list, update, and delete workbooks. - **Workbook template lifecycle:** Create, read, list, update, and delete workbook templates. - **Shared and private workbooks:** The [Workbooks REST API](https://learn.microsoft.com/en-us/rest/api/application-insights/workbooks/create-or-update) defines `kind` values `shared` and `user`. The Azure CLI `workbook create` command currently exposes only `--kind shared` (see [az monitor app-insights workbook create](https://learn.microsoft.com/en-us/cli/azure/monitor/app-insights/workbook#az-monitor-app-insights-workbook-create)). - **Category filtering:** For [list by resource group](https://learn.microsoft.com/en-us/rest/api/application-insights/workbooks/list-by-resource-group), the `category` query parameter is one of `workbook`, `TSG`, `performance`, or `retention` (the same set the CLI accepts on `workbook list --category`). The `properties.category` value stored on a workbook resource is a separate string that you set at creation time. - **Serialized data storage:** Store the full workbook JSON definition in the `serializedData` field. - **Gallery configuration:** Define workbook templates with gallery metadata for easy discovery. ## Limitations - **No rendering or query execution:** The workbook content is stored as a JSON string but KQL queries within the workbook are not executed. - **No Azure Portal integration:** Workbooks cannot be previewed or rendered via LocalStack. - **No linked resource validation:** Source ID references to Application Insights components or subscriptions are accepted but not validated. ## Samples Explore end-to-end examples in the [LocalStack for Azure Samples](https://github.com/localstack/localstack-azure-samples) repository. ## API Coverage # Welcome to LocalStack for Snowflake Docs > LocalStack for Snowflake allows you to develop and test your Snowflake data pipelines entirely on your local machine! import { OverviewCards, HeroCards } from '../../../components/OverviewCards'; // Import SVGs from assets folder import rocketIcon from '../../../assets/images/GettingStarted_Color.svg'; import buildingsIcon from '../../../assets/images/Enterprise_Color.svg'; import wrenchIcon from '../../../assets/images/Tooling_color.svg'; import connectionsIcon from '../../../assets/images/Integrations_Color.svg'; import cubeIcon from '../../../assets/images/LSAWS_Color.svg'; import starburstIcon from '../../../assets/images/Capabilities_Color.svg'; ## What would you like to do today? # Overview > Advanced capabilities and features available in LocalStack for Snowflake. import SectionCards from '../../../../components/SectionCards.astro'; LocalStack for Snowflake provides advanced capabilities that enhance your development workflow and enable sophisticated testing scenarios beyond basic Snowflake service emulation. # Configuration > Overview of configuration options in LocalStack for Snowflake. LocalStack exposes various configuration options to control its behaviour. With `lstk`, these options can be passed as `LOCALSTACK_`-prefixed environment variables when starting the container: ```bash LOCALSTACK_DEBUG=1 lstk start ``` Alternatively, set them as named environment profiles in your config file and reference them from the container block: ```toml # .lstk/config.toml [[containers]] type = "snowflake" env = ["debug"] [env.debug] DEBUG = "1" ``` ```bash lstk start ``` See [Passing environment variables to the container](/aws/developer-tools/running-localstack/lstk/configuration/#passing-environment-variables-to-the-container) for details. ## Core Options that affect the core Snowflake emulator functionality. | Variable | Example Values | Description | |----------|----------------------|-------------------------------------------------------------------------------------------------------------| | `DEBUG` | `0` (default) \| `1` | Flag to increase log level and print more verbose logs (useful for troubleshooting issues) | | `SF_LOG` | `trace` | Specify the log level. Currently overrides the `DEBUG` configuration. `trace` for detailed request/response | | `SF_S3_ENDPOINT` | `s3.localhost.localstack.cloud:4566` (default) | Specify the S3 endpoint to use for the Snowflake emulator. | | `SF_S3_ENDPOINT_EXTERNAL` | `s3.localhost.localstack.cloud:4566` | S3 endpoint for file uploads to return to external clients. Defaults to `SF_S3_ENDPOINT` if not set. | | `SF_AWS_ENDPOINT_URL` | `localhost:4566` (default) | AWS services endpoint for connecting to other AWS services (SQS, SNS, etc.) from the Snowflake emulator. | | `DNS_NAME_PATTERNS_TO_RESOLVE_UPSTREAM` | `*.s3.amazonaws.com` (example) | List of domain names that should NOT be resolved to the LocalStack container, but instead always forwarded to the upstream resolver (S3 for example). this would be required when importing data into a stage from an external S3 bucket on the real AWS cloud. Comma-separated list of Python-flavored regex patterns. | | `SF_HOSTNAMES` | `snowflake.localhost.localstack.cloud,snowflake.internal` | Comma-separated list of hostnames that should route to the Snowflake emulator. If set, only these hostnames are matched. If unset, LocalStack also matches any hostname with a `snowflake.` subdomain (i.e., `*snowflake.*` or `*.snowflake.*`) for backward compatibility. | `SF_CSV_IMPORT_MAX_ROWS` | `50000` (default) | Maximum number of rows to import from CSV files into tables | | `SF_DEFAULT_USER` | `test` (default) | Specify the default user to be used by the Snowflake emulator. | | `SF_DEFAULT_PASSWORD` | `test` (default) | Specify the default password to be used by the Snowflake emulator. | ### Custom Snowflake hostnames By default, the Snowflake emulator accepts requests for hostnames such as `snowflake.localhost.localstack.cloud` and other `*.snowflake.*` hostnames. If you expose the emulator through a custom DNS name, for example in Kubernetes or behind an ingress, set `SF_HOSTNAMES` to the exact hostnames clients use to reach the emulator. When you use `lstk`, set this as a named environment profile in your config file: ```toml # .lstk/config.toml [[containers]] type = "snowflake" env = ["custom"] [env.custom] SF_HOSTNAMES = "snowflake.internal.example.com,snowflake.internal,snowflake.localhost.localstack.cloud" ``` ```bash lstk start ``` The first hostname in `SF_HOSTNAMES` is used as the primary hostname for local connection defaults and generated URLs. When `SF_HOSTNAMES` is set, the default wildcard fallback is disabled, and only the configured hostnames are routed to the Snowflake emulator. Include `snowflake.localhost.localstack.cloud` in the list, as shown above, if you want the default hostname to continue working. `SF_HOSTNAMES` controls Host-header routing only. It does not configure DNS or TLS for custom hostnames. Configure each hostname to resolve to the LocalStack host from every client that connects to the emulator. For example, add the following entries to the client's `/etc/hosts` file when LocalStack runs on the same machine: ```text title="/etc/hosts" 127.0.0.1 snowflake.internal.example.com 127.0.0.1 snowflake.internal ``` The default LocalStack certificate does not match custom domains. Configure a matching custom TLS certificate before connecting through a custom hostname. ::::caution Do not use `insecure_mode=True` in the Snowflake Connector for Python to work around a certificate hostname mismatch. This deprecated option disables certificate revocation checks, but the connector still verifies the certificate and hostname. :::: ::::note `SF_HOSTNAME_REGEX` is no longer supported. If you previously used `SF_HOSTNAME_REGEX`, migrate to `SF_HOSTNAMES` and list each hostname explicitly. :::: If your custom hostname also needs a matching TLS certificate, use LocalStack's standard certificate configuration options: ```toml # .lstk/config.toml [[containers]] type = "snowflake" env = ["custom"] [env.custom] SF_HOSTNAMES = "snowflake.internal.example.com" CUSTOM_SSL_CERT_PATH = "/var/lib/localstack/custom/cert.pem" SKIP_SSL_CERT_DOWNLOAD = "1" ``` ```bash lstk start ``` The file referenced by `CUSTOM_SSL_CERT_PATH` must contain a certificate and private key that match the hostname used by your Snowflake clients. For more general guidance on adding trusted certificates to LocalStack, see [Custom TLS certificates](/aws/developer-tools/security-testing/custom-tls-certificates/). ## CLI `lstk` is configured through its config file rather than through environment variables. See [Configuration](/aws/developer-tools/running-localstack/lstk/configuration/) on the `lstk` page for the config file search order, the field reference, and how to define named environment profiles. ## Docker Options to configure how LocalStack interacts with Docker. | Variable | Example Values | Description | | - | - | - | | `LOCALSTACK_VOLUME_DIR` | `~/.cache/localstack/volume` (on Linux) | The location on the host of the LocalStack volume directory mount. | | `DOCKER_FLAGS` | | Allows to pass custom flags (e.g., volume mounts) to "docker run" when running LocalStack in Docker. | | `DOCKER_SOCK` | `/var/run/docker.sock` | Path to local Docker UNIX domain socket | | `DOCKER_BRIDGE_IP` | `172.17.0.1` | IP of the Docker bridge used to enable access between containers | | `LEGACY_DOCKER_CLIENT` | `0`\|`1` | Whether LocalStack should use the command-line Docker client and subprocess execution to run Docker commands, rather than the Docker SDK. | | `DOCKER_CMD` | `docker` (default), `sudo docker`| Shell command used to run Docker containers (only used in combination with `LEGACY_DOCKER_CLIENT`) | | `FORCE_NONINTERACTIVE` | | When running with Docker, disables the `--interactive` and `--tty` flags. Useful when running headless. | # Initialization Hooks > Writing SQL scripts to initialize your Snowflake emulator import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction LocalStack for Snowflake supports automatically executing `*.sf.sql` files via Init Hooks when mounted into the Docker container. A script can be added to one of these stages in the lifecycle: - `BOOT`: the container is running, but LocalStack hasn’t started - `START`: the Python process is running, and LocalStack is starting - `READY`: LocalStack is ready for requests - `SHUTDOWN`: LocalStack is shutting down A script can be in one of four states: `UNKNOWN`, `RUNNING`, `SUCCESSFUL`, or `ERROR`. By default, scripts are in the `UNKNOWN` state when first discovered. ## Getting started To begin, create a script called `test.sf.sql` with the following SQL statements: ```sql CREATE DATABASE foobar123; CREATE DATABASE test123; SHOW DATABASES; ``` Mount the script into `/etc/localstack/init/ready.d/` using Docker Compose or `lstk`: ```yaml showLineNumbers version: "3.8" services: localstack: container_name: "${LOCALSTACK_DOCKER_NAME:-localstack-main}" image: localstack/snowflake ports: - "127.0.0.1:4566:4566" - "127.0.0.1:4510-4559:4510-4559" - "127.0.0.1:443:443" environment: - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} - DEBUG=1 volumes: - "/path/to/test.sf.sql:/etc/localstack/init/ready.d/test.sf.sql" # ready hook - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" - "/var/run/docker.sock:/var/run/docker.sock" ``` Declare the bind mount and the `DEBUG` profile in your config file, then start LocalStack: ```toml # .lstk/config.toml [[containers]] type = "snowflake" env = ["debug"] volumes = ["/path/to/test.sf.sql:/etc/localstack/init/ready.d/test.sf.sql"] [env.debug] DEBUG = "1" # Optionally enable DEBUG, not required for init hooks but helps diagnose init hook issues ``` ```bash lstk start ``` Start the Snowflake emulator, and the following logs will appear: ```bash DEBUG --- [et.reactor-0] s.analytics.handler : REQ: POST /queries/v1/query-request {"sqlText": "CREATE DATABASE foobar123", ... DEBUG --- [et.reactor-0] s.analytics.handler : REQ: POST /queries/v1/query-request {"sqlText": "CREATE DATABASE test123", ... DEBUG --- [et.reactor-0] s.analytics.handler : REQ: POST /queries/v1/query-request {"sqlText": "SHOW DATABASES", ... ``` # State Management > Get started with State Management in LocalStack for Snowflake import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction State Management in LocalStack allows you to save and load the state of your LocalStack instance. LocalStack is ephemeral in nature, so when you stop and restart your LocalStack instance, all the data is lost. With State Management, you can save the state of your LocalStack instance and load it back when you restart your LocalStack instance. State Management in LocalStack encompasses the following features: - **Export & Import State**: Export and import the state of your LocalStack instance on your local machine as a local file. - **Persistence**: Persist the state of your LocalStack instance on your local machine using a configuration variable. State Management is an essential feature that supports various use-cases, such as pre-seeding your fresh LocalStack instance with data, sharing your LocalStack instance’s state with your team, fostering collaboration, and more. ## Persistence LocalStack’s Persistence mechanism enables the saving and restoration of the entire LocalStack state. It functions as a **pause and resume** feature, allowing you to take a snapshot of your LocalStack instance and save this data to disk. This mechanism ensures a quick and efficient way to preserve and continue your work with Snowflake resources locally. To start snapshot-based persistence, launch LocalStack with the `--persist` command-line option, or the configuration option `PERSISTENCE=1`. This setting instructs LocalStack to save all local Snowflake resources and their respective application states into the LocalStack Volume Directory. Upon restarting LocalStack, you'll be able to resume your activities exactly where you left off. ```bash export LOCALSTACK_AUTH_TOKEN= lstk start --persist ``` To resume a persisted session after the emulator has stopped, or to restart a session while keeping persistence on, pass `--persist` again: ```bash lstk stop lstk start --persist ``` ```bash lstk restart --persist ``` ```yaml showLineNumbers ... image: localstack/snowflake environment: - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} - PERSISTENCE=1 volumes: - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack" ``` ```bash showLineNumbers docker run \ -e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} \ -e PERSISTENCE=1 \ -v ./volume:/var/lib/localstack \ -p 4566:4566 \ localstack/snowflake ``` :::note Snapshots may not be compatible across different versions of LocalStack. It is possible that snapshots from older versions can be restored, but there are no guarantees as to whether LocalStack will start into a consistent state. We are actively working on a solution for this problem. ::: ## Export/Import State The Export/Import State feature enables you to export the state of your LocalStack instance into a file, and then later import it into another LocalStack instance. This feature is useful when you want to save your LocalStack instance’s state for later use. ### Export the State To export the state, you can run the following command: ```bash lstk snapshot save '' ``` You can use the `` argument to specify a file path to export the state to. If you do not specify a file path, the state will be exported to the current working directory into an auto-named snapshot file. See the [lstk snapshot documentation](/aws/developer-tools/running-localstack/lstk/snapshots/) for the full list of supported save destinations, including local files and remote Cloud Pods. ### Import the State To import the state, you can run the following command: ```bash lstk snapshot load '' ``` The `` argument is required and specifies the file path to import the state from. The file should be generated from a previous export. # Changelog > Changelog for the latest releases of the LocalStack for Snowflake. The LocalStack for Snowflake changelog tracks updates to LocalStack’s Snowflake support, including new features, enhancements, and compatibility fixes. Stay up to date on changes across official versioned releases of LocalStack’s Snowflake support. :::note Starting with the end-of-March 2026 release, LocalStack for Snowflake follows [calendar versioning](https://calver.org/) in the `YYYY.MM.patch` format. For example, `2026.03.0` is the initial March 2026 release. ::: ## 2026.08.0 - Add Snowflake-Next preview, a new-generation emulator planned to become the default soon. In internal testing it ran the regression suite more than 10 times faster than the current emulator. Enable it to try `STREAM` (including `STREAM ON VIEW`), zero-copy `CLONE`, and Time Travel. - `RESULT_SCAN` now processes results from `SHOW`, `DESC`, `LIST`, and other status-returning commands, handles empty `SHOW` results, and returns Snowflake-style errors for failed or in-progress queries instead of unrelated syntax errors. - `LISTAGG` and `ARRAY_AGG` support multiple ordered aggregates with separate `WITHIN GROUP` clauses in a single `SELECT` statement, each using independent ordering. - Long-running queries now remain in progress until execution finishes, instead of returning early. - `SHOW IN SCHEMA` now limits results to the requested schema for views, materialized views, sequences, and file formats. - `SHOW VIEWS` identifies materialized views via the `is_materialized` field; `SHOW MATERIALIZED VIEWS` returns the view definition in the `text` field. - `ALTER PIPE`, `DESC PIPE`, and `DROP PIPE` now resolve fully qualified names independent of the session's current schema. - `DROP PIPE IF EXISTS` no longer errors when the pipe doesn't exist. - `CREATE TAG IF NOT EXISTS` is now idempotent. - Dropping a database or schema now removes only the tags it owns, preserving tags in other namespaces. - Migrate the Docker image base from Debian sid (rolling) to Debian trixie (stable release). ## 2026.07.0 - Tables and data stored with `PERSISTENCE=1` remain available after the emulator restarts. - DDL statements now commit an open transaction before they run, matching Snowflake and making newly created objects visible to other sessions. - `MAX(GREATEST(...))` works when source columns share names with aggregate aliases. - `UNPIVOT` preserves passthrough columns when its input comes from a chain of `SELECT *` CTEs. - Boolean comparisons inside `COALESCE` work when the result is used with `NOT`, `AND`, or `WHERE`. - `QUALIFY` branches can be combined with `UNION ALL` without leaking internal columns into the result. - `SHOW USER FUNCTIONS` respects schema scope and no longer exposes internal emulator functions. - Scripting keywords such as `loop`, `while`, and `cursor` can qualify columns when used as table aliases. - JavaScript stored procedures can call `ResultSet.getColumnValue()` with either a column name or a column index. ## 2026.06.0 - 'Added support for `PERCENTILE_CONT` and `PERCENTILE_DISC`, the standard percentile aggregate functions used for medians, quartiles, and other distribution cutoffs.' - '`MEDIAN` is now fixed to return correct results, since it shares its underlying implementation with `PERCENTILE_CONT`.' - '`UNPIVOT` no longer crashes with an unhandled `IndexError` in certain query shapes.' - '`GREATEST` and `LEAST` now preserve the input type, and generic `MIN`/`MAX` resolution was fixed alongside them.' - '`ROUND` precision derivation now accommodates an increased scale, fixing incorrect results for some decimal roundings.' - 'Fixed handling of certain boolean comparison expressions that previously produced incorrect results.' - '`CONVERT_TIMESTAMP` now handles `NULL` inputs correctly instead of failing.' - '`DATEDIFF` now handles an unquoted date part argument used inside a window function ORDER BY.' - 'Fixed a bug affecting character/string conversion.' - '`QUALIFY` column resolution is fixed for queries that cross-join a CTE and reference a column via its CTE alias.' - 'Table qualifiers added during alias expansion are now quoted correctly, fixing visibility issues with `UNION ALL` over a CTE combined with `QUALIFY`.' - '`IS NULL` now evaluates correctly against a bare `NULL` value projected through a CTE or subquery.' - 'Unaliased derived tables in a `JOIN` clause will no longer cause the query to fail.' - 'Nested `QUALIFY` rewrites no longer create an ambiguous internal column.' - 'Parenthesized queries of the form `(WITH ... SELECT ...)` are no longer incorrectly rejected.' - '`WITH RECURSIVE` will now enable recursive CTEs to be properly recognized and executed as recursive.' - 'Scripting keywords used as implicit table aliases are no longer misinterpreted during table reference parsing.' - 'SQL procedure bodies using `DECLARE` blocks, including `OBJECT` and `NUMBER`-typed variables and bodies containing `UPDATE ... RETURNING`, no longer fail to create.' - '`CREATE TASK` now supports multi-statement `BEGIN ... END` bodies.' - '`DROP STAGE IF EXISTS` no longer raises an error when the referenced stage does not exist.' - 'Bumped `jackson-databind` to `2.18.8`, addressing CVE-2026-54512 and CVE-2026-54513.' - 'Bumped `dulwich` and `kclpy-ext` to patched versions, addressing three High-severity CVEs.' - 'Bumped the bundled Apache NiFi (OpenFlow) component to version 2.10.0.' ## 2026.05.0 - Update Docker image tagging policy to introduce `dev` and `nightly` tags, while aligning `latest` to mirror `stable` - Add support for `TRY_CAST( AS ARRAY)` conversions - Support `ARRAY_AGG(DISTINCT ...)` execution with variant path expressions - Support preservation of typed `NUMBER(p,s)` values in `MIN` and `MAX` aggregates - Improve `COUNT(DISTINCT ...)` handling with nested `CASE` predicates and literals - Improve window functions (`LAG`, `LEAD`) to preserve qualified column references in `ORDER BY` - Improve pattern matching (`LIKE`, `ILIKE`) on values derived from `LATERAL FLATTEN` loops - Improve `QUALIFY` query handling combining `SELECT *` with aliased expressions in CTEs - Improve implicit type coercion for aliased `NULL` and `VARCHAR` values in `UNION` / `UNION ALL` queries - Improve translation and execution for SQL UDFs and stored procedures starting with CTEs, set operations, or blocks - Improve Snowpipe concurrency and message polling responsiveness by scheduling `COPY INTO` work per pipe - Fix statement splitting for multi-statement queries containing inline, line, or block SQL comments - Fix error shapes for invalid SQL syntax and compilation failures to return Snowflake-compatible codes instead of raw backend errors ## 2026.04.0 - Add support for AWS Glue Iceberg REST catalog integrations via `CREATE CATALOG INTEGRATION` - Add support for new SQL date/time functions (`DAY`, `DAYOFMONTH`, `DAYOFYEAR`, `QUARTER`, `WEEK`, `YEAROFWEEK`) - Support six-argument form for `TIMESTAMP_LTZ_FROM_PARTS` - Support unquoted date/time parts in `DATE_PART` and `EXTRACT` functions - Add support for `INFORMATION_SCHEMA.PROCEDURES` view via metadata FDW mechanism - Add `ROW_COUNT` and `BYTES` metadata to `INFORMATION_SCHEMA.TABLES` and `SHOW OBJECTS` - Enhance `SHOW OBJECTS` to support `is_interactive` columns and scoped filters (`IN`, `STARTS WITH`, `LIMIT`) - Improve routine tracking for UDFs and stored procedures, including overloaded routines and `IDENTIFIER($var)` - Improve Snowpipe notification processing and performance for shared S3 buckets - Enhance Web App with multi-statement query execution and side-by-side layout for Query History and results - Enhance Web App with tabbed query results, truncated result banners, and improved SQL autocomplete - Enhance Web App Resource Browser with dedicated views and icons for databases, schemas, tables, and views ## 2026.03.0 - Add support for `GET_DDL` for tables, views, and databases - Add support for `FULL JOIN` and complex join conditions - Add support for looping constructs (`WHILE`, `LOOP`, `FOR`, `REPEAT`) in Snowflake scripting - Add support for custom `EXCEPTION` handling, `RAISE`, and `SQLERRM/SQLSTATE/SQLCODE` - Add support for 50+ functions including `HLL`, `APPROX_TOP_K`, `REGEXP_SUBSTR_ALL`, and `JAROWINKLER_SIMILARITY` - Add support for encryption (`ENCRYPT_RAW`) and compression (`COMPRESS`) functions - Add support for stage metadata and URL functions (`BUILD_STAGE_FILE_URL`, `FL_GET_SIZE`, etc.) - Add support for `SHOW/UNSET VARIABLES` and session variable management - Add initial support for `CREATE DYNAMIC ICEBERG TABLE` - Support `CTAS` with inline column constraints (`NOT NULL`, `PRIMARY KEY`) - Support nested `DECLARE` blocks and `IDENTIFIER()` references in stored procedures - Support `INSERT INTO ... (SELECT ...)` with parenthesized subqueries - Enhance Web App with Monaco SQL editor, Query History, and hierarchical Resource Tree - Enhance `CALL` parsing and `EXECUTE AS OWNER/CALLER` parity - Enhance performance for `DATASKETCHES_HLL` on large datasets - Improve Snowpipe latency for large JSON uploads - Fix dropping of dependent views when dropping tables ## 1.7.0 - Add support for `OBJECT_AGG`, `QUERY_HISTORY_BY_USER`, `BOOLAND_AGG`, `BOOLOR_AGG`, `APPROX_PERCENTILE`, `LIKE`, `ILIKE`, and math functions - Add support for date arithmetics - Add support for `INFORMATION_SCHEMA.COLUMNS`, `INFORMATION_SCHEMA.SCHEMATA`, and `INFORMATION_SCHEMA.DATABASES` views - Add support for anonymous code blocks - Add support for if-else clauses in Snowflake scripting - Support `ALTER DYNAMIC TABLE` and dynamic tables enhancements - Support multiple variable assignment and `IDENTIFIER()` with session variables - Support distinct clause in window functions - Enhance `TERSE` support for `SHOW` commands - Enhance `IDENTIFIER()` support for `CREATE/DROP SCHEMA` with fully-qualified names - Enhance `INFORMATION_SCHEMA` handling for same-name passthrough and identifier resolution - Enhance handling of internal identifiers - Improve routing of table functions to internal `INFORMATION_SCHEMA` - Raise error for non-existent `INFORMATION_SCHEMA` views - Fix null handling in timestamp/date/time implicit casts - Fix `PIVOT` with CTEs - Fix `SYSTEM$BOOTSTRAP_DATA_REQUEST` to support varargs - Fix evaluating values for describe-only queries ## 1.6.0 - Add support for SQL `REGEXP_LIKE`, `SPLIT_PART`, `STRIP_NULL_VALUE`, `TRY_PARSE_JSON` function - Integrate Snowflake with S3 tables - Improve dynamic table query handling and error reporting - Support access delegation mode in Polaris catalog - Fix `DATEADD` function to support column references as input arguments - Fix casting of types for interval arithmetics - Enhance parity for `TO_DATE` with large datetimes - Bump Apache NiFi version to 2.7.1 - Enable cross-database queries in SELECT statements - Fix schema-qualified identifier handling and standardize canonical name usage - Verify Accept header for `GET/PUT` statements - Add support for rows column in `SHOW OBJECTS` - Enhance parity for location and other response fields in `PUT` uploads - Enhance support for large numbers and auto-casting of `DECIMAL/DOUBLE` with mixed `VALUES` - Add update stats to response of DML operations - Fixes schema resolution for `CREATE STREAM` statements ## 1.5.0 - Add support for `POSITION`, `CHARINDEX`, `LEFT`, `RIGHT`, `LENGTH/LEN` SQL function - Fix long-running queries by continuing to run them asynchronously - Enhance parity for `SELECT AVG OVER ROWS` aggregate queries - Enhance parity and metadata for large timestamps - Add logic to disable SF client telemetry for the emulator - Enhance parity for GET expressions with quoted file refs - Preserve case for quoted identifiers in SELECT results - Enhance parity to GET a single stage file directly - Support Iceberg CTAS (`CREATE ICEBERG TABLE AS SELECT`) ## 1.4.0 - Add support for `SELECT * EXCLUDE (...)` - Add support for `PI` function - Enhance parity for query binding values and numeric timestamps with TZ - Support `UPDATE ... FROM ... SET ...` syntax - Add support for query bindings from stage files - Add support for switching JSON/Arrow result encoding in session parameters - Enable returning of result batches for large query results - Fix VARCHAR type/modifier when inferred from SELECT statements - Return exact column type/modifiers for `DESCRIBE VIEW` results - Fix default type modifiers when inferred from SELECT statements - Add logic to clean up temporary stages on session termination - Raise `DuplicateMergeKey` on `MERGE` duplicates - Fix duplicate columns in `DESCRIBE TABLE` queries - Add support for inline foreign key syntax - Properly handle error responses for async queries - Fix `GROUP BY` positional references being incorrectly wrapped ## 1.3.0 - Retry failing db commands and enhance handling of transactions - Upgrade to Python 3.13 - Expand Alias References - Support python-only queries in snowflake scripting - Remove byte order mark (BOM) in query string to support SnowConvert tool - Add resultSetMetaData for v2 API responses - Fix multiline `CREATE TASK` - Patch `SHOW OBJECTS` with the new columns - Add patch to fix parsing CREATE TABLE within BEGIN blocks - Enhance `LAG`/`LEAD` functions - Fix variant usage in `FLATTEN` - Enhance support for procedures with INSERT statements - Enhance parity for `SHOW VIEWS` to show view definitions - Enhance scope and object-type support for `SHOW OBJECTS` queries - Enhance parity for `SHOW VIEWS` queries - Add `TIMELIMIT` parameter to `GENERATOR` function - Enhance support for `GRANT` with `APPLICATION ROLEs` - Handle cross db references in `INFORMATION_SCHEMA.TABLES` view - Enhance parity of column types for `DESCRIBE VIEW` queries ## 1.2.0 - Add support for `SQUARE`, `FACTORIAL`, `UNIFORM`, `SYSTEM$ALLOWLIST`, `ARRAYS_ZIP`, `CURRENT_ORGANIZATION_USER`, `QUERY_HISTORY` functions - Enhance metadata for varchar type - Add support for `PIVOT` operation - Introduce `SF_S3_ENDPOINT_EXTERNAL` config - Add logic to add DQM records on `TRIGGER_ON_CHANGES` - Add implementation for `PUT /api/v2/databases/{name}` - Enhance parity for different flavors of `REVOKE` statements - Add support for quoted stage references - Fix drop tables with multi-level identifiers - Enhance parity for GRANT TO statements - Fix FULL JOINs between columns of different types - Properly delete stage files on table `REPLACE` - Add initial CRUD support for network rules - Fix column dependencies and `WHERE/ORDER BY` references in subqueries - Add initial CRUD support for API INTEGRATIONs - Enhance processing of UPDATE queries with table aliases - Add initial support for Openflow via Apache NiFi - Fix parsing timestamps with milliseconds - Fix CREATE OR REPLACE statements when schema name is specified in resource identifier - Enhance parity for database roles and grants - Add initial CRUD support for SECRETs - Support UDFs with handler code imported from stage file - Add initial CRUD support for resource monitors - Add initial CRUD support for masking policies - Support `SHOW REPLICATION ACCOUNTS` - Enhance parity for creating transient schemas - Enhance `CREATE MASKING POLICY` handling ## 1.1.0 - Add support for functions: `ROUND`, `SYSDATE`, `STARTSWITH`, `ENDSWITH`, `MD5`, `SUBSTR`, `HAVERSINE`, `LAST_DAY`, `TRY_CAST`, `TRUNCATE`, `PERCENT_RANK`, `SYSTEM$TASK_DEPENDENTS_ENABLE` - Implement `TRY_*` conversion functions - Add number formatting parameter to `TO_DOUBLE` and `TO_DECIMAL` - Add initial support for data metric functions (DMFs) - Add `SF_AWS_ENDPOINT_URL` config var to make AWS endpoint configurable - `TIME` and `TIMESTAMP` processing improvements - Add initial CRUD support for compute pools - Add support for `OR REPLACE` on `CREATE STREAM` statements - Add support for `IF NOT EXISTS` on `CREATE TASK` statements - Enhance GROUP BY over aggregate columns - Add initial CRUD support for security integrations - Handle if exists when dropping tags - Enhance account name handling - Add timezone, fractional seconds formatting to `TO_CHAR` - Enhance parity for sorting VARIANT-encoded values - Add initial support for streams on `CHANGE_TRACKING` tables - Enhance parity for subqueries with column aliases - Add support for dict bindings with named query parameters - Boost performance of multi-statement queries - Implement support for more database endpoints in REST API - Fix encoding of numeric NaN - Enhance storage integration statements - Added support for `IN` clause in `SHOW INDEXES` queries - Enhance parity for result encoding of async queries - Add support for using `on_error=continue` used in `COPY INTO` statements - Add `LIMIT` support for `SHOW DATABASES` - Enhance parity for type casts on results returned from `FLATTEN` - Enhance parsing of nested variable assignments ## 1.0.1 - Implementation and migration to the new type system - Enhance persistence support for storing/reloading native apps - Add support for cross-db CTAS and CVAS statements - Enhance support for `ALTER DATABASE` statements - Add initial support for `COLLATE` table columns - Add support for `DESCRIBE SCHEMA` - Enhance support for `ALTER SCHEMA` queries - Add support for suspending tasks - Add initial support for table change tracking - Implement `CONCAT` function - Miscellaneous fixes for Streamlit apps - Add support for secure functions - Add exception handler for REST API - Fix `TO_DATE` for JSON object attributes - Enhance parity for managing Users and Roles - Implement `LOWER` and `UPPER` functions - Fix `DROP TABLE` query - Add initial support for `GRANT OWNERSHIP` statements - Enhance CRUD support for `ROW ACCESS POLICIES` - Support temporary views and dropping temp objects at session end - Enahance parity for `CREATE` and `DROP ROLE` - Fix for multi-account DB initialization - Fix for `TO_DATE` conversion with date format string - Fix insertion of numeric values from staged CSV files - Fix return value of `TO_DECIMAL` for int values - Fix parity with SnowSQL for `SHOW ROLES` - Fix create user or role response - Add support for `ALTER SESSION UNSET` - Native app deployment fixes - Add support for `SHOW TELEMETRY EVENT DEFINITIONS IN APPLICATION` - Support `ALTER` and `DESCRIBE APPLICATION` - Support `DESCRIBE PROCEDURE` - Add support for function `SYSTEM$VALIDATE_NATIVE_APP_SETUP` - Add dispatcher request deserialization for REST API - Enhance parity on metadata results for copy into command - Fix for numeric bool values ## 1.0.0 - Add support for `SHOW/ALTER FUNCTION` - Fix incompatibilities with GO driver and SnowSQL client - Add support for `SHOW INDEXES` - Improve timestamp string support in `TO_TIMESTAMP` - Fix casting values to array - Cast `MERGE INTO/UPDATE` commands arguments to target type - Make `TO_BOOLEAN` work with all boolean strings - Enhance parity for parsing URLs with whitespaces in `PUT` commands - Enhance and add support for metadata columns in parquet format - Enhance CRUD support for external volumes - Add support for `EXECUTE TASK` - Fix identifier parsing in stages and file formats - Enhance `SHOW TABLES` feature parity - Handle `DATE` and `TIME` functions - Add initial Iceberg support - Enhance parity for GRANT statements and DB permissions - Add initial support for password-less auth using RSA key - Enhance parity for queries over staged JSON files - Add initial support for Catalog Integrations - Support prepared statements in ODBC driver - Enhance decimals parity - Add initial support for granting `APPLICATION ROLE` - Enhance parity for SQL procedures with SF-native statements - Add support for lateral column references on `SELECT` - Fix `SHOW TABLE` with schema scope for Flyway - Decode field delimiters passed as hex or octal values - Remove modifiers from binary columns - Fix permissions to clone default database - Add support for numeric paramstyle - Enhance support for `CASE` expressions in `BEGIN..END` blocks - Refreshed UI - Enhance parity for `DESCRIBE DATABASE` queries - Enable local deployment of Streamlit Native Apps - Enhance parity for materialized view queries - Enhance parity for `SHOW DYNAMIC TABLES` - Fix handling of `IF EXISTS` statements within transactions - Enhance support for `BEGIN` code blocks with multiple command statements - Enhance logic for native apps and permission grants - Enhance parity for Native Apps that contain streamlit apps - Add auto-conversion of strings to `ARRAY/OBJECT` types - Add support for Polaris catalog ## 0.3.0 - Add support for multi-account setups - Add initial support for Java UDFs - Add initial support for native apps and `APPLICATION PACKAGEs` - Add support for row access policies - Add support for AVRO file format - Add support for `ALTER STAGE` and `ALTER STREAM` statements - Enhance `SHOW COLUMNS/SCHEMAS` feature parity - Add support for `EQUAL_NULL` and variant type check functions - Add support for `ARRAY_REVERSE` function - Add support for `DESCRIBE/SHOW SEQUENCES` - Enhance timestamp handling and timezone support - Add support for metadata columns in JSON - Add support for `INSERT OVERWRITE` queries - Support ZTSD compression - Enhance `TO_CHAR` with date and number formatting - Support dropping multiple columns in single `ALTER` statement - Add support for TIMEZONE session parameter - Make default user and password configurable - Fix various timestamp and timezone-related issues - Improve query performance by re-using PG connections per session - Enhance parity for comparison of VARIANTs and JSON objects - Enhance column aliasing for joins - Fix type mapping for rowtype metadata to enhance compatibility with .NET clients - Cast `INSERT INTO` arguments to target type - Add support for `REGEXP_SUBSTR` function - Prohibit Postgres specific data types - Add `HEADER` and `FIELD_OPTIONALLY_ENCLOSED_BY` support for COPY INTO - Support subquery in copy into location - Support not equal operator - Enhance parity for $N references in WHERE clauses - Increase identifiers length to 255 char - Improve JDBC driver compatibility - Support LS alias to LIST files from stages ## 0.2.5 - Change storage integration to not be associated with DB/schema - Add enhanced support for SHOW ROLES and USE ROLE - Enhance parity for COUNT(..) with NULL values - Add multiple statements support - Fix staging issues with parsing - Enhance logic for timestamp assignments in MERGE INTO queries - Add proper passing of stage parameters and validations - Add support for `SYSTEM$CLIENT_VERSION_INFO`/`OBJECT_CONSTRUCT_KEEP_NULL`/`TIMESTAMPADD`/`DATEADD`/`TIMEADD` functions - Fix timestamp_ltz comparison operators issue - Fix INSERT result count type - Show tables metadata/information for dynamic tables - Include Snowflake emulator version in the logs - Update web app layout - Enhance parity for NUMBER/FIXED data types in JDBC results - Add support for querying from `INFORMATION_SCHEMA.TABLES` - Fix Arrow encoding for large decimal numeric values - Fix session for /api/v2/statements endpoint - Improve file formats handling in COPY INTO - Enhance parity for COPY INTO with multiple stage files - Update metadata for integer - Enhance parity around selecting NULL values for DATE columns - Add initial support for materialized views - Add format parameter for `TO_TIMESTAMP` function - Add support for async execution of multi-statement queries - Add support for patterns in select from stages - Allow table column projections for COPY INTO queries from stage files - Add support for numeric operators with mixed/variant types - Add initial support for `MAP` data type and util functions - Create `INFORMATION_SCHEMA.FUNCTIONS` table - Add support for querying metadata filename in select from stage statements - Enhance parity for primary/foreign keys - Add support for IF EXISTS clauses in ALTER COLUMN queries - Add initial support for user-defined transaction management ## 0.2.4 - Support `POWER`/`POW`/`DIV0NULL`/`IFNULL` functions - Add support for COPY INTO location - Add initial support for table/database clones - Establish parity with snowflake when csv imports - Allow binding multiple values - Fix database and schema names in copy into table - Fix `executeUpdate` in JDBC - Support `ORDER` and `NOORDER` from AUTOINCREMENT column def - Add initial logic and tests for replicating resources between accounts - Convert empty csv values to null - Add support and tests for `LATERAL` queries - Add initial support for EXECUTE IMMEDIATE - Add initial support for tags - Add identifier support to SELECT queries - Fix inserting timestamp values correctly - Fix timestamps in insert for current_timestamp - Add support for init scripts - Fix handling timestamp values on update - Add support for `DESCRIBE STAG` - Add initial support for storage integrations - Enhance parity of `TIMESTAMP_LTZ` - Create clone db using identifiers - Add mock support for replication databases to fix TF issues - Fix logic for setting session parameters - Add support for some extended GRANT statements - Support `ALTER SEQUENCE` - Support creating stages with storage integrations - Terraform create database fixes - Improve general error handling - Add initial support for `SHOW GRANTS TO/OF` ## 0.2.3 - Add initial support for `OBJECT_KEYS`/`PARSE_IP`/`DESCRIBE FUNCTION` functions - Add support for various functions including `DATE_TRUNC`, `NVL2`, `LEAST`, `GREATEST`, and more - Enhance parity for creation/deletion of schemas with fully qualified names - Enhance parity for inserting timestamps with subsecond precision - Enhance parity for CTAS with nested subqueries - Enhance parity for id placeholders in JDBC prepared statements - Enhance parity for metadata queries and schema lookup - Adjust `MIN_BY`/`MAX_BY` aggregate functions - Properly extract db/schema parameters for JDBC connections - Implement trigonometric and hyperbolic functions - Add support for GET stage files ## 0.2.2 - Add initial support for hybrid tables and dynamic tables - Add support for `OBJECT_CONSTRUCT_KEEP_NULL`/`AS_DOUBLE`/`AS_INTEGER`/`AS_NUMBER`/`AS_CHAR` - Add `/result` API endpoint to retrieve query results - Track original types in internal VARIANTs - Enhance parity for `SHOW WAREHOUSES` queries - Support for `SHOW TASKS` - Enhance parsing of stage params - Fix selection of columns when querying stage files - Automatically adjust PG JIT support if LLVM libs are missing - Enhance custom JSON parsing to allow escaped characters - Enhance parity of `TIMESTAMP_LTZ` for Flyway compatibility ## 0.2.1 - Add initial support for Iceberg tables - Add initial support for external volumes and Snowflake pipes - Support `LIST`/`REMOVE` queries for staged files - Support `SHOW PIPES` queries - Support `COPY GRANTS` in `CREATE TABLE` queries - Add multiple new SQL functions - Implement `RANK`/`DENSE_RANK` - Implement various conversion functions - Enhance logic for `TO_CHAR` - Support window queries with `QUALIFY` - Support `COUNT_IF` aggregate functions - Make `CREATE SERVER` queries idempotent - Fix compatibility issues - Add MUI data-grid for results table in UI - Add squashing of Docker image to reduce size ## 0.2.0 - Support various SQL functions (`BITAND`, `FLATTEN`, `RANDOM`, etc.) - Add Snowflake proxy request handler - Add initial version of simple UI view - Fix execution of CTAS queries with UNION selects - Fix logic for PUT file uploads - Support parsing incomplete JSON - Enhance support for TABLESAMPLE queries - Add initial support for temporary and transient tables - Add Snowflake v2 SQL APIs - Fix `describeOnly` `INSERT` queries ## 0.1.26 - Support `CONVERT_TIMEZONE`, `IFF` SQL functions - Implement `ALTER WAREHOUSE` as no-op - Implement time functions - Enhance JDBC driver compatibility - Fix Arrow encoding - Support SQL queries from within JS UDFs - Add various performance improvements - Implement additional date functions - Add support for loading data from public S3 buckets ## 0.1.25 - Enhance support for various SHOW queries - Add initial persistence support for Snowflake store - Enhance parity for timestamp types - Fix SHOW PARAMETERS for Terraform compatibility - Set up CI build for Docker image ## 0.1.24 - Enhance parity around user-defined PROCEDUREs - Fix upper-casing for various functions - Enhance support for UNIQUE column constraints - Add initial support for cross-DB resource sharing ## 0.1.23 - Add initial simple scheduler to periodically run tasks ## 0.1.22 - Fix query transforms for `ADD COLUMN` queries - Fix wrapping of `VALUES` subquery in braces - Add initial CRUD support for `TASK`s - Support `DROP PRIMARY KEY` queries - Migrate use of `localstack.http` to `rolo` ## 0.1.21 - Add support for consuming table stream records via DML statements ## 0.1.20 - Initial simple support for table streams - Add support for `SHOW DATABASES`, `SHOW VIEWS` - Enhance parity for Arrow results - Fix various identifier and query issues ## 0.1.19 - Return `SELECT` results in arrow format for pandas compatibility - Add `add_months` function - Fix UDFs with raw expressions - Upgrade to Postgres v15 - Various parity and performance improvements ## 0.1.18 - Add support for various array and aggregation functions - Enhance `FILE FORMAT` operations - Fix `CTAS` queries - Support `INFER_SCHEMA(..)` for parquet files - Improve identifier handling ## 0.1.17 - Support creation/deletion of stages - Add `IS_ARRAY` function - Remove `DuckDB` based `DB` engine - Refactor codebase to use `QueryProcessor` interface - Enhance column name handling ## 0.1.16 - Add support for `SHOW PROCEDURES` and `SHOW IMPORTED KEYS` - Add basic support for session parameters ## 0.1.15 - Fix result type conversation for `GET_PATH(..)` util function ## 0.1.14 - Enhance parity around `SHOW` queries - Add more array util functions - Fix `STRING_AGG` functionality ## 0.1.13 - Support `CURRENT_*` functions - Enhance `LISTAGG` for distinct values - Add test for `JS` UDFs with exports ## 0.1.12 - Cast params for `string_agg`/`listagg` - Fix parity for upper/lowercase names ## 0.1.11 - Enhance parity for array aggregation functions - Improve timestamp timezone handling - Add case-sensitive identifier tracking ## 0.1.10 - Add query transforms for `CLUSTER BY` - Add `SF_S3_ENDPOINT` config - Various parity fixes ## 0.1.9 - Add support for `Python` UDFs - Enhance function creation parity - Add analytics setup ## 0.1.8 - Add `SF_LOG` config for request/response trace logging ## 0.1.7 - Add initial support for `JavaScript` UDFs - Enhance DB/table creation responses - Improve streaming logic ## 0.1.6 - Introduce session state for DB/schema retention - Support async queries and `result_scan(..)` ## 0.1.5 - Enhance `DESCRIBE TABLE` results - Support `MIN_BY`/`MAX_BY` aggregate functions ## 0.1.4 - Add logic to parse and replace `DB` references in queries ## 0.1.3 - Add `DBEngine` abstraction - Add experimental support for `duckdb` - Enhance `JSON` query support ## 0.1.2 - Add CSV file ingestion from Snowflake stage to table ## 0.1.1 - Initial support for `Kafka` connector - Add `snowpipe`/streaming APIs ## 0.1.0 - Initial release of the extension # lstk CLI > Overview, installation, and quick start for lstk, the modern CLI for managing LocalStack. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction `lstk` is a high-performance command-line interface for LocalStack, built in Go. It provides a built-in terminal UI (TUI) for interactive use and plain text output for CI/CD pipelines and scripting. `lstk` handles the full emulator lifecycle: authentication, pulling the Docker image, starting, stopping, and restarting the container, streaming logs, and checking status. It can also save and load emulator state (as local snapshots or Cloud Pods) reset running state, run AWS CLI commands against the emulator, and manage the on-disk volume. Running `lstk` with no arguments takes you through the entire startup flow automatically. `lstk` also proxies developer tools so they run directly against LocalStack: the AWS CLI (`lstk aws`), the Azure CLI (`lstk az`), Terraform (`lstk terraform`), the AWS CDK (`lstk cdk`), and the AWS SAM CLI (`lstk sam`). :::tip[Recommended] `lstk` is the recommended way to run and manage LocalStack. The [legacy LocalStack CLI](/aws/developer-tools/running-localstack/localstack-cli/) is deprecated. ::: This section is split into focused pages: - **Overview** (this page): installation, quick start, global options, and shell completions. - [Authentication](/snowflake/developer-tools/lstk/authentication/): logging in and out, and how `lstk` resolves your auth token. - [Configuration](/snowflake/developer-tools/lstk/configuration/): the `config.toml` file, emulator types, environment variables, and volumes. - [Lifecycle commands](/snowflake/developer-tools/lstk/lifecycle-commands/): `start`, `stop`, `restart`, `status`, `logs`, `reset`, `volume`. - [Cloud & IaC commands](/snowflake/developer-tools/lstk/cloud-and-iac-commands/): `aws`, `az`, `terraform`, `cdk`, `sam`. - [Snapshots](/snowflake/developer-tools/lstk/snapshots/): save and load emulator state with `snapshot save`/`load`/`list`/`remove`/`show`. - [Automation & CI](/snowflake/developer-tools/lstk/automation/): non-interactive mode, structured output, targeting an external emulator, and environment variables. - [Setup & maintenance](/snowflake/developer-tools/lstk/setup-and-maintenance/): `setup`, `config`, `update`, and offline/enterprise environments. - [FAQ & Troubleshooting](/snowflake/developer-tools/lstk/faq-and-troubleshooting/). ## Prerequisites - [Docker](https://docs.docker.com/get-docker/) installed and running. - A [LocalStack account](https://www.localstack.cloud/pricing) with a [license](/snowflake/getting-started/auth-token/#managing-your-license), and `lstk` handles authentication for you (see [Authentication](/snowflake/developer-tools/lstk/authentication/)). ## Installation ```bash brew install localstack/tap/lstk ``` Homebrew also installs shell completions for bash, zsh, and fish automatically. ```bash npm install -g @localstack/lstk ``` Download the binary for your platform from [GitHub Releases](https://github.com/localstack/lstk/releases), extract it, and place it on your `PATH`. Verify the installation: ```bash lstk --version ``` ### Updating `lstk` can update itself. It detects how it was originally installed (Homebrew, npm, or binary) and uses the matching update method: ```bash # Check for updates without installing lstk update --check # Update to the latest version lstk update ``` See the [`update`](/snowflake/developer-tools/lstk/setup-and-maintenance/#update) command for details, including the start-time update notification. ## Quick start ```bash lstk ``` Running `lstk` without arguments performs the full startup sequence: authenticates you automatically, pulls the latest image if needed, and starts the LocalStack container. In an interactive terminal it launches the TUI; in a non-interactive environment it prints plain text output. On the very first interactive run, `lstk` prompts you to pick which emulator to run (AWS, Snowflake, or Azure) and writes your choice to `config.toml`. See [Emulator types](/snowflake/developer-tools/lstk/configuration/#emulator-types) for the available options. For CI or headless environments, set `LOCALSTACK_AUTH_TOKEN` and use `--non-interactive`: ```bash LOCALSTACK_AUTH_TOKEN= lstk --non-interactive ``` CI environments require a CI Auth Token; a personal Developer Auth Token cannot be used there. ## Global options These options are available for all commands: | Option | Description | |:--------------------|:------------------------------------------------------------------------------| | `--config ` | Path to a specific TOML config file | | `--endpoint-url ` | Target an existing, externally-managed emulator at this URL instead of discovering one via local Docker. See [Targeting an external emulator](/snowflake/developer-tools/lstk/automation/#targeting-an-external-emulator). | | `--non-interactive` | Disable the interactive TUI, use plain output | | `--json` | Emit a single machine-readable JSON envelope on stdout instead of human-oriented output. Supported by `start`, `stop`, `status`, `reset`, and `update`; any other command rejects it. See [Structured output](/snowflake/developer-tools/lstk/automation/#structured-output). | | `--persist` | Persist emulator state across restarts (on `start`/bare `lstk` and `restart`) | | `--type `, `-t ` | Emulator type to start: `aws`, `snowflake`, or `azure` (on `start`/bare `lstk`; records the choice in config). See [Selecting the emulator with `--type`](/snowflake/developer-tools/lstk/lifecycle-commands/#selecting-the-emulator-with---type). | | `--snapshot ` | Snapshot REF to auto-load after start (on `start`/bare `lstk`; overrides config for one run) | | `--no-snapshot` | Skip auto-loading the configured snapshot (on `start`/bare `lstk`) | | `--timeout ` | Startup readiness deadline for `start`/bare `lstk`, as a Go duration; overrides `LSTK_STARTUP_TIMEOUT` for one run. See [`start`](/snowflake/developer-tools/lstk/lifecycle-commands/#start). | | `-v`, `--version` | Print the version and exit | | `-h`, `--help` | Print help and exit | These apply to both interactive and non-interactive (scripted/CI) use, see [Automation & CI](/snowflake/developer-tools/lstk/automation/) for the details behind `--non-interactive`, `--json`, and `--endpoint-url`. ## Shell completions `lstk` includes completion scripts for bash, zsh, fish, and powershell. If you installed via Homebrew, completions are set up automatically. For manual setup: ```bash # Load in current session eval "$(lstk completion bash)" # Persist (Linux) lstk completion bash > /etc/bash_completion.d/lstk # Persist (macOS with Homebrew) lstk completion bash > $(brew --prefix)/etc/bash_completion.d/lstk ``` :::note Use `eval "$(lstk completion bash)"` rather than `source <(lstk completion bash)`. The `lstk` script works with or without the `bash-completion` package (it bundles a fallback for stock macOS bash 3.2), but `source <(...)` is a silent no-op on that shell. ::: ```bash # Load in current session source <(lstk completion zsh) # Persist (Linux) lstk completion zsh > "${fpath[1]}/_lstk" # Persist (macOS with Homebrew) lstk completion zsh > $(brew --prefix)/share/zsh/site-functions/_lstk ``` ```bash # Load in current session lstk completion fish | source # Persist lstk completion fish > ~/.config/fish/completions/lstk.fish ``` Restart your shell after persisting completions. ### `completion` Generate shell completion scripts. ```bash lstk completion [bash|zsh|fish|powershell] ``` See [Shell completions](#shell-completions) above for setup instructions. # lstk Authentication > How lstk resolves your auth token, and the login and logout commands. `lstk` resolves your auth token in the following order: 1. **System keyring**: a token stored by a previous `lstk login`. 2. **`LOCALSTACK_AUTH_TOKEN` environment variable**: used only when the keyring has no token. 3. **Browser login**: triggered automatically in interactive mode when neither of the above provides a token. :::caution The keyring token takes precedence over `LOCALSTACK_AUTH_TOKEN`. If you set or change the environment variable but a keyring token already exists, the environment variable is ignored. Run `lstk logout` to clear the stored keyring token first. ::: ## Logging in ```bash lstk login ``` Opens a browser window for authentication and stores the resulting token in your system keyring. This command requires an interactive terminal. See the [`login`](#login) command below for the full flow and the endpoints it uses. ## Logging out ```bash lstk logout ``` Removes the stored credentials from the system keyring and the file-based fallback, and clears the cached license. `logout` cannot clear a token supplied via `LOCALSTACK_AUTH_TOKEN`; if you authenticated that way, unset the variable instead. See the [`logout`](#logout) command below for the full behavior. ## File-based token storage On systems where the system keyring is unavailable, `lstk` automatically falls back to storing the token in a file (`/auth-token`, mode `0600`). You can force file-based storage by setting: ```bash export LSTK_KEYRING=file ``` ## `login` Authenticate with LocalStack via a browser-based device authorization flow and store the resulting credential in your system keyring. This command requires an interactive terminal. ```bash lstk login ``` `lstk` opens your default browser to the LocalStack Web Application, shows a one-time code, and waits for you to approve the request. If the browser cannot open automatically, `lstk` prints the URL to visit manually. On success it stores the **license token** returned by the platform (not the raw browser bearer token). If you are already authenticated — either `LOCALSTACK_AUTH_TOKEN` is set or a token already exists in storage — `login` prints `You're already logged in` and exits without starting a new flow. In non-interactive mode (piped output, CI, or `--non-interactive`), `login` fails with `login requires an interactive terminal`. The `--config ` flag selects which `config.toml` is loaded, which affects `keyring`, `web_app_url`, and `api_endpoint` resolution. :::note If you approve the request in the browser only *after* pressing a key in the terminal, `lstk` reports `auth request not confirmed - please complete the authentication in your browser`. Re-run `lstk login` and approve in the browser before continuing. ::: The credential is written to the system keyring (service `lstk`, key `lstk.auth-token`). When the keyring is unavailable — or `LSTK_KEYRING=file` is set — `lstk` stores it in a file at `/auth-token` (mode `0600`) instead. Endpoints used by the flow can be overridden via config or environment: | Config key | Env var | Default | Description | |:---------------|:--------------------|:-------------------------------|:-----------------------------------------------------------------------------| | `keyring` | `LSTK_KEYRING` | (system keyring) | Set to `file` to force file-based token storage instead of the OS keyring. | | `web_app_url` | `LSTK_WEB_APP_URL` | `https://app.localstack.cloud` | Base URL used to build the browser authorization link. | | `api_endpoint` | `LSTK_API_ENDPOINT` | `https://api.localstack.cloud` | LocalStack platform API endpoint used for the device flow and license token. | ```bash # Force file-based token storage during login LSTK_KEYRING=file lstk login # Use a specific config file lstk --config ./.lstk/config.toml login ``` ## `logout` Remove stored authentication credentials. ```bash lstk logout lstk logout --non-interactive ``` `logout` deletes the auth token from your system keyring (falling back to the file-based token at `/auth-token` when the keyring is unavailable or `LSTK_KEYRING=file` is set) and removes the cached license file. On success it prints `Logged out successfully`. The outcome depends on how you are authenticated: | Situation | Behavior | |:----------|:---------| | A token is stored (from `lstk login`) | The token is deleted from the keyring and file fallback, the cached license is removed, and `lstk` prints `Logged out successfully`. | | No stored token, but `LOCALSTACK_AUTH_TOKEN` is set | Nothing is deleted. `lstk` prints a note that you are authenticated via the environment variable and to unset it to log out. | | No stored token and no `LOCALSTACK_AUTH_TOKEN` | `lstk` prints `Not currently logged in` and exits successfully. | :::note `logout` never clears the `LOCALSTACK_AUTH_TOKEN` environment variable, and it does not stop running emulators. If a LocalStack emulator is still running after logout, `lstk` prints a note reminding you it is running in the background; run `lstk stop` to stop it. ::: # lstk Automation & CI > Non-interactive mode, structured JSON output, targeting an external emulator, environment variables, tracing, and logging for scripting lstk. ## Interactive and non-interactive mode `lstk` automatically selects its output mode: - **Interactive mode** (TUI): used when both stdin and stdout are connected to a terminal. Commands like `start`, `stop`, `restart`, `status`, `login`, `update`, and the confirmation prompts of `reset`/`volume clear` display a Bubble Tea-powered terminal UI. - **Non-interactive mode** (plain text): used when the output is piped, redirected, or running in CI. Force this in a TTY with `--non-interactive`. ```bash # Force plain output even in an interactive terminal lstk --non-interactive start ``` :::note `lstk login` requires an interactive terminal; if you need to authenticate in CI, set `LOCALSTACK_AUTH_TOKEN` instead. Commands that mutate state without prompting in CI (`reset`, `volume clear`) require `--force`. `lstk setup aws` works non-interactively — it writes the profile with defaults and needs `--force` only to overwrite a conflicting `localstack` profile. ::: ## Targeting an external emulator By default `lstk` discovers the emulator it manages through local Docker. The `--endpoint-url ` global flag (or the `LSTK_ENDPOINT_URL` environment variable) instead points a command at an emulator `lstk` did not start, a Docker Compose or host-network deployment, one running in CI or on another machine, or a LocalStack cloud-hosted ephemeral instance. ```bash # Run against an emulator reachable at a custom URL lstk aws --endpoint-url http://localhost:4566 s3 ls # Equivalent via the environment LSTK_ENDPOINT_URL=https://my-ephemeral-instance.localstack.cloud lstk status ``` The endpoint is resolved from, in order of precedence: the `--endpoint-url` flag, `LSTK_ENDPOINT_URL`, then `AWS_ENDPOINT_URL` (a full synonym for `LSTK_ENDPOINT_URL`, one tier lower). Both `http://` and `https://` URLs are accepted (any other scheme is rejected), and the scheme is preserved end-to-end, so `https://` ephemeral instances work. The commands that accept an external endpoint are the ones that only *talk to* an already-running emulator: [`aws`](/snowflake/developer-tools/lstk/cloud-and-iac-commands/#aws), [`az`](/snowflake/developer-tools/lstk/cloud-and-iac-commands/#az), [`terraform`](/snowflake/developer-tools/lstk/cloud-and-iac-commands/#terraform)/`tf`, [`cdk`](/snowflake/developer-tools/lstk/cloud-and-iac-commands/#cdk), [`sam`](/snowflake/developer-tools/lstk/cloud-and-iac-commands/#sam), [`status`](/snowflake/developer-tools/lstk/lifecycle-commands/#status), [`reset`](/snowflake/developer-tools/lstk/lifecycle-commands/#reset), and the [`snapshot`](/snowflake/developer-tools/lstk/snapshots/) `save`/`load`/`remove` subcommands (including the `lstk save`/`lstk load` aliases) and `list s3://…`. Commands that manage the emulator's lifecycle or on-disk state have no remote equivalent and **reject** any endpoint source: `start`, the bare `lstk`, `stop`, `restart`, `logs`, and `volume`. The emulator's type (AWS, Azure, or Snowflake) is auto-detected by probing the endpoint's health API, there is no override flag or config setting, and an inconclusive probe is a hard failure. The AWS-only tools (`terraform`, `cdk`, `sam`) reject an endpoint whose detected type is not AWS. ## Structured output The global `--json` flag makes a command emit a single, machine-readable JSON object on stdout instead of human-oriented text, for scripting and CI. JSON support is available per command: `start`, `stop`, `status`, `reset`, and `update` accept `--json`. Any other command rejects it with an error envelope (`error.code: NOT_JSON_CAPABLE`) rather than silently printing plain text. Every JSON-capable command writes **exactly one** JSON object with the following envelope shape: ```json { "schemaVersion": 1, "command": "stop", "status": "ok", "data": { "emulators": [ { "type": "snowflake", "name": "localstack-snowflake", "wasRunning": true } ] }, "warnings": [], "error": null } ``` | Field | Type | Description | |:----------------|:-----------------|:--------------------------------------------------------------------------------------------------------| | `schemaVersion` | integer | Wire-format version of the envelope, always `1` for this schema. Check it once before parsing. | | `command` | string | The command that produced the envelope (e.g. `"stop"`, `"reset"`). | | `status` | string | `"ok"` or `"error"` — branch on this first. | | `data` | object or `null`| Command-specific result. Non-null when `status` is `"ok"`, `null` when it is `"error"`. | | `warnings` | array | Non-fatal notices, always present (empty array when there are none). Each entry is `{ "code", "message" }`. | | `error` | object or `null`| The machine-readable failure. Non-null when `status` is `"error"`, `null` otherwise. | When `status` is `"error"`, the `error` object carries a stable `code` (e.g. `EMULATOR_NOT_RUNNING`, `CONFIRMATION_REQUIRED`, `RUNTIME_UNAVAILABLE`), a coarse `category`, a human-readable `message` (informational only — branch on `code`, not `message`), and a `retryable` boolean: ```json { "schemaVersion": 1, "command": "reset", "status": "error", "data": null, "warnings": [], "error": { "code": "CONFIRMATION_REQUIRED", "category": "USAGE", "message": "reset requires confirmation; use --force to skip in non-interactive mode", "retryable": false } } ``` ### Exit codes For a full enumeration, read `error.code` from the envelope; the process exit code carries only the two most common, mechanically-remediable failures: | Exit code | Meaning | |:----------|:-------------------------------------------------------------------------------------------| | `0` | `status: "ok"`. | | `1` | `status: "error"` for any code other than the two below. | | `2` | A Cobra-level usage error that occurred before `--json` could be recognized (plain-text error on stderr, not an envelope). | | `3` | `error.code == "CONFIRMATION_REQUIRED"` (re-run with `--force`). | | `4` | `error.code == "AUTH_REQUIRED"` (run `lstk login` or set `LOCALSTACK_AUTH_TOKEN`). | :::note `--json` implies non-interactive behavior: no TUI and no prompts. Combining it with a destructive command that would otherwise prompt (`reset`) still requires `--force`, which surfaces as `CONFIRMATION_REQUIRED` (exit code `3`) when omitted. ::: ## Environment variables The following environment variables configure `lstk` itself (not the LocalStack container): | Variable | Description | |:-------------------------------|:---------------------------------------------------------------------------------------------------------------------| | `LOCALSTACK_AUTH_TOKEN` | Auth token for non-interactive runs or to skip browser login. Takes precedence over a token stored in the keyring. | | `LSTK_ENDPOINT_URL` | Target an existing, externally-managed emulator at this URL (equivalent to `--endpoint-url`). `AWS_ENDPOINT_URL` is a lower-precedence synonym. See [Targeting an external emulator](#targeting-an-external-emulator). | | `LOCALSTACK_HOST` | Override the host (and optional port) used when resolving and printing the emulator endpoint, and when writing the AWS CLI profile. Bypasses the `localhost.localstack.cloud` DNS probe. | | `LOCALSTACK_DISABLE_EVENTS` | Set to `1` to disable anonymous telemetry event reporting. | | `DOCKER_HOST` | Override the Docker daemon socket (e.g. `unix:///home/user/.colima/default/docker.sock`). | | `LSTK_KEYRING` | Set to `file` to force file-based token storage instead of the system keyring. | | `LSTK_STARTUP_TIMEOUT` | Startup readiness deadline for `lstk start`, as a Go duration (e.g. `90s`, `2m`). Zero/unset uses the per-mode default (20s interactive, 60s non-interactive). See [`start`](/snowflake/developer-tools/lstk/lifecycle-commands/#start). | | `LSTK_MERGE_STRATEGY` | Default merge strategy for `snapshot load` / `load` (`account-region-merge`, `overwrite`, or `service-merge`) when `--merge` is not passed. An explicit `--merge` always wins. | | `LSTK_OTEL` | Set to `1` to enable OpenTelemetry trace export (disabled by default). See [OpenTelemetry tracing](#opentelemetry-tracing). | | `LSTK_GITHUB_TOKEN` | Optional GitHub token used when checking for or downloading `lstk` updates (raises GitHub API rate limits). | | `LSTK_API_ENDPOINT` | Override the LocalStack platform API base URL. Default: `https://api.localstack.cloud`. | | `LSTK_WEB_APP_URL` | Override the LocalStack Web Application URL used for browser login. Default: `https://app.localstack.cloud`. | When `LSTK_OTEL` is enabled, the standard `OTEL_EXPORTER_OTLP_*` environment variables are honored by the OpenTelemetry SDK. ### Container runtime discovery `lstk` talks to a Docker-compatible runtime and works with Docker Desktop, Rancher Desktop, Colima, OrbStack, Lima, and Podman. When `DOCKER_HOST` is not set, it resolves the daemon endpoint in this order: 1. **`DOCKER_HOST`**, if set, always wins. 2. **`DOCKER_CONTEXT`** or the active Docker CLI context, when it is non-default and reachable (a stale or unreachable context is skipped rather than failing). 3. On **Linux**, a live `/var/run/docker.sock`, a running Docker daemon is preferred over a co-installed runtime such as Podman. 4. A probe of known runtime sockets (Docker Desktop, Rancher Desktop, Colima, OrbStack, Podman, Lima). Each candidate is dialed, not just checked for existence, so a leftover socket file never shadows a live daemon. 5. The Docker SDK's own default. If no runtime is reachable, the error tailors its suggested start command (`rdctl start`, `colima start`, `podman machine start`, …) to the runtime it detects. Set `DOCKER_HOST` to point at a specific socket to bypass discovery entirely. ### Container-injected variables `lstk` injects several environment variables into the LocalStack container on every start, in addition to any profiles you configure: | Variable | Default value | Description | |:-----------------------------|:-------------------------------------------------|:---------------------------------------------| | `LOCALSTACK_AUTH_TOKEN` | (your resolved token) | Passed from the CLI to activate the license. | | `GATEWAY_LISTEN` | `:4566,:443` | Ports the emulator binds inside the container. | | `MAIN_CONTAINER_NAME` | `localstack-snowflake` | Container name for internal references. | | `LOCALSTACK_HOST` | `localhost.localstack.cloud:` | Hostname/port the emulator advertises. | | `LOCALSTACK_PERSISTENCE` | `1` (only with `--persist`) | Enables state persistence across restarts. | | `LOCALSTACK_CLIENT_NAME` | `lstk` | Identifies the client that started the emulator. | | `LOCALSTACK_CLIENT_VERSION`| (the `lstk` version) | Version of the client that started the emulator. | When a Docker socket is detected it is bind-mounted into the container and `DOCKER_HOST=unix:///var/run/docker.sock` is injected so the emulator can spawn its own containers. `lstk` also forwards host environment variables matching `CI` and `LOCALSTACK_*` (the host `LOCALSTACK_AUTH_TOKEN` is dropped so it cannot override the token resolved by `lstk`). The container also gets port mappings for `4566`, `443`, and the service port range `4510-4559`. :::note `GATEWAY_LISTEN` is read from the container's resolved environment (set it via an `[env.*]` profile), not hardcoded. Beyond controlling which ports the emulator binds, its host part sets the host publish IP for all published ports: a value like `GATEWAY_LISTEN = "0.0.0.0:4566,0.0.0.0:443"` exposes the emulator beyond loopback (e.g. on a remote host), whereas the default binds to `127.0.0.1` only. ::: ## OpenTelemetry tracing `lstk` can export traces of its own command execution over OTLP/HTTP. Tracing is **disabled by default**. Enable it with: ```bash LSTK_OTEL=1 lstk start ``` When enabled, every command is wrapped in a span (e.g. `lstk.start`) recording the exit code and any error. `lstk` does not hardcode an export target, so the OpenTelemetry Go SDK reads the standard `OTEL_EXPORTER_OTLP_*` environment variables automatically (default target: OTLP/HTTP at `localhost:4318`). You need an OTLP-compatible backend running to receive the traces. ## Logging `lstk` writes its own diagnostic logs to `lstk.log` in the same directory as the active config file. This is separate from the LocalStack container logs (which you view with [`lstk logs`](/snowflake/developer-tools/lstk/lifecycle-commands/#logs)). - The log file is created automatically and appended to across runs. - When the file exceeds **1 MB**, it is cleared on the next run. - Use `lstk config path` to find the config directory; `lstk.log` sits alongside `config.toml`. # lstk Cloud & IaC Commands > The aws, az, terraform, cdk, and sam commands that proxy cloud and infrastructure-as-code tools against LocalStack. `lstk` proxies developer tools so they run directly against LocalStack. :::note Like `lstk aws`, the `az`, `terraform`, `cdk`, and `sam` proxies do not start the emulator — start it first with [`lstk start`](/snowflake/developer-tools/lstk/lifecycle-commands/#start). Each requires the corresponding third-party CLI to be installed and on your `PATH`. ::: :::note When you interrupt a proxied tool (for example Ctrl+C or `kill` during `lstk terraform apply`), `lstk` forwards the termination signal to the wrapped tool and waits for it to shut down cleanly rather than killing it outright, so operations like releasing a Terraform state lock can complete. The wrapped tool's real exit code is passed through unchanged. ::: ## `aws` Run AWS CLI commands against the running LocalStack emulator. `lstk aws` proxies your host `aws` CLI with the endpoint, credentials, and region pre-configured, so you don't have to pass `--endpoint-url` or set test credentials yourself. ```bash lstk aws s3 ls lstk aws sqs list-queues lstk aws s3 mb s3://my-bucket ``` It is equivalent to running: ```bash aws --endpoint-url http://localhost:4566 ``` with `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_DEFAULT_REGION` set automatically. Everything after `lstk aws` is forwarded verbatim to the host `aws` binary, including AWS CLI flags such as `--region` or `--output`. The exit code and `stdout`/`stderr` of the underlying `aws` process are passed through unchanged, so piping and interactive subcommands work as expected. | Option | Description | |:--------------------|:--------------------------------------------------------------------------------------------------| | `--non-interactive` | Suppress the loading spinner. Unlike other commands, this flag is stripped before invoking `aws` (not forwarded). | :::note `lstk aws` does not start the emulator. The AWS emulator must already be running (`lstk start`), Docker must be healthy, and the host `aws` CLI must be installed and on your `PATH`. ::: ### Credentials and region `lstk aws` injects credentials in one of two ways: - **Profile mode**: if a complete `localstack` profile exists in both `~/.aws/config` and `~/.aws/credentials`, `lstk` appends `--profile localstack` and lets `aws` read the region, credentials, and endpoint from that profile. - **Profile-less mode**: if the profile is not present, `lstk` runs `aws` with `AWS_ACCESS_KEY_ID=test`, `AWS_SECRET_ACCESS_KEY=test`, and `AWS_DEFAULT_REGION=us-east-1` injected only when those variables are not already set in your environment. In this mode it also prints an informational note: `No AWS profile found, run 'lstk setup aws'`. Run [`lstk setup aws`](/snowflake/developer-tools/lstk/setup-and-maintenance/#setup-aws) to create the `localstack` profile for use with the AWS CLI and SDKs. ### Endpoint resolution By default, `lstk` probes whether `localhost.localstack.cloud` resolves to `127.0.0.1` and uses `localhost.localstack.cloud:` if so, otherwise it falls back to `127.0.0.1:`. Set [`LOCALSTACK_HOST`](/snowflake/developer-tools/lstk/automation/#environment-variables) to override the host:port used to reach LocalStack and skip the DNS probe. The port comes from the AWS container's `port` in `config.toml` (default `4566`). ## `az` Run Azure CLI commands against the running LocalStack Azure emulator. `lstk az` runs `az` with an isolated `AZURE_CONFIG_DIR` in which a custom Azure cloud is registered against LocalStack's endpoints, so your global `~/.azure` configuration is left untouched and plain `az` keeps talking to real Azure. Run [`lstk setup azure`](/snowflake/developer-tools/lstk/setup-and-maintenance/#setup-azure) once before using this mode. Everything after `lstk az` is forwarded verbatim to the host `az` binary, and its exit code and output are passed through unchanged. ```bash lstk az group list lstk az storage account list ``` The Azure CLI has no `--endpoint-url`/`--profile` equivalent, so the isolation relies entirely on the dedicated config directory prepared by `setup azure`. ### Global interception (optional) If a script must invoke plain `az` (not `lstk az`), you can redirect your **global** `~/.azure` to LocalStack instead: ```bash # Point global 'az' at the LocalStack Azure emulator lstk az start-interception # Switch back to real Azure lstk az stop-interception ``` `start-interception` registers and activates the `LocalStack` cloud in your global Azure configuration so every `az` invocation targets LocalStack until you stop it. `stop-interception` switches the active cloud back to `AzureCloud` (override with `--cloud `) and re-enables instance discovery, but only when `LocalStack` is still the active cloud, to avoid clobbering an unrelated selection. :::caution Interception changes global state that affects every `az` command in any terminal. Use the isolated `lstk az ` mode unless you specifically need plain `az` to target LocalStack. ::: ## `terraform` Run Terraform against LocalStack, using LocalStack endpoints as AWS provider overrides. `lstk terraform` (alias `lstk tf`) generates a provider-override file and forwards your arguments to the real `terraform` binary. :::note `lstk terraform` targets the AWS emulator. To use Terraform with the other emulators, see the relevant emulator docs. ::: ```bash lstk terraform init lstk terraform --region us-west-2 plan lstk tf apply ``` lstk-specific flags must appear **before** the Terraform action: | Option | Default | Description | |:------------------|:---------------------|:---------------------------------------| | `--region ` | `us-east-1` | Deployment region. | | `--account ` | `test` | Target AWS account id (12 digits). | Relevant environment variables: `AWS_ENDPOINT_URL` (override the auto-resolved endpoint), `LSTK_TF_CMD` (binary to invoke, e.g. `tofu`; default `terraform`), `LSTK_TF_OVERRIDE_FILE_NAME` (override file name; default `localstack_providers_override.tf`), `LSTK_TF_DRY_RUN` (generate the override file but do not run Terraform), `AWS_REGION` (fallback for `--region`), and `AWS_ACCESS_KEY_ID` (fallback for `--account`). ## `cdk` Run the AWS CDK against LocalStack. Requires the AWS CDK CLI version `2.177.0` or newer on your `PATH`. ```bash lstk cdk bootstrap lstk cdk --region us-west-2 deploy lstk cdk synth ``` The only lstk-specific flag (before the CDK action) is `--region ` (default `us-east-1`); CDK always targets the default LocalStack account `000000000000`, so there is no `--account` flag. Relevant environment variables: `AWS_ENDPOINT_URL`, `AWS_ENDPOINT_URL_S3`, `LSTK_CDK_CMD` (default `cdk`), and `AWS_REGION`. ## `sam` Run the AWS SAM CLI against LocalStack. Requires the AWS SAM CLI version `1.95.0` or newer on your `PATH` (older versions ignore `AWS_ENDPOINT_URL` and would target real AWS). ```bash lstk sam build lstk sam --region us-west-2 deploy lstk sam validate ``` lstk-specific flags (before the SAM action): `--region ` (default `us-east-1`) and `--account ` (12 digits, default `000000000000`). Relevant environment variables: `AWS_ENDPOINT_URL`, `AWS_ENDPOINT_URL_S3`, `LSTK_SAM_CMD` (default `sam`), `AWS_REGION` (fallback for `--region`), and `AWS_ACCESS_KEY_ID` (fallback for `--account`). :::note Compared with `samlocal`, image/container-based Lambda (ECR) deploys and nested CloudFormation stacks are not supported; use `samlocal` for those workflows. ::: # lstk Configuration > The lstk config.toml file, emulator types, environment variables, custom images, and volume mounts. `lstk` uses a TOML configuration file, created automatically on first run. ## Config file search order `lstk` uses the first `config.toml` it finds in this order: 1. `./.lstk/config.toml`: project-local config in the current directory. 2. `$HOME/.config/lstk/config.toml`: user config (created here if `$HOME/.config/` exists). 3. OS default: - **macOS**: `$HOME/Library/Application Support/lstk/config.toml` - **Windows**: `%AppData%\lstk\config.toml` - **Linux**: `$XDG_CONFIG_HOME/lstk/config.toml` or `$HOME/.config/lstk/config.toml` On first run, the config is created at path #2 if `$HOME/.config/` already exists; otherwise at the OS default (#3). To see the active config file path: ```bash lstk config path ``` To use a specific config file: ```bash lstk --config /path/to/config.toml start ``` ## Default configuration The default `config.toml` created on first run. The `type` field reflects whichever emulator you chose at first run (see [Emulator types](#emulator-types)); the example below shows the `snowflake` default: ```toml [[containers]] type = "snowflake" # Emulator type. Supported: "aws", "snowflake", "azure" tag = "latest" # Docker image tag, e.g. "latest", "2026.4" port = "4566" # Host port the emulator will be accessible on # image = "" # Full image override (e.g. an internal mirror or offline image) # volume = "" # Host directory for persistent state (default: OS cache dir) # volumes = [] # Docker-style "host:container[:ro]" bind mounts (see Volumes) # env = [] # Named environment profiles to apply (see [env.*] sections below) # snapshot = "" # Snapshot REF to auto-load after start (AWS only) ``` ## Config field reference | Field | Type | Default | Description | |:-----------|:---------|:-----------|:-----------------------------------------------------------------------------------------------------| | `type` | string | `"aws"` | Emulator type. One of `"aws"`, `"snowflake"`, `"azure"`. Run a single `[[containers]]` block at a time. See [Emulator types](#emulator-types). | | `tag` | string | `"latest"` | Docker image tag (`"latest"`, `"2026.4"`, etc.). Useful for pinning a specific version. Zero-padded months (`"2026.04"`) are normalized to `"2026.4"`. | | `port` | string | `"4566"` | Host port the emulator listens on (1–65535). The in-container port is always `4566`. | | `image` | string | (default) | Full image reference that overrides the default Docker Hub image, e.g. an internal-registry mirror or a locally loaded offline image. If it already carries a tag, `tag` is ignored; otherwise `tag` (or `latest`) is appended. | | `volume` | string | (OS cache) | Host directory for persistent emulator state. Defaults to `/lstk/volume/`. See also `volumes`. | | `volumes` | string[] | `[]` | Docker-style `"host:container[:ro]"` bind mounts (e.g. init hooks). May also carry the persistence mount (target `/var/lib/localstack`). See [Volume mounts](#volume-mounts). | | `env` | string[] | `[]` | List of named environment profiles to inject into the container (see below). | | `snapshot` | string | `""` | Snapshot REF (e.g. `pod:my-baseline` or a local path) to auto-load after the emulator starts. AWS emulator only; not available for Snowflake. See [Auto-loading a snapshot on start](/aws/developer-tools/running-localstack/lstk/lifecycle-commands/#auto-loading-a-snapshot-on-start) in the AWS docs. | :::note There is no `update_prompt` config key. `lstk` always checks for available updates on startup. Once you choose to skip a version, `lstk` records it under the `[cli]` table as `update_skipped_version` and stops prompting for that version. This value is written automatically and is not meant to be hand-edited (see [`update`](/snowflake/developer-tools/lstk/setup-and-maintenance/#update)). ::: ## Emulator types `lstk` can run more than one kind of emulator. The `type` field in your `config.toml` selects which one: | Type | Docker image | Description | |:------------|:------------------------------|:-------------------------------------| | `aws` | `localstack/localstack-pro` | LocalStack AWS emulator (default). | | `snowflake` | `localstack/snowflake` | LocalStack Snowflake emulator. | | `azure` | `localstack/localstack-azure` | LocalStack Azure emulator. | On the first interactive run, `lstk` prompts you to pick an emulator (`a` for AWS, `s` for Snowflake, `z` for Azure) and writes your choice to `config.toml`. In non-interactive mode the default `aws` emulator is used if no config file is found. Lifecycle commands operate on the emulators defined in your `config.toml`. Run a single `[[containers]]` block at a time; the AWS-specific commands (`status` resources, `aws`, `reset`, `setup aws`) require an `aws` emulator to be configured. :::note The AWS emulator's license is validated by `lstk` before the container starts. The Snowflake and Azure emulators validate their own license inside the container at startup, so `lstk` skips its pre-flight license check for them. If your license does not include the selected emulator, the container exits and `lstk` reports the missing entitlement. ::: ## Passing environment variables to the container Define reusable environment profiles under `[env.]` and reference them in your container config: ```toml [[containers]] type = "snowflake" tag = "latest" port = "4566" env = ["debug"] [env.debug] DEBUG = "1" SF_LOG = "debug" ``` When `lstk start` runs, the key-value pairs from each referenced profile are injected as environment variables into the LocalStack container. Keys are uppercased automatically. :::note If you reference an `env` profile name that doesn't exist in your config, `lstk` returns an error: `environment "..." referenced in container config not found`. ::: In addition to your custom profiles, `lstk` always injects several variables into the container. See [Container-injected variables](/snowflake/developer-tools/lstk/automation/#container-injected-variables) for the full list. ## Custom container image By default the emulator image is pulled from Docker Hub (`localstack/localstack-pro`, `localstack/snowflake`, or `localstack/localstack-azure` depending on `type`). Set `image` on a container block to override it — for example, to pull from an internal-registry mirror or to run a locally loaded image in an air-gapped environment: ```toml [[containers]] type = "snowflake" image = "registry.internal.example.com/localstack/snowflake" tag = "2026.4" ``` If `image` already carries a tag (e.g. `...:2026.4`), the separate `tag` field is ignored; otherwise `tag` (or `latest`) is appended. See [Offline and enterprise environments](/snowflake/developer-tools/lstk/setup-and-maintenance/#offline-and-enterprise-environments) for how `lstk` falls back to a locally present image when a pull fails. ## Volume mounts Beyond the single persistence directory set by `volume`, a container block can declare arbitrary Docker-style bind mounts with `volumes`. Each entry is a `"host:container[:ro]"` spec — useful, for example, for mounting a [Snowflake init hook](/snowflake/capabilities/init-hooks/) script into `/etc/localstack/init/{boot,start,ready,shutdown}.d`: ```toml [[containers]] type = "snowflake" port = "4566" volumes = [ "./test.sf.sql:/etc/localstack/init/ready.d/test.sf.sql", "./data:/var/lib/localstack", ] ``` - A `volumes` entry whose container target is `/var/lib/localstack` sets the persistence directory (the same mount `volume` configures); this is what [`lstk volume path`](/snowflake/developer-tools/lstk/lifecycle-commands/#volume) and [`lstk volume clear`](/snowflake/developer-tools/lstk/lifecycle-commands/#volume) resolve. - Relative host sources and a leading `~/` are resolved against the config file's directory. This differs from the legacy `volume` field, whose value is passed to Docker verbatim. - Setting the persistence directory through both `volume` and a `volumes` entry with a different source is a validation error. `volume` and `volumes` overlap only for the persistence mount: `volume` can *only* set the persistence directory, while `volumes` is a superset that can also express init hooks and other mounts. ## Using a project-local config Place a `.lstk/config.toml` in your project directory. When you run `lstk` from that directory, the local config takes precedence over the global one. This lets each project pin its own emulator type, image tag, and environment profiles. For example, a project that targets the Snowflake emulator can keep its own config: ```toml # .lstk/config.toml [[containers]] type = "snowflake" port = "4566" ``` A project that also wants to pin a specific image tag and enable a debug profile might instead use: ```toml # .lstk/config.toml [[containers]] type = "snowflake" tag = "2026.4" port = "4566" env = ["dev"] [env.dev] DEBUG = "1" PERSISTENCE = "1" ``` # lstk FAQ & Troubleshooting > Frequently asked questions and common issues when using lstk. ## FAQ ### Can I use `lstk` with Docker Compose? Yes, for the commands that talk to an already-running emulator. `lstk start`, `lstk stop`, and the other lifecycle commands manage their own Docker container and are not meant to drive a Compose-managed instance, so don't point `lstk stop` at one. But if you run LocalStack from a `docker-compose.yml`, you can still use `lstk`'s emulator-facing commands against it, `aws`, `az`, `terraform`/`cdk`/`sam`, `status`, `reset`, and `snapshot`, by passing `--endpoint-url ` (or setting `LSTK_ENDPOINT_URL`) to target the Compose deployment. See [Targeting an external emulator](/snowflake/developer-tools/lstk/automation/#targeting-an-external-emulator) for the commands that accept an endpoint, and the [Docker Compose installation guide](/snowflake/getting-started/installation/#docker-compose) for the Compose setup itself. ### Which Docker image does `lstk` use? It depends on the emulator type configured in your `config.toml`. The AWS emulator uses `localstack/localstack-pro`, the Snowflake emulator uses `localstack/snowflake`, and the Azure emulator uses `localstack/localstack-azure`. All require a valid auth token (including the free Hobby tier). See [Emulator types](/snowflake/developer-tools/lstk/configuration/#emulator-types). ### How do I pass configuration options like `DEBUG` or `PERSISTENCE` to the container? Use environment profiles in your `config.toml`. Define the variables under an `[env.]` section and reference that name in the `env` list of your container config. See [Passing environment variables to the container](/snowflake/developer-tools/lstk/configuration/#passing-environment-variables-to-the-container) for details. ### How do I save and restore emulator state? Use [`lstk snapshot save`](/snowflake/developer-tools/lstk/snapshots/#snapshot-save) to capture the running Snowflake emulator's state to a local file or a Cloud Pod, and [`lstk snapshot load`](/snowflake/developer-tools/lstk/snapshots/#snapshot-load) (or the `lstk save` / `lstk load` aliases) to restore it. To drop in-memory state without writing a snapshot, use [`lstk reset`](/snowflake/developer-tools/lstk/lifecycle-commands/#reset) (AWS emulator only). ### How do I pin a specific LocalStack version? Set the `tag` field in your `config.toml` to a specific version tag: ```toml [[containers]] type = "snowflake" tag = "2026.4" port = "4566" ``` ## Troubleshooting ### Port 443 already in use By default, LocalStack publishes both port `4566` and port `443` (controlled by the `GATEWAY_LISTEN` variable). On some systems port 443 is already taken, Windows with Hyper-V, IIS, or VPN software, or an ingress proxy such as Rancher Desktop's Traefik. Because port 443 comes from the **default** `GATEWAY_LISTEN`, a busy 443 is **not fatal**: `lstk` drops that publication with a warning and starts anyway, and HTTPS is still served on the edge port `4566`. You only need to act if you want to silence the warning or bind 443 elsewhere. To skip port 443 entirely, override `GATEWAY_LISTEN` to bind only to `4566`: ```toml [[containers]] type = "snowflake" tag = "latest" port = "4566" env = ["nossl"] [env.nossl] GATEWAY_LISTEN = "0.0.0.0:4566" ``` :::note A port you list **explicitly** in a custom `GATEWAY_LISTEN` is treated as a hard requirement, so a busy one there fails the start rather than being dropped. Only the `443` from the default value is best-effort. ::: ### Docker is not running `lstk` requires a running Docker daemon. If Docker is not reachable, you will see an error like: ```text Error: runtime not healthy ``` **Fix:** Start your container runtime. `lstk` works with Docker Desktop, Rancher Desktop, Colima, OrbStack, Lima, and Podman, start the Docker daemon (`sudo systemctl start docker` on Linux) or the relevant VM (`rdctl start`, `colima start`, `podman machine start`, …). When the runtime is unavailable, `lstk`'s error tailors its suggested start command to whichever runtime it detects. You can also point `lstk` at a specific socket with `DOCKER_HOST`. See [Container runtime discovery](/snowflake/developer-tools/lstk/automation/#container-runtime-discovery) for how the daemon is located. ### Authentication required in non-interactive mode When running without a TTY (e.g. in CI), `lstk` cannot open a browser for login. If no token is found in the keyring or environment, it fails: ```text authentication required: set LOCALSTACK_AUTH_TOKEN or run in interactive mode ``` **Fix:** Set the `LOCALSTACK_AUTH_TOKEN` environment variable before running `lstk`: ```bash export LOCALSTACK_AUTH_TOKEN= lstk --non-interactive start ``` You can find your auth token on the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens). ### License validation failed If your auth token is invalid, expired, or not linked to an active license, the LocalStack container exits with a license error: ```text The license activation failed for the following reason: No credentials were found in the environment. ``` **Fix:** - Verify your token is valid at the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens). - Make sure the token is set correctly, either via `lstk login` or the `LOCALSTACK_AUTH_TOKEN` environment variable. - A stale token or cached license no longer requires a manual `lstk logout`: when the platform definitively rejects it, `lstk` drops the cached license and, in an interactive terminal, prompts you to log in again and retries automatically. In non-interactive mode, run `lstk logout && lstk login` (or set a valid `LOCALSTACK_AUTH_TOKEN`) and re-run. ### Image pull failed If `lstk` cannot pull the Docker image, check your network connection and Docker configuration. On corporate networks, you may need to configure Docker's proxy settings, see [How do I configure LocalStack to use my corporate HTTP and HTTPS proxy?](/aws/getting-started/faq/#how-do-i-configure-localstack-to-use-my-corporate-http-and-https-proxy). ### Unknown environment profile If your container config references an `env` profile that doesn't exist, `lstk` returns: ```text environment "myprofile" referenced in container config not found ``` **Fix:** Make sure the profile name in the `env` list matches an `[env.]` section in your `config.toml`: ```toml [[containers]] type = "snowflake" env = ["myprofile"] # must match the section name below [env.myprofile] DEBUG = "1" ``` ### Getting help If the steps above don't resolve your issue, see [Get Help](/snowflake/help-support/get-help/) for the available support channels, including the support email and in-app chat. # lstk Lifecycle Commands > The start, stop, restart, status, logs, reset, and volume commands for managing the LocalStack emulator with lstk. `lstk` uses a flat command structure. Running `lstk` with no command is equivalent to `lstk start`. ## `start` Start the LocalStack emulator. Launches the TUI in interactive terminals and prints plain output otherwise. `lstk start` launches the emulator defined in the first `[[containers]]` entry of the resolved `config.toml` (not necessarily AWS). ```bash lstk start lstk start --persist lstk start --non-interactive ``` | Option | Description | |:--------------------|:-----------------------------------------------------------------------------| | `--persist` | Persist emulator state across restarts (sets `LOCALSTACK_PERSISTENCE=1` in the container) | | `--type `, `-t ` | Select the emulator to start (`aws`, `snowflake`, or `azure`) non-interactively, recording the choice in `config.toml`. See [Selecting the emulator with `--type`](#selecting-the-emulator-with---type). | | `--snapshot ` | Auto-load this snapshot after the emulator starts, overriding the configured `snapshot` for one run (AWS only) | | `--no-snapshot` | Skip auto-loading the configured `snapshot` for this run | | `--timeout ` | Maximum time to wait for the emulator to become ready, as a Go duration (e.g. `90s`, `2m`). Overrides `LSTK_STARTUP_TIMEOUT` for this run; `0` uses the per-mode default. | | `--non-interactive` | Disable the interactive TUI and use plain output | `lstk start` forwards host environment variables prefixed with `LOCALSTACK_` to the emulator (the host `LOCALSTACK_AUTH_TOKEN` is dropped so it cannot override the token `lstk` resolved). See [Container-injected variables](/snowflake/developer-tools/lstk/automation/#container-injected-variables). `lstk` applies a readiness deadline while waiting for the emulator to come up (a crash during startup is detected instantly, with its exit code, and does not wait for the deadline). In an interactive terminal the deadline defaults to 20 seconds and is only a recoverable prompt — you can keep waiting or stop; in non-interactive mode it defaults to 60 seconds and is fatal, leaving the container running for inspection. Override the deadline for a single run with `--timeout` (a Go duration such as `90s` or `2m`), or for every run with [`LSTK_STARTUP_TIMEOUT`](/snowflake/developer-tools/lstk/automation/#environment-variables); an explicit `--timeout` wins over the environment variable, and `--timeout 0` falls back to the per-mode default. The flag is available on `start` and the bare `lstk` command only — `restart` and the snapshot auto-start path do not expose it. By default the emulator starts with a fresh state on every run. Pass `--persist` to keep data across restarts: `lstk` injects `LOCALSTACK_PERSISTENCE=1` into the container so state is written to the mounted [`volume`](/snowflake/developer-tools/lstk/configuration/#config-field-reference) and reloaded on the next start. When persistence is active, the AWS emulator's startup summary includes a `• Persistence: Enabled` line. ```bash # Start with persistent state lstk start --persist ``` :::note `--persist` is a flag on `start` (and the bare `lstk` command) and on [`restart`](#restart). For finer-grained control, you can also set `PERSISTENCE = "1"` in an environment profile (see [Passing environment variables to the container](/snowflake/developer-tools/lstk/configuration/#passing-environment-variables-to-the-container)). ::: `start` supports [`--json`](/snowflake/developer-tools/lstk/automation/#structured-output) (as does the bare `lstk` command, which reports `"command": "start"`): the `data` payload is a flat object describing the started emulator, its `emulator` type, `container` name, `endpoint`, `version`, whether it was `alreadyRunning`, and whether `persistence` is enabled. ### Selecting the emulator with `--type` `--type` (shorthand `-t`, also available on the bare `lstk` command) is the non-interactive answer to the first-run emulator picker. It selects which emulator to start (`aws`, `snowflake`, or `azure`) and **records the choice in `config.toml`**, so lifecycle commands (`stop`, `status`, `logs`, `volume`, snapshot auto-load) stay in sync with what you started. ```bash # Start the Snowflake emulator, recording the choice in config lstk start --type snowflake # Shorthand lstk start -t azure ``` - On first run, the config is created with the selected type. - If the configured type already matches, `--type` is a no-op. - If it differs, `lstk` rewrites the `type` line in place (comments and formatting preserved) and prints a note naming the config file. When switching an existing config to a different type: - A custom `image` is a **hard error** — it pins a specific product that cannot be reinterpreted under a new emulator type. Use a separate config (`--config`) for that profile instead. - A non-`latest` `tag` and any `volume`/`volumes` mounts are kept, but `lstk` warns that they may be product-specific. - `port`, `env`, and `snapshot` are kept silently. `--type` is a flag only; passing the emulator as a positional (`lstk start azure`) is rejected with a hint pointing at `--type`. ## `stop` Stop the running LocalStack emulator. Stops every emulator container defined in the resolved `config.toml` (the `[[containers]]` entries), with a 30-second stop timeout per container. ```bash lstk stop lstk stop --non-interactive ``` `stop` fails fast if the Docker runtime is not healthy (for example, Docker is not running), or if a configured emulator is not currently running (`LocalStack is not running`). In an interactive terminal it shows an animated "Stopping LocalStack..." spinner and a styled confirmation; in non-interactive mode it prints the same progress and result as plain text. `stop` supports [`--json`](/snowflake/developer-tools/lstk/automation/#structured-output): the `data` payload lists each configured emulator and whether it `wasRunning`. ## `restart` Stop and restart the LocalStack emulator. Performs a stop of the running emulator followed by a fresh start, using the same auth, config, and Docker settings as [`start`](#start). Launches the TUI in interactive terminals and prints plain output otherwise. ```bash lstk restart lstk restart --persist ``` | Option | Description | |:-------------|:-------------------------------------------| | `--persist` | Persist emulator state across the restart | By default, emulator state is **not** retained across the restart and the container starts clean. Pass `--persist` to keep the emulator's state so it survives the restart. ## `status` Show the status of a running emulator and its deployed resources. Before contacting the emulator, `lstk` checks that the Docker runtime is healthy; if it is not, the command reports `runtime not healthy` and exits with a non-zero status. ```bash lstk status lstk --non-interactive status ``` For each emulator configured in your `config.toml` (the `[[containers]]` entries), `status` reports whether it is running and, if so, prints an instance summary: ```text ✔︎ LocalStack Snowflake Emulator is running • Endpoint: snowflake.localhost.localstack.cloud:4566 • Container: localstack-snowflake • Version: 2026.9.0.dev113 • Uptime: 30s ``` - **Endpoint** is the live `host:port`, queried from Docker, so it stays correct even if the configured `port` was changed while the container kept running. - **Persistence** appears only for the AWS emulator and only when persistence is enabled. - **Uptime** is computed from the container's start time and is omitted if it cannot be determined. If an emulator is not running, `status` prints an error and exits non-zero without checking the remaining emulators: ```text LocalStack Snowflake Emulator is not running Start LocalStack: lstk See help: lstk -h ``` For the **AWS emulator**, `status` additionally lists deployed resources. When resources exist it prints a summary line followed by a table; when none exist it prints `No resources deployed`. ```text ~ 3 resources · 2 services Service Resource Region Account S3 my-bucket us-east-1 000000000000 SQS my-queue us-east-1 000000000000 ``` In an interactive terminal the output is rendered through the TUI; in non-interactive mode (or with `--non-interactive`) the same content is printed as plain text, with the resource table shown at full width when stdout is not a TTY. The Snowflake and Azure emulators show the instance summary only and never report resources. `status` supports [`--json`](/snowflake/developer-tools/lstk/automation/#structured-output): the `data` payload lists one entry per configured emulator with its running state, health, version, and host. For the AWS emulator it also includes a `resourceSummary` and the deployed `resources`, which `--no-resources` omits for a faster response when polling. `--json` also honors [`--endpoint-url`](/snowflake/developer-tools/lstk/automation/#targeting-an-external-emulator) to report on an emulator `lstk` did not start. ## `logs` Show or stream emulator logs. ```bash lstk logs [options] ``` | Option | Description | |:------------|:-----------------------------------------| | `--follow`, `-f` | Stream logs in real-time. Without this flag, `lstk` prints the currently available logs and exits. | | `--verbose`, `-v` | Show all logs without filtering. By default, `lstk` drops noisy lines (internal request logs, provider chatter); `--verbose` shows every line verbatim. | | `--tail `, `-n ` | Show only the last `N` lines from the end of the logs. Accepts a non-negative integer or `all` (the default, showing all available lines). | By default, `lstk logs` reads from the first configured emulator container and applies a noise filter. In an interactive terminal, lines are color-coded by log level (`DEBUG`, `INFO`, `WARN`, `ERROR`); in non-interactive mode, raw log lines are written to stdout. Example: ```bash # Print current filtered logs and exit lstk logs # Stream filtered logs in real-time lstk logs --follow # Show only the last 100 lines lstk logs --tail 100 # Stream all logs without filtering lstk logs --follow --verbose ``` ## `reset` Discard the running AWS emulator's in-memory state (all created resources such as S3 buckets and Lambda functions are dropped). The emulator **keeps running**; only its state is cleared. `reset` is **AWS-only** and errors out with `reset is only supported for the AWS emulator` for the Snowflake and Azure emulators. ```bash lstk reset lstk reset --force ``` | Option | Description | |:----------|:------------------------------------------------------------------| | `--force` | Skip the confirmation prompt. Required in non-interactive mode. | In interactive mode, `reset` prompts for confirmation before clearing state. In non-interactive mode it fails unless `--force` is passed: ```text reset requires confirmation; use --force to skip in non-interactive mode ``` `reset` supports [`--json`](/snowflake/developer-tools/lstk/automation/#structured-output): on success the `data` payload reports the reset emulator and `"reset": true`. :::note `reset` clears in-memory state only. It does **not** wipe the on-disk volume (certificates, persistence data, cached tools). To clear that, stop the emulator and run [`lstk volume clear`](#volume-clear). ::: ## `volume` Manage the emulator volume: the host directory that holds persistent state such as certificates, downloaded tools, and persistence data. ```bash lstk volume path lstk volume clear [options] ``` ### `volume path` Prints the resolved volume directory for every emulator in your config, one per line. With the default config (a single `aws` emulator) it prints one path. Each path is the container's configured `volume` value, or the default OS cache location if `volume` is unset (`~/Library/Caches/lstk/volume/localstack-aws` on macOS, `~/.cache/lstk/volume/localstack-aws` on Linux). ```bash # Print the volume directory for each configured emulator lstk volume path ``` ### `volume clear` Removes all data from the emulator volume directory, resetting cached state. It operates on all configured emulators by default, or a single one with `--type`. Before clearing, it lists each target as `: ()`. | Option | Description | |:----------------|:-----------------------------------------| | `--force` | Skip the confirmation prompt | | `--type ` | Clear only the emulator of this type | ```bash # Clear all configured emulator volumes (prompts for confirmation) lstk volume clear # Clear only the AWS emulator volume lstk volume clear --type aws # Skip the confirmation prompt lstk volume clear --force # Clear without prompting in a non-interactive environment lstk volume clear --type snowflake --force ``` In an interactive terminal, `lstk volume clear` prompts `Clear volume data? This cannot be undone` before deleting anything; choosing **NO** or pressing Ctrl+C cancels with no changes. In non-interactive mode, `--force` is required, otherwise the command fails with `volume clear requires confirmation; use --force to skip in non-interactive mode`. :::caution If the volume contains files owned by `root` (created by Docker), clearing fails with a permission error. Re-run with elevated privileges: ```bash sudo lstk volume clear ``` ::: # lstk Setup & Maintenance > The setup, config, and update commands, and running lstk in offline or enterprise environments. ## `setup` Set up CLI integration for an emulator type. `lstk setup` is a grouping command with no action of its own; the work is done by its subcommands, `setup aws` and `setup azure`. ```bash lstk setup aws lstk setup azure ``` :::note There is no `setup snowflake` subcommand. `setup aws` and `setup azure` exist because `lstk` wraps the AWS and Azure CLIs and needs to point their own profiles at the emulator; the Snowflake emulator has no CLI for `lstk` to wrap in that way. Snowflake clients connect through their own `config.toml` connection profiles instead, and a single one can hold multiple named connections. See [Integrations](/snowflake/integrations/) for how to connect SnowSQL, drivers, BI tools, and other clients to the emulator. ::: ### `setup aws` Create or update a `localstack` profile in `~/.aws/config` and `~/.aws/credentials` so the AWS CLI and SDKs can target LocalStack. ```bash lstk setup aws lstk setup aws --force ``` | Option | Description | |:----------|:-----------------------------------------------------------------------------------------| | `--force` | Overwrite an existing `localstack` profile whose values differ, and skip the confirmation prompt. | On an interactive terminal it prompts (Y/n) before making changes. In non-interactive mode (piped output, CI, or `--non-interactive`) it writes the profile with defaults without prompting and exits `0`; a failed write or check returns a non-zero exit code so automation notices. Overwriting an existing `localstack` profile whose values differ requires `--force` (which also skips the interactive prompt); creating a fresh profile, completing a partial one, or leaving an already-correct profile in place never needs it. It writes the following profile (existing unrelated profiles are preserved): ```ini # ~/.aws/config [profile localstack] region = us-east-1 output = json endpoint_url = http://localhost.localstack.cloud:4566 # ~/.aws/credentials [localstack] aws_access_key_id = test aws_secret_access_key = test ``` Afterwards, target LocalStack by passing `--profile localstack` or exporting `AWS_PROFILE`: ```bash export AWS_PROFILE=localstack aws s3 ls ``` The endpoint host is resolved the same way as for [`lstk aws`](/snowflake/developer-tools/lstk/cloud-and-iac-commands/#endpoint-resolution) (probing `localhost.localstack.cloud` and falling back to `127.0.0.1`), and [`LOCALSTACK_HOST`](/snowflake/developer-tools/lstk/automation/#environment-variables) overrides the host and port written into the profile. The port comes from your AWS emulator's configured `port` (default `4566`); if no `aws` emulator is configured, the command fails with `no aws emulator configured`. If the `localstack` profile is already configured correctly, `lstk` reports `LocalStack AWS profile is already configured.` and makes no changes. :::note The former `lstk config profile` command has been removed; use `lstk setup aws`. ::: ### `setup azure` Prepare an isolated Azure CLI configuration directory (under the `lstk` config dir, via `AZURE_CONFIG_DIR`) that routes [`lstk az`](/snowflake/developer-tools/lstk/cloud-and-iac-commands/#az) commands to the LocalStack Azure emulator. Your global `~/.azure` configuration is left untouched. ```bash lstk setup azure # alias: lstk setup az ``` `setup azure` registers a custom Azure cloud (`LocalStack`) whose endpoints point at the LocalStack Azure emulator, activates it, disables Azure CLI instance discovery and telemetry, and performs a one-time dummy service-principal login — all inside a dedicated config directory under the `lstk` config dir (via `AZURE_CONFIG_DIR`). It requires the `az` CLI to be installed and a running LocalStack Azure emulator. Run this once; afterwards use `lstk az ` to run Azure CLI commands against LocalStack. To instead redirect your **global** `az` (so existing scripts run unmodified against LocalStack), see [`lstk az start-interception`](/snowflake/developer-tools/lstk/cloud-and-iac-commands/#global-interception-optional). ## `config` Manage CLI configuration. `config` has no behavior of its own; run it with a subcommand. ### `config path` Print the resolved path to the active `config.toml`. ```bash lstk config path ``` This subcommand is read-only: it never creates or initializes a config file. If `--config ` is set, it prints that path verbatim. Otherwise it prints the already-loaded config path, the first existing config in the search order, or the path where a config would be created on first run. ## `update` Check for and apply updates to the `lstk` CLI itself. `lstk` auto-detects how it was installed (Homebrew, npm, or direct binary) and updates using that same method. Development builds (version `dev`) are skipped, and updates are checked against the latest [GitHub release](https://github.com/localstack/lstk/releases/latest). ```bash lstk update [options] ``` | Option | Description | |:--------------------|:------------------------------------------------------------| | `--check` | Check for updates without installing them | | `--non-interactive` | Use plain output instead of the TUI (update logic unchanged) | | `--json` | Emit the result as a JSON envelope (see [Structured output](/snowflake/developer-tools/lstk/automation/#structured-output)). With `--check`, `data` reports `currentVersion`/`latestVersion`/`updateAvailable`; after an applied update, `updatedVersion`/`updated`/`method`. | Examples: ```bash # Check for updates without installing lstk update --check # Update to the latest version lstk update # Update with plain (non-TUI) output lstk update --non-interactive ``` By install method: - **Homebrew** (binary under a `Caskroom` path): runs `brew upgrade localstack/tap/lstk`. - **npm** (binary under `node_modules`): runs `npm install -g @localstack/lstk@latest`. - **Binary** (anything else): downloads the release asset for your OS/arch from GitHub, extracts it, and replaces the running executable in place. With `--check`, `lstk` only reports whether a newer version is available and exits without downloading or installing anything. :::note Set `LSTK_GITHUB_TOKEN` to send an authenticated GitHub request and avoid API rate limits during update checks. It is optional; updates also work unauthenticated. ::: If more than one `lstk` installation is found on your `PATH` (for example a Homebrew binary and an npm one), `lstk update` and the start-time update notification print a warning listing each location, its install method, and which one is currently running, so you can tell which binary an update will actually replace. ### Update notification on start Separately from `lstk update`, `lstk` checks for a newer version when you run `lstk start` (the default command), using a short timeout that fails silently if GitHub is unreachable. In an interactive terminal, when an update is available `lstk` prints the new version and a release-notes link, then prompts: ```text Update lstk to latest version? > Update now [U] Remind me next time [R] Skip this version [S] ``` - **Update now [U]**: downloads and applies the update, then asks you to re-run your command. - **Remind me next time [R]**: does nothing; you are reminded on the next run. - **Skip this version [S]**: records the version in `config.toml` so you are not prompted about it again. In non-interactive mode the notification is not a prompt — `lstk` emits a single note (`Update available: (run lstk update)`) and continues. When you choose **Skip this version**, `lstk` writes the skipped version under a `[cli]` table: ```toml [cli] update_skipped_version = "0.5.0" ``` While this value matches the latest available version, the start-time update notification for that version is suppressed. This key is managed automatically and is not intended to be edited by hand. ## Offline and enterprise environments There is no `--offline` flag. Instead, `lstk` degrades gracefully when common enterprise blockers (Docker Hub unreachable, a proxy/TLS interceptor, or an unreachable license server) prevent an internet request: - **Image pull**: if the image pull fails but the image is already present locally, `lstk` warns and uses the local image instead of failing. In interactive mode you can also press Esc to abort an in-progress pull and fall back to the local image. - **License pre-flight**: when the pinned image is already present locally, `lstk` skips its pre-flight license check so a fully offline start is not blocked; the emulator validates the license itself once it starts. When a check does run, a transport-level failure (offline, proxy, or certificate error) is treated as non-fatal and the emulator validates the license instead. A definitive server rejection (HTTP 400/401/403) is handled differently: `lstk` drops the cached license and, in an interactive terminal, offers to log in again and retries the start once with the refreshed credentials (a rejected token often just predates a license purchase or plan change); in non-interactive mode it fails with an error pointing at `lstk logout && lstk login` or a valid `LOCALSTACK_AUTH_TOKEN`. The pre-flight is also skipped — with a warning — when the license server does not recognize the image *tag format* (for example a `dev` nightly or a custom internal-mirror tag): that is not a verdict on the license, so `lstk` defers to the emulator's own startup check rather than blocking the start. - **Telemetry and update checks** are best-effort and fail silently when offline. Pair this behavior with a custom [`image`](/snowflake/developer-tools/lstk/configuration/#custom-container-image) that points at an internal-registry mirror or a locally loaded image to run `lstk` in an air-gapped environment. # lstk Snapshots > Save, load, list, remove, and show emulator snapshots with lstk, including S3 remotes. ## `snapshot` Manage emulator snapshots. A snapshot captures the running emulator's state, either as a local file on disk, as a Cloud Pod on the LocalStack platform, or in your own S3 bucket. The `snapshot` command groups five subcommands — `save`, `load`, `list`, `remove`, and `show`. The first two are also exposed as the top-level aliases `lstk save` and `lstk load`. :::note Snapshots are best supported on the **AWS emulator**. `snapshot save`/`load` (and the `save`/`load` aliases) also work for the Snowflake emulator, but its snapshot support is experimental and not fully tested — `lstk` prints a warning such as `Snapshot support for the snowflake emulator is experimental and not fully tested.` Azure emulator persistence is still a work in progress and is not yet supported. ::: ## `snapshot save` Save a snapshot of the running emulator's state. The emulator must already be running; this command does **not** auto-start it. ```bash # Auto-named snapshot file in the current directory lstk snapshot save # Save to a specific local path lstk snapshot save ./my-snapshot # Save to a Cloud Pod on the LocalStack platform (requires auth) lstk snapshot save pod:my-baseline # Save to your own S3 bucket (pod name is auto-generated if omitted) lstk snapshot save my-pod s3://my-bucket/prefix # Limit the snapshot to a subset of services lstk snapshot save --services s3,lambda ``` The optional `[destination]` argument takes one of these forms: | Destination | Description | |:---------------------------------|:--------------------------------------------------------------------------------------------------| | (omitted) | Auto-generates a timestamped snapshot file in the current directory (`./snapshot--.snapshot`). | | local path | Writes a snapshot archive to that path. The `.snapshot` extension is forced. | | `pod:` | Saves a Cloud Pod to the LocalStack platform. Requires authentication. | | ` s3://bucket/prefix` | Saves to your own S3 bucket. The pod name is a separate positional (auto-generated when omitted). See [S3 remotes](#s3-remotes). | Pod operations require an auth token (`LOCALSTACK_AUTH_TOKEN` or a prior `lstk login`); local-file snapshots do not. By default a snapshot captures every service's state. Pass `-s`/`--services` with a comma-separated list to limit it to a subset; this applies uniformly to local files, `pod:` Cloud Pods, and `s3://` remotes. | Option | Description | |:--------------------|:------------------------------------------------------------------------------------------------| | `--services `, `-s ` | Comma-separated list of services to include in the snapshot (all services by default). Applies to local, `pod:`, and `s3://` destinations. | | `--profile ` | AWS profile to read S3 credentials from (used only for `s3://` destinations). Defaults to `AWS_*` env vars, then `AWS_PROFILE`. | ## `snapshot load` Load a snapshot into the emulator, **auto-starting it first** if it is not already running. ```bash # Load a local snapshot by path or name lstk snapshot load my-baseline lstk snapshot load ./checkpoint # Load from a Cloud Pod (requires auth) lstk snapshot load pod:my-baseline # Load from your own S3 bucket (pod name is required) lstk snapshot load my-pod s3://my-bucket/prefix # Control how the snapshot merges with running state lstk snapshot load pod:my-baseline --merge=overwrite # Preview what a Cloud Pod load would change, without applying it lstk snapshot load pod:my-baseline --dry-run ``` The `REF` argument is required and identifies a local path/name or a `pod:` Cloud Pod. To load from S3, pass the pod name followed by an `s3://bucket/prefix` location (see [S3 remotes](#s3-remotes)). | Option | Description | |:---------------------|:------------------------------------------------------------------------------------------------------------| | `--merge ` | How the loaded state combines with running state. One of `account-region-merge` (default), `overwrite`, `service-merge`. | | `--dry-run` | Preview the resource additions and modifications the load would produce, per service, without changing any state. Supported for `pod:` refs only; requires a running emulator (it does not auto-start one). | | `--profile ` | AWS profile to read S3 credentials from (used only for `s3://` sources). Defaults to `AWS_*` env vars, then `AWS_PROFILE`. | - `account-region-merge` (default): the snapshot wins on any `(service, account, region)` overlap. - `overwrite`: running state is reset first, then the snapshot is imported onto a clean state. - `service-merge`: the snapshot wins per resource; non-overlapping resources are combined. Set [`LSTK_MERGE_STRATEGY`](/snowflake/developer-tools/lstk/automation/#environment-variables) to change the default strategy used when `--merge` is not passed; an explicit `--merge` always wins. Pass `--dry-run` with a `pod:` ref to preview a load before committing to it: `lstk` queries the platform and prints, per service, how many resources the snapshot would add or modify under the chosen merge strategy, without touching running state. It is supported for `pod:` refs only (other refs are rejected) and requires the emulator to already be running, since it does not auto-start one. ### `save`/`load` aliases `snapshot save` and `snapshot load` are also exposed as the top-level aliases `lstk save` and `lstk load`. The aliases behave identically: ```bash lstk save pod:my-baseline lstk load ./checkpoint ``` ## `snapshot list` List the Cloud Pod snapshots available on the LocalStack platform. By default, only snapshots you created are listed; pass `--all` to include every snapshot in your organization. This subcommand operates on Cloud Pods, so it requires authentication. ```bash # Snapshots you created lstk snapshot list # Every snapshot in your organization lstk snapshot list --all # List snapshots in your own S3 bucket (requires a running emulator) lstk snapshot list s3://my-bucket/prefix ``` Passing an `s3://bucket/prefix` location lists snapshots stored in your own S3 bucket instead of the platform (see [S3 remotes](#s3-remotes)). Unlike the platform listing, this queries the emulator, so it requires a running emulator. | Option | Description | |:--------------------|:--------------------------------------------------------------| | `--all` | List all snapshots in your organization, not just your own. | | `--profile ` | AWS profile to read S3 credentials from (used only with an `s3://` location). Defaults to `AWS_*` env vars, then `AWS_PROFILE`. | ## `snapshot remove` Delete a Cloud Pod snapshot from the LocalStack platform. Only cloud snapshots (the `pod:` prefix) can be removed; local snapshots are plain files you delete yourself. This operation cannot be undone. ```bash lstk snapshot remove pod:my-baseline # Skip the confirmation prompt (required in non-interactive mode) lstk snapshot remove pod:my-baseline --force ``` The required `REF` argument must be a `pod:` Cloud Pod reference. | Option | Description | |:----------|:-------------------------------------------------------------------------| | `--force` | Skip the confirmation prompt. Required when running non-interactively. | ## `snapshot show` Show metadata for a single Cloud Pod snapshot on the LocalStack platform: its name, created date, size, LocalStack version, message, the services it contains, and per-service resource counts (resource counts render only when the platform has them for that snapshot). This subcommand is cloud-only and requires authentication. ```bash lstk snapshot show pod:my-baseline ``` The required `REF` argument must be a `pod:` Cloud Pod reference. ## S3 remotes `snapshot save`, `load`, and `list` can target a snapshot stored in your **own S3 bucket** by passing an `s3://bucket/prefix` location. The pod name (the snapshot's identity within the bucket) is a positional separate from the `s3://` location — required for `load`, auto-generated for `save` when omitted, and unused for `list`. ```bash lstk snapshot save my-pod s3://my-bucket/prefix lstk snapshot load my-pod s3://my-bucket/prefix lstk snapshot list s3://my-bucket/prefix ``` Credentials follow AWS CLI precedence: `--profile ` wins, otherwise the static `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` (plus optional `AWS_SESSION_TOKEN`) environment variables, otherwise the profile named by `AWS_PROFILE`. Only static credentials are supported (no SSO, assume-role, or `credential_process`), and credentials must never be embedded in the URL. `lstk` runs a pre-flight check that the target bucket exists and errors out rather than letting the emulator auto-create a bucket on a typo. Because the transfer is performed by the emulator (not the CLI), S3 remotes require a **running emulator**, and `list s3://…` in particular queries the emulator rather than the platform API. :::note `remove` and `show` do not support S3; they operate on Cloud Pods only. ::: # User Interface > Get started with LocalStack for Snowflake Web User Interface ## Introduction The Snowflake emulator provides a User Interface (UI) via the [LocalStack Web Application](https://app.localstack.cloud/). The User Interface allows you to: * Run SQL queries and view results using a Query Editor. * View detailed request/response traces of API calls. To access the User Interface, you need to start the Snowflake emulator and access the **Snowflake** tab in your default instance of the LocalStack Web Application. This User Interface is available only when the Snowflake emulator is running. Please note that it does not connect to the real Snowflake cloud environment or any other external service on the Internet. :::note Please note that the Snowflake User Interface is still experimental and under active development. ::: ## Getting started This guide is designed for users new to the Snowflake emulator Web UI. Start your Snowflake emulator using the following command: ```bash export LOCALSTACK_AUTH_TOKEN= lstk start ``` Navigate to [**https://app.localstack.cloud/inst/default/snowflake**](https://app.localstack.cloud/inst/default/snowflake) to access the User Interface. ### Run SQL queries The User Interface provides a **SQL Worksheet** tab that allows you to run SQL queries and view results. The editor includes SQL syntax highlighting and autocomplete for SQL keywords, Snowflake built-in functions, and schema objects such as databases, tables, and columns. ![Running SQL queries](/images/snowflake/snowflake-run-queries.png) Use the **Local Resources** panel on the right to explore your schema objects in a hierarchical tree. Each node type — database, schema, table, view, column, and folder — is represented by a dedicated icon. Views are displayed as a distinct category alongside tables within each schema. ![Local Resources](/images/snowflake/local-resources.png) You can run the current statement by placing your cursor inside it, or execute only a selected portion of SQL from the editor. #### Multi-statement query support ![Multi Statement Query](/images/snowflake/snow-ui-multi-statement-query.png) The SQL Worksheet supports executing multiple SQL statements in a single run. When a multi-statement query is submitted, results are displayed in a scrollable tabbed interface, with each tab labeled **Statement 1**, **Statement 2**, and so on. If a statement fails, its corresponding tab is highlighted to make it easy to identify errors. :::note A truncation banner is displayed when query results exceed 100 rows, indicating that only the first 100 rows are shown. ::: ### View Query History The User Interface provides a **Query History** tab that displays recently executed queries along with their execution details. Each entry includes metadata such as the query ID, SQL text, execution status, duration, rows returned, and the database, schema, and warehouse used during execution. You can search and filter the query history to quickly locate queries by text, execution status, or time range. ![Query history](/images/snowflake/snowflake-query-history.png) # Feature Coverage > Overview of the implemented Snowflake features in LocalStack ## Resource Types and Operations This page provides a list of Snowflake query features (resource types and operations) that are supported in the LocalStack emulator. The content will be updated as additional query features and functions are implemented. ### Applications | |ALTER|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----| |**APPLICATION**|✅|✅|✅|✅|✅| ### Application Packages | |ALTER|CREATE|DROP|SHOW| |----|----|----|----|----| |**APPLICATION PACKAGE**|✅|✅|✅|✅| ### Catalog Integration | |ALTER|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----| |**CATALOG INTEGRATION**|❌|✅|❌|✅|✅| ### Databases | |ALTER|CREATE|DESCRIBE|DROP|SHOW|UNDROP|USE| |----|----|----|----|----|----|----|----| |**DATABASE**|✅|✅|✅|✅|✅|❌|✅| ### Data and File Operations | |COPY INTO|GET|LIST|PUT|REMOVE| |----|----|----|----|----|----| |**FILE OPERATIONS**|✅|✅|✅|✅|✅| ### Dynamic Tables | |ALTER|CREATE|DESCRIBE|DROP|SHOW|UNDROP| |----|----|----|----|----|----|----| |**DYNAMIC TABLE**|✅|✅|✅|✅|✅|❌| ### External Tables | |ALTER|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----| |**EXTERNAL TABLE**|❌|❌|❌|❌|❌| ### External Volumes | |ALTER|CREATE|DESCRIBE|DROP|SHOW|UNDROP| |----|----|----|----|----|----|----| |**EXTERNAL VOLUME**|✅|✅|✅|✅|✅|❌| ### File Formats | |ALTER|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----| |**FILE FORMAT**|✅|✅|✅|✅|✅| ### Functions | |ALTER|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----| |**FUNCTION**|✅|✅|✅|✅|✅| ### Hybrid Tables | |CREATE|SHOW| |----|----|----| |**HYBRID TABLE**|✅|✅| ### Iceberg Tables | |ALTER|CREATE|DESCRIBE|DROP|SHOW|UNDROP| |----|----|----|----|----|----|----| |**ICEBERG TABLE**|❌|✅|❌|❌|❌|❌| ### Indexes | |CREATE|DROP|SHOW| |----|----|----|----| |**INDEX**|✅|✅|✅| ### Materialized Views | |ALTER|CREATE|DESCRIBE|DROP|SHOW|TRUNCATE| |----|----|----|----|----|----|----| |**MATERIALIZED VIEW**|✅|✅|✅|✅|✅|✅| ### Pipes | |ALTER|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----| |**PIPE**|✅|✅|✅|✅|✅| ### Procedures | |ALTER|CALL|CALL WITH|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----|----|----| |**PROCEDURE**|❌|✅|❌|✅|✅|✅|✅| ### Roles | |ALTER|CREATE|DROP|GRANT|REVOKE|SHOW|USE| |----|----|----|----|----|----|----|----| |**ROLE**|❌|✅|✅|❌|❌|✅|✅| ### Row Access Policies | |ALTER|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----| |**ROW ACCESS POLICY**|✅|✅|✅|✅|✅| ### Schemas | |ALTER|CREATE|DESCRIBE|DROP|SHOW|UNDROP|USE| |----|----|----|----|----|----|----|----| |**SCHEMA**|✅|✅|✅|✅|✅|❌|✅| ### Sequences | |ALTER|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----| |**SEQUENCE**|✅|✅|✅|✅|✅| ### Shares | |ALTER|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----| |**SHARE**|❌|✅|❌|✅|✅| ### Stages | |ALTER|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----| |**STAGE**|✅|✅|✅|✅|✅| ### Storage Integrations | |ALTER|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----| |**STORAGE INTEGRATION**|❌|✅|✅|✅|✅| ### Streams | |ALTER|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----| |**STREAM**|✅|✅|✅|✅|✅| ### Streamlits | |ALTER|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----| |**STREAMLIT**|✅|✅|✅|✅|✅| ### Tables | |ALTER|CREATE|DESCRIBE|DROP|SHOW|TRUNCATE|UNDROP| |----|----|----|----|----|----|----|----| |**TABLE**|✅|✅|✅|✅|✅|❌|❌| ### Tags | |ALTER|CREATE|DROP|SHOW|UNDROP| |----|----|----|----|----|----| |**TAG**|✅|✅|✅|✅|❌| ### Tasks | |ALTER|CREATE|DESCRIBE|DROP|EXECUTE|SHOW| |----|----|----|----|----|----|----| |**TASK**|✅|✅|✅|✅|✅|✅| ### Users | |ALTER|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----| |**USER**|✅|✅|❌|✅|✅| ### Views | |ALTER|CREATE|DESCRIBE|DROP|SHOW| |----|----|----|----|----|----| |**VIEW**|✅|✅|✅|✅|✅| ### Warehouses | |ALTER|CREATE|DESCRIBE|DROP|SHOW|USE| |----|----|----|----|----|----|----| |**WAREHOUSE**|✅|✅|✅|✅|✅|✅| # Features > Browse LocalStack's implemented Snowflake features and explore their capabilities import SearchableSnowflakeFeatures from '../../../../components/SearchableSnowflakeFeatures.astro'; # Accounts > Get started with Accounts in LocalStack for Snowflake ## Introduction An account is a unique identifier for a Snowflake instance within an organization. It acts as a container for resources and operations related to data storage, processing, and management. The Snowflake emulator lets you connect to and manage resources in different accounts. ## Getting Started This guide explains how to start and connect to the Snowflake emulator using specific accounts. You can specify any account name when connecting to the Snowflake emulator. If you don't, all resources will be managed by the default `test` account. Depending on the Snowflake Driver you choose, you can pass `account` accordingly. ### Connect using Snowflake Connection Object If the Snowflake driver provides a connection object, you can pass the `account` parameter in the connection object. Example using the Snowflake Connector for Python: ```python showLineNumbers sf_conn_obj = sf.connect( account="your_account", # other parameters ) ``` Example using the NodeJS Driver for Snowflake: ```javascript showLineNumbers var connection = snowflake.createConnection({ account: "your_account", // other parameters }); ``` ### Connect using Connection String You can also specify the account for Snowflake drivers that let you connect with a connection string. Example establishing a JDBC connection: ```text jdbc:snowflake://snowflake.localhost.localstack.cloud:4566/?account=your_account ``` ### Check Current Account Once successfully connected, you can verify which account you are connected to by executing the following SQL command: ```sql SELECT CURRENT_ACCOUNT_NAME(); ``` The query statement will return the name of the account you are currently connected to. Output should look like: ```sql +------------------------------------------+ | CURRENT_ACCOUNT_NAME() | |------------------------------------------| | YOUR_ACCOUNT | +------------------------------------------+ ``` # API Integrations > Get started with API Integrations in LocalStack for Snowflake ## Introduction API integrations in Snowflake provide a secure way to configure trust between Snowflake and external cloud providers such as AWS API Gateway. They are typically used when creating external functions or API-based workflows. The LocalStack Snowflake emulator now supports basic **CRUD operations** for API integrations, which are currently mocked and not functional. This is useful for testing Terraform configurations or other automation flows that depend on these objects. Currently, this feature is partially mocked and designed primarily to unblock end-to-end test coverage. Behavior may not fully reflect production Snowflake semantics. ## Getting started This guide assumes you already have the Snowflake emulator running and a SQL client connected. You can manage API integrations using standard SQL statements such as `CREATE API INTEGRATION`, `ALTER API INTEGRATION`, and others. ## Create, alter, and drop an API integration ### Create an API integration You can create a new API integration using the `CREATE API INTEGRATION` command: ```sql CREATE API INTEGRATION my_integration API_PROVIDER = aws_api_gateway API_AWS_ROLE_ARN = 'arn:aws:iam::000000000000:role/r1' API_ALLOWED_PREFIXES = ('https://xyz.execute-api.us-east-1.amazonaws.com/test') ENABLED = TRUE; ``` ### Show integrations You can list all existing API integrations with: ```sql SHOW API INTEGRATIONS; ``` ### Describe integration You can inspect the details of an integration using: ```sql DESCRIBE API INTEGRATION my_integration; ``` ### Alter an integration You can modify an existing API integration, for example disabling it: ```sql ALTER API INTEGRATION my_integration SET ENABLED = FALSE; ``` ### Drop an integration You can remove an integration with: ```sql DROP API INTEGRATION my_integration; ``` # Authentication > Get started with authentication in Snowflake ## Introduction Snowflake supports [multiple authentication methods](https://docs.snowflake.com/en/user-guide/authentication-policies). The Snowflake emulator supports the following authentication methods: * Username and password * RSA key pair authentication This guide demonstrates how to use the Snowflake emulator to authenticate using both methods. ## Username and password To authenticate using a username and password, you can set the `user` and `password` parameters in the connection string. The values for these parameters can be set to `test` in the Snowflake emulator. Since the Snowflake emulator is a local instance, the username and password can be the same, and the authentication mechanism is mocked. Here's an example of how to connect to the Snowflake emulator using a username and password in a Python script: ```python showLineNumbers import snowflake.connector as sf sf_conn_obj = sf.connect( user="test", password="test", account="test", database="test", host="snowflake.localhost.localstack.cloud", ) ``` The default username and password are set to `test` and can be changed using `SF_DEFAULT_USER` and `SF_DEFAULT_PASSWORD` when starting the Snowflake emulator. :::note It is not recommended to use your production credentials in the Snowflake emulator. ::: ## RSA key pair authentication The Snowflake emulator supports RSA key-based authentication, allowing users to log in without a password by using a private key and a configured public key. To enable this, create a private key and a public key pair. ```bash openssl genrsa -out private_key.pem 2048 openssl rsa -in private_key.pem -pubout -out public_key.pem ``` Then set a public key for the user: ```sql ALTER USER your_user_name SET RSA_PUBLIC_KEY=''; ``` Then authenticate with the private key using the Snowflake client: ```python showLineNumbers import snowflake.connector conn = snowflake.connector.connect( user='your_user_name', account='your_account_identifier', private_key_file='/path/to/private_key.pem', # Add other parameters as needed ) ``` :::note The Snowflake emulator does not validate key contents—RSA authentication is mocked for local testing only. ::: # Clones > Get started with Clones in LocalStack for Snowflake ## Introduction Cloning in Snowflake allows you to create a quick, zero-copy duplicate of an existing database, schema, or table. This feature enables users to replicate data structures and content for testing or development without duplicating the underlying storage. The Snowflake emulator supports database cloning, enabling you to create quick duplicates of databases, schemas, or tables. Currently, [`CREATE ... CLONE`](https://docs.snowflake.com/en/sql-reference/sql/create-clone) is supported by LocalStack. ## Getting started This guide assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client to execute the queries below. The following sections guide you through creating a database, inserting data into a table, and then cloning the database to verify data integrity. ### Create a Database The following SQL snippet demonstrates how to create a database named `test_db`. ```sql CREATE DATABASE test_db;` ``` The expected output is: ```sql +--------------------------------------------+ | status | |--------------------------------------------| | Database TEST_DB successfully created. | +--------------------------------------------+ 0 Row(s) produced. Time Elapsed: 0.123s ``` ### Create a Table Once the database is created, you can create a table within it. The following SQL statement demonstrates how to create a table named `test_table` with columns for `id` and `name`. ```sql CREATE TABLE test_table (id INT, name TEXT);` ``` The expected output is: ```sql +----------------------------------------+ | status | |----------------------------------------| | Table TEST_TABLE successfully created. | +----------------------------------------+ 0 Row(s) produced. Time Elapsed: 0.067s ``` ### Insert Data To insert data into the `test_table`, use the `INSERT INTO` statement. This example inserts a single row with the values `(1, 'test')`. ```sql INSERT INTO test_table VALUES (1, 'test'); ``` The expected output is: ```sql +----------------------------------------+ | status | |----------------------------------------| | 1 Row(s) inserted. | +----------------------------------------+ 1 Row(s) produced. Time Elapsed: 0.024s ``` ### Create a Clone With data now in `test_table`, you can create a clone of the entire `test_db` database. This will produce a new database, `test_db_clone`, containing all objects and data from the original `test_db`. ```sql CREATE DATABASE test_db_clone CLONE test_db; ``` The expected output is: ```sql +--------------------------------------------+ | status | |--------------------------------------------| | Database TEST_DB_CLONE successfully created. | +--------------------------------------------+ 0 Row(s) produced. Time Elapsed: 0.101s ``` ### Verify Data in Clone To confirm that the data has been cloned, query the `test_table` in the `test_db_clone` database. ```sql SELECT * FROM test_db_clone.test_table; ``` The expected output is: ```sql +----+------+ | id | name | |----|------| | 1 | test | +----+------+ 1 Row(s) produced. Time Elapsed: 0.012s ``` # Compute Pools > Get started with Compute Pools in LocalStack for Snowflake ## Introduction Compute Pools in Snowflake are account-level collections of virtual machine nodes. These pools automatically scale between configurable minimum and maximum node limits based on demand. They function as the foundational infrastructure for containerized data applications within Snowflake's ecosystem, similar to virtual warehouses. The Snowflake emulator provides a CRUD (Create, Read, Update, Delete) interface for Compute Pools, allowing you to mock the creation and management of Compute Pools in your local environment. ## Getting started This guide is designed for users new to Compute Pools and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client in order to execute the queries further below. In this guide, you will create a Compute Pool, display the Compute Pool details, alter the Compute Pool configuration, and drop the Compute Pool. ### Create a Compute Pool You can create a Compute Pool using the `CREATE COMPUTE POOL` statement. In this example, you can create a Compute Pool called `my_compute_pool`: ```sql CREATE COMPUTE POOL my_compute_pool MIN_NODES = 1 MAX_NODES = 1 INSTANCE_FAMILY = CPU_X64_XS; ``` ### Describe Compute Pool You can view detailed information about a Compute Pool using the `DESCRIBE COMPUTE POOL` statement: ```sql DESCRIBE COMPUTE POOL my_compute_pool; ``` The output should be: ```sql name |state |min_nodes|max_nodes|instance_family|num_services|num_jobs|auto_suspend_secs|auto_resume|active_nodes|idle_nodes|target_nodes|created_on |resumed_on |updated_on |owner |comment|is_exclusive|application|error_code|status_message | ---------------+--------+---------+---------+---------------+------------+--------+-----------------+-----------+------------+----------+------------+-----------------------+-----------------------+-----------------------+------+-------+------------+-----------+----------+-------------------------------------------+ MY_COMPUTE_POOL|STARTING| 1| 1|CPU_X64_XS | 0| 0| 3600|true | 0| 0| 1|1970-01-01 05:30:00.000|1970-01-01 05:30:00.000|1970-01-01 05:30:00.000|PUBLIC| |false | | |Compute pool is starting for last 0 minutes| ``` ### Show Compute Pools You can display the Compute Pools using the `SHOW COMPUTE POOLS` statement: ```sql SHOW COMPUTE POOLS LIKE 'my_compute_pool'; ``` The output should be: ```sql name |state |min_nodes|max_nodes|instance_family|num_services|num_jobs|auto_suspend_secs|auto_resume|active_nodes|idle_nodes|target_nodes|created_on |resumed_on |updated_on |owner |comment|is_exclusive|application| ---------------+--------+---------+---------+---------------+------------+--------+-----------------+-----------+------------+----------+------------+-----------------------+-----------------------+-----------------------+------+-------+------------+-----------+ MY_COMPUTE_POOL|STARTING| 1| 1|CPU_X64_XS | 0| 0| 3600|true | 0| 0| 1|1970-01-01 05:30:00.000|1970-01-01 05:30:00.000|1970-01-01 05:30:00.000|PUBLIC| |false | | ``` ### Alter Compute Pool You can modify the configuration of an existing Compute Pool using the `ALTER COMPUTE POOL` statement. In this example, you can increase the maximum number of nodes: ```sql ALTER COMPUTE POOL my_compute_pool SET MAX_NODES = 2; ``` You can verify the change by describing the Compute Pool again: ```sql DESCRIBE COMPUTE POOL my_compute_pool; ``` ### Drop Compute Pool You can drop the Compute Pool using the `DROP COMPUTE POOL` statement: ```sql DROP COMPUTE POOL my_compute_pool; ``` # Cross-Database Resource Sharing > Get started with cross-database resource sharing in the Snowflake emulator ## Introduction Snowflake data providers can easily share data from various databases using secure views. These views can include schemas, tables, and other views from one or more databases, as long as they're part of the same account. The Snowflake emulator supports cross-database resource sharing, allowing you to share a secure view that references objects from multiple databases. This guide walks you through the process of creating databases, schemas, tables, and views, and sharing them with other databases. ## Getting started This guide is designed for users new to cross-database resource sharing and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to the Snowflake emulator using an SQL client. In this guide, we'll walk through a series of Snowflake SQL statements to create databases, schemas, tables, views, and a share. ### Create databases Create three databases to represent the three different organizations that will share resources. In this example, we'll create databases for `db_name1`, `db_name2`, and `db_name3`. ```sql showLineNumbers CREATE DATABASE db_name1_actual; CREATE DATABASE db_name2_actual; CREATE DATABASE db_name3_actual; ``` ### Create schemas Create a schema in each database to represent the shared resources. In this example, you can create a schema called `sch` in each database. ```sql showLineNumbers CREATE SCHEMA db_name1_actual.sch; CREATE SCHEMA db_name2_actual.sch; CREATE SCHEMA db_name3_actual.sch; ``` ### Create tables Create a table in each schema to represent the shared resources. In this example, you can create a table called `table1` in `db_name1_actual.sch`, `table2` in `db_name2_actual.sch`, and `table3` in `db_name3_actual.sch`. ```sql showLineNumbers CREATE TABLE db_name1_actual.sch.table1 (id INT); CREATE TABLE db_name2_actual.sch.table2 (id INT); CREATE TABLE db_name3_actual.sch.table3 (id INT); ``` ### Insert Data into Tables You can now insert data into the tables to represent the shared resources. In this example, we'll insert a single row into each table. ```sql showLineNumbers INSERT INTO db_name1_actual.sch.table1 (id) VALUES (1); INSERT INTO db_name2_actual.sch.table2 (id) VALUES (2); INSERT INTO db_name3_actual.sch.table3 (id) VALUES (3); ``` ### Create Views You can create a view `view1` based on `table1` in `db_name1_actual`. ```sql CREATE VIEW db_name1_actual.sch.view1 AS SELECT * FROM db_name1_actual.sch.table1; ``` ### Create Secure View You can creates a secure view `view3` in `db_name3_actual.sch` by joining data from different tables. ```sql showLineNumbers CREATE SECURE VIEW db_name3_actual.sch.view3 AS SELECT view1.id AS View1Id, table2.id AS table2id, table3.id AS table3id FROM db_name1_actual.sch.view1 view1, db_name2_actual.sch.table2 table2, db_name3_actual.sch.table3 table3; ``` ### Create Share and Grant Permissions You can create a share `s_actual` and grant usage permissions on the `db_name3_actual` database and its schema. ```sql showLineNumbers CREATE SHARE s_actual; GRANT USAGE ON DATABASE db_name3_actual TO SHARE s_actual; GRANT USAGE ON SCHEMA db_name3_actual.sch TO SHARE s_actual; ``` ### Query Data from Secure View You can now query data from the secure view `view3` in `db_name3_actual.sch`. ```sql SELECT * FROM db_name3_actual.sch.view3; ``` The expected output is: ```plaintext (1, 2, 3) ``` # Data Metric Functions > Get started with Data Metric Functions in LocalStack for Snowflake ## Introduction Snowflake [Data Metric Functions (DMFs)](https://docs.snowflake.com/en/user-guide/data-quality-intro) lets you monitor the freshness, completeness, and quality of your data by attaching system or user-defined metrics to tables and columns. LocalStack for Snowflake supports you to add a metric schedule to table (enable functions in a table), attach DMFs to table column, run the DMFs manually, and get results from DMFs. ## Getting started This guide is designed for users new to Data Metric Functions and assumes basic knowledge of SQL and Snowflake. Start LocalStack for Snowflake and connect to it using a SQL client in order to execute the queries further below. In this guide, you will learn how to: - Define system/user metrics like `COUNT`, `UNIQUE`, `NULL`, or `DUPLICATE` - Schedule those metrics on tables - Attach metrics to columns - Run them manually or on a schedule - Query results from the DMF state table ### 1. Create a Data Metric Function Run the following query to create your Data Metric Function: ```sql CREATE OR REPLACE FUNCTION check_values(ARG_T TABLE(c1 STRING)) RETURNS NUMBER AS $$ SELECT COUNT(*) FROM ARG_T WHERE c1 IS NULL $$; ``` ### 2. Create a Table Run the following query to create your table: ``` sql CREATE TABLE customers ( id NUMBER, name STRING, email STRING ); ``` ### 3. Add a Metric Schedule to a Table Run the following query to add a metric to your table: ```sql ALTER TABLE customers SET DATA_METRIC_SCHEDULE = 'TRIGGER_ON_CHANGES'; ``` ### 4. Attach a DMF to a Column Run the following query to attach Data Metric Functions to a specific column in your table: ``` sql ALTER TABLE customers ADD DATA METRIC FUNCTION check_values ON (email); ``` ### 5. Run a DMF Manually Run the following query to manually run Data Metric Functions in your table: ``` sql SELECT check_values(SELECT * FROM customers); ``` ### 6. Query Results from the DMF Run the following query to see the results of the Data Metric Functions in your table: ``` sql SELECT * FROM SNOWFLAKE.LOCAL.DATA_QUALITY_MONITORING_RESULTS WHERE TABLE_NAME = 'CUSTOMERS'; ``` # Dynamic Tables > Get started with Dynamic Tables in LocalStack for Snowflake ## Introduction Snowflake Dynamic Tables enable a background process to continuously load new data from sources into the table, supporting both delta and full load operations. A dynamic table automatically updates to reflect query results, removing the need for a separate target table and custom code for data transformation. This table is kept current through regularly scheduled refreshes by an automated process. The Snowflake emulator supports Dynamic Tables, allowing you to create and manage Dynamic Tables locally. In the current emulator implementation, Dynamic Tables are backed by views and are always current. `TARGET_LAG` and refresh scheduling have no effect. Full staleness and refresh-timing support is planned for the future. ## Getting started This guide is designed for users new to Dynamic Tables and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client in order to execute the queries further below. In this guide, you will create a table, create a dynamic table, insert data into the table, and query the dynamic table. ### Create a table You can create a table using the `CREATE TABLE` statement. Run the following query to create a table: ```sql CREATE TABLE example_table_name (id int, name text); ``` The output should be: ```sql +-----------------------------------------------+ | status | | ----------------------------------------------+ | Table EXAMPLE_TABLE_NAME successfully created.| +-----------------------------------------------+ ``` ### Create a dynamic table You can create a dynamic table using the `CREATE DYNAMIC TABLE` statement. Run the following query to create a dynamic table: ```sql showLineNumbers CREATE OR REPLACE DYNAMIC TABLE t_12345 TARGET_LAG = '1 minute' WAREHOUSE = 'test' REFRESH_MODE = auto INITIALIZE = on_create AS SELECT id, name FROM example_table_name; ``` The output should be: ```sql +-----------------------------------------------+ | result | | ----------------------------------------------+ Dynamic table T_12345 successfully created. | +-----------------------------------------------+ ``` ### Insert data into the table You can insert data into the table using the `INSERT INTO` statement. Run the following query to insert data into the table: ```sql INSERT INTO example_table_name(id, name) VALUES (1, 'foo'), (2, 'bar'); ``` The output should be: ```sql | count | | -----+ | | 2 | ``` ### Query the dynamic table You can query the dynamic table using the `SELECT` statement. Run the following query to query the dynamic table: ```sql SELECT * FROM t_12345; ``` The output should be: ```sql +----+------+ | ID | NAME | | ---+------+ | 1 | foo | | 2 | bar | +----+------+ ``` ## Dynamic Iceberg Tables Dynamic Iceberg Tables combine the auto-refresh capabilities of Dynamic Tables with the Apache Iceberg open table format. This allows you to create materialized views that automatically update from source queries while storing data in Iceberg format on external object storage (such as S3). A Dynamic Iceberg Table consists of three key concepts: - **Dynamic table**: A materialized view that auto-refreshes from a source query on a schedule (controlled by `TARGET_LAG` and `WAREHOUSE`). - **Iceberg table**: A table stored in Apache Iceberg format on external object storage (configured via `EXTERNAL_VOLUME`, `CATALOG`, and `BASE_LOCATION`). - **External volume**: A Snowflake object that references an S3 bucket as the backing storage for Iceberg data files. ### Create an S3 bucket Create a local S3 bucket using the `mb` command with `lstk aws`: ```bash lstk aws s3 mb s3://test-bucket ``` ### Create an external volume Create an external volume to define the storage location for the Iceberg data files: ```sql showLineNumbers CREATE OR REPLACE EXTERNAL VOLUME test_volume STORAGE_LOCATIONS = ( ( NAME = 'aws-s3-test' STORAGE_PROVIDER = 'S3' STORAGE_BASE_URL = 's3://test-bucket/' STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::000000000000:role/s3-role' ) ) ``` ### Create a source table Create a source table that the Dynamic Iceberg Table will refresh from: ```sql CREATE TABLE source_table (id INT, name TEXT); INSERT INTO source_table(id, name) VALUES (1, 'foo'), (2, 'bar'); ``` ### Create a Dynamic Iceberg Table Create a Dynamic Iceberg Table using the `CREATE DYNAMIC ICEBERG TABLE` statement: ```sql showLineNumbers CREATE DYNAMIC ICEBERG TABLE my_dynamic_iceberg_table TARGET_LAG = '2 minutes' WAREHOUSE = test REFRESH_MODE = INCREMENTAL INITIALIZE = on_create EXTERNAL_VOLUME = 'test_volume' CATALOG = 'SNOWFLAKE' BASE_LOCATION = 'my_table_data' AS SELECT id, name FROM source_table; ``` The output should be: ```sql +----------------------------------------------------------+ | result | |----------------------------------------------------------+ | Dynamic table MY_DYNAMIC_ICEBERG_TABLE successfully created. | +----------------------------------------------------------+ ``` ### Query the Dynamic Iceberg Table You can query the Dynamic Iceberg Table using a standard `SELECT` statement: ```sql SELECT * FROM my_dynamic_iceberg_table ORDER BY id; ``` The output should be: ```sql +----+------+ | ID | NAME | |----+------+ | 1 | foo | | 2 | bar | +----+------+ ``` ### Show Dynamic Tables You can view the metadata of your Dynamic Iceberg Table using the `SHOW DYNAMIC TABLES` command. The `is_iceberg` column indicates whether the table is a Dynamic Iceberg Table: ```sql SHOW DYNAMIC TABLES LIKE 'my_dynamic_iceberg_table'; ``` ### Rename a Dynamic Iceberg Table You can rename a Dynamic Iceberg Table using the `ALTER DYNAMIC TABLE` statement: ```sql ALTER DYNAMIC TABLE my_dynamic_iceberg_table RENAME TO my_renamed_table; ``` ### Drop a Dynamic Iceberg Table You can drop a Dynamic Iceberg Table using the `DROP DYNAMIC TABLE` statement: ```sql DROP DYNAMIC TABLE my_renamed_table; ``` The output should be: ```sql +------------------------------------------+ | status | |------------------------------------------+ | MY_RENAMED_TABLE successfully dropped. | +------------------------------------------+ ``` ## Current Limitations The following limitations apply to Dynamic Tables in the current Snowflake emulator: - **Always-current refresh behavior**: Dynamic Tables are backed by views in the current implementation, meaning they always reflect the latest state of their source query. As a result, `TARGET_LAG` and `REFRESH_MODE` settings are accepted but have no effect on refresh timing or staleness behavior. # Glue Iceberg REST Catalog > Get started with Glue Iceberg REST Catalog in LocalStack for Snowflake ## Introduction [AWS Glue](https://docs.aws.amazon.com/glue/) exposes an [Iceberg REST endpoint](https://docs.aws.amazon.com/glue/latest/dg/connect-glu-iceberg-rest.html) that lets external query engines read and write Iceberg tables managed in the Glue Data Catalog. When the Glue catalog is federated to [S3 Tables](/aws/services/s3tables/), the REST endpoint serves Iceberg tables stored in S3 Tables buckets. The Snowflake emulator can connect to this Glue Iceberg REST endpoint through a `CATALOG INTEGRATION` of source `ICEBERG_REST` with `CATALOG_API_TYPE = AWS_GLUE`. The integration uses AWS SigV4 to sign catalog requests and `VENDED_CREDENTIALS` to obtain scoped credentials for the underlying S3 Tables data files, so you can query the same tables from Snowflake and from PyIceberg without duplicating data. ## Getting started This guide walks through creating an Iceberg table in S3 Tables through the Glue Iceberg REST endpoint, registering that table with the Snowflake emulator through a Glue catalog integration, and querying it with SQL. It assumes basic knowledge of the AWS CLI, our [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command, and Snowflake. In this guide, you will: - Create an S3 Tables bucket and namespace - Register a federated `s3tablescatalog` in Glue - Create and populate an Iceberg table through the Glue Iceberg REST endpoint with PyIceberg - Create a Snowflake catalog integration that points at the Glue REST endpoint - Create an Iceberg table in Snowflake that references the remote Glue table and query it Start your Snowflake emulator and connect to it using an SQL client in order to execute the queries below. Make sure to install Python and `pyiceberg[s3fs,pyarrow]` packages before starting. ### Create an S3 Tables bucket and namespace The Glue Iceberg REST endpoint serves tables stored in S3 Tables. Create a table bucket: ```bash lstk aws s3tables create-table-bucket --name my-table-bucket ``` ```bash title="Output" { "arn": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket" } ``` Now create a namespace to hold the Iceberg table: ```bash lstk aws s3tables create-namespace \ --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket \ --namespace my_namespace ``` ```bash title="Output" { "tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket", "namespace": [ "my_namespace" ] } ``` ### Register the Glue federated catalog Glue exposes S3 Tables buckets through a federated catalog. Register a catalog named `s3tablescatalog` that federates to all S3 Tables buckets in the account: ```bash showLineNumbers lstk aws glue create-catalog \ --name s3tablescatalog \ --catalog-input '{ "FederatedCatalog": { "Identifier": "arn:aws:s3tables:us-east-1:000000000000:bucket/*", "ConnectionName": "aws:s3tables" }, "CreateTableDefaultPermissions": [], "CreateDatabaseDefaultPermissions": [] }' ``` Confirm the catalog was registered: ```bash lstk aws glue get-catalogs ``` The response includes a `CatalogList` entry with `Name: s3tablescatalog` and a `FederatedCatalog` block pointing at S3 Tables. Snowflake will reference this catalog through its `WAREHOUSE` identifier in the form `:s3tablescatalog/`. ### Create and populate the table Use PyIceberg to talk to the Glue Iceberg REST endpoint at `http://glue.localhost.localstack.cloud:4566/iceberg`. The same endpoint is later used by the Snowflake catalog integration, so creating the table through PyIceberg first lets you confirm that signing and federation are configured correctly. Save the script as `setup_glue_iceberg.py`: ```python showLineNumbers import pyarrow as pa from pyiceberg.catalog.rest import RestCatalog from pyiceberg.schema import Schema from pyiceberg.types import NestedField, StringType, LongType LOCALSTACK_URL = "http://localhost.localstack.cloud:4566" GLUE_URL = "http://glue.localhost.localstack.cloud:4566" ACCOUNT_ID = "000000000000" TABLE_BUCKET_NAME = "my-table-bucket" NAMESPACE = "my_namespace" TABLE_NAME = "customer_orders" REGION = "us-east-1" catalog = RestCatalog( name="glue_catalog", uri=f"{GLUE_URL}/iceberg", warehouse=f"{ACCOUNT_ID}:s3tablescatalog/{TABLE_BUCKET_NAME}", **{ "s3.region": REGION, "s3.endpoint": LOCALSTACK_URL, "client.access-key-id": ACCOUNT_ID, "client.secret-access-key": "test", "rest.sigv4-enabled": "true", "rest.signing-name": "glue", "rest.signing-region": REGION, }, ) schema = Schema( NestedField(field_id=1, name="order_id", field_type=StringType(), required=False), NestedField(field_id=2, name="customer_name", field_type=StringType(), required=False), NestedField(field_id=3, name="amount", field_type=LongType(), required=False), ) catalog.create_table(identifier=(NAMESPACE, TABLE_NAME), schema=schema) table = catalog.load_table((NAMESPACE, TABLE_NAME)) table.append(pa.table({ "order_id": ["ORD001", "ORD002", "ORD003"], "customer_name": ["Alice", "Bob", "Charlie"], "amount": [100, 250, 175], })) print(f"Tables in {NAMESPACE}: {catalog.list_tables(NAMESPACE)}") ``` Run the script: ```bash python setup_glue_iceberg.py ``` ```bash title="Output" Tables in my_namespace: [('my_namespace', 'customer_orders')] ``` ### Create the catalog integration Connect to the Snowflake emulator with your SQL client of choice and create the catalog integration. The `REST_CONFIG` block declares the Glue REST endpoint and warehouse, and the `REST_AUTHENTICATION` block configures AWS SigV4 signing against the `glue` service. ```sql showLineNumbers CREATE OR REPLACE CATALOG INTEGRATION glue_rest_catalog_int CATALOG_SOURCE = ICEBERG_REST TABLE_FORMAT = ICEBERG CATALOG_NAMESPACE = 'my_namespace' REST_CONFIG = ( CATALOG_URI = 'http://glue.localhost.localstack.cloud:4566/iceberg' CATALOG_API_TYPE = AWS_GLUE WAREHOUSE = '000000000000:s3tablescatalog/my-table-bucket' ACCESS_DELEGATION_MODE = VENDED_CREDENTIALS ) REST_AUTHENTICATION = ( TYPE = AWS_SIGV4 AWS_ACCESS_KEY_ID = '000000000000' AWS_SECRET_ACCESS_KEY = 'test' AWS_REGION = 'us-east-1' AWS_SERVICE = 'glue' ) ENABLED = TRUE REFRESH_INTERVAL_SECONDS = 60 COMMENT = 'Glue Iceberg REST catalog integration'; ``` The key fields are: - `CATALOG_API_TYPE = AWS_GLUE` selects the Glue dialect of the Iceberg REST protocol. This skips the `/api/catalog` suffix that Polaris-style endpoints expect. - `WAREHOUSE` is the Glue catalog identifier in the form `:s3tablescatalog/` and points at the federated catalog created above. - `ACCESS_DELEGATION_MODE = VENDED_CREDENTIALS` instructs the catalog to return scoped S3 credentials so Snowflake can read the underlying data files without a separate external volume. - `REST_AUTHENTICATION` uses `AWS_SIGV4` with `AWS_SERVICE = 'glue'`. `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` accept any LocalStack-compatible credentials. You can verify the integration with: ```sql SHOW CATALOG INTEGRATIONS; ``` ### Create an Iceberg table referencing the Glue catalog Reference the existing Glue table by its fully-qualified name and let Snowflake infer the schema from the catalog metadata: ```sql showLineNumbers CREATE OR REPLACE ICEBERG TABLE iceberg_customer_orders CATALOG = 'glue_rest_catalog_int' CATALOG_TABLE_NAME = 'my_namespace.customer_orders' AUTO_REFRESH = TRUE; ``` `CATALOG_TABLE_NAME` uses the `.` format from the Glue catalog. With `AUTO_REFRESH = TRUE`, Snowflake re-reads the table metadata on the schedule defined by the integration's `REFRESH_INTERVAL_SECONDS`. ### Query the table Query the table like any other Snowflake table: ```sql SELECT * FROM iceberg_customer_orders; ``` ```sql title="Output" +----------+---------------+--------+ | ORDER_ID | CUSTOMER_NAME | AMOUNT | +----------+---------------+--------+ | ORD001 | Alice | 100 | | ORD002 | Bob | 250 | | ORD003 | Charlie | 175 | +----------+---------------+--------+ ``` Rows appended through PyIceberg are visible to Snowflake on the next metadata refresh, and any further changes you make on the Glue side propagate through the same `glue_rest_catalog_int` integration. # Hybrid Tables > Get started with Hybrid Tables in LocalStack for Snowflake ## Introduction Snowflake Hybrid tables, also known as Unistore hybrid tables, support fast, single-row operations by enforcing unique constraints for required primary keys and including indexes to speed up data retrieval. These tables are designed to optimize support for both analytical and transactional workloads simultaneously, underpinning Snowflake's Unistore architecture. The Snowflake emulator supports Hybrid tables, allowing you to create and manage Hybrid tables locally. ## Getting started This guide is designed for users new to Hybird tables and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client in order to execute the queries further below. In this guide, you will create a Hybrid table, display the Hybrid tables, and drop the Hybrid table. ### Create a Hybrid table You can create a Hybrid table using the `CREATE HYBRID TABLE` statement. In this example, you can create a Hybrid table called `test-table`: ```sql CREATE HYBRID TABLE "test-table"(id int, name TEXT, PRIMARY KEY(id)); ``` The output should be: ```sql +------------------------------------------+ | status | |------------------------------------------| | Table "test-table" successfully created. | +------------------------------------------+ ``` ### Show Hybrid tables You can display the Hybrid tables using the `SHOW HYBRID TABLES` statement: ```sql SHOW HYBRID TABLES LIKE 'test-table'; ``` The output should be: ```sql +-----------------------------------------------------------------------------------------------------+ created_on |name |database_name|schema_name|comment|rows|bytes|owner |owner_role_type| -----------------------+----------+-------------+-----------+-------+----+-----+------+---------------+ 1970-01-01 05:30:00.000|test-table|TEST |PUBLIC | |1000| 1000|PUBLIC|ROLE | +-----------------------------------------------------------------------------------------------------+ ``` ### Drop Hybrid table You can drop the Hybrid table using the `DROP HYBRID TABLE` statement: ```sql DROP TABLE "test-table"; ``` The output should be: ```sql +------------------------------------------+ | status | | -----------------------------------------+ | TEST-TABLE successfully dropped. | +------------------------------------------+ ``` # Iceberg Tables > Get started with Iceberg Tables in LocalStack for Snowflake ## Introduction Iceberg tables uses [Apache Iceberg](https://iceberg.apache.org/) open table format specification to provide an abstraction layer on data files stored in open formats. Iceberg tables for Snowflake offer schema evolution, partitioning, and snapshot isolation to manage the table data efficiently. The Snowflake emulator supports Iceberg tables, allowing you to create and manage Iceberg tables locally. You can use Iceberg tables to query data in Snowflake tables using the Iceberg table format by using external volumes, with data stored in local/remote S3 buckets. ## Getting started This guide is designed for users new to Iceberg tables and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client in order to execute the queries further below. In this guide, you will create an external volume, and an Iceberg table to store and query data using the Iceberg table format. ### Create an S3 bucket You can create a local S3 bucket using the `mb` command with `lstk aws`. ```bash lstk aws s3 mb s3://test-bucket ``` ### Create an external volume You can create an external volume using the `CREATE OR REPLACE EXTERNAL VOLUME` statement. The external volume is used to define the location of the files that Iceberg will use to store the table data. ```sql showLineNumbers CREATE OR REPLACE EXTERNAL VOLUME test_volume STORAGE_LOCATIONS = ( ( NAME = 'aws-s3-test' STORAGE_PROVIDER = 'S3' STORAGE_BASE_URL = 's3://test-bucket/' STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::000000000000:role/s3-role' ENCRYPTION=(TYPE='AWS_SSE_S3') ) ) ``` ### Grant access to Snowflake role You can grant access to the Snowflake role using the `GRANT USAGE ON EXTERNAL VOLUME` statement. ```sql GRANT USAGE ON EXTERNAL VOLUME test_volume TO ROLE PUBLIC ``` ### Create Iceberg table You can create an Iceberg table using the `CREATE ICEBERG TABLE` statement. The Iceberg table is used to define the schema and location of the table data. ```sql CREATE ICEBERG TABLE test_table (c1 TEXT) CATALOG='SNOWFLAKE', EXTERNAL_VOLUME='test_volume', BASE_LOCATION='test' ``` ### Insert and select data You can insert and select data from the Iceberg table using the `INSERT INTO` and `SELECT` statements. ```sql INSERT INTO test_table(c1) VALUES ('test'), ('foobar') SELECT * FROM test_table ``` The output should be: ```plaintext +------+ | C1 | |------| | test | | foobar | +------+ ``` You can also list the content of the S3 bucket: ```bash lstk aws s3 ls --recursive s3://test-bucket/ ``` # Masking Policies > Get started with Masking Policies in LocalStack for Snowflake ## Introduction Masking policies are schema-level objects that let you define column-level data protection rules in Snowflake. They determine how sensitive data is displayed depending on the context of the query and the role of the user. For example, a masking policy can ensure that full values are shown to administrators while obfuscating values for regular users. The Snowflake emulator in LocalStack now supports **basic CRUD operations** for masking policies, which are currently mocked and not functional. While the full integration of masking policies with table data is not yet supported, you can use these operations to experiment with policy definitions and query their metadata locally. ## Getting started Masking policies is intended for local development and testing. It is useful for validating schema migration scripts, Terraform workflows, or integration tests that reference masking policies. ## Create, alter, and drop a masking policy ### Create a masking policy You can define a masking policy using the `CREATE MASKING POLICY` statement: ```sql CREATE MASKING POLICY ssn_mask AS (val STRING) RETURNS STRING -> CASE WHEN CURRENT_ROLE() IN ('FULL_ACCESS_ROLE') THEN val ELSE 'XXX-XX-XXXX' END; ``` This policy shows the full value of a column only to users with the `FULL_ACCESS_ROLE`. All other users see a masked version. ### Alter a masking policy You can update an existing masking policy using `ALTER MASKING POLICY`: ```sql ALTER MASKING POLICY ssn_mask SET BODY -> CASE WHEN CURRENT_ROLE() IN ('FULL_ACCESS_ROLE', 'AUDITOR_ROLE') THEN val ELSE 'XXX-XX-XXXX' END; ``` This modification expands access to include the `AUDITOR_ROLE`. ### Show masking policies List existing masking policies using: ```sql SHOW MASKING POLICIES; ``` The result displays available masking policies and their properties. ### Drop a masking policy Remove a policy using: ```sql DROP MASKING POLICY ssn_mask; ``` This deletes the policy definition from the emulator. :::note ## Limitations - LocalStack currently supports only the CRUD operations (`CREATE`, `ALTER`, `SHOW`, `DROP`) for masking policies. - Applying masking policies to tables and enforcing them during queries is not supported yet. - Use this feature primarily for validating schema definitions and testing IaC workflows. ::: # Materialized Views > Get started with Materialized Views in LocalStack for Snowflake ## Introduction Materialized views are a feature of Snowflake that allows you to create a persistent view of a table. This view is pre-computed and stored in the database, allowing for faster queries and improved performance. The Snowflake emulator supports Materialized Views, allowing you to accurately test materialized view logic and behavior in local development environments. ## Getting started This guide is designed for users new to Materialized Views and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client to execute the queries below. The following sections guide you through creating materialized views, inserting data into source tables, querying from views, and performing operations like rename, describe, and drop. ### Create a materialized view To create a materialized view, use the `CREATE MATERIALIZED VIEW` statement. The following example creates a view `order_view` that selects specific columns from the `orders` table. ```sql showLineNumbers CREATE TABLE IF NOT EXISTS orders ( id INT, product TEXT, shipped BOOLEAN ); CREATE MATERIALIZED VIEW IF NOT EXISTS order_view AS SELECT id, product FROM orders; ``` ### Insert data into source table Inserting new data into the base table automatically refreshes the materialized view in the background. ```sql INSERT INTO orders(id, product, shipped) VALUES (1, 'Book', FALSE), (2, 'Pen', TRUE); ``` ### Query from materialized view You can query a materialized view just like a regular table. The view reflects the data from the source table as of its most recent refresh. ```sql SELECT * FROM order_view; ``` The output should be: ```sql ID|PRODUCT| --+-------+ 1|Book | 2|Pen | ``` ### Describe the view Use `DESCRIBE MATERIALIZED VIEW` to inspect the schema of the view, including column names and types. ```sql DESCRIBE MATERIALIZED VIEW order_view; ``` The output should be: ```sql name |type|kind |null?|default|primary key|unique key|check|expression|comment|policy name|privacy domain| -------+----+------+-----+-------+-----------+----------+-----+----------+-------+-----------+--------------+ ID |INT4|COLUMN|Y | |N |N | | | | | | PRODUCT|TEXT|COLUMN|Y | |N |N | | | | | | ``` ### Rename and drop view You can rename and drop materialized views using standard SQL statements. ```sql ALTER MATERIALIZED VIEW order_view RENAME TO order_view_new; DROP MATERIALIZED VIEW IF EXISTS order_view_new; ``` # Metadata Views > Get started with Metadata Views in LocalStack for Snowflake ## Introduction Snowflake provides metadata views and table functions to query information about database objects, schemas, tables, columns, and query history. These views are available through two schemas: - **INFORMATION_SCHEMA**: A read-only schema present in every Snowflake database, scoped to that database's objects. - **ACCOUNT_USAGE**: Available in the special `SNOWFLAKE` database, providing account-wide views that span all databases. The Snowflake emulator supports querying metadata views, allowing you to inspect the structure of your local Snowflake objects using the same SQL syntax as the Snowflake service. ## INFORMATION_SCHEMA The `INFORMATION_SCHEMA` schema contains views that return metadata about objects within the current database. You can query it using `.INFORMATION_SCHEMA.` or simply `INFORMATION_SCHEMA.` when a database context is active. ### TABLES The `TABLES` view returns metadata about tables and views in the current database. The following columns are returned: | Column | Data Type | Description | | --- | --- | --- | | TABLE_CATALOG | TEXT | Name of the database containing the table | | TABLE_SCHEMA | TEXT | Name of the schema containing the table | | TABLE_NAME | TEXT | Name of the table | | TABLE_OWNER | TEXT | Owner of the table | | TABLE_TYPE | TEXT | Type of object (BASE TABLE, VIEW, etc.) | | IS_TRANSIENT | TEXT | Whether the table is transient | | CLUSTERING_KEY | TEXT | Clustering key expression | | ROW_COUNT | INTEGER | Approximate number of rows | | BYTES | INTEGER | Approximate size in bytes | | RETENTION_TIME | INTEGER | Data retention period in days | | SELF_REFERENCING_COLUMN_NAME | TEXT | Name of the self-referencing column for typed tables | | REFERENCE_GENERATION | TEXT | How the self-referencing column value is generated | | USER_DEFINED_TYPE_CATALOG | TEXT | Database of the user-defined type for typed tables | | USER_DEFINED_TYPE_SCHEMA | TEXT | Schema of the user-defined type for typed tables | | USER_DEFINED_TYPE_NAME | TEXT | Name of the user-defined type for typed tables | | IS_INSERTABLE_INTO | TEXT | Whether rows can be inserted into the table | | IS_TYPED | TEXT | Whether this is a typed table | | COMMIT_ACTION | TEXT | Action taken on commit for temporary tables | | CREATED | TIMESTAMP_LTZ | Timestamp when the table was created | | LAST_ALTERED | TIMESTAMP_LTZ | Timestamp when the table was last modified | | LAST_DDL | TIMESTAMP_LTZ | Timestamp of the last DDL operation | | LAST_DDL_BY | TEXT | User who last performed a DDL operation | | AUTO_CLUSTERING_ON | TEXT | Whether automatic clustering is enabled | | COMMENT | TEXT | Comment on the table | | IS_TEMPORARY | TEXT | Whether the table is temporary | | IS_ICEBERG | TEXT | Whether the table is an Iceberg table | | IS_DYNAMIC | TEXT | Whether the table is a Dynamic table | | IS_IMMUTABLE | TEXT | Whether the table is immutable | | IS_HYBRID | TEXT | Whether the table is a Hybrid table | ### COLUMNS The `COLUMNS` view returns metadata about the columns of tables and views in the current database. The following columns are returned: | Column | Data Type | Description | | --- | --- | --- | | TABLE_CATALOG | TEXT | Name of the database | | TABLE_SCHEMA | TEXT | Name of the schema | | TABLE_NAME | TEXT | Name of the table | | COLUMN_NAME | TEXT | Name of the column | | ORDINAL_POSITION | INTEGER | Position of the column within the table | | COLUMN_DEFAULT | TEXT | Default value expression of the column | | IS_NULLABLE | TEXT | Whether the column allows NULL values | | DATA_TYPE | TEXT | Snowflake data type of the column | | CHARACTER_MAXIMUM_LENGTH | INTEGER | Maximum length for character data types | | CHARACTER_OCTET_LENGTH | INTEGER | Maximum octet length for character data types | | NUMERIC_PRECISION | INTEGER | Precision for numeric data types | | NUMERIC_PRECISION_RADIX | INTEGER | Radix for numeric precision | | NUMERIC_SCALE | INTEGER | Scale for numeric data types | | DATETIME_PRECISION | INTEGER | Fractional seconds precision for datetime types | | INTERVAL_TYPE | TEXT | Interval type qualifier | | INTERVAL_PRECISION | INTEGER | Interval precision | | CHARACTER_SET_CATALOG | TEXT | Not applicable in Snowflake (always NULL) | | CHARACTER_SET_SCHEMA | TEXT | Not applicable in Snowflake (always NULL) | | CHARACTER_SET_NAME | TEXT | Not applicable in Snowflake (always NULL) | | COLLATION_CATALOG | TEXT | Not applicable in Snowflake (always NULL) | | COLLATION_SCHEMA | TEXT | Not applicable in Snowflake (always NULL) | | COLLATION_NAME | TEXT | Not applicable in Snowflake (always NULL) | | DOMAIN_CATALOG | TEXT | Not applicable in Snowflake (always NULL) | | DOMAIN_SCHEMA | TEXT | Not applicable in Snowflake (always NULL) | | DOMAIN_NAME | TEXT | Not applicable in Snowflake (always NULL) | | UDT_CATALOG | TEXT | Not applicable in Snowflake (always NULL) | | UDT_SCHEMA | TEXT | Not applicable in Snowflake (always NULL) | | UDT_NAME | TEXT | Not applicable in Snowflake (always NULL) | | SCOPE_CATALOG | TEXT | Not applicable in Snowflake (always NULL) | | SCOPE_SCHEMA | TEXT | Not applicable in Snowflake (always NULL) | | SCOPE_NAME | TEXT | Not applicable in Snowflake (always NULL) | | MAXIMUM_CARDINALITY | INTEGER | Not applicable in Snowflake (always NULL) | | DTD_IDENTIFIER | TEXT | Not applicable in Snowflake (always NULL) | | IS_SELF_REFERENCING | TEXT | Whether the column is self-referencing | | IS_IDENTITY | TEXT | Whether the column is an identity column | | IDENTITY_GENERATION | TEXT | How identity values are generated | | IDENTITY_START | TEXT | Start value for identity columns | | IDENTITY_INCREMENT | TEXT | Increment for identity columns | | IDENTITY_MAXIMUM | TEXT | Maximum value for identity columns | | IDENTITY_MINIMUM | TEXT | Minimum value for identity columns | | IDENTITY_CYCLE | TEXT | Whether the identity column cycles | | IDENTITY_ORDERED | TEXT | Whether the identity column is ordered | | SCHEMA_EVOLUTION_RECORD | TEXT | Schema evolution record for the column | | COMMENT | TEXT | Comment on the column | ### SCHEMATA The `SCHEMATA` view returns metadata about schemas in the current database. The following columns are returned: | Column | Data Type | Description | | --- | --- | --- | | CATALOG_NAME | TEXT | Name of the database | | SCHEMA_NAME | TEXT | Name of the schema | | SCHEMA_OWNER | TEXT | Owner of the schema (NULL for INFORMATION_SCHEMA) | | IS_TRANSIENT | TEXT | Whether the schema is transient | | IS_MANAGED_ACCESS | TEXT | Whether managed access is enabled | | RETENTION_TIME | NUMBER | Data retention period in days | | DEFAULT_CHARACTER_SET_CATALOG | TEXT | Not applicable in Snowflake (always NULL) | | DEFAULT_CHARACTER_SET_SCHEMA | TEXT | Not applicable in Snowflake (always NULL) | | DEFAULT_CHARACTER_SET_NAME | TEXT | Not applicable in Snowflake (always NULL) | | SQL_PATH | TEXT | Not applicable in Snowflake (always NULL) | | CREATED | TIMESTAMP_LTZ | Timestamp when the schema was created | | LAST_ALTERED | TIMESTAMP_LTZ | Timestamp when the schema was last modified | | COMMENT | TEXT | Comment on the schema | | REPLICABLE_WITH_FAILOVER_GROUPS | TEXT | Whether the schema can be replicated with failover groups | | OWNER_ROLE_TYPE | TEXT | Type of role that owns the schema | ### VIEWS The `VIEWS` view returns metadata about views in the current database. The following columns are returned: | Column | Data Type | Description | | --- | --- | --- | | TABLE_CATALOG | TEXT | Name of the database | | TABLE_SCHEMA | TEXT | Name of the schema | | TABLE_NAME | TEXT | Name of the view | | TABLE_OWNER | TEXT | Owner of the view | | VIEW_DEFINITION | TEXT | Full SQL definition of the view, prefixed with `CREATE VIEW . AS` | | CHECK_OPTION | TEXT | Check option (always `NONE`) | | IS_UPDATABLE | TEXT | Whether the view is updatable | | INSERTABLE_INTO | TEXT | Whether rows can be inserted into the view | | IS_SECURE | TEXT | Whether the view is a secure view | | CREATED | TIMESTAMP_LTZ | Timestamp when the view was created | | LAST_ALTERED | TIMESTAMP_LTZ | Timestamp when the view was last modified | | LAST_DDL | TIMESTAMP_LTZ | Timestamp of the last DDL operation | | LAST_DDL_BY | TEXT | User who last performed a DDL operation | | COMMENT | TEXT | Comment on the view | ### DATABASES The `DATABASES` view returns metadata about databases accessible in the account. The following columns are returned: | Column | Data Type | Description | | --- | --- | --- | | DATABASE_NAME | TEXT | Name of the database | | DATABASE_OWNER | TEXT | Owner of the database | | IS_TRANSIENT | TEXT | Whether the database is transient | | COMMENT | TEXT | Comment on the database | | CREATED | TIMESTAMP_LTZ | Timestamp when the database was created | | LAST_ALTERED | TIMESTAMP_LTZ | Timestamp when the database was last modified | | RETENTION_TIME | NUMBER | Data retention period in days | | TYPE | TEXT | Database type (e.g., `STANDARD`) | | REPLICABLE_WITH_FAILOVER_GROUPS | TEXT | Whether the database can be replicated with failover groups | | OWNER_ROLE_TYPE | TEXT | Type of role that owns the database | ### PROCEDURES The `PROCEDURES` view returns metadata about user-defined stored procedures in the current database. The following columns are returned: | Column | Data Type | Description | | --- | --- | --- | | PROCEDURE_CATALOG | TEXT | Name of the database containing the procedure | | PROCEDURE_SCHEMA | TEXT | Name of the schema containing the procedure | | PROCEDURE_NAME | TEXT | Name of the procedure | | PROCEDURE_OWNER | TEXT | Role that owns the procedure | | ARGUMENT_SIGNATURE | TEXT | Snowflake-style argument signature, e.g. `(INPUT_VAR VARCHAR, SCORE FLOAT)` | | DATA_TYPE | TEXT | Return type of the procedure | | CHARACTER_MAXIMUM_LENGTH | INTEGER | Maximum length when the return type is `VARCHAR` | | CHARACTER_OCTET_LENGTH | INTEGER | Maximum octet length when the return type is `VARCHAR` | | NUMERIC_PRECISION | INTEGER | Precision when the return type is `NUMBER` | | NUMERIC_PRECISION_RADIX | INTEGER | Radix used to express precision (always `10` for `NUMBER`) | | NUMERIC_SCALE | INTEGER | Scale when the return type is `NUMBER` | | PROCEDURE_LANGUAGE | TEXT | Language the procedure is written in (e.g., `SQL`, `JAVASCRIPT`, `PYTHON`) | | PROCEDURE_DEFINITION | TEXT | Body of the procedure as provided at creation time | | CREATED | TIMESTAMP_LTZ | Timestamp when the procedure was created | | LAST_ALTERED | TIMESTAMP_LTZ | Timestamp when the procedure was last modified | | COMMENT | TEXT | Comment on the procedure | | EXTERNAL_ACCESS_INTEGRATIONS | TEXT | External access integrations referenced by the procedure | | SECRETS | TEXT | Secrets referenced by the procedure | | RUNTIME_VERSION | TEXT | Runtime version of the procedure | | PACKAGES | TEXT | Packages referenced by the procedure | | INSTALLED_PACKAGES | TEXT | Packages installed for the procedure | The view only returns procedures and skips functions with the same name. ## INFORMATION_SCHEMA Table Functions In addition to views, `INFORMATION_SCHEMA` provides table functions that return tabular results. These are invoked using the `TABLE()` syntax. ### QUERY_HISTORY The `QUERY_HISTORY` table function returns the recent query execution history for the current account. The function accepts the following parameters: | Parameter | Type | Default | Description | | --- | --- | --- | --- | | RESULT_LIMIT | NUMBER | 100 | Maximum number of rows to return (range: 1–10000) | | END_TIME_RANGE_START | TIMESTAMP_LTZ | NULL | Return queries whose end time is at or after this value | | END_TIME_RANGE_END | TIMESTAMP_LTZ | NULL | Return queries whose end time is before this value | | INCLUDE_CLIENT_GENERATED_STATEMENT | BOOLEAN | NULL | Whether to include client-generated statements (while an accepted parameter, it's not available yet in the emulator, so it defaults to False) | :::note The `END_TIME_RANGE_START` and `END_TIME_RANGE_END` parameters are accepted but time-based filtering is not yet applied in the emulator. ::: ### QUERY_HISTORY_BY_USER The `QUERY_HISTORY_BY_USER` table function returns the recent query execution history for a specific user. It returns the same columns as `QUERY_HISTORY`. The function accepts the following parameters: | Parameter | Type | Default | Description | | --- | --- | --- | --- | | USER_NAME | VARCHAR | NULL | User whose history to return; defaults to the current session user | | END_TIME_RANGE_START | TIMESTAMP_LTZ | NULL | Return queries whose end time is at or after this value | | END_TIME_RANGE_END | TIMESTAMP_LTZ | NULL | Return queries whose end time is before this value | | RESULT_LIMIT | NUMBER | 100 | Maximum number of rows to return (range: 1–10000) | | INCLUDE_CLIENT_GENERATED_STATEMENT | BOOLEAN | NULL | Whether to include client-generated statements | ### TAG_REFERENCES The `TAG_REFERENCES` table function returns all tag assignments for a specific object within the current database. The function accepts the following positional parameters: | Parameter | Type | Description | | --- | --- | --- | | OBJECT_NAME | VARCHAR | Name of the object to look up tag assignments for | | OBJECT_DOMAIN | VARCHAR | Domain of the object (e.g., `table`, `column`, `schema`, `database`) | ## ACCOUNT_USAGE The `ACCOUNT_USAGE` schema is available in the special `SNOWFLAKE` database and provides views with account-wide visibility across all databases. Query it using `SNOWFLAKE.ACCOUNT_USAGE.`. You can also use the `TABLES` view to audit metadata for all tables in your account, `account_usage.tables`. ### TAG_REFERENCES The `ACCOUNT_USAGE.TAG_REFERENCES` view returns all tag-to-object associations across the entire account. Unlike the `INFORMATION_SCHEMA.TAG_REFERENCES()` table function, this view returns every tag assignment without requiring you to specify a particular object. The following columns are returned: | Column | Data Type | Description | | --- | --- | --- | | TAG_DATABASE | TEXT | Name of the database containing the tag | | TAG_SCHEMA | TEXT | Name of the schema containing the tag | | TAG_ID | NUMBER | Internal identifier of the tag | | TAG_NAME | TEXT | Name of the tag | | TAG_VALUE | TEXT | Value assigned to the tag | | OBJECT_DATABASE | TEXT | Database of the tagged object (NULL for `DATABASE`, `ROLE`, and `WAREHOUSE` domains) | | OBJECT_SCHEMA | TEXT | Schema of the tagged object (NULL for `DATABASE`, `SCHEMA`, `ROLE`, and `WAREHOUSE` domains) | | OBJECT_ID | NUMBER | Internal identifier of the tagged object | | OBJECT_NAME | TEXT | Name of the tagged object | | OBJECT_DELETED | TIMESTAMP_LTZ | Timestamp when the tagged object was deleted, if applicable | | DOMAIN | TEXT | Type of the tagged object (`TABLE`, `COLUMN`, `SCHEMA`, `DATABASE`, `ROLE`, `WAREHOUSE`, etc.) | | COLUMN_ID | TEXT | Internal identifier of the tagged column | | COLUMN_NAME | TEXT | Name of the tagged column when `DOMAIN` is `COLUMN`; otherwise NULL | | APPLY_METHOD | TEXT | Method by which the tag was applied | For more information on Snowflake metadata views, refer to the [Snowflake INFORMATION_SCHEMA documentation](https://docs.snowflake.com/en/sql-reference/info-schema) and the [Snowflake ACCOUNT_USAGE documentation](https://docs.snowflake.com/en/sql-reference/account-usage). # Native Apps > Get started with Native Apps in LocalStack for Snowflake ## Introduction Snowflake Native Apps are applications built and executed directly within the Snowflake Data Cloud platform. These apps can be used to extend the capabilities of Snowflake by integrating with external services, automating workflows, and building custom data applications. These apps are developed using Snowflake-native tools (e.g., Snowflake SQL, Snowflake API, and JavaScript) and can be distributed on the Snowflake Marketplace. The Snowflake emulator supports creating & deploying Native Apps locally with the same statements as the Snowflake service. ## Getting started This guide is designed for users new to Native Apps and assumes basic knowledge of Snow CLI and Snowflake. Start your Snowflake emulator and connect to it using the Snow CLI in order to execute the commands further below. In this guide, you will locally deploy a Native App using an existing Application Package. ### Clone the repository Clone the [Native Apps repository](https://github.com/snowflakedb/native-apps-examples) and navigate to the `tasks-streams` directory: ```bash git clone https://github.com/snowflakedb/native-apps-examples.git cd native-apps-examples/tasks-streams ``` ### Deploy Native App Deploy the Native App using the Snow CLI: ```bash snow app run --connection localstack ``` The following output should be displayed: ```bash Creating new application package tasks_streams_app_pkg_username in account. Checking if stage tasks_streams_app_pkg_username.app_src.stage exists, or creating a new one if none exists. Performing a diff between the Snowflake stage: stage and your local deploy_root: /Users/username/code/localstack/native-apps-examples/tasks-streams/output/deploy. Local changes to be deployed: added: app/manifest.yml -> manifest.yml added: app/setup_script.sql -> setup_script.sql added: src/module-ui/src/environment.yml -> streamlit/environment.yml added: src/module-ui/src/ui.py -> streamlit/ui.py Updating the Snowflake stage from your local /Users/username/code/localstack/native-apps-examples/tasks-streams/output/deploy directory. Validating Snowflake Native App setup script. Creating new application object tasks_streams_app_username in account. Application 'TASKS_STREAMS_APP_username' created successfully. Your application object (tasks_streams_app_username) is now available: https://app.snowflake.com/test/test/#/apps/application/TASKS_STREAMS_APP_username ``` ### Access Native App You can access the Native App by visiting your preferred browser and navigating to the following URL: ```bash https://snowflake.localhost.localstack.cloud:4566/apps/test/test/TASKS_STREAMS_APP_username/ ``` :::note The URL above is an example. Change the outputted URL by: 1. Replacing `https://app.snowflake.com` with `https://snowflake.localhost.localstack.cloud:4566`. 2. Changing the path structure from `/#/apps/application/` to `/apps/test/test/`. You can make additional changes depending on your local setup. ::: The following app should be displayed: ![Native App](/images/snowflake/native-app.png) # Network Rules > Get started with Network Rules in LocalStack for Snowflake ## Introduction Network rules are schema-level objects in Snowflake that allow you to group network identifiers (such as IP addresses, ports, or private endpoints) into logical units. They are used to define which network traffic should be allowed or blocked. The Snowflake emulator in LocalStack supports basic CRUD operations (`CREATE`, `ALTER`, `DROP`, `SHOW`) for network rules. This enables you to create and manage network rule objects locally for testing and schema validation. :::note While you can create and manage network rules, their use in enforcing network access policies is not yet supported in the emulator. ::: ## Getting started This guide is designed for users new to network rules and assumes you are already connected to your Snowflake emulator with a SQL client. The following examples demonstrate how to create, alter, show, and drop network rules. ### Create a network rule You can create a network rule using the `CREATE NETWORK RULE` statement. The example below creates a network rule that allows ingress traffic from a specific IPv4 address: ```sql showLineNumbers CREATE NETWORK RULE allow_ip_rule TYPE = IPV4 MODE = INGRESS VALUE_LIST = ('192.168.1.1') COMMENT = 'Allow traffic from 192.168.1.1'; ``` ### Show network rules You can list all network rules in your schema using the `SHOW NETWORK RULES` statement: ```sql SHOW NETWORK RULES; ``` ### Alter a network rule You can modify an existing network rule using the `ALTER NETWORK RULE` statement. The example below updates the comment: ```sql ALTER NETWORK RULE allow_ip_rule SET COMMENT = 'Updated description'; ``` ### Drop a network rule You can delete an existing network rule with the `DROP NETWORK RULE` statement: ```sql DROP NETWORK RULE allow_ip_rule; ``` :::note ## Limitations - Only CRUD operations are supported in the emulator. - Network rules cannot yet be enforced or attached to other Snowflake objects. - Use this feature for schema validation and testing SQL workflows, not for actual network access control. ::: # Openflow > Get started with Openflow in LocalStack for Snowflake ## Introduction Openflow is Snowflake’s data movement service that provides a unified platform for building, scaling, and managing data pipelines. It is powered by Apache NiFi and enables flexible data ingestion, transformation, and integration across diverse sources and destinations. The Snowflake emulator in LocalStack supports **basic Openflow functionality** by using Apache NiFi. This allows you to experiment locally with Openflow concepts, such as creating processors and running SQL queries against the Snowflake emulator. You can access the Openflow UI when the emulator is running at: ``` https://snowflake.localhost.localstack.cloud:4566/openflow/ ``` :::note Openflow in LocalStack for Snowflake is intended for local experimentation. It does not provide the full set of managed Openflow capabilities available in Snowflake’s cloud platform. ::: ## Getting started To begin using Openflow in LocalStack: 1. Start your Snowflake emulator. 2. Open the following URL in your browser: `https://snowflake.localhost.localstack.cloud:4566/openflow/` ![Openflow running locally via Apache NiFi](/images/snowflake/openflow-feature/openflow-nifi.png) The first load may take some time, as Apache NiFi dependencies are downloaded and initialized. Once the UI is available, you can create and configure NiFi processors directly in your browser. ### Running a query with ExecuteSQL The following example demonstrates how to run a simple query against the Snowflake emulator using the `ExecuteSQL` processor. 1. Add an ExecuteSQL processor: Drag the `ExecuteSQL` processor onto the canvas in the Openflow UI. 2. Configure the processor: - Set the **Database Connection Pooling Service** to use the default `Snowflake Connection Pool`. ![Processor](/images/snowflake/openflow-feature/processor.png) - Enter a query, for example: ```sql SELECT 123; ``` - In the **Relationships** tab, configure the processor to terminate or retry on `failure` and `success`. ![Terminate Processor](/images/snowflake/openflow-feature/terminate-processor.png) 3. Start the processor: Right-click the processor and choose **Start**. The processor will run the SQL query against the Snowflake emulator. ![Start Processor](/images/snowflake/openflow-feature/start-processor.png) 4. Verify execution: In the emulator logs, you should see the executed query: ```sql Running query (account/DB/schema TEST/TEST/public): SELECT 123 ``` :::note ## Limitations - The initial download of Apache NiFi is large (~750 MB) and may take several minutes. - Only basic UI and processor creation are supported. Advanced Openflow functionality, including governance, AI capabilities, and managed connectors, is not included in the local emulator. ::: # Polaris Catalog > Get started with Polaris Catalog in LocalStack for Snowflake ## Introduction [Polaris Catalog](https://github.com/apache/polaris) is a unified data catalog that provides a single view of all your data assets across Snowflake and external sources. It enables you to discover, understand, and govern your data assets, making it easier to find and use the right data for your analytics and machine learning projects. The Snowflake emulator supports creating Iceberg tables with Polaris catalog. Currently, [`CREATE CATALOG INTEGRATION`](https://docs.snowflake.com/en/sql-reference/sql/create-catalog-integration-open-catalog) is supported by LocalStack. LocalStack also provides a `localstack/polaris` Docker image that can be used to create a local Polaris REST catalog. ## Getting started This guide is designed for users new to Iceberg tables with Polaris catalog and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client in order to execute the queries further below. This guide shows how to use the Polaris REST catalog to create Iceberg tables in the Snowflake emulator, by: - Launching the Polaris Catalog service - Setting up an external volume - Creating a catalog integration - Creating an Iceberg table - Querying the Iceberg table ### Start Polaris catalog container The following command starts the Polaris catalog container using the `localstack/polaris` Docker image: ```bash showLineNumbers docker run -d --name polaris-test \ -p 8181:8181 -p 8182:8182 \ -e AWS_REGION=us-east-1 \ -e AWS_ACCESS_KEY_ID=test \ -e AWS_SECRET_ACCESS_KEY=test \ -e AWS_ENDPOINT_URL=http://localhost.localstack.cloud:4566 \ -e POLARIS_BOOTSTRAP_CREDENTIALS=default-realm,root,s3cr3t \ -e polaris.realm-context.realms=default-realm \ -e quarkus.otel.sdk.disabled=true \ localstack/polaris:latest ``` Wait for Polaris to become healthy: ```bash curl -X GET http://localhost:8182/health ``` ### Authenticate and create Polaris catalog Set variables and retrieve an access token: ```bash showLineNumbers REALM="default-realm" CLIENT_ID="root" CLIENT_SECRET="s3cr3t" BUCKET_NAME="test-bucket-$(openssl rand -hex 4)" CATALOG_NAME="polaris" TOKEN=$(curl -s -X POST http://localhost:8181/api/catalog/v1/oauth/tokens \ -H "Polaris-Realm: $REALM" \ -d "grant_type=client_credentials&client_id=$CLIENT_ID&client_secret=$CLIENT_SECRET&scope=PRINCIPAL_ROLE:ALL" | jq -r '.access_token') ``` The `TOKEN` variable will contain the access token. Create a catalog: ```bash showLineNumbers curl -s -X POST http://localhost:8181/api/management/v1/catalogs \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "catalog": { "name": "'"$CATALOG_NAME"'", "type": "INTERNAL", "properties": { "default-base-location": "s3://'"$BUCKET_NAME"'/test" }, "storageConfigInfo": { "storageType": "S3_COMPATIBLE", "allowedLocations": ["s3://'"$BUCKET_NAME"'/"], "s3.roleArn": "arn:aws:iam::000000000000:role/'"$BUCKET_NAME"'", "region": "us-east-1", "s3.pathStyleAccess": true, "s3.endpoint": "http://localhost:4566" } } }' ``` Grant necessary permissions to the catalog: ```bash showLineNumbers curl -s -X PUT http://localhost:8181/api/management/v1/catalogs/polaris/catalog-roles/catalog_admin/grants \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{"type": "catalog", "privilege": "TABLE_WRITE_DATA"}' ``` ### Create a bucket Create a bucket using the `lstk aws` command: ```bash lstk aws s3 mb s3://$BUCKET_NAME ``` ### Create an external volume In your SQL client, create an external volume using the `CREATE EXTERNAL VOLUME` statement: ```sql showLineNumbers CREATE EXTERNAL VOLUME polaris_volume STORAGE_LOCATIONS = ( ( NAME = aws_s3_test STORAGE_PROVIDER = S3 STORAGE_BASE_URL = 's3://test-bucket/' STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::000000000000:role/test-bucket' ENCRYPTION = (TYPE = AWS_SSE_S3) ) ) ALLOW_WRITES = TRUE; ``` ### Create catalog integration Create a catalog integration using the `CREATE CATALOG INTEGRATION` statement: ```sql showLineNumbers CREATE CATALOG INTEGRATION polaris_catalog CATALOG_SOURCE = ICEBERG_REST TABLE_FORMAT = ICEBERG CATALOG_NAMESPACE = 'test_namespace' REST_CONFIG = ( CATALOG_URI = 'http://localhost:8181', CATALOG_NAME = 'polaris' ) REST_AUTHENTICATION = ( TYPE = OAUTH, OAUTH_CLIENT_ID = 'root', OAUTH_CLIENT_SECRET = 's3cr3t', OAUTH_ALLOWED_SCOPES = (PRINCIPAL_ROLE:ALL) ) ENABLED = TRUE REFRESH_INTERVAL_SECONDS = 60 COMMENT = 'Polaris catalog integration'; ``` ### Create and query an Iceberg table Now create the table using the Polaris catalog and volume: ```sql showLineNumbers CREATE ICEBERG TABLE polaris_iceberg_table (c1 TEXT) CATALOG = 'polaris_catalog', EXTERNAL_VOLUME = 'polaris_volume', BASE_LOCATION = 'test/test_namespace'; ``` Insert and query data: ```sql INSERT INTO polaris_iceberg_table(c1) VALUES ('test'), ('polaris'), ('iceberg'); SELECT * FROM polaris_iceberg_table; ``` The output should be: ```sql +----------+ | c1 | |----------| | iceberg | | foobar | | test | +----------+ ``` All data will be persisted under: ```bash lstk aws s3 ls s3://$BUCKET_NAME/test/test_namespace/ ``` You will see: - `data/` with `.parquet` files - `metadata/` with Iceberg metadata files ## Configuration options The following configuration options are available for the Polaris Catalog Docker image provided by LocalStack: | Environment Variable | Description | Default Value | Required | | ------------------------------- | -------------------------------------------------------------------------- | ------------- | --------------------------- | | `AWS_REGION` | The AWS region to use | `us-east-1` | Yes | | `AWS_ACCESS_KEY_ID` | AWS access key ID for accessing AWS services | - | Yes when using AWS services | | `AWS_SECRET_ACCESS_KEY` | AWS secret access key for accessing AWS services | - | Yes when using AWS services | | `AWS_ENDPOINT_URL` | Custom endpoint URL for AWS services (e.g., for LocalStack) | - | No | | `POLARIS_BOOTSTRAP_CREDENTIALS` | Initial realm, username, and password in format: `realm,username,password` | - | Yes | | `polaris.realm-context.realms` | List of realms to create/use | - | Yes | | `quarkus.otel.sdk.disabled` | Disable OpenTelemetry SDK | `false` | No | The following logging options are available for the Polaris Catalog Docker image: | Logging Option | Description | | ----------------------------------------------------- | ----------------------------------------------------------------------- | | `quarkus.log.level` | Sets the overall logging level (e.g., DEBUG) | | `quarkus.log.console.level` | Sets the console logging level (e.g., DEBUG) | | `quarkus.log.category."org.apache.polaris".level` | Sets the logging level specifically for the Polaris components | | `quarkus.log.category."org.apache.polaris".min-level` | Sets the minimum logging level for the Polaris components (e.g., TRACE) | # Resource Monitors > Get started with Resource Monitors in LocalStack for Snowflake ## Introduction Resource monitors in Snowflake allow administrators to track and control credit usage for warehouses and accounts. They help manage costs by defining limits and triggering actions (such as suspending warehouses) when thresholds are reached. The Snowflake emulator offers CRUD support for resource monitors. These objects are placeholders only, they do not track usage or enforce limits. This allows you to test Terraform configurations or automation flows that reference resource monitors without enabling their actual functionality. ## Getting started This guide is designed for users new to resource monitors and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client in order to execute the queries below. In this guide, you will: - Create a resource monitor. - View its properties. - Alter it to adjust quotas. - Drop it when it is no longer needed. ### Create a resource monitor You can create a resource monitor with the `CREATE RESOURCE MONITOR` statement: ```sql CREATE RESOURCE MONITOR test_monitor WITH CREDIT_QUOTA = 100 TRIGGERS ON 80 PERCENT DO SUSPEND; ``` This example creates a monitor named `test_monitor` with a quota of 100 credits. When 80% of the quota is reached, it suspends associated warehouses. ### Show resource monitors You can list all resource monitors in the emulator with: ```sql SHOW RESOURCE MONITORS; ``` ### Describe a resource monitor You can view the properties of a specific monitor with: ```sql DESCRIBE RESOURCE MONITOR test_monitor; ``` ### Alter a resource monitor You can change the quota or triggers of an existing resource monitor using `ALTER RESOURCE MONITOR`: ```sql ALTER RESOURCE MONITOR test_monitor SET CREDIT_QUOTA = 200; ``` ### Drop a resource monitor When a monitor is no longer needed, you can drop it with: ```sql DROP RESOURCE MONITOR IF EXISTS test_monitor; ``` # REST API > Get started with REST API Endpoints in LocalStack for Snowflake ## Introduction The [Snowflake REST API](https://docs.snowflake.com/en/developer-guide/snowflake-rest-api/snowflake-rest-api) provides REST API endpoints that allow you to manage schemas and tables in Snowflake. Snowflake REST APIs let you use the programming language of your choice to build your integrations. LocalStack for Snowflake supports REST API endpoints that let you manage your Snowflake data locally. ## Supported Snowflake REST API endpoints LocalStack for Snowflake supports the following REST API endpoints to manage your Snowflake data locally: | Supported Endpoint | Description | |--------------------|-------------| | `GET /api/v2/databases` | Lists databases. | | `POST /api/v2/databases` | Creates a database. | | `GET /api/v2/databases/` | Fetches a database. | | `PUT /api/v2/databases/` | Creates a new, or alters an existing, database. | # Row Access Policies > Get started with Row Access Policies in LocalStack for Snowflake ## Introduction Row access policies (RAPs) are a feature of Snowflake that allows you to control access to specific rows in a table. This is useful for implementing security policies, such as restricting access to certain users or groups. The Snowflake emulator supports Row Access Policies, allowing developers to implement and test fine-grained, row-level access controls locally. These policies define conditions that determine which rows are visible to the querying user. ## Getting started This guide is designed for users new to Row Access Policies and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client to execute the queries below. The following sections demonstrate how to create a row access policy, attach it to a table, and observe how it filters rows during queries based on policy logic. ### Create row access policy Use the `CREATE ROW ACCESS POLICY` statement to define a filter condition. This policy will restrict row visibility based on column values. ```sql showLineNumbers CREATE OR REPLACE ROW ACCESS POLICY id_filter_policy AS (id INT) RETURNS BOOLEAN -> id IN (1, 2); ``` ### Apply policy to a table Create a table and bind the row access policy to one of its columns using the `WITH ROW ACCESS POLICY` clause. ```sql showLineNumbers CREATE TABLE accounts ( id INT ) WITH ROW ACCESS POLICY id_filter_policy ON (id); ``` Insert sample data into the table: ```sql INSERT INTO accounts(id) VALUES (1), (2), (3); ``` ### Query table with policy applied Querying the table will now return only rows that match the access policy condition. ```sql SELECT * FROM accounts; ``` The output should be: ```sql ID| --+ 1| 2| ``` ### Remove access policy You can remove a row access policy from a table using the `ALTER TABLE ... DROP ROW ACCESS POLICY` statement. ```sql ALTER TABLE accounts DROP ROW ACCESS POLICY id_filter_policy; ``` Re-run the query to view all rows: ```sql SELECT * FROM accounts; ``` The output should be: ```sql ID| --+ 1| 2| 3| ``` # Sample Data > Get started with Sample Data in LocalStack for Snowflake ## Introduction Snowflake provides sample datasets that allow users to test and develop queries without needing to import their own data. These sample datasets include TPC-H benchmark data, which is commonly used for evaluating database performance and practicing SQL queries. The Snowflake emulator supports importing Snowflake's sample datasets using the `FROM SHARE SFC_SAMPLES` syntax. This enables you to create a local `snowflake_sample_data` database with TPC-H benchmark data for testing and development purposes. ## Getting started This guide is designed for users new to Sample Data and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using a SQL client to execute the queries below. The following sections guide you through importing sample data and querying the TPC-H benchmark dataset. ### Import Sample Data To import the sample data, use the `CREATE DATABASE ... FROM SHARE` statement. The following example demonstrates how to import Snowflake's sample data. ```sql CREATE DATABASE SNOWFLAKE_SAMPLE_DATA FROM SHARE SFC_SAMPLES.SAMPLE_DATA; ``` This creates a `snowflake_sample_data` database with the following structure: | Object | Name | | --- | --- | | Database | `snowflake_sample_data` | | Schema | `tpch_sf1` | | Table | `orders` | ### Query the Sample Data Once the sample data is imported, you can query the `orders` table in the `tpch_sf1` schema. The following example demonstrates how to query the sample data. ```sql SELECT * FROM snowflake_sample_data.tpch_sf1.orders LIMIT 5; ``` You can also filter the data using the `WHERE` clause. The following example demonstrates how to filter the data by `O_ORDERKEY`. ```sql SELECT * FROM snowflake_sample_data.tpch_sf1.orders WHERE O_ORDERKEY = 3000001; ``` ### Schema Details The `orders` table in the `snowflake_sample_data.tpch_sf1` schema follows the TPC-H benchmark schema with the following columns: | Column | Type | Description | | --- | --- | --- | | O_ORDERKEY | NUMBER(38,0) | Order key | | O_CUSTKEY | NUMBER(38,0) | Customer key | | O_ORDERSTATUS | VARCHAR(1) | Order status | | O_TOTALPRICE | NUMBER(12,2) | Total price | | O_ORDERDATE | DATE | Order date | | O_ORDERPRIORITY | VARCHAR(15) | Order priority | | O_CLERK | VARCHAR(15) | Clerk identifier | | O_SHIPPRIORITY | NUMBER(38,0) | Shipping priority | | O_COMMENT | VARCHAR(79) | Comments | ## Current Limitations The sample data feature currently has the following limitations: - The dataset contains sample data with limited rows in the `orders` table. - The full TPC-H dataset is not yet implemented. - Only the `tpch_sf1` schema and `orders` table are available. For more information on Snowflake's sample data, refer to the [official Snowflake documentation](https://docs.snowflake.com/en/user-guide/sample-data-using). # Secrets > Get started with Secrets in LocalStack for Snowflake ## Introduction Secrets in Snowflake provide a secure way to store sensitive credentials, such as usernames and passwords, for use with external integrations. They allow you to centralize authentication information and manage access consistently across your Snowflake environment. The Snowflake emulator offers CRUD support, which are currently mocked and not functional. This makes it possible to test workloads locally that rely on secure credential management without needing a live Snowflake account. ## Getting started This guide is designed for users new to Secrets and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client in order to execute the queries below. In this guide, you will: 1. Create a secret. 2. Show and describe existing secrets. 3. Alter a secret. 4. Drop a secret. ### Create a Secret You can create a secret using the `CREATE SECRET` statement. The following example creates a password-based secret: ```sql CREATE SECRET my_secret TYPE = PASSWORD USERNAME = 'myuser' PASSWORD = 'mypassword123'; ``` ### Show Secrets You can list all secrets in the account using the `SHOW SECRETS` command: ```sql SHOW SECRETS; ``` ### Describe Secret You can view the details of a specific secret using the `DESCRIBE SECRET` command: ```sql DESCRIBE SECRET my_secret; ``` ### Alter Secret You can update the properties of an existing secret with the `ALTER SECRET` command. For example, changing the password: ```sql ALTER SECRET my_secret SET PASSWORD = 'newpassword456'; ``` ### Drop Secret You can remove a secret using the `DROP SECRET` statement: ```sql DROP SECRET IF EXISTS my_secret; ``` # Security Integrations > Get started with Security Integrations in LocalStack for Snowflake ## Introduction Security Integration is a Snowflake object that acts as a bridge between Snowflake and an external identity, API, or provisioning service. Security Integrations simplify single sign-on, token-based API access, and automated user/role management while keeping sensitive keys encrypted and auditable within Snowflake. The Snowflake emulator lets you test Security Integrations locally by mocking their creation and management. You can set up a Snowflake OAuth-based security integration that handle user authentication for Snowflake access. ## Getting started This guide is designed for users new to Security Integrations and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client in order to execute the queries further below. In this guide, you will create a Security Integration, display the Security Integration details, alter the Security Integration configuration, and drop the Security Integration. ### Create a Security Integration You can create a Security Integration using the `CREATE SECURITY INTEGRATION` statement. In this example, you can create an OAuth-based Security Integration called `my_oauth_integration`: ```sql CREATE SECURITY INTEGRATION my_oauth_integration TYPE = OAUTH ENABLED = true OAUTH_CLIENT = CUSTOM OAUTH_CLIENT_TYPE = 'PUBLIC' OAUTH_REDIRECT_URI = 'https://example.com/callback' OAUTH_ISSUE_REFRESH_TOKENS = true; ``` ### Describe Security Integration You can view detailed information about a Security Integration using the `DESCRIBE SECURITY INTEGRATION` statement: ```sql DESCRIBE SECURITY INTEGRATION my_oauth_integration; ``` The output should display various properties of the Security Integration: ```sql property |property_type|property_value |property_default| ------------------------------------------+-------------+----------------------------------+----------------+ BLOCKED_ROLES_LIST |List |[] |[] | COMMENT |String | | | ENABLED |Boolean |RuntimeException: Unknow json type|false | NETWORK_POLICY |String | | | OAUTH_ALLOWED_AUTHORIZATION_ENDPOINTS |List |[] |[] | OAUTH_ALLOWED_TOKEN_ENDPOINTS |List |[] |[] | OAUTH_ALLOW_NON_TLS_REDIRECT_URI |Boolean |false |false | OAUTH_AUTHORIZATION_ENDPOINT |String | | | OAUTH_CLIENT_ID |String | | | OAUTH_CLIENT_RSA_PUBLIC_KEY_2_FP |String | | | OAUTH_CLIENT_RSA_PUBLIC_KEY_FP |String | | | OAUTH_CLIENT_TYPE |String |PUBLIC |CONFIDENTIAL | OAUTH_ENFORCE_PKCE |Boolean |false |false | OAUTH_ISSUE_REFRESH_TOKENS |Boolean |RuntimeException: Unknow json type|true | OAUTH_REDIRECT_URI |String |https://example.com/callback | | OAUTH_REFRESH_TOKEN_VALIDITY |Integer |7776000 |7776000 | OAUTH_SINGLE_USE_REFRESH_TOKENS_REQUIRED |Boolean |false |false | OAUTH_TOKEN_ENDPOINT |String | | | OAUTH_USE_SECONDARY_ROLES |String |NONE |NONE | PRE_AUTHORIZED_ROLES_LIST |List |[] |[] | USE_PRIVATELINK_FOR_AUTHORIZATION_ENDPOINT|Boolean |false |false | ``` ### Alter Security Integration You can modify the configuration of an existing Security Integration using the `ALTER SECURITY INTEGRATION` statement. In this example, you can disable the integration: ```sql ALTER SECURITY INTEGRATION my_oauth_integration SET ENABLED = false; ``` ### Show Security Integrations You can display the Security Integrations using the `SHOW SECURITY INTEGRATIONS` statement: ```sql SHOW SECURITY INTEGRATIONS LIKE 'my_oauth_integration'; ``` ### Drop Security Integration You can drop the Security Integration using the `DROP SECURITY INTEGRATION` statement: ```sql DROP SECURITY INTEGRATION my_oauth_integration; ``` # Snowpipe > Get started with Snowpipe in LocalStack for Snowflake ## Introduction Snowpipe allows you to load data into Snowflake tables from files stored in an external stage. Snowpipe continuously loads data from files in a stage into a table as soon as the files are available. Snowpipe uses a queue to manage the data loading process, which allows you to load data into Snowflake tables in near real-time. The Snowflake emulator supports Snowpipe, allowing you to create and manage Snowpipe objects in the emulator. You can use Snowpipe to load data into Snowflake tables from files stored in a local directory or a local/remote S3 bucket. ## Getting started This guide is designed for users new to Snowpipe and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client in order to execute the queries further below. In this guide, you will create a stage, and a pipe to load data from a local S3 bucket into a Snowflake table. ### Create an S3 bucket You can create a local S3 bucket using the `mb` command with `lstk aws`. ```bash lstk aws s3 mb s3://test-bucket ``` ### Create a stage You can create a stage using the `CREATE STAGE` command. The stage is used to define the location of the files that Snowpipe will load into the table. ```sql showLineNumbers CREATE STAGE test_stage URL='s3://test-bucket' CREDENTIALS = ( aws_key_id='test' aws_secret_key='test' ) FILE_FORMAT = (TYPE = 'JSON') ``` ### Create a table You can create a table using the `CREATE TABLE` command. The table is used to store the data that Snowpipe loads from the stage. ```sql CREATE TABLE my_test_table(record VARIANT) ``` ### Create a pipe You can create a pipe using the `CREATE PIPE` command. The pipe is used to define the data loading process from the stage to the table. ```sql CREATE PIPE test_pipe AUTO_INGEST = TRUE AS COPY INTO my_test_table FROM @test_stage/ FILE_FORMAT = (TYPE = 'JSON') ``` ### Get pipe details You can use the `DESC PIPE` command to get the details of the pipe you created. ```sql DESC PIPE test_pipe ``` Retrieve the `notification_channel` value from the output of the `DESC PIPE` query. You will use this value to add a notification configuration to the S3 bucket. ### Create bucket notification You can use the [`PutBucketNotificationConfiguration`](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketNotificationConfiguration.html) API to create a bucket notification configuration that sends notifications to Snowflake when new files are uploaded to the S3 bucket. ```bash showLineNumbers lstk aws s3api put-bucket-notification-configuration \ --bucket test-bucket \ --notification-configuration file://notification.json ``` The `notification.json` file should contain the following configuration: ```json showLineNumbers { "QueueConfigurations": [ { "Id": "test-queue", "QueueArn": "arn:aws:sqs:us-east-1:000000000000:sf-snowpipe-TEST", "Events": ["s3:ObjectCreated:*"] } ], "EventBridgeConfiguration": {} } ``` Replace the `QueueArn` with the ARN of the queue provided in the `DESC PIPE` query output. ### Copy data to the stage Copy a JSON file to the S3 bucket to trigger the pipe to load the data into the table. Create a JSON file named `test.json` with the following content: ```json {"name": "Alice", "age": 30} {"name": "Bob", "age": 42} ``` Upload the file to the S3 bucket: ```bash lstk aws s3 cp test.json s3://test-bucket/ ``` ### Check the data After uploading the file to the S3 bucket in the previous step, the contents of the JSON file should get inserted into the table automatically by the `test_pipe` pipe. You can check the data in the table using the following query: ```sql SELECT * FROM my_test_table ``` # SQL API > Get started with SQL API in LocalStack for Snowflake ## Introduction The [Snowflake SQL API](https://docs.snowflake.com/en/developer-guide/sql-api/about-sql-api) allows you to submit SQL statements for execution over HTTP. LocalStack for Snowflake supports the SQL API, enabling you to execute single or multiple SQL statements locally. ## Multi-Statement Execution The Snowflake emulator supports [submitting multiple SQL statements in a single request](https://docs.snowflake.com/en/developer-guide/sql-api/submitting-multiple-statements). Separate each statement with a semicolon (`;`) and specify the statement count using one of the following methods: ### Session-level configuration Set `MULTI_STATEMENT_COUNT` in the session parameters. This setting is persistent for the entire session. When set to `0`, the emulator accepts any number of statements without requiring an exact count. If set to a non-zero value, you must specify exactly that many statements in each batch. ```python showLineNumbers conn = snowflake.connector.connect( user="test", password="test", account="test", host="snowflake.localhost.localstack.cloud", session_parameters={"MULTI_STATEMENT_COUNT": 0} ) with conn.cursor() as cur: cur.execute("SELECT 1; SELECT 2; SELECT 3") print(list(cur)) # First result cur.nextset() print(list(cur)) # Second result ``` ### Request-level configuration Specify `num_statements` per query using the Python connector's `execute()` method. ```python showLineNumbers with conn.cursor() as cur: cur.execute("SELECT 1; SELECT 2; SELECT 3", num_statements=3) print(list(cur)) # First result cur.nextset() print(list(cur)) # Second result ``` If `MULTI_STATEMENT_COUNT` does not match the actual number of statements, an error is returned: ``` Actual statement count did not match the desired statement count . ``` # Stages > Get started with Stages in LocalStack for Snowflake import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction Stages are a way to load data into Snowflake. You can use stages to load data from files in a variety of formats, including CSV, JSON, and Parquet. You can also use stages to load data from external cloud storage services, such as Amazon S3, Google Cloud Storage, and Microsoft Azure Blob Storage. The Snowflake emulator supports stages, allowing you to load data into Snowflake using the same commands and syntax as the Snowflake service. ## Getting started This guide is designed for users new to Stages and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client in order to execute the queries further below. In this guide, you will create a database and a table for storing data. You will then create a stage to load data into the table. ### Download the sample data You can download the sample data by [clicking on this link](/artifacts/getting-started.zip) and downloading this in your machine. Unzip the file and save the contents to a directory on your local machine. ### Create a database & table You can create a database using the `CREATE DATABASE` command. In this example, you can create a database called `snowflake_tutorials`: ```sql CREATE OR REPLACE DATABASE snowflake_tutorials; ``` Similarly, you can create a table using the `CREATE TABLE` command. In this example, you can create a table called `employees` in `snowflake_tutorials.public`: ```sql showLineNumbers CREATE OR REPLACE TABLE employees ( first_name STRING , last_name STRING , email STRING , streetaddress STRING , city STRING , start_date DATE ); ``` ### Create a stage You can now create a stage using the `CREATE STAGE` command. In this example, you can create a stage called `employees_stage`: ```sql CREATE OR REPLACE STAGE employees_stage FILE_FORMAT = csv; ``` ### Upload data to the stage In this example, you can upload the CSV files to the table stage provided for `employees` table. ```sql PUT file://./employees0*.csv @@employees_stage AUTO_COMPRESS=TRUE; ``` ```sql PUT file://C:\temp\employees0*.csv @@employees_stage AUTO_COMPRESS=TRUE; ``` The expected output is: ```sql +-----------------+--------------------+-------------+-------------+--------------------+--------------------+----------+---------+ | source | target | source_size | target_size | source_compression | target_compression | status | message | |-----------------+--------------------+-------------+-------------+--------------------+--------------------+----------+---------| | employees01.csv | employees01.csv.gz | 370 | 0 | NONE | GZIP | SKIPPED | | | employees02.csv | employees02.csv.gz | 364 | 0 | NONE | GZIP | SKIPPED | | | employees03.csv | employees03.csv.gz | 407 | 0 | NONE | GZIP | SKIPPED | | | employees04.csv | employees04.csv.gz | 375 | 0 | NONE | GZIP | SKIPPED | | | employees05.csv | employees05.csv.gz | 404 | 0 | NONE | GZIP | SKIPPED | | +-----------------+--------------------+-------------+-------------+--------------------+--------------------+----------+---------+ ``` ## Loading files from S3 You can also load data from an S3 bucket using the `CREATE STAGE` command. Create a new S3 bucket named `testbucket` and upload the [employees CSV files](/artifacts/getting-started.zip) to the bucket. You can use LocalStack's `lstk aws` command to create the S3 bucket and upload the files. ```bash lstk aws s3 mb s3://testbucket lstk aws s3 cp employees0*.csv s3://testbucket ``` In this example, you can create a stage called `my_s3_stage` to load data from an S3 bucket: ```sql showLineNumbers CREATE STAGE my_s3_stage STORAGE_INTEGRATION = s3_int URL = 's3://testbucket/' FILE_FORMAT = csv; ``` You can further copy data from the S3 stage to the table using the `COPY INTO` command: ```sql showLineNumbers COPY INTO mytable FROM @my_s3_stage PATTERN='.*employees.*.csv'; ``` # Storage Integrations > Get started with Storage Integrations in LocalStack for Snowflake ## Introduction Snowflake storage integrations enable access to external cloud storage like Amazon S3, Google Cloud Storage, and Azure Blob Storage. They manage authentication through generated IAM roles, enhancing security and simplifying data operations without exposing sensitive credentials. This approach centralizes and controls access, streamlining workflows across major cloud platforms. The Snowflake emulator supports storage integrations, allowing you to test interactions with external storage using the same commands and syntax as the Snowflake service. ## Getting started This guide is designed for users new to Storage Integration and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client in order to execute the queries further below. In this guide, you will create a Snowflake Storage Integration with Amazon S3 and creating an external stage to load data. ### Create an S3 bucket You can create a local S3 bucket using the `mb` command with `lstk aws`. ```bash lstk aws s3 mb s3://testbucket ``` Upload some sample CSV file into the S3 bucket using the following command: ```bash lstk aws s3 cp file.csv s3://testbucket ``` ### Create a Storage Integration You can now create a Storage Integration named `s_example` which will connect Snowflake to your S3 bucket using the following statement: ```sql showLineNumbers CREATE STORAGE INTEGRATION s_example TYPE = EXTERNAL_STAGE ENABLED = TRUE STORAGE_PROVIDER = 'S3' STORAGE_AWS_ROLE_ARN = 'arn:aws:iam::000000000000:role/testrole' STORAGE_ALLOWED_LOCATIONS = ('s3://testbucket'); ``` The expected output is: ```sql +---------------------------------------------+ | ?COLUMN? | |---------------------------------------------| | Integration S_EXAMPLE successfully created. | +---------------------------------------------+ ``` ### Describe the Storage Integration After creating the storage integration, you can retrieve important configuration by running the following statement: ```sql DESCRIBE STORAGE INTEGRATION s_example; ``` The expected output is: ```sql +---------------------------+---------------+-----------------------------------------+------------------+ | property | property_type | property_value | property_default | |---------------------------+---------------+-----------------------------------------+------------------| | ENABLED | Boolean | true | false | | STORAGE_PROVIDER | String | S3 | | | STORAGE_ALLOWED_LOCATIONS | List | s3://testbucket | [] | | STORAGE_BLOCKED_LOCATIONS | List | | [] | | STORAGE_AWS_IAM_USER_ARN | String | arn:aws:iam::000000000000:user/test | | | STORAGE_AWS_ROLE_ARN | String | arn:aws:iam::000000000000:role/testrole | | | STORAGE_AWS_EXTERNAL_ID | String | TEST_SFCRole=test | | | COMMENT | String | | | +---------------------------+---------------+-----------------------------------------+------------------+ 8 Row(s) produced. Time Elapsed: 0.050s ``` ### Create a stage You can now create an external stage using the following statement: ```sql showLineNumbers CREATE STAGE stage_example STORAGE_INTEGRATION = s_example URL = 's3://testbucket' FILE_FORMAT = (TYPE = CSV); ``` The expected output is: ```sql +------------------------------------------------+ | ?COLUMN? | |------------------------------------------------| | Stage area STAGE_EXAMPLE successfully created. | +------------------------------------------------+ 0 Row(s) produced. Time Elapsed: 0.083s ``` ### List the files To list the files in the stage, you can run the following statement: ```sql LIST @stage_example; ``` The output will show the `files.csv` file that we uploaded earlier to the S3 bucket. # Stored Procedures > Get started with Stored Procedures in LocalStack for Snowflake ## Introduction Stored Procedures uses [Snowflake's Procedures API](https://docs.snowflake.com/en/developer-guide/stored-procedure/stored-procedures-api). The API consists of objects and the methods in those objects. You can create stored procedures, execute SQL via embedded scripts, and call these using Snowflake’s supported methods. The methods that we support thus far are: - `snowflake.execute()` - `snowflake.createStatement()` - `statement.execute()` - `resultSet.next()` - `resultSet.getColumnValue()` ## Getting started This guide is designed for users new to Stored Procedures and assumes basic knowledge of Snowflake. Start LocalStack for Snowflake and execute [Snowflake stored procedures](https://docs.snowflake.com/en/developer-guide/stored-procedure/stored-procedures-api#object-snowflake). ## JavaScript In LocalStack for Snowflake, you can create JavaScript Stored Procedures to define reusable logic using Snowflake’s JavaScript API. These procedures allow you to embed SQL execution inside JavaScript functions, enabling flexible control flow, conditionals, and result handling within your data workflows. ### Creating a simple JavaScript procedure The following is a simple JavaScript procedure that makes use of some of the most important methods of Snowflake JavasScript Procedures API: ```javascript showLineNumbers CREATE OR REPLACE PROCEDURE minimal_proc() RETURNS STRING LANGUAGE JAVASCRIPT AS $$ var stmt = snowflake.createStatement({sqlText: "SELECT 'hello world'"}); var rs = stmt.execute(); rs.next(); return rs.getColumnValue(1); $$; ``` # Streamlit > Get started with Streamlit in LocalStack for Snowflake ## Introduction Snowflake provides SQL commands to create and modify a `STREAMLIT` object. Streamlit is a Python library that allows you to create web applications with simple Python scripts. With Streamlit, you can create interactive web applications without having to learn complex web development technologies. The Snowflake emulator supports Streamlit, allowing you to create Streamlit applications using the same commands and syntax as the Snowflake service. ## Getting started This guide is designed for users new to Streamlit and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client in order to execute the queries further below. In this guide, you will create a Streamlit application, describe the Streamlit application, and drop the Streamlit application. ### Create an application You can create a Streamlit application using the `CREATE STREAMLIT` command. In this example, you can create a Streamlit application called `testapp` with a `@teststage` stage and a `test.py` script: ```sql CREATE STREAMLIT TESTAPP ROOT_LOCATION = '@teststage' MAIN_FILE = 'test.py'; ``` The output shows that the Streamlit application `testapp` was successfully created. ```sql +-----------------------------------------+ | ?COLUMN? | |-----------------------------------------| | Streamlit TESTAPP successfully created. | +-----------------------------------------+ ``` ### Describe the application You can show the Streamlit application using the `DESCRIBE STREAMLIT` command. In this example, you can describe the Streamlit application `testapp`: ```sql DESCRIBE STREAMLIT testapp; ``` The output shows the details of the Streamlit application `testapp`. ```sql +---------+-------+---------------+-----------+-----------------+---------+------------------+---------------+ | name | title | root_location | main_file | query_warehouse | url_id | default_packages | user_packages | |---------+-------+---------------+-----------+-----------------+---------+------------------+---------------| | TESTAPP | NULL | @teststage | test.py | NULL | testurl | ... | | ``` ### Drop the application You can drop the Streamlit application using the `DROP STREAMLIT` command. In this example, you can drop the Streamlit application `testapp`: ```sql DROP STREAMLIT testapp; ``` The output shows that the Streamlit application `testapp` was successfully dropped. ```sql +-----------------------------------------+ | ?COLUMN? | |-----------------------------------------| | Streamlit TESTAPP successfully dropped. | +-----------------------------------------+ ``` ## Connecting Streamlit to Snowflake emulator To connect to the Snowflake emulator while developing locally, Streamlit provides a way to store secrets and connection details in the project. To run the sample against Snowflake emulator, your local `~/.streamlit/secrets.toml` should look like this: ```toml showLineNumbers [snowpark] user = "test" password = "test" account = "test" warehouse = "test" database = "STREAMLIT_DEMO" schema = "STREAMLIT_USER_PROFILER" role = "test" host = "snowflake.localhost.localstack.cloud" ``` ## Limitations Currently, the Snowflake emulator supports CRUD operations to create Streamlit application entries in the Snowflake emulator, but support for hosting the Web UIs of these Streamlit apps is not yet available. Users can run Streamlit apps locally by using the `streamlit run main.py` command and connecting to the local Snowflake instance. # Streams > Get started with Streams in LocalStack for Snowflake ## Introduction Streams allow you to track changes made to a table. Streams capture changes made to a table, such as inserts, updates, and deletes, and store the changes in a log that you can query to see what changes have been made. The Snowflake emulator supports managed streams. You can create a stream locally to track changes made to an emulated Snowflake table and query the stream to see what changes have been made. ## Getting started This guide is designed for users new to Streams and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client to execute the queries below. The following sections guide you through a simple example of using Streams to track changes in a table that stores information about gym members. We will create tables to store member information and their signup dates, and then use a stream to capture changes made to the members' table. ### Create tables The following SQL snippet demonstrates how to create a table named `members` to store the names and fees paid by members of a gym, and a table named `signup` to store the dates when gym members joined. ```sql showLineNumbers -- Create a table to store the names and fees paid by members of a gym CREATE TABLE IF NOT EXISTS members ( id NUMBER(8) NOT NULL, name VARCHAR(255) DEFAULT NULL, fee NUMBER(3) NULL ); -- Create a table to store the dates when gym members joined CREATE TABLE IF NOT EXISTS signup ( id NUMBER(8), dt DATE ); ``` ### Create a Stream To create a stream, use the `CREATE STREAM` statement. The following example demonstrates how to create a stream named `member_check` to track changes made to the `members` table. ```sql CREATE STREAM IF NOT EXISTS member_check ON TABLE members; ``` ### Insert Data To insert data into the `members` and `signup` tables, use the `INSERT INTO` statement. The following example demonstrates how to insert data into the `members` and `signup` tables. ```sql showLineNumbers INSERT INTO members (id,name,fee) VALUES (1,'Joe',0), (2,'Jane',0), (3,'George',0), (4,'Betty',0), (5,'Sally',0); INSERT INTO signup VALUES (1,'2018-01-01'), (2,'2018-02-15'), (3,'2018-05-01'), (4,'2018-07-16'), (5,'2018-08-21'); ``` ### Query Stream for Changes To query the stream for changes, use the `SELECT` statement. The following example demonstrates how to query the `member_check` stream for changes. ```sql SELECT * FROM member_check; ``` The expected output is: ```sql +----+--------+-----+-----------------+-------------------+---------------------+ | ID | NAME | FEE | METADATA$ACTION | METADATA$ISUPDATE | METADATA$ROW_ID | |+----+--------+-----+-----------------+-------------------+--------------------| | 1 | Joe | 0 | INSERT | False | f05ac800-394b-4007-ab6b-28e1a915769e | | 2 | Jane | 0 | INSERT | False | ab54a93e-3eb5-45fb-85f9-0e5f208e02dc | | 3 | George | 0 | INSERT | False | 0e061182-fb1b-4a54-b018-61ada3feba35 | | 4 | Betty | 0 | INSERT | False | 4dcf24c3-c25e-4e89-b0ec-cb20fbf1275c | | 5 | Sally | 0 | INSERT | False | 1e3abb7e-f3f0-4a78-8fc1-d80e2dfdaaf7 | +----+--------+-----+-----------------+-------------------+---------------------+ ``` # Tags > Get started with Tags in LocalStack for Snowflake ## Introduction Snowflake tags allow you to categorize and manage Snowflake objects by associating custom metadata with them. These tags support governance, cost tracking, and data lineage by enabling organizations to label resources with business-relevant information. The Snowflake emulator supports tags, allowing you to apply these tags to the local Snowflake tables, views, and databases using the same commands and syntax as the Snowflake service. ## Getting started This guide is designed for users new to tagging in Snowflake and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client to execute the queries below. The following sections guide you through an example of creating a database, adding a tag, and using the tag to track metadata in Snowflake. ### Create a Database The following SQL snippet demonstrates how to create a database named `tag_test_db`. ```sql CREATE DATABASE IF NOT EXISTS tag_test_db; ``` The expected output is: ```sql +--------------------------------------------+ | status | |--------------------------------------------| | Database TAG_TEST_DB successfully created. | +--------------------------------------------+ 0 Row(s) produced. Time Elapsed: 0.163s ``` ### Create a Tag To create a tag, use the `CREATE TAG` statement. The following example demonstrates how to create a tag named `tag1`. ```sql CREATE TAG tag1; ``` The expected output is: ```sql +--------------------------------+ | ?COLUMN? | |--------------------------------| | Tag TAG1 successfully created. | +--------------------------------+ 0 Row(s) produced. Time Elapsed: 0.057s ``` ### Assign Tag to Database To assign a tag to a database, use the `ALTER DATABASE` statement. The following example demonstrates how to assign the tag `tag1` with the value `'test 123'` to the `tag_test_db` database. ```sql ALTER DATABASE tag_test_db SET TAG tag1 = 'test 123'; ``` The expected output is: ```sql +----------------------------------+ | ?column? | |----------------------------------| | Statement executed successfully. | +----------------------------------+ 0 Row(s) produced. Time Elapsed: 0.026s ``` ### Query Tag Value To retrieve the value of a tag assigned to a database, use the `SELECT SYSTEM$GET_TAG` statement. The following example demonstrates how to query the value of `tag1` assigned to the `tag_test_db` database. ```sql SELECT SYSTEM$GET_TAG('tag1', 'tag_test_db', 'database'); ``` The expected output is: ```sql +----------------+ | SYSTEM$GET_TAG | |----------------| | test 123 | +----------------+ 1 Row(s) produced. Time Elapsed: 0.565s ``` ### Query Tag References To view all references of a tag within a database, use the `INFORMATION_SCHEMA.TAG_REFERENCES` function. The following example demonstrates how to query the `tag_test_db` database for references to the `tag1` tag. ```sql SELECT * FROM TABLE(tag_test_db.INFORMATION_SCHEMA.TAG_REFERENCES('tag_test_db', 'database')); ``` The expected output is: ```sql +--------------+------------+----------+-----------+----------+-----------------+---------------+-------------+----------+-------------+ | TAG_DATABASE | TAG_SCHEMA | TAG_NAME | TAG_VALUE | LEVEL | OBJECT_DATABASE | OBJECT_SCHEMA | OBJECT_NAME | DOMAIN | COLUMN_NAME | |--------------+------------+----------+-----------+----------+-----------------+---------------+-------------+----------+-------------| | TAG_TEST_DB | PUBLIC | TAG1 | test 123 | DATABASE | NULL | NULL | TAG_TEST_DB | DATABASE | NULL | +--------------+------------+----------+-----------+----------+-----------------+---------------+-------------+----------+-------------+ 1 Row(s) produced. Time Elapsed: 0.528s ``` # Tasks > Get started with Tasks in LocalStack for Snowflake ## Introduction Tasks are user-defined objects that enable the automation of repetitive SQL operations in Snowflake. You can use tasks to schedule SQL statements, such as queries, DDL, and DML operations, to run at a specific time or at regular intervals. The Snowflake emulator provides a CRUD (Create, Read, Update, Delete) interface to manage tasks. ## Getting started This guide is designed for users new to Tasks and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to the Snowflake emulator using an SQL client. ### Create a Task To create a task, use the `CREATE TASK` statement. The following example demonstrates how to create a task named `test_task` that inserts a record into a table named `sample_table` every minute. ```sql showLineNumbers CREATE TASK test_task WAREHOUSE = 'test' SCHEDULE = '1 MINUTE' AS INSERT INTO sample_table(ID) VALUES (123); ``` ### Drop a Task To drop a task, use the `DROP TASK` statement. The following example demonstrates how to drop the `test_task` task. ```sql DROP TASK IF EXISTS test_task; ``` ### Resume a Task To start or resume a task, use the `ALTER TASK` statement. The following example demonstrates how to resume the `test_task` task. ```sql ALTER TASK test_task RESUME; ``` ### Query the table To query the table, use the `SELECT` statement. The following example demonstrates how to query the `sample_table` table. ```sql SELECT * FROM sample_table; ``` The expected output is: ```plaintext 123 ``` # Transaction Management > Get started with Transaction Management in LocalStack for Snowflake ## Introduction Transaction Management is a feature that allows you to manage transactions in Snowflake. You can use Transaction Management to create a transaction management system that is specific to your application. The Snowflake emulator supports Transaction Management, allowing you to emulate realistic database operations that require precise control over when changes are committed or rolled back. ## Getting started This guide is designed for users new to Transaction Management and assumes basic knowledge of SQL and Snowflake. Start your Snowflake emulator and connect to it using an SQL client to execute the queries below. The following sections demonstrate how to start and manage named transactions, check transaction state, control visibility across sessions, and monitor all active transactions using a simple orders table. ### Create a table The following SQL snippet creates a table named orders to store order IDs. We will use this table to test transactional behavior. ```sql CREATE TABLE IF NOT EXISTS orders ( id INT ); ``` ### Begin transaction and insert data Use the `BEGIN` or `START TRANSACTION` statement to begin a transaction and optionally assign it a name. Data inserted during the transaction will not be visible to other sessions until committed. ```sql BEGIN NAME mytxn; INSERT INTO orders VALUES (1), (2); ``` ### Commit transaction Use the `COMMIT` statement to save the changes made during the transaction. ```sql COMMIT; ``` The expected output is: ```sql +------------------------+ | status | |------------------------| | Transaction committed | +------------------------+ ``` ### View the current transaction Use the `CURRENT_TRANSACTION()` function to get the ID of the currently active transaction. ```sql SELECT CURRENT_TRANSACTION() AS txn; ``` After starting a transaction, this function returns a non-null ID. ```sql START TRANSACTION NAME mytxn2; SELECT CURRENT_TRANSACTION() AS txn; ``` The expected output is: ```sql +--------------------------------------+ | TXN | |--------------------------------------| | 6f9bfa42-88d7-4c8c-bc7c-3db1d69d1552 | +--------------------------------------+ ``` ### Show all active transactions Use the `SHOW TRANSACTIONS` statement to view all active transactions. ```sql SHOW TRANSACTIONS; ``` ### Rollback transaction To undo uncommitted changes, use the `ROLLBACK` statement. Subsequent rollbacks have no effect. ```sql showLineNumbers BEGIN; INSERT INTO orders VALUES (3), (4); ROLLBACK; ``` # User-Defined Functions > Get started with User-Defined Functions in SQL, Node.js, Java & Python with LocalStack for Snowflake ## Introduction User-Defined Functions (UDFs) are functions that you can create to extend the functionality of your SQL queries. Snowflake supports UDFs in different programming languages, including SQL, JavaScript, Python, Java, and Scala. The Snowflake emulator supports User-Defined Functions (UDFs) in SQL, JavaScript, Java, and Python. You can create UDFs to extend the functionality of your SQL queries. This guide demonstrates how to create and execute UDFs in SQL, JavaScript, Java, and Python. ## SQL In the Snowflake emulator, you can create scalar SQL UDFs and table-returning SQL UDFs, also known as User-Defined Table Functions (UDTFs). Start your Snowflake emulator and connect to it using a SQL client to execute the queries below. ### Create a Scalar SQL UDF You can create a scalar SQL UDF using the `CREATE FUNCTION` statement. The following example creates a SQL UDF that receives an amount and percentage as input and returns the amount with the percentage added. ```sql showLineNumbers CREATE OR REPLACE FUNCTION add_percentage(amount FLOAT, percentage FLOAT) RETURNS FLOAT AS 'amount + (amount * percentage / 100)'; ``` ### Execute a Scalar SQL UDF You can execute a scalar SQL UDF using the `SELECT` statement. ```sql SELECT add_percentage(100, 8); ``` The result of the query is `108`. ### Create a Table-Returning SQL UDF You can create a table-returning SQL UDF using the `RETURNS TABLE` clause. The following example creates a SQL UDF that returns rows matching the specified minimum amount. ```sql showLineNumbers CREATE OR REPLACE FUNCTION orders_above(min_amount FLOAT) RETURNS TABLE (order_id INTEGER, total FLOAT) AS $$ SELECT order_id, total FROM ( SELECT 1 AS order_id, 42.50 AS total UNION ALL SELECT 2 AS order_id, 120.00 AS total UNION ALL SELECT 3 AS order_id, 255.25 AS total ) AS orders WHERE total >= min_amount $$; ``` ### Execute a Table-Returning SQL UDF You can execute a table-returning SQL UDF from a `FROM` clause using the `TABLE` keyword. ```sql SELECT * FROM TABLE(orders_above(100)); ``` The result of the query is: ```sql +----------+--------+ | ORDER_ID | TOTAL | |----------+--------| | 2 | 120.00 | | 3 | 255.25 | +----------+--------+ ``` ## JavaScript In the Snowflake emulator, you can create JavaScript UDFs to extend the functionality of your SQL queries. Start your Snowflake emulator and connect to it using a SQL client to execute the queries below. ### Create a JavaScript UDF You can create a JavaScript UDF using the `CREATE FUNCTION` statement. The following example creates a JavaScript UDF that receives a number as input and adds 5 to it. ```sql showLineNumbers CREATE OR REPLACE FUNCTION add5(n double) RETURNS double LANGUAGE JAVASCRIPT AS 'return N + 5;'; ``` ### Execute a JavaScript UDF You can execute a JavaScript UDF using the `SELECT` statement. The following example executes the UDF created in the previous step. ```sql SELECT add5(10); ``` The result of the query is `15`. ## Java In the Snowflake emulator, you can create Java UDFs to extend the functionality of your SQL queries. The following modes are supported: - **Inline Java Code** via the `AS` clause - **Staged JAR Files** via the `IMPORTS` clause. Start your Snowflake emulator and connect to it using a SQL client to execute the queries below. ### Create a Java UDF (inline code) You can define a Java UDF using the `CREATE FUNCTION` statement and provide the Java source inline with the `AS` clause. ```sql showLineNumbers CREATE OR REPLACE FUNCTION echo_inline(x VARCHAR) RETURNS VARCHAR LANGUAGE JAVA CALLED ON NULL INPUT HANDLER = 'TestFunc.echoVarchar' AS ' class TestFunc { public static String echoVarchar(String x) { return x; } } '; ``` ### Execute the Java UDF (inline code) Once created, you can call the Java UDF using a standard `SELECT` statement. ```sql SELECT echo_inline('hello world'); ``` The result of the query is: ```sql +---------------------+ | ECHO_INLINE | |---------------------| | hello world | +---------------------+ ``` ### Create a Java UDF from a JAR file You can also compile your Java code into a `.jar` file, upload it to a Snowflake stage, and reference it using the `IMPORTS` clause. ```sql showLineNumbers -- Assume the JAR file has been uploaded to @mystage/testfunc.jar CREATE OR REPLACE FUNCTION echo_from_jar(x VARCHAR) RETURNS VARCHAR LANGUAGE JAVA CALLED ON NULL INPUT HANDLER = 'TestFunc.echoVarchar' IMPORTS = ('@mystage/testfunc.jar'); ``` ### Execute the Java UDF from a JAR file Once created, you can call the Java UDF using a standard `SELECT` statement. ```sql SELECT echo_from_jar('from jar'); ``` The result of the query is: ```sql +---------------------+ | ECHO_FROM_JAR | |---------------------| | from jar | +---------------------+ ``` ## Python In the Snowflake emulator, you can create User-Defined Functions (UDFs) in Python to extend the functionality of your SQL queries. Start your Snowflake emulator and connect to it using a SQL client to execute the queries below. ### Create a Python UDF You can create a Python UDF using the `CREATE FUNCTION` statement. The following example creates a Python UDF that takes a string as input and returns the string with a prefix. ```sql showLineNumbers CREATE OR REPLACE FUNCTION sample_func(sample_arg TEXT) RETURNS VARCHAR LANGUAGE PYTHON RUNTIME_VERSION='3.8' HANDLER='sample_func' AS $$ def sample_func(i): return 'echo: ' + i $$; ``` ### Execute a Python UDF You can execute a Python UDF using the `SELECT` statement. The following example executes the Python UDF created in the previous step. ```sql SELECT sample_func('foobar'); ``` The result of the query is `echo: foobar`. ## Secure Functions Secure UDFs are user-defined functions that protect sensitive information and prevent unauthorized users from viewing function definitions, underlying data, or implementation details. LocalStack supports Secure UDFs, allowing you to tests sensitive data privacy & security controls. To create a Secure UDF, you need to use the `SECURE` keyword in the `CREATE FUNCTION` statement. ```sql CREATE OR REPLACE SECURE FUNCTION secure_func(x VARCHAR) RETURNS VARCHAR LANGUAGE PYTHON RUNTIME_VERSION='3.8' HANDLER='secure_func' AS $$ def secure_func(i): return 'echo: ' + i $$; ``` # Overview > Introduction to LocalStack for Snowflake, covering core use cases, local data pipeline development, and deployment options for development and testing. import { SectionCards } from '../../../../components/SectionCards.tsx'; LocalStack for Snowflake provides a local, Snowflake-compatible emulator that runs within a single container on your local machine or CI environment. It lets you develop and test Snowflake data pipelines, SQL, and integrations without a real Snowflake account or consuming Snowflake credits. See our [Feature Coverage](/snowflake/feature-coverage/) and [SQL Functions](/snowflake/sql-functions/) docs for what's supported. ### Core Use Cases - **Accelerate data pipeline development**: Run dbt, Airflow, and Snowpark pipelines against a local Snowflake-compatible endpoint instead of waiting on cloud credits or shared environments. - **Automate integration testing**: Run SQL and data pipeline tests in CI against a local emulator, without touching a real Snowflake account. - **Validate SQL compatibility**: Check queries, UDFs, and stored procedures against LocalStack before deploying to production. - **Experimental sandbox**: Prototype Snowflake features such as Streams, Tasks, or storage integrations in a risk-free, local environment. LocalStack for Snowflake also supports [state management](/snowflake/capabilities/state-management/) for persisting data across restarts, and [init hooks](/snowflake/capabilities/init-hooks/) for bootstrapping schemas and data on startup. ## Start with the basics :::note Browse the [Feature Coverage](/snowflake/feature-coverage/) and [Capabilities](/snowflake/capabilities/) docs to see what LocalStack for Snowflake supports beyond this `getting-started` flow. ::: # AI & Agent Workflows > Use LocalStack for Snowflake with AI coding assistants and MCP clients. ## Introduction LocalStack gives AI coding assistants a local Snowflake-compatible environment to work against. Instead of letting an agent run SQL experiments against a real Snowflake account, ask your agent to create schemas, run queries, inspect results, and test data pipelines in LocalStack first. This is useful when you want to: - Prototype SQL, schemas, and data pipeline logic from natural language prompts. - Validate AI-generated SQL, dbt models, or Snowpark code before using a real Snowflake account. - Give an AI assistant a safe place to run queries, inspect results, and iterate on a data pipeline. ## Common workflows There are two common ways to use LocalStack for Snowflake in AI-assisted development: - Use the [LocalStack MCP Server](/aws/developer-tools/running-localstack/mcp-server/) when your AI assistant supports MCP clients such as Cursor, Claude, Codex, or OpenCode. The server includes a dedicated Snowflake tool that runs SQL against your local emulator via the Snowflake CLI. - Use LocalStack with the [Snowflake CLI](/snowflake/integrations/snow-cli/) directly when you want the agent to generate SQL, dbt models, or Snowpark code that you review and run locally. Unlike the AWS emulator, `lstk` does not proxy the Snowflake CLI the way it proxies `aws`, `terraform`, or `cdk` commands. Agent workflows that run SQL directly use the Snowflake CLI's own `localstack` connection profile, as set up in [Local Development](/snowflake/getting-started/local-development/#step-2-connect-the-snowflake-cli). ## Quick Setup LocalStack provides an [`agents.md`](https://docs.localstack.cloud/agents.md) file with the full instructions your AI agent needs to get started with LocalStack, including how to configure the MCP server for the AWS, Snowflake, and Azure emulators. You can give the file directly to your agent or copy and paste the prompt below. ```text Fetch https://docs.localstack.cloud/agents.md and follow the instructions to set up LocalStack on my machine. ``` For manual setup of the MCP server, you can follow the steps below. ## Connect an MCP client The LocalStack MCP Server connects MCP-compatible clients to your LocalStack environment. Once configured, your AI assistant can use LocalStack tools to start the Snowflake emulator, run SQL queries and files against it via the Snowflake CLI, inspect logs, and manage state. Start the MCP server with an interactive setup wizard: ```bash npx -y @localstack/localstack-mcp-server init ``` :::note The MCP server runs locally and talks to a LocalStack instance. Your AI assistant is the MCP client. For full installation instructions, detailed setup, and the full tool reference (including the Snowflake-specific `localstack-snowflake-client` tool), see the [LocalStack MCP Server guide](/aws/developer-tools/running-localstack/mcp-server/). You need a valid [Auth Token](/snowflake/getting-started/auth-token/) to configure the server, and the [Snowflake CLI](https://docs.snowflake.com/en/developer-guide/snowflake-cli/index) (`snow`) installed on your `PATH` if you want the agent to run SQL directly. ::: ## Example prompt sequence After LocalStack and your preferred AI tooling are configured, you can use a sequence like this: ```text Start the LocalStack Snowflake emulator. ``` ```text Create a database, schema, and table for storing customer orders, then insert a few sample rows. ``` ```text Run a query that summarizes total order value by customer and show me the results. ``` ```text Write a dbt model that reproduces this summary, and validate it against the LocalStack Snowflake emulator. ``` This keeps the feedback loop local while still giving the assistant a realistic Snowflake-compatible target. ## Review before applying to Snowflake AI-generated SQL and data pipeline code still needs review. Treat LocalStack as the first validation step, not as a replacement for code review, tests, or production deployment controls. Before applying changes to a real Snowflake account, check that: - The generated schema and queries match your intended data model. - Roles, warehouses, and resource names are appropriate for your project. - Tests pass against LocalStack. - You understand any changes the assistant made to pipeline code or configuration. ## Next steps - Configure the [LocalStack MCP Server](/aws/developer-tools/running-localstack/mcp-server/) if your AI assistant supports MCP. - Browse the [Feature Coverage](/snowflake/feature-coverage/) reference, or check the [Getting Started FAQ](/snowflake/getting-started/faq/) for common setup questions. # Auth Token > Configure and manage your LocalStack Auth Token to activate LocalStack for Snowflake and access licensed features. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## What is an Auth Token? An Auth Token is a mandatory credential required to start the Snowflake emulator and activate licensed features. It links your running LocalStack instance to your workspace license and unlocks the services and capabilities available to your account. Auth Tokens are issued at the workspace level in [app.localstack.cloud](https://app.localstack.cloud) and are not specific to any single LocalStack product. The same token works across every LocalStack product your account has access to, including LocalStack for AWS, Snowflake, and Azure. You can manage Auth Tokens from the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens) in the LocalStack Web Application. :::danger[Credential security] Auth Tokens provide access to your license and workspace. Do not commit tokens to version control. If a token is exposed, rotate it immediately in the LocalStack Web Application. ::: ## Token types | Token Type | Scope | Use Case | | :--- | :--- | :--- | | **Developer Token** | Individual | Local development workstations. Managed per user. | | **CI Auth Token** | Workspace | Automated pipelines and shared runners. Managed by workspace admins. | ## Managing your license To use the LocalStack for Snowflake emulator, a license with access to Snowflake is required. You can get a license by [signing up for a free LocalStack account](https://www.localstack.cloud/pricing) and starting a trial, or by exploring additional features with a paid offering. After initiating your trial or acquiring a license, assign it to a user: 1. Navigate to the [Users & Licenses page](https://app.localstack.cloud/workspace/members). 2. Identify the target user in **Workspace Members**. 3. Select the appropriate **Member Role**. 4. Save the configuration to activate the license for that identity. If you have joined a workspace, you need to be assigned a license by the workspace administrator. :::note LocalStack cannot activate the Snowflake emulator unless the token belongs to a user or workspace with an assigned license, even if the Auth Token itself is valid. ::: To view your own assigned license, visit the [My License page](https://app.localstack.cloud/workspace/my-license). For more details on inviting users, assigning licenses, or managing roles, see [Users and Licenses](/aws/organizations-admin/managing-users-licenses/). ## Configure your token Authentication requirements vary based on your chosen execution method. ### lstk The `lstk` CLI automates the authentication lifecycle. On initial execution, it triggers a browser-based OAuth flow and stores the resulting token in your system keyring. No manual environment variable configuration is required. ```bash lstk start ``` You can alternatively set the `LOCALSTACK_AUTH_TOKEN` environment variable in your shell session; `lstk` uses it when no keyring token is present. See [Authentication](/aws/developer-tools/running-localstack/lstk/authentication/) for the full resolution order. ### Docker and Docker Compose For direct container execution, inject the token as an environment variable. For complete startup examples, see the [Docker Compose](/snowflake/getting-started/installation/#docker-compose) and [Docker CLI](/snowflake/getting-started/installation/#docker-cli) installation options. **Docker CLI:** ```bash -e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN} ``` **Docker Compose:** ```yaml environment: - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN} ``` ### CI environments CI environments require a CI Auth Token. Developer Auth Tokens cannot be used in CI. CI Auth Tokens are available on the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens) and are configured similarly to Developer Auth Tokens. For complete examples, see the [CI Integration guide](/snowflake/getting-started/ci-cd/). ## Verify activation Verify the activation status by querying the Snowflake emulator's session endpoint: ```bash curl -d '{}' snowflake.localhost.localstack.cloud:4566/session ``` ```powershell Invoke-WebRequest -Method POST -Body '{}' -Uri http://snowflake.localhost.localstack.cloud:4566/session ``` ```json title="Output" { "success": true } ``` You can also check the container logs for a message indicating successful license activation: ```bash [...] Successfully activated license ``` Otherwise, check the [Troubleshooting](#troubleshooting) section below. ## Rotate a token Rotate an Auth Token if it has been exposed, shared accidentally, or stored in a place where it should not be. Go to the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens) and select the reset option for the affected token. After rotation, update every local shell, container configuration, or CI secret that used the old token. ## Troubleshooting The Snowflake emulator requires a successful license activation to start. If activation fails, the container exits and prints an error message similar to: ```bash =============================================== License activation failed! Reason: The credentials defined in your environment are invalid. Please make sure to set the LOCALSTACK_AUTH_TOKEN variable to a valid auth token. You can find your Auth Token in the LocalStack web app https://app.localstack.cloud. Due to this error, LocalStack has quit. The Snowflake emulator can only be used with a valid license. ``` The most common causes are listed below. ### Missing credentials You need to provide an Auth Token to start the Snowflake emulator. You can find your Auth Token on the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens) in the LocalStack Web Application. If you are using `lstk`, run `lstk login` to authenticate through a browser-based flow, or set the `LOCALSTACK_AUTH_TOKEN` environment variable directly. ### Invalid license The issue may occur if there is no valid license linked to your account (for example, because it has expired), or if the license has not been assigned to your user. You can check your license status in the LocalStack Web Application on the [My License page](https://app.localstack.cloud/workspace/my-license). If your license does not grant access to the Snowflake emulator, [contact us](https://localstack.cloud/contact/) to upgrade. ### License server unreachable LocalStack initiates offline activation when the license server is unreachable, requiring re-activation every 24 hours. Log output may indicate issues with your machine resolving the LocalStack API domain, which can be verified using a tool like `dig`: ```bash dig api.localstack.cloud ``` If the result shows a status other than `status: NOERROR`, your machine is unable to resolve this domain. Certain corporate DNS servers may filter requests to specific domains. Reach out to your network administrator to safelist the `localstack.cloud` domain. If you continue to have problems with license activation, or if the steps above do not help, do not hesitate to [contact us](https://localstack.cloud/contact/). ## Next steps After configuring your Auth Token, continue to the [Local Development guide](/snowflake/getting-started/local-development/) to start the Snowflake emulator and run your first query. # CI Integration > Use LocalStack for Snowflake in CI pipelines to run integration tests against a local Snowflake-compatible emulator. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction LocalStack for Snowflake helps you run integration tests in CI against an emulated Snowflake instance. Your pipeline starts the Snowflake emulator inside the CI job, connects the Snowflake CLI or your test harness (dbt, Airflow, etc.), runs tests against the local endpoint, and then discards the environment when the job ends. ## How LocalStack works in CI A typical CI job with LocalStack for Snowflake follows this flow: 1. Check out your application code. 2. Start the Snowflake emulator in the CI runner. 3. Configure a CI Auth Token through the CI provider's secret manager. 4. Connect with the Snowflake CLI or your test harness. 5. Run integration tests against the LocalStack endpoint. 6. Collect logs, test reports, and artifacts from the job. This gives every pipeline run a fresh Snowflake-compatible environment without creating cloud resources in a real Snowflake account. ## What changes from local development CI runs are usually more constrained than local development: - Use a dedicated **CI Auth Token** instead of a personal Developer Token. - Store `LOCALSTACK_AUTH_TOKEN` as a protected CI secret. - Start LocalStack non-interactively as part of the job. - Treat the LocalStack container as ephemeral unless your workflow explicitly saves state. - Export logs and test reports before the runner shuts down. Docker and Docker Compose are still common ways to run containers inside CI runners, but they are not CI tools by themselves. For container startup details, see the [Installation guide](/snowflake/getting-started/installation/#container-and-orchestration-tools). ## Choose your CI provider Start with the CI system you use. These snippets show the basic emulator startup shape for each provider. :::note For brevity, these snippets show only the LocalStack startup shape. They assume your CI Auth Token is already exposed to the job as the `LOCALSTACK_AUTH_TOKEN` environment variable. Store it as a secret in your CI provider before running them, and see [Authentication in CI](#authentication-in-ci) below. ::: ```yaml - name: Start LocalStack for Snowflake run: | npm install -g @localstack/lstk lstk start --type snowflake --non-interactive env: LOCALSTACK_AUTH_TOKEN: ${{ secrets.LOCALSTACK_AUTH_TOKEN }} ``` ```yaml version: '2.1' jobs: localstack-snowflake-test: machine: image: ubuntu-2204:current steps: - checkout - run: name: Install lstk command: npm install -g @localstack/lstk - run: name: Start LocalStack for Snowflake command: lstk start --type snowflake --non-interactive ``` ```yaml stages: - test localstack-snowflake-test: stage: test image: node:20 services: - docker:dind script: - npm install -g @localstack/lstk - lstk start --type snowflake --non-interactive ``` `lstk start --non-interactive` blocks until the emulator reports healthy, or exits non-zero if it fails to start within the readiness deadline (60 seconds by default). This means no separate wait step is required. Override the deadline with `--timeout` or [`LSTK_STARTUP_TIMEOUT`](/aws/developer-tools/running-localstack/lstk/automation/#environment-variables) if your runner needs more time to pull the image. You can also start the emulator directly with Docker or Docker Compose inside your CI job — see the [Installation guide](/snowflake/getting-started/installation/#container-and-orchestration-tools) for the container configuration, and the existing [Continuous Integration](/snowflake/integrations/continuous-integration/) guide for complete Docker-based examples. ## Authentication in CI CI environments should use a CI Auth Token. Create one from the [Auth Tokens page](https://app.localstack.cloud/workspace/auth-tokens), then store it as `LOCALSTACK_AUTH_TOKEN` in your CI provider's secret manager. Do not commit tokens to your repository or write them directly into workflow files. For more details on token types and rotation, see the [Auth Token guide](/snowflake/getting-started/auth-token/). ## State in CI Most CI jobs should start with a clean LocalStack instance. A fresh instance makes test runs reproducible and avoids hidden dependencies between jobs. If your pipeline needs state across jobs or workflow stages, see [State Management](/snowflake/capabilities/state-management/) to save and restore named LocalStack state snapshots. ## Next steps After choosing your CI provider, continue to [AI & Agent Workflows](/snowflake/getting-started/ai-workflows/) to learn how AI coding assistants can help generate, run, and test Snowflake SQL against LocalStack. # FAQ > Frequently asked questions about LocalStack for Snowflake ## Core FAQs ### Are Snowflake v2 APIs supported? Yes, the LocalStack for Snowflake supports the Snowflake v2 SQL API (`/api/v2/*` endpoints), as well as the legacy v1 SQL API (which is still being used by a large portion of Snowflake client libraries and SDKs) ### Why are my Snowflake tests failing? LocalStack for Snowflake is now GA. If your tests are failing, it could be due to a lack of support for certain Snowflake features. We recommend checking the [function coverage](/snowflake/sql-functions/) to see the list of supported SQL functions and [feature coverage](/snowflake/feature-coverage/) to see the list of supported features. If you encounter any issues, you can connect with us for [support](#support-faqs). ### Why does the LocalStack for Snowflake run on `snowflake.localhost.localstack.cloud`? The LocalStack for Snowflake operates on `snowflake.localhost.localstack.cloud`. This is a DNS name that resolves to a local IP address (`127.0.0.1`) to make sure the connector interacts with the local APIs. In addition, we also publish an SSL certificate that is automatically used inside LocalStack, in order to enable HTTPS endpoints with valid certificates. Note: In case you are deploying the LocalStack for Snowflake in a Kubernetes cluster or some other non-local environment, you may need to add an entry to the `/etc/hosts` file of any client machine or Kubernetes pod that attempts to connect to the LocalStack for Snowflake pod via the `snowflake.localhost.localstack.cloud` domain name. ## Integration FAQs ### How do I enable detailed debug logs? You can set the `SF_LOG=trace` environment variable in the Snowflake container to enable detailed trace logs that show all the request/response message. If you're using `lstk`, define an environment profile in your `config.toml` and reference it from your container block: ```toml [[containers]] type = "snowflake" port = "4566" env = ["debug"] [env.debug] DEBUG = "1" SF_LOG = "trace" ``` ```bash lstk start ``` If you're using `docker-compose`, simply add these variables to the `environment` section of the YAML configuration file instead. ### The `snowflake.localhost.localstack.cloud` hostname doesn't resolve on my machine, what can I do? On some systems, including some newer versions of MacOS, the domain name `snowflake.localhost.localstack.cloud` may not resolve properly. If you are encountering network issues and your Snowflake client drivers are unable to connect to the emulator, you may need to manually add the following entry to your `/etc/hosts` file: ```bash 127.0.0.1 snowflake.localhost.localstack.cloud ``` ### Which Docker image tag should I use for LocalStack for Snowflake? As of the LocalStack for Snowflake 2026.05.0 release, the published Docker image tags follow the same policy used across the wider LocalStack image set: | Tag | Updated when | Recommended for | |---|---|---| | `latest` / `stable` | Tagged releases only (e.g. `2026.05.0`) | Most users, stable, release-quality builds | | `dev` | Every merged commit on `main` (the main development branch) | Users who need the latest unreleased changes | | `YYYY.MM.patch` (e.g. `2026.05.0`) | Never (pinned) | Fully reproducible environments where no changes are acceptable | Previously, `latest` tracked untagged changes from `main`. It now mirrors `stable` and is only updated on official tagged releases. If you were relying on `localstack/snowflake:latest` to pick up the most recent unreleased changes (for example, in CI or Docker Compose), switch to the `dev` tag instead. The `nightly` tag is no longer published for LocalStack for Snowflake. Use the `dev` tag to track untagged changes from `main`. ## Support FAQs ### How can I get help or support with LocalStack for Snowflake? If you're experiencing an issue with LocalStack for Snowflake, read our [Help & Support Guide](https://docs.localstack.cloud/snowflake/help-support/) for developers and enterprise teams. # Installation > Install LocalStack for Snowflake with lstk, Docker, or Docker Compose. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction LocalStack provides multiple installation paths depending on your development environment and requirements. We recommend a CLI-based installation for the most consistent local startup experience. Use [`lstk`](#lstk) to install, authenticate, and start LocalStack for Snowflake with minimal setup. LocalStack for Snowflake features require an [Auth Token](/snowflake/getting-started/auth-token/) to activate your running instance. `lstk` handles authentication through a browser-based login flow, while Docker and CI workflows can use `LOCALSTACK_AUTH_TOKEN`. ## lstk `lstk` is a lightweight CLI for LocalStack that manages the authentication and container lifecycle in a single workflow. **Requirement:** You must have a working [Docker installation](https://docs.docker.com/get-docker/) before proceeding. ### Install lstk ```bash brew install localstack/tap/lstk ``` ```bash npm install -g @localstack/lstk ``` Download the binary for your platform from the [GitHub Releases](https://github.com/localstack/lstk/releases) and add it to your `PATH`. ### Start lstk ```bash lstk start ``` The first execution initiates a browser-based login flow and, in an interactive terminal, prompts you to pick which emulator to run — choose `s` for Snowflake. Your choice is written to `config.toml` and used as the default on subsequent runs. Subsequent starts use credentials stored in your system keyring. ### Already using lstk with a different default emulator? If your global `config.toml` already defaults to a different emulator (for example, AWS), target Snowflake for a specific project instead by creating a project-local `.lstk/config.toml`: ```toml # .lstk/config.toml [[containers]] type = "snowflake" port = "4566" ``` For more details, see the [lstk documentation](/aws/developer-tools/running-localstack/lstk/). ## Container and orchestration tools Use these methods when you need explicit container configuration or want to run LocalStack alongside other orchestrated services. For everyday local development, `lstk` is usually simpler — and `lstk` is also a good fit for CI, see [CI Integration](/snowflake/getting-started/ci-cd/) for `lstk`-based CI examples. ### Docker Compose Use Docker Compose when you want a reusable configuration file that can be shared across a team or checked into a project repository. Create a `docker-compose.yml` with the following configuration: ```yaml showLineNumbers services: localstack-snowflake: container_name: '${LOCALSTACK_DOCKER_NAME:-localstack-snowflake}' image: localstack/snowflake ports: - '127.0.0.1:4566:4566' # LocalStack Gateway - '127.0.0.1:4510-4559:4510-4559' # external services port range - '127.0.0.1:443:443' # LocalStack HTTPS Gateway environment: # Activate LocalStack for Snowflake: https://docs.localstack.cloud/snowflake/getting-started/auth-token/ - LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} # required - DEBUG=${DEBUG:-0} - PERSISTENCE=${PERSISTENCE:-0} volumes: - '${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack' - '/var/run/docker.sock:/var/run/docker.sock' ``` Execute `docker compose up` to start. ### Docker CLI Use the Docker CLI for one-off starts or when you want to test a container configuration before moving it into Compose: ```bash docker run \ --rm -it \ --name localstack-snowflake \ -p 127.0.0.1:4566:4566 \ -p 127.0.0.1:4510-4559:4510-4559 \ -p 127.0.0.1:443:443 \ -e LOCALSTACK_AUTH_TOKEN=${LOCALSTACK_AUTH_TOKEN:?} \ -v /var/run/docker.sock:/var/run/docker.sock \ localstack/snowflake ``` :::note The Docker Compose and Docker CLI examples above use the same runtime settings: - The `4566` port exposes the LocalStack Gateway, including the Snowflake-compatible SQL API. - The `4510-4559` range exposes external service ports used by services that bind additional endpoints. - The `443` port exposes the LocalStack HTTPS Gateway. - Docker reuses a local image if one already exists. Pull explicitly or pin an image tag, such as `localstack/snowflake:`, when you need reproducible CI or team environments. - Configuration variables can be prefixed with `LOCALSTACK_` in Docker. For instance, setting `LOCALSTACK_PERSISTENCE=1` is equivalent to `PERSISTENCE=1`. For more details, see the general [Docker images](/aws/customization/other-installations/docker-images/) and [networking](/aws/customization/networking/) documentation, which applies across LocalStack products, and the [Snowflake configuration](/snowflake/capabilities/configuration/) reference. ::: ## Graphical user interfaces (GUIs) ### LocalStack Desktop Manage local instances via a standalone desktop application. [Download LocalStack Desktop here](https://app.localstack.cloud/download). ## Troubleshooting Installation issues typically fall into one of three areas: getting your chosen install method working, activating your license, or reaching the emulator over the network. ### lstk If you installed via [`lstk`](#lstk) and LocalStack fails to start, authenticate, or pull its image, see [lstk troubleshooting](/aws/developer-tools/running-localstack/lstk/faq-and-troubleshooting/#troubleshooting). For first-run authentication (browser login, keyring tokens, or `LOCALSTACK_AUTH_TOKEN` in CI), refer to [Authentication](/aws/developer-tools/running-localstack/lstk/authentication/) and the [Auth Token](/snowflake/getting-started/auth-token/) guides. ### Docker Compose and Docker CLI If you started LocalStack with [Docker Compose](#docker-compose) or the [Docker CLI](#docker-cli): - **License or credential errors**: see [Auth Token troubleshooting](/snowflake/getting-started/auth-token/#troubleshooting). - **Hostname resolution or network issues**: see the [Getting Started FAQ](/snowflake/getting-started/faq/). Ensure you have exported `LOCALSTACK_AUTH_TOKEN` in your shell before running `docker compose up` or `docker run`. ### View logs Stream container logs using the command that matches your install method: ```bash lstk logs ``` For `lstk` CLI diagnostics (separate from container logs), see [Logging](/aws/developer-tools/running-localstack/lstk/automation/#logging). ```bash docker compose logs -f localstack-snowflake ``` ```bash docker logs -f localstack-snowflake ``` Use the container name from your `docker run --name` flag if you set one. ### Network connectivity If your Snowflake client cannot reach the emulator after installation, see the [Getting Started FAQ](/snowflake/getting-started/faq/#the-snowflakelocalhostlocalstackcloud-hostname-doesnt-resolve-on-my-machine-what-can-i-do). ## Next steps Now that you've completed installation, proceed to the [Auth Token guide](/snowflake/getting-started/auth-token/) to activate LocalStack and prepare your environment for local development. # Local Development > Start LocalStack for Snowflake, connect the Snowflake CLI, and run your first query. ## Introduction This guide walks you through starting the Snowflake emulator, connecting the Snowflake CLI, and running your first SQL query. You will perform the entire workflow on your local machine without a Snowflake account or consuming Snowflake credits. A successful walkthrough results in: - **A running emulator**: A local, Snowflake-compatible endpoint. - **A Snowflake CLI connection**: A `localstack` connection profile pointed at your local emulator. - **Sample data**: A database, schema, and table populated with sample records via a CSV upload. ## Prerequisites - [Docker](https://docs.docker.com/get-docker/) engine installed and running. - A [LocalStack account](https://app.localstack.cloud/sign-up) with a license that includes Snowflake features — see the [Auth Token guide](/snowflake/getting-started/auth-token/). - [Snowflake CLI](https://docs.snowflake.com/en/developer-guide/snowflake-cli/installation/installation) (`snow`) installed. If you haven't installed LocalStack yet, follow the [installation guide](/snowflake/getting-started/installation/) to get started. ## Step 1: Install and start the Snowflake emulator Start the Snowflake emulator: ```bash lstk start ``` The first run triggers a browser-based authentication flow. After authentication, `lstk` pulls the LocalStack for Snowflake image and initializes the container, then prints a confirmation banner once the emulator is ready. ## Step 2: Connect the Snowflake CLI Create a connection profile that points the Snowflake CLI at your local emulator: ```bash snow connection add \ --connection-name localstack \ --user test_user \ --password TestPassword1! \ --account test-account \ --host snowflake.localhost.localstack.cloud ``` :::note You might be prompted for additional optional parameters, such as the connection port, database name, or warehouse name. These can be skipped. ::: Test the connection: ```bash snow connection test --connection localstack ``` ```bash title="Output" +--------------------------------------------------------+ | key | value | |-----------------+--------------------------------------| | Connection name | localstack | | Status | OK | | Host | snowflake.localhost.localstack.cloud | | Account | test-account | | User | test_user | +--------------------------------------------------------+ ``` A `Status` of `OK` confirms the Snowflake CLI can reach your local emulator. :::note This guide uses the Snowflake CLI, but LocalStack for Snowflake also works with [SnowSQL](/snowflake/integrations/snow-sql/), [DBeaver](/snowflake/integrations/dbeaver/), and the [LocalStack Web Application](/snowflake/developer-tools/user-interface/). See [Integrations](/snowflake/integrations/) for connection instructions for each. ::: ## Step 3: Run your first query Open an interactive SQL session using the `localstack` connection profile: ```bash snow sql --connection localstack ``` In this session, we'll create a student records database that demonstrates how to create databases, schemas, and tables, upload data using a stage, and query the results. Create the database and use it: ```sql CREATE DATABASE IF NOT EXISTS STUDENT_RECORDS_DEMO; USE DATABASE STUDENT_RECORDS_DEMO; ``` ```bash title="Output" +-----------------------------------------------------+ | status | |-----------------------------------------------------| | Database STUDENT_RECORDS_DEMO successfully created. | +-----------------------------------------------------+ ``` Create a schema and use it: ```sql CREATE SCHEMA IF NOT EXISTS PUBLIC; USE SCHEMA PUBLIC; ``` Create the `STUDENT_DATA` table: ```sql CREATE OR REPLACE TABLE STUDENT_DATA ( student_id VARCHAR(50), first_name VARCHAR(100), last_name VARCHAR(100), email VARCHAR(200), enrollment_date DATE, gpa FLOAT, major VARCHAR(100) ); ``` ```bash title="Output" +------------------------------------------+ | status | |------------------------------------------| | Table STUDENT_DATA successfully created. | +------------------------------------------+ ``` Create a file format and a stage for uploading files: ```sql CREATE OR REPLACE FILE FORMAT csv_format TYPE = CSV FIELD_DELIMITER = ',' SKIP_HEADER = 1 NULL_IF = ('NULL', 'null') EMPTY_FIELD_AS_NULL = TRUE; CREATE OR REPLACE STAGE student_data_stage FILE_FORMAT = csv_format; ``` Exit the SQL session (`!exit` or `Ctrl+D`), and create a `student_data.csv` file with sample records: ```csv student_id,first_name,last_name,email,enrollment_date,gpa,major S001,John,Smith,john.smith@university.edu,2023-08-15,3.75,Computer Science S002,Alice,Johnson,alice.johnson@university.edu,2023-08-15,3.92,Mathematics S003,Bob,Williams,bob.williams@university.edu,2022-08-15,3.45,Engineering S004,Carol,Brown,carol.brown@university.edu,2024-01-10,3.88,Physics S005,David,Davis,david.davis@university.edu,2023-08-15,2.95,Biology ``` Upload the CSV file to the stage: ```bash snow sql --connection localstack \ --query "PUT file://student_data.csv @student_data_stage AUTO_COMPRESS=TRUE;" ``` :::note Adjust the file path to the location of your `student_data.csv` file. ::: ```bash title="Output" source |target |source_size|target_size|source_compression|target_compression|status |message| ----------------+-------------------+-----------+-----------+------------------+------------------+--------+-------+ student_data.csv|student_data.csv.gz| 425| 262|NONE |GZIP |UPLOADED| | ``` Load the data from the stage into the table, then query it: ```bash snow sql --connection localstack --query " COPY INTO STUDENT_DATA FROM @student_data_stage ON_ERROR = 'CONTINUE'; SELECT COUNT(*) AS total_students FROM STUDENT_DATA; " ``` ```bash title="Output" +----------------+ | TOTAL_STUDENTS | |----------------| | 5 | +----------------+ ``` The Snowflake CLI executed every statement against the locally emulated Snowflake instance. Because no actual Snowflake resources are created, you won't consume any real Snowflake credits. ## Step 4: Inspect resources View the state of your local database via the [LocalStack Web Application](https://app.localstack.cloud/). Navigate to the **Snowflake** tab to inspect your running resources using the **SQL Worksheet**, which provides syntax highlighting, autocomplete, and a resource tree for your databases, schemas, and tables. ![Running SQL queries using LocalStack Web Application](/images/snowflake/snowflake-web-ui.png) For more on the Web Application's Snowflake tooling, see [User Interface](/snowflake/developer-tools/user-interface/). ## Step 5: Clean up Stop your LocalStack container to remove all emulated resources. LocalStack is ephemeral by default; stopping the instance clears the state. ```bash lstk stop ``` To persist your database, schema, and table data across restarts, see [State Management](/snowflake/capabilities/state-management/). Remove the local file you created in this guide: ```bash rm student_data.csv ``` ## Explore more - **Load data from cloud storage**: Use [Storage Integrations](/snowflake/features/storage-integrations/) (currently supporting AWS S3) or a script (see [Snowflake Drivers](/snowflake/integrations/snowflake-drivers/)). - **Automate data ingestion**: Configure [Snowpipe](/snowflake/features/snowpipe/) for automated data ingestion from external sources. - **Use your favorite tools**: Continue developing against LocalStack for Snowflake with your preferred [integrations](/snowflake/integrations/). ## Next steps You have successfully connected the Snowflake CLI and run your first query against a local Snowflake-compatible emulator. Proceed to the [CI/CD guide](/snowflake/getting-started/ci-cd/) to learn how to integrate LocalStack for Snowflake into your automated continuous integration pipelines. # Overview > How to get Help and Support for LocalStack for Snowflake. LocalStack for Snowflake provides multiple support options to help you troubleshoot issues, understand features, and integrate the platform into your workflows. The level of support available depends on your subscription plan. Our support team can assist with: - Troubleshooting LocalStack-specific issues - Understanding LocalStack features and functionality - Integration guidance for LocalStack in your application - Best practices for working with LocalStack services For non-technical inquiries, such as billing or account-related questions, contact us at [support@localstack.cloud](mailto:support@localstack.cloud). ## Support options LocalStack offers different support plans with varying levels of access, response times, and communication channels. Use the following sections to find the support option that best fits your needs: - [Get Help](/snowflake/help-support/get-help/): Learn which support channel to use based on your situation. - [Report an Issue](/snowflake/help-support/report-issue/): Submit a support request with the required details. - [Support Plans](/snowflake/help-support/support-plans/): Compare available plans and included features. - [Enterprise Support](/snowflake/help-support/enterprise-support/): Explore dedicated support options for enterprise customers. :::note Support is currently provided in `English` only. As an international team, this ensures clear and consistent communication across all regions. ::: # Enterprise Support > How to request Enterprise Support for LocalStack for Snowflake. ## Introduction Enterprise support offers organizations personalized resources, direct communication channels with the LocalStack team, and flexible service level agreements (SLAs) to meet specific business requirements. The key components of our enterprise support offering include: - **Direct Slack Connect or Teams Channel**: A dedicated Slack Connect or Teams channel is available to maintain a direct communication link with the LocalStack engineering team. This setup ensures quick issue resolution and streamlined collaboration, improving overall service efficiency. - **Dedicated Customer Success Manager (CSM) and Technical Account Manager (TAM)**: Enterprise customers are assigned a CSM and SA. The CSM acts as a strategic advisor to help fully utilize LocalStack's offerings, while the SA provides expert technical assistance in designing and optimizing solutions tailored to your needs. - **Custom Service Level Agreements (SLAs)**: Tailor your service levels and response times to meet your organization's requirements. Custom SLAs can be negotiated to align with your business objectives and ensure optimal system performance. - **Support Ticketing Portal**: Access the [support ticketing portal](https://support.localstack.cloud/portal) to view, create, and respond to support tickets, ensuring organized tracking of all queries. - **Real-time Chat Support**: Real-time Level 1 (L1) chat support is available during support operating hours. While immediate resolutions are prioritized, complex issues may require additional time and resources for thorough handling. ## Customer portal A customer portal is a home behind a login where customers can view, open, and reply to their support tickets. Currently, the **customer portal** is only **available to Enterprise customers**. You can find the customer portal here: [https://support.localstack.cloud/portal](https://support.localstack.cloud/portal). ![Customer portal for enterprise support](/images/aws/customer-portal.png) ## Signing up for Enterprise Support If you are a member of an organization with an enterprise LocalStack subscription, you will receive an invitation to create an account and join the LocalStack Support Portal via email. Follow the instructions in the email and set up your account by clicking on the **Sign up** button. You will be asked to create a password. Once you do so, you will be able to log in and start using the customer portal to create, view, and engage with tickets. ## Creating a Support Ticket You can open a new ticket with LocalStack support by going to the **Create a Support Ticket** link. You will be redirected to a form where you will have to provide certain information to file a new support ticket. ![Filing a support ticket](/images/aws/file-a-support-ticket.png) The form consists of two parts. One is basic information, which is mandatory to fill out, and additional information, which adds more context to your issue but is not mandatory. Once all the mandatory fields are filled out, you can create a new support ticket by clicking on the Submit button. When the ticket is submitted, it's reported to LocalStack support, who will get back to you on that query as soon as possible. A ticket will show up in the ticket list as soon as it’s submitted. ### Basic Information You need to fill out the following fields, which are mandatory to open a new ticket: - **Type**: Choose the type of your query from the following options: - **Issue**: Select this when you are facing an issue using LocalStack. - **General inquiry**: Select this when you have a general question regarding LocalStack. - **Feature request**: Select this when you are looking for a feature that is not yet implemented in LocalStack. - **Ticket name**: Provide a descriptive name for the ticket that summarizes your inquiry. - **Description**: Provide a comprehensive description of your inquiry, explaining all the details that will help us understand your query. ### Additional Information - **CI Issue?** If the query is related to a CI issue, select the one that best fits your query from the dropdown. - **Operating system**: From the dropdown, select the operating system you are using. - **Affected Services**: From the dropdown, select the AWS service that is affected in your query. - **File upload**: Here you can provide any additional files that you believe would be helpful for LocalStack support (e.g., screenshots, log files, etc.). # Get Help > Choose the right support channel for your LocalStack issue or question. If you need help with LocalStack, choosing the right support channel can help you get a faster and more effective response. This guide explains when to use each available support option. ## Choose the right support channel ### Community Slack Use the [LocalStack Slack Community](https://localstack.cloud/slack) for: - Quick questions - General guidance - Discussions with other users and maintainers Best for: - Early-stage troubleshooting - Learning from others’ experiences :::note Community support is provided on a best-effort basis and is not guaranteed. ::: ### Support email Contact [support@localstack.cloud](mailto:support@localstack.cloud) for: - Reporting bugs - Requesting new features - Technical questions - Account-related inquiries - Issues requiring direct support Best for: - Reproducible issues - Tracking known problems or feature requests - Ongoing issues that require follow-up - Questions that are not suited for public channels ### Web application chat Use the [LocalStack Web Application](https://app.localstack.cloud/) chat for: - Submitting support requests - Asking technical questions directly To create a support request: 1. Open the LocalStack Web Application 2. Click the chat icon in the bottom right corner 3. Select **Technical Question** 4. Enter your details and submit Best for: - Direct interaction with the support team - Submitting issues without leaving the app ### Enterprise support channels Enterprise customers have access to additional support options, including: - Dedicated Slack or Teams channel - Support ticketing portal - Real-time chat support For more details, see [Enterprise Support](/snowflake/help-support/enterprise-support/). ## Before you reach out Before contacting support, we recommend: - Reviewing the documentation and FAQs - Verifying your configuration settings - Checking logs for errors or warnings - Ensuring your setup meets system requirements Providing clear and complete information helps us respond more quickly and effectively. # Report an Issue > Submit a support request with the right information to help resolve your issue faster. If you’re experiencing an issue with LocalStack for Snowflake, providing clear and complete information helps our team identify and resolve the problem more quickly. This guide explains what to check before reporting an issue and what details to include in your request. ## Before reporting an issue Before reaching out, we recommend: - Reviewing the documentation and FAQs - Verifying your configuration settings - Ensuring your setup meets system requirements - Checking logs for errors or warnings Many common issues can be resolved quickly by validating your setup and reviewing existing resources. ## What to include To help us troubleshoot your issue efficiently, please include the following information: - **Logs** Snowflake emulator container logs with the environment variables `SF_LOG=trace` and `DEBUG=1` enabled - **Query (if applicable)** The query that triggered the issue - **Client details** Client tool or driver used - **Connection parameters** Excluding sensitive information - **Additional logs (if available)** Client tool or driver logs Providing detailed information upfront helps reduce back-and-forth and speeds up resolution time. ## How to submit an issue You can report an issue using one of the following methods: ### Web application 1. Open the [LocalStack Web Application](https://app.localstack.cloud/) 2. Click the chat icon in the bottom right corner 3. Select **Technical Question** 4. Enter the required details and submit ### Support email Send your request to [support@localstack.cloud](mailto:support@localstack.cloud). ## What happens next After submitting your request: - Our support team will review your issue - You may be contacted for additional details - Issues are prioritized based on urgency and impact Response times depend on your support plan. For details, see [Support Plans](/snowflake/help-support/support-plans/). # Support Plans > Compare LocalStack support plans, features, and response expectations. LocalStack offers multiple support plans with different levels of access, response times, and communication channels. Choose the plan that best fits your needs based on the level of support and responsiveness required. ## Plan overview | **Plan** | **Support Tier** | | --- | --- | | Trial | Standard Support | | Base | Priority Support | | Enterprise | Enterprise Support | ## Feature comparison | **Features** | **Standard** | **Priority** | **Enterprise** | | --- | --- | --- | --- | | Documentation access | ✅ | ✅ | ✅ | | Community support | ✅ | ✅ | ✅ | | Operational support | ✅ | ✅ | ✅ | | 1:1 technical support | ✅ | ✅ | ✅ | | Screen sharing sessions | | ✅ | ✅ | | Third-party tools support | | Limited | Limited | | Faster response times | | ✅ | ✅ | | Real-time chat support | | | ✅ | | Support ticketing portal | | | ✅ | | Service Level Agreements (SLAs) | | | ✅ | | Direct Slack/Teams channel | | | ✅ | | Dedicated CSM & TAM | | | ✅ | ## Response expectations ### Standard support (Trial) - Best-effort support - No guaranteed response times - Typical response time: **24–48 hours** during business hours ### Priority support (Base) - **First response:** within 24 hours - **Follow-up responses:** within 24 hours - Responses provided during business hours ### Enterprise support Enterprise support includes custom SLAs and dedicated communication channels. For full details, see [Enterprise Support](/snowflake/help-support/enterprise-support/). ## Support scope LocalStack support focuses on helping you use and integrate LocalStack effectively. ### Included - Troubleshooting LocalStack-specific issues - Guidance on features and functionality - Integration support for LocalStack workflows - Best practices for using LocalStack services ### Limitations Support does **not** include: - **Customer-specific code** Debugging or modifying custom applications, scripts, or workflows - **Third-party tools (Standard plan)** External tools, plugins, or development environments - **Advanced third-party troubleshooting (Priority plan)** Only basic integration guidance is provided for officially supported tools - **Snowflake in production** Support is limited to LocalStack’s emulated services ## Support channels by plan | **Channel** | **Standard** | **Priority** | **Enterprise** | | --- | --- | --- | --- | | Slack community | ✅ | ✅ | ✅ | | GitHub Discussions | ✅ | ✅ | ✅ | | Support email | ✅ | ✅ | ✅ | | Web application chat | ✅ | ✅ | ✅ | | Support ticketing portal | | | ✅ | | Dedicated Slack/Teams channel | | | ✅ | ## Support hours Support is available: - Monday to Friday - 6:00 AM – 9:00 PM UTC , CET, and ET time zones Excludes: - January 1 - May 1 - November 1 - December 24, 25, and 31 # Overview > Use your favorite development tools with LocalStack for Snowflake. import SectionCards from '../../../../components/SectionCards.astro'; LocalStack for Snowflake supports a wide range of tools and integrations from the data development ecosystem. This section covers tools that are officially supported and tested with LocalStack for Snowflake. # Airflow > Use Airflow to run local ETL jobs against the Snowflake emulator ## Introduction Apache [Airflow](https://airflow.apache.org) is a platform for running data-centric workflows and scheduled compute jobs. LocalStack supports the [AWS Managed Workflows for Apache Airflow](https://docs.localstack.cloud/user-guide/aws/mwaa/) (MWAA) service to run Airflow jobs locally. You can use Airflow to interact with the LocalStack Snowflake emulator and run ETL (Extract-Transform-Load) jobs, using the Airflow `SnowflakeOperator` for running queries against Snowflake. On this page we outline how to set up the connection between local Airflow and the Snowflake emulator. ## Create an Airflow environment via MWAA in LocalStack In order to create an Airflow environment in local MWAA, we can use the [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command: ```bash showLineNumbers lstk aws s3 mb s3://my-mwaa-bucket lstk aws mwaa create-environment --dag-s3-path /dags \ --execution-role-arn arn:aws:iam::000000000000:role/airflow-role \ --network-configuration {} \ --source-bucket-arn arn:aws:s3:::my-mwaa-bucket \ --airflow-version 2.6.3 \ --name my-mwaa-env ``` ## Create an Airflow DAG script that connects to LocalStack Snowflake We can then create a local file `my_dag.py` with the Airflow DAG definition, for example: ```python showLineNumbers import datetime import json from airflow import settings from airflow.models import Connection, DAG from airflow.providers.snowflake.operators.snowflake import SnowflakeOperator # prepare session and connection info session = settings.Session() conn_id = "c1" try: # try to look up local Snowflake connection in the session session.query(Connection).filter(Connection.conn_id == conn_id).one() except Exception: # create new Snowflake connection, if it doesn't exist yet conn = Connection( conn_id=conn_id, conn_type="snowflake", login="test", password="test", extra=json.dumps({"account": "test", "host": "snowflake.localhost.localstack.cloud", "port": 4566}) ) session.add(conn) session.commit() # create DAG my_dag = DAG( "sf_dag1", start_date=datetime.datetime.utcnow(), default_args={"snowflake_conn_id": conn_id}, catchup=False, ) # add Snowflake operator to DAG sf_task_1 = SnowflakeOperator( task_id="sf_query_1", dag=my_dag, sql=""" CREATE TABLE IF NOT EXISTS test(id INT); COPY INTO test (id) VALUES (1), (2), (3); SELECT * FROM test """, ) ``` ### Patching the `SnowflakeOperator` in the DAG script In order to use the `SnowflakeOperator` in your Airflow DAG, a small patch is required in the code. The code listings below contain the patch for different Airflow versions - simply copy the relevant snippet and paste it into the top of your DAG script (e.g., `my_dag.py`). **Airflow version 2.6.3 and above**: ```python showLineNumbers # --- # patch for local Snowflake connection, for Airflow 2.6.3 and above from airflow.providers.snowflake.hooks.snowflake import SnowflakeHook def _get_conn_params(self): result = self._get_conn_params_orig() conn = self.get_connection(self.snowflake_conn_id) extra_dict = conn.extra_dejson if extra_dict.get("host"): result["host"] = extra_dict["host"] if extra_dict.get("port"): result["port"] = extra_dict["port"] return result SnowflakeHook._get_conn_params_orig = SnowflakeHook._get_conn_params SnowflakeHook._get_conn_params = _get_conn_params # --- # ... rest of your DAG script below ... ``` **Airflow version 2.9.2 and above**: ```python showLineNumbers # --- # patch for local Snowflake connection, for Airflow 2.9.2 / 2.10.1 from airflow.providers.snowflake.hooks.snowflake import SnowflakeHook @property def _get_conn_params(self): result = self._get_conn_params_orig conn = self.get_connection(self.snowflake_conn_id) extra_dict = conn.extra_dejson if extra_dict.get("host"): result["host"] = extra_dict["host"] if extra_dict.get("port"): result["port"] = extra_dict["port"] return result SnowflakeHook._get_conn_params_orig = SnowflakeHook._get_conn_params SnowflakeHook._get_conn_params = _get_conn_params # --- # ... rest of your DAG script below ... ``` :::note In a future release, we're looking to integrate these patches directly into the LocalStack environment, such that users do not need to apply these patches in DAG scripts manually. ::: ## Deploying the DAG to Airflow Next, we copy the `my_dag.py` file to the `/dags` folder within the `my-mwaa-bucket` S3 bucket, to trigger the deployment of the DAG in Airflow: ```bash lstk aws s3 cp my_dag.py s3://my-mwaa-bucket/dags/ ``` You should then be able to open the Airflow UI (e.g., http://localhost.localstack.cloud:4510/dags) to view the status of the DAG and trigger a DAG run. # Continuous Integration > Get started with Snowflake emulator in continuous integration (CI) environments. import { Tabs, TabItem } from '@astrojs/starlight/components'; ## Introduction This guide explains how to set up the Snowflake emulator in a continuous integration (CI) environment. You can use the emulator to test your Snowflake integration without connecting to the real Snowflake instance. To install the Snowflake emulator, you need to set up the [`lstk` CLI](/snowflake/developer-tools/lstk/), which authenticates, pulls the Docker image, and starts the container for you. After starting the Docker container, an endpoint (`snowflake.localhost.localstack.cloud`) is made available to connect to the emulated Snowflake instance, which you can use to test your data integrations in CI environments. ## Configure the LocalStack CI Key Before you begin, set up the LocalStack CI key to activate the Snowflake emulator in your CI environment. LocalStack requires a CI Key for use in continuous integration (CI) or similar machine environments. Each instance startup in a CI or comparable environment consumes one CI token. To create a CI key, follow these steps: 1. Open the [LocalStack Web Application](https://app.localstack.cloud). 2. Click the [**CI Keys**](https://app.localstack.cloud/workspace/ci-keys) on the left navigation pane. 3. Navigate to the **Generate CI Key** tab, enter a description, and click **Generate CI Key**. 4. Copy the generated CI key and set it as the `LOCALSTACK_AUTH_TOKEN` environment variable in your CI environment. ## Setup the CI configuration The following examples demonstrate how to set up the emulator in GitHub Actions, CircleCI, and GitLab CI. ```yaml showLineNumbers name: LocalStack Test on: [ push, pull_request ] jobs: localstack-action-test: name: 'Test LocalStack GitHub Action' runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Start Snowflake emulator run: | npm install -g @localstack/lstk lstk start --type snowflake --timeout 90s echo "Startup complete" env: LOCALSTACK_AUTH_TOKEN: ${{ secrets.LOCALSTACK_AUTH_TOKEN }} ``` ```yaml showLineNumbers version: 2.1 jobs: example-job: machine: image: ubuntu-2004:2022.04.1 steps: - checkout - run: name: Start Snowflake emulator command: | npm install -g @localstack/lstk lstk start --type snowflake --timeout 90s echo "Startup complete" workflows: version: 2 build: jobs: - example-job ``` ```yaml showLineNumbers image: docker:20.10.16 stages: - test test: stage: test variables: DOCKER_HOST: tcp://docker:2375 DOCKER_TLS_CERTDIR: "" LOCALSTACK_AUTH_TOKEN: $LOCALSTACK_AUTH_TOKEN services: - name: docker:20.10.16-dind alias: docker command: ["--tls=false"] before_script: - apk update - apk add --no-cache nodejs npm - npm install -g @localstack/lstk script: - dind_ip="$(getent hosts docker | cut -d' ' -f1)" - echo "${dind_ip} localhost.localstack.cloud " >> /etc/hosts - DOCKER_HOST="tcp://${dind_ip}:2375" lstk start --type snowflake --timeout 90s ``` # DBeaver > Use DBeaver to interact with the Snowflake emulator ## Introduction [DBeaver](https://dbeaver.io/) is a free and open-source universal database tool for developers, database administrators, and analysts. DBeaver provides a wide range of features, such as executing SQL statements, viewing and editing data, managing database objects, and more. You can use DBeaver to interact with the Snowflake emulator using the same commands and syntax as the Snowflake service. With DBeaver, you can manage Snowflake resources locally, such as databases, schemas, tables, stages, and more. ## Configuring DBeaver In this guide, you will learn how to configure DBeaver to interact with the Snowflake emulator. ### Install DBeaver To install DBeaver, follow the instructions in the [official DBeaver documentation](https://dbeaver.io/download/). ### Create a new connection To create a new connection in DBeaver, follow these steps: - Open DBeaver. Go to the top menu, select **Database**, and choose **New Database Connection**. In the **Connect to database** window, pick **All** databases and search for **Snowflake**, then click **Next**. - In the **Connect to database** window, switch to the **Main** tab. Enter your Snowflake user details: - **Host**: `snowflake.localhost.localstack.cloud` - **Port**: `4566` - **Database**: `test` (case insensitive) - **User**: `test` - **Password**: `test` ![New connection in DBeaver](/images/snowflake/dbeaver-new-connection.png) - Click **Test Connection**. - If the connection test succeeds, click **Finish**. The Snowflake database will appear in DBeaver's Database Navigator. You can verify the connection by running a query to check the Snowflake version: `SELECT CURRENT_VERSION();` # dbt > Use dbt to interact with the Snowflake emulator ## Introduction [dbt (data build tool)](https://www.getdbt.com/) is a transformation workflow tool that enables data analysts and engineers to transform data in their warehouses by writing modular SQL. dbt handles version control, documentation, and modularity for data transformations. The Snowflake emulator supports dbt, allowing you to develop and test your dbt models locally without connecting to a production Snowflake instance. ## Configuring dbt In this guide, you will learn how to configure dbt to interact with the Snowflake emulator. ### Install dbt First, install dbt with the Snowflake adapter: ```bash pip install dbt-snowflake ``` ### Configure dbt Profile Create or modify your `profiles.yml` file (typically located in `~/.dbt/profiles.yml`) to include the connection details for the Snowflake emulator: ```yaml showLineNumbers localstack_snowflake: outputs: dev: account: localstack host: snowflake.localhost.localstack.cloud port: 4566 database: test password: test role: test schema: public threads: 1 type: snowflake user: test warehouse: test target: dev ``` ### Test the Connection To verify your dbt configuration is working correctly with the Snowflake emulator, run: ```bash dbt debug --profile localstack_snowflake ``` You should see output indicating a successful connection to the Snowflake emulator. ### Running dbt Commands Once configured, you can run standard dbt commands against the Snowflake emulator. ### Example dbt Model Here's a simple example of a dbt model which creates a table with a single row that you can use to test the integration: ```sql showLineNumbers -- models/example_model.sql {{ config(materialized='table') }} SELECT 1 as id, 'test' as name ``` ### Example Tests You can test your models using dbt's generic tests. Add the following to your `models/schema.yml`: ```yaml showLineNumbers version: 2 models: - name: example_model description: "A simple example model with generic tests" columns: - name: id description: "The primary key for this table" tests: - unique - not_null - name: name description: "The name field" tests: - not_null ``` You can run all models and tests with the following commands: ```bash # Run all models dbt run --profile localstack_snowflake # Run tests dbt test --profile localstack_snowflake ``` ### Project Structure A typical dbt project structure when working with the Snowflake emulator might look like this: ``` my_dbt_project/ ├── dbt_project.yml ├── models/ │ ├── example_model.sql │ └── schema.yml └── README.md ``` Example `dbt_project.yml`: ```yaml showLineNumbers name: 'my_dbt_project' version: '1.0.0' config-version: 2 profile: 'localstack_snowflake' model-paths: ["models"] test-paths: ["tests"] analysis-paths: ["analyses"] macro-paths: ["macros"] target-path: "target" clean-targets: - "target" - "dbt_packages" models: my_dbt_project: materialized: table ``` ## Best Practices 1. **Version Control**: Keep your dbt models and configurations in version control 2. **Testing**: Write tests for your models to ensure data quality 3. **Documentation**: Document your models using dbt's built-in documentation features 4. **Modularity**: Break down complex transformations into smaller, reusable models :::note It's a good practice to always test your dbt models locally with the Snowflake emulator before deploying to production, to save time and resources. ::: # Flyway > Use Flyway to interact with the Snowflake emulator ## Introduction [Flyway](https://flywaydb.org/) is an open-source database migration tool that simplifies the process of managing and applying database migrations. Flyway supports various databases, including Snowflake, allowing you to manage database schema changes, version control, and data migration in a structured and automated way. The Snowflake emulator can connect with Flyway, to apply database migrations, and manage database schema changes locally. ## Configuring Flyway In this guide, you will learn how to configure Flyway to interact with the Snowflake emulator. ### Install Flyway You can install Flyway on the [official Flyway website](https://flywaydb.org/download/). Download the Flyway Desktop & command-line tool and install it on your local machine. ### Create a new Flyway project To create a new Flyway project, follow these steps: * Open the Desktop application. * Click **New Project**. * Enter the project name and select the database type as **Snowflake**. * Click **Create project**. ### Connect Flyway to Snowflake To connect Flyway to the Snowflake emulator, follow these steps: * Click **Add target database +**. * Enter username as `test` and password as `test`. * Enter JDBC URL as `jdbc:snowflake://http://snowflake.localhost.localstack.cloud:4566/?db=test&schema=PUBLIC&JDBC_QUERY_RESULT_FORMAT=JSON`. * Click on **Test connection**. If the connection test succeeds, you can start applying database migrations using Flyway. # Pulumi > Use Pulumi to interact with the Snowflake emulator ## Introduction [Pulumi](https://pulumi.com/) is an Infrastructure-as-Code (IaC) framework that allows you to define and provision infrastructure using familiar programming languages. Pulumi supports a wide range of cloud providers and services, including AWS, Azure, Google Cloud, and more. The Snowflake emulator supports Pulumi, allowing you to define and provision Snowflake resources using the same commands and syntax as the Snowflake service. You can use Pulumi to create, update, and delete Snowflake resources locally, such as databases, schemas, tables, stages, and more. ## Configuring Pulumi In this guide, you will learn how to configure Pulumi to interact with the Snowflake emulator. ### Set up Snowflake provider To use Pulumi with the Snowflake emulator, you need to configure the Snowflake provider in your Pulumi configuration file. Create a blank Pulumi project, and add the following environment variables to your Pulumi stack: ```bash pulumi config set snowflake:account test pulumi config set snowflake:region test pulumi config set snowflake:username test pulumi config set snowflake:password test pulumi config set snowflake:host snowflake.localhost.localstack.cloud ``` You can install the Snowflake provider in any of the programming languages supported by Pulumi, such as Python, JavaScript, TypeScript, and Go. The following example shows how to install the Snowflake provider for your TypeScript project: ```bash npm install @pulumi/snowflake ``` ### Create Snowflake resources You can now use Pulumi to create Snowflake resources using the Snowflake provider. The following example shows how to create a Snowflake database using Pulumi: ```javascript showLineNumbers import * as snowflake from "@pulumi/snowflake"; const simple = new snowflake.Database("simple", { comment: "test comment", dataRetentionTimeInDays: 3, }); ``` ### Deploy the Pulumi configuration You can now deploy the Pulumi configuration to create the Snowflake resources locally. Run the following command to deploy the Pulumi configuration: ```bash pulumi up ``` The expected output should show the resources being created in the Snowflake emulator: ```bash Enter your passphrase to unlock config/secrets Previewing update (snowflake): Type Name Plan Info + pulumi:pulumi:Stack pulumi-snowflake-sample-snowflake create + └─ snowflake:index:Database simple create Resources: + 2 to create Do you want to perform this update? yes Updating (snowflake): Type Name Status Info + pulumi:pulumi:Stack pulumi-snowflake-sample-snowflake created (0.48s) 2 Resources: + 2 created Duration: 5s ``` # Snowflake CLI > Use Snowflake CLI to interact with the Snowflake emulator. ## Introduction Snowflake CLI is a command-line interface (CLI) for Snowflake. You can use Snowflake CLI to interact with the Snowflake emulator. Snowflake CLI provides a set of commands to manage and interact with Snowflake accounts, databases, warehouses, and more. You can connect Snowflake CLI to the Snowflake emulator using a connection profile. A connection profile is a set of parameters that define the connection to a Snowflake account. You can create, list, and test connection profiles using Snowflake CLI. ## Configuring Snowflake CLI In this guide, you will learn how to configure Snowflake CLI to interact with the Snowflake emulator using a `localstack` connection profile. :::note For installation instructions, follow the [official Snowflake documentation](https://docs.snowflake.com/en/developer-guide/snowflake-cli/installation/installation) for your operating system. This ensures you install the correct and most up-to-date version of the Snowflake CLI. ::: ### Create a connection profile To configure Snowflake CLI to interact with the Snowflake emulator, create a connection profile using the following command: ```bash snow connection add \ --connection-name localstack \ --user test \ --password test \ --account test \ --host snowflake.localhost.localstack.cloud ``` You might be prompted to enter more optional parameters, such as the connection port, database name, warehouse name, authentication method, and more. These are however optional and can be skipped. After a successful configuration, the `localstack` connection profile is ready to use. ### List your connection profiles To list all the connection profiles configured in Snowflake CLI, execute the following command: ```bash snow connection list ``` The output should be: ```bash +-----------------------------------------------------------------------------------+ | connection_name | parameters | |-----------------+-----------------------------------------------------------------| | localstack | {'account': 'test', 'user': 'test', 'password': '****', | | | 'host': 'snowflake.localhost.localstack.cloud'} | +-----------------------------------------------------------------------------------+ ``` ### Test the connection To test the connection to the Snowflake emulator, execute the following command: ```bash snow connection test --connection localstack ``` The output should be: ```bash +--------------------------------------------------------+ | key | value | |-----------------+--------------------------------------| | Connection name | localstack | | Status | OK | | Host | snowflake.localhost.localstack.cloud | | Account | test | | User | test | +--------------------------------------------------------+ ``` ### Run a query using the connection profile To run a query using the connection profile, execute the following command: ```bash snow sql --query "CREATE DATABASE mytestdb;" --connection localstack ``` You can see all the databases in your Snowflake emulator using the following command: ```bash snow sql --query "SHOW DATABASES;" --connection localstack ``` You can create a schema using the following commands: ```bash snow sql --query "CREATE SCHEMA mytestdb.mytestschema;" --connection localstack ``` # SnowSQL > Use SnowSQL to interact with the Snowflake emulator ## Introduction [SnowSQL](https://docs.snowflake.com/en/user-guide/snowsql.html) is a command-line client for Snowflake that allows you to interact with the Snowflake service using SQL commands. SnowSQL provides a wide range of features, such as executing SQL statements, loading data, unloading data, and more. The Snowflake emulator supports SnowSQL, allowing you to interact with the Snowflake emulator using the same commands and syntax as the Snowflake service. You can use SnowSQL to connect to the Snowflake emulator, execute SQL commands, and manage Snowflake resources locally, such as databases, schemas, tables, stages, and more. ## Configuring SnowSQL In this guide, you will learn how to configure SnowSQL to interact with the Snowflake emulator. ### Install SnowSQL To install SnowSQL, follow the instructions in the [official SnowSQL documentation](https://docs.snowflake.com/en/user-guide/snowsql-install-config.html). ### Start SnowSQL To start SnowSQL, execute the following command: ```bash showLineNumbers export SNOWSQL_PWD=test snowsql \ -a test \ -u test \ -h snowflake.localhost.localstack.cloud \ -p 4566 \ -d test \ -w test \ -r test \ -s test ``` In the above command: - `-a` specifies the account name. - `-u` specifies the username. - `-h` specifies the host name. - `-p` specifies the port number. - `-d` specifies the database name. - `-w` specifies the warehouse name. - `-r` specifies the role name. - `-s` specifies the schema name. After a successful configuration, you can use SnowSQL to interact with the Snowflake emulator. ```bash * SnowSQL * v1.2.32 Type SQL statements or !help test#test@test.test> ``` ### Run SQL commands You can execute SQL commands using SnowSQL. For example, to create a new database, execute the following command: ```bash CREATE DATABASE test_db; +----------------------------------------+ | status | |----------------------------------------| | Database TEST_DB successfully created. | +----------------------------------------+ 0 Row(s) produced. Time Elapsed: 0.198s ``` # Snowflake Drivers > Get started with Snowflake Drivers in LocalStack for Snowflake ## Introduction Snowflake Drivers enable the use of programming languages like Go, C#, and Python for developing applications that interact with Snowflake. The Snowflake emulator facilitates testing Snowflake integration without connecting to the actual Snowflake instance. This guide provides instructions on connecting the Snowflake emulator with various drivers. ## Snowflake Connector for Python The Snowflake Connector for Python (`snowflake-connector-python`) is a Python library that facilitates connecting Python programs to Snowflake databases and executing operations. Utilize this connector to link to the Snowflake emulator for testing your Snowflake integrations in Python. To install the Snowflake Connector for Python, execute the following command: ```bash pip install snowflake-connector-python ``` The Snowflake emulator operates on `snowflake.localhost.localstack.cloud` - note that this is a DNS name that resolves to a local IP address (`127.0.0.1`) to make sure the connector interacts with the local APIs. Connect to the emulator using the following Python code: ```python showLineNumbers import snowflake.connector as sf conn = sf.connect( user="test", password="test", account="test", database="test", host="snowflake.localhost.localstack.cloud", ) ``` Subsequently, create a warehouse named `test_warehouse`, a database named `testdb`, and a schema named `testschema` using the Snowflake Connector for Python: ```python showLineNumbers conn.cursor().execute("CREATE WAREHOUSE IF NOT EXISTS test_warehouse") conn.cursor().execute("CREATE DATABASE IF NOT EXISTS testdb") conn.cursor().execute("USE DATABASE testdb") conn.cursor().execute("CREATE SCHEMA IF NOT EXISTS testschema") ``` ## Node.js Driver The Snowflake Node.js driver facilitates connecting to Snowflake and executing operations on Snowflake databases using Node.js. Use this driver to link to the Snowflake emulator for testing Snowflake integration. To install the Snowflake Node.js driver, execute the following command: ```bash npm install snowflake-sdk ``` The Snowflake emulator runs on `snowflake.localhost.localstack.cloud`. Connect to the emulator using the following JavaScript code: ```javascript showLineNumbers var snowflake = require('snowflake-sdk'); var connection = snowflake.createConnection({ username: 'test', password: 'test', account: 'test', database: 'test', // snowflake-sdk version 1.9.3 and later supports host property and can be used instead of accessUrl like: // host: 'snowflake.localhost.localstack.cloud', accessUrl: 'https://snowflake.localhost.localstack.cloud', }); connection.connect(function(err, conn) { if (err) { console.error('Unable to connect: ' + err.message); } else { console.log('Successfully connected as id: ' + connection.getId()); } }); ``` Execute a query to create a database named `testdb` and verify the results using the following JavaScript code: ```javascript showLineNumbers connection.execute({ sqlText: 'CREATE DATABASE testdb', complete: function(err, stmt, rows) { if (err) { console.error('Failed to execute statement due to the following error: ' + err.message); } else { console.log('Successfully executed statement: ' + stmt.getSqlText()); } } }); ``` ## Go Driver The Go Snowflake driver provides a way to connect to Snowflake and perform database operations using Go. You can use this driver to connect to the Snowflake emulator for testing your Snowflake integrations in Go. To install the Go Snowflake driver, execute the following command: ```bash go get github.com/snowflakedb/gosnowflake ``` The connection string follows the format `username:password@host:port/database?account=account_name`. For the emulator use: `test:test@snowflake.localhost.localstack.cloud:4566/test?account=test` Here's an example of how to connect to the Snowflake emulator using Go: ```go showLineNumbers package main import ( "database/sql" "fmt" "log" _ "github.com/snowflakedb/gosnowflake" ) func main() { // Connection string connectionString := "test:test@snowflake.localhost.localstack.cloud:4566/test?account=test" // Connect to LocalStack Snowflake db, err := sql.Open("snowflake", connectionString) if err != nil { log.Fatalf("Failed to connect to Snowflake: %v", err) } defer db.Close() // Ping the database to verify the connection if err := db.Ping(); err != nil { log.Fatalf("Failed to ping Snowflake: %v", err) } fmt.Println("Successfully connected to Snowflake!") // Execute a simple query rows, err := db.Query("SELECT 123") if err != nil { log.Fatalf("Failed to execute query: %v", err) } defer rows.Close() // Process the result var version string for rows.Next() { if err := rows.Scan(&version); err != nil { log.Fatalf("Failed to scan row: %v", err) } fmt.Printf("Query result: %s\n", version) } if err := rows.Err(); err != nil { log.Fatalf("Error iterating rows: %v", err) } } ``` # Snowpark > Get started with Snowpark in LocalStack for Snowflake ## Introduction Snowpark library is a developer library for querying and processing data at scale in Snowflake. Snowflake currently provides Snowpark libraries for three languages: Java, Python, and Scala. The Snowflake emulator facilitates testing Snowpark queries without connecting to the actual Snowflake instance. This guide provides instructions on using the Snowflake emulator in conjunction with Snowpark. ## Snowpark for Python The Snowflake emulator supports the development and testing of Snowpark Python code in a local development environment. You can install the Snowpark Python library using the following command: ```bash pip install snowflake-snowpark-python ``` In this getting started guide, we'll use the Snowpark Python library to establish a connection to the Snowflake emulator and employ a DataFrame to query a table named `sample_product_data`. ### Create a Snowpark Session The Snowflake emulator operates on `snowflake.localhost.localstack.cloud`. To create a Snowpark session in Python, use the following code: ```python showLineNumbers from snowflake.snowpark import * from snowflake.snowpark.functions import * connection_parameters = { "user": "test", "password": "test", "account": "test", "warehouse": "test", "host": "snowflake.localhost.localstack.cloud", } session = Session.builder.configs(connection_parameters).create() ``` ### Create a table You can create a table named `sample_product_data` and fill the table with some data by executing SQL statements. Add the following Python code to create the table: ```python showLineNumbers session.sql('CREATE OR REPLACE TABLE sample_product_data (id INT, parent_id INT, category_id INT, name VARCHAR, serial_number VARCHAR, key INT, "3rd" INT)').collect() [Row(status='Table SAMPLE_PRODUCT_DATA successfully created.')] session.sql(""" INSERT INTO sample_product_data VALUES (1, 0, 5, 'Product 1', 'prod-1', 1, 10), (2, 1, 5, 'Product 1A', 'prod-1-A', 1, 20), (3, 1, 5, 'Product 1B', 'prod-1-B', 1, 30), (4, 0, 10, 'Product 2', 'prod-2', 2, 40), (5, 4, 10, 'Product 2A', 'prod-2-A', 2, 50), (6, 4, 10, 'Product 2B', 'prod-2-B', 2, 60), (7, 0, 20, 'Product 3', 'prod-3', 3, 70), (8, 7, 20, 'Product 3A', 'prod-3-A', 3, 80), (9, 7, 20, 'Product 3B', 'prod-3-B', 3, 90), (10, 0, 50, 'Product 4', 'prod-4', 4, 100), (11, 10, 50, 'Product 4A', 'prod-4-A', 4, 100), (12, 10, 50, 'Product 4B', 'prod-4-B', 4, 100) """).collect() ``` You can verify that the table was created and the data was inserted by executing the following Python code: ```python session.sql("SELECT count(*) FROM sample_product_data").collect() ``` The following output should be displayed: ```bash [Row(COUNT=12)] ``` ### Construct a DataFrame You can now construct a DataFrame from the data in the `sample_product_data` table using the following Python code: ```python df_table = session.table("sample_product_data") ``` You can see the first 10 rows of the DataFrame by executing the following Python code: ```python df_table.show(10) ``` The following output should be displayed: ```bash ------------------------------------------------------------------------------------- |"ID" |"PARENT_ID" |"CATEGORY_ID" |"NAME" |"SERIAL_NUMBER" |"KEY" |"3RD" | ------------------------------------------------------------------------------------- |1 |0 |5 |Product 1 |prod-1 |1 |10 | |2 |1 |5 |Product 1A |prod-1-A |1 |20 | |3 |1 |5 |Product 1B |prod-1-B |1 |30 | |4 |0 |10 |Product 2 |prod-2 |2 |40 | |5 |4 |10 |Product 2A |prod-2-A |2 |50 | |6 |4 |10 |Product 2B |prod-2-B |2 |60 | |7 |0 |20 |Product 3 |prod-3 |3 |70 | |8 |7 |20 |Product 3A |prod-3-A |3 |80 | |9 |7 |20 |Product 3B |prod-3-B |3 |90 | |10 |0 |50 |Product 4 |prod-4 |4 |100 | ------------------------------------------------------------------------------------- ``` ### Transform the DataFrame You can perform local transformations on the DataFrame. For example, you can filter rows with the value of 'id' equal to 1: ```python showLineNumbers df = session.table("sample_product_data").filter(col("id") == 1) df.show() ``` The following output should be displayed: ```bash ------------------------------------------------------------------------------------ |"ID" |"PARENT_ID" |"CATEGORY_ID" |"NAME" |"SERIAL_NUMBER" |"KEY" |"3RD" | ------------------------------------------------------------------------------------ |1 |0 |5 |Product 1 |prod-1 |1 |10 | ------------------------------------------------------------------------------------ ``` Furthermore, you can also select specific columns: ```python showLineNumbers df = session.table("sample_product_data").select(col("id"), col("name"), col("serial_number")) df.show() ``` The following output should be displayed: ```bash --------------------------------------- |"ID" |"NAME" |"SERIAL_NUMBER" | --------------------------------------- |1 |Product 1 |prod-1 | |2 |Product 1A |prod-1-A | |3 |Product 1B |prod-1-B | |4 |Product 2 |prod-2 | |5 |Product 2A |prod-2-A | |6 |Product 2B |prod-2-B | |7 |Product 3 |prod-3 | |8 |Product 3A |prod-3-A | |9 |Product 3B |prod-3-B | |10 |Product 4 |prod-4 | --------------------------------------- ``` # Terraform > Use Terraform to interact with the Snowflake emulator ## Introduction [Terraform](https://terraform.io/) is an Infrastructure-as-Code (IaC) framework developed by HashiCorp. It enables users to define and provision infrastructure using a high-level configuration language. Terraform uses HashiCorp Configuration Language (HCL) as its configuration syntax. The Snowflake emulator supports Terraform, allowing you to define and provision Snowflake resources using the same commands and syntax as the Snowflake service. You can use Terraform to create, update, and delete Snowflake resources locally, such as databases, schemas, tables, and stages. ## Configuring Terraform In this guide, you will learn how to configure Terraform to interact with the Snowflake emulator. ### Setup Snowflake provider To use Terraform with the Snowflake emulator, you need to configure the Snowflake provider in your Terraform configuration file. The following example shows how to configure the Snowflake provider: ```hcl showLineNumbers terraform { required_providers { snowflake = { source = "Snowflake-Labs/snowflake" version = "= 0.92" } } } provider "snowflake" { account = "test" user = "test" password = "test" role = "test" host = "snowflake.localhost.localstack.cloud" } ``` :::note Instead of manually specifying the `host`, you can export the `SNOWFLAKE_HOST` environment variable to set the Snowflake host. Here is an example: ```bash export SNOWFLAKE_HOST=snowflake.localhost.localstack.cloud ``` ::: ### Create Snowflake resources You can now use Terraform to create Snowflake resources using the Snowflake provider. The following example shows how to create a Snowflake database using Terraform: ```hcl showLineNumbers resource "snowflake_database" "example" { name = "example" comment = "example database" data_retention_time_in_days = 3 } ``` ### Deploy the Terraform configuration You can now deploy the Terraform configuration using the following command: ```bash terraform init terraform apply ``` The `terraform init` command initializes the Terraform configuration, and the `terraform apply` command creates the Snowflake database. # Sample Apps > Sample Apps to help LocalStack for Snowflake users adopt real-world scenarios to rapidly and conveniently create, configure, and test applications locally. import ApplicationsShowcase from "../../../components/ApplicationsShowcase.astro"; # SQL Functions Coverage > Overview of the implemented Snowflake SQL functions in LocalStack import SqlFunctionsCoverage from '../../../components/snowflake-coverage/SqlFunctionsCoverage'; ## Overview This table provides a list of all Snowflake system-defined SQL functions, scalar or table, emulated by LocalStack. The content will be updated as additional query features and functions are implemented.

# Connecting Snowpark to Lambda using LocalStack > In this tutorial, you will explore how you can use Snowpark to connect to a locally running AWS Lambda using LocalStack. ## Introduction In this tutorial, you will explore how to connect Snowpark to AWS Lambda locally using LocalStack. Snowpark allows you to query, process, and transform data in a variety of ways using Snowpark Python. In this example, we create a Lambda function that uses Snowpark to: - Establish a connection to a local Snowflake database provided by LocalStack. - Create a cursor object and execute a simple query to create a table. - Insert rows and execute an insert query with executemany and a query to select data from the table. - Fetch the results and execute a query to get the current timestamp. The code in this tutorial is available on [GitHub](https://github.com/localstack-samples/localstack-snowflake-samples/tree/main/lambda-snowpark-connector). ## Prerequisites - [`lstk`](/snowflake/getting-started/installation/) with a [`LOCALSTACK_AUTH_TOKEN`](/snowflake/getting-started/auth-token/) - [LocalStack for Snowflake](/snowflake/getting-started/) - [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) & [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command - Python 3.10 installed locally ## Create the Lambda function Create a new directory for your lambda function and navigate to it: ```bash showLineNumbers mkdir -p lambda-snowpark cd lambda-snowpark ``` Create a new file named `handler.py` and add the following code: ```python showLineNumbers import snowflake.connector as sf def lambda_handler(event, context): snowflake_params = { 'user': 'test', 'password': 'test', 'account': 'test', 'warehouse': 'test', 'database': 'test', 'schema': 'TEST_SCHEMA', 'host': 'snowflake.localhost.localstack.cloud' } # Establish a connection connection = sf.connect(**snowflake_params) try: # Create a cursor object cursor = connection.cursor() # Execute the query to create a table cursor.execute("create or replace table ability(name string, skill string )") # Rows to insert rows_to_insert = [('John', 'SQL'), ('Alex', 'Java'), ('Pete', 'Snowflake')] # Execute the insert query with executemany cursor.executemany("insert into ability (name, skill) values (%s, %s)", rows_to_insert) # Execute a query to select data from the table cursor.execute("select name, skill from ability") # Fetch the results result = cursor.fetchall() print("Total # of rows:", len(result)) print("Row-1 =>", result[0]) print("Row-2 =>", result[1]) # Execute a query to get the current timestamp cursor.execute("SELECT CURRENT_TIMESTAMP()") current_timestamp = cursor.fetchone()[0] print("Current timestamp from Snowflake:", current_timestamp) finally: # Close the cursor cursor.close() # Close the connection connection.close() return { 'statusCode': 200, 'body': "Successfully connected to Snowflake and inserted rows!" } ``` In the above code: - You import the `snowflake.connector` module to establish a connection to the Snowflake database. - You define the `lambda_handler` function, which is the entry point for the Lambda function. - You define the `snowflake_params` dictionary with `snowflake.localhost.localstack.cloud` as the host to connect to the Snowflake emulator. - You establish a connection to the Snowflake database using the `sf.connect` method. - You create a cursor object and execute a query to create a table. - You insert rows and execute an insert query with `executemany` and a query to select data from the table. - You fetch the results and execute a query to get the current timestamp. - You close the cursor and the connection. ## Install the dependencies You can now install the dependencies for your Lambda function. These include: - `snowflake-connector-python` to connect to the Snowflake database. - `boto3` and `botocore` to interact with AWS services. Run the following command: ```bash showLineNumbers pip3 install \ --platform manylinux2014_x86_64 \ --implementation cp \ --python-version 3.10 \ --only-binary=:all: --upgrade \ --target ./libs \ snowflake-connector-python==2.7.9 boto3==1.26.153 botocore==1.29.153 ``` ## Package the Lambda function Package the Lambda function and its dependencies into a ZIP file. Run the following command: ```bash showLineNumbers mkdir -p build cp handler.py build/ cp -r libs/* build/ (cd build && zip -q -r function-py.zip .) ``` You have now created a ZIP file named `function-py.zip` that contains the Lambda function and its dependencies. ## Start the LocalStack container Start your LocalStack container in your preferred terminal/shell. ```bash showLineNumbers export LOCALSTACK_AUTH_TOKEN= LOCALSTACK_DEBUG=1 \ LOCALSTACK_LAMBDA_RUNTIME_ENVIRONMENT_TIMEOUT=180 \ lstk start --type snowflake ``` > The `LOCALSTACK_DEBUG=1` environment variable is set to enable debug logs. It would allow you to see the SQL queries executed by the Lambda function. The `LOCALSTACK_LAMBDA_RUNTIME_ENVIRONMENT_TIMEOUT` environment variable is set to increase the Lambda function's timeout to 180 seconds. ## Deploy the Lambda function You can now deploy the Lambda function to LocalStack using `lstk aws`. Run the following command: ```bash showLineNumbers lstk aws lambda create-function \ --function-name localstack-snowflake-lambda-example \ --runtime python3.10 \ --timeout 180 \ --zip-file fileb://build/function-py.zip \ --handler handler.lambda_handler \ --role arn:aws:iam::000000000000:role/lambda-role ``` After successfully deploying the Lambda function, you will receive a response with the details of the function. You can now invoke the function using `lstk aws`: ```bash showLineNumbers lstk aws lambda invoke --function-name localstack-snowflake-lambda-example \ --cli-binary-format raw-in-base64-out \ --payload '{"body": "test"}' output.txt ``` You will receive a response with the details of the invocation. You can view the output in the `output.txt` file. To see the SQL queries executed by the Lambda function, check the logs by navigating to LocalStack logs (`lstk logs`). ```bash 2024-02-07T17:33:36.763 DEBUG --- [ asgi_gw_3] l.s.l.i.version_manager : [localstack-snowflake-lambda-example-b0813b21-ad5f-4ec7-8fb4-53147df9695e] Total # of rows: 3 2024-02-07T17:33:36.763 DEBUG --- [ asgi_gw_3] l.s.l.i.version_manager : [localstack-snowflake-lambda-example-b0813b21-ad5f-4ec7-8fb4-53147df9695e] Row-1 => ('John', 'SQL') 2024-02-07T17:33:36.763 DEBUG --- [ asgi_gw_3] l.s.l.i.version_manager : [localstack-snowflake-lambda-example-b0813b21-ad5f-4ec7-8fb4-53147df9695e] Row-2 => ('Alex', 'Java') 2024-02-07T17:33:36.771 DEBUG --- [ asgi_gw_3] l.s.l.i.version_manager : [localstack-snowflake-lambda-example-b0813b21-ad5f-4ec7-8fb4-53147df9695e] Current timestamp from Snowflake: 2024-02-07T17:33:36 ``` ## Conclusion LocalStack's core cloud emulator supports a wide range of AWS services, including Lambda, and provides a local environment for developing and testing your serverless applications. You can now use LocalStack's Snowpark emulator to connect to locally running AWS services, and develop/test your Snowpark & AWS applications locally. # Credit Scoring with Snowpark & LocalStack > In this tutorial, you will explore how you can use LocalStack for Snowflake with Snowpark for data analysis and visualization. ## Introduction In this tutorial, you will learn how you can use the Snowflake emulator with Snowpark for Python and your favorite Python libraries for data analysis. The Jupyter Notebook and the dataset used in this tutorial are available on [GitHub](https://github.com/localstack-samples/localstack-snowflake-samples/tree/main/credit-scoring-with-snowpark). ## Prerequisites - [`lstk`](/snowflake/getting-started/installation/) with a [`LOCALSTACK_AUTH_TOKEN`](/snowflake/getting-started/auth-token/) - [LocalStack for Snowflake](/snowflake/getting-started/) - [Snowpark](/snowflake/integrations/snowpark) with other Python libraries - [Jupyter Notebook](https://jupyter.org/install#jupyter-notebook) You should also download [`credit_files.csv`](https://github.com/localstack-samples/localstack-snowflake-samples/blob/main/credit-scoring-with-snowpark/credit_files.csv) and [`credit_request.csv`](https://github.com/localstack-samples/localstack-snowflake-samples/blob/main/credit-scoring-with-snowpark/credit_request.csv) files from the LocalStack repository. The files should be present in the same directory as your Jupyter Notebook. ## Start the Snowflake emulator Start your LocalStack container in your preferred terminal/shell. ```bash export LOCALSTACK_AUTH_TOKEN= lstk start ``` ## Create a Snowpark session The next step is to configure the Snowflake emulator. The Snowflake emulator runs on `snowflake.localhost.localstack.cloud`. You can use the Snowpark to connect to the locally running Snowflake server. Start Jupyter Notebook and create a new notebook. Add the following code to connect to the Snowflake emulator: ```python showLineNumbers from snowflake.snowpark import * from snowflake.snowpark.functions import * connection_parameters = { "user": "test", "password": "test", "account": "test", "warehouse": "test", "host": "snowflake.localhost.localstack.cloud", } session = Session.builder.configs(connection_parameters).create() ``` In the above configuration, you can set `user`, `password`, `account`, and `warehouse` as `test` to avoid passing any production values. You can now run Snowflake SQL queries on your local machine. ```python showLineNumbers session.sql("create or replace database credit_bank").collect() session.sql("use schema credit_bank.public").collect() print(session.sql("select current_warehouse(), current_database(), current_schema(), current_user(), current_role()").collect()) ``` ```bash showLineNumbers [Row(?COLUMN?='TEST', CURRENT_DATABASE='CREDIT_BANK', CURRENT_SCHEMA='public', ?COLUMN?='TEST', GET_CURRENT_ROLE='PUBLIC')] ``` ## Create the tables You can now create two tables associated with this tutorial: - `CREDIT_FILES`: This table contains the credit on files along with the credit standing whether the loan is being repaid or if there are actual issues with reimbursing the credit. - `CREDIT_REQUESTS`: This table contains the new credit requests that the bank needs to provide approval on. Run the following code to create the `credit_df` table: ```python showLineNumbers import pandas as pd credit_files = pd.read_csv('credit_files.csv') session.write_pandas(credit_files,"CREDIT_FILES",auto_create_table='True') credit_df = session.table("CREDIT_FILES") credit_df.schema ``` ```bash StructType([StructField('CREDIT_REQUEST_ID', LongType(), nullable=True), StructField('CREDIT_AMOUNT', LongType(), nullable=True), StructField('CREDIT_DURATION', LongType(), nullable=True), StructField('PURPOSE', StringType(), nullable=True), StructField('INSTALLMENT_COMMITMENT', LongType(), nullable=True), StructField('OTHER_PARTIES', StringType(), nullable=True), StructField('CREDIT_STANDING', StringType(), nullable=True), StructField('CREDIT_SCORE', LongType(), nullable=True), StructField('CHECKING_BALANCE', LongType(), nullable=True), StructField('SAVINGS_BALANCE', LongType(), nullable=True), StructField('EXISTING_CREDITS', LongType(), nullable=True), StructField('ASSETS', StringType(), nullable=True), StructField('HOUSING', StringType(), nullable=True), StructField('QUALIFICATION', StringType(), nullable=True), StructField('JOB_HISTORY', LongType(), nullable=True), StructField('AGE', LongType(), nullable=True), StructField('SEX', StringType(), nullable=True), StructField('MARITAL_STATUS', StringType(), nullable=True), StructField('NUM_DEPENDENTS', LongType(), nullable=True), StructField('RESIDENCE_SINCE', LongType(), nullable=True), StructField('OTHER_PAYMENT_PLANS', StringType(), nullable=True)]) ``` In a similar fashion, you can create the `credit_req_df` table: ```python showLineNumbers credit_requests = pd.read_csv('credit_request.csv') session.write_pandas(credit_requests,"CREDIT_REQUESTS",auto_create_table='True') credit_req_df = session.table("CREDIT_REQUESTS") credit_req_df.schema ``` ```bash StructType([StructField('CREDIT_REQUEST_ID', LongType(), nullable=True), StructField('CREDIT_AMOUNT', LongType(), nullable=True), StructField('CREDIT_DURATION', LongType(), nullable=True), StructField('PURPOSE', StringType(), nullable=True), StructField('INSTALLMENT_COMMITMENT', LongType(), nullable=True), StructField('OTHER_PARTIES', StringType(), nullable=True), StructField('CREDIT_SCORE', LongType(), nullable=True), StructField('CHECKING_BALANCE', LongType(), nullable=True), StructField('SAVINGS_BALANCE', LongType(), nullable=True), StructField('EXISTING_CREDITS', LongType(), nullable=True), StructField('ASSETS', StringType(), nullable=True), StructField('HOUSING', StringType(), nullable=True), StructField('QUALIFICATION', StringType(), nullable=True), StructField('JOB_HISTORY', LongType(), nullable=True), StructField('AGE', LongType(), nullable=True), StructField('SEX', StringType(), nullable=True), StructField('MARITAL_STATUS', StringType(), nullable=True), StructField('NUM_DEPENDENTS', LongType(), nullable=True), StructField('RESIDENCE_SINCE', LongType(), nullable=True), StructField('OTHER_PAYMENT_PLANS', StringType(), nullable=True)]) ``` ## Analyze the data You can now analyze the data using Snowpark and your favorite Python libraries. For example, you can fetch the first 5 rows of the `credit_df` table: ```python credit_df.toPandas().head() ``` You can similarly fetch the first 5 rows of the `credit_req_df` table: ```python credit_req_df.toPandas().head() ``` You can also visualize the numeric features and categorical features. For example, you can visualize the histogram of the `credit_df` table: ```python credit_df.toPandas().hist(figsize=(15,15)) ```

![credit_df_hist](/images/snowflake/credit_df_hist.png)

You can also visualize the categorical features of the `credit_df` table: ```python showLineNumbers import matplotlib.pyplot as plt import seaborn as sns sns.set(style="darkgrid") fig, axs = plt.subplots(5, 2, figsize=(15, 30)) df = credit_df.toPandas() sns.countplot(data=df, y="PURPOSE", ax=axs[0,0]) sns.countplot(data=df, x="OTHER_PARTIES", ax=axs[0,1]) sns.countplot(data=df, x="CREDIT_STANDING", ax=axs[1,0]) sns.countplot(data=df, x="ASSETS", ax=axs[1,1]) sns.countplot(data=df, x="HOUSING", ax=axs[2,0]) sns.countplot(data=df, x="QUALIFICATION", ax=axs[2,1]) sns.countplot(data=df, x="SEX", ax=axs[3,0]) sns.countplot(data=df, x="MARITAL_STATUS", ax=axs[3,1]) sns.countplot(data=df, x="OTHER_PAYMENT_PLANS", ax=axs[4,0]) sns.stripplot(y="PURPOSE", x="CREDIT_AMOUNT", data=df, hue='CREDIT_STANDING', jitter=True, ax=axs[4,1]) plt.show() ```

![credit_df_cat](/images/snowflake/credit_df_cat.png) ## Conclusion You can now perform further experimentations with the Snowflake emulator. For example, you can use the Snowpark API to run queries to get various insights, such as determining the range of loans per different category. # Querying S3 Tables with Snowflake > In this tutorial, you will learn how to integrate AWS S3 Tables with Snowflake to query Iceberg tables stored in S3 Tables buckets through LocalStack. ## Introduction In this tutorial, you will explore how to connect Snowflake to AWS S3 Tables locally using LocalStack. S3 Tables is a managed Apache Iceberg table catalog that uses S3 storage, providing built-in maintenance features like automatic compaction and snapshot management. With LocalStack's Snowflake emulator, you can create catalog integrations that connect to S3 Tables and query Iceberg tables without needing cloud resources. This integration allows you to: - Create catalog integrations to connect Snowflake to S3 Tables. - Query existing Iceberg tables stored in S3 Tables buckets. - Leverage automatic schema inference from external Iceberg tables. ## Prerequisites - [`lstk`](/snowflake/getting-started/installation/) with a [`LOCALSTACK_AUTH_TOKEN`](/snowflake/getting-started/auth-token/) - [LocalStack for Snowflake](/snowflake/getting-started/) - [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) & [`lstk aws`](/aws/developer-tools/running-localstack/lstk/cloud-and-iac-commands/#aws) command - Python 3.10+ with `pyiceberg` and `pyarrow` installed ## Start LocalStack Start your LocalStack container with the Snowflake emulator enabled. ```bash export LOCALSTACK_AUTH_TOKEN= lstk start ``` ## Create S3 Tables resources Before configuring Snowflake, you need to create S3 Tables resources using the AWS CLI. This includes a table bucket and a namespace. ### Create a table bucket Create a table bucket to store your Iceberg tables. ```bash lstk aws s3tables create-table-bucket --name my-table-bucket ``` ```bash title="Output" { "arn": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket" } ``` ### Create a namespace Create a namespace within the table bucket to organize your tables. ```bash lstk aws s3tables create-namespace \ --table-bucket-arn arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket \ --namespace my_namespace ``` ```bash title="Output" { "tableBucketARN": "arn:aws:s3tables:us-east-1:000000000000:bucket/my-table-bucket", "namespace": [ "my_namespace" ] } ``` ## Create and populate a table in S3 Tables To query data from Snowflake using `CATALOG_TABLE_NAME`, the S3 Tables table must have a defined schema and contain data. Use PyIceberg to create a table with schema and populate it with data. First, install the required Python packages: ```bash pip install "pyiceberg[s3fs,pyarrow]" boto3 ``` Create a Python script named `setup_s3_tables.py` with the following content: ```python import pyarrow as pa from pyiceberg.catalog.rest import RestCatalog from pyiceberg.schema import Schema from pyiceberg.types import NestedField, StringType, LongType # Configuration LOCALSTACK_URL = "http://localhost.localstack.cloud:4566" S3TABLES_URL = "http://s3tables.localhost.localstack.cloud:4566" TABLE_BUCKET_NAME = "my-table-bucket" NAMESPACE = "my_namespace" TABLE_NAME = "customer_orders" REGION = "us-east-1" # Create PyIceberg REST catalog pointing to S3 Tables catalog = RestCatalog( name="s3tables_catalog", uri=f"{S3TABLES_URL}/iceberg", warehouse=TABLE_BUCKET_NAME, **{ "s3.region": REGION, "s3.endpoint": LOCALSTACK_URL, "client.access-key-id": "000000000000", "client.secret-access-key": "test", "rest.sigv4-enabled": "true", "rest.signing-name": "s3tables", "rest.signing-region": REGION, }, ) # Define table schema schema = Schema( NestedField(field_id=1, name="order_id", field_type=StringType(), required=False), NestedField(field_id=2, name="customer_name", field_type=StringType(), required=False), NestedField(field_id=3, name="amount", field_type=LongType(), required=False), ) # Create table in S3 Tables catalog.create_table( identifier=(NAMESPACE, TABLE_NAME), schema=schema, ) print(f"Created table: {NAMESPACE}.{TABLE_NAME}") # Reload the table to get the latest metadata table = catalog.load_table((NAMESPACE, TABLE_NAME)) # Populate table with sample data data = pa.table({ "order_id": ["ORD001", "ORD002", "ORD003"], "customer_name": ["Alice", "Bob", "Charlie"], "amount": [100, 250, 175], }) table.append(data) print("Inserted sample data into table") # Verify table exists tables = catalog.list_tables(NAMESPACE) print(f"Tables in namespace: {tables}") ``` Run the script to create the table and populate it with data: ```bash python setup_s3_tables.py ``` ```bash title="Output" Created table: my_namespace.customer_orders Inserted sample data into table Tables in namespace: [('my_namespace', 'customer_orders')] ``` ## Connect to the Snowflake emulator Connect to the locally running Snowflake emulator using an SQL client of your choice (such as DBeaver). The Snowflake emulator runs on `snowflake.localhost.localstack.cloud`. You can use the following connection parameters: | Parameter | Value | |-----------|-------| | Host | `snowflake.localhost.localstack.cloud` | | User | `test` | | Password | `test` | | Account | `test` | | Warehouse | `test` | ## Create a catalog integration Create a catalog integration to connect Snowflake to your S3 Tables bucket. The catalog integration defines how Snowflake connects to the external Iceberg REST catalog provided by S3 Tables. ```sql CREATE OR REPLACE CATALOG INTEGRATION s3tables_catalog_integration CATALOG_SOURCE=ICEBERG_REST TABLE_FORMAT=ICEBERG CATALOG_NAMESPACE='my_namespace' REST_CONFIG=( CATALOG_URI='http://s3tables.localhost.localstack.cloud:4566/iceberg' CATALOG_NAME='my-table-bucket' ) REST_AUTHENTICATION=( TYPE=AWS_SIGV4 AWS_ACCESS_KEY_ID='000000000000' AWS_SECRET_ACCESS_KEY='test' AWS_REGION='us-east-1' AWS_SERVICE='s3tables' ) ENABLED=TRUE REFRESH_INTERVAL_SECONDS=60; ``` In the above query: - `CATALOG_SOURCE=ICEBERG_REST` specifies that the catalog uses the Iceberg REST protocol. - `TABLE_FORMAT=ICEBERG` indicates the table format. - `CATALOG_NAMESPACE='my_namespace'` sets the default namespace to query tables from. - `REST_CONFIG` configures the connection to the LocalStack S3 Tables REST API endpoint. - `REST_AUTHENTICATION` configures AWS SigV4 authentication for the S3 Tables service. - `REFRESH_INTERVAL_SECONDS=60` sets how often Snowflake refreshes metadata from the catalog. ## Create an Iceberg table referencing S3 Tables Create an Iceberg table in Snowflake that references the existing S3 Tables table using `CATALOG_TABLE_NAME`. The schema is automatically inferred from the external table. ```sql CREATE OR REPLACE ICEBERG TABLE iceberg_customer_orders CATALOG='s3tables_catalog_integration' CATALOG_TABLE_NAME='my_namespace.customer_orders' AUTO_REFRESH=TRUE; ``` In the above query: - `CATALOG` references the catalog integration created in the previous step. - `CATALOG_TABLE_NAME` specifies the fully-qualified table name in the format `namespace.table_name`. - `AUTO_REFRESH=TRUE` enables automatic refresh of table metadata. - No column definitions are needed as the schema is inferred from the existing S3 Tables table. ## Query the Iceberg table You can now query the Iceberg table like any other Snowflake table. The schema (columns) are automatically available from the external table. ```sql SELECT * FROM iceberg_customer_orders; ``` ```sql title="Output" +----------+---------------+--------+ | order_id | customer_name | amount | +----------+---------------+--------+ | ORD001 | Alice | 100 | | ORD002 | Bob | 250 | | ORD003 | Charlie | 175 | +----------+---------------+--------+ ``` ## Conclusion In this tutorial, you learned how to integrate AWS S3 Tables with Snowflake using LocalStack. You created S3 Tables resources, populated a table with data using PyIceberg, configured a catalog integration in Snowflake, and queried Iceberg tables stored in S3 Tables buckets using `CATALOG_TABLE_NAME`. The S3 Tables integration enables you to: - Query data stored in S3 Tables using familiar Snowflake SQL syntax. - Leverage automatic schema inference from external Iceberg catalogs. - Develop and test your data lakehouse integrations locally without cloud resources. LocalStack's Snowflake emulator combined with S3 Tables support provides a complete local environment for developing and testing multi-platform data analytics workflows.