Continuous Integration
Introduction
Section titled “Introduction”This guide explains how to set up the Snowflake emulator in a continuous integration (CI) environment. You can use the emulator to test your Snowflake integration without connecting to the real Snowflake instance.
To install the Snowflake emulator, you need to set up the LocalStack CLI and pull the Docker image. After starting the Docker container, an endpoint (snowflake.localhost.localstack.cloud
) is made available to connect to the emulated Snowflake instance, which you can use to test your data integrations in CI environments.
Configure the LocalStack CI Key
Section titled “Configure the LocalStack CI Key”Before you begin, set up the LocalStack CI key to activate the Snowflake emulator in your CI environment. LocalStack requires a CI Key for use in continuous integration (CI) or similar machine environments. Each instance startup in a CI or comparable environment consumes one CI token.
To create a CI key, follow these steps:
- Open the LocalStack Web Application.
- Click the CI Keys on the left navigation pane.
- Navigate to the Generate CI Key tab, enter a description, and click Generate CI Key.
- Copy the generated CI key and set it as the
LOCALSTACK_AUTH_TOKEN
environment variable in your CI environment.
Setup the CI configuration
Section titled “Setup the CI configuration”The following examples demonstrate how to set up the emulator in GitHub Actions, CircleCI, and GitLab CI.
name: LocalStack Teston: [ push, pull_request ]
jobs: localstack-action-test: name: 'Test LocalStack GitHub Action' runs-on: ubuntu-latest steps: - uses: actions/checkout@v3
- name: Start Snowflake emulator run: docker pull localstack/snowflake:latest pip install localstack localstack start --stack snowflake -d echo "Waiting for LocalStack startup..." localstack wait -t 15 echo "Startup complete" env: LOCALSTACK_AUTH_TOKEN: ${{ secrets.LOCALSTACK_AUTH_TOKEN }}
version: 2.1
orbs: python: circleci/python@2.0.3
jobs: example-job: machine: image: ubuntu-2004:2022.04.1
steps: - checkout
- run: name: Start LocalStack command: | pip3 install localstack docker pull localstack/snowflake localstack start --stack snowflake -d
echo "Waiting for LocalStack startup..." localstack wait -t 30 echo "Startup complete"
workflows: version: 2 build: jobs: - example-job
image: docker:20.10.16
stages: - test
test: stage: test variables: DOCKER_HOST: tcp://docker:2375 DOCKER_TLS_CERTDIR: "" LOCALSTACK_AUTH_TOKEN: $LOCALSTACK_AUTH_TOKEN
services: - name: docker:20.10.16-dind alias: docker command: ["--tls=false"]
before_script: - apk update - apk add gcc musl-dev linux-headers py3-pip python3 python3-dev - python3 -m pip install localstack script: - docker pull localstack/snowflake:latest - dind_ip="$(getent hosts docker | cut -d' ' -f1)" - echo "${dind_ip} localhost.localstack.cloud " >> /etc/hosts - DOCKER_HOST="tcp://${dind_ip}:2375" localstack start --stack snowflake -d - localstack wait -t 15