Openflow
Introduction
Section titled “Introduction”Openflow is Snowflake’s data movement service that provides a unified platform for building, scaling, and managing data pipelines. It is powered by Apache NiFi and enables flexible data ingestion, transformation, and integration across diverse sources and destinations.
The Snowflake emulator in LocalStack supports basic Openflow functionality by using Apache NiFi. This allows you to experiment locally with Openflow concepts, such as creating processors and running SQL queries against the Snowflake emulator.
You can access the Openflow UI when the emulator is running at:
https://snowflake.localhost.localstack.cloud:4566/openflow/
Getting started
Section titled “Getting started”To begin using Openflow in LocalStack:
- Start your Snowflake emulator.
- Open the following URL in your browser:
https://snowflake.localhost.localstack.cloud:4566/openflow/
The first load may take some time, as Apache NiFi dependencies are downloaded and initialized.
Once the UI is available, you can create and configure NiFi processors directly in your browser.
Running a query with ExecuteSQL
Section titled “Running a query with ExecuteSQL”The following example demonstrates how to run a simple query against the Snowflake emulator using the ExecuteSQL
processor.
-
Add an ExecuteSQL processor: Drag the
ExecuteSQL
processor onto the canvas in the Openflow UI. -
Configure the processor:
- Set the Database Connection Pooling Service to use the default
Snowflake Connection Pool
.
- Enter a query, for example:
SELECT 123;
- In the Relationships tab, configure the processor to terminate or retry on
failure
andsuccess
.
- Start the processor: Right-click the processor and choose Start. The processor will run the SQL query against the Snowflake emulator.
- Verify execution: In the emulator logs, you should see the executed query:
Running query (account/DB/schema TEST/TEST/public): SELECT 123