Streaming from databases

For streaming data from databases, you need to read data in your pipeline with the "Query" task:

When the pipeline is triggered, this task will make an SQL query to the database, you have configured in your integration and use the response data for the entire pipeline.

We support a wide range of SQL integrations to read from and write data to. You can find more information on how to configure such integrations on their specific pages:

• PostgreSQL

• Snowflake

• MySQL

• AWS RDS

• Google Big Query

• Google Cloud SQL

• ClickHouse

• Immuta

• ElasticSearch

• PlanetScale

Some examples of the streaming pipelines from the API:

Streaming from database to API

Streaming from database to database

Same as in the case of streaming from APIs, the pipelines above are just examples and your real pipelines will most probably be more complicated and will contain more transformation and processing of data: