Data Source

What is a Data Source?

Data Source is any origin from which data is ingested into GlassFlow pipelines. Data Sources can be databases, streaming services, or real-time applications that generate events that need to be processed.

Ways to Connect to Data Sources

Managed Integrations

Integrations are fully managed services and provide seamless, out-of-the-box connectivity to common data sources. These integrations are fully managed, supported and maintained by the GlassFlow team. These integrations make it easier to connect to data sources without infrastructure setup.

Example Use Case: Integrating well-known services like Amazon SQS, Google PubSub, or databases like Postgres using CDC.

Using Python SDK

Python SDK lets you send data from any data source to the GlassFlow pipelines. Create a custom data source connector for your specific data source.

Example Use Case: Integrate the SDK directly in your python application or build a custom integrate with a data source that does not have a managed GlassFlow integration.

Data Transformation

Data Sink