Get Started
Get Started with GlassFlow Python SDK to build and manage pipelines.
Prerequisites
Before starting, ensure you have:
- A GlassFlow account. Sign up here if you don't have one.
- A personal access token from the GlassFlow account you just created. You can get yours in the profile section of your account
- Python 3.x installed on your system.
- Pip installed to manage project packages.
Installation
Install glassflow python SDK locally:
Quickstart
To create an example pipeline, run the following helper command, which will create a hello-world example pipeline on GlassFlow.
It will send some example data to the pipeline and display the transformed result on the console.
You can explore this example pipeline on GlassFlow Web App.
Example Pipeline - Step by Step
The following steps show how to create a pipeline and send-receive events from the pipeline.
Get Personal Access Token
GlassFlow Python SDK connects to GlassFlow server using Personal Access Token.
You can get your token from the profile section in the web-application.
Define Your Pipeline Transformation Code
Create a transform.py file with your transformation code.
A basic echo example can be like:
If your pipeline transformation code needs external Python dependencies, you can add them on a requirements.txt file.
Example:
Save transform.py and requirements.txt locally to use them in the next step.
Create a Pipeline and Send Events
Create a python script called create_pipeline.py with the following code. This creates an example space on GlassFlow and
creates a pipeline called echo-pipeline. It also sends events to the pipeline and consumes transformed events back from the pipeline.
In the above example, you defined a custom transformation function in python that is executed in real-time for every event in the pipeline.
You used the Python SDK to send data to the pipeline and consume the transformed events back from the pipeline in real-time.
The output after running the above code should look like this:
As you see, with a few lines of code, you are able to create a real-time data pipeline completely in python. With GlassFlow, you do not need to set up or configure any infrastructure and your pipeline can scale to millions of events.
Explore other sections to learn how to set up more advanced pipelines, work with integrations and use CI/CD to maintain pipelines in production.