Running Eureka server
Running Vyne query server
Running Pipelines Orchestrator
Taxi schema for ingested data
The pipeline Runner receives pipeline descriptions through a REST API. On reception, it is going to instantiate the two input and output Transports. Those transports will be responsible to initiate and maintain connections and communication to the datasources
Whilst the runners expose a REST API to schedule pipelines, it is recommended to use the Pipelines Orchestrator API to submit and manage your pipelines.
As mentioned in the Pipelines Orchestrator documentation, the Pipeline Runners write their current state into their Eureka instance metadata. On top of that, the Pipeline Runners also communicate back events to the Pipelines Orchestator through the Event API. Currently, only the logs of the runners are reported back to the Orchestrator and no action except of logging is performed on reception of those events. Future versions will be more elaborated.