0.19.0
0.19 is a feature release, containing multiple new features, including some breaking changes.
Taxi version
The Taxi version has not changed in this release, and remains on 1.30.2.
Breaking Changes
Pipeline Runner Changes
If using a bespoke Pipeline Runner application, you'll need to make the following changes:
Remove the following annotation from the Spring boot application class (i.e. the type annotated with @SpringBootApplication
)
@EnableConfigurationProperties(VyneCacheConfiguration::class)
Add the following imports:
import io.vyne.spring.config.VyneSpringCacheConfiguration
import io.vyne.spring.config.VyneSpringHazelcastConfiguration
import io.vyne.spring.config.VyneSpringProjectionConfiguration
import io.vyne.spring.http.auth.HttpAuthConfig
import org.springframework.context.annotation.Import
and annotate the Spring boot application class with these additional statements:
@Import(HttpAuthConfig::class)
@EnableConfigurationProperties(VyneSpringCacheConfiguration::class, VyneSpringProjectionConfiguration::class, VyneSpringHazelcastConfiguration::class)
Update 'vyne' configuration block in your application.yaml with the new mandatory projection and hazelcast configuration blocks. As an example, update:
vyne:
schema:
publicationMethod: EUREKA
queryService:
name: QUERY-SERVICE
pipelinesOrchestratorService:
name: PIPELINES-ORCHESTRATOR
caskService:
name: CASK
queryServiceUrl: http://query-service
graph:
vyneGraphBuilderCache:
baseSchemaCacheSize: 100
graphWithFactTypesCacheSize: 100
baseSchemaGraphCacheSize: 100
vyneDiscoverGraphQuery:
schemaGraphCacheSize: 5
searchPathExclusionsCacheSize: 200000
as:
vyne:
schema:
publicationMethod: EUREKA
queryService:
name: QUERY-SERVICE
pipelinesOrchestratorService:
name: PIPELINES-ORCHESTRATOR
caskService:
name: CASK
queryServiceUrl: http://query-service
graph:
vyneGraphBuilderCache:
baseSchemaCacheSize: 100
graphWithFactTypesCacheSize: 100
baseSchemaGraphCacheSize: 100
vyneDiscoverGraphQuery:
schemaGraphCacheSize: 5
searchPathExclusionsCacheSize: 200000
projection:
distributionMode: LOCAL # One of [LOCAL | HAZELCAST] LOCAL = projections are performed on the instance process the query, HAZELCAST = distribute projection work to eligible members
distributionPacketSize: 100 # Number of TypedInstances in a distributed workload - only applicable where distributionMode = HAZELCAST
distributionRemoteBias: 10 # Number of nodes in the hazelcast cluster before work is preferentially distributed to remote nodes
hazelcast:
enabled: false # Enable or disable starting a hazelcast cluster
discovery: MULTICAST # One of aws/multicast/swarm
memberTag: vyne-query-service # Member tags allow delegation of responsibilities
eurekaUri: http://127.0.0.1:8761/eureka/
Embedded Vyne
Applications using Vyne in embedded mode will need to make the following changes upon updating to 0.19.0
.
Remove the following annotation from the Spring boot 'application class' (i.e. the type annotated with @SpringBootApplication
)
@EnableConfigurationProperties(VyneCacheConfiguration::class)
Add the following imports:
import io.vyne.spring.config.VyneSpringCacheConfiguration
import io.vyne.spring.config.VyneSpringHazelcastConfiguration
import io.vyne.spring.config.VyneSpringProjectionConfiguration
import io.vyne.spring.http.auth.HttpAuthConfig
import org.springframework.context.annotation.Import
and annotate the Spring boot application class with these additional statements:
@Import(HttpAuthConfig::class)
@EnableConfigurationProperties(VyneSpringCacheConfiguration::class, VyneSpringProjectionConfiguration::class, VyneSpringHazelcastConfiguration::class)
Update Vyne configuration block in your application.yaml
with the new mandatory projection and hazelcast configuration blocks.
As an example, update:
vyne:
schema:
publicationMethod: EUREKA
queryService:
name: QUERY-SERVICE
pipelinesOrchestratorService:
name: PIPELINES-ORCHESTRATOR
caskService:
name: CASK
queryServiceUrl: http://query-service
graph:
vyneGraphBuilderCache:
baseSchemaCacheSize: 100
graphWithFactTypesCacheSize: 100
baseSchemaGraphCacheSize: 100
vyneDiscoverGraphQuery:
schemaGraphCacheSize: 5
searchPathExclusionsCacheSize: 200000
as:
vyne:
queryServiceUrl: http://localhost
schema:
publicationMethod: EUREKA
queryService:
name: query-service
graph:
vyneGraphBuilderCache:
baseSchemaCacheSize: 100
graphWithFactTypesCacheSize: 100
baseSchemaGraphCacheSize: 100
vyneDiscoverGraphQuery:
schemaGraphCacheSize: 5
searchPathExclusionsCacheSize: 200000
projection:
distributionMode: LOCAL # One of [LOCAL | HAZELCAST] LOCAL = projections are performed on the instance process the query, HAZELCAST = distribute projection work to eligible members
distributionPacketSize: 100 # Number of TypedInstances in a distributed workload - only applicable where distributionMode = HAZELCAST
distributionRemoteBias: 10 # Number of nodes in the hazelcast cluster before work is preferentially distributed to remote nodes
hazelcast:
enabled: false # Enable or disable starting a hazelcast cluster
discovery: MULTICAST # One of aws/multicast/swarm
memberTag: vyne-query-service # Member tags allow delegation of responsibilities
eurekaUri: http://127.0.0.1:8761/eureka/
File Schema Server Change
file-schema-server renamed as schema-server and hence starting from version 0.19.0 the corresponding docker image name is vyneco/schema-server
New Features
Cask message retention
Casks can now be configured to retain messages for a specified duration. Messages which are older than this duration are periodically purged.
See Configuring cask retention for more details
Writing Cask messages to Kafka
Casks can now be configured to output messages to Kafka as they are ingested. For casks with a @PrimaryKey
annotation, messages contain both the old and the new states. Otherwise, the full message is output.
Enabling this setting is per-cask, and comes with a performance overhead at ingestion time.
See Writing Cask Messages to kafka for more details.
Authentication headers
Vyne can now be configured to include authentication headers when calling upstream services.
See Authentication Vyne to other services for more details.
Distributing projection work over a cluster
Vyne can now be deployed with a cluster of nodes, and configured to parallelize the projection work of projecting queries.
See clustered deployments for details of how to deploy Vyne in this configuration. This is only supported for query server workloads, and not currently available for embedded clients.
Support for continuous queries in Vyne
Vyne now supports continuous queries - ie., queries from data sources that provide infinite streams of data.
Any service can publish support for continuous queries, by exposing a text/event-stream
endpoint, and defining
the return type as Stream<T>
in the schema.
See Continuous queries for more details.
Continuous query support in Casks
Support for continuous queries has been added to Casks, and is enabled by default.
See Continuous queries with Casks for more details.