Schema Server

Overview

This component is responsible for reading Taxi files and distributing them to all the connected vyne components.

At the moment there are two supported modes of operation:

  • Local files

  • Git

Local files mode

In this mode you specify folder where you keep all the schemas. Any modifications in that folder will be automatically detected and propagated between Vyne components. Example docker-compose environment can be found here.

Sample docker-compose file

version: "3.4"
services:
vyne:
image: vyneco/vyne:latest-snapshot
ports:
- 5701-5705
- 9022:9022
environment:
PROFILE: embedded-discovery,inmemory-query-history
file-schema-server:
image: vyneco/file-schema-server:latest-snapshot
ports:
- 5701-5705
volumes:
- /path/to/local/vyne/schemas:/var/lib/vyne/schemas
environment:
OPTIONS: \
--taxi.schema-local-storage=/var/lib/vyne/schemas \
--eureka.uri=http://vyne:9022
cask:
image: vyneco/cask:latest-snapshot
depends_on:
- vyne
ports:
- 5701-5705
- 15432:5432
- 8800:8800
environment:
PROFILE: local
OPTIONS: --eureka.uri=http://vyne:9022

Save the file to your desired location e.g. /dev/vyne/demo/docker-compose.yml.

To start schema server environment: docker-compose up

To stop: ctrl+c or docker-compose down

To add test schema, create a new file e.g. /dev/vyne/demo/schemas/order.taxi with this content:

type alias OrderId as String
type alias Symbol as String
type alias Price as Decimal
type alias Side as String
type alias OrderDate as Instant
type Order {
id: OrderId by column(1)
symbol : Symbol by column(2)
price : Price by column(3)
side: Side by column(4)
@Between
date: OrderDate by column(5)
}

Save it and navigate to http://localhost:9022/schema-explorer, you should see your newly defined schema there.

GIT Mode

In GIT mode, the schema server periodically pulls a predefined git branch to the local folder.

Sample docker-compose configuration:

version: "3.4"
services:
vyne:
image: vyneco/vyne:latest-snapshot
ports:
- 5701-5705
- 9022:9022
environment:
PROFILE: embedded-discovery,inmemory-query-history
file-schema-server:
image: vyneco/file-schema-server:latest-snapshot
ports:
- 5701-5705
volumes:
- /path/to/local/vyne/schemas:/var/lib/vyne/schemas
environment:
OPTIONS: \
--eureka.uri=http://vyne:9022 \
--taxi.schema-local-storage=/var/lib/vyne/schemas \
--taxi.gitCloningJobEnabled=true \
--taxi.gitCloningJobPeriodMs=5000 \
--taxi.gitSchemaRepos[0].name=taxonomy \
--taxi.gitSchemaRepos[0].uri=git@gitlab.com:notional/taxonomy.git \
--taxi.gitSchemaRepos[0].branch=master \
--taxi.gitSchemaRepos[0].sshPrivateKeyPath=/home/ubuntu/.ssh/id_rsa
cask:
image: vyneco/cask:latest-snapshot
depends_on:
- vyne
ports:
- 5701-5705
- 15432:5432
- 8800:8800
environment:
PROFILE: local
OPTIONS: --eureka.uri=http://vyne:9022

Schema server parameters

Option name

Default

Description

taxi.schema-local-storage

--

Location where all the Taxi schema files are stored

taxi.gitCloningJobEnabled

false

Set to true to enable synchronization of schema changes from Git.

taxi.gitCloningJobPeriodMS

300000

(5 minutes)

Schema refresh interval in milliseconds.

taxi.gitSchemaRepos[n].*

--

An array of repositories to pull data from.

Possible options here

  • taxi.gitSchemaRepos[0].name - unique name of respository,

  • taxi.gitSchemaRepos[0].uri - git uri,

  • taxi.gitSchemaRepos[0].branch - branch name, e.g. master,

  • taxi.gitSchemaRepos[0].sshPrivateKeyPath - path to ssh key with read access to the repository.

There can be multiple repositories added here.

Constraints: each repository name, should be unique!