/

Schema Server

Learn how to launch schema server


Overview

The Schema Server is responsible for reading Taxi files and distributing them to all the connected vyne components.

The server may be configured to read files from a local file system, or from a remote git repository.

Local files mode

In this mode you specify folder where you keep all the schemas. Any modifications in that folder will be automatically detected and propagated between Vyne components. Example docker-compose environment can be found here.

Sample docker-compose file

version: "3.4"
services:
   vyne:
      image: vyneco/vyne:latest-snapshot
      ports:
         - 5701-5705
         - 9022:9022
      environment:
         PROFILE: embedded-discovery,inmemory-query-history

   file-schema-server:
      image: vyneco/file-schema-server:latest-snapshot
      ports:
         - 5701-5705
      volumes:
         - /path/to/local/vyne/schemas:/var/lib/vyne/schemas
      environment:
         OPTIONS: \
            --taxi.schema-local-storage=/var/lib/vyne/schemas \
            --eureka.uri=http://vyne:9022

   cask:
      image: vyneco/cask:latest-snapshot
      depends_on:
         - vyne
      ports:
         - 5701-5705
         - 15432:5432
         - 8800:8800
      environment:
         PROFILE: local
         OPTIONS: --eureka.uri=http://vyne:9022

Save the file to your desired location e.g. /dev/vyne/demo/docker-compose.yml.

To start schema server environment: docker-compose up

To stop: ctrl+c or docker-compose down

To add test schema, create a new file e.g. /dev/vyne/demo/schemas/order.taxi with this content:

type alias OrderId as String
type alias Symbol as String
type alias Price as Decimal
type alias Side as String
type alias OrderDate as Instant

type Order {
    id: OrderId by column(1)
    symbol : Symbol by column(2)
    price : Price by column(3)
    side: Side by column(4)
    @Between
    date: OrderDate by column(5)
}

Save it and navigate to http://localhost:9022/schema-explorer, you should see your newly defined schema there.

Configuring change detection mode

There are two different types of change detection available for detecting changes to source files made in the local file system.

The mode is defined by setting --taxi.change-detection-method as a startup parameter.

Watch mode (Default)

--taxi.change-detection-method=watch

When watch mode is enable, file system events trigger changes. This provides the fastest feedback, and is a sensible default.

Watch mode cannot be used when running the schema server within a docker container with a host mounted volume for the taxi sources. This is due to limitations in Java's FileWatcher API. In this scenario, Poll mode should be enabled

Poll mode

--taxi.change-detection-method=poll --taxi.schema-poll-interval-seconds=2

Poll mode will periodically poll the directory for changes, using a configurable poll interval.

Remote git repository

The file schema server may be configured to periodically pull a predefined git branch to the local folder.

Sample docker-compose configuration:

version: "3.4"
services:
   vyne:
      image: vyneco/vyne:latest-snapshot
      ports:
         - 5701-5705
         - 9022:9022
      environment:
         PROFILE: embedded-discovery,inmemory-query-history

   file-schema-server:
      image: vyneco/file-schema-server:latest-snapshot
      ports:
         - 5701-5705
      volumes:
         - /path/to/local/vyne/schemas:/var/lib/vyne/schemas
      environment:
         OPTIONS: \
            --eureka.uri=http://vyne:9022 \
            --taxi.schema-local-storage=/var/lib/vyne/schemas \
            --taxi.gitCloningJobEnabled=true \
            --taxi.gitCloningJobPeriodMs=5000 \ 
            --taxi.gitSchemaRepos[0].name=taxonomy \
            --taxi.gitSchemaRepos[0].uri=git@gitlab.com:notional/taxonomy.git \
            --taxi.gitSchemaRepos[0].branch=master \
            --taxi.gitSchemaRepos[0].sshPrivateKeyPath=/home/ubuntu/.ssh/id_rsa

   cask:
      image: vyneco/cask:latest-snapshot
      depends_on:
         - vyne
      ports:
         - 5701-5705
         - 15432:5432
         - 8800:8800
      environment:
         PROFILE: local
         OPTIONS: --eureka.uri=http://vyne:9022

Schema server parameters

Option nameDefaultDescription
taxi.schema-local-storage--Location where all the Taxi schema files are stored
taxi.change-detection-methodwatchPossible options: watch | poll. See configuring change detection for more detail.
taxi.gitCloningJobEnabledfalseSet to true to enable synchronization of schema changes from Git.
taxi.gitCloningJobPeriodMS

300000

(5 minutes)

Schema refresh interval in milliseconds.
taxi.gitSchemaRepos[n].*--

An array of repositories to pull data from.

Possible options here

  • taxi.gitSchemaRepos[0].name - unique name of respository,
  • taxi.gitSchemaRepos[0].uri - git uri,
  • taxi.gitSchemaRepos[0].branch - branch name, e.g. master,
  • taxi.gitSchemaRepos[0].sshPrivateKeyPath - path to ssh key with read access to the repository.

There can be multiple repositories added here.

Constraints: each repository name, should be unique!