Ingesting data into a cask

Cask API allows for an easy way of ingesting your data and exposing it as a query-able API in Vyne.

Cask offers two APIs:

  • Websocket API

  • Rest API

In both cases, a cask is created if not already present when data is submitted.

Websocket API

Websockets are used for publishing a continuous stream of data to a cask.

Once a Websocket session is established, the client can submit many ingestion requests. The body of the ingestion request is just a text.

The format of the URL is as follows:

ws://localhost:8800/cask/[contentType]/[typeReference]?debug=true&[otherParams]

contentType:

  • csv

  • json

typeReference:

  • Fully qualified name of the taxi type that will be used to ingest data (e.g.com.acme.BlogPost)

General parameters:

  • debug - enables ingester response (default debug=false)

  • nullValue - any number of parameter & separated. e.g. nullValue=EMPTY will result in ingester not mapping those values

otherParams:

  • csvDelimiter - csv file column separator (default value ",")

  • csvFirstRecordAsHeader - treat first csv row as list of column names (default value true)

URI examples

JSON

ws://cask:8800/cask/json/com.acme.BlogPost

ws://cask:8800/cask/json/com.acme.BlogPost?debug=true

CSV

ws://cask:8800/cask/csv/com.acme.BlogPost?debug=true&csvDelimiter=,

ws://cask:8800/cask/csv/com.acme.BlogPost?csvDelimiter=;&nullValue=NULL&nullValue=N/A

ws://cask:8800/cask/csv/com.acme.BlogPost?csvDelimiter=:&csvFirstRecordAsHeader=false

Payload examples

JSON

[
{
"id": "order1",
"symbol": "BTCUSD",
"price": "10000",
"side": "SELL",
"date": "2020-01-01T12:00:00.000Z"
},
{
"id": "order2",
"symbol": "ETHUSD",
"price": "300",
"side": "BUY",
"date": "2020-01-02T12:00:00.000Z"
}
]

CSV

id,symbol,price,side,date
order1,BTCUSD,10001,SELL,2020-01-01T12:00:00.000Z
order2,ETHUSD,301,SELL,2020-01-02T12:00:00.000Z

Rest API

Rest API is designed to offer fast and easy way to upload data to Cask. The API is identical to Websocket API (the only difference is /api in the url).

POST http://cask:8800/api/ingest/[contentType]/[typeReference]?debug=true&[otherParams]

Examples

CSV

POST http://localhost:8800/api/ingest/csv/vyne.casks.Order?debug=true&csvDelimiter=,
Payload:
id,symbol,price,side,date
order1,BTCUSD,10001,SELL,2020-01-01T12:00:00.000Z
order2,ETHUSD,301,SELL,2020-01-02T12:00:00.000Z

JSON

POST http://localhost:8800/api/ingest/json/vyne.casks.Order?debug=true
Payload:
{
"id": "order1",
"symbol": "BTCUSD",
"price": "10000",
"side": "SELL",
"date": "2020-02-01T12:00:00.000Z"
}