Transaction Ser/Des

1. Introduction

This document contains description of raw transaction protocol as provided by transaction-serdes modules. With default configuration, transaction-serdes is listening on 127.0.0.1:6378/tcp.

As this module provides unlimited access to OpenDAF transaction mechanism, it's imperative not to expose this interface to the world, i.e. transaction-serdes shall not listen on 0.0.0.0:6378/tcp. External clients shall access this interface via ssh tunnel. User credentials are of course required for such access.

2. Protocol description

Protocol is a symmetric, very native, low-level, binary, and its description will be similar :-) It's purpose is a high-speed transaction streaming between OpenDAF components. Symmetric means client and server both use the same stream structure and can both transmit and receive transactions.

There is no padding / alignment in serialized data structures.

2.1 Notations used

notation	description
`{ type }`	describes serialized layout of given structure type
`[ type ]`	represents one instance of serialized structure of given type
`type name`	represents discrete field of given data type named "name" (discrete data types have serialized layout identical to in-memory layout)
`[N * type]`	represents N instances of discrete type or serialized structures type
`<type1\|type2\|...\|typeN>`	represents any of discrete or serialized structure types mentioned
`//`	line comment
`/* */`	block comment

2.2 Datatypes used

datatype	description
string	variable-length null-terminated string
uint16_t, uint32_t	standard C types
int	signed integer type, 32-bit on both 32-bit and 64-bit platforms
float	IEEE-754 32-bit float
double	IEEE-754 64-bit float

2.3 Protocol stream structure

Protocol stream carries only [ TransactionContainer ] entities.

{ TransactionContainer }
    - int transaction_type          // 0x0001 - SetTransactionMaskTransaction
                                    // 0x0100 - MeasurementUpdateTransaction
                                    // 0x0200 - CommandWriteTransaction
                                    // 0x0400 - InitialMeasurementUpdateTransaction
                                    // 0x0800 - InitialCommandWriteTransaction
                                    // 0x1000 - InitialAlarmStateChangeTransaction
                                    // 0x2000 - AlarmStateChangeTransaction
    - uint32_t transaction_size     // size of transaction field following
    - <[SetTransactionMaskTransaction]|[MeasurementUpdateTransaction]|[CommandWriteTransaction]|AlarmStateChangeTransaction]> transaction

2.4 Transaction layouts

{ SetTransactionMaskTransaction }
    - uint64_t mask         // bitmask of transactions peer should start transmitting to us

{ MeasurementUpdateTransaction }
    - uint32_t n_updates    // # of updates following
    - [ n_updates * [ MeasurementUpdate ] ] updates

{ InitialMeasurementUpdateTransaction }
    - [ MeasurementUpdateTransaction] initial_state

{ CommandWriteTransaction }
    - uint32_t n_writes     // # of writes following
    - [ n_writes * [ CommandWrite] ] writes

{ InitialCommandWriteTransaction }
    - [ CommandWriteTransaction ] initial_state

{ AlarmStateChangeTransaction }
    - uint32_t n_changes    // # of changes following
    - [ Timestamp ] timestamp
    - string authority
    - [ n_changes * [ AlarmStateChange ] ]

2.5 Auxiliary structures layout

{ InitialAlarmStateChangeTransaction }
    - [ AlarmStateChangeTransaction ] initial_state

{ MeasurementUpdate }
    - string measurement_name
    - [ VTQ ] vtq

{ CommandWrite }
    - string command_name
    - [ VT ] vt

{ AlarmStateChange }
    - string alarm_name
    - int state         // 0 - INACT_ACK, 1 - ACT_UNACK, 2 - ACT_ACK, 3 - INACT_UNACK

{ VTQ }
    - [ VT ] vt
    - uint16_t quality      // quality & 0x00C0 => C0 = good, 00 = bad, 80 = uncertain, 40 = unused

{ VT }
    - [ Value ] value
    - [ T ] timestamp

{ T }
    - uint64_t value    // [] = us

{ Value }
    - uint8_t datatype                                  // 0 - empty, 1 - integer, 2 - long, 3 - float, 4 - double, 5 - binary, 6 - quaternary, 7 - string
    - <int|int64_t|float|double|uint8_t|string> value   // non-trivial datatype mapping: empty -> nothing, binary -> uint8_t, quaternary -> uint8_t

3. Usage scenarios

3.1 Realtime data acquisition

This scenario assumes clients wants to read all data from OpenDAF in realtime.

Sequence of events: 1. client connects to transaction serdes 2. client transmits SetTransactionMaskTransaction with either mask = 0x0100 or mask = 0x0500 3. server transmits SetTransactionMaskTransaction too, client can safely skip it 4. server transmits InitialMeasurementUpdateTransaction (if 0x0400 is present in mask) 5. servers continuously transmits MeasurementUpdateTransactions