OSF5 – Specific documentation

This document describes all aspects of Open Streaming Format Version 5 (OSF5) that go beyond the general OSF description.
It supplements the file osf_general.md, which explains all structures common to OSF4 and OSF5.

OSF5 is the further development of OSF4. It uses JSON as the standard format for the metablock and simplifies the control byte.
At the same time, OSF5 remains fully backward compatible with OSF4.

Magic headers in OSF5

• Allowed identifiers:

  • OSF5
  • OSF4 (for backward compatibility)
  • OCEAN_STREAMING_FORMAT4 (historical)

• Format:

  OSF5 84512\n

• Recognition of the meta block:

  • First character < → XML (OSF4 metablock)
  • First character { → JSON (OSF5 metablock)

• Properties:

  • JSON metablock by default.
  • OSF5 parsers can read OSF4 XML.

Metablock in OSF5 (JSON)

OSF5 uses JSON for the metablock by default.
The structure is functionally equivalent to the XML variant from OSF4, but is easier to parse and optimized for embedded systems.

Example:

{
  "osf": {
    "version": 5,
    "created_utc": "2025-07-27T12:00:00Z",
    "creator": "smartdevice:15002000001",
    "channels": [
      {
        "index": 0,
        "name": "Sensor/Temperature",
        "channeltype": "scalar",
        "datatype": "double",
        "timeincrement": 1000000,
        "sizeoflengthvalue": 2,
        "physicalunit": "°C"
      }
    ]
  }
}

• Difference to OSF4: JSON instead of XML, but same logical content.
• Compatibility: OSF5 can still interpret OSF4 XML.

Simplifications in the control byte (block types)

OSF5 adopts the block structure from OSF4, but reduces the number of block types used and simplifies their interpretation.

Changes compared to OSF4:

• bcContinuedRelStampData is no longer used.
• bcStatusEvent and bcMessageEvent are omitted.
• Bit 7 for single/multiple value remains unchanged.

Supported block types in OSF5:

Value	Enum	Description
0	bcReserved	Reserved for internal purposes.
2	bcTimebaseRealign	Time base adjustment (rarely used).
5	bcContinuedData	Continue data with fixed sampling rate.
6	bcStartData	Start block with fixed sampling rate.
8	bcAbsTimeStampData	Data with absolute timestamp.

Supported data types in OSF5

OSF5 supports all data types from OSF4 as well as two new composite types for multi-component values:

Data type	Size	Description
bool	1 byte	True/False
int8	1 byte	Signed integer
int16	2 bytes	Signed integer
int32	4 bytes	Signed integer
int64	8 bytes	Signed integer
float	4 bytes	IEEE 754 single precision
double	8 bytes	IEEE 754 double precision
string	variable	UTF-8, length determined by block
binary	variable	Any byte sequence, optionally with mimetype
candata	16 bytes	Structure for CAN frames
gpsdata	24 bytes	Structure for GPS positions
pair	16 bytes	Two double values, e.g. X/Y or real/imaginary
triple	24 bytes	Three double values, e.g., 3D acceleration

Structure pair:

|

struct pair {
    double v1;
    double v2;
};

Structure triple:

struct triple {
    double v1;
    double v2;
    double v3;
};

Trailer and info block

OSF5 no longer supports info data blocks (0xFFFF) or magic trailers.
The end of the file is defined solely by reaching the last completely written data block.

• Consequence:
  • Parsers read the file up to the last consistent block.
  • There is no additional statistical or meta information at the end of the file.
  • For time interval information, the file must be analyzed completely or partially.

Why OSF5 no longer uses a trailer or info block

In OSF4, there was an optional info data block (index 0xFFFF) and a magic trailer at the end of the file to make meta information and the time interval of the file quickly available.
In OSF5, we deliberately removed both elements.

Main reason: Reduced implementation effort

• Writing the info block requires final processing of all channels and time information.
• For the reading side, this means additional code that must be maintained and tested alongside the regular data stream.
• Since OSF is designed to read files up to the last complete block anyway, the trailer and info block do not provide any technical added value that justifies the effort.

Additional advantages

• Simpler specification: Fewer special cases, leaner implementation on the embedded and PC side.
• No duplicate information storage: All relevant metadata is available in the meta block and in the data blocks themselves.
• Analysis tools can do the same: Time intervals and statistics can be derived from the regular blocks during reading.

Conclusion:
The decision to dispense with trailers and info blocks in OSF5 is primarily a decision in favor of a simpler, more easily maintainable implementation.
The robustness of the format remains completely intact—it results from the block structure and not from additional elements at the end of the file.

Special features and limitations of OSF5

• JSON as the main format, XML only for backward compatibility.
• Simplified control byte with fewer block types.
• New data types pair and triple.
• No trailers or info data blocks at the end of the file.
• bcContinuedRelStampData, bcStatusEvent, bcMessageEvent are no longer generated.

Example of an OSF5 file

OSF5 2048
{
  "osf": {
    "version": 5,
    "created_utc": "2025-07-27T12:00:00Z",
    "creator": "smartdevice:15002000001",
    "channels": [
      {
        "index": 0,
        "name": "Sensor/Temperature",
        "channeltype": "scalar",
        "datatype": "double",
        "timeincrement": 1000000
      },
      {
        "index": 1,
        "name": "Sensor/ForcePath",
        "channeltype": "scalar",
        "datatype": "pair"
      }
    ]
  }
}
[BEGIN OF BINARY DATA]