This document describes a draft protocol for serializing messages between numerical models and between numerical modesl and other programs.
There is already a common protocol in use for model messages, MPI. This works great for communication within a model. Since MPI2 it can also be used to setup ad hoc communication between models.
There are scenarios where communication through MPI is not the logical approach. The main use cases are:
- Inter language communication (Javascript/C#/Windows support for MPI is unavailbel/lagging)
- Flexible process structures (starting different programs on different machines)
- Communcating through the web.
Here we describe the serialization protocol that can be used as a layer on top of alternative messaging protocols such as ZMQ and WebSockets.
Here we mainly focus on sending over arrays, of rank n, of simple types. with a variable number of attributes. We base our data model on the Variables and Attributes from the Common Data Model.
The message content is a variable, the variable is split up into a metadata message and the data.
The metadata message is always utf8 encoded and is send in JSON format. It contains at least the following attributes:
{
name: "variable",
shape: [3,3],
dtype: "float64"
}
The data message intersects the buffer protocol and the ArrayBuffer protocol.
An extended example:
{
name: "variable",
shape: [3,3],
dtype: "float64",
continuguous: "C",
attributes: {
standard_name: "sea_surface_altiude",
units: "m"
},
strides: [[0,1],[0,2]]
}
