Sending Raw Data

There are two variants of raw data. There is raw data for larger transfers such as e.g. zip files and lightweight raw data for sending small messages with custom data.

Raw Data

Sending raw data can be used to send any arbitrary data, e.g. zip files. The data to be sent is split into chunks that are then sent through the mesh. The data to be sent is called the "payload".

General Overview

Every raw data transaction (except raw_data_light, which will be explained later) starts with a raw_data_start message. This message includes the amount of chunks in the transaction and the protocol of the payload. Once the receiver receives this message, it answers with a raw_data_start_received message, which indicates to the sender that the receiver is ready to receive the chunks.

When the sender receives the raw_data_start_received message, it starts sending all raw_data_chunk objects. In addition of payload data, every raw_data_chunk includes a chunk ID, which is a unique (regarding the current transaction) ascending number, starting at 1 for the first raw_data_chunk object. chunk ID 0 is reserved for the raw_data_start. Using this chunk ID, and the information of the amount of chunks form the raw_data_start message, the receiver is able to determine the last chunk in the transaction. Once it receives the last chunk, it reports back to the sender using a single raw_data_report. This message includes the chunk IDs of up to three missing chunks. If all chunks were received, the list of missing chunks is empty, which tells the sender that the transaction was successful. If however, the list of missing chunks is not empty, the sender must send the chunks with the corresponding chunk IDs again. The last chunk IDs of the previous raw_data_report message acts as a last chunk, regarding the sending of additional raw_data_report. This sending of raw_data_chunk and raw_data_report is repeated until raw_data_report has an empty list of missing chunk IDs.

All devices involved in the communication (sender, receiver, and the mesh) are able to cancel the communication by sending a raw_data_error message to the sender as well as the receiver. If the sender or the receiver is the device that hung up the transmission, it is allowed to not send the error to itself. The raw_data_error message includes an error code, indicating the reason for the cancellation. It is possible to receive a raw_data_error message without an open transmission. This can happen if the sender cancels the transmission using a raw_data_error, but this error is dropped during the transmission. The receiver then might send another raw_data_error indicating a timeout while the sender already canceled the communication. Such messages without an open transmission may be discarded.

Implementation

As any other message in the mesh, every message in the raw send protocol could be dropped. This section describes how an implementation must behave in such scenarios. It also describes the obligations of the sender and the receiver.

Dropped message	Reaction
raw_data_start raw_data_start_received	The sender must send the raw_data_start again after a timeout of 10 seconds or stop the transmission. The receiver has to be able to handle several successive raw_data_start messages with the same content (in case the raw_data_start_received message is dropped) and close a dropped connection after a timeout of 15 seconds (in case the sender does not send another raw_data_start).
raw_data_chunk	Missing chunks are reported in raw_data_report once the last chunk is received. These missing chunks must be resent.
Last raw_data_chunk or raw_data_report	Using the ChunkID, both sender and receiver are able to identify the last data chunk. If this message or the raw_data_report are dropped, the sender must send the last chunk again. This means that the receiver is only allowed to save the last chunk ID once the first chunk after a raw_data_report is received, not immediately after the raw_data_report is sent.
raw_data_error	If a raw_data_error message is dropped, the sender or receiver has already canceled the transmission, leading to the sending of another raw_data_error upon receiving an invalid out-of-transmission message or a raw_data_error indicating a timeout. In the rare cases where the origin of the raw_data_error is the mesh itself, it could happen that both raw_data_error messages are dropped. In such cases the connection is still up, but will probably create another raw_data_error once the ill-formed chunk is sent again.

Dropped message

Reaction

raw_data_start
raw_data_start_received

The sender must send the raw_data_start again after a timeout of 10 seconds or stop the transmission. The receiver has to be able to handle several successive raw_data_start messages with the same content (in case the raw_data_start_received message is dropped) and close a dropped connection after a timeout of 15 seconds (in case the sender does not send another raw_data_start).

raw_data_chunk

Missing chunks are reported in raw_data_report once the last chunk is received. These missing chunks must be resent.

Last raw_data_chunk or raw_data_report

Using the ChunkID, both sender and receiver are able to identify the last data chunk. If this message or the raw_data_report are dropped, the sender must send the last chunk again. This means that the receiver is only allowed to save the last chunk ID once the first chunk after a raw_data_report is received, not immediately after the raw_data_report is sent.

raw_data_error

If a raw_data_error message is dropped, the sender or receiver has already canceled the transmission, leading to the sending of another raw_data_error upon receiving an invalid out-of-transmission message or a raw_data_error indicating a timeout. In the rare cases where the origin of the raw_data_error is the mesh itself, it could happen that both raw_data_error messages are dropped. In such cases the connection is still up, but will probably create another raw_data_error once the ill-formed chunk is sent again.

Transmission Start

raw_data_start [receiverId] [destinationModuleId] [numChunks] [protocolId] {requestHandle = 0} {metadataHex}

This command starts a raw data transmission. The payload shall be sent using raw_data_chunk messages.

Command Parameter	Type	Description
receiverId	u16	The node ID that this message should be sent to
destinationModuleId	u8	The module ID is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.
numChunks	u24	Number of Chunks for the total message. Must not be 0.
protocolId	u8	One of the protocol IDs mentioned in the table below
requestHandle	u8	A handle that can be used to distinguish between different raw data transmissions (Default: 0)
metadataHex	HexString or Base64	Optional binary metadata to send, e.g. 11:22:33:44 or ESIzRA==. The maximum length is 40 bytes of binary data (hex/base64 encoded string will be longer). It can be used to store request specific data like a UUID of a transmission.

Command Parameter

Type

Description

receiverId

u16

The node ID that this message should be sent to

destinationModuleId

The module ID is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.

numChunks

u24

Number of Chunks for the total message. Must not be 0.

protocolId

One of the protocol IDs mentioned in the table below

requestHandle

A handle that can be used to distinguish between different raw data transmissions (Default: 0)

metadataHex

HexString or Base64

Optional binary metadata to send, e.g. 11:22:33:44 or ESIzRA==. The maximum length is 40 bytes of binary data (hex/base64 encoded string will be longer). It can be used to store request specific data like a UUID of a transmission.

Raw Data Protocol ID	Name	Description
0	Invalid	Invalid protocol ID
1	HTTP	A raw HTTP request or response
2	GZIPPED_JSON	A JSON that was gzipped
3	KBM	Mway proprietary building automation protocol
4 - 199	Reserved	Not yet used
200 - 255	User defined	May be different in each implementation

Raw Data Protocol ID

Name

Description

Invalid

Invalid protocol ID

HTTP

A raw HTTP request or response

GZIPPED_JSON

A JSON that was gzipped

KBM

Mway proprietary building automation protocol

4 - 199

Reserved

Not yet used

200 - 255

User defined

May be different in each implementation

If received by a device with a terminal, raw_data_start is printed out like this:

// without metadata
{
	"nodeId":5,
	"type":"raw_data_start",
	"module":4,
	"numChunks":3,
	"protocol":1,
	"fmKeyId":2,
	"requestHandle":0
}

// with metadata
{
	"nodeId":5,
	"type":"raw_data_start",
	"module":4,
	"numChunks":3,
	"protocol":1,
	"fmKeyId":2,
	"requestHandle":0,
	"metadata":"ESIzRA=="
}

Accept Transmission

raw_data_start_received [receiverId] [destinationModuleId] {requestHandle = 0} {metadataHex = ""}

Once a raw_data_start is received, the receiver shall send the sender a raw_data_start_received message.

Command Parameter	Type	Description
receiverId	u16	The node ID that this message should be sent to
destinationModuleId	u8	The module ID is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.
requestHandle	u8	A handle that can be used to distinguish between different raw data transmissions (Default: 0)
metadataHex	HexString or Base64	Optional binary metadata to send, e.g. 11:22:33:44 or ESIzRA==. The maximum length is 40 bytes of binary data (hex/base64 encoded string will be longer). It can be used to store request specific data like a UUID of a transmission.

Command Parameter

Type

Description

receiverId

u16

The node ID that this message should be sent to

destinationModuleId

The module ID is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.

requestHandle

A handle that can be used to distinguish between different raw data transmissions (Default: 0)

metadataHex

HexString or Base64

If received by a JSON-capable device, the raw_data_start_received is printed out like this:

// without metadata
{
	"nodeId":5,
	"type":"raw_data_start_received",
	"module":4,
	"requestHandle":0
}

// with metadata
{
	"nodeId":5,
	"type":"raw_data_start_received",
	"module":4,
	"requestHandle":0,
	"metadata":"ESIzRA=="
}

Subsequent Chunk Messages

raw_data_chunk [receiverId] [destinationModuleId] [chunkId] [payloadHex] {requestHandle = 0}

Once a raw transmission has been started, the appropriate number of chunks should follow in the correct order. Once the last chunk is received by the receiver, it is possible to reassemble and parse the whole message. The moduleId is present in all chunks so that they can be assigned to the correct stream and to avoid clashes between different modules. A module can send intermittent data streams if is uses different request handles.

Command Parameter	Type	Description
receiverId	u16	The NodeID that this message should be sent to
destinationModuleId	u8	The ModuleId is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.
chunkId	u24	ID of this data chunk starting from 1.
payloadHex	HexString or Base64String	The binary data to send. E.g. AA:BB:CC. The maximum length is 60 bytes for HexStrings, 120 bytes for Base64Strings.
requestHandle	u8	A handle that can be used to distinguish between different raw data transmissions (Default: 0)

Command Parameter

Type

Description

receiverId

u16

The NodeID that this message should be sent to

destinationModuleId

The ModuleId is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.

chunkId

u24

ID of this data chunk starting from 1.

payloadHex

HexString or Base64String

The binary data to send. E.g. AA:BB:CC. The maximum length is 60 bytes for HexStrings, 120 bytes for Base64Strings.

requestHandle

A handle that can be used to distinguish between different raw data transmissions (Default: 0)

If received by a JSON-capable device, the raw_data_chunk is printed out like this:

{
	"nodeId":5,
	"type":"raw_data_chunk",
	"module":4,
	"chunkId":1,
	"payload":"abcdeQ==",
	"requestHandle":0
}

Send Report

raw_data_report [receiverId] [destinationModuleId] [MissingChunkIds] {requestHandle = 0}

Once the last chunk is received, the receiver sends this message to the sender, indicating either a successful transmission (empty missing chunk IDs) or informs the sender about missing chunk IDs.

Command Parameter	Type	Description
receiverId	u16	The node ID that this message should be sent to
destinationModuleId	u8	The module ID is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.
MissingChunkIds	Comma-separated integers or the literal string "-" (without "")	Up to three chunk IDs of missing chunks. Must not contain spaces! E.g. 2,17,312
requestHandle	u8	A handle that can be used to distinguish between different raw data transmissions. If the transmission has been started with a request handle, this message must use the same request handle. (Default: 0)

Command Parameter

Type

Description

receiverId

u16

The node ID that this message should be sent to

destinationModuleId

The module ID is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.

MissingChunkIds

Comma-separated integers or the literal string "-" (without "")

Up to three chunk IDs of missing chunks. Must not contain spaces! E.g. 2,17,312

requestHandle

A handle that can be used to distinguish between different raw data transmissions. If the transmission has been started with a request handle, this message must use the same request handle. (Default: 0)

If received by a JSON-capable device, the raw_data_report is printed out like this:

{
	"nodeId":5,
	"type":"raw_data_report",
	"module":4,
	"missing":[2,17,312],
	"requestHandle":0
}

Or in cases where the transmission was successful:

{
	"nodeId":5,
	"type":"raw_data_report",
	"module":4,
	"missing":[],
	"requestHandle":0
}

Send Report Desired

raw_data_report_desired [receiverId] [destinationModuleId] {requestHandle = 0}

The "Report Desired" message is an optional message that can speed up transmissions in cases where the current last chunk was dropped. It is send from the sender to the receiver, indicating that the sender wishes to get a raw_data_report from the receiver. The receiver however may completely ignore this message, completely relying on the timeouts. The sender may also never send it, also completely relying on the timeouts.

Command Parameter	Type	Description
receiverId	u16	The node ID that this message should be sent to
destinationModuleId	u8	The module ID is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.
requestHandle	u8	A handle that can be used to distinguish between different raw data transmissions (Default: 0)

Command Parameter

Type

Description

receiverId

u16

The node ID that this message should be sent to

destinationModuleId

The module ID is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.

requestHandle

A handle that can be used to distinguish between different raw data transmissions (Default: 0)

If received by a JSON-capable device, the raw_data_report_desired is printed out like this:

{
	"nodeId":5,
	"type":"raw_data_report_desired",
	"module":4,
	"requestHandle":0
}

Send Error

raw_data_error [receiverId] [destinationModuleId] [errorCode] [destination] {requestHandle = 0}

This command indicates that some error occurred and the transmission must be closed. The packet will be sent to the receiver as well as the sender.

Error Code	Name	Meaning
0	Reserved
1	Unexpected end of transmission	Three timeouts happened without receiving a message from the transmission partner.
2	Not in a transmission	A raw_data_chunk or raw_data_report was received without an open transmission.
3	Malformed Message	A message was received which was malformed and did not fit any other error code.
4	Unsupported Protocol	The receiver is unable to interpret the given protocol.
5	Malformed GZIP	The receiver got all chunks but could could not unpack the GZIP.
6	Malformed Type	Thrown in case of a GZIP communication. The unpacking worked, but the provided type inside the GZIP JSIN was unknown.
7	Invalid Chunk ID	The given chunk ID was out of range.
8	Transmission Aborted	The transmission was aborted on purpose.
9	Metadata Unparsable	The metadata that was sent in raw_data_start could not be parsed.
10	Not Interested	The receiver is not interested in the transmission because of the parsed metadata.
11	Out of Memory	There is not enough memory left to continue the transmission
12	Terminal not Reachable	A terminal to forward the data is not connected or not available.
13	Implementation not Reachable	The implementation is not ready to process any data.
…	Reserved	Not yet used
200 - 255	User defined	May be different in each implementation.

Error Code

Name

Meaning

Reserved

Unexpected end of transmission

Three timeouts happened without receiving a message from the transmission partner.

Not in a transmission

A raw_data_chunk or raw_data_report was received without an open transmission.

Malformed Message

A message was received which was malformed and did not fit any other error code.

Unsupported Protocol

The receiver is unable to interpret the given protocol.

Malformed GZIP

The receiver got all chunks but could could not unpack the GZIP.

Malformed Type

Thrown in case of a GZIP communication. The unpacking worked, but the provided type inside the GZIP JSIN was unknown.

Invalid Chunk ID

The given chunk ID was out of range.

Transmission Aborted

The transmission was aborted on purpose.

Metadata Unparsable

The metadata that was sent in raw_data_start could not be parsed.

Not Interested

The receiver is not interested in the transmission because of the parsed metadata.

Out of Memory

There is not enough memory left to continue the transmission

Terminal not Reachable

A terminal to forward the data is not connected or not available.

Implementation not Reachable

The implementation is not ready to process any data.

…

Reserved

Not yet used

200 - 255

User defined

May be different in each implementation.

Destination Code	Name	Meaning
1	Sender	The error is sent to the sender.
2	Receiver	The error is sent to the receiver.
3	Both	The error is sent both to the sender and the receiver.

Destination Code

Name

Meaning

Sender

The error is sent to the sender.

Receiver

The error is sent to the receiver.

Both

The error is sent both to the sender and the receiver.

If received by a JSON-capable device, the raw_data_error is printed out like this:

{
	"nodeId":5,
	"type":"raw_data_error",
	"module":4,
	"error":1,
	"destination":1
	"requestHandle":0
}

Lightweight Raw data

Sending messages that fit into a single mesh packet can be done by using raw_data_light with minimal overhead and almost no implementation effort.

Terminal Command

raw_data_light [receiverId] [destinationModuleId] [protocolId] [payload] {requestHandle = 0}

Sends a single chunk of arbitrary data to the receiver. There is no guarantee that the message is transmitted if the mesh is in the process of restructuring or busy. The sender should provide some resending logic if desired.

Command Parameter	Type	Description
receiverId	u16	The node ID that this message should be sent to
destinationModuleId	u8	The module ID is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.
protocolId	u8	One of the protocol IDs mentioned in the table of raw_data_start
payloadHex	HexString or Base64String	The binary data to send. E.g. AA:BB:CC. The maximum length is 60 bytes for HexStrings, 120 bytes for Base64Strings.
requestHandle	u8	A handle that can be used to distinguish between different raw data transmissions (Default: 0)

Command Parameter

Type

Description

receiverId

u16

The node ID that this message should be sent to

destinationModuleId

The module ID is used for giving context to this message. If the transmission should only be printed on the receiver and otherwise be ignored by the firmware, it must be set to 0.

protocolId

One of the protocol IDs mentioned in the table of raw_data_start

payloadHex

HexString or Base64String

The binary data to send. E.g. AA:BB:CC. The maximum length is 60 bytes for HexStrings, 120 bytes for Base64Strings.

requestHandle

A handle that can be used to distinguish between different raw data transmissions (Default: 0)

Terminal Response

If received by a JSON-capable device, then raw_data_light is printed out like this:

{
	"nodeId":5,
	"type":"raw_data_light",
	"module":4,
	"protocol":2,
	"payload":"abcdeQ==",
	"requestHandle":0
}

The mesh message is documented here.

Messages

Raw Data Start Message

Bytes	Type	Name	Description
8	connPacketModule	Conn Packet Module	Message Type = 54, Action Type = 0.
3	u24	Number of Chunks	The total amount of raw_data_chunk messages for this transmission.
1	u8	Protocol ID	See above for a list of valid protocol IDs.
4	u32	fmKeyId	See EncryptionKeys Heading.
0-40	u8	metadataHex	Optional metadata

Bytes

Type

Name

Description

connPacketModule

Conn Packet Module

Message Type = 54, Action Type = 0.

u24

Number of Chunks

The total amount of raw_data_chunk messages for this transmission.

Protocol ID

See above for a list of valid protocol IDs.

u32

fmKeyId

See EncryptionKeys Heading.

0-40

metadataHex

Optional metadata

Raw Data Start Received Message

Bytes	Type	Name	Description
8	connPacketModule	Conn Packet Module	Message Type = 54, Action Type = 1.
0-40	u8	metadataHex	Same metadata as included in the raw data start message

Bytes

Type

Name

Description

connPacketModule

Conn Packet Module

Message Type = 54, Action Type = 1.

0-40

metadataHex

Same metadata as included in the raw data start message

Raw Data Error Message

Bytes	Type	Name	Description
8	connPacketModule	Conn Packet Module	Message Type = 54, Action Type = 4.
1	u8	Error Code	See above for a list of possible error codes.
1	u8	Destination Code	See above for a list of possible Destination Codes.

Bytes

Type

Name

Description

connPacketModule

Conn Packet Module

Message Type = 54, Action Type = 4.

Error Code

See above for a list of possible error codes.

Destination Code

See above for a list of possible Destination Codes.

Raw Data Chunk message

Bytes	Type	Name	Description
8	connPacketModule	Conn Packet Module	Message Type = 54, Action Type = 2.
3	u24	Chunk ID	The ID of this chunk. The first chunk has ID 1.
1	u8	reserved	A reserved value that must be set to 0.
1-60	u8[1-60]	payload	The payload.

Bytes

Type

Name

Description

connPacketModule

Conn Packet Module

Message Type = 54, Action Type = 2.

u24

Chunk ID

The ID of this chunk. The first chunk has ID 1.

reserved

A reserved value that must be set to 0.

1-60

u8[1-60]

payload

The payload.

Raw Data Report Message

Bytes	Type	Name	Description
8	connPacketModule	Conn Packet Module	Message Type = 54, Action Type = 3.
12	u32[3]	missings	The IDs of the missing chunks.

Bytes

Type

Name

Description

connPacketModule

Conn Packet Module

Message Type = 54, Action Type = 3.

u32[3]

missings

The IDs of the missing chunks.

Raw Data Report Desired Message

Bytes	Type	Name	Description
8	connPacketModule	Conn Packet Module	Message Type = 54, Action Type = 5.

Bytes

Type

Name

Description

connPacketModule

Conn Packet Module

Message Type = 54, Action Type = 5.

Raw Data Light Message

Bytes	Type	Name	Description
5	connPacketHeader	header	Message Type = 55
1	u8	moduleId	The id of the module that should process the message. 0 to only print it at the gateway.
1	u8	requestHandle	Used to identify the message
1	RawDataProtocolId	protocolId	Raw Data Protocol Id as given above
1-60	u8[1-60]	payload	The custom payload to send

Bytes

Type

Name

Description

connPacketHeader

header

Message Type = 55

moduleId

The id of the module that should process the message. 0 to only print it at the gateway.

requestHandle

Used to identify the message

RawDataProtocolId

protocolId

Raw Data Protocol Id as given above

1-60

u8[1-60]

payload

The custom payload to send