Underlying Transport over MQTT
The main goal of this document is to provide instruction on how to expand the transport layer for OOI Networking of the
Semantic-Dataover the Message Queue Telemetry Transport (MQTT) to be compliant with the specifications mentioned in the section Normative references.
Implementation of the
Messagesexchange over the MQTT protocol is out of the scope of this project. The library intentionally is designed to use any transport protocol meeting some basic requirements using external components. External components implementing MQTT connectivity can be seamlessly integrated with this SDK using dependency injection concept illustrated by the following domain model:
Figure 1 Domain Model
Instruction for implementer is covered in the section Notices for Implementer.
The following documents, in whole or in part, are normatively referenced in this document and are indispensable for its application.
The Message Queue Telemetry Transport (MQTT) is an open standard application layer protocol. It is a
Serverpublish/subscribe messaging transport protocol. The protocol runs over TCP/IP, WebSocket or over other network protocols that provide ordered, lossless, bi-directional connections. According to the specification its features include:
- Use of the publish/subscribe message pattern which provides one-to-many message distribution and decoupling of applications.
- A messaging transport that is agnostic to the content of the payload.
- A small transport overhead and protocol exchanges minimized to reduce network traffic.
- A mechanism to notify interested parties when an abnormal disconnection occurs.
The MQTT protocol defines a binary protocol used to send and receive
Application Message. The
Application Messageis the data carried by the protocol across the network for the application. When
Application Messagesare transported they have an associated
Quality of Service(
QoS) and a
Topic Nameis the label attached to an
Application Messagewhich is matched against the
Subscriptionsknown to the
Serversends a copy of the
Application Messageto each Client that has a matching
Server(called also a broker) acts as an intermediary between
Clientswhich have made
Application Messagespublished by
- Processes subscribe and unsubscribe requests from
Application Messagesthat match
Application Messagesso they can be delivered even if the
Subscriberis not online.
Network Connectionto the
Server. It can:
Application Messagesthat other
Clientsmight be interested in.
Application Messagesthat it is interested in receiving.
- Unsubscribe (dispose of
Subscription) to remove a request for
- Disconnect from the
Topic Filterand a maximum
Subscriptionis associated with a single
Sessioncan contain more than one
Sessionhas a different
Topic Filteris an expression contained in a
Subscription, to indicate an interest in one or more topics. A
Topic Filtercan include wildcard characters.
The session is a stateful interaction between a
Sessionslast only as long as the
Network Connection, others can span multiple consecutive
Network Connectionsbetween a
The interaction between the
Serveris controlled using
Control Packet. The
Control Packetis a packet of data that is sent across the
Network Connection. The specification defines fourteen different types of
Control Packet, one of which (the PUBLISH packet) is used to convey
- "At most once", where messages are delivered according to the best efforts of the operating environment. Message loss can occur. This level could be used, for example, with ambient sensor data where it does not matter if an individual reading is lost as the next one will be published soon after.
- "At least once", where messages are assured to arrive but duplicates can occur.
- "Exactly once", where message are assured to arrive exactly once. This level could be used, for example, with billing systems where duplicate or lost messages could lead to incorrect charges being applied.
The connection has
Keep Aliveattribute. It is a time interval measured in seconds and expressed as a 16-bit word. It is the maximum time interval that is permitted to elapse between the point in time at which the Client finishes transmitting one
Control Packetand the point it starts sending the next. It is the responsibility of the
Clientto ensure that the interval between
Control Packetsbeing sent does not exceed the
The specification introduces a concept of topic levels. The topic level separator is used to introduce structure into the
Topic Name. If present, it divides the
Topic Nameinto multiple topic levels. A
Topic Filtercan contain special wildcard characters, which allow
Clientto subscribe to multiple topics at once.
The OPC.UA.PubSub offers the publish/subscribe communication pattern as an option to client-server pattern and is a consistent part of the OPC UA specifications suit. The detailed description of the OPC.UA.PubSub has been covered by the document OPC Unified Architecture Part 14: PubSub Main Technology Features.
The specification defines the following actors:
Publisher: is the actor that pushes
NetworkMessagestructures to an underlying transport layer.
Subscriber: is the actor that consumes data encapsulated by the
NetworkMessagestructure, which is polled from the underlying transport layer.
According to the specification the
Subscriberdon't have any subscriptions management functionality, namely, they follow a communication paradigm called unsolicited notification. When unsolicited notification occurs, a client receives a message that it has never requested. The
Subscribermust use a filtering mechanism to process only messages it is interested in.
MQTT does not provide a mechanism for specifying the encoding of the
Application Messagewhich means the OPC.UA.PubSub
Subscriberentities shall be configured in advance with knowledge of the expected encoding.
Publisherentities should only publish
NetworkMessagesusing a single encoding to a unique MQTT
The messages sent through MQTT are limited to one per
Application Message. It is expected that the software used to receive the message can process it without needing to know that it was transported via MQTT instead of UDP for example.
It is recommended that the MetaDataQueueName as described in OPC.UA.PubSub is configured as a sub-topic of the related
QueueNamewith the name
$Metadata. The MQTT RETAIN flag shall be set for metadata messages.
The implementation choses packet and message size limits depending on the capabilities of the operating system or the capabilities of the device the application is running on. They can be made configurable through configuration model extensions or by other means.
Security with MQTT is primarily provided by a TLS connection between the
Server, however, this requires that the
Servermust be trusted. For that reason, it may be necessary to provide end-to-end security. Applications that require end-to-end security with MQTT need to use the binary message encoding and apply security protection defined in the OPC.UA.PubSub specification.
The syntax of the MQTT transporting protocol URL used has the following form:
The default port is 8883.
The syntax for an MQTT URL over Web Sockets has the following form:
The default port is 443.
KeepAliveTimeis set on a
WriterGroup, a value slightly higher than the configured value of the group in seconds should be set as MQTT
Keep Aliveensuring that the connection is disconnected if the keep alive message was not sent by any writer in the specified time.
BrokerTransportQualityOfServicevalues map to MQTT
Quality of Servicesettings as follows:
- AtMostOnce_1 is mapped to MQTT QoS 0.
- AtLeastOnce_2 is mapped to MQTT QoS 1.
- ExactlyOnce_3 is mapped to MQTT Qos 2.
In the article Reactive Networking of Semantic-Data Library the section Message Transport contains description covering instruction for the external MQTT handling components. An example how to implement the Transport layer over the UDP protocol is illustrated by the project UA Data Example Application. This application uses two implementations of the
ConsumerMessageHandlerFactory- to create communication infrastructure for the consumer role
ProducerMessageHandlerFactory- to create communication infrastructure for the producer role
It has been implements by the following classes providing the required interfaces: