If you would like a better understanding of MQTT, or are having a tough time wrapping your head around what is Sparkplug and how does it relate to MQTT, this post is for you!
The Message Queuing Telemetry Transport Protocol, known as MQTT, has become extremely popular for IoT and IIoT applications, and for good reason. This simple, lightweight, and open transport protocol enables devices and clients to publish information to a central MQTT broker. Other devices, clients, and additional brokers can in turn, subscribe to the messages from the MQTT broker as well as publish messages of their own back to the broker.
Arlen Nipper and Andy Stanford-Clark co-invented MQTT while trying to solve a specific problem for Phillips 66 in the late 1990's. So instead of following the same model that had been used for decades, the team decided to think outside of the current poll-response industrial automation box; MQTT was the result of that effort.
That problem, still relevant today was...
"How do we improve data access in the field for our entire enterprise without killing bandwidth?"
Purposefully, the majority of implementation information was left out of the original MQTT specification. This decision provided for maximum flexibility and is the reason that Facebook, Amazon Web Services, and Microsoft Azure were early adopters and have found the protocol so appealing.
However, to meet the interoperability needs of the industrial automation community, MQTT needed to have implementation standards defined. This has been achieved with the creation of the Sparkplug specification for MQTT. Sparkplug allows for several important IIoT enabling benefits, as described by Arlen Nipper in a previous interview (December 2019) with Inductive Automation.
"Four years ago, a specification was developed called Sparkplug. Sparkplug is a specification that defines how to use MQTT in a mission-critical, real-time OT environment. It defines a standard MQTT Namespace that's optimized for industrial application use cases. It defines an efficient MQTT Payload definition that's extensible but that's been optimized for SCADA, tag and metric representations. It defines how best to use the MQTT state management in real-time SCADA implementations. And lastly, it defines some high-availability MQTT architectures addressing both redundancy and scale."
But why do we really need a specification for MQTT? Wouldn't a specification reduce the flexibility of what MQTT offers? I spoke through these questions with Walker Reynolds of Intellic Integration, a friend as well as a superb systems integrator. Walker brought some great insights to the discussion.
He summarized, "Yes MQTT is flexible – you can basically publish any payload to any topic but this can create a mess in the topic namespace. Additionally, building IoT applications using many MQTT clients publishing into the same broker, without any uniformity in the payload structure takes additional time. Sparkplug B, specifically, is the specification written by Arlen Nipper and team for publishing industrial data and provides the structure, compression and uniformity needed to normalize disparate data into a unified structure at the broker."
And Walker is right (BTW, you should watch some of his videos). Sparkplug provides the structure for anyone that wants to create an IoT or IIoT application to use MQTT without having to worry about interoperability between their project and other applications. Furthermore, Arlen and his team were wise enough to realize that the Sparkplug specification needed to be a collaborative effort within the industrial automation community. That is why they chose to move ownership of the specification to the Eclipse Foundation and form the MQTT Sparkplug Working Group, a collection of end users, integrators, and vendors who work together to drive the evolution and adoption of the Sparkplug protocol with the end goal of enabling interoperable and scalable IIoT solutions.
Beyond these benefits, using MQTT Sparkplug provides more advantages. Let's look at a few of the main reasons you might want to consider an MQTT Sparkplug architecture.
Less spaghetti.
Consider a traditional architecture and the number of connections that need balanced between devices, servers, and client tools.
The MQTT Sparkplug architecture becomes drastically simpler. Just point your publishing devices to a single broker and define a single topic or multiple topics the device should publish under. Then, your subscribing clients can use those same topics to filter which data they want to receive from the broker.
Uses considerably less bandwidth.
Since it is not poll-response, MQTT does not communicate unless a change occurs, drastically reducing bandwidth requirements. Additionally, the Sparkplug specification is able to achieve 3x higher compression on the packets it transports.
Johnathan Hottell, a Senior SCADA Manager with Edge Controls, published his findings comparing MQTT Sparkplug bandwidth usage to other industry protocols like OPC UA and Modbus.
In that study, Hottell demonstrated that MQTT Sparkplug can actually provide from 75% to 99.5% more bandwidth efficiency than using Modbus in real-world applications! That's extreme but true.
As you can see from the chart, which outlines the 'Total Cost of Ownership' of a days worth of data transferred from an actual oilfield application, the savings over another pub/sub protocol like OPC UA are also incredible.
Less bandwidth means higher resolution.
An additional benefit is the increase in data granularity. Since you are only communicating changes, you can theoretically increase your scan class without increasing your bandwidth usage. For instance, if a discrete tag typically changes 6 times a day, using poll-response you would have to balance how often you scan the tag vs the amount of bandwidth you are prepared to sacrifice. If you chose to poll the tag every 15 minutes, you would communicate data 96 times per day to catch 6 actual value changes, with the potential of having up to a 14:59 delay from the time the value changes to the time you are aware. Using MQTT, you would become immediately aware of the tag change AND only send the 6 value changes per day.
State awareness.
MQTT Sparkplug brokers are aware of the state of their clients and can communicate that state to other clients. This works using birth and death certificates and the MQTT Last Will and Testament (LWT) feature. When a publishing client connects to a broker, it issues a birth certificate along with it's LWT payload to the broker. Any MQTT clients that choose to subscribe to the same topic as the publishing client, will be made aware by the broker whenever the publishing client is online or offline. Additionally, if the publishing client is offline, the broker will provide the subscribing client with the LWT payload. This ensures that stale data does not get delivered to subscribing clients and is accomplished using a pre-configured 'keep alive timer' or 'heart beat check' between broker and publishing clients that is admin configurable. In conclusion, MQTT Sparkplug state awareness allows subscribing clients to always understand whether the publishing client is online, and if not, be made aware of when the client went offline, and what was the last known value.
For a great overview of MQTT, as well as the birth and death certificate, and the LWT feature, watch this great video from Opto 22 (and if you need some epic hardware, definitely check them out, they play heavily in the MQTT space).
Security, security, security!
Network security is inherited because MQTT is based on TCP/IP. This means that the MQTT specification moves at the same pace as TCP/IP. Additionally, this also drastically assists in keeping the specification simple!
Think you would like to try MQTT on one of your future projects, or want to learn more about it? Let us know and we can introduce you to multiple hardware and software companies that have made MQTT Sparkplug a standard offering in their suite of tools!
