Controlling Devices - The Good, The Bad and The Ugly

As you can see, we write a lot of device drivers.  To do this, we have to get intimately acquainted with the protocols that a device's manufacturer has implemented to provide control of their device.  In an ideal world, these protocols would be well designed, fit for purpose and fully documented.  We're not living in an ideal world, and depending on where you look you may well come to the conclusion that things are far from ideal - in fact, there are some truly horrible protocols and implementations out there.  This article describes some of the challenges we face when implementing a protocol handler for a driver.  It's also an offer to any manufacturer that perhaps doesn't have the expertise in house to define a well-designed control protocol; Ultamation can help.

What is a control protocol anyway?

In this context, We're talking about the language that a control system, such as a Crestron processor, speaks to communicate with another device; say, an amplifier or an intruder alarm panel.

Protocols come in all shapes and sizes (more on that later) and their job is to convey commands and feedback between two devices.  the design goals should encompass:

  • Be unambiguous - the protocol has to be clear and unambiguous.  You tell the device what you want, and it should understand you without any trouble.  Likewise, when the device responds, or generates unsolicited feedback, you should be able to infer what the data relates to without prior knowledge.  It's surprising how many protocols make this simple mistake.
  • Be efficient - if you want a device to perform well, you need to consider efficiency or message generation and parsing.  This is an important consideration for implementing a protocol handler as you many not be able to control how rapidly messages are thrown at you, and both ends of the conversation should handle being overwhelmed gracefully.
  • Consider the lowest common denominator - some developers have their favourite language and libraries.  If you work with web technologies, the path of least resistance might be an HTTPS based protocol with OAuth.  Easy for you, hard for someone trying to implement a client on a microcontroller in a language that doesn't have native support for secure sockets, or a convenient way to gather an authentication token from the account holder.  It's worth bearing in mind that your device may be controlled by very simple device. 

Transports

It's worth mentioning that protocols and transports are very different.  A transport could be considered to be the road that the protocol messages we send travel along.  The optimal case is that a protocol doesn't care how its messages go from A to B.  There are many great examples of manufacturers who implement a protocol that is identical whether it uses a serial (e.g. RS232, RS485, RS422) transport or uses an IP (e.g. TCP, UDP) transport.  There are equally as many examples where manufacturers introduce transport specific elements into their protocols.  Give yourselves a D-

Where multiple transports are supported, not only should it be unnecessary to confuse the transport and protocol, it allows a developer to maintain a common code base, and this improves software quality and efficiency.

The general approach that a manufacturer choses for their control protocol's transport can have a big impact on how the control implementation performs.  As I mentioned above, the choice of technology can make implementation simple or complex, but some technologies are just inherently poor choices for certain types of control.

As an example HTTP/RESTful APIs are very common.  This is likely because they are so accessible, have ubiquitous support in web applications which will likely be implemented within the device already and are generally easy to understand.  However, they are stateless and generally pretty slow.  Each request must be received, handled by the HTTP server, and passed off to the handler.  This may be fine if all you want to do is switch a device on or off, or change an input, but when it comes to tracking a volume slider in real time, they just don't cut it.

When I'm designing and implementing a control protocol I would always favour better performance at the cost of implementation effort.

If I change the input from the front panel, how do we communicate this back to the controller?  With HTTP your only solution is to poll - which is a poor solution for all sorts of reasons or use hacks like HTTP long polling - did I mention that is a hack to get around the deficiency in the HTTP request/response model?

Web Sockets might provide "best of both worlds" solution, but if you don't already have Web Socket client support, you have a lot of work to achieve a result that perhaps might been better served with a simple TCP socket.  IT requires a deeper understanding of TCP and sockets, but - to be fair - you are implementing a machine-machine interface, so that's probably worth getting a grasp of anyway.

The Protocol

There are so many ways to design a protocol, and the correct approach is really down to application.  That could mean designing a message structure that is human readable (i.e. ASCII) so that you can communicate with the device via a terminal emulator, or you might want more efficiency with a pure binary payload.  Either way, messages must be expressed in such a way that a client can easily identify where they start and stop.  Error detection may also be a requirement, though remember that TCP handles this for you, so anything else you add is pointless if you ONLY support TCP.  If you may also use serial or UDP transports, this is where some error handling will be needed. 

JSON is another popular payload.  It's human readable, it's pretty concise, fairly easy to parse and there are lots of tools to help you validate syntax etc.  It's also very easy to create some pretty freakish structures that can make deserialisation into objects an absolute nightmare.  Just because it validates, it doesn't mean it's well designed, and it's well worth adding in some additional specificity into the structure to make it unambiguous (and deserialisable!) at the expense of some extra characters - if you're really on an efficiency drive, then you're barking up the wrong tree with JSON in the first place.

Google's Protobuf is an example of a very efficient, flexible and robust message payload - the trade off is it's complicated to implement message reader/writers.

At the other end of the scale there are formats such as SOAP.  Fine if you're writing web discoverable business services.  Not really suitable for M2M communications.

Documentation

Once we have a protocol designed, implemented and tested, it's ready to be released into the world.  But not until it has been documented.

I would suggest that at the VERY least, a public protocol specification should cover the following:

  • Transport details - Serial comspec (9600-8N1, hardware pinout etc.) or TCP port for communication
  • The general packet structure for the payload along with any additional notes that apply to transport specifics (authentication, handshaking, heartbeats etc.)
  • Complete payload documentation describing all commands, values and valid ranges - this gets forgotten so often
    • If I'm setting volume, how am I to know if the protocol expects 0-100, -90-0, 1.0-50.0?
  • An example for each command, including the possible expected, and error, responses

 

 The reality is we may get a spreadsheet with an incomplete table of messages that do specific things, but don't explain the structure, how checksums are calculated, the allowed values (which may need to be given for specific models) and no mention of valid responses or even whether one should be expected!  Yes, it can be used to reverse engineer a protocol, but this shouldn't be part of the driver development process.

It goes without saying that documentation should also be accurate.  I recently had to request the serial comspec for a device as it wasn't mentioned in the documentation.  After waiting too long, I broke out the logic analyser, measured the time between a pulse at 8.6us and established that we were using 115200 baud.  Then I received an email from the manufacturer telling me to use 192000 baud.  Wrong on so many levels ;)

If you need help pulling your device protocols into shape so that your device is easy to integrate, performs well and is reliable, we'd be pleased to offer our help based on many years of experience.