Controlling Devices - The Good, The Bad and The Ugly

As you can see, we write a lot of device drivers. To do this, we have to get intimately acquainted with the protocols that a device's manufacturer has implemented to provide control of their device. In an ideal world, these protocols would be well designed, fit for purpose and fully documented. We're not living in an ideal world, and depending on where you look you may well come to the conclusion that things are far from ideal - in fact, there are some truly horrible protocols and implementations out there. This article describes some of the challenges we face when implementing a protocol handler for a driver. It's also an offer to any manufacturer that perhaps doesn't have the expertise in house to define a well-designed control protocol; Ultamation can help.

What is a control protocol anyway?

In this context, We're talking about the language that a control system, such as a Crestron processor, speaks to communicate with another device; say, an amplifier or an intruder alarm panel.

Protocols come in all shapes and sizes (more on that later) and their job is to convey commands and feedback between two devices. the design goals should encompass:

Be unambiguous - the protocol has to be clear and unambiguous. You tell the device what you want, and it should understand you without any trouble. Likewise, when the device responds, or generates unsolicited feedback, you should be able to infer what the data relates to without prior knowledge. It's surprising how many protocols make this simple mistake.
Be efficient - if you want a device to perform well, you need to consider efficiency or message generation and parsing. This is an important consideration for implementing a protocol handler as you many not be able to control how rapidly messages are thrown at you, and both ends of the conversation should handle being overwhelmed gracefully.
Consider the lowest common denominator - some developers have their favourite language and libraries. If you work with web technologies, the path of least resistance might be an HTTPS based protocol with OAuth. Easy for you, hard for someone trying to implement a client on a microcontroller in a language that doesn't have native support for secure sockets, or a convenient way to gather an authentication token from the account holder. It's worth bearing in mind that your device may be controlled by very simple device.

Transports

It's worth mentioning that protocols and transports are very different. A transport could be considered to be the road that the protocol messages we send travel along. The optimal case is that a protocol doesn't care how its messages go from A to B. There are many great examples of manufacturers who implement a protocol that is identical whether it uses a serial (e.g. RS232, RS485, RS422) transport or uses an IP (e.g. TCP, UDP) transport. There are equally as many examples where manufacturers introduce transport specific elements into their protocols. Give yourselves a D-

Where multiple transports are supported, not only should it be unnecessary to confuse the transport and protocol, it allows a developer to maintain a common code base, and this improves software quality and efficiency.

The general approach that a manufacturer choses for their control protocol's transport can have a big impact on how the control implementation performs. As I mentioned above, the choice of technology can make implementation simple or complex, but some technologies are just inherently poor choices for certain types of control.

As an example HTTP/RESTful APIs are very common. This is likely because they are so accessible, have ubiquitous support in web applications which will likely be implemented within the device already and are generally easy to understand. However, they are stateless and generally pretty slow. Each request must be received, handled by the HTTP server, and passed off to the handler. This may be fine if all you want to do is switch a device on or off, or change an input, but when it comes to tracking a volume slider in real time, they just don't cut it.

When I'm designing and implementing a control protocol I would always favour better performance at the cost of implementation effort.

If I change the input from the front panel, how do we communicate this back to the controller? With HTTP your only solution is to poll - which is a poor solution for all sorts of reasons or use hacks like HTTP long polling - did I mention that is a hack to get around the deficiency in the HTTP request/response model?

Web Sockets might provide "best of both worlds" solution, but if you don't already have Web Socket client support, you have a lot of work to achieve a result that perhaps might been better served with a simple TCP socket. IT requires a deeper understanding of TCP and sockets, but - to be fair - you are implementing a machine-machine interface, so that's probably worth getting a grasp of anyway.

The Protocol

There are so many ways to design a protocol, and the correct approach is really down to application. That could mean designing a message structure that is human readable (i.e. ASCII) so that you can communicate with the device via a terminal emulator, or you might want more efficiency with a pure binary payload. Either way, messages must be expressed in such a way that a client can easily identify where they start and stop. Error detection may also be a requirement, though remember that TCP handles this for you, so anything else you add is pointless if you ONLY support TCP. If you may also use serial or UDP transports, this is where some error handling will be needed.

JSON is another popular payload. It's human readable, it's pretty concise, fairly easy to parse and there are lots of tools to help you validate syntax etc. It's also very easy to create some pretty freakish structures that can make deserialisation into objects an absolute nightmare. Just because it validates, it doesn't mean it's well designed, and it's well worth adding in some additional specificity into the structure to make it unambiguous (and deserialisable!) at the expense of some extra characters - if you're really on an efficiency drive, then you're barking up the wrong tree with JSON in the first place.

Google's Protobuf is an example of a very efficient, flexible and robust message payload - the trade off is it's complicated to implement message reader/writers.

At the other end of the scale there are formats such as SOAP. Fine if you're writing web discoverable business services. Not really suitable for M2M communications.

Documentation

Once we have a protocol designed, implemented and tested, it's ready to be released into the world. But not until it has been documented.

I would suggest that at the VERY least, a public protocol specification should cover the following:

Transport details - Serial comspec (9600-8N1, hardware pinout etc.) or TCP port for communication
The general packet structure for the payload along with any additional notes that apply to transport specifics (authentication, handshaking, heartbeats etc.)
Complete payload documentation describing all commands, values and valid ranges - this gets forgotten so often
- If I'm setting volume, how am I to know if the protocol expects 0-100, -90-0, 1.0-50.0?
An example for each command, including the possible expected, and error, responses

The reality is we may get a spreadsheet with an incomplete table of messages that do specific things, but don't explain the structure, how checksums are calculated, the allowed values (which may need to be given for specific models) and no mention of valid responses or even whether one should be expected! Yes, it can be used to reverse engineer a protocol, but this shouldn't be part of the driver development process.

It goes without saying that documentation should also be accurate. I recently had to request the serial comspec for a device as it wasn't mentioned in the documentation. After waiting too long, I broke out the logic analyser, measured the time between a pulse at 8.6us and established that we were using 115200 baud. Then I received an email from the manufacturer telling me to use 192000 baud. Wrong on so many levels ;)

If you need help pulling your device protocols into shape so that your device is easy to integrate, performs well and is reliable, we'd be pleased to offer our help based on many years of experience.

Getting a Crestron Home Driver into Development

We regularly get contact from integrators and manufacturers about writing drivers; usually an email that looks something like this:

"We need a Crestron Home driver for the "ACME Super-Deluxe Awesomeness". It's used by everybody in my area and it'd be amazing if we had support for this in Crestron Home. Please let me know if you can do it and how much it will cost."

I've written the same reply to so many of these emails that it seems like a good idea to give the nuts and bolt of the answer here. So - here we go...

The bottom line - there is no short answer here, as whether one can write an integration driver for any home automation system is partly a technical question, partly commercial, and sometimes even touches on policy for the platform. The questions that need to be answered (in no particular order) are:

Does the device have an API (Application Programming Interface)?
Fundamentally - if the "ACME Super Deluxe Awesomeness" doesn't have an integration API - i.e. there is no programmatic way of controlling the device, then you can probably guess where that leaves us. It's true that some, very simple, devices don't need an API - perhaps they're just controlled by a simple relay - but that leads to a different question (later) and this perhaps a very small edge case, in terms of what we're talking about here.
Is that API publicly available?
Many devices have APIs, but they're not all published, and sometimes they are actively kept hidden - such as in the case of Apple's MediaRemote protocol. Using the Apple example is intentional, as we've shown that this isn't necessarily a barrier to getting a working integration module, but the point here is - it makes the task considerably harder. This isn't related to the functions of the device either. Something like a power distribution unit may have very simple functionality - switching an outlet on or off - but to make that happen may require complex authentication or involve encryption to avoid someone switching things on and off when they shouldn't. Having the API/protocol documented is a key consideration here.
Is the API any good?
Not all control protocols are created equal. For every well designed, well implemented and well document protocol, there are many more that are badly designed, poorly implemented and the documentation is vague or just plain wrong. The better the API, the easier the task of implementing it for Crestron Home.
What's the development effort?
This is a function of different aspects but, primarily, how complicated the API is to implement and how much functionality is required. APIs requiring OAuth or encryption will be more challenging than one that uses a simple ASCII CR/LF delimitered protocol. A device that requires only requires turning on and off will be much simpler that a device that has complex audio DSP control. We can only estimate development effort once we have visibility of the API and an unambiguous specification of what is required - but this will generally be measured in man-days of effort.

These points have all been technical in nature, but that isn't the only consideration when we schedule work for a new driver.
Is development equipment available?
It's been done before, but writing an integration module against documentation only is a gamble. Inevitably, the first time you fire up a new driver you immediately find a fundamental issue with the communications. Having equipment to test against is essential. That doesn't necessarily mean you need a complete system - being able to test against the communications board of a display may be sufficient (as long as it can function without the glass). Some manufacturers are happy to provide equipment which makes things much easier. Some manufacturers aren't quite as accommodating - those manufacturers don't get a driver.
There is a middle ground where people offer to "be the tester". As helpful as this is, it's far less efficient than having the device in the development environment. During a day in development, we may iterate through 20 or 30 changes and being able to debug directly, watch the low level communications and identify defects in real-time is the only way to go. When it comes to beta-testing, then we welcome integrators who can really put a device driver through its paces.
Who's covering the cost?
One way or another, we need to cover the financial cost of the development effort (and testing, any peripheral equipment, rent, power, etc.) and this is where we can take two different approaches.
Our preferred approach is that we develop a driver, bear the cost of the development, and then recoup that investment through incremental sales over time. For this approach to be viable, we need to be reasonably confident that the finished driver will sell in sufficient quantity to - at least - break even, and if the product is particularly niche, or the development effort is likely to be very high with only limited sales, then the driver is probably not a good candidate for our work pipeline.
The other common option is for a manufacturer to bear the cost of development. They naturally have a vested interest in having their device supported in Crestron Home so that would open sales opportunities that would otherwise be closed. This then means that the module is free to use to integrators.
The least common option is for an integrator to cover the full cost so that they can have some sort of exclusivity over a particular device to create a unique proposition for their clients. The risk being that someone else could take a similar view and undermine that exclusivity.
Does it "fit"?
This is a particular consideration for Crestron Home. Crestron Home uses the "Crestron Certified Driver" framework for device drivers which you could look at as a driver being block of a certain shape that will only fit into the hole of the corresponding shape in Crestron Home. So - displays have one shape, cable boxes have a different shape and some things simply don't have a shape at all. To make things more confusing, Crestron Home does support some device types, such as lighting, HVAC or media streamers that aren't available for driver developers and have to be "baked into" the Crestron Home product. Lutron and Sonos are examples of this.
When we're looking at developing a driver, we can only do this for driver types that are supported, or try to be creative and fit an unsupported type into one of the supported types - for example - our NVR drivers are masquerading to look like cable boxes from Crestron Home's point of view.
Crestron Home is expanding the list of driver types all the time, so this constraint is gradually diminishing, and we also have a special device type called "Extension Devices" which opens up the opportunities for this that can't yet be accommodated by one of the standard types, with the caveat that Crestron Home doesn't really integrate these devices other than providing simple commands in sequences and via the UI.
Timing?
Sometimes the timing just isn't right. You may need a driver for a project right now and we may be able to develop a driver quickly enough for that need; but if Crestron have already got a driver in their development pipeline and, say, it gets released shortly after, the market for the driver vanishes over night. There's certainly something to be said for being first, but this is another angle that needs consideration.

If the answers to these items are generally positive, or if you'd like to discuss the possibility of having a driver developed for Crestron Home, then we would love to hear from you. Just contact This email address is being protected from spambots. You need JavaScript enabled to view it.

About Us

As a multi–award winning company specialising in high-end Crestron home automation we are used to writing modules for jobs we are working on. With one of the, globally, few Gold Crestron Master Programmers at the head of our team, we excel in the more challenging modules and have written drivers for equipment that had previously been considered too complex to integrate.

We now enjoy a reputation for high quality, tried and tested modules which we have made available to other Crestron programmers and integrators, both saving them time in completeing jobs and being able to offer sophisticated integration, that otherwise might have been unachievable for their clients.

Ultamation is our vehicle to deliver these products to you!

Along with requests from manufacturers and integrators to develop modules for their products, we’re constantly looking to provide support for innovative solutions – from car turntables to steam rooms.

We also develop software solutions for the Crestron programming community, such as the hugely popular “Theme Creator” for UI designers and the “SIMPLified” programmer’s assistant.

Mocking Crestron

Introduction

Far from what you might think from the title, this article is not about poking fun at the world’s leading automation system. Far from it: Crestron is the only platform that has successfully made the transition from proprietary development environments and niche, specialised programming, to mainstream software development practices. This article covers one aspect of that transition by describing one approach to “unit testing” – the practice of repeatedly testing small elements of a larger system to ensure they satisfy their design parameters – using “mock” objects. This gives confidence that your code, when employed in a larger system, will yield the expected behaviour.