In our last post we unpacked Interoperability, including Open Standards. Continuing this theme, we will look at how solution developers implement standards compliance and the problems that arise.
Mandating that vendors (and internal systems) comply with Open Standards is a strategy used by organisations to drive interoperability. The assumption is that Open Standards compliant components will be interoperable.
In this post we examine the many reasons why that assumption does not always hold in real-world situations. This analysis will be from the Software perspective, since generally network equipment does a better job of component interoperability than software. This post will also cover general aspects of standards compliance in this post, and the specific aspects of API’s in the next post.
Whether the standard is “de jure” or “de facto”, there are three basic approaches to implementing software compliance with the standards:
This approach consists of two parts:
A “reference implementation” is a software component that is warranted to comply with the standard and is a known reference against which to validate a developed component. This should also include a set of standard test cases that verify compliance and / or highlight issues.
Vendors often provide the test results as evidence and characterisation of the level of compliance.
Benefits of this approach
This is the highest level of compliance possible against a standard. Two components that have been validated against the standard will be interoperable at the lowest common level to which they have both passed the test.
Problems with this approach
A reference implementation must exist and be available, however this is not always the case. The reference implementation must be independently developed and certified, often by the standards body themselves.
This approach is similar to the Reference Implementation approach. Firstly, the documented standard is a controlling design input. However the second part (validation against the standard) is both optional and highly variable. At the most basic level, compliance could be just that the vendor asserts component compliance with the standard. Alternatively, compliance may be validated by comparison between the developed component and the documentation, and there are many ways to do this at varying levels of accuracy and cost.
Benefits of this approach
The main benefits of this approach is that the design is driven to the standard, and at this level it is equivalent to the Reference implementation approach.
Problems with this approach
Validation without a reference implementation is highly manual and potentially subject to interpretation. This type of validation is very expensive which creates cost pressure for vendors to only partially validate, especially on repeat version upgrades and enhancements.
In this case the standard is used as one input to the design, but not as the controlling input. The intent is not compliance but alignment. The product may use the same or similar underlying technologies as defined in the standards (e.g. REST interfaces or the same underlying data representation standards such as XML or JSON. The vendor may adopt a similar component architecture (e.g. microservices) to the standard.
Benefits of this approach
At best, this approach may provide a foundation for future compliance.
Problems with this approach
In general, the vendor is designing their product to be “not incompatible” with the standard, without taking on the cost of full compliance.
Standards compliance is expensive to implement, regardless of the approach taken. So each vendor will take on their own approach, based on their own situations and context. A vendor may:
There are a wide range of implementations and the results are highly variable. The important thing to remember is that a claim of “standards compliance” can mean many things.
From a starting point of the intent to comply (or at least claim compliance), and using any of the strategies above, a vendor can be non-compliant in many ways:
Nothing can be assumed about standards compliance, other than that each vendor’s claims must be validated. The other part of this issue is Application Program Interfaces (API) interoperability. We will cover this in the next post. Stay tuned.