Introducing the AEGIS Interoperability Scorecard
As passionate advocates for health IT interoperability, the AEGIS team is not shy about about calling for faster progress (what are we waiting for? and 2014 Health IT Week rant)–nor are we shy about celebrating success (OPA’s thought leadership) when we see it.
One difficulty with providing this kind of feedback, however, has been the absence of a succinct vocabulary.
What exactly defines operational interoperability, and how does one measure that in a way that is most relevant, independent and understandable?
We have been thinking hard about this question for a long time. Since 2009, AEGIS has been at the center of the evolving process of validating that health IT systems (and organizations using those systems) can safely exchange health information according to the rules defined by evolving specifications and standards.
Through the use of the AEGIS Developers Integration Lab (DIL), we have gathered thousands of real data points about what goes well (and not so well) as health IT systems attempt to share information, and there are key measurements that can be distilled into an interoperability scorecard.
Our intention with this scorecard is to empower users of health IT systems with an objective measurement of the continuously validated interoperability of that system.
Think “Interoperability Maturity Model” (IMM)
One could think of the scorecard as a measure of Interoperability Maturity. As tempting (and fitting) as this may be, instead of expressing scores in terms of “levels” as is done with other maturity models, we will start with a simple numerical value like shown above.
This illustration implies a simple calculation which applies equal weight to five basic interoperability functions: Security, Patient Discovery, Document Query, Document Retrieval and Document Content. This scoring evaluates abilities in these areas as both an initiator and responder. In reality, this scorecard calculation will most likely evolve to apply different weights to these dimensions.
Security, for example, is pretty critical, so it is likely to have the highest weight in the scorecard calculation. Document content (i.e., the accuracy and completeness of clinical data returned in the Retrieve Document response) is the whole point of interoperability, so it is likely to also have a higher weight than the remaining three components of the score.
Expect the Interoperability Scorecard algorithm to evolve as the Health IT industry continues to mature and as more buyers of these system demand to know the Interoperability Score before making purchasing decisions.
NOT yet another Certification
Note that we are not talking about “certification” here; knowing that a given release of a product meets the meaningful use (MU) requirements at a point in time is a fine use case for certification, but interoperability can change whenever any piece of the overall system changes.
Interoperability testing is NOT “one and done.”
We get that the health IT vendor community is sick of the seemingly endless cycle of new certification requirements. In fact, because of all the factors which can change interoperability, one-time certification means very little when it comes to the ongoing ability of health IT systems to interoperate.
What is meaningful is the ongoing ability to safely and accurately exchange health information, and the AEGIS Interoperability Scorecard quantifies that ability.
“As of” Date: Your Safety Freshness Metric
Note the “As of:” date in the scorecard. This tells you when this system
last proved it is interoperable. But what do I mean by “proved it is interoperable?” Great question.
I mean that the system–as currently configured and integrated with all other system components–has been tested to objectively demonstrate that it remains interoperable. This gets right to the heart of that “Interoperability testing is NOT ‘one and done.'” quote above.
Every time a system component changes, even if that change is not to the “interoperability” functionality directly, the interoperability behavior of he overall system can (and often does) change. As a result, interoperability needs to be revalidated by testing. We call this continuous interoperability–the state of continuing validation that a Health IT stack in a specific operating environment can safely exchange clinical information according to all the applicable specifications and standards.
Next Steps: Let’s start with Getting Real
We have thrown out many terms and envelope-stretching concepts here. It would be unrealistic to expect everyone to immediately grok everything in this post, so we offer the following starting points:
Health IT vendors: Let’s get real about what it means to be truly interoperable. Products aren’t interoperable; configured Health IT stacks interoperate, and whenever any component changes, the entire stack needs to be revalidated to prove ongoing interoperability. Embrace the Interoperability Scorecard as a competitive advantage.
Health IT buyers: Demand ongoing proof of interoperability via a dated Interoperability Score from your Health IT solutions providers. Know that you need continuous validation that your entire solution remains interoperability, and demand that this proof is part of your vendor contracts.