VA health care portal: Still down

On March 1, the U.S. Department of Veterans Affairs shut down a portal between its health care facilities and a Defense Department health information system. The problem: The portal (called the Bidirectional Health Information Exchange, or BHIE) was delivering patient information that was sometimes about the wrong patient.

When the system tells you that a female patient has been given a prescription for Levitra, you know something's wrong.

In my Computerworld column this week about the problem, I wrote about the key role a user played in spotting the problem and avoiding catastrophe. What I didn't have space for in print is exactly how the problem happened in the first place.

According to the VA, what it came down to was old code.

Really old code.

How old? That section of code was written in a programming language that's no longer used or supported at the VA. (No, they wouldn't say which language it was.) The VA's IT department had to track down the original programmers to make the fix.

There's also another angle on the age of the code: It wasn't originally implemented to handle the amount of traffic it eventually got.

See, the glitch was caused by multiple instances of (what was supposed to be) a unique identifier. Sounds like sloppy programming, right? Except that code was originally designed for a single node handling relatively low levels of traffic. It worked fine at that level.

The problem didn't show up (at least not regularly) when the software was cloned on multiple nodes. It was only when there were large numbers of nodes, and traffic ramped up in a big way, that the unique identifier sometimes failed to be unique.

That's the trouble with reusing old code: Just because it works for its intended purpose doesn't mean it will keep working when you scale up its use by orders of magnitude. You've gotta keep refactoring -- and testing.

In the end, that unique-identifier bug turned out to be a fairly easy fix. But in the course of testing, a different bug showed up: Sometimes the portal returned medical records that were correct but incomplete.

That problem is turning out to be a lot harder to eliminate. According to the online newsletter NextGov, the VA's CIO says the BHIE portal will likely be down until at least mid-April for testing, and VA doctors will continue to get information from the DOD's medical records system by phone, fax and paper until it's working reliably.