When a flaw is discovered in an FDA-regulated product that can impact that product’s ability to perform effectively and safely, they are required by the FDA to issue a “recall”. Recalls are often newsworthy – we hear of recalls for drugs (when they have newly discovered side effects or are mislabeled) and even every day products like cat food (for contamination). Recalls for medical devices are often less newsworthy, but can be incredibly expensive.
The Recall Process
When a device manufacturer learns that there is a problem with one of their products, FDA regulations require that the vendor issues a recall. As defined in 21 CFR 7, the FDA states:
“Recall is an effective method of removing or correcting consumer products that are in violation of laws administered by the Food and Drug Administration. Recall is a voluntary action that takes place because manufacturers and distributors carry out their responsibility to protect the public health and well-being from products that present a risk of injury or gross deception or are otherwise defective.“
When a vendor discovers a flaw in a product, they are obligated to either issue a correction (i.e., a patch or update) or a removal (taking the device off of the market). All recalls must be reported to the FDA and announced publicly. For most medical device manufacturers, recalls are costly–a 2017 McKinsey report estimates that a single recall of a medical device can cost up to $600 million dollars. And even issuing small recalls can cost a vendor many millions of dollars.
The challenge comes when cybersecurity vulnerabilities are reported that potentially require a recall. Because a vulnerability can easily affect large parts of a vendor’s product line (e.g. if a large number of products share an operating system or a particular system component), a large medical device manufacturer could be required to issue recalls for a significant portion of their product lines on a regular basis. This leads some manufacturers to take a more constrained approach to vulnerability reporting, performing an update only on devices or product lines that have been the subject of explicit and targeted reports.
The FDA anticipated these issues in the 2016 FDA Postmarket Guidance, stating that vendors can supply security patches without issuing a recall when three conditions are met:
- There are no known serious adverse events or deaths associated with the vulnerability;
- “….no later than 30 days after learning of the vulnerability, the manufacturer communicates with its customers and user community regarding the vulnerability, identifies interim compensating controls, and develops a remediation plan to bring the residual risk to an acceptable level.”
- “…no later than 60 days after learning of the vulnerability, the manufacturer fixes the vulnerability, validates the change, and distributes the deployable fix to its customers and user community such that the residual risk is brought down to an acceptable level. “
This may seem like a reasonable set of conditions. But imagine the situation that a vulnerability is reported in a product that was built from 2007-2010 and has been in the market since that time. As anyone who has worked in a software company can attest, finding developers, testers and product managers who know how a particular product that was developed in a prior decade works (and then spinning up infrastructure to fix the system, test the system and issue a patch) often takes months. And if a vulnerability exists across a significant portion of their products, meeting these 30 and 60 day timelines can be nearly impossible.
This leaves many security patches necessitating a recall, and the medical device manufacturers bearing the cost of issuing them.
For these reasons around cost, many medical device manufacturers take an extremely limited view of vulnerability reports, issuing fixes only to the systems that they have been explicitly notified (by customers or security researchers) about, rather than taking a broad view and trying to find vulnerabilities across the majority of their product lines. In fact, if vulnerabilities are found in reviews of premarket devices that may share components or code with devices that are currently on the market, those vulnerabilities are often not confirmed to be on, nor are patches issued for, in-market devices. (This is why events like the Biohacking Village at Defcon usually only include premarket devices. Rarely would a vendor submit themselves voluntarily to the risk of having to do recalls)
These dynamics present significant headwinds for a hospital’s ability to identify and protect itself against the vulnerabilities in their clinical environments. Clinical technologies’ operating systems are almost always nearing end-of-life, and perverse incentives keep vendors from issuing security patches or reporting vulnerabilities in their products–due to regulatory complexity and because they are incentivized to avoid proactively seeking them out. This means that many vulnerabilities can exist in the devices within their network that they do not know about and that there are no patches for. And, because the vendors have not confirmed that vulnerabilities exist on those products, there are no CVEs nor other indicators to security products that they should be looking for or protecting those devices from exploits of those vulnerabilities.
These “Dark” vulnerabilities that exist but cannot be patched or monitored for can leave the users of these devices (whether small hospitals or large health systems) largely defenseless. They only know about a small number of vulnerabilities in the devices in their environment. And, even when they do know about the vulnerabilities, they have no way of detecting when they are exploited.