Researchers are solving the issue of chronic underreporting of medical device adverse events. In the New England Journal of Medicine last month, they described their use of DELTA (Data Extraction and Longitudinal Analysis), a statistics software that prospectively monitors safety events stored in registries and databases. Moving forward from the current voluntary reporting method by physicians and patients, this FDA-backed software automates the correlation of adverse events with associated medical devices. In their proof-of-concept example comparing 146,288 patients, the software showed that Mynx (Cardinal Health), a vascular closure device, had a 50% increased relative risk of a vascular complications than other vascular closure devices (1.2% versus 0.8% adverse event rates, respectively).
“The primary goal was to explore/validate this approach, which is a prospective, computer-based surveillance of clinical repository data using automated data monitoring tools to look for differences in the performances of existing and approved medical devices,” said Dr. Frederic Resnic, lead author of the study and chair of the department of cardiovascular medicine at Lahey Hospital & Medical Center (Burlington, MA, U.S.). He likens it to a safety gadget used in most North American homes: “the best way to think about it is like a smoke detector in a kitchen, which is constantly monitoring the environment.” This automatic data monitoring can be much more effective than what’s currently in place.
In the U.S., the bulk of safety assurance revolves around manual reporting by hospitals and physicians. Such reports are sparse: they are voluntary and rare safety events are often difficult to correlate on a case-by-case basis. According to Dr. Resnic, this is problematic because 1) more than 99.8% of attributable events go unreported, and 2) the few reports that are filed are mostly inadequate and overwhelming for the FDA’s monitoring programs. “The FDA is wading through this avalanche of incomplete information to try to find the needle in the haystack – the evidence that there might be a problem with the device.”
With DELTA, assessing medical device safety could become much more efficient. DELTA triggered a safety warning nine months into this study, and sustained the warning until study completion. Here, vascular closure devices simply served as a proof-of-concept example to show that DELTA could provide a method to evaluate all medical devices for their safety. “[My] knowledge of prior observational data from large registries suggested differences in the performance of medical devices. We expected there to be a risk increase of 50% of vascular complications [of the Mynx]. But I want to point out that this isn’t unique to this device. Mynx was a convenient example that I knew something about, but there are hundreds of device approvals every year that are based on very small datasets.”
As with anything used for medical purposes, all medical devices go through rigorous testing before they hit the market for patients. However, they’re tested in small cohorts, relative to the large numbers of patients using it after approval. Regulatory agencies like the Food and Drug Administration (FDA) in the U.S. or Health Canada in Canada review the results, and if the performance is appropriate, the device is approved and sold. While this is an efficient way to evaluate the performance of a medical device for approval, the small datasets fail to capture complete performance profiles of the devices. “[Mynx] was approved in 2007 on the basis of 95 patients of a single-arm study done in centers in Europe,” said Dr. Resnic. “That is effectively the totality of clinical evidence that went into the approval of the device. And then, we simply don’t look after that, and we wait to hear if there is a problem or not.” Post-market surveillance sheds light on the safety profile in a much larger dataset, detecting more events and allowing stronger comparisons to other devices. Here, Dr. Resnic and his colleagues were able to find a minor difference in performance, that otherwise would not have been detected, because of the automated data analysis.
For Dr. Resnic, the current system of surveillance must be improved. “Right now, in many ways we’re like ostriches, who put their heads in the sand when they’re afraid or confused. This is the absence of exploring the real-world data that’s already been collected. In my perspective, this is a failure of safety culture for healthcare in general. We have to think about using the data we’ve already collected to ensure that every patient is receiving the best available treatment. We want to make sure that of what gets released into the market, we have accountability to ensure that they are performing the way we expect them to perform. Patients assume that we are doing this. We owe it to them and to ourselves to be more rigorous to ensure medical devices are safe.”
The DELTA software is available to public health organizations as a free, open source, and modular toolset.
Paper in NEJM: Registry-Based Prospective, Active Surveillance of Medical-Device Safety…
Lahey Hospital and Medical Center news release: First-of-its-Kind Study Focuses on Medical Device Safety…
