0
We're unable to sign you in at this time. Please try again in a few minutes.
Retry
We were able to sign you in, but your subscription(s) could not be found. Please try again in a few minutes.
Retry
There may be a problem with your account. Please contact the AMA Service Center to resolve this issue.
Contact the AMA Service Center:
Telephone: 1 (800) 262-2350 or 1 (312) 670-7827  *   Email: subscriptions@jamanetwork.com
Error Message ......
Research Letters |

Inclusion of Comparative Effectiveness Data in High-Risk Cardiovascular Device Studies at the Time of Premarket Approval FREE

Connie E. Chen, MD; Sanket S. Dhruva, MD; Rita F. Redberg, MD, MSc
[+] Author Affiliations

Author Affiliations: Department of Medicine, Stanford Hospital and Clinics, Palo Alto, California (Dr Chen); Division of Cardiovascular Medicine, University of California, Davis (Dr Dhruva); and Division of Cardiology, University of California, San Francisco (Dr Redberg; redberg@medicine.ucsf.edu).


Letters Section Editor: Jody W. Zylke, MD, Senior Editor.

More Author Information
JAMA. 2012;308(17):1740-1742. doi:10.1001/jama.2012.14491.
Text Size: A A A
Published online

To the Editor: Use of a comparator group is not required in studies used for approval of medical devices, in contrast to drugs, which require 2 randomized controlled trials. The US Food and Drug Administration (FDA) defines valid scientific evidence as “partially controlled studies, studies and objective trials without matched controls, well-documented case histories conducted by qualified experts, and reports of significant human experience.”1 A study of new drugs in the United States found that 51% were not supported by comparative effectiveness data at the time of FDA approval.2 Unlike drugs, device trials frequently use control data from previous trials (historical controls) and performance benchmarks imputed from prior studies (objective performance criteria; OPC) in evaluating outcomes. However, the lack of active controls raises concerns about selection bias and validity.

High-risk cardiovascular devices requiring premarket approval are defined as those that support or sustain life and include many invasive or implantable devices. Minor variations in manufacture do not require approval. In a review of such devices approved from 2000-2007, 52% of primary end points had controls, 31% of which were retrospective.3 In this study, we examined the type of controls used by the FDA in evaluating comparative effectiveness during premarket approval of cardiovascular devices.

Data were extracted from publicly available FDA summaries of safety and effectiveness data (herein called summaries).4 Summaries representing all high-risk cardiovascular devices approved by the FDA between January 1, 2000, and December 31, 2011, were analyzed. The selection and characteristics of premarket applications have been previously described.2

All primary end points were examined to determine the type of controls with which the device was compared (Box).3 We defined comparative effectiveness research as use of active controls in evaluating primary end points.1 We described control group characteristics by approval year and device class and examined whether number needed to treat was calculated. The Kruskal-Wallis test was used to examine differences between device classes. Statistical significance was determined at P < .05 (2-sided). Statistical analyses were performed using Stata version 10 (StataCorp).

Box. Coding of Studies and Primary End Points

  • Active control

    The control group comprised individuals recruited for current study and received a validated intervention that was not the device being studied (ie, comparing a drug-eluting stent with a bare metal stent). Use of a historical control group was excluded.

    For first-in-class devices, the active control group could be standard surgical intervention (carotid endarterectomy vs carotid stenting) or maximum medical management (catheter ablation vs flecainide for recurrent paroxysmal atrial fibrillation).

    Few trials relied on placebo groups such as sham intravascular brachytherapy or cardiac resynchronization device with biventricular pacing functionality turned off. Active control trials could be blinded or unblinded.

  • Historical control

    The data for the control group was generated from previously conducted studies.

    The significance of the end point of interest had to be evaluated against the historical data using a case-control or other statistically matched method.

  • Objective performance criteria or performance goal

    A study that relied on a numerical benchmark derived from previously published studies of comparable devices or from expert opinion (such as previous Food and Drug Administration consensus statements).

  • No control

    No control group was recruited for the current study and also used none of the performance goals described above.

    As an example, 2 single-group studies of a new porcine aortic valve provided data on postoperative outcomes (heart failure and hemodynamic evaluation) without comparison with other available valves. Twenty-three valve-related deaths were noted among the 337 treatment patients without comparison with a control.

The 121 summaries identified contained 203 supporting clinical studies. These studies used 353 primary end points, of which 40% (n = 140) were evaluated against an active control, 13% (n = 45) against a historical control, 26% (n = 90) against an OPC, and 22% (n = 78) against no control. Thirty-six percent (74/203) of studies included at least 1 primary end point evaluated against an active control; 48% (58/121) of devices had an active control in at least 1 supporting study, while 35% (42/121) of devices were approved without data from any control or OPC.

The use of active controls varied by device category (Table). The number of devices approved each year was too small to allow statistical analysis, but there did not appear to be any trends over time. Of the 90 end points defined using OPC, 28% (n = 25) relied on performance goals established in previous FDA guidance documents, 26% (n = 23) on previous trial data, and 19% (n = 17) on reviews of previously published literature. Twenty-one percent (25/121) of studies did not provide an OPC source.

Table Graphic Jump LocationTable. Use of Comparative Effectiveness Criteria by Application Year and Device Category

Only 3 end points were identified in which an active control may not have been possible. No summary commented on the number needed to treat for any primary end point.

Most high-risk cardiovascular devices were approved without comparative effectiveness data. A minority of end points, studies, and summaries since 2000 included an active comparator group. Use of active controls varied significantly by device class.

We categorized the control group used in studies conducted for premarket approval of high-risk cardiovascular devices, but we cannot comment on internal validity or rigor of the studies included or on other key aspects of the trial designs.

It is hard to know if a new device is safe and more effective than alternative treatments unless it is compared with conventional treatment. While occasionally use of active controls may not be possible, such as ventricular assist devices, more frequent use of a comparator group may help to better inform clinical and regulatory decisions.

Author Contributions: Dr Redberg had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Chen, Dhruva, Redberg.

Acquisition of data: Chen, Dhruva, Redberg.

Analysis and interpretation of data: Chen, Dhruva, Redberg.

Drafting of the manuscript: Chen, Dhruva.

Critical revision of the manuscript for important intellectual content: Chen, Dhruva, Redberg.

Statistical analysis: Chen.

Study supervision: Redberg.

Conflict of Interest Disclosures: The authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Redberg reported being a member of the US Food and Drug Administration Circulatory System Devices Panel and a member of the California Technology Assessment Forum. Dr Redberg is the editor of the Archives of Internal Medicine. No other conflicts of interest were reported.

Additional Contributions: We thank Jean Slutsky, PA, MSPH (Center for Outcomes and Evidence at Agency for Healthcare Research and Quality), and Vanessa Jacoby, MD, MAS (University of California, San Francisco, Department of Obstetrics, Gynecology, and Reproductive Sciences), for their comments and contributions. Neither received compensation for their contributions.

Food and Drug Administration.  Determination of safety and effectiveness (§860.7): valid scientific evidence. http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/HowtoMarketYourDevice/PremarketSubmissions/PremarketApprovalPMA/ucm050419.htm. Accessed July 10, 2012
Goldberg NH, Schneeweiss S,  et al.  Availability of comparative effectiveness data at the time of drug approval in the United States.  JAMA. 2011;305(17):1786-1789
Link to Article
Dhruva SS, Bero LA, Redberg RF. The strength of study evidence examined by the FDA in pre-market approval of cardiovascular devices.  JAMA. 2009;302(24):2679-2685
Link to Article
Food and Drug Administration.  FDA summary of safety and effectiveness data (SSED) clinical section checklist. http://www.fda.gov/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDRH/CDRHTransparency/ucm220413.htm. Accessed September 17, 2012

Figures

Tables

Table Graphic Jump LocationTable. Use of Comparative Effectiveness Criteria by Application Year and Device Category

References

Food and Drug Administration.  Determination of safety and effectiveness (§860.7): valid scientific evidence. http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/HowtoMarketYourDevice/PremarketSubmissions/PremarketApprovalPMA/ucm050419.htm. Accessed July 10, 2012
Goldberg NH, Schneeweiss S,  et al.  Availability of comparative effectiveness data at the time of drug approval in the United States.  JAMA. 2011;305(17):1786-1789
Link to Article
Dhruva SS, Bero LA, Redberg RF. The strength of study evidence examined by the FDA in pre-market approval of cardiovascular devices.  JAMA. 2009;302(24):2679-2685
Link to Article
Food and Drug Administration.  FDA summary of safety and effectiveness data (SSED) clinical section checklist. http://www.fda.gov/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDRH/CDRHTransparency/ucm220413.htm. Accessed September 17, 2012
CME
Also Meets CME requirements for:
Browse CME for all U.S. States
Accreditation Information
The American Medical Association is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians. The AMA designates this journal-based CME activity for a maximum of 1 AMA PRA Category 1 CreditTM per course. Physicians should claim only the credit commensurate with the extent of their participation in the activity. Physicians who complete the CME course and score at least 80% correct on the quiz are eligible for AMA PRA Category 1 CreditTM.
Note: You must get at least of the answers correct to pass this quiz.
Your answers have been saved for later.
You have not filled in all the answers to complete this quiz
The following questions were not answered:
Sorry, you have unsuccessfully completed this CME quiz with a score of
The following questions were not answered correctly:
Commitment to Change (optional):
Indicate what change(s) you will implement in your practice, if any, based on this CME course.
Your quiz results:
The filled radio buttons indicate your responses. The preferred responses are highlighted
For CME Course: A Proposed Model for Initial Assessment and Management of Acute Heart Failure Syndromes
Indicate what changes(s) you will implement in your practice, if any, based on this CME course.

Multimedia

Some tools below are only available to our subscribers or users with an online account.

Web of Science® Times Cited: 8

Related Content

Customize your page view by dragging & repositioning the boxes below.

Articles Related By Topic
Related Collections
PubMed Articles