February 1, 2015 (Vol. 35, No. 3)

Sean Tunis, M.D. president & CEO Center for Medical Technology Policy

New Working Group to Evaluate Clinical Utility of NGS-Based Testing to Inform Coverage

After many years of steady development in DNA sequencing, the scientific community is reaching an exciting threshold in understanding the genetic riddles behind cancerous tumors. Newly developing next-generation sequencing (NGS) techniques can rapidly analyze large quantities of DNA, offering a greatly enhanced understanding of the molecular complexity of disease.

NGS promises to accelerate the understanding of cancer, helping to define tumors’ biological pathways and their genetic characteristics. It is already beginning to yield a more accurate picture of disease and making possible an expanding choice of targeted therapies. However, assessing the clinical utility of NGS-based testing, as well as the benefits and risks of off-label prescribing of targeted therapies, are complex and controversial issues.

A number of basic questions remain about the technical performance of NGS-based diagnostic tests. For example, it is unclear how best to compare results obtained across different NGS platforms or, indeed, with findings from older analytical techniques. While there are not yet uniform biologic sample standards, cross-validated bioinformatics algorithms across different analytic platforms, or consistent standards for the annotation of detected variants, these issues are being worked on by the FDA, NIST, CAP, and other relevant organizations. Assuming these issues can be resolved satisfactorily, there is the equally challenging issue of determining how best to generate evidence to demonstrate that use of these tests will improve health outcomes for patients.

Clinical and coverage decisions regarding the use of NGS-based tests for oncology and other diseases depend on the availability of reliable evidence that use of these tests will improve patient care. Different stakeholders have very different views about what sort of clinical studies are necessary to make good decisions.

Randomized trials are generally considered the most rigorous method for demonstrating the benefits and harms of health interventions. Because of the prohibitive expense and significant time commitment required to complete randomized trials, demonstrating clinical utility is often a challenge for diagnostic test developers. And, in some cases, small patient populations make such trials extremely difficult to complete.

In addition, the sheer volume of data coming from NGS-based analyses (a bounty of mutations which might be targets for anticancer therapy) complicates questions about the clinical circumstances under which new biomarker information can be confidently used by doctors, recommended by clinical guideline developers, and reimbursed by health plans.

So how do test developers analyze this wealth of data in order to give the payers, clinicians, and patients what they need to make informed decisions? In short, how can we really leverage all that NGS has to offer, and in doing so advance cancer diagnostic and treatment tools? Everyone in this industry—developers, payers, and policy makers—needs a clearly defined, reasonable, and broadly accepted framework for evaluating the evidence of the clinical utility of NGS-based diagnostics testing through studies that provide dependable results while also being feasible, timely, and affordable.


Dr. Sean Tunis of the Baltimore-based Center for Medical Technology Policy, which is dedicated to providing stakeholders with the evidence they need to make informed clinical decisions.

Evaluating Clinical Utility

Earlier this year, the Green Park Collaborative (GPC), an initiative of the Center for Medical Technology Policy (CMTP), launched a cooperative effort to develop methodological guidance on the design of studies needed to evaluate promising NGS tests for cancer diagnosis and treatment. The GPC is a multistakeholder forum established to clarify the evidence expectations of public and private payers, in order to inform coverage and payment decisions.

In GPC’s independent environment, representatives of the life sciences industry, payers, regulators, researchers, clinicians, and patients are working to develop standards of evidence for evaluating the clinical utility for NGS-based diagnostics testing. These standards seek to strike an acceptable balance between validity, feasibility, timelines, and cost, recognizing it is necessary to find a viable compromise that reflects the legitimate needs of multiple stakeholders with respect to the development and adoption of new genome-based diagnostics. This collaborative effort is comparable to the type of technical dialogue that occurs in the context of regulatory science, and which we refer to as “reimbursement science.”

To help inform this effort, GPC has invited experts across many disciplines to participate in discussions with the goal of drafting recommendations for the design of studies intended to evaluate the clinical utility of NGS-based testing in oncology. These experts include representatives of commercial health plans, CMS, NGS test developers, FDA, patient groups, research methods experts, oncology guideline organizations, clinical oncologists, drug/biologic companies, and other experts and stakeholders.

As with other work of the GPC, the recommendations are intended to assist test developers in designing studies that would support reimbursement by public and private payers. The inclusion of these payers in the dialogue will ensure that the recommendations accurately reflect the expectations of those decision makers.

These recommendations will be complimentary to the work being done by the Roundtable on Consensus Standards for Multiplex Cancer Genomic Testing, which is convened by the American Society of Clinical Oncology (ASCO), The Association for Molecular Pathology (AMP), and the College of American Pathologists (CAP).


The rapid development of NGS-based tests for use in oncology has highlighted this technology’s tremendous potential to improve clinical care and patient outcomes.

Evidence Generation

Specifically, the Working Group is now working to address several questions on evidence generation, including:

  • How should the utility of an NGS panel as a whole be determined? Does each gene on a panel need to be individually assessed in clinical trials? Or can other modes of data collection and assessment be used depending on the design and purpose of the panel?
  • Should standards of utility vary according to whether the primary purpose of the marker is predictive, prognostic, or diagnostic? (A variant might be predictive for one tumor site/type combination and prognostic for another.)
  • Should standards of utility vary according to medical need of the patient (e.g., does a different standard become correct when seriously ill patients have exhausted conventional therapies)? That is, the risk/benefit profile may vary with the patient’s current condition. Should “interim standards” be considered for situations where information is lacking but patient need is great?
  • Variants often occur on a spectrum from the most-common to rare or “N of 1” mutations. What threshold should be used to say that a variant is so rare that population-based studies are impractical or impossible to perform?
  • When treatment responses for rare cancers are recorded, how should this data be accounted for in larger bodies of evidence? (What should be the role of case reports?)

Because for payers the demonstration of utility is only germane to “actionable” tests, the workgroup will also explore the meaning of actionability of tests.

The rapid development of NGS-based tests for use in oncology has highlighted this technology’s tremendous potential to improve clinical care and patient outcomes. At the same time, it underscores the need for a clearly defined, consistent, and broadly accepted framework for evaluating the evidence of the clinical utility of tests in well-designed, feasible studies. We look forward to sharing our results with industry, policymakers, and payers.





























Sean Tunis, M.D. (sean.tunis@cmtpnet.org), is the president and CEO of the Center for Medical Technology Policy (CMTP), an independent nonprofit dedicated to developing a health care system where patients, clinicians, policymakers, and payers have the evidence they need to make informed health decisions.