Leonard Freedman, Ph.D. President Global Biological Standards Institute
GBSI is taking action to reduce the widespread and systemic irreproducibility of research findings.
Disease research is a critical element in the fight against debilitating and deadly conditions. On the research bench, scientists are designing and executing preclinical research studies that can form the foundation of the next breakthrough drug.
During the past several years, however, concerns about the quality of research have garnered the attention of multiple stakeholders in the scientific community1,2,3,4. Specifically, the life science community has begun taking a documented look at something that was once considered a natural, acceptable, and relatively unavoidable part of the research ecosystem: widespread and systemic irreproducibility of research findings.
In the United States alone, almost $115 billion is estimated to be spent annually on life science research, about half of which is spent on preclinical work. Of that preclinical research, a conservatively estimated 50% is not reproducible. In consequence, it’s not a stretch to state that tens of millions of dollars of research funds have been spent on research that, with the application of some simple yet effective standards, could have been reproducible— and therefore more effective.
Irreproducibility is most often the result of unintentional cumulative errors or flaws in the research process, including reference materials, study design, laboratory protocols, and data analysis and reporting. Consider cell lines, which have been used for decades in cancer and other areas of biomedical research, including vaccine development, reconstructive medicine, and cell biology research. Cell lines, however, have their issues and can be the source or contributor to a lack of reproducibility in life science research due to widespread use of misidentified or contaminated cell lines. This has been reported to range from a conservative low of 15% to a high of 36%. In fact, misidentification of cell lines has been an historic and pervasive issue, resulting in failed clinical trials and substantial delays in therapeutic drug development.
The prevalence of contaminated cancer cell line usage is well-documented—despite the availability of the short-tandem repeat (STR) analysis as an accepted standard to authenticate cell lines, and its relatively low cost (approximately $200 per assay). One well-known case is that of Adriamycin-resistant breast adenocarcinoma cell lines, or MCF-7/AdrR, where more than 300 studies used this misidentified cell line before they were found to be derived from human ovarian carcinoma cells. Another compelling case is the misidentified cell line MDA-MB-435. More than 1,000 articles have been published on this cell line as a triple-negative, metastatic breast cancer cell line that grows well in culture. It was later reported to have an identical STR profile to that of the melanoma cell line M14. Since then, a controversy has raged as to the provenance of either cell line, and all resulting data and results generated using these lines are not reliable. Even after its true identity was revealed in 2004, researchers continued to include the MDA-MB-435 cell line model in grant applications for breast cancer research.
Recently, the Global Biological Standards Institute (GBSI) held a forum on the issue of cell authentication in the backdrop of the irreproducibility crisis. Keynote speaker Keith Yamamoto, Ph.D., vice chancellor for Research, University of California, San Francisco, executive vice dean, UCSF School of Medicine, offered this insight at the meeting: “The current standard for cell authentication, STR analysis, is unknown to many investigators, ignored by others. Embracing this standard, or refinements that may follow, will require a global cultural shift by all stakeholders—researchers, publishers, funders, and regulators—but failure to do so puts our endeavor at risk.”
Fortunately, this issue is increasingly recognized and many leaders in the life sciences field are taking action. Scientific journals are starting to request that authors show proof of cell-line authentication prior to publishing. The National Institutes of Health recently released a set of proposed principles and guidelines for reporting preclinical research following a joint workshop held in June between the NIH, Nature Publishing Group, and Science. They were endorsed by a large number of journals and societies. And the Prostate Cancer Foundation now requires all funding applicants to authenticate their cell lines at the start of research and at intervals during the research process.
Howard Soule, Ph.D., chief science officer, Prostate Cancer Foundation and senior fellow, Milken Institute, explained this rationale, “The life sciences community must embrace the use of standards to improve the credibility, reproducibility, and translatability of preclinical research using cancer cell lines. At the Foundation, we are ensuring enhanced scientific fidelity by requiring that all funded researchers use the authenticated cells that they are given at the beginning of research, and we annually send samples to an independent lab for STR and PCR analysis.”
Further, the GBSI convened a cell-authentication standard task force and held a BioPolicy Summit to address this issue with key leaders from government, industry, academia, and journal publishing to address irreproducibility and improve research fidelity and overall reproducibility. From this, some recommendations for best practices and key standards in life science research emerged regarding cell authentication, including:
- Change the Culture—Practices of the research community should embrace the importance of cell authentication.
- Use Standards and Best Practices—Documented authentic, contaminant-free cell lines should be used in research.
- Establish Dedicated Funding—Research grants should include the costs to address cell authentication.
- Authenticate to Publish—Journals should require documentation of cell authentication.
- Commit to Train—Graduate students and postdoctoral fellows should receive greater education on the importance of cell authentication in research.
- Invest in New Technologies—There should be greater investment in the development of novel tools for cell authentication.
While the practice of cell authentication is relatively easily implemented, the culture shift necessary to embrace cell authentication may take time. These are not easy discussions, but great minds engaging in this dialogue can catalyze change and transform the way we conduct research. A vibrant conversation has been spawned leading to important actions that will support the entire life science community in building a more powerful research process—resulting in increased reproducibility, faster breakthroughs, and drug discoveries on behalf of the patients who ultimately benefit from the amazing science that is happening around the world.
1 Ioannidis JP. PLoS Med. 2005 August; 2(8):e124.
2 Mobley A. PLoS One. 2013 May; 8(5):e63221.
3 Westphal S. Unverified science. Boston Globe. 2013 July 8.
4 Jasny, B. et. al., Science 2011 December 2011; 334 (6060):1225.
Dr. Freedman is the founding President of GBSI. He has over 30 years of research, management, and program development experience in molecular and cell biology, biomedical research, and drug discovery in both the private sector and academia. He is a recognized leader in the field of nuclear hormone receptors. (firstname.lastname@example.org)