O. Joseph Trask Jr. Head, Cellular Imaging Core The Hamner Institutes for Health Sciences
Paul A. Johnston School of Pharmacy, University of Pittsburgh
The Importance of Calibration Standards and Reproducibility for High Content Screening
There is an old folk saying popularized by the late Bert Lance that reads “if it ain't broke, don't fix it.” The application of this philosophy to scientific endeavors creates major problems if scientists fail to appreciate when the instruments used to collect and analyze data are neither properly operated nor calibrated. Such behaviors undermine the widely held expectation that scientists do everything possible to ensure the quality and reproducibility of their data. Recently, the Society of Biomolecular Imaging and Informatics (SBI2) emerged to become engaged in a technology discipline that has grown exponentially over the past 20 years without adopting either standards or guidelines. The procedures to validate the integrity and performance of automated imaging platforms and the image analysis methods used to produce data for publication or drive business decisions are not routinely available to the end user. Rather, the majority of laboratories using automated imaging platforms for high content screening (HCS) typically employ reference control compounds (activators and/or inhibitors) to demonstrate the performance of their assays. Although this practice may currently be considered acceptable by most programs and facilities in academia, biopharmaceutical discovery, and research institutions in nonclinical settings, there are some major underlying assumptions that may not always be valid: the instrument has been calibrated correctly and is being operated optimally to acquire images; the image analysis algorithm has been appropriately optimized and applied in an unbiased manner; and that the data are suitably annotated, stored, and retrievable following archive. The truth is, “any image is data, good or bad,” and therefore it is critical that the imager be set up to acquire the best images possible and they be stored, analyzed, and annotated appropriately.
The advantage of implementing standards in HCS is to provide a mechanism for the scientific community to directly compare data generated across multiple HCS platforms. These standards not only provide a means to calibrate instrumentation, assays, and results, but also can be applied to cross-reference data generated in one laboratory against other laboratories to verify the reproducibility of HCS data independently. Adoption of a standardized annotation and nomenclature to describe data analysis methods and outputs will facilitate understanding and comparisons within the scientific community. Standardized biological data is not a new concept, and the urgent need and support for standards is becoming even more evident in research as scientists are challenged with a plethora of data and the inability to reproduce published work.1,2 Leonard Freedman from the Global Biological Standards and colleagues recently published a commentary in PLoS Biology about the cost of irreproducibility in preclinical sciences.3 The fact that scientific research is not always reproducible is not surprising, and to a certain extent this idea has become accepted by many that have worked in the field for years. What makes the commentary of Freedman et al. of greater importance is the mind-boggling estimate of $28 billion in economic losses resulting from only 50% of scientific work being reproducible. With the global shrinkage in sponsored research funds from both private and government agencies, the scientific community needs to minimize the waste in spending and improve the reproducibility of published research to address the urgent need to bring new therapies to market.
To read the resto of this article click here.
ASSAY & Drug Development Technologies, published by Mary Ann Liebert, Inc., offers a unique combination of original research and reports on the techniques and tools being used in cutting-edge drug development. GEN presents here one article "Standardization of High Content Imaging and Informatics." Authors of the paper are O. Joseph Trask Jr., and Paul A. Johnston.
