September 15, 2016 (Vol. 36, No. 16)

Roberto Polakiewicz Ph.D. CSO Cell Signaling Technology

It’s All About Validation

The reproducibility of scientific studies has become a major issue, leading to a lack of trust in scientific results from the academic and pharmaceutical research communities.  While issues around reproducibility have been discussed for years, calls for action have been infrequent and half-hearted. Beginning in about 2012, a number of articles, letters, and editorials started appearing in Nature, Science and other publications, with some going so far as to call this a “reproducibility crisis.”1,2

The ongoing dialogue has included the role of improperly validated research reagents, such as antibodies, with blame falling at the feet of reagent vendors, researchers, and journals. This article will highlight how the lack of consistent research on antibody validation has contributed to the reproducibility crisis and the role of vendors from Cell Signaling Technology’s (CST) perspective in making research more robust and reproducible.

The number of antibody suppliers contributing commercial antibodies to the Research Use Only (RUO) space has steadily increased over the past several years. As a result, the marketplace has become populated with vendors that vary widely in validation practices and quality standards. The end result is that antibodies against the same target are available from  multiple companies, with inconsistent validation data to confirm sensitivity, specificity, reactivity, and lot-to-lot consistency.

For example, a published report evaluating PTEN expression in patient tissue samples by immunohistochemistry (IHC) found that despite being tested and recommended by their respective companies for this research application, only one of 11 antibodies could reliably predict positive and negative expressing tumors. Six of the 11 antibodies nonspecifically stained cell lines that were known to be negative for PTEN expression.3 This study is one demonstration illustrating the difficulty of finding properly characterized antibodies in the commercial marketplace.

The consequence of improperly characterized antibodies being published in the literature can be wide reaching. Take for example the predicted link between enterovirus infections and type 1 diabetes. Data from seroprevalence and epidemiological studies provide a correlative link to suggest enterovirus infection precipitates the onset of type 1 diabetes.4

However, the antibody-based studies used to establish a causal link have relied heavily on clone 5-D8/1, which was generated against an enteroviral capsid protein. This antibody has been used to show the presence of virus in infected tissues. These data support the hypothesis that enteroviral infection leads to the loss of the insulin-secreting β-cells within the pancreas either directly or by promoting a deleterious autoimmune response.5 Unfortunately, it is now widely known that this clone also recognizes two mitochondrial proteins, creatine kinase β-type and ATP synthase β-subunit.6

This finding has forced researchers to ask whether the antibody is revealing the presence of the virus within β-cells or if the positive signal seen in mitochondria-rich pancreatic tissues is simply an artifact.4,5,7 Estimates suggest that enteroviral infections affect up to a billion people worldwide every year.8,9 Without a clear causal link, the scientific community is limited in its ability to predict the course that would best serve the public good globally. In this case, having a more rigorously characterized antibody may have helped investigators come to a consensus without having to spend the effort reconciling a controversy.


The lack of consistent research on antibody validation has contributed to the scientific reproducibility crisis. [extender01 / Getty Images]

Antibody Supplier’s Responsibility

Antibody suppliers have a responsibility to the scientific community to do all they can to ensure the antibodies they release into the research marketplace are binding specifically to their intended target. This means establishing criteria by which antibodies should be validated for specificity, sensitivity, reactivity, and a lot-to-lot consistency. To ensure adoption, these standards must be decided upon and agreed to by vendors, researchers, and journals alike.

Improperly characterized primary antibodies may be just one factor contributing to the reproducibility crisis, but it is a factor that can be addressed. It will take all of us in the scientific community to achieve a solution.

For this reason the Global Biological Standards Institute (GBSI) has organized a meeting in Asilomar, California September 25–27. Attending will be thought leaders from academic and industry labs, commercial antibody vendors, and journal editors.  The objective to define the minimum acceptable validation requirements for RUO antibodies is the start of a long-term community effort to improve research reproducibility.  Together we can set achievable standards for antibody validation that vendors can embrace, and result in antibodies that are worthy of the research they support.

References
1. Begley CG, Ellis LM (2012) Drug development: Raise standards for preclinical cancer research. Nature 483(7391), 531–3.
2. Prinz F, Schlange T, Asadullah K (2011) Believe it or not: how much can we rely on published data on potential drug targets?  Nat Rev Drug Discov 10(9), 712.
3. Sangale Z, Prass C, Carlson A, Tikishvili E, Degrado J, Lanchbury J, Stone S (2011) A robust immunohistochemical assay for detecting PTEN expression in human tumors. Appl. Immunohistochem. Mol. Morphol. 19(2), 173–83.
4. Oberste MS, Pallansch MA (2003) Establishing evidence for enterovirus infection in chronic disease.  Ann. N. Y. Acad. Sci. 1005, 23–31.
5. Coppieters KT, von Herrath M (2013) Antibody cross-reactivity and the viral aetiology of type 1 diabetes.  J. Pathol. 230(1), 1–3.
6. Hansson SF, Korsgren S, Pontén F, Korsgren O (2013) Enteroviruses and the pathogenesis of type 1 diabetes revisited: cross-reactivity of enterovirus capsid protein (VP1) antibodies with human mitochondrial proteins.  J. Pathol. 229(5), 719–28.
7. Richardson SJ, Leete P, Dhayal S, Russell MA, Oikarinen M, Laiho JE, Svedin E, Lind K, Rosenling T, Chapman N, Bone AJ, Foulis AK, Frisk G, Flodstrom-Tullberg M, Hober D, Hyoty H, Morgan NG (2014) Evaluation of the fidelity of immunolabelling obtained with clone 5D8/1, a monoclonal antibody directed against the enteroviral capsid protein, VP1, in human pancreas.  Diabetologia 57(2), 392–401.
8. Morens, DM., Pallansch, MA., 1995 Epidemiology. In Human Enterovirus Infections, 3-23, ASM Press. Washington, D.C.
9. Pallansch, MA., Roos, RP., 2001. Enteroviruses: poliovirus, coxsackievirus, echovirus, and newer enteroviruses. In Field’s Virology, 723-75. Lippincott/Williams & Wilkins. Philadelphia/Baltimore.

 

Roberto Polakiewicz, Ph.D. (rpolakiewicz @cellsignal.com), is CSO at Cell Signaling
Technology
.

Previous articleDiscovery of Rapidly Adapting Antibody Opens Door for Universal Flu Vaccines
Next articleGenomic Signatures Clearly Dictate Schizophrenic Drug Responses