November 1, 2016 (Vol. 36, No. 19)
Bioassay Developers Are Using Statistics to Chase Down Stragglers and Showing Sensitivity to Mechanisms of Action to Keep Studies on Track
Bioassays are scientific experiments involving the use of cell cultures, tissue preparations, or living animals or plants to determine the biological activity of a substance, such as a hormone or drug. Due to the inherent variability of the testing platform, good bioassay design is crucial. While reference material that contains the same active analyte as the test sample is available, no general calibration standards exist as each batch of animals or living material may behave differently.
Bioassay design must contend with inherently complex experimental systems. Accordingly, it calls for scientific innovation at the biological level. But it also needs other kinds of innovation, such as statistical innovation.
More and more statisticians are working on bioassay projects and companies have begun to systematically include statisticians in their bioassay working groups to improve data analysis. Regulatory bodies concur that research modernization can reduce the cost of such analyses.
The statistician’s contribution to bioassay modernization often involves modular design. A modular design has elements that may be reused in multiple assays. There are practical, statistical, and performance advantages when these design elements are also statistical blocks.
Ideally, the block should be a smart collection of reference and test doses. Incorporating multiple test doses into the same block and utilizing the same reference data saves time. And if the modules contain the same amount of test samples that will be typically assayed when development is complete, the number of plates/blocks needed to get a good estimate of potency will be known.
Assay developers investigate factors such as the number of cells and the duration of incubation. If developers assign replicate blocks to each level of each factor, while holding other factors constant in a “one factor at a time” design, the number of blocks required will be very large and there will be no information about factor interactions. Alternatively, developers can study multiple parameters in combination by using a full factorial experiment, which examines all possible combinations of all factors and levels.
“A fractional factorial design can reduce the size of a factorial experiment,” says David Lansky, Ph.D., president, Precision Bioassay. “It is a powerful way to learn a lot with complex designs.
“For a simple example, imagine a cube and consider each axis of the cube a factor. If we use two diagonally opposite corners on the bottom of the cube and the other diagonals on the top of the cube, we have tested four conditions that provide a lot of information about the space of the cube. Depending on sample availability, this can be broken down into blocks of partial diagonal pairs one day, and others the next.”
When two products are similar, their dose-response curves have the same shape, with only a horizontal shift from one to the other. Parallelism or similarity tests are used to prove that similarity and to ensure that reported potency estimates have the desired meaning.
Both the U.S. Pharmacopeia and the European Pharmacopeia require that parallelism be shown to compute relative potency. Classic tests (Chi-squared, F-ratio) are known to have poor performance. Classical statistical tests of a null hypothesis that there is no slope difference use the absence of evidence that curves are not parallel as proof that they are parallel.
A practical consequence of this approach is the tendency to reject similarity in very precise assays, while accepting similarity in very imprecise ones. To address these risks, statisticians developed equivalence tests. While they are conceptually more accurate, they can require many experiments to build acceptance criteria, with potentially significant cost and time implications.
Precision Bioassay recommends an approach based on scaled measures of similarity. These scaled differences have substantially lower variance and bias than ratios because they use long-term averages in the denominator. If the bounds for these scaled differences are set appropriately, the median bias in potency due to nonsimilarity will be limited.
The tolerable amount of bias in potency depends on the intended use of the assay and the product. For products with a narrow therapeutic window (such as Coumadin), it is important that there be little bias in potency, while for a product with a wide therapeutic window (a vaccine, for example), much more bias in potency may be acceptable.
According to Perceval Sondag, senior statistician, nonclinical statistics, Arlenda, there is a temptation to keep analysis as simple as possible and not add what might be interpreted as overcomplication. However, innovation is not restricted to science; it can also include innovative statistical methods. Scientists should not be afraid to use advanced methods, particularly if they lead to better and more cost-effective solutions.
If only 8 to 15 experimental plates are used to derive acceptance criteria, results may not be accurate. Computation simulation can address this risk. Arlenda’s algorithm can simulate 10,000 or more experimental plates in just a matter of hours. And because already available data developed for the bioassay validation is used, there is no additional experimental cost.
Biosimilars, biobetters, and innovator drugs need simple, easy-to-use, robust, and precise bioassays based on the mechanism of action.
Existing bioassays often have long, complex, multistep protocols that can contribute to increased variability and reduce precision or accuracy. For example, the bioassay associated with Avastin, although well characterized, is a proliferation assay that was developed over a decade ago and is challenging to use. Biosimilar drug developers need novel ways to get mechanism of action–based bioassays for existing molecules.
DiscoverX works on understanding the signaling pathways associated with a specific receptor or drug target, then develops a bioassay based on the native biology of that receptor.
A subset of optimized bioassays is provided in which the innovator drug molecule has been tested and a method developed and qualified to get the appropriate response. Alternatively, a portfolio of cell-based assays for specific targets is available in which specific molecule testing has not yet been accomplished. In this scenario, cell lines are provided that respond to the specific target of the drugs.
Another improvement, the ability to use frozen cells that are created in large cell banks, is changing biologics development and long-term lot testing. Frozen cells eliminate the use for continuous culture, improve the efficiency of the lab, reduce time and cost, and increase reproducibility and flexibility.
“We enable faster development of robust reproducible bioassays and make assay development teams more efficient,” comments Abhi Saharia, Ph.D., director, cell-based assays and biologics, DiscoverX. “They can either use one of our existing, qualified bioassays or move directly into method development using a targeted cell line.”
The recently introduced KILR cell-based assays allow specific measurement of targeted cell death in a co-culture environment with immune effector cells. Applications from screening to lot release testing largely focus on immuno-oncology drugs. The bioassay enables the development or testing of a variety of different drugs, such as testing antibodies for ADCC (antibody-dependent cell-mediated cytotoxicity) or ADCP (antibody-dependent cellular phagocytosis), as well as bispecific antibodies for T-cell redirection and cellular therapies such as chimeric antigen receptor (CAR) T cells.
Alternative measures of cell death, such as technologies based on calcein or other dyes tend to be leaky. Those that measure general cell death cannot differentiate between the deaths of effector versus the target cells, and cell sorting is intensive, not scalable or transferrable.
Cancer immunotherapies cover a wide range of treatments. In vitro assays using immune and tumor cell lines or primary immune cells are used routinely to study one specific function of the therapy or products. Despite the promise of immunotherapy, more tools and assays are needed to understand the underlying mechanism of action and to reveal the unresponsiveness of certain types of tumors and patients to specific therapies.
ImmunXperts offers a suite of T-cell assays consisting of mixed lymphocyte reaction assays, antigen-specific T-cell activation assays, such as cytomegalovirus reactivation assays and polyclonal T-cell activation assays, for the functional screening of immune-checkpoint inhibitors. These assays involve the major players of the immune system, such as dendritic cells, natural killer cells, regulatory T cells, and responding T cells, thereby mimicking the human immune system in a tissue culture plate. Depending on the target and mechanism of action, more focus can be given to certain subpopulations.
Analysis by flow cytometry allows multiplexing. In addition, a large number of donors and donor sets are available to assess the functionality in a test population representing varying types of responders, as well as aliquots of prelabeled, cryopreserved responder and stimulator cells.
Macrophages also play an important role during cancer development by promoting the immunosuppressive environment. An in vitro macrophage assay to assess the effect of test molecules on the polarization state toward M1 (antitumor) macrophages or M2 (pro-tumor) macrophages is under development.
“The question remains of how results can be translated to the actual in vivo situations,” insists Sofie Pattijn, chief technology officer, ImmunXperts. “The large variability in anti-tumor and patient responses makes it very difficult to represent in a single in vitro test.”
Despite ongoing efforts in three-dimensional culture models, and platforms such as impedance-based detection of cell death and proliferation, not all properties can be assessed in vitro. Drug bioavailability, distribution, and absorption rate can vary depending on the route of administration.
Complex assays using primary cells are prone to variation, and most laboratories use in-house optimized protocols. For the vast majority of the assays, no standards are available and selected immune-modulating molecules are included for benchmarking purposes.
Plus, no universal methods of data analysis and reporting are applied. The Cancer Immunotherapy Consortium of the Cancer Research Institute (CRI-CIC) and the Association for Cancer Immunotherapy (CIMT) have undertaken efforts to harmonize and standardize the clinical subspecialty of immuno-oncology to improve reproducibility and contribute to a better in vitro/in vivo correlation.