January 15, 2017 (Vol. 37, No. 2)

Carl-Johan Ivarsson President and Co-founder Qlucore

Five-Step Approach Can Ensure Repeatable and Significant Results

The GEN Special Section on Big Data consists of four articles:
Precision Medicine Research in the Million-Genome Era
Utilizing Machine-Learning Capabilities
NGS Big Data Issues for Biomanufacturing
Visualization for Advanced Big Data Analysis


A common challenge affecting many scientists, especially those working in the area of molecular biology, is the vast amount of data that is created by their experiments. With such a large volume of data to consider, advanced software tools are required to interpret their data effectively.

Until now, computer software designed for this purpose has focused on being able to handle increasingly vast amounts of data, and to a large extent applying standard statistical methods presented to the user in a technical specialist-oriented user interface.  As a result, the possibility for the scientist/researcher to approach and interpret data has partly been set aside, and a lot of data analysis can only be performed by specialist bioinformaticians and biostatisticians.  In most cases, however, this model has several drawbacks, since it is typically the scientist who knows the most about the specific area being studied.

Visualization in combination with well-selected algorithms and methods can overcome some of the described challenges and allow a broader range of users to explore and analyze data. The active use of visualization techniques provides a powerful way of identifying important structures and patterns very quickly. Visualization, which provides the user with feedback that is easy to understand, is also an important tool from an organizational point of view, since it stimulates innovation as a result of more scientists now being able to analyze and discuss data and results.


Qlucore Omics Explorer is used by a broad range of professions, from bioinformaticians, biologists, statisticians, and chemists to medical doctors, students, and professors. Four synchronized plots with various aspects of the variables are shown.

Five-Step Approach

We recommend a five-step method to ensure repeatable and significant results when using visualization to identify new subgroups and patterns in data. The purposes of this analysis phase can be several. The most common is to try to identify completely new groups or patterns in data. Another purpose can be to explore data to detect only expected patterns, which is a good form of quality control. 

By applying this five-step method, it is possible to investigate large and complex datasets without being an expert in  statistics.  The technique is described below in more detail, but some basics need to be in place at the start. This approach can be applied to any type of high-dimensional data and examples from the life-science industry include: RNA-Seq, gene expression arrays, proteomics, DNA methylation, and metabolomics.

 First of all, the high dimensional data needs to be reduced to lower dimensions so that it can be plotted in 3D. We recommend the use of Principal Component Analysis (PCA) for this purpose. Tools to color data to enhance the information are also required, as well as filters and tools to select and deselect parts of the dataset.

 At this stage, researchers can begin the five-step visualization process by detecting and removing the strongest signal present in the active dataset. Once this signal is identified, it can be removed in order to see whether there are any other obscured (but still detectable) signals present. Removing a strong signal will usually result in the reduction of both the number of active samples and/or variables (features).

Step two of the process is to assess the signal-to-noise ratio in the data by using PCA, projection score, and randomization. The projection score will indicate the strength of the visually detected signal or pattern.

Step three is to remove “noise” by variance filtering. If researchers can see a significant signal-to-noise ratio in their active dataset, they should try to remove some of the active variables that are most likely contributing to the noise. To identify the required amount of variance filtering, the user can use variance filtering supervised by PCA visualization and projection score. By testing many different variance settings, it is possible to more easily find clear patterns.

Step four offers the option of performing statistical tests that can be applied to any/all of the other stages of the five-step process: either during the initial analysis, when a step is repeated, at the end of a step, or not at all. The groups to test can be either predefined or a selection of those that were identified during the iterative process. (It is recommended to verify found structure and groups in a second dataset).

The final step uses graphs to refine the search for subgroups or clusters. Connecting samples in networks or graphs, for example, makes it possible to move into higher dimensions (i.e., more than the three that can be represented in a 3D PCA plot), since the graph created in a sample plot is based on the distances in the space of all active variables, and can therefore provide more insight into the structure of the data.

 These five steps are then repeated until there are no more structures to be found. When used in this way, visualization can be used as a powerful tool for researchers. Data can be visualized in a clear way, scientists can identify any interesting and/or significant results easily by themselves, without having to rely on specialist bioinformaticians and biostatisticians. Instead, the scientist can cooperate with the bioinformaticians to achieve even more interesting results.





























Carl-Johan Ivarsson (carl-johan.ivarsson@qlucore.com) is president and co-founder of Qlucore.

This site uses Akismet to reduce spam. Learn how your comment data is processed.