An internal study comprising 30 different image-analysis projects carried out by different laboratories around the world using a range of different image-analysis software indicates that, on average, 75% of the true positives are missed due to errors made in spot detection and matching. Needless to say, this represents a huge untapped potential in 2-D gel-based proteomics
The Combined Correctness Principle
To date, there have been no metrics for image-analysis quality that can reliably ensure a low ratio of both false positives and false negatives among the results. The only solution for researchers is to painstakingly go through every single spot and edit the image-analysis results where necessary. Inevitably, this will introduce unwanted bias in the analysis, not to mention the toll it takes on time and nerves.
The need for such metrics is clear, seeing that both false positives and false negatives stem from errors made during spot detection and/or spot matching. It seems obvious that the key lies in reliably increasing the correctness in these steps.
In order to increase the correctness of both spot detection and spot matching, what we call combined correctness, a standard of measurement is needed. Only when combined correctness can be reliably quantified can it also be optimized.
Taking this line of thought further and assuming that there truly is a significant correlation between combined correctness and the rate of false positives and false negatives, the introduction of such clearly defined metrics will provide us with the necessary tool to measure and hence increase the true discovery potential in 2-D gel-image analysis.
Measuring Combined Correctness
The first step in measuring combined correctness is to define distinct categories that describe what constitutes correct or incorrect spot detection or matching. These categories need to be independent of the different approaches used for image analysis today. By applying these categories to the image-analysis data in a statistically relevant way, it is possible to measure and calculate the combined correctness for 2-D gel-image analysis.
For spot detection, all spots can be categorized into one of the four following classes as outlined in Figure 3: (A) correct, (B) false, (C) misshaped, and (D) missing. Using these four classes, it is possible to determine the overall spot-detection correctness in any 2-D gel-image analysis.
In a similar manner, each pair-match (matching of one spot in one gel to a spot in another gel) can be categorized into either of four classes: (A) correct matching, (B) incorrect matching, (C) correct nonmatching, and (D) incorrect nonmatching. Applying this will provide us with a measurement for the overall matching correctness.