Research published today in PLOS ONE indicates that the setting in which a scientific peer review panel evaluates grant applications does not necessarily impact the outcome of the review process. However, the study found that the average amount of discussion panelists engage in during the review is reduced.
The investigational team examined more than 1,600 grant application reviews coordinated by the American Institute of Biological Sciences Scientific Peer Advisory and Review Services (AIBS SPARS) on behalf of a federal agency over a four-year period. The researchers compared two years when grant applications were reviewed using an in-person peer review panel to two years when panels were conducted via teleconference or videoconference.
“There are no studies exploring whether the review setting significantly alters the quality of the peer review process,” said Stephen Gallo, M.D., the lead author of this study and technical operations manager for AIBS SPARS.
“Our goal is always a reliable and high-quality peer-review process. It is important that we understand the strengths and weaknesses of different peer review methods,” added Scott Glisson, director of AIBS SPARS and an author of this study.
The findings appear in “Teleconference Versus Face-To-Face Scientific Peer Review of Grant Applications: Effects on Review Outcomes” published in PLOS ONE.
“Little difference was found in most of the review metrics between face-to-face and teleconference settings,” said Dr. Gallo. Application scoring was only modestly affected, and reviewers used the full scoring range regardless of review setting. The reviewer ratings were highly reliable in both settings.
Often, the greatest anticipated difference between in-person and teleconference panels is the amount of time allocated to discussing applications. This study found teleconference or videoconference panels allocated less time to application discussions than in-person panels.
“We should know whether the reduced amount of discussion and peripheral interactions that occur in a teleconference setting influence the final panel outcomes, and, ultimately the productivity of the research that is funded,” noted Dr. Gallo.