Yolanda Gil (left) and Ewa Deelman
ISI’s Ewa Deelman, Yolanda Gil and six colleagues from other institutions have published an article in Science magazine’s policy forum.
Called “Enhancing reproducibility for computational methods,” the piece argues that “access to the computational steps taken to process data and generate findings is as important as access to data themselves.”
By enabling a researcher to independently reach the same scientific conclusions as another researcher, possibly with different data sets or methods, reproducibility supports scientific knowledge building and education. The Science article is a step toward making computational methods more reproducible.
Science is the flagship publication of the American Association for the Advancement of Science (AAAS). As the world’s oldest and largest general science organization, the AAAI says Science reaches 10 million people through members and affiliations with more than 250 scientific societies and academies.
The article stems a three-workshop series organized by the AAAS to address transparency and reproducibility in science. The first workshop resulted in publication of Transparency and Openness Promotion (TOP) Guidelines that have been signed onto by more than 600 journals.
The second workshop endorsed preservation of samples and data in field sciences. The third and final workshop included more than 30 invited experts in reproducibility, including Deelman and Gil, both research directors at ISI and research faculty in the University of Southern California’s Viterbi School of Engineering. Gil is also president-elect of the Association for the Advancement of Artificial Intelligence.
The article proposes a set of Reproducibility Enhancement Principles (REP) for scientific authors, publishers, and research funders such as US federal agencies. Principles emphasize sharing and citing not only data and software used in computational studies, but also the workflows involved.
Deelman and Gil have collaborated for several years on computational workflows, and led the development of the Pegasus and WINGS workflow systems respectively. Deelman’s Pegasus was used to manage complex data analytics from the LIGO experiment that led to the discovery of gravitational waves. Gil and her colleagues also have trained hundreds of scientists on reproducible research and open science best practices as part of the “Scientific Paper of the Future” initiative.
Research reproducibility has become a priority given that many studies report on failures to reproduce published results. In December 2016, the US Senate passed the American Innovation and Competitiveness Act, which directs the National Research Council to study reproducibility and design mechanisms for the National Science Foundation to improve transparency.
Earlier this year, the USC Academic Senate released “Rigor and Transparency in the Conduct of Research at USC“, which USC endorses and promotes through training workshops and other events.
Published on December 22nd, 2016
Last updated on July 13th, 2021
