RR links

From Reproducible Research

These are some links to related work about reproducible research, reproducible research papers, etc.

Contents 1 Other people and labs doing RR efforts 2 Journals with RR initiatives 3 Articles about RR (chronologically) 3.1 Talks and other 'informal' write-ups about reproducible research 4 Tools 4.1 Open Source 4.2 Commercial 5 Blogs 5.1 Reproducible Research 5.2 Related Topics

Other people and labs doing RR efforts

• Reproducible electronic documents: Jon Claerbout and his colleagues at the Stanford Exploration Project initiated (to our knowledge) the discussions about reproducible research.

• Wavelab: David Donoho and his colleagues at the Stanford Statistics Department developed Matlab code to reproduce their results on wavelets.

• Reproducible Neurophysiological Data Analysis: a page by Christophe Pouzat on reproducible research in neurophysiology using R and Sweave.

• Sensorscope: the wireless environmental sensing network developed at EPFL. Detailed descriptions of the sensor platform are available for those interested to reproduce the setup. Documented datasets are also available for people interested to reuse the data.

• Xin Li's source code collection for reproducible research, with links to code for various image processing algorithms.

• Al Hero's lab applies reproducible research for their publications.

• Andrew Davison at CNRS works on facilitating reproducible simulations using Python.

• MD Anderson Cancer Center: Bioinformatics hosts supplementary material for a number of their publications including code and data.

• StatReport, a description of reproducible statistical reporting at the Department of Biostatistics, Vanderbilt University.

• Jon Wellner's page with links about reproducible research.

• VisionBib.Com contains a very large bibliography of computer vision papers, as well as listings of vision-related code and datasets.

Journals with RR initiatives

• Annals of Internal Medicine: When a paper is accepted, the authors are asked explicitly whether their paper is reproducible. If yes, links are provided to the study protocol, data, and/or statistical code.

• IEEE Transactions on Signal Processing: in the acceptance e-mail from the editor-in-chief, the authors are encouraged to make their code and data available online.

• The Insight Journal: An online, open access journal in medical imaging that requires code as an integral part of the publication. They also allow for online post-publication reviews.

Articles about RR (chronologically)

• D. E. Knuth, Literate Programming, The Computer Journal, vol. 27, no. 2, pp. 97–111, May 1984.

• K. Price, Anything You Can Do, I Can Do Better (No You Can’t)..., Computer Vision, Graphics, and Image Processing, Vol. 36, pp. 387-391, 1986, doi:10.1016/0734-189X(86)90083-6.

• J. Claerbout, Electronic documents give reproducible research a new meaning, in Proc. 62nd Ann. Int. Meeting of the Soc. of Exploration Geophysics, 1992, pp. 601–604.

• J. B. Buckheit and D. L. Donoho, WaveLab and Reproducible Research, Dept. of Statistics, Stanford University, Tech. Rep. 474, 1995.

• R. Koenker, Reproducible Econometric Research, Department of Econometrics, University of Illinois, Urbana-Champaign, IL, Tech. Rep., 1996.

• M. Schwab, M. Karrenbach, and J. Claerbout, Making scientific computations reproducible, Computing in Science & Engineering, vol. 2, no. 6, pp. 61–67, Nov. 2000.

• H. D. Vinod, Care and feeding of reproducible econometrics, Journal of Econometrics, vol. 100, no. 1, pp. 87–88, Jan. 2001.

• Jan de Leeuw, Reproducible Research. The Bottom Line, Department of Statistics, UCLA, Department of Statistics Papers, Paper 2001031101, March 2001.

• F. Leisch, Sweave: Dynamic generation of statistical reports using literate data analysis, in Compstat 2002 — Proceedings in Computational Statistics, W. Härdle and B. Rönz, Eds. Physica Verlag, Heidelberg, 2002, pp. 575–580, ISBN 3-7908-1517-9.

• A. J. Rossini and F. Leisch, Literate statistical practice, UW Biostatistics Working Paper Series 194, University of Washington, WA, USA, 2003.

• J. Quirk, Computational Science: "Same Old Silence, Same Old Mistakes" - "Something More Is Needed", Adaptive Mesh Refinement - Theory and Applications, Proceedings of the Chicago Workshop on Adaptive Mesh Refinement Methods, pp. 3-28, Sept. 2003, doi:10.1007/3-540-27039-6_1.

• R. Gentleman and D. Temple Lang, Statistical Analyses and Reproducible Research, Bioconductor Project Working Papers, Working Paper 2, May 2004.

• S. Pakin, Reproducible Network Benchmarks with coNCePTuaL, Proc. Euro-Par '04, pp. 64–71, Lecture Notes in Computer Science Vol. 3149, Sept. 2004.

• M. Barni and F. Perez-Gonzalez, Pushing Science into Signal Processing, IEEE Signal Processing Magazine, vol. 22, no. 4, pp. 119–120, July 2005.

• R. D. Peng, F. Dominici, and S. L. Zeger, Reproducible Epidemiologic Research, American Journal of Epidemiology, 2006.

• H. A. Piwowar, R. S. Day, and D. B. Fridsma, Sharing detailed research data is associated with increased citation rate, PLoS ONE, vol. 2, no. 3, p. e308, March 2007.

• M. Barni, F. Perez-Gonzalez, P. Comesaña, and G. Bartoli, Putting reproducible signal processing into practice: A case study in watermarking, in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 4, April 2007, pp. 1261–1264.

• S. Fomel and G. Hennenfent, Reproducible computational experiments using scons, in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 4, pp. 1257–1260, April 2007.

• J. Kovacevic, How to encourage and publish reproducible research, in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 4, April 2007, pp. 1273–1276.

• P. Marziliano, Reproducible research: A case study of sampling signals with finite rate of innovation, in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 4, April 2007, pp. 1265–1268.

• J. Vandewalle, J. Suykens, B. De Moor, and A. Lendasse, State of the art and evolutions in public data sets and competitions for system identification, time series prediction and pattern recognition, in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 4, April 2007, pp. 1269–1272.

• P. Vandewalle, G. Barrenetxea, I. Jovanovic, A. Ridolfi, and M. Vetterli, Experiences with reproducible research in various facets of signal processing research, in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 4, April 2007, pp. 1253–1256.

• D. L. Donoho, A. Maleki, I. U. Rahman, M. Shahram and V. Stodden, Reproducible Research in Computational Harmonic Analysis, Computing in Science and Engineering, vol. 11, no. 1, pp. 8-18, Jan./Feb. 2009, doi:10.1109/MCSE.2009.15.

• R. J. LeVeque, Python Tools for Reproducible Research on Hyperbolic Problems, Computing in Science and Engineering, vol. 11, no. 1, pp. 19-27, Jan./Feb. 2009, doi:10.1109/MCSE.2009.13

• R. D. Peng and S. P. Eckel, Distributed Reproducible Research Using Cached Computations, Computing in Science and Engineering, vol. 11, no. 1, pp. 28-34, Jan./Feb. 2009, doi:10.1109/MCSE.2009.6

• V. Stodden, The Legal Framework for Reproducible Scientific Research: Licensing and Copyright, Computing in Science and Engineering, vol. 11, no. 1, pp. 35-40, Jan./Feb. 2009, doi:10.1109/MCSE.2009.19

• P. Vandewalle, J. Kovacevic and M. Vetterli, Reproducible Research in Signal Processing - What, why, and how, IEEE Signal Processing Magazine, vol. 26, no. 3, May 2009, pp. 37-47.

Talks and other 'informal' write-ups about reproducible research

• K. Coombes, Sweave: First steps toward reproducible analyses, presentation given at UT M. D. Anderson Cancer Center, Feb. 2007.

• G. Wilson, High-Performance Computing Considered Harmful, May 2008. See also a related interview with Greg Wilson.

• Interview with Roger Barga on Trident, a workbench for scientific workflow for oceanography, Aug. 2008.

Tools

Open Source

• AMRITA: a cross between a document preparation system, a computational engine, and a programming language.

• coNCePTuaL: A Network Correctness and Performance Testing Language.

• Madagascar: an open-source software package for multidimensional data analysis and reproducible computational experiments.

• RRepository: a repository setup for making reproducible research publications available online, based on EPrints.

• Sweave
  • The Sweave Homepage: by Friedrich Leisch
  • A Sweave Demo: a package to do literate programming and good documentation using the statistical software R, by Charlie Geyer.
  • Troubleshooting Sweave: by John Cook.

• StatWeave: software whereby you can embed statistical code (e.g., SAS, R, Stata, etc.) into a LaTeX or OpenOffice document. A bit like Sweave, but for more languages, developed by Russell V. Lenth.

• ThePub: an alternative setup for making reproducible research publications available online, using Java.

Commercial

• Inference: a tool for performing reproducible research from within Microsoft Office (Word, Excel) documents, with links to scripts in Matlab, R, etc.

Blogs

Reproducible Research

• Reproducible Research Ideas: This site's blog about reproducible research.

• RRPlanet Blog: RRPlanet blog about reproducible research.

Related Topics

• The Endeavour: John D. Cook's blog about statistics, programming, and reproducible research.

• EPrints News: latest news from the developers of the EPrints repository software.

• Open Access Archivangelism: Stevan Harnad's blog about Open Access and related issues.

• The Third Bit: Greg Wilson's blog, also containing comments on reproducible research, life in academia, software engineering, etc.

• Victoria Stodden: blog about internet and democracy, open science, intellectual property, etc.

• Madagascar Development Blog.