Ethics and reproducible research in statistics & computer science

Fred Boehm
May 23, 2014

IRES Program

• Graduate School’s Integrating Research Ethics and Scholarship
• Year-long fellowship program for graduate students and post-docs
• 15 fellows (from across UW-Madison departments) for 2013-2014 year
• Become a departmental resource for issues related to research ethics
• Teach and lead a small group discussion to your peers
• Design, implement, and assess one research ethics event for your department and one for campus

IRES Fellows Learning Outcomes

1. Critically evaluate issues that relate to research ethics that arise in research/scholarship.
   
2. Analyze case studies to communicate your understanding of the responsible conduct of research/scholarship.
   
3. Apply effective pedagogies in implementing research ethics events.

IRES Fellows Learning Outcomes

4. Explain policies at the institutional, state, and federal levels that relate to the responsible conduct of research/scholarship.
   
5. Apply professional codes of conduct and ethical principles to frame ethical questions, problems, and decisions that arise in research/scholarship.
   
6. View yourself as an advocate for research integrity.

Reproducibility and research ethics

• Concern is motivated by high-profile cases of research misconduct
  • Anil Potti & Joseph Nevins genomics scandal at Duke University
    
• Computational software tools ease implementation
  • R has literate programming systems: knitr & Sweave
  • R now has rctrack, too

What is reproducible research?

• How to define it? Ideas?

What are researchers’ ethical obligations to other researchers?

• Are we required to “check” others’ work (M.D. Anderson team in the Duke scandal)?

• Why or why not?

What are shortcomings or limitations of current tools for reproducible research?

• Limitations of knitr & Sweave?
  
• Some computing tasks may depend on many data files, R script files, and other factors that can (potentially) vary over time
  • Data: Microarray image files, Sequence alignment files
    
  • Computer code: R packages (versions), primary analysis code
    
  • It may be impractical (and undesirable) to force researchers to manually record these changes

Liu & Pounds’s rctrack: What is it & what does it do?

• An R package that automatically collects and archives details for reproducible computing

• Computationally collects & archives details of R computations
  • More user-friendly than linux’s CDE
  • Accessible to users of many operating systems: Windows, MacOS, linux

Liu & Pounds’s rctrack: What is it & what does it do?

• Integrates with literate programming software in R (Sweave & knitr)
  
• Includes details of pseudo-random number generation
  
• Archives can be used for R package development
  
• Capable of comparing two instances of archives to identify differences

Implementing rctrack

Implementing rctrack

Fred’s reproducible research & ethics resource webpage

Fred’s reproducible research & ethics resources

References

• An R package that automatically collects and archives details for reproducible computing

• rctrack project page (free download)

• CDE software project (for linux)