What is Open Research?

https://debruine.github.io/talks/UofG-open-res/

Lisa DeBruine

Housekeeping

Schedule/Notes: tinyurl.com/uofg-open-res-22

  • Toilets
  • Breaks
  • Water
  • Venue

Key concepts

Collaboration

All research roles are appropriated valued and credited

Accessibility

Stakeholders can access research outputs

Transparency

Processes are documented for others to assess and learn from

Reproducibility

Image of a map map showing the 'path' from data to tools to results to code

Scriberia. CC-BY 4.0. DOI: 10.5281/zenodo.3332807

Key practices

Contributorship

  • Conceptualization
  • Data curation
  • Formal analysis
  • Funding acquisition
  • Investigation
  • Methodology
  • Project administration
  • Resources
  • Software
  • Supervision
  • Validation
  • Visualization
  • Writing (original draft)
  • Writing (review & editing)

Tenzing: App/Paper

Preprints

arXiv logo (word with the X like a chain link)

AfricArXiv logo with Africa made of colourful stripes

bioRxiv logo (word with red R)

edArXiv logo (word with a quill and fountain for the i)

MetaArXiv logo (white Meta on a multicoloured background)

medRxiv logo (word with blue R)

psyArXiv logo (Psi, A, and Chi in red boxes)

earthRxiv logo (padlock where half the round part is a globe)

SocArXiv logo (white word on a black background)

zenodo logo (word on a blue background)

List of 62 preprint servers

Open Materials/Data/Code

A LEGO minifig version of Margaret Hamilton, lead of the Apollo mission NASA software team, holding a newspaper that says 'Men walk on the moon', standing next to a stack of computer code printouts

Computational Reproducibility

Of 35 articles published in Cognition with usable data, Hardwicke et al. (2018) found:

  • only 11 could be reproduced independently
  • 11 were reproducible with the original authors’ help
  • 13 were not reproducible even by the original authors

An analysis by Nuijten et al. (2016) of over 250K p-values reported in 8 major psych journals from 1985 to 2013 found that:

  • half the papers had at least one inconsistent p-value
  • 1/8 of papers had errors that could affect conclusions
  • errors more likely to be erroneously significant than not

Why Code? talk expanding on this

Preregistration

The features of general-purpose preregistration services. See link for text version.

Haroz (2022) Comparison of Preregistration Platforms, 10.31222/osf.io/zry2u

Registered Reports

LEGO minifigs measuring each other, next to a flow chart of the RR process: Develop idea, design study, stage 1 review, collect and analyse data, write report, stage 2 review, publish report.

osf.io/rr/

Replication

An idealized version of the hypothetico-deductive model of the scientific method. Various potential threats to this model exist (indicated in red), including lack of replication, hypothesizing after the results are known (HARKing), poor study design, low statistical power, analytical flexibility, P-hacking, publication bias and lack of data sharing.

From A manifesto for reproducible science, 10.1038/s41562-016-0021

Open Research Support at University of Glasgow

Matt Mahon
Slides (via SharePoint)