Ethics and Open Research

https://debruine.github.io/talks/UofG-open-res-ethics/

Lisa DeBruine
tech.lgbt/@debruine

Key concepts

Collaboration

All research roles are appropriated valued and credited

Accessibility

Stakeholders can access research outputs

Transparency

Processes are documented for others to assess and learn from

Replication Crisis

An idealized version of the hypothetico-deductive model of the scientific method. Various potential threats to this model exist (indicated in red), including lack of replication, hypothesizing after the results are known (HARKing), poor study design, low statistical power, analytical flexibility, P-hacking, publication bias and lack of data sharing.

From A manifesto for reproducible science, 10.1038/s41562-016-0021

Reproducibility

Image of a map map showing the 'path' from data to tools to results to code

Scriberia. CC-BY 4.0. DOI: 10.5281/zenodo.3332807

Key practices

Open Materials/Data/Code

OSF logo - blue dots

Open Science Framework

UKDS logo - a rainbow box

UK Data Service

Databrary logo - a leaf

Databrary

Zenodo

Zenodo

GitHub logo - an octocat

GitHub

Registry of Research Data Repositories: Re3Data

Glasgow Open Research Data Repositories

DMP for Personal Data

Glasgow Research Data Management

Analytic Reproducibility

An analysis by Nuijten et al. (2016) of over 250K p-values reported in 8 major psych journals from 1985 to 2013 found that:

  • half the papers had at least one inconsistent p-value
  • 1/8 of papers had errors that could affect conclusions
  • errors more likely to be erroneously significant than not

Of 35 articles published in Cognition with usable data, Hardwicke et al. (2018) found:

  • only 11 could be reproduced independently
  • 11 were reproducible with the original authors’ help
  • 13 were not reproducible even by the original authors

Why Code? talk expanding on this

Computational Reproducibility

Of 62 Registered Reports in psychology published from 2014–2018, 36 had data and analysis code, 31 could be run, and 21 reproduced all the main results (Obels et al, 2020)

Preregistration

The features of general-purpose preregistration services. See link for text version.

Haroz (2022) Comparison of Preregistration Platforms, 10.31222/osf.io/zry2u

Registered Reports

LEGO minifigs measuring each other, next to a flow chart of the RR process: Develop idea, design study, stage 1 review, collect and analyse data, write report, stage 2 review, publish report.

osf.io/rr/

Preprints

arXiv logo (word with the X like a chain link)

AfricArXiv logo with Africa made of colourful stripes

bioRxiv logo (word with red R)

edArXiv logo (word with a quill and fountain for the i)

MetaArXiv logo (white Meta on a multicoloured background)

medRxiv logo (word with blue R)

psyArXiv logo (Psi, A, and Chi in red boxes)

earthRxiv logo (padlock where half the round part is a globe)

SocArXiv logo (white word on a black background)

zenodo logo (word on a blue background)

List of 62 preprint servers

Resources

Issues for Ethics Committees

  • Address data sharing in template consent forms
  • Consistency of data management plans across consent, info, DMP, DPIA
  • Registered report-friendly procedures
  • Team research-friendly procedures
  • Should ethics committees do methodological review? (Lakens, 2023, Nature)