Good Research Practice

https://debruine.github.io/talks/GRP/

University of Glasgow Values

Ambition & Excellence

Curiosity & Discovery

Integrity & Truth

Inclusive Community

  • “We value the quality of our research over its quantity”
  • “How research is done is as important as what is done”

Research Culture Priorities

  • Research Recognition: All colleagues are valued for their varied contributions to a diverse range of research activities and outputs
  • Collegiality: People actively work together to support each other to succeed
  • Research Integrity: We engage with and produce research that meets the highest standards of integrity
  • Open Research: We are committed to openness, transparency, rigour, and reproducibility
  • Career Development: We support all members of our research ecology to advance in their chosen career path

Culture Change Pyramid

What is Good Research Practice?

Key Concepts

Collaboration

All research roles are appropriated valued and credited

Accessibility

Stakeholders can access research outputs

Transparency

Processes are documented for others to assess and learn from

Who is GRP for?

UKRN Open Research Acrooss Disciplines

Reproducibility

Image of a map map showing the 'path' from data to tools to results to code

Scriberia. CC-BY 4.0. DOI: 10.5281/zenodo.3332807

Replication Crisis

An idealized version of the hypothetico-deductive model of the scientific method. Various potential threats to this model exist (indicated in red), including lack of replication, hypothesizing after the results are known (HARKing), poor study design, low statistical power, analytical flexibility, P-hacking, publication bias and lack of data sharing.

From A manifesto for reproducible science, 10.1038/s41562-016-0021

Key practices

Contributorship

  • Conceptualization
  • Data curation
  • Formal analysis
  • Funding acquisition
  • Investigation
  • Methodology
  • Project administration
  • Resources
  • Software
  • Supervision
  • Validation
  • Visualization
  • Writing (original draft)
  • Writing (review & editing)

Tenzing: App/Paper

Open Materials/Data/Code

OSF logo - blue dots

Open Science Framework

UKDS logo - a rainbow box

UK Data Service

Databrary logo - a leaf

Databrary

Zenodo

Zenodo

GitHub logo - an octocat

GitHub

Registry of Research Data Repositories: Re3Data

Glasgow Open Research Data Repositories

DMP for Personal Data

Glasgow Research Data Management

Analytic Reproducibility

An analysis by Nuijten et al. (2016) of over 250K p-values reported in 8 major psych journals from 1985 to 2013 found that:

  • half the papers had at least one inconsistent p-value
  • 1/8 of papers had errors that could affect conclusions
  • errors more likely to be erroneously significant than not

Of 35 articles published in Cognition with usable data, Hardwicke et al. (2018) found:

  • only 11 could be reproduced independently
  • 11 were reproducible with the original authors’ help
  • 13 were not reproducible even by the original authors

Why Code? talk expanding on this

Computational Reproducibility

Of 62 Registered Reports in psychology published from 2014–2018, 36 had data and analysis code, 31 could be run, and 21 reproduced all the main results (Obels et al, 2020)

Preregistration

The features of general-purpose preregistration services. See link for text version.

Haroz (2022) Comparison of Preregistration Platforms, 10.31222/osf.io/zry2u

Registered Reports

LEGO minifigs measuring each other, next to a flow chart of the RR process: Develop idea, design study, stage 1 review, collect and analyse data, write report, stage 2 review, publish report.

osf.io/rr/

Preprints

arXiv logo (word with the X like a chain link)

AfricArXiv logo with Africa made of colourful stripes

bioRxiv logo (word with red R)

edArXiv logo (word with a quill and fountain for the i)

MetaArXiv logo (white Meta on a multicoloured background)

medRxiv logo (word with blue R)

psyArXiv logo (Psi, A, and Chi in red boxes)

earthRxiv logo (padlock where half the round part is a globe)

SocArXiv logo (white word on a black background)

zenodo logo (word on a blue background)

List of 62 preprint servers

Next Steps

  • Read Easing into Open Science
  • Get an ORCiD
  • Ask your team about open research practices and tools in your areas
  • Think about open research (especially open data) in the planning stages

Flowchart of main steps and practices. Conceptualisation: journal club, project workflow; Design: Preregistration, Registered reports, data sharing planning; Analysis: Reproducible code; Reporting: transparent writing; Dissemination: Preprints, data sharing

Open Science research practices across the research cycle

Good Research Practice Network

  • 25 School/Unit Advisors
  • 4 College Champions
    • Arts & Humanities: Dahlia Porter
    • MVLS: Simon Milling
    • Science & Engineering: Sonja Franke-Arnold
    • Social Sciences: Louise Kennefick
  • Academic Lead for GRP (Lisa DeBruine)
  • Research Integrity Team (Amanda McKenna, Sam Oakley, Elinor Toland)
  • Various research support staff

Other Resources

Good Research Practice Network

Research Integrity

Open Research

UK Reproducibility Network

The Turing Way

Framework for Open and Reproducible Research Training (FORRT)

Thank You!

https://debruine.github.io/talks/GRP/