Academic Lead for Good Research Practice

https://debruine.github.io/talks/ALfGRP/

Lisa DeBruine

Who am I?

• Prof Lisa DeBruine
  
• School of Psychology & Neuroscience
  
• UK Reproducibility Network Institutional Lead for UofG
  
• Chair of APA Open Science and Methodology Expert Panel
• Associate Director of the Psychological Science Accelerator
• PsyTeachR team for reproducible research educational resources

ALfGRP Role Description

In this role you will provide academic leadership, working with colleagues across the university research ecosystem and externally, to design and implement a framework to ensure the integration of activities relevant to improving good research practice, including, but not limited to, research integrity, research ethics (non-clinical), reproducible and transparent research, open research, responsible research and the use of AI in research.

The postholder will also drive the alignment of training, policies and awareness to ensure that our researchers apply the highest standards of practice in the planning, execution and reporting of their research.

Duties

• to provide strategic leadership across the University on the University’s commitment to driving continuous improvement in research practice across all areas of research activity;
• to provide academic expertise contributing to and shaping the University’s strategic planning processes on matters relating to research practice improvement;
• accountable for the reporting of activities to the University’s Research Planning & Strategy Committee;
• the role includes delegated academic oversight of various Concordats and sector agreements to which the University is a signatory, including the Concordat to Support Research Integrity, the Concordat on Open Research Data, as well as elements of the San Francisco Declaration on Research Assessment (DORA) and the Knowledge Exchange Concordat that relate to research practice improvement. The postholder will be the institutional lead for the UK Reproducibility Network (UKRN);
• to provide academic direction to the Professional Services teams who support the development and promotion of good research practice;
• to identify and champion innovations in ways of working that serve to improve research practice, taking into account how priorities vary across disciplines. Evaluate researchers ways of working across the University, and the provisions available to support improvement in research practice in-order to identify and development new & innovative solutions designed to improve overall research practices across the organisation;
• to champion the development and implementation of policies, initiatives, training, and/or awareness-raising, to deliver continuous improvement in research practice across disciplines and career stages, including an evaluation of interventions;
• to act as a role model and academic advocate for good research practice by applying exemplary standards of professionalism in the University’s pursuit of improved research practice, for example by applying rigour and transparency in the planning, execution, evaluation, and reporting of actions;
• to represent and promote the University externally in activities led by funders and other relevant sectoral organisations, including as the University of Glasgow institutional lead for the UK Reproducibility Network (UKRN);
• to establish and build strong relationships with key individuals in major external funders, including UKRI and its constituent Research Councils, and other relevant organisations, including the UK Committee on Research Integrity (CORI)that are mutually beneficial to the achievement of strategic objectives as detailed within the Research Strategy;
• to provide situational awareness of, and steering institutional responses to, the evolving landscape for good research practice across the sector, ensuring this is reflected across the provisions and solutions available across the organisation which are developed and designed to enhance the University’s research practices;
• to work closely with key internal stakeholders, including Deans of Research, Deans of Graduate Studies, Research Services, Research Information Management team, Research Integrity Champions and Advisors, and key individuals and groups in Schools, Colleges and University Services in order to further develop the University’s research environment and creating a positive research culture;
• to identify and lead areas of activity in which the University can demonstrate a reputation as sector-leading in enhancing research quality;
• to secure external sources of funding, as appropriate, to drive improvements in research quality and sector leadership.

Values

Ambition & Excellence

Curiosity & Discovery

Integrity & Truth

Inclusive Community

• “We value the quality of our research over its quantity”
• “How research is done is as important as what is done”

Culture Change Pyramid

SCARF Model

Status

Valuing rigour over prestige

Certainty

Clear training and resources

Autonomy

Trusted discipline-specific advisors

Relatedness

Create communities of practice

Fairness

Fair recognition and reward

Current Tasks

• Revising the integrity/GRP advisor network
  
• Chairing the DORA working group
  
• Advising on research practice indicators
  
• Recruiting a network of AI reps across Scottish unis
  
• Forming ideas for a best research practice framework

What is Good Research Practice?

Key Concepts

Collaboration

All research roles are appropriated valued and credited

Accessibility

Stakeholders can access research outputs

Transparency

Processes are documented for others to assess and learn from

Replication Crisis

From A manifesto for reproducible science, 10.1038/s41562-016-0021

Reproducibility

Scriberia. CC-BY 4.0. DOI: 10.5281/zenodo.3332807

Key practices

Open Materials/Data/Code

Open Science Framework

UK Data Service

Databrary

Zenodo

GitHub

Registry of Research Data Repositories: Re3Data

Glasgow Open Research Data Repositories

DMP for Personal Data

Glasgow Research Data Management

Analytic Reproducibility

An analysis by Nuijten et al. (2016) of over 250K p-values reported in 8 major psych journals from 1985 to 2013 found that:

• half the papers had at least one inconsistent p-value
• 1/8 of papers had errors that could affect conclusions
• errors more likely to be erroneously significant than not

Of 35 articles published in Cognition with usable data, Hardwicke et al. (2018) found:

• only 11 could be reproduced independently
• 11 were reproducible with the original authors’ help
• 13 were not reproducible even by the original authors

Why Code? talk expanding on this

Computational Reproducibility

Of 62 Registered Reports in psychology published from 2014–2018, 36 had data and analysis code, 31 could be run, and 21 reproduced all the main results (Obels et al, 2020)

Preregistration

Haroz (2022) Comparison of Preregistration Platforms, 10.31222/osf.io/zry2u

Registered Reports

osf.io/rr/

Preprints

List of 62 preprint servers

Research Publications and Copyright Policy

For the purpose of open access, the author(s) has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission.

• Policy

Resources

Glasgow Open Research

UK Reproducibility Network

The Turing Way

Framework for Open and Reproducible Research Training (FORRT)

Thank You!

https://debruine.github.io/talks/ALfGRP/