8 Chapter 8: Procedures

Learning Objectives:

• Understand the importance of ethical practices in data collection and research procedures.
• Learn about the process of pre-registration of studies and its role in enhancing research transparency and reproducibility.
• Explore the principles of planning reproducible research, including data accessibility, methods documentation, and code and analysis reproducibility.
• Gain insight into the ethical considerations surrounding participant privacy, informed consent, minimizing harm and risk, and data collection integrity.
• Discuss the responsible use of research data, with a focus on the ABCD study as an example.
• Examine the comprehensive steps involved in data collection and ethical procedures in research, from obtaining ethical approval to ongoing consent and communication.

Key Terms:

• Data Collection Procedures: The specific methods and steps outlined in the research protocol for gathering data, including the use of instruments, timing of data collection, and protocols for ensuring data accuracy and security.
• Ethical Practices in Data Collection: Guidelines and principles that ensure the dignity, rights, and welfare of research participants are respected throughout the data collection process.
• Preregistration of Studies: The process of registering the study design, methods, and analysis plan with a scientific body before conducting the research to enhance transparency and credibility.
• Reproducible Research: Research practices that allow others to replicate and verify the results by providing access to data and clear documentation of methods and analyses.
• Informed Consent: The process of providing participants with complete information about the study, including its purpose, procedures, risks, and benefits, and obtaining their voluntary agreement to participate.

Introduction to Procedures in Data Collection & Ethics

Data collection procedures detail the specific methods and steps for gathering data. These procedures describe the tools and instruments used, such as surveys or laboratory tests, and outline how and when data will be collected. They also include protocols for ensuring the accuracy, consistency, and security of the collected data. Additionally, researchers must adhere to ethical guidelines while carrying out research procedures. Ethics in research, at its core, research ethics is about respecting the dignity and rights of all individuals involved in the study. It involves making thoughtful decisions that balance scientific advancement with the well-being of participants.

Preregistration of Studies

Preregistration means registering a study design with a scientific body before conducting the research. Preregistration should be done prior to data collection procedures and/or data analysis. This proactive step is taken to ensure that the research is transparent and that the results are credible and reproducible, and preregistration is increasingly becoming a standard (Lindsay, Simons et al. 2016).

There are many benefits of preregistration. One of the primary goals of preregistration is to increase statistically reproducible research. It discourages practices such as “p-hacking” (searching for significant results in different subsets of data), “HARKing” (hypothesizing after results are known), and the misuse of “researcher degrees of freedom” (making data-dependent analysis decisions) (Munafò, Nosek et al. 2017). By outlining the analysis plan in advance, researchers are held accountable to their original hypotheses and methods, reducing the temptation to engage in these practices.

Therefore, preregistration enhances transparency by providing a public record of the intended research methodology, which can be compared with the final published results. This comparison helps to reduce publication bias, as it becomes more difficult to selectively report results based on their outcomes. Preregistration also increases the credibility of research findings, as it demonstrates a commitment to methodological rigor and integrity.

There are several platforms available for preregistering studies, with the Open Science Framework (OSF) and ClinicalTrials.gov being among the most widely used. These platforms provide a structured format for researchers to document their study plans and make them accessible to the public. When preregistering a study, researchers typically include a description of the study design, hypotheses, variables, data collection procedures, and planned statistical analyses. It is also important to specify how data will be collected and processed.

Planning Reproducible Research

Beyond preregistration, researchers should also act on several considerations to increase the reproducibility of their studies. Reproducibility refers to research findings that are reliable and can be independently verified. Planning for reproducible research involves several key components (Munafò, Nosek et al. 2017):

• Data Accessibility: This refers to researchers sharing their data (accompanied by clear instructions for management and analysis), such that other researchers can access the same information and follow the same procedures to verify or reproduce the results. For example, the Human Microbiome Project exemplifies data accessibility by providing open access to its data, allowing researchers worldwide to validate and extend the findings.
• Methods Documentation: This refers to researchers keeping detailed documentation of their methods used to obtain and analyze data. This transparency allows others to understand, evaluate, and replicate the study. The ENCODE project, for instance, offers extensive documentation of its experimental protocols and data analysis pipelines, serving as a model for methods documentation.
• Code and Analysis Reproducibility: This refers to sharing the code used for data analysis, including statistical analyses, visualizations, and modeling. The Cancer Genome Atlas project, for example, shares its bioinformatics code on public repositories, enabling other researchers to replicate their analyses and build upon their work.

Reproducible research practices have several benefits. It enhances the quality of science by enabling validation and extension of existing work. It accelerates scientific progress by facilitating collaboration and knowledge exchange among scientists. And it helps reduce the risk of wasted efforts due to irreproducible findings. In addition to preregistration, a prerequisite step to data collection should be clear plans to produce reproducible research.

Ethical Practices in Data Collection

Prior to outlining data collection procedures, researchers must follow ethical standards in data collection to protect and promote the integrity of science. These practices are guided by key ethical principles such as respect for persons, beneficence, and justice, which aim to protect participants and uphold the quality of the research process. Main features of ethical practices in data collection include (Morse 2007, Guraya, London et al. 2014):

• Ensuring Participant Privacy and Confidentiality: Researchers must protect participants’ privacy and confidentiality by employing strategies such as anonymization, where identifying information is removed, or pseudonymization, where identifiers are replaced with pseudonyms. Secure data storage methods, such as encrypted databases, are also essential to prevent unauthorized access to sensitive information.
• Informed Consent in Data Collection: Participants must be fully informed about how their data will be used, stored, and shared. They should understand their rights, including the right to withdraw from the study at any time, and provide their consent willingly and without coercion.
• Minimizing Harm and Risk: Researchers must minimize any potential harm or risk to participants by identifying and assessing risks associated with data collection and implementing measures to mitigate them. Special consideration should be given to vulnerable populations (e.g., children) or sensitive topics (e.g. illicit activities or contexts of stigma).
• Data Collection Integrity: Researchers must maintain integrity in data collection by avoiding bias and ensuring that data are accurately recorded, including transparent and honest reporting practices to maintain data integrity.
• Ethical Use of Digital Tools and Technologies: Researchers must be vigilant in protecting participants’ information, including attention to data privacy and data security.
• Disclosing Conflicts of Interest: Researchers must be transparent about any potential conflicts of interest that could influence the research process or outcomes. Conflicts of interest can arise from financial ties, personal relationships, or other factors that may affect the objectivity of the study. It is crucial to disclose these conflicts to ensure the integrity of the research and maintain trust with participants and the scientific community. For example, if a researcher receives funding from a pharmaceutical company for a study on a new medication, this should be clearly stated to avoid any perception of bias.
• Cultural Sensitivity and Respect: Researchers must recognize and respect cultural differences and adapt their data collection methods to be culturally appropriate and sensitive. This ensures that participants feel respected and valued. In certain cases, they may involve community engagement, in which researchers incorporate individuals, communities, and stakeholders into research processes (Adhikari, Pell et al. 2020).
• Ethical Data Sharing and Publication: Researchers must follow guidelines for ethically sharing data and ensure that shared data are de-identified. Ethical publication practices involve transparent reporting and avoiding the manipulation of data.
• Legal and Regulatory Compliance: Researchers must comply with legal and regulatory requirements related to data protection, such as the General Data Protection Regulation (GDPR). Adherence to these laws and regulations is crucial for ethical data collection.

Ethical practices in data collection are essential to safeguarding participants’ rights and wellbeing, ensuring the validity of research, and maintaining the trustworthiness of the scientific community. Let’s consider a concrete example. Social media platforms contain vast amounts of personal information that researchers might find valuable for understanding health behaviors and outcomes. However, the use of this data raises significant ethical questions about privacy and consent. Imagine a research project aimed at determining how social media usage impacts mental health among teenagers. Researchers plan to use data gathered from social media platforms to analyze posting patterns, online interactions, and sentiment analysis.

Key Ethical Questions in the Example:

• Consent: Have the teenagers (or their guardians, if minors) given informed consent for researchers to access and analyze their social media profiles?
• Privacy: How will researchers ensure that the personal information collected from social media is kept confidential? What measures are in place to prevent the identification of individual participants from the data?
• Data Security: What steps will be taken to secure the data against unauthorized access, especially when such data can be sensitive and personally revealing?

Consider how you might feel if someone analyzed your social media profiles to study your health without your explicit consent or knowledge. Such a scenario highlights the importance of ethical considerations in protecting individuals’ rights in health research.

Responsible Use of ABCD Data

To access ABCD data, researchers much sign a Data Use Agreement/Certificate (DUC) for ABCD data. The prerequisite is that researchers must be affiliated with an NIH-recognized research institution and have an active Federalwide Assurance, as well as a research-related need to access the data. Responsible data use of ABCD includes using the data only for research, not reidentifying subjects, keeping the data secure, working only with approved users, deleting the data after use, and submitting results to the NIMH Data Archive (NDA). Violating these terms can result in penalties, and the agreement is valid for one year.

Additionally, there are a range of other responsible use considerations (Simmons, Conley et al. 2021). It’s important to respect the youths and families whose lives form the basis of the research data, consider the influence of social context on developmental outcomes, and be aware of the heterogeneity of youths’ experiences. Researchers should use multifactorial variables to estimate context and experience, situate findings in the broader social context, avoid deterministic language, be modest with conclusions, recognize that unfavorable outcomes in one environment may be adaptations in another, and consider how findings may be misinterpreted (Simmons, Conley et al. 2021).

Data Collection & Ethical Procedures in Research

This section outlines the core components of data collection and ethical procedures in research:

• Obtain Ethical Approval: Prior to initiating the study, researchers must obtain approval from an Institutional Review Board (IRB) or an equivalent ethics committee. This ensures that the study meets ethical standards for protecting participants’ rights and welfare. The IRB review process evaluates the risks and benefits of the study, as well as the informed consent process. Researchers submit a research protocol to the IRBs, which includes the study design details covered in Chapters 3-8 of this textbook.
• Recruit Participants and Informed Consent: The next step of data collection procedures is to carry out the study’s participant recruitment strategy (covered in Chapter 6), based on specific inclusion and exclusion criteria outlined in the study protocol. Researchers must obtain informed consent from participants before their involvement in the study. This process involves explaining the study’s purpose, procedures, potential risks and benefits, and participants’ rights, such as the right to withdraw at any time.
• Randomization and Baseline Assessment: In randomized clinical trials, researchers must describe their process for randomly assigning participants to different groups (e.g., intervention vs. control), and how participants will arrive to these groups. Typically, before the intervention begins, initial data are collected on participants’ health and other relevant variables, which provides a reference point for assessing the effects of the intervention.
• Data Collection: Researchers need to describe how they will use instruments (described in Chapter 7) to collect data. Researchers must maintain integrity in data collection by avoiding bias and ensuring that data are accurately recorded, including transparent and honest reporting practices, which involves meticulous documentation of data collection methods. In clinical trials, researchers must describe in detail the step-by-step process of the experimental intervention, including, what the researchers will do, when, and for how long.
• Monitoring and Quality Control: In clinical trials, researchers must describe how they monitor participants for adherence to the protocol, participant safety, and data quality. This may involve regular meetings of a data monitoring committee and ongoing quality control checks.
• Data Management and Analysis: Researchers must ensure secure storage of data and have clear protocols for data access and sharing. Researchers need to develop a data management plan at the outset of a project and detail how data will be handled throughout the research lifecycle. Utilizing appropriate software and tools for data management can enhance efficiency and ensure compliance with ethical and legal standards for data protection.
• Ongoing Consent and Communication: In longitudinal studies, it is important to maintain ongoing consent and communication with participants. This involves regularly updating participants on the study’s progress and re-confirming their consent at each assessment point.

The above listed procedures apply to both experimental and observational studies, although certain steps are not done in observational research – such as dividing participants into treatment and control groups.

Addressing Ethical Dilemmas in Research

Researchers often encounter ethical dilemmas that require careful consideration and decision-making. For example, conflicts of interest may arise when personal or financial interests could influence research outcomes. Researchers must disclose such conflicts and ensure they do not compromise the integrity of the study. Confidentiality issues can emerge when handling sensitive participant data, requiring stringent measures to protect privacy. Navigating these dilemmas requires a commitment to ethical principles, transparent communication, and adherence to established guidelines.

ABCD Data Collection Procedures & Protocols

The official ABCD website has visual overviews of ABCD data collection procedures. ABCD coordinators and study groups have publications outlining details of data collection procedures within specific domains and instrumentation. For example, below is an overview of ABCD neuroimaging data collection procedures, derived from “The Adolescent Brain Cognitive Development (ABCD) study: Imaging acquisition across 21 sites” (Casey, Cannonier et al. 2018). The imaging component of the ABCD study was developed to measure brain structure and function relevant to adolescent development and addiction.

• Equipment and Software: The ABCD imaging protocol is harmonized across three 3T scanner platforms: Siemens Prisma, General Electric 750, and Philips. The use of multi-channel coils capable of multiband echo planar imaging (EPI) acquisitions is standardized across sites. Special stimulus presentation and response collection equipment and software are used for task-based fMRI scans.
• ABCD Scan Protocol: The scan session consists of a fixed order of scan types, including 3D T1- and 3D T2-weighted images, diffusion-weighted images, resting state fMRI, and task-based fMRI. The order of the three fMRI tasks (Monetary Incentive Delay, Stop Signal Task, and EN-back Task) is randomized across subjects.
• Pre-Scan Assessments and Training: Participants undergo MR screening, simulation, and motion compliance training before the scan. An arousal questionnaire is administered immediately prior to scanning.
• Scan Session Details: Scanning Parameters: The imaging parameters are harmonized for the three scanner platforms.
• Motion Detection and Correction: Real-time motion detection and correction are implemented for structural scans. The FIRMM system is used for motion detection in resting state fMRI scans at Siemens sites.
• fMRI Tasks: The ABCD study includes three task-based fMRI assessments – the Monetary Incentive Delay Task, the Stop Signal Task, and the EN-back Task. Each task measures different aspects of brain function relevant to addiction and adolescent development.
• Post-Scan Assessments: After the scan, participants are administered the arousal questionnaire again, followed by the EN-back Recognition Memory Task and the Monetary Incentive Delay Task Post-Scan Questionnaire.

Examples of Procedures Sections in Studies Published with ABCD

Below are two examples of sections of data collection and ethics procedures in studies published with ABCD data.

Example 1:

Cullen, Breda, et al. “Cognitive Function in People With Familial Risk of Depression.” JAMA psychiatry 80.6 (2023): 610-620.

Research question: Are hypothesized associations between familial risk of depression and lower cognitive performance evident across the life span for both family history and genetic risk measures? The study aims to investigate associations between familial risk of depression and cognitive performance in four independent cohorts, using both family history and genetic risk measures, to understand whether these associations are evident across the life span .

Procedures: The procedures are outlined under Methods, quoted at length here:

Method

This study used a cohort design within TGS, ABCD, and UK Biobank, with family history data collected at one assessment wave and cognitive outcomes measured at a later wave. In Add Health, the family history data and cognitive data were only available at the same wave, and so these analyses were cross-sectional. Each cohort’s study procedures were approved by the relevant institutional review board or ethics committee, and participants gave written informed consent. This study follows the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

Example 2:

Kochvar, Andrew, et al. “Genetic and environmental influences on early-age susceptibility and initiation of nicotine-containing product use: A twin-pairs study.” Tobacco Prevention & Cessation 9 (2023).

Research Question: The study has a few interrelated questions: Do genetic and environmental factors influence youth early-age nicotine-containing product (NCP) use behaviors? Do identical pairs have a higher correlation in NCP use behaviors than fraternal twin pairs? Does the heritability vary between NCP use susceptibility and initiation?

Procedures: Quoted at length from the Methods Section:

Methods

Participants are asked for in-person assessment sessions once a year for behavioral and biospecimen collections, with brief remote assessments (e.g. youth substance exposure/use) at 6 months between inperson sessions. All parents or guardians provided written informed consent, and children gave written assent. Study procedures were approved by the UC San Diego Central Institutional Review Board (IRB) and each local institutional IRB. This article followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline for cohort studies. Data from this study were collected from the ABCD Twin Hub, a sub-study of monozygotic (MZ) and dizygotic (DZ) same-sex twins from four sites in Minnesota, Colorado, Virginia, and Missouri.

Example 3:

Owens, Max M., et al. “Test–retest reliability of the neuroanatomical correlates of impulsive personality traits in the adolescent brain cognitive development study.” Journal of Psychopathology and Clinical Science 132.6 (2023): 779.

Research question: How replicable are the neuroanatomical correlates of impulsive personality traits across childhood and adolescence, specifically from age 9/10 to age 11/12 in the Adolescent Brain Cognitive Development Study? The study aims to investigate the stability and replicability of the structural brain correlates of impulsive personality traits, as measured by the UPPS-P Impulsive Behavior Scale, in a large sample of youths over a two-year period (Owens, Hyatt et al. 2023).

Procedures: The author’s detail ethical commitments under Methods: “Ethical considerations and study procedures of the ABCD Study are discussed extensively elsewhere (Casey et al., 2018; Clark et al., 2018). Locally, this study was approved by the Hamilton Integrated Research Ethics Board, in protocol number 14830-C, titled ‘Investigating the Neurobiology of Self-Control in Publicly Available Neuroimaging Datasets’” (Owens, Hyatt et al. 2023). However, the data collection procedures for the study are not explicitly detailed in a separate section within the article. The “Method” section does provide some information on data collection, particularly in the subsections “Participants,” “Measures,” and “MRI Preprocessing.” These subsections describe the recruitment of participants, the administration of the UPPS-P Impulsive Behavior Scale, and the processing of MRI data, which are all key components of the data collection process in this study. This example illustrates that, depending on the study, researchers may or may not publish an explicit section on data collection procedures. While this is not common, it most likely happens in publications with secondary datasets, such as ABCD.

Summary

In this chapter, we covered the essential aspects of research procedures, focusing on data collection and ethics. We discussed pre-registration, reproducible research, and ethical practices in data collection. We also examined the responsible use of data, particularly using the ABCD study as an example. The next chapter concerns how to analyze data that has been collected.