Non-Experimental Design

PSYC 641 - Spring 2025

Welcome

Announcements

  • Syllabus grade breakdown:

    Component % of Grade
    Participation& Engagement 30%
    Project Proposal 5%
    Design & Measures Draft 10%
    Design & Measures Peer Review 5%
    Data Analytic Plan Draft 10%
    Final Report 20%
    Final Presentation 20%

Overview

  • Reading Question Check in

  • Defining Non-Experimental

  • Highlighting the different techniques

  • Methodological Components

  • Apply it to a Study!

  • What do we do with the stats??

Reading Questions Check-in

Take a moment and review the questions. Highlight those that you still find unclear.

Link to Document

Non-Experimental Designs

What are they?

What are they?

  • Research in which our variables tend to be observed rather than manipulated

  • Absence of random assignment

  • Or Both!

  • We tend to use non-experimental designs when we cannot satisfy the conditions above

    • Instances where we want to examine some level of causality

Types of Non-Experimental Designs

  • Single-case designs

  • Correlational

    • Naturalistic Observation

    • Archival Data

  • Quasi-experimental

  • Qualitative

Single Case Designs (SCD)

  • Following a single individual over time

  • When would this apply in your field/area?

    • Low base rate of what you’re looking at (e.g., OCD, Schizophrenia)

    • Intervention work

SCD produced 3 levels of knowledge

  • Purely descriptive

    • Shows pattern of change for an individual

  • Correlational

    • Covariation of different variables over time (trajectories and a predictor)

  • Causal

    • Answer questions about the effects of another variable

https://ou-books.gitlab.io/scda---single-case-design-analyses/scda-book.pdf

Types of Non-Experimental Designs

  • Single-case designs

  • Correlational

    • Naturalistic Observation

    • Archival Data

  • Quasi-experimental

  • Qualitative

  • They all tend to mix together

Correlational

No independent variable is manipulated. Interested in the relationship between variables.

  • Naturalistic Observation: What we did a few weeks ago!

    • Being clear about what you are observing

  • Archival Data

    • Using existing data sources to answer questions

      • Databases (LDBase, NHANES, Add Health, HDX) Other Examples?

        • Census Data, Pew Research Center, Social Determinants of Health (USDA)

        • Integrated Data Analysis

      • Publicly Accessible (Reddit Posts, Tweets, Reviews)

Quasi-Experimental

“Resembling” an experiment

  • Often times we don’t have the ability to randomize to groups

    • Instead, investigate naturally occurring groups

  • Have to deal with differences across the groups

  • Pretest-Posttest Design

    • How does time play into this?

    • The role of time (e.g., maturation, regression to the mean, spontaneous remission)

Designing A Study

How to Start a Study?

What are the steps that you take to design a study?

Our area of focus will be on testing the placebo effect on sleep quality

  1. Come up with a question!
  2. Does the placebo effect impact an individual’s perceived sleep quality
    1. Placebo Effect - We will lie and say that they slept well; We will say they slept worse too
    2. Ratings of perceived sleep quality
  3. Consider Boundaries or limitations
  4. Look at what has been used in the past
    1. Consider population and the sample we will be using
  5. What resources do we have??
  6. What do we do about hypotheses??
  7. How do we measure these things (PSG vs. Fake Sensor)
    1. Make sure measures are valid and align
    2. Need to determine the “mismatch” and operationalize what this is
  8. Considering Time

Building Blocks to a study

Pick and choose the things that are going to most accurately assess your question

1. Observations/Measurements

When measurements are taken (pretest, posttest, multiple time points)

Types of measures used (standardized tools, observation protocols)

2. Treatments or Interventions

Implementation strategies

Dosage or intensity levels

3. Groups

Experimental groups (receive intervention)

Control groups (no intervention)

Comparison groups (alternative intervention)

4. Assignment Strategies

Random assignment

Matching

Self-selection

Cutoff-based assignment (as in regression discontinuity designs)

5. Time Elements

Data collection points (past, present, future)

Time frame of research

Number and spacing of observations

But what about statistics??