what is a single case design in research

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Perspective
• Published: 22 November 2022

Single case studies are a powerful tool for developing, testing and extending theories

• Lyndsey Nickels   ORCID: orcid.org/0000-0002-0311-3524 1 , 2 ,
• Simon Fischer-Baum   ORCID: orcid.org/0000-0002-6067-0538 3 &
• Wendy Best   ORCID: orcid.org/0000-0001-8375-5916 4  

Nature Reviews Psychology volume  1 ,  pages 733–747 ( 2022 ) Cite this article

650 Accesses

5 Citations

26 Altmetric

Metrics details

• Neurological disorders

Psychology embraces a diverse range of methodologies. However, most rely on averaging group data to draw conclusions. In this Perspective, we argue that single case methodology is a valuable tool for developing and extending psychological theories. We stress the importance of single case and case series research, drawing on classic and contemporary cases in which cognitive and perceptual deficits provide insights into typical cognitive processes in domains such as memory, delusions, reading and face perception. We unpack the key features of single case methodology, describe its strengths, its value in adjudicating between theories, and outline its benefits for a better understanding of deficits and hence more appropriate interventions. The unique insights that single case studies have provided illustrate the value of in-depth investigation within an individual. Single case methodology has an important place in the psychologist’s toolkit and it should be valued as a primary research tool.

This is a preview of subscription content, access via your institution

Access options

Subscribe to this journal

Receive 12 digital issues and online access to articles

55,14 € per year

only 4,60 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Microdosing with psilocybin mushrooms: a double-blind placebo-controlled study

A systematic review and multivariate meta-analysis of the physical and mental health benefits of touch interventions

The language network as a natural kind within the broader landscape of the human brain

Corkin, S. Permanent Present Tense: The Unforgettable Life Of The Amnesic Patient, H. M . Vol. XIX, 364 (Basic Books, 2013).

Lilienfeld, S. O. Psychology: From Inquiry To Understanding (Pearson, 2019).

Schacter, D. L., Gilbert, D. T., Nock, M. K. & Wegner, D. M. Psychology (Worth Publishers, 2019).

Eysenck, M. W. & Brysbaert, M. Fundamentals Of Cognition (Routledge, 2018).

Squire, L. R. Memory and brain systems: 1969–2009. J. Neurosci. 29 , 12711–12716 (2009).

Article   PubMed   PubMed Central   Google Scholar  

Corkin, S. What’s new with the amnesic patient H.M.? Nat. Rev. Neurosci. 3 , 153–160 (2002).

Article   PubMed   Google Scholar  

Schubert, T. M. et al. Lack of awareness despite complex visual processing: evidence from event-related potentials in a case of selective metamorphopsia. Proc. Natl Acad. Sci. USA 117 , 16055–16064 (2020).

Behrmann, M. & Plaut, D. C. Bilateral hemispheric processing of words and faces: evidence from word impairments in prosopagnosia and face impairments in pure alexia. Cereb. Cortex 24 , 1102–1118 (2014).

Plaut, D. C. & Behrmann, M. Complementary neural representations for faces and words: a computational exploration. Cogn. Neuropsychol. 28 , 251–275 (2011).

Haxby, J. V. et al. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293 , 2425–2430 (2001).

Hirshorn, E. A. et al. Decoding and disrupting left midfusiform gyrus activity during word reading. Proc. Natl Acad. Sci. USA 113 , 8162–8167 (2016).

Kosakowski, H. L. et al. Selective responses to faces, scenes, and bodies in the ventral visual pathway of infants. Curr. Biol. 32 , 265–274.e5 (2022).

Harlow, J. Passage of an iron rod through the head. Boston Med. Surgical J . https://doi.org/10.1176/jnp.11.2.281 (1848).

Broca, P. Remarks on the seat of the faculty of articulated language, following an observation of aphemia (loss of speech). Bull. Soc. Anat. 6 , 330–357 (1861).

Google Scholar  

Dejerine, J. Contribution A L’étude Anatomo-pathologique Et Clinique Des Différentes Variétés De Cécité Verbale: I. Cécité Verbale Avec Agraphie Ou Troubles Très Marqués De L’écriture; II. Cécité Verbale Pure Avec Intégrité De L’écriture Spontanée Et Sous Dictée (Société de Biologie, 1892).

Liepmann, H. Das Krankheitsbild der Apraxie (“motorischen Asymbolie”) auf Grund eines Falles von einseitiger Apraxie (Fortsetzung). Eur. Neurol. 8 , 102–116 (1900).

Article   Google Scholar  

Basso, A., Spinnler, H., Vallar, G. & Zanobio, M. E. Left hemisphere damage and selective impairment of auditory verbal short-term memory. A case study. Neuropsychologia 20 , 263–274 (1982).

Humphreys, G. W. & Riddoch, M. J. The fractionation of visual agnosia. In Visual Object Processing: A Cognitive Neuropsychological Approach 281–306 (Lawrence Erlbaum, 1987).

Whitworth, A., Webster, J. & Howard, D. A Cognitive Neuropsychological Approach To Assessment And Intervention In Aphasia (Psychology Press, 2014).

Caramazza, A. On drawing inferences about the structure of normal cognitive systems from the analysis of patterns of impaired performance: the case for single-patient studies. Brain Cogn. 5 , 41–66 (1986).

Caramazza, A. & McCloskey, M. The case for single-patient studies. Cogn. Neuropsychol. 5 , 517–527 (1988).

Shallice, T. Cognitive neuropsychology and its vicissitudes: the fate of Caramazza’s axioms. Cogn. Neuropsychol. 32 , 385–411 (2015).

Shallice, T. From Neuropsychology To Mental Structure (Cambridge Univ. Press, 1988).

Coltheart, M. Assumptions and methods in cognitive neuropscyhology. In The Handbook Of Cognitive Neuropsychology: What Deficits Reveal About The Human Mind (ed. Rapp, B.) 3–22 (Psychology Press, 2001).

McCloskey, M. & Chaisilprungraung, T. The value of cognitive neuropsychology: the case of vision research. Cogn. Neuropsychol. 34 , 412–419 (2017).

McCloskey, M. The future of cognitive neuropsychology. In The Handbook Of Cognitive Neuropsychology: What Deficits Reveal About The Human Mind (ed. Rapp, B.) 593–610 (Psychology Press, 2001).

Lashley, K. S. In search of the engram. In Physiological Mechanisms in Animal Behavior 454–482 (Academic Press, 1950).

Squire, L. R. & Wixted, J. T. The cognitive neuroscience of human memory since H.M. Annu. Rev. Neurosci. 34 , 259–288 (2011).

Stone, G. O., Vanhoy, M. & Orden, G. C. V. Perception is a two-way street: feedforward and feedback phonology in visual word recognition. J. Mem. Lang. 36 , 337–359 (1997).

Perfetti, C. A. The psycholinguistics of spelling and reading. In Learning To Spell: Research, Theory, And Practice Across Languages 21–38 (Lawrence Erlbaum, 1997).

Nickels, L. The autocue? self-generated phonemic cues in the treatment of a disorder of reading and naming. Cogn. Neuropsychol. 9 , 155–182 (1992).

Rapp, B., Benzing, L. & Caramazza, A. The autonomy of lexical orthography. Cogn. Neuropsychol. 14 , 71–104 (1997).

Bonin, P., Roux, S. & Barry, C. Translating nonverbal pictures into verbal word names. Understanding lexical access and retrieval. In Past, Present, And Future Contributions Of Cognitive Writing Research To Cognitive Psychology 315–522 (Psychology Press, 2011).

Bonin, P., Fayol, M. & Gombert, J.-E. Role of phonological and orthographic codes in picture naming and writing: an interference paradigm study. Cah. Psychol. Cogn./Current Psychol. Cogn. 16 , 299–324 (1997).

Bonin, P., Fayol, M. & Peereman, R. Masked form priming in writing words from pictures: evidence for direct retrieval of orthographic codes. Acta Psychol. 99 , 311–328 (1998).

Bentin, S., Allison, T., Puce, A., Perez, E. & McCarthy, G. Electrophysiological studies of face perception in humans. J. Cogn. Neurosci. 8 , 551–565 (1996).

Jeffreys, D. A. Evoked potential studies of face and object processing. Vis. Cogn. 3 , 1–38 (1996).

Laganaro, M., Morand, S., Michel, C. M., Spinelli, L. & Schnider, A. ERP correlates of word production before and after stroke in an aphasic patient. J. Cogn. Neurosci. 23 , 374–381 (2011).

Indefrey, P. & Levelt, W. J. M. The spatial and temporal signatures of word production components. Cognition 92 , 101–144 (2004).

Valente, A., Burki, A. & Laganaro, M. ERP correlates of word production predictors in picture naming: a trial by trial multiple regression analysis from stimulus onset to response. Front. Neurosci. 8 , 390 (2014).

Kittredge, A. K., Dell, G. S., Verkuilen, J. & Schwartz, M. F. Where is the effect of frequency in word production? Insights from aphasic picture-naming errors. Cogn. Neuropsychol. 25 , 463–492 (2008).

Domdei, N. et al. Ultra-high contrast retinal display system for single photoreceptor psychophysics. Biomed. Opt. Express 9 , 157 (2018).

Poldrack, R. A. et al. Long-term neural and physiological phenotyping of a single human. Nat. Commun. 6 , 8885 (2015).

Coltheart, M. The assumptions of cognitive neuropsychology: reflections on Caramazza (1984, 1986). Cogn. Neuropsychol. 34 , 397–402 (2017).

Badecker, W. & Caramazza, A. A final brief in the case against agrammatism: the role of theory in the selection of data. Cognition 24 , 277–282 (1986).

Fischer-Baum, S. Making sense of deviance: Identifying dissociating cases within the case series approach. Cogn. Neuropsychol. 30 , 597–617 (2013).

Nickels, L., Howard, D. & Best, W. On the use of different methodologies in cognitive neuropsychology: drink deep and from several sources. Cogn. Neuropsychol. 28 , 475–485 (2011).

Dell, G. S. & Schwartz, M. F. Who’s in and who’s out? Inclusion criteria, model evaluation, and the treatment of exceptions in case series. Cogn. Neuropsychol. 28 , 515–520 (2011).

Schwartz, M. F. & Dell, G. S. Case series investigations in cognitive neuropsychology. Cogn. Neuropsychol. 27 , 477–494 (2010).

Cohen, J. A power primer. Psychol. Bull. 112 , 155–159 (1992).

Martin, R. C. & Allen, C. Case studies in neuropsychology. In APA Handbook Of Research Methods In Psychology Vol. 2 Research Designs: Quantitative, Qualitative, Neuropsychological, And Biological (eds Cooper, H. et al.) 633–646 (American Psychological Association, 2012).

Leivada, E., Westergaard, M., Duñabeitia, J. A. & Rothman, J. On the phantom-like appearance of bilingualism effects on neurocognition: (how) should we proceed? Bilingualism 24 , 197–210 (2021).

Arnett, J. J. The neglected 95%: why American psychology needs to become less American. Am. Psychol. 63 , 602–614 (2008).

Stolz, J. A., Besner, D. & Carr, T. H. Implications of measures of reliability for theories of priming: activity in semantic memory is inherently noisy and uncoordinated. Vis. Cogn. 12 , 284–336 (2005).

Cipora, K. et al. A minority pulls the sample mean: on the individual prevalence of robust group-level cognitive phenomena — the instance of the SNARC effect. Preprint at psyArXiv https://doi.org/10.31234/osf.io/bwyr3 (2019).

Andrews, S., Lo, S. & Xia, V. Individual differences in automatic semantic priming. J. Exp. Psychol. Hum. Percept. Perform. 43 , 1025–1039 (2017).

Tan, L. C. & Yap, M. J. Are individual differences in masked repetition and semantic priming reliable? Vis. Cogn. 24 , 182–200 (2016).

Olsson-Collentine, A., Wicherts, J. M. & van Assen, M. A. L. M. Heterogeneity in direct replications in psychology and its association with effect size. Psychol. Bull. 146 , 922–940 (2020).

Gratton, C. & Braga, R. M. Editorial overview: deep imaging of the individual brain: past, practice, and promise. Curr. Opin. Behav. Sci. 40 , iii–vi (2021).

Fedorenko, E. The early origins and the growing popularity of the individual-subject analytic approach in human neuroscience. Curr. Opin. Behav. Sci. 40 , 105–112 (2021).

Xue, A. et al. The detailed organization of the human cerebellum estimated by intrinsic functional connectivity within the individual. J. Neurophysiol. 125 , 358–384 (2021).

Petit, S. et al. Toward an individualized neural assessment of receptive language in children. J. Speech Lang. Hear. Res. 63 , 2361–2385 (2020).

Jung, K.-H. et al. Heterogeneity of cerebral white matter lesions and clinical correlates in older adults. Stroke 52 , 620–630 (2021).

Falcon, M. I., Jirsa, V. & Solodkin, A. A new neuroinformatics approach to personalized medicine in neurology: the virtual brain. Curr. Opin. Neurol. 29 , 429–436 (2016).

Duncan, G. J., Engel, M., Claessens, A. & Dowsett, C. J. Replication and robustness in developmental research. Dev. Psychol. 50 , 2417–2425 (2014).

Open Science Collaboration. Estimating the reproducibility of psychological science. Science 349 , aac4716 (2015).

Tackett, J. L., Brandes, C. M., King, K. M. & Markon, K. E. Psychology’s replication crisis and clinical psychological science. Annu. Rev. Clin. Psychol. 15 , 579–604 (2019).

Munafò, M. R. et al. A manifesto for reproducible science. Nat. Hum. Behav. 1 , 0021 (2017).

Oldfield, R. C. & Wingfield, A. The time it takes to name an object. Nature 202 , 1031–1032 (1964).

Oldfield, R. C. & Wingfield, A. Response latencies in naming objects. Q. J. Exp. Psychol. 17 , 273–281 (1965).

Brysbaert, M. How many participants do we have to include in properly powered experiments? A tutorial of power analysis with reference tables. J. Cogn. 2 , 16 (2019).

Brysbaert, M. Power considerations in bilingualism research: time to step up our game. Bilingualism https://doi.org/10.1017/S1366728920000437 (2020).

Machery, E. What is a replication? Phil. Sci. 87 , 545–567 (2020).

Nosek, B. A. & Errington, T. M. What is replication? PLoS Biol. 18 , e3000691 (2020).

Li, X., Huang, L., Yao, P. & Hyönä, J. Universal and specific reading mechanisms across different writing systems. Nat. Rev. Psychol. 1 , 133–144 (2022).

Rapp, B. (Ed.) The Handbook Of Cognitive Neuropsychology: What Deficits Reveal About The Human Mind (Psychology Press, 2001).

Code, C. et al. Classic Cases In Neuropsychology (Psychology Press, 1996).

Patterson, K., Marshall, J. C. & Coltheart, M. Surface Dyslexia: Neuropsychological And Cognitive Studies Of Phonological Reading (Routledge, 2017).

Marshall, J. C. & Newcombe, F. Patterns of paralexia: a psycholinguistic approach. J. Psycholinguist. Res. 2 , 175–199 (1973).

Castles, A. & Coltheart, M. Varieties of developmental dyslexia. Cognition 47 , 149–180 (1993).

Khentov-Kraus, L. & Friedmann, N. Vowel letter dyslexia. Cogn. Neuropsychol. 35 , 223–270 (2018).

Winskel, H. Orthographic and phonological parafoveal processing of consonants, vowels, and tones when reading Thai. Appl. Psycholinguist. 32 , 739–759 (2011).

Hepner, C., McCloskey, M. & Rapp, B. Do reading and spelling share orthographic representations? Evidence from developmental dysgraphia. Cogn. Neuropsychol. 34 , 119–143 (2017).

Hanley, J. R. & Sotiropoulos, A. Developmental surface dysgraphia without surface dyslexia. Cogn. Neuropsychol. 35 , 333–341 (2018).

Zihl, J. & Heywood, C. A. The contribution of single case studies to the neuroscience of vision: single case studies in vision neuroscience. Psych. J. 5 , 5–17 (2016).

Bouvier, S. E. & Engel, S. A. Behavioral deficits and cortical damage loci in cerebral achromatopsia. Cereb. Cortex 16 , 183–191 (2006).

Zihl, J. & Heywood, C. A. The contribution of LM to the neuroscience of movement vision. Front. Integr. Neurosci. 9 , 6 (2015).

Dotan, D. & Friedmann, N. Separate mechanisms for number reading and word reading: evidence from selective impairments. Cortex 114 , 176–192 (2019).

McCloskey, M. & Schubert, T. Shared versus separate processes for letter and digit identification. Cogn. Neuropsychol. 31 , 437–460 (2014).

Fayol, M. & Seron, X. On numerical representations. Insights from experimental, neuropsychological, and developmental research. In Handbook of Mathematical Cognition (ed. Campbell, J.) 3–23 (Psychological Press, 2005).

Bornstein, B. & Kidron, D. P. Prosopagnosia. J. Neurol. Neurosurg. Psychiat. 22 , 124–131 (1959).

Kühn, C. D., Gerlach, C., Andersen, K. B., Poulsen, M. & Starrfelt, R. Face recognition in developmental dyslexia: evidence for dissociation between faces and words. Cogn. Neuropsychol. 38 , 107–115 (2021).

Barton, J. J. S., Albonico, A., Susilo, T., Duchaine, B. & Corrow, S. L. Object recognition in acquired and developmental prosopagnosia. Cogn. Neuropsychol. 36 , 54–84 (2019).

Renault, B., Signoret, J.-L., Debruille, B., Breton, F. & Bolgert, F. Brain potentials reveal covert facial recognition in prosopagnosia. Neuropsychologia 27 , 905–912 (1989).

Bauer, R. M. Autonomic recognition of names and faces in prosopagnosia: a neuropsychological application of the guilty knowledge test. Neuropsychologia 22 , 457–469 (1984).

Haan, E. H. F., de, Young, A. & Newcombe, F. Face recognition without awareness. Cogn. Neuropsychol. 4 , 385–415 (1987).

Ellis, H. D. & Lewis, M. B. Capgras delusion: a window on face recognition. Trends Cogn. Sci. 5 , 149–156 (2001).

Ellis, H. D., Young, A. W., Quayle, A. H. & De Pauw, K. W. Reduced autonomic responses to faces in Capgras delusion. Proc. R. Soc. Lond. B 264 , 1085–1092 (1997).

Collins, M. N., Hawthorne, M. E., Gribbin, N. & Jacobson, R. Capgras’ syndrome with organic disorders. Postgrad. Med. J. 66 , 1064–1067 (1990).

Enoch, D., Puri, B. K. & Ball, H. Uncommon Psychiatric Syndromes 5th edn (Routledge, 2020).

Tranel, D., Damasio, H. & Damasio, A. R. Double dissociation between overt and covert face recognition. J. Cogn. Neurosci. 7 , 425–432 (1995).

Brighetti, G., Bonifacci, P., Borlimi, R. & Ottaviani, C. “Far from the heart far from the eye”: evidence from the Capgras delusion. Cogn. Neuropsychiat. 12 , 189–197 (2007).

Coltheart, M., Langdon, R. & McKay, R. Delusional belief. Annu. Rev. Psychol. 62 , 271–298 (2011).

Coltheart, M. Cognitive neuropsychiatry and delusional belief. Q. J. Exp. Psychol. 60 , 1041–1062 (2007).

Coltheart, M. & Davies, M. How unexpected observations lead to new beliefs: a Peircean pathway. Conscious. Cogn. 87 , 103037 (2021).

Coltheart, M. & Davies, M. Failure of hypothesis evaluation as a factor in delusional belief. Cogn. Neuropsychiat. 26 , 213–230 (2021).

McCloskey, M. et al. A developmental deficit in localizing objects from vision. Psychol. Sci. 6 , 112–117 (1995).

McCloskey, M., Valtonen, J. & Cohen Sherman, J. Representing orientation: a coordinate-system hypothesis and evidence from developmental deficits. Cogn. Neuropsychol. 23 , 680–713 (2006).

McCloskey, M. Spatial representations and multiple-visual-systems hypotheses: evidence from a developmental deficit in visual location and orientation processing. Cortex 40 , 677–694 (2004).

Gregory, E. & McCloskey, M. Mirror-image confusions: implications for representation and processing of object orientation. Cognition 116 , 110–129 (2010).

Gregory, E., Landau, B. & McCloskey, M. Representation of object orientation in children: evidence from mirror-image confusions. Vis. Cogn. 19 , 1035–1062 (2011).

Laine, M. & Martin, N. Cognitive neuropsychology has been, is, and will be significant to aphasiology. Aphasiology 26 , 1362–1376 (2012).

Howard, D. & Patterson, K. The Pyramids And Palm Trees Test: A Test Of Semantic Access From Words And Pictures (Thames Valley Test Co., 1992).

Kay, J., Lesser, R. & Coltheart, M. PALPA: Psycholinguistic Assessments Of Language Processing In Aphasia. 2: Picture & Word Semantics, Sentence Comprehension (Erlbaum, 2001).

Franklin, S. Dissociations in auditory word comprehension; evidence from nine fluent aphasic patients. Aphasiology 3 , 189–207 (1989).

Howard, D., Swinburn, K. & Porter, G. Putting the CAT out: what the comprehensive aphasia test has to offer. Aphasiology 24 , 56–74 (2010).

Conti-Ramsden, G., Crutchley, A. & Botting, N. The extent to which psychometric tests differentiate subgroups of children with SLI. J. Speech Lang. Hear. Res. 40 , 765–777 (1997).

Bishop, D. V. M. & McArthur, G. M. Individual differences in auditory processing in specific language impairment: a follow-up study using event-related potentials and behavioural thresholds. Cortex 41 , 327–341 (2005).

Bishop, D. V. M., Snowling, M. J., Thompson, P. A. & Greenhalgh, T., and the CATALISE-2 consortium. Phase 2 of CATALISE: a multinational and multidisciplinary Delphi consensus study of problems with language development: terminology. J. Child. Psychol. Psychiat. 58 , 1068–1080 (2017).

Wilson, A. J. et al. Principles underlying the design of ‘the number race’, an adaptive computer game for remediation of dyscalculia. Behav. Brain Funct. 2 , 19 (2006).

Basso, A. & Marangolo, P. Cognitive neuropsychological rehabilitation: the emperor’s new clothes? Neuropsychol. Rehabil. 10 , 219–229 (2000).

Murad, M. H., Asi, N., Alsawas, M. & Alahdab, F. New evidence pyramid. Evidence-based Med. 21 , 125–127 (2016).

Greenhalgh, T., Howick, J. & Maskrey, N., for the Evidence Based Medicine Renaissance Group. Evidence based medicine: a movement in crisis? Br. Med. J. 348 , g3725–g3725 (2014).

Best, W., Ping Sze, W., Edmundson, A. & Nickels, L. What counts as evidence? Swimming against the tide: valuing both clinically informed experimentally controlled case series and randomized controlled trials in intervention research. Evidence-based Commun. Assess. Interv. 13 , 107–135 (2019).

Best, W. et al. Understanding differing outcomes from semantic and phonological interventions with children with word-finding difficulties: a group and case series study. Cortex 134 , 145–161 (2021).

OCEBM Levels of Evidence Working Group. The Oxford Levels of Evidence 2. CEBM https://www.cebm.ox.ac.uk/resources/levels-of-evidence/ocebm-levels-of-evidence (2011).

Holler, D. E., Behrmann, M. & Snow, J. C. Real-world size coding of solid objects, but not 2-D or 3-D images, in visual agnosia patients with bilateral ventral lesions. Cortex 119 , 555–568 (2019).

Duchaine, B. C., Yovel, G., Butterworth, E. J. & Nakayama, K. Prosopagnosia as an impairment to face-specific mechanisms: elimination of the alternative hypotheses in a developmental case. Cogn. Neuropsychol. 23 , 714–747 (2006).

Hartley, T. et al. The hippocampus is required for short-term topographical memory in humans. Hippocampus 17 , 34–48 (2007).

Pishnamazi, M. et al. Attentional bias towards and away from fearful faces is modulated by developmental amygdala damage. Cortex 81 , 24–34 (2016).

Rapp, B., Fischer-Baum, S. & Miozzo, M. Modality and morphology: what we write may not be what we say. Psychol. Sci. 26 , 892–902 (2015).

Yong, K. X. X., Warren, J. D., Warrington, E. K. & Crutch, S. J. Intact reading in patients with profound early visual dysfunction. Cortex 49 , 2294–2306 (2013).

Rockland, K. S. & Van Hoesen, G. W. Direct temporal–occipital feedback connections to striate cortex (V1) in the macaque monkey. Cereb. Cortex 4 , 300–313 (1994).

Haynes, J.-D., Driver, J. & Rees, G. Visibility reflects dynamic changes of effective connectivity between V1 and fusiform cortex. Neuron 46 , 811–821 (2005).

Tanaka, K. Mechanisms of visual object recognition: monkey and human studies. Curr. Opin. Neurobiol. 7 , 523–529 (1997).

Fischer-Baum, S., McCloskey, M. & Rapp, B. Representation of letter position in spelling: evidence from acquired dysgraphia. Cognition 115 , 466–490 (2010).

Houghton, G. The problem of serial order: a neural network model of sequence learning and recall. In Current Research In Natural Language Generation (eds Dale, R., Mellish, C. & Zock, M.) 287–319 (Academic Press, 1990).

Fieder, N., Nickels, L., Biedermann, B. & Best, W. From “some butter” to “a butter”: an investigation of mass and count representation and processing. Cogn. Neuropsychol. 31 , 313–349 (2014).

Fieder, N., Nickels, L., Biedermann, B. & Best, W. How ‘some garlic’ becomes ‘a garlic’ or ‘some onion’: mass and count processing in aphasia. Neuropsychologia 75 , 626–645 (2015).

Schröder, A., Burchert, F. & Stadie, N. Training-induced improvement of noncanonical sentence production does not generalize to comprehension: evidence for modality-specific processes. Cogn. Neuropsychol. 32 , 195–220 (2015).

Stadie, N. et al. Unambiguous generalization effects after treatment of non-canonical sentence production in German agrammatism. Brain Lang. 104 , 211–229 (2008).

Schapiro, A. C., Gregory, E., Landau, B., McCloskey, M. & Turk-Browne, N. B. The necessity of the medial temporal lobe for statistical learning. J. Cogn. Neurosci. 26 , 1736–1747 (2014).

Schapiro, A. C., Kustner, L. V. & Turk-Browne, N. B. Shaping of object representations in the human medial temporal lobe based on temporal regularities. Curr. Biol. 22 , 1622–1627 (2012).

Baddeley, A., Vargha-Khadem, F. & Mishkin, M. Preserved recognition in a case of developmental amnesia: implications for the acaquisition of semantic memory? J. Cogn. Neurosci. 13 , 357–369 (2001).

Snyder, J. J. & Chatterjee, A. Spatial-temporal anisometries following right parietal damage. Neuropsychologia 42 , 1703–1708 (2004).

Ashkenazi, S., Henik, A., Ifergane, G. & Shelef, I. Basic numerical processing in left intraparietal sulcus (IPS) acalculia. Cortex 44 , 439–448 (2008).

Lebrun, M.-A., Moreau, P., McNally-Gagnon, A., Mignault Goulet, G. & Peretz, I. Congenital amusia in childhood: a case study. Cortex 48 , 683–688 (2012).

Vannuscorps, G., Andres, M. & Pillon, A. When does action comprehension need motor involvement? Evidence from upper limb aplasia. Cogn. Neuropsychol. 30 , 253–283 (2013).

Jeannerod, M. Neural simulation of action: a unifying mechanism for motor cognition. NeuroImage 14 , S103–S109 (2001).

Blakemore, S.-J. & Decety, J. From the perception of action to the understanding of intention. Nat. Rev. Neurosci. 2 , 561–567 (2001).

Rizzolatti, G. & Craighero, L. The mirror-neuron system. Annu. Rev. Neurosci. 27 , 169–192 (2004).

Forde, E. M. E., Humphreys, G. W. & Remoundou, M. Disordered knowledge of action order in action disorganisation syndrome. Neurocase 10 , 19–28 (2004).

Mazzi, C. & Savazzi, S. The glamor of old-style single-case studies in the neuroimaging era: insights from a patient with hemianopia. Front. Psychol. 10 , 965 (2019).

Coltheart, M. What has functional neuroimaging told us about the mind (so far)? (Position Paper Presented to the European Cognitive Neuropsychology Workshop, Bressanone, 2005). Cortex 42 , 323–331 (2006).

Page, M. P. A. What can’t functional neuroimaging tell the cognitive psychologist? Cortex 42 , 428–443 (2006).

Blank, I. A., Kiran, S. & Fedorenko, E. Can neuroimaging help aphasia researchers? Addressing generalizability, variability, and interpretability. Cogn. Neuropsychol. 34 , 377–393 (2017).

Niv, Y. The primacy of behavioral research for understanding the brain. Behav. Neurosci. 135 , 601–609 (2021).

Crawford, J. R. & Howell, D. C. Comparing an individual’s test score against norms derived from small samples. Clin. Neuropsychol. 12 , 482–486 (1998).

Crawford, J. R., Garthwaite, P. H. & Ryan, K. Comparing a single case to a control sample: testing for neuropsychological deficits and dissociations in the presence of covariates. Cortex 47 , 1166–1178 (2011).

McIntosh, R. D. & Rittmo, J. Ö. Power calculations in single-case neuropsychology: a practical primer. Cortex 135 , 146–158 (2021).

Patterson, K. & Plaut, D. C. “Shallow draughts intoxicate the brain”: lessons from cognitive science for cognitive neuropsychology. Top. Cogn. Sci. 1 , 39–58 (2009).

Lambon Ralph, M. A., Patterson, K. & Plaut, D. C. Finite case series or infinite single-case studies? Comments on “Case series investigations in cognitive neuropsychology” by Schwartz and Dell (2010). Cogn. Neuropsychol. 28 , 466–474 (2011).

Horien, C., Shen, X., Scheinost, D. & Constable, R. T. The individual functional connectome is unique and stable over months to years. NeuroImage 189 , 676–687 (2019).

Epelbaum, S. et al. Pure alexia as a disconnection syndrome: new diffusion imaging evidence for an old concept. Cortex 44 , 962–974 (2008).

Fischer-Baum, S. & Campana, G. Neuroplasticity and the logic of cognitive neuropsychology. Cogn. Neuropsychol. 34 , 403–411 (2017).

Paul, S., Baca, E. & Fischer-Baum, S. Cerebellar contributions to orthographic working memory: a single case cognitive neuropsychological investigation. Neuropsychologia 171 , 108242 (2022).

Feinstein, J. S., Adolphs, R., Damasio, A. & Tranel, D. The human amygdala and the induction and experience of fear. Curr. Biol. 21 , 34–38 (2011).

Crawford, J., Garthwaite, P. & Gray, C. Wanted: fully operational definitions of dissociations in single-case studies. Cortex 39 , 357–370 (2003).

McIntosh, R. D. Simple dissociations for a higher-powered neuropsychology. Cortex 103 , 256–265 (2018).

McIntosh, R. D. & Brooks, J. L. Current tests and trends in single-case neuropsychology. Cortex 47 , 1151–1159 (2011).

Best, W., Schröder, A. & Herbert, R. An investigation of a relative impairment in naming non-living items: theoretical and methodological implications. J. Neurolinguistics 19 , 96–123 (2006).

Franklin, S., Howard, D. & Patterson, K. Abstract word anomia. Cogn. Neuropsychol. 12 , 549–566 (1995).

Coltheart, M., Patterson, K. E. & Marshall, J. C. Deep Dyslexia (Routledge, 1980).

Nickels, L., Kohnen, S. & Biedermann, B. An untapped resource: treatment as a tool for revealing the nature of cognitive processes. Cogn. Neuropsychol. 27 , 539–562 (2010).

Download references

Acknowledgements

The authors thank all of those pioneers of and advocates for single case study research who have mentored, inspired and encouraged us over the years, and the many other colleagues with whom we have discussed these issues.

Author information

Authors and affiliations.

School of Psychological Sciences & Macquarie University Centre for Reading, Macquarie University, Sydney, New South Wales, Australia

Lyndsey Nickels

NHMRC Centre of Research Excellence in Aphasia Recovery and Rehabilitation, Australia

Psychological Sciences, Rice University, Houston, TX, USA

Simon Fischer-Baum

Psychology and Language Sciences, University College London, London, UK

You can also search for this author in PubMed   Google Scholar

Contributions

L.N. led and was primarily responsible for the structuring and writing of the manuscript. All authors contributed to all aspects of the article.

Corresponding author

Correspondence to Lyndsey Nickels .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Reviews Psychology thanks Yanchao Bi, Rob McIntosh, and the other, anonymous, reviewer for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Nickels, L., Fischer-Baum, S. & Best, W. Single case studies are a powerful tool for developing, testing and extending theories. Nat Rev Psychol 1 , 733–747 (2022). https://doi.org/10.1038/s44159-022-00127-y

Download citation

Accepted : 13 October 2022

Published : 22 November 2022

Issue Date : December 2022

DOI : https://doi.org/10.1038/s44159-022-00127-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Transl Behav Med
• v.4(3); 2014 Sep

Optimizing behavioral health interventions with single-case designs: from development to dissemination

Jesse dallery.

Department of Psychology, University of Florida, P. O. box 112250, Gainesville, FL 32611 USA

Bethany R Raiff

Department of Psychology, Rowan University, Glassboro, USA

Over the past 70 years, single-case design (SCD) research has evolved to include a broad array of methodological and analytic advances. In this article, we describe some of these advances and discuss how SCDs can be used to optimize behavioral health interventions. Specifically, we discuss how parametric analysis, component analysis, and systematic replications can be used to optimize interventions. We also describe how SCDs can address other features of optimization, which include establishing generality and enabling personalized behavioral medicine. Throughout, we highlight how SCDs can be used during both the development and dissemination stages of behavioral health interventions.

Research methods are tools to discover new phenomena, test theories, and evaluate interventions. Many researchers have argued that our research tools have become limited, particularly in the domain of behavioral health interventions [ 1 – 9 ]. The reasons for their arguments vary, but include an overreliance on randomized controlled trials, the slow pace and high cost of such trials, and the lack of attention to individual differences. In addition, advances in mobile and sensor-based data collection now permit real-time, continuous observation of behavior and symptoms over extended durations [ 3 , 10 , 11 ]. Such fine-grained observation can lead to tailoring of treatment based on changes in behavior, which is challenging to evaluate with traditional methods such as a randomized trial.

In light of the limitations of traditional designs and advances in data collection methods, a growing number of researchers have advocated for alternative research designs [ 2 , 7 , 10 ]. Specifically, one family of research designs, known as single-case designs (SCDs), has been proposed as a useful way to establish the preliminary efficacy of health interventions [ 3 ]. In the present article, we recapitulate and expand on this proposal, and argue that they can be used to optimize health interventions.

We begin with a description of what we consider to be a set of criteria, or ideals, for what research designs should accomplish in attempting to optimize an intervention. Admittedly, these criteria are self-serving in the sense that most of them constitute the strengths of SCDs, but they also apply to other research designs discussed in this volume. Next, we introduce SCDs and how they can be used to optimize treatment using parametric and component analyses. We also describe how SCDs can address other features of optimization, which include establishing generality and enabling personalized behavioral medicine. Throughout, we also highlight how these designs can be used during both the development and dissemination of behavioral health interventions. Finally, we evaluate the extent to which SCDs live up to our ideals.

AN OPTIMIZATION IDEAL

During development and testing of a new intervention, our methods should be efficient, flexible, and rigorous. We would like efficient methods to help us establish preliminary efficacy, or “clinically significant patient improvement over the course of treatment” [ 12 ] (p. 137). We also need flexible methods to test different parameters or components of an intervention. Just as different doses of a drug treatment may need to be titrated to optimize effects, different parameters or components of a behavioral treatment may need to be titrated to optimize effects. It should go without saying that we also want our methods to be rigorous, and therefore eliminate or reduce threats to internal validity.

Also, during development, we would like methods that allow us to assess replications of effects to establish the reliability and generality of an intervention. Replications, if done systematically and thoughtfully, can answer questions about for whom and under what conditions an intervention is effective. Answering these questions speaks to the generality of research findings. As Cohen [ 13 ] noted in a seminal article: “For generalization, psychologists must finally rely, as has been done in all the older sciences, on replication” (p. 997). Relying on replications and establishing the conditions under which an intervention works could also lead to more targeted, efficient dissemination efforts.

During dissemination, when an intervention is implemented in clinical practice, we again would like to know if the intervention is producing a reliable change in behavior for a particular individual. (Here, “we” may refer to practitioners in addition to researchers.) With knowledge derived from development and efficacy testing, we may be able to alter components of an intervention that impact its effectiveness. But, ideally, we would like to not only alter but verify whether these components are working. Also, recognizing that behavior change is idiosyncratic and dynamic, we may need methods that allow ongoing tailoring and testing. This may result in a kind of personalized behavioral medicine in which what gets personalized, and when, is determined through experimental analysis.

In addition, during both development and dissemination, we want methods that afford innovation. We should have methods that allow rapid, rigorous testing of new treatments, and which permit incorporating new technologies to assess and treat behavior as they become available. This might be thought of as systematic play. Whatever we call it, it is a hallmark of the experimental attitude in science.

INTRODUCTION TO SINGLE-CASE DESIGNS

SCDs include an array of methods in which each participant, or case, serves as his or her own control. Although these methods are conceptually rooted in the study of cognition and behavior [ 14 ], they are theory-neutral and can be applied to any health intervention. In a typical study, some behavior or symptom is measured repeatedly during all conditions for all participants. The experimenter systematically introduces and withdraws control and intervention conditions, and assesses effects of the intervention on behavior across replications of these conditions within and across participants. Thus, these studies include repeated, frequent assessment of behavior, experimental manipulation of the independent variable (the intervention or components of the intervention), and replication of effects within and across participants.

The main challenge in conducting a single-case experiment is collecting data of the same behavior or symptom repeatedly over time. In other words, a time series must be possible. If behavior or symptoms cannot be assessed frequently, then SCDs cannot be used (e.g., on a weekly basis, at a minimum, for most health interventions). Fortunately, technology is revolutionizing methods to collect data. For example, ecological momentary assessment (EMA) enables frequent input by an end-user into a handheld computer or mobile phone [ 15 ]. Such input occurs in naturalistic settings, and it usually occurs on a daily basis for several weeks to months. EMA can therefore reveal behavioral variation over time and across contexts, and it can document effects of an intervention on an individual’s behavior [ 15 ]. Sensors to record physical activity, medication adherence, and recent drug use also enable the kind of assessment required for single-case research [ 10 , 16 ]. In addition, advances in information technology and mobile phones can permit frequent assessment of behavior or symptoms [ 17 , 18 ]. Thus, SCDs can capitalize on the ability of technology to easily, unobtrusively, and repeatedly assess health behavior [ 3 , 18 , 19 ].

SCDs suffer from several misconceptions that may limit their use [ 20 – 23 ]. First, a single case does not mean “ n of 1.” The number of participants in a typical study is almost always more than 1, usually around 6 but sometimes as many as 20, 40, or more participants [ 24 , 25 ]. Also, the unit of analysis, or “case,” could be individual participants, clinics, group homes, hospitals, health care agencies, or communities [ 1 ]. Given that the unit of analysis is each case (i.e., participant), a single study could be conceptualized as a series of single-case experiments. Perhaps a better label for these designs would be “intrasubject replication designs” [ 26 ]. Second, SCDs are not limited to interventions that produce large, immediate changes in behavior. They can be used to detect small but meaningful changes in behavior and to assess behavior that may change slowly over time (e.g., learning a new skill) [ 27 ]. Third, SCDs are not quasi-experimental designs [ 20 ]. The conventional notions that detecting causal relations requires random assignment and/or random sampling are false [ 26 ]. Single-case experiments are fully experimental and include controls and replications to permit crisp statements about causal relations between independent and dependent variables.

VARIETIES OF SINGLE-CASE DESIGNS

The most relevant SCDs to behavioral health interventions are presented in Table  1 . The table also presents some procedural information and advantages and disadvantages for each design. (The material below is adapted from [ 3 ]) There are also a number of variants of these designs, enabling flexibility in tailoring the design based on practical or empirical considerations [ 27 , 28 ]. For example, there are several variants to circumvent long periods of assessing behavior during baseline conditions, which may be problematic if the behavior is dangerous, before introducing a potentially effective intervention [ 28 ].

Several single-case designs, including general procedures, advantages, and disadvantages

Procedural controls must be in place to make inferences about causal relations, such as clear, operational definitions of the dependent variables, reliable and valid techniques to assess the behavior, and the experimental design must be sufficient to rule out alternative hypotheses for the behavior change. Table  2 presents a summary of methodological and assessment standards to permit conclusions about treatment effects [ 29 , 30 ]. These standards were derived from Horner et al. [ 29 ] and from the recently released What Works Clearinghouse (WWC) pilot standards for evaluating single-case research to inform policy and practice (hereafter referred to as the SCD standards) [ 31 ].

Quality indicators for single-case research [ 29 ]

All of the designs listed in Table  1 entail a baseline period of observation. During this period, the dependent variable is measured repeatedly under control conditions. For example, Dallery, Glenn, and Raiff [ 24 ] used a reversal design to assess effects of an internet-based incentive program to promote smoking cessation, and the baseline phase included self-monitoring, carbon monoxide assessment of smoking status via a web camera, and monetary incentives for submitting videos. The active ingredient in the intervention, incentives contingent on objectively verified smoking abstinence, was not introduced until the treatment phase.

The duration of the baseline and the pattern of the data should be sufficient to predict future behavior. That is, the level of the dependent variable should be stable enough to predict its direction if the treatment was not introduced. If there is a trend in the direction of the anticipated treatment effect during baseline, or if there is too much variability, the ability to detect a treatment effect will be compromised. Thus, stability, or in some cases a trend in the direction opposite the predicted treatment effect, is desirable during baseline conditions.

In some cases, the source(s) of variability can be identified and potentially mitigated (e.g., variability could be reduced by automating data collection, standardizing the setting and time for data collection). However, there may be instances when there is too much variability during baseline conditions, and thus, detecting a treatment effect will not be feasible. There are no absolute standards to define what “too much” variability means [ 27 ]. Excessive variability is a relative term, which is typically determined by a comparison of performance within and between conditions (e.g., between baseline and intervention conditions) in a single-case experiment. The mere presence of variability does not mean that a single-case approach should be abandoned, however. Indeed, identifying the sources of variability and/or assessing new measurement strategies can be evaluated using SCDs. Under these conditions, the outcome of interest is not an increase or a decrease in some behavior or symptom but a reduction in variability. Once accomplished, the researcher has not only learned something useful but is also better prepared to evaluate the effects of an intervention to increase or decrease some health behavior.

REVERSAL DESIGNS

In a reversal design, a treatment is introduced after the baseline period, and then a baseline period is re-introduced, hence, the “reversal” in this design (also known as an ABA design, where “A” is baseline and “B” is treatment). Using only two conditions, such as a pre-post design, is not considered sufficient to demonstrate experimental control because other sources of influence on behavior cannot be ruled out [ 31 , 32 ]. For example, a smoking cessation intervention could coincide with a price increase in cigarettes. By returning to baseline conditions, we could assess and possibly rule out the influence of the price increase on smoking. Researchers also often use a reversal to the treatment condition. Thus, the experiment ends during a treatment period (an ABAB design). Not only is this desirable from the participant’s perspective but it also provides a replication of the main variable of interest—the treatment [ 33 ].

Figure  1 displays an idealized, ABAB reversal design, and each panel shows data from a different participant. Although all participants were exposed to the same four conditions, the duration of the conditions differed because of trends in the conditions. For example, for participant 1, the beginning of the first baseline condition displays a consistent downward trend (in the same direction as the expected text-message treatment effects). If we were to introduce the smoking cessation-related texts after only five or six baseline sessions, it would be unclear if the decrease in smoking was a function of the independent variable. Therefore, continuing the baseline condition until there is no visible trend helps build our confidence about the causal role of the treatment when it is introduced. The immediate decrease in the level of smoking for participant 1 when the treatment is introduced also implicates the treatment. We can also detect, however, an increasing trend in the early portion of the treatment condition. Thus, we need to continue the treatment condition until there is no undesirable trend before returning to the baseline condition. Similar patterns can be seen for participants 2–4. Based on visual analysis of Fig.  1 , we would conclude that treatment is exerting a reliable effect on smoking. But, the meaningfulness of this effect requires additional considerations (see the section below on “ Visual, Statistical, and Social Validity Analysis ”).

Example of a reversal design showing experimental control and replications within and between subjects. Each panel represents a different participant, each of whom experienced two baseline and two treatment conditions

Studies using reversal designs typically include at least four or more participants. The goal is to generate enough replications, both within participants and across participants, to permit a confident statement about causal relations. For example, several studies on incentive-based treatment to promote drug abstinence have used 20 participants in a reversal design [ 24 , 25 ]. According to the SCD standards, there must be a minimum of three replications to support conclusions about experimental control and thus causation. Also, according to the SCD standards, there must be at least three and preferably five data points per phase to allow the researcher to evaluate stability and experimental effects [ 31 ].

There are two potential limitations of reversal designs in the context of behavioral health interventions. First, the treatment must be withdrawn to demonstrate causal relations. Some have raised an ethical objection about this practice [ 11 ]. However, we think that the benefits of demonstrating that a treatment works outweigh the risks of temporarily withdrawing treatment (in most cases). The treatment can also be re-instituted in a reversal design (i.e., an ABAB design). Second, if the intervention produces relatively permanent changes in behavior, then a reversal to pre-intervention conditions may not be possible. For example, a treatment that develops new skills may imply that these skills cannot be “reversed.” Some interventions do not produce permanent change and must remain in effect for behavior change to be maintained, such as some medications and incentive-based procedures. Under conditions where behavior may not return to baseline levels when treatment is withdrawn, alternative designs, such as multiple-baseline designs, should be used.

MULTIPLE-BASELINE DESIGNS

In a multiple-baseline design, the durations of the baselines vary systematically for each participant in a so-called staggered fashion. For example, one participant may start treatment after five baseline days, another after seven baseline days, then nine, and so on. After baseline, treatment is introduced, and it remains until the end of the experiment (i.e., there are no reversals). Like all SCDs, this design can be applied to individual participants, clusters of individuals, health care agencies, and communities. These designs are also referred to as interrupted time-series designs [ 1 ] and stepped wedge designs [ 7 ].

The utility of these designs is derived from demonstrating that change occurs when, and only when, the intervention is directed at a particular participant (or whatever the unit of analysis happens to be [ 28 ]). The influence of other factors, such as idiosyncratic experiences of the individual or self-monitoring (e.g., reactivity), can be ruled out by replicating the effect across multiple individuals. A key to ruling out extraneous factors is a stable enough baseline phase (either no trends or a trend in the opposite direction to the treatment effect). As replications are observed across individuals, and behavior changes when and only when treatment is introduced, confidence that behavior change was caused by the treatment increases.

As noted above, multiple-baseline designs are useful for interventions that teach new skills, where behavior would not be expected to “reverse” to baseline levels. Multiple-baseline designs also obviate the ethical concern about withdrawing treatment (as in a reversal design) or using a placebo control comparison group (as in randomized trials), as all participants are exposed to the treatment with multiple-baseline designs.

Figure  2 illustrates a simple, two-condition multiple-baseline design replicated across four participants. As noted above, the experimenter should introduce treatment only when the data appear stable during baseline conditions. The durations of the baseline conditions are staggered for each participant, and the dependent variable increases when, and only when, the independent variable is introduced for all participants. The SCD standards requires at least six phases (i.e., three baseline and three treatment) with at least five data points per phase [ 31 ]. Figure  2 suggests reliable increases in behavior and that the treatment was responsible for these changes.

Example of a multiple-baseline design showing experimental control and replications between subjects. Each row represents a different participant, each of whom experienced a baseline and treatment. The baseline durations differed across participants

CHANGING CRITERION DESIGN

The changing criterion design is also relevant to optimizing interventions [ 34 ]. In a changing criterion design, a baseline is conducted until stability is attained. Then, a treatment goal is introduced, and goals are made progressively more difficult. Behavior should track the introduction of each goal, thus demonstrating control by the level of the independent variable [ 28 ]. For example, Kurti and Dallery [ 35 ] used a changing criterion design to increase activity in six sedentary adults using an internet-based contingency management program to promote walking. Weekly step count goals were gradually increased across 5-day blocks. The step counts for all six participants increased reliably with each increase in the goals, thereby demonstrating experimental control of the intervention. This design has many of the same benefits of the multiple-baseline design, namely that a reversal is not required for ethical or potentially practical reasons (i.e., irreversible treatment effects).

VISUAL, STATISTICAL, AND SOCIAL VALIDITY ANALYSIS

Analyzing the data from SCDs involves three questions: (a) Is there a reliable effect of the intervention? (b) What is the magnitude of the effect? and (c) Are the results clinically meaningful and socially valid [ 31 ]? Social validity refers to the extent to which the goals, procedures, and results of an intervention are socially acceptable to the client, the researcher or health care practitioner, and society [ 36 – 39 ]. The first two questions can be answered by visual and statistical analysis, whereas the third question requires additional considerations.

The SCD standards prioritizes visual analysis of the time-series data to assess the reliability and magnitude of intervention effects [ 29 , 31 , 40 ]. Clinically significant change in patient behavior should be visible. Visual analysis prioritizes clinically significant change in health-related behavior as opposed to statistically significant change in group behavior [ 13 , 41 , 42 ]. Although several researchers have argued that visual analysis may be prone to elevated rates of type 1 error, such errors may be limited to a narrow range of conditions (e.g., when graphs do not contain contextual information about the nature of the plotted behavioral data) [ 27 , 43 ]. Furthermore, in recent years, training in visual analysis has become more formalized and rigorous [ 44 ]. Perhaps as a result, Kahng and colleagues found high reliability among visual analysts in judging treatment effects based on analysis of 36 ABAB graphs [ 45 ]. The SCD standards recommends four steps and the evaluation of six features of the graphical displays for all participants in a study, which are displayed in Table  3 [ 31 ]. As the visual analyst progresses through the steps, he or she also uses the six features to evaluate effects within and across experimental phases.

Four steps and six outcome measures to evaluate when conducting visual analysis of time-series data

In addition to visual analysis, several regression-based approaches are available to analyze time-series data, such as autoregressive models, robust regression, and hierarchical linear modeling (HLM) [ 46 – 49 ]. A variety of non-parametric statistics are also available [ 27 ]. Perhaps because of the proliferation of statistical methods, there is a lack of consensus about which methods are most appropriate in light of different properties of the data (e.g., the presence of trends and autocorrelation [ 43 , 50 ], the number of data points collected, etc.). A discussion of statistical techniques is beyond the scope of this paper. We recommend Kazdin’s [ 27 ] or Barlow and colleague’s [ 28 ] textbooks as useful resources regarding statistical analysis of time-series data. The SCD standards also includes a useful discussion of statistical approaches for data analysis [ 31 ].

A variety of effect size calculations have been proposed for SCDs [ 13 , 51 – 54 ]. Although effect size estimates may allow for rank ordering of most to least effective treatments [ 55 ], most estimates do not provide metrics that are comparable to effect sizes derived from group designs [ 31 ]. However, one estimate that provides metrics comparable to group designs has been developed and tested by Shadish and colleagues [ 56 , 57 ]. They describe a standardized mean difference statistic ( d ) that is equivalent to the more conventional d in between-groups experiments. The d statistic can also be used to compute power based on the number of observations in each condition and the number of cases in an experiment [ 57 ]. In addition, advances in effect size estimates has led to several meta-analyses of results from SCDs [ 48 , 58 – 61 ]. Zucker and associates [ 62 ] explored Bayesian mixed-model strategy to combining SCDs using, which allowed population-level claims about the merits of different intervention strategies.

Determining whether the results are clinically meaningful and socially valid can be informed by visual and most forms of statistical analysis (i.e., not null-hypothesis significance testing) [ 42 , 63 ]. One element in judging social validity concerns the clinical meaningfulness of the magnitude of behavior change. This judgment can be made by the researcher or clinician in light of knowledge of the subject matter, and perhaps by the client being treated. Depending on factors such as the type of behavior and the way in which change is measured, the judgment can also be informed by previous research on a minimal clinically important difference (MCID) for the behavior or symptom under study [ 64 , 65 ]. The procedures used to generate the effect also require consideration. Intrusive procedures may be efficacious yet not acceptable. The social validity of results and procedures should be explicitly assessed when conducting SCD research, and a variety of tools have emerged to facilitate such efforts [ 37 ]. Social validity assessment should also be viewed as a process [ 37 ]. That is, it can and should be assessed at various time points as an intervention is developed, refined, and eventually implemented. Social validity may change as the procedures and results of an intervention are improved and better appreciated in the society at large.

OPTIMIZATION METHODS AND SINGLE-CASE DESIGNS

The SCDs described above provide an efficient way to evaluate the effects of a behavioral intervention. However, in most of the examples above, the interventions were held constant during treatment periods; that is, they were procedurally static (cf. [ 35 ]). This is similar to a randomized trial, in which all components of an intervention are delivered all at once and held constant throughout the study. However, the major difference between the examples above and traditional randomized trials is efficiency: SCDs usually require less time and fewer resources to demonstrate that an intervention can change behavior. Nevertheless, a single, procedurally static single-case experiment does not optimize treatment beyond showing whether or not it works.

One way to make initial efficacy testing more dynamic would be to conduct a series of single-case experiments in which aspects of the treatment are systematically explored. For example, a researcher could assess effects of different frequencies, timings, or tailoring dimensions of a text-based intervention to promote physical activity. Such manipulation could also be conducted in separate experiments conducted by the same or different researchers. Some experiments may reveal larger effects than others, which could then lead to further replications of the effects of the more promising intervention elements. This iterative development process, with a focus on systematic manipulation of treatment elements and replications of effects within and across experiments, could lead to an improved intervention within a few years’ time. Arguably, this process could yield more clinically useful information than a procedurally static randomized trial conducted over the same period [ 5 , 17 ].

To further increase the efficiency of optimizing treatment, different components or parameters of an intervention can be systematically evaluated within and across single-case experiments. There are two ways to optimize treatment using these methods: parametric and component analyses.

PARAMETRIC ANALYSIS

Parametric analysis involves exposing participants to a range of values of the independent variable, as opposed to just one or two values. To qualify as a parametric analysis, three is the minimum number of values that must be evaluated, as this number is the minimum to evaluate the function form relating the independent to the dependent variable. One goal of a parametric analysis is to identify the optimal value that produces a behavioral outcome. Another goal is to identify general patterns of behavior engendered by a range of values of the independent variable [ 26 , 63 ].

Many behavioral health interventions can be delivered at different levels [ 66 ] and are therefore amenable to parametric analysis. For example, text-based prompts can be delivered at different frequencies, incentives can be delivered at different magnitudes and frequencies, physical activity can occur at different frequencies and intensities, engagement in a web-based program can occur at different levels, medications can be administered at different doses and frequencies, and all of the interventions could be delivered for different durations.

The repeated measures, and resulting time-series data, that are inherent to all SCDs (e.g., reversal and multiple-baseline designs) make them useful designs to conduct parametric analyses. For example, two doses of a medication, low versus high, labeled B and C, respectively, could be assessed using a reversal design [ 67 ]. There may be several possible sequences to conduct the assessment such as ABCBCA or ABCABCA. If C is found to be more effective of the two, it might behoove the researcher to replicate this condition using an ABCBCAC design. A multiple baseline across participants could also be conducted to assess the two doses, one dose for each participant, but this approach may be complicated by individual variability in medication effects. Instead, the multiple-baseline approach could be used on a within-subject basis, where the durations of not just the baselines but of the different dose conditions are varied across participants [ 68 ].

Guyatt and colleagues [ 5 ] provide an excellent discussion about how parametric analysis can be used to optimize an intervention. The intervention was amitriptyline for the treatment of fibrositis. The logic and implications of the research tactics, however, also apply to other interventions that have parametric dimensions. At the time that the research was conducted, a dose of 50 mg/day was the standard recommendation for patients. To determine whether this dose was optimal for a given individual, the researchers first exposed participants to low doses, and if no response was noted relative to placebo, then they systematically increased the dose until a response was observed, or until they reached the maximum of 50 mg/day. In general, their method involved a reversal design in which successively higher doses alternated with placebo. So, for example, if one participant did not respond to a low dose, then doses might be increased to generate an ABCD design, where each successive letter represents a higher dose (other sequences were arranged as well). Parametrically examining doses in this way, and examining individual subject data, the researchers found that some participants responded favorably at lower doses than 50 mg/day (e.g., 10 or 20 mg/day). This was an important finding because the higher doses often produced unwanted side effects. Once optimal doses were identified for individuals, the researchers were able to conduct further analyses using a reversal design, exposing them to either their optimal dose or placebo on different days.

Guyatt and colleagues also investigated the minimum duration of treatment necessary to detect an effect [ 5 ]. Initially, all participants were exposed to the medication for 4 weeks. Visual analysis of the time-series data revealed that medication effects were apparent within about 1–2 weeks of exposure, making a 4-week trial unnecessary. This discovery was replicated in a number of subjects and led them to optimize future, larger studies by only conducting a 2-week intervention. Investigating different treatment durations, such as this, is also a parametric analysis.

Parametric analysis can detect effects that may be missed using a standard group design with only one or two values of the independent variable. For example, in the studies conducted by Guyatt and colleagues [ 5 ], if only the lowest dose of amitriptyline had been investigated using a group approach, the researchers may have incorrectly concluded that the intervention was ineffective because this dose only worked for some individuals. Likewise, if only the highest dose had been investigated, it may have been shown to be effective, but potentially more individuals would have experienced unnecessary side effects (i.e., the results would have low social validity for these individuals). Perhaps most importantly, in contrast to what is typically measured in a group design (e.g., means, confidence intervals, etc.), optimizing treatment effects is fundamentally a question about an individual ’ s behavior.

COMPONENT ANALYSIS

A component analysis is “any experiment designed to identify the active elements of a treatment condition, the relative contributions of different variables in a treatment package, and/or the necessary and sufficient components of an intervention” [ 69 ]. Behavioral health interventions often entail more than one potentially active treatment element. Determining the active elements may be important to increase dissemination potential and decrease cost. Single-case research designs, in particular the reversal and multiple-baseline designs, may be used to perform a component analysis. The essential experimental ingredients, regardless of the method, are that the independent variable(s) are systematically introduced and/or withdrawn, combined with replication of effects within and/or between subjects.

There are two main variants of component analyses: the dropout and add-in analyses. In a dropout analysis, the full treatment package is presented following a baseline phase and then components are systematically withdrawn from the package. A limitation of dropout analyses is when components produce irreversible behavior change (i.e., learning a new skill). Given that most interventions seek to produce sustained changes in health-related behavior, dropout analyses may have limited applicability. Instead, in add-in analyses, components can be assessed individually and/or in combination before the full treatment package is assessed [ 69 ]. Thus, a researcher could conduct an ABACAD design, where A is baseline, B and C are the individual components, and D is the combination of the two B and C components. Other sequences are also possible, and which one is selected will require careful consideration. For example, sequence effects should be considered, and researchers could address these effects through counterbalancing, brief “washout” periods, or explicit investigation of these effects [ 26 ]. If sequence effects cannot be avoided, combined SCD and group designs can be used to perform a component analysis. Thus, different components of a treatment package can be delivered between two groups, and within each group, a SCD can be used to assess effects of each combination of components. Although very few component analyses have assessed health behavior or symptoms per se as the outcome measure, there are a variety of behavioral interventions that have been evaluated using component analysis [ 63 ]. For example, Sanders [ 70 ] conducted a component analysis of an intervention to decrease lower back pain (and increase time standing/walking). The analysis consisted of four components: functional analysis of pain behavior (e.g., self-monitoring of pain and the conditions that precede and follow pain), progressive relaxation training, assertion training, and social reinforcement of increased activity. Sanders concluded that both relaxation training and reinforcement of activity were necessary components (see [ 69 ] for a discussion of some limitations of this study).

Several conclusions can be drawn about the effects of the various components in changing behavior. The data should first be evaluated to determine the extent to which the effects of individual components are independent of one another. If they are, then the effects of the components are additive. If they are not, then the effects are multiplicative, or the effects of one component depend on the presence of another component. Figure  3 presents simplified examples of these two possibilities using a reversal design and short data streams (adapted from [ 69 ]). The panel on the left shows additive effects, and the panel on the right shows multiplicative effects. The data also can be analyzed to determine whether each component is necessary and sufficient to produce behavior change. For instance, the panel on the right shows that neither the component labeled X (e.g., self-monitoring of health behavior) nor the component labeled Y (e.g., counseling to change health behavior) is sufficient, and both components are necessary. If two components produce equal changes in behavior, and the same amount of change when both are combined, then either component is sufficient but neither is necessary.

Two examples of possible results from a component analysis. BSL baseline, X first component, Y second component. The panel on the left shows an additive effect of components X and Y, and the panel of the right shows a multiplicative effect of components X and Y

The logic of the component analyses described here is similar to new methods derived from an engineering framework [ 2 , 9 , 71 ]. During the initial stages of intervention development, researchers use factorial designs to allocate participants to different combinations of treatment components. These designs, called fractional factorials because not all combinations of components are tested, can be used to screen promising components of treatment packages. The components tested may be derived from theory or working assumptions about which components and combinations will be of interest, which is the same process used to guide design choices in SCD research. Just as engineering methods seek to isolate and combine active treatment components to optimize interventions, so too do single-case methods. The main difference between approaches is the focus on the individual as the unit of analysis in SCDs.

OPTIMIZING WITH REPLICATIONS AND ESTABLISHING GENERALITY

Another form of optimization is an understanding of the conditions under which an intervention may be successful. These conditions may relate to particular characteristics of the participant (or whatever the unit of analysis happens to be) or to different situations. In other words, optimizing an intervention means establishing its generality.

In the context of single-case research, generality can be demonstrated experimentally in several ways. The most basic way is via direct replication [ 26 ]. Direct replication means conducting the same experiment on the same behavioral problem across several individuals (i.e., a single-case experiment). For example, Raiff and Dallery [ 72 ] achieved a direct replication of the effects of internet-based contingency management (CM) on adherence to glucose testing in four adolescents. One goal of the study was to establish experimental control by the intervention and to minimize as many extraneous factors as possible. Overall, direct replication can help establish generality across participants. It cannot answer questions about generality across settings, behavior change agents, target behaviors, or participants that differ in some way from the original experiment (e.g., to adults diagnosed with type 1 diabetes). Instead, systematic replication can answer these questions. In a systematic replication, the methods from previous direct replication studies are used in a new setting, target behavior, group of participants, and so on [ 73 ]. The Raiff and Dallery study, therefore, was also a systematic replication of effects of internet-based CM to promote smoking cessation to a new problem and to a new group of participants because the procedure had originally been tested with adult smokers [ 24 ]. Effects of internet-based CM for smoking cessation also were systematically replicated in an application to adolescent smokers using a single-case design [ 74 ].

Systematic replication also occurs with parametric manipulation [ 63 ]. In other words, rather than changing the type of participants or setting, we change the value of the independent variable. In addition to demonstrating an optimal effect, parametric analysis may also reveal boundary conditions. These may be conditions under which an intervention no longer has an effect, or points of diminishing returns in which further increases in some parameter produce no further increases in efficacy. For example, if one study was conducted showing that 30 min of moderate exercise produced a decrease in cigarette cravings, a systematic replication, using parametric analysis, might be conducted to determine the effects of other exercise durations (e.g., 5, 30, 60 min) on cigarette craving to identify the boundary parameters (i.e., the minimum and maximum number of minutes of exercise needed to continue to see changes in cigarette craving). Boundary conditions are critical in establishing generality of an intervention. In most cases, the only way to assess boundary conditions is through experimental, parametric analysis of an individual’s behavior.

By carefully choosing the characteristics of the individuals, settings, or other relevant variables in a systematic replication, the researcher can help identify the conditions under which a treatment works. To be sure, as with any new treatment, failures will occur. However, the failure does not detract from the prior successes: “…a procedure can be quite valuable even though it is effective under a narrow range of conditions, as long as we know what those conditions are” [ 75 ]. Such information is important for treatment recommendations in a clinical setting, and scientifically, it means that the conditions themselves may become the subject of experimental analysis.

This discussion leads to a type of generality called scientific generality [ 63 ], which is at the heart of a scientific understanding of behavioral health interventions (or any intervention for that matter). As described by Branch and Pennypacker [ 63 ], scientific generality is characterized by knowledgeable reproducibility, or knowledge of the factors that are required for a phenomenon to occur. Scientific generality can be attained through parametric and component analysis, and through systematic replication. One advantage of a single-case approach to establishing generality is that a series of strategic studies can be conducted with some degree of efficiency. Moreover, the data intimacy afforded by SCDs can help achieve scientific generality about behavioral health interventions.

PERSONALIZED BEHAVIORAL MEDICINE

Personalized behavioral medicine involves three steps: assessing diagnostic, demographic, and other variables that may influence treatment outcomes; assigning an individual to treatment based on this information; and using SCDs to assess and tailor treatment. The first and second steps may be informed by outcomes using SCDs. In addition, the clinician may be in a better position to personalize treatment with knowledge derived from a body of SCD research about generality, boundary conditions, and the factors that are necessary for an effect to occur. (Of course, this information can come from a variety of sources—we are simply highlighting how SCDs may fit in to this process.)

In addition, with advances in genomics and technology-enabled behavioral assessment prior to treatment (i.e., a baseline phase), the clinician may further target treatment to the unique characteristics of the individual [ 76 ]. Genetic testing is becoming more common before prescribing various medications [ 17 ], and it may become useful to predict responses for treatments targeting health behavior. Baseline assessment of behavior using technology such as EMA may allow the clinician to develop a tailored treatment protocol. For example, assessment could reveal the temporal patterning of risky situations, such as drinking alcohol, having an argument, or long periods of inactivity. A text-based support system could be tailored such that the timings of texts are tied to the temporal pattern of the problem behavior. The baseline assessment may also be useful to simply establish whether a problem exists. Also, the data path during baseline may reveal that behavior or symptoms are already improving prior to treatment, which would suggest that other, non-treatment variables are influencing behavior. Perhaps more importantly, compared to self-report, baseline conditions provide a more objective benchmark to assess effects of treatment on behavior and symptoms.

In addition to greater personalization at the start of treatment, ongoing assessment and treatment tailoring can be achieved with SCDs. Hayes [ 77 ] described how parametric and component analyses can be conducted in clinical practice. For example, reversal designs could be used to conduct a component analysis. Two components, or even different treatments, could be systematically introduced alone and together. If the treatments are different, such comparisons would also yield a kind of comparative effectiveness analysis. For example, contingency contracting and pharmacotherapy for smoking cessation could be presented alone using a BCBC design (where B is contracting and C is pharmacotherapy). A combined treatment could also be added, and depending on results, a return to one or the other treatment could follow (e.g., BCDCB, where D is the combined treatment). Furthermore, if a new treatment becomes available, it could be tested relative to an existing standard treatment in the same fashion. One potential limitation of such designs is when a reversal to baseline conditions (i.e., no treatment) is necessary to document treatment effects. Such a return to baseline may be challenging for ethical, reimbursement, and other issues.

Multiple-baseline designs also can be used in clinical contexts. Perhaps the simplest example would be a multiple baseline across individuals with similar problems. Each individual would experience an AB sequence, where the durations of the baseline phases vary. Another possibility is to target different behavior in the same individual in a multiple-baseline across behavior design. For example, a skills training program to improve social behavior could target different aspects of such behavior in a sequential fashion, starting with eye contact, then posture, then speech volume, and so on. If behavior occurs in a variety of distinct settings, the treatment could be sequentially implemented across these settings. Using the same example, treatment could target social behavior at family events, work, and different social settings. It can be problematic if generalization of effects occurs, but it may not necessarily negate the utility of such a design [ 27 ].

Multiple-baseline designs can be used in contexts other than outpatient therapy. Biglan and associates [ 1 ] argued that such designs are particularly useful in community interventions. For example, they described how a multiple baseline across communities and even states could be used to assess effects of changes in drinking age on car crashes. These designs may be especially useful to evaluate technology-based health interventions. A web-based program could be sequentially rolled out to different schools, communities, or other clusters of individuals. Although these research designs are also referred to as interrupted time series and stepped wedge designs, we think it may be more likely for researchers and clinicians to access the rich network of resources, concepts, and analytic tools if these designs are subsumed under the category of multiple-baseline designs.

The systematic comparisons afforded by SCDs can answer several key questions relevant to optimization. The first question a clinician may have is whether a particular intervention will work for his or her client [ 27 ]. It may be that the client has such a unique history and profile of symptoms, the clinician may not be confident about the predictive validity of a particular intervention for his or her client [ 6 ]. SCDs can be used to answer this question. Also, as just described, they can address which of two treatments work better, whether adding two treatments (or components) together works better than either one alone, which level of treatment is optimal (i.e., a parametric analysis), and whether a client prefers one treatment over another (i.e., via social validity assessment). Furthermore, the use of SCDs in practice conforms to the scientist-practitioner ideal espoused by training models in clinical psychology and allied disciplines [ 78 ].

OPTIMIZING FROM DEVELOPMENT TO DISSEMINATION

We are now in a position to evaluate whether SCDs live up to our ideals about optimization. During development, SCDs may obviate some logistical issues in using between-group designs to conduct initial efficacy testing [ 3 , 8 ]. Specifically, the costs and duration needed to conduct a SCD to establish preliminary efficacy would be considerably lower than traditional randomized designs. Riley and colleagues [ 8 ] noted that randomized trials take approximately 5.5 years from the initiation of enrollment to publication, and even longer from the time a grant application is submitted. In addition to establishing whether a treatment works, SCDs have the flexibility to efficiently address which parameters and components are necessary or optimal. In light of traditional methods to establish preliminary efficacy and optimize treatments, Riley and colleagues advocated for “rapid learning research systems.” SCDs are one such system.

Although some logistical issues may be mitigated by using SCDs, they do not necessarily represent easy alternatives to traditional group designs. They require a considerable amount of data per participant (as opposed to a large number of individuals in a group), enough participants to reliably demonstrate experimental effects, and systematic manipulation of variables over a long duration. For the vast majority of research questions, however, SCDs can reduce the resource and time burdens associated with between group designs and allow the investigator to detect important treatment parameters that might otherwise have been missed.

SCDs can minimize or eliminate a number of threats to internal validity. Although a complete discussion of these threats is beyond the scope of this paper (see [ 1 , 27 , 28 ]), the standards listed in Table  1 can provide protection against most threats. For example, the threat known as “testing” refers to the fact that repeated measurement alone may change behavior. To address this, baseline phases need to be sufficiently long, and there must be enough within and/or between participant replications to rule out the effect of testing. Such logic applies to a number of other potential threats (e.g., instrumentation, history, regression to the mean, etc.). In addition, a plethora of new analytic techniques can supplement experimental techniques to make inferences about causal relations. Combining SCD results in meta-analyses can yield information about comparative effects of different treatments, and combing results using Bayesian methods may yield information about likely effects at the population level.

Because of their efficiency and rigor, SCDs permit systematic replications across types of participants, behavior problems, and settings. This research process has also led to “gold-standard,” evidence-based treatments in applied behavior analysis and education [ 29 , 79 ]. More importantly, in several fields, such research has led to scientific understanding of the conditions under which treatment may be effective or ineffective [ 79 , 80 ]. The field of applied behavior analysis, for example, has matured to the extent that individualized assessment of the causes of problem behavior must occur before treatment recommendations.

Our discussion of personalized behavioral medicine highlighted how SCDs can be used in clinical practice to evaluate and optimize interventions. The advent of technology-based assessment makes SCDs much easier to implement. Technology could propel a “super convergence” of SCDs and clinical practice [ 76 ]. Advances in technology-based assessment can also promote the kind of systematic play central to the experimental attitude. It can also allow testing of new interventions as they become available. Such translational efforts can occur in several ways: from laboratory and other controlled settings to clinical practice, from SCD to SCD within clinical practice, and from randomized efficacy trials to clinical practice.

Over the past 70 years, SCD research has evolved to include a broad array of methodological and analytic advances. It also has generated evidence-based practices in health care and related disciplines such as clinical psychology [ 81 ], substance abuse [ 82 , 83 ], education [ 29 ], medicine [ 4 ], neuropsychology [ 30 ], developmental disabilities [ 27 ], and occupational therapy [ 84 ]. Although different methods are required for different purposes, SCDs are ideally suited to optimize interventions, from development to dissemination.

Acknowledgments

We wish to thank Paul Soto for comments on a previous draft of this manuscript. Preparation of this paper was supported in part by Grants P30DA029926 and R01DA023469 from the National Institute on Drug Abuse.

Conflict of interest

The authors have no conflicts of interest to disclose.

Implications

Practitioners: practitioners can use single-case designs in clinical practice to help ensure that an intervention or component of an intervention is working for an individual client or group of clients.

Policy makers: results from a single-case design research can help inform and evaluate policy regarding behavioral health interventions.

Researchers: researchers can use single-case designs to evaluate and optimize behavioral health interventions.

Contributor Information

Jesse Dallery, Phone: +1-352-3920601, Fax: +1-352-392-7985, Email: ude.lfu@yrellad .

Bethany R Raiff, Email: ude.nawor@ffiar .

• Subject List
• Take a Tour
• For Authors
• Subscriber Services
• Publications
• African American Studies
• African Studies
• American Literature
• Anthropology
• Architecture Planning and Preservation
• Art History
• Atlantic History
• Biblical Studies
• British and Irish Literature
• Childhood Studies
• Chinese Studies
• Cinema and Media Studies
• Communication
• Criminology
• Environmental Science
• Evolutionary Biology
• International Law
• International Relations
• Islamic Studies
• Jewish Studies
• Latin American Studies
• Latino Studies
• Linguistics
• Literary and Critical Theory
• Medieval Studies
• Military History
• Political Science
• Public Health
• Renaissance and Reformation
• Social Work
• Urban Studies
• Victorian Literature
• Browse All Subjects

How to Subscribe

• Free Trials

In This Article Expand or collapse the "in this article" section Single-Case Experimental Designs

Introduction, general overviews and primary textbooks.

• Textbooks in Applied Behavior Analysis
• Types of Single-Case Experimental Designs
• Model Building and Randomization in Single-Case Experimental Designs
• Visual Analysis of Single-Case Experimental Designs
• Effect Size Estimates in Single-Case Experimental Designs
• Reporting Single-Case Design Intervention Research

Related Articles Expand or collapse the "related articles" section about

About related articles close popup.

Lorem Ipsum Sit Dolor Amet

Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia Curae; Aliquam ligula odio, euismod ut aliquam et, vestibulum nec risus. Nulla viverra, arcu et iaculis consequat, justo diam ornare tellus, semper ultrices tellus nunc eu tellus.

• Action Research
• Ambulatory Assessment in Behavioral Science
• Effect Size
• Mediation Analysis
• Path Models
• Research Methods for Studying Daily Life

Other Subject Areas

Forthcoming articles expand or collapse the "forthcoming articles" section.

• Data Visualization
• Remote Work
• Workforce Training Evaluation
• Find more forthcoming articles...
• Export Citations
• Share This Facebook LinkedIn Twitter

Single-Case Experimental Designs by S. Andrew Garbacz , Thomas R. Kratochwill LAST REVIEWED: 29 July 2020 LAST MODIFIED: 29 July 2020 DOI: 10.1093/obo/9780199828340-0265

Single-case experimental designs are a family of experimental designs that are characterized by researcher manipulation of an independent variable and repeated measurement of a dependent variable before (i.e., baseline) and after (i.e., intervention phase) introducing the independent variable. In single-case experimental designs a case is the unit of intervention and analysis (e.g., a child, a school). Because measurement within each case is conducted before and after manipulation of the independent variable, the case typically serves as its own control. Experimental variants of single-case designs provide a basis for determining a causal relation by replication of the intervention through (a) introducing and withdrawing the independent variable, (b) manipulating the independent variable across different phases, and (c) introducing the independent variable in a staggered fashion across different points in time. Due to their economy of resources, single-case designs may be useful during development activities and allow for rapid replication across studies.

Several sources provide overviews of single-case experimental designs. Barlow, et al. 2009 includes an overview for the development of single-case experimental designs, describes key considerations for designing and conducting single-case experimental design research, and reviews procedural elements, assessment strategies, and replication considerations. Kazdin 2011 provides detailed coverage of single-case experimental design variants as well as approaches for evaluating data in single-case experimental designs. Kratochwill and Levin 2014 describes key methodological features that underlie single-case experimental designs, including philosophical and statistical foundations and data evaluation. Ledford and Gast 2018 covers research conceptualization and writing, design variants within single-case experimental design, definitions of variables and associated measurement, and approaches to organize and evaluate data. Riley-Tillman and Burns 2009 provides a practical orientation to single-case experimental designs to facilitate uptake and use in applied settings.

Barlow, D. H., M. K. Nock, and M. Hersen, eds. 2009. Single case experimental designs: Strategies for studying behavior change . 3d ed. New York: Pearson.

A comprehensive reference about the process of designing and conducting single-case experimental design studies. Chapters are integrative but can stand alone.

Kazdin, A. E. 2011. Single-case research designs: Methods for clinical and applied settings . 2d ed. New York: Oxford Univ. Press.

A complete overview and description of single-case experimental design variants as well as information about data evaluation.

Kratochwill, T. R., and J. R. Levin, eds. 2014. Single-case intervention research: Methodological and statistical advances . New York: Routledge.

The authors describe in depth the methodological and analytic considerations necessary for designing and conducting research that uses a single-case experimental design. In addition, the text includes chapters from leaders in psychology and education who provide critical perspectives about the use of single-case experimental designs.

Ledford, J. R., and D. L. Gast, eds. 2018. Single case research methodology: Applications in special education and behavioral sciences . New York: Routledge.

Covers the research process from writing literature reviews, to designing, conducting, and evaluating single-case experimental design studies.

Riley-Tillman, T. C., and M. K. Burns. 2009. Evaluating education interventions: Single-case design for measuring response to intervention . New York: Guilford Press.

Focuses on accelerating uptake and use of single-case experimental designs in applied settings. This book provides a practical, “nuts and bolts” orientation to conducting single-case experimental design research.

back to top

Users without a subscription are not able to see the full content on this page. Please subscribe or login .

Oxford Bibliographies Online is available by subscription and perpetual access to institutions. For more information or to contact an Oxford Sales Representative click here .

• About Psychology »
• Meet the Editorial Board »
• Abnormal Psychology
• Academic Assessment
• Acculturation and Health
• Action Regulation Theory
• Addictive Behavior
• Adolescence
• Adoption, Social, Psychological, and Evolutionary Perspect...
• Advanced Theory of Mind
• Affective Forecasting
• Affirmative Action
• Ageism at Work
• Allport, Gordon
• Alzheimer’s Disease
• Analysis of Covariance (ANCOVA)
• Animal Behavior
• Animal Learning
• Anxiety Disorders
• Art and Aesthetics, Psychology of
• Artificial Intelligence, Machine Learning, and Psychology
• Assessment and Clinical Applications of Individual Differe...
• Attachment in Social and Emotional Development across the ...
• Attention-Deficit/Hyperactivity Disorder (ADHD) in Adults
• Attention-Deficit/Hyperactivity Disorder (ADHD) in Childre...
• Attitudinal Ambivalence
• Attraction in Close Relationships
• Attribution Theory
• Authoritarian Personality
• Bayesian Statistical Methods in Psychology
• Behavior Therapy, Rational Emotive
• Behavioral Economics
• Behavioral Genetics
• Belief Perseverance
• Bereavement and Grief
• Biological Psychology
• Birth Order
• Body Image in Men and Women
• Bystander Effect
• Categorical Data Analysis in Psychology
• Childhood and Adolescence, Peer Victimization and Bullying...
• Clark, Mamie Phipps
• Clinical Neuropsychology
• Clinical Psychology
• Cognitive Consistency Theories
• Cognitive Dissonance Theory
• Cognitive Neuroscience
• Communication, Nonverbal Cues and
• Comparative Psychology
• Competence to Stand Trial: Restoration Services
• Competency to Stand Trial
• Computational Psychology
• Conflict Management in the Workplace
• Conformity, Compliance, and Obedience
• Consciousness
• Coping Processes
• Correspondence Analysis in Psychology
• Counseling Psychology
• Creativity at Work
• Critical Thinking
• Cross-Cultural Psychology
• Cultural Psychology
• Daily Life, Research Methods for Studying
• Data Science Methods for Psychology
• Data Sharing in Psychology
• Death and Dying
• Deceiving and Detecting Deceit
• Defensive Processes
• Depressive Disorders
• Development, Prenatal
• Developmental Psychology (Cognitive)
• Developmental Psychology (Social)
• Diagnostic and Statistical Manual of Mental Disorders (DSM...
• Discrimination
• Dissociative Disorders
• Drugs and Behavior
• Eating Disorders
• Ecological Psychology
• Educational Settings, Assessment of Thinking in
• Embodiment and Embodied Cognition
• Emerging Adulthood
• Emotional Intelligence
• Empathy and Altruism
• Employee Stress and Well-Being
• Environmental Neuroscience and Environmental Psychology
• Ethics in Psychological Practice
• Event Perception
• Evolutionary Psychology
• Expansive Posture
• Experimental Existential Psychology
• Exploratory Data Analysis
• Eyewitness Testimony
• Eysenck, Hans
• Factor Analysis
• Festinger, Leon
• Five-Factor Model of Personality
• Flynn Effect, The
• Forensic Psychology
• Forgiveness
• Friendships, Children's
• Fundamental Attribution Error/Correspondence Bias
• Gambler's Fallacy
• Game Theory and Psychology
• Geropsychology, Clinical
• Global Mental Health
• Habit Formation and Behavior Change
• Health Psychology
• Health Psychology Research and Practice, Measurement in
• Heider, Fritz
• Heuristics and Biases
• History of Psychology
• Human Factors
• Humanistic Psychology
• Implicit Association Test (IAT)
• Industrial and Organizational Psychology
• Inferential Statistics in Psychology
• Insanity Defense, The
• Intelligence
• Intelligence, Crystallized and Fluid
• Intercultural Psychology
• Intergroup Conflict
• International Classification of Diseases and Related Healt...
• International Psychology
• Interviewing in Forensic Settings
• Intimate Partner Violence, Psychological Perspectives on
• Introversion–Extraversion
• Item Response Theory
• Law, Psychology and
• Lazarus, Richard
• Learned Helplessness
• Learning Theory
• Learning versus Performance
• LGBTQ+ Romantic Relationships
• Lie Detection in a Forensic Context
• Life-Span Development
• Locus of Control
• Loneliness and Health
• Mathematical Psychology
• Meaning in Life
• Mechanisms and Processes of Peer Contagion
• Media Violence, Psychological Perspectives on
• Memories, Autobiographical
• Memories, Flashbulb
• Memories, Repressed and Recovered
• Memory, False
• Memory, Human
• Memory, Implicit versus Explicit
• Memory in Educational Settings
• Memory, Semantic
• Meta-Analysis
• Metacognition
• Metaphor, Psychological Perspectives on
• Microaggressions
• Military Psychology
• Mindfulness
• Mindfulness and Education
• Minnesota Multiphasic Personality Inventory (MMPI)
• Money, Psychology of
• Moral Conviction
• Moral Development
• Moral Psychology
• Moral Reasoning
• Nature versus Nurture Debate in Psychology
• Neuroscience of Associative Learning
• Nonergodicity in Psychology and Neuroscience
• Nonparametric Statistical Analysis in Psychology
• Observational (Non-Randomized) Studies
• Obsessive-Complusive Disorder (OCD)
• Occupational Health Psychology
• Olfaction, Human
• Operant Conditioning
• Optimism and Pessimism
• Organizational Justice
• Parenting Stress
• Parenting Styles
• Parents' Beliefs about Children
• Peace Psychology
• Perception, Person
• Performance Appraisal
• Personality and Health
• Personality Disorders
• Personality Psychology
• Person-Centered and Experiential Psychotherapies: From Car...
• Phenomenological Psychology
• Placebo Effects in Psychology
• Play Behavior
• Positive Psychological Capital (PsyCap)
• Positive Psychology
• Posttraumatic Stress Disorder (PTSD)
• Prejudice and Stereotyping
• Pretrial Publicity
• Prisoner's Dilemma
• Problem Solving and Decision Making
• Procrastination
• Prosocial Behavior
• Prosocial Spending and Well-Being
• Protocol Analysis
• Psycholinguistics
• Psychological Literacy
• Psychological Perspectives on Food and Eating
• Psychology, Political
• Psychoneuroimmunology
• Psychophysics, Visual
• Psychotherapy
• Psychotic Disorders
• Publication Bias in Psychology
• Reasoning, Counterfactual
• Rehabilitation Psychology
• Relationships
• Reliability–Contemporary Psychometric Conceptions
• Religion, Psychology and
• Replication Initiatives in Psychology
• Research Methods
• Risk Taking
• Role of the Expert Witness in Forensic Psychology, The
• Sample Size Planning for Statistical Power and Accurate Es...
• Schizophrenic Disorders
• School Psychology
• School Psychology, Counseling Services in
• Self, Gender and
• Self, Psychology of the
• Self-Construal
• Self-Control
• Self-Deception
• Self-Determination Theory
• Self-Efficacy
• Self-Esteem
• Self-Monitoring
• Self-Regulation in Educational Settings
• Self-Report Tests, Measures, and Inventories in Clinical P...
• Sensation Seeking
• Sex and Gender
• Sexual Minority Parenting
• Sexual Orientation
• Signal Detection Theory and its Applications
• Simpson's Paradox in Psychology
• Single People
• Single-Case Experimental Designs
• Skinner, B.F.
• Sleep and Dreaming
• Small Groups
• Social Class and Social Status
• Social Cognition
• Social Neuroscience
• Social Support
• Social Touch and Massage Therapy Research
• Somatoform Disorders
• Spatial Attention
• Sports Psychology
• Stanford Prison Experiment (SPE): Icon and Controversy
• Stereotype Threat
• Stereotypes
• Stress and Coping, Psychology of
• Student Success in College
• Subjective Wellbeing Homeostasis
• Taste, Psychological Perspectives on
• Teaching of Psychology
• Terror Management Theory
• Testing and Assessment
• The Concept of Validity in Psychological Assessment
• The Neuroscience of Emotion Regulation
• The Reasoned Action Approach and the Theories of Reasoned ...
• The Weapon Focus Effect in Eyewitness Memory
• Theory of Mind
• Therapy, Cognitive-Behavioral
• Thinking Skills in Educational Settings
• Time Perception
• Trait Perspective
• Trauma Psychology
• Twin Studies
• Type A Behavior Pattern (Coronary Prone Personality)
• Unconscious Processes
• Video Games and Violent Content
• Virtues and Character Strengths
• Women and Science, Technology, Engineering, and Math (STEM...
• Women, Psychology of
• Work Well-Being
• Wundt, Wilhelm
• Privacy Policy
• Cookie Policy
• Legal Notice
• Accessibility

Powered by:

• [66.249.64.20|185.39.149.46]
• 185.39.149.46

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

15.1 The basics of single-system research design

Learning objectives.

Learners will be able to…

• Identify why social workers might use single-subjects design.
• Describe the two stages of single-subjects design.
• Previously imported from Mauldin 2019
• WRITTEN BY TRMC

Single-systems research design, sometimes called single-subject or single-case research design, is distinct from other research methodologies in that, as its name indicates, only one person, group, policy, etc. (i.e., system) is being studied. Because clinical social work often involves one-on-one practice, single-subjects designs are often used by social workers to ensure that their interventions are having a positive effect. Single-subjects designs are used to demonstrate that social work intervention has its intended effects.  Single-subjects designs are most compatible with clinical modalities such as cognitive-behavioral therapy which incorporate as part of treatment client self-monitoring, clinician data analysis, and quantitative measurement. It is routine in this therapeutic model to track, for example, the number of intrusive thoughts experienced between counseling sessions. Moreover, practitioners spend time each session reviewing changes in patterns during the therapeutic process, using it to evaluate and fine-tune the therapeutic approach. Although researchers have used single-subjects designs with less positivist therapies, such as narrative therapy, the single-subjects design is generally used in therapies with more quantifiable outcomes. The results of single-systems studies help ensure that social workers are not providing useless or counterproductive interventions to their clients.

By definition, single-systems design only attempts to explain causality in one case; therefore the results are not generalizable. Because their results are not generalizable, single-systems studies do not meet the strict definition of research , and are generally used in practice settings only. While the results will not be generalizable, they do provide important insight into the effectiveness of clinical interventions. Social work researchers teaching in BSW and MSW programs will teach single-systems design to their students, and on occasion may publish results of single systems research design to further knowledge about interventions.

The two main stages of single-systems research design

Single-systems designs follow the logic of experimental design by attempting to compare conditions when no intervention or treatment is applied to conditions when there is an intervention. To do this, SSRDs involve repeated measurements over time, usually in two stages and attempts to identify changes in a behavioral outcome (i.e., the dependent variable) s a result of an intervention, program, or treatment (i.e., the independent variable). The dependent variable is measured repeatedly during two distinct phases: the baseline stage and the treatment stage .

The baseline stage is the period of time before the intervention starts. During the baseline stage, the social worker is collecting data about the problem the treatment is hoping to address.  For example, a person with substance use issues may binge drink on the weekends but cut down their drinking during the work week.  A social worker might ask the client to record the number of drinks that they consume each day.  By looking at this, we could evaluate the level of alcohol consumption.  For other clients, the social worker might assess other indicators, such as the number of arguments the client had when they were drinking or whether or not the client blacked out as a result of drinking.  Whatever measure is used to assess the targeted problem, that measure is the dependent variable in the single-subjects design.

The baseline stage should last until a pattern emerges in the dependent variable.  This requires at least three different occasions of measurement, but it can often take longer.  During the baseline stage, the social worker looks for one of three types of patterns (Engel & Schutt, 2016).  The dependent variable may (1) be stable over time, (2) exhibit a trend where it is increasing or decreasing over time, or (3) have a cycle of increasing and decreasing that is repeated over time.  Establishing a pattern can prove difficult in clients whose behaviors vary widely.

Ideally, social workers would start measurement for the baseline stage before starting the intervention. This provides the opportunity to determine the baseline pattern.  Unfortunately, that may be impractical or unethical to do in practice if it entails withholding important treatment. In that case, a retrospective baseline can be attained by asking the client to recollect data from before the intervention started.  The drawback to this is the information is likely to be less reliable than a baseline data recorded in real time. The baseline stage is important because with only one subject, there is no control group. Thus, we have to see if our intervention is effective by comparing the client before treatment to and during and after treatment.  In this way, the baseline stage provides the same type of information as a control group — what it looks like when there is not treatment given.

The next stage is the treatment stage , and it refers to the time in which the treatment is administered by the social worker. Repeated measurements are taken during this stage to see if there is change in the dependent variable during treatment.

One way to analyze the data from a single-subjects design is to visually examine a graphical representation of the results.  An example of a graph from a single-subjects design is shown in Figure 11.1.  The x -axis is time, as measured in months. The y -axis is the measure of the problem we’re trying to change (i.e., the dependent variable).

In Figure 11.1, the y -axis is caseload size. From 1998 to July of 1991, there was no treatment. This is the baseline phase, and we can examine it for a pattern. There was an upward trend at the beginning of the baseline phase, but it looks as if the caseloads began to decrease around October 1989. The vertical line indicates when the intervention began (around July 1991). Once the intervention occurred, the downward trend continues. In this case, it is not clear if there was a change due to the intervention or if it was a continuation of a trend that began in October of 1989.

Key Takeaways

• Social workers conduct single-subjects research designs to make sure their interventions are effective.
• Single-subjects designs use repeated measures before and during treatment to assess the effectiveness of an intervention.
• Single-subjects designs often use a graphical representation of numerical data to look for patterns.

a systematic investigation, including development, testing, and. evaluation, designed to develop or contribute to generalizable knowledge

The stage in single-subjects design in which a baseline level or pattern of the dependent variable is established

The stage in single subjects research design in which the treatment or intervention is delivered

Doctoral Research Methods in Social Work Copyright © by Mavs Open Press. All Rights Reserved.

Share This Book

Our websites may use cookies to personalize and enhance your experience. By continuing without changing your cookie settings, you agree to this collection. For more information, please see our University Websites Privacy Notice .

Neag School of Education

Educational Research Basics by Del Siegle

Single subject research.

“ Single subject research (also known as single case experiments) is popular in the fields of special education and counseling. This research design is useful when the researcher is attempting to change the behavior of an individual or a small group of individuals and wishes to document that change. Unlike true experiments where the researcher randomly assigns participants to a control and treatment group, in single subject research the participant serves as both the control and treatment group. The researcher uses line graphs to show the effects of a particular intervention or treatment.  An important factor of single subject research is that only one variable is changed at a time. Single subject research designs are “weak when it comes to external validity….Studies involving single-subject designs that show a particular treatment to be effective in changing behavior must rely on replication–across individuals rather than groups–if such results are be found worthy of generalization” (Fraenkel & Wallen, 2006, p. 318).

Suppose a researcher wished to investigate the effect of praise on reducing disruptive behavior over many days. First she would need to establish a baseline of how frequently the disruptions occurred. She would measure how many disruptions occurred each day for several days. In the example below, the target student was disruptive seven times on the first day, six times on the second day, and seven times on the third day. Note how the sequence of time is depicted on the x-axis (horizontal axis) and the dependent variable (outcome variable) is depicted on the y-axis (vertical axis).

Once a baseline of behavior has been established (when a consistent pattern emerges with at least three data points), the intervention begins. The researcher continues to plot the frequency of behavior while implementing the intervention of praise.

In this example, we can see that the frequency of disruptions decreased once praise began. The design in this example is known as an A-B design. The baseline period is referred to as A and the intervention period is identified as B.

Another design is the A-B-A design. An A-B-A design (also known as a reversal design) involves discontinuing the intervention and returning to a nontreatment condition.

Sometimes an individual’s behavior is so severe that the researcher cannot wait to establish a baseline and must begin with an intervention. In this case, a B-A-B design is used. The intervention is implemented immediately (before establishing a baseline). This is followed by a measurement without the intervention and then a repeat of the intervention.

Multiple-Baseline Design

Sometimes, a researcher may be interested in addressing several issues for one student or a single issue for several students. In this case, a multiple-baseline design is used.

“In a multiple baseline across subjects design, the researcher introduces the intervention to different persons at different times. The significance of this is that if a behavior changes only after the intervention is presented, and this behavior change is seen successively in each subject’s data, the effects can more likely be credited to the intervention itself as opposed to other variables. Multiple-baseline designs do not require the intervention to be withdrawn. Instead, each subject’s own data are compared between intervention and nonintervention behaviors, resulting in each subject acting as his or her own control (Kazdin, 1982). An added benefit of this design, and all single-case designs, is the immediacy of the data. Instead of waiting until postintervention to take measures on the behavior, single-case research prescribes continuous data collection and visual monitoring of that data displayed graphically, allowing for immediate instructional decision-making. Students, therefore, do not linger in an intervention that is not working for them, making the graphic display of single-case research combined with differentiated instruction responsive to the needs of students.” (Geisler, Hessler, Gardner, & Lovelace, 2009)

Regardless of the research design, the line graphs used to illustrate the data contain a set of common elements.

Generally, in single subject research we count the number of times something occurs in a given time period and see if it occurs more or less often in that time period after implementing an intervention. For example, we might measure how many baskets someone makes while shooting for 2 minutes. We would repeat that at least three times to get our baseline. Next, we would test some intervention. We might play music while shooting, give encouragement while shooting, or video the person while shooting to see if our intervention influenced the number of shots made. After the 3 baseline measurements (3 sets of 2 minute shooting), we would measure several more times (sets of 2 minute shooting) after the intervention and plot the time points (number of baskets made in 2 minutes for each of the measured time points). This works well for behaviors that are distinct and can be counted.

Sometimes behaviors come and go over time (such as being off task in a classroom or not listening during a coaching session). The way we can record these is to select a period of time (say 5 minutes) and mark down every 10 seconds whether our participant is on task. We make a minimum of three sets of 5 minute observations for a baseline, implement an intervention, and then make more sets of 5 minute observations with the intervention in place. We use this method rather than counting how many times someone is off task because one could continually be off task and that would only be a count of 1 since the person was continually off task. Someone who might be off task twice for 15 second would be off task twice for a score of 2. However, the second person is certainly not off task twice as much as the first person. Therefore, recording whether the person is off task at 10-second intervals gives a more accurate picture. The person continually off task would have a score of 30 (off task at every second interval for 5 minutes) and the person off task twice for a short time would have a score of 2 (off task only during 2 of the 10 second interval measures.

I also have additional information about how to record single-subject research data .

I hope this helps you better understand single subject research.

I have created a PowerPoint on Single Subject Research , which also available below as a video.

I have also created instructions for creating single-subject research design graphs with Excel .

Fraenkel, J. R., & Wallen, N. E. (2006). How to design and evaluate research in education (6th ed.). Boston, MA: McGraw Hill.

Geisler, J. L., Hessler, T., Gardner, R., III, & Lovelace, T. S. (2009). Differentiated writing interventions for high-achieving urban African American elementary students. Journal of Advanced Academics, 20, 214–247.

Del Siegle, Ph.D. University of Connecticut [email protected] www.delsiegle.info

Revised 02/02/2024

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 10: Single-Subject Research

Single-Subject Research Designs

Learning Objectives

• Describe the basic elements of a single-subject research design.
• Design simple single-subject studies using reversal and multiple-baseline designs.
• Explain how single-subject research designs address the issue of internal validity.
• Interpret the results of simple single-subject studies based on the visual inspection of graphed data.

General Features of Single-Subject Designs

Before looking at any specific single-subject research designs, it will be helpful to consider some features that are common to most of them. Many of these features are illustrated in Figure 10.2, which shows the results of a generic single-subject study. First, the dependent variable (represented on the  y -axis of the graph) is measured repeatedly over time (represented by the  x -axis) at regular intervals. Second, the study is divided into distinct phases, and the participant is tested under one condition per phase. The conditions are often designated by capital letters: A, B, C, and so on. Thus Figure 10.2 represents a design in which the participant was tested first in one condition (A), then tested in another condition (B), and finally retested in the original condition (A). (This is called a reversal design and will be discussed in more detail shortly.)

Another important aspect of single-subject research is that the change from one condition to the next does not usually occur after a fixed amount of time or number of observations. Instead, it depends on the participant’s behaviour. Specifically, the researcher waits until the participant’s behaviour in one condition becomes fairly consistent from observation to observation before changing conditions. This is sometimes referred to as the steady state strategy  (Sidman, 1960) [1] . The idea is that when the dependent variable has reached a steady state, then any change across conditions will be relatively easy to detect. Recall that we encountered this same principle when discussing experimental research more generally. The effect of an independent variable is easier to detect when the “noise” in the data is minimized.

Reversal Designs

The most basic single-subject research design is the  reversal design , also called the  ABA design . During the first phase, A, a  baseline  is established for the dependent variable. This is the level of responding before any treatment is introduced, and therefore the baseline phase is a kind of control condition. When steady state responding is reached, phase B begins as the researcher introduces the treatment. There may be a period of adjustment to the treatment during which the behaviour of interest becomes more variable and begins to increase or decrease. Again, the researcher waits until that dependent variable reaches a steady state so that it is clear whether and how much it has changed. Finally, the researcher removes the treatment and again waits until the dependent variable reaches a steady state. This basic reversal design can also be extended with the reintroduction of the treatment (ABAB), another return to baseline (ABABA), and so on.

The study by Hall and his colleagues was an ABAB reversal design. Figure 10.3 approximates the data for Robbie. The percentage of time he spent studying (the dependent variable) was low during the first baseline phase, increased during the first treatment phase until it leveled off, decreased during the second baseline phase, and again increased during the second treatment phase.

Why is the reversal—the removal of the treatment—considered to be necessary in this type of design? Why use an ABA design, for example, rather than a simpler AB design? Notice that an AB design is essentially an interrupted time-series design applied to an individual participant. Recall that one problem with that design is that if the dependent variable changes after the treatment is introduced, it is not always clear that the treatment was responsible for the change. It is possible that something else changed at around the same time and that this extraneous variable is responsible for the change in the dependent variable. But if the dependent variable changes with the introduction of the treatment and then changes  back  with the removal of the treatment (assuming that the treatment does not create a permanent effect), it is much clearer that the treatment (and removal of the treatment) is the cause. In other words, the reversal greatly increases the internal validity of the study.

There are close relatives of the basic reversal design that allow for the evaluation of more than one treatment. In a  multiple-treatment reversal design , a baseline phase is followed by separate phases in which different treatments are introduced. For example, a researcher might establish a baseline of studying behaviour for a disruptive student (A), then introduce a treatment involving positive attention from the teacher (B), and then switch to a treatment involving mild punishment for not studying (C). The participant could then be returned to a baseline phase before reintroducing each treatment—perhaps in the reverse order as a way of controlling for carryover effects. This particular multiple-treatment reversal design could also be referred to as an ABCACB design.

In an  alternating treatments design , two or more treatments are alternated relatively quickly on a regular schedule. For example, positive attention for studying could be used one day and mild punishment for not studying the next, and so on. Or one treatment could be implemented in the morning and another in the afternoon. The alternating treatments design can be a quick and effective way of comparing treatments, but only when the treatments are fast acting.

Multiple-Baseline Designs

There are two potential problems with the reversal design—both of which have to do with the removal of the treatment. One is that if a treatment is working, it may be unethical to remove it. For example, if a treatment seemed to reduce the incidence of self-injury in a developmentally disabled child, it would be unethical to remove that treatment just to show that the incidence of self-injury increases. The second problem is that the dependent variable may not return to baseline when the treatment is removed. For example, when positive attention for studying is removed, a student might continue to study at an increased rate. This could mean that the positive attention had a lasting effect on the student’s studying, which of course would be good. But it could also mean that the positive attention was not really the cause of the increased studying in the first place. Perhaps something else happened at about the same time as the treatment—for example, the student’s parents might have started rewarding him for good grades.

One solution to these problems is to use a  multiple-baseline design , which is represented in Figure 10.4. In one version of the design, a baseline is established for each of several participants, and the treatment is then introduced for each one. In essence, each participant is tested in an AB design. The key to this design is that the treatment is introduced at a different  time  for each participant. The idea is that if the dependent variable changes when the treatment is introduced for one participant, it might be a coincidence. But if the dependent variable changes when the treatment is introduced for multiple participants—especially when the treatment is introduced at different times for the different participants—then it is extremely unlikely to be a coincidence.

As an example, consider a study by Scott Ross and Robert Horner (Ross & Horner, 2009) [2] . They were interested in how a school-wide bullying prevention program affected the bullying behaviour of particular problem students. At each of three different schools, the researchers studied two students who had regularly engaged in bullying. During the baseline phase, they observed the students for 10-minute periods each day during lunch recess and counted the number of aggressive behaviours they exhibited toward their peers. (The researchers used handheld computers to help record the data.) After 2 weeks, they implemented the program at one school. After 2 more weeks, they implemented it at the second school. And after 2 more weeks, they implemented it at the third school. They found that the number of aggressive behaviours exhibited by each student dropped shortly after the program was implemented at his or her school. Notice that if the researchers had only studied one school or if they had introduced the treatment at the same time at all three schools, then it would be unclear whether the reduction in aggressive behaviours was due to the bullying program or something else that happened at about the same time it was introduced (e.g., a holiday, a television program, a change in the weather). But with their multiple-baseline design, this kind of coincidence would have to happen three separate times—a very unlikely occurrence—to explain their results.

In another version of the multiple-baseline design, multiple baselines are established for the same participant but for different dependent variables, and the treatment is introduced at a different time for each dependent variable. Imagine, for example, a study on the effect of setting clear goals on the productivity of an office worker who has two primary tasks: making sales calls and writing reports. Baselines for both tasks could be established. For example, the researcher could measure the number of sales calls made and reports written by the worker each week for several weeks. Then the goal-setting treatment could be introduced for one of these tasks, and at a later time the same treatment could be introduced for the other task. The logic is the same as before. If productivity increases on one task after the treatment is introduced, it is unclear whether the treatment caused the increase. But if productivity increases on both tasks after the treatment is introduced—especially when the treatment is introduced at two different times—then it seems much clearer that the treatment was responsible.

In yet a third version of the multiple-baseline design, multiple baselines are established for the same participant but in different settings. For example, a baseline might be established for the amount of time a child spends reading during his free time at school and during his free time at home. Then a treatment such as positive attention might be introduced first at school and later at home. Again, if the dependent variable changes after the treatment is introduced in each setting, then this gives the researcher confidence that the treatment is, in fact, responsible for the change.

Data Analysis in Single-Subject Research

In addition to its focus on individual participants, single-subject research differs from group research in the way the data are typically analyzed. As we have seen throughout the book, group research involves combining data across participants. Group data are described using statistics such as means, standard deviations, Pearson’s  r , and so on to detect general patterns. Finally, inferential statistics are used to help decide whether the result for the sample is likely to generalize to the population. Single-subject research, by contrast, relies heavily on a very different approach called  visual inspection . This means plotting individual participants’ data as shown throughout this chapter, looking carefully at those data, and making judgments about whether and to what extent the independent variable had an effect on the dependent variable. Inferential statistics are typically not used.

In visually inspecting their data, single-subject researchers take several factors into account. One of them is changes in the  level  of the dependent variable from condition to condition. If the dependent variable is much higher or much lower in one condition than another, this suggests that the treatment had an effect. A second factor is  trend , which refers to gradual increases or decreases in the dependent variable across observations. If the dependent variable begins increasing or decreasing with a change in conditions, then again this suggests that the treatment had an effect. It can be especially telling when a trend changes directions—for example, when an unwanted behaviour is increasing during baseline but then begins to decrease with the introduction of the treatment. A third factor is  latency , which is the time it takes for the dependent variable to begin changing after a change in conditions. In general, if a change in the dependent variable begins shortly after a change in conditions, this suggests that the treatment was responsible.

In the top panel of Figure 10.5, there are fairly obvious changes in the level and trend of the dependent variable from condition to condition. Furthermore, the latencies of these changes are short; the change happens immediately. This pattern of results strongly suggests that the treatment was responsible for the changes in the dependent variable. In the bottom panel of Figure 10.5, however, the changes in level are fairly small. And although there appears to be an increasing trend in the treatment condition, it looks as though it might be a continuation of a trend that had already begun during baseline. This pattern of results strongly suggests that the treatment was not responsible for any changes in the dependent variable—at least not to the extent that single-subject researchers typically hope to see.

The results of single-subject research can also be analyzed using statistical procedures—and this is becoming more common. There are many different approaches, and single-subject researchers continue to debate which are the most useful. One approach parallels what is typically done in group research. The mean and standard deviation of each participant’s responses under each condition are computed and compared, and inferential statistical tests such as the  t  test or analysis of variance are applied (Fisch, 2001) [3] . (Note that averaging  across  participants is less common.) Another approach is to compute the  percentage of nonoverlapping data  (PND) for each participant (Scruggs & Mastropieri, 2001) [4] . This is the percentage of responses in the treatment condition that are more extreme than the most extreme response in a relevant control condition. In the study of Hall and his colleagues, for example, all measures of Robbie’s study time in the first treatment condition were greater than the highest measure in the first baseline, for a PND of 100%. The greater the percentage of nonoverlapping data, the stronger the treatment effect. Still, formal statistical approaches to data analysis in single-subject research are generally considered a supplement to visual inspection, not a replacement for it.

Key Takeaways

• Single-subject research designs typically involve measuring the dependent variable repeatedly over time and changing conditions (e.g., from baseline to treatment) when the dependent variable has reached a steady state. This approach allows the researcher to see whether changes in the independent variable are causing changes in the dependent variable.
• In a reversal design, the participant is tested in a baseline condition, then tested in a treatment condition, and then returned to baseline. If the dependent variable changes with the introduction of the treatment and then changes back with the return to baseline, this provides strong evidence of a treatment effect.
• In a multiple-baseline design, baselines are established for different participants, different dependent variables, or different settings—and the treatment is introduced at a different time on each baseline. If the introduction of the treatment is followed by a change in the dependent variable on each baseline, this provides strong evidence of a treatment effect.
• Single-subject researchers typically analyze their data by graphing them and making judgments about whether the independent variable is affecting the dependent variable based on level, trend, and latency.
• Does positive attention from a parent increase a child’s toothbrushing behaviour?
• Does self-testing while studying improve a student’s performance on weekly spelling tests?
• Does regular exercise help relieve depression?
• Practice: Create a graph that displays the hypothetical results for the study you designed in Exercise 1. Write a paragraph in which you describe what the results show. Be sure to comment on level, trend, and latency.

Long Descriptions

Figure 10.3 long description: Line graph showing the results of a study with an ABAB reversal design. The dependent variable was low during first baseline phase; increased during the first treatment; decreased during the second baseline, but was still higher than during the first baseline; and was highest during the second treatment phase. [Return to Figure 10.3]

Figure 10.4 long description: Three line graphs showing the results of a generic multiple-baseline study, in which different baselines are established and treatment is introduced to participants at different times.

For Baseline 1, treatment is introduced one-quarter of the way into the study. The dependent variable ranges between 12 and 16 units during the baseline, but drops down to 10 units with treatment and mostly decreases until the end of the study, ranging between 4 and 10 units.

For Baseline 2, treatment is introduced halfway through the study. The dependent variable ranges between 10 and 15 units during the baseline, then has a sharp decrease to 7 units when treatment is introduced. However, the dependent variable increases to 12 units soon after the drop and ranges between 8 and 10 units until the end of the study.

For Baseline 3, treatment is introduced three-quarters of the way into the study. The dependent variable ranges between 12 and 16 units for the most part during the baseline, with one drop down to 10 units. When treatment is introduced, the dependent variable drops down to 10 units and then ranges between 8 and 9 units until the end of the study. [Return to Figure 10.4]

Figure 10.5 long description: Two graphs showing the results of a generic single-subject study with an ABA design. In the first graph, under condition A, level is high and the trend is increasing. Under condition B, level is much lower than under condition A and the trend is decreasing. Under condition A again, level is about as high as the first time and the trend is increasing. For each change, latency is short, suggesting that the treatment is the reason for the change.

In the second graph, under condition A, level is relatively low and the trend is increasing. Under condition B, level is a little higher than during condition A and the trend is increasing slightly. Under condition A again, level is a little lower than during condition B and the trend is decreasing slightly. It is difficult to determine the latency of these changes, since each change is rather minute, which suggests that the treatment is ineffective. [Return to Figure 10.5]

• Sidman, M. (1960). Tactics of scientific research: Evaluating experimental data in psychology . Boston, MA: Authors Cooperative. ↵
• Ross, S. W., & Horner, R. H. (2009). Bully prevention in positive behaviour support. Journal of Applied Behaviour Analysis, 42 , 747–759. ↵
• Fisch, G. S. (2001). Evaluating data from behavioural analysis: Visual inspection or statistical models.  Behavioural Processes, 54 , 137–154. ↵
• Scruggs, T. E., & Mastropieri, M. A. (2001). How to summarize single-participant research: Ideas and applications.  Exceptionality, 9 , 227–244. ↵

The researcher waits until the participant’s behaviour in one condition becomes fairly consistent from observation to observation before changing conditions. This way, any change across conditions will be easy to detect.

A study method in which the researcher gathers data on a baseline state, introduces the treatment and continues observation until a steady state is reached, and finally removes the treatment and observes the participant until they return to a steady state.

The level of responding before any treatment is introduced and therefore acts as a kind of control condition.

A baseline phase is followed by separate phases in which different treatments are introduced.

Two or more treatments are alternated relatively quickly on a regular schedule.

A baseline is established for several participants and the treatment is then introduced to each participant at a different time.

The plotting of individual participants’ data, examining the data, and making judgements about whether and to what extent the independent variable had an effect on the dependent variable.

Whether the data is higher or lower based on a visual inspection of the data; a change in the level implies the treatment introduced had an effect.

The gradual increases or decreases in the dependent variable across observations.

The time it takes for the dependent variable to begin changing after a change in conditions.

The percentage of responses in the treatment condition that are more extreme than the most extreme response in a relevant control condition.

Research Methods in Psychology - 2nd Canadian Edition Copyright © 2015 by Paul C. Price, Rajiv Jhangiani, & I-Chant A. Chiang is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

Single-Case Design, Analysis, and Quality Assessment for Intervention Research

Affiliation.

• 1 Biomechanics & Movement Science Program, Department of Physical Therapy, University of Delaware, Newark, Delaware (M.A.L., A.B.C., I.B.); and Division of Educational Psychology & Methodology, State University of New York at Albany, Albany, New York (M.M.).
• PMID: 28628553
• PMCID: PMC5492992
• DOI: 10.1097/NPT.0000000000000187

Background and purpose: The purpose of this article is to describe single-case studies and contrast them with case studies and randomized clinical trials. We highlight current research designs, analysis techniques, and quality appraisal tools relevant for single-case rehabilitation research.

Summary of key points: Single-case studies can provide a viable alternative to large group studies such as randomized clinical trials. Single-case studies involve repeated measures and manipulation of an independent variable. They can be designed to have strong internal validity for assessing causal relationships between interventions and outcomes, as well as external validity for generalizability of results, particularly when the study designs incorporate replication, randomization, and multiple participants. Single-case studies should not be confused with case studies/series (ie, case reports), which are reports of clinical management of a patient or a small series of patients.

Recommendations for clinical practice: When rigorously designed, single-case studies can be particularly useful experimental designs in a variety of situations, such as when research resources are limited, studied conditions have low incidences, or when examining effects of novel or expensive interventions. Readers will be directed to examples from the published literature in which these techniques have been discussed, evaluated for quality, and implemented.

• Cohort Studies
• Medical Records*
• Quality Assurance, Health Care*
• Randomized Controlled Trials as Topic
• Research Design*

Grants and funding

• R21 HD076092/HD/NICHD NIH HHS/United States

A systematic review of applied single-case research published between 2016 and 2018: Study designs, randomization, data aspects, and data analysis

• Published: 26 October 2020
• Volume 53 , pages 1371–1384, ( 2021 )

Cite this article

• René Tanious 1 &
• Patrick Onghena 1  

5737 Accesses

29 Citations

21 Altmetric

Explore all metrics

Single-case experimental designs (SCEDs) have become a popular research methodology in educational science, psychology, and beyond. The growing popularity has been accompanied by the development of specific guidelines for the conduct and analysis of SCEDs. In this paper, we examine recent practices in the conduct and analysis of SCEDs by systematically reviewing applied SCEDs published over a period of three years (2016–2018). Specifically, we were interested in which designs are most frequently used and how common randomization in the study design is, which data aspects applied single-case researchers analyze, and which analytical methods are used. The systematic review of 423 studies suggests that the multiple baseline design continues to be the most widely used design and that the difference in central tendency level is by far most popular in SCED effect evaluation. Visual analysis paired with descriptive statistics is the most frequently used method of data analysis. However, inferential statistical methods and the inclusion of randomization in the study design are not uncommon. We discuss these results in light of the findings of earlier systematic reviews and suggest future directions for the development of SCED methodology.

Similar content being viewed by others

The use of cronbach’s alpha when developing and reporting research instruments in science education.

The Impact of Peer Assessment on Academic Performance: A Meta-analysis of Control Group Studies

Ethical Considerations of Conducting Systematic Reviews in Educational Research

Avoid common mistakes on your manuscript.

Introduction

In single-case experimental designs (SCEDs) a single entity (e.g., a classroom) is measured repeatedly over time under different manipulations of at least one independent variable (Barlow et al., 2009 ; Kazdin, 2011 ; Ledford & Gast, 2018 ). Experimental control in SCEDs is demonstrated by observing changes in the dependent variable(s) over time under the different manipulations of the independent variable(s). Over the past few decades, the popularity of SCEDs has risen continuously as reflected in the number of published SCED studies (Shadish & Sullivan, 2011 ; Smith, 2012 ; Tanious et al., 2020 ), the development of domain-specific reporting guidelines (e.g., Tate et al., 2016a , 2016b ; Vohra et al., 2016 ), and guidelines on the quality of conduct and analysis of SCEDs (Horner, et al., 2005 ; Kratochwill et al., 2010 , 2013 ).

The What Works Clearinghouse guidelines

In educational science in particular, the US Department of Education has released a highly influential policy document through its What Works Clearinghouse (WWC) panel (Kratochwill et al., 2010 ) Footnote 1 . The WWC guidelines contain recommendations for the conduct and visual analysis of SCEDs. The panel recommended visually analyzing six data aspects of SCEDs: level, trend, variability, overlap, immediacy of the effect, and consistency of data patterns. However, given the subjective nature of visual analysis (e.g., Harrington, 2013 ; Heyvaert & Onghena, 2014 ; Ottenbacher, 1990 ), Kratochwill and Levin ( 2014 ) later called the formation of a panel for recommendations on the statistical analysis of SCEDs “ the highest imminent priority” (p. 232, emphasis in original) on the agenda of SCED methodologists. Furthermore, Kratochwill and Levin—both members of the original panel—contended that advocating for design-specific randomization schemes in line with the recommendations by Edgington ( 1975 , 1980 ) and Levin ( 1994 ) would constitute an important contribution to the development of updated guidelines.

Developments outside the WWC guidelines

Prior to the publication of updated guidelines, important progress had already been made in the development of SCED-specific statistical analyses and design-specific randomization schemes not summarized in the 2010 version of the WWC guidelines. Specifically, three interrelated areas can be distinguished: effect size calculation, inferential statistics, and randomization procedures. Note that this list includes effect size calculation even though the 2010 WWC guidelines include some recommendations for effect size calculation, but with the reference that further research is “badly needed” (p. 23) to develop novel effect size measures comparable to those used in group studies. In the following paragraphs, we give a brief overview of the developments in each area.

Effect size measures

The effect size measures mentioned in the 2010 version of the WWC guidelines mainly concern the data aspect overlap: percentage of non-overlapping data (Scruggs, Mastropieri, & Casto, 1987 ), percentage of all non-overlapping data (Parker et al., 2007 ), and percentage of data points exceeding the median (Ma, 2006 ). Other overlap-based effect size measures are discussed in Parker et al. ( 2011 ). Furthermore, the 2010 guidelines discuss multilevel models, regression models, and a standardized effect size measure proposed by Shadish et al. ( 2008 ) for comparing results between participants in SCEDs. In later years, this measure has been further developed for other designs and meta-analyses (Hedges et al., 2012 ; Hedges et al., 2013 ; Shadish et al., 2014 ) Without mentioning any specific measures, the guidelines further mention effect sizes that compare the different conditions within a single unit and standardize by dividing by the within-phase variance. These effect size measures quantify the data aspect level. Beretvas and Chung ( 2008 ) proposed for example to subtract the mean of the baseline phase from the mean of the intervention phase, and subsequently divide by the pooled within-case standard deviation. Other proposals for quantifying the data aspect level include the slope and level change procedure which corrects for baseline trend (Solanas et al., 2010 ), and the mean baseline reduction which is calculated by subtracting the mean of treatment observations from the mean of baseline observations and subsequently dividing by the mean of the baseline phase (O’Brien & Repp, 1990 ). Efforts have also been made to quantify the other four data aspects. For an overview of the available effect size measures per data aspect, the interested reader is referred to Tanious et al. ( 2020 ). Examples of quantifications for the data aspect trend include the split-middle technique (Kazdin, 1982 ) and ordinary least squares (Kromrey & Foster-Johnson, 1996 ), but many more proposals exist (see e.g., Manolov, 2018 , for an overview and discussion of different trend techniques). Fewer proposals exist for variability, immediacy, and consistency. The WWC guidelines recommend using the standard deviation for within-phase variability. Another option is the use of stability envelopes as suggested by Lane and Gast ( 2014 ). It should be noted, however, that neither of these methods is an effect size measure because they are assessed within a single phase. For the assessment of between-phase variability changes, Kromrey and Foster-Johnson ( 1996 ) recommend using variance ratios. More recently, Levin et al. ( 2020 ) recommended the median absolute deviation for the assessment of variability changes. The WWC guidelines recommend subtracting the mean of the last three baseline data points from the first three intervention data points to assess immediacy. Michiels et al. ( 2017 ) proposed the immediate treatment effect index extending this logic to ABA and ABAB designs. For consistency of data patterns, only one measure currently exists, based on the Manhattan distance between data points from experimentally similar phases (Tanious et al., 2019 ).

Inferential statistics

Inferential statistics are not summarized in the 2010 version of the WWC guidelines. However, inferential statistics do have a long and rich history in debates surrounding the methodology and data analysis of SCEDs. Excellent review articles detailing and explaining the available methods for analyzing data from SCEDs are available in Manolov and Moeyaert ( 2017 ) and Manolov and Solanas ( 2018 ). In situations in which results are compared across participants within or between studies, multilevel models have been proposed. The 2010 guidelines do mention multilevel models, but with the indication that more thorough investigation was needed before their use could be recommended. With few exceptions, such as the pioneering work by Van den Noortgate and Onghena ( 2003 , 2008 ), specific proposals for multilevel analysis of SCEDs had long been lacking. Not surprisingly, the 2010 WWC guidelines gave new impetus for the development of multilevel models for meta-analyzing SCEDs. For example, Moeyaert, Ugille, et al. ( 2014b ) and Moeyaert, Ferron, et al. ( 2014a ) discuss two-level and three-level models for combining results across single cases. Baek et al. ( 2016 ) suggested a visual analytical approach for refining multilevel models for SCEDs. Multilevel models can be used descriptively (i.e., to find an overall treatment effect size), inferentially (i.e., to obtain a p value or confidence interval), or a mix of both.

• Randomization

One concept that is closely linked to inferential statistics is randomization. In the context of SCEDs, randomization refers to the random assignment of measurements to treatment levels (Onghena & Edgington, 2005 ). Randomization, when ethically and practically feasible, can reduce the risk of bias in SCEDs and strengthen the internal validity of the study (Tate et al., 2013 ). To incorporate randomization into the design, specific randomization schemes are needed, as previously stated (Kratochwill & Levin, 2014 ). In alternation designs, randomization can be introduced by randomly alternating the sequence of conditions, either unrestricted or restricted (e.g., maximum of two consecutive measurements under the same condition) (Onghena & Edgington, 1994 ). In phase designs (e.g., ABAB), multiple baseline designs, and changing criterion designs, where no rapid alternation of treatments takes place, it is possible to randomize the moment of phase change after a minimum number of measurements has taken place in each phase (Marascuilo & Busk, 1988 ; Onghena, 1992 ). In multiple baseline designs, it is also possible to predetermine different baseline phase lengths for each tier and then randomly allocate participants to different baseline phase lengths (Wampold & Worsham, 1986 ). Randomization tests use the randomization actually present in the design for quantifying the probability of the observed effect occurring by chance. These tests are among the earliest data analysis techniques specifically proposed for SCEDs (Edgington, 1967 , 1975 , 1980 ).

The main aim of the present paper is to systematically review the methodological characteristics of recently published SCEDs with an emphasis on the data aspects put forth in the WWC guidelines. Specific research questions are:

What is the frequency of the various single-case design options?

How common is randomization in the study design?

Which data aspects do applied researchers include in their analysis?

What is the frequency of visual and statistical data analysis techniques?

For systematic reviews of SCEDs predating the publication of the WWC guidelines, the interested reader is referred to Hammond and Gast ( 2010 ), Shadish and Sullivan ( 2011 ), and Smith ( 2012 ).

Justification for publication period selection

The present systematic review deals with applied SCED studies published in the period from 2016 to 2018. The reasons for the selection of this period are threefold: relevance, sufficiency, and feasibility. In terms of relevance, there is a noticeable lack of recent systematic reviews dealing with the methodological characteristics of SCEDs in spite of important developments in the field. Apart from the previously mentioned reviews predating the publication of the 2010 WWC guidelines, only two reviews can be mentioned that were published after the WWC guidelines. Solomon ( 2014 ) reviewed indicators of violations of normality and independence in school-based SCED studies until 2012. More recently, Woo et al. ( 2016 ) performed a content analysis of SCED studies published in American Counseling Association journals between 2003 and 2014. However, neither of these reviews deals with published SCEDs in relation to specific guidelines such as WWC. In terms of sufficiency, a three-year period can give sufficient insight into recent trends in applied SCEDs. In addition, it seems reasonable to assume a delay between the publication of guidelines such as WWC and their impact in the field. For example, several discussion articles regarding the WWC guidelines were published in 2013. Wolery ( 2013 ) and Maggin et al. ( 2013 ) pointed out perceived weaknesses in the WWC guidelines, which in turn prompted a reply by the original authors (Hitchcock et al., 2014 ). Discussions like these can help increase the exposure of the guidelines among applied researchers. In terms of feasibility, it is important to note that we did not set any specification on the field of study for inclusion. Therefore, the period of publication had to remain feasible and manageable to read and code all included publications across all different study fields (education, healthcare, counseling, etc.).

Data sources

We performed a broad search of the English-language SCED literature using PubMed and Web of Science. The choice for these two search engines was based on Gusenbauer and Haddaway ( 2019 ), who assessed the eligibility of 26 search engines for systematic reviews. Gusenbauer and Haddaway came to the conclusion that PubMed and Web of Science could be used as primary search engines in systematic reviews, as they fulfilled all necessary requirements such as functionality of Boolean operators and reproducibility of search results in different locations and at different times. We selected only these two of all eligible search engines to keep the size of the project manageable and to prevent excessive overlap between the results. Table 1 gives an overview of the search terms we used and the number of hits per search query. This list does not exclude duplicates between the search terms and between the two search engines. For all designs containing the term “randomized” (e.g., randomized block design) we added the Boolean operator AND specified that the search results must also contain either the term “single-case” or “single-subject”. An initial search for randomized designs without these specifications yielded well over 1000 results per search query.

Study selection

We specifically searched for studies published between 2016 and 2018. We used the date of first online publication to determine whether an article met this criterion (i.e., articles that were published online during this period, even if not yet published in print). Initially, the abstracts and article information of all search results were scanned for general exclusion criteria. In a first step, all articles that fell outside the date range of interest were excluded, as well as articles for which the full text was not available or only available against payment. We only included articles written in English. In a second step, all duplicate articles were deleted. From the remaining unique search results, all articles that did not use any form of single-case experimentation were excluded. Such studies include for example non-experimental forms of case studies. Lastly, all articles not reporting any primary empirical data were excluded from the final sample. Thus, purely methodological articles were discarded. Methodological articles were defined as articles that were within the realm of SCEDs but did not report any empirical data or reported only secondary empirical data. Generally, these articles propose new methods for analyzing SCEDs or perform simulation studies to test existing methods. Similarly, commentaries, systematic reviews, and meta-analyses were excluded from the final sample, as such articles do not contain primary empirical data. In line with systematic review guidelines (Staples & Niazi, 2007 ), the second author verified the accuracy of the selection process. Ten articles were randomly selected from an initial list of all search results for a joint discussion between the authors, and no disagreements about the selection emerged. Figure 1 presents the study attrition diagram.

Study attrition diagram

Coding criteria

For all studies, the basic design was coded first. For coding the design, we followed the typology presented in Onghena and Edgington ( 2005 ) and Tate et al. ( 2016a ) with four overarching categories: phase designs, alternation designs, multiple baseline designs, and changing criterion designs. For each of these categories, different design options exist. Common variants of phase designs include for example AB and ABAB, but other forms also exist, such as ABC. Within the alternation designs category the main variants are the completely randomized design, the alternating treatments designs, and the randomized block design. Multiple baseline designs can be conducted across participants, behaviors, or settings. They can be either concurrent, meaning that all participants start the study at the same time, or non-concurrent. Changing criterion designs can employ either a single-value criterion or a range-bound criterion. In addition to these four overarching categories, we added a design category called hybrid Footnote 2 . The hybrid category consists of studies using several design strategies combined, for example a multiple baseline study with an integrated alternating treatments design. For articles reporting more than one study, each study was coded separately. For coding the basic design, we followed the authors’ original description of the study.

Randomization was coded as a dichotomous variable, i.e., either present or not present. In order to be coded as present, some form of randomization had to be present in the design itself, as previously defined in the randomization section. Studies with a fixed order of treatments or phase change moments with randomized stimulus presentation, for example, were coded as randomization not present.

Data aspect

A major contribution of the WWC guidelines was the establishment of six data aspects for the analysis of SCEDs: level, trend, variability, overlap, immediacy, and consistency. Following the guidelines, these data aspects can be defined operationally as follows. Level is the mean score within a phase. The straight line best fitting the data within a phase refers to the trend. The standard deviation or range in a phase represents the data aspect variability. The proportion of data points overlapping between adjacent phases is the data aspect overlap. The immediacy of an effect is assessed by a comparison of the last three data points of an intervention with the first three data points of the subsequent intervention. Finally, consistency Footnote 3 is assessed by comparing data patterns from experimentally similar interventions. In multiple baseline designs, consistency can be assessed horizontally (within series) when more than one phase change is present, and vertically (across series) by comparing experimentally similar phases across participants, behaviors, or settings. It was of course possible that studies reported more than one data aspect or none at all. For studies reporting more than one data aspect, each data aspect was coded separately.

Data analysis

The data analysis methods were coded directly from the authors’ description in the “data analysis” section. If no such section was present, the data analysis methods were coded according to the presentation of the results. Generally, two main forms of data analysis for SCEDs can be distinguished: visual and statistical analysis. In the visual analytical approach, a time series graph of the dependent variable under the different experimental conditions is analyzed to determine treatment effectiveness. The statistical analytical approach can be roughly divided into two categories: descriptive and inferential statistics. Descriptive statistics summarize the data without quantifying the uncertainty in the description. Examples of descriptive statistics include means, standard deviations, and effect sizes. Inferential statistics imply an inference from the observed results to unknown parameter values and quantify the uncertainty for doing so, for example, by providing p values and confidence intervals.

Number of participants

Finally, for each study we coded the number of participants, counting only participants who appeared in the results section. Participants who dropped out prematurely and whose data were not analyzed, were not counted.

General results

For each coding category, the interrater agreement was calculated with the formula \( \frac{\mathrm{no}.\kern0.5em \mathrm{of}\ \mathrm{agreements}}{\mathrm{no}.\kern0.5em \mathrm{of}\ \mathrm{agreements}+\mathrm{no}.\kern0.5em \mathrm{of}\ \mathrm{disagreements}} \) based on ten randomly selected articles. The interrater agreement was as follow: design (90%), analysis (60%), data aspect (80%), randomization (100%), number of participants (80%). Given the initial moderate agreement for analysis, the two authors discussed discrepancies and then reanalyzed a new sample of ten randomly selected articles. The interrater reliability for analysis then increased to 90%.

In total, 406 articles were included in the final sample, which represented 423 studies. One hundred thirty-eight of the 406 articles (34.00%) were published in 2016, 150 articles (36.95%) were published in 2017, and 118 articles (29.06%) were published in 2018. Out of the 423 studies, the most widely used form of SCEDs was the multiple baseline design, which accounted for 49.65% ( N  = 210) of the studies included in the final sample. Across all studies and designs, the median number of participants was three (IQR = 3). The most popular data analysis technique across all studies was visual analysis paired with descriptive statistics, which was used in 48.94% ( N  = 207) of the studies. The average number of data aspects analyzed per study was 2.61 ( SD =  1.63). The most popular data aspect across all designs and studies was level (83.45%, N =  353). Overall, 22.46% ( N  = 95) of the 423 studies included randomization in the design. However, these results vary between the different designs. In the following sections, we therefore present a summary of the results per design. A detailed overview of all the results per design can be found in Table 2 .

Results per design

Phase designs.

Phase designs accounted for 25.53% ( N  = 108) of the studies included in the systematic review. The median number of participants for phase designs was three (IQR = 4). Visual analysis paired with descriptive statistics was the most popular data analysis method for phase designs (40.74%, N  = 44), and the majority of studies analyzed several data aspects (54.62%, N  = 59); 20.37% ( N  = 22) did not report any of the six data aspects. The average number of data aspects analyzed in phase designs was 2.02 ( SD =  2.07). Level was the most frequently analyzed data aspect for phase designs (73.15%, N  = 79). Randomization was very uncommon in phase designs and was included in only 5.56% ( N  = 6) of the studies.

Alternation designs

Alternation designs accounted for 14.42% ( N  = 61) of the studies included in the systematic review. The median number of participants for alternation designs was three (IQR = 1). More than half of the alternation design studies used visual analysis paired with descriptive statistics (57.38%, N  = 35). The majority of alternation design studies analyzed several data aspects (75.41%, N  = 46), while 11.48% ( N  = 7) did not report which data aspect was the focus of analysis. The average number of data aspects analyzed in alternation designs was 2.38 ( SD =  2.06). The most frequently analyzed data aspect for alternation designs was level (85.25%, N =  52). Randomization was used in the majority of alternation designs (59.02%, N  = 36).

Multiple baseline designs

Multiple baseline designs, by a large margin the most prevalent design, accounted for nearly half of all studies (49.65%, N  = 210) included in the systematic review. The median number of participants for multiple baseline designs was four (IQR = 4). A total of 49.52% ( N  = 104) of multiple baseline studies were analyzed using visual analysis paired with descriptive statistics, and the vast majority (80.95%, N  = 170) analyzed several data aspects, while only 7.14% ( N  = 15) did not report any of the six data aspects. The average number of data aspects analyzed in multiple baseline designs was 3.01 ( SD =  1.61). The most popular data aspect was level, which was analyzed in 87.62% ( N =  184) of all multiple baseline designs. Randomization was not uncommon in multiple baseline designs (20.00%, N  = 42).

Changing criterion design

Changing criterion designs accounted for 1.42% ( N  = 6) of the studies included in the systematic review. The median number of participants for changing criterion designs was three (IQR = 0); 66.67% ( N =  4) of changing criterion designs were analyzed using visual analysis paired with descriptive statistics. Half of the changing criterion designs analyzed several data aspects ( N =  3), and one study (16.67%) did not report any data aspect. The average number of data aspects analyzed in changing criterion designs was 1.83 ( SD =  1.39). The most popular data aspect was level (83.33%, N  = 5). None of the changing criterion design studies included randomization in the design.

Hybrid designs

Hybrid designs accounted for 8.98% ( N  = 38) of the studies included in the systematic review. The median number of participants for hybrid designs was three (IQR = 2). A total of 52.63% ( N  = 20) of hybrid designs were analyzed with visual analysis paired with descriptive statistics, and the majority of studies analyzed several data aspects (73.68%, N  = 28); 10.53% ( N  = 4) did not report any of the six data aspects. The average number of data aspects considered for analysis was 2.55 ( SD =  2.02). The most popular data aspect was level (86.84%, N  = 33). Hybrid designs showed the second highest proportion of studies including randomization in the study design (28.95%, N  = 11).

Results per data aspect

Out of the 423 studies included in the systematic review, 72.34% ( N =  306) analyzed several data aspects, 16.08% ( N =  68) analyzed one data aspect, and 11.58% ( N =  49) did not report any of the six data aspects.

Across all designs, level was by far the most frequently analyzed data aspect (83.45%, N =  353). Remarkably, nearly all studies that analyzed more than one data aspect included the data aspect level (96.73%, N =  296). Similarly, for studies analyzing only one data aspect, there was a strong prevalence of level (83.82%, N =  57). For studies that only analyzed level, the most common form of analysis was visual analysis paired with descriptive statistics (54.39%, N =  31).

Trend was the third most popular data aspect. It was analyzed in 45.39% ( N =  192) of all studies included in the systematic review. There were no studies in which trend was the only data aspect analyzed, meaning that trend was always analyzed alongside other data aspects, making it difficult to isolate the analytical methods specifically used to analyze trend.

Variability

The data aspect variability was analyzed in 59.10% ( N =  250) of the studies, making it the second most prominent data aspect. A total of 80.72% ( N =  247) of all studies analyzing several data aspects included variability. However, variability was very rarely the only data aspect analyzed. Only 3.3% ( N =  3) of the studies analyzing only one data aspect focused on variability. All three studies that analyzed only variability did so using visual analysis.

The data aspect overlap was analyzed in 35.70% ( N =  151) of all studies and was thus the fourth most analyzed data aspect. Nearly half of all studies analyzing several data aspects included overlap (47.08%, N =  144). For studies analyzing only one data aspect, overlap was the second most common data aspect after level (10.29%, N =  7). The most common mode of analysis for these studies was descriptive statistics paired with inferential statistics (57.14%, N =  4).

The immediacy of the effect was assessed in 28.61% ( N =  121) of the studies, making it the second least analyzed data aspect; 39.22% ( N =  120) of the studies analyzing several data aspects included immediacy. Only one study analyzed immediacy as the sole data aspect, and this study used visual analysis.

Consistency

Consistency was analyzed in 9.46% ( N =  40) of the studies and was thus by far the least analyzed data aspect. It was analyzed in 13.07% ( N =  40) of the studies analyzing several data aspects and was never the focus of analysis for studies analyzing only one data aspect.

As stated previously, 72.34% ( N =  306) of all studies analyzed several data aspects. For these studies, the average number of data aspects analyzed was 3.39 ( SD =  1.18). The most popular data analysis technique for several data aspects was visual analysis paired with descriptive statistics (56.54%, N =  173).

Not reported

As mentioned previously, 11.58% ( N =  49) did not report any of the six data aspects. For these studies, the most prominent analytical technique was visual analysis alone (61.22%, N =  30). Of all studies not reporting any of the six data aspects, the highest proportion was phase designs (44.90%, N =  22).

Results per analytical method

Visual analysis, without the use of any descriptive or inferential statistics, was the analytical method used in 16.78% ( N =  71) of all included studies. Of all studies using visual analysis, the majority were multiple baseline design studies (45.07%, N =  32). The majority of studies using visual analysis did not report any data aspect (42.25%, N =  30), closely followed by several data aspects (40.85%, N =  29). Randomization was present in 20.53% ( N =  16) of all studies using visual analysis.

Descriptive statistics

Descriptive statistics, without the use of visual analysis, was the analytical method used in 3.78% ( N =  16) of all included studies. The most common designs for studies using descriptive statistics were phase designs and multiple baseline designs (both 43.75%, N =  7). Half of the studies using descriptive statistics (50.00%, N =  8) analyzed the data aspect level, and 37.5% ( N =  6) analyzed several data aspects. One study (6.25%) using descriptive statistics included randomization.

Inferential statistics, without the use of visual analysis, was the analytical method used in 2.84% ( N =  12) of all included studies. The majority of studies using inferential statistics were phase designs (58.33%, N =  7) and did not report any of the six data aspects (58.33%, N =  7). Of the remaining studies, three (25.00%) reported several data aspects, and two (16.67%) analyzed the data aspect level. Two studies (16.67) using inferential statistical analysis included randomization.

Descriptive and inferential statistics

Descriptive statistics combined with inferential statistics, but without the use of visual analysis, accounted for 5.67% ( N  = 24) of all included studies. The majority of studies using this combination of analytical methods were multiple baseline designs (62.5%, N =  15), followed by phase designs (33.33%, N =  8). There were no alternation or hybrid designs using descriptive and inferential statistics. Most of the studies using descriptive and inferential statistics analyzed several data aspects (41.67%, N =  10), followed by the data aspect level (29.17%, N =  7); 16.67% ( N =  4) of the studies using descriptive and inferential statistics included randomization.

Visual and descriptive statistics

As mentioned previously, visual analysis paired with descriptive statistics was the most popular analytical method. This method was used in nearly half (48.94%, N  = 207) of all included studies. The majority of these studies were multiple baseline designs (50.24%, N =  104), followed by phase designs (21.25%, N =  44). This method of analysis was prevalent across all designs. Nearly all of the studies using this combination of analytical methods analyzed either several data aspects (83.57%, N =  173) or level only (14.98%, N =  31). Randomization was present in 19.81% ( N =  41) of all studies using visual and descriptive analysis.

Visual and inferential statistics

Visual analysis paired with inferential statistics accounted for 2.60% ( N  = 11) of the included studies. The largest proportion of these studies were phase designs (45.45%, N  = 5), followed by multiple baseline designs and hybrid designs (both 27.27%, N =  3). This combination of analytical methods was thus not used in alternation or changing criterion designs. The majority of studies using visual analysis and inferential statistics analyzed several data aspects (72.73%, N =  8), while 18.18% ( N =  2) did not report any data aspect. One study (9.10%) included randomization.

Visual, descriptive, and inferential statistics

A combination of visual analysis, descriptive statistics, and inferential statistics was used in 18.44% ( N =  78) of all included studies. The majority of the studies using this combination of analytical methods were multiple baseline designs (56.41%, N =  44), followed by phase designs (23.08%, N =  18). This analytical approach was used in all designs except changing criterion designs. Nearly all studies using a combination of these three analytical methods analyzed several data aspects (97.44%, N =  76). These studies also showed the highest proportion of randomization (38.46%, N =  30).

None of the above

A small proportion of studies did not use any of the above analytical methods (0.95%, N =  4). Three of these studies (75%) were phase designs and did not report any data aspect. One study (25%) was a multiple baseline design that analyzed several data aspects. Randomization was not used in any of these studies.

To our knowledge, the present article is the first systematic review of SCEDs specifically looking at the frequency of the six data aspects in applied research. The systematic review has shown that level is by a large margin the most widely analyzed data aspect in recently published SCEDs. The second most popular data aspect from the WWC guidelines was variability, which was usually assessed alongside level (e.g., a combination of mean and standard deviation or range). The fact that these two data aspects are routinely assessed in group studies may be indicative of a lack of familiarity with SCED-specific analytical methods by applied researchers, but this remains speculative. Phase designs showed the highest proportion of studies not reporting any of the six data aspects and the second lowest number of data aspects analyzed on average, only second to changing criterion designs. This was an unexpected finding given that the WWC guidelines were developed specifically in the context of (and with examples of) phase designs. The multiple baseline design showed the highest number of data aspects analyzed and at the same time the lowest proportion of studies not analyzing any of the six data aspects.

These findings regarding the analysis and reporting of the six data aspects need more contextualization. The selection of data aspects for the analysis depends on the research questions and expected data pattern. For example, if the aim of the intervention is a gradual change over time, then trend becomes more important. If the aim of the intervention is a change in level, then it is import to also assess trend (to verify that the change in level is not just a continuation of a baseline trend) and variability (to assess whether the change in level is caused by excessive variability). In addition, assessing consistency can add information on whether the change in level is consistent over several repetitions of experimental conditions (e.g., in phase designs). Similarly, if an abrupt change in level of target behavior is expected after changing experimental conditions, then immediacy becomes a more relevant data aspect in addition to trend, variability, and level. The important point here is that oftentimes the research team has an idea of the expected data pattern and should choose the analysis of data aspects accordingly. The strong prevalence of level found in the present review could be indicative of a failure to assess other data aspects that may be relevant to demonstrate experimental control over an independent variable.

In line with the findings of earlier systematic reviews (Hammond & Gast, 2010 ; Shadish & Sullivan, 2011 ; Smith, 2012 ), the multiple baseline design continues to be the most frequently used design, and despite the advancement of sophisticated statistical methods for the analysis of SCEDs, two thirds of all studies still relied on visual analysis alone or visual analysis paired with descriptive statistics. A comparison to the findings of Shadish and Sullivan further reveals that the number of participants included in SCEDs has remained steady over the past decade at around three to four participants. The relatively small number of changing criterion designs in the present findings is partly due to the fact that changing criterion designs were often combined with other designs and thus coded in the hybrid category, even though we did not formally quantify that. This finding is supported by the results of Shadish and Sullivan, who found that changing criterion designs are more often used as part of hybrid designs than as a standalone design. Hammond and Gast even excluded changing criterion design from their review due to its low prevalence. They found a total of six changing criterion designs published over a period of 35 years. It should be noted, however, that the low prevalence of changing criterion designs is not indicative of the value of this design.

Regarding randomization, the results cannot be interpreted against earlier benchmarks, as neither Smith nor Shadish and Sullivan or Hammond and Gast quantified the proportion of randomized SCEDs. Overall, randomization in the study design was not uncommon. However, the proportion of randomized SCEDs differed greatly between different designs. The results showed that alternating treatments designs have the highest proportion of studies including randomization. This result was to be expected given that alternating treatments designs are particularly suited to incorporate randomization. In fact, when Barlow and Hayes ( 1979 ) first introduced the alternating treatments design, they emphasized randomization as an important part of the design: “Among other considerations, each design controls for sequential confounding by randomizing the order of treatment […]” (p. 208). Besides that, alternating treatments designs could work with already existing randomization procedures, such as the randomized block procedure proposed by Edgington ( 1967 ). The different design options for alternating treatments designs (e.g., randomized block design) and accompanying randomization procedures are discussed in detail in Manolov and Onghena ( 2018 ). For multiple baseline designs, a staggered introduction of the intervention is needed. Proposals to randomize the order of the introduction of the intervention have been around since the 1980s (Marascuilo & Busk, 1988 ; Wampold & Worsham, 1986 ). These randomization procedures have their counterparts in group studies where particpants are randomdly assigned to treatments or different blocks of treatments. Other randomization procedures for multiple baseline designs are discussed in Levin et al. ( 2018 ). These include the restricted Marascuilo–Busk procedure proposed by Koehler and Levin and the randomization test procedure proposed by Revusky. For phase designs and changing criterion designs, the incorporation of randomization is less evident. For phase designs, Onghena ( 1992 ) proposed a method to randomly determine the moment of phase change between two succesive phases. However, this method is rather uncommon and has no counterpart in group studies. Specific randomization schemes for changing criterion designs have only very recently been proposed (Ferron et al., 2019 ; Manolov et al., 2020 ; Onghena et al., 2019 ), and it remains to be seen how common they will become in applied SCEDs.

Implications for SCED research

The results of the systematic review have several implications for SCED research regarding methodology and analyses. An important finding of the present study is that the frequency of use of randomization differs greatly between different designs. For example, while phase designs were found to be the second most popular design, randomization is used very infrequently for this design type. Multiple baseline designs, as the most frequently used design, showed a higher percentage of randomized studies, but only every fifth study used randomization. Given that randomization in the study design increases the internal and statistical conclusion validity irrespective of the design, it seems paramount to further stress the importance of the inclusion of randomization beyond alternating treatments designs. Another implication concerns the analysis of specific data aspects. While level was by a large margin the most popular data aspect, it is important to stress that conclusions based on only one data aspect may be misleading. This seems particularly relevant for phase designs, which were found to contain the highest proportion of studies not reporting any of the six data aspects and the lowest proportion of studies analyzing several data aspects (apart from changing criterion designs, which only accounted for a very small proportion of the included studies). A final implication concerns the use of analytical methods, in particular triangulation of different methods. Half of the included studies used visual analysis paired with descriptive statistics. These methods should of course not be discarded, as they generate important information about the data, but they cannot make statements regarding the uncertainty of a possible intervention effect. Therefore, triangulation of visual analysis, descriptive statistics, and inferential statistics should form an important part of future guidelines on SCED analysis.

Reflections on updated WWC guidelines

Updated WWC guidelines were recently published, after the present systematic review had been conducted (What Works Clearinghouse, 2020a , 2020c ). Two major changes in the updated guidelines are of direct relevance to the present systematic review: (a) the removal of visual analysis for demonstrating intervention effectiveness and (b) recommendation for a design comparable effect size measure for demonstrating intervention effects (D-CES, Pustejovsky et al., 2014 ; Shadish et al., 2014 ). This highlights a clear shift away from visual analysis towards statistical analysis of SCED data, especially compared to the 2010 guidelines. These changes in the guidelines have prompted responses from the public, to which What Works Clearinghouse ( 2020b ) published a statement addressing the concerns. Several concerns relate to the removal of visual analysis. In response to a concern that visual analysis should be reinstated, the panel clearly states that “visual analysis will not be used to characterize study findings” (p. 3). Another point from the public concerned the analysis of studies where no effect size can be calculated (e.g., due to unavailability of raw data). Even in these instances, the panel does not recommend visual analysis. Rather, “the WWC will extract raw data from those graphs for use in effect size computation” (p. 4). In light of the present findings, these statements are particularly noteworthy. Given that the present review found a strong continued reliance on visual analysis, it remains to be seen if and how the updated WWC guidelines impact the analyses conducted by applied SCED researchers.

Another update of relevance in the recent guidelines concerns the use of design categories. While the 2010 guidelines were demonstrated with the example of a phase design, the updated guidelines include quality rating criteria for each major design option. Given that the present results indicate a very low prevalence of the changing criterion design in applied studies, the inclusion of this design in the updated guidelines may increase the prominence of the changing criterion design. For changing criterion designs, the updated guidelines recommend that “the reversal or withdrawal (AB) design standards should be applied to changing criterion designs” (What Works Clearinghouse, 2020c , p. 80). With phase designs being the second most popular design choice, this could further facilitate the use of the changing criterion design.

While other guidelines on conduct and analysis (e.g., Tate et al., 2013 ), as well as members of the 2010 What Works Clearinghouse panel (Kratochwill & Levin, 2014 ), have clearly highlighted the added value of randomization in the design, the updated guidelines do not include randomization procedures for SCEDs. Regarding changes between experimental conditions, the updated guidelines state that “the independent variable is systematically manipulated, with the researcher determining when and how the independent variable conditions change” (What Works Clearinghouse, 2020c , p. 82). While the frequency of use of randomization differs considerably between different designs, the present review has shown that overall randomization is not uncommon. The inclusion of randomization in the updated guidelines may therefore have offered guidance to applied researchers wishing to incorporate randomization into their SCEDs, and may have further contributed to the popularity of randomization.

Limitations and future research

One limitation of the current study concerns the used databases. SCEDs that were published in journals that are not indexed in these databases may not have been included in our sample. A similar limitation concerns the search terms used in the systematic search. In this systematic review, we focused on the common names “single-case” and “single-subject.” However, as Shadish and Sullivan ( 2011 ) note, SCEDs go by many names. They list several less common alternative terms: instrasubject replication design (Gentile et al., 1972 ), n -of-1 design (Center et al., 1985 -86), intrasubject experimental design (White et al., 1989 ), one-subject experiment (Edgington, 1980 ), and individual organism research (Michael, 1974 ). Even though these terms date back to the 1970s and 1980s, a few authors may still use them to describe their SCED studies. Studies using these terms may not have come up during the systematic search. It should furthermore be noted that we followed the original description provided by the authors for the coding of the design and analysis to reduce bias. We therefore made no judgments regarding the correctness or accuracy of the authors’ naming of the design and analysis techniques.

The systematic review offers several avenues for future research. The first avenue may be to explore more in depth the reasons for the unequal distribution of data aspects. As the systematic review has shown, level is assessed far more often than the other five data aspects. While level is an important data aspect, failing to assess it alongside other data aspects can lead to erroneous conclusions. Gaining an understanding of the reasons for the prevalence of level, for example through author interviews or questionnaires, may help to improve the quality of data analysis in applied SCEDs.

In a similar vein, a second avenue of future research may explore why randomization is much more prevalent in some designs. Apart from the aforementioned differences in randomization procedures between designs, it may be of interest to gain a better understanding of the reasons that applied researchers see for randomizing their SCEDs. As the incorporation of randomization enhances the internal validity of the study design, promoting the inclusion of randomization for designs other than alternation designs will help in advancing the credibility of SCEDs in the scientific community. Searching the methodological sections of the articles that used randomization may be a first step to gain a better understanding of why applied researchers use randomization. Such a text search may reveal how the authors discuss randomization and which reasons they name for randomizing. A related question is how the randomization was actually carried out. For example, was the randomization carried out a priori or in a restricted way taking into account the evolving data pattern? A deeper understanding of the reasons for randomizing and the mechanisms of randomization may be gained by author interviews or questionnaires.

A third avenue of future research may explore in detail the specifics of inferential analytical methods used to analyze SCED data. Within the scope of the present review, we only distinguished between visual, descriptive and inferential statistics. However, deeper insight into the inferential analysis methods and their application to SCED data may help to understand the viewpoint of applied researchers. This may be achieved through a literature review of articles that use inferential analysis. Research questions for such a review may include: Which inferential methods do applied SCED researchers use and what is the frequency of these methods? Are these methods adapted to SCED methodology? And how do applied researchers justify their choice for an inferential method? Similar questions may also be answered for effect size measures understood as descriptive statistics. For example, why do applied researchers choose a particular effect size measure over a competing one? Are these effect size measures adapted to SCED research?

Finally, future research may go into greater detail about the descriptive statistics used in SCEDs. In the present review, we distinguished between two major categories: descriptive and inferential statistics. Effect sizes that were not accompanied by a standard error, confidence limits, or by the result of a significance test were coded in the descriptive statistics category. Effect sizes do however go beyond merely summarizing the data by quantifying the treatment effect between different experimental conditions, contrary to within phase quantifications such as the mean and standard deviation. Therefore, future research may examine in greater detail the use of effect sizes separately from other descriptive statistics such the mean and standard deviation. Such research could focus in depth on the exact methods used to quantify each data aspect in the form of either a quantification (e.g., mean or range) or an effect size measure (e.g., standardized mean difference or variance ratios).

The What Works Clearinghouse panel ( 2020a , 2020c ) has recently released an updated version of the guidelines. We will discuss the updated guidelines in light of the present findings in the Discussion section.

As holds true for most single-case designs, the same design is often described with different terms. For example, Ledford and Gast ( 2018 ) call these designs combination designs, and Moeyaert et al. ( 2020 ) call them combined designs. Given that this is a purely terminological question, it is hard to argue in favor of one term over the other. We do, however, prefer the term hybrid, given that it emphasizes that neither of the designs remains in its pure form. For example, a multiple baseline design with alternating treatments is not just a combination of a multiple baseline design and an alternating treatments design. It is rather a hybrid of the two. This term is also found in recent literature (e.g., Pustejovski & Ferron, 2017 ; Swan et al., 2020 ).

For the present systematic review, we strictly followed the data aspects as outlined in the 2010 What Works Clearinghouse guidelines. While the assessment of consistency of effects is an important data aspect, this data aspect is not described in the guidelines. Therefore, we did not code it in the present review.

Baek, E. K., Petit-Bois, M., Van den Noortgate, W., Beretvas, S. N., & Ferron, J. M. (2016). Using visual analysis to evaluate and refine multilevel models of single-case studies. The Journal of Special Education, 50 , 18-26. https://doi.org/10.1177/0022466914565367 .

Article   Google Scholar  

Barlow, D. H., & Hayes, S. C. (1979). Alternating Treatments Design: One Strategy for Comparing the Effects of Two Treatments in a Single Subject. Journal of Applied Behavior Analysis, 12 , 199-210. https://doi.org/10.1901/jaba.1979.12-199 .

Article   PubMed   PubMed Central   Google Scholar  

Barlow, D. H., Nock, M. K., & Hersen, M. (2009). Single case experimental designs: Strategies for studying behavior change ( 3rd ). Pearson.

Beretvas, S. N., & Chung, H. (2008). A review of meta-analyses of single-subject experimental designs: Methodological issues and practice. Evidence-Based Communication Assessment and Intervention, 2 , 129-141. https://doi.org/10.1080/17489530802446302 .

Center, B. A., Skiba, R. J., & Casey, A. (1985-86). A Methodology for the Quantitative Synthesis of Intra-Subject Design research. Journal of Special Education, 19 , 387–400. https://doi.org/10.1177/002246698501900404 .

Edgington, E. S. (1967). Statistical inference from N=1 experiments. The Journal of Psychology, 65 , 195-199. https://doi.org/10.1080/00223980.1967.10544864 .

Article   PubMed   Google Scholar  

Edgington, E. S. (1975). Randomization tests for one-subject operant experiments. The Journal of Psychology, 90 , 57-68. https://doi.org/10.1080/00223980.1975.9923926 .

Edgington, E. S. (1980). Random assignment and statistical tests for one-subject experiments. Journal of Educational Statistics, 5 , 235-251.

Ferron, J., Rohrer, L. L., & Levin, J. R. (2019). Randomization procedures for changing criterion designs. Behavior Modification https://doi.org/10.1177/0145445519847627 .

Gentile, J. R., Roden, A. H., & Klein, R. D. (1972). An analysis-of-variance model for the intrasubject replication design. Journal of Applied Behavior Analysis, 5 , 193-198. https://doi.org/10.1901/jaba.1972.5-193 .

Gusenbauer, M., & Haddaway, N. R. (2019). Which academic search systems are suitable for systematic Reviews or meta-analyses? Evaluating retrieval qualities of Google Scholar, PubMed and 26 other Resources. Research Synthesis Methods https://doi.org/10.1002/jrsm.1378 .

Hammond, D., & Gast, D. L. (2010). Descriptive analysis of single subject research designs: 1983—2007. Education and Training in Autism and Developmental Disabilities, 45 , 187-202.

Google Scholar  

Harrington, M. A. (2013). Comparing visual and statistical analysis in single-subject studies. Open Access Dissertations , Retrieved from http://digitalcommons.uri.edu/oa_diss .

Hedges, L. V., Pustejovsky, J. E., & Shadish, W. R. (2012). A standardized mean difference effect size for single case designs. Research Synthesis Methods, 3 , 224-239. https://doi.org/10.1002/jrsm.1052 .

Hedges, L. V., Pustejovsky, J. E., & Shadish, W. R. (2013). A standardized mean difference effect size for multiple baseline designs across individuals. Research Synthesis Methods, 4 , 324-341. https://doi.org/10.1002/jrsm.1086 .

Heyvaert, M., & Onghena, P. (2014). Analysis of single-case data: Randomization tests for measures of effect size. Neuropsychological Rehabilitation, 24 , 507-527. https://doi.org/10.1080/09602011.2013.818564 .

Hitchcock, J. H., Horner, R. H., Kratochwill, T. R., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2014). The What Works Clearinghouse single-case design pilot standards: Who will guard the guards? Remedial and Special Education, 35 , 145-152. https://doi.org/10.1177/0741932513518979 .

Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71 , 165-179. https://doi.org/10.1177/001440290507100203 .

Kazdin, A. E. (1982). Single-case research designs: Methods for clinical and applied settings. Oxford University Press.

Kazdin, A. E. (2011). Single-case research designs: Methods for clinical and applied settings ( 2nd ). Oxford University Press.

Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single-case designs technical documentation. Retrieved from What Works Clearinghouse: https://files.eric.ed.gov/fulltext/ED510743.pdf

Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2013). Single-case intervention research design standards. Remedial and Special Education, 34 , 26-38. https://doi.org/10.1177/0741932512452794 .

Kratochwill, T. R., & Levin, J. R. (2014). Meta- and statistical analysis of single-case intervention research data: Quantitative gifts and a wish list. Journal of School Psychology, 52 , 231-235. https://doi.org/10.1016/j.jsp.2014.01.003 .

Kromrey, J. D., & Foster-Johnson, L. (1996). Determining the efficacy of intervention: The use of effect sizes for data analysis in single-subject research. The Journal of Experimental Education, 65 , 73-93. https://doi.org/10.1080/00220973.1996.9943464 .

Lane, J. D., & Gast, D. L. (2014). Visual analysis in single case experimental design studies: Brief review and guidelines. Neuropsychological Rehabilitation, 24 , 445-463. https://doi.org/10.1080/09602011.2013.815636 .

Ledford, J. R., & Gast, D. L. (Eds.) (2018). Single case research methodology: Applications in special education and behavioral sciences (3rd). Routledge.

Levin, J. R. (1994). Crafting educational intervention research that's both credible and creditable. Educational Psychology Review, 6 , 231-243. https://doi.org/10.1007/BF02213185 .

Levin, J. R., Ferron, J. M., & Gafurov, B. S. (2018). Comparison of randomization-test procedures for single-case multiple-baseline designs. Developmental Neurorehabilitation, 21 , 290-311. https://doi.org/10.1080/17518423.2016.1197708 .

Levin, J. R., Ferron, J. M., & Gafurov, B. S. (2020). Investigation of single-case multiple-baseline randomization tests of trend and variability. Educational Psychology Review . https://doi.org/10.1007/s10648-020-09549-7 .

Ma, H.-H. (2006). Quantitative synthesis of single-subject researches: Percentage of data points exceeding the median. Behavior Modification, 30 , 598-617. https://doi.org/10.1177/0145445504272974 .

Maggin, D. M., Briesch, A. M., & Chafouleas, S. M. (2013). An application of the What Works Clearinghouse standards for evaluating single-subject research: Synthesis of the self-management literature base. Remedial and Special Education, 34 , 44-58. https://doi.org/10.1177/0741932511435176 .

Manolov, R. (2018). Linear trend in single-case visual and quantitative analyses. Behavior Modification, 42 , 684-706. https://doi.org/10.1177/0145445517726301 .

Manolov, R., & Moeyaert, M. (2017). Recommendations for choosing single-case data analytical techniques. Behavior Therapy, 48 , 97-114. https://doi.org/10.1016/j.beth.2016.04.008 .

Manolov, R., & Onghena, P. (2018). Analyzing data from single-case alternating treatments designs. Psychological Methods, 23 , 480-504. https://doi.org/10.1037/met0000133 .

Manolov, R., & Solanas, A. (2018). Analytical options for single-case experimental designs: Review and application to brain impairment. Brain Impairment, 19 , 18-32. https://doi.org/10.1017/BrImp.2017.17 .

Manolov, R., Solanas, A., & Sierra, V. (2020). Changing Criterion Designs: Integrating Methodological and Data Analysis Recommendations. The Journal of Experimental Education, 88 , 335-350. https://doi.org/10.1080/00220973.2018.1553838 .

Marascuilo, L., & Busk, P. (1988). Combining statistics for multiple-baseline AB and replicated ABAB designs across subjects. Behavioral Assessment, 10 , 1-28.

Michael, J. (1974). Statistical inference for individual organism research: Mixed blessing or curse? Journal of Applied Behavior Analysis, 7 , 647-653. https://doi.org/10.1901/jaba.1974.7-647 .

Michiels, B., Heyvaert, M., Meulders, A., & Onghena, P. (2017). Confidence intervals for single-case effect size measures based on randomization test inversion. Behavior Research Methods, 49 , 363-381. https://doi.org/10.3758/s13428-016-0714-4 .

Moeyaert, M., Akhmedjanova, D., Ferron, J. M., Beretvas, S. N., & Van den Noortgate, W. (2020). Effect size estimation for combined single-case experimental designs. Evidence-Based Communication Assessment and Intervention, 14 , 28-51. https://doi.org/10.1080/17489539.2020.1747146 .

Moeyaert, M., Ferron, J. M., Beretvas, S. N., & Van den Noortgate, W. (2014a). From a single-level analysis to a multilevel analysis of single-case experimental designs. Journal of School Psychology, 52 , 191-211. https://doi.org/10.1016/j.jsp.2013.11.003 .

Moeyaert, M., Ugille, M., Ferron, J. M., Beretvas, S. N., & Van den Noortgate, W. (2014b). Three-level analysis of single-case experimental data: Empirical validation. The Journal of Experimental Education, 82 , 1-21. https://doi.org/10.1080/00220973.2012.745470 .

O’Brien, S., & Repp, A. C. (1990). Reinforcement-based reductive procedures: A review of 20 years of their use with persons with severe or profound retardation. Journal of the Association for Persons with Severe Handicaps, 15 , 148–159. https://doi.org/10.1177/154079699001500307 .

Onghena, P. (1992). Randomization tests for extensions and variations of ABAB single-case experimental designs: A rejoinder. Behavioral Assessment, 14 , 153-172.

Onghena, P., & Edgington, E. S. (1994). Randomization tests for restricted alternating treatment designs. Behaviour Research and Therapy, 32 , 783-786. https://doi.org/10.1016/0005-7967(94)90036-1 .

Onghena, P., & Edgington, E. S. (2005). Customization of pain treatments: Single-case design and analysis. The Clinical Journal of Pain, 21 , 56-68. https://doi.org/10.1097/00002508-200501000-00007 .

Onghena, P., Tanious, R., De, T. K., & Michiels, B. (2019). Randomization tests for changing criterion designs. Behaviour Research and Therapy, 117 , 18-27. https://doi.org/10.1016/j.brat.2019.01.005 .

Ottenbacher, K. J. (1990). When is a picture worth a thousand p values? A comparison of visual and quantitative methods to analyze single subject data. The Journal of Special Education, 23 , 436-449. https://doi.org/10.1177/002246699002300407 .

Parker, R. I., Hagan-Burke, S., & Vannest, K. (2007). Percentage of all non-overlapping data (PAND): An alternative to PND. The Journal of Special Education, 40 , 194-204. https://doi.org/10.1177/00224669070400040101 .

Parker, R. I., Vannest, K. J., & Davis, J. L. (2011). Effect Size in Single-Case Research: A Review of Nine Nonoverlap Techniques. Behavior Modification, 35 , 303-322. https://doi.org/10.1177/0145445511399147 .

Pustejovski, J. E., & Ferron, J. M. (2017). Research synthesis and meta-analysis of single-case designs. In J. M. Kaufmann, D. P. Hallahan, & P. C. Pullen, Handbook of Special Education (pp. 168-185). New York: Routledge.

Chapter   Google Scholar  

Pustejovsky, J. E., Hedges, L. V., & Shadish, W. R. (2014). Design-comparable effect sizes in multiple baseline designs: A general modeling framework. Journal of Educational and Behavioral Statistics, 39 , 368-393. https://doi.org/10.3102/1076998614547577 .

Scruggs, T. E., Mastropieri, M. A., & Casto, G. (1987). The quantitative synthesis of single-subject research: Methodology and validation. Remedial and Special Education, 8 , 24-33. https://doi.org/10.1177/074193258700800206 .

Shadish, W. R., Hedges, L. V., & Pustejovsky, J. E. (2014). Analysis and meta-analysis of single-case designs with a standardized mean difference statistic: A primer and applications. Journal of School Psychology, 52 , 123–147. https://doi.org/10.1016/j.jsp.2013.11.005 .

Shadish, W. R., Rindskopf, D. M., & Hedges, L. V. (2008). The state of the science in the meta-analysis of single-case experimental designs. Evidence-Based Communication Assessment and Intervention, 2 , 188-196. https://doi.org/10.1080/17489530802581603 .

Shadish, W. R., & Sullivan, K. J. (2011). Characteristics of single-case designs used to assess intervention effects in 2008. Behavior Research Methods, 43 , 971-980. https://doi.org/10.3758/s13428-011-0111-y .

Smith, J. D. (2012). Single-case experimental designs: A systematic review of published research and current standards. Psychological Methods, 17 , 510-550. https://doi.org/10.1037/a0029312 .

Solanas, A., Manolov, R., & Onghena, P. (2010). Estimating slope and level change in N=1 designs. Behavior Modification, 34 , 195-218. https://doi.org/10.1177/0145445510363306 .

Solomon, B. G. (2014). Violations of school-based single-case data: Implications for the selection and interpretation of effect sizes. Behavior Modification, 38 , 477-496. https://doi.org/10.1177/0145445513510931 .

Staples, M., & Niazi, M. (2007). Experiences using systematic review guidelines. The Journal of Systems and Software, 80 , 1425-1437. https://doi.org/10.1016/j.jss.2006.09.046 .

Swan, D. M., Pustejovsky, J. E., & Beretvas, S. N. (2020). The impact of response-guided designs on count outcomes in single-case experimental design baselines. Evidence-Based Communication Assessment and Intervention, 14 , 82-107. https://doi.org/10.1080/17489539.2020.1739048 .

Tanious, R., De, T. K., Michiels, B., Van den Noortgate, W., & Onghena, P. (2019). Consistency in single-case ABAB phase designs: A systematic review. Behavior Modification https://doi.org/10.1177/0145445519853793 .

Tanious, R., De, T. K., Michiels, B., Van den Noortgate, W., & Onghena, P. (2020). Assessing consistency in single-case A-B-A-B phase designs. Behavior Modification, 44 , 518-551. https://doi.org/10.1177/0145445519837726 .

Tate, R. L., Perdices, M., Rosenkoetter, U., McDonald, S., Togher, L., Shadish, W. R., … Vohra, S. (2016b). The Single-Case Reporting guideline In BEhavioural Interventions (SCRIBE) 2016: Explanation and Elaboration. Archives of Scientific Psychology, 4 , 1-9. https://doi.org/10.1037/arc0000026 .

Tate, R. L., Perdices, M., Rosenkoetter, U., Shadish, W. R., Vohra, S., Barlow, D. H., … Wilson, B. (2016a). The Single-Case Reporting guideline In BEhavioural interventions (SCRIBE) 2016 statement. Aphasiology, 30 , 862-876. https://doi.org/10.1080/02687038.2016.1178022 .

Tate, R. L., Perdices, M., Rosenkoetter, U., Wakim, D., Godbee, K., Togher, L., & McDonald, S. (2013). Revision of a method quality rating scale for single-case experimental designs and n-of-1 trials: The 15-item Risk of Bias in N-of-1 Trials (RoBiNT) Scale. Neuropsychological Rehabilitation, 23 , 619-638. https://doi.org/10.1080/09602011.2013.824383 .

Van den Noortgate, W., & Onghena, P. (2003). Hierarchical linear models for the quantitative integration of effect sizes in single-case research. Behavior Research Methods, Instruments, & Computers, 35 , 1-10. https://doi.org/10.3758/bf03195492 .

Van den Noortgate, W., & Onghena, P. (2008). A multilevel meta-analysis of single-subject experimental design studies. Evidence-Based Communication Assessment and Intervention, 2 , 142-151. https://doi.org/10.1080/17489530802505362 .

Vohra, S., Shamseer, L., Sampson, M., Bukutu, C., Schmid, C. H., Tate, R., … Group, TC (2016). CONSORT extension for reporting N-of-1 trials (CENT) 2015 statement. Journal of Clinical Epidemiology, 76 , 9–17. https://doi.org/10.1016/j.jclinepi.2015.05.004 .

Wampold, B., & Worsham, N. (1986). Randomization tests for multiple-baseline designs. Behavioral Assessment, 8 , 135-143.

What Works Clearinghouse. (2020a). Procedures Handbook (Version 4.1). Retrieved from Institute of Education Sciences: https://ies.ed.gov/ncee/wwc/Docs/referenceresources/WWC-Procedures-Handbook-v4-1-508.pdf

What Works Clearinghouse. (2020b). Responses to comments from the public on updated version 4.1 of the WWC Procedures Handbook and WWC Standards Handbook. Retrieved from Institute of Education Sciences: https://ies.ed.gov/ncee/wwc/Docs/referenceresources/SumResponsePublicComments-v4-1-508.pdf

What Works Clearinghouse. (2020c). Standards Handbook, version 4.1. Retrieved from Institute of Education Sciences: https://ies.ed.gov/ncee/wwc/Docs/referenceresources/WWC-Standards-Handbook-v4-1-508.pdf

White, D. M., Rusch, F. R., Kazdin, A. E., & Hartmann, D. P. (1989). Applications of meta-analysis in individual-subject research. Behavioral Assessment, 11 , 281-296.

Wolery, M. (2013). A commentary: Single-case design technical document of the What Works Clearinghouse. Remedial and Special Education , 39-43. https://doi.org/10.1177/0741932512468038 .

Woo, H., Lu, J., Kuo, P., & Choi, N. (2016). A content analysis of articles focusing on single-case research design: ACA journals between 2003 and 2014. Asia Pacific Journal of Counselling and Psychotherapy, 7 , 118-132. https://doi.org/10.1080/21507686.2016.1199439 .

Download references

Author information

Authors and affiliations.

Faculty of Psychology and Educational Sciences, Methodology of Educational Sciences Research Group, KU Leuven, Tiensestraat 102, Box 3762, B-3000, Leuven, Belgium

René Tanious & Patrick Onghena

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to René Tanious .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

(DOCX 110 kb)

Rights and permissions

Reprints and permissions

About this article

Tanious, R., Onghena, P. A systematic review of applied single-case research published between 2016 and 2018: Study designs, randomization, data aspects, and data analysis. Behav Res 53 , 1371–1384 (2021). https://doi.org/10.3758/s13428-020-01502-4

Download citation

Accepted : 09 October 2020

Published : 26 October 2020

Issue Date : August 2021

DOI : https://doi.org/10.3758/s13428-020-01502-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Single-case experimental designs
• Visual analysis
• Statistical analysis
• Data aspects
• Systematic review
• Find a journal
• Publish with us
• Track your research

Research Funding Principles and Priorities

Research funding principles.

• We prioritize research that advances the Society’s strategic plan, with a focus on stopping the disease in its tracks, restoring what has been lost, and ending MS forever through a cure for all forms of MS.
• We maintain a diverse research portfolio that includes short- and long-term investments, balances risks and rewards, and funds research globally.
• We balance investments in research initiated by individual investigators with investments in Society-directed research, all focused on bridging knowledge gaps, seizing opportunities and addressing research priorities.
• We promote synergies between researchers in diverse disciplines, and develop strategic partnerships to accelerate progress.
• We invest in all types of research including: laboratory models, human studies, population-based approaches, and data-intensive investigations that leverage both researcher and patient reported outcomes.
• We support the full spectrum of basic, translational, and clinical research. We also strive to break down barriers to commercial development to ensure that new treatments and wellness approaches are available as quickly as possible.
• We use independent experts to ensure that each research proposal receives a fair, competent and objective assessment of its scientific merit, relevance to MS, relevance to the MS community, and alignment with the Society’s research priorities.
• We attract and support new investigators to foster a robust future workforce focused on finding solutions for MS.
• We expect Society-funded researchers to adhere to rigorous experimental methods and reporting practices.
• We believe that sharing data and resources enhances research and speeds scientific discovery. Data and resources developed through Society-sponsored research will be made available to other researchers after publication in an expeditious manner.

Research Priorities: Pathways to Multiple Sclerosis Cures

• Biomarkers/Screening tools that identify MS in its earliest stages with enough confidence to initiate interventions
• Biological processes driving early MS compared to later stages of disease
• Understanding the heterogeneity of pre-symptom phases of MS in diverse populations
• Biomarkers of prognosis and therapeutic response in individuals
• Pathways driving non-lesional pathology/neurodegeneration involved in progressive stages of disease
• New molecular targets and therapeutic approaches for neuroprotection
• Interventions that target the earliest disease-causing pathways
• Understanding the roles of aging, sex, ethnicity, race, and genetics in MS pathology and response to therapies
• Physiological mechanisms, molecular targets and therapeutic approaches to promote myelin and neural repair
• Clear understanding of the functional heterogeneity of cells involved in repair
• Better physiologic, fluid biomarkers, imaging, and functional measures for earlier readouts of remyelination, reversal of tissue damage, and functional recovery
• Understanding of the roles of aging, sex, ethnicity, race, and genetics in tissue restoration
• Better animal models to study repair
• Understanding how rehabilitation and exercise impact the central nervous system and the extent that they can facilitate CNS repair processes
• Sensitive, valid, and clinically meaningful measures of disability and tools that establish measurable relationships between physiologic findings and meaningful recovery of function
• Sufficiently powered intervention studies that incorporate endpoints focused on type and dose parameters, targeting (including, but not limited to) cognitive or motor rehabilitation, resilience, diet, exercise, electrical stimulation, pain, fatigue, depression, anxiety, and bladder impairment
• Outcome measures, biologic, behavioral or technology driven that can be used to individually tailor interventions
• Innovative approaches that translate research findings to clinical practice and daily disease management
• Identify all relevant risk factors for MS, windows of risk, and determine whether any risk factor is necessary and sufficient to cause disease
• Understanding the contributions of genetic/epigenetic factors and environmental interactions to MS risk
• Understanding the roles of sex, ethnicity, and race with MS risk
• Understanding the early pathological pathways/events that lead to the initiation of MS
• Screening tools that identify MS in its earliest stages with enough confidence to trigger intervention
• Discovery of biomarkers that detect early MS before symptoms appear
• Interventions that target the earliest disease-causing pathways and the ability to determine which treatment will work for which person

An official website of the Department of Health & Human Services

• Search All AHRQ Sites
• Email Updates

1. Use quotes to search for an exact match of a phrase.

2. Put a minus sign just before words you don't want.

3. Enter any important keywords in any order to find entries where all these terms appear.

• The PSNet Collection
• All Content
• Perspectives
• Current Weekly Issue
• Past Weekly Issues
• Curated Libraries
• Clinical Areas
• Patient Safety 101
• The Fundamentals
• Training and Education
• Continuing Education
• WebM&M: Case Studies
• Training Catalog
• Submit a Case
• Improvement Resources
• Innovations
• Submit an Innovation
• About PSNet
• Editorial Team
• Technical Expert Panel

Technology as a Tool for Improving Patient Safety

Introduction .

In the past several decades, technological advances have opened new possibilities for improving patient safety. Using technology to digitize healthcare processes has the potential to increase standardization and efficiency of clinical workflows and to reduce errors and cost across all healthcare settings. 1 However, if technological approaches are designed or implemented poorly, the burden on clinicians can increase. For example, overburdened clinicians can experience alert fatigue and fail to respond to notifications. This can lead to more medical errors. As a testament to the significance of this topic in recent years, several government agencies [(e.g. the Agency for Healthcare Research and Quality (AHRQ) and the Centers for Medicare and Medicaid services (CMS)] have developed resources to help healthcare organizations integrate technology, such as the Safety Assurance Factors for EHR Resilience (SAFER) guides developed by the Office of the National Coordinator for Health Information Technology (ONC). 2,3,4  However, there is some evidence that these resources have not been widely used.5 Recently, the Centers for Medicare & Medicaid Services (CMS) started requiring hospitals to use the SAFER guides as part of the FY 2022 Hospital Inpatient Prospective Payment Systems (IPPS), which should raise awareness and uptake of the guides. 6

During 2022, research into technological approaches was a major theme of articles on PSNet. Researchers reviewed all relevant articles on PSNet and consulted with Dr. A Jay Holmgren, PhD, and Dr. Susan McBride, PhD, subject matter experts in health IT and its role in patient safety. Key topics and themes are highlighted below.  

Clinical Decision Support  

The most prominent focus in the 2022 research on technology, based on the number of articles published on PSNet, was related to clinical decision support (CDS) tools. CDS provides clinicians, patients, and other individuals with relevant data (e.g. patient-specific information), purposefully filtered and delivered through a variety of formats and channels, to improve and enhance care. 7   

Computerized Patient Order Entry  

One of the main applications of CDS is in computerized patient order entry (CPOE), which is the process used by clinicians to enter and send treatment instructions via a computer application. 8 While the change from paper to electronic order entry itself can reduce errors (e.g., due to unclear handwriting or manual copy errors), research in 2022 showed that there is room for improvement in order entry systems, as well as some promising novel approaches. 

Two studies looked at the frequency of and reasons for medication errors in the absence of CDS and CPOE and demonstrated that there was a clear patient safety need. One study found that most medication errors occurred during the ordering or prescribing stage, and both this study and the other study found that the most common medication error was incorrect dose. Ongoing research, such as the AHRQ Medication Safety Measure Development project, aims to develop and validate measure specifications for wrong-patient, wrong-dose, wrong-medication, wrong-route, and wrong-frequency medication orders within EHR systems, in order to better understand and capture health IT safety events.9 Errors of this type could be avoided or at least reduced through the use of effective CPOE and CDS systems. However, even when CPOE and CDS are in place, errors can still occur and even be caused by the systems themselves. One study reviewed duplicate medication orders and found that 20% of duplicate orders resulted from technological issues, including alerts being overridden, alerts not firing, and automation issues (e.g., prefilled fields). A case study last year Illustrated one of the technological issues, in this case a manual keystroke error, that can lead to a safety event. A pharmacist mistakenly set the start date for a medication to the following year rather than the following day , which the CPOE system failed to flag. The authors recommended various alerts and coding changes in the system to prevent this particular error in the future.  

There were also studies in 2022 that showed successful outcomes of well-implemented CPOE systems. One in-depth pre-post, mixed-methods study showed that a fully implemented CPOE system significantly reduced specific serious and commonly occurring prescribing and procedural errors. The authors also presented evidence that it was cost-effective and detailed implementation lessons learned drawn from the qualitative data collected for the study. A specific CPOE function that demonstrated statistically significant improvement in 2022 was automatic deprescribing of medication orders and communication of the relevant information to pharmacies. Deprescribing is the planned and supervised process of dose reduction or stopping of a medication that is no longer beneficial or could be causing harm. That study showed an immediate and sustained 78% increase in successful discontinuations after implementation of the software. A second study on the same functionality determined that currently only one third to one half of medications are e-prescribed, and the study proposed that e-prescribing should be expanded to increase the impact of the deprescribing software. It should be noted, however, that the systems were not perfect and that a small percentage of medications were unintentionally cancelled. Finally, an algorithm to detect patients in need of follow-up after test results was developed and implemented in another study . The algorithm showed some process improvements, but outcome measures were not reported. 

Usability  

Usability of CDS systems was a large focus of research in 2022. Poorly designed systems that do not fit into existing workflows lead to frustrated users and increase the potential for errors. For example, if users are required to enter data in multiple places or prompted to enter data that are not available to them, they could find ways to work around the system or even cease to use it, increasing the potential for patient safety errors. The documentation burden is already very high on U.S. clinicians, 10 so it is important that novel technological approaches do not add to this burden but, if possible, alleviate it by offering a high level of usability and interoperability.  

One study used human-factored design in creating a CDS to diagnose pulmonary embolism in the Emergency Department and then surveyed clinician users about their experiences using the tool. Despite respondents giving the tool high usability ratings and reporting that the CDS was valuable, actual use of the tool was low. Based on the feedback from users, the authors proposed some changes to increase uptake, but both users and authors mentioned the challenges that arise when trying to change the existing workflow of clinicians without increasing their burden. Another study gathered qualitative feedback from clinicians on a theoretical CDS system for diagnosing neurological issues in the Emergency Department. In this study too, many clinicians saw the potential value in the CDS tool but had concerns about workflow integration and whether it would impact their ability to make clinical decisions. Finally, one study developed a dashboard to display various risk factors for multiple hospital-acquired infections and gathered feedback from users. The users generally found the dashboard useful and easy to learn, and they also provided valuable feedback on color scales, location, and types of data displayed. All of these studies show that attention to end user needs and preferences is necessary for successful implementation of CDS.  However, the recent market consolidation in Electronic Health Record vendors may have an impact on the amount of user feedback gathered and integrated into CDS systems. Larger vendors may have more resources to devote to improving the usability and design of CDS, or their near monopolies in the market may not provide an incentive to innovate further. 11 More research is needed as this trend continues.  

Alerts and Alarms 

Alerts and alarms are an important part of most CDS systems, as they can prompt clinicians with important and timely information during the treatment process. However, these alerts and alarms must be accurate and useful to elicit an appropriate response. The tradeoff between increased safety due to alerts and clinician alert fatigue is an important balance to strike. 12

Many studies in 2022 looked at clinician responses to medication-related alerts, including override and modification rates. Several of the studies found a high alert override rate but questioned the validity of using override rates alone as a marker of CDS effectiveness and usability. For example, one study looked at drug allergy alerts and found that although 44.8% of alerts were overridden, only 9.3% of those were inappropriately overridden, and very few overrides led to an adverse allergic reaction. A study on “do not give” alerts found that clinicians modified their orders to comply with alert recommendations after 78% of alerts but only cancelled orders after 26% of alerts. A scoping review looked at drug-drug interaction alerts and found similar results, including high override rates and the need for more data on why alerts are overridden. These findings are supported by another study that found that the underlying drug value sets triggering drug-drug interaction alerts are often inconsistent, leading to many inappropriate alerts that are then appropriately overridden by clinicians. These studies suggest that while a certain number of overrides should be expected, the underlying criteria for alert systems should be designed and regularly reviewed with specificity and sensitivity in mind. This will increase the frequency of appropriate alerts that foster indicated clinical action and reduce alert fatigue. 

There also seems to be variability in the effectiveness of alert systems across sites. One study looked at an alert to add an item to the problem list if a clinician placed an order for a medication that was not indicated based on the patient’s chart. The study found about 90% accuracy in alerts across two sites but a wide difference in the frequency of appropriate action between the sites (83% and 47%). This suggests that contextual factors at each site, such as culture and organizational processes, may impact success as much as the technology itself.  

A different study looked at the psychology of dismissing alerts using log data and found that dismissing alerts becomes habitual and that the habit is self-reinforcing over time. Furthermore, nearly three quarters of alerts were dismissed within 3 seconds. This indicates how challenging it can be to change or disrupt alert habits once they are formed. 

Artificial Intelligence and Machine Learning  

In recent years, one of the largest areas of burgeoning technology in healthcare has been artificial intelligence (AI) and machine learning. AI and machine learning use algorithms to absorb large amounts of historical and real-time data and then predict outcomes and recommend treatment options as new data are entered by clinicians. Research in 2022 showed that these techniques are starting to be integrated into EHR and CDS systems, but challenges remain. A full discussion of this topic is beyond the scope of this review. Here we limit the discussion to several patient-safety-focused resources posted on PSNet in 2022.  

One of the promising aspects of AI is its ability to improve CDS processes and clinician workflow overall. For example, one study last year looked at using machine learning to improve and filter CDS alerts. They found that the software could reduce alert volume by 54% while maintaining high precision. Reducing alert volume has the potential to alleviate alert fatigue and habitual overriding. Another topic explored in a scoping review was the use of AI to reduce adverse drug events. While only a few studies reviewed implementation in a clinical setting (most evaluated algorithm technical performance), several promising uses were found for AI systems that predict risk of an adverse drug event, which would facilitate early detection and mitigate negative effects.  

Despite enthusiasm for and promising applications of AI, implementation is slow. One of the challenges facing implementation is the variable quality of the systems. For example, a commonly used sepsis detection model was recently found to have very low sensitivity. 13 Algorithms also drift over time as new data are integrated, and this can affect performance, particularly during and after large disturbances like the COVID-19 pandemic. 14 There is also emerging research about the impact of AI algorithms on racial and ethnic biases in healthcare; at the time of publication of this essay, an AHRQ EPC was conducting a review of evidence on the topic. 15  These examples highlight the fact that AI is not a “set it and forget it” application; it requires monitoring and customization from a dedicated resource to ensure that the algorithms perform well over time. A related challenge is the lack of a strong business case for using high-quality AI. Because of this, many health systems choose to use out-of-the-box AI algorithms, which may be of poor quality overall (or are unsuited to particular settings) and may also be “black box” algorithms (i.e., not customizable by the health system because the vendor will not allow access to the underlying code). 16 The variable quality and the lack of transparency may cause mistrust by clinicians and overall aversion to AI interventions.  

In an attempt to address these concerns, one article in 2022 detailed best practices for AI implementation in health systems, focusing on the business case. Best practices include using AI to address a priority problem for the health system rather than treating it as an end itself. Additionally, testing the AI using the health system’s patients and data to demonstrate applicability and accuracy for that setting, confirming that the AI can provide a return on investment, and ensuring that the AI can be implemented easily and efficiently are also important. Another white paper described a human-factors and ergonomics framework for developing AI in order to improve the implementation within healthcare systems, teams, and workflows. The federal government and international organizations have also published AI guidelines, focusing on increasing trustworthiness (National Artificial Intelligence Initiative) 17 and ensuring ethical governance (World Health Organization). 18   

Conclusion and Next Steps 

As highlighted in this review, the scope and complexity of technology and its application in healthcare can be intimidating for healthcare systems to approach and implement. Researchers last year thus created a framework that health systems can use to assess their digital maturity and guide their plans for further integration.  

The field would benefit from more research in several areas in upcoming years. First and foremost, high-quality prospective outcome studies are needed to validate the effectiveness of the new technologies. Second, more work is needed on system usability, how the systems are integrated into workflows, and how they affect the documentation burden placed on clinicians. For CDS specifically, more focus is needed on patient-centered CDS (PC CDS), which supports patient-centered care by helping clinicians and patients make the best decisions given each individual’s circumstances and preferences. 19 AHRQ is already leading efforts in this field with their CDS Innovation Collaborative project. 20 Finally, as it becomes more common to incorporate EHR scribes to ease the documentation burden, research on their impact on patient safety will be needed, especially in relation to new technological approaches. For example, when a scribe encounters a CDS alert, do they alert the clinician in all cases? 

In addition to the approaches mentioned in this article, other emerging technologies in early stages of development hold theoretical promise for improving patient safety. One prominent example is “computer vision,” which uses cameras and AI to gather and process data on what physically happens in healthcare settings beyond what is captured in EHR data, 21 including being able to detect immediately that a patient fell in their room. 22  

As technology continues to expand and improve, researchers, clinicians, and health systems must be mindful of potential stumbling blocks that could impede progress and threaten patient safety. However, technology presents a wide array of opportunities to make healthcare more integrated, efficient, and safe.  

• Cohen CC, Powell K, Dick AW, et al. The Association Between Nursing Home Information Technology Maturity and Urinary Tract Infection Among Long-Term Residents . J Appl Gerontol . 2022;41(7):1695-1701. doi: 10.1177/07334648221082024. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9232878/
• https://www.healthit.gov/topic/safety/safer-guides
• https://cds.ahrq.gov/cdsconnect/repository
• https://www.cms.gov/about-cms/obrhi
• McBride S, Makar E, Ross A, et al. Determining awareness of the SAFER guides among nurse informaticists. J Inform Nurs. 2021;6(4). https://library.ania.org/ania/articles/713/view
• Sittig DF, Sengstack P, Singh H. Guidelines for US hospitals and clinicians on assessment of electronic health record safety using SAFER guides. J ama . 2022;327:719-720.
• https://library.ahima.org/doc?oid=300027#.Y-6RhXbMKHt
• https://www.healthit.gov/faq/what-computerized-provider-order-entry#:~:text=Computerized%20provider%20order%20entry%20(CPOE,paper%2C%20fax%2C%20or%20telephone
• https://digital.ahrq.gov/2018-year-review/research-spotlights/leveragin…
• Holmgren AJ, Downing NL, Bates DW, et al. Assessment of electronic health record use between US and non-US health systems. JAMA Intern Med. 2021;181:251-259. https://doi.org/10.1001/jamainternmed.2020.7071
• Holmgren AJ, Apathy NC. Trends in US hospital electronic health record vendor market concentration, 2012–2021. J Gen Intern Med. 2022. https://link.springer.com/article/10.1007/s11606-022-07917-3#citeas
• Co Z, Holmgren AJ, Classen DC, et al. The tradeoffs between safety and alert fatigue: data from a national evaluation of hospital medication-related clinical decision support. J Am Med Inform Assoc. 2020;27:1252-1258. https://pubmed.ncbi.nlm.nih.gov/32620948/
• Wong A, Otles E, Donnelly JP, et al. External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Intern Med. 2021;181:1065-1070. https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2781307
• Parikh RB, Zhang Y, Kolla L, et al. Performance drift in a mortality prediction algorithm among patients with cancer during the SARS-CoV-2 pandemic. J Am Med Inform Assoc. 2022;30:348-354. https://academic.oup.com/jamia/advance-article/doi/10.1093/jamia/ocac221/6835770?login=false
• https://effectivehealthcare.ahrq.gov/products/racial-disparities-health…
• https://www.statnews.com/2022/05/24/market-failure-preventing-efficient-diffusion-health-care-ai-software/
• https://www.ai.gov/strategic-pillars/advancing-trustworthy-ai/
• Ethics and governance of artificial intelligence for health (WHO guidance). Geneva: World Health Organization; 2021. https://www.who.int/publications/i/item/9789240029200
• Dullabh P, Sandberg SF, Heaney-Huls K, et al. Challenges and opportunities for advancing patient-centered clinical decision support: findings from a horizon scan. J Am Med Inform Assoc. 2022: 29(7):1233-1243. doi: 10.1093/jamia/ocac059. PMID: 35534996; PMCID: PMC9196686.
• https://cds.ahrq.gov/cdsic
• Yeung S, Downing NL, Fei-Fei L, et al. Bedside computer vision: moving artificial intelligence from driver assistance to patient safety. N Engl J Med. 2018;387:1271-1273. https://www.nejm.org/doi/10.1056/NEJMp1716891
• Espinosa R, Ponce H, Gutiérrez S, et al. A vision-based approach for fall detection using multiple cameras and convolutional neural networks: a case study using the UP-Fall detection dataset. Comput Biol Med. 2019;115:103520. https://doi.org/10.1016/j.compbiomed.2019.103520

This project was funded under contract number 75Q80119C00004 from the Agency for Healthcare Research and Quality (AHRQ), U.S. Department of Health and Human Services. The authors are solely responsible for this report’s contents, findings, and conclusions, which do not necessarily represent the views of AHRQ. Readers should not interpret any statement in this report as an official position of AHRQ or of the U.S. Department of Health and Human Services. None of the authors has any affiliation or financial involvement that conflicts with the material presented in this report. View AHRQ Disclaimers

Perspective

Perspectives on Safety

Annual Perspective

Patient Safety Innovations

Suicide Prevention in an Emergency Department Population: ED-SAFE

WebM&M Cases

The Retrievals. August 9, 2023

Agent of change. August 1, 2018

Amid lack of accountability for bias in maternity care, a California family seeks justice. August 16, 2023

Mirror, Mirror on the Wall: An Update on the Quality of American Health Care Through the Patient's Lens. April 12, 2006

Improving patient safety by shifting power from health professionals to patients. October 25, 2023

Patient Safety Primers

Discharge Planning and Transitions of Care

Medicines-related harm in the elderly post-hospital discharge. March 27, 2019

Emergency department crowding: the canary in the health care system. November 3, 2021

Advancing Patient Safety: Reviews From the Agency for Healthcare Research and Quality's Making Healthcare Safer III Report. September 2, 2020

Exploring Alternatives To Malpractice Litigation. January 15, 2014

Making Healthcare Safer III. March 18, 2020

Special Section: Patient Safety. May 24, 2006

The Science of Simulation in Healthcare: Defining and Developing Clinical Expertise. November 19, 2008

Compendium of Strategies to Prevent HAIs in Acute Care Hospitals 2014. September 1, 2014

Quality, Safety, and Noninterpretive Skills. November 11, 2015

Patient Safety. November 21, 2018

Ambulatory Safety Nets to Reduce Missed and Delayed Diagnoses of Cancer

Remote response team and customized alert settings help improve management of sepsis.

Using sociotechnical theory to understand medication safety work in primary care and prescribers' use of clinical decision support: a qualitative study. May 24, 2023

Human factors and safety analysis methods used in the design and redesign of electronic medication management systems: a systematic review. May 17, 2023

Journal Article

Reducing hospital harm: establishing a command centre to foster situational awareness.

The potential for leveraging machine learning to filter medication alerts. May 4, 2022

Improving the specificity of drug-drug interaction alerts: can it be done? April 6, 2022

A qualitative study of prescribing errors among multi-professional prescribers within an e-prescribing system. December 23, 2020

The tradeoffs between safety and alert fatigue: data from a national evaluation of hospital medication-related clinical decision support. July 29, 2020

Assessment of health information technology-related outpatient diagnostic delays in the US Veterans Affairs health care system: a qualitative study of aggregated root cause analysis data. July 22, 2020

Reducing drug prescription errors and adverse drug events by application of a probabilistic, machine-learning based clinical decision support system in an inpatient setting. August 21, 2019

Improving medication-related clinical decision support. March 7, 2018

The frequency of inappropriate nonformulary medication alert overrides in the inpatient setting. April 6, 2016

The effect of provider characteristics on the responses to medication-related decision support alerts. July 15, 2015

Best practices: an electronic drug alert program to improve safety in an accountable care environment. July 1, 2015

Impact of computerized physician order entry alerts on prescribing in older patients. March 25, 2015

Differences of reasons for alert overrides on contraindicated co-prescriptions by admitting department. December 17, 2014

Connect With Us

Sign up for Email Updates

To sign up for updates or to access your subscriber preferences, please enter your email address below.

Agency for Healthcare Research and Quality

5600 Fishers Lane Rockville, MD 20857 Telephone: (301) 427-1364

• Accessibility
• Disclaimers
• Electronic Policies
• HHS Digital Strategy
• HHS Nondiscrimination Notice
• Inspector General
• Plain Writing Act
• Privacy Policy
• Viewers & Players
• U.S. Department of Health & Human Services
• The White House
• Don't have an account? Sign up to PSNet

Submit Your Innovations

Please select your preferred way to submit an innovation.

Continue as a Guest

Track and save your innovation

in My Innovations

Edit your innovation as a draft

Continue Logged In

Please select your preferred way to submit an innovation. Note that even if you have an account, you can still choose to submit an innovation as a guest.

Continue logged in

New users to the psnet site.

Access to quizzes and start earning

CME, CEU, or Trainee Certification.

Get email alerts when new content

matching your topics of interest

in My Innovations.

Advisory boards aren’t only for executives. Join the LogRocket Content Advisory Board today →

• Product Management
• Solve User-Reported Issues
• Find Issues Faster
• Optimize Conversion and Adoption

21 UX case studies to learn from in 2024

UX case studies are the heart of your design portfolio. They offer a peek into your design process, showcasing how you tackle challenges, your methods, and your results. For recruiters, these case studies serve as a metric for evaluating your skills, problem-solving abilities, and talent.

If you’re considering creating your own UX case study in 2024 but don’t know where to start, you’re in the right place. This article aims to inspire you with 21 carefully hand-picked UX case study examples, each offering valuable lessons.

But before we dive into these examples, let’s address a question that might be lingering: Is a UX case study truly worth the effort?

Is it worth creating a UX case study?

The short answer is yes.

Remember how in math class, showing your workings was even more important than getting the correct answer? UX case studies are like that for designers. They are more than just showcasing the final product (the polished website or app); they detail the steps taken to get there (the research, user testing, and design iterations). By showing your design process, you give potential employers or clients a peek into your thought process and problem-solving skills.

A well-laid-out case study has many benefits, including the following:

Building credibility

As case studies provide evidence of your expertise and past successes, they can build credibility and trust with potential employers or clients.

Educational value

By showing your design process, you provide valuable insights and learnings for other designers and stakeholders.

Differentiation

A compelling case study can leave a lasting impression on potential recruiters and clients, helping you stand out.

Iterative improvement

A case study is like a roadmap of each project, detailing the highs, lows, failures, and successes. This information allows you to identify areas for improvement, learn from mistakes, and refine your approach in subsequent projects.

Now that you know why a stand-out case study is so important, let’s look at 21 examples to help you get creative. The case studies will fall under five categories:

• Language learning app
• Learning app
• Travel agency app
• Intelly healthcare app
• Cox Automotive
• Swiftwash laundry
• Wayfaro trip planner
• New York Times app redesign
• Disney+ app redesign
• Fitbit redesign
• Ryanair app redesign
• Forbes app redesign
• Enhancing virtual teaching with Google Meet
• Airbnb’s global check-in tool
• Spotify home shortcuts
• AI-powered spatial banking for Apple Vision Pro
• Sage Express

In this section, we’ll explore case studies that take us through the complete design journey of creating a digital product from scratch.

1. Language learning app

If you’re a designer looking to get your foot in the door, this is one case study you need to check out. It’s so well detailed that it helped this designer land their first role as a UX designer:

Created by Christina Sa, this case study tackles the all-too-common struggle of learning a new language through a mobile app. It takes us through the process of designing a nontraditional learning app that focuses on building a habit by teaching the Korean language using Korean media such as K-pop, K-drama, and K-webtoon.

Over 200k developers and product managers use LogRocket to create better digital experiences

Key takeaway

This case study shows how a structured design process, user-centered approach, and effective communication can help you stand out. The creator meticulously laid out their design process from the exploratory research phase to the final prototype, even detailing how the case study changed their view on the importance of a design process.

If you’re searching for a comprehensive case study that details every step of the design process, look no further. This one is for you:

This impressive case study by Finna Wang explores the creation of a fan-focused responsive platform for Jambb, an already existing social platform. The creator starts by identifying the problem and then defines the project scope before diving into the design process.

This case study shows us the importance of an iterative problem-solving approach. It identifies a problem (pre-problem statement), creates a solution, tests the solution, and then revises the problem statement based on the new findings.

3. Learning app

If you need a highly visual case study that takes you through every step of the design process in an engaging way, this one is for you:

This case study walks us through the design of a platform where users can find experts to explain complex topics to them in a simple and friendly manner. It starts by defining the scope of work, then progresses through research, user journeys, information architecture, user flow, initial design, and user testing, before presenting the final solution.

This case study demonstrates effective ways to keep readers engaged while taking them through the steps of a design process. By incorporating illustrations and data visualization, the designer communicates complex information in an engaging manner, without boring the readers.

If you’re in search of a case study that details the design process but is also visually appealing, you should give this one a look:

This case study by Orbix Studio takes us through the process of designing GiveHub, a fundraising app that helps users set up campaigns for causes they’re passionate about. It starts with an overview of the design process, then moves on to identifying the challenges and proposing solutions, before showing us how the solutions are brought to life.

This case study illustrates how a visually engaging design and clear organization can make your presentation easy to grasp.

5. Travel agency app

This case study is quite popular on Behance, and it’s easy to see why:

The case study takes us through the process of creating a travel app that lets users compare travel packages from various travel agencies or groups. The creators set out a clear problem statement, propose a solution, and then show us the step-by-step implementation process. The incorporation of data visualization tools makes this case study easy to digest.

This is another case study that shows the importance of using a clearly defined design process. Going by its popularity on Behance, you can tell that the step-by-step process breakdown was well worth the effort.

6. Intelly healthcare app

If you’re looking for a UX case study that explores the design journey for both mobile and desktop versions of an app, this is one you should check out:

This case study explores the process of creating Intelly, an app that transforms patient care with telemedicine, prescription management, and real-time tracking. The case study begins with a clear design goal, followed by a layout of existing problems and design opportunities. The final design is a mobile app for patients and a desktop app for doctors.

This case study highlights the importance of proactive problem-solving and creative thinking in the design process. The creators laid out some key problems, identified design opportunities in them, and effectively leveraged them to create an app.

7. Cox Automotive

If you prefer a results-oriented case study, you’ll love this one:

This case study delves into how Cox Automotive’s Manheim division, used LogRocket to optimize their customers’ digital experience for remote car auctions. It starts by highlighting the three key outcomes before giving us an executive summary of the case study. The rest of the case study takes us through the process of achieving the highlighted outcomes.

A key takeaway from this case study is the significance of using user data and feedback to enhance the digital experience continuously. Cox Automotive used LogRocket to identify and address user-reported issues, gain insights into customer behaviors, and make data-driven decisions to optimize their product.

These case studies are more focused on the visual aspects of the design process, teaching us a thing or two about presentation and delivery.

If you love a case study that scores high on aesthetics with vivid colors, cool illustrations, and fun animations, you need to check this one out:

This case study takes us on a visual journey of creating Rebank, a digital product aimed at revolutionizing the baking industry. It starts with the research process, moves on to branding and style, and then takes us through the different screens, explaining what each one offers.

This case study illustrates the value of thinking outside the box. Breaking away from the conventional design style of financial products makes it a stand-out case study.

9. Swiftwash Laundry

If you’re looking for a case study that prioritizes aesthetics and visual appeal, you should check this one out:

This case study by Orbix Studio gives us a peek into how they created Swiftwash, a laundry service app. It takes us through the steps involved in creating an intuitive, user-friendly, and visually appealing interface.

If there’s one thing to take away from this case study, it’s the value of presenting information in a straightforward manner. Besides being easy on the eye, this case study is also easy to digest. The creators lay out the problem and detail the steps taken to achieve a solution, in an easy-to-follow way, while maintaining a high visual appeal.

10. Wayfaro trip planner

If you’re looking for a concise case study with clean visuals, you should definitely check this one out:

This Behance case study takes us through the design of Wayfaro, a trip planner app that allows users to plan their itineraries for upcoming journeys. The creators dive straight into the visual design process, showing us aspects such as branding and user flow, and explaining the various features on each screen.

This case study shows us the power of an attractive presentation. Not only is the mobile app design visually appealing, but the design process is presented in a sleek and stylish manner.

App redesign

These case studies delve into the redesign of existing apps, offering valuable insights into presentation techniques and problem-solving approaches.

11. New York Times app redesign

If you’re looking for an app redesign case study that’s impactful yet concise, this one is for you:

This study details the creation of “Timely,” a design feature to address issues with the NYT app such as irrelevant content, low usage, and undesirable coverage. It takes us through the process of identifying the problem, understanding audience needs, creating wireframes, and prototyping.

This case study shows us that you don’t always need to overhaul the existing app when redesigning. It suggests a solution that fits into the current information setup, adding custom graphics to the mobile app. Starting with a simple problem statement, it proposes a solution to address the app’s issues without changing what customers already enjoy.

12. Disney+ app redesign

If you’re looking for an engaging case study that’s light on information, you should check out this one:

This case study by Andre Carioca dives right into giving the user interface a little facelift to make it more fun and engaging. By employing compelling storytelling and appealing visuals, the creator crafts a narrative that’s a delight to read.

Given how popular this case study is on Behance, you can tell that the designer did something right. It shows how injecting a little playfulness can elevate your case study and make it more delightful.

13. Fitbit redesign

If you want an in-depth case study that doesn’t bore you to sleep, this one is for you:

This case study by Stacey Wang takes us through the process of redesigning Fitbit, a wearable fitness tracker. The creator starts by understanding personas and what users expect from a fitness tracker.

Next was the development of use cases and personas. Through a series of guerrilla tests, they were able to identify user pain points. The redesign was centered around addressing these pain points.

This case study highlights the importance of clear organization and strong visual communication. The creator goes in-depth into the intricacies of redesigning the Fitbit app, highlighting every step, without boring the readers.

14. Ryanair app redesign

If you’re bored of the usual static case studies and need something more interactive, this app redesign is what you’re looking for:

This case study takes us through the process of giving the Ryanair app a fresh look. Besides the clean aesthetics and straightforward presentation, the incorporation of playful language and interactive elements makes this case study captivating.

This case study shows how adding a bit of interactivity to your presentation can elevate your work.

15. Forbes app redesign

This case study starts by explaining why the redesign was needed and dives deep into analyzing the current app. The creator then takes us through the research and ideation phases and shares their proposed solution. After testing the solution, they made iterations based on the results.

When it comes to redesigning an existing product, it’s a good idea to make a strong case for why the redesign was needed in the first place.

UX research

These case studies are centered around UX research, highlighting key research insights to enhance your design process.

16. Enhancing virtual teaching with Google Meet

This case study by Amanda Rosenburg, Head of User Experience Research, Google Classroom shows us how listening to user feedback can help make our products more useful and inclusive to users.

To improve the virtual teaching experience on Google Meet, the team spent a lot of time getting feedback from teachers. They then incorporated this feedback into the product design, resulting in new functionality like attendance taking, hand raising, waiting rooms, and polls. Not only did these new features improve the user experience for teachers and students, but they also created a better user experience for all Google Meet users.

When there isn’t room for extensive user research and you need to make quick improvements to the user experience, it’s best to go straight to your users for feedback.

17. Airbnb’s global check-in tool

This case study by Vibha Bamba, Design Lead on Airbnb’s Host Success team, shows us how observing user behaviors inspired the creation of a global check-in tool:

By observing interactions between guests and hosts, the Airbnb team discovered a design opportunity. This led to the creation of visual check-in guides for Airbnb guests, which they can access both offline and online.

There’s a lot to be learned from observing user behavior. Don’t limit yourself to insights obtained from periodic research. Instead, observe how people interact with your product in their daily lives. The insights obtained from such observations can help unlock ingenious design opportunities.

18. Spotify Home Shortcuts

This case study by Nhi Ngo, a Senior User Researcher at Spotify shows us the importance of a human perspective in a data-driven world:

When the Spotify team set out to develop and launch the ML-powered Shortcuts feature on the home tab, they hit a brick wall with the naming. A/B tests came back inconclusive. In the end, they had to go with the product designer’s suggestion of giving the feature a name that would create a more human and personal experience for users.

This led to the creation of a humanistic product feature that evoked joy in Spotify’s users and led to the incorporation of more time-based features in the model, making the content more time-sensitive for users.

Although data-driven research is powerful, it doesn’t hold all the answers. So in your quest to uncover answers through research, never lose sight of the all-important human perspective.

Artificial intelligence

The following case studies are centered around the design of AI-powered products.

19. AI-powered spatial banking for Apple Vision Pro

If you want to be wowed by a futuristic case study that merges artificial intelligence with spatial banking, you should check this out:

In this revolutionary case study, UXDA designers offer a sneak peek into the future with a banking experience powered by AI. They unveil their vision of AI-powered spatial banking on the visionOS platform, showcasing its features and their AI use cases.

This case study shows us the importance of pushing boundaries to create innovative experiences that cater to user needs and preferences.

20. Sage Express

If what you need is an AI case study that isn’t information-dense, this one is for you:

This case study by Arounda takes us through the design of Sage Express, an AI-powered data discovery tool that automatically extracts patterns, tendencies, and insights from data. It outlines the challenge, proposes a solution, and details the journey of bringing the proposed solution to life. But it doesn’t stop there: it also shows the actual results of the design using tangible metrics.

This case study underscores the importance of showing your outcomes in tangible form. You’ve worked hard on a project, but what were the actual results?

If you’re looking for a clean and well-structured AI case study, this will be helpful:

This case study takes us through the process of creating Delfi, an AI-driven banking financial report system. It details the entire design process from onboarding to prototype creation.

If there’s one thing to learn from this case study, it’s how a well-structured presentation can simplify complex information. Although the case study is heavy on financial data, the organized layout not only enhances visual appeal but also aids comprehension.

This article has shown you 21 powerful case study examples across various niches, each providing valuable insights into the design process. These case studies demonstrate the importance of showcasing the design journey, not just the final polished product.

When creating your own case study, remember to walk your users through the design process, the challenges you faced, and your solutions. This gives potential recruiters and clients a glimpse of your creativity and problem-solving skills.

And finally, don’t forget to add that human touch. Let your personality shine through and don’t be afraid to inject a little playfulness and storytelling where appropriate. By doing so, you can craft a case study that leaves a lasting impression on your audience.

Header image source: IconScout

LogRocket : Analytics that give you UX insights without the need for interviews

LogRocket lets you replay users' product experiences to visualize struggle, see issues affecting adoption, and combine qualitative and quantitative data so you can create amazing digital experiences.

See how design choices, interactions, and issues affect your users — get a demo of LogRocket today .

Share this:

• Click to share on Twitter (Opens in new window)
• Click to share on Reddit (Opens in new window)
• Click to share on LinkedIn (Opens in new window)
• Click to share on Facebook (Opens in new window)

Stop guessing about your digital experience with LogRocket

Recent posts:.

A UX analysis of Arc, Opera, and Edge: The future of browser interfaces

Are we experiencing a new age of browsers? Let’s break down the UX design of the most popular browsers and upcoming stars.

Service design in action: Creating memorable experiences for small businesses

Service design can help our organizations innovate customer experience and build brand loyalty — and it’s great for small businesses.

Combating addictive design is the UX challenge of 2024

Digital addiction is bad for your mental and physical health. Learn what trends encourage this concerning behavior and how to avoid it.

What is DesignOps? The essentials of DesignOps

DesignOps is a set of best practices and principles that aims to streamline the effectiveness of design teams.

Leave a Reply Cancel reply

• Share full article

For more audio journalism and storytelling, download New York Times Audio , a new iOS app available for news subscribers.

• April 29, 2024   •   47:53 Trump 2.0: What a Second Trump Presidency Would Bring
• April 26, 2024   •   21:50 Harvey Weinstein Conviction Thrown Out
• April 25, 2024   •   40:33 The Crackdown on Student Protesters
• April 24, 2024   •   32:18 Is $60 Billion Enough to Save Ukraine?
• April 23, 2024   •   30:30 A Salacious Conspiracy or Just 34 Pieces of Paper?
• April 22, 2024   •   24:30 The Evolving Danger of the New Bird Flu
• April 19, 2024   •   30:42 The Supreme Court Takes Up Homelessness
• April 18, 2024   •   30:07 The Opening Days of Trump’s First Criminal Trial
• April 17, 2024   •   24:52 Are ‘Forever Chemicals’ a Forever Problem?
• April 16, 2024   •   29:29 A.I.’s Original Sin
• April 15, 2024   •   24:07 Iran’s Unprecedented Attack on Israel
• April 14, 2024   •   46:17 The Sunday Read: ‘What I Saw Working at The National Enquirer During Donald Trump’s Rise’

Harvey Weinstein Conviction Thrown Out

New york’s highest appeals court has overturned the movie producer’s 2020 conviction for sex crimes, which was a landmark in the #metoo movement..

Hosted by Katrin Bennhold

Featuring Jodi Kantor

Produced by Nina Feldman ,  Rikki Novetsky and Carlos Prieto

Edited by M.J. Davis Lin and Liz O. Baylen

Original music by Dan Powell and Elisheba Ittoop

Engineered by Chris Wood

Listen and follow The Daily Apple Podcasts | Spotify | Amazon Music

When the Hollywood producer Harvey Weinstein was convicted of sex crimes four years ago, it was celebrated as a watershed moment for the #MeToo movement. Yesterday, New York’s highest court of appeals overturned that conviction.

Jodi Kantor, one of the reporters who broke the story of the abuse allegations against Mr. Weinstein in 2017, explains what this ruling means for him and for #MeToo.

On today’s episode

Jodi Kantor , an investigative reporter for The New York Times.

Background reading

The verdict against Harvey Weinstein was overturned by the New York Court of Appeals.

Here’s why the conviction was fragile from the start .

There are a lot of ways to listen to The Daily. Here’s how.

We aim to make transcripts available the next workday after an episode’s publication. You can find them at the top of the page.

The Daily is made by Rachel Quester, Lynsea Garrison, Clare Toeniskoetter, Paige Cowett, Michael Simon Johnson, Brad Fisher, Chris Wood, Jessica Cheung, Stella Tan, Alexandra Leigh Young, Lisa Chow, Eric Krupke, Marc Georges, Luke Vander Ploeg, M.J. Davis Lin, Dan Powell, Sydney Harper, Mike Benoist, Liz O. Baylen, Asthaa Chaturvedi, Rachelle Bonja, Diana Nguyen, Marion Lozano, Corey Schreppel, Rob Szypko, Elisheba Ittoop, Mooj Zadie, Patricia Willens, Rowan Niemisto, Jody Becker, Rikki Novetsky, John Ketchum, Nina Feldman, Will Reid, Carlos Prieto, Ben Calhoun, Susan Lee, Lexie Diao, Mary Wilson, Alex Stern, Dan Farrell, Sophia Lanman, Shannon Lin, Diane Wong, Devon Taylor, Alyssa Moxley, Summer Thomad, Olivia Natt, Daniel Ramirez and Brendan Klinkenberg.

Our theme music is by Jim Brunberg and Ben Landsverk of Wonderly. Special thanks to Sam Dolnick, Paula Szuchman, Lisa Tobin, Larissa Anderson, Julia Simon, Sofia Milan, Mahima Chablani, Elizabeth Davis-Moorer, Jeffrey Miranda, Renan Borelli, Maddy Masiello, Isabella Anderson and Nina Lassam.

Katrin Bennhold is the Berlin bureau chief. A former Nieman fellow at Harvard University, she previously reported from London and Paris, covering a range of topics from the rise of populism to gender. More about Katrin Bennhold

Jodi Kantor is a Pulitzer Prize-winning investigative reporter and co-author of “She Said,” which recounts how she and Megan Twohey broke the story of sexual abuse allegations against Harvey Weinstein, helping to ignite the #MeToo movement.    Instagram • More about Jodi Kantor

Advertisement