rural definition of health a systematic literature review

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Publications
Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

Advanced Search
Journal List
BMC Public Health

Rural definition of health: a systematic literature review

Charles gessert.

Essentia Institute of Rural Health, 502 E 2nd St, Duluth, MN 55805 USA

Stephen Waring

Lisa bailey-davis.

Geisinger Center for Health Research, Danville, PA 17822 USA

Melissa Roberts

LCF Research, Albuquerque, NM 87106 USA

Jeffrey VanWormer

Marshfield Clinic Research Foundation, Marshfield, WI 54449 USA

We conducted a systematic review of the literature to assess the extent and strength of evidence for rural definition of health. Studies were eligible for inclusion if they were published in English, reported on original research and presented findings or commentary relevant to rural definition of health, were published over the last 40 years, and were based on observations of rural U.S., Canadian, or Australian populations. Two reviewers were assigned to each selected article and blinded to the other reviewer’s comments. For discordant reviews, a third blinded review was performed.

Of the 125 published articles identified from the literature, 34 included commentary or findings relevant to a rural definition of health. Of these studies, 6 included an urban comparison group. Few studies compared rural and urban definitions of health directly. Findings relevant to rural definition of health covered a broad range; however, good health was commonly characterized as being able to work, reciprocate in social relationships, and maintain independence.

This review largely confirmed many general characteristics on rural views of health, but also documented the extensive methodological limitations, both in terms of quantity and quality, of studies that empirically compare rural vs. urban samples. Most notably, the evidence base in this area is weakened by the frequent absence of parallel comparison groups and standardized assessment tools.

Conclusions

To engage and activate rural patients in their own healthcare, a better understanding of the health beliefs in rural populations is needed. This review suggests that rural residents may indeed hold distinct views on how to define health, but more rigorous studies are needed to confirm these findings.

Electronic supplementary material

The online version of this article (doi:10.1186/s12889-015-1658-9) contains supplementary material, which is available to authorized users.

The passage of the Affordable Care Act (ACA) in 2010 heralded a significant shift in American health care, with increased emphasis on population health, disease prevention, and cost containment. These new areas of emphasis are prominent in several initiatives that have emerged in the wake of the ACA, including Accountable Care Organizations (ACOs), which tie provider reimbursements to measures of quality of care and reductions in the total cost of care for assigned populations of patients. The Patient Centered Outcomes Research Institute (PCORI) also emerged, charged with examining the relative health outcomes, clinical effectiveness, and appropriateness of medical treatments. Underlying these initiatives is a renewed emphasis on patient engagement in health care and, more specifically, on patient activation.

While patient engagement and patient activation are often used interchangeably, patient engagement denotes the broad involvement of patients and caregivers in all aspects of health care is based on the principle of shared responsibility [ 1 ]. Patient activation , a component of patient engagement, emphasizes patients’ willingness and ability to take independent actions to manage their health and care [ 2 ]. Patient activation has been found to be associated with better health outcomes, better health care experiences, and reduced probability of adverse markers such as emergency department use, obesity, and smoking [ 2 , 3 ]. Health care systems are encouraging primary care providers to practice patient-centered care, employing strategies that engage with and activate patients. This approach is grounded in the perspective that care should focus on patients and finding a common ground between patients and clinicians when choosing preventive and treatment care pathways.

This focus on patient-centered care challenges policy makers, health care administrators, clinicians, and patient advocates to understand the factors that contribute to effective patient activation. Individuals define “health” on the basis of their personal health-related beliefs, values and knowledge. Improved understanding of how patients think about and define their health is needed to more effectively “activate” patients, and to nurture and support patients’ efforts to improve their health.

For more than 25 years, researchers have suggested that rural populations may have a distinct view of health that differs from other non-rural populations. Seminal research in 1987 by Weinert and Long reported that rural people predominantly associated health with “the ability to work” [ 4 ], but were less likely to regard cosmetic, comfort, or life-prolonging aspects of health as important. In subsequent work, Weinert and Burman concluded that the rural “function-based definition of health” may contribute to delays in seeking health care, even in the face of grave illness [ 5 ]. In a study of the health beliefs of rural elders, Davis et al. found that subjects described health in terms of autonomy and self-reliance; they feared loss of health primarily because it could lead to “being a burden to others” [ 6 ].

These early studies indicate the need for a richer understanding of rural “frames” for health, but they lacked a direct comparison to health views from non-rural counterparts. If rural and non-rural populations indeed commonly think about or define their health differently, then efforts to engage such populations in promoting and preserving health must be better informed, particularly as health care providers increasingly focus on patient-centered care. The purpose of this study was to systematically review and critique the extent and strength of the published literature regarding how persons living in rural areas define health. In addition, we sought comparisons between rural and urban concepts of health. We were interested in findings that could guide improved patient engagement and patient activation in rural communities of the United States and similar industrialized countries. We specifically examined health values and beliefs as constructs rather than knowledge per se , as knowledge generation is better understood as a continuous process influenced by values, beliefs, motivation, skills, and context [ 7 , 8 ].

We conducted a systematic review of the literature to assess the current scientific understanding of rural definitions of health [ 9 ]. The online databases PubMed, CINAHL, PsycINFO, AnthroSource, and Sociological Abstracts were searched, followed by a manual search of the reference sections of studies identified through the online database search. Key search terms that were used were “rural population, “attitude to health,” “health behavior,” “health promotion,” “health belief,” and “health values.” Studies were eligible for inclusion if they were published in English, reported on original research, presented findings or commentary relevant to rural definitions of health, were published over the last 40 years (01/01/1972-03/31/14), and were based on observations of rural U.S., Canadian, or Australian populations. These countries were selected because of their large rural populations, including remote/frontier communities, and their access to Western health care. No restrictions were placed on sample size, research design, or length of follow-up.

For data extraction and synthesis, two reviewers were assigned to each article and blinded to the other reviewer’s comments. For discordant reviews, a third blinded review was performed. Articles were reviewed for content, methodology and rigor, with information collected on study design, characteristics of the study population, whether articles related to rural definitions of health, the definition of rural, and whether there was a comparison group (e.g., rural vs. urban). All information was captured in a Microsoft Excel spreadsheet for summarizing and comparisons. Further details on the review process are provided in Additional file 1 .

The review process identified 34 articles as having fulfilled the criterion of assessing how rural residents define health. These articles were assigned an evidence grade of A, B, or C depending on methodological quality and supporting evidence of the conclusions, based on a previously used adaptation of the American Diabetes Association's (ADA) evidence grading system [ 10 , 11 ].

Because this was a retrospective review of data from previous published studies, no patient informed consent procedures were applicable, and the study was exempt from review by the Essentia Health Institutional Review Board.

Three hundred and eighty two articles were identified using the study’s search terms; an additional fifteen articles were identified as cited references during the review process. From these, 125 articles were selected for initial review by the lead author. Ninety-one articles were excluded because they did not report on original research or were outside of the scope of the current inquiry. Thirty-four were utilized for this review; 4 were commentaries about a rural definition of health [ 4 , 12 - 14 ] and 30 contained findings relevant to a rural definition of health (see Figure 1 ). Of the latter 30 studies, 6 [ 15 - 20 ] included an urban comparison group (see Table 1 ). The remaining 24 articles [ 6 , 21 - 43 ] did not include a comparison group. Few studies compared rural and urban definitions of health directly.

An external file that holds a picture, illustration, etc.
Object name is 12889_2015_1658_Fig1_HTML.jpg

PRISMA diagram of literature review process.

Published research relevant to rural definition of health: with comparison groups (N = 6) and without comparison groups (N = 24)

The 6 studies that compared findings from both rural and urban populations were of primary interest. Findings relevant to a rural definition of health covered a broad range; however, good health was commonly characterized as being able to work, reciprocate in social relationships, and maintain independence (see Table 1 ). In a focus group study by Gessert et al., rural responders were more likely to express greater willingness to accept ill health and even death as natural phenomena, whereas urban residents expressed stronger aversion to death and greater insistence on aggressive end of life care [ 15 ]. In a study to examine factors influencing individual capacity to manage coronary artery disease risk, both gender and culture (rural vs urban) were identified [ 18 ]. Rural residents expressed belief that a “work hard, eat hard” attitude kept them healthy despite the stress of their work and living in a rural environment. Additionally, rural residents would only seek a physician’s help if physical functioning was severely impaired [ 18 ]. Rural vs. urban differences were also evident in driving behavior, with rural residents more likely to participate in risky behaviors and less likely to have confidence in the utility of safety interventions [ 16 ]. Another study found that persons living in the most remote environments were more likely to hold highly stigmatized attitudes toward mental health care and these views were strongly predictive of willingness to actually seek care [ 17 ].

Comments from participants in several of the reviewed studies (see Table 1 ) centered on three traits that influenced their definition of health: independence, stoicism, and fatalism. Thorson et al. found that rural elders were less likely than urban elders to turn to health care providers for issues they considered non-urgent, regardless of how long a particular symptom had been present [ 20 ]. Hoyt et al. concluded that the agrarian ideology of self-reliance and rugged independence, coupled with a lessened sense of confidentiality and increased pressure to conform due to the smaller, more intimate nature of smaller rural environments, was not conducive to seeking mental health care, particularly for males [ 17 ]. Attitudes of rural and urban residents toward seeking medical care were similar in the Harju et al. study [ 16 ], but were somewhat incongruent with self-reported care seeking behaviors. Fear of hospitals was associated with medical adherence in rural residents and good health habits in urbanites [ 16 ].

Original research articles that did not include a comparison group (n = 29) also revealed influential themes among rural residents’ definitions of health: autonomy, avoiding medical care, and spiritual health. Rural elders participating in a study in Alberta (Canada) reported that ability to work and ability to function, irrespective of symptoms or underlying illness, was their definition of “health” [ 29 ]. In a focus group study of individuals from rural communities in Wyoming, “cowboy up to continue doing what you have to do” was a prevailing theme in responses pertaining to how participants viewed health [ 34 ]. Arcury et al., reporting from interviews of elderly residents in two rural communities in North Carolina, concluded that the residents’ definition of health integrated physical, mental, spiritual, and social aspects of health [ 21 ]. Another study of rural elderly in New Mexico reported that the common definition of health consisted of remaining autonomous and independent, avoiding contact with the health care system [ 23 ]. Lastly, from a study that included interviews of rural health providers in Colorado, one provider’s perspective, based on a 90 year old patient still engaged in ranching, was that work at any age gave patients a sense of purpose that kept them going regardless of the physical challenges of getting around [ 35 ].

This review assessed the extent and strength of evidence regarding how rural people in the United States, Canada and Australia view health differently than their urban counterparts. The overarching objective of this review was to better understand rural definitions of health and how they might be applied in health education messaging and patient engagement/ activation strategies related to disease prevention and treatment. This review largely confirmed many general characteristics previously observed on rural views of health, but also documented the extensive methodological limitations of studies that empirically compared rural vs. urban samples. The evidence in this area is particularly weakened by the routine absence of parallel comparison groups and standardized assessment tools, among other limitations.

Despite these limitations, several consistent characteristics of a rural definition of health were identified. Rural populations tend to emphasize functional aspects of health, especially the preservation of the ability to work and to fulfill (traditional) social roles. Rural people tend to frame health in terms of independence and self-sufficiency, and to accept ill health with higher degrees of stoicism and seemingly more fatalism. If more rigorous future studies can confirm these findings in rural populations, health education and patient engagement/activation programs can be better structured in ways that capitalize on the strong underlying motivations to preserve independence through good health practices. Our findings suggest that rural populations might be more responsive to health messages that emphasize physical function, independence, self-sufficiency, and the ability to reciprocate in social roles and perceived obligations.

Projects designed to improve the health of rural populations face a number of challenges. At a macro level, rural settings are not homogenous in terms of culture, economic hardship, or sense of history/community. Accordingly, findings from one rural community or similar group of rural communities may not be applicable to other rural communities or regions. Much of the previous research on rural health reported findings from primarily agrarian samples, which is an increasingly small subset of rural settings and not necessarily similar to other rural areas that rely heavily on manufacturing, forestry, or subsistence occupations. This distinction has become more pronounced in recent years with the growth of rural recreation and retirement communities, as well as other rural environments where the agrarian economy or culture has limited influence.

Individual characteristics are also important in rural health attitudes and beliefs. Several investigators reported that religious or spiritual health was an integral part of the definition of health in the rural communities studied. Socioeconomic status is recognized as a key factor in health attitudes and practices, yet few studies in the current review controlled for the socioeconomic status of rural participants. Age and length of time in the community may also be important because some of the most distinctive rural definitions of health were held by older residents (particularly those who had a life-long history of rural residence). The current review also suggests that some work histories such as lifelong farming or ranching may be associated with the more distinct views of health framed by physical function and capacity to work. A better understanding of rural attitudes and beliefs is needed to engage and activate rural residents in managing their health and care. Thus, further study of how rural residents define health will contribute to the implementation of patient-centered care in rural communities.

This study was limited principally by its focus on industrialized Western countries. Additional research is needed both to examine rural concepts of health in a wider range of settings, especially in the developing world. This study was also limited by the paucity of rigorous studies that compared rural and urban perceptions of health directly. This is a rich arena for future research.

There is increasing interest in engaging and activating patients in their own healthcare. To do so effectively in rural areas, a better understanding of the health beliefs in rural populations is needed. This review suggests that rural residents may indeed define health in their own way (e.g., functional independence). However, a formal assessment of the risk of bias was not performed in this paper because the vast majority of studies were qualitative and did not include direct comparisons between rural vs. non-rural samples. As such, selection bias remains an overshadowing concern in this collective body of literature, highlighting the need for more rigorous studies to confirm our findings. Research on rural definitions of health is further complicated by continuously changing rural lifestyles and landscapes as demographics and economic emphases shift. Despite such challenges, however, further research on rural health beliefs and attitudes is critical as American healthcare reform legislation calls for broader, systems-based strategies to improve the public’s health. To better engage and activate rural patients in their own healthcare, a better understanding of the health beliefs of targeted rural populations is needed.

Acknowledgements

The authors would like to acknowledge Allise Taran (Essentia Institute of Rural Health) for assisting with final preparation of the manuscript.

Abbreviations

Additional file.

Literature search and review methodology.

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

All six of the authors participated actively in the conception and design of this review of the literature; the selection and review of articles from the literature; the drafting, review and editing of the manuscript; and the preparation of the final draft. All of the authors agree to be accountable for all aspects of the work. All authors read and approved the final manuscript.

Authors’ information

CG has conducted and published research on rural health issues, including access to care in rural areas and rural-urban differences in end-of-life care. SW currently conducts and publishes research on rural health issues, particularly in the elderly. LBD has conducted and published research that observed rural and urban disparities in the prevalence of childhood obesity while controlling for socio-economic factors. PC has conducted research regarding rural and tribal communities and health (including behavioral health) outcomes and prevention strategies. MR has focused on patient satisfaction with care as part of her chronic disease and primary care outcomes research. JV has led several studies on rural disparities in preventive healthcare, as well as a large rural health improvement initiative in Minnesota.

Contributor Information

Charles Gessert, Email: moc.oohay@tressegc .

Stephen Waring, Email: [email protected] .

Lisa Bailey-Davis, Email: ude.regnisieg@sivadyeliabdl .

Pat Conway, Email: [email protected] .

Melissa Roberts, Email: gro.hcraeserfcl@streborm .

Jeffrey VanWormer, Email: [email protected] .

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

We're Hiring!
Help Center

Rural definition of health: a systematic literature review

2015, BMC public health

The advent of patient-centered care challenges policy makers, health care administrators, clinicians, and patient advocates to understand the factors that contribute to effective patient activation. Improved understanding of how patients think about and define their health is needed to more effectively "activate" patients, and to nurture and support patients' efforts to improve their health. Researchers have intimated for over 25 years that rural populations approach health in a distinct fashion that may differ from their non-rural counterparts. We conducted a systematic review of the literature to assess the extent and strength of evidence for rural definition of health. Studies were eligible for inclusion if they were published in English, reported on original research and presented findings or commentary relevant to rural definition of health, were published over the last 40 years, and were based on observations of rural U.S., Canadian, or Australian populations. Tw...

Related Papers

Palliative Medicine

Steve Iliffe

Canadian journal on aging = La revue canadienne du vieillissement

Juanita Bacsu

Existing cognitive health literature focuses on the perspectives of older adults with dementia. However, little is known about the ways in which healthy older adults without dementia understand their cognitive health. In rural communities, early dementia diagnosis may be impeded by numerous factors including transportation challenges, cultural obstacles, and inadequate access to health and support services. Based on participant observation and two waves of 42 semi-structured interviews, this study examined healthy, rural older adults' perceptions of cognitive health. By providing an innovative theoretical foundation informed by local perspectives and culture, findings reveal a complex and multidimensional view of cognitive health. Rural older adults described four key areas of cognitive health ranging from independence to social interaction. As policy makers, community leaders, and researchers work to address the cognitive health needs of the rural aging demographic, it is essen...

Rural and Remote Health

Prairie Women's Centre of Excellence, …

Brigette Krieg

Janice Linton

FROM AUTHOR]; Copyright of Journal of Cross-Cultural Gerontology is the property of Springer Science & Business Media B.V. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) LaVallie, D. L., Wolf, F. M., Jacobsen, C., & Buchwald, D. (2008). Barriers to cancer clinical trial participation among native elders. Ethnicity and Disease, 18(2), 210-217. Objectives: American Indians/Alaska Native: are underrepresented in clinical trials. There fore, they must participate in large-scak cancer clinical trials to ensure the generaliz ability of trial results and improve their acces: to high-quality treatment. Our ...

International Journal of Environmental Research and Public Health

Reproductive health disparities in the Appalachian region may be driven by barriers to healthcare access. However, the barriers specific to accessing family planning services in Appalachia have not yet been identified from the perspectives of Appalachian community members. Moreover, it is unclear how community members might perceive elevated levels of opioid use in the region to impact family planning practices. To fill this gap in knowledge, the current qualitative study explored community perspectives about family planning in Appalachia in the context of the opioid epidemic for the purpose of developing a survey instrument based on these responses. We conducted three video call focus group interviews with community stakeholders, those who live, work and are invested in Appalachia (N = 16), and analyzed the responses using Levesque, Harris, and Russell’s (2013) five pillars of healthcare access as a framework to categorize family planning practices and perceptions of service needs ...

Tom Shakespeare

We aim to explore the barriers to accessing primary care for socio-economically disadvantaged older people in rural areas. Using a community recruitment strategy, fifteen people over 65 years, living in a rural area, and receiving financial support were recruited for semi-structured interviews. Four focus groups were held with rural health professionals. Interviews and focus groups were audio-recorded and transcribed. Thematic analysis was used to identify barriers to primary care access. Older people's experience can be understood within the context of a patient perceived set of unwritten rules or social contract-an individual is careful not to bother the doctor in return for additional goodwill when they become unwell. However, most found it difficult to access primary care due to engaged telephone lines, availability of appointments, interactions with receptionists; breaching their perceived social contract. This left some feeling unwelcome, worthless or marginalised, especia...

Journal of Nursing Scholarship

Kathy Magilvy

Diversity of Research in Health Journal

Roger Pilon

RELATED PAPERS

International Journal of Qualitative Methods

Lic. Maria Gabriela Mangas

BMC Public Health

Carol Cummings

Fenna Van Nes

Nursing Ethics

Inbal Hochwald

The Gerontologist

Jordan Lewis

Jordan Lewis, Ph.D.

Joshua Hauser

Sue Lawrence

Rob Beamish

BMC Pregnancy and Childbirth

Tamara Mulenga

The Journal of Rural Health

Cynthia Peden-mcalpine

Journal of Rural Health

Robin Kruse

Journal of Aging and Health

Betty Black

Asian Nursing Research

farahnaz mohammadi

The Journal of Cardiovascular Nursing

Karen Harkness , Kay Currie

Journal of Cross Cultural Gerontology

Jordan P Lewis

Charles Furlotte

International Journal of Geriatric Psychiatry

Health Expectations

Marcella Ambenne

African Health Sciences

Sharon Koehn

BMC Medical Ethics

Joan Bottorff

BMC Health Services Research

INQUIRY: The Journal of Health Care Organization, Provision, and Financing

Drona Rasali

International Journal of Mental Health Systems

Rishav Koirala

Tracy Wasylak

John Stanifer

BMC Geriatrics

Jiong Tu 涂炯

Kate Riegle van West

Annual review of nursing research

Deirdre Thornlow

Bonnie Duran

Ketaki Chowkhani

INNOVATIONS in pharmacy

ellis owusu-dabo

2019 AAG Conference Oral Presentations – Abstracts

Daphne Flynn , Eden Potter [she/her] , Jon McCormack

Emily Hauenstein

Madhusudan Subedi

Canadian Journal on Aging / La Revue canadienne du vieillissement

Wendy Hulko

ACC 2024 Congress

Catch up on all the top stories and latest evidence in cardiology research with our ACC 2024 Congress hub page.

Springer Medicine

Open Access 01-12-2015 | Research article

Rural definition of health: a systematic literature review

Authors: Charles Gessert, Stephen Waring, Lisa Bailey-Davis, Pat Conway, Melissa Roberts, Jeffrey VanWormer

Published in: BMC Public Health | Issue 1/2015

Conclusions

Please log in to get access to this content, other articles of this issue 1/2015.

Research article

Tuberculosis treatment delays and associated factors within the Zimbabwe national tuberculosis programme

Reaching older people with pa delivered in football clubs: the reach, adoption and implementation characteristics of the extra time programme, do discrimination, residential school attendance and cultural disruption add to individual-level diabetes risk among aboriginal people in canada, does the social context of early alcohol use affect risky drinking in adolescents prospective cohort study, physician and patient concordance of report of tobacco cessation intervention in primary care in india, efficient national surveillance for health-care-associated infections.

Medical Journals
Webcasts & Webinars
CME & eLearning
Newsletters
ESMO Congress 2023
2023 ERS Congress
ESC Congress 2023
EHA2023 Hybrid Congress
2023 ASCO Annual Meeting Coverage
Advances in Alzheimer’s
About Springer Medicine
Diabetology
Endocrinology
Gastroenterology
Geriatrics and Gerontology
Gynecology and Obstetrics
Infectious Disease
Internal Medicine
Respiratory Medicine
Rheumatology

Impact of Community Health Workers on Access to Care for Rural Populations in the United States: A Systematic Review

Published: 24 November 2021
Volume 47 , pages 539–553, ( 2022 )

Cite this article

rural definition of health a systematic literature review

Carole R. Berini ORCID: orcid.org/0000-0003-1772-7970 1 , 2 ,
Heather S. Bonilha 3 &
Annie N. Simpson 1

2009 Accesses

17 Citations

2 Altmetric

Explore all metrics

Community Health Worker (CHW) interventions have shown potential to reduce inequities for underserved populations. However, there is a lack of support for CHW integration in the delivery of health care. This may be of particular importance in rural areas in the Unites States where access to care remains problematic. This review aims to describe CHW interventions and their outcomes in rural populations in the US. Peer reviewed literature was searched in PubMed and PsycINFO for articles published in English from 2015 to February 2021. Title and abstract screening was performed followed by full text screening. Quality of the included studies was assessed using the Downs and Black score. A total of 26 studies met inclusion criteria. The largest proportion were pre-post program evaluation or cohort studies (46.2%). Many described CHW training (69%). Almost a third (30%) indicated the CHW was integrated within the health care team. Interventions aimed to provide health education (46%), links to community resources (27%), or both (27%). Chronic conditions were the concern for most interventions (38.5%) followed by women’s health (34.6%). Nearly all studies reported positive improvement in measured outcomes. In addition, studies examining cost reported positive return on investment. This review offers a broad overview of CHW interventions in rural settings in the United States. It provides evidence that CHW can improve access to care in rural settings and may represent a cost-effective investment for the healthcare system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Access to health services among culturally and linguistically diverse populations in the Australian universal health care system: issues and challenges

Resham B. Khatri & Yibeltal Assefa

Public health system challenges in the Free State, South Africa: a situation appraisal to inform health system strengthening

B. Malakoane, J. C. Heunis, … W. H. Kruger

Aboriginal community controlled health organisations address health equity through action on the social determinants of health of Aboriginal and Torres Strait Islander peoples in Australia

O. Pearson, K. Schwartzkopff, … on behalf of the Leadership Group guiding the Centre for Research Excellence in Aboriginal Chronic Disease Knowledge Translation and Exchange (CREATE)

Data Availability

All data is from the published literature.

Code Availability

not applicable.

Bolin, J. N., Bellamy, G. R., Ferdinand, A. O., et al. (2015). Rural healthy people 2020: new decade, same challenges. The Journal of Rural Health, 31 (3), 326–333.

Article Google Scholar

Cardarelli, R., Bausch, G., Murdock, J., & Chyatte, M. R. (2018). Return-on-Investment (ROI) analyses of an inpatient lay health worker model on 30-day readmission rates in a rural community hospital. The Journal of Rural Health, 34 (4), 411–422.

Cardarelli, R., Horsley, M., Ray, L., et al. (2018). Reducing 30-day readmission rates in a high-risk population using a lay-health worker model in Appalachia Kentucky. Health Education Research, 33 (1), 73–80.

Center for Disease Control and Prevention. (2020). About Rural Health. Centers for Disease Control and Prevention. https://www.cdc.gov/ruralhealth/about.html

Cramer, M. E., Mollard, E. K., Ford, A. L., Kupzyk, K. A., & Wilson, F. A. (2018). The feasibility and promise of mobile technology with community health worker reinforcement to reduce rural preterm birth. Public Health Nursing, 35 (6), 508–516.

Crespo, R., Christiansen, M., Tieman, K., & Wittberg, R. (2020). An emerging model for community health worker-based chronic care management for patients with high health care costs in rural Appalachia. Preventing Chronic Disease, 17 , E13.

Downs, S. H., & Black, N. (1998). The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. Journal of Epidemiology & Community Health, 52 (6), 377–384.

Article CAS Google Scholar

Falk, D., Falk, D., Cubbin, C., et al. (2018). Increasing breast and cervical cancer screening in rural and border texas with friend to friend plus patient navigation. Journal of Cancer Education, 33 (4), 798–805.

Falk, D., Foley, K., Weaver, K. E., Jones, B., & Cubbin, C. (2020). An Evaluation of Breast and Cervical Cancer Screening Outcomes in an Education and Patient Navigation Program in Rural and Border Texas. Journal of Cancer Education, 10, Epub ahead of print.

Fawcett, K., Neff, R., Freese Decker, C., & Faber, J. (2018). Achieving IHIʼs triple aim by utilizing core health program with community health workers in rural communities. Family & Community Health, 41 (4), 255–264.

Felix, H. C., Ali, M., Bird, T. M., Cottoms, N., & Stewart, M. K. (2019). Are community health workers more effective in identifying persons in need of home and community-based long-term services than standard-passive approaches. Home Health Care Services Quarterly, 38 (3), 194–208.

Felix, H. C., Mays, G. P., Stewart, M. K., Cottoms, N., & Olson, M. (2011). Medicaid savings resulted when community health workers matched those with needs to home and community care. Health Affairs, 30 (7), 1366–1374.

Feltner, F., Thompson, S., Baker, W., & Slone, M. (2017). Community health workers improving diabetes outcomes in a rural Appalachian population. Social Work in Health Care, 56 (2), 115–123.

Glenn, L. E., Nichols, M., Enriquez, M., & Jenkins, C. (2020). Impact of a community-based approach to patient engagement in rural, low-income adults with type 2 diabetes. Public Health Nursing, 37 (2), 178–187.

Hopper, L. N., Blackman, K. F., Page, R. A., et al. (2017). Seeds of HOPE: incorporating community-based strategies to implement a weight-loss and empowerment intervention in Eastern North Carolina. North Carolina Medical Journal, 78 (4), 230–236.

Hunter, J. B., de Zapien, J. G., Papenfuss, M., Fernandez, M. L., Meister, J., & Giuliano, A. R. (2004). The impact of a “Promotora” on increasing routine chronic disease prevention among women aged 40 and older at the US-Mexico Border. Health Education & Behavior, 31 (4), 18–28.

Jack, H. E., Arabadjis, S. D., Sun, L., Sullivan, E. E., & Phillips, R. S. (2016). Impact of community health workers on use of healthcare services in the United States: a systematic review. Journal of General Internal Medicine, 32 (3), 325–344.

Kitzman, P., Hudson, K., Sylvia, V., Feltner, F., & Lovins, J. (2017). Care coordination for community transitions for individuals post-stroke returning to low-resource rural communities. Journal of Community Health, 42 (3), 565–572.

Krantz, M. J., Coronel, S. M., Whitley, E. M., Dale, R., Yost, J., & Estacio, R. O. (2013). Effectiveness of a community health worker cardiovascular risk reduction program in public health and health care settings. American Journal of Public Health, 103 (1), e19–e27.

Krok-Schoen, J. L., Oliveri, J. M., Young, G. S., Katz, M. L., Tatum, C. M., & Paskett, E. D. (2015). Evaluating the stage of change model to a cervical cancer screening intervention among Ohio Appalachian women. Women and Health, 56 (4), 468–4886.

Logan, R. I., & Castañeda, H. (2020). Addressing health disparities in the rural United States: advocacy as caregiving among community health workers and Promotores de Salud. International Journal of Environmental Research and Public Health, 17 (24), 10.

Luque, J., Tarasenko, Y., Reyes-Garcia, C., et al. (2017). Salud es Vida: a Cervical Cancer Screening Intervention for Rural Latina Immigrant Women. Journal of Cancer Education, 32 (4), 690–699.

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Group, T. P. (2009) Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLOS Medicine, 6 (7), e1000097

NACHW. (2021). CHW Networks and Training Programs. NACHW. https://nachw.org/membership/chw-networks-and-certification-programs/

Nkonki, L., Tugendhaft, A., & Hofman, K. (2017). A systematic review of economic evaluations of CHW interventions aimed at improving child health outcomes. Human Resources for Health, 15 (1), 19.

Postma, J. M., Smalley, K., Ybarra, V., & Kieckhefer, G. (2011). The Feasibility and acceptability of a home-visitation, asthma education program in a rural, Latino a population. The Journal of Asthma, 48 (2), 139–146.

Probst, J., Zahnd, W., & Breneman, C. (2019). Declines in pediatric mortality fall short for rural US children. Health Affairs, 38 (12), 2069–2076.

Reed, P. H., & Hulton, L. J. (2017). Community health workers in collaboration with case managers to improve quality of life for patients with heart failure. Professional Case Management, 22 (3), 144–148.

Riedy, C. A., Weinstein, P., Mancl, L., Garson, G., Huebner, C. E., Milgrom, P., Grembowski, D., Shepherd-Banigan, M., Smolen, D., & Sutherland, M. (2015). Dental attendance among low-income women and their children following a brief motivational counseling intervention: a community randomized trial. Social Science & Medicine, 1982 (144), 9–18.

Roland, K. B., Milliken, E. L., Rohan, E. A., et al. (2017). Use of Community health workers and patient navigators to improve cancer outcomes among patients served by federally qualified health centers: a systematic literature review. Health Equity, 1 (1), 61–76.

Rural Health Information Hub. (2020). Substance Use and Misuse in Rural Areas. Rural Health Information Hub. https://www.ruralhealthinfo.org/topics/substance-use#:~:text=Though%20often%20perceived%20to%20be,in%20towns%20of%20every%20size .

Ryabov, I. (2014). Cost-effectiveness of community health workers in controlling diabetes epidemic on the US–Mexico border. Public Health, 128 (7), 636–642.

Samuel-Hodge, C. D., Gizlice, Z., Allgood, S. D., et al. (2020). Strengthening community-clinical linkages to reduce cardiovascular disease risk in rural NC: feasibility phase of the CHANGE study. BMC Public Health, 20 (1), 264.

Schleiff, M. J., Peters, D. H., & Surkan, P. J. (2020). Comparative case analysis of the role of community health workers in rural and low-income populations of west Virginia and the United States. The Journal of Ambulatory Care Management, 43 (3), 205–220.

Smith, L., Atherly, A., Campbell, J., Flattery, N., Coronel, S., & Krantz, M. (2019). Cost-effectiveness of a statewide public health intervention to reduce cardiovascular disease risk. BMC Public Health, 19 (1), 1234.

Suther, S., Battle, A. M., Battle-Jones, F., & Seaborn, C. (2016). Utilizing health ambassadors to improve type 2 diabetes and cardiovascular disease outcomes in Gadsden County, Florida. Evaluation and Program Planning, 55 , 17–26.

Thompson, B., Carosso, E. A., Jhingan, E., et al. (2017). Results of a randomized controlled trial to increase cervical cancer screening among rural Latinas. Cancer, 123 (4), 666–674.

Trump, L. J., & Mendenhall, T. J. (2017). Community health workers in diabetes care: a systematic review of randomized controlled trials. Families Systems & Health, 35 (3), 320–340.

Turner, B. J., Liang, Y., Rodriguez, N., et al. (2018). Randomized trial of a low-literacy chronic pain self-management program: analysis of secondary pain and psychological outcome measures. The Journal of Pain, 19 (12), 1471–1479. https://doi.org/10.1016/j.jpain.2018.06.010

Vilchis, H., Onstad, L. E., Benavidez, R., et al. (2019). Una Mano Amiga: pilot test of a patient navigator program for Southwest New Mexico. Journal of Cancer Education, 34 (1), 173–179.

Weaver, A., & Lapidos, A. (2018). Mental health interventions with community health workers in the United States: a systematic review. Journal of Health Care for the Poor and Underserved, 29 (1), 159–180.

Yaemsiri, S., Alfier, J. M., Moy, E., et al. (2019). Healthy people 2020: rural areas lag in achieving targets for major causes of death. Health Affairs (Project Hope), 38 (12), 2027–2031.

Download references

Acknowledgements

The authors thank Teri Lynn Herbert for her expert assistance in finalizing the database search strategies.

There was no funding for this study.

Author information

Authors and affiliations.

Department of Healthcare Leadership and Management, Medical University of South Carolina, Charleston, SC, 29425, USA

Carole R. Berini & Annie N. Simpson

Department of Family Medicine, Medical University of South Carolina, 5 Charleston Center Dr., Suite 263, MSC192, Charleston, SC, 29425, USA

Carole R. Berini

Department of Rehabilitation Sciences, Medical University of South Carolina, Charleston, SC, 29425, USA

Heather S. Bonilha

You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the conceptualization of the study; the first author performed the initial literature search and rating, the second author served as second rater, all authors contributed to subsequent analysis; the first author produced the initial draft, all authors critically revised the work and approved the final manuscript.

Corresponding author

Correspondence to Carole R. Berini .

Ethics declarations

Conflict of interest.

Authors have no conflicts of interests to disclose.

Ethical Approval

Because the study is based on previously published information, no institutional review board approval is needed.

Consent to Participate

Not applicable.

Consent for Publication

Additional information, publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Search strategies

PubMed: ((Rural OR “rural health”) AND (“continuum of care” OR “Access to care” OR “health systems” OR “Women’s health” OR “disease prevention” OR “patient education” OR “managed care” OR “program cost” OR “health care system” OR "cost–benefit" OR "cost-effectiveness" OR "ED visit" OR “fees” OR "emergency department visit" OR "emergency room visit" OR "ER visit" OR "urgent care use" OR "urgent care visit" OR cost OR expenditure OR expense OR finance OR hospitalization OR payment OR profit OR readmission OR “Return on investment” OR savings OR utilization OR “insurance, health” [MeSH Terms] OR "costs and cost analysis"[MeSH Terms] OR "Medicaid/economics"[MeSH Terms] OR "Medicaid/utilization"[MeSH Terms] OR "Models, econometric"[MeSH Terms]) AND ("Abuse Counselor" OR "Birth attendant" OR "Case coordinator" OR "Community coordinator" OR "Community health advocate" OR "Community health aide" OR "Community health educator" OR "Community health representative" OR "Community health worker" OR "Community liaison" OR "Community organizer" OR "Community social worker" OR "Doula" OR "Family advocate" OR "Family support worker" OR "Health adviser" OR "Health advisor" OR "Health advocate" OR "Health agent" OR "Health assistant" OR "Health attendant" OR “health care volunteer” OR "Health coach" OR "Health Communicator" OR "Health Educator" OR "Home Care Worker" OR "Home health aide" OR "Home Visitor" OR "Intake specialist" OR "Lay attendant" OR "Lay health advisor" OR “Lay health adviser” OR "Lay health worker" OR "Lay personnel" OR "Medical representative" OR "Mental Health Worker" OR "Nutrition educator" OR "Outreach Coordinator" OR "Outreach Educator" OR "Outreach Worker" OR "Parent Aide" OR "Parent Liaison" OR "Patient navigator" OR "Peer advocate" OR "Peer coach" OR "Peer leader" OR "Promotora” OR "Recovery coach" OR Paraprofessional OR "community health workers"[MeSH Terms] OR "health educators"[MeSH Terms] OR "patient navigation"[MeSH Terms] OR "home health aides"[MeSH Terms])).

PsycINFO: (Rural OR “rural health”) AND (“continuum of care” OR “Access to care” OR “health systems” OR “Women’s health” OR “disease prevention” OR “patient education” OR “managed care” OR “program cost” OR “health care system” OR "cost–benefit" OR "cost-effectiveness" OR "ED visit" OR “fees” OR "emergency department visit" OR "emergency room visit" OR "ER visit" OR "urgent care use" OR "urgent care visit" OR cost OR expenditure OR expense OR finance OR hospitalization OR payment OR profit OR readmission OR “Return on investment” OR savings OR utilization) AND ("Abuse Counselor" OR "Birth attendant" OR "Case coordinator" OR "Community coordinator" OR "Community health advocate" OR "Community health aide" OR "Community health educator" OR "Community health representative" OR "Community health worker" OR "Community liaison" OR "Community organizer" OR "Community social worker" OR "Doula" OR "Family advocate" OR "Family support worker" OR "Health adviser" OR "Health advisor" OR "Health advocate" OR "Health agent" OR "Health assistant" OR "Health attendant" OR “health care volunteer” OR "Health coach" OR "Health Communicator" OR "Health Educator" OR "Home Care Worker" OR "Home health aide" OR "Home Visitor" OR "Intake specialist" OR "Lay attendant" OR "Lay health advisor" OR “Lay health adviser” OR "Lay health worker" OR "Lay personnel" OR "Medical representative" OR "Mental Health Worker" OR "Nutrition educator" OR "Outreach Coordinator" OR "Outreach Educator" OR "Outreach Worker" OR "Parent Aide" OR "Parent Liaison" OR "Patient navigator" OR "Peer advocate" OR "Peer coach" OR "Peer leader" OR "Promotora” OR "Recovery coach" OR Paraprofessional).

Rights and permissions

Reprints and permissions

About this article

Berini, C.R., Bonilha, H.S. & Simpson, A.N. Impact of Community Health Workers on Access to Care for Rural Populations in the United States: A Systematic Review. J Community Health 47 , 539–553 (2022). https://doi.org/10.1007/s10900-021-01052-6

Download citation

Accepted : 12 November 2021

Published : 24 November 2021

Issue Date : June 2022

DOI : https://doi.org/10.1007/s10900-021-01052-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Community Health Worker
Rural Health
Underserved Populations
Access to Health Care
Find a journal
Publish with us
Track your research

Europe PMC requires Javascript to function effectively.

Either your web browser doesn't support Javascript or it is currently turned off. In the latter case, please turn on Javascript support in your web browser and reload this page.

Search life-sciences literature (43,960,127 articles, preprints and more)

Free full text
Citations & impact
Similar Articles

Rural definition of health: a systematic literature review.

Author information, affiliations.

Gessert C 1
Bailey-Davis L 2
Roberts M 3
VanWormer J 4

ORCIDs linked to this article

Roberts M | 0000-0002-1314-4906
Bailey-Davis L | 0000-0002-8781-1521
Waring S | 0000-0002-9674-0083

BMC Public Health , 14 Apr 2015 , 15: 378 https://doi.org/10.1186/s12889-015-1658-9 PMID: 25879818 PMCID: PMC4406172

Abstract

Conclusions, free full text .

Rural definition of health: a systematic literature review

Charles gessert.

Essentia Institute of Rural Health, 502 E 2nd St, Duluth, MN 55805 USA

Stephen Waring

Lisa bailey-davis.

Geisinger Center for Health Research, Danville, PA 17822 USA

Melissa Roberts

LCF Research, Albuquerque, NM 87106 USA

Jeffrey VanWormer

Marshfield Clinic Research Foundation, Marshfield, WI 54449 USA

Electronic supplementary material

The online version of this article (10.1186/s12889-015-1658-9) contains supplementary material, which is available to authorized users.

PRISMA diagram of literature review process.

Acknowledgements

The authors would like to acknowledge Allise Taran (Essentia Institute of Rural Health) for assisting with final preparation of the manuscript.

Abbreviations

Additional file.

Additional file 1: (138K, pdf)

Literature search and review methodology.

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

Authors’ information

Contributor Information

Charles Gessert, Email: moc.oohay@tressegc .

Stephen Waring, Email: [email protected] .

Lisa Bailey-Davis, Email: ude.regnisieg@sivadyeliabdl .

Pat Conway, Email: [email protected] .

Melissa Roberts, Email: gro.hcraeserfcl@streborm .

Jeffrey VanWormer, Email: [email protected] .

Full text links

Read article at publisher's site: https://doi.org/10.1186/s12889-015-1658-9

Citations & impact

Impact metrics, citations of article over time, alternative metrics.

Altmetric item for https://www.altmetric.com/details/3928610

Smart citations by scite.ai Smart citations by scite.ai include citation statements extracted from the full text of the citing article. The number of the statements may be higher than the number of citations provided by EuropePMC if one paper cites another multiple times or lower if scite has not yet processed some of the citing articles. Explore citation contexts and check if this article has been supported or disputed. https://scite.ai/reports/10.1186/s12889-015-1658-9

Article citations, double disparity of sexual minority status and rurality in cardiometabolic hospitalization risk: a secondary analysis using linked population-based data..

Gupta N , Cookson SR

Healthcare (Basel) , 11(21):2854, 30 Oct 2023

Cited by: 0 articles | PMID: 37957999 | PMCID: PMC10650143

Rural-urban disparities in health outcomes, clinical care, health behaviors, and social determinants of health and an action-oriented, dynamic tool for visualizing them.

Weeks WB , Chang JE , Pagán JA , Lumpkin J , Michael D , Salcido S , Kim A , Speyer P , Aerts A , Weinstein JN , Lavista JM

PLOS Glob Public Health , 3(10):e0002420, 03 Oct 2023

Cited by: 0 articles | PMID: 37788228 | PMCID: PMC10547156

Rural health and rural industries: Opportunities for partnership and action.

Scott KA , Elliott KC , Lincoln J , Flynn MA , Hill R , Hall DM

J Rural Health , 40(2):401-405, 05 Sep 2023

Cited by: 0 articles | PMID: 37669228

A perspective on the role of language about pain.

van Rysewyk S

Front Pain Res (Lausanne) , 4:1251676, 02 Aug 2023

Cited by: 0 articles | PMID: 37599863 | PMCID: PMC10433376

The impact of health literacy interventions on glycemic control and self-management outcomes among type 2 diabetes mellitus: A systematic review.

Butayeva J , Ratan ZA , Downie S , Hosseinzadeh H

J Diabetes , 15(9):724-735, 05 Jul 2023

Cited by: 2 articles | PMID: 37407516 | PMCID: PMC10509520

Work disability prevention in rural healthcare workers.

Franche RL , Murray EJ , Ostry A , Ratner PA , Wagner SL , Harder HG

Rural Remote Health , 10(4):1502, 16 Oct 2010

Cited by: 5 articles | PMID: 20964467

The patient experience of patient-centered communication with nurses in the hospital setting: a qualitative systematic review protocol.

Newell S , Jordan Z

JBI Database System Rev Implement Rep , 13(1):76-87, 01 Jan 2015

Cited by: 46 articles | PMID: 26447009

Student and educator experiences of maternal-child simulation-based learning: a systematic review of qualitative evidence protocol.

MacKinnon K , Marcellus L , Rivers J , Gordon C , Ryan M , Butcher D

JBI Database System Rev Implement Rep , 13(1):14-26, 01 Jan 2015

Cited by: 18 articles | PMID: 26447004

Telehealth services in rural and remote Australia: a systematic review of models of care and factors influencing success and sustainability.

Bradford NK , Caffery LJ , Smith AC

Rural Remote Health , 16(4):3808, 17 Oct 2016

Cited by: 89 articles | PMID: 27744708

The effectiveness of internet-based e-learning on clinician behavior and patient outcomes: a systematic review protocol.

Sinclair P , Kable A , Levett-Jones T

JBI Database System Rev Implement Rep , 13(1):52-64, 01 Jan 2015

Cited by: 74 articles | PMID: 26447007

Europe PMC is part of the ELIXIR infrastructure

Systematic review
Open access
Published: 14 March 2012

The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research

Shannon Wiltsey Stirman 1 , 2 , 3 ,
John Kimberly 4 ,
Natasha Cook 1 , 2 ,
Amber Calloway 1 , 2 , 3 ,
Frank Castro 1 , 2 , 3 &
Martin Charns 2 , 5 , 6

Implementation Science volume 7 , Article number: 17 ( 2012 ) Cite this article

55k Accesses

759 Citations

39 Altmetric

Metrics details

The introduction of evidence-based programs and practices into healthcare settings has been the subject of an increasing amount of research in recent years. While a number of studies have examined initial implementation efforts, less research has been conducted to determine what happens beyond that point. There is increasing recognition that the extent to which new programs are sustained is influenced by many different factors and that more needs to be known about just what these factors are and how they interact. To understand the current state of the research literature on sustainability, our team took stock of what is currently known in this area and identified areas in which further research would be particularly helpful. This paper reviews the methods that have been used, the types of outcomes that have been measured and reported, findings from studies that reported long-term implementation outcomes, and factors that have been identified as potential influences on the sustained use of new practices, programs, or interventions. We conclude with recommendations and considerations for future research.

Two coders identified 125 studies on sustainability that met eligibility criteria. An initial coding scheme was developed based on constructs identified in previous literature on implementation. Additional codes were generated deductively. Related constructs among factors were identified by consensus and collapsed under the general categories. Studies that described the extent to which programs or innovations were sustained were also categorized and summarized.

Although "sustainability" was the term most commonly used in the literature to refer to what happened after initial implementation, not all the studies that were reviewed actually presented working definitions of the term. Most study designs were retrospective and naturalistic. Approximately half of the studies relied on self-reports to assess sustainability or elements that influence sustainability. Approximately half employed quantitative methodologies, and the remainder employed qualitative or mixed methodologies. Few studies that investigated sustainability outcomes employed rigorous methods of evaluation (e.g., objective evaluation, judgement of implementation quality or fidelity). Among those that did, a small number reported full sustainment or high fidelity. Very little research has examined the extent, nature, or impact of adaptations to the interventions or programs once implemented. Influences on sustainability included organizational context, capacity, processes, and factors related to the new program or practice themselves.

Conclusions

Clearer definitions and research that is guided by the conceptual literature on sustainability are critical to the development of the research in the area. Further efforts to characterize the phenomenon and the factors that influence it will enhance the quality of future research. Careful consideration must also be given to interactions among influences at multiple levels, as well as issues such as fidelity, modification, and changes in implementation over time. While prospective and experimental designs are needed, there is also an important role for qualitative research in efforts to understand the phenomenon, refine hypotheses, and develop strategies to promote sustainment.

Peer Review reports

All systems and organizations are faced with the challenge of implementing new practices at one time or another, yet many of the innovations that are initially successful fail to become part of the habits and routines of the host organizations and communities. Why do some take root and flourish while others languish? Recognizing the need to promote the use of best practices to achieve better outcomes in healthcare, many government agencies and community organizations have devoted significant resources to promoting research on evidence-based practices (EBPs), clinical guideline implementation, and quality-improvement programs [ 1 , 2 ]. The National Institutes of Health, for example, have given priority to research on the implementation of best practices and evidence-based interventions, and many systems and communities have endeavored to implement specific healthcare interventions or programs to promote improved health outcomes. One consequence of these emerging priorities is the rapid development of the field of implementation science. Most studies in this field thus far have focused on identifying the factors that are critical to the success of initial implementation efforts. While this is a promising start, policy makers and other stakeholders are increasingly concerned with the long-term impact of their investment. However, as Greenhalgh and her colleagues (2004) pointed out in their review of the dissemination and implementation literature, there is a "near absence of studies focusing primarily on the sustainability of complex service innovations" [ 3 ].

The results of program evaluation and research to date suggest that sustainability must be studied as a distinct and dynamic phenomenon [ 4 , 5 ]. Although a variety of factors may create conditions that facilitate initial implementation, their presence or influence may diminish over time [ 6 – 8 ]. Even when initial implementation efforts are successful, interventions or programs do not necessarily continue as originally implemented. At times, discontinuation of a particular intervention may be the result of development or discovery of more effective, efficient, or compatible practices [ 9 ]. Adaptations, partial continuation of a program or intervention, or integration of new practices may occur in response to new evidence, changes in priorities or resource availability, or other contextual influences. At other times, however, failure to maintain an effective program or intervention at a sufficient level of quality, intensity, or comprehensiveness once implemented is at odds with the original goals and intentions of the host systems or organizations [ 10 – 12 ]. New practices may simply be added on top of existing ones rather than becoming fully integrated [ 13 ], which may make them particularly vulnerable to erosion over time [ 14 ]. Unintentional "slippage" can occur as a result of factors such as local staffing conditions, lack of resources, or competing demands [ 4 ]. If these processes result in failure to achieve desired outcomes, negative appraisals of the value of the interventions themselves [ 15 ] can in turn make discontinuation more likely. Understanding these processes and determining how to foster the continuation of effective practices at a level that is sufficient to yield desired health outcomes is at least as important as understanding how to implement them in the first place [ 16 ].

Many factors make it difficult to study sustainability and draw conclusions in the current literature. A fundamental challenge is the tension that exists between the continuation of interventions as originally designed and the need to adapt them for use in contexts that may differ in important ways from those in which they were originally developed and tested [ 5 , 16 , 17 ]. A number of conceptualizations of sustainability have been proposed that reflect differing priorities and perspectives on this issue [ 18 ]. In some models, the intervention, rather than the system into which it is introduced, is the focal point of interest. Such models tend to identify a set of factors or conditions that increase the likelihood of sustainability of a specific intervention [ 17 ]. This approach is very different from models and studies that examine sustainability from an ecological or complex-systems perspective. These models emphasize the interconnection between broader environmental forces, contextual influences, and the program or intervention itself [ 19 , 20 ]. The differing approaches have important implications for the way that research is conducted and the conclusions that can be drawn. For example, the former perspective may reflect an emphasis on determinants of the preservation, fidelity to, or discontinuation of a program or intervention. In contrast, research conducted from an ecological perspective would seek to understand the ways in which the intervention and the local context mutually adapt and evolve [ 21 ] and how this process impacts sustainability. Additional challenges to the study of sustainability and interpretation of the literature include the numerous definitions and related but not entirely equivalent terms that have been used in differing fields, and variation in the timing and method of assessment employed across studies. Furthermore, the assessment of programs, practices, and interventions as varied as community-level prevention programs, medical records systems, psychotherapies, and quality-improvement programs will necessarily limit the extent to which assessment can be standardized.

To better understand the state of research on sustainability to date, we reviewed studies that investigated whether or to what extent programs or interventions that had previously been implemented were sustained, and those that sought to understand factors that influence their sustainment. We present an overview of the ways that some key research considerations have been addressed from this perspective in a variety of fields, and we allow what we found using this approach to guide our synthesis of the results and recommendations. For the purposes of this review, we consider relevant studies to be those that identified interventions, procedures, or programs that were implemented to achieve specific program-, patient-, or population-level benefits. We reviewed studies that examined (1) sustainability outcomes [ 22 ] (such as the continuation of some or all components [ 23 ] or the desired recipient-level outcomes that occurred after initial efforts to implement, fund, or study a new practice were complete) or (2) influences on the sustainment of these programs or innovations. This review included studies that used a variety of terms to describe sustainability (e.g., "maintenance," "durability," "institutionalization," and "routinization" [ 9 , 24 , 25 ]) and the decision to discontinue or the failure to sustain programs or interventions (e.g., "de-adoption," "divestment," "exnovation," and "discontinuation" [ 6 , 26 , 27 ]). However, for the purpose of consistency in the current review, we will primarily use the terms "sustainability" (or "sustainment") and "discontinuation," respectively. Specific questions that guided our review include the following:

How has sustainability been defined?

At what levels and units of analysis has it been studied?

What research methods have been used?

Over what time periods?

What outcomes have been reported in the empirical literature?

What were the findings?

What has research told us to date about influences on sustainment?

Our findings provide an overview of the current state of the research literature on the sustainment of specific interventions and programs that were implemented to achieve particular goals or benefits. By looking broadly at efforts to study the phenomenon, it may be possible to distill those considerations that should be integral to programs of research that examine the sustainability of specific interventions, programs, and practices [ 23 ]. Based on these findings, we will make a number of recommendations for defining, assessing, and studying this topic in future research.

Search method

We searched the MEDLINE, ISI, PsycINFO, Academic Search Premier, Health Source, ERIC, and Google Scholar databases using the terms "sustainability," "implementation," "long-term implementation," "routinization," "discontinuation," "de-adoption," "durability," "institutionalization," "maintenance," "capacity building," and "knowledge utilization." Truncated forms of these terms (e.g., "sustai*", "routini*", "institutionali*") and alternative spellings were included in the search. We also employed a snowballing strategy, in which we searched the reference sections of reviews and theoretical papers on implementation and sustainability [ 2 – 4 , 19 , 23 , 25 , 28 , 29 ] and those found in our review. We searched the tables of contents of key journals and journals that had published more than one relevant study on sustainability. These journals included the following: Academy of Management Review, Academy of Management Journal, Administrative Science Quarterly, American Journal of Public Health, Administration and Policy in Mental Health and Mental Health Services Research, American Journal of Evaluation, Implementation Science, Health Services Management Research, Health Services Research, Healthcare Management Review, Journal of Healthcare Management, The Journal of Nursing Administration, The Journal of General Internal Medicine, Medical Care Research and Review, Millbank Quarterly , and Psychiatric Services . Additionally, we examined papers that had cited influential models or reviews of implementation or sustainability [ 2 , 3 , 5 , 19 , 25 , 30 – 32 ]. Finally, we provided the list of articles that were found using these strategies to four individuals known to the investigators who study implementation or sustainability and asked them to share additional articles that they were aware of that had not been included. This yielded nine additional studies.

Inclusion and exclusion criteria

Our inclusion criteria included peer-reviewed studies that addressed sustainability of specific interventions or programs, were written in English, and were published or in press by July 2011. Because sustainability has been defined in numerous ways, we included all studies in which the authors used one of the terms described above or in which an effort was made to determine the extent to which a program or intervention continued after an initial period of training, implementation, or study. Studies were coded only if they included a methodology or procedure designed to identify (1) the status of the program after the initial implementation effort or funding has ended (e.g., fidelity, percent implemented, presence or absence of key components, or discontinuation); (2) the program-, service-, or recipient-level outcomes measured after external support or funding was withdrawn; or (3) the influences on the persistence of the implementation, whether or not the primary focus of the article was sustainability. Articles were excluded if they (1) reported only on initial implementation efforts, (2) were purely narrative accounts or papers on "lessons learned" that did not examine sustainability using qualitative or quantitative research methodologies, (3) reported only long-term follow-up of individuals after a clinical trial or intervention study, or (4) contained insufficient information to determine whether inclusion or exclusion criteria were met (e.g., ambiguity or failure to report the timeframe during which measures were collected). Studies were considered to focus on initial implementation efforts if the original training, supervision, monitoring, or funding support was ongoing throughout the time period of the research (unless monitoring was considered a central element of the program or conducted strictly to assess sustainability, with minimal or no feedback provided).

Review methods

All titles and abstracts retrieved by electronic searching were reviewed by one reviewer, who screened out papers that were not related to implementation (e.g., articles related to sustainable agriculture or discontinuation of medications in the context of clinical interventions). Where it was not possible to exclude articles based on title and abstract, full text versions were obtained and their eligibility was assessed. Full text versions of all potentially relevant articles identified from the reference lists of included articles were obtained. Papers related to implementation were screened independently by two reviewers, and those studies that did not meet the inclusion criteria were excluded. Raters agreed on 95% of the papers that were excluded and agreed on the reasons for exclusion for 91% of the articles. Disagreements were resolved through discussion and consensus.

Figure 1 is a modified PRISMA (which stands for Preferred Reporting Items for Systematic Reviews and Meta-Analyses [ 33 ]) diagram summarizing the selection process, which includes reasons that potentially relevant papers were excluded. A total of 460 published articles were found and considered; 125 were determined to be relevant for coding (An additional file lists studies included in the review [see Additional file 1 ]). Of the papers included, 100 focused on or explicitly addressed sustainability. The remainder of the papers contained follow-up data on implementation from an intervention or training study, or focused primarily on dissemination or implementation but included information about sustainability.

Diagram of Study Selection and Exclusion Process .

An initial coding scheme was developed based on constructs identified in previous conceptualizations of implementation [ 2 – 4 , 9 , 34 , 35 ] and sustainability [ 4 , 5 , 17 , 20 , 36 – 44 ]. Additional codes were generated deductively by the raters if a construct or process identified in the literature was not represented in the coding scheme. Related constructs among potential influences on sustainability were identified by consensus and collapsed under the general categories described in the findings. Thirty percent of the papers included in the review were coded by two raters and rater agreement was assessed. Agreement (Cohen's kappa) ranged from .85 to 1 ("substantial" to "almost perfect" [ 45 ]) on the broad categories and from .61 to 1 ("moderate" to "almost perfect" [ 45 ]) on more specific categories, which were later collapsed into the three broad categories. Additionally, disagreements on four items that were coded at lower frequencies with moderate agreement (.61-.80) were resolved by discussion and consultation if necessary with co-authors, resulting in consensus ratings. Two coders also rated 40% of the health-related (medical, public health/health promotion, or mental health) studies that reported sustainability outcomes for assessment method and the presence or absence of an indication of the level of quality or fidelity. Raters agreed on 93% of the ratings for assessment method and 90% for indication of quality or fidelity. The few disagreements were resolved through discussion.

Characteristics of included studies

Area of study.

Our search procedure identified studies from a variety of fields. Forty-one (33%) of the studies reported on medical interventions or healthcare programs, 42 (34%) on public health or health promotion programs, 33 (27%) on mental or behavioral health interventions, and 9 (7%) on educational interventions. Eighty-eight (72%) of the studies examined either programs or multicomponent interventions as opposed to a single procedure or intervention, such as a discrete medical procedure.

Unit of analysis

The majority of the studies (67; 54%) reported on sustainability at multiple implementation sites or settings, followed by studies that reported on sustainability at the individual or provider level (15; 12%). The remainder of the studies reported on sustainability within single systems or communities (20; 16%), at a single site (11; 9%), among individual providers within sites (7; 6%), or at the team level (5; 4%).

Timeframe examined

Studies were coded for the last post-implementation timeframe reported. Most studies (80; 64%) occurred two years or more past the initial implementation. Seven (6%) reported outcomes at less than 12 months post-implementation, 20 (16%) at 12 months, and 15 (12%) between 12 and 24 months post-implementation.

Terms and definitions

Sustainability was defined in a number of ways, and different terms were used to refer to the continuation of an innovation within an organization or community. Table 1 includes a listing of authors whose definitions were cited in the literature as working definitions of sustainability, as well as the frequency with which terms related to sustainability were used. Sixty-five percent of the studies examined did not present a definition. Among the studies that did present definitions, definitions were most commonly generated by the investigator. The most commonly used term in the studies examined was "sustainability," which was used in 62% of the articles. Those who cited a specific, published definition as their operational definition most frequently cited Scheirer's definition [ 25 ], which was based on the framework set forth by Shediac-Rizkallah and Bone [ 5 ], whose review was the second most commonly cited. Both identified multiple aspects of sustainability: continued benefits, continued activities, and continued capacity.

Methods used

Almost half of the studies reviewed employed self-report measures (54; 43%; nearly all were developed specifically for the project or study) or interviews (50; 40%) to assess sustainability or its influences. Fifty-four (43%) included some form of observation, 35 (28%) involved record review, and 23 (19%) included assessment of the fidelity/integrity of an intervention or practice. Eight (7%) of the studies reported on sustainability after an intervention had been implemented in a clinical trial. Solely quantitative approaches were used in 68 (54%) of the studies, qualitative approaches alone were used in 27 (22%), and 28 (23%) of the studies employed both qualitative and quantitative strategies. Nearly all examinations of sustainability were naturalistic rather than experimental. However, seven studies (6%) involved experimental manipulation of training or implementation strategies and assessed self-reported use, skill, or fidelity at a follow-up.

Outcomes reported

Studies discussed or reported on a variety of outcomes, and some reported multiple outcomes. Fifty-seven health-related studies (45% of the studies reviewed) reported outcomes such as the proportion of sites or providers sustaining, or the proportion of eligible patients receiving an intervention. The remaining studies did not report sustainability outcomes or reported data in such a way that it was not possible to determine the extent to which an intervention or practice was continued. For example, some reported on factors related to sustainability, without describing sustainability outcomes. Among the 57 studies that reported outcomes, 51 reported the proportion of sites or providers sustaining or discontinuing an intervention or program. The remaining studies reported the percent of patients or communities that received an intervention during a follow-up period. Seventy-five (60%) of the reviewed studies reported changes in the rate of program implementation and/or recipient outcomes, and two studies reported changes in both. Twenty-seven (22%) of the studies reported some form of health outcome (sustained impact or increases/decreases in desired outcomes), 14 of which were published in or after 2010.

Summary of findings

Figure 2 contains a summary of the sustainability outcomes reported for medical, public health/health promotion, and mental health studies. In general, a wide range of outcomes was reported. Rates of continuation of some, but not all, program or intervention elements ("partial sustainability") were relatively high across fields and units of analysis. Sixteen studies employed a form of independent observation and/or fidelity assessment to evaluate sustainability outcomes. In light of the literature that self-report assessments are often inaccurate [ 59 ], the figure distinguishes studies that employed observation from those that solely employed self-reports. Few studies that included independent observation or validation reported high rates of continuation at the site or setting level. The studies that reported on full sustainability or high fidelity at the provider level indicated that fewer than half of the observed providers sustained the practices at a high level of skill, intensity, or fidelity.

Sustainability Outcomes By Field . Note: More rigorous studies are defined as having included independent or objective observation and a judgment of fidelity, quality, or level of implementation. Ranges are provided when multiple studies reported these rates.

Of the 75 studies that reported on changes in implementation or recipient-level outcomes after initial implementation efforts or funding had ended, 56 studies reported on the intervention or program implementation. Of these, 19 reported lower levels of implementation after initial implementation efforts had ended, 17 reported an increase, and 3 reported no change or a similar level of implementation. Seventeen studies reported varying changes in rates across different intervention or program components. Twenty-one studies assessed changes in outcomes: 5 reported a decrease in desired outcomes, 10 reported an increase, and 1 reported no change. The remaining five studies reported multiple outcomes or indicators that varied in the extent to which they sustained.

Associated elements and influences

Thirty studies employed quantitative methodologies to identify predictors, correlates, or associated factors of sustainability. Thirty-six studies employed qualitative or mixed methodologies to identify influences on or processes associated with sustainability. Twenty of these studies specified that they were guided by a conceptual framework. Four broad categories of potential influences emerged in our coding process: influences related to the innovation, organizational context, capacity (internal and external), and processes. These four categories were common among each of the health-related fields we examined. However, only eight of the quantitative studies that examined elements related to sustainability included all four areas in their analyses, and 12 examined factors related to both the organization and capacity (typically, characteristics or attitudes of the workforce). Twelve of the studies, all of which employed qualitative or mixed methodologies, found that elements in all four categories were associated with sustainability.

Table 2 summarizes the findings, which are organized by study method (qualitative or quantitative) and health-related field (medicine/health care, public health/health promotion, and mental health). Findings regarding specific innovation characteristics and contextual factors were fairly consistent across medical/health care, public health/health promotion, and mental health studies. Findings related to capacity varied somewhat across fields. The presence of a champion was a less frequent finding for public health studies; funding was a much more common finding in this area. Workforce-related findings (e.g., adequate staffing, attributes of personnel) were less frequent findings in health care, and community support was less frequently identified as associated with the sustainment of mental health programs or interventions. Findings related to processes emerged most commonly in qualitative studies and were identified most commonly in public health programs. Perhaps due to the nature of the instruments or assessment procedures used in quantitative studies, processes were rarely identified. Engagement of stakeholders was more frequently associated with sustainability for public health studies, and adaptation of the intervention and alignment between the innovation and the setting were less frequently found in mental health studies.

We examined 125 published papers to identify the different methodologies, types of innovations studied, timeframes examined, definitions used, outcomes examined, and factors examined in research on sustainability to date. Discussions in the literature over the past two decades [ 5 , 25 , 32 ] regarding the importance of sustainability appear to have resulted in an increase in research on this topic. However, our review found relatively few comprehensive or methodologically rigorous studies. The majority of the studies were retrospective. Most did not provide an operational definition of sustainability, and fewer than half appeared to be guided by a published definition or model of the concept. Few employed independent evaluation or observation.

Sustainability outcomes

Because of the variety of results reported in the studies we reviewed, it is difficult to quantify or generalize about the extent to which new programs and practices are sustained. However, three findings are notable. First, similar to findings from a previous review [ 10 ], we found that those studies that provided information about levels or extent of implementation generally indicated that partial sustainability was more common than continuation of the entire program or intervention, even when full implementation was initially achieved. Most projects did not maintain all aspects as originally designed or implemented. However, in the studies that we reviewed, it was not possible to determine the impact of partially sustaining interventions on recipient-level outcomes. Further, virtually no studies revealed the nature of the changes made, the reasons for the changes, or the process by which adaptations or decisions to discontinue elements of the program or intervention were made. A second key finding is that in the past five years, there has been an increase in the number of studies that reported data on the sustainability of patient- or recipient-level benefits. Future studies that further examine these outcomes will be critical to understanding whether, and to what extent, the health-related benefits of implementation efforts can be sustained over time. Finally, the studies that employed independent fidelity ratings to assess sustainability at the provider level indicated that fewer than half of the providers sampled continued the practice or intervention at high levels of fidelity. These findings suggest that the development and study of fidelity-maintenance strategies, such as training and supervision, audit and feedback, building triggers into the process of care, checklists, or reminders, may be particularly important for the sustainment of interventions that require a high degree of fidelity to produce the intended health benefits [ 60 – 62 ].

Influences on sustainability

Our review found that although terminologies and areas of emphasis differ somewhat across fields, influences on sustainability relate to the context (both outer, e.g., policies, legislation; and inner, e.g., culture, structure), the innovation itself (e.g., fit, adaptability, and effectiveness), processes (e.g., fidelity monitoring, evaluation, efforts to align the intervention and the setting), and the capacity to sustain (e.g., funding, resources, workforce characteristics and stability, interpersonal processes). Some qualitative findings also supported the conceptual literature that suggests an interrelation and interaction between these factors [ 63 ]. The broad categories of influences that our findings appeared to fit overlap most closely with the components of Shediac-Rizkallah and Bone's (1998) and Scheirers' (2005) conceptualizations, which were some of the more commonly cited definitions in the studies that we reviewed [ 5 ]. Within these broad categorizations, however, the key elements that were identified varied considerably.

Findings related to capacity were relatively common in both quantitative and qualitative research. For example, both qualitative and quantitative methodologies identified influences related to the workforce as associated with sustainability. These included the stability of the workforce and attributes of the workforce, such as their skills and attitudes. Additionally, qualitative studies identified the support or participation of key stakeholders and funding as important influences. Funding was rarely measured or included in the analyses, perhaps because studies took place after the initially allocated funding and resources had been removed. While some studies explicitly assessed or discussed the availability of new funds to support the programs that were being studied, most did not indicate whether additional funding had been obtained or allocated. However, influences such as sufficient resources and staffing that were identified in qualitative studies may be indicators of the adequacy of funds.

Other elements that are included in conceptualizations of sustainability [ 18 ] rarely emerged in the hypotheses or findings of the studies that we reviewed. Evaluation, feedback, and other quality-improvement processes were also less well represented than expected. Program or intervention effectiveness was identified in only nine studies, despite a fairly common emphasis on the importance of observable benefits within the implementation literature. In contrast to the relatively consistent emphasis on characteristics of the innovation within conceptualizations of sustainability and the broader implementation literature, fewer studies than expected found that characteristics of the innovation were associated with sustainability [ 3 , 9 , 64 ]. The dearth of findings related to innovation characteristics may be due to the lack of influence of the innovation on sustainability, but it may also be due to researchers' lack of attention to these constructs. Some researchers may have viewed innovation characteristics as more central to adoption decisions than to sustainment. Others may have overlooked innovation characteristics because they were examining a single innovation or organization and thus lacked sufficient variability to study the relative impact of factors such as fit or the intervention's complexity. Not surprisingly, among the innovation characteristics that were identified, the fit of the program or intervention with the system or organization and the degree to which the intervention or program could be modified were most common. Finally, given the amount of discussion on leadership, organizational climate, and culture in the literature on implementation and sustainability [ 5 , 20 , 35 , 39 , 65 , 66 ], we expected greater representation of these constructs in the studies that were reviewed.

Processes and interactions were associated with sustainability in nearly three-quarters of the qualitative studies. Integration of the program into policies, collaboration among stakeholders, and ongoing support were commonly identified processes. Findings related to processes that emerged in qualitative studies may explain why factors such as culture and climate were rarely identified in the studies we reviewed, despite their prominence in the implementation literature. Those who were interviewed in qualitative studies may have been more likely to describe noticeable processes and interactions that are evidence of a particular culture or leadership style than to characterize the culture of an organization. For example, some processes identified in our review, such as integration into policies or standards, may serve as "culture-embedding mechanisms" [ 67 , 68 ]. Similarly, negotiation, relationships, and shared decision making may be fostered by effective leaders and found more commonly in contexts that are prepared to nurture and sustain new practices. Studies with quantitative designs were less likely to identify processes and interactions of this nature, perhaps due to their design and research questions, as well as the models and measures that were employed. Important processes may have been subsumed, identified, or obscured under related and more readily measured constructs. The process-related findings highlight the importance of investigating the ways in which influences at multiple levels may interact to impact sustainability [ 19 ]. For example, some processes that were identified in our review suggest mutual adaptation between the intervention and the organization or system (e.g., adaptation of the intervention to improve fit, alignment of the organizational procedures with the intervention), or important interactions between stakeholders in various roles (e.g., negotiation, navigating competing demands). Such findings suggest that interplay between contextual factors and the innovation itself is to be expected given the dynamic nature of the complex systems into which innovations are introduced [ 19 , 69 ].

In summary, the findings that we presented above illustrate the variability in methods, outcomes, and potential influences that have been studied to date. Based on the empirical literature that we reviewed, it is difficult to generalize about influences on sustainability and the long-term impact of implementation efforts. As research develops further, refinements in conceptualizations and study designs will lead to results that are more easily interpreted. Below, we discuss some considerations and recommendations for such research.

Recommendations for advancing the empirical literature

Defining sustainability.

An important limitation to the body of research on sustainability that we reviewed is the high proportion of studies that did not present a working definition or demonstrate evidence of guidance by a model of sustainability. The way that the concept is defined and conceptualized has important implications for how it is investigated. At a basic level, the studies that we reviewed focused on the continuation of the programs and practices that were implemented within organizations, systems, or communities after initial implementation efforts or funding ended [ 70 ]. While such a broad definition applies across a number of disciplines and contexts, research based on such a definition can yield results that are difficult to interpret, particularly when the studies conclude that some aspects of a program or innovation continued while others did not. Thus, we recommend that both a definition and a conceptual framework be carefully chosen to guide research in this area.

In light of our review, we suggest that investigators consider several factors in choosing a definition to guide their research on the sustainment of interventions or programs and that they clearly specify their research questions regarding each factor. These factors are (1) whether, and to what extent, the core elements (the elements most closely associated with desired health benefits) [ 23 , 32 , 64 ] are maintained; (2) the extent to which desired health benefits are maintained or improved upon over time after initial funding or supports have been withdrawn; (3) the extent, nature, and impact of modifications to the core and adaptable/peripheral elements of the program or innovation [ 23 , 32 ]; and (4) continued capacity to function at the required level to maintain the desired benefits. A program or intervention's impact may be considered sustained if desired health benefits remain at or above the level achieved during implementation and this increase can be attributed to continuation of the program. A program or intervention may be considered to be sustained at a given point in time if, after initial implementation support has been withdrawn, core elements are maintained (e.g., remain recognizable [ 13 ] or delivered at a sufficient level of fidelity or intensity to yield desired health outcomes [ 59 , 62 , 71 ]) and adequate capacity for continuation of these elements is maintained.

Defining outcomes or desired benefits

As our discussion of elements of sustainability above indicates, the desired impact and benefits of the program or intervention should be identified. Additionally, stakeholder goals for sustainability (e.g., Must the program be sustained at the same level, or improved upon? To what extent is a lower level of implementation fidelity or a partially sustained program consistent with stakeholders' goals for the project? At what point, and under what circumstances, is discontinuation, modification, or implementation of a more effective, efficient, or better-fitting intervention advisable?) should be considered in the interpretation of findings. The type of innovation and setting will drive some of these considerations. For specific interventions identified to improve patient-level outcomes (e.g., reduce rates of infection, relieve symptoms), these health benefits may be considered to be the "bottom line." For programs formed to identify and implement multiple interventions to achieve health-related goals, outcomes such as indicators that programs are being implemented, the existence and functioning of a decision-making body, and coordination between multiple agencies or stakeholders may be critical outcomes in addition to population-level outcomes such as reduced rates of disease or infection.

Choosing an appropriate timeframe

When studying the sustainment of a program or intervention, a timeframe that is sufficiently beyond an initial implementation effort to provide meaningful information must be chosen. Although there may be no obvious indicator for determining when initial implementation efforts have "ended," it can be useful conceptually to separate the period of initial implementation from a post-implementation phase. Most of the studies we reviewed examined sustainability two or more years after implementation, consistent with suggestions in the literature [ 24 ]. Although many existing conceptualizations imply that sustainability cannot be studied until full implementation is achieved [ 31 ] and funding is withdrawn [ 4 ], some programs may never be fully implemented due to a variety of forces within or external to systems and organizations. For example, some may have funding withdrawn before full implementation is achieved [ 5 ], yet these programs may achieve success in maintaining some components of the programs over time. Additionally, as most studies that we reviewed measured sustainability at a single time point, they may have masked what several conceptualizations present as a dynamic phenomenon. To advance what is currently known about sustainability over time and to capture variations over time, we suggest that researchers assess sustainability over several years rather than at a single time.

Studying fidelity and adaptation

Consistent with discussions of sustainability that suggest that adaptation and evolution of the practices and innovations are to be expected [ 25 , 32 , 72 , 73 ], a number of studies that we reviewed indicated that some form of modification had occurred. While such changes may be made to interventions or programs in response to contextual influences, such as shifting priorities or availability of resources, the process and nature of adaptations may vary considerably between projects. Most studies that we reviewed did not describe adaptations or examine their impact on health-related outcomes. To facilitate a greater understanding through future research, some clarity regarding adaptation and fidelity is necessary. Additional research is needed to assess the conditions under which fidelity, or different types and degrees of adaptations, are important for the achievement of specific health benefits. While it is important to differentiate sustainment from entrenchment, which may prevent further innovation or adoption of more effective practices [ 16 , 31 , 32 ], it is also critical to understand when, and to what components of a particular program or intervention, fidelity is necessary. Fidelity has been conceptualized in the mental health literature as a combination of adherence to a prescribed set of practices at adequate dose or intensity, competence in delivery, and differentiation from other interventions [ 59 , 74 ], with judgments of competence taking response to certain contextual factors into account [ 23 ]. In the medical literature, it has been defined as "the extent to which the system provides patients the precise interventions they need, delivered properly, precisely when they need them" [ 71 ]. Evidence has emerged that for some interventions, a higher level of fidelity or intensity may be required to produce desired health benefits [ 11 , 62 , 74 ]. In these cases, insufficient levels of fidelity may in fact indicate that a program was not sustained at the level necessary to promote these outcomes. On the other hand, the success of some programs (e.g., community-based health promotion programs) may be less dependent on the implementation of a set of procedures with fidelity than on the flexibility and adaptive capacity of the system or organization that implements the program. In such cases, the range of possible or even necessary adaptations within the program might be quite broad [ 75 , 76 ] and may reflect new priorities or response to local conditions [ 77 ]. This type of ongoing evaluation, modification, and replacement of elements or procedures as necessary is an approach advocated in organizational learning and continuous quality-improvement literatures [ 78 – 80 ]. Theory in this area suggests that an appropriate balance between exploration of new methods while exploiting existing knowledge regarding effective strategies may in fact result in more sustainable and successful programs [ 81 , 82 ].

Simply measuring fidelity and characterizing modifications as deviations may obscure the very refinements that facilitate the continued use of some innovations. A period of mutual adaptation [ 83 ] is probably common between initial implementation and institutionalization, and some innovations may continually evolve [ 76 ]. To advance the field, subsequent research should include further attention to the nature of the modifications that occur and the process by which modifications are made [ 84 , 85 ]. Even for those interventions for which there is evidence that fidelity is important, there may be aspects that can be adapted and modified, while preserving desired outcomes [ 23 , 86 , 87 ], provided that the critical elements are conducted or delivered at adequate levels of fidelity. Several types of modification, at either a molecular or molar level [ 74 , 88 ], may occur as practitioners, communities, and systems implement specific programs and interventions. For example, tailored adaptation may be guided by available evidence and remain faithful to identified core elements [ 23 ], with an eye towards facilitating desired health benefits. Evolution may occur if procedures are modified in light of the emergence of new evidence [ 89 ]. Replacement may occur if more compatible or effective interventions or procedures are identified [ 9 ]. Adaptations that result in reversion or erosion fail to preserve the core elements of an intervention, which may in turn result in a failure to preserve desired health outcomes. In such cases, if the intervention that was originally introduced becomes unrecognizable [ 13 ], it may be considered to have been discontinued. We therefore recommend that when the intervention is the focal unit of interest, in addition to identifying methods of assessing fidelity, researchers study periods of adaptation [ 90 ] and characterize the nature of modifications made to interventions. It will also be important to understand more about the nature of possible trade-offs that are made between fidelity and sustainability and how stakeholders make such decisions.

Identifying core elements, or components that are critical for the achievement of desired outcomes, is also a critical area for future study. Developers of many complex interventions have not yet pursued these questions. Isolating elements of innovative practices and examining their relative contributions to the overall impact of the practice can be challenging and may not be feasible or desirable in some situations. However, when available, this information can facilitate a streamlined or pragmatic implementation effort that retains the aspects found to be most effective and successful in everyday practice [ 16 ]. As a positive impact on intended recipients is the ultimate goal of implementation, we recommend that researchers include a consideration of these important matters in their efforts to study sustainability.

Conceptualization, measurement, and assessment of influences on sustainability

Findings from our review suggest that the study of influences on sustainability is nascent. Fewer studies than we expected identified influences that are found in existing conceptualizations of sustainability. It is possible that these findings result from a lack of guidance by a theoretical framework, given that fewer than one-third of the studies that we reviewed were guided by an explicit model. To advance research in this area, we recommend that researchers identify models or frameworks of sustainability [ 91 ] that are most appropriate for their projects and research questions. In doing so, consideration should be given to the issues regarding fidelity, the potential for adaptation, and the nature of the system that will be studied. As many models of sustainability have not been evaluated [ 19 , 20 , 92 ], we do not yet know enough about which models are valid and appropriate [ 93 ] for differing programs and circumstances [ 16 ]. Thus, efforts to evaluate conceptualizations of sustainability can further advance the field.

There is also room for improvement in methods employed to characterize intervention sustainability and its influences. Beyond fidelity measures used in clinical trials, there are few procedures or benchmarks to guide researchers in efforts to identify the extent to which interventions and programs were continued as implemented. Pluye and colleagues operationalized definitions of three degrees of sustainability for public health programs (weak, moderate, and high) but did not develop a formal assessment instrument. However, they did develop a 15-question interview to assess degrees of sustainability [ 41 ]. The Level of Institutionalization scale has been developed to gauge the extent to which key activities for a health-promotion program have occurred [ 94 , 95 ], and the authors suggest that the measure can be modified easily for a variety of health-promotion programs and settings. When fidelity is necessary to sustain outcomes, observation using a set of criteria for adequate skill, adherence, or intensity will improve the precision with which results are reported. As in other areas, self-reports of fidelity are likely to be imprecise [ 59 ]. The development of valid, yet low time- and cost-intensive, observation or monitoring strategies would represent a significant advance [ 59 ]. Triangulation of information gathered through multiple methods may ultimately be most informative. As fidelity measures are generally not designed to assess, describe, or elucidate the nature and consequences of adaptations, methods of assessment in this area must also be advanced [ 72 ].

Typically, in research that employs surveys to measure influences on sustainment, the instruments were developed for the specific projects or implementation efforts [ 40 ], and psychometric properties were almost never reported. The development of a wholly unique procedure for assessing the sustainability of each intervention or program limits the conclusions that can be drawn from the literature as a whole. In lieu of specific measures, other studies employed survey results or information about setting characteristics collected during the implementation process to identify predictors [c.f.96]. Thus, assessment and analytic strategies employed to date may not have captured the appropriate influences and their interactions. Multilevel measurement of sustainability [ 20 , 29 ], based on sound conceptualization, is necessary to allow for greater methodological rigor and interpretability of findings [ 91 ], and some measures have been developed for this purpose. Mancini and Marek developed a 29-item Program Sustainability Index to assess six factors related to the sustainability of community-based programs [ 40 ]. An instrument was also developed based on the National Health Service Institute for Innovation and Improvement's Health Service Sustainability model. While the model was intended to be used in the planning and early stages of implementation to evaluate the likelihood that an innovation will be sustained, the authors suggest that it can be used at any phase of a project [ 13 ]. Both of these surveys assess factors and processes at multiple levels and can be used to examine the relationship and interactions among differing elements and levels, although further research on their validity and applicability to a broad range of programs or interventions is warranted.

Qualitative and mixed methodologies that assess potential influences across multiple levels will continue to be necessary to refine hypotheses, explore results, understand the relationships between sustainability drivers, and facilitate the development of interventions to promote the sustainability of effective programs and practices. Our review indicates that qualitative studies yielded a wider variety of findings and have highlighted processes and constructs that warrant further study. However, the vast majority did not provide interview guides to clarify how they assessed activities, processes, or influences associated with sustainability. This step will improve the interpretability and replicability of future research. Furthermore, prospective research on sustainability and efforts to identify influences and interventions that lead to sustainable implementation efforts will be of critical importance [ 23 ]. Elucidating the ways in which influences interact to enhance or challenge sustainability will ultimately facilitate an understanding from a complex-systems perspective and may also lead to the development of strategies to promote sustainability in contexts and circumstances in which certain factors are absent or less than optimal. For example, whether specific influences (e.g., leadership, culture) or processes can serve as protective or compensatory factors in the absence of other elements (e.g., funding) remains to be determined. Findings in this area can ultimately lead to the development and improvement of strategies that promote the continuation of effective programs and interventions.

Limitations

Some limitations to our review are important to acknowledge. In this project, we reviewed studies that identified specific programs or interventions and investigated either the extent to which those interventions and/or the desired benefits were sustained or factors that influenced their sustainability. Information generated from this review can inform researchers about what has not yet been sufficiently explored and stakeholders about what may be important to consider when monitoring specific programs or interventions that they have chosen to implement. We did not specifically seek studies that examined the adaptive capacity of systems, and we did not take an ecological or developmental perspective in our review [ 19 , 97 ]. Such perspectives are valuable for future research and much can be learned by broadening the research questions beyond whether or not an intervention continued as originally implemented. However, from a number of stakeholder perspectives, and given the substantial resources that have been devoted to implementing effective practices to date, there is also value to understanding the findings and limitations of the existing body of research that has investigated whether and how interventions and their health benefits have been sustained [ 16 ]. Thus, in this review, the ways in which we presented our findings, conclusions that we drew, and recommendations that we made were shaped by an effort to understand more about sustainment or discontinuation from this perspective and by the state of the existing literature that has addressed sustainability in this manner.

Although we attempted to identify studies from a variety of fields using a number of search strategies, the diffuse nature of the literature on sustainability and the variety of terms used may have limited our ability to complete an exhaustive review. Additionally, we sought to look broadly across literatures from a number of fields, but the applicability of some findings to any one innovation may be somewhat limited. We sought to learn what the available findings could tell us about the extent to which specific practices or programs have become rooted and sustained within organizations and communities, in order to conduct the most comprehensive review possible. By "casting a broad net" in terms of the fields and methodologies that were represented in our review, we intended to identify methods, strategies, constructs, and findings that may not have been considered within some individual fields. In these studies, the extent to which a program or intervention had continued was generally assessed at a single point in time, limiting conclusions that could be drawn about changes over time. Thus, we chose to present ranges of sustainment that had been found within particular fields in lieu of a definitive statement about whether or to what extent sustainment could be expected for particular innovations.

In the early efforts to study the sustainability of specific programs and interventions that we reviewed, we have identified a body of literature that is fragmented and underdeveloped. In addition to previously noted challenges, limited funding for monitoring programs after initial implementation, challenges to observation in real-time, and the lack of validated measures have complicated the study of sustainability, and much of what is known to date has been determined through post hoc research [ 4 ]. The current paper contributes to the literature by reviewing the research on sustainability that has been conducted to date. Our goals in this review were to examine the ways that researchers have approached this challenging topic thus far and to contribute to the development of an agenda for future, high-quality research.

Each of our recommendations to advance what is known about sustainability will require time, resources, and funding, all of which have been relatively limited across the fields that we reviewed. With prospective studies of implementation efforts underway, investigators could make a substantial contribution to the field by planning follow-up studies that assess the degree to which the programs or practices are maintained and the nature and implications of changes that are made once implemented. Furthermore, researchers and policy makers should be encouraged to consider the question of sustainability when developing implementation programs and research. Appropriate planning, assessment, and allocation of funds would result in much better understanding of why and how some interventions and programs last and others do not. In an era of increasing budget pressures and greater scrutiny of new investments, nothing could be more desirable as a practical matter as well.

McHugh RK, Barlow DH: The dissemination and implementation of evidence-based psychological treatments. A review of current efforts. American Psychology. 2010, 65 (2): 73-84.

Google Scholar

Proctor EK: Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health Ment Health Serv Res. 2009, 36 (1): 24-35. 10.1007/s10488-008-0197-4.

Greenhalgh T: How to Spread Good Ideas: A systematic review of the literature on diffusion, dissemination, and sustainability of innovations in health service delivery and organization, in Report for the National Co-ordinating Centre for NHS Service Delivery and Organisation R & D (NCCSDO). 2004, London: University College, 362-

Bowman C: Measuring persistence of implementation: QUERI Series. Implement Sci. 2008, 3 (1): 21-10.1186/1748-5908-3-21.

PubMed PubMed Central Google Scholar

Shediac-Rizkallah MC, Bone LR: Planning for the sustainability of community-based health programs: conceptual frameworks and future directions for research, practice and policy. Health Educ Res. 1998, 13 (1): 87-108. 10.1093/her/13.1.87.

CAS PubMed Google Scholar

Massatti RR: The de-adoption of innovative mental health practices (IMHP): why organizations choose not to sustain an IMHP. Adm Policy Ment Health. 2008, 35 (1-2): 50-65. 10.1007/s10488-007-0141-z.

PubMed Google Scholar

Scheirer MA: The life cycle of an innovation: adoption versus discontinuation of the fluoride mouth rinse program in schools. J Health Soc Behav. 1990, 31 (2): 203-215. 10.2307/2137173.

Seffrin B, Panzano PC, Roth D: What Gets Noticed: How Barrier and Facilitator Perceptions Relate to the Adoption and Implementation of Innovative Mental Health Practices. Community Ment Health J. 2008, 44 (6): 475-10.1007/s10597-008-9151-x.

Rogers EM: Diffusion of Innovations. 2003, New York: The Free Press, A Division of Simon & Schuster, Inc, 1-551.

McHugo GJ: Fidelity to assertive community treatment and client outcomes in the New Hampshire dual disorders study. Psychiatr Serv. 1999, 50 (6): 818-

McGlynn EA: The quality of health care delivered to adults in the United States. N Engl J Med. 2003, 348: 2635-2645. 10.1056/NEJMsa022615.

Jerrell JM, Ridgely MS: The relative impact of treatment program [] robustness' and [] dosage'on client outcomes. Eval Program Plann. 1999, 22 (3): 323-330. 10.1016/S0149-7189(99)00023-3.

Maher L, Gustafson DA: Sustainability Model and Guide: Evans;. 2007

von Krogh G, Roos J: A perspective on knowledge, competence and strategy. Pers Rev. 1995, 24 (3): 56-10.1108/00483489510089650.

Aarons GA, Palinkas LA: Implementation of evidence, evidence-based practice in child welfare: service provider perspective. Adm Policy Ment Health Serv Res. 2007, 34: 411-419. 10.1007/s10488-007-0121-3.

Scheirer MA, Dearing JW: An agenda for research on the sustainability of public health programs. Am J Public Health. 2011, 101 (11): 2059-2067. 10.2105/AJPH.2011.300193.

Racine DP: Reliable effectiveness: a theory on sustaining and replicating worthwhile innovations. Adm Policy Ment Health. 2006, 33 (3): 356-387. 10.1007/s10488-006-0047-1.

Stirman SW: Sustainability: A systematic review of methods and conceptualizations. in Fourth Annual NIH Conference on the Science of Dissemination and Implementation. 2011, Bethesda, MD

Gruen RL: Sustainability science: an integrated approach for health-programme planning. Lancet. 2008, 372 (9649): 1579-10.1016/S0140-6736(08)61659-1.

Aarons G, Hurlburt M, Horwitz S: Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors. Adm Policy Ment Health Ment Health Serv Res. 2011, 38 (1): 4-23. 10.1007/s10488-010-0327-7.

Kirsh SR, Lawrence RH, Aron DC: Tailoring an intervention to the context and system redesign related to the intervention: A case study of implementing shared medical appointments for diabetes. Implement Sci. 2008, 3 (1): 34-10.1186/1748-5908-3-34.

Scheirer MA, Hartling G, Hagerman D: Defining sustainability outcomes of health programs: Illustrations from an on-line survey. Eval Program Plann. 2008, 31 (4): 335-346. 10.1016/j.evalprogplan.2008.08.004.

Scheirer MA, J Dearing: An Agenda for Research on the Sustainability of Public Health Programs. 2011, American Journal of Public Health,

Glasgow RE, Vogt TM, Boles SM: Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999, 89: 1322-1327. 10.2105/AJPH.89.9.1322.

CAS PubMed PubMed Central Google Scholar

Scheirer MA: Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am J Eval. 2005, 23: 320-347.

Kimberly JR, Evanisko MJ: Organizational innovation: The influence of individual, organizational, and contextual factors on hospital adoption of technological and administrative innovations. Acad Manage J. 1981, 24 (4): 689-713.

Kimberly JR, Starbuck WH: Managerial Innovation, in Handbook of Organizational Design. Edited by: Nystrom PC. 1981, New York: Oxford University Press, 84-104.

Durlak J, DuPre E: Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation. Am J Community Psychol. 2008, 41: 327-350. 10.1007/s10464-008-9165-0.

Swerissen H, Crisp BR: The sustainability of health promotion interventions for different levels of social organization. Health Promot Int. 2004, 19 (1): 123-130. 10.1093/heapro/dah113.

Pettigrew A: On studying organizational cultures. Adm Sci Q. 1979, 24: 570-581. 10.2307/2392363.

Fixsen D: Implementation Research: A Synthesis of the Literature. 2005

Greenhalgh T: Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Q. 2004, 82 (4): 581-629. 10.1111/j.0887-378X.2004.00325.x.

Moher D: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009, 6 (7): e1000097-10.1371/journal.pmed.1000097.

Marty D: Factors influencing consumer outcome monitoring in implementation of evidence-based practices: results from the National EBP Implementation Project. Adm Policy Ment Health Ment Health Serv Res. 2008, 35 (3): 204-10.1007/s10488-007-0157-4.

Glisson C: Assessing the organizational social context (OSC) of mental health services: Implications for research and practice. Adm Policy Ment Health Ment Health Serv Res. 2008, 35 (1-2): 98-10.1007/s10488-007-0148-5.

Feldstein AC, Glasgow RE, Smith DH: A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Jt Comm J Qual Patient Saf. 2008, 34 (4): 228-243.

Glaser EM: Durability of Innovations in Human Service Organizations. Sci Commun. 1981, 3 (2): 167-185. 10.1177/107554708100300204.

Goodman RM: Development of level of institutionalization scales for health promotion programs. Health Educ Q. 1993, 20: 161-178. 10.1177/109019819302000208.

Johnson K: Building capacity and sustainable prevention innovations: a sustainability planning model. Eval Program Plann. 2004, 27: 135-149. 10.1016/j.evalprogplan.2004.01.002.

CAS Google Scholar

Mancini JA, Marek LI: Sustaining community-based programs for families: Conceptualization and measurement. Fam Relat. 2004, 53 (4): 339-347. 10.1111/j.0197-6664.2004.00040.x.

Pluye P: Program sustainability: focus on organizational routines. Health Promot Int. 2004, 19 (4): 489-500. 10.1093/heapro/dah411.

Sarriot EG: A methodological approach and framework for sustainability assessment in NGO-implemented primary health care programs. Int J Health Plann Manage. 2004, 19 (1): 23-41. 10.1002/hpm.744.

Silimperi DR: A framework for institutionalizing quality assurance. Int J Qual Health Care. 2002, 14 (suppl 1): 67-073.

Yin RK, Quick SK, Bateman PM, Marks EL: Changing Urban Bureaucracies: How New Practices Become Routinized. 1978, Santa Monica: Rand Corp, 1-155.

Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33 (1): 159-174. 10.2307/2529310.

Bossert TJ: Can they get along without us? Sustainability of donor-supported health projects in Central America and Africa. Soc Sci Med. 1990, 30 (9): 1015-1023. 10.1016/0277-9536(90)90148-L.

Bartholomew L: Planning Health Promotion Programs: An Intervention Mapping Approach. 2006, San Francisco: Jossey-Bass

Botvin GJ: Preventing adolescent drug abuse through a multimodal cognitive-behavioral approach: results of a 3-year study. J Consult Clin Psychol. 1990, 58 (4): 437-

Rohrbach LA, Graham JW, Hansen WB: Diffusion of a school-based substance abuse prevention program: predictors of program implementation. Prev Med. 1993, 22 (2): 237-260. 10.1006/pmed.1993.1020.

Claquin P: Sustainability of EPI: Utopia or Sine Qua Non Condition of Child Survival, in Resources for Child Health Project. 1989, Arlington; REACH

Rabin B: Individual and setting level predictors of the implementation of a skin cancer prevention program: a multilevel analysis. Implement Sci. 2010, 5: 40-53. 10.1186/1748-5908-5-40.

Stetler CB: Improving quality of care through routine, successful implementation of evidence-based practice at the bedside: an organizational case study protocol using the Pettigrew and Whipp model of strategic change. Implement Sci. 2007, 2: 3-15. 10.1186/1748-5908-2-3.

Sibthorpe BM, Glasgow NJ, Wells RW: Emergent themes in tlie sustainability of primary health care innovation. Med J Aust. 2005, 183 (Suppl 10): S77-S80.

Lafond AK: Improving the quality of investment in health: lessons on sustainability. Health Policy Plan. 1995, 10: 63-76.

Ovretveit J: Making temporary quality improvement continuous. 2003, Stockholm: Swedish Association of County Councils

Winett RA, King AC, Altman DG: Health Psychology and Public Health: An Integrative Approach. 1989, Pergamon Press

Pluye P, Potvin L, Denis J-L: Making public health programs last: conceptualizing sustainability. Eval Program Plann. 2004, 27 (2): 121-133. 10.1016/j.evalprogplan.2004.01.001.

Goodman RM, Steckler AB: A model for the institutionalization of health promotion programs. Fam Community Health. 1987, 11: 63-78.

Schoenwald SK: Toward the Effective and Efficient Measurement of Implementation Fidelity. 2011, Administration and Policy in Mental Health and Mental Health Services Research, 38-

Dusenbury L: A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003, 18 (2): 237-256. 10.1093/her/18.2.237.

Henggeler SW: Multisystemic therapy with violent and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. J Consult Clin Psychol. 1997, 65 (5): 821-833.

Woolf SH, Johnson RE: Inattention to the fidelity of health care delivery is costing lives. Am J Public Health. 2007, 97 (10): 1732-1733. author reply 1733

Nilsen P: Towards improved understanding of injury prevention program sustainability. Saf Sci. 2005, 43 (10): 815-10.1016/j.ssci.2005.08.015.

Damschroder LJ: Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009, 4: 50-10.1186/1748-5908-4-50.

Hemmelgarn AL, Glisson C, James LR: Organizational culture and climate: implications for services and interventions research. Clin Psychol Sci Pract. 2006, 13 (1): 73-10.1111/j.1468-2850.2006.00008.x.

VanDeusen Lukas C: Strengthening organizations to implement evidence-based clinical practice. Health Care Manage Rev. 2010, 35 (3): 235-245.

Aarons G, Hurlburt M, Horwitz S: Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors. Administration and Policy in Mental Health and Mental Health Services Research,

Schein E: Organizational Culture and Leadership. 2004, San Francisco: Jossey-Bass

Clarke GN, Kazdin AE: Improving the transition from basic efficacy research to effectiveness studies: Methodological issues and procedures. Methodological issues & strategies in clinical research (3rd ed.). 2003, Washington: American Psychological Association, 569-3

Blasinsky M, Goldman HH, Unutzer J: Project IMPACT: a report on barriers and facilitators to sustainability. Adm Policy Ment Health Ment Health Serv Res. 2006, 33 (6): 718-729. 10.1007/s10488-006-0086-7.

Woolf SH, Johnson RE: The break-even point: when medical advances are less important than improving the fidelity with which they are delivered. Ann Fam Med. 2005, 3 (6): 545-552. 10.1370/afm.406.

Kimberly J, Rye CB: The Morphology of Innovation, in Handbook of Culture, Organization, and Work. Edited by: Bhagat RS, Steers RM. 2009, London: Cambridge University Press, 197-218.

Kimberly JR, de Pouvourville G, D'Aunno T: The Globalization of Managerial Innovation in Health Care. 2008, London: Cambridge University Press

Waltz J: Testing the integrity of a psychotherapy protocol: assessment of adherence and competence. J Consult Clin Psychol. 1993, 61 (4): 620-630.

Jana S: The Sonagachi Project: a sustainable community intervention program. AIDS Educ Prev. 2004, 16 (5): 405-414. 10.1521/aeap.16.5.405.48734.

Plochg T: Collaborating while competing? The sustainability of community-based integrated care initiatives through a health partnership. BMC Health Serv Res. 2006, 20: 6-37.

Blakely CH: The fidelity-adaptation debate: implications for the implementation of public sector social programs. Am J Community Psychol. 1987, 15 (3): 253-268. 10.1007/BF00922697.

Langley GJ: The improvement guide: a practical approach to enhancing organizational performance. 2009, New York: Jossey-Bass

Spencer BA: Models of organization and total quality management: a comparison and critical evaluation. Academy of Management Review. 1994, 19 (3): 446-471.

Weiss CH: Have we learned anything new about the use of evaluation?. Am J Eval. 1998, 19 (1): 21-

Crossan MM, Lane HW, White RE: An organizational learning framework: From intuition to institution. 1999, 24 (3): 522-537. Academy of Management Review

March JG: Exploration and exploitation in organizational learning. 1991, Organization Science, 2 (1): 71-87.

Berman P: The study of macro and micro implementation of social policy. 1978, Santa Monica: Rand Corporation

Hill LG, Maucione K, Hood BK: A focused approach to assessing program fidelity. Prev Sci. 2007, 8: 25-34. 10.1007/s11121-006-0051-4.

Lundgren L: Modifications of evidence-based practices in community-based addiction treatment organizations: A qualitative research study addictive behaviors. Addict Behav. 2011, 36 (6): 630-635. 10.1016/j.addbeh.2011.01.003.

Levitt JT: The flexible application of a manualized treatment for PTSD symptoms and functional impairment related to the 9/11 World Trade Center attack. Behav Res Ther. 2007, 45 (7): 1419-10.1016/j.brat.2007.01.004.

SW S, CF : Moving From the Laboratory to the Real World: Implementation Science and Effectiveness in Practice Settings, in Toward an Integrated Approach to Trauma Focused Therapy: Placing Evidence-Based Interventions in an Expanded Psychological Context. Edited by: R McMackin, et al. American Psychological Association: Washington, D.C,

Perepletchikova F, Treat TA, Kazdin AE: Treatment integrity in psychotherapy research: Analysis of the studies and examination of the associated factors. J Consult Clin Psychol. 2007, 75 (6): 829-841.

Davies B, Tremblay D, Edwards N: Sustaining evidence-based practice systems and measuring the impacts, in Evaluating the Impact of Implementing Evidence-Based Practice. Edited by: Bick D, Ingram ID. 2010, Oxford: Wiley Blackwell

Tyre M, Orlikowski W: Windows of opportunity: temporal patterns in technical adaptation in organizations. Organ Sci. 1994, 5 (1): 98-105. 10.1287/orsc.5.1.98.

Charns MP: Multilevel Interventions: Measurement and Measures. J Natl Cancer Inst.

Glanz K, Bishop DB: The role of behavioral science theory in development and implementation of public health interventions. Annu Rev Public Health. 2010, 31: 399-418. 10.1146/annurev.publhealth.012809.103604.

Edward JR: Testing ecological models: the meaning of validation. Ecol Model. 1996, 90 (3): 229-244. 10.1016/0304-3800(95)00152-2.

Steckler AB, Goodman RM, McLeory KR, Davis S, Koch G: Measuring the diffusion of innovative health promotion programs. Am J Health Promot. 1992, 6 (3): 214-224. 10.4278/0890-1171-6.3.214.

Barab SA, Redman BK, Froman RD: Measurement characteristics of the levels of institutionalization scales: examining reliability and validity. J Nurs Meas. 1998, 6: 19-33.

Ruch-Ross H: Evaluation of community-based health projects: the healthy tomorrows experience. Pediatrics. 2008, 122 (3): e564-e572. 10.1542/peds.2006-3273.

Patton MQ: Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. 2010, New York: The Guilford Press

Abraham AJ, Knudsen HK, Roman PM: A longitudinal examination of alcohol pharmacotherapy adoption in substance use disorder treatment programs: patterns of sustainability and discontinuation. J Stud Alcohol Drugs. 2011, 72 (4): 669-677.

Ahluwalia IB, Robinson D, Vallely L, Gieseker KE, Kabakama A: Sustainability of community-capacity to promote safer motherhood in northwestern Tanzania: what remains?. Glob Health Promot. 2010, 17 (1): 39-49. 10.1177/1757975909356627.

Aitaoto N, Tsark J, Braun KL: Sustainability of the pacific diabetes today coalitions. Prev Chronic Dis. 2009, 6 (4): A130-A138.

Amazigo U, Okeibunor J, Matovu V, Zouré H, Bump J, Seketeli A: Performance of predictors: evaluating sustainability in community-directed treatment projects of the African programme for onchocerciasis control. Soc Sci Med. 2007, 64 (10): 2070-2082. 10.1016/j.socscimed.2007.01.018.

August GJ, Bloomquist ML, Lee SS, Realmuto GM, Hektner JM: Can evidence-based prevention programs be sustained in community practice settings? The early risers' advanced-stage effectiveness trial. Prev Sci. 2006, 7 (2): 151-165. 10.1007/s11121-005-0024-z.

Austin G, Bell T, Caperchione C, Mummery WK: Translating research to practice: using the RE-AIM framework to examine an evidence-based physical activity intervention in primary school settings. Health Promot Pract. 2011, 12 (6): 932-941. 10.1177/1524839910366101.

Babl FE, Krieser D, Belousoff J, Theophilos T: Evaluation of a paediatric procedural sedation training and credentialing programme: sustainability of change. Emerg Med J. 2010, 27 (8): 577-581. 10.1136/emj.2009.077024.

Baer JS, Ball SA, Campbell BK, Miele GM, Schoener EP, Tracy K: Training and fidelity monitoring of behavioral interventions in multi-site addictions research. Drug Alcohol Depend. 2007, 87 (2): 107-118. 10.1016/j.drugalcdep.2006.08.028.

Bailie RS, Robinson G, Kondalsamy-Chennakesavan SN, Halpin S, Wang Z: Investigating the sustainability of outcomes in a chronic disease treatment programme. Soc Sci Med. 2006, 63 (6): 1661-1670. 10.1016/j.socscimed.2006.04.010.

Barnett LM, Van Beurden E, Eakin EG, Beard J, Dietrich U, Newman B: Program sustainability of a community-based intervention to prevent falls among older Australians. Health Promot Int. 2004, 19 (3): 281-288. 10.1093/heapro/dah302.

Baum F, Jolley G, Hicks R, Saint K, Parker S: What makes for sustainable Healthy Cities initiatives?-a review of the evidence from Noarlunga, Australia after 18 years. Health Promot Int. 2006, 21 (4): 259-265. 10.1093/heapro/dal023.

Beery WL, Senter S, Cheadle A, Greenwald HP, Pearson D, Brousseau R, Nelson GD: Evaluating the legacy of community health initiatives: a conceptual framework and example from the California Wellness Foundation's health improvement initiative. Am J Evaluation. 2005, 26: 150-165. 10.1177/1098214005275627.

Bere E, Veierod MB, Bjelland M, Klepp KI: Free school fruit-sustained effect 1 year later. Health Educ Res. 2006, 21 (2): 268-275.

Bisset S, Potvin L: Expanding our conceptualization of program implementation: lessons from the genealogy of a school-based nutrition program. Health Educ Res. 2006, 22 (5): 737-746. 10.1093/her/cyl154.

Blasinsky M, Goldman HH, Unutzer J: Project IMPACT: a report on barriers and facilitators to sustainability. Adm Policy Ment Health. 2006, 33 (6): 718-729. 10.1007/s10488-006-0086-7.

Bowman C, Sobo E, Asch S, Gifford A: The H. I. V. Hepatitis Quality Enhancement Research Initiative: Measuring persistence of implementation: QUERI series. Implement Sci. 2008, 3 (1): 21-10.1186/1748-5908-3-21.

Bratcht NF, Finnegan JR, Rissel C, Weisbrod R, Gleason J, Corbett J, Mortenson S: Community ownership and program coninuation following a health demonstration program. Health Educ Res. 1994, 9: 243-255. 10.1093/her/9.2.243.

Brand C, Landgren F, Hutchinson A, Jones C, MacGregor L, Campbell D: Clinical practice guidelines: barriers to durability after effective early implementation. Intern Med J. 2005, 35 (3): 162-169. 10.1111/j.1445-5994.2004.00763.x.

Bunik M, Federico MJ, Beaty B, Rannie M, Olin JT, Kempe A: Quality improvement for asthma care within a hospital-based teaching clinic. Acad Pediatr. 2011, 11 (1): 58-65. 10.1016/j.acap.2010.10.004.

Cherry RA, West CE, Hamilton MC, Rafferty CM, Hollenbeak CS, Caputo GM: Reduction of central venous catheter associated blood stream infections following implementation of a resident oversight and credentialing policy. Patient Saf Surg. 2011, 5: 15-10.1186/1754-9493-5-15.

DeWein M, Miller L: The effects of a classroom-based intervention on aggression-related injuries. Child and Youth Care Forum. 2009, 38 (4): 201-218. 10.1007/s10566-009-9076-8.

Dückers MLA, Wagner C, Vos L, Groenewegen PP: Understanding organisational development, sustainability, and diffusion of innovations within hospitals participating in a multilevel quality collaborative. Implement Sci. 2011, 6 (1): 18-10.1186/1748-5908-6-18.

Ebert-May D, Derting TL, Hodder J, Momsen JL, Long TM, Jardeleza SE: What we say is not what we do: effective evaluation of faculty professional development programs. Bioscience. 2011, 61 (7): 550-558. 10.1525/bio.2011.61.7.9.

Edvarsson K, Garvare R, Ivarsson A, Eurenius E, Morgen I, Nystrom M: Sustainable practice change: professionals' experiences with a multisectoral child health promotion programme in Sweden. BMC Health Serv Res. 2011, 11: 61-10.1186/1472-6963-11-61.

Eliason RN: Towards sustainability in village health care in rural Cameroon. Health Promot Int. 1999, 14: 301-306. 10.1093/heapro/14.4.301.

Ellingson K, Muder RR, Jain R, Kleinbaum D, Feng PJ, Cunningham C, Squier C, Lloyd J, Edwards J, Gebski V, Jernigan J: Sustained reduction in the clinical incidence of methicillin-resistant Staphylococcus aureu colonization or infection associated with a multifaceted infection control intervention. Infect Control Hosp Epidemiol. 2011, 32 (1): 1-8. 10.1086/657665.

Epstein JN, Langberg JM, Lichtenstein PK, Kolb RC, Stark LJ: Sustained improvement in pediatricians' ADHD practice behaviors in the context of a community-based quality improvement initiative. Child Health Care. 2010, 39 (4): 296-311. 10.1080/02739615.2010.515931.

Escoffery C, Glanz K, Hall D, Elliott T: A multi-method process evaluation for a skin cancer prevention diffusion trial. Eval Health Prof. 2009, 32 (2): 184-203. 10.1177/0163278709333154.

Evashwick C, Ory M: Organizational characteristics of successful innovative health care programs sustained over time. Fam Community Health. 2003, 26 (3): 177-193.

Feinberg ME, Bontempo DE, Greenberg MT: Predictors and level of sustainability of community prevention coalitions. Am J Prev Med. 2008, 34 (6): 495-501. 10.1016/j.amepre.2008.01.030.

Fonck K, Claeys P, Bashir F, Bwayo J, Fransen L, Temmerman M: Syphilis control during pregnancy: effectiveness and sustainability of a decentralized program. Am J Public Health. 2001, 91 (5): 705-707.

Glaser EM, Backer TE: Durability of innovations: how goal attainment scaling programs fare over time. Community Ment Health J. 1980, 16 (2): 130-143. 10.1007/BF00778585.

Glisson C, Schoenwald SK, Kelleher K, Landsverk J, Hoagwood KE, Mayberg S, Green P: Therapist turnover and new program sustainability in mental health clinics as a function of organizational culture, climate, and service structure. Adm Policy Ment Health. 2008, 35 (1-2): 124-133. 10.1007/s10488-007-0152-9.

Godley SH, Garner BR, Smith JE, Meyers RJ, Godley MD: A large scale dissemination and implementation model for evidence based treatment and continuing care. Clin Psychol Sci Pract. 2011, 18 (1): 67-83. 10.1111/j.1468-2850.2011.01236.x.

Goetz MB, Hoang T, Henry SR, Knapp H, Anaya HD, Gifford AL, Asch SM: Evaluation of the sustainability of an intervention to increase HIV testing. J Gen Intern Med. 2009, 24 (12): 1275-1280. 10.1007/s11606-009-1120-8.

Goodson P, Murphy Smith M, Evans A, Meyer B, Gottlieb NH: Maintaining prevention in practice: survival of PPIP in primary care settings. Am J Prev Med. 2001, 20 (3): 184-189. 10.1016/S0749-3797(00)00310-X.

Greenwood CR, Tapia Y, Abbott M, Walton C: A building-based case study of evidence-based literacy practices: implementation, reading behavior, and growth in reading fluency, K-4. J Spec Educ. 2003, 37 (2): 95-110. 10.1177/00224669030370020401.

Gruen RL, Elliott JH, Nolan ML, Lawton PD, Parkhill A, McLaren CJ, Lavis JN: Sustainability science: an integrated approach for health-programme planning. Lancet. 2008, 372 (9649): 1579-1589. 10.1016/S0140-6736(08)61659-1.

Gurtler RE, Kitron U, Cecere MC, Segura EL, Cohen JE: Sustainable vector control and management of Chagas disease in the Gran Chaco, Argentina. Proc Natl Acad Sci USA. 2007, 104 (41): 16194-16199. 10.1073/pnas.0700863104.

Harris DL, Henry RC, Bland CJ, Starnaman SM, Voytek KL: Lessons learned from implementing multidisciplinary health professions educational models in community settings. J Interprof Care. 2003, 17 (1): 7-20. 10.1080/1356182021000044111.

Helfrich CD, Damschroder LJ, Hagedorn HJ, Daggett GS, Sahay A, Ritchie M, Damush T, Guihan M, Ullrich PM, Stetler CB: A critical synthesis of literature on the promoting action on research implementation in health services (PARIHS) framework. Implement Sci. 2010, 5: 82-10.1186/1748-5908-5-82.

Henggeler SW, Sheidow AJ, Cunningham PB, Donohue BC, Ford JD: Promoting the implementation of an evidence-based intervention for adolescent marijuana abuse in community settings: testing the use of intensive quality assurance. J Clin Child Adolesc Psychol. 2008, 37 (3): 682-689. 10.1080/15374410802148087.

Higuchi KS, Davies BL, Edwards N, Ploeg J, Virani T: Implementation of clinical guidelines for adults with asthma and diabetes: a three-year follow-up evaluation of nursing care. J Clin Nurs. 2011, 20 (9-10): 1329-1338. 10.1111/j.1365-2702.2010.03590.x.

Hii JL, Chee KC, Vun YS, Awang J, Chin KH, Kan SK: Sustainability of a successful malaria surveillance and treatment program in a Runggus community in Sabah, east Malaysia. Southeast Asian J Trop Med Public Health. 1996, 27 (3): 512-521.

Hoelscher DM, Feldman HA, Johnson CC, Lytle LA, Osganian SK, Parcel GS, Kelder SH, Stone EJ, Nader PR: School-based health education programs can be maintained over time: results from the CATCH Institutionalization study. Prev Med. 2004, 38 (5): 594-606. 10.1016/j.ypmed.2003.11.017.

Hoffman KA, Ford II, James H, Choi D, Gustafson DH, McCarty D: Replication and sustainability of improved access and retention within the Network for the Improvement of Addiction Treatment. Drug Alcohol Depend. 2008, 98 (1-2): 63-69. 10.1016/j.drugalcdep.2008.04.016.

Hogg W, Baskerville N, Nykiforuk C, Mallen D: Improved preventive care in family practices with outreach facilitation: understanding success and failure. J Health Serv Res Policy. 2002, 7 (4): 195-201. 10.1258/135581902320432714.

Hoque BA, Juncker T, Sack RB, Ali M, Aziz KM: Sustainability of a water, sanitation and hygiene education project in rural Bangladesh: a 5-year follow-up. Bulletin WHO. 1996, 74 (4): 431-437.

Jansen M, Harting J, Ebben N, Kroon B, Stappers J, VanEngelshoven E: deVries, N: The concept of sustainability and the use of outcome indicators: a case study to continue a successful health counselling intervention. Fam Pract. 2008, 25: i32-i37. 10.1093/fampra/cmn066.

Johnson C, Fargo J, Kahle JB: The cumulative and residual impact of a systemic reform program on teacher change and student learning of science. Sch Sci Math. 2010, 110 (3): 144-159. 10.1111/j.1949-8594.2010.00017.x.

Kalafat J, Ryerson DM: The implementation and institutionalization of a school-based youth suicide prevention program. J Prim Prev. 1999, 19 (3): 157-175. 10.1023/A:1022658826641.

Kay BH, Tuyet Hanh TT, Le NH, Quy TM, Nam VS, Hang PV, Yen NT, Hill PS, Vos T, Ryan PA: Sustainability and cost of a community-based strategy against Aedes aegypti in northern and central Vietnam. AmJTrop Med Hyg. 2010, 82 (5): 822-

Kellie SM, Timmins A, Brown C: A statewide collaborative to reduce methicillin-resistant Staphylococcus aureu bacteremias in New Mexico. Jt Comm J Qual Patient Saf. 2011, 37: 154-162.

Kennedy MT, Fiss PC: Institutionalization, framing, and diffusion: the logic of TQM adoption and implementation decisions among U.S. hospitals. Acad Manage J. 2009, 52 (5): 897-918.

Klingner J, Vaughn S: Tejero Hughes M, Arguelles ME: Sustaining research-based practices in reading: a 3-year follow-up. Rem Spec Ed. 1999, 20 (5): 263-275. 10.1177/074193259902000502.

Knapp H, Anaya HD, Goetz MB: Attributes of an independently self-sustaining implementation: nurse-administered HIV rapid testing in VA primary care. Qual Manag Health Care. 2010, 19 (4): 292-297.

Knippenberg R, Soucat A, Oyegbite K, Sene M, Bround D, Pangu K, Hopwood I, Grandcourt R, Tinguiri KL, Fall I, Ammassari S, Alihonou E: Sustainability of primary health care including expanded program of immunizations in Bamako Initiative programs in West Africa: an assessment of 5 years' field experience in Benin and Guinea. Int J Health Plann Manage. 1997, 12: S9-S28. 10.1002/(SICI)1099-1751(199706)12:1+<S9::AID-HPM471>3.3.CO;2-U.

Knudsen HK, Studts JL: Availability of nicotine replacement therapy in substance use disorder treatment: longitudinal patterns of adoption, sustainability, and discontinuation. Drug Alcohol Depend. 2011, 118 (2-3): 244-250. 10.1016/j.drugalcdep.2011.03.028.

Kolko DJ, Iselin A-MR, Gully KJ: Evaluation of the sustainability and clinical outcome of Alternatives for Families: A Cognitive-Behavioral Therapy (AF-CBT) in a child protection center. Child Abuse Negl. 2011, 35 (2): 105-116. 10.1016/j.chiabu.2010.09.004.

LaPelle NR, Zapka J, Ockene JK: Sustainability of public health programs: the example of tobacco treatment services in Massachusetts. Am J Public Health. 2006, 96: 1363-1369. 10.2105/AJPH.2005.067124.

Lee AJ, Bonson APV, Yarmirr D, O'Dea K, Mathews JD: Sustainability of a successful health and nutrition program in a remote Aboriginal community. Med J Aust. 1995, 162: 632-635.

Lee PW, Dietrich AJ, Oxman TE, Williams JWJ, Barry SL: Sustainable impact of a primary care depression intervention. J Am Board Fam Med. 2007, 20 (5): 427-433. 10.3122/jabfm.2007.05.070045.

Lichtenstein E, Thompson B, Nettekoven L, Corbett K: Durability of tobacco control activities in 11 North American communities: life after the community intervention trial for smoking cessation (COMMIT). Health Educ Res. 1996, 11 (4): 527-534. 10.1093/her/11.4.527.

Lieber J, Butera G, Hanson M, Palmer S, Horn E, Czaja C: Sustainability of a preschool curriculum: what encourages continued use among teachers?. NHSA Dialog. 2010, 13 (4): 225-242.

Lodl K, Stevens G: Coalition sustainability: long-term successes & lessons learned. J Ext. 2002, 40 (1): 1-8.

Loman SL, Rodriguez BJ, Horner RH: Sustainability of a targeted intervention package: first step to success in Oregon. J Emot Behav Disord. 2010, 18 (3): 178-191. 10.1177/1063426610362899.

Lyon AR, Stirman SW, Kerns SE, Bruns EJ: Developing the mental health workforce: review and application of training approaches from multiple disciplines. Adm Policy Ment Health. 2011, 38 (4): 238-253. 10.1007/s10488-010-0331-y.

Martin GW, Herie MA, Turner BJ, Cunningham JA: A social marketing model for disseminating research-based treatments to addictions treatment providers. Addiction. 1998, 93 (11): 1703-1715. 10.1046/j.1360-0443.1998.931117038.x.

Massatti RR, Sweeney HA, Panzano PC, Roth D: The de-adoption of innovative mental health practices (IMHP): why organizations choose not to sustain an IMHP. Adm Policy Ment Health. 2008, 35 (1-2): 50-65. 10.1007/s10488-007-0141-z.

Maticka-Tyndale E, Wildish J, Gichuru M: Thirty-month quasi-experimental evaluation follow-up of a national primary school HIV intervention in Kenya. Sex Educ: Sex Soc Learn. 2010, 10 (2): 113-130.

Mayer J, Mooney B, Gundlapalli A, Harbarth S, Stoddard GJ, Rubin MA, Eutropius L, Brinton B, Samore MH: Dissemination and sustainability of a hospital-wide hand hygiene program emphasizing positive reinforcement. Infect Control Hosp Epidemiol. 2011, 32 (1): 59-66. 10.1086/657666.

McCormick LK, Steckler AB, McLeroy KR: Diffusion of innovations in schools: a study of adoption and implementation of school-based tobacco prevention curricula. Am J Health Promot. 1995, 9 (3): 210-219. 10.4278/0890-1171-9.3.210.

McDermott R, Tulip F, Schmidt B, Sinha A: Sustaining better diabetes care in remote indigenous Australian communities. Qual Saf Health Care. 2004, 13 (4): 295-298. 10.1136/qhc.13.4.295.

McHugo GJ, Drake RE, Whitley R, Bond GR, Campbell K, Rapp CA, Goldman HH, Lutz WJ, Finnerty MT: Fidelity outcomes in the national implementing evidence-based practices project. Psychiatr Serv. 2007, 58 (10): 1279-1284. 10.1176/appi.ps.58.10.1279.

Miller WR, Yahne CE, Moyers TB, Martinez J, Pirritano M: A randomized trial of methods to help clinicians learn motivational interviewing. J Consult Clin Psychol. 2004, 72 (6): 1050-1062.

Morgenstern LB, Bartholomew LK, Grotta JC, Staub L, King M, Chan W: Sustained benefit of a community and professional intervention to increase acute stroke therapy. Arch Intern Med. 2003, 163 (18): 2198-2202. 10.1001/archinte.163.18.2198.

Nease DE, Nutting PA, Graham DG, Dickinson WP, Gallagher KM, Jeffcott-Pera M: Sustainability of depression care improvements: success of a practice change improvement collaborative. J Am Board Fam Med. 2010, 23 (5): 598-605. 10.3122/jabfm.2010.05.090212.

Nilsen P, Timpka T, Nordenfelt L, Lindqvist K: Towards improved understanding of injury prevention program sustainability. Saf Sci. 2005, 43 (10): 815-10.1016/j.ssci.2005.08.015.

O'Loughlin J, Renaud L, Richard L, Gomez LS, Paradis G: Correlates of the sustainability of community-based heart health promotion interventions. Prev Med. 1998, 27 (5): 702-712. 10.1006/pmed.1998.0348.

Osganian SK, Hoelscher DM, Zive M, Mitchell PD, Snyder P, Webber LS: Maintenance of effects of the Eat Smart School Food Service Program: results from the CATCH-ON study. Health Educ Behav. 2003, 30 (4): 418-433. 10.1177/1090198103253509.

Paine-Andrews A, Fisher JL, Campuzano MK, Fawcett SB, Berkley-Patton J: Promoting sustainability of community health initiatives: an empirical case study. Health Promot Pract. 2000, 1: 248-258. 10.1177/152483990000100311.

Perlstein PH, Kotagal UR, Schoettker PJ, HD , Farrell MK, Gerhardt WE, Alfaro MP: Sustaining the implementation of an evidence-based guideline for bronchiolitis. Arch Pediatr Adolesc Med. 2000, 154 (10): 1001-1007.

Plochg T, Delnoij DM, Hoogedoorn NP, Klazinga NS: Collaborating while competing? The sustainability of community-based integrated care initiatives through a health partnership. BMC Health Serv Res. 2006, 20: 6-37.

Pronovost PJ, Goeschel CA, Colantuoni E, Watson S, Lubomski LH, Berenholtz SM, Thompson DA, Sinopoli DJ, Cosgrove S, Sexton JB, Marsteller JA, Hyzy RC, Welsh R, Posa P, Schumacher K, Needham D: Sustaining reductions in catheter related bloodstream infections in Michigan intensive care units: observational study. BMJ. 2010, 340: c309-c313. 10.1136/bmj.c309.

Rabin B, Nehl E, Elliott T, Deshpande A, Brownson R, Glanz K: Individual and setting level predictors of the implementation of a skin cancer prevention program: a multilevel analysis. Implement Sci. 2010, 5: 40-53. 10.1186/1748-5908-5-40.

Rapoport A: Sustainability of teachers' international experiences: conditions for institutionalization of international program outcomes. Educ Res Int. 2011, 2011: 1-9.

Rau R, Rumpeltin C, Hoop R, Pfeiffer H, Drees J, Paas B, Schmitz-Buhl G, Geraedts M: Five years "Healthy Lower Rhine...Against Stroke": implementation of a regional, intersectoral and sustainable public health program. J Public Health. 2010, 18 (1): 29-34. 10.1007/s10389-009-0278-1.

Rog D, Boback N, Barton-Villagrana H, Marrone-Bennett P, Cardwell J, Hawdon J, Diaz J, Jenkins P, Kridler J, Reischl T: Sustaining collaboratives: a cross-site analysis of the National Funding Collaborative on Violence Prevention. Eval Program Plann. 2004, 27 (3): 249-261. 10.1016/j.evalprogplan.2003.10.001.

Rollins A, Salyers M, Tsai J, Lydick J: Staff turnover in statewide implementation of ACT: relationship with ACT fidelity and other team characteristics. Adm Policy Ment Health. 2009, 37 (5): 417-426.

Rosenberg A, Hartwig K, Merson M: Government-NGO collaboration and sustainability of orphans and vulnerable children projects in southern Africa. Eval Program Plann. 2008, 31 (1): 51-60. 10.1016/j.evalprogplan.2007.08.005.

Rubin FH, Neal K, Fenlon K, Hassan S, Inouye SK: Sustainability and scalability of the hospital elder life program at a community hospital. J Am Geriatr Soc. 2011, 59 (2): 359-365. 10.1111/j.1532-5415.2010.03243.x.

Ruch-Ross H, Keller D, Miller N, Bassewitz J, Melinkovich P: Evaluation of community-based health projects: the healthy tomorrows experience. Pediatrics. 2008, 122 (3): e564-e572. 10.1542/peds.2006-3273.

Sadof MD, Boschert KA, Brandt SJ, Motyl AP: An analysis of predictors of sustainability efforts at the Inner-City Asthma Intervention sites: after the funding is gone. Ann Allergy Asthma Immunol. 2006, 97 (S1): S31-S35.

Sanci L, Coffey C, Patton G, Bowes G: Sustainability of change with quality general practitioner education in adolescent health: a 5-year follow-up. Med Educ. 2005, 39: 557-560. 10.1111/j.1365-2929.2005.02172.x.

Sanci LA, Coffey CM, Veit FC, Carr-Gregg M, Patton GC, Day N, Bowes G: Evaluation of the effectiveness of an educational intervention for general practitioners in adolescent health care: randomised controlled trial. BMJ. 2000, 320 (7229): 224-230. 10.1136/bmj.320.7229.224.

Schetzina KE, Dalton WT, Pfortmiller DT, Robinson HF, Lowe EF, Stern HP: The Winning With Wellness pilot project: rural Appalachian elementary student physical activity and eating behaviors and program implementation 4 years later. Fam Community Health. 2011, 34 (2): 154-162.

Schoenwald SK, Carter RE, Chapman JE, Sheidow AJ: Therapist adherence and organizational effects on change in youth behavior problems one year after multisystemic therapy. Adm Policy Ment Health. 2008, 35 (5): 379-394. 10.1007/s10488-008-0181-z.

Sebotsa MLD, Dannhauser A, Jooste PL, Joubert G: Assessment of the sustainability of the iodine-deficiency disorders control program in Lesotho. Food and Nutrition Bulletin. 2007, 28 (3): 337-347.

Simonsen B, Eber L, Black AC, Sugai G, Lewandowski H, Sims B, Myers D: Illinois statewide positive behavioral interventions and supports: evolution and impact on student outcomes across years. J Posit Behav Interv. 2012, 14: 5-16. 10.1177/1098300711412601.

Stange KC, Goodwin MA, Zyzanski SJ, Dietrich AJ: Sustainability of a practice-individualized preventive service delivery intervention. Am J Prev Med. 2003, 25 (4): 296-300. 10.1016/S0749-3797(03)00219-8.

Steadman HJ, Cocozza JJ, Dennis DL, Lassiter MG, Randolph FL, Goldman H, Blasinsky M: Successful program maintenance when federal demonstration dollars stop: the ACCESS program for homeless mentally ill persons. Adm Policy Ment Health. 2002, 29 (6): 481-493. 10.1023/A:1020776310331.

Stetler CB, Ritchie JA, Rycroft-Malone J, Schultz AA, Charns MP: Institutionalizing evidence-based practice: an organizational case study using a model of strategic change. Implement Sci. 2009, 4: 78-10.1186/1748-5908-4-78.

Stevens B, Peikes D: When the funding stops: do grantees of the Local Initiative Funding Partners Program sustain themselves?. Eval Program Plann. 2006, 29 (2): 153-161. 10.1016/j.evalprogplan.2006.01.009.

Stroul BA, Manteuffel BA: The sustainability of systems of care for children's mental health: lessons learned. J Behav Health Serv Res. 2007, 34 (3): 237-259. 10.1007/s11414-007-9065-3.

Swain K, Whitley R, McHugo GJ, Drake RE: The sustainability of evidence-based practices in routine mental health agencies. Community Ment Health J. 2009, 46 (2): 119-129.

Thompson B, Lichtenstein E, Corbett K, Nettekoven L, Feng Z: Durability of tobacco control efforts in the 22 Community Intervention Trial for Smoking Cessation (COMMIT) communities 2 years after the end of intervention. Health Educ Res. 2000, 15 (3): 353-366. 10.1093/her/15.3.353.

Thorsen AV, Lassen AD, Tetens I, Hels O, Mikkelsen BE: Long-term sustainability of a worksite canteen intervention of serving more fruit and vegetables. Public Health Nutr. 2010, 13 (10): 1647-1652. 10.1017/S1368980010001242.

Tibbits M, Bumbarger B, Kyler S, Perkins D: Sustaining evidence-based interventions under real-world conditions: results from a large-scale diffusion project. Prev Sci. 2010, 11 (3): 252-262. 10.1007/s11121-010-0170-9.

Toledo Romani ME, Vanlerberghe V, Perez D, Lefevre P, Ceballos E, Bandera D, Baly Gil A, Van der Stuyft P: Achieving sustainability of community-based dengue control in Santiago de Cuba. Soc Sci Med. 2007, 64 (4): 976-10.1016/j.socscimed.2006.10.033.

Ullrich S, McCutcheon H, Parker B: Reclaiming time for nursing practice in nutritional care: outcomes of implementing Protected Mealtimes in a residential aged care setting. J Clin Nurs. 2010, 20 (9-10): 1339-1348.

Visrutaratna S, Lindan CP, Sirhorachai A, Mandel JS: "Superstar" and"model brothel": developing and evaluating a condom promotion program for sex establishments in Chiang Mai, Thailand. AIDS. 1995, 9 (1): S69-S75.

Wallin L, Bostrom A, Wikblad K, Ewald U: Sustainability in changing clinical practice promotes evidence-based nursing care. J Adv Nurs. 2003, 41 (5): 509-518. 10.1046/j.1365-2648.2003.02574.x.

Westrick SC, Breland ML: Sustainability of pharmacy-based innovations: the case of in-house immunization services. J Am Pharm Assoc. 2009, 49 (4): 500-508. 10.1331/JAPhA.2009.08055.

Whitford DL, Roberts SH, Griffin S: Sustainability and effectiveness of comprehensive diabetes care to a district population. Diabet Med. 2004, 21 (11): 1221-1228. 10.1111/j.1464-5491.2004.01324.x.

Woltmann EM, Whitley R, McHugo GJ, Brunette M, Torrey WC, Coots L, Lynde D, Drake RE: The role of staff turnover in the implementation of evidence-based practices in mental health care. Psychiatr Serv. 2008, 59 (7): 732-737. 10.1176/appi.ps.59.7.732.

Wong LC, Amega B, Barker R, Connors C, Dulla ME, Ninnal A, Cumaiyi MM, Kolumboort L, Currie BJ: Factors supporting sustainability of a community-based scabies control program. Australas J Dermatol. 2002, 43 (4): 274-277. 10.1046/j.1440-0960.2002.00626.x.

Wong ML, Chan KW, Koh D: A sustainable behavioral intervention to increase condom use and reduce gonorrhea among sex workers in Singapore: 2-year follow-up. Prev Med. 1998, 27: 891-900. 10.1006/pmed.1998.0398.

Wright C, Catty J, Watt H, Burns T: A systematic review of home treatment services: classification and sustainability. Soc Psychiatry Psychiatr Epidemiol. 2004, 39 (10): 789-796.

Wright DB: Care in the country: a historical case study of long-term sustainability in 4 rural health centers. Am J Public Health. 2009, 99 (9): 1612-1618. 10.2105/AJPH.2008.146050.

Xian Y, Pan W, Peterson ED, Heidenreich PA, Cannon CP, Hernandez AF, Friedman B, Holloway RG, Fonarow GC: Are quality improvements associated with the Get With the Guidelines-Coronary Artery Disease (GWTG-CAD) program sustained over time?: A longitudinal comparison of GWTG-CAD hospitals versus non-GWTG-CAD hospitals. Am Heart J. 2010, 159 (2): 207-214. 10.1016/j.ahj.2009.11.002.

Download references

Acknowledgements

We wish to thank the implementation researchers who suggested additional studies for inclusion in our review, our reviewers, and Dr. Sarah Beehler for their comments on previous drafts.

The preparation of this article was supported through funding from the National Institute of Mental Health (R00 MH 01800 [Dr. Stirman] and T32 MH 019836 [Dr. Castro; PI: Keane]); through the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis, which is funded through an award from the National Institute of Mental Health (R25 MH080916-01A2) and the Department of Veterans Affairs, Health Services Research & Development Service, Quality Enhancement Research Initiative (QUERI). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health, the National Institutes of Health, or the Department of Veterans Affairs.

Dr. Castro is now affiliated with the VA Maryland Healthcare System

Author information

Authors and affiliations.

Women's Health Sciences Division, National Center for PTSD, Boston, MA, USA

Shannon Wiltsey Stirman, Natasha Cook, Amber Calloway & Frank Castro

VA Boston Healthcare System, Boston, MA, USA

Shannon Wiltsey Stirman, Natasha Cook, Amber Calloway, Frank Castro & Martin Charns

Department of Psychiatry, Boston University, Boston, MA, USA

Shannon Wiltsey Stirman, Amber Calloway & Frank Castro

Department of Healthcare Management, The Wharton School of the University of Pennsylvania, Philadelphia, PA, USA

John Kimberly

VA Center for Organization, Leadership, and Management Research, Boston, MA, USA

Martin Charns

Department of Health Policy and Management, Boston University School of Public Health, Boston, MA, USA

You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shannon Wiltsey Stirman .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors' contributions

SWS conceptualized the study, contributed to the data collection and coding, and was the predominant contributor to this article. NC and AC assisted with the coding of articles and compilation and interpretation of results. MC, JK, and FC made significant contributions to the conceptual framework and the interpretation of results. All authors read and modified drafts and approved the final manuscript.

Electronic supplementary material

13012_2011_461_moesm1_esm.doc.

Additional file 1: Articles Included in the Review [ 37 , 54 , 98 – 220 ]. (DOC 88 KB)

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2, rights and permissions.

Open Access This article is published under license to BioMed Central Ltd. This is an Open Access article is distributed under the terms of the Creative Commons Attribution License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Wiltsey Stirman, S., Kimberly, J., Cook, N. et al. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implementation Sci 7 , 17 (2012). https://doi.org/10.1186/1748-5908-7-17

Download citation

Received : 22 April 2011

Accepted : 14 March 2012

Published : 14 March 2012

DOI : https://doi.org/10.1186/1748-5908-7-17

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Innovation Characteristic
Implementation Effort
Initial Implementation
Sustainability Outcome
Full Text Version

Implementation Science

ISSN: 1748-5908

Submission enquiries: Access here and click Contact Us
General enquiries: [email protected]

Help & FAQ

Rural tourism: A systematic literature review on definitions and challenges

Research output : Journal article publication › Journal article › Academic research › peer-review

The definition of rural tourism remains unclear and only a few studies have mapped the current state of knowledge in this field. Through a systematic quantitative literature review, this study extends the previous literature by investigating rural tourism definitions and challenges faced within developed and developing contexts. The analysis of definitions reveals four key aspects of rural tourism that include location, sustainable development, community-based features, and experiences. While rural tourism in both developed and developing contexts emphasised location as a main defining characteristic, sustainable development and community-based aspects appear prominently in the literature related to developing countries, and the experience dimension appears more frequently in the literature related to developed countries. The results suggest that rural destinations face internal and external challenges. The greatest challenges for developed and developing contexts arise from issues related to internal resources, although external challenges were found to be greater in developed contexts. The mapping of the current state of knowledge suggests several directions for future research in this domain, and response to the pandemic.

Definitions and challenges
Developed countries
Developing countries
Rural tourism
Systematic literature review

ASJC Scopus subject areas

Tourism, Leisure and Hospitality Management

More information

10.1016/j.jhtm.2021.03.001

Fingerprint

Rural Tourism Keyphrases 100%
Systematic Literature Review Keyphrases 100%
Sustainable Development Social Sciences 100%
Community-Based Social Sciences 100%
Systematic Review Social Sciences 100%
Current-State Computer Science 100%
Developing Context Keyphrases 60%
Developed Countries Social Sciences 50%

T1 - Rural tourism

T2 - A systematic literature review on definitions and challenges

AU - Rosalina, Putu Devi

AU - Dupre, Karine

AU - Wang, Ying

N1 - Funding Information: The first author would like to acknowledge Directorate General of Higher Education - Indonesian Ministry of Education and Culture; Institut Pariwisata dan Bisnis Internasional in Bali for their extensive support to sponsor this doctoral study. Also, the gratitude extends to Griffith Institute for Tourism (GIFT) for supporting the development of the paper. Publisher Copyright: © 2021 The Authors

PY - 2021/6

Y1 - 2021/6

N2 - The definition of rural tourism remains unclear and only a few studies have mapped the current state of knowledge in this field. Through a systematic quantitative literature review, this study extends the previous literature by investigating rural tourism definitions and challenges faced within developed and developing contexts. The analysis of definitions reveals four key aspects of rural tourism that include location, sustainable development, community-based features, and experiences. While rural tourism in both developed and developing contexts emphasised location as a main defining characteristic, sustainable development and community-based aspects appear prominently in the literature related to developing countries, and the experience dimension appears more frequently in the literature related to developed countries. The results suggest that rural destinations face internal and external challenges. The greatest challenges for developed and developing contexts arise from issues related to internal resources, although external challenges were found to be greater in developed contexts. The mapping of the current state of knowledge suggests several directions for future research in this domain, and response to the pandemic.

AB - The definition of rural tourism remains unclear and only a few studies have mapped the current state of knowledge in this field. Through a systematic quantitative literature review, this study extends the previous literature by investigating rural tourism definitions and challenges faced within developed and developing contexts. The analysis of definitions reveals four key aspects of rural tourism that include location, sustainable development, community-based features, and experiences. While rural tourism in both developed and developing contexts emphasised location as a main defining characteristic, sustainable development and community-based aspects appear prominently in the literature related to developing countries, and the experience dimension appears more frequently in the literature related to developed countries. The results suggest that rural destinations face internal and external challenges. The greatest challenges for developed and developing contexts arise from issues related to internal resources, although external challenges were found to be greater in developed contexts. The mapping of the current state of knowledge suggests several directions for future research in this domain, and response to the pandemic.

KW - Definitions and challenges

KW - Developed countries

KW - Developing countries

KW - Rural tourism

KW - Rurality

KW - Systematic literature review

UR - http://www.scopus.com/inward/record.url?scp=85102799138&partnerID=8YFLogxK

U2 - 10.1016/j.jhtm.2021.03.001

DO - 10.1016/j.jhtm.2021.03.001

M3 - Journal article

AN - SCOPUS:85102799138

SN - 1447-6770

JO - Journal of Hospitality and Tourism Management

JF - Journal of Hospitality and Tourism Management

Finished Papers

Prostate cancer incidence and mortality in rural men--a systematic review of the literature

Affiliation.

1 University of Auckland, and Urology Department, Waikato Hospital, Hamilton, New Zealand.
PMID: 22616627

Introduction: Prostate cancer is a common health problem in men worldwide. This systematic review has been undertaken to determine if there are differences in incidence of and mortality from prostate cancer between rural and urban men. The understanding of geographical patterns of prostate cancer incidence and mortality is necessary in order to identify and assess any disparities between rural and urban residents in gaining access to healthcare services, such as screening, diagnosis and treatment.

Methods: Medline, CINAHL and Embase were searched using relevant mesh phrases, such as 'prostate cancer incidence rural' or 'prostatic neoplasms mortality rural'. Secondary literature and reports not published in peer-reviewed journals were included if inclusion criteria had been met. The following inclusion criteria were applied: cohort (population-based study) of adult men, diagnosis of prostate cancer, comparing rural and urban groups, and incidence or mortality with available statistical parameters as outcome.

Results: In total, 25 studies were found to fit the inclusion criteria. Sixteen cohort studies were identified that examined incidence of prostate cancer in rural and urban populations, while 18 studies focused on mortality. Nine of these publications discussed both aspects. Twenty of these studies were published in scientific journals, while five were reports identified through secondary literature search. Prostate cancer incidence was found to be higher in urban men, while mortality patterns seemed to vary to some degree depending on different definitions of rural/urban groups, as well as on variations in demographic factors and study periods. There is evidence, however, that after prostate-specific antigen testing was introduced death rates tended to be higher in rural men with prostate cancer.

Conclusions: The review of the literature showed that in spite of inconsistent definitions of rural/urban categories among studies the majority reported higher incidence rates in urban men. This finding suggests that rural men are less likely to be screened and less likely to be subsequently diagnosed with prostate cancer. Although mortality patterns tended to be heterogeneous, there was some evidence that rural residents with prostate cancer experience higher death rates. It would be beneficial if future studies take into consideration factors such as stage at initial diagnosis, ethnicity, and socioeconomic and health status when assessing differences in cancer outcomes. Few studies in this review accounted for one or more of these variables, although there are indications that they contribute to differences in prostate cancer incidence and mortality between rural and urban populations.

Publication types

Research Support, Non-U.S. Gov't
Systematic Review
Cohort Studies
Mass Screening
Prostatic Neoplasms / diagnosis*
Prostatic Neoplasms / mortality*
Rural Population*
Urban Population

Finished Papers

Support team is ready to answer any questions at any time of day and night

Customer Reviews

Testimonials

Bennie Hawra

offers a great selection of professional essay writing services. Take advantage of original, plagiarism-free essay writing. Also, separate editing and proofreading services are available, designed for those students who did an essay and seek professional help with polishing it to perfection. In addition, a number of additional essay writing services are available to boost your customer experience to the maximum!

Advanced writer

Add more quality to your essay or be able to obtain a new paper within a day by requesting a top or premium writer to work on your order. The option will increase the price of your order but the final result will be totally worth it.

Top order status

Every day, we receive dozens of orders. To process every order, we need time. If you’re in a great hurry or seek premium service, then choose this additional service. As a result, we’ll process your order and assign a great writer as soon as it’s placed. Maximize your time by giving your order a top status!

SMS updates

Have you already started to write my essay? When it will be finished? If you have occasional questions like that, then opt-in for SMS order status updates to be informed regarding every stage of the writing process. If you’re pressed for time, then we recommend adding this extra to your order.

Plagiarism report

Is my essay original? How do I know it’s Turnitin-ready? Very simple – order us to attach a detailed plagiarism report when work is done so you could rest assured the paper is authentic and can be uploaded to Turnitin without hesitating.

1-page summary

World’s peace isn’t riding on essay writing. If you don’t have any intent on reading the entire 2000-word essay that we did for you, add a 1-page summary to your order, which will be a short overview of your essay one paragraph long, just to be in the loop.

Customer Reviews

IMAGES

(PDF) Rural definition of health: A systematic literature review
Rural definition of health: a systematic literature review
systematic literature review use cases
How to write a systematic literature review [9 steps]
How to Conduct a Systematic Review
Systematic literature review phases.

VIDEO

Bringing Healthcare to Rural Communities
Systematic Literature Review
A Comprehensive Systematic Literature Review on Intrusion Detection Systems
Online Extremism Detection A Systematic Literature Review With Emphasis on Datasets, Classification

COMMENTS

Rural definition of health: a systematic literature review
Methods. We conducted a systematic review of the literature to assess the extent and strength of evidence for rural definition of health. Studies were eligible for inclusion if they were published in English, reported on original research and presented findings or commentary relevant to rural definition of health, were published over the last 40 years, and were based on observations of rural U ...
Rural Definition of Health: A Systematic Literature Review
As providers increasingly focus on patient-centered care, it becomes crucial to understand how rural populations attitudes towards health and healthcare differ from their urban counterparts.
Rural definition of health: a systematic literature review
Gessert et al. BMC Public Health (2015) 15:378 DOI 10.1186/s12889-015-1658-9 RESEARCH ARTICLE Open Access Rural definition of health: a systematic literature review Charles Gessert1, Stephen Waring1*, Lisa Bailey-Davis2, Pat Conway1, Melissa Roberts3 and Jeffrey VanWormer4 Abstract Background: The advent of patient-centered care challenges policy makers, health care administrators, clinicians ...
Rural definition of health: a systematic literature review
The passage of the Affordable Care Act (ACA) in 2010 heralded a significant shift in American health care, with increased emphasis on population health, disease prevention, and cost containment. These new areas of emphasis are prominent in several …
PDF Access to specialty healthcare in urban versus rural US populations: a
This sys-tematic literature review therefore (1) thematically sum-marizes recent studies describing characteristics of urban and rural specialty care access in the US, (2) iden-tifies thematic gaps in Levesque et al.'s framework [22], and (3) incorporates results into an expanded framework specific to US urban versus rural specialty care access.
Health disparities among rural individuals with mental health
There is growing concern about the availability of health care services for rural patients. This systematic literature review evaluates original research on health disparities among rural and urban populations with mental health (MH) conditions in North America. Using the Preferred Reporting Items for Systematic Reveiws and Meta-Analyses guidelines, we used four electronic databases (Pubmed ...
PDF Impact of Community Health Workers on Access to Care for Rural
et al. [17]. The present systematic review adapted methodol-ogy from Jack et al. in order to include more recent literature and to specically address issues pertaining to rural settings. Search Strategy A string of keywords was developed based on the previously published systematic review and other keywords identied in relevant literature.
Access to specialty healthcare in urban versus rural US ...
Abstract. Background: Access to healthcare is a poorly defined construct, with insufficient understanding of differences in facilitators and barriers between US urban versus rural specialty care. We summarize recent literature and expand upon a prior conceptual access framework, adapted here specifically to urban and rural specialty care.
30316 PDFs
Aim: The aim of this systematic review was to examine the literature regarding rural healthcare delivery for women with any type of diabetes in pregnancy, and subsequent maternal and infant outcomes.
Primary Health Care Interventions for Potentially Preventable Ear, Nose
Primary health care prevention strategies are typically categorized into 3 levels as primary, secondary, and tertiary prevention. 29 Primary and secondary prevention aim to prevent disease or injury before it occurs through health promotion initiatives and screening for early disease. Tertiary prevention is the treatment once a disease has been identified (including medication prescriptions ...
Rural definition of health: a systematic literature review
Europe PMC is an archive of life sciences journal literature. https://orcid.org
PRISMA statement
Here you can access information about the PRISMA reporting guidelines, which are designed to help authors transparently report why their systematic review was done, what methods they used, and what they found. The main PRISMA reporting guideline (the PRISMA 2020 statement) primarily provides guidance for the reporting of systematic reviews ...
A Systematic Review of the Strategies to Address Health Worker Shortage
Sample: 16 peer-review articles that address strategies to recruit and retain health workers in rural and remote areas of low- and middle-income countries. Methods: A systematic literature review was conducted following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.
The sustainability of new programs and innovations: a review of the
The introduction of evidence-based programs and practices into healthcare settings has been the subject of an increasing amount of research in recent years. While a number of studies have examined initial implementation efforts, less research has been conducted to determine what happens beyond that point. There is increasing recognition that the extent to which new programs are sustained is ...
Rural tourism: A systematic literature review on definitions and
Abstract. The definition of rural tourism remains unclear and only a few studies have mapped the current state of knowledge in this field. Through a systematic quantitative literature review, this study extends the previous literature by investigating rural tourism definitions and challenges faced within developed and developing contexts.
Rural tourism: A systematic literature review on definitions and
Turismo comunitário Jorge Alexander Mora Forero Alexander Forero. Sociology, Environmental Science. 2022. Rural tourism is characterized for being mainly community-based and for having links of associativity and cooperativism among farmers. The objective of this research is to describe the actions of the….
Rural Definition Of Health A Systematic Literature Review
Rural Definition Of Health A Systematic Literature Review. Accept. 4.8/5. 20 Customer reviews. Nursing Management Business and Economics History +104. Nursing Management Business and Economics Psychology +69. Nursing Management Business and Economics Ethnicity Studies +90.
Prostate cancer incidence and mortality in rural men--a systematic
Abstract. Introduction: Prostate cancer is a common health problem in men worldwide. This systematic review has been undertaken to determine if there are differences in incidence of and mortality from prostate cancer between rural and urban men. The understanding of geographical patterns of prostate cancer incidence and mortality is necessary ...
Rural Definition Of Health A Systematic Literature Review
Rural Definition Of Health A Systematic Literature Review. We are inclined to write as per the instructions given to you along with our understanding and background research related to the given topic. The topic is well-researched first and then the draft is being written. Order preparation While our expert is working on your order, you will be ...
Rural Definition Of Health A Systematic Literature Review
Rural Definition Of Health A Systematic Literature Review | Best Writing Service. 4.7/5. 1 (888)814-4206 1 (888)499-5521. 100% Success rate. ID 5683. Create New Order.
Rural Definition Of Health A Systematic Literature Review
Only 3% of all applicants are accepted to work with us and even these 3% have a training program and a two-month trial period ahead. We value our reputation and only hire true experts with years of experience in academic writing behind their backs. Nonetheless, being a professional writers service has its challenges.
Water fluoridation in Australia: A systematic review
Water fluoridation is considered a safe and effective public health strategy to improve oral health. This review aimed to systematically summarize the available evidence of water fluoridation in Australia, focusing on the history, health impacts, cost effectiveness, challenges, and limitations. A systematic search was conducted on the Ovid Medline, Web of Science, Scopus, ProQuest Central ...

Rural definition of health: a systematic literature review

Stephen Waring

Melissa Roberts

Jeffrey VanWormer

Conclusions

Electronic supplementary material

Acknowledgements

Abbreviations

Contributor Information

Rural definition of health: a systematic literature review

Related Papers

RELATED PAPERS

RELATED TOPICS

ACC 2024 Congress

Springer Medicine

Rural definition of health: a systematic literature review

Conclusions

Tuberculosis treatment delays and associated factors within the Zimbabwe national tuberculosis programme

Impact of Community Health Workers on Access to Care for Rural Populations in the United States: A Systematic Review

Cite this article

Access this article

Similar content being viewed by others

Access to health services among culturally and linguistically diverse populations in the Australian universal health care system: issues and challenges

Public health system challenges in the Free State, South Africa: a situation appraisal to inform health system strengthening

Aboriginal community controlled health organisations address health equity through action on the social determinants of health of Aboriginal and Torres Strait Islander peoples in Australia

Data Availability

Code Availability

Acknowledgements

Author information

Contributions

Corresponding author

Ethics declarations

Ethical Approval

Consent to Participate

Consent for Publication

Search strategies

Rights and permissions

About this article

Share this article

Search life-sciences literature (43,960,127 articles, preprints and more)

Rural definition of health: a systematic literature review.

ORCIDs linked to this article

Abstract

Rural definition of health: a systematic literature review

Stephen Waring

Melissa Roberts

Jeffrey VanWormer

Electronic supplementary material

Acknowledgements

Abbreviations

Contributor Information

Full text links

Citations & impact

Rural-urban disparities in health outcomes, clinical care, health behaviors, and social determinants of health and an action-oriented, dynamic tool for visualizing them.

Rural health and rural industries: Opportunities for partnership and action.

A perspective on the role of language about pain.

The impact of health literacy interventions on glycemic control and self-management outcomes among type 2 diabetes mellitus: A systematic review.

Similar Articles

Work disability prevention in rural healthcare workers.

The patient experience of patient-centered communication with nurses in the hospital setting: a qualitative systematic review protocol.

Student and educator experiences of maternal-child simulation-based learning: a systematic review of qualitative evidence protocol.

Telehealth services in rural and remote Australia: a systematic review of models of care and factors influencing success and sustainability.

The effectiveness of internet-based e-learning on clinician behavior and patient outcomes: a systematic review protocol.

The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research

Conclusions

Search method

Inclusion and exclusion criteria

Review methods

Characteristics of included studies

Unit of analysis

Timeframe examined

Terms and definitions

Methods used

Outcomes reported

Summary of findings

Associated elements and influences

Sustainability outcomes

Influences on sustainability

Recommendations for advancing the empirical literature

Defining outcomes or desired benefits