what are methods in a research paper

When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

• PLOS Biology
• PLOS Climate
• PLOS Complex Systems
• PLOS Computational Biology
• PLOS Digital Health
• PLOS Genetics
• PLOS Global Public Health
• PLOS Medicine
• PLOS Mental Health
• PLOS Neglected Tropical Diseases
• PLOS Pathogens
• PLOS Sustainability and Transformation
• PLOS Collections
• How to Write Your Methods

Ensure understanding, reproducibility and replicability

What should you include in your methods section, and how much detail is appropriate?

Why Methods Matter

The methods section was once the most likely part of a paper to be unfairly abbreviated, overly summarized, or even relegated to hard-to-find sections of a publisher’s website. While some journals may responsibly include more detailed elements of methods in supplementary sections, the movement for increased reproducibility and rigor in science has reinstated the importance of the methods section. Methods are now viewed as a key element in establishing the credibility of the research being reported, alongside the open availability of data and results.

A clear methods section impacts editorial evaluation and readers’ understanding, and is also the backbone of transparency and replicability.

For example, the Reproducibility Project: Cancer Biology project set out in 2013 to replicate experiments from 50 high profile cancer papers, but revised their target to 18 papers once they understood how much methodological detail was not contained in the original papers.

What to include in your methods section

What you include in your methods sections depends on what field you are in and what experiments you are performing. However, the general principle in place at the majority of journals is summarized well by the guidelines at PLOS ONE : “The Materials and Methods section should provide enough detail to allow suitably skilled investigators to fully replicate your study. ” The emphases here are deliberate: the methods should enable readers to understand your paper, and replicate your study. However, there is no need to go into the level of detail that a lay-person would require—the focus is on the reader who is also trained in your field, with the suitable skills and knowledge to attempt a replication.

A constant principle of rigorous science

A methods section that enables other researchers to understand and replicate your results is a constant principle of rigorous, transparent, and Open Science. Aim to be thorough, even if a particular journal doesn’t require the same level of detail . Reproducibility is all of our responsibility. You cannot create any problems by exceeding a minimum standard of information. If a journal still has word-limits—either for the overall article or specific sections—and requires some methodological details to be in a supplemental section, that is OK as long as the extra details are searchable and findable .

Imagine replicating your own work, years in the future

As part of PLOS’ presentation on Reproducibility and Open Publishing (part of UCSF’s Reproducibility Series ) we recommend planning the level of detail in your methods section by imagining you are writing for your future self, replicating your own work. When you consider that you might be at a different institution, with different account logins, applications, resources, and access levels—you can help yourself imagine the level of specificity that you yourself would require to redo the exact experiment. Consider:

• Which details would you need to be reminded of? 
• Which cell line, or antibody, or software, or reagent did you use, and does it have a Research Resource ID (RRID) that you can cite?
• Which version of a questionnaire did you use in your survey? 
• Exactly which visual stimulus did you show participants, and is it publicly available? 
• What participants did you decide to exclude? 
• What process did you adjust, during your work? 

Tip: Be sure to capture any changes to your protocols

You yourself would want to know about any adjustments, if you ever replicate the work, so you can surmise that anyone else would want to as well. Even if a necessary adjustment you made was not ideal, transparency is the key to ensuring this is not regarded as an issue in the future. It is far better to transparently convey any non-optimal methods, or methodological constraints, than to conceal them, which could result in reproducibility or ethical issues downstream.

Visual aids for methods help when reading the whole paper

Consider whether a visual representation of your methods could be appropriate or aid understanding your process. A visual reference readers can easily return to, like a flow-diagram, decision-tree, or checklist, can help readers to better understand the complete article, not just the methods section.

Ethical Considerations

In addition to describing what you did, it is just as important to assure readers that you also followed all relevant ethical guidelines when conducting your research. While ethical standards and reporting guidelines are often presented in a separate section of a paper, ensure that your methods and protocols actually follow these guidelines. Read more about ethics .

Existing standards, checklists, guidelines, partners

While the level of detail contained in a methods section should be guided by the universal principles of rigorous science outlined above, various disciplines, fields, and projects have worked hard to design and develop consistent standards, guidelines, and tools to help with reporting all types of experiment. Below, you’ll find some of the key initiatives. Ensure you read the submission guidelines for the specific journal you are submitting to, in order to discover any further journal- or field-specific policies to follow, or initiatives/tools to utilize.

Tip: Keep your paper moving forward by providing the proper paperwork up front

Be sure to check the journal guidelines and provide the necessary documents with your manuscript submission. Collecting the necessary documentation can greatly slow the first round of peer review, or cause delays when you submit your revision.

Randomized Controlled Trials – CONSORT The Consolidated Standards of Reporting Trials (CONSORT) project covers various initiatives intended to prevent the problems of  inadequate reporting of randomized controlled trials. The primary initiative is an evidence-based minimum set of recommendations for reporting randomized trials known as the CONSORT Statement . 

Systematic Reviews and Meta-Analyses – PRISMA The Preferred Reporting Items for Systematic Reviews and Meta-Analyses ( PRISMA ) is an evidence-based minimum set of items focusing  on the reporting of  reviews evaluating randomized trials and other types of research.

Research using Animals – ARRIVE The Animal Research: Reporting of In Vivo Experiments ( ARRIVE ) guidelines encourage maximizing the information reported in research using animals thereby minimizing unnecessary studies. (Original study and proposal , and updated guidelines , in PLOS Biology .) 

Laboratory Protocols Protocols.io has developed a platform specifically for the sharing and updating of laboratory protocols , which are assigned their own DOI and can be linked from methods sections of papers to enhance reproducibility. Contextualize your protocol and improve discovery with an accompanying Lab Protocol article in PLOS ONE .

Consistent reporting of Materials, Design, and Analysis – the MDAR checklist A cross-publisher group of editors and experts have developed, tested, and rolled out a checklist to help establish and harmonize reporting standards in the Life Sciences . The checklist , which is available for use by authors to compile their methods, and editors/reviewers to check methods, establishes a minimum set of requirements in transparent reporting and is adaptable to any discipline within the Life Sciences, by covering a breadth of potentially relevant methodological items and considerations. If you are in the Life Sciences and writing up your methods section, try working through the MDAR checklist and see whether it helps you include all relevant details into your methods, and whether it reminded you of anything you might have missed otherwise.

Summary Writing tips

The main challenge you may find when writing your methods is keeping it readable AND covering all the details needed for reproducibility and replicability. While this is difficult, do not compromise on rigorous standards for credibility!

• Keep in mind future replicability, alongside understanding and readability.
• Follow checklists, and field- and journal-specific guidelines.
• Consider a commitment to rigorous and transparent science a personal responsibility, and not just adhering to journal guidelines.
• Establish whether there are persistent identifiers for any research resources you use that can be specifically cited in your methods section.
• Deposit your laboratory protocols in Protocols.io, establishing a permanent link to them. You can update your protocols later if you improve on them, as can future scientists who follow your protocols.
• Consider visual aids like flow-diagrams, lists, to help with reading other sections of the paper.
• Be specific about all decisions made during the experiments that someone reproducing your work would need to know.

Don’t

• Summarize or abbreviate methods without giving full details in a discoverable supplemental section.
• Presume you will always be able to remember how you performed the experiments, or have access to private or institutional notebooks and resources.
• Attempt to hide constraints or non-optimal decisions you had to make–transparency is the key to ensuring the credibility of your research.
• How to Write a Great Title
• How to Write an Abstract
• How to Report Statistics
• How to Write Discussions and Conclusions
• How to Edit Your Work

The contents of the Peer Review Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

The contents of the Writing Center are also available as a live, interactive training session, complete with slides, talking points, and activities. …

There’s a lot to consider when deciding where to submit your work. Learn how to choose a journal that will help your study reach its audience, while reflecting your values as a researcher…

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• What Is a Research Methodology? | Steps & Tips

What Is a Research Methodology? | Steps & Tips

Published on 25 February 2019 by Shona McCombes . Revised on 10 October 2022.

Your research methodology discusses and explains the data collection and analysis methods you used in your research. A key part of your thesis, dissertation, or research paper, the methodology chapter explains what you did and how you did it, allowing readers to evaluate the reliability and validity of your research.

It should include:

• The type of research you conducted
• How you collected and analysed your data
• Any tools or materials you used in the research
• Why you chose these methods
• Your methodology section should generally be written in the past tense .
• Academic style guides in your field may provide detailed guidelines on what to include for different types of studies.
• Your citation style might provide guidelines for your methodology section (e.g., an APA Style methods section ).

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

How to write a research methodology, why is a methods section important, step 1: explain your methodological approach, step 2: describe your data collection methods, step 3: describe your analysis method, step 4: evaluate and justify the methodological choices you made, tips for writing a strong methodology chapter, frequently asked questions about methodology.

Prevent plagiarism, run a free check.

Your methods section is your opportunity to share how you conducted your research and why you chose the methods you chose. It’s also the place to show that your research was rigorously conducted and can be replicated .

It gives your research legitimacy and situates it within your field, and also gives your readers a place to refer to if they have any questions or critiques in other sections.

You can start by introducing your overall approach to your research. You have two options here.

Option 1: Start with your “what”

What research problem or question did you investigate?

• Aim to describe the characteristics of something?
• Explore an under-researched topic?
• Establish a causal relationship?

And what type of data did you need to achieve this aim?

• Quantitative data , qualitative data , or a mix of both?
• Primary data collected yourself, or secondary data collected by someone else?
• Experimental data gathered by controlling and manipulating variables, or descriptive data gathered via observations?

Option 2: Start with your “why”

Depending on your discipline, you can also start with a discussion of the rationale and assumptions underpinning your methodology. In other words, why did you choose these methods for your study?

• Why is this the best way to answer your research question?
• Is this a standard methodology in your field, or does it require justification?
• Were there any ethical considerations involved in your choices?
• What are the criteria for validity and reliability in this type of research ?

Once you have introduced your reader to your methodological approach, you should share full details about your data collection methods .

Quantitative methods

In order to be considered generalisable, you should describe quantitative research methods in enough detail for another researcher to replicate your study.

Here, explain how you operationalised your concepts and measured your variables. Discuss your sampling method or inclusion/exclusion criteria, as well as any tools, procedures, and materials you used to gather your data.

Surveys Describe where, when, and how the survey was conducted.

• How did you design the questionnaire?
• What form did your questions take (e.g., multiple choice, Likert scale )?
• Were your surveys conducted in-person or virtually?
• What sampling method did you use to select participants?
• What was your sample size and response rate?

Experiments Share full details of the tools, techniques, and procedures you used to conduct your experiment.

• How did you design the experiment ?
• How did you recruit participants?
• How did you manipulate and measure the variables ?
• What tools did you use?

Existing data Explain how you gathered and selected the material (such as datasets or archival data) that you used in your analysis.

• Where did you source the material?
• How was the data originally produced?
• What criteria did you use to select material (e.g., date range)?

The survey consisted of 5 multiple-choice questions and 10 questions measured on a 7-point Likert scale.

The goal was to collect survey responses from 350 customers visiting the fitness apparel company’s brick-and-mortar location in Boston on 4–8 July 2022, between 11:00 and 15:00.

Here, a customer was defined as a person who had purchased a product from the company on the day they took the survey. Participants were given 5 minutes to fill in the survey anonymously. In total, 408 customers responded, but not all surveys were fully completed. Due to this, 371 survey results were included in the analysis.

Qualitative methods

In qualitative research , methods are often more flexible and subjective. For this reason, it’s crucial to robustly explain the methodology choices you made.

Be sure to discuss the criteria you used to select your data, the context in which your research was conducted, and the role you played in collecting your data (e.g., were you an active participant, or a passive observer?)

Interviews or focus groups Describe where, when, and how the interviews were conducted.

• How did you find and select participants?
• How many participants took part?
• What form did the interviews take ( structured , semi-structured , or unstructured )?
• How long were the interviews?
• How were they recorded?

Participant observation Describe where, when, and how you conducted the observation or ethnography .

• What group or community did you observe? How long did you spend there?
• How did you gain access to this group? What role did you play in the community?
• How long did you spend conducting the research? Where was it located?
• How did you record your data (e.g., audiovisual recordings, note-taking)?

Existing data Explain how you selected case study materials for your analysis.

• What type of materials did you analyse?
• How did you select them?

In order to gain better insight into possibilities for future improvement of the fitness shop’s product range, semi-structured interviews were conducted with 8 returning customers.

Here, a returning customer was defined as someone who usually bought products at least twice a week from the store.

Surveys were used to select participants. Interviews were conducted in a small office next to the cash register and lasted approximately 20 minutes each. Answers were recorded by note-taking, and seven interviews were also filmed with consent. One interviewee preferred not to be filmed.

Mixed methods

Mixed methods research combines quantitative and qualitative approaches. If a standalone quantitative or qualitative study is insufficient to answer your research question, mixed methods may be a good fit for you.

Mixed methods are less common than standalone analyses, largely because they require a great deal of effort to pull off successfully. If you choose to pursue mixed methods, it’s especially important to robustly justify your methods here.

Next, you should indicate how you processed and analysed your data. Avoid going into too much detail: you should not start introducing or discussing any of your results at this stage.

In quantitative research , your analysis will be based on numbers. In your methods section, you can include:

• How you prepared the data before analysing it (e.g., checking for missing data , removing outliers , transforming variables)
• Which software you used (e.g., SPSS, Stata or R)
• Which statistical tests you used (e.g., two-tailed t test , simple linear regression )

In qualitative research, your analysis will be based on language, images, and observations (often involving some form of textual analysis ).

Specific methods might include:

• Content analysis : Categorising and discussing the meaning of words, phrases and sentences
• Thematic analysis : Coding and closely examining the data to identify broad themes and patterns
• Discourse analysis : Studying communication and meaning in relation to their social context

Mixed methods combine the above two research methods, integrating both qualitative and quantitative approaches into one coherent analytical process.

Above all, your methodology section should clearly make the case for why you chose the methods you did. This is especially true if you did not take the most standard approach to your topic. In this case, discuss why other methods were not suitable for your objectives, and show how this approach contributes new knowledge or understanding.

In any case, it should be overwhelmingly clear to your reader that you set yourself up for success in terms of your methodology’s design. Show how your methods should lead to results that are valid and reliable, while leaving the analysis of the meaning, importance, and relevance of your results for your discussion section .

• Quantitative: Lab-based experiments cannot always accurately simulate real-life situations and behaviours, but they are effective for testing causal relationships between variables .
• Qualitative: Unstructured interviews usually produce results that cannot be generalised beyond the sample group , but they provide a more in-depth understanding of participants’ perceptions, motivations, and emotions.
• Mixed methods: Despite issues systematically comparing differing types of data, a solely quantitative study would not sufficiently incorporate the lived experience of each participant, while a solely qualitative study would be insufficiently generalisable.

Remember that your aim is not just to describe your methods, but to show how and why you applied them. Again, it’s critical to demonstrate that your research was rigorously conducted and can be replicated.

1. Focus on your objectives and research questions

The methodology section should clearly show why your methods suit your objectives  and convince the reader that you chose the best possible approach to answering your problem statement and research questions .

2. Cite relevant sources

Your methodology can be strengthened by referencing existing research in your field. This can help you to:

• Show that you followed established practice for your type of research
• Discuss how you decided on your approach by evaluating existing research
• Present a novel methodological approach to address a gap in the literature

3. Write for your audience

Consider how much information you need to give, and avoid getting too lengthy. If you are using methods that are standard for your discipline, you probably don’t need to give a lot of background or justification.

Regardless, your methodology should be a clear, well-structured text that makes an argument for your approach, not just a list of technical details and procedures.

Methodology refers to the overarching strategy and rationale of your research. Developing your methodology involves studying the research methods used in your field and the theories or principles that underpin them, in order to choose the approach that best matches your objectives.

Methods are the specific tools and procedures you use to collect and analyse data (e.g. interviews, experiments , surveys , statistical tests ).

In a dissertation or scientific paper, the methodology chapter or methods section comes after the introduction and before the results , discussion and conclusion .

Depending on the length and type of document, you might also include a literature review or theoretical framework before the methodology.

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to test a hypothesis by systematically collecting and analysing data, while qualitative methods allow you to explore ideas and experiences in depth.

A sample is a subset of individuals from a larger population. Sampling means selecting the group that you will actually collect data from in your research.

For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

Statistical sampling allows you to test a hypothesis about the characteristics of a population. There are various sampling methods you can use to ensure that your sample is representative of the population as a whole.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, October 10). What Is a Research Methodology? | Steps & Tips. Scribbr. Retrieved 2 April 2024, from https://www.scribbr.co.uk/thesis-dissertation/methodology/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a literature review | guide, template, & examples, what is a theoretical framework | a step-by-step guide.

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 6. The Methodology
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

The methods section describes actions taken to investigate a research problem and the rationale for the application of specific procedures or techniques used to identify, select, process, and analyze information applied to understanding the problem, thereby, allowing the reader to critically evaluate a study’s overall validity and reliability. The methodology section of a research paper answers two main questions: How was the data collected or generated? And, how was it analyzed? The writing should be direct and precise and always written in the past tense.

Kallet, Richard H. "How to Write the Methods Section of a Research Paper." Respiratory Care 49 (October 2004): 1229-1232.

Importance of a Good Methodology Section

You must explain how you obtained and analyzed your results for the following reasons:

• Readers need to know how the data was obtained because the method you chose affects the results and, by extension, how you interpreted their significance in the discussion section of your paper.
• Methodology is crucial for any branch of scholarship because an unreliable method produces unreliable results and, as a consequence, undermines the value of your analysis of the findings.
• In most cases, there are a variety of different methods you can choose to investigate a research problem. The methodology section of your paper should clearly articulate the reasons why you have chosen a particular procedure or technique.
• The reader wants to know that the data was collected or generated in a way that is consistent with accepted practice in the field of study. For example, if you are using a multiple choice questionnaire, readers need to know that it offered your respondents a reasonable range of answers to choose from.
• The method must be appropriate to fulfilling the overall aims of the study. For example, you need to ensure that you have a large enough sample size to be able to generalize and make recommendations based upon the findings.
• The methodology should discuss the problems that were anticipated and the steps you took to prevent them from occurring. For any problems that do arise, you must describe the ways in which they were minimized or why these problems do not impact in any meaningful way your interpretation of the findings.
• In the social and behavioral sciences, it is important to always provide sufficient information to allow other researchers to adopt or replicate your methodology. This information is particularly important when a new method has been developed or an innovative use of an existing method is utilized.

Bem, Daryl J. Writing the Empirical Journal Article. Psychology Writing Center. University of Washington; Denscombe, Martyn. The Good Research Guide: For Small-Scale Social Research Projects . 5th edition. Buckingham, UK: Open University Press, 2014; Lunenburg, Frederick C. Writing a Successful Thesis or Dissertation: Tips and Strategies for Students in the Social and Behavioral Sciences . Thousand Oaks, CA: Corwin Press, 2008.

Structure and Writing Style

I.  Groups of Research Methods

There are two main groups of research methods in the social sciences:

• The e mpirical-analytical group approaches the study of social sciences in a similar manner that researchers study the natural sciences . This type of research focuses on objective knowledge, research questions that can be answered yes or no, and operational definitions of variables to be measured. The empirical-analytical group employs deductive reasoning that uses existing theory as a foundation for formulating hypotheses that need to be tested. This approach is focused on explanation.
• The i nterpretative group of methods is focused on understanding phenomenon in a comprehensive, holistic way . Interpretive methods focus on analytically disclosing the meaning-making practices of human subjects [the why, how, or by what means people do what they do], while showing how those practices arrange so that it can be used to generate observable outcomes. Interpretive methods allow you to recognize your connection to the phenomena under investigation. However, the interpretative group requires careful examination of variables because it focuses more on subjective knowledge.

II.  Content

The introduction to your methodology section should begin by restating the research problem and underlying assumptions underpinning your study. This is followed by situating the methods you used to gather, analyze, and process information within the overall “tradition” of your field of study and within the particular research design you have chosen to study the problem. If the method you choose lies outside of the tradition of your field [i.e., your review of the literature demonstrates that the method is not commonly used], provide a justification for how your choice of methods specifically addresses the research problem in ways that have not been utilized in prior studies.

The remainder of your methodology section should describe the following:

• Decisions made in selecting the data you have analyzed or, in the case of qualitative research, the subjects and research setting you have examined,
• Tools and methods used to identify and collect information, and how you identified relevant variables,
• The ways in which you processed the data and the procedures you used to analyze that data, and
• The specific research tools or strategies that you utilized to study the underlying hypothesis and research questions.

In addition, an effectively written methodology section should:

• Introduce the overall methodological approach for investigating your research problem . Is your study qualitative or quantitative or a combination of both (mixed method)? Are you going to take a special approach, such as action research, or a more neutral stance?
• Indicate how the approach fits the overall research design . Your methods for gathering data should have a clear connection to your research problem. In other words, make sure that your methods will actually address the problem. One of the most common deficiencies found in research papers is that the proposed methodology is not suitable to achieving the stated objective of your paper.
• Describe the specific methods of data collection you are going to use , such as, surveys, interviews, questionnaires, observation, archival research. If you are analyzing existing data, such as a data set or archival documents, describe how it was originally created or gathered and by whom. Also be sure to explain how older data is still relevant to investigating the current research problem.
• Explain how you intend to analyze your results . Will you use statistical analysis? Will you use specific theoretical perspectives to help you analyze a text or explain observed behaviors? Describe how you plan to obtain an accurate assessment of relationships, patterns, trends, distributions, and possible contradictions found in the data.
• Provide background and a rationale for methodologies that are unfamiliar for your readers . Very often in the social sciences, research problems and the methods for investigating them require more explanation/rationale than widely accepted rules governing the natural and physical sciences. Be clear and concise in your explanation.
• Provide a justification for subject selection and sampling procedure . For instance, if you propose to conduct interviews, how do you intend to select the sample population? If you are analyzing texts, which texts have you chosen, and why? If you are using statistics, why is this set of data being used? If other data sources exist, explain why the data you chose is most appropriate to addressing the research problem.
• Provide a justification for case study selection . A common method of analyzing research problems in the social sciences is to analyze specific cases. These can be a person, place, event, phenomenon, or other type of subject of analysis that are either examined as a singular topic of in-depth investigation or multiple topics of investigation studied for the purpose of comparing or contrasting findings. In either method, you should explain why a case or cases were chosen and how they specifically relate to the research problem.
• Describe potential limitations . Are there any practical limitations that could affect your data collection? How will you attempt to control for potential confounding variables and errors? If your methodology may lead to problems you can anticipate, state this openly and show why pursuing this methodology outweighs the risk of these problems cropping up.

NOTE :   Once you have written all of the elements of the methods section, subsequent revisions should focus on how to present those elements as clearly and as logically as possibly. The description of how you prepared to study the research problem, how you gathered the data, and the protocol for analyzing the data should be organized chronologically. For clarity, when a large amount of detail must be presented, information should be presented in sub-sections according to topic. If necessary, consider using appendices for raw data.

ANOTHER NOTE : If you are conducting a qualitative analysis of a research problem , the methodology section generally requires a more elaborate description of the methods used as well as an explanation of the processes applied to gathering and analyzing of data than is generally required for studies using quantitative methods. Because you are the primary instrument for generating the data [e.g., through interviews or observations], the process for collecting that data has a significantly greater impact on producing the findings. Therefore, qualitative research requires a more detailed description of the methods used.

YET ANOTHER NOTE :   If your study involves interviews, observations, or other qualitative techniques involving human subjects , you may be required to obtain approval from the university's Office for the Protection of Research Subjects before beginning your research. This is not a common procedure for most undergraduate level student research assignments. However, i f your professor states you need approval, you must include a statement in your methods section that you received official endorsement and adequate informed consent from the office and that there was a clear assessment and minimization of risks to participants and to the university. This statement informs the reader that your study was conducted in an ethical and responsible manner. In some cases, the approval notice is included as an appendix to your paper.

III.  Problems to Avoid

Irrelevant Detail The methodology section of your paper should be thorough but concise. Do not provide any background information that does not directly help the reader understand why a particular method was chosen, how the data was gathered or obtained, and how the data was analyzed in relation to the research problem [note: analyzed, not interpreted! Save how you interpreted the findings for the discussion section]. With this in mind, the page length of your methods section will generally be less than any other section of your paper except the conclusion.

Unnecessary Explanation of Basic Procedures Remember that you are not writing a how-to guide about a particular method. You should make the assumption that readers possess a basic understanding of how to investigate the research problem on their own and, therefore, you do not have to go into great detail about specific methodological procedures. The focus should be on how you applied a method , not on the mechanics of doing a method. An exception to this rule is if you select an unconventional methodological approach; if this is the case, be sure to explain why this approach was chosen and how it enhances the overall process of discovery.

Problem Blindness It is almost a given that you will encounter problems when collecting or generating your data, or, gaps will exist in existing data or archival materials. Do not ignore these problems or pretend they did not occur. Often, documenting how you overcame obstacles can form an interesting part of the methodology. It demonstrates to the reader that you can provide a cogent rationale for the decisions you made to minimize the impact of any problems that arose.

Literature Review Just as the literature review section of your paper provides an overview of sources you have examined while researching a particular topic, the methodology section should cite any sources that informed your choice and application of a particular method [i.e., the choice of a survey should include any citations to the works you used to help construct the survey].

It’s More than Sources of Information! A description of a research study's method should not be confused with a description of the sources of information. Such a list of sources is useful in and of itself, especially if it is accompanied by an explanation about the selection and use of the sources. The description of the project's methodology complements a list of sources in that it sets forth the organization and interpretation of information emanating from those sources.

Azevedo, L.F. et al. "How to Write a Scientific Paper: Writing the Methods Section." Revista Portuguesa de Pneumologia 17 (2011): 232-238; Blair Lorrie. “Choosing a Methodology.” In Writing a Graduate Thesis or Dissertation , Teaching Writing Series. (Rotterdam: Sense Publishers 2016), pp. 49-72; Butin, Dan W. The Education Dissertation A Guide for Practitioner Scholars . Thousand Oaks, CA: Corwin, 2010; Carter, Susan. Structuring Your Research Thesis . New York: Palgrave Macmillan, 2012; Kallet, Richard H. “How to Write the Methods Section of a Research Paper.” Respiratory Care 49 (October 2004):1229-1232; Lunenburg, Frederick C. Writing a Successful Thesis or Dissertation: Tips and Strategies for Students in the Social and Behavioral Sciences . Thousand Oaks, CA: Corwin Press, 2008. Methods Section. The Writer’s Handbook. Writing Center. University of Wisconsin, Madison; Rudestam, Kjell Erik and Rae R. Newton. “The Method Chapter: Describing Your Research Plan.” In Surviving Your Dissertation: A Comprehensive Guide to Content and Process . (Thousand Oaks, Sage Publications, 2015), pp. 87-115; What is Interpretive Research. Institute of Public and International Affairs, University of Utah; Writing the Experimental Report: Methods, Results, and Discussion. The Writing Lab and The OWL. Purdue University; Methods and Materials. The Structure, Format, Content, and Style of a Journal-Style Scientific Paper. Department of Biology. Bates College.

Writing Tip

Statistical Designs and Tests? Do Not Fear Them!

Don't avoid using a quantitative approach to analyzing your research problem just because you fear the idea of applying statistical designs and tests. A qualitative approach, such as conducting interviews or content analysis of archival texts, can yield exciting new insights about a research problem, but it should not be undertaken simply because you have a disdain for running a simple regression. A well designed quantitative research study can often be accomplished in very clear and direct ways, whereas, a similar study of a qualitative nature usually requires considerable time to analyze large volumes of data and a tremendous burden to create new paths for analysis where previously no path associated with your research problem had existed.

To locate data and statistics, GO HERE .

Another Writing Tip

Knowing the Relationship Between Theories and Methods

There can be multiple meaning associated with the term "theories" and the term "methods" in social sciences research. A helpful way to delineate between them is to understand "theories" as representing different ways of characterizing the social world when you research it and "methods" as representing different ways of generating and analyzing data about that social world. Framed in this way, all empirical social sciences research involves theories and methods, whether they are stated explicitly or not. However, while theories and methods are often related, it is important that, as a researcher, you deliberately separate them in order to avoid your theories playing a disproportionate role in shaping what outcomes your chosen methods produce.

Introspectively engage in an ongoing dialectic between the application of theories and methods to help enable you to use the outcomes from your methods to interrogate and develop new theories, or ways of framing conceptually the research problem. This is how scholarship grows and branches out into new intellectual territory.

Reynolds, R. Larry. Ways of Knowing. Alternative Microeconomics . Part 1, Chapter 3. Boise State University; The Theory-Method Relationship. S-Cool Revision. United Kingdom.

Yet Another Writing Tip

Methods and the Methodology

Do not confuse the terms "methods" and "methodology." As Schneider notes, a method refers to the technical steps taken to do research . Descriptions of methods usually include defining and stating why you have chosen specific techniques to investigate a research problem, followed by an outline of the procedures you used to systematically select, gather, and process the data [remember to always save the interpretation of data for the discussion section of your paper].

The methodology refers to a discussion of the underlying reasoning why particular methods were used . This discussion includes describing the theoretical concepts that inform the choice of methods to be applied, placing the choice of methods within the more general nature of academic work, and reviewing its relevance to examining the research problem. The methodology section also includes a thorough review of the methods other scholars have used to study the topic.

Bryman, Alan. "Of Methods and Methodology." Qualitative Research in Organizations and Management: An International Journal 3 (2008): 159-168; Schneider, Florian. “What's in a Methodology: The Difference between Method, Methodology, and Theory…and How to Get the Balance Right?” PoliticsEastAsia.com. Chinese Department, University of Leiden, Netherlands.

• << Previous: Scholarly vs. Popular Publications
• Next: Qualitative Methods >>
• Last Updated: Apr 5, 2024 1:38 PM
• URL: https://libguides.usc.edu/writingguide

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2023 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

How to Write a Methods Section for a Psychology Paper

Tips and Examples of an APA Methods Section

Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Emily is a board-certified science editor who has worked with top digital publishing brands like Voices for Biodiversity, Study.com, GoodTherapy, Vox, and Verywell.

Verywell / Brianna Gilmartin 

The methods section of an APA format psychology paper provides the methods and procedures used in a research study or experiment . This part of an APA paper is critical because it allows other researchers to see exactly how you conducted your research.

Method refers to the procedure that was used in a research study. It included a precise description of how the experiments were performed and why particular procedures were selected. While the APA technically refers to this section as the 'method section,' it is also often known as a 'methods section.'

The methods section ensures the experiment's reproducibility and the assessment of alternative methods that might produce different results. It also allows researchers to replicate the experiment and judge the study's validity.

This article discusses how to write a methods section for a psychology paper, including important elements to include and tips that can help.

What to Include in a Method Section

So what exactly do you need to include when writing your method section? You should provide detailed information on the following:

• Research design
• Participants
• Participant behavior

The method section should provide enough information to allow other researchers to replicate your experiment or study.

Components of a Method Section

The method section should utilize subheadings to divide up different subsections. These subsections typically include participants, materials, design, and procedure.

Participants 

In this part of the method section, you should describe the participants in your experiment, including who they were (and any unique features that set them apart from the general population), how many there were, and how they were selected. If you utilized random selection to choose your participants, it should be noted here.

For example: "We randomly selected 100 children from elementary schools near the University of Arizona."

At the very minimum, this part of your method section must convey:

• Basic demographic characteristics of your participants (such as sex, age, ethnicity, or religion)
• The population from which your participants were drawn
• Any restrictions on your pool of participants
• How many participants were assigned to each condition and how they were assigned to each group (i.e., randomly assignment , another selection method, etc.)
• Why participants took part in your research (i.e., the study was advertised at a college or hospital, they received some type of incentive, etc.)

Information about participants helps other researchers understand how your study was performed, how generalizable the result might be, and allows other researchers to replicate the experiment with other populations to see if they might obtain the same results.

In this part of the method section, you should describe the materials, measures, equipment, or stimuli used in the experiment. This may include:

• Testing instruments
• Technical equipment
• Any psychological assessments that were used
• Any special equipment that was used

For example: "Two stories from Sullivan et al.'s (1994) second-order false belief attribution tasks were used to assess children's understanding of second-order beliefs."

For standard equipment such as computers, televisions, and videos, you can simply name the device and not provide further explanation.

Specialized equipment should be given greater detail, especially if it is complex or created for a niche purpose. In some instances, such as if you created a special material or apparatus for your study, you might need to include an illustration of the item in the appendix of your paper.

In this part of your method section, describe the type of design used in the experiment. Specify the variables as well as the levels of these variables. Identify:

• The independent variables
• Dependent variables
• Control variables
• Any extraneous variables that might influence your results.

Also, explain whether your experiment uses a  within-groups  or between-groups design.

For example: "The experiment used a 3x2 between-subjects design. The independent variables were age and understanding of second-order beliefs."

The next part of your method section should detail the procedures used in your experiment. Your procedures should explain:

• What the participants did
• How data was collected
• The order in which steps occurred

For example: "An examiner interviewed children individually at their school in one session that lasted 20 minutes on average. The examiner explained to each child that he or she would be told two short stories and that some questions would be asked after each story. All sessions were videotaped so the data could later be coded."

Keep this subsection concise yet detailed. Explain what you did and how you did it, but do not overwhelm your readers with too much information.

Tips for How to Write a Methods Section

In addition to following the basic structure of an APA method section, there are also certain things you should remember when writing this section of your paper. Consider the following tips when writing this section:

• Use the past tense : Always write the method section in the past tense.
• Be descriptive : Provide enough detail that another researcher could replicate your experiment, but focus on brevity. Avoid unnecessary detail that is not relevant to the outcome of the experiment.
• Use an academic tone : Use formal language and avoid slang or colloquial expressions. Word choice is also important. Refer to the people in your experiment or study as "participants" rather than "subjects."
• Use APA format : Keep a style guide on hand as you write your method section. The Publication Manual of the American Psychological Association is the official source for APA style.
• Make connections : Read through each section of your paper for agreement with other sections. If you mention procedures in the method section, these elements should be discussed in the results and discussion sections.
• Proofread : Check your paper for grammar, spelling, and punctuation errors.. typos, grammar problems, and spelling errors. Although a spell checker is a handy tool, there are some errors only you can catch.

After writing a draft of your method section, be sure to get a second opinion. You can often become too close to your work to see errors or lack of clarity. Take a rough draft of your method section to your university's writing lab for additional assistance.

A Word From Verywell

The method section is one of the most important components of your APA format paper. The goal of your paper should be to clearly detail what you did in your experiment. Provide enough detail that another researcher could replicate your study if they wanted.

Finally, if you are writing your paper for a class or for a specific publication, be sure to keep in mind any specific instructions provided by your instructor or by the journal editor. Your instructor may have certain requirements that you need to follow while writing your method section.

Frequently Asked Questions

While the subsections can vary, the three components that should be included are sections on the participants, the materials, and the procedures.

• Describe who the participants were in the study and how they were selected.
• Define and describe the materials that were used including any equipment, tests, or assessments
• Describe how the data was collected

To write your methods section in APA format, describe your participants, materials, study design, and procedures. Keep this section succinct, and always write in the past tense. The main heading of this section should be labeled "Method" and it should be centered, bolded, and capitalized. Each subheading within this section should be bolded, left-aligned and in title case.

The purpose of the methods section is to describe what you did in your experiment. It should be brief, but include enough detail that someone could replicate your experiment based on this information. Your methods section should detail what you did to answer your research question. Describe how the study was conducted, the study design that was used and why it was chosen, and how you collected the data and analyzed the results.

Erdemir F. How to write a materials and methods section of a scientific article ? Turk J Urol . 2013;39(Suppl 1):10-5. doi:10.5152/tud.2013.047

Kallet RH. How to write the methods section of a research paper . Respir Care . 2004;49(10):1229-32. PMID: 15447808.

American Psychological Association.  Publication Manual of the American Psychological Association  (7th ed.). Washington DC: The American Psychological Association; 2019.

American Psychological Association. APA Style Journal Article Reporting Standards . Published 2020.

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Organizing Academic Research Papers: 6. The Methodology

• Purpose of Guide
• Design Flaws to Avoid
• Glossary of Research Terms
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Choosing a Title
• Making an Outline
• Paragraph Development
• Executive Summary
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tertiary Sources
• What Is Scholarly vs. Popular?
• Qualitative Methods
• Quantitative Methods
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Annotated Bibliography
• Dealing with Nervousness
• Using Visual Aids
• Grading Someone Else's Paper
• How to Manage Group Projects
• Multiple Book Review Essay
• Reviewing Collected Essays
• About Informed Consent
• Writing Field Notes
• Writing a Policy Memo
• Writing a Research Proposal
• Acknowledgements

The methods section of a research paper provides the information by which a study’s validity is judged. The method section answers two main questions: 1) How was the data collected or generated? 2) How was it analyzed? The writing should be direct and precise and written in the past tense.

Importance of a Good Methodology Section

You must explain how you obtained and analyzed your results for the following reasons:

• Readers need to know how the data was obtained because the method you choose affects the results and, by extension, how you likely interpreted those results.
• Methodology is crucial for any branch of scholarship because an unreliable method produces unreliable results and it misappropriates interpretations of findings .
• In most cases, there are a variety of different methods you can choose to investigate a research problem. Your methodology section of your paper should make clear the reasons why you chose a particular method or procedure .
• The reader wants to know that the data was collected or generated in a way that is consistent with accepted practice in the field of study. For example, if you are using a questionnaire, readers need to know that it offered your respondents a reasonable range of answers to choose from.
• The research method must be appropriate to the objectives of the study . For example, be sure you have a large enough sample size to be able to generalize and make recommendations based upon the findings.
• The methodology should discuss the problems that were anticipated and the steps you took to prevent them from occurring . For any problems that did arise, you must describe the ways in which their impact was minimized or why these problems do not affect the findings in any way that impacts your interpretation of the data.
• Often in social science research, it is useful for other researchers to adapt or replicate your methodology. Therefore, it is important to always provide sufficient information to allow others to use or replicate the study . This information is particularly important when a new method had been developed or an innovative use of an existing method has been utilized.

Bem, Daryl J. Writing the Empirical Journal Article . Psychology Writing Center. University of Washington; Lunenburg, Frederick C. Writing a Successful Thesis or Dissertation: Tips and Strategies for Students in the Social and Behavioral Sciences . Thousand Oaks, CA: Corwin Press, 2008.

Structure and Writing Style

I. Groups of Research Methods

There are two main groups of research methods in the social sciences:

• The empirical-analytical group approaches the study of social sciences in a similar manner that researchers study the natural sciences. This type of research focuses on objective knowledge, research questions that can be answered yes or no, and operational definitions of variables to be measured. The empirical-analytical group employs deductive reasoning that uses existing theory as a foundation for hypotheses that need to be tested. This approach is focused on explanation .
• The interpretative group is focused on understanding phenomenon in a comprehensive, holistic way . This research method allows you to recognize your connection to the subject under study. Because the interpretative group focuses more on subjective knowledge, it requires careful interpretation of variables.

II. Content

An effectively written methodology section should:

• Introduce the overall methodological approach for investigating your research problem . Is your study qualitative or quantitative or a combination of both (mixed method)? Are you going to take a special approach, such as action research, or a more neutral stance?
• Indicate how the approach fits the overall research design . Your methods should have a clear connection with your research problem. In other words, make sure that your methods will actually address the problem. One of the most common deficiencies found in research papers is that the proposed methodology is unsuited to achieving the stated objective of your paper.
• Describe the specific methods of data collection you are going to use , such as, surveys, interviews, questionnaires, observation, archival research. If you are analyzing existing data, such as a data set or archival documents, describe how it was originally created or gathered and by whom.
• Explain how you intend to analyze your results . Will you use statistical analysis? Will you use specific theoretical perspectives to help you analyze a text or explain observed behaviors?
• Provide background and rationale for methodologies that are unfamiliar for your readers . Very often in the social sciences, research problems and the methods for investigating them require more explanation/rationale than widely accepted rules governing the natural and physical sciences. Be clear and concise in your explanation.
• Provide a rationale for subject selection and sampling procedure . For instance, if you propose to conduct interviews, how do you intend to select the sample population? If you are analyzing texts, which texts have you chosen, and why? If you are using statistics, why is this set of statisics being used? If other data sources exist, explain why the data you chose is most appropriate.
• Address potential limitations . Are there any practical limitations that could affect your data collection? How will you attempt to control for potential confounding variables and errors? If your methodology may lead to problems you can anticipate, state this openly and show why pursuing this methodology outweighs the risk of these problems cropping up.

NOTE :  Once you have written all of the elements of the methods section, subsequent revisions should focus on how to present those elements as clearly and as logically as possibly. The description of how you prepared to study the research problem, how you gathered the data, and the protocol for analyzing the data should be organized chronologically. For clarity, when a large amount of detail must be presented, information should be presented in sub-sections according to topic.

III.  Problems to Avoid

Irrelevant Detail The methodology section of your paper should be thorough but to the point. Don’t provide any background information that doesn’t directly help the reader to understand why a particular method was chosen, how the data was gathered or obtained, and how it was analyzed. Unnecessary Explanation of Basic Procedures Remember that you are not writing a how-to guide about a particular method. You should make the assumption that readers possess a basic understanding of how to investigate the research problem on their own and, therefore, you do not have to go into great detail about specific methodological procedures. The focus should be on how you applied a method , not on the mechanics of doing a method. NOTE: An exception to this rule is if you select an unconventional approach to doing the method; if this is the case, be sure to explain why this approach was chosen and how it enhances the overall research process. Problem Blindness It is almost a given that you will encounter problems when collecting or generating your data. Do not ignore these problems or pretend they did not occur. Often, documenting how you overcame obstacles can form an interesting part of the methodology. It demonstrates to the reader that you can provide a cogent rationale for the decisions you made to minimize the impact of any problems that arose. Literature Review Just as the literature review section of your paper provides an overview of sources you have examined while researching a particular topic, the methodology section should cite any sources that informed your choice and application of a particular method [i.e., the choice of a survey should include any citations to the works you used to help construct the survey].

It’s More than Sources of Information! A description of a research study's method should not be confused with a description of the sources of information. Such a list of sources is useful in itself, especially if it is accompanied by an explanation about the selection and use of the sources. The description of the project's methodology complements a list of sources in that it sets forth the organization and interpretation of information emanating from those sources.

Azevedo, L.F. et al. How to Write a Scientific Paper: Writing the Methods Section. Revista Portuguesa de Pneumologia 17 (2011): 232-238; Butin, Dan W. The Education Dissertation A Guide for Practitioner Scholars . Thousand Oaks, CA: Corwin, 2010; Carter, Susan. Structuring Your Research Thesis . New York: Palgrave Macmillan, 2012; Lunenburg, Frederick C. Writing a Successful Thesis or Dissertation: Tips and Strategies for Students in the Social and Behavioral Sciences . Thousand Oaks, CA: Corwin Press, 2008. Methods Section . The Writer’s Handbook. Writing Center. University of Wisconsin, Madison; Writing the Experimental Report: Methods, Results, and Discussion . The Writing Lab and The OWL. Purdue University; Methods and Materials . The Structure, Format, Content, and Style of a Journal-Style Scientific Paper. Department of Biology. Bates College.

Writing Tip

Statistical Designs and Tests? Do Not Fear Them!

Don't avoid using a quantitative approach to analyzing your research problem just because you fear the idea of applying statistical designs and tests. A qualitative approach, such as conducting interviews or content analysis of archival texts, can yield exciting new insights about a research problem, but it should not be undertaken simply because you have a disdain for running a simple regression. A well designed quantitative research study can often be accomplished in very clear and direct ways, whereas, a similar study of a qualitative nature usually requires considerable time to analyze large volumes of data and a tremendous burden to create new paths for analysis where previously no path associated with your research problem had existed.

Another Writing Tip

Knowing the Relationship Between Theories and Methods

There can be multiple meaning associated with the term "theories" and the term "methods" in social sciences research. A helpful way to delineate between them is to understand "theories" as representing different ways of characterizing the social world when you research it and "methods" as representing different ways of generating and analyzing data about that social world. Framed in this way, all empirical social sciences research involves theories and methods, whether they are stated explicitly or not. However, while theories and methods are often related, it is important that, as a researcher, you deliberately separate them in order to avoid your theories playing a disproportionate role in shaping what outcomes your chosen methods produce.

Introspectively engage in an ongoing dialectic between theories and methods to help enable you to use the outcomes from your methods to interrogate and develop new theories, or ways of framing conceptually the research problem. This is how scholarship grows and branches out into new intellectual territory.

Reynolds, R. Larry. Ways of Knowing. Alternative Microeconomics. Part 1, Chapter 3. Boise State University; The Theory-Method Relationship . S-Cool Revision. United Kingdom.

• << Previous: What Is Scholarly vs. Popular?
• Next: Qualitative Methods >>
• Last Updated: Jul 18, 2023 11:58 AM
• URL: https://library.sacredheart.edu/c.php?g=29803
• QuickSearch
• Library Catalog
• Databases A-Z
• Publication Finder
• Course Reserves
• Citation Linker
• Digital Commons
• Our Website

Research Support

• Ask a Librarian
• Appointments
• Interlibrary Loan (ILL)
• Research Guides
• Databases by Subject
• Citation Help

Using the Library

• Reserve a Group Study Room
• Renew Books
• Honors Study Rooms
• Off-Campus Access
• Library Policies
• Library Technology

User Information

• Grad Students
• Online Students
• COVID-19 Updates
• Staff Directory
• News & Announcements
• Library Newsletter

My Accounts

• Interlibrary Loan
• Staff Site Login

FIND US ON  

• Affiliate Program

• UNITED STATES
• 台灣 (TAIWAN)
• TÜRKIYE (TURKEY)
• Academic Editing Services
• - Research Paper
• - Journal Manuscript
• - Dissertation
• - College & University Assignments
• Admissions Editing Services
• - Application Essay
• - Personal Statement
• - Recommendation Letter
• - Cover Letter
• - CV/Resume
• Business Editing Services
• - Business Documents
• - Report & Brochure
• - Website & Blog
• Writer Editing Services
• - Script & Screenplay
• Our Editors
• Client Reviews
• Editing & Proofreading Prices
• Wordvice Points
• Partner Discount
• Plagiarism Checker
• APA Citation Generator
• MLA Citation Generator
• Chicago Citation Generator
• Vancouver Citation Generator
• - APA Style
• - MLA Style
• - Chicago Style
• - Vancouver Style
• Writing & Editing Guide
• Academic Resources
• Admissions Resources

How to Write a Methods Section for a Research Paper

A common piece of advice for authors preparing their first journal article for publication is to start with the methods section: just list everything that was done and go from there. While that might seem like a very practical approach to a first draft, if you do this without a clear outline and a story in mind, you can easily end up with journal manuscript sections that are not logically related to each other. 

Since the methods section constitutes the core of your paper, no matter when you write it, you need to use it to guide the reader carefully through your story from beginning to end without leaving questions unanswered. Missing or confusing details in this section will likely lead to early rejection of your manuscript or unnecessary back-and-forth with the reviewers until eventual publication. Here, you will find some useful tips on how to make your methods section the logical foundation of your research paper.

Not just a list of experiments and methods

While your introduction section provides the reader with the necessary background to understand your rationale and research question (and, depending on journal format and your personal preference, might already summarize the results), the methods section explains what exactly you did and how you did it. The point of this section is not to list all the boring details just for the sake of completeness. The purpose of the methods sections is to enable the reader to replicate exactly what you did, verify or corroborate your results, or maybe find that there are factors you did not consider or that are more relevant than expected. 

To make this section as easy to read as possible, you must clearly connect it to the information you provide in the introduction section before and the results section after, it needs to have a clear structure (chronologically or according to topics), and you need to present your results according to the same structure or topics later in the manuscript. There are also official guidelines and journal instructions to follow and ethical issues to avoid to ensure that your manuscript can quickly reach the publication stage.

Table of Contents:

• General Methods Structure: What is Your Story? 
• What Methods Should You Report (and Leave Out)? 
• Details Frequently Missing from the Methods Section

More Journal Guidelines to Consider 

• Accurate and Appropriate Language in the Methods

General Methods Section Structure: What Is Your Story? 

You might have conducted a number of experiments, maybe also a pilot before the main study to determine some specific factors or a follow-up experiment to clarify unclear details later in the process. Throwing all of these into your methods section, however, might not help the reader understand how everything is connected and how useful and appropriate your methodological approach is to investigate your specific research question. You therefore need to first come up with a clear outline and decide what to report and how to present that to the reader.

The first (and very important) decision to make is whether you present your experiments chronologically (e.g., Experiment 1, Experiment 2, Experiment 3… ), and guide the reader through every step of the process, or if you organize everything according to subtopics (e.g., Behavioral measures, Structural imaging markers, Functional imaging markers… ). In both cases, you need to use clear subheaders for the different subsections of your methods, and, very importantly, follow the same structure or focus on the same topics/measures in the results section so that the reader can easily follow along (see the two examples below).

If you are in doubt which way of organizing your experiments is better for your study, just ask yourself the following questions:

• Does the reader need to know the timeline of your study? 
• Is it relevant that one experiment was conducted first, because the outcome of this experiment determined the stimuli or factors that went into the next?
• Did the results of your first experiment leave important questions open that you addressed in an additional experiment (that was maybe not planned initially)?
• Is the answer to all of these questions “no”? Then organizing your methods section according to topics of interest might be the more logical choice.

If you think your timeline, protocol, or setup might be confusing or difficult for the reader to grasp, consider adding a graphic, flow diagram, decision tree, or table as a visual aid.

What Methods Should You Report (and Leave Out)?

The answer to this question is quite simple–you need to report everything that another researcher needs to know to be able to replicate your study. Just imagine yourself reading your methods section in the future and trying to set up the same experiments again without prior knowledge. You would probably need to ask questions such as:

• Where did you conduct your experiments (e.g., in what kind of room, under what lighting or temperature conditions, if those are relevant)? 
• What devices did you use? Are there specific settings to report?
• What specific software (and version of that software) did you use?
• How did you find and select your participants?
• How did you assign participants into groups?  
• Did you exclude participants from the analysis? Why and how?
• Where did your reagents or antibodies come from? Can you provide a Research Resource Identifier (RRID) ?
• Did you make your stimuli yourself or did you get them from somewhere?
• Are the stimuli you used available for other researchers?
• What kind of questionnaires did you use? Have they been validated?
• How did you analyze your data? What level of significance did you use?
• Were there any technical issues and did you have to adjust protocols?

Note that for every experimental detail you provide, you need to tell the reader (briefly) why you used this type of stimulus/this group of participants/these specific amounts of reagents. If there is earlier published research reporting the same methods, cite those studies. If you did pilot experiments to determine those details, describe the procedures and the outcomes of these experiments. If you made assumptions about the suitability of something based on the literature and common practice at your institution, then explain that to the reader.

In a nutshell, established methods need to be cited, and new methods need to be clearly described and briefly justified. However, if the fact that you use a new approach or a method that is not traditionally used for the data or phenomenon you study is one of the main points of your study (and maybe already reflected in the title of your article), then you need to explain your rationale for doing so in the introduction already and discuss it in more detail in the discussion section .

Note that you also need to explain your statistical analyses at the end of your methods section. You present the results of these analyses later, in the results section of your paper, but you need to show the reader in the methods section already that your approach is either well-established or valid, even if it is new or unusual. 

When it comes to the question of what details you should leave out, the answer is equally simple ‒ everything that you would not need to replicate your study in the future. If the educational background of your participants is listed in your institutional database but is not relevant to your study outcome, then don’t include that. Other things you should not include in the methods section:

• Background information that you already presented in the introduction section.
• In-depth comparisons of different methods ‒ these belong in the discussion section.
• Results, unless you summarize outcomes of pilot experiments that helped you determine factors for your main experiment.

Also, make sure your subheadings are as clear as possible, suit the structure you chose for your methods section, and are in line with the target journal guidelines. If you studied a disease intervention in human participants, then your methods section could look similar to this:

Since the main point of interest here are your patient-centered outcome variables, you would center your results section on these as well and choose your headers accordingly (e.g., Patient characteristics, Baseline evaluation, Outcome variable 1, Outcome variable 2, Drop-out rate ). 

If, instead, you did a series of visual experiments investigating the perception of faces including a pilot experiment to create the stimuli for your actual study, you would need to structure your methods section in a very different way, maybe like this:

Since here the analysis and outcome of the pilot experiment are already described in the methods section (as the basis for the main experimental setup and procedure), you do not have to mention it again in the results section. Instead, you could choose the two main experiments to structure your results section ( Discrimination and classification, Familiarization and adaptation ), or divide the results into all your test measures and/or potential interactions you described in the methods section (e.g., Discrimination performance, Classification performance, Adaptation aftereffects, Correlation analysis ).

Details Commonly Missing from the Methods Section

Manufacturer information.

For laboratory or technical equipment, you need to provide the model, name of the manufacturer, and company’s location. The usual format for these details is the product name (company name, city, state) for US-based manufacturers and the product name (company name, city/town, country) for companies outside the US.

Sample size and power estimation

Power and sample size estimations are measures for how many patients or participants are needed in a study in order to detect statistical significance and draw meaningful conclusions from the results. Outside of the medical field, studies are sometimes still conducted with a “the more the better” approach in mind, but since many journals now ask for those details, it is better to not skip this important step.

Ethical guidelines and approval

In addition to describing what you did, you also need to assure the editor and reviewers that your methods and protocols followed all relevant ethical standards and guidelines. This includes applying for approval at your local or national ethics committee, providing the name or location of that committee as well as the approval reference number you received, and, if you studied human participants, a statement that participants were informed about all relevant experimental details in advance and signed consent forms before the start of the study. For animal studies, you usually need to provide a statement that all procedures included in your research were in line with the Declaration of Helsinki. Make sure you check the target journal guidelines carefully, as these statements sometimes need to be placed at the end of the main article text rather than in the method section.

Structure & word limitations

While many journals simply follow the usual style guidelines (e.g., APA for the social sciences and psychology, AMA for medical research) and let you choose the headers of your method section according to your preferred structure and focus, some have precise guidelines and strict limitations, for example, on manuscript length and the maximum number of subsections or header levels. Make sure you read the instructions of your target journal carefully and restructure your method section if necessary before submission. If the journal does not give you enough space to include all the details that you deem necessary, then you can usually submit additional details as “supplemental” files and refer to those in the main text where necessary.

Standardized checklists

In addition to ethical guidelines and approval, journals also often ask you to submit one of the official standardized checklists for different study types to ensure all essential details are included in your manuscript. For example, there are checklists for randomized clinical trials, CONSORT (Consolidated Standards of Reporting Trials) , cohort, case-control, cross‐sectional studies, STROBE (STrengthening the Reporting of OBservational studies in Epidemiology ), diagnostic accuracy, STARD (STAndards for the Reporting of Diagnostic accuracy studies) , systematic reviews and meta‐analyses PRISMA (Preferred Reporting Items for Systematic reviews and Meta‐Analyses) , and Case reports, CARE (CAse REport) .

Make sure you check if the manuscript uses a single- or double-blind review procedure , and delete all information that might allow a reviewer to guess where the authors are located from the manuscript text if necessary. This means that your method section cannot list the name and location of your institution, the names of researchers who conducted specific tests, or the name of your institutional ethics committee.  

Accurate and Appropriate Language in the Methods Section

Like all sections of your research paper, your method section needs to be written in an academic tone . That means it should be formal, vague expressions and colloquial language need to be avoided, and you need to correctly cite all your sources. If you describe human participants in your method section then you should be especially careful about your choice of words. For example, “participants” sounds more respectful than “subjects,” and patient-first language, that is, “patients with cancer,” is considered more appropriate than “cancer patients” by many journals.

Passive voice is often considered the standard for research papers, but it is completely fine to mix passive and active voice, even in the method section, to make your text as clear and concise as possible. Use the simple past tense to describe what you did, and the present tense when you refer to diagrams or tables. Have a look at this article if you need more general input on which verb tenses to use in a research paper . 

Lastly, make sure you label all the standard tests and questionnaires you use correctly (look up the original publication when in doubt) and spell genes and proteins according to the common databases for the species you studied, such as the HUGO Gene Nomenclature Committee database for human studies .  

Visit Wordvice AI’s AI Text Editor to receive a free grammar check and English editing services (including manuscript editing , paper editing , and dissertation editing ) before submitting your manuscript to journal editors.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

4 Writing the Materials and Methods (Methodology) Section

The Materials and Methods section briefly describes how you did your research. In other words, what did you do to answer your research question? If there were materials used for the research or materials experimented on you list them in this section. You also describe how you did the research or experiment. The key to a methodology is that another person must be able to replicate your research—follow the steps you take. For example if you used the internet to do a search it is not enough to say you “searched the internet.” A reader would need to know which search engine and what key words you used.

Open this section by describing the overall approach you took or the materials used. Then describe to the readers step-by-step the methods you used including any data analysis performed. See Fig. 2.5 below for an example of materials and methods section.

Writing tips:

• Explain procedures, materials, and equipment used
• Example: “We used an x-ray fluorescence spectrometer to analyze major and trace elements in the mystery mineral samples.”
• Order events chronologically, perhaps with subheadings (Field work, Lab Analysis, Statistical Models)
• Use past tense (you did X, Y, Z)
• Quantify measurements
• Include results in the methods! It’s easy to make this mistake!
• Example: “W e turned on the machine and loaded in our samples, then calibrated the instrument and pushed the start button and waited one hour. . . .”

Technical Writing @ SLCC Copyright © 2020 by Department of English, Linguistics, and Writing Studies at SLCC is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License , except where otherwise noted.

Share This Book

How to write the methods section of a research paper

Affiliation.

• 1 Respiratory Care Services, San Francisco General Hospital, NH:GA-2, 1001 Potrero Avenue, San Francisco, CA 94110, USA. [email protected]
• PMID: 15447808

The methods section of a research paper provides the information by which a study's validity is judged. Therefore, it requires a clear and precise description of how an experiment was done, and the rationale for why specific experimental procedures were chosen. The methods section should describe what was done to answer the research question, describe how it was done, justify the experimental design, and explain how the results were analyzed. Scientific writing is direct and orderly. Therefore, the methods section structure should: describe the materials used in the study, explain how the materials were prepared for the study, describe the research protocol, explain how measurements were made and what calculations were performed, and state which statistical tests were done to analyze the data. Once all elements of the methods section are written, subsequent drafts should focus on how to present those elements as clearly and logically as possibly. The description of preparations, measurements, and the protocol should be organized chronologically. For clarity, when a large amount of detail must be presented, information should be presented in sub-sections according to topic. Material in each section should be organized by topic from most to least important.

• Biomedical Research*
• Research Design
• Writing* / standards

• Privacy Policy

Buy Me a Coffee

Home » Research Paper – Structure, Examples and Writing Guide

Research Paper – Structure, Examples and Writing Guide

Table of Contents

Research Paper

Definition:

Research Paper is a written document that presents the author’s original research, analysis, and interpretation of a specific topic or issue.

It is typically based on Empirical Evidence, and may involve qualitative or quantitative research methods, or a combination of both. The purpose of a research paper is to contribute new knowledge or insights to a particular field of study, and to demonstrate the author’s understanding of the existing literature and theories related to the topic.

Structure of Research Paper

The structure of a research paper typically follows a standard format, consisting of several sections that convey specific information about the research study. The following is a detailed explanation of the structure of a research paper:

The title page contains the title of the paper, the name(s) of the author(s), and the affiliation(s) of the author(s). It also includes the date of submission and possibly, the name of the journal or conference where the paper is to be published.

The abstract is a brief summary of the research paper, typically ranging from 100 to 250 words. It should include the research question, the methods used, the key findings, and the implications of the results. The abstract should be written in a concise and clear manner to allow readers to quickly grasp the essence of the research.

Introduction

The introduction section of a research paper provides background information about the research problem, the research question, and the research objectives. It also outlines the significance of the research, the research gap that it aims to fill, and the approach taken to address the research question. Finally, the introduction section ends with a clear statement of the research hypothesis or research question.

Literature Review

The literature review section of a research paper provides an overview of the existing literature on the topic of study. It includes a critical analysis and synthesis of the literature, highlighting the key concepts, themes, and debates. The literature review should also demonstrate the research gap and how the current study seeks to address it.

The methods section of a research paper describes the research design, the sample selection, the data collection and analysis procedures, and the statistical methods used to analyze the data. This section should provide sufficient detail for other researchers to replicate the study.

The results section presents the findings of the research, using tables, graphs, and figures to illustrate the data. The findings should be presented in a clear and concise manner, with reference to the research question and hypothesis.

The discussion section of a research paper interprets the findings and discusses their implications for the research question, the literature review, and the field of study. It should also address the limitations of the study and suggest future research directions.

The conclusion section summarizes the main findings of the study, restates the research question and hypothesis, and provides a final reflection on the significance of the research.

The references section provides a list of all the sources cited in the paper, following a specific citation style such as APA, MLA or Chicago.

How to Write Research Paper

You can write Research Paper by the following guide:

• Choose a Topic: The first step is to select a topic that interests you and is relevant to your field of study. Brainstorm ideas and narrow down to a research question that is specific and researchable.
• Conduct a Literature Review: The literature review helps you identify the gap in the existing research and provides a basis for your research question. It also helps you to develop a theoretical framework and research hypothesis.
• Develop a Thesis Statement : The thesis statement is the main argument of your research paper. It should be clear, concise and specific to your research question.
• Plan your Research: Develop a research plan that outlines the methods, data sources, and data analysis procedures. This will help you to collect and analyze data effectively.
• Collect and Analyze Data: Collect data using various methods such as surveys, interviews, observations, or experiments. Analyze data using statistical tools or other qualitative methods.
• Organize your Paper : Organize your paper into sections such as Introduction, Literature Review, Methods, Results, Discussion, and Conclusion. Ensure that each section is coherent and follows a logical flow.
• Write your Paper : Start by writing the introduction, followed by the literature review, methods, results, discussion, and conclusion. Ensure that your writing is clear, concise, and follows the required formatting and citation styles.
• Edit and Proofread your Paper: Review your paper for grammar and spelling errors, and ensure that it is well-structured and easy to read. Ask someone else to review your paper to get feedback and suggestions for improvement.
• Cite your Sources: Ensure that you properly cite all sources used in your research paper. This is essential for giving credit to the original authors and avoiding plagiarism.

Research Paper Example

Note : The below example research paper is for illustrative purposes only and is not an actual research paper. Actual research papers may have different structures, contents, and formats depending on the field of study, research question, data collection and analysis methods, and other factors. Students should always consult with their professors or supervisors for specific guidelines and expectations for their research papers.

Research Paper Example sample for Students:

Title: The Impact of Social Media on Mental Health among Young Adults

Abstract: This study aims to investigate the impact of social media use on the mental health of young adults. A literature review was conducted to examine the existing research on the topic. A survey was then administered to 200 university students to collect data on their social media use, mental health status, and perceived impact of social media on their mental health. The results showed that social media use is positively associated with depression, anxiety, and stress. The study also found that social comparison, cyberbullying, and FOMO (Fear of Missing Out) are significant predictors of mental health problems among young adults.

Introduction: Social media has become an integral part of modern life, particularly among young adults. While social media has many benefits, including increased communication and social connectivity, it has also been associated with negative outcomes, such as addiction, cyberbullying, and mental health problems. This study aims to investigate the impact of social media use on the mental health of young adults.

Literature Review: The literature review highlights the existing research on the impact of social media use on mental health. The review shows that social media use is associated with depression, anxiety, stress, and other mental health problems. The review also identifies the factors that contribute to the negative impact of social media, including social comparison, cyberbullying, and FOMO.

Methods : A survey was administered to 200 university students to collect data on their social media use, mental health status, and perceived impact of social media on their mental health. The survey included questions on social media use, mental health status (measured using the DASS-21), and perceived impact of social media on their mental health. Data were analyzed using descriptive statistics and regression analysis.

Results : The results showed that social media use is positively associated with depression, anxiety, and stress. The study also found that social comparison, cyberbullying, and FOMO are significant predictors of mental health problems among young adults.

Discussion : The study’s findings suggest that social media use has a negative impact on the mental health of young adults. The study highlights the need for interventions that address the factors contributing to the negative impact of social media, such as social comparison, cyberbullying, and FOMO.

Conclusion : In conclusion, social media use has a significant impact on the mental health of young adults. The study’s findings underscore the need for interventions that promote healthy social media use and address the negative outcomes associated with social media use. Future research can explore the effectiveness of interventions aimed at reducing the negative impact of social media on mental health. Additionally, longitudinal studies can investigate the long-term effects of social media use on mental health.

Limitations : The study has some limitations, including the use of self-report measures and a cross-sectional design. The use of self-report measures may result in biased responses, and a cross-sectional design limits the ability to establish causality.

Implications: The study’s findings have implications for mental health professionals, educators, and policymakers. Mental health professionals can use the findings to develop interventions that address the negative impact of social media use on mental health. Educators can incorporate social media literacy into their curriculum to promote healthy social media use among young adults. Policymakers can use the findings to develop policies that protect young adults from the negative outcomes associated with social media use.

References :

• Twenge, J. M., & Campbell, W. K. (2019). Associations between screen time and lower psychological well-being among children and adolescents: Evidence from a population-based study. Preventive medicine reports, 15, 100918.
• Primack, B. A., Shensa, A., Escobar-Viera, C. G., Barrett, E. L., Sidani, J. E., Colditz, J. B., … & James, A. E. (2017). Use of multiple social media platforms and symptoms of depression and anxiety: A nationally-representative study among US young adults. Computers in Human Behavior, 69, 1-9.
• Van der Meer, T. G., & Verhoeven, J. W. (2017). Social media and its impact on academic performance of students. Journal of Information Technology Education: Research, 16, 383-398.

Appendix : The survey used in this study is provided below.

Social Media and Mental Health Survey

• How often do you use social media per day?
• Less than 30 minutes
• 30 minutes to 1 hour
• 1 to 2 hours
• 2 to 4 hours
• More than 4 hours
• Which social media platforms do you use?
• Others (Please specify)
• How often do you experience the following on social media?
• Social comparison (comparing yourself to others)
• Cyberbullying
• Fear of Missing Out (FOMO)
• Have you ever experienced any of the following mental health problems in the past month?
• Do you think social media use has a positive or negative impact on your mental health?
• Very positive
• Somewhat positive
• Somewhat negative
• Very negative
• In your opinion, which factors contribute to the negative impact of social media on mental health?
• Social comparison
• In your opinion, what interventions could be effective in reducing the negative impact of social media on mental health?
• Education on healthy social media use
• Counseling for mental health problems caused by social media
• Social media detox programs
• Regulation of social media use

Thank you for your participation!

Applications of Research Paper

Research papers have several applications in various fields, including:

• Advancing knowledge: Research papers contribute to the advancement of knowledge by generating new insights, theories, and findings that can inform future research and practice. They help to answer important questions, clarify existing knowledge, and identify areas that require further investigation.
• Informing policy: Research papers can inform policy decisions by providing evidence-based recommendations for policymakers. They can help to identify gaps in current policies, evaluate the effectiveness of interventions, and inform the development of new policies and regulations.
• Improving practice: Research papers can improve practice by providing evidence-based guidance for professionals in various fields, including medicine, education, business, and psychology. They can inform the development of best practices, guidelines, and standards of care that can improve outcomes for individuals and organizations.
• Educating students : Research papers are often used as teaching tools in universities and colleges to educate students about research methods, data analysis, and academic writing. They help students to develop critical thinking skills, research skills, and communication skills that are essential for success in many careers.
• Fostering collaboration: Research papers can foster collaboration among researchers, practitioners, and policymakers by providing a platform for sharing knowledge and ideas. They can facilitate interdisciplinary collaborations and partnerships that can lead to innovative solutions to complex problems.

When to Write Research Paper

Research papers are typically written when a person has completed a research project or when they have conducted a study and have obtained data or findings that they want to share with the academic or professional community. Research papers are usually written in academic settings, such as universities, but they can also be written in professional settings, such as research organizations, government agencies, or private companies.

Here are some common situations where a person might need to write a research paper:

• For academic purposes: Students in universities and colleges are often required to write research papers as part of their coursework, particularly in the social sciences, natural sciences, and humanities. Writing research papers helps students to develop research skills, critical thinking skills, and academic writing skills.
• For publication: Researchers often write research papers to publish their findings in academic journals or to present their work at academic conferences. Publishing research papers is an important way to disseminate research findings to the academic community and to establish oneself as an expert in a particular field.
• To inform policy or practice : Researchers may write research papers to inform policy decisions or to improve practice in various fields. Research findings can be used to inform the development of policies, guidelines, and best practices that can improve outcomes for individuals and organizations.
• To share new insights or ideas: Researchers may write research papers to share new insights or ideas with the academic or professional community. They may present new theories, propose new research methods, or challenge existing paradigms in their field.

Purpose of Research Paper

The purpose of a research paper is to present the results of a study or investigation in a clear, concise, and structured manner. Research papers are written to communicate new knowledge, ideas, or findings to a specific audience, such as researchers, scholars, practitioners, or policymakers. The primary purposes of a research paper are:

• To contribute to the body of knowledge : Research papers aim to add new knowledge or insights to a particular field or discipline. They do this by reporting the results of empirical studies, reviewing and synthesizing existing literature, proposing new theories, or providing new perspectives on a topic.
• To inform or persuade: Research papers are written to inform or persuade the reader about a particular issue, topic, or phenomenon. They present evidence and arguments to support their claims and seek to persuade the reader of the validity of their findings or recommendations.
• To advance the field: Research papers seek to advance the field or discipline by identifying gaps in knowledge, proposing new research questions or approaches, or challenging existing assumptions or paradigms. They aim to contribute to ongoing debates and discussions within a field and to stimulate further research and inquiry.
• To demonstrate research skills: Research papers demonstrate the author’s research skills, including their ability to design and conduct a study, collect and analyze data, and interpret and communicate findings. They also demonstrate the author’s ability to critically evaluate existing literature, synthesize information from multiple sources, and write in a clear and structured manner.

Characteristics of Research Paper

Research papers have several characteristics that distinguish them from other forms of academic or professional writing. Here are some common characteristics of research papers:

• Evidence-based: Research papers are based on empirical evidence, which is collected through rigorous research methods such as experiments, surveys, observations, or interviews. They rely on objective data and facts to support their claims and conclusions.
• Structured and organized: Research papers have a clear and logical structure, with sections such as introduction, literature review, methods, results, discussion, and conclusion. They are organized in a way that helps the reader to follow the argument and understand the findings.
• Formal and objective: Research papers are written in a formal and objective tone, with an emphasis on clarity, precision, and accuracy. They avoid subjective language or personal opinions and instead rely on objective data and analysis to support their arguments.
• Citations and references: Research papers include citations and references to acknowledge the sources of information and ideas used in the paper. They use a specific citation style, such as APA, MLA, or Chicago, to ensure consistency and accuracy.
• Peer-reviewed: Research papers are often peer-reviewed, which means they are evaluated by other experts in the field before they are published. Peer-review ensures that the research is of high quality, meets ethical standards, and contributes to the advancement of knowledge in the field.
• Objective and unbiased: Research papers strive to be objective and unbiased in their presentation of the findings. They avoid personal biases or preconceptions and instead rely on the data and analysis to draw conclusions.

Advantages of Research Paper

Research papers have many advantages, both for the individual researcher and for the broader academic and professional community. Here are some advantages of research papers:

• Contribution to knowledge: Research papers contribute to the body of knowledge in a particular field or discipline. They add new information, insights, and perspectives to existing literature and help advance the understanding of a particular phenomenon or issue.
• Opportunity for intellectual growth: Research papers provide an opportunity for intellectual growth for the researcher. They require critical thinking, problem-solving, and creativity, which can help develop the researcher’s skills and knowledge.
• Career advancement: Research papers can help advance the researcher’s career by demonstrating their expertise and contributions to the field. They can also lead to new research opportunities, collaborations, and funding.
• Academic recognition: Research papers can lead to academic recognition in the form of awards, grants, or invitations to speak at conferences or events. They can also contribute to the researcher’s reputation and standing in the field.
• Impact on policy and practice: Research papers can have a significant impact on policy and practice. They can inform policy decisions, guide practice, and lead to changes in laws, regulations, or procedures.
• Advancement of society: Research papers can contribute to the advancement of society by addressing important issues, identifying solutions to problems, and promoting social justice and equality.

Limitations of Research Paper

Research papers also have some limitations that should be considered when interpreting their findings or implications. Here are some common limitations of research papers:

• Limited generalizability: Research findings may not be generalizable to other populations, settings, or contexts. Studies often use specific samples or conditions that may not reflect the broader population or real-world situations.
• Potential for bias : Research papers may be biased due to factors such as sample selection, measurement errors, or researcher biases. It is important to evaluate the quality of the research design and methods used to ensure that the findings are valid and reliable.
• Ethical concerns: Research papers may raise ethical concerns, such as the use of vulnerable populations or invasive procedures. Researchers must adhere to ethical guidelines and obtain informed consent from participants to ensure that the research is conducted in a responsible and respectful manner.
• Limitations of methodology: Research papers may be limited by the methodology used to collect and analyze data. For example, certain research methods may not capture the complexity or nuance of a particular phenomenon, or may not be appropriate for certain research questions.
• Publication bias: Research papers may be subject to publication bias, where positive or significant findings are more likely to be published than negative or non-significant findings. This can skew the overall findings of a particular area of research.
• Time and resource constraints: Research papers may be limited by time and resource constraints, which can affect the quality and scope of the research. Researchers may not have access to certain data or resources, or may be unable to conduct long-term studies due to practical limitations.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

How to Cite Research Paper – All Formats and...

Data Collection – Methods Types and Examples

Delimitations in Research – Types, Examples and...

Research Paper Format – Types, Examples and...

Research Process – Steps, Examples and Tips

Research Design – Types, Methods and Examples

• Locations and Hours
• UCLA Library
• Research Guides
• Research Tips and Tools

Advanced Research Methods

Writing the research paper.

• What Is Research?
• Library Research
• Writing a Research Proposal

Before Writing the Paper

Methods, thesis, and hypothesis, clarity, precision, and academic expression, format your paper, typical problems, a few suggestions, avoid plagiarism.

• Presenting the Research Paper
• Try to find a subject that really interests you.
• While you explore the topic, narrow or broaden your target and focus on something that gives the most promising results.
• Don't choose a huge subject if you have to write a 3 page long paper, and broaden your topic sufficiently if you have to submit at least 25 pages.
• Consult your class instructor (and your classmates) about the topic.
• Find primary and secondary sources in the library.
• Read and critically analyse them.
• Take notes.
• Compile surveys, collect data, gather materials for quantitative analysis (if these are good methods to investigate the topic more deeply).
• Come up with new ideas about the topic. Try to formulate your ideas in a few sentences.
• Review your notes and other materials and enrich the outline.
• Try to estimate how long the individual parts will be.
• Do others understand what you want to say?
• Do they accept it as new knowledge or relevant and important for a paper?
• Do they agree that your thoughts will result in a successful paper?
• Qualitative: gives answers on questions (how, why, when, who, what, etc.) by investigating an issue
• Quantitative:requires data and the analysis of data as well
• the essence, the point of the research paper in one or two sentences.
• a statement that can be proved or disproved.
• Be specific.
• Avoid ambiguity.
• Use predominantly the active voice, not the passive.
• Deal with one issue in one paragraph.
• Be accurate.
• Double-check your data, references, citations and statements.

Academic Expression

• Don't use familiar style or colloquial/slang expressions.
• Write in full sentences.
• Check the meaning of the words if you don't know exactly what they mean.
• Avoid metaphors.
• Almost the rough content of every paragraph.
• The order of the various topics in your paper.
• On the basis of the outline, start writing a part by planning the content, and then write it down.
• Put a visible mark (which you will later delete) where you need to quote a source, and write in the citation when you finish writing that part or a bigger part.
• Does the text make sense?
• Could you explain what you wanted?
• Did you write good sentences?
• Is there something missing?
• Check the spelling.
• Complete the citations, bring them in standard format.

Use the guidelines that your instructor requires (MLA, Chicago, APA, Turabian, etc.).

• Adjust margins, spacing, paragraph indentation, place of page numbers, etc.
• Standardize the bibliography or footnotes according to the guidelines.

• EndNote and EndNote Basic by UCLA Library Last Updated Mar 18, 2024 667 views this year
• Zotero by UCLA Library Last Updated Jan 18, 2023 507 views this year

(Based on English Composition 2 from Illinois Valley Community College):

• Weak organization
• Poor support and development of ideas
• Weak use of secondary sources
• Excessive errors
• Stylistic weakness

When collecting materials, selecting research topic, and writing the paper:

• Be systematic and organized (e.g. keep your bibliography neat and organized; write your notes in a neat way, so that you can find them later on.
• Use your critical thinking ability when you read.
• Write down your thoughts (so that you can reconstruct them later).
• Stop when you have a really good idea and think about whether you could enlarge it to a whole research paper. If yes, take much longer notes.
• When you write down a quotation or summarize somebody else's thoughts in your notes or in the paper, cite the source (i.e. write down the author, title, publication place, year, page number).
• If you quote or summarize a thought from the internet, cite the internet source.
• Write an outline that is detailed enough to remind you about the content.
• Read your paper for yourself or, preferably, somebody else. 
• When you finish writing, check the spelling;
• Use the citation form (MLA, Chicago, or other) that your instructor requires and use it everywhere.

Plagiarism : somebody else's words or ideas presented without citation by an author

• Cite your source every time when you quote a part of somebody's work.
• Cite your source  every time when you summarize a thought from somebody's work.
• Cite your source  every time when you use a source (quote or summarize) from the Internet.

Consult the Citing Sources research guide for further details.

• << Previous: Writing a Research Proposal
• Next: Presenting the Research Paper >>
• Last Updated: Jan 4, 2024 12:24 PM
• URL: https://guides.library.ucla.edu/research-methods

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Neurol Res Pract

How to use and assess qualitative research methods

Loraine busetto.

1 Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120 Heidelberg, Germany

Wolfgang Wick

2 Clinical Cooperation Unit Neuro-Oncology, German Cancer Research Center, Heidelberg, Germany

Christoph Gumbinger

Associated data.

Not applicable.

This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions, and focussing on intervention improvement. The most common methods of data collection are document study, (non-) participant observations, semi-structured interviews and focus groups. For data analysis, field-notes and audio-recordings are transcribed into protocols and transcripts, and coded using qualitative data management software. Criteria such as checklists, reflexivity, sampling strategies, piloting, co-coding, member-checking and stakeholder involvement can be used to enhance and assess the quality of the research conducted. Using qualitative in addition to quantitative designs will equip us with better tools to address a greater range of research problems, and to fill in blind spots in current neurological research and practice.

The aim of this paper is to provide an overview of qualitative research methods, including hands-on information on how they can be used, reported and assessed. This article is intended for beginning qualitative researchers in the health sciences as well as experienced quantitative researchers who wish to broaden their understanding of qualitative research.

What is qualitative research?

Qualitative research is defined as “the study of the nature of phenomena”, including “their quality, different manifestations, the context in which they appear or the perspectives from which they can be perceived” , but excluding “their range, frequency and place in an objectively determined chain of cause and effect” [ 1 ]. This formal definition can be complemented with a more pragmatic rule of thumb: qualitative research generally includes data in form of words rather than numbers [ 2 ].

Why conduct qualitative research?

Because some research questions cannot be answered using (only) quantitative methods. For example, one Australian study addressed the issue of why patients from Aboriginal communities often present late or not at all to specialist services offered by tertiary care hospitals. Using qualitative interviews with patients and staff, it found one of the most significant access barriers to be transportation problems, including some towns and communities simply not having a bus service to the hospital [ 3 ]. A quantitative study could have measured the number of patients over time or even looked at possible explanatory factors – but only those previously known or suspected to be of relevance. To discover reasons for observed patterns, especially the invisible or surprising ones, qualitative designs are needed.

While qualitative research is common in other fields, it is still relatively underrepresented in health services research. The latter field is more traditionally rooted in the evidence-based-medicine paradigm, as seen in " research that involves testing the effectiveness of various strategies to achieve changes in clinical practice, preferably applying randomised controlled trial study designs (...) " [ 4 ]. This focus on quantitative research and specifically randomised controlled trials (RCT) is visible in the idea of a hierarchy of research evidence which assumes that some research designs are objectively better than others, and that choosing a "lesser" design is only acceptable when the better ones are not practically or ethically feasible [ 5 , 6 ]. Others, however, argue that an objective hierarchy does not exist, and that, instead, the research design and methods should be chosen to fit the specific research question at hand – "questions before methods" [ 2 , 7 – 9 ]. This means that even when an RCT is possible, some research problems require a different design that is better suited to addressing them. Arguing in JAMA, Berwick uses the example of rapid response teams in hospitals, which he describes as " a complex, multicomponent intervention – essentially a process of social change" susceptible to a range of different context factors including leadership or organisation history. According to him, "[in] such complex terrain, the RCT is an impoverished way to learn. Critics who use it as a truth standard in this context are incorrect" [ 8 ] . Instead of limiting oneself to RCTs, Berwick recommends embracing a wider range of methods , including qualitative ones, which for "these specific applications, (...) are not compromises in learning how to improve; they are superior" [ 8 ].

Research problems that can be approached particularly well using qualitative methods include assessing complex multi-component interventions or systems (of change), addressing questions beyond “what works”, towards “what works for whom when, how and why”, and focussing on intervention improvement rather than accreditation [ 7 , 9 – 12 ]. Using qualitative methods can also help shed light on the “softer” side of medical treatment. For example, while quantitative trials can measure the costs and benefits of neuro-oncological treatment in terms of survival rates or adverse effects, qualitative research can help provide a better understanding of patient or caregiver stress, visibility of illness or out-of-pocket expenses.

How to conduct qualitative research?

Given that qualitative research is characterised by flexibility, openness and responsivity to context, the steps of data collection and analysis are not as separate and consecutive as they tend to be in quantitative research [ 13 , 14 ]. As Fossey puts it : “sampling, data collection, analysis and interpretation are related to each other in a cyclical (iterative) manner, rather than following one after another in a stepwise approach” [ 15 ]. The researcher can make educated decisions with regard to the choice of method, how they are implemented, and to which and how many units they are applied [ 13 ]. As shown in Fig.  1 , this can involve several back-and-forth steps between data collection and analysis where new insights and experiences can lead to adaption and expansion of the original plan. Some insights may also necessitate a revision of the research question and/or the research design as a whole. The process ends when saturation is achieved, i.e. when no relevant new information can be found (see also below: sampling and saturation). For reasons of transparency, it is essential for all decisions as well as the underlying reasoning to be well-documented.

Iterative research process

While it is not always explicitly addressed, qualitative methods reflect a different underlying research paradigm than quantitative research (e.g. constructivism or interpretivism as opposed to positivism). The choice of methods can be based on the respective underlying substantive theory or theoretical framework used by the researcher [ 2 ].

Data collection

The methods of qualitative data collection most commonly used in health research are document study, observations, semi-structured interviews and focus groups [ 1 , 14 , 16 , 17 ].

Document study

Document study (also called document analysis) refers to the review by the researcher of written materials [ 14 ]. These can include personal and non-personal documents such as archives, annual reports, guidelines, policy documents, diaries or letters.

Observations

Observations are particularly useful to gain insights into a certain setting and actual behaviour – as opposed to reported behaviour or opinions [ 13 ]. Qualitative observations can be either participant or non-participant in nature. In participant observations, the observer is part of the observed setting, for example a nurse working in an intensive care unit [ 18 ]. In non-participant observations, the observer is “on the outside looking in”, i.e. present in but not part of the situation, trying not to influence the setting by their presence. Observations can be planned (e.g. for 3 h during the day or night shift) or ad hoc (e.g. as soon as a stroke patient arrives at the emergency room). During the observation, the observer takes notes on everything or certain pre-determined parts of what is happening around them, for example focusing on physician-patient interactions or communication between different professional groups. Written notes can be taken during or after the observations, depending on feasibility (which is usually lower during participant observations) and acceptability (e.g. when the observer is perceived to be judging the observed). Afterwards, these field notes are transcribed into observation protocols. If more than one observer was involved, field notes are taken independently, but notes can be consolidated into one protocol after discussions. Advantages of conducting observations include minimising the distance between the researcher and the researched, the potential discovery of topics that the researcher did not realise were relevant and gaining deeper insights into the real-world dimensions of the research problem at hand [ 18 ].

Semi-structured interviews

Hijmans & Kuyper describe qualitative interviews as “an exchange with an informal character, a conversation with a goal” [ 19 ]. Interviews are used to gain insights into a person’s subjective experiences, opinions and motivations – as opposed to facts or behaviours [ 13 ]. Interviews can be distinguished by the degree to which they are structured (i.e. a questionnaire), open (e.g. free conversation or autobiographical interviews) or semi-structured [ 2 , 13 ]. Semi-structured interviews are characterized by open-ended questions and the use of an interview guide (or topic guide/list) in which the broad areas of interest, sometimes including sub-questions, are defined [ 19 ]. The pre-defined topics in the interview guide can be derived from the literature, previous research or a preliminary method of data collection, e.g. document study or observations. The topic list is usually adapted and improved at the start of the data collection process as the interviewer learns more about the field [ 20 ]. Across interviews the focus on the different (blocks of) questions may differ and some questions may be skipped altogether (e.g. if the interviewee is not able or willing to answer the questions or for concerns about the total length of the interview) [ 20 ]. Qualitative interviews are usually not conducted in written format as it impedes on the interactive component of the method [ 20 ]. In comparison to written surveys, qualitative interviews have the advantage of being interactive and allowing for unexpected topics to emerge and to be taken up by the researcher. This can also help overcome a provider or researcher-centred bias often found in written surveys, which by nature, can only measure what is already known or expected to be of relevance to the researcher. Interviews can be audio- or video-taped; but sometimes it is only feasible or acceptable for the interviewer to take written notes [ 14 , 16 , 20 ].

Focus groups

Focus groups are group interviews to explore participants’ expertise and experiences, including explorations of how and why people behave in certain ways [ 1 ]. Focus groups usually consist of 6–8 people and are led by an experienced moderator following a topic guide or “script” [ 21 ]. They can involve an observer who takes note of the non-verbal aspects of the situation, possibly using an observation guide [ 21 ]. Depending on researchers’ and participants’ preferences, the discussions can be audio- or video-taped and transcribed afterwards [ 21 ]. Focus groups are useful for bringing together homogeneous (to a lesser extent heterogeneous) groups of participants with relevant expertise and experience on a given topic on which they can share detailed information [ 21 ]. Focus groups are a relatively easy, fast and inexpensive method to gain access to information on interactions in a given group, i.e. “the sharing and comparing” among participants [ 21 ]. Disadvantages include less control over the process and a lesser extent to which each individual may participate. Moreover, focus group moderators need experience, as do those tasked with the analysis of the resulting data. Focus groups can be less appropriate for discussing sensitive topics that participants might be reluctant to disclose in a group setting [ 13 ]. Moreover, attention must be paid to the emergence of “groupthink” as well as possible power dynamics within the group, e.g. when patients are awed or intimidated by health professionals.

Choosing the “right” method

As explained above, the school of thought underlying qualitative research assumes no objective hierarchy of evidence and methods. This means that each choice of single or combined methods has to be based on the research question that needs to be answered and a critical assessment with regard to whether or to what extent the chosen method can accomplish this – i.e. the “fit” between question and method [ 14 ]. It is necessary for these decisions to be documented when they are being made, and to be critically discussed when reporting methods and results.

Let us assume that our research aim is to examine the (clinical) processes around acute endovascular treatment (EVT), from the patient’s arrival at the emergency room to recanalization, with the aim to identify possible causes for delay and/or other causes for sub-optimal treatment outcome. As a first step, we could conduct a document study of the relevant standard operating procedures (SOPs) for this phase of care – are they up-to-date and in line with current guidelines? Do they contain any mistakes, irregularities or uncertainties that could cause delays or other problems? Regardless of the answers to these questions, the results have to be interpreted based on what they are: a written outline of what care processes in this hospital should look like. If we want to know what they actually look like in practice, we can conduct observations of the processes described in the SOPs. These results can (and should) be analysed in themselves, but also in comparison to the results of the document analysis, especially as regards relevant discrepancies. Do the SOPs outline specific tests for which no equipment can be observed or tasks to be performed by specialized nurses who are not present during the observation? It might also be possible that the written SOP is outdated, but the actual care provided is in line with current best practice. In order to find out why these discrepancies exist, it can be useful to conduct interviews. Are the physicians simply not aware of the SOPs (because their existence is limited to the hospital’s intranet) or do they actively disagree with them or does the infrastructure make it impossible to provide the care as described? Another rationale for adding interviews is that some situations (or all of their possible variations for different patient groups or the day, night or weekend shift) cannot practically or ethically be observed. In this case, it is possible to ask those involved to report on their actions – being aware that this is not the same as the actual observation. A senior physician’s or hospital manager’s description of certain situations might differ from a nurse’s or junior physician’s one, maybe because they intentionally misrepresent facts or maybe because different aspects of the process are visible or important to them. In some cases, it can also be relevant to consider to whom the interviewee is disclosing this information – someone they trust, someone they are otherwise not connected to, or someone they suspect or are aware of being in a potentially “dangerous” power relationship to them. Lastly, a focus group could be conducted with representatives of the relevant professional groups to explore how and why exactly they provide care around EVT. The discussion might reveal discrepancies (between SOPs and actual care or between different physicians) and motivations to the researchers as well as to the focus group members that they might not have been aware of themselves. For the focus group to deliver relevant information, attention has to be paid to its composition and conduct, for example, to make sure that all participants feel safe to disclose sensitive or potentially problematic information or that the discussion is not dominated by (senior) physicians only. The resulting combination of data collection methods is shown in Fig.  2 .

Possible combination of data collection methods

Attributions for icons: “Book” by Serhii Smirnov, “Interview” by Adrien Coquet, FR, “Magnifying Glass” by anggun, ID, “Business communication” by Vectors Market; all from the Noun Project

The combination of multiple data source as described for this example can be referred to as “triangulation”, in which multiple measurements are carried out from different angles to achieve a more comprehensive understanding of the phenomenon under study [ 22 , 23 ].

Data analysis

To analyse the data collected through observations, interviews and focus groups these need to be transcribed into protocols and transcripts (see Fig.  3 ). Interviews and focus groups can be transcribed verbatim , with or without annotations for behaviour (e.g. laughing, crying, pausing) and with or without phonetic transcription of dialects and filler words, depending on what is expected or known to be relevant for the analysis. In the next step, the protocols and transcripts are coded , that is, marked (or tagged, labelled) with one or more short descriptors of the content of a sentence or paragraph [ 2 , 15 , 23 ]. Jansen describes coding as “connecting the raw data with “theoretical” terms” [ 20 ]. In a more practical sense, coding makes raw data sortable. This makes it possible to extract and examine all segments describing, say, a tele-neurology consultation from multiple data sources (e.g. SOPs, emergency room observations, staff and patient interview). In a process of synthesis and abstraction, the codes are then grouped, summarised and/or categorised [ 15 , 20 ]. The end product of the coding or analysis process is a descriptive theory of the behavioural pattern under investigation [ 20 ]. The coding process is performed using qualitative data management software, the most common ones being InVivo, MaxQDA and Atlas.ti. It should be noted that these are data management tools which support the analysis performed by the researcher(s) [ 14 ].

From data collection to data analysis

Attributions for icons: see Fig. ​ Fig.2, 2 , also “Speech to text” by Trevor Dsouza, “Field Notes” by Mike O’Brien, US, “Voice Record” by ProSymbols, US, “Inspection” by Made, AU, and “Cloud” by Graphic Tigers; all from the Noun Project

How to report qualitative research?

Protocols of qualitative research can be published separately and in advance of the study results. However, the aim is not the same as in RCT protocols, i.e. to pre-define and set in stone the research questions and primary or secondary endpoints. Rather, it is a way to describe the research methods in detail, which might not be possible in the results paper given journals’ word limits. Qualitative research papers are usually longer than their quantitative counterparts to allow for deep understanding and so-called “thick description”. In the methods section, the focus is on transparency of the methods used, including why, how and by whom they were implemented in the specific study setting, so as to enable a discussion of whether and how this may have influenced data collection, analysis and interpretation. The results section usually starts with a paragraph outlining the main findings, followed by more detailed descriptions of, for example, the commonalities, discrepancies or exceptions per category [ 20 ]. Here it is important to support main findings by relevant quotations, which may add information, context, emphasis or real-life examples [ 20 , 23 ]. It is subject to debate in the field whether it is relevant to state the exact number or percentage of respondents supporting a certain statement (e.g. “Five interviewees expressed negative feelings towards XYZ”) [ 21 ].

How to combine qualitative with quantitative research?

Qualitative methods can be combined with other methods in multi- or mixed methods designs, which “[employ] two or more different methods [ …] within the same study or research program rather than confining the research to one single method” [ 24 ]. Reasons for combining methods can be diverse, including triangulation for corroboration of findings, complementarity for illustration and clarification of results, expansion to extend the breadth and range of the study, explanation of (unexpected) results generated with one method with the help of another, or offsetting the weakness of one method with the strength of another [ 1 , 17 , 24 – 26 ]. The resulting designs can be classified according to when, why and how the different quantitative and/or qualitative data strands are combined. The three most common types of mixed method designs are the convergent parallel design , the explanatory sequential design and the exploratory sequential design. The designs with examples are shown in Fig.  4 .

Three common mixed methods designs

In the convergent parallel design, a qualitative study is conducted in parallel to and independently of a quantitative study, and the results of both studies are compared and combined at the stage of interpretation of results. Using the above example of EVT provision, this could entail setting up a quantitative EVT registry to measure process times and patient outcomes in parallel to conducting the qualitative research outlined above, and then comparing results. Amongst other things, this would make it possible to assess whether interview respondents’ subjective impressions of patients receiving good care match modified Rankin Scores at follow-up, or whether observed delays in care provision are exceptions or the rule when compared to door-to-needle times as documented in the registry. In the explanatory sequential design, a quantitative study is carried out first, followed by a qualitative study to help explain the results from the quantitative study. This would be an appropriate design if the registry alone had revealed relevant delays in door-to-needle times and the qualitative study would be used to understand where and why these occurred, and how they could be improved. In the exploratory design, the qualitative study is carried out first and its results help informing and building the quantitative study in the next step [ 26 ]. If the qualitative study around EVT provision had shown a high level of dissatisfaction among the staff members involved, a quantitative questionnaire investigating staff satisfaction could be set up in the next step, informed by the qualitative study on which topics dissatisfaction had been expressed. Amongst other things, the questionnaire design would make it possible to widen the reach of the research to more respondents from different (types of) hospitals, regions, countries or settings, and to conduct sub-group analyses for different professional groups.

How to assess qualitative research?

A variety of assessment criteria and lists have been developed for qualitative research, ranging in their focus and comprehensiveness [ 14 , 17 , 27 ]. However, none of these has been elevated to the “gold standard” in the field. In the following, we therefore focus on a set of commonly used assessment criteria that, from a practical standpoint, a researcher can look for when assessing a qualitative research report or paper.

Assessors should check the authors’ use of and adherence to the relevant reporting checklists (e.g. Standards for Reporting Qualitative Research (SRQR)) to make sure all items that are relevant for this type of research are addressed [ 23 , 28 ]. Discussions of quantitative measures in addition to or instead of these qualitative measures can be a sign of lower quality of the research (paper). Providing and adhering to a checklist for qualitative research contributes to an important quality criterion for qualitative research, namely transparency [ 15 , 17 , 23 ].

Reflexivity

While methodological transparency and complete reporting is relevant for all types of research, some additional criteria must be taken into account for qualitative research. This includes what is called reflexivity, i.e. sensitivity to the relationship between the researcher and the researched, including how contact was established and maintained, or the background and experience of the researcher(s) involved in data collection and analysis. Depending on the research question and population to be researched this can be limited to professional experience, but it may also include gender, age or ethnicity [ 17 , 27 ]. These details are relevant because in qualitative research, as opposed to quantitative research, the researcher as a person cannot be isolated from the research process [ 23 ]. It may influence the conversation when an interviewed patient speaks to an interviewer who is a physician, or when an interviewee is asked to discuss a gynaecological procedure with a male interviewer, and therefore the reader must be made aware of these details [ 19 ].

Sampling and saturation

The aim of qualitative sampling is for all variants of the objects of observation that are deemed relevant for the study to be present in the sample “ to see the issue and its meanings from as many angles as possible” [ 1 , 16 , 19 , 20 , 27 ] , and to ensure “information-richness [ 15 ]. An iterative sampling approach is advised, in which data collection (e.g. five interviews) is followed by data analysis, followed by more data collection to find variants that are lacking in the current sample. This process continues until no new (relevant) information can be found and further sampling becomes redundant – which is called saturation [ 1 , 15 ] . In other words: qualitative data collection finds its end point not a priori , but when the research team determines that saturation has been reached [ 29 , 30 ].

This is also the reason why most qualitative studies use deliberate instead of random sampling strategies. This is generally referred to as “ purposive sampling” , in which researchers pre-define which types of participants or cases they need to include so as to cover all variations that are expected to be of relevance, based on the literature, previous experience or theory (i.e. theoretical sampling) [ 14 , 20 ]. Other types of purposive sampling include (but are not limited to) maximum variation sampling, critical case sampling or extreme or deviant case sampling [ 2 ]. In the above EVT example, a purposive sample could include all relevant professional groups and/or all relevant stakeholders (patients, relatives) and/or all relevant times of observation (day, night and weekend shift).

Assessors of qualitative research should check whether the considerations underlying the sampling strategy were sound and whether or how researchers tried to adapt and improve their strategies in stepwise or cyclical approaches between data collection and analysis to achieve saturation [ 14 ].

Good qualitative research is iterative in nature, i.e. it goes back and forth between data collection and analysis, revising and improving the approach where necessary. One example of this are pilot interviews, where different aspects of the interview (especially the interview guide, but also, for example, the site of the interview or whether the interview can be audio-recorded) are tested with a small number of respondents, evaluated and revised [ 19 ]. In doing so, the interviewer learns which wording or types of questions work best, or which is the best length of an interview with patients who have trouble concentrating for an extended time. Of course, the same reasoning applies to observations or focus groups which can also be piloted.

Ideally, coding should be performed by at least two researchers, especially at the beginning of the coding process when a common approach must be defined, including the establishment of a useful coding list (or tree), and when a common meaning of individual codes must be established [ 23 ]. An initial sub-set or all transcripts can be coded independently by the coders and then compared and consolidated after regular discussions in the research team. This is to make sure that codes are applied consistently to the research data.

Member checking

Member checking, also called respondent validation , refers to the practice of checking back with study respondents to see if the research is in line with their views [ 14 , 27 ]. This can happen after data collection or analysis or when first results are available [ 23 ]. For example, interviewees can be provided with (summaries of) their transcripts and asked whether they believe this to be a complete representation of their views or whether they would like to clarify or elaborate on their responses [ 17 ]. Respondents’ feedback on these issues then becomes part of the data collection and analysis [ 27 ].

Stakeholder involvement

In those niches where qualitative approaches have been able to evolve and grow, a new trend has seen the inclusion of patients and their representatives not only as study participants (i.e. “members”, see above) but as consultants to and active participants in the broader research process [ 31 – 33 ]. The underlying assumption is that patients and other stakeholders hold unique perspectives and experiences that add value beyond their own single story, making the research more relevant and beneficial to researchers, study participants and (future) patients alike [ 34 , 35 ]. Using the example of patients on or nearing dialysis, a recent scoping review found that 80% of clinical research did not address the top 10 research priorities identified by patients and caregivers [ 32 , 36 ]. In this sense, the involvement of the relevant stakeholders, especially patients and relatives, is increasingly being seen as a quality indicator in and of itself.

How not to assess qualitative research

The above overview does not include certain items that are routine in assessments of quantitative research. What follows is a non-exhaustive, non-representative, experience-based list of the quantitative criteria often applied to the assessment of qualitative research, as well as an explanation of the limited usefulness of these endeavours.

Protocol adherence

Given the openness and flexibility of qualitative research, it should not be assessed by how well it adheres to pre-determined and fixed strategies – in other words: its rigidity. Instead, the assessor should look for signs of adaptation and refinement based on lessons learned from earlier steps in the research process.

Sample size

For the reasons explained above, qualitative research does not require specific sample sizes, nor does it require that the sample size be determined a priori [ 1 , 14 , 27 , 37 – 39 ]. Sample size can only be a useful quality indicator when related to the research purpose, the chosen methodology and the composition of the sample, i.e. who was included and why.

Randomisation

While some authors argue that randomisation can be used in qualitative research, this is not commonly the case, as neither its feasibility nor its necessity or usefulness has been convincingly established for qualitative research [ 13 , 27 ]. Relevant disadvantages include the negative impact of a too large sample size as well as the possibility (or probability) of selecting “ quiet, uncooperative or inarticulate individuals ” [ 17 ]. Qualitative studies do not use control groups, either.

Interrater reliability, variability and other “objectivity checks”

The concept of “interrater reliability” is sometimes used in qualitative research to assess to which extent the coding approach overlaps between the two co-coders. However, it is not clear what this measure tells us about the quality of the analysis [ 23 ]. This means that these scores can be included in qualitative research reports, preferably with some additional information on what the score means for the analysis, but it is not a requirement. Relatedly, it is not relevant for the quality or “objectivity” of qualitative research to separate those who recruited the study participants and collected and analysed the data. Experiences even show that it might be better to have the same person or team perform all of these tasks [ 20 ]. First, when researchers introduce themselves during recruitment this can enhance trust when the interview takes place days or weeks later with the same researcher. Second, when the audio-recording is transcribed for analysis, the researcher conducting the interviews will usually remember the interviewee and the specific interview situation during data analysis. This might be helpful in providing additional context information for interpretation of data, e.g. on whether something might have been meant as a joke [ 18 ].

Not being quantitative research

Being qualitative research instead of quantitative research should not be used as an assessment criterion if it is used irrespectively of the research problem at hand. Similarly, qualitative research should not be required to be combined with quantitative research per se – unless mixed methods research is judged as inherently better than single-method research. In this case, the same criterion should be applied for quantitative studies without a qualitative component.

The main take-away points of this paper are summarised in Table ​ Table1. 1 . We aimed to show that, if conducted well, qualitative research can answer specific research questions that cannot to be adequately answered using (only) quantitative designs. Seeing qualitative and quantitative methods as equal will help us become more aware and critical of the “fit” between the research problem and our chosen methods: I can conduct an RCT to determine the reasons for transportation delays of acute stroke patients – but should I? It also provides us with a greater range of tools to tackle a greater range of research problems more appropriately and successfully, filling in the blind spots on one half of the methodological spectrum to better address the whole complexity of neurological research and practice.

Take-away-points

Acknowledgements

Abbreviations, authors’ contributions.

LB drafted the manuscript; WW and CG revised the manuscript; all authors approved the final versions.

no external funding.

Availability of data and materials

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Retirement planning – a systematic review of literature and future research directions

• Published: 28 October 2023

Cite this article

• Kavita Karan Ingale   ORCID: orcid.org/0000-0003-3570-4211 1 &
• Ratna Achuta Paluri 2  

506 Accesses

Explore all metrics

Rising life expectancy and an aging population across nations are leading to an increased need for long-term financial savings and a focus on the financial well-being of retired individuals amidst changing policy framework. This study is a systematic review based on a scientific way of producing high-quality evidence based on 191 articles from the Scopus and Web of Science databases. It adopts the Theory, Context, Characteristics, and Method (TCCM) framework to analyze literature. This study provides collective insights into financial decision-making for retirement savings and identifies constructs for operationalizing and measuring financial behavior for retirement planning. Further, it indicates the need for an interdisciplinary approach. Though cognitive areas were studied extensively, the non-cognitive areas received little attention. Qualitative research design is gaining prominence in research over other methods, with the sparse application of mixed methods design. The study’s TCCM framework explicates several areas for further research. Furthermore, it guides the practice and policy by integrating empirical evidence and concomitant findings. Coherent synthesis of the extant literature reconciles the highly fragmented field of retirement planning. No research reports prospective areas for further analysis based on the TCCM framework on retirement planning, which highlights the uniqueness of the study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

A Research Proposal to Examine Psychological Factors Influence on Financial Planning for Retirement in China

Domains and determinants of retirement timing: A systematic review of longitudinal studies

Micky Scharn, Ranu Sewdas, … Allard J. van der Beek

Reinventing Retirement

Deanna L. Sharpe

Data Availability

The research data will be made available on request.

Acknowledgment.

Elderly population is defined as a population aged 65 years and over.

Defined benefit plan guarantees benefits to the employee, while defined contribution plan requires employees to decide on their own investment and bear the financial risks identified with it.

“The old-age dependency ratio is defined as the number of individuals aged 65 and over per 100 people of working age defined as those at ages 20 to 64”(OECD 2023 ).

Adams GA, Rau BL (2011) Putting off tomorrow to do what you want today: planning for Retirement. Am Psychol 66(3):180–192. https://doi.org/10.1037/a0022131

Article   Google Scholar  

Aegon Cfor, Longevity, Retirement ICR (2016) The Aegon Retirement Readiness Survey 2016. In The Aegon Retirement Readiness Survey 2016 . https://www.aegon.com/contentassets/c6a4b1cdded34f1b85a4f21d4c66e5d3/2016-aegon-retirement-readiness-report-india.pdf

Agarwalla SK, Barua SK, Jacob J, Varma JR (2015) Financial Literacy among Working Young in Urban India. World Development , 67 (2013), 101–109. https://doi.org/10.1016/j.worlddev.2014.10.004

Ajzen I (1991) The theory of Planned Behavior. Organ Behav Hum Decis Process 50:179–211. https://doi.org/10.47985/dcidj.475

Anderson A, Baker F, Robinson DT (2017) Precautionary savings, retirement planning, and misperceptions of financial literacy. J Financ Econ 126(2):383–398. https://doi.org/10.1016/j.jfineco.2017.07.008

Atkinson A, Messy FA (2011) Assessing financial literacy in 12 countries: an OECD/INFE international pilot exercise. J Pension Econ Finance 10(4):657–665. https://doi.org/10.1017/S1474747211000539`

Aydin AE, Akben Selcuk E (2019) An investigation of financial literacy, money ethics, and time preferences among college students: a structural equation model. Int J Bank Mark 37(3):880–900. https://doi.org/10.1108/IJBM-05-2018-0120

Bapat D (2020) Antecedents to responsible financial management behavior among young adults: the moderating role of financial risk tolerance. Int J Bank Mark 38(5):1177–1194. https://doi.org/10.1108/IJBM-10-2019-0356

Beckett A, Hewer P, Howcroft B (2000) An exposition of consumer behaviour in the financial services industry. Int J Bank Mark 18(1):15–26. https://doi.org/10.1108/02652320010315325

Białowolski P (2019) Economic sentiment as a driver for household financial behavior. J Behav Experimental Econ 80(August 2017):59–66. https://doi.org/10.1016/j.socec.2019.03.006

Binswanger J, Carman KG (2012) How real people make long-term decisions: the case of retirement preparation. J Economic Behav Organ 81(1):39–60. https://doi.org/10.1016/j.jebo.2011.08.010

Brounen D, Koedijk KG, Pownall RAJ (2016) Household financial planning and savings behavior. J Int Money Finance 69:95–107. https://doi.org/10.1016/j.jimonfin.2016.06.011

Brown R, Jones M (2015) Mapping and exploring the topography of contemporary financial accounting research. Br Acc Rev 47(3):237–261. https://doi.org/10.1016/j.bar.2014.08.006

Brown S, Gray D (2016) Household finances and well-being in Australia: an empirical analysis of comparison effects. J Econ Psychol 53:17–36. https://doi.org/10.1016/j.joep.2015.12.006

Brown S, Taylor K (2014) Household finances and the big five personality traits. J Econ Psychol 45:197–212. https://doi.org/10.1016/j.joep.2014.10.006

Brown S, Taylor K (2016) Early influences on saving behaviour: analysis of British panel data. J Bank Finance 62:1–14. https://doi.org/10.1016/j.jbankfin.2015.09.011

Brüggen EC, Post T, Schmitz K (2019) Interactivity in online pension planners enhances engagement with retirement planning – but not for everyone. J Serv Mark 33(4):488–501. https://doi.org/10.1108/JSM-02-2018-0082

Bruggen E, Post T, Katharina S (2019) Interactivity in online pension planners enhances engagement with retirement planning but not for everyone. J Serv Mark 33(4):488–501

Calcagno R, Monticone C (2015) Financial literacy and the demand for financial advice. J Bank Finance 50:363–380. https://doi.org/10.1016/j.jbankfin.2014.03.013

Campbell JY (2006) Household finance. J Finance 61(4):1553–1604. https://doi.org/10.1111/j.1540-6261.2006.00883.x

Choudhury K (2015) Service quality and customers’ behavioural intentions: class and mass banking and implications for the consumer and society. Asia Pac J Mark Logistics 27(5):735–757

Chowdhry N, Jung J, Dholakia U (2018) Association for consumer research. Adv Consum Res 42:42–46

Google Scholar  

Clark GL, Knox-Hayes J, Strauss K (2009) Financial sophistication, salience, and the scale of deliberation in UK retirement planning. Environ Plann A 41(10):2496–2515. https://doi.org/10.1068/a41265

Clark R, Lusardi A, Mitchell OS (2017) Employee Financial Literacy and Retirement Plan Behavior: a case study. Econ Inq 55(1):248–259. https://doi.org/10.1111/ecin.12389

Collins JM, Urban C (2016) The role of information on Retirement Planning: evidence from a field study. Econ Inq 54(4):1860–1872. https://doi.org/10.1111/ecin.12349

Creswell J (2009) Research Design Qualitative Quantitative and Mixed Methods Approaches. In Sage Publishing: Vol. Third edit . https://doi.org/10.1002/tl.20234

Csorba L (2020) The determining factors of financial culture, financial literacy, and financial behavior. Public Finance Q 65:67–83. https://doi.org/10.35551/PFQ_2020_1_6

Davidoff T, Gerhard P, Post T (2017) Reverse mortgages: what homeowners (don’t) know and how it matters. J Economic Behav Organ 133:151–171. https://doi.org/10.1016/j.jebo.2016.11.007

Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly: Management Information Systems 13(3):319–339. https://doi.org/10.2307/249008

Devlin J (2001) Consumer evaluation and competitive advantage in retail financial services - a research agenda. Eur J Mark 35(5/6):639–660

Dholakia U, Tam L, Yoon S, Wong N (2016) The ant and the grasshopper: understanding personal saving orientation of consumers. J Consum Res 43(1):134–155. https://doi.org/10.1093/jcr/ucw004

Dolls M, Doerrenberg P, Peichl A, Stichnoth H (2018) Do retirement savings increase in response to information about retirement and expected pensions? J Public Econ 158(July 2017):168–179. https://doi.org/10.1016/j.jpubeco.2017.12.014

Dragos SL, Dragos CM, Muresan GM (2020) From intention to the decision in purchasing life insurance and private pensions: different effects of knowledge and behavioural factors. J Behav Experimental Econ 87(March):101555. https://doi.org/10.1016/j.socec.2020.101555

Drever AI, Odders-white E, Kalish CW, Hoagland EM, Nelms EN, Drever AI, Odders-white E, Charles W, Else-quest NM, Hoagland EM, Nelms EN (2015) Foundations of Financial Weil-Being: Insights into the Role of Executive Function, Financial Socialization, and Experience-Based Learning in Childhood and Youth Source : The Journal of Consumer Affairs, Vol. 49, No. 1, Special Issue on Starting Ea. The Journal of Consumer Affairs , 49 (1)

Duflo E, Saez E (2002) Participation and investment decisions in a retirement plan: the influence of colleagues’ choices. J Public Econ 85(1):121–148. https://doi.org/10.1016/S0047-2727(01)00098-6

Duxbury D, Summers B, Hudson R, Keasey K (2013) How people evaluate defined contribution, annuity-based pension arrangements: a behavioral exploration. J Econ Psychol 34:256–269. https://doi.org/10.1016/j.joep.2012.10.008

Earl J, Bednall T, Muratore A (2015) A matter of time: why some people plan for retirement and others do not. Work Aging and Retirement 1(2):181–189. https://doi.org/10.1093/workar/wau005

Employees Benefits Research Institute (2020) EBRI Retirement Confidence Survey Report (Issue 202)

Engel JF, Kollat DT, Blackwell RD (1968) A model of consumer motivation and behavior. In: Research in consumer behavior. Holt, Rinehart and Winston, Inc., New York, pp 3–20

Erasmus A, Boshoff E, Rousseau G (2001) Consumer decision-making models within the discipline of consumer science: a critical approach. J Family Ecol Consumer Sci /Tydskrif Vir Gesinsekologie En Verbruikerswetenskappe 29(1):82–90. https://doi.org/10.4314/jfecs.v29i1.52799

Farrell L, Fry TRL, Risse L (2016) The significance of financial self-efficacy in explaining women’s personal finance behaviour. J Econ Psychol 54:85–99

Fernandes D, Lynch JG, Netemeyer RG (2014) Financial literacy, financial education, and downstream financial behaviors. Manage Sci 60(8):1861–1883. https://doi.org/10.1287/mnsc.2013.1849

Filbec G, Ricciardi V, Evensky H, Fan S, Holzhauer H, Spieler A (2017) Behavioral finance: a panel discussion. J Behav Experimental Finance 15:52–58. https://doi.org/10.1016/j.jbef.2015.07.003

Fishbein M (1979) A theory of reasoned action: some applications and implications. Nebraska Symposium on Motivation 27:65–116

Fisher PJ, Montalto CP (2010) Effect of saving motives and horizon on saving behaviors. J Econ Psychol 31(1):92–105. https://doi.org/10.1016/j.joep.2009.11.002

Flores SAM, Vieira KM (2014) Propensity toward indebtedness: an analysis using behavioral factors. J Behav Exp Finance 3:1–10

Foxall GR, Pallister JG (1998) Measuring purchase decision involvement for financial services: comparison of the Zaichkowsky and Mittal scales. Int J Bank Mark 16(5):180–194. https://doi.org/10.1108/02652329810228181

Friedman M (1957) Introduction to “A theory of the consumption function”. In: A theory of the consumption function. Princeton University Press, pp 1–6

Frydman C, Camerer CF (2016) The psychology and neuroscience of financial decision making. Trends Cogn Sci 20(9):661–675. https://doi.org/10.1016/j.tics.2016.07.003

Gardarsdóttir RB, Dittmar H (2012) The relationship of materialism to debt and financial well-being: the case of Iceland’s perceived prosperity. J Econ Psychol 33(3):471–481. https://doi.org/10.1016/j.joep.2011.12.008

Gathergood J (2012) Self-control, financial literacy and consumer over-indebtedness. J Econ Psychol 33(3):590–602

Gerhard P, Gladstone JJ, Hoffmann AOI (2018) Psychological characteristics and household savings behavior: the importance of accounting for latent heterogeneity. J Economic Behav Organ 148:66–82. https://doi.org/10.1016/j.jebo.2018.02.013

Gibbs PT (2009) Time, temporality, and

Goedde-Menke M, Lehmensiek-Starke M, Nolte S (2014) An empirical test of competing hypotheses for the annuity puzzle. J Econ Psychol 43:75–91

Gough O, Nurullah M (2009) Understanding what drives the purchase decision in pension and investment products. J Financial Serv Mark 14(2):152–172. https://doi.org/10.1057/fsm.2009.14

Griffin B, Loe D, Hesketh B (2012) Using Proactivity, Time Discounting, and the theory of Planned Behavior to identify predictors of Retirement Planning. Educ Gerontol 38(12):877–889. https://doi.org/10.1080/03601277.2012.660857

Gritten A (2011) New insights into consumer confidence in financial services. Int J Bank Mark 29(2):90–106. https://doi.org/10.1108/02652321111107602

Grohmann A (2018) Financial literacy and financial behavior: Evidence from the emerging Asian middle class. Pacific Basin Finance Journal , 48 (November 2017), 129–143. https://doi.org/10.1016/j.pacfin.2018.01.007

Grohmann A, Kouwenberg R, Menkhoff L (2015) Childhood roots of financial literacy. J Econ Psychol 51:114–133. https://doi.org/10.1016/j.joep.2015.09.002

Hair JF, Sarstedt M, Ringle CM, Mena JA (2012) An assessment of the use of partial least squares structural equation modeling in marketing research . 414–433. https://doi.org/10.1007/s11747-011-0261-6

Hanna SD, Kim KT, Chen SCC (2016) Retirement savings. In: Handbook of consumer finance research, pp 33–43

Harrison T, Waite K, White P (2006) Analysis by paralysis: the pension purchase decision process. Int J Bank Mark 24(1):5–23. https://doi.org/10.1108/02652320610642317

Hastings J, Mitchell O (2011) How financial literact and impatience shape retirement wealth and investment behaviors. Pengaruh Harga Diskon Dan Persepsi Produk Terhadap Nilai Belanja Serta Perilaku Pembelian Konsumen, NBER Working paper, 1–28

Hauff J, Carlander A, Amelie G, Tommy G, Holmen M (2016) Breaking the ice of low financial involvement: does narrative information format from a trusted sender increase savings in mutual funds? Int J Bank Mark 34(2):151–170

Hentzen JK, Hoffmann A, Dolan R, Pala E (2021) Artificial intelligence in customer-facing financial services: a systematic literature review and agenda for future research. Int J Bank Mark. https://doi.org/10.1108/IJBM-09-2021-0417

Hershey DA, Mowen JC (2000) Psychological determinants of financial preparedness for retirement. Gerontologist 40(6):687–697. https://doi.org/10.1093/geront/40.6.687

Hershey DA, Henkens K, Van Dalen HP (2007) Mapping the minds of retirement planners: a cross-cultural perspective. J Cross-Cult Psychol 38(3):361–382. https://doi.org/10.1177/0022022107300280

Hershey DA, Jacobs-Lawson JM, McArdle JJ, Hamagami F (2007b) Psychological foundations of financial planning for retirement. J Adult Dev 14(1–2):26–36. https://doi.org/10.1007/s10804-007-9028-1

Hershey DA, Jacobs-Lawson JM, McArdle JJ, Hamagami F (2008) Psychological foundations of financial planning for retirement. J Adult Dev 14(1–2):26–36. https://doi.org/10.1007/s10804-007-9028-1

Hershfield H, Goldstein D, Sharpe W, Fox J, Yeykelis L, Carstensen L, Bailenson J (2011) Increasing saving behavior through age-progressed renderings of the future self. J Mark Res 48:23–37

Hoffmann AOI, Broekhuizen TLJ (2009) Susceptibility to and impact of interpersonal influence in an investment context. J Acad Mark Sci 37:488–503

Hoffmann AOI, Broekhuizen TLJ (2010) Understanding investors’ decisions to purchase innovative products: drivers of adoption timing and range. Int J Res Mark 27(4):342–355. https://doi.org/10.1016/j.ijresmar.2010.08.002

Hoffmann AOI, Plotkina D (2020a) Positive framing when assessing the personal resources to manage one’s finances increases consumers’ retirement self-efficacy and improves retirement goal clarity. Psychol Mark 38(12):2286–2304. https://doi.org/10.1002/mar.21563

Hoffmann AOI, Plotkina D (2020b) Why and when does financial information affect retirement planning intentions and which consumers are more likely to act on them? Journal of Business Research , 117 (September 2019), 411–431. https://doi.org/10.1016/j.jbusres.2020.06.023

Hoffmann AOI, Plotkina D (2021) Let your past define your future. How recalling successful financial experiences can increase beliefs of self-efficacy in financial planning. J Consum Aff 55(3):847–871. https://doi.org/10.1111/joca.12378

Hoffmann AOI, Risse L (2020) Do good things come in pairs? How personality traits help explain individuals’ simultaneous pursuit of a healthy lifestyle and financially responsible behavior. J Consum Aff 54(3):1082–1120. https://doi.org/10.1111/joca.12317

Hsiao YJ, Tsai WC (2018) Financial literacy and participation in the derivatives markets. J Bank Finance 88:15–29

Huhmann BA, McQuitty S (2009) A model of consumer financial numeracy. Int J Bank Mark 27(4):270–293. https://doi.org/10.1108/02652320910968359

Huston SJ (2010) Measuring financial literacy. J Consum Aff 44(2):296–316. https://doi.org/10.1111/j.1745-6606.2010.01170.x

Ijevleva K, Arefjevs I (2014) Analysis of the Aggregate Financial Behaviour of customers using the Transtheoretical Model of Change. Procedia - Social and Behavioral Sciences 156(April):435–438. https://doi.org/10.1016/j.sbspro.2014.11.217

Ingale KK, Paluri RA (2020) Financial literacy and financial behavior: a bibliometric analysis. Rev Behav Finance. https://doi.org/10.1108/RBF-06-2020-0141

Jacobs-Lawson J, Hershey D (2005) Influence of future time perspective, financial knowledge, and financial risk tolerance on retirement savings behavior. Financial Serv Rev 14:331–344. https://doi.org/10.1088/1751-8113/44/8/085201

Jappelli T, Padula M (2013) Investment in financial literacy and saving decisions. J Bank Finance 37(8):2779–2792. https://doi.org/10.1016/j.jbankfin.2013.03.019

Kadoya Y, Rahim Khan MS (2020) Financial literacy in Japan: new evidence using financial knowledge, behavior, and attitude. Sustain (Switzerland) 12(9). https://doi.org/10.3390/su12093683

Kamil NSSN, Musa R, Sahak SZ (2014) Examining the Role of Financial Intelligence Quotient (FiQ) in explaining credit card usage behavior: a conceptual Framework. Procedia - Social and Behavioral Sciences 130:568–576. https://doi.org/10.1016/j.sbspro.2014.04.066

Kerry MJ (2018) Psychological antecedents of retirement planning: a systematic review. Front Psychol 9(OCT). https://doi.org/10.3389/fpsyg.2018.01870

Kerry MJ, Embretson SE (2018) An experimental evaluation of competing age predictions of future time perspective between workplace and retirement domains. Front Psychol 8(JAN):1–9. https://doi.org/10.3389/fpsyg.2017.02316

Kiliyanni AL, Sivaraman S (2016) The perception-reality gap in financial literacy: evidence from the most literate state in India. Int Rev Econ Educ 23:47–64. https://doi.org/10.1016/j.iree.2016.07.001

Kimiyaghalam F, Mansori S, Safari M, Yap S (2017) Parents’ influence on retirement planning in Malaysia. Family Consumer Sci Res J 45(3):315–325

Klapper L, Lusardi A, Panos GA (2013) Financial literacy and its consequences: evidence from Russia during the financial crisis. J Bank Finance 37(10):3904–3923

Koehler DJ, Langstaff J, Liu WQ (2015) A simulated financial savings task for studying consumption and retirement decision-making. J Econ Psychol 46:89–97. https://doi.org/10.1016/j.joep.2014.12.004

Kramer MM (2016) Financial literacy, confidence, and financial advice seeking. Journal of Economic Behavior and Organization , 131 (June 2015), 198–217. https://doi.org/10.1016/j.jebo.2016.08.016

Kumar S, Tomar S, Verma D (2019) Women’s financial planning for retirement: systematic literature review and future research agenda. Int J Bank Mark 37(1):120–141. https://doi.org/10.1108/IJBM-08-2017-0165

Kwon KN, Lee J (2009) The effects of reference point, knowledge, and risk propensity on the evaluation of financial products. J Bus Res 62(7):719–725. https://doi.org/10.1016/j.jbusres.2008.07.002

Landerretche OM, Martínez C (2013) Voluntary savings, financial behavior, and pension finance literacy: evidence from Chile. J Pension Econ Finance 12(3):251–297. https://doi.org/10.1017/S1474747212000340

Lee T (2017) (David). Clear, conspicuous, and improving: US corporate websites for critical financial literacy in retirement. International Journal of Bank Marketing , 35 (5), 761–780. https://doi.org/10.1108/IJBM-01-2016-0010

Liang C-J, Wang Wen‐Hung, Farquhar JD (2009) (2009). The influence of customer perceptions on financial performance in financial services. International Journal of Bank Marketing , 27 (2), 129–149

Liberman N, Trope Y (2003) Construal level theory of intertemporal judgment and decision. In: Loewenstein G, Read D, Baumeister R (eds) Time and decision: economic and psychological perspectives on intertemporal choice, pp 245–276

Lim KL, Soutar GN, Lee JA (2013) Factors affecting investment intentions: a consumer behaviour perspective. J Financ Serv Mark 18:301–315

Lin C, Hsiao YJ, Yeh CY (2017) Financial literacy, financial advisors, and information sources on demand for life insurance. Pac Basin Finance J 43(March):218–237. https://doi.org/10.1016/j.pacfin.2017.04.002

Lown JM (2011) Development and validation of a Financial Self-Efficacy Scale. J Financial Couns Plann 22(2):54–63

Lusardi A, Mitchell OS (2007) Baby Boomer retirement security: the roles of planning, financial literacy, and housing wealth. J Monet Econ 54(1):205–224. https://doi.org/10.1016/j.jmoneco.2006.12.001

Maloney M, McCarthy A (2017) Understanding pension communications at the organizational level: insights from bounded rationality theory & implications for HRM. Hum Resource Manage Rev 27(2):338–352. https://doi.org/10.1016/j.hrmr.2016.08.001

Marjanovic Z, Fiksenbaum L, Greenglass E (2018) Financial threat correlates with acute economic hardship and behavioral intentions that can improve one’s personal finances and health. J Behav Experimental Econ 77(April):151–157. https://doi.org/10.1016/j.socec.2018.09.012

Marques S, Mariano J, Lima ML, Abrams D (2018) Are you talking to the future me? The moderator role of future self-relevance on the effects of aging salience in retirement savings. J Appl Soc Psychol 48(7):360–368. https://doi.org/10.1111/jasp.12516

McKechnie S (1992) Consumer buying behaviour in financial services: an overview. Int J Bank Mark 10(5):5–39. https://doi.org/10.1108/02652329210016803

Milner T, Rosenstreich D (2013a) A review of consumer decision-making models and development of a new model for financial services. J Financial Serv Mark 18(2):106–120. https://doi.org/10.1057/fsm.2013.7

Milner T, Rosenstreich D (2013b) Insights into mature consumers of financial services. J Consumer Mark 30(3):248–257. https://doi.org/10.1108/07363761311328919

Mitchell OS, Mukherjee A (2017) Assessing the demand for micro pensions among India’s poor. J Econ Ageing 9:30–40. https://doi.org/10.1016/j.jeoa.2016.05.004

Mitchell O, Utkus S (2003) Lessons from Behavioral Finance for Retirement Plan Design (PRC WP 2003-6). http://prc.wharton.upenn.edu/prc/prc.html

Modigliani F, Brumberg RH (1954) Utility analysis and the consumption function: an interpretation of cross-section data. In: Kurihara KK (ed) Post-Keynesian economics. Rutgers University Press, New Brunswick, pp 388–436

Moher D, Liberati A, Tetzlaff J, Altman DG, Altman D, Antes G, Atkins D, Barbour V, Barrowman N, Berlin JA, Clark J, Clarke M, Cook D, D’Amico R, Deeks JJ, Devereaux PJ, Dickersin K, Egger M, Ernst E, …, Tugwell P (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6(7). https://doi.org/10.1371/journal.pmed.1000097

Monti M, Pelligra V, Martignon L, Berg N (2014) Retail investors and financial advisors: new evidence on trust and advice taking heuristics. J Bus Res 67(8):1749–1757. https://doi.org/10.1016/j.jbusres.2014.02.022

Mouna A, Anis J (2017) Financial literacy in Tunisia: its determinants and its implications on investment behavior. Res Int Bus Finance 39:568–577

Mullainathan S, Thaler R (2000) Massachusetts Institute of Technology Department of Economics Working Paper Series . September

Nga KH, Yeoh KK (2018) An exploratory model on retirement savings behaviour: a Malaysian study. Int J Bus Soc 19(3):637–659

OECD (2023) Old-age dependency ratio (indicator). https://doi.org/10.1787/e0255c98-en . Accessed 13 Oct 2023

Onwuegbuzie AJ, Collins KM (2007) A typology of mixed methods sampling designs in social science research. Qualitative Rep 12(2):474–498

Pallister JG, Wang HC, Foxall GR (2007) An application of the style/involvement model to financial services. Technovation 27(1–2):78–88. https://doi.org/10.1016/j.technovation.2005.10.001

Pan L, Pezzuti T, Lu W, Pechmann C (2019) Hyperopia and frugality: different motivational drivers and yet similar effects on consumer spending. J Bus Res 95(August 2018):347–356

Parise G, Peijnenburg K (2017) Understanding the Determinants of Financial Outcomes and Choices: The Role of Noncognitive Abilities. BIS Working Papers

Paul J, Rosado-Serrano A (2019) Gradual internationalization vs Born-Global/International new venture models: a review and research agenda. Int Mark Rev 36(6):830–858. https://doi.org/10.1108/IMR-10-2018-0280

Paul J, Criado AR (2020) The art of writing literature review: what do we know and what do we need to know? Int Bus Rev 29(4):101717. https://doi.org/10.1016/j.ibusrev.2020.101717

Paul J, Khatri P, Kaur Duggal H (2023) Frameworks for developing impactful systematic literature reviews and theory building: what, why and how? J Decis Syst 00(00):1–14. https://doi.org/10.1080/12460125.2023.2197700

Petkoska J, Earl JK (2009) Understanding the influence of demographic and psychological variables on Retirement Planning. Psychol Aging 24(1):245–251. https://doi.org/10.1037/a0014096

Piotrowska M (2019) The importance of personality characteristics and behavioral constraints for retirement saving. Econ Anal Policy 64:194–220

Plath DA, Stevenson TH (2005) Financial services consumption behavior across Hispanic American consumers. J Bus Res 58(8):1089–1099. https://doi.org/10.1016/j.jbusres.2004.03.003

Poterba JM (2015) Saver heterogeneity and the challenge of assessing retirement saving adequacy. Natl Tax J 68(2):377–388. https://doi.org/10.17310/ntj.2015.2.06

Potrich ACG, Vieira KM, Kirch G (2018) How well do women do when it comes to financial literacy? Proposition of an indicator and analysis of gender differences. J Behav Experimental Finance 17:28–41. https://doi.org/10.1016/j.jbef.2017.12.005

Rai D, Lin CW (2019) (Wilson). The influence of implicit self-theories on consumer financial decision making. Journal of Business Research , 95 (August 2018), 316–325. https://doi.org/10.1016/j.jbusres.2018.08.016

Ramalho TB, Forte D (2019) Financial literacy in Brazil – do knowledge and self-confidence relate with behavior? RAUSP Manage J 54(1):77–95. https://doi.org/10.1108/RAUSP-04-2018-0008

Rana J, Paul J (2017) Consumer behavior and purchase intention for organic food: a review and research agenda. J Retailing Consumer Serv 38(June):157–165. https://doi.org/10.1016/j.jretconser.2017.06.004

Ranyard R, McNair S, Nicolini G, Duxbury D (2020) An item response theory approach to constructing and evaluating brief and in-depth financial literacy scales. J Consum Aff 54(3):1121–1156. https://doi.org/10.1111/joca.12322

RBI Household Finance Committee (2017) Indian household finance. Reserve Bank of India, Mumbai

Ruefenacht M, Schlager T, Maas P, Puustinen P (2015) Drivers of long-term savings behavior from consumer’s perspective. Electron Libr 34(1):1–5

Scholz JK, Seshadri A, Khitatrakun S (2006) Are Americans saving “optimally” for retirement? J Polit Econ 114(4):607–643

Schuabb T, França LH, Amorim SM (2019) Retirement savings model tested with Brazilian private health care workers. Front Psychol 10(JULY):1–11. https://doi.org/10.3389/fpsyg.2019.01701

Schuhen M, Schurkmann S (2014) International Review of Economics Education. Int Rev Econ Educ 16:1–11

Segel-Karpas D, Werner P (2014) Perceived financial retirement preparedness and its correlates: a national study in Israel. Int J Aging Hum Dev 79(4):279–301. https://doi.org/10.1177/0091415015574177

Seth H, Talwar S, Bhatia A, Saxena A, Dhir A (2020) Consumer resistance and inertia of retail investors: Development of the resistance adoption inertia continuance (RAIC) framework. Journal of Retailing and Consumer Services , 55 (August 2019), 102071. https://doi.org/10.1016/j.jretconser.2020.102071

Sewell M (2008) Behavioural finance. Economist 389(8604):1–13. https://doi.org/10.1057/9780230280786_5

Shefrin HM, Thaler RH (1988) The behavioral life‐cycle hypothesis. Econ Inq 26(4):609–643

Shim S, Serido J, Tang C (2012) The ant and the grasshopper revisited: the present psychological benefits of saving and future oriented financial behavior. J Econ Psychol 33(1):155–165

Simon HA (1978) Information-processing theory of human problem solving. In: Handbook of learning and cognitive processes, vol 5, pp 271–295

Sivaramakrishnan S, Srivastava M, Rastogi A (2017) Attitudinal factors, financial literacy, and stock market participation. Int J Bank Mark 34(1):1–5

Snyder H (2019) Literature review as a research methodology: an overview and guidelines. J Bus Res 104(August):333–339. https://doi.org/10.1016/j.jbusres.2019.07.039

Stawski RS, Hershey DA, Jacobs-Lawson JM (2007) Goal clarity and financial planning activities as determinants of retirement savings contributions. Int J Aging Hum Dev 64(1):13–32. https://doi.org/10.2190/13GK-5H72-H324-16P2

Steinert JI, Zenker J, Filipiak U, Movsisyan A, Cluver LD, Shenderovich Y (2018) Do saving promotion interventions increase household savings, consumption, and investments in Sub-saharan Africa? A systematic review and meta-analysis. World Dev 104:238–256. https://doi.org/10.1016/j.worlddev.2017.11.018

Steinhart Y, Mazursky D (2010) Purchase availability and involvement antecedents among financial products. Int J Bank Mark 28(2):113–135. https://doi.org/10.1108/02652321011018314

Strömbäck C, Lind T, Skagerlund K, Västfjäll D, Tinghög G (2017) Does self-control predict financial behavior and financial well-being? J Behav Experimental Finance 14:30–38. https://doi.org/10.1016/j.jbef.2017.04.002

Strömbäck C, Skagerlund K, Västfjäll D, Tinghög G (2020) Subjective self-control but not objective measures of executive functions predict financial behavior and well-being. Journal of Behavioral and Experimental Finance , 27 . https://doi.org/10.1016/j.jbef.2020.100339

Tam L, Dholakia U (2014) Saving in cycles: how to get people to save more money. Psychol Sci 25(2):531–537. https://doi.org/10.1177/0956797613512129

Tang N, Baker A (2016) Self-esteem, financial knowledge and financial behavior. J Econ Psychol 54:164–176

Tate M, Evermann J, Gable G (2015) An integrated framework for theories of individual attitudes toward technology. Inform Manage 52(6):710–727. https://doi.org/10.1016/j.im.2015.06.005

Taylor MP, Jenkins SP, Sacker A (2011) Financial capability and psychological health. J Econ Psychol 32(5):710–723. https://doi.org/10.1016/j.joep.2011.05.006

Tennyson S, Yang HK (2014) The role of life experience in long-term care insurance decisions. Journal of Economic Psychology , 42 (2014), 175–188. https://doi.org/10.1016/j.joep.2014.04.002

Thaler BRH (1994) Psychology and savings policies. Am Econ Rev 84(2):175–179. http://www.jstor.org/stable/3132220

Thaler R (1980) Toward a positive theory of consumer choice. J Econ Behav Organ 1:39–60

Thaler RH (2005) Advances in behavioral finance. Adv Behav Finance 2:1–694. https://doi.org/10.2307/2329257

Thaler R, Shefrin H (1981) An economic theory of self-control. J Polit Econ 89(2):392–406

Tomar S, Kent Baker H, Kumar S, Hoffmann AOI (2021) Psychological determinants of retirement financial planning behavior. Journal of Business Research , 133 (November 2020), 432–449. https://doi.org/10.1016/j.jbusres.2021.05.007

Topa G, Moriano JA, Depolo M, Alcover CM, Morales JF (2009) Antecedents and consequences of retirement planning and decision-making: a meta-analysis and model. J Vocat Behav 75(1):38–55. https://doi.org/10.1016/j.jvb.2009.03.002

Topa G, Moriano JA, Depolo M, Alcover CM, Moreno A (2011) Retirement and wealth relationships: Meta-analysis and SEM. Res Aging 33(5):501–528. https://doi.org/10.1177/0164027511410549

Tranfield D, Denyer D, Smart P (2003) Towards a methodology for developing evidence-informed management knowledge by means of systematic review. Br J Manag 14(3):207–222. https://doi.org/10.1111/1467-8551.00375

Ülkümen G, Cheema A (2011) Framing goals to influence personal savings: the role of specificity and construal level. J Mark Res 48(6):958–969. https://doi.org/10.1509/jmr.09.0516

United Nations, Department of Economic and Social, Affairs PD (2020) (2019). World Population Ageing 2019. In United Nations . http://link.springer.com/chapter/10.1007/ 978-94-007-5204-7_6

Utkarsh, Pandey A, Ashta A, Spiegelman E, Sutan A (2020) Catch them young: impact of financial Socialization, financial literacy and attitude towards money on the financial well-being of young adults. Int J Consumer Stud 44(6):531–541. https://doi.org/10.1111/ijcs.12583

Valente TW, Paredes P, Poppe P (1998) Matching the message to the process: the relative ordering of knowledge, attitudes, and practices in behavior change research. Hum Commun Res 24(3):366–385

Van Rooij M, Teppa F (2014) Personal traits and individual choices: taking action in economic and non-economic decisions. J Econ Behav Organ 100:33–43

van Rooij M, Lusardi A, Alessie R (2011) Financial literacy and stock market participation. J Financ Econ 101(2):449–472. https://doi.org/10.1016/j.jfineco.2011.03.006

Van Rooij MCJ, Lusardi A, Alessie RJM (2011a) Financial literacy and retirement planning in the Netherlands. J Econ Psychol 32(4):593–608. https://doi.org/10.1016/j.joep.2011.02.004

van Schie RJG, Dellaert BGC, Donkers B (2015) Promoting later planned retirement: construal level intervention impact reverses with age. J Econ Psychol 50:124–131. https://doi.org/10.1016/j.joep.2015.06.010

Venkatesh V, Morris M, Davis G, Davis F (2003) Factors influencing the Use of M-Banking by academics: Case Study sms-based M-Banking. MIS Q 27(3):425–478

Vitt LA (2004) Consumers’ financial decisions and the psychology of values. J Financial Service Professionals 58(November):68–77. http://search.ebscohost.com/login.aspx?direct=true &db=bth&AN=14888952&site=ehost-live

Wang L, Lu W, Malhotra NK (2011) Demographics, attitude, personality, and credit card features correlate with credit card debt: a view from China. J Econ Psychol 32(1):179–193. https://doi.org/10.1016/j.joep.2010.11.006

World Economic Forum (2019) Investing in (and for) our future. Issue June. www.weforum.org

Xia T, Wang Z, Li K (2014) Financial literacy overconfidence and stock market participation. Soc Indic Res 119(3):1233–1245. https://doi.org/10.1007/s11205-013-0555-9

Xiao JJ, Chen C, Chen F (2014) Consumer financial capability and financial satisfaction. Soc Indic Res 118(1):415–432. https://doi.org/10.1007/s11205-013-0414-8

Yeung DY, Zhou X (2017) Planning for retirement: longitudinal effect on retirement resources and post-retirement well-being. Front Psychol 8:1300

Zhou R, Pham MT (2004) Promotion and prevention across mental accounts: when financial products dictate consumers’ investment goals. J Consum Res 31(1):125–135. https://doi.org/10.1086/383429

Download references

Acknowledgements

Authors would like to acknowledge the academicians and researchers who guided the search of the article and would like to thank the experts for the valuable inputs to refine the work.

There is no funding received for this research.

Author information

Authors and affiliations.

Symbiosis International (Deemed University), Pune, India

Kavita Karan Ingale

Symbiosis Institute of Operations Management, Symbiosis International (Deemed) University, Pune, India

Ratna Achuta Paluri

You can also search for this author in PubMed   Google Scholar

Contributions

Both authors contributed to the conceptualization, research design, methodology, analysis of the data,writing of the manuscript and its revision.

Corresponding author

Correspondence to Kavita Karan Ingale .

Ethics declarations

Conflict of interest.

The authors declare that there is no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Ingale, K.K., Paluri, R.A. Retirement planning – a systematic review of literature and future research directions. Manag Rev Q (2023). https://doi.org/10.1007/s11301-023-00377-x

Download citation

Received : 14 December 2022

Accepted : 04 October 2023

Published : 28 October 2023

DOI : https://doi.org/10.1007/s11301-023-00377-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Retirement planning
• Systematic literature review
• Financial behavior
• Household finance
• Long-term savings
• Pension plan
• Financial literacy
• TCCM framework
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 02 April 2022

A qualitative study of rural healthcare providers’ views of social, cultural, and programmatic barriers to healthcare access

• Nicholas C. Coombs 1 ,
• Duncan G. Campbell 2 &
• James Caringi 1  

BMC Health Services Research volume  22 , Article number:  438 ( 2022 ) Cite this article

22k Accesses

19 Citations

10 Altmetric

Metrics details

Ensuring access to healthcare is a complex, multi-dimensional health challenge. Since the inception of the coronavirus pandemic, this challenge is more pressing. Some dimensions of access are difficult to quantify, namely characteristics that influence healthcare services to be both acceptable and appropriate. These link to a patient’s acceptance of services that they are to receive and ensuring appropriate fit between services and a patient’s specific healthcare needs. These dimensions of access are particularly evident in rural health systems where additional structural barriers make accessing healthcare more difficult. Thus, it is important to examine healthcare access barriers in rural-specific areas to understand their origin and implications for resolution.

We used qualitative methods and a convenience sample of healthcare providers who currently practice in the rural US state of Montana. Our sample included 12 healthcare providers from diverse training backgrounds and specialties. All were decision-makers in the development or revision of patients’ treatment plans. Semi-structured interviews and content analysis were used to explore barriers–appropriateness and acceptability–to healthcare access in their patient populations. Our analysis was both deductive and inductive and focused on three analytic domains: cultural considerations, patient-provider communication, and provider-provider communication. Member checks ensured credibility and trustworthiness of our findings.

Five key themes emerged from analysis: 1) a friction exists between aspects of patients’ rural identities and healthcare systems; 2) facilitating access to healthcare requires application of and respect for cultural differences; 3) communication between healthcare providers is systematically fragmented; 4) time and resource constraints disproportionately harm rural health systems; and 5) profits are prioritized over addressing barriers to healthcare access in the US.

Conclusions

Inadequate access to healthcare is an issue in the US, particularly in rural areas. Rural healthcare consumers compose a hard-to-reach patient population. Too few providers exist to meet population health needs, and fragmented communication impairs rural health systems’ ability to function. These issues exacerbate the difficulty of ensuring acceptable and appropriate delivery of healthcare services, which compound all other barriers to healthcare access for rural residents. Each dimension of access must be monitored to improve patient experiences and outcomes for rural Americans.

Peer Review reports

Unequal access to healthcare services is an important element of health disparities in the United States [ 1 ], and there remains much about access that is not fully understood. The lack of understanding is attributable, in part, to the lack of uniformity in how access is defined and evaluated, and the extent to which access is often oversimplified in research [ 2 ]. Subsequently, attempts to address population-level barriers to healthcare access are insufficient, and access remains an unresolved, complex health challenge [ 3 , 4 , 5 ]. This paper presents a study that aims to explore some of the less well studied barriers to healthcare access, particularly those that influence healthcare acceptability and appropriateness.

In truth, healthcare access entails a complicated calculus that combines characteristics of individuals, their households, and their social and physical environments with characteristics of healthcare delivery systems, organizations, and healthcare providers. For one to fully ‘access’ healthcare, they must have the means to identify their healthcare needs and have available to them care providers and the facilities where they work. Further, patients must then reach, obtain, and use the healthcare services in order to have their healthcare needs fulfilled. Levesque and colleagues critically examined access conceptualizations in 2013 and synthesized all ways in which access to healthcare was previously characterized; Levesque et al. proposed five dimensions of access: approachability, acceptability, availability, affordability and appropriateness [ 2 ]. These refer to the ability to perceive, seek, reach, pay for, and engage in services, respectively.

According to Levesque et al.’s framework, the five dimensions combine to facilitate access to care or serve as barriers. Approachability indicates that people facing health needs understand that healthcare services exist and might be helpful. Acceptability represents whether patients see healthcare services as consistent or inconsistent with their own social and cultural values and worldviews. Availability indicates that healthcare services are reached both physically and in a timely manner. Affordability simplifies one’s capacity to pay for healthcare services without compromising basic necessities, and finally, appropriateness represents the fit between healthcare services and a patient’s specific healthcare needs [ 2 ]. This study focused on the acceptability and appropriateness dimensions of access.

Before the novel coronavirus (SARS-CoV-2; COVID-19) pandemic, approximately 13.3% of adults in the US did not have a usual source of healthcare [ 6 ]. Millions more did not utilize services regularly, and close to two-thirds reported that they would be debilitated by an unexpected medical bill [ 7 , 8 , 9 ]. Findings like these emphasized a fragility in the financial security of the American population [ 10 ]. These concerns were exacerbated by the pandemic when a sudden surge in unemployment increased un- and under-insurance rates [ 11 ]. Indeed, employer-sponsored insurance covers close to half of Americans’ total cost of illness [ 12 ]. Unemployment linked to COVID-19 cut off the lone outlet to healthcare access for many. Health-related financial concerns expanded beyond individuals, as healthcare organizations were unequipped to manage a simultaneous increase in demand for specialized healthcare services and a steep drop off for routine revenue-generating healthcare services [ 13 ]. These consequences of the COVID-19 pandemic all put additional, unexpected pressure on an already fragmented US healthcare system.

Other structural barriers to healthcare access exist in relation to the rural–urban divide. Less than 10% of US healthcare resources are located in rural areas where approximately 20% of the American population resides [ 14 ]. In a country with substantially fewer providers per capita compared to many other developed countries, persons in rural areas experience uniquely pressing healthcare provider shortages [ 15 , 16 ]. Rural inhabitants also tend to have lower household income, higher rates of un- or under-insurance, and more difficulty with travel to healthcare clinics than urban dwellers [ 17 ]. Subsequently, persons in rural communities use healthcare services at lower rates, and potentially preventable hospitalizations are more prevalent [ 18 ]. This disparity often leads rural residents to use services primarily for more urgent needs and less so for routine care [ 19 , 20 , 21 ].

The differences in how rural and urban healthcare systems function warranted a federal initiative to focus exclusively on rural health priorities and serve as counterpart to Healthy People objectives [ 22 ]. The rural determinants of health, a more specific expression of general social determinants, add issues of geography and topography to the well-documented social, economic and political factors that influence all Americans’ access to healthcare [ 23 ]. As a result, access is consistently regarded as a top priority in rural areas, and many research efforts have explored the intersection between access and rurality, namely within its less understood dimensions (acceptability and appropriateness) [ 22 ].

Acceptability-related barriers to care

Acceptability represents the dimension of healthcare access that affects a patient’s ability to seek healthcare, particularly linked to one’s professional values, norms and culture [ 2 ]. Access to health information is an influential factor for acceptable healthcare and is essential to promote and maintain a healthy population [ 24 ]. According to the Centers for Disease Control and Prevention, health literacy or a high ‘health IQ’ is the degree to which individuals have the ability to find, understand, and use information and services to inform health-related decisions and actions for themselves and others, which impacts healthcare use and system navigation [ 25 ]. The literature indicates that lower levels of health literacy contribute to health disparities among rural populations [ 26 , 27 , 28 ]. Evidence points to a need for effective health communication between healthcare organizations and patients to improve health literacy [ 24 ]. However, little research has been done in this area, particularly as it relates to technologically-based interventions to disseminate health information [ 29 ].

Stigma, an undesirable position of perceived diminished status in an individual’s social position, is another challenge that influences healthcare acceptability [ 30 ]. Those who may experience stigma fear negative social consequences in relation to care seeking. They are more likely to delay seeking care, especially among ethnic minority populations [ 31 , 32 ]. Social media presents opportunities for the dissemination of misleading medical information; this runs further risk for stigma [ 33 ]. Stigma is difficult to undo, but research has shown that developing a positive relationship with a healthcare provider or organization can work to reduce stigma among patients, thus promoting healthcare acceptability [ 34 ].

A provider’s attempts to engage patients and empower them to be active decision-makers regarding their treatment has also been shown to improve healthcare acceptability. One study found that patients with heart disease who completed a daily diary of weight and self-assessment of symptoms, per correspondence with their provider, had better care outcomes than those who did not [ 35 ]. Engaging with household family members and involved community healers also mitigates barriers to care, emphasizing the importance of a team-based approach that extends beyond those who typically provide healthcare services [ 36 , 37 ]. One study, for instance, explored how individuals closest to a pregnant woman affect the woman’s decision to seek maternity care; partners, female relatives, and community health-workers were among the most influential in promoting negative views, all of which reduced a woman’s likelihood to access care [ 38 ].

Appropriateness-related barriers to care

Appropriateness marks the dimension of healthcare access that affects a patient’s ability to engage, and according to Levesque et al., is of relevance once all other dimensions (the ability to perceive, seek, reach and pay for) are achieved [ 2 ]. The ability to engage in healthcare is influenced by a patient’s level of empowerment, adherence to information, and support received by their healthcare provider. Thus, barriers to healthcare access that relate to appropriateness are often those that indicate a breakdown in communication between a patient with their healthcare provider. Such breakdown can involve a patient experiencing miscommunication, confrontation, and/or a discrepancy between their provider’s goals and their own goals for healthcare. Appropriateness represents a dimension of healthcare access that is widely acknowledged as an area in need of improvement, which indicates a need to rethink how healthcare providers and organizations can adapt to serve the healthcare needs of their communities [ 39 ]. This is especially true for rural, ethnic minority populations, which disproportionately experience an abundance of other barriers to healthcare access. Culturally appropriate care is especially important for members of minority populations [ 40 , 41 , 42 ]. Ultimately, patients value a patient-provider relationship characterized by a welcoming, non-judgmental atmosphere [ 43 , 44 ]. In rural settings especially, level of trust and familiarity are common factors that affect service utilization [ 45 ]. Evidence suggests that kind treatment by a healthcare provider who promotes patient-centered care can have a greater overall effect on a patient’s experience than a provider’s degree of medical knowledge or use of modern equipment [ 46 ]. Of course, investing the time needed to nurture close and caring interpersonal connections is particularly difficult in under-resourced, time-pressured rural health systems [ 47 , 48 ].

The most effective way to evaluate access to healthcare largely depends on which dimensions are explored. For instance, a population-based survey can be used to measure the barrier of healthcare affordability. Survey questions can inquire directly about health insurance coverage, care-related financial burden, concern about healthcare costs, and the feared financial impacts of illness and/or disability. Many national organizations have employed such surveys to measure affordability-related barriers to healthcare. For example, a question may ask explicitly about financial concerns: ‘If you get sick or have an accident, how worried are you that you will not be able to pay your medical bills?’ [ 49 ]. Approachability and availability dimensions of access are also studied using quantitative analysis of survey questions, such as ‘Is there a place that you usually go to when you are sick or need advice about your health?’ or ‘Have you ever delayed getting medical care because you couldn’t get through on the telephone?’ In contrast, the remaining two dimensions–acceptability and appropriateness–require a qualitative approach, as the social and cultural factors that determine a patient’s likelihood of accepting aspects of the services that are to be received (acceptability) and the fit between those services and the patient’s specific healthcare needs (appropriateness) can be more abstract [ 50 , 51 ]. In social science, qualitative methods are appropriate to generate knowledge of what social events mean to individuals and how those individuals interact within them; these methods allow for an exploration of depth rather than breadth [ 52 , 53 ]. Qualitative methods, therefore, are appropriate tools for understanding the depth of healthcare providers’ experiences in the inherently social context of seeking and engaging in healthcare.

In sum, acceptability- and appropriateness-related barriers to healthcare access are multi-layered, complex and abundant. Ensuring access becomes even more challenging if structural barriers to access are factored in. In this study, we aimed to explore barriers to healthcare access among persons in Montana, a historically underserved, under-resourced, rural region of the US. Montana is the fourth largest and third least densely populated state in the country; more than 80% of Montana counties are classified as non-core (the lowest level of urban/rural classification), and over 90% are designated as health professional shortage areas [ 54 , 55 ]. Qualitative methods supported our inquiry to explore barriers to healthcare access related to acceptability and appropriateness.

Participants

Qualitative methods were utilized for this interpretive, exploratory study because knowledge regarding barriers to healthcare access within Montana’s rural health systems is limited. We chose Montana healthcare providers, rather than patients, as the population of interest so we may explore barriers to healthcare access from the perspective of those who serve many persons in rural settings. Inclusion criteria required study participants to provide direct healthcare to patients at least one-half of their time. We defined ‘provider’ as a healthcare organization employee with clinical decision-making power and the qualifications to develop or revise patients’ treatment plans. In an attempt to capture a group of providers with diverse experience, we included providers across several types and specialties. These included advanced practice registered nurses (APRNs), physicians (MDs and DOs), and physician assistants (PAs) who worked in critical care medicine, emergency medicine, family medicine, hospital medicine, internal medicine, pain medicine, palliative medicine, pediatrics, psychiatry, and urgent care medicine. We also included licensed clinical social workers (LCSWs) and clinical psychologists who specialize in behavioral healthcare provision.

Recruitment and Data Collection

We recruited participants via email using a snowball sampling approach [ 56 ]. We opted for this approach because of its effectiveness in time-pressured contexts, such as the COVID-19 pandemic, which has made healthcare provider populations hard to reach [ 57 ]. Considering additional constraints with the pandemic and the rural nature of Montana, interviews were administered virtually via Zoom video or telephone conferencing with Zoom’s audio recording function enabled. All interviews were conducted by the first author between January and September 2021. The average length of interviews was 50 min, ranging from 35 to 70 min. There were occasional challenges experienced during interviews (poor cell phone reception from participants, dropped calls), in which case the interviewer remained on the line until adequate communication was resumed. All interviews were included for analysis and transcribed verbatim into NVivo Version 12 software. All qualitative data were saved and stored on a password-protected University of Montana server. Hard-copy field notes were securely stored in a locked office on the university’s main campus.

Data analysis included a deductive followed by an inductive approach. This dual analysis adheres to Levesque’s framework for qualitative methods, which is discussed in the Definition of Analytic Domains sub-section below. Original synthesis of the literature informed the development of our initial deductive codebook. The deductive approach was derived from a theory-driven hypothesis, which consisted of synthesizing previous research findings regarding acceptability- and appropriateness-related barriers to care. Although the locations, patient populations and specific type of healthcare services varied by study in the existing literature, several recurring barriers to healthcare access were identified. We then operationalized three analytic domains based on these findings: cultural considerations, patient-provider communication, and provider-provider communication. These domains were chosen for two reasons: 1) the terms ‘culture’ and ‘communication’ were the most frequently documented characteristics across the studies examined, and 2) they each align closely with the acceptability and appropriateness dimensions of access to healthcare, respectively. In addition, ‘culture’ is included in the definition of acceptability and ‘communication’ is a quintessential aspect of appropriateness. These domains guided the deductive portion of our analysis, which facilitated the development of an interview guide used for this study.

Interviews were semi-structured to allow broad interpretations from participants and expand the open-ended characterization of study findings. Data were analyzed through a flexible coding approach proposed by Deterding and Waters [ 58 ]. Qualitative content analysis was used, a method particularly beneficial for analyzing large amounts of qualitative data collected through interviews that offers possibility of quantifying categories to identify emerging themes [ 52 , 59 ]. After fifty percent of data were analyzed, we used an inductive approach as a formative check and repeated until data saturation, or the point at which no new information was gathered in interviews [ 60 ]. At each point of inductive analysis, interview questions were added, removed, or revised in consideration of findings gathered [ 61 ]. The Standards for Reporting Qualitative Research (SRQR) was used for reporting all qualitative data for this study [ 62 ]. The first and third authors served as primary and secondary analysts of the qualitative data and collaborated to triangulate these findings. An audit approach was employed, which consisted of coding completed by the first author and then reviewed by the third author. After analyses were complete, member checks ensured credibility and trustworthiness of findings [ 63 ]. Member checks consisted of contacting each study participant to explain the study’s findings; one-third of participants responded and confirmed all findings. All study procedures were reviewed and approved by the Human Subjects Committee of the authors’ institution’s Institutional Review Board.

Definitions of Analytic Domains

Cultural considerations.

Western health systems often fail to consider aspects of patients’ cultural perspectives and histories. This can manifest in the form of a providers’ lack of cultural humility. Cultural humility is a process of preventing imposition of one’s worldview and cultural beliefs on others and recognizing that everyone’s conception of the world is valid. Humility cultivates sensitive approaches in treating patients [ 64 ]. A lack of cultural humility impedes the delivery of acceptable and appropriate healthcare [ 65 ], which can involve low empathy or respect for patients, or dismissal of culture and traditions as superstitions that interfere with standard treatments [ 66 , 67 ]. Ensuring cultural humility among all healthcare employees is a step toward optimal healthcare delivery. Cultural humility is often accomplished through training that can be tailored to particular cultural- or gender-specific populations [ 68 , 69 ]. Since cultural identities and humility have been marked as factors that can heavily influence patients’ access to care, cultural considerations composed our first analytic domain. To assess this domain, we asked participants how they address the unique needs of their patients, how they react when they observe a cultural behavior or attitude from a patient that may not directly align with their treatment plan, and if they have received any multicultural training or training on cultural considerations in their current role.

Patient-provider communication

Other barriers to healthcare access can be linked to ineffective patient-provider communication. Patients who do not feel involved in healthcare decisions are less likely to adhere to treatment recommendations [ 70 ]. Patients who experience communication difficulties with providers may feel coerced, which generates disempowerment and leads patients to employ more covert ways of engagement [ 71 , 72 ]. Language barriers can further compromise communication and hinder outcomes or patient progress [ 73 , 74 ]. Any miscommunication between a patient and provider can affect one’s access to healthcare, namely affecting appropriateness-related barriers. For these reasons, patient-provider communication composed our second analytic domain. We asked participants to highlight the challenges they experience when communicating with their patients, how those complications are addressed, and how communication strategies inform confidentiality in their practice. Confidentiality is a core ethical principle in healthcare, especially in rural areas that have smaller, interconnected patient populations [ 75 ].

Provider-Provider Communication

A patient’s journey through the healthcare system necessitates sufficient correspondence between patients, primary, and secondary providers after discharge and care encounters [ 76 ]. Inter-provider and patient-provider communication are areas of healthcare that are acknowledged to have some gaps. Inconsistent mechanisms for follow up communication with patients in primary care have been documented and emphasized as a concern among those with chronic illness who require close monitoring [ 68 , 77 ]. Similar inconsistencies exist between providers, which can lead to unclear care goals, extended hospital stays, and increased medical costs [ 78 ]. For these reasons, provider-provider communication composed our third analytic domain. We asked participants to describe the approaches they take to streamline communication after a patient’s hospital visit, the methods they use to ensure collaborative communication between primary or secondary providers, and where communication challenges exist.

Healthcare provider characteristics

Our sample included 12 providers: four in family medicine (1 MD, 1 DO, 1 PA & 1 APRN), three in pediatrics (2 MD with specialty in hospital medicine & 1 DO), three in palliative medicine (2 MDs & 1 APRN with specialty in wound care), one in critical care medicine (DO with specialty in pediatric pulmonology) and one in behavioral health (1 LCSW with specialty in trauma). Our participants averaged 9 years (range 2–15) as a healthcare provider; most reported more than 5 years in their current professional role. The diversity of participants extended to their patient populations as well, with each participant reporting a unique distribution of age, race and level of medical complexity among their patients. Most participants reported that a portion of their patients travel up to five hours, sometimes across county- or state-lines, to receive care.

Theme 1: A friction exists between aspects of patients’ rural identities and healthcare systems

Our participants comprised a collection of medical professions and reported variability among health-related reasons their patients seek care. However, most participants acknowledged similar characteristics that influence their patients’ challenges to healthcare access. These identified factors formed categories from which the first theme emerged. There exists a great deal of ‘rugged individualism’ among Montanans, which reflects a self-sufficient and self-reliant way of life. Stoicism marked a primary factor to characterize this quality. One participant explained:

True Montanans are difficult to treat medically because they tend to be a tough group. They don’t see doctors. They don’t want to go, and they don’t want to be sick. That’s an aspect of Montana that makes health culture a little bit difficult.

Another participant echoed this finding by stating:

The backwoods Montana range guy who has an identity of being strong and independent probably doesn’t seek out a lot of medical care or take a lot of medications. Their sense of vitality, independence and identity really come from being able to take care and rely on themselves. When that is threatened, that’s going to create a unique experience of illness.

Similar responses were shared by all twelve participants; stoicism seemed to be heavily embedded in many patient populations in Montana and serves as a key determinant of healthcare acceptability. There are additional factors, however, that may interact with stoicism but are multiply determined. Stigma is an example of this, presented in this context as one’s concern about judgement by the healthcare system. Respondents were openly critical of this perception of the healthcare system as it was widely discussed in interviews. One participant stated:

There is a real perception of a punitive nature in the medical community, particularly if I observe a health issue other than the primary reason for one’s hospital visit, whether that may be predicated on medical neglect, delay of care, or something that may warrant a report to social services. For many of the patients and families I see, it’s not a positive experience and one that is sometimes an uphill barrier that I work hard to circumnavigate.

Analysis of these factors suggest that low use of healthcare services may link to several characteristics, including access problems. Separately, a patient’s perceived stigma from healthcare providers may also impact a patient’s willingness to receive services. One participant put it best by stating

Sometimes, families assume that I didn’t want to see them because they will come in for follow up to meet with me but end up meeting with another provider, which is frustrating because I want to maintain patients on my panel but available time and resource occasionally limits me from doing so. It could be really hard adapting to those needs on the fly, but it’s an honest miss.

When a patient arrives for a healthcare visit and experiences this frustration, it may elicit a patient’s perceptions of neglect or disorganization. This ‘honest miss’ may, in turn, exacerbate other acceptable-related barriers to care.

Theme 2: Facilitating access to healthcare requires application of and respect for cultural differences

The biomedical model is the standard of care utilized in Western medicine [ 79 , 80 ]. However, the US comprises people with diverse social and cultural identities that may not directly align with Western conceptions of health and wellness. Approximately 11.5% of the Montana population falls within an ethnic minority group. 6.4% are of American Indian or Alaska Native origin, 0.5% are of Black or African American origin, 0.8% are of Asian origin and 3.8% are of multiple or other origins. [ 81 ]. Cultural insensitivity is acknowledged in health services research as an active deterrent for appropriate healthcare delivery [ 65 ]. Participants for this study were asked how they react when a patient brings up a cultural attitude or behavior that may impact the proposed treatment plan. Eight participants noted a necessity for humility when this occurs. One participant conceptualized this by stating:

When this happens, I learn about individuals and a way of life that is different to the way I grew up. There is a lot of beauty and health in a non-patriarchal, non-dominating, non-sexist framework, and when we can engage in such, it is really expansive for my own learning process.

The participants who expressed humility emphasized that it is best to work in tandem with their patient, congruently. Especially for those with contrasting worldviews, a provider and a patient working as a team poses an opportunity to develop trust. Without it, a patient can easily fall out of the system, further hindering their ability to access healthcare services in the future. One participant stated:

The approach that ends up being successful for a lot of patients is when we understand their modalities, and they have a sense we understand those things. We have to show understanding and they have to trust. From there, we can make recommendations to help get them there, not decisions for them to obey, rather views based on our experiences and understanding of medicine.

Curiosity was another reaction noted by a handful of participants. One participant said:

I believe patients and their caregivers can be engaged and loving in different ways that don’t always follow the prescribed approach in the ways I’ve been trained, but that doesn’t necessarily mean that they are detrimental. I love what I do, and I love learning new things or new approaches, but I also love being surprised. My style of medicine is not to predict peoples’ lives, rather to empower and support what makes life meaningful for them.

Participants mentioned several other characteristics that they use in practice to prevent cultural insensitivity and support a collaborative approach to healthcare. Table 1 lists these facilitating characteristics and quotes to explain the substance of their benefit.

Consensus among participants indicated that the use of these protective factors to promote cultural sensitivity and apply them in practice is not standardized. When asked, all but two participants said they had not received any culturally-based training since beginning their practice. Instead, they referred to developing skills through “on the job training” or “off the cuff learning.” The general way of medicine, one participant remarked, was to “throw you to the fire.” This suggested that use of standardized cultural humility training modules for healthcare providers was not common practice. Many attributed this to time constraints.

Individual efforts to gain culturally appropriate skills or enhance cultural humility were mentioned, however. For example, three participants reported that they attended medical conferences to discuss cultural challenges within medicine, one participant sought out cultural education within their organization, and another was invited by Native American community members to engage in traditional peace ceremonies. Participants described these additional efforts as uncommon and outside the parameters of a provider’s job responsibilities, as they require time commitments without compensation.

Additionally, eight participants said they share their personal contact information with patients so they may call them directly for medical needs. The conditions and frequency with which this is done was variable and more common among providers in specialized areas of medicine or those who described having a manageable patient panel. All who reported that they shared their personal contact information described it as an aspect of rural health service delivery that is atypical in other, non-rural healthcare systems.

Theme 3: Communication between healthcare providers is systematically fragmented

Healthcare is complex and multi-disciplinary, and patients’ treatment is rarely overseen by a single provider [ 82 ]. The array of provider types and specialties is vast, as is the range of responsibilities ascribed to providers. Thus, open communication among providers both within and between healthcare systems is vital for the success of collaborative healthcare [ 83 ]. Without effective communication achieved between healthcare providers, the appropriate delivery of healthcare services may be become compromised. Our participants noted that they face multiple challenges that complicate communication with other providers. Miscommunication between departments, often implicating the Emergency Department (ED), was a recurring point noted among participants. One participant who is a primary care physician said:

If one of my patients goes to the ER, I don’t always get the notes. They’re supposed to send them to the patient’s primary care doc. The same thing happens with general admissions, but again, I often find out from somebody else that my patient was admitted to the hospital.

This failure to communicate can negatively impact the patient, particularly if time sensitivity or medical complexity is essential to treatment. A patient’s primary care physician is the most accurate source of their medical history; without an effective way to obtain and synthesize a patient’s health information, there may be increased risk of medical error. One participant in a specialty field stated:

One of the biggest barriers I see is obtaining a concise description of a patient’s history and needs. You can imagine if you’re a mom and you’ve got a complicated kid. You head to the ER. The ER doc looks at you with really wide eyes, not knowing how to get information about your child that’s really important.

This concern was highlighted with a specific example from a different participant:

I have been unable to troubleshoot instances when I send people to the ER with a pretty clear indication for admission, and then they’re sent home. For instance, I had an older fellow with pretty severe chronic kidney disease. He presented to another practitioner in my office with shortness of breath and swelling and appeared to have newly onset decompensated heart failure. When I figured this out, I sent him to the ER, called and gave my report. The patient later came back for follow up to find out not only that they had not been admitted but they lost no weight with outpatient dialysis . I feel like a real opportunity was missed to try to optimize the care of the patient simply because there was poor communication between myself and the ER. This poor guy… He ended up going to the ER four times before he got admitted for COVID-19.

In some cases, communication breakdown was reported as the sole cause of a poor outcome. When communication is effective, each essential member of the healthcare team is engaged and collaborating with the same information. Some participants called this process ‘rounds’ when a regularly scheduled meeting is staged between a group of providers to ensure access to accurate patient information. Accurate communication may also help build trust and improve a patient’s experience. In contrast, ineffective communication can result in poor clarity regarding providers’ responsibilities or lost information. Appropriate delivery of healthcare considers the fit between providers and a patient’s specific healthcare needs; the factors noted here suggest that provider-provider miscommunication can adversely affect this dimension of healthcare access.

Another important mechanism of communication is the sharing of electronic medical records (EMRs), a process that continues to shift with technological advances. Innovation is still recent enough, however, for several of our study participants to be able to recall a time when paper charts were standard. Widespread adoption and embrace of the improvements inherent in electronic medical records expanded in the late 2000’s [ 84 ]. EMRs vastly improved the ability to retain, organize, safeguard, and transfer health information. Every participant highlighted EMRs at one point or another and often did so with an underlying sense of anger or frustration. Systematic issues and problems with EMRs were discussed. One participant provided historical context to such records:

Years back, the government aimed to buy an electronic medical record system, whichever was the best, and a number of companies created their own. Each were a reasonable system, so they all got their checks and now we have four completely separate operating systems that do not talk to each other. The idea was to make a router or some type of relay that can share information back and forth. There was no money in that though, so of course, no one did anything about it. Depending on what hospital, clinic or agency you work for, you will most likely work within one of these systems. It was a great idea; it just didn’t get finished.

Seven participants confirmed these points and their impacts on making coordination more difficult, relying on outdated communication strategies more often than not. Many noted this even occurs between facilities within the same city and in separate small metropolitan areas across the state. One participant said:

If my hospital decides to contract with one EMR and the hospital across town contracts with another, correspondence between these hospitals goes back to traditional faxing. As a provider, you’re just taking a ‘fingered crossed’ approach hoping that the fax worked, is picked up, was put in the appropriate inbox and was actually looked at. Information acquisition and making sure it’s timely are unforeseen between EMRs.

Participants reported an “astronomic” number of daily faxes and telephone calls to complete the communication EMRs were initially designed to handle. These challenges are even more burdensome if a patient moves from out of town or out of state; obtaining their medical records was repeatedly referred to as a “chore” so onerous that it often remains undone. Another recurring concern brought up by participants regarded accuracy within EMRs to lend a false sense of security. They are not frequently updated, not designed to be family-centered and not set up to do anything automatically. One participant highlighted these limitations by stating:

I was very proud of a change I made in our EMR system [EPIC], even though it was one I never should have had to make. I was getting very upset because I would find out from my nursing assistant who read the obituary that one of my patients had died. There was a real problem with the way the EMR was notifying PCP’s, so I got an EPIC-level automated notification built into our EMR so that any time a patient died, their status would be changed to deceased and a notification would be sent to their PCP. It’s just really awful to find out a week later that your patient died, especially when you know these people and their families really well. It’s not good care to have blind follow up.

Whether it be a physical or electronic miscommunication between healthcare providers, the appropriate delivery of healthcare can be called to question

Theme 4: Time and resource constraints disproportionately harm rural health systems

Several measures of system capacity suggest the healthcare system in the US is under-resourced. There are fewer physicians and hospital beds per capita compared to most comparable countries, and the growth of healthcare provider populations has stagnated over time [ 15 ]. Rural areas, in particular, are subject to resource limitations [ 16 ]. All participants discussed provider shortages in detail. They described how shortages impact time allocation in their day-to-day operations. Tasks like patient intakes, critical assessments, and recovering information from EMRs take time, of which most participants claimed to not have enough of. There was also a consensus in having inadequate time to spend on medically complex cases. Time pressures were reported to subsequently influence quality of care. One participant stated:

With the constant pace of medicine, time is not on your side. A provider cannot always participate in an enriching dialogue with their patients, so rather than listen and learn, we are often coerced into the mindset of ‘getting through’ this patient so we can move on. This echoes for patient education during discharge, making the whole process more arduous than it otherwise could be if time and resources were not as sparse.

Depending on provider type, specialty, and the size of patient panels, four participants said they have the luxury of extending patient visits to 40 + minutes. Any flexibility with patient visits was regarded as just that: a luxury. Very few providers described the ability to coordinate their schedules as such. This led some study participants to limit the number of patients they serve. One participant said:

We simply don’t have enough clinicians, which is a shame because these people are really skilled, exceptional, brilliant providers but are performing way below their capacity. Because of this, I have a smaller case load so I can engage in a level of care that I feel is in the best interest of my patients. Everything is a tradeoff. Time has to be sacrificed at one point or another. This compromise sets our system up to do ‘ok’ work, not great work.

Of course, managing an overly large number of patients with high complexity is challenging. Especially while enduring the burden of a persisting global pandemic, participants reflected that the general outlook of administering healthcare in the US is to “do more with less.” This often forces providers to delegate responsibilities, which participants noted has potential downsides. One participant described how delegating patient care can cause problems.

Very often will a patient schedule a follow up that needs to happen within a certain time frame, but I am unable to see them myself. So, they are then placed with one of my mid-level providers. However, if additional health issues are introduced, which often happens, there is a high-risk of bounce-back or need to return once again to the hospital. It’s an inefficient vetting process that falls to people who may not have specific training in the labs and imaging that are often included in follow up visits. Unfortunately, it’s a forlorn hope to have a primary care physician be able to attend all levels of a patient’s care.

Several participants described how time constraints stretch all healthcare staff thin and complicate patient care. This was particularly important among participants who reported having a patient panel exceeding 1000. There were some participants, however, who praised the relationships they have with their nurse practitioners and physician’s assistants and mark transparency as the most effective way to coordinate care. Collectively, these clinical relationships were built over long standing periods of time, a disadvantage to providers at the start of their medical career. All but one participant with over a decade of clinical experience mentioned the usefulness of these relationships. The factors discussed in Theme 4 are directly linked to the Availability dimension of access to healthcare. A patient’s ability to reach care is subject to the capacity of their healthcare provider(s). Additionally, further analysis suggests these factors also link to the Appropriateness dimension because the quality of patient-provider relationships may be negatively impacted if a provider’s time is compromised.

Theme 5: Profits are prioritized over addressing barriers to healthcare access in the US.

The US healthcare system functions partially for-profit in the public and private sectors. The federal government provides funding for national programs such as Medicare, but a majority of Americans access healthcare through private employer plans [ 85 ]. As a result, uninsurance rates influence healthcare access. Though the rate of the uninsured has dropped over the last decade through expansion of the Affordable Care Act, it remains above 8 percent [ 86 ]. Historically, there has been ethical criticism in the literature of a for-profit system as it is said to exacerbate healthcare disparities and constitute unfair competition against nonprofit institutions. Specifically, the US healthcare system treats healthcare as a commodity instead of a right, enables organizational controls that adversely affect patient-provider relationships, undermines medical education, and constitutes a medical-industrial complex that threatens influence on healthcare-related public policy [ 87 ]. Though unprompted by the interviewer, participants raised many of these concerns. One participant shared their views on how priorities stand in their practice:

A lot of the higher-ups in the healthcare system where I work see each patient visit as a number. It’s not that they don’t have the capacity to think beyond that, but that’s what their role is, making sure we’re profitable. That’s part of why our healthcare system in the US is as broken as it is. It’s accentuated focus on financially and capitalistically driven factors versus understanding all these other barriers to care.

Eight participants echoed a similar concept, that addressing barriers to healthcare access in their organizations is largely complicated because so much attention is directed on matters that have nothing to do with patients. A few other participants supported this by alluding to a “cherry-picking” process by which those at the top of the hierarchy devote their attention to the easiest tasks. One participant shared an experience where contrasting work demands between administrators and front-line clinical providers produces adverse effects:

We had a new administrator in our hospital. I had been really frustrated with the lack of cultural awareness and curiosity from our other leaders in the past, so I offered to meet and take them on a tour of the reservation. This was meant to introduce them to kids, families and Tribal leaders who live in the area and their interface with healthcare. They declined, which I thought was disappointing and eye-opening.

Analysis of these factors suggest that those who work directly with patients understand patient needs better than those who serve in management roles. This same participant went on to suggest an ulterior motive for a push towards telemedicine, as administrators primarily highlight the benefit of billing for virtual visits instead of the nature of the visits themselves.

This study explored barriers and facilitators to healthcare access from the perspective of rural healthcare providers in Montana. Our qualitative analysis uncovered five key themes: 1) a friction exists between aspects of patients’ rural identities and healthcare systems; 2) facilitating access to healthcare requires application of and respect for cultural differences; 3) communication between healthcare providers is systematically fragmented; 4) time and resource constraints disproportionately harm rural health systems; and 5) profits are prioritized over addressing barriers to healthcare access in the US. Themes 2 and 3 were directly supported by earlier qualitative studies that applied Levesque’s framework, specifically regarding healthcare providers’ poor interpersonal quality and lack of collaboration with other providers that are suspected to result from a lack of provider training [ 67 , 70 ]. This ties back to the importance of cultural humility, which many previous culture-based trainings have referred to as cultural competence. Cultural competence is achieved through a plethora of trainings designed to expose providers to different cultures’ beliefs and values but induces risk of stereotyping and stigmatizing a patient’s views. Therefore, cultural humility is the preferred idea, by which providers reflect and gain open-ended appreciation for a patient’s culture [ 88 ].

Implications for Practice

Perhaps the most substantial takeaway is how embedded rugged individualism is within rural patient populations and how difficult that makes the delivery of care in rural health systems. We heard from participants that stoicism and perceptions of stigma within the system contribute to this, but other resulting factors may be influential at the provider- and organizational-levels. Stoicism and perceived stigma both appear to arise, in part, from an understandable knowledge gap regarding the care system. For instance, healthcare providers understand the relations between primary and secondary care, but many patients may perceive both concepts as elements of a single healthcare system [ 89 ]. Any issue experienced by a patient when tasked to see both a primary and secondary provider may result in a patient becoming confused [ 90 ]. This may also overlap with our third theme, as a disjointed means of communication between healthcare providers can exacerbate patients’ negative experiences. One consideration to improve this is to incorporate telehealth programs into an existing referral framework to reduce unnecessary interfacility transfers; telehealth programs have proven effective in rural and remote settings [ 91 ].

In fact, telehealth has been rolled out in a variety of virtual platforms throughout its evolution, its innovation matched with continued technological advancement. Simply put, telehealth allows health service delivery from a distance; it allows knowledge and practice of clinical care to be in a different space than a patient. Because of this, a primary benefit of telehealth is its impact on improving patient-centered outcomes among those living in rural areas. For instance, text messaging technology improves early infant diagnosis, adherence to recommended diagnostic testing, and participant engagement in lifestyle change interventions [ 92 , 93 , 94 ]. More sophisticated interventions have found their way into smartphone-based technology, some of which are accessible even without an internet connection [ 95 , 96 ]. Internet accessibility is important because a number of study participants noted internet connectivity as a barrier for patients who live in low resource communities. Videoconferencing is another function of telehealth that has delivered a variety of health services, including those which are mental health-specific [ 97 ], and mobile health clinics have been used in rural, hard-to-reach settings to show the delivery of quality healthcare is both feasible and acceptable [ 98 , 99 , 100 ]. While telehealth has potential to reduce a number of healthcare access barriers, it may not always address the most pressing healthcare needs [ 101 ]. However, telehealth does serve as a viable, cost-effective alternative for rural populations with limited physical access to specialized services [ 102 ]. With time and resource limitations acknowledged as a key theme in our study, an emphasis on expanding telehealth services is encouraged as it will likely have significant involvement on advancing healthcare in the future, especially as the COVID-19 pandemic persists [ 103 ].

Implications for Policy

One could argue that most of the areas of fragmentation in the US healthcare system can be linked to the very philosophy on which it is based: an emphasis on profits as highest priority. Americans are, therefore, forced to navigate a health service system that does not work solely in their best interests. It is not surprising to observe lower rates of healthcare usage in rural areas, which may be a result from rural persons’ negative views of the US healthcare system or a perception that the system does not exist to support wellness. These perceptions may interact with ‘rugged individualism’ to squelch rural residents’ engagement in healthcare. Many of the providers we interviewed for this study appeared to understand this and strived to improve their patients’ experiences and outcomes. Though these efforts are admirable, they may not characterize all providers who serve in rural areas of the US. From a policy standpoint, it is important to recognize these expansive efforts from providers. If incentives were offered to encourage maximum efforts be made, it may lessen burden due to physician burnout and fatigue. Of course, there is no easy fix to the persisting limit of time and resources for providers, problems that require workforce expansion. Ultimately, though, the current structure of the US healthcare system is failing rural America and doing little to help the practice of rural healthcare providers.

Implications for Future Research

It is important for future health systems research efforts to consider issues that arise from both individual- and system-level access barriers and where the two intersect. Oftentimes, challenges that appear linked to a patient or provider may actually stem from an overarching system failure. If failures are critically and properly addressed, we may refine our understanding of what we can do in our professional spaces to improve care as practitioners, workforce developers, researchers and advocates. This qualitative study was exploratory in nature. It represents a step forward in knowledge generation regarding challenges in access to healthcare for rural Americans. Although mental health did not come up by design in this study, future efforts exploring barriers to healthcare access in rural systems should focus on access to mental healthcare. In many rural areas, Montana included, rates of suicide, substance use and other mental health disorders are highly prevalent. These characteristics should be part of the overall discussion of access to healthcare in rural areas. Optimally, barriers to healthcare access should continue to be explored through qualitative and mixed study designs to honor its multi-dimensional stature.

Strengths and Limitations

It is important to note first that this study interviewed healthcare providers instead of patients, which served as both a strength and limitation. Healthcare providers were able to draw on numerous patient-provider experiences, enabling an account of the aggregate which would have been impossible for a patient population. However, accounts of healthcare providers’ perceptions of barriers to healthcare access for their patients may differ from patients’ specific views. Future research should examine acceptability- and appropriateness-related barriers to healthcare access in patient populations. Second, study participants were recruited through convenience sampling methods, so results may be biased towards healthcare providers who are more invested in addressing barriers to healthcare access. Particularly, the providers interviewed for this study represented a subset who go beyond expectations of their job descriptions by engaging with their communities and spending additional uncompensated time with their patients. It is likely that a provider who exhibits these behavioral traits is more likely to participate in research aimed at addressing barriers to healthcare access. Third, the inability to conduct face-to-face interviews for our qualitative study may have posed an additional limitation. It is possible, for example, that in-person interviews might have resulted in increased rapport with study participants. Notwithstanding this possibility, the remote interview format was necessary to accommodate health risks to the ongoing COVID-19 pandemic. Ultimately, given our qualitative approach, results from our study cannot be generalizable to all rural providers’ views or other rural health systems. In addition, no causality can be inferred regarding the influence of aspects of rurality on access. The purpose of this exploratory qualitative study was to probe research questions for future efforts. We also acknowledge the authors’ roles in the research, also known as reflexivity. The first author was the only author who administered interviews and had no prior relationships with all but one study participant. Assumptions and pre-dispositions to interview content by the first author were regularly addressed throughout data analysis to maintain study integrity. This was achieved by conducting analysis by unique interview question, rather than by unique participant, and recoding the numerical order of participants for each question. Our commitment to rigorous qualitative methods was a strength for the study for multiple reasons. Conducting member checks with participants ensured trustworthiness of findings. Continuing data collection to data saturation ensured dependability of findings, which was achieved after 10 interviews and confirmed after 2 additional interviews. We further recognize the heterogeneity in our sample of participants, which helped generate variability in responses. To remain consistent with appropriate means of presenting results in qualitative research however, we shared minimal demographic information about our study participants to ensure confidentiality.

The divide between urban and rural health stretches beyond a disproportionate allocation of resources. Rural health systems serve a more complicated and hard-to-reach patient population. They lack sufficient numbers of providers to meet population health needs. These disparities impact collaboration between patients and providers as well as the delivery of acceptable and appropriate healthcare. The marker of rurality complicates the already cumbersome challenge of administering acceptable and appropriate healthcare and impediments stemming from rurality require continued monitoring to improve patient experiences and outcomes. Our qualitative study explored rural healthcare providers’ views on some of the social, cultural, and programmatic factors that influence access to healthcare among their patient populations. We identified five key themes: 1) a friction exists between aspects of patients’ rural identities and healthcare systems; 2) facilitating access to healthcare requires application of and respect for cultural differences; 3) communication between healthcare providers is systematically fragmented; 4) time and resource constraints disproportionately harm rural health systems; and 5) profits are prioritized over addressing barriers to healthcare access in the US. This study provides implications that may shift the landscape of a healthcare provider’s approach to delivering healthcare. Further exploration is required to understand the effects these characteristics have on measurable patient-centered outcomes in rural areas.

Availability of data and materials

The datasets generated and/or analyzed during the current study are not publicly available due to individual privacy could be compromised but are available from the corresponding author on reasonable request.

Ethics approval and consent to participate.

All study procedures and methods were carried out in accordance with relevant guidelines and regulations from the World Medical Association Declaration of Helsinki. Ethics approval was given by exempt review from the Institutional Review Board (IRB) at the University of Montana (IRB Protocol No.: 186–20). Participants received oral and written information about the study prior to interview, which allowed them to provide informed consent for the interviews to be recorded and used for qualitative research purposes. No ethical concerns were experienced in this study pertaining to human subjects.

Consent for publication.

The participants consented to the publication of de-identified material from the interviews.

Riley WJ. Health Disparities: Gaps in Access, Quality and Affordability of Medical Care. Trans Am Clin Climatol Assoc. 2012;123:167–74.

PubMed   PubMed Central   Google Scholar  

Levesque J, Harris MF, Russell G. Patient-centred access to health care: conceptualising access at the interface of health systems and populations. Int J Equity Health. 2013;12:18. https://doi.org/10.1186/1475-9276-12-18 .

Article   PubMed   PubMed Central   Google Scholar  

Serban N. A Multidimensional Framework for Measuring Access. In: Serban N. Healthcare System Access: Measurement, Inference, and Intervention. New Jersey: John Wiley & Sons; 2019. p. 13–59.

Google Scholar  

Roncarolo F, Boivin A, Denis JL, et al. What do we know about the needs and challenges of health systems? A scoping review of the international literature. BMC Health Serv Res. 2017;17:636. https://doi.org/10.1186/s12913-017-2585-5 .

Cabrera-Barona P, Blaschke T. Kienberger S. Explaining Accessibility and Satisfaction Related to Healthcare: A Mixed-Methods Approach. Soc Indic Res. 2017;133,719–739. https://doi.org/10.1007/s11205-016-1371-9 .

Coombs NC, Meriwether WE, Caringi J, Newcomer SR. Barriers to healthcare access among U.S. adults with mental health challenges: A population-based study. SSM Popul Health. 2021;15:100847. https://doi.org/10.1016/j.ssmph.2021.100847 .

Farietta TP, Lu B, Tumin R. Ohio’s Medicaid Expansion and Unmet Health Needs Among Low-Income Women of Reproductive Age. Matern Child Health. 2018;22(12):1771–9. https://doi.org/10.1007/s10995-018-2575-1 .

Article   Google Scholar  

Sommers BD, Blendon RJ, Orav EJ, Epstein AM. Changes in Utilization and Health Among Low-Income Adults After Medicaid Expansion or Expanded Private Insurance. JAMA Intern Med. 2016;1;176(10):1501–1509. https://doi.org/10.1001/jamainternmed.2016.4419 .

National Alliance on Mental Illness: The Doctor is Out. Continuing Disparities in Access to Mental and Physical Health Care. 2017. https://www.nami.org/Support-Education/Publications-Reports/Public-Policy-Reports/The-Doctor-is-Out . Accessed 29 Aug 2021.

Pollitz K, Lopes L, Kearney A, Rae M, Cox C, Fehr F, Rousseau D, Kaiser Family Foundation. US Statistics on Surprise Medical Billing. JAMA. 2020;323(6):498. https://doi.org/10.1001/jama.2020.0065 .

Kahraman C, Orobello C, Cirella GT. hanging Dynamics with COVID-19: Future Outlook. In: XX Cirella GT, editor. Human Settlements: Urbanization, Smart Sector Development, and Future Outlook. Singapore: Springer; 2022. p. 235–52.

Book   Google Scholar  

Blumenthal D, Fowler EJ, Abrams M, Collins SR. Covid-19 — Implications for the Health Care System. N Engl J Med. 2020;383(15):1483–8. https://doi.org/10.1056/NEJMsb2021088 .

Article   CAS   PubMed   Google Scholar  

Defêche J, Azarzar S, Mesdagh A, Dellot P, Tytgat A, Bureau F, Gillet L, Belhadj Y, Bontems S, Hayette M-P, Schils R, Rahmouni S, Ernst M, Moutschen M, Darcis G. In-Depth Longitudinal Comparison of Clinical Specimens to Detect SARS-CoV-2. Pathogens. 2021;10(11):1362. https://doi.org/10.3390/pathogens10111362 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Rosenblatt RA, Hart LG. Physicians and rural America. West J Med. 2000;173(5):348–51. https://doi.org/10.1136/ewjm.173.5.348 .

Kaiser Family Foundation Analysis of OECD Data. 2020. https://www.healthsystemtracker.org/chart-collection/u-s-health-care-resources-compare-countries/#item-physicians-density-per-1000-population-2000-2018 . Accessed 31 Oct 2021.

Government Accounting Office. Physician Workforce: Locations and Types of Graduate Training Were Largely Unchanged, and Federal Efforts May Not Be Sufficient to Meet Needs. 2017. https://www.gao.gov/assets/gao-17-411.pdf . Accessed 31 Oct 2021.

The Kaiser Family Foundation: The Uninsured in Rural America. 2003. https://www.kff.org/wp-content/uploads/2013/01/the-uninsured-in-rural-america-update-pdf.pdf . Accessed 15 Sept 2021.

Wright B, Potter AJ, Trivedi AN, Mueller KJ. The Relationship Between Rural Health Clinic Use and Potentially Preventable Hospitalizations and Emergency Department Visits Among Medicare Beneficiaries. J Rural Health. 2018;34(4):423–30. https://doi.org/10.1111/jrh.12253 .

Article   PubMed   Google Scholar  

Weichelt B, Bendixsen C, Patrick T. A Model for Assessing Necessary Conditions for Rural Health Care’s Mobile Health Readiness: Qualitative Assessment of Clinician-Perceived Barriers. JMIR Mhealth Uhealth. 2019;7(11): e11915. https://doi.org/10.2196/11915 .

Mangundu M, Roets L, Janse van Rensberg E. Accessibility of healthcare in rural Zimbabwe: The perspective of nurses and healthcare users. Afr J Prim Health Care Fam Med. 2020;12(1):e1-e7. https://doi.org/10.4102/phcfm.v12i1.2245 .

Rahayu YYS, Araki T, Rosleine D. Factors affecting the use of herbal medicines in the universal health coverage system in Indonesia. J Ethnopharmacol. 2020;260: 112974. https://doi.org/10.1016/j.jep.2020.112974 .

Bolin JN, Bellamy GR, Ferdinand AO, et al. Rural Healthy People 2020: New Decade, Same Challenges. Journal of Rural Health. 2015;31(3):326–33.

Reid S. The rural determinants of health: using critical realism as a theoretical framework. Rural Remote Health. 2019;19(3):5184. https://doi.org/10.22605/RRH5184 .

Wynia MK, Osborn CY. Health literacy and communication quality in health care organizations. J Health Commun. 2010;15 Suppl 2(Suppl 2):102–115. https://doi.org/10.1080/10810730.2010.499981 .

Centers for Disease Control and Prevention. What is Health Literacy? 2022. https://www.cdc.gov/healthliteracy/learn/index.html . Accessed 25 Feb 2022.

Davis TC, Arnold CL, Rademaker A, et al. Differences in barriers to mammography between rural and urban women. J Womens Health (Larchmt). 2012;21(7):748–55. https://doi.org/10.1089/jwh.2011.3397 .

Halverson J, Martinez-Donate A, Trentham-Dietz A, et al. Health literacy and urbanicity among cancer patients. J Rural Health. 2013;29(4):392–402. https://doi.org/10.1111/jrh.12018 .

Zahnd WE, Scaife SL, Francis ML. Health literacy skills in rural and urban populations. Am J Health Behav. 2009;33(5):550–7. https://doi.org/10.5993/ajhb.33.5.8 .

Dogba MJ, Dossa AR, Breton E, Gandonou-Migan R. Using information and communication technologies to involve patients and the public in health education in rural and remote areas: a scoping review. BMC Health Serv Res. 2019;19(1):128. https://doi.org/10.1186/s12913-019-3906-7 .

Committee on the Science of Changing Behavioral Health Social Norms, Board on Behavioral, Cognitive, and Sensory Sciences, Division of Behavioral and Social Sciences and Education; National Academies of Sciences, Engineering, and Medicine. In: Ending Discrimination Against People with Mental and Substance Use Disorders: The Evidence for Stigma Change. Washington: National Academics Press; 2016. p. 33–52.

Wu Y, Zhou H, Wang Q, Cao M, Medina A, Rozelle S. Use of maternal health services among women in the ethnic rural areas of western China. BMC Health Serv Res. 2019;19(1):179. https://doi.org/10.1186/s12913-019-3996-2 .

Meyer E, Hennink M, Rochat R, et al. Working Towards Safe Motherhood: Delays and Barriers to Prenatal Care for Women in Rural and Peri-Urban Areas of Georgia. Matern Child Health J. 2016;20(7):1358–65. https://doi.org/10.1007/s10995-016-1997-x .

Heinrich S. Medical science faces the post-truth era: a plea for the grassroot values of science. Curr Opin Anaesthesiol. 2020;33(2):198–202. https://doi.org/10.1097/ACO.0000000000000833 .

Esquivel MM, Chen JC, Woo RK, et al. Why do patients receive care from a short-term medical mission? Survey study from rural Guatemala. J Surg Res. 2017;215:160–6. https://doi.org/10.1016/j.jss.2017.03.056 .

Park LG, Dracup K, Whooley MA, et al. Symptom Diary Use and Improved Survival for Patients With Heart Failure [published correction appears in Circ Heart Fail. 2017 Dec;10(12):]. Circ Heart Fail. 2017;10(11):e003874. https://doi.org/10.1161/CIRCHEARTFAILURE.117.003874 .

Taleb F, Perkins J, Ali NA, et al. Transforming maternal and newborn health social norms and practices to increase utilization of health services in rural Bangladesh: a qualitative review. BMC Pregnancy Childbirth. 2015;15:75. https://doi.org/10.1186/s12884-015-0501-8 .

Billah SM, Hoque DE, Rahman M, et al. Feasibility of engaging “Village Doctors” in the Community-based Integrated Management of Childhood Illness (C-IMCI): experience from rural Bangladesh. J Glob Health. 2018;8(2): 020413. https://doi.org/10.7189/jogh.08.020413 .

Kaiser JL, Fong RM, Hamer DH, et al. How a woman’s interpersonal relationships can delay care-seeking and access during the maternity period in rural Zambia: An intersection of the Social Ecological Model with the Three Delays Framework. Soc Sci Med. 2019;220:312–21. https://doi.org/10.1016/j.socscimed.2018.11.011 .

Rao KD, Sheffel A. Quality of clinical care and bypassing of primary health centers in India. Soc Sci Med. 2018;207:80–8. https://doi.org/10.1016/j.socscimed.2018.04.040 .

Lee YT, Lee YH, Kaplan WA. Is Taiwan’s National Health Insurance a perfect system? Problems related to health care utilization of the aboriginal population in rural townships. Int J Health Plann Manage. 2019;34(1):e6–10. https://doi.org/10.1002/hpm.2653 .

Lyford M, Haigh MM, Baxi S, Cheetham S, Shahid S, Thompson SC. An Exploration of Underrepresentation of Aboriginal Cancer Patients Attending a Regional Radiotherapy Service in Western Australia. Int J Environ Res Public Health. 2018;15(2):337. https://doi.org/10.3390/ijerph15020337 .

Article   PubMed Central   Google Scholar  

Rohr JM, Spears KL, Geske J, Khandalavala B, Lacey MJ. Utilization of Health Care Resources by the Amish of a Rural County in Nebraska. J Community Health. 2019;44(6):1090–7. https://doi.org/10.1007/s10900-019-00696-9 .

Johnston K, Harvey C, Matich P, et al. Increasing access to sexual health care for rural and regional young people: Similarities and differences in the views of young people and service providers. Aust J Rural Health. 2015;23(5):257–64. https://doi.org/10.1111/ajr.12186 .

Legido-Quigley H, Naheed A, de Silva HA, et al. Patients’ experiences on accessing health care services for management of hypertension in rural Bangladesh, Pakistan and Sri Lanka: A qualitative study. PLoS ONE. 2019;14(1): e0211100. https://doi.org/10.1371/journal.pone.0211100 .

Shaw B, Amouzou A, Miller NP, Bryce J, Surkan PJ. A qualitative exploration of care-seeking pathways for sick children in the rural Oromia region of Ethiopia. BMC Health Serv Res. 2017;17(1):184. https://doi.org/10.1186/s12913-017-2123-5 .

Larson E, Vail D, Mbaruku GM, Kimweri A, Freedman LP, Kruk ME. Moving Toward Patient-Centered Care in Africa: A Discrete Choice Experiment of Preferences for Delivery Care among 3,003 Tanzanian Women. PLoS ONE. 2015;10(8): e0135621. https://doi.org/10.1371/journal.pone.0135621 .

Spleen AM, Lengerich EJ, Camacho FT, Vanderpool RC. Health care avoidance among rural populations: results from a nationally representative survey. J Rural Health. 2014;30(1):79–88. https://doi.org/10.1111/jrh.12032 .

Weisgrau S. Issues in rural health: access, hospitals, and reform. Health Care Financ Rev. 1995;17(1):1–14.

CAS   PubMed   PubMed Central   Google Scholar  

National Center for Health Statistics. 2018. https://www.cdc.gov/nchs/surveys.htm . Accessed 29 Aug 2021.

Sekhon M, Cartwright M, Francis JJ. Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC Health Serv Res. 2017;17(1):88. https://doi.org/10.1186/s12913-017-2031-8 .

Dyer TA, Owens J, Robinson PG. The acceptability of healthcare: from satisfaction to trust. Community Dent Health. 2016;33(4):242–51. https://doi.org/10.1922/CDH_3902Dyer10 .

Padgett DK. Qualitative and Mixed Methods in Public Health. Thousand Oaks, California: SAGE Publications Inc.; 2012.

Tolley EE, Ulin PR, Mack N, Robinson ET, Succop SM. Qualitative Methods in Public Health: A Field Guide for Applied Research. 2nd ed. San Francisco, California: John Wiley & Sons, Inc.; 2016.

Ingram DD, Franco SJ. 2013 NCHS urban-rural classification scheme for counties. In: National Center for Health Statistics: Vital Health Statistics. 2014. https://www.cdc.gov/nchs/data/series/sr_02/sr02_166.pdf . Accessed 31 Oct 2021.

Health Resources & Services Administration. Health Professional Shortage Area Find. 2021. https://data.hrsa.gov/tools/shortage-area/hpsa-find . Accessed 31 Oct 2021.

Shaghaghi A, Bhopal RS, Sheikh A. Approaches to Recruiting “Hard-To-Reach” Populations into Re-search: A Review of the Literature. Health Promot Perspect. 2011;1(2):86–94. https://doi.org/10.5681/hpp.2011.009 .

Sadler GR, Lee HC, Lim RS, Fullerton J. Recruitment of hard-to-reach population subgroups via adaptations of the snowball sampling strategy. Nurs Health Sci. 2010;12(3):369–74. https://doi.org/10.1111/j.1442-2018.2010.00541.x .

Deterding NM, Waters MC. Flexible Coding of In-depth Interviews: A Twenty-first-century Approach. Sociological Methods & Research. 2021;50(2):708–39. https://doi.org/10.1177/0049124118799377 .

Schreier M, Stamann C, Janssen M, Dahl T, Whittal A. Qualitative Content Analysis: Conceptualizations and Challenges in Research Practice-Introduction to the FQS Special Issue" Qualitative Content Analysis I". InForum Qualitative Sozialforschung/Forum: Qualitative Social Research. 2019;20:26 DEU.

Guest G, Namey E, Chen M. A simple method to assess and report thematic saturation in qualitative research. PLoS ONE. 2020;15(5): e0232076. https://doi.org/10.1371/journal.pone.0232076 .

Mayring P. Qualitative Content Analysis. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research. 2000;1(2). https://doi.org/10.17169/fqs-1.2.1089 .

O'Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Academic Medicine, Vol. 89, No. 9 / Sept 2014. https://doi.org/10.1097/ACM.0000000000000388 .

Carter N, Bryant-Lukosius D, DiCenso A, Blythe J, Neville AJ. The use of triangulation in qualitative research. Oncol Nurs Forum. 2014;41(5):545–7. https://doi.org/10.1188/14.ONF.545-547 .

Miller S. Cultural humility is the first step to becoming global care providers. J Obstet Gynecol Neonatal Nurs. 2009;38(1):92–3. https://doi.org/10.1111/j.1552-6909.2008.00311.x .

Prasad SJ, Nair P, Gadhvi K, Barai I, Danish HS, Philip AB. Cultural humility: treating the patient, not the illness. Med Educ Online. 2016;21:30908. https://doi.org/10.3402/meo.v21.30908 .

George MS, Davey R, Mohanty I, Upton P. “Everything is provided free, but they are still hesitant to access healthcare services”: why does the indigenous community in Attapadi, Kerala continue to experience poor access to healthcare? Int J Equity Health. 2020;19(1):105. https://doi.org/10.1186/s12939-020-01216-1 .

Romanelli M, Hudson KD. Individual and systemic barriers to health care: Perspectives of lesbian, gay, bisexual, and transgender adults. Am J Orthopsychiatry. 2017;87(6):714–28. https://doi.org/10.1037/ort0000306 .

Bailie J, Schierhout G, Laycock A, et al. Determinants of access to chronic illness care: a mixed-methods evaluation of a national multifaceted chronic disease package for Indigenous Australians. BMJ Open. 2015;5(11): e008103. https://doi.org/10.1136/bmjopen-2015-008103 .

Reeve C, Humphreys J, Wakerman J, Carter M, Carroll V, Reeve D. Strengthening primary health care: achieving health gains in a remote region of Australia. Med J Aust. 2015;202(9):483–7. https://doi.org/10.5694/mja14.00894 .

Swan LET, Auerbach SL, Ely GE, Agbemenu K, Mencia J, Araf NR. Family Planning Practices in Appalachia: Focus Group Perspectives on Service Needs in the Context of Regional Substance Abuse. Int J Environ Res Public Health. 2020;17(4):1198. https://doi.org/10.3390/ijerph17041198 .

Kabia E, Mbau R, Oyando R, et al. “We are called the et cetera”: experiences of the poor with health financing reforms that target them in Kenya. Int J Equity Health. 2019;18(1):98. https://doi.org/10.1186/s12939-019-1006-2 .

Ho JW, Kuluski K, Im J. “It’s a fight to get anything you need” - Accessing care in the community from the perspectives of people with multimorbidity. Health Expect. 2017;20(6):1311–9. https://doi.org/10.1111/hex.12571 .

Latif A, Mandane B, Ali A, Ghumra S, Gulzar N. A Qualitative Exploration to Understand Access to Pharmacy Medication Reviews: Views from Marginalized Patient Groups. Pharmacy (Basel). 2020;8(2):73. https://doi.org/10.3390/pharmacy8020073 .

Tschirhart N, Diaz E, Ottersen T. Accessing public healthcare in Oslo, Norway: the experiences of Thai immigrant masseuses. BMC Health Serv Res. 2019;19(1):722. https://doi.org/10.1186/s12913-019-4560-9 .

Ireland S, Belton S, Doran F. “I didn’t feel judged”: exploring women’s access to telemedicine abortion in rural Australia. J Prim Health Care. 2020;12(1):49–56. https://doi.org/10.1071/HC19050 .

Stokes T, Tumilty E, Latu ATF, et al. Improving access to health care for people with severe chronic obstructive pulmonary disease (COPD) in Southern New Zealand: qualitative study of the views of health professional stakeholders and patients. BMJ Open. 2019;9(11): e033524. https://doi.org/10.1136/bmjopen-2019-033524 .

Jafar TH, Ramakrishnan C, John O, et al. Access to CKD Care in Rural Communities of India: a qualitative study exploring the barriers and potential facilitators. BMC Nephrol. 2020;21(1):26. https://doi.org/10.1186/s12882-020-1702-6 .

Doetsch J, Pilot E, Santana P, Krafft T. Potential barriers in healthcare access of the elderly population influenced by the economic crisis and the troika agreement: a qualitative case study in Lisbon, Portugal. Int J Equity Health. 2017;16(1):184. https://doi.org/10.1186/s12939-017-0679-7 .

Engel GL. The need for a new medical model: a challenge for biomedicine. Science. 1977;196(4286):129–36. https://doi.org/10.1126/science.847460 .

Risberg G, Hamberg K, Johansson E.E. Gender perspective in medicine: a vital part of medical scientific rationality. A useful model for comprehending structures and hierarchies within medical science. BMC Med. 2006;4(1):1.

Montana Census & Economic Information Center. 2021. https://ceic.mt.gov/ . Accessed 18 Sept 2021.

Smith M, Saunders R, Stuckhardt L, McGinnis JM, Committee on the Learning Health Care System in America, Institute of Medicine. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington: National Academies Press; 2013.

O’Daniel M, Rosenstein AH, Professional Communication and Team Collaboration. In: Hughes RG, editor. Chapter 33: Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Agency for Healthcare Research and Quality: Rockville; 2008.

Evans RS. Electronic Health Records: Then, Now, and in the Future. Yearb Med Inform. 2016;Suppl 1(Suppl 1):S48-S61. https://doi.org/10.15265/IYS-2016-s006 .

Tikkanen R, Osborn R, Mossialos E, Djordjevic A, Wharton GA. International Health Care System Profiles: United States. The Commonwealth Fund. 2020. https://www.commonwealthfund.org/international-health-policy-center/countries/united-states . Accessed 19 Sept 2021.

Berchick ER, Barnett JC, Upton RD. Health Insurance Coverage in the United States. Washington: United States Census Bureau; 2018.

Brock DW, Buchanan A. Ethical Issues in For-Profit Health Care. In: Gray BH, editor. For-Profit Enterprise in Health Care. Washington: National Academics Press; 1986.

Lekas HM, Pahl K, Fuller Lewis C. Rethinking Cultural Competence: Shifting to Cultural Humility. Health Serv Insights. 2020;13:1178632920970580. https://doi.org/10.1177/1178632920970580 .

Beaulieu MD. Primary and secondary care: Breaking down barriers for our patients with chronic diseases. Can Fam Physician. 2013;59(2):221.

Crafford L, Jenkins LS. Why seek a second consultation at an emergency centre? A qualitative study. Afr J Prim Health Care Fam Med. 2017;9(1):e1–8. https://doi.org/10.4102/phcfm.v9i1.1397 .

Sorensen MJ, von Recklinghausen FM, Fulton G, et al. Secondary overtriage: The burden of unnecessary interfacility transfers in a rural trauma system. JAMA Surg. 2013;148:763–8.

Sutcliffe CG, Thuma PE, van Dijk JH, et al. Use of mobile phones and text messaging to decrease the turnaround time for early infant HIV diagnosis and notification in rural Zambia: an observational study. BMC Pediatr. 2017;17(1):66. https://doi.org/10.1186/s12887-017-0822-z .

Baldwin LM, Morrison C, Griffin J, et al. Bidirectional Text Messaging to Improve Adherence to Recommended Lipid Testing. J Am Board Fam Med. 2017;30(5):608–14. https://doi.org/10.3122/jabfm.2017.05.170088 .

Albright K, Krantz MJ, Backlund Jarquín P, DeAlleaume L, Coronel-Mockler S, Estacio RO. Health Promotion Text Messaging Preferences and Acceptability Among the Medically Underserved. Health Promot Pract. 2015;16(4):523–32. https://doi.org/10.1177/1524839914566850 .

Cramer ME, Mollard EK, Ford AL, Kupzyk KA, Wilson FA. The feasibility and promise of mobile technology with community health worker reinforcement to reduce rural preterm birth. Public Health Nurs. 2018;35(6):508–16. https://doi.org/10.1111/phn.12543 .

Gbadamosi SO, Eze C, Olawepo JO, et al. A Patient-Held Smartcard With a Unique Identifier and an mHealth Platform to Improve the Availability of Prenatal Test Results in Rural Nigeria: Demonstration Study. J Med Internet Res. 2018;20(1): e18. https://doi.org/10.2196/jmir.8716 .

Trondsen MV, Tjora A, Broom A, Scambler G. The symbolic affordances of a video-mediated gaze in emergency psychiatry. Soc Sci Med. 2018;197:87–94. https://doi.org/10.1016/j.socscimed.2017.11.056 .

Schwitters A, Lederer P, Zilversmit L, et al. Barriers to health care in rural Mozambique: a rapid ethnographic assessment of planned mobile health clinics for ART. Glob Health Sci Pract. 2015;3(1):109–16. https://doi.org/10.9745/GHSP-D-14-00145 .

Kojima N, Krupp K, Ravi K, et al. Implementing and sustaining a mobile medical clinic for prenatal care and sexually transmitted infection prevention in rural Mysore, India. BMC Infect Dis. 2017;17(1):189. https://doi.org/10.1186/s12879-017-2282-3 .

Gorman SE, Martinez JM, Olson J. An assessment of HIV treatment outcomes among utilizers of semi-mobile clinics in rural Kenya. AIDS Care. 2015;27(5):665–8. https://doi.org/10.1080/09540121.2014.986053 .

Lee S, Black D, Held ML. Factors Associated with Telehealth Service Utilization among Rural Populations. J Health Care Poor Underserved. 2019;30(4):1259–72. https://doi.org/10.1353/hpu.2019.0104 .

Agha Z, Schapira RM, Maker AH. Cost effectiveness of telemedicine for the delivery of outpatient pulmonary care to a rural population. Telemed J E Health. 2002;8(3):281–91. https://doi.org/10.1089/15305620260353171 .

Doraiswamy S, Abraham A, Mamtani R, Cheema S. Use of Telehealth During the COVID-19 Pandemic: Scoping Review. J Med Internet Res. 2020;22(12): e24087. https://doi.org/10.2196/24087 .

Download references

Acknowledgements

This research was supported by the Center for Biomedical Research Excellence award (P20GM130418) from the National Institute of General Medical Sciences of the National Institute of Health. The first author was also supported by the University of Montana Burnham Population Health Fellowship. We would like to thank Dr. Christopher Dietrich, Dr. Jennifer Robohm and Dr. Eric Arzubi for their contributions on determining inclusion criteria for the healthcare provider population used for this study.

 This research did not receive any specific grant from funding agencies in the public, commercial, and not-for-profit sectors. 

Author information

Authors and affiliations.

School of Public & Community Health Sciences, University of Montana, 32 Campus Dr, Missoula, MT, 59812, USA

Nicholas C. Coombs & James Caringi

Department of Psychology, University of Montana, 32 Campus Dr, Missoula, MT, 59812, USA

Duncan G. Campbell

You can also search for this author in PubMed   Google Scholar

Contributions

The authors confirm contribution to the paper as follows: study conception and design: NC and JC; data collection: NC; analysis and interpretation of results: NC and JC; draft manuscript preparation: NC, DC and JC; and manuscript editing: NC, DC and JC. All authors reviewed the results and approved the final version of the manuscript.

Corresponding author

Correspondence to Nicholas C. Coombs .

Ethics declarations

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Coombs, N.C., Campbell, D.G. & Caringi, J. A qualitative study of rural healthcare providers’ views of social, cultural, and programmatic barriers to healthcare access. BMC Health Serv Res 22 , 438 (2022). https://doi.org/10.1186/s12913-022-07829-2

Download citation

Received : 11 January 2022

Accepted : 21 March 2022

Published : 02 April 2022

DOI : https://doi.org/10.1186/s12913-022-07829-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Access to healthcare
• Rural health
• Qualitative methods

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 26 March 2024

Predicting and improving complex beer flavor through machine learning

• Michiel Schreurs   ORCID: orcid.org/0000-0002-9449-5619 1 , 2 , 3   na1 ,
• Supinya Piampongsant 1 , 2 , 3   na1 ,
• Miguel Roncoroni   ORCID: orcid.org/0000-0001-7461-1427 1 , 2 , 3   na1 ,
• Lloyd Cool   ORCID: orcid.org/0000-0001-9936-3124 1 , 2 , 3 , 4 ,
• Beatriz Herrera-Malaver   ORCID: orcid.org/0000-0002-5096-9974 1 , 2 , 3 ,
• Christophe Vanderaa   ORCID: orcid.org/0000-0001-7443-5427 4 ,
• Florian A. Theßeling 1 , 2 , 3 ,
• Łukasz Kreft   ORCID: orcid.org/0000-0001-7620-4657 5 ,
• Alexander Botzki   ORCID: orcid.org/0000-0001-6691-4233 5 ,
• Philippe Malcorps 6 ,
• Luk Daenen 6 ,
• Tom Wenseleers   ORCID: orcid.org/0000-0002-1434-861X 4 &
• Kevin J. Verstrepen   ORCID: orcid.org/0000-0002-3077-6219 1 , 2 , 3  

Nature Communications volume  15 , Article number:  2368 ( 2024 ) Cite this article

49k Accesses

847 Altmetric

Metrics details

• Chemical engineering
• Gas chromatography
• Machine learning
• Metabolomics
• Taste receptors

The perception and appreciation of food flavor depends on many interacting chemical compounds and external factors, and therefore proves challenging to understand and predict. Here, we combine extensive chemical and sensory analyses of 250 different beers to train machine learning models that allow predicting flavor and consumer appreciation. For each beer, we measure over 200 chemical properties, perform quantitative descriptive sensory analysis with a trained tasting panel and map data from over 180,000 consumer reviews to train 10 different machine learning models. The best-performing algorithm, Gradient Boosting, yields models that significantly outperform predictions based on conventional statistics and accurately predict complex food features and consumer appreciation from chemical profiles. Model dissection allows identifying specific and unexpected compounds as drivers of beer flavor and appreciation. Adding these compounds results in variants of commercial alcoholic and non-alcoholic beers with improved consumer appreciation. Together, our study reveals how big data and machine learning uncover complex links between food chemistry, flavor and consumer perception, and lays the foundation to develop novel, tailored foods with superior flavors.

Similar content being viewed by others

BitterSweet: Building machine learning models for predicting the bitter and sweet taste of small molecules

Rudraksh Tuwani, Somin Wadhwa & Ganesh Bagler

Sensory lexicon and aroma volatiles analysis of brewing malt

Xiaoxia Su, Miao Yu, … Tianyi Du

Predicting odor from molecular structure: a multi-label classification approach

Kushagra Saini & Venkatnarayan Ramanathan

Introduction

Predicting and understanding food perception and appreciation is one of the major challenges in food science. Accurate modeling of food flavor and appreciation could yield important opportunities for both producers and consumers, including quality control, product fingerprinting, counterfeit detection, spoilage detection, and the development of new products and product combinations (food pairing) 1 , 2 , 3 , 4 , 5 , 6 . Accurate models for flavor and consumer appreciation would contribute greatly to our scientific understanding of how humans perceive and appreciate flavor. Moreover, accurate predictive models would also facilitate and standardize existing food assessment methods and could supplement or replace assessments by trained and consumer tasting panels, which are variable, expensive and time-consuming 7 , 8 , 9 . Lastly, apart from providing objective, quantitative, accurate and contextual information that can help producers, models can also guide consumers in understanding their personal preferences 10 .

Despite the myriad of applications, predicting food flavor and appreciation from its chemical properties remains a largely elusive goal in sensory science, especially for complex food and beverages 11 , 12 . A key obstacle is the immense number of flavor-active chemicals underlying food flavor. Flavor compounds can vary widely in chemical structure and concentration, making them technically challenging and labor-intensive to quantify, even in the face of innovations in metabolomics, such as non-targeted metabolic fingerprinting 13 , 14 . Moreover, sensory analysis is perhaps even more complicated. Flavor perception is highly complex, resulting from hundreds of different molecules interacting at the physiochemical and sensorial level. Sensory perception is often non-linear, characterized by complex and concentration-dependent synergistic and antagonistic effects 15 , 16 , 17 , 18 , 19 , 20 , 21 that are further convoluted by the genetics, environment, culture and psychology of consumers 22 , 23 , 24 . Perceived flavor is therefore difficult to measure, with problems of sensitivity, accuracy, and reproducibility that can only be resolved by gathering sufficiently large datasets 25 . Trained tasting panels are considered the prime source of quality sensory data, but require meticulous training, are low throughput and high cost. Public databases containing consumer reviews of food products could provide a valuable alternative, especially for studying appreciation scores, which do not require formal training 25 . Public databases offer the advantage of amassing large amounts of data, increasing the statistical power to identify potential drivers of appreciation. However, public datasets suffer from biases, including a bias in the volunteers that contribute to the database, as well as confounding factors such as price, cult status and psychological conformity towards previous ratings of the product.

Classical multivariate statistics and machine learning methods have been used to predict flavor of specific compounds by, for example, linking structural properties of a compound to its potential biological activities or linking concentrations of specific compounds to sensory profiles 1 , 26 . Importantly, most previous studies focused on predicting organoleptic properties of single compounds (often based on their chemical structure) 27 , 28 , 29 , 30 , 31 , 32 , 33 , thus ignoring the fact that these compounds are present in a complex matrix in food or beverages and excluding complex interactions between compounds. Moreover, the classical statistics commonly used in sensory science 34 , 35 , 36 , 37 , 38 , 39 require a large sample size and sufficient variance amongst predictors to create accurate models. They are not fit for studying an extensive set of hundreds of interacting flavor compounds, since they are sensitive to outliers, have a high tendency to overfit and are less suited for non-linear and discontinuous relationships 40 .

In this study, we combine extensive chemical analyses and sensory data of a set of different commercial beers with machine learning approaches to develop models that predict taste, smell, mouthfeel and appreciation from compound concentrations. Beer is particularly suited to model the relationship between chemistry, flavor and appreciation. First, beer is a complex product, consisting of thousands of flavor compounds that partake in complex sensory interactions 41 , 42 , 43 . This chemical diversity arises from the raw materials (malt, yeast, hops, water and spices) and biochemical conversions during the brewing process (kilning, mashing, boiling, fermentation, maturation and aging) 44 , 45 . Second, the advent of the internet saw beer consumers embrace online review platforms, such as RateBeer (ZX Ventures, Anheuser-Busch InBev SA/NV) and BeerAdvocate (Next Glass, inc.). In this way, the beer community provides massive data sets of beer flavor and appreciation scores, creating extraordinarily large sensory databases to complement the analyses of our professional sensory panel. Specifically, we characterize over 200 chemical properties of 250 commercial beers, spread across 22 beer styles, and link these to the descriptive sensory profiling data of a 16-person in-house trained tasting panel and data acquired from over 180,000 public consumer reviews. These unique and extensive datasets enable us to train a suite of machine learning models to predict flavor and appreciation from a beer’s chemical profile. Dissection of the best-performing models allows us to pinpoint specific compounds as potential drivers of beer flavor and appreciation. Follow-up experiments confirm the importance of these compounds and ultimately allow us to significantly improve the flavor and appreciation of selected commercial beers. Together, our study represents a significant step towards understanding complex flavors and reinforces the value of machine learning to develop and refine complex foods. In this way, it represents a stepping stone for further computer-aided food engineering applications 46 .

To generate a comprehensive dataset on beer flavor, we selected 250 commercial Belgian beers across 22 different beer styles (Supplementary Fig.  S1 ). Beers with ≤ 4.2% alcohol by volume (ABV) were classified as non-alcoholic and low-alcoholic. Blonds and Tripels constitute a significant portion of the dataset (12.4% and 11.2%, respectively) reflecting their presence on the Belgian beer market and the heterogeneity of beers within these styles. By contrast, lager beers are less diverse and dominated by a handful of brands. Rare styles such as Brut or Faro make up only a small fraction of the dataset (2% and 1%, respectively) because fewer of these beers are produced and because they are dominated by distinct characteristics in terms of flavor and chemical composition.

Extensive analysis identifies relationships between chemical compounds in beer

For each beer, we measured 226 different chemical properties, including common brewing parameters such as alcohol content, iso-alpha acids, pH, sugar concentration 47 , and over 200 flavor compounds (Methods, Supplementary Table  S1 ). A large portion (37.2%) are terpenoids arising from hopping, responsible for herbal and fruity flavors 16 , 48 . A second major category are yeast metabolites, such as esters and alcohols, that result in fruity and solvent notes 48 , 49 , 50 . Other measured compounds are primarily derived from malt, or other microbes such as non- Saccharomyces yeasts and bacteria (‘wild flora’). Compounds that arise from spices or staling are labeled under ‘Others’. Five attributes (caloric value, total acids and total ester, hop aroma and sulfur compounds) are calculated from multiple individually measured compounds.

As a first step in identifying relationships between chemical properties, we determined correlations between the concentrations of the compounds (Fig.  1 , upper panel, Supplementary Data  1 and 2 , and Supplementary Fig.  S2 . For the sake of clarity, only a subset of the measured compounds is shown in Fig.  1 ). Compounds of the same origin typically show a positive correlation, while absence of correlation hints at parameters varying independently. For example, the hop aroma compounds citronellol, and alpha-terpineol show moderate correlations with each other (Spearman’s rho=0.39 and 0.57), but not with the bittering hop component iso-alpha acids (Spearman’s rho=0.16 and −0.07). This illustrates how brewers can independently modify hop aroma and bitterness by selecting hop varieties and dosage time. If hops are added early in the boiling phase, chemical conversions increase bitterness while aromas evaporate, conversely, late addition of hops preserves aroma but limits bitterness 51 . Similarly, hop-derived iso-alpha acids show a strong anti-correlation with lactic acid and acetic acid, likely reflecting growth inhibition of lactic acid and acetic acid bacteria, or the consequent use of fewer hops in sour beer styles, such as West Flanders ales and Fruit beers, that rely on these bacteria for their distinct flavors 52 . Finally, yeast-derived esters (ethyl acetate, ethyl decanoate, ethyl hexanoate, ethyl octanoate) and alcohols (ethanol, isoamyl alcohol, isobutanol, and glycerol), correlate with Spearman coefficients above 0.5, suggesting that these secondary metabolites are correlated with the yeast genetic background and/or fermentation parameters and may be difficult to influence individually, although the choice of yeast strain may offer some control 53 .

Spearman rank correlations are shown. Descriptors are grouped according to their origin (malt (blue), hops (green), yeast (red), wild flora (yellow), Others (black)), and sensory aspect (aroma, taste, palate, and overall appreciation). Please note that for the chemical compounds, for the sake of clarity, only a subset of the total number of measured compounds is shown, with an emphasis on the key compounds for each source. For more details, see the main text and Methods section. Chemical data can be found in Supplementary Data  1 , correlations between all chemical compounds are depicted in Supplementary Fig.  S2 and correlation values can be found in Supplementary Data  2 . See Supplementary Data  4 for sensory panel assessments and Supplementary Data  5 for correlation values between all sensory descriptors.

Interestingly, different beer styles show distinct patterns for some flavor compounds (Supplementary Fig.  S3 ). These observations agree with expectations for key beer styles, and serve as a control for our measurements. For instance, Stouts generally show high values for color (darker), while hoppy beers contain elevated levels of iso-alpha acids, compounds associated with bitter hop taste. Acetic and lactic acid are not prevalent in most beers, with notable exceptions such as Kriek, Lambic, Faro, West Flanders ales and Flanders Old Brown, which use acid-producing bacteria ( Lactobacillus and Pediococcus ) or unconventional yeast ( Brettanomyces ) 54 , 55 . Glycerol, ethanol and esters show similar distributions across all beer styles, reflecting their common origin as products of yeast metabolism during fermentation 45 , 53 . Finally, low/no-alcohol beers contain low concentrations of glycerol and esters. This is in line with the production process for most of the low/no-alcohol beers in our dataset, which are produced through limiting fermentation or by stripping away alcohol via evaporation or dialysis, with both methods having the unintended side-effect of reducing the amount of flavor compounds in the final beer 56 , 57 .

Besides expected associations, our data also reveals less trivial associations between beer styles and specific parameters. For example, geraniol and citronellol, two monoterpenoids responsible for citrus, floral and rose flavors and characteristic of Citra hops, are found in relatively high amounts in Christmas, Saison, and Brett/co-fermented beers, where they may originate from terpenoid-rich spices such as coriander seeds instead of hops 58 .

Tasting panel assessments reveal sensorial relationships in beer

To assess the sensory profile of each beer, a trained tasting panel evaluated each of the 250 beers for 50 sensory attributes, including different hop, malt and yeast flavors, off-flavors and spices. Panelists used a tasting sheet (Supplementary Data  3 ) to score the different attributes. Panel consistency was evaluated by repeating 12 samples across different sessions and performing ANOVA. In 95% of cases no significant difference was found across sessions ( p  > 0.05), indicating good panel consistency (Supplementary Table  S2 ).

Aroma and taste perception reported by the trained panel are often linked (Fig.  1 , bottom left panel and Supplementary Data  4 and 5 ), with high correlations between hops aroma and taste (Spearman’s rho=0.83). Bitter taste was found to correlate with hop aroma and taste in general (Spearman’s rho=0.80 and 0.69), and particularly with “grassy” noble hops (Spearman’s rho=0.75). Barnyard flavor, most often associated with sour beers, is identified together with stale hops (Spearman’s rho=0.97) that are used in these beers. Lactic and acetic acid, which often co-occur, are correlated (Spearman’s rho=0.66). Interestingly, sweetness and bitterness are anti-correlated (Spearman’s rho = −0.48), confirming the hypothesis that they mask each other 59 , 60 . Beer body is highly correlated with alcohol (Spearman’s rho = 0.79), and overall appreciation is found to correlate with multiple aspects that describe beer mouthfeel (alcohol, carbonation; Spearman’s rho= 0.32, 0.39), as well as with hop and ester aroma intensity (Spearman’s rho=0.39 and 0.35).

Similar to the chemical analyses, sensorial analyses confirmed typical features of specific beer styles (Supplementary Fig.  S4 ). For example, sour beers (Faro, Flanders Old Brown, Fruit beer, Kriek, Lambic, West Flanders ale) were rated acidic, with flavors of both acetic and lactic acid. Hoppy beers were found to be bitter and showed hop-associated aromas like citrus and tropical fruit. Malt taste is most detected among scotch, stout/porters, and strong ales, while low/no-alcohol beers, which often have a reputation for being ‘worty’ (reminiscent of unfermented, sweet malt extract) appear in the middle. Unsurprisingly, hop aromas are most strongly detected among hoppy beers. Like its chemical counterpart (Supplementary Fig.  S3 ), acidity shows a right-skewed distribution, with the most acidic beers being Krieks, Lambics, and West Flanders ales.

Tasting panel assessments of specific flavors correlate with chemical composition

We find that the concentrations of several chemical compounds strongly correlate with specific aroma or taste, as evaluated by the tasting panel (Fig.  2 , Supplementary Fig.  S5 , Supplementary Data  6 ). In some cases, these correlations confirm expectations and serve as a useful control for data quality. For example, iso-alpha acids, the bittering compounds in hops, strongly correlate with bitterness (Spearman’s rho=0.68), while ethanol and glycerol correlate with tasters’ perceptions of alcohol and body, the mouthfeel sensation of fullness (Spearman’s rho=0.82/0.62 and 0.72/0.57 respectively) and darker color from roasted malts is a good indication of malt perception (Spearman’s rho=0.54).

Heatmap colors indicate Spearman’s Rho. Axes are organized according to sensory categories (aroma, taste, mouthfeel, overall), chemical categories and chemical sources in beer (malt (blue), hops (green), yeast (red), wild flora (yellow), Others (black)). See Supplementary Data  6 for all correlation values.

Interestingly, for some relationships between chemical compounds and perceived flavor, correlations are weaker than expected. For example, the rose-smelling phenethyl acetate only weakly correlates with floral aroma. This hints at more complex relationships and interactions between compounds and suggests a need for a more complex model than simple correlations. Lastly, we uncovered unexpected correlations. For instance, the esters ethyl decanoate and ethyl octanoate appear to correlate slightly with hop perception and bitterness, possibly due to their fruity flavor. Iron is anti-correlated with hop aromas and bitterness, most likely because it is also anti-correlated with iso-alpha acids. This could be a sign of metal chelation of hop acids 61 , given that our analyses measure unbound hop acids and total iron content, or could result from the higher iron content in dark and Fruit beers, which typically have less hoppy and bitter flavors 62 .

Public consumer reviews complement expert panel data

To complement and expand the sensory data of our trained tasting panel, we collected 180,000 reviews of our 250 beers from the online consumer review platform RateBeer. This provided numerical scores for beer appearance, aroma, taste, palate, overall quality as well as the average overall score.

Public datasets are known to suffer from biases, such as price, cult status and psychological conformity towards previous ratings of a product. For example, prices correlate with appreciation scores for these online consumer reviews (rho=0.49, Supplementary Fig.  S6 ), but not for our trained tasting panel (rho=0.19). This suggests that prices affect consumer appreciation, which has been reported in wine 63 , while blind tastings are unaffected. Moreover, we observe that some beer styles, like lagers and non-alcoholic beers, generally receive lower scores, reflecting that online reviewers are mostly beer aficionados with a preference for specialty beers over lager beers. In general, we find a modest correlation between our trained panel’s overall appreciation score and the online consumer appreciation scores (Fig.  3 , rho=0.29). Apart from the aforementioned biases in the online datasets, serving temperature, sample freshness and surroundings, which are all tightly controlled during the tasting panel sessions, can vary tremendously across online consumers and can further contribute to (among others, appreciation) differences between the two categories of tasters. Importantly, in contrast to the overall appreciation scores, for many sensory aspects the results from the professional panel correlated well with results obtained from RateBeer reviews. Correlations were highest for features that are relatively easy to recognize even for untrained tasters, like bitterness, sweetness, alcohol and malt aroma (Fig.  3 and below).

RateBeer text mining results can be found in Supplementary Data  7 . Rho values shown are Spearman correlation values, with asterisks indicating significant correlations ( p  < 0.05, two-sided). All p values were smaller than 0.001, except for Esters aroma (0.0553), Esters taste (0.3275), Esters aroma—banana (0.0019), Coriander (0.0508) and Diacetyl (0.0134).

Besides collecting consumer appreciation from these online reviews, we developed automated text analysis tools to gather additional data from review texts (Supplementary Data  7 ). Processing review texts on the RateBeer database yielded comparable results to the scores given by the trained panel for many common sensory aspects, including acidity, bitterness, sweetness, alcohol, malt, and hop tastes (Fig.  3 ). This is in line with what would be expected, since these attributes require less training for accurate assessment and are less influenced by environmental factors such as temperature, serving glass and odors in the environment. Consumer reviews also correlate well with our trained panel for 4-vinyl guaiacol, a compound associated with a very characteristic aroma. By contrast, correlations for more specific aromas like ester, coriander or diacetyl are underrepresented in the online reviews, underscoring the importance of using a trained tasting panel and standardized tasting sheets with explicit factors to be scored for evaluating specific aspects of a beer. Taken together, our results suggest that public reviews are trustworthy for some, but not all, flavor features and can complement or substitute taste panel data for these sensory aspects.

Models can predict beer sensory profiles from chemical data

The rich datasets of chemical analyses, tasting panel assessments and public reviews gathered in the first part of this study provided us with a unique opportunity to develop predictive models that link chemical data to sensorial features. Given the complexity of beer flavor, basic statistical tools such as correlations or linear regression may not always be the most suitable for making accurate predictions. Instead, we applied different machine learning models that can model both simple linear and complex interactive relationships. Specifically, we constructed a set of regression models to predict (a) trained panel scores for beer flavor and quality and (b) public reviews’ appreciation scores from beer chemical profiles. We trained and tested 10 different models (Methods), 3 linear regression-based models (simple linear regression with first-order interactions (LR), lasso regression with first-order interactions (Lasso), partial least squares regressor (PLSR)), 5 decision tree models (AdaBoost regressor (ABR), extra trees (ET), gradient boosting regressor (GBR), random forest (RF) and XGBoost regressor (XGBR)), 1 support vector regression (SVR), and 1 artificial neural network (ANN) model.

To compare the performance of our machine learning models, the dataset was randomly split into a training and test set, stratified by beer style. After a model was trained on data in the training set, its performance was evaluated on its ability to predict the test dataset obtained from multi-output models (based on the coefficient of determination, see Methods). Additionally, individual-attribute models were ranked per descriptor and the average rank was calculated, as proposed by Korneva et al. 64 . Importantly, both ways of evaluating the models’ performance agreed in general. Performance of the different models varied (Table  1 ). It should be noted that all models perform better at predicting RateBeer results than results from our trained tasting panel. One reason could be that sensory data is inherently variable, and this variability is averaged out with the large number of public reviews from RateBeer. Additionally, all tree-based models perform better at predicting taste than aroma. Linear models (LR) performed particularly poorly, with negative R 2 values, due to severe overfitting (training set R 2  = 1). Overfitting is a common issue in linear models with many parameters and limited samples, especially with interaction terms further amplifying the number of parameters. L1 regularization (Lasso) successfully overcomes this overfitting, out-competing multiple tree-based models on the RateBeer dataset. Similarly, the dimensionality reduction of PLSR avoids overfitting and improves performance, to some extent. Still, tree-based models (ABR, ET, GBR, RF and XGBR) show the best performance, out-competing the linear models (LR, Lasso, PLSR) commonly used in sensory science 65 .

GBR models showed the best overall performance in predicting sensory responses from chemical information, with R 2 values up to 0.75 depending on the predicted sensory feature (Supplementary Table  S4 ). The GBR models predict consumer appreciation (RateBeer) better than our trained panel’s appreciation (R 2 value of 0.67 compared to R 2 value of 0.09) (Supplementary Table  S3 and Supplementary Table  S4 ). ANN models showed intermediate performance, likely because neural networks typically perform best with larger datasets 66 . The SVR shows intermediate performance, mostly due to the weak predictions of specific attributes that lower the overall performance (Supplementary Table  S4 ).

Model dissection identifies specific, unexpected compounds as drivers of consumer appreciation

Next, we leveraged our models to infer important contributors to sensory perception and consumer appreciation. Consumer preference is a crucial sensory aspects, because a product that shows low consumer appreciation scores often does not succeed commercially 25 . Additionally, the requirement for a large number of representative evaluators makes consumer trials one of the more costly and time-consuming aspects of product development. Hence, a model for predicting chemical drivers of overall appreciation would be a welcome addition to the available toolbox for food development and optimization.

Since GBR models on our RateBeer dataset showed the best overall performance, we focused on these models. Specifically, we used two approaches to identify important contributors. First, rankings of the most important predictors for each sensorial trait in the GBR models were obtained based on impurity-based feature importance (mean decrease in impurity). High-ranked parameters were hypothesized to be either the true causal chemical properties underlying the trait, to correlate with the actual causal properties, or to take part in sensory interactions affecting the trait 67 (Fig.  4A ). In a second approach, we used SHAP 68 to determine which parameters contributed most to the model for making predictions of consumer appreciation (Fig.  4B ). SHAP calculates parameter contributions to model predictions on a per-sample basis, which can be aggregated into an importance score.

A The impurity-based feature importance (mean deviance in impurity, MDI) calculated from the Gradient Boosting Regression (GBR) model predicting RateBeer appreciation scores. The top 15 highest ranked chemical properties are shown. B SHAP summary plot for the top 15 parameters contributing to our GBR model. Each point on the graph represents a sample from our dataset. The color represents the concentration of that parameter, with bluer colors representing low values and redder colors representing higher values. Greater absolute values on the horizontal axis indicate a higher impact of the parameter on the prediction of the model. C Spearman correlations between the 15 most important chemical properties and consumer overall appreciation. Numbers indicate the Spearman Rho correlation coefficient, and the rank of this correlation compared to all other correlations. The top 15 important compounds were determined using SHAP (panel B).

Both approaches identified ethyl acetate as the most predictive parameter for beer appreciation (Fig.  4 ). Ethyl acetate is the most abundant ester in beer with a typical ‘fruity’, ‘solvent’ and ‘alcoholic’ flavor, but is often considered less important than other esters like isoamyl acetate. The second most important parameter identified by SHAP is ethanol, the most abundant beer compound after water. Apart from directly contributing to beer flavor and mouthfeel, ethanol drastically influences the physical properties of beer, dictating how easily volatile compounds escape the beer matrix to contribute to beer aroma 69 . Importantly, it should also be noted that the importance of ethanol for appreciation is likely inflated by the very low appreciation scores of non-alcoholic beers (Supplementary Fig.  S4 ). Despite not often being considered a driver of beer appreciation, protein level also ranks highly in both approaches, possibly due to its effect on mouthfeel and body 70 . Lactic acid, which contributes to the tart taste of sour beers, is the fourth most important parameter identified by SHAP, possibly due to the generally high appreciation of sour beers in our dataset.

Interestingly, some of the most important predictive parameters for our model are not well-established as beer flavors or are even commonly regarded as being negative for beer quality. For example, our models identify methanethiol and ethyl phenyl acetate, an ester commonly linked to beer staling 71 , as a key factor contributing to beer appreciation. Although there is no doubt that high concentrations of these compounds are considered unpleasant, the positive effects of modest concentrations are not yet known 72 , 73 .

To compare our approach to conventional statistics, we evaluated how well the 15 most important SHAP-derived parameters correlate with consumer appreciation (Fig.  4C ). Interestingly, only 6 of the properties derived by SHAP rank amongst the top 15 most correlated parameters. For some chemical compounds, the correlations are so low that they would have likely been considered unimportant. For example, lactic acid, the fourth most important parameter, shows a bimodal distribution for appreciation, with sour beers forming a separate cluster, that is missed entirely by the Spearman correlation. Additionally, the correlation plots reveal outliers, emphasizing the need for robust analysis tools. Together, this highlights the need for alternative models, like the Gradient Boosting model, that better grasp the complexity of (beer) flavor.

Finally, to observe the relationships between these chemical properties and their predicted targets, partial dependence plots were constructed for the six most important predictors of consumer appreciation 74 , 75 , 76 (Supplementary Fig.  S7 ). One-way partial dependence plots show how a change in concentration affects the predicted appreciation. These plots reveal an important limitation of our models: appreciation predictions remain constant at ever-increasing concentrations. This implies that once a threshold concentration is reached, further increasing the concentration does not affect appreciation. This is false, as it is well-documented that certain compounds become unpleasant at high concentrations, including ethyl acetate (‘nail polish’) 77 and methanethiol (‘sulfury’ and ‘rotten cabbage’) 78 . The inability of our models to grasp that flavor compounds have optimal levels, above which they become negative, is a consequence of working with commercial beer brands where (off-)flavors are rarely too high to negatively impact the product. The two-way partial dependence plots show how changing the concentration of two compounds influences predicted appreciation, visualizing their interactions (Supplementary Fig.  S7 ). In our case, the top 5 parameters are dominated by additive or synergistic interactions, with high concentrations for both compounds resulting in the highest predicted appreciation.

To assess the robustness of our best-performing models and model predictions, we performed 100 iterations of the GBR, RF and ET models. In general, all iterations of the models yielded similar performance (Supplementary Fig.  S8 ). Moreover, the main predictors (including the top predictors ethanol and ethyl acetate) remained virtually the same, especially for GBR and RF. For the iterations of the ET model, we did observe more variation in the top predictors, which is likely a consequence of the model’s inherent random architecture in combination with co-correlations between certain predictors. However, even in this case, several of the top predictors (ethanol and ethyl acetate) remain unchanged, although their rank in importance changes (Supplementary Fig.  S8 ).

Next, we investigated if a combination of RateBeer and trained panel data into one consolidated dataset would lead to stronger models, under the hypothesis that such a model would suffer less from bias in the datasets. A GBR model was trained to predict appreciation on the combined dataset. This model underperformed compared to the RateBeer model, both in the native case and when including a dataset identifier (R 2  = 0.67, 0.26 and 0.42 respectively). For the latter, the dataset identifier is the most important feature (Supplementary Fig.  S9 ), while most of the feature importance remains unchanged, with ethyl acetate and ethanol ranking highest, like in the original model trained only on RateBeer data. It seems that the large variation in the panel dataset introduces noise, weakening the models’ performances and reliability. In addition, it seems reasonable to assume that both datasets are fundamentally different, with the panel dataset obtained by blind tastings by a trained professional panel.

Lastly, we evaluated whether beer style identifiers would further enhance the model’s performance. A GBR model was trained with parameters that explicitly encoded the styles of the samples. This did not improve model performance (R2 = 0.66 with style information vs R2 = 0.67). The most important chemical features are consistent with the model trained without style information (eg. ethanol and ethyl acetate), and with the exception of the most preferred (strong ale) and least preferred (low/no-alcohol) styles, none of the styles were among the most important features (Supplementary Fig.  S9 , Supplementary Table  S5 and S6 ). This is likely due to a combination of style-specific chemical signatures, such as iso-alpha acids and lactic acid, that implicitly convey style information to the original models, as well as the low number of samples belonging to some styles, making it difficult for the model to learn style-specific patterns. Moreover, beer styles are not rigorously defined, with some styles overlapping in features and some beers being misattributed to a specific style, all of which leads to more noise in models that use style parameters.

Model validation

To test if our predictive models give insight into beer appreciation, we set up experiments aimed at improving existing commercial beers. We specifically selected overall appreciation as the trait to be examined because of its complexity and commercial relevance. Beer flavor comprises a complex bouquet rather than single aromas and tastes 53 . Hence, adding a single compound to the extent that a difference is noticeable may lead to an unbalanced, artificial flavor. Therefore, we evaluated the effect of combinations of compounds. Because Blond beers represent the most extensive style in our dataset, we selected a beer from this style as the starting material for these experiments (Beer 64 in Supplementary Data  1 ).

In the first set of experiments, we adjusted the concentrations of compounds that made up the most important predictors of overall appreciation (ethyl acetate, ethanol, lactic acid, ethyl phenyl acetate) together with correlated compounds (ethyl hexanoate, isoamyl acetate, glycerol), bringing them up to 95 th percentile ethanol-normalized concentrations (Methods) within the Blond group (‘Spiked’ concentration in Fig.  5A ). Compared to controls, the spiked beers were found to have significantly improved overall appreciation among trained panelists, with panelist noting increased intensity of ester flavors, sweetness, alcohol, and body fullness (Fig.  5B ). To disentangle the contribution of ethanol to these results, a second experiment was performed without the addition of ethanol. This resulted in a similar outcome, including increased perception of alcohol and overall appreciation.

Adding the top chemical compounds, identified as best predictors of appreciation by our model, into poorly appreciated beers results in increased appreciation from our trained panel. Results of sensory tests between base beers and those spiked with compounds identified as the best predictors by the model. A Blond and Non/Low-alcohol (0.0% ABV) base beers were brought up to 95th-percentile ethanol-normalized concentrations within each style. B For each sensory attribute, tasters indicated the more intense sample and selected the sample they preferred. The numbers above the bars correspond to the p values that indicate significant changes in perceived flavor (two-sided binomial test: alpha 0.05, n  = 20 or 13).

In a last experiment, we tested whether using the model’s predictions can boost the appreciation of a non-alcoholic beer (beer 223 in Supplementary Data  1 ). Again, the addition of a mixture of predicted compounds (omitting ethanol, in this case) resulted in a significant increase in appreciation, body, ester flavor and sweetness.

Predicting flavor and consumer appreciation from chemical composition is one of the ultimate goals of sensory science. A reliable, systematic and unbiased way to link chemical profiles to flavor and food appreciation would be a significant asset to the food and beverage industry. Such tools would substantially aid in quality control and recipe development, offer an efficient and cost-effective alternative to pilot studies and consumer trials and would ultimately allow food manufacturers to produce superior, tailor-made products that better meet the demands of specific consumer groups more efficiently.

A limited set of studies have previously tried, to varying degrees of success, to predict beer flavor and beer popularity based on (a limited set of) chemical compounds and flavors 79 , 80 . Current sensitive, high-throughput technologies allow measuring an unprecedented number of chemical compounds and properties in a large set of samples, yielding a dataset that can train models that help close the gaps between chemistry and flavor, even for a complex natural product like beer. To our knowledge, no previous research gathered data at this scale (250 samples, 226 chemical parameters, 50 sensory attributes and 5 consumer scores) to disentangle and validate the chemical aspects driving beer preference using various machine-learning techniques. We find that modern machine learning models outperform conventional statistical tools, such as correlations and linear models, and can successfully predict flavor appreciation from chemical composition. This could be attributed to the natural incorporation of interactions and non-linear or discontinuous effects in machine learning models, which are not easily grasped by the linear model architecture. While linear models and partial least squares regression represent the most widespread statistical approaches in sensory science, in part because they allow interpretation 65 , 81 , 82 , modern machine learning methods allow for building better predictive models while preserving the possibility to dissect and exploit the underlying patterns. Of the 10 different models we trained, tree-based models, such as our best performing GBR, showed the best overall performance in predicting sensory responses from chemical information, outcompeting artificial neural networks. This agrees with previous reports for models trained on tabular data 83 . Our results are in line with the findings of Colantonio et al. who also identified the gradient boosting architecture as performing best at predicting appreciation and flavor (of tomatoes and blueberries, in their specific study) 26 . Importantly, besides our larger experimental scale, we were able to directly confirm our models’ predictions in vivo.

Our study confirms that flavor compound concentration does not always correlate with perception, suggesting complex interactions that are often missed by more conventional statistics and simple models. Specifically, we find that tree-based algorithms may perform best in developing models that link complex food chemistry with aroma. Furthermore, we show that massive datasets of untrained consumer reviews provide a valuable source of data, that can complement or even replace trained tasting panels, especially for appreciation and basic flavors, such as sweetness and bitterness. This holds despite biases that are known to occur in such datasets, such as price or conformity bias. Moreover, GBR models predict taste better than aroma. This is likely because taste (e.g. bitterness) often directly relates to the corresponding chemical measurements (e.g., iso-alpha acids), whereas such a link is less clear for aromas, which often result from the interplay between multiple volatile compounds. We also find that our models are best at predicting acidity and alcohol, likely because there is a direct relation between the measured chemical compounds (acids and ethanol) and the corresponding perceived sensorial attribute (acidity and alcohol), and because even untrained consumers are generally able to recognize these flavors and aromas.

The predictions of our final models, trained on review data, hold even for blind tastings with small groups of trained tasters, as demonstrated by our ability to validate specific compounds as drivers of beer flavor and appreciation. Since adding a single compound to the extent of a noticeable difference may result in an unbalanced flavor profile, we specifically tested our identified key drivers as a combination of compounds. While this approach does not allow us to validate if a particular single compound would affect flavor and/or appreciation, our experiments do show that this combination of compounds increases consumer appreciation.

It is important to stress that, while it represents an important step forward, our approach still has several major limitations. A key weakness of the GBR model architecture is that amongst co-correlating variables, the largest main effect is consistently preferred for model building. As a result, co-correlating variables often have artificially low importance scores, both for impurity and SHAP-based methods, like we observed in the comparison to the more randomized Extra Trees models. This implies that chemicals identified as key drivers of a specific sensory feature by GBR might not be the true causative compounds, but rather co-correlate with the actual causative chemical. For example, the high importance of ethyl acetate could be (partially) attributed to the total ester content, ethanol or ethyl hexanoate (rho=0.77, rho=0.72 and rho=0.68), while ethyl phenylacetate could hide the importance of prenyl isobutyrate and ethyl benzoate (rho=0.77 and rho=0.76). Expanding our GBR model to include beer style as a parameter did not yield additional power or insight. This is likely due to style-specific chemical signatures, such as iso-alpha acids and lactic acid, that implicitly convey style information to the original model, as well as the smaller sample size per style, limiting the power to uncover style-specific patterns. This can be partly attributed to the curse of dimensionality, where the high number of parameters results in the models mainly incorporating single parameter effects, rather than complex interactions such as style-dependent effects 67 . A larger number of samples may overcome some of these limitations and offer more insight into style-specific effects. On the other hand, beer style is not a rigid scientific classification, and beers within one style often differ a lot, which further complicates the analysis of style as a model factor.

Our study is limited to beers from Belgian breweries. Although these beers cover a large portion of the beer styles available globally, some beer styles and consumer patterns may be missing, while other features might be overrepresented. For example, many Belgian ales exhibit yeast-driven flavor profiles, which is reflected in the chemical drivers of appreciation discovered by this study. In future work, expanding the scope to include diverse markets and beer styles could lead to the identification of even more drivers of appreciation and better models for special niche products that were not present in our beer set.

In addition to inherent limitations of GBR models, there are also some limitations associated with studying food aroma. Even if our chemical analyses measured most of the known aroma compounds, the total number of flavor compounds in complex foods like beer is still larger than the subset we were able to measure in this study. For example, hop-derived thiols, that influence flavor at very low concentrations, are notoriously difficult to measure in a high-throughput experiment. Moreover, consumer perception remains subjective and prone to biases that are difficult to avoid. It is also important to stress that the models are still immature and that more extensive datasets will be crucial for developing more complete models in the future. Besides more samples and parameters, our dataset does not include any demographic information about the tasters. Including such data could lead to better models that grasp external factors like age and culture. Another limitation is that our set of beers consists of high-quality end-products and lacks beers that are unfit for sale, which limits the current model in accurately predicting products that are appreciated very badly. Finally, while models could be readily applied in quality control, their use in sensory science and product development is restrained by their inability to discern causal relationships. Given that the models cannot distinguish compounds that genuinely drive consumer perception from those that merely correlate, validation experiments are essential to identify true causative compounds.

Despite the inherent limitations, dissection of our models enabled us to pinpoint specific molecules as potential drivers of beer aroma and consumer appreciation, including compounds that were unexpected and would not have been identified using standard approaches. Important drivers of beer appreciation uncovered by our models include protein levels, ethyl acetate, ethyl phenyl acetate and lactic acid. Currently, many brewers already use lactic acid to acidify their brewing water and ensure optimal pH for enzymatic activity during the mashing process. Our results suggest that adding lactic acid can also improve beer appreciation, although its individual effect remains to be tested. Interestingly, ethanol appears to be unnecessary to improve beer appreciation, both for blond beer and alcohol-free beer. Given the growing consumer interest in alcohol-free beer, with a predicted annual market growth of >7% 84 , it is relevant for brewers to know what compounds can further increase consumer appreciation of these beers. Hence, our model may readily provide avenues to further improve the flavor and consumer appreciation of both alcoholic and non-alcoholic beers, which is generally considered one of the key challenges for future beer production.

Whereas we see a direct implementation of our results for the development of superior alcohol-free beverages and other food products, our study can also serve as a stepping stone for the development of novel alcohol-containing beverages. We want to echo the growing body of scientific evidence for the negative effects of alcohol consumption, both on the individual level by the mutagenic, teratogenic and carcinogenic effects of ethanol 85 , 86 , as well as the burden on society caused by alcohol abuse and addiction. We encourage the use of our results for the production of healthier, tastier products, including novel and improved beverages with lower alcohol contents. Furthermore, we strongly discourage the use of these technologies to improve the appreciation or addictive properties of harmful substances.

The present work demonstrates that despite some important remaining hurdles, combining the latest developments in chemical analyses, sensory analysis and modern machine learning methods offers exciting avenues for food chemistry and engineering. Soon, these tools may provide solutions in quality control and recipe development, as well as new approaches to sensory science and flavor research.

Beer selection

250 commercial Belgian beers were selected to cover the broad diversity of beer styles and corresponding diversity in chemical composition and aroma. See Supplementary Fig.  S1 .

Chemical dataset

Sample preparation.

Beers within their expiration date were purchased from commercial retailers. Samples were prepared in biological duplicates at room temperature, unless explicitly stated otherwise. Bottle pressure was measured with a manual pressure device (Steinfurth Mess-Systeme GmbH) and used to calculate CO 2 concentration. The beer was poured through two filter papers (Macherey-Nagel, 500713032 MN 713 ¼) to remove carbon dioxide and prevent spontaneous foaming. Samples were then prepared for measurements by targeted Headspace-Gas Chromatography-Flame Ionization Detector/Flame Photometric Detector (HS-GC-FID/FPD), Headspace-Solid Phase Microextraction-Gas Chromatography-Mass Spectrometry (HS-SPME-GC-MS), colorimetric analysis, enzymatic analysis, Near-Infrared (NIR) analysis, as described in the sections below. The mean values of biological duplicates are reported for each compound.

HS-GC-FID/FPD

HS-GC-FID/FPD (Shimadzu GC 2010 Plus) was used to measure higher alcohols, acetaldehyde, esters, 4-vinyl guaicol, and sulfur compounds. Each measurement comprised 5 ml of sample pipetted into a 20 ml glass vial containing 1.75 g NaCl (VWR, 27810.295). 100 µl of 2-heptanol (Sigma-Aldrich, H3003) (internal standard) solution in ethanol (Fisher Chemical, E/0650DF/C17) was added for a final concentration of 2.44 mg/L. Samples were flushed with nitrogen for 10 s, sealed with a silicone septum, stored at −80 °C and analyzed in batches of 20.

The GC was equipped with a DB-WAXetr column (length, 30 m; internal diameter, 0.32 mm; layer thickness, 0.50 µm; Agilent Technologies, Santa Clara, CA, USA) to the FID and an HP-5 column (length, 30 m; internal diameter, 0.25 mm; layer thickness, 0.25 µm; Agilent Technologies, Santa Clara, CA, USA) to the FPD. N 2 was used as the carrier gas. Samples were incubated for 20 min at 70 °C in the headspace autosampler (Flow rate, 35 cm/s; Injection volume, 1000 µL; Injection mode, split; Combi PAL autosampler, CTC analytics, Switzerland). The injector, FID and FPD temperatures were kept at 250 °C. The GC oven temperature was first held at 50 °C for 5 min and then allowed to rise to 80 °C at a rate of 5 °C/min, followed by a second ramp of 4 °C/min until 200 °C kept for 3 min and a final ramp of (4 °C/min) until 230 °C for 1 min. Results were analyzed with the GCSolution software version 2.4 (Shimadzu, Kyoto, Japan). The GC was calibrated with a 5% EtOH solution (VWR International) containing the volatiles under study (Supplementary Table  S7 ).

HS-SPME-GC-MS

HS-SPME-GC-MS (Shimadzu GCMS-QP-2010 Ultra) was used to measure additional volatile compounds, mainly comprising terpenoids and esters. Samples were analyzed by HS-SPME using a triphase DVB/Carboxen/PDMS 50/30 μm SPME fiber (Supelco Co., Bellefonte, PA, USA) followed by gas chromatography (Thermo Fisher Scientific Trace 1300 series, USA) coupled to a mass spectrometer (Thermo Fisher Scientific ISQ series MS) equipped with a TriPlus RSH autosampler. 5 ml of degassed beer sample was placed in 20 ml vials containing 1.75 g NaCl (VWR, 27810.295). 5 µl internal standard mix was added, containing 2-heptanol (1 g/L) (Sigma-Aldrich, H3003), 4-fluorobenzaldehyde (1 g/L) (Sigma-Aldrich, 128376), 2,3-hexanedione (1 g/L) (Sigma-Aldrich, 144169) and guaiacol (1 g/L) (Sigma-Aldrich, W253200) in ethanol (Fisher Chemical, E/0650DF/C17). Each sample was incubated at 60 °C in the autosampler oven with constant agitation. After 5 min equilibration, the SPME fiber was exposed to the sample headspace for 30 min. The compounds trapped on the fiber were thermally desorbed in the injection port of the chromatograph by heating the fiber for 15 min at 270 °C.

The GC-MS was equipped with a low polarity RXi-5Sil MS column (length, 20 m; internal diameter, 0.18 mm; layer thickness, 0.18 µm; Restek, Bellefonte, PA, USA). Injection was performed in splitless mode at 320 °C, a split flow of 9 ml/min, a purge flow of 5 ml/min and an open valve time of 3 min. To obtain a pulsed injection, a programmed gas flow was used whereby the helium gas flow was set at 2.7 mL/min for 0.1 min, followed by a decrease in flow of 20 ml/min to the normal 0.9 mL/min. The temperature was first held at 30 °C for 3 min and then allowed to rise to 80 °C at a rate of 7 °C/min, followed by a second ramp of 2 °C/min till 125 °C and a final ramp of 8 °C/min with a final temperature of 270 °C.

Mass acquisition range was 33 to 550 amu at a scan rate of 5 scans/s. Electron impact ionization energy was 70 eV. The interface and ion source were kept at 275 °C and 250 °C, respectively. A mix of linear n-alkanes (from C7 to C40, Supelco Co.) was injected into the GC-MS under identical conditions to serve as external retention index markers. Identification and quantification of the compounds were performed using an in-house developed R script as described in Goelen et al. and Reher et al. 87 , 88 (for package information, see Supplementary Table  S8 ). Briefly, chromatograms were analyzed using AMDIS (v2.71) 89 to separate overlapping peaks and obtain pure compound spectra. The NIST MS Search software (v2.0 g) in combination with the NIST2017, FFNSC3 and Adams4 libraries were used to manually identify the empirical spectra, taking into account the expected retention time. After background subtraction and correcting for retention time shifts between samples run on different days based on alkane ladders, compound elution profiles were extracted and integrated using a file with 284 target compounds of interest, which were either recovered in our identified AMDIS list of spectra or were known to occur in beer. Compound elution profiles were estimated for every peak in every chromatogram over a time-restricted window using weighted non-negative least square analysis after which peak areas were integrated 87 , 88 . Batch effect correction was performed by normalizing against the most stable internal standard compound, 4-fluorobenzaldehyde. Out of all 284 target compounds that were analyzed, 167 were visually judged to have reliable elution profiles and were used for final analysis.

Discrete photometric and enzymatic analysis

Discrete photometric and enzymatic analysis (Thermo Scientific TM Gallery TM Plus Beermaster Discrete Analyzer) was used to measure acetic acid, ammonia, beta-glucan, iso-alpha acids, color, sugars, glycerol, iron, pH, protein, and sulfite. 2 ml of sample volume was used for the analyses. Information regarding the reagents and standard solutions used for analyses and calibrations is included in Supplementary Table  S7 and Supplementary Table  S9 .

NIR analyses

NIR analysis (Anton Paar Alcolyzer Beer ME System) was used to measure ethanol. Measurements comprised 50 ml of sample, and a 10% EtOH solution was used for calibration.

Correlation calculations

Pairwise Spearman Rank correlations were calculated between all chemical properties.

Sensory dataset

Trained panel.

Our trained tasting panel consisted of volunteers who gave prior verbal informed consent. All compounds used for the validation experiment were of food-grade quality. The tasting sessions were approved by the Social and Societal Ethics Committee of the KU Leuven (G-2022-5677-R2(MAR)). All online reviewers agreed to the Terms and Conditions of the RateBeer website.

Sensory analysis was performed according to the American Society of Brewing Chemists (ASBC) Sensory Analysis Methods 90 . 30 volunteers were screened through a series of triangle tests. The sixteen most sensitive and consistent tasters were retained as taste panel members. The resulting panel was diverse in age [22–42, mean: 29], sex [56% male] and nationality [7 different countries]. The panel developed a consensus vocabulary to describe beer aroma, taste and mouthfeel. Panelists were trained to identify and score 50 different attributes, using a 7-point scale to rate attributes’ intensity. The scoring sheet is included as Supplementary Data  3 . Sensory assessments took place between 10–12 a.m. The beers were served in black-colored glasses. Per session, between 5 and 12 beers of the same style were tasted at 12 °C to 16 °C. Two reference beers were added to each set and indicated as ‘Reference 1 & 2’, allowing panel members to calibrate their ratings. Not all panelists were present at every tasting. Scores were scaled by standard deviation and mean-centered per taster. Values are represented as z-scores and clustered by Euclidean distance. Pairwise Spearman correlations were calculated between taste and aroma sensory attributes. Panel consistency was evaluated by repeating samples on different sessions and performing ANOVA to identify differences, using the ‘stats’ package (v4.2.2) in R (for package information, see Supplementary Table  S8 ).

Online reviews from a public database

The ‘scrapy’ package in Python (v3.6) (for package information, see Supplementary Table  S8 ). was used to collect 232,288 online reviews (mean=922, min=6, max=5343) from RateBeer, an online beer review database. Each review entry comprised 5 numerical scores (appearance, aroma, taste, palate and overall quality) and an optional review text. The total number of reviews per reviewer was collected separately. Numerical scores were scaled and centered per rater, and mean scores were calculated per beer.

For the review texts, the language was estimated using the packages ‘langdetect’ and ‘langid’ in Python. Reviews that were classified as English by both packages were kept. Reviewers with fewer than 100 entries overall were discarded. 181,025 reviews from >6000 reviewers from >40 countries remained. Text processing was done using the ‘nltk’ package in Python. Texts were corrected for slang and misspellings; proper nouns and rare words that are relevant to the beer context were specified and kept as-is (‘Chimay’,’Lambic’, etc.). A dictionary of semantically similar sensorial terms, for example ‘floral’ and ‘flower’, was created and collapsed together into one term. Words were stemmed and lemmatized to avoid identifying words such as ‘acid’ and ‘acidity’ as separate terms. Numbers and punctuation were removed.

Sentences from up to 50 randomly chosen reviews per beer were manually categorized according to the aspect of beer they describe (appearance, aroma, taste, palate, overall quality—not to be confused with the 5 numerical scores described above) or flagged as irrelevant if they contained no useful information. If a beer contained fewer than 50 reviews, all reviews were manually classified. This labeled data set was used to train a model that classified the rest of the sentences for all beers 91 . Sentences describing taste and aroma were extracted, and term frequency–inverse document frequency (TFIDF) was implemented to calculate enrichment scores for sensorial words per beer.

The sex of the tasting subject was not considered when building our sensory database. Instead, results from different panelists were averaged, both for our trained panel (56% male, 44% female) and the RateBeer reviews (70% male, 30% female for RateBeer as a whole).

Beer price collection and processing

Beer prices were collected from the following stores: Colruyt, Delhaize, Total Wine, BeerHawk, The Belgian Beer Shop, The Belgian Shop, and Beer of Belgium. Where applicable, prices were converted to Euros and normalized per liter. Spearman correlations were calculated between these prices and mean overall appreciation scores from RateBeer and the taste panel, respectively.

Pairwise Spearman Rank correlations were calculated between all sensory properties.

Machine learning models

Predictive modeling of sensory profiles from chemical data.

Regression models were constructed to predict (a) trained panel scores for beer flavors and quality from beer chemical profiles and (b) public reviews’ appreciation scores from beer chemical profiles. Z-scores were used to represent sensory attributes in both data sets. Chemical properties with log-normal distributions (Shapiro-Wilk test, p  <  0.05 ) were log-transformed. Missing chemical measurements (0.1% of all data) were replaced with mean values per attribute. Observations from 250 beers were randomly separated into a training set (70%, 175 beers) and a test set (30%, 75 beers), stratified per beer style. Chemical measurements (p = 231) were normalized based on the training set average and standard deviation. In total, three linear regression-based models: linear regression with first-order interaction terms (LR), lasso regression with first-order interaction terms (Lasso) and partial least squares regression (PLSR); five decision tree models, Adaboost regressor (ABR), Extra Trees (ET), Gradient Boosting regressor (GBR), Random Forest (RF) and XGBoost regressor (XGBR); one support vector machine model (SVR) and one artificial neural network model (ANN) were trained. The models were implemented using the ‘scikit-learn’ package (v1.2.2) and ‘xgboost’ package (v1.7.3) in Python (v3.9.16). Models were trained, and hyperparameters optimized, using five-fold cross-validated grid search with the coefficient of determination (R 2 ) as the evaluation metric. The ANN (scikit-learn’s MLPRegressor) was optimized using Bayesian Tree-Structured Parzen Estimator optimization with the ‘Optuna’ Python package (v3.2.0). Individual models were trained per attribute, and a multi-output model was trained on all attributes simultaneously.

Model dissection

GBR was found to outperform other methods, resulting in models with the highest average R 2 values in both trained panel and public review data sets. Impurity-based rankings of the most important predictors for each predicted sensorial trait were obtained using the ‘scikit-learn’ package. To observe the relationships between these chemical properties and their predicted targets, partial dependence plots (PDP) were constructed for the six most important predictors of consumer appreciation 74 , 75 .

The ‘SHAP’ package in Python (v0.41.0) was implemented to provide an alternative ranking of predictor importance and to visualize the predictors’ effects as a function of their concentration 68 .

Validation of causal chemical properties

To validate the effects of the most important model features on predicted sensory attributes, beers were spiked with the chemical compounds identified by the models and descriptive sensory analyses were carried out according to the American Society of Brewing Chemists (ASBC) protocol 90 .

Compound spiking was done 30 min before tasting. Compounds were spiked into fresh beer bottles, that were immediately resealed and inverted three times. Fresh bottles of beer were opened for the same duration, resealed, and inverted thrice, to serve as controls. Pairs of spiked samples and controls were served simultaneously, chilled and in dark glasses as outlined in the Trained panel section above. Tasters were instructed to select the glass with the higher flavor intensity for each attribute (directional difference test 92 ) and to select the glass they prefer.

The final concentration after spiking was equal to the within-style average, after normalizing by ethanol concentration. This was done to ensure balanced flavor profiles in the final spiked beer. The same methods were applied to improve a non-alcoholic beer. Compounds were the following: ethyl acetate (Merck KGaA, W241415), ethyl hexanoate (Merck KGaA, W243906), isoamyl acetate (Merck KGaA, W205508), phenethyl acetate (Merck KGaA, W285706), ethanol (96%, Colruyt), glycerol (Merck KGaA, W252506), lactic acid (Merck KGaA, 261106).

Significant differences in preference or perceived intensity were determined by performing the two-sided binomial test on each attribute.

Reporting summary

Further information on research design is available in the  Nature Portfolio Reporting Summary linked to this article.

Data availability

The data that support the findings of this work are available in the Supplementary Data files and have been deposited to Zenodo under accession code 10653704 93 . The RateBeer scores data are under restricted access, they are not publicly available as they are property of RateBeer (ZX Ventures, USA). Access can be obtained from the authors upon reasonable request and with permission of RateBeer (ZX Ventures, USA).  Source data are provided with this paper.

Code availability

The code for training the machine learning models, analyzing the models, and generating the figures has been deposited to Zenodo under accession code 10653704 93 .

Tieman, D. et al. A chemical genetic roadmap to improved tomato flavor. Science 355 , 391–394 (2017).

Article   ADS   CAS   PubMed   Google Scholar  

Plutowska, B. & Wardencki, W. Application of gas chromatography–olfactometry (GC–O) in analysis and quality assessment of alcoholic beverages – A review. Food Chem. 107 , 449–463 (2008).

Article   CAS   Google Scholar  

Legin, A., Rudnitskaya, A., Seleznev, B. & Vlasov, Y. Electronic tongue for quality assessment of ethanol, vodka and eau-de-vie. Anal. Chim. Acta 534 , 129–135 (2005).

Loutfi, A., Coradeschi, S., Mani, G. K., Shankar, P. & Rayappan, J. B. B. Electronic noses for food quality: A review. J. Food Eng. 144 , 103–111 (2015).

Ahn, Y.-Y., Ahnert, S. E., Bagrow, J. P. & Barabási, A.-L. Flavor network and the principles of food pairing. Sci. Rep. 1 , 196 (2011).

Article   CAS   PubMed   PubMed Central   Google Scholar  

Bartoshuk, L. M. & Klee, H. J. Better fruits and vegetables through sensory analysis. Curr. Biol. 23 , R374–R378 (2013).

Article   CAS   PubMed   Google Scholar  

Piggott, J. R. Design questions in sensory and consumer science. Food Qual. Prefer. 3293 , 217–220 (1995).

Article   Google Scholar  

Kermit, M. & Lengard, V. Assessing the performance of a sensory panel-panellist monitoring and tracking. J. Chemom. 19 , 154–161 (2005).

Cook, D. J., Hollowood, T. A., Linforth, R. S. T. & Taylor, A. J. Correlating instrumental measurements of texture and flavour release with human perception. Int. J. Food Sci. Technol. 40 , 631–641 (2005).

Chinchanachokchai, S., Thontirawong, P. & Chinchanachokchai, P. A tale of two recommender systems: The moderating role of consumer expertise on artificial intelligence based product recommendations. J. Retail. Consum. Serv. 61 , 1–12 (2021).

Ross, C. F. Sensory science at the human-machine interface. Trends Food Sci. Technol. 20 , 63–72 (2009).

Chambers, E. IV & Koppel, K. Associations of volatile compounds with sensory aroma and flavor: The complex nature of flavor. Molecules 18 , 4887–4905 (2013).

Pinu, F. R. Metabolomics—The new frontier in food safety and quality research. Food Res. Int. 72 , 80–81 (2015).

Danezis, G. P., Tsagkaris, A. S., Brusic, V. & Georgiou, C. A. Food authentication: state of the art and prospects. Curr. Opin. Food Sci. 10 , 22–31 (2016).

Shepherd, G. M. Smell images and the flavour system in the human brain. Nature 444 , 316–321 (2006).

Meilgaard, M. C. Prediction of flavor differences between beers from their chemical composition. J. Agric. Food Chem. 30 , 1009–1017 (1982).

Xu, L. et al. Widespread receptor-driven modulation in peripheral olfactory coding. Science 368 , eaaz5390 (2020).

Kupferschmidt, K. Following the flavor. Science 340 , 808–809 (2013).

Billesbølle, C. B. et al. Structural basis of odorant recognition by a human odorant receptor. Nature 615 , 742–749 (2023).

Article   ADS   PubMed   PubMed Central   Google Scholar  

Smith, B. Perspective: Complexities of flavour. Nature 486 , S6–S6 (2012).

Pfister, P. et al. Odorant receptor inhibition is fundamental to odor encoding. Curr. Biol. 30 , 2574–2587 (2020).

Moskowitz, H. W., Kumaraiah, V., Sharma, K. N., Jacobs, H. L. & Sharma, S. D. Cross-cultural differences in simple taste preferences. Science 190 , 1217–1218 (1975).

Eriksson, N. et al. A genetic variant near olfactory receptor genes influences cilantro preference. Flavour 1 , 22 (2012).

Ferdenzi, C. et al. Variability of affective responses to odors: Culture, gender, and olfactory knowledge. Chem. Senses 38 , 175–186 (2013).

Article   PubMed   Google Scholar  

Lawless, H. T. & Heymann, H. Sensory evaluation of food: Principles and practices. (Springer, New York, NY). https://doi.org/10.1007/978-1-4419-6488-5 (2010).

Colantonio, V. et al. Metabolomic selection for enhanced fruit flavor. Proc. Natl. Acad. Sci. 119 , e2115865119 (2022).

Fritz, F., Preissner, R. & Banerjee, P. VirtualTaste: a web server for the prediction of organoleptic properties of chemical compounds. Nucleic Acids Res 49 , W679–W684 (2021).

Tuwani, R., Wadhwa, S. & Bagler, G. BitterSweet: Building machine learning models for predicting the bitter and sweet taste of small molecules. Sci. Rep. 9 , 1–13 (2019).

Dagan-Wiener, A. et al. Bitter or not? BitterPredict, a tool for predicting taste from chemical structure. Sci. Rep. 7 , 1–13 (2017).

Pallante, L. et al. Toward a general and interpretable umami taste predictor using a multi-objective machine learning approach. Sci. Rep. 12 , 1–11 (2022).

Malavolta, M. et al. A survey on computational taste predictors. Eur. Food Res. Technol. 248 , 2215–2235 (2022).

Lee, B. K. et al. A principal odor map unifies diverse tasks in olfactory perception. Science 381 , 999–1006 (2023).

Mayhew, E. J. et al. Transport features predict if a molecule is odorous. Proc. Natl. Acad. Sci. 119 , e2116576119 (2022).

Niu, Y. et al. Sensory evaluation of the synergism among ester odorants in light aroma-type liquor by odor threshold, aroma intensity and flash GC electronic nose. Food Res. Int. 113 , 102–114 (2018).

Yu, P., Low, M. Y. & Zhou, W. Design of experiments and regression modelling in food flavour and sensory analysis: A review. Trends Food Sci. Technol. 71 , 202–215 (2018).

Oladokun, O. et al. The impact of hop bitter acid and polyphenol profiles on the perceived bitterness of beer. Food Chem. 205 , 212–220 (2016).

Linforth, R., Cabannes, M., Hewson, L., Yang, N. & Taylor, A. Effect of fat content on flavor delivery during consumption: An in vivo model. J. Agric. Food Chem. 58 , 6905–6911 (2010).

Guo, S., Na Jom, K. & Ge, Y. Influence of roasting condition on flavor profile of sunflower seeds: A flavoromics approach. Sci. Rep. 9 , 11295 (2019).

Ren, Q. et al. The changes of microbial community and flavor compound in the fermentation process of Chinese rice wine using Fagopyrum tataricum grain as feedstock. Sci. Rep. 9 , 3365 (2019).

Hastie, T., Friedman, J. & Tibshirani, R. The Elements of Statistical Learning. (Springer, New York, NY). https://doi.org/10.1007/978-0-387-21606-5 (2001).

Dietz, C., Cook, D., Huismann, M., Wilson, C. & Ford, R. The multisensory perception of hop essential oil: a review. J. Inst. Brew. 126 , 320–342 (2020).

CAS   Google Scholar  

Roncoroni, Miguel & Verstrepen, Kevin Joan. Belgian Beer: Tested and Tasted. (Lannoo, 2018).

Meilgaard, M. Flavor chemistry of beer: Part II: Flavor and threshold of 239 aroma volatiles. in (1975).

Bokulich, N. A. & Bamforth, C. W. The microbiology of malting and brewing. Microbiol. Mol. Biol. Rev. MMBR 77 , 157–172 (2013).

Dzialo, M. C., Park, R., Steensels, J., Lievens, B. & Verstrepen, K. J. Physiology, ecology and industrial applications of aroma formation in yeast. FEMS Microbiol. Rev. 41 , S95–S128 (2017).

Article   PubMed   PubMed Central   Google Scholar  

Datta, A. et al. Computer-aided food engineering. Nat. Food 3 , 894–904 (2022).

American Society of Brewing Chemists. Beer Methods. (American Society of Brewing Chemists, St. Paul, MN, U.S.A.).

Olaniran, A. O., Hiralal, L., Mokoena, M. P. & Pillay, B. Flavour-active volatile compounds in beer: production, regulation and control. J. Inst. Brew. 123 , 13–23 (2017).

Verstrepen, K. J. et al. Flavor-active esters: Adding fruitiness to beer. J. Biosci. Bioeng. 96 , 110–118 (2003).

Meilgaard, M. C. Flavour chemistry of beer. part I: flavour interaction between principal volatiles. Master Brew. Assoc. Am. Tech. Q 12 , 107–117 (1975).

Briggs, D. E., Boulton, C. A., Brookes, P. A. & Stevens, R. Brewing 227–254. (Woodhead Publishing). https://doi.org/10.1533/9781855739062.227 (2004).

Bossaert, S., Crauwels, S., De Rouck, G. & Lievens, B. The power of sour - A review: Old traditions, new opportunities. BrewingScience 72 , 78–88 (2019).

Google Scholar  

Verstrepen, K. J. et al. Flavor active esters: Adding fruitiness to beer. J. Biosci. Bioeng. 96 , 110–118 (2003).

Snauwaert, I. et al. Microbial diversity and metabolite composition of Belgian red-brown acidic ales. Int. J. Food Microbiol. 221 , 1–11 (2016).

Spitaels, F. et al. The microbial diversity of traditional spontaneously fermented lambic beer. PLoS ONE 9 , e95384 (2014).

Blanco, C. A., Andrés-Iglesias, C. & Montero, O. Low-alcohol Beers: Flavor Compounds, Defects, and Improvement Strategies. Crit. Rev. Food Sci. Nutr. 56 , 1379–1388 (2016).

Jackowski, M. & Trusek, A. Non-Alcohol. beer Prod. – Overv. 20 , 32–38 (2018).

Takoi, K. et al. The contribution of geraniol metabolism to the citrus flavour of beer: Synergy of geraniol and β-citronellol under coexistence with excess linalool. J. Inst. Brew. 116 , 251–260 (2010).

Kroeze, J. H. & Bartoshuk, L. M. Bitterness suppression as revealed by split-tongue taste stimulation in humans. Physiol. Behav. 35 , 779–783 (1985).

Mennella, J. A. et al. A spoonful of sugar helps the medicine go down”: Bitter masking bysucrose among children and adults. Chem. Senses 40 , 17–25 (2015).

Wietstock, P., Kunz, T., Perreira, F. & Methner, F.-J. Metal chelation behavior of hop acids in buffered model systems. BrewingScience 69 , 56–63 (2016).

Sancho, D., Blanco, C. A., Caballero, I. & Pascual, A. Free iron in pale, dark and alcohol-free commercial lager beers. J. Sci. Food Agric. 91 , 1142–1147 (2011).

Rodrigues, H. & Parr, W. V. Contribution of cross-cultural studies to understanding wine appreciation: A review. Food Res. Int. 115 , 251–258 (2019).

Korneva, E. & Blockeel, H. Towards better evaluation of multi-target regression models. in ECML PKDD 2020 Workshops (eds. Koprinska, I. et al.) 353–362 (Springer International Publishing, Cham, 2020). https://doi.org/10.1007/978-3-030-65965-3_23 .

Gastón Ares. Mathematical and Statistical Methods in Food Science and Technology. (Wiley, 2013).

Grinsztajn, L., Oyallon, E. & Varoquaux, G. Why do tree-based models still outperform deep learning on tabular data? Preprint at http://arxiv.org/abs/2207.08815 (2022).

Gries, S. T. Statistics for Linguistics with R: A Practical Introduction. in Statistics for Linguistics with R (De Gruyter Mouton, 2021). https://doi.org/10.1515/9783110718256 .

Lundberg, S. M. et al. From local explanations to global understanding with explainable AI for trees. Nat. Mach. Intell. 2 , 56–67 (2020).

Ickes, C. M. & Cadwallader, K. R. Effects of ethanol on flavor perception in alcoholic beverages. Chemosens. Percept. 10 , 119–134 (2017).

Kato, M. et al. Influence of high molecular weight polypeptides on the mouthfeel of commercial beer. J. Inst. Brew. 127 , 27–40 (2021).

Wauters, R. et al. Novel Saccharomyces cerevisiae variants slow down the accumulation of staling aldehydes and improve beer shelf-life. Food Chem. 398 , 1–11 (2023).

Li, H., Jia, S. & Zhang, W. Rapid determination of low-level sulfur compounds in beer by headspace gas chromatography with a pulsed flame photometric detector. J. Am. Soc. Brew. Chem. 66 , 188–191 (2008).

Dercksen, A., Laurens, J., Torline, P., Axcell, B. C. & Rohwer, E. Quantitative analysis of volatile sulfur compounds in beer using a membrane extraction interface. J. Am. Soc. Brew. Chem. 54 , 228–233 (1996).

Molnar, C. Interpretable Machine Learning: A Guide for Making Black-Box Models Interpretable. (2020).

Zhao, Q. & Hastie, T. Causal interpretations of black-box models. J. Bus. Econ. Stat. Publ. Am. Stat. Assoc. 39 , 272–281 (2019).

Article   MathSciNet   Google Scholar  

Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical Learning. (Springer, 2019).

Labrado, D. et al. Identification by NMR of key compounds present in beer distillates and residual phases after dealcoholization by vacuum distillation. J. Sci. Food Agric. 100 , 3971–3978 (2020).

Lusk, L. T., Kay, S. B., Porubcan, A. & Ryder, D. S. Key olfactory cues for beer oxidation. J. Am. Soc. Brew. Chem. 70 , 257–261 (2012).

Gonzalez Viejo, C., Torrico, D. D., Dunshea, F. R. & Fuentes, S. Development of artificial neural network models to assess beer acceptability based on sensory properties using a robotic pourer: A comparative model approach to achieve an artificial intelligence system. Beverages 5 , 33 (2019).

Gonzalez Viejo, C., Fuentes, S., Torrico, D. D., Godbole, A. & Dunshea, F. R. Chemical characterization of aromas in beer and their effect on consumers liking. Food Chem. 293 , 479–485 (2019).

Gilbert, J. L. et al. Identifying breeding priorities for blueberry flavor using biochemical, sensory, and genotype by environment analyses. PLOS ONE 10 , 1–21 (2015).

Goulet, C. et al. Role of an esterase in flavor volatile variation within the tomato clade. Proc. Natl. Acad. Sci. 109 , 19009–19014 (2012).

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Borisov, V. et al. Deep Neural Networks and Tabular Data: A Survey. IEEE Trans. Neural Netw. Learn. Syst. 1–21 https://doi.org/10.1109/TNNLS.2022.3229161 (2022).

Statista. Statista Consumer Market Outlook: Beer - Worldwide.

Seitz, H. K. & Stickel, F. Molecular mechanisms of alcoholmediated carcinogenesis. Nat. Rev. Cancer 7 , 599–612 (2007).

Voordeckers, K. et al. Ethanol exposure increases mutation rate through error-prone polymerases. Nat. Commun. 11 , 3664 (2020).

Goelen, T. et al. Bacterial phylogeny predicts volatile organic compound composition and olfactory response of an aphid parasitoid. Oikos 129 , 1415–1428 (2020).

Article   ADS   Google Scholar  

Reher, T. et al. Evaluation of hop (Humulus lupulus) as a repellent for the management of Drosophila suzukii. Crop Prot. 124 , 104839 (2019).

Stein, S. E. An integrated method for spectrum extraction and compound identification from gas chromatography/mass spectrometry data. J. Am. Soc. Mass Spectrom. 10 , 770–781 (1999).

American Society of Brewing Chemists. Sensory Analysis Methods. (American Society of Brewing Chemists, St. Paul, MN, U.S.A., 1992).

McAuley, J., Leskovec, J. & Jurafsky, D. Learning Attitudes and Attributes from Multi-Aspect Reviews. Preprint at https://doi.org/10.48550/arXiv.1210.3926 (2012).

Meilgaard, M. C., Carr, B. T. & Carr, B. T. Sensory Evaluation Techniques. (CRC Press, Boca Raton). https://doi.org/10.1201/b16452 (2014).

Schreurs, M. et al. Data from: Predicting and improving complex beer flavor through machine learning. Zenodo https://doi.org/10.5281/zenodo.10653704 (2024).

Download references

Acknowledgements

We thank all lab members for their discussions and thank all tasting panel members for their contributions. Special thanks go out to Dr. Karin Voordeckers for her tremendous help in proofreading and improving the manuscript. M.S. was supported by a Baillet-Latour fellowship, L.C. acknowledges financial support from KU Leuven (C16/17/006), F.A.T. was supported by a PhD fellowship from FWO (1S08821N). Research in the lab of K.J.V. is supported by KU Leuven, FWO, VIB, VLAIO and the Brewing Science Serves Health Fund. Research in the lab of T.W. is supported by FWO (G.0A51.15) and KU Leuven (C16/17/006).

Author information

These authors contributed equally: Michiel Schreurs, Supinya Piampongsant, Miguel Roncoroni.

Authors and Affiliations

VIB—KU Leuven Center for Microbiology, Gaston Geenslaan 1, B-3001, Leuven, Belgium

Michiel Schreurs, Supinya Piampongsant, Miguel Roncoroni, Lloyd Cool, Beatriz Herrera-Malaver, Florian A. Theßeling & Kevin J. Verstrepen

CMPG Laboratory of Genetics and Genomics, KU Leuven, Gaston Geenslaan 1, B-3001, Leuven, Belgium

Leuven Institute for Beer Research (LIBR), Gaston Geenslaan 1, B-3001, Leuven, Belgium

Laboratory of Socioecology and Social Evolution, KU Leuven, Naamsestraat 59, B-3000, Leuven, Belgium

Lloyd Cool, Christophe Vanderaa & Tom Wenseleers

VIB Bioinformatics Core, VIB, Rijvisschestraat 120, B-9052, Ghent, Belgium

Łukasz Kreft & Alexander Botzki

AB InBev SA/NV, Brouwerijplein 1, B-3000, Leuven, Belgium

Philippe Malcorps & Luk Daenen

You can also search for this author in PubMed   Google Scholar

Contributions

S.P., M.S. and K.J.V. conceived the experiments. S.P., M.S. and K.J.V. designed the experiments. S.P., M.S., M.R., B.H. and F.A.T. performed the experiments. S.P., M.S., L.C., C.V., L.K., A.B., P.M., L.D., T.W. and K.J.V. contributed analysis ideas. S.P., M.S., L.C., C.V., T.W. and K.J.V. analyzed the data. All authors contributed to writing the manuscript.

Corresponding author

Correspondence to Kevin J. Verstrepen .

Ethics declarations

Competing interests.

K.J.V. is affiliated with bar.on. The other authors declare no competing interests.

Peer review

Peer review information.

Nature Communications thanks Florian Bauer, Andrew John Macintosh and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information, peer review file, description of additional supplementary files, supplementary data 1, supplementary data 2, supplementary data 3, supplementary data 4, supplementary data 5, supplementary data 6, supplementary data 7, reporting summary, source data, source data, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Schreurs, M., Piampongsant, S., Roncoroni, M. et al. Predicting and improving complex beer flavor through machine learning. Nat Commun 15 , 2368 (2024). https://doi.org/10.1038/s41467-024-46346-0

Download citation

Received : 30 October 2023

Accepted : 21 February 2024

Published : 26 March 2024

DOI : https://doi.org/10.1038/s41467-024-46346-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: Translational Research newsletter — top stories in biotechnology, drug discovery and pharma.

Help | Advanced Search

Computer Science > Human-Computer Interaction

Title: advancing explainable autonomous vehicle systems: a comprehensive review and research roadmap.

Abstract: Given the uncertainty surrounding how existing explainability methods for autonomous vehicles (AVs) meet the diverse needs of stakeholders, a thorough investigation is imperative to determine the contexts requiring explanations and suitable interaction strategies. A comprehensive review becomes crucial to assess the alignment of current approaches with the varied interests and expectations within the AV ecosystem. This study presents a review to discuss the complexities associated with explanation generation and presentation to facilitate the development of more effective and inclusive explainable AV systems. Our investigation led to categorising existing literature into three primary topics: explanatory tasks, explanatory information, and explanatory information communication. Drawing upon our insights, we have proposed a comprehensive roadmap for future research centred on (i) knowing the interlocutor, (ii) generating timely explanations, (ii) communicating human-friendly explanations, and (iv) continuous learning. Our roadmap is underpinned by principles of responsible research and innovation, emphasising the significance of diverse explanation requirements. To effectively tackle the challenges associated with implementing explainable AV systems, we have delineated various research directions, including the development of privacy-preserving data integration, ethical frameworks, real-time analytics, human-centric interaction design, and enhanced cross-disciplinary collaborations. By exploring these research directions, the study aims to guide the development and deployment of explainable AVs, informed by a holistic understanding of user needs, technological advancements, regulatory compliance, and ethical considerations, thereby ensuring safer and more trustworthy autonomous driving experiences.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is Qualitative Research? | Methods & Examples

What Is Qualitative Research? | Methods & Examples

Published on June 19, 2020 by Pritha Bhandari . Revised on June 22, 2023.

Qualitative research involves collecting and analyzing non-numerical data (e.g., text, video, or audio) to understand concepts, opinions, or experiences. It can be used to gather in-depth insights into a problem or generate new ideas for research.

Qualitative research is the opposite of quantitative research , which involves collecting and analyzing numerical data for statistical analysis.

Qualitative research is commonly used in the humanities and social sciences, in subjects such as anthropology, sociology, education, health sciences, history, etc.

• How does social media shape body image in teenagers?
• How do children and adults interpret healthy eating in the UK?
• What factors influence employee retention in a large organization?
• How is anxiety experienced around the world?
• How can teachers integrate social issues into science curriculums?

Table of contents

Approaches to qualitative research, qualitative research methods, qualitative data analysis, advantages of qualitative research, disadvantages of qualitative research, other interesting articles, frequently asked questions about qualitative research.

Qualitative research is used to understand how people experience the world. While there are many approaches to qualitative research, they tend to be flexible and focus on retaining rich meaning when interpreting data.

Common approaches include grounded theory, ethnography , action research , phenomenological research, and narrative research. They share some similarities, but emphasize different aims and perspectives.

Note that qualitative research is at risk for certain research biases including the Hawthorne effect , observer bias , recall bias , and social desirability bias . While not always totally avoidable, awareness of potential biases as you collect and analyze your data can prevent them from impacting your work too much.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Each of the research approaches involve using one or more data collection methods . These are some of the most common qualitative methods:

• Observations: recording what you have seen, heard, or encountered in detailed field notes.
• Interviews:  personally asking people questions in one-on-one conversations.
• Focus groups: asking questions and generating discussion among a group of people.
• Surveys : distributing questionnaires with open-ended questions.
• Secondary research: collecting existing data in the form of texts, images, audio or video recordings, etc.
• You take field notes with observations and reflect on your own experiences of the company culture.
• You distribute open-ended surveys to employees across all the company’s offices by email to find out if the culture varies across locations.
• You conduct in-depth interviews with employees in your office to learn about their experiences and perspectives in greater detail.

Qualitative researchers often consider themselves “instruments” in research because all observations, interpretations and analyses are filtered through their own personal lens.

For this reason, when writing up your methodology for qualitative research, it’s important to reflect on your approach and to thoroughly explain the choices you made in collecting and analyzing the data.

Qualitative data can take the form of texts, photos, videos and audio. For example, you might be working with interview transcripts, survey responses, fieldnotes, or recordings from natural settings.

Most types of qualitative data analysis share the same five steps:

• Prepare and organize your data. This may mean transcribing interviews or typing up fieldnotes.
• Review and explore your data. Examine the data for patterns or repeated ideas that emerge.
• Develop a data coding system. Based on your initial ideas, establish a set of codes that you can apply to categorize your data.
• Assign codes to the data. For example, in qualitative survey analysis, this may mean going through each participant’s responses and tagging them with codes in a spreadsheet. As you go through your data, you can create new codes to add to your system if necessary.
• Identify recurring themes. Link codes together into cohesive, overarching themes.

There are several specific approaches to analyzing qualitative data. Although these methods share similar processes, they emphasize different concepts.

Qualitative research often tries to preserve the voice and perspective of participants and can be adjusted as new research questions arise. Qualitative research is good for:

• Flexibility

The data collection and analysis process can be adapted as new ideas or patterns emerge. They are not rigidly decided beforehand.

• Natural settings

Data collection occurs in real-world contexts or in naturalistic ways.

• Meaningful insights

Detailed descriptions of people’s experiences, feelings and perceptions can be used in designing, testing or improving systems or products.

• Generation of new ideas

Open-ended responses mean that researchers can uncover novel problems or opportunities that they wouldn’t have thought of otherwise.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Researchers must consider practical and theoretical limitations in analyzing and interpreting their data. Qualitative research suffers from:

• Unreliability

The real-world setting often makes qualitative research unreliable because of uncontrolled factors that affect the data.

• Subjectivity

Due to the researcher’s primary role in analyzing and interpreting data, qualitative research cannot be replicated . The researcher decides what is important and what is irrelevant in data analysis, so interpretations of the same data can vary greatly.

• Limited generalizability

Small samples are often used to gather detailed data about specific contexts. Despite rigorous analysis procedures, it is difficult to draw generalizable conclusions because the data may be biased and unrepresentative of the wider population .

• Labor-intensive

Although software can be used to manage and record large amounts of text, data analysis often has to be checked or performed manually.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square goodness of fit test
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

There are five common approaches to qualitative research :

• Grounded theory involves collecting data in order to develop new theories.
• Ethnography involves immersing yourself in a group or organization to understand its culture.
• Narrative research involves interpreting stories to understand how people make sense of their experiences and perceptions.
• Phenomenological research involves investigating phenomena through people’s lived experiences.
• Action research links theory and practice in several cycles to drive innovative changes.

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

There are various approaches to qualitative data analysis , but they all share five steps in common:

• Prepare and organize your data.
• Review and explore your data.
• Develop a data coding system.
• Assign codes to the data.
• Identify recurring themes.

The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 22). What Is Qualitative Research? | Methods & Examples. Scribbr. Retrieved April 5, 2024, from https://www.scribbr.com/methodology/qualitative-research/

Is this article helpful?

Pritha Bhandari

Other students also liked, qualitative vs. quantitative research | differences, examples & methods, how to do thematic analysis | step-by-step guide & examples, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• Quantum Research

Landmark IBM error correction paper published on the cover of Nature

Ibm has created a quantum error-correcting code about 10 times more efficient than prior methods — a milestone in quantum computing research..

27 Mar 2024

Rafi Letzter

Share this blog

Today, the paper detailing those results was published as the cover story of the scientific journal Nature. 1

Last year, we demonstrated that quantum computers had entered the era of utility , where they are now capable of running quantum circuits better than classical computers can. Over the next few years, we expect to find speedups over classical computing and extract business value from these systems. But there are also algorithms with mathematically proven speedups over leading classical methods that require tuning quantum circuits with hundreds of millions, to billions, of gates. Expanding our quantum computing toolkit to include those algorithms requires us to find a way to compute that corrects the errors inherent to quantum systems — what we call quantum error correction.

Read how a paper from IBM and UC Berkeley shows a path toward useful quantum computing

Quantum error correction requires that we encode quantum information into more qubits than we would otherwise need. However, achieving quantum error correction in a scalable and fault-tolerant way has, to this point, been out of reach without considering scales of one million or more physical qubits. Our new result published today greatly reduces that overhead, and shows that error correction is within reach.

While quantum error correction theory dates back three decades, theoretical error correction techniques capable of running valuable quantum circuits on real hardware have been too impractical to deploy on quantum system. In our new paper, we introduce a new code, which we call the gross code , that overcomes that limitation.

This code is part of our broader strategy to bring useful quantum computing to the world.

While error correction is not a solved problem, this new code makes clear the path toward running quantum circuits with a billion gates or more on our superconducting transmon qubit hardware.

What is error correction?

Quantum information is fragile and susceptible to noise — environmental noise, noise from the control electronics, hardware imperfections, state preparation and measurement errors, and more. In order to run quantum circuits with millions to billions of gates, quantum error correction will be required.

Error correction works by building redundancy into quantum circuits. Many qubits work together to protect a piece of quantum information that a single qubit might lose to errors and noise.

On classical computers, the concept of redundancy is pretty straightforward. Classical error correction involves storing the same piece of information across multiple bits. Instead of storing a 1 as a 1 or a 0 as a 0, the computer might record 11111 or 00000. That way, if an error flips a minority of bits, the computer can treat 11001 as 1, or 10001 as 0. It’s fairly easy to build in more redundancy as needed to introduce finer error correction.

Things are more complicated on quantum computers. Quantum information cannot be copied and pasted like classical information, and the information stored in quantum bits is more complicated than classical data. And of course, qubits can decohere quickly, forgetting their stored information.

Research has shown that quantum fault tolerance is possible, and there are many error correcting schemes on the books. The most popular one is called the “surface code,” where qubits are arranged on a two-dimensional lattice and units of information are encoded into sub-units of the lattice.

But these schemes have problems.

First, they only work if the hardware’s error rates are better than some threshold determined by the specific scheme and the properties of the noise itself — and beating those thresholds can be a challenge.

Second, many of those schemes scale inefficiently — as you build larger quantum computers, the number of extra qubits needed for error correction far outpaces the number of qubits the code can store.

At practical code sizes where many errors can be corrected, the surface code uses hundreds of physical qubits per encoded qubit worth of quantum information, or more. So, while the surface code is useful for benchmarking and learning about error correction, it’s probably not the end of the story for fault-tolerant quantum computers.

Exploring “good” codes

The field of error correction buzzed with excitement in 2022 when Pavel Panteleev and Gleb Kalachev at Moscow State University published a landmark paper proving that there exist asymptotically good codes — codes where the number of extra qubits needed levels off as the quality of the code increases.

This has spurred a lot of new work in error correction, especially in the same family of codes that the surface code hails from, called quantum low-density parity check, or qLDPC codes. These qLDPC codes are quantum error correcting codes where the operations responsible for checking whether or not an error has occurred only have to act on a few qubits, and each qubit only has to participate in a few checks.

But this work was highly theoretical, focused on proving the possibility of this kind of error correction. It didn’t take into account the real constraints of building quantum computers. Most importantly, some qLDPC codes would require many qubits in a system to be physically linked to high numbers of other qubits. In practice, that would require quantum processors folded in on themselves in psychedelic hyper-dimensional origami, or entombed in wildly complex rats’ nests of wires.

In our paper, we looked for fault-tolerant quantum memory with a low qubit overhead, high error threshold, and a large code distance.

Bravyi, S., Cross, A., Gambetta, J., et al. High-threshold and low-overhead fault-tolerant quantum memory. Nature (2024). https://doi.org/10.1038/s41586-024-07107-7

In our Nature paper, we specifically looked for fault-tolerant quantum memory with a low qubit overhead, high error threshold, and a large code distance.

Let’s break that down:

Fault-tolerant: The circuits used to detect errors won't spread those errors around too badly in the process, and they can be corrected faster than they occur

Quantum memory: In this paper, we are only encoding and storing quantum information. We are not yet doing calculations on the encoded quantum information.

High error threshold: The higher the threshold, the higher amount of hardware errors the code will allow while still being fault tolerant. We were looking for a code that allowed us to operate the memory reliably at physical error rates as high as 0.001, so we wanted a threshold close to 1 percent.

Large code distance: Distance is the measure of how robust the code is — how many errors it takes to completely flip the value from 0 to 1 and vice versa. In the case of 00000 and 11111, the distance is 5. We wanted one with a large code distance that corrects more than just a couple errors. Large-distance codes can suppress noise by orders of magnitude even if the hardware quality is only marginally better than the code threshold. In contrast, codes with a small distance become useful only if the hardware quality is significantly better than the code threshold.

Low qubit overhead: Overhead is the number of extra qubits required for correcting errors. We want the number of qubits required to do error correction to be far less than we need for a surface code of the same quality, or distance.

We’re excited to report that our team’s mathematical analysis found concrete examples of qLDPC codes that met all of these required conditions. These fall into a family of codes called “Bivariate Bicycle (BB)” codes. And they are going to shape not only our research going forward, but how we architect physical quantum systems.

The gross code

While many qLDPC code families show great promise for advancing error correction theory, most aren’t necessarily pragmatic for real-world application. Our new codes lend themselves better to practical implementation because each qubit needs only to connect to six others, and the connections can be routed on just two layers.

To get an idea of how the qubits are connected, imagine they are put onto a square grid, like a piece of graph paper. Curl up this piece of graph paper so that it forms a tube, and connect the ends of the tube to make a donut. On this donut, each qubit is connected to its four neighbors and two qubits that are farther away on the surface of the donut. No more connections needed.

The good news is we don’t actually have to embed our qubits onto a donut to make these codes work — we can accomplish this by folding the surface differently and adding a few other long-range connectors to satisfy mathematical requirements of the code. It’s an engineering challenge, but much more feasible than a hyper-dimensional shape.

We explored some codes that have this architecture and focused on a particular [[144,12,12]] code. We call this code the gross code because 144 is a gross (or a dozen dozen). It requires 144 qubits to store data — but in our specific implementation, it also uses another 144 qubits to check for errors, so this instance of the code uses 288 qubits. It stores 12 logical qubits well enough that fewer than 12 errors can be detected. Thus: [[144,12,12]].

Using the gross code, you can protect 12 logical qubits for roughly a million cycles of error checks using 288 qubits. Doing roughly the same task with the surface code would require nearly 3,000 qubits.

This is a milestone. We are still looking for qLDPC codes with even more efficient architectures, and our research on performing error-corrected calculations using these codes is ongoing. But with this publication, the future of error correction looks bright.

Fig. 1 | Tanner graphs of surface and BB codes.

Fig. 1 | Tanner graphs of surface and BB codes. a, Tanner graph of a surface code, for comparison. b, Tanner graph of a BB code with parameters [[144, 12, 12]] embedded into a torus. Any edge of the Tanner graph connects a data and a check vertex. Data qubits associated with the registers q(L) and q(R) are shown by blue and orange circles. Each vertex has six incident edges including four short-range edges (pointing north, south, east and west) and two long-range edges. We only show a few long-range edges to avoid clutter. Dashed and solid edges indicate two planar subgraphs spanning the Tanner graph, see the Methods. c, Sketch of a Tanner graph extension for measuring Z ˉ \={Z} and X ˉ \={X} following ref. 50, attaching to a surface code. The ancilla corresponding to the X ˉ \={X} measurement can be connected to a surface code, enabling load-store operations for all logical qubits by means of quantum teleportation and some logical unitaries. This extended Tanner graph also has an implementation in a thickness-2 architecture through the A and B edges (Methods).

Fig. 2 | Syndrome measurement circuit.

Fig. 2 | Syndrome measurement circuit. Full cycle of syndrome measurements relying on seven layers of CNOTs. We provide a local view of the circuit that only includes one data qubit from each register q(L) and q(R) . The circuit is symmetric under horizontal and vertical shifts of the Tanner graph. Each data qubit is coupled by CNOTs with three X-check and three Z-check qubits: see the Methods for more details.

Why error correction matters

Today, our users benefit from novel error mitigation techniques — methods for reducing or eliminating the effect of noise when calculating observables, alongside our work suppressing errors at the hardware level. This work brought us into the era of quantum utility. IBM researchers and partners all over the world are exploring practical applications of quantum computing today with existing quantum systems. Error mitigation lets users begin looking for quantum advantage on real quantum hardware.

But error mitigation comes with its own overhead, requiring running the same executions repeatedly so that classical computers can use statistical methods to extract an accurate result. This limits the scale of the programs you can run, and increasing that scale requires tools beyond error mitigation — like error correction.

Last year, we debuted a new roadmap laying out our plan to continuously improve quantum computers over the next decade. This new paper is an important example of how we plan to continuously increasing the complexity (number of gates) of the quantum circuits that can be run on our hardware. It will allow us to transition from running circuits with 15,000 gates to 100 million, or even 1 billion gates.

Bravyi, S., Cross, A.W., Gambetta, J.M. et al. High-threshold and low-overhead fault-tolerant quantum memory. Nature 627, 778–782 (2024). https://doi.org/10.1038/s41586-024-07107-7

Start using our 100+ qubit systems

Keep exploring.

Exploring measurement error mitigation with the Mthree Qiskit extension

Computing with error-corrected quantum computers.

The era of quantum utility must also be the era of responsible quantum computing

Logical gates with magic state distillation