how to do an effective research

15 Steps to Good Research

• Define and articulate a research question (formulate a research hypothesis). How to Write a Thesis Statement (Indiana University)
• Identify possible sources of information in many types and formats. Georgetown University Library's Research & Course Guides
• Judge the scope of the project.
• Reevaluate the research question based on the nature and extent of information available and the parameters of the research project.
• Select the most appropriate investigative methods (surveys, interviews, experiments) and research tools (periodical indexes, databases, websites).
• Plan the research project. Writing Anxiety (UNC-Chapel Hill) Strategies for Academic Writing (SUNY Empire State College)
• Retrieve information using a variety of methods (draw on a repertoire of skills).
• Refine the search strategy as necessary.
• Write and organize useful notes and keep track of sources. Taking Notes from Research Reading (University of Toronto) Use a citation manager: Zotero or Refworks
• Evaluate sources using appropriate criteria. Evaluating Internet Sources
• Synthesize, analyze and integrate information sources and prior knowledge. Georgetown University Writing Center
• Revise hypothesis as necessary.
• Use information effectively for a specific purpose.
• Understand such issues as plagiarism, ownership of information (implications of copyright to some extent), and costs of information. Georgetown University Honor Council Copyright Basics (Purdue University) How to Recognize Plagiarism: Tutorials and Tests from Indiana University
• Cite properly and give credit for sources of ideas. MLA Bibliographic Form (7th edition, 2009) MLA Bibliographic Form (8th edition, 2016) Turabian Bibliographic Form: Footnote/Endnote Turabian Bibliographic Form: Parenthetical Reference Use a citation manager: Zotero or Refworks

Adapted from the Association of Colleges and Research Libraries "Objectives for Information Literacy Instruction" , which are more complete and include outcomes. See also the broader "Information Literacy Competency Standards for Higher Education."

How to Conduct Effective Research: Tips and Tricks for Beginners

• 11 July 2023

Introduction

Research serves as the foundation for advances in every conceivable subject and is the cornerstone of progress. Starting a research journey can be intimidating for novices. But if you have the correct tools and approaches, it may be a worthwhile endeavor. This blog article seeks to instruct new researchers on how to conduct fruitful research by offering helpful advice and strategies to help them become productive researchers.

1. Define Your Research Question

Determining your research question is the first stage in conducting successful research. A well-crafted question gives you direction and helps you focus your efforts on a certain objective. Ask a question that is as specific as you can. A better inquiry may be, “How is climate change affecting crop production in California?” rather than, “What are the effects of climate change?”

2. Develop a Research Plan

A roadmap is essential for effective research. Describe your strategy while considering the type of information you require, potential sources, and a rough time frame. You can effectively manage your time by taking this step, which will also give your research structure and keep you on track.

3. Understand the Types of Research

There are two basic categories of research: primary and secondary. Primary research is when you collect your own first-hand information, such as through surveys or experiments. Using information gathered by another person, such as data from books, papers, or scholarly articles, is known as secondary research. Knowing the difference can help you choose the approach that best fits your research issue and available resources.

4. Use a Variety of Sources

Your research’s scope and depth are constrained if you rely solely on one type of source. Utilize a variety of credible websites, books, scholarly articles, podcasts, and even multimedia sources like documentaries. Remember that the trustworthiness of your study is influenced by the caliber of your sources.

5. Learn How to Use Databases Effectively

Your time will be much reduced if you learn how to use databases. Scholarly articles can be found in abundance in databases like JSTOR, PubMed, and Google Scholar, among others. Recall that various databases have various advantages; thus, investigate a few to determine which ones best meet your requirements.

6. Master Effective Search Techniques

The research process can be significantly sped up by using excellent search techniques. Learn how to broaden or narrow your search using Boolean operators (AND, OR, NOT), phrase searching using quotation marks, and truncation symbols. You can quickly uncover additional pertinent sources using these methods.

7. Evaluate Your Sources

Information is not all created equal. Always assess the trustworthiness and veracity of your sources. Verify the author’s credentials, the publishing date, and the veracity of the information. Websites with the.edu,.gov, or.org extensions are usually more trustworthy than those with the.com extension.

8. Keep Track of Your Sources

To properly reference your sources and prevent plagiarism, keeping track of them is crucial. Making a system (such as a spreadsheet) where you enter all the relevant citation data as you go is an excellent practice. Also very beneficial are reference management programs like EndNote, Mendeley, or Zotero..

9. Take Effective Notes

Making thorough notes can help you remember knowledge and draw connections across sources. To improve understanding, try to summarise the main ideas in your own words. To show the data and connections, you can also utilize charts, mind maps, or colors.

10. Analyze and Synthesize Your Findings

Making meaning of the information you have gathered involves analysis and synthesis. You must comprehend the situation, evaluate many points of view, spot trends, and reach conclusions. The development of your arguments or the development of your hypotheses depends on this step.

11. Don’t Be Afraid to Ask for Help

Finally, if you are stuck, don’t be afraid to seek for assistance. Teachers, classmates with more expertise, and librarians can all offer helpful advice. Keep in mind that even experienced researchers occasionally require help.

One size does not fit all when it comes to research. Different projects call for various approaches and plans. However, you can become a successful researcher by defining your question, creating a plan, utilizing a variety of sources, learning efficient search strategies, evaluating your sources, keeping track of references, taking good notes, analyzing and synthesizing data, and asking for assistance when necessary. The most crucial advice is to always be curious and to keep learning. Enjoy your research!

It’s important to keep in mind that understanding how to perform excellent research is a journey that requires constant practice, learning, and development. Research can be a fulfilling and enlightening experience as you gain experience and get more familiar with the procedure. Wishing you well as you pursue your research!

Share This Post:

Leave feedback about this cancel reply.

• Quality 5 4 3 2 1
• Price 5 4 3 2 1
• Service 5 4 3 2 1

• PRO Courses Guides New Tech Help Pro Expert Videos About wikiHow Pro Upgrade Sign In
• EDIT Edit this Article
• EXPLORE Tech Help Pro About Us Random Article Quizzes Request a New Article Community Dashboard This Or That Game Popular Categories Arts and Entertainment Artwork Books Movies Computers and Electronics Computers Phone Skills Technology Hacks Health Men's Health Mental Health Women's Health Relationships Dating Love Relationship Issues Hobbies and Crafts Crafts Drawing Games Education & Communication Communication Skills Personal Development Studying Personal Care and Style Fashion Hair Care Personal Hygiene Youth Personal Care School Stuff Dating All Categories Arts and Entertainment Finance and Business Home and Garden Relationship Quizzes Cars & Other Vehicles Food and Entertaining Personal Care and Style Sports and Fitness Computers and Electronics Health Pets and Animals Travel Education & Communication Hobbies and Crafts Philosophy and Religion Work World Family Life Holidays and Traditions Relationships Youth
• Browse Articles
• Learn Something New
• Quizzes Hot
• This Or That Game New
• Train Your Brain
• Explore More
• Support wikiHow
• About wikiHow
• Log in / Sign up
• Education and Communications

How to Research Effectively

Last Updated: June 26, 2023 Fact Checked

This article was co-authored by Jennifer Mueller, JD . Jennifer Mueller is an in-house legal expert at wikiHow. Jennifer reviews, fact-checks, and evaluates wikiHow's legal content to ensure thoroughness and accuracy. She received her JD from Indiana University Maurer School of Law in 2006. There are 15 references cited in this article, which can be found at the bottom of the page. This article has been fact-checked, ensuring the accuracy of any cited facts and confirming the authority of its sources. This article has been viewed 75,995 times.

Whether you're working on a research paper for school, a fact page for a website, or an article for a magazine, your final product will rise or fall based on the quality of your research. To research effectively, you must plan your project in advance so you can make the most efficient use of your time and resources. Effective research is as much about planning as it is about the research itself. Analyze your sources carefully and cite them properly in your final report. [1] X Research source

Organizing Your Project

• If you're working on a research paper for school, you can use the page requirements (or limits) as a guide to narrow your topic.
• Use the same approach if you're writing an article for a website or print publication, and have been given a word limit.
• For example, "butterflies" probably is too broad for a 20-page research paper. If you know you want to write about butterflies, narrow your topic to a particular species of butterflies, or butterflies found in a particular geographic area.
• You may have to do some basic preliminary research to appropriately narrow your topic. This gives you a sense of how much information is available. If it's more than you could possibly dig through, consider narrowing your topic again.

• Typically, you'll want to create a basic thesis statement for your final report. This is what you ultimately want your research to prove.
• Creating a thesis statement before you've even begun your research might seem backwards, but to research effectively, you need to know what you're looking for.
• For example, maybe you have narrowed your butterfly topic to focus on why butterflies are attracted to a particular park in your hometown. Your thesis statement at this stage might be "Butterflies are attracted to Panorama Park because of the flowers planted there."
• Keep in mind that you can adjust your thesis statement later if your research shows that it's inaccurate or inappropriate.
• To return to the butterfly example, your thesis statement means you are focusing your research on the park you named and the flowers that are planted in that park. If you find out through your research that the butterflies are not particularly fond of those flowers, you may have to adjust your thesis.

• Essentially, the scope of your project is how far you're going to go. For example, if you're writing a research paper on the role of the Germans in the American Revolutionary War, the role of the French in the same war would be outside the scope of your project.
• The scope of your project may be dictated to you, for example through an assignment sheet from your teacher or professor.
• Keep your scope in mind as you're planning your research strategy. You don't want to get bogged down in your research or spend too much time chasing after information that turns out to be irrelevant.
• If you find something that doesn't fall within the scope of your project, you can quickly discard it and move on to something else.

• For example, if you're doing research on the butterflies in a local park, surveys of local residents on how they feel about the butterflies probably isn't going to be helpful for you. However, research conducted by students at a nearby university might be beneficial.
• Focus on the type of source you need. If you're working on a political science or sociology research paper, polls and statistics may be relevant.
• However, if you're writing a paper in the hard sciences, you want to focus on scientific experiments and journal articles.
• If your teacher or professor has given you a reading list or identified good sources, start there. Consider it a jumping-off point rather than something to hit as a last resort.

• You may want to do basic internet searches just so you can get an idea of how effective your key words will be.
• For example, if you're writing a research paper on butterflies in a local park, the key word "butterfly" will bring up entirely too much information to be useful for you.
• Try to make your key words as specific as possible. Once you have your basic key words, look for synonyms of those words, or other related words or phrases.
• Write all the words you brainstorm on a sheet of paper so you don't forget them or lose track of them.
• Leave plenty of space around each word so you can take notes or make comments on the effectiveness of each word search as you research.

• One of the easiest methods to organize your research notes is to buy a stack of index cards. At the top of each card, you'll write the name of the source you're taking notes about.
• If you're required to use a particular citation format, you might want to create a formatted citation for the source at the top of the card. It will save you time later on when you're working on your final paper.
• Beyond index cards, you'll also want to gather highlighters, tabs or flags, and adhesive notes. These will help you keep track of information you find in print sources and organize your thoughts as you research.

Finding Good Sources

• Whether a source is considered primary depends on its relation to the topic. Some sources can be primary for one topic but not for another.
• For example, if you're doing research on D-Day, newspaper articles from June 6, 1944 or a few days afterward would be considered primary sources. However, newspaper articles from 2014 would not.
• Primary sources are the heart of research, and what differentiates an actual research project from a book report. If you're reading what someone else wrote about your topic and then summarizing what you read, that's a book report – not research.
• Focusing on primary sources means you may have to get off the internet and go to a library. Some primary sources, such as historic documents and newspapers, have been digitized and are available online. However, many have not.
• Additionally, databases and digital archives of primary sources often are only accessible by subscription. Public and university libraries often maintain subscriptions to these digital resources, but you'll have to use a library computer to get to them.

• If you're using the index-card method, jot down a few words to indicate the idea or fact you found in that source. Include a page number if available so you can cite to it directly or quickly find it again.
• If you see something in a source you want to quote, put it on your notecard. However, do something to make it stand out from the rest of your notes so you don't inadvertently include it in your report as your own words.
• For example, you may want to highlight a quote so it stands out at a glance as a quote, and not your own notes. You also could write it in a different color.
• Likewise, as you read a source you may have thoughts of your own, or additional questions you want to explore. Highlight those or write them in a different color so you can instantly recognize them as your own thoughts, not something that should be attributed to your source.

• The authors of secondary sources are not involved directly in the subject they're discussing – otherwise they would be primary sources.
• Rather, they are using their own knowledge and expertise to interpret, comment, analyze, investigate, or contextualize something.
• Secondary sources can be extremely valuable to you in making sense of the primary sources you've found, but it's important to avoid over-using them.
• With secondary sources, the most important thing to keep in mind when taking notes are ideas. If an author you read has an idea or theory about the topic you're researching that resonates with you, take out a note card and write that idea down. Then if you bring it up in your report you can credit them with having the idea.
• Taking a theory or idea you read somewhere else and acting like you came up with it is plagiarism. The most effective research builds on ideas and theories introduced by others, rather than stealing them.

• When something was published can be a strong indicator of whether you should rely on it. Ideally, you should check the date of publication before you even read the material.
• Whether dates matter depends on the subject you're researching. If you're researching a historical event, you'll likely be making use of material dated any time from the date of the event to the present.
• However, in other areas of research older sources are likely to be outdated and unusable. For example, if you were researching how schoolchildren use the internet, you wouldn't want to use an article written in the early 2000s as a source. Use of the internet has changed significantly since then.
• If an author is biased, it can undermine their authority, even if they have tremendous expertise on the subject you're researching.
• For example, suppose you're researching the butterflies in a local park. You find a lengthy, detailed, and informative article in a reputable science journal that discusses the very butterflies you're researching.
• However, the article is written by a scientist who is infamous for their denial of climate change. This position means they aren't well-respected by most of the scientific community.
• If their bias has little to nothing to do with the subject at hand, or if they wrote the butterfly article before they spoke out against climate change, you might still be able to use that source.
• If you do plan to use a source with a potentially biased or compromised author, use caution. If you're working on a project for school, ask your teacher or professor about it.

• These sources can be helpful for you in finding new articles and sources to use, but you typically don't want to cite them directly in your research.
• Newspaper and magazine articles also can be tertiary sources. Pay attention to what you're reading. If the writer is talking about, for example, a series of studies conducted at different universities that came to opposing conclusions, seek out the studies themselves rather than citing to that article.
• At the same time, tertiary sources can be a quick way to find new sources, and to understand more about a potentially dense article or book before you dive into it yourself.

Citing Your Research

• To understand plagiarism, put yourself in the author's shoes. They put a lot of work into their own project to come up with those words. Properly citing them acknowledges that work and shows that you appreciate it.
• Moreover, you are showing your teachers, editors, or colleagues that you are an upstanding and respectful member of the academic or journalistic community – and that you're making a productive contribution to the field of research.
• Be aware that many high schools, universities, and other publications use plagiarism checkers that will find words you have copied.
• Even if you think you won't get caught because you're using a relatively obscure source, the proliferation of plagiarism checkers makes it more likely that you will.

• Always provide a citation for facts – things that can be proven true or false. This is so your reader can confirm the fact that you've presented in your report. The best source for a fact is always a primary source.
• You also should cite opinions, analysis, speculation, or interpretation that is not your own. For example, suppose you have a source for your butterfly research that believes the butterflies are attracted to the park because people walk their dogs there and butterflies love dogs.
• If you want to use this opinion in your report, credit it to the specific person that said it. Using phrases like "studies show" or "some scientists believe" won't tell your reader anything.
• Instead, you might write "Meredith Monarch, butterfly expert at the University of Lepidoptera, speculated that butterflies frequent the park because they enjoy frolicking with the dogs there."

• Citation styles or formats may seem arbitrary to you, but for those in the field, the punctuation and formatting serve as short-hand. Someone familiar with a particular style can look at a citation and instantly know what each thing means.
• Different fields use different citation methods, which can be confusing if you've done research in several areas of study.
• For example, if you're writing a science paper, you typically will use APA (American Psychological Association) style.
• However, for a research paper in history or political science, you'll more likely use the Turabian style.

• Generally, you only want to have a direct quote from a source when the words themselves are of particular importance, or if the language is so original and succinct it would be nearly impossible to phrase it better.
• For example, if you're writing a research paper on the Gettysburg Address, you likely will include direct quotes from the speech itself, since those words would hold particular importance.
• A source's original words can provide color or enhance the meaning of your own thoughts, or can be quoted as an example of a particular line of thought.
• When you paraphrase, you're putting the information from a source into your own words, rather than quoting the source directly.
• Keep in mind that paraphrasing doesn't mean simply changing a few words to synonymous words, or altering the word order.
• For example, "Quick brown foxes jump over lazy dogs" would not be an acceptable paraphrase of "The quick brown fox jumped over the lazy dog."

Expert Q&A

You might also like.

• ↑ https://library.georgetown.edu/tutorials/research-guides/15-steps
• ↑ https://libguides.usc.edu/writingguide/narrowtopic
• ↑ https://libguides.usc.edu/writingguide/introduction
• ↑ https://libguides.usc.edu/writingguide/citingsources
• ↑ https://umassglobal.libguides.com/keywords
• ↑ https://umb.libguides.com/PrimarySources/secondary
• ↑ https://advice.writing.utoronto.ca/researching/notes-from-research/
• ↑ https://open.lib.umn.edu/writingforsuccess/chapter/11-4-strategies-for-gathering-reliable-information/
• ↑ https://libguides.mit.edu/citing
• ↑ https://ohiostate.pressbooks.pub/choosingsources/chapter/degree-of-bias/
• ↑ https://ohiostate.pressbooks.pub/choosingsources/chapter/primary-secondary-tertiary-sources/
• ↑ https://edu.gcfglobal.org/en/useinformationcorrectly/avoiding-plagiarism/1/
• ↑ https://openoregon.pressbooks.pub/wrd/chapter/crediting-your-sources/
• ↑ https://pitt.libguides.com/citationhelp
• ↑ https://owl.purdue.edu/owl/research_and_citation/using_research/quoting_paraphrasing_and_summarizing/index.html

About This Article

To research effectively, look through library databases and digital archives to find helpful, reliable sources related to your topic. For example, if you’re researching something in the science field, you can visit your university library to look at articles in their scientific journals. While going through your sources, take notes by jotting down useful ideas on a notecard along with the page numbers you found them on. You may also want to highlight information in order to easily refer to what's important later on. For more tips from our co-author, including how to quote and cite a source responsibly, keep reading! Did this summary help you? Yes No

• Send fan mail to authors

Reader Success Stories

Muhammad Hussain

Nov 28, 2019

Did this article help you?

Mar 21, 2017

Featured Articles

Trending Articles

Watch Articles

• Terms of Use
• Privacy Policy
• Do Not Sell or Share My Info
• Not Selling Info

wikiHow Tech Help Pro:

Develop the tech skills you need for work and life

• Follow us on Facebook
• Follow us on Twitter
• Criminal Justice
• Environment
• Politics & Government
• Race & Gender

Expert Commentary

Research strategy guide for finding quality, credible sources

Strategies for finding academic studies and other information you need to give your stories authority and depth

Republish this article

This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License .

by Keely Wilczek, The Journalist's Resource May 20, 2011

This <a target="_blank" href="https://journalistsresource.org/home/research-strategy-guide/">article</a> first appeared on <a target="_blank" href="https://journalistsresource.org">The Journalist's Resource</a> and is republished here under a Creative Commons license.<img src="https://journalistsresource.org/wp-content/uploads/2020/11/cropped-jr-favicon-150x150.png" style="width:1em;height:1em;margin-left:10px;">

Knowing how to conduct deeper research efficiently and effectively is a critical skill for journalists — especially in the information age. It is, like other facets of the profession such as interviewing, a matter of practice and establishing good habits. And once you find a successful routine for information-gathering, it will pay dividends time and again.

Journalists need to be able to do many kinds of research. This article focuses on creating a research strategy that will help you find academic studies and related scholarly information. These sources can, among other things, give your stories extra authority and depth — and thereby distinguish your work. You can see examples of such studies — and find many relevant ones for your stories — by searching the Journalist’s Resource database . But that is just a representative sample of what exists in the research world.

The first step is to create a plan for seeking the information you need. This requires you to take time initially and to proceed with care, but it will ultimately pay off in better results. The research strategy covered in this article involves the following steps:

Get organized

Articulate your topic, locate background information.

• Identify your information needs

List keywords and concepts for search engines and databases

Consider the scope of your topic, conduct your searches, evaluate the information sources you found, analyze and adjust your research strategy.

Being organized is an essential part of effective research strategy. You should create a record of your strategy and your searches. This will prevent you from repeating searches in the same resources and from continuing to use ineffective terms. It will also help you assess the success or failure of your research strategy as you go through the process. You also may want to consider tracking and organizing citations and links in bibliographic software such as Zotero . (See this helpful resource guide about using Zotero.)

Next, write out your topic in a clear and concise manner. Good research starts with a specific focus.

For example, let’s say you are writing a story about the long-range health effects of the explosion at the Chernobyl Nuclear Power Plant based on a study published in Environmental Health Perspectives titled, “The Chernobyl Accident 20 Years On: An Assessment of the Health Consequences and the International Response.” (The study is summarized in Journalist’s Resource here .)

A statement of your topic might be, “Twenty years after the Chernobyl disaster, scientists are still learning the affects of the accident on the health of those who lived in the surrounding area and their descendants.”

If you have a good understanding of the Chernobyl disaster, proceed to the next step, “Identify the information you need.” If not, it’s time to gather background information. This will supply you with the whos and the whens of the topic. It will also provide you with a broader context as well as the important terminology.

Excellent sources of background information are subject-specific encyclopedias and dictionaries, books, and scholarly articles, and organizations’ websites. You should always consult more than one source so you can compare for accuracy and bias.

For your story about Chernobyl, you might want to consult some of the following sources:

• Frequently Asked Chernobyl Questions , International Atomic Agency
• Chernobyl Accident 1986 , World Nuclear Association
• Chernobyl: Consequences of the Catastrophe for People and the Environment , New York Academy of Sciences, 2009.
• “Chernobyl Disaster,” Encyclopedia Britannica, last updated 2013.

Identify the information you need

What information do you need to write your story? One way to determine this is to turn your overall topic into a list of questions to be answered. This will help you identify the type and level of information you need. Some possible questions on consequences of the Chernobyl accident are:

• What are the proven health effects?
• What are some theorized health effects?
• Is there controversy about any of these studies?
• What geographic area is being studied?
• What are the demographic characteristics of the population being studied?
• Was there anything that could have been done at the time to mitigate these effects?

Looking at these questions, it appears that scientific studies and scholarly articles about those studies, demographic data, disaster response analysis, and government documents and publications from the Soviet Union and Ukraine would be needed.

Now you need to determine what words you will use to enter in the search boxes within resources. One way to begin is to extract the most important words and phrases from the questions produced in the previous step. Next, think about alternative words and phrases that you might use. Always keep in mind that different people may write or talk about the same topic in different ways. Important concepts can referred to differently or be spelled differently depending on country of origin or field of study.

For the Chernobyl health story, some search keyword options are: “Chernobyl,” “Chornobyl”; “disaster,” “catastrophe,” “explosion”; “health,” “disease,” “illness,” “medical conditions”; “genetic mutation,” “gene mutation,” “germ-line mutation,” “hereditary disease.” Used in different combinations, these can unearth a wide variety of resources.

Next you should identify the scope of your topic and any limitations it puts on your searches. Some examples of limitations are language, publication date, and publication type. Every database and search engine will have its own rules so you may need to click on an advanced search option in order to input these limitations.

It is finally time to start looking for information but identifying which resources to use is not always easy to do. First, if you are part of an organization, find out what, if any, resources you have access to through a subscription. Examples of subscription resources are LexisNexis and JSTOR. If your organization does not provide subscription resources, find out if you can get access to these sources through your local library. Should you not have access to any subscription resources appropriate for your topic, look at some of the many useful free resources on the internet.

Here are some examples of sources for free information:

• PLoS , Public Library of Science
• Google Scholar
• SSRN , Social Science Research Network
• FDsys , U.S. Government documents and publications
• World Development Indicators , World Bank
• Pubmed , service of the U.S. National Library of Medicine

More quality sites, and search tips, are here among the other research articles at Journalist’s Resource.

As you only want information from the most reliable and suitable sources, you should always evaluate your results. In doing this, you can apply journalism’s Five W’s (and One H):

• Who : Who is the author and what are his/her credentials in this topic?
• What: Is the material primary or secondary in nature?
• Where: Is the publisher or organization behind the source considered reputable? Does the website appear legitimate?
• When: Is the source current or does it cover the right time period for your topic?
• Why: Is the opinion or bias of the author apparent and can it be taken into account?
• How: Is the source written at the right level for your needs? Is the research well-documented?

Were you able to locate the information you needed? If not, now it is time to analyze why that happened. Perhaps there are better resources or different keywords and concepts you could have tried. Additional background information might supply you with other terminology to use. It is also possible that the information you need is just not available in the way you need it and it may be necessary to consult others for assistance like an expert in the topic or a professional librarian.

Keely Wilczek is a research librarian at the Harvard Kennedy School. Tags: training

About The Author

Keely Wilczek

Basic Steps in the Research Process

The following steps outline a simple and effective strategy for writing a research paper. Depending on your familiarity with the topic and the challenges you encounter along the way, you may need to rearrange these steps.

Step 1: Identify and develop your topic

Selecting a topic can be the most challenging part of a research assignment. Since this is the very first step in writing a paper, it is vital that it be done correctly. Here are some tips for selecting a topic:

• Select a topic within the parameters set by the assignment. Many times your instructor will give you clear guidelines as to what you can and cannot write about. Failure to work within these guidelines may result in your proposed paper being deemed unacceptable by your instructor.
• Select a topic of personal interest to you and learn more about it. The research for and writing of a paper will be more enjoyable if you are writing about something that you find interesting.
• Select a topic for which you can find a manageable amount of information. Do a preliminary search of information sources to determine whether existing sources will meet your needs. If you find too much information, you may need to narrow your topic; if you find too little, you may need to broaden your topic.
• Be original. Your instructor reads hundreds of research papers every year, and many of them are on the same topics (topics in the news at the time, controversial issues, subjects for which there is ample and easily accessed information). Stand out from your classmates by selecting an interesting and off-the-beaten-path topic.
• Still can't come up with a topic to write about? See your instructor for advice.

Once you have identified your topic, it may help to state it as a question. For example, if you are interested in finding out about the epidemic of obesity in the American population, you might pose the question "What are the causes of obesity in America ?" By posing your subject as a question you can more easily identify the main concepts or keywords to be used in your research.

Step 2 : Do a preliminary search for information

Before beginning your research in earnest, do a preliminary search to determine whether there is enough information out there for your needs and to set the context of your research. Look up your keywords in the appropriate titles in the library's Reference collection (such as encyclopedias and dictionaries) and in other sources such as our catalog of books, periodical databases, and Internet search engines. Additional background information may be found in your lecture notes, textbooks, and reserve readings. You may find it necessary to adjust the focus of your topic in light of the resources available to you.

Step 3: Locate materials

With the direction of your research now clear to you, you can begin locating material on your topic. There are a number of places you can look for information:

If you are looking for books, do a subject search in One Search . A Keyword search can be performed if the subject search doesn't yield enough information. Print or write down the citation information (author, title,etc.) and the location (call number and collection) of the item(s). Note the circulation status. When you locate the book on the shelf, look at the books located nearby; similar items are always shelved in the same area. The Aleph catalog also indexes the library's audio-visual holdings.

Use the library's  electronic periodical databases  to find magazine and newspaper articles. Choose the databases and formats best suited to your particular topic; ask at the librarian at the Reference Desk if you need help figuring out which database best meets your needs. Many of the articles in the databases are available in full-text format.

Use search engines ( Google ,  Yahoo , etc.) and subject directories to locate materials on the Internet. Check the  Internet Resources  section of the NHCC Library web site for helpful subject links.

Step 4: Evaluate your sources

See the  CARS Checklist for Information Quality   for tips on evaluating the authority and quality of the information you have located. Your instructor expects that you will provide credible, truthful, and reliable information and you have every right to expect that the sources you use are providing the same. This step is especially important when using Internet resources, many of which are regarded as less than reliable.

Step 5: Make notes

Consult the resources you have chosen and note the information that will be useful in your paper. Be sure to document all the sources you consult, even if you there is a chance you may not use that particular source. The author, title, publisher, URL, and other information will be needed later when creating a bibliography.

Step 6: Write your paper

Begin by organizing the information you have collected. The next step is the rough draft, wherein you get your ideas on paper in an unfinished fashion. This step will help you organize your ideas and determine the form your final paper will take. After this, you will revise the draft as many times as you think necessary to create a final product to turn in to your instructor.

Step 7: Cite your sources properly

Give credit where credit is due; cite your sources.

Citing or documenting the sources used in your research serves two purposes: it gives proper credit to the authors of the materials used, and it allows those who are reading your work to duplicate your research and locate the sources that you have listed as references. The  MLA  and the  APA  Styles are two popular citation formats.

Failure to cite your sources properly is plagiarism. Plagiarism is avoidable!

Step 8: Proofread

The final step in the process is to proofread the paper you have created. Read through the text and check for any errors in spelling, grammar, and punctuation. Make sure the sources you used are cited properly. Make sure the message that you want to get across to the reader has been thoroughly stated.

Additional research tips:

• Work from the general to the specific -- find background information first, then use more specific sources.
• Don't forget print sources -- many times print materials are more easily accessed and every bit as helpful as online resources.
• The library has books on the topic of writing research papers at call number area LB 2369.
• If you have questions about the assignment, ask your instructor.
• If you have any questions about finding information in the library, ask the librarian.

Contact Information

Craig larson.

Librarian 763-424-0733 [email protected] Zoom:  myzoom   Available by appointment

Get Started

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• HHS Author Manuscripts

How to Conduct Responsible Research: A Guide for Graduate Students

Alison l. antes.

1 Department of Medicine, Division of General Medical Sciences, Washington University School of Medicine, St. Louis, Missouri, 314-362-6006

Leonard B. Maggi, Jr.

2 Department of Medicine, Division of Molecular Oncology, Siteman Cancer Center, Washington University School of Medicine, St. Louis, Missouri, 314-362-4102

Researchers must conduct research responsibly for it to have an impact and to safeguard trust in science. Essential responsibilities of researchers include using rigorous, reproducible research methods, reporting findings in a trustworthy manner, and giving the researchers who contributed appropriate authorship credit. This “how-to” guide covers strategies and practices for doing reproducible research and being a responsible author. The article also covers how to utilize decision-making strategies when uncertain about the best way to proceed in a challenging situation. The advice focuses especially on graduate students but is appropriate for undergraduates and experienced researchers. The article begins with an overview of the responsible conduct of research, research misconduct, and ethical behavior in the scientific workplace. The takeaway message is that responsible conduct of research requires a thoughtful approach to doing research to ensure trustworthy results and conclusions and that researchers receive fair credit.

INTRODUCTION

Doing research is stimulating and fulfilling work. Scientists make discoveries to build knowledge and solve problems, and they work with other dedicated researchers. Research is a highly complex activity, so it takes years for beginning researchers to learn everything they need to know to do science well. Part of this large body of knowledge is learning how to do research responsibly. Our purpose in this article is to provide graduate students a guide for how to perform responsible research. Our advice is also relevant to undergraduate researchers and for principal investigators (PIs), postdocs, or other researchers who mentor beginning researchers and wish to share our advice.

We begin by introducing some fundamentals about the responsible conduct of research (RCR), research misconduct, and ethical behavior. We focus on how to do reproducible science and be a responsible author. We provide practical advice for these topics and present scenarios to practice thinking through challenges in research. Our article concludes with decision-making strategies for addressing complex problems.

What is the responsible conduct of research?

To be committed to RCR means upholding the highest standards of honesty, accuracy, efficiency, and objectivity ( Steneck, 2007 ). Each day, RCR requires engaging in research in a conscientious, intentional fashion that yields the best science possible ( “Research Integrity is Much More Than Misconduct,” 2019 ). We adopt a practical, “how-to” approach, discussing the behaviors and habits that yield responsible research. However, some background knowledge about RCR is helpful to frame our discussion.

The scientific community uses many terms to refer to ethical and responsible behavior in research: responsible conduct of research, research integrity, scientific integrity, and research ethics ( National Academies of Science, 2009 ; National Academies of Sciences Engineering and Medicine, 2017 ; Steneck, 2007 ). A helpful way to think about these concepts is “doing good science in a good manner” ( DuBois & Antes, 2018 ). This means that the way researchers do their work, from experimental procedures to data analysis and interpretation, research reporting, and so on, leads to trustworthy research findings and conclusions. It also includes respectful interactions among researchers both within research teams (e.g., between peers, mentors and trainees, and collaborators) and with researchers external to the team (e.g., peer reviewers). We expand on trainee-mentor relationships and interpersonal dynamics with labmates in a companion article ( Antes & Maggi, 2021 ). When research involves human or animal research subjects, RCR includes protecting the well-being of research subjects.

We do not cover all potential RCR topics but focus on what we consider fundamentals for graduate students. Common topics covered in texts and courses on RCR include the following: authorship and publication; collaboration; conflicts of interest; data management, sharing, and ownership; intellectual property; mentor and trainee responsibilities; peer review; protecting human subjects; protecting animal subjects; research misconduct; the role of researchers in society; and laboratory safety. A number of topics prominently discussed among the scientific community in recent years are also relevant to RCR. These include the reproducibility of research ( Baker, 2016 ; Barba, 2016 ; Winchester, 2018 ), diversity and inclusion in science ( Asplund & Welle, 2018 ; Hofstra et al., 2020 ; Meyers, Brown, Moneta-Koehler, & Chalkley, 2018 ; National Academies of Sciences Engineering and Medicine, 2018a ; Roper, 2019 ), harassment and bullying ( Else, 2018 ; National Academies of Sciences Engineering and Medicine, 2018b ; “ No Place for Bullies in Science,” 2018 ), healthy research work environments ( Norris, Dirnagl, Zigmond, Thompson-Peer, & Chow, 2018 ; “ Research Institutions Must Put the Health of Labs First,” 2018 ), and the mental health of graduate students ( Evans, Bira, Gastelum, Weiss, & Vanderford, 2018 ).

The National Institutes of Health (NIH) ( National Institutes of Health, 2009 ) and the National Science Foundation ( National Science Foundation, 2017 ) have formal policies indicating research trainees must receive education in RCR. Researchers are accountable to these funding agencies and the public which supports research through billions in tax dollars annually. The public stands to benefit from, or be harmed by, research. For example, the public may be harmed if medical treatments or social policies are based on untrustworthy research findings. Funding for research, participation in research, and utilization of the fruits of research all rely on public trust ( Resnik, 2011 ). Trustworthy findings are also essential for good stewardship of scarce resources ( Emanuel, Wendler, & Grady, 2000 ). Researchers are further accountable to their peers, colleagues, and scientists more broadly. Trust in the work of other researchers is essential for science to advance. Finally, researchers are accountable for complying with the rules and policies of their universities or research institutions, such as rules about laboratory safety, bullying and harassment, and the treatment of animal research subjects.

What is research misconduct?

When researchers intentionally misrepresent or manipulate their results, these cases of scientific fraud often make the news headlines ( Chappell, 2019 ; O’Connor, 2018 ; Park, 2012 ), and they can seriously undermine public trust in research. These cases also harm trust within the scientific community.

The U.S. defines research misconduct as fabrication, falsification, and plagiarism (FFP) ( Department of Health and Human Services, 2005 ). FFP violate the fundamental ethical principle of honesty. Fabrication is making up data, and falsification is manipulating or changing data or results so they are no longer truthful. Plagiarism is a form of dishonesty because it includes using someone’s words or ideas and portraying them as your own. When brought to light, misconduct involves lengthy investigations and serious consequences, such as ineligibility to receive federal research funding, loss of employment, paper retractions, and, for students, withdrawal of graduate degrees.

One aspect of responsible behavior includes addressing misconduct if you observe it. We suggest a guide titled “Responding to Research Wrongdoing: A User-Friendly Guide” that provides advice for thinking about your options if you think you have observed misconduct ( Keith-Spiegel, Sieber, & Koocher, 2010 ). Your university will have written policies and procedures for investigating allegations of misconduct. Making an allegation is very serious. As Keith-Spiegel et al.’s guide indicates, it is important to know the evidence that supports your claim, and what to expect in the process. We encourage, if possible, talking to the persons involved first. For example, one of us knew of a graduate student who reported to a journal editor their suspicion of falsified data in a manuscript. It turned out that the student was incorrect. Going above the PI directly to the editor ultimately led to the PI leaving the university, and the student had a difficult time finding a new lab to complete their degree. If the student had first spoken to the PI and lab members, they could have learned that their assumptions about the data in the paper were wrong. In turn, they could have avoided accusing the PI of a serious form of scientific misconduct—making up data—and harming everyone’s scientific career.

What shapes ethical behavior in the scientific workplace?

Responsible conduct of research and research misconduct are two sides of a continuum of behavior—RCR upholds the ideals of research and research misconduct violates them. Problematic practices that fall in the middle but are not defined formally as research misconduct have been labeled as detrimental research practices ( National Academies of Sciences Engineering and Medicine, 2017 ). Researchers conducting misleading statistical analyses or PIs providing inadequate supervision are examples of the latter. Research suggests that characteristics of individual researchers and research environments explain (un)ethical behavior in the scientific workplace ( Antes et al., 2007 ; Antes, English, Baldwin, & DuBois, 2018 ; Davis, Riske-Morris, & Diaz, 2007 ; DuBois et al., 2013 ).

These two influences on ethical behavior are helpful to keep in mind when thinking about your behavior. When people think about their ethical behavior, they think about their personal values and integrity and tend to overlook the influence of their environment. While “being a good person” and having the right intentions are essential to ethical behavior, the environment also has an influence. In addition, knowledge of standards for ethical research is important for ethical behavior, and graduate students new to research do not yet know everything they need to. They also have not fully refined their ethical decision-making skills for solving professional problems. We discuss strategies for ethical decision-making in the final section of this article ( McIntosh, Antes, & DuBois, 2020 ).

The research environment influences ethical behavior in a number of ways. For example, if a research group explicitly discusses high standards for research, people will be more likely to prioritize these ideals in their behavior ( Plemmons et al., 2020 ). A mentor who sets a good example is another important factor ( Anderson et al., 2007 ). Research labs must also provide individuals with adequate training, supervision and feedback, opportunities to discuss data, and the psychological safety to feel comfortable communicating about problems, including mistakes ( Antes, Kuykendall, & DuBois, 2019a , 2019b ). On the other hand, unfair research environments, inadequate supervision, poor communication, and severe stress and anxiety may undermine ethical decision-making and behavior; particularly when many of these factors exist together. Thus, (un)ethical behavior is a complex interplay of individual factors (e.g., personality, stress, decision-making skills) and the environment.

For graduate students, it is important to attend to what you are learning and how the environment around you might influence your behavior. You do not know what you do not know, and you necessarily rely on others to teach you responsible practices. So, it is important to be aware. Ultimately, you are accountable for your behavior. You cannot just say “I didn’t know.” Rather, just like you are curious about your scientific questions, maintain a curiosity about responsible behavior as a researcher. If you feel uncomfortable with something, pay attention to that feeling, speak to someone you trust, and seek out information about how to handle the situation. In what follows, we cover key tips for responsible behavior in the areas of reproducibility and authorship that we hope will help you as you begin.

HOW TO DO REPRODUCIBLE SCIENCE

The foremost responsibility of scientists is to ensure they conduct research in such a manner that the findings are trustworthy. Reproducibility is the ability to duplicate results ( Goodman, Fanelli, & Ioannidis, 2016 ). The scientific community has called for greater openness, transparency, and rigor as key remedies for lack of reproducibility ( Munafò et al., 2017 ). As a graduate student, essential to fostering reproducibility is the rigor of your approach to doing experiments and handling data. We discuss how to utilize research protocols, document experiments in a lab notebook, and handle data responsibly.

Utilize research protocols

1. learn and utilize the lab’s protocols.

Research protocols describe the step-by-step procedures for doing an experiment. They are critical for the quality and reproducibility of experiments. Lab members must learn and follow the lab’s protocols with the understanding that they may need to make adjustments based on the requirements of a specific experiment.

Also, it is important to distinguish between the experiment you are performing and analyzing the data from that experiment. For example, the experiment you want to perform might be to determine if loss of a gene blocks cell growth. Several protocols, each with pros and cons, will allow you to examine “cell growth.” Using the wrong experimental protocol can produce data that leads to muddled conclusions. In this example, the gene does block cell growth, but the experiment used to produce the data that you analyze to understand cell growth is wrong, thus giving a result that is a false negative.

When first joining a lab, it is essential to commit to learning the protocols necessary for your assigned research project. Researchers must ensure they are proficient in executing a protocol and can perform their experiments reliably. If you do not feel confident with a protocol, you should do practice runs if possible. Repetition is the best way to work through difficulties with protocols. Often it takes several attempts to work through the steps of a protocol before you will be comfortable performing it. Asking to watch another lab member perform the protocol is also helpful. Be sure to watch closely how steps are performed, as often there are minor steps taken that are not written down. Also, experienced lab members may do things as second nature and not think to explicitly mention them when working through the protocol. Ask questions of other lab members so that you can improve your knowledge and gain confidence with a protocol. It is better to ask a question than potentially ruin a valuable or hard-to-get sample.

Be cautious of differences in the standing protocols in the lab and how you actually perform the experiment. Even the most minor deviations can seriously impact the results and reproducibility of an experiment. As mentioned above, often there are minor things that are done that might not be listed in the protocol. Paying attention and asking questions are the best ways to learn, in addition to adding notes to the protocol if you find minor details are missing.

2. Develop your own protocols

Often you will find that a project requires a protocol that has not been performed in the lab. If performing a new experiment in the lab and no protocol exists, find a protocol and try it. Protocols can be obtained from many different sources. A great source is other labs on campus, as you can speak directly to the person who performs the experiment. There are many journal sources as well, such as Current Protocols, Nature Protocols, Nature Methods, and Cell STAR Methods . These methods journals provide the most detailed protocols for experiments often with troubleshooting tips. Scientific papers are the most common source of protocols. However, keep in mind that due to the common brevity of methods sections, they often omit crucial details or reference other papers that may not contain a complete description of the protocol.

3. Handle mistakes or problems promptly

At some point, everyone encounters problems with a protocol, or realizes they made a mistake. You should be prepared to handle this situation by being able to detail exactly how you performed the experiment. Did you skip a step? Shorten or lengthen a time point? Did you have to make a new buffer or borrow a labmate’s buffer? There are too many ways an experiment can go wrong to list here but being able to recount all the steps you performed in detail will help you work through the problem. Keep in mind that often the best way to understand how to perform an experiment is learning from when something goes wrong. This situation requires you to critically think through what was done and understand the steps taken. When everything works perfectly, it is easy to pay less attention to the details, which can lead to problems down the line.

It is up to you to be attentive and meticulous in the lab. Paying attention to the details may feel like a pain at first, or even seem overwhelming. Practice and repetition will help this focus on details become a natural part of your lab work. Ultimately, this skill will be essential to being a responsible scientist.

Document experiments in a lab notebook

1. recognize the importance of a lab notebook.

Maintaining detailed documentation in a lab notebook allows researchers to keep track of their experiments and generation of data. This detailed documentation helps you communicate about your research with others in the lab, and serves as a basis for preparing publications. It also provides a lasting record for the lab that exists beyond your time in the lab. After graduate students leave the lab, sometimes it is necessary to go back to the results of older experiments. A complete and detailed notebook is essential, or all of the time, effort, and resources are lost.

2. Learn the note-keeping practices in your lab

When you enter a new lab, it is important to understand how the lab keeps notebooks and the expectations for documentation. Being conscientious about documentation will make you a better scientist. In some labs, the PI might routinely examine your notebook, while in other labs you may be expected to maintain a notebook, but it may not be regularly viewed by others. It is tempting to become relaxed in documentation if you think your notebook may not be reviewed. Avoid this temptation; documentation of your ideas and process will improve your ability to think critically about research. Further, even if the PI or lab members do not physically view your notebook, you will need to communicate with them about your experiments. This documentation is necessary to communicate effectively about your work.

3. Organize your lab notebook

Different labs use different formats; some use electronic notebooks while others handwritten notebooks. The contents of a good notebook include the purpose of the experiment, the details of the experimental procedure, the data, and thoughts about the results. To effectively document your experiment, there are 5 critical questions that the information you record should be able to answer.

• Why I am doing this experiment? (purpose)
• What did I do to perform the experiment? (protocol)
• What are the results of what I did? (data, graphs)
• What do I think about the results?
• What do I think are the next steps?

We also recommend a table of contents. It will make the information more useful to you and the lab in the future. The table of contents should list the title of the experiment, the date(s) it was performed, and the page numbers on which it is recorded. Also, make sure that you write clearly and provide a legend or explanation of any shorthand or non-standard abbreviation you use. Often labs will have a combination of written lab notebooks and electronic data. It is important to reference where electronic data are located that go with each experiment. The idea is to make it as easy as possible to understand what you did and where to find all the data (electronic and hard copy) that accompanies your experiment.

Keeping a lab notebook becomes easier with practice. It can be thought of almost like journaling about your experiment. Sometimes people think of it as just a place to paste their protocol and a graph or data. We strongly encourage you to include your thoughts about why you made the decisions you made when conducting the experiment and to document your thoughts about next steps.

4. Commit to doing it the right way

A common reason to become lax in documentation is feeling rushed for time. Although documentation takes time, it saves time in the long-run and fosters good science. Without good notes, you will waste time trying to recall precisely what you did, reproduce your findings, and remember what you thought would be important next steps. The lab notebook helps you think about your research critically and keep your thoughts together. It can also save you time later when writing up results for publication. Further, well-documented data will help you draft a cogent and rigorous dissertation.

Handle data responsibly

1. keep all data.

Data are the product of research. Data include raw data, processed data, analyzed data, figures, and tables. Many data today are electronic, but not all. Generating data requires a lot of time and resources and researchers must treat data with care. The first essential tip is to keep all data. Do not discard data just because the experiment did not turn out as expected. A lot of experiments do not turn out to yield publishable data, but the results are still important for informing next steps.

Always keep the original, raw data. That is, as you process and analyze data, always maintain an unprocessed version of the original data.

Universities and funding agencies have data retention policies. These policies specify the number of years beyond a grant that data must be kept. Some policies also indicate researchers need to retain original data that served as the basis for a publication for a certain number of years. Therefore, your data will be important well beyond your time in graduate school. Most labs require you to keep samples for reanalysis until a paper is published, then the analyzed data are enough. If you leave a lab before a paper is accepted for publication, you are responsible for ensuring your data and original samples are well documented for others to find and use.

2. Document all data

In addition to keeping all data, data must be well-organized and documented. This means that no matter the way you keep your data (e.g., electronic or in written lab notebooks), there is a clear guide—in your lab notebook, a binder, or on a lab hard drive—to finding the data for a particular experiment. For example, it must be clear which data produced a particular graph. Version control of data is also critical. Your documentation should include “metadata” (data about your data) that tracks versions of the data. For example, as you edit data for a table, you should save separate versions of the tables, name the files sequentially, and note the changes that were made to each version.

3. Backup your data

You should backup electronic data regularly. Ideally, your lab has a shared server or cloud storage to backup data. If you are supposed to put your data there, make sure you do it! When you leave the lab, it must be possible to find your data.

4. Perform data analysis honestly and competently

Inappropriate use of statistics is a major concern in the scientific community, as the results and conclusions will be misleading if done incorrectly ( DeMets, 1999 ). Some practices are clearly an abuse of statistics, while other inappropriate practices stem from lack of knowledge. For example, a practice called “p-hacking” describes when researchers “collect or select data or statistical analyses until nonsignificant results become significant” ( Head, Holman, Lanfear, Kahn, & Jennions, 2015 ). In addition to avoiding such misbehavior, it is essential to be proficient with statistics to ensure you do statistical procedures appropriately. Learning statistical procedures and analyzing data takes many years of practice, and your statistics courses may only cover the basics. You will need to know when to consult others for help. In addition to consulting members in your lab or your PI, your university may have statistical experts who can provide consultations.

5. Master pressure to obtain favored results

When you conduct an experiment, the results are the results. As a beginning researcher, it is important to be prepared to manage the frustration of experiments not turning out as expected. It is also important to manage the real or perceived pressure to produce favored results. Investigators can become wedded to a hypothesis, and they can have a difficult time accepting the results. Sometimes you may feel this pressure coming from yourself; for example, if you want to please your PI, or if you want to get results for a certain publication. It is important to always follow the data no matter where it leads.

If you do feel pressure, this situation can be uncomfortable and stressful. If you have been meticulous and followed the above recommendations, this can be one great safeguard. You will be better able to confidently communicate your results to the PI because of your detailed documentation, and you will be more confident in your procedures if the possibility of error is suggested. Typically, with enough evidence that the unexpected results are real, the PI will concede. We recommend seeking the support of friends or colleagues to vent and cope with stress. In the rare case that the PI does not relent, you could turn to an advisor outside the lab if you need advice about how to proceed. They can help you look at the data objectively and also help you think about the interpersonal aspects of navigating this situation.

6. Communicate about your data in the lab

A critical element of reproducible research is communication in the lab. Ideally, there are weekly or bi-weekly meetings to discuss data. You need to develop your communication skills for writing and speaking about data. Often you and your labmates will discuss experimental issues and results informally during the course of daily work. This is an excellent way to hone critical thinking and communication skills about data.

Scenario 1 – The Protocol is Not Working

At the beginning of a rotation during their first year, a graduate student is handed a lab notebook and a pen and is told to keep track of their work. There does not appear to be a specific format to follow. There are standard lab protocols that everyone follows, but minor tweaks to the protocols do not seem to be tracked from experiment to experiment in the standard lab protocol nor in other lab notebooks. After two weeks of trying to follow one of the standard lab protocols, the student still cannot get the experiment to work. The student has included the appropriate positive and negative controls which are failing, making the experiment uninterpretable. After asking others in the lab for help, the graduate student learns that no one currently in the lab has performed this particular experiment. The former lab member who had performed the experiment only lists the standard protocol in their lab notebook.

How should the graduate student start to solve the problem?

Speaking to the PI would be the next logical step. As a first-year student in a lab rotation, the PI should expect this type of situation and provide additional troubleshooting guidance. It is possible that the PI may want to see how the new graduate student thinks critically and handles adversity in the lab. Rather than giving an answer, the PI might ask the student to work through the problem. The PI should give guidance, but it may not be an immediate fix for the problem. If the PI’s suggestions fail to correct the problem, asking a labmate or the PI for the contact information of the former lab member who most recently performed the experiment would be a reasonable next step. The graduate student’s conversations with the PI and labmates in this situation will help them learn a lot about how the people in the lab interact.

Most of the answers for these types of problems will require you as a graduate student to take the initiative to answer. They will require your effort and ingenuity to talk to other lab members, other labs at the university, and even scour the literature for alternatives. While labs have standard protocols, there are multiple ways to do many experiments, and working out an alternative will teach you more than when everything works. Having to troubleshoot problems will result in better standard protocols in the lab and better science.

HOW TO BE A RESPONSIBLE AUTHOR

Researchers communicate their findings via peer-reviewed publications, and publications are important for advancing in a research career. Many graduate students will first author or co-author publications in graduate school. For good advice on how to write a research manuscript, consult the Current Protocols article “How to write a research manuscript” ( Frank, 2018 ). We focus on the issues of assigning authors and reporting your findings responsibly. First, we describe some important basics: journal impact factors, predatory journals, and peer review.

What are journal impact factors?

It is helpful to understand journal impact factors. There is criticism about an overemphasis on impact factors for evaluating the quality or importance of researchers’ work ( DePellegrin & Johnston, 2015 ), but they remain common for this purpose. Journal impact factors reflect the average number of times articles in a journal were cited in the last two years. Higher impact factors place journals at a higher rank. Approximately 2% of journals have an impact factor of 10 or higher. For example, Cell, Science, and Nature have impact factors of approximately 39, 42, and 43, respectively. Journals can be great journals but have lower impact factors; often this is because they focus on a smaller specialty field. For example, Journal of Immunology and Oncogene are respected journals, but their impact factors are about 4 and 7, respectively.

Research trainees often want to publish in journals with the highest possible impact factor because they expect this to be viewed favorably when applying to future positions. We encourage you to bear in mind that many different journals publish excellent science and focus on publishing where your work will reach the desired audience. Also, keep in mind that while a high impact factor can direct you to respectable, high-impact science, it does not guarantee that the science in the paper is good or even correct. You must critically evaluate all papers you read no matter the impact factor.

What are predatory journals?

Predatory journals have flourished over the past few years as publishing science has moved online. An international panel defined predatory journals as follows ( Grudniewicz et al., 2019 ):

Predatory journals and publishers are entities that prioritize self-interest at the expense of scholarship and are characterized by false or misleading information, deviation from best editorial and publication practices, a lack of transparency, and/or the use of aggressive and indiscriminate solicitation practices. (p. 211)

Often young researchers receive emails soliciting them to submit their work to a journal. There are typically small fees (around $99 US) requested but these fees will be much lower than open access fees of reputable journals (often around $2000 US). A warning sign of a predatory journal is outlandish promises, such as 24-hour peer review or immediate publication. You can find a list of predatory journals created by a postdoc in Europe at BeallsList.net ( “Beall’s List of Potential Predatory Journals and Publishers,” 2020 ).

What is peer review?

Peer reviewers are other scientists who have the expertise to evaluate a manuscript. Typically 2 or 3 reviewers evaluate a manuscript. First, an editor performs an initial screen of the manuscript to ensure its appropriateness for the journal and that it meets basic quality standards. At this stage, an editor can decide to reject the manuscript and not send it to review. Not sending a paper for peer review is common in the highest impact journals that receive more submissions per year than can be reviewed and published. For average-impact journals and specialty journals, typically your paper will be sent for peer review.

In general, peer review focuses on three aspects of a manuscript: research design and methods, validity of the data and conclusions, and significance. Peer reviewers assess the merit and rigor of the research design and methodology, and they evaluate the overall validity of the results, interpretations, and conclusions. Essentially, reviewers want to ensure that the data support the claims. Additionally, reviewers evaluate the overall significance, or contribution, of the findings, which involves the novelty of the research and the likelihood that the findings will advance the field. Significance standards vary between journals. Some journals are open to publishing findings that are incremental advancements in a field, while others want to publish only what they deem as major advancements. This feature can distinguish the highest impact journals which seek the most significant advancements and other journals that tend to consider a broader range of work as long as it is scientifically sound. It is important to keep in mind that determining at the stage of review and publication whether a paper is “high impact” is quite subjective. In reality, this can only really be determined in retrospect.

The key ethical issues in peer review are fairness, objectivity, and confidentiality ( Shamoo & Resnik, 2015 ). Peer reviewers are to evaluate the manuscript on its merits and not based on biases related to the authors or the science itself. If reviewers have a conflict of interest, this should be disclosed to the editor. Confidentiality of peer review means that the reviewers should keep private the information; they should not share the information with others or use it to their benefit. Reviewers can ultimately recommend that the manuscript is rejected, revised, and resubmitted (major or minor revisions), or accepted. The editor evaluates the reviewers’ feedback and makes a judgment about rejecting, accepting, or requesting a revision. Sometimes PIs will ask experienced graduate students to assist with peer reviewing a manuscript. This is a good learning opportunity. The PI should disclose to the editor that they included a trainee in preparing the review.

Assign authorship fairly

Authorship gives credit to the people who contributed to the research. This includes thinking of the ideas, designing and performing experiments, interpreting the results, and writing the paper. Two key questions regarding authorship include: 1 - Who will be an author? 2 - What will be the order in which authors are listed? These seem simple on the surface but can get quite complex.

1. Know authorship guidelines

Authorship guidelines published by journals, professional societies, and universities communicate key principles of authorship and standards for earning authorship. The core ethical principle of assigning authorship is fairness in who receives credit for the work. The people who contributed to the work should get credit for it. This seems simply enough, but determining authorship can (and often does) create conflict.

Many universities have authorship guidelines, and you should know the policies at your university. The International Committee of Medical Journal Editors (ICMJE) provides four criteria for determining who should be an author ( International Committee of Medical Journal Editors, 2020 ). These criteria indicate that an author should do all of the following: 1) make “substantial contributions” to the development of the idea or research design, or to acquiring, analyzing, or interpreting the data, 2) write the manuscript or revise it a substantive way, 3) give approval of the final manuscript (i.e., before it is submitted for review, and after it is revised, if necessary), and 4) agree to be responsible for any questions about the accuracy or integrity of the research.

Several types of authorship violate these guidelines and should be avoided. Guest authorship is when respected researchers are added out of appreciation, or to have the manuscript be perceived more favorably to get it published or increase its impact. Gift authorship is giving authorship to reward an individual, or as a favor. Ghost authorship is when someone made significant contributions to the paper but is not listed as an author. To increase transparency, some journals require authors to indicate how each individual contributed to the research and manuscript.

2. Apply the guidelines

Conflicts often arise from disagreements about how much people contributed to the research and whether those contributions merit authorship. The best approach is an open, honest, and ongoing discussion about authorship, which we discuss in #3 below. To have effective, informed conversations about authorship, you must understand how to apply the guidelines to your specific situation. The following is a simple rule of thumb that indicates there are three components of authorship. We do not list giving final approval of the manuscript and agreeing to be accountable, but we do consider these essentials of authorship.

• Thinking – this means contributing to the ideas leading to the hypothesis of the work, designing experiments to address the hypothesis, and/or analyzing the results in the larger context of the literature in the field.
• Doing – this means performing and analyzing the experiments.
• Writing – this means editing a draft, or writing the entire paper. The first author often writes the entire first draft.

In our experience, a first author would typically do all three. They also usually coordinate the writing and editing process. Co-authors are typically very involved in at least two of the three, and are somewhat involved in the other. The PI, who oversees and contributes to all three, is often the last, or “senior author.” The “senior author” is typically the “corresponding author”—the person listed as the individual to contact about the paper. The other co-authors are listed between the first and senior author either alphabetically, or more commonly, in order from the largest to smallest contribution.

Problems in assigning authorship typically arise due to people’s interpretations of #1 (thinking) and #2 (doing)—what and how much each individual contributed to a project’s design, execution, and analysis. Different fields or PIs may have their own slight variations on these guidelines. The potential conflicts associated with assigning authorship lead to the most common recommendation for responsibly assigning authorship: discuss authorship expectations early and revisit them during the project.

3. Discuss authorship with your collaborators

Publications are important for career advancement, so you can see why people might be worried about fairness in assigning authorship. If the problem arises from a lack of a shared understanding about contributions to the research, the only way to clarify this is an open discussion. This discussion should ideally take place very early at the beginning of a project, and should be ongoing. Hopefully you work in a laboratory that makes these discussions a natural part of the research process; this makes it much easier to understand the expectations upfront.

We encourage you to speak up about your interest in making a contribution that would merit authorship, especially if you want to earn first authorship. Sometimes norms about authoring papers in a lab make it clear you are expected to first and co-author publications, but it is best to communicate your interest in earning authorship. If the project is not yours, but you wish to collaborate, you can inquire what you may be able to contribute that would merit authorship.

If it is not a norm in your lab to discuss authorship throughout the life of projects, then as a graduate student you may feel reluctant to speak up. You could initiate a conversation with a more senior graduate student, a postdoc, or your PI, depending on the dynamics in the group. You could ask generally about how the lab approaches assignment of authorship, but discussing a specific project and paper may be best. It may feel awkward to ask, but asking early is less uncomfortable than waiting until the end of the project. If the group is already drafting a manuscript and you are told that your contribution is insufficient for authorship, this situation is much more discouraging than if you had asked earlier about what is expected to earn authorship.

How to report findings responsibly

The most significant responsibility of authors is to present their research accurately and honestly. Deliberately presenting misleading information is clearly unethical, but there are significant judgment calls about how to present your research findings. For example, an author can mislead by overstating the conclusions given what the data support.

1. Commit to presenting your findings honestly

Any good scientific manuscript writer will tell you that you need to “tell a good story.” This means that your paper is organized and framed to draw the reader into the research and convince them of the importance of the findings. But, this story must be sound and justified by the data. Other authors are presenting their findings in the best, most “publishable” light, so it is a balancing act to be persuasive but also responsible in presenting your findings in a trustworthy manner. To present your findings honestly, you must be conscious of how you interpret your data and present your conclusions so that they are accurate and not overstated.

One misbehavior known as “HARKing,” Hypothesis After the Results are Known, occurs when hypotheses are created after seeing the results of an experiment, but they are presented as if they were defined prior to collecting the data ( Munafò et al., 2017 ). This practice should be avoided. HARKing may be driven, in part, by a concern in scientific publishing known as publication bias. This bias is a preference that reviewers, editors, and researchers have for papers describing positive findings instead of negative findings ( Carroll, Toumpakari, Johnson, & Betts, 2017 ). This preference can lead to manipulating one’s practices, such as by HARKing, so that positive findings can be reported.

It is important to note that in addition to avoiding misbehaviors such as HARKing, all researchers are susceptible to a number of more subtle traps in judgment. Even the most well-intentioned researcher may jump to conclusions, discount alternative explanations, or accept results that seem correct without further scrutiny ( Nuzzo, 2015 ). Therefore, researchers must not only commit to presenting their findings honestly but consider how they can counteract such traps by slowing down and increasing their skepticism towards their findings.

2. Provide an appropriate amount of detail

Providing enough detail in a manuscript can be a challenge with the word limits imposed by most journals. Therefore, you will need to determine what details to include and which to exclude, or potentially include in the supplemental materials. Methods sections can be long and are often the first to be shortened, but complete methods are important for others to evaluate the research and to repeat the methods in other studies. Even more significant is making decisions about what experimental data to include and potentially exclude from the manuscript. Researchers must determine what data is required to create a complete scientific story that supports the central hypothesis of the paper. On the other hand, it is not necessary or helpful to include so much data in the manuscript, or in supplemental material, that the central point of the paper is difficult to discern. It is a tricky balance.

3. Follow proper citation practices

Of course, responsible authorship requires avoiding plagiarism. Many researchers think that plagiarism is not a concern for them because they assume it is always done intentionally by “copying and pasting” someone else’s words and claiming them as your own. Sometimes poor writing practices, such as taking notes from references without distinguishing between direct quotes and paraphrased material, can lead to including material that is not quoted properly. More broadly, proper citation practices include accurately and completely referencing prior studies to provide appropriate context for your manuscript.

4. Attend to the other important details

The journal will require several pieces of additional information, such as disclosure of sources of funding and potential conflicts of interest. Typically, graduate students do not have relationships that constitute conflicts of interest, but a PI who is a co-author may. In submitting a manuscript, also make sure to acknowledge individuals not listed as authors but who contributed to the work.

5. Share data and promote transparency

Data sharing is a key facet of promoting transparency in science ( Nosek et al., 2015 ). It will be important to know the expectations of the journals in which you wish to publish. Many top journals now require data sharing; for example, sharing your data files in an online repository so others have access to the data for secondary use. Funding agencies like NIH also increasingly require data sharing. To further foster transparency and public trust in research, researchers must deposit their final peer-reviewed manuscripts that report on research funded by NIH to PubMed Central. PubMed makes biomedical and life science research publicly accessible in a free, online database.

Scenario 2 – Authors In Conflict

To prepare a manuscript for publication, a postdoc’s data is added to a graduate student’s thesis project. After working together to combine the data and write the paper, the postdoc requests co-first authorship on the paper. The graduate student balks at this request on the basis that it is their thesis project. In a weekly meeting with the lab’s PI to discuss the status of the paper, the graduate student states that they should divide the data between the authors as a way to prove that the graduate student should be the sole first author. The PI agrees to this attempt to quantify how much data each person contributed to the manuscript. All parties agree the writing and thinking were equally shared between them. After this assessment, the graduate student sees that the postdoc actually contributed more than half of the data presented in the paper. The graduate student and a second graduate student contributed the remaining data; this means the graduate student contributed much less than half of the data in the paper. However, the graduate student is still adamant that they must be the sole first author of the paper because it is their thesis project.

Is the graduate student correct in insisting that it is their project, so they are entitled to be the sole first author?

Co-first authorship became popular about 10 years ago as a way to acknowledge shared contributions to a paper in which authors worked together and contributed equally. If the postdoc contributed half of the data and worked with the graduate student to combine their interpretations and write the first draft of the paper, then the postdoc did make a substantial contribution. If the graduate student wrote much of the first draft of the paper, contributed significantly to the second half of data, and played a major role in the thesis concept and design, this is also a major contribution. We summarized authorship requirements as contributing to thinking, doing, and writing, and we noted that a first author usually contributes to all of these. The graduate student has met all 3 elements to claim first authorship. However, it appears that the postdoc has also met these 3 requirements. Thus, it is at least reasonable for the postdoc to ask about co-first authorship.

The best way to move forward is to discuss their perspectives openly. Both the graduate student and postdoc want first authorship on papers to advance their careers. The postdoc feels they contributed more to the overall concept and design than the graduate student is recognizing, and the postdoc did contribute half of the data. This is likely frustrating and upsetting for the postdoc. On the other hand, perhaps the postdoc is forgetting how much a thesis becomes like “your baby,” so to speak. The work is the graduate student’s thesis, so it is easy to see why the graduate student would feel a sense of ownership of it. Given this fact, it may be hard for the graduate student to accept the idea that they would share first-author recognition for the work. Yet, the graduate student should consider that the manuscript would not be possible without the postdoc’s contribution. Further, if the postdoc was truly being unreasonable, then the postdoc could make the case for sole first authorship based on contributing the most data to the paper, in addition to contributing ideas and writing the paper. The graduate student should consider that the postdoc may be suggesting co-first authorship in good faith.

As with any interpersonal conflict, clear communication is key. While it might be temporarily uncomfortable to voice their views and address this disagreement, it is critical to avoiding permanent damage to their working relationship. The pair should consider each other’s perspectives and potential alternatives. For example, if the graduate student is first author and the postdoc second, at a minimum they could include an author note in the manuscript that describes the contribution of each author. This would make it clear the scope of the postdoc’s contribution, if they decided not to go with co-first authorship. Also, the graduate student should consider their assumptions about co-first authorship. Maybe they assume it makes it appear they contributed less, but instead, perhaps co-first authorship highlights their collaborative approach to science. Collaboration is a desirable quality many (although arguably not all) research organizations look for when they are hiring.

They will also need to speak with others for advice. The pair should definitely speak with the PI who could provide input about how these cases have been handled in the past. Ultimately, if they cannot reach an agreement, the PI, who is likely to be the last or “senior” author, may make the final decision. They should also speak to the other graduate student who is an author.

If either individual is upset with the situation, they will want to discuss it when they have had time to cool down. This might mean taking a day before discussing, or speaking with someone outside of the lab for support. Ideally, all authors on this paper would have initiated this conversation earlier, and the standards in the lab for first authorship would be discussed routinely. Clear communication may have avoided the conflict.

HOW TO USE DECISION-MAKING STRATEGIES TO NAVIGATE CHALLENGES

We have provided advice on some specific challenges you might encounter in research. This final section covers our overarching recommendation that you adopt a set of ethical decision-making strategies. These strategies help researchers address challenges by helping them think through a problem and possible alternatives ( McIntosh et al., 2020 ). The strategies encourage you to gather information, examine possible outcomes, consider your assumptions, and address emotional reactions before acting. They are especially helpful when you are uncertain how to proceed, face a new problem, or when the consequences of a decision could negatively impact you or others. The strategies also help people be honest with themselves, such as when they are discounting important factors or have competing goals, by encouraging them to identify outside perspectives and test their motivations. You can remember the strategies using the acronym SMART .

1. S eek Help

Obtain input from others who can be objective and that you trust. They can assist you with assessing the situation, predicting possible outcomes, and identifying potential options. They can also provide you with support. Individuals to consult may be peers, other faculty, or people in your personal life. It is important that you trust the people you talk with, but it is also good when they challenge your perspective, or encourage you to think in a new way about a problem. Keep in mind that people such as program directors and university ombudsmen are often available for confidential, objective advice.

2. M anage Emotions

Consider your emotional reaction to the situation and how it might influence your assessment of the situation, and your potential decisions and actions. In particular, identify negative emotions, like frustration, anxiety, fear, and anger, as they particularly tend to diminish decision-making and the quality of interactions with others. Take time to address these emotions before acting, for example, by exercising, listening to music, or simply taking a day before responding.

3. A nticipate Consequences

Think about how the situation could turn out. This includes for you, for the research team, and anyone else involved. Consider the short, middle-term, and longer-term impacts of the problem and your potential approach to addressing the situation. Ideally, it is possible to identify win-win outcomes. Often, however, in tough professional situations, you may need to select the best option from among several that are not ideal.

4. R ecognize Rules and Context

Determine if any ethical principles, professional policies, or rules apply that might help guide your choices. For instance, if the problem involves an authorship dispute, consider the authorship guidelines that apply. Recognizing the context means considering the situational factors that could impact your options and how you proceed. For example, factors such as the reality that ultimately the PI may have the final decision about authorship.

5. T est Assumptions and Motives

Examine your beliefs about the situation and whether any of your thoughts may not be justified. This includes critically examining the personal motivations and goals that are driving your interpretation of the problem and thoughts about how to resolve it.

These strategies do not have to be engaged in order, and they are interrelated. For example, seeking help can help you manage emotions, test assumptions, and anticipate consequences. Go back to the scenarios and our advice throughout this article, and you will see many of our suggestions align with these strategies. Practice applying SMART strategies when you encounter a problem and they will become more natural.

Learning practices for responsible research will be the foundation for your success in graduate school and your career. We encourage you to be reflective and intentional as you learn and hope that our advice helps you along the way.

ACKNOWLEDGEMENTS

This work was supported by the National Human Genome Research Institute (Antes, K01HG008990) and the National Center for Advancing Translational Sciences (UL1 TR002345).

LITERATURE CITED

• Anderson MS, Horn AS, Risbey KR, Ronning EA, De Vries R, & Martinson BC (2007). What Do Mentoring and Training in the Responsible Conduct of Research Have To Do with Scientists’ Misbehavior? Findings from a National Survey of NIH-Funded Scientists . Academic Medicine , 82 ( 9 ), 853–860. doi: 10.1097/ACM.0b013e31812f764c [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Antes AL, Brown RP, Murphy ST, Waples EP, Mumford MD, Connelly S, & Devenport LD (2007). Personality and Ethical Decision-Making in Research: The Role of Perceptions of Self and Others . Journal of Empirical Research on Human Research Ethics , 2 ( 4 ), 15–34. doi: 10.1525/jer.2007.2.4.15 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Antes AL, English T, Baldwin KA, & DuBois JM (2018). The Role of Culture and Acculturation in Researchers’ Perceptions of Rules in Science . Science and Engineering Ethics , 24 ( 2 ), 361–391. doi: 10.1007/s11948-017-9876-4 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Antes AL, Kuykendall A, & DuBois JM (2019a). The Lab Management Practices of “Research Exemplars” that Foster Research Rigor and Regulatory Compliance: A Qualitative Study of Successful Principal Investigators . PloS One , 14 ( 4 ), e0214595. doi: 10.1371/journal.pone.0214595 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Antes AL, Kuykendall A, & DuBois JM (2019b). Leading for Research Excellence and Integrity: A Qualitative Investigation of the Relationship-Building Practices of Exemplary Principal Investigators . Accountability in Research , 26 ( 3 ), 198–226. doi: 10.1080/08989621.2019.1611429 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Antes AL, & Maggi LB Jr. (2021). How to Navigate Trainee-Mentor Relationships and Interpersonal Dynamics in the Lab . Current Protocols Essential Laboratory Techniques. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Asplund M, & Welle CG (2018). Advancing Science: How Bias Holds Us Back . Neuron , 99 ( 4 ), 635–639. doi: 10.1016/j.neuron.2018.07.045 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Baker M (2016). Is There a Reproducibility Crisis? Nature , 533 , 452–454. doi: 10.1038/533452a [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Barba LA (2016). The Hard Road to Reproducibility . Science , 354 ( 6308 ), 142. doi: 10.1126/science.354.6308.142 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Beall’s List of Potential Predatory Journals and Publishers . (2020). Retrieved from https://beallslist.net/#update [ Google Scholar ]
• Carroll HA, Toumpakari Z, Johnson L, & Betts JA (2017). The Perceived Feasibility of Methods to Reduce Publication Bias . PloS One , 12 ( 10 ), e0186472–e0186472. doi: 10.1371/journal.pone.0186472 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Chappell B (2019). Duke Whistleblower Gets More Than $33 Million in Research Fraud Settlement . NPR. Retrieved from https://www.npr.org/2019/03/25/706604033/duke-whistleblower-gets-more-than-33-million-in-research-fraud-settlement [ Google Scholar ]
• Davis MS, Riske-Morris M, & Diaz SR (2007). Causal Factors Implicated in Research Misconduct: Evidence from ORI Case Files . Science and Engineering Ethics , 13 ( 4 ), 395–414. doi: 10.1007/s11948-007-9045-2 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• DeMets DL (1999). Statistics and Ethics in Medical Research . Science and Engineering Ethics , 5 ( 1 ), 97–117. doi: 10.1007/s11948-999-0059-9 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Department of Health and Human Services. (2005). 42 CFR Parts 50 and 93 Public Health Service Policies on Research Misconduct; Final Rule. Retrieved from https://ori.hhs.gov/sites/default/files/42_cfr_parts_50_and_93_2005.pdf [ Google Scholar ]
• DePellegrin TA, & Johnston M (2015). An Arbitrary Line in the Sand: Rising Scientists Confront the Impact Factor . Genetics , 201 ( 3 ), 811–813. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• DuBois JM, Anderson EE, Chibnall J, Carroll K, Gibb T, Ogbuka C, & Rubbelke T (2013). Understanding Research Misconduct: A Comparative Analysis of 120 Cases of Professional Wrongdoing . Account Res , 20 ( 5–6 ), 320–338. doi: 10.1080/08989621.2013.822248 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• DuBois JM, & Antes AL (2018). Five Dimensions of Research Ethics: A Stakeholder Framework for Creating a Climate of Research Integrity . Academic Medicine , 93 ( 4 ), 550–555. doi: 10.1097/ACM.0000000000001966 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Else H (2018). Does Science have a Bullying Problem? Nature , 563 , 616–618. doi: 10.1038/d41586-018-07532-5 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Emanuel EJ, Wendler D, & Grady C (2000). What Makes Clinical Research Ethical ? Journal of the American Medical Association , 283 ( 20 ), 2701–2711. doi:jsc90374 [pii] [ PubMed ] [ Google Scholar ]
• Evans TM, Bira L, Gastelum JB, Weiss LT, & Vanderford NL (2018). Evidence for a Mental Health Crisis in Graduate Education . Nature Biotechnology , 36 ( 3 ), 282–284. doi: 10.1038/nbt.4089 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Frank DJ (2018). How to Write a Research Manuscript . Current Protocols Essential Laboratory Techniques , 16 ( 1 ), e20. doi: 10.1002/cpet.20 [ CrossRef ] [ Google Scholar ]
• Goodman SN, Fanelli D, & Ioannidis JPA (2016). What Does Research Reproducibility Mean? Science Translational Medicine , 8 ( 341 ), 341ps312. doi: 10.1126/scitranslmed.aaf5027 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Grudniewicz A, Moher D, Cobey KD, Bryson GL, Cukier S, Allen K, … Lalu MM (2019). Predatory journals: no definition, no defence . Nature , 576 ( 7786 ), 210–212. doi: 10.1038/d41586-019-03759-y [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Head ML, Holman L, Lanfear R, Kahn AT, & Jennions MD (2015). The Extent and Consequences of P-Hacking in Science . PLoS Biology , 13 ( 3 ), e1002106. doi: 10.1371/journal.pbio.1002106 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hofstra B, Kulkarni VV, Munoz-Najar Galvez S, He B, Jurafsky D, & McFarland DA (2020). The Diversity–Innovation Paradox in Science . Proceedings of the National Academy of Sciences , 117 ( 17 ), 9284. doi: 10.1073/pnas.1915378117 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• International Committee of Medical Journal Editors. (2020). Defining the Role of Authors and Contributors . Retrieved from http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authors-and-contributors.html
• Keith-Spiegel P, Sieber J, & Koocher GP (2010). Responding to Research Wrongdoing: A User-Friendly Guide . Retrieved from http://users.neo.registeredsite.com/1/4/0/20883041/assets/RRW_11-10.pdf
• McIntosh T, Antes AL, & DuBois JM (2020). Navigating Complex, Ethical Problems in Professional Life: A Guide to Teaching SMART Strategies for Decision-Making . Journal of Academic Ethics . doi: 10.1007/s10805-020-09369-y [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Meyers LC, Brown AM, Moneta-Koehler L, & Chalkley R (2018). Survey of Checkpoints along the Pathway to Diverse Biomedical Research Faculty . PloS One , 13 ( 1 ), e0190606–e0190606. doi: 10.1371/journal.pone.0190606 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Munafò MR, Nosek BA, Bishop DVM, Button KS, Chambers CD, Percie du Sert N, … Ioannidis JPA (2017). A manifesto for reproducible science . Nature Human Behaviour , 1 ( 1 ), 0021. doi: 10.1038/s41562-016-0021 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• National Academies of Science. (2009). On Being a Scientist: A Guide to Responsible Conduct in Research . Washington DC: National Academics Press. [ Google Scholar ]
• National Academies of Sciences Engineering and Medicine. (2017). Fostering Integrity in Research . Washington, DC: The National Academies Press [ Google Scholar ]
• National Academies of Sciences Engineering and Medicine. (2018a). An American Crisis: The Growing Absence of Black Men in Medicine and Science: Proceedings of a Joint Workshop . Washington, DC: The National Academies Press. [ Google Scholar ]
• National Academies of Sciences Engineering and Medicine. (2018b). Sexual harassment of women: climate, culture, and consequences in academic sciences, engineering, and medicine : National Academies Press. [ Google Scholar ]
• National Institutes of Health. (2009). Update on the Requirement for Instruction in the Responsible Conduct of Research . NOT-OD-10-019 . Retrieved from https://grants.nih.gov/grants/guide/notice-files/NOT-OD-10-019.html
• National Science Foundation. (2017). Important Notice No. 140 Training in Responsible Conduct of Research – A Reminder of the NSF Requirement . Retrieved from https://www.nsf.gov/pubs/issuances/in140.jsp
• No Place for Bullies in Science. (2018). Nature , 559 ( 7713 ), 151. doi: 10.1038/d41586-018-05683-z [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Norris D, Dirnagl U, Zigmond MJ, Thompson-Peer K, & Chow TT (2018). Health Tips for Research Groups . Nature , 557 , 302–304. doi: 10.1038/d41586-018-05146-5 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Nosek BA, Alter G, Banks GC, Borsboom D, Bowman SD, Breckler SJ, … Yarkoni T (2015). Scientific Standards . Promoting an Open Research Culture. Science , 348 ( 6242 ), 1422–1425. doi: 10.1126/science.aab2374 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Nuzzo R (2015). How Scientists Fool Themselves - and How They Can Stop . Nature , 526 , 182–185. [ PubMed ] [ Google Scholar ]
• O’Connor A (2018). More Evidence that Nutrition Studies Don’t Always Add Up . The New York Times. Retrieved from https://www.nytimes.com/2018/09/29/sunday-review/cornell-food-scientist-wansink-misconduct.html [ Google Scholar ]
• Park A (2012). Great Science Frauds . Time. Retrieved from https://healthland.time.com/2012/01/13/great-science-frauds/slide/the-baltimore-case/ [ Google Scholar ]
• Plemmons DK, Baranski EN, Harp K, Lo DD, Soderberg CK, Errington TM, … Esterling KM (2020). A Randomized Trial of a Lab-embedded Discourse Intervention to Improve Research Ethics . Proceedings of the National Academy of Sciences , 117 ( 3 ), 1389. doi: 10.1073/pnas.1917848117 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Research Institutions Must Put the Health of Labs First. (2018). Nature , 557 ( 7705 ), 279–280. doi: 10.1038/d41586-018-05159-0 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Research Integrity is Much More Than Misconduct . (2019). ( 570 ). doi: 10.1038/d41586-019-01727-0 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Resnik DB (2011). Scientific Research and the Public Trust . Science and Engineering Ethics , 17 ( 3 ), 399–409. doi: 10.1007/s11948-010-9210-x [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Roper RL (2019). Does Gender Bias Still Affect Women in Science? Microbiology and Molecular Biology Reviews , 83 ( 3 ), e00018–00019. doi: 10.1128/MMBR.00018-19 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Shamoo AE, & Resnik DB (2015). Responsible Conduct of Research (3rd ed.). New York: Oxford University Press. [ Google Scholar ]
• Steneck NH (2007). ORI Introduction to the Responsible Conduct of Research (Updated ed.). Washington, D.C.: U.S. Government Printing Office. [ Google Scholar ]
• Winchester C (2018). Give Every Paper a Read for Reproducibility . Nature , 557 , 281. doi: 10.1038/d41586-018-05140-x [ PubMed ] [ CrossRef ] [ Google Scholar ]

How to Do Research: A Step-By-Step Guide: Get Started

• Get Started
• 1a. Select a Topic
• 1b. Develop Research Questions
• 1c. Identify Keywords
• 1d. Find Background Information
• 1e. Refine a Topic
• 2a. Search Strategies
• 2d. Articles
• 2e. Videos & Images
• 2f. Databases
• 2g. Websites
• 2h. Grey Literature
• 2i. Open Access Materials
• 3a. Evaluate Sources
• 3b. Primary vs. Secondary
• 3c. Types of Periodicals
• 4a. Take Notes
• 4b. Outline the Paper
• 4c. Incorporate Source Material
• 5a. Avoid Plagiarism
• 5b. Zotero & MyBib
• 5c. MLA Formatting
• 5d. MLA Citation Examples
• 5e. APA Formatting
• 5f. APA Citation Examples
• 5g. Annotated Bibliographies

Research Essentials Video Tutorials

Related guides.

• Elmira College Writing Center Get one-on-one assistance for all types of writing.

Recommended Websites

• Purdue University's Online Writing Lab (OWL)

Research Process Overview

Step 1.  Develop a topic Select a Topic | Develop Research Questions | Identify Keywords | Find Background Information | Refine a Topic

Step 2. Locate information Search Strategies | Books | eBooks | Articles  | Videos & Images | Databases | Websites | Grey Literature

Step 3. Evaluate and analyze information Evaluate Sources | Primary vs Secondary | Types of Periodicals

Step 4. Write, organize, and communicate information Take Notes | Outline the Paper | Incorporate Source Material

Step 5. Cite sources Avoid Plagiarism | Zotero & MyBib | MLA | APA | Chicago Style | Annotated Bibliographies

For research help,  use one of the following options:

Ask the GTL

• Next: Step 1: Develop a Topic >>
• Last Updated: Feb 21, 2024 11:01 AM
• URL: https://libguides.elmira.edu/research

Michael Nielsen

Principles of Effective Research

Note: The entire essay is here in postscript format .

By Michael A. Nielsen

This essay is intended as a letter to both myself and others, to hold up in the sharpest possible terms an ideal of research I believe is worth working toward. I’ve deliberately limited the essay to 10 pages, hoping that the resulting omissions are compensated by the forced brevity. This is a rather personal essay; it’s not the sort of thing I’d usually make publicly available. I’ve made the essay public in order to heighten my commitment to the project, and in the hope that other people will find it stimulating, and perhaps offer some thoughts of their own.

A few words of warning. My primary audience is myself, and some of the advice is specific to my career situation [*], and therefore may not be directly applicable to others. And, of course, it’s all just my opinion anyway. I hope, however, that it’ll still be stimulating and helpful.

[*] I’m a theoretical physicist; I lead a small research group at a large Australian University; I have a permanent position, with no teaching duties for the next few years; I have several colleagues on the faculty with closely related interests.

The philosophy underlying the essay is based on a famous quote attributed to Aristotle: “We are what we repeatedly do. Excellence, then, is not an act but a habit.” Underlying all our habits are models (often unconscious) of how the world works. I’m writing this essay to develop an improved personal model of how to be an effective researcher, a model that can be used as the basis for concrete actions leading to the development of new habits.

Fundamental principles

The fundamental principles of effective research are extremely similar to those for effectiveness in any other part of life. Although the principles are common sense, that doesn’t mean they’re common practice, nor does it mean that they’re easy to internalize. Personally, I find it a constant battle to act in accord with these principles, a battle requiring ongoing reflection, rediscovery and renewed commitment.

Integrating research into the rest of your life

Research is, of course, only a part of life, and must be understood in relation to the rest of life. The foundation of effective research is a strong motivation or desire to do research. If research is not incredibly exciting, rewarding and enjoyable, at least some of the time, then why not do something else that is? For the purposes of this essay, I’ll assume that you already have a strong desire to do research [*].

[*] People sometimes act or talk as though desire and motivation cannot be changed. Within limits, I think that’s wrong, and we can mold our own motivations. But that’s a subject for another essay.

Motivation and desire alone are not enough. You also need to have the rest of your life in order to be an effective researcher. Make sure you’re fit. Look after your health. Spend high quality time with your family. Have fun. These things require a lot of thought and effort to get right. If you don’t get them right, not only will your life as a whole be less good, your research will suffer. So get these things right, and make sure they’re integrated with your research life.

As an example, I once spent three years co-authoring a technical book, and for the final eighteen months I concentrated on the book almost exclusively, to the neglect of my health, relationships, and other research. It is tempting to ask the question “Was the neglect worth the benefits?” But that is the wrong question, for while the neglect paid short-term dividends in increased productivity, over the total period of writing the book I believe it probably cost me productivity, and it certainly did after the book was complete. So not only did I become less fit and healthy, and see my relationships suffer, the book took longer to complete than if I’d had my life in better order.

Principles of personal behaviour: proactivity, vision, and discipline

I believe that the foundation of effective research is to internalize a strong vision of what you want to achieve, to work proactively towards that vision, taking personal responsibility for successes and failures. You need to develop disciplined work habits, and to achieve balance between self-development and the actual creative research process .

Proactivity and personal responsibility

Effective people are proactive and take personal responsibility for the events in their lives. They form a vision of how they want their life to be, and work toward achieving that vision. They identify problems in their lives, and work toward solutions to those problems.

Isn’t this obvious, banal advice? I heard a story years ago in which a representative from McDonald’s was asked what gave McDonald’s the edge in the fast food industry. They replied that McDonald’s took care of the little things, like making sure that their restaurants and surrounds were always extremely clean. Representatives of other fast food companies replied incredulously that surely that was not the reason McDonald’s did so well, for “anyone could do that”. “But only McDonald’s does” was the response. The heart of personal effectiveness is not necessarily any special knowledge or secret: it is doing the basics consistently well.

When it comes to proactivity and responsibility, it seems to be incredibly difficult to internalize these principles and act on them consistently. Almost everyone says and thinks they are proactive and responsible, but how many of us truly respond to the force of external circumstance in the most proactive manner?

My belief is that the reason it is difficult to be consistently proactive and responsible is that over the short term it is often significantly easier to abdicate responsibility and behave in a reactive fashion. In my opinion, there are three basic ways this can occur.

The first way is to blame external circumstances for our problems. “We don’t have enough grant money.” “I have to teach too much.” “My supervisor is no good.” “My students are no good.” “I don’t have enough time for research.” When challenged on what actions we are taking to rectify the situation, we will claim that it’s the fault of other people, or of circumstances beyond our control, relieving ourselves of the burden of doing anything to solve the problem.

In short, we abdicate responsibility, preferring to blame others. This is easier over the short term, since it’s easier to complain than it is to take action, but is not a recipe for long-term happiness or effectiveness. Furthermore, we will usually deny that it is within our power to take actions to improve our situation. After all, if it was in our power, it would be us who is responsible, and our entire worldview is based upon blaming others for our own problems.

The second way of abdicating responsibility is to get caught up in displacement activities. These may give us a short-term fix, especially if they win us the approbation of other people, perhaps for responding to requests that they label urgent. Over the long run such displacement activities are ultimately unfulfilling, representing time lost from our lives.

The third way of abdicating responsibility is by getting down on yourself, worrying and feeling bad for not overcoming one’s difficulties. Winston Churchill spoke of the “black dog” of depression that overtook him during times when his political career was in eclipse. Personally, I sometimes get really down when things are not going well, and get caught up in a cycle of worry and analysis, without constructively addressing my problems. Of course, the right way to respond to a bad situation is not to beat yourself up, but rather to admit that, yes, things are going badly, to figure out exactly what problems you are facing, write out possible solutions, prioritize and implement them, without getting too worried or hamstrung by the whole process.

Why are these three options so attractive? Why do we so often choose to respond in this way to the challenges of life rather than taking things on with a proactive attitude that acknowledges that we’re responsible for our own life? What all three options share in common is that over the short-term abdicating responsibility for our problems is easier than taking responsibility for meeting the challenges of life.

A specific example that I believe speaks to many of us is when we’re having some sort of difficulty or conflict with another person. How many of us put off confronting the problem, preferring instead to hope that the problem will resolve itself? Yet, properly managed – a difficult thing to do, most likely requiring considerable preparation and aforethought – it’s nearly always better to talk with the person about the problem until you arrive at a mutual understanding of both your points of view, both sets of interests, and can resolve the issue on a basis of shared trust.

How can we learn to become proactive? I don’t know of any easy way. One powerful way is to be inspired by examples of proactive people. This can either be through direct personal contact, or indirectly through biographies, history, movies and so on. I like to set aside regular time for such activities. Another powerful tool for learning proactivity is to remind ourselves regularly of the costs and benefits of proactivity and responsibility versus reactivity and irresponsibility. These costs and benefits are easy to forget, unless you’re constantly being reminded that complaints, self-doubt, blame of others and of self are actually the easy short-term way out, and that chances are that you can construct a better life for yourself, at the cost of needing to do some hard work over the short term.

In the context of research, this means constantly reminding yourself that you are the person ultimately responsible for your research effectiveness. Not the institution you find yourself in. Not your colleagues, or supervisor. Not the society you are living in. All these things influence your research career, and may be either a help or a hindrance (more on that later), but in the final analysis if things are not working well it is up to you to take charge and change them.

Effective people have a vision of what they’d like to achieve. Ideally, such a vision incorporates both long-term values and goals, as well as shorter-term goals. A good vision answers questions like: What sort of researcher would I like to become? What areas of research am I interested in? How am I going to achieve competence in those areas? Why are those areas interesting? How am I going to continue growing and expanding my horizons? What short-term steps will I take to achieve those goals? How will I balance the long-term goals with the short-term realities of the situation I find myself in? For example, if you’re in a temporary job and need to get another job soon, it’s probably not such a great idea to devote all your time to learning some new subject, without any visible outcome.

A vision is not something you develop overnight. You need to work at it, putting time aside for the process, and learning to integrate it into your everyday life. It’s a challenging process, but over the long run it’s also extremely rewarding. History shows that great actions usually are the outcome of great purpose, even if the action that resulted was not the original purpose. Your vision doesn’t always need to be of a great purpose; it’s good to work on the little stuff, some of the time. But you should occasionally set yourself some big, ambitious goal, a goal that gets you excited, that makes you want to get up in the morning, and where you’ve developed a confidence in your own mind that you have a chance of achieving that goal. Such a great purpose inspires in a way that the humdrum cannot; it makes things exciting and worthwhile if you feel you’re working towards some genuinely worthy end. I believe this is particularly important in the more abstract parts of research (like theoretical physics), where it can require some work to make a personal, emotional connection to one’s own research. Having a clear vision of a great end is one very good way of making such a connection. When you don’t do this, you can get stuck in the rut of the everyday; you need to get out of that rut, to develop a bigger vision.

Finally, a good vision is not inflexible. It’s something that gets changed as you go along, never lightly, but frequently. The importance of having the vision is that it informs your everyday and every week decisions, giving you a genuinely exciting goal to work towards.

Self-discipline

Effective people are self-disciplined . They work both hard and smart, in the belief that you reap what you sow. How does one achieve such self-discipline? It’s a difficult problem. Wayne Bennett, one of the most successful coaches in the history of the sport of Rugby League, sums the problem up well when he says “I’ve had more trouble with myself than any other man I’ve ever met”.

It is a tempting but ultimately counterproductive fallacy to believe that self-discipline is merely a matter of will , of deciding what it is that you want to do, and then doing it. Many other factors affect self-discipline, and it’s important to understand those other factors. Furthermore, if you believe that it’s all a matter of willpower then you’re likely to get rather depressed when you fall short, sapping your confidenc, and resulting in less disciplined behaviour.

I now describe three factors important in achieving self-discipline.

The first factor is having clarity about what one wants to achieve, why one wants to achieve it, and how to go about achieving it. It’s easy to work hard if you’re clear about these three things, and you’re excited about what you’re doing. Conversely, I think the main cause of aimlessness and procrastination is when you lack clarity on one or more of these points.

The second factor affecting self-discipline is one’s social environment. Researchers are typically under little immediate social pressure to produce research results. Contrast this with the example of professional athletes, who often have an entire support system of coaches, managers and trainers in place, focused around the task of increasing their effectiveness. When a researcher stays out late, sleeps in, and gets a late start, no-one minds; when a professional athlete does, they’re likely to receive a blast from their coach.

Access to a social environment which encourages and supports the development of research skills and research excellence can make an enormous difference to all aspect of one’s research, including self-discipline. The key is to be accountable to other people. Some simple ways of achieving such accountability are to take on students, to collaborate with colleagues, or to set up mentoring relationships with colleagues.

The third factor affecting self-discipline is a special kind of honesty, honesty to oneself, about oneself. It’s extremely easy to kid ourselves about what we do and who we are. A colleague once told me of a friend of his who for some time used a stopwatch to keep track of how much research work he did each week. He was shocked to discover that after factoring in all the other activities he engaged in each day – interruptions, email, surfing the net, the phone, fruitless meetings, chatting with friends, and so on – he was averaging only half an hour of research per day. I wouldn’t be surprised if this was typical of many researchers. The good news, of course, is that building this kind of awareness lays the foundation for personal change, for achieving congruence between our behavioural goals and how we actually behave, in short, for achieving self-discipline.

Aspects of research: self-development and the creative process

Research involves two main aspects, self-development and the creative process of research. We’ll discuss the specifics of each aspect below, but for now I want to concentrate on the problem of achieving balance between the two, for I believe it is a common and significant mistake to concentrate too much on one aspect to the exclusion of the other.

People who concentrate mostly on self-development usually make early exits from their research careers. They may be brilliant and knowledgeable, but they fail to realize their responsibility to make a contribution to the wider community. The academic system usually ensures that this failure is recognized, and they consequently have great difficulty getting jobs. Although this is an important problem, in this essay I will focus mostly on the converse problem, the problem of focusing too much on creative research, to the exclusion of self-development.

There are a lot of incentives for people to concentrate on creative research to the exclusion of self-development. Throughout one’s research career, but particularly early on, there are many advantages to publishing lots of papers. Within limits, this is a good thing, especially for young researchers: it brings you into the community of researchers; it gives you the opportunity to learn how to write well, and give good presentations; it can help keep you motivated. I believe all researchers should publish at least a few papers each year, essentially as an obligation to the research and wider community; they should make some contribution, even if only a small one, on a relatively unimportant topic.

However, some people end up obsessed with writing as many papers as possible, as quickly as possible. While the short-term rewards of this are attractive (jobs, grants, reputation and prizes), the long-term costs are significant. In particular, it can lead to stagnation, and plateauing as a researcher. To achieve one’s full potential requires a balancing act: making a significant and regular enough research contribution to enable oneself to get and keep good jobs, while continuing to develop one’s talents, constantly renewing and replenishing oneself. In particular, once one has achieved a certain amount of job security (a long-term or permanent job) it may make sense to shift the balance so that self-development takes on a larger role.

For many people (myself included) who have concentrated mainly on making creative research contributions earlier in their careers, this can be a difficult adjustment to make, as it requires changing one’s sense of what is important. Furthermore, there is a constant pull towards concentrating on research over self-development, since there are often short-term incentives to sacrifice self-development for research (“I’ve got to get this paper out now”), but rarely vice versa. To balance these tendencies, we need to remember that nobody, no matter how talented, is born an effective researcher; that distinction can only be obtained after a considerable amount of hard work and personal change, and there is no reason to suppose that just because one is now able to publish lots of papers that one has peaked as a researcher.

In my opinion, creative research is best viewed as an extension of self-development, especially an extension of a well-developed reading program. I don’t believe the two can be completely pried apart, as the two interact in interesting non-linear ways. I’m now going to talk in a little more detail about both processes, keeping in mind that the ultimate goal of research is new ideas, insights, tools and technologies, and this goal must inform the process of self-development.

Developing research strengths

The foundation is a plan for the development of research strengths. What are you interested in? Given your interests, what are you going to try to learn? The plan needs to be driven by your research goals, but should balance short-term and long-term considerations. Some time should be spent on things that appear very likely to lead to short-term research payoff. Equally well, some time needs to be allocated to the development of strengths that may not have much immediate pay-off, but over the longer-term will have a considerable payoff.

In targeting areas of development, an important goal to keep in mind is that you want to develop unique combinations of abilities. You need to develop unique combinations of talents which give you a comparative advantage over other people. Do what you can do better than anybody ; to mangle a quote from Lincoln, nobody can be better than everybody all of the time, but anybody can be better than everybody some of the time.

In my opinion the reason most people fail to do great research is that they are not willing to pay the price in self-development. Say some new field opens up that combines field X and field Y. Researchers from each of these fields flock to the new field. My experience is that virtually none of the researchers in either field will systematically learn the other field in any sort of depth. The few who do put in this effort often achieve spectacular results.

Finally, a note on how to go about developing some new research strength. A mistake I’m prone to make, and I know some others are as well, is to feel as though some degree of completeness is required in understanding a research field. In fact, in any given research field there are usually only a tiny number of papers that are really worth reading. You are almost certainly better off reading deeply in the ten most important papers of a research field than you are skimming the top five hundred.

These ideas carry over to the problem of staying current in your fields of interest: I believe that you can stay quite current by (a) quickly skimming a great deal of work, to keep track of what is known, and what sort of problems people are thinking about, and (b) based on that skimming, picking a dozen or so papers each year to read deeply, in the belief that they contain the most important research results of the year. This is not the only deep reading you’ll need to do; you’ll also need to do some which is related to the immediate problems that you’re working on. But you certainly should do some such deep reading.

Develop a high-quality research environment

There is a considerable amount of research showing that people consistently underestimate the effect of the environment on personal effectiveness. This is particularly important in an academic environment where there are usually many short-term social pressures that are not directly related to research effectiveness – teaching, writing letters of recommendation and referee reports, committee work, academic politics. By contrast, in most institutions there are few short-term social pressures to do great research work.

Some of the highest-leverage work you can do involves improving your environment so that social pressures work for you as a researcher, rather than against you. Discussing this in detail would require another essay of length at least equal to that of the present one, but I will make a few remarks.

The first is that improving your environment is something anyone can do; students, in particular, often underestimate the magnitude of the changes they can bring about. Anyone can start a seminar series, develop a discussion area, create a lounge, organize a small workshop, or organize a reading group. Furthermore, although all these things are hard to do well, if you’re willing to do critical evaluations, experiment and try radical changes, preferably in partnership with equally committed people, things are likely to improve a great deal.

Second, institutions have long memories, so changes that you make in your environment will stick around for a long time. This means that once something is working well, chances are it’ll continue to work well without much help from you – and you can move on to improve some other aspect of your environment. Furthermore, each positive change you make actually improves your leverage with other people. I’ve known undergraduate students who had made so many creative positive contributions to their departments that their influence with canny senior faculty was comparable to the influence of other senior faculty.

The creative process

The problem-solver and the problem-creator.

Different people have different styles of creative work. I want to discuss two different styles that I think are particularly useful in understanding the creative process. I call these the problem-solver and the problem-creator styles. They’re not really disjoint or exclusive styles of working, but rather idealizations which are useful ways of thinking about how people go about creative work.

The problem-solver: This is the person who works intensively on well-posed technical problems, often problems known (and sometimes well-known) to the entire research community in which they work. The best problem-solvers are often extremely technically proficient and hard-working. Problem-solvers often attach great social cache to the level of difficulty of the problem they solve, without necessarily worrying so much about other indicators of the importance of the problem.

The problem-creator: This is a rarer working style. Problem-creators may often write papers that are technically rather simple, but ask an interesting new question, or pose an old problem in a new way, or demonstrate a simple but fruitful connection that no-one previously realized existed.

Of course, the problem-solver and the problem-creator are idealizations; all researchers exemplify both styles, to some extent. But they are also useful models to clarify our thinking about the creative process. One distinction between the two styles is how proactive one is in identifying problems, with the problem-solver being much more passive, while the problem-creator is extremely proactive. By contrast, the problem-solver needs to be much more proactive in developing their problem-solving skills. Both styles of research can be extremely successful.

Problem-solvers have numerous social advantages in research, and for that reason I believe they tend to be more common. In particular, it is relatively easy to recognize (and then reward) people who solve problems that are of medium or high levels of difficulty. This has rewards both in terms of the immediate esteem of one’s peers – physicists love to trade legends about brilliant colleagues who immediately see through to the solution of some difficult problems or another – and also in the hunt for jobs and other tangible forms of recognition. It takes more time (and thus can be more difficult) to recognize people whose work is technically rather simple, but whose questions may eventually open up whole new lines of enquiry.

The advantage in being a problem-creator is that there is a sizeable comparative advantage in opening up an entirely new problem area, and thus being the first into that problem area. You can work hard to get a basic foundation in the skills needed in that problem area, and then clean up many of the fundamental problems.

The skills of the problem-creator

Our training as physicists focuses pretty heavily on becoming problem-solvers; we tend not to get much training as problem-creators. One reason I’m discussing these two working styles at some length is to dispel the common idea that creative research is necessarily primarily about problem-solving. It’s true that many people have very successful research career as problem-solvers. But you can also consciously decide to invest more time and effort into developing as a problem-creator. I now describe some of the skills involved in problem-creation.

Developing a taste for what’s important: What do you think are the characteristics of important science? What makes one area thrive, while another dies away? What sorts of unifying ideas are the most useful? What have been the most important developments in your field? Why are they important? What were the apparently promising ideas that didn’t pan out? Why didn’t they pan out? You need to be thinking constantly about these issues, both in concrete terms, and also in the abstract, developing both a general feeling for what is important (and what is not), and also some specific beliefs about what is important and what is not in your fields of interest. Richard Hamming describes setting aside time each week for “Great Thoughts”, time in which he would focus on and discuss with others only things that he believed were of the highest importance. Systematically setting aside time to think (and talk with colleagues) about where the important problems are is an excellent way of developing as a problem-creator.

On this topic, let me point out one myth that exerts a powerful influence (often subconsciously) on people: the idea that difficulty is a good indicator of the importance of a problem. It is true that an elegant solution to a difficult problem (even one not a priori important) often contains important ideas. However, I believe that most people consistently over rate the importance of difficulty. Often far more important is what your work enables, the connections that it makes apparent, the unifying themes uncovered, the new questions asked, and so on.

Internal and external standards for what is important: Some of the most thought-provoking advice on physics that I ever heard was at a colloquium given by eminent physicist Max Dresden. He advised young people in the audience not to work towards a Nobel Prize, but instead to aim their research in directions that they personally find fun and interesting. I thought his advice quite sound in some regards: for some people it is extremely tempting to regard external recognition as the be-all and end-all of research success, and the Nobel Prize is perhaps the highest form of external recognition in physics. Dresden is right, in the sense that working with a primary goal of winning a Nobel Prize would be pointless and degrading; far better to work in an area one personally finds enjoyable.

On the other hand, the Nobel Prizes are usually given for very good reasons: they reward some of the most interesting work in all of physics. There is, admittedly, a political element, with certain fields being favoured, and so on. Nonetheless, imagine a world in which one of these discoveries had not been awarded a Prize for some reason. Would you be proud to have your name associated with that discovery, even so, and regard the work on it as time well spent? In every case I can think of, that certainly is the case for me, and I suspect it’s true for most other physicists.

I believe this highlights an interesting point about what makes something interesting and important. A person working toward a Nobel Prize or some other form of external recognition has, in some sense, decided to abdicate their personal decision about what is important and interesting. The external community of physicists (in this case, represented by the Nobel Committee) is what makes their decision: if it might win a Nobel, it’s important.

Balancing this observation, this is not to say that your decision about what is interesting and important should be yours along. People who work in isolation rarely end up making contributions that are all that significant. Your decision about what is important should be informed by others: talk to your peers, find out what they think is important, look in the textbooks and history books and biographies, and, yes, look at what wins prizes (of all sorts).

But at the end of the day you’ve got to form your own independent standards for what is interesting and important and worth doing, and make judgments about where you should be making a contribution, based on those standards. I think better advice from Dresden would have been to aim to produce work of the highest possible caliber, but according to what you have come to believe is important.

Exploring for problems: Obviously, all researchers do some of this. For the problem-solver, the process of exploring for problems often works along the following lines: keep moving around, looking for problems that you consider (a) well-posed, or able to be well-posed after some work on your part, (b) likely to fall within a reasonable time to the arsenal of tools at your disposal (perhaps with some small expansion of that arsenal), and (c) below some minimum thresholds of interest and difficulty. Once you’ve found a problem of this sort, you work hard on the problem, solve it, and publish.

Problem-creators may be rather more systematic about exploring for problems. For example, they may occasionally set time aside to survey the landscape of a field, looking not just for problems, but trying to identify larger patterns. What types of questions do people in the field tend to ask? Can we abstract away patterns in those questions? What other fields might there be links to? What are the few most important problems in the field? Problem-creators set aside time for doing this kind of systematic exploration, and do it in a disciplined way, often with feedback from others.

Surveying the landscape can be particularly revealing. A lot of people work in fashionable subfields of a larger field primarily because there are lots of other people working in that subfield. The problems they work on may be technically complicated, especially after a few years, when the most basic questions have been answered. This is compensated by the fact that it’s extremely comforting to work within a field where there is a standard narrative explaining the importance of the field, some canonical models for what problems are interesting, and a willing audience of people ready to appreciate your work. In addition, working in such subfields gives younger people a chance to show off their technical prowess (sometimes, not unlike elk spoiling for a fight) to peers in a position to recommend them for valuable faculty positions.

Getting ahead of the game: There are many important problems, and sometimes an entire field comes to some agreement about what is important: proving the Riemann Hypothesis, or understanding high temperature superconductivity. Sometimes, however, there is a problem either not appreciated at all, or only dimly appreciated, that is equal in importance to such gems. Consider the creation of the scanning tunneling microscope – the basic idea had been around for years, yet nobody had ever seriously tried to build the device. The inventors put it together on a shoestring, and created one of the major tools of modern physics. Or consider David Deutsch and Richard Feynman’s creation of the field of quantum computing, by framing the right questions (“What would a quantum mechanical computer be capable of?” and “Would it be faster than a classical computer?”). One of the big ways you can get ahead as a researcher is by identifying and then solving problems that are important, but perhaps not terribly difficult, ahead of everyone else.

Identify the messes: In a nice article about how he does research, physicist Steven Weinberg emphasized the importance of identifying the messes. What areas of physics appear to be a state of mess? Funnily enough, one of the signs of this can be that it’s very hard to understand. For a long time – and to some extent this persists today – physics texts on general relativity were very difficult to understand. The tensor calculus in them was often confusing and difficult to understand. There was a good reason for this: the basic definitions in the subject of differential geometry, although laid down in the 19th century, didn’t really reach their modern form until the mid part of the twentieth century, and then took considerable time to migrate to physics. The reason a lot of the discussion of tensor calculus in physics texts is confusing is because, very often, it is confused , being written by people who don’t have quite the right definitions (meaning, in this case, simplest, most elegant and natural) in mind.

When you identify such a mess, the natural inclination of many people is to shy away, to find something that is easier to understand. But a field that is a mess is really an opportunity. Chances are good that there are deep unifying and simplifying concepts still waiting to be understood and developed by someone – perhaps you.

The skills of the problem-solver

As I’ve already said, our technical training as physicists focuses a lot more on problem-solving than problem-creation, so I’m not going to say a lot about the skills needed to be a problem-solver. But I will make a few general remarks that I find helpful.

Clarity, goals, and forward momentum: In my opinion, there is little that is more important in research than building forward momentum. Being clear about some goal, even if that goal is the wrong goal, or the clarity is illusory, is tremendously powerful. For the most part, it’s better to be doing something, rather than nothing, provided , of course, that you set time aside frequently for reflection and reconsideration of your goals. Much of the time in research is spent in a fog, and taking the time to set clear goals can really help lift the fog.

Have multiple formulations: One of the most common mistakes made by researchers is to hold on very closely to a particular problem formulation. They will stick closely to a particular formulation of a problem, without asking if they can achieve insights on related problems. The important thing is to be able to make some progress: if you can find a related problem, or reformulate a problem in a way that permits you to move forward, that is progress.

Spontaneous discovery as the outcome of self-development: For me this is one of the most common ways of making discoveries. Many people’s basic research model is to identify a problem they find interesting, and then spend a lot of time working on just that problem. In fact, if you keep your mind open while engaging in exploration, and are working at the edge of what is known, you’ll often see huge opportunities open wide in front of you, provided you keep developing your range of skills.

Working on important problems

It’s important that you work towards being able to solve important problems. This sounds silly, but people don’t do this for any number of reasons. I want to talk a little about those reasons, and how to avoid them.

Reason 1: Lack of self-development. Many people don’t spend enough time on self-development. If you stop your development at the level which resulted in your first paper, it’s unlikely you’ll solve any major problems. More realistically, for many people self-development is an incidental thing, something that happens while they’re on the treadmill of trying to solve problems, generate papers, and so on, or while teaching. While such people will develop, it’s unlikely that doing so in such an ad hoc way will let them address the most important problems.

Reason 2: The treadmill of small problems. Social factors such as the need to publish, get grants, and so on, encourage people to work only on unimportant problems, without addressing the important problems. This can be a difficult treadmill to get off.

My belief is that the way to start out in a research career is by working primarily on small and relatively tractable problems, where you have a good chance of success. You then continue the process of self-development, gradually working up to more important problems (which also tend to be more difficult, although, as noted above, difficulty is most emphatically not the same as importance). The rare exception is important problems that are also likely to be technically easy; if you’re lucky you may find such a problem early in your career, or be handed one. If so, solve it quickly!

Even later on, when you’ve developed to the point that you can realistically expect to be able to attack important problems, it’s still useful to tackle a mixture of more and less important problems. The reason is that tackling smaller problems ensures that you make a reasonable contribution to science, and that you continue to take an active part in the research community. Even Andrew Wiles continued to publish papers and work on other problems during his work on Fermat’s Last Theorem, albeit at a rather low rate. If he had not, he would have lost contact with an entire research community, and losing such contact would likely have made a significant negative difference to his work on Fermat’s Last Theorem.

Reason 3: The intimidation factor. Even if people have spent enough time on self-development that they have a realistic chance of attacking big problems, they still may not. The reason is that they have a fear of working on something unsuccessfully. Imagine Andrew Wiles feeling if he had worked on Fermat’s Last Theorem for several decades, and completely failed. For most people, the fear of ending up in such a situation is enough to discourage them from doing this.

The great mathematician Andrei Kolmogorov described an interesting trick that he used to get around this problem. Rather than investing all his time and effort on attacking the problem, he’d put the problem into a larger context. He’d announce a seminar series in which he’d lecture on material that he thought would be related to the problem. He’d write a set of lecture notes (often turning into a book) on material related to the problem. That way, he lowered the psychological pressure on himself. Rather than investing all his effort in an attack on the problem – which might ultimately be a complete waste of time – he knew that he’d produce something of value. By making the research process part of a larger endeavour, he ensured that the process was a success no matter how it came out, even if he failed to solve the problem, or was scooped by someone else. It’s a case of not putting all of one’s psychological eggs in one basket.

Richard Feynman described a related trick. He said that when he started working on a problem he would try to convince himself that he had some kooky insight into the problem that it was very unlikely anybody else had. He admitted that very often this belief was erroneous, or that, even if original, his initial insight often wasn’t very good. But he claimed that he found that he could fool himself into thinking that he had the “inside track” on the problem as a result, and this was essential to getting up the forward momentum necessary to really make a big dint in a difficult problem.

Committing to work on an important problem: For the difficult problems, I think commitment is really a process rather than a moment. You may decide to prepare a lecture to talk about a problem. If that is interesting, you enjoy it, and you feel like you have some insight, you might decide to prepare a few lectures. If that goes well, perhaps you’ll start to prepare more lectures, write a review, and maybe make a really big contribution. It’s all about building up more and more insight. Ideally, you’ll do this as part of some larger process, with social support around you.

People who only attack difficult problems: There is a converse to the problem I’ve been talking about, which is people who are only interested in attacking problems that are both difficult and important. This affliction can affect people at any stage of their career, but it manifests itself in somewhat different ways at different stages.

In the case of the beginner, this is like a beginning pole vaulter insisting on putting the bar at 5 meters from the time they begin, rather than starting at some more reasonable height. Unless exceptionally pigheaded, such a person will never learn to vault 5 meters successfully, simply because they will never learn anything from failure at a more realistic starting height. This sounds prima facie ridiculous, but I have seen people burn out by following exactly this strategy.

The case of the more experienced researcher is more difficult. As I’ve emphasized, once you’ve reached an appropriate level of development I think it’s important to spend some time working on the most important problems. But if that’s all you do, there are some very significant drawbacks. In particular, by attacking only the most important and most difficult problems an experienced researcher (a) takes themselves out of circulation, (b) stops making ongoing contributions, (c) loses the habit of success, and (d) risks losing morale, which is so important to research success. I think the solution is to balance one’s work on the more and less important problems: you need to schedule time to do the more important stuff, but should also make sure that you spend some time on less high-risk activities.

In both cases, the explanation is often, at least in part, intellectual macho. Theorists can be a pretty judgmental lot about the value of their own work, and the work of others. This helps lead some into the error of only working on big problems, and not sometimes working on little problems, for the fun of it, for the contact it brings with colleagues, and for the rewarding and enjoyable sense of making a real contribution of some significance.

This essay has been translated into Chinese by Buhua Liu.

10 comments

Excellent stuff! It reads like a researcher’s “Enchiridion.”

I’ve written about some of these issues on my own site, in particular, the need to try to find something important to work on. In my experience, the difficulty of trying to work on large problems is even greater in the industrial setting. Amazingly trivial things are perfectly capable of eating away your days, and your career.

Have you read Robert Root-Bernstein’s book “Discovering”? He addresses the question of how people make important discoveries, and concludes that some of it is (or could be) learned behavior. It’s an odd book, but well worth seeking out. Many of its conclusions are very similar to yours.

Derek: Thanks for the kind words. I’ve never read Epictetus, but I’m quite a fan of Marcus Aurelius’ “Meditations”, so perhaps that’s the influence you’re seeing.

As an academic, I actually did quite a bit of agonizing over whether or not to cite my sources, and ended up deciding not to in the main, viewing this more like a magazine article or opinion piece than a piece of research. One day I’d like to go back and construct a nice annotated bibliography. (Possibly with the book you recommend, which I’ll certainly take a look at.)

A very realistic portrait of the challenges involved in tackling significant problems. Maintaining habits which lead to excellence is very difficult in any field. I find often scientists assume research skills don’t translate into business, but critical thinking and the ability to properly dissect a problem into manageable chunks is a rare and valuable skill.

What an enjoyable and useful look at the process of research. What I find particularly useful is that there is a “balance” in all areas, “nothing in excess,” approach here, i.e. the advice to have home and life in order, in the long-term, to produce quality. Classical, rather than the romantic “burn brightly and fade away.” I am also intrigued by your categories of researcher: the problem solver and problem creator. I find these very accurate descriptions in my own field, English, (more specifically rhetoric and composition) and in my own ideal of how I want to pursue research – the big questions of language and writing research (what is persuasive language? how do we teach writing?) alternate with “smaller,” practical issues of research (what is an effective way to teach this essay, format a bibliography, etc.)

Inspiring read. Thanks.

Ed and Susan: Thankyou to you both for the kind words.

On categories of researcher: I find it remarkable and surprising to hear that this kind of divide holds also in English. I was certainly only thinking of theoretical physics (and, to some extent, mathematics) when I wrote that!

I’m a postgraduate and began to do research in theoretical physics just now .This article is quite helpful and useful for me. Thank you for writing the nice article! I will recommend it to my friends.

This piece definitely resonates with me. I hope your students listen and understand the advantage they have if you communicate this to them early. I believe most people obtain PhDs but are not taught *how* to do research – which is what you discuss; they only solve *some* problem and by osmosis are to pick up the *how*. Most pick this up from many post-docs and jobs if they are lucky enough to work with good researchers, a horribly inefficient process.

You may be interested in looking at what has been written about John Wheeler’s approach to working with his graduate students (I read a summary somewhere, which I now misplaced). You may have more materials and colleagues you can access. It always amazed me that so many of his students did excellent work. It was not that they were all geniuses and he just knew how to pick them; there was definite training here — on how to solve problems of increasing difficulty, and to always work on one short-range, and one long-range problem, at the same time.

One final note: any ideas on scientists in industry? In particular, I found the following pressures eventually lead to a state of brownian motion: the pressure that all work “has to be profitable”, the “instant expert” syndrome and tracking and justifying time by the minute.

Thanks, fc.

On Wheeler: I’d be _extremely_ interested to see the document you refer to. Like you, I am amazed at all that has been accomplished by the people he worked with. I doubt it was chance.

On brownian motion in research: even in academia, I find I sometimes end up feeling the same kind of pressures you mention. Personally, I find one useful solution is to regularly schedule a short period – even just an hour or half an hour a day – in which I isolate myself from other concerns, and just concentrate on one thing that I believe is truly important, deliberately allowing myself to work on it in an unhurried way, without interruptions. No phone, no email, no web, and no working on stuff that is urgent, but neither fun nor important!

Please answer me this question “WHAT DEFINES A GOOD RESEARCH”

is there any importance of research?

Comments are closed.

No internet connection.

All search filters on the page have been cleared., your search has been saved..

• All content
• Dictionaries
• Encyclopedias
• Expert Insights

Foundations

• How-to Guides
• Journal Articles
• Little Blue Books
• Little Green Books
• Project Planner
• Tools Directory
• Sign in to my profile My Profile

• Sign in Signed in
• My profile My Profile

You might be interested in

Bring research methods, statistics and evaluation to life.

Find practice datasets to help you master qualitative and quantitative data analysis.

Introductory overviews to all the major methods to help get you on your way

Find stories of real research to learn from and be inspired by

Navigating away from this page will delete your results

Please save your results to "My Self-Assessments" in your profile before navigating away from this page.

Sign in to my profile

Sign up for a free trial and experience all Sage Learning Resources have to offer.

You must have a valid academic email address to sign up.

Get off-campus access

• View or download all content my institution has access to.

Sign up for a free trial and experience all Sage Research Methods has to offer.

• view my profile
• view my lists

• Developing a Research Question

by acburton | Mar 22, 2024 | Resources for Students , Writing Resources

Selecting your research question and creating a clear goal and structure for your writing can be challenging – whether you are doing it for the first time or if you’ve done it many times before. It can be especially difficult when your research question starts to look and feel a little different somewhere between your first and final draft. Don’t panic! It’s normal for your research question to change a little (or even quite a bit) as you move through and engage with the writing process. Anticipating this can remind you to stay on track while you work and that it’ll be okay even if the literature takes you in a different direction.

What Makes an Effective Research Question?

The most effective research question will usually be a critical thinking question and should use “how” or “why” to ensure it can move beyond a yes/no or one-word type of answer. Consider how your research question can aim to reveal something new, fill in a gap, even if small, and contribute to the field in a meaningful way; How might the proposed project move knowledge forward about a particular place or process? This should be specific and achievable!

The CEWC’s Grad Writing Consultant Tariq says, “I definitely concentrated on those aspects of what I saw in the field where I believed there was an opportunity to move the discipline forward.”

General Tips

Do your research.

Utilize the librarians at your university and take the time to research your topic first. Try looking at very general sources to get an idea of what could be interesting to you before you move to more academic articles that support your rough idea of the topic. It is important that research is grounded in what you see or experience regarding the topic you have chosen and what is already known in the literature. Spend time researching articles, books, etc. that supports your thesis. Once you have a number of sources that you know support what you want to write about, formulate a research question that serves as the interrogative form of your thesis statement.

Grad Writing Consultant Deni advises, “Delineate your intervention in the literature (i.e., be strategic about the literature you discuss and clear about your contributions to it).”

Start Broadly…. then Narrow Your Topic Down to Something Manageable

When brainstorming your research question, let your mind veer toward connections or associations that you might have already considered or that seem to make sense and consider if new research terms, language or concepts come to mind that may be interesting or exciting for you as a researcher. Sometimes testing out a research question while doing some preliminary researching is also useful to see if the language you are using or the direction you are heading toward is fruitful when trying to search strategically in academic databases. Be prepared to focus on a specific area of a broad topic.

Writing Consultant Jessie recommends outlining: “I think some rough outlining with a research question in mind can be helpful for me. I’ll have a research question and maybe a working thesis that I feel may be my claim to the research question based on some preliminary materials, brainstorming, etc.” — Jessie, CEWC Writing Consultant

Try an Exercise

In the earliest phase of brainstorming, try an exercise suggested by CEWC Writing Specialist, Percival! While it is normally used in classroom or workshop settings, this exercise can easily be modified for someone working alone. The flow of the activity, if done within a group setting, is 1) someone starts with an idea, 2) three other people share their idea, and 3) the starting person picks two of these new ideas they like best and combines their original idea with those. The activity then begins again with the idea that was not chosen. The solo version of this exercise substitutes a ‘word bank,’ created using words, topics, or ideas similar to your broad, overarching theme. Pick two words or phrases from your word bank, combine it with your original idea or topic, and ‘start again’ with two different words. This serves as a replacement for different people’s suggestions. Ideas for your ‘word bank’ can range from vague prompts about mapping or webbing (e.g., where your topic falls within the discipline and others like it), to more specific concepts that come from tracing the history of an idea (its past, present, future) or mapping the idea’s related ideas, influences, etc. Care for a physics analogy? There is a particle (your topic) that you can describe, a wave that the particle traces, and a field that the particle is mapped on.

Get Feedback and Affirm Your Confidence!

Creating a few different versions of your research question (they may be the same topic/issue/theme or differ slightly) can be useful during this process. Sharing these with trusted friends, colleagues, mentors, (or tutors!) and having conversations about your questions and ideas with other people can help you decide which version you may feel most confident or interested in. Ask colleagues and mentors to share their research questions with you to get a lot of examples. Once you have done the work of developing an effective research question, do not forget to affirm your confidence! Based on your working thesis, think about how you might organize your chapters or paragraphs and what resources you have for supporting this structure and organization. This can help boost your confidence that the research question you have created is effective and fruitful.

Be Open to Change

Remember, your research question may change from your first to final draft. For questions along the way, make an appointment with the Writing Center. We are here to help you develop an effective and engaging research question and build the foundation for a solid research paper!

Example 1: In my field developing a research question involves navigating the relationship between 1) what one sees/experiences at their field site and 2) what is already known in the literature. During my preliminary research, I found that the financial value of land was often a matter of precisely these cultural factors. So, my research question ended up being: How do the social and material qualities of land entangle with processes of financialization in the city of Lahore. Regarding point #1, this question was absolutely informed by what I saw in the field. But regarding point #2, the question was also heavily shaped by the literature. – Tariq

Example 2: A research question should not be a yes/no question like “Is pollution bad?”; but an open-ended question where the answer has to be supported with reasons and explanation. The question also has to be narrowed down to a specific topic—using the same example as before—”Is pollution bad?” can be revised to “How does pollution affect people?” I would encourage students to be more specific then; e.g., what area of pollution do you want to talk about: water, air, plastic, climate change… what type of people or demographic can we focus on? …how does this affect marginalized communities, minorities, or specific areas in California? After researching and deciding on a focus, your question might sound something like: How does government policy affect water pollution and how does it affect the marginalized communities in the state of California? -Janella

Our Newest Resources!

• Transitioning to Long-form Writing
• Integrating Direct Quotations into Your Writing
• Nurturing a Growth Mindset to Overcome Writing Challenges and Develop Confidence in College Level Writing
• An Introduction to Paraphrasing, Summarizing, and Quoting

Additional Resources

• Graduate Writing Consultants
• Instructor Resources
• Student Resources
• Quick Guides and Handouts
• Self-Guided and Directed Learning Activities

Princeton Correspondents on Undergraduate Research

How to Make a Successful Research Presentation

Turning a research paper into a visual presentation is difficult; there are pitfalls, and navigating the path to a brief, informative presentation takes time and practice. As a TA for  GEO/WRI 201: Methods in Data Analysis & Scientific Writing this past fall, I saw how this process works from an instructor’s standpoint. I’ve presented my own research before, but helping others present theirs taught me a bit more about the process. Here are some tips I learned that may help you with your next research presentation:

More is more

In general, your presentation will always benefit from more practice, more feedback, and more revision. By practicing in front of friends, you can get comfortable with presenting your work while receiving feedback. It is hard to know how to revise your presentation if you never practice. If you are presenting to a general audience, getting feedback from someone outside of your discipline is crucial. Terms and ideas that seem intuitive to you may be completely foreign to someone else, and your well-crafted presentation could fall flat.

Less is more

Limit the scope of your presentation, the number of slides, and the text on each slide. In my experience, text works well for organizing slides, orienting the audience to key terms, and annotating important figures–not for explaining complex ideas. Having fewer slides is usually better as well. In general, about one slide per minute of presentation is an appropriate budget. Too many slides is usually a sign that your topic is too broad.

Limit the scope of your presentation

Don’t present your paper. Presentations are usually around 10 min long. You will not have time to explain all of the research you did in a semester (or a year!) in such a short span of time. Instead, focus on the highlight(s). Identify a single compelling research question which your work addressed, and craft a succinct but complete narrative around it.

You will not have time to explain all of the research you did. Instead, focus on the highlights. Identify a single compelling research question which your work addressed, and craft a succinct but complete narrative around it.

Craft a compelling research narrative

After identifying the focused research question, walk your audience through your research as if it were a story. Presentations with strong narrative arcs are clear, captivating, and compelling.

• Introduction (exposition — rising action)

Orient the audience and draw them in by demonstrating the relevance and importance of your research story with strong global motive. Provide them with the necessary vocabulary and background knowledge to understand the plot of your story. Introduce the key studies (characters) relevant in your story and build tension and conflict with scholarly and data motive. By the end of your introduction, your audience should clearly understand your research question and be dying to know how you resolve the tension built through motive.

• Methods (rising action)

The methods section should transition smoothly and logically from the introduction. Beware of presenting your methods in a boring, arc-killing, ‘this is what I did.’ Focus on the details that set your story apart from the stories other people have already told. Keep the audience interested by clearly motivating your decisions based on your original research question or the tension built in your introduction.

• Results (climax)

Less is usually more here. Only present results which are clearly related to the focused research question you are presenting. Make sure you explain the results clearly so that your audience understands what your research found. This is the peak of tension in your narrative arc, so don’t undercut it by quickly clicking through to your discussion.

• Discussion (falling action)

By now your audience should be dying for a satisfying resolution. Here is where you contextualize your results and begin resolving the tension between past research. Be thorough. If you have too many conflicts left unresolved, or you don’t have enough time to present all of the resolutions, you probably need to further narrow the scope of your presentation.

• Conclusion (denouement)

Return back to your initial research question and motive, resolving any final conflicts and tying up loose ends. Leave the audience with a clear resolution of your focus research question, and use unresolved tension to set up potential sequels (i.e. further research).

Use your medium to enhance the narrative

Visual presentations should be dominated by clear, intentional graphics. Subtle animation in key moments (usually during the results or discussion) can add drama to the narrative arc and make conflict resolutions more satisfying. You are narrating a story written in images, videos, cartoons, and graphs. While your paper is mostly text, with graphics to highlight crucial points, your slides should be the opposite. Adapting to the new medium may require you to create or acquire far more graphics than you included in your paper, but it is necessary to create an engaging presentation.

The most important thing you can do for your presentation is to practice and revise. Bother your friends, your roommates, TAs–anybody who will sit down and listen to your work. Beyond that, think about presentations you have found compelling and try to incorporate some of those elements into your own. Remember you want your work to be comprehensible; you aren’t creating experts in 10 minutes. Above all, try to stay passionate about what you did and why. You put the time in, so show your audience that it’s worth it.

For more insight into research presentations, check out these past PCUR posts written by Emma and Ellie .

— Alec Getraer, Natural Sciences Correspondent

Share this:

• Share on Tumblr

• Link to facebook
• Link to linkedin
• Link to twitter
• Link to youtube
• Writing Tips

Efficient Research: 5 Ways to Make the Most of Your Study Time

3-minute read

• 29th August 2016

Life as a student can be busy, so you might need to cram a lot into a short period of time. Knowing how to conduct efficient research will therefore give you more time to focus on other things.

This is especially vital when working on more than one assignment at the same time, or when balancing work and your studies. To help out, we’ve come up with a few guidelines for conducting efficient research.

1. Create a Research Plan

One big mistake some students make when researching an essay is diving straight into the library (or ignoring it until a week before the deadline). But this will mean your reading is unfocused, costing you additional time and effort.

By creating a research plan you ensure that you work more effectively. Be sure to consider the time you have to conduct your research, the questions you’re seeking to answer and what you intend to argue.

Once you’ve done this, you should be able to focus your research on the most relevant texts.

2. Have a Search Strategy

If you are looking for sources via an electronic database or the internet, it’s important to have a search strategy. This involves identifying key terms, knowing the databases you’ll be searching and using Boolean operators to narrow your search to the most relevant material.

Some universities require you to document your search strategy while writing up research, but even if this isn’t the case it can still be helpful to give this a little consideration.

Find this useful?

Subscribe to our newsletter and get writing tips from our editors straight to your inbox.

3. Location, Location, Location…

Once you’ve found relevant sources, make sure you also pick a good place to read them . This should be somewhere free from distractions, where you can be most productive.

4. Efficient and Effective Note Taking

The way in which you take notes is also important. Try summarising what you read, rather than copying it out verbatim. This will help you retain information and saves time overall (you can copy out key quotations, if required).

Other time-saving tips include using abbreviations where possible (e.g. ‘w/’ for ‘with’, ‘b/c’ for ‘because’, etc.) and noting down the publication information for sources as you read them (which will save time when you come to write your reference list). It’s also worth trying out different note-taking techniques , as we all learn in different ways.

5. Be Organised!

Not that we want to be accused of being anally retentive, but the benefits of a good filing system shouldn’t be underestimated.

Sorting everything into neatly labelled folders might not seem like an efficient use of your time to begin with, but when you need to check your notes while writing an essay or revising for a test you’ll definitely appreciate having made the effort!

Share this article:

Post A New Comment

Get help from a language expert. Try our proofreading services for free.

4-minute read

The Benefits of Using an Online Proofreading Service

Proofreading is important to ensure your writing is clear and concise for your readers. Whether...

2-minute read

6 Online AI Presentation Maker Tools

Creating presentations can be time-consuming and frustrating. Trying to construct a visually appealing and informative...

What Is Market Research?

No matter your industry, conducting market research helps you keep up to date with shifting...

8 Press Release Distribution Services for Your Business

In a world where you need to stand out, press releases are key to being...

How to Get a Patent

In the United States, the US Patent and Trademarks Office issues patents. In the United...

The 5 Best Ecommerce Website Design Tools 

A visually appealing and user-friendly website is essential for success in today’s competitive ecommerce landscape....

Make sure your writing is the best it can be with our expert English proofreading and editing.

College Info Geek

How to Do Research in 7 Simple Steps

C.I.G. is supported in part by its readers. If you buy through our links, we may earn an affiliate commission. Read more here.

It’s 2 am, and you’re on your fifth cup of coffee (or was it your sixth?). You’re crouched at a table in some dark corner of the library surrounded by fifteen open books. Equally as many tabs are open on your laptop, and you still haven’t written a word of the paper that’s due in 7 hours.

Many things can explain how you got to this point, including procrastination , poor organization , and a messy schedule .

Very often, however, the problem is a lack of research skills .

And it’s not your fault. High school does a poor job of teaching you how to do research, and most college classes do little better. It feels like you’re expected to figure it out through trial and error.

I think we can do better than that, however. In this guide, I’m going to show you the 7-step process for researching everything from a 10-page term paper to a final presentation. Not only will you learn how to do better research; you’ll also learn how to research more efficiently.

What Is Research?

Before we go any further, what  is  research?

At its core, research is an attempt to answer a question. This could be anything from “How can we reduce infant mortality rates?” to “Why does salt make food taste good?”

To answer your question, you consult books, academic papers, newspaper articles, historical records, or anything else that could be helpful. The broad term for these things is “sources.”

And, usually, once you’ve done the research, you present or summarize it in some way. In many cases, this means writing an essay or another type of scholarly paper, but it could also mean giving a presentation or even creating a YouTube video.

Even if you have no interest in academia, research is an extremely useful skill to learn. When you know how to do research, it’s much easier to improve your life and work more effectively . Instead of having to ask someone every time you have a question, research will help you solve problems yourself (and help others in turn).

Note:  Research can also mean conducting surveys, performing experiments, or going on archaeological digs. While these activities are crucial for advancing human knowledge, I won’t be discussing them here. This article focuses on the research you can do with only a library and an internet connection.

The 7 Steps of the Research Process

Research can feel overwhelming, but it’s more manageable when you break it down into steps. In my experience, the research process has seven main steps:

• Find a topic
• Refine your topic
• Find key sources
• Take notes on your sources
• Create your paper or presentation
• Do additional research as necessary
• Cite your sources

Let’s look at each of these steps in more detail.

1. Find a Topic

If you don’t have a topic, your research will be undirected and inefficient. You’ll spend hours reading dozens of sources, all because you didn’t take a few minutes to develop a topic.

How do you come up with a topic? My number one suggestion is to create a mind map.

A mind map is a visual way to generate ideas. Here’s how it works:

• Get a piece of paper and a pen. Make sure the paper isn’t too small — you want lots of room for your ideas.
• Draw an oval in the center of the paper.
• Inside that oval, write a super vague topic. Start with whatever your professor has assigned you.
• Draw lines from the oval towards the edges of the paper.
• Draw smaller ovals connected to each of these lines.
• Inside the smaller ovals, write more specific ideas/topics related to the central one.
• Repeat until you’ve found 3-5 topic ideas.

When I write it out step by step, it sounds kind of strange. But trust me, it works . Anytime I’m stuck on a writing assignment, this method is my go-to. It’s basically magic.

To see what mind mapping looks like in practice, check out this clip:

Want to create a digital mind map like the one Thomas uses in the video? Check out Coggle .

2. Refine Your Topic

Okay, so now you have a list of 3-5 topics. They’re all still pretty general, and you need to narrow them down to one topic that you can research in depth.

To do this, spend 15 minutes doing some general research on each topic. Specifically, take each topic and plug it into your library’s catalog and database search tools.

The details of this process will vary from library to library. This is where consulting a librarian can be super helpful. They can show you how to use the tools I mentioned, as well as point you to some you probably don’t know about.

Furthermore, I suggest you ask your professor for recommendations. In some cases, they may even have created a resource page specifically for your assignment.

Once you’ve found out where to search, type in your topic. I like to use a mixture of the library catalog, a general academic database like EBSCO Host , and a search on Google Scholar .

What exactly are you trying to find? Basically, you’re trying to find a topic with a sufficient quantity and variety of sources.

Ideally, you want something with both journal articles and books, as this demonstrates that lots of scholars are seriously engaging with the topic.

Of course, in some cases (if the topic is very cutting edge, for example), you may be only able to find journal articles. That’s fine, so long as there are enough perspectives available.

Using this technique, you’ll be able to quickly eliminate some topics. Be ruthless. If you’re not finding anything after 15 minutes, move on. And don’t get attached to a topic.

Tip: If you find two topics with equal numbers of sources available, ask your professor to help you break the tie. They can give you insight into which topic is super common (and thus difficult to write about originally), as well as which they find more interesting.

Now that you have your topic, it’s time to narrow down your sources.

3. Find Key Sources

If you’ve picked a good topic, then you probably have lots of sources to work with. This is both a blessing and a curse. A variety of sources shows that there’s something worth saying about your topic, and it also gives you plenty of material to cite.

But this abundance can quickly turn into a nightmare in which you spend hours reading dense, mind-numbing material without getting any closer to actually producing a paper.

How do you keep this from happening? Choose 3–5  key sources and focus on them intently. Sure, you may end up needing more sources, especially if this is a long paper or if the professor requires it. But if you start out trying to read 15 sources, you’re likely to get overwhelmed and frustrated.

Focusing on a few key sources is powerful because it:

• Lets you engage deeply with each source.
• Gives you a variety of perspectives.
• Points you to further resources.
• Keeps you focused.

4. Read and Take Notes

But what do you do with these sources, exactly? You need to read them the right way . Follow these steps to effectively read academic books and articles:

Go through the article and look at the section headings. If any words or terms jump out at you, make note of them. Also, glance at the beginning sentences of each section and paragraph to get an overall idea of the author’s argument.

The goal here isn’t to comprehend deeply, but to prime your mind for effective reading .

Write down any questions you have after skimming the article, as well as any general questions you hope the article can answer. Always keep your topic in mind.

Read Actively

Now, start reading. But don’t just passively go through the information like you’re scrolling through Tumblr. Read with a pen or pencil in hand , underlining any unfamiliar terms or interesting ideas.

Make notes in the margins about other sources or concepts that come to mind. If you’re reading a library book, you can make notes on a separate piece of paper.

Once you’ve finished reading, take a short break. Have a cup of tea or coffee. Go for a walk around the library. Stretch. Just get your mind away from the research for a moment without resorting to distracting, low-density fun .

Now come back to the article and look at the things you underlined or noted. Gather these notes and transfer them to a program like Evernote .

If you need to look up a term, do that, and then add that definition to your notes. Also, make note of any sources the author cites that look helpful.

But what if I’m reading a book?   Won’t this take forever?  No, because you’re not going to read the entire book.

For most research you’ll do in college, reading a whole academic book is overkill . Just skim the table of contents and the book itself to find chapters or sections that look relevant.

Then, read each of those in the same way you would read an article. Also, be sure to glance at the book’s bibliography, which is a goldmine for finding additional sources.

Note: The above method is a variation on the classic SQ3R method , adapted slightly since we’re not interested in taking notes from textbooks .

5. Create Your Paper or Presentation

“You can’t turn in raw research.”

Research is crucial to crafting a great paper or presentation, but it’s also a great way to procrastinate. I had classmates in college who would spend 8 hours researching a 5-page paper. That’s way too much!

At some point, you need to stop researching and start writing (or whatever method you’re using to present your research).

How do you decide when to stop researching? There’s no strict rule, but in general I wouldn’t spend more than 30 minutes per page of the final paper.

So if the final paper is supposed to be 10 pages, don’t spend more than 5 hours researching it.

6. Do Additional Research (As Necessary)

Once you’ve started writing the draft of your paper, you’ll probably find a few gaps. Maybe you realize that one scholar’s argument isn’t relevant to your paper, or that you need more information for a particular section. In this case, you are free to return to researching as necessary.

But again, beware the trap of procrastination masquerading as productivity! Only do as much additional research as you need to answer your question. Don’t get pulled into rabbit holes or dragged off on tangents. Get in there, do your research, and get back to writing .

To keep yourself focused, I suggest keeping a separate document or piece of paper nearby to note points that need additional research.

Every time you encounter such a point, make note of it in the document and then keep writing. Only stop when you can’t get any further without additional research.

It’s much better to get a full draft done first. Otherwise, you risk suffering a cognitive switching penalty , making it harder to regain your focus.

7. Cite Your Sources

Whether you’re creating an oral presentation, essay, or video, you’ll need to cite your sources. Plagiarism is a serious offense, so don’t take any chances.

How to cite your sources depends on the subject and the professor’s expectations. Chicago, MLA, and APA are the most common citation formats to use in college, but there are thousands more.

Luckily, you don’t need to painstakingly type each of your citations by hand or slog through a style manual. Instead, you can use a tool like Zotero to track and generate your citations. To make things even easier, install the Zotero Connector browser extension. It can automatically pull citation information from entries in an online library catalog.

Once you’ve collected all of your sources, Zotero can generate a properly formatted works cited page or bibliography at just the click of a button.

For help setting up and using Zotero, read this guide . If you need further assistance, ask a librarian.

Go Research With Confidence

I hope you now understand how to do research with more confidence. If you follow the procedures I’ve covered in this article, you’ll waste less time, perform more effective research, and ultimately have the material for a winning essay.

Curious about how to use your research to write a great research paper? Check out this guide .

Image Credits: picking book from shelf

Conduct High Quality Online Research: Process, Types, Tools, Tips & More

If there’s one constant in modern life it’s this: research. No matter the topic, it’s imperative that most of us conduct thorough research for a variety of purposes online.

We research products and options when we want to buy something. We research markets and competitors when we want to sell something. We research topics and exes when we want to know or learn something.

We do research on the internet for so many different reasons, it can be hard to think about “online research” as one task—but if you add it all up, many of us spend a lot of time doing research on the internet. So there’s some serious value in understanding how to do that research more thoroughly, accurately, and quickly.

In this article we will cover:

• PROCESS:  The online research process
• METHODS:  Research methods and strategies
• TYPES:  Some of the most common types of research you can do online
• TIPS:  7 tips for better online research
• TOOLS:  Research tools and companies to improve and expedite the research process
• RESOURCES:  35 great internet research resources
• DELEGATING:  How you can delegate your research to a virtual assistant (VA)

The Online Research Process in 6 Steps

Broadly speaking, the typical online research project goes through 6 key steps. While you probably don’t tick off all these steps every time you research something online, following them can help ensure your research is complete, accurate, and useful.

Let’s talk about what those steps are and why each one is worthwhile for just about any online research you do.

1. Choose and define your topic of interest

This first step is where you’ll get specific about just what it is you’re looking for. What’s your end-goal? Why are you conducting this research? What are you hoping to learn or achieve?

For market research, this might be developing a full understanding of the competitors in the space and their positioning. For product research, you might be trying to arrive at the best option for you to buy.

The key is to make a comprehensive list of the research questions you want to answer and the individual items that interest you. This list will help inform where and how you do your research and ensure you don’t wind up with a bunch of information that doesn’t help or interest you.

2. Determine which fields of study you’ll need to look into

This step will help you define and narrow down the type of journals, databases, websites, etc. that you’ll look to for information.

For example, if you’re doing product research and you want to know how valuable existing customers find a given product, you may turn to prominent third-party review websites. If you’re doing medical research, you may look into the relevant medical journals for your topic.

3. See what research has been done and conclusions have been drawn

Step 3 is likely the part of the process you most often associate with “research.” Now’s the time to dig into your research sources, read up on the topic, and look to see how other people have answered the questions you laid out for your research.

The important part of this step is to stay organized and on-task. It’s easy to get lost in all the information, so it’s best to have a clear process and to keep your sources and learnings organized.

4. Evaluate your sources and information

In today’s digital world, this step is even more important than the rest. No matter the topic of your research, you need to take the time to understand and evaluate your sources . Who’s writing about the topic? Why are they interested or invested in it? Do they have anything to gain from what they’re saying?

This step is when you can identify any biases you or your sources have. Think of these biases as gaps in your research—and fill them in with opposing viewpoints and additional information. ‍

5. Determine additional research data collection methods needed and conduct

Whether as a result of biases or something else, it’s not uncommon to find gaps in the research that’s already been done. When that happens, you may consider conducting your own primary research to help fill in those holes in your information.

For example, if you’re missing qualitative market research, you may choose to conduct an online focus group of consumers in that market. For medical research, filling in the gaps might mean conducting an extensive clinical trial. For research into your own customers, on the other hand, it might be as simple as sending out a brief online survey asking for feedback. You can also use online survey platforms to reach a broader base.

6. Organize your full body of research and draw conclusions

Once steps 1 through 5 are finished, you’re ready to start digging into your body of research and drawing your conclusions. This is where you’ll make a final decision on which product to buy or identify where in the market to position your own business, for example.

Online Research Methods & Strategies

When you think about “online research,” what sort of research method do you imagine? Many of us likely think about Googling and reading articles—and that is one method for doing research online. But it isn’t the only one—far from it.

Below are some of the other common online resources for research methods and strategies you can draw on during your research.

Content analysis and social media or social network analysis

Content analysis is the typical web search and read method of conducting research. In this case, you’re consuming secondary research that’s already been conducted and learning from that.

Focus groups

A focus group is when you bring together a group of people to take part in a guided discussion—often this discussion is about their experience with a particular product, brand, political campaign, ad, or TV series/movie. You might picture these happening in-person, but they can also be conducted online using video chat or conferencing software.

Interviews are similar to focus groups—you’re asking real people for very specific information. The difference is that interviews are more often done one-on-one versus in a group. Interviews can also follow a less conversational and more transactional question-answer approach.

Questionnaires and surveys

Questionnaires and surveys share the question-and-answer approach of an interview, but they aren’t typically done live or in real-time. Surveys can be emailed or mailed out to respondents or shared on social media. The respondent completes the questionnaire on their own time and returns it to the researcher when finished.

Web-based experiments

Web-based experiments follow a more regimented and traditional set of processes designed to yield scientifically significant results. There are three main types of experiments:

• Controlled experiments
• Natural experiments
• Field experiments

While the topic varies, many of these experiments can be adapted to take place online.

Clinical trials

Clinical trials are a type of experiment most often done in medical and psychological research. In a clinical trial, the experiment is designed to answer a very specific set of questions. The classic example of a clinical trial is a drug or pharmaceutical trial—designed to answer whether a particular drug affects a given disease or injury.

Online ethnography 

In an ethnographic study, the researcher essentially lives among their research subjects and observes their behavior, social structures, and more. Ethnography is most commonly used in behavioral research like sociological and anthropological studies. Online ethnography simply refers to the method by which the researcher interacts with subjects—online.

Common Types of Online Research

Online research comes in all shapes and forms, but talking about “research” in the abstract can feel a little nebulous. To help you wrap your head around the kinds of online research we’re referring to for our purposes, here are some of the most common types of online research.

Basic Research

Basic research refers to broad studies and experiments done, not to answer a specific question or prove a hypothesis, but to create a foundation for additional studies or experiments.

For example, a study of how caffeine affects the brain would be considered basic research. Its results would increase general knowledge on the topic and likely inspire more specific experimentation.

Here’s another example of what basic research looks like—and how it can often blend into applied research:

• EXAMPLE: via Verywell Mind
• RESEARCH: To start, “researchers might conduct basic research on how stress levels impact students academically, emotionally, and socially.” That might involve content analysis of existing research on the topic, empirical research around students’ moods and performance, and interviews or surveys completed by the students themselves.
• FINDINGS: At the end of the basic research process, researchers have a better understanding of how stress impacts students—but they don’t know why stress has those effects or how to change or solve the effect.
• CONCLUSIONS: Because of that, “the results of these theoretical explorations might lead to further studies designed to solve specific problems. Researchers might initially observe that students with high stress levels are more prone to dropping out of college before graduating. As a result, scientists might then design research to determine what interventions might best lower these stress levels. Such studies would be examples of applied research.”

Quantitative Research

Quantitative research involves studying something using statistical or mathematical techniques and it’s used to understand how often a particular phenomenon occurs. The “quantitative” part of this type of research refers simply to numbers.

Here’s a common example of what quantitative research looks like in action:

• EXAMPLE: via QuestionPro
• RESEARCH: “If any organization would like to conduct a customer satisfaction (CSAT) survey, a customer satisfaction survey template can be used. Data can be collected by asking a net promoter score (NPS) question, matrix table questions, etc.”
• FINDINGS: The survey method above provides “data in the form of numbers that can be analyzed and worked upon.”
• CONCLUSIONS: “Through this survey, an organization can collect quantitative data and metrics on the goodwill of the brand or organization in the mind of the customer based on multiple parameters such as product quality, pricing, customer experience, etc.”

Qualitative Research

Qualitative research , on the flipside, focuses more on observations and non-numerical qualities. It’s used to answer questions about how and why phenomena occur, versus how often.

Here’s an example of what a typical qualitative research study looks like:

• RESEARCH: “A bookstore owner who is looking for ways to improve their sales and customer outreach. An online community of members who were loyal patrons of the bookstore were interviewed and related questions were asked and the questions were answered by them.” 
• FINDINGS: “At the end of the interview, it was realized that most of the books in the stores were suitable for adults and there were not enough options for children or teenagers.”
• CONCLUSIONS: “By conducting this qualitative research the bookstore owner realized the shortcomings and the feelings of readers. Through this research now the bookstore owner can keep books for different age categories and can improve his sales and customer outreach.”

Market Research and Competitive Research

Market research and competitive research refer to gathering information about a particular industry and the companies currently doing business in it. It often involves mapping out the positioning of competing companies or products and is usually done by the companies in the market (or those hoping to be).

Here’s what a typical market research study looks like:

• EXAMPLE: A software company is looking to launch a new product into an unfamiliar market.
• RESEARCH: They conduct research to figure out the features their product will need, what price will be competitive, and where in the market there’s an opportunity to serve an underserved segment of consumers. Research includes basic informational research about competitors, their products, and pricing, content analysis of industry publications, and focus groups with potential customers.
• FINDINGS: The company finds that a small but dedicated segment of consumers in the market have a particular need that isn’t being met by any of the current competitors in the space.
• CONCLUSIONS: They design their product to solve that specific issue and create marketing and advertising campaigns targeted toward only that small niche market.

Customer Research

Customer research is when a business seeks to learn more about their customers (or their competitors’ customers). Often, customer and consumer research are included in the overall market research process we mentioned above.

Here’s what a typical customer research study looks like:

• EXAMPLE: via Hotjar
• RESEARCH: A software company wanted to learn more about what their customers needed from their software, and how they could build a better product and customer experience. They used on-page surveys on their website and some observational research to dig deeper into their customers.
• FINDINGS: Based on their research, the company created in-depth customer personas that exemplified their 3 most common customers, who they are, and what challenges they face.
• CONCLUSIONS: Based on what the company learned about challenges faced by one particular customer segment, they improved a particular feature of the product to improve that customer’s experience. 

Other Common Types of Research

• Comparative research , done primarily in the social sciences, refers to studies that compare a given data set across different geographic locations or cultures. For example, a study may look at the differences in poverty between the U.S. and Canada.
• Medical research can make up a wide range of studies and experiments. The most obvious example is clinical drug trials, which are run to determine the efficacy and safety of new pharmaceuticals. But medical research can also involve observational studies to better understand new diseases and other basic research.
• Legal research most typically refers to two scenarios: 1) finding an answer to a particular legal question or decision that needs to be made and 2) looking for precedent to support a legal argument.
• Product research refers to research done by companies to better understand what their customers are looking for. It can be done during the ideation or new product development phase or to further improve an existing product.
• Empirical research data is collected by observation. In other words, it’s a record of someone’s experience, defined via the 5 senses. For example, an experiment done to figure out if listening to happy music improves subjects’ moods would be considered empirical research.
• Descriptive research is done with the intention of better understanding something. Customer and consumer research are often done in a descriptive way—describing customers and their attributes rather than trying to explain or quantify them.
• Experimental research refers to a more rigid research process than many other research types listed here. In experiential research, researchers follow the research method. They utilize strictly controlled experiments in which one variable is altered and the results either support or refute a specific hypothesis.
• Exploratory research is similar to basic research. It’s done with the goal of better understanding a given problem or phenomenon, and its findings typically inform further research to solve the problem.

Tips for Better, Faster Online Research

Whether you’re new to conducting research online or you’ve been doing it for years, there are always tips and tricks you can employ to streamline, strengthen, and refocus your research process. With that in mind, here are our top tips for conducting high-quality research online.

Know the Information You’re Looking For

With all of the information available on the internet, it’s really easy to get lost. Maybe you end up chasing down rabbit holes or trying to answer new questions every time they arise. Either way, you’re distracted from answering the original questions you set out to.

That’s why it’s so important to get clear about what those questions are, and hold yourself to researching those answers. This is what steps 1 and 2 in our online research process above are designed to help with.

Get Clear About Your Goal for Researching

While similar to the previous tip, defining your goal for research is more action-oriented. When you get answers to the questions outlined above, what will you do with them? All the questions you seek to answer with your online research should serve this overarching goal—helping you make a decision or choose your next course of action.

For example, your goal for travel research might be to choose and book a destination for your next family vacation. For competitive research, your goal may be to identify a niche audience to target within your industry.

Check the Abstract First

If you’re using scientific papers, medical studies, legal reviews, and other academic research, you know you’re in for some dense, lengthy reading. So before you commit to reading anything, check out the abstract first. If you don’t find anything compelling in the abstract, you can safely skip that paper.

Have a System and Stay Organized

As we mentioned before, the internet completely changes the stakes when it comes to research. There’s almost no limit to the amount of research you can do. That’s why it’s vital that you create a system for determining which information you’ll look at, plus how and where you’ll store it. Here are a few suggestions for staying organized:

• Create Google Drive folders to store PDFs and other documents
• Create a designated folder in your Bookmarks to store websites and URLs
• Use a reference management software (like Mendeley ) designed to help organize extensive research
• Delegate the organizing part to a virtual assistant (VA)

Get Started with a Virtual Assistant

Avoid analysis paralysis.

Online research can be incredibly valuable in helping you make informed decisions on a whole range of topics—but it is possible to take research too far, ending up with way more information than you can adequately process. Avoiding analysis paralysis is the only way to ensure your research makes your life easier, instead of the other way around.

Clearly outlining your goals and questions to answer is a good first step in avoiding analysis paralysis. The second part comes down to recognizing when you have enough information to make a decision. Once that happens, it’s usually time to set the research aside and act.

Evaluate Your Sources and Check Your Own Biases

In the time of #fakenews and corporation-funded scientific research, it’s more important than ever to evaluate your sources for online research. To start, just get in the habit of paying attention to who ran the study, wrote the paper, or created an article.

From there, you can look deeper into their objectivity (or lack thereof). Ask yourself whether the researcher has something to gain or lose from the information they’re sharing. Are they interpreting objective information through their own angle? Equally important: how current is the information presented?

In addition to evaluating the objectivity of your research sources, it’s even more important to identify and be aware of your own biases toward the subject matter.

Delegate Research to a Virtual Assistant

Whether you lack the time, expertise, or just desire to conduct thorough online research, there are many reasons to delegate your research to someone else. Online research, in particular, can easily be handled by a virtual assistant ( more on that later! )

Research Tools and Resources to Help with Your Online Research

When the time comes to dive into online research, most of us default to starting with an internet search on Google, followed by trying different search terms and combing through endless search result listings. That’s a fine place to start, but there are also tons of other reputable databases and search engines that can help you get straight to the most accurate and up-to-date research on just about any topic.

Below, we recommend 13 tools that can help you find reputable sources, organize your research, and even conduct your own primary research.

For General Research Articles

• Google Scholar and Google Books
• Library of Congress and LexisNexis
• Project Gutenberg
• Student’s Online Research Guide via AllConnect
• Yale University Research Guides by Subject

Academic Journals

• AcademicJournals.org

For Specialized Research

• Medical: BioMed Central , The Lancet , New England Journal of Medicine , NCBI (Nat’l Center for Biotechnology News)
• Legal: American Law Reports
• Business and industry: Nielsen and Pew Research Center

Online Research Management and Organization

Online research companies.

• 20|20 Research
• Facts ‘n Figures

Virtual Research Assistant Companies

Other great online resources for research.

If you’re looking for more info on various aspects of researching online, here are a few more top-notch resources you can reference.

• For psychological, sociological, and other behavior research: Psychology.org
• For business (market, competitive, and new product) research: QuestionPro
• For market research: Inc.
• To better understand online research and “big data:” Online Research Methods, Quantitative by Hocevar and Flanagin
• On conducting your own survey research: SurveyMonkey
• For legal, news, and public records: LexisNexis online library

Delegating Research to a Virtual Assistant

The advice and resources above are enough to turn anyone into a pro online researcher—but do you really have the time or desire to do your own research online? Regardless of how well done, effective internet research always requires one big investment: time . There’s no getting around the time investment it takes to conduct valuable online research.

Instead of investing that time out of your own busy schedule, you could outsource your online research efforts to a virtual assistant (VA). That way, you get the benefit of making informed decisions without spending days or even weeks wading through abstracts and research articles.

When you work with a Delegated virtual research assistant:

• You can hand-off basic research, competitor and market research, comparative research and more from day one
• You can work with your VA and train them to handle more specialized types of research like medical and legal

In both cases, as your VA gains experience working with you, they’ll get better and better at pulling together exactly the kind of research and insights you’re looking for. Some aspects of research they can tackle include:

• Pulling together research articles and data
• Research annotation and summaries
• Research management and organization
• Various aspects of conducting primary research

How does this work?

We know that delegating something as broad and nebulous as “research” can feel a little foreign if you haven’t outsourced it before. Most of the concerns we hear from people are very quickly quelled by the time savings that come with delegating their research.

That said, if you’re feeling unsure, here are a few of the questions we hear frequently:

How does all this work?

Your Delegated VA is available to you whenever you need them. They can pull together research articles and sources, organize and annotate them, present research summaries and conclusions, and help with many of the tasks involved with conducting your own primary research.

What kind of research can a Delegated VA handle?

Delegated VAs can handle these types of research right off the bat:

• Basic research
• Market research
• Competitive research
• Comparative research
• Data research
• Information research

That said, with a little guidance and training from you, our VAs can take over just about any kind of research you need done.

How will my VA know what information to look for?

Initially, your VA will base this judgment on the information you provide to them. Any information you ask for, they’ll pull together for you. For basic research, they’ll be able to handle most anything you need.

For more specialized research areas (like medical and legal research), your VA may need a little more help from you in the beginning. Rest assured, after a few projects, they’ll be able to handle just about everything you can throw at them.

Can my VA handle next steps after research is done?

If you provide your Delegated VA with the access and information they need to take the next step, they can do that—whether that’s booking a trip based on travel research, purchasing their recommended product, or something else entirely.

How will my VA communicate with me?

Your Delegated VA will communicate with you any way you prefer. If you choose to communicate via Slack, email, phone, or morse code, your VA will work with your preferences to streamline communication.

Wrapping Up

Whether it’s product research, medical research, or something else entirely, conducting thorough and accurate research online takes time—and going without isn’t a great option either.

If you don’t have the time, desire, or expertise to perform your own internet research, you can easily turn the keys over to a virtual assistant. With a little guidance, they can handle a lot more than you may think.

Then, you can spend less time Googling around and more time acting on your research findings.

Glossary of Online Research Terms to Know

Research problem and research question: The central question your research sets out to answer, or the central problem your research sets out to solve.

Correlation: A connection or relationship between two variables.

Causation: A connection or relationship between two variables where a change in one variable creates a change in the other.

Findings: The results and conclusions of your research.

Scientific method: An empirical, step-by-step method whereby hypotheses are formed and experiments/observations either affirm or disprove the hypothesis.

Sampling method: A method for collecting data from a small sample of a given population.

Research methodology: The specific techniques and procedures you use to identify and analyze information about your topic.

Control group: A group within an experiment to which no changes are made in the variable being studied. Control groups are used for comparison to better identify how changes in a variable affect the other group.

Experimental group: The group within an experiment which is changed or manipulated.

Primary research: Data collected directly by you, the researcher.

Secondary research: Data previously collected by other researchers.

Hypothesis: An educated guess or theory about how an experiment will turn out.

Abstract: A brief summary of the contents of a research paper or study.

Bias: Assumptions made without credible evidence, often that skew the ultimate outcome of a study. Bias can be caused by beliefs held by the researcher or by errors in sampling or data analysis.

Kiera's a content writer who works with SaaS and ecommerce companies. Located in Boston, MA, she loves cinnamon coffee and a good baseball game. Catch up with her @Kieraabbamonte or KieraAbbamonte.com.

Start growing your business and reducing overhead with Delegated

1. meet your assistant
, 2. create a workflow
, 3. start delegating
, other resources.

Guide to Time Management Skills

Guide to Email Management

Guide to Scheduling Like a Pro

Loved by top companies and entrepreneurs.

Specialized Tasks

• Open access
• Published: 23 March 2024

Technology, data, people, and partnerships in addressing unmet social needs within Medicaid Managed Care

• Rachel Hogg-Graham 1 ,
• Allison M. Scott 2 ,
• Emily R. Clear 1 ,
• Elizabeth N. Riley 1 &
• Teresa M. Waters 3  

BMC Health Services Research volume  24 , Article number:  368 ( 2024 ) Cite this article

183 Accesses

Metrics details

Individuals with unmet social needs experience adverse health outcomes and are subject to greater inequities in health and social outcomes. Given the high prevalence of unmet needs among Medicaid enrollees, many Medicaid managed care organizations (MCOs) are now screening enrollees for unmet social needs and connecting them to community-based organizations (CBOs) with knowledge and resources to address identified needs. The use of screening and referral technology and data sharing are often considered key components in programs integrating health and social services. Despite this emphasis on technology and data collection, research suggests substantial barriers exist in operationalizing effective systems.

We used qualitative methods to examine cross-sector perspectives on the use of data and technology to facilitate MCO and CBO partnerships in Kentucky, a state with high Medicaid enrollment, to address enrollee social needs. We recruited participants through targeted sampling, and conducted 46 in-depth interviews with 26 representatives from all six Kentucky MCOs and 20 CBO leaders. Qualitative descriptive analysis, an inductive approach, was used to identify salient themes.

We found that MCOs and CBOs have differing levels of need for data, varying incentives for collecting and sharing data, and differing valuations of what data can or should do. Four themes emerged from interviewees’ descriptions of how they use data, including 1) to screen for patient needs, 2) to case manage, 3) to evaluate the effectiveness of programs, and 4) to partner with each other. Underlying these data use themes were areas of alignment between MCOs/CBOs, areas of incongruence, and areas of tension (both practical and ideological). The inability to interface with community partners for data privacy and ownership concerns contributes to division. Our findings suggest a disconnect between MCOs and CBOs regarding terms of their technology interfacing despite their shared mission of meeting the unmet social needs of enrollees.

Conclusions

While data and technology can be used to identify enrollee needs and determine the most critical need, it is not sufficient in resolving challenges. People and relationships across sectors are vital in connecting enrollees with the community resources to resolve unmet needs.

Peer Review reports

Introduction

Individuals with unmet social needs, like food and housing insecurity and transportation challenges, experience higher rates of adverse health outcomes [ 1 , 2 , 3 , 4 , 5 , 6 , 7 ] and are subject to greater inequities in health and social outcomes [ 8 ]. Unmet social needs are especially prevalent among Medicaid enrollees [ 9 ]. For this reason, state Medicaid programs are particularly interested in testing strategies that encourage and incentivize Medicaid managed care organizations (MCOs) to identify and address the complex social needs of enrollees [ 10 , 11 ]. Many Medicaid MCOs are now screening enrollees for their unmet social needs and connecting them to community-based organizations (CBOs) better equipped with knowledge and resources to address these needs [ 12 , 13 ].

The use of screening and referral technology and data sharing are often considered key components in programs integrating health and social services to address social needs [ 12 , 14 ]. Data sharing infrastructure has been highlighted as a way to streamline coordination and social need resolution [ 12 , 14 ]. In some instances, successful integration has facilitated strong connections between health and social services organizations, ensuring that patients move efficiently between sectors [ 14 , 15 , 16 ]. Despite this emphasis on technology and data collection and some positive integration, research suggests substantial barriers exist in operationalizing effective systems [ 12 , 17 ]. CBOs often have limited resources, financial and personnel, to put toward the use of advanced social need screening and referral systems [ 12 , 17 , 18 , 19 ]. The reliance on grant funding and other time-limited resource streams likely presents another barrier in the adoption of tools [ 17 ]. CBOs can also be hesitant to adopt technology and data systems owned by MCOs, hospitals, and other clinically oriented organizations because of data privacy and HIPAA-related issues [ 16 , 20 ].

Research examining health and community partnerships has identified technology adoption by CBOs and other social services organizations as an important barrier to collaboration [ 14 , 15 , 17 ]. Most prior studies examining data and technology include clinical organization perspectives on the use of tools but do not include robust information from community partners [ 12 , 14 , 16 ]. Further, those studies that do include perspectives from multiple organization types on the integration of health and social services are not focused on adopting screening and referral systems. Technology typically emerges in subthemes, and the evidence included does not provide in-depth information on benefits and challenges from both community and clinical partners [ 17 ].

This study examines CBO and MCO perspectives on the use of technology in social need screening and referral. The qualitative analysis presented here is part of a larger mixed methods study examining how Kentucky (KY) MCOs address unmet social needs in partnership with community organizations [ 21 ]. KY offers a unique opportunity to examine strategies addressing Medicaid enrollee needs. Just under 29% of all KY residents are enrolled in Medicaid, making it the third highest enrollment among US states [ 22 ]. KY is also geographically diverse, with distinct urban, rural, and Appalachian regions.

Setting and study population

A project Stakeholder Advisory Board (SAB), including representatives from all Medicaid MCOs, academia, a community-based organization, the State Department for Medicaid Services, and enrollees, met quarterly to provide expertise, guide research, and assist with the dissemination of study results. MCO representatives serving on our SAB were asked to 1) identify individuals in their organization leading efforts to address unmet social needs and population health outcomes among their enrollees and 2) identify CBOs they work closely with in their social need referral process. As part of a targeted sampling strategy, identified contacts were invited via email by the research team to participate in key informant interviews to discuss how MCOs and CBOs address social needs. Inclusion criteria were that participants were at least 18 years old, were employed at an MCO/CBO in Kentucky, and were willing to engage in an interview in English. A total of 32 MCO contacts were invited and 33 CBOs, giving us response rates of 81% and 58% respectively.

Participants

Our sample of 46 participants comprised 26 representatives from 6 MCOs (ranging from 3 to 6 participants per MCO) and 20 representatives from 19 unique CBOs. MCO participants represented various organizational roles, including vice presidents, directors, population health, case management, and community engagement. CBO participants represented roles including directors, Chief Executive Officers, Chief Operating Officers, Medical Coordinators, Presidents, Chief Engagement officers, program managers, and outreach coordinators. The services provided by community-based organizations included food security, health, housing, employment, and work readiness, refugee and immigrant services, and community support; many CBOs addressed multiple social needs. CBO interviewees represented organizations operating in both urban and rural areas of the state.

Data collection

In-depth one-on-one interviews with 46 stakeholders from identified CBOs ( n  = 20) and MCOs ( n  = 26) were conducted between May 24, 2021, and November 8, 2021. Interviews were conducted via Zoom, audio-recorded, and transcribed verbatim. The qualitative researcher and facilitator conducting these interviews have extensive training and experience with structural interviewing using a semi-structured interview guide. The guide used was developed for this study [ 23 ].

Data analysis

We conducted an iterative content analysis of the transcribed interview data using qualitative descriptive analysis [ 24 ], an inductive, low-inference method designed to gain an accurate understanding of a phenomenon in the everyday terms of stakeholders. Our data analysis unfolded in two stages. The first stage involved open coding [ 25 ], in which the transcripts were independently coded by two authors and one study team member (AM, ER, and HS), who then met to discuss and reach consensus on the central themes in the data related to technology and data sharing. In this meeting, the authors identified the themes of to screen for patient needs, to case manage, to evaluate the effectiveness of programs, and to partner with each other. The second stage of analysis involved focused coding, with the three individuals again independently coding transcripts for subthemes within each identified central theme. The coders met again to compare findings and finalize themes (and subthemes for Theme 4). At this time, we recognized that there were areas of alignment, incongruence, and tension between the responses of participants from MCOs and CBOs, and we reached agreement in this meeting about which themes demonstrated each dynamic. Finally, all authors met a third time to review the subthemes and select illustrative quotations for each. All analytic decisions were made through discussion until consensus was reached. We used the team-based approach to reaching consensus, which considered dependability and trustworthiness of the data [ 26 ]. This paper focuses on responses addressing technology platforms and data sharing to support MCO and CBO partnerships.

We identified several themes related to the use of technology and data in MCO-CBO partnerships to address enrollee social needs. MCOs and CBOs noted differing levels of need for data, differing incentives for collecting and sharing data, and differing valuations of what data can or should do. MCO and CBO interviewees described how they collect and use data in their work, which fell into four major themes: to screen for patient needs, to case manage, to evaluate the effectiveness of programs, and to partner with each other. Within these themes, the interview responses illuminated areas of alignment between MCOs/CBOs, incongruence, and tension (both practical and ideological; see Table  1 ).

Theme 1. Alignment on collecting data to identify and prioritize patient needs

Using data to identify and prioritize patient needs was largely an area of alignment for MCOs and CBOs. All MCOs and nearly all CBOs recognized the value of data in this area. As one CBO noted,

“By completing the needs assessment with our families, it helps the case managers understand your immediate needs.”

Similarly, MCOs often used the data for targeted programming and social needs referrals,

“ When our members are enrolled, we attempt to engage them in our health risk assessment. And so that health risk assessment is going to not only ask them questions about their specific health, but also about some additional needs that would help us be able to identify them at enrollment and also to be able to target them for programs and other [benefits].”

Several MCO and CBO interviewees also discussed using the data to understand individual enrollee/client needs and to track overall trends among their clients. As one MCO shared,

“The end of 2021, we had a tremendous amount of referrals for food. And so maybe we need to look at doing some of our community investment work and partnering with additional providers and community partners that are in that space for next year.”

There were some differences between MCOs and CBOs in the formality and degree to which social need data was collected. MCO interviewees, particularly those on the front lines of this work, could describe detailed and comprehensive data screening metrics for patient needs and how needs were tracked in their data systems. Using data on patient needs to identify areas for intervention was described as an essential part of patient care:

“We use the screening data, not just to meet the individual member need, but to also inform health equity and types of programs that we bring to play...”

CBO interviewees, on the other hand, had greater variability in their responses about the importance of using data on social needs at an organizational-level. Most described data as having potential value but stopped short of calling it essential for their operations. One CBO stated,

“I don't know what I would do with the information if we had it.”

Conversely, one food-oriented CBO reported that they collect demographic data and use that to help with distribution,

“So think about the local pantry that I talked about earlier. Because we know, we drive a truck into [KY County]. We know that the last five times that we've been in [KY County], we saw, on average, 150 households at each of those five visits. That tells us how much product to put on the truck so that we don't run out.”

Theme 2. Differences in organizational capacity, mission, and resources influenced variability in data use to support case management

Using data to support case management activities was an area of both alignment and incongruence between MCOs and CBOs. All MCOs and many CBOs saw value in using data systems to identify resources available, track referrals and follow-ups, keep notes, and stay in contact with patients. However, there was considerable variability in the sophistication of the data systems. Most MCOs reported elaborate data tracking systems designed specifically for screening, referral, and tracking (e.g., combining medical records applications with Unite Us [ 27 ] or Find Help (formerly Aunt Bertha [ 28 ]). Some CBOs have systems designed specifically for tracking data (e.g., Electronic Health Systems or Vesta [ 29 ]), whereas others employ systems not designed specifically for tracking (e.g., Microsoft Excel spreadsheets). Most CBOs used informal data collection to screen for needs (e.g., Post-it notes, memory, a hand-written planner), and several CBOs reported that they did not use formal data systems to screen and track patient needs at all,

“Are you kidding me? No books. What I usually tell anybody who's working with me is to either email me or text me, and that's my filing system.”

MCO interviewees were more likely to report using data analytics to support and enhance case management. Frontline MCO workers spoke about this aspect of data use more often than executives, and many saw data systems as the answer to case management problems. As one MCO stated,

“We do have a case management system that keeps track. So, we are able to schedule calls. They're able to pop back up on a calling queue, so that we're able to check in with members and attempt to continuously reach out to them. So, that's kind of how we try to make sure that those members don't fall through the cracks by continuously following up.”

Most CBOs indicated that case management occurred but was more personalized and less attached to data and technology use,

“We have a database that we use for client notes. We just record case notes in there. Some of our caseworkers keep basic Excel spreadsheets on their specific clients and what they're working on. Most of that would be informal.”

Only one MCO specifically mentioned the limits of data systems for tracking and the need for a personal touch in case management, a perspective more in line with most CBO interviewees. The MCO shared this when discussing platform capabilities, stating,

“We have a case management platform, of course, where we document everything, because just like everywhere else, if you don't write it down, it didn't happen, but a lot of it is just that manual follow-up and that human touch.”

The variability in tracking system sophistication and capabilities between MCOs and CBOs was also frequently highlighted as one of the critical challenges in collaboration and a notable source of frustration for both sides. When discussing their partnerships with MCOs and data sharing, one CBO stated,

“They really wanted to know about it. And so had to spend considerable time with them about, ‘This is what we do, this is how stuff works.’ And including it's like, ‘No, we can't track. We have no way of tracking [MCO] clientele through the [KY food security] program’."

While MCO interviewees often noted this tension in collaboration, they were aware that capacity and resources typically made it harder for CBOs to track and collect data. One MCO interviewee noted,

“I think the challenge is just the data piece and the complexity of the regulations that we have to navigate, all for good reason. When you're talking about how to best leverage those community resources, if we can't kind of have those data exchanges, it makes it so much more difficult. And so when you're trying to get at outcomes or have simplified referral processes, it just makes it harder because you may not be able to get through, they may not have the HIPAA, the high-tech clearance or whatever it is. It's expensive for them to have to do that.”

Theme 3. Funding and reimbursement structures shaped how MCOs and CBOs used data to evaluate program effectiveness

We found limited alignment between MCO and CBO perspectives on using data to evaluate social need programming and partnerships. Instead, evaluation was an area fraught with incongruencies and tension between the two sectors. The financial incentives and pressures for using data differ substantially between MCOs and CBOs. MCOs reported using data to evaluate the financial impact or effectiveness of programs (particularly claims data/utilization metrics) and partnerships to justify investments or show MCO executives that meeting unmet social needs is good business. As one MCO interviewee explained,

“I think every anything that we’re doing with the community-based partner, we’re studying all that. We’re studying the reduction, so I’m able to say, okay, because we have this member in this [CBO program], in this residential treatment program, not only mama’s healthier, baby is not born exposed to opiates, no NICU, ER utilization down. I think that’s the neat thing, there’s your answer, right?”

One reason MCOs seem to be driving data collection for demonstrated effectiveness/return on investment is that they are heavily regulated in terms of how they can invest funds,

“We are doing payment innovation, we want to take money out of what’s being spent on health care and invest it into social services and that is not easy.”

As another MCO highlighted continued investment often depends on what they can demonstrate,

“Sometimes, there are finance guidelines, right? Like when I’m fighting for my budget, they’ll say, ‘Well, where’s the return on investment numbers?’.”

Conversely, only a few CBOs used data-driven evaluation to support their financial operations. When CBOs did report using data for evaluation, it was typically in relation to using outcomes data in grant writing to gain funding specifically from MCOs, data which may not serve any other useful purpose for the CBO. As one CBO stated,

“Another kind of pain point, and for like one of the managed care companies that we contract with, they give us $8,000 a year. But the requirements to receive that $8,000 is very data heavy. We have to go through and pull all this data, get different releases signed with the participants. It’s great to have extra money, but it’s also a lot of work and nothing really being tied to it, if that makes sense. They just want the data to be able to review and any good outcomes and success stories and stuff like that, which is great. But it’s a lot of work for not a lot of money.”

Theme 4. Tension in using data to partner with other MCOs and CBOs

Both MCO and CBO interviewees described several reasons why they engage in data sharing within MCO-CBO partnerships (e.g., to garner funding, demonstrate effectiveness, or enhance case management), even if the values and importance placed on data sharing differed between agency types. When data sharing existed or was being contemplated, interviewees still described several barriers to sharing, both practical and ideological.

Overwhelmingly, CBO interviewees expressed a perception that they had to report data to the MCOs to prove impact so MCOs would maintain the partnership or provide funding. The first subtheme revealed a notable ideological difference between the MCOs/CBOs regarding whether data was useful to evaluate program effectiveness . While data-driven evaluation is routine and relied upon by most MCOs, many CBO interviewees perceived that data and metrics could harm their operations, diverting time and energy from serving clients and that there is much about program effectiveness that simply cannot be captured using formal data tracking systems. When discussing the course of their partnerships with MCOs, one CBO highlighted,

“So what does that support look like? Well, it is financial support for it. And, initially, it was very much focused on their clientele with [MCO] clientele and trying to track metrics about the impact that having access to better nutrition was going to have on the outcomes for their folks, right? So over the course of two years, I mean, we were able to show, "we," and I mean that collectively, we're able to show that it does have a positive impact. I mean, for [MCO], I think it's safe to say that they realize that it is more cost-effective to invest upfront in increasing access to healthy food better than the back end, to drugs and health care costs and all that kind of stuff. So they have, again, they have maintained that partnership.”

Indeed, most MCOs expressed wanting data from their CBO partners to justify the relationship and a reluctance to build relationships if data capacity is not present. One MCO discussed this directly, stating,

“They come us and they send us their flyer and they're like, "We want [MCO] to partner with us on our heart walk and we want you to give us $20,000." We still get a lot of people that do that because that's their old business model. Most of the time, we don't engage with those types of organizations. I always say, we want to hear from someone and I will take a meeting always if a community-based organization says, "We have an evidence-based solution that is solving for X," or "We have a solution that is solving for X and we want to work with you to help us prove that it's evidence-based," or we have research capabilities...”

Subtheme 2 illustrates how underlying the data sharing tension between CBOs and MCOs are challenges related to the need for more effective and user-friendly interfacing between tracking and referral systems, as well as the limited capacity of CBOs to track and analyze data . As mentioned, the sophistication of CBO data systems is highly variable, and even those organizations with more advanced tracking systems struggle with data sharing. When asked about data sharing, one CBO noted,

“Well that's another pain point. In my history, in my experience, every health plan has their own data system that don't talk to one another, that are very convoluted and messy. Right now we're filling stuff in on an Excel spreadsheet.”

Several MCOs also highlighted this as a challenge. As one MCO stated,

“Our system is designed to deal with hospital systems and health care providers, there's many different levels. I mean we go through a pretty comprehensive system and you have to have all kinds of, meet all kinds of requirements, share data, and different pieces that for a small community-based organization providing housing services, they might not even have the capacity to meet those requirements.”

Although some CBOs reported sharing data with MCOs willingly and saw this sharing as a natural facet of their partnership, other CBOs described significant concerns about data privacy and ownership ( subtheme 3 ). They noted how important data privacy was to the clients they served and how their organization valued serving their clients without the need to collect personal data or share it. Some CBO interviewees indicated that sharing or even collecting private client data might compromise their ability to do their work and serve their clients well,

“We respect their privacy, and we will never do any sharing of their data. In fact, a lot of people who come to us, one of the reasons they're with us is because we do not require them to show an ID.”

Subtheme 4 revealed how CBO and MCO interviewees expressed concerns about relying on data and technology as the solution to social need screening and referral systems building . Interviewees felt that data does not adequately capture utilization or partnership benefits. Primarily, this was attributed to issues related to data quality. One MCO interviewee highlighted this when discussing the challenges of understanding the quality of social need services:

“We also don't have a really long track record of managing quality for this type of provider. We have very distinct report cards and quality cards for every hospital in the state of Kentucky. I can tell you what the outcomes for [Hospital 1] compared to [Hospital 2] and compared to [Hospital 3]. We have very clear metrics on those types of things. We do not have that for the sort of soft services, especially since we don't pay for them.”

Most CBOs articulated challenges with data quality centered on their perception that data does not tell the whole story about what is happening at their organization and in the community. As one CBO noted,

“ We have a people problem. And I think right now there are a lot of hospitals and other organizations, MCOs, that want to kind of tech their way out of this. [T]hey're looking for technological [solutions] to try to streamline and expand services to folks. And that's just not really the answer. You need people.”

MCO interviewees recognized that databases and their tracking systems may be limited in what they capture. In subtheme 5 , several noted their technological ability to comprehensively track organizations in a community as a significant limitation . Maintaining accurate data has also been challenging because of community organization turnover and closures. As one MCO highlighted,

“These national repositories don't have the local knowledge so they don't know the churches that do the hot meals and they don't know the small organizations that are getting up and off their feet and tied to this one or that one, or it's an offshoot of whatever. There are some smaller organizations that don't always get into those big directories and you don't always know about them unless you have boots on the ground, people who live and work in the community and actually know what those are.”

Similarly, another MCO highlighted CBO data capacity as a major challenge in their partnerships, stating,

“Biggest challenges. I guess, you could say data might be the challenges, to close the loop around the return on investment on some of these organizations that are not ... They just don't have the staffing, or the professional leadership, if you will, to do all the tracking. The ones that do, do it very well. The ones that don't, it's just that they don't have the resources.”

In the final subtheme, all MCO interviewees acknowledged that CBOs are doing good work , even if that cannot be quantified, and the ability to share that data is often related to CBO capacity and resources. One MCO shared,

“[Food Pantry CBO] who's just like [Named Female] and her husband [Named Male], they might be the greatest people and we might know that members like going there versus the other food bank because [Named Female] like bakes brownies and gives them a hug and we want to quantify that but also it's just not realistic because they don't have the infrastructure sometimes that's needed to prove the business case, solidify the partnership and ultimately inform policy.”

Our study found alignment as well as discordance between MCOs and CBOs about how and when to leverage technology and data despite their shared mission to meet the unmet social needs of enrollees. Our findings offer important insights regarding why data and technology may create a barrier to effective MCO-CBO partnerships, potentially hindering efforts to improve health and social outcomes. They also provide guidance and identify key considerations for developing programs and partnerships that may be more effective in coordinating efforts between the two organizations.

As we observed in Themes 1 (Alignment on collecting data to identify and prioritize patient needs) and 2 (Differences in organizational capacity, mission, and resources influenced variability in data use to support case management), results suggest that data and technology can be important tools in screening and referral for social needs, but they are far from a universal panacea. Our data indicate that both logistical and cultural disconnects between MCOs and CBOs significantly limit data collection and sharing for coordination of services. On the logistical side, CBOs have extremely limited capacity (software, workforce) to collect and share data. Several participants reported serious concerns with collecting and sharing confidential client information. To make matters worse, MCOs use a range of proprietary and sophisticated referral and tracking systems that severely tax the resources and capacity of CBOs. On the cultural side, while MCOs view data and technology as essential to partnering with CBOs to meet enrollee social needs, CBOs do not. In fact, as we found in Theme 3 (Funding and reimbursement structures shaped how MCOs and CBOs used data to evaluate program effectiveness), many CBOs see data collection as a necessary evil to garner funding from potential donors. Instead, they emphasize the relationship-honoring aspects of their work as a core value.

Solutions that only focus on providing data collection and tracking technology to CBOs are unlikely to be completely successful because they fail to address the disparate cultures found in MCOs vs. CBOs. This conclusion is robustly supported by Theme 4 from our analysis (Tension in using data to partner with other MCOs and CBOs).In many ways, CBOs may view MCO efforts to grow their technological capacity as imposing profit-seeking values, norms, and structure rather than seeking true understanding and partnership. CBOs’ low enthusiasm for and capacity to use data can create difficulty for MCOs when MCOs rely on CBOs for data to justify their funding streams and partnerships. This fundamental disconnect is likely to severely impede partnership efforts without reevaluating the strengths and values each sector brings to the collaborative [ 30 ].

Successful partnerships are built on shared interest and trust [ 31 ]. Our study suggests a strong alignment between MCOs and CBOs in addressing the social needs of highly vulnerable Medicaid beneficiaries. This values alignment may offer a foundation for partnership. Our work underscores a key finding across studies on cross-sector partnerships integrating health and social services, more work must be done to build trust and understand each other’s organizational values [ 17 , 19 , 32 ]. MCOs and CBOs need each other to address social determinants of health (SDOH) effectively. MCOs have the resources and responsibility for finding more effective ways to support their beneficiaries. CBOs are ‘on the ground’ and have the trust of the clients they serve (many of whom are Medicaid enrollees). Forums that create a level playing field for both types of organizations and facilitate safe conversations to build trust are essential.

The Department of Health and Human Services (DHHS) has developed a three-pronged strategy for addressing SDOH: (1) better data, (2) improving health and social services connections, and (3) whole-of-government collaborations [ 8 ]. Our study suggests that their second strategy is essential and could be far more difficult than many imagine. Facilitating honest conversations about identifying and addressing the challenges in building these connections is a critical first step. Because many challenges involve “hearts and minds” and organizational culture, addressing these challenges will need to be a slow and iterative process. Moving forward, organizations like MCOs and other clinical partners must carefully consider how data and social need screening and referral technology can be a value-add to CBOs and not another burden on their already strained capacity.

Limitations

While our sample included at least one representative from all six state MCOs and nineteen different CBOs, the generalizability of study results may not apply to other states. However, many of the MCOs in KY operate in national markets and often use similar strategies in different geographic areas. Insights likely shed light on similar efforts and challenges in other states and markets. Future studies examining the use of data and technology nationally in social need resolutions would provide confirmation of the results we present and any potential geographic variability. Additionally, participant perspectives may not necessarily represent their MCOs or CBOs. Finally, our cross-sectional view of technology and referral platforms provides a snapshot of current processes; a more in-depth longitudinal study would capture changes over time as technology constantly evolves.

Despite a shared mission to meet unmet social needs, MCOs and CBOs do not agree on how and when to leverage technology and data. This discordance is a significant barrier to effective partnerships. Technology offers powerful tools for identifying and prioritizing enrollee needs and connecting them with services. However, trust and a shared understanding of organizational cultures and goals are critically needed to allow technology to realize its potential. Current efforts to build effective MCO-CBO partnerships should focus on creating a level playing field for all organizations and a space for honest conversations that can build strong connections and sustainable relationships across sectors.

Availability of data and materials

Deidentified aggregated data is available from the corresponding author ([email protected]) on reasonable request.

Gottlieb L, Tobey R, Cantor J, Hessler D, Adler NE. Integrating social and medical data to improve population health: opportunities and barriers. Health Aff. 2016;35(11):2116–23.

Article   Google Scholar  

Seligman HK, Laraia BA, Kushel MB. Food insecurity is associated with chronic disease among low-income NHANES participants. J Nutr. 2010;140(2):304–10.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Silverman J, Krieger J, Kiefer M, Hebert P, Robinson J, Nelson K. The relationship between food insecurity and depression, diabetes distress and medication adherence among low-income patients with poorly-controlled diabetes. J Gen Intern Med. 2015;30:1476–80.

Article   PubMed   PubMed Central   Google Scholar  

Berkowitz SA, Hulberg AC, Hong C, Stowell BJ, Tirozzi KJ, Traore CY, Atlas SJ. Addressing basic resource needs to improve primary care quality: a community collaboration programme. BMJ Qual Saf. 2016;25(3):164–72.

Article   PubMed   Google Scholar  

Cole MB, Nguyen KH. Unmet social needs among low-income adults in the United States: Associations with health care access and quality. Health Serv Res. 2020;55:873–82.

Fiori KP, Heller CG, Rehm CD, Parsons A, Flattau A, Braganza S, Lue K, Lauria M, Racine A. Unmet social needs and no-show visits in primary care in a US northeastern urban health system, 2018–2019. Am J Public Health. 2020;110(S2):S242–50.

Alley DE, Asomugha CN, Conway PH, Sanghavi DM. Accountable health communities—addressing social needs through Medicare and Medicaid. N Engl J Med. 2016;374(1):8–11.

Article   CAS   PubMed   Google Scholar  

De Lew N, Sommers BD. Addressing social determinants of health in federal programs. InJAMA Health Forum 2022;3(3):e221064-e221064). American Medical Association.

Thompson T, McQueen A, Croston M, Luke A, Caito N, Quinn K, Funaro J, Kreuter MW. Social needs and health-related outcomes among Medicaid beneficiaries. Health Educ Behav. 2019;46(3):436–44.

Moreno-Camacho CA, Montoya-Torres JR, Jaegler A, Gondran N. Sustainability metrics for real case applications of the supply chain network design problem: A systematic literature review. J Clean Prod. 2019;10(231):600–18.

Apenteng BA, Kimsey L, Opoku ST, Owens C, Peden AH, Mase WA. Addressing the social needs of Medicaid enrollees through managed care: lessons and promising practices from the field. Popul Health Manag. 2022;25(1):119–25.

Cartier Y, Fichtenberg C, Gottlieb LM. Implementing Community Resource Referral Technology: Facilitators And Barriers Described By Early Adopters: A review of new technology platforms to facilitate referrals from health care organizations to social service organizations. Health Aff. 2020;39(4):662–9.

Center for Health Care Strategies [Internet]. Supporting social service and health care partnerships to address health-related social needs: case study series. [updated 2018; cited 2023 Nov 9] Available from: https://www.chcs.org/project/partnership-healthy-outcomes-bridging-community-based-human-services-health-care/ . Accessed March 2, 2023.

Klein S, Hostetter M. Leveraging Technology to Find Solutions to Patients’ Unmet Social Needs. The Commonwealth Fund; June 21, 2017. Available from: https://www.commonwealthfund.org/publications/2017/jun/leveraging-technology-find-solutions-patients-unmet-social-needs

Blavin F, Smith LB, Ramos C, Ozanich G, Horn A. Opportunities to Improve Data Interoperability and Integration to Support Value-Based Care. 2022.

Massar RE, Berry CA, Paul MM. Social needs screening and referral in pediatric primary care clinics: a multiple case study. BMC Health Serv Res. 2022;22(1):1369.

Hogg‐Graham R, Edwards K, L Ely T, Mochizuki M, Varda D. Exploring the capacity of community‐based organisations to absorb health system patient referrals for unmet social needs. Health Soc Care Commun. 2021;29(2):487–95.

Amarashingham R, Xie B, Karam A, Nguyen N, Kapoor B. Using community partnerships to integrate health and social services for high-need, high-cost patients. Issue Brief (Commonw Fund). 2018;2018:1–11.

PubMed   Google Scholar  

Agonafer EP, Carson SL, Nunez V, Poole K, Hong CS, Morales M, et al. Community-based organizations’ perspectives on improving health and social service integration. BMC Public Health. 2021;21(1):1–12.

Petchel S, Gelmon S, Goldberg B. The Organizational Risks Of Cross-Sector Partnerships: A Comparison Of Health And Human Services Perspectives: A legal and policy review to identify potential funding streams specifically for Accountable Communities For Health infrastructure activities. Health Aff. 2020;39(4):574–81.

Hogg-Graham R, Scott AM, Stahl H, Riley E, Clear ER, Waters TM. COVID-19 and MCO-community partnerships to address enrollee social needs. Am J Managed Care. 2023;29(3).

KFF. Medicaid State Fact Sheets 2023 [Available from: https://www.kff.org/interactive/medicaid-state-fact-sheets/ .

Hogg-Graham R, Scott AM, Waters TM. Medicaid Managed Care Organizations and Community Based Organizations Social Need Strategies Interview Guide. University of Kentucky. 2021.

Sandelowski M. Using qualitative research. Qual Health Res. 2004;14(10):1366–86.

Thornberg R, Charmaz K. Grounded theory and theoretical coding. The SAGE handbook of qualitative data analysis. 2014;2014(5):153–69.

Cascio MA, Lee E, Vaudrin N, Freedman DA. A team-based approach to open coding: Considerations for creating intercoder consensus. Field Methods. 2019;31(2):116–30.

Unite Us. Cross-sector collaboration software powered by community. [updated 2023; cited 2023 Nov 9]. Available from: https://uniteus.com/ .

FindHelp.org. Social Care Technology [Internet]. [updated 2023; cited 2023 Nov 9]. Available from: https://company.findhelp.com/ .

The Partnership Center. Vesta [Internet]. [updated 2023; cited 2023 Nov 9]. 2023; Available from: https://thepcl.net/vesta.html .

Varda DM, Retrum JH. Collaborative performance as a function of network members’ perceptions of success. Public Perform Manag Rev. 2015;38(4):632–53.

Varda DM, Chandra A, Stern SA, Lurie N. Core dimensions of connectivity in public health collaboratives. J Public Health Manag Pract. 2008;14(5):E1-7.

Byhoff E, Taylor LA. Massachusetts community-based organization perspectives on Medicaid redesign. Am J Prev Med. 2019;57(6):S74–81.

Download references

Acknowledgements

The authors would like to thank the Study Advisory Board for their help in guiding the research.

This research was supported by a Robert Wood Johnson Foundation grant as part of the Research in Transforming Health and Health Systems Program (Grant ID 77256). Research reported in this publication was also supported by the Kentucky Cabinet for Health and Family Services, Department for Medicaid Services under Agreement C2517 titled “Medicaid Managed Care Organizational Strategies to Address Enrollee Unmet Social Needs.” The content is solely the responsibility of the authors and does not necessarily represent the official views of the Cabinet for Health and Family Services, Department for Medicaid Services.

Author information

Authors and affiliations.

Department of Health Management and Policy, College of Public Health, University of Kentucky, 111 Washington Ave, 107B, Lexington, KY, USA

Rachel Hogg-Graham, Emily R. Clear & Elizabeth N. Riley

Department of Communication, University of Kentucky, Lexington, KY, USA

Allison M. Scott

Institute for Public and Preventive Health, Augusta University, Augusta, GA, USA

Teresa M. Waters

You can also search for this author in PubMed   Google Scholar

Contributions

Concept and design (RH-G, AMS, TMW); acquisition of data (RH-G, AMS, ERC, TMW); analysis and interpretation of data (RH-G, AMS, ER, TMW); drafting of the manuscript (RH-G, AMS, ER, ERC, TMW); critical revision of the manuscript for important intellectual content (RH-G, AMS, ER, TMW); provision of patients or study materials (RH-G, ERC); obtaining funding (RH-G, TMW); administrative, technical, or logistic support (RH-G, ERC, TMW); and supervision (RH-G).

Corresponding author

Correspondence to Rachel Hogg-Graham .

Ethics declarations

Ethics approval and consent to participate.

All research activities involving human subjects have been reviewed and approved by the Institutional Review Board at the University of Kentucky. Informed consent was verbally obtained by all participants. All methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Hogg-Graham, R., Scott, A.M., Clear, E.R. et al. Technology, data, people, and partnerships in addressing unmet social needs within Medicaid Managed Care. BMC Health Serv Res 24 , 368 (2024). https://doi.org/10.1186/s12913-024-10705-w

Download citation

Received : 13 November 2023

Accepted : 11 February 2024

Published : 23 March 2024

DOI : https://doi.org/10.1186/s12913-024-10705-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Social determinants of health
• Managed care organizations
• Health care organizations and systems

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

• Share full article

Advertisement

Supported by

What the Data Says About Pandemic School Closures, Four Years Later

The more time students spent in remote instruction, the further they fell behind. And, experts say, extended closures did little to stop the spread of Covid.

By Sarah Mervosh ,  Claire Cain Miller and Francesca Paris

Four years ago this month, schools nationwide began to shut down, igniting one of the most polarizing and partisan debates of the pandemic.

Some schools, often in Republican-led states and rural areas, reopened by fall 2020. Others, typically in large cities and states led by Democrats, would not fully reopen for another year.

A variety of data — about children’s academic outcomes and about the spread of Covid-19 — has accumulated in the time since. Today, there is broad acknowledgment among many public health and education experts that extended school closures did not significantly stop the spread of Covid, while the academic harms for children have been large and long-lasting.

While poverty and other factors also played a role, remote learning was a key driver of academic declines during the pandemic, research shows — a finding that held true across income levels.

Source: Fahle, Kane, Patterson, Reardon, Staiger and Stuart, “ School District and Community Factors Associated With Learning Loss During the COVID-19 Pandemic .” Score changes are measured from 2019 to 2022. In-person means a district offered traditional in-person learning, even if not all students were in-person.

“There’s fairly good consensus that, in general, as a society, we probably kept kids out of school longer than we should have,” said Dr. Sean O’Leary, a pediatric infectious disease specialist who helped write guidance for the American Academy of Pediatrics, which recommended in June 2020 that schools reopen with safety measures in place.

There were no easy decisions at the time. Officials had to weigh the risks of an emerging virus against the academic and mental health consequences of closing schools. And even schools that reopened quickly, by the fall of 2020, have seen lasting effects.

But as experts plan for the next public health emergency, whatever it may be, a growing body of research shows that pandemic school closures came at a steep cost to students.

The longer schools were closed, the more students fell behind.

At the state level, more time spent in remote or hybrid instruction in the 2020-21 school year was associated with larger drops in test scores, according to a New York Times analysis of school closure data and results from the National Assessment of Educational Progress , an authoritative exam administered to a national sample of fourth- and eighth-grade students.

At the school district level, that finding also holds, according to an analysis of test scores from third through eighth grade in thousands of U.S. districts, led by researchers at Stanford and Harvard. In districts where students spent most of the 2020-21 school year learning remotely, they fell more than half a grade behind in math on average, while in districts that spent most of the year in person they lost just over a third of a grade.

( A separate study of nearly 10,000 schools found similar results.)

Such losses can be hard to overcome, without significant interventions. The most recent test scores, from spring 2023, show that students, overall, are not caught up from their pandemic losses , with larger gaps remaining among students that lost the most ground to begin with. Students in districts that were remote or hybrid the longest — at least 90 percent of the 2020-21 school year — still had almost double the ground to make up compared with students in districts that allowed students back for most of the year.

Some time in person was better than no time.

As districts shifted toward in-person learning as the year went on, students that were offered a hybrid schedule (a few hours or days a week in person, with the rest online) did better, on average, than those in places where school was fully remote, but worse than those in places that had school fully in person.

Students in hybrid or remote learning, 2020-21

80% of students

Some schools return online, as Covid-19 cases surge. Vaccinations start for high-priority groups.

Teachers are eligible for the Covid vaccine in more than half of states.

Most districts end the year in-person or hybrid.

Source: Burbio audit of more than 1,200 school districts representing 47 percent of U.S. K-12 enrollment. Note: Learning mode was defined based on the most in-person option available to students.

Income and family background also made a big difference.

A second factor associated with academic declines during the pandemic was a community’s poverty level. Comparing districts with similar remote learning policies, poorer districts had steeper losses.

But in-person learning still mattered: Looking at districts with similar poverty levels, remote learning was associated with greater declines.

A community’s poverty rate and the length of school closures had a “roughly equal” effect on student outcomes, said Sean F. Reardon, a professor of poverty and inequality in education at Stanford, who led a district-level analysis with Thomas J. Kane, an economist at Harvard.

Score changes are measured from 2019 to 2022. Poorest and richest are the top and bottom 20% of districts by percent of students on free/reduced lunch. Mostly in-person and mostly remote are districts that offered traditional in-person learning for more than 90 percent or less than 10 percent of the 2020-21 year.

But the combination — poverty and remote learning — was particularly harmful. For each week spent remote, students in poor districts experienced steeper losses in math than peers in richer districts.

That is notable, because poor districts were also more likely to stay remote for longer .

Some of the country’s largest poor districts are in Democratic-leaning cities that took a more cautious approach to the virus. Poor areas, and Black and Hispanic communities , also suffered higher Covid death rates, making many families and teachers in those districts hesitant to return.

“We wanted to survive,” said Sarah Carpenter, the executive director of Memphis Lift, a parent advocacy group in Memphis, where schools were closed until spring 2021 .

“But I also think, man, looking back, I wish our kids could have gone back to school much quicker,” she added, citing the academic effects.

Other things were also associated with worse student outcomes, including increased anxiety and depression among adults in children’s lives, and the overall restriction of social activity in a community, according to the Stanford and Harvard research .

Even short closures had long-term consequences for children.

While being in school was on average better for academic outcomes, it wasn’t a guarantee. Some districts that opened early, like those in Cherokee County, Ga., a suburb of Atlanta, and Hanover County, Va., lost significant learning and remain behind.

At the same time, many schools are seeing more anxiety and behavioral outbursts among students. And chronic absenteeism from school has surged across demographic groups .

These are signs, experts say, that even short-term closures, and the pandemic more broadly, had lasting effects on the culture of education.

“There was almost, in the Covid era, a sense of, ‘We give up, we’re just trying to keep body and soul together,’ and I think that was corrosive to the higher expectations of schools,” said Margaret Spellings, an education secretary under President George W. Bush who is now chief executive of the Bipartisan Policy Center.

Closing schools did not appear to significantly slow Covid’s spread.

Perhaps the biggest question that hung over school reopenings: Was it safe?

That was largely unknown in the spring of 2020, when schools first shut down. But several experts said that had changed by the fall of 2020, when there were initial signs that children were less likely to become seriously ill, and growing evidence from Europe and parts of the United States that opening schools, with safety measures, did not lead to significantly more transmission.

“Infectious disease leaders have generally agreed that school closures were not an important strategy in stemming the spread of Covid,” said Dr. Jeanne Noble, who directed the Covid response at the U.C.S.F. Parnassus emergency department.

Politically, though, there remains some disagreement about when, exactly, it was safe to reopen school.

Republican governors who pushed to open schools sooner have claimed credit for their approach, while Democrats and teachers’ unions have emphasized their commitment to safety and their investment in helping students recover.

“I do believe it was the right decision,” said Jerry T. Jordan, president of the Philadelphia Federation of Teachers, which resisted returning to school in person over concerns about the availability of vaccines and poor ventilation in school buildings. Philadelphia schools waited to partially reopen until the spring of 2021 , a decision Mr. Jordan believes saved lives.

“It doesn’t matter what is going on in the building and how much people are learning if people are getting the virus and running the potential of dying,” he said.

Pandemic school closures offer lessons for the future.

Though the next health crisis may have different particulars, with different risk calculations, the consequences of closing schools are now well established, experts say.

In the future, infectious disease experts said, they hoped decisions would be guided more by epidemiological data as it emerged, taking into account the trade-offs.

“Could we have used data to better guide our decision making? Yes,” said Dr. Uzma N. Hasan, division chief of pediatric infectious diseases at RWJBarnabas Health in Livingston, N.J. “Fear should not guide our decision making.”

Source: Fahle, Kane, Patterson, Reardon, Staiger and Stuart, “ School District and Community Factors Associated With Learning Loss During the Covid-19 Pandemic. ”

The study used estimates of learning loss from the Stanford Education Data Archive . For closure lengths, the study averaged district-level estimates of time spent in remote and hybrid learning compiled by the Covid-19 School Data Hub (C.S.D.H.) and American Enterprise Institute (A.E.I.) . The A.E.I. data defines remote status by whether there was an in-person or hybrid option, even if some students chose to remain virtual. In the C.S.D.H. data set, districts are defined as remote if “all or most” students were virtual.

An earlier version of this article misstated a job description of Dr. Jeanne Noble. She directed the Covid response at the U.C.S.F. Parnassus emergency department. She did not direct the Covid response for the University of California, San Francisco health system.

How we handle corrections

Sarah Mervosh covers education for The Times, focusing on K-12 schools. More about Sarah Mervosh

Claire Cain Miller writes about gender, families and the future of work for The Upshot. She joined The Times in 2008 and was part of a team that won a Pulitzer Prize in 2018 for public service for reporting on workplace sexual harassment issues. More about Claire Cain Miller

Francesca Paris is a Times reporter working with data and graphics for The Upshot. More about Francesca Paris

• International edition
• Australia edition
• Europe edition

What is preventive chemotherapy and how effective is it? The Princess of Wales’ treatment explained

As Princess of Wales begins treatment after cancer diagnosis, we answer some key questions

The Princess of Wales has begun preventive chemotherapy after her doctors discovered she had cancer following major abdominal surgery in January. What is preventive chemotherapy and how effective can it be?

What is preventive chemotherapy?

The treatment, known formally as adjuvant chemotherapy, is a course of anti-cancer drugs that is given to mop up any cancer cells that might remain in the body after primary cancer treatment, which is usually surgical removal of a tumour.

What does it do?

The therapy aims to reduce the risk of the original cancer coming back and spreading. This can happen when cancer cells, which are too small to detect with hospital scans and tests, are left behind after surgery. The risk of cancer returning tends to be smaller if the cancer is caught at a very early stage, before it has had a chance to spread, but greater if the disease is found at a later stage, or has spread to nearby lymph nodes.

How does the therapy work?

Most cancer chemotherapy drugs target rapidly dividing cells. A typical course of preventive chemotherapy lasts three to six months depending on the type and stage of the cancer, which is determined by examining the cancer removed during surgery. Occasionally, courses of adjuvant chemotherapy are given over several years.

How many courses of therapy do people have?

It depends on the nature of the original tumour that was picked up after surgery. When cancer is discovered during operations for other conditions, the tumour is often at an early stage, when subsequent chemotherapy is more effective. “This is likely to mean that a single course of chemotherapy will be sufficient to ensure that if any cancer cells are present, they will be destroyed,” says Prof Lawrence Young, the director of the Warwick Cancer Research Centre at the University of Warwick.

How effective is the therapy?

It is particularly effective for breast, bowel and lung cancer, but can be recommended for other forms of the disease, too. For example, adjuvant chemotherapy is often used after surgery for epithelial ovarian cancer, the most common form of ovarian cancer, because of the risk of the disease returning. Doctors decide whether adjuvant therapy is likely to have a benefit based on the type of cancer, how advanced the disease is, and other properties of the tumour.

Are there side-effects?

No chemotherapy is completely harmless. The side-effects depend on the specific drugs given, but patients can experience tiredness, nausea, vomiting, diarrhoea, an increased risk of picking up infections, and a loss of appetite. The side-effects arise because the drugs affect not only cancer cells but all rapidly dividing cells, including hair, bone marrow, skin and the lining of the digestive system. The damage to healthy tissues tends to be temporary, however, and side-effects usually disappear once the treatment is over.

Younger people often tolerate chemotherapy better than older patients and experience fewer side-effects because of their “greater functional reserves” and the “ability of young tissues to heal more rapidly,” says Dr Mangesh Thorat, an honorary reader at Queen Mary University of London and consultant breast surgeon at Homerton university hospital. Because of this, younger people may be given higher doses of the drugs, which are more likely to wipe out any cancer cells that remain in their bodies.

How long does it take to recover?

It depends on the patient and the particular drugs given, but it can take several months for a person to be back to full strength. Since younger people tend to be healthier and fitter than older people, their recovery times are usually shorter.

• Catherine, Princess of Wales

More on this story

Princess of Wales’ diagnosis: cancers in young are rising, but so are survival rates

UK scientists working on breast cancer monitor fitted in bra

UK cancer study shows big fall in death rates since early 1990s

Sussex man doing well a year and a half after new brain cancer treatment

Drug that could slow womb cancer to be rolled out by NHS in England

Study offers hope in identifying high-risk prostate cancer patients

UK trails other countries on waiting times for cancer treatment, study finds

Cancer experts call on philanthropists to help fund ‘golden age’ of research

Most viewed.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

Research Methods | Definitions, Types, Examples

Research methods are specific procedures for collecting and analyzing data. Developing your research methods is an integral part of your research design . When planning your methods, there are two key decisions you will make.

First, decide how you will collect data . Your methods depend on what type of data you need to answer your research question :

• Qualitative vs. quantitative : Will your data take the form of words or numbers?
• Primary vs. secondary : Will you collect original data yourself, or will you use data that has already been collected by someone else?
• Descriptive vs. experimental : Will you take measurements of something as it is, or will you perform an experiment?

Second, decide how you will analyze the data .

• For quantitative data, you can use statistical analysis methods to test relationships between variables.
• For qualitative data, you can use methods such as thematic analysis to interpret patterns and meanings in the data.

Table of contents

Methods for collecting data, examples of data collection methods, methods for analyzing data, examples of data analysis methods, other interesting articles, frequently asked questions about research methods.

Data is the information that you collect for the purposes of answering your research question . The type of data you need depends on the aims of your research.

Qualitative vs. quantitative data

Your choice of qualitative or quantitative data collection depends on the type of knowledge you want to develop.

For questions about ideas, experiences and meanings, or to study something that can’t be described numerically, collect qualitative data .

If you want to develop a more mechanistic understanding of a topic, or your research involves hypothesis testing , collect quantitative data .

You can also take a mixed methods approach , where you use both qualitative and quantitative research methods.

Primary vs. secondary research

Primary research is any original data that you collect yourself for the purposes of answering your research question (e.g. through surveys , observations and experiments ). Secondary research is data that has already been collected by other researchers (e.g. in a government census or previous scientific studies).

If you are exploring a novel research question, you’ll probably need to collect primary data . But if you want to synthesize existing knowledge, analyze historical trends, or identify patterns on a large scale, secondary data might be a better choice.

Descriptive vs. experimental data

In descriptive research , you collect data about your study subject without intervening. The validity of your research will depend on your sampling method .

In experimental research , you systematically intervene in a process and measure the outcome. The validity of your research will depend on your experimental design .

To conduct an experiment, you need to be able to vary your independent variable , precisely measure your dependent variable, and control for confounding variables . If it’s practically and ethically possible, this method is the best choice for answering questions about cause and effect.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Your data analysis methods will depend on the type of data you collect and how you prepare it for analysis.

Data can often be analyzed both quantitatively and qualitatively. For example, survey responses could be analyzed qualitatively by studying the meanings of responses or quantitatively by studying the frequencies of responses.

Qualitative analysis methods

Qualitative analysis is used to understand words, ideas, and experiences. You can use it to interpret data that was collected:

• From open-ended surveys and interviews , literature reviews , case studies , ethnographies , and other sources that use text rather than numbers.
• Using non-probability sampling methods .

Qualitative analysis tends to be quite flexible and relies on the researcher’s judgement, so you have to reflect carefully on your choices and assumptions and be careful to avoid research bias .

Quantitative analysis methods

Quantitative analysis uses numbers and statistics to understand frequencies, averages and correlations (in descriptive studies) or cause-and-effect relationships (in experiments).

You can use quantitative analysis to interpret data that was collected either:

• During an experiment .
• Using probability sampling methods .

Because the data is collected and analyzed in a statistically valid way, the results of quantitative analysis can be easily standardized and shared among researchers.

Prevent plagiarism. Run a free check.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square test of independence
• Statistical power
• Descriptive statistics
• Degrees of freedom
• Pearson correlation
• Null hypothesis
• Double-blind study
• Case-control study
• Research ethics
• Data collection
• Hypothesis testing
• Structured interviews

Research bias

• Hawthorne effect
• Unconscious bias
• Recall bias
• Halo effect
• Self-serving bias
• Information bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

A sample is a subset of individuals from a larger population . Sampling means selecting the group that you will actually collect data from in your research. For example, if you are researching the opinions of students in your university, you could survey a sample of 100 students.

In statistics, sampling allows you to test a hypothesis about the characteristics of a population.

The research methods you use depend on the type of data you need to answer your research question .

• If you want to measure something or test a hypothesis , use quantitative methods . If you want to explore ideas, thoughts and meanings, use qualitative methods .
• If you want to analyze a large amount of readily-available data, use secondary data. If you want data specific to your purposes with control over how it is generated, collect primary data.
• If you want to establish cause-and-effect relationships between variables , use experimental methods. If you want to understand the characteristics of a research subject, use descriptive methods.

Methodology refers to the overarching strategy and rationale of your research project . It involves studying the methods used in your field and the theories or principles behind them, in order to develop an approach that matches your objectives.

Methods are the specific tools and procedures you use to collect and analyze data (for example, experiments, surveys , and statistical tests ).

In shorter scientific papers, where the aim is to report the findings of a specific study, you might simply describe what you did in a methods section .

In a longer or more complex research project, such as a thesis or dissertation , you will probably include a methodology section , where you explain your approach to answering the research questions and cite relevant sources to support your choice of methods.

Is this article helpful?

Other students also liked, writing strong research questions | criteria & examples.

• What Is a Research Design | Types, Guide & Examples
• Data Collection | Definition, Methods & Examples

More interesting articles

• Between-Subjects Design | Examples, Pros, & Cons
• Cluster Sampling | A Simple Step-by-Step Guide with Examples
• Confounding Variables | Definition, Examples & Controls
• Construct Validity | Definition, Types, & Examples
• Content Analysis | Guide, Methods & Examples
• Control Groups and Treatment Groups | Uses & Examples
• Control Variables | What Are They & Why Do They Matter?
• Correlation vs. Causation | Difference, Designs & Examples
• Correlational Research | When & How to Use
• Critical Discourse Analysis | Definition, Guide & Examples
• Cross-Sectional Study | Definition, Uses & Examples
• Descriptive Research | Definition, Types, Methods & Examples
• Ethical Considerations in Research | Types & Examples
• Explanatory and Response Variables | Definitions & Examples
• Explanatory Research | Definition, Guide, & Examples
• Exploratory Research | Definition, Guide, & Examples
• External Validity | Definition, Types, Threats & Examples
• Extraneous Variables | Examples, Types & Controls
• Guide to Experimental Design | Overview, Steps, & Examples
• How Do You Incorporate an Interview into a Dissertation? | Tips
• How to Do Thematic Analysis | Step-by-Step Guide & Examples
• How to Write a Literature Review | Guide, Examples, & Templates
• How to Write a Strong Hypothesis | Steps & Examples
• Inclusion and Exclusion Criteria | Examples & Definition
• Independent vs. Dependent Variables | Definition & Examples
• Inductive Reasoning | Types, Examples, Explanation
• Inductive vs. Deductive Research Approach | Steps & Examples
• Internal Validity in Research | Definition, Threats, & Examples
• Internal vs. External Validity | Understanding Differences & Threats
• Longitudinal Study | Definition, Approaches & Examples
• Mediator vs. Moderator Variables | Differences & Examples
• Mixed Methods Research | Definition, Guide & Examples
• Multistage Sampling | Introductory Guide & Examples
• Naturalistic Observation | Definition, Guide & Examples
• Operationalization | A Guide with Examples, Pros & Cons
• Population vs. Sample | Definitions, Differences & Examples
• Primary Research | Definition, Types, & Examples
• Qualitative vs. Quantitative Research | Differences, Examples & Methods
• Quasi-Experimental Design | Definition, Types & Examples
• Questionnaire Design | Methods, Question Types & Examples
• Random Assignment in Experiments | Introduction & Examples
• Random vs. Systematic Error | Definition & Examples
• Reliability vs. Validity in Research | Difference, Types and Examples
• Reproducibility vs Replicability | Difference & Examples
• Reproducibility vs. Replicability | Difference & Examples
• Sampling Methods | Types, Techniques & Examples
• Semi-Structured Interview | Definition, Guide & Examples
• Simple Random Sampling | Definition, Steps & Examples
• Single, Double, & Triple Blind Study | Definition & Examples
• Stratified Sampling | Definition, Guide & Examples
• Structured Interview | Definition, Guide & Examples
• Survey Research | Definition, Examples & Methods
• Systematic Review | Definition, Example, & Guide
• Systematic Sampling | A Step-by-Step Guide with Examples
• Textual Analysis | Guide, 3 Approaches & Examples
• The 4 Types of Reliability in Research | Definitions & Examples
• The 4 Types of Validity in Research | Definitions & Examples
• Transcribing an Interview | 5 Steps & Transcription Software
• Triangulation in Research | Guide, Types, Examples
• Types of Interviews in Research | Guide & Examples
• Types of Research Designs Compared | Guide & Examples
• Types of Variables in Research & Statistics | Examples
• Unstructured Interview | Definition, Guide & Examples
• What Is a Case Study? | Definition, Examples & Methods
• What Is a Case-Control Study? | Definition & Examples
• What Is a Cohort Study? | Definition & Examples
• What Is a Conceptual Framework? | Tips & Examples
• What Is a Controlled Experiment? | Definitions & Examples
• What Is a Double-Barreled Question?
• What Is a Focus Group? | Step-by-Step Guide & Examples
• What Is a Likert Scale? | Guide & Examples
• What Is a Prospective Cohort Study? | Definition & Examples
• What Is a Retrospective Cohort Study? | Definition & Examples
• What Is Action Research? | Definition & Examples
• What Is an Observational Study? | Guide & Examples
• What Is Concurrent Validity? | Definition & Examples
• What Is Content Validity? | Definition & Examples
• What Is Convenience Sampling? | Definition & Examples
• What Is Convergent Validity? | Definition & Examples
• What Is Criterion Validity? | Definition & Examples
• What Is Data Cleansing? | Definition, Guide & Examples
• What Is Deductive Reasoning? | Explanation & Examples
• What Is Discriminant Validity? | Definition & Example
• What Is Ecological Validity? | Definition & Examples
• What Is Ethnography? | Definition, Guide & Examples
• What Is Face Validity? | Guide, Definition & Examples
• What Is Non-Probability Sampling? | Types & Examples
• What Is Participant Observation? | Definition & Examples
• What Is Peer Review? | Types & Examples
• What Is Predictive Validity? | Examples & Definition
• What Is Probability Sampling? | Types & Examples
• What Is Purposive Sampling? | Definition & Examples
• What Is Qualitative Observation? | Definition & Examples
• What Is Qualitative Research? | Methods & Examples
• What Is Quantitative Observation? | Definition & Examples
• What Is Quantitative Research? | Definition, Uses & Methods

What is your plagiarism score?

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

How to Improve a Meeting (When You’re Not in Charge)

• Tijs Besieux
• Amy C. Edmondson

You can use these tactics before, during, or after.

Research shows that many employees find meetings ineffective, inefficient, or unproductive, but unless you work in a psychologically safe environment, where candor is encouraged, you’re unlikely to raise your hand and speak up about what’s wrong — especially if you’re not the person running the meeting. The authors have identified a series of interventions that anyone can use:

• The helpful workaround is useful when there is low psychological safety on the team or with the person conducting the meeting. Here, you don’t explicitly name the thing that bothers you, but you ask a question that can help resolve the feelings of frustration you may be experiencing in the meeting.
• The solution-centric proposal works when there is moderate psychological safety. Here, you are to point to the problem area by asking a question that demonstrates how resolving the issue will increase the effectiveness of the meeting.
• The constructive confrontation is for high psychological safety situations. With this approach, you decide to explicitly name the problem and give feedback to forge a productive solution.

Organizations hold meetings all the time — lots of them. In the U.S. alone, 55 million meetings happen daily, but how many of them are actually useful? According to most employees and managers — not very many.

• Tijs Besieux is an independent researcher at Harvard Business School. He is a senior advisor at &samhoud and the founder of Voice Up.
• Amy C. Edmondson is the Novartis Professor of Leadership and Management at Harvard Business School. Her latest book is Right Kind of Wrong: The Science of Failing Well (Atria Books, forthcoming in September 2023).

Partner Center