what is data collection in nursing research

Introduction to qualitative nursing research

This type of research can reveal important information that quantitative research can’t.

• Qualitative research is valuable because it approaches a phenomenon, such as a clinical problem, about which little is known by trying to understand its many facets.
• Most qualitative research is emergent, holistic, detailed, and uses many strategies to collect data.
• Qualitative research generates evidence and helps nurses determine patient preferences.

Research 101: Descriptive statistics

Differentiating research, evidence-based practice, and quality improvement

How to appraise quantitative research articles

All nurses are expected to understand and apply evidence to their professional practice. Some of the evidence should be in the form of research, which fills gaps in knowledge, developing and expanding on current understanding. Both quantitative and qualitative research methods inform nursing practice, but quantitative research tends to be more emphasized. In addition, many nurses don’t feel comfortable conducting or evaluating qualitative research. But once you understand qualitative research, you can more easily apply it to your nursing practice.

What is qualitative research?

Defining qualitative research can be challenging. In fact, some authors suggest that providing a simple definition is contrary to the method’s philosophy. Qualitative research approaches a phenomenon, such as a clinical problem, from a place of unknowing and attempts to understand its many facets. This makes qualitative research particularly useful when little is known about a phenomenon because the research helps identify key concepts and constructs. Qualitative research sets the foundation for future quantitative or qualitative research. Qualitative research also can stand alone without quantitative research.

Although qualitative research is diverse, certain characteristics—holism, subjectivity, intersubjectivity, and situated contexts—guide its methodology. This type of research stresses the importance of studying each individual as a holistic system (holism) influenced by surroundings (situated contexts); each person develops his or her own subjective world (subjectivity) that’s influenced by interactions with others (intersubjectivity) and surroundings (situated contexts). Think of it this way: Each person experiences and interprets the world differently based on many factors, including his or her history and interactions. The truth is a composite of realities.

Qualitative research designs

Because qualitative research explores diverse topics and examines phenomena where little is known, designs and methodologies vary. Despite this variation, most qualitative research designs are emergent and holistic. In addition, they require merging data collection strategies and an intensely involved researcher. (See Research design characteristics .)

Although qualitative research designs are emergent, advanced planning and careful consideration should include identifying a phenomenon of interest, selecting a research design, indicating broad data collection strategies and opportunities to enhance study quality, and considering and/or setting aside (bracketing) personal biases, views, and assumptions.

Many qualitative research designs are used in nursing. Most originated in other disciplines, while some claim no link to a particular disciplinary tradition. Designs that aren’t linked to a discipline, such as descriptive designs, may borrow techniques from other methodologies; some authors don’t consider them to be rigorous (high-quality and trustworthy). (See Common qualitative research designs .)

Sampling approaches

Sampling approaches depend on the qualitative research design selected. However, in general, qualitative samples are small, nonrandom, emergently selected, and intensely studied. Qualitative research sampling is concerned with accurately representing and discovering meaning in experience, rather than generalizability. For this reason, researchers tend to look for participants or informants who are considered “information rich” because they maximize understanding by representing varying demographics and/or ranges of experiences. As a study progresses, researchers look for participants who confirm, challenge, modify, or enrich understanding of the phenomenon of interest. Many authors argue that the concepts and constructs discovered in qualitative research transcend a particular study, however, and find applicability to others. For example, consider a qualitative study about the lived experience of minority nursing faculty and the incivility they endure. The concepts learned in this study may transcend nursing or minority faculty members and also apply to other populations, such as foreign-born students, nurses, or faculty.

Qualitative nursing research can take many forms. The design you choose will depend on the question you’re trying to answer.

A sample size is estimated before a qualitative study begins, but the final sample size depends on the study scope, data quality, sensitivity of the research topic or phenomenon of interest, and researchers’ skills. For example, a study with a narrow scope, skilled researchers, and a nonsensitive topic likely will require a smaller sample. Data saturation frequently is a key consideration in final sample size. When no new insights or information are obtained, data saturation is attained and sampling stops, although researchers may analyze one or two more cases to be certain. (See Sampling types .)

Some controversy exists around the concept of saturation in qualitative nursing research. Thorne argues that saturation is a concept appropriate for grounded theory studies and not other study types. She suggests that “information power” is perhaps more appropriate terminology for qualitative nursing research sampling and sample size.

Data collection and analysis

Researchers are guided by their study design when choosing data collection and analysis methods. Common types of data collection include interviews (unstructured, semistructured, focus groups); observations of people, environments, or contexts; documents; records; artifacts; photographs; or journals. When collecting data, researchers must be mindful of gaining participant trust while also guarding against too much emotional involvement, ensuring comprehensive data collection and analysis, conducting appropriate data management, and engaging in reflexivity.

Data usually are recorded in detailed notes, memos, and audio or visual recordings, which frequently are transcribed verbatim and analyzed manually or using software programs, such as ATLAS.ti, HyperRESEARCH, MAXQDA, or NVivo. Analyzing qualitative data is complex work. Researchers act as reductionists, distilling enormous amounts of data into concise yet rich and valuable knowledge. They code or identify themes, translating abstract ideas into meaningful information. The good news is that qualitative research typically is easy to understand because it’s reported in stories told in everyday language.

Evaluating a qualitative study

Evaluating qualitative research studies can be challenging. Many terms—rigor, validity, integrity, and trustworthiness—can describe study quality, but in the end you want to know whether the study’s findings accurately and comprehensively represent the phenomenon of interest. Many researchers identify a quality framework when discussing quality-enhancement strategies. Example frameworks include:

• Trustworthiness criteria framework, which enhances credibility, dependability, confirmability, transferability, and authenticity
• Validity in qualitative research framework, which enhances credibility, authenticity, criticality, integrity, explicitness, vividness, creativity, thoroughness, congruence, and sensitivity.

With all frameworks, many strategies can be used to help meet identified criteria and enhance quality. (See Research quality enhancement ). And considering the study as a whole is important to evaluating its quality and rigor. For example, when looking for evidence of rigor, look for a clear and concise report title that describes the research topic and design and an abstract that summarizes key points (background, purpose, methods, results, conclusions).

Application to nursing practice

Qualitative research not only generates evidence but also can help nurses determine patient preferences. Without qualitative research, we can’t truly understand others, including their interpretations, meanings, needs, and wants. Qualitative research isn’t generalizable in the traditional sense, but it helps nurses open their minds to others’ experiences. For example, nurses can protect patient autonomy by understanding them and not reducing them to universal protocols or plans. As Munhall states, “Each person we encounter help[s] us discover what is best for [him or her]. The other person, not us, is truly the expert knower of [him- or herself].” Qualitative nursing research helps us understand the complexity and many facets of a problem and gives us insights as we encourage others’ voices and searches for meaning.

When paired with clinical judgment and other evidence, qualitative research helps us implement evidence-based practice successfully. For example, a phenomenological inquiry into the lived experience of disaster workers might help expose strengths and weaknesses of individuals, populations, and systems, providing areas of focused intervention. Or a phenomenological study of the lived experience of critical-care patients might expose factors (such dark rooms or no visible clocks) that contribute to delirium.

Successful implementation

Qualitative nursing research guides understanding in practice and sets the foundation for future quantitative and qualitative research. Knowing how to conduct and evaluate qualitative research can help nurses implement evidence-based practice successfully.

When evaluating a qualitative study, you should consider it as a whole. The following questions to consider when examining study quality and evidence of rigor are adapted from the Standards for Reporting Qualitative Research.

Jennifer Chicca is a PhD candidate at the Indiana University of Pennsylvania in Indiana, Pennsylvania, and a part-time faculty member at the University of North Carolina Wilmington.

Amankwaa L. Creating protocols for trustworthiness in qualitative research. J Cult Divers. 2016;23(3):121-7.

Cuthbert CA, Moules N. The application of qualitative research findings to oncology nursing practice. Oncol Nurs Forum . 2014;41(6):683-5.

Guba E, Lincoln Y. Competing paradigms in qualitative research . In: Denzin NK, Lincoln YS, eds. Handbook of Qualitative Research. Thousand Oaks, CA: SAGE Publications, Inc.;1994: 105-17.

Lincoln YS, Guba EG. Naturalistic Inquiry . Thousand Oaks, CA: SAGE Publications, Inc.; 1985.

Munhall PL. Nursing Research: A Qualitative Perspective . 5th ed. Sudbury, MA: Jones & Bartlett Learning; 2012.

Nicholls D. Qualitative research. Part 1: Philosophies. Int J Ther Rehabil . 2017;24(1):26-33.

Nicholls D. Qualitative research. Part 2: Methodology. Int J Ther Rehabil . 2017;24(2):71-7.

Nicholls D. Qualitative research. Part 3: Methods. Int J Ther Rehabil . 2017;24(3):114-21.

O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: A synthesis of recommendations. Acad Med . 2014;89(9):1245-51.

Polit DF, Beck CT. Nursing Research: Generating and Assessing Evidence for Nursing Practice . 10th ed. Philadelphia, PA: Wolters Kluwer; 2017.

Thorne S. Saturation in qualitative nursing studies: Untangling the misleading message around saturation in qualitative nursing studies. Nurse Auth Ed. 2020;30(1):5. naepub.com/reporting-research/2020-30-1-5

Whittemore R, Chase SK, Mandle CL. Validity in qualitative research. Qual Health Res . 2001;11(4):522-37.

Williams B. Understanding qualitative research. Am Nurse Today . 2015;10(7):40-2.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Post Comment

NurseLine Newsletter

• First Name *
• Last Name *
• Hidden Referrer

*By submitting your e-mail, you are opting in to receiving information from Healthcom Media and Affiliates. The details, including your email address/mobile number, may be used to keep you informed about future products and services.

Test Your Knowledge

Recent posts.

Nearly 100 measles cases reported in the first quarter, CDC says

Infections after surgery are more likely due to bacteria already on your skin than from microbes in the hospital − new research

Honoring our veterans

Supporting the multi-generational nursing workforce

Vital practitioners

From data to action

Many travel nurses opt for temporary assignments because of the autonomy and opportunities − not just the big boost in pay

Effective clinical learning for nursing students

Nurse safety in the era of open notes

Collaboration: The key to patient care success

Health workers fear it’s profits before protection as CDC revisits airborne transmission

Why COVID-19 patients who could most benefit from Paxlovid still aren’t getting it

Human touch

Leadership style matters

My old stethoscope

Data Collection and Entry

Tips for collecting qualitative and quantitative data including available platforms for survey research, strategies for conducting interviews, ways of coding and organizing different types of data, guidance for entering data into a database, and transcription services recommended by UW-Madison. Links to training guides for various software for data analysis and guidance for handling missing data. Main menu | Comments/Suggestions

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Data Collection | Definition, Methods & Examples

Data Collection | Definition, Methods & Examples

Published on June 5, 2020 by Pritha Bhandari . Revised on June 21, 2023.

Data collection is a systematic process of gathering observations or measurements. Whether you are performing research for business, governmental or academic purposes, data collection allows you to gain first-hand knowledge and original insights into your research problem .

While methods and aims may differ between fields, the overall process of data collection remains largely the same. Before you begin collecting data, you need to consider:

• The  aim of the research
• The type of data that you will collect
• The methods and procedures you will use to collect, store, and process the data

To collect high-quality data that is relevant to your purposes, follow these four steps.

Table of contents

Step 1: define the aim of your research, step 2: choose your data collection method, step 3: plan your data collection procedures, step 4: collect the data, other interesting articles, frequently asked questions about data collection.

Before you start the process of data collection, you need to identify exactly what you want to achieve. You can start by writing a problem statement : what is the practical or scientific issue that you want to address and why does it matter?

Next, formulate one or more research questions that precisely define what you want to find out. Depending on your research questions, you might need to collect quantitative or qualitative data :

• Quantitative data is expressed in numbers and graphs and is analyzed through statistical methods .
• Qualitative data is expressed in words and analyzed through interpretations and categorizations.

If your aim is to test a hypothesis , measure something precisely, or gain large-scale statistical insights, collect quantitative data. If your aim is to explore ideas, understand experiences, or gain detailed insights into a specific context, collect qualitative data. If you have several aims, you can use a mixed methods approach that collects both types of data.

• Your first aim is to assess whether there are significant differences in perceptions of managers across different departments and office locations.
• Your second aim is to gather meaningful feedback from employees to explore new ideas for how managers can improve.

Prevent plagiarism. Run a free check.

Based on the data you want to collect, decide which method is best suited for your research.

• Experimental research is primarily a quantitative method.
• Interviews , focus groups , and ethnographies are qualitative methods.
• Surveys , observations, archival research and secondary data collection can be quantitative or qualitative methods.

Carefully consider what method you will use to gather data that helps you directly answer your research questions.

When you know which method(s) you are using, you need to plan exactly how you will implement them. What procedures will you follow to make accurate observations or measurements of the variables you are interested in?

For instance, if you’re conducting surveys or interviews, decide what form the questions will take; if you’re conducting an experiment, make decisions about your experimental design (e.g., determine inclusion and exclusion criteria ).

Operationalization

Sometimes your variables can be measured directly: for example, you can collect data on the average age of employees simply by asking for dates of birth. However, often you’ll be interested in collecting data on more abstract concepts or variables that can’t be directly observed.

Operationalization means turning abstract conceptual ideas into measurable observations. When planning how you will collect data, you need to translate the conceptual definition of what you want to study into the operational definition of what you will actually measure.

• You ask managers to rate their own leadership skills on 5-point scales assessing the ability to delegate, decisiveness and dependability.
• You ask their direct employees to provide anonymous feedback on the managers regarding the same topics.

You may need to develop a sampling plan to obtain data systematically. This involves defining a population , the group you want to draw conclusions about, and a sample, the group you will actually collect data from.

Your sampling method will determine how you recruit participants or obtain measurements for your study. To decide on a sampling method you will need to consider factors like the required sample size, accessibility of the sample, and timeframe of the data collection.

Standardizing procedures

If multiple researchers are involved, write a detailed manual to standardize data collection procedures in your study.

This means laying out specific step-by-step instructions so that everyone in your research team collects data in a consistent way – for example, by conducting experiments under the same conditions and using objective criteria to record and categorize observations. This helps you avoid common research biases like omitted variable bias or information bias .

This helps ensure the reliability of your data, and you can also use it to replicate the study in the future.

Creating a data management plan

Before beginning data collection, you should also decide how you will organize and store your data.

• If you are collecting data from people, you will likely need to anonymize and safeguard the data to prevent leaks of sensitive information (e.g. names or identity numbers).
• If you are collecting data via interviews or pencil-and-paper formats, you will need to perform transcriptions or data entry in systematic ways to minimize distortion.
• You can prevent loss of data by having an organization system that is routinely backed up.

Finally, you can implement your chosen methods to measure or observe the variables you are interested in.

The closed-ended questions ask participants to rate their manager’s leadership skills on scales from 1–5. The data produced is numerical and can be statistically analyzed for averages and patterns.

To ensure that high quality data is recorded in a systematic way, here are some best practices:

• Record all relevant information as and when you obtain data. For example, note down whether or how lab equipment is recalibrated during an experimental study.
• Double-check manual data entry for errors.
• If you collect quantitative data, you can assess the reliability and validity to get an indication of your data quality.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Likert scale

Research bias

• Implicit bias
• Framing effect
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

When conducting research, collecting original data has significant advantages:

• You can tailor data collection to your specific research aims (e.g. understanding the needs of your consumers or user testing your website)
• You can control and standardize the process for high reliability and validity (e.g. choosing appropriate measurements and sampling methods )

However, there are also some drawbacks: data collection can be time-consuming, labor-intensive and expensive. In some cases, it’s more efficient to use secondary data that has already been collected by someone else, but the data might be less reliable.

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

Reliability and validity are both about how well a method measures something:

• Reliability refers to the  consistency of a measure (whether the results can be reproduced under the same conditions).
• Validity   refers to the  accuracy of a measure (whether the results really do represent what they are supposed to measure).

If you are doing experimental research, you also have to consider the internal and external validity of your experiment.

Operationalization means turning abstract conceptual ideas into measurable observations.

For example, the concept of social anxiety isn’t directly observable, but it can be operationally defined in terms of self-rating scores, behavioral avoidance of crowded places, or physical anxiety symptoms in social situations.

Before collecting data , it’s important to consider how you will operationalize the variables that you want to measure.

In mixed methods research , you use both qualitative and quantitative data collection and analysis methods to answer your research question .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2023, June 21). Data Collection | Definition, Methods & Examples. Scribbr. Retrieved April 17, 2024, from https://www.scribbr.com/methodology/data-collection/

Is this article helpful?

Pritha Bhandari

Other students also liked, qualitative vs. quantitative research | differences, examples & methods, sampling methods | types, techniques & examples, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Fastest Nurse Insight Engine

• MEDICAL ASSISSTANT
• Abdominal Key
• Anesthesia Key
• Basicmedical Key
• Otolaryngology & Ophthalmology
• Musculoskeletal Key
• Obstetric, Gynecology and Pediatric
• Oncology & Hematology
• Plastic Surgery & Dermatology
• Clinical Dentistry
• Radiology Key
• Thoracic Key
• Veterinary Medicine
• Gold Membership

Collecting and Managing Data

Chapter 20 Collecting and Managing Data   http://evolve.elsevier.com/Grove/practice/ Data collection is one of the most exciting parts of research. After all the planning, writing, and negotiating, you should be eager and well prepared for this active part of research. The passion that comes from wanting to know the answer to your research question brings a sense of excitement and eagerness to start collecting your data. However, before you leap into data collection, you need to spend some time carefully planning this adventure and pilot test each step. Planning data collection begins with identifying all the data to be collected. The data to be collected are determined by the research questions, objectives, or hypotheses of the proposed study. As you develop the data collection plan, be sure that you gather all the data needed to answer the research questions, achieve the study objectives, or test the hypotheses. Chapter 16 includes detailed information about measurement, so the focus in this chapter is on the logistical and pragmatic aspects of quantitative data collection. Data collection strategies for qualitative studies are described in Chapter 12 . To start planning the data collection process, you need to determine the best mode by which the data can be collected. Factors that influence the plan to collect and enter data into a database for analysis include cost, time, the availability of assistance, and the need for consistency. The development of the data collection plan is followed by developing data collection forms and a codebook for data entry. Conducting a pilot test with a small group of subjects is the next recommended step. The pilot test may result in modifications of the plan, and then the actual data collection can begin. During data collection, various problems may arise. Potential situations are described in this chapter along with problem-solving strategies. The chapter concludes with the discussion of data entry and management. Data Collection Modes Data can be collected by interview (face-to-face or telephone); observations; focus groups; self-administered questionnaires (online or hard copy); or extraction from existing documents such as patient medical records, motor vehicle department accident records, or state birth records ( Figure 20-1 ). Many factors need to be considered when a researcher is deciding on the mode for collecting data. Harwood and Hutchinson (2009) describe four factors that need to be part of your decision-making process: (1) purpose and complexity of the study, (2) availability of financial and physical resources, (3) characteristics of study participants and how best to gain access to them from the population, and (4) your skills and preferences as a researcher. Figure 20-1 Data collection modes. Researcher-Administered or Participant-Administered Instruments If you need a subject’s accurate blood pressure or height and weight, a self-report measure may be neither valid nor reliable for the purpose of your study. However, if the purpose of your study can be accomplished with a self-report survey method, you must decide whether the format will be researcher-administered or self-administered. It may be best for the researcher to administer self-report paper-and-pencil instruments if the potential subjects have minimal language or literacy ability, whereas it may be best to consider electronic data collection or medical record extraction if the subjects are likely to have hearing impairments, transportation problems, or physical difficulties. If the researcher is administering the survey, will it be in person or by telephone? If self-administered, will the participant complete a pencil-and-paper copy or an online electronic copy? Internet survey centers specialize in this mode of data collection and have expert help or tutorials for assessing the best mode for your study purpose. For example, in deciding on a telephone survey, how many times will you try to reach a potential subject before you give up, what days of the week or hours of the day will you call and how might that bias your sample or their responses, and how will you accurately determine the response rate ( Harwood, 2009 )? If you decide on a mailed paper-and-pencil survey, what will you do with undelivered or incomplete returns? Will you search for correct mailing addresses and try again? Will you send a reminder if the survey is not received within a particular time frame, and, if so, what time frame will you give a respondent, and how many reminders will you send ( Harwood, 2009 )? Electronic Data Collection When you are using an existing instrument, you may need permission to convert the questions into an online format, a special type of form that allows the data to be scanned into a database, or into an application for a phone or other electronic device. Each of these modes of data collection may require special hardware and software. Universities, schools of nursing, and funded researchers are purchasing these sometimes expensive products because the costs of acquiring the hardware and software are considerably less than the costs of entering data manually. Scannable Forms Other software allows the preparation of special data collection forms that rely on optical character recognition (OCR), which requires exact placement on the page for each potential response. To maintain the precise location of each response on print copies of these instruments, careful attention must be given to printing or copying these forms. The complete form is scanned, and the answers (data) are automatically recorded in a database. Additional features include data accuracy verification, selective data extraction and analysis, auditing and tracking, and flexible export interfaces. Figure 20-2 shows the scannable version of the Parents and Newborn Screening Survey developed by Patricia Newcomb, PhD, RN, CPNP, and Barbara True, MSN, CNS. Subjects completing the survey fill in the circle that corresponds to the appropriate option for each question. Figure 20-2 Scannable form: Parents and Newborn Screening Survey. (Developed by Patricia Newcomb, PhD, RN, CPNPN, and Barbara True, MSN, CNS; Teleform designed by Denise Cauble and Whitney Mildren, Graduate Research Assistants and PhD students, College of Nursing, The University of Texas at Arlington.) Online Data Collection Computer software packages developed by a variety of companies (e.g., Zoomerang and SurveyMonkey) enable researchers to provide an online copy of instruments and other data collection forms. These types of software programs have unique features that allow the researcher to develop point-and-click automated forms that can be distributed electronically. The following questions need to be considered with use of these programs. For an online survey, is it a secure site for the purposes of confidentiality and anonymity? How will you ensure that only eligible participants complete the survey? Will potential subjects receive a personalized email from you with a link to a website? How will you obtain the email addresses? Can you offer help if the subjects have any questions about your study? Online services can be easy to use for both the researcher and study participants but may be costly and require specific assurances about confidentiality of data and anonymity of subjects. The National Institutes of Health (NIH) supports a secure Internet environment for building online data surveys and data management packages ( Harris et al., 2009 ). This service, developed by experts at Vanderbilt University, is called REDCap (Research Electronic Data Capture) and may be available at your university research site ( http://project-redcap.org/ ). Im et al. (2007) conducted a survey in the United States of gender and ethnic differences in the experience of cancer pain. These researchers administered their questionnaire over the Internet and through a paper-and-pencil format based on subject preference. The following excerpt describes the data collection procedure for their study: “To administer the Internet questionnaire, a Web site conforming to the Health Insurance Portability and Accountability Act standards, the System Administration, Networking, and Security Institute Federal Bureaus of Investigation recommendations, and the Institutional Review Board [IRB] policy of the institution where the researchers were affiliated was developed and published on an independent, dedicated Web site server. When potential participants visited the project Web site, informed consent was obtained by asking them to click a button labeled I agree to participate . After this, questions on specific diagnoses, cancer therapies, and medications were asked, and the appropriateness of answers was checked automatically through a server-side program; participants were connected automatically to the Internet survey web page if the answers were appropriate. “Upon request, pen-and-pencil questionnaires were provided by mail to the community consultants, who distributed the questionnaires in person only to those who were identified as cancer patients. These questionnaires accompanied hard copies of the same informed consent form included in the Internet format of the questionnaire, and the pen-and-pencil questionnaire included a sentence ‘Filling out this questionnaire means that you are aged over 18 years old and giving your consent to participate in this survey.’ After the self-administered questionnaires were completed, community consultants retrieved all except five (these were mailed directly to the research team by the participants) in person at the community settings and mailed them to the research team. Supplementing pen-and-pencil questionnaires was essential to recruit the target number of ethnic minority cancer patients across the nation who did not have access to the Internet but were interested in participating in the study. Among the 276 participants who were recruited through community settings, 246 … used the pen-and-pencil questionnaires. … There were no statistically significant differences in psychometric properties between the Internet format and the pen-and-pencil format of the questionnaire. … It took an average of 30-40 minutes for the participants to complete either the Internet format or the pen-and-pencil format of the questionnaire.” ( Im et al., 2007 , pp. 299-300) Im et al. (2007) maximized their sample size and obtained a more representative sample by giving participants an option to complete their questionnaire on the Internet or using paper-and-pencil format. The researchers took steps to ensure that the data collected by the two formats were comparable by testing for significant differences and finding none. The time to complete the Internet and paper-and-pencil questionnaires did not vary. Im et al. (2007) also ensured that an ethical study was conducted and subjects’ rights were protected. The additional advantage of Internet data collection is that responses can be time/date stamped. For example, if subjects are instructed to complete the questionnaire before bedtime, the time can be verified. If subjects are instructed to complete a daily diary, date of entry would be documented, and subjects would be discouraged from entering all diary days on the last day just before returning the diary to the researcher ( Fukuoka, Kamitani, Dracup, & Jong, 2011 ). Computer-Based Data Collection With the advent of laptop and tablet computers, data collectors can code data directly into an electronic file at the data collection site. If a computer is used for data collection, a program must be written for entering, cleaning, and storing data. A computer enables users to collect large amounts of data with few errors that can be readily analyzed with a variety of statistical software packages. In addition to researchers using technology at the point of data collection to record data, technology has made it possible to interface physiological monitoring systems with computers for data collection. An advantage of using computers for the acquisition and storage of physiological data is the increased accuracy and precision that can be achieved by reducing errors associated with manually recording or transcribing physiological data from a monitor. Another advantage is that more data points can be recorded electronically than could be recorded manually. Computers linked to physiological monitoring systems can store multiple data for multiple indicators, such as blood pressures, oxygen saturation levels, and sleep stages. Because data can be electronically recorded, data collection is less labor intensive, and the data are ready to analyze more quickly. The initial cost of equipment may be high, but it is reasonable when the cost of hiring and training human data collectors is considered. There are some concerns with the use of computerized data acquisition systems, but physiological data are usually best gathered and stored directly into a computer database to ensure accurate, complete data collection. Physiological data typically require large computer storage space. The computer-equipment interface may require more space in an already crowded clinical setting; when possible, existing equipment should be used to collect data. Purchasing the equipment, setting it up, and installing the software can be time-consuming and expensive at the start of your project. Thus, initial studies usually require substantial funding. Another concern is that the nurse researcher may focus on the machine and technology and neglect observing and interacting with the subject. The most serious disadvantage of computerized data collection is the possibility of measurement error that can occur with equipment malfunctions and software errors. Regular maintenance and calibrations, or reliability checks of the equipment and software, reduce this problem. The benefits of collecting repeated measures over time may outweigh the risk of missing data because of poor compliance. For example, collecting continuous rectal temperature data from a subject is easier and less burdensome than asking the subject to measure an oral temperature every 1 to 2 hours. Savian, Paratz, and Davies (2006) conducted a single-blind randomized, crossover study with 14 mechanically ventilated intensive care unit patients. “[The purpose of the study was to determine the effectiveness of] manual hyperinflation (MHI) and ventilator hyperinflation (VHI) on respiratory mechanics (static compliance [C st ]), oxygenation (arterial oxygen tension [PaO 2 ]/fraction of inspired oxygen [FIO 2 ] ratio), and secretion removal (wet weight of sputum and peak expiratory flow rate [PEFR]) at different levels of PEEP [positive end-expiratory pressure] … a secondary aim was to investigate the hemodynamics heart rate [HR], mean arterial pressure [MAP] and metabolic response (carbon dioxide output [VCO 2 ]) during MHI and VHI.” ( Savian et al., 2006 , p. 335) The computerized systems used to collect and record data in the study by Savian et al. (2006) are detailed in the following excerpt: “PEFR and CO 2 [carbon dioxide] production were measured using a flow and CO 2 sensor connected to the patient’s airways and to the CO 2 SMO [carbon dioxide] respiratory mechanics monitor (CO 2 SMO Plus Model 8000, Novametrix Medical Systems Inc., Wallingford, CT). All information from the CO 2 SMO monitor was simultaneously recorded in the Analysis Plus computer program. “Static lung compliance was recorded by the static measures function device on the Bennett 7200 ventilator where a plateau pressure was obtained by including an inspiratory pause of 2 seconds into the mandatory breath. … “PaO 2 /FIO 2 ratio was calculated from arterial blood samples taken immediately before and immediately after MHI and VHI. Four milliliters of arterial blood were drawn into a syringe containing heparin and analyzed by a blood gas machine (Bayer Australian Limited 865, Pymble, NSW, CAN 000128 714). This procedure was standardized across subjects. “HR and MAP were read directly from the monitoring system (Merlin pressure module M1006A Hewlett Packard, Palo Alto, CA) and recorded every minute before, during, and for 5 minutes after MHI and VHI.” ( Savian et al., 2006 , p. 336) The use of computerized data collection by Savian et al. (2006) enabled them to collect repeated measures on several physiological variables in an accurate and precise way. The data were collected by sensors and stored in the computer to reduce error and facilitate data analysis. Phones and Other Electronic Devices Software applications for mobile phones have evolved from personal digital assistants (PDAs) that allow the researcher to collect and download data directly into the computer from observations as they occur. Healthcare providers load applications that facilitate accurate assessment, diagnosis, and pharmacological and nonpharmacological management of patients. PDAs are also used to store deidentified data from office computers in a form that is easily transportable. PDA software is currently available that may help nurse practitioners collect data for research. Multiple nurse practitioners involved in a research project could forward data electronically from PDAs to a central research site for analysis. Encrypted electronic devices are needed to protect the confidentiality of data during transmission. These electronic devices can be misplaced or stolen, threatening confidentiality. Researchers need to protect the data with a security code to ensure that no one but themselves can access data in these formats. Mobile phones and computers are becoming more similar with the increased sophistication of applications for mobile phones. Some of these applications can be used to collect various data. Other electronic devices include pill containers that record when pills are accessed and watches with timers to remind participants to take certain health-related actions. However, the use of these devices for research may require considerable preparation. You may need to hire programmers with the needed expertise, and you may need to purchase, rent, or borrow the needed number of devices or monitors. Factors Influencing Data Collection When planning data collection, cost, time, the availability of assistance, and the need for consistency are critical factors to consider. The researcher balances these factors with the need to maintain the reliability and validity of the study in the development of the data collection plan. Cost Factors Cost is a major consideration when planning a study. Measurement tools, such as continuous electrocardiogram monitors (Holter monitor), wrist activity monitors (accelerometers), spirometers, pulse oximeters, or glucometers, used in physiological studies may need to be rented, purchased, or loaned from the manufacturer or other company. You may need to pay a fee to use instruments or questionnaires. Some instruments and questionnaires are available only if a copy is purchased for each participant. Data collection forms may need to be formatted or developed for electronic use. In some cases, printing costs for materials such as teaching materials or questionnaires that will be used during the study must be considered. Providing the required copy of the signed consent form doubles the expense of consent forms. Small payments to participants in the form of cash or gift cards should be considered as compensation for a subject’s time and effort in providing the data. Sometimes childcare may need to be provided for parents and other caregivers who would not otherwise be able to participate in your study. In some studies, postage is an additional expense. There may be costs involved in coding the data for entry into the computer and for conducting data analyses. Consultation with a statistician early in the development of a research project and during data analysis must also be budgeted. You may need to hire someone who can remain blinded for data entry or analysis or someone who can type the final report, develop graphics or presentations, or type and edit manuscripts for publication. In addition to the above-described direct costs of a research project, there are costs associated with the researcher’s time and travel to and from the study site. You also must estimate the expense of presenting the research findings at conferences and include those expenses in the budget. To prevent unexpected expenses from delaying the study, examine all costs in an organized manner. A budget is best developed early in the planning process and revised as plans are modified. Seeking funding for at least part of the study costs can facilitate the conduct of a study. Time Factors Researchers often underestimate the time required for participants to complete data collection forms and for the research team to recruit and enroll subjects for a study. The first aspect of time—the participant’s time commitment—must be determined early in the process because the time needed for participant involvement must be included in the informed consent process and document. While conducting your pilot study, make note of the time required to collect data from a subject. You may need to revise your timeline and consent form to reflect the expected time commitment accurately. The second aspect of time—the time needed to complete data collection—is especially challenging to predict because events during the data collection period sometimes are not under the researcher’s control. For example, a sudden heavy staff workload may make data collection temporarily difficult or impossible, or the number of potential subjects might be reduced for a period. In some situations, researchers must obtain permission from each subject’s physician before they are permitted to collect data on that subject. Activities required for this stipulation, such as contacting physicians, explaining the study, and obtaining permission, require extensive time. In some cases, potential subjects are lost before the researcher can obtain the mandatory permission, extending the time required to obtain the necessary number of subjects. How long will it take to identify potential subjects, explain the study, and obtain consent? How much time will be needed for activities such as completing questionnaires or obtaining physiological measures? Novice researchers have difficulty making reasonable estimates of time and costs related to a study. Validating the time and cost estimates with an experienced researcher can be very informative. Experienced researchers know the challenges of data collection and have learned that data collection may take two to three times longer than predicted. If the cost and time factors are prohibitive, you may need to simplify your study so that fewer variables are measured, fewer instruments are used, or fewer subjects are needed. Make the design less complex, and use fewer data collectors. A blinded intervention study involves more research staff and is generally not feasible for a novice researcher. These are serious modifications, however, with implications for the validity of the findings, so you and your team should thoroughly examine the consequences before making such revisions. If preliminary time or cost estimates go beyond expectations, you can revise the time schedules and budget with new projections for completing the study. Consistency Consistency in data collection across subjects is critical. What time of year will data be collected? For example, if you collect data during holiday seasons, data about sleeping, eating, or exercising may vary. Pediatric patients with asthma may experience more symptoms during the winter months than during summer. Planning data collection for a study of symptom management with this population would need to take this possibility into consideration. The specific days and hours of data collection may influence the consistency of the data collected and must be carefully considered. For example, the energy level and state of mind of subjects from whom data are gathered in the morning may differ from that of subjects from whom data are gathered in the evening. With hospitalized study participants, visitors are more likely to be present at certain times of day and may interfere with data collection or influence participant responses. Patient care routines vary with the time of day. In some studies, the care recently received or the care currently being provided may alter the data you gather. The subjects you approach on Saturday to participate in the study may differ from the subjects you approach on weekday mornings. Subjects seeking care on Saturday may have a full-time job, whereas subjects seeking care on weekday mornings may be either unemployed or too ill to work. Who will collect the data? If you decide to use data collectors, they must be trained in responsible conduct of research and issues of informed consent, ethics, and confidentiality and anonymity (see Chapter 9 ). They must be informed about the research project, familiar with the instruments to be used, and have equivalent training in the data collection process. In addition to training, data collectors need written guidelines or protocols that indicate which instruments to use, the order in which to introduce the instruments, how to administer the instruments, and a time frame for the data collection process ( Harwood, 2009 ; Kang, Davis, Habermann, Rice, & Broome, 2005 ). If more than one person is collecting the data, consistency among data collectors (interrater reliability) must be ensured through testing (see Chapter 16 ). The training needs to continue until interrater reliability estimates are at least 85% to 90% agreement between the expert and the trainee or trainees. Waltz, Strickland, and Lenz (2010) suggest that a minimum of 10% of the data needs to be compared across raters before interrater reliability can be adequately reported. The trained data collector’s interrater reliability with the expert trainer should be assessed intermittently throughout data collection to ensure consistency from the first to the last participant in the study. Data collectors also must be encouraged to identify and record any problems or variations in the environment that affect the data collection process. The description of the training of the data collectors is usually reported in the methods section of an article so that others can assess the data collection process ( Harwood & Hutchinson, 2009 ). Availability of Assistance Who is going to help you with the study? If you are a student, will your mentor or supervising faculty member participate? Does your mentor or supervising faculty member have research assistants who could assist in your study? Will nurses, physicians, and other health professionals assist with recruitment? Do they have time to do this? Are they willing to help? Will the researcher collect all the data, or will data collectors be employed for this purpose? Can data collectors be nurses working in the area? Data collection may be delayed when nurses providing patient care are also expected to be data collectors. Even when a nurse agrees to help you with subject recruitment or data collection, patient care takes priority over data collection and increases the risk for missing data or missing the opportunity to enroll eligible subjects. If clinicians are going to recruit subjects or collect data, the clinicians need to complete training for protection of human subjects during research. An IRB requires documentation of this training for each person involved in recruitment and data collection. If you are going to be doing all the data collection yourself, will you be available every day of the week? What hours will you be available? If others will be involved in collecting data, allow time for training on data collection procedures. You need to be available by telephone or other means for questions and emergencies when others are collecting data for your study. Keeping these factors in mind, you are now ready to plan the data collection process for your study. Data Collection and Coding Plan The factors of cost, time, availability of assistance, and need for consistency shape the data collection plan that you develop. A data collection plan details how you will implement your study. The plan for collecting data is specific to the study being conducted and requires that you consider some common elements of research. You need to map out procedures you will use to collect data, anticipate the time and cost of data collection, develop data collection forms that ease data entry, and prepare a codebook that will help you to code the variables to be entered in a database. This extensive planning increases the accuracy of the data collected and the validity of the study findings. The validity and strength of the findings from several carefully planned studies increase the quality of the research evidence that is then available for implementing into clinical practice ( Melnyk & Fineout-Overholt, 2010 ). Identifying data include variables such as patient record number, home address, and date of birth (see Chapter 9 ). Avoid collecting these data unless they are essential to answer the research question. For example, collect a patient’s age instead of date of birth. Review regulations by the Health Insurance Portability and Accountability Act about the participant’s private health information ( www.hhs.gov/ocr/hipaa ). The methodology of a study may include contacting subjects later for additional data collection. In this case, you will need to obtain the subject’s address and telephone number and protect the information appropriately. Names and phone numbers of family members or friends may also be useful if subjects are likely to move or may be difficult to contact. This information can be obtained only with subjects’ permission as part of their informed consent. Consider the importance of each piece of data and the subject’s time required to collect it. If the data can be obtained from patient records or any other written sources, you do not need to ask the subject to provide this information. To collect data from a patient’s records, make sure to include permission to do this in the consent form, and ensure that the IRB has authorized your team to do this. Data Collection Forms Before data collection begins, you may need to develop or modify forms on which to record data. These forms can be used to record demographic data, information from the patient record, observations, or values from physiological measures. The demographic variables commonly collected in nursing studies include age, gender, race, education, income, employment status, diagnosis, and marital status. You may want to collect additional demographic data if researchers have identified participant characteristics that affect the study variables. You also might need to collect other data that may be extraneous or confounding variables, such as the subject’s physician, stage of illness, length of illness or hospitalization, complications, date of data collection, time of day and day of week of data collection, and any untoward events that occur during the data collection period. If there are only women in your sample, the subject’s age and reproductive status, parity, and number of children in the home may be confounding variables. In a study of patients with ventilator-associated pneumonia, the researcher needs to record the length of time between when the patient was intubated and when ventilator-associated pneumonia was diagnosed. The researcher for this study also needs to record whether the patient had a preexisting pulmonary disease. Data collection forms must be designed so that the data are easily recorded, coded, and entered into the computer. You need to decide whether data will be collected in raw form or coded at the time of collection. Coding in quantitative studies is the process of transforming data into numerical symbols that can be entered easily into the computer. For example, variables such as race, gender, ethnicity, and diagnoses can be categorized and given numerical labels. For gender, the male category could be identified by a “1” and the female category by a “2.” You may also want to include an “other” category (coded “3”) for participants who are transgendered or transsexual. To be able to compare your sample with samples in federally funded studies, you may need to separate the questions about ethnicity and race. In 2003, the Office of Management and Budget of the U.S. government directed researchers and others collecting data for federal purposes or at federal expense to separate the questions of race and ethnicity ( Office of Minority Health, 2010 ). At the same time, the Office of Management and Budget specified the categories for each. The following questions are correct according to these federal guidelines. How would a subject who is biracial or multiracial complete the form? You may want to word the question to ask the participant’s primary race or allow multiple responses. Ethnicity (1)  Hispanic or Latino (2)  Non-Hispanic or Latino

Share this:

• Click to share on Twitter (Opens in new window)
• Click to share on Facebook (Opens in new window)

Related posts:

• Interpreting Research Outcomes
• Review of Relevant Literature
• Disseminating Research Findings
• Selecting a Quantitative Research Design

Stay updated, free articles. Join our Telegram channel

Comments are closed for this page.

Full access? Get Clinical Tree

Collecting Nursing Research Data 24 Hours a Day: Challenges, Lessons, and Recommendations

Affiliations.

• 1 Mary Lou Sole is dean and professor, Orlando Health Endowed Chair in Nursing, University of Central Florida College of Nursing, and clinical scientist, Orlando Health, Orlando, Florida. Steven Talbert is assistant clinical professor, University of Central Florida College of Nursing. Melody Bennett is study project director, Orlando Health. Aurea Middleton is clinical research coordinator, Orlando Health. Lara Deaton is clinical research coordinator, Orlando Health. Daleen Penoyer is director, Center for Nursing Research, Orlando Health, Orlando. [email protected].
• 2 Mary Lou Sole is dean and professor, Orlando Health Endowed Chair in Nursing, University of Central Florida College of Nursing, and clinical scientist, Orlando Health, Orlando, Florida. Steven Talbert is assistant clinical professor, University of Central Florida College of Nursing. Melody Bennett is study project director, Orlando Health. Aurea Middleton is clinical research coordinator, Orlando Health. Lara Deaton is clinical research coordinator, Orlando Health. Daleen Penoyer is director, Center for Nursing Research, Orlando Health, Orlando.
• PMID: 29961666
• PMCID: PMC6050978
• DOI: 10.4037/ajcc2018448

Background: Research on many routine nursing interventions requires data collection around the clock each day (24/7). Strategies for implementing and coordinating a study 24/7 are not discussed in the literature, and best practices are needed.

Objective: To identify strategies incorporated into implementing a nursing intervention trial 24/7, including key lessons learned.

Methods: Strategies to facilitate implementation of a clinical trial of a nursing intervention with patients undergoing mechanical ventilation are shared. Challenges and changes for future studies also are discussed.

Results: Adequate planning, including a detailed operations manual, guides study implementation. Staffing is the most challenging and costly part of a study but is essential to a study's success. Other important strategies include communication among the study personnel and with collaborators and direct care staff. An electronic method of recording study-related data also is essential.

Conclusions: A nursing clinical trial that requires interventions on a 24/7 basis can be done with thorough planning, staffing, and continuous quality improvement activities.

©2018 American Association of Critical-Care Nurses.

Publication types

• Clinical Trial Protocol
• Research Support, N.I.H., Extramural
• Communication
• Data Collection* / methods
• Data Collection* / standards
• Inservice Training
• Nursing Research* / economics
• Nursing Research* / organization & administration
• Nursing Research* / standards
• Oral Hygiene / methods
• Personnel Staffing and Scheduling
• Randomized Controlled Trials as Topic
• Research Design
• Respiration, Artificial / adverse effects
• Respiratory Aspiration / prevention & control
• Time Factors

Grants and funding

• R01 NR014508/NR/NINR NIH HHS/United States

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Appl Clin Inform
• v.6(1); 2015

Data Collection Methods in Health Services Research

M.n. sarkies.

2 Monash Health, Allied Health, Melbourne, Victoria, Australia

3 Monash Health, Allied Health Research Unit, Melbourne, Victoria, Australia

K.-A. Bowles

7 Monash University, Physiotherapy Department, Allied Health Research Unit, Melbourne, Victoria, Australia

E.H. Skinner

4 Monash University, Allied Health Research Unit, Melbourne, Victoria, Australia

8 Western Health, Allied Health, Melbourne, Victoria, Australia

D. Mitchell

6 Monash University, Physiotherapy Department, Melbourne, Victoria, Australia

L. O’Brien

5 Monash University, Occupational Therapy Department, Melbourne, Victoria, Australia

1 Melbourne Health, Allied Health, Melbourne, Victoria, Australia

T.P. Haines

Hospital length of stay and discharge destination are important outcome measures in evaluating effectiveness and efficiency of health services. Although hospital administrative data are readily used as a data collection source in health services research, no research has assessed this data collection method against other commonly used methods.

Determine if administrative data from electronic patient management programs are an effective data collection method for key hospital outcome measures when compared with alternative hospital data collection methods.

Prospective observational study comparing the completeness of data capture and level of agreement between three data collection methods; manual data collection from ward-based sources, administrative data from an electronic patient management program (i.PM), and inpatient medical record review (gold standard) for hospital length of stay and discharge destination.

Manual data collection from ward-based sources captured only 376 (69%) of the 542 inpatient episodes captured from the hospital administrative electronic patient management program. Administrative data from the electronic patient management program had the highest levels of agreement with inpatient medical record review for both length of stay (93.4%) and discharge destination (91%) data.

This is the first paper to demonstrate differences between data collection methods for hospital length of stay and discharge destination. Administrative data from an electronic patient management program showed the highest level of completeness of capture and level of agreement with the gold standard of inpatient medical record review for both length of stay and discharge destination, and therefore may be an acceptable data collection method for these measures.

1. Background

Hospital length of stay and discharge destination are important outcome measures used in health services research. Length of stay is often used as a measure of healthcare efficiency by researchers, clinicians, administrators, and policy makers in planning the delivery of health services [ 1–4 ]. Hospital discharge destination is an influencing factor on length of stay [ 3 ] providing a means of quantifying numerous measures such as; requirements for sub-acute inpatient care; changes in level of care; requirement for community services following discharge, and hospital death. Due to their importance, researchers use these measures as key indicators of effectiveness and efficiency when evaluating hospital service provision.

There are numerous methods by which data may be collected for research and hospital administrative purposes. Observational length of stay and discharge destination data can be manually collected from ward-based sources including; nursing handover records, paper-based ward discharge/transfer records, paper-based inpatient medical records, direct observation by experienced personnel, and 24hour recall of key hospital personnel (e.g. Nurse Unit Manager). However this is a time-intensive data collection method which is difficult to fund in the current environment where research funding is increasingly more competitive. Retrospective data may be collected via review of scanned inpatient medical records post hospital discharge. While this approach has previously been used as a gold standard measure for multiple outcomes [ 5–11 ], transforming medical records into research data is resource intensive and requires exceptional knowledge and skill in medical context and research [ 12 ]. An alternative to these traditional methods of hospital data collection has been to extract electronic administrative data. Retrospective hospital administrative data has become a commonly used source of inexpensive and readily available information. Administrative data is not normally entered specifically for research purposes, with previous literature indicating the use of administrative data in adverse events and coding for billing purposes may result in inaccurate data [ 12–23 ].

Interestingly despite the importance and frequent reporting of hospital length of stay and discharge destination measures, we were unable to identify any published empirical research comparing methods of data collection for these outcome measures. With this range of potential data sources and data collection approaches, it is important to consider the relative completeness and agreement between different data extraction methods. This research is therefore essential to ensure the validity of data collection for research that is used to inform decision making around health policy and clinical care.

2. Objectives

The purpose of this study was therefore to determine the completeness of capture and level of agreement between three different data collection approaches in measuring length of stay and discharge destination. These were:

• Observational data manually collected from ward-based sources by a research assistant
• retrospective administrative data extraction from an electronic patient management program (i.PM), and
• retrospective review of scanned inpatient medical records post discharge from hospital (gold standard).

3.1 Study setting

This study was performed in conjunction with a larger stepped-wedge randomised controlled trial examining the effectiveness of acute weekend allied health services [ 24 ] and was approved by Monash Health Human Ethics Committee (Reference Number 13327B). The study was conducted at a major 520 bed public hospital providing acute and sub-acute services in urban South-East Melbourne, Australia and occurred during the first two weeks in February 2014. The study period and wards were selected in accordance with the stepped-wedge randomised controlled trial and included the acute assessment unit, neurosciences, plastics, surgical, orthopaedic, and two general medical wards. As there were no exclusion criteria, the study cohort included the total sample of consecutive individuals discharged from the study wards during the study period.

3.2 Outcome measures

Two outcome measures from the stepped-wedge randomised controlled trial were used in this analysis. These were selected as they are outcome measures in the larger project that were extracted from multiple sources and are key indicators of inpatient hospital effectiveness and efficiency.

• Hospital length of stay . Hospital length of stay was reported in days and was determined by subtracting the date of hospital admission from the date of hospital discharge.
• Discharge destination . Discharge destination is the location the patient is residing immediately post hospital discharge and can include: home, other hospital, rehabilitation facility, other supported residential facility (including retirement villages, supported residential services, respite and transitional care), low level care (hostel), high level care (nursing home) or death.

3.3 Data collection approaches

Five research assistants from allied health backgrounds collected hospital admission date, hospital discharge date and discharge destination via three methods. All research assistants received on-site training from a hospital researcher prior to collecting data.

• Observational data manually collected by four research assistants from ward-based sources including nursing handover records, paper-based inpatient medical records, paper-based ward discharge/transfer records and verbal handover from ward staff based on the previous 24 hours. This data collection method was a pragmatic approach intended to replicate how this data would be collected in a large clinical trial with limited resources. Nursing handover records were updated daily by the nurse in charge. Ward transfer records were updated continuously by ward administrative staff Monday to Friday between 0730 – 2000 hours and 0730 – 1300 hours Saturdays, and nursing staff during all other hours.
• Retrospective data extraction by one research assistant using administrative data from an electronic patient management program (i.Patient Manager CSC, Falls Church Virginia, USA). i.Patient Manager (i.PM) is the most widely used Patient Administrative System in Australian and New Zealand public hospitals. i.PM allows all administrative aspects of an inpatient episode to be securely tracked within a centralised database accessible by healthcare staff from multiple sites within the same health service [ 25 ]. Admission and discharge information is entered into i.PM by administrative staff during Monday to Friday between 0730 – 2000 hours and 0730 – 1300 hours Saturdays, and nursing staff during all other hours. This information includes date and time of hospital admission and hospital discharge, in addition to discharge destination.
• Retrospective review of scanned inpatient medical records by two research assistants following patient discharge . All paper-based inpatient medical records are routinely scanned by health record administrative staff to form an integrated digital record within 48 hours of patient discharge. This record can then be reviewed electronically. For the purposes of this study, the retrospective review of scanned inpatient medical records was considered the gold standard data collection method. This justification is founded in the consideration of the medico-legal record of the patient admission as the primary source of information [ 26 ] and has previously been used as a gold standard measure when assessing multiple other outcomes including diagnostic accuracy and rates of adverse events but not for hospital length of stay or discharge destination [ 5–11 , 27 ].

3.4 Procedure

Wards were attended daily by a research assistant between 0800 and 1200 hours. Observational data collected manually from ward based sources was entered directly into a survey tool (SurveyMonkey Inc. Palo Alto California, USA) via an electronic tablet device (iPad, Apple Inc, Cupertino CA, USA) at time of daily collection. Hospital admission and discharge data were collected from nursing handover records, paper-based inpatient medical records and ward admission and discharge records. If any data were unavailable or there were any discrepancies between data sources, research assistants clarified data through discussions with the nurse in charge or ward administrative staff. Data was exported from the survey tool into a Microsoft Office Excel spreadsheet (Microsoft, Redmond WA, USA).

Retrospective administrative data extracted from i.PM was entered into a separate Excel spreadsheet after patients had been discharged from hospital to ensure full availability and capture of inpatient episode data.

Similarly, research assistants independently extracted hospital admission date, hospital discharge date and discharge destination from the scanned inpatient medical records and entered data into a separate Excel spreadsheet. Inter-rater reliability analysis using Cohen’s Kappa coefficient was performed to determine consistency among the two research assistants, finding 93.8% agreement (Kappa=0.92) for length of stay and 100% agreement (Kappa=1.00) for discharge destination.

3.5 Analysis

3.5.1 completeness of data capture.

The computer program used to retrospectively review scanned medical records post discharge from hospital did not have a method for calculating total hospital admissions and hospital discharges between set dates. Therefore, for the purpose of this project, the researchers deemed a comparative analysis between retrospective administrative data collected from i.PM, and prospective data collection from ward-based sources as the most suitable assessment of the completeness of data capture. The number and percentage of data captured via each data collection method was presented using descriptive statistics.

3.5.2 Level of Agreement

Investigators compared level of agreement for the 376 admission and discharge data sets captured completely by all three data collection methods. Level of agreement between data collection from scanned medical record review, electronic patient management program (i.PM), and data collected from ward-based sources was analysed using a Bland-Altman comparison for length of stay and Cohen’s Kappa for discharge destination. To determine whether agreement between data collection methods was related to the day of week, of discharge data were entered into univariate logistic regression analysis as independent variables, agreement with inpatient medical record review was entered as the dependent variable.

Statistical analyses were performed using Stata (Version 13, StataCorp, College Station, Texas, USA).

4.1 Completeness of data capture

▶ Figure 1 outlines the data collection and analysis process, while ▶ Table 1 outlines the data cleaning process resulting in the identification of 376 complete hospital admission and hospital discharge data sets captured by all three data collection methods. This data set contained 178 (47%) females and 198 (53%) males with an average age of 59 ± 21 years.

Data collection and analyses process flowchart

Data cleaning process

Of the 542 complete sets of hospital admission and hospital discharge data captured retrospectively by i.PM, only 376 (69%) complete data sets were captured via manual data collection from ward-based sources by a research assistant. It should be noted that data collection from ward-based sources did not produce any unique data set that wasn’t captured via i.PM.

4.2 Level of agreement

Descriptive data describing the level of agreement between data collection methods for length of stay and discharge destination are displayed in ▶ Table 2 .

Agreement between ward-based sources, electronic patient management program and scanned inpatient medical records

4.2.1 Length of Stay

Bland-Altman comparison between length of stay data collected via scanned inpatient medical records and ward-based sources by a research assistant resulted in limits of agreement of –5.323 to 5.637 with a mean difference of 0.157 (-0.121 – 0.435). Pitman’s Test in variance resulted in r=0.186 (p=0.00).

Bland-Altman comparison between length of stay data collected via scanned inpatient medical records and retrospective administrative data from the electronic patient management program (i.PM) resulted in limits of agreement of –1.564 to 1.415 with a mean difference of –0.074 (-150 – 0.001). Pitman’s Test in variance resulted in r=0.026 (p=0.613).

Agreement between data collection methods for hospital length of stay are displayed as Bland-Altman plots (▶ Figure 2 ), with differences in agreement based on day of week of discharge displayed in ▶ Table 3 .

Bland-Altman plots comparing length of stay using different data collection methods.

Agreement between ward-based sources, electronic patient management program and medical record review for length of stay based on day of week of discharge

4.2.2 Discharge Destination

Cohen’s Kappa coefficient of agreement between discharge destination data collected via scanned inpatient medical records and ward-based sources by a research assistant resulted in 83.1% agreement with a k=0.63 (p=0.00).

Cohen’s Kappa coefficient of agreement between discharge destination data collected via scanned inpatient medical records and retrospective administrative data from the electronic patient management program (i.PM) resulted in 92.0% agreement with a k=0.81 (p=0.00).

Differences in agreement based on day of week of discharge are displayed in ▶ Table 4 .

Agreement between ward-based sources, electronic patient management program and medical record review for discharge destination based on day of week of discharge

5. Discussion

Data accuracy has previously been examined for electronic patient medical records and hospital administration data for multiple clinical outcome measures including diagnostic accuracy and rates of adverse events [ 5–11 , 27 ]. However, to the author’s knowledge, this is the first investigation to compare and demonstrate differences in completeness of capture and agreement between hospital data collection methods for length of stay and discharge destination.

Our results demonstrate that the administrative data-set extracted from i.PM captured a larger set of complete hospital admission and hospital discharge data than the ward-based data-set. Importantly there were no unique data captured from ward-based sources that were not captured via i.PM. Therefore, researchers, hospital administrators and clinicians may be able to rely on electronic patient management programs as an effective method of capturing occasions of inpatient episodes of care.

There are many potential contributing factors to the discrepancy in completeness of capture between observational data collected from ward-based sources, and retrospective administrative data extraction from i.PM. Research assistants cannot be present on all wards at all times of the day, so it is possible that data may be missed for patients whose ward admission and discharge occurred when research assistants were not present on the ward, highlighting a limitation of this data collection method. There may have also been limited opportunity for research staff to access all ward resources for all patients on the ward. For example, if a patient was in the operating room, having a procedure or in radiology, their inpatient medical records are not located in the ward but are with the patient. Prospective observational data collection has been used in research examining hospital adverse events [ 29 , 30 ] where data collectors were integrated into the ward environment and attended regular meetings. This method of data collection is difficult to fund in the current environment where competition for research funding is increasingly fierce. With increasing use of health information technology globally [ 31 , 32 ] traditional research methodology must evolve to embrace opportunities to collect reliable retrospective data from electronic sources. Our results support the use of retrospective administrative data collection from electronic patient management programs to capture inpatient episodes of care.

Using a Bland-Altman comparison strong levels of agreement in length of stay were observed between all three investigated data collection methods, with a slightly larger mean difference observed in data from ward-based sources. Less significant p values in Pittman’s Test of variance when comparing electronic administrative data may have been due to the high levels of agreement in this data. There was no significant difference in agreement between different days of the week, with weekend data from ward-based sources demonstrating a slight trend towards improved agreement. Although this may appear to indicate that any method of data collection could be used in the hospital setting, when displayed in Bland-Altman plots data from ward-based sources demonstrated lower levels of agreement as length of stay increased when compared with inpatient medical record review ( Figure 2 A ). As can be seen in Figure 2 (A) , one patient had a recorded hospital length of stay that was greatly different in the ward-based source data collection method, compared to the other 2 methods. On further examination, this patient was an inpatient in the health services’ “Hospital in the Home Program” prior to being admitted to the ward. The length of stay collected from the ward-based source only represented a portion of the patient’s total length of stay, however, this information was not clear to research assistants when collecting this patient’s data from ward-based sources. This may indicate that for patients with long “outlier” hospital length of stays, data collected from ward-based sources is less likely to be correct. These findings have further relevance in the funding of clinical trials where resources may be sought for staffing to manually collect data. This study suggests that retrospective data collection via electronic patient management programs can improve data completeness and agreement compared to manual ward-based data collection, when measuring hospital length of stay.

Using Cohen’s Kappa coefficient the electronic patient management program (i.PM) demonstrated the highest levels of agreement with the gold standard retrospective review of scanned medical records for discharge destination data. Discharge destination data collected from ward-based sources on a Monday was significantly less likely to agree with review of inpatient medical records. This may be due to change of staff from weekend to weekday staff, or that wards may have been busier on a Monday morning. Lower levels of agreement in data from ward-based sources generally, may be due to inaccuracy in recording of discharge destination by ward staff as this data is not usually recorded for research purposes and there appeared to be a perception by ward staff that accurate recording of discharge destination was of a lower organisational importance.

The knowledge of agreement between different hospital data collection methods remains limited despite the importance of data accuracy for research validity and health services quality management. This study was performed in seven wards in a single acute urban Melbourne hospital, which limits the external generalisability of the findings. In addition, as the study only measured a narrow scope of outcome measures using a single program, conclusions should be drawn cautiously for other electronic patient management programs and outcome measures. While inter-rater reliability was tested between research assistants reviewing inpatient medical records, this analysis was not performed on research assistants collecting data from ward-based sources. Due to the simplicity of the task in transcribing (collecting) data from one source to another it was deemed unlikely that there would be systematic bias between research assistants, however authors do recognise this as a limitation to this study. Future research should continue to examine accuracy in methods of hospital data collection, for the purposes of research and health services management. Due to the limitations of this research, the authors recommend future investigation to examine the cost effectiveness of data collection methods, as well as a broader scope of outcome measures, for example; unplanned hospital readmissions, rapid response teams, and patient adverse events (falls, pressure injuries, and deep vein thrombosis). Inclusion of cost effectiveness of data collection methods using time-intensive comparisons in future research could help to identify what level of investment is essential for the valid and accurate conduct of clinical trials in the hospital setting. Evaluation across multiple hospital sites using different electronic management programs in future research will improve the ability to generalise results across health services.

6. Conclusion

Although previous research has sourced data from electronic patient management programs, this is the first paper to demonstrate difference in completeness of data capture and level of agreement between data collection methods for hospital length of stay and discharge destination. Administrative data from an electronic patient management program showed the highest level of completeness of capture and level of agreement with the gold standard of inpatient medical record review for both length of stay and discharge destination, and therefore may be an acceptable data collection method for these measures.

Acknowledgments

The authors thank staff from Monash Health, Western Health, Melbourne Health, and Monash University, who contributed their time and effort to this investigation, particularly Monash Health ward staff, Allied Health WISER Unit, Health Information Services, and Monash University Allied Health Research Unit.

Clinical Relevance Statement

Electronic administrative data has become a key source of inexpensive and readily available information for hospital outcome measures such as hospital length of stay and discharge destination. Interestingly while previous studies have shown the use of administrative data in adverse events and coding for billing purposes may result in inaccurate data, no such studies were identified examining the use of administrative data for key outcome measures such as length of stay and discharge destination. This is the first paper to demonstrate that the administrative data from electronic patient management programs are an effective data collection method for key hospital outcome measures (length of stay and discharge destination). Future clinicians and hospital administrators can utilise opportunities to collect reliable hospital length of stay and discharge destination data from electronic sources, while researchers can improve data collection methodology when using length of stay and discharge destination as research outcome measures.

Conflict Of Interest

This study was performed in conjunction with a larger stepped-wedge randomised controlled trial examining the effectiveness of acute weekend allied health services. The results of our manuscript for publication helped shape the data collection methodology for the stepped-wedge trial mentioned above. All investigators involved in this manuscript are involved in the stepped-wedge randomised control trial. This research was funded by a Partnership Grant from the Australian National Health and Medical Research Council. Professor Terry Haines is supported by a Career Development Fellowship (Level 2) from the National Health and Medical Research Council. The NHMRC has had no direct role in the writing of the manuscript or the decision to submit it for publication.

Protection Of Human Subjects

This paper was an observational study only collecting routinely available data from the hospital systems and did not involve any direct intervention to participants. This study was approved by Monash Health Human Ethics Committee (Reference number: 13327B).