ADVANTAGES AND DISADVANTAGES OF SURVEYS

There are several advantages to surveys (Table 1). The ease and cost-efficient nature of this study design is very appealing. While each mode offers its own unique advantages and disadvantages, in general, the most frequent type of survey in healthcare epidemiology and antimicrobial stewardship is the online survey. The availability of user-friendly sophisticated software to develop and deploy surveys has made it very convenient, even for surveys that require complex programming. Since there is no interviewer involved, respondents may be more willing to share information but conversely, there is no possibility for probing or clarifying responses. Although there is considerable variation,Reference Abbo, Lo and Sinkowitz-Cochran ⁴ average response rates for online surveys range from 20% to 30%. Nonresponse bias stemming from a low response rate can lead to biased survey results if the respondents have differing characteristics than nonrespondents. For example, in a survey to examine antimicrobial stewardship practices, institutions with robust stewardship programs may be more likely to respond than institutions with little or no antimicrobial stewardship activity. There is no universally agreed-upon minimum response rate for online surveys but capturing data to the extent possible on the characteristics of the respondents and the nonrespondents may allow an assessment of the impact of response rates on the study results.

TABLE 1 Advantages and Disadvantages of Using Surveys in Healthcare Epidemiology Research

NOTE. Adapted from http://www.fao.org/ag/humannutrition/33251-0a0b0928c3532ed36c3b6624d54d73f86.pdf.

Interventions that have been shown to increase response rates for online surveys include clarity in the instructions, motivation for participation, choosing a topic that is highly relevant to the audience, brevity, survey length and ease of completing the survey, incentives and rewards for participation, and repeat reminders to nonrespondents.Reference Funkhouser, Vellala and Baltuck ^{5 –} Reference Fan and Yan ⁷ Useful resources include http://www.qualtrics.com/blog/how-to-increase-online-survey-response-rates/ and http://socialnorms.org/what-is-an-acceptable-survey-response-rate/.

Other modes of survey administration that may have applicability for healthcare epidemiology and antimicrobial stewardship are phone interviews and face-to-face surveys. The advantage of phone interviews is that they are less likely to be self-selective. Also, there is a higher completion rate and quality control since the interviewer can guide the respondents to any questions not understood as opposed to leaving them unanswered. The main disadvantage is that they are generally limited to a maximum of about 15 minutes. The longer a phone survey continues, the more people will “drop out” and not fully answer all the questions. Questions must be simple and clearly stated. Since those responding cannot see or read the questions, complicated or long questions are not appropriate for telephone surveys. Respondents might be biased toward giving socially desirable answers to the interviewer. For face-to-face sessions, response rates are better but this is the most expensive and time-consuming method of administering surveys.

PITFALLS AND TIPS

As with other research designs, many pitfalls in surveys are avoidable through adequate planning and development of a specific research question prior to design and deployment and careful consideration of the population under study (Table 2). Each question in a survey must be deliberately positioned and included only if the responses generated will contribute to answering the research question. Ambiguous, complex, or vague questions and inconsistency in definitions, scale, or wording can compromise the validity of a survey. For questionnaire design, attention should be paid to the layout, the length and general organization, and skip patterns. All too often, surveys that are too long and poorly organized are doomed to fail because of the participant burden they entail.

TABLE 2 Pitfalls in Questionnaire Design and Development

NOTE. Adapted from Choi et al.Reference Choi and Pak ¹⁷

The layout of the survey needs to be consistent, respondent-friendly, and not laborious. Several excellent texts on survey design provide guidance on the preferred layout of surveys.Reference Fowler ¹

The mode of survey should be planned well in advance, and the pros and cons of each mode should be considered, depending upon the aim of the survey and the population being covered. Many surveys in healthcare epidemiology and antimicrobial stewardship are electronic and self-administered. Although that option is convenient, it may lead to issues if respondents have different interpretations of questions than intended.

The study question should drive consideration of whether a survey’s shortcomings, such as recall bias and social desirability bias, may be important enough to consider an alternative study design. For example, a survey in healthcare epidemiology that asks detailed questions regarding extensive recall may not provide the information that is being sought.

Pilot testing of a newly developed or adapted survey instrument is essential. At a minimum, pilot testing should be done for comprehensibility, face validity, participant burden, and layout, including accuracy of skip patterns.

Many of these pitfalls related to survey design and layout can be avoided by using an existing standardized, validated survey if it meets the needs of the study. It can be very time-consuming and burdensome to develop questions from scratch. Several research groups have used the Society for Healthcare Epidemiology of America Research Network to conduct surveys in healthcare epidemiology and antimicrobial stewardship, and it would be worthwhile to peruse the Society for Healthcare Epidemiology of America Research Network website for an existing survey instrument that could lend itself to adaptation.

A second major category of pitfalls is in the deployment of surveys. The measurement and consideration of the denominator are important to be able to calculate response rates and understand how nonrespondents will be identified and compared with respondents (eg, demographic or other characteristics to decrease selection bias).

A common issue that should be addressed is the effect of respondent identifiability. This can be separated into 2 categories: anonymity and confidentiality. The study question should drive decision-making regarding whether a survey should be anonymous, confidential, or neither. If the survey contains sensitive questions, anonymity is the preferred approach since it may lead to more honest responses. However, an anonymous survey cannot be authenticated, meaning that responses cannot be restricted to be one per individual. For online surveys, survey software can be used to make responses anonymous. If it is desirable to track participant responses, such as in longitudinal surveys, confidential surveys may be a better option.

Sample size considerations for surveys require knowledge of the population size, the margin of error, and the confidence interval. Survey development and deployment software often come with an online calculator to determine sample size (https://www.surveymonkey.com/mp/sample-size-calculator/), which is a useful early step as in any other research method.

Statistical analysis of surveys ranges from relatively easy with descriptive analyses to quite complex in case of weighted population surveys. For existing surveys such as (for example) the Agency for Healthcare Research and Quality Culture of Safety Survey, review of documentation and understanding the questions, the outcomes, and the data collected are especially essential. An analysis plan should be created ahead of time. Creating table shells that can be completed later with data is a useful exercise for planning the analysis phase. There are no current national guidelines or standards regarding the reporting of survey results.

QUALITATIVE RESEARCH IN THE CONTEXT OF SURVEYS

Qualitative research is a vast science with its own sets of methods, so an in-depth discussion of this science is beyond the scope of this paper. Here we discuss only those aspects of qualitative research that may complement survey design and/or analysis. Qualitative approaches are becoming more common in healthcare research and are dependent on conceptual, rather than numerical, analysis. Qualitative research is used to discern complex social processes, understand how a particular phenomenon is perceived by the study patients, and discover beliefs, values, or motivations underlying their behavior. Reference Fitzpatrick and Boulton ^{8 ,} Reference Curry, Nembhard and Bradley ⁹ Methods have expanded over the past 2 decades to include ethnography, grounded theory, case studies, phenomenological research, narrative research, participatory action research, and discourse analysis. These methods can be employed when little knowledge exists about a particular research area and can also generate hypotheses for future quantitative research.Reference Curry, Nembhard and Bradley ⁹ In situations for which quantitative data alone will not suffice, the addition of qualitative approaches can be helpful; the combination is called the “mixed methods” approach.Reference Curry, Nembhard and Bradley ⁹

Philosophically, qualitative research takes a constructivist approach and tends to be participatory and collaborative in nature. Qualitative methods usually occur within a natural setting with the researcher as key to the process. Qualitative information can validate accuracy of quantitative data from surveys or can be used in advance of survey development as a step to identify key themes. For example, a researcher can hold focus groups of caregivers for those with “lived experience” of a healthcare-associated infection to examine key themes important to caregivers. These key themes can then be used as a starting point for survey development.

The methodology for conducting unbiased, credible, and rigorous qualitative research has been published elsewhere.Reference Curry, Nembhard and Bradley ^{9 ,} Reference Patton ¹⁰ One critical decision is thinking about whom or what to include in the sample, and in contrast to quantitative research, there is no predetermined sample size. Rather, sampling stops when the saturation end point, which is defined as the point at which the encounters no longer provide new information, has been reached.Reference Kuper, Lingard and Levinson ¹¹ Sampling should include negative or deviant cases to ensure the inclusion of all perspectives.Reference Anderson ¹² Details regarding criteria for selecting and recruiting study participants, the research setting, ethical governance (eg, informed consent, maintenance of confidentiality), methods for recording and transcribing data, data analysis (including whether it was repeated by other researchers to guarantee reliability), and consideration for how the researcher(s) could have introduced bias should be provided.Reference Anderson ^{12 ,} Reference Mays and Pope ¹³ Researchers should carefully examine the target audience and spend time reviewing theories, frameworks, or models that may assist in the organization and interpretation of data.

There are several advantages that qualitative research provides (Table 3). Issues and subjects can be evaluated in more depth and detail; interviews are not limited to particular questions and can be redirected or guided by researchers in real time; and the direction and framework of research can be revised quickly as soon as fresh information and findings emerge.Reference Anderson ¹² On the other hand, readers may not be familiar with the terms and concepts of qualitative research nor how to appraise such work.Reference Mays and Pope ^{13 ,} Reference Kuper, Reeves and Levinson ¹⁴ Other potential pitfalls are that the quality of such studies is heavily dependent on the experience and skills of the investigators; the presence of the researcher(s) during data gathering may affect responses; the quantity of the data can make interpretation and analysis time-consuming; and the personal biases of the researcher(s) may influence data interpretation.Reference Anderson ¹² Because of the complexity and ambiguity of healthcare environments and healthcare roles, contextual factors cannot be ignored. Qualitative methods can build the context or the “story” needed to further understand a phenomenon or issue.

TABLE 3 Advantages and Disadvantages of Qualitative ResearchReference Anderson ^{12 –} Reference Kuper, Reeves and Levinson ¹⁴

EXAMPLES OF SURVEY METHODS IN HEALTHCARE EPIDEMIOLOGY AND ANTIMICROBIAL STEWARDSHIP

An example of a survey is one by Drees et al.Reference Drees, Pineles, Harris and Morgan ¹⁵ The investigators conducted a cross-sectional survey of members of the Society for Healthcare Epidemiology of America Research Network to investigate variation in definitions and isolation procedures for multidrug-resistant gram-negative bacteria. In this study respondents were from 15 countries and 26 states in the United States, with a 39% response rate. The investigators acknowledged the low response rate and that respondents were selected from a convenience sample, which could cause selection bias. Since respondents were part of a healthcare epidemiology network, they are probably more motivated to participate and perhaps give socially desirable answers compared with nonrespondents. No information was available for nonrespondents, which is a limitation. Data on the number or rates of multidrug-resistant organism infections in each institution were not available; therefore, variations in practice were not adjusted for the frequency with which the respondents encounter these organisms. Despite the limitations, the survey provides helpful information to understand practices in a large group of hospitals and different geographic locations; the results are useful to understand baseline behaviors in a selected sample and possibly guide future studies to standardize best practices in the isolation of multidrug-resistant organisms.

An example of a mixed methods survey is the work by Glanz et al titled “A Mixed Methods Study of Parental Vaccine Decision Making and Parent-Provider Trust.”Reference Glanz, Wagner and Narwaney ¹⁶ Seven focus groups were conducted with vaccine-hesitant parents in a single health plan in Colorado. On the basis of findings from the focus groups, the investigators developed a survey that was mailed to a stratified sample of parents who accepted, delayed, or refused vaccinations for one of their children. The study included a review of health records to target, sample, and compare distinct groups of parents on the basis of their actual vaccine behavior and identified parents who delayed and those who refused vaccinations. Strengths of this study include a good response rate of 52%. The investigators also report a subanalysis comparing responders with nonresponders, which did not reveal any significant differences and provides valuable information when interpreting the results to minimize selection bias. Performing focus group interviews allowed better understanding of the patient’s perceptions of the providers’ knowledge and balanced vs unbalanced information provided to the parents. Combining qualitative and quantitative data as performed in this study is useful in designing meaningful survey questions to address research topics and guide future targeted interventions for specific populations. Limitations of these types of surveys include the fact that these are more time-consuming, and funding is usually needed in order to conduct these studies because resources are required for training and conducting focus groups.

MAJOR TAKE-HOME POINTS

Researchers embarking on a survey in healthcare epidemiology or antimicrobial stewardship must consider multiple issues (outlined in a checklist in Table 4). All surveys require a specific research question and planning for the type of subjects who will form the survey population in order to ensure that the survey population is the correct one for the questions of interest. An analysis plan should be developed that takes into account assessment of response bias. For new questionnaires, it is critical that the guidelines for layout, question design, formatting, and pilot testing area be followed.

TABLE 4 Checklist of Key Considerations When Developing a Survey in Healthcare Epidemiology