This article describes an active-learning exercise using fast food delivery to achieve several learning objectives in an undergraduate research design course. Having food delivered to the classroom incorporates multiple core social science concepts in a fun and engaging way that allows students to effectively translate methodological skills to their independent research projects. When the exercise is completed, students will be able to:
-
1. Design an observational study
-
2. Explain the role of assumptions in social science
-
3. Explain the role of operationalization in social science
-
4. Construct measures to operationalize concepts
-
5. Test an original hypothesis
-
6. Create an original dataset
-
7. Analyze an original dataset
The article proceeds as follows. First, it demonstrates the importance of using engaging active-learning techniques in a research methods course. Next, it defines the steps necessary to execute the fast food delivery activity. The article concludes with a discussion of the activity, alternatives, challenges, and assessment methods.
ACTIVE LEARNING AND RESEARCH METHODS
The traditional lecture-based classroom is becoming less effective as more open-source lecture resources are available to students. Students want increased engagement, and active learning is more popular (Raymond and Usherwood Reference Raymond and Usherwood2013). They do not want to be passive consumers of content. There is a wide range of active-learning techniques (Bromley Reference Bromley2013) with numerous benefits. Active learning helps students to learn and retain material (Archer and Miller Reference Archer and Miller2011), comprehend complex concepts (Omelicheva and Avdeyeva Reference Omelicheva and Avdeyeva2008), perform better on quizzes (Levin-Banchik Reference Levin-Banchik2018; Powner and Allendoerfer Reference Powner and Allendoerfer2008), achieve higher-order learning outcomes (Archer and Miller Reference Archer and Miller2011), and it increases student enjoyment (Hendrickson Reference Hendrickson2019).
Positive learning outcomes are crucially important in research methods courses in which students learn how knowledge is generated, what quality research looks like, and why the field can make claims about social phenomena. Common active-learning techniques in methods courses include discussions, searching databases, and using statistical software to analyze data. Important concepts such as theory development and operationalization are taught by dissecting academic articles (Fisher and Justwan Reference Fisher and Justwan2018) and student papers (Murphy Reference Murphy2015). However, even clearly developed articles present challenges for students trying to understand theoretical concepts, research design, operationalization, and data. Most articles lack a discussion of why measures were not included or how a design changed because of external constraints—important components of the research process (Pfeffer and Rogalin Reference Pfeffer and Rogalin2012). Participation in these activities is time consuming and feels forced, reducing student commitment that hinders actual (Tews et al. Reference Tews, Jackson, Ramsay and Michel2015) and perceived (Deslauriers et al. Reference Deslauriers, McCarty, Miller, Callaghan and Kestin2018) learning.
Positive learning outcomes are crucially important in a research methods course in which students learn how knowledge is generated, what quality research looks like, and why the field can make claims about social phenomena.
Research design and operationalization present unique challenges because students must acquire knowledge and skills to successfully conduct independent research (Adriaensen, Kerremans, and Slootmaeckers Reference Adriaensen, Kerremans and Slootmaeckers2015). Students struggle to acquire these crucial skills when material is dry and there are few engaging experiences. Acknowledging these limitations, methods faculty created more relatable learning opportunities. Some are as simple as having outside faculty share research experiences so that students can contextualize the research process (Pfeffer and Rogalin Reference Pfeffer and Rogalin2012). Others focus on leveraging current student knowledge to operationalize seemingly well understood concepts (Asal et al. Reference Asal, Jahanbani, Lee and Ren2018), creating collaborative teams of student researchers (Campisi and Finn Reference Campisi and Finn2011; Ishiyama Reference Ishiyama2012; Lundahl Reference Lundahl2008), using games to teach concepts (Kollars and Rosen Reference Kollars and Rosen2017; McCarty Reference McCarty2019), and using material that is more relevant to a student’s everyday life (Rosen Reference Rosen2018). These methods help students better connect to material, achieve the benefits of active learning, and provide a more complete picture of the research process by providing real-life context.
Rosen (Reference Rosen2018) made significant strides in creating a more relatable research experience by developing a semester-long project focused on eating breakfast. The activity requires students to define the “best” breakfast, develop measurements, eat at restaurants, and report their results in a final research paper. Although this is an important contribution, there are limitations.
First, Rosen’s activity is a semester-long project requiring a significant time commitment. Several class sessions are dedicated to the preparation, collection, and analysis of the data necessary for the final project. This comes at the cost of other material, requiring the instructor to reduce or eliminate content. The fast food delivery exercise allows the instructor more flexibility, with much of the work completed in a limited number of class sessions with the option of continuing the activity through the semester. Even if it is conducted during only a single week, the activity provides a conceptual anchor for students as they learn more about the research process.
Second, there are issues of equity involved in the Rosen (Reference Rosen2018, 174) activity. There are two paths to collecting data: “…(1) a mix of observation and evaluation, in which students generate a questionnaire, visit their assigned restaurant, and evaluate it according to a set of criteria and procedures; or (2) a survey of their fellows students or community.” The pressures of a group decision and prospect of the additional work associated with an Institutional Review Board approval steer the choice in the direction of eating at restaurants regardless of a student’s individual or financial concerns. Rosen admitted that students have never chosen the survey route. The fast food delivery does not overcome the financial concerns, but it shifts the burden to the instructor. Although the associated cost might dissuade faculty, they have that option as they develop a syllabus. Students are unaware that they will incur a financial cost when registering for the class.
Third, as Rosen (Reference Rosen2018, 177) noted, the breakfast activity does not engage in hypothesis testing, a crucial component of political science research. The fast food delivery activity fills this gap.
FAST FOOD DELIVERY ACTIVE-LEARNING EXERCISE
This section describes the steps for using the active-learning activity. Additional resources including discussion questions, examples, and assignments are in the supplemental materials.
Timing
Providing flexibility so that instructors can align the activity with learning objectives in their course is important. They should review the activity and see when it best suits their class needs. It is recommended to start the activity after students develop their own research question. With their research question decided, the next step is thinking about a design to answer that question. Fast food delivery serves as a conceptual anchor for students to reference as the course moves through more complex ideas and they develop a design of their own.
The instructor must dedicate at least two sessions to the activity. An observational study is developed in the first session and the food is ordered at the beginning of the next session, followed by a discussion after the food arrives. While the class waits for the food to arrive, the instructor may cover other material. If the instructor decides to end the activity here, students will achieve learning objectives 1–4. Ordering deliveries throughout the remainder of the semester will achieve learning objectives 5–7. Additional class sessions are needed later in the semester to analyze the data.
Step 1: Research Question
To begin the activity, the instructor shows students several TV, print, or internet ads from a local restaurant that delivers. Restaurants known for their speedy delivery service are best. The showmanship of the instructor is necessary to convey how hungry they are and that they need food fast. The instructor guides student discussion to develop a research question focused on the speed of the delivery—ideally, “What is a fast delivery?”
Step 2: Research Design
The class discusses the best way to investigate the research question. This allows for a brief review of the different types of methods available to social scientists: observational study, comparative method, survey research, and data/archival work. The most appropriate method for this exercise is an observational study, in which the instructor orders from the restaurant and has food delivered to the classroom. Students collect data based on what they observe.
Step 3: Operationalization, Theory, and Assumptions
This arguably is the most complex component of the activity because multiple learning objectives are achieved simultaneously. However, this is similar to the actual research process, which does not happen linearly.
The instructor begins this discussion by asking students what the best measure is to address the research question. The instructor explains that this is the value determined by another factor in the observational study—that is, the dependent variable. Students quickly determine that time is the appropriate concept for fast delivery, but the operationalization is less clear. Students must decide what constitutes a delivery. For instance, when does the delivery officially start and end? The class discusses the implications of these decisions and how they impact the research process. Early coding decisions impact the direction of the research, how questions are framed, and the outcomes.
Next, the class discusses the independent variables that will affect how fast the delivery occurs. In many cases, operationalization occurs in reverse during this step, with students identifying measurable outcomes and then connecting them to larger concepts. For instance, if the class determines that the number of order alterations and additional items have the most significant impact, they are measuring order complexity.
The class continues to develop the list of independent variables. With each variable, the instructor should ask three questions. First, how will the class ensure that the measure is replicable? If weather is a concern for students, how will they measure that? Is it enough for them to look out the window or do they need to agree on a weather website? Replicability is core to the social science process. Having students think about procedure when determining variables is crucial to replicability.
Second, what can the class actually observe? The best data are not always available and researchers must find alternative measures or make assumptions defended with logic. Social scientists must observe the world they are in and use the data available to them as best they can. The students do not know how the delivery person gets to the classroom; they just know that the food arrives. However, by using Google Maps to determine the distance of the restaurant to the classroom, students develop an assumption about the mode of transportation. This allows the instructor to address the validity and limits of assumptions and how they impact the research process.
Social scientists must observe the world they are in and use the data available to them as best they can.
Third, what impact will this variable have on delivery time? For each independent variable, students develop a causal theory to explain the effect on the dependent variable. Although theory development is rudimentary—that is, increases in distance will increase delivery time because of an increased chance of encountering obstacles—students can see how measures and concepts relate to theory and the research question.
Step 4: Hypotheses
If the instructor ends the activity after one delivery, hypothesis generation will be limited to delivery time. Given the observational study developed, students should hypothesize a time for “fast delivery.” This is limited because it does not capture any variation. Ordering multiple times during the semester will generate multiple data points. Students should think about the causal effect of independent variables and develop testable hypotheses based on the theories developed in step 3. The instructor needs to ensure that students are identifying appropriate hypotheses. For instance, distance might be a factor in delivery time, but in this iteration of the activity, distance is static and would not generate a testable hypothesis.
Step 5: Data Collection
Students must understand how to systematically collect and organize the data. The instructor sets up an Excel worksheet to input data and should control a master worksheet; however, students can maintain their own if they have computer access. Constructing the Excel worksheet gives students valuable insights that they do not receive when they are given completed datasets such as the American National Election Studies. These datasets are large, difficult to navigate, and disconnected from the collection process. Collecting and entering data after each delivery demonstrates the collection process.
Step 6: Ordering
Before ordering, the class decides what is ordered and when. When students are allowed to choose and receive the order, they become more invested in the activity. Instructors can use this as an opportunity to talk about random selection and sampling. Randomizer applications mean that all students have an equal opportunity of being chosen. Multiple orders during the semester allow the instructor to talk about selection with and without replacement. Running the activity without replacement increases the chances that a student will be able to choose and receive a free meal. The decision about when to order might encourage students to revisit variables. If it is the same day and time—for example, the beginning of Tuesday’s class—additional measures might not be needed. If the instructor orders for every class or has multiple sections, additional variables are appropriate.
Instructors should assign three students to time the delivery, which is used to discuss intercoder reliability. Inevitably, students will end up with different times. All three times should be entered into the dataset, but the class should decide which rules to apply in deciding the official time: mean, median, mode, greatest value, or lowest value. The instructor can discuss how this decision impacts the study.
Step 7: Analysis
The final part of the exercise is at the end of the semester after weeks of data collection. Students must determine if hypotheses or “fast delivery” are reflected in the data. It is unlikely that the reality of the data will directly match the hypothesized “fast delivery” time; therefore, students use the data to determine the mean, standard deviations, and z-scores. Having a base knowledge of these concepts is helpful but the instructor also can use these data to teach basic statistical concepts.
Worksheets that ask for statistical information and analysis of the data are helpful; examples are provided in the supplemental materials. The instructor can lead the discussion to determine how close to the mean the hypothesized speed of a “fast delivery” must be to confirm the class’s hypothesis—one standard deviation, for instance.
During the final analysis, students revisit earlier operationalizations and assumptions to discuss their impact. The instructor challenges students to think about what they learned from the activity and what they might change. This discussion highlights the importance of assumptions and operationalization in social science research.
LEARNING OBJECTIVES
Active learning has many benefits. It engages students in the learning process, encourages understanding of complex concepts, and helps with the retention of material. The fast food delivery activity is not only engaging; it also is a fun way for students to draw on their lived experience and learn about research methods. As the course progresses and students engage with more difficult concepts, the instructor can draw from this activity and data to reinforce concepts such as design decisions, operationalization, observations, dependent and independent variables, theory development, and data analysis. Thinking through the decisions necessary to successfully develop a research project prepares students to digest more complex political science articles and engage in independent research.
Active learning has many benefits. It engages students in the learning process, encourages understanding of complex concepts, and helps with the retention of material. The fast food delivery activity is not only engaging, it also is a fun way for students to draw on their lived experience and learn about research methods.
Much like the research process, accomplishing the learning objectives does not proceed linearly. Table 1 summarizes when learning objectives are achieved during the activity.
Table 1 Achieving Fast Food Delivery Learning Objectives
ASSESSMENT
Assessment can occur in several ways, and more information is provided in the supplemental materials. First, after completing the observational design, the instructor assigns articles and asks students to identify certain components: research question, design type, operationalizations, and cases. Instructors also can assign articles that measure the same concept differently and discuss the research implications. Second, the instructor can provide concepts found in political science and ask students to operationalize them: power, war, democracy, and justice. Third, worksheets can be provided to help in understanding basic statistical concepts using the data collected.
VARIATIONS AND LIMITATIONS
This activity is best used in upper-level political science research methods courses. However, the flexibility of the activity allows for wider application. Steps 1–3 can be useful in a first-year political science course to teach the basics of research design. Setting up the study, ordering, and debriefing can be accomplished in a single week. The activity covers many key learning objectives and provides an anchor for students as they think about more advanced concepts. By ordering from multiple restaurants, instructors also can make the activity more complex to challenge undergraduates.
Discussions can be structured in various ways. Instead of class discussions, the instructor can use the pair-and-share method in which small groups develop operationalization measures and then discuss as a class. Groups can work on all variables or each group can work on operationalizing a different concept. Online discussion boards provide students the opportunity to think through their ideas, post them, and receive feedback from colleagues. Online discussion-board assignments before class can be used to ensure that students come to class prepared.
The major limitation of this exercise is the cost associated with the activity. Whereas a single delivery might be a nominal cost, ordering multiple times during the semester significantly increases the cost. Instructors are encouraged to seek support from their department, dean, and provost. This is an engaging way to teach complex topics in a challenging class.
Substantively, there are topics taught in methods classes not covered by the fast food delivery activity. For example, literature reviews are fundamental to successfully framing a research project in the discipline. However, separating the activity from literature helps students to feel more comfortable participating when they are not concerned about giving a “wrong answer” that does not align with a reading.
Finally, this activity showcases many complex topics in political science. The burden is on the instructor to manage discussions, make connections, and ensure that students are learning the material.
CONCLUSION
Active learning is increasingly popular because it provides novel ways to engage students and teach material. Methods courses serve a crucial role in political science curricula, teaching students how to think about and systematically investigate political questions. Although methods courses necessarily engage students through the conduct of research and use of statistical software, they regularly receive lower evaluations because it is not the type of engaged learning typical in other courses. The fast food delivery activity contributes to the few innovative engagement activities in research methods courses. Ordering fast food allows the instructor to develop an observational study and achieve multiple learning objectives. The activity is flexible, allowing instructors to use it to introduce or reinforce concepts. What is important to this activity is the subjective nature of the variables being defined and the reliance on students’ lived experiences. It allows students to engage with important political science concepts with limited prior knowledge or the concern about providing a wrong answer. The activity focuses on argumentative logic and replicability, which are necessary skills for successful political scientists.
Supplementary Materials
To view supplementary material for this article, please visit http://dx.doi.org/10.1017/S1049096521001128.
