Julia Gressick PhD is an Associate Professor of Instructional Technology at Indiana University South Bend. Joel B. Langston is a graduate student of Liberal Studies and Manager of Media Services at Indiana University South Bend. Communication regarding this manuscript should be directed to Julia Gressick at: Department of Teacher Education, Indiana University South Bend, School of Education, 1700 Mishawaka Ave., Post Office Box 7111, South Bend, IN \u00a046634-7111. Phone: (574) 520-4281, Fax: (574) 520-4550,Email: jgressic@iusb.edu.<\/p>\n
Abstract<\/b><\/p>\n
Fostering critical thinking skills is a ubiquitous goal across disciplines and social contexts. Productive solutions to educational, content-based and social problems can emerge through well-reasoned conversation. How best to support the development of these skills has been a topic of debate.\u00a0 In this study, we investigated the design and effectiveness of a card-based game focused on undergraduate student understanding of common fallacies in thinking.\u00a013 Fallacies<\/i>\u00a0was designed with the intention of improving students\u2019 reasoning. In our study, we completed an iterative design phase, play testing phase and have collected data on student learning outcomes from two semesters as a result of classroom implementation. Results indicate that 13 Fallacies<\/i> improved student understanding of common fallacies in thinking and promoted social reasoning for at-risk undergraduate students.<\/p>\n
<\/p>\n
Show us your hand: How a card game improved critical thinking for at-risk college students<\/b><\/p>\n
A common goal of instructors across disciplines in higher education is to foster undergraduates\u2019 critical thinking and argumentative reasoning skills (e.g. Gellin, 2003). Many agree it is important for students to understand errors in thinking, recognize persuasion, and distinguish among opinion, reasoned judgments and facts (Halpern, 2003). These skills are essential for success across domains. Further, their importance holds the power to transform lives through productive social communication and engagement. Moreover, critical thinking skills consistently rank among top competencies that employers seek in post-secondary degree-holding candidates (Casner-Lotto & Barrington, 2006). \u00a0How best to cultivate and support development of these skills, however, has been a topic of debate (e.g. Cavagnetto, 2010). Despite our natural human tendencies to argue from a young age (Hay & Ross, 1982), there are many errors we commit in our daily thinking. For example, premises might be unacceptable, based solely on opinion and unrelated or inconsistent with conclusions (Kuhn, 2010). Experts who are cited may not be credible and important information might be missing from students\u2019 arguments. Recognizing errors in our thinking can be a challenge since they often seem persuasive and resemble sound reasoning despite their unsound nature (Toulmin, Rieke & Janik, 1984). Whether committing these fallacies is intentional with the goal of persuasion or simply an oversight, it is important for undergraduate students to understand fallacies in order to defend against them while improving the strength of the arguments they advance.<\/p>\n
Gaming as a way of learning and reviewing content has become an increasingly popular way to engage students (Swanson, 2014). Leveraging the ubiquitous human affinity across cultures for engaging in play, some games go beyond domain knowledge (e.g. Squire & Jan, 2007) to focus on more general skills, like argumentation. While digital games have become increasingly popular, card-based games help to bridge the digital divide and provide access to learning where technological resources might not be readily available (Bochennek, Wittekindt, Zimmermann & Klingebiel, 2007). Card games as pedagogy also foster collaborative learning and essential 21st Century habits of mind, including critical thinking and productive discourse skills (Reese & Wells, 2007).<\/p>\n
Considering broad critical thinking goals of higher education and the recognized benefits of pedagogical games, we have designed and implemented a card game called 13 Fallacies <\/i>in a course that prepares diverse, incoming, students who are identified as having barriers for college success. This game is aimed at scaffolding students\u2019 recognition of fallacies in others\u2019 thinking and avoiding them in their own social negotiation. Our research was guided by the following research questions: 1. Does playing 13 Fallacies <\/i>promote students\u2019 understanding of common fallacies in thinking, as evidenced by performance on a written assessment? 2. Does a guided video activity followed by playing 13 Fallacies <\/i>improve students\u2019 understanding of common fallacies in thinking when compared to playing 13 Fallacies <\/i>alone? 3. Do students feel positively about 13 Fallacies<\/i> as a learning tool, as evidenced by self-reported survey responses?<\/p>\n
Theoretical framework<\/b><\/p>\n
We have framed the development of 13 Fallacies<\/i> in Kuh\u2019s (2008) high impact practices, which are beneficial for diverse college students\u2019 learning outcomes, affect and overall development. 13 Fallacies<\/i> connects to Kuh\u2019s high impact practices since it is a collaborative learning approach designed to improve undergraduates\u2019 critical thinking skills while building a community of learners.<\/p>\n
Further guiding this research are theories of argumentative reasoning (Toulmin, 1958). Toulmin, Rieke & Janik (1984) assert that recognizing fallacies in thinking is an important component of reasoning and constructing sound arguments. Framed as \u201ca kind of sensitivity training\u201d, Toulmin, Rieke & Janik define the distinction between recognizing others\u2019 errors as an important component of avoiding them in one\u2019s own thinking. In 13 Fallacies<\/i> students are expected to both advance and defend arguments made about common fallacies.<\/p>\n
This research is also guided by the potential cognitive and motivational benefits of engaging in game play (Gee, 2003). Games can promote risk taking in a non-threatening manner because failure is offloaded to the process of play and becomes less associated with the individual. Further, the choices afforded by gameplay promote student agency in decision making, which may lead to increased engagement and motivation.<\/p>\n
We describe 13 Fallacies<\/i> as a \u201clightly-contextualized\u201d learning experience. The content of the game was written considering the concept of general problem solving skills (Perkins & Salomon, 1989). When the focus of learning is the development of a set of skills, in this case fallacy recognition as a means to improve critical thinking skills, providing a generalizable context can benefit and assist transfer of skills to other domains. Since 13 Fallacies<\/i> was played in a course for incoming freshmen designed to cultivate skills essential to their success in future classes, we carefully considered how to provide meaningful context without overly situating the skills within a specific domain. In turn, this also prepares students for future learning (Bransford & Schwartz, 1999) and success in other courses. Because of this consideration, 13 Fallacies<\/i> can be adopted to various contexts to teach and learn about fallacies in thinking.<\/p>\n
Methodology<\/b><\/p>\n
This paper reports on the development of 13 Fallacies<\/i> and two iterations of data collection. To develop the game, we used a design-based research approach (Brown, 1992; Collins, 1990). Using this approach allowed us to produce an instructional intervention and systematically examine resulting student learning in a classroom environment. We have separated iterations of development into two phases: initial development<\/i> and classroom implementation<\/i>. Initial development involved the design and play testing of 13 Fallacies<\/i>. The classroom implementation phase focused on the wide-scale introduction of the game that resulted from the initial development phase and its influence on student learning outcomes in two conditions. We collected data during Fall 2015 and 2016 semesters. During each semester, 13 Fallacies<\/i> was played 10 times over the course of five weeks. \u00a0Each session lasted 30 minutes. We have obtained human subjects approval for all research reported in this paper.<\/p>\n
After iteratively designing and play testing 13 Fallacies<\/i>, for our first study, we administered isomorphic pre- and post-assessments in initial data collection to measure the effectiveness of the game. Assessments measured students\u2019 ability to identify common fallacies covered in the game. Before the first session students completed a pre-assessment and a post-assessment was administered on the last day of the semester.<\/p>\n
Our second study in Fall 2016 used an in vivo experimental design. In vivo experimental design refers to research that involves manipulating smaller elements of instruction and determining the effect of these manipulations on student learning (e.g. Aleven & Koedinger 2002; Koedinger, Aleven, Roll & Baker, 2009; Salden & Aleven, 2009; Salden & Koedinger, 2009). This approach differs from traditional experimental design because the study is situated in an authentic learning setting, in this case a college course, rather than in a lab setting. While similar to design-based research (Brown & Campione, 1996; Collins, 1990; The Design-Based Research Collective, 2003), the primary distinction is the inclusion of a comparison group and a single iteration of instruction.<\/p>\n
Data sources<\/i><\/p>\n
The context of this research was an academic skills course for diverse groups of incoming freshmen college students at a Midwestern regional state university campus. The course, U100<\/i>, focuses on the development of essential academic and critical thinking skills to prepare students for future college courses and promote retention. An essential component of this goal is to provide rigorous, relevant and relation-centered experiences for students. Enrolled in this course are the University\u2019s most at-risk students who are conditionally admitted because they do not meet minimum admission requirements. They enter college with SAT scores as low as the mid-700\u2019s and possess minimal skills to navigate higher education.<\/p>\n
In Fall 2015 all students played 13 Fallacies<\/i> after watching a brief video explaining the rules. In Fall 2016, we were interested in analyzing the impact of a supplemental instructional video activity on student learning. Students watched a video about fallacies in thinking and completed a worksheet. To analyze the effectiveness of this approach, some U100<\/i> sections used the video and worksheet and others did not. We administered pre- and post-assessments across these conditions to determine whether the video supplement improved students\u2019 learning outcomes when compared to gameplay alone. 72 students completed pre- and post-tests in Fall 2015 and 77 students completed pre- and post-tests in Fall 2016.<\/p>\n
Description of 13 Fallacies <\/i><\/p>\n
13 Fallacies<\/i> is played in small groups and designed around a mechanic similar to Apples to Apples<\/i>. In a round of play, each player draws 5 fallacy cards that provide a definition and example of a specific common fallacy in thinking (e.g. card stacking, appeal to pity). One player\u2019s role is the judge, and this role rotates among all players to comprise a round of play. The player in the judge role reveals a \u201cscenario\u201d card that provides a situation that contains at least three fallacies (Authors, 2015). Figure 1 illustrates fallacy and scenario cards. The scenarios are framed as relevant civic instances that relate to the experience of undergraduate students at a regional state campus. For example: There should not be an attendance policy for college classes. Students are either trusted to show up to class, or they are treated like grade school kids. Just look at all that college students have accomplished. Be a teacher who really cares about your students and drop the attendance policy<\/i>. <\/i><\/p>\n Figure 1. Fallacy and scenario cards from 13 Fallacies.<\/strong><\/p><\/div>\n The game is organized into four phases, described in Table 1. Each player other than the judge plays a fallacy card and then justifies why theirs is the best description of the scenario. The player selected by the judge receives a token signifying the best response, and the player with the most tokens at the end of the game is the winner.<\/p>\n Table 1. Phases of Play in 13 Fallacies.<\/strong><\/p><\/div>\n Results and discussion<\/b><\/p>\n Designing 13 Fallacies<\/i><\/p>\n To design 13 Fallacies<\/i>, we began with a merger of academic goals, theories of learning, and game design mechanics (Schell, 2008). After developing a prototype, we play tested the game through three iterations. During each play test we collected field notes, then conducted follow-up discussions with participants. After our first test, we identified areas that needed improvement. They included the number of cards each player would have in their hand and a mechanism for students to confirm whether the judge\u2019s ruling was accurate for a specific scenario. As a result we modified the game\u2019s rules to include players having five cards rather than 12; we also provided a key for the instructors in order to check the accuracy of the judge\u2019s ruling. Evidence from field notes and player interviews supported these improvements. Another concern that arose was how to scaffold the social negotiation that occurs once all of the cards are played. After our second play test, we developed four phases of the game, described above in Table 1. During a third play test we observed that both mechanism and approach were engaging students and we proceeded to produce the game for wide-scale implementation.<\/p>\n Study 1: 13 Fallacies promotes understanding <\/i><\/p>\n During the Fall 2015 semester, all students enrolled in U100<\/i> played 13 Fallacies<\/i> under the same conditions. Our quantitative analysis of student\u2019s pre and post-test data showed that student\u2019s ability to identify common fallacies improved after playing 13 Fallacies<\/i>. The average pre-test score was 28% (SD = 14.83). This suggests that students\u2019 initial knowledge of common fallacies prior to gameplay was limited. The post-test score average was 70.25% (SD = 12.75). The average individual gain between pre- and post-test was 36.92% (SD = 16.08). The gain in student scores was statistically significant, t(71) = 19.509, p<.001<\/i>, which indicates that 13 Fallacies<\/i> helped students learn to identify common fallacies in thinking.<\/p>\n Anecdotal analyses of student writing following the post-test indicated that students were able to not only identify common fallacies in others\u2019 thinking, but to provide written justifications of their own reasoning. This is illustrated by the examples in Table 2.<\/p>\n Table 2. Student Justifications of Fallacy Identifications.<\/strong><\/p><\/div>\n Study 2: direct instruction does not enhance learning<\/i><\/p>\n In a second study we were interested in analyzing the role of an instructional video activity as a supplement to U100<\/i> students\u2019 play of 13 Fallacies<\/i>. \u00a0In Fall 2016 some students viewed an instructional video about fallacies that was supplemented with a worksheet. To analyze the effectiveness of this approach, some U100<\/i> instructors used the worksheet with the video and others did not. We conducted pre and post-test analyses across these two conditions. Students who were in classes that used the worksheet and video as a supplement to gameplay (n = 39) had an average pre-test score of 26.50% (SD = 14.79) and post-test score of 69.23% (SD = 9.78). The mean individual gain in performance was 42.74% (SD = 13.54). Individuals in the comparison group, which did not complete the worksheet video activity (n = 38), had an average pre-test score of 27.85% (SD = 14.14) and post-test score of 73.03% (SD = 11.03). The mean individual gain in performance was 45.18% (SD = 15.21). These results indicate that mean student learning gains were approximately the same across conditions. There was no significant difference between the gain scores of the two groups, t(75) = -.744, p = .2295.<\/i> This suggests that both approaches to instruction helped students to learn. Results suggest there was no added benefit to direct instruction that used the worksheet and video.<\/p>\n Table 3. Mean Assessment Scores Across Conditions<\/strong><\/p><\/div>\n Student perceptions of 13 Fallacies<\/i><\/p>\n We were also interested in student perceptions of 13 Fallacies<\/i> as a learning tool, as compared to more traditional instructional approaches. To measure this, students completed a brief survey at the end of their post-test in Fall 2016. Students were asked to respond whether they strongly agree (4), agree (3), disagree (2) or strongly disagree (1) with the following statements: 1. Playing 13 Fallacies<\/i> helped me to learn about errors in thinking. 2. Playing 13 Fallacies<\/i> helped me to better communicate with my classmates. 3. I would have preferred listening to a lecture than playing 13 Fallacies<\/i>. 4. I would have preferred writing a paper about this than playing 13 Fallacies<\/i>.<\/p>\n Table 4 reports the percentages of students who indicated each rating for these statements. Across conditions, most students indicated they felt 13 Fallacies<\/i> helped them to learn and promoted collaboration with classmates. Furthermore, the majority of U100<\/i> students indicated that they preferred the game-based approach to learning when compared to listening to a lecture or writing a paper on the topic of fallacies in thinking.<\/p>\n Table 4. Student Perceptions of 13 Fallacies.<\/strong><\/p><\/div>\n Along with rating their agreement with these statements, students were provided with an opportunity to provide additional written comments about the game. 14 students supplied additional feedback and the comments indicated that students enjoyed playing the game (e.g. \u201cthe game is actually really helpful\u201d) or offered suggestions for how the implementation might be improved (e.g. \u201cI wish we got more time to play the game\u201d).<\/p>\n Conclusion<\/b><\/p>\n Playing 13 Fallacies<\/i> improved students\u2019 understanding of common fallacies in thinking for conditionally-admitted incoming freshmen attending a regional state university. In a second study, where we compared gameplay alone with a supplemental video and worksheet activity, we found no significant difference in student learning outcomes between these conditions. This suggests that students did not benefit from the additional instructional activity and that learning occurred through the process of play. Students self-reported that they felt the game helped them to learn about fallacies and to better communicate with classmates. Students also indicated that they preferred gameplay over listening to a lecture or writing a paper about the topic.<\/p>\n This research contributes to a larger discussion on how best to increase the quality of undergraduates\u2019 critical thinking and argumentation skills by focusing on identifying common fallacies in self, others and cultural beliefs. Through adopting a \u2018lightly-contextualized\u2019 approach, students are provided with opportunities to actively engage with their peers while analyzing their own and others\u2019 thinking as a means to develop habits of mind necessary for success in college. Further, since our research targeted a group of diverse, conditionally-admitted, at-risk undergraduate students, our goal was to provide scaffolding for the skills of productive argumentation that are often nuanced or bound to explicit contexts, hidden in a way that prevents abstraction and transfer to new domains (Perkins & Salomon, 1989). Through our design, we make explicit the common errors in thinking and provide opportunity that promises to promote enduring understanding, prepare students for future learning, and create a more equitable learning experience for students who might have limited development and prior knowledge of these skills. 13 Fallacies<\/i> has the potential to be adapted to various contexts as a way to promote students\u2019 critical thinking, argumentation skills and general communication.<\/p>\n References<\/b><\/p>\n Aleven, V. and Koedinger, K.R. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computer-based Cognitive Tutor. Cognitive Science<\/i>, 26(2).<\/p>\n Bochennek, K., Wittekindt, B., \u00a0Zimmermann, S.Y. and Klingebiel, T. (2007). More Than Mere Games: A Review of Card and Board Games for Medical Education.\u00a0Medical Teacher<\/i>\u00a029(9), 941-948.<\/p>\n Bransford, J.D. and Schwartz, D.L. (1999). Rethinking transfer: A simple proposal with multiple implications. Review of Research in Education<\/i> 24: 61-101.<\/p>\n Brown, A.L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning <\/i> Sciences<\/i> 2(2): 141-178.<\/p>\n Brown, A.L. and Campione, J.C. (1996). Psychological theory and the design of innovative learning environments. In Schauble, L. and Glaser, R. (eds) Innovations in learning: New environments for education<\/i>. Hillsdale, NJ: Lawrence Erlbaum Associates.<\/p>\n Cavagnetto, A.R. (2010). Argument to foster scientific literacy: A review of argument interventions in K-12 science contexts. Review of Educational Research<\/i> 80(3): 336-371.<\/p>\n Casner-Lotto, J. and Barrington, L. (2006). Are They Really Ready to Work? Employers’ Perspectives on the Basic Knowledge and Applied Skills of New Entrants to the 21st Century U.S. Workforce. Report by Partnership for 21st Century Skills<\/i>.<\/p>\n Collins, A. (1990). Toward a Design Science of Education<\/i>. New York, NY: Center for Technology in Education.<\/p>\n Gee, J. (2003). What Video Games Have to Teach Us about Learning and Literacy <\/i>(2nd ed). Houndmills, Basingstoke, Hampshire, England; Palgrave Macmillan.<\/p>\n Gellin, A. (2003). The effect of undergraduate student involvement on critical thinking: A meta-analysis of the literature 1991-2000.\u00a0Journal of college student <\/i> development<\/i> 44(6): 746-762.<\/p>\n Halpern, D. (2003). Thought and Knowledge (4th ed). Mahwah, NJ: Erlbaum.<\/p>\n Hay, D.F. and Ross, H.S. (1982). The social nature of early conflict. Child Development<\/i> 53: 105-113.<\/p>\n Koedinger, K.R., Aleven, V., Roll, I. and Baker, R. (2009). In vivo experiments on whether supporting metacognition in intelligent tutoring systems yields robust learning. In Hacker, D.J., Dunlosky, J. and Graesser, A.C. (eds) Handbook of Metacognition in Education, The Educational Psychology Series<\/i>. New York: Routledge: pp.897-964.<\/p>\n Kuh, G.D. (2008). High-impact educational practices: What they are, who has access to them, and why they matter.<\/i> Washington, DC: AACU.<\/p>\n Kuhn, D. (2010). Teaching and learning science as argument. Science Education<\/i>: 810-824.<\/p>\n [Authors, 2015, omitted for peer review]<\/p>\n Perkins, D.N. and & Salomon, G. (1989). Are cognitive skills context bound? Educational Researcher<\/i> 18 (1): 16-25.<\/p>\n Reese, C. and Wells, T. (2007). Teaching Academic Discussion Skills With a Card Game.\u00a0Simulation & Gaming\u00a0<\/i>38(4): 546-555.<\/p>\n Salden, R.J.C.M. and Aleven, V. (2009). In vivo experimentation on self-explanations across domains. Symposium at the Thirteenth Biennial Conference of the European Association for Research on Learning and Instruction.<\/i><\/p>\n Salden, R.J.C.M. and Koedinger, K.R. (2009). In vivo experimentation on worked examples across domains. Symposium at the Thirteenth Biennial Conference of the European Association for Research on Learning and Instruction.<\/i><\/p>\n Schell, J. (2014). The Art of Game Design: A book of lenses<\/i>. Boston, MA: Elsevier.<\/p>\n Squire, K.D. and Jan, M. (2007). Mad City Mystery: Developing Scientific Argumentation Skills With a Place-based Augmented Reality Game on Handheld Computers.\u00a0Journal of Science Education and Technology\u00a0<\/i>16(1): 5-29.<\/p>\n Swanson, K. (2014). Digital games and learning: A world of opportunities. \u00a0Tech & Learning <\/i>34(6): 22.<\/p>\n The Design-Based Research Collective (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher<\/i> 32(1): 5-8.<\/p>\n Toulmin, S. (1958). The Uses of Argument. <\/i>Cambridge: Cambridge University Press.<\/p>\n Toulmin, S., Rieke, R. and Janik, A. (1984). An introduction of reasoning. New York: Macmillan.<\/p>\n <\/p>\n","protected":false},"excerpt":{"rendered":" Julia Gressick, Ph.D. Joel B. Langston Author Note Julia Gressick PhD is an Associate Professor of Instructional Technology at Indiana University South Bend. Joel B. Langston is a graduate student of Liberal Studies and Manager of Media Services at Indiana University South Bend. Communication regarding this manuscript should be directed to Julia Gressick at: Department… <\/p>\n![\"\"](\"https:\/\/everypiecematters.com\/jget\/wp-content\/uploads\/2019\/03\/figure1.png\")
![\"\"](\"https:\/\/everypiecematters.com\/jget\/wp-content\/uploads\/2019\/03\/Screen-Shot-2019-03-17-at-3.54.27-AM.png\")
![\"\"](\"https:\/\/everypiecematters.com\/jget\/wp-content\/uploads\/2019\/03\/Screen-Shot-2019-03-17-at-3.55.25-AM.png\")
![\"\"](\"https:\/\/everypiecematters.com\/jget\/wp-content\/uploads\/2019\/03\/Screen-Shot-2019-03-17-at-3.58.33-AM.png\")
![\"\"](\"https:\/\/everypiecematters.com\/jget\/wp-content\/uploads\/2019\/03\/Screen-Shot-2019-03-17-at-3.59.27-AM.png\")