Phases of Learning: A Framework for Teaching AP World History Based on Cognitive Psychology by Tiffany Bullock A project submitted in partial fulfillment of the requirements for the degree of MASTER OF EDUCATION with an emphasis in CURRICULUM AND INSTRUCTION WEBER STATE UNIVERSITY Ogden, Utah June 23, 2023 Approved Penée W. Stewart, Ph.D. Clay Rasmussen Clay Rasmussen (Nov 30, 2023 07:58 MST) Clay L Rasmussen, Ph.D. Stephanie Speicher, Ph.D. Phases of Learning 2 Table of Contents List of Tables ...................................................................................................................................3 Abstract ............................................................................................................................................4 Acknowledgments .......................................................................................................................5 Nature of the Problem ......................................................................................................................7 Literature Review.............................................................................................................................9 Bloom’s Taxonomy, Historical Thinking Skills and the AP History Curricula ..........................9 The Flipped Classroom .............................................................................................................13 Retrieval Practice ......................................................................................................................18 Purpose...........................................................................................................................................20 Method ...........................................................................................................................................21 Context ......................................................................................................................................21 Procedure ...................................................................................................................................21 Organization of the Phases of Learning using Bloom’s Taxonomy .....................................23 Evaluation ..................................................................................................................................23 Results ............................................................................................................................................24 Feedback on Phase 1: Know and Understand ...........................................................................24 Feedback on Phase 2: Evaluate and Create ...............................................................................25 Feedback on Phase 3: Evaluate and Create ...............................................................................26 Feedback on the Phases of Learning Curricular Framework as a Whole .................................27 Discussion ......................................................................................................................................30 Recommendations and Dissemination ...........................................................................................32 References ......................................................................................................................................33 Appendix A ....................................................................................................................................38 Appendix B ....................................................................................................................................53 Appendix C ....................................................................................................................................55 Phases of Learning 3 List of Tables Table 1: AP Historical Thinking Skills…………………………………………………………..10 Table 2: AP History Reasoning Processes………………………………………………………. 11 Table 3: AP World History: Modern Exam Results by Year ……………………………………31 Phases of Learning 4 Abstract The Phases of Learning is a curricular framework designed for content-heavy courses that also require students to demonstrate high-order cognitive skills. This framework was specifically designed for the AP World History: Modern course with the aim to help students learn and retain a large amount of historical content (historical facts and narrative) while learning and progressing in the higher-cognitive order AP Historical Thinking and Reasoning Skills and pass the AP exam at the end of the school year. The Phases of Learning method divides each of the major units of study into three “phases” that build in cognitive difficulty based on Bloom’s Taxonomy. This framework also incorporates retrieval practice, a practice proven in cognitive psychology to be effective for the memorization and retention of large amounts of information. In the Phases of Learning framework, students learn the same content three times progressively as they hone their historical thinking and reasoning skills. This repetition of the content, combined with multiple opportunities for retrieval practice, helps students commit the content to memory as they continue to draw from their content knowledge in demonstrating their historical skills in their writing. A tutorial on this framework was developed to help educators implement it in their own practice. Four AP World History teachers used the tutorial to evaluate the Phases of Learning, and all four are considering adopting it. Keywords: cognitive psychology, Bloom’s taxonomy, cognitive skills, curricular framework, content retention, historical thinking skills, historical reasoning skills, world history, retrieval practice. Phases of Learning 5 Acknowledgments I would like to thank all of those who assisted me in completing this project. I would like to thank my committee chair, Dr. Penée Stewart, for all of her warm and kind encouragement and for her enthusiasm for my project. I also owe a big thank you to Dr. Louise Moulding, Dr. Stephanie Speicher, and Dr. Clay Rasmussen for offering great insights and direction. I am grateful that I started this project with over a decade of teaching experience to draw from. This is the culmination of my experience in the Weber State University Master of Education program. I benefitted tremendously from my master's courses in my own practice. Each course helped me better understand my student’s needs and the most innovative and effective ways to meet them. As an educator of 8th, 9th, 10th, and 11th-grade students, I always strived to try out new ideas, but this program helped me find more research-supported ideas, which made all the difference. I am grateful that during my coursework in this program, I was chosen to open a brand new high school and take on the AP World History course there. This teaching assignment stretched me as an educator like no other course! Its unique challenges forced me to be innovative and the result is this curriculum project. Just before taking on this new challenge, my dear friend, Rebecca Goekeritz, introduced me to a little book called Make it Stick. This book introduced me to new research in cognitive psychology that I had never heard of before, much of which felt counter-intuitive and opposed most of my teaching strategies at the time! This book and the research summarized within its pages became the backbone of this curriculum project. My students deserve to be thanked here as well. Over 400 students experienced the development of this curricular framework! They trusted me and followed my direction, and did the hard work. I am so grateful for these brilliant young people! Phases of Learning 6 Finally, I need to express tremendous gratitude to my husband, Dr. Casey Bullock, who gave me much-needed, gentle encouragement to complete my master's program after our surprise miracle daughter, Birdie, came into our lives. These two are my favorite people in the whole world and gave me the inspiration I needed to finish what I had started. 7 Nature of the Problem Some educators are tasked with content-heavy courses that make teaching and learning challenging. A great example of this type of course is Advanced Placement (AP) World History: Modern. These educators have the enormous challenge of teaching a large amount of content as well as critical cognitive skills and disciplinary practices (Aidinopoulou & Sampson, 2017; Gaughan, 2014). What is generally referred to as “content” are the seemingly innumerable historical terms and facts: people, places, events, ideas, and processes from the historical record. One would think this is all one needs to learn in a history class, but history is so much more than the content. Students must not only know the history but also have the skills necessary to do history (Wineburg, 2001). Historical thinking skills and reasoning processes include to name a few, contextualization, argumentation, sourcing, and comparison. These skills are developed by analyzing historical evidence, formulating a thesis, identifying patterns and turning points, and comparing historical periods and regions (Aidinopoulou & Sampson, 2017; “AP World History: Modern,” 2020). These skills are usually demonstrated in historical research and writing. Knowing and understanding the content is a prerequisite to demonstrating these skills. So, the challenge for World History teachers is enormous: volumes of content that need to be learned to hone students’ higher cognitive skills. Using Bloom’s revised taxonomy to understand the different cognitive natures of these objectives can help teachers plan the most effective delivery of instruction. Learning the content falls under the lower cognitive categories of Remember and Understand in the taxonomy as well as the Factual Knowledge dimension. The historical thinking skills and reasoning processes fall under the higher cognitive categories of Apply, Analyze, Evaluate, or Create in the taxonomy as well as the Conceptual Knowledge and Procedural Knowledge dimensions (Krathwohl, 2002). Phases of Learning 8 Factual knowledge should be relegated to homework, while the skills best practiced under the teacher’s supervision should be practiced in class. However, if the content is relegated to outside of class, how can teachers best help students learn and understand it and retain it to be used in class and pass the AP exam at the end of the school year? This presents another enormous challenge. AP World History educators need a methodology that moves their students through the content quickly and builds up their understanding of the higher cognitive skills required on the AP exam. They need activities that maximize retention that students can do confidently outside of class without the teacher’s supervision. In the last decade, the “Flipped Classroom” (FC) concept gained popularity because it freed up class time usually taken up by lectures to work on high-order thinking skills. Video lectures relegated this more passive learning activity to outside of class (Bergmann & Sams, 2012; Finkel, 2012; Gaughan, 2014; Kim et al., 2014; Milman, 2014; Roach, 2014; Zainuddin & Halili, 2016). Lectures are often used in fast-paced, content-heavy survey courses because they offer the illusion that much content is being “covered” in a relatively short period (Gaughan, 2014; Snyder et al., 2016). The FC does not, however, maximize retention on its own. Students passively viewing lecture videos and taking notes is ineffective for long-term retention. In addition, some courses, like AP World History, still have too much content for this solution, as video lectures cannot replace assigned readings in the necessary depth and detail required of the course. As a general rule, lecture videos need to be reasonably short – especially in K12 courses – so some content-heavy courses still have too much content to be taught in lecture homework videos (Gaughan, 2014). Many of these types of courses also rely on students reading the Phases of Learning 9 textbook outside of the classroom. It is unreasonable to ask students to read a chapter of text AND watch a lecture video outside of class. Content-heavy courses also face the issue of only covering topics once as a class and relying on students to re-read their notes or the textbook to review for summative assessments. Recent research in cognitive science has shown that re-reading notes or the textbook is ineffective for retention (Brown et al., 2014; Carpenter, 2009; Dunlosky et al., 2013; Chan et al., 2006). So, the traditional approach to teaching these types of courses, even with the implementation of the FC, does not set learners up for long-term success. If re-reading notes and texts are not effective, what is? Self-quizzing, also known as retrieval practice, has proven to be one of the most effective ways for students to retain more information and perform better on summative assessments (Brown et al., 2014; McDermott et al., 2014; Roediger et al., 2011). Combined with “spacing” and “interleaving,” this strategy maximizes retention. “Spacing” refers to the spacing of retrieval practice sessions far enough apart to make retrieval more difficult. Ideally, retrieval practice should be done just as one begins to forget the information (Brown et al., 2014; Carpenter et al., 2009). “Interleaving” is the practice of mixing unrelated information in a retrieval practice session. Essentially, the more the brain struggles during retrieval, the more permanent the knowledge becomes (Brown et al., 2014). What if these more effective strategies for retention were built into a course rather than relying on students to figure out the more effective ways to study for tests on their own time? Literature Review Bloom’s Taxonomy, Historical Thinking Skills, and the AP History Curricula Benjamin S. Bloom’s Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook I: Cognitive Domain was initially developed by a team he put Phases of Learning 10 together as a measuring tool to help write comprehensive exam questions (Bloom et al., 1956). It has since become a cornerstone in curriculum development as it helps educators avoid overusing objectives that require only recognition or recall in pursuit of more cognitively challenging learning objectives (Krathwohl, 2002). In 2002, David R. Krathwohl and his team of experts revised Bloom’s Taxonomy by changing the fifth and sixth domains from Synthesis and Evaluation to Evaluate and Create and adding Knowledge Dimensions (Krathwohl, 2002). Today the revised taxonomy is still a valuable tool educators use to help them strive for the higher cognitive domains and processes in their curricula. Teaching history today involves both low and high-order cognitive domains. Content knowledge – the who, what, when, how, and why of history – aligns with Bloom’s two lower categories of Knowledge and Comprehension as well as the Factual Knowledge dimension (Krathwohl, 2002). In the past, this is where most history teaching and learning stayed. In line with the general shift in education to stretch students’ capabilities with higher-order thinking, the history classroom has transformed in the last 30 years from being primarily about memorizing historical facts and narratives to one of investigation through the development of historical thinking skills (HTS) (Wineburg, 2001). In response to this shift, the College Board made sweeping changes to the AP history exams in 2001 and again in 2017 in assessing students’ HTS. While there are still content learning objectives that students must know, there are now six AP Historical Thinking Skills, as shown in Table 1, and three Reasoning Processes, as shown in Table 2, in the AP history curricula. Table 1 AP Historical Thinking Skills Phases of Learning Skill # Skill 1 Skill Title Developments and Processes Skill 2 Sourcing and Situation Skill 3 Claims and Evidence in Sources Skill 4 Contextualization Skill 5 Making Connections Skill 6 Argumentation 11 Skill Description Identify and explain historical developments and processes. Analyze sourcing and situation of primary and secondary sources. Analyze arguments in primary and secondary sources. Analyze the context of historical events, developments, or processes. Using historical reasoning processes (comparison, causation, continuity, and change), analyze patterns and connections between and among historical developments and processes. Develop an argument. Table 2 AP History Reasoning Processes (RP) RP# RP1 RP Title Comparison RP2 Causation RP3 Continuity and Change RP Description Describe similarities and/or differences between different historical developments or processes. Describe the causes and/or effects of a specific historical development or process. Describe patterns of continuity and/or change over time. The College Board tests these skills in addition to students’ understanding of the learning objectives in all four parts of the history exams – even the multiple-choice (MC) section. In this portion of the exam, students must use their content knowledge and HTS/HRS in “stimulusresponse” MC questions. The MC questions are grouped into sets of three or four, and each set has one or more stimuli that the students must interpret. The stimuli may be primary or secondary texts, images (artwork, photos, posters, cartoons, etc.), charts, graphs, data tables, or maps. These questions require students to synthesize what they know about historical developments and processes (the content) with their interpretation of the stimuli to come to the correct answer. All HTS and HRS are assessed in this section of the test (“AP World History: Modern,” 2020). Phases of Learning 12 After completing 55 multiple-choice questions in 55 minutes, students then begin the three Short-Answer Questions (SAQs). The first SAQ includes a secondary source stimulus, and the second includes a primary source stimulus. Students can choose between two question sets, neither of which has stimuli, for their third SAQ. Students must write out their responses in detail and recall specific historical terms. Students may encounter any of the HTS or HRS in this section of the exam (“AP World History: Modern,” 2020). The Document-Based Question (DBQ) essay is designed to showcase students’ abilities to use historical evidence to formulate and support an argument in response to a prompt. Students are given seven documents to analyze and a prompt. They then must formulate a thesis in response to the prompt and use the documents to defend their thesis statement. In addition, students are asked to contextualize the topic of the prompt by relating it to broader historical developments and processes. For another point, they must incorporate a specific and detailed piece of evidence that is NOT found in the document set – it must come from their own knowledge of the historical content. Students are also asked to source three of the documents by explaining how or why the document’s point of view, purpose, historical situation, and/or audience is relevant to their argument (“AP World History: Modern,” 2020). The final section of the AP History Exam is the Long Essay Question (LEQ) essay. Because this essay is written using historical evidence the examinee recalled, students are given three prompts to choose from to write one essay. Students are asked to respond to a prompt with a historically defensible thesis that establishes a line of reasoning. They are also asked to relate the topic of the prompt to a broader historical context. The evidence portion of this essay must all be derived from the student’s knowledge of the historical narrative. Students who cannot recall Phases of Learning 13 specific people, places, events, and other historical terms will score poorly on this part of the exam (“AP World History: Modern,” 2020). It is evident by the nature and design of the AP History courses and exams that students must simultaneously memorize a great deal of historical content as well as develop sophisticated historical thinking and reasoning skills. All portions of the exam require both content and skill. The Flipped Classroom Since the FC began to gain traction ten years ago, many articles have been published, including periodicals and research studies on the FC in most disciplines at both the K-12 and collegiate levels. In general, existing literature includes many claimed benefits and challenges of the FC and generalizable recommendations for best practices across all disciplines and educational levels. Literature specific to the K-12 history classroom supports the FC in three specific areas: student satisfaction and motivation, the facilitation of more student-centered and differentiated class activities, and the cultivation of students’ historical thinking and higher-order thinking skills (Aidinopoulou & Sampson, 2017; Gaughan, 2014; Snyder et al., 2016). Proponents of the FC method claim that the main benefit is a shift from teacher-centered, passive learning to student-centered, active learning (Bergmann & Sams, 2012; Finkel, 2012; Gaughan, 2014; Kim et al., 2014; McCrea, 2012; Milman, 2014; Roach, 2014; Zainuddin & Halili, 2016). Researchers Kim, Kim, Khera, and Getman (2014) stressed that in the FC, “students have the opportunity to become more active and interactive through group activities rather than passively listening to lectures” (p. 37). In a student-centered classroom, students have greater accountability for their learning. Zainuddin and Halili (2016) stated, “The use of the traditional learning approach, which focuses on the instructor as the centre of knowledge is irrelevant in today’s digital age” (p. 314). In a student-centered classroom, the instructor acts Phases of Learning 14 more as a facilitator of learning and is more accessible during face-to-face class time to give oneon-one instruction, monitor discussions, motivate students, and give feedback on students’ performance (Bergmann & Sams, 2012; Gaughan, 2014; Morgan, 2014; Roach, 2014; Zainuddin & Halili, 2016). Many periodicals and studies also claim that these benefits of the FC resulted in increased test scores compared with the traditional classroom (AlJaser, 2017; Balaban et al., 2016; Morgan, 2014; Zainuddin & Halili, 2016). The benefits of student-centered learning are widely accepted in education. However, many educators struggle to balance the large amount of content in their curriculum with studentcentered activities. Bergmann and Sams suggested that the FC model is perfect for courses with a heavy amount of material to cover, which is typical in many social studies courses (Bergmann & Sams, 2012; Snyder et al., 2016). Instructors with limited class time and extensive material to cover resort to lecture-based instruction. The FC alleviates this constraint (Gaughan, 2014; Roach, 2014). Lage, Platt, and Treglia (2000) found that this method allowed them to incorporate group work and active learning without sacrificing course coverage. Some have reported that covering more content in less time allowed high-achieving students to go beyond the essential content into extra enrichment activities (Finkel, 2012). Research on the FC in K-12 history courses is very limited. A mixed-method study to assess the effectiveness of the FC in a 9th-grade world history course reported greater student satisfaction but no improvement in test scores compared to the control group (Snyder et al., 2016). The first part of the five-year study featured a qualitative analysis of student engagement regarding the screencasts as reported on a survey. The qualitative data showed high levels of satisfaction with the screencasts as an educational tool. Students preferred the screencasts to other types of homework and class activities over in-class lectures. Part two of the study Phases of Learning 15 introduced a second experimental group that used interactive screencasts with quizzes. Students were required to complete quizzes throughout the screencast before continuing the screencast. Quantitative data from pre-and post-test scores were compared from three different 9th-grade world history courses that used three different methods of instruction: 1) a traditional, nonflipped classroom (the control group), 2) a flipped classroom with non-interactive screencasts, and 3) a flipped classroom with interactive screencasts. The quantitative analysis revealed no statistically significant differences in pre-and post-test scores in any of the classrooms. Snyder et al. concluded that the qualitative data show strong support for screencasting as a learning tool; however, the findings did not “firmly endorse the use of screencasts over other methods of instruction” (p. 43). (Snyder et al., 2016) A more recent study of a 5th-grade history class showed similar results. Two classes participated in the study: the control group was a traditional history classroom, and the experimental group used the flipped classroom method. The study measured learning outcomes based on international standards for teaching history: historical content memorization and historical thinking skills (HTS) cultivation. Their findings showed similar learning outcomes in the area of historical content memorization. However, in the area of HTS cultivation, the experimental group showed statistically significant improvement over the control group. The improvement shown by the experimental group in this area was attributed to the fact that the flipped classroom used much more class time on student-centered HTS cultivation activities (Aidinopoulou & Sampson, 2017). This study supports the claim that the flipped classroom allows for higher forms of cognitive development (Zainuddin & Halili, 2016). Most research on the FC in collegiate social studies courses supports claims of increased student satisfaction. A world history professor at a Colorado State University-Pueblo found that Phases of Learning 16 most of her students reported favorably on the FC on a survey (Gaughan, 2014). Seventy-five percent of the students self-reported that they watched most or all of the videos and found them useful. Most students also reported that the videos helped them prepare for class discussion all or most of the time. Most students also reported that the class discussions had the most significant impact on their learning. This result aligned with the primary goal of the professor in choosing to implement the FC in her world history course (Gaughan, 2014). A mixed-method study of three courses at a university, one of which was a sociology course, provided qualitative findings that students were satisfied and felt the more student-oriented activities resulted in a greater understanding of the course content (Kim et al., 2014). Another study of student perceptions of a partially-flipped collegiate-level microeconomics found that 76% of students felt that flipped learning helped them learn, and 94% felt that the class was more interactive than other courses (Roach, 2014). A few studies in social studies support claims of increased student performance. A university Principles of Economics course studied the effectiveness of the FC on student performance. They reported that, on average, students in the FC scored about 6.5 percentage points better than students in the control group on the final exam. The same study also supports the claim that the FC improves students’ ability to answer higher-order thinking questions. The FC significantly improved performance on comprehension, analysis, and application questions with gains of at least half of one standard deviation in performance (Balaban et al., 2016). A study conducted at an all-female college in Saudi Arabia found that the FC improved academic performance in an education course. The FC experimental group significantly outperformed the control group on pre- and post-achievement tests (Al Jaser, 2017). Phases of Learning 17 The FC has specific challenges, which have sparked some criticism of this model. A chief concern is that students will not watch the screencasts (Gaughan, 2014). The 2016 study of 9thgrade world history courses reported that 20% of the students admitted in a survey that they were not properly engaging in the screencasts either by fast-forwarding them to complete the notes more quickly or by copying notes from a classmate (Snyder et al., 2016). Others have suggested that videos are less engaging than live lectures (Tormey & Henchy, 2008) and that video lectures are too monotonous (Palaigeorgiou & Despotakis, 2010). Perhaps the most common challenge and criticism is the cost of implementing this method. Recording the screencasts and preparing meaningful activities to be completed during face-to-face instruction requires enormous work upfront (Balaban et al., 2016; Roach, 2014; Zainuddin & Halili, 2016). Another criticism is that not all teachers successfully apply the FC model (Halili & Zainuddin, 2015). The quality and length of the screencasts are significant factors in the success of this method of instruction (Gaughan, 2014; Zainuddin & Halili, 2016). Critics also point out that the FC places much emphasis on lectures which many consider to be an antiquated method of instruction (Morgan, 2014). Many articles on the FC focus on design principles and recommendations for practice. Even with the vast amount of videos available online, teachers are encouraged to produce their own videos (Finkel, 2012). According to one article, students prefer hearing their teacher’s voice (Morgan, 2014). One practitioner recommends that teachers keep their videos concise. The length was reported as the main reason students did not watch them (Gaughan, 2014). The FC is strongly recommended to history teachers looking for a way to incorporate more student-centered, hands-on HTS activities into their content-heavy course; however, current literature does not address the fact that assigned readings make up a large portion of Phases of Learning 18 outside-of-class activities in most history courses. Textbooks or other secondary reading materials are required when the breadth and depth of content are too heavy to cover in class. Because content videos should remain reasonable in length, in many instances, videos cannot replace assigned readings altogether. The FC does, however, present a solution to use class time more effectively than content coverage in the form of lectures which is an essential aspect of this project. Retrieval Practice In a traditional world history classroom, a single content topic is typically learned once during homework-assigned readings and perhaps reviewed once again in class. Students are then encouraged to “study” past readings and notes to prepare for summative assessments. However, research has shown that this strategy is not nearly as effective as retrieval practice (RP) despite being the primary mode of study for many students (Brown et al., 2014; Carpenter, 2009; Dunlosky et al., 2013; Chan et al., 2006; Glaser & Richter, 2022) due to the fact that most students think repeated rereading is the most effective way to study (Agarwal et al., 2008). Cognitive psychologists Henry L. Roediger III and Mark A. McDaniel teamed up with writer Peter C. Brown to publish much of their research in a book for all audiences that explains in layman’s terms what has been dubbed as “the testing effect” – that as a person practices recalling or retrieving new knowledge, long-term retention of the material is enhanced (Brown et al., 2014; Gonzalez, 2015). Their book, Make it Stick: The Science of Successful Learning, became a best seller and has inspired people in all vocations to think differently about how they learn and retain new knowledge. Make it Stick highlights several of Roediger and McDaniel’s published research on this potent strategy, including a 2011 study that compared student test performance after rereading with performance after no stakes quizzing. Rereading as a form of Phases of Learning 19 study produced an average score of 83%, while RP produced an average of 91% - a difference of almost a full letter grade (Roediger et al., 2011). Another classroom study found that, compared with rereading, RP improved retention considerably – by between 10 and 20 percentage points (McDermott, 2014). Researchers have repeatedly tested the advantage of RP, comparing it with other learning strategies besides rereading and looking for ways to enhance the so-called “testing effect” even further. One way to enhance the testing effect is by offering informative feedback (Leggett et al., 2018). This feedback may come in the form of a subsequent restudy, which was found further to enhance RP (Arnold et al., 2013). Researchers also found that retention was improved when learners were shown the correct answers during an RP session. Even unsuccessful retrieval attempts enhanced retention when the correct answers were immediately given (Kornell et al., 2009; Kornell et al., 2015; Richland et al., 2009). RP needs to be effortful to be most effective for retention. Research has shown that “spaced” practice is far superior to “massed practice” (Brown et al., 2014). Because spaced practice allows some forgetting to occur, the retrieval is more difficult, may require more mental operations, and may require the learner to recall other related concepts in addition to that being retrieved (Carpenter, 2009; Chan et al., 2006; Leggett et al., 2018). Another way to increase the difficulty of RP is to interleave topics (Brown et al., 2014). Retrieval practice, spacing, and interleaving were compared with seven other standard learning techniques in a 54-page monograph that rated their “utility” as low, moderate, or high based on the authors’ criteria. The ten techniques discussed included elaborative interrogation, self-explanation, summarization, highlighting/underlining, keyword mnemonic, imagery use for text learning, rereading, practice testing (RP), distributed practice (spacing), and interleaved Phases of Learning 20 practice. RP and spacing practice are the only two of these ten rated high utility (Dunlosky et al., 2013). The benefits of RP do not end with improved retention. It is also effective for learners of all capabilities and has even been shown to be of more significant benefit for students with lower working memory capacities (Agarwal et al., 2017). Its effectiveness requires no individual learning prerequisites (Glaser & Richter, 2022). In addition, researchers have also found that RP improves metacognition because the learner judges their own learning more accurately after RP sessions (Agarwal et al., 2008; Carpenter et al., 2016). Retrieval practice is a proven and potent learning tool that should be built into any content-heavy curriculum. Educators of these types of courses should provide as many opportunities for no-stakes RP as possible to increase their students’ long-term retention of the content. Purpose The purpose of this project was to create a curricular framework for the AP World History: Modern course based on Bloom’s taxonomy that utilized the most effective methods of instruction to help students learn and retain a large amount of historical content (historical facts and narrative) while learning and progressing in the higher-cognitive order AP Historical Thinking and Reasoning Skills and pass the AP exam. The name Phases of Learning was chosen for this new instructional framework as it divides major units of study into three “phases” that build on each other in cognitive difficulty and that are based on Bloom’s Taxonomy. Teachermade homework videos were incorporated to help students memorize new historical terms. Students learned the same content three times progressively as they honed their historical thinking and reasoning skills. This repetition of the content, combined with multiple opportunities for retrieval practice, including interleaving and spacing, helped students commit Phases of Learning 21 the content to memory as they continued to draw from their content knowledge in demonstrating their skills. After the Phases of Learning framework was developed, a walkthrough tutorial was created for other AP World History teachers to evaluate and potentially adopt in their classrooms. Method Context The AP World History: Modern (APWH: M) course is among the most challenging AP courses to teach and learn. The course was modified from AP World History to APWH: M in 2020 to alleviate the challenge of breadth in the original course, which included all of human history. The new course description includes only c. 1200 to the present, but while the breadth of the course was significantly decreased, the depth of those 800 years increased considerably. The course remains incredibly challenging with the amount of content and skills required to pass the AP exam. In the last year of the old course, the global pass rate was 55.4%. In the three years since, under the new curriculum, the pass rates have been 60.2% in 2020, 53.1% in 2021, and 62.1% in 2022 (data retrieved from personal AP score reports). The Phases of Learning curriculum was developed and implemented at a suburban high school in northern Utah between the years of 2019 and 2023. Enrollment in AP World History: Modern is comparatively high and fluctuates between 70-100 students a year or 3-4 sections (classes). The school’s pass rates have ranged from 82.4% to 95.9% since the school’s opening in 2018. Students enrolling in this course are motivated to pass the AP exam to earn college credit. Most of the students that take the course complete their homework on time. Procedure Phases of Learning 22 The College Board published “The AP World History: Modern Course and Exam Description” (CED) in 2018. The CED includes units, broad learning objectives with a few illustrative examples, themes, HTS and HRS, and a few general ideas for instruction. Beyond the course outline of objectives, themes, and the requisite skills, quite a lot is left to the teachers to determine themselves, such as what specific historical terms students should master (beyond a few select illustrative examples in the CED) and detailed lesson plans and materials. This project incorporated all objectives, themes, and skills as outlined in the CED; however, the following were developed in this project to address the unique challenges this course presents: • The CED divides the 800 years of history covered in the course into four historical periods: the late post-classical period (c. 1200 to c. 1450), the premodern period (c. 1450 to c. 1750), the modern period (c. 1750 to c. 1900), and the contemporary period (c. 1900 to the present). These four historical periods made up the units of study which were also divided into three phases based on Bloom’s taxonomy. • A list of crucial historical terms (people, places, events, ideas, innovations, developments, and processes) was developed for each historical period. These formed the basis of the RP that will be incorporated into instruction. • A schedule/calendar for each unit/historical period was developed, laying out the three Phases of Learning. • Homework videos and no-stakes retrieval practice activities were developed and incorporated into every phase of learning. • Summative assessments for each phase were developed. These assessments increased in cognitive difficulty with each succeeding phase. Phases of Learning 23 Organization of the Phases of Learning using Bloom’s Taxonomy Phase 1, based on Bloom’s lower tiers of “know” and “understand,” focused on the memorization and retention of the most important historical terms using retrieval practice, including spacing and interleaving. Because all of the content in this phase is new to the learner, homework video explanations of terms from the teacher were used to introduce the content terms. Class time was devoted to teacher clarification and practice in the form of games. During Phase 2, modeled after Bloom’s intermediate tiers of “apply” and “analyze,” learners added to their knowledge of the historical terms through homework textbook reading and guided reading notes. Students continued to use their content knowledge while practicing historical thinking and reasoning skills during class. Class instruction focused on additional context and refining these skills under the teacher’s supervision. Phase 3, modeled after Bloom’s top tiers of “evaluate” and “create,” furthered learners’ understanding of the content with synthesis articles and videos demonstrating the highest-order thinking skills. In this phase, learners were ready to formulate their own arguments using historical evidence. The repetition of learning the same content while increasing the cognitive difficulty of the skills while utilizing the most effective strategies for retention makes this framework ideal for content-heavy courses that require higher-order thinking and retention for whole-year summative assessments (like the AP exam). Evaluation A tutorial on the Phases of Learning curricular structure was developed to explain it and its benefits to other APWH: M teachers (Appendix A). An email was sent to fellow APWH: M teachers within the same suburban school district in northern Utah, seeking volunteers to evaluate the Phases of Learning curricular framework. Four teachers seeking a better way to Phases of Learning 24 teach the course volunteered to view the tutorial and all the course materials (given to them via a shared folder) and complete an evaluation using a provided questionnaire (Appendix B). Each of these four teachers was new to the course, having only taught it for one year; however, none are very new to the profession – all have experience teaching other courses. All evaluators were from the same suburban school district in northern Utah. One of the evaluators stated their own frustration with teaching this challenging course in an email after being invited to evaluate this project: “I’ve been constantly frustrated with the amount of content we’re asked to teach, in conjunction with HTS and HRS. I tried for a time the flipped classroom concept, but had no success. Student use of the videos was extremely low and it was challenging to make a video short enough to keep their attention, but still cover the content. I think I’m a good lecturer, but don’t like the amount of lecturing I feel like I have to do. I’ve tried to incorporate retrieval practice but find that I’m quick to discard it when there’s a time crunch (which is most of the school year.) I’ve tried a variety of other methods. Some I’ve liked, some the students have liked, but all seem to take more time than we have available. Socratic seminars, Harkness discussions, causal map discussions, group projects, solo projects. I’ve tried a lot.” Results The evaluators were asked to view the tutorial (see Appendix A), evaluate each of the three Phases of Learning, and give feedback on the curricular structure as a whole, using a provided questionnaire (see Appendix B). Feedback on Phase 1: Know and Understand Phases of Learning 25 All evaluators agreed that starting with basic content memorization set up a foundation of knowledge to build upon. On the effectiveness of this phase, one evaluator wrote that it was beneficial to begin at the students’ current level of learning, assuming that most students would come into the course with little or no experience with the content. Another evaluator saw the benefit of phase 1 in helping students establish habits regarding homework and class participation. Another commented that learning key historical terms for an entire historical period would help students “gain big-picture understandings of historical narratives while helping with retention.” Another insight given was that new content learned during phase 1 was delivered in more “easily digestible bites,” preparing them for “more intense and higher forms of cognition.” Overall, evaluators saw phase 1 as a beneficial start to student learning; however, one evaluator expressed concern that too much class time was being used in retrieval practice games and thought some skills should begin to be taught during this phase. This educator also suggested that retrieval practice be continued into phase 2. Feedback on Phase 2: Apply and Analyze When asked about the most effective aspects of phase 2, evaluators understood the intention of its design very clearly. One noted that phase 2 increased complexity without overtaxing students and that introducing analytical thinking in short-answer questions (SAQs) was a good preparation for the more complex DBQ and LEQ essays introduced later in phase 3. Another evaluator pointed out that students are able to test out their newly acquired phase 1 knowledge and combine it with the new skills learned in class. All evaluators agreed that teacher-led in-class instruction demonstrating HTS and HRS using content learned at home in their textbook reading and note-taking was the most valuable aspect of phase 2. The use of Phases of Learning 26 primary sources to add complexity and nuance to their growing understanding of the content was explicitly singled out as a critical use of class time. One evaluator noted that many students in 10th grade in northern Utah view history as “discipline concerned solely with facts” and that phase 2 lessons would help them understand what historians actually do – interpret historical evidence. When asked about the least effective aspects of phase 2, the evaluators focused on the homework and test corrections during this phase. One evaluator expressed concern with the style of the phase 2 reading notes guides because they may encourage students to “fish” for the correct answers in the text rather than read it for comprehension. This person suggested more openended questions. This evaluator also suggested the use of timelines in their notes to help them picture and understand the chronology better. Another reviewer also expressed concern regarding the reading and note-taking homework during this phase as being more of a burden on students. This person suggested introducing the textbook in class to demonstrate note-taking and annotation (close reading) skills. Another evaluator suggested improvements to the test corrections form used after the phase 2 MCQ and SAQ test by adding a column for them to explain why the correct answer is correct or why the incorrect answer is incorrect, as well as rewrite their SAQ response. Feedback on Phase 3: Evaluate and Create One evaluator highlighted using more complex texts during phase 3 as especially beneficial because students would be introduced to new ways to think about what they have already learned in previous phases. Another appreciated that students would be more prepared for the high-order cognitive skills, evaluate and create, after phases 1 and 2. They pointed out that many sophomores may not have been previously challenged at this level before taking this Phases of Learning 27 course, so it was important to wait for students to have sufficient background before delving into a skill like argumentation. The other two evaluators singled out the scaffolded writing lessons as the most beneficial aspect of phase 3. Most AP World History teachers would agree that the DBQ and LEQ essays are the most complex parts of the AP exam and the most difficult to teach. Having class time dedicated to teaching and modeling how to most effectively analyze the evidence, write a thesis statement, contextualize the topic of the prompt, point out nuances in the sources, and perhaps even present a counterargument is crucial to student success. One evaluator explained that “argumentation is a challenging skill, and as writing is a difficult skill to teach anyways, chunking the steps of the writing and thinking process for students provides them with steps to follow when it comes to writing and arguing on their own. Additionally, focusing on a few DBQ/LEQ skills at a time prevents student overwhelm.” Only two evaluators had suggestions for improving phase 3. One suggested that the homework during this phase could incorporate more writing practice. The other expressed concern about not having enough time to give students individualized feedback on their writing practice and that new AI technology may be implemented to give students immediate feedback on their practice writing. Feedback on the Phase of Learning Curricular Framework as a Whole On the benefits and effectiveness of the Phases of Learning, two evaluators provided helpful insights best stated in their own words. One said that “the most important benefit of this framework is the ability to interact with a large amount of course content multiple times and at varying levels of complexity. This is the greatest challenge for a teacher of a course like AP World History. Too much content, too little time.” The other evaluator wrote: Phases of Learning 28 “This framework’s careful attention to learning historical narratives before using knowledge to craft arguments provides a helpful order of operations for other teachers to follow. The guided practice and resources students have for at-home studying (videos, multiple textbooks, primary sources, etc.) provide support for students in ways that benefit a variety of preferences. However, the most valuable part of this curricular framework is the time/space it allows for full class periods dedicated specifically to argumentation and writing practice!” The other two evaluators echoed sentiments in their feedback on the individual phases. When asked about weaknesses in the framework, three of the four expressed concerns about having enough time to implement all three phases for all four historical periods. One of the four worried that with all that was being asked of students, some may not have the “time, resources, or academic skills to learn so much of the content at home.” Suggestions to improve the framework included using discussion circles, such as Socratic or Harkness, to help students engage with complex texts. A different reviewer wondered if the reading notes could be replaced with formative assessments with the same effect. One evaluator’s suggestion is worth including in its entirety in their own words: “I wonder if creating more overlap between Phase 1 and Phase 2 could ensure students get more consistent skill practice. I wonder if retrieval games could be spaced out to be included in class during both phases and if some source work could be helpful in analyzing the perspective and historical context of primary sources in class during Phase 1. I wonder if these types of activities, paired with retrieval practice, could ultimately boost student retention while improving their skills.” Phases of Learning 29 The fourth evaluator stated they felt they lacked sufficient experience teaching the course to make suggestions. When asked if they plan to implement the Phases of Learning framework in their classrooms, all four said they would adopt it and adjust as needed. Each of their statements regarding their own implementation is as follows: • “Absolutely. I struggle every year to cover the amount of content that is required for this course while simultaneously teaching students writing and analytical skills. If this curriculum works as described, and Mrs. Bullock’s students’ exam results indicate that it does, it may solve the content vs. skills vs. time problem.” • “I do plan on adopting this framework in my AP World History class next year. Time will tell how much I use, adapt, or substitute over the coming months.” • “I am a first-year teacher of AP World and I felt like I was drowning ALL year. I need to make adjustments that make the learning simpler and a process that makes sense to both my students and myself. I feel like this does just that. Would I consider adopting this framework into my teaching? YES!” • “Yes! While I would make some adjustments in combining aspects of Phase 1 and Phase 2, I know my students need more retrieval practice of content in order to be successful in crafting historical arguments. This type of retrieval practice and focus on terms would benefit my students as they need specific examples to support the broad trends and patterns they are responsible for knowing in world history. I also love the use of primary and secondary sources in class and at home to provide lots of writing practice in class. The more extended writing time students have with a teacher, the better! Writing and historical argumentation are extremely difficult skills, and most sophomores are novices.” Phases of Learning 30 Discussion The professional evaluation of the Phases of Learning framework did highlight a few flaws and needed improvements that are worth consideration. Some of the activities and skills in the phases could be overlapped more, especially as the year progresses. For instance, retrieval practice, which is heavily implemented in phase 1, needs to be continued into phases 2 and 3 to continue improvements in content retention. Also, students could benefit from being introduced to Phase 2 skills in Phase 1 and to Phase 3 skills while in Phase 2. Making the phases too rigid may not serve the students best, especially after students have been through all three phases at least one time because, at that point, they would have been introduced to most of the skills and may be able to move on to the highest order thinking skills earlier than phase 3. The Phases of Learning system does allow teachers to do this. Lessons, activities, and homework can be adjusted and improved without removing the entire framework. That one evaluator was already considering their own tweaks while adopting the framework as a whole shows that there is flexibility to do so. One resounding truth became clear during the evaluation: AP World History: Modern teachers struggle similarly with too much content, too many skills, and too little time. The problems this project set out to address are very real to educators. At the very least, the Phases of Learning curricular framework provides order to perceived chaos for the educator and learner. Placing all learning objectives within Bloom’s Taxonomy and addressing them systematically according to cognitive difficulty provides a logical order to the demands of the course. In addition to this more logical ordering of objectives, the specific attention to the latest research in cognitive psychology regarding retention helps teachers better prepare students for the AP exam. Those new to this approach to the APWH: M course saw how this framework would help them Phases of Learning 31 help their students navigate the course. That all four evaluators are planning to adopt the Phases of Learning in their own classrooms speaks volumes. The AP Exam pass rates, as shown in Table 3, were also compelling to the evaluators as possible evidence of its efficacy. The Phases of Learning framework was implemented as it was developed over four school years. The framework was improved each year until all the goals outlined in the procedure were accomplished in 2023. Students were taught about Bloom’s Taxonomy and the research behind the practices in cognitive psychology that were being used in their learning experience to achieve student buy-in to the Phases of Learning. Table 3 presents AP exam scores from 2019 to 2023 of the students that were taught using the Phases of Learning during its development. It should be noted that the 2019 scores are from before the Phases of Learning framework was entirely conceived and before the College Board changed the course from AP World History to AP World History: Modern, as explained earlier. Table 3 AP World History: Modern Exam Results by Year Year 2019 Number of Students who took the AP 91 Exam Percentage of Students who Passed with a 82.4% score of 3 or higher Percentage of Students who Scored a 3, 34% “Qualified” Percentage of Students who Scored a 4, 34% Very Well Qualified” Percentage of Students who Scored a 5, 14% “Extremely Well Qualified” 2020 83 2021 88 2022 70 2023 73 90.4% 88.6% 91.4% 95.9% 31% 28.8% 21% 23% 37% 39.8% 37% 40% 22% 22.7% 33% 33% The most notable improvements in student performance on the AP Exam can be seen in the distribution of the scores. As the Phases of Learning methodology was being adjusted and improved over the last four years, the percentage of students scoring at 5 – the highest score on Phases of Learning 32 AP exams – more than doubled. By 2022, 70% of students scored at a 4 or 5, which according to the College Board, represents “very well qualified” and “extremely well qualified.” Pass rates across all AP exams at this particular school are generally higher than the state and national averages; however, to have a large majority of students scoring above the minimum “passing” score is exceptional. This is evidence that the Phases of Learning framework improves student retention of historical content and their ability to demonstrate higher-order cognitive skills on the exam. Recommendations and Dissemination Although the Phases of Learning was developed for AP World History: Modern, educators struggling to teach any course with similar demands could consider adopting this framework. Because it is a framework rather than a curriculum, it is transferable to other fields of study. Begin by determining where the learning objectives fit into Bloom’s Taxonomy. Then use retrieval practice to help students commit vocabulary and important terms to memory and end with the most demanding cognitive skills. Reserve precious class time for modeling, teaching, and practicing the most complex cognitive skills and trust students to learn essential content at home. Encourage students to become active participants in their learning and help them feel growth and progress in their journey. The Phases of Learning promotes all these recommendations and could be the key to success for many more educators and students. As a result of the professional evaluation, the Phases of Learning framework is being introduced fully in the AP World History: Modern courses in at least two other high schools in northern Utah following the completion of this project. AP World History, and courses like it, present a tremendous challenge to educators and it is hoped that the Phases of Learning can be the solution many are seeking. Ideas to disseminate the Phases of Learning framework include Phases of Learning 33 presenting at educator conferences, podcast media, or AP seminars. There is a national Facebook group of AP World History educators where ideas and teaching materials are shared. This is a great platform to share the Phases of Learning approach to teaching this course. The tutorial will be shared and question and answer sessions over Zoom will be offered for those who are interested. References Aidinopoulou, V. & Sampson, D. G. (2017). An action research study from implementing the flipped classroom model in primary school history teaching and learning. Journal of Educational Technology & Society, 20(1), 237–247. Agarwal, P. K., Karpicke, J. D., Kang, S. H. K., Roediger III, H. L., & McDermott, K. B. (2008). Examining the testing effect with open- and closed-book tests. Applied Cognitive Psychology, 22(7), 861–876. https://doi-org.hal.weber.edu/10.1002/acp.1391 Agarwal, P. K., D'Antonio, L., Roediger III, H. L., McDermott, K. B., & McDaniel, M. A. (2014). Classroom-based programs of retrieval practice reduce middle school and high school students’ test anxiety. Journal of Applied Research in Memory and Cognition, 3(3), 131-139. https://doi-org.hal.weber.edu/10.1016/j.jarmac.2014.07.002 Agarwal, P. K., Finley, J. R., Rose, N. S., & Roediger, H. L. 3rd. (2017) Benefits from retrieval practice are greater for students with lower working memory capacity. Memory, 25, 764– 771, https://doi-org.hal.weber.edu/10.1080/09658211.2016.1220579 Al Jaser, A. M. (2017). Effectiveness of Using Flipped Classroom Strategy in Academic Achievement and Self-Efficacy among Education Students of Princess Nourah Bint Abdulrahman University. English Language Teaching, 10(4), 67–77. Phases of Learning 34 AP World History: Modern: Course and Exam Description. (2020). Retrieved October 15, 2022, from https://apcentral.collegeboard.org/media/pdf/ap-world-history-modern-course-andexam-description.pdf Arnold, K. M., & McDermott, K. B. (2013). Test-potentiated learning: Distinguishing between direct and indirect effects of tests. Journal of Experimental Psychology: Learning, Memory, and Cognition, 39(3), 940-945. https://doi.org/10.1037/a0029199 Balaban, R. A., Gilleskie, D. B., & Tran, U. (2016). A quantitative evaluation of the flipped classroom in a large lecture principles of economics course. Journal of Economic Education, 47(4), 269–287. https://doi.org/10.1080/00220485.2016.1213679 Bergmann, J., & Sams, A. (2012). Flip Your Classroom : Reach Every Student in Every Class Every Day (Vol. 1st ed). Eugene, Or: ISTE. Bloom, B. S. (1956). Taxonomy of educational objectives: The classification of educational goals. D. McKay Co., Inc. Brown, P. C. (2014). Make it stick: The science of successful learning. The Belknap Press of Harvard University Press. Carpenter, S. K. (2009). Cue strength as a moderator of the testing effect: The benefits of elaborative retrieval. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(6), 1563–1569. doi:https://doi-org.hal.weber.edu/10.1037/a0017021 Carpenter, S. K., Pashler, H., & Cepeda, N. J. (2009). Using tests to enhance 8th grade students' retention of U.S. history facts. Applied Cognitive Psychology, 23(6), 760-771. https://doi.org/10.1002/acp.1507 Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students' learning with effective learning techniques: Promising directions Phases of Learning 35 from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4–58. https://doi.org/10.1177/1529100612453266 Finkel, E. (2012). Flipping the script in K12. District Administration, 48(10), 28–34. Gaughan, J. E. (2014). The flipped classroom in world history. The History Teacher, 47(2), 221– 244. Glaser, J., & Richter, T. (2022). The testing effect in the lecture hall: does it depend on learner prerequisites? Psychology Learning & Teaching, 0(0). https://doiorg.hal.weber.edu/10.1177/14757257221136660 Gonzalez, J. (2015, July 15). Interview with Peter C. Brown [Audio podcast]. Cult of Pedagogy. Retrieved from https://www.cultofpedagogy.com/stick/ Kim, M. K., Kim, S. M., Khera, O., & Getman, J. (2014). The experience of three flipped classrooms in an urban university: An exploration of design principles. Internet & Higher Education, 22, 37–50. https://doi.org/10.1016/j.iheduc.2014.04.003 Kornell, N., Hays, M. J., & Bjork, R. A. (2009). Unsuccessful retrieval attempts enhance subsequent learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(4), 989-998. https://doi-org.hal.weber.edu/10.1037/a0015729 Kornell, N., Klein, P. J., & Rawson, K. A. (2015). Retrieval attempts enhance learning, but retrieval success (versus failure) does not matter. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41(1), 283-294. https://doi.org/10.1037/a0037850 Krathwohl, D. R. (2002). A revision of Bloom's Taxonomy: an overview. Theory into Practice, 41(4), 212+. Phases of Learning 36 Lage, M. J., Platt, G. J., & Treglia, M. (2000). Inverting the classroom: A gateway to creating an inclusive learning environment. The Journal of Economic Education, 31(1), 30–43. https://doi.org/10.1080/00220480009596759 Leggett, J. M. I., Burt, J. S., & Carroll, A. (2019). Retrieval practice can improve classroom review despite low practice test performance. Applied Cognitive Psychology, 33(5), 759– 770. https://doi-org.hal.weber.edu/10.1002/acp.3517 McCrea, S. (2012). Transforming Teachers, Transforming Schools: Turning “Sages” into “Guides on the Side.” Turkish Online Journal of Distance Education, 13(3), 11–16. McDermott, K. B., Agarwal, P. K., D'Antonio, L., Roediger III, H. L., & McDaniel, M. A. (2014). Both multiple-choice and short-answer quizzes enhance later exam performance in middle and high school classes. Journal of Experimental Psychology: Applied, 20(1), 3-21. https://doi-org.hal.weber.edu/10.1037/xap0000004 Milman, N. B. (2014). The flipped classroom strategy. Distance Learning, 11(4), 9–11. Morgan, H. (2014). Focus on technology: Flip your classroom to increase academic achievement. Childhood Education, 90(3), 239–241. https://doi.org/10.1080/00094056.2014.912076 Palaigeorgiou, G., & Despotakis, T. (2010). Known and unknown weaknesses in software animated demonstrations (screencasts): a study in self-paced learning settings. Journal of Information Technology Education: Research, 9, 081–098. Richland, L. E., Kornell, N., & Kao, L. S. (2009). The pretesting effect: Do unsuccessful retrieval attempts enhance learning? Journal of Experimental Psychology: Applied, 15(3), 243-257. https://doi-org.hal.weber.edu/10.1037/a0016496 Phases of Learning 37 Roach, T. (2014). Student perceptions toward flipped learning: New methods to increase interaction and active learning in economics. International Review of Economics Education, 17(Supplement C), pp. 74–84. https://doi.org/10.1016/j.iree.2014.08.003 Roediger, H. L., Agarwal, P. K., McDaniel, M. A., & McDermott, K. B. (2011). Test‐enhanced learning in the classroom: Long‐term improvements from quizzing. Journal of Experimental Psychology: Applied, 17(4), 382–395. https://doi.org/10.1037/a0026252 Snyder, C., Besozzi, D., Paska, L., & Oppenlander, J. (2016). Is flipping worth the fuss: A mixed methods case study of screencasting in the social studies classroom. American Secondary Education, 45(1), 28–45. Tormey, R., & Henchy, D. (2008). Re-imagining the traditional lecture: an action research approach to teaching student teachers to ‘do’ philosophy. Teaching in Higher Education, 13(3), 303–314. https://doi.org/10.1080/13562510802045337 Wineburg, S. S. (2001). Historical thinking and other unnatural acts: Charting the future of teaching the past. Temple University Press. Zainuddin, Z., & Halili, S. H. (2016). Flipped classroom research and trends from different fields of study. The International Review of Research in Open and Distributed Learning, 17(3). https://doi.org/10.19173/irrodl.v17i3.2274 Phases of Learning 38 Appendix A Phases of Learning 39 Phases of Learning 40 Phases of Learning 41 Phases of Learning 42 Phases of Learning 43 Phases of Learning 44 Phases of Learning 45 Phases of Learning 46 Phases of Learning 47 Phases of Learning 48 Phases of Learning 49 Phases of Learning 50 Phases of Learning 51 Phases of Learning 52 Appendix B Phases of Learning Framework Questionnaire Phases of Learning 53 Thank you for taking the time to review the Phases of Learning framework for teaching AP World History: Modern. Please answer the following questions and provide as much detail as possible. Phase 1: Know and Understand What is the most effective aspect of Phase 1? What is the least effective aspect of Phase 1? Do you have any suggestions to improve the effectiveness of Phase 1? Phase 2: Apply and Analyze What is the most effective aspect of Phase 2? What is the least effective aspect of Phase 2? Do you have any suggestions to improve the effectiveness of Phase 2? Phase 3: Evaluate and Synthesize What is the most effective aspect of Phase 3? What is the least effective aspect of Phase 3? Do you have any suggestions to improve the effectiveness of Phase 3? Phases of Learning 54 Phases of Learning Overall What do you think about the overall effectiveness of the Phases of Learning framework? Do you have any suggestions to improve the Phases of Learning framework? What are the benefits of this curricular framework? What are the weaknesses of this curricular framework? Would you consider adopting this curricular framework in your own teaching? Why or Why not? Phases of Learning 55 Appendix C T. Bullock - Phases of Learning - FINAL Final Audit Report 2023-12-01 Created: 2023-11-29 By: Ellynn Raynor (ellynnraynor@weber.edu) Status: Signed Transaction ID: CBJCHBCAABAAQ5lF3ok7-bQoI0uIAPghwbrLCX8MlLKm "T. Bullock - Phases of Learning - FINAL" History Document created by Ellynn Raynor (ellynnraynor@weber.edu) 2023-11-29 - 11:34:29 PM GMT Document emailed to Penee Stewart (pstewart@weber.edu) for signature 2023-11-29 - 11:35:34 PM GMT Email viewed by Penee Stewart (pstewart@weber.edu) 2023-11-30 - 2:01:29 AM GMT Document e-signed by Penee Stewart (pstewart@weber.edu) Signature Date: 2023-11-30 - 4:15:16 AM GMT - Time Source: server Document emailed to Clay Rasmussen (clayrasmussen1@weber.edu) for signature 2023-11-30 - 4:15:19 AM GMT Email viewed by Clay Rasmussen (clayrasmussen1@weber.edu) 2023-11-30 - 2:57:44 PM GMT Document e-signed by Clay Rasmussen (clayrasmussen1@weber.edu) Signature Date: 2023-11-30 - 2:58:45 PM GMT - Time Source: server Document emailed to Stephanie Speicher (stephaniespeicher@weber.edu) for signature 2023-11-30 - 2:58:48 PM GMT Email viewed by Stephanie Speicher (stephaniespeicher@weber.edu) 2023-12-01 - 1:27:32 PM GMT Document e-signed by Stephanie Speicher (stephaniespeicher@weber.edu) Signature Date: 2023-12-01 - 4:56:50 PM GMT - Time Source: server Agreement completed. 2023-12-01 - 4:56:50 PM GMT