Research Foundation Paper
Rafe Mazzeo, Ph.D
Stanford University’s Education Program for Gifted Youth (EPGY) is a suite of online multimedia courses based on over 40 years of pedagogical and cognitive science research carried out at the university. The courses include Mathematics, Language Arts, Physics, and Computer Science for students in K-12. While originally designed to accelerate the learning of gifted students, multiple research studies have proven the efficacy of these courses for learners of all abilities. EPGY endeavors to
- Provide students, regardless of where they live, with high-quality courses developed from research-based pedagogical methods.
- Formulate instructional material that integrates easily into regular school environments.
- Personalize instruction and accommodate individual differences in student learning, providing materials suitably tailored for each student.
- Allow students to progress at their own paces and to accelerate and enrich their education.
Recognizing that EPGY now stands at a critical crossroads of potential outreach and opportunity for technological enhancement, Stanford University has identified Redbird Advanced Learning to develop and introduce the next generation of EPGY courses. Building upon our distinguished legacy, this new development will be guided by continuing Stanford research. Redbird Advanced Learning, a Silicon Valley-based enterprise, is part of Rocket Group, a company with a rich history in diverse educational offerings aiming to improve the lives of students through technology, research, and innovation.
This paper summarizes the proven foundational components of EPGY and describes the enhanced learning technologies that Redbird Mathematics—the next iteration of EPGY K7 Math—will debut for the 2014-15 school year.
EPGY has evolved from a solid foundation of over 40 years of active research in computer-based education at Stanford University. EPGY courses are research-based, validated by experiment, and honored with numerous awards and widely recognized for groundbreaking efforts in e-Learning. Key features include innovative curriculum and educational technology, personalized instructional and technical support, and validated achievement results.
Superior Curriculum and Technology
EPGY courses contain world-class curriculum and instructional technology developed at Stanford University. A combination of multimedia instruction and automated assessment of student work provides a highly individualized, self-paced educational experience. Content is continually tailored to address each student’s ongoing performance, thereby ensuring that students across a wide range of proficiency levels remain fully engaged during the learning process. Students who demonstrate mastery of math content move quickly to the next concept, while those who require additional instructional attention receive material designed to address their needs. The resulting personalization accelerates learning and enhances student achievement.
Research-Based Approach, Validated by Results
EPGY course algorithms—the code that defines how the program responds to student input—are based on a long history of research in computer-based education and learning theory that began in the 1960s at Stanford University. The learning model delivered by EPGY consists of definitions for each student of which topics to work on next, what constitutes mastery, when to assign different types of instruction, and how frequently to review previously learned material (Suppes et al, 2014).
As EPGY expanded into schools in the 1990s, studies with large numbers of students established and confirmed EPGY’s efficacy with students of all abilities. This research also identified implementation models that could be used by teachers to aim for greater student gains (Suppes et al 2012, 2013, 2014). Additionally, student performance data was regularly examined and used to adjust the learning model and to modify course content.
A pivotal project completed in 2008 studied over 2,000 Title I students who were using EPGY math. This study showed that students who did more careful and sustained work in EPGY math during the school year, as measured by total exercises answered correctly on the first attempt (CFA), performed better on end-of-year state tests than students in a matched peer control group who did not use EPGY math. Lower-performing students gained more ground by participating in EPGY regardless of the amount of use, while average students who completed over 2,000 CFA during the year achieved significant gains compared to their peers (Suppes et al, 2013). Subsequent studies with larger groups of students consistently proved that careful and sustained work in EPGY correlated with higher achievement gains (Suppes et al 2014). Additionally, predictive models were developed to anticipate student achievement results, providing teachers with the opportunity to perform early interventions.
A recent examination of EPGY’s efficacy studies performed by NYU’s Metropolitan Center for Research on Equity and Transformation of Schools concluded that EPGY has a positive effect on achievement for K-8 students in both mathematics and English language arts and furthermore, schools can use EPGY data to help identify struggling learners who may need additional interventions. (Ahrams & Noguera, 2014)
Recognizing the Need for Continual Program Development
Stanford University and Redbird Advanced Learning recognize that students, technology, and educational goals continue to evolve; therefore EPGY must grow and adapt to the changing landscape.
Educators Are Beginning to Implement Common Core State Standards (CCSS). CCSS provides an exceptional opportunity to deploy the breadth and depth of the current EPGY offering and to frame areas for continuing development.
Today’s Students Are Digital Natives. EPGY has always been committed to providing educational experiences of the highest integrity to hold student interest. Continued effort is required to develop innovative student material that is deeply engaging, highly personalized, aligned to current technological innovations, and sensitive to preferences in learning modality.
The Classroom Environment is Evolving. The EPGY blended-learning environment already excels by integrating teaching and technology. Ongoing development is required to keep up with the pace of emerging demands of classroom instruction.
Redbird Mathematics: Building on the Distinguished Legacy of EPGY
Stanford University and Redbird Advanced Learning are expanding EPGY into Redbird Mathematics. Key features include
- Adaptive instruction and learning experiences to meet the specific needs of each student,
- An engaging and effective variety of instructional approaches to develop student thinking,
- Continual and immediate feedback to motivate and engage students,
- Focus on helping students develop a rigorous understanding of mathematical domains that build the foundations for success in Algebra 1, and
- Self-paced progress through content gives students control over their own learning.
Adaptive instruction and learning experiences meet the specific needs of each student.
Individual performance directs the personalization of instruction, allowing each student to experience learning that meets his or her specific needs. Working on suitably tailored content allows students to be challenged and motivated; working on material at the wrong level leads to disengagement.
Research and Best Practice
An analysis of gifted students’ individual difference while using mathematics courses offered by Stanford University focused on modeling student performance over a long period of time. The study found that performance varies widely with the group for a variety of different measures, including error rates, times to completion, progress rates, and latency of response (Cope & Suppes, 2002).
“Using several large student samples, the four-year statistical assessment of state test performance is the focus of this report” … the main conclusion is students can make significant learning gains … “when taught on an individualized basis, at their current level of competence”. (Suppes et al, 2014)
Gathering data on student progress is important for revealing growth and the need to modify instruction or other teacher actions. (Stecker, Fuchs, & Fuchs, 2005)
Teachers’ regular use of formative assessment improves their students’ learning, especially if teachers have additional guidance on using the assessment to design and to individualize instruction. (National Mathematics Advisory Panel, 2008, p.xxiii)
Simply accelerating students into the next grade-level course designed for all math students is not appropriate for helping advanced students develop their mathematical talents. Gifted students need math that is both accelerated and enriched. (Saul, Assouline, & Sheffield, 2010)
Building on the foundation and proven effectiveness of EPGY, Redbird Mathematics provides each student with the content and pacing he or she needs for personalization, motivation, engagement, and success. Progression through the content is determined by continually updated information about student actions in the program. Redbird Mathematics leads the next generation of data utilization by providing performance-based instructional recommendations, suggesting high-value actions teachers can take with individual or small groups of students to meet their needs and supplement the software.
Redbird Mathematics reports on frequency of Correct First Attempt (CFA), data that is associated with student precision and careful engagement. Teachers who encourage their students to increase their CFAs see greater student growth in math performance.
Redbird Mathematics continues and extends EPGY’s tradition of using adaptive learning to individualize students’ experiences. Students who need to improve their performance may get further modes of instruction, extra practice, expanded help and hints, and even additional lessons on math concepts earlier in learning progressions to ensure that they develop strong foundations for future work. Other students may progress more quickly through lessons, accelerating through content they have mastered, to spend more time on mathematics that challenges them. In addition, these high-performing students may encounter material and experiences that are qualitatively different than regular on-grade study.
Like EPGY, Redbird Mathematics gives teachers access to the accumulated data that is gathered on their students. Teachers are offered suggestions about which students have similar instructional needs and what material would be best addressed by teachers with individuals, small groups, or even the whole class.
An engaging and effective variety of instructional approaches to develop student thinking.
Each student responds uniquely to different instructional approaches, therefore it is essential to provide multiple representations to deepen and enrich mathematical understandings. Systematic, explicit, and visual mathematical experiences have proven positive learning effects.
Research and Best Practice
“The motion engine also weaves together the various strands that make up each course, at each step drawing the next presented concept from which every strand will keep the student advancing through the grade while keeping progress aligned across the strands. The complex algorithm driving this adaptation of the movement through the course ensures that students are focusing their time in the course on concepts that challenge them while matching their mastery level, so the students stay engaged and rewarded in the learning process.” (Suppes et al, 2012)
“The exercises are designed to give students an opportunity to practice what they have learned through activities that range from answering simple knowledge-based questions to conducting virtual experiments, and responding to open-ended questions. …tutorial help designed to mimic the behavior and support an expert tutor might give—guiding the student towards the correct answer without directly stating it—and providing additional exercises.” (Ahram & Noguera, 2014)
Performance is improved when instruction accommodates the differences in learning needs. (Sousa & Tomlinson, 2011)
A meta-analysis of fifty studies shows that systematic and explicit instruction had a strong positive effect for both special-education and low-achieving students. Explicit and systematic instruction typically entails explanations and demonstrations of specific strategies coupled with opportunities for students to solve carefully sequenced problems and get feedback. (National Mathematics Advisory Panel, 2008).
“…studies that included visual representations along with other components of explicit instruction tended to produce significant positive results” (National Mathematics Advisory Panel, 2008).
Over its years in the vanguard of online learning, EPGY introduced an innovative range of instructional approaches allowing students to interact with mathematics online in novel ways. With the impending advent of Partnership for Assessment and Readiness for College and Careers (PARCC) and Smarter Balanced (SBAC) assessments, new interfaces and instructional models are being introduced in Redbird Mathematics, taking advantage of new technological advances and constantly evolving classroom needs.
Numerous effective instructional approaches were introduced in EPGY. Students experienced
- Direct instruction lectures,
- Interactive lectures,
- Inquiry-based learning in which they performed a series of actions and drew conclusions based on the results,
- Experience-based learning where they solved problems and learned from feedback regarding what they were successful and unsuccessful with, and
- Games to provide engaging practice and develop fluency.
Redbird Mathematics expands these approaches to address new demands of high-stakes tests such as PARCC and SBAC. New developments in technology allows for greater interactivity in problems and lectures, including widgets and learning items with associated sets of instruction, helps, hints, and feedback that provide a network of instruction. Redbird Mathematics extends the game offerings to provide lessons that support conceptual development, fluency with procedures, and development of coding and logic skills.
Continual and immediate feedback motivates and engages students.
Assessment need not be separate from mathematical work. Formative assessment can be gathered during students’ work as evidence of progress and to inform instructional opportunities for students.
Research and Best Practice
“Given the well-recognized difficulty of developing instructional regimes that measurably improve the performance of Title I students, EPGY Math and Language Arts courses have tried to do this by using important technological features: (i) progress is individualized for each student according to a stochastic motion driven by a learning model embedded in a computer program; (ii) responses to exercises are continually made by students seeing supporting visual displays and hearing associated audio lectures, ordinarily no longer than 90 s; (iii)hints, tailored to a student’s wrong answers, are given by a computer program; (iv) immediate reinforcement determined by computer evaluation of individual responses is given at the end of each exercise.” (Suppes et al, 2014)
“The software has a diagnostic component that assesses the student work and provides immediate feedback…” (Ahram & Noguera, 2014)
A research-proven algorithm evaluates each student’s progress towards mastery by measuring the pattern and increasing frequency of Correct First Attempts. This allows the program to position each student in his or her zone of proximal development where the student is challenged to tackle new material, rather than showcase what he or she already knows. In Redbird Mathematics this algorithm is applied to instructional clusters and material in the program is expanded or condensed based on its determination of student performance and instructional needs.
A focus on helping students develop a rigorous understanding of mathematical domains that builds the foundations for success in Algebra 1.
To be mathematically proficient and prepared for further STEM studies, students need a solid understanding of foundational domains. The National Math Panel and the Common Core State Standards (CCSS) define these essential domains for Mathematics at the elementary and middle grades as focusing on whole numbers, fractions and particular aspects geometry and measurement. A rigorous foundation in these domains includes conceptual understanding, procedural fluency, and applications to problems.
Once students master these essential domains, the mathematical experiences may broaden to other important topics, including geometry and logic.
Research and Best Practice
“A major goal for K-8 mathematics education should be proficiency with fractions (including decimals, percent, and negative fractions), for such proficiency is foundational for algebra and, at the present time, seems to be severely underdeveloped. Proficiency with whole numbers is a necessary precursor for the study of fractions, as are aspects of measurement and geometry. These three areas—whole numbers, fractions, and particular aspects of geometry and measurement—are the Critical Foundations of Algebra.” (National Mathematics Advisory Panel, 2008)
“…an important subset of the major work in grades K-8 is the progression that leads towards middle-school algebra.” (National Governors Association, CCSSO, Achieve, Council of the Great City Schools, NASBE, 2012)
An experiment was undertaken with academically talented fifth- and sixth-grade students who were taught logic as part of their curriculum and then tested compared to college students. Results showed that “…the upper quartile of elementary school students can achieve a significant conceptual and technical mastery of elementary mathematical logic. The level of mastery is 85 to 90 percent of that achieved by comparable university students.” (Suppes & Binford, 1965)
“What is striking about all of our research findings is the strong predictive power of spatial skills on students’ math skills. The present results support the hypothesis that spatial skills are an important underpinning of students’ math abilities.” (Casey, 2013)
Redbird Mathematics continues the EPGY tradition of rigorous instruction on important mathematics topics. Special attention to the rigor described by the Common Core State Standards for Mathematics and the inclusion of enriching learning experiences ensure students are prepared and appropriately challenged. Each student receives a personalized experience to deliver the mix of topics that is appropriate for him or her.
Leveled games provide logic and geometry puzzles and practice to engage, challenge, and enrich the mathematical experiences of all students. This expands students’ mathematical experiences beyond the focus of the Common Core in a format that appeals to students and encourages them to invest additional time in mathematical pursuits.
Self-paced progress through content gives students control over their own learning.
Students who progress at their own pace experience increased motivation and engagement. This supports the goal of increasing student independent learning skills.
Research and Best Practice
The EPGY structure allows for an individualized and self-paced curriculum that rises to meet each student’s needs as he or she uses the computer program; students who master a particular concept quickly are able to move on to different and more advanced concepts, while struggling students are provided the additional time and support they need to master the same concept. (Ahram & Noguera, 2014)
Studies using students’ EPGY exercise-response data relative to their performance on standardized mathematics and English language-arts exams demonstrate that EPGY is an effective supplemental learning tool for boosting student achievement. (Ahram & Noguera, 2014)
Accelerated secondary and college students used a computer-assisted elementary logic course developed at Stanford University. The course allowed students to explore different paths to a solution, with varying degrees of exploration controlled by their teacher. The instructional system was designed to increase the level of active student participation and researchers found that students reacted positively and used many features of the program. (Goldberg & Suppes, 1972)
Technology that gives students control over the pace of their learning increases students’ sense of independence, motivation, and engagement. (Anderson-Inman & Horney, 2007)
Students who had control over their own pace performed better at problem-solving transfer tasks than students who could not control their pace. (Mayer & Moreno, 2003)
Redbird Mathematics embraces the potential of self-paced learning that was uncovered by EPGY. Not only can students accelerate through content, but they also have choices about movement through content clusters based on their readiness and understanding of precursor material. Students may also move through multi-level games based on their performance, providing an optimized level of challenge and engagement that ensures students invest the time they need to be successful.
Ongoing Stanford research will continue to drive the features of Redbird Mathematics, and efficacy studies performed on Redbird Mathematics will inform research. This interdependence of foundational research and efficacy studies ensure that Redbird Mathematics will continually evolve to address the instructional needs and goals of students, educators, and parents by utilizing research, best practices, and the newest advances in technology and personalized learning.
Appendix | References
Ahram, R. & Noguera, P. (2014) Metropolitan Center for Research on Equity and the Transformation of Schools. Using the Education Program for Gifted Youth (EPGY) in Schools to Support and Advance Student Achievement. New York: NYU.
Anderson-Inman, L. & Horney, M. (2007). Supported eText: Assistive technology through text transformations. Reading Research Quarterly, 42(1), 153-160.
Casey, Beth. (2013). Scaffolding Young Math Learners to Be Effective Spatial Problem Solvers. Boston: Boston College. Retrieved from here
Cope, E. W., & Suppes, P. (2002b). Gifted students' individual differences in distance-learning computer-based calculus and linear algebra. Instructional Science, 30(2), 79-110.
Goldberg, A., Suppes, P. (1972). A computer-assisted instruction program for exercises on finding axioms. Educational Studies in Mathematics 4 (1972) 429-449.
Mayer, R.E. & Moreno, R. (2003), Nine ways to reduce cognitive load in multimedia learning. Educational Psychologist, 38(1), 43-52.
National Council of Teachers of Mathematics. (2007). Principles and standards of school mathematics. Reston, VA: Author.
National Governors Association, CCSSO, Achieve, Council of the Great City Schools, NASBE. (2012) K-8 Publishers’ Criteria for the CommonCore State Standards for Mathematics. Retrieved from here
National Mathematics Advisory Panel. (2008) Foundations for success: The final report of the National Mathematics Advisory Panel. Washington, DC: U.S. Department of Education. Retrieved from here
Saul, M., Assouline, S., & Sheffield, L. (2010) The Peak in the Middle: Developing Mathematically Gifted Students in the Middle Grades, Reston, VA: National Council of Teachers of Mathematics.
Sousa, D.A. & Tomlinson, C.A. (2011). Differentiation and the brain: How neuroscience supports the learner-friendly classroom. Bloomington, IN: Solution Tree Press.
Stecker, P., Fuchs, L., & Fuchs, D. (2005). Using curriculum-based measurement to improve student achievement: Review of research. Psychology in the Schools, 42(8), 795-819.
Suppes, Flickinger, Macken, Cook, Liang (2012). Description of the EPGY Stanford University Online Courses for Mathematics and Language Arts, International Society for Technology in Education Annual (ISTE) 2012 Conference
Suppes, P., & Zanotti, M. (1996). Mastery learning in elementary mathematics: theory and data. In P. Suppes, & M. Zanotti (Eds.), Foundations of probability with applications (pp. 149–188). New York: Cambridge University Press.
Suppes, P., Binford, F. (1965). Experimental teaching of mathematical logic in elementary schools. The Arithmetic Teacher, 12, 1965, 187-195
Suppes, P., Holland, P. W., Hu, Y., & Vu, M.-T. (2013). Effectiveness of an individualized computer-driven online math K-5 course in eight California title I elementary schools. Educational Assessment, 18(3), 162–181.
Suppes, P., Liang, T., Macken, E. E., & Flickinger, D. P. (2014). Positive technological and negative pre-test-score effects in a four-year assessment of low socioeconomic status K-8 student learning in computer-based math and language arts courses. Computers & Education, 71, 23-32.