Summer Slide and Its Lessons for COVID-19 Related Learning Loss

Tara O'Neill Hayes, Madeline VanHorn

Executive Summary

In an effort to understand the learning losses that occurred during the COVID-19 pandemic, some have looked to research on learning losses during the summer, but the research provides no consistent conclusion regarding the scope or magnitude of the problem.
The research on summer learning losses does offer, however, a few common points of agreement that are relevant to conversations about learning losses during the pandemic: Students learn most during the school year, achievement gaps are well established during early childhood, and children learn at a greater rate in younger years.
Since the research suggests younger children and those already behind likely suffered the most during the pandemic, policymakers and educators should focus their efforts to provide enhanced support to these children in particular.

Introduction

Students around the world had their education thrown into chaos for the past year and a half as the COVID-19 pandemic forced schools to close, at least temporarily, and then to operate remotely for much of the past year. These closures and virtual learning experiences are likely to have had significant consequences on students’ learning, and parents, educators, and policymakers are rightly concerned. Many are trying to anticipate the effects and plan mitigation efforts, with the “summer slide” providing a seemingly useful analogy.

“Summer slide” refers to the learning loss that elementary, middle, and high school students may experience during the summer months when school is not in session. The concept has been studied for decades, but there is little consensus regarding the scope or magnitude of the problem, the students most at risk, whether losses are more likely to occur or be worse in early grades or later grades, the reason for any losses, or whether losses even really occur. Some researchers prefer the terms “summer slowdown” or “summer stagnation,” as their research has shown less of a loss and more of a deceleration in learning. While most studies tend to find some degree of loss, typically more so in math than in reading and more often among students of low socioeconomic status, enough flaws in the research have been identified and opposing results found that no real conclusion is easily drawn.

In short, the data around the summer slide are quite contradictory and inconclusive. Further, the COVID-19 pandemic and the resulting closures and adapted learning environments persisted for so long that, even if the data on summer learning loss were consistent, that data would still provide only limited insight into the potential effects of the COVID-19 pandemic. That said, policymakers can glean some limited conclusions about whom the school closures might have affected the most.

Assessing the Evidence on Summer Learning Loss

Early Research: A Large Impact

A 1996 meta-analysis of early research found that student achievement declined each summer, on average, by one month’s worth of school year instruction. Declines were greater in math than in reading, although income-related reading gaps grew during the summer, and the losses were greater at higher grade levels.[i] Researchers suggested the more significant declines in math were due to fewer opportunities to practice such skills as well as the fact that math, as opposed to reading, requires fact- and procedure-based knowledge, which is more easily forgotten than conceptual knowledge.[ii] This early research, however, has been criticized for overstating the impact of seasonal loss due to measurement issues.

More Recent Research: Mixed Results

Some more recent studies continued to find significant evidence that summer slides exist. When put under the microscope, however, these claims similarly fell apart.

A large, often-cited study from 2007 analyzing testing data for Baltimore students from 1^st through 9^th grade in the 1980s found that two-thirds of the reading comprehension achievement gap between high and low socioeconomic status (SES) students by the 9^th grade was explained by their access to summer learning resources during their elementary education; the other third of the gap was present at the time of the initial test in first grade.[iii] Both of these factors support the notion that most of the achievement gap is attributable to differences in available resources outside the formal school setting. Yet recent analysis in which researchers attempted to replicate these results using nationally representative data from the 2000s shows that differences in testing and scoring methodology between the assessments previously used and those now in use erase all of the learning gaps originally identified.[iv]

In 2016 a study of over half a million 2^nd through 9^th graders found that students, on average, lost 25 – 30 percent of the learning they gained during the school year, and Black and Latino students typically gained less during the school year and lost more during the summer than White students.[v]

The same authors published new research in 2020 using data for 18 million students across all 50 states from 2008 through 2016, making it one of the largest studies on the subject to date.[vi] School year learning gains, on average, were found to be largest in the early years, with roughly equal gains across reading and math. During the summer, most students lost some of their school year gains, with greater losses for math than reading; the losses diminished over time, but to a lesser degree than the school-year gains. They found wide variance in summer learning outcomes, however, with students at the top of the range experiencing an acceleration of learning and students at the bottom of the range losing nearly all their school year gains. The authors were unable to articulate a reason for the variation in gains and losses, simply noting other research finding race/ethnicity and SES account for only 4 percent of that variance.[vii]

A study reviewing data for 3.4 million Kindergarten through 8^thgrade students who took the Northwest Evaluation Association Measures of Academic Progress (MAP) reading and math assessments between 2016 and 2018 found more than three-fourths of students experienced some learning loss over summer vacation in either subject, with average median losses of roughly two months, which increased in higher grade levels.[viii] Yet there were significant variations in the magnitude of losses; the strongest predictor of whether a student would experience learning loss over the summer was how much they gained during the school year, with the biggest losses occurring to students with the greatest gains; these findings contradict the 2016 findings that found Black and Latino students gained less and lost more as well as the 2020 study showing that those who gained the most during the school year also made gains during the summer.[ix] Neighborhood poverty status was only moderately correlated with differences.[x]

Re-evaluation: Evidence of a Minimal Impact

Researcher Paul von Hippel, who contributed to early literature on the subject finding significant losses, has more recently become doubtful of the role summer learning loss plays after unsuccessfully replicating his original study, and suggests the idea of a summer slide is vastly overstated. The findings of his 2004 study indicated that nearly every learning gap between student populations grew fastest in the summer months and increased with age.[xi] In new research, using the same nationally representative Early Childhood Longitudinal Surveys (ECLS) as well as a more recent cohort along with an improved method for scaling test scores from one year to the next, the learning gap between high and low SES students was found to already be substantial at the start of Kindergarten and to shrink in the subsequent years of schooling.[xii] Only during the first summer vacation did test score variance rise, leading the authors to conclude the effect of summer is overstated and to adopt the terms “summer slowdown” or “summer stagnation.”[xiii] Von Hippel continues to find schools to be the “great equalizers” because of how achievement gaps shrink during the school year.[xiv]

A 2016 analysis of the most recent ECLS similarly found minimal evidence of learning loss over the summers after first grade. In fact, only high-SES students were found to experience a summer learning loss, which was detected in reading in the summer after first grade.[xv]

A Measurement Issue

Given the vast array of findings and inconsistencies, researchers have worked to find an explanation for these differences. It has since been shown that early studies used exams that were not vertically scaled, making them poor candidates for comparing results over time.[xvi] Further, these “fixed-form” tests were highly prone to distortions: Simple changes to the questions or asking more questions led to significant fluctuations in the alleged achievement gap.[xvii] Newer tests, including the two most commonly used tests today—the MAP and the ECLS—use adaptive questions that increase or decrease in difficulty depending on whether the student is answering correctly. Both of these tests show that the rate of learning decelerates as children get older and that most of the achievement gap is well established by Kindergarten.[xviii] Every study also shows that nearly all students learn at a faster pace during the school year than the summer.[xix]

Still, the magnitude of any summer slide varies significantly between the MAP and ECLS. The ECLS shows minimal summer learning loss of just two weeks, on average, in the first summer, and during the second summer, reading loss remains consistent with the first summer, but in math, students actually made gains.[xx] The MAP test, on the other hand, shows losses of a full month in the first summer for both reading and math, and losses of three full months in the second summer for both.[xxi]

Other challenges relate to the timing of the tests. Research suggests that students are less engaged with testing in the fall semester than the spring semester, which may hinder their performance.[xxii] It is also important to consider that some schools administer tests further into the school year when students have already relearned material, whereas others are tested earlier. Finally, tests given in the fall have been found to be more difficult than those administered in the spring.[xxiii] This discrepancy may be because in the fall teachers want to clearly understand where their students are in order to know where to begin their teaching, whereas spring tests are used for accountability and performance checks, often tied to financial rewards or penalties.

What to Plan for: COVID-19 Impact

Amid the pandemic, most schools in the United States and across the globe first closed and then turned to online learning, for which many school systems and educators were ill prepared.[xxiv] In total these changes affected 55.1 million U.S. students, raising concerns about the severity of learning loss that may have occurred with virtual learning for such an extended period of time.[xxv] Of even greater concern is the impact on students who faced additional challenges: lacking a high-speed internet connection or the necessary equipment, not having parents at home to help them learn and keep focused, or any number of other challenges that may have made learning in such an environment more difficult.

Researchers have looked at COVID-19 learning loss in the same way as summer learning loss: the COVID slide and the COVID slowdown.[xxvi] While race and socioeconomic status may not be as strong an indicator for seasonal learning loss as initially believed, they may play a significant role in COVID learning loss, as low SES families were particularly hard hit by the economic impact of the virus and had fewer resources available to make learning at home possible; as a result, these students likely experienced more instability.

Even as schools began to reopen in the spring, reasons for concern regarding racial divides in learning loss persisted. While 80 percent of public schools were open with some capacity for in-person learning as of February 2021, a survey of 8^th grade students found that 78 percent of Asian American students and 80 percent of Black and Hispanic students were still in full-time remote learning.[xxvii] The National Parents Advocacy Union reports that mistrust of the educational system is one of the primary reasons families of color chose to opt out of in-person instruction.[xxviii]

Consistent Findings

The COVID-19 pandemic caused severe disruptions in nearly all children’s learning environments for a year or more. While past research regarding summer learning loss has produced inconsistent results, making it difficult to use as a predictor for potential fallout from the pandemic, there are a few common findings worth keeping in mind. Studies consistently show that students learn most during the school year, that achievement gaps are well established during early childhood, and that children learn at a greater rate in younger years. Thus, educators should expect that little learning occurred while schools were closed, the youngest children likely suffered the most, and those who were already behind are probably still behind, perhaps more so. Efforts to combat the losses that most likely occurred should therefore target these children in particular.

[i] https://journals.sagepub.com/doi/10.3102/00346543066003227

[ii] https://journals.sagepub.com/doi/10.3102/00346543066003227

[iii] https://www.researchgate.net/publication/242779815_Lasting_Consequences_of_the_Summer_Learning_Gap

[iv] https://www.educationnext.org/is-summer-learning-loss-real-how-i-lost-faith-education-research-results/

[v] https://www.educationnext.org/summer-learning-loss-what-is-it-what-can-we-do-about-it/#_edn3

[vi] https://www.edworkingpapers.com/ai19-82

[vii] https://journals.sagepub.com/doi/pdf/10.1177/0038040718801760

[viii] https://kappanonline.org/rethinking-summer-slide-the-more-you-gain-the-more-you-lose/

[ix] https://kappanonline.org/rethinking-summer-slide-the-more-you-gain-the-more-you-lose/

[x] https://kappanonline.org/rethinking-summer-slide-the-more-you-gain-the-more-you-lose/

[xi] https://journals.sagepub.com/doi/10.1177/000312240406900501

[xii] https://journals.sagepub.com/doi/pdf/10.1177/0038040718801760