Do SAT-Mandatory States Explain Declining Scores?

Yesterday, I wrote about how it was likely the case that some of the decline in SAT scores  was due to states and districts requiring students to take the SAT. At the request of several esteemed readers, I did a back-of-the-envelope calculation to see how much of the change in SAT scores over the last five years is due to states requiring all students to take the SAT (hat tip to Kan-Ye Test (love the name!) for pointing me to the data). Between 2011 and 2015, Delaware, the District of Columbia, and Idaho moved from having some of their students take the SAT (14,765) to having all of their students (32,236) take the SAT. Meanwhile, the average SAT score fell from 1500 to 1490.

Based on 2011 state-by-state data, I recalculated average 2015 SAT scores while substituting 2011 participation levels and scores for 2015  levels and scores in those three states. Erasing the additional 17,471 test-takers (and their average SAT of 1292) from those three states was enough to raise the average SAT score of 1.6 million other test-takers by 2.1 points. These three states explain approximately 21% of the decline in SAT scores, as outlined below.

Required SAT states explain at least 21% of the decline in SAT scores since 2011
States Num. students Avg. SAT
DC, DE, & ID (2011) 14,765 1445
DC, DE, & ID (2015) 32,236 1362
All others (2015) 1,614,887 1493
Total (using ’11 DC, DE, & ID) 1,629,652 1492
Total (using ’15 DC, DE, & ID) 1,647,123 1490

I’d still love to see the College Board pull out data from the districts which moved to require the SAT, as it’s entirely possible that half of the decline in SAT scores could just be due to students who were required to take the test. They’ve got the data, and I hope they take a look!

Why SAT Scores Going Down May Be Just Fine

The average score for students taking the venerable SAT exam in 2014-2015 was 1490, seven points below last year’s scores and the lowest score since the writing section was added in 2005. Not surprisingly, this drop is generating a lot of media coverage—much of it focused on how high schools are failing America’s children. But while high schools may very well be a concern (and those of us in colleges shouldn’t get off without criticism, either), I contend that the decline in SAT scores may be just fine.

The simple reason for my lack of concern is that the decline may very well be due to more students taking the exams in response to new state laws and district rules in several states requiring or encouraging testing. For example, Idaho required beginning in 2012 that students had to take the ACT or SAT to graduate—and that the state would cover SAT costs for students. In 2011-2012, 27% of Idaho students took the SAT and got an average score of 1613, while practically all Idaho students in 2014-2015 took the SAT and got an average score of 1372. (The District of Columbia, Delaware, and Maine—the other three jurisdictions where basically everyone takes the SAT—had similarly low scores.) Either Idaho high schools imploded over a three-year window, or the types of students who weren’t previously taking the test didn’t have the same level of ability on standardized tests as the 27% of students who were likely considering selective four-year colleges.

The chart below shows the relationship between the percentage of students taking the SAT and scores (data available via the Washington Post). The R-squared is 0.82, suggesting that 82% of the variation in state-level test scores can be explained by the percentage of students tested in each state.

sat_2015

What I would like to see is some comparisons across similar types of students over time. Among students who signal a clear intent to go to a four-year college, are SAT scores declining? Or is the entire decline driven by different students taking the test? And are students considering college for the first time because they took the SAT and did reasonably well? There is value to everyone taking a standardized test across states (given the differences in state high school exams), but it’s inappropriate to look at trends over time with such large differences in the types of students taking the test.