(Still) Don’t Dismiss Performance Funding Research

I like the idea of funding public colleges and universities based in part on their former students’ outcomes—and I’m far from the only one. Something in the ballpark of three dozen states have adopted some sort of a performance-based funding (PBF) system, with more states currently discussing the program. Given that many states currently fund colleges based on a combination of enrollment levels and historical allocations that can be woefully out-of-date, tying some funding to outcomes has an intuitive appeal.

However, as a researcher of accountability policies in higher education, I am concerned that some colleges may be responding to PBF in unintended ways. At this point, as I briefly summarized in a recent piece at The Conversation, there is evidence that PBF may adversely affect access to college for moderately prepared students as well as the types of postsecondary credentials awarded. My newest contribution was a recently published article in the Journal of Education Finance that found both two-year and four-year colleges subject to PBF saw less Pell revenue than other colleges not subject to PBF.

Since that article finished the peer review and copy editing processes and was posted online two weeks ago, I’ve been expecting a response from one of the largest organizations advocating for PBF. HCM Strategists, a DC-based advocacy group that is quite effective in lobbying and policy development, has traditionally been a strong supporter of PBF. (Disclaimer: I’ve gotten funding from them for a project on a different topic in the past.) In 2013, an HCM director responded to a high-quality paper by David Tandberg and Nick Hillman (that was later published in JEF) by writing an Inside Higher Ed piece called “Don’t Dismiss Performance Funding.” In this piece, they call the research “flawed” and “simplistic,” neither of which are particularly true. I wrote a blog post called “Don’t Dismiss Performance Funding Research,” in which I wasn’t too pleased with their response.

Today, HCM director Martha Snyder has a much more nuanced IHE essay on my and Luke’s work entitled “Jumping to Conclusions,” saying that our work should not be used “to draw any meaningful conclusions” on PBF. Snyder discusses what she perceives as some of the limitations of our work. The most notable one is that multiple types of PBF policies are lumped together in the analyses. That is necessary due to data limitations—there is no comprehensive archive of the nuances of PBF plans prior to the early 2010s. However, general trends in PBF policies across states are partially captured by the year fixed effects in the regression (standard practice in panel analyses), which also help to account for these factors.

Snyder also suggests that some states have been encouraging students to enroll in community colleges, which is definitely the case (although somewhat less so prior to 2012-13, the last year of our analysis due to the pace at which new data become available). If this were true, it would explain decreases in per-FTE Pell revenue at four-year colleges, but also increase Pell revenues at two-year colleges. Instead, we saw nearly identical negative point estimates, which raise further cause for concern. (Could this affect for-profit enrollment? I can’t really tell with federal data, but a state-level analysis here would be great.)

I appreciate HCM’s work in helping states implement more modern funding programs, but it is imperative that influential policy organizations work with the research community before drawing any meaningful conclusions about the potential unintended consequences of PBF—especially as the stakes become higher for students and colleges alike. The small, but growing, body of literature on colleges’ responses to PBF suggests that collaboration among interested parties would be far more productive than attempting to dismiss findings from peer-reviewed research that suggest caution may be in order. I’m happy to do what I can to summarize the literature on unintended consequences while working to move forward policy discussions on future versions of PBF.

Will Colleges Send Out Financial Aid Packages Earlier Next Year?

I’m looking forward to college students being able to submit the Free Application for Federal Student Aid (FAFSA) three months earlier next year. Instead of being able to submit starting January 1 for the 2017-18 academic year, students will be able to submit beginning October 1—giving students an additional three months to complete the form thanks to using ‘prior prior year’ (PPY) income and asset data. This means that students can get an estimate of their eligibility for federal grants and loans as soon as late fall, which has the potential to help inform the college choice process.

But there is no guarantee that students will get their final financial aid package from the college any earlier as a result of prior prior year. Recognizing this, Undersecretary of Education Ted Mitchell recently sent a letter to college presidents asking colleges to send out their aid packages earlier in order for students to fully benefit from PPY. Will colleges follow suit? I expect that some will, but the colleges with the greatest ability to offer institutional grant aid probably won’t. Below, I explain why.

The types of colleges that can easily respond to PPY by getting aid packages out earlier are those institutions with rolling admissions deadlines. (Essentially, it’s first-come, first-served among students who meet whatever admissions criteria are present—less-selective four-year and virtually all two-year colleges operate in this manner.) Some of these colleges already offer their own grant aid upon admission, but these colleges tend to have less grant aid to offer on account of relatively low sticker prices and fewer institutional resources. Additionally, these colleges often take applications well into the spring and summer—after students can already file the FAFSA under current rules.

It is less likely that the relatively small number of highly-selective colleges that get a disproportionate amount of media coverage will respond to PPY by getting financial aid offers out any earlier. For example, the Ivy League institutions didn’t even release their admissions notifications for students applying through the regular route until the last day of March, which gives students plenty of time to complete the FAFSA under current rules. Moving up the notification date to January is definitely feasible under PPY, but it requires students to apply earlier—and thus take tests like the ACT or SAT earlier. All students are supposed to commit to one college by May 1, giving students one month under current rules to compare aid packages and make a decision. Colleges may oppose extending this decision period as students have more time to compare offers and potentially request more money from colleges.

I suspect the Department of Education sent their letter to colleges in an effort to get the admissions notification dates at selective colleges moved up, but this goes against the incentives in place at some colleges to reduce the comparison shopping period. Prior prior year still allows students to get their federal aid eligibility earlier, which is a good thing. But for quite a few students, they won’t get their complete financial aid package any earlier.

Fewer Poor Students Are Being Enrolled in State Universities–Here’s Why

This post initially appeared at The Conversation, and is co-authored with my Seton Hall colleague Luke Stedrak.

States have traditionally provided funding for public colleges and universities based on a combination of the number of students enrolled and how much money they were allocated previously.

But, in the face of increasingly tight budgets and pressures to demonstrate their effectiveness to legislators, more and more states are tying at least some higher education funding to student outcomes.

As of 2015, 32 states have implemented a funding system that is based in part on students’ performance in at least some of their colleges. In such states, a portion of state funding is based on metrics such as the number of completed courses or the number of graduates.

Research shows that performance-based funding (PBF) has not moved the needle on degree completions in any substantial way. Our research focuses on the unintended consequences of such funding policies – whether colleges have responded to funding incentives in ways that could hurt disadvantaged students.

We find evidence that these systems may be reducing access for low-income students at public colleges.

Just a popular political strategy?

What is performance-based funding (PBF)? And does it improve college completion rates?

Performance funding, the idea of tying funding to outcomes instead of enrollment, was first adopted in Tennessee in 1979. It spread across the country in waves in the 1990s and 2000s, with some states dropping and adding programs as state budget conditions and political winds changed. In this decade, several states have implemented systems tying most or all of state funding to outcomes.

By basing funding on outcomes such as course completions and the number of degrees awarded, PBF has become a politically popular strategy to improve student outcomes. It has received strong support from the Bill and Melinda Gates and Lumina Foundations – two big players in the higher education landscape.

However, the best available evidence suggests that PBF systems generally do not move the needle on degree completions in any substantial way.

For example, a study of Washington state’s PBF program by Nick Hillman of Wisconsin, David Tandberg of Florida State and Alisa Hicklin Fryar of University of Oklahoma showed no effects on associate degree completion at two-year colleges. The study found positive effects on certificates in technical fields that took less time to complete, but those were the ones that were not as valuable in the labor market.

When Tandberg and Hillman conducted a nationwide study, they found no effect overall of PBF programs on degree completions at two-year and four-year colleges.

However, the small number of PBF programs that had been in effect for at least seven years (giving colleges plenty of time to change their practices in response) did appear to increase the number of bachelor’s degrees awarded by a few percentage points.

More selective and lower standards

While there is no significant evidence of impact, there have been many unintended consequences of this policy.

There is a growing body of evidence, for example, that shows that colleges may be trying to change both their student body and their academic standards in order to meet the state’s performance goals as well as their own priorities.

A research team at Teachers College who interviewed administrators in three states with “high-stakes” PBF systems (Indiana, Ohio and Tennessee) found that colleges facing PBF were both becoming more selective in accepting students and lowering academic standards among current students in an effort to have more students graduate.

A new study by Mark Umbricht and Frank Fernandez at Penn State and Justin Ortagus at University of Florida used data on incoming students to show that Indiana colleges increased selectivity in response to PBF.

They estimated that Indiana colleges lowered admissions rates by nearly 10 percent and increased ACT scores by nearly a full point compared to similar colleges in other states.

In our research, published recently in the Journal of Education Finance, we examined whether public two-year and four-year colleges nationwide changed how they either received or spent money in response to performance funding systems.

We found that colleges generally did not change spending on instruction or research, but they did see significantly less revenue from federal Pell Grants that are primarily given to students with family incomes below US$60,000 per year, suggesting fewer low-income students enrolled. We estimated a statistically significant decline in Pell revenue of about 2 percent at both two-year and four-year colleges.

We also found that four-year colleges offered more institutional grant aid, potentially in the form of merit-based scholarships to attract higher-income students with a greater likelihood of success.

Implications for policy

Although research suggests that performance funding systems have not been particularly effective in increasing the number of degrees that public colleges grant, the fact is that PBF is being adopted in more states. For example, five more states have adopted PBF since 2014, with additional states debating whether to adopt plans of their own.

We believe, this is unlikely to go away anytime soon.

And many states’ existing funding systems are highly inequitable. They favor research universities over less-selective colleges, even though less-selective colleges enroll the lion’s share of low-income students.

States should consider placing provisions in both their enrollment-based and performance-based funding systems to encourage colleges to continuing to enroll an economically diverse student body.

Several states, such as Arkansas, Ohio and Florida, provide additional incentives for graduating Pell Grant recipients. But states need to ensure that these additional funds are sufficient to encourage colleges to enroll academically qualified students from low-income families as well.

To do this, states would need to take three concrete steps. First, states should provide incentives for colleges to not raise admissions standards beyond what is needed to succeed in coursework. Second, they could also provide additional funds for graduating students who require a modest amount of remedial coursework (courses to build skills of less-prepared students), before taking college-level classes.

And finally, it is important that state policymakers and college leaders have honest conversations about the goals of PBF systems and what colleges need to improve their performance. This could help reduce the unintended outcomes.