How the New Carnegie Classifications Scrambled College Rankings

Carnegie classifications are one of the wonkiest, most inside baseball concepts in the world of higher education policy. Updated every three years by the good folks at Indiana University, these classifications serve as a useful tool to group similar colleges based on their mix of programs, degree offerings, and research intensity. And since I have been considered “a reliable source of deep-weeds wonkery” in the past, I wrote about the most recent changes to Carnegie classifications earlier this year.

But for most people outside institutional research offices, the first time the updated Carnegie classifications really got noticed was with this fall’s college rankings season. Both the Washington Monthly rankings that I compile and the U.S. News rankings that I get asked to comment about quite a bit rely on Carnegie classifications to define the group of national universities. We both use the Carnegie doctoral/research university category for this, putting master’s institutions to a master’s university category (us) or regional universities (U.S. News). With the number of Carnegie research universities spiking from 334 in the 2015 classifications to 423 in the most recent 2018 classifications, this introduces a bunch of new universities into the national rankings.

To be more exact, 92 universities appeared in Washington Monthly’s national university rankings for the first time this year, with nearly all of these universities coming out of the master’s rankings last year. The full dataset of these colleges and their rankings in both the US News and Washington Monthly rankings can be downloaded here, but I will highlight a few colleges that cracked the top 100 in either ranking below:

Santa Clara University: #54 in US News, #137 in Washington Monthly

Loyola Marymount University: #64 in US News, #258 in Washington Monthly

Gonzaga University: #79 in US News, #211 in Washington Monthly

Elon University: #84 in US News, #282 in Washington Monthly

Rutgers University-Camden: #166 in US News, #57 in Washington Monthly

Towson University: #197 in US News, #59 in Washington Monthly

Mary Baldwin University: #272 in US News, #35 in Washington Monthly

These new colleges appearing in the national university rankings means that other colleges got squeezed down the rankings. Given the priority that many colleges and their boards place on the US News rankings, it’s a tough day on some campuses. Meanwhile, judging by press releases, the new top-100 national universities are probably having a good time right now.

Some Updates on the State Performance Funding Data Project

Last December, I publicly announced a new project with Justin Ortagus of the University of Florida and Kelly Rosinger of Pennsylvania State University that would collect data on the details of states’ performance-based funding (PBF) systems. We have spent the last nine months diving even deeper into policy documents and obscure corners of the Internet as well as talking with state higher education officials to build our dataset. Now is a good chance to come up for air for a few minutes and provide an update on our project and our status going forward.

First, I’m happy to share that data collection is moving along pretty well. We gave a presentation at the State Higher Education Executives Officers Association’s annual policy conference in Boston in early August and were also able to make some great connections with people from more states at the conference. We are getting close to having a solid first draft of a 20-plus year dataset on state-level policies, and are working hard to build institution-level datasets for each state. As we discuss in the slide deck, our painstaking data collection process is leading us to question some of the prior typologies of performance funding systems. We will have more to share on that in the coming months, but going back to get data on early PBF systems is quite illuminating.

Second, our initial announcement about the project included a one-year, $204,528 grant from the William T. Grant Foundation to fund our data collection efforts. We recently received $373,590 in funding from Arnold Ventures and the Joyce Foundation to extend the project through mid-2021. This will allow us to build a project website, analyze the data, and disseminate results to policymakers and the public.

Finally, we have learned an incredible amount about data collection over the last couple of years working together as a team. (And I couldn’t ask for better colleagues!) One thing that we learned is that there is little guidance to researchers on how to collect the types of detailed data needed to provide useful information to the field. We decided to write up a how-to guide on data collection and analyses, and I’m pleased to share our new article on the topic in AERA Open. In this article (which is fully open access), we share some tips and tricks for collecting data (the Wayback Machine might as well be a member of our research team at this point), as well as how to do difference-in-differences analyses with continuous treatment variables. Hopefully, this article will encourage other researchers to launch similar data collection efforts while helping them avoid some of the missteps that we made early in our project.

Stay tuned for future updates on our project, as we will have some exciting new research to share throughout the next few years!

How Much Money Should Students Borrow for College?

Well-known personal finance personality Dave Ramsey apparently tweeted out something about paying for college yesterday. When I went to click on the link, I got the following notice from Twitter.

As far as I am aware, this is the first time that someone has blocked me on Twitter (I only use the blocking function for likely Russian bots and people who have made racist or sexist statements against people I know). And I’m pretty sure I know the reason why Ramsey blocked me—this interview that I did with Money magazine last spring in which I noted that his advice to avoid all college debt is a generally bad idea for students. Limiting ridiculous credit card debt and having a plan to pay off debt eventually are sound recommendations, but delaying the labor market benefits of a college credential to work your way through debt-free just doesn’t make long-term sense for most people. (And research shows that borrowing for college can improve student outcomes.)

On the other hand, students also should be reasonable about how much they borrow for college. While I was pulling up the Money magazine link for the previous paragraph, I noticed that their feature piece today is on someone with $185,000 in student loan debt and who is struggling to make minimum payments. There are key details missing in the piece, such as how much of the debt is for graduate school, the person’s income, and how much interest has capitalized, but this is certainly a cause for concern absent additional information (which seems to be missing from many of these pieces).

So this brings me to a question that I get asked quite a bit: how much should students borrow for college? To me, the correct answer is generally “it depends”—but most people don’t like that classic answer from a tenured professor. This leads me to specify some basic ground rules that students and their families should consider before signing that Master Promissory Note.

For undergraduate students: I am generally not concerned if students take out the maximum amount in federal student loans in their own name while in college. Younger (dependent) students can typically take out up to $31,000 in federal loans, while independent students can take out up to $57,500. These loans have generous income-driven repayment options that reduce payments if college doesn’t work out financially for a student. (Any forgiven balance outside of Public Service Loan Forgiveness may be taxed, but my guess is that Congress patches that fix on an annual basis going forward.)

Beyond that amount, students and their families may be able to get Parent PLUS or private loans, which generally require a co-signer and varied levels of creditworthiness in order to qualify. Parent PLUS loans scare me (as I talked about last fall with NPR), as they require many parents to pay loans into their retirement and have much lower credit standards than private loans. Students and their families need to have long and hard conversations about borrowing beyond the federal student loan limit to see if parents or co-signers can afford to repay those loans and the student’s likely ability to help parents repay those debts.

For graduate students: Most six-figure student debt burdens are from graduate school, since students can borrow up to the full cost of attendance in their own name through the Grad PLUS program and many of these programs tend to be expensive. New program-level College Scorecard data show that a large number of master’s and professional doctorate programs graduate students with more than $100,000 in debt. And I can speak to this personally as my wife and I are down to the final year of payments on her $110,000 in law school debt.

Income-driven repayment plans and PSLF extend to all graduate debt through the federal government, so there is some protection against low earnings after attending graduate school. However, unless a student is sure that he or she is going to be in a public service field and receive PSLF, it is important to keep loan balances in mind. Income-driven repayment plans require 20 years of payments instead of the ten years under PSLF, and this requires committing ten percent of your income over 150% of the poverty line for much of your prime earning years. Sit down with your family and try to get a handle on expected future earnings (program-level earnings data will come out this fall) and other expenses such as childcare and housing and see what is affordable. $100,000 in debt is extremely manageable for a two-income household making $150,000 per year, but much harder for a single adult making $60,000 per year.

The right answer for how much a student should borrow for college depends quite a bit on individual circumstances, but in general the modest federal loan limits for undergraduate students are manageable for most graduates with the help of income-driven repayment programs. Dave Ramsey may be an influential voice in the personal finance world, but following all of his advice on paying for college is likely to be a losing proposition for many students.

Trends in For-Profit Colleges’ Reliance on Federal Funds

One of the many issues currently derailing bipartisan agreement on federal Higher Education Act reauthorization is how to treat for-profit colleges. Democrats and their ideologically-aligned interest groups, such as Elizabeth Warren and the American Federation of Teachers, have called on Congress to cut off all federal funds to for-profit colleges—a position that few publicly took before this year. Meanwhile, Republicans have generally pushed for all colleges to be held to the same accountability standards, as evidenced by the Department of Education’s recent decision to rescind the Obama-era gainful employment era regulations that primarily focused on for-profit colleges. (Thankfully, program-level debt to earnings data—which was used to calculate gainful employment metrics—will be available for all programs later this year.)

I am spending quite a bit of time thinking about gainful employment right now as I work on a paper with one of my graduate students that examines whether programs at for-profit colleges that failed the gainful employment metrics shut down at higher rates than similar colleges that passed. Look for a draft of this paper to be out later this year, and I welcome feedback from the field as soon as we have something that is ready to share.

But while I was putting together the dataset for that paper, I realized that new data on the 90/10 rule came out with basically no attention last December. (And this is how blog posts are born, folks!) This rule requires for-profit colleges to get at least 10% of their revenue from sources other than federal Title IV financial aid (veterans’ benefits count toward the non-Title IV funds). Democrats who are not calling for the end of federal student aid to for-profits are trying to get 90/10 changed to 85/15 and putting veterans’ benefits in with the rest of federal aid, while Republicans are trying to eliminate the rule entirely. (For what it’s worth, here are my thoughts about a potential compromise.)

With the release of the newest data (covering fiscal years ending in the 2016-17 award year), there are now ten years of 90/10 rule data available on Federal Student Aid’s website. I have written in the past about how much for-profit colleges rely on federal funds, and this post extends the dataset from the 2007-08 through the 2016-17 award years. I limited the sample to colleges located in the 50 states and Washington, DC as well as to the 965 colleges that reported data over all ten years that data have been publicly released. The general trends in the reliance on Title IV revenues are similar when looking at the full sample, which ranges from 1,712 to 1,999 colleges across the ten years.

The graphic below shows how much the median college in the sample relied on Title IV federal financial aid revenues in each of the ten years of available data. The typical institution’s share of revenue coming from federal financial aid increased sharply from 63.2% in 2007-08 to 73.6% in 2009-10. At least part of this increase is attributable to two factors: the Great Recession making more students eligible for need-based financial aid (and encouraging an increase in college enrollment) and increased generosity of the Pell Grant program. Title IV reliance peaked at 76.0% in 2011-12 and has declined each of the most recent five years, reaching 71.5% in 2016-17.

Award Year Reliance on Title IV (pct)
2007-08 63.2
2008-09 68.3
2009-10 73.6
2010-11 74.0
2011-12 76.0
2012-13 75.5
2013-14 74.6
2014-15 73.2
2015-16 72.5
2016-17 71.5
Number of colleges 965

I then looked at reliance on Title IV aid by a college’s total revenues in the 2016-17 award year, dividing colleges into less than $1 million (n=318), $1 million-$10 million (n=506), $10 million-$100 million (n=122), and more than $100 million (n=19). The next graphic highlights that the groups all exhibited similar patterns of change over the last decade. The smallest colleges tended to rely on Title IV funds the least, while colleges with revenue of between $10 million and $100 million in 2016-17 had the highest shares of funds coming from federal financial aid. However, the differences among the groups were less than five percentage points from 2009-10 forward.

For those interested in diving deeper into the data, I highly recommend downloading the source spreadsheets from Federal Student Aid along with the explanations for colleges that have exceeded the 90% threshold. I have also uploaded an Excel spreadsheet of the 965 colleges with data in each of the ten years examined above.

How to Maintain Research Productivity

This summer is my first summer after receiving tenure at Seton Hall. While tenure and promotion to associate professor officially do not kick in until the start of the next academic year in August, there have already been some changes to my job responsibilities. The most notable change is that I have taken over as the director of the higher education graduate programs at Seton Hall, which means taking on a heaping helping of administrative work that is needed to make things run smoothly. While this work does come with a teaching reduction during the academic year, it’s a year-round job that takes a hefty bite out of my schedule. (And yes, professors do work—which is often unpaid—during the summer!)

Over the past few years, a few other factors have contributed to sharply reduce the amount of available time that I have to work on research. Since I teach in a doctoral program, faculty members are typically asked to chair more and more dissertation committees as they gain more experience. I also spend quite a bit of time on the road giving talks and being in meetings on higher education policy issues across the country, which is a great opportunity to catch up on reading dissertations in transit but makes it hard to write. These demands have really hit hard over the last few months, which is why blog posts have been relatively few and far between this year.

I had the chance to participate in a panel discussion through Seton Hall’s Center for Faculty Development last academic year on the topic of maintaining research productivity. I summarize some of my key points below, and people who are interested can listen to the entire podcast. Hopefully, some of these tips are especially useful for new faculty members who are beginning the exciting transition into a tenure-track position and often face more demands on their time than they faced in the past.

(1) Take care of yourself. One challenge of being a faculty member is that an unusually large proportion of our time is unstructured. Even for colleagues who teach three or four classes a semester (I teach two), direct teaching and office hour obligations may only be 20 hours per week. But the amount of work to do is seemingly infinite, resulting in pressures to work absurd hours. Set a reasonable bound on the number of hours that you are willing to work each week and stick to it the best that you can. Also make sure to have some hobbies to get away from the computer. I enjoy running, gardening, and cooking—as demonstrated by these homemade pizzas from last weekend.

(2) Keep your time allocation in mind. In addition to not working too many hours each week, it is important to be spending time on what is eventually rewarded. If your annual review or tenure/promotion guidelines specify that your evaluation is based 40% on research, 40% on teaching, and 20% on service, it is an issue to be spending 25 hours each week on teaching. Talk with experienced faculty members or trusted colleagues about what you can do to improve your teaching efficiency. If efficiency isn’t the issue, it’s time to talk with trusted colleagues about what can be done (if anything) to protect your time for research. I do my best to block off two days each week for research during the academic year, although that does get tough with travel, conference calls, and interviews.

Another helpful hint is structuring assignment due dates so you don’t get overwhelmed. I usually have a conference to attend during the middle of the semester, so I schedule the due date for midterm papers to be right before the trip. That way, I can read papers on the train or plane (since I’m not good at writing away from my trusted home office).

(3) Guard your most productive writing time. Most faculty members that I talk with have a much harder time getting into a research mindset than getting into a teaching or service mindset. This means that for many people, their writing time needs to be the time of day in which they are at their sharpest. Being able to control when you teach and meet with students is often outside your control, but deciding when to answer e-mails and prepare classes typically is. It’s hard enough to write, so blocking off several of your most productive hours each week to write is a must when tenure and promotion depend on it. Conference calls and nonessential meetings can fit nicely into the rest of your week.

(4) Collaborations can be awesome. (Caveat: Make sure your discipline/institution rewards collaborative research first. Most do, but some don’t.) In the tenure and promotion process, it is crucial for faculty members to be able to demonstrate their own research agenda and contribution to their field of study. But strategically using collaborations in addition to sole-authored projects can be a wonderful way to maintain research productivity and stay excited about your work. I have been fortunate to work with a number of great collaborators over the last few years, and just had a great time last week going out to Penn State to meet with my colleagues on a fun research project on state performance funding policies. These collaborations motivate me to keep working on new projects!

Colleagues, I would love to hear your thoughts about how you keep your research agenda moving forward amid a host of other demands. Either comment below or send me a note; I would love to do a follow-up post with more suggestions!

Some Thoughts on Program-Level College Scorecard Data

The U.S. Department of Education has been promising to make program-level outcome data available on the College Scorecard for several years now. The Obama administration started the underlying data collection after releasing the initial Scorecard to the public in 2015, and the Trump administration elevated this topic by issuing an executive order earlier this year. I was at a technical review panel at ED last month on this topic, and I just noticed earlier today that members of the public can now comment on our two-day discussion in one of Washington’s most scenic windowless conference rooms.

So I was surprised to see a press release this afternoon announcing that the College Scorecard had been updated in several important ways. This update includes more than just program-level data. The public-facing site now has data on certificate-granting institutions, as well as using IPEDS data on graduation rates that go beyond first-time, full-time students. Needless to say, I’m happy to see both of these improvements, even though I am somewhat skeptical that students pursuing vocational certificates will access the public-facing Scorecard to the same extent that students seeking bachelor’s degrees will.

But this blog post focuses on program-level Scorecard data, which are preliminary and will be updated as soon as later this year. I used the combined 2015-16 and 2016-17 dataset (the most recent year available), which includes data on all graduates who received federal financial aid. This means that coverage is better for certain programs than others; for example, law schools are better covered than PhD programs since relatively few PhD students borrow compared to law students. The dataset contains 194,575 programs across 6,094 institutions.

Here are some highlights:

  • Median debt data are only available for 42,430 programs (21.8% of the sample), as small programs do not have data shown due to privacy concerns. But based on IPEDS completions, about 70% of students are enrolled in programs where debt data are available.
  • Here are the average median debt burdens by credential level:
    • Undergraduate certificate: $10,953 (n=4,146)
    • Associate: $15,134 (n=5,952)
    • Bachelor’s: $23,382 (n=23,649)
    • Graduate certificate: $48,513 (n=266)
    • Master’s: $42,335 (n=7,011)
    • First professional: $141,310 (n=660)
    • Doctoral: $95,715 (n=716)
  • 172 programs had over $200,000 in median debt, and it looks like the top 116 programs are all in health sciences. The data are preliminary, but Roseman University of Health Sciences’s dentistry program has the top listed debt burden at a cool $410,213. Meanwhile, 3,970 programs had median debt burdens below $10,000.

I am thrilled to see program-level debt data, both as a researcher (if I only had more time to sit down and dive into the data!) and as the newly-minted director of higher education graduate programs. Thanks to this dataset, I now know that roughly half of the students in the educational leadership doctoral program (K-12 and higher ed) at Seton Hall borrow, and median debt among graduates is $63,045. I hope that colleges around the country use this tool to get a handle on their graduates’ situations now that data are available for more than just those programs that were covered by gainful employment.

Oh, and about gainful employment. Once earnings data come out (which hopefully will be soon), it will be possible to calculate a debt-to-earnings ratio for programs that cover a large number of students even without the sanctions present in the now-mothballed gainful employment regulations. Also expect to see loan repayment rates in the updated Scorecard, which will shed some interesting light on income-driven repayment rate usage and the implications for students and taxpayers.

Why the Next Secretary of Education Should Come from Higher Ed

Elizabeth Warren is one of several Democratic presidential candidates who is highlighting education as a key policy issue in their campaigns. A few weeks after announcing an ambitious proposal to forgive nearly half of all outstanding student debt and strip for-profit colleges’ access to federal financial aid (among other issues), she returned to the topic in advance of a town hall event with the American Federation of Teachers in Philadelphia. In a tweet, Warren promised that her Secretary of Education would be a public school teacher.

This would be far from unprecedented: both Rod Paige (under George W. Bush) and John King (under Barack Obama) were public school teachers. But if Warren or any other Democrat wants to influence American education to the greatest extent possible, the candidate should appoint someone from higher education instead of K-12 education. (The same also applies to Donald Trump, who apparently will need a new Secretary of Education if he wins a second term.) Below, I discuss a few reasons why ED’s next leader should come from higher ed.

First, the Every Student Succeeds Act, signed into law in 2015, shifted a significant amount of power from ED to the states. This means that the federal government’s power in K-12 education has shifted more toward the appropriations process, which is controlled by Congress. Putting a teacher in charge of ED may result in better K-12 policy, but the change is likely to be small due to the reduced amount of discretion.

Meanwhile, on the higher education side of the ranch, I still see a comprehensive Higher Education Act reauthorization as being unlikely before 2021—even though Lamar Alexander is promising a bill soon. I could see a narrowly-targeted bill on FAFSA simplification getting through Congress, but HEA reauthorization is going to be tough in three main areas: for-profit college accountability, income-driven student loan repayment plans, and social issues (Title IX, campus safety, and free speech). Warren’s proposal last month probably makes HEA reauthorization tougher as it will pull many Senate Democrats farther to the left.

This means that ED will continue to have a great amount of power to make policy through the negotiated rulemaking process under the current HEA. Both the Obama and Trump administrations used neg reg to shape policies without going through Congress, and a Democratic president is likely to rely on ED to undo Trump-era policies. Meanwhile, a second-term Trump administration will still have a number of loose ends to tie up given the difficulty of getting the sheer number of regulatory changes through the process by November 1 of this year (the deadline to have rules take effect before the 2020 election).

I fully realize that promising a public school teacher as Secretary of Education is a great political statement to win over teachers’ unions—a key ally for Democrats. But in terms of changing educational policies, candidates should be looking toward higher education veterans who can help them reshape a landscape in which there is more room to maneuver.

Which Colleges Failed the Latest Financial Responsibility Test?

Every year, the U.S. Department of Education is required to issue a financial responsibility score for private nonprofit and for-profit colleges, which serves as a crude measure of an institution’s financial health. Colleges are scored on a scale from -1.0 to 3.0, with colleges scoring 0.9 or below failing the test (and having to put up a letter of credit) and colleges scoring between 1.0 and 1.4 being placed in a zone of additional oversight.

Ever since I first learned of the existence of this metric five or six years ago, I have been bizarrely fascinated by its mechanics and how colleges respond to the score as an accountability pressure. I have previously written about how these scores are only loosely correlated with college closures in the past and also wrote an article about how colleges do not appear to change their fiscal priorities as a result of receiving a low score.

ED typically releases financial responsibility scores with no fanfare, and it looks like they updated their website with new scores in late March without anyone noticing (at least based on a Google search of the term “financial responsibility score”). I was adding a link to the financial responsibility score to a paper I am writing and noticed that the newest data—for the fiscal year ending between July 1, 2016 and June 30, 2017—was out. So here is a brief summary of the data.

Of the 3,590 colleges (at the OPEID level) that were subject to the financial responsibility test in 2016-17, 269 failed, 162 were in the oversight zone, and 3,159 passed. Failure rates were higher in the for-profit sector than in the nonprofit sector, as the table below indicates.

Financial responsibility scores by institutional type, 2016-17.

Nonprofit For-profit Total
Fail (-1.0 to 0.9) 82 187 269
Zone (1.0 to 1.4) 58 104 162
Pass (1.5 to 3.0) 1,559 1,600 3,159
Total 1,699 1,891 3,590

 

Among the 91 institutions with the absolute lowest score of -1.0, 85 were for-profit. And many of them were a part of larger chains. Education Management Corporation (17), Education Affiliates, Inc. (19), and Nemo Investor Aggregator (11) were responsible for more than half of the -1 scores. Most of the Education Affiliates (Fortis) and Nemo (Cortiva) campuses still appear to be open, but Education Management Corporation (Argosy, Art Institutes) recently suffered a spectacular collapse.

I am increasingly skeptical of financial responsibility scores as a useful measure of financial health because they are so backwards-looking. The data are already three years old, which is an eternity for a college on the brink of collapse (but perhaps not awful for a cash-strapped nonprofit college with a strong will to live on). I joined Kenny Megan from the Bipartisan Policy Center to write an op-ed for Roll Call on a better way to move forward with collecting more updated financial health measures, and I would love your thoughts on new ways to proceed!

Three New Articles on Performance-Based Funding Policies

As an academic, few things make me happier than reading cutting-edge research conducted by talented scholars. So I was thrilled to see three new articles on a topic near and dear to my heart—performance-based funding (PBF) in higher education—come out in top-tier journals. In this post, I briefly summarize the three articles and look at where the body of research is heading.

Nathan Favero (American University) and Amanda Rutherford (Indiana University). “Will the Tide Lift all Boats? Examining the Equity Effects of Performance Funding Policies in U.S. Higher Education.” Research in Higher Education.

In this article, the authors look at state PBF policies (divided into earlier 1.0 policies and later 2.0 policies) to examine whether PBF affects four-year colleges within a state differently. They found evidence that the wave of 2.0 policies may negatively affect less-selective and less-resourced public universities, while 1.0 policies affected colleges in relatively similar ways. In a useful Twitter thread (another reason why all policy-relevant researchers should be on Twitter!), Nathan discusses the implications on equity.

Lori Prince Hagood (University System of Georgia). “The Financial Benefits and Burdens of Performance Funding in Higher Education.” Educational Evaluation and Policy Analysis.

Lori’s article digs into the extent that PBF policies affect per-student state appropriations at four-year colleges, defining PBF as whether a state had any policy funded in a given year. The first item worth noting from the paper is that per-student funding in PBF states has traditionally been lower than in non-PBF states. This may change going forward as states with more generous funding (such as California) are now adopting PBF policies. Lori’s main finding is that selective and research universities tend to see increased state funding following the implementation of PBF, while less-selective institutions see decreased funding, raising concerns about equity.

As an aside, I had the pleasure of discussing an earlier version of this paper at the 2017 Association for the Study of Higher Education conference (although I had forgotten about that until Lori sent me a nice note when the article came out). I wrote in my comments at that time: “I think it has potential to go to a good journal with a modest amount of additional work.” I’m not often right, but I’m glad I was in this case!

Denisa Gándara (Southern Methodist University). “Does Evidence Matter? An Analysis of Evidence Use in Performance-Funding Policy Design.” The Review of Higher Education.

Denisa’s article is a wonderful read alongside the other two because it does not use difference-in-differences techniques to look at quantitative effects of PBF. Instead, she digs into how the legislative sausage of a PBF policy is actually made by studying the policy processes in Colorado (which adopted PBF across two-year and four-year colleges) and Texas (which never adopted PBF in the four-year sector). Her interviews reveal that PBF models in other states and national advocacy groups such as Complete College America and HCM Strategists were far more influential than lowly academic researchers.

In a Twitter thread about her new article, Denisa highlighted the following statement:

As a fellow researcher who also talks with policymakers on a regular basis, I have quite a few thoughts on this statement. Policymakers (including in blue states) are increasingly hesitant to give colleges more money without tying a portion of those funds to student outcomes, and other ways of funding colleges also raise equity concerns. So expect PBF to expand in the next several years.

Does this mean that academic research on PBF is irrelevant? I don’t think so. Advocacy organizations are at least partially influenced by academic research; for example, see how the research on equity metrics in PBF policies has shaped their work. It is the job of researchers to keep raising critical questions about the design of PBF policies, and it is also our job to conduct more nuanced analyses that dive into the details of how policies are constructed. That is why my new project with Kelly Rosinger of Penn State and Justin Ortagus of the University of Florida to collect these details over time excites me so much—it is what the field needs to keep building upon great studies such as the ones highlighted here.