New Working Paper on the Effects of Gainful Employment Regulations

As debates regarding Higher Education Act reauthorization continue in Washington, one of the key sticking points between Democrats and Republicans is the issue of accountability for the for-profit sector of higher education. Democrats typically want to have tighter for-profit accountability measures, while Republicans either want to loosen regulations or at the very least hold all colleges to the same standards where appropriate.

The case of federal gainful employment (GE) regulations is a great example of partisan differences regarding for-profit accountability. The Department of Education spent much of its time during the Obama administration trying to implement regulations that would have stripped away aid from programs (mainly at for-profit colleges) that could not pass debt-to-earnings ratios. They finally released the first year of data in January 2017—in the final weeks of the Obama administration. The Trump administration then set about undoing the regulations and finally did so earlier this year. (For those who like reading the Federal Register, here is a link to all of the relevant documents.)

There has been quite a bit of talk in the higher ed policy world that GE led colleges to close poor-performing programs, and Harvard closing its poor-performing graduate certificate program in theater right after the data dropped received a lot of attention. But to this point, there has been no rigorous empirical research examining whether the GE regulations changed colleges’ behaviors.

Until now. Together with my sharp PhD student Zhuoyao Liu, I set out to examine whether the owners of for-profit colleges closed lousy programs or colleges after receiving information about their performance.

You can download our working paper, which we are presenting at the Association for the Study of Higher Education conference this week, here.

For-profit colleges can respond more quickly to new information than nonprofit colleges due to a more streamlined governance process and a lack of annoying tenured faculty, and they are also more motivated to make changes if they expect to lose money going forward. It is worth noting that no college should have expected to lose federal funding due to poor GE performance since the Trump administration was on its way in when the dataset was released.

Data collection for this project took a while. For 4,998 undergraduate programs at 1,462 for-profit colleges, we collected information on whether the college was still open using the U.S. Department of Education’s closed school database. Looking at whether programs were still open took a lot more work. We went to college websites, Facebook pages for mom-and-pop operations, and used the Wayback Machine to find information on whether a program appeared to still be open as of February 2019.

After doing that, we used a regression discontinuity research design to look at whether passing GE outright (relative to not passing) or being in the oversight zone (versus failing) affected the likelihood of college or program closures. While the results for the zone versus fail analyses were not consistently significant across all of our bandwidth and control variable specifications, there were some interesting findings for the passing versus not passing comparisons. Notably, programs that passed GE were much less likely to close than those that did not pass. This suggests that for-profit colleges, possibly encouraged by accrediting agencies and/or state authorizing agencies, closed lower-performing programs and focused their resources on their best-performing programs.

We are putting this paper out as a working paper as a first form of peer review before undergoing the formal peer review process at a scholarly journal. We welcome all of your comments and hope that you find this paper useful—especially as the Department of Education gets ready to release program-level earnings data in the near future.

Comments on the Proposed Gainful Employment Regulations

The U.S. Department of Education is currently accepting public comments (through September 13) on their proposal to rescind the Obama administration’s gainful employment regulations, which had the goal of tying federal financial aid eligibility to whether graduates of certain vocationally-focused programs had an acceptable debt-to-earnings ratio. My comments are reprinted below.

September 4, 2018

Annmarie Weisman

U.S. Department of Education

400 Maryland Avenue SW, Room 6W245

Washington, DC 20202

Re: Comments on the proposed rescinding of the gainful employment regulations

Dear Annmarie,

My name is Robert Kelchen and I am an assistant professor of higher education at Seton Hall University.[1] As a researcher who studies financial aid, accountability policies, and higher education finance, I have been closely following the Department of Education (ED)’s 2017-18 negotiated rulemaking efforts regarding gainful employment. I write to offer my comments on certain aspects of the proposed rescinding of the regulations.

First, as an academic, I was pleasantly surprised to see ED immediately referring to a research paper in making its justification to change the debt-to-earnings (D/E) threshold. But that quickly turned into dismay as it became clear that ED had incorrectly interpreted what Sandy Baum and Saul Schwartz wrote a decade ago after Baum clarified the findings of the paper in a blog post.[2] I am not wedded to any particular threshold regarding D/E ratios, but I would recommend that ED reach out to researchers before using their findings in order to make sure they are being interpreted correctly.

Second, the point that D/E ratios can be affected by the share of adult students, who have higher loan limits than dependent students, is quite valid. But it can potentially be addressed in one of two ways if D/E ratios are reported in the future. One option is to report D/E ratios separately for independent and dependent students separately, but that runs the risk of creating more issues of small cell sizes by splitting the sample. Another option is to cap the amount of independent student borrowing credited toward D/E ratios at the same level as dependent students (also addressing the possibility that some dependent students have higher limits due to Parent PLUS loan applications being rejected). This is less useful from a consumer information perspective, but could solve issues regarding high-stakes accountability.

Third, ED’s point about gainful employment using a ten-year amortization period for certificate programs while also offering 20-year repayment plans under REPAYE is well-taken. Switching to a 20-year period would allow some lower-performing programs to pass the D/E test, but it is reasonable given that ED offers a loan repayment plan of that period. (I also approach the idea that programs would lose Title IV eligibility under the prior administration’s regulations as being highly unlikely based on experiences with very few colleges losing eligibility based on high cohort default rates.) In any case, aligning amortization periods to repayment plan periods makes sense.

Fourth, I am highly skeptical that requiring institutions to disclose various outcomes on their own websites would have much value. Net price calculators, which colleges are required to post under the Higher Education Act, are a prime example. Research has shown that many colleges place these calculators on obscure portions of their website and that information is often up to five years out of date.[3] Continuing to publish centralized data on outcomes is far preferable than letting colleges do their own thing, and highlights the importance of continuing to publish outcomes information without any pauses in the data.

Fifth, while providing median debt and median earnings data allows analysts to continue to calculate a D/E ratio, there is no harm in continuing to provide such a ratio in the future alongside the raw data. There is no institutional burden for doing so, and it is possible that some prospective students may find that ratio to be more useful than simply looking at median debt. At the very least, ED should conduct several focus groups to make sure that D/E ratios lack value before getting rid of them.

Sixth, while it is absolutely correct to note that people working in certain service industries receive a high portion of their overall compensation in tips, I find it dismaying as a taxpayer that there is no interest in creating incentives for individuals to report their income as required by law. A focus on D/E ratios created a possibility for colleges to encourage their students to follow the law and accurately report their incomes in order to increase earnings relative to debt payments. ED should instead work with IRS and colleges to help protect taxpayers by making sure that everyone pays income taxes as required.

In closing, I do not have a strong preference about whether ED ties Title IV eligibility to program-level D/E thresholds due to my skepticism that any sanctions would actually be enforced.[4] However, I strongly oppose efforts by ED to completely stop publishing program-level student outcomes data until the College Scorecard data are ready (which could be a few years). Continuing to publish data on certificate graduates’ outcomes in the interim is an essential step since all sectors of higher education already have to report certificate outcomes—meaning that keeping these data treat all sectors equally. Publishing outcomes of degree programs would be nice, but not as important since only some colleges would be included.

As I showed with my colleagues in the September/October issue of Washington Monthly magazine, certificate students’ outcomes vary tremendously both within and across CIP codes as well as within different types of higher education institutions.[5] Once the College Scorecard data are ready, this dataset can be phased out. But in the meantime, continuing to publish data meets a key policy goal of fostering market-based accountability in higher education.

[1] All opinions reflected in this commentary are solely my own and do not represent the views of my employer or funders.

[2] Baum, S. (2018, August 22). DeVos misinterprets the evidence in seeking gainful employment deregulation. Urban Wire. https://www.urban.org/urban-wire/devos-misrepresents-evidence-seeking-gainful-employment-deregulation.

[3] Anthony, A. M., Page, L. C., & Seldin, A. (2016). In the right ballpark? Assessing the accuracy of net price calculators. Journal of Student Financial Aid, 46(2), 25-50. Cheng, D. (2012). Adding it all up 2012: Are college net price calculators easy to find, use, and compare? Oakland, CA: The Institute for College Access and Success.

[4] For more reasons why I am skeptical that all-or-nothing accountability systems such as the prior administration’s gainful employment regulations would actually be effective, see my book Higher Education Accountability (Johns Hopkins University Press, 2018).

[5] Washington Monthly (2018, September/October). 2018 best colleges for vocational certificates. https://washingtonmonthly.com/2018-vocational-certificate-programs.

What New Gainful Employment and Borrower Defense Rules May Look Like

President Trump is fond of negotiating, as can be evidenced through his long business career and many promises to renegotiate a whole host of international agreements. Federal higher education policy is also fond of negotiation, thanks to a process called negotiated rulemaking that brings a range of stakeholders together for an arduous series of negotiations regarding key changes to federal policies. Notably, if stakeholders do not come to an agreement, the Department of Education can write its own rules—something that the Obama administration did on multiple occasions. (For more on the nitty-gritty of negotiated rulemaking, I highly recommend Rebecca Natow’s new book on the topic.)

In a long-expected announcement, the Department of Education announced Wednesday morning that it would be renegotiating two key higher education regulations (gainful employment and borrower defense to repayment) that were initially negotiated during the Obama administration, with the first meetings beginning next month. To get an idea of how expected these announcements were, here are the stock prices for Adtalem (DeVry) and Capella right after the announcement (which began to break around 11:30 AM ET). Note the fairly small movement in share prices, suggesting that changes were baked into stock prices pretty well.

It is extremely likely that the negotiated rulemaking committees won’t be able to come to an agreement (again), so the new rules will reflect the Trump administration’s higher education priorities. Here is my take on what the two rules might look like.

Gainful Employment

The Obama administration first announced its intention to tie federal financial aid eligibility for select vocational programs (disproportionately at for-profit colleges) in 2009 and entered negotiated rulemaking in 2009-10. The first rules, released in 2011, were struck down in 2012 due a lack of a “reasoned basis” for the criteria used. The second attempt entered negotiated rulemaking in 2013, survived legal challenges in 2015, and began to take effect with the first data release in early 2017. Nearly all of the programs that failed in the first year were at for-profit colleges, but this also led to Harvard shutting down a failing graduate theater program. No colleges have lost aid eligibility yet, as two failing years are required before a college is at risk of losing funds.

The Trump administration is likely to take one of three paths in changing gainful employment regulations:

Path 1: Expand the rules to cover everyone. One of the common critiques against the current regulations is that they only cover nondegree programs at nonprofit colleges in addition to nearly all programs at for-profit colleges. For example, doctoral programs in education at Capella University are covered by gainful employment, while my program at Seton Hall University is not. Requiring all programs to be covered by gainful employment would both preserve the goals of the original regulations while silencing some of the concerns. But this would face intense pressure from colleges that are not currently covered (particularly private nonprofits).

Path 2: Restrict the rules to cover only the most at-risk programs. It is possible that gainful employment metrics could be used along other risk factors (such as heightened cash monitoring status or high student loan default rates) to determine federal loan eligibility. If written a certain way, this would free nearly all programs from the rules without completely unwinding the regulations.

Path 3: Make the rates for informational purposes instead of accountability purposes. This is the most likely outcome in my view. The Trump administration can provide useful consumer information without tying federal funds (a difficult thing to actually do, anyway). In this case, I could see all programs being included since the data will be somewhat lower-stakes.

Borrower Defense to Repayment

Unlike gainful employment, borrower defense to repayment regulations were set to affect for-profit and nonprofit colleges relatively equally. Here is what I wrote back in October about the regulations when they were announced.

These wide-ranging regulations, which will take effect on July 1, 2017 (a summary is available here) allow individuals with student loans to get relief if there is a breach of contract or court decision affecting that college or if there is “a substantial misrepresentation by the school about the nature of the educational program, the nature of financial changes, or the employability of graduates.” The language regarding “substantial misrepresentation” could have the largest impact for both for-profit and nonprofit colleges, as students will have six years to bring lawsuits if loans are made after July 1, 2017.

These regulations have been halted and will not take effect until a new round of negotiated rulemaking takes place. They were generally unpopular among colleges, as evidenced by a strong lobbying effort from historically black colleges that were worried about the vague definition of “misrepresentation.” The outcome of this negotiated rulemaking session is likely to be a significant rollback of the scope to cover only the most egregious examples of fraud.

Although these two sets of negotiated rulemaking sessions are likely to mainly be for show due to the Department of Education’s final ability to write rules when the committee deadlocks, they will provide insight into how various portions of the higher education community view the federal role in accountability under the Trump administration. The Department of Education doesn’t livestream these meetings (a real shame), but I’ll be following along on Twitter with great interest. Pass the popcorn, please?

Highlights from the Gainful Employment Data Release

In one of the Obama administration’s final education policy actions, the U.S. Department of Education released a long-awaited dataset of earnings and debt burdens under the gainful employment accountability regulations. These regulations, which survived several legal challenges from the for-profit college sector, require programs that are defined to be vocationally-oriented in nature (the majority of programs at for-profit colleges and a small subset of nondegree programs at public and private nonprofit colleges) to meet one of two debt-to-earnings metrics in order to continue receiving federal financial aid.

Option 1 (annual earnings): The average student loan payment of graduates in a program must be less than 8% of either mean or median earnings in order to pass. Payments between 8% and 12% of income puts programs “in the zone,” while payments above 12% of income result in a failure.

Option 2 (discretionary income): The average student loan payment of graduates in a program must be less than 20% of discretionary income (earnings above 150% of the federal poverty line) in order to pass. Payments between 20% and 30% of discretionary income puts programs “in the zone,” while payments above 30% of discretionary income result in a failure.

Any colleges that fail both metrics twice in a three-year period (using both mean and median earnings) or colleges in the oversight zone for four consecutive years are currently at risk of losing access to federal financial aid. However, both the Trump administration and Congressional Republicans have expressed interest in scrapping this accountability metric, meaning that colleges may not actually face sanctions in the future.

This data release covered 8,637 programs at 2,616 colleges, with about two-thirds of these programs being at for-profit institutions. Overall, 803 programs (9.3%) failed and 1,239 programs (14.4%) were in the oversight zone, with the remaining 76% of programs passing. As shown below, there were large differences in the pass rates by type of institution (note: the incorrect headers on the original post have been fixed). No public colleges failed (likely due to lower tuition levels because of state and local subsidies), and failure rates in the private nonprofit sector were also fairly low. Yet Harvard, Johns Hopkins, and the University of Southern California all had one program fail—leaving these prestigious institutions with some egg on their face. (UPDATE: Harvard suspended admissions for their graduate program in theater that failed gainful employment within one week of the data release.)

Distribution of gainful employment scores by sector and level.
Percentage of programs
Sector Fail Zone Pass N
Public, <2 year 0.0 0.7 99.3 293
Public, 2-3 year 0.0 0.3 99.7 1,898
Public, 4+ year 0.0 0.3 99.7 302
Private nonprofit, <2 year 0.0 10.3 89.7 78
Private nonprofit, 2-3 year 3.5 22.0 74.6 173
Private nonprofit, 4+ year 4.7 9.0 86.3 212
For-profit, <2 year 4.4 19.7 76.0 1,460
For-profit, 2-3 year 11.5 20.1 68.4 2,042
For-profit, 4+  year 22.5 21.4 56.1 2,174
Total 9.3 14.4 76.4 8,637
Source: U.S. Department of Education.
Notes:
(1) Percentages may not add up to 100 due to rounding.
(2) The “total” row excludes five foreign colleges.

 

For-profit colleges that only offer shorter programs (primarily certificates) did pretty well in the gainful employment metrics, with only 4% failing and 20% in the oversight zone. The worst outcomes were by far among four-year for-profit colleges, with 23% failing and 21% in the oversight zone. These poorer outcomes are not being driven by the large for-profit chains. DeVry, Kaplan, Strayer, and Phoenix combined to have just 16 programs fail, while four colleges (Vaterott, Sanford-Brown, the Art Institute of Phoenix, and Virginia College) all had at least 19 programs fail.

I then examined how the different sectors of colleges performed on the debt-to-earnings ratios for both annual income and discretionary income, with the distributions of ratios shown on the charts below. (Red vertical lines represent the cutoffs for being in the oversight zone (left) and failing (right).) These graphs confirm that public colleges have the lowest debt-to-earnings ratios, followed by private nonprofit colleges and for-profit colleges.

gainful_annual_jan17

gainful_disc_jan17

There are three important drawbacks of this data release that are worth emphasizing. First, 133 programs, all at for-profit colleges, are still in the process of appealing their classification (67 that failed and 66 that are in the oversight zone). Second, this only includes a small subset of programs at public and private nonprofit colleges even as similar programs are covered at for-profit colleges. For example, for-profit law schools are included in the gainful employment regulations (and the outcomes aren’t always great). But law programs at nonprofit law schools aren’t covered by the regulations, even though the goal at the end of the program is similar and many colleges expect their law schools to generate excess revenue for their university. Third, by only covering people who completed a program, colleges with low completion rates may look good even if the quality of education induces students to leave the program in disgust.

Regardless of whether federal financial aid dollars are tied to graduates’ debt-to-earnings ratios, it is important to make more program-level outcome data available to students, their families, and the general public. There have been discussions about including program-level data in the College Scorecard, but that is far from a certainty at this point. At the very least, the incoming Trump administration should propose making comparable earnings and debt available for vocationally-focused degree programs at public and private nonprofit colleges.

Gainful Employment and the Federal Ability to Sanction Colleges

The U.S. Department of Education’s second attempt at “gainful employment” regulations, which apply to the majority of vocationally-oriented programs at for-profit colleges and certain nondegree programs at public and private nonprofit colleges, was released to the public this morning. The Department’s first effort in 2010 was struck down by a federal judge after the for-profit sector challenged a loan repayment rate metric on account of it requiring additional student data collection that would be illegal under current federal law.

The 2014 measure was widely expected to contain two components: a debt-to-earning s ratio that required program completers to have annual loan debt be less than 8% of total income or 20% of “discretionary income” above 150% of the poverty line, and a cohort default rate measure that required fewer than 30% of program borrowers (regardless of completion status) to default on federal loans in less than three years. As excellent articles on the newly released measure in The Chronicle of Higher Education and Inside Higher Ed this morning detail, the cohort default rate measure was unexpectedly dropped from the final regulation. This change in rules, Inside Higher Ed reports, would reduce the number of affected programs from 1,900 to 1,400 and the number of affected students from about one million to 840,000.

There will be a number of analyses of the exact details of gainful employment over the coming days (I highly recommend anything written by Ben Miller at the New America Foundation), but I want to briefly discuss on what the changes to the gainful employment rule mean for other federal accountability policies. Just over a month ago, the Department of Education released cohort default rate data, but they tweaked a calculation at the last minute that had the effect of allowing more colleges to get under the 30% default rate threshold at least once in three years to avoid sanctions.

The last-minute changes to both gainful employment and cohort default rate accountability measures highlight the political difficulty of the current sanctioning system, which is on an all-or-nothing basis. When the only funding lever the federal government uses is so crude, colleges have a strong incentive to lobby against rules that could effectively shut them down. It is long past time for the Department of Education to consider sliding sanctions against colleges with less-than-desirable outcomes if the goal is to eventually cut off financial aid to the poorest performing institutions.

Finally, the successful lobbying efforts of different sectors of higher education make it appear less likely that the Obama Administration’s still-forthcoming Postsecondary Institution Ratings System (PIRS) will be able to tie financial aid to college ratings. This measure still requires Congressional approval, but the Department of Education’s willingness to propose sanctions has been substantially weakened over the last month. It remains to be seen if the Department of Education under the current administration will propose how PIRS will be tied to aid before the clock runs out on the Obama presidency.

Should There Be Gainful Employment for College Athletes?

College athletics, particularly the big-revenue sports of NCAA Division I football and basketball, have been in the news lately for less-than-athletic reasons. The recent push by the Northwestern football team to unionize has led to further discussion of whether college athletes* should be compensated beyond their athletic scholarships. And the University of Connecticut’s national championship team in men’s basketball comes a year after they were banned from the tournament due to woeful academic performance and an eight percent graduation rate. (Big congrats to the UConn women’s team, who won another national championship while graduating 92% of students!)

Now things may not be quite as bad as they look. The NCAA’s preferred measure of academic progress is the Academic Progress Rate (APR), which is scored from 0 to 1000 based on retention and eligibility of athletes. Colleges aren’t penalized for athletes who leave without a degree, as long as they stay eligible while competing. This measure is likely more reasonable for athletes who leave for the professional ranks, but this excludes students who exhaust their eligibility and do not become professionals. The APR doesn’t take graduation into account—a significant limitation in this case.

I can’t help think of what could happen if the general principles of gainful employment—a hot political topic in the vocational portions of higher education—would apply to students with athletic scholarships. While the primary metrics of the current gainful employment proposal (debt to income ratios) may not apply to students with full scholarships, some sort of earning and employment measure could be used to track the future success of former athletes. If former players on college teams were unable to obtain professional athletic or academic major-related employment, the team could be subject to sanctions.

I’d love to hear your thoughts on gainful employment for college athletes in the comment section. I’m not taking an actual stand in favor or against this idea, but it’s something potentially worth additional discussion.

* I’m sure the NCAA would rather that I call them “student-athletes,” but I use “athletes” and “students” where appropriate.