College Accountability and the Obama Budget Proposal

The Fiscal Year 2015 $3.9 trillion budget document from the Obama Administration includes a request of $68.6 billion in discretionary funds for the Department of Education, up $1.3 billion from 2014 funding. This excludes a great deal of mandatory spending on entitlements, including student loan costs/subsidies, some Pell Grant funding, and some other types of financial aid. (Mandatory spending is much harder to eliminate than discretionary funding, as illustrated by this helpful CBO summary.) The budget is also a reflection of the Administration’s priorities, even if many components are unlikely to be approved by Congress. For a nice summary of the Department of Education’s request, see this policy brief from the New America Foundation.

On the higher education front, the Obama budget implies that accountability will be a key priority of the Department of Education. The Administration made two key requests in this area: $10 million to fund continued development of the Postsecondary Institution Ratings System (PIRS) and $647 million for a fund to reward colleges that enroll and graduate Pell recipients. There was a holdover request for $4 billion in mandatory funds for a version of Race to the Top in higher education, but few in the higher education policy community are taking this plan seriously.

The $10 million for PIRS would go toward “further development and refinement of a new college rating system” (see p. T-156). This request is a signal that the Administration is taking the development of PIRS seriously, but the $10 million in funds suggests that large-scale additional data collection is unlikely to happen in the near future. It is also unlikely that the federal government will work to audit IPEDS data for the rating, something that I called for in my recent policy brief on ratings. Even if the specific $10 million request for PIRS is not acted upon, the Department of Education will use other discretionary funds to move forward.

The $647 million request for College Opportunity and Graduation Bonuses, if approved, would provide bonuses to colleges that are successful in enrolling and graduating large numbers of Pell recipients. I view this as a first attempt to tie federal funds to college performance using metrics that are likely to be in PIRS. I would be surprised if any Pell Grant funds get reallocated through college ratings except for perhaps a handful of very low-performing colleges, but it is possible to get some additional bonus funds tied to ratings.

I had a poll on a blog post a couple weeks ago asking for readers’ thoughts of the likelihood that PIRS would be tied to student financial aid dollars by 2018. The majority of the respondents gave this less than a 50% chance of happening, and I am inclined to agree as well. The Administration’s budget priorities suggest a serious push toward tying some funds to performance, although it is worth emphasizing that a future Congress and President must agree.

What are your thoughts of the Obama Administration’s higher education budget, particularly about accountability? If you have any comments to share, please do so and continue the conversation!

New Policy Brief on College Ratings

I am pleased to announce the release of my newest policy brief, “Moving Forward with Federal College Ratings: Goals, Metrics, and Recommendations” through my friends at the Wisconsin Center for the Advancement of Postsecondary Education (WISCAPE). In the brief, I outline the likely goals of the Obama Administration’s proposed Postsecondary Institution Ratings System (PIRS), discuss some potential outcome measures, and provide recommendations for a fair and effective ratings system given available data. I welcome your comments on the brief!

Will Federal Aid Be Tied to College Ratings? (Poll)

With all of the discussion of what will be included in the proposed Postsecondary Institution Ratings System (PIRS), there has been relatively little discussion about whether federal Title IV financial aid will actually be tied to the ratings by 2018—as the President has specified. I would love to get your thoughts on the feasibility by taking the following poll, and leaving any additional comments below.

 

 

I’ll share my thoughts in a subsequent post, so stay tuned!

Senator Warren’s Interest Rate Follies

First-term Senator Elizabeth Warren (D-MA) is a darling of the progressive Left, and she has been mentioned as a possible Presidential candidate in 2016 (although she has stated she’s not running). One of the ways she has gained support with the Democratic base is through her many public statements about the federal government’s purported profit on student loans, which she cites to be $51 billion in Fiscal Year 2013. Given the huge profit, she has introduced legislation to drop interest rates to the overnight borrowing rate at the Federal Reserve: 0.75%.

Her argument suffers from one main problem: student loans carry risk for the federal government. (She made my 2013 not-top-ten list for this reason.) The Congressional Budget Office, where the $51 billion estimate came from, uses federal borrowing costs as a discount rate. This discount rate is very low, in part because the federal government is viewed as very unlikely to default (even with the possibility of debt ceiling shenanigans). As a result, numerous groups have suggested the use of fair-value accounting, in which the risk of default is considered. Indeed, the Washington Post’s fact-checking blog gave Senator Warren’s statement of a $51 billion profit “two Pinocchios” because it did not consider fair-value accounting.

[On Twitter, the wonderful Libby Nelson notes that my explanation of fair-value accounting vs. federal regulations is unclear. Here is a nice CBO summary of the different methods.]

With the debate over student loan profits and accounting methods as a backdrop, the release of Friday’s Government Accountability Office report on federal student loans was eagerly anticipated in the higher education community. The title of the report succinctly summarizes the rest of the document: “Borrower Interest Rates Cannot Be Set in Advance to Precisely and Consistently Balance Federal Revenues and Costs.” This resulted in a few howlers from policy analysts, including this gem from Matt Chingos at Brookings:

Karen Weise at Bloomberg was a little more diplomatic with her summary of the report:

The report itself is fairly dry, but it does emphasize something that should be kept in mind when considering the costs of student loan programs. Due to the growing prevalence of extended payment plans, increased rates of income-based repayment plan usage, and the continued risk of defaults, the actual amount of the subsidy or cost on student loans will not be known for 40 years after disbursement.  Each of these individual variables could also have a large effect on the long-run subsidy or cost; for example, a higher-than-expected rate of income-based repayment participation could increase program costs.

The following paragraph on pages 18 and 19 sums up a key point of the report:

“As of the end of fiscal year 2013, it is estimated that the government will generate about $66 billion in subsidy income from the 2007 to 2012 loan cohorts as a group. However, current estimates for this group of loan cohorts are based predominantly on forecasted cash flow data derived from assumptions about future loan performance. As more information on actual cash flows for these loans becomes available, subsidy cost estimates will change. As a result, it is unclear whether these loan cohorts will ultimately generate subsidy income, as currently estimated, or whether they will result in subsidy costs to the government. This will not be known with certainty until all cash flows have been recorded after loans have been repaid or discharged—which may be as many as 40 years from when the loans were originally disbursed.”

I read this paragraph as providing possible evidence that interest rates may have been set relatively high compared to the federal cost of borrowing. (Recall that the interest rates for subsidized Stafford loans declined from 6.8% in 2007 to 3.4% in 2011, while Treasury rates were at historic lows.) The spread between the 10-year Treasury yield in May versus the interest rate on subsidized Stafford loans has been the following for the past seven years:

Year Stafford (pct) 10-yr T-note (pct) Spread (pct)
2007 6.8 4.75 2.05
2008 6.0 3.88 2.12
2009 5.6 3.29 2.31
2010 4.5 3.42 1.08
2011 3.4 3.17 0.23
2012 3.4 1.80 1.60
2013 3.86 1.93 1.93

This interest rate spread is statutorily set at 2.05% for subsidized Stafford loans in the future, roughly the long-run average. So while future GAO reports a few years after disbursement may find similar results, what we’ll all be waiting for is longer-term data to see if the estimates hold true. The federal government doesn’t necessarily have a great history of long-run cost projections, so I’m expecting this spread to disappear over time. (And keep in mind this report doesn’t fully account for risk.)

Yet Senator Warren and eight other Democrats released a press release on Friday afternoon with the headline of “Democratic Senators Highlight Obscene Government Profits Off Student Loan Program.” They focused entirely on the initial projection of a $66 billion profit over five years and entirely ignored the long-run uncertainty highlighted by the GAO. This press release is a great example of selecting only the most favored parts of a report, while ignoring other important details along the way. Again, the Twittersphere (myself included) expressed its thoughts:

On a more fundamental note, I think that Senator Warren and colleagues are misguided in their efforts to continue lowering student loan interest rates. Given the reality that higher education funding is a zero-sum game, I would much rather see funds used to support the Pell Grant, work-study, and other upfront sources of aid for students than slightly lower loan payments after students have already left college. (The same argument holds against tax credits.) Senator Warren may not be running for President (yet), but she’s in the running for my 2014 not-top-ten list.

My State of the Union Wish List

My State of the Union Wish List

I don’t have tremendously high expectations for tonight’s State of the Union address regarding higher education, given the priority placed on other topics such as social mobility and potentially even foreign affairs. However, I would be thrilled if President Obama and/or the small army of Republicans responding touched on any of the three following items:

(1) Don’t make grand claims on fixing the rising burden of student loan debt. While student loan debt has crossed $1 trillion, it’s unclear whether any of the proposals out there would seriously help students after they leave college—let alone encourage students to attend college. Last year’s fight over interest rates was an example of tinkering around the edges. The push for so-called “Pay it Forward” plans might help, but these plans are a long way from enrolling students and may have adverse consequences. I encourage the President and Republicans to bring up these ideas, but don’t overpromise here.

(2) Talk about access to college for more than just high-achieving, low-income students. The recent White House summit on these students is a nice PR push, but will do little to improve college access. Most of these students are going to college somewhere, although some are attending less-selective institutions. As Matt Chingos at Brookings notes, focusing on the problem of “undermatching” won’t move the college completion margin in any substantial way. Focus on trying to increase college access for more than just the small number of very well-prepared students.

(3) Don’t overpromise on college ratings. While the push for federal ratings is moving forward this year, these ratings won’t be released for at least several more months. And once the ratings get released, there is no guarantee that students use the ratings in any meaningful way (although it’s possible). Additionally, tying aid to ratings takes an act of Congress and won’t happen during the current administration. Keep plugging forward on the ratings work, but don’t make them sound like the solution to all of our problems.

I’m looking forward to the speech tonight, and please send along your wish list through either the comments section or via Twitter!

Can Maintenance of Effort Programs Fund Public Higher Education?

The American Association of State Colleges and Universities released a policy paper this week calling for the federal government to enact (and fund) a program designed to encourage states to increase their support for public higher education. The AASCU brief rightly notes that per-student funding for public higher education has fallen over the past three decades (the magnitude of which is overstated somewhat due to their choice in inflation adjustments), and they propose a potential solution in the form of a maintenance of effort provision.

AASCU’s proposal would give colleges a partial match of their higher education appropriations, as long as per-FTE funding to institutions is higher than 50% of the value of the maximum Pell Grant and did not decline from the previous year’s value. The value of the matching funds would go up as state appropriations to institutions increased. They estimate that their hypothesized program would cost something in the neighborhood of $10-$15 billion per year, which could be paid for by cutting waste, fraud, and abuse in current financial aid systems (particularly among for-profits) and by implementing some sort of risk-sharing for student loans—which I’ve written on recently.

However, I view the plan as having a fatal flaw. By only including state appropriations to institutions in the calculation—and not requiring that the matching funds be spent on higher education—states can game the system to get additional money from the federal government. States could reduce funding to their financial aid programs and direct those funds toward institutional appropriations in order to get federal dollars, which could be used for K-12 education, healthcare, or tax cuts.

If states followed the incentive to eliminate all grant aid and fund institutions instead, tuition would likely decrease (something that AASCU institutions would appreciate). The most recent NASSGAP survey of state aid programs found that states spend $9.4 billion per year on grant aid, two-thirds of which is allocated based on financial need. Putting this money into state appropriations would cost the federal government several billion dollars, with no guarantees of any additional funding for students or institutions.

I have a hard time seeing Congress approving this maintenance of effort plan, regardless of the merits. Lobbyists for the private nonprofit and for-profit sectors are likely to strongly oppose this measure, as are lobbying groups for K-12 education, healthcare, and corrections spending (behind the scenes) since higher education is often cut at the expense of higher ed. In addition, this is likely to be a nonstarter in the House due to its placing restrictions on state priorities.

I’m glad to see this proposal from AASCU, but I don’t see it becoming law anytime soon. I would suggest that they follow up with some more details on their proposed risk-sharing program, as well as how elements of this plan could be incorporated into the Obama Administration’s proposed college ratings.

The 2013 Higher Education Not Top Ten List

Yesterday, I put out my top-ten list of higher education policy and finance issues from 2013. And today, I’m back with a list of not-top-ten events from the year (big thanks to Justin Chase Brown for inspiring me to write this post). These are events that left me shaking my head in disbelief or wondering how someone could fail so dramatically.

(Did I miss anything? Start the discussion below!)

10. Monsters University isn’t real. The higher education community was abuzz this summer with the premiere of Pixar’s newest movie about one of the few universities outside Fear Tech specializing in scaring studies. The Monsters University website is quite good, and as Jens Larson at U of Admissions Marketing notes, it’s hard to distinguish from many Title IV-participating institutions. I’ll use this blog post to announce my willingness to give a lecture or two at Monsters University. (As an aside, since the two main characters didn’t graduate, their post-college success may not help MU’s scores in a college rating system.)

9. Brent Musburger set men back at least five decades in the course of 30 seconds. His public ogling of the girlfriend of Alabama quarterback A.J. McCarron during January’s BCS championship game instantly became a YouTube sensation. Musburger shouldn’t have listened to his partner in The Waterboy, Dan Fouts, who urged him to not hold anything back in the last game of the season. McCarron, on the other hand, is preparing to play Oklahoma in the Sugar Bowl on January 2.

8. Rankings and ratings are not the same thing. While college leaders tend not to like the Obama Administration’s proposed Postsecondary Institution Rating System, it is important to emphasize the difference between rankings and ratings. Rankings assign unique values to each institution (like the college football or basketball polls), while ratings lump colleges into broad categories (think A-F grades). Maybe since I work on college rankings, I’m particularly annoyed by the confusion. In any case, it’s enough to make my list.

7. Mooooove over: The College Board has another rough year. This follows a rough 2012 for the publishers of the SAT, as more students took the ACT than the SAT for the first time last year. But in 2013, the redesign of the SAT got pushed back from 2015 to 2016, giving the ACT more time to gain market share. The College Board followed that up with a head-scratching example of “brand-ing,” passing out millions of cow stickers to students taking the PSAT. If these weren’t enough, the College Board also runs the CSS Profile, a supplemental (and not free) application for financial aid required by many expensive institutions. Rachel Fishman at New America has written extensively about the concerns of the Profile.

6. Gordon Gee is the most interesting man in higher education. The well-traveled university president began 2013 leading Ohio State University, but left the post this summer after his 2012 comments disparaging Notre Dame, Catholic priests, and the ability of the Southeastern Conference to read came to light. Yet, he and his large bowtie collection will be heading to West Virginia University this spring as he assumes the role of interim president. There is still no word if the Little Sisters of the Poor will show up on WVU’s 2014 football schedule.

5. Rate My Professor is a lousy measure of institutional teaching quality. I’m not going to fully dismiss Rate My Professor, as I do believe it can be correlated with an individual professor’s teaching quality. But a Yahoo! Finance piece claiming to have knowledge of the 25 colleges with the worst professors cross the boundaries of absurd. I quickly wrote a response to that piece, noting that controlling for a student’s grade and the difficulty of the course are essential in order to try to isolate teaching quality. This was by far my most-viewed blog of 2013.

4. Elizabeth Warren’s interest rate follies. The Democratic Senator from Massachusetts became even more of a progressive darling this spring when she announced a plan to tie student loan interest rates to the Federal Reserve’s overnight borrowing rate—0.75%. Unfortunately, this plan made no sense on several dimensions. While overnight borrowing has nearly no risk, student loans (over a ten-year period) have considerable risk. Additionally, if interest rates were set this low, money would have to come from somewhere else. I would much rather see the subsidy go upfront to students through larger Pell Grants than through lower interest payments after leaving college. Fortunately, Congress listened to smart people like Jason Delisle at New America and her plan went nowhere.

3. The Common Application fails early applicants. The Common Application, used by a substantial number of elite colleges, did not work for some students applying in October and November. The reason was that the Common App’s new software didn’t work and they failed to leave the previous version available in case of problems. Although this didn’t affect the vast majority of students who aspire to attend less-selective institutions, it certainly got the chattering classes talking.

2. The federal government shut down and budget games ensued all year long. The constant partisan battle culminated with a sixteen-day shutdown in October, bringing much of the Department of Education to a screeching halt. While the research community used Twitter to trade downloaded copies of IPEDS data and government reports, other disruptions were more substantial. 2013 also featured sequestration of some education spending, although it looks like the budget process might return to regular order for the next two years.

1. Georgetown Law finds a way to stick taxpayers with the entire cost of law school. It is no secret that law school is an expensive proposition, with six-figure debt burdens becoming the norm at many institutions. But some of the loans can be forgiven if students pursue public service careers for a decade, a program that was designed to help underpaid and overworked folks like public defenders or prosecuting attorneys.

Georgetown’s Loan Repayment Assistance Program advertises that “public interest borrowers might now pay a single penny on their loans—ever!” To do this, the law school increased tuition to cover the cost of 10 years’ worth of loan payments under income-based repayment for students making under $75,000 per year. Students take out Grad PLUS loans to fund this upfront, but never have to pay a dime of those loans back as Georgetown makes the payments. Jason Delisle and Alex Holt, who busted this scheme wide open this summer, estimate that students will have over $150,000 in loans forgiven—and put on the backs of taxpayers.  Although Georgetown tries to defend the practice as being good for society, it is extremely hard to make that argument.

Honorable mentions: #Karma, lousy attacks on performance-based funding research, financial stability of athletics at Rutgers and Maryland, and parking at 98% of campuses.

The Year of Higher Education Policy in Review

As 2013 draws to a close, it’s time to take a look back at some of the biggest happenings (or non-happenings) of the year. Some of these items would have been on the list for several years, but others (including the top happening of the year) are brand-new for 2013. Enjoy the list!

10. There is still some hope in the academic job market. In spite of continued concerns about the working conditions of adjuncts (as exemplified in the case of former Duquesne adjunct Margaret Mary Vojtko—read both the original op-ed and a thoughtful retelling of her life story), the tenure-track job market may just be springing back to life after a few lean years. I’m thankful to be one of those success stories, as I got a great job offer from Seton Hall University before defending my dissertation at the University of Wisconsin-Madison. (Look at my faculty webpage…I’m bona fide and I love my job!) But, in other disciplines, the rough market continues.

9. We heard more noise about reauthorizing the Higher Education Act, but no action. The HEA, which dates back to 1965, is supposed to be renewed in 2014. And Congress is saying all the right things about renewing the HEA, including holding a series of hearings on reforming the Pell Grant. However, it is hard to find anyone in academia or the policy community who thinks that is likely. After all, No Child Left Behind (the Elementary and Secondary Education Act) expired in 2007. If I had to put money on a reauthorization date, I would go for 2017.

8. The higher ed policy world gets RADDical. During late 2012 and early 2013, 17 organizations and teams released white papers as a part of the Gates Foundation-funded Redesigning Aid Design and Delivery (RADD) project. The recommendations of the groups ranged widely (see this nice summary from the National Association of Student Financial Aid Administrators, one of the participating organizations), but all groups suggested substantial changes from the status quo. It’s worth noting that the recommendation shared across the largest number of reports is stabilizing or increasing Pell funding, which could be a tough political lift in the current fiscal environment. This effort was not without its skeptics, as this well-commented Chronicle piece on the influence of Gates funds details.

7. The FAFSA changes to recognize same-sex parents, but is still complicated. Despite the push among many of the RADD grantees and at least some interest in Congress, the FAFSA ends 2013 as perhaps being more complicated than it was at the beginning of the year. This is because the venerable form changed to recognize the existence of same-sex marriages after this year’s Supreme Court ruling and political pressure before the ruling took place. The net result is that some students will see less aid. I would also be remiss if I didn’t mention my work with NASFAA on the feasibility of using prior prior year financial data to determine aid eligibility. That might get tied into the next HEA authorization.

6. Congress reached a reasonable solution on student loan interest rates. Put your shocked face on, folks—Congress did accomplish something without causing too much pain to students or financial aid offices. Interest rates on undergraduate subsidized Stafford loans were set to increase from 3.4% to 6.8% on July 1 (and actually did for a few weeks), leading to the hashtag #DontDoubleMyRate. The rates ended up being tied to 10-year Treasury notes, yielding a rate of under 4% this year; however, advocates note that the rate is likely to rise over time. Thankfully, Senator Warren’s plan to set interest rates based on the Federal Reserve discount window (which is nearly riskless) never received serious discussion.

5. MOOCs expand, but their outcomes are questioned. Massive open online courses (MOOCs) are seen by some as having the potential to change how higher education is delivered, but it is safe to say that not all faculty support them—as evidenced at San Jose State. MOOCs have also been hammered for low completion rates, which are often below 10%. The always-astute Kevin Carey notes, however, that the low completion rates are partially due to people who sign up for the course but never really attempt to complete them. Additionally, large numbers of students may still be completing the course, even if completion rates are low. This issue will only get hotter during 2014.

4. Student loan debt grows amid possible reforms. The Institute for College Access and Success (TICAS) recently put out its annual report on student debt loads—and the results aren’t pretty. The average debt load of graduates was $29,400 in 2012, and 71% of students took out debt. (Even more concerning is the fact that TICAS can’t even get data on a lot of colleges’ graduates.) Increasing debt loads have led to innovative plans to make college more affordable. The most-discussed plan is Oregon’s Pay it Forward proposal, which would be a type of income-based repayment covering tuition and fees in that state. While I have serious concerns about whether the program could work (but think it’s worth a demonstration program), my dear friend and dissertation mentor Sara Goldrick-Rab makes her opposition clear.

3. One of the nation’s more prominent community colleges might actually lose its accreditation. The City College of San Francisco is currently slated to lose its accreditation next summer if they do not meet 357 goals set by the Accrediting Commission for Junior and Community Colleges. Since students cannot qualify for federal Title IV financial aid if they attend an unaccredited college, this would effectively shut down an institution that had nearly 100,000 students. Students and faculty went after the accreditor and nearly shut it down, although it was recently announced that the accreditor could operate for another year. I still think that CCSF will keep its accreditation, but the damage (in terms of enrollment) may already be done.

2. Gainful employment continues to be a hot political topic. The Obama Administration proposed gainful employment regulations several years ago, in which vocationally-oriented colleges would lose Title IV eligibility if they had poor employment and loan repayment outcomes. These rules have been in and out of court for several years, and a new set is now being developed. The Department of Education tried to reach consensus with stakeholders last week, but failed; this means that ED will write its own rules. For all the developments that will happen in 2014, I’ll defer you to Ben Miller’s great work covering the topic.

1. PIRS roars to the public’s attention, and colleges are not happy. As regular readers of this blog know, I’m the methodologist for Washington Monthly’s annual college rankings. Yet I was completely floored when President Obama announced the impending development of a college ratings system for the 2014-15 academic year. (The official title—Postsecondary Institution Rating System or PIRS—just got released yesterday.) Thankfully, I was able to recover quickly enough to go on MSNBC the next night to talk about the proposal.

The Department of Education has done a lot of listening on the college ratings proposal, and the vast majority of the feedback in the higher education community appears to be negative. A recently released poll of college presidents highlights the opposition amid concerns of the ratings favoring highly selective institutions. (Yet the only measure that a majority of college presidents supported using was graduation rates—a measure strongly tied to selectivity.) This recent conference panel also shows some of the issues facing the ratings.

While the long-term goal is to tie ratings to financial aid by 2018 or so, I don’t see this as being likely to happen given its requirement of Congressional approval. However, the ratings could potentially help students even if institutions don’t like the bright lights of accountability. Let’s just say that the discussion around the release of the first ratings this summer should be spicy.

I’ll post a not-top-ten list of higher education policy issues later this week. Send me your suggestions for that piece, and let me know what you think of this list!

Don’t Dismiss Performance Based Funding Research

Performance-based funding (PBF), in which at least a small portion of state higher education appropriations are tied to outcomes, is a hot political topic in many states. According to the National Conference of State Legislatures and work by Janice Friedel and others, 22 states have PBF in place, seven more are transitioning to PBF, and ten more have discussed a switch.

The theory of PBF is simple: if colleges are incentivized to focus on improving student retention and graduation rates, they will redirect effort and funds from other areas to do so. PBF should work if two conditions hold:

(1) Colleges must currently be using their resources in ways that do not strongly correlate with student success, a point of contention with many institutions. If colleges are already operating in a way that maximizes student success, then PBF will not have an impact. PBF could also have negative effects if colleges end up using resources less effectively than they currently are.

(2) The expected funding tied to performance must be larger than the expected cost of changing institutional practices. Most state PBF systems currently tie small amounts of state appropriations to outcomes, which could result in the cost of making changes smaller than the benefits. Colleges also need to be convinced that PBF systems will be around for the long run instead of until the next governor ends the plan or state budget crises cut any funds for PBF. Otherwise, they may choose to wait out the current PBF system and not make any changes. Research by Kevin Dougherty and colleagues through the Community College Research Center highlights the unstable nature of many PBF systems.

For these reasons, the expected impacts of state PBF plans on student outcomes may not be positive. A recent WISCAPE policy brief by David Tandberg, an assistant professor at Florida State University, and Nicholas Hillman, an assistant professor at the University of Wisconsin-Madison, examines whether PBF plans appear to affect the number of associate’s and bachelor’s degrees awarded by institutions in affected states. Their primary findings are that although some states had significantly significant gains in degrees awarded (four at the four-year level and four at the two-year level), other states had significant declines (four at the four-year level and five at the two-year level). Moreover, PBF was most effective in inducing additional degree completions in states with long-running programs.

The general consensus in the research community is that more work needs to be done to understand the effects of state performance-based funding policies on student outcomes. PBF policies differ considerably by state, and it is too early to evaluate the impact of policies on states that have recently adopted the systems.

For these reasons, I was particularly excited to read the Inside Higher Ed piece by Nancy Shulock and Martha Snyder entitled, “Don’t Dismiss Performance Funding,” in response to Tandberg and Hillman’s policy brief. Shulock and Snyder are well-known individuals in the policy community and work for groups with significant PBF experience. However, their one-sided look at the research and cavalier assumptions about the authors’ motives upset me to the point that writing this response became necessary.

First of all, ad hominem attacks about the motives of well-respected researchers should never be a part of a published piece, regardless of the audience. Shulock and Snyder’s reference to the authors’ “surprising lack of curiosity about their own findings” is both an unfair personal attack and untrue. Tandberg and Hillman not only talk about the eight states with some positive impacts, they also discuss the nine states with negative impacts and a larger number of states with no statistically significant effects. Yet Shulock and Snyder do not bother mentioning the states with negative effects in their piece.

Shulock and Snyder are quite willing to attack Tandberg and Hillman for a perceived lack of complexity in their statistical model, particularly regarding their lack of controls for “realism and complexities.” In the academic community, criticisms like this are usually followed up with suggestions on how to improve the model given available data. Yet they fail to do so.

It is also unusual to see a short policy brief like this receive such a great degree of criticism, particularly when the findings are null, the methodology is not under serious question, and the authors are assistant professors. As a new assistant professor myself, I hope that this sort of criticism does not deter untenured faculty and graduate students from pursuing research in policy-relevant fields.

I teach higher education finance to graduate students, and one of the topics this semester was performance-based funding and accountability policy. If Shulock and Snyder submitted their essay for my class, I would ask for a series of revisions before the end of the semester. They need to provide empirical evidence in support of their position and to accurately describe the work done by Tandberg and Hillman. They deserve to have their research fairly characterized in the public sphere.

What Should Be in the President’s College Ratings?

President Obama’s August announcement that his administration would work to develop a college rating system by 2015 has been the topic of a great deal of discussion in the higher education community. While some prominent voices have spoken out against the ratings system (including my former dissertation advisor at Wisconsin, Sara Goldrick-Rab), the Administration appears to have redoubled its efforts to create a rating system during the next eighteen months. (Of course, that assumes the federal government’s partial shutdown is over by then!)

As the ratings system is being developed, Secretary Duncan and his staff must make a number of important decisions:

(1) Do they push for ratings to be tied to federal financial aid (requiring Congressional authorization), or should they just be made available to the public as one of many information sources?

(2) Should they be designed to highlight the highest-performing colleges, or should they call out the lowest-performing institutions?

(3) Should public, private nonprofit, and for-profit colleges be held to separate standards?

(4) Should community colleges be included in the ratings?

(5) Will outcome measures be adjusted for student characteristics (similar to the value-added models often used in K-12 education)?

After these decisions have been made, then the Department of Education can focus on selecting possible outcomes. Graduation rates and student loan default rates are likely to be a part of the college ratings, but what other measures could be considered—both now and in the future? An expanded version of gainful employment, which is currently used for vocationally-oriented programs, is certainly a possibility, as is some measure of earnings. These measures may be subject to additional legal challenges. Some measure of cost may also make its way into the ratings, rewarding colleges that operate in a more efficient manner.

I would like to hear your thoughts (in the comments section below) about whether these ratings are a good idea and what measures should be included. And when the Department of Education starts accepting comments on the ratings, likely sometime in 2014, I encourage you to submit your thoughts directly to them!