New Experimental Evidence on the Effectiveness of Need-Based Financial Aid

My first experience doing higher education research began in the spring 2008, when I (then a graduate student in economics) responded to an e-mail from an education professor at the University of Wisconsin who was looking for students to help her with an interesting new study. Sara Goldrick-Rab was co-leading an evaluation of the Wisconsin Scholars Grant (WSG)—a rare case of need-based financial aid being given to students from low-income families via random assignment. Over the past decade, the Wisconsin Hope Lab team published articles on the effectiveness of the WSG in improving on-time graduation rates among university students and on changing students’ work patterns.

A decade later, we were able to conduct a follow-up study to examine the outcomes of treatment and control group students who started college between 2008 and 2011. This sort of long-term analysis of financial aid programs has rarely been conducted—and the two best existing evaluations (of the Cal Grant and the West Virginia PROMISE program) are on programs with substantial merit-based components. Eligibility for the WSG was solely based on financial need (conditional on being a first-time, full-time student), providing the first long-term experimental evaluation of a need-based program.

Along with longtime collaborators from our days in Wisconsin (Drew Anderson of the RAND Corporation, Katharine Broton of the University of Iowa, and Sara Goldrick-Rab of Temple University), I am pleased to announce the release of our new working paper on the long-term effects of the WSG to kick off the opening of the new Hope Center for College, Community and Justice at Temple University. We found some evidence that students who began at four-year colleges who were assigned to receive the WSG had improved academic outcomes. The positive impacts on degree completion for the initial cohort of students in 2008 did fade out over a period of up to nine years, but the grant still helped students complete their degrees more quickly than the comparison group. Additionally, there was a positive impact on six-year graduation rates in later cohorts, with treatment students in the 2011 cohort being 5.4 percentage points more likely to graduate than the control group.

The grant generated clear increases in the percentage of students who both declared and completed STEM majors, even though the grant made no mentions whatsoever of STEM and had no major requirements. A second new paper by Katharine Broton and David Monaghan of Shippensburg University found that university students assigned to treatment were eight percentage points more likely to declare a STEM major, while our paper estimated a 3.6 percentage point increase in the likelihood of graduating with a STEM major. This strongly suggests that additional need-based financial aid can free students to pursue a wider range of majors, including ones that may require more expensive textbooks and additional hours spent in laboratory sessions.

However, the WSG did not generate across-the-board positive impacts. Impacts on persistence, degree completion, and transfer for students who began at two-year colleges were generally null, which could be due to the smaller size of the grant ($1,800 per year at two-year colleges versus $3,500 at four-year colleges) or the rather unusual population of first-time, full-time students attending mainly transfer-focused two-year colleges. We also found no effects of the grant on graduate school enrollment among students who started at four-year colleges, although this trend is worth re-examining in the future as people may choose to enroll after several years of work experience.

It has been an absolute delight to reunite with my longstanding group of colleagues to conduct this long-term evaluation of the WSG. We welcome any comments on our working paper and look forward to continuing our work in this area through the Hope Center.

The Vast Array of Net Price Calculators

Net price calculators are designed to give students and their families a clear idea of how much college will cost them each year after taking available financial aid into account. All colleges have to post a net price calculator under the Higher Education Opportunity Act of 2008, but these calculators take a range of different form. The Department of Education has proposed a standardized “shopping sheet” which has been adopted by some colleges, but there is still a wide amount of variation in net price calculators across institutions. This is shown in a 2012 report by The Institute for College Access and Success, using 50 randomly selected colleges across the country.

In this blog post, I examine net price calculators from six University of Wisconsin System institutions for the 2013-14 academic year. Although these colleges might be expected to have similar net price calculators and cost assumptions, this is far from the case as shown in the below screenshots.  In all cases, I used the same student conditions—an in-state, dependent, zero-EFC student.

Two of the six colleges selected (the University of Wisconsin Colleges and UW-La Crosse) require students to enter several screens of financial and personal information in order to get an estimate of their financial aid package. While that can be useful for some students, there should be an option to directly enter the EFC for students who have filed the FAFSA or are automatically eligible for a zero EFC. For the purposes of this post, I stopped there with those campuses—as some students may decide to do.

(UW Colleges and UW-La Crosse, respectively)

UW Colleges Net Price Calculator

La Crosse Net Price Calculator

UW-Milwaukee deserves special commendation for clearly listing the net price before mentioning loans and work-study. Additionally, they do not list out each grant a student could expect to receive, simplifying the information display (although this does have its tradeoffs).

Milwaukee Net Price Calculator

The other three schools examined (Eau Claire, Madison, and Oshkosh) list out each type of financial aid and present an unmet need figure (which can be zero) before reporting the estimated net price of attendance. Students may read these calculators and think that no borrowing is necessary in order to attend college, while this is not the case. The net price should be listed first, since this tool is a net price calculator.

(UW-Eau Claire, UW-Madison, and UW-Oshkosh, respectively)

Eau Claire Net Price Calculator

Madison Net Price CalculatorOshkosh Net Price Calculator

The net price calculators also differ in their terminologies for different types of financial aid. For example, UW-Eau Claire calls the Wisconsin Higher Education Grant the “Wisconsin State Grant,” which appears nowhere else in the information students receive. The miscellaneous and travel budgets vary by more than $1000 across the four campuses with net price calculators, highlighting the subjective nature of these categories. However, they are very important to students because they cannot receive more in financial aid than their total cost of attendance. If colleges want to report a low net price, they have incentives to report low living allowances.

I was surprised to see the amount of variation in net price calculators across UW System institutions. I hope that financial aid officers and data managers from these campuses can continue to work together to refine best practices and present a more unified net price calculator.

New Recommendations for Performance-Based Funding in Wisconsin

Performance-based funding for Wisconsin’s technical colleges is at the forefront of Governor Walker’s higher education budget for the next biennium. In previous blog posts (here, here, and here), I have briefly discussed some of the pros and cons of moving to a performance-based funding model for a diverse group of postsecondary institutions.

This week, Nick Hillman, Sara Goldrick-Rab, and I released a policy brief with recommendations for performance-based funding in Wisconsin through WISCAPE. In the brief, we discuss how performance-based funding has operated in other states, as well as recommendations for how to operate PBF in Wisconsin. Our key points are the following:

(1) Performance-based funding seeks to switch the focus from enrollment to completion.

(2) Successful performance-based funding starts small and is developed via collaboration.

(3) Colleges with different missions should have different performance metrics.

(4) Multiple measures of success are necessary to reduce the possibility of perverse incentives.

Wisconsin’s proposal appears to meet some of these key points, but some concerns do remain. My primary concern is the speed with which funding will shift to performance—from 10% in 2014-15 to 100% by 2019-20. This may not be enough time for colleges to adjust their actions, so this timeline should be adjusted as needed.

Technical Colleges Debate Tying Funding to Job Placement

In advance of Wisconsin Governor Scott Walker’s budget address tomorrow evening, last week’s release of plans to tie state funding for technical colleges to performance measures has generated a great deal of discussion. One of the most discussed portions of his plan (press release here) is his proposal to tie funding to job placement rates, particularly in high-demand fields. Most colleges seem to support the idea of getting better data on job placement rates, but using that measure in an accountability system has sparked controversy.

Madison Area Technical College came out last week in opposition to the Governor’s proposal, as covered by a recent article in the Capital Times. The article mentions comments by provost Terry Webb that job placement rates are partially influenced by factors outside the college’s control, such as job availability, location, and individual preferences. These concerns are certainly real, especially given the difficulty of tracking students who may leave the state in search of a great job opportunity.

However, Gateway Technical College came out in support of funding based on job placement rates, according to an article in the Racine Journal Times (hat tip to Noel Radomski for the link). Gateway president Bryan Albrecht supports the plan on account of the college’s high job placement rates among graduates (85%, among those who responded to a job placement survey with a 78% response rate, although only 55% were employed in their field of study). The college seems confident in its ability to change programs as needed in order to keep up with labor market demands, even in the face of a difficult economy in southeast Wisconsin.

The differing reactions of these two technical colleges show the difficulty of developing a performance-based funding system which works for all stakeholders. Madison College, along with three other technical colleges in the state, has liberal arts transfer programs with University of Wisconsin System institutions. These students may graduate with an associate’s degree and not immediately enter the labor market, or even successfully transfer before getting the degree. The funding system, which will be jointly developed by the Wisconsin Technical College System and the state’s powerful Department of Administration, should keep those programs in mind so to not unfairly penalize students with dual vocational/transfer missions.

More on Wisconsin’s Workforce Development Proposal

Today, Wisconsin Governor Scott Walker released more information about his proposal to improve the state’s workforce development system through an additional $100 million in state appropriations. These proposals have the potential to affect the priorities of Wisconsin institutions of higher education, particularly the Wisconsin Technical College System. While most of the key points of the proposal are directly from his special workforce development commission’s report last August (see my analyses here and here), the additional details provided in this press release provide more concrete information about the Governor’s soon-to-be-released budget proposal.

Three items in Gov. Walker’s proposal are in legislation separate from the state budget: workforce training grants to a mix of colleges, businesses and economic development organizations, a new Office of Skills Development to administer the grants, and a labor market information system designed to help link students and workers to available jobs and track labor market trends. The labor market information system has the potential to provide high school and college students with information that can help them decide their course of study, but getting the information to students in a timely manner may be difficult. It can be a useful tool for high school juniors who want to figure out a possible career, but it may be four or five years before the student is ready to go into the workforce. A lot can happen in that period of time. In any case, these records should be linked to K-12 and higher education datasets so the effectiveness of the new system can be evaluated.

The big change in higher education policy comes from the proposed shift to performance-based funding (PBF) in the Wisconsin Technical College System. Under PBF, colleges are funded based on outcomes (such as graduation and job placement rates) instead of based on enrollment or other historical factors. This plan starts with 10% of base funding being used for PBF in 2014-15, rising to 100% by 2020. Although other states have similar plans to completely shift to PBF, I am skeptical that a majority of funding will ever be tied to performance for political reasons. (Note that if Gov. Walker serves a second term and declines to run for a third, he would leave office in January of 2019—before this takes effect.)

Few details are currently available about the proposed funding formula for WTCS, as it will be developed by WTCS and the state Department of Administration. But the press release does note that the formula will prioritize job placement and enrollment in high-demand programs, something which is likely to be opposed by WTCS campuses with strong university transfer programs (such as Madison Area Technical College). These concerns will likely be kept in mind as a PBF system is developed.

Finally, the press release calls for the development of a common core of 30 credits (approximately ten courses) that will be fully transferrable across the UW System, WTCS, and participating Wisconsin private colleges. This will likely be opposed by a number of UW System universities as a loss of autonomy and a perceived lowering of academic standards. I would expect the common core to be mandated, but some colleges will attempt to deny full transferability of certain courses; for example, a college algebra class at a technical college might be classified as an elective math credit at a UW System university instead of as a college algebra class.

Governor Walker’s budget address will take place on February 20, and I will have a complete analysis of his higher education programs later this week. More details may be released before that time, such as in this unusual Sunday press release.

Wisconsin Higher Education Policy Issues for 2013

2013 marks a potential benchmark year for state higher education policy debates. More tends to happen in odd-numbered years because politicians are farther away from elections and more willing to make difficult budget decisions—and the influx of federal stimulus dollars is rapidly drying up. In Wisconsin, 2013 is a particularly important year as discussions begin on the state’s biennial budget. The American Association of State Colleges and Universities, an association representing primarily non-flagship public four-year schools, has released its list of the top ten state policy issues for 2013. They are the following:

(1)    Increasing college performance

(2)    Funding for public colleges and universities

(3)    Tuition prices and policy

(4)    State grant aid programs

(5)    Academic preparation for college

(6)    Immigration policy

(7)    Competency-based education

(8)    Concealed carry on campus

(9)    Workforce/economic development

(10) For-profit college regulation

Not all of these issues are a major concern in Wisconsin (such as whether to grant in-state concern to illegal immigrants who graduated from a Wisconsin public high school), are particularly relevant to student success (such as concealed carry regulations), or are likely to change much (tuition policy). My take on the five most important issues facing the Wisconsin Legislature in 2013 are the following:

Priority #1: Workforce and economic development

Although many in the academic community might disagree with how I have these key issues ordered, the Legislature is clearly focused on workforce and economic development. I expect a focus on vocational and technical education in 2013, as outlined in an August 2012 report by Tim Sullivan, special consultant on economic, workforce, and education development. I’ve written about this report in a previous blog post; overall, the key points in the proposal are reasonable, as long as the Legislature doesn’t go off on a tangent regarding immigration policy or setting unreasonable expectations.

Priority #2: Increasing college performance

Legislation was passed in the previous session that required colleges to make certain accountability information public. (I analyzed UW-Madison’s 2012 report in a post last August.) This legislation didn’t really have any teeth in terms of changing a university’s funding level. This looks very likely to change in 2013, as performance-based funding is going to be a key point of discussion. As Gov. Walker outlined in a speech last fall, he is pushing for some of the higher education funding to be based on a college’s performance in key areas, such as graduation rates and possibly enrolling Pell Grant recipients. I’ll have much more to say about performance-based funding in future blog posts, but for now I will emphasize the importance of using some sort of value-added measure as part of the performance score. (I’ve written quite a bit on this in the past, as well.)

Priority #3: Competency-based education

Wisconsin has become a leader in competency-based education in specialized degree programs, allowing students to earn credit for prior knowledge in certain areas. Unlike some states, which are contracting with the not-for-profit Western Governors University, Wisconsin is doing their effort in-house through the University of Wisconsin System. This experiment will be watched closely around the nation to see whether students take up the program in meaningful numbers as well as whether it will be cost-effective.

Priority #4: State grant aid programs

In 2012, the Legislature tasked the Higher Education Aids Board, the state’s agency administering need-based and merit-based grant programs, with exploring ways to consolidate and modernize the state’s financial aid system. The report, released in December, failed to suggest any meaningful changes that would help ensure a more reasonable distribution of financial aid to students. I hope that the Legislature will reconsider ways to reduce the number of separate need-based grants in order to have a more streamlined and student-friendly aid system, but I am not terribly optimistic.

Priority #5: Funding for public colleges and universities

After several rough budget cycles, Wisconsin looks to be in reasonable fiscal health entering the 2013-15 biennium. As such, Wisconsin higher education is requesting a funding increase over the 2011-13 cycle. The University of Wisconsin System is requesting a $224 million increase (1.9%), while the Wisconsin Technical College System is requesting an additional $92 million (a 31.6% increase). Most of the requested increases for the UW System are designated for meeting the accountability goals, while most of WTCS’s requested increases are designated for meeting workforce shortages in high-demand occupations. These requested increases show the importance of the top two priorities on my list to Wisconsin legislators.

 

I expect 2013 to be a much calmer year in Wisconsin politics than the past several years, but no less important to the higher education community. Hopefully, the state will continue to make progress in meeting key performance goals and fostering student success.

Transparency and Teacher Education Programs

I am a firm believer in the public’s right to know nearly everything about government-funded institutions unless there is a clear and compelling reason for privacy. For that reason, I have been following the University of Wisconsin System’s fight against the National Council on Teacher Quality (NCTQ), a group seeking to make information on the standards of teacher education programs public. In conjunction with U.S. News and World Report, NCTQ is compiling course syllabi, textbooks, student handbooks, and other information to rate education schools based on whether they are adequately preparing future K-12 teachers for their professions.

This review process has been objected to by many public colleges and universities (the full list is here) on the grounds that the proposed methodology is inadequate for rating colleges. (Yet these same colleges boast about their U.S. News rankings in other aspects, although the rankings are just as flawed.)The University of Wisconsin System has long refused to cooperate with NCTQ on this, as evidenced by their March 2011 letter to NCTQ.

Yet the UW System and many other public universities are failing the public trust by refusing to make important information produced by public employees available at a reasonable cost. The Wisconsin Institute for Law and Liberty, a Milwaukee-based public interest law firm, sued the UW System last January on behalf of NCTQ to get the records turned over. WILL’s suit was ultimately successful in obtaining its objective, as the UW System agreed to turn over the relevant materials and pay WILL nearly $10,000 in damages and fees after obtaining additional privacy assurances.

Wisconsin taxpayers and students will foot the bill for the UW System’s initial refusal to make information public under open records laws. This is a big PR mistake for Wisconsin higher education, as it gives the appearance that universities think they are above accountability—this isn’t a good thing in the current political climate, to say the least.

Now on to the meat of the new rankings, which should come out sometime this year. There are 17 standards which will be a part of the rankings, centered on four areas:

(1)    Selectivity of teacher education programs and students’ incoming academic characteristics

(2)    Teacher knowledge of subject matter

(3)    Classroom management and student teaching skills

(4)    Outcomes of graduates’ future classes on state tests

As regular readers of this blog know, I’m not a fan of the selectivity criterion. If a college does a good job of training teachers, who cares about their ACT score? But the other three measures are certainly important; the question is whether the available data will be sufficient to accurately rate programs and provide stakeholders with useful information.

I expect a big fuss when these ratings are released, just like there is a big fuss whenever the U.S. News undergraduate rankings are released every fall. While I’m concerned about the ability to draw conclusions from available data, these ratings will provide information about whether institutions are collecting relevant types of data (such as their graduates’ outcomes) and certainly won’t be any worse than the peer rating part of the undergraduate rankings that has existed for nearly three decades.

Streamlining Financial Aid in Wisconsin

The Wisconsin Higher Educational Aids Board, the state’s agency administering need-based and merit-based financial aid programs, was recently tasked with forming a commission on financial aid consolidation and modernization. The commission had two primary charges:

(1)    Explore consolidating all state need-based grants into one program.

(2)    Study options for providing grants to students attending college less than half-time.

The current system of need-based grants has separate grants for four different sectors of Wisconsin higher education: the University of Wisconsin System (UWS), the Wisconsin Technical College System (WTCS), the state’s tribal colleges, and the private, non-profit sector (WAICU).  Sadly, HEAB’s final report, which was recently released, failed to streamline the complicated financial aid system in Wisconsin. Each of the four sectors’ grants currently has separate pools of funding, and the report encourages this practice to continue.

The current system of awarding grants by sector needs to be revamped. Buried on page 42 of the report is the current distribution of funding by sector:

Sector Num. Eligible Awarded (%) Spent ($) Unfunded ($) Max Award ($)
UW System 43,808 70.1 58,321,266 32,922,506 2,384
WTCS 74,284 26.2 18,326,312 63,835,738 1,084
Tribal 1,204 26.0 441,963 1,593,276 1,800
Privates 17,935 58.6 26,613,208 23,291,709 2,900
Total 137,231 44.4 103,702,749 121,643,229  

 

This distribution makes absolutely no sense, in both the percent of eligible students awarded grant money (due to budget constraints) and the maximum award. I can’t speak to the needs of students attending the tribal colleges due to my lack of knowledge of these institutions and the students’ other financial aid awards, but it seems logical to have the same percentage of students receive need-based aid across systems. Given the lower cost of tuition for the technical colleges, I can see why they are receiving smaller grants.

I also don’t see a compelling reason for the state to give more aid to students attending private colleges than those attending public colleges. It is true that the state saves money if a student attends a private college (by being able to appropriate less money for the public sector), but I seriously doubt that students will change their decision to attend a private college if their grant aid is cut by about $500. This is especially the case since some students attending private colleges can receive need-based aid even if they are ineligible for the federal Pell Grant, which is not the case for public colleges.

The report also called for the status quo regarding the lack of eligibility for state grants if a student attended college less than half-time (five or fewer credits per semester). This would only be reversed if each sector supported changing the eligibility rules, sufficient funding became available, and HEAB had additional staff to monitor the additional students, conditions which are unlikely to be met anytime soon.

In my view, the commission completely failed to respond to its charge as little was done to streamline financial aid in Wisconsin or fix persistent inequities in the funding system. The Legislature should seriously consider combining all need-based grant programs into one pot even though the stakeholders on the committee disagree.

Need-Based Financial Aid and College Persistence: Experimental Evidence from Wisconsin

This afternoon, the Institute for Research on Poverty at the University of Wisconsin-Madison released our new paper on the effects of the Wisconsin Scholars Grant on Pell Grant recipients attending public universities in the state of Wisconsin. In short, we find that randomly assigning some students additional financial aid yields modest positive effects on college retention rates, but getting to that result has been extremely complicated.

Read the paper here and let me know what you think!  We have submitted this paper to a journal and hope to get good news in the next couple months.

Right Idea, Wrong Time

It’s election season once again, so President Obama is coming back to Madison for a large campaign event right smack dab in the middle of the University of Wisconsin-Madison campus Thursday afternoon. Given the amount of security required to host a Presidential visit (regardless of the purpose), it is not surprising that all of the buildings on Bascom Hill will be closed on Thursday. This campaign rally will require all classes in affected buildings to be moved—many of them will likely be cancelled despite this being midterm exam season for undergraduates.

I am always happy to have politicians come to campus to ask for the community’s support, but two things just grate me the wrong way about the visit. The first thing is the timing. When Obama came to campus the previous two times (February 2008 and September 2010), his events were scheduled later in the day. While classes were still moved from the immediate area of Bascom Hill for the 2010 visit, the rally was held later in the afternoon so more classes could be held. Ann Althouse, prominent blogger and faculty member in the UW Law School, isn’t too happy about the class disruption:

“Nice for the campaign, but positioned to maximize disruption of regular classes. Is that a bug or a feature? If there are no classes and it’s a class day, students are around and they are free to attend. Classes are being cancelled to supply the photogenic crowd for the President?”

Badgers are a pretty photogenic lot. (It’s hard to be humble when you’re from Wisconsin, after all.) But starting the event at, say, 4 PM instead of noon would allow for a much more normal day of classes. For reference, recall the hubbub about having a night football game on the Thursday before classes even started. I’m guessing that the folks complaining about a night football game aren’t complaining about the President’s campaign stop—I’m happy to complain about both.

I have one more gripe about the rally: in order to get into the event in the heart of campus, people have to register with the President’s campaign team. I don’t have any problems with metal detectors and tight security (there are plenty of crazy people out there), but requiring registration with an aggressive political campaign team to attend an on-campus event does not support sifting and winnowing. (To be fair, Romney’s folks do the same thing to harvest voter information—but he is never coming to far-left Madison.)

I have taken steps to cancel or postpone all of my events on campus on Thursday and will likely listen to the rally online. Hopefully, all of the people displaced by the campaign event can have a fairly normal day of work if they so choose.