New Recommendations for Performance-Based Funding in Wisconsin

Performance-based funding for Wisconsin’s technical colleges is at the forefront of Governor Walker’s higher education budget for the next biennium. In previous blog posts (here, here, and here), I have briefly discussed some of the pros and cons of moving to a performance-based funding model for a diverse group of postsecondary institutions.

This week, Nick Hillman, Sara Goldrick-Rab, and I released a policy brief with recommendations for performance-based funding in Wisconsin through WISCAPE. In the brief, we discuss how performance-based funding has operated in other states, as well as recommendations for how to operate PBF in Wisconsin. Our key points are the following:

(1) Performance-based funding seeks to switch the focus from enrollment to completion.

(2) Successful performance-based funding starts small and is developed via collaboration.

(3) Colleges with different missions should have different performance metrics.

(4) Multiple measures of success are necessary to reduce the possibility of perverse incentives.

Wisconsin’s proposal appears to meet some of these key points, but some concerns do remain. My primary concern is the speed with which funding will shift to performance—from 10% in 2014-15 to 100% by 2019-20. This may not be enough time for colleges to adjust their actions, so this timeline should be adjusted as needed.

On MOOCs and Money

Massively open online courses (MOOCs) have become the newest fashionable trend in higher education. These courses, which are open to anyone and can cover anything from songwriting to combinatorial game theory (which sounds both fun and exceedingly challenging), have begun to gain recognition among policymakers and the public alike. The American Council on Education announced that five MOOCs from Coursera would be recommended for college credit, hastening the move into the technology.

The University of Wisconsin-Madison announced last night that its faculty would offer four MOOCs in conjunction with Coursera as a part of the university’s Educational Innovations program. While the MOOCs (in video games and learning, higher education globalization, evolution, and economics/finance) would not currently be offered for credit, the potential certainly exists for credit in the future. These courses could be a part of the Flexible Option program through the University of Wisconsin System, which has gained additional support in Governor Walker’s new budget.

With all of the potential promise of MOOCS to help students get access to higher education, there are still many concerns to be addressed. Coursera recently had to call off a MOOC on the fundamentals of online education, as the technology wasn’t ready for 40,000 students. Issues of access are also important with MOOCs, as they appeal to students who prefer more independent learning and are ready to handle that sort of delivery concern. We also know little about whether MOOCs are effective in promoting student learning, both compared to an in-person class or even to nothing at all.

From a university’s perspective, MOOCs present both promises and pitfalls. If a university can develop a successful MOOC (in the sense of gaining public support), it is a potential way to increase funding through either state appropriations (as could be the case in Wisconsin) or donations from satisfied students. If a few entry-level courses (such as UC-Irvine’s offerings of algebra and pre-calculus) were to be offered for credit, it could serve as a potential way for students to select colleges or familiarize themselves with higher-level coursework.

Current university students (and faculty) should be concerned about where the faculty time for developing these MOOCs comes from. Many faculty at research universities (the ones who are likely to develop MOOCs) are teaching one or two courses per semester in an environment where teaching is valued less than research. These courses would have to be developed on a professor’s own time or count as part of the service component—it is essential that teaching loads not be reduced for developing MOOCs unless the university is somehow compensated. An option for compensation is to have foundations help fund initial course development in the form of faculty buyouts.

I am glad that the University of Wisconsin-Madison is starting small with MOOCs, as these courses have potential to help improve student learning on the margin at this point in time. If a college can develop a MOOC for an entry-level class that can be cost-effective, I’ll be a lot more optimistic. But for right now, it’s a neat way to see new research in specialized fields and could certainly be a way for advanced undergraduate students to take an “elective” course in their field of study. I am waiting for more research before fully jumping on the MOOC bandwagon.

Technical Colleges Debate Tying Funding to Job Placement

In advance of Wisconsin Governor Scott Walker’s budget address tomorrow evening, last week’s release of plans to tie state funding for technical colleges to performance measures has generated a great deal of discussion. One of the most discussed portions of his plan (press release here) is his proposal to tie funding to job placement rates, particularly in high-demand fields. Most colleges seem to support the idea of getting better data on job placement rates, but using that measure in an accountability system has sparked controversy.

Madison Area Technical College came out last week in opposition to the Governor’s proposal, as covered by a recent article in the Capital Times. The article mentions comments by provost Terry Webb that job placement rates are partially influenced by factors outside the college’s control, such as job availability, location, and individual preferences. These concerns are certainly real, especially given the difficulty of tracking students who may leave the state in search of a great job opportunity.

However, Gateway Technical College came out in support of funding based on job placement rates, according to an article in the Racine Journal Times (hat tip to Noel Radomski for the link). Gateway president Bryan Albrecht supports the plan on account of the college’s high job placement rates among graduates (85%, among those who responded to a job placement survey with a 78% response rate, although only 55% were employed in their field of study). The college seems confident in its ability to change programs as needed in order to keep up with labor market demands, even in the face of a difficult economy in southeast Wisconsin.

The differing reactions of these two technical colleges show the difficulty of developing a performance-based funding system which works for all stakeholders. Madison College, along with three other technical colleges in the state, has liberal arts transfer programs with University of Wisconsin System institutions. These students may graduate with an associate’s degree and not immediately enter the labor market, or even successfully transfer before getting the degree. The funding system, which will be jointly developed by the Wisconsin Technical College System and the state’s powerful Department of Administration, should keep those programs in mind so to not unfairly penalize students with dual vocational/transfer missions.

Should Campuses be Able to Limit Student Loans?

The National Association of Student Financial Aid Administrators jumped into the financial aid reform debate this week with the release of their policy paper as a part of the Gates Foundation’s Reimagining Aid Delivery and Design (RADD) project. Many of the recommendations are similar to other papers in the panel (including proposals to increase the maximum Pell Grant for certain students and providing more information for students and their families to make better college decisions)—and an additional recommendation of exploring an early commitment program for Pell recipients is informed by some of my research, which is pretty nifty.

The NASFAA report does make one recommendation which will likely prove to be highly controversial—limiting eligibility for student loans for certain groups of students in a clear effort to reduce student loan default rates. First, NASFAA suggests that students who do not meet a baseline level of academic preparation (perhaps a combination of ACT/SAT scores and high school GPA) would not be initially eligible to take out federal student loans. This proposal would be similar to the academic eligibility index used by the NCAA to determine student-athletes’ ability to play college sports. This proposal could have the effect of ending the open-access institution as we know it, depending on exactly where the cutoff is set. While it is true that students with lower standardized test scores are less likely to complete college, I’m very hesitant to place a substantial barrier to college entry—especially for students who did not enroll in college directly after completing high school.

The report also contains a recommendation allowing colleges to restrict groups of students’ ability to borrow if the financial aid officer feels that the loan funds are not needed or risky. For example, education majors’ loans may be limited compared to business majors because of their lower annual earnings (and reduced repayment abilities). Restricting access to loans by program characteristics (instead of individual characteristics) reduces the burden on financial aid officers, but also fails to take individual characteristics into account unless a student appeals for professional judgment.

The proposal to limit student loans will penalize students who cannot pay for college by any other means—especially for dependent students who cannot get parental support to pay for their expected family contribution. Additionally, many students cannot borrow the maximum amount of loans under current rules, which base eligibility in part on the estimated cost of attendance. Research suggests that this posted cost of attendance may be much lower than the actual cost of attending college, as institutions have an incentive to make the college look as affordable as possible.

While I am concerned about these particular portions of NASFAA’s proposal, they raise concerns that are of genuine merit and concern in the financial aid and policy communities. I would be surprised if they become a part of federal rules in any meaningful way, but this does show the diversity of opinions within the RADD group and the importance of listening to as many stakeholders as possible before redesigning the financial aid system.

More on Wisconsin’s Workforce Development Proposal

Today, Wisconsin Governor Scott Walker released more information about his proposal to improve the state’s workforce development system through an additional $100 million in state appropriations. These proposals have the potential to affect the priorities of Wisconsin institutions of higher education, particularly the Wisconsin Technical College System. While most of the key points of the proposal are directly from his special workforce development commission’s report last August (see my analyses here and here), the additional details provided in this press release provide more concrete information about the Governor’s soon-to-be-released budget proposal.

Three items in Gov. Walker’s proposal are in legislation separate from the state budget: workforce training grants to a mix of colleges, businesses and economic development organizations, a new Office of Skills Development to administer the grants, and a labor market information system designed to help link students and workers to available jobs and track labor market trends. The labor market information system has the potential to provide high school and college students with information that can help them decide their course of study, but getting the information to students in a timely manner may be difficult. It can be a useful tool for high school juniors who want to figure out a possible career, but it may be four or five years before the student is ready to go into the workforce. A lot can happen in that period of time. In any case, these records should be linked to K-12 and higher education datasets so the effectiveness of the new system can be evaluated.

The big change in higher education policy comes from the proposed shift to performance-based funding (PBF) in the Wisconsin Technical College System. Under PBF, colleges are funded based on outcomes (such as graduation and job placement rates) instead of based on enrollment or other historical factors. This plan starts with 10% of base funding being used for PBF in 2014-15, rising to 100% by 2020. Although other states have similar plans to completely shift to PBF, I am skeptical that a majority of funding will ever be tied to performance for political reasons. (Note that if Gov. Walker serves a second term and declines to run for a third, he would leave office in January of 2019—before this takes effect.)

Few details are currently available about the proposed funding formula for WTCS, as it will be developed by WTCS and the state Department of Administration. But the press release does note that the formula will prioritize job placement and enrollment in high-demand programs, something which is likely to be opposed by WTCS campuses with strong university transfer programs (such as Madison Area Technical College). These concerns will likely be kept in mind as a PBF system is developed.

Finally, the press release calls for the development of a common core of 30 credits (approximately ten courses) that will be fully transferrable across the UW System, WTCS, and participating Wisconsin private colleges. This will likely be opposed by a number of UW System universities as a loss of autonomy and a perceived lowering of academic standards. I would expect the common core to be mandated, but some colleges will attempt to deny full transferability of certain courses; for example, a college algebra class at a technical college might be classified as an elective math credit at a UW System university instead of as a college algebra class.

Governor Walker’s budget address will take place on February 20, and I will have a complete analysis of his higher education programs later this week. More details may be released before that time, such as in this unusual Sunday press release.

Another Random List of “Best Value” Colleges

Getting a good value for attending college is on the mind of most prospective students and their families, and as a result, numerous publishers of college rankings have come out with lists of “best value” colleges. I have highlighted the best value college lists from Kiplinger’s and U.S. News in previous posts, as well as discussing my work incorporating a cost component into Washington Monthly’s rankings. Today’s entry in this series comes from the Princeton Review,  a company better known for test preparation classes and private counseling, but they are also in the rankings business.

The Princeton Review released its list of its “Best Value Colleges” today in conjunction with USA Today, and the list is heavily populated with a “who’s who” list of selective, wealthy colleges and universities. Among the top ten private colleges, several of them are wealthy enough to be able to waive all tuition and fees for their few students from modest financial backgrounds. The top ten public institutions do tend to attract a fair number of out-of-state and full-pay students, although there is one surprise name on the list (North Carolina State University—well done!). More data on the top 150 colleges can be found here.

My main complaint with this ranking system, as with other best value colleges lists, is with the methodology. They begin by narrowing their sample from about 2,000 colleges to 650—what they call “the nation’s academically best undergraduate institutions.” This effectively limits the utility of these rankings to students who score a 25 or higher on the ACT, or even higher if students wish to qualify for merit-based grant aid. Student selectivity is further awarded in the academic rating, even though this has no guarantee of future academic performance. Much of the academic and financial aid ratings measures come from student surveys, which are fraught with selection bias. Basically, many colleges handpick the students who take these surveys, which results in an optimistic set of opinions being registers. I wish I could say more about their methodology and point values, but no information is available.

The top 150 list (which can be found here by state) certainly favors wealthy, prestigious colleges with a few exceptions (University of South Dakota, University of Tennessee-Martin, and Southern Utah University, for example). In Wisconsin, only Madison and Eau Claire (two of the three most selective universities in the UW System) made the list. In the Big Ten, there are some notable omissions—Iowa (but Iowa State is included), Michigan State (but Michigan is included), Ohio State, and Penn State.

The best value rankings try to provide information about what college will cost, and whether some colleges provide better “bang for the buck” than others. Providing useful information is an important endeavor, as this recent article in the Chronicle emphasizes. However, the Princeton Review’s list provides useful information to only a small number of academically elite students, many of whom have the financial means to pay for college without taking on much debt. This is illustrated by the accompanying USA Today article featuring the rankings, which notes that fewer than half of all students attending Best Value Colleges take on debt, compared to two-thirds of students nationwide. This differential isn’t just a result of the cost of attendance, but instead the student’s ability to pay for college.

Bill Gates on Measuring Educational Effectiveness

The Bill and Melinda Gates Foundation has become a very influential force in shaping research in health and education policy over the past decade, both due to the large sums of money the foundation has spent funding research in these areas and because of the public influence that someone as successful as Bill Gates can have. (Disclaimer: I’ve worked on several projects which have received Gates funding.) In both the health and education fields, the Gates Foundation is focusing on the importance of being able to collect data and measure a program’s effectiveness. This is evidenced by the Gates Foundation’s annual letter to the public, which I recommend reading.

In the education arena, the Gates letter focuses on creating useful and reliable K-12 teacher feedback and evaluation systems. They have funded a project called Measures of Effective Teaching, which finds some evidence that it is possible to measure teacher effectiveness in a repeatable manner that can be used to help teachers improve. (A hat tip to my friend Trey Miller, who worked on the report.) To me, the important part of the MET report is that multiple measures of teacher effectiveness, including evaluations, observations, and student scores, need to be used when consider teaching effectiveness.

The Gates Foundation is also moving into performance measurement in higher education. I have been a part of one of Gates’s efforts in this arena—a project examining best practices in input-adjusted performance metrics. What this essentially means is that colleges should be judged based on some measure of their “value added” instead of the raw performance of their students. Last week, Bill Gates commented to a small group of journalists that college rankings are doing the exact opposite (as reported by Luisa Kroll of Forbes):

“The control metric shouldn’t be that kids aren’t so qualified. It should be whether colleges are doing their job to teach them. I bet there are community colleges and other colleges that do a good job in this area, but US News & World Report rankings pushes you away from that.”

The Forbes article goes on to mention that Gates would like to see metrics that focus on the performance of students from low-income families and the effectiveness of teacher education programs. Both of these measures are currently in progress, and are likely to continue moving forward given the Gates Foundation’s deep pockets and influence.