More Information on the Education Week Article

I was happy to learn this morning that my research on value-added with respect to college graduation rates (with Doug Harris) was covered in an Education Week blog post by Sarah Sparks. While I am glad to get media coverage for this week, the author never reached out to me to make sure her take on the article was accurate. (I had a radio show in college and this was one of the things that was drilled into my head, so I am probably a little too sensitive regarding fact-checking.) As a result, there are some concerns with the Ed Week post that need to be addressed. My concerns are as follows:

(1)    The blog post states that we “analyzed data on six-year graduation rates, ACT or SAT placement-test scores and the percentage of students receiving federal need-based Pell grants at 1,279 colleges and universities from all 50 states from 2006-07 through 2008-09.” While that is true, we also used a range of other demographic and institutional measures in our value-added models. Using ACT/SAT scores and Pell Grants to predict graduation rates explains only about 60% of the variation in institutional graduation rates, while including the additional demographic measures that we use explains an additional 15% or so of the variation. The post should have briefly mentioned this, as it helps set our work apart from previous work (and particularly the U.S. News rankings).

(2)    After generating the predicted graduation rate and comparing it to the actual graduation rate, we adjust for cost in two different ways. In what we call the student/family model, we adjust for the net price of attendance (this is what I used in the Washington Monthly rankings this year). And in the policymaker model, we adjust for educational expenditures per full-time equivalent student. The blog post characterizes our rankings as “value-added rankings and popularity with families.” While the popularity with families is an accurate depiction of the student/family model, the term “value-added rankings” doesn’t reflect the policymaker model that well.

(3)    While we do present the schools in the top ten of our measures by Carnegie classification, we spend a great amount of time discussing the issues of confidence intervals and statistical significance. Even if a school has the highest value-added score, its score is generally not different from other high-performing institutions. We present the top-ten lists for illustrative purposes only and would encourage readers not to consider the lists to be perfect.

As an aside, there are five other papers in the Context for Success working group which also examine measuring college value-added that were not mentioned in the article, plus an outstanding literature review by Tom Bailey and Di Xu. I highly recommend reading through the summaries of those articles to learn more about the state of research in this field.

UPDATE (10/29): I had a wonderful e-mail conversation with the author and the above points have now been addressed. Chalk this up as another positive experience with the education press.

Need-Based Financial Aid and College Persistence: Experimental Evidence from Wisconsin

This afternoon, the Institute for Research on Poverty at the University of Wisconsin-Madison released our new paper on the effects of the Wisconsin Scholars Grant on Pell Grant recipients attending public universities in the state of Wisconsin. In short, we find that randomly assigning some students additional financial aid yields modest positive effects on college retention rates, but getting to that result has been extremely complicated.

Read the paper here and let me know what you think!  We have submitted this paper to a journal and hope to get good news in the next couple months.

Using Input-Adjusted Measures to Estimate College Performance

I have been privileged to work with HCM Strategists over the past two years on a Gates Foundation-funded project to explore how to use input-adjusted measures to estimate a college’s performance. Although the terminology sounds fancy, the basic goal of the project is to figure out better ways to measure whether a college does a good job educating the types of students that it actually enrolls. It doesn’t make any sense to measure a highly selective and well-resourced flagship university against an open-access commuter college; doing so is akin to comparing my ability to run a marathon with that of an elite professional athlete. Just like me finishing a marathon is a much more substantial accomplishment, getting a first-generation student with modest academic preparation to graduate is a much bigger deal than someone whom everyone expected to race through their coursework with ease.

The seven-paper project was officially unveiled in Washington on Friday, and I was able to make it out there for the release. My paper (joint work with Doug Harris) is essentially a policymaker’s version of our academic paper on the pitfalls of popular rankings. It’s worth a read if you want to find out more about my research beyond the Washington Monthly rankings.  Additional media coverage can be found in The Chronicle of Higher Education and Inside Higher Ed.

As a side note, it’s pretty neat that the Inside Higher Ed article links to the “authors” page of the project’s website (which includes my bio and information) under the term “prominent scholars.” I know I’m by no means a prominent scholar, but maybe some of that will rub off the others via association.

Improving Income-Based Repayment

As regular readers of this blog know, I am keenly interested in exploring the cost-effectiveness of policies affecting the world of education. This week, the New America Foundation released a report detailing changes made by Congress and the Obama Administration to income-based repayment. Income-based repayment allows people to pay back their student loans by paying a fixed percentage of their income over a long period of time; this differs from traditional loan payments in the sense that loan payments can do down if income is low and up if income is high.

The recent actions of the good folks in Washington resulted in a system that substantially reduces the payments for people who take on a lot of debt (generally those who attend very expensive colleges or get professional degrees). Giving heavy subsidies to high-income, well-educated people isn’t the most cost-effective strategy and encourages the cost of higher education to rise even higher.

I combined with Sara Goldrick-Rab, my friendly neighborhood dissertation chair and someone who occupies a distinctly different political space from me, to write a piece for The Chronicle of Higher Education on how to improve income-based repayment. Take a read and let me know what you think. The New America people are certainly interested in considering changes to their proposal, and so am I.

As an aside, it always feels nice to get some publicity for your thoughts, especially while navigating the job market. Stay tuned for my next endeavor…coming soon!

Is College Cost Certainty a Possibility?

There are only a few certainties in life, such as death, taxes, and Murphy’s Law holding true at the most inopportune times. For nearly everyone, however, knowing the cost of college more than a few months in advance is definitely not one of those certainties. But given the high sticker price of attending college (which is not tremendously useful for most people), what can be done to provide cost certainty for students and their families over the course of several years?

As a part of a nifty series of essays on possible ways to overhaul the college experience, Beckie Supiano of The Chronicle of Higher Education takes a quick look at what has been done to provide better cost information. She focuses on a useful goal—being able to provide students with the net price of education (tuition less grant aid) over the course of several years. While this is a great idea in theory, it quickly runs into several problems in practice:

(1)    Colleges only control a fraction of grant aid, especially for financially needy students. Most federal need-based grants require that a student be eligible to receive the Pell Grant; if family income rises by a small amount, the loss of aid can be devastating. This also makes forecasting net price nearly impossible for middle-income families.

(2)    Most colleges cannot forecast their available resources several years in advance. Public colleges are at the whims of economic circumstances and the state government, while many private colleges rely on a combination on endowment revenue and rear ends in seats in order to make ends meet.

(3)    Colleges, like most of us out there, tend to be what economists call risk-averse. In English, that means that we don’t like being exposed to uncertainty. Students and their families currently bear the brunt of the uncertainty with respect to higher education pricing, but locking in a price regardless of the economic circumstances would shift that risk to the college. Most colleges would likely set a very high initial price in order to account for this uncertainty.

A small number of states have experimented with guaranteed tuition plans through prepaid tuition programs, which helps provide at least some certainty (although they do not guarantee set amounts of financial aid). Alabama’s program ran out of money during the financial crisis and is currently closed to new enrollment while facing legal challenges. A similar program in Illinois is also drastically underfunded, which is a common theme in the Land of Lincoln.

The article makes a key mistake in discussing cost certainty—it ignores the fact that known cost increases still represent cost certainty. If a college guarantees that tuition will go up by five percent per year for the next five years, students and parents can still have an idea of what college will cost in the future. The State University of New York system took a similar path in 2011, allowing each university in the system to raise tuition by up to $300 per year for five years. This proposal is quite useful as it puts increases in dollar terms, which are easier to understand than percentages.

I am interested in providing more information on cost certainty, but my research focuses on the financial aid side of the cost equation instead of the tuition side. I am currently working on two studies examining whether students and their families can receive earlier notifications of their financial aid, with results hopefully to come in the next few months. It is far from perfect, but it does help provide a little more information in an uncertain world.

Overvaluing Harvard

Many parents went to send their children to what they consider to be the best colleges and universities. For quite a few of these families, this means that Junior should go to fair Harvard (after all, it’s the top-rated university by U.S. News and World Report). But few families are willing to go as far as Gerald and Lily Chow of Hong Kong.

An article in the Boston Globe tells the sad saga of the Chow family and how they got duped out of $2.2 million by a former Harvard professor who claimed to be able to get the family’s two sons into the university. The family filed suit against defendant Mark Zimmy’s company after claiming fraud after their children did not get accepted there (although they did get into other elite colleges). Zimmy’s website is still active and targets Chinese students, many of whom have little knowledge of the American educational system. Needless to say, I am interested in how this case proceeds through the legal system.

I am pretty familiar with the academic literature studying the returns to attending a prestigious college. Although there are possibly some additional benefits of attending a more prestigious college to students from disadvantaged backgrounds, the literature is quite clear that the typical student should not expect to benefit by over one million dollars for attending Harvard compared to a slightly less prestigious college. It’s safe to say that the Chow family was likely going to waste their money, even if Mr. Zimmy was able to get their children into Harvard.

The Top of the Ninth

This time of year, my thoughts turn fairly often to baseball. This is especially true this year with my beloved St. Louis Cardinals in the playoffs. The familiar sounds of the game’s great announcers are the background of my summer, and are particularly well-suited for listening while working. Today’s lengthy playoff games (three and a half hours for a regular nine-inning game) made me think of George Carlin’s famous dialogue on why he preferred baseball over football. The best part of the dialogue is the following:

“Baseball has no time limit: we don’t know when it’s gonna end – might have extra innings.
Football is rigidly timed, and it will end even if we’ve got to go to sudden death.”

As I work well into an October evening filled with tightly played postseason games, this quote makes me think about graduate school. Enrolling in a PhD program is a lot like playing baseball—there is no rigidly enforced time limit (at least since the end of curfews about three decades ago) and extra innings are unlimited in theory. Few other sports, with the exception of playoff hockey and cricket, have such indeterminate endings.

My journey through graduate school has often felt like an exciting playoff baseball game. Through my five-plus years in graduate school, both in economics and education policy, I have experienced the highest of highs (incredible research opportunities and working with amazing people) and the lowest of lows (scoring below the posted minimum score on an exam). But days like today make me feel like I’m entering the top of the ninth inning of graduate school with a comfortable lead.

Today marked a very exciting day in my time in graduate school. I have spent at least three years working with a research team on a paper examining the effects of a randomly assigned need-based grant program here in Wisconsin. We finally finished the umpteenth rewrite of the paper and sent it off to a very good journal. The paper should be posted on our study’s website in the next few days, but the main punchline is that financial aid does have modest positive effects on students’ persistence through college. To come up with this estimate, we used a pretty nifty econometric strategy of instrumental variables with treatment-by-site interactions; for baseball fans, think of it as advanced sabermetrics.

Additionally, I have been making good progress in applying for assistant professor positions in both education and public policy schools. In working on my application materials, I realize how much I have learned and grown in my time in graduate school. Five years ago, I couldn’t have imagined what I would be doing today, which is pretty amazing. I won’t know where I will be next year for several months as I approach free agency this spring, but I am looking forward to getting called up to the academic big leagues.

Majoring in Football?

Unlike some in the higher education world, I am often a fan of big-time college athletics. They do provide important benefits to both the university and the broader community, such as social cohesion, increased levels of public support, and (under the right circumstances) economic development. However, my support is limited to when the following conditions are met:

(1)    Athletics must interfere with academics as little as possible for the broader campus community. I understand that athletes’ schedules will be difficult to maintain, but let’s can the nuttiness of weekday evening football games. Other sports can have evening events during the week, but they don’t shut down campus like football does.

(2)    Students must not be forced to provide massive subsidies for athletics programs, as is often the case at non-BCS (think directional state universities with Division I athletics) colleges. Here in Big Ten land, passionate alumni bases giving oodles of money to athletic departments and the successful Big Ten Network have reduced athletic subsidies to a minimum.

(3)    Athletes must care about academics as much as other students (which may not necessarily be that much). Here at Wisconsin, I’ve interacted with a fair number of athletes and nearly all of the experiences have been with students who clearly appreciate the value of a free education and take their academics seriously.

I was extremely disappointed to learn about the case of Ohio State third-string freshman quarterback Cardale Jones,  who clearly violated condition (3) above. He unwisely tweeted the following statement last Friday:

“”Why should we have to go to class if we came here to play FOOTBALL, we ain’t come to play SCHOOL classes are POINTLESS”

Jones was suspended for last Saturday’s game against Nebraska, in which the Huskers were shucked by the score of 63-38. Although Jones would have been unlikely to play, at least Ohio State took some action.

Individuals like Jones are likely in college in the first place because there is not a serious minor league system in football, unlike in baseball and hockey. But given the fact that he is playing at a state-supported university of higher education, he needs to keep his thoughts to himself. Plenty of people in college don’t care much about classes, but they don’t have the same public stage as a Buckeye football player.

As a side note, the comments on the Inside Higher Ed note on this situation are worth a read.

New Data on the Returns to College

Many people love to hate college rankings, but they have traditionally been one of the most easily digestible sources of information about institutions of higher education. We know very little about the outcomes of students who attend a particular college over time, so we tend to rely on simplistic measures such as graduation rates or measures of prestige. It is difficult to follow and assess the outcomes of students once they leave a given college for multiple reasons:

(1)    A substantial percentage of students transfer colleges at least once. A recent report estimated that about one-third of students who enrolled in fall 2006 were enrolled elsewhere sometime in the next five years. The growth of the National Student Clearinghouse has made following students easier, but it is difficult to figure out how to split the credit for successful outcomes across the colleges that a given student attends.

(2)    While the group of students to be assessed (everyone!) sounds straightforward, most of the push has been to focus on the outcomes of graduates. This makes for a reasonable comparison group across colleges, but colleges have different graduation rates. It makes sense to focus on all students who entered a college, but this would lower the returns to college (and doesn’t fit well with selective colleges, where everyone is assumed to graduate).

(3)    Some people choose to postpone entry into the full-time labor market, whether for good reasons (such as starting a family) or for more dubious reasons (such as getting a master’s degree and working on a PhD). Given the lack of a federal data system, other students will not be observed if they move out-of-state to work.

Even with all of the limitations of measuring student outcomes once they leave college, I am heartened to see states starting to track the labor market outcomes of students who attended public colleges and stay in-state. This requires the merging of two data systems that don’t always exist in some states and don’t talk to each other in others—state higher education data systems and unemployment insurance (UI) records. Two states, Arkansas and Tennessee, just launched websites with labor market information for graduates from their public institutions of higher education. While the sample included is far from perfect, it still provides useful data to many students, families, and policymakers.

Not surprisingly, many in academia are worried about these new measures, as they prioritize one of the purposes of higher education (employment) at the expense of other important purposes (such as critical thinking and higher-order learning). The comments on this recent Chronicle of Higher Education article are worth a read. I am concerned about policymakers solely relying on these imperfect measures of student outcomes, but stakeholders should be able to have more information about the effectiveness of colleges on as many outcomes as possible.

Right Idea, Wrong Time

It’s election season once again, so President Obama is coming back to Madison for a large campaign event right smack dab in the middle of the University of Wisconsin-Madison campus Thursday afternoon. Given the amount of security required to host a Presidential visit (regardless of the purpose), it is not surprising that all of the buildings on Bascom Hill will be closed on Thursday. This campaign rally will require all classes in affected buildings to be moved—many of them will likely be cancelled despite this being midterm exam season for undergraduates.

I am always happy to have politicians come to campus to ask for the community’s support, but two things just grate me the wrong way about the visit. The first thing is the timing. When Obama came to campus the previous two times (February 2008 and September 2010), his events were scheduled later in the day. While classes were still moved from the immediate area of Bascom Hill for the 2010 visit, the rally was held later in the afternoon so more classes could be held. Ann Althouse, prominent blogger and faculty member in the UW Law School, isn’t too happy about the class disruption:

“Nice for the campaign, but positioned to maximize disruption of regular classes. Is that a bug or a feature? If there are no classes and it’s a class day, students are around and they are free to attend. Classes are being cancelled to supply the photogenic crowd for the President?”

Badgers are a pretty photogenic lot. (It’s hard to be humble when you’re from Wisconsin, after all.) But starting the event at, say, 4 PM instead of noon would allow for a much more normal day of classes. For reference, recall the hubbub about having a night football game on the Thursday before classes even started. I’m guessing that the folks complaining about a night football game aren’t complaining about the President’s campaign stop—I’m happy to complain about both.

I have one more gripe about the rally: in order to get into the event in the heart of campus, people have to register with the President’s campaign team. I don’t have any problems with metal detectors and tight security (there are plenty of crazy people out there), but requiring registration with an aggressive political campaign team to attend an on-campus event does not support sifting and winnowing. (To be fair, Romney’s folks do the same thing to harvest voter information—but he is never coming to far-left Madison.)

I have taken steps to cancel or postpone all of my events on campus on Thursday and will likely listen to the rally online. Hopefully, all of the people displaced by the campaign event can have a fairly normal day of work if they so choose.