The Top of the Ninth

This time of year, my thoughts turn fairly often to baseball. This is especially true this year with my beloved St. Louis Cardinals in the playoffs. The familiar sounds of the game’s great announcers are the background of my summer, and are particularly well-suited for listening while working. Today’s lengthy playoff games (three and a half hours for a regular nine-inning game) made me think of George Carlin’s famous dialogue on why he preferred baseball over football. The best part of the dialogue is the following:

“Baseball has no time limit: we don’t know when it’s gonna end – might have extra innings.
Football is rigidly timed, and it will end even if we’ve got to go to sudden death.”

As I work well into an October evening filled with tightly played postseason games, this quote makes me think about graduate school. Enrolling in a PhD program is a lot like playing baseball—there is no rigidly enforced time limit (at least since the end of curfews about three decades ago) and extra innings are unlimited in theory. Few other sports, with the exception of playoff hockey and cricket, have such indeterminate endings.

My journey through graduate school has often felt like an exciting playoff baseball game. Through my five-plus years in graduate school, both in economics and education policy, I have experienced the highest of highs (incredible research opportunities and working with amazing people) and the lowest of lows (scoring below the posted minimum score on an exam). But days like today make me feel like I’m entering the top of the ninth inning of graduate school with a comfortable lead.

Today marked a very exciting day in my time in graduate school. I have spent at least three years working with a research team on a paper examining the effects of a randomly assigned need-based grant program here in Wisconsin. We finally finished the umpteenth rewrite of the paper and sent it off to a very good journal. The paper should be posted on our study’s website in the next few days, but the main punchline is that financial aid does have modest positive effects on students’ persistence through college. To come up with this estimate, we used a pretty nifty econometric strategy of instrumental variables with treatment-by-site interactions; for baseball fans, think of it as advanced sabermetrics.

Additionally, I have been making good progress in applying for assistant professor positions in both education and public policy schools. In working on my application materials, I realize how much I have learned and grown in my time in graduate school. Five years ago, I couldn’t have imagined what I would be doing today, which is pretty amazing. I won’t know where I will be next year for several months as I approach free agency this spring, but I am looking forward to getting called up to the academic big leagues.

Majoring in Football?

Unlike some in the higher education world, I am often a fan of big-time college athletics. They do provide important benefits to both the university and the broader community, such as social cohesion, increased levels of public support, and (under the right circumstances) economic development. However, my support is limited to when the following conditions are met:

(1)    Athletics must interfere with academics as little as possible for the broader campus community. I understand that athletes’ schedules will be difficult to maintain, but let’s can the nuttiness of weekday evening football games. Other sports can have evening events during the week, but they don’t shut down campus like football does.

(2)    Students must not be forced to provide massive subsidies for athletics programs, as is often the case at non-BCS (think directional state universities with Division I athletics) colleges. Here in Big Ten land, passionate alumni bases giving oodles of money to athletic departments and the successful Big Ten Network have reduced athletic subsidies to a minimum.

(3)    Athletes must care about academics as much as other students (which may not necessarily be that much). Here at Wisconsin, I’ve interacted with a fair number of athletes and nearly all of the experiences have been with students who clearly appreciate the value of a free education and take their academics seriously.

I was extremely disappointed to learn about the case of Ohio State third-string freshman quarterback Cardale Jones,  who clearly violated condition (3) above. He unwisely tweeted the following statement last Friday:

“”Why should we have to go to class if we came here to play FOOTBALL, we ain’t come to play SCHOOL classes are POINTLESS”

Jones was suspended for last Saturday’s game against Nebraska, in which the Huskers were shucked by the score of 63-38. Although Jones would have been unlikely to play, at least Ohio State took some action.

Individuals like Jones are likely in college in the first place because there is not a serious minor league system in football, unlike in baseball and hockey. But given the fact that he is playing at a state-supported university of higher education, he needs to keep his thoughts to himself. Plenty of people in college don’t care much about classes, but they don’t have the same public stage as a Buckeye football player.

As a side note, the comments on the Inside Higher Ed note on this situation are worth a read.

New Data on the Returns to College

Many people love to hate college rankings, but they have traditionally been one of the most easily digestible sources of information about institutions of higher education. We know very little about the outcomes of students who attend a particular college over time, so we tend to rely on simplistic measures such as graduation rates or measures of prestige. It is difficult to follow and assess the outcomes of students once they leave a given college for multiple reasons:

(1)    A substantial percentage of students transfer colleges at least once. A recent report estimated that about one-third of students who enrolled in fall 2006 were enrolled elsewhere sometime in the next five years. The growth of the National Student Clearinghouse has made following students easier, but it is difficult to figure out how to split the credit for successful outcomes across the colleges that a given student attends.

(2)    While the group of students to be assessed (everyone!) sounds straightforward, most of the push has been to focus on the outcomes of graduates. This makes for a reasonable comparison group across colleges, but colleges have different graduation rates. It makes sense to focus on all students who entered a college, but this would lower the returns to college (and doesn’t fit well with selective colleges, where everyone is assumed to graduate).

(3)    Some people choose to postpone entry into the full-time labor market, whether for good reasons (such as starting a family) or for more dubious reasons (such as getting a master’s degree and working on a PhD). Given the lack of a federal data system, other students will not be observed if they move out-of-state to work.

Even with all of the limitations of measuring student outcomes once they leave college, I am heartened to see states starting to track the labor market outcomes of students who attended public colleges and stay in-state. This requires the merging of two data systems that don’t always exist in some states and don’t talk to each other in others—state higher education data systems and unemployment insurance (UI) records. Two states, Arkansas and Tennessee, just launched websites with labor market information for graduates from their public institutions of higher education. While the sample included is far from perfect, it still provides useful data to many students, families, and policymakers.

Not surprisingly, many in academia are worried about these new measures, as they prioritize one of the purposes of higher education (employment) at the expense of other important purposes (such as critical thinking and higher-order learning). The comments on this recent Chronicle of Higher Education article are worth a read. I am concerned about policymakers solely relying on these imperfect measures of student outcomes, but stakeholders should be able to have more information about the effectiveness of colleges on as many outcomes as possible.

Right Idea, Wrong Time

It’s election season once again, so President Obama is coming back to Madison for a large campaign event right smack dab in the middle of the University of Wisconsin-Madison campus Thursday afternoon. Given the amount of security required to host a Presidential visit (regardless of the purpose), it is not surprising that all of the buildings on Bascom Hill will be closed on Thursday. This campaign rally will require all classes in affected buildings to be moved—many of them will likely be cancelled despite this being midterm exam season for undergraduates.

I am always happy to have politicians come to campus to ask for the community’s support, but two things just grate me the wrong way about the visit. The first thing is the timing. When Obama came to campus the previous two times (February 2008 and September 2010), his events were scheduled later in the day. While classes were still moved from the immediate area of Bascom Hill for the 2010 visit, the rally was held later in the afternoon so more classes could be held. Ann Althouse, prominent blogger and faculty member in the UW Law School, isn’t too happy about the class disruption:

“Nice for the campaign, but positioned to maximize disruption of regular classes. Is that a bug or a feature? If there are no classes and it’s a class day, students are around and they are free to attend. Classes are being cancelled to supply the photogenic crowd for the President?”

Badgers are a pretty photogenic lot. (It’s hard to be humble when you’re from Wisconsin, after all.) But starting the event at, say, 4 PM instead of noon would allow for a much more normal day of classes. For reference, recall the hubbub about having a night football game on the Thursday before classes even started. I’m guessing that the folks complaining about a night football game aren’t complaining about the President’s campaign stop—I’m happy to complain about both.

I have one more gripe about the rally: in order to get into the event in the heart of campus, people have to register with the President’s campaign team. I don’t have any problems with metal detectors and tight security (there are plenty of crazy people out there), but requiring registration with an aggressive political campaign team to attend an on-campus event does not support sifting and winnowing. (To be fair, Romney’s folks do the same thing to harvest voter information—but he is never coming to far-left Madison.)

I have taken steps to cancel or postpone all of my events on campus on Thursday and will likely listen to the rally online. Hopefully, all of the people displaced by the campaign event can have a fairly normal day of work if they so choose.

An Elite Take on College Rankings

As a conservative, small-town Midwesterner, I get a great deal of amusement out of the education coverage in the New York Times. I have never quite understood the newspaper’s consistent focus on the most elite portions of America’s educational systems, from kindergartens which cost more than most colleges (is the neighborhood school really that bad) to the special section of the website regarding the Ivy League. In that light, I was interested when several friends sent me the NYT’s take on college rankings and surprised to find a discussion that didn’t focus solely on the Ivy League.

In Saturday’s edition of the paper, columnist Joe Nocera noted some of the limitations of the U.S. News and World Report college rankings, such as rewarding selectivity and spending more money regardless of outcomes. (I’ve written plenty on this topic.) He notes that the Washington Monthly rankings do seek to reward colleges which effectively educate their students, and also states that a reduced focus on institutional prestige might help reduce student stress.

I am hardly a fan of Nocera (who is best known for comparing Tea Party supporters to terrorists), but the piece is worth a read. I highly recommend reading through the comments on the article, as they show a sharp divide between commenters who believe that attending solid—but not elite—colleges is a good investment and those who believe strongly in attending an elite institution. For those of us who are not regular readers of the Gray Lady, the comments also give us an idea of what some of America’s elite think about the value of certain types of higher education.

The Wisconsin Idea in Action

One of the factors which attracted me to the University of Wisconsin-Madison for graduate school was the Wisconsin Idea—the belief that the boundaries of the university should be the boundaries of the state. (Yes, that is much more important than being able to see my beloved Packers on television each week—and I’m a shareholder in the team.) As the University of Wisconsin System was formed in the early 1970s, the Wisconsin Idea has been adopted by the rest of the state’s public colleges and universities. While some people say that the Wisconsin Idea has passed its prime due to the focus on arcane research topics, I still think the idea is alive in well.

I saw a great example of the Wisconsin Idea in action at UW-Parkside that made the state newspapers this morning. Two Parkside students did research for a class project and discovered that moving prisoners’ medical records from paper to electronic formats could save millions of dollars and likely improve patient outcomes. This is a win-win for the students (who gain valuable research experience and analytic skills), the university (which gets great publicity), and the state (which should be able to save money).

I have been privileged to study the Wisconsin public higher education system for the past four-plus years through the Wisconsin Scholars Longitudinal Study. It is not uncommon for someone at UW-Madison to look down their noses at the rest of the UW System, but it is critical to recognize the contributions of the entire system toward making Wisconsin a better place to live.

Sticker Shock in Choosing Colleges: What Can Be Done?

Very few items are priced in the same manner as a college education. While the price of some items, such as cars and houses, can be negotiated downward from a posted (sticker) price, the actual price and the sticker price are usually in the same ballpark. However, the difference between the sticker price and the actual price paid can be enormous in higher education. This has posed a substantial problem to students and their families, especially those with less knowledge of the collegegoing and financial aid processes.

Until recently, students had to apply for financial aid to get an idea of how much college would actually cost them. The latest iteration of the Higher Education Opportunity Act, signed in 2008, required that institutions place a net price calculator on their website by last October. This calculator uses basic financial information such as income, household size, and dependency status to estimate a student’s expected family contribution (EFC), which would then give students an idea of their grant aid.

The need for more transparent information on the actual cost of college is shown by a recently released poll conducted by the College Board and Art & Science Group, LLC. These groups polled a nonrandom sample of SAT test-takers applying to mainly selective four-year colleges and universities in late 2011 and early 2012 and found that nearly 60% of low and middle-income families ruled out colleges solely because of the sticker price. This is in spite of generous need-based financial aid programs at some expensive, well-endowed colleges.

Given that the survey was conducted right as net price calculators became mandatory, it is likely the case that more students are aware of these tools by now. But it is unlikely that net price calculators have been used as much as possible, especially by first-generation students. To make the net price more apparent, the Department of Education has put forth a proposed “Shopping Sheet” that can be easily compared across colleges. This proposal has advocates in Washington, but there are reasonable concerns that a one-size-fits-all model may not benefit all colleges.

As an economist, I hope that better information can help students and their families make good decisions about whether to go to college and where to attend. However, I am also hesitant to believe that requiring uniform information across colleges will result in something useful.

Public Research at its Finest: The 2012 Ig Nobel Prize Winners

It is no secret that academics research some obscure topics—and are known to write about these topics in ways that obfuscate the importance of such research. This is one reason why former Senator William Proxmire (D-WI) started the Golden Fleece Awards to highlight research that he did not consider cost-effective. Here are some examples, courtesy of the Wisconsin Historical Society. (Academia has started to push back through the Golden Goose Awards, conceived by Rep. Jim Cooper (D-TN).)

Some of these potentially strange topics either have potentially useful applications or are just plain thought-provoking. To recognize some of the most unusual research in a given year, some good chaps at Harvard organized the first Ig Nobel Prize ceremony in 1991. This wonderful tradition continues to this day, with the 2012 ceremony being held yesterday. Real Nobel Prize winners are even known to hand out the awards!

Ten awards are handed out each year, so it is difficult to pick the best award. My initial thought was to highlight the Government Accountability Office’s report titled, “Actions Needed to Evaluate the Impact of Efforts to Estimate Costs of Reports and Studies,” but this sort of report is not unusual in the federal government. So I’ll single out a nice little article on whether multiple comparisons bias can result in brain wave activities for a dead Atlantic salmon (no word on whether the study participant was consumed after completion of the study) as my favorite award. Multiple comparisons bias is certainly real and the authors provide a nice example of how to lie with statistics, but the subject tested sure is unusual. I encourage people to take a look at the other awards and try to figure out how these research projects got started. Some seem more useful than others, but that is the nature of academic research.

The Annals of Improbable Research, the folks who put on the Ig Nobel ceremony, also have three hair clubs for scientists: The Luxuriant Flowing Hair Club for Scientists, the Luxuriant Former Hair Club for Scientists, and the Luxuriant Facial Hair Club for Scientists.

Here is the full video of the ceremony.

Knowing Before You Go

Knowing Before You Go

The American Enterprise Institute today hosted a discussion of the Student Right to Know Before You Go Act, introduced by Senator Ron Wyden (D-OR) and co-sponsored by Senator Marco Rubio (R-FL). The two senators, both of whom are known for working across party lines, briefly discussed the legislation and were then followed by a panel of higher education experts. Video of the discussion will be available on AEI’s website shortly.

The goal of the legislation, as the senators discuss in a column in USA Today, is to provide more information about labor market and other important outcomes to students and their families. While labor market outcomes are rarely available in any systemic manner, this legislation would support states which release the data both at the school level and by academic programs. This sort of information cannot be collected at the federal level due to a restriction placed in Section 134 of the Higher Education Act reauthorization in 2008, which bans the Department of Education from having a student-level data system of the sort used in some states.

While nearly everyone across the political spectrum agrees that making additional data available is good for students and their families, there are certainly concerns about the proposed legislation. One concern is that the availability of employment data will make more rigorous accountability systems feasible, even though state-level data systems can only track students who stay within that state. This concern is shared by colleges, which tend to loathe regulation, and some conservatives, who don’t feel that the federal government should regulate higher education.

Additionally, measuring employment outcomes does place more of a focus on generating employment over some of the other goals of college (such as learning for learning’s sake). The security of these large unit-record datasets is also a concern of some people; I am less concerned about this given the difficulty of accessing deidentified data. (I’ve worked with the data from Florida, which has possibly the most advanced state-level data system. Getting access is extremely difficult.)

Although I certainly recognize those concerns, I strongly support this piece of legislation. It would reduce reporting requirements for colleges, since they would work primarily with states instead of the federal government. (In that respect, the legislation is quite conservative.) It makes more data available to all stakeholders in education and provides researchers with more opportunities to examine promising educational practices and intervention. Finally, it allows for states to make more informed decisions about how to allocate their scarce resources.

I don’t expect this legislation to go anywhere during this session of Congress, even with bipartisan support. Let’s see what happens next session, by which time I hope we are away from the “fiscal cliff.”

The Limitations of “Data-Driven” Decisions

It’s safe to say that I am a data-driven person. I am an economist of education by training, and I get more than a little giddy when I get a new dataset that can help me examine an interesting policy question (and even more exciting when I can get the dataset coded correctly). But there are limits to what quantitative analysis can tell us, which comes as no surprise to nearly everyone in the education community (but can be surprising to some other researchers). Given my training and perspectives, I found an Education Week article on the limitations of data-driven decisions by Alfie Kohn, a noted critic of quantitative analyses in education, interesting.

Kohn writes that our reliance on quantifiable measures (such as test scores) in education result in the goals of education being transformed to meet those measures. He also notes that educators and policymakers have frequently created rubrics to quantify performance that used to be more qualitatively assessed, such as writing assignments. These critiques are certainly valid and should be kept in mind at all times, but then his clear agenda against what is often referred to as data-driven decision making shows through.

Toward the end of his essay, he launches into a scathing criticism of the “pseudoscience” of value-added models, in which students’ gains on standardized tests or other outcomes are estimated over time. While nobody in the education or psychometric communities is (or should be) claiming that value-added models give us a perfect measure of student learning, they do provide us with at least some useful information. A good source for more information on value-added models and data-driven decisions in K-12 education can be found in a book by my longtime mentor and dissertation committee member Doug Harris (with a foreword by the president of the American Federation of Teachers).

Like it or not, policy debates in education are being increasingly being shaped by the available quantitative data in conjunction with more qualitative sources such as teacher evaluations. I certainly don’t put full faith in what large-scale datasets can tell us, but it is abundantly clear that the accountability movement at all levels of education is not going away anytime soon. If Kohn disagrees with the type of assessment going on, he should propose an actionable alternative; otherwise, his objections cannot be taken seriously.