NBER Reporter Online
NBER Reporter OnLine: 2011 Number 1PDF Version (includes NBER Profiles, Conferences)Previous Issues NBER Program Reports Subscribe to the hard copy NBER Reporter. |
Program Report: Economics of Education |
Caroline M. Hoxby*
The Economics of Education Program is both exciting and productive, currently adding new Working Papers at the rate of 7.5 per month -- a 50 percent increase from the rate at the time of my last program report in fall 2006. The number of papers submitted for a typical Program Meeting is often ten times the number of available slots, and attendance at those meetings is high.
I am particularly proud of three aspects of the Program. The first is the quality of the research being produced and the methods used by members, including some of the latest, most rigorous methods in applied micro-econometrics. The second is the fact that members use some of the richest, most comprehensive datasets in economics -- many of these datasets were initially compiled by schools or school-related organizations, and program members deserve enormous credit for their resourcefulness in making them useful for economic research by establishing strong, collegial relationships with data providers, convincing schools to conduct randomized and other policy experiments, matching data from diverse sources, and themselves surveying or testing people when data otherwise would be missing. Third, program members produce research that is policy relevant, credible to policymakers, and grounded in economic logic.
The NBER's Higher Education Working Group was integrated into the Economics of Education Program in 2009. We made the integration an occasion to celebrate the leadership of Charles T. Clotfelter, director of that working group, who oversaw an immense improvement in the quality of research on the economics of higher education. Although the practical policy questions differ across the two levels of education, all of the methods, much of the data, and much of the deep economic logic are shared.
Areas of Continuing Interest and New Interest
In my last review, I focused on three areas in which research was advancing particularly rapidly: the analysis of peer effects; the estimation of teachers' effects on achievement; and making sense of students' college choices (not just whether to attend college in the first place, but which schools to attend and whether to persist at each school). These three areas continue to be highly productive. For instance, Elias Bruegmann and C. Kirabo Jackson (15202) demonstrate that, when a teacher whose own effect on achievement is strongly positive moves into a new school, her new colleagues improve. They further show that the colleagues' improved ability to raise achievement is attributable to their changing, not merely to selection. That is, incumbent teachers in the new school raise their performance. For another example, we now have substantial evidence on what happens to a student who goes to a school where other students are high-achieving: his own achievement rises. This evidence relies on regression discontinuity methods, that is, on comparing the later achievement of students who are just above and just below some admissions threshold, where the threshold is not known to students when they apply. Christian Pop-Eleches and Miguel Urquiola (16886) study this situation in Romania; Damon Clark ("Elite Schools and Academic Performance", presented at the spring 2007 Program Meeting) studies this situation in England; and C. Kirabo Jackson (16598) studies this situation in Trinidad and Tobago. Turning to college-going behavior, some of the most interesting new research provides rigorous evidence on how students respond to scholarships and other financial aid designed to improve their college outcomes. Aimee Chin and Chinhui Juhn (15932) show that allowing undocumented students to pay in-state tuition (usually just one-third to one-half of out-of-state tuition) has no statistically significant effect on their college attendance. Stephens Desjardins and Brian McCall ("The Impact of the Gates Millenium Scholars Program", presented at the spring 2008 Program Meeting) show that Gates Scholarships very modestly improved persistence among the low-income minority students eligible for them.
Since my last report, several new themes also have emerged in Economics of Education research. Two notable ones are the importance of information and the role of incentives for students, teachers, and schools. Because any program review is necessarily selective, I focus here mainly on illustrating these new themes.
The Importance of Information
Much of the existing research on education concerns the change in some concrete resource: a salary increase for teachers; a reduction in class size; a scholarship or other financial aid for students; the extension of compulsory schooling; or the opening of a program. Although such resource changes often can be shown to change educational outcomes, their effects typically are much smaller than proponents believed they would be. Also, two students with similar prior achievement often react to resources in very different ways. For instance, although making financial aid more generous causes some students to attend college or to persist longer in college, a good share of students do not respond. Frustratingly for researchers, the students who do not respond often look very similar to the students who do. (On this point, see for instance the Desjardins and McCall paper mentioned above.) Put another way, researchers have been unable to show that policymakers could control and improve most people's educational outcomes simply by controlling policies that are concerned with educational resources.
Responding to the weak explanatory power of resource-type policies, researchers increasingly have wondered whether differences in students' and families' information can account for variation in educational outcomes. Recent findings from behavioral economics, which often show that apparently small differences in the content or framing of information can have large effects, have only intensified education researchers' focus on information. There are practical reasons to focus on information as well: information interventions tend to be very inexpensive compared to resource-type interventions (so that even modest benefits may outweigh costs) and often have positive spillovers (useful information given to one person tends to spread to other people).
Eric Bettinger, Bridget Long, Philip Oreopoulos, and Lisa Sanbonmatsu (15361) designed an experiment in coordination with the tax preparer H&R Block. Some families with college-aged children were randomly assigned to be given information on their child's eligibility for government-based financial aid and on local college-going options. Some families also were randomly assigned to receive help in filing the federal application for financial aid ("FAFSA"). The results, which are highly credible owing to the randomized design, suggest that the intervention that combined information and FAFSA help actually caused people to be 25 to 30 percent more likely to enroll in college. These effects are dramatic in size for such a modest intervention -- one that, if implemented routinely, would cost only a few dollars per family.
Todd Stinebrickner and Ralph Stinebrickner (14810) investigate whether students learn about their academic ability in college and make decisions about persisting in a logical way, based on that information. To study this queston, they combine rich administrative data from Berea College with data from surveys they conducted themselves. Thus, they are able to observe not just students' academic behavior, such as their course-taking patterns and the grades they earn, but also students' beliefs about their academic aptitude and expectations about college completion. The authors show that students enter college with beliefs about their academic ability that are both optimistic and diffuse. Moreover, the students update their beliefs in the manner predicted by the Bayesian learning model. Students' learning about their own aptitude explains much of their decision to drop out of college.
Amanda Pallais ("Why Not Apply?" presented at the spring 2008 Program Meeting) shows that an apparently tiny change in ACT policy produced a 20 percent increase in students' applications to colleges. The change was that ACT, one of the two college aptitude testing organization in the United States, gave students four free score reports instead of three. Because an additional score report cost only $6 before and after the policy change, the intervention was negligible when viewed against the background of family income or the potential returns to college attendance. Yet, the policy change caused about 40 percent of students to send their scores to an additional school. This generated some additional information for students because, when a student who is a plausible applicant sends his scores to a school, that school responds with brochures and other materials describing its offerings. It is striking that such a modest change in information produced such sizable effects on behavior.
Avery and Turner ("Playing the College Application Game", presented at the fall 2009 Program Meeting) and Avery and Hoxby ("The Missing One-Offs", presented at the 2010 Summer Institute) demonstrate that low-income students apply to fewer and less selective colleges than their more affluent counterparts who have the same test scores and achievement in high school. This fact holds even for low-income students whose achievement is so high that they qualify for free tuition and living expenses at the most selective colleges in the United States. The authors of these papers assemble an array of evidence that indicates that low-income students lack information about college-going. While it is hard to argue that these students do not have access to materials (since most colleges' materials are readily available online), they have few contacts with people who attended selective colleges. They are frequently too isolated geographically to find a critical mass of college-going peers or advisors. In fact, the latter paper shows that it would not even make sense for selective colleges' staff to visit the schools or cities of most low-income, high-achieving students: they are simply too isolated for the benefits of such visits to outweigh the costs. The bottom line is that information interventions might be warranted, but they may prove hard to design -- see Avery (16359).
Informational differences among students are also important in primary and secondary education. Parag Pathak and Tayfun Sonmez (16783; also "Leveling the Playing Field," 2008 Summer Institute) show that school choice mechanisms that are susceptible to strategic manipulation tend to generate better outcomes for families who are more informed. That is, although all students have the same opportunities under these mechanisms, students who understand how the mechanisms work and which schools are in demand end up enrolling in schools that are higher in their preference rankings. These better informed students disproportionately have parents who are affluent and educated. Thus, superior information is one reason why students' outcomes are correlated with their family's socioeconomic circumstances.
Abigail Wozniak and Ofer Malamud (16463) explore another reason why students from more educated families have better outcomes. They investigate the long-standing hypothesis that more educated people respond more elastically to changes in opportunities. (Theodore W. Schultz often is credited with originating this idea. See Bowman, 1980, cited below.) Specifically, Wozniak and Malamud investigate people who were induced to attend college because they had a higher risk of being drafted for the Vietnam War. They use draft induction risk as an instrument for attending and graduating from college, and they show that college education makes a person more likely to subsequently choose his labor market experience based on expected earnings, as opposed to the market's mere proximity to his place of origin.
School report cards -- simple reports that describe students' achievement in absolute terms and relative to other local schools -- are very inexpensive to provide. Asim Khwaja, Tahir Andrabi, and Jishnu Das ("Report Cards," spring 2009 Program Meeting) arranged to provide reports in 112 randomly selected educational markets in Pakistan. The intervention was purely informational: no explicit rewards or punishments were included. The authors find that the report cards improved learning by 0.10 standard deviations and increased enrollment slightly. Private schools that were initially bad -- those with below median scores at baseline -- improved especially strongly: learning gains were 0.34 standard deviations. Private schools that were initially good did not improve learning but did cut their fees. Government schools were somewhat less responsive than private schools. The authors interpret these results as showing that report cards generate competitive pressure on schools to increase price-adjusted quality.
Jonah Rockoff, Douglas Staiger, Thomas Kane, and Eric Taylor (16240) study another informational intervention that appears small yet had big effects. They evaluate the effect of a program in which New York City school principals were provided with estimates of how much each of their teachers had raised students' test scores. Principals were randomly assigned to this program, so the study's findings are highly credible. The authors show that principals update their beliefs about teachers' effects in accordance with the Bayesian learning model: for instance, principals update their beliefs more when the estimates provided to them are more precise and their own prior opinions are less precise. More importantly, principals are likelier to retain their effective teachers (and not retain their ineffective ones) when they are provided with the estimated teacher effects. The change in the sensitivity of retention to performance improves student achievement by a statistically significant though small amount. Here, it is worthwhile to remember the cost-benefit ratios typical of information interventions: although the change in achievement is small, the cost of the intervention is very small on an ongoing basis.
Finally, Eric Taylor and John Tyler (16877) examine a highly reputed teacher evaluation system and find that it improves teachers' performance, as measured by their effects on student achievement. While the cost-benefit ratio of the program they study is not as impressive as the results of the information program in New York City (16240), the improvement that Taylor and Tyler see is entirely within teacher. As a rule, it has been hard for researchers to produce credible evidence that teachers improve simply through being evaluated and then informed about how to improve their instruction. Even if such evaluation systems are an expensive means of improving achievement relative to some of the informational interventions described above, they remain inexpensive relative to most resource-type interventions.
Incentives for Students, Teachers, and Schools
Even though improving incentives is often more expensive than improving information, incentive-type interventions are often much less expensive than resource-type interventions, especially when their relative efficacy is taken into account. This is shown by an array of recent research done by program members.
Joshua Angrist, Daniel Lang, and Philip Oreopoulos (12790) and Joshua Angrist, Philip Oreopoulos, and Tyler Williams (16643) explore incentives for students to improve their grades in a Canadian university. In the former paper, they study students who are randomly assigned to receive a merit scholarship if they maintain solid grades. In the second paper, they study students who are randomly assigned to receive cash for better grades: $100 for each grade of 70 or better and an additional $20 for each percentage point above 70 percent. They find that the merit scholarship improved the grades and persistence of female students, though not of males. Interestingly, they also find that the availability of the merit scholarship caused female students to seek out more help with their courses: they were more likely to take advantage of supplemental instructional services. In the latter paper, the authors find that the cash rewards improved males' achievement, though not females! The effects on males are modest overall, but larger for males who understood the function linking performance to rewards.
Judith Scott-Clayton ("On Money and Motivation", fall 2008 program meeting) studies a West Virginia incentive scheme for college students. The program offered free tuition to students who maintained a certain minimum course load and minimum GPA (2.75 in the freshmen year, 3.0 thereafter). Since students were not randomly assigned to the program, Scott-Clayton exploits differences in the timing of implementation and discontinuities in the eligibility formula to generate credible estimates. Not only does she find substantial effects on achievement, she also finds that the effects are highly concentrated around the thresholds for annual scholarship renewal, indicating that the program's effects come via the incentives it provides, not simply via relaxing financial constraints.
C. Kirabo Jackson (15722) studies incentives for students and teachers based on Advanced Placement (AP) scores. The program he analyzes ("APIP") pays high school students and their teachers between $100 and $500 per score of three or above on an AP exam. To give a sense of magnitude of rewards that a person could earn, the maximum that a teacher has ever earned in one year is $11,500, and the maximum that a student has ever earned in high school is $1,400. Because the program is not randomly assigned to schools, Jackson has to use a detrended difference-in-differences strategy: essentially, the achievement trends of schools that adopted the program earlier are compared to the achievement trends of schools that adopted it later. Because the program's sponsors were not able to roll out the program in a single year to every school interested in adoption, the late adopters are fairly idiosyncratically selected from among schools who applied. Thus, the results are quite credible. Jackson finds that students who participate in the program are more likely to attend college and persist in college beyond their freshman year. In addition, Black and Hispanic students are more likely to graduate from college.
Eric Bettinger (16333) examines cash incentives for students funded by a philanthropist in Coshocton, a poor city in the Appalachian area of Ohio. Schools and grades in the city were randomly assigned to have their students get rewards of up $75 per year for "proficient" scores and $100 per year for "advanced" scores on Ohio's statewide exams. Bettinger finds that the incentives improve math scores by 0.15 standard deviations but he does not find similar effects on other subject exams. The Coshocton program was highly beneficial relative to its costs: the program costs were only fifteen hundredths of 1 percent (0.15 percent) of the district's per-pupil expenditures. The effects of this inexpensive program on achievement were 250 times what we would predict if the district had spent the same amount on class size reduction. (The class size comparison is based on Project Star, which generates some of the highest credibly estimated effects of class size reduction.)
Karthik Muralidharan and Venkatesh Sundararaman (15323) investigate performance pay for teachers, using a program in India that they themselves largely designed. Hundreds of schools were randomly assigned to have their teachers receive higher pay for higher students' scores. Hundreds of schools were assigned to an alternative treatment that gave them additional resources equal to the value of the performance pay. Muralidharan and Sundararaman find that students in incentive schools improved their performance by 0.28 and 0.16 standard deviations in math and language tests, relative to control scores. Students scored significantly higher on "conceptual" as well as "mechanical" components of the tests and also performed better on subjects for which no incentives were given. These results suggest that the students' gains in achievement were authentic, not mere "teaching to the test." The gains in schools that simply received the extra resources were one-third to one-half as large as the incentive-driven gains.
Several authors have examined what happens when schools face incentives. For instance, Jonah Rockoff and Lesley Turner (14564) and Hanley Chiang ("Accountability Pressure on Failing Schools," fall 2008 Program Meeting) use regression discontinuity methods to show that schools that "just fail" according to their state's accountability program raise their students' achievement more than schools that "just pass." In these two studies, failing schools faced several possible consequences: students could transfer out, principals could lose their jobs, and schools could be closed completely (though this was rare). Since the passing thresholds were unknown to schools in advance, the regression discontinuity designs produce convincing results.
A very different source of school incentives -- competitive pressures generated by private school vouchers -- is analyzed by David Figlio and Cassandra Hart (16056) and by Winnie Chan and Robert McMillan ("School Choice and Public School Performance, fall 2009 program meeting). Although the authors investigate programs in different locations -- Figlio and Hart analyze a Florida corporate tax credit program and Chan and McMillan analyze a tax credit program in Ontario -- both teams of authors exploit variation in pressure on public schools that arises through pre-existing differences in the local availability of private schools. Both teams find that public schools respond to the potential loss of students to private schools by raising their students' achievement. Neither team of authors finds evidence that differential student sorting (poor students disproportionately leaving the public schools) accounts for the improvement.
Summing Up
New themes emerge in research because researchers find themselves convinced by previous studies that some questions remain answered, thereby exposing other questions as likely to be important. Thus, I think that it is a measure of the success of the NBER's Economics of Education Program that, although some recent research extends and elaborates themes I identified previously, I did not predict the themes of much recent research in my previous program review. In particular, it is encouraging that so much current research focuses on issues like information and incentives that economists have long regarded as important. That information and incentive-type interventions also tend to have propitious cost-benefit ratios is a bonus. Finally, it is important that NBER researchers continue to pioneer rigorous methodological designs and create good data that allow them to analyze such interventions.
Endnotes:
M.J. Bowman, "Theodore W. Schultz's Contributions to Economics," The Scandinavian Journal of Economics, Vol. 82, No. 1 (1980), pp. 80-107.
* Hoxby is the Director of the NBER's Program on Economics of Education and the Scott and Donya Bommer Professor of Economics at Stanford University. The numbers in parends throughout this report refer to NBER Working Papers. A complete list of NBER Education Working Papers.