NBER Reporter: Spring 2001
Efficiency and Equity in Education
Education is of interest to many economists because of its perceived importance for a wide variety of economic issues. But like others in society, economists also have a personal interest in education -- having been students and perhaps taught themselves, having had children who are students, and often having formed strong opinions about educational policy through their own experiences. This combined professional and personal interest in education undoubtedly has heightened the interest in school research and led to stronger reactions to the policy implications of that research.
A major strand of my work concerns what determines student achievement -- what economists generally would call part of "human capital quality" -- and, most importantly, what role schools and governmental policy play in this equation. The results of this research reveal a complicated picture of determining factors that have subsequent implications for other areas of research and policy undertakings.
However, overshadowing all other findings is the fact that measurable attributes of teachers and schools bear little systematic relationship to student performance. This finding is controversial, at least partly because of its policy implications.
Some Background
The concept of human capital, while part of economics for several centuries, has only recently become central to both theoretical and empirical analyses. In the 1960s and 1970s, Theodore W. Shultz, Gary Becker, and Jacob Mincer laid the foundation of this theory. Their analyses framed the issues of investment in individual skills and provided insights into their empirical relevance. However, most early analysis concentrated on the quantity of individual human capital--not the quality or its determination--and its implications for subsequent wages or health. Specifically, the impact of schools on "quality" was not addressed. In fact, the best early study on the role of schools in skill formation was conducted outside the field of economics in the "Coleman Report." 1 This government publication, dictated by the Civil Rights Act of 1964, suggested that schools had little to do with human capital as measured by cognitive achievement. This pioneering analysis focused on quality issues but contained a variety of fundamental analytical flaws. 2 The Coleman Report commonly has been interpreted as showing that "schools do not matter," because its analysis indicated that family background, followed by peer influence, and least of all school attributes, determined student achievement. While its methodology was problematic, the report motivated a broad inquiry into the role of schools.
Analysis of Educational Production
Economists naturally think in terms of a production model, where school and other influences go in and student achievement comes out. However, the concept has been implemented in a variety of ways.
The standard statistical analysis relates student outcomes to family profiles and observable characteristics of schools and teachers. This approach, which frequently relies on schools' administrative records, has been applied to a wide range of U.S. schools. But these studies failed to reveal that school resources influence student performance in any systematic way, a finding I've developed in a series of papers. 3 In close to 300 studies examining the influence of class size reduction on student performance, nearly equally balanced positive and negative effects were uncovered. The dearth of statistically significant results (14 percent on each side) also underscores the fact that the vast majority of these studies reveal no relationship at all. Studies on teachers' graduate education and experience, as well as on per-pupil spending, yield similar mixed results. These findings, while once surprising to many, have now become the conventional wisdom.
The most significant misinterpretation of these standard studies -- the central problem with analyzing the Coleman Report -- is the conclusion that schools do not matter in education. Another closely related line of research has pointed out the flaws in that interpretation. Specifically, while the commonly measured attributes of teachers do not appear to be important in any systematic way, there remain large and consistent differences among teachers and schools.
Alternative statistical studies have used a general fixed-effect approach to estimation of teacher and school quality. In this methodology, differences in the growth of student achievement during a specific grade are related to the student's teacher. These studies, which measure teacher quality implicitly by the performance of students, invariably find large and significant differences among teachers. 4 Yet differences in teacher quality still are unrelated to the measured characteristics of teachers, class size, and the like.
These estimated differences in teacher quality reveal how strongly teachers affect student achievement (and justify many parents' interest in ensuring that their children are placed with specific teachers). Variations resulting from teacher quality dwarf those that result from class size or from measurable characteristics of teachers. 5 Moreover, this research demonstrates the ability of schools to overcome deficits in family background, especially among low-income students. 6 The ability of teachers to significantly affect the performance of low achievers justifies the historic attention to compensatory policies, although the policies actually implemented have not been very successful because they have not focused on teacher quality.
Because greater resources are not systematically related to higher student performance, schools may be inefficient. Greater inputs of resources simply do not translate into higher outcomes. This conclusion has obvious implications for policy, since many people want to use inputs, or resources, to bring about desired changes. These types of policies are discussed below.
Controversies
The previous conclusions about the lack of importance of measured inputs to schools come from summarizing the results of all the underlying statistical studies (given minimal quality criteria) that are available. However, differences in the quality of the studies will affect their conclusions. Therefore some clear criteria for quality need to be established. First, studies must take into account the varying education policies of the 50 states, recognizing that the states are primarily responsible for organizing and funding schools. States differ substantially in their funding approaches, in labor laws, in teacher certification and hiring requirements, and more. Many standard studies of educational production draw data from across states, but fail to measure the differences in these states' policy environments. Such studies suffer from specification errors, and these problems are compounded if aggregate state level data are used. 7
Second, studies should take into account the educational background of the students tested. Educational policies are cumulative, so performance of twelfth graders is the product of more than just the instruction they receive during that grade. However, many studies ignore the students' history, largely because of missing data. To circumvent these problems and to avoid some of the largest missing-data issues, a number of studies have estimated value-added models: that is, models that analyze achievement across a short time span, such as a single grade, and take into account the achievement of students at the beginning of the analysis period. 8
One way to understand the importance of these determinants of study quality is to exclude studies that use multistate data and to focus exclusively on those with a value-added design. When this is done, the evidence that resources minimally affect performance is even stronger. 9
Another historical controversy, largely moot at this time, relates to the measurement of educational outcomes. A majority of the studies of student performance consider variations in test scores as the measure of outcomes. The justification for this is that tests reveal skills that are valued in the labor market, as has been shown in a variety of studies. However, an important 1992 study by David Card and Alan B. Krueger offered the possibility that the output measure (test score) affected the conclusion about the inputs (education quality). Card and Krueger found that school resources have significant effects when performance is measured by subsequent labor market earnings rather than test scores. 10 After considerable analysis and debate, though, it appears that measurement issues are not the most significant cause of differences in results. 11 Direct analyses of resources and earnings do not confirm the differences; the variation appears to result from other analytical differences. 12 In fact, the findings for different measures of outcomes seem to be qualitatively similar. 13 Parallel analysis finds similar results when macroeconomic growth is the final output; that is, measured achievement is important, but input measures are not. 14
An entirely different approach to uncovering the impacts of varying school resources involves using random assignment experimental methodology for education studies. The benefits of this approach have been demonstrated in medical and agricultural studies. Random assignment to treatment and control groups minimizes model misspecification and bias in estimates of treatment effect. A random assignment experiment on class size reduction in Tennessee has been interpreted as providing a strong case for the approach. That experiment, Project STAR, found that kindergarten students in small classes (13-17 students) scored better than those in regular size classes (22-25 students). 15 The gains were relatively small and isolated in the first year that students were in smaller classes, though. Do these findings seem to contradict prior studies that show no systematic relationship between class size and performance? Krueger (1999) suggests that the results indeed may be consistent with prior econometric results because many of the earlier studies may not have been equipped to detect the small effects of class size differences that the STAR experiment did. Thus, in a policy sense, the results need not conflict because small gains from very expensive programs do not make such policies very attractive. Nonetheless, on the methodological side, random-assignment experiments clearly have tremendous advantages in assessing the effects of these kinds of policies.
Equity and the Financing of Education
One of the most significant policy issues of the past 30 years has been how states should fund local schools. While most states have used a compensatory aid formula to ameliorate some local governments' difficulty in raising taxes, these measures have only partially solved the problem. The issue became the subject of court action in the 1960s with the California case of Serrano v. Priest. Suits in many other states followed. As a result, some significant narrowing of spending variations occurred, because of both court rulings and independent legislative actions. 16Nonetheless, increased funding is not closely related to school quality -- as my research shows -- then changes in spending are not likely to move us toward more equitable provision of education. 17
Even though these court challenges have been going on for three decades, there has been surprisingly little direct analysis of their impacts. The one study of the effect of the original Serrano case on student achievement found no lessening of the variation in student outcomes after spending was equalized across districts. 18 A broader analysis of the distribution of earnings outcomes related to variations in district spending across the United States similarly finds no beneficial effect on the earnings distribution except perhaps for black females. 19
Interpretations and Puzzles
My research suggests that there is inefficiency in the provision of schooling; it does not indicate that schools do not matter. Nor does it indicate that money and resources never affect achievement. The accumulated research simply says that there is no clear, systematic relationship between resources and student outcomes.
Is this surprising? Some would argue that it is not plausible because parents decide on school spending and that fact alone should provide a discipline to schools. But at the same time, there are reasons why government provision of resources may be inefficient -- including lack of effective competition, bureaucratic decisionmaking, the costs of moving to a different school district, and the lack of good measures for assessing the "value-added" of schools. Clearly the political economy of educational decisionmaking needs further study. Any such research should also include better information about the character of household decisionmaking, both in choosing school districts and in supporting alternative policies.
The main conclusion of my research is that policy decisions should not focus on school resources, because the impact of resources on student achievement is unknown at this time. The solution is to establish teacher incentives -- rewards or consequences related to student outcomes -- and then to permit local schools to make appropriate choices. 20 Vouchers, merit pay, contracting out, and the like may be alternative ways to establish performance incentives.
Unfortunately, not much is known about alternative incentive schemes: how to structure them and what kinds of outcomes can be expected. Schools currently have few, if any, incentives for improving performance (as measured by student outcome). In addition, there is little empirical data on the effectiveness of incentive programs. Some is beginning to be available -- for example, from the Milwaukee voucher program -- but it applies only to very specific kinds of programs.
A final issue is the implication of these analyses for other kinds of studies. 21 Most studies involving human capital consider its effect on other aspects of behavior. But the inefficient production of human capital introduces natural measurement problems. Direct spending is no longer a good measure of quality because it has no perceivable bearing on performance. Further, families have considerable influence on student achievement, implying that school resources are only part of the equation. Both factors suggest that measuring student achievement only by resource investment could lead to distortion.
1 J. S. Coleman, et al., Equality of Educational Opportunity, Washington, DC: U.S. Government Printing Office, 1966.
2 E. A. Hanushek and J. F. Kain, "On the Value of 'Equality of Educational Oopportunity' as a Guide to Public Policy," in On Equality of Educational Opportunity, F. Mosteller and D. P. Moynihan, eds., New York: Random House, 1972.
3 E. A. Hanushek, "Throwing Money at Schools," Journal of Policy Analysis and Management, 1 (1) (Fall 1981), pp. 19-41; "The Economics of Schooling: Production and Efficiency in Public Schools," Journal of Economic Literature, 24 (3) (September 1986), pp. 1141-77; "The Impact of Differential Expenditures on School Performance," Educational Researcher, 18 (4) (May 1989), pp. 45-51; and "Assessing the Effects of School Resources on Student Performance: An Update," Educational Evaluation and Policy Analysis, 19 (2) (Summer 1997), pp. 141-64.
4 See, for example, E. A. Hanushek, "Teacher Characteristics and Gains in Student Achievement: Estimation Using Micro Data," American Economic Review, 60 (2) (May 1971), pp. 280-8; and "The Trade-off Between Child Quantity and Quality," Journal of Political Economy, 100 (1) (February 1992), pp. 84-117; R. J. Murnane, Impact of School Resources on the Learning of Inner City Children, Cambridge, MA: Ballinger, 1975; and D. Armor, et al., Analysis of the School Preferred Reading Program in Selected Los Angeles Minority Schools, Santa Monica, CA: Rand Corp., 1976.
5 S. G. Rivkin, E. A. Hanushek, and J. F. Kain, "Teachers, Schools, and Academic Achievement," NBER Working Paper No. 6691, August 1998.
6 The calculations of teacher quality are based on differences in achievement growth across classrooms. Having a teacher one standard deviation above the mean for four to five years running will overcome the average difference in performance between those on free or reduced lunch and those not.
7 In order to obtain unbiased estimates of the effects of school inputs, it must be the case either that variations in state policies do not matter or that there is no correlation between policies and school inputs. The latter condition, while more plausible at the individual student level, is very unlikely at the state aggregate level. See E. A Hanushek, S. G. Rivkin, and L. L. Taylor, "Aggregation and the Estimated Effects of School Resources," NBER Working Paper No. 5548, April 1996, and Review of Economics and Statistics, 78 (4) (November 1996), pp. 611-27.
8 For a discussion of empirical specifications, see E. A. Hanushek, "Conceptual and Empirical Issues in the Estimation of Educational Production Functions," Journal of Human Resources, 14 (3) (Summer 1979), pp. 351-88.
9 E. A. Hanushek, "Assessing the Effects of School Resources on Student Performance: An Update," Educational Evaluation and Policy Analysis, 19 (2) (Summer 1997), pp. 141-64; and "The Evidence on Class Size," in Earning and Learning: How Schools Matter, S. E. Mayer and P. E. Peterson, eds., Washington, DC: Brookings Institution, 1999.
10 D. Card and A. B. Krueger, "Does School Quality Matter? Returns to Education and the Characteristics of Public Schools in the United States," Journal of Political Economy,100 (1) (February 1992), pp. 1-40.
11 See Does Money Matter? The Effect of School Resources on Student Achievement and Adult Success, G. T. Burtless, ed., Washington, DC: Brookings Institution, 1996. An alternative way to reconcile Card's and Krueger's results is to note that resources might have a larger effect when the level of spending is less. Their schooling goes back to the 1930s. On the other hand, the evidence for developing countries does not appear much stronger; E. A. Hanushek, "Interpreting Recent Research on Schooling in Developing Countries," World Bank Research Observer, 10 (2) (August 1995), pp. 227-46.
12 J. Betts, "Is There a Link Between School Inputs and Earnings? Fresh Scrutiny of an Old Literature," in Does Money Matter? The Effect of School Resources on Student Achievement and Adult Success, G. T. Burtless, ed., Washington, DC: Brookings Institution, 1996; J. S. Heckman, A. Layne-Farrar, and P. E. Todd, "Does Measured School Quality Really Matter? An Examination of the Earnings-Quality Relationship," NBER Working Paper No. 5274, September 1995, and in Does Money Matter?; and E. A. Hanushek, S. G. Rivkin, and L. L. Taylor, "Aggregation and the Estimated Effects of School Resources," NBER Working Paper No. 5548, April 1996, and Review of Economics and Statistics, 78 (4) (November 1996), pp. 611-27.
13 E. A. Hanushek, "Assessing the Effects of School Resources on Student Performance: An Update," Educational Evaluation and Policy Analysis, 19 (2) (Summer 1997), pp. 141-64.
14 E. A. Hanushek and D. D. Kimko, "Schooling, Labor Force Quality, and the Growth of Nations," American Economic Review, 90 (5) (December 2000), pp. 1184-1208.
15 The results are described in E. Word et al., Student/Teacher Achievement Ratio (STAR), Tennessee's K-3 Class Size Study: Final Summary Report, 1985-1990, Nashville, TN: Tennessee State Department of Education, 1990, and introduced to the economics literature in A. B. Krueger, "Experimental Estimates of Education Production Functions," Quarterly Journal of Economics, 114 (2) (May 1999), pp. 497-532. The superior design of the random assignment experiment was nonetheless compromised by a series of implementation problems: see E. A. Hanushek, "Some Findings from an Independent Investigation of the Tennessee STAR Experiment and from Other Investigations of Class Size Effects," Educational Evaluation and Policy Analysis, 21 (2) (Summer 1999), pp. 143-63. See also E. A. Hanushek, "The Evidence on Class Size," in Earning and Learning: How Schools Matter, S. E. Mayer and P. E. Peterson, eds. Washington, DC: Brookings Institution, 1999. Unfortunately, there are no attempts to replicate the experiment.
16 S. E. Murray, W. N. Evans, and R. M. Schwab, "Education Finance Reform and the Distribution of Education Resources," American Economic Review, 88 (4) (September 1998), pp. 789-812.
17 E. A. Hanushek, "When School Finance 'Reform' May Not Be Good Policy," Harvard Journal on Legislation, 28 (2) (Summer 1991), pp. 423-56, has a discussion of various aspects of the common litigation.
18 T. A. Downes, "Evaluating the Impact of School Finance Reform on the Provision of Public Education: The California Case," SPAN STYLE="text-decoration: underline">National Tax Journal, 45 (4) (December 1992), pp. 405-19
19 E. A. Hanushek and J. A. Somers, "Schooling, Inequality, and the Impact of Government," NBER Working Paper No. 7450, December 1999, and in The Causes and Consequences of Increasing Inequality, F. R. Welch, ed., Chicago: University of Chicago Press, 2001.
20 E. A. Hanushek, et al., Making Schools Work: Improving Performance and Controlling Costs, Washington, DC: Brookings Institution, 1994.
21 E. A. Hanushek, "Measuring Investment in Education," Journal of Economic Perspectives, 10 (4) (Fall 1996), pp. 9-30.